Improving swift-lldb support for Path Remappings

Background:

Xcode assumes absolute paths for everything, which makes it particularly inflexible for distributed and portable builds.

Bazel makes use of relative paths from a single execution root, meaning it may embed relative paths (not absolute paths) into binaries. This is a problem with debugging with Xcode: Xcode runs lldb from the / directory instead of execution root, meaning these relative paths can not be resolved. Thus, we need workarounds to remap the paths.

Workarounds:

In lldb, there are two main ways to remap source paths for a binary:

  • target.source-map: Remap paths at the target level, we can set this globally for Xcode but it will apply to all projects
  • dSYM DBGSourcePathRemapping : Remap paths at the module (dSYM) level

Problem: Remapping paths without dSYMs as dSYMs are costly in terms of time

dSYM remappings work incredibly well - for this reason, Bazel's Xcode integration currently requires dSYMs for Swift. However, they come at a major cost for incremental builds - linking together all debug information is expensive. For this reason I've been looking into no longer requiring dSYMs for Swift, but now I'm running into a problem that target.source-map can't solve.

Underlying problem: swift-lldb will try to load a SwiftASTContext from a Module and then try to resolve Swift AST data from the DWARFDebugMap.

For Bazel, these paths are relative, so by default lldb will not be able to find them since Xcode runs lldb from /. However, target.source-map can't apply here as only a module is given to SwiftASTContext, not a target. And I'm trying to move away from requiring dSYMs, so that remapping option isn't here.

I'm not quite sure how to deal with this, any ideas? @Adrian_Prantl

Quick brainstorming:

  • Some equivalent target.source-map lldb setting for modules
  • Some equivalent to dSYM files that only contain remapping (is an empty dSYM file valid?)
1 Like

There are several way to attack this problem.

A quick and dirty hack would be to modify the linker invocation. The AST filenames in DWARFDebugMap are N_AST symbol table entries created by the linker for every -add-ast-path command line option. Since you control the build system, you should be able to edit the paths before sending them to the ld.

You could write a post-processing tool that edits the N_AST stabs entries.

We could add a symbols.obj-map setting that work analogously to target.source-map. However, this brings up an interesting question. Regular object files are found by LLDB very similarly by following N_OSO stabs entries. Why can LLDB find the .o files in your use-case, but not the AST files? (You can dump the symtab by running dsymutil -s on the binary.) I'd like to understand that before deciding on a solution first.

Post-processing seems a bit overkill since as you said we can modify the invocation as we control the build system. We could make the invocation use absolute paths, in which case lldb should be able to find the binary (assuming it was linked locally).

For some reason the N_OSO stabs refer to files by absolute paths even though AFAIK we only use relative paths when referring to .a and .o files for the linker, maybe those are explicitly made absolute before embedding, unlike the N_AST entries?

Are the .o files passed as absolute paths to the linker? If yes, then that's where they come from. If no, I'd suspect the linker makes them absolute.

I'm currently (independently of this) working on eliminating the -add-ast-path option from the linker and instead embed the path directly into DWARF, in which case the Swift compiler would control the path and the build system wouldn't need to know about it.

Nope, confirmed that we pass them as relative paths, the linker must be making them absolute (which makes sense as otherwise dsymutil/lldb would not be able to load them unless it knew the build root).

Anyways, I'll try making the add_ast_path absolute and report back.

In my experience I've seen absolute paths for Bazel-produced debug symbols, i.e. /var/tmp/_jdovey_xxxxxxxx/<uuid>/execroot/__main__/…. I've been using -fdebug-prefix-map (and -debug-prefix-map in swiftc v5) when invoked from Xcode to change these to match Xcode's expectations.

Regarding the linker making absolute paths in N_OSO stabs, I'd look in

https://opensource.apple.com/source/ld64/ld64-409.12/src/ld/OutputFile.cpp.auto.html

search for assureFullPath.

Not 100% sure that's it, but certainly a smoking gun :gun:

1 Like

Interesting, it appeared to work locally with swift-lldb built on the Swift 5.1 branch, but when I try to test with Xcode 10.2 I get an error with duplicate definitions (was this recently fixed between Swift 5.0 and 5.1?):

EDIT: This seems like an unrelated issue, this workaround appears to be working.

<module-includes>:3:9: note: in file included from <module-includes>:3:
#import "../../../../../../../../../<redacted 5 dirs>/FooInternal.h"
        ^

/private/var/tmp/_bazel_davg/c4ea4f0ebd1679d2e8a68dee5920b740/execroot/workspace/bazel-out/ios-x86_64-min10.0-applebin_ios-ios_x86_64-dbg/genfiles/<redacted 5 dirs>/Foo.modulemaps/../../../../../../../../../<redacted 5 dirs>/FooInternal.h:1:9: note: in file included from /private/var/tmp/_bazel_davg/c4ea4f0ebd1679d2e8a68dee5920b740/execroot/workspace/bazel-out/ios-x86_64-min10.0-applebin_ios-ios_x86_64-dbg/genfiles/<redacted 5 dirs>/Foo.modulemaps/../../../../../../../../../<redacted 5 dirs>/FooInternal.h:1:
#import "<redacted 5 dirs>/Foo.h"
        ^

error: /private/var/tmp/_bazel_davg/c4ea4f0ebd1679d2e8a68dee5920b740/execroot/workspace/<redacted 5 dirs>/Foo.h:140:1: error: duplicate interface definition for class 'FooClass'
@interface FooClass : NSObject
^

/private/var/tmp/_bazel_davg/c4ea4f0ebd1679d2e8a68dee5920b740/execroot/workspace/bazel-out/ios-x86_64-min10.0-applebin_ios-ios_x86_64-dbg/genfiles/<redacted 5 dirs>/Foo.modulemaps/../../../../../../../../../<redacted 5 dirs>/Foo.h:140:12: note: previous definition is here
@interface FooClass : NSObject

Update here: there is an ld flag to make the OSO entries relative (see milen.me — Apple’s Linker & Deterministic Builds), it may be worth revisiting whether we can make the remapping work for the Swift module as well so we don't have to make the path absolute.