Improving Path Remappings for ExtraClangArgs/SwiftASTContext

Background:

SwiftASTContext when parsing swiftmodules currently applies the working-directory flag if found in the ExtraClangOptions (see here).

This works well in most use cases, but in the case of distributed builds - for distributed builds (on remote machines), the paths may be invalid (as they're a path on the remote machine, not the local one).

Problem:

For Bazel, which supports distributed builds, we pass relative paths for all compilation inputs as well as a -fdebug-prefix-map=<execution root>=. to make all embedded paths relative. This works well, except for the folowing:

  • SwiftASTContext doesn't respect any previous -fdebug-prefix-map= in the extra args, only remappings specified from the dSYMs/target source map.
  • The SearchPaths embedded inside the swift modules are also absolute paths (therefore also remote paths); I'm not sure if this is even consumed (or remapped) by SwiftASTContext or at all.

I'm not sure about the second point, but due to the first point, distributed Swift builds are failing to load modules since the paths are absolute paths for the remote machine instead of the local one.

I think the best way to fix this would be to add -fdebug-prefix-map= support to SwiftASTContext's ClangArgs handling, but I'm not sure. If the second point is an issue as well, we'll also need to fix up those.

Any ideas @Adrian_Prantl?

2 Likes

This was an issue I called out in the description of my original PR to implement -debug-prefix-map. We (Bazel) currently pass -Xfrontend -serialize-debugging-options to swiftc, which is what causes the search paths and ClangImporter flags to be written into the .swiftmodule file. (Oddly, these "debug" options also affect search path behavior during compilation: SR-7845.)

Unfortunately, the discussion at the time led us to defer remapping those flags because we would have had to just run through the raw command line text and make sure to handle a number of combinations like -I foo vs. -Ifoo, -fmodule-map-file foo vs -fmodule-map-file=foo, and so on.

Later on, however, essentially the same command line manipulating logic had to be added to lldb anyway: https://github.com/apple/swift-lldb/blob/stable/source/Symbol/SwiftASTContext.cpp#L1470-L1493

As I see it, there are a couple options that get us further along the right path:

  1. Figure out how to get debugging working without passing -Xfrontend -serialize-debugging-options during compilation.
  2. Add the same flag remapping logic from lldb to the compiler itself so that it applies to .swiftmodule files, not just DI.

But I'm not sure what unintended consequences might come out of doing one or both of those.