Swift behavior of -gmodules and dSYMs

Keith · April 16, 2019, 9:11pm

We build our Swift 5 app with bazel which allows us to utilize a remote cache for downloading artifacts that have previously been built. When testing this behavior and generating a dSYM, we got this warning:

warning: /private/var/tmp/_bazel_iosci/f749baf807251022363d6bcea6ad530d/execroot/lyftios/bazel-out/ios-armv7-min10.0-applebin_ios-ios_armv7-dbg/bin/_objc_module_cache/2AFT5C5VLSSQV/UIKit-3IG826SVDQUFO.pcm: No such file or directory
note: while processing /private/var/tmp/_bazel_iosci/f749baf807251022363d6bcea6ad530d/execroot/lyftios/bazel-out/ios-armv7-min10.0-applebin_ios-ios_armv7-dbg/bin/_objc_module_cache/2AFT5C5VLSSQV/UIKit-3IG826SVDQUFO.pcm
note: Linking a static library that was built with -gmodules, but the module cache was not found.  Redistributable static libraries should never be built with module debugging enabled.  The debug experience will be degraded due to incomplete debug information.
warning: /private/var/tmp/_bazel_iosci/f749baf807251022363d6bcea6ad530d/execroot/lyftios/bazel-out/ios-armv7-min10.0-applebin_ios-ios_armv7-dbg/bin/_objc_module_cache/2AFT5C5VLSSQV/ObjectiveC-TF7LBA6W6M7C.pcm: No such file or directory
note: while processing /private/var/tmp/_bazel_iosci/f749baf807251022363d6bcea6ad530d/execroot/lyftios/bazel-out/ios-armv7-min10.0-applebin_ios-ios_armv7-dbg/bin/_objc_module_cache/2AFT5C5VLSSQV/ObjectiveC-TF7LBA6W6M7C.pcm
warning: /private/var/tmp/_bazel_iosci/f749baf807251022363d6bcea6ad530d/execroot/lyftios/bazel-out/ios-armv7-min10.0-applebin_ios-ios_armv7-dbg/bin/_objc_module_cache/2AFT5C5VLSSQV/Foundation-M22A4D3IGJY9.pcm: No such file or directory
note: while processing /private/var/tmp/_bazel_iosci/f749baf807251022363d6bcea6ad530d/execroot/lyftios/bazel-out/ios-armv7-min10.0-applebin_ios-ios_armv7-dbg/bin/_objc_module_cache/2AFT5C5VLSSQV/Foundation-M22A4D3IGJY9.pcm
warning: /private/var/tmp/_bazel_iosci/f749baf807251022363d6bcea6ad530d/execroot/lyftios/bazel-out/ios-armv7-min10.0-applebin_ios-ios_armv7-dbg/bin/_objc_module_cache/2AFT5C5VLSSQV/CoreGraphics-HOC1SLJ4ZMTL.pcm: No such file or directory
note: while processing /private/var/tmp/_bazel_iosci/f749baf807251022363d6bcea6ad530d/execroot/lyftios/bazel-out/ios-armv7-min10.0-applebin_ios-ios_armv7-dbg/bin/_objc_module_cache/2AFT5C5VLSSQV/CoreGraphics-HOC1SLJ4ZMTL.pcm
warning: /private/var/tmp/_bazel_iosci/f749baf807251022363d6bcea6ad530d/execroot/lyftios/bazel-out/ios-armv7-min10.0-applebin_ios-ios_armv7-dbg/bin/_objc_module_cache/2AFT5C5VLSSQV/SwiftShims-3HHJ0DU6KU482.pcm: No such file or directory
note: while processing /private/var/tmp/_bazel_iosci/f749baf807251022363d6bcea6ad530d/execroot/lyftios/bazel-out/ios-armv7-min10.0-applebin_ios-ios_armv7-dbg/bin/_objc_module_cache/2AFT5C5VLSSQV/SwiftShims-3HHJ0DU6KU482.pcm

This warnings comes from here in dsymutil seemingly when creating a dSYM from a binary that linked a static library that was built on another machine.

The warning indicates that we are passing -gmodules, causing the issue. -gmodules seems to only be a flag to clang, which we are never passing during our build (although we are passing -g in this case). When you do pass this flag, the handling logic adds 2 other flags to the invocation: -fmodule-format=obj, which is also added by Swift here and -dwarf-ext-refs which, based on this comment, seems to be the one causing this.

I'm curious if the behavior of clang's -gmodules flag is the default in Swift, through this flag or others, and if so how we could avoid this so debugging isn't degraded on developer machines when some of the libraries are pulled from a shared cache.

We've also had a difficult time reproducing this without building our entire app. What are the specific factors that lead to the absolute path of a pcm to end up in debug info?

jrose · April 16, 2019, 9:14pm

-gmodules is indeed the default in Swift. cc @Adrian_Prantl to talk more about it.

Adrian_Prantl · April 16, 2019, 11:00pm

We use -gmodules to be able to reconstruct Clang types from DWARF, when LLDB wasn't able to import a dependent Clang module from source.

Note that this warning is safe to ignore when you can guarantee that all your Clang modules can be imported from source code at debug time.

If you have to build a static archive and run dsymutil on a different machine that doesn't have access to the Clang module cache you can pass -Xfrontend -no-clang-module-breadcrumbs (https://github.com/apple/swift/pull/23796/) to Swift to disable -gmodule debug info. It would be better, however, to ensure that the module cache is still accessible when dsymutil is run.

Keith · April 16, 2019, 11:37pm

It would be better, however, to ensure that the module cache is still accessible when dsymutil is run.

Since it's actually the static archive that was built with the missing module cache and then downloaded to a separate machine to run dsymutil, I don't think that will work for the remote build use case.

when you can guarantee that all your Clang modules can be imported from source code at debug time

How does this importing happen? Does this mean that assuming there's a new module cache, or just that they exist locally within Xcode, or something else?

you can pass -Xfrontend -no-clang-module-breadcrumbs

Nice timing! Is passing a flag like this the expected path forward for redistributed builds in the future? If so is there a path to making this flag not a frontend flag?

Adrian_Prantl · April 17, 2019, 11:55pm

How does this importing happen? Does this mean that assuming there's a new module cache, or just that they exist locally within Xcode, or something else?

You'll need to have the header (.h) files and the module.modulemap in a location where it can be found. You should be able to tweak -fdebug-prefix-map to ensure this assuming that you have a fully buildable (but not necessarily built) copy of the sources on the machine where you are running LLDB.

Is passing a flag like this the expected path forward for redistributed builds in the future?

No, that's more a stopgap, since it degrades the debug info self-containedness and robustness. The cleanest solution would be to transition explicit modules that are tracked by your build system, but there is zero support in the Swift compiler to make this happen and adding support would be highly non-trivial. A more realistic approach might be to write a "dsymtutil-static" tool that collects the DWARF from the module cache before building the static archive. Or, we could teach dsymutil to rebuild the module cache from the breadcrumb information in the .o files' DWARF. This requires the header be present on the machine that runs dsymutil, but that seems like a reasonable requirement.

brentley · March 2, 2020, 11:46pm

We’ve run into this as well. With it being nearly a year since this was last discussed, what is the current recommended way to have portable .o files and static archives?

Edit: While -no-clang-module-breadcrumbs removes some references to pcm files, we've found that DW_AT_gnu_dwo_name is still sometimes set, resulting in non-portable files.

Keith · June 26, 2020, 11:49pm

It sounds like at this point we're probably just going to wait for explicit module builds to solve this warning I guess? In the meantime some projects are just going to hide it: https://github.com/bazelbuild/tulsi/pull/136