Background: a TBD file is a text-based stub, that lists all the symbols of a dynamic library, that can be linked against in leui of the full dylib. This allows more parallelisation in the build system, allowing linking of libraries to proceed in parallel, even if they have dependencies between them.
Currently swiftc supports generating TBD files with
-emit-tbd, but this only works in whole-module optimisation mode: in an incremental build, a particular frontend invocation only sees the symbols in a single file, and thus cannot emit the complete TBD file. There's two possibilities for addressing this:
- emit the TBD file only when the full AST module can be seen, meaning either during a WMO build, or during the merge-modules step of an incremental one
- emit the TBD files per-file, and invoke an external tool (tapi) to merge them into the final output, in parallel with merge-modules and/or linking.
The first is slightly less incremental, since the merging of TBD files and merging of modules cannot happen in parallel. However, emitting the TBD file is not an expensive step, even compared to the limited
-merge-modules step, which is already a serialization point in an incremental build of a library (and, note that even merging them with the external tool as above is a serialization point too). For instance:
- generating the TBD file (which contains ~12000 symbols) for the whole stdlib, in WMO mode, takes 35ms. The rest of the build takes 14s (WMO -Onone) or 60s (WMO -O).
- generating the TBD file as part of
Basicmodule (which seems to be the largest in swiftpm) takes 3ms, while the whole
-merge-modulesinvocation takes 210ms.
(These are with a release no-asserts swiftc, without LTO.)
The second is similar to how each frontend invocation emits separate object files and swiftmodules, and then the driver invokes
swift -frontend -merge-modules after doing the normal build steps. However, unlike
swiftc, tapi is not in the path on Apple systems by default, and, AFAIK, doesn't exist on any non-Apple systems (although I don't know if any linker supports TBD files on non-Apple systems just yet). Going with this approach makes it more difficult to emit TBD files "by hand", such as in a non-Xcode build system, or when debugging a problem: the build system and the developer (respectively) has to reconstruct the appropriate path to tapi. Additionally, there's not a fundamental performance gain for incremental builds: it is still a serialisation point, and
-merge-modules still exists and needs to run.
As such, I'm inclined towards avoiding the complexity of an external tool and going with approach 1. However, I'd like to just double check that I'm not missing something: are there downsides to the first approach other than adding slightly more work to a serial step? Are there upsides to the second approach beyond performance?
(Apple people: this is a public post, so if there's some internal reason please email me instead.)