I'm trying to understand a number of linker warnings that I'm getting on Windows, but not on macOS, when I build a command line utility using the Swift Package Manager.
The utility depends on the Swift Argument Parser, and when I build on Windows I get about 80 linker warnings of the same kind – locally defined symbol imported (LNK4217) – for various Argument Parser related symbols.
Here is an example (paths redacted by me): lld-link: warning: C:\[...]\.build\x86_64-unknown-windows-msvc\debug\ArgumentParser.build\DumpHelpGenerator.swift.o: locally defined symbol imported: $s22ArgumentParserToolInfo0cD2V0V7commandAcA07CommanddE0V_tcfC (defined in C:\[...]\.build\x86_64-unknown-windows-msvc\debug\ArgumentParserToolInfo.build\ToolInfo.swift.o) [LNK4217]
Of course being just warnings, the utility not only builds successfully but also runs successfully and produces the desired file transformations.
Looking up the warning code, it appears to be related to the lack of static linking in Swift on Windows. But, surely this is just about linking Swift libraries to host C code, right?
Do package dependencies not compile statically on Windows?
Is there something I could do to avoid these warnings?
I will answer some of the questions out of order as they are all tied together in an odd way.
The only current solution to avoid the warnings is to not use SPM. This is the crux of the issue. You can also experiment with antimony if you like, though that is still very much a work in progress.
No, the warning has nothing to do with the source language, it is about the linkage model that was used to build the code and link the code. This primarily impacts Swift code being linked against Swift code rather than C code being linked to Swift code as most portable C/C++ codebases correctly annotate the public interfaces with the appropriate DLL storage (i.e. __declspec(dllimport) or __declspec(dllexport)).
Compilation is a complex topic. The problem is SPM, not static linking support for package dependencies. The state of Windows currently is that only the Swift standard library cannot be statically linked, but anything else is purely a developer decision. However, unlike macOS and Linux, you need to be explicit about the linkage. This means that if you are statically linking, you need to build the module with -static and ensure that static linking is used. If you are dynamically linking, you need to build the module without-static and ensure that dynamic linking is used (i.e. you use the import library).
SPM on the other hand will always build everything for dynamic linking and then use object libraries to link all the targets statically. This is the cause of the warning spew - SPM built the module one way and then used it a different way.
Windows does prefer dynamic linking, and that is the better default IMO. However, SPM's build model does not allow us the proper control over the linkage model.
Maybe this is a question for SPM folks rather than for you, but since SPM build products should (approximately) never go into a dynamic library, should it just build everything for static linking?
Sure, we can do that as long as we simultaneously drop support for products specified as .library(name:type:targets:) where type is .dynamic. Specifically, we would need to drop the dynamic case from Product.Library.LibraryType | Apple Developer Documentation.
Building something for static linking will immediately make it ineligible for dynamic linking as the symbols are internalised. The DLL storage is absolutely mandatory to get correct as that is what forms the moral equivalent to the GOT (the IAT and EAT). Windows heavily skews towards dynamic linking and I think that is the model we should be supporting (at least on Windows).
At the same time, this would mostly make the builds very much uninteresting. As a concrete example, the difference between static and dynamic linking for swift-format was ~70MiB. That is for a single module product. It also means that you can not support any type of plugin architecture with SPM.
IMO, such a restricted build tool is wholly uninteresting and I might even go so far as to say unusable.
I would propose the counter-question to the SPM folks: perhaps we can drop all static linking of targets and only support dynamic linking of targets?
To be honest, I'm kind of surprised SPM offers that as an option to begin with, since linking a tree of packages into one dynamic library seems like a great way to invite runtime module collisions if the same packages get built into multiple dynamic libraries that end up in the same process. If it does offer dynamic linking, then
seems like the right thing to do. Dynamic libraries should only transitively depend on other dynamic libraries and not link in static libraries.
I definitely agree with your point that it is a risky proposition. I do not have the context of why this feature was supported.
Yes, this is correct. However, it is complicated by the fact that a module built for dynamic linking will export its public interface. There is a 64K limit on the exports in a single module. Therefore, if you grow your binary (I'm looking at you SourceKit-LSP!), you will no longer be able to build with SPM.
Honestly, with different requirements from different user groups, I fear that the only viable solution here is to fix the build model. This is easier said than done and why I originally embarked on Antimony. It fundamentally changes the build model and evolving that from the current state of SPM was more challenging to do a quick prototype.
This comes from the fact that the ordinal field in the IAT/EAT is a uint16_t, limiting the ordinal to 64K. This is part of the PE/COFF design from the original OMF days and does not have an extension defined in the NT kernel.
Is that a hard limit, though? I thought the ordinal was only used as a hint and the loader would still do a binary search by symbol name if the export ordinal doesn't match. (Not to say that we couldn't also be more parsimonious with the symbols we export too.)
Yes, sadly it is a hard limit. Dynamic linking by ordinal is only performed if the IAT explicitly does not mention the name. The link by name is preferred. However, each entry is implicitly assigned an ordinal. So while the ordinal is taken as a hint, it is a requirement .
The ordinal linkage is even more interesting as it requires you to explicitly specify a def file to ensure that the ordinal assignment is consistent over time.
Being more frugal with exports would certainly be very helpful - on all platforms really - but might make it possible to support larger binaries like SourceKit-LSP which has long blown past this limit and cannot be built with SPM on Windows.
Would you have any particular symbols that you can think of that we are currently making public (from an ABI perspective) which do not need to be? I think that if there are easy wins here, this would be a valuable immediate change to pursue.
The big difference with -static is that we no longer expose the public accessible symbols and instead treat it as internal from an ABI perspective. It reduces the exported surface and is why it immediately disqualifies the module from being used for building a dynamic library or an rdynamic executable (does SPM have a model for that?).
Not off the top of my head without breaking Darwin's ABI, but I could imagine places where without an ABI stability constraint, we could probably stand to export a single symbol referencing a struct made up of related entities rather than a bunch of related symbols, such as with the many related metadata symbols that get exported for types and protocol conformances.
I guess for Swift we could resolve symbols ourselves on Windows (i.e. the runtime could grow a mechanism for symbol resolution, and we could put appropriate data in a custom section), and only export things that need to be accessible to/from C/C++.
(What I'm thinking is that Swift is the only language currently that will directly call Swift symbols, hence they don't have to be exported using the EAT or imported using the IAT — we could have our own mechanism and it wouldn't break anything.)
Yes, it has. There is a GSoC proposal to help address the required DLL storage annotations to enable building plugins for clang. That should also help with the DLL builds of LLVM. This is part of what makes the Windows toolchain so large storage-wise - we need to statically link LLVM. This also then prevents some other things LTCG as the costs become exorbitant.
This feels like we would be growing a significant amount of complexity. We would be building an in-process dynamic linker with everything that entails and then incurring costs for the necessary data processing and page dirtying. Instinctively, this feels like a poor trade-off.
On the topic of dynamic vs static linking in Swift PM. Most libraries do not specify their linkage mode. This is currently only used as an escape hatch for libraries that for some reason must be dynamically linked.
The default is nil which means the build system can choose which way to link the module. Most of the time this is fully opaque to the user and allows build systems to avoid duplicate symbols by deciding to build a dynamic library instead of linking a static library into multiple dynamic targets (libraries or executables).
On the top of my head I don't know a single package in the server ecosystem that explicitly specifies the linkage type. So I don't see how this is currently the limiting factor for Swift PM on Windows. It seems to me that the Swift PM on windows should default to static linking everything into the end executable unless some module defines dynamic as the linkage type.
Right, this is a possibility, trading off pre-computation for runtime computation. The indexed load is likely not too much more expensive with modern CPUs, but certainly something to consider, and may have some impact on cache coherency as well.
When the developer has control over the modules and their linkage, I believe that it is still possible to fit within the confines of the 64K exports. It is still a good idea to get a rough estimate of the number of export symbols per exported class/struct/generic to be able to share what the cost is for a public interface that is exported. This should also help developers make appropriate choices for their design.
There are places within the current ABI where some symbols are optional and only exported to allow for optimizations in clients; for instance, when a struct is @frozen or library evolution is disabled and the struct has a fully static, non-generic-dependent layout, then we export its type metadata symbol directly for clients to reference, when in the more general case you would need to go through the metadata accessor referenced through the nominal type descriptor. Building for library evolution might end up reducing the overall number of symbols exported since it would inhibit many of these optimizations from happening passively (though AIUI there are other complications with enabling library evolution within the SwiftPM environment, for whatever reason).
.