How does/should static linking work on non-Apple platforms?

Using Swift
1

I was wondering about how static linking works for Swift today and how that compares to what might be useful and desired. There is a lot of text here to lay the context before getting to the actual question, so please pardon the long message.

My understanding is that there are two flags that are relevant to this today:

  1. -static-stdlib: statically link the standard library
  2. -static-executable: statically link the entire executable
An aside on `-static-library`

There is a related flag which is -static-library. However, as static libraries are not linked at build time, this merely changes the compilation to form a static library. This is important for Windows as this changes the compiler behaviour - public symbols will not be exported to ensure that the consumer of the library does not vend the ABI surface.

-static-executable forms a completely static executable: fully statically linked C, C++, and Swift runtimes and any dependencies. This is a completely static binary. This is reasonable if you do not need any type of dynamic linking at all.

The interesting case however is -static-stdlib. First of all, this is a misnomer AIUI, it does not control the static linking of the Swift standard library, but rather all Swift SDK libraries, i.e., it controls the static linking of the standard library, the runtime, and the core libraries (which now includes the non-corelibs Foundation). Granted that the runtime is always statically linked to the standard library, the effective point becomes: -static-stdlib also indirectly controls the static linking of Foundation and dispatch.

With that context, I finally arrive to my motivating question: does it make sense or is it ever useful to consider a statically linked standard library and a dynamically linked core library? That is, could it ever be beneficial to statically link Foundation and dynamically link swiftCore?

I can imagine cases where a host application is written in Swift but does not use Foundation and a plugin would like to use a small portion of Foundation. Similarly, I can see potential applications to the embedded environments. This obviously is unlikely to ever be useful for the Apple platforms given as these libraries are shipped with the OS, but for other platforms, there is the open question. It could also be that I am forgetting some reasoning for this to be prohibited (e.g. the inverted case of static standard library and dynamic core libraries would not be reasonable to support as one runs afoul of the one runtime per address space restriction).

CC: @kubamracek @al45tair

1 Like
2

I'm not sure that's worth supporting, honestly. Foundation and Dispatch are (at least today) built in lock-step with the language runtime (and indeed the compiler), and I think it really does make sense for us to have an all-static or all-dynamic policy on the core libraries for that reason.

3

Mixing static Foundation with dynamic swiftCore may cause runtime conflicts. For plugins, dynamic linking helps reduce binary size, while static linking suits embedded systems. Have you tested different linker flags?

4

This isn't about changing the build compiler but rather the linkage model. You would still have them built at the same point, it is about the linkage that the modules use. It may very well be that it is not useful. I do wonder if we should either put it on a roadmap or mark it explicitly as something that the PSG should mark as something that the project does not wish to explore (similar to how the LSG has a number of commonly rejected proposals).

I'm not sure I see how they would cause conflicts if there is a single copy in the address space or if they do not escape their types. Could you please help me understand how this may occur? Perhaps being more concrete about the type of conflicts you are thinking of could be useful.

Not sure what linker flags you have in mind. I imagine that most will be irrelevant here. Particularly for PE/COFF, /opt:ref and /opt:icf are enabled by default unlike ELF. DCE is unable to kick in due to forced retention of protocol conformances.

5

I assume you're thinking about the case where static Foundation was linked against a different version of libswiftCore than the one the main program ended up being linked to; that would potentially be a problem, if that was a thing we allowed to happen. I don't think that's what @compnerd is thinking about here though, and Swift on Windows isn't ABI stable so technically speaking that would be UB anyway.

6

Good point! Version mismatches could lead to conflicts, but since Swift on Windows isn’t ABI stable, it’s already in undefined behavior territory. Curious to hear @compnerd’s perspective on this!

7

As @al45tair said - that isn't the case that I was thinking about. This is a fully matching set of libraries. The question is more about a mixed mode linking: static Foundation, dynamic swiftCore. The more important one to consider is static swiftDispatch, dynamic libdispatch, dynamic BlocksRuntime.

The ABI of BlocksRuntime is not Windows friendly, and requires special changes to the compiler (and as of current upstream LLVM main, I've introduced a new flag for account for the vtable layout). However, this is a compile-time flag, and we need to know when building code whether the linkage for libclosure is static or dynamic. This is something that there isn't much precedent in Swift for: to date, we do not have a mechanism to indicate the indicate linkage for well-known libraries (we do explicitly pass macros through SPM for parts of the toolchain). However, the only actual control is either static swiftCore or static swiftCore, objc2, c++, c++abi, c.

Clearly, -static-executable doesn't work for a dynamic library (consider Linux - glibc must be dynamically linked for dl* functions to be usable). So, should the assumption be that -static-stdlib should only be static linking for swiftCore and dynamic linking for libdispatch, BlocksRuntime, Foundation? XCTest and Testing require dynamic linking today AFAIK. So it makes sense that the others would also be dynamically linked. If not, libdispatch and BlocksRuntime really do want to be dynamically linked I think - so that means that -static-stdlib means that you statically link swiftCore, Foundation, and swiftDispatch, but dynamically link libdispatch, BlocksRuntime, XCTest/Testing? That seems like it can be confusing for users.

1 Like
8

The idea of selectively statically linking the standard library while dynamically linking core libs like Foundation has potential benefits, like for embedded apps. But the "one runtime per address space" restriction is likely a technical limitation.

9

I like to think about Swift libraries as "just source code", including Foundation, including the standard library.

The idea that some libraries are "core" and that we can decide how we want users to use them is much more fuzzy and unclear when we take more low-levels/systems/embedded use cases into account. Flexibility matters to users, and matters more to power users. And especially around how linking is done maximum flexibility is important to certain group of users -- notably embedded developers, but this also to any lower-level library developers too. Of course with reasonable defaults so that most users never have to deal with the complexity.

1 Like