[Pre-Pitch] Statically linking the Swift runtime libraries by default on Linux

Some "better" DCE is theoretically possible in the sense that you might be able to shave off some of the standard library functions that are freestanding. However, the rest are going to stick around no matter what due to strongly connected clusters due to VWTs. If you want better DCE, I recommend that you build with -function-sections. That should give you the benefits of the DCE for any non-public symbol even in shared library builds.

As to the problem with internalizing symbols - that is one of the things that is currently problematic for supporting static libraries on Windows with Swift code.

2 Likes

I do agree that it is better for distribution, but only when building outside of package managers, which is not the norm on Linux distributions. For most distributions, the default linkage type is shared - and that is often codified into policies for distribution of software. I think that we should default to the safer assumption for the platform - that shared linkage is in effect.

The "strongly connected cluster" problem is something we're working on:

https://reviews.llvm.org/D104496

This should allow for trees of metadata to be stripped when the parts that are used from generated code, such as type descriptors, are not actually referenced by the binary. Then we'll be able to strip much more of the standard library and other packages, and statically-linked binaries will be much closer to pay-for-what-you-use.

A distribution distributing Swift packages as shared library binaries is still a hypothetical at this point, and it isn't obvious to me that the way C libraries happen to be traditionally distributed can be ported to other languages without modifications. If a distribution wants to ship Swift shared library bindings, then they're going to need to establish an ABI for those shared libraries, otherwise the entire Swift world they distribute will need to be rebuilt in lockstep with every new compiler release, which is not much better than static linking. If a distribution wants to do that work, then they should also be able to configure the compiler toolchain they distribute to default to building dynamically-linked binaries using their ABI as well. In that situation, it would still be very unlikely that a Swift toolchain built from source would be compatible with that distribution, so it would still make sense to default to static linking for such a toolchain.

How do distributions handle distributing Rust packages (if they do at all)? Dynamic linking is fundamentally incompatible with Rust's compilation model. It seems like another approach for distributions with newer languages like this would be to treat them like scripting languages, distributing packages as source code and rebuilding on-the-fly.

5 Likes

Circling back to some of the feedback, perhaps a better way to characterize the proposal is for SwiftPM to statically link the Swift runtime library when building executables on/for platforms where such libraries aren't part of the system already (e.g. Darwin) and that support static linking (e.g. Linux)

Following this line of thought perhaps we should deprecate SwiftPM --static-swift-stdlib flag and replace it with --{enable|disable}-static-swift-runtime, having it enabled by default and using the disable form for opt out.

1 Like

If we can enable static linking by a single flag, it’s okay to not make it the default🤔 The biggest problem today is we need to figure out which packages to install and which libraries to link against. The behavior differs largely across Linux distributions and also results in a large numbers of additional linker flags.

Maybe it's just me, but I would think of "Swift runtime" to actually suggest less components to be linked (not even the standard library, but really just the runtime), rather than more. I don't have a great suggestion that actually conveys that Foundation, Dispatch etc would also be statically linked if they're being used.

1 Like

--static-toolchain-libraries?

Would --static-swift-dependencies make more sense here?

As far as I understand, this flag would forbid dynamically linking such executable with any Swift library, not just toolchain libraries? That is, products of dependencies specified as .library(type: .dynamic) in their respective Package.swift manifests would cause build errors?

Otherwise, how would the issue with duplication of symbols of toolchain libraries be avoided?

If my assumptions above are correct, the only dynamically linked library for such executable would be only libc, right?

I don't think that is necessarily the case. Consider a system library that is included; the linker is free to dynamically link against a dependency, and can be forced to dynamically link just as well via pkg-config.

I'm +1 on this. Ignoring the semantics of whatever the flag is or future plans to try and provide a fully statically linked binary, from a server app deployment point of view this makes sense.

Statically linking the (Swift) standard libraries makes deployment easier as you don't need to dig around and ensure that you copy over the different libraries. This not only simplifies deployment/build scripts for things like Lambda and Docker, it should also speed up cold start times on Lambda etc.

For 99% of cases where most people are deploying a single binary onto a system this makes sense. For those that need dynamic linking (e.g. those with multiple binaries in a system) as long as they can turn it off it should be fine. Making the default better for the majority where those in the minority have the option to opt out makes sense :+1:

7 Likes

+1 on this. Also note that I plan to revive some works in Support fully-static linking · Issue #8 · bazelbuild/rules_swift · GitHub Thus, would appreciate any changes in the flag name happens on the Swift driver / SwiftPM level, not swiftc level.

1 Like

With SE-0332 coming into the picture, one alternative design direction could be to create a Linux oriented "packaging" plugin which would (among other things) build applications for Linux with the runtime static linking flags (-Xswiftc -static-stdlib) saving the need to change SwiftPM's default behavior. What do folks think about such alternative?

1 Like

I don't think that helps with the goal of changing the default so that new users get the right behavior. I can also make a shell script wrapper or Makefile today which is conceptually and UX-wise pretty similar to what a plugin would give us.

3 Likes

After we dropped the ICU dependency, statically linking stdlib should make more sense than before. I’m on making this as the new default, a plugin doesn’t help that much.

1 Like

While I deeply love command plugins, I think changing the default to static linking is still the right choice here. It truly is "the default" and right way to package for Linux distributions IMHO.

7 Likes

Over one year later, do we now have static linking as default on Linux? If yes, what about FoundationNetworking? FoundationNetworking always made problems for me when statically linked.

2 Likes

The proposal is accepted. But the implementation is not yet merged.