In Swift 5.3.1 we introduced static linking on Linux. With this feature, one can pass the --static-swift-stdlib flag to SwiftPM's swift build command (or long form -Xswiftc -static-stdlib) in order to create a statically linked program, with some caveats pointed out in the original post and which are less important to this pitch.
On Linux, this is often the preferred way to build, since the statically linked program is easier to deploy to the target server or otherwise share. Members of the Swift on Server community brought up the idea to change SwiftPM use this mode by default when building executables on Linux, with an opt-out option when it is not needed.
This post is to collect feedback on this idea, and we would love to hear your opinion!
I can state my reason for supporting this change: I work on this stuff daily and I still forget itâs even an option when I run into missing dependencies on a node I âjust want to run that sample app real quickâ, and end up wiggling around with docker ending up with other issues (docker image not having matching kernel, so ending up with issues installing perf in there etc).
I can imagine for people only doing this even more rarely or for the first time itâs even more confusing and annoying, which makes it harder to attract new users on the server.
People who care about reducing the binary size can always opt out; and by that time they know what theyâre looking to solve
Makes sense to me. We run alot of stuff on AWS Ubuntu Linux, spinning up GRPC nodes that talk to each other to do random agent-y things, and we definitely would rather it be easiest- think of it this way- the other thing that happens is you end up building locally on your running node and exposing more of your IP- when you can just pump a binary to the right place.
Pragmatic to change the default to lower initial friction and for casual usage. Those who build more complex systems and that wants to share can then opt out - as long as itâs documented how to do that, I canât see the problem.
Before that can happen, we'd need to ditch glibc (and everything that depends on glibc) and switch to eg musl. Glibc doesn't support fully static binaries (even if you build a "static" binary, it'll dynamically load stuff for anything that touches NSS like DNS, user/group info, ...). So you'll still need the exact same version of the glibc so files on your target system.
Having the option of switching to musl and building fully static binaries would be super cool (you'd give up full compatibility with the system's DNS settings etc but on a server that's acceptable I think) but that's not a small feat. We'd need a different C++ standard library as well because libstdc++ depends on glibc .
I'm +1 on this direction. It would be great if we can do so, but Glibc doesnât recommend being statically linked. I would suggest simplifying the arguments first.
Also, we may need a guide in this field. I tried to statically link SwiftTencentSCFRuntime examples on CentOS 7, but ended up failing with CurrencyExchange. Collecting and specifying the linked libraries is a terrible experience, and it only works for exactly the current distro. I didnât find a better way to make an executable statically linked easily for all platforms.
I presume there would be a way to flip it back to dynamic based linking? Using swift for non-server based linux applications might be some times better served with dynamic libraries. As long as there is an option to disable that then it sounds reasonable to favor that usage pattern.
I'm strongly against this. The only reasonable way to link to glibc is dynamically - as has been stated upthread. This requires splitting out the Glibc module and a fair amount of ancillary support to actually enable. I think that if we do the necessary pre-requisite work, then this potentially becomes more interesting.
Speaking of static linking, I just noticed if I made my Package type:.static that my static let initializers in my classes get called more than once. Seems weird... (this is on 5.5 release)
What happens if you have a shared library and an executable that depends on that? If you statically link the executable with the stdlib and it dlopens the library, you just have a ODR violation. Mixing static and shared linkage is perilous. Would we also diagnose these types of problems? What happens if you have an executable that dynamically links against a library that is statically linked against the standard library and the executable also statically linked the standard library? This is also a problematic situation and amongst those that need to be diagnosed.
It would be good for us to make sure the compiler builds these statically-linked variations of the Swift runtime and packages with everything having hidden visibility, so that Swift symbols don't leak out of dynamic libraries that might use Swift internally. That should address these concerns. There is no stable ABI on Linux, so people are hopefully not distributing dynamic libraries that export Swift API, though I agree that having a shared Swift runtime library is the only way to support those environments. But for static linking is much better for distribution and performance for everybody else, so it seems like the right default.
This makes sense to me and if we do this, I think we should generalize it to defaulting to --static-swift-stdlib for any platform where the option is fully supported and the Swift libs aren't part of the OS (which I believe is any non-Darwin platform at this time).
Any chance we can get better dead code stripping out of this endeavor? Thats one of the advantages of pressing everything together like this. I would add to the warning about how we must hide symbols then- ive seen many cases of dylibs suddenly shoved together doing very strange things when forced to be static and having their symbols blended.
.