Static linking on Linux
I am happy to announce that with the release of Swift 5.3.1 statically linking the Swift stdlib components is now fully supported on Linux. This includes linking against Dispatch and the different Foundation components. Additionally building self-contained binaries is now possible by manually adding a few linker flags.
How to use it?
The Swift compiler supports two different static linking modes:
static-stdlib
The -static-stdlib
flag tells the Swift compiler to only statically link the components of the Swift stdlib, including Foundation and Dispatch. Any transitive dependencies will still be dynamically linked. This mode is very useful when deploying to a target that is known to have all Swift dependencies installed, but not Swift itself. It also allows running several applications that use different Swift versions on the same host without any special configuration.
static-executable
This mode tells the Swift compiler to build a self contained binary. It will statically link any dependency, including system dependencies. This mode is useful when deploying to a host that is not known to have any of Swift's dependencies installed, but produces larger binaries and makes deploying security patches harder, because the app has to be rebuilt against the patched libraries and redeployed.
Building fully self contained binaries with -static-executable
works out of the box for Dispatch and Foundation, because they don't have any additional dependencies, but requires the user to provide transitive dependencies for FoundationXML and FoundationNetworking. The depedencies can be provided to the linker through the -Xlinker
flag, e.g. swiftc -Xlinker -lxml2
.
What dependencies need to be passed can be determined by looking at the modulemap for the particular library. For FoundationXML that is located under /usr/lib/swift_static/CFXMLInterface/module.map
, for FoundationNetworking is is located under /usr/lib/swift_static/CFURLSessionInterface/module.map
.
The steps for FoundationXML and FoundationNetworking are the same, but let's look at FoundationXML as an example.
The content of the module map file looks as follows:
module CFXMLInterface [extern_c] [system] {
umbrella header "CFXMLInterface.h"
link "CFXMLInterface"
link "xml2"
}
The only direct dependency here is libxml2, but that itself has additional dependencies that need to be passed to the linker as well. The easiest way to determine those transitive dependencies is by asking pkg-config
:
pkg-config --libs --static libxml-2.0
This command returns all the flags necessary to statically link against libxml2. On Ubuntu 18.04 the dependencies are as follows:
-lxml2 -licui18n -licuuc -licudata -lz -llzma -lm
So a complete Swift compiler invocation to create a self contained executable that uses FoundationXML would look something like:
swiftc -static-executable -Xlinker -licui18n -Xlinker -licuuc -Xlinker -licudata -Xlinker -lz -Xlinker -llzma -o myApp main.swift
-lxml2
and -lm
can be omitted, as they are already included in the linker flags.
One might wonder why we don't include these depdencies in the module map as well. The short answer is that different systems may have different versions and the dependencies installed, which in turn may have different transitive depdencies. On CentOS8 for example the dependencies for libxml2 are:
-lxml2 -lz -llzma -lm
For projects utilizing Swift Package Manager, those flags should be added to the linkerSettings
of the target:
// swift-tools-version:5.3
import PackageDescription
let package = Package(
name: "MyApp",
dependencies: [],
targets: [
.target(
name: "MyApp",
dependencies: [],
linkerSettings: [
.linkedLibrary("icui18n"),
.linkedLibrary("icuuc"),
.linkedLibrary("icudata"),
.linkedLibrary("z"),
.linkedLibrary("lzma")
]
)
]
)
Additionally we need to pass the -static-executable
flag to the compiler, so the invocation looks as follows:
swift build -Xswiftc -static-executable
What was broken and how did we fix it?
Dispatch and Foundation are partially written in C and have dependencies on other C libraries like libxml2 and curl. When using dynamic linking, the CoreFoundation libraries (the part that's written in C) are included in the shared objects that also contain the Swift part, so there is only one library file per module. In the static linking case the C libraries are separate and have to be explicitly linked against. To interface with C libraries, Swift relies on Clang module maps, which among other things also contain the libraries to link against. Because there are no separate C libraries in the dynamic linking case, the module maps don't contain any linking instructions, so when statically linking against Foundation, the linker could not find the symbols that are part of the C library.
To fix this, we first needed to generate separate module maps for the dynamic and static linking cases. The relevant patches for this are:
- Add modulemaps that work for statically compiled Foundation [#2850]
- Add modulemap for static compilation [#544]
Doing this also uncovered that the compiler did not properly compute the resource path when statically linking and instead always used the shared resource folder. This was fixed in the following patch:
- Properly compute resource folder when linking statically [#33168]
I hope this helps shed some light on static linking in Swift 5.3.1.