Static linking on Linux in Swift 5.3.1

Static linking on Linux

I am happy to announce that with the release of Swift 5.3.1 statically linking the Swift stdlib components is now fully supported on Linux. This includes linking against Dispatch and the different Foundation components. Additionally building self-contained binaries is now possible by manually adding a few linker flags.

How to use it?

The Swift compiler supports two different static linking modes:

static-stdlib

The -static-stdlib flag tells the Swift compiler to only statically link the components of the Swift stdlib, including Foundation and Dispatch. Any transitive dependencies will still be dynamically linked. This mode is very useful when deploying to a target that is known to have all Swift dependencies installed, but not Swift itself. It also allows running several applications that use different Swift versions on the same host without any special configuration.

static-executable

This mode tells the Swift compiler to build a self contained binary. It will statically link any dependency, including system dependencies. This mode is useful when deploying to a host that is not known to have any of Swift's dependencies installed, but produces larger binaries and makes deploying security patches harder, because the app has to be rebuilt against the patched libraries and redeployed.

Building fully self contained binaries with -static-executable works out of the box for Dispatch and Foundation, because they don't have any additional dependencies, but requires the user to provide transitive dependencies for FoundationXML and FoundationNetworking. The depedencies can be provided to the linker through the -Xlinker flag, e.g. swiftc -Xlinker -lxml2.

What dependencies need to be passed can be determined by looking at the modulemap for the particular library. For FoundationXML that is located under /usr/lib/swift_static/CFXMLInterface/module.map, for FoundationNetworking is is located under /usr/lib/swift_static/CFURLSessionInterface/module.map.

The steps for FoundationXML and FoundationNetworking are the same, but let's look at FoundationXML as an example.

The content of the module map file looks as follows:

module CFXMLInterface [extern_c] [system] {
    umbrella header "CFXMLInterface.h"

    link "CFXMLInterface"
    link "xml2"
}

The only direct dependency here is libxml2, but that itself has additional dependencies that need to be passed to the linker as well. The easiest way to determine those transitive dependencies is by asking pkg-config:

pkg-config --libs --static libxml-2.0

This command returns all the flags necessary to statically link against libxml2. On Ubuntu 18.04 the dependencies are as follows:

-lxml2 -licui18n -licuuc -licudata -lz -llzma -lm

So a complete Swift compiler invocation to create a self contained executable that uses FoundationXML would look something like:

swiftc -static-executable -Xlinker -licui18n -Xlinker -licuuc -Xlinker -licudata -Xlinker -lz -Xlinker -llzma -o myApp main.swift

-lxml2 and -lm can be omitted, as they are already included in the linker flags.

One might wonder why we don't include these depdencies in the module map as well. The short answer is that different systems may have different versions and the dependencies installed, which in turn may have different transitive depdencies. On CentOS8 for example the dependencies for libxml2 are:

-lxml2 -lz -llzma -lm

For projects utilizing Swift Package Manager, those flags should be added to the linkerSettings of the target:

// swift-tools-version:5.3
import PackageDescription

let package = Package(
   name: "MyApp",
   dependencies: [],
   targets: [
       .target(
           name: "MyApp",
           dependencies: [],
           linkerSettings: [
             .linkedLibrary("icui18n"),
             .linkedLibrary("icuuc"),
             .linkedLibrary("icudata"),
             .linkedLibrary("z"),
             .linkedLibrary("lzma")
           ]
       )
   ]
)

Additionally we need to pass the -static-executable flag to the compiler, so the invocation looks as follows:

swift build -Xswiftc -static-executable

What was broken and how did we fix it?

Dispatch and Foundation are partially written in C and have dependencies on other C libraries like libxml2 and curl. When using dynamic linking, the CoreFoundation libraries (the part that's written in C) are included in the shared objects that also contain the Swift part, so there is only one library file per module. In the static linking case the C libraries are separate and have to be explicitly linked against. To interface with C libraries, Swift relies on Clang module maps, which among other things also contain the libraries to link against. Because there are no separate C libraries in the dynamic linking case, the module maps don't contain any linking instructions, so when statically linking against Foundation, the linker could not find the symbols that are part of the C library.

To fix this, we first needed to generate separate module maps for the dynamic and static linking cases. The relevant patches for this are:

  • Add modulemaps that work for statically compiled Foundation [#2850]
  • Add modulemap for static compilation [#544]

Doing this also uncovered that the compiler did not properly compute the resource path when statically linking and instead always used the shared resource folder. This was fixed in the following patch:

  • Properly compute resource folder when linking statically [#33168]

I hope this helps shed some light on static linking in Swift 5.3.1.

55 Likes

This is really interesting article...but do you know how to create a SwiftPM script that works both on Linux and Mac ? because these libs are linux only.

Yes, you can define a computed property in your Package.swift that produces different settings based on the platform and then just reference that in your package description:

// swift-tools-version:5.3
import PackageDescription

var linkerSettings: [LinkerSetting]? {
  #if os(Linux)
  return [
    .linkedLibrary("icui18n"),
    .linkedLibrary("icuuc"),
    .linkedLibrary("icudata"),
    .linkedLibrary("z"),
    .linkedLibrary("lzma")
  ]
  #else
  return nil
  #endif
}

let package = Package(
   name: "MyApp",
   dependencies: [],
   targets: [
       .target(
           name: "MyApp",
           dependencies: [],
           linkerSettings: linkerSettings
       )
   ]
)
4 Likes

Has anyone tried if this works with libraries like Swift Crypto, and maybe with something more complex depending on it, like Vapor?

Is there a way to enable this in Package.swift? I tried adding the -static-stdlib flag to my target, but that doesn't appear to be working when I have dependencies.

If I add it to just swiftSettings like so:

import PackageDescription

let package = Package(
    name: "MyCommand",
    dependencies: [
        // dependencies added here
    ],
    targets: [
        .target(
            name: "MyCommand",
            dependencies: [
                // dependencies added here
            ],
            swiftSettings: [
                .unsafeFlags(["-static-stdlib"], .when(platforms: [.linux])),
            ]),
    ]
)

it doesn't seem to change anything (as confirmed by ldd on the compiled binary).

If I also add it to linkerSettings like this:

import PackageDescription

let package = Package(
    name: "MyCommand",
    dependencies: [
        // dependencies added here
    ],
    targets: [
        .target(
            name: "MyCommand",
            dependencies: [
                // dependencies added here
            ],
            swiftSettings: [
                .unsafeFlags(["-static-stdlib"], .when(platforms: [.linux])),
            ],
            linkerSettings: [
                .unsafeFlags(["-static-stdlib"], .when(platforms: [.linux])),
            ]),
    ]
)

I get a whole bunch of error: undefined reference to errors from the linker.

I think it's because the dependencies aren't built with that flag, since if I remove the dependencies, the build works (and ldd confirms the static compilation).

Is there a way to apply swiftSettings and linkerSettings to dependencies in Package.swift?

1 Like
Terms of Service

Privacy Policy

Cookie Policy