Static linking on Linux in Swift 5.3.1

Static linking on Linux

I am happy to announce that with the release of Swift 5.3.1 statically linking the Swift stdlib components is now fully supported on Linux. This includes linking against Dispatch and the different Foundation components. Additionally building self-contained binaries is now possible by manually adding a few linker flags.

How to use it?

The Swift compiler supports two different static linking modes:

static-stdlib

The -static-stdlib flag tells the Swift compiler to only statically link the components of the Swift stdlib, including Foundation and Dispatch. Any transitive dependencies will still be dynamically linked. This mode is very useful when deploying to a target that is known to have all Swift dependencies installed, but not Swift itself. It also allows running several applications that use different Swift versions on the same host without any special configuration.

static-executable

This mode tells the Swift compiler to build a self contained binary. It will statically link any dependency, including system dependencies. This mode is useful when deploying to a host that is not known to have any of Swift's dependencies installed, but produces larger binaries and makes deploying security patches harder, because the app has to be rebuilt against the patched libraries and redeployed.

Building fully self contained binaries with -static-executable works out of the box for Dispatch and Foundation, because they don't have any additional dependencies, but requires the user to provide transitive dependencies for FoundationXML and FoundationNetworking. The depedencies can be provided to the linker through the -Xlinker flag, e.g. swiftc -Xlinker -lxml2.

What dependencies need to be passed can be determined by looking at the modulemap for the particular library. For FoundationXML that is located under /usr/lib/swift_static/CFXMLInterface/module.map, for FoundationNetworking is is located under /usr/lib/swift_static/CFURLSessionInterface/module.map.

The steps for FoundationXML and FoundationNetworking are the same, but let's look at FoundationXML as an example.

The content of the module map file looks as follows:

module CFXMLInterface [extern_c] [system] {
    umbrella header "CFXMLInterface.h"

    link "CFXMLInterface"
    link "xml2"
}

The only direct dependency here is libxml2, but that itself has additional dependencies that need to be passed to the linker as well. The easiest way to determine those transitive dependencies is by asking pkg-config:

pkg-config --libs --static libxml-2.0

This command returns all the flags necessary to statically link against libxml2. On Ubuntu 18.04 the dependencies are as follows:

-lxml2 -licui18n -licuuc -licudata -lz -llzma -lm

So a complete Swift compiler invocation to create a self contained executable that uses FoundationXML would look something like:

swiftc -static-executable -Xlinker -licui18n -Xlinker -licuuc -Xlinker -licudata -Xlinker -lz -Xlinker -llzma -o myApp main.swift

-lxml2 and -lm can be omitted, as they are already included in the linker flags.

One might wonder why we don't include these depdencies in the module map as well. The short answer is that different systems may have different versions and the dependencies installed, which in turn may have different transitive depdencies. On CentOS8 for example the dependencies for libxml2 are:

-lxml2 -lz -llzma -lm

For projects utilizing Swift Package Manager, those flags should be added to the linkerSettings of the target:

// swift-tools-version:5.3
import PackageDescription

let package = Package(
   name: "MyApp",
   dependencies: [],
   targets: [
       .target(
           name: "MyApp",
           dependencies: [],
           linkerSettings: [
             .linkedLibrary("icui18n"),
             .linkedLibrary("icuuc"),
             .linkedLibrary("icudata"),
             .linkedLibrary("z"),
             .linkedLibrary("lzma")
           ]
       )
   ]
)

Additionally we need to pass the -static-executable flag to the compiler, so the invocation looks as follows:

swift build -Xswiftc -static-executable

What was broken and how did we fix it?

Dispatch and Foundation are partially written in C and have dependencies on other C libraries like libxml2 and curl. When using dynamic linking, the CoreFoundation libraries (the part that's written in C) are included in the shared objects that also contain the Swift part, so there is only one library file per module. In the static linking case the C libraries are separate and have to be explicitly linked against. To interface with C libraries, Swift relies on Clang module maps, which among other things also contain the libraries to link against. Because there are no separate C libraries in the dynamic linking case, the module maps don't contain any linking instructions, so when statically linking against Foundation, the linker could not find the symbols that are part of the C library.

To fix this, we first needed to generate separate module maps for the dynamic and static linking cases. The relevant patches for this are:

  • Add modulemaps that work for statically compiled Foundation [#2850]
  • Add modulemap for static compilation [#544]

Doing this also uncovered that the compiler did not properly compute the resource path when statically linking and instead always used the shared resource folder. This was fixed in the following patch:

  • Properly compute resource folder when linking statically [#33168]

I hope this helps shed some light on static linking in Swift 5.3.1.

61 Likes

This is really interesting article...but do you know how to create a SwiftPM script that works both on Linux and Mac ? because these libs are linux only.

Yes, you can define a computed property in your Package.swift that produces different settings based on the platform and then just reference that in your package description:

// swift-tools-version:5.3
import PackageDescription

var linkerSettings: [LinkerSetting]? {
  #if os(Linux)
  return [
    .linkedLibrary("icui18n"),
    .linkedLibrary("icuuc"),
    .linkedLibrary("icudata"),
    .linkedLibrary("z"),
    .linkedLibrary("lzma")
  ]
  #else
  return nil
  #endif
}

let package = Package(
   name: "MyApp",
   dependencies: [],
   targets: [
       .target(
           name: "MyApp",
           dependencies: [],
           linkerSettings: linkerSettings
       )
   ]
)
4 Likes

Has anyone tried if this works with libraries like Swift Crypto, and maybe with something more complex depending on it, like Vapor?

Is there a way to enable this in Package.swift? I tried adding the -static-stdlib flag to my target, but that doesn't appear to be working when I have dependencies.

If I add it to just swiftSettings like so:

import PackageDescription

let package = Package(
    name: "MyCommand",
    dependencies: [
        // dependencies added here
    ],
    targets: [
        .target(
            name: "MyCommand",
            dependencies: [
                // dependencies added here
            ],
            swiftSettings: [
                .unsafeFlags(["-static-stdlib"], .when(platforms: [.linux])),
            ]),
    ]
)

it doesn't seem to change anything (as confirmed by ldd on the compiled binary).

If I also add it to linkerSettings like this:

import PackageDescription

let package = Package(
    name: "MyCommand",
    dependencies: [
        // dependencies added here
    ],
    targets: [
        .target(
            name: "MyCommand",
            dependencies: [
                // dependencies added here
            ],
            swiftSettings: [
                .unsafeFlags(["-static-stdlib"], .when(platforms: [.linux])),
            ],
            linkerSettings: [
                .unsafeFlags(["-static-stdlib"], .when(platforms: [.linux])),
            ]),
    ]
)

I get a whole bunch of error: undefined reference to errors from the linker.

I think it's because the dependencies aren't built with that flag, since if I remove the dependencies, the build works (and ldd confirms the static compilation).

Is there a way to apply swiftSettings and linkerSettings to dependencies in Package.swift?

1 Like

Yes, I think that is indeed because the flag needs to be passed to all projects, including the dependencies. I think there is already a patch for SPM that would allow you to pass -static-swift-stdlib to swift build instead of the compiler flag, but there is no option to enable it in the package definition.

@tomerd @NeoNacho @abertelrud would be the right people to talk to about adding an option for it.

Hey, I am not sure what I am doing wrong / missing, or if I completely misunderstood. I have a simple executable SPM project that has a single external dependency on swift-argument-parser and uses FoundationNetworking. I want to build it with -static-stdlib, but I get lots of errors during linking ...

$ docker run --rm -v (pwd):(pwd) -w (pwd) -it \
         swift:5.3.1 \
         swift build -Xswiftc -static-stdlib

/usr/lib/swift_static/linux/libFoundation.a(Data.swift.o):Data.swift.o:function _withStackOrHeapBuffer: error: undefined reference to '_CFIsMainThread'
/usr/lib/swift_static/linux/libFoundation.a(Data.swift.o):Data.swift.o:function $sSmsE6insert_2aty7ElementQzn_5IndexQztF10Foundation4DataV_Tg5: error: undefined reference to '_CFIsMainThread'
/usr/lib/swift_static/linux/libFoundation.a(Data.swift.o):Data.swift.o:function $sSmsE6remove2at7ElementQz5IndexQz_tF10Foundation4DataV_Tg5: error: undefined reference to '_CFIsMainThread'
/usr/lib/swift_static/linux/libFoundation.a(Data.swift.o):Data.swift.o:function $sSmsE14removeSubrangeyySny5IndexQzGF10Foundation4DataV_Tg5: error: undefined reference to '_CFIsMainThread'
/usr/lib/swift_static/linux/libFoundation.a(JSONEncoder.swift.o):JSONEncoder.swift.o:function $s10Foundation12_JSONDecoder33_12768CA107A31EF2DCE034FD75B541C9LLC5unbox_2asSdSgyp_SdmtKFTf4ndn_n: error: undefined reference to 'kCFBooleanTrue'
/usr/lib/swift_static/linux/libFoundation.a(JSONEncoder.swift.o):JSONEncoder.swift.o:function $s10Foundation12_JSONDecoder33_12768CA107A31EF2DCE034FD75B541C9LLC5unbox_2asSdSgyp_SdmtKFTf4ndn_n: error: undefined reference to 'kCFBooleanFalse'
/usr/lib/swift_static/linux/libFoundation.a(JSONEncoder.swift.o):JSONEncoder.swift.o:function $s10Foundation12_JSONDecoder33_12768CA107A31EF2DCE034FD75B541C9LLC5unbox_2asSbSgyp_SbmtKFTf4ndn_n: error: undefined reference to 'CFBooleanGetTypeID'
/usr/lib/swift_static/linux/libFoundation.a(JSONEncoder.swift.o):JSONEncoder.swift.o:function $s10Foundation12_JSONDecoder33_12768CA107A31EF2DCE034FD75B541C9LLC5unbox_2asSiSgyp_SimtKFTf4ndn_n: error: undefined reference to 'kCFBooleanTrue'
/usr/lib/swift_static/linux/libFoundation.a(JSONEncoder.swift.o):JSONEncoder.swift.o:function $s10Foundation12_JSONDecoder33_12768CA107A31EF2DCE034FD75B541C9LLC5unbox_2asSiSgyp_SimtKFTf4ndn_n: error: undefined reference to 'kCFBooleanFalse'
/usr/lib/swift_static/linux/libFoundation.a(JSONEncoder.swift.o):JSONEncoder.swift.o:function $s10Foundation12_JSONDecoder33_12768CA107A31EF2DCE034FD75B541C9LLC5unbox_2ass4Int8VSgyp_AHmtKFTf4ndn_n: error: undefined reference to 'kCFBooleanTrue'
/usr/lib/swift_static/linux/libFoundation.a(JSONEncoder.swift.o):JSONEncoder.swift.o:function $s10Foundation12_JSONDecoder33_12768CA107A31EF2DCE034FD75B541C9LLC5unbox_2ass4Int8VSgyp_AHmtKFTf4ndn_n: error: undefined reference to 'kCFBooleanFalse'
/usr/lib/swift_static/linux/libFoundation.a(JSONEncoder.swift.o):JSONEncoder.swift.o:function $s10Foundation12_JSONDecoder33_12768CA107A31EF2DCE034FD75B541C9LLC5unbox_2ass5Int16VSgyp_AHmtKFTf4ndn_n: error: undefined reference to 'kCFBooleanTrue'
/usr/lib/swift_static/linux/libFoundation.a(JSONEncoder.swift.o):JSONEncoder.swift.o:function $s10Foundation12_JSONDecoder33_12768CA107A31EF2DCE034FD75B541C9LLC5unbox_2ass5Int16VSgyp_AHmtKFTf4ndn_n: error: undefined reference to 'kCFBooleanFalse'
/usr/lib/swift_static/linux/libFoundation.a(JSONSerialization.swift.o):JSONSerialization.swift.o:function $s10Foundation17JSONSerializationC17isValidJSONObjectySbypFZ0cdE8InternalL_ySbypSgF: error: undefined reference to 'CFNumberIsFloatType'
/usr/lib/swift_static/linux/libFoundation.a(NSArray.swift.o):NSArray.swift.o:function $s10Foundation7NSArrayC7_cfinfo33_C60286EB6896636ED9A0C67CDB8DE899LLAA7_CFInfoVvpfi: error: undefined reference to 'CFArrayGetTypeID'
/usr/lib/swift_static/linux/libFoundation.a(NSArray.swift.o):NSArray.swift.o:function $s10Foundation7NSArrayC7objects5countACSPyyXlGSg_Sitcfc: error: undefined reference to 'CFArrayGetTypeID'
/usr/lib/swift_static/linux/libFoundation.a(NSArray.swift.o):NSArray.swift.o:function $s10Foundation7NSArrayC4copy4withypAA6NSZoneVSg_tF: error: undefined reference to 'CFArrayGetTypeID'
/usr/lib/swift_static/linux/libFoundation.a(NSArray.swift.o):NSArray.swift.o:function $s10Foundation7NSArrayC11mutableCopy4withypAA6NSZoneVSg_tF: error: undefined reference to 'CFArrayGetTypeID'
/usr/lib/swift_static/linux/libFoundation.a(NSArray.swift.o):NSArray.swift.o:function $s10Foundation7NSArrayC11sortedArray4from7options15usingCompae[2K
ratorSayypGAA8_NSRangeV_AA13NSSortOptionsVAA16ComparisonResultOyp_yptXEtF: error: undefined reference to 'CFSortIndexes'
/usr/lib/swift_static/linux/libFoundation.a(NSArray.swift.o):NSArray.swift.o:function $sSo10CFArrayRefa10FoundationE18_unsafeTypedBridgeSayxGyAC13_CFBridgeableRzlF: error: undefined reference to 'CFArrayGetCount'
/usr/lib/swift_static/linux/libFoundation.a(NSArray.swift.o):NSArray.swift.o:function $sSo10CFArrayRefa10FoundationE18_unsafeTypedBridgeSayxGyAC13_CFBridgeableRzlF: error: undefined reference to 'CFArrayGetValueAtIndex'
/usr/lib/swift_static/linux/libFoundation.a(NSArray.swift.o):NSArray.swift.o:function $s10Foundation14NSMutableArrayC4sort5usingySayAA16NSSortDescriptorCG_tF: error: undefined reference to 'CFArraySortValues'
/usr/lib/swift_static/linux/libFoundation.a(NSData.swift.o):NSData.swift.o:function $s10Foundation6NSDataC5_base33_6A2A18DBA81B32BFAF1406C41D05FDF5LLAA7_CFInfoVvpfi: error: undefined reference to 'CFDataGetTypeID'
/usr/lib/swift_static/linux/libFoundation.a(NSData.swift.o):NSData.swift.o:function $s10Foundation6NSDataC9_cfObjectSo9CFDataRefavg: error: undefined reference to 'kCFAllocatorSystemDefault'
/usr/lib/swift_static/linux/libFoundation.a(NSData.swift.o):NSData.swift.o:function $s10Foundation6NSDataC9_cfObjectSo9CFDataRefavg: error: undefined reference to 'CFDataCreate'
/usr/lib/swift_static/linux/libFoundation.a(NSData.swift.o):NSData.swift.o:function $s10Foundation6NSDataC9_cfTypeIDSuvg: error: undefined reference to 'CFDataGetTypeID'
/usr/lib/swift_static/linux/libFoundation.a(NSData.swift.o):NSData.swift.o:function $s10Foundation6NSDataC5_init33_6A2A18DBA81B32BFAF1406C41D05FDF5LL5bytes6length4copy11deallocatorySvSg_SiSbySv_SitcSgtF: error: undefined reference to '_CFDataInit'
/usr/lib/swift_static/linux/libFoundation.a(NSData.swift.o):NSData.swift.o:function $s10Foundation6NSDataC5_init33_6A2A18DBA81B32BFAF1406C41D05FDF5LL5bytes6length4copy11deallocatorySvSg_SiSbySv_SitcSgtF: error: undefined reference to '_CFDataInit'
/usr/lib/swift_static/linux/libFoundation.a(NSData.swift.o):NSData.swift.o:function $s10Foundation6NSDataCACycfC: error: undefined reference to 'CFDataGetTypeID'
/usr/lib/swift_static/linux/libFoundation.a(NSData.swift.o):NSData.swift.o:function $s10Foundation6NSDataCACycfC: error: undefined reference to '_CFDataInit'
/usr/lib/swift_static/linux/libFoundation.a(NSData.swift.o):NSData.swift.o:function $s10Foundation6NSDataCACycfc: error: undefined reference to 'CFDataGetTypeID'
/usr/lib/swift_static/linux/libFoundation.a(NSData.swift.o):NSData.swift.o:function $s10Foundation6NSDataCACycfc: error: undefined reference to '_CFDataInit'
/usr/lib/swift_static/linux/libFoundation.a(NSData.swift.o):NSData.swift.o:function $s10Foundation6NSDataCfd: error: undefined reference to '_CFDeinit'
/usr/lib/swift_static/linux/libFoundation.a(NSData.swift.o):NSData.swift.o:function $s10Foundation6NSDataC6lengthSivg: error: undefined reference to 'CFDataGetLength'
/usr/lib/swift_static/linux/libFoundation.a(NSData.swift.o):NSData.swift.o:function $s10Foundation6NSDataC5bytesSVvg: error: undefined reference to 'CFDataGetBytePtr'
/usr/lib/swift_static/linux/libFoundation.a(NSData.swift.o):NSData.swift.o:function $s10Foundation6NSDataC4hashSivg: error: undefined reference to '_CFNonObjCHash'
/usr/lib/swift_static/linux/libFoundation.a(NSData.swift.o):NSData.swift.o:function $s10Foundation6NSDataC8getBytes_6lengthySv_SitF: error: undefined reference to 'CFDataGetBytes'
/usr/lib/swift_static/linux/libFoundation.a(NSData.swift.o):NSData.swift.o:function $s10Foundation6NSDataC8getBytes_5rangeySv_AA8_NSRangeVtF: error: undefined reference to 'CFDataGetBytes'
/usr/lib/swift_static/linux/libFoundation.a(NSData.swift.o):NSData.swift.o:function $s10Foundation13NSMutableDataC12mutableBytesSvvg: error: undefined reference to 'CFDataGetMutableBytePtr'
/usr/lib/swift_static/linux/libFoundation.a(NSData.swift.o):NSData.swift.o:function $s10Foundation13NSMutableDataC6lengthSivg: error: undefined reference to 'CFDataGetLength'
/usr/lib/swift_static/linux/libFoundation.a(NSData.swift.o):e[2K
NSData.swift.o:function $s10Foundation13NSMutableDataC6lengthSivs: error: undefined reference to 'CFDataSetLength'
/usr/lib/swift_static/linux/libFoundation.a(NSData.swift.o):NSData.swift.o:function $s10Foundation13NSMutableDataC6append_6lengthySV_SitF: error: undefined reference to 'CFDataAppendBytes'
/usr/lib/swift_static/linux/libFoundation.a(NSData.swift.o):NSData.swift.o:function $s10Foundation13NSMutableDataC14increaseLength2byySi_tF: error: undefined reference to 'CFDataGetLength'
/usr/lib/swift_static/linux/libFoundation.a(NSData.swift.o):NSData.swift.o:function $s10Foundation13NSMutableDataC14increaseLength2byySi_tF: error: undefined reference to 'CFDataSetLength'
/usr/lib/swift_static/linux/libFoundation.a(NSData.swift.o):NSData.swift.o:function $s10Foundation13NSMutableDataC12replaceBytes2in04withE0yAA8_NSRangeV_SVtF: error: undefined reference to 'CFDataReplaceBytes'
/usr/lib/swift_static/linux/libFoundation.a(NSData.swift.o):NSData.swift.o:function $s10Foundation13NSMutableDataC12replaceBytes2in04withE06lengthyAA8_NSRangeV_SVSgSitF: error: undefined reference to 'CFDataReplaceBytes'
/usr/lib/swift_static/linux/libFoundation.a(NSDictionary.swift.o):NSDictionary.swift.o:function $s10Foundation12NSDictionaryC7_cfinfo33_8C53A72A73712D943F7602D2573DFCFBLLAA7_CFInfoVvpfi: error: undefined reference to 'CFDictionaryGetTypeID'
/usr/lib/swift_static/linux/libFoundation.a(NSDictionary.swift.o):NSDictionary.swift.o:function $s10Foundation12NSDictionaryC7objects7forKeys5countACSPyyXlGSg_SPyAA8NSObjectCGSgSitcfc: error: undefined reference to 'CFDictionaryGetTypeID'
/usr/lib/swift_static/linux/libFoundation.a(NSDictionary.swift.o):NSDictionary.swift.o:function $s10Foundation12NSDictionaryC4copy4withypAA6NSZoneVSg_tF: error: undefined reference to 'CFDictionaryGetTypeID'
/usr/lib/swift_static/linux/libFoundation.a(NSDictionary.swift.o):NSDictionary.swift.o:function $s10Foundation12NSDictionaryC11mutableCopy4withypAA6NSZoneVSg_tF: error: undefined reference to 'CFDictionaryGetTypeID'
/usr/lib/swift_static/linux/libFoundation.a(NSError.swift.o):NSError.swift.o:function $s10Foundation7NSErrorC9_cfObjectSo10CFErrorRefavg: error: undefined reference to 'kCFAllocatorSystemDefault'
/usr/lib/swift_static/linux/libFoundation.a(NSError.swift.o):NSError.swift.o:function $s10Foundation7NSErrorC9_cfObjectSo10CFErrorRefavg: error: undefined reference to 'CFErrorCreate'
/usr/lib/swift_static/linux/libFoundation.a(NSError.swift.o):NSError.swift.o:function $s10Foundation7NSErrorC4hashSivg: error: undefined reference to 'CFHash'
/usr/lib/swift_static/linux/libFoundation.a(NSError.swift.o):NSError.swift.o:function $sSo10CFErrorRefa10FoundationE9_nsObjectAC7NSErrorCvg: error: undefined reference to 'CFErrorCopyUserInfo'
/usr/lib/swift_static/linux/libFoundation.a(NSError.swift.o):NSError.swift.o:function $sSo10CFErrorRefa10FoundationE9_nsObjectAC7NSErrorCvg: error: undefined reference to 'CFErrorGetDomain'
/usr/lib/swift_static/linux/libFoundation.a(NSError.swift.o):NSError.swift.o:function $sSo10CFErrorRefa10FoundationE9_nsObjectAC7NSErrorCvg: error: undefined reference to 'objc_retainAutoreleasedReturnValue'
/usr/lib/swift_static/linux/libFoundation.a(NSError.swift.o):NSError.swift.o:function $sSo10CFErrorRefa10FoundationE9_nsObjectAC7NSErrorCvg: error: undefined reference to 'CFErrorGetCode'
/usr/lib/swift_static/linux/libFoundation.a(NSError.swift.o):NSError.swift.o:function $sSo10CFErrorRefa10FoundationE7_domainSSvg: error: undefined reference to 'CFErrorGetDomain'
/usr/lib/swift_static/linux/libFoundation.a(NSError.swift.o):NSError.swift.o:function $sSo10CFErrorRefa10FoundationE7_domainSSvg: error: undefined reference to 'objc_retainAutoreleasedReturnValue'
/usr/lib/swift_static/linux/libFoundation.a(NSError.swift.o):NSError.swift.o:function $sSo10CFErrorRefa10FoundationE5_codeSivg: error: undefined reference to 'CFErrorGetCode'
/usr/lib/swift_static/linux/libFoundation.a(NSError.swift.o):NSError.swift.o:function $sSo10CFErrorRefa10FoundationE9_userInfoyXlSgvg: error: undefined reference to 'CFErrorCopyUserInfo'
/usr/lib/swift_static/le[2K
inux/libFoundation.a(NSError.swift.o):NSError.swift.o:function $sSo10CFErrorRefas5Error10FoundationsACP7_domainSSvgTW: error: undefined reference to 'CFErrorGetDomain'
/usr/lib/swift_static/linux/libFoundation.a(NSError.swift.o):NSError.swift.o:function $sSo10CFErrorRefas5Error10FoundationsACP7_domainSSvgTW: error: undefined reference to 'objc_retainAutoreleasedReturnValue'
/usr/lib/swift_static/linux/libFoundation.a(NSError.swift.o):NSError.swift.o:function $sSo10CFErrorRefas5Error10FoundationsACP5_codeSivgTW: error: undefined reference to 'CFErrorGetCode'
/usr/lib/swift_static/linux/libFoundation.a(NSError.swift.o):NSError.swift.o:function $sSo10CFErrorRefas5Error10FoundationsACP9_userInfoyXlSgvgTW: error: undefined reference to 'CFErrorCopyUserInfo'
/usr/lib/swift_static/linux/libFoundation.a(NSKeyedArchiver.swift.o):NSKeyedArchiver.swift.o:function $s10Foundation15NSKeyedArchiverC17archiveRootObject_6toFileSbyp_SStFZ: error: undefined reference to 'kCFAllocatorSystemDefault'
/usr/lib/swift_static/linux/libFoundation.a(NSKeyedArchiver.swift.o):NSKeyedArchiver.swift.o:function $s10Foundation15NSKeyedArchiverC17archiveRootObject_6toFileSbyp_SStFZ: error: undefined reference to '_CFWriteStreamCreateFromFileDescriptor'
/usr/lib/swift_static/linux/libFoundation.a(NSKeyedArchiver.swift.o):NSKeyedArchiver.swift.o:function $s10Foundation15NSKeyedArchiverC17archiveRootObject_6toFileSbyp_SStFZ: error: undefined reference to 'CFWriteStreamOpen'
/usr/lib/swift_static/linux/libFoundation.a(NSKeyedArchiver.swift.o):NSKeyedArchiver.swift.o:function $s10Foundation15NSKeyedArchiverC17archiveRootObject_6toFileSbyp_SStFZ: error: undefined reference to 'CFWriteStreamClose'
/usr/lib/swift_static/linux/libFoundation.a(NSKeyedArchiver.swift.o):NSKeyedArchiver.swift.o:function $s10Foundation15NSKeyedArchiverC13_writeXMLData33_885F476806621E9C5CBC3167C719EC73LLySbAA12NSDictionaryCF: error: undefined reference to 'kCFAllocatorSystemDefault'
/usr/lib/swift_static/linux/libFoundation.a(NSKeyedArchiver.swift.o):NSKeyedArchiver.swift.o:function $s10Foundation15NSKeyedArchiverC13_writeXMLData33_885F476806621E9C5CBC3167C719EC73LLySbAA12NSDictionaryCF: error: undefined reference to '_CFPropertyListCreateXMLDataWithExtras'
/usr/lib/swift_static/linux/libFoundation.a(NSKeyedArchiver.swift.o):NSKeyedArchiver.swift.o:function $s10Foundation15NSKeyedArchiverC13_writeXMLData33_885F476806621E9C5CBC3167C719EC73LLySbAA12NSDictionaryCF: error: undefined reference to 'CFPropertyListWrite'
/usr/lib/swift_static/linux/libFoundation.a(NSKeyedArchiver.swift.o):NSKeyedArchiver.swift.o:function $s10Foundation15NSKeyedArchiverC14finishEncodingyyF: error: undefined reference to '__CFBinaryPlistWriteToStream'
/usr/lib/swift_static/linux/libFoundation.a(NSKeyedArchiverHelpers.swift.o):NSKeyedArchiverHelpers.swift.o:function $s10Foundation19_NSKeyedArchiverUIDC5_baseAA7_CFInfoVvpfi: error: undefined reference to '_CFKeyedArchiverUIDGetTypeID'
/usr/lib/swift_static/linux/libFoundation.a(NSKeyedArchiverHelpers.swift.o):NSKeyedArchiverHelpers.swift.o:function $s10Foundation19_NSKeyedArchiverUIDC9_cfTypeIDSuvg: error: undefined reference to '_CFKeyedArchiverUIDGetTypeID'
/usr/lib/swift_static/linux/libFoundation.a(NSKeyedArchiverHelpers.swift.o):NSKeyedArchiverHelpers.swift.o:function $s10Foundation19_NSKeyedArchiverUIDC4hashSivg: error: undefined reference to 'CFHash'
/usr/lib/swift_static/linux/libFoundation.a(NSKeyedArchiverHelpers.swift.o):NSKeyedArchiverHelpers.swift.o:function $s10Foundation19_NSKeyedArchiverUIDC5valueACs6UInt32V_tcfC: error: undefined reference to '_CFKeyedArchiverUIDGetTypeID'
/usr/lib/swift_static/linux/libFoundation.a(NSKeyedArchiverHelpers.swift.o):NSKeyedArchiverHelpers.swift.o:function $s10Foundation19_NSKeyedArchiverUIDC5valueACs6UInt32V_tcfc: error: undefined reference to '_CFKeyedArchiverUIDGetTypeID'
/usr/lib/swift_static/linux/libFoundation.a(NSKeyedArchiverHelpers.swift.o):NSKeyedArchiverHelpers.swift.o:function $s10Foundation19_NSKeyedArchiverUIDCfd: error: undefined reference to '_CFDeinit'
/usr/lib/swift_static/linux/libFoundation.a(NSKee[2K
yedArchiverHelpers.swift.o):NSKeyedArchiverHelpers.swift.o:function $s10Foundation19_NSKeyedArchiverUIDCfD: error: undefined reference to '_CFDeinit'
/usr/lib/swift_static/linux/libFoundation.a(NSKeyedUnarchiver.swift.o):NSKeyedUnarchiver.swift.o:function $s10Foundation17NSKeyedUnarchiverC15unarchiveObject8withFileypSgSS_tFZ: error: undefined reference to 'CFReadStreamCreateWithFile'
/usr/lib/swift_static/linux/libFoundation.a(NSKeyedUnarchiver.swift.o):NSKeyedUnarchiver.swift.o:function $s10Foundation17NSKeyedUnarchiverC15unarchiveObject8withFileypSgSS_tFZ: error: undefined reference to 'CFReadStreamOpen'
/usr/lib/swift_static/linux/libFoundation.a(NSKeyedUnarchiver.swift.o):NSKeyedUnarchiver.swift.o:function $s10Foundation17NSKeyedUnarchiverC15unarchiveObject8withFileypSgSS_tFZ: error: undefined reference to 'CFReadStreamClose'
/usr/lib/swift_static/linux/libFoundation.a(NSLocale.swift.o):NSLocale.swift.o:function $s10Foundation8NSLocaleC5_base33_0706E4AB06E3E8E1B389A0E2B07B4002LLAA7_CFInfoVvpfi: error: undefined reference to 'CFLocaleGetTypeID'
/usr/lib/swift_static/linux/libFoundation.a(NSLocale.swift.o):NSLocale.swift.o:function $s10Foundation8NSLocaleC6object6forKeyypSgAC0E0V_tF: error: undefined reference to 'CFLocaleGetValue'
/usr/lib/swift_static/linux/libFoundation.a(NSLocale.swift.o):NSLocale.swift.o:function $s10Foundation8NSLocaleC6object6forKeyypSgAC0E0V_tF: error: undefined reference to 'objc_retainAutoreleasedReturnValue'
/usr/lib/swift_static/linux/libFoundation.a(NSLocale.swift.o):NSLocale.swift.o:function $s10Foundation8NSLocaleC11displayName6forKey5valueSSSgAC0F0V_SStF: error: undefined reference to 'CFLocaleCopyDisplayNameForPropertyValue'
/usr/lib/swift_static/linux/libFoundation.a(NSLocale.swift.o):NSLocale.swift.o:function $s10Foundation8NSLocaleCfd: error: undefined reference to '_CFDeinit'
/usr/lib/swift_static/linux/libFoundation.a(NSLocale.swift.o):NSLocale.swift.o:function $s10Foundation8NSLocaleC6encode4withyAA7NSCoderC_tF: error: undefined reference to 'CFLocaleGetIdentifier'
/usr/lib/swift_static/linux/libFoundation.a(NSLocale.swift.o):NSLocale.swift.o:function $s10Foundation8NSLocaleC7currentAA6LocaleVvgZ: error: undefined reference to 'CFLocaleCopyCurrent'
/usr/lib/swift_static/linux/libFoundation.a(NSLocale.swift.o):NSLocale.swift.o:function $s10Foundation8NSLocaleC6systemAA6LocaleVvgZ: error: undefined reference to 'CFLocaleGetSystem'
/usr/lib/swift_static/linux/libFoundation.a(NSLocale.swift.o):NSLocale.swift.o:function $s10Foundation8NSLocaleC26availableLocaleIdentifiersSaySSGvgZ: error: undefined reference to 'CFLocaleCopyAvailableLocaleIdentifiers'
/usr/lib/swift_static/linux/libFoundation.a(NSLocale.swift.o):NSLocale.swift.o:function $s10Foundation8NSLocaleC16isoLanguageCodesSaySSGvgZ: error: undefined reference to 'CFLocaleCopyISOLanguageCodes'
/usr/lib/swift_static/linux/libFoundation.a(NSLocale.swift.o):NSLocale.swift.o:function $s10Foundation8NSLocaleC15isoCountryCodesSaySSGvgZ: error: undefined reference to 'CFLocaleCopyISOCountryCodes'
/usr/lib/swift_static/linux/libFoundation.a(NSLocale.swift.o):NSLocale.swift.o:function $s10Foundation8NSLocaleC16isoCurrencyCodesSaySSGvgZ: error: undefined reference to 'CFLocaleCopyISOCurrencyCodes'
/usr/lib/swift_static/linux/libFoundation.a(NSLocale.swift.o):NSLocale.swift.o:function $s10Foundation8NSLocaleC22commonISOCurrencyCodesSaySSGvgZ: error: undefined reference to 'CFLocaleCopyCommonISOCurrencyCodes'
/usr/lib/swift_static/linux/libFoundation.a(NSLocale.swift.o):NSLocale.swift.o:function $s10Foundation8NSLocaleC18preferredLanguagesSaySSGvgZ: error: undefined reference to 'CFLocaleCopyPreferredLanguages'
/usr/lib/swift_static/linux/libFoundation.a(NSLocale.swift.o):NSLocale.swift.o:function $s10Foundation8NSLocaleC10components20fromLocaleIdentifierSDyS2SGSS_tFZ: error: undefined reference to 'CFLocaleCreateComponentsFromLocaleIdentifier'
/usr/lib/swift_static/linux/libFoundation.a(NSLocale.swift.o):NSLocale.swift.o:function $s10Foundation8NSLocaleC16localeIdentifier14fromComponentsSSSDyS2SG_tFZ: error: undefined reference to 'CFLocale[2K
eCreateLocaleIdentifierFromComponents'
/usr/lib/swift_static/linux/libFoundation.a(NSLocale.swift.o):NSLocale.swift.o:function $s10Foundation8NSLocaleC25canonicalLocaleIdentifier4fromS2S_tFZ: error: undefined reference to 'CFLocaleCreateCanonicalLocaleIdentifierFromString'
/usr/lib/swift_static/linux/libFoundation.a(NSLocale.swift.o):NSLocale.swift.o:function $s10Foundation8NSLocaleC27canonicalLanguageIdentifier4fromS2S_tFZ: error: undefined reference to 'CFLocaleCreateCanonicalLanguageIdentifierFromString'
/usr/lib/swift_static/linux/libFoundation.a(NSLocale.swift.o):NSLocale.swift.o:function $s10Foundation8NSLocaleC16localeIdentifier21fromWindowsLocaleCodeSSSgs6UInt32V_tFZ: error: undefined reference to 'CFLocaleCreateLocaleIdentifierFromWindowsLocaleCode'
/usr/lib/swift_static/linux/libFoundation.a(NSLocale.swift.o):NSLocale.swift.o:function $s10Foundation8NSLocaleC17windowsLocaleCode04fromD10Identifiers6UInt32VSS_tFZ: error: undefined reference to 'CFLocaleGetWindowsLocaleCodeFromLocaleIdentifier'
/usr/lib/swift_static/linux/libFoundation.a(NSLocale.swift.o):NSLocale.swift.o:function $s10Foundation8NSLocaleC18characterDirection11forLanguageAC0fD0OSS_tFZ: error: undefined reference to 'CFLocaleGetLanguageCharacterDirection'
/usr/lib/swift_static/linux/libFoundation.a(NSLocale.swift.o):NSLocale.swift.o:function $s10Foundation8NSLocaleC13lineDirection11forLanguageAC0fD0OSS_tFZ: error: undefined reference to 'CFLocaleGetLanguageLineDirection'
/usr/lib/swift_static/linux/libFoundation.a(NSLocale.swift.o):NSLocale.swift.o:function $s10Foundation8NSLocaleC16localeIdentifierACSS_tcfcTf4gn_n: error: undefined reference to 'CFLocaleGetTypeID'
/usr/lib/swift_static/linux/libFoundation.a(NSLocale.swift.o):NSLocale.swift.o:function $s10Foundation8NSLocaleC16localeIdentifierACSS_tcfcTf4gn_n: error: undefined reference to '_CFLocaleInit'
/usr/lib/swift_static/linux/libFoundation.a(NSLog.swift.o):NSLog.swift.o:function $s10Foundation6NSLogvyySS_s14CVaListPointerVtF: error: undefined reference to 'CFLog1'
/usr/lib/swift_static/linux/libFoundation.a(NSLog.swift.o):NSLog.swift.o:function $s10Foundation5NSLogyySS_s7CVarArg_pdtF: error: undefined reference to 'CFLog1'
/usr/lib/swift_static/linux/libFoundation.a(NSNumber.swift.o):NSNumber.swift.o:function $ss4Int8V10FoundationE7exactlyABSgAC8NSNumberC_tcfC: error: undefined reference to 'CFNumberCreate'

... [ and a whole lot more of those ]

$s20FoundationNetworking11_EasyHandleC15_CurlStringListCyAESaySSGcfcTf4gg_n: error: undefined reference to 'CFURLSessionSListAppend'
/usr/lib/swift_static/linux/libFoundationNetworking.a(EasyHandle.swift.o):EasyHandle.swift.o:function $s20FoundationNetworking11_EasyHandleC3set13requestMethodySS_tFySPys4Int8VGXEfU_TA: error: undefined reference to 'CFURLSessionOptionCUSTOMREQUEST'
/usr/lib/swift_static/linux/libFoundationNetworking.a(EasyHandle.swift.o):EasyHandle.swift.o:function $s20FoundationNetworking11_EasyHandleC15_CurlStringListC6appendyySSFySPys4Int8VGXEfU_TA: error: undefined reference to 'CFURLSessionSListAppend'
clang-10: error: linker command failed with exit code 1 (use -v to see invocation)
<unknown>:0e[2K
: error: e[2K
link command failed with exit code 1 (use -v to see invocation)
e[2K
[0/1] Linking github-authenticator

I also tried -static-executable but this also failed, so I wanted to go one step at a time. My understanding is, with -static-stdlib the executable should run on a system without Swift installed but with libcurl available. Is this correct? And with -static-executable it could even be deployed to a system with neither Swift nor curl installed, right?

Thanks for any insights!

I was able to reproduce your issue and found a workaround. It links properly for me when importing Foundation in the main project. I‘m not sure why it doesn‘t properly include it from the ArgumentParser dependency. Would you mind filing a bug on https://bugs.swift.org/ ?

I did what you said...but got other issues :
/usr/lib/swift_static/linux/libFoundation.a(FileManager.swift.o):FileManager.swift.o:function $s10Foundation11FileManagerC17_attributesOfItem6atPath26includingPrivateAttributesSDyAA0B12AttributeKeyVypGSS_SbtKF: warning: Using 'getgrgid' in statically linked applications requires at runtime the sha

I have severals issues like this....

The warning: Using '...' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking comes from Glibc which doesnt really like being statically linked into programs.

Certain functionality especially related to user and group functions using PAM (Pluggable Authentication Modules) can require glibc to dynamically open modules which is generally incompatible with statically linking the library in.

The following C program demonstrates the same warning:

$ cat getgrgid.c 
#include <stdio.h>
#include <sys/types.h>
#include <grp.h>

int main() {
        struct group *gr = getgrgid(0);

        if (gr) {
                printf("name: %s\n", gr->gr_name);
        }
        return 0;
}
$ clang -Wall -O -o getgrgid -static getgrgid.c 
/usr/bin/ld: /tmp/getgrgid-897e40.o: in function `main':
getgrgid.c:(.text+0x4): warning: Using 'getgrgid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
$ ./getgrgid 
name: root

It will still work most of the time but can fail if a complex PAM setup exists on the host.

3 Likes

Thanks for taking a look. I actually had already an import for Foundation. But I also overlooked that I was using some Darwin/Glibc function without importing. When I added this import, the -static-stdlib worked well.

2 Likes

I can use linking on Linux for OpenGL lib ?
I build simple system lib opengl (GitHub - TKNgu/SwiftOpenGL) and add glad as code C mixing but when build some error: "cannot find 'glGenBuffers' in scope" when mover to mixing code C everything OK.
Sorry for my english, I can ask in here ?

let package = Package(
    name: "SwiftGUI",
    dependencies: [
        // Dependencies declare other packages that this package depends on.
        .package(url: "https://github.com/TKNgu/SwiftOpenGL.git", from: "1.0.4"),
    ],
    targets: [
        // Targets are the basic building blocks of a package. A target can define a module or a test suite.
        // Targets can depend on other targets in this package, and on products in packages this package depends on.
        .target(
            name: "SwiftGUI",
            dependencies: ["GLAD"]),
        .target(
            name: "GLAD",
            path: "./Sources/glad"),
        .testTarget(
            name: "SwiftGUITests",
            dependencies: ["SwiftGUI"]),
    ]
)

func main() throws {
    var vertexBuffer: GLuint = 0
    glGenBuffers(1, &vertexBuffer)
}

do {
    try main()
} catch {
    print("Error")
}

I could be wrong because I'm not 100% sure I have the full context, but it looks like you will need to import a module that references the OpenGL headers.

For that you will need to look at creating a Swift package with a module.modulemap in the include directory that imports the headers. Then you can import that module.

Yes I have system module for OpenGL:

module SwiftOpenGL [system] {
  header "macros.h"
  module glfw {
    header "/usr/local/include/GLFW/glfw3.h"
    link "/usr/local/lib/libglfw.so"
  }
  module gl {
    header "/usr/include/GL/gl.h"
    header "/usr/include/GL/glext.h"
    header "/usr/include/X11/X.h"
    header "/usr/include/GL/glx.h"
    link "GL"
  }
  export *
}

I can create window (glfw) and call some open gl function: glClearColor(Float(0.2), Float(0.3),Float(0.3), Float(1.0)), glClear(UInt32(GL_COLOR_BUFFER_BIT)).

P/S: Sometime i can build but when rm -rf .build rebuild bug come back

I thing have find bug is GLAD need include befor GLFW/glfw3.h. When build glfw as system module glfw alaways include first show all function glad can't define

#ifdef __gl_h_
#error OpenGL header already included, remove this include, glad already provides it
#endif
#define __gl_h_

One question swift can build call function glColor4d() if define like this ? (I think not it like OpaquePointer)

typedef void (APIENTRYP PFNGLCOLOR4DPROC)(GLdouble red, GLdouble green, GLdouble blue, GLdouble alpha);
GLAPI PFNGLCOLOR4DPROC glad_glColor4d;
#define glColor4d glad_glColor4d

Unfortunately, I'm not familiar enough with the raw OpenGL libraries to be able to help you with these questions. I think it'd be better if you started a new thread with these questions, because I don't think the issues you're having are related to static linking (I can see you're linking to a .so file, which implies dynamic linking)

1 Like

Hey it is me again, I am trying to get a -static-executable running again... Before I go down the rabbit whole and remove file after file, to see where the mistake is, maybe somebody can point me in the right direction.

Long story short, my executable works fine when using -static-stdlib, but when I use -static-executable I get a Segmentation Fault. I will try to setup a more isolated reproducer, but maybe somebody can already give me a hint, on what might be a cause in this scenario... Note, this time I am not using any third-party system dependencies except Foundation. The rest is NIO, AsyncHTTPClient, Soto.

What is the backtrace when you hit the fault? (or what does even crash, swiftc or your program? :-))

Hehe, sorry, my program crashes. Unfortunately it does not print a backtrace (although swift-backtrace is installed).

I ran the program in lldb then I get this suspicious output:

warning: (aarch64) /src/.build/debug/contenttoolcli unable to load swift module "AsyncHTTPClient" (failed to get module "AsyncHTTPClient" from AST context:
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h"
         ^

error: /src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h:22:10: error: 'CNIOBoringSSL.h' file not found
#include "CNIOBoringSSL.h"
         ^

error: could not build C module 'CNIOBoringSSLShims'
)
warning: (aarch64) /src/.build/debug/contenttoolcli unable to load swift module "Backtrace" (failed to get module "AsyncHTTPClient" from AST context:
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h"
         ^

error: /src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h:22:10: error: 'CNIOBoringSSL.h' file not found
#include "CNIOBoringSSL.h"
         ^

error: could not build C module 'CNIOBoringSSLShims'
)
warning: (aarch64) /src/.build/debug/contenttoolcli unable to load swift module "CoreMetrics" (failed to get module "AsyncHTTPClient" from AST context:
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h"
         ^

error: /src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h:22:10: error: 'CNIOBoringSSL.h' file not found
#include "CNIOBoringSSL.h"
         ^

error: could not build C module 'CNIOBoringSSLShims'
)
warning: (aarch64) /src/.build/debug/contenttoolcli unable to load swift module "Crypto" (failed to get module "AsyncHTTPClient" from AST context:
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h"
         ^

error: /src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h:22:10: error: 'CNIOBoringSSL.h' file not found
#include "CNIOBoringSSL.h"
         ^

error: could not build C module 'CNIOBoringSSLShims'
)
warning: (aarch64) /src/.build/debug/contenttoolcli unable to load swift module "INIParser" (failed to get module "AsyncHTTPClient" from AST context:
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h"
         ^

error: /src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h:22:10: error: 'CNIOBoringSSL.h' file not found
#include "CNIOBoringSSL.h"
         ^

error: could not build C module 'CNIOBoringSSLShims'
)
warning: (aarch64) /src/.build/debug/contenttoolcli unable to load swift module "Logging" (failed to get module "AsyncHTTPClient" from AST context:
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h"
         ^

error: /src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h:22:10: error: 'CNIOBoringSSL.h' file not found
#include "CNIOBoringSSL.h"
         ^

error: could not build C module 'CNIOBoringSSLShims'
)
warning: (aarch64) /src/.build/debug/contenttoolcli unable to load swift module "Metrics" (failed to get module "AsyncHTTPClient" from AST context:
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h"
         ^

error: /src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h:22:10: error: 'CNIOBoringSSL.h' file not found
#include "CNIOBoringSSL.h"
         ^

error: could not build C module 'CNIOBoringSSLShims'
)
warning: (aarch64) /src/.build/debug/contenttoolcli unable to load swift module "NIO" (failed to get module "AsyncHTTPClient" from AST context:
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h"
         ^

error: /src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h:22:10: error: 'CNIOBoringSSL.h' file not found
#include "CNIOBoringSSL.h"
         ^

error: could not build C module 'CNIOBoringSSLShims'
)
warning: (aarch64) /src/.build/debug/contenttoolcli unable to load swift module "NIOConcurrencyHelpers" (failed to get module "AsyncHTTPClient" from AST context:
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h"
         ^

error: /src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h:22:10: error: 'CNIOBoringSSL.h' file not found
#include "CNIOBoringSSL.h"
         ^

error: could not build C module 'CNIOBoringSSLShims'
)
warning: (aarch64) /src/.build/debug/contenttoolcli unable to load swift module "NIOFoundationCompat" (failed to get module "AsyncHTTPClient" from AST context:
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h"
         ^

error: /src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h:22:10: error: 'CNIOBoringSSL.h' file not found
#include "CNIOBoringSSL.h"
         ^

error: could not build C module 'CNIOBoringSSLShims'
)
warning: (aarch64) /src/.build/debug/contenttoolcli unable to load swift module "NIOHTTP1" (failed to get module "AsyncHTTPClient" from AST context:
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h"
         ^

error: /src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h:22:10: error: 'CNIOBoringSSL.h' file not found
#include "CNIOBoringSSL.h"
         ^

error: could not build C module 'CNIOBoringSSLShims'
)
warning: (aarch64) /src/.build/debug/contenttoolcli unable to load swift module "NIOHTTPCompression" (failed to get module "AsyncHTTPClient" from AST context:
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h"
         ^

error: /src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h:22:10: error: 'CNIOBoringSSL.h' file not found
#include "CNIOBoringSSL.h"
         ^

error: could not build C module 'CNIOBoringSSLShims'
)
warning: (aarch64) /src/.build/debug/contenttoolcli unable to load swift module "NIOSSL" (failed to get module "AsyncHTTPClient" from AST context:
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h"
         ^

error: /src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h:22:10: error: 'CNIOBoringSSL.h' file not found
#include "CNIOBoringSSL.h"
         ^

error: could not build C module 'CNIOBoringSSLShims'
)
warning: (aarch64) /src/.build/debug/contenttoolcli unable to load swift module "NIOTLS" (failed to get module "AsyncHTTPClient" from AST context:
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h"
         ^

error: /src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h:22:10: error: 'CNIOBoringSSL.h' file not found
#include "CNIOBoringSSL.h"
         ^

error: could not build C module 'CNIOBoringSSLShims'
)
warning: (aarch64) /src/.build/debug/contenttoolcli unable to load swift module "NIOTransportServices" (failed to get module "AsyncHTTPClient" from AST context:
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h"
         ^

error: /src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h:22:10: error: 'CNIOBoringSSL.h' file not found
#include "CNIOBoringSSL.h"
         ^

error: could not build C module 'CNIOBoringSSLShims'
)
warning: (aarch64) /src/.build/debug/contenttoolcli unable to load swift module "SotoCognitoIdentity" (failed to get module "AsyncHTTPClient" from AST context:
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h"
         ^

error: /src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h:22:10: error: 'CNIOBoringSSL.h' file not found
#include "CNIOBoringSSL.h"
         ^

error: could not build C module 'CNIOBoringSSLShims'
)
warning: (aarch64) /src/.build/debug/contenttoolcli unable to load swift module "SotoCognitoIdentityProvider" (failed to get module "AsyncHTTPClient" from AST context:
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h"
         ^

error: /src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h:22:10: error: 'CNIOBoringSSL.h' file not found
#include "CNIOBoringSSL.h"
         ^

error: could not build C module 'CNIOBoringSSLShims'
)
warning: (aarch64) /src/.build/debug/contenttoolcli unable to load swift module "SotoCore" (failed to get module "AsyncHTTPClient" from AST context:
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h"
         ^

error: /src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h:22:10: error: 'CNIOBoringSSL.h' file not found
#include "CNIOBoringSSL.h"
         ^

error: could not build C module 'CNIOBoringSSLShims'
)
warning: (aarch64) /src/.build/debug/contenttoolcli unable to load swift module "SotoCrypto" (failed to get module "AsyncHTTPClient" from AST context:
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h"
         ^

error: /src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h:22:10: error: 'CNIOBoringSSL.h' file not found
#include "CNIOBoringSSL.h"
         ^

error: could not build C module 'CNIOBoringSSLShims'
)
warning: (aarch64) /src/.build/debug/contenttoolcli unable to load swift module "SotoSignerV4" (failed to get module "AsyncHTTPClient" from AST context:
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h"
         ^

error: /src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h:22:10: error: 'CNIOBoringSSL.h' file not found
#include "CNIOBoringSSL.h"
         ^

error: could not build C module 'CNIOBoringSSLShims'
)
warning: (aarch64) /src/.build/debug/contenttoolcli unable to load swift module "SotoXML" (failed to get module "AsyncHTTPClient" from AST context:
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h"
         ^

error: /src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h:22:10: error: 'CNIOBoringSSL.h' file not found
#include "CNIOBoringSSL.h"
         ^

error: could not build C module 'CNIOBoringSSLShims'
)
warning: (aarch64) /src/.build/debug/contenttoolcli unable to load swift module "contenttoolcli" (failed to get module "AsyncHTTPClient" from AST context:
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h"
         ^

error: /src/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSLShims/include/CNIOBoringSSLShims.h:22:10: error: 'CNIOBoringSSL.h' file not found
#include "CNIOBoringSSL.h"
         ^

error: could not build C module 'CNIOBoringSSLShims'
)
Process 311 stopped
* thread #1, name = 'contenttoolcli', stop reason = signal SIGSEGV: invalid address (fault address: 0xe0)
    frame #0: 0x0000fffff7750690
->  0xfffff7750690: ldrh   w2, [x3, x2]
    0xfffff7750694: tbz    w2, #0xd, 0xfffff77506ac
    0xfffff7750698: ldrb   w0, [x1, #0x1]!
    0xfffff775069c: ubfiz  x2, x0, #1, #8
Target 0: (contenttoolcli) stopped.

Package.swift

let package = Package(
    name: "contenttoolcli",
    platforms: [
        .macOS(.v10_15)
    ],
    products: [
        .executable(name: "contenttoolcli", targets: [ "contenttoolcli" ]),
    ],
    dependencies: [
        .package(url: "https://github.com/apple/swift-argument-parser", .upToNextMinor(from: "0.3.0")),
        .package(url: "https://github.com/soto-project/soto.git", from: "5.0.0"),
        .package(url: "https://github.com/apple/swift-crypto", from: "1.1.0"),
        .package(url: "https://github.com/swift-server/swift-backtrace.git", from: "1.0.0"),
        .package(url: "https://github.com/swift-server/async-http-client.git", from: "1.0.0"),
        .package(url: "https://github.com/apple/swift-nio.git", from: "2.19.0"),
    ],
    targets: [
        .target(
            name: "contenttoolcli",
            dependencies: [
                .product(name: "ArgumentParser", package: "swift-argument-parser"),
                .product(name: "SotoCognitoIdentityProvider", package: "soto"),
                .product(name: "SotoCognitoIdentity", package: "soto"),
                .product(name: "Crypto", package:  "swift-crypto"),
                .product(name: "Backtrace", package: "swift-backtrace"),
                .product(name: "AsyncHTTPClient", package: "async-http-client"),
                .product(name: "NIO", package: "swift-nio"),
                .product(name: "NIOHTTP1", package: "swift-nio"),
            ])

Build command (on ARM, maybe I should try on intel)...

docker run -v (pwd):/src -w /src -it --privileged th089/swift:5.3.2 swift build --product contenttoolcli -c debug -Xswiftc -g -Xswiftc -static-executable

hi @t089 is the code something you can share? the LLDB output is ref herring imo - there was an issue (which got fixed in 5.4) with loading certain modulemaps

if you can't share the code, can you confirm you are installing swift-backtrace as the very first thing the program does?

cc @drexin