Can You (Dynamically) Link Swift Libraries on Linux?

this is something that has bedeviled generations and generations of Swift Developers for hundreds of years! SwiftPM and dynamic linking Use a dynamic library in a swift package on Linux Swift shared libraries as plugins Dynamic library support on Linux with library evolution · Issue #5714 · swiftlang/swift-package-manager · GitHub

in this thread, i’m going to try and learn how to dynamically link Swift libraries on Linux, with the eventual goal of teaching SwiftPM how to do this automatically, and upstreaming those changes to swiftlang/swift-package-manager.

before i start, i’d like to thank Ordo One who will be sponsoring me to work on this problem over the next few weeks, for the benefit of server-side Swift and the community at large :D

:face_with_spiral_eyes: why would anyone want to do this???

many of us in the server world have a Big App with a small component that changes very rapidly, much more rapidly than the rest of the app. this small component might be something like a filter, or an algorithm, or a plugin that is being constantly tuned.

we could, for argument’s sake, try and turn this component into data that can be consumed by the Big App, which would probably involve designing a bytecode and an interpreter, and maybe even a whole interpreted domain-specific programming language. but that’s Really Hard. we’d rather just write this thing In Swift, and let Swift code call Swift code.

macOS has XCFrameworks. but on Linux, SwiftPM wants us to recompile the Big App from source and redeploy the Big App every time the filter changes, and we don’t want to do that. what we really want instead is to have the Big App link the filter as a dynamic library, and redeploy the dynamic library as needed.

:moyai: those who came before

Swift speaks C, and C is very good at dynamic linking, so a lot of people doing dynamic linking today from Swift on Linux are doing it with C targets. but we don’t want to write the filter in C, we want to write it in Swift.

we could write the filter in Swift, expose a C interface, and interact with the Swift library as if it were a C library, but that really limits what we can do with the library. we want to call Swift code from Swift code, through interfaces that understand Swift.

to my knowledge, only one person on the planet has figured out how to do this and shared it with the world, and that’s @tiborbodecs over at theswiftdev.com.

:magic_wand: building a .so binary

Tibor wrote his tutorial on macOS, but he said it also works on Linux, so right now, i’m just going to Blindly Copy what Tibor did and see if i can replicate his result.

in Package.swift, i have

// swift-tools-version:6.0
import PackageDescription

let package = Package(
    name: "DynamicLinkingTest",
    products: [
        .library(name: "MyLibrary", type: .dynamic, targets: ["MyLibrary"]),
    ],
    dependencies: [
    ],
    targets: [
        .target(name: "MyLibrary",
            swiftSettings: [
                .unsafeFlags(["-emit-module", "-emit-library"])
            ]),
    ]
)

and inside Sources/MyLibrary/, i have

public
struct Algorithm
{
    public static dynamic func hello() -> String
    {
        "Hello World!"
    }
}

when i do swift build, SwiftPM builds a libMyLibrary.so library for me, and also vomits a copy of the artifact into the project root, for some reason. (that’s probably Intermediates from build plugin generated files written to working directory · Issue #7930 · swiftlang/swift-package-manager · GitHub )

$ swift build
Building for debugging...
[10/10] Linking libMyLibrary.so
Build complete! (0.83s)

the original binary lives in .build/debug, and it’s a real shared Linux library!

$ nm .build/debug/libMyLibrary.so 
0000000000000c50 T $s9MyLibrary9AlgorithmV5helloSSyFZ
0000000000007038 D $s9MyLibrary9AlgorithmV5helloSSyFZTX
0000000000000ce8 R $s9MyLibrary9AlgorithmV5helloSSyFZTx
0000000000000ca0 T $s9MyLibrary9AlgorithmVACycfC
0000000000000cd0 r $s9MyLibrary9AlgorithmVMF
0000000000000cb0 T $s9MyLibrary9AlgorithmVMa
0000000000006d58 d $s9MyLibrary9AlgorithmVMf
0000000000000d24 R $s9MyLibrary9AlgorithmVMn
0000000000006d68 D $s9MyLibrary9AlgorithmVN
0000000000000d0c r $s9MyLibraryMXM
                 U $sSS21_builtinStringLiteral17utf8CodeUnitCount7isASCIISSBp_BwBi1_tcfC
                 U $sytWV
0000000000006d78 d _DYNAMIC
0000000000006fe8 d _GLOBAL_OFFSET_TABLE_
                 w _ITM_deregisterTMCloneTable
                 w _ITM_registerTMCloneTable
0000000000007030 d _ZL14__backtraceRef
0000000000000a30 t _ZL23swift_image_constructorv
0000000000007050 b _ZN12_GLOBAL__N_18sectionsE
0000000000005220 r __FRAME_END__
0000000000000d64 r __Swift_AST
0000000000007048 d __TMC_END__
0000000000007048 d __TMC_LIST__
0000000000007048 D __bss_start
                 w __cxa_finalize
00000000000009e0 t __do_global_dtors_aux
0000000000006d40 d __do_global_dtors_aux_fini_array_entry
0000000000007028 d __dso_handle
0000000000006d48 d __frame_dummy_init_array_entry
                 w __gmon_start__
0000000000000ce6 r __start_swift5_accessible_functions
0000000000000ce0 r __start_swift5_assocty
0000000000000ce0 r __start_swift5_builtin
0000000000000ce0 r __start_swift5_capture
0000000000000cd0 r __start_swift5_fieldmd
0000000000000ce6 r __start_swift5_mpenum
0000000000000cc9 r __start_swift5_protocol_conformances
0000000000000cc9 r __start_swift5_protocols
0000000000000ce0 r __start_swift5_reflstr
0000000000000ce6 r __start_swift5_replac2
0000000000000ce6 r __start_swift5_replace
0000000000000ce6 r __start_swift5_runtime_attributes
0000000000000ce6 r __start_swift5_tests
0000000000000ccc r __start_swift5_type_metadata
0000000000000ce0 r __start_swift5_typeref
0000000000000ce6 r __stop_swift5_accessible_functions
0000000000000ce0 r __stop_swift5_assocty
0000000000000ce0 r __stop_swift5_builtin
0000000000000ce0 r __stop_swift5_capture
0000000000000ce0 r __stop_swift5_fieldmd
0000000000000ce6 r __stop_swift5_mpenum
0000000000000cc9 r __stop_swift5_protocol_conformances
0000000000000cc9 r __stop_swift5_protocols
0000000000000ce0 r __stop_swift5_reflstr
0000000000000ce6 r __stop_swift5_replac2
0000000000000ce6 r __stop_swift5_replace
0000000000000ce6 r __stop_swift5_runtime_attributes
0000000000000ce6 r __stop_swift5_tests
0000000000000cd0 r __stop_swift5_type_metadata
0000000000000ce6 r __stop_swift5_typeref
0000000000000d40 r __swift_reflection_version
0000000000007048 D _edata
0000000000007160 D _end
0000000000000cbc t _fini
00000000000008e8 t _init
                 U _swift_backtrace_isThunkFunction
0000000000007048 b completed.0
0000000000000970 t deregister_tm_clones
0000000000000a20 t frame_dummy
00000000000009a0 t register_tm_clones
                 U swift_addNewDSOImage
                 U swift_getFunctionReplacement
0000000000000ce0 r symbolic _____ 9MyLibrary9AlgorithmV

:mechanical_leg: using an .so binary

i create an executable target in the same SwiftPM project called Client.

    products: [
        .library(name: "MyLibrary", type: .dynamic, targets: ["MyLibrary"]),
+       .executable(name: "Client", targets: ["Client"]),
    ],
    dependencies: [
    ],
    targets: [
+       .executableTarget(name: "Client",
+           dependencies: [
+               .target(name: "MyLibrary"),
+           ]),

        .target(name: "MyLibrary",
            swiftSettings: [
                .unsafeFlags(["-emit-module", "-emit-library"])
            ]),
    ]
import MyLibrary

print(MyLibrary.Algorithm.hello())

the problem is, when we build this, SwiftPM just links it statically anyway. (it will do this even if the target is part of a dynamic product from another package.)

we know this because you can delete the libMyLibrary.so, and the Client executable will still work.

$ rm .build/debug/libMyLibrary.so 
$ .build/debug/Client
Hello World!

$ ldd .build/debug/Client          
        linux-vdso.so.1 (0x00007327d7c5d000)
        libswiftSwiftOnoneSupport.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswiftSwiftOnoneSupport.so (0x00007327d7c08000)
        libswiftCore.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswiftCore.so (0x00007327d7400000)
        libswift_Concurrency.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswift_Concurrency.so (0x00007327d7b71000)
        libswift_StringProcessing.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswift_StringProcessing.so (0x00007327d7325000)
        libswift_RegexParser.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswift_RegexParser.so (0x00007327d7211000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007327d6e00000)
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007327d6a00000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007327d7128000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007327d7b3d000)
        /lib64/ld-linux-x86-64.so.2 (0x00007327d7c5f000)
        libdispatch.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libdispatch.so (0x00007327d7add000)
        libswiftGlibc.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswiftGlibc.so (0x00007327d7115000)
        libBlocksRuntime.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libBlocksRuntime.so (0x00007327d7ad7000)

:exploding_head: linking an .so binary manually

it seems we’ve got to delete the dependency edge from Client to MyLibrary to stop static linking from taking place. funny enough, SwiftPM can still compile code that depends on other targets that are not actually dependencies, it just fails at link time. i always considered this a bug, but i guess for this experiment, it’s also a handy feature.

if we do a straight build, it will fail because Swift still needs the module interfaces for MyLibrary, and SwiftPM (sometimes!) builds the targets in the wrong order, since it is missing the dependency graph information.

$ swift build 
Building for debugging...
error: emit-module command failed with exit code 1 (use -v to see invocation)
/swift/dynamic-test/Sources/Client/Client.swift:1:8: error: no such module 'MyLibrary'
1 | import MyLibrary
  |        `- error: no such module 'MyLibrary'
2 | 
3 | print(MyLibrary.Algorithm.hello())
/swift/dynamic-test/Sources/Client/Client.swift:1:8: error: no such module 'MyLibrary'
1 | import MyLibrary
  |        `- error: no such module 'MyLibrary'
2 | 
3 | print(MyLibrary.Algorithm.hello())
[12/16] Linking libMyLibrary.so

so we’ve got to build the products individually, and it fails, which is exactly what we were expecting.

$ swift build --product MyLibrary
Building for debugging...
[2/2] Linking libMyLibrary.so
Build of product 'MyLibrary' complete! (0.46s)

$ swift build --product Client   
Building for debugging...
error: link command failed with exit code 1 (use -v to see invocation)
/swift/dynamic-test/Sources/Client/Client.swift:3: error: undefined reference to '$s9MyLibrary9AlgorithmV5helloSSyFZ'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
[6/7] Linking Client

$ swift demangle s9MyLibrary9AlgorithmV5helloSSyFZ 
$s9MyLibrary9AlgorithmV5helloSSyFZ ---> static MyLibrary.Algorithm.hello() -> Swift.String

we need to tell the linker to link MyLibrary manually, and the way to do that is probably with -Xlinker flags. the .so lives in .build/debug, so i pass that as -L.build/debug, and the file is named libMyLibrary.so, so i pass that as -lMyLibrary.

$ swift build --product Client -Xlinker -L.build/debug -Xlinker -lMyLibrary
Building for debugging...
[2/2] Linking Client
Build of product 'Client' complete! (0.44s)

let’s run it!

$ .build/debug/Client
Hello World!

$ ldd .build/debug/Client
        linux-vdso.so.1 (0x00007450cf5a5000)
        libswiftSwiftOnoneSupport.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswiftSwiftOnoneSupport.so (0x00007450cf554000)
        libswiftCore.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswiftCore.so (0x00007450cee00000)
        libswift_Concurrency.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswift_Concurrency.so (0x00007450ced69000)
        libswift_StringProcessing.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswift_StringProcessing.so (0x00007450cec8e000)
        libswift_RegexParser.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswift_RegexParser.so (0x00007450ceb7a000)
***     libMyLibrary.so => /swift/dynamic-test/.build/debug/libMyLibrary.so (0x00007450cf54a000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007450ce800000)
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007450ce400000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007450cea91000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007450cf518000)
        /lib64/ld-linux-x86-64.so.2 (0x00007450cf5a7000)
        libdispatch.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libdispatch.so (0x00007450cea31000)
        libswiftGlibc.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswiftGlibc.so (0x00007450cf503000)
        libBlocksRuntime.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libBlocksRuntime.so (0x00007450cf4ff000)

and unlike last time, it really is dependent on the dynamic library, because if i delete it, the app fails!

$ rm .build/debug/libMyLibrary.so 
$ .build/debug/Client
.build/debug/Client: error while loading shared libraries: libMyLibrary.so: cannot open shared object file: No such file or directory

:woman_zombie: updating the library

now let’s try updating the code inside the library and see if we can change its behavior without recompiling Client.

SwiftPM on VSCode is notorious for doing Weird Stuff In The Background, so let’s first take a screenshot of our build artifacts to make sure it’s not trying to be Too Helpful and rebuilding Client behind our backs.

$ ls -l .build/x86_64-unknown-linux-gnu/debug/
total 80
-rwxr-xr-x 1 ubuntu ubuntu 35408 Jan 24 22:47 Client
drwxr-xr-x 2 ubuntu ubuntu  4096 Jan 24 22:52 Client.build
drwxr-xr-x 2 ubuntu ubuntu  4096 Jan 24 22:45 Client.product
drwxr-x--- 5 ubuntu ubuntu  4096 Jan 24 22:52 ModuleCache
drwxr-xr-x 2 ubuntu ubuntu  4096 Jan 24 22:45 Modules
drwxr-xr-x 2 ubuntu ubuntu  4096 Jan 24 22:52 MyLibrary.build
drwxr-xr-x 2 ubuntu ubuntu  4096 Jan 24 22:44 MyLibrary.product
-rw-r--r-- 1 ubuntu ubuntu 10899 Jan 24 22:47 description.json
drwxr-x--- 3 ubuntu ubuntu  4096 Jan 24 22:43 index
-rw-r--r-- 1 ubuntu ubuntu    75 Jan 24 22:43 swift-version--75355AAB4E86B17C.txt

let’s change the code

    public static dynamic func hello() -> String
    {
+       "Hi Barbie!"
    }

rebuild MyLibrary

$ swift build --product MyLibrary
Building for debugging...
[9/9] Linking libMyLibrary.so
Build of product 'MyLibrary' complete! (0.80s)

run Client

$ .build/debug/Client
Hi Barbie!

make sure Client has not changed

$ ls -l .build/x86_64-unknown-linux-gnu/debug/
total 116
-rwxr-xr-x 1 ubuntu ubuntu 35408 Jan 24 22:47 Client

it works!

:thinking: do we really need dynamic?

i started off by cargo culting everything Tibor did in an article from 4 years ago, so naturally i wonder which of these steps are actually necessary today on Swift 6.0.3.

the dynamic keyword is related to Objective C interop, and there is no Objective C runtime on Linux, so that’s a prime target for elimination.

as it turns out, dynamic does nothing today. everything we did before works without dynamic.

:thinking: do we really need -emit-module, -emit-library?

no. everything we did before also works without those two flags. SwiftPM now adds those flags automatically, which can be seen in swift build -v --product MyLibrary | grep emit.

:cold_sweat: is this safe???

the change that we tested was pretty minor, we were just changing the value of a returned string. no interfaces actually changed. how do we know if our binary libraries are actually compatible with the Big App?

well, for a really trivial example, we can change the signature of Algorithm.hello, and the library will just fail to load at runtime.

+   public static func hello() -> Int
    {
+       0
    }
$ .build/debug/Client
.build/debug/Client: symbol lookup error: .build/debug/Client: undefined symbol: $s9MyLibrary9AlgorithmV5helloSSyFZ

this should Surprise No One; return types are part of the symbol mangling ABI.

but this can get Way Worse! consider an enum like this one:

public
enum SupportedCases
{
    case a

    public
    static var current:Self { .a }
}

we can use it like this:

import MyLibrary
switch MyLibrary.SupportedCases.current
{
case .a:
    print("a")
}

and run it like this:

$ .build/debug/Client
a

but now if we add a new case, return that new case from current, and recompile just the library, we get the completely wrong behavior! probably, when the compiler built Client, it assumed SupportedCases had only one case, and so it optimized the Client code to always print a.

public
enum SupportedCases
{
    case a
    case b

    public
    static var current:Self { .b }
}
$ .build/debug/Client
a

:monkey: enabling Library Evolution

as someone who spends a lot of time optimizing multi-target projects, i am usually trying to turn off resilience. but in this situation, we want more resilience, not less. the flag to enable this is -enable-library-evolution.

$ swift build --product MyLibrary -Xswiftc -enable-library-evolution

this will cause the compiler to complain if you write code in Client that assumes things about MyLibrary that are supposed to be hidden behind a resilience barrier.

switch MyLibrary.SupportedCases.current
{
case .a:
    print("a")
case .b:
    print("b")

+ @unknown default:
+     print("unknown")
}
$ swift build --product Client -Xlinker -L.build/debug -Xlinker -lMyLibrary
Building for debugging...
[8/8] Linking Client
Build of product 'Client' complete! (0.77s)

$ .build/debug/Client
b

:hocho: changing a resilient enum

let’s add a new case to SupportedCases.

public
enum SupportedCases
{
    case a
    case b
+   case c

    public
+   static var current:Self { .c }
}
$ swift build --product MyLibrary -Xswiftc -enable-library-evolution
Building for debugging...
[7/7] Linking libMyLibrary.so
Build of product 'MyLibrary' complete! (0.80s)

$ .build/debug/Client
unknown

it works!

:running_woman: can we make SwiftPM do this for us?

so i’ve got the incantations down, but this workflow just Really Sucks, and i can’t see it scaling up to a project of realistic size. this is stuff our build system is supposed to manage for us!

of all the flags we’re using, -enable-library-evolution is probably the easiest to integrate. i can just add that to Package.swift and SwiftPM will forward it to swiftc instead of me having to retype it every time i recompile MyLibrary.

        .target(name: "MyLibrary",
            swiftSettings: [
+               .unsafeFlags(["-enable-library-evolution"]),
            ]),

but with Client, we want SwiftPM to understand the dependency relationship between it and MyLibrary without actually statically linking MyLibrary into the final executable.

let’s find out what flags are different between the two ways of building Client with the help of the -v option.

$ swift build -v --product Client -Xlinker -L.build/debug -Xlinker -lMyLibrary
$ swift build -v --product Client 

the meaningful differences, i think, are that these arguments are present in swift-autolink-extract, clang, and ld when linking statically

.build/x86_64-unknown-linux-gnu/debug/MyLibrary.build/Algorithm.swift.o 
.build/x86_64-unknown-linux-gnu/debug/MyLibrary.build/MyLibrary.swiftmodule.o

and our custom -L .build/debug, -lMyLibrary are missing, which suggests that SwiftPM is not doing anything fancy here behind the scenes, it is literally just telling clang to add those two extra .o files to the build.

while flags like -enable-library-evolution can be added fairly easily to SwiftPM’s generated commands, there doesn’t seem to be a way to suppress them. i expect that i will have to modify the source code of SwiftPM to get it to do this automatically.

30 Likes

Super interesting reading, thank you for having put this together. I did some work on that too based on the work from Tibor.

The experimental Swift plugin for SAM DSL does that. It locates a DSL file (similar to Package.swift, but for SAM), compiles it, links it and runs it. It links to a shared library which is also compiled dynamically.

2 Likes

Great investigation and write up. Thanks for sharing! I particularly appreciated the ostensibly tangential parts about library evolution and other compiler flags - it’s still not easy to find good clear information about what these are for and when to use them.

It’s a shame that we need to use „unsafe“ swift flags to get part of the way there - IMO there is still way too much that falls under the unsafe umbrella that is actually just fine.

2 Likes

i have a proof-of-concept fork of SwiftPM that gets the build system to do all of this automatically, given an .artifactbundle directory containing the dynamic library.

:woman_cook: preparing a dynamic library

build a shared library with -enable-library-evolution, take its .so and its .swiftmodule files, and stuff them in an Artifactbundle resembling this:

📂 test.artifactbundle
    📂 MyLibrary
        ⚙️ libMyLibrary.so
        ⚙️ MyLibrary.swiftmodule
    📝 info.json
{
    "schemaVersion": "1.0",
    "artifacts": {
        "MyLibrary": {
            "type": "library",
            "version": "0.0.0",
            "variants": [{ "path": "MyLibrary" }]
        }
    }
}

:woman_astronaut: using a dynamic library

declare a binaryTarget with the path pointing to the Artifactbundle, and make it a dependency of your executable target.

let package:Package = .init(
    name: "DynamicLinkingTest",
    products: [
        // .library(name: "MyLibrary", type: .dynamic, targets: ["MyLibrary"]),
        .executable(name: "Client", targets: ["Client"]),
    ],
    dependencies: [
    ],
    targets: [
        .binaryTarget(
            name: "MyLibrary",
            path: "test.artifactbundle"),

        .executableTarget(name: "Client",
            dependencies: [
                .target(name: "MyLibrary"),
            ]),

        // .target(name: "MyLibrary",
        //     swiftSettings: [
        //         .unsafeFlags(["-enable-library-evolution"]),
        //     ]),
    ]
)

build the executable

swift-build --product Client

by default, it will be linked but not loaded, since in a “realistic” deployment, there will be no .build/debug directory.

$ ldd .build/debug/Client
        linux-vdso.so.1 (0x00007aca257ed000)
        libswiftSwiftOnoneSupport.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswiftSwiftOnoneSupport.so (0x00007aca2579c000)
        libswiftCore.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswiftCore.so (0x00007aca25000000)
        libswift_Concurrency.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswift_Concurrency.so (0x00007aca25705000)
        libswift_StringProcessing.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswift_StringProcessing.so (0x00007aca24f25000)
        libswift_RegexParser.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswift_RegexParser.so (0x00007aca24e11000)
***     libMyLibrary.so => not found
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007aca24a00000)
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007aca24600000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007aca24d28000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007aca256d1000)
        /lib64/ld-linux-x86-64.so.2 (0x00007aca257ef000)
        libdispatch.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libdispatch.so (0x00007aca24cc8000)
        libswiftGlibc.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswiftGlibc.so (0x00007aca24cb5000)
        libBlocksRuntime.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libBlocksRuntime.so (0x00007aca256cb000)

so for testing purposes tell the linker where the library is on your host

$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PWD/test.artifactbundle/MyLibrary
$ ldd .build/debug/Client
        linux-vdso.so.1 (0x000078a070847000)
        libswiftSwiftOnoneSupport.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswiftSwiftOnoneSupport.so (0x000078a0707f6000)
        libswiftCore.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswiftCore.so (0x000078a070000000)
        libswift_Concurrency.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswift_Concurrency.so (0x000078a07075f000)
        libswift_StringProcessing.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswift_StringProcessing.so (0x000078a06ff25000)
        libswift_RegexParser.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswift_RegexParser.so (0x000078a06fe11000)
***     libMyLibrary.so => /swift/dynamic-test/test.artifactbundle/MyLibrary/libMyLibrary.so (0x000078a070754000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x000078a06fa00000)
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x000078a06f600000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x000078a06fd28000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x000078a070722000)
        /lib64/ld-linux-x86-64.so.2 (0x000078a070849000)
        libdispatch.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libdispatch.so (0x000078a06fcc8000)
        libswiftGlibc.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswiftGlibc.so (0x000078a07070d000)
        libBlocksRuntime.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libBlocksRuntime.so (0x000078a070709000)
3 Likes

Thanks for doing all this investigation and work @taylorswift! I actually use Swift professionally on Linux, almost exclusively compiling dynamic libraries, but I do so with CMake where it’s pretty trivial to do. I too have noticed that SPM isn’t really happy about anything other than recompiling all targets and dependencies, so it’s great to see it becoming more robust.

1 Like

I am also interested in dynamic linking since we have a small Linux target that we deploy our apps to. And it would be nice to not have to statically link all swiftpm dependencies for each app that runs on the target.

I'll add my thanks to the others for this deep dive (and for sharing). I also had tried at least some of this so it is nice to see someone else's results.

I'm wondering if you happened to have looked into something that stymied me while looking into this: When trying to compile a dynamic library that has a dependency, I could get SwiftPM to output a .so for the top-level library but not the dependency. Do you know of a way to convince SwiftPM to traverse the dependency tree for dynamic libraries?