this is something that has bedeviled generations and generations of Swift Developers for hundreds of years! SwiftPM and dynamic linking Use a dynamic library in a swift package on Linux Swift shared libraries as plugins Dynamic library support on Linux with library evolution · Issue #5714 · swiftlang/swift-package-manager · GitHub
in this thread, i’m going to try and learn how to dynamically link Swift libraries on Linux, with the eventual goal of teaching SwiftPM how to do this automatically, and upstreaming those changes to swiftlang/swift-package-manager.
before i start, i’d like to thank Ordo One who will be sponsoring me to work on this problem over the next few weeks, for the benefit of server-side Swift and the community at large :D
why would anyone want to do this???
many of us in the server world have a Big App with a small component that changes very rapidly, much more rapidly than the rest of the app. this small component might be something like a filter, or an algorithm, or a plugin that is being constantly tuned.
we could, for argument’s sake, try and turn this component into data that can be consumed by the Big App, which would probably involve designing a bytecode and an interpreter, and maybe even a whole interpreted domain-specific programming language. but that’s Really Hard. we’d rather just write this thing In Swift, and let Swift code call Swift code.
macOS has XCFrameworks. but on Linux, SwiftPM wants us to recompile the Big App from source and redeploy the Big App every time the filter changes, and we don’t want to do that. what we really want instead is to have the Big App link the filter as a dynamic library, and redeploy the dynamic library as needed.
those who came before
Swift speaks C, and C is very good at dynamic linking, so a lot of people doing dynamic linking today from Swift on Linux are doing it with C targets. but we don’t want to write the filter in C, we want to write it in Swift.
we could write the filter in Swift, expose a C interface, and interact with the Swift library as if it were a C library, but that really limits what we can do with the library. we want to call Swift code from Swift code, through interfaces that understand Swift.
to my knowledge, only one person on the planet has figured out how to do this and shared it with the world, and that’s @tiborbodecs over at theswiftdev.com.
building a .so
binary
Tibor wrote his tutorial on macOS, but he said it also works on Linux, so right now, i’m just going to Blindly Copy what Tibor did and see if i can replicate his result.
in Package.swift, i have
// swift-tools-version:6.0
import PackageDescription
let package = Package(
name: "DynamicLinkingTest",
products: [
.library(name: "MyLibrary", type: .dynamic, targets: ["MyLibrary"]),
],
dependencies: [
],
targets: [
.target(name: "MyLibrary",
swiftSettings: [
.unsafeFlags(["-emit-module", "-emit-library"])
]),
]
)
and inside Sources/MyLibrary/
, i have
public
struct Algorithm
{
public static dynamic func hello() -> String
{
"Hello World!"
}
}
when i do swift build
, SwiftPM builds a libMyLibrary.so
library for me, and also vomits a copy of the artifact into the project root, for some reason. (that’s probably Intermediates from build plugin generated files written to working directory · Issue #7930 · swiftlang/swift-package-manager · GitHub )
$ swift build
Building for debugging...
[10/10] Linking libMyLibrary.so
Build complete! (0.83s)
the original binary lives in .build/debug
, and it’s a real shared Linux library!
$ nm .build/debug/libMyLibrary.so
0000000000000c50 T $s9MyLibrary9AlgorithmV5helloSSyFZ
0000000000007038 D $s9MyLibrary9AlgorithmV5helloSSyFZTX
0000000000000ce8 R $s9MyLibrary9AlgorithmV5helloSSyFZTx
0000000000000ca0 T $s9MyLibrary9AlgorithmVACycfC
0000000000000cd0 r $s9MyLibrary9AlgorithmVMF
0000000000000cb0 T $s9MyLibrary9AlgorithmVMa
0000000000006d58 d $s9MyLibrary9AlgorithmVMf
0000000000000d24 R $s9MyLibrary9AlgorithmVMn
0000000000006d68 D $s9MyLibrary9AlgorithmVN
0000000000000d0c r $s9MyLibraryMXM
U $sSS21_builtinStringLiteral17utf8CodeUnitCount7isASCIISSBp_BwBi1_tcfC
U $sytWV
0000000000006d78 d _DYNAMIC
0000000000006fe8 d _GLOBAL_OFFSET_TABLE_
w _ITM_deregisterTMCloneTable
w _ITM_registerTMCloneTable
0000000000007030 d _ZL14__backtraceRef
0000000000000a30 t _ZL23swift_image_constructorv
0000000000007050 b _ZN12_GLOBAL__N_18sectionsE
0000000000005220 r __FRAME_END__
0000000000000d64 r __Swift_AST
0000000000007048 d __TMC_END__
0000000000007048 d __TMC_LIST__
0000000000007048 D __bss_start
w __cxa_finalize
00000000000009e0 t __do_global_dtors_aux
0000000000006d40 d __do_global_dtors_aux_fini_array_entry
0000000000007028 d __dso_handle
0000000000006d48 d __frame_dummy_init_array_entry
w __gmon_start__
0000000000000ce6 r __start_swift5_accessible_functions
0000000000000ce0 r __start_swift5_assocty
0000000000000ce0 r __start_swift5_builtin
0000000000000ce0 r __start_swift5_capture
0000000000000cd0 r __start_swift5_fieldmd
0000000000000ce6 r __start_swift5_mpenum
0000000000000cc9 r __start_swift5_protocol_conformances
0000000000000cc9 r __start_swift5_protocols
0000000000000ce0 r __start_swift5_reflstr
0000000000000ce6 r __start_swift5_replac2
0000000000000ce6 r __start_swift5_replace
0000000000000ce6 r __start_swift5_runtime_attributes
0000000000000ce6 r __start_swift5_tests
0000000000000ccc r __start_swift5_type_metadata
0000000000000ce0 r __start_swift5_typeref
0000000000000ce6 r __stop_swift5_accessible_functions
0000000000000ce0 r __stop_swift5_assocty
0000000000000ce0 r __stop_swift5_builtin
0000000000000ce0 r __stop_swift5_capture
0000000000000ce0 r __stop_swift5_fieldmd
0000000000000ce6 r __stop_swift5_mpenum
0000000000000cc9 r __stop_swift5_protocol_conformances
0000000000000cc9 r __stop_swift5_protocols
0000000000000ce0 r __stop_swift5_reflstr
0000000000000ce6 r __stop_swift5_replac2
0000000000000ce6 r __stop_swift5_replace
0000000000000ce6 r __stop_swift5_runtime_attributes
0000000000000ce6 r __stop_swift5_tests
0000000000000cd0 r __stop_swift5_type_metadata
0000000000000ce6 r __stop_swift5_typeref
0000000000000d40 r __swift_reflection_version
0000000000007048 D _edata
0000000000007160 D _end
0000000000000cbc t _fini
00000000000008e8 t _init
U _swift_backtrace_isThunkFunction
0000000000007048 b completed.0
0000000000000970 t deregister_tm_clones
0000000000000a20 t frame_dummy
00000000000009a0 t register_tm_clones
U swift_addNewDSOImage
U swift_getFunctionReplacement
0000000000000ce0 r symbolic _____ 9MyLibrary9AlgorithmV
using an .so
binary
i create an executable target in the same SwiftPM project called Client
.
products: [
.library(name: "MyLibrary", type: .dynamic, targets: ["MyLibrary"]),
+ .executable(name: "Client", targets: ["Client"]),
],
dependencies: [
],
targets: [
+ .executableTarget(name: "Client",
+ dependencies: [
+ .target(name: "MyLibrary"),
+ ]),
.target(name: "MyLibrary",
swiftSettings: [
.unsafeFlags(["-emit-module", "-emit-library"])
]),
]
import MyLibrary
print(MyLibrary.Algorithm.hello())
the problem is, when we build this, SwiftPM just links it statically anyway. (it will do this even if the target is part of a dynamic
product from another package.)
we know this because you can delete the libMyLibrary.so
, and the Client
executable will still work.
$ rm .build/debug/libMyLibrary.so
$ .build/debug/Client
Hello World!
$ ldd .build/debug/Client
linux-vdso.so.1 (0x00007327d7c5d000)
libswiftSwiftOnoneSupport.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswiftSwiftOnoneSupport.so (0x00007327d7c08000)
libswiftCore.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswiftCore.so (0x00007327d7400000)
libswift_Concurrency.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswift_Concurrency.so (0x00007327d7b71000)
libswift_StringProcessing.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswift_StringProcessing.so (0x00007327d7325000)
libswift_RegexParser.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswift_RegexParser.so (0x00007327d7211000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007327d6e00000)
libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007327d6a00000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007327d7128000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007327d7b3d000)
/lib64/ld-linux-x86-64.so.2 (0x00007327d7c5f000)
libdispatch.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libdispatch.so (0x00007327d7add000)
libswiftGlibc.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswiftGlibc.so (0x00007327d7115000)
libBlocksRuntime.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libBlocksRuntime.so (0x00007327d7ad7000)
linking an .so
binary manually
it seems we’ve got to delete the dependency edge from Client
to MyLibrary
to stop static linking from taking place. funny enough, SwiftPM can still compile code that depends on other targets that are not actually dependencies, it just fails at link time. i always considered this a bug, but i guess for this experiment, it’s also a handy feature.
if we do a straight build, it will fail because Swift still needs the module interfaces for MyLibrary
, and SwiftPM (sometimes!) builds the targets in the wrong order, since it is missing the dependency graph information.
$ swift build
Building for debugging...
error: emit-module command failed with exit code 1 (use -v to see invocation)
/swift/dynamic-test/Sources/Client/Client.swift:1:8: error: no such module 'MyLibrary'
1 | import MyLibrary
| `- error: no such module 'MyLibrary'
2 |
3 | print(MyLibrary.Algorithm.hello())
/swift/dynamic-test/Sources/Client/Client.swift:1:8: error: no such module 'MyLibrary'
1 | import MyLibrary
| `- error: no such module 'MyLibrary'
2 |
3 | print(MyLibrary.Algorithm.hello())
[12/16] Linking libMyLibrary.so
so we’ve got to build the products individually, and it fails, which is exactly what we were expecting.
$ swift build --product MyLibrary
Building for debugging...
[2/2] Linking libMyLibrary.so
Build of product 'MyLibrary' complete! (0.46s)
$ swift build --product Client
Building for debugging...
error: link command failed with exit code 1 (use -v to see invocation)
/swift/dynamic-test/Sources/Client/Client.swift:3: error: undefined reference to '$s9MyLibrary9AlgorithmV5helloSSyFZ'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
[6/7] Linking Client
$ swift demangle s9MyLibrary9AlgorithmV5helloSSyFZ
$s9MyLibrary9AlgorithmV5helloSSyFZ ---> static MyLibrary.Algorithm.hello() -> Swift.String
we need to tell the linker to link MyLibrary
manually, and the way to do that is probably with -Xlinker
flags. the .so
lives in .build/debug
, so i pass that as -L.build/debug
, and the file is named libMyLibrary.so
, so i pass that as -lMyLibrary
.
$ swift build --product Client -Xlinker -L.build/debug -Xlinker -lMyLibrary
Building for debugging...
[2/2] Linking Client
Build of product 'Client' complete! (0.44s)
let’s run it!
$ .build/debug/Client
Hello World!
$ ldd .build/debug/Client
linux-vdso.so.1 (0x00007450cf5a5000)
libswiftSwiftOnoneSupport.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswiftSwiftOnoneSupport.so (0x00007450cf554000)
libswiftCore.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswiftCore.so (0x00007450cee00000)
libswift_Concurrency.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswift_Concurrency.so (0x00007450ced69000)
libswift_StringProcessing.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswift_StringProcessing.so (0x00007450cec8e000)
libswift_RegexParser.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswift_RegexParser.so (0x00007450ceb7a000)
*** libMyLibrary.so => /swift/dynamic-test/.build/debug/libMyLibrary.so (0x00007450cf54a000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007450ce800000)
libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007450ce400000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007450cea91000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007450cf518000)
/lib64/ld-linux-x86-64.so.2 (0x00007450cf5a7000)
libdispatch.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libdispatch.so (0x00007450cea31000)
libswiftGlibc.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libswiftGlibc.so (0x00007450cf503000)
libBlocksRuntime.so => /home/ubuntu/x86_64/6.0.3/usr/lib/swift/linux/libBlocksRuntime.so (0x00007450cf4ff000)
and unlike last time, it really is dependent on the dynamic library, because if i delete it, the app fails!
$ rm .build/debug/libMyLibrary.so
$ .build/debug/Client
.build/debug/Client: error while loading shared libraries: libMyLibrary.so: cannot open shared object file: No such file or directory
updating the library
now let’s try updating the code inside the library and see if we can change its behavior without recompiling Client
.
SwiftPM on VSCode is notorious for doing Weird Stuff In The Background, so let’s first take a screenshot of our build artifacts to make sure it’s not trying to be Too Helpful and rebuilding Client
behind our backs.
$ ls -l .build/x86_64-unknown-linux-gnu/debug/
total 80
-rwxr-xr-x 1 ubuntu ubuntu 35408 Jan 24 22:47 Client
drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 24 22:52 Client.build
drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 24 22:45 Client.product
drwxr-x--- 5 ubuntu ubuntu 4096 Jan 24 22:52 ModuleCache
drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 24 22:45 Modules
drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 24 22:52 MyLibrary.build
drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 24 22:44 MyLibrary.product
-rw-r--r-- 1 ubuntu ubuntu 10899 Jan 24 22:47 description.json
drwxr-x--- 3 ubuntu ubuntu 4096 Jan 24 22:43 index
-rw-r--r-- 1 ubuntu ubuntu 75 Jan 24 22:43 swift-version--75355AAB4E86B17C.txt
let’s change the code
public static dynamic func hello() -> String
{
+ "Hi Barbie!"
}
rebuild MyLibrary
$ swift build --product MyLibrary
Building for debugging...
[9/9] Linking libMyLibrary.so
Build of product 'MyLibrary' complete! (0.80s)
run Client
$ .build/debug/Client
Hi Barbie!
make sure Client
has not changed
$ ls -l .build/x86_64-unknown-linux-gnu/debug/
total 116
-rwxr-xr-x 1 ubuntu ubuntu 35408 Jan 24 22:47 Client
it works!
do we really need dynamic
?
i started off by cargo culting everything Tibor did in an article from 4 years ago, so naturally i wonder which of these steps are actually necessary today on Swift 6.0.3.
the dynamic
keyword is related to Objective C interop, and there is no Objective C runtime on Linux, so that’s a prime target for elimination.
as it turns out, dynamic
does nothing today. everything we did before works without dynamic
.
do we really need -emit-module
, -emit-library
?
no. everything we did before also works without those two flags. SwiftPM now adds those flags automatically, which can be seen in swift build -v --product MyLibrary | grep emit
.
is this safe???
the change that we tested was pretty minor, we were just changing the value of a returned string. no interfaces actually changed. how do we know if our binary libraries are actually compatible with the Big App?
well, for a really trivial example, we can change the signature of Algorithm.hello
, and the library will just fail to load at runtime.
+ public static func hello() -> Int
{
+ 0
}
$ .build/debug/Client
.build/debug/Client: symbol lookup error: .build/debug/Client: undefined symbol: $s9MyLibrary9AlgorithmV5helloSSyFZ
this should Surprise No One; return types are part of the symbol mangling ABI.
but this can get Way Worse! consider an enum
like this one:
public
enum SupportedCases
{
case a
public
static var current:Self { .a }
}
we can use it like this:
import MyLibrary
switch MyLibrary.SupportedCases.current
{
case .a:
print("a")
}
and run it like this:
$ .build/debug/Client
a
but now if we add a new case, return that new case from current
, and recompile just the library, we get the completely wrong behavior! probably, when the compiler built Client
, it assumed SupportedCases
had only one case, and so it optimized the Client
code to always print a
.
public
enum SupportedCases
{
case a
case b
public
static var current:Self { .b }
}
$ .build/debug/Client
a
enabling Library Evolution
as someone who spends a lot of time optimizing multi-target projects, i am usually trying to turn off resilience. but in this situation, we want more resilience, not less. the flag to enable this is -enable-library-evolution
.
$ swift build --product MyLibrary -Xswiftc -enable-library-evolution
this will cause the compiler to complain if you write code in Client
that assumes things about MyLibrary
that are supposed to be hidden behind a resilience barrier.
switch MyLibrary.SupportedCases.current
{
case .a:
print("a")
case .b:
print("b")
+ @unknown default:
+ print("unknown")
}
$ swift build --product Client -Xlinker -L.build/debug -Xlinker -lMyLibrary
Building for debugging...
[8/8] Linking Client
Build of product 'Client' complete! (0.77s)
$ .build/debug/Client
b
changing a resilient enum
let’s add a new case to SupportedCases
.
public
enum SupportedCases
{
case a
case b
+ case c
public
+ static var current:Self { .c }
}
$ swift build --product MyLibrary -Xswiftc -enable-library-evolution
Building for debugging...
[7/7] Linking libMyLibrary.so
Build of product 'MyLibrary' complete! (0.80s)
$ .build/debug/Client
unknown
it works!
can we make SwiftPM do this for us?
so i’ve got the incantations down, but this workflow just Really Sucks, and i can’t see it scaling up to a project of realistic size. this is stuff our build system is supposed to manage for us!
of all the flags we’re using, -enable-library-evolution
is probably the easiest to integrate. i can just add that to Package.swift
and SwiftPM will forward it to swiftc
instead of me having to retype it every time i recompile MyLibrary
.
.target(name: "MyLibrary",
swiftSettings: [
+ .unsafeFlags(["-enable-library-evolution"]),
]),
but with Client
, we want SwiftPM to understand the dependency relationship between it and MyLibrary
without actually statically linking MyLibrary
into the final executable.
let’s find out what flags are different between the two ways of building Client
with the help of the -v
option.
$ swift build -v --product Client -Xlinker -L.build/debug -Xlinker -lMyLibrary
$ swift build -v --product Client
the meaningful differences, i think, are that these arguments are present in swift-autolink-extract
, clang
, and ld
when linking statically
.build/x86_64-unknown-linux-gnu/debug/MyLibrary.build/Algorithm.swift.o
.build/x86_64-unknown-linux-gnu/debug/MyLibrary.build/MyLibrary.swiftmodule.o
and our custom -L .build/debug
, -lMyLibrary
are missing, which suggests that SwiftPM is not doing anything fancy here behind the scenes, it is literally just telling clang
to add those two extra .o
files to the build.
while flags like -enable-library-evolution
can be added fairly easily to SwiftPM’s generated commands, there doesn’t seem to be a way to suppress them. i expect that i will have to modify the source code of SwiftPM to get it to do this automatically.