AWS Lambda functions and the Linux Static SDK - works!

johannesweiss · June 24, 2024, 11:04am

Thanks for checking! I'm somewhat surprised that it didn't make a much bigger difference . @al45tair wdyt, wouldn't you have expected more too?

al45tair · June 24, 2024, 11:07am

No, I don't think so. The majority of the debug information is likely from the libraries rather than from Swift code built with swift build.

(Also, FWIW, I don't think swift-backtrace currently knows how to handle -gline-tables-only. That's on my to-do list.)

johannesweiss · June 24, 2024, 11:15am

We're doing it for both C & Swift in that command which should cover most code, incl. BoringSSL.

Hmm, seems to work for me:

`-O -gline-tables-only`

works (inlined frames & line numbers), click to see full output

root@380a8d052ac3:/tmp/foo# rm -f test && swiftc -gline-tables-only -O test.swift && ls -lah && SWIFT_BACKTRACE=interactive=no ./test
total 84K
drwxr-xr-x 4 root root  128 Jun 24 11:12 .
drwxrwxrwt 1 root root 4.0K Jun 24 11:12 ..
-rwxr-xr-x 1 root root  74K Jun 24 11:12 test
-rw-r--r-- 1 root root  117 Jun 24 11:10 test.swift

💣 Swift runtime failure: precondition failure

Thread 0 "test" crashed:

0 [inlined] [system] 0x0000aaaaaca10a10 Swift runtime failure: precondition failure in test at /tmp/foo/<compiler-generated>
1 [inlined]          0x0000aaaaaca10a10 bar in test at /tmp/foo/test.swift:2:5
2 [inlined]          0x0000aaaaaca10a10 foo in test at /tmp/foo/test.swift:6:5
3                    0x0000aaaaaca10a10 main() in test at /tmp/foo/test.swift:11:5
4 [ra]               0x0000ffff9a8573fc <unknown> in libc.so.6
5 [ra]               0x0000ffff9a8574cc <unknown> in libc.so.6


Registers:

 x0 0x0000000000000001  1
 x1 0x0000fffff85598e8  45 a8 55 f8 ff ff 00 00 00 00 00 00 00 00 00 00  E¨Uøÿÿ··········
 x2 0x0000fffff85598f8  4c a8 55 f8 ff ff 00 00 6b a8 55 f8 ff ff 00 00  L¨Uøÿÿ··k¨Uøÿÿ··
 x3 0x0000aaaaaca10a04  fd 7b bf a9 fd 03 00 91 01 00 00 94 20 00 20 d4  ý{¿©ý······· · Ô
 x4 0x0000000000000000  0
 x5 0x9e9203d76bc81010  11426199428353495056
 x6 0x0000000000000000  0
 x7 0x0000000000000000  0
 x8 0x0000000000000000  0
 x9 0x0000aaaaaca30048  03 00 00 00 00 00 00 00 30 00 a3 ac aa aa 00 00  ········0·£¬ªª··
x10 0x0000ffff9b2b5490  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ················
x11 0x0000000000000000  0
x12 0x2020202020202020  2314885530818453536
x13 0x0000000000000000  0
x14 0x0000000000000000  0
x15 0x0000000000000008  8
x16 0x0000ffff9b2c9774  fd 7b bb a9 fd 03 00 91 f7 63 03 a9 37 01 00 f0  ý{»©ý···÷c·©7··ð
x17 0x0000ffff9a9cb080  74 97 2c 9b ff ff 00 00 c0 96 8c 9a ff ff 00 00  t·,·ÿÿ··À···ÿÿ··
x18 0x0000000000000000  0
x19 0x0000fffff85598e8  45 a8 55 f8 ff ff 00 00 00 00 00 00 00 00 00 00  E¨Uøÿÿ··········
x20 0x0000000000000001  1
x21 0x0000aaaaaca2fd70  03 00 00 00 00 00 00 00 e8 ff a2 ac aa aa 00 00  ········èÿ¢¬ªª··
x22 0x0000ffff9b2f1040  70 23 2f 9b ff ff 00 00 0e 00 00 00 00 00 00 00  p#/·ÿÿ··········
x23 0x0000aaaaaca10a04  fd 7b bf a9 fd 03 00 91 01 00 00 94 20 00 20 d4  ý{¿©ý······· · Ô
x24 0x0000ffff9a9ca000  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ················
x25 0x0000000000000000  0
x26 0x0000fffff85598f8  4c a8 55 f8 ff ff 00 00 6b a8 55 f8 ff ff 00 00  L¨Uøÿÿ··k¨Uøÿÿ··
x27 0x0000aaaaaca2fd70  03 00 00 00 00 00 00 00 e8 ff a2 ac aa aa 00 00  ········èÿ¢¬ªª··
x28 0x0000000000000000  0
 fp 0x0000fffff8559760  70 97 55 f8 ff ff 00 00 fc 73 85 9a ff ff 00 00  p·Uøÿÿ··üs··ÿÿ··
 lr 0x0000aaaaaca10a10  20 00 20 d4 20 00 20 d4 fd 7b bf a9 fd 03 00 91   · Ô · Ôý{¿©ý···
 sp 0x0000fffff8559760  70 97 55 f8 ff ff 00 00 fc 73 85 9a ff ff 00 00  p·Uøÿÿ··üs··ÿÿ··
 pc 0x0000aaaaaca10a10  20 00 20 d4 20 00 20 d4 fd 7b bf a9 fd 03 00 91   · Ô · Ôý{¿©ý···


Images (11 omitted):

0x0000aaaaaca10000–0x0000aaaaaca10bf8 <no build ID>                            test      /tmp/foo/test
0x0000ffff9a830000–0x0000ffff9a9b7404 9e27cf97f03940293c6df3b107674ceda9c825d8 libc.so.6 /usr/lib/aarch64-linux-gnu/libc.so.6

Backtrace took 0.00s

Trace/breakpoint trap

Just `-O` (no `-g...`):

no line numbers, no inlined frames (as expected), click to see full output

root@380a8d052ac3:/tmp/foo# rm -f test && swiftc -O test.swift && ls -lah && SWIFT_BACKTRACE=interactive=no ./test
total 80K
drwxr-xr-x 4 root root  128 Jun 24 11:12 .
drwxrwxrwt 1 root root 4.0K Jun 24 11:12 ..
-rwxr-xr-x 1 root root  72K Jun 24 11:12 test
-rw-r--r-- 1 root root  117 Jun 24 11:10 test.swift

💣 Program crashed: System trap at 0x0000aaaad8400a10

Thread 0 "test" crashed:

0      0x0000aaaad8400a10 main() in test
1 [ra] 0x0000ffffbc7b73fc <unknown> in libc.so.6
2 [ra] 0x0000ffffbc7b74cc <unknown> in libc.so.6


Registers:

 x0 0x0000000000000001  1
 x1 0x0000ffffe0e569d8  45 78 e5 e0 ff ff 00 00 00 00 00 00 00 00 00 00  Exåàÿÿ··········
 x2 0x0000ffffe0e569e8  4c 78 e5 e0 ff ff 00 00 6b 78 e5 e0 ff ff 00 00  Lxåàÿÿ··kxåàÿÿ··
 x3 0x0000aaaad8400a04  fd 7b bf a9 fd 03 00 91 01 00 00 94 20 00 20 d4  ý{¿©ý······· · Ô
 x4 0x0000000000000000  0
 x5 0xe6dd9c63859eae5b  16635624549833944667
 x6 0x0000000000000000  0
 x7 0x0000000000000000  0
 x8 0x0000000000000000  0
 x9 0x0000aaaad8420048  03 00 00 00 00 00 00 00 30 00 42 d8 aa aa 00 00  ········0·BØªª··
x10 0x0000ffffbd210490  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ················
x11 0x0000000000000000  0
x12 0x2020202020202020  2314885530818453536
x13 0x0000000000000000  0
x14 0x0000000000000000  0
x15 0x0000000000000008  8
x16 0x0000ffffbd224774  fd 7b bb a9 fd 03 00 91 f7 63 03 a9 37 01 00 f0  ý{»©ý···÷c·©7··ð
x17 0x0000ffffbc92b080  74 47 22 bd ff ff 00 00 c0 96 82 bc ff ff 00 00  tG"½ÿÿ··À··¼ÿÿ··
x18 0x0000000000000000  0
x19 0x0000ffffe0e569d8  45 78 e5 e0 ff ff 00 00 00 00 00 00 00 00 00 00  Exåàÿÿ··········
x20 0x0000000000000001  1
x21 0x0000aaaad841fd70  03 00 00 00 00 00 00 00 e8 ff 41 d8 aa aa 00 00  ········èÿAØªª··
x22 0x0000ffffbd24c040  70 d3 24 bd ff ff 00 00 0e 00 00 00 00 00 00 00  pÓ$½ÿÿ··········
x23 0x0000aaaad8400a04  fd 7b bf a9 fd 03 00 91 01 00 00 94 20 00 20 d4  ý{¿©ý······· · Ô
x24 0x0000ffffbc92a000  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ················
x25 0x0000000000000000  0
x26 0x0000ffffe0e569e8  4c 78 e5 e0 ff ff 00 00 6b 78 e5 e0 ff ff 00 00  Lxåàÿÿ··kxåàÿÿ··
x27 0x0000aaaad841fd70  03 00 00 00 00 00 00 00 e8 ff 41 d8 aa aa 00 00  ········èÿAØªª··
x28 0x0000000000000000  0
 fp 0x0000ffffe0e56850  60 68 e5 e0 ff ff 00 00 fc 73 7b bc ff ff 00 00  `håàÿÿ··üs{¼ÿÿ··
 lr 0x0000aaaad8400a10  20 00 20 d4 20 00 20 d4 fd 7b bf a9 fd 03 00 91   · Ô · Ôý{¿©ý···
 sp 0x0000ffffe0e56850  60 68 e5 e0 ff ff 00 00 fc 73 7b bc ff ff 00 00  `håàÿÿ··üs{¼ÿÿ··
 pc 0x0000aaaad8400a10  20 00 20 d4 20 00 20 d4 fd 7b bf a9 fd 03 00 91   · Ô · Ôý{¿©ý···


Images (11 omitted):

0x0000aaaad8400000–0x0000aaaad8400bf8 <no build ID>                            test      /tmp/foo/test
0x0000ffffbc790000–0x0000ffffbc917404 9e27cf97f03940293c6df3b107674ceda9c825d8 libc.so.6 /usr/lib/aarch64-linux-gnu/libc.so.6

Backtrace took 0.00s

Trace/breakpoint trap

`-O -g` (SwiftPM default for `-c release`)

works (inline frames and line numbers, as expected), click to see full output

root@380a8d052ac3:/tmp/foo# rm -f test && swiftc -g -O test.swift && ls -lah && SWIFT_BACKTRACE=interactive=no ./test
total 84K
drwxr-xr-x 4 root root  128 Jun 24 11:12 .
drwxrwxrwt 1 root root 4.0K Jun 24 11:12 ..
-rwxr-xr-x 1 root root  74K Jun 24 11:12 test
-rw-r--r-- 1 root root  117 Jun 24 11:10 test.swift

💣 Swift runtime failure: precondition failure

Thread 0 "test" crashed:

0 [inlined] [system] 0x0000aaaab4d40a10 Swift runtime failure: precondition failure in test at /tmp/foo/<compiler-generated>
1 [inlined]          0x0000aaaab4d40a10 bar() in test at /tmp/foo/test.swift:2:5
2 [inlined]          0x0000aaaab4d40a10 foo() in test at /tmp/foo/test.swift:6:5
3                    0x0000aaaab4d40a10 main() in test at /tmp/foo/test.swift:11:5
4 [ra]               0x0000ffff95e773fc <unknown> in libc.so.6
5 [ra]               0x0000ffff95e774cc <unknown> in libc.so.6


Registers:

 x0 0x0000000000000001  1
 x1 0x0000ffffd0185ce8  45 68 18 d0 ff ff 00 00 00 00 00 00 00 00 00 00  Eh·Ðÿÿ··········
 x2 0x0000ffffd0185cf8  4c 68 18 d0 ff ff 00 00 6b 68 18 d0 ff ff 00 00  Lh·Ðÿÿ··kh·Ðÿÿ··
 x3 0x0000aaaab4d40a04  fd 7b bf a9 fd 03 00 91 01 00 00 94 20 00 20 d4  ý{¿©ý······· · Ô
 x4 0x0000000000000000  0
 x5 0x9f0910e45fa8b4a0  11459709299864417440
 x6 0x0000000000000000  0
 x7 0x0000000000000000  0
 x8 0x0000000000000000  0
 x9 0x0000aaaab4d60048  03 00 00 00 00 00 00 00 30 00 d6 b4 aa aa 00 00  ········0·Ö´ªª··
x10 0x0000ffff968cc490  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ················
x11 0x0000000000000000  0
x12 0x2020202020202020  2314885530818453536
x13 0x0000000000000000  0
x14 0x0000000000000000  0
x15 0x0000000000000008  8
x16 0x0000ffff968e0774  fd 7b bb a9 fd 03 00 91 f7 63 03 a9 37 01 00 f0  ý{»©ý···÷c·©7··ð
x17 0x0000ffff95feb080  74 07 8e 96 ff ff 00 00 c0 96 ee 95 ff ff 00 00  t···ÿÿ··À·î·ÿÿ··
x18 0x0000000000000000  0
x19 0x0000ffffd0185ce8  45 68 18 d0 ff ff 00 00 00 00 00 00 00 00 00 00  Eh·Ðÿÿ··········
x20 0x0000000000000001  1
x21 0x0000aaaab4d5fd70  03 00 00 00 00 00 00 00 e8 ff d5 b4 aa aa 00 00  ········èÿÕ´ªª··
x22 0x0000ffff96908040  70 93 90 96 ff ff 00 00 0e 00 00 00 00 00 00 00  p···ÿÿ··········
x23 0x0000aaaab4d40a04  fd 7b bf a9 fd 03 00 91 01 00 00 94 20 00 20 d4  ý{¿©ý······· · Ô
x24 0x0000ffff95fea000  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ················
x25 0x0000000000000000  0
x26 0x0000ffffd0185cf8  4c 68 18 d0 ff ff 00 00 6b 68 18 d0 ff ff 00 00  Lh·Ðÿÿ··kh·Ðÿÿ··
x27 0x0000aaaab4d5fd70  03 00 00 00 00 00 00 00 e8 ff d5 b4 aa aa 00 00  ········èÿÕ´ªª··
x28 0x0000000000000000  0
 fp 0x0000ffffd0185b60  70 5b 18 d0 ff ff 00 00 fc 73 e7 95 ff ff 00 00  p[·Ðÿÿ··üsç·ÿÿ··
 lr 0x0000aaaab4d40a10  20 00 20 d4 20 00 20 d4 fd 7b bf a9 fd 03 00 91   · Ô · Ôý{¿©ý···
 sp 0x0000ffffd0185b60  70 5b 18 d0 ff ff 00 00 fc 73 e7 95 ff ff 00 00  p[·Ðÿÿ··üsç·ÿÿ··
 pc 0x0000aaaab4d40a10  20 00 20 d4 20 00 20 d4 fd 7b bf a9 fd 03 00 91   · Ô · Ôý{¿©ý···


Images (11 omitted):

0x0000aaaab4d40000–0x0000aaaab4d44b50 <no build ID>                            test      /tmp/foo/test
0x0000ffff95e50000–0x0000ffff95fd7404 9e27cf97f03940293c6df3b107674ceda9c825d8 libc.so.6 /usr/lib/aarch64-linux-gnu/libc.so.6

Backtrace took 0.00s

Trace/breakpoint trap

My test program:

root@380a8d052ac3:/tmp/foo# cat test.swift 
func bar() {
    preconditionFailure()
}

func foo() {
    bar()
}

@inline(never)
func main() {
    foo()
}

main()

al45tair · June 24, 2024, 11:35am

Is that on macOS? It'll work there, I think. But not on Linux, I think, at least not presently.

Other than the Swift runtime, the C library, the C++ library, ICU and so on… so there's still quite a bit of code that will have debug information regardless (because it's in the .a files you're linking with).

johannesweiss · June 24, 2024, 1:25pm

That was Linux, specifically the swift:5.10 Docker container.

Of course, yes, that's fair. Do we build those with full debug info?
EDIT: I guess you're saying that whatever savings -gline-tables-only could make might be dwarfed by the size of the binary libraries we ship with Swift.

rauhul · June 24, 2024, 2:36pm

You can also try lto with --experimental-lto-mode full

al45tair · June 26, 2024, 9:39am

Though note that the static libraries in the SDK aren't built with LTO enabled. I did try, but hit some problems; it'd be nice to have an LTO enabled version available in future, for sure, but right now you'll only get LTO for your own code.

pro100filipp · September 17, 2025, 9:40am

Hello, any thoughts on implementing this as a part of standard flow? Currently after stripping sizes differ not so much (imho). Results:

dynamic linking on macOS w/o stripping: `10M`
static linking Linux w/o stripping: `88M`
dynamic linking on macOS after stripping: `4.6M`
static linking Linux after stripping: `17M`

Terminal output

pro100filipp@iMac-Filipp hello % swift build -c release
[1/1] Planning build
Building for production...
warning: input verification failed
note: while processing /Users/pro100filipp/Projects/Swift/hello/.build/arm64-apple-macosx/release/hello.build/main.swift.o
[56/56] Linking hello
Build complete! (55.32s)
pro100filipp@iMac-Filipp hello % swift build -c release --swift-sdk swift-6.2-RELEASE_static-linux-0.0.1
warning: multiple Swift SDKs match ID `swift-6.2-RELEASE_static-linux-0.0.1` and host triple arm64-apple-macosx26.0, selected one at /Users/pro100filipp/Library/org.swift.swiftpm/swift-sdks/swift-6.2-RELEASE_static-linux-0.0.1.artifactbundle/swift-6.2-RELEASE_static-linux-0.0.1/swift-linux-musl
Building for production...
[56/56] Linking hello
Build complete! (40.61s)
pro100filipp@iMac-Filipp hello % ls -alh .build/aarch64-swift-linux-musl/release/hello
Permissions Size User         Date Modified Name
.rwxr-xr-x@  88M pro100filipp 17 Sep 12:17  󰡯 .build/aarch64-swift-linux-musl/release/hello
pro100filipp@iMac-Filipp hello % ls -lah .build/arm64-apple-macosx/release/hello
Permissions Size User         Date Modified Name
.rwxr-xr-x@  10M pro100filipp 17 Sep 12:16  󰡯 .build/arm64-apple-macosx/release/hello
pro100filipp@iMac-Filipp hello % llvm-objcopy --strip-unneeded .build/aarch64-swift-linux-musl/release/hello
pro100filipp@iMac-Filipp hello % ls -lah .build/aarch64-swift-linux-musl/release/hello
Permissions Size User         Date Modified Name
.rwxr-xr-x@  17M pro100filipp 17 Sep 12:20  󰡯 .build/aarch64-swift-linux-musl/release/hello
pro100filipp@iMac-Filipp hello % strip .build/arm64-apple-macosx/release/hello
pro100filipp@iMac-Filipp hello % ls -lah .build/arm64-apple-macosx/release/hello
Permissions Size User         Date Modified Name
.rwxr-xr-x@ 4.6M pro100filipp 17 Sep 12:20  󰡯 .build/arm64-apple-macosx/release/hello

Tested on minimal example provided in tutorial

sebsto · September 17, 2025, 10:12am

Thank @pro100filipp for sharing these data.

Offering an option to use the static SDK and stripping the binary is on our task list right after v2 will be published.

github.com/swift-server/swift-aws-lambda-runtime

[plugin] add option to compile with the Static Linux SDK instead of docker

opened 05:29PM - 19 Jun 24 UTC

sebsto

kind/enhancement 🆕 semver/minor status/needs-design

### Expected behavior Now that the [Static Linux SDK](https://www.swift.org/d…ocumentation/articles/static-linux-getting-started.html) has been released, it open the doors to cross-platform compilation from Mac that doesn't require Docker or depend on Amazon Linux. The binaries created with the Static Linux SDK work "as is" on AWS Lambda. I propose to add an option to the Archiver plugin to compile the binary with the Static Linux SDK instead of Docker. When this option is enabled (it might become the default at some stage): 1. the plugin would check if the Linux Static SDK is installed, and install the correct version automatically when it's not installed. 2. the plugin will check the current platform architecture (or a command line flag to force a specific architecture) 3. the plugin will then compile the Lambda functions with `swift build -c release --swift-sdk aarch64-swift-linux-musl` (or x64_86) 4. the plugin will create a `bootstrap` sym link to the executable and package the sym link and the executable in a ZIP file However, **some pre-requisites are necessary before to release this enhancement** - Swift 6 must be generally available. At the moment, installing the toolchain requires manual steps that we don't want to include in the archive plugin code - The Swift runtime for AWS Lambda must compile with musl (see #331) - The [segmentation fault problem must be fixed](https://forums.swift.org/t/distributing-binaries-for-linuxes-static-linking/21366/24), to avoid further ceremony to get the toolchain working. ### Actual behavior The archiver plugin requires Docker and an Amazon Linux 2 image with the Swift toolchain installed ### Steps to reproduce n/a ### If possible, minimal yet complete reproducer code (or URL to code) Here is the script I use to test this on macOS Sonoma ``` export TOOLCHAINS=org.swift.600202406131a PATH_TO_TOOLCHAIN=/Library/Developer/Toolchains/swift-6.0-DEVELOPMENT-SNAPSHOT-2024-06-13-a.xctoolchain LAMBDA_USE_LOCAL_DEPS=true DYLD_LIBRARY_PATH=$PATH_TO_TOOLCHAIN/usr/lib/swift/macosx $PATH_TO_TOOLCHAIN/usr/bin/swift build -c release --swift-sdk aarch64-swift-linux-musl pushd .build/aarch64-swift-linux-musl/release cp UrlLambda bootstrap zip UrlLambda.zip bootstrap cp SQSLambda bootstrap zip SQSLambda.zip bootstrap cp HttpApiLambda bootstrap zip HttpApiLambda.zip bootstrap rm bootstrap popd ``` ### What version of this project (`swift-aws-lambda-runtime`) are you using? 1.0.0-alpha1 ### Swift version ``` ✗ swift --version && uname -a Apple Swift version 6.0-dev (LLVM 6e13b8be259f20f, Swift 372de4670b9cd55) Target: arm64-apple-macosx14.0 Darwin bcd07458a91c 23.5.0 Darwin Kernel Version 23.5.0: Wed May 1 20:12:58 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T6000 arm64 ``` ### Amazon Linux 2 docker image version _No response_

Fell free to comment or contribute if you want.

AWS Lambda functions and the Linux Static SDK - works!

-O -gline-tables-only

Just -O (no -g...):

-O -g (SwiftPM default for -c release)

`-O -gline-tables-only`

Just `-O` (no `-g...`):

`-O -g` (SwiftPM default for `-c release`)