Cross-Compilation on Mac to Ubuntu Fails

Dear Swift Community,

I'm trying to cross-compile a binary on my Mac for Ubuntu 20.04, but I'm facing some issues.

After building the Ubuntu SDK with this command:

/path/to/swift-package-manager/Utilities/build_ubuntu_cross_compilation_toolchain ~/toolchains swift-5.3.3-RELEASE-osx.pkg swift-5.3.3-RELEASE-ubuntu20.04.tar.gz

I tried to compile a simple hello-world package:

$ swift package init --type executable
$ swift build --destination ~/toolchains/cross-toolchain/ubuntu-xenial-destination.json
/Users/user/toolchains/cross-toolchain/swift.xctoolchain/usr/bin/ld.lld: error: cannot open Scrt1.o: No such file or directory
/Users/user/toolchains/cross-toolchain/swift.xctoolchain/usr/bin/ld.lld: error: cannot open crti.o: No such file or directory
/Users/user/toolchains/cross-toolchain/swift.xctoolchain/usr/bin/ld.lld: error: cannot open crtbeginS.o: No such file or directory
/Users/user/toolchains/cross-toolchain/swift.xctoolchain/usr/bin/ld.lld: error: unable to find library -lgcc
/Users/user/toolchains/cross-toolchain/swift.xctoolchain/usr/bin/ld.lld: error: unable to find library -lgcc_s
/Users/user/toolchains/cross-toolchain/swift.xctoolchain/usr/bin/ld.lld: error: unable to find library -lc
/Users/user/toolchains/cross-toolchain/swift.xctoolchain/usr/bin/ld.lld: error: unable to find library -lgcc
/Users/user/toolchains/cross-toolchain/swift.xctoolchain/usr/bin/ld.lld: error: unable to find library -lgcc_s
/Users/user/toolchains/cross-toolchain/swift.xctoolchain/usr/bin/ld.lld: error: cannot open crtendS.o: No such file or directory
/Users/user/toolchains/cross-toolchain/swift.xctoolchain/usr/bin/ld.lld: error: cannot open crtn.o: No such file or directory
clang-10: error: linker command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: link command failed with exit code 1 (use -v to see invocation)
[0/1] Linking test

Passing -sdk to swiftc doesn't work either:

$ swift build --destination ~/toolchains/cross-toolchain/ubuntu-xenial-destination.json -Xswiftc -sdk -Xswiftc ~/toolchains/cross-toolchain/ubuntu-xenial.sdk
[1/1] Planning build

clang-10: error: no such file or directory: '/Users/jefflebrun/toolchains/cross-toolchain/ubuntu-xenial.sdk/usr/lib/swift/linux/x86_64/swiftrt.o'
<unknown>:0: error: link command failed with exit code 1 (use -v to see invocation)
[3/4] Linking test

Nor does passing ~/toolchains/cross-toolchain/swift.xctoolchain to -sdk:

$ swift build --destination ~/toolchains/cross-toolchain/ubuntu-xenial-destination.json -Xswiftc -sdk -Xswiftc ~/toolchains/cross-toolchain/swift.xctoolchain

[1/1] Planning build

/Users/jefflebrun/toolchains/cross-toolchain/swift.xctoolchain/usr/bin/ld.lld: error: cannot open Scrt1.o: No such file or directory
/Users/jefflebrun/toolchains/cross-toolchain/swift.xctoolchain/usr/bin/ld.lld: error: cannot open crti.o: No such file or directory
/Users/jefflebrun/toolchains/cross-toolchain/swift.xctoolchain/usr/bin/ld.lld: error: cannot open crtbeginS.o: No such file or directory
/Users/jefflebrun/toolchains/cross-toolchain/swift.xctoolchain/usr/bin/ld.lld: error: unable to find library -lgcc
/Users/jefflebrun/toolchains/cross-toolchain/swift.xctoolchain/usr/bin/ld.lld: error: unable to find library -lgcc_s
/Users/jefflebrun/toolchains/cross-toolchain/swift.xctoolchain/usr/bin/ld.lld: error: unable to find library -lc
/Users/jefflebrun/toolchains/cross-toolchain/swift.xctoolchain/usr/bin/ld.lld: error: unable to find library -lgcc
/Users/jefflebrun/toolchains/cross-toolchain/swift.xctoolchain/usr/bin/ld.lld: error: unable to find library -lgcc_s
/Users/jefflebrun/toolchains/cross-toolchain/swift.xctoolchain/usr/bin/ld.lld: error: cannot open crtendS.o: No such file or directory
/Users/jefflebrun/toolchains/cross-toolchain/swift.xctoolchain/usr/bin/ld.lld: error: cannot open crtn.o: No such file or directory
clang-10: error: linker command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: link command failed with exit code 1 (use -v to see invocation)
[3/4] Linking test

Jeff

@LebJe the script (without telling you, sorry) builds a cross compilation toolchain for Ubuntu Xenial which is 16.04 but you're trying to use it with Swift for Ubuntu 20.04. I'm not 100% sure this is the issue but that's definitely not quite right.

To be honest, that cross compilation toolchain builder script is more an example what needs to be done to build a cross-compilation toolchain rather than being a real solution. The folks over at the SwiftCrossCompilers project have done more work improving on that script. Unfortunately, the newest configuration they have is Swift 5.1.1 for Ubuntu 18.04.

CC @Helge_Hess1 / @rvsrvs to see if they've built any newer cross-compilation toolchains.

I've gotten 5.3 working on 64 -bit platforms. Haven't caught up with 5.4 yet. I need to spend some time to create a new release. @Helge_Hess1 has his stuff up-to-date. The big issue for me is that 32-bit hasn't worked since 5.1 so if you are targeting the Pi Zero (for example) or 32-bit on Pi 2,3, 4 there's really nothing we can do. @uraimo had posted radars on the problems there, but as far as I know they have not been addressed.

1 Like

I try to keep the SPMDestinations current, I'm currently at 5.3.3 for Ubuntu 16.04 (Intel). If someone wants an Ubuntu 20 X compiler, I'm happy to add that. Please just file an issue asking for it. Same if I got out of date :slight_smile:

I do not provide X compilers targeting ARM Linux (i.e. Raspberry) yet, though the Intel X compilers seem to work fine on an M1 host (i.e. you can build Intel Linux binaries on Apple silicon using the Ubuntu SPMDestinations compiler).

For some context: SPMDestinations is a cleaned up version of @johannesweiss' build_ubuntu_cross_compilation_toolchain example script. It splits the setup into three distinct phases (I think grabbing the host compiler, the destination compiler and building the cross Linux SDK - with Homebrew caching the expensive steps).

P.S.: For the fun of it, I also once built a Linux-ARM to macOS Intel cross compiler: swift-arm2mac-x-compile-toolchain :slight_smile:

1 Like

@johannesweiss I tried cross-compiling using the latest Ubuntu Zenial toolchain and the build failed with the same errors. Thankfully the solution provided by @Helge_Hess1 works.

@Helge_Hess1 Thank you for SPMDestinations, what's even better is that the 16.04 toolchain works on 20.04 if you statically link the standard library:

$ brew install SPMDestinations/tap/spm-dest-5.3-x86_64-ubuntu16.04
$ swift build --static-swift-stdlib --destination /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-ubuntu16.04.xtoolchain/destination.json

$ docker run --rm -v $(pwd):/src -w /src -it ubuntu:latest
root@c12a1cb3622e:/src# ./.build/debug/test
Hello, World!
root@c12a1cb3622e:/src#

@rvsrvs Thanks for the insight! Thankfully the solution provided by @Helge_Hess1 worked for me, just like you said.

1 Like

Uh, that static works is cool, I didn't get it to work w/ Amazon yet. I also started a 20.04 package, we'll see how that goes.

Note that the toolchain may not contain all package headers you might need, if you miss something, let me know (ideally using the issues) - it is easy to add more stuff.

Update: So apparently static linking doesn't work, similar to the issue I have in AL (Can't load `Dispatch` when statically compiling `Foundation` · Issue #11 · SPMDestinations/homebrew-tap · GitHub). It is a little weird because the static flags seem to affect the header lookup (it doesn't fail in linking but in module resolution).

The Swift 5.3.3 Ubuntu 20.04 Focal cross compiler in SPMDestinations is ready and seems to work fine with regular dynamic stdlib compiles, i.e. the Macro examples work and run.

It can be installed using:

brew install spmdestinations/tap/spm-dest-5.3-x86_64-ubuntu20.04

Then compile a project using:

swift build --destination /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-ubuntu20.04.xtoolchain/destination.json

And you'll get proper results:

Zini18:Examples helge$ file .build/x86_64-unknown-linux/debug/express-simple-lambda
.build/x86_64-unknown-linux/debug/express-simple-lambda: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, with debug_info, not stripped

Note that the Docker images still have no proper ld.so configuration, so either set LD_LIBRARY_PATH manually or do:

echo '/usr/lib/swift/linux'            > /etc/ld.so.conf.d/swift.conf;\
echo '/usr/lib/swift/clang/lib/linux' >> /etc/ld.so.conf.d/swift.conf;\
echo '/usr/lib/swift/pm'              >> /etc/ld.so.conf.d/swift.conf;\
ldconfig

to get things working. Have fun!

P.S.: Works on M1 as well.

1 Like