1

Hello Everyone,

I am trying to cross compile a swift library using swift version 5.6.2 on ubuntu 20.04 for a 32-bit arm linux system. The swift library that I am building depends on a system library that is found in the sysroot for the target system. There is also a version for the host in /usr/local/lib.

If I try and build for the target using swift build everything works fine:

swift build --destination destination.json
...
Build complete! (4.27s)

However, when I try to cross-compile the tests, it finds the host library and fails to link:

swift build --build-tests --destination destination.json
Building for debugging...
ld.lld: error: /usr/local/lib/<hostlib>.so is incompatible with elf32-littlearm
ld.lld: error: /usr/local/lib/<hostlib>.so is incompatible with elf32-littlearm
ld.lld: error: /usr/local/lib/<hostlib>.so is incompatible with elf32-littlearm
ld.lld: error: /usr/local/lib/<hostlib>.so is incompatible with elf32-littlearm
ld.lld: error: /usr/local/lib/<hostlib>.so is incompatible with elf32-littlearm
ld.lld: error: /usr/local/lib/l<hostlib>.so is incompatible with elf32-littlearm
clang-13: error: linker command failed with exit code 1 (use -v to see invocation)

In order to ensure that pkgconfig isn't finding the wrong .pc file for the system library, I have added the pkgconfig directory for the target to the PKG_CONFIG_PATH environment variable:

PKG_CONFIG_PATH="/dir/to/sysroot/usr/lib/pkgconfig" swift build --build-tests --destination.json

and I get similar results. I have also deleted the pkgconfig file in /usr/local/lib/pkgconfig just in case and it still finds the host lib. I then executed the build using some verbose flags and I found some weird flags being passed to swiftc:

/path/to/usr/bin/swiftc -L/usr/local/lib ...

I don't suppose anyone has run into anything similar or knows if I am doing anything wrong?

2

Can you upload your destination.json? What sys root are you using? Debian 11? Buildroot? Yocto? How are you building your Std lib?

3

I am using a sysroot for a proprietary custom linux build. Saying this, I don't think the sysroot is the issue. As I have said, swift build works, it just doesn't work when --build-tests is included. I can also cross-compile C/C++ programs for the target. I have built the stdlib using the scripts provided in this swift-armv7 repo. Below is my destination.json:

{
   "version":1,
   "sdk":"/usr/local/armv7-unknown-linux-gnueabihf/sysroot",
   "toolchain-bin-dir":"/usr/local/armv7-unknown-linux-gnueabihf/tools/bin",
   "target":"armv7-unknown-linux-gnueabihf",
   "dynamic-library-extension":"so",
   "extra-cc-flags":[
      "-fPIC",
      "-I/usr/local/armv7-unknown-linux-gnueabihf/staging/include",
      "-B/usr/local/armv7-unknown-linux-gnueabihf/sysroot/usr/lib/armv7-unknown-linux-gnueabihf/9.2.0"
   ],
   "extra-swiftc-flags":[
      "-target", "armv7-unknown-linux-gnueabihf",
      "-use-ld=lld",
      "-tools-directory", "/usr/local/armv7-unknown-linux-gnueabihf/tools/bin",
      "-Xlinker", "-rpath", "-Xlinker", "/usr/lib/swift/linux",
      "-Xlinker", "-rpath", "-Xlinker", "/usr/lib/swift/linux/armv7-unknown-linux-gnueabihf",
      "-Xlinker", "-L/usr/local/armv7-unknown-linux-gnueabihf/staging/lib",
      "-Xlinker", "-L/usr/local/armv7-unknown-linux-gnueabihf/sysroot",
      "-Xlinker", "-L/usr/local/armv7-unknown-linux-gnueabihf/sysroot/lib",
      "-Xlinker", "-L/usr/local/armv7-unknown-linux-gnueabihf/sysroot/usr/lib",
      "-Xlinker", "-L/usr/local/armv7-unknown-linux-gnueabihf/sysroot/usr/lib/swift",
      "-Xlinker", "-L/usr/local/armv7-unknown-linux-gnueabihf/sysroot/usr/lib/swift/linux",
      "-Xlinker", "-L/usr/local/armv7-unknown-linux-gnueabihf/sysroot/usr/lib/swift/linux/armv7",
      "-Xlinker", "-L/usr/local/armv7-unknown-linux-gnueabihf/sysroot/usr/lib/armv7-unknown-linux-gnueabihf/9.2.0",
      "-Xlinker", "--build-id=sha1",
      "-I/usr/local/armv7-unknown-linux-gnueabihf/sysroot/usr/include",
      "-I/usr/local/armv7-unknown-linux-gnueabihf/sysroot/usr/lib/swift",
      "-resource-dir", "/usr/local/armv7-unknown-linux-gnueabihf/sysroot/usr/lib/swift",
      "-Xclang-linker", "-B/usr/local/armv7-unknown-linux-gnueabihf/sysroot/usr/lib",
      "-Xclang-linker", "-B/usr/local/armv7-unknown-linux-gnueabihf/sysroot/usr/lib/armv7-unknown-linux-gnueabihf/9.2.0",
      "-Xclang-linker", "-L/usr/local/armv7-unknown-linux-gnueabihf/staging/lib",
      "-sdk", "/usr/local/armv7-unknown-linux-gnueabihf/sysroot"
   ],
   "extra-cpp-flags":[
      "-I/usr/local/armv7-unknown-linux-gnueabihf/staging/include",
      "-lstdc++"
   ]
}
4

For some extra info, if I delete the lib on the host, then compiling with --build-tests does succeed and executes on the target platform. So the flags being passed from the destination.json are working, they are just being placed after the -L/usr/local/lib when being passed to swiftc from the package manager.

5

That is weird that cross-compiling the package works but building the tests pulls in the host system library, could be a bug in SPM or are you doing something different with your SPM package manifest for your test targets that might be causing this? I too recently had to use that PKG_CONFIG_PATH override when cross-compiling a Swift executable for Android, though I was only cross-compiling the executable there, not the tests like you are.

On a side note, your destination JSON is ridiculously long and redundant: there should be no reason to add the target and sdk flags again inside extra-swiftc-flags for example. Take a look at my Android destination JSON for a more minimal example- I'm down to only two extra-swiftc-flags, though that is in part because some of that Android-specific logic has been merged into the Swift toolchain and clang by now- but you may well need more include paths for your sysroot if you have that organized differently. SPM doesn't recognize dynamic-library-extension in the JSON file: these are the six flags it recognizes, along with the version. Anyway, I don't see anything in there that would be causing this problem, but it may help to clean your config up.

One way to debug this would be to try building the tests in verbose mode with the host library package installed and make note of the failing command, then try it again but with the entire host library package uninstalled, including its pkgconfig file. I know you've been trying deleting various files individually, but you should try uninstalling the entire host library package instead, in case you missed some file it's still picking up. If that works, compare the failing command's flags to the now working one and file an SPM bug noting the difference.

6

Thanks for the help. Turns out there was a different system library in a package that was being pulled in that was setting the prefix incorrectly inside its .pc file. So this meant that it was setting the prefix to /usr/local and adding /usr/local/lib to the linker search paths. Thanks @Finagolfin for the info on the destination.json, will definitely apply those changes going forward.