Building the Swift Project on Linux with lld instead of gold

Development Compiler
,
21

Glad to hear you got it working.

We would need to see what file was being built. SwiftRT-ELF-WASM.cpp only refers to the symbol, it doesn't define it. The compiler is supposed to emit it into the objects so that swiftrt.o can extract them and make them visible to the runtime. We'd need to see the full swiftc invocation above the link failure to have a better idea of what was getting the symbol removed.

22

Ah, you're right — I should've posted the invocation because I would've noticed this case wouldn't be covered by --extra-swift-args:

FAILED: lib/lib_InternalSwiftStaticMirror.so
: && /home/itai/Development/swift-built/swift-5.10.1-RELEASE-fedora39/usr/bin/clang++
      -fPIC -Wno-unknown-warning-option -Werror=unguarded-availability-new -fno-stack-protector
      -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time
      -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings
      -Wcast-qual -Wmissing-field-initializers -Wimplicit-fallthrough -Wcovered-switch-default
      -Wno-class-memaccess -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor
      -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported
      -fdiagnostics-color -ffunction-sections -fdata-sections -Werror=switch -Wdocumentation
      -Wimplicit-fallthrough -Wunreachable-code -Woverloaded-virtual -DOBJC_OLD_DISPATCH_PROTOTYPES=0
      -O3 -DNDEBUG -Wl,--version-script,"/home/itai/Development/swift-project/build/hosttools/swift-linux-x86_64/tools/libStaticMirror/libStaticMirror.exports"
      -Wl,--gc-sections -Wl,--gc-sections  -target
      x86_64-unknown-linux-gnu -fuse-ld=gold -shared -Wl,-soname,lib_InternalSwiftStaticMirror.so
      -o lib/lib_InternalSwiftStaticMirror.so
      tools/libStaticMirror/CMakeFiles/libStaticMirror.dir/libStaticMirror.cpp.o
      tools/libStaticMirror/CMakeFiles/libStaticMirror.dir/c-include-check.c.o -L/home/itai/Development/swift-project/build/hosttools/llvm-linux-x86_64/./lib
      -L/home/itai/Development/swift-project/build/hosttools/swift-linux-x86_64/lib
      -L/home/itai/Development/swift-built/swift-5.10.1-RELEASE-fedora39/usr/lib/swift/linux
      -L/home/itai/Development/swift-built/swift-5.10.1-RELEASE-fedora39/usr/lib/swift/linux/x86_64
      -Wl,-rpath,"\$ORIGIN:\$ORIGIN/./swift/linux" /home/itai/Development/swift-project/build/hosttools/llvm-linux-x86_64/lib/libLLVMObject.a
      /home/itai/Development/swift-project/build/hosttools/llvm-linux-x86_64/lib/libLLVMSupport.a
      /home/itai/Development/swift-project/build/hosttools/llvm-linux-x86_64/lib/libLLVMDemangle.a
      -lpthread -ldl /home/itai/Development/swift-built/swift-5.10.1-RELEASE-fedora39/usr/lib/swift/linux/x86_64/swiftrt.o
      -lswiftCore  lib/libswiftStaticMirror.a
      lib/libswiftRemoteInspection.a /home/itai/Development/swift-project/build/hosttools/llvm-linux-x86_64/lib/libLLVMObject.a
      /home/itai/Development/swift-project/build/hosttools/llvm-linux-x86_64/lib/libLLVMIRReader.a
      /home/itai/Development/swift-project/build/hosttools/llvm-linux-x86_64/lib/libLLVMBitReader.a
      /home/itai/Development/swift-project/build/hosttools/llvm-linux-x86_64/lib/libLLVMAsmParser.a
      /home/itai/Development/swift-project/build/hosttools/llvm-linux-x86_64/lib/libLLVMCore.a
      /home/itai/Development/swift-project/build/hosttools/llvm-linux-x86_64/lib/libLLVMRemarks.a
      /home/itai/Development/swift-project/build/hosttools/llvm-linux-x86_64/lib/libLLVMBitstreamReader.a
      /home/itai/Development/swift-project/build/hosttools/llvm-linux-x86_64/lib/libLLVMMCParser.a
      /home/itai/Development/swift-project/build/hosttools/llvm-linux-x86_64/lib/libLLVMMC.a
      /home/itai/Development/swift-project/build/hosttools/llvm-linux-x86_64/lib/libLLVMDebugInfoCodeView.a
      /home/itai/Development/swift-project/build/hosttools/llvm-linux-x86_64/lib/libLLVMTextAPI.a
      /home/itai/Development/swift-project/build/hosttools/llvm-linux-x86_64/lib/libLLVMBinaryFormat.a
      /home/itai/Development/swift-project/build/hosttools/llvm-linux-x86_64/lib/libLLVMSupport.a
      -lrt -lm /usr/lib64/libz.so /usr/lib64/libzstd.so /usr/lib64/libtinfo.so /home/itai/Development/swift-project/build/hosttools/llvm-linux-x86_64/lib/libLLVMDemangle.a
      lib/libswiftDemangling.a  -lpthread  -ldl && :

Looks like I'll need to add the flag to swift/tools/libStaticMirror/CMakeLists.txt and try again.

23

Unsure why it doesn't work if you patched the CMake config and passed in the flags, may help to look in build/hosttools/swift-linux-x86_64/CMakeCache.txt and see if those flags all show up.

24

Okay, after a few weeks (!) of experimentation, I've managed to get source builds going for Gentoo. The base changes are currently live on GURU, though I've put out a few cleanup/fix commits that haven't made it out yet.

Some notes:

  • I tried to see if it was possible to get Swift to link using bfd for Gentoo systems that don't have lld and the answer appears to be a resounding "no" (hence why Swift normally defaults to gold); so LLD is required in all cases, both at build time and at runtime
  • Part of what took so long was trying to get Swift built inside of the Portage sandbox:
    1. Packages are built inside of a very restrictive sandbox (great!), which is a different environment from building locally inside of, e.g., your home folder, which means different failure modes
    2. When compilation fails inside of the sandbox, there's no way to "go in", fix things up, and try again from that point — which means that my poor CPU has built LLVM so many times at this point that the source code is etched into its branch predictors
  • I was able to use the pre-built Fedora 39 compiler to build Swift, but not in one go. I never managed to figure out why, but attempting to build a full toolchain in a single go with the prebuilt compiler (with a dummy ld.gold pointing to ld.lld in the PATH) always got stuck in building lldb with something attempting to link using bfd and failing. When this would happen, deleting the lldb build dir and re-running the build script (with the same args) would then build successfully ¯\(ツ)/¯
  • I ended up forgoing the pre-built compiler and using a self-bootstrapping 3-stage build that's been tightened up a little from @etcwilde's original scripts:
    • stage0 builds just LLVM+Clang, CMark, and a bare Swift compiler with no Swift driver/macros:
      "${S}/swift/utils/build-script" \
    --verbose-build \
    --release \
    --install-destdir="${S}/stage0" \
    --extra-cmake-options="${extra_cmake_options}" \
    --bootstrapping=off \
    --build-swift-libexec=false \
    --llvm-install-components='llvm-ar;llvm-cov;llvm-profdata;IndexStore;clang;clang-resource-headers;compiler-rt;clangd;lld;LTO;clang-features-file' \
    --llvm-targets-to-build=host \
    --skip-build-benchmarks \
    --skip-early-swift-driver --skip-early-swiftsyntax \
    --skip-test-cmark \
    --skip-test-linux \
    --skip-test-swift \
    --install-all
      Because LLVM+Clang don't change between invocations, this is the only stage I build them at, and I reuse the build dir in its entirety
    • stage1 then bootstraps a Swift compiler with the Swift driver + macros and the minimal library dependencies:
      export PATH="${S}/stage0/usr/bin:${original_path}"
"${S}/swift/utils/build-script" \
    --verbose-build \
    --release \
    --install-destdir="${S}/stage1" \
    --extra-cmake-options="${extra_cmake_options}" \
    --build-swift-libexec=false \
    --cmark --skip-test-cmark \
    --foundation --skip-test-foundation \
    --libdispatch --skip-test-libdispatch \
    --llbuild --skip-test-llbuild \
    --skip-build-benchmarks \
    --skip-build-llvm \
    --skip-test-linux \
    --skip-test-swift \
    --swift-driver --skip-test-swift-driver \
    --swiftpm --skip-test-swiftpm \
    --xctest --skip-test-xctest \
    --install-all
    • stage2 then builds a full toolchain:
      export PATH="${S}/stage1/usr/bin:${original_path}"
"${S}/swift/utils/build-script" \
   --verbose-build \
   --release \
   --install-destdir="${S}/stage2" \
   --extra-cmake-options="${extra_cmake_options}" \
   --build-swift-libexec=false \
   --foundation --skip-test-foundation \
   --indexstore-db --skip-test-indexstore-db \
   --libdispatch --skip-test-libdispatch \
   --llbuild --skip-test-llbuild \
   --lldb --skip-test-lldb \
   --skip-build-benchmarks \
   --skip-build-llvm \
   --skip-test-linux \
   --skip-test-swift \
   --sourcekit-lsp --skip-test-sourcekit-lsp \
   --swift-driver --skip-test-swift-driver \
   --swift-install-components='autolink-driver;compiler;clang-resource-dir-symlink;stdlib;swift-remote-mirror;sdk-overlay;static-mirror-lib;toolchain-tools;license;sourcekit-inproc' \
   --swiftdocc --skip-test-swiftdocc \
   --swiftpm --skip-test-swiftpm \
   --xctest --skip-test-xctest \
   --install-all
    • Avoiding rebuilding LLVM+Clang between stages saves a huge amount of time (45min per stage on my machine), as does building and installing local copies of curl, libicu, and libxml2, which are present on the filesystem as dependencies
      • With extra-cmake-options, I also avoid building test code + binaries which tend to be built by default:
        local _extra_cmake_options=(
    '-DSWIFT_USE_LINKER=lld',
    '-DBUILD_TESTING:BOOL=NO',
    '-DSWIFT_INCLUDE_TESTS:BOOL=NO',
    '-DSWIFT_INCLUDE_TEST_BINARIES:BOOL=NO',
    '-DCOMPILER_RT_BUILD_ORC:BOOL=NO'
)
    • Given the build product sharing between these stages, a 2-stage build process with the pre-built compiler didn't save enough time over a 3-stage process to warrant the additional 500Mb+ download, in my eyes

In all, this was... an interesting exercise... and I hope that the process for adapting this to building Swift 6 will be a bit easier now.

25

Very cool! I think Swift 6 may bring new challenges (at least it has challenged me in trying to get it to cross compile for armv7)...but I'm looking forward to trying a build on a local Gentoo and see how it goes!

1 Like