Building the Swift Project on Linux with lld instead of gold

[itaiferber]

Okay, after a few weeks (!) of experimentation, I've managed to get source builds going for Gentoo. The base changes are currently live on GURU, though I've put out a few cleanup/fix commits that haven't made it out yet.

Some notes:

• I tried to see if it was possible to get Swift to link using bfd for Gentoo systems that don't have lld and the answer appears to be a resounding "no" (hence why Swift normally defaults to gold); so LLD is required in all cases, both at build time and at runtime
• Part of what took so long was trying to get Swift built inside of the Portage sandbox:
  1. Packages are built inside of a very restrictive sandbox (great!), which is a different environment from building locally inside of, e.g., your home folder, which means different failure modes
  2. When compilation fails inside of the sandbox, there's no way to "go in", fix things up, and try again from that point — which means that my poor CPU has built LLVM so many times at this point that the source code is etched into its branch predictors
• I was able to use the pre-built Fedora 39 compiler to build Swift, but not in one go. I never managed to figure out why, but attempting to build a full toolchain in a single go with the prebuilt compiler (with a dummy ld.gold pointing to ld.lld in the PATH) always got stuck in building lldb with something attempting to link using bfd and failing. When this would happen, deleting the lldb build dir and re-running the build script (with the same args) would then build successfully ¯\(ツ)/¯
• I ended up forgoing the pre-built compiler and using a self-bootstrapping 3-stage build that's been tightened up a little from @etcwilde's original scripts:
  • stage0 builds just LLVM+Clang, CMark, and a bare Swift compiler with no Swift driver/macros:

    "${S}/swift/utils/build-script" \
        --verbose-build \
        --release \
        --install-destdir="${S}/stage0" \
        --extra-cmake-options="${extra_cmake_options}" \
        --bootstrapping=off \
        --build-swift-libexec=false \
        --llvm-install-components='llvm-ar;llvm-cov;llvm-profdata;IndexStore;clang;clang-resource-headers;compiler-rt;clangd;lld;LTO;clang-features-file' \
        --llvm-targets-to-build=host \
        --skip-build-benchmarks \
        --skip-early-swift-driver --skip-early-swiftsyntax \
        --skip-test-cmark \
        --skip-test-linux \
        --skip-test-swift \
        --install-all
    

    Because LLVM+Clang don't change between invocations, this is the only stage I build them at, and I reuse the build dir in its entirety
  • stage1 then bootstraps a Swift compiler with the Swift driver + macros and the minimal library dependencies:

    export PATH="${S}/stage0/usr/bin:${original_path}"
    "${S}/swift/utils/build-script" \
        --verbose-build \
        --release \
        --install-destdir="${S}/stage1" \
        --extra-cmake-options="${extra_cmake_options}" \
        --build-swift-libexec=false \
        --cmark --skip-test-cmark \
        --foundation --skip-test-foundation \
        --libdispatch --skip-test-libdispatch \
        --llbuild --skip-test-llbuild \
        --skip-build-benchmarks \
        --skip-build-llvm \
        --skip-test-linux \
        --skip-test-swift \
        --swift-driver --skip-test-swift-driver \
        --swiftpm --skip-test-swiftpm \
        --xctest --skip-test-xctest \
        --install-all
    

  • stage2 then builds a full toolchain:

    export PATH="${S}/stage1/usr/bin:${original_path}"
    "${S}/swift/utils/build-script" \
       --verbose-build \
       --release \
       --install-destdir="${S}/stage2" \
       --extra-cmake-options="${extra_cmake_options}" \
       --build-swift-libexec=false \
       --foundation --skip-test-foundation \
       --indexstore-db --skip-test-indexstore-db \
       --libdispatch --skip-test-libdispatch \
       --llbuild --skip-test-llbuild \
       --lldb --skip-test-lldb \
       --skip-build-benchmarks \
       --skip-build-llvm \
       --skip-test-linux \
       --skip-test-swift \
       --sourcekit-lsp --skip-test-sourcekit-lsp \
       --swift-driver --skip-test-swift-driver \
       --swift-install-components='autolink-driver;compiler;clang-resource-dir-symlink;stdlib;swift-remote-mirror;sdk-overlay;static-mirror-lib;toolchain-tools;license;sourcekit-inproc' \
       --swiftdocc --skip-test-swiftdocc \
       --swiftpm --skip-test-swiftpm \
       --xctest --skip-test-xctest \
       --install-all
    

  • Avoiding rebuilding LLVM+Clang between stages saves a huge amount of time (45min per stage on my machine), as does building and installing local copies of curl, libicu, and libxml2, which are present on the filesystem as dependencies
    • With extra-cmake-options, I also avoid building test code + binaries which tend to be built by default:

      local _extra_cmake_options=(
          '-DSWIFT_USE_LINKER=lld',
          '-DBUILD_TESTING:BOOL=NO',
          '-DSWIFT_INCLUDE_TESTS:BOOL=NO',
          '-DSWIFT_INCLUDE_TEST_BINARIES:BOOL=NO',
          '-DCOMPILER_RT_BUILD_ORC:BOOL=NO'
      )
      

  • Given the build product sharing between these stages, a 2-stage build process with the pre-built compiler didn't save enough time over a 3-stage process to warrant the additional 500Mb+ download, in my eyes

In all, this was... an interesting exercise... and I hope that the process for adapting this to building Swift 6 will be a bit easier now.