Okay, after a few weeks (!) of experimentation, I've managed to get source builds going for Gentoo. The base changes are currently live on GURU, though I've put out a few cleanup/fix commits that haven't made it out yet.
Some notes:
- I tried to see if it was possible to get Swift to link using
bfd
for Gentoo systems that don't havelld
and the answer appears to be a resounding "no" (hence why Swift normally defaults togold
); so LLD is required in all cases, both at build time and at runtime - Part of what took so long was trying to get Swift built inside of the Portage sandbox:
- Packages are built inside of a very restrictive sandbox (great!), which is a different environment from building locally inside of, e.g., your home folder, which means different failure modes
- When compilation fails inside of the sandbox, there's no way to "go in", fix things up, and try again from that point — which means that my poor CPU has built LLVM so many times at this point that the source code is etched into its branch predictors
- I was able to use the pre-built Fedora 39 compiler to build Swift, but not in one go. I never managed to figure out why, but attempting to build a full toolchain in a single go with the prebuilt compiler (with a dummy
ld.gold
pointing told.lld
in thePATH
) always got stuck in buildinglldb
with something attempting to link usingbfd
and failing. When this would happen, deleting thelldb
build dir and re-running the build script (with the same args) would then build successfully ¯\(ツ)/¯ - I ended up forgoing the pre-built compiler and using a self-bootstrapping 3-stage build that's been tightened up a little from @etcwilde's original scripts:
stage0
builds just LLVM+Clang, CMark, and a bare Swift compiler with no Swift driver/macros:
Because LLVM+Clang don't change between invocations, this is the only stage I build them at, and I reuse the build dir in its entirety"${S}/swift/utils/build-script" \ --verbose-build \ --release \ --install-destdir="${S}/stage0" \ --extra-cmake-options="${extra_cmake_options}" \ --bootstrapping=off \ --build-swift-libexec=false \ --llvm-install-components='llvm-ar;llvm-cov;llvm-profdata;IndexStore;clang;clang-resource-headers;compiler-rt;clangd;lld;LTO;clang-features-file' \ --llvm-targets-to-build=host \ --skip-build-benchmarks \ --skip-early-swift-driver --skip-early-swiftsyntax \ --skip-test-cmark \ --skip-test-linux \ --skip-test-swift \ --install-all
stage1
then bootstraps a Swift compiler with the Swift driver + macros and the minimal library dependencies:export PATH="${S}/stage0/usr/bin:${original_path}" "${S}/swift/utils/build-script" \ --verbose-build \ --release \ --install-destdir="${S}/stage1" \ --extra-cmake-options="${extra_cmake_options}" \ --build-swift-libexec=false \ --cmark --skip-test-cmark \ --foundation --skip-test-foundation \ --libdispatch --skip-test-libdispatch \ --llbuild --skip-test-llbuild \ --skip-build-benchmarks \ --skip-build-llvm \ --skip-test-linux \ --skip-test-swift \ --swift-driver --skip-test-swift-driver \ --swiftpm --skip-test-swiftpm \ --xctest --skip-test-xctest \ --install-all
stage2
then builds a full toolchain:export PATH="${S}/stage1/usr/bin:${original_path}" "${S}/swift/utils/build-script" \ --verbose-build \ --release \ --install-destdir="${S}/stage2" \ --extra-cmake-options="${extra_cmake_options}" \ --build-swift-libexec=false \ --foundation --skip-test-foundation \ --indexstore-db --skip-test-indexstore-db \ --libdispatch --skip-test-libdispatch \ --llbuild --skip-test-llbuild \ --lldb --skip-test-lldb \ --skip-build-benchmarks \ --skip-build-llvm \ --skip-test-linux \ --skip-test-swift \ --sourcekit-lsp --skip-test-sourcekit-lsp \ --swift-driver --skip-test-swift-driver \ --swift-install-components='autolink-driver;compiler;clang-resource-dir-symlink;stdlib;swift-remote-mirror;sdk-overlay;static-mirror-lib;toolchain-tools;license;sourcekit-inproc' \ --swiftdocc --skip-test-swiftdocc \ --swiftpm --skip-test-swiftpm \ --xctest --skip-test-xctest \ --install-all
- Avoiding rebuilding LLVM+Clang between stages saves a huge amount of time (45min per stage on my machine), as does building and installing local copies of
curl
,libicu
, andlibxml2
, which are present on the filesystem as dependencies- With
extra-cmake-options
, I also avoid building test code + binaries which tend to be built by default:local _extra_cmake_options=( '-DSWIFT_USE_LINKER=lld', '-DBUILD_TESTING:BOOL=NO', '-DSWIFT_INCLUDE_TESTS:BOOL=NO', '-DSWIFT_INCLUDE_TEST_BINARIES:BOOL=NO', '-DCOMPILER_RT_BUILD_ORC:BOOL=NO' )
- With
- Given the build product sharing between these stages, a 2-stage build process with the pre-built compiler didn't save enough time over a 3-stage process to warrant the additional 500Mb+ download, in my eyes
In all, this was... an interesting exercise... and I hope that the process for adapting this to building Swift 6 will be a bit easier now.