Utils/build-* scripts are confusing; trying to build on Alpine

Development Compiler
1

Hello!

So the long story short is that I would like to build Swift for Alpine, since there are no packages for it by default - neither in the main or community repo and although there seems to be a static SDK, it is advertised for glibc distros. But, Alpine runs musl. So I suppose its static, but still links to the libc (static vs. freestanding). Hence, I want to build it from source.

But after running ./utils/build-script -j(nproc) from source, all that I achieved is a fully built LLVM toolchain and the whole workspace eating a crispy 55GB of space. I tried build-toolchain before that, but that also didn’t go well.

So, I started reading the help output of the build scripts…but grew no wiser.

That’s why I come here: I am on aarch64-linux-musl (to summarize as a triplet). What flags do I pass to either of the build scripts to make this work out? As for hardware, it’s a Radxa Orion O6.

Thank you and kind regards!

2

This has been attempted before, but won't work out of the box, as it requires a heavy modification of the build script to handle this triple. Additionally, you probably won't be able to build the package manager without manual hacks modifying stack size of swift-frontend in ELF binaries directly. SwiftPM dependencies cause stack overflow in certain LLVM passes with the tiny stack size that Musl uses by default.

Overall, this requires a good amount of work but can be done. There's no active work AFAIK on keeping this setup maintained, and with the Swift SDK for Musl the demand for running the toolchain on Alpine has been very low. Yours is the first request I've seen in a few years.

1 Like
3

For us to help, you'd have to tell us if it failed and with what error, including what was output in the build/ directory.

Unfortunately, in order to build the Swift toolchain on Alpine, you need a working Swift compiler for the platform, which may mean cross-compiling a bootstrap Swift compiler first.

I doubt this, adding new triples to build-script is pretty easy, plus much of the Swift stdlib and corelibs porting work has already been done by @al45tair for the static Musl SDK.

4

For us to help, you'd have to tell us if it failed and with what error, including what was output in the build/ directory.

Here is the moment it stops:

-- The Swift compiler identification is unknown
CMake Error at /root/work/swift-syntax/CMakeLists.txt:13 (project):
  No CMAKE_Swift_COMPILER could be found.

  Tell CMake where to find the compiler by setting either the environment
  variable "SWIFTC" or the CMake cache entry CMAKE_Swift_COMPILER to the full
  path to the compiler, or to the compiler name if it is in the PATH.


-- Configuring incomplete, errors occurred!
ERROR: command `['/root/work/swift/utils/build-script-impl', '--workspace', '/root/work', '--build-dir', '/root/work/build/Ninja-RelWithDebInfoAssert', '--install-prefix', '/usr', '--host-target', 'linux-aarch64', '--stdlib-deployment-targets=linux-aarch64', '--host-cc', '/usr/bin/clang', '--host-cxx', '/usr/bin/clang++', '--darwin-xcrun-toolchain', 'default', '--darwin-deployment-version-osx=13.0', '--darwin-deployment-version-ios=16.0', '--darwin-deployment-version-tvos=16.0', '--darwin-deployment-version-watchos=6.0', '--darwin-deployment-version-xros=1.0', '--cmake', '/usr/bin/cmake', '--llvm-build-type', 'RelWithDebInfo', '--swift-build-type', 'RelWithDebInfo', '--swift-stdlib-build-type', 'RelWithDebInfo', '--lldb-build-type', 'RelWithDebInfo', '--foundation-build-type', 'RelWithDebInfo', '--libdispatch-build-type', 'RelWithDebInfo', '--xctest-build-type', 'RelWithDebInfo', '--llbuild-build-type', 'RelWithDebInfo', '--swift-enable-assertions', 'true', '--swift-stdlib-enable-assertions', 'true', '--swift-stdlib-enable-strict-availability', 'false', '--swift-analyze-code-coverage', 'false', '--llbuild-enable-assertions', 'true', '--lldb-assertions', 'true', '--cmake-generator', 'Ninja', '--cross-compile-append-host-target-to-destdir', 'true', '--cross-compile-build-swift-tools', 'true', '--build-jobs', '12', '--lit-jobs', '12', '--common-cmake-options=-G Ninja -DCMAKE_C_COMPILER:PATH=/usr/bin/clang -DCMAKE_CXX_COMPILER:PATH=/usr/bin/clang++ -DCMAKE_Swift_COMPILER:PATH= -DCMAKE_LIBTOOL:PATH=/usr/bin/libtool -DCMAKE_AR:PATH=/usr/bin/ar -DCMAKE_RANLIB:PATH=/usr/bin/ranlib -DLLVM_VERSION_MAJOR:STRING=21 -DLLVM_VERSION_MINOR:STRING=0 -DLLVM_VERSION_PATCH:STRING=0 -DCLANG_VERSION_MAJOR:STRING=21 -DCLANG_VERSION_MINOR:STRING=0 -DCLANG_VERSION_PATCH:STRING=0 -DCMAKE_MAKE_PROGRAM=/usr/bin/ninja', '--build-args=-j12', '--dsymutil-jobs', '1', '--build-swift-libexec', 'true', '--swift-enable-backtracing', 'true', '--build-swift-clang-overlays', 'true', '--build-swift-remote-mirror', 'true', '--swift-source-dirname', 'swift', '--swift-cmake-options=-DCMAKE_EXPORT_COMPILE_COMMANDS=TRUE -DSWIFT_FORCE_OPTIMIZED_TYPECHECKER:BOOL=FALSE -DSWIFT_STDLIB_ENABLE_STDLIBCORE_EXCLUSIVITY_CHECKING:BOOL=FALSE -DSWIFT_ENABLE_EXPERIMENTAL_DIFFERENTIABLE_PROGRAMMING:BOOL=TRUE -DSWIFT_ENABLE_EXPERIMENTAL_CONCURRENCY:BOOL=TRUE -DSWIFT_ENABLE_EXPERIMENTAL_CXX_INTEROP:BOOL=TRUE -DSWIFT_ENABLE_CXX_INTEROP_SWIFT_BRIDGING_HEADER:BOOL=TRUE -DSWIFT_ENABLE_EXPERIMENTAL_DISTRIBUTED:BOOL=TRUE -DSWIFT_ENABLE_BACKTRACING:BOOL=TRUE -DSWIFT_ENABLE_EXPERIMENTAL_OBSERVATION:BOOL=TRUE -DSWIFT_ENABLE_SYNCHRONIZATION:BOOL=TRUE -DSWIFT_ENABLE_VOLATILE:BOOL=TRUE -DSWIFT_ENABLE_RUNTIME_MODULE:BOOL=TRUE -DSWIFT_STDLIB_STATIC_PRINT=FALSE -DSWIFT_FREESTANDING_IS_DARWIN:BOOL=FALSE -DSWIFT_STDLIB_BUILD_PRIVATE:BOOL=TRUE -DSWIFT_STDLIB_ENABLE_UNICODE_DATA=TRUE -DSWIFT_SHOULD_BUILD_EMBEDDED_STDLIB=TRUE -DSWIFT_SHOULD_BUILD_EMBEDDED_STDLIB_CROSS_COMPILING=FALSE -DSWIFT_STDLIB_BUILD_SYMBOL_GRAPHS:BOOL=FALSE -DSWIFT_TOOLS_LD64_LTO_CODEGEN_ONLY_FOR_SUPPORTING_TARGETS:BOOL=FALSE -DSWIFT_ENABLE_EXPERIMENTAL_PARSER_VALIDATION:BOOL=TRUE -USWIFT_DEBUGINFO_NON_LTO_ARGS -DSWIFT_ENABLE_NEW_RUNTIME_BUILD:BOOL=FALSE -DSWIFT_DARWIN_TEST_DEPLOYMENT_VERSION_OSX:STRING=13.0 -DSWIFT_DARWIN_TEST_DEPLOYMENT_VERSION_IOS:STRING=16.0 -DSWIFT_DARWIN_TEST_DEPLOYMENT_VERSION_TVOS:STRING=16.0 -DSWIFT_DARWIN_TEST_DEPLOYMENT_VERSION_WATCHOS:STRING=6.0 -DSWIFT_DARWIN_TEST_DEPLOYMENT_VERSION_XROS:STRING=1.0', '--build-stdlib-deployment-targets', 'all', '--ninja-bin=/usr/bin/ninja', '--install-destdir', '/root/work/build/Ninja-RelWithDebInfoAssert/toolchain-linux-aarch64', '--skip-build-foundation', '--skip-build-xctest', '--skip-build-lldb', '--skip-build-llbuild', '--skip-build-libcxx', '--skip-build-libdispatch', '--skip-build-libxml2', '--skip-build-zlib', '--skip-build-curl', '--build-swift-dynamic-stdlib', '--build-swift-dynamic-sdk-overlay', '--skip-build-android', '--skip-test-swift', '--skip-test-lldb', '--skip-test-llbuild', '--skip-test-xctest', '--skip-test-foundation', '--skip-test-libdispatch', '--skip-test-benchmarks', '--android-deploy-device-path', '/data/local/tmp', '--host-libtool', '/usr/bin/libtool', '--extra-cmake-options=-USWIFT_DARWIN_SUPPORTED_ARCHS -DSWIFT_PATH_TO_SWIFT_SYNTAX_SOURCE:PATH=/root/work/swift-syntax -DSWIFT_BUILD_SWIFT_SYNTAX:BOOL=TRUE -DSWIFTSYNTAX_ENABLE_ASSERTIONS:BOOL=TRUE -DSWIFT_EARLY_SWIFT_DRIVER_BUILD=/root/work/build/Ninja-RelWithDebInfoAssert/earlyswiftdriver-linux-aarch64/release/bin', '--llvm-lit-args=-sv --param color_output', '--llvm-install-components=llvm-ar;llvm-cov;llvm-profdata;IndexStore;clang;clang-resource-headers;compiler-rt;clangd;LTO;lld', '--musl-path=/usr/local/musl', '--linux-static-archs=x86_64;aarch64', '--only-execute', 'linux-aarch64-swift-build']` terminated with a non-zero exit status 1, aborting

And, here is the build output:

Yours is the first request I've seen in a few years.

Oof. Well, Alpine is most often used for deployments in containerized workloads to be fair - and less so as a “home OS”. I just kinda ended up using it a lot more as I started to know and understand it’s internals a little better. This was also in part to PostmarketOS which is Alpine-based, effectively.

5

I am aware what Alpine is used for, but for a containerized deployment the obvious workflow for Swift is to cross-compile with the Musl Swift SDK and copy the product over to the container image. Would that not work in your case?

6

No…because Alpine is my host OS. :slight_smile: It’s where I experiment with compilers, tools, frameworks and the likes the most.

But it’s good to know that apps can be cross-compiled. Will keep that in mind for other things I may end up doing with Swift.

7

OK, that is the issue I mentioned, that you need a prebuilt Swift compiler to build a Swift compiler from source, as it is partially written in Swift. Take a look at the Getting Starting guide, which has this and a lot of other useful info.

What you can do this first time is install Alpine's support for glibc, download the Swift compiler built for whichever linux distro works best with Alpine, then add it to your PATH. You may want to test building small executables with it first to make sure it works well, then run build-script again to build the Swift compiler, and this time it should pick up that prebuilt compiler from your PATH.

As Max says, some patching of the build-script and other portions of the compiler repo will be required- see his old, unmerged pull that he linked for an idea- but the good news is that the stdlib and corelibs like Foundation already work well with Musl used in Alpine. The most work may be required to port the Swift package manager and its current llbuild and future swift-build backends, don't know about the stack issue Max hit.

If you want to work on this and upstream your patches, there is an old github issue for Alpine support that you can use to post your progress and ask questions of any of the three of us.