Build fails to find swiftrt.o in Swiftly's lib directory

Development Compiler
,
1

I am following the recommended guide (swift/docs/HowToGuides/GettingStarted.md at main · swiftlang/swift · GitHub) to compile Swift on my local machine (Ubuntu 24.04 LTS). I installed a prebuilt Swift release toolchain using Swiftly (guide: Swift.org - Install Swift) and then built swift.

utils/build-script --release-debuginfo --xctest --sccache --debug-swift

The build fails with this error.

ninja: error: '/home/rup/.local/share/swiftly/lib/swift/linux/x86_64/swiftrt.o', needed by 'lib/libswiftDemangle.so', missing and no known rule to make it

I checked I don't have a lib directory in ~/.local/share/swiftly.

A build log that I found has 8 lines which I quote below. This could provide some context about the stage at which the build failed. (I could not find a full build log; any pointer to one is appreciated.)

swift-project/swift/build/.build_script_log

{"event": "start", "command": "Building earlyswiftdriver", "duration": "0.00"}
{"event": "end", "command": "Building earlyswiftdriver", "duration": "121.19"}
{"event": "start", "command": "Building cmark", "duration": "0.00"}
{"event": "end", "command": "Building cmark", "duration": "2.17"}
{"event": "start", "command": "Building llvm", "duration": "0.00"}
{"event": "end", "command": "Building llvm", "duration": "2881.73"}
{"event": "start", "command": "linux-x86_64-swift-build", "duration": "0.00"}
{"event": "end", "command": "linux-x86_64-swift-build", "duration": "12.68"}

This ninja build script shows what invoked the compilation:

swift-project/build/Ninja-RelWithDebInfoAssert+swift-DebugAssert/swift-linux-x86_64/build.ninja

#############################################

Link the shared library lib/libswiftDemangle.so

build lib/libswiftDemangle.so: CXX_SHARED_LIBRARY_LINKER__swiftDemangle_Debug lib/SwiftDemangle/CMakeFiles/swiftDemangle.dir/SwiftDemangle.cpp.o lib/SwiftDemangle/CMakeFiles/swiftDemangle.dir/MangleHack.cpp.o | /home/rup/.local/share/swiftly/lib/swift/linux/x86_64/swiftrt.o lib/libswiftDemangling.a || clang-tablegen-targets intrinsics_gen lib/libswiftDemanglin
g.a
LANGUAGE_COMPILE_FLAGS = -Wno-unknown-warning-option -Werror=unguarded-availability-new -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-ty
pe -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -g
LINK_FLAGS = -target x86_64-unknown-linux-gnu -fuse-ld=gold -Xlinker --build-id=sha1
LINK_LIBRARIES = -Wl,-rpath,"$$ORIGIN:$$ORIGIN/./swift/linux" -lpthread -ldl /home/rup/.local/share/swiftly/lib/swift/linux/x86_64/swiftrt.o -lswiftCore lib/libswiftDemangling.a -lpthread -ldl
LINK_PATH = -L/home/rup/swifthack/local/swift-project/build/Ninja-RelWithDebInfoAssert+swift-DebugAssert/llvm-linux-x86_64/./lib -L/home/rup/swifthack/local/swift-project/build/Ninja-RelWithDebInfoAssert+swift-DebugAssert/swift-linux-x86_64/lib -L/home/rup/.local/share/swiftly/lib/swift/linux -L/home/rup/.local/share/swiftly/lib/swift/linux/x86_64
OBJECT_DIR = lib/SwiftDemangle/CMakeFiles/swiftDemangle.dir
POST_BUILD = :
PRE_LINK = :
SONAME = libswiftDemangle.so
SONAME_FLAG = -Wl,-soname,
TARGET_COMPILE_PDB = lib/SwiftDemangle/CMakeFiles/swiftDemangle.dir/
TARGET_FILE = lib/libswiftDemangle.so
TARGET_PDB = lib/libswiftDemangle.pdb
pool = local_jobs

What could I have missed out, in the pre-build steps, in order to prep for the build?

2

I don't think compatibility of swiftly and the Swift toolchain build script has been tested. As discussed in a similar thread Swift compiler building error after cloning the repo - #26 by Max_Desiatov, it's best to replicate the CI environment as closely as possible, which does not use swiftly at all.

I'd recommend using the Docker image that CI uses for building the toolchain. If you prefer Ubuntu 24.04, it's this one: swift-docker/swift-ci/main/ubuntu/24.04/Dockerfile at main · swiftlang/swift-docker · GitHub

3

@Max_Desiatov, thank you. I went through the forum thread that you pointed me to. Even though the issue being discussed in that thread was different from mine, (I had Swiftly in the loop and pointers to Swiftly were missing), it seems to me that an identical replication of the CI build environment would guarantee a local build that runs to completion. I was able to run the CI build (Docker) and it completed without any errors (it took 4.5 hours). Henceforth, to hack on Swift, I will use the CI build.

I think, a build on the local machine, as described in the how-to (swift/docs/HowToGuides/GettingStarted.md at main · swiftlang/swift · GitHub) is not going to work for a lot of folks. How can I help improve that page? Since I spent some time digging into the two ways of building the Swift compiler for development, I can add a few notes, about pitfalls, to that page. If you think that's worth the effort, please let me know, or point me to someone with whom I could discuss a rewrite.

4

Yes, please do, that would be greatly appreciated. That page is maintained by @anthonylatsis and @xedin IIUC

1 Like
5

For posterity, I've also filed an issue on swiftly: Unable to build the Swift toolchain with swiftly · Issue #333 · swiftlang/swiftly · GitHub

2 Likes
6

@Max_Desiatov I don't want to start a new topic on failing builds because this could be something that I am doing wrong. For unrelated reasons I had to install Ubuntu 24.04 and start afresh, but now the Docker builds are failing. Here are two examples. For both the instances, I used Dockerfiles with sccache (v 0.10.0) installation commands added. I enabled Docker by installing Docker Engine instead of Docker Desktop.

I used the Dockerfile at swift-docker/swift-ci/main/ubunut/24.04/Dockerfile and a docker run invocation exactly as specified in the swift-docker repo README.

For this build, a Python script complains: "file not found."

> Traceback (most recent call last):
>   File "/home/build-user/swift/benchmark/scripts/build_script_helper.py", line 61, in <module>
>     main()
>   File "/home/build-user/swift/benchmark/scripts/build_script_helper.py", line 55, in main
>     perform_build(args, swiftbuild_path, "debug", "Benchmark_Onone", "-Onone")
>   File "/home/build-user/swift/benchmark/scripts/build_script_helper.py", line 37, in perform_build
>     shutil.copy(binpath, finalpath)
>   File "/usr/lib/python3.12/shutil.py", line 435, in copy
>     copyfile(src, dst, follow_symlinks=follow_symlinks)
>   File "/usr/lib/python3.12/shutil.py", line 260, in copyfile
>     with open(src, 'rb') as fsrc:
>          ^^^^^^^^^^^^^^^
> FileNotFoundError: [Errno 2] No such file or directory: '/home/build-user/build/buildbot_linux/benchmarks-linux-x86_64/Benchmark_Onone/debug/SwiftBench'

The second time I built, I used the same Dockerfile with the Swift version changed to 6.1 (here's a copy at gist: Dockerfile), and this time wasmkit complains:

> --- Building wasmkit ---
> + env DEBIAN_FRONTEND=noninteractive HOME=/home/build-user HOSTNAME=e9fe79ef6100 LC_CTYPE=C.UTF-8 'NINJA_STATUS=[%f/%t][%p][%es] ' PATH=/opt/swift/6.1/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PWD=/home/build-user SHLVL=0 SWIFTCI_USE_LOCAL_DEPS=1 SWIFT_BRANCH=swift-6.1-release SWIFT_PLATFORM=ubuntu24.04 SWIFT_PREFIX=/opt/swift/6.1 SWIFT_TAG=swift-6.1-RELEASE SWIFT_VERSION=6.1 SWIFT_WEBROOT=https://download.swift.org TOOLCHAINS=default XDG_CACHE_HOME=/home/build-user/build/buildbot_linux/.cache _=./swift/utils/build-script /opt/swift/6.1/usr/bin/swift-build --product wasmkit-cli --package-path /home/build-user/wasmkit --build-path /home/build-user/build/buildbot_linux/wasmkit-linux-x86_64 --configuration release
> error: 'wasmkit': package 'wasmkit' is using Swift tools version 999.0.0 but the installed version is 6.1.0
> error: 'wasmkit': package 'wasmkit' is using Swift tools version 999.0.0 but the installed version is 6.1.0
> ERROR: command `['/opt/swift/6.1/usr/bin/swift-build', '--product', 'wasmkit-cli', '--package-path', '/home/build-user/wasmkit', '--build-path', '/home/build-user/build/buildbot_linux/wasmkit-linux-x86_64', '--configuration', 'release']` terminated with a non-zero exit status 1, aborting

It seems that a "version 999.0.0" is there on purpose to force evaluation of some predicate false and prevent something from happening.

At the end of it all, can someone point me to a Dockerfile for Ubuntu 24.04 that works, or an off-the-shelf Docker image so that I can hasten my way to having a working build?

7

This is happening because Bump WasmKit in `update-checkout-config.json` to 0.1.5 by MaxDesiatov · Pull Request #80820 · swiftlang/swift · GitHub hasn't been merged. cc @mishal_shah @bnbarham

1 Like