5.8 release build failure on Linux

sochot · May 19, 2023, 3:14am

Hi, to whom it may concern, or whoever can help, here is where the 5.8 Release code fails to build on aarch64 Linux:

-- Building Swift Remote Mirror for SDKs: LINUX

-- Found Python3: /usr/bin/python3.9 (found version "3.9.2") found components: Interpreter
-- Found LibEdit: /usr/include (found version ".")
-- Looking for el_wgets
-- Looking for el_wgets - found
-- Looking for wait4
-- Looking for wait4 - found
-- Looking for proc_pid_rusage
-- Looking for proc_pid_rusage - not found
-- Performing Test is_supported
-- Performing Test is_supported - Success
-- Performing Test CXX_SUPPORTS_FORMAT_NONLITERAL_WARNING
-- Performing Test CXX_SUPPORTS_FORMAT_NONLITERAL_WARNING - Success
-- Performing Test CXX_SUPPORTS_GLOBAL_CONSTRUCTORS_WARNING
-- Performing Test CXX_SUPPORTS_GLOBAL_CONSTRUCTORS_WARNING - Success
-- Performing Test CXX_SUPPORTS_EXIT_TIME_DESTRUCTORS_WARNING
-- Performing Test CXX_SUPPORTS_EXIT_TIME_DESTRUCTORS_WARNING - Success
-- Using Experimental String Processing library for _RegexParser (/home/swiftuser/swift-source/swift-experimental-string-processing).
-- Using Experimental String Processing library for _StringProcessing (/home/swiftuser/swift-source/swift-experimental-string-processing).
-- Using Experimental String Processing library for RegexBuilder (/home/swiftuser/swift-source/swift-experimental-string-processing).
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.2")
-- Found UUID: /usr/include
-- Found Git: /usr/bin/git (found version "2.30.2")
-- Swift version: 5.9
-- Swift vendor:
-- Swift compiler version:
-- Embedded clang compiler version:
-- Using Experimental String Processing library for libswift _RegexParser (/home/swiftuser/swift-source/swift-experimental-string-processing).
-- Looking for include file xpc/xpc.h
-- Looking for include file xpc/xpc.h - not found
-- Copying over early SwiftDriver executable.
-- From: /home/swiftuser/swift-source/build/buildbot_linux/swift-linux-aarch64/../earlyswiftdriver-linux-aarch64/release/bin/swift-driver
-- To: /home/swiftuser/swift-source/build/buildbot_linux/swift-linux-aarch64/./bin/swift-driver
-- From: /home/swiftuser/swift-source/build/buildbot_linux/swift-linux-aarch64/../earlyswiftdriver-linux-aarch64/release/bin/swift-help
-- To: /home/swiftuser/swift-source/build/buildbot_linux/swift-linux-aarch64/./bin/swift-help
CMake Error at cmake/modules/AddSwift.cmake:981 (llvm_add_tool_symlink):
Unknown CMake command "llvm_add_tool_symlink".
Call Stack (most recent call first):
tools/driver/CMakeLists.txt:157 (add_swift_tool_symlink)

-- Configuring incomplete, errors occurred!
See also "/home/swiftuser/swift-source/build/buildbot_linux/swift-linux-aarch64/CMakeFiles/CMakeOutput.log".
See also "/home/swiftuser/swift-source/build/buildbot_linux/swift-linux-aarch64/CMakeFiles/CMakeError.log".
ERROR: command terminated with a non-zero exit status 1, aborting

I deleted swift-source and unfortunately I didn't keep the Cmake logs.

Finagolfin · May 19, 2023, 8:05am

We would need more info on how you invoked this build. Looking at your CMake log, there should have been a /home/swiftuser/swift-source/llvm-project/ source directory and a /home/swiftuser/swift-source/build/buildbot_linux/llvm-linux-aarch64/ build directory as part of your build, where the Swift build can find LLVM CMake config files that it needs.

It is failing because it cannot find the CMake command llvm_add_tool_symlink() from the LLVM CMake config, so something has gone wrong that your Swift CMake config cannot find that required LLVM CMake config.

Max_Desiatov · May 19, 2023, 9:56am

Could you provide more information on how exactly you're trying to build? Are you utilizing build-script? Do you have all of the sources checked out for the correct branch scheme with update-checkout? What arguments are you passing to build-script for building?

sochot · May 19, 2023, 5:33pm

Here is my build script:

#!/bin/bash
mkdir swift-source
cd swift-source
git clone https://github.com/apple/swift.git
pushd swift
git checkout swift-5.8-RELEASE
popd
./swift/utils/update-checkout --clone
time nice swift/utils/build-script -j2 --preset=buildbot_linux,no_test \
install_destdir=~/swift-install installable_package=~/swift.tgz

Max_Desiatov · May 19, 2023, 5:37pm

Can you also clarify which exact Linux distribution and its version you are building on?

sochot · May 19, 2023, 5:39pm

64-bit Debian aarch64. It's Raspberry Pi OS.
If I recall, only Ubuntu is officially supported but I'd never use that.

Max_Desiatov · May 19, 2023, 5:42pm

LTS versions of Ubuntu, UBI9, CentOS 7, and Amazon Linux 2 are supported. Other distributions are not tested on CI and aren't guaranteed to work. If you're willing to submit a PR with a fix for a distribution of your choice not on this list, I personally would be happy to review it.

sochot · May 19, 2023, 5:54pm

I'm already in the guinea pig role so yes, if I can get it to work, I can submit a PR. Question is though, is there anything about my above bash script that stands out as being wrong? I used the same script months ago to build Swift successfully.

Max_Desiatov · May 19, 2023, 5:59pm

./swift/utils/update-checkout --clone will clone everything off the main branch, you should pass an additional --tag swift-5.8-RELEASE option to make sure all repositories are on corresponding tags.

sochot · May 19, 2023, 6:11pm

Ok thanks. I'll post again a couple days when the build nears completion ;-), or whenever it fails.

futurejones · May 20, 2023, 1:51am

Hi @sochot ,
If you are looking for prebuilt packages of Swift for Raspberry Pi OS (Debian Bullseye) it is available in the Swift Community Apt Repository

sochot · May 21, 2023, 4:27pm

It's more interesting if I compile it myself. Did they compile it themselves or are they repackaging the Ubuntu package for Debian?

sochot · May 22, 2023, 7:48pm

Here's some build failure output. swiftc build failure output - Pastebin.com

What you'll see there is the second build after a failed build.
I did a build, which took 2 days on the Raspberry Pi 4 with 8GB RAM.
It failed with lots of errors about redefined symbols etc., so I re-ran the build.
The output is from that that 2nd build attempt.

Finagolfin · May 22, 2023, 8:51pm

Known issue when compiling on linux with another Swift compiler in your PATH, uninstalling the prebuilt system Swift compiler should get this error to disappear. Building the Swift toolchain on linux doesn't require a prebuilt Swift compiler yet, so uninstalling it won't break anything.

Max_Desiatov · May 22, 2023, 10:17pm

+1 to having a clean environment. I agree that not having previous Swift installations on Linux makes things easier.

Additionally, I can confirm that the main branch with buildbot_linux,no_test preset builds fine on Debian Bullseye on aarch64 for me. The list of apt packages installed is the same as for Ubuntu 22.04 Jammy, except their exact versions of course. I'll try to build 5.9 soon, but if there's indeed a breakage in 5.8 for Debian, I'm not sure if a merge window is still open for another 5.8.x release.

sochot · May 23, 2023, 12:06pm

Ah that might be it. I've got 5.9dev in my path. OK I'll fix that and retry. Thanks.

sochot · May 23, 2023, 12:10pm

Is there a plan to require swiftc in future? That could cause problems, don't you think?

If a buggy compiler is released, it could prevent new compiler builds.
If a state actor installs a backdoor in a compiler, it could perpetuate itself in each new compiler.

Max_Desiatov · May 23, 2023, 3:01pm

These concerns are just as applicable to clang as they're applicable to swiftc. In fact, actions to mitigate these would be similar if not the same for both compilers.

On topic, I can confirm that the release/5.8 branch scheme builds successfully on aarch64 on Debian Bullseye, I've used this Docker image:

FROM debian:bullseye

RUN groupadd -g 998 build-user && \
    useradd -m -r -u 998 -g build-user build-user

ENV DEBIAN_FRONTEND="noninteractive"

RUN apt-get -y update && apt-get -y install \
  build-essential       \
  clang                 \
  cmake                 \
  git                   \
  icu-devtools          \
  libcurl4-openssl-dev  \
  libedit-dev           \
  libicu-dev            \
  libncurses5-dev       \
  libpython3-dev        \
  libsqlite3-dev        \
  libxml2-dev           \
  ninja-build           \
  pkg-config            \
  python2               \
  python-six            \
  python2-dev           \
  python3-six           \
  python3-distutils     \
  rsync                 \
  swig                  \
  systemtap-sdt-dev     \
  tzdata                \
  uuid-dev              \
  zip

USER build-user

WORKDIR /home/build-user

RUN git clone https://github.com/apple/swift.git
RUN ./swift/utils/update-checkout --clone --scheme release/5.8

RUN ./swift/utils/build-script --preset buildbot_linux,no_test \
    install_destdir=/home/build-user/swift-install \
    installable_package=/home/build-user/swift-5.8-on-debian-bullseye.tar.gz

Finagolfin · May 24, 2023, 3:50am

Yes.

Mostly only when bringing up new platforms, which would require cross-compiling the stdlib and compiler initially.

Unlikely, particularly since it's a multi-stage process planned.

Why would this only be an issue when bootstrapping in Swift: you're saying this because you'd rely on someone else's prebuilt compiler that you didn't build from source yourself? Possible, but there exist other pure OSS toolchains now, so you could use one of those other prebuilt toolchains instead.

sochot · May 25, 2023, 7:01pm

Most malware gets onto computers because someone downloads pre-built software that was either malicious by design or had malware added to it e.g. due to a server hack. Therefore ideally, one should build all software from the source code like you see in Gentoo Linux. There are limits to how much software hygiene can be achieved because we have to trust something, which is usually the OS and its toolchain. But even there, a state actor could easily influence any company or an employee at a company to add malware into the toolchain.