What is the current state of getting Swift 6.0 compiling with Amazon Linux 2023 Docker images?

Server
1

I understand there are many threads scattered throughout this forum as well as the rest of the internet on different issues related to getting latest Swift versions compiling for Amazon Linux 2023 on Docker. The last I heard is that there was a known issue related to the linker. I wanted to have a consolidated and recent thread to discuss what is required to get this to the finish line.

2 Likes
2

Over at Vapor's Penny Bot we've also had so many AmazonLinux2 issues lately, so I'm also eagerly waiting for AmazonLinux 2023 support.
From GitHub Actions actions not working due to outdated Glibc, to weird runtime Swift crashes.

4

It's on my todo list when Lambda Runtime 2.0 beta will be out. I hope to work on this in Aug.
Last time I tried, the update-checkout script does not fetch all the dependencies, which caused the build to fail (yams and many other were missing)
But I have to dive into the root cause.

If someone has time to help on this, all help is welcome !

The high level steps are

#!/bin/bash

## Install and Remove conflicting dependencies 

sudo dnf install python3-pip -y
sudo dnf remove python3-requests python-urllib3 -y

## Install Swift build dependencies
sudo dnf -y install     \
  clang            \
  cmake            \
  curl-devel       \
  gcc-c++          \
  git              \
  glibc-static     \
  libbsd-devel     \
  libedit-devel    \
  libicu-devel     \
  libuuid-devel    \
  libxml2-devel    \
  ncurses-devel    \
  ninja-build      \
  python3-pexpect  \
  pkgconfig        \
  procps-ng        \
  python           \
  python3-devel    \
  python3-six      \
  python3-psutil   \
  python3-pkgconfig \
  rsync            \
  sqlite-devel     \
  swig             \
  tzdata           \
  unzip            \
  uuid-devel       \
  wget             \
  which            \
  zip              \
  tar

## Install Swift 5.10 to bootstrap compilation of Swift 6.0 
ARCH_NAME="$(rpm --eval '%{_arch}')"
case "${ARCH_NAME##*-}" in 
    'x86_64') 
        OS_ARCH_SUFFIX='' 
        ;;
    'aarch64') 
        OS_ARCH_SUFFIX='-aarch64'
        ;;
    *) echo >&2 "error: unsupported architecture: '$ARCH_NAME'"; exit 1 ;;
esac

SWIFT_PLATFORM=amazonlinux2 
SWIFT_BRANCH=swift-5.10-release
SWIFT_VERSION=swift-5.10-RELEASE
SWIFT_WEBROOT=https://download.swift.org   
SWIFT_WEBDIR="$SWIFT_WEBROOT/$SWIFT_BRANCH/$(echo $SWIFT_PLATFORM | tr -d .)$OS_ARCH_SUFFIX"
SWIFT_BIN_URL="$SWIFT_WEBDIR/$SWIFT_VERSION/$SWIFT_VERSION-$SWIFT_PLATFORM$OS_ARCH_SUFFIX.tar.gz"
wget $SWIFT_BIN_URL
tar xfvz $SWIFT_VERSION-$SWIFT_PLATFORM$OS_ARCH_SUFFIX.tar.gz
export PATH=/home/ec2-user/$SWIFT_VERSION-$SWIFT_PLATFORM$OS_ARCH_SUFFIX/usr/bin/:$PATH

echo Using Swift version $(swift --version)

## Install LD GOLD 
## This is used by Swift 5.10 and not available by default on Amazon Linux 2023
## Swift 6 selects the ldd linker by default on Amazon Linux 2023,
## so we just need ld.gold to build Swift 6, not at runtime
## https://github.com/swiftlang/swift/pull/72049
sudo dnf install gmp-devel mpfr-devel texinfo bison git gcc-c++ -y
mkdir ld.gold && cd ld.gold
git clone --depth 1 git://sourceware.org/git/binutils-gdb.git binutils
mkdir build && cd build
../binutils/configure --enable-gold --enable-plugins --disable-werror
make all-gold
cd gold
make all-am
cd ..
cp gold/ld-new /usr/bin/ld.gold
cd ~
/usr/bin/ld.gold -v

mkdir swift-project && cd swift-project   
git clone https://github.com/apple/swift.git swift
./swift/utils/update-checkout --clone --all-repositories

## Create an EBS snapshot or an AMI at this point because the above takes ~30 minutes

## Build Instructions 

## Specify the release you want to build
## IMPORTANT : to build a version >= 5.9, you must install Swift 5.8 or later and make sure `swift` is in the PATH
SWIFT_VERSION=6.1
./swift/utils/update-checkout --scheme release/$SWIFT_VERSION
DIST_DIR=$HOME/dist 
mkdir $DIST_DIR

## Building 
# simple build
# ./swift/utils/build-script --release-debuginfo --skip-early-swift-driver

# or 

# build and package
# ./swift/utils/build-script --preset=buildbot_linux,no_assertions,no_test install_destdir=$DIST_DIR installable_package=$DIST_DIR/swift-${SWIFT_VERSION}-amazonlinux2023.tar.gz

# Build the toolchain
export BUNDLE_PREFIX=com.amazon.aws
./swift/utils/build-toolchain $BUNDLE_PREFIX

## Cleaning in between builds
rm -rf ./build
rm -rf $DIST_DIR
1 Like
5

Once we will manage to compile and use the result of the compilation on a fresh machine, there are two next steps, that will involve folks at Apple

1/ recreate the build procedure to create a docker image and submit a PR here

2/ Have the build added to Apple CI and Swift's web site. (no idea how to do that :-), but we're not there yet )

2 Likes