Amazon linux 2 is going to be superseded by Amazon linux 2023 (AL2023). but all of the swift docker images for Amazon linux are still for AL2. when will we get docker images for AL2023?
It's being worked on
7 Likes
has there been any progress in this area?
I'm also curious about this, any update would be nice. Thank you. :)
I was able to build Swift on Amazon Linux 2023 using the following Dockerfile, maybe it's a good startingpoint, I know it's a hack, but still it works... 
FROM amazonlinux:2023
RUN dnf -y install binutils gcc git unzip glibc-static gzip libbsd libcurl-devel libedit libicu libstdc++-static libuuid libxml2-devel tar tzdata zlib-devel sqlite-libs
RUN dnf -y install dirmngr --allowerasing
ARG SWIFT_SIGNING_KEY=A62AE125BBBFBB96A6E042EC925CC1CCED3D1561
ARG SWIFT_PLATFORM=amazonlinux2
ARG SWIFT_BRANCH=swift-5.9.1-release
ARG SWIFT_VERSION=swift-5.9.1-RELEASE
ARG SWIFT_WEBROOT=https://download.swift.org
ENV SWIFT_SIGNING_KEY=$SWIFT_SIGNING_KEY \
SWIFT_PLATFORM=$SWIFT_PLATFORM \
SWIFT_BRANCH=$SWIFT_BRANCH \
SWIFT_VERSION=$SWIFT_VERSION \
SWIFT_WEBROOT=$SWIFT_WEBROOT
RUN set -e; \
ARCH_NAME="$(uname -m)"; \
url=; \
case "${ARCH_NAME##*-}" in \
'x86_64') \
OS_ARCH_SUFFIX=''; \
;; \
'aarch64') \
OS_ARCH_SUFFIX='-aarch64'; \
;; \
*) echo >&2 "error: unsupported architecture: '$ARCH_NAME'"; exit 1 ;; \
esac; \
SWIFT_WEBDIR="$SWIFT_WEBROOT/$SWIFT_BRANCH/$(echo $SWIFT_PLATFORM | tr -d .)$OS_ARCH_SUFFIX" \
&& SWIFT_BIN_URL="$SWIFT_WEBDIR/$SWIFT_VERSION/$SWIFT_VERSION-$SWIFT_PLATFORM$OS_ARCH_SUFFIX.tar.gz" \
&& SWIFT_SIG_URL="$SWIFT_BIN_URL.sig" \
&& echo $SWIFT_BIN_URL \
&& export GNUPGHOME="$(mktemp -d)" \
&& curl -fsSL "$SWIFT_BIN_URL" -o swift.tar.gz "$SWIFT_SIG_URL" -o swift.tar.gz.sig \
&& gpg --batch --quiet --keyserver keyserver.ubuntu.com --recv-keys "$SWIFT_SIGNING_KEY" \
&& gpg --batch --verify swift.tar.gz.sig swift.tar.gz \
&& tar -xzf swift.tar.gz --directory / --strip-components=1 \
&& chmod -R o+r /usr/lib/swift \
&& rm -rf "$GNUPGHOME" swift.tar.gz.sig swift.tar.gz
RUN swift --version
1 Like
as of today, VSCode stopped supporting Glibc 2.26, which means VSCode no longer supports devcontainers with Amazon Linux 2022.
for most other language communities, this is a nonevent, as almost everyone using languages that are not called swift should have upgraded to Amazon Linux 2023 long ago.
this doesn’t actually build an Amazon Linux 2023 toolchain, it just installs an Amazon Linux 2 toolchain on an Amazon Linux 2023 host.
with @Finagolfin ’s indispensable help, i was able to modify @tiborbodecs ’s dockerfile to create a dockerfile that builds a 5.10 toolchain for Amazon Linux 2023 by bootstrapping from a 5.9.2 compiler.
FROM amazonlinux:2023
RUN yum -y update
# install sysadmin basics
RUN yum -y install sudo passwd
# install swift dependencies
RUN yum install shadow-utils -y
RUN yum -y group install "development tools"
RUN yum -y install \
cmake \
curl-devel \
git \
glibc-static \
libbsd-devel \
libedit-devel \
libicu-devel \
libuuid-devel \
libxml2-devel \
ncurses-devel \
pkgconfig \
procps-ng \
python \
python-devel \
python-pkgconfig \
python-six \
python3-devel \
python3-psutil \
rsync \
sqlite-devel \
swig \
tzdata \
unzip \
uuid-devel \
wget \
which \
zip
RUN mkdir -p /usr/local/lib/python3.7/site-packages/
ARG SWIFT_PLATFORM=amazonlinux2
ARG SWIFT_VERSION=5.9.2
ARG SWIFT_BRANCH=swift-${SWIFT_VERSION}-release
ARG SWIFT_TAG=swift-${SWIFT_VERSION}-RELEASE
ARG SWIFT_WEBROOT=https://download.swift.org
ARG SWIFT_PREFIX=/opt/swift/${SWIFT_VERSION}
ENV SWIFT_PLATFORM=$SWIFT_PLATFORM \
SWIFT_VERSION=$SWIFT_VERSION \
SWIFT_BRANCH=$SWIFT_BRANCH \
SWIFT_TAG=$SWIFT_TAG \
SWIFT_WEBROOT=$SWIFT_WEBROOT \
SWIFT_PREFIX=$SWIFT_PREFIX
RUN dnf -y install dirmngr --allowerasing
RUN dnf swap gnupg2-minimal gnupg2-full
RUN set -e; \
ARCH_NAME="$(rpm --eval '%{_arch}')"; \
url=; \
case "${ARCH_NAME##*-}" in \
'x86_64') \
OS_ARCH_SUFFIX=''; \
;; \
'aarch64') \
OS_ARCH_SUFFIX='-aarch64'; \
;; \
*) echo >&2 "error: unsupported architecture: '$ARCH_NAME'"; exit 1 ;; \
esac; \
SWIFT_WEBDIR="$SWIFT_WEBROOT/$SWIFT_BRANCH/$(echo $SWIFT_PLATFORM | tr -d .)$OS_ARCH_SUFFIX" \
&& SWIFT_BIN_URL="$SWIFT_WEBDIR/$SWIFT_TAG/$SWIFT_TAG-$SWIFT_PLATFORM$OS_ARCH_SUFFIX.tar.gz" \
&& SWIFT_SIG_URL="$SWIFT_BIN_URL.sig" \
&& echo $SWIFT_BIN_URL \
# - Download the GPG keys, Swift toolchain, and toolchain signature, and verify.
&& export GNUPGHOME="$(mktemp -d)" \
&& curl -fsSL "$SWIFT_BIN_URL" -o swift.tar.gz "$SWIFT_SIG_URL" -o swift.tar.gz.sig \
&& curl -fSsL https://swift.org/keys/all-keys.asc | gpg --import - \
&& gpg --batch --verify swift.tar.gz.sig swift.tar.gz \
# - Unpack the toolchain, set libs permissions, and clean up.
&& mkdir -p $SWIFT_PREFIX \
&& tar -xzf swift.tar.gz --directory $SWIFT_PREFIX --strip-components=1 \
&& chmod -R o+r $SWIFT_PREFIX/usr/lib/swift \
&& rm -rf "$GNUPGHOME" swift.tar.gz.sig swift.tar.gz
ENV PATH="${SWIFT_PREFIX}/usr/bin:${PATH}"
RUN yum install -y gcc-c++
RUN sudo yum install -y libmpc-devel
RUN sudo yum install -y texinfo
RUN git clone --depth 1 git://sourceware.org/git/binutils-gdb.git binutils
RUN mkdir binutils.build
RUN cd binutils.build
RUN ../binutils/configure --enable-gold --enable-plugins --disable-werror
RUN make all-gold
RUN mv gold/ld-new /usr/bin/ld.gold
RUN cd ..
# create the `ec2-user`, and switch to her
RUN useradd -ms /bin/bash ec2-user
RUN passwd -d ec2-user
RUN usermod -aG wheel ec2-user
USER ec2-user
WORKDIR /home/ec2-user/
ENV SWIFT_BRANCH_CHECKOUT=release/5.10
RUN git clone --depth 1 --branch $SWIFT_BRANCH_CHECKOUT https://github.com/apple/swift
WORKDIR /home/ec2-user/swift
RUN utils/update-checkout --clone --scheme $SWIFT_BRANCH_CHECKOUT
COPY preset.ini build-preset-ext.ini
RUN cat build-preset-ext.ini >> utils/build-presets.ini
RUN utils/build-script \
--preset buildbot_linux_amazon_linux_2023 \
install_destdir=/home/ec2-user/swift-install \
installable_package=/home/ec2-user/swift-DEVELOPMENT-SNAPSHOT-$(date +'%F')-a.tar.gz
# cleanup build scaffolding
RUN sudo cp /home/ec2-user/swift-DEVELOPMENT-SNAPSHOT-* /
RUN sudo rm -r /opt/swift
RUN sudo rm -r /binutils
RUN sudo rm -r /binutils.build
RUN sudo rm -rf ~/*
# generate script that will run on terminal creation,
# enables showing the PWD prompt
RUN echo "PS1='\w\$ '" >> .bashrc
RUN echo "force_color_prompt=yes" >> .bashrc
ENV TERM xterm-256color
CMD sleep infinity
although i was able to build this image, i could not actually push it anywhere as the layers are over 28 GB in total.
I didn't try this for this particular case, but I think you should be able to use a multi-stage build to avoid the intermediate layers.
1 Like
ha, i had no idea that feature existed, i admit i am still very much a docker novice. i managed to push a stripped down version of the toolchain i built the other day to tayloraswift/nightly-5.10-amazonlinux2023 if anyone wants to try it out.
here’s the updated dockerfile, which i am sure could benefit from some code review:
FROM amazonlinux:2023 as toolchain-build
RUN yum -y update
# install sysadmin basics
RUN yum -y install sudo passwd
# install swift dependencies
RUN yum install shadow-utils -y
RUN yum -y group install "development tools"
RUN yum -y install \
cmake \
curl-devel \
git \
glibc-static \
libbsd-devel \
libedit-devel \
libicu-devel \
libuuid-devel \
libxml2-devel \
ncurses-devel \
pkgconfig \
procps-ng \
python \
python-devel \
python-pkgconfig \
python-six \
python3-devel \
python3-psutil \
rsync \
sqlite-devel \
swig \
tzdata \
unzip \
uuid-devel \
wget \
which \
zip
RUN mkdir -p /usr/local/lib/python3.7/site-packages/
ARG SWIFT_PLATFORM=amazonlinux2
ARG SWIFT_VERSION=5.9.2
ARG SWIFT_BRANCH=swift-${SWIFT_VERSION}-release
ARG SWIFT_TAG=swift-${SWIFT_VERSION}-RELEASE
ARG SWIFT_WEBROOT=https://download.swift.org
ARG SWIFT_PREFIX=/opt/swift/${SWIFT_VERSION}
ENV SWIFT_PLATFORM=$SWIFT_PLATFORM \
SWIFT_VERSION=$SWIFT_VERSION \
SWIFT_BRANCH=$SWIFT_BRANCH \
SWIFT_TAG=$SWIFT_TAG \
SWIFT_WEBROOT=$SWIFT_WEBROOT \
SWIFT_PREFIX=$SWIFT_PREFIX
RUN dnf -y install dirmngr --allowerasing
RUN dnf swap gnupg2-minimal gnupg2-full
RUN set -e; \
ARCH_NAME="$(rpm --eval '%{_arch}')"; \
url=; \
case "${ARCH_NAME##*-}" in \
'x86_64') \
OS_ARCH_SUFFIX=''; \
;; \
'aarch64') \
OS_ARCH_SUFFIX='-aarch64'; \
;; \
*) echo >&2 "error: unsupported architecture: '$ARCH_NAME'"; exit 1 ;; \
esac; \
SWIFT_WEBDIR="$SWIFT_WEBROOT/$SWIFT_BRANCH/$(echo $SWIFT_PLATFORM | tr -d .)$OS_ARCH_SUFFIX" \
&& SWIFT_BIN_URL="$SWIFT_WEBDIR/$SWIFT_TAG/$SWIFT_TAG-$SWIFT_PLATFORM$OS_ARCH_SUFFIX.tar.gz" \
&& SWIFT_SIG_URL="$SWIFT_BIN_URL.sig" \
&& echo $SWIFT_BIN_URL \
# - Download the GPG keys, Swift toolchain, and toolchain signature, and verify.
&& export GNUPGHOME="$(mktemp -d)" \
&& curl -fsSL "$SWIFT_BIN_URL" -o swift.tar.gz "$SWIFT_SIG_URL" -o swift.tar.gz.sig \
&& curl -fSsL https://swift.org/keys/all-keys.asc | gpg --import - \
&& gpg --batch --verify swift.tar.gz.sig swift.tar.gz \
# - Unpack the toolchain, set libs permissions, and clean up.
&& mkdir -p $SWIFT_PREFIX \
&& tar -xzf swift.tar.gz --directory $SWIFT_PREFIX --strip-components=1 \
&& chmod -R o+r $SWIFT_PREFIX/usr/lib/swift \
&& rm -rf "$GNUPGHOME" swift.tar.gz.sig swift.tar.gz
ENV PATH="${SWIFT_PREFIX}/usr/bin:${PATH}"
RUN yum install -y gcc-c++
RUN sudo yum install -y libmpc-devel
RUN sudo yum install -y texinfo
RUN git clone --depth 1 git://sourceware.org/git/binutils-gdb.git binutils
RUN mkdir binutils.build
RUN cd binutils.build
RUN ../binutils/configure --enable-gold --enable-plugins --disable-werror
RUN make all-gold
RUN mv gold/ld-new /usr/bin/ld.gold
RUN cd ..
# create the `ec2-user`, and switch to her
RUN useradd -ms /bin/bash ec2-user
RUN passwd -d ec2-user
RUN usermod -aG wheel ec2-user
USER ec2-user
WORKDIR /home/ec2-user/
ENV SWIFT_BRANCH_CHECKOUT=release/5.10
RUN git clone --depth 1 --branch $SWIFT_BRANCH_CHECKOUT https://github.com/apple/swift
WORKDIR /home/ec2-user/swift
RUN utils/update-checkout --clone --scheme $SWIFT_BRANCH_CHECKOUT
COPY preset.ini build-preset-ext.ini
RUN cat build-preset-ext.ini >> utils/build-presets.ini
RUN utils/build-script \
--preset buildbot_linux_amazon_linux_2023 \
install_destdir=/home/ec2-user/swift-install \
installable_package=/home/ec2-user/swift-DEVELOPMENT-SNAPSHOT-$(date +'%F')-a.tar.gz
FROM amazonlinux:2023
COPY --from=toolchain-build /home/ec2-user/swift-install /swift-install
RUN cp -r /swift-install/usr/bin/* /usr/bin
RUN cp -r /swift-install/usr/include/* /usr/include/
RUN cp -r /swift-install/usr/libexec/* /usr/libexec/
RUN cp -r /swift-install/usr/lib/* /usr/lib/
RUN cp -r /swift-install/usr/local/* /usr/local
RUN cp -r /swift-install/usr/share/* /usr/share
RUN yum -y update
# install sysadmin basics
RUN yum -y install sudo passwd
# install swift dependencies
RUN yum install shadow-utils -y
RUN yum -y group install "development tools"
RUN yum -y install \
cmake \
curl-devel \
git \
glibc-static \
libbsd-devel \
libedit-devel \
libicu-devel \
libuuid-devel \
libxml2-devel \
ncurses-devel \
pkgconfig \
procps-ng \
python \
python-devel \
python-pkgconfig \
python-six \
python3-devel \
python3-psutil \
rsync \
sqlite-devel \
swig \
tzdata \
unzip \
uuid-devel \
wget \
which \
zip
# create the `ec2-user`, and switch to her
RUN useradd -ms /bin/bash ec2-user
RUN passwd -d ec2-user
RUN usermod -aG wheel ec2-user
USER ec2-user
WORKDIR /home/ec2-user/
# generate script that will run on terminal creation,
# enables showing the PWD prompt
RUN echo "PS1='\w\$ '" >> .bashrc
RUN echo "force_color_prompt=yes" >> .bashrc
ENV TERM xterm-256color
CMD sleep infinity
i suppose the only remaining obstacle for me is: how do i map the version of this toolchain back to something compiler developers would understand? as far as i can tell, i built the toolchain at a completely arbitrary commit, with completely arbitrary versions of its dependencies, and when i run swift --version i get something like:
$ swift --version
Swift version 5.10-dev (LLVM dbfaba0078e9380, Swift 63c8b551eb2f613)
Target: x86_64-unknown-linux-gnu
which will not bode well for reporting/reproducing bugs in the compiler.
GitHub has made some changes and you can no longer checkout a repository using the "checkout" action in an swift:amazonlinux2 container since it requires a different version of GlibC.
I'm sure i'll find a workaround, but suboptimal ...
/usr/bin/docker exec ff16e801308479c02bef5bb517e52d428b497ba3d073bc3ceffdf090fd6ee3d4 sh -c "cat /etc/*release | grep ^ID"
/__e/node20/bin/node: /lib64/libm.so.6: versionGLIBC_2.27' not found (required by /__e/node20/bin/node) /__e/node20/bin/node: /lib64/libc.so.6: versionGLIBC_2.28' not found (required by /__e/node20/bin/node)
EDIT: switched to manual git-clone-ing instead of using actions/checkout.
2 Likes