Amazon linux 2023 support?

Amazon linux 2 is going to be superseded by Amazon linux 2023 (AL2023). but all of the swift docker images for Amazon linux are still for AL2. when will we get docker images for AL2023?

It's being worked on

7 Likes

has there been any progress in this area?

I'm also curious about this, any update would be nice. Thank you. :)

I was able to build Swift on Amazon Linux 2023 using the following Dockerfile, maybe it's a good startingpoint, I know it's a hack, but still it works... :sweat_smile:

FROM amazonlinux:2023

RUN dnf -y install binutils gcc git unzip glibc-static gzip libbsd libcurl-devel libedit libicu libstdc++-static libuuid libxml2-devel tar tzdata zlib-devel sqlite-libs 

RUN dnf -y install dirmngr --allowerasing

ARG SWIFT_SIGNING_KEY=A62AE125BBBFBB96A6E042EC925CC1CCED3D1561
ARG SWIFT_PLATFORM=amazonlinux2
ARG SWIFT_BRANCH=swift-5.9.1-release
ARG SWIFT_VERSION=swift-5.9.1-RELEASE
ARG SWIFT_WEBROOT=https://download.swift.org

ENV SWIFT_SIGNING_KEY=$SWIFT_SIGNING_KEY \
    SWIFT_PLATFORM=$SWIFT_PLATFORM \
    SWIFT_BRANCH=$SWIFT_BRANCH \
    SWIFT_VERSION=$SWIFT_VERSION \
    SWIFT_WEBROOT=$SWIFT_WEBROOT

RUN set -e; \
    ARCH_NAME="$(uname -m)"; \
    url=; \
    case "${ARCH_NAME##*-}" in \
        'x86_64') \
            OS_ARCH_SUFFIX=''; \
            ;; \
        'aarch64') \
            OS_ARCH_SUFFIX='-aarch64'; \
            ;; \
        *) echo >&2 "error: unsupported architecture: '$ARCH_NAME'"; exit 1 ;; \
    esac; \
    SWIFT_WEBDIR="$SWIFT_WEBROOT/$SWIFT_BRANCH/$(echo $SWIFT_PLATFORM | tr -d .)$OS_ARCH_SUFFIX" \
    && SWIFT_BIN_URL="$SWIFT_WEBDIR/$SWIFT_VERSION/$SWIFT_VERSION-$SWIFT_PLATFORM$OS_ARCH_SUFFIX.tar.gz" \
    && SWIFT_SIG_URL="$SWIFT_BIN_URL.sig" \
    && echo $SWIFT_BIN_URL \
    && export GNUPGHOME="$(mktemp -d)" \
    && curl -fsSL "$SWIFT_BIN_URL" -o swift.tar.gz "$SWIFT_SIG_URL" -o swift.tar.gz.sig \
    && gpg --batch --quiet --keyserver keyserver.ubuntu.com --recv-keys "$SWIFT_SIGNING_KEY" \
    && gpg --batch --verify swift.tar.gz.sig swift.tar.gz \
    && tar -xzf swift.tar.gz --directory / --strip-components=1 \
    && chmod -R o+r /usr/lib/swift \
    && rm -rf "$GNUPGHOME" swift.tar.gz.sig swift.tar.gz

RUN swift --version

1 Like

as of today, VSCode stopped supporting Glibc 2.26, which means VSCode no longer supports devcontainers with Amazon Linux 2022.

for most other language communities, this is a nonevent, as almost everyone using languages that are not called swift should have upgraded to Amazon Linux 2023 long ago.

this doesn’t actually build an Amazon Linux 2023 toolchain, it just installs an Amazon Linux 2 toolchain on an Amazon Linux 2023 host.

with @Finagolfin ’s indispensable help, i was able to modify @tiborbodecs ’s dockerfile to create a dockerfile that builds a 5.10 toolchain for Amazon Linux 2023 by bootstrapping from a 5.9.2 compiler.

FROM amazonlinux:2023

RUN yum -y update
# install sysadmin basics
RUN yum -y install sudo passwd

# install swift dependencies
RUN yum install shadow-utils -y
RUN yum -y group install "development tools"
RUN yum -y install \
  cmake            \
  curl-devel       \
  git              \
  glibc-static     \
  libbsd-devel     \
  libedit-devel    \
  libicu-devel     \
  libuuid-devel    \
  libxml2-devel    \
  ncurses-devel    \
  pkgconfig        \
  procps-ng        \
  python           \
  python-devel     \
  python-pkgconfig \
  python-six       \
  python3-devel    \
  python3-psutil   \
  rsync            \
  sqlite-devel     \
  swig             \
  tzdata           \
  unzip            \
  uuid-devel       \
  wget             \
  which            \
  zip

RUN mkdir -p /usr/local/lib/python3.7/site-packages/

ARG SWIFT_PLATFORM=amazonlinux2
ARG SWIFT_VERSION=5.9.2
ARG SWIFT_BRANCH=swift-${SWIFT_VERSION}-release
ARG SWIFT_TAG=swift-${SWIFT_VERSION}-RELEASE
ARG SWIFT_WEBROOT=https://download.swift.org
ARG SWIFT_PREFIX=/opt/swift/${SWIFT_VERSION}

ENV SWIFT_PLATFORM=$SWIFT_PLATFORM \
    SWIFT_VERSION=$SWIFT_VERSION \
    SWIFT_BRANCH=$SWIFT_BRANCH \
    SWIFT_TAG=$SWIFT_TAG \
    SWIFT_WEBROOT=$SWIFT_WEBROOT \
    SWIFT_PREFIX=$SWIFT_PREFIX

RUN dnf -y install dirmngr --allowerasing
RUN dnf swap gnupg2-minimal gnupg2-full

RUN set -e; \
    ARCH_NAME="$(rpm --eval '%{_arch}')"; \
    url=; \
    case "${ARCH_NAME##*-}" in \
        'x86_64') \
            OS_ARCH_SUFFIX=''; \
            ;; \
        'aarch64') \
            OS_ARCH_SUFFIX='-aarch64'; \
            ;; \
        *) echo >&2 "error: unsupported architecture: '$ARCH_NAME'"; exit 1 ;; \
    esac; \
    SWIFT_WEBDIR="$SWIFT_WEBROOT/$SWIFT_BRANCH/$(echo $SWIFT_PLATFORM | tr -d .)$OS_ARCH_SUFFIX" \
    && SWIFT_BIN_URL="$SWIFT_WEBDIR/$SWIFT_TAG/$SWIFT_TAG-$SWIFT_PLATFORM$OS_ARCH_SUFFIX.tar.gz" \
    && SWIFT_SIG_URL="$SWIFT_BIN_URL.sig" \
    && echo $SWIFT_BIN_URL \
    # - Download the GPG keys, Swift toolchain, and toolchain signature, and verify.
    && export GNUPGHOME="$(mktemp -d)" \
    && curl -fsSL "$SWIFT_BIN_URL" -o swift.tar.gz "$SWIFT_SIG_URL" -o swift.tar.gz.sig \
    && curl -fSsL https://swift.org/keys/all-keys.asc | gpg --import -  \
    && gpg --batch --verify swift.tar.gz.sig swift.tar.gz \
    # - Unpack the toolchain, set libs permissions, and clean up.
    && mkdir -p $SWIFT_PREFIX \
    && tar -xzf swift.tar.gz --directory $SWIFT_PREFIX --strip-components=1 \
    && chmod -R o+r $SWIFT_PREFIX/usr/lib/swift \
    && rm -rf "$GNUPGHOME" swift.tar.gz.sig swift.tar.gz

ENV PATH="${SWIFT_PREFIX}/usr/bin:${PATH}"

RUN yum install -y gcc-c++

RUN sudo yum install -y libmpc-devel
RUN sudo yum install -y texinfo

RUN git clone --depth 1 git://sourceware.org/git/binutils-gdb.git binutils
RUN mkdir binutils.build
RUN cd binutils.build
RUN ../binutils/configure --enable-gold --enable-plugins --disable-werror
RUN make all-gold
RUN mv gold/ld-new /usr/bin/ld.gold
RUN cd ..

# create the `ec2-user`, and switch to her
RUN useradd -ms /bin/bash ec2-user
RUN passwd -d ec2-user
RUN usermod -aG wheel ec2-user
USER ec2-user

WORKDIR /home/ec2-user/

ENV SWIFT_BRANCH_CHECKOUT=release/5.10

RUN git clone --depth 1 --branch $SWIFT_BRANCH_CHECKOUT https://github.com/apple/swift

WORKDIR /home/ec2-user/swift

RUN utils/update-checkout --clone --scheme $SWIFT_BRANCH_CHECKOUT

COPY preset.ini build-preset-ext.ini
RUN cat build-preset-ext.ini >> utils/build-presets.ini

RUN utils/build-script \
    --preset buildbot_linux_amazon_linux_2023 \
    install_destdir=/home/ec2-user/swift-install \
    installable_package=/home/ec2-user/swift-DEVELOPMENT-SNAPSHOT-$(date +'%F')-a.tar.gz

# cleanup build scaffolding 

RUN sudo cp /home/ec2-user/swift-DEVELOPMENT-SNAPSHOT-* /
RUN sudo rm -r /opt/swift
RUN sudo rm -r /binutils
RUN sudo rm -r /binutils.build
RUN sudo rm -rf ~/*

# generate script that will run on terminal creation,
# enables showing the PWD prompt
RUN echo "PS1='\w\$ '" >> .bashrc
RUN echo "force_color_prompt=yes" >> .bashrc
ENV TERM xterm-256color

CMD sleep infinity

although i was able to build this image, i could not actually push it anywhere as the layers are over 28 GB in total.

I didn't try this for this particular case, but I think you should be able to use a multi-stage build to avoid the intermediate layers.

1 Like

ha, i had no idea that feature existed, i admit i am still very much a docker novice. i managed to push a stripped down version of the toolchain i built the other day to tayloraswift/nightly-5.10-amazonlinux2023 if anyone wants to try it out.

here’s the updated dockerfile, which i am sure could benefit from some code review:

FROM amazonlinux:2023 as toolchain-build

RUN yum -y update
# install sysadmin basics
RUN yum -y install sudo passwd

# install swift dependencies
RUN yum install shadow-utils -y
RUN yum -y group install "development tools"
RUN yum -y install \
  cmake            \
  curl-devel       \
  git              \
  glibc-static     \
  libbsd-devel     \
  libedit-devel    \
  libicu-devel     \
  libuuid-devel    \
  libxml2-devel    \
  ncurses-devel    \
  pkgconfig        \
  procps-ng        \
  python           \
  python-devel     \
  python-pkgconfig \
  python-six       \
  python3-devel    \
  python3-psutil   \
  rsync            \
  sqlite-devel     \
  swig             \
  tzdata           \
  unzip            \
  uuid-devel       \
  wget             \
  which            \
  zip

RUN mkdir -p /usr/local/lib/python3.7/site-packages/

ARG SWIFT_PLATFORM=amazonlinux2
ARG SWIFT_VERSION=5.9.2
ARG SWIFT_BRANCH=swift-${SWIFT_VERSION}-release
ARG SWIFT_TAG=swift-${SWIFT_VERSION}-RELEASE
ARG SWIFT_WEBROOT=https://download.swift.org
ARG SWIFT_PREFIX=/opt/swift/${SWIFT_VERSION}

ENV SWIFT_PLATFORM=$SWIFT_PLATFORM \
    SWIFT_VERSION=$SWIFT_VERSION \
    SWIFT_BRANCH=$SWIFT_BRANCH \
    SWIFT_TAG=$SWIFT_TAG \
    SWIFT_WEBROOT=$SWIFT_WEBROOT \
    SWIFT_PREFIX=$SWIFT_PREFIX

RUN dnf -y install dirmngr --allowerasing
RUN dnf swap gnupg2-minimal gnupg2-full

RUN set -e; \
    ARCH_NAME="$(rpm --eval '%{_arch}')"; \
    url=; \
    case "${ARCH_NAME##*-}" in \
        'x86_64') \
            OS_ARCH_SUFFIX=''; \
            ;; \
        'aarch64') \
            OS_ARCH_SUFFIX='-aarch64'; \
            ;; \
        *) echo >&2 "error: unsupported architecture: '$ARCH_NAME'"; exit 1 ;; \
    esac; \
    SWIFT_WEBDIR="$SWIFT_WEBROOT/$SWIFT_BRANCH/$(echo $SWIFT_PLATFORM | tr -d .)$OS_ARCH_SUFFIX" \
    && SWIFT_BIN_URL="$SWIFT_WEBDIR/$SWIFT_TAG/$SWIFT_TAG-$SWIFT_PLATFORM$OS_ARCH_SUFFIX.tar.gz" \
    && SWIFT_SIG_URL="$SWIFT_BIN_URL.sig" \
    && echo $SWIFT_BIN_URL \
    # - Download the GPG keys, Swift toolchain, and toolchain signature, and verify.
    && export GNUPGHOME="$(mktemp -d)" \
    && curl -fsSL "$SWIFT_BIN_URL" -o swift.tar.gz "$SWIFT_SIG_URL" -o swift.tar.gz.sig \
    && curl -fSsL https://swift.org/keys/all-keys.asc | gpg --import -  \
    && gpg --batch --verify swift.tar.gz.sig swift.tar.gz \
    # - Unpack the toolchain, set libs permissions, and clean up.
    && mkdir -p $SWIFT_PREFIX \
    && tar -xzf swift.tar.gz --directory $SWIFT_PREFIX --strip-components=1 \
    && chmod -R o+r $SWIFT_PREFIX/usr/lib/swift \
    && rm -rf "$GNUPGHOME" swift.tar.gz.sig swift.tar.gz

ENV PATH="${SWIFT_PREFIX}/usr/bin:${PATH}"

RUN yum install -y gcc-c++

RUN sudo yum install -y libmpc-devel
RUN sudo yum install -y texinfo

RUN git clone --depth 1 git://sourceware.org/git/binutils-gdb.git binutils
RUN mkdir binutils.build
RUN cd binutils.build
RUN ../binutils/configure --enable-gold --enable-plugins --disable-werror
RUN make all-gold
RUN mv gold/ld-new /usr/bin/ld.gold
RUN cd ..

# create the `ec2-user`, and switch to her
RUN useradd -ms /bin/bash ec2-user
RUN passwd -d ec2-user
RUN usermod -aG wheel ec2-user
USER ec2-user

WORKDIR /home/ec2-user/

ENV SWIFT_BRANCH_CHECKOUT=release/5.10

RUN git clone --depth 1 --branch $SWIFT_BRANCH_CHECKOUT https://github.com/apple/swift

WORKDIR /home/ec2-user/swift

RUN utils/update-checkout --clone --scheme $SWIFT_BRANCH_CHECKOUT

COPY preset.ini build-preset-ext.ini
RUN cat build-preset-ext.ini >> utils/build-presets.ini

RUN utils/build-script \
    --preset buildbot_linux_amazon_linux_2023 \
    install_destdir=/home/ec2-user/swift-install \
    installable_package=/home/ec2-user/swift-DEVELOPMENT-SNAPSHOT-$(date +'%F')-a.tar.gz



FROM amazonlinux:2023 
COPY --from=toolchain-build /home/ec2-user/swift-install /swift-install

RUN cp -r /swift-install/usr/bin/* /usr/bin
RUN cp -r /swift-install/usr/include/* /usr/include/
RUN cp -r /swift-install/usr/libexec/* /usr/libexec/
RUN cp -r /swift-install/usr/lib/* /usr/lib/
RUN cp -r /swift-install/usr/local/* /usr/local 
RUN cp -r /swift-install/usr/share/* /usr/share

RUN yum -y update
# install sysadmin basics
RUN yum -y install sudo passwd

# install swift dependencies
RUN yum install shadow-utils -y
RUN yum -y group install "development tools"
RUN yum -y install \
  cmake            \
  curl-devel       \
  git              \
  glibc-static     \
  libbsd-devel     \
  libedit-devel    \
  libicu-devel     \
  libuuid-devel    \
  libxml2-devel    \
  ncurses-devel    \
  pkgconfig        \
  procps-ng        \
  python           \
  python-devel     \
  python-pkgconfig \
  python-six       \
  python3-devel    \
  python3-psutil   \
  rsync            \
  sqlite-devel     \
  swig             \
  tzdata           \
  unzip            \
  uuid-devel       \
  wget             \
  which            \
  zip

# create the `ec2-user`, and switch to her
RUN useradd -ms /bin/bash ec2-user
RUN passwd -d ec2-user
RUN usermod -aG wheel ec2-user
USER ec2-user

WORKDIR /home/ec2-user/

# generate script that will run on terminal creation,
# enables showing the PWD prompt
RUN echo "PS1='\w\$ '" >> .bashrc
RUN echo "force_color_prompt=yes" >> .bashrc
ENV TERM xterm-256color

CMD sleep infinity

i suppose the only remaining obstacle for me is: how do i map the version of this toolchain back to something compiler developers would understand? as far as i can tell, i built the toolchain at a completely arbitrary commit, with completely arbitrary versions of its dependencies, and when i run swift --version i get something like:

$ swift --version
Swift version 5.10-dev (LLVM dbfaba0078e9380, Swift 63c8b551eb2f613)
Target: x86_64-unknown-linux-gnu

which will not bode well for reporting/reproducing bugs in the compiler.