"Install Swift" GitHub Action

slashmo · 2021-11-18T10:22:01.066Z

Hi folks I’d like to pitch my GitHub Action slashmo/install-swift to be endorsed by the SSWG. The action lets you install any* version of Swift available on swift.org, while caching previously installed versions to make subsequent CI runs as fast as possible.

* While the action currently only supports Ubuntu I’d love to add support for other platforms as well. See future direction for more details.

Motivation

Recently, I needed to test a Swift library using different toolchains, with both release and development versions on GitHub Actions. While there were actions available like fwal/setup-swift which use a hard-coded list of installable Swift versions, I wanted to create one that’s more dynamic so that users can specify any version available on swift.org/downloads.

install-swift will figure out which version of Ubuntu it runs on to assemble the correct download URL. Based on the combination of Ubuntu & Swift version, it caches the downloaded toolchain so that on future executions of the CI running in the same context, it can be downloaded directly from GitHub instead. This enables a significant speed-up. In a demo project using the action, the cached toolchain installed in 16 seconds as opposed to the original 41 seconds.

Future Direction

Support more platforms

GitHub Actions have three ways of running:

Hosted runners (Ubuntu, macOS, Windows)
Self-hosted runners (Supports other Linux distributions and architectures)
Docker (Runs inside a given Docker container)

Currently, the action only supports the Ubuntu based GitHub Actions runners (that is ubuntu-18.04 & ubuntu-20.04). There shouldn’t be anything standing in the way of it also supporting other runners like macOS & Windows. This addition is already being tracked in GitHub Issues. We might also want to support all the distributions and architectures Swift is supported on, so that the action can also be useful for self-hosted runners and running in docker.

Support wildcard installs

If there’s interest, the action could also handle installing Swift versions based on wildcards like 5, then figure out the latest version of Swift 5 and install it.

To support this it would be nice for swift.org to expose some kind of list of all available versions. This could also enable wildcards like latest-main.

Example Workflow

Here’s an example of the action in action. You can check it out in the example repo: https://github.com/slashmo/xctest-async-linux/blob/main/.github/workflows/test.yaml

name: Test
on:
  push:

jobs:
  test:
    runs-on: ubuntu-20.04

    steps:
    - name: Install Swift
      uses: slashmo/install-swift@v0.1.0
      with:
        version: swift-DEVELOPMENT-SNAPSHOT-2021-11-12-a

    - name: Checkout
      uses: actions/checkout@v2

    - name: Run Tests
      run: swift test --parallel


      
          ARG OS_MIN_VER=04
          ARG SWIFT_WEBROOT=https://download.swift.org/development
          
          ENV SWIFT_SIGNING_KEY=$SWIFT_SIGNING_KEY \
              SWIFT_PLATFORM=$SWIFT_PLATFORM \
              OS_MAJOR_VER=$OS_MAJOR_VER \
              OS_MIN_VER=$OS_MIN_VER \
              OS_VER=$SWIFT_PLATFORM$OS_MAJOR_VER.$OS_MIN_VER \
              SWIFT_WEBROOT="$SWIFT_WEBROOT/$SWIFT_PLATFORM$OS_MAJOR_VER$OS_MIN_VER"
          
          RUN echo "${SWIFT_WEBROOT}/latest-build.yml"
          
          RUN set -e; \
              # - Grab curl here so we cache better up above
              export DEBIAN_FRONTEND=noninteractive \
              && apt-get -q update && apt-get -q install -y curl && rm -rf /var/lib/apt/lists/* \
              # - Latest Toolchain info
              && export $(curl -s ${SWIFT_WEBROOT}/latest-build.yml | grep 'download:' | sed 's/:[^:\/\/]/=/g')  \
              && export $(curl -s ${SWIFT_WEBROOT}/latest-build.yml | grep 'download_signature:' | sed 's/:[^:\/\/]/=/g')  \
              && export DOWNLOAD_DIR=$(echo $download | sed "s/-${OS_VER}.tar.gz//g") \
              && echo $DOWNLOAD_DIR > .swift_tag \

Something similar can be found in all other nightly images as well.

adam-fowler · 2021-11-18T11:26:17.423Z

Fabian I believe the latest.yml is only available for development builds. I don't know of any reporting of the latest version of a release.

If this GitHub action can be extended to support other Linux platforms, macOS and Windows it will be awesome. Well done @slashmo

hassila · 2021-11-18T12:03:12.149Z

Awesome @slashmo, thanks!

If the work of @futurejones (New Swift Package Repository for Ubuntu/Debian Linux Distributions) and @stevapple (GitHub - stevapple/swiftbox: Use Swift out of the Box on Ubuntu, CentOS(RHEL) and Amazon Linux) and this script would have an offspring that was officially blessed... :-)

Finagolfin · 2021-11-18T13:11:42.040Z

Nice, I wrote a separate small CI job to get the latest Swift release, devel, and trunk toolchain for my Android CI on github Actions. I'd gladly switch to your setup instead, but it looks like you don't support the development tag builds from the release/5.5 branch yet. If you add that and a way to just get the latest of each sort of release or snapshot toolchain, I'll switch over to your Action.

compnerd · 2021-11-18T17:36:14.571Z

As @ktoso mentioned, GitHub - compnerd/gha-setup-swift: Setup Swift (on Windows) on GitHub Actions Builders supports Windows. I've added Linux support as well to it, and I know that adding macOS is fairly trivial, just needs a bit of testing - I have something on a branch for that already).

The design choice difference was that I did not want the automatic update functionality. This is to ensure that re-runs of jobs use the same configuration and it is controllable (sometimes there is an issue in a snapshot, and you cannot control it if it is fully automatic). Yes, it could be the default is the latest, but to me it is less cognitive load to have everything be spelt out.

There is one thing that I dislike about my actions's implementation - namely the branch/tag pair feels like it should be possible to derive from a singular value. I'd be happy to have someone else help improve that if they are interested.

kmahar · 2021-11-18T18:07:58.162Z

This is great! +1

tomerd · 2021-11-19T17:28:25.986Z

adam-fowler:

Fabian I believe the latest.yml is only available for development builds. I don't know of any reporting of the latest version of a release

cc @mishal_shah can help with that

tomerd · 2021-11-19T17:28:44.833Z

also +1 great to see this

0xTim · 2021-11-22T11:33:39.439Z

You would think for people who work together would be better at talking I've been sherlocked!

+1 from me, very useful and lots of extra features that can be added over time

fwal · 2021-11-29T12:45:16.889Z

Hi all!

I'm the author of the "setup-swift" action mentioned (GitHub - swift-actions/setup-swift: GitHub Action that setup a Swift environment).
Just wanted to pitch in that I think that a consolidated effort in this area would be great, it seems like the current different solutions all solve different parts of the equation of a fully fledged solution

slashmo · 2022-01-31T17:54:06.765Z

Hey folks, thank you all for the comments and apologies for my late reply!

Changes since last year

@PoshAlpaca joined as co-maintainer
Now supports installing on macOS (#11)
The correct system libraries are now installed depending on the Ubuntu version (#7)
Upcoming release snapshots, e.g. swift-56-development-main, are now supported (#18)

Replies

fabianfett:

This already exists today and is used when building the nightly docker images. For ubuntu 20.04 the lookup url is:

Cool! It would be great to have something similar for the other types of releases as well. (#19)

adam-fowler:

If this GitHub action can be extended to support other Linux platforms, macOS and Windows it will be awesome.

Thanks Adam, as mentioned above macOS support is now implemented. We have an umbrella issue for supporting additional platforms/distributions here (#1).

hassila:

If the work of @futurejones (New Swift Package Repository for Ubuntu/Debian Linux Distributions) and @stevapple (GitHub - stevapple/swiftbox: Use Swift out of the Box on Ubuntu, CentOS(RHEL) and Amazon Linux) and this script would have an offspring that was officially blessed... :-)

Oh, didn't know these existed! They both look interesting. Do you have anything in mind for that offspring?

Finagolfin:

I'd gladly switch to your setup instead, but it looks like you don't support the development tag builds from the release/5.5 branch yet.

Thanks for the suggestion! Release/* branch snapshots are now supported. We definitely also want to enable specifying wildcard-like versions as you mention. Here's the issue: (#19)

compnerd:

The design choice difference was that I did not want the automatic update functionality. This is to ensure that re-runs of jobs use the same configuration and it is controllable (sometimes there is an issue in a snapshot, and you cannot control it if it is fully automatic). Yes, it could be the default is the latest, but to me it is less cognitive load to have everything be spelt out.

Thanks for the details on your design choice! Does that also apply to release versions? As for install-swift, we definitely want to support both kinds of specificity when dealing with dev snapshots. At the moment we only support hard-coded snapshots, but we'd also like to supporting something like latest-trunk as an alternative.

0xTim:

You would think for people who work together would be better at talking I've been sherlocked!

Haha, sorry about surprising you

fwal:

Just wanted to pitch in that I think that a consolidated effort in this area would be great, it seems like the current different solutions all solve different parts of the equation of a fully fledged solution

Thanks for chiming in! I've been thinking about how we could best consolidate the efforts, and would like to suggest that we have both an install-swift and setup-swift action, where setup-swift has built-in support for things like .swiftpm & build artifact caching and is what most people use, and install-swift a pure installation action. By making setup-swift a composite action, it could then use install-swift for the installation part. What do you think? CC @0xTim as you created the caching issues (#3 & #4).

Thanks again for all your feedback, looking forward to fleshing out the action some more.