"Install Swift" GitHub Action

slashmo · November 18, 2021, 10:22am

Hi folks I’d like to pitch my GitHub Action slashmo/install-swift to be endorsed by the SSWG. The action lets you install any* version of Swift available on swift.org, while caching previously installed versions to make subsequent CI runs as fast as possible.

* While the action currently only supports Ubuntu I’d love to add support for other platforms as well. See future direction for more details.

Motivation

Recently, I needed to test a Swift library using different toolchains, with both release and development versions on GitHub Actions. While there were actions available like fwal/setup-swift which use a hard-coded list of installable Swift versions, I wanted to create one that’s more dynamic so that users can specify any version available on swift.org/downloads.

install-swift will figure out which version of Ubuntu it runs on to assemble the correct download URL. Based on the combination of Ubuntu & Swift version, it caches the downloaded toolchain so that on future executions of the CI running in the same context, it can be downloaded directly from GitHub instead. This enables a significant speed-up. In a demo project using the action, the cached toolchain installed in 16 seconds as opposed to the original 41 seconds.

Future Direction

Support more platforms

GitHub Actions have three ways of running:

Hosted runners (Ubuntu, macOS, Windows)
Self-hosted runners (Supports other Linux distributions and architectures)
Docker (Runs inside a given Docker container)

Currently, the action only supports the Ubuntu based GitHub Actions runners (that is ubuntu-18.04 & ubuntu-20.04). There shouldn’t be anything standing in the way of it also supporting other runners like macOS & Windows. This addition is already being tracked in GitHub Issues. We might also want to support all the distributions and architectures Swift is supported on, so that the action can also be useful for self-hosted runners and running in docker.

Support wildcard installs

If there’s interest, the action could also handle installing Swift versions based on wildcards like 5, then figure out the latest version of Swift 5 and install it.

To support this it would be nice for swift.org to expose some kind of list of all available versions. This could also enable wildcards like latest-main.

Example Workflow

Here’s an example of the action in action. You can check it out in the example repo: https://github.com/slashmo/xctest-async-linux/blob/main/.github/workflows/test.yaml

name: Test
on:
  push:

jobs:
  test:
    runs-on: ubuntu-20.04

    steps:
    - name: Install Swift
      uses: slashmo/install-swift@v0.1.0
      with:
        version: swift-DEVELOPMENT-SNAPSHOT-2021-11-12-a

    - name: Checkout
      uses: actions/checkout@v2

    - name: Run Tests
      run: swift test --parallel


      
          ARG OS_MIN_VER=04
          ARG SWIFT_WEBROOT=https://download.swift.org/development
          
          ENV SWIFT_SIGNING_KEY=$SWIFT_SIGNING_KEY \
              SWIFT_PLATFORM=$SWIFT_PLATFORM \
              OS_MAJOR_VER=$OS_MAJOR_VER \
              OS_MIN_VER=$OS_MIN_VER \
              OS_VER=$SWIFT_PLATFORM$OS_MAJOR_VER.$OS_MIN_VER \
              SWIFT_WEBROOT="$SWIFT_WEBROOT/$SWIFT_PLATFORM$OS_MAJOR_VER$OS_MIN_VER"
          
          RUN echo "${SWIFT_WEBROOT}/latest-build.yml"
          
          RUN set -e; \
              # - Grab curl here so we cache better up above
              export DEBIAN_FRONTEND=noninteractive \
              && apt-get -q update && apt-get -q install -y curl && rm -rf /var/lib/apt/lists/* \
              # - Latest Toolchain info
              && export $(curl -s ${SWIFT_WEBROOT}/latest-build.yml | grep 'download:' | sed 's/:[^:\/\/]/=/g')  \
              && export $(curl -s ${SWIFT_WEBROOT}/latest-build.yml | grep 'download_signature:' | sed 's/:[^:\/\/]/=/g')  \
              && export DOWNLOAD_DIR=$(echo $download | sed "s/-${OS_VER}.tar.gz//g") \
              && echo $DOWNLOAD_DIR > .swift_tag \

Something similar can be found in all other nightly images as well.

adam-fowler · November 18, 2021, 11:26am

Fabian I believe the latest.yml is only available for development builds. I don't know of any reporting of the latest version of a release.

If this GitHub action can be extended to support other Linux platforms, macOS and Windows it will be awesome. Well done @slashmo

hassila · November 18, 2021, 12:03pm

Awesome @slashmo, thanks!

If the work of @futurejones (New Swift Package Repository for Ubuntu/Debian Linux Distributions) and @stevapple (GitHub - stevapple/swiftbox: Use Swift out of the Box on Ubuntu, CentOS(RHEL) and Amazon Linux) and this script would have an offspring that was officially blessed... :-)

Finagolfin · November 18, 2021, 1:11pm

Nice, I wrote a separate small CI job to get the latest Swift release, devel, and trunk toolchain for my Android CI on github Actions. I'd gladly switch to your setup instead, but it looks like you don't support the development tag builds from the release/5.5 branch yet. If you add that and a way to just get the latest of each sort of release or snapshot toolchain, I'll switch over to your Action.

compnerd · November 18, 2021, 5:36pm

As @ktoso mentioned, GitHub - compnerd/gha-setup-swift: Setup Swift (on Windows) on GitHub Actions Builders supports Windows. I've added Linux support as well to it, and I know that adding macOS is fairly trivial, just needs a bit of testing - I have something on a branch for that already).

The design choice difference was that I did not want the automatic update functionality. This is to ensure that re-runs of jobs use the same configuration and it is controllable (sometimes there is an issue in a snapshot, and you cannot control it if it is fully automatic). Yes, it could be the default is the latest, but to me it is less cognitive load to have everything be spelt out.

There is one thing that I dislike about my actions's implementation - namely the branch/tag pair feels like it should be possible to derive from a singular value. I'd be happy to have someone else help improve that if they are interested.

kmahar · November 18, 2021, 6:07pm

This is great! +1

tomerd · November 19, 2021, 5:28pm

cc @mishal_shah can help with that

tomerd · November 19, 2021, 5:28pm

also +1 great to see this

0xTim · November 22, 2021, 11:33am

You would think for people who work together would be better at talking I've been sherlocked!

+1 from me, very useful and lots of extra features that can be added over time

fwal · November 29, 2021, 12:45pm

Hi all!

I'm the author of the "setup-swift" action mentioned (GitHub - swift-actions/setup-swift: GitHub Action that setup a Swift environment).
Just wanted to pitch in that I think that a consolidated effort in this area would be great, it seems like the current different solutions all solve different parts of the equation of a fully fledged solution

slashmo · January 31, 2022, 5:54pm

Hey folks, thank you all for the comments and apologies for my late reply!

Changes since last year

@PoshAlpaca joined as co-maintainer
Now supports installing on macOS (#11)
The correct system libraries are now installed depending on the Ubuntu version (#7)
Upcoming release snapshots, e.g. swift-56-development-main, are now supported (#18)

Replies

Cool! It would be great to have something similar for the other types of releases as well. (#19)

Thanks Adam, as mentioned above macOS support is now implemented. We have an umbrella issue for supporting additional platforms/distributions here (#1).

Oh, didn't know these existed! They both look interesting. Do you have anything in mind for that offspring?

Thanks for the suggestion! Release/* branch snapshots are now supported. We definitely also want to enable specifying wildcard-like versions as you mention. Here's the issue: (#19)

Thanks for the details on your design choice! Does that also apply to release versions? As for install-swift, we definitely want to support both kinds of specificity when dealing with dev snapshots. At the moment we only support hard-coded snapshots, but we'd also like to supporting something like latest-trunk as an alternative.

Haha, sorry about surprising you

Thanks for chiming in! I've been thinking about how we could best consolidate the efforts, and would like to suggest that we have both an install-swift and setup-swift action, where setup-swift has built-in support for things like .swiftpm & build artifact caching and is what most people use, and install-swift a pure installation action. By making setup-swift a composite action, it could then use install-swift for the installation part. What do you think? CC @0xTim as you created the caching issues (#3 & #4).

Thanks again for all your feedback, looking forward to fleshing out the action some more.

JacobHearst · October 21, 2024, 3:05pm

Has there been any more discussion about this topic? It seems like people (including myself) are generally in favor of the idea of a "blessed" solution for installing and setting up Swift on all supported platforms. Does this just need to go through the SSWG incubation process?

gregcotten · October 21, 2024, 4:10pm

You'd imagine once Swiftly becomes a bit more mature (supporting Windows, for one), you could make a GitHub Action wrapping that rather than doing something entirely bespoke in Node, for example.

I would say the "best" Github Action I have used for so far is SwiftyLab/setup-swift, mainly because it works on macOS/Linux/Windows and supports the latest versions of Swift. Would love to see a canonical Github Action at least blessed by swiftlang, if not outright maintained.

JacobHearst · October 21, 2024, 4:33pm

That's the action that I use currently and it's quite nice. I haven't tried the one that started this thread but having my workflow be as simple as simple as what's below is a beautiful thing

name: Build and test

on: push

jobs:
  build:
    strategy:
      fail-fast: false
      matrix:
        os: [macos-latest, ubuntu-latest, windows-latest]
    runs-on: ${{ matrix.os }}

    steps:
    - uses: actions/checkout@v4
    - name: Setup Swift
      uses: SwiftyLab/setup-swift@v1.8.0
      with:
        swift-version: '6.0.0'
    - name: Build
      run: swift build -v
    - name: Run tests
      run: swift test -v

taylorswift · October 21, 2024, 8:04pm

the biggest issue i’ve had with existing installer actions is they don’t get updated in a timely fashion, so it can be a while until the latest Swift releases show up in the installer action. so i now use

which supports caching and just takes URL path components as its toolchain selector.

barnard-b · October 21, 2024, 9:14pm

We have been using Swiftly in our Ubuntu actions, these two steps usually take around 45 seconds.

- name: Install Swiftly
  run: curl -L https://swiftlang.github.io/swiftly/swiftly-install.sh | bash -s -- --disable-confirmation
- name: Install Swift
  run: swiftly install 6.0.1

Finagolfin · October 22, 2024, 2:54am

There was a new official github workflows repository announced last month.