Running Swift package tests with GitHub Actions

GitHub provides a solution for running tests on commits which they call

name: tests

on:
  - push
  - pull_request
  
jobs:

  linux:
    name: Linux
    runs-on: ubuntu-latest
    container:
      image: swift:5.1
    steps:
    - name: Checkout
      uses: actions/checkout@v1
    - name: Test
      run: swift test

Different build steps

Our initial configuration has one Test step to test the library. As the Actions UI gives indication of which steps pass or fail, it would be nice to split up the build step from the test step so that we can see at a glance whether it’s the build that failed or a test that failed.

steps:
- name: Checkout
  uses: actions/checkout@v1
- name: Swift Version
  run: swift --version
- name: Release Build
  run: swift build -v -c release
- name: Debug Build
  run: swift build -v -c debug
- name: Debug Test
  run: swift test -v -c debug

Adding more steps is trivial and here I’ve added different steps for release and debug builds, as well as using the option -v to provide a more verbose output, which may be useful for identifying failures that may occur.

I’ve not tested the release build because that prevents use of @testable but if you don’t need that, then you could also try running tests against the release build too.

–enable-test-discovery

LinuxMain.swift and XCTestManifests.swift are files that are generated on macOS machines to list out the tests to run, so that Linux machines can run XCTMain on all the tests. When writing a library

Swift 5.1 introduced a new flag called --enable-test-discovery which allows tests to be discovered on Linux, removing the need for.

## Using a matrix for multiple Swift versions

It would be super swanky if we could run our tests against all the swift versions we claim to support. Fortunately for us, the Swift project provides pre-built Docker containers for each version of Swift and GitHub actions allows us to define custom variables, each with a list of values and it will run each combination using the same set of steps.

For us, we only want to add one variable, which we use to supply a list of image names, swift:5.1 and swift:5.2. Then to reference them, we can use $ for the value of the image.

linux:
  strategy:
    fail-fast: false
    matrix:
      image:
        - "swift:5.1"
        - "swift:5.2"
  name: Linux
  runs-on: ubuntu-latest
  container:
    image: $

fail-fast prevents a failed run of one image from cancelling the other. I use this because I would want to see if a failure is unique to a particular version of Swift, so want to see the results of all runs. I can imagine this doesn’t apply to all projects however.

macOS

For the Mac, we can add a new job that runs on the latest version. I’m sure there’s a way to get a specific version of swift used on here, but for now I run the same steps only on the latest version of macOS using the latest version of the compiler. It can take some time for new versions to appear as GitHub updates Xcode on its machines, so sometimes I’ve had to remove this job until the machines have been updated.

macos:
  name: MacOS
  runs-on: macos-latest
  steps:
  - name: Checkout
    uses: actions/checkout@v1
  - name: Swift Version
    run: swift --version
  - name: Release Build
    run: swift build -v -c release
  - name: Debug Build
    run: swift build -v -c debug
  - name: Debug Test
    run: swift test -v -c debug

Complete tests.yml

This is the complete tests.yml file I use for my projects, wiht the only difference being which versions of Swift are supported. Once the image for the new version of Swift is available, I add that to my image list.

name: tests

on:
  - push
  - pull_request

jobs:

  linux:
    strategy:
      fail-fast: false
      matrix:
        image:
          - "swift:5.1"
          - "swift:5.2"
    name: Linux
    runs-on: ubuntu-latest
    container:
      image: $
    steps:
    - name: Checkout
      uses: actions/checkout@v1
    - name: Swift Version
      run: swift --version
    - name: Release Build
      run: swift build --enable-test-discovery -v -c release
    - name: Debug Build
      run: swift build --enable-test-discovery -v -c debug
    - name: Debug Test
      run: swift test --enable-test-discovery -v -c debug

  macos:
    name: MacOS
    runs-on: macos-latest
    steps:
    - name: Checkout
      uses: actions/checkout@v1
    - name: Swift Version
      run: swift --version
    - name: Release Build
      run: swift build -v -c release
    - name: Debug Build
      run: swift build -v -c debug
    - name: Debug Test
      run: swift test -v -c debug

Nightly builds

Recently, the Swift project also announced nightly images for the latest versions of Swift – one for the cutting edge branch and one for the latest code of the next announced version of Swift, which is currently 5.3.

The jobs part is similar, with the exception of when it runs. I set this up with cron syntax to run at midnight every night. The images are located on DockerHub as swiftlang/swift:nightly and ``swiftlang/swift:nightly-5.3`, so those are the images I use. I update these along with the tests.yml when the new version is released.

name: nightlies

on:
  schedule:
    - cron: '0 0 * * * '

jobs:

  linux:
    strategy:
      fail-fast: false
      matrix:
        image:
          - "swift:nightly"
          - "swift:nightly-5.3"
    name: Linux
    runs-on: ubuntu-latest
    container:
      image: swiftlang/$
    steps:
    - name: Checkout
      uses: actions/checkout@v1
    - name: Swift Version
      run: swift --version
    - name: Release Build
      run: swift build --enable-test-discovery -v -c release
    - name: Debug Build
      run: swift build --enable-test-discovery -v -c debug
    - name: Debug Test
      run: swift test --enable-test-discovery -v -c debug