Swift test --filter is slow; or: how to run tests fast on CI?

Non-solution 1

swift test does have a job option but that controls the number of tests to run in parallel when executing the entire test suite. The GitLab feature seems something specialized where the jobs are split into N jobs and only a single split job runs in an invocation. This is not something that is supported by SwiftPM.

Non-solution 2

Your hypothesis is right and --filter spawns a new subprocess for individual test cases. This causes the setup code run for every single test case which is costly in your case. We should work towards improving the parallelism heuristics in the swift test tool. Can you file a bug for this?

Here are some workarounds that might work better for your use case:

Workaround 1:

You can move your costly tests into a local package in the same repository and then run those in a separate job. Assuming you're using Xcode to develop on desktop, you can create an empty workspace and add both of these packages so you can run tests of either package locally without having to switch between them.

Workaround 2:

This is a bit "hacky" but should be an OK workaround: Conditionally add/remove test targets from the package manifest. Here's an example from SwiftPM's own manifest. This also requires automatic test discovery.

Workaround 3:

One other option is driving the test execution process yourself. This is a bit risky/advanced as test execution is platform dependent and the underlying implementation detail can evolve overtime. With that disclaimer, here's how you can execute the tests yourself on Linux:

  1. Perform the build with tests

$ swift build --build-tests

  1. List the tests in JSON format

$ .build/debug/<PackageName>PackageTests.xctest --dump-tests-json

  1. Execute the subset of tests that you want on the class or test case granularity

$ .build/debug/<PackageName>PackageTests.xctest <TestTargetName>.<TestsClassName>

or

$ .build/debug/<PackageName>PackageTests.xctest <TestTargetName>.<TestsClassName>/<TestMethodName>