Our test suite takes a long time to run, e.g. up to 20 minutes on CI. In order to iterate faster, we would love to split up tests and run them in parallel, but the "obvious" approaches to it don't seem to work. This is particularly surprising for non-solution 2 (using swift test --filter), which seems like a bug or shortcoming to me. We did find a workaround, but it's a bit awkward and hard to extend.
Non-solution 1
The first idea is to use swift test --parallel. While this could potentially speed up tests on a machine with multiple cores, at least Gitlab only seems to run a single core per job, so in this case there is no speedup by choosing this option. While there is an option for Gitlab to run multiple instances of the same job in parallel, it is my understanding that this would require swift test --parallel to be able to receive an argument indicating the number of the job (similar to the example in the Gitlab documentation). I don't think such a capability exists?
Non-solution 2
Alternatively, we might just manually create different jobs that test subsets of the test cases independently; this is fine since we know e.g. which test sets take a long time, and it's also more deterministic than the previous version, even though it requires a bit more maintenance. While XCTest unfortunately doesn't seem to support advanced tagging and filtering like some other tools, for our use cases the basic regex-based filter would essentially suffice. Unfortunately, we discovered that splitting tests up by the use of swift test --filter makes the whole CI build significantly slower than not splitting tests up. Since swift test --filter also produces output in quite a different way than normal swift test (e.g. it doesn't print a summary of all the tests at the end), my working hypothesis is that using the --filter option somehow causes the test framework to re-run some app setup code for every test which incurs quite a performance overhead in our use case (due to costly initialisation logic in our app) as opposed to having this setup code run only once. I wonder if this is known behaviour or some kind of bug?
Workaround
The only working solution that we've found so far is a bit messy; it consists in basically just deleting test files on CI before running the tests (for this to work, --enable-test-discovery needs to be used in lieu of LinuxMain.swift). I don't think it's a very neat solution, but it seems to work and speeds our builds up significantly.