Sure! I can add notes here about anything that comes to mind regarding the benchmark suite in general, not just the swiftpm part. Don't expect too much though, as I'm only looking into this as time permits.
One thing I noticed was that if I allowed Xcode 10 beta to update the project to 4.2, I got over 300 errors or something, enough for me to give up and restart the process using Xcode 9.4, or perhaps try with cmake instead.
No, my goal is just to get acquainted with the SBS, since I've, from the perspective of an amateur, been thinking a bit about how to benchmark issues that are highly context-dependent (eg compiling with -static-stdlib can make some specific code faster or slower depending on context, importing a module that isn't even used can sometimes make some code slower, adding or removing some spaces in a string that is printed after measuring the time of a hot loop can impact the optimizability of the timed code, using &+ can sometimes be slower than using + , etc.).
In short, some issues are impossible to demonstrate or benchmark unless the test is completely stand-alone, compiled with some specific flags etc. So such tests can't eg import TestUtils, or have some specific runFunction or BenchmarkInfo struct, since anything like that might interfere with what's meant to be tested.
So the software that is responsible for compiling, running and collect data from these kinds of tests would have to be as separated from the tests as possible.
This is why I asked the following:
Perhaps you know if and where I can find it?
That's a big improvement! Perhaps the docs could mention how to do a shallow clone, I had to look that up.