Hey! This seems like an interesting discussion and I wanted to see if there was an opportunity for me to contribute/help in working towards a solution.
Although the initial traits document does list --all-trait-combinations as a possible future direction, I agree with @Cyberbeni that it can get very intensive very quickly. It may still be a useful flag to have in some scenarios, but I’m unsure whether it effectively solves the problem being discussed here.
There is a a related issue #9523, which explores allowing a test target to request a non-default trait. That could address one non-default configuration at a time, but it doesn't appear to cover the broader requirement of building and testing several trait configurations in one invocation.
Perhaps the solution here is adding a configuration parameter to the Package in the Package.swift file that can declare the configurations/combinations of traits that will be used by this package? it could declare configurations for no traits, selected traits, default traits, or all traits.
Test targets could then indicate which of those configurations they should participate in. Running the test suite would require SwiftPM to build and test each selected configuration independently…
This way, we avoid running all 2^n combinations of traits, and only focus in on ones that are truly required by our package.
Still, I suppose some design questions remain on how we would go about this:
- Does it make sense for plain
swift testrun every test trait configuration declared by the package author? - Or, should this require an explicit option like
swift test --all-configurationsfor instance? - If the user passes a
--traitsargument, should that override the declared test configurations and produce a single test run?
I’d very much be interested in helping investigate or implement this, but I wanted to first ask whether this general direction makes sense? I’m assuming this would have to go through the evolution process in this case because of the scope (please correct me if I am wrong), but would love to hear and discuss if you think there are any other implementations that may help us work our way towards this solution.
Looking forward to discussing more!