Capture all issues in a block without reports, for property-based testing

Hi,

I'm working on a library for property-based testing (a.k.a. QuickCheck) designed for the new Swift Testing framework, as an up-to-date replacement for SwiftCheck. It currently supports running tests with random inputs, while also being able to supply a specific random seed to reproduce failing cases.

One aspect of property-based testing that Swift Testing is currently not suited for is shrinking, which involves calling a failing test repeatedly with different inputs to find a minimal failing case. The closest function is withKnownIssue(), but this will still report every suppressed issue as a "known issue". Finding the actual Issue I want to display will also add around 20 "known" Issues which are irrelevant to the output.

I'd like Swift Testing to add a way to suppress all issues being recorded inside a block, while also returning which issues were suppressed (or at least the amount of issues). I could then use it to run all tests, and in the case of failures, run them again while changing the inputs. Finally, run it without the suppression to actually record all issues.

3 Likes