Recursive Suite Traits must conform to TestTrait or else they crash

Development Swift Testing
1

I just stumbled across this while teaching myself how to use TestScoping. If a Suite Trait test scoping fails to also conform to TestSuite, I observe a runtime crash.

Am I doing something wrong or could the runtime be smarter about detecting the missing conformance and fail more gracefully?

Code fragment:

// DANGER: missing TestTrait conformance
struct TestConfiguration: SuiteTrait, TestScoping {
    let isRecursive: Bool = true
    func provideScope(for test: Test, testCase: Test.Case?, performing function: @Sendable () async throws -> Void) async throws {
        try await function()
    }
}

extension Trait where Self == TestConfiguration {
    static var testConfiguration: Self { Self() }
}

@Suite(.testConfiguration)
struct DemoSuite {
    @Test func demo1() {}
}
2 Likes
2

For the record, here's the crash I get

Screenshot 2025-04-02 at 12.46.26 PM
Screenshot 2025-04-02 at 12.46.26 PM793Ă—329 64.2 KB

3

This is by design (it's an assertion failure), however we are planning to revisit the design of suite traits and adjust this behaviour.

@briancroom, @smontgomery, and I need to squabble over discuss it a bit more, but IMHO it is likely that we would treat the trait you have as recursively applying to suites nested in the outermost suite, but not to tests, and that you'd need to add TestTrait conformance to make it also apply to tests in that suite.

I don't think we currently have a GitHub issue tracking this change, so please feel free to file one!

1 Like
4

It makes sense that one would need the TestTrait conformance to be able to wrap test functions. I'm just wondering if there's better way to handle this condition. I was able to figure out, but others might not be so lucky.

5

Yes, we can do better, which was the point of my previous message I think. :slightly_smiling_face: If that's not what you mean, can you please clarify?

6

That is what I meant :smile:

For the record, I logged issue #$1048 in the GitHub repo

4 Likes
7

I ran into this issue myself as well.

It would crash in a weird way without any logs, so I assumed it was either a problem with my environment or a bug in swift-testing.

Once I set both SuiteTrait and TestTrait, things worked as expected.

I had assumed that the provideScope method of SuiteTrait would be called—especially since there’s a testCase argument.

But it turns out that if I set isRecursive on SuiteTrait, its provideScope method won’t be called, and instead the one from TestTrait will be used.

@mystic-mandrill Thanks for posting on the forum—that helped me figure this out.

@grynspan In my environment, assertion messages weren’t shown.

Maybe that’s because swift-testing is running in release build?

8

To be clear, I wasn't seeing any kind of assertion message either. It was @grynspan who identified the assertion from my stack trace.

Note that the "TestCase" argument to the callback has a boolean isSuite to differentiate between suites and test functions.

Can you elaborate on what you mean by differentiating between provideScope methods for SuiteTrait and TestTrait? In my implementation, I have a single struct that conforms to both protocols, so there's just one unique provideScope method. How are you doing it?

public struct SuiteSetup: SuiteTrait, TestTrait, TestScoping {
    public let isRecursive: Bool = true

    public init() {}

    public func provideScope(for test: Test, testCase: Test.Case?, performing testFunction: @Sendable () async throws -> Void) async throws {
        // do some global setup stuff
        try await testFunction()
        // do some global cleanup stuff
    }
}