How do I apply a CustomExecutionTrait to a Suite?

Related Projects swift-testing
1

For context, I'm trying to convert a UI test suite from XCTest to swift-testing. There are a few requirements, but the one I'm attacking at the moment is I need to initialize the UI test harness (which in turn launches the app) exactly once for each test suite, and I need to tear it down (which stops the app) after all the tests have run.

It seems CustomExecutionTrait is the right way to do this, but I'm not sure how to apply it to a test suite. What would that look like? Here's what I've got:

@_spi(Experimental) import Testing
struct SetupDriverTrait: CustomExecutionTrait, SuiteTrait {
    @Sendable
    func execute(_ function: @escaping @Sendable () async throws -> Void, for test: Test, testCase: Test.Case?) async throws {
        print("Before: \(test).name)")
        try await function()
        print("After: \(test).name)")
    }
}
2

@Suite is modelled at runtime as an instance of Test whose isSuite property has a value of true—the difference between @Suite and @Test is one of syntactic sugar, for the most part. So the instance of test you are given in this context should refer to your suite.

(Be aware that there's a known issue in the implementation of CustomExecutionTrait that may cause it to execute at the wrong time relative to a suite.)

A formalized interface for running code before/after a suite is tracked by API or mechanism to perform an action once before/after running all of the tests in a suite type · Issue #36 · apple/swift-testing · GitHub.

3

Thanks for the quick reply! I've familiarized myself with #36, and I'm okay with this being experimental at the moment.

The part I'm having trouble with is what I actually need to type to associate the trait with a suite or test. I've looked at CustomExecutionTraitTests.swift but not being a swift expert I'm not sure how to adapt that.

4

To associate a trait with a suite, you need some way to instantiate the trait. With the code you have here, you can just write SetupDriverTrait():

@Suite(SetupDriverTrait()) struct MyTests {
  ...
}

However, you'll note the pattern we use in the testing library generally involves a static factory function. The syntax for creating one that will be picked up correctly by Swift in this position is a bit weird, but is effectively:

extension Trait where Self == SetupDriverTrait {
  static func setupDriver(args: ...) -> Self {
    ...
  }

  // OR:

  static var setupDriver: Self {
    ...
  }
}

The name of the function is up to you.

1 Like
5

Ahhhhh I see. I'll go give this a try. Thank you!

6

I think I see what you mean about the issue with CustomExecutionTrait running at the wrong time for Suites. I see "before" and "after" both printing before any tests in the suite are run. Is there a bug tracking that, or no because it's experimental?

7

The feature is indeed experimental at this time (as is all of swift-testing!) @Dennis is looking into if/when/how we could promote it to a supported API.

1 Like