XCTestObservation callbacks for Swift Testing

Development Swift Testing
1

Am I right in saying that XCTestObservation calls are never hit when registering a conforming class to XCTestObservationCenter.shared?

I’m just trying to get some callbacks to know when a retry is about to begin for some bookkeeping code to clear its state but adding breakpoints shows these functions like testCaseDidFinish(_:)are never called.

Is there an alternative callback I could hook in to?

2

Are you using XCTest or Swift Testing? XCTestObserver only works with XCTest. We haven't yet designed an equivalent API for Swift Testing, although it's on the to-do list.

1 Like
3

@grynspan that's right, Swift Testing.

Ah thought that was the case, no problem.

I guess I thought Swift Testing might be a wrapper around XCTest so would still be able to use the XCTestObserver under the hood but interesting to know it’s not!

4

Correct, Swift Testing is not a wrapper of XCTest.

1 Like
5

For some use cases it may be possible to accomplish what you’re trying to do using a custom Test Scoping Trait.

6

Thanks @smontgomery but I think in my case I won’t be able to use the test scoping traits as I need to hook in to the retries to reset a counter.

More specifically, my use case is to reset a counter between retries inside Pointfree’s snapshot testing framework which relies on XCTest observations but doesn’t currently work for Swift Testing - but it sounds like this is a current limitation of Swift Testing (for now)

7

Not sure what you are trying to accomplish or if this helps, but we do provide a snapshots test trait that resets counters between test runs for Swift Testing:

8

Hi @mbrandonw - I’m just looking in to a bug in my own wrapper around your snapshot testing library which allows for the use of macros, similar to Swift Testing’s @Test/@Suite, in the shape of @SnapshotTest/@SnapshotSuite to conveniently snapshot test views:

The issue I’m hitting is on retries of my test suites the counter continues to count up, creating new snapshot references which subsequently fail instead of resetting on each retry.

Maybe it’s best for me to submit a bug/open a discussion on your snapshot testing library’s repo to look in to this more, but from what I can see by adding breakpoints in to the below code is that they’re never hit because my wrapper uses Swift Testing:

swift-snapshot-testing/Sources/SnapshotTesting/AssertSnapshot.swift at d7e40607dcd6bc26543f5d9433103f06e0b28f8f · pointfreeco/swift-snapshot-testing · GitHub

Is it possible for me to use that trait (or override the counter) when calling through to SnapshotTesting.verifySnapshot()?

9

I believe everything would work automatically as long as tests written with your macros continue to apply the test trait:

@Suite(.snapshots)
@SnapshotSuite
struct MySnapshots {
  …
}

Even if you aren't overriding any of the settings of SnapshotTesting it always a good idea to use the .snapshots trait specifically for overriding that internal state between tests.

1 Like
10

@mbrandonw Perfect, this is exactly what I wanted for my macro.

I’m assuming at this point a new counter is initialised on each run of the suite meaning retries start from 0 and tests with arguments use the same instance so benefit from the counter adding values?

Either way this is what I was looking for so thanks very much!

Also, FWIW, in my code the .snapshots trait goes inside my generated @Suite so the user will never have to write it out manually:

(Annoyingly for now I have to also add the @Suite above my @SnapshotSuite macro for Xcode to recognise the tests and be able to run from the test diamond from the Xcode GUI)