Testing Publisher Value Updates with Async Tasks: How to Properly Wait for Changes in Tests?

Development Swift Testing
, ,
1

Hi everyone,

I’ve been working on testing a class that uses a @Published property, where an async task is triggered by a method, and eventually, the published value is updated. Initially, I had trouble testing the asynchronous behavior because the test would fail unless the value update was synchronous. After some reflection, I realized the key issue: I wasn’t properly waiting for the publisher to send the updated value before making my assertion.

Here’s a simplified version of my code:

final class SUT {
    @Published
    private(set) var myImportantValue: String

    func eventDidHappen() {
        // Triggers an async task that eventually updates myImportantValue in a thread-safe way
        // I avoid marking this as async to keep it transparent for the caller. The consumer just observes the published value.
    }
}

The test outline looked like this:

@Test
func should_update_important_value_when_event_happens() async throws {
    let sut = SUT()

    var didPublish = false
    sut.myImportantValue
        .dropFirst()
        .sink { _ in
            didPublish = true
        }

    sut.eventDidHappen()

    // Initially, I was expecting the following assertion to work, but it fails for async updates
    #expect(didPublish) // Fails unless the change is synchronous
}

The problem was that I didn’t account for the fact that the value change happens asynchronously, and thus, I wasn’t giving the publisher enough time to emit the updated value. After some thought, I realized that in order to make the test reflect actual consumer behavior, I need to wait for the value change within the test, between triggering the event and asserting the result.

I missed the point that the proper approach would be to properly wait for the change in the sink before asserting with #expect(didPublish).

To solve this, I created a custom TestExpectation to wait for the event to be triggered:

@Test
func should_update_important_value_when_event_happens() async throws {
    let sut = SUT()

    var didPublish = TestExpectation()
    sut.myImportantValue
        .dropFirst()
        .sink { _ in
            didPublish.fulfill()
        }

    sut.eventDidHappen()

    try await fulfillment(of: didPublish, timeout: 1)

    #expect(didPublish)
}

I derived this from how I formerly implemented such tests using XCTest framework with its XCTExpectation, hence the similarity.

The solution with the custom implementaion works well, but I also wanted to check if there’s anything in the Swift Testing framework that could simplify this.

Looking forward to hearing your thoughts!

2

XCTestExpectation is from XCTest and is not going to work with Swift Testing. Although it will create the necessary delay, if the expectation fails, Swift Testing won't know. As well, a timeout of 1 second may spuriously fail if the system is under load and the read takes more than 1 second to complete.

Instead of calling sink(), you can read the first value from values which is delivered asynchronously to your code. Something like this should work:

    let sut = SUT()

    var didPublish = TestExpectation()
    let values = sut.myImportantValue
        .dropFirst()
        .values

    sut.eventDidHappen()

    let didPublish = await values.first { _ in true }
    #expect(didPublish)
3

I’m just curious about the workaround you described. Are you saying that you implemented your own type named TestExpectation resembling XCTest’s XCTestExpectation type? Or was that a typo in your message and you’re saying you were attempting to use XCTestExpectation itself from a Swift Testing test?

4

Hello @smontgomery,

Yes, I implemented my own naive version of XCTests XCTestExpectation. It's really simple. It only has a fulfill() Method and the fulfillment(of:timeout) function just checks in a while loop if its fulfilled unless the timeout expires.

I already tried with XCTestExpectation in SwiftTesting and its not working. Mostly because the XCTestCase.fulfillment(of:timeout) method is not available in an Swift Testing test. ^^

I realize the naming of my custom implementation may not have been the best choice for this post.

I hope I could clarify on the confusion now.

5

Hey @grynspan

Thx for you reply. I was not aware of this possibility on an Publisher.

What I'm wondering is, if there is a way to let the awaiting of the value time out, in case the publisher does not publish anything. Which would mean something is broken in the implementation of SUT.

I also wonder if this would work with an synchronous implementation of the publishers value change.

My test should tell if the publisher fires as expected, regardless if the updating mechanic behind the scenes is doing it immediately or in an asynchronous fashion.

6

There have been a few requests for supporting timeouts in some fashion with confirmation(). There's general demand for timeouts with Swift Concurrency above and beyond testing, and we'd prefer to leverage whatever solution might be added to the stdlib rather than building our own.

That said, I am not a Combine expert and it may be the case that Combine has its own timeout mechanism you could leverage with values. Or, if you really had to roll your own timeout, you could do something like:

func withTimeLimit(
  _ timeLimit: Duration,
  _ body: @escaping @Sendable () async throws -> Void,
  timeoutHandler: @escaping @Sendable () -> Void
) async throws {
  try await withThrowingTaskGroup(of: Void.self) { group in
    group.addTask {
      // If sleep() returns instead of throwing a CancellationError, that means
      // the timeout was reached before this task could be cancelled, so call
      // the timeout handler.
      try await SuspendingClock().sleep(for: timeLimit)
      timeoutHandler()
    }
    group.addTask(operation: body)

    defer {
      group.cancelAll()
    }
    try await group.next()!
  }
}

This exact code is used in Swift Testing to implement the .timeLimit() trait, and you could use it here with finer granularity than the trait allows:

try await withTimeLimit(.seconds(10)) {
  // run your code here.
} timeoutHandler: {
  Issue.record("My code timed out!")
}

As far as I know, values works with synchronously-delivered elements too.