I have a Stub type that records its parameters and return value whenever it is called. It stores these in a calls array, and has computed properties for accessing the callArguments, among other things. I also have an EventBus type with a fire-and-forget style postEvent(_:) function that asynchronously delivers events to observers.
I’m trying to write some tests for a specific use of the event bus. I want to verify that when an event is received, the event bus logs a telemetry event:
let telemetryEventLogger = MockTelemetryEventLogger(logEventStub: Stub())
let eventBus = EventBus()
// And more setup…
// System under test
let event = SomeEvent()
eventBus.post(event)
// Wait for the event to be delivered
try await Task.sleep(for: .milliseconds(200))
// Verify that the event was logged to the telemetry logger
#expect(telemetryEventLogger.logEventStub.callArguments == [event])
This works, but the sleep is obviously suboptimal.
I modified the Stub type to be Observable, so now we can track changes to the calls. Using this, I changed the code to:
// Same setup code as above
// Start observing changes to the stub’s call count
let logEventStub = telemetryEventLogger.logEventStub
let observations = Observations({ !logEventStub.calls.isEmpty })
let observationTask = Task.immediate {
return await observations.first { $0 == true }
}
// System under test
let event = SomeEvent()
eventBus.post(event)
// Wait for the stub to be called before doing verifications
await _ = observationTask.value
// Same verification code as above
When this works, it runs in only 0.005 seconds (success!), but once every 20–100 runs, the test will time out having never observed the change. I assume there’s some race between the observation starting and the system under test, but nothing I do can work around it. I’ve tried wrapping the system-under-test in a Task, adding Task.yield() before posting the event, and increasing the priority of the observation task. Nothing seems to work.
It sounds like what might help is to then is to focus on making that operation deterministic and predictable when running tests. How are you currently dispatching events to observers?
Since posting last night, I’ve found a fast workaround for this problem that doesn’t use Observations. I emit a value to an async stream from my mocked telemetry logger, which I wait for in my tests.
That said, I think the question about Observations remains an interesting one. My workaround basically does what I thought Observations would do, but it doesn’t intermittently fail like the Observations approach does. There will surely be other situations in which someone wants to await an observation like this, but it’s not obvious that it wouldn’t have some subtle, intermittent race as well.
My understanding is that Task.immediate can't always be used to set up observation and ensure the loop is already "listening" for values before continuing, even though it (empirically) works most of the time. Nothing in the AsyncIteratorProtocol guarantees this to work.
That said, I thought this would work for Observations unless an actor hop was required due to the observation and the iterator being in different isolation regions. I can't really tell from the source if there are any additional suspension points in the process of calling next() before the continuation is registered, though.
+1 on this, it's the only truly reliable solution I've found when facing this problem. Sadly it's not always feasible to do so. Sometimes you do want to test whether a fire-and-forget function will eventually trigger a new value in an async sequence.
One option at that point is to build your own custom AsyncSequence type on a continuation to "deadlock" your test code. Your production code can continue to run from the regular AsyncStream or AsyncChannel but the test code injects in this custom dependency.