Task's closure not called in async XCTest function

Using Swift
, , ,
1

I came across an issue when writing unit tests for a "Finite State Automaton plus Side Effect" machinery.

I was able to reduce the code to illustrate the problem as follows:

    func testExampleIssueAsync2() async throws {
        let expectCompleted = expectation(description: "completed")
        Task {
            expectCompleted.fulfill()
        }
        wait(for: [expectCompleted], timeout: 1)
    }

On macOS (current version) and iOS 14 (Simulator) the test returns as a PASS.

For iOS 15 the test returns as a FAIL. In this case the Task's closure has not been called before the test function returns (it will be called afterwards).

Looking at the stack trace gives us already a clue why this is the case. Also, when adding @MainActor to the test function, it fails also in macOS and iOS 14.

The observed differences stem from the underlying type of the dispatch queue: a "global concurrent queue" vs a "serial queue", vs "cooperative serial" queue where the test function is executing.

In case of a serial queue, the task's closure will only be executed after the enclosing function completes. So, effectively, the test cannot be performed like this.

What would be a proper approach to get reliable and correct tests?

Background

Now, obviously there is no reason to use async in this snippet above. However, the real test object is a Finite State Automaton (FSA) realised with Swift Combine. It has a transition and an output function (merged into one "update" function). The FSA's Output are "Effect" values which may spawn functions performing any side effects, like a Task with an async closure, or a publisher producing zero or more "events". Events will be fed back to the FSA. And, it will never stop running, and never blocking. (note, it's akin to "The Composable Architecture" which has been mentioned several times already here in this forum).

Then, there are also dependencies, which need to be setup which will be injected later when these effects do actually run, so that it runs a mocked service and not a real network service.

Now, some dependencies may be Swift actors, and thus require an async context when they need to be setup. Now, we are here, where we need the async test function ;)

I am also a bit concerned about the robustness of the approach in a production setup. Will the whole machinery run correctly when enclosed within a task, as opposed to when starting from the main thread? Of course, it IS expected and absolutely required that the tasks will run. The FSA has no notion about an async context, this is pure Swift Combine. The Tasks in side effects itself don't know its parent - but should utilise structured concurrency automatically (tasks may be detached, too if needed). Cancellation of tasks can be explicitly managed by the FSA by sending it corresponding events, which may be originate from a user. Also, when a task gets canceled due to structured concurrency, the FSA will take care of this fact and will (well, should) behave as expected.

2

Easiest way to wait for work like that is to simply await Task {}.value. If you need to use explicit waits, I believe there are now async versions which properly suspend so other async work can finish on the same actor.

1 Like
3

hi Jon, thanks for your reply. :)

Yes, explicitly awaiting the task would put the closure into the enclosing function - and it would work.

However, I think, the issue needs further explanation:
If you are familiar with The Composable Architecture (which I am using to explain the issue), the Task {...} would be executed within an Effect. Now, in order to have some useful unit tests, one want to use the already created store from the app together with the already existing update function, and of course all the already defined effects. The Store is defined in a third party package, and the update function may be implemented in the App as well as the effects. Effects may even be implemented in other packages.

What one want to do in the unit test, is simply providing a "Test Environment", which contains the mocked services, which an Effect is reading from the "Environment" and then utilising in oder to do its job.

So, since the Effect will be used AS_IS, there is no way to just change the implementation detail of the effect to make it "testable", when we call the unit test from within an async context.

Note, the Task will be executed as expected when the calling context is not async, or if this is macOS, or iOS 14, AND (both) not annotated with @MainActor.

On iOS 15, and having an async context (like when using a Unit test function), the Task's closure will not be called.

So, the question is, what can be done, to make unit testing work in this case, when we using an async context. May be there is no option yet. And we simple can't.

4

I found the solution - it was just an oversight and I missed that XCTest already provides a function for this:

@MainActor func waitForExpectations(timeout: TimeInterval, handler: XCWaitCompletionHandler? = nil)

func testExampleIssueAsync2() async throws {
    let expectCompleted = expectation(description: "completed")
    Task {
        expectCompleted.fulfill()
    }
    await waitForExpectations(timeout: 1)
}

Now, the test function awaits the Task to be completed, however I am not completely sure, this is the right approach.

1 Like