Mixing async/await code with waitForExpectations in tests causes hang

Using Swift
,
1

I've encountered a problem with awaiting expectations in XCTestCase when testing async code. I want to test code that is supposed to execute operations in background and notify me about the result.

The problem is that once background operation finishes nothing happens. Instead my expectation times out after the specified time. Below is a simplified model and test case that simulates my code. Running it displays only "before doSomethingAsync" and "in doSomethingAsync" messages in console. It looks to me like execution is not returning to main actor while waitForExpectations is running.

@MainActor
final class ViewModel: ObservableObject {
    let didRun: () -> Void

    init(didRun: @escaping () -> Void) {
        self.didRun = didRun
    }

    func doSomething() {
        Task {
            print("before doSomethingAsync")
            await doSomethingAsync()
            print("after doSomethingAsync")
            didRun()
        }
    }

    nonisolated func doSomethingAsync() async {
        print("in doSomethingAsync")
    }
}

final class AsyncTestTests: XCTestCase {
    func testDoSomething() async throws {
        let exp = expectation(description: "task finished")
        let viewModel = await ViewModel(didRun: {
            exp.fulfill()
        })

        await viewModel.doSomething()

        await waitForExpectations(timeout: 5)
    }
}

Is there anything that I'm doing incorrectly here or perhaps is it an issue with waitForExpectations function?

I'm using Xcode 14.2 with iOS 16.2 simulator.

2 Likes
2

I noticed that there is a note in documentation that says “Clients should not manipulate the run loop while using this API.”. Perhaps this means that something like I’m trying to do is not supported. If that is the case is there an alternative approach?

3

Would something like this work?

final class AsyncTestTests: XCTestCase {
    func testDoSomething() async throws {
        var didRun = false
        let viewModel = await ViewModel(didRun: {
            didRun = true
        })

        await viewModel.doSomething()

        XCTAssertTrue(didRun)
    }
}
4

Unfortunately that won’t work. doSomething launches a Task that won’t complete until later. There is an await in test code only because it runs outside of MainActor. I could annotate whole test with @MainActor like below but that won’t resolve the issue.

final class AsyncTestTests: XCTestCase {
    @MainActor
    func testDoSomething() async throws {
        let exp = expectation(description: "task finished")
        let viewModel = ViewModel(didRun: {
            exp.fulfill()
        })

        viewModel.doSomething()

        waitForExpectations(timeout: 5)
    }
}
5

Per Xcode 14.3 Beta release notes Apple confirmed that it was an issue in these functions and provided replacement.

Fixed: XCTestCase.wait(for:timeout:enforceOrder:) and related methods are now marked unavailable in concurrent Swift functions because they can cause a test to deadlock. Instead, you can use the new concurrency-safe XCTestCase.fulfillment(of:timeout:enforceOrder:) method. (91453026)

3 Likes
6

If you’re still curious about the root cause, I think the issue with waitForExpectations in async code is that calling it blocks the main thread/queue. Since the async tasks need to run on the main queue too, they wait for the queue to clear up, but the queue never clears up because the XCTestExpectation blocking it is waiting for the async tasks — creating a deadlock.

In general you should avoid blocking any execution context that’s running async code (whether that’s the main thread or a global async executor) because that violates the forward progress invariant. There’s a great video on this from WWDC 2021: see Swift concurrency: Behind the scenes - WWDC21 - Videos - Apple Developer.

2 Likes
7

@Tunous
Did using fulfillment api helped you here eventually?

8

Yes the issue doesn't happen when using fulfillment api.

1 Like