Fixed working directory per test case

Development Swift Testing
1

I'm working on converting tests from XCTest to Swift Testing. While that went well for the most part already, I struggle with test cases that run on files on disk - either in resource folders as part of the repository or in temporary directories.

The idea is to use traits that handle the directory switch. However, due to the async nature of the Swift Testing runtime, the current directory (as a global state) can change while a single test case is executed. Whether a suite runs .serialized, entirely on a single actor or with --no-parallel doesn't change this behavior.

My naïve implementation of a .workingDirectory(path) trait is like:

public struct WorkingDirectory: TestScoping, SuiteTrait, TestTrait {
    fileprivate let path: String

    public func provideScope(
        for _: Test,
        testCase _: Test.Case?,
        performing function: () async throws -> Void
    ) async throws {
        let previousDirectory = FileManager.default.currentDirectoryPath
        #expect(FileManager.default.changeCurrentDirectoryPath(path))
        try await function()
        #expect(FileManager.default.changeCurrentDirectoryPath(previousDirectory))
    }
}

public extension Trait where Self == WorkingDirectory {
    static func workingDirectory(_ path: @autoclosure () -> String) -> Self { Self(path: path()) }
}

Because of the suspension point at try await function(), test execution can be interrupted by another task that might change the directory as well, so that the test later fails due to an incorrect working directory.

Is there a way to run these tests reliably within Swift Testing or is file system access and its async nature inherent incompatible?

2

The current working directory is global mutable state and generally unsafe to rely on in concurrent code. So bear that in mind as you design your Swift project and your tests. If at all possible, don't use the current working directory in your code.

I suspect this PR would be of interest to you as it would let you easily mark tests as dependent on this state.

In the absence of that API, consider placing the affected tests in a single suite and then marking that suite .serialized. If you're finding the current working directory is changing even when your tests are serialized, that indicates that something is changing it, but there's not much more advice I can give you.

3 Likes
3

That PR looks super exciting!

4

That's pretty much what I expected. Thanks for the confirmation the for sharing the PR. I'm also looking forward to it being merged and released!