Mock URLProtocol with strict swift 6 concurrency

Hi all. I am trying to mock some network calls by using URLProtocol, I'm also trying to do this in a strict swift 6 compliant way... I've tried wrapping a request handler in an actor and even tried just using a hash table of URLs and responses... I either get 'passing closure as a 'sending' parameter risks causing data races between code in the current task and concurrent execution of the closure' if I wrap the request handler in an actor then use it in a subclass of URLProtocol or I get 'static property 'mockResponses' is not concurrency-safe because it is nonisolated global shared mutable state' when using a hash table inside the subclass. Has anyone successsfully set this up with strict swift 6? I tried annotating with nonisolated(unsafe) as well and that didn’t work.


import Foundation

actor RequestHandlerStorage {
    private var requestHandler: (@unchecked @Sendable (URLRequest) async throws -> (HTTPURLResponse, Data))?

    func setHandler(_ handler: @unchecked @Sendable @escaping (URLRequest) async throws -> (HTTPURLResponse, Data)) async {
        requestHandler = handler
    }

    func executeHandler(for request: URLRequest) async throws -> (HTTPURLResponse, Data) {
        guard let handler = requestHandler else {
            throw MockURLProtocolError.noRequestHandler
        }
        return try await handler(request)
    }
}

final class MockURLProtocol: URLProtocol {

    private static let requestHandlerStorage = RequestHandlerStorage()

    static func setHandler(_ handler: @unchecked @Sendable @escaping (URLRequest) async throws -> (HTTPURLResponse, Data)) async {
        await requestHandlerStorage.setHandler { request in
            try await handler(request)
        }
    }

    func executeHandler(for request: URLRequest) async throws -> (HTTPURLResponse, Data) {
        return try await Self.requestHandlerStorage.executeHandler(for: request)
    }

    override class func canInit(with request: URLRequest) -> Bool {
        return true
    }

    override class func canonicalRequest(for request: URLRequest) -> URLRequest {
        return request
    }

    override func startLoading() {

        Task {
            do {
                let (response, data) = try await self.executeHandler(for: request)
                client?.urlProtocol(self, didReceive: response, cacheStoragePolicy: .notAllowed)
                client?.urlProtocol(self, didLoad: data)
                client?.urlProtocolDidFinishLoading(self)
            } catch {
                client?.urlProtocol(self, didFailWithError: error)
            }
        }

    }

    override func stopLoading() {}
}


enum MockURLProtocolError: Error {
    case noRequestHandler
    case invalidURL
}
1 Like

You've run into one of the rarer but more challenging problems - inheritance with an ObjC type that doesn't support concurrency well.

(I'm sure you've noticed that @unchecked is not used in this way, so skipping that.)

The immediate problem is that startLoading a) has no isolation but b) participates in concurrency via Task. When a type has no isolation, getting it to work with concurrency is much harder. But, in this case, all of the safe tools I can think of to fix this involve changing the definition of startLoading, which we cannot do, because that's not under our control.

Right now, you can make this work by marking MockURLProtocol as @unchecked Sendable. However, it does not look like URLProtocol is actually thread-safe? So, you'll have to be particularly careful with such an opt-out, as the inheritance opens the doors for even more traps that a regular unchecked type might.

(you may also want to keep your eye on the "isolated overloads" concept from [Prospective Vision] Improving the approachability of data-race safety, because I think it could help in this case)

1 Like

Thanks!! The goal is to use for testing scenarios. I am thinking that if I don't try to run the tests in parallel, it should be fine. Hopefully with swift 6, more objc code can be laid to rest. I'll try the @unchecked Sendable and see how it goes.

1 Like

The following compiles and the tests pass

import Foundation

actor RequestHandlerStorage {
    private var requestHandler: ( @Sendable (URLRequest) async throws -> (HTTPURLResponse, Data))?

    func setHandler(_ handler: @Sendable @escaping (URLRequest) async throws -> (HTTPURLResponse, Data)) async {
        requestHandler = handler
    }

    func executeHandler(for request: URLRequest) async throws -> (HTTPURLResponse, Data) {
        guard let handler = requestHandler else {
            throw MockURLProtocolError.noRequestHandler
        }
        return try await handler(request)
    }
}

final class MockURLProtocol: URLProtocol, @unchecked Sendable {

    private static let requestHandlerStorage = RequestHandlerStorage()

    static func setHandler(_ handler: @Sendable @escaping (URLRequest) async throws -> (HTTPURLResponse, Data)) async {
        await requestHandlerStorage.setHandler { request in
            try await handler(request)
        }
    }

    func executeHandler(for request: URLRequest) async throws -> (HTTPURLResponse, Data) {
        return try await Self.requestHandlerStorage.executeHandler(for: request)
    }

    override class func canInit(with request: URLRequest) -> Bool {
        return true
    }

    override class func canonicalRequest(for request: URLRequest) -> URLRequest {
        return request
    }

    override func startLoading() {

        Task {
            do {
                let (response, data) = try await self.executeHandler(for: request)
                client?.urlProtocol(self, didReceive: response, cacheStoragePolicy: .notAllowed)
                client?.urlProtocol(self, didLoad: data)
                client?.urlProtocolDidFinishLoading(self)
            } catch {
                client?.urlProtocol(self, didFailWithError: error)
            }
        }

    }

    override func stopLoading() {}
}


enum MockURLProtocolError: Error {
    case noRequestHandler
    case invalidURL
}
3 Likes

However, it does not look like URLProtocol is actually thread-safe?

That threading rules for URLProtocol are weird, poorly documented, and intimately tied to the run loop )-:


Folks, if you’re doing anything serious with URLProtocol, I recommend that you carefully work through the notes in Read Me About CustomHTTPProtocol.

Share and Enjoy

Quinn “The Eskimo!” @ DTS @ Apple

8 Likes

Thanks to your suggestions regarding MockURLProtocol and Actor, I’ve managed to build a test environment in Xcode 16 using the new Swift 6 concurrency features. I’ve adapted the setup slightly to fit my needs, introducing a HandlerType enum in our actor RequestHandlerStorage to handle success and failure cases separately.

Here’s the tricky part: when I run the tests individually, they all pass. However, running them together results in a failure. Specifically, a test expected an error to be thrown (e.g., when adding a URLResponse instead of an HTTPURLResponse), but instead, the test passes with success unexpectedly. The output is:

Expectation failed: an error was expected but none was thrown. Instead, the result was (12 bytes, <NSHTTPURLResponse: 0x6000032fde60> { URL: http://any-url.com/ } { Status Code: 200, Headers ...

This issue doesn’t occur when running the tests separately.

Tests:

import Testing
import CodingFeed

@Suite("HTTP Client Session")
struct URLSessionHTTPClientTests {
    
    @Test("Get from URL performs a GET request with URL", arguments:
        [TestHelpers.Network.anyURL()],
        [TestHelpers.Network.makeItemJSON([])]
    )
    func test_perforsGETRequestWithURL(url: URL, data: Data) async throws {
        let sut = makeSUT()
        
        await MockURLProtocol.setSuccessHandler { request in
            #expect(request.url == url)
            #expect(request.httpMethod == "GET")
            return (TestHelpers.Network.makeResponse(for: url), data)
        }
        
        _ = try? await sut.get(from: url)
    }
    
    @Test("Get from URL fails when response is not HTTP response", arguments:
        [TestHelpers.Network.anyURL()],
        [TestHelpers.Network.makeItemJSON([])]
    )
    func test_getFromURL_failsOnNonHTTPResponse(url: URL, data: Data) async throws {
        let sut = makeSUT()
        
        await MockURLProtocol.setFailureHandler { _ in
            let nonHTTPResponse = URLResponse(
                url: url,
                mimeType: nil,
                expectedContentLength: 0,
                textEncodingName: nil
            )
            return (nonHTTPResponse, data)
        }
        
        await #expect(throws: (any Error).self) {
            try await sut.get(from: url)
        }
    }
    
    @Test("Get from URL returns data and HTTP response", arguments:
        [TestHelpers.Network.anyURL()],
        [TestHelpers.Network.makeItemJSON([])]
    )
    func test_getFromURL_returnsDataAndHTTPResponse(url: URL, data: Data) async throws {
        let sut = makeSUT()
        
        await MockURLProtocol.setSuccessHandler { _ in
            return (TestHelpers.Network.makeResponse(for: url), data)
        }
        
        let (receivedData, receivedResponse) = try await sut.get(from: url)
        
        #expect(receivedData == data)
        #expect(receivedResponse.url == TestHelpers.Network.makeResponse(for: url).url)
        #expect(receivedResponse.statusCode == TestHelpers.Network.makeResponse(for: url).statusCode)
    }
    
    // MARK: Helpers
    
    private func makeSUT() -> HTTPClient {
        let configuration = URLSessionConfiguration.ephemeral
        configuration.protocolClasses = [MockURLProtocol.self]
        let session = URLSession(configuration: configuration)
        return URLSessionHTTPClient(session: session)
    }
    
    private actor RequestHandlerStorage {
        enum HandlerType {
            case success((@Sendable (URLRequest) async throws -> (HTTPURLResponse, Data)))
            case failure((@Sendable (URLRequest) async throws -> (URLResponse, Data)))
            case none
        }
        
        private var currentHandler: HandlerType = .none
        
        func setSuccessHandler(_ handler: @Sendable @escaping (URLRequest) async throws -> (HTTPURLResponse, Data)) async {
            currentHandler = .success(handler)
        }
        
        func setFailureHandler(_ handler: @Sendable @escaping (URLRequest) async throws -> (URLResponse, Data)) async {
            currentHandler = .failure(handler)
        }
        
        func executeHandler(for request: URLRequest) async throws -> (URLResponse, Data) {
            switch currentHandler {
            case .success(let handler):
                return try await handler(request)
            case .failure(let handler):
                return try await handler(request)
            case .none:
                throw TestHelpers.Network.anyError()
            }
        }
    }

    private final class MockURLProtocol: URLProtocol, @unchecked Sendable {
        private static let storage = RequestHandlerStorage()
        
        static func setSuccessHandler(_ handler: @Sendable @escaping (URLRequest) async throws -> (HTTPURLResponse, Data)) async {
            await storage.setSuccessHandler(handler)
        }
        
        static func setFailureHandler(_ handler: @Sendable @escaping (URLRequest) async throws -> (URLResponse, Data)) async {
            await storage.setFailureHandler(handler)
        }
        
        func executeHandler(for request: URLRequest) async throws -> (URLResponse, Data) {
            return try await Self.storage.executeHandler(for: request)
        }
        
        override class func canInit(with request: URLRequest) -> Bool {
            return true
        }
        
        override class func canonicalRequest(for request: URLRequest) -> URLRequest {
            return request
        }
        
        override func startLoading() {
            Task {
                do {
                    let (response, data) = try await self.executeHandler(for: request)
                    client?.urlProtocol(self, didReceive: response, cacheStoragePolicy: .notAllowed)
                    client?.urlProtocol(self, didLoad: data)
                    client?.urlProtocolDidFinishLoading(self)
                } catch {
                    client?.urlProtocol(self, didFailWithError: error)
                }
            }
        }
        
        override func stopLoading() {}
    }
}

and here is what we are actually testing:

import Foundation

public protocol HTTPClient {
    func get(from url: URL) async throws -> (Data, HTTPURLResponse)
}

public class URLSessionHTTPClient: HTTPClient {
    private let session: URLSession
    
    public init(session: URLSession = .shared) {
        self.session = session
    }
    
    private struct UnexpectedValueRepresentation: Error {}
    
    public func get(from url: URL) async throws -> (Data, HTTPURLResponse) {
        let (data, response) = try await session.data(from: url)
        
        guard let httpResponse = response as? HTTPURLResponse else {
            throw UnexpectedValueRepresentation()
        }
        
        return (data, httpResponse)
    }
}

I’m working on implementing a modular approach while keeping SOLID principles in mind. However, I’m still getting familiar with Modern Concurrency in Swift.

Here are the steps I’ve tried so far:
• Clearing RequestHandlerStorage before and after each test – no success.
• Using a defer block to ensure cleanup at the end of a function’s scope, but I dropped this approach as defer blocks cannot contain async calls in Swift – no success.
• Registering and unregistering MockURLProtocol before and after each test – no success.

Despite these efforts, the issue persists. Does anyone have suggestions or insights on how to resolve this?

Thank you in advance for your help!

May I ask, why you are even using a custom URLProtocol here? You already have abstracted your HTTPClient. Wouldn't it be easier to create a MockHTTPClient and return it in your makeSUT()?

2 Likes

I haven’t caught up completely on this thread, but I wanted to point out that swift testing runs tests in parallel by default. If you’re using URLProtocol with only a single callback handler then you will suffer from races.

You can resolve this by marking these tests with the serialized trait:

But my advice is to rearchitect your use of URLSession such that your tests do not rely on URLProtocol. In the past I’ve handled this by mocking URLSession itself and injecting that, but there are other approaches that would also work.

3 Likes

The reason for using URLProtocol instead of a simple MockHTTPClient is that we're specifically testing the URLSessionHTTPClient implementation, not just the HTTPClient protocol contract.

While we do have the HTTPClient protocol abstraction (which is great for high-level testing), when testing URLSessionHTTPClient we need to verify:

  1. The actual URLSession behavior and integration (which only way I know is subclassing old legacy URLProtocol and if there is any better approach with "modern" swift concurrency will be really glad to hear it)
  2. Correct handling of URLSession-specific responses and errors
  3. Proper configuration of URLSession requests

Even if we switch to Alamofire or Moya in the future, they still use URLSession under the hood, so these tests remain valuable for network layer.

1 Like

Thanks for pointing that out! I’ve actually read the Apple documentation on Swift testing a few times, specifically trying to figure out how to run tests non-parallel since I discovered the race condition. I did read about test suites, but I completely missed the traits section.. after some frustration, I finally Googled the problem and found this post—which was the key to solving it. Huge thanks for this! Let’s just say 100% of my code coverage success now rests on your shoulders. :sunglasses:

Regarding the advice to rearchitect away from URLProtocol: while I see the value in mocking URLSession itself, I prefer sticking to this legacy class for now URLProtocol It’s the only way for me to observe API calls without actually making them, and I like keeping things closely tied to the URLSession framework-specific behavior. Open to new suggestions and advices if any one know any better approach how to test not only abstraction of interface/protocol, but for testing the actual URLSession implementation details!

Thanks again for the insight—it saved me a ton of time and frustration!

You are right. I specifically removed parallel from my testing so that I don't hit any race conditions.

1 Like

when testing URLSessionHTTPClient we need to verify … actual URLSession behavior

That’s where things fall apart with the URLProtocol approach. Folks do it because they want to test how their code interacts with URLSession. However, URLSession behaves very differently in the face of a URLProtocol. So, you’re not testing how your code interacts with the session’s behaviour, you’re testing how your code interacts with your understanding of the session’s behaviour, which isn’t all that helpful IMO.

Share and Enjoy

Quinn “The Eskimo!” @ DTS @ Apple

6 Likes

Hi Quinn,

Thanks for sharing your perspective - as you and other folks keep mentioning the same thing, it definitely made me reflect on the limitations of using URLProtocol. I discovered years ago that it doesn't fully replicate the real-world behavior of URLSession, and this gap can lead to potential blind spots in testing.

I'm currently using URLProtocol as part of a layered testing strategy. While it doesn't capture all URLSession nuances, it allows me to efficiently test core networking logic like request composition, response handling, and error scenarios in isolation. This gives me fast feedback during development over my remote module as separated macOS frameworks where the target is to include only Foundation, as the codebase will be shared across different Apple devices that will consume from that framework.

That said, I see URLProtocol as a useful tool for specific scenarios, particularly when I want to isolate and test how my client interacts with mocked requests and responses without depending on real network interactions. While it may not cover all of URLSession's nuances, it's been valuable for ensuring my code constructs and handles requests/responses as expected across the Apple ecosystem.

I agree that pairing these tests with integration tests against a real server or URLSession in its natural state would provide a more comprehensive picture of behavior (which is part of my plan for CI/CD). I'll definitely keep your advice in mind for refining my testing strategy to better account for potential gaps in emulating real-world URLSession behavior, especially given this will be a foundation for part of my multi-device application. At this level of Foundation framework testing, I'm not aware of other approaches that would provide similar capabilities, open to hear more constructive feedbacks from community.

Thanks again for pointing this out! I really appreciate the insight!

Best regards,
Georgi

Thanks for the perspective. What alternatives do you suggest?

I generally recommend that you introduce an abstraction layer between your request-response code and your network transport. You can then:

  • Test the request-response code with an in-memory transport

  • Test the network transport against a server

That raises the question of what that abstraction layer looks like. Rather than try to mock URLSession itself, with its large and complex API, I recommend that you create a much smaller interface that’s focused on your product’s specific needs. For many apps that interface can be as simple as a throwing async function that takes a URLRequest and returns an HTTPURLResponse.

Share and Enjoy

Quinn “The Eskimo!” @ DTS @ Apple

6 Likes