Why do I need explicit @Sendable to capture a Mutex in a sending closure?

Using Swift
1

Hi all :wave: The following snippet (a quick attempt at parallelizing some CPU-intensive computation) has an error related to capturing a Mutex in a sending closure (the addTask { ... } closure).

I don't fully understand why this is an issue in the first place:

struct Foo {
    
    static func cpuIntensiveWork() async -> Int {
        try! await Task.sleep(for: .seconds(5))
        return Int.random(in: 0...10)
    }
    
    func parallelComputation() async -> Int {
        let fullResult = Mutex<Int>(0)
        
        await withTaskGroup(of: Void.self) { taskGroup in
            taskGroup.addTask { // ❌ Passing closure as a 'sending' parameter risks causing data races between code in the current task and concurrent execution of the closure
                // Note: closure captures 'fullResult' which is accessible to code in the current task
                let localResult = await Self.cpuIntensiveWork()
                fullResult.withLock { state in
                    state += localResult
                }
            }
        }
        
        return fullResult.withLock { $0 }
    }
}

Here fullResult is Sendable, and no other values are captured, which I think means (and this is maybe where my understanding is flawed) the closure is safe to use from any context, and therefore sendable, so it should be fit to be passed to a sending closure parameter. The compiler doesn't seem to be able to realize this though. Explicitly annotating the closure with @Sendable seems to work:

func parallelComputation() async -> Int {
    let fullResult = Mutex<Int>(0)
    
    await withTaskGroup(of: Void.self) { taskGroup in
        taskGroup.addTask { @Sendable in // βœ…
            let localResult = await Self.cpuIntensiveWork()
            fullResult.withLock { state in
                state += localResult
            }
        }
    }
    
    return fullResult.withLock { $0 }
}

But I'm curious, why is the explicit @Sendable annotation required? AFAICT, capturing other sendable values is fine.

(I know there are better methods than Mutex for the code shown here, this is just a tiny example).

2 Likes
2

first i want to highlight some recent similar topics that may be of interest:

  1. Sending, inout sending, Mutex
  2. Sending to TaskGroup child task

now, on to my speculations as to what's going on here...

note that this behavior is not limited to Mutex; you can end up with the same diagnostic from this reduced example with a subset of the Mutex API:

struct NC: ~Copyable {}
extension NC: @unchecked Sendable {}

func f() async {
    let nc = NC()
    await withTaskGroup(of: Void.self) { taskGroup in
        taskGroup.addTask {
                          `- error: passing closure as a 'sending' parameter risks causing data races between code in the current task and concurrent execution of the closure
            _ = nc
                `- note: closure captures reference to mutable let 'nc' which is accessible to code in the current task
        }        
    }
}

now, removing the ~Copyable suppression syntax resolves the issue[1], so presumably the diagnostic has something to do with the type being non-copyable.

consider a similar configuration, where we have a mutable capture of a Sendable type, e.g.

func f() async {
    var i = 0
    await withTaskGroup(of: Void.self) { taskGroup in
        taskGroup.addTask {
                          `- error: passing closure as a 'sending' parameter risks causing data races between code in the current task and concurrent execution of the closure
            _ = i
                `- note: closure captures reference to mutable var 'i' which is accessible to code in the current task
        }
    }
}

the diagnostics are essentially identical, which suggests the region-based isolation checking is perhaps treating these two cases as equivalent. i can imagine why this might happen; a mutable capture of a Sendable type is sort of like wrapping the sendable entity in a (non-sendable) 'box' of sorts. similarly, a non-copyable type can perhaps be thought of as a value contained in a 'mutable box', with some additional requirements about how things can move in and out of it.

it seems like this is probably a bug – a Sendable type (and in particular Mutex) should be able to be passed across these isolation boundaries without issue.

  1. as does the addition of an explicit @Sendable annotation on the closure, as you illustrate in your example. β†©οΈŽ

3

You could put the mutex in a class declaration for your use case

class MutexWrapper<T>: Sendable {
    let mutex: Mutex<T>
    init(value: sending T) {
        mutex = .init(initialValue: value)
    } 
}

func parallelComputation() async -> Int {
    let fullResult = MutexWrapper<Int>(value: 0)
    
    await withTaskGroup(of: Void.self) { taskGroup in
        taskGroup.addTask {
            let localResult = await Self.cpuIntensiveWork()
            fullResult.mutex.withLock { state in
                state += localResult
            }
        }
    }
    
    return fullResult.mutex.withLock { $0 }
}
4

I was just surprised to hit this myself when migrating from Atomics.ManagedAtomic to Synchronization.Atomic:

import Atomics

func Ζ’() async {
    let v = ManagedAtomic(false)
    await withDiscardingTaskGroup { taskGroup in
        taskGroup.addTask { // βœ…
            v.store(true, ordering: .releasing)
        }
    }
}

import Synchronization

func Ζ’() async {
    let v = Atomic(false)
    await withDiscardingTaskGroup { taskGroup in
        taskGroup.addTask { // πŸ›‘ Passing closure as a 'sending' parameter risks causing data races between code in the current task and concurrent execution of the closure
            v.store(true, ordering: .releasing)
        }
    }
}

Marking the closure passed to taskGroup as @Sendable does indeed work around the issue, but I'm surprised the compiler doesn't somehow "forward" the borrow as-is:

func call(_ body: sending @escaping () async -> Void) async {
    await body()
}

func Ζ’1() async {
    let v = Atomic(false)

    // a single layer works
    await call { // βœ…
        v.store(true, ordering: .releasing)
    }
}

func Ζ’2() async {
    let v = Atomic(false)

    // nested `sending` does not...
    await call {
        await call { // πŸ›‘ Passing closure as a 'sending' parameter risks causing data races between code in the current task and concurrent execution of the closure
            v.store(true, ordering: .releasing)
        }
    }
}

func Ζ’3() async {
    let v = Atomic(false)

    // ... unless the inner closure is explicitly `@Sendable`
    await call {
        await call { @Sendable in // βœ…
            v.store(true, ordering: .releasing)
        }
    }
}

Is this something that should be expected to work (eventually), or should the diagnostics here be made more explicit? (/cc @Joe_Groff, maybe?)

5

these all appear to work on the current nightly: Compiler Explorer

2 Likes
6

Ah, nice! Thanks for checking.

1 Like