Can anyone shed some deeper light on why the following code doesn't compile?
import Foundation
import Synchronization
final class A {
let state = Mutex(State())
struct State {
var callbacks: [UUID: () async -> ()] = [:]
}
func a(_ callback: sending @escaping () async -> ()) {
state.withLock {
$0.callbacks[UUID()] = callback // 'inout sending' parameter '$0' cannot be task-isolated at end of function
}
}
}
The error is:
'inout sending' parameter '$0' cannot be task-isolated at end of function
My evidently incomplete mental model is that the closure parameter is statically known to be "disconnected" (by virtue of the sending keyword), and that this should mean that passing the closure into the mutex-protected state does not infect the state with another isolation region. Rather, the disconnected closure gets absorbed into the isolation region of the state. What am I missing?
I think your mental model is right (either that or ours are both wrong in a similar way ). I think it is a bug that this does not work (exhibits A, B, C). Speculating a bit, but I think it may be due to closure captures generally not getting annotated as sending when lowered to SIL (with stuff involving async let being a possible exception).
The specific limitation there is that you can't do things in a closure that can only safely be done once, like transferring a value into a disconnected region. We'd have to know how often the closure is called.
Exactly, the same issue I ran into while trying to replace my custom lock implementation with Mutex. While doing so, I also discovered a race condition involving closures that return a non-Sendable value and are not marked as sending.
@NotTheNHK, I think the issue in OP's example wasn't caused by sending return value, because it can be reproduced with simpified code below.
struct State {
var callback: () async -> ()
}
func foo(_ fn: (inout sending State) -> Void) {}
struct S {
func test(_ callback: sending @escaping () async -> ()) {
foo {
$0.callback = callback
} // error: 'inout sending' parameter '$0' cannot be task-isolated at end of function [#RegionIsolation]
}
}
I agree with OP and @jamieQ that the code should compile. ~IMO "task-isolated" in the diagnostic is incorrect, as demonstrated in the following code. It produce similar diagnostic, but I believe the local variable callback is in disconnected region, not task-isolated.~
class NS {}
struct State {
var callback: NS
}
func foo(_ fn: (inout sending State) -> Void) {}
struct S {
func test() {
let callback = NS()
foo {
$0.callback = callback
} // error: 'inout sending' parameter '$0' cannot be task-isolated at end of function [#RegionIsolation]
}
}
As the above example doesn't use closure, it also indicates that the issue isn't specific to closure. I found @John_McCall's explanation very helpful (IIUC what he said is a general rule and applies to both sending and non-sending closures), but I suspect it doesn't apply to OP's example.
PS: IIUC a sending value or closure shouldn't be transferred inside a closure, but it should be OK to access the value or call the closure in another closure.
class NS { var value = 0 }
func outer() async {
let ns = NS()
await middle { ns.value += 1 }
}
func middle(_ fn: sending @escaping () async -> ()) async {
await inner { await fn() } // sending closure is wrapped in another sending closure
}
func inner(_ fn: sending @escaping () async -> ()) async {
await fn()
}
Sorry, my mistake. The diagnostic is correct. The value is task-isolated inside the closure. So I think the OP's example doesn't work because of inout sending parameter's restriction, which applies to both captured class instances and closures.
foo { // <- these braces delimit a closure expression
$0.callback = callback
}
The compiler is giving a somewhat confusing diagnostic here, but it is correct that this code cannot be allowed to compile absent whole-program information. callback cannot be transferred into $0.callback because it remains referenced by the closure capture. If foo called the closure twice with two different variables, callback could end up referenced by two different regions simultaneously.
Sorry for the late response. Yes, you're correct. Below are notes to future me and others who have the same confusion.
TLDR: Swift currently doesn't support specifying a closure should only run once. As a workaround, a captured value is always task-isolated so that the value can't be transferred to a different isolation at all.
Example scenarios include A) passing a captured value to a function as sending parameter, B) capturing it again in a task closure, C) assigning it to a inout sending parameter, D) returning it as a sending value, and so on.
class NS {}
@concurrent
func send(_ ns: NS) async {}
@concurrent
func send2(_ ns: sending NS) async {}
func test() async {
let ns = NS()
// Scenario A
let a1 = { await send2(ns) } // Not OK
let a2 = { await send(ns) } // OK
// Scenario B
let b1 = { Task { print(ns) } } // Not OK
let b2 = { await { await send(ns)}() } // OK
// Scenario C
let c1: (inout sending NS) async -> Void = { $0 = ns } // Not OK
let c2: (inout NS) async -> Void = { $0 = ns } // OK
// Scenario D
let d1: () async -> sending NS = { ns } // Not OK
let d2: () async -> NS = { ns } // OK
}
People (including me) commonly confuse this because they expect a captured value which was originally in disconnected region or was a sending parameter should be in disconnected region inside the closure and can be transferred further. But this isn't supported yet. The key to avoid the confusion is to remember that, unlike function parameters, captured value can currently only be task-isolated.
EDIT: The above discussion applies only to captured value which was originally in disconnected region. It doesn’t apply to value which was in actor isolation region. The latter is always in actor isolation region.
Oh, you're right. I didn't realize I was compiling in Swift 5 mode.
<source>:23:5: error: 'inout sending' parameter '$0' cannot be task-isolated at end of function [#RegionIsolation]
21 | foo {
22 | $0.callback = replace(&callback, with: {})
23 | }
| |- error: 'inout sending' parameter '$0' cannot be task-isolated at end of function [#RegionIsolation]
| `- note: task-isolated '$0' risks causing races in between task-isolated uses and caller uses since caller assumes value is not actor isolated
24 | }
25 | }
@Nobody1707, I think what you had in mind was probably something like the following. The idea is to put a sending value in a Sendable wrapper struct and pass the wrapper value to closure to avoid using RBI.
struct Disconnected<Value>: @unchecked Sendable {
private var value: Value
init (value: sending Value) {
self.value = value
}
consuming func take() -> sending Value {
return value
}
}
typealias Callback = () async -> Void
struct State {
var callback: Disconnected<Callback>
}
func foo(_ fn: (inout sending State) -> Void) {}
func test(_ callback: Disconnected<Callback>) {
foo {
$0.callback = callback
}
}
The code compiles but it's buggy. The issue is with Disconnected. It must be non-copyable, otherwise the "@unchecked Sendable" is incorrect (that's why compiler fails to catch the potential data race issue).
With the code change, however, the above approach doesn't work because Swift currently doesn't allow consuming a non-copyable value in closure. Even if this is supported in the future, I still expect compiler won't allow similar code because if there was an approach to transfer a captured value to a different isolation (it'a inout sending parameter in this case), it would cause data race as John explained.
EDIT: on a second thought, if it's supported to consume a captured non-copyable value in a closure in the future, that closure will have to run only once. So the wrapper approach may work in the future.
It's a little annoying that you have to wrap the Optional in another type to get this to work, but it's nice that you can make this work. Isn't swap(newValue: nil) just take() with extra steps? Nevermind, I just noticed that take returns the unwrapped value.
Indeed, I forgot the Optional.take workaround. So, with the code you suggested, it's possible to implement a run-once closure which checks and crashs at runtime. Given how often the question pops up, I'm surprised I didn't see this general solution before. Below I apply it to all my previous tests. Note testB. To use this approach to pass a sending value to a nested sending closure, multiple Disconnected wrappers are required, one at each level. This is because Disconnected reference is mutable.
class NS {}
@concurrent
func send(_ ns: NS) async {}
@concurrent
func send2(_ ns: sending NS) async {}
func testA(_ ns: sending NS) {
var dis = Disconnected(value: Optional(ns))
let fn = {
let ns = dis.swap(newValue: nil)!
await send2(ns)
}
}
func testB(_ ns: sending NS) {
var dis1 = Disconnected(value: Optional(ns))
let fn = {
let ns = dis1.swap(newValue: nil)!
var dis2 = Disconnected(value: Optional(ns))
Task {
let ns = dis2.swap(newValue: nil)!
print(ns)
}
}
}
func testC(_ ns: sending NS) {
var dis = Disconnected(value: Optional(ns))
let fn: (inout sending NS) async -> Void = {
let ns = dis.swap(newValue: nil)!
$0 = ns
}
}
func testD(_ ns: sending NS) {
var dis = Disconnected(value: Optional(ns))
let fn: () async -> sending NS = {
let ns = dis.swap(newValue: nil)!
return ns
}
}
testB emulates real world code like this:
func test(fn: sending @escaping () async -> Void) async {
await withTaskGroup(of: Void.self) { group in
group.addTask { await fn() } // This doesn't compile
}
}
A minor comment on Disconnect code. Is it necessary to use consuming and sending together? I see this is from the original code in AsyncAlgorithms, but IMO it's overkilling because
consuming is pretty much useless and misleading for copyable value because of implicit copy (note Value is copyable in most real world cases).
More importantly, sending is sufficient. In my understanding the main difference between sending and consuming is the former applies to a region. In Disconnect's init and swap APIs, what matter is the entire region not a single value.
PS: I think this is an example why it's not ideal to depend on Owership in concurrency code.
EDIT: never mind, I figured it out. consuming is required when Value is non-copyable.
). I think it is a bug that this does not work (exhibits