Recently, I had isolated closures in mind and how they interact with non-isolated asynchronous functions (@concurrent), and found that the compiler naively determines that sending the closure (value) to a different isolation region is unsafe, since the value’s type (() async -> Void / () -> Void) is non-Sendable:
@concurrent
func f(_ body: () async -> Void) async {}
@SomeGlobalActor
func isolatedToGlobalActor() async {
await f {} // error: sending value of non-Sendable type '() async -> ()'
}
func isolatedToActorInstance(_ `actor`: isolated some Actor) async {
await f { _ = `actor` } // error: sending value of non-Sendable type '() async -> ()'
}
Note: This also applies to explicitly isolated closures with an isolated parameter. The solution here should be straightforward: infer @Sendable as is done with explicitly global-actor-isolated closures and functions.
My question is, what do we think the fix should be? Obviously, we could simply leave it as-is. My first thought was that we could infer @Sendable, as we already do with explicit global-actor-isolation. However, this would break the implicit invariant that @Sendableclosures are statically non-isolated.[1]
Then I thought we could use @isolated(any), but a function type annotated with @isolated(any) does not imply @Sendable either, which I think was the wrong decision. Alternatively, we could perform better analysis. If the non-Sendable value is a function type, we could check whether that type is statically isolated.
Personally, I think expanding upon @isolated(any) would be the best choice.
Let’s disregard @_inheritActorContext and @_inheritActorContext(always) for now. ↩︎
This is a topic of great interest to me, and I'm happy to see some discussion around it.
In the first case with await f {}, this looks to me like RBI is unable to determine that sending the function into f is safe, even though it is. My gut feeling is this a limitation and not (necessarily) a bug.
The second case is much more involved. It's difficult for me to make up my mind on what should happen. Closure isolation rules are complex and doubly-so when isolated parameters are involved.
Exactly what ways are you thinking about expanding @isolated(any) ?
@isolated(...) should be able to represent specific isolation.
@isolated(...) should imply @Sendable when the closure is actually isolated.
Move away from implicit static isolation, so that functions and closures that are statically isolated reflect their isolation explicitly. Currently, only explicit global-actor isolation or explicit actor-instance isolation (isolated keyword) is visible in the type system.
This would require some fairly complex and esoteric type conversion (are there any languages that support something like this?), where an isolated closure or function could “morph” into different types depending on the caller’s isolation context:
@concurrent
func f(_ body: () async -> Void) async {
// Call to `f` in `A.work` // Fine since `body` is `async`
print(type(of: body)) // @Sendable @isolated(A) () -> Void AKA () async -> Void
}
func ff(_ body: () -> Void) {
// Call to `ff` in `A.work` // Fine since `ff` does not cross any isolation boundary
print(type(of: body)) // @Sendable @isolated(A) () -> Void AKA () -> Void
}
actor A {
func work() async {
let c: () -> Void = { _ = self } // @Sendable @isolated(A) () -> Void AKA () -> Void
await f(c) // @Sendable @isolated(A) () -> Void AKA () async -> Void
f(c) // @Sendable @isolated(A) -> Void AKA () -> Void
}
}
I hope anyone understands what I’m trying to express :D
Could you elaborate? Both closures passed to f are non-isolated from the type system’s perspective.
Edit: Obviously, solving this in RBI should be much simpler than this.
Yeah, I also think the first case is not a bug, because under the current rule, the closure {} is deduced to be @SomeGlobalActor isolated, that makes it unqualified to be in a disconnected region, so it cannot not be sent in to f. I'm really sympathetic to this case.
To make it work, we must erase the isolation manually, one way is await f { @concurrent in } ...
A minor change causes it fails to compile, with similar diagnostic in OP's original post. IMO this indicates the root cause isn't caused by sendability but by type inference.
- let fn: @MainActor () -> Void = {}
+ let fn: () -> Void = {}
BTW, I find the following code compiles but I don't think they should:
@globalActor
actor SomeGlobalActor {
static let shared = SomeGlobalActor()
}
@concurrent
func f(_ body: @MainActor () async -> Void) async {}
@SomeGlobalActor
func isolatedToGlobalActor() async {
await f {}
}
func isolatedToActorInstance(_ `actor`: isolated some Actor) async {
await f { _ = `actor` }
}
The only difference from @NotTheNHK's original code is:
EDIT: I figured out why it's OK to convert SomeGlobalActor isolated closure to MainActor isolated closure. It's because compiler automatically adds a wrapper.
Can you elaborate your idea of SE-0434 more? Why do you think the {} in isolatedToGlobalActor from OP should not be global actor isolated?
The result of isolation inference can be verified actually. For example if we put the code from OP into this explorer, the output contains:
Beginning processing: $s6output21isolatedToGlobalActoryyYaFyyYaXEfU_
Demangled: closure #1 in isolatedToGlobalActor()
╾++++++++++++++++++++++++++++++╼
Dump:
// closure #1 in isolatedToGlobalActor()
// Isolation: global_actor. type: MainActor
Edit: Ah I may understand what you are trying to say here, you are asking: since the closure is (inferred) MainActor isolated, it should be @Sendable, and RBI analysis should not be needed.
This is a very interesting question, and has been brought up several times. My understanding is that this is a chicken-and-egg problem: on one hand, the isolation of a closure is affected by whether the closure is constrained to be @Sendable, and on the other hand, its isolation can in turn affect its sendability.
There isn't a perfect solution now. As far as I can tell, the application of SE-0434 on closures is often limited to closures that are explicitly global actor isolated.
Yes, that's what I meant. I wasn't aware of the difference and figured it out just now (OP actually mentioned it in the original post, sorry for the distraction).
Did you mean 1) changing f to the following so that the closures in isolatedToGlobalActor and isolatedToActorInstance are inferred with @isolated(any) isolation, and 2) adding Sendable support to @isolated(any) closures?
Once you implement item 2, your original code (excluding isolatedToGlobalActor) would work too. That might lead to user questions like "what's the use of @isolated(any) if the code works with or without it?". A possible answer is that the one with @isolated(any) works like a generic function and the one without it works as a wrapper. Both works, but which one is the recommended?
In current implementation if a closure captures an isolated parameter its isolation is inferred as actor isolated, which IIUC isn't the same as @isolated(any). The inference rules in your new implementation will be more complex and need to take into account not only if the closure captures isolated parameter but also the isolation of the contextual type.
isolatedToGlobalActor function compiles if it's nonisolated. I checked SIL of the closure inside isolatedToGlobalActor(). It didn't have @Sendable in its signature. So the code worked because of RBI? That lead to a question: why RBI works in this case but not if isolatedToGlobalActor is global actor isolated?
@concurrent
func f(_ body: () async -> Void) async {}
@globalActor
actor SomeGlobalActor {
static let shared = SomeGlobalActor()
}
- @SomeGlobalActor
func isolatedToGlobalActor() async {
await f {}
}
Then I made the following change. My experiment showed that body ran in MainActor. This indicated that compiler inferred the isolation of the closure {} inside isolatedToGlobalActor as nonisolated(nonsending) directly, rather than using the usual wrapping approach (that is, a @concurrent closure wrapped inside a nonisolated(nonsending) closure). This is the behavior that @bjhomer would like to have in another thread, but the code didn't have sending or @Sendable hack. I wonder what caused the difference?
Sorry for taking so long to respond here. These are complex topics!
One thing I have found very tricky is the relationship between @isolated(any) , @Sendable and sending. In the @isolated(any) proposal, it is mentioned that originally the intention was to make @isolated(any) imply @Sendable. However, this was removed because RBI and sending were starting to take shape. But due to unfortunate timing, this was never revisited.
@isolated(any) functions really need to be sending at a minimum, and I believe that should be implied. I hope to see some movement here before too long.
Now, the proposal also goes into a little detail on what it would mean for an explicit type to be included as an argument to @isolated. And my understanding, which is superficial, is that such a change would be extremely complex.
Now, if I'm following what you are saying, I think you are heading towards the idea that @isolated(MainActor) would be equivalent to @MainActor. But, there's a problem. Because, at least today, it is possible to create new instances of type MainActor that are not MainActor.shared.
As for the case of isolated parameters, this is an area of great complexity and subtly. Closure captures do affect static isolation, but I find this to be both unintuitive and problematic in a number of situations. I'm not saying what you've suggested is unreasonable - more that how things are implemented today is unreasonable.
This would indeed be a problem, but I don’t see a reason why we can’t use @isolated(SomeInstance) for instance-isolation and @isolated(@SomeGlobalActor) for global-isolation. It should also be fairly straightforward to warn when someone writes @isolated(SomeGlobalActor) instead of @isolated(@SomeGlobalActor).
First of all, I want to retract my earlier statement that my idea is “fairly complex and esoteric.” After some more thought, I believe this would be simpler to understand and more explicit than what we have currently. Specifically, if you have an actor isolated closure of type () -> Void (in its concise form), then you can pass and call that closure synchronously from the same actor, while calling it from a different isolation domain, requires await.
Instance-Isolation:
actor A {
func getAIsolatedClosure() -> () -> Void {
return { @isolated(self) in }
// The user simply specifies the function signature as the return type.
// But since we explicitly isolate the closure to this instance, its "expanded"
// type is @isolated(A) @Sendable () -> Void.
}
func callAIsolatedClosureFromA() {
let c = getAIsolatedClosure()
c() // Okay
// Since this is the same isolation domain, we automatically "qualify"
// a type of: @isolated(A) @Sendable () -> Void to run synchronously.
}
}
func passIsolatedClosureToNonIsolatedContext(_ c: () -> Void) {
c()
// error: Calling @isolated(A) closure `c` from a different isolation domain
// requires `c` to be called from an `async` context and marked with `await`
}
func passIsolatedClosureToNonIsolatedAsyncContext(_ c: () async -> Void) async {
await c() // Okay
}
@concurrent func nonIsolatedContext() async {
let a = A()
passIsolatedClosureToNonIsolatedContext(await a.getAIsolatedClosure())
// It's okay to pass the closure, but calling it requires `await`
await passIsolatedClosureToNonIsolatedAsyncContext(a.getAIsolatedClosure())
// Okay
}
Global-Isolation:
@globalActor
actor GA {
static let shared = A()
}
@isolated(GA) // warning: instance-isolation @isolated(GA) is different from global-isolation `@isolated(@GA)`
func instanceGAIsolated() {}
@isolated(@GA)
func globalGAIsolated() {
instanceGAIsolated()
// error: calling instance-isolated (@isolated(GA)) function
// `instanceGAIsolated` from global isolation @isolated(@GA) crosses
// isolation boundary. requires await ...
}
That’s a good question, and honestly I haven’t given @isolated(any) much thought yet. The idea is still very rough, with many outstanding questions, especially around its interaction with existing syntax. But I like the idea more and more :D
Slightly off-topic...I didn't expect foo2(fn) would compile, because IMO foo2 takes a @concurrent () async -> Void closure. From what I can tell in SIL (I'm not familiar with it but I think I read it correctly in this case), the thunk that converts fn to the argument of foo2 hops to MainActor actor directly. This is the case even after I changed foo2's parameter type to @concurrent () async -> Void explicitly.
The above conversion also works for actor-isolated closures. It's only that actor-isolated closure is non-Sendable. The following works. Note bar1 and bar2 are synchronous functions because the closure is non-Sendable. The conversion isn't really useful because the closure can't be transferred to a different isolation. Actually, the thunks that perform the conversion don't have hop_to_executor instructions, which would cause issue if the closure was Sendable.
One main difference between global actor isolated closure and actor isolated closure is that global actor's shared instance can be accessed globally. My initial thought was that meant it's easiler to implement Sendable for global actor isolated closure because the shared instance is always accessible when a synchronous closure is transferred to a different isolation and needs to be converted to an async function. But then I realized that it might be feasible for actor isolated closure too because the conversion must occur at the moment a closure leaves the actor's isolation, which means that actor is still in scope and accessible. Just my random thought.
But I don't understand the purpose of the explicit annotation @isolated(self) in your example. What does it enable?
As shown in my examples above, this isn't specific to your solution. It exists in the current implementation too.
EDIT: ah, I get it. The purpose of your explicit annotation is to help infer isolation of the closure. But I think what's equivalently important is the isolation and sendability of the type of closure returned by getAIsolatedClosure():
`func getAIsolatedClosure() -> () -> Void`
I think your annotation only affects the type of the closure defined in the function body but doesn't affect the type seen by getAIsolatedClosure()'s caller?
BTW, isn't annotation like this part of the ongoing closure isolation control project?