Capturing functions vs casting functions

Evolution Discussion
1

Thanks to SE-0434, it is possible to capture non-sendable function in an actor-isolated (and thus sendable) closure if it is isolated to the current context.

If closure does nothing but calling the captured function, then this is effectively casting.

But direct casting is not allowed. Even through pretty much the same logic applies here.

@MainActor
func test(completion: @escaping () -> Void) {
    let a: @MainActor () -> Void = { completion() } // ok
    let b: @MainActor () -> Void = completion // error:  using non-Sendable parameter 'completion' in a context expecting a '@Sendable' closure
}

Any reasons why this restriction could not be lifted?

2 Likes
2

I'm not sure why that cast wasn't supported! But it's probably for the best.

let a: @MainActor ... is not actually data race safe in light of Deinit for non-Sendable types unfortunately. completion may capture non-Sendable values with deinits, but will be released synchronously from any isolation by the capture stored in a. b has the same issue, but even worse, completion wasn't even "aware" that it was given this isolation.

I'm working on a writeup about all the permutations of this hole and how we might close it, but it seems very likely it will involve a mechanism for isolating the deinits of closures based on their isolation. To do that for b would require the compiler to wrap completion in a global actor isolated thunk, making it equivalent to a (point being, we can't just cast it, it is a representation change). We could support doing that implicitly though.

This will crash at runtime due to concurrent mutation:

final class Log { var text = "" }

final class Recorder {
  let log: Log
  init(_ l: Log) { log = l }
  deinit { log.text += "x" }
}

@MainActor func makeSendableClosureUsingGAI(_ r: Recorder) -> @MainActor () -> Void {
  { _ = r }
}

let n = 10_000

@MainActor func dataRace() async {
  let log = Log()
  await withDiscardingTaskGroup { g in
    for _ in 0..<n {
      let sendableWrapper = makeSendableClosureUsingGAI(Recorder(log))
      g.addTask { _ = sendableWrapper }
    }
  }
  print(log.text.count, "expected", n)
}

await dataRace()
5 Likes
3

SE-0461 has a section discussing conversion like this. It has a valid example which is a bit similar to your code. But that example doesn't compile on nightly. Not sure if it's due to the deinit issue Aviva mentioned. I filed a bug just in case it's not intentional.

class NotSendable {
  var value = 0
}

nonisolated(nonsending)
func convert(closure: () -> Void) async {
  let ns = NotSendable()
  let disconnectedClosure = {
    ns.value += 1
  }
  let valid: @MainActor () -> Void = disconnectedClosure // okay
  await valid()

  // ...
}
3 Likes
4

I think that is different although it doesn't work for the same reason. I don't think SE-0434 and SE-0461 work correctly together, or at least 461 is underspecified in the context of 434? Sema doesn't have enough information to decide if this cast is safe (disconnected is not a Sema concept), and I'm not clear if 461 is taking into account that 434 means actor isolated can imply Sendable, I think 434 gives the better rule. Maybe 461 was hoping Sema would defer this to RBI? If you wrote this, it would be legal:

  let disconnectedClosure = { @MainActor in
    ns.value += 1
  }

because ns would be sent into disconnectedClosure, which RBI will ensure is ok.

You could also treat a function as sending when it is disconnected, that is fine, RBI will ensure it is ok. It's a similar idea as capturing it, except capturing it when it's already in an isolation can go further, and call it repeatedly.

class NotSendable {
  var value = 0
}

nonisolated(nonsending)
func convert(closure: () -> Void) async {
  let ns = NotSendable()
  let disconnectedClosure = {
    ns.value += 1
  }
  await take(disconnectedClosure)
}

@MainActor
func take(_ f: sending () -> ()) {
	f()
}

But Sema doesn't know which functions are sending, and can't reason about the cast shown in 461. For Sema to allow the cast in 461, Sema / RBI also needs to prevent use of the function after casting!

Capturing functions vs casting functions is asking about implicit to explicit isolation cast, which I think is "more" ok, since it is casting a @MainActor isolated (due to being a param of a @MainActor func which Sema knows) and doesn't need to prevent use of the original value. 461 describes a cast which is informed by RBI, and needs to prevent use of the original value.

The same pattern which works in the original question works for the 461 example:

class NotSendable {
  var value = 0
}

nonisolated(nonsending)
func convert(closure: () -> Void) async {
  let ns = NotSendable()
  let disconnectedClosure = {
    ns.value += 1
  }
  let valid: @MainActor () -> Void = { disconnectedClosure() } // okay
  await valid()
}

but the only safe way to call disconnectedClosure after capturing it is if it was @MainActor.

2 Likes
5

TBH, I am a bit confused why RBI is not part of Sema. Intuitively to me it seems like it should be. Regions are not part of the type system, but this still sounds like a semantical analysis to me. But, anyway. Sema knows at least about @Sendable. It can allow all non-Sendable to actor-isolated casts to pass and let RBI validate that original value is disconnected or in a region already connected to that actor.

I think neither SE-0434 nor SE-0461 affect validity of this cast. Even if @MainActor-isolated function is not sendable, we still need to connect region of the initial value with @MainActor. And if this can be done, cast is safe. And synchronous cases in the section in SE-0461 do not depend on any changes introduced by SE-0461, they sound like pre-proposal reality. Further text describes rules for casting @concurrent async functions, which is what the proposal adds on top.

My conclusion is that SE-0414 is sufficient to make this cast valid, even though text of the SE-0414 never mentions it explicitly.

I'm not sure what you mean here. sending is an attribute of the function parameter/result that indicates requirements/promise that value is disconnected.

Yes, for captures RBI already does that. Actually, RBI prefers to do it other way around - it complains about the capture if there is a use after capture.

6

It's an implementation detail, but a pretty important one so it informs a lot of design. RBI operates on a control flow graph discovered from SIL, which reasons about which code follows which other code, when values are used, the final use of values. Sema operates on an abstract syntax tree, which can reason about the types at play in operations, but can't reason about which operations follow which other operations, and under what circumstances. In theory, anything Sema wishes to do that is flow sensitive (meaning, ok dependent on the flow of values through a function) could be handed off and coordinated with RBI logic, but in practice deferring stuff like this can be hard!

To make it more concrete, if Sema wanted to allow a function cast only if the function isn't used again, it cannot answer the question "is this function used again". It could see if the function is referenced again, but in general it can't tell if that reference is reachable from the cast, that requires control flow. For example, you can imagine code that casts a function, then returns, inside a conditional, then uses the function again after the conditional. Flow sensitive analysis can determine that this is safe, but Sema isn't able to reason about if the use after the conditional can be reached. So what it would need to do is always permit these casts when they might work, and RBI would also need to validate casts under these rules. Anywhere we do that wrong leads to a bug :)

yeah, this could theoretically be done, that's what I was trying to get at I think.

Sending is also a way RBI can think about values, "in the sending region" / "sending eligible" / "could be passed to a sending parameter or used as a sending result". A variable could be sending in this sense even though we don't allow the keyword to be written there. Disconnected is a better way to describe it.

I know! But captures are a little different for the implementation. Afaik RBI just doesn't arbitrate on if casts are allowed today, although it could in the future to make casts like these work. Right now casting stuff is all settled before we leave Sema.

2 Likes