Sendable Closures that are also isolated and synchronous?

Using Swift
1

Does anyone have advice for defining or passing an isolated and synchronous closure that should also be Sendable?

Suppose I start with an actor. I need to perform some work with withTaskCancellationHandler. I begin an operation and run a handler if this task is cancelled.

actor A {
  func operation() {
    
  }
  
  func handler() {
    
  }
  
  func f1() async {
    await withTaskCancellationHandler(
      operation: self.operation,
      onCancel: self.handler
    )
  }
}

This compiles with no errors.

But what if I need some more state here? Some additional information that should be communicated to operation and handler?

extension A {
  func operation<T: Sendable>(_ value: T) {
    
  }
  
  func handler<T: Sendable>(_ value: T) {
    
  }
}
  
extension A {
  func f2<T: Sendable>(_ value: T) async {
    await withTaskCancellationHandler(
      operation: {
        self.operation(value)
      },
      onCancel: {
        self.handler(value)
      }
    )
  }
}

This fails with an error:

error: call to actor-isolated instance method 'handler' in a synchronous nonisolated context [#ActorIsolatedCall]
 25 |   }
 26 |   
 27 |   func handler<T: Sendable>(_ value: T) {
    |        `- note: calls to instance method 'handler' from outside of its actor context are implicitly asynchronous
 28 |     
 29 |   }
    :
 37 |       },
 38 |       onCancel: {
 39 |         self.handler(value)
    |              `- error: call to actor-isolated instance method 'handler' in a synchronous nonisolated context [#ActorIsolatedCall]
 40 |       }
 41 |     )

At this point I'm not completely clear I understand. The withTaskCancellationHandler function should AFAIK inherit the isolation of this actor.

Let me try a closure to capture my state and pass that directly to withTaskCancellationHandler:

extension A {
  func f3<T: Sendable>(_ value: T) async {
    let onCancel = {
      self.handler(value)
    }
    
    await withTaskCancellationHandler(
      operation: {
        self.operation(value)
      },
      onCancel: onCancel
    )
  }
}

This fails again with a different error:

error: converting non-Sendable function value to '@Sendable () -> Void' may introduce data races
 53 |         self.operation(value)
 54 |       },
 55 |       onCancel: onCancel
    |                 `- error: converting non-Sendable function value to '@Sendable () -> Void' may introduce data races
 56 |     )
 57 |   }

Ok. Let me try making the closure Sendable:

extension A {
  func f4<T: Sendable>(_ value: T) async {
    let onCancel = { @Sendable in
      self.handler(value)
    }
    
    await withTaskCancellationHandler(
      operation: {
        self.operation(value)
      },
      onCancel: onCancel
    )
  }
}

Here's another error:

error: call to actor-isolated instance method 'handler' in a synchronous nonisolated context [#ActorIsolatedCall]
 25 |   }
 26 |   
 27 |   func handler<T: Sendable>(_ value: T) {
    |        `- note: calls to instance method 'handler' from outside of its actor context are implicitly asynchronous
 28 |     
 29 |   }
    :
 61 |   func f4<T: Sendable>(_ value: T) async {
 62 |     let onCancel = { @Sendable in
 63 |       self.handler(value)
    |            `- error: call to actor-isolated instance method 'handler' in a synchronous nonisolated context [#ActorIsolatedCall]
 64 |     }
 65 |

My closure is synchronous but not isolated to my actor. Fair enough. Here is another attempt with a nested local function:

extension A {
  func f5<T: Sendable>(_ value: T) async {
    func onCancel() {
      self.handler(value)
    }
    
    await withTaskCancellationHandler(
      operation: {
        self.operation(value)
      },
      onCancel: onCancel
    )
  }
}

Another Sendable error:

error: converting non-Sendable function value to '@Sendable () -> Void' may introduce data races
 83 |         self.operation(value)
 84 |       },
 85 |       onCancel: onCancel
    |                 `- error: converting non-Sendable function value to '@Sendable () -> Void' may introduce data races
 86 |     )
 87 |   }

Here is another attempt:

extension A {
  func f6<T: Sendable>(_ value: T) async {
    @Sendable func onCancel() {
      self.handler(value)
    }
    
    await withTaskCancellationHandler(
      operation: {
        self.operation(value)
      },
      onCancel: onCancel
    )
  }
}

Which gets me back to where I was before:

error: call to actor-isolated instance method 'handler' in a synchronous nonisolated context [#ActorIsolatedCall]
 25 |   }
 26 |   
 27 |   func handler<T: Sendable>(_ value: T) {
    |        `- note: calls to instance method 'handler' from outside of its actor context are implicitly asynchronous
 28 |     
 29 |   }
    :
 91 |   func f6<T: Sendable>(_ value: T) async {
 92 |     @Sendable func onCancel() {
 93 |       self.handler(value)
    |            `- error: call to actor-isolated instance method 'handler' in a synchronous nonisolated context [#ActorIsolatedCall]
 94 |     }
 95 |

But since it's a function now instead of a closure
 can I use SE-0420 to pass isolation directly?

extension A {
  func f7<T: Sendable>(_ value: T) async {
    @Sendable func onCancel(
      isolation: isolated (any Actor)? = #isolation
    ) {
      self.handler(value)
    }
    
    await withTaskCancellationHandler(
      operation: {
        self.operation(value)
      },
      onCancel: onCancel
    )
  }
}

This works on making my nested location function isolated to my actor. But there are two new errors:

error: actor-isolated synchronous local function 'onCancel(isolation:)' cannot be marked as '@Sendable'
105 | extension A {
106 |   func f7<T: Sendable>(_ value: T) async {
107 |     @Sendable func onCancel(
    |     `- error: actor-isolated synchronous local function 'onCancel(isolation:)' cannot be marked as '@Sendable'
108 |       isolation: isolated (any Actor)? = #isolation
109 |     ) {

/Users/rick/Desktop/ActorDemo/Sources/ActorDemo/main.swift:117:17: error: cannot convert value of type '@Sendable (isolated (any Actor)?) -> ()' to expected argument type '@Sendable () -> Void'
115 |         self.operation(value)
116 |       },
117 |       onCancel: onCancel
    |                 `- error: cannot convert value of type '@Sendable (isolated (any Actor)?) -> ()' to expected argument type '@Sendable () -> Void'
118 |     )
119 |   }

I can try and remove the Sendable:

extension A {
  func f8<T: Sendable>(_ value: T) async {
    func onCancel(
      isolation: isolated (any Actor)? = #isolation
    ) {
      self.handler(value)
    }
    
    await withTaskCancellationHandler(
      operation: {
        self.operation(value)
      },
      onCancel: onCancel
    )
  }
}

But this just moves my error around somewhere else:

error: converting non-Sendable function value to '@Sendable () -> Void' may introduce data races
132 |         self.operation(value)
133 |       },
134 |       onCancel: onCancel
    |                 `- error: converting non-Sendable function value to '@Sendable () -> Void' may introduce data races
135 |     )
136 |   }

/Users/rick/Desktop/ActorDemo/Sources/ActorDemo/main.swift:134:17: error: cannot convert value of type '(isolated (any Actor)?) -> ()' to expected argument type '@Sendable () -> Void'
132 |         self.operation(value)
133 |       },
134 |       onCancel: onCancel
    |                 `- error: cannot convert value of type '(isolated (any Actor)?) -> ()' to expected argument type '@Sendable () -> Void'
135 |     )
136 |   }

It seems like what I'm trying to do should be possible without major ceremony. Is something like this blocked on closure isolation control?

1 Like
Updating `ImageDownloader` actor sample to support cancellation?
2

The cancellation handler runs immediately when the task is cancelled, which means it may run concurrently with the operation. Therefore, it's not possible to isolate both the operation and the task cancellation handler to the actor, because then it wouldn't be possible to execute them concurrently.

AFAIK the isolation parameter only ever applies to the operation, which is documented to execute "on the calling execution context". The cancellation handler may be called from anywhere, which is why it requires @Sendable.

I'm surprised that the first example compiles, though :thinking:

6 Likes
3

I agree the first example is ill-formed.

Fortunately, it now produces an error with the nightly build: Compiler Explorer.

4 Likes
4

I wonder if this is related to this bug: Function parameter isolation not respected , although the latter even compiles fine with the nightly build.

The documentation is misleading here. This is only true under specific circumstances. If you don’t capture the isolation directly or the context is not global-actor isolated it currently executes on the GCE: Compiler Explorer

I discussed this in somewhat more detail here: [Pre-Pitch]: updating `with{Checked|Unsafe}Continuation` to support typed throws (and perhaps nonisolated(nonsending)) - #8 by NotTheNHK and here Why is a Task closure defined in an Actor's method nonisolated? . I also want to point out that we have only been making this claim since Swift 5.8. In earlier Swift versions, this was actually true up until SE-0338. The good news is that we can address this with nonisolated(nonsending), and there is ongoing work to do so.

As for the question, you could use a lock or spawn a new task. However, you need to be careful with locks, as they can easily lead to deadlocks, as the documentation warns.

2 Likes
5

Mostly agree with and echo the chorus here, but did want to point out that isolated, synchronous, sendable functions can be represented and used if you really want to (e.g. see Observations), but calling them synchronously is pretty awkward, and can subvert data race safety (exhibit A, exhibit B).

FYI this change was merged fairly recently that intends to address these issues.

1 Like
6

Thanks for the info! While creating the Godbolt example, I noticed that the nightly build already has the updated semantics.

1 Like
7

Hmm
 this is all sort of confusing me now.

The "current" documentation from the project lead says that isolation is "The actor that the operation and cancellation handler are isolated to."[1]

I believe this is being pulled from the latest release/6.3 commit:

It looks like this landed from here:

It looks like this was recently moved around a little on latest main:

But I still see a similar guarantee about operation and handler sharing an isolation:

  1. withTaskCancellationHandler(operation:onCancel:isolation:) | Apple Developer Documentation ↩

8

This might get a little confusing. The documentation, as stated in my previous comment, is incorrect in that only in specific cases will operation execute on the caller’s executor. However, handler will always execute on the passed in actor’s executor since it is synchronous (a value of nil represents the GCE here). Both @Sendable and @concurrent do not force code to execute on the GCE. However, if you mark an asynchronous closure with @Sendable that is currently statically isolated, it will lose its static isolation and will execute on the GCE, except when you pass it to a nonisolated(nonsending) function parameter. In that case, it will be “dynamically isolated”, i.e., it will run on the caller’s executor, which could be the GCE.

P.S. I think I may have found a bug in nonisolated(nonsending), but I need to investigate further.

P.P.S. I can provide some sample code if that would help.

Edit: We need to keep both versions of withTaskCancellationHandler, nonisolated(nonsending), and #isolation since the change is source breaking.

1 Like
9

I don't think this is true. As far as I can tell, onCancel will always be run by the thread that calls task.cancel(), and is not affected by the isolation parameter of withTaskCancellationHandler in any way.

Take this bit from the task.cancel() documentation (but this behavior is also verifiable by looking at the backtrace that calls onCancel):

cancel may [...] synchronously run arbitrary cancellation-handler code associated with the canceled task.

If the thread that calls task.cancel() executes the cancellation handlers as part of the cancel() method, and given there are no suspension points anywhere (both task.cancel() and onCancel are synchronous), it seems impossible for onCancel to statically enforce any isolation.

The only "isolation" onCancel can have is the dynamic isolation of the executor that calls task.cancel(), but there's no way to know (much less enforce) at compile time which isolation that'll be. Hence why it needs to be @Sendable, it quite literally may be called from anywhere.

1 Like
10

Yes, I was talking about which executor the code will execute on, not the actor (isolation) itself, if that wasn’t clear.

Interesting. However, I couldn’t verify that. Could you provide a sample?

1 Like
11

In this sample:

func isMainThread() -> Bool {
    return Thread.isMainThread
}

let task = Task.detached {
    await withTaskCancellationHandler(
        operation: { @MainActor in
            print("operation() started -> isMainActor: \(isMainThread()), thread: \(pthread_self())")
            try? await Task.sleep(for: .seconds(2))
            print("operation() finished -> isMainActor: \(isMainThread()), thread: \(pthread_self())")
        },
        onCancel: {
            print("onCancel() -> isMainActor: \(isMainThread()), thread: \(pthread_self())")
        },
        isolation: MainActor.shared
    )
}

await Task.detached {
    try? await Task.sleep(for: .seconds(1))
    print("cancel() -> isMainActor: \(isMainThread()), thread: \(pthread_self())")
    task.cancel()
}.value

I get this output:

operation() started -> isMainActor: true, thread: 0x202e105c0
cancel() -> isMainActor: false, thread: 0x16fe87000
onCancel() -> isMainActor: false, thread: 0x16fe87000
operation() finished -> isMainActor: true, thread: 0x202e105c0

Which makes sense: the Main Actor can't execute both operation and onCancel concurrently.

You can also see that cancel() and onCancel happen on the same thread.

3 Likes
12

Thanks! Now that I think about it, it makes obvious sense.

1 Like