Is there a way for a nonsendable type to receive an asynchronous callback?

Using Swift
,
1

I've been trying to figure this out and I think the answer is no, but I'd like to check:

Is it possible for a non-Sendable type to perform some detached asynchronous operation, with a completion handler that calls back to some method on the instance?

final class NonSendable {

  func doSomething() {
    Task.detached {
        // <... do some work ...>
        // < -- HERE -- >
        self.otherFunc()
    }
  }

  func otherFunc() {
  }
}

At the line marked HERE, as far as I can tell, there is no way to call otherFunc.

So I was trying to figure what a non-Sendable type even means in Swift, and I found this from the region-based isolation proposal:

As defined above, all non-Sendable values in a Swift program belong to some isolation region. An isolation region is isolated to an actor's isolation domain, a task's isolation domain, or disconnected from any specific isolation domain:

[...]

As the program executes, an isolation region can be passed across isolation boundaries, but an isolation region can never be accessed by multiple isolation domains at once. When a region R1 is merged into another region R2 that is isolated to an actor, R1 becomes protected by that isolation domain and cannot be passed or accessed across isolation boundaries again.

If an isolation region (containing a non-Sendable type) is able to move across isolation domains, then it would make sense why it is not possible to safely invoke a callback from some arbitrary other domain - because you don't know whether the region has moved since your task was launched.

In other words, these kind of functions "pin" your non-sendable type (and its isolation region) to the isolation domain they currently live on, until some point which cannot be reasoned about statically (so indefinitely). That's fine for the use-case I have in mind, but I don't believe it's expressible in the language (is it?). I couldn't find anything like that in the proposal or future directions, but I may have missed it.

There are quite a lot of types which are not Sendable, i.e. they cannot be used concurrently from different isolation domains, but you can create them from any isolation domain and then (maybe directly from creation or once you start using them) safely use them only from that same domain. And that is how they are used, and in theory they can support asynchronous callbacks, but I can't find anything in Swift concurrency to model this.

Have I overlooked something? Is there a solution for these kinds of types either currently or planned?

2

I'm a bit confused by what the external use looks like here that would guarantee safety absent further machinery: NonSendable can't see inside doSomething how it's being used externally, and so it would have no way to know statically that the usage here is safe—to me it appears fundamentally unsafe.

There might be a version which uses an isolated parameter as well as sending to ensure that the internal task is isolated to whatever the isolation is that called doSomething—but then self would be forever merged with the surrounding isolation of the caller of doSomething (precisely because it could have done something like spawn an internal async callback on the given isolation). Would that also satisfy the use case here?

3

Yeah that's how it seemed to me - because non-sendable types can move between isolation domains, there's no safe way to make an asynchronous callback to them.

So if we take it that this is indeed inherently not possible in the current language design, I suppose I could describe what I'm looking for in a solution: a class that can hold some non-sendable state, and which provides a fixed isolation domain that I know is safe to access that state from.

Now, clearly the thing I've just described is an actor. But using an actor in this instance is not what I want because they always define their own isolation domain. I don't want this data to live in a separate isolation domain; I want a kind of actor which can also be assigned (once, so maybe transferred/sendinged) to an existing isolation domain.

If the isolation checker knows you are in the same domain as this new kind of actor, you could call it synchronously, similar to @isolated(any) closures. For example, if you have one of these values stored in a UI context it would be isolated to the main actor, and you could call it synchronously from main actor contexts, and if you had one stored inside an actor, it would adopt the isolation of that actor and be callable synchronously within the parent actor.

And I suppose that becoming a kind of actor - and having a single, known isolation domain - would actually make these types Sendable, which makes life much easier.

1 Like
4

Gotcha, this makes sense. I think in a couple of threads this has come up as allowing a class which is otherwise non-sendable (due to holding some mutable state) to have a blessed property declared as something like:

public let isolation: isolated any Actor

which would then let each instance know that it's immutably bound to some isolation domain. Perhaps given the precedent for function types this could be spelled as @isolated(any) class.

I believe there's a version that works at the expense of adding boilerplate to every method, and it looks like this:

final class NonSendable {
  func doSomething(_ isolation: isolated any Actor = #isolation) {
    Task {
      _ = isolation
      // <... do some work ...>
      // < -- HERE -- >
      otherFunc("doSomething") // OK, no change of isolation domain
    }
  }

  func otherFunc() {
  }
}

Essentially, if each individual method on NonSendable expresses that it must be called with some known isolation then that isolation can be used for the async callback.

2 Likes
5

I think this is actually quite a big hole in Swift's concurrency story.

I want to focus on the actor naming because isolation in Swift fundamentally is modelled in terms of actors. And what I'm actually asking for here is a composable island of isolation.

Actors are able to "protect" arbitrary non-sendable state and operations and make them sendable because they are tied to a single isolation domain which the compiler knows how to access and can somewhat reason about the boundaries of - enough that whichever isolation domain you happen to be in, it knows how to switch to the serialised timeline that this non-sendable state lives on so you can access it safely.

So anyway, let me reduce the shape of this hole to its essential characteristics:

  • We have some non-sendable state with identity (e.g. it contains a resource handle or something)

  • We need to respond to events (incl. async continuations), so handlers for those events need the ability to copy an owned reference to it

Those are the essentials, I think: has identity, needs to respond to events.

Of course, we want something that is composable - sure, the component may be receiving callbacks from asynchronous workers/event sources, but we still want its state to live in the same concurrency context as its container (as is typical for values in Swift) so the container can easily use the component.

Looking at requirement 1, it seems we want to encapsulate our state in something with identity, so either a class/actor, or a noncopyable struct. Requirement 2 rules out noncopyable structs, so our options are:

A Class

Classes are the traditional solution to this kind of thing, and we can summarise the options Swift concurrency gives us by considering whether our class should be Sendable:

  • If we do not mark it as Sendable, it is subject to the limitations on nonsendable types described in this thread: it is assumed to have no fixed isolation, always on the move switching isolation domains. That isn't what we want to model, and it's not suitable for our purpose, as it cannot respond to events; event handlers know of no isolation from which they can call back to it safely.

  • So we must mark it as Sendable, but this means it must be directly callable from every isolation. Because the language lacks the ability to express isolation of a classes, it doesn't know which isolation to switch to in order to use instances of this class safely. So it assumes it is callable concurrently from everywhere, at any time, and it's up to us to try make that work somehow :man_shrugging:

    It is not very easy to actually implement a Sendable class over non-sendable state from scratch. In most cases, this is going to involve wrapping all your internal state in a Mutex or dispatching everything to some internal queue (basically a poor-man's actor). This is technically making it maximally thread-safe, but it's only really necessary because the language doesn't know how to switch to our actual isolation.

An Actor

Actors seemed really great when they were pitched because, as I described above, they offer an island of isolation for non-sendable state. Concretely, that allows you to locally reason about events from async workers a single piece of state, with the compiler ensuring all the calls are serialised and called from the appropriate context.

However, it is difficult to actually use actors in your program because they simply do not compose in a concurrency sense.

(struct/class) Foo {

  let component: MyActor

  func useComponent() {
    component.anything() // Error: crosses isolation boundary, must be await-ed. Means you need async/Task {...} everywhere.
  }

}

I think the real benefit of having islands of isolation for developers is that they should form the foundation of building code that can compose and use concurrency without fear of introducing data races. But actors, with the limitations they have today, are unable to fill that need.

...but wait, what's this? A third way??

There is one more option. The language offers us one, singular way to express isolation for classes in a way that can be used to build composable... components with static concurrency checking: global actors.

If you mark everything as @MainActor, then the compiler is finally able to reason about things that should be in the same isolation domain. I am fairly convinced this is why we're considering proposals such as SE-0466 Control default actor isolation inference:

A lot of code is effectively “single-threaded”. For example, most executables, such as apps, command-line tools, and scripts, start running on the main actor and stay there unless some part of the code does something concurrent (like creating a Task). If there isn’t any use of concurrency, the entire program will run sequentially, and there’s no risk of data races — every concurrency diagnostic is necessarily a false positive! It would be good to be able to take advantage of that in the language, both to avoid annoying programmers with unnecessary diagnostics and to reinforce progressive disclosure.

The easiest and best way to model single-threaded code is with a global actor. Everything on a global actor runs sequentially, and code that isn’t isolated to that actor can’t access the data that is. All programs start running on the global actor MainActor , and if everything in the program is isolated to the main actor, there shouldn’t be any concurrency errors.

In other words, I think the problem is deeper than the proposal lets on: it's not about just avoiding annoying diagnostics for the sake of progressive disclosure; global actors are a necessary cludge to make simple things work in Swift concurrency because they are the only way to express that multiple components with identity have the same, non-floating isolation.

So yeah, I think we need composable actors.

1 Like
6

I'm not sure I understand this point—this is kind of the opposite of how I conceptualize non-Sendable values. They must either be completely disconnected from any isolation domain (in which case they're eligible to be moved between domains as sending parameters), or else they've been merged into some isolation domain at which point they are fixed—being non-Sendable they're no longer eligible to move to a different isolation domain.

With the new default of isolation inheritance for async functions I'm curious if you have a toy example of where the usability hole remains—I think ideally we'd be able to model this with a non-Sendable class. An instance of such class would internally be isolation-agnostic, and expected to be used from some parent 'root' isolation, be that a global actor, some actor instance, or as a local Task-isolated value. And such a type would, yes, have to take care internally to make sure that it doesn't cross isolation domains, because it doesn't know what that parent isolation might be. Is problem that the current default isolation inheritance permits nil to be passed for the isolation parameter, and so it becomes verbose to express "this method may only be called with an actual, concrete isolation available"?

1 Like
7

I’m quite confused by the thread…

This description

does make sense to me, as I was one who have been missing this capability to control async calls. At the state of 6.2, however, this has been improved - now we can make isolation sticky and just model classes as non-Sendable, relying on updated semantics. We still lack control for Task calls with isolation I believe, but there are workarounds IIRC.

At the meantime, design using global actors to define isolated subsystems, that consist from groups of types, are valid and even still useful solution for many cases. Swift Concurrency model embraces this explicit isolation a lot. I was fighting a bit this way of things, but then experimented with several cases of such subsystems design with global actors and really liked that.

So I’m not sure what do you mean it has no fixed isolation, but instances of non-Sendable type stick to one isolation where they were created and only if they in disconnected region they can be transferred to another isolation. They are extremely conservative in that question.

8

If this is what you're getting at, I think you might be able to kludge this today by accepting a non-optional isolated any Actor parameter in the init of your type and then preconditionFailure-ing out of all async methods in your type if guard let isolation fails—your init would ensure that instances of your types are always merged with some concrete isolation upon creation, which should then avoid it ever being passed to a domain without concrete isolation.