SE-0313: Improved control over actor isolation

Evolution Proposal Reviews
22

The closure is "in the actor code". It's the unnamed-function equivalent to defining a method on the actor, no more and no less.

The primary motivation for isolated parameters in this proposal is to allow us to generalize from a method on an actor:

actor A {
  var counter = 0

  func inc() { counter += 1 }
}

to being able to write a free function with the same capabilities:

func inc(a: isolated A) {
  a.counter += 1
}

This feature we're discussing lets you generalize from a method on an actor to a closure that's defined within the actor.

It's independent of the actor reentrancy model. [isolated self] on a closure is no less explicit than calling an actor method from that closure.

The problem with the last line isn't about Sendable at all: it's about two isolated parameters in the same call being fulfilled by different actor arguments, so there is no single executor that we can jump to.

Let's step back slightly, because the statement "isolated actor references aren't Sendable" isn't a well-formed one to make. Types are Sendable or non-Sendable, values cannot be Sendable or non-Sendable because values don't conform to protocols. Types conform to protocols.

The argument for allowing multiple isolated parameters came from here:

  • "Therefore, we prohibit the definition of a function with more than one isolated parameter:" --> Why? This can be narrowly useful (e.g. unsafe casts when two actor instances are known to be on the same executor) and there is no reason to ban it. Why exclude a (narrowly useful) valid case and add the compiler complexity? I don't see any safety benefit to this.

I don't believe we've seen a use case for this that doesn't require unsafe casts. We've removed several other "unsafe" features (e.g., nonisolated(unsafe)) that had stronger use cases, based on the principle that we want to start with the safe model and learn more before we add unsafe things.

The argument that it is less complex to allow multiple isolated parameters than it is to prohibit them hasn't held up, as this discussion has repeatedly shown.

Multiple isolated parameters should be removed from this proposal. They can be added back when there is a way to safely indicate that various different actor instances share the same executor. Then, they'll be worth the complexity they add.

Doug

9 Likes
23

One critical topic I hope to be discussed in the proposal is Async requirements require duplication of protocols (which is raised in the google doc of another proposal Exploration: Type System Considerations for Actor Proposal). If I have an isolated ownership of an actor, I suppose I can cast it into existentials or do other POP programming. But this proposal doesn't seem to address these issues at the current stage. Do we really need to define almost identical protocols of sync/async versions?

24

After reading this syntax-directed proposal and the other type-directed proposal (Exploration: Type System Considerations for Actor Proposal) , it seems easier to understand and reason about in the type-directed approach (@sync and @async actor types). I don't need to remember more complicated syntax rules (though it may be quite natural from a language designer's perspective), and I'm thrilled that I can use existing POP techniques and other third-party Swift libraries when the actor model is integrated into the language.

In the Alternatives Considered section, the author explains why the syntax-directed approach is preferred:

  1. The proposal treat isolated as a type modifier similar as inout, because "it provides a simpler, value-centric model".

    While I agree that it provides a value-centric model, I don't really think it provides a simpler model. What simplicity means to me is that we address more issues with less rules. The proposal should explain more on why this value-centric model is simpler.

  2. The type of an actor's self can change within nested contexts, such as closures, between @sync and non- @sync.

    It really seems natural to me that if a @sync actor is captured in an async closure, it will be automatically casted to @async type. Even this is a new syntax rule that modifies the captured variables behavior in the current language, I don't see strong reason why we should resist this change.

  3. The design relies heavily on the implicit conversion from @sync MyActor to MyActor .

    Again this implicit conversion seems natural to me. I don't understand why this should be the reason that we don't choose the type-directed approach. The author should explain WHY this implicit conversion is a bad idea.

  4. A @sync actor type is a subtype of the corresponding (non- @sync ) actor type. A non- @sync actor type conforms to Sendable (it's safe to share it across concurrency domains), but its corresponding @sync subtype does not conform to Sendable .

    My question is that: if we allow @sync actor type safely converts to @async actor type, does this behavior really means that a @sync actor type is a subtype of the @async actor type? I can take a simple example here: we can convert T to Optional<T> in Swift, but does this really means T is a subtype of Optional<T>? I don't think so.

So I can conclude that the counterarguments that this proposal raised against the type-directed approach (@sync and @async actor types) doesn't really hold from my view. I hope the author could add more rational and discuss more on this :)

25

Apologies if I am asking too basic of a question here but why do we need the isolated param (above) when the below "works fine"?

// Why is await not recommended here when in the `isolated` case the suspension still there just implicit?
func inc(a: A) async {
  // Assuming async properties
  await a.counter += 1
}

// Is this not a viable alternative?
extension A {
  func inc2() {
    self.counter += 1
  }
}
26

Why does this "seem natural"? I cannot point to any other place in the language where a value changes type within a nested context, and we've resisted such changes in the past (e.g., an value x: Int? becoming type Int inside an if x != nil).

We're generally quite careful about adding implicit conversions, because they can have unintended consequences. In this case, the type-directed approach effectively needs two implicit conversions, one in each direction. Going from @sync -> @async is always fine (and needs to happen at some point), but you also need to be able to go from @async -> @sync in certain limited places. This is an "interesting" conversion because it's presence means you are interacting with an actor asynchronously, so each such conversion needs to be part of something that can be done asynchronously (function call, read from an actor property, etc.) and must be covered by an await.

That's a fair point. I suppose I was thinking about how the sequence of conversions you take can affect type identity, which I find a bit odd. For example, consider this:

func f(a: @sync MyActor) {
  let a1: Any = a
  let a2: Sendable = a
  let a3: Any = a2
}

What are the types dynamically stored in a1 and a3? They're actually different, because it's a1 will have dynamic type @sync MyActor, while a3 will have dynamic type @async MyActor because it had to be Sendable to initialize a2. Per your comment about optional and conversions, I bet you can construct a similar case with Optional<T>, so maybe this oddity is a big deal.

Let's go more dynamic, though. What if now I write:

if let a4 = a3 as? @sync MyActor { ... }

Does this dynamic cast fail, because the type stored in a3 is @async MyActor, or is the runtime dynamically checking whether we are currently executing on the this actor and allow the cast to succeed?

Happy to discuss!

There's more rationale in one of the early pitch thread documents, but essentially the argument is that the self of an actor shouldn't be special: you should be able to write a free functions and closures that can have synchronous access to the actors, for example, because that's normal factoring of code. It's very similar to how mutating methods of structs/enums are really just inout on the self parameter, so mutating just isn't all that special.

isolated parameters on closures also let you write things like the runOnActor operation mentioned above.

Doug

2 Likes
27

Thanks, I'd like to share more of my thoughts here!

Let's continue discuss the points, first with the value changes type within a nested context issue:

In the type-directed proposal, it says

So from my point of view we are making an analogy between the @sync/@async actor type switching and the implicit inout YourStruct. But I regard it as a trivial issue if you think this behavior is unnatural, because we can choose an explicit way instead:

let myActor: @sync MyActor = ...
detach {
  await myActor.doSomething() // Compile error! we always reject implicit capture of @sync actors inside @sendable closure
}

detach { [async myActor]
  await myActor.doSomething() // No compile error. we explicit capture myActor into an @async actor object
}

As you can see, it's a bit like [weak self] pattern in the language.

So actually what make you worried is the reverse direction of casting you mentioned in the proposal. In the type-directed proposal, there are both safe and unsafe ways proposed to let us cast from @async actor to @sync actor:

// This is the safe way to cast
extension AnyActor {
  func doWithinActor<T>(fn: (@sync Self) -> T) async -> T { 
 }
}

// This is the unsafe way to cast
func unsafeActorInternalsCast<T: AnyActor>(_ x: @async T) -> @sync T

Can you share more thoughts on these safe/unsafe methods? I'm not sure what makes you worried/unhappy with the current design of casting @async MyActor to @sync MyActor.

Edited: Can you explain more about the oddity here? I guess you are saying that a1 and a3 should have the same type (while they aren't in the case). Does the implicit casting of @sync MyActor to @async MyActor make you confused here? I can argue that when we convert T to Optional<T>, we are making a normal value conversion. But when we convert @sync MyActor to @async MyActor, we are actually making a type promotion here.
Update: Since Sendable is just a marker protocol (which doesn't contribute to binary file), can we really declare an existential of Sendable like let a2: Sendable?

I think the dynamic cast would succeed, because we are still passing an object with dynamic type of @sync MyActor.

P.S. When I am consuming the type-directed proposal, I also find it's not perfect. For instance, we may still need nonisolated let to make some properties/methods accessed synchronously from outside, but still have an implicit @async self within their bodies. But I hope that we can combine the pros from both the value-directed and type-directed approaches, which could be the foundation for future actor usage :)

28

I recognize the closure is in the actor code. I'm trying to point out that the await is not.

Suspensions in the actor model allow reentrancy, which is critical "safety" problem (logic safety, not in memory safety) we need to help programmers with. The actor model was carefully built to address this by making potential suspensions explicit with an await in your code. This violates that principle in an (as far as I can tell) pretty unprecedented way.

I don't see why we would add syntactic sugar to do that in any case, and I don't see why we would start out a base type system proposal like this with sugar in it. If you're interested in sugaring away awaits, then it should be done in a more consistent way, and we should carefully weigh the global tradeoffs to logic safety.

Yes, I'm not arguing about this. I'm arguing about a specific form of syntactic sugar that is added in this proposal that causes an implicit jump back into the actor context. I'm arguing that jump should be explicit. My request is a tiny change to the proposal, it does not undermine the goals as you seem to think.

I'm keenly aware that "isolated" is not a type modifier in this proposal, seriously I remember the iteration that got us here. I am trying to make analogies to another model that you also know well in a way to communicate an issue with you. You are pedantically pointing out that we're not using that model, but the issue I'm pointing out still holds.

Feel free to replace "isolated actor references aren't Sendable " with "isolated actor references need to be rejected in the same places that require Sendable".

The closest analog in the language (inout) supports multiple parameters and has an analogous safety issue. It is checked dynamically (exclusivity checking) just like this can.

I'm tired of arguing about this, all the points have already been made. I agree it isn't essential to the model and can be added later.

-Chris

3 Likes
29

This proposal (value-directed approach) can actually be separated into two new features:

  1. Allow nonisolated keyword before property/method declarations in an actor type.

    This feature is based on the fact that some data of an actor is non-isolated, so we need a safe mechanism to visit this part synchronously from outside. This also enables actors conform to some pre-async-era protocols with their non-isolated members.

    From my point of view, this feature isn't much controversial. It solves a simple problem with a simple way.

  2. Add isolated as a parameter modifier, as well as to capture list in closures.

    This feature is based on the scenario that sometimes we want to synchronously visit an actor, as long as we are in the isolated domain of it.

    I think this feature is quite controversial, because it solves a much smaller scope of issues than the type-directed approach, while differ from it in a source-breaking way. Once we choose this value-directed approach, it would be much harder for us to solve the issues that could be solved in the type-directed approach (e.g. duplication of sync/async protocols, unnecessary async methods on actors forced by protocol abstractions). I hope the authors could discuss these issues in the main body of proposal, so that we could have a clear picture of how actors will interact with protocols in the future.

30

I can't imagine us accepting the above.

[weak self] is a good point! It's technically a new declaration, but it feels like it's changing the type of self in a nested context.

To be clear, this is expressed as

extension Actor {
  func doWithinActor<T>(fn: (isolated Self) -> T) async -> T { ... }
}

in SE-0313, and it's safe either way.

This is inexpressible in SE-0313 in this form. You could do some tricks to create an unsafe variant of doWithinActor, though:

extension Actor {
  nonisolated
  func unsafeDoAssumingWithinActor<T>(fn: (isolated Self) -> T) -> T {
    typealias UnsafeFn = (Self) -> T
    return unsafeBitCast(fn, to: UnsafeFn.self)(self)
  }
}

I consider that to be a better API to unsafeActorInternalsCast, because it narrowly scopes the unsafe access.

The oddity is that well-formed conversion paths, @sync MyActor -> Sendable -> Any and @sync MyActor -> Any, produce values with different dynamic types.

Yes, you can. It has the same representation as an Any. It's even important and useful if you need to type-erase and still pass something across concurrency domains.

I don't believe this is correct. a2 cannot contain a value with dynamic type @sync MyActor, because @sync MyActor is not Sendable. It contains a value with dynamic type @async MyActor.

For that case to succeed, we would need to implement a runtime check that permits an @async A -> @sync A cast by checking whether we're running on the actor. This is doable, and might be a good idea, but let's make sure that we're on the same page about how the model would work.

Doug

1 Like
31

Yes, we can separate the two features. nonisolated can be described as making an instance member of an actor not be treated as isolated, so it doesn't actually depend at all on the presence of isolated parameters. This is how nonisolated was described in the SE-0306 before it got split out into SE-0313.

If indeed there consensus is that we got nonisolated right but isolated parameters need more work, then splitting the two apart is a reasonable way to make progress.

Doug

3 Likes
32

Glad that we have a workable approach to solve the type-change-in-nested-scope issue :)

Do you mean that it's safer because we couldn't pass around isolated Self to other parts of the code? I agree on this point. But I am also thinking we may not need an unsafe casting method in the first place.

Thanks for the explanation! Now I'm thinking if we should still allow implicit conversion from @sync MyActor to Sendable, since we have an explicit way to capture @sync MyActor as @async MyActor in a Sendable closure using [async myActor]. If we prohibit this kind of implicit conversion, I suppose we will no longer have a conversion path like @sync MyActor -> Sendable. Will the oddity go now?

If we can dynamically check that whether we're running on an actor (not the thread/queue it's backed), that would really of great help.

33

An abbreviated review because, unfortunately, deadlines for SE-0311 and SE-0313 are coincident.

I think this proposal does address a significant problem, and I'm please with the overall direction. I think it does fit well, in general, with the overall feel and direction of Swift's concurrency story.

Having read through the dialogue so far I would share @Chris_Lattner3's concern about any non-locality of await signaling possible suspension points. If indeed this is an aspect of the proposal that can be refactored and added later as sugar, then I'd be in support of doing so and re-evaluating at a later point whether the sugar is warranted.

With respect to the isolated part that's new to SE-0313:

Way back when, we had a whole proposal to move inout to the right of the colon because it's a type modifier. As a corollary, we've recently put property wrappers to the left of the colon because it's not a type modifier but a parameter modifier. This proposal breaks with this distinction: it makes isolated a parameter modifier but puts it to the right of the colon.

Now it is true that, without an @ sigil, putting isolated to the left of the colon might be a beat more difficult to read: func foo(isolated _ bankAccount: BankAccount). But in the context of syntax highlighting it really shouldn't be a deal-breaking problem. I'm wary of blowing apart a distinction that we've assiduously cultivated in order to help users understand what's a type modifier and what's not.

3 Likes
34

With inout, we have it to the right of the colon:

func f(a: inout Int) { ... }

The value type of the parameter a, when you use it, is Int. There is no type inout Int in the type system. Rather, the inout is associated with that parameter within the function's type, i.e., the function type includes the inout:

let fn = f    // type of 'fn' is '(inout Int) -> Void'

Property wrappers and result builders on parameters are a bit different. They are part of the declaration but don't affect the type of the parameter when it's used or the function type. Hence, they go on the left.

The way I think of it is that when you go from the declaration of a function to its type, you drop the stuff to the left of the : and keep the stuff to the right of the :.

isolated parameters are following the precedent set by inout: they don't affect the type of the parameter, but they do affect the type of the function.

Doug

9 Likes
35

Accepted with Revision

The core team (with discussion with the proposal authors) has decided to accept a subset of the proposal. The revised (and accepted) proposal has the following parts removed:

  • Support for multiple isolated parameters
  • The proposed changes for closure isolation control

The removed parts, which were the most controversial sources in the review discussion, can be revisited in a future Swift Evolution proposal.

The review discussion also discussed the tradeoffs between a syntax-driven versus type-driven approach. The core team discussed the tradeoffs of the two methods and observed that a type-based approach would (by design) pervade the type system and potentially result in a more complicated system. The core team thus preferred the syntax-based approach in the proposal.

Reviews are a critical part of evolving Swift! Thank you to everyone who participated in this review!

11 Likes