SE-0313: Improved control over actor isolation

Evolution Proposal Reviews
#16

They've been elided by the ...:

func acceptSendable(_ operation: @Sendable () async -> Void) {
  detach {
    await operation()
  }
}

This is no different from an await when calling a async method on an actor.

I suspect that you didn't understand my point. Consider this:

actor A {
  var array: [Int] = []
  var array2: [Int] = []

  func slowAppendToSecondArray() {
    array.forEach {
      array2.append($0)
    }    
  }
}

The capture of self in the closure passed to forEach must be isolated, or else we wouldn't be allowed synchronous access to array2. The proposal is allowing you to be explicit about that implicit behavior:

    array.forEach { [isolated self] in
      array2.append($0)
    }

It is the right constraint to check: all isolated parameters in a call must be provided with the same argument value, or it is impossible to guarantee proper isolation.

You're pointing out an additional constraint: if that argument value is not itself isolated, then we have a cross-actor access. The call must be asynchronous is subject to Sendable checking. This is shown early on in the proposal, where we asynchronously call when passing a non-isolated argument to an isolated parameter.

Having separate parameters that must all be provided with the same argument value sounds silly, because we've put the multiple-isolated-parameters cart before the custom-executors horse. The actual rule would be that all of the argument values must have the same executor, and that's the executor we hop to if we aren't already guaranteed to be on that executor. The trick is in making it reasonable to share executors across different actors in a manner that allows us to statically determine that they all share an executor.

You're conflating two issues, that of guaranteeing that we have a consistent actor on which the code runs in the presence of multiple isolated parameters (what I thought you were asking) and that of hopping to the appropriate actor if we're not already on it (what you were apparently asking). Both of these are independently addressed in the proposal, and the model remains sound. I am happy to clarify the proposal text, add examples, and spell out in more detail how this checking works.

I've concluded that it is a mistake for this proposal to admit multiple isolated parameters at all. We should ban multiple isolated parameters. Without custom executors, multiple isolated parameters have no practical value without falling back to unsafe hacks. And once we do get custom executors, we'll have to go and revise the rule anyway to rewrite it in terms of executors (more importantly, statically relationships among executors). It's better to keep things simpler now and generalize when we have tools to do it well, rather than pre-emptively adding features that can only be used unsafely.

Doug

[EDIT: Fixed the definition of acceptSendable to add a detach]

7 Likes
#17

I'm more than happy that you came to that conclusions in the end, Doug :grinning_face_with_smiling_eyes:

It is not a point I was willing to throw pour more gas into the fire for, but I never understood the argument for allowing these making the language simpler, if anything it is just adding more weirdness.

As I mentioned before the actual value of this, and how the same thing is used nowadays but without compiler help of course in actor and event loop systems, will only be unlocked with custom executors:

That's the actual useful piece, and we're still quite far away from it... :slight_smile:

1 Like
#18

Shouldn’t acceptSendable be async?

1 Like
#19

Ah, thanks. I've updated my other reply to put a detach in there so we can await where needed.

Doug

2 Likes
#20

If running on the same executor (The Actor’s) is the goal of isolated parameters then why should we introduce this feature now? Seems to be it’s more about executors (custom|actor) than just about actor isolation. Specially since it seems that multiple isolated parameters it’s not desired.

#21

Aha, got it. Thanks I missed that.

Thanks Doug, I missed that!

That said, I am still not certain this is the model we want. The concerns about bugs introduced by unexpected actor reentrancy are a major potential issue with the actors design. While I agree with you that there is an await in the call chain, I still have a few questions:

  1. Isn't it concerning that it isn't "in the actor code"? This is making the author of the actor anticipate suspensions/reentrancy at await sites as usual, but also makes them look for "closures that take an isolated self parameter and are async". This complicates / defeats the simplicity of the 'await marking' approach, and undermines a significant amount of local reasoning that would otherwise exist.

  2. Why is this a better model than not having the magic? I haven't heard rationale for why we'd want to introduce this -- it isn't essential to the model or to expressivity. It is just stated as included behavior without much rationale.

  3. Given that this is effectively another syntactic sugar extension, why put it into this essential proposal introducing the essential type system capabilities? Sugar proposals have a higher bar than utility proposals - beyond explaining "what they make better", they have to also defend "why they are worth it on balance" which is a more difficult discussion.

Overall, the proposal doesn't seem to include much or any discussion about these points. I'm very sensitive to making the actor reentrancy model more complicated given that this is one of the boldest design stakes taken by the actor design.

Ok, got it, makes sense! This is similar to how unowned captures work.

I think that we are trying to address different concerns. My read from your comment leads me to believe that you're interested in proving that two different isolated actor references are on the same executor for safety purposes. I'm trying for something weaker and more important (to me): I want to make sure we reject incorrect code. Incidentally, I misstated when I said that the behavior occurs with a single actor reference, you are correct that it requires two. Here's what I'm trying to ensure that we reject correctly with an analogy for comparison:

actor MyActor {
  func takeClass(_ x : NSMutableString) {...}
  func takeIsolated(_ x : isolated MyActor) {...}

  func test(a : isolated MyActor, otherActor: MyActor, str: NSMutableString) {
     // Ok to pass isolated things and classes internally to an actor.
     self.takeClass(str)   // ok
     self.takeIsolated(a) // ok

     // Not ok to pass these to other non-isolated actors.
    otherActor.takeClass(str) // error: NSMutableString isn't @Sendable
    otherActor.takeIsolated(a)  // error: cannot isolated actor references aren't @Sendable
  }
}

I'm specifically bringing up the last error there. I'm pointing out that the sendability check for cross-actor hops needs to be applied to isolated actor references like that.

Yes, I'm just asking for clarity around the rules being spelled out in the proposal.

I completely disagree. Banning this has no apparent value, undermines a future direction, and complicates the language. We discussed this quite a bit in previous rounds of the proposal. I think that this is really important for (admittedly advanced!) use cases, and I don't think that banning it has any value.

-Chris

2 Likes
#22

The closure is "in the actor code". It's the unnamed-function equivalent to defining a method on the actor, no more and no less.

The primary motivation for isolated parameters in this proposal is to allow us to generalize from a method on an actor:

actor A {
  var counter = 0

  func inc() { counter += 1 }
}

to being able to write a free function with the same capabilities:

func inc(a: isolated A) {
  a.counter += 1
}

This feature we're discussing lets you generalize from a method on an actor to a closure that's defined within the actor.

It's independent of the actor reentrancy model. [isolated self] on a closure is no less explicit than calling an actor method from that closure.

The problem with the last line isn't about Sendable at all: it's about two isolated parameters in the same call being fulfilled by different actor arguments, so there is no single executor that we can jump to.

Let's step back slightly, because the statement "isolated actor references aren't Sendable" isn't a well-formed one to make. Types are Sendable or non-Sendable, values cannot be Sendable or non-Sendable because values don't conform to protocols. Types conform to protocols.

The argument for allowing multiple isolated parameters came from here:

  • "Therefore, we prohibit the definition of a function with more than one isolated parameter:" --> Why? This can be narrowly useful (e.g. unsafe casts when two actor instances are known to be on the same executor) and there is no reason to ban it. Why exclude a (narrowly useful) valid case and add the compiler complexity? I don't see any safety benefit to this.

I don't believe we've seen a use case for this that doesn't require unsafe casts. We've removed several other "unsafe" features (e.g., nonisolated(unsafe)) that had stronger use cases, based on the principle that we want to start with the safe model and learn more before we add unsafe things.

The argument that it is less complex to allow multiple isolated parameters than it is to prohibit them hasn't held up, as this discussion has repeatedly shown.

Multiple isolated parameters should be removed from this proposal. They can be added back when there is a way to safely indicate that various different actor instances share the same executor. Then, they'll be worth the complexity they add.

Doug

8 Likes
#23

One critical topic I hope to be discussed in the proposal is Async requirements require duplication of protocols (which is raised in the google doc of another proposal Exploration: Type System Considerations for Actor Proposal). If I have an isolated ownership of an actor, I suppose I can cast it into existentials or do other POP programming. But this proposal doesn't seem to address these issues at the current stage. Do we really need to define almost identical protocols of sync/async versions?

#24

After reading this syntax-directed proposal and the other type-directed proposal (Exploration: Type System Considerations for Actor Proposal) , it seems easier to understand and reason about in the type-directed approach (@sync and @async actor types). I don't need to remember more complicated syntax rules (though it may be quite natural from a language designer's perspective), and I'm thrilled that I can use existing POP techniques and other third-party Swift libraries when the actor model is integrated into the language.

In the Alternatives Considered section, the author explains why the syntax-directed approach is preferred:

  1. The proposal treat isolated as a type modifier similar as inout, because "it provides a simpler, value-centric model".

    While I agree that it provides a value-centric model, I don't really think it provides a simpler model. What simplicity means to me is that we address more issues with less rules. The proposal should explain more on why this value-centric model is simpler.

  2. The type of an actor's self can change within nested contexts, such as closures, between @sync and non- @sync.

    It really seems natural to me that if a @sync actor is captured in an async closure, it will be automatically casted to @async type. Even this is a new syntax rule that modifies the captured variables behavior in the current language, I don't see strong reason why we should resist this change.

  3. The design relies heavily on the implicit conversion from @sync MyActor to MyActor .

    Again this implicit conversion seems natural to me. I don't understand why this should be the reason that we don't choose the type-directed approach. The author should explain WHY this implicit conversion is a bad idea.

  4. A @sync actor type is a subtype of the corresponding (non- @sync ) actor type. A non- @sync actor type conforms to Sendable (it's safe to share it across concurrency domains), but its corresponding @sync subtype does not conform to Sendable .

    My question is that: if we allow @sync actor type safely converts to @async actor type, does this behavior really means that a @sync actor type is a subtype of the @async actor type? I can take a simple example here: we can convert T to Optional<T> in Swift, but does this really means T is a subtype of Optional<T>? I don't think so.

So I can conclude that the counterarguments that this proposal raised against the type-directed approach (@sync and @async actor types) doesn't really hold from my view. I hope the author could add more rational and discuss more on this :)

#25

Apologies if I am asking too basic of a question here but why do we need the isolated param (above) when the below "works fine"?

// Why is await not recommended here when in the `isolated` case the suspension still there just implicit?
func inc(a: A) async {
  // Assuming async properties
  await a.counter += 1
}

// Is this not a viable alternative?
extension A {
  func inc2() {
    self.counter += 1
  }
}
#26

Why does this "seem natural"? I cannot point to any other place in the language where a value changes type within a nested context, and we've resisted such changes in the past (e.g., an value x: Int? becoming type Int inside an if x != nil).

We're generally quite careful about adding implicit conversions, because they can have unintended consequences. In this case, the type-directed approach effectively needs two implicit conversions, one in each direction. Going from @sync -> @async is always fine (and needs to happen at some point), but you also need to be able to go from @async -> @sync in certain limited places. This is an "interesting" conversion because it's presence means you are interacting with an actor asynchronously, so each such conversion needs to be part of something that can be done asynchronously (function call, read from an actor property, etc.) and must be covered by an await.

That's a fair point. I suppose I was thinking about how the sequence of conversions you take can affect type identity, which I find a bit odd. For example, consider this:

func f(a: @sync MyActor) {
  let a1: Any = a
  let a2: Sendable = a
  let a3: Any = a2
}

What are the types dynamically stored in a1 and a3? They're actually different, because it's a1 will have dynamic type @sync MyActor, while a3 will have dynamic type @async MyActor because it had to be Sendable to initialize a2. Per your comment about optional and conversions, I bet you can construct a similar case with Optional<T>, so maybe this oddity is a big deal.

Let's go more dynamic, though. What if now I write:

if let a4 = a3 as? @sync MyActor { ... }

Does this dynamic cast fail, because the type stored in a3 is @async MyActor, or is the runtime dynamically checking whether we are currently executing on the this actor and allow the cast to succeed?

Happy to discuss!

There's more rationale in one of the early pitch thread documents, but essentially the argument is that the self of an actor shouldn't be special: you should be able to write a free functions and closures that can have synchronous access to the actors, for example, because that's normal factoring of code. It's very similar to how mutating methods of structs/enums are really just inout on the self parameter, so mutating just isn't all that special.

isolated parameters on closures also let you write things like the runOnActor operation mentioned above.

Doug

2 Likes
#27

Thanks, I'd like to share more of my thoughts here!

Let's continue discuss the points, first with the value changes type within a nested context issue:

In the type-directed proposal, it says

So from my point of view we are making an analogy between the @sync/@async actor type switching and the implicit inout YourStruct. But I regard it as a trivial issue if you think this behavior is unnatural, because we can choose an explicit way instead:

let myActor: @sync MyActor = ...
detach {
  await myActor.doSomething() // Compile error! we always reject implicit capture of @sync actors inside @sendable closure
}

detach { [async myActor]
  await myActor.doSomething() // No compile error. we explicit capture myActor into an @async actor object
}

As you can see, it's a bit like [weak self] pattern in the language.

So actually what make you worried is the reverse direction of casting you mentioned in the proposal. In the type-directed proposal, there are both safe and unsafe ways proposed to let us cast from @async actor to @sync actor:

// This is the safe way to cast
extension AnyActor {
  func doWithinActor<T>(fn: (@sync Self) -> T) async -> T { 
 }
}

// This is the unsafe way to cast
func unsafeActorInternalsCast<T: AnyActor>(_ x: @async T) -> @sync T

Can you share more thoughts on these safe/unsafe methods? I'm not sure what makes you worried/unhappy with the current design of casting @async MyActor to @sync MyActor.

Edited: Can you explain more about the oddity here? I guess you are saying that a1 and a3 should have the same type (while they aren't in the case). Does the implicit casting of @sync MyActor to @async MyActor make you confused here? I can argue that when we convert T to Optional<T>, we are making a normal value conversion. But when we convert @sync MyActor to @async MyActor, we are actually making a type promotion here.
Update: Since Sendable is just a marker protocol (which doesn't contribute to binary file), can we really declare an existential of Sendable like let a2: Sendable?

I think the dynamic cast would succeed, because we are still passing an object with dynamic type of @sync MyActor.

P.S. When I am consuming the type-directed proposal, I also find it's not perfect. For instance, we may still need nonisolated let to make some properties/methods accessed synchronously from outside, but still have an implicit @async self within their bodies. But I hope that we can combine the pros from both the value-directed and type-directed approaches, which could be the foundation for future actor usage :)

#28

I recognize the closure is in the actor code. I'm trying to point out that the await is not.

Suspensions in the actor model allow reentrancy, which is critical "safety" problem (logic safety, not in memory safety) we need to help programmers with. The actor model was carefully built to address this by making potential suspensions explicit with an await in your code. This violates that principle in an (as far as I can tell) pretty unprecedented way.

I don't see why we would add syntactic sugar to do that in any case, and I don't see why we would start out a base type system proposal like this with sugar in it. If you're interested in sugaring away awaits, then it should be done in a more consistent way, and we should carefully weigh the global tradeoffs to logic safety.

Yes, I'm not arguing about this. I'm arguing about a specific form of syntactic sugar that is added in this proposal that causes an implicit jump back into the actor context. I'm arguing that jump should be explicit. My request is a tiny change to the proposal, it does not undermine the goals as you seem to think.

I'm keenly aware that "isolated" is not a type modifier in this proposal, seriously I remember the iteration that got us here. I am trying to make analogies to another model that you also know well in a way to communicate an issue with you. You are pedantically pointing out that we're not using that model, but the issue I'm pointing out still holds.

Feel free to replace "isolated actor references aren't Sendable " with "isolated actor references need to be rejected in the same places that require Sendable".

The closest analog in the language (inout) supports multiple parameters and has an analogous safety issue. It is checked dynamically (exclusivity checking) just like this can.

I'm tired of arguing about this, all the points have already been made. I agree it isn't essential to the model and can be added later.

-Chris

3 Likes
#29

This proposal (value-directed approach) can actually be separated into two new features:

  1. Allow nonisolated keyword before property/method declarations in an actor type.

    This feature is based on the fact that some data of an actor is non-isolated, so we need a safe mechanism to visit this part synchronously from outside. This also enables actors conform to some pre-async-era protocols with their non-isolated members.

    From my point of view, this feature isn't much controversial. It solves a simple problem with a simple way.

  2. Add isolated as a parameter modifier, as well as to capture list in closures.

    This feature is based on the scenario that sometimes we want to synchronously visit an actor, as long as we are in the isolated domain of it.

    I think this feature is quite controversial, because it solves a much smaller scope of issues than the type-directed approach, while differ from it in a source-breaking way. Once we choose this value-directed approach, it would be much harder for us to solve the issues that could be solved in the type-directed approach (e.g. duplication of sync/async protocols, unnecessary async methods on actors forced by protocol abstractions). I hope the authors could discuss these issues in the main body of proposal, so that we could have a clear picture of how actors will interact with protocols in the future.

#30

I can't imagine us accepting the above.

[weak self] is a good point! It's technically a new declaration, but it feels like it's changing the type of self in a nested context.

To be clear, this is expressed as

extension Actor {
  func doWithinActor<T>(fn: (isolated Self) -> T) async -> T { ... }
}

in SE-0313, and it's safe either way.

This is inexpressible in SE-0313 in this form. You could do some tricks to create an unsafe variant of doWithinActor, though:

extension Actor {
  nonisolated
  func unsafeDoAssumingWithinActor<T>(fn: (isolated Self) -> T) -> T {
    typealias UnsafeFn = (Self) -> T
    return unsafeBitCast(fn, to: UnsafeFn.self)(self)
  }
}

I consider that to be a better API to unsafeActorInternalsCast, because it narrowly scopes the unsafe access.

The oddity is that well-formed conversion paths, @sync MyActor -> Sendable -> Any and @sync MyActor -> Any, produce values with different dynamic types.

Yes, you can. It has the same representation as an Any. It's even important and useful if you need to type-erase and still pass something across concurrency domains.

I don't believe this is correct. a2 cannot contain a value with dynamic type @sync MyActor, because @sync MyActor is not Sendable. It contains a value with dynamic type @async MyActor.

For that case to succeed, we would need to implement a runtime check that permits an @async A -> @sync A cast by checking whether we're running on the actor. This is doable, and might be a good idea, but let's make sure that we're on the same page about how the model would work.

Doug

1 Like
#31

Yes, we can separate the two features. nonisolated can be described as making an instance member of an actor not be treated as isolated, so it doesn't actually depend at all on the presence of isolated parameters. This is how nonisolated was described in the SE-0306 before it got split out into SE-0313.

If indeed there consensus is that we got nonisolated right but isolated parameters need more work, then splitting the two apart is a reasonable way to make progress.

Doug

3 Likes
#32

Glad that we have a workable approach to solve the type-change-in-nested-scope issue :)

Do you mean that it's safer because we couldn't pass around isolated Self to other parts of the code? I agree on this point. But I am also thinking we may not need an unsafe casting method in the first place.

Thanks for the explanation! Now I'm thinking if we should still allow implicit conversion from @sync MyActor to Sendable, since we have an explicit way to capture @sync MyActor as @async MyActor in a Sendable closure using [async myActor]. If we prohibit this kind of implicit conversion, I suppose we will no longer have a conversion path like @sync MyActor -> Sendable. Will the oddity go now?

If we can dynamically check that whether we're running on an actor (not the thread/queue it's backed), that would really of great help.

#33

An abbreviated review because, unfortunately, deadlines for SE-0311 and SE-0313 are coincident.

I think this proposal does address a significant problem, and I'm please with the overall direction. I think it does fit well, in general, with the overall feel and direction of Swift's concurrency story.

Having read through the dialogue so far I would share @Chris_Lattner3's concern about any non-locality of await signaling possible suspension points. If indeed this is an aspect of the proposal that can be refactored and added later as sugar, then I'd be in support of doing so and re-evaluating at a later point whether the sugar is warranted.

With respect to the isolated part that's new to SE-0313:

Way back when, we had a whole proposal to move inout to the right of the colon because it's a type modifier. As a corollary, we've recently put property wrappers to the left of the colon because it's not a type modifier but a parameter modifier. This proposal breaks with this distinction: it makes isolated a parameter modifier but puts it to the right of the colon.

Now it is true that, without an @ sigil, putting isolated to the left of the colon might be a beat more difficult to read: func foo(isolated _ bankAccount: BankAccount). But in the context of syntax highlighting it really shouldn't be a deal-breaking problem. I'm wary of blowing apart a distinction that we've assiduously cultivated in order to help users understand what's a type modifier and what's not.

3 Likes
#34

With inout, we have it to the right of the colon:

func f(a: inout Int) { ... }

The value type of the parameter a, when you use it, is Int. There is no type inout Int in the type system. Rather, the inout is associated with that parameter within the function's type, i.e., the function type includes the inout:

let fn = f    // type of 'fn' is '(inout Int) -> Void'

Property wrappers and result builders on parameters are a bit different. They are part of the declaration but don't affect the type of the parameter when it's used or the function type. Hence, they go on the left.

The way I think of it is that when you go from the declaration of a function to its type, you drop the stuff to the left of the : and keep the stuff to the right of the :.

isolated parameters are following the precedent set by inout: they don't affect the type of the parameter, but they do affect the type of the function.

Doug

9 Likes
#35

Accepted with Revision

The core team (with discussion with the proposal authors) has decided to accept a subset of the proposal. The revised (and accepted) proposal has the following parts removed:

  • Support for multiple isolated parameters
  • The proposed changes for closure isolation control

The removed parts, which were the most controversial sources in the review discussion, can be revisited in a future Swift Evolution proposal.

The review discussion also discussed the tradeoffs between a syntax-driven versus type-driven approach. The core team discussed the tradeoffs of the two methods and observed that a type-based approach would (by design) pervade the type system and potentially result in a more complicated system. The core team thus preferred the syntax-based approach in the proposal.

Reviews are a critical part of evolving Swift! Thank you to everyone who participated in this review!

10 Likes
[Accepted with revisions] SE-0313: Improved control over actor isolation
Terms of Service

Privacy Policy

Cookie Policy