[Pitch #4] Actors

It's close. You won't be able to await inside the add function, so you would need something more like this:

nonisolated func add(_ query: Query) {
  Task.runDetached {
    await self.asyncAdd(query)
  }
}

where Task.runDetached is part of the Structured Concurrency proposal under review now.

We've seen this come up a bit when porting code, and have a prototyped feature we call @asyncHandler that automates this a bit. It would allow your example to compress down to:

@asyncHandler func add(_ query: Query) {
  // body of your asyncAdd
}

Doug

2 Likes

Yeah, I think that we'll eventually want to support setters but not inout. I agree that starting conservatively is the right way to go in any case.

Right. To be clear, the reason I'd prefer to remove it from the proposal is more for "reasons of principle" than "practical reasons". I agree that the utility of such things are very narrow, but adding special cases like this to the compiler reduces orthogonality, I don't think it will detect problems in practice.

If people trip on this in the future then we can look at why that is happening in practice and add a warning or whatever the best remedy is based on the details later. This is better done lazily than proactively IMO.

I understand the claim, and I hear your very clear insistence, but you're sort of ignoring my point. The issues involved have nothing to do with actors. The exact same issues arise with structured concurrency. Furthermore, the model proposed doesn't work as far as I understand it. It would be good to clarify the model if it does work, perhaps it just isn't coming out in the proposal.

No, it really isn't. I'll split this out to a separate post because it really is the crux of my concern with this proposal as written.

Ok, it makes sense to split it out of the proposal and add it later if unclear about this, but this is highly precedented with AnyClass and it seems like it will have resilience issues if added after Actors 1.0, so this would benefit from some significant thought.

Right, I agree.

While I understand the concern, Actors already have a good way to deal with this: access control. The utility of something like a "run" method is that it gives a cross-actor caller the ability to work with multiple public properties without the risk of interruption between them. This is a pretty important utility for actors with publicly exposed state. Same thing for access to sync methods (not just state) without interruption.

If you don't want people poking at your state and sync methods like this, then don't mark them public!

Thank you for expanding the issue, but I really do understand how this works. I think you're missing my argument though, which is basically:

  1. This catches no bugs, because even if you define such a method, you can't invoke it using safe swift. There is no hole this closes.

  2. There are narrow but valid use cases for this.

  3. Keeping this uncheck is simpler and more orthogonal in the compiler - it composes out.

The argument for rejecting this is basically that "most uses of this are misguided". While we do diagnose some (very few!) such things with warnings, this is generally not what swift does -- and for good reasons. This is akin to warning on all force unwraps.

-Chris

No, it's not that the uses are misguided, it is that the it is semantic nonsense, unless the extra edge case handling is implemented and we compiler enforce the correct usage.

There is no such thing as "run simultaneously on two actors." It is only because a series of special conditions that make this a thing that can exist: specific exact same, serial executor used by both, which also has to be provable at compile time.

Practically speaking, how would this function be even implemented: where would this function body be scheduled? There's no correct answer for this. A function with 2+ isolated parameters has no idea where to be scheduled, and picking one over the other is arbitrary and wrong in any case -- either one actor will be accessed unsafely, or the other one.

Unless we implement the checking I discussed above of course, but we don't have that and I don't think it's planned in the initial scope.

I guess you mean simply because it should not be possible to get hands on two isolated actors, because if we're in some actor isolated closure, we can't capture the isolated actor anyway etc?

This may be true but is somewhat accidental...

Above you claim you cannot invoke it wrongly using safe swift; how would this be invoked in the narrow use-case that you claim here, and be safe?

I would rather we push unsafe uses to one specific unsafe way to interact with actors: which can be the addition of the withUnsafeActorInternals (which I quite like btw!), or fomr form of nonisolated(unsafe)/@actorIndependent(unsafe).

Rather than allowing definition of these weird functions, we should today push people to "well, this is pretty surely wrong, use unsafe things if you really know it's not", rather than allow defining those double-isolated functions which are pretty much always wrong.

And then in step 2 we can make those be provably safe -- and I definitely see a lot of use cases for these, especially on the server side where reusing the same event-loop for many pieces of a pipeline is a very popular pattern.

Splitting this out, I think there is a very big issue here that is not coming through clearly in my feedback. I strongly believe that nonisolated should be split out to a follow on proposal, but I am /not/ arguing that we drop it from the model. This is exactly how we're handling global actors - split out to a follow-on proposal - even though they are also critical to the model.

Through the evolution of the actors proposal, we've progressively pared back the functionality ascribed to actors, generalizing that functionality and making it orthogonal to actors. This has made actors and structured concurrency more of peers that share the same implementation details (e.g. Sendable).

What are the fundamental nature of actors?

An Actor is a nominal type that collects a bag of state together and allows one to define methods on it, providing isolation through async messaging of its mailbox. This is a simple and beautiful idea!

By analogy, the non-concurrent part of swift has closures and structs/classes. Closures allow one to capture state and provide a single operation against that state. A struct allows you to explicitly declare the state you want to represent, and define multiple operations against it. This is what I meant in the intro rewrite when I said actors are a "state first" design like structs/classes when compared to closures.

Nominal types are great, because they allow a bunch of cool things that closures cannot do: e.g. they can conform to protocols, allow POP abstractions built with them, are the key for resilience, and support for multiple methods that interact with that state.

@sendable closures in structured concurrency and actors provide the same duality for the concurrent world!

The problem with the presentation of the proposal

Getting to my problem, the proposal does not express this simple nature! Where actors should be a simple nominal type with isolated self and the sync/async promotion dance going on, instead the proposal spends most of its words talking about complexity related to nonisolated, which isn't general and isn't the fundamental nature of actors. nonisolated is all about /breaking/ the fundamental property that actors provide - isolation of data.

To reiterate, I agree with you that nonisolated may be an important part of the pragmatic eventual user model, but it is an additive feature, and the presence of all the nonisolated complexity in this proposal is distracting and makes it very difficult to evaluate and reason about the actual actor part of the proposal.

The basic actors proposal (ignoring the nonisolated stuff) is not good enough

Furthermore, the proposed protocol model has a significant number of problems that will cause excessive use of async, does not allow protocol abstraction across actor definitions, doesn't interact well with generics, and has other problems I laid out in the motivation of the whitepaper I wrote. While the introduction of 'nonisolated' parameters is a positive step, there are still major holes in the proposal.

At the end of the day, I hear you state vociferously that things are "non-negotiable" but the model you're proposing is missing basic abstraction facilities that all other nominal types support. I can't see how introduction of actors without these basic features would be considered acceptable.

In short, while it isn't "necessary" to split all the non-isolated complexity out of the proposal, doing so would make it much easier to evaluate the core of actors, land it cleanly, then build on top of it. "non-isolated" members are not an acceptable substitute for proper interaction with the protocol and generics system in Swift.

-Chris

6 Likes

Actor.run { }

Yes but not in the way you're suggesting to use access control. It is the run, defined by specific actors that should be either internal or public, depending on the library's design. If it is predefined on the Actor protocol, we lose this flexibility.

If the function exists on Actor, it will be public/internal depending on what MyActor is. I may want to offer this ability to my library but not to everyone using the library.

Yes and that's fine, but should be opt in by actors which actually do this. Nothing I'm saying is preventing people from defining such function. I specifically argue for offering it on MainActor even after all.

That is not the concern; the concern is operations performed on this actor, please read my IOActor example again -- it is not about any state or any functions of the IOActor being touched. It is purely that someone may throw arbitrary computation at this actor.

And perhaps it has to be public because e.g. it has to be passed somewhere. It is easy to say "so don't make it public" but the reality of API design with actors is that it often is useful to allow passing them around, and if we force every actor to offer this completely arbitrary capability, we can't escape it.

If run exists on every actor ever defined, there's no way to "hide" or prevent people from using this.

If it is not pre-defined, anyone who wants this capability is free to define it for actors which want to support such usage pattern.

The actor model allows defining multiple actors that share an executor, so it is not semantic nonsense. The fact that the type system doesn't allow one to express that doesn't mean it is nonsense. We support unsafe casts specifically to allow Swift programmers to escape the type system. It is perfectly reasonable to touch the mutable state of two actors that share a context, and can be useful in advanced cases. If it weren't important, then we shouldn't allow actors to share a context.

In this case, we could even support a safe cast from an actor type to isolated actor type that dynamically checks that two actors are on the same executor. This would be the actor-cast equivalent of x!.

I think you're missing how these things work - functions aren't "scheduled", they are invoked from existing things (e.g. other actors or sturctured concurrency tasks) that are themselves scheduled.

Consider:

func foo(a: isolated MyActor, b: isolated MyActor) {...}

It's obvious where this gets invoked: it is only invokable from within MyActor's concurrency domain. You can do this trivially by passing self as both arguments, but if you have another actor instance that shares an executor, it is safe to cast that and pass it as well. This is all properly defined (but, as I mention above, I agree this is not going to be widely used ;-).

You seem hung up on whether the compiler's type system can statically enforce that, and I'm making the argument that this is not how Swift treats diagnostics in general and that there is no reason to deviate from the standard approach.

I'm claiming four things:

  1. It is trivially safe to pass self to both arguments to foo above, no unsafety involved.
  2. You can use withUnsafeActorInternals if you know another actor instance is on the same executor as the current actor.
  3. You can probably define a withSafeActorInternals(otherActor, thisActor) cast method that is memory safe by failing/trapping if thisActor and otherActor are on different executors.
  4. Banning this doesn't prevent any bugs in safe code.

Can you explain if you disagree with point #4 and show an example?

I also find this whole discussion very interesting given the proposed safety holes with nonisolated(unsafe) etc which would be a much more appealing footgun to offer people ;-)

-Chris

Hi all, I'm new to both the concurrency pitches and speaking up on Evolution in general, so apologies if some of these questions have obvious answers I'm just not finding in the documents. Maybe they'll at least prompt some example tuning?

I don't see any examples of chained calls in async/await, so I'm not sure if it's possible under the current phase of proposals. I'm trying to understand how something like the following might work:

struct S {
  var balance: Int = 0
  mutating func deposit(_ money: Int) { balance += money }
}
class C {
  var balance: Int = 0
  func deposit(_ money: Int) { balance += money }
}
actor A {
  var vally = S()
  var reffy = C()
}
var accountBook = A()
await accountBook.vally.deposit(10)
await accountBook.reffy.deposit(10)

My first guess is that both of the await lines would fail for one of two reasons: either await doesn't handle chained calls or because they'd decompose as equivalent to

// left A isolation at ;, x-actor ref to setter prohibited
{ var s = await account.vally; s.deposit(10); } 
// left A isolation at ;, x-actor ref to non-sendable ref type prohibited
{ var c = await account.reffy; c.deposit(10); } 

But my more hopeful guess is that one or both of them would decompose to the following:

await account.runOnActor { a in a.vally.deposit(10); }
await account.runOnActor { a in a.vally.deposit(10); }

This would be my first assumption for a fully finished actor model, because Swift IME has spent a lot of effort on devolving a type's interface to supporting member types, but I'm not sure if that's enabled by the current version of await and actor isolation.

I only started reading the concurrency pitches today yesterday, but my first pass of this one left me concerned that refactoring a given class to an actor will implicitly change all my properties to the equivalent of private(set) and force me to declare new setX methods that I already eschewed when designing the original class, because the members' interfaces served as extensions of the owner's interface. I understand the goals of

  1. Cannot pass mutable state across actor boundaries
  2. Prefer not to encourage storms of async a.x += 1; async a.y += 1; ...
  3. Prefer not to encourage large runOnActors of arbitrary computation

But these are all in tension with each other and I don't see explanation of how this design encourages or enables the healthy middle ground of messaging an actor to call one mutable method on one of its childrenā€”because you've already been trying to right-size mutation, which is why you grouped some data up in a supporting type. The closest thing to a suggestion I saw was essentially redeclaring the mutable interface of each member. I'm not sure that is adequate to help us use phase 1 refactors to decide what we want out of phase 2.

Which brings me to my second question:

Is isolated only applicable to a function parameter of actor type, or can it be used for any one parameter? I assume not, because I could not find any examples of this use, but IIUC non/isolated is otherwise an attribute of an actor's members so using it to refer to a parameter-of-actor-type made me doubt my understanding for a moment; once I translated it to a: isolator A it was fine. I certainly understand the desire to keep keyword count low. I also can't avoid pitching the flavorful starring A as an alternative though :wink:

That said, I really like the concept of allowing isolated on any parameter type, as a way to opt a function into an arbitrary actor's executor without adding the actor to the parameter list. I'm picturing something like the following:

actor Hero { // freshly retyped from class
  var stats = Stats()
}
func ponder(hero: isolated Hero) { ..; buffAll(hero.stats) }
class Stats {
  var hitting: Int = 10
  var thinking: Int = 10
  var tolerating: Int = 10
}
func buffAll(_ s: Stats) {
  s.hitting += 1
  s.thinking += 1
  s.tolerating += 1
}

// bad; Stats is non-sendable ref type (and maybe ill-formed to wrap async in a sync call?)
await buffAll(someHero.stats);
// signature updated as part of Hero refactor
func buffAll(_ s: isolated Stats) { ā€¦ }
// ok; semantically similar to
// someHero.runOnActor { hero in buffAll(hero.stats) }
await buffAll(someHero.stats)
await buffAll(someMonster.stats) // actor Monster also benefits!

Is that something this pitch currently admits? If so, I think I'd appreciate an example or two in the pitch documentā€”it certainly clarifies that isolated isn't actually being repurposed in a function signature, but is the same modifier we apply to member decls. Considering Swift often uses multiple related types to support the final interface of a type, it would be nice to see examples of how supporting types might update alongside the newly minted actor.

If this pitch doesn't currently consider thatā€”could it? IIUC the compiler already needs to know the actor isolating each argument at each call site. Here's my reasoning:

  1. There is nothing special about self as of pitch 4; it is just a decl isolated to a Hero
  2. stats is another decl isolated to a Hero
  3. Some compiler operator isolating(nonisolated Actor) exists such that await ponder(someHero) decomposes to await isolating(someHero) { a in ponder(a) }
  4. The compiler uses some sort of isolatorOf() operator to check each of these decls at each call site, to determine things like "is this x-actor ref declared let in its parent?"
  5. buffAll(s: isolating Stats) could decompose to isolating(isolatorOf(someHero.stats)) { a in buffAll(a.stats) }
  6. That decomposition could, possibly as future work, be a projection with a keypath or internal equivalent: isolating(someHero.stats) => isolating(someHero, projecting: \.stats) { s in buffAll(s) }

This would also be another plausible justification for allowing multiple isolated parameters: to support functions that modify two non-actors that share an owning actor/executor.

extension Stats {
  // or global func steal(from: isolated Stats, toPay: isolated Stats)
  // only safe when (isolator(self) == isolator(other))
  isolated func steal(from other: isolated Stats) { 
      other.tolerating -= 1
      tolerating += 1
  }
}
actor HeroOfTime {
  var link = Stats()
  var fairy = Stats()
  // ok with or without non-actor isolated annotations
  func emergency() { link.steal(from: fairy) } 
}
// ok: isolatorOf(fairy) == isolatorOf(link)
await aTimelyHero.link.steal(from: aTimelyHero.fairy); 

I believe (hope) this would be an extension of the previous reasoning plus Chris's first claim:

  1. it is trivially safe to pass self to multiple isolated parameters of a function
  2. this is true because both copies of self trivially share an isolation context
  3. Both link and fairy are also isolated data members of HeroOfTime
  4. Their shared isolation context already needs to be checked in emergency even without the new modifiers.

One more reason this is potentially interesting IMHO: one of the "workarounds" for my first buffAll example would be to re-type Stats as an actor, because actor references are Sendable. Semantically, Stats is just a way to group up some related Ints that a lot of actors use; it's not really appropriate for it to synchronize separately from its owners, who will synchronously manage other properties in response to stat updates. This means I may want actor Stats to be passed its owner's executor upon initialization. Even if I expect all Hero->Stats messages to be optimized onto the same thread, Hero contains many synchronous calls to buffAll and they would have to be rewritten to async, muddying the waters on Stats's supporting role within its owner.

I'm not saying that's the correct architecture, but it's one I can see the current limitations encouraging, with some classes becoming actors out of convenience, who semantically ought to be managed by their owner's executor. Another pitch looks like it will enable that shared executor, so the ability to say "I know I can reference my member actor's members synchronously because I know they're isolated to my context" would be nice.

Okay last question: why are key paths currently disallowed, out of curiosity? Is it just a matter of waiting on Sendable to be accepted before it's workable, or are there some other aspects of actor that would make a readonly IsolatedKeyValue hard?

Doug

Yes, this is the correct reading of the proposal.

I suspect it is your #3 that would end up being the right answer here, or extending the actor type to add a new method that performs the mutation you need.

isolated can only apply to parameters of actor type. It sounds like we should make that clearer and perhaps have an example with an "error: parameter of type 'Int' cannot be 'isolated'" or similar.

Since Stats is a class, a parameter of type Stats cannot be isolated per this proposal. However, I will certainly add more examples showing how isolation parameters can be used.

If key paths are meant to be Sendable, then you don't want a key path that refers to actor-isolated state but can be evaluated for a non-isolated instance of the actor.

Doug

I disagree with this because of where the unsafe code would be.

If declaring functions with multiple isolated parameters were legal, e.g.:

func x(a: isolated A, b: isolated B) {} 

this gives people the impression that this "totally can somehow work, yay". While it only can be working in a very narrow case. Developers seeing such function are led to just "passing whatever makes this work", which would be obtained via withUnsafeInternalState(a) { a in withUnsafeInternalState(b) { b in x(a: a, b: b) } } which the vast majority of the time is wrong.

I claim, that such unsafe patterns should be done inside the functions which to the unsafe tricks, i.e. that the function should be:

/// Actors A and B must be on the same exact serial executor, or this function will trap.
func unsafelyX(a: A, b: B) {
  precondition(a.executor == b.executor) (and they're known to be a serial executor)
  withUnsafeActor(a) { a in withUnsafeActor(b) { b in ... } }
}

This is less wrong in the promises the function signature makes IMHO. It does not promise things about the function magically being isoalted to "two" actors, which simply isn't a thing. The tricks we can do with more knowlage about execution are valid and useful, but they are unsafe tricks and should remains such, and not pretending in function signatures we can do something that we actually can't.

If there was some way to unsafely obtain such isolated A, it is possible for this function to actually be unsafe.

In other words: It is about the where the unsafety is exposed, and I would never want a function that only uses safe spellings, to suddenly become unsafe - which the "multiple isolated parameters" would cause (unless we do the long, involved static proving stuff).


Alternatively, it could be allowed to say "multiple isolated(unsafe) actor parameters" are ok, since this puts the word unsafe into the function signature again, alleviating the concern about a safe-looking function actually not being safe.

But I'm not a huge fan of making this pattern so easy to begin with. I am very much interested in making such uses efficient, safe, and nice in the long run though -- especially since on the server such pattern appears a lot, esp. when using event loop based IO systems, such as NIO.


No, not really hung up on it, but rather I'm showcaseing how hard it is for this "multiple isolated parameters" to actually be safe. This is why I'm saying we should not allow this, and if we end up really wanting to support this, these are the steps that would need to be taken for it to be statically safe.

It definitely is a lot of analysis, and I'm not pushing for it at all, but it could be done if there is enough need for it. It would be a natural loosening up of the model, because we added more checking and are confident this code can ge safe.

I'm also specifically calling out the executor hop-avoidance work which may render this entirely unnecessary, because perhaps hop-avoidance will be good enough. We don't know yet, and this is why I don't think we should just allow signatures which promise more than they are able to guarantee.

Yeah, very much agreed.

Rather than just slapping a run on all actors, since you're designing your actor to encapsulate some specific work, such "change a bunch of options together" should be e.g. exposed as some:

protocol WorkerSettings { 
  var throttle: Int {get set}
  var max: Int {get set}
}

actor Worker: WorkerSettings { 
  var throttle: Int
  var max: Int  
  func reconfigure(_ body: inout WorkerSettings -> ()) {
    body(&self)
  }
}

worker.reconfigure { settings in 
  settings.throttle = 10
  settings.max = 100
}

I think this is a much cleaner and future proof design, rather than just exposing some pretty much arbitrary "run random stuff here" run function.

As I said before though, you can just expose such run(body) if you really wanted to -- but it's up to specific actors to decide if they want to do this or not. Rather than the entire concurrency story encouraging this.

// or really, Iā€™d have the settings be a var settings then itā€™s nicer, but to prove the point about mutating multiple things at once without interleaving this is spelled like this.

So, I think we fundamentally disagree about what kinds of protocols are important for actors to conform to. For me, most interactions with actors are naturally asynchronous---they are coordinators and controllers and owners of data models that need to be consistent.

I am not strongly motivated by the arguments in your white paper because I fundamentally don't see actors as being "the data" that you operate on, because "the data for me should usually be a value type. Your motivation provides this example:

actor MyDataActor {
  var data: Data
  func compressData() -> Data { use(data) ā€¦ details omitted ā€¦ }
  func doThing() {
    let compressed = compressData()
  }
}

public protocol DataProcessible {
    var data: Data { get }
}
extension DataProcessible {
  func compressData() -> Data {
    use(data) 
    ā€¦ details omitted ā€¦ 
  }
}

I don't think I would model things things this way at all. Data is presumably a value type, so compressData would be an algorithm on it that returns a new Data value. It doesn't belong on the actor type. doThing() might be a completely reasonable operation, but I expect it will be something like:

  func doThing() {
    let compressed = data.compressed()
    // do more to the compressed data
    data = resulting-data
  }

I'm trying to come up with another example I've come across to swap in to bolster your argument, but I have not come across one yet. I don't want an actor to be a Sequence: I either want it to store a sequence as a member or I want it to be an AsyncSequence`. I've come across various existing async protocols that are spelled with completion handlers:

protocol Server {
  func send<Message: MessageType>(
    message: Message,
    completionHandler: (Result<Message.Reply>) -> Void
  )
}

And with nonisolated and Task.runDetached, I've taught actors to conform to them:

actor MyActorServer {
  func send<Message: MessageType>(message: Message) async throws -> Message.Reply {
    // actual implementation
  }
}

extension MyActorServer: Server {
  nonisolated func send<Message: MessageType>(
    message: Message,
    completionHandler: (Result<Message.Reply>) -> Void
  ) {
    Task.runDetached {
      do {
          let reply = try await send(message: message)
          completionHandler(.success(reply))
      } catch {
        completionHandler(.failure(error))
      }
    }
  }
}

(Yes, I'll add this to the proposal)

I could perhaps buy that nonisolated can be separated out because it makes the proposal confusing, but I completely disagree with your assessment that the proposal is missing basic abstraction facilities. Actors can interact with asynchronous protocols, which fits well with their nature. They can aggregate other types---usually value types---that can conform to whatever protocols are needed for the synchronous "computation" bit. Actors do not need to themselves be the abstraction for synchronous computation, because we have better types---value types---to cover those cases.

Doug

1 Like

Can we make a simpler and more succinct proposal based on what has been commonly agreed on? This pitch is already in round 4, which may be an indicator that there is too much things to discuss and settle in a single proposal. By putting things that causing arguments and disagreement into a separate pitch we can discuss in a more clear and narrow scope.

1 Like

While I think that something like run will be very important and useful, I don't think it is important to include in this round of proposal. It is an additive library feature that can be considered when the design is more baked out. It seems best to elide it for now until we get more experience to guide the direction.

There will be a lot of different useful library features we should consider in time, and it is much more difficult to remove them later than add them later.

Ummm, yes, they "totally can somehow work, yay", that's why it should be legal. As you say, the cases are present but narrow.

I made this argument before, but there is nothing unsafe about this. It seems completely reasonable to allow:

actor MyActor {
  var state : Int

  // Designed to share an executor with other instances of MyActor, and implemented here.
  func  doThing(other: MyActor) {
    // This would trap if the two actors are dynamically on different executors.
    withActorOnCurrentExecutor(self, other) { other in  // other rebound as isolated
      other.state += 1
      self.state += 1
    }
  } 

or equivalently:

func doThingWithTwoActors(one: isolated MyActor, two: MyActor) {
    withActorOnCurrentExecutor(one, two) { two in  // two rebound as isolated
      two.state += 1
      self.state += 1
    }
}

While I agree with you that the use-case is narrow, so is the use case for shared executor actors. If you think the use-case is non-existent, then we should remove shared executor actors. Your argument that this is somehow only useful with unsafe code is very mystifying to me.

As I've mentioned before, this behavior naturally composes out of the underlying mechanism of how "isolated" parameters work. I agree with you that the keyword isolated isn't necessarily the right name for these things though!

There is no unsafety implicit in this feature. You're proposing that we ban a safe construct because someone may use unsafe code incorrectly somewhere else.

-Chris

Reading your post, I think that the whitepaper I wrote is seriously confusing things. This is my fault, and I can cut it back and rewrite it if helpful just let me know. In the absence of that, please look past any discussion about conforming to existing protocols (e.g. Sequence) and the discussion about redundancy between sync and async proposals. Also, if I revise it, I will remove any proposed solutions, because I care about the problem getting solved, I don't care about any particular outcome (though I'm happy to discuss tradeoffs of course).


I'll try to keep this simple:

My concern here is that we need abstraction facilities to work across actor types. If I have two actors that share common aspects of their design, then I will want to abstract across that with generic operations and protocol extensions (just like I do for structs, classes and enums). As proposed, the actors design does not allow this, because it forces all the protocol requirements to be async.

This isn't acceptable to me. Generics, protocol extensions, and PoP patterns should work across actors -- including the sync internal implementation details -- or else actor types are not first class.

The proposal doesn't address this, and the complexity around "nonisolated" is masking/confusing this and making analysis of the core feature more complicated.

-Chris

2 Likes

Agreed, thanks.

Open to revisiting those in the future, but it is definitely to early to add such very broad functions to all actors, for all eternity :+1:

Rather than all actors, I think this will be useful to offer on a few specific ones (e.g. MainActor).


We're backtracking and adding new API to make it less unsafe/scary during this discussion :wink:

Your initial phrasing assumed only "there can be multiple isolated parameters" and the existence of "withUnsafeActorIsolatedState()", which I think is a foot gun combination, leading people to attempt to use this, get it slightly wrong, and nothing is checked in this world -- though the double isolated function now is actually concurrency unsafe, because of who/how called it.

Indeed, the introduction of some withCheckedActorsOnSameSerialExecutor(a, b, ...) { ... } would be memory/concurrency safe (or crash), and is a much better spelling for this, rather than leading people towards just doing withUnsafe to fulfil these functions, and potentially getting it wrong all to easily.

What I'm not comfortable with is allowing multiple isolated parameters, because it makes it seem way safer than it actually is for authors of such function. Authoring such function should be scary and authors should be notified about it, exactly because if called incorrectly it actually is unsafe, because a plain withUnsafeActorInternalState would always succeed (if the actor is local), and thus we could pass actors on different executors to such x(isolated A, isolated B) function, making it racy.

That, specifically, is the API shape I don't want to end up with.

If we wanted to explore and offer APIs to withCheckedActorsOnSameSerialExecutor(...) { isolated a, isolated b in }, as you mention next, that would be much better, and safer:

I would say though this should not go hand-in-hand with allowing multiple isoalted parameters on nonisolated functions -- rather, that is something I would like to enable later, once we've either gained more ways to make this provably safe, or the need for it is so strong that we accept the unsafety potential.

This function is more clear:

  • it is statically known that if invoked from non actor code, it will may (subject to optimizations, though we don't do those today) cause a hop to isolated MyActor's executor
  • its internals are explicit about the same executor assumption (and function name should be as well)

I specifically spent a lot of time convincing the team that we will need multiple actors sharing (serial) executors in the first place, it was not a part of the considered design until we went through many discussions which finally resulted in the custom executors pitch. So it's not that I disagree with the use-case, just with the manner of exposing it to end users.

I don't know how you arrived at this conclusion?

The usefulness of passing self as both arguments to a function of very small significance, this I think we can agree with? (I.e. the x(a: self, b: self) case). And, today, for all other cases users would have to reach for the withUnsafeActorIsolatedState which yes, already is unsafe code. I welcome the discussion about the withCheckedActorsOnSameSerialExecutor it would definitely be a better tool to offer developers who really need to do this (e.g. NIO style apps will be in that camp), but I also think it's a follow up thing, rather than core to the proposal.

As such, because the risks of many-isolated parameters, I'd rather ban them until we have a complete understanding how we want to be using them. What specific features we want to and will be able to offer, and how much it actually matters. Especially while keeping in mind our future work with custom executors and cross-actor-hop-avoidance -- as we want to optimize away any such hops as much as possible when two actors are detected to be on the same serial executor during runtime.

Long story short, I would prefer to restrict and slowly open up this, rather than open up and invite people to have wrong assumptions about what isolated means and does. We can and should definitely revisit this though, but once we actually have custom executors and have a practical real world understanding of the actual needs and their implications.

Hey all, I've made a bunch of changes on the road to a "pitch #5", based on feedback here. You can preview the changes at https://github.com/DougGregor/swift-evolution/pull/62, and here's the summary:

  • Changes in the fifth pitch:
    • Drop the prohibition on having multiple isolated parameters. We don't need to ban it.
    • Replace Sendable with Sendable to better track SE-0302.
    • Add the Actor protocol back, as an empty protocol whose details will be filled in with a subsequent proposal for [custom executors][customexecs].
    • Replace ConcurrentValue with Sendable and @concurrent with @sendable to track the evolution of [SE-0302][se302].
    • Clarify the presentation of actor isolation checking.
    • Add more examples for non-isolated declarations.
    • Added a section on isolated or "sync" actor types.

From my perspective, the proposal is not complete without nonisolated, and my read of the situation is that there is no actual disagreement about whether nonisolated is a good idea, or it's semantics, or even it's spelling (now). Yes, the proposal would be somewhat smaller without it, but at the cost of having yet another concurrency proposal that scatters the knowledge further. It fits well alongside isolated parameters to have the modifier that disables the implicit isolated on self. If we rewound the clock to the pre-Swift 1.0 days, could we imagine introducing inout without also introducing mutating?

Global actors was large, so we split it out of the actors proposal. async let was both controversial and confusing the discussion of structured concurrency, so we split it out of that proposal. Both were separable. For me, nonisolated doesn't meet that bar.

The actual disagreement, as I understand it, is whether another feature, called isolated conformances in the future directions of this proposal, is also a critical part of the model. We have neither a set of agreed-upon examples nor a complete design to evaluate that feature. My own experience is that I have not needed this feature. We should keep exploring it further, but we shouldn't hold up progress on actors themselves waiting for it.

Doug

5 Likes

I know a new pitch is coming soon, but I think this applies to both.

It's still not clear to me what isolated and nonisolated really mean and when you would use either of them. Can we get a brief summary? Particularly nonisolated is rather confusing. Does it give safe read-only access to the isolated self? This example from the pitch(es) doesn't make sense to me:

extension BankAccount {
  // Produce an account number string with all but the last digits replaced with "X", which
  // is safe to put on documents.
  nonisolated var safeAccountNumberDisplayString: String {
    let digits = String(accountNumber)
    return String(repeating: "X", count: digits.count - 4) + String(digits.suffix(4))
  }
}

How is the access to accountNumber safe within this computed property? Is it safe because it isn't a mutation? Or are is this safe because it's a property on self, where an external access of .accountNumber would always be unsafe?

On a related note, I really think we need a way to make synchronous mutations safe for external access, even if it requires some internal manual work. Otherwise we have to add async methods which are otherwise unnecessary. For instance, we found it critically important during the development of Alamofire 5 to be able to synchronously update Request state despite the existence of an internal queue where most mutation happens. This is important for something like cancellation, where the user's cancel() call must immediately update the internal state so the ongoing async work that checks isCancelled can see the updated value as soon as possible. It's can also be important for the user's other synchronous access to be able to see the updated state immediately. To accomplish this we used internal locking. Perhaps actors could offer something similar, rather than requiring manual locking?

Thank you and @Douglas_Gregor for your replies! Your answers have been helpful.

I actually did have something similar to this reconfigure-on-a-single-var in an "alternatives considered" section I cut for brevity. I twisted my original Hero example up a bit to avoid chained calls and inout to keep the number of interacting features low. If I untwist it:

actor Hero {
  var stats = Stats()
  var ally: Hero
  func ponder() { ..other work...; stats.buffAll() }
}

struct Stats {
  var hitting: Int = 10
  var thinking: Int = 10
  var tolerating: Int = 10

  mutating buffAll() {
    s.hitting += 1
    s.thinking += 1
    s.tolerating += 1
  }
}

My understanding is that the following would not work:

// 1: not okay; buffAll ends up called cross-actor
// equivalent to { var s = await someHero.stats; s.buffAll }
await someHero.stats.buffAll()

So I'd still need a closure-based method to call mutating methods on stats (or any method if Stats were still a class.

extension Hero {
  // 2: inout needed now that Stats is properly a value type
  func updatingStats(_ body: inout Stats async -> ()) {
    body(&stats)
  }
}
// ok; buffAll called within closure executed within someHero
await someHero.updatingStats { stats in stats.buffAll() }

I have some thoughts on this pattern. That said, I want to stop and check my understanding that call 1 would cause a cross-actor violation. I fear that, by twisting up my first example to make it about a function instead of method, I missed some way that actors enable method calls on data members.

You're right, I'd instead want such a key path to be evaluated via an isolated subscript, which would require me to await if I were on another actor, right? Then the next problem is what relationship that MustBeIsolatedKeyPath has with a MayBeNonisolatedKeyPath, which could be evaluated in a nonisolated subscript. I'd expect both of these subscripts to exist on most if not all actors, but I don't know what relationship they'd have with the current KeyPath hierarchy. That's why I was asking about what the current conceptualization and bottlenecks might be.

Maybe we can get to a shared understanding without asking you to write another white paper.

Let's ignore all of the conformances to existing protocols from the white paper: Sequence, Iterator, Equatable, etc., I think that means we're focusing in on DataProcessible:

public protocol DataProcessible {
    var data: Data { get }
}
extension DataProcessible {
  func compressData() -> Data {
    use(data) 
    ā€¦ details omitted ā€¦ 
  }
}

and its examples.

I think you might be saying something here that is fundamentally different from what's in the white paper. All of the existing protocols referenced in the white paper that we're ignoring now are protocols that value types or classes conform to. The white paper clearly calls out the intent that DataProcessible also work for structs, enums, and classes:

Furthermore, because DataProcessible isnā€™t unnecessarily async, other structs and classes may usefully conform to it as well.

In your comments above (not the white paper), you talk exclusively about abstracting across different actor types "just like I do for structs, classes and enums." So here's a question for you. Let's say that DataProcessible looks like this:

public protocol DataProcessible: Actor {
    var data: Data { get }
}

i.e., it is a protocol that only actor types can conform to. Now, when we define an extension on it:

extension DataProcessible {
  func compressData() -> Data {
    use(data) 
    ā€¦ details omitted ā€¦ 
  }
}

the type of compressData is <Self: Actor> (isolated Self) -> () -> Data, i.e., it's a synchronous function running on the actor. An actor type can conform to it with a synchronous data (example pulled from your white paper):

actor MyDataActor : DataProcessible {
  var data: Data

  func doThing() {
    let compressed = compressData()
  }
}

You can use DataProcessible synchronously when you have an isolated parameter for it, asynchronously otherwise:

func f<T: DataProcessible>(a: isolated T, b: T) {
  a.compressData() // okay to be synchronous
  await b.compressData() // must be asynchronous
}

However, this would not allow, e.g.,

class MyOldProcessor : DataProcessible { // error: MyOldProcessor is not an actor
  var data: Data
}

Does the above, where you can write synchronous protocols that work across actor types but cannot be conformed-to by non-actor types, address your concerns about "generics, protocol extensions, and PoP patterns [working] across actors"?

EDIT: brought up an implementation of what I'm proposing.

Doug