[Proposal] Distributed Actor Isolation

[ktoso (Konrad 'ktoso' Malawski 🐟🏴‍☠️)]

@Douglas_Gregor

Yes, that sounds right. This feels a lot like SE-0313, where we took a lexical property ( self in a method is isolated) and generalized it to be part of the type system with isolated parameters.

Right, I feel we'll want to revisit this but leaving it out of the proposals for now.

It actually plays fantastically with:

@Douglas_Gregor

Stored properties cannot ever be access from outside the distributed actor. They cannot be declared distributed nor nonisolated .

Per the discussion above, we don't have to be this restrictive.

I didn't lift that restriction yet because I felt we needed local to make use of it. Specifically we'd be able to implement the "better" when local, and allow for this:

distributed actor Counter {
  // ok, allow declaring nonisolated properties:
  nonisolated let counter: AtomicInt // silly but valid nonisolated use
}

func test(c: Counter) { 
  c.counter // error: distributed actor-isolated property, even though noniso
  c.whenLocal { (c: local Counter) in // no async hop here (!)
    counter.increment()
  }
}

// but also... I wonder if we could even allow the following:
func test(system: DistributedActorSystem) {
  let c: local Counter = Counter(system: system)
  c.counter // OK, we know it's local
  func test(c)
}

func test(c: Counter) {
 // lost the `local Counter` so usual isolation applies.
}

I can see this be very useful in testing.

I agree with your statement here:

@Douglas_Gregor

I think the restriction you want is that stored properties cannot be accessed if the actor isn't known to be local. Or, you can write this in terms of the more general rule governing the addition of effects: if we would need to add the throws effect to read the stored property, then the access is ill-formed.

Yeah that's right, I think I can reformulate this like that and we'll reimplement how nonisolated is interpreted to match this, I think that's a much better model than just banning nonisolated on stored properties

---

While we're mentioning that local keyword, I think it would also play well with protocol conformances actually... We can conform to such a thing on our "local" side, but we cannot actually call it unless the base of the call is known local:

protocol PA {
  func hello() async 
}

distributed actor DA: PA {
  // it's not throwing in the protocol, so we cannot conform to it using
  // distributed func, but it could totally be conformed to by the "local" side...
  local func hello() { ... } 
}

I wonder if this could be legal. It seemed to me like it could, but I have not dived very deep into thee local conformance question

I'll reword the nonisolated restriction into the rule stated above for now, thanks!

---

@Douglas_Gregor

The associated type name DistributedActorSystem is really long. How about just ActorSystem here (since we're already in the DistributedActor protocol), and then DistributedActorSystem can remain the name of the protocol?

Hah, I struggled with naming here to be honest, so thanks for another pair of eyes on it! I was a bit worried about "just" ActorSystem for two reasons 1) it sounded like the normal actors don't have an "actor system" but semantically all actors really form systems, just that here we need the explicit type; 2) I guess I'm still attached to the ActorSystem meaning "the cluster system" but that's my historical attachment and a mistake to stick to it

Re-reading all this again: I think you're right, associatedtype ActorSystem should work well. Thanks for chiming in.

@Douglas_Gregor

• The type alias Identity plays exactly the same role as the type ID from the Identifiable protocol. Should we use ID here everywhere rather than having two names for the same thing?

Yes, that's right, it's fulfilling the exact role as Identifiable's ID really.

And I took a stab at hiding the ActorIdentity protocol entirely in this proposal already, we only have associatedtype Identity: Hashable & Sendable the previously known protocol ActorIdentity is gone.

I'll admit I was reluctant about this because how (subjectively) it made actor system APIs look ugly (system.assignID, system.resignID, system.decodeID), but "consistency is king ", so honestly there's only one right thing to do here: make it ID everywhere.

@Douglas_Gregor

The property id should have type Identity (or ID if you take my suggestion immediately prior to this), not ActorIdentity .

Whoops thanks, that's a leftover, I thought i got rid of all of ActorIdentity, I'll grep through again as I change everything to ID.

@Douglas_Gregor

• As noted above, I think this should be named DistributedActorSystem )
• You can default SerializationRequirement to Codable in the definition, e.g., associatedtype SerializationRequirement = Codable

Sounds good on both

@Douglas_Gregor

While I can see why AnyDistributedActorSystem can be useful (e.g., to decide between different actor system implementations dynamically), I don't think we should use it as a default, because it pushes folks toward a less-efficient, type-erased implementation. Rather, I'd expect each library to provide a DefaultDistributedActorSystem so there is a good default tailored to the libraries you use. One can then opt in to type erasure.

Good point, and very true -- people will usually use exactly one type of system. Maybe two if they know exactly what they're doing.

I have one use case in mind where there will be two actor systems, but really they're handling the same transport, just that one of the systems assigns a Codable ID and the other one not. This will allow us to opt-in certain actors for "sending them around" which is a thing we want to have tight control over in XPC for example.

@Douglas_Gregor

As an aside, AnyDistributedActorSystem either needs to be given a complete API or its mention should be removed.

Given the above, I think let's better remove it completely for now. It could make a comeback, we'll see.

---

@Douglas_Gregor

protocol ClusterActor: DistributedActor 
  where Self.DistributedActorSystem == ClusterSystem {
}

Ah, interesting one... will do

@Douglas_Gregor

A distributed actor's designated initializer must always contain exactly one DistributedActorSystem parameter. This is because the lifecycle and messaging of a distributed actor is managed by the system.

Hmm. Should we recognize this parameter because it has the same type as DistributedActorSystem , or should we also require the argument name to match (e.g., it must be called actorSystem )? I have a vague preference for the latter, because it's easier to syntactically match, and type identity is not always an obvious thing... one could turn an Int into a distributed actor system and make it very, very annoying write initializers that work.

This is done by type indeed, and I had hoped to keep that, it is nice to be able to say:

Worker(id: 123, on: nodeA)
// init(id: Int, on system: ClusterSystem) { ... }

True about making random types conform to DistributedActorSystem which then makes it messy, but in reality is this something to really worry about? I think not, but perhaps you've seen some cases indicating otherwise in the past?

@Douglas_Gregor

I don't think I understand why we're making the designated vs. convenience initializer distinction here, especially when part of the discussion of SE-0327 is about eliminating this distinction.

Ah yes, the reason is deeply intertwined with actor initializers, and also Kavon's recent work. I didn't dive into the depths of that in this proposal as it can get very long... but I'll try to summarize and put a short version into the proposal as well.

This is because the point at which an actor is "fully initialized" must immediately invoke system.actorReady(self), and the point where "fully initialized" happens, is only in designated initializers. So what we mean by "convenience" here is really only "an init that does delegate, to an init that does not delegate", since the non-delegating initializer is at the root of the initialization and is the one which will cause both the actor to become fully initialized (with all the isolation implications Kavon is working on), as well as ready-ed (meaning it may begin receiving incoming messages).

In that sense we really only care about "non delegating initializer" which is the one where the id = system.assignID(...) and system.actorReady(self) calls must be made. Delegating initializerswhich in practice are spelled as convenience initializers. We'd love to not have to say convenience because it doesn't matter for actors at all (because no inheritance), but if they'll continue to exist or not depends on SE-0327.

---

@Douglas_Gregor

If a distributed actor's Identity conforms to Codable , the distributed actor automatically gains a Codable conformance as well.

I'm struggling a bit to understand how this specific use of Codable relates to SerializationRequirement .

The actual implementation of the requirements in those protocols might end up being synthesized (they will for Codable ) or could come from some implementations in a protocol extension. Is that the general rule here, or is Codable special for some reason?

The plan I had in mind was indeed specialized for Codable only.

Codable is special in one way: the decoding has to be implemented via an initializer (Decodable.init(from:) -> Self) and that is not possible to implement using our "resolve" capability in present day Swift. Specifically this piece:

// distributed actor Player: Codable, ... {
  nonisolated public init(from decoder: Decoder) throws {
    // ... 
    let id: Identity = try system.decodeIdentity(from: decoder)
    self = try Self.resolve(id, using: system) // !!!

Because actors are reference types, like classes, today they hit the following restriction here: error: cannot assign to value: 'self' is immutable. So what we do for distributed actors is to just lift this restriction for this initializer because this actually works, but is just blocked at the typesystem level.

@Joe_Groff actually worked on this a while ago Allow `self = x` in class convenience initializers and as I discussed this with him a while ago it basically works, but we'd need a separate SE proposal to unlock this for all types. I was hoping to do this separately, and then every class/actor could implement such things.

Other serialization mechanisms can do some func static func deserialize(as:) -> Self which would work fine.

Should it be the case that a distributed actor should implicitly conform to the protocols in SerializationRequirement whenever its Identity type conforms to the protocols in SerializationRequirement ?

It does not really have any relation to SerializationRequirement, it just so happens that an actor system transport that makes use of SerializationRequirement = Codable and makes the ID: Codable, effectively is saying "actors which I manage, and assign my ID to, are Codable, and therefore can be passed to distributed methods".

Though you're right now that I re-read this that other serialization mechanisms would be forced into a more annoying adoption route, they would have to do:

protocol CoolActor: CoolMessage, DistributedActor {
  func toCoolMessage() throws -> CoolMessage
  static func fromCoolMessage(message: CoolMessage) throws -> Self
}

as we can't do the extension DistributedActor: CoolMessage where ID: CoolMessage {} trick...

I'll be giving this piece more thought now as I'm working through the runtime with the new serializaiton calls that @xedin has been working on

I don't think we can say "all protocols that the ID conforms to the actor does as well" that seems too random, the ID could be conforming to all kinds of things after all... If we needed to open this up I wonder if that'd be another typealias...? That would be unfortunate additional complexity, but I also think this can remain out of scope until proven necessary perhaps?