1

Earlier today I discussed with @ktoso about my attempt to create an automatic registering of DistributedActor implementing a "distributed protocol" or resolvable protocol in the sense of SE-0428

Here is the problem is more details. Imagine you have a protocol Greeter in a library Greeter

@Resolvable
protocol Greeter: DistributedActor where ActorSystem: DistributedActorSystem<any Codable> {
    distributed func greet(name: String) -> String
}

Thanks to SE-0428 a stub distributed actor will be generated by the macro:

distributed actor $Greeter<ActorSystem>: Greeter, Distributed._DistributedActorSub where ActorSystem: DistributedActorSystem<any Codable> {
}

// extension with stubs implementation of Greeter protocol

Now we have an implementation EnglishGreeter of this protocol:

distributed actor EnglishGreeter: Greeter {
    distributed func greet(name: String) -> String {
        "Hello \(name)!"
    } 
}

A client will resolve the distributed actor using:
$Greeter.resolve(id: actorId, using: actorSystem)

Of course to do that we need to know actorId and that it can be resolved as a $Greeter

First instinct is to call a method like actorSystem.register(id: actorId, as: $Greeter.self) and broadcast the relationship to all nodes in the distributed actor system. A similar things is done in ClusterSystem with dedicated typed key if I understood it correctly (even if ClusterSystem didn't adopt SE-0428 (yet?)).

Now the question is how, in the ActorSystem, can we register automatically this distributed actor ?

The ideal place is in the DistributedActorSystem in
func actorReady<Act>(_ actor: Act) as the actor is fully initialised and ready and we have the actor instance at our disposal.

Problem is: we have no way to "see" that the Actor is implementing one or more resolvable protocols and what are the stub types. Broadcasting to all nodes "here is the actorId of EnglishGreeter" would not allow any client node to resolve it to $Greeter as EnglishGreeter type is unknown to the client.

We discussed two solutions with @ktoso

1/ Using a protocol refinement:

protocol DistributedProtocol {
    associatedtype Stub: Self
}

extension Greeter: DistributedProtocol {
    typealias Stub = $Greeter
}

func actorReady<Act: DistributedActor>(_ actor: Act) {
    if let actorDisitributedProtocol = act as? any DistributedProtocol {
        register(actorDistributedProtocol)
    }
}

func register<Act: DistributedProtocol>(_ actor: Act) {
     // Do something with Act.Stub.self and actor.actorId
}

But this approach limit the ability of a distributed actor to implement two protocols that are resolvable.

2/ Rely on runtime metadata and have a function that for a given type provide all the stubs types
_implementedDistributedProtocols<Act: DistributedActor>(_ type: Act.Type) -> [any Any.Type] .
So for EnglishGreeter it would return [ $Greeter.self ]

Do you think this problem should be addressed somehow ?

1 Like
2

Yeah we definitely have more work to do with the resolvable protocols. The registering is one tricky bit, and another seems to be passing them around -- such that a remote peer who does not have EnglishGreeter but that's a separate topic...

I personally am thinking that the runtime function we talked about and you mention here as 2/ is probably viable -- we need some form of dynamism here... It'd be probably preferable if we have this as static information rather than walk protocols...

I don't actually mind if we have to register the "exposed" protocols, so if we had to do:

CoolActorSystem { settings in 
  settings.advertise/expose/publish((any Greeter).self)

That's not too bad...

In systems looking to use @Resolvable so far we've used register((any Greeter).self, factory func) which bound it to a concrete implementation as well...

We have a related problem that we missed during review though, as a remote may want to:

distributed func get() -> any Greeter { 
   return EnglishGreeter(...)
}

which would encode as EnglishGreeter, but the client side must resolve this using $Greeter.resolve -- since they don't have english greeter. I'll admit we missed this during review and we need to find a solution for it. Luckily though it just builds on top of the existing resolvable protocols.

These two issues seem a bit similar to me... we need some more dynamism so the system can know when to use a proxy etc etc...

Thank you for starting the thread, I'll see what we can come up with here.

4

Protocol are annotated with the @Resolvable macro so the protocol refinement with DistributedProtocol could be made automatic by the macro I suppose.

So you are registering the protocol + factory on both side: client and server I suppose ?

About giving the ability to return distributed actors / distributed protocol that would be a nice addition to the system. Is it already possible to exchange concrete implementation of DistributedActor like that ? Never tested it but will give it a try. I see there is a actorSystemKey CodingUserInfoKey so may be already possible !