Sorry for the slow reply here @kavon, and thanks for raising a good topic to discuss here!
I was actively looking into related things for the last few days, so wanted to focus there and be able to provide a good discussion here, so let's dive in.
So in normal usage the executor would never be used in practice, because:
- calls to
distributed func on remote actors are effectively calls to nonisolated thunks that notice the reference is remote, and instead of calling the distributed func hello() on the distributed actor, redirect the call to an invocation to its ActorSystem's remoteCall.
So in that sense, an executor never "matters" for remote distributed function calls -- we never use it, because the code-path for remote actors is always nonisolated for all distributed calls. And non-distributed calls are not allowed externally. If we are "inside" a distributed actor (i.e. have an isolated DistributedActor), we are guaranteed that it is local, so all the usual Actor semantics about the executor hold.
Now, my assumption when proposing this API shape was that there's two ways to implement an unownedExecutor for a distributed actor:
- as reference to some global shared executor
nonisolated var unownedExecutor: UnownedSerialExecutor { MainActor.sharedUnownedExecutor }
- the same way we configure other things about distributed actors, by carrying the information in the
ID which is the only stored property the user has a lot of control over (though as usual it depends how much control the specific actor system allows you here),
- we do this a lot actually in practice, so if this seems weird I assure you it's not as weird as it seems
- this boils down to having an ActorID have e.g.
id.preferredExecutor: UnownedExecutor? that you'd use this way to implement the ID (The ActorID can have all kinds of "preferences", and I do think this is a good pattern to stick to, like "which region to prefer" for lookups etc. etc.):
- I (wrongly! so thank you for reminding me to look into this deeper) assumed we'll then implement the
unownedExecutor as id.preferredExecutor ?? buildDefaultActorExecutorRef(self)
- I was wrong here though, because we don't expose the
Builtin.buildDefaultActorExecutorRef(self) and probably shouldn't 
So yes, I agree with your proposal to change the user-facing part here to ? and at the same time I don't think we should make the exposed interface and how distributed actors interact with the rest of the compiler different... So I think we can achieve both goals bo doing the following: make the user-facing API more explicit that "this only matters for local actor" but also keep the synthesized unownedExecutor the same way as default actors to it today:
public protocol DistributedActor: AnyActor {
nonisolated var localUnownedExecutor: UnownedSerialExecutor? { get }
}
distributed actor Worker {
nonisolated var localUnownedExecutor: UnownedSerialExecutor? { ... }
// KEEP the synthesized implementation, but make it do this:
// nonisolated var localUnownedExecutor: UnownedSerialExecutor { // do not mark as default actor though
// localUnownedExecutor ?? Builtin.buildDefaultActorExecutorRef(self)
// }
}
This way we don't have to special case a lot of places in the compiler but keep the deciding "is it a default actor" as well as accessess by the compiler when it wants to make hop_to_executor still the same as they are today. I think this way we can nicely keep the "devirtualize" logic for existing default distributed actors as well... 
I'm going through implementing / validating this idea though in order to make sure all the layout questions work out if we did this. Maybe hanging the hop_to_executor logic won't be too bad... I'll give this a shot too, though I would like to not have to change all places that touch the unownedExecutor but just keep them the same code-paths basically. As far as assert/precondition/assume APIs go, this should be working out fine still, but I'll also verify.
Thanks @kavon for mentioning this bit!
