Hello everyone,
continuing the series of of Distributed improvements I've been working on recently.
This proposal introduces the @remoteCall(...) attribute which can be applied to distributed func/var declarations. It allows expressing semantic expectations that the runtime system should respect when turning a remote method call into the underlying networking/messaging.
The most requested semantic is @remoteCall(oneway) which allows an actor system to leverage transport-native "fire and forget" messaging. Some transports have built-in ways to send a message without awaiting for a response (e.g. sending a websocket message, or an UDP datagram, etc.)
We also propose a slightly more controversial but necessary for some transports semantic called @remoteCall(blocking) which requests the actor system makes attempts to perform a "synchronous rpc" if a transport has such mechanism. This is fully expected to never be employed or respected by networking systems, however is a crucial primitive some forms of IPC.
By annotating these semantics expressively on distributed method declarations, the semantics are clear to the reader, and the compiler may slightly check such functions differently, e.g. a @remoteCall(oneway) method cannot return a value, since we are not waiting for the server's response.
There's a number of nuanced details to these, so please refer to the full proposal text here.
Some input from practice, in case it helps shape the semantics. At one of our companies we build POS / back-office systems with one shared Swift domain module compiled into three kinds of binaries: a server (Swift on Linux, PostgreSQL), a native iOS app (Core Data / SQLite, must work offline) and a manager/web app ( on SwitftWASM. We evaluated distributed actors for this and they didn't quite fit — not because of the transport, but because of where the local/remote decision lives. With distributed it's per actor instance; in our case it's per function, per deployment profile, and sometimes per call:
nextDocumentNumber(series:) is local on a POS (each cash desk owns its own sequence), remote on the manager app, local again on the server.
chargeToAccount(_:) is remote on every POS because accounts are shared across the venue — except in a single-POS venue where a settings flag makes it run locally against the local DB, with the persistence layer syncing deltas later.
Same function, same module, unchanged in all targets; the linked runtime plus configuration decide. We're currently building this in our own framework: a per-function attribute resolved at call time, with runtime overrides on top for operational diagnostics. Three observations from that which I think are relevant to @remoteCall:
The declaration-level check is the valuable part. Having the compiler enforce "a oneway function cannot return a value" mirrors what we found most useful in our macro: diagnostics at the declaration (unreachable rule, parameters must be Codable & Sendable if the call can go remote). Discovering these at the actor-system level, at runtime, was the pain point. +1 on the compiler checking these.
Semantics vs. transport capability. The proposal says blocking is "expected to never be respected by networking systems". From our experience the gap between "what the declaration asks for" and "what this transport can do" is exactly where bugs hide, so I'd argue for the attribute being observable by the actor system (e.g. passed through the invocation / remoteCall API) so a system can fail fast or diagnose when it can't honor it, rather than silently ignoring it. Is that the intended design?
Which delivery guarantee does oneway name? The pitch describes it as transport-native fire-and-forget (websocket message, UDP datagram), i.e. no result and no guarantee of receipt. From our experience there's a middle semantic that business apps need far more often: "don't make me wait for the processing, but do tell me the message was accepted" (a broker ack, an HTTP 202, a persisted outbox entry). On a POS that's the difference between "don't block the cashier" and "maybe lose the sale". Is oneway intended to strictly mean the former, leaving the latter to the actor system's own API, or is there room for the attribute (or a future one) to express "accepted, not completed"? If it stays strictly fire-and-forget, it might be worth the proposal saying so explicitly, since many people reading oneway will assume at least at-least-once delivery.
Thanks for chiming in, I've been iterating on the design here as well.
2] Blocking - I revisited some of the use-cases that were driving this semantic and we ended up concluding there's not a strong enough case for this, so I think we're going to drop introducing this for the time being.
It's up to a specific actor system to define that, however we will encourage doing the "at least it was sent" confirmation. So, since remote calls are always throws you can implement this semantic.
I'm also looking into redesigning the oneway semantic such that:
we should allow sending from synchronous contexts: try something.ack()
and we should handle the same requirement for normal actor methods; it seems the vast majority of Task{ await one.thing() } are actually void methods and could benefit from a nicer syntax as well as runtime optimizations
I read the proposal and I think the oneway semantics make total sense. This is a pattern that today often requires spawning unstructured tasks which can be entirely avoided with the proposed changes. A minor comment when reading the implementation suggestion for oneway calls is that I was wondering if they should still check for local back pressure i.e. should they check if the existing local queue has a certain depth and wait until more room is available? Once it made it into the local queue they can then resume without waiting for a response. Curious how the systems mentioned in the prior art handle this.
I personally think that there is general usefulness in a @blocking annotation that can be added to methods that will lead to dynamic checking if the context is allowed to be blocked. I could see something like this generalize to distributed actors as well. I think splitting it out into a separate proposal makes sense though.
You could write it this way, though your only choice is to throw is that happens. Which may not really be very actionable, but it is possible.
They don't Classic pitfall of all those mentioned systems is flooding queues with oneway messages; Even in the good ol' Erlang you could run into this, and why folks rely on gen_server and request reply most of the time (i.e. using call rather than cast).
In Akka, where tell is the default, it was the reason we went on to create reactive-streams really -- flow control over uni-directional signals isn't accidental in that protocol, it's partially because it is implementable using oneway messaging.
Thanks @ktoso, read through the proposal and it looks good -- agree with oneway naming and very happy to see this overall.
Can concur with that prior art systems did not handle this great (did a fair bit of (Portable) Distributed Objects work in 1994-1996 with still some visible scars -- as a minor trivial nit is the 1988 attribution of the NeXT DO implementation seems a bit early, a quick google attributes it to NS 3.0 (1992)).
Thanks for pushing this forward, this will be useful for our distributed system implementation.
Classic pitfall of all those mentioned systems is flooding queues with oneway messages; Even in the good ol' Erlang you could run into this, and why folks rely on gen_server and request reply most of the time (i.e. using