Reviews are an important part of the Swift evolution process. All review feedback should be either on this forum thread or, if you would like to keep your feedback private, directly to me as the review manager via the forum messaging feature or email. When contacting the review manager directly, please put "SE-0428" in the subject line.
Try it out
Toolchains with this feature implemented are now available:
The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:
What is your evaluation of the proposal?
Is the problem being addressed significant enough to warrant a change to Swift?
Does this proposal fit well with the feel and direction of Swift?
If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
Given we have distributed actor, it seems like the most natural spelling for a distributed protocol would be distributed protocol rather than @DistributedProtocol protocol.
I imagine there are implementation benefits for the macro approach on the compiler side, in that it lets the functionality be implemented more independently from the compiler itself. Are there other motivations behind the @DistributedProtocol protocol spelling?
If it's primarily an implementation design consideration, I wonder if it could make sense to have the surface spelling be distributed protocol, but maintain the current implementation architecture by having that syntax implicitly add an underlying @_DistributedProtocol macro that implements the functionality.
Remember reading proposal and also was wondering. Could be wrong and @ktoso would correct, but think it's just coincidence that we have protocol in Swift and protocol in networking terminology, otherwise it would have been @DistributedProtocol trait or @DistributedProtocol typeclass or whatever.
Think also faced a problem, could test on small demo with two systems when it will be ready.
It's good the proposal permits something other than Codable as a SerializationRequirement.
I can imagine services where the serialization is optimized for a given (e.g., high-performance) call. Currently I think that would be modeled with a distinct actor, and all the serialization on that actor would have to follow that serialization.
i.e., Conversely, I'm not sure how it would work e.g., to have different serialization for the result handler, even if the proposal were changed to permit that.
Either way, it's unclear what kinds of serializations could be implemented: would the current scheme permit a bootstrap phase, or the opportunity to configure e.g., local-only (in-process?) channels?
I'd welcome the proposal discussing the impact on the potential for alternative serialization and proxying mechanisms, just to be sure no doors are closed unnecessarily.
Specifying that the syntax distributed protocol expands a macro would go against some of the philosophy of the macro system, which is that macro expansions should be explicitly visible in the source (# or @ for freestanding or attached, respectively).
If what you really mean is "should this be a language feature, not a macro?", I think that the macro approach is preferable because of its flexibility and the ability to inspect what it does.
Yeah distributed protocol is quite tempting I have to say, but as Doug explained we're able to get away with "just" macros for part of the feature and there's some benefits from being able to just expand and look at the code it produces. It is useful and "less magical" to newcomers this way -- although the way the remote calls still use some special sauce, but at least the stubs the macro creates are easily understandable this way.
It is a bit of a wrinkle that we have to annotate here, but on the other hand -- I very much expect more annotations and information to be put into the @DistributedProtocol like things for versioning which will be very very important.
So... in practice I think we'll have annotations like this anyway, so might already endorse it -- and then add more parameters to it, like @DistributedProtocol(version: .v10) etc.
Yeah it's since day 1 been a goal to allow various serialization mechanisms, even if today most systems use Codable.
For example, if someone wanted to have some "store the structs raw bytes in the message and just send it" one could do that if the protocol required such capability. I'm quite interested in some high performance IPC like that, but have not really worked on new serialization mechanisms so far.
This proposal doesn't change anything here from the initial distributed actor design tbh, distributed methods can use some specific serialization mechanism -- whatever that might be.
Do note though that the serialization is bound to a specific actor system -- otherwise our typesafety inside the remoteCall and encoder/decoder would go out the window -- so it isn't per-actor, but per-actor system what serialization you choose to use. I'm not sure what you mean with the phases; since the requirement is a static property, it cannot change depending on time or situation where the actor system is used.
In process actors (i.e. distributed "known to be local" actors) simply do not serialize their messages and it is plain method calls. It is interesting to be able to force serialization on local boundaries for testing purposes but we don't do so today.
It is somewhat possible to make even more generic systems by introducing some "common serialization protocol" but we don't have this yet -- I think we could explore this once we had some good bytes types in the standard library, but today we don't have such common protocol.
IMHO the DistributedProtocol is a language feature that requires not only a macro
to function. The existing distributed actor works much the same and also generates boilerplate code under the hood. There’s no harm to align the syntax and, since the DistributedProtocol is something that’s used explicitly with distributed actor and distributed func, we’d better not confuse it with other macros even if they behave alike.
I’d like to quote the following paragraph from the Vision for Macros:
Although the DistributedProtocol macro can provide the syntax for such functionality, this feature is indeed a language feature that requires specific ABI, instead of pure syntactic sugar. Eliding the fact is, in some way, harmful for language users, providing wrong signals that such language feature is achievable purely by the macro system.
This means we should, at least, make it @DistributedProtocol distributed protocol to distinguish from regular protocols. Since we may not want users to craft a distributed protocol without the DistributedProtocol macro, eliminating @DistributedProtocol and making it the default behavior is a boost for both clarity, simplicity and accuracy.
I think this is a brilliant addition that solves some real pain points in my limited experience using distributed actors. I've read the proposal fully, and think it's a great way to address this, since macros can be expanded to understand the 'magic' that's happening.
I've done peer-to-peer networking before, and think that this would really change the complexity of writing those systems.
The only thing I miss is that I haven't been able to dive into code yet, with the try-out toolchain not working correctly.
Thanks for your input and efforts to verify the feature @Joannis_Orlandos!
I was looking into why the macro wasn't found properly, and long story short: There were general issues with compatibility of custom toolchains where they were looking for macros nowadays. @rintaro did a fix (here).
You can download this toolchain for macOS: [Pull Request] Swift Build Toolchain - macOS #1201 [Jenkins] (and subsequent ones which contain Rintaro's fix), but you'll have to manually provide the paths to look for macros as follows in Xcode:
Effectively you need to pass: -Xfrontend -load-plugin-library -Xfrontend /Library/Developer/Toolchains/swift-PR-72629-1201.xctoolchain//usr/lib/swift/host/plugins/libSwiftMacros.dylib
You'll still need an actor system that is able to resolve and call things on protocols, but this generally should not take much work to adopt -- as all the logic in remoteCall is really going to be the same.
Since the entirety of the review period passed without a functional toolchain, we'll also extend the review through April 2 so that any community members who wish to try the feature out have a chance to.
Finally checked implementation, played with different setup—1) same system, but different instances of it; 2) two separate systems. All working fine, definitely see ways where this could be very helpful. At least will play myself creating some client-server communication.
As a consumer myself, think macro makes sense—all the building blocks for distributed actors is already there, and this one is something on top and niche, would prefer it will do some magic for me so I can focus on building distributed systems itself more.
Looking forward for it to be approved and merged!
Here a small example, like how clean and easy it is to use those protocol:
I've been thinking about this more, and I realized that the problem for me is that @DistributedProtocol is the wrong name. Everything about the design feels right, but the name doesn't reflect what it's doing.
The language already has "distributed" protocols. They're protocols that refine DistributedActor and have distributed functions in them. Something like this (from the proposal) is already a "distributed protocol":
and distributed actor types can conform to it. The language handles distributed actor isolation as part of its semantics, and provides detail to the runtime to manage the distributed actor.
Adding @DistributedProtocol to this protocol isn't making it a distributed protocol, because it already was one. The attribute is creating the client stub type that lets you get a remote instance of a DistributedAsyncSequence without having any access to that concrete type. That's what this macro name needs to capture---the extra functionality you get for adding the attribute. Perhaps something like @RemoteStubs or @ClientStubs or @RemoteClient would capture what it's doing.
Yeah that's a fair point, since we're not making distributed protocol do the thing, it might be worth exploring rather what this macro really gives us.
There's a bunch of words which I really want to avoid, for example "remote" never should show up in sources really, because we always either have known-local or "could be remote, we don't know" so RemoteClient is kind of true, that's the primary use case of these, but one can still use these with local values. Client and Stubs words I try to avoid in general because we're trying to have developers not think too much about stubs etc, just focus on the language and actors hm... I very much see the point that the current name isn't great though.
When we look at it from that angle, it is "it enables resolve() on this protocol" which isn't possible without this macro. All such "extends capabilities" things tend to be called "-able" in Swift, so there's actually a word we can use here:
Since this macro enables using resolve() on protocols like this, it is @Resolvable -- i.e. it is a protocol i can resolve(). All concrete distributed actor types are "resovable" in the sense that MyActor.resolve() exits, and this @Resolvable/@Distributed.Resolvable really adds the $MyProtocol.resolve().
I'd be quite happy with that, and I think it'll grow well in the future when we have to add "stable names" or other metadata to the macro, like @Resolvable(stableName: "OldProtocolNameForWireCompatReasons") etc (not designed yet, but I'm sure we'll have to at some point).
