Migrating Cluster system to Swift structured concurrency

Server Distributed Actors
, ,
1

Last and this year I had some progress in developing examples (e.g. swift-chat) or brief prototypes [1] of extension plugins on cluster system, like event sourcing and virtual actors. Also with the help of community we're slowly fixing issues in cluster system itself.

While Cluster system is stable and working, it was written before Swift's structured concurrency was introduced and actually uses its own concurrency runtime. This is kinda blocker for me right now, as to develop things further I want system to be stable and up to date with current state of the art. With @ktoso we agreed that best solution right now would be to rewrite system from scratch, here is a issue ticket explaining the motivation and steps: Migrate Cluster system to Swift structured concurrency Ā· Issue #1215 Ā· apple/swift-distributed-actors Ā· GitHub.

I've finally started to reimplement cluster system slowly, and while doing thisā€”realised that steps on how to implement your own cluster system from scratch could be interesting for people who wants to work with distributed actors. Swift is unique in a way that one of the main parts of distributed actors is bring your own cluster runtime. While Cluster system aims for server side environment, backed by swift-nio, there are many other potential use cases.

While I'm not sure I can fully document it[2], I've decided to create a separate topic here where will try to note what was done and explain each steps as a guideā€”so that one can read and understand those steps and complexity. Guess, in any case, these notes will be beneficial for later documentation.

So, let's try to dive in and explore this together!

A quick note: Iā€™m learning a lot myself while working on this project, so please donā€™t hesitate to ask questions, point out issues, or suggest improvements. Any discussion around this topic really helps motivate me to keep developing and writing more about it.:slightly_smiling_face:

  1. Though fully working! As mentioned later I just need to focus on migrating topic before finishing plugins. ā†©ļøŽ

  2. At least now, as I don't have a lot of time. Maybe after migrating will create a separate project for that. ā†©ļøŽ

19 Likes
2

Weā€™ve been chatting for a while, so Iā€™m very supportive of the effort.

Maintanance of the cluster has fallen behind as you notedā€¦ but it would be very viable to ā€œreviveā€ it through a fresh start like this. Iā€™m supportive of this effort and happy to remain involved merge PRs and offer advice etc.

I should also perhaps address the general status of distributed actors ā€“ theyā€™re (the language feature) doing very well, and Iā€™m supporting a bunch of adopters that I canā€™t share too much information about though. The cluster was very much a proof of what we can build using the language feature, and sadly has not received much adoption, unlike the language feature itself. Either way, the feature is doing well and weā€™re planning improvements to it.

Thank you for picking up the cluster work while Iā€™m busy more on the language side of things,

Thanks and godspeed!

8 Likes
3

Thx!

Yeah, I also emphasized in the post that everything is actually working ā€” especially the language features. Itā€™s just the Cluster system that needs to be updated to move forward.

Thereā€™s some interest in helping with the migration, so Iā€™ll try to organize that (anyone welcome, so just ping me in dm!). In the meantime, Iā€™ll start posting some thoughts here about distributed actors and the steps Iā€™ve already taken.

3 Likes