In this topic I asked a question about testing for actor re-entrance. Unfortunately, testing for that is very difficult, perhaps not really possible.
The topic also moved more towards design for the use case we were discussing (the server component of a multiplayer real time PvE game).
Since that discussion I've tried out various designs for the server component. And I thought to share my experiences. While doing so I learned more about concurrency beyond the basics and that might benefit others as well.
The use case
The game is a real time multiplayer game. It is fully server authorited, meaning that all of the game state is held on the server. Every action gets performed on the server. Clients only see the results.
The game has a persistent world, meaning there are no 'matches' or 'matchmaking'. You just create a player and you are set to go.
The clients only show game state. And are able to send commands to the server where they are executed.
The server also has an update loop to update time based state, as well as control the environment (in particular the mobs)
The server is a basic Vapor server. All interactions with clients are simple HTTP requests (yes, not even web sockets). I try to use the async/await versions of the Vapor services as much as possible.
The problem
The server needs to respond to multiple clients sending in commands, requesting state and the servers update loop while keeping a consistent simulation of the game world (which is now only entities and maps). As actions from entities can affect other entities, some form of transaction consistency is nice. I.e. if you give another entity 100bucks, the transaction should complete in full or not at all. But it should not be done 'half'.
For the high level of design for the server, I tried three approaches:
World is an actor, entities are structs. All commands are placed in a queue and processed in the actor one at a time. Most of what happens with entities is synchronous. And because they are Sendable, they are easy to pass around. However, managing all the copies of entities (as they are values) is risky. It's very easy to change an entity somewhere, only for the change to get lost because I forgot to persist it back into the world.
Worlds is a struct, entities are actors. This was suggested in the topic. Making the world unchangeable makes a couple of things a lot easier. And having the entities as reference types makes a lot more sense. However, because of all the actors, almost everything ended up being async. And suspension points were everywhere. Tests still passed, but that doesn't give me much confidence as the tests are all nice and isolated.
World is a struct, entities are classes. A simulation as an actor to bring them all together. Here, the world and entities are NOT sendable. (this was inspired by a post I read somewhere that non-sendable types still matter) The world completely lives in synchronous space. The simulation is a very small layer that brings all the requests coming in, into synchronous actions. Entity state is returned as structs towards the world outside of the simulation as a sort of 'snapshots' of the entities.
(maps are immutable, always a struct, and never an issue)
After seeing all three options, I decided to go with the third. The code it delivers is the most straightforward. And from a conceptual point, it makes sense to me: I have a simulation that really is its own isolated thing. It is really simple to add features because the simulation is completely synchronous in iets working, and that fits well with my test driven way of working.
There are two downsides though:
keeping an EntityInfo struct that mirrors Entity (class) properties in sync is a chore. And easy to mess up. Perhaps a Macro could help here?;
one big simulation will be unable to scale beyond a certain point. If the entities themselves are more independently simulated (option 2 above), scaling would probably be a lot easier. But that means keeping a simulation that is distributed among all the various entities consistent and this is something that is way outside of my current ability.
Anyway, hope this helps somebody.
My key take away is: once you start working with Swift Concurrency, you need to think in terms of isolation domains. Not just "I'll change every class into an actor", but really think about what parts of the system require concurrency. And which ones don't.
I agree that it seems a much nicer option out of all. In terms of scalability, you can take a look into distributed actors, my current understanding is that exactly the case they are designed for (might be wrong, haven’t looked at them yet).
The thing is, it requires making entities sendable.
And that is tricky for classes with mutable state. (I don’t feel confident just marking them @unchecked Sendable).
So then they need to become actors. And then we are at option 2 again.
In a way there is still one source of truth: the classes are the truth. The struct version is only created from the class. And only contains immutable properties. It’s as if you took a “photo” of the class state at a point in time.
It's a good topic! Would like to see some updates.
Multiplayer game means it's already concurrent by default, so guess choosing third option—everything should be consistent and code will end up having locks or mutex everywhere?
There is forth option, where everything is an actor having it's own states, but in order to understand benefits different questions should be asked:
How is player represented? What is World actually? Can player connect simultaneously to several games? What happens if player leaves the game? What happens if player disconnects for some time? What if server crashes? Should the world state be stored somewhere and restored? Will everything run on single server or several? How this will scale?
and so on...
Covering those of course gives complexity and for small scale can understand why going with classes and structs could be better, especially if you used to.
btw you can do streaming with http (even bidirectional), there are some examples using openapi (check for event-streams- examples)
It's a good topic! Would like to see some updates.
Glad you enjoy it! Though, I don't intend to let this become a devlog for the game. I'll post interesting Swift questions/experiences.
There is forth option, where everything is an actor having it's own states, but in order to understand benefits different questions should be asked:
How is player represented?
Currently the player is just an entity like all others. Only controlled by a specific player, instead of by AI (which all the mobs are). This makes the mobs in a way over engineered, because they don't have need for complex stats, leveling, equipment, etc. But for now it's OK.
What is World actually? Can player connect simultaneously to several games? What happens if player leaves the game? What happens if player disconnects for some time? What if server crashes? Should the world state be stored somewhere and restored? Will everything run on single server or several? How this will scale?
and so on...
The world at this moment is just: a collection of entities and a collection of (immutable) maps. That's it, meaning that anything that has any behavior automatically becomes an entity.
I expect a form of drop in / drop out gameplay (like in an MMORPG, but at a way smaller scale) and persistence. A DB makes sense, although you can get a lot of mileage even out of a flat file
For now the classes work. And I expect it will be easier to move from classes to actors, than the other way around.
btw you can do streaming with http (even bidirectional), there are some examples using openapi (check for event-streams- examples)
Thinking first about isolation, and how you want that interrupted (if you do) with async calls is a terrific way to start designing something like this. So much of the conversation in evolution focused on Sendability, and it was a prominent enough term, that I think it’s warped the thinking a bit.
Classes are amazing when kept within an isolation domain, so referencing classes from with a struct, or even an actor, makes a lot of sense to me. I found that when I stepped back and looked at the design and API interaction from an isolation perspective, it made the choices about structure and API from a concurrency perspective MUCH easier.
The mistake I made earlier (and what I’m still prone to do) is to make too many things an actor, tossing everything into their own little isolation zone, meaning a lot of potential hops. My code wasn’t that performance sensitive, so it didn’t really bite me hard - but it was an easy mistake to make coming into designing for concurrency.
