I don't think the proposed design is as centered around actors as you think. Fundamentally, the vision of concurrency in our proposal is that the program is structured into many lightweight concurrent tasks which are scheduled opaquely by the system onto true threads. Actors provide a mechanism for managing shared mutable state in that concurrent environment, and they have nice compositional advantages over some of the alternatives, and Apple does have a need to model critical systems in our environment that are managed by actors; but of course actors aren't the only such mechanism.
Our goal for what we call "actor isolation" is to capture and enforce the semantic properties necessary to ensure general data isolation. Actors naturally provide a lot of static structure that can make satisfying those properties easier to prove. Other, more dynamic mechanisms often take more care because the rules that make them safe are often difficult to express without, say, a system of static unique ownership. But we still intend to offer such a system in time.
Really? I understand âinterleavingâ to mean that a thread may pause execution of one function and resume execution of code in another function before either function returns. IIUC that's what happens in _read and _modify at the point of yield, and also what happens at await. What am I missing?
Maybe it depends on whether you think of the executor as primarily operating on tasks or partial tasks. Don't executors that serialize their partial tasks interleave their tasks, instead of executing them in parallel?
Hi Brennan, thanks for chiming in, it's been a while
I can't help but feel there is some mismatch between what is being read into the proposals based on various people's backgrounds vs. what the proposals really are saying.
A lot of this is perhaps just that there's a lot of text and concepts to process, and we don't yet have a working implementation (these are early pitches bear in mind) so all the finer details one tends to fill in using one's past experiences, which may not actually be what the proposals are saying.
Also, I've worked with actors for enough years to know that just the mention of the word sometimes "triggers" people You raised some concerns I think deserve more context to fully grasp what we're doing here. But really John says it best:
Exactly as John said above â actors are but one of the pieces to the puzzle.
The Swift Concurrency story is not "everything is an actor". That would be terrible advice.
These proposals purposefully introduce many ways to deal with concurrency, including structured concurrency, and escape hatches to build otherwise synchronized or safe datatypes, all working together in the same process.
Actors are fantastic for some types of work, and they are a great, safe, default to reach for when one just wants to get stuff done (e.g. some relatively boring "controller" types in iOS or server apps). They are not a silver bullet and nowhere in these proposals do we say they are.
We have converted some existing apps to use the current main-branch compiler to get a feeling if we're solving real issues and making the code much nicer or not. We have found the simple conversion of the omnipresent "I need to manually remember to hop to the right queue every time I call something"-style to actors has not only vastly simplified those classes, but also immediately uncovered actual concurrency bugs in them (!), even though we don't have the "year 2" safety checks in place yet.
You mention Akka and prior experience with it as one of the points of concern, so allow me to address this directly, because there is a lot of context here that readers might miss out on:
I think it's important to be precise here, as when you talk about "akka actors" you specifically refer to the untyped (by now: legacy) APIs of Akka (because that must have been 8~4 years ago). I suspect not much Akka Streams, and definitely no Typed usage (though typed is also very verbose, so it'd be a mixed bag) by that timeline either.
It is also worth pointing out that during those years there was somewhat of a "craze" to use actors for everything, even if they wouldn't necessarily fit. Their untyped nature, lack of easy request/reply and other issues of course make it hard for "just" concurrent programming. Akka never was focused on the concurrency to be honest, it always was about the distributed parts. For better or worse, intense marketing and hype pushed it into many places where it might not have been a great fit.
I think this is very important to call out because: Sure, I agree (!), and we can do much better than that.
And even if you were to ask the core akka team about using actors for "just concurrency" you very often would get us to tell you "maybe that's overkill." Please note that the last 5~6 years of new development of Akka all have been typed APIs: it kicked off Reactive Streams (which became part of the JDK, and inspired Combine's take on streams), and introduced Typed Actors. Very often over those years we would tell people "just use a stream" and "don't use an actor" or more specifically "actors are a nice lower-level building block".
In a way we're approaching actors in swift from the opposite side: make actors feel great and part of the system/language, rather than build actors and force everything to work with them (which "library only" implementations have to do).
This is another point I fully understand and you can rest assured that we are doing better than what Akka can offer here.
I would not let experience with one specific implementation taint the entire idea of actors (esp. experience with an untyped system)
Often times in Akka systems one needed to use child actors to isolate some workflow, because lack of async-await, so one has to put the "linear execution" into a child.
With Swift Actors we're well set up to do much better than thatâby suspending/awaiting actors we can keep our linear logic, avoiding having to spawn children just for the sake of "linearizing work". Yes, we need to talk about reentrancy (more below) though to do this.
Again, absolutely agree and we are doing much better with Swift's actors!
For others, allow me to provide context what the ceremony mentioned here is:
// imagine "akka swift"
struct Greet { let name: String; let replyTo: ActorRef<Greeting> }
struct Greeting { let greeting: String }
someone.tell(Greet(name: "Bob", replyTo: context.myself))
// and myself has to be ActorRef<Greeting>, which actually it often isn't
// so additional complexity creeps in with adapters and "correlation" of
// requests and replies... lots of complexity.
// or (boilerplate because the system is typed!)
let future = greeter.ask(timeout: .seconds(3)) { replyTo in
Greeting(name: "Bob", replyTo: replyTo)
}
// but futures are _not_ safe to use inside actors, so one has to
// be very careful using them, use special pipeTo(self) patterns etc.
// The futures are unsafe to close over things etc. -> lots of complexity.
So that's a lot of boilerplate (especially since it is typed), and it can't be easily "command click" navigated or understood the same way as just (async) function calls.
In Swift, these interactions are as natural as any normal non-actor code:
let greeting = await greeter.greet(name: "Bob")
You don't even care if it is an actor or not here to be honest, that's up to greeter to decide how it wants to implement things. Yes, we need to have the reentrancy discussion, and we should have it soon (I've been working on a writeup for discussion), please be a little bit more patient for that bit â it's coming.
Long story short: Swift Actors are Swift Actors. They are not "let's copy paste some other specific runtime to Swift," and it's not doing them justice by simplifying them to that.
Swift actors are vastly more user friendly and safe than what a pure library approach to implementing an runtime could ever achieve.
You absolutely can define standalone async functions; It just means they're not bound to any specific executor. If you want to bind them to a specific executor you'd throw them on an actor, be it global or instantiated.
I think this and other questions will clear up as we keep updating the proposals and implementations. Thanks for raising all the questions and concerns.
topic: actor reentrancy is hard
Yes, absolutely! I said this in many threads already.
I, for one, agree that must revisit this, and I've been signalling "we will revisit this" throughout other threads, and the reality is that simply currently we have not revisited this yet (there's so many things in flight implementation wise right now...), and the current proposal just is what it is. I'm working on a discussion piece so we can figure out reentrance â once we have discussed more I hope we can add a proper section and plan to the proposals.
Speaking for myself, only reentrant actors definitely is very risky, and throws complexity at end-users rather than the runtime. As you say, all other notable actor runtimes are non-reentrant by default, and for good reasons, but allow opting out when necessary.
Added to our todo-list to expand in discussion/rationale sections.
Short answer: we have tons of classes which "protect their state with a queue" so these naturally become actors. If necessary they can synchronize using other methods, like locks if they decide that's better. But actors are a safe default and easy migration path for a lot of delegates.
I remain somewhat confused about why the exceeding worry about this.
We absolutely see and support such types, even in actors sometimes "reality strikes" and one wants to send such type around. We're absolutely aware of this and the model supports it. If this is about "why are we not introducing Swift APIs for locks etc" then I suppose it's just an ordering thing â we want to offer the nice/safe/good-default for people who are struggling today. Experts can always continue what they are doing today, using locks, swift-atomics or anything else.
I really feel we're aligned in our goals and how to get there, but need to find shared vocabulary to express this As we're getting more implementation pieces in place, and update proposals with more details this will become easier.
The existence of actors does not prevent or subsume other use-cases which don't fit them.
The core abstraction of Swift Concurrency are Tasks, and actor-isolation is, as John explains very well a simple to understand, easy to reason about model to think about "isolated state":
Thanks again for the feedback, Brennan et al!
I hope this helps addressing some of the concerns, and we'll be updating proposals as we have more details fleshed out.
Sorry, that bullet was badly phrased. We absolutely understand that you can define standalone async functions without referring to actors.
We were trying to say that the async functions proposal itselfâwhich defines what an async function isâuses actors to describe their semantics. So based on our reading, having async functions in the language appears to depend on having actors in the language, which seems like a layer inversion. By contrast, it seems perfectly natural to us that the description of actors depends on the description of async functions. We feel pretty good about buying into async functions, and much less confident about actors, so we'd be more comfortable with a design that doesn't couple them bidirectionally. That apparent coupling is one of the things that makes actors seem central to the design.
The other thing that makes actors seem central to the design is that the proposals give lots of attention to ensuring thread-safety of accesses to properties of reference-semantic actors, but none at all to closing the existing thread-safety holes (globals and mutable captures) for value types, which in many ways are the low-hanging fruit. I acknowledge that @Douglas_Gregor has said there's been some implementation work to address mutable value captures, but it's not described in the proposals, we only heard about it today, and I don't really understand what I've heard yet.
We may be completely misunderstanding the design, of course, but all we have to go on is what's written in the proposals. What we outlined in the part of the post you're quoting seems like it could be designed and evolved more incrementally, whereas the proposals as written seem to describe a system that can't do anything useful until the language takes on all the complexity associated with actors⌠which is a lot.
Please do NOT follow the path of javascript, where the whole ecosystem is poisoned by await keywords. Code should run synchronously by default. If an async routine shall branch off, this can be denoted with a "do" or "go" keyword similar to the approach in the Go language.
Javascript is not poisoned by await, NodeJS libraries are.
This is the case unless the code is blocking, and blocking code is a legacy from the way we used to do when we had no better way.
Blocking a thread is almost never a good idea, even more if this is the main thread. And as this is almost never the right way to go, it should not be the default.
Folks, I just posted an update on the evolution of the concurrency proposals over on a separate thread. Thanks for all of the wonderful discussion---this is an exciting area for Swift.
First off - Iâm really excited and thankful to the Swift community for considering the Actor model for concurrency. My first exposure to Actors was in a Distributed Smalltalk in the early 1990âs and it was really nice to work in. More recently I worked in Scala/Akka and have toyed with Erlang/Elixir.
To me - the concepts of Supervisors, Supervision Trees, ActorRefâs and dealing with actor lifecycle policies (restart/reset/etc) seem like foundational things in any actor system (noting âActor Systemâ is also a formal concept in some implementations).
Are these dealt with in this roadmap under different names?
I think even putting some basic types in place early on would ease the path later on when we will have to grapple with the inevitable distributed systems problems (actor replication, quorums, actor failure) that will come when devs [at Apple and elsewhere] start wanting to distribute actor systems on Kubernetes
If we donât I see many devs (re)inventing their own âactor factoryâ and âactor managerâ types.
We're a bit far away from focusing on sugar, so let's maybe revisit this later on once things are up and running. But what you're proposing to be honest perhaps should even just work�
On that topic though, I'd personally (no idea if others oppose this view or not) rather introduce an
protocol Awaitable {
associatedtype Value
func get() async -> Value // get() to be same as the Task.Handle
}
// and a throwing version of that
which would make awaiting a bit nicer on such types:
await handle.get() // Task.Handle
await handle // using the Awaitable sugar
Anyway, not sure if the rest of the team is opposed to this or we just didn't get to it yet -- the focus is getting things working now, sugar can follow later
Iâm really coming around to this, not by reference to Zio and Trio but for Swift-specific reasons: modelling cancellation as throwable errors is not a good fit.
Although itâs sometimes desirable to âlimp through and finishâ as John said, cancellation is more like a universal error than a recoverable error (as defined in the Error Handling Rationale) in that it can arise at any time and isnât directly tied to what the cancelled code is doing.
In other words, cancellability can (and should) be seen as a different effect than throwing. Explicitly marking functions as async cancellable and call them as await possiblycancel foo() wouldnât be very popular, though.
If support for cancellation is expected to be common, and it probably should be, it would be more reasonable to have await imply cancellability, use defer (or RAII with move-only types) for cleanup, and provide an escape hatch like uncancellable functions or a withoutCancelling {} context.
We're absolutely not going to have an asynchronous-impact model for cancellation, where cancelling a task immediately triggers that task to unwind; that's been a failure in language after language. Given that, I don't think it's actually a universal-error scenario where the programmer has really no chance to react appropriately. Cancellation will asynchronously notify the task that it's been cancelled, and if it's in the middle of waiting for some operation to complete, it can choose to react to that by, say, cutting the operation short and reporting failure. Cancellation is then just an extra (and opt-in) source of failure for the operation, which is perfectly reasonable to model by throwing an error.
Now, there are a few operations where cancellation is really the only reason it can fail, like waiting on a timer. We can decide how best to report to cancellation in these cases; maybe we should have throwing and non-throwing variants of them, where the non-throwing variant simply ignores cancellation or ends early â whatever is most reasonable for the API. But we don't want a world where arbitrary code can suddenly terminate for an invisible-to-the-reader, impossible-to-reproduce reason.
Would this approach scale to other coroutines, like _modify? IIUC the only difference between await and something like yield in a generator is that we can pass objects around at the suspension point.
Iâd like us to have more coroutine features in the future (I find generators about 1000x easier to write than complex, multi-stage iterators, for instance), and it would be nice if there was a coherent programming model between them. So if await requires explicit cancellation checks, yield should, too.
My point was that you canât express that they only fail via cancellation without typed throws. If we can express it we probably donât need non-throwing variants that ignore cancellation. That would trivially compose on top of the throws CancellationError variant.
The only difference is that we need to unwind a coroutine from the yield; it cannot be meaningfully âhandledâ by the modify/read accessor, nor can the accessor access or replace the reason for unwinding.
In a side conversation I started to wonder about the nature of these bugs and how they were found. So I thought I'd just ask. Were they reentrancy bugs or something else? And what helped you find them?
The type system telling you the code was wrong
The boilerplate reduction making it clear something was wrong when it had been obscured
You raised some concerns I think deserve more context to fully grasp what we're doing here. But really John says it best: