Swift Concurrency Roadmap

I think it would be very natural to express that as an async API. Whether it's a good fit for an actor is an interesting question. To me, something can be logically be an "actor" — that is, it fills a similar role as a language-supported actor — without necessarily being a good fit for the language-supported actor features. The difference between a well-encapsulated actor class and an ordinary class with primarily async methods is very little, basically just the semantics of extension methods (i.e. whether they logically execute on the actor's executor or not).

1 Like

Wow, this looks great, I very much look forward to using it all in the future :clap::clap::clap:

The roadmap states that:

But (sometimes) we have multiple releases per year. Is there at least an idea of how long the first phase will take to be done? Months, years (decades :stuck_out_tongue:)?

1 Like

will it need to implement ownership and borrowing?

2 Likes

You mean the entire async/await features, actors, and all the shebangs? Apparently not, judging from all the pitches.

The main problem I see with this is that it seems to encourage anti-patterns that have the potential to do much more harm than good. This document seems already to be recommending anti-patterns like using queues instead of locks to protect shared state and using private queues to protect access to properties (which is what actors are). The maintainer of the libdispatch at Apple has repeatedly explained that developers should not do this. I have compiled together a page that sums up all this, including many references at the bottom of the document:

Multithreading is a complex thing, developers need to think very hard about it, about threads and how to use them wisely in their programs. The libdispatch maintainer explained that going wide by default is a mistake. Developers need to start serially and only apply concurrency when needed and with great care. My worry is that developers will misuse this just as they are misusing the libdispatch, they will make actors out of everything, have way too many queues, dispatch way too small execution blocks and get terrible performance out of it.

12 Likes

Do you have some examples in mind? It's hard to tell right now if your recommendation is incongruent with the design as things like prefer long queue aren't exactly applicable.

Theres also the question of control...

The inner implementation will implement a thread pool? if yes, with how many threads it will start?
If no, so it will dispatch a new thread for each time?

What about thread affinity and resources that need to assure they run in the same thread it runs before?

The actors pattern is cool and all but in my experience at least they are no the best pattern for every multithreading scenario. So forcing this directly in the language, of course it will make it easy for most people, but as i 've said before if they were built with the current language constructions it would be even better..

Edit: to make it clear.. ? for instance point to optional<> what about an async<> than a sugar could be used of course, but async<> could than be optmized for different cases.. IDK, just an idea..

1 Like

Right. I think the libdispatch has done a fair amount of harm to the platform because it made it feel like multithreading was easy, developers started using it without understanding what was going on under the hood and terrible performance ensued. This is the same thing but on steroids.

Having personally gone the way of the libdispatch since it was introduced in Snow Leopard, having written a lot of heavily async code, and eventually having understood the hard way that it could not work well without carefully thinking about threads, I am quite afraid of the long-running consequences of something like this.

My first advice to developers about multithreading today is: don't do it, unless you know you need to introduce concurrency and if you do, apply it with care. Most programs don't need a great deal of concurrency and most are not good candidate for it (because they lack long enough non-contended tasks to execute concurrently). This proposal will make everyone use concurrency and multitrhreading with no understanding of the implications.

8 Likes

I guess the advice is simple: Use as few actors as possible.

Also actors are great because they force a share nothing sort of attitude. Its great because its simple and clean without the need of complicated locks.

But, they are bad for some cases because they may copy heavily too much between threads. So they will behave poorly if you need to pass to much stuff between threads.

In those cases threads being able to share with fine-grained locks would be a better approach..

4 Likes

This is very exciting. Swift solving these problems will have a profound impact on me, my career and even the industry going forward. Thank you to all for the effort you’ve put into it.

What’s the story for back-deployment to earlier OS versions? Will we have to wait a handful of years after the dust has settled to use it (e.g.: because it’s dependent on some platform specific features)?

4 Likes

Posting on behalf of @saeta and @compnerd (and on my own behalf of course).

Some of the systems programming folks we've been talking with would dispute whether that is good. They point out that, aside from having lower absolute performance costs than queues, in a lock-based system there are proven tools for deadlock detection and debugging. [^1]

While we think an async/await system is probably essential for Swift, we are also concerned that this proposal may be ignoring the foundations that proven concurrent programming is built upon. By starting from the middle (e.g. without updating the memory and ownership models), choices incompatible with systems programming may be baked into the language.

Most of all—and this is difficult because we do appreciate the obvious effort that has gone into these proposals on behalf of the whole Swift community—a massive document set is being exposed for the first time, with very little prior opportunity to discuss the approach here in the forums. That makes it something of a fait accompli, which nobody can reasonably challenge at a fundamental level without implying a huge cost to the proposers.

[1] Incidentally we don’t understand the claim being made that deadlocks are eliminated, since as far as we know, two actors can easily wait on one another’s results.

26 Likes

Also, unfair locks perform much, much better under contention than fair locks or queues.

I guess that's a livelock and not a deadlock, as the APIs are async?

2 Likes

Worse, they don't prevent interleaving the code (AFAIU). So another "partial task" can be executed on actor1 while it's waiting for the result from actor2. So less deadlock, but could lead to unexpected result to untrained eyes (or even trained ones). We can go to Async Function thread if we want to discuss this. I think it's mentioned there.

My understanding is that livelocks involve unfairness where a lock is repeatedly denied rather than a deadlock where there is lack of progress. Is there a terminology difference here?

My experience as a programmer that was already here before the libdispatch was introduced is that developers will use way too many actors, just as they use way too many queues. Actors also seem to make it super easy to dispatch very small tasks, such as protecting shared state and properties. When this is out there, people will use it and it will be impossible to explain to them that they should really not use it that much.

3 Likes

As I see it, Actors are one of the main building blocks of safe concurrent code. Plus there's the distributed actors... So I think it's absolutely awesome to get this in as a first class citizen. Even if it currently can lead to less efficient use. Also, wouldn't it be possible for a future runtime to decide, possibly at runtime, that some actors belong together, and hence merge their dispatch queues (if they use dispatch queues)?

And async/await is really great for simplifying concurrent code in a big way.

I think these proposals are amazing, and I really look forward to using them :+1:

2 Likes

We're working closely with the libdispatch team on making sure the level of real concurrency is appropriate for the underlying system. It's always possible we'll still get it wrong of course, but don't assume that the current libdispatch and pthread primitives are all we have available :slight_smile:

Having put a lot of effort into removing multithreading from systems to make them go faster over the last few years, this is something I'm trying to keep a close eye on.

45 Likes

Yeah, one of the underappreciated aspects of having lightweight tasks is being able to write concurrent code without inflicting multithreading on yourself when one thread will do.

14 Likes