Ok so this is super interesting. Because while I 100% agree that FP patterns can make concurrency adoption easier, I do not see how it could have made the language or its concepts any simpler. But, I'm also not particularly good or well-versed in FP, so I may be missing something important and/or subtle.
Swift may not be a pure FP language like Haskell, but its almost as functional as its granddaddy OCaml. The only thing that's really missing is "everything is an expression". Which I think was mostly due to to Chris Lattner not wanting to spook the C programmers too much back in 2010.
Even mutating is functional, due to the way that inout is semantically defined as copy-in-copy-out. We do have a couple of things that poke holes in that (classes & unsafe pointers), but again, not pure.
Swift's Concurrency isn't making Swift any more functional than it already was, or even moving people to use more functional constructs, it's just covering up the holes in our already functional mutation system while in concurrent code.
I'm not completely sure I saw that same exact philosophy in practice. I was at FB ten years ago and two big languages for front-end engineering were JS for WWW and ObjC++ for iPhone. Both of those languages were for the most part "object oriented" without the same first class values types that we have in Swift. Product engineers built on mutable object references because that is the tool the language gave you.
Over time engineers at FB built tools to "mimic" immutability on objects. This became the ImmutableJS library for JS and the Remodel library for ObjC++. At the time, product engineers for iPhone did have to start coding with concurrency in mind… but JS at the time had no real first class concurrency model. Better support for concurrency in ImmutableJS was a side effect… the primary goal was the mental model of immutable data and the ability to reason locally.
IMO Apple does take time to explain that local reasoning is the primary reason they recommend value types.
2 Likes
This reminds me… some time ago I heard a wonderful argument for immutability from a professor at Portland State University that really stuck with me. I'll replicate it here since I think people will find it interesting:
- You can approximate the complexity of a program as being the product of the number of pieces of state and the number of places that state is mutated (N * M)
- Both pieces of state and mutation sites scale roughly linearly in code size
- Therefore the complexity of shared-mutable-state programs is quadratic in the size of the program
And indeed aside from forming a truly compelling argument for immutability/value semantics (quadratic -> linear complexity scaling), an explanation for the value of encapsulation in OO programs also naturally falls out: encapsulation changes (N*M) to (N1 * M1) + (N2 * M2), where N1/M1/N2/M2 are much smaller than the original N/M.
[edit] Also this explains why students rarely, in my experience, see the value in either. Homework-sized programs just aren't large enough for complexity scaling to be a major concern.
34 Likes
Speaking of the concurrency manifesto, I keep going back and looking at its stated aims and thinking about whether they have actually achieved them: —
- Design: Swift should provide (just) enough language and library support for programmers to know what to reach for when a concurrent abstractions are needed. There should be a structured "right" way to achieve most tasks.
- Maintenance: The use of those abstractions should make Swift code easier to reason about. For example, it is often difficult to know what data is protected by which GCD queue and what the invariants are for a heap based data structure.
- Safety: Swift's current model provides no help for race conditions, deadlock and other concurrency problems. Completion handlers can get called on a surprising queue. These issues should be improved, and we would like to get to a "safe by default" programming model.
- Scalability: Particularly in server applications, it is desirable to have hundreds of thousands of tasks that are active at a time (e.g. one for every active client of the server).
- Performance: As a stretch goal, it would be great to improve performance, e.g. by reducing the number of synchronization operations performed, and perhaps even reducing the need for atomic accesses on many ARC operations. The compiler should be aided by knowing how and where data can cross task boundaries.
- Excellence: More abstractly, we should look to the concurrency models provided by other languages and frameworks, and draw together the best ideas from wherever we can get them, aiming to be better overall than any competitor.
That said, it is absolutely essential that any new model coexists with existing concurrency constructs and existing APIs. We cannot build a conceptually beautiful new world without also building a pathway to get existing apps into it.
— from Swift Concurrency Manifesto · GitHub
Let's take those one at a time:
Design: Swift should provide (just) enough language and library support for programmers to know what to reach for when a concurrent abstractions are needed. There should be a structured "right" way to achieve most tasks.
Is that true? Say I want to avoid synchronization problems in my code: I still have a choice of a global actor, a local actor, one of the new semaphore/mutex tools they've released, the old locks eg OSAllocatedUnfairLock, and so on. And I don't think it is clear what is best to use, especially within an existing codebase where adopting actors can be very hard — because you can't always easily introduced concurrent bits of the code into existing deep code paths. And it changes what needs to be Sendable and what doesn't too. The tradeoffs are complicated.
Maintenance: The use of those abstractions should make Swift code easier to reason about. For example, it is often difficult to know what data is protected by which GCD queue and what the invariants are for a heap based data structure.
I think the existence of this thread in the first place puts doubt on this one. Actors, Sendable, sending, crossing actor boundaries, the difference between isolation context, isolation domain, isolation region, and so on. These are not simple things to reason about — especially as sometimes they only exist as compile time concepts unlike GCD queues.
Plus invariants are still not simply described anyway. Compare with say, MPE where you have a directive to mark a section of code directly as a critical section. In Swift that kind of thing is covered by a (probably global?) actor where the actor function itself is synchronous. That is not clarity and could easily be broken when a future change requires that code to become async.
Safety: Swift's current model provides no help for race conditions, deadlock and other concurrency problems. Completion handlers can get called on a surprising queue. These issues should be improved, and we would like to get to a "safe by default" programming model.
Data races may be eliminated — if and only if everything is compiled with swift 6 and trusted of course. But general race conditions remain and you get no help with those.
Thread deadlocks may technically not exist but logical flows can still deadlock easily — one clear example is using withContinuation and then failing to call the continuation. Swift can runtime detect and warn about that sometimes but not always.
And reentrancy is alive and well — instead of completion handlers getting called on a surprising queue, now any code after an await can be called on a surprising thread. If you adopt the concurrency model completely and properly, then sure maybe it doesn't matter what thread you are on. But that is not always possible and bugs from actor reentrancy do exist.
"Safety" as a concept in all these discussions is interesting. It is a certain, technical meaning of safety that is not necessarily actual safety. Memory safety is definitely a good goal to have, but it is not synonymous at all with the kind of help they are implying here. Take MainActor.assumeIsolated as an example — this is safe by the technical definition because no undefined bahaviour can happen because we crash the application instead. Personally, I don't think of my application as being able to crash as "safe". Another example is the opt-in task cancellation. You now can't guarantee that some task won't live forever and deadlock your app, because it has to allow itself to be cancelled cooperatively. That is literally less help with deadlocks than we had before, not more. (see withThrowingTaskGroup doesn't (re)throw the error - #11 by rurza)
Scalability: Particularly in server applications, it is desirable to have hundreds of thousands of tasks that are active at a time (e.g. one for every active client of the server).
Performance: As a stretch goal, it would be great to improve performance, e.g. by reducing the number of synchronization operations performed, and perhaps even reducing the need for atomic accesses on many ARC operations. The compiler should be aided by knowing how and where data can cross task boundaries.
These goals are the ones that have the best claim to actually having been achieved. How many Swift developers were actually suffering from these problems though?
Excellence: More abstractly, we should look to the concurrency models provided by other languages and frameworks, and draw together the best ideas from wherever we can get them, aiming to be better overall than any competitor.
We could argue about this but personally I don't think they are anywhere near being able to claim they are better than any other competitor.
That said, it is absolutely essential that any new model coexists with existing concurrency constructs and existing APIs. We cannot build a conceptually beautiful new world without also building a pathway to get existing apps into it.
We only need to look at this thread again and see they failed at this. Eg: —
I deeply resent the hours-upon-hours wasted working around concurrency challenges with Apple’s own preconcurrency frameworks. (I’d love to see research into the total loss of productivity across the industry that all of this has introduced! It makes me think wistfully about the Swift 2 to Swift 3 naming convention transition, and that was a PITA.)
I feel like the victim of a “move fast and break things” philosophy, which is all fine and dandy if you’re the one “moving fast”, but is far less fun for those of us living with these “broken things”.
The old way and the new way are so estranged they had to introduce new mutexes and locks etc because the new way wasn't good enough in all cases.
So in conclusion the goals of the original manifesto can only really score 2/7 at this point. And given the route they have taken, it seemed they valued the performance and scalability over all the other points. If there wasn't a limited thread pool, a lot of these other issues would be easier. They constantly reference Erlang in the manifesto as support for the actor model, but Erlang actors do not really resemble Swift actors at all, and the thread pool is one of the reasons that is so. Unfortunately it is far too late to backtrack on that now and I don't know how one would start going about fixing it. There has been an emphasis on documentation in this thread, but I would argue that few to none of the problems I listed above would be solved by documentation. Even the concept of a "clear right way to do something" — it's not clear if you have to read a book first.
16 Likes
I spend a lot of time working with Swift concurrency. I do not think it is either useful nor really fair to argue with any of this. No one can tell you your experiences or conclusions are incorrect. But I think what we can do is explore tangible ways to help. And I'm, honestly, quite encouraged to see some agreement that documentation (while necessary and good) is not really a solution.
I think there are three core problems.
- The language is hard to use in a number of common situations
- Many APIs are difficult to use in their current state
- Swift 6 has been adopted too aggressively
I absolutely do believe that the vision document for improving approachability will help tremendously. I think it's premature to say if it will help enough, but I'm quite optimistic.
I'm unsure how to meaningfully comment on the state of Apple's SDKs. There was quite a bit of progress with Xcode 16, and incrementally more with its point releases. There remains considerable work to do.
On the third point, it almost feels insulting to say. And I really don't mean it that way, especially since some concurrency features were introduced years ago. Why wouldn't you? But, for most projects, the Swift 6 language mode makes no sense. It can require a deep understanding to use successfully in many situations. And this isn't always clear until you're quite far into the migration progress.
10 Likes
Let me add a fourth: a lot of developers are having a really hard time understanding the diagnostics emitted by the compiler.
10 Likes
At the very least, written-out documentation is easily searchable in a way that WWDC videos are not.
6 Likes
You could be right, maybe it isn't helpful. I guess my point is this manifesto was a very strong statement with wide support — and these bullet points are still a grand vision worth aiming at. So maybe the future could look to actually achieving more of them.
(As for fairness, I'll let people make their own minds up on that one, but I did try to make put the arguments in terms beyond simply my own experiences as much as I could.)
I do want to raise something about your point about Swift 6 adoption though: —
In discussing the use or not use of concurrency recently, I've heard a lot of people say things like "you don't need to use Swift 6", "it's optional, just don't use it" etc. Now, if you are writing a project on your own, sure you can just not use it. But if you are writing a framework, or work on an app beyond a certain size, that isn't something you have control over.
And concurrency is not only found in Swift 6. Swift 5 has all these features and therefore issues too — and if you don't turn the warnings on, has basically all of the disadvantages with none of the advantages. As Matt Massicote put it here Making Mistakes with Swift Concurrency | massicotte.org : —
Swift 5 mode + no warnings can be an extremely unsafe dialect of the language. There are basically no rules, and you cannot use a compiler-based concurrency system when the compiler’s checks are disabled.
When you do this, you aren’t just at risk of building systems that work incorrectly, you are also building an incorrect mental model of how the language works. This is very bad. It may feel like it is getting you closer to Swift 6, but it could be doing the exact opposite.
In a Swift 5 world, in a codebase where anyone can start writing code with actors and async, you are flying completely without a safety net. So it isn't Swift 6 adoption itself that has been too aggressive: once you are in the Swift Concurrency world then the only really rational thing is to move toward the strictest and latest compiler possible as soon as you can, as that the safest and (should be) the best to use too.
Rather, it is the introduction of all these concurrency pieces: async/await, actors, and so on, before the language was ready to support their proper use: both from a safety and QoL perspective. Things like default arguments to functions not having isolation to me is a clear indicator of these features not being production ready.
Swift has always been willing to take some risks and adopt to change and that has been a strength over the years, despite causing some pain points here and there. But it never before felt like a pre-production research language — until concurrency.
So I think we are agreeing that whether or not it went too far, it went too fast.
4 Likes
Yeah this is a very important point too. Just like the start of the thread example with the function signature being hard to read, many of the compiler errors are extremely difficult to understand unless you've studied the new concurrency model very deeply. And even then they are are often unclear about specifics. Like when you get stuff like
Non-sendable type 'Whatever' returned by implicitly
asynchronous call to nonisolated function cannot cross
actor boundary.
Even once you've parsed that sentence, which is not a piece of cake, it is not clear what called or returned what and where the actor boundary it is unhappy about actually is.
4 Likes
And to make it worse: if you want to use SwiftUI for example (because you will be so much more productive compared to UIKit) even in the most basic app, the moment you use View.task { ... }, say for a network call, you open the floodgates and can't escape concurrency anymore. Similar situation with even a trivial CLI app.
Point being, concurrency is imposed on the developer. The reason is kind of understood, a mobile phone can have 6 or more cores these days. Without imposing concurrency, most of the apps on the App Store would've been using only say 15-20% of the computing power available.
So true!
5 Likes
@mattie My apologies I misread you, I thought you meant it was not useful or unfair for me to be arguing these points, which isn't actually what you said. Sorry about that, with you now! 
1 Like
I was just about to clarify, but you beat me to it!
I do, though, just want to really underscore that I subscribe to the philosophy that if someone says something is complicated, it is. Even if one hundred people piled on here and rated this thing 7/7, I still think it is absolutely fair and justified to see disagreement.
For example, I don't know if concurrency should get a pass on performance 
Sure, this is all pretty subjective stuff. And I do mostly have interest in practical progress from where we are now. But I also think fair and kindly-delivered criticism is far, far more valuable than praise.
4 Likes
We have all been flying without a safety net for a long time. Swift didn’t make concurrency hard or complex, it has been such all the time. The difference is that this is now highlighted much more prominently.
What new concurrency approach does is that it surfaces all the issues that has already been in a concurrency. Yes, it sometimes has false-positives, and yes — some of diagnostics are still cryptic. But this is still a step forward.
6 Likes
I get what you are saying.
But the point here is that if you introduce async/await into a Swift 5 code base without compiler feedback, there's basically a 100% chance of you running code in the background accidentally because of how non-isolated async functions work today. You'll also likely create ordering issues. So, in this regard, it is even less safe.
And then there's the design implications of fixing this, which can require substantial work.
2 Likes
Very few applications are grinding away at CPU-bound work though. To a very close approximation, all apps on the App Store are single-threaded. They have just small amounts of work done here and there in the background to keep the main thread responsive.
Just wanted to acknowledge this. I agree. I originally was going to say "concurrency was adopted too aggressively", but I felt dumb. It's hard to fault anyone for adopting a language feature like this. It wasn't even behind a feature flag.
Also want to acknowledge this too. I was kind of grouping this up in the whole "language is hard to use" point in my mind, but the way I worded it really didn't capture that at all and I'm glad you brought it up.
2 Likes
I would like to bring this up again, to stir some discussion.
Here is one problem, which really limits the utility of the concurrency: I can start a task from a synchronous function, but I can't get anything back from that task. Funnily enough, I can cancel the task though.
To speed up the work in a synchronous function, currently I have two options available (good old, reliable friends):
- GCD; and
- XPC services (but only on macOS)
I am just wondering why Swift's concurrency does not address the performance needs of the synchronous functions. There are a lot of synchronous functions which can not be made async.
Wouldn't addressing this need really make Swift's concurrency more useful and also approachable?
1 Like
What do you mean by that? How do you speed up work?
Honestly, I do not think the performance angle is particularly compelling. I'm not disputing that this use-case exists, but I think you can make a much better case for this from an interoperability and convenience perspective. It comes up a lot in those contexts.
A half-baked solution would be to allow await within synchronous functions. Doing this would flip a bit in the runtime that records the current dynamic isolation and blocks while the work is progressing. The runtime would then be in a mode that would crash the process if the thread pool is totally occupied when needing to switch back to this current isolation. Or something like that. I'm not 100% sure, but I think that's a general idea that would allow this to work with reasonably good deadlock prevention.
Of course, I'm very worried this would be used as a crutch to work around poor designs. But, sometimes, things aren't in your control and/or you just don't have time to deal with it right now. So, perhaps a little scary ceremony is worthwhile.
func synchronousFunction() {
let value = await(unsafe) someAsyncFunction()
}
Why not put the runtime in a mode where it will spawn more threads if the thread pool is totally occupied? That would be safer for everything except a risk of thread explosion -- and thread explosion is not a concern for me in any of the situations I find the need to block a sync function until an async one is done.
2 Likes