Has Swift's concurrency model gone too far?

While porting some of my libraries to Swift 6 in preparation for my next projects, it struck me for the first time in my 5-6 years of Swift programming, and maybe echoing some of the posts on this forum: has Swift become an overengineered language? Can it be simplified, or will it only get more and more complex?

That happened when I was looking at the Task constructor's signature:

@discardableResult
public init(priority: TaskPriority? = nil,
    operation: sending @escaping @isolated(any) () async -> Success)

Take just the operation argument. It's a closure that is sending, escaping, declares any isolation (I don't understand this part very well yet), it's async and it returns Success. That's a whole bunch of facts - 7 to be precise - you need to know about just one parameter of this constructor.

I understand that all 7 make sense and there's nothing you can do about it within the current strict concurrency model.

My absolute respect goes to people who have built this incredible language. I love Swift, and I understand that it is at the cutting edge of today's paradigm shifts in our industry. In order to survive in the multi-core era we live in, you do need more reliable concurrency and Swift delivers that.

However I can't help but think at times, has it gone too far? We went from semaphores, the simplest fundamental building block of concurrency, which is also error-prone, yes, to a monster of a paradigm that kind of guarantees correctness but may leave both you and even the compiler overwhelmed. (Let alone, you do need to occasionally resort to semaphores where structured concurrency hits its limits.)

Bottom line, I want to ask the language maintainers: could the next evolutionary step for Swift be simplification of the existing language rather than new features? Really, isn't it time to step back, look at it and maybe say: "how can we simplify things?"

Thank you!

40 Likes
12 Likes

Ditto, here. :slight_smile:

Also, the lack of unified, cohesive, comprehensive documentation makes it really unpleasant to adopt concurrency.

PS: One of the things, that really bothers me is the deep chasm between the synchronous world and the asynchronous world - it is one way only.

11 Likes

I also think we need a book like "Swift concurrency - the definitive guide". It has to be written either by the language architects or by the community. Currently the only authoritative documents are SE proposals, but as concurrency evolves fast many contents in those proposals have become inaccurate. Sometimes it's not even obvious that a SE exists (for example, I have recently known of SE-0411 by accident), not to mention to fully understand them. On the other hand, the problem with the current official user document (concurrency chapter in TSPL) is it's too basic. It's good for beginners, but it can hardly help to answer most concurrency questions posted in this forum.

I'd hope the book could cover the history of how the feature evolves, the design considerations, the relations of various features, and some implementation details (because I think that help user to build their mental models) and examples. I'd also hope the contents can be presented in a more user-friendly way than SE proposals (SE proposals are excellent documents, the only problem is that they assume knowledge in compiler internals, which most of us lack of. This is a place where a book can help by explaining more background when needed).

I'm aware that concurrency is still evolving, so there may be future revisions of the book. But I personally think a comprehensive and authoritative book (either in paper format or wiki format) is very much needed for Swift 6 users .

23 Likes

For me, the problem isn’t necessarily that the concurrency model has “gone too far”. (One can make that argument, but I won’t; I almost admire Swift 6’s lofty ambitions in this regard.) My problem is that they’ve gone too fast. I think it was simply too soon to transition from Swift 5’s well-intentioned “strict concurrency checking” warnings into hard errors in Swift 6.

It may seem ironic to say that they’ve gone “too fast” (as it’s taken us years to get here), but it seems wildly inappropriate to impose all these new Swift concurrency errors upon us when there are vast swaths of native frameworks have not transitioned to Swift 6 concurrency, yet. I have zero problems writing my own safe Swift 6 concurrency code; the vast majority of my headaches stem from interfacing with Apple’s preconcurrency frameworks or compiling Apple’s own code samples in Swift 6 mode.

We rely upon and must interface with these legacy frameworks. (And I’m not talking about third-party frameworks, but rather Apple’s own frameworks.) I deeply resent the hours-upon-hours wasted working around concurrency challenges with Apple’s own preconcurrency frameworks. (I’d love to see research into the total loss of productivity across the industry that all of this has introduced! It makes me think wistfully about the Swift 2 to Swift 3 naming convention transition, and that was a PITA.)

I feel like the victim of a “move fast and break things” philosophy, which is all fine and dandy if you’re the one “moving fast”, but is far less fun for those of us living with these “broken things”.

And don’t get me wrong: I am somewhat sympathetic to your observation regarding the closure decoration example of the Task initializer. Yes, it feels a little disquieting (and this is an example of only some of the closure qualifiers). I think another illustrative example is SE-0417, which is a great feature, but, if you need a literal flow-chart to figure out what’s going on, maybe that is a indication of some deeper problem. And there are dozens of concurrency designs that I think could benefit from refinements (constrained actor reentrancy, better integration of computationally intensive work, weird syntactic idiosyncrasies, etc.).

But with all of that said, my highest priority issue is that we shouldn’t be bound to Swift 6 rules when the frameworks we use are not.

17 Likes

Someone has tried to deliver this book :slight_smile: , see: https://practicalswiftconcurrency.com/

@donny_wals (bat signal)

4 Likes

For the universal adoption Swift looks for maybe a lot of this complexity is needed, some of it (maybe the hardest) is how many choices there are and subtle dialects (soon another one if you want memory safety / lifetimes guaranteed by the compiler or not… rust has a much steeper initial curve, but I think it does plateau / rise much slower after that jump), how many things are inferred, how much sugar there is.

Anyways, while Swift may need it, I am still unconvinced universal languages are better than learning domain specific solutions and most of all that the kind of apps we target iPhone for (iPad could be a bit of a change, but for practical purposes, iPhone apps dwarf all other Apple targets) are much much better served by this compared to good ol’ Obj-C, operation queues, etc…Maybe the initial Swift, but current Swift does not feel like built by those app developers for app developers. I am not saying it is necessarily bad, but there seems to be a philosophical difference between the two.

7 Likes

The new concurrency is incredibly neat and a phenomenal accomplishment. Hats off to all of you who pulled it off!

But I join the others here in calling for comprehensive documentation. Forcing us to constantly browse through 50+ related proposals to revisit information makes the learning curve far too steep and undermines the narrative of Swift being approachable. The fact that the migration guide is superior to the inadequate official guide speaks for itself.

18 Likes

This is something I think about a lot. Are these things you truly need to know to be successful? I think it is 100% natural to want to understand everything. You're looking at documentation for an essential API. How could it be you can use this without knowing exactly what each of these does?

I believe it actually is unnecessary and usually just a distraction.

Take the @isolated(any). It gives the callee (Task.init here) the ability to inspect the static isolation of the function argument. So in this case, there's this scary-looking annotation that, as far as I know, does not have any impact at all on callers.

Making things worse, this particular constructor is actually even more complex than you are seeing in the published documentation. I think discussions around ways to simplify/streamline/sugarize syntax here is worthwhile. I have some ideas, but I'd love to hear more!

So I guess I'm saying that I'm skeptical documentation alone will ever be able to overcome complex syntax. Yes, that syntax exists for a reason, and you cannot just eliminate it without giving up power. But that syntax also makes some APIs pretty darn intimidating.

12 Likes

Not like I'm saying it's not, but could be that this creates perception that Swift concurrency is too far? Semaphore is a synchronization primitive, concurrency is way bigger topic than just synchronization, though very important of course. I know there are industry techniques at the moment, but if one wants to built concurrency safe language there are a lot challenges to tackle.

1 Like

To be fair, I wasn't comparing semaphores to structured concurrency, I wanted to highlight how far we have gone in the past ~ 50 years. My point is more high-level and more philosophical: why should structured concurrency be so hard to learn?

What you will likely be facing today if you start in Swift 6 is:

  • a lot of seemingly unnecessary await's in your code (let's call it "async contamination")
  • a lot of cryptic concurrency compiler errors that are not immediately obvious
  • just a lot of "why can't this be simpler" moments with no immediate clear answers if you try to think from a language designer's perspective

(Bonus: and don't get me started on Apple's frameworks that are lagging behind. Just try AVFoundation alone...)

For example, I think Swift would benefit from having an attribute for pure functions, i.e. functions with no side effects that would be safe to call from anywhere. This implies detecting side effects by the compiler and I'm not sure about the complexity of it, but the language would benefit from it big time. Nonisolated is not good enough for this since it doesn't detect side effects should they creep into my function.

Then, in any more or less useful mobile app you are going to have global variables that you want to access fast, i.e. you don't want to build actors around them because await calls would truly contaminate your entire code base; it's where you resort to semaphores.

These and many other things leave the impression that Swift's structured concurrency is not mature enough and therefore maybe should not be enforced the way Swift 6 did.

I don't have any specific suggestions yet (apart from the pure function idea), but I would just love to see some discussion around the direction Swift is taking from here.

3 Likes

my 2¢ on this is that there are really two separate problems here – Apple frameworks that have not been updated to be “Concurrency friendly”, and really poor/superficial documentation.

i don’t really use the Apple frameworks, so i don‘t have much to say there. but regarding the second problem, i feel that Swift Evolution has a cultural problem of treating documentation as an afterthought and there is a sort of “we’ll ship the proposal and let the random people on the internet fill in the documentation aspect” attitude that hasn’t worked out very well.

i think that it would be really valuable for it to be a requirement that every new Evolution proposal be accompanied by a tutorial for how to use the feature, that is more than simply a “specification dump” for implementers.

37 Likes

I have encountered so much code that needs a simple thread-safe type, marks it as @unchecked Sendable, and does locking incorrectly internally. Concurrency, as a concept, is just hard.

That doesn't mean things cannot be improved. They can! And I think the vision document linked above will go a long way.

3 Likes

Having followed the Apple ecosystem pretty closely… I can confirm that quality "long-form" documentation was commonly shipped directly from Apple for major frameworks. The trend seems to have been to migrate to documenting with WWDC lectures and sample code projects. It's not necessarily a good or bad choice… but it's a tradeoff. Personally… I would be happy with more long-form documentation but I don't believe I have any kind of objective metric to quantify why the time to write that documentation should be calibrated as impactful.

5 Likes

I would wager most @unchecked Sendable code exists to satisfy the compiler, not to document the developer's belief about the safety of their code.

I agree 100% with the sentiment that those who don't live and breathe Swift Concurrency likely do not understand it, and will simply add attributes and modifiers until the compiler is happy.

7 Likes

I agree that docs have trouble keeping up with Evolution proposals in general, but also particularly with concurrency because it has been in flux for some time, the vision is long and complicated, and implementations often fall short.

But for an implementation deliverable for Evolution proposals, I wonder if tests would be more natural, feasible, and detailed than docs, which are gated by relatively ineffable style, audience, and complexity considerations.

The test suite should demonstrate a representative sample of supported behaviors. The tests have to be written anyway; publishing them enables Swift users to assess the feature immediately and directly in the common language of Swift instead of waiting for intermediary interpreters.

In cases where the implementation falls short of the proposal, it would really help if the test suite also demonstrated missing and even failing API (as such). Gaps and issues are well-nigh inevitable, and tracking them helps users, writers, and implementers alike.

With test suites, external observers could track implementation improvements with high granularity by observing how the test suite changes for each release; that in turn might trigger faster API uptake, library/app upgrades, and even (dare we ask?) documentation.

(I'm also guessing that public test suites would reduce the lag and increase accuracy for training AI on new API's, if that's your thing.)

1 Like

My worry that desprite the fact that topic of quality documentation has been lately quite often mentioned in this context, gains no interest whatsoever. Swift has had one of the best docs on the language, but concurrency is covered poorly.

For instance, if one wants to get grasp of concurrency features, they has to go through the proposals on the topic. That’s quite a lot of time to determine them, order by priority and disclosure, and then study. Even if you did something like this, to look up certain features there is no better option than Google “ swift evolution github”, and here I assume that you know what the feature is, which is impossible to if you haven’t studied them in the first place. At least, this is my experience getting into concurrency model up to this day.

I think it worth to consider getting some priorities, maybe within the community, to address this. Because right now information is scattered. I’ve been reading yesterday concurrency documentation on GRDB release and it’s magnificent — just there are a lot of details to catch about concurrency in general, that are missing in TSPL book or anywhere else in docs.

7 Likes

I strongly agree about the documentation being a big issue. Information about things like @isolated(any), region-based isolation, and the exact syntax of regex literals are only really documented behind Swift evolution proposals, which are unorganized, don’t show up in search engine results, and are filled with technical details that are irrelevant to developers using these features. The lack of proper documentation leads to a bad experience for anyone trying to use these features.

3 Likes

No bet :grimacing:

But my point was people frequently also do not understand "traditional" concurrency. They just don't have a compiler yelling at them in that case.

I'm really into the idea of additional documentation! However, I remain skeptical that more documentation will help with the problem that motivated the original post.

6 Likes

Here's the original signature, including the underscored attribute that is omitted from generated documentation:

@discardableResult
public init(
    priority: TaskPriority? = nil,
    @_inheritActorContext operation: sending @escaping @isolated(any) () async -> Success
)

I'd love to apply three simplifications:

  1. Make @_inheritActorContext the default behavior for an @isolated(any) function
  2. Make use of the ideas that the proposal which added @isolated(any) notes and have it imply sending
  3. Exploit the fact that I cannot think of a way that a synchronous function could do useful work with an asynchronous closure argument unless it is allowed to escape

2 might be cheating, and 3 could just be impossible and/or a bad idea. I haven't thought about either too hard. But, regardless, it's interesting to imagine.

@discardableResult
public init(
    priority: TaskPriority? = nil,
    operation: @isolated(any) () async -> Success
)
5 Likes