[Prospective Vision] Improving the approachability of data-race safety

Evolution Pitches
,
81

That is the beauty of it: if it is non-sendable, there are no multiple concurrent callers, because it cannot be sent.

edited addition to this point:
I think this is one of the most underutilized (or maybe least understood/internalized) properties of non-sendables, and I suspect this is the reason why people over-reach for global actors. Non-Sendable types, in combination with inherited isolation and the sending keyword for transferring, are what you should reach for first IMHO. Then locks/mutexes or actors, and only then global actors.
edit 2: I was referring to mutable shared state/classes. More generally, sendable structs should probably be the first weapon of choice.

If it is Sendable, you will have to manage locking or the compiler will complain.

6 Likes
82

I don't get it. Why should one reach first for a non-Sendable type? For example, a struct with Sendable properties will be Sendable automatically, but under your assumption should this be considered an issue?

Sendable means "it's not going to cause data races", which is something always desirable. In some cases the compiler will be able to infer automatically that, yes, that type is not going to cause data races, but in other the dev will need to tell the compiler to trust them with @unchecked: what's the issue with this?

EDIT: my bad, I didn't interpret the message right

83

I was referring to classes and "protecting" their members, as the poster above was describing.

Obviously there is nothing wrong with sendable structs, those are also my first choice. So, to conclude my "imho generally do this" list, sendable structs are first.

I would not be so liberal with @unchecked Sendable: avoid and try to properly refactor mutable state into mutex boxes would be my advise.

I'll edit my post above to make my intention clearer, thanks!

1 Like
84

ah I got it now, I agree wholeheartedly

2 Likes
85

Iā€™m sympathetic to this argument. Obviously there are many different kinds of developers using swift in very different ways like app/UI developers who have to be tied to the main actor and have lots of issues with swift 6 errors and just trying to simply use UI apis and then complex library developers trying to do what the Vapor developer talked about up thread (geez that sounds hard).

Iā€™m super pumped about the main actor everything in the module idea. As a UI engineer for a long time my rule has been for as much code as possible to run on the main thread and ONLY jump off when you need to. It makes things MUCH simpler to reason about AND with UI programming you want to avoid async calls that would lead to loading states in your UI as much as possible to provide a very responsive UI.

The number of times I need to reach out of the main thread for big things (like large classes or subsystems) is pretty small. Things like downloads, caches, DB access.

Anyways all that to say, what I really think would be useful for this discussion is moving onto the proposal phase of the main thread idea so we could understand a lot more about it and how it would work and what exactly would differ.

If we need to understand a lot more or approve of the main thread idea to ā€œapproveā€ the vision doc then I think I need a preview of what a proposal would look like with lots of examples.

1 Like
86

Is there a single large document from the swift team about the entirety of swift concurrency that is always kept up to date with there latest approved proposals?

If not, could that be proposed as part of this vision doc?

6 Likes
87

I have no doubt that opting in to this whole process has great value for some projects/teams.

I agree that developers make mistakes.
If the compiler could stop mistakes for free, then it would be obvious that we should opt in to that safety.
Things become less obvious when the safety comes at a price. To me, the price of the safety that swift 6 offers is not worth the value it gives.

Again - as an opt in feature, I think 'safe mode' is magnificent.

As with most things - the question is cost vs benefit.
For me, the pain just isn't worth the benefit.

And of course - there are still plenty of other areas where developers can make mistakes.
Indeed - I suspect that the contortions required to satisfy the compiler for data-race safety will probably lead to more complex code which inherently breeds different classes of errors.

To take a trivial example - Apple has released new APIs this year which are async only. The one I particularly want to use crashes repeatably due to an out of bounds array access. I don't know if that was written with swift6 safety - but I assume swift 6 safety would not have helped.

1 Like
88

I think this is an interesting point but I think there will still be pain points. The simplest example that comes to mind is a non isolated function (again the default for a async function in a non isolated class or struct) wanting to call something that is MainActor.

2 Likes
89

Thinking about this a bit more: I have a huge code base and at the app level everything kind of already assumes main actor. Turning on swift 6 errors is very very hard. If I could flip a switch and make it a main actor module maybe it would get rid of a lot of those warnings and make it easier to adopt swift 6. I donā€™t know but that would be my hope.

2 Likes
90

this is quite awkward for libraries, because the tradeoffs are dramatically different between Swift 5.10 and Swift 6.0.

in 5.10, everything must be Sendable, because RBI doesnā€™t exist in 5.10. in 6.0 itā€™s just the opposite.

there are not a lot of good solutions to this. RBI is not something that can be ā€œbackportedā€ to an older compiler. and Sendable is not something that can be revoked later due to API stability considerations.

2 Likes
91

One solution I'd love to see would be to have a much less strict Swift 6 option.

Let me run in something akin to 5.10 mode.
Then I can turn on strictness in files/modules or the whole app when I judge it useful.

Just to take one trivial example - I could switch my code to swift 6 and use sending to pass an NSImage out of isolation rather than having to declare it as Sendable as I do today.

Odds are that a lot of my classes could turn on strictness this way with no or small fixes.

This would be more akin to how we migrated to arc where I could enable it at a file level with a comment.

92

I agree that Swift 6 is a very difficult language to use, and that something needs to be done to improve the situation, and I'm not really against the idea of a mode that isolates everything to the main actor by default ā€” that definitely makes it easier to work with Apple's SDKs.

What I do have is a vague sense of unease, that the problem with Swift 6 isn't so much Swift 6, as the state of Apple's SDKs. For example, AVFoundation, Security, etc. are more or less impossible to interact with from Swift 6, but I don't think that's mostly Swift's problem.

(I also think there's a limit to what MainActor-only can accomplish ā€” for example, AVFoundation won't actually be usable even in the proposed dialect, since it explicitly gives you synchronous, @Sendable callbacks that are expected to invoke @MainActor methods, which would still be illegal)

I also think that there's an aspect of "throwing the baby out with the bathwater" in the vision, in that if we encourage people to uses classes, inheritance, and global variables in the MainActor-only mode, they are only making life harder for themselves in the long run ā€” those things are bad for concurrency, yes, but they're also generally harder to work with and reason about than immutable data and value types.

tl;dr: I think the current state of pain is less about Swift 6 itself than that the world hasn't caught up yet. I'd love to see what the world looks like in a couple of years, when class isn't the first thing recommended by beginner tutorials, when Apple has had more time to produce Swift-friendly APIs, and be able to use the hindsight there, to decide whether this is really a change we need to make now.

My experience is that writing Swift 6 "from scratch", and with APIs that are designed for it, isn't really harder than Swift 5. It's just that, for the moment, I mostly have to create those APIs for myself (which is hard).

12 Likes
93

This is an absolutely magnificent example of the folly of the current Swift 6 approach.

A package developer wants to provide clients with a way to consume his log messages.

This does not seem like a rocket-science level problem.

The contortions he goes through - only to end up with code that compiles, and then hard crashes would make you think he was trying to extract something truly HARD from the language.

The guy has clearly spent a bunch of time on this and written an article on it. He can only make an educated guess why it crashes.

https://jaredsinclair.com/2024/11/12/beware-unchecked.html

It's a perfect example of complexity and a new crash as a result of the contortions required to do something that used to be (and should be) simple. Imagining that regular coders have time to understand this kind of complexity is....

It get's better. Update at the end:

Matt Massicotte writes:

Whatā€™s happening here is the compiler is reasoning ā€œthis closure is not Sendable so it couldnā€™t possibly change isolation from where it was formed and therefore its body must be MainActor tooā€ but your unchecked type allows this invariant to be violated. This kind of thing comes up a lot in many forms, and itā€™s hard to debugā€¦

[emphasis mine]

The compiler has reasoned that you couldn't possibly have sent a non-sendable across an isolation barrier. So, even though you disabled the compile time warning, it's going to generate a deliberate crash. This makes the language safer???

1 Like
95

(Apologies I typo'ed and then clicked the wrong button!)

You can use sending in the Swift 5 language mode! All the new 6 compiler language features are available in that mode. And that's definitely the better option than marking NSImage unchecked Sendable, if that's what you mean, because it isn't and that could cause issues anywhere that extension is visible.

96

That doesn't work for me. My guess is because I'm sending across a package boundary...

But who knows- this stuff is all dark arts...
It's not like I can go and check the documentation. So in the end, I just do the hacky thing that lets me get on with work...

97

And therein, perhaps, is the heart of the problem:

  • Swift 6 throws a lot of errors at you when you try to play in pre-Swift-6 worlds
  • The errors often don't have obvious solutions, and some of the solutions that will fix the errors will just punt your problems down the line. You actually have to understand the things Swift 6 wants of you, and understand the solutions the language makes available, to be able to fix them correctly.
  • most people don't have that experience yet
  • some people are unwilling to put in the effort to gain that understanding

Taking that Jared Sinclair blog post as an example, the problem is that he's not understood what it means to have a global closure, used @unchecked Sendable to bypass the compiler's ability to check things are safe, and they're not. Here's what happens if you use a type that actually obeys Swift's concurrency rules instead:

Screenshot 2024-11-28 at 12.50.43 PM
Screenshot 2024-11-28 at 12.50.43 PM2028Ɨ442 84.9 KB

On the other hand, what do those errors mean? This is literally the first time I've ever seen that error, and it's doing nothing to help me understand what I've done wrong. (rhetorical question ā€” the correct solution here is to change to typealias LoggingSink = @Sendable (String) -> Void. I know that, but how can we expect anyone to arrive at that conclusion from that error?)

That all said, we're deeply into "creating APIs for Swift 6" territory here, not in "using Swift 6". If he depended on swift-log, his log handler would be forced to be Sendable, and the problem would go away. On the other hand, and to bring this all back on-topic, even from MainActor-only-ville, because swift-log is doing the right thing in general, he still can't create a log handler without understanding Sendable.

7 Likes
98

The only thing I disagree with here is your use of the word 'yet' !

To my mind, this isn't fairly simple stuff where people will easily catch up over time. This is mind meltingly convoluted stuff where most people (myself included) will never catch up.

And this mind meltingly convoluted complexity is required in my case to do a 100% safe return of an NSImage from a function that made it.

Swift has become a mental puzzle game where you need to figure out complicated wrappers, architecture, indirection to achieve things that should be natural.

I'm choosing to 'play in pre swift 6 worlds' because at the moment, that seems like the best way to stay with a language that is broadly functional and doesn't require this level of understanding and architectural conformity for me to build things. ...And it's still painful with these ever increasing warnings that I can't turn off!

Regarding the error - You're right that you can't expect people to figure out these things from the errors given - but I think you're looking at the wrong level of the problem.

The real problem is that it shouldn't be hard for a package to vend error messages to a client.

student: I want my package to pass messages to clients

teacher: First one must understand a generic mutex on a static enum function with custom getter and setter as a method to pass Sendable Strings...

Giving better error messages might make that complexity a little easier to deal with, but it would be missing the fundamental issue.

4 Likes
99

Picking on this to emphasize one of my points ā€” NSImage has a legacy design that's very problematic in Swift 6:

  • open class
  • mutable data

You'll be fine if you keep all your images on the main actor, but if not, you're in trouble.

(CGImage is immutable, and therefore avoids all these problems, but I'd also love to see a modern CoW value-typed bitmap image type somewhere...)

But again, the problem is less Swift 6 per se, and more with the API you've chosen/need to interact with.

3 Likes
100

"Yes, and..."?

What about: instead of just targeting data-race safety, support a "base" module mode for all introductory best practices?

A base module would have the defaults and subset of features intended to avoid complex degrees of freedom and unnecessary ceremony, with more guidance from the compiler, but would be expressly transitional.

Sample base restrictions:

  • Library or executable only
  • No top-level code, aside from one fixed entry point:
    • func main(arguments: [String]) -> Int {}
      • i.e., no throw from main
    • (or @main inside a struct)
  • Latest language mode
  • No plugins
  • No swift settings: experimental/upcoming features, unsafe flags, etc.
    • No other settings: linker, cxx, etc.

Base enhancements (noticed on emigration):

  • No 'Package.swift' necessary for a file or folder of sources with swift -base {file or folder}
  • Single-threaded presumption
  • Compiler has base diagnostic mode with messages tailored for new developers
    • e.g., string indexing, perhaps high-traffic Apple framework issues
  • On macOS, Info.plist at root of resources is incorporated into executable (and {src}/Resources is adopted as such)
  • (others?)

Since the goal is to lead with current best practices but use this mainly as a starting point, Swift compatibility guarantees would be limited: a base module will be source-compatible in the same language mode, except in later versions it may gain warnings or errors (and behave differently after warning), offering more guidance and possibly ousting people from behaviors newly considered bad (hopefully with automated source migrations).

This could be seen as a dialect, but a transitional one in favor of simplicity and good practice, which could help make Swift "a better first language" without taxing experienced developers with limits or noisy feedback.

101

This reminds me of the old joke.

For a spherical horse on an infinite flat plane - my horse racing prediction technology is 100% accurate.

This is why I refer to Swift 6 as a 'science project'

If you allow yourself the luxury of ignoring Apple's frameworks and classes - then the constraints it imposes seem more reasonable.

In reality, I'm going to be using NSImage, Codable, Core Bluetooth, etc

A language which doesn't play nicely with those is no use to me.

10 Likes