[Prospective Vision] Improving the approachability of data-race safety

If I am understanding right, the justification for having different default isolation based on target has to do with libraries exclusively.

It sounds like a dialect would exist only for the convenience of some (possibly large subset) of library authors? Or are there technical reasons why a universal MainActor-by-default would be problematic?

3 Likes

I think it's not just convenience, it's the wrong default. If your intent is to write a library that can be used from any isolation — presumably the right idea for data structures, databases, I/O, and most other libraries — Swift should not force you to specifically remember to add an extra attribute to everything in your library.

9 Likes

I don't think your suggested approach is that different from the vision presented here.

I agree that all new language features should be documented in the language guide. I also think that programmers need more in-depth information on concurrency than makes sense to include in the language guide. Several folks here have mentioned the value in having more guidance on recommended code patterns for various kinds of concurrency use cases, and those wouldn't belong in the language guide. The kind of resource that programmers need to migrate to a new language mode is also very different than the content of the language guide -- this is what The Swift Concurrency Migration Guide: Redirect is for, which covers Swift's data-race safety model in-depth and describes how to migrate to the Swift 6 language mode, how to fix common concurrency errors, etc.

I'm all for improving documentation coverage and the kinds of documentation that Swift.org offers. Those improvements aren't covered by the vision document simply because those changes don't need an extensive review process like language changes do -- documentation doesn't have long term source compatibility and ABI constraints, documentation is easy to tweak once it's been produced, etc, so gating it behind a heavy weight process isn't necessary.

I personally agree that documentation in the language guide should be a requirement of the Swift evolution process. However, that's a general policy change that increases the already high bar for proposal authors, and needs to be discussed separately from this vision for approachability of data-race safety.

Data-race safety is opt in today.

There is only one case where the Swift 6 language mode is the default, which is when you have a Swift package that requires Swift 6.0 tools in the package manifest and doesn't otherwise specify a prior language mode. It's debatable whether or not this is correct behavior, and I'm open to having Swift package creation using 6.0 tools explicitly specify the Swift 5 language mode by default. For all other forms of project creation, and for existing projects, the Swift 6 language mode is not the default, and programmers opt into strict concurrency checking, either by bumping up the checking level of the -strict-concurrency setting or by opting into the Swift 6 language mode. The purpose of the Swift 6 language mode is to enable static data-race safety, so if you don't want that checking, e.g. because the false-positive rate is too high in your code base, then sticking with the Swift 5 language mode is a perfectly reasonable choice.

I know that a lot of folks have run into compiler warnings about Sendable violations when building with minimal concurrency checking when frameworks they use add @Sendable to completion handles, etc. Those are typically compiler bugs. For example, there was a recent issue where people building with -strict-concurrency=minimal (the default in language modes < 6) were seeing a lot of new data-race safety warnings when using DispatchQueue.concurrentPerform, which I just fixed, including on release/6.0.

This is not really different from building with -strict-concurrency=complete in language modes <6, where all data-race safety issues are warnings. I know a lot of programmers will build with minimal checking, and periodically turn on strict checking to evaluate their concurrency use by going through the warnings and fixing them or ignoring them, which is similar to the workflow you're describing. That workflow could certainly be streamlined to give folks an easier way to perform one build with strict concurrency checking enabled, rather than going back and forth to their build settings to toggle those options.

A solution to this problem is described in the vision document here.

The document isn't suggesting that programmers throw away existing uses of concurrency. The idea is that most executable code that isn't explicitly using concurrency is always running on the main actor in practice today. Having to annotate all of that code with @MainActor is not useful because that code is already safe.

8 Likes

This paragraph means you can pass a nonisolated function to an API that is expecting a global-actor-isolated function. The API can only call the function from the global actor it specified, which is fine because a nonisolated function can be called from anywhere. Does that make sense? I can clarify the text and/or add a code example.

I see this as a long-term default, and you can choose to configure the default as part of creating a new project or target. You certainly could turn off "main actor by default" mode later on into a project's development, and we could offer an easy way to do that by making the inferred @MainActor annotations explicit, and vice versa. But I don't think of this as a "training wheels mode" that the project and its programmers eventually don't need, and I don't think the maturity of a code base necessarily matches the experience level of its contributors either. While a code base may get increasingly more sophisticated over time with respect to its concurrency usage, the contributor base might be continuously welcoming new members with varying levels of experience. I also don't think the maturity of a code base is an indication of where actively written code runs - there is plenty of active development on "sufficiently mature" executable projects where new code is written to run on the main actor, e.g. for new user-facing features.

6 Likes

I think it's not just convenience, it's the wrong default. If your intent is to write a library that can be used from any isolation — presumably the right idea for data structures, databases, I/O, and most other libraries — Swift should not force you to specifically remember to add an extra attribute to everything in your library.

But on the flip side, why is it better to force app developers to add extra MainActor attributes to everything? What percent of Swift usage is by app developers, 95%?

1 Like

That is exactly why the vision proposes that should be a language mode to set the default.

1 Like

This is the approach I was taking when I first started working on this vision document. But I don't think this is enough to mitigate false-positive errors in single threaded code. Of the 5 pain points listed under the Single-threaded code and its challenges under Swift 6 section:

  • global and static variables,
  • conformances of main-actor-isolated types to non-isolated protocols,
  • class deinitializers,
  • overrides of non-isolated superclass methods in a main-actor-isolated subclass, and
  • calls to main-actor-isolated functions from the platform SDK.

The "main actor by default" mode is what solves the problem for global and static variables, class deinitializers, and calls to main-actor-isolated functions from the platform SDK.

Yes, I have thought about this, but I think this approach falls short because it does not help with the @MainActor proliferation that occurs as soon as you need to access that global variable.

Many Objective-C APIs are built on shared mutable state, which makes those APIs more difficult to use in the Swift 6 language mode where there are more restrictions on how mutable state is used. However, I don't share the perspective that these pain points aren't Swift's problem, nor do I agree that the solution to the problem is idiomatic Swift APIs. It has never been a goal for programmers to stop using Objective-C patterns in order to write Swift code. Rather, the goal is to enable writing modern, idiomatic Swift code incrementally within an existing code bases that uses Objective-C patterns. The same is true of Swift 6. The goal is not to force programmers to re-write all of their code to rid their uses of classes, inheritance, global variables, etc. These features are perfectly reasonable tools to use in many cases, and there are ways to use these language features safely in Swift 6 mode.

There are several ways in which the language makes these APIs harder to use than necessary. For example, the execution semantics of nonisolated async functions makes nearly all async functions bridged from Objective-C completion handler functions impossible to call from the main actor in the Swift 6 language mode. I believe this is a problem not with the API itself, but with the way the API is imported into Swift. I also think the interactions between global actor isolation and protocol conformances / subclasses make these patterns unnecessarily difficult to express.

Yes, I agree that there are limits to what the main actor by default mode can accomplish. If a library exposes its use of concurrency through its public API, e.g. by requiring synchronous @Sendable closures that are called off the main actor, then clients still have to adhere to those rules.

6 Likes

This seems to me a very funny thing to be saying at this stage.

As soon as Sendable was proposed, it immediately became clear that a huge swathe of Apple's existing platform APIs would never be safe in Swift 6 — whether that's RunLoop or NSLock from the 80s, Operation or URLResponse from the 2000s, or any of Combine from only 5 years ago, the introduction of Sendable put a vast quantity of existing APIs on the chopping block.

I thought that was the point. That we were ushering in a new world of memory-safety in the face of parallelism, so that anybody would be able to produce correct parallel code and effectively utilize (to the degree possible) Apple's increasingly-parallel hardware. That we were accepting that one couldn't make an omelette without breaking eggs, and that these particular eggs were worth breaking.

And honestly, I've never expected that Swift would remain an ObjC-compatibility layer — from that very first WWDC touting protocol-oriented programming and my first steps in this new language, it was clear that Swift was its own thing, that there were better ways to build Swift APIs and write Swift code than sticking to ObjC's patterns. Being able to use ObjC APIs has always felt like an afterthought in Swift. The introduction of SwiftUI only served to make it clear how much better Apple's platform APIs could be by actually making use of Swift. Every WWDC I hang out to find out which ObjC framework is being layered over or replaced with something more natively Swifty, more expressive, more powerful.

To now come and say... oops, this past decade of Swift's evolution was all a mistake, perfect compatibility with 35-year-old ObjC APIs are still the primary driving force for Swift, sounds... well, less like a decision of the Swift language team, and more like an edict from someone who doesn't understand the consequences, if I'm being honest.

No amount of implicit @MainActor is going to make half of Foundation usable again, or any of Combine safe. If the goal is truly for perfect compatibility with all the existing platform APIs, that's provided by Swift 5 mode (maybe with a toggle to hide all the warnings).

7 Likes

Hello Holly, thanks for the extensive reply in the thread. I had a small follow up question I wanted to ask here: “ guidance on recommended code patterns for various kinds of concurrency use cases, and those wouldn't belong in the language guide” why wouldn’t it belong in the language guide?

Someone picking up Swift today or “tomorrow” would reasonably expect to have it in the guide, not in a migration document from a version of Swift they never learned (not sure the migration document would help them much).

3 Likes

Just so I understand, are you saying these APIs are impossible to use from Swift 6, or just really difficult?

That is not at all what I said. I said the goal was to enable writing modern, idiomatic Swift code incrementally within an existing Objective-C code base. Objective-C code can stay as it is and/or it can be rewritten over time in Swift. On Swift 6 migration, it is not a goal for programmers to have to rewrite all of their code to get rid of Objective-C patterns in order to migrate to Swift 6. The point is modernizing code incrementally. If programmers had to rewrite huge amounts of their code in order to migrate to Swift 6, that just makes it an insurmountable task.

I also think that, interoperability aside, language features like classes and global variables are fine features to reach for in some cases. Others might disagree. But me stating that does not imply that I believe the past decade of Swift's evolution was all a mistake.

16 Likes

That is exactly why the vision proposes that should be a language mode to set the default.

The default is whatever Swift ships with out of the box, not what configuration you can pass. If I'm reading the vision doc correctly, Swift will ship by default with today's behavior.

IMO, per-module mode will be regretful. Large codebases are a mix of app code and libraries, and this inevitable dialect will make reading Swift locally, on github, etc. problematic. I know Apple disfavors breaking changes, but if there's a case for Swift 7, it's to fix today's concurrency defaults.

1 Like

One caveat I have with that list of pain points in particular is that while I see the immense value of getting rid of those pain points for truly single-threaded code, my experience with some of those has been that they nudge you in the direction of writing better Swift code. The MainActor default would also get rid of that nice effect, sadly.

Take global variables as an example: currently, if I inadvertently make some global state mutable, the compiler warns me, and I have to make a decision. Typically, I choose to make the variable a let or move away from a global variable to something that is passed around in initializers (which is most often the right choice!).

This decision has a tiny cost if done at the very beginning. However, if the variable is automatically annotated with @MainActor instead, and later on I need to use that variable from outside the Main Actor (for example, when slowly introducing parallelism into an app!) reversing that decision could be very painful.

I can think of similar examples with protocol conformances. Do I really want a Codable type to be @MainActor? Is there really no chance that I'd want to encode/decode instances of this type from a background thread in the future?

I know it's not the compiler's job to educate developers or prevent bad architectural choices. But so far this nice side-effect has been such a big part of why my experience with Swift 6 / Strict Concurrency has been so positive that I couldn't help mentioning it.

6 Likes

Do I really want a Codable type to be @MainActor ?

Isn't this a solvable problem by Apple annotating Codable?

Thanks for clarifying. Assuming this vision doc moves forward into reality, is it possible to give some rough idea of when this would be available in a Swift toolchain? I think my project would seriously consider using the main actor default for almost all of our "feature" modules (and some libraries as well) that use Swift Concurrency.

1 Like

I agree. You know how people talk about async causing chain reactions of everything that touches it needing to become async? It seems to me that in addition to that currently everything that touches async needs to become thread safe (or sendable or worry about safety). What I really like about this change is that (for an app or script) you can use async a ton without having to think about a jump through all the hoops of thread safety.

1 Like

I think this direction will alleviate much of the pain of dealing with concurrency, so I'm happy for that.

The main concern I have is that it will be more difficult when looking at a piece of code to know what isolation applies to it. This is already somewhat of a problem—if someone pastes a function on StackOverflow or something, we don't know what the isolation is without knowing whether it's inside an isolated type. And even that can be hard to determine; the type could be directly isolated via a @MainActor, but that isolation could also be inherited from a superclass or via a protocol conformance. (And either of those could be inherited directly, or via some other transitive conformance.)

This vision document just adds one more way that a function's isolation could be determined. It's a trickier one, though, because the isolation is not specified anywhere in code; it's instead a build configuration issue.

This readability concern is acknowledged in the vision document, which I appreciate. I do think that addressing that concern will be an important part of making this change palatable. The vision document suggests that "it would be reasonable for IDEs to be able to present isolation information for the current context". That would be helpful.

We should also make sure that documentation tools (e.g. DocC) clearly document that isolation information. Currently, DocC notes the isolation for types, but doesn't show the effective isolation of functions and properties. It may be helpful to make that isolation information available more broadly in the documentation.

8 Likes

Perhaps we could revisit the "isolation inference from protocol requirements" feature as part of this effort. It never sat right with me, either from a language design standpoint (we don't "infer" static, mutating and so on from a protocol requirement; why is @MainActor any different?) or from an implementation point of view (this kind of analysis leads to hard-to-fix "circular reference" errors and slow compile times).

20 Likes

Another (excellent) article by massicotte

He does a fantastic job of explaining what is going on with all the mental complexity that pops up when you try to do the simplest most contrived bit of async work.

He does a great job of explaining this.

But my takeaway is that the need for all this explanation shows the fundamental problems with the language as it stands.

I have just come back from a week rolling out software I built to some teams of users.

Every time they find something complex - I could conclude that they just need more training ('Skill Issue') - or I could ask myself why they found it counterintuitive/complex to use ('Design Issue').

I lean towards the latter as a default.

5 Likes

Thanks for the detailed response @hborla

To some specifics:

Re documentation - beyond the point that documentation is important, I actually think running a process of generating accurate documentation would be immensely valuable to the team.

The act of figuring out how to communicate what swift does would force you to confront more directly the reality of the current situation.

I agree with you that the documentation is not the place for general style guides, or recommended patterns. However the behaviour should be fully described.

I think you're missing an opportunity here. Technically this is of course true, but I think forcing documentation would add massive value to the review process by really bringing the the fore 'We're telling users do to what???'

Also - the reality is that while technically the language could be documented after release - that hasn't happened so far.

It is. Long may this remain so.

If the vision group were to publicly declare that they expect it to be opt-in for the forseeable future, then I would be much less concerned.

However - my fear is that as with other Apple changes, the new soon becomes mandatory. If not technically, then in practice.

This is great. Again - I'd love to see an explicit statement about how much 'leakage' is expected/allowed from swift 6 to swift 5.
For example, I recently had to mark NSImage as Sendable to resolve a warning in swift 5 mode - where I'm explicitly using sendable in a way that I think should fix the warning.

Great to hear that this is (probably) a bug, rather than deliberate imposition on swift 5.

I'm not 100% following what you're suggesting there, but if it allows me to conform to Codable with an isolated type (potentially calling async encode(to:) ) as appropriate - then I'll be a happy camper.

I have to disagree on this.

  1. By creating an 'easy' mode for the language - you're implicitly telling people to start in easy mode. So, you're throwing away a lot.
    That's an implicit admission that the language has failed to 'make basic use of concurrency simple and easy.'

  2. I think explicit annotation is (generally) good.

The bottleneck for writing code isn't the typing part - it is the thinking part.
Explicit annotations make the thinking part clearer. Someone reading the code knows what they're looking at because it is explicit in the local context.

Of course - there is a balance to be had here, and tooling could make this explicit (green checkmark on any implicitly single thread files as an example approach)

1 Like