Concurrency in Swift 5 and 6

Hey all,

Swift has always been designed to be safe-by-default. This informs runtime features like bounds checking, integer overflow checking, and automatic reference counting, and type system features like optionals. But this safety has never extended to data races, which are easily created in practice and hard to completely eliminate.

An explicit goal of the concurrency effort for Swift 6 is to make safe-by-default also extend to data races. The type system and language model work together, through features like actors, Sendable, and global actors, to eliminate data races.

However, there is a lot of Swift code out there that was developed without the Swift concurrency model. Some portion of that code is going to have to change to fit within the constraints of Swift’s concurrency model, but it’s not going to be rewritten in a month or even a year. We need to consider what it’s going to look like to migrate the Swift ecosystem to Swift 6, and expect that this process will take years. Along the way, we need to ensure that:

  • Swift 5.x and Swift 6 code can interoperate well as concurrency continues to evolve,
  • Incremental adoption of the concurrency model produces incrementally better and safer code, and
  • Each module, package, and program can adopt the concurrency module at its own speed without unnecessarily breaking clients.

Given the need for interoperability with existing code over years, we can restate the goal in a different way:

A program that uses the Swift 6 language mode throughout will be safe-by-default from data races.

Learning from prior transitions

Swift has been evolving for a number of years, and we’ve learned a bit about how to manage transitions. Here’s a quick summary.

  • Swift 2→3: Swift 3 was a major transition for the language, and in many ways it was a painful one. First, the scope of the change was large, with a huge number of APIs changing names and many syntax changes to the language itself. Second, it was an all-or-nothing transition: with no interoperability between Swift 2 and Swift 3 code, one could not start to migrate a module until all of its dependencies had already made the switch to Swift 3. This bottom-up rewrite of the Swift ecosystem was painful and slow, effectively requiring many module authors to have two distinct versions of their code. Bottom-up rewrites like this aren’t possible given the size of the Swift ecosystem today.
  • Swift 3→4→5: The syntax changes since Swift 3 has been more modest than the Swift 2→3 transformation, but more importantly, the model is different: Swift 3/4/5 code can all coexist in a program. The Swift compiler supported all of the language modes together, allowing one to mix modules compiled for the Swift 3 mode with modules compiled in Swift 4 or 5. This allowed each module to move to a newer language mode at its own pace, without breaking its clients. We also allowed features from “newer” Swift versions to be used in older language modes: for example, conditional conformances to protocols were introduced in Swift 4.1, but that compiler allowed those conformances to be written and used in the Swift 3 language mode. This allows incremental adoption of new features even before a module fully moves to a new language mode.
  • Limiting @objc inference: The move from Swift 3→4 changed the rules around @objc in a manner that broke code at runtime. To assist here, the Swift 4 compiler introduced additional compilation flags to specifically identify those parts of a Swift 3 module that would be affected by the change, with a mix of compile-time and run-time warnings. This allowed developers to slowly adapt to the change within the Swift 3 language mode before “flipping the switch” to adopt Swift 4 and the rule change.
  • Memory exclusivity enforcement: On the road to Swift 5, Swift tightened up rules about exclusive access to memory. For example, it became illegal to form two inout references to the same memory location at the same time. This was treated as a compiler change and was not subject to different language modes (and there’s no opt out). Instead, it was staged in over more than a year, with compiler error messages and runtime checking, so the ecosystem could adapt to the new rules.

If we take those altogether, we want to avoid anything that requires us to remake the world from the bottom up. We need to embrace individual modules moving toward the new language mode one-by-one, in any order. We should make it easy to incrementally adopt aspects of the new language model before making the jump to the new language mode, and get incremental gains from doing so. If at any point the only way forward for the developer is to switch the language mode and fix 100 errors before they can make progress, we have failed.

Concurrency in Swift 5 and 6

To enable the incremental rollout of Swift 6’s safe-by-default for data races, we’ll allow the use of concurrency features in Swift 5 with a “relaxed” enforcement model that allows incremental adoption:

  • Certain rules to maintain the concurrency model’s guarantees will only be enforced within code that has adopted some aspect of the concurrent model (e.g., within code that is async or is defined within an actor). For example, the closure passed to DispatchQueue.async will be required to be @Sendable only when called from code that has directly adopted some aspect of the concurrency model. This begins the reinterpretation of existing APIs in the manner that will be required in Swift 6.
  • Sendable conformances can be defined and will be checked for structural correctness, but Sendable requirements will not be universally enforced, e.g., when passing a value across actors. This allows Sendable conformances to be built up throughout the ecosystem without blocking incremental adoption on a bottom-up process.
  • A compiler option ( -warn-concurrency ) will be provided to warn about Swift 5 code that will be invalid under the Swift 6 rules. For example, Sendable will be checked but missing Sendable conformances will produce a warning (rather than an error). Developers can use this to prepare for Swift 6, by highlighting the places where fixes will be needed, but without needing to fully adopt Swift 6. The behavior of this flag will evolve as we narrow in on the semantics of Swift 6.
  • A compiler option ( -enable-actor-data-race-checks ) will be provided to diagnose any runtime data races caused by missed checks in Swift 5. For example, if Swift 5 code executes a non- @Sendable closure concurrently, this flag will detect at runtime that a synchronous actor-isolated function was called on the wrong executor.

Swift 6 will not be available as part of the initial release of Swift Concurrency. Instead, the initial release of Swift Concurrency provides the fundamental tools for concurrent programming in the manner that will help eliminate many data races. Swift 5 will then continue to provide source compatibility with prior versions of Swift, allowing developers to incrementally adopt the new concurrency features. Feedback from the use of the concurrency features in Swift 5 will direct further refinements and improvements to the concurrency model, so that Swift 6 can deliver on its promise to be safe-by-default from data races.

Staging tools for Swift 5 and 6

The Swift compiler can manage the Swift 5 and 6 language modes, making specific diagnostics silent/warning/error as appropriate. However, to facilitate a smooth transition, we need to be able to distinguish between APIs that have always used the concurrency features (and therefore should always follow the safety rules) vs. those that existed prior to the introduction to concurrency yet need to work safely with it. For example, consider the DispatchQueue.async method mentioned earlier. It looks something like this today:

// Existing, Swift 5 definition
extension DispatchQueue {
  func async(execute work: @escaping () -> Void)
}

In Swift 6, we need its closure parameter to be @Sendable :

// Correct Swift 6 definition
extension DispatchQueue {
  func async(execute work: @Sendable @escaping () -> Void)
}

There are a few options for dealing with this:

  • Deprecate or remove the API in Swift 6, which will potentially cause excess code churn
  • Change the API in Swift 6 and have Swift 5 ignore @Sendable closure requirements, which would make Swift 5 code using concurrency less safe than it could be (e.g., we’d allow actor-isolated closures to be passed to this function)
  • Annotate the API to state that it used to be non- @Sendable but should be @Sendable in Swift 6.

The Swift compiler has a few hidden attributes to implement the last of these options. We’ll likely need to augment or replace these attributes with a more holistic scheme. For this API, we use the parameter attribute @_unsafeSendable :

extension DispatchQueue {
  func async(@_unsafeSendable execute work: @escaping () -> Void)
}

The @_unsafeSendable parameter attribute says that the parameter’s type will be treated as @Sendable in Swift 6 and within a context in Swift 5 code that has adopted concurrency, but will be non- @Sendable everywhere else. Unlike @Sendable , it also has no ABI impact, so an existing API can be augmented with this attribute without breaking either source compatibility or ABI for existing code.

Similarly, declarations can be marked with the “unsafe” form of a global actor, meaning that they should be considered to be a part of the global actor, but that should only be enforced in Swift 6 code or Swift 5 code that has adopted concurrency. For example:

@MainActor(unsafe) func renderUI() { ... }

This function will be treated as @MainActor when in code that has adopted concurrency or is compiled in Swift 6 mode, and unspecified elsewhere. In practice, this affects the type of the function:

let fn1 = renderUI // type is () -> Void in Swift 5, @MainActor () -> Void in Swift 6

func newCode() async {
  let fn2 = renderUI // type is @MainActor () -> Void everywhere
}

Global actors and @Sendable are the places where we have found that we needed the “unsafe” attribute variants to allow for evolving Swift toward Swift 6. There are likely more such places, and we’ll want to develop an overall approach that developers can understand and embrace to make for a smooth migration from Swift 5 to Swift 6, providing incremental benefits for incremental adoption of concurrency features.

Doug

64 Likes

Would it be valuable to differentiate truly unsafe usage from versioned usage? @MainActor(versioned) would make it clear that in Swift 6 it's fully safe, but in 5.x it may not be enforced. That also lets use keep the unsafe versions for truly unsafe usage, which may be necessary even in pure Swift 6 mode.

8 Likes

Yes, and "versioned" much better describes what this is actually for.

Doug

8 Likes

And just to clarify, my understanding is that the concurrency features announced at WWDC21 are part of Swift 5.5 with Swift 6 being a release beyond that.

Is that correct?

That’s correct. The upcoming release is Swift 5.5 and has all of the concurrency features we’ve been discussing. We don’t have a timeline for Swift 6 yet.

Doug

9 Likes

Hi Doug, thank you for this post, it’s very useful for planning ahead.

However, this opens up a question for me, although I'm not sure if today anyone has the answer. If you plan to deploy the safe versions in Swift 6... would that mean that anyone that in the future cannot update to Swift 6 if they plan to support iOS 14?

3 Likes

Can you say anything about how this might look, i.e. what new features or language changes it might depend upon?

At this point, there isn't much in the way of new language features we need to do here. Rather, it's a matter of rolling out Sendable checking throughout the ecosystem (something we'll have to do incrementally, and enabling more dynamic checks to deal with places where code that is outside the model (C code, Swift 5 code that hasn't enabled checking, etc.) could introduce problems.

Now, I'm not completely happy with how Sendable interacts with some parts of the language (a few of those issues are mentioned here), but the general model is sound.

The major missing piece for the language is that we currently don't have anything that bans access to mutable module-scope or static variables. I suspect that's a small change (in terms of language design), and the implementation has some latent code to outright ban it, but nobody has come up with a proposal thus far.

Doug

12 Likes

We have a large codebase which we try to migrate to the swift concurrency, under the pressure that when Swift 6 comes out, our project will be uncompilable due to incomplete compliance to Sendable protocol. We discovered typically:

  1. When we make a small change, say conform a type to Sendable, compiler generates many warnings, that we have to fix, by fixing one warning, more warnings are generated.
  2. It's tricky to handle 3rd party libraries, some of which are released in binary format, which is even more challenging to workaround compiler warnings related to those libraries.
  3. There is no way we can tell when all of the 3rd party libraries we use will be fully migrated to swift concurrency for Swift 6. There is constant risk/concern that even one single 3rd party library could break the compiling of the project.

My impression regarding the Sendable protocol and its implications. I think it's not very developer-friendly, considering that it solves just a small issue (race condition) among so many problems in software engineering, yet it requires so much effort to do the migration on existing code base. It's an order of magnitude of efforts compared to migrations between earlier Swift versions.

1 Like

There are still changes coming to the compiler's Sendable checking and Apple's own frameworks that will affect this. So while this is interesting for new projects, existing projects really shouldn't transition now, aside from maybe making new code Sendable-safe. As you've noted, there's also no tooling in Xcode to automate any of this process or otherwise help with the transition, which I don't expect until we have an actual timeline for Swift 6. Personally I don't expect it until WWDC 2024 at the earliest.

6 Likes

To add to this, the Swift 6 compiler will have Swift 5 language modes, so any code that shouldn't compile under Swift 6 will still compile under Swift 5.

Additionally, if you're conforming a type to Sendable and getting a ton of warnings (which we're currently going through in Vapor at the moment), start from the very low-level types and work up rather than the other way around. That way you won't get loads of warnings

4 Likes

This is definitely an area where the compiler or Xcode could really help. Rather than producing warning or error messages with simply the next type down the chain that doesn't conform the Sendable (or really any recursive protocol like Equatable or Washable as well), it would be nice if the compiler could give us the full chain of non conforming types we'd need to update to get the type we're looking at to conform. If that's too much work (perhaps a max depth), it would still be better to show more information rather than the single layer we get now.

3 Likes