[Announcement] Planning for Swift Atomics v1.1

Swift Atomics 1.0 shipped 19 months ago -- it's long overdue for a new minor release.

I'm preparing a Swift Atomics 1.1 release that will bring this package up to date with recent developments. I'm currently planning to ship the following changes:

  1. The minimum required toolchain version will increase from Swift 5.1 to Swift 5.6. This lets us get rid of a large list of workarounds for long-since resolved compiler bugs, to simplify the package and prepare it for the noncopyable future.

    (Note: This will not disrupt existing projects that depend on swift-atomics: the package manager will only offer the new update to clients that are actually able to build it. Clients that are stuck on older toolchains will continue to see the existing 1.0 releases; they don't need to do any manual action to avoid the new release.)

  2. The Swift Runtime has been requiring double-wide atomics since Swift ~5.5, so the toolchain bump will let Swift Atomics unconditionally ship the AtomicReference construct on all supported platforms. This change will finally enable widespread use of atomic strong references in Swift code.

    Atomic strong references are a crucial tool for (easily) avoiding memory reclamation issues in concurrent data structure work. Having this unconditionally available opens the door for some fundamentally new directions where Swift was never able to venture before. (E.g., it's difficult to overstate how much a Swift-native concurrent dictionary type would benefit the systems programming use case.)

  3. ManagedAtomic will finally become conditionally Sendable when its Value is Sendable. This has proved to be a constant pain point for users, especially when first trying out the package, or in the simplest production use cases. (That said, I do hope that folks will not start passing around naked atomic values in any case that is more complicated than a simple atomic counter.)

  4. Swift Atomics 1.1 will resolve the Swift 5.7+ type checker warning when a non-final class conforms to AtomicReference.

    class Base: AtomicReference {}
    // warning: Non-final class 'Base' cannot safely conform to protocol
    //    'AtomicValue', which requires that 'Self.AtomicRepresentation.Value'
    //    is exactly equal to 'Self'; this is an error in Swift 6
    

    The warning complains about a real type safety issue when subclasses of such types are used as the generic argument of ManagedAtomic or UnsafeAtomic. Swift Atomics 1.1, we'll change things so the declaration above will compile without warning, but the code below will produce an error:

    class Derived: Base {} // OK
    
    let ref1: ManagedAtomic<Base> = .init(Derived()) // OK
    
    let ref2: ManagedAtomic<Derived> = .init(Derived())
    // 1.0:
    //    no error, silent type safety violation
    // 1.1+: 
    //    error: 'ManagedAtomic' requires the types 'Derived' 
    //    and 'Base' be equivalent
    

    This can technically be considered source breaking, as such code did build in Swift Atomics 1.0. However, it was always an undiagnosed type safety violation, so this was never actually valid Swift code. This, in addition to AtomicReference not being available on Linux, gives me some confidence that this change will not cause problems. (Please do reach out if you know it'll be a problem for you!)

I encourage interested parties to try building the package from the main branch before the 1.1.0 tag ships, and report any problems as soon as possible, to prevent disruption. (Note: some of the changes above have not landed on main yet. I'll follow up in this thread when we have a release candidate that is ready for testing.)

On noncopyable types

Non-copiable types are quickly approaching, and they are an important step towards modeling synchronization primitives such as atomics as Swift native constructs.

Swift Atomics 1.1 does some cleanup work in preparation for that, but it will not introduce better replacements for UnsafeAtomic/ManagedAtomic just yet, because no shipping Swift compiler supports move-only types today.

Additionally, SE-0390 will also not be nearly enough on its own -- we will need a considerable amount of additional language work to actually support atomics, and that may or may not happen in time for 5.9. (This followup work is yet to be designed in detail. It has not yet been pitched on S-E, but I expect it will be adapting what we learned from this thread: Exposing the Memory Locations of Class Instance Variables)

I expect Swift Atomics 1.2 will introduce Atomic<Value> soon after the language matures enough to support it, if and when that happens. (I also expect the Standard Library to start providing the same or (similar) construct at that point, eventually replacing the need for this package altogether.)

30 Likes

this is great news! i’ve tested locally and opened a PR on one of my projects (swift-mongodb) and confirmed that i can build the project using the official swift-atomics package on linux. on top of that, i managed to delete 1,063 lines of AtomicReference workarounds. wow!

5 Likes

Reads like nice progress is being made. Thanks for working on this!

This bit:

“That said, I do hope that folks will not start passing around naked atomic values in any case that is more complicated than a simple atomic counter.)”

Made me wonder: “Why do you hope this? What situation(s) tempt(s) people to do this? What are better alternatives?”… etc.

(Disclaimer: I have not looked at Swift Atomics at all yet, have been away from programming Swift for 7 months, and am not running at full speed currently, so it might just be me)

3 Likes

Atomic values are fundamental to managing concurrency, but they are far too low level to be used lightly, or to be passed around between components as if they were harmless little integers. These things are scary. They bite. They leave scars that never heal.

The best way to deal with atomics is to avoid using them them at all.

If that doesn’t work, the next best option is to strictly encapsulate them as implementation details in a higher-level synchronization/concurrency construct that is actually suitable for use without wearing a hazmat suit. (A concurrent dictionary for caching stuff would be a great example of this — the Swift runtime is chock full of those. Reflection and Observables are two topics randomly grabbed from recent evolution headlines that could really use a Swift-native implementations for that. Other examples would be concurrent FIFO queues, or similar list-like things.)

Atomics enable brave folks with sad, but steely eyes to start writing those things directly in Swift, rather than shamefully calling down to a legacy language, such as C++.

Atomics, locks, and similar synchronization primitives will breath life into the (presently) hollow promise that is @unchecked Sendable.

The Sendable conformance is largely irrelevant in this context — because the point of these things is to allow people to do the gruesome work of manually implementing Sendable conformances on things other than plain old bags of values.

However, sometimes all we need is a simple, unsophisticated atomic counter, and for that, ManagedAtomic conforming to Sendable is going to come really useful. (As long as we keep in mind that these things will wreak havoc at the slightest provocation. A function that takes or returns an atomic value, or a closure that captures one will always have a vaguely revolting smell about it.)

24 Likes

Very clearly stated. Thank you.
This seems like very useful context/perspective to include in the documentation for SwiftAtomics, if it’s not already (I apologize for not having read them yet - life went sideways on me).

Thank you again for working on this.

2 Likes

I don't have substantive feedback on this plan other than that from the perspective of SwiftNIO this all sounds fantastic. We're very happily using swift-atomics, and none of the roadmap items here cause us any concern whatsoever, and will be great for the ecosystem as a whole.

We're envisioning dropping 5.5 support in line with the Swift 5.8 release anyway, so as a practical matter raising the minimum version to 5.6 will have approximately zero impact on us or our users going forward.

6 Likes

(Note: some of the changes above have not landed on main yet. I'll follow up in this thread when we have a release candidate that is ready for testing.)

All changes that we currently plan to ship in 1.1.0 have now landed on the main branch.

If you have a project that uses Swift Atomics, this would be a good time to try building it with the version specification temporarily replaced with one that brings in the latest commit from the main branch. Please report if you run into any issues!

I have not yet gone through the full pre-release test matrix, so there may be unexpected issues on some platforms and/or some of the supported toolchain versions. My own testing will likely catch those soon, but reports of such problems would still be very welcome! (And they will be rewarded with speedy fixes.)

Beyond the changes discussed above, the main branch also contains the following three additional new features, also intended to ship in 1.1:

  1. It is now possible for RawRepresentable types to opt into atomic support for Optionals that wrap them, as long as their raw value also supports optional atomics:

    struct SomeValue: RawRepresentable, AtomicOptionalWrappable {
      var rawValue: UnsafeRawPointer
    
      init(rawValue: UnsafeRawPointer) {
        self.rawValue = rawValue
      }
    }
    
    let atomic: ManagedAtomic<SomeValue> // OK
    let atomic2: ManagedAtomic<SomeValue?> // Also OK!
    
  2. weakCompareExchange now comes with a variant that takes just one memory ordering, bringing it on equal footing with compareExchange, that has had one from the start:

    let atomic = ManagedAtomic<Int>(42)
    
    var original = 0
    var exchanged = false
    repeat {
      (exchanged, original) = atomic.weakCompareExchange(
        expected: original,
        desired: 23,
        ordering: .relaxed) // no need to specify a separate failureOrdering!
    } while !exchanged
    
  3. The code base now supports an experimental build configuration where C atomics are replaced by direct use of Swift compiler builtins. This will ship in 1.1, but it will not be possible to enable it without editing the package manifest. It is primarily intended for people who vendor the package into projects that use a different build system. This functionality isn't stable: it is provided as a courtesy, but it may disappear or it may in fact become the default in future versions of the package.

18 Likes

The current candidate commit for the 1.1 release is 6c89474e.

Prerelease testing is in progress, and it is expected to complete soon.

5 Likes

Swift Atomics 1.1 is out: Release Swift Atomics 1.1.0 · apple/swift-atomics · GitHub

13 Likes

hooray!!!

Seconded! hooray!!