Swift Concurrency: Feedback Wanted!

[ktoso (Konrad 'ktoso' Malawski 🐟🏴‍☠️)]

In Swift 5.5 we’re not fully enforcing the Sendable checks, we will enable these in Swift 6 — see here for a detailed writeup:

Concurrency in Swift 5 and 6

Hey all, Swift has always been designed to be safe-by-default. This informs runtime features like bounds checking, integer overflow checking, and automatic reference counting, and type system features like optionals. But this safety has never extended to data races, which are easily created in practice and hard to completely eliminate. An explicit goal of the concurrency effort for Swift 6 is to make safe-by-default also extend to data races. The type system and language model work together, t…

[JJJ (Jens Jakob Jensen)]

@Terje

If the class Foo conforms to Sendable the compiler will emit errors about Foo.value being mutable.

You need @unchecked Sendable to make something Sendable that the compiler doesn't see as obviously true.

[moreindirection]

From the post @ktoso linked to, you want to use the -warn-concurrency compiler option to get the behavior you're looking for.

[Torust (Thomas Roughton)]

A few comments from an afternoon of converting code over:

• TaskGroup (or at least TaskGroup<Void>) could use a wait() function that waits for all elements in the group rather than immediately cancelling.

• The lack of reasync functions for things like withUnsafePointer, withExtendedLifetime, Optional.map, and Sequence operations currently is a real pain point. I would say that pretty much every rethrows function in the standard library or Foundation should also be reasync.

• As far as I'm aware, we don't have a good solution for making async versions of protocols.
  For example, ideally there'd be an AsyncDecodable protocol, which carries the same requirements as Decodable except with an async signature (i.e. init(from decoder: Decoder) async throws). I wonder if that's a common enough need (a variant of an existing protocol whose methods and closure parameters are async when called from an async context) that it deserves special treatment; e.g. async Codable could be a distinct subtype of Codable. I'm not sure how that would scale in terms of code size – from my limited understanding, doubling the number of protocol conformances for each app seems a heavy price to pay.

• As a possible alternative solution for the async protocol issue, it would be very useful to have an escape hatch for a sync function to turn into an async function. I realise that this means there could either be suspension points in non-awaited code or blocking/potentially deadlocked code; however, given we don't have the ability to make all the protocols or closure-taking functions in all the external libraries that we use async there's sometimes no other real option. Hacking around this with a DispatchGroup.wait() doesn't seem ideal, so if a more efficient option is implementable in the runtime then it would be great to add.

• Now that async get vars are allowed, it would be nice to support static lets which are assigned with the result of an async expression.

• If it's not already, Void should be @Sendable. An earlier build rejected Void as e.g. a TaskGroup result type, but I'm not sure if the reason it's allowed now is because checking has been relaxed for Swift 5.5 or because the @Sendable conformance has been added.

• Probably just a bug, but @unchecked Sendable doesn't currently work; you have to use UnsafeSendable instead.

[Lantua]

@Torust

TaskGroup (or at least TaskGroup<Void> ) could use a wait() function that waits for all elements in the group rather than immediately cancelling.

If you exit the task group without throwing, it should await pending child tasks without cancellation.

[Torust (Thomas Roughton)]

@Lantua

If you exit the task group without throwing, it should await pending child tasks without cancellation.

Okay, cool, that makes sense – I probably just got mixed up with the proposed rules for async let and didn’t properly test.

[toph42 (Topher Hickman)]

@Torust

As a possible alternative solution for the async protocol issue, it would be very useful to have an escape hatch for a sync function to turn into an async function.

I’m confused. Can’t you just call synchronous functions inside async {} to do that?

[wowbagger (冀卓疌)]

I think that only works if you don't need whatever inside the async block to complete before the sync function returns.

[sharplet (Adam Sharp)]

I’m using an actor right now primarily for isolation, so none of methods are actually async at all. This makes it really easy to reason about thread safety (no reentrancy), while still providing a nice async interface to make it easier to reason about what code is running in what context, and where the potential suspension points are.

The feedback I have is that because the public API is async in practice, it feels logical to me that a synchronous actor method should be able to fulfill an asynchronous protocol requirement. The way I’m imagining this working is that if you’re ever interacting with a dynamic instance that’s not know to be self, or is erased by the protocol type, you’re forced to call the method asynchronously. But the normal synchronous dispatch still works the same way it does today.

What do you think about this as a potential enhancement? Or is the lack of this just a gap in the current implementation? It feels similar to how non-throwing methods can fulfill throwing protocol requirements.

[Jon_Shier]

I just noticed there's no way to create a continuation with a concrete Error type, only Never or Error. Is there time for an amendment to that proposal to add those APIs?

[xwu (Xiaodi Wu)]

@Jon_Shier

I just noticed there's no way to create a continuation with a concrete Error type, only Never or Error . Is there time for an amendment to that proposal to add those APIs?

This was not overlooked during review; apparently, it is the intended design, and the possibility of creating a continuation with a concrete Error type is merely future-proofing "in case" we get typed errors.

@Joe_Groff

@xwu

In the same vein, it seems that the design as-is doesn't really make use of a parameterized error type. It seems like either with*ThrowingContinuation should use a parameterized error type too, or the *Continuation types should not.

This is the issue with including error parameters in these APIs in general; they're more for future proofing in case we add typed errors in the future, even if today you can only really use them with Never and Error . Since we've decided to also include error type parameters in other parts of the API, like Task.Handle and Task.Group , it seems worth it to be consistent here.

[xwu (Xiaodi Wu)]

@sharplet

The feedback I have is that because the public API is async in practice, it feels logical to me that a synchronous actor method should be able to fulfill an asynchronous protocol requirement.

This should already be the case:

Note that an actor can satisfy [an] asynchronous requirement with a synchronous one, in which case the normal notion of asynchronously accessing a synchronous declaration on an actor applies. For example:

protocol Server {
  func send<Message: MessageType>(message: Message) async throws -> Message.Reply
}

actor MyServer : Server {
  func send<Message: MessageType>(message: Message) throws -> Message.Reply { ... }  // okay, asynchronously accessed from clients of the protocol
}

[sharplet (Adam Sharp)]

Thanks! It appears not to be the case yet, here's an example compiler error:

SyncActor/SyncActor/Repository.swift:15:8: error: actor-isolated instance method 'clone(from:)' cannot be used to satisfy a protocol requirement
  func clone(from url: URL) throws {}
       ^
SyncActor/SyncActor/Repository.swift:15:8: note: add '@asyncHandler' to function 'clone(from:)' to create an implicit asynchronous context
  func clone(from url: URL) throws {}
       ^
  @asyncHandler 
SyncActor/SyncActor/Repository.swift:15:8: note: add 'nonisolated' to 'clone(from:)' to make this instance method not isolated to the actor
  func clone(from url: URL) throws {}
       ^
  nonisolated 

Adding @asyncHandler as the fix-it suggests results in the error "'asyncHandler' attribute is only valid when experimental concurrency is enabled". I'll file a bug about it in any case.

[Terje]

Background info: I am trying to make a (core data inspired) data model that leverages the type system as much as possible together with features like resultBuilder.

0302-concurrent-value-and-concurrent-closures says that (literal) key paths are ok if the underlying types are sendable. Yet every kind of key path I try gives the same error:
Stored property '...path' of 'Sendable'-conforming generic struct 'AttributeDescription' has non-sendable type ...
I tried to make the AttributeDescription @unchecked Sendable just for the time being but that gives Unknown attribute 'unchecked'

According to swift-evolution the proposal is accepted but not listed as implemented in Swift 5.5.
However this post does list this proposal and Sendable works in other circumstances.
Is KeyPath special is some way or more likely what did I miss?

Running beta 2 of both Xcode and Monterey with these compiler flags -Xfrontend -enable-experimental-concurrency -warn-concurrency

protocol Entity : Sendable
{
}

protocol Attribute : Sendable
{
}

struct AttributeDescription<E,A> : Sendable where E:Entity, A:Attribute 
{
    let path: KeyPath<E,A>             //preferred
    let paritalPath: PartialKeyPath<E> //half way there
    let anyPath: AnyKeyPath            //if need be
    let literalPath = \Person.name     // not even this ???
}


extension String : Attribute 
{
}

struct Person : Entity
{
    let name: String
}

[jmjauer (Johannes Auer)]

The new concurrency features/api is great.

One thing I still find confusing is the structured concurrency api. It would be great if someone would show some real world examples using this api.

[josephlord]

I think I love the structured concurrency and I've been having a good play (and have largely converted a personal app over already). That said in some experimentation I'm being stymied and I'm not sure if it is understanding (possibly documentation) gaps or bugs in current implementation.

I've been experimenting with creating my own simple AsyncSequences in ways that actually cut across a lot of the structured concurrency features. I know AsyncStream should make this simpler but I wanted to implement manually to see how it really worked (and I also couldn't find AsyncStream so maybe it isn't available yet).

[Edit: I think I've found the issue, something frequently goes wrong when an continuation is resumed from an actor context. I think I have a solution where the actor manages the continuation but returns it to be called from outside the actor instead of calling it directly itself. Raised as: SR-14875]

I think in particular I'm not seeing the reentrancy behaviour I expect. For example this doesn't work (or at least works only occasionally - maybe I'm missing something obvious):

AsyncTimerActorSequence

@available(macOS 12.0, iOS 15.0, *)
public struct AsyncTimerActorSequence : AsyncSequence {
    
    public typealias AsyncIterator = Iterator
    public typealias Element = Void
    
    let interval: TimeInterval
    
    public init(interval: TimeInterval) {
        self.interval = interval
    }
    
    public func makeAsyncIterator() -> Iterator {
        let itr = Iterator()
        Task {
            await itr.start(interval: interval)
        }
        return itr
    }
    
    public actor Iterator : AsyncIteratorProtocol {
        
        private var timer: Timer?
        private var continuation: CheckedContinuation<(), Never>?
        
        fileprivate init() {}
        
        fileprivate func start(interval: TimeInterval) {
            let t = Timer(fire: .now, interval: interval, repeats: true) { [weak self] _ in
                guard let s = self else { return }
                Task.detached {
                    await s.fireContinuation()
                }
            }
            timer = t
            RunLoop.main.add(t, forMode: .default)
        }
        
        private func fireContinuation()  {
            continuation?.resume()
            continuation = nil
        }
        
        public func next() async throws -> ()? {
            await withCheckedContinuation { continuation in
                self.continuation = continuation
            }
            return ()
        }
        
        deinit {
            timer?.invalidate()
        }
    }
}

When I use a class instead it seems to work well although I worry about races setting and firing the continuation which is why I leant on the actor. I thought the reentrancy at suspension points meant this would work.

It might be that there is some information that I'm so far missing, possibly in relation to limitations of reentrancy in Actors or the interaction between Actors and continuations.

"AsyncTimerSequence (class based iterator - works but I worry about races setting/getting/clearing continuation

@available(macOS 12.0, iOS 15.0, *)
public struct AsyncTimerSequence : AsyncSequence {
    
    public typealias AsyncIterator = Iterator
    public typealias Element = Void
    
    let interval: TimeInterval
    
    public init(interval: TimeInterval) {
        self.interval = interval
    }
    
    public func makeAsyncIterator() -> Iterator {
        Iterator(interval: interval)
    }
    
    public final class Iterator : AsyncIteratorProtocol {
        
        private var timer: Timer?
        private var continuation: CheckedContinuation<(), Never>?
        
        init(interval: TimeInterval) {
            let t = Timer(fire: .now, interval: interval, repeats: true) { [weak self] _ in
                if let continuation = self?.continuation {
                    // I worry about a race condition here but I think it is safe if we don't resume the continuation until it has been cleared.
                    self?.continuation = nil
                    continuation.resume()
                }
            }
            timer = t
            RunLoop.main.add(t, forMode: .default)
        }
        
        public func next() async throws -> ()? {
            await withCheckedContinuation { continuation in
                self.continuation = continuation
            }
        }
        
        deinit {
            timer?.invalidate()
        }
    }
}

Note that these versions are only robust to the situation where next() is not called multiple times concurrently (I have a version with an array as a queue - need to look up the state of dequeue) but I don't think I've seen anything in the documentation disallowing concurrent calls to next(). I think it is easy to write sequences which don't handle that properly and it may be a source of bugs. Clearly in the normal for try await let foo in sequence { you won't get concurrent calls but if people start unrolling or async leting a number of items I can see it happening.

An additional albeit temporary complication is differences between builds currently available as the features and syntax evolves and also various compiler flags that affect the concurrency behaviour. Are the 5.5 snapshots the right ones to be using or should I be using the trunk snapshots or whichever happens to be the newest build on a given day? Is there a good way to track what the changes are between snapshots or compared with Xcode releases?

[hfhbd]

It would be nice, if Task.sleep would also accept a non-negative TimeIntervall. Personally, nanoseconds in UInt64 are hard to read.

let duration: TimeIntervall = 2.0
await Task.sleep(duration)
// vs
await Task.sleep(2 * 1_000_000_000)  // Two seconds

Or, with this extension functions:

private extension TimeInterval {
    var seconds: TimeInterval {
        self
    }
    var minutes: TimeInterval {
        seconds * 60
    }
    var hours: TimeInterval {
        minutes * 60
    }
    var days: TimeInterval {
        hours * 24
    }
}

private extension Int {
    var seconds: TimeInterval {
        Double(self)
    }
    var minutes: TimeInterval {
        seconds * 60.0
    }
    var hours: TimeInterval {
        minutes * 60.0
    }
    var days: TimeInterval {
        hours * 24.0
    }
}

await Task.sleep(2.seconds)

In Kotlin, a similar function exists

delay(2.seconds)

// with
suspend fun delay(duration: Duration)
val Int.seconds get() = Duration.seconds(this)

[ktoso (Konrad 'ktoso' Malawski 🐟🏴‍☠️)]

Thanks for the feedback, we aimed to provide such nice “duration” based api but had to defer it out of this release because we want to holistically address the general topic of time and deadline APIs together. Hope we’ll get back to it soon enough Once we have a duration and deadline types we can offer a new sleep that would take such arguments

[David_Smith]

(Also note that days are not uniform lengths, typically they range from 23 to 25 hours long. From past experience, designing duration related APIs that correctly traffic in days is nontrivial and produces results that surprise users of the API)

[technogen (Gor Gyolchanyan)]

Quick question: If Actor types do not support inheritance, then why is final actor legal?