[Pitch #2] Structured Concurrency

I posted my thoughts in the other thread about @Lantua’s suggestions re. task control (specifically cancellation). TL;DR: I don’t believe we need Task.Handle or “all” Task functionality in the continuation APIs: those serve to control a task form the outside, but here we’re “inside” a task. Specifically for cancellation, I think we need isTaskCancelled and to be able to install a taskCancellationHandler.

There is a huge amount of text about this topic, but I didn't see a discussion about a detail that really does not feel right for me:
enum Task { }.

Firstly, this seems to be a workaround for the lack of namespaces in Swift, which by itself imho already warrants some reflection. Do we really want to establish enums that aren't actual enums, just to save a new keyword which would convey intend clearly?

There is a big precedent with Combine, but whereas it is quite clear that Publishers is not a real type, Task is not plural — and it wouldn't be that uncommon to have instances of task-objects (which might have a role similar to the proposed Task.Handle).

I didn't waste time for a complete alternative design, but it would probably resemble Apple Developer Documentation.

4 Likes

What we've generally talked about here is creating a task from a closure such as { @MainActor in ... }, although I presume we could have a more narrow syntax.

This effectively means there's no concurrency, or no concurrency by default, if you're starting from a context that has an executor. That seems to undermine the point of async let, which is to initiate concurrent work that you'll ask for later.

I understand the first part about async let being too subtle for introducing concurrent work, even if I don't necessarily agree.

The second part I don't really understand, though, at least not on its own: any asynchronous function has blocks of synchronous code with asynchronous calls in between. Maybe it has no asynchronous calls. That's fine, and it's part of the reason that you can implicitly convert a synchronous function into an async function, which was deemed important in the async/await proposal (we added that in response to pitch feedback). Forcing the concurrent task produced by an async let to have an await doesn't really fit with that design, and as a restriction I don't think it's valuable... except if what you want out of async let is something that is not generally going to run concurrently.

Doug

2 Likes

FWIW, this is already established in the standard library, e.g., in CommandLine. There are other cases, IIRC.

Doug

This has been bothering me a lot with the Task design. Continuation is but one example where we are inside synchronous code but we might want to check whether the task we're running in has been cancelled. Other cases might be, e.g., some long-running synchronous operation where we might want to be able to check for cancellation:

func updateRecords(_ records: [Record], isCancelled: () -> Bool) {
  for record in records {
    if isCancelled() { return }
    record.update(...)
  }
}

That's just not possible in the design here, because you can only check cancellation if you have a Task.Handle (you won't for child tasks in the design we have) or you are in asynchronous code. That seems like a big hole to me.

We're also spending a lot of time trying to figure out how to call async functions without await because Task has a whole bunch of static members on it that are async because we are trying not to have instances of Task running around.

What if instead we embrace the notion that we can have Task instances to query tasks? That it would be okay to store them, pass them down to synchronous code, share them, etc. You would be able to get the the task instance from the currently-running asynchronous function:

struct Task {
  static func current() async -> Task
}

but that would effectively be the only "instantaneous" async operation one would deal with. Checking for and triggering cancellation would be synchronous instance members:

extension Task {
  var isCancelled: Bool { get }
  mutating func cancel() { }
}

and we could make them useful value types with Equatable and Hashable conformances. One would be able to get the Task out of a Task.Handle, e.g.,

extension Task.Handle {
  var task: Task { get }
}

Now, APIs that can only make sense on the current task because they need to maintain structure--things like Task.withGroup and Task.withCancellationHandler--would stay as static async methods. However, by doing it this way, we much more strongly communicate that static async very strongly implies things that can only logically be performed on the current async task.

I'd like to turn this into a real sketch of the Task API, but all of the discussion here has changed my mind on Task as a type that has instances, and I wanted to get the basic ideas out there first.

Doug

6 Likes

Dear authors, I understand that some proposed API has precedent in UIKit or / and Dispatch, however I disagree that we have to re-use the same outdated / domain specific naming scheme for Task.Priority cases!? I don‘t have a better set of names to suggest, but I strongly feel that a modern and general purpose API such as the Task API should not use a naming scheme borrowed from the UI domain. For example I don‘t think it would fit server based applications.

9 Likes

Absolutely agreed we need a way to support these.

But I don't fully agree that "Task can be a value" solves this automatically all the way. We cannot just rely on "please pass the Task object to sync functions" since that only works if one controls the full invocation chain to be able to pass such Task value, and often one may not be able to do this), more on this below.

So the TL;DR; is that all comes back to the need of static func unsafeCurrent() -> UnsafeCurrentTask?, but let's dive in and explore why.


To be able to implement the simple read operations on tasks we have two options:

  • implement as async functions and thus be guaranteed to have a Task
  • implement as sync functions, using unsafeCurrent() and have good default values for the read values

What I mean by the second point is, we could implement isCanceled as follows:

extension Task { 
  static var isCanceled: Bool { // NOT ASYNC!
    Task.unsafeCurrent()?.isCanceled ?? false
  }
}

Where unsafeCurrent() is func unsafeCurrent() -> UnsafeCurrentTask? which contains all read and write operations, but one must be super careful to never invoke those from any other task than that current task (thus the Unsafe prefix; I believe @John_McCall was okey with this spelling of it).

So this impl makes a lot of sense for isCanceled specifically: if we're not in a task, there's no cancellation to be set, so we should never react to it, so the default is to return false. Thanks to this rather than static func isCanceled() async -> Bool we could express it as static var isCanceled: Bool and implement it correctly.

Deadlines in the future can also be implemented like this would be "infinite" if no task is available, so we'd be able to keep that API consistent in the future:

extension Task { 
  static var deadline: Task.Deadline { 
    Task.unsafeCurrent()?.deadline ?? Task.Deadline.never
  }
}

Thus usages of that specific call become non awaiting and usable from non-async functions:

if Task.isCanceled { ... } 

The same about checkCancellation():

extension Task { 
  static func checkCancellation() throws { 
    if self.isCanceled { throw ... } 
  }
}

If we wanted to also offer the task as a value, this is fine, and we can offer:

extension Task { 
  var isCanceled: Bool { ... }
}

In general I think it's nice to allow current but I don't think it necessarily means removing the static functions to be honest... Specifically since the number of APIs that are okey as member functions on Task are less than the ones that work on the current task implicitly (analysis below).

All discussions about something in a async functions all rely on the existence of unsafeCurrent, which relies on the ABI of setting tasks into a thread-local by all swift operations which I believe is our plan anyway right?

Long story short, to properly solve this, we'll need unsafeCurrent IMHO, and it also solves task local values, cancelation, deadlines and general access to a task from a non-async function. Coming back to your original example, it'll look like this:

// func updateRecords(_ records: [Record], isCancelled: () -> Bool) {
//  for record in records {
//    if isCancelled() { return }
//    record.update(...)
//  }
// }

func updateRecords(_ records: [Record]) {
  for record in records {
    if Task.isCanceled() { return }
    record.update(...)
  }
}

Okey let's talk about the Task as value then... :slight_smile:

I think that's fine but it is not a complete answer to the problem at hand -- saying that out loud right away, just to not get caught up that "that's it" :wink: At least, not without the unsafeCurrent() version available as well.

I want to confirm what you mean here. Is it:

  • (1) to apply compiler magic that Task.current() needs no await
    • rather than introduce the @instantaneous allowed to be put on random functions
  • or (2) that thanks to moving things to instance functions, it becomes less terrible to look at,
    • and the Task can be passed to some sync function, and that sync function can then also check cancellation on the task via the instance function.

(1) helps a bit... since indeed we can then write:

if Task.current().isCancelled { ... } // (1) because "current" is magical / instant

but (2) does not really solve things; So I want to confirm if we mean (1) or (2) here.

Specifically, (2) breaks down if something is being called "through frameworks" and those happen to be a sync function; And I'd want to be able to check if I'm cancelled or not, but the framework invoked me through a sync function... So I'm stuck, and I cannot fix that other framework because I don't own it.

For completeness, option (3) is the introduction of the attribute and have it available to others...

  • (3) introduce @instantaneous (or other spelling) and allow async functions to implement this.

Alternatively...

Consider basing those APIs a lot around synchronous and unsafeCurrent based implementations, just as I've shown above with the isCanceled. I think we can do great with this and propose a complete set of functions for the API a little bit below.


Okey, so by this reasoning a task local's bind operation withLocal is the same category of function -- as it needs to be on a task, and it needs to maintain structure, so it remains: static func withLocal(...).

However, a task local's read operation local(\.key) can be successfully implemented as not async if we had Task.unsafeCurrent. This is because we can return the TaskLocalKey.defaultValue() if no task is available, or a task is available but the value is not set.

So by inspecting this rule...

"APIs that can only make sense on the current task because they need to maintain structure [...] would stay as static async methods"

So we'd arrive at:

extension Task { 
 static func withLocal<Key, BodyResult>(
   _ key: KeyPath<TaskLocalValues, Key>,
   boundTo value: Key.Value,
   body: @escaping () async -> BodyResult
 ) async -> BodyResult { ... } // `reasync` even, if it existed
}
extension Task { 
 static func local<Key>(_ keyPath: KeyPath<TaskLocalValues, Key>)
   -> Key.Value where Key: TaskLocalKey { ... } 

Note that it is not async, similar to isCanceled because we can implement it correctly with Task.unsafeCurrent() -- we return Key.defaultValue() if either no task is available, or the key is not bound.

Both remain static... I think this abides to the rule you just formulated there, because they really only must be invoked on "current", and may never be as instance functions on task from another task, because of how they strongly depend on the structure of task lifetimes (and we don't want synchronization on those accesses either).


Okey... so summing up all those thoughts–and riding on the existence of unsafeCurrent–I think we could provide a great consistent API that ends up like this, @Douglas_Gregor:

Task :: static async functions

Getting the task instance

  • static func current() async -> Task
    • Would this one then get the magic @instantaneous logic? If yes then I'm on board with this (and getting unsafeCurrent)

Task local value binding

  • static func withLocal(_:boundTo:body:) async rethrows -> BodyResult

Creating tasks

  • static func withGroup(...) async (throws) -> GroupResult

Cancellation handlers since these strongly relate to structure of tasks too

  • static func withCancellationHandler(handler:operation:) async rethrows

Voluntary Suspension

  • statif func yield() async
  • any other "wait 1 second" or similar if we wanted those

Task :: static sync functions

Getting unsafe task, must not be shared nor accessed from any other task; returns nil if no task in current scope (e.g. we're called from some random pthread)

  • static var unsafeCurrent: UnsafeCurrentTask { get }

Querying cancelation, regardless if in a task or not

  • static var isCanceled: Bool { get }
    • implementation as explained above
  • static func checkCancellation() throws
  • :question: static var priority: Task.Priority? { Task.unsafeCurrent?.priority }
    • this one is a bit weird, since we do not have a good default to use for when not in a Task I think... so we can either:
      • not have this function at all as a static, and only have it on instances
      • have this static as well, but it is optional here...
    • I'd be happy with not having this at all.

Note that these isCanceled static versions are NOT async and this is on purpose to allow checking from synchronous loops after all.

Creating detached tasks

  • static func runDetached(...) -> Task.Handle

Task local reading is synchronous, and returns default value if key not bound, or no task is available

  • func local(_:) -> Key.Value // can return Key.defaultValue

Task :: instance functions

  • var isCanceled: Bool { get }
  • func checkCancellation() throws
  • var priority: Task.Priority { get }
  • in the future also deadline here fits well.

Those are fine and possible to implement by access also from other tasks so they can be on the instance -- task locals cannot, so they remain as static.

Task locals MUST NOT be accessible through this API, as a Task could be passed around and accessed by other tasks, that's not safe.

UnsafeCurrentTask :: instance functions

This must only be invoked from the current task, passing it around, storing it, or any access from another task or thread is illegal and may crash hard.

  • var isCanceled: Bool { get }
  • func checkCancellation() throws
  • var priority: Task.Priority { get }
  • in the future also deadline here fits well.
  • func local(_:) -> Key.Value -- this allows for task locals outside of async functions, excellent.

This seems pretty good to me... but all these API shapes ride on the existence of unsafeCurrent... hopefully we could get that? as the sync/async and static/instance placement of functions look pretty good (assuming the above outline is what you had in mind).

Thanks for digging into this Doug,

// PS: Yeah I'm spelling it as isCanceled since we got some feedback for consistence that we should move to this... pending a PR to adjust it should happen soon I guess. Long story, but we reached out to confirm what spellings APIs should use.

PPS: I'm not sure if it should be static func isCanceled() -> Bool or static var isCanceled: Bool -- either way it is O(1) however (!) it does imply hitting a thread-local to get access to the Task... so it's a bit heavier... Perhaps it makes sense to keep those as ()?

I saw this restriction a few times, but don't quite get how it came to be. Is it because there's actually no task instance stored inside UnsafeCurrentTask? How is task stored/identified/accessed at runtime?

That’s more of a question for the task locals pitch semantics, but let’s quickly address it...

It isn’t about the task, it is about the specific fields and their storage.

As I mentioned, reading canceled status is safe since it’s just an atomic load of an atomic status integer in the task. Other more complex operations on the task may not be safe — that’s it.

Access to task-locals outside of the task is racy semantically (if it is executing, and entering/exiting withLocal blocks, those are mutating the bound values stack — so it is racy what value you’d observe from outside of the task), as well as “weird” (why the hell would you do that) so we don’t allow for it.

Sure, it can be made thread-safe (by using a treiber stack (simple lock free stack)), but we don’t have to since logically we don’t want to allow external access anyway, so why would we. It does not make sense to read them outside the task from arbitrary other tasks — so we can avoid synchronization there, making accesses to the locals cheaper. Perhaps we’ll do the lockfree stack anyway, but it does not make sense to me to allow “arbitrary random task can read your task locals” and we should not do this.

For the implementation on how it’s stored, you can refer to the task locals PR which contains a full implementation along with many docs explaining it.

1 Like

I also wanted to highlight another reason why I thought in the above snippet that the APIs are mirrored in the static and instance.

I may be wrong here... Let me know if my concern is moot, because we would @inline(__always_) static func current() and that would actually be enough to help the compiler optimize this even cross module? (i.e. I’m not sure about TLB cost can be optimized away well or not — do you know @Douglas_Gregor? My prior experience on the JVM and reading up some Intel manuals had me worried here a bit.)

I think it may be beneficial to have the APIs mirrored on the Task instance and statically, because it means we can avoid hitting the thread-local in which a task would be stored in tight loops, e.g.:

func a() async { 
  while true { 
    if Task.isCanceled { return }
  }
}

would need to:

  • read the task thread local (which assumes ABI will be such that the task is in a thread local),
  • read the atomic status and check cancellation

Since we’re always going to be “back” on the same task here, it would make sense to:


func a() async { 
  let task = /* await? */ Task.current()
  while true { 
    if task.isCanceled { return }
  }
}

which only does the atomic op, without having to go through the thread local...

So... Thread locals can be very fast, but if they’re accessed super frequently like cancellation in tight loops might, it can become an issue (reference, at least for intel chips), thus I think the fact that isCanceled would exist as static func and instance func both — one for rare use, and the secondary for “if in tight loop, get the task once and then reuse it to check cancelation” for small performance optimizations like that.

Same as the usual advice:

You may ask if the function “get_value” cannot be inlined for whatever reason, is it possible to reduce the cost of accessing a thread-local variable? The answer is “yes”. Since in this example, the thread-local variable is read-only, you can assign the thread-local variable to a local variable outside the “for” loop, and then use the local variable inside the loop, as shown below. [from above intel docs link]

Tho at the same time there’s

TL;DR (linux writeup)

If you have statically linked code, you can use a thread local variable like another one: there is only one TLS block and its entire data is known at link-time, which allows the pre-calculation of all offsets.

If you are using dynamic modules , there will be a lookup each time you access the thread local variable, but you have some options in order to avoid it in speed-critical code.

So... given Swift and it’s compilation model, I’m not super sure if I should be worried or not — does force inlining solve the concern or not? :slight_smile:

It’s been a while since I read it, but my understanding of the task-local storage pitch was that semantically, task-locals are immutable and each withLocal block is a new nested task. If this is the case, observing the locals of another task would not be semantically racy, and the problems you describe would only arise in observing the state of an executor.

That would be too expensive and the core team suggested we implement it in a different way — and I agree, and it is implemented more efficiently already, by not creating new tasks (which are much heavier than just adding 1 pointer to a stack).

Regardless if it can be made work even at high performance costs (which imho disqualify such solution anyway, task locals MUST be really fast), it is semantically weird and nonsensical to allow other tasks access another tasks local state. That’s the meaning of task local state — no-one else may access it.

I had a comment and a question about using variables declared with async let.

I understand why try and await are needed when the variables are used in the example code since they are potential suspension points.

func makeDinner() async throws -> Meal {
  async let veggies = chopVegetables()
  async let meat = marinateMeat()
  async let oven = preheatOven(temperature: 350)

  let dish = Dish(ingredients: await [try veggies, meat])
  return try await oven.cook(dish, duration: .hours(3))
}

But to my eye, this has the potential of cluttering the call site. In the provided example there is only one argument, but if multiple arguments were async let the readability of the call site would suffer due to the number of try and await keywords.

At least for me, the clarity of what is happening in the call becomes diluted by all the information of what is being waiting for and whether those things can throw:

let dish = Dish(ingredients: await [try veggies, meat], cookware: await pan, secretIngredient: try await sauce)

From the Detailed design section about async let it appears that you can explicitly await using code like this:

async let veggies = chopVegetables()

try await veggies

My question is, once you have awaited for the value in this way, are the try and await keywords required for subsequent usage, since the value is already returned without error?

Would the following be valid code?

func makeDinner() async throws -> Meal {
   async let veggies = chopVegetables()
    async let meat = marinateMeat()
    async let oven = preheatOven(temperature: 350)
   
    await meat
    try await veggies

    let dish = Dish(ingredients: [veggies, meat])

    try await oven
    return oven.cook(dish, duration: .hours(3))
}

If so, then the call sites could be simplified if a developer so desired.

The more general question would be once an async let variable has been awaited on in any fashion, do subsequent uses require the await or try await keywords?

I may be reading something into the proposal that isn't there, since this doesn't seem to be explicitly mentioned.

2 Likes

I disagree. Very often asynchronous work is I/O bound, and what you want to do is initiate the work (start the I/O) and then do something else (perhaps starting additional asynchronous I/O) before waiting for the results. The concurrency comes from the fact that the "work" being performed happens outside of your process (perhaps even outside of the CPU entirely).

The current design means that if you're initiating that kind of work on a single-threaded executor then you are forced to either wait for each result immediately or forced to use more complex syntax (which as far as I can tell has not even yet been proposed). That makes this syntax both harder to use and harder to learn.

I believe that I/O-bound asynchronous APIs are and should be more common than CPU-bound asynchronous APIs, which means this pattern of starting working that doesn't actually use any CPU time is going to be common. It should be easy to do that from the main thread. That, in my mind, is one of the huge benefits of async/await. Simplifying the complexity of UI code that starts and waits for asynchronous results is a huge win for async/await, but this design makes that win much smaller.

That's not what I said, though. Here's an example to illustrate:

func fAsync() async {...}
func fSync() {...}

func caller() async {
    async let t1 = fAsync(); // okay
    async let t2 = fSync(); // should be an error
}

None of that has to do with whether you used await in fAsync or on the same line as async let. The difference is that the expression in the first line could have await instead, whereas the expression in the second line could not. So if the expression itself has no asynchronous call (therefore it could not contain an await) then the assignment to an async let variable should be disallowed.

My argument is that the async let syntax should be reserved for calls to async functions, and that it shouldn't change in any way how that async function gets called (i.e., which executor it uses or which thread context it is initially called on). It should only allow for separation of the initial call from the suspension point. It should not be used to introduce multithreading. If you want to run some synchronous CPU-bound function concurrently then you should have to use an API for that. That API may use an async function to allow you to await it, but it should be an API or at least a syntax that has a clear scope.

In my experience in both using async/await with UI code in C# and in helping other people to learn how to use async/await in C#, I believe strongly that blurring the lines between "waiting for asynchronous results" and "introducing multithreading" is going to make the feature both harder to use and harder to learn. It makes it difficult for people to form a clear mental model of what async/await actually does. It's much easier to understand and use when it consistently means "a new syntax for waiting for the results of asynchronous results" and not "a new way to implement multithreaded code". We have a strong need for the former, but I'm not sure we have much need at all for the latter.

7 Likes

Note though that non-async function are implicitly converted to async. Differentiate the two in this manner would have to be an exception to that rule.

1 Like

If you consider that an exception then we already have an exception in that this would be disallowed:

func f() {...}

func caller() async {
    await f(); // error
}

You could wrap it like this to work around the error:

func caller() async {
    let fWrapped: () -> Void async = f
    await fWrapped() // okay
}

If you did that then the call to fWrapped (and thus f) would complete synchronously in the same stack as caller.

Likewise, if you did the same with async let:

func caller() async {
    let fWrapped: () -> Void async = f
    async let t = fWrapped()
    await t
}

Then the behavior should be the same: the call to fWrapped and thus f should complete synchronously in the same stack as caller.

You can place a single try await at the beginning if there are no async closures involved. It's one of the benefits of not having a Future-like type as building block for asynchronicity:

let dish = try await Dish(ingredients: [veggies, meat], cookware: pan, secretIngredient: sauce)

At first sight it seems a clear and intuitive design. I have a few considerations, though:

  • await meat alone would give a warning of unused value, so you would be forced to use _ = await meat. The same applies for the other try await isolated expressions;
  • you can await a value without realizing it, e.g. by using await print(meat), would that count as awaited too?;
  • if you move await meat on a different line you may need to move the await somewhere else.

That would certainly be less cluttered. The code example in the proposal led me to believe the individual arguments would each need to be annotated. (Why wouldn't the spelling you mention almost always be preferred to interweaving the keywords with the arguments?)

The proposal includes the example:

{
  async
let 
    ok = "ok",
    (yay, nay) = ("yay", throw Nay())
  
  await ok
  try await yay
  // okay
}

which led me to believe that await ok or try await yay in the code example would not cause a warning. The proposal doesn't explicitly say one way or another.

I would imagine so, since it needs to be be present to print it.

I was thinking of await meat sort of like guard let item = optionalItem. If I move a reference to meat or item before the appropriate line, I'd need to move where I await or unwrap.

But for me at the moment, it's less about whether it should or shouldn't work this way, and more about clarifying how these things are currently intended to behave.

If async let is used in a UIActor context, would the spawned task still be run concurrently? Or does this depend on whether the initialiser is also annotated with ‘UIActor`?

Eg.

func load1() async -> String {
  await loadUrl(textField1.text)
}

func load2() async -> String {
  await loadUrl(textField2.text)
}

@asyncHandler
func buttonPressed() async {
  async let res1 = load1()
  async let res2 = load2()
  textField3.text = await “\(res1) \(res2)”
}

I'm not sure if you're asking me or the pitch's authors. My understanding is that as currently written the answer is that it depends on how the function (or class?) is annotated.

My argument is that it should not make a difference. Calling load1 or load2 directly with await or by using async let should make no difference in terms of which call stack is initially used by the call (before the suspension point) or which executor is responsible for the task.

Your example shows a reason why that's so important: your functions both use UI objects and therefore must run on the UI thread. If changing their caller from await to async let changes the thread that they run on (even if the caller remains on the UI thread) then it's trivial to introduce a bug in your application. That would make this feature really painful to use. The default behavior of async/await should as much as possible be to make code like this Just Work. Refactoring to change where the await happens by using async let should not break things.

4 Likes

In other words, does async let support a form of definitive initialization?

(I've been curious about this as well.)

1 Like