[Pitch #2] Structured Concurrency

[xAlien95 (Stefano De Carolis)]

@michelf

@michelf

Does that mean async let introduces a suspension point too?

I still haven't got an answer to that question. I find it strange that group.add requires an await but async let (which is supposed to be sugar over it) doesn't.

I guess it's explicitly mentioned in the Detailed design section:

Any reference to a variable that was declared in an async let is a potential suspension point, equivalent to a call to an asynchronous function, so it must occur within an await expression. The initializer of the async let is considered to be enclosed by an implicit await expression.

The answer is: await is implicit when using async let and it isn't when using group.add (at the moment at least).

[michelf (Michel Fortin)]

@xAlien95

The answer is: await is implicit when using async let and it isn't when using group.add (at the moment at least).

Note that group.add has two awaits: one before group.add, and one inside the closure:

await group.add { 
   DinnerChild.chopVegetables(await chopVegetables())
}

Both awaits are made implicit by async let.

It's fine to omit the one in the closure since you have to await on it anyway when referencing the variable (as stated in the text you quoted). But the await before group.add would introduce a suspension point on the async let declaration line, which I think will surprise most people when using async let:

async let x = somethingAsync()
// suspension point here
async let y = somethingAsync()
// suspension point here
async let z = somethingAsync()
// suspension point here
...
await (x, y, z)
// suspension point here, obvious because of `await`

It's clever that group.add can introduce back-pressure by suspending the caller in the middle of adding a subtask. But most people won't expect a suspension point to exist on the async let declaration line. The await should be made explicit to make that clear.

And you can't make this explicit by adding an await after the "=" since it'll appear to be part of the expression executed concurrently on the other thread/queue/executor. There is not much choice:

await async let x = somethingAsync()

A bit verbose, but at least now it's clear that the function can be suspended on this line.

[johannesweiss]

@Douglas_Gregor

@johannesweiss

What's the API to query the current executor? Or if there's no API to query it, what does it mean for the function to "know" its executor?

There's no API for this. A function "knowing" its executor means that it can hop back to that executor after (e.g.) calling out to another async function that might change executors (say, because it runs on an actor).

Okay, so in this piece of code

func foo() async -> Void {
    // [A]: running on executor X
    await bar()
    // [B]: which executor am I on now?
}

are you saying that [A] and [B] are always executed on the same executor? My understanding was that this isn't always the case.

@Douglas_Gregor

We haven't exposed the ability to provide a custom executor beyond the enqueue operation of actors . Some day, probably, but I don't expect it will be part of this round of proposals.

Hmm, so how would I implement a thread pool that has N threads (N >= 1 and decided at runtime) giving me N executors?

It's also very surprising to me that the proposal reads "For the most part, programmers should not have to work directly with partial tasks unless they are implementing a custom executor." but it's not actually possible to implement a custom executor from what I understand.

@Douglas_Gregor

I see your point here, but highest , lowest , etc. are almost entirely devoid of meaning. Having the UI metaphor at least gives some sense of when to use the priorities, even if it's not exactly the domain.

I'm very open to different naming suggestions. But how is it obvious for example that utility is a higher priority than background? Do you think that if somebody had asked you, say 10 years ago, to order default, utility,userInteractive, userInitiated, and background by priority you'd have come up with the "right" answer? I would've probably guessed that the user* ones are higher than default and that utility/background are lower but apart from that I don't think I would've necessarily picked the "correct" order.

[Douglas_Gregor]

@Lantua

One more question, it does seem that the task groups (and async let ) cancels all of their subtasks only when they exit the scope by throwing an error . Is there a reason we don't want to cancel if the scope exits normally (by reaching the end or using return )?

You might have created those child tasks for their effects. An earlier draft of the proposal actually had an example of this (create child tasks to go confirm a bunch of orders; there’s no value to return on success); perhaps I should bring it back.

Doug

@Lantua

I don't want the nuance of why the scope ends to differentiate the cancellation behaviour, especially when "error reporting" doesn't limit to throwing errors, but also returning Bool or Optional .

The difference between a return and a thrown error is quite significant in Swift; cancelling on thrown error but not a normal return matches well with the intended use of error handling.

Doug

[IOOI (Lars Sonchocky-Helldorf)]

[Nit]: The link to the discussion in the draft still points to the old discussion [Concurrency] Structured concurrency which is not wrong per se but I think a link to this discussion here should be added.

[Douglas_Gregor]

@johannesweiss

Okay, so in this piece of code

func foo() async -> Void {
    // [A]: running on executor X
    await bar()
    // [B]: which executor am I on now?
}

are you saying that [A] and [B] are always executed on the same executor? My understanding was that this isn't always the case.

I don’t think we want “always” here, no. If foo were within an actor we guarantee that we’d jump back to the actor.

@johannesweiss

Hmm, so how would I implement a thread pool that has N threads (N >= 1 and decided at runtime) giving me N executors?

It's also very surprising to me that the proposal reads "For the most part, programmers should not have to work directly with partial tasks unless they are implementing a custom executor." but it's not actually possible to implement a custom executor from what I understand.

We should strike this sentence until we’re ready to define custom executors. Even if we do custom executors now, it would be a different proposal.

@johannesweiss

I'm very open to different naming suggestions. But how is it obvious for example that utility is a higher priority than background ? Do you think that if somebody had asked you, say 10 years ago, to order default , utility , userInteractive , userInitiated , and background by priority you'd have come up with the "right" answer?

The names don’t have to be perfectly self-evident, but communicating some meaning behind the intent in the range of “a human is waiting on this” vs. “do this when you have idle cycles” seems beneficial. If we’re to only consider the ordering without meaning, then we might as well have an Int rather than an enum.

Doug

[Douglas_Gregor]

@michelf

It's clever that group.add can introduce back-pressure by suspending the caller in the middle of adding a subtask. But most people won't expect a suspension point to exist on the async let declaration line. The await should be made explicit to make that clear.

The intent of the proposal is that the async let does not introduce a suspension point, for the reasons you’ve described. This is a difference in the desugarinf that we forgot to mention in the desugaring example. I’ll update it.

Doug

[Lantua]

@Douglas_Gregor

You might have created those child tasks for their effects.

I see, so the asynchrony (and concurrency) model is meant to be used with effects, not just for calculation. That part of the decision has been eluding me for quite some time.

I still have some uneasy feelings about some actor optimization like combining partial tasks of the same executor or skipping an empty partial task if we're going for effectful programming. Though I'll need to think about it some more (and it's irrelevant here anyway).

@Douglas_Gregor

The difference between a return and a thrown error is quite significant in Swift;

But while significant, throwing errors isn't the only way in Swift to propagate errors. Even TaskGroup.add, like many Cocoa imports, returns Bool to indicate its success/failure. Not to mention failable initializers and other Cocoa imports that return nil to indicate failure.

Plus, opting-out of cancellation is easier, too:

// Opt-out of cancellation
Task.withGroup {
  while await group.get() { }
}

do {
  async let foo = _

  _ = await foo
}

Opting-in for child task requires you to throw a dummy error regardless of the success/failure condition:

// Opt-in for cancellation
Task.withGroup {
  group.cancelAll()
}

try {
  do {
    async let foo = _

    throw CancellationError()
  }
} catch {
  // Might actually succeed, but we want
  // to cancel long-running unused tasks.
}

[johannesweiss]

@Douglas_Gregor

@johannesweiss

are you saying that [A] and [B] are always executed on the same executor? My understanding was that this isn't always the case.

I don’t think we want “always” here, no. If foo were within an actor we guarantee that we’d jump back to the actor.

Ok, thanks. I think this should definitely be spelled out explicitly in the proposal because I think this matters a lot.

@Douglas_Gregor

@johannesweiss

Hmm, so how would I implement a thread pool that has N threads (N >= 1 and decided at runtime) giving me N executors?

It's also very surprising to me that the proposal reads "For the most part, programmers should not have to work directly with partial tasks unless they are implementing a custom executor." but it's not actually possible to implement a custom executor from what I understand.

We should strike this sentence until we’re ready to define custom executors. Even if we do custom executors now, it would be a different proposal.

I think we absolutely need the ability to create custom executors from the get go but I do agree that the sentence should be removed from this proposal. And if we were to actually punt on custom executors, this should be spelled out very explicitly too. For the evolution process to work properly, I do think that we should be as explicit as possible so that people really understand what'll be possible and what not.

Without custom executors, is it correct that I can't make async functions run on a specific OS thread (pthread_t or similar)? Because I think this ability is crucial.

Just like UIKit and friends will need to run the UI code on the main thread, other systems will need to run their code on specific threads too. And I don't think we can get away with mandating 2 thread hops (to and from the specific OS thread) for every async function if its implementation requires going to a specific thread. For server systems that would for example mean sacrificing a lot of performance for async/await which would be more than sad.

Am I understanding correctly that in the current version of the proposal, the UIActor is special and cannot be replicated in user code (say for another UI framework that also requires the UI code to run on a specific thread)?

@Douglas_Gregor

The names don’t have to be perfectly self-evident, but communicating some meaning behind the intent in the range of “a human is waiting on this” vs. “do this when you have idle cycles” seems beneficial. If we’re to only consider the ordering without meaning, then we might as well have an Int rather than an enum.

I'm totally on board with an Int (or an enum backed by an Int). I would still offer names like high/low etc because it's not always obvious if a higher number means higher or lower priority.

If the names aren't self-evident, I don't really see why we should have the names as the canonical way of specifying these priorities. Of course, it makes a lot of sense to also have names like userInteractive but IMHO they should map to the canonical spelling which would not hard-code the UI metaphor. If I may take the server example again, what the word user represents for example is probably quite unclear.

[benrimmington]

extension Task {
  static func withGroup<TaskResult, BodyResult>(
    resultType: TaskResult.Type,
    returning returnType: BodyResult.Type = BodyResult.self,
    body: (inout Task.Group<TaskResult>) async throws -> BodyResult
  ) async throws -> BodyResult
}

• The resultType: TaskResult.Type could become elementType: Element.Type, especially if the task group will conform to AsyncSequence.

• Why is the returning returnType parameter needed?

• The body is @escaping in the current implementation.

---

extension Task.Group { 
  mutating func add(
    overridingPriority: Priority? = nil,
    operation: @escaping () async throws -> TaskResult
  ) async -> Bool
}

• The operation: @escaping could become element: @autoclosure @escaping.

• The return type is Task.Handle with @discardableResult in the current implementation.

[s-k (Stephen Kockentiedt)]

Edit: My question was already answered:

@Douglas_Gregor

The intent of the proposal is that the async let does not introduce a suspension point

Original question

@xAlien95

I guess it's explicitly mentioned in the Detailed design section:

Any reference to a variable that was declared in an async let is a potential suspension point, equivalent to a call to an asynchronous function, so it must occur within an await expression. The initializer of the async let is considered to be enclosed by an implicit await expression.

The answer is: await is implicit when using async let and it isn't when using group.add (at the moment at least).

I understand this differently: The implicit await is placed after the =. Can someone please confirm this?

Otherwise, this could be a very subtle bug (assuming interleaving is possible):

self.balance = 42
async let valueNeededLater = otherInstance.doSomething()
print(self.balance) // Can this be something else than 42, even if doSomething() does not touch balance?

In other words: Is there a potential suspension point between self.balance = 42 and print(self.balance)?

[s-k (Stephen Kockentiedt)]

@John_McCall

withGroup does wait for the child tasks to all complete

I think this is very subtle behavior. In my opinion, this will lead to performance bugs because people will assume that all remaining tasks are cancelled.

I would feel better if the body of the task group must either

return .afterAwaitingRemainingTasks(result)

or

return .afterCancellingRemainingTasks(result)

This would be the most explicit solution. However, I am open to other solutions, which are more expilicit than the current one.

@ktoso: I believe you wrote something about this issue in another thread.

[John_McCall]

@s-k

I think this is very subtle behavior. In my opinion, this will lead to performance bugs because people will assume that all remaining tasks are cancelled.

They are cancelled. Cancellation does not itself force child tasks to immediately complete, and structured concurrency does mean that we can't just leave cancelled tasks out there to complete in their own time; thus, we have to wait for them.

[s-k (Stephen Kockentiedt)]

So even without calling group.cancelAll() explicitly, remaining tasks are automatically cancelled when the body returns? The proposal says otherwise.

[Lantua]

@s-k

remaining tasks are automatically cancelled when the body returns?

The group cancels if withGroup body throws. It won't cancel if the block exits via normal return.

The three ways in which a task group can be cancelled are:
1. When an error is thrown out of the body of withGroup ,
2. When the task in which the task group itself was created is cancelled, or
3. When the cancelAll() operation is invoked.

For child task, it's cancel if the block surrounding it throws (I think?)

[John_McCall]

I think we've gone back and forth about this, so I'm not surprised if I'm misremembering. Originally it was a postcondition failure if you returned normally with tasks still in flight. I suppose not cancelling the remaining tasks is useful if the tasks don't have meaningful returns.

Regardless, withGroup has to wait for the tasks to actually complete.

[Jon_Shier]

I'm still reading the proposal but I noticed what seemed to be a discrepancy between the examples I've seen in this thread and the proposal itself.

extension Task.Handle {
  /// Cancel the task referenced by this handle.
  func cancel()
  
  /// Determine whether the task was cancelled.
  var isCancelled: Bool { get }
}

In the proposal, isCancelled is a property, but I've seen it used as an async function several times. Which one is correct? Or are the examples I've seen for different things? Personally I prefer a property over a method, if only for fluency and to match Swift's existing style, though I know async properties weren't part of the async await proposal (I really think they should be).

Edit: Reading further, this appears to be the difference between a Handle's isCancelled and Task's static isCancelled(). I'm pretty sure that's going to be confusing but we'll see.

Edit 2: Understanding the static Task context more (but perhaps not enough), it does seem odd to me that cancel() and isCancelled() are async in that context, but not on the Handle itself. Why the difference?

[Jon_Shier]

It seems like, in this method:

extension Task.Group { 
  /// Add a task to the group.
  mutating func add(
      overridingPriority: Priority? = nil,
      operation: @escaping () async throws -> TaskResult
  ) async -> Bool
}

overridingPriority should be atOverridingPriority, since you're providing a new Priority, not setting some priority which should be overridden.

[Jon_Shier]

Overall the proposal looks great, though I'm still not fully comfortable only seeing part of the picture at any given moment.

I do have a few questions (which may have already been answered).

1. What are the intended uses of Handle values? Could I offer them for multiple observation? That is, could I have a network framework that offers a var user: Task.Handle<User, NetworkError> value for any consumers needing the current User without issues?
2. Relatedly, and it doesn't appear to exist at this level, but is there a solution for multiple callbacks, like observed properties or streams? For instance, the above example is only really useful if there's a single User value. Updates to the User value would have to be provided by something other than a Handle, right?. So would async systems that provide multiple values still use closures?
3. How heavy are the underlying Task values (or whatever representation is controlled by the Task API)? Backpressure is mentioned, but what sort of numbers are we talking about here? Has there been any testing to see how many tasks a typical app may produce at any given moment? Are there any scalability goals here? Ultimately, how much are users supposed to care about the number of tasks they're creating?
4. Similar to 2, will there be any future or higher level facility to add multiple cancellation handlers, or other observational callbacks? Or is it up to higher level APIs to vend representations that could offer similar functionality?

[MichaelB]

Amazing work @Douglas_Gregor.

I see value in a name like spawn let instead of “overloading” async in this context. You discuss this at the very end of the proposal.

In the proposal you say: “The initializer of the async let is considered to be enclosed by an implicit try expression.”

This nuance makes me feel like a distinct word would be more clear rather than reusing async in this context.