[Pitch #2] Structured Concurrency

Evolution Pitches
1

Hi all,

We've been revising the Structured Concurrency pitch document a bit to try to explain more about the design goals and fill in some gaps based on the discussion in the first pitch thread. Here's a list of the changes in this second pitch document:

  • Added a "desugaring" of async let to task groups and more motivation for the structured-concurrency parts of the design.
  • Reflowed the entire proposal to focus on the general description of structured concurrency first, the programming model with syntax next, and then details of the language features and API design last.
  • Reworked the presentation of the Task APIs with more rationale for the design.
  • Added more discussion of why futures aren't more prominent.
  • "Task nursery" has been replaced with "task group".
  • Added support for asynchronous @main and top-level code.
  • Specify that try is not required in the initializer of an async let, because the thrown error is only observable when reading from one of the variables.
  • withUnsafe(Throwing)Continuation functions have been moved out of the Task type.
  • Note that an async let variable can only be captured by a non-escaping closure.
  • Removed the requirement that an async let variable be awaited on all paths.

The latest draft is available here.

Comments welcome!

Doug

27 Likes
2

I'm still not entirely sure which use cases structured concurrency is meant to be used for.

Use case 1: Increase performance by using multiple CPU cores. In this case the programmer needs to make sure that there is no concurrent read/write or write/write access to shared data (by using value types, ActorSendable, mutexes, etc).

Use case 2: Reduce latency by parallelizing IO. In this case no parallel code execution is needed and the programmer would be best served if all code is executed on the originating thread (so no precautions are necessary to prevent data races).

Structured concurrency is a solution for case 1 but not for case 2, correct? Case 2 would be solved by combining structured concurrency with a reentrant actor?

Is it planned to have a way to mark code that uses async let as "single-threaded" without binding it to an actor?

Thanks!

3

I found this article helpful way back in the day:

https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/

My short summary of it is: structured concurrency adds another building block for asynchronous programming, roughly analogous to control flow constructs for synchronous programming.

7 Likes
4

I think it's a combination of both cases. Take SwiftNIO which is pretty much everything a modern server system should have: it uses a limited number of kernel threads in conjunction with "soft" concurrency, probably the most dangerous and the most efficient combination that is known today.

I think one of the goals of Swift concurrency should be making SwiftNIO much cleaner and nicer to deal with, among other things. I don't know if the goal of the proposal was that, or whether SwiftNIO will be rewritten (maybe even from scratch!), but ultimately it's how I see the purpose of modern concurrency facility in any programming language: make the combination of kernel threads and soft concurrency idiomatic.

This in fact will put Swift above many popular dynamic languages where kernel threads are not supported and therefore server systems based on those languages are very incomplete. The full use of all available CPU cores on the server is becoming a big prize in this race among languages, in my view.

Edit: I didn't mean literally kernel threads in OS terms, of course

6 Likes
5

It is a good article, but it’s worth noting that the author is only interested in @ lassejansen’s use case 2 (Trio is entirely single-threaded), while Swift structured concurrency appears to be aimed at both use cases without distinguishing clearly between them.

1 Like
6

So if this:

async let veggies = chopVegetables()

is equivalent to this:

await group.add { 
   DinnerChild.chopVegetables(await chopVegetables())
}

Why doesn't the async let line have an await when group.add does?

I was pretty sure at first this meant await before group.add did not really introduce a suspension point and was required solely because group.add needs to run from an async context. But then in the detailed design section, it says:

The add operation is async to allow for a form of back-pressure. If the executor on which the new task will be scheduled is oversubscribed, the add call itself can suspend to slow the creation of new tasks.

Does that mean async let introduces a suspension point too? There should be an await on the async let line if this is the case.

It's confusing that await is used for these APIs:

try await Task.checkCancellation()

await Task.isCancelled()

await Task.currentPriority()

There should be no await unless calling those might actually introduce a suspension point. We should use await to mean "here may be a suspension point", not "here's a function that requires an async context". Otherwise you train people to consider await as likely spurious, like I did at first with group.add.

Shouldn't there be a try on the DinnerChild.chopVegetables(await chopVegetables()) line?

2 Likes
7

I've long wanted to add to the proposal an @instantaneous also known as "can not actually suspend", specifically because those APIs.

It would have the effect of:

  1. allowing calls to those functions without await
  2. forbid the use of await inside the function body of the instantaneous function.

i.e.:

@instantaneous func isCancelled() async -> Bool { ... } 

if Task.isCancelled() { ... }

The wording I thought of this being instantaneous is because "can not suspend", but totally open to other names.

The same semantics would really help [Pitch] Task Local Values, reading a task local value never suspends, and having to await on them is pretty annoying:

let id = await Task.local(\.traceID) // meh

Perhaps this is the right proposal to include this new capability?

7 Likes
8

Perhaps. By introducing this here, where we have several APIs that would make use of it, its use and meaning would be self-explanatory.

Isn't "instantaneous" synonym with "synchronous" here? It feels like @instantaneous and async are cancelling each other, like matter and anti-matter.

If we mean "synchronous but needs a task context" maybe we could just omit async and name the attribute so it signals the task context requirement:

@taskLocal func isCancelled() -> Bool { ... }

And perhaps getting rid of async could allow isCancelled to be a property:

@taskLocal var isCancelled: Bool { ... }

which would fell more natural.

And then people will do this: 
@taskLocal var traceID: Int { Task.local(\.traceID) }

but that's probably a topic for the other pitch.

10 Likes
9

I have thoughts about this proposal generally having to do with the user experience. As a start it would be helpful, I think, to name things exactly as we mean them.

If the meaning desired here is “cannot suspend,” I would hope that we would name the annotation @nonsuspending or something similar. Or we could write async(nonsuspending).

6 Likes
10

Thank you! This looks mostly really good, just a few questions:

An asynchronous function that is currently running always knows the executor that it's running on.

What's the API to query the current executor? Or if there's no API to query it, what does it mean for the function to "know" its executor?

Swift provides a default executor implementation, but both actor classes and global actors (described in separate proposals) can suppress this and provide their own implementation.

How can non-actor classes provide a custom executor? And what API/magic registration/... do custom executors need to implement?

extension Task {
/// Describes the priority of a task.
enum Priority: Int, Comparable {
/// The task is important for user interaction, such as animations, event handling, or
/// updating your app's user interface 
case userInteractive

/// The task was initiated by the user and prevents the user from actively using
/// your app.
case userInitiated

/// Default priority for tasks. 
case `default`

/// Priority for a utility function that the user does not track actively.
case utility

/// Priority for maintenance or cleanup tasks.
case background

I do understand that at this moment in time, the majority of Swift code written are indeed apps for Apple platforms which is probably why you're proposing to hard-code the priority names to make sense in a system where a user interacts with a UI. But IMHO Swift should be a general purpose language and I don't think we should hardcode the priorities to have a "UI meaning". Why not just highest, high, default, low, lowest and map the proposed spellings to that (userInteractive -> highest, ..., background -> lowest).

And finally, (this is actually a quote from the async/await proposal):

However, many asynchronous functions are not just asynchronous: they’re also associated with specific actors (which are the subject of a separate proposal), and they’re always supposed to run as part of that actor. Swift does guarantee that such functions will in fact return to their actor to finish executing.

What mechanism is used for this hopping-back to a certain actor, can this also be used with custom executors?

16 Likes
11

Can you elaborate on what happens if cancellation occurs while a partial task is currently executing? The proposal says cancellation handlers are invoked immediately when cancellation occurs, but it seems like it wouldn’t be possible until a suspension point is reached. I’m specifically thinking about a case where two child tasks running on a non-exclusive executor are executing concurrently and one throws as an example scenario.

Actually, come to think of it I’d love to see more on non-exclusive executors in general. They are alluded to but are not discussed in detail. If child tasks are able to execute concurrently then how are data races avoided?

12

First, I found the pitch document high quality, so well done there. I have some feedback on it:

I found the description of what happens when an error is thrown confusing. I initially thought the document said that tasks are not implicitly awaited when an error is thrown, only cancelled. Rereading the child task section, it was this sentence that confused me:

On exiting the body of the makeDinner() function with this error, any child tasks that have not yet completed (marinating the meat or preheating the oven, maybe both) will be automatically cancelled.

The cancellation section did clarify this for me but it may still be worth mentioning here too that cancellation is cooperative rather than forcibly stopping tasks.

I also noticed this function declaration is seemingly missing async throws and the body missing a .get:

func eat(mealHandle: Task.Handle<Meal, Error>) {
  let meal = try await mealHandle()
  meal.eat() // yum
}
13

A task can install cancellation handlers that will be invoked immediately (i.e. concurrently from the task's perspective) if the task is cancelled. This is to allow code that's blocking on some slow operation to wrap it up immediately (and perhaps free up any resources associated with that operation); for example, if you had an async function for sending a request to a server, and it used withUnsafeContinuation to resume the task when the server responded, you could install a cancellation handler to send a cancellation message to the server and resume the task immediately. Since the handler runs concurrently, you would need some sort of atomic latch to ensure that you didn't resume the continuation multiple times. Swift will ensure that you don't have to worry about the cancellation handler being invoked twice, though, or about data races between its invocation and its uninstallation.

Whether a task throws is a mostly independent question from cancellation. They are related only in that certain functions may choose to react to cancellation by throwing. A task throwing does not mean it was cancelled, and it doesn't by itself cancel any sibling or parent tasks, so I'm not sure how to interpret your specific case.

1 Like
14

I was under the same impression that throwing would cancel tasks. Perhaps it was because of this part:

Task group cancellation

There are several ways in which a task group can be cancelled. In all cases, all of the tasks in the group are cancelled and no more tasks can be added to the group ( add will return false ). The three ways in which a task group can be cancelled are:

  1. When an error is thrown out of the body of withGroup ,
15

One of the ways to cancel a task group is

  1. When an error is thrown out of the body of withGroup ,

I think we should refine it to

When body of withGroup returns, either by throwing or returning a value,

Since we don't want dangling tasks after the group is done with the values. I'd go so far as to wait for all subtasks to return/throw before returning from withGroup as well, but that could be a hard call.

Is the child task cancelled when it goes out of scope (without awaiting)? I think we should, but I couldn't fine any mention of it.

Should we rename Child tasks to Subtasks?

16

withGroup does wait for the child tasks to all complete; if that isn’t clear in the document, we need to fix that.

4 Likes
17

If a parent task gives up on waiting for its child tasks, for whatever reason, they are cancelled (and waited for).

18

This bit of the proposal is what I had in mind:

child tasks in the proposed structured-concurrency model are (intentionally) more restricted than general-purpose futures. Unlike in a typical futures implementation, a child task does not persist beyond the scope in which it was created. By the time the scope exits, the child task must either have completed, or it will be implicitly awaited. When the scope exits via a thrown error, the child task will be implicitly cancelled before it is awaited.

Bringing it back to our example, note that the chopVegetables() function might throw an error if, say, there is an incident with the kitchen knife. That thrown error completes the child task for chopping the vegetables. The error will then be propagated out of the makeDinner() function, as expected. On exiting the body of the makeDinner() function with this error, any child tasks that have not yet completed (marinating the meat or preheating the oven, maybe both) will be automatically cancelled

19

Okay. To answer what I think your question is, if a scope makes a couple child tasks, and one of them throws, and the parent task awaits that task, and it doesn’t handle the thrown error and just lets it unwind from the scope which created the child tasks, then all the still-running child tasks will be cancelled, and the error will continue propagating in the parent task as soon as all of those child tasks complete. But this is just compositional semantics: awaiting a throwing task handle receives the result of that task, including rethrowing any error it threw, and running child tasks are cancelled when the spawning scope exits.

3 Likes
20

Yeah, absolutely.

You'll be happy to know after posting this I spent a sleepless night debating if I should get up and post that this likely should bee called @suspends(never)... :wink: Kind of also taking into account the suspends feedback that @kavon had in other threads.