Hi, folks.
Any concurrency design has to provide some way of actually running code concurrently. We propose embracing the approach of structured concurrency, in which concurrent programs are organized into tasks and child tasks rather than simply a sea of formally-unrelated threads and jobs. Structured concurrency allows us to neatly address a large number of systemic problems with task management that would otherwise need a lot of ad hoc support.
This is part of a series of pitches that jointly form the Swift concurrency proposal. You can find a roadmap for the entire proposal here. I have posted a portion of the proposal below; the full text is also available. This proposal is closely related to the proposal for asynchronous functions; it is probably best to start with that proposal, then read this one.
The text of this proposal has changed since this initial post; please read the copy at the link rather than the text below.
Introduction
async/await is a language mechanism for writing natural, efficient asynchronous code. Asynchronous functions (introduced with async) can give up the thread on which they are executing at any given suspension point (marked with await), which is necessary for building highly-concurrent systems.
However, the async/await proposal does not introduce concurrency per se: ignoring the suspension points within an asynchronous function, it will execute in essentially the same manner as a synchronous function. This proposal introduces support for structured concurrency in Swift, enabling concurrency execution of asynchronous code with a model that is ergonomic, predictable, and admits efficient implementation.
Motivation
For a simple example, let's make dinner, asynchronously:
func chopVegetables() async throws -> [Vegetable] { ... }
func marinateMeat() async -> Meat { ... }
func preheatOven(temperature: Double) async throws -> Oven { ... }
// ...
func makeDinner() async throws -> Meal {
let veggies = await try chopVegetables()
let meat = await marinateMeat()
let oven = await try preheatOven(temperature: 350)
let dish = Dish(ingredients: [veggies, meat])
return await try oven.cook(dish, duration: .hours(3))
}
Each step in our dinner preparation is an asynchronous operation, so there are numerous suspension points. While waiting for the vegetables to be chopped, makeDinner won't block a thread: it will suspend until the vegetables are available, then resume. Presumably, many dinners could be in various stages of preparation, with most suspended until their current step is completed.
However, even though our dinner preparation is asynchronous, it is still sequential. It waits until the vegetables have been chopped before starting to marinate the meat, then waits again until the meat is ready before preheating the oven. Our hungry patrons will be very hungry indeed by the time dinner is finally done.
To make dinner preparation go faster, we need to perform some of these steps concurrently. To do so, we can break down our recipe into different tasks that can happen in parallel. The vegetables can be chopped at the same time that the meat is marinating and the oven is preheating. Sometimes there are dependencies between tasks: as soon as the vegetables and meat are ready, we can combine them in a dish, but we can't put that dish into the oven until the oven is hot. All of these tasks are part of the larger task of making dinner. When all of these tasks are complete, dinner is served.
This proposal aims to provide the necessary tools to carve work up into smaller tasks that can run concurrently, to allow tasks to wait for each other to complete, and to effectively manage the overall progress of a task.
Proposed solution
Our approach follows the principles of structured concurrency. All asynchronous functions run as part of an asynchronous task. Tasks can conveniently make child tasks that will perform work concurrently. This creates a hierarchy of tasks, and information can conveniently flow up and down the hierarchy, making it convenient to manage the whole thing holistically.
Child tasks
This proposal introduces an easy way to create child tasks with async let:
func makeDinner() async throws -> Meal {
async let veggies = try chopVegetables()
async let meat = marinateMeat()
async let oven = try preheatOven(temperature: 350)
let dish = Dish(ingredients: await [veggies, meat])
return await try oven.cook(dish, duration: .hours(3))
}
async let is similar to a let, in that it defines a local constant that is initialized by the expression on the right-hand side of the =. However, it differs in that the initializer expression is evaluated in a separate, concurrently-executing child task. On completion, the child task will initialize the variables in the async let and complete.
Because the main body of the function executes concurrently with its child tasks, it is possible that makeDinner will reach the point where it needs the value of an async let (say, veggies) before that value has been produced. To account for that, reading a variable defined by an async let is treated as a suspension point, and therefore must be marked with await. The task will suspend until the child task has completed initialization of the variable, and then resume.
One can think of async let as introducing a (hidden) future, which is created at the point of declaration of the async let and whose value is retrieved at the await. In this sense, async let is syntactic sugar to futures.
However, child tasks in the proposed structured-concurrency model are (intentionally) more restricted than general-purpose futures. Unlike in a typical futures implementation, a child task does not persist beyond the scope in which is was created. By the time the scope exits, the child task must either have completed, or it will be implicitly cancelled. This structure both makes it easier to reason about the concurrent tasks that are executing within a given scope, and also unlocks numerous optimization opportunities for the compiler and runtime.
Bringing it back to our example, note that the chopVegetables() function might throw an error if, say, there is an incident with the kitchen knife. That thrown error completes the child task for chopping the vegetables. The error will then be propagated out of the makeDinner() function, as expected. On exiting the body of the makeDinner() function, any child tasks that have not yet completed (marinating the meat or preheating the oven, maybe both) will be automatically cancelled.
Nurseries
The async let construct makes it easy to create a set number of child tasks and associate them with variables. However, the construct does not work as well with dynamic workloads, where we don't know the number child tasks we will need to create because (for example) it is dependent on the size of a data structure. For that, we need a more dynamic construct: a task nursery.
A nursery defines a scope in which one can create new child tasks programmatically. As with all child tasks, the child tasks within the nursery must complete when the scope exits or they will be implicitly cancelled. Nurseries also provide utilities for working with the child tasks, e.g., by waiting until the next child task completes.
To stretch our example even further, let's consider our chopVegetables() operation, which produces an array of Vegetable values. With enough cooks, we could chop our vegetables even faster if we divided up the chopping for each kind of vegetable.
Let's start with a sequential version of chopVegetables():
/// Sequentially chop the vegetables.
func chopVegetables() async throws -> [Vegetable] {
var veggies: [Vegetable] = gatherRawVeggies()
for i in veggies.indices {
veggies[i] = await try veggies[i].chopped()
}
return veggies
}
Introducing async let into the loop would not produce any meaningful concurrency, because each async let would need to complete before the next iteration of the loop could start. To create child tasks programmatically, we introduce a new nursery scope via withNursery:
/// Sequentially chop the vegetables.
func chopVegetables() async throws -> [Vegetable] {
// Create a task nursery where each task produces (Int, Vegetable).
Task.withNursery(resultType: (Int, Vegetable).self) { nursery in
var veggies: [Vegetable] = gatherRawVeggies()
// Create a new child task for each vegetable that needs to be
// chopped.
for i in rawVeggies.indices {
await try nursery.add {
(i, veggies[i].chopped())
}
}
// Wait for all of the chopping to complete, slotting each result
// into its place in the array as it becomes available.
while let (index, choppedVeggie) = await try nursery.next() {
veggies[index] = choppedVeggie
}
return veggies
}
}
The withNursery(resultType:body:) function introduces a new scope in which child tasks can be created (using the nursery's add(_:) method). The next() method waits for the next child task to complete, providing the result value from the child task. In our example above, each child task carries the index where the result should go, along with the chopped vegetable.
As with the child tasks created by async let, if the closure passed to withNursery exits without having completed all child tasks, any remaining child tasks will automatically be cancelled.
Detached tasks
Thus far, every task we have created is a child task, whose lifetime is limited by the scope in which is created. This does not allow for new tasks to be created that outlive the current scope.
The runDetached operation creates a new task. It accepts a closure, which will be executed as the body of the task. Here, we create a new, detached task to make dinner:
let dinnerHandle = Task.runDetached {
await makeDinner()
}
The result of runDetached is a task handle, which can be used to retrieve the result of the operation when it completes (via get()) or cancel the task if the result is no longer desired (via cancel()). Unlike child tasks, detached tasks aren't cancelled even if there are no remaining uses of their task handle, so runDetached is suitable for operations for which the program does not need to observe completion.
