Concurrency 101

Using Swift
1

How are async functions implemented?

To understand this, I am reading the transcript for Explore Swift performance talk given by @John_McCall.

When he explains how async functions work:

func awaitAll(tasks: [Task<Int, Never>]) async -> [Int] {
    var results = [Int]()
    for task in tasks {
        results.append(await task.value)
    }
    return results
}

At one point, he refers to some local functions, which confuses me.

So far, we’ve only been talking about synchronous functions.

What about async functions? The central idea with async functions is that C threads are a precious resource, and holding on to a C thread just to block is not making good use of that resource.

As a result, async functions are implemented in two special ways: First, they keep their local state on a separate stack from the C stack, and second, they’re actually split into multiple functions at runtime.

So, let’s look at an example async function.

There’s one potential suspension point, await, in this function.

All of these local functions have uses that cross that suspension point, so they can’t be saved on the C stack.

We just talked about how sync functions allocate their local memory on the C stack by subtracting from the stack pointer.

Async functions conceptually work the same way, except they don’t allocate out of a large, contiguous stack.

Instead, async tasks hold on to one or more slabs of memory.

When an async function wants to allocate memory on the async stack, it asks the task for memory.

What are those local functions in the above context?

Are they the multiple functions below?

As a result, async functions are implemented in two special ways: First, they keep their local state on a separate stack from the C stack, and second, they’re actually split into multiple functions at runtime.

Can anyone clarify?

Update: local functions should be local variables, see the answer and confirmation below.

2

I’d suggest to watch video instead of only transcript: it has visual representation of these things, that probably give the better understanding what all of that means. You might still not take it from the first time, but there are a really good illustration of what is local functions and how example is split into multiple, I just don’t feel confident to rephrase anything from there that would be that much accurate description.

2 Likes
3

I think this was meant to be ”local variables”. The video clearly highlights variables that live across the suspension point. I would recommend watching it if you haven’t.

Also, you might find Asynchronous Functions in Swift from the 2021 LLVM Dev Mtg interesting. It goes into more detail on how this is implemented at a lower level.

2 Likes
4

It makes sense now. Thank you.

5

Thank you. I have some hearing loss; I am unable to resolve the sounds in videos clearly, thanks to the aging process. :slight_smile:

6

If two async functions happen to run on the same thread, do they run in parallel concurrently (async is about concurrency after all)? If yes, how context switching points are determined, and how often they occur? (given that the functions do not explicitly yield execution flow)?

1 Like
7

If two functions run on the same thread then they can't run in parallel by definition, but you probably meant isolation. In either case two async functions can't run in parallel unless they yield, i.e. they have await inside, and even in this case the execution of the parts before and after each await is serialized. (Hope I'm correct on this!)

2 Likes
8

Sorry for the wrong term. Already corrected. "in parallel" usually means "in different threads". I meant "concurrently".

I meant concurrency. This post gives a good explanation of how concurrency works without parallelism. It shows context switch points. My question is: What creates these switch points if my concurrent functions do not yield? Are these points created at all in this case? And if they are created, how often they occur?

1 Like
9

That's right. I misspoke, and unfortunately we didn't catch it during editing.

2 Likes
10

You have suspension points to mark that, so then around awaits — which are these markers, — where function is suspended, thread can switch to other work.

Yielding as suggested above is a way to introduce suspension points in otherwise synchronous/long running code: you can try that by defining two functions that loop over some range and yield periodically to see how it affects synchronous operations.

If there is no suspension, function will continue execution as is, “blocking” thread for the work. Swift doesn’t introduce any artificial suspension points additionally to one’s defined with awaits, because if it would introduce, meaning any function randomly can suspend at any time, you wouldn’t be able to rely on synchronous functions anymore and distinction between sync and async one’s become meaningless. That’s why we need this “function coloring” as well: to make a clear distinction between functions that can suspend and that cannot.

3 Likes
11

Given that the number of processor cores is finite, I think this is what happens at low level. If a function takes too long to complete, hogging its thread, thus causing it to use all of its time quantum, the scheduler intervenes and context switches to another thread because the original thread has used up its time quantum.

Please correct me if this is not close enough to reality.

12

As far as I understand, this is true for parallel execution flow in different threads (when number of threads is more than number of CPU cores). My question though was about a particular case in Swift concurrency when two or more concurrent functions are running in the same thread, e.g. started from main thread and annotated as @MainActor.

13

Would be really cool to have an indepth talk about executor and scheduling. Overall " Explore Swift performance" is super informative. Wish there was more content like that.

14

Thank you, this is exactly what I wanted to know.

1 Like
15

If you occupy the main actor with something that runs forever with no awaits, your app will stop responding to user events forever.

4 Likes
16

This is what I wanted to know in regard to Swift concurrency (e.g. that it is possible to block main thread with async calls). Not only about main thread, but any thread. Swift concurrency uses some pre-allocated thread pool with limited number of threads, and if an app creates many async calls at the same time, some concurrent functions end up running at the same thread at the same time. I wanted to know how are they managed to run concurrently at the same thread.

Now my understanding is that if there are no awaits in them, they run sequentially - one then another. And if there is an await in a concurrent function, it basically splits the function in two (if there is only one await in the function that not placed in a loop), and these two chunks of code can execute intermittently with other such chunks of code, possibly from other concurrent functions running on the same thread. Not much parallelism actually within a single thread. One function at a time or half a function at a time is not that different.

I think that real benefit of using async/await is the fact that most likely (but this is not guaranteed) an async call will end up running in a different thread, actually running in parallel.

17

That's not possible. A single thread, think of it as a "carrier", can only ever be executing "one piece of code". In this case such a "thing" is a task. It runs "on" a given thread, until suspension, at which point that thread can be used to run some other task.

I would also warn against calling things "concurrent function", that's not a thing that exists. Functions can be asynchronous but there's no such thing as a "concurrent function"; Execution can be concurrent, but a function itself not really.

async/await is "just" a means to achieve cooperative multitasking.

The benefit of async/await is that such code "looks" linear/sequential and is easier to reason about than doing the same with callbacks or continuation passing style, or futures etc.

Parallelism ("things actually happening at the same time") only exists between multiple tasks. Which async/await doesn't really do by itself at all: A bunch of await a; await b; await c lines in a single task won't ever execute in parallel -- they're in a single task, and thus execute sequentially. It's not guarnteed what specific thread they'll use (unless a specific actor uses a specific thread using a custom executor etc).

Parallelism appears with multiple tasks and they run some things "at the same time" (using multiple actual carrier threads).

Interleaving of execution may happen at points in code marked with await. async/await enables this interleaved execution, which we need because we're in a cooperative scheduling world, so the tasks need to "give back the thread" when they're about to suspend, so some other task may use it.

Interleaving enables concurrency even without actual parallelism. Tasks may execute concurrently, but if the underlying runtime had a single thread for example, they'd never execute in parallel. It'd still be concurrent execution - thanks to the interleaving - but it would not be parallel. If the same example had more threads backing those tasks, they may execute concurrently in parallel. In other words, concurrency is a prerequisite for parallel execution, but it's not a 1:1 relationship. Concurrent code may end up executing not in parallel at all because of scheduling decisions which are only known at runtime.

You may want to check this video as well: Swift concurrency: Behind the scenes - WWDC21 - Videos - Apple Developer which discusses context switching etc.

Hope this helps clear things up a bit.

10 Likes
18

Sorry for my bad English. Now I know that code is running "on" a thread, not "in" or "at".

19

Thank you for the link. This video provides an example of a hypothetical news feed app, and explains why Swift concurrency is better than GCD in terms of performance and efficiency: If an app wants to download/update 1000 feeds at once, GCD version of the app would create 1000 threads, which is too much. We have thread explosion, app (nearly) stops working. Swift concurrency version of the app does not create 1000 threads, and thus is better. It is not clear for me however if Swift concurrency version of the app is able to download/update 1000 feeds at once. Is it viable to do so using Swift concurrency? My opinion is that in any case (GCD or Swift concurrency) we should limit the number of concurrent tasks to a reasonable amount.

20

Using neither option you would be able to download 1000 at once, because you have limited resources on the device – mostly by the number of cores. It possible, though, that all 1000 might make progress (in case we put aside networking limitations), if after reading a chunk data for one request it switches to read for another request, and so on.

func load1000feeds() async {
    await withDiscardingTaskGroup { group in
        for i in 0..<1000 {
            group.addTask { await downloadFeed(at: in) }
        }
    }
}

func downloadFeed(at index: Int) async {
    let dataStream = ...
    let chunkSize = ...
    while !dataStream.isEOF() {
        read(chunkSize, from: dataStream)
        await Task.yield()  // allowing other tasks to make a progress
    }
}

With GCD we had to limit amount of queues, and consequently, threads, by roughly implementing the same limited pool scattered across the project. With new concurrency that's not the case anymore. What we want to limit currently is amount of suspension points – await's: hops to executors aren't cheap, they (I guess) cheaper than full context switch, but still ain't free. I think consequently you'll have less tasks running, but you don't exactly control task creation.

2 Likes