Async/await status?

[adamkemp]

As far as I'm aware C# doesn't use a linked list to represent the call stack, at least not formally. The resulting Task object may be consumed by a non-async function, and you can also await a non-async function. Maybe you could consider completion callbacks to be a kind of linked list, but they can be more complex than that (consider Task.WhenAny or Task.WhenAll) so the compiler can't really assume that async functions must fit into a simple stack structure.

FWIW, I am still a strong proponent of tying async/await to a formal promise object with an API, especially if you really do need to invent some kind of new way of handling stacks to avoid it. Using the formal promise object allows for a lot of patterns that could be very difficult without it, and makes it compose well with higher-level constructs (such as the Task.When* functions listed above).

I'm kind of afraid that we may be overthinking this feature and trying to make it more complex than it really has to be.

[Joe_Groff]

@adamkemp

As far as I'm aware C# doesn't use a linked list to represent the call stack, at least not formally. The resulting Task object may be consumed by a non- async function, and you can also await a non- async function. Maybe you could consider completion callbacks to be a kind of linked list, but they can be more complex than that (consider Task.WhenAny or Task.WhenAll ) so the compiler can't really assume that async functions must fit into a simple stack structure.

It doesn't directly use a linked list, but that is the effect that falls out from the compiler transform when an async function calls another. If you have:

async Task<B> foo() {
  a();
  await bar();
  return b();
}

async Task<D> bar() {
  c();
  await bas();
  return d();
}

async Task<E> bas() {
  return e();
}

then notionally, this transforms into:

Task<B> foo() {
  a();
  return bar().then { b(); }
}

Task<D> bar() {
  c();
  return bas().then { d() };
}

Task<E> bas() {
  return Task<E>(e());
}

where each "frame" of the async computation either allocates a new Task, or allocates a closure to chain on to the task from the awaited computation below it, creating a daisy chain of allocations.

@adamkemp

FWIW, I am still a strong proponent of tying async/await to a formal promise object with an API, especially if you really do need to invent some kind of new way of handling stacks to avoid it. Using the formal promise object allows for a lot of patterns that could be very difficult without it, and makes it compose well with higher-level constructs (such as the Task.When* functions listed above).

One could argue that compiler transformation into promise objects is a "new way of handling stacks" too. It's an effective one in .NET land, but it seems to be more of a mixed bag in JavaScript and Python land, where there seems to be a growing backlash against the incidental complexities of promises. If we avoid privileging promises, then I feel like that ends up with a simpler programmer model, even if the implementation ends up more complex, because there's less opportunity for the underlying promise model to leak into the user's mental model. It also allows for people to implement different models for organizing async tasks, such as structured concurrency, without them feeling second-class compared to promises. A non-promise-based language design also allows us more flexibility in considering implementation approaches, which might be the cause of the apparent "overthinking" here.

[adamkemp]

That's not actually how C#'s compiler transform works, though I'm guessing you already know that.

I don't think it really make sense to consider it as some special handling of stacks, unless you consider any hoisting of locals into objects as special handling of stacks. That's the kind of transform that's already done for things like closures, and we don't need any "growing stack" for those either.

I'm just not sure where this need for a new kind of growable stack comes from. What am I missing?

To me the important traits I want for this feature are:

1. Composability:

• Can I call multiple async methods and then wait on them as a group?

2. Interoperability:

• Can I call async methods from non-async methods in a way that allows me to still handle their completion somehow?
• Can I easily await things that signal completion but aren't async methods?

3. Configurability:

• Can I influence how the completion of a call I just made gets scheduled? Ex: sometimes you want the completion to come back on the calling context (like the main queue) and sometimes you don't care. It's useful to be able to control that, and it can be bad for performance if the choice is forced on you by the compiler.

C# solved some of those by having the compiler support awaiting things based on API pattern matching (i.e., "does this type have a method with this name and signature?") and some of them by having a useful promise API that implements the expected pattern and then builds on it.

How will these problems be solved without having some kind of API to interact with?

[robnik]

Some of this conversation is beyond me, but I'm curious what the transform is if you have a link or can explain it.

Maybe one reason for concern about the stack in Swift is because C# (probably Java too) uses a garbage collector that allows their "heap" allocations to be almost as fast as stack allocations - all they do is advance a pointer. Later the GC will move and compact longer living objects. Maybe Swift needs to do something manually here to get the same kind of memory performance. ?

[lukasa (Cory Benfield)]

@Joe_Groff

In languages like C#/JavaScript/Python, where async/await is sugar for working with promise objects, then when an async function calls into another one, there's a linked list of promises formed for each "frame" in the async call stack, and each one could generally be pre-allocated to a fixed size, yeah. In the case of Swift, I think we'd want to reduce overhead by using one allocation for the whole async callstack.

While @adamkemp is dealing with C#, I’ll point out that this isn’t really quite what Python does either. Python’s async/await is sugar for driving coroutines and awaitables. In Python the futures that make the base of the awaitable abstraction are rarely chained into a linked list. Instead, they are yielded by the bottom of the coroutine stack all the way to the coroutine runner, which attaches a single callback to throw the result of the future back down that coroutine stack.

Now, it’s definitely more accurate to say that Python’s coroutine call stack is still essentially a linked list, but that’s unrelated to the promises: it’s just a reflection of not having a better place in the runtime to put these stacks than on the heap.

[adamkemp]

This link explains how the transform works, and the followup post shows the various ways you can extend it in code (i.e., how to make your own types awaitable).

[Joe_Groff]

@adamkemp

That's not actually how C#'s compiler transform works, though I'm guessing you already know that.

Sure, the actual transform is more complex; I'm just trying to show how the "linked list" of contexts forms when async functions call each other.

@adamkemp

I don't think it really make sense to consider it as some special handling of stacks, unless you consider any hoisting of locals into objects as special handling of stacks. That's the kind of transform that's already done for things like closures, and we don't need any "growing stack" for those either.

Well, the purpose of a callstack is store the locals being used by the execution of a function while other functions below it run. In the case of an async function that can be suspended, those locals have to live somewhere so that they can be restored by whoever resumes them. In the C# model, the chain of state machines takes on this role. Each individual async function gets its own state machine, so none of them individually have to grow or shrink, but then you have added indirection between the stack machines when async functions call other async functions.

@adamkemp

I'm just not sure where this need for a new kind of growable stack comes from. What am I missing?

It's not strictly necessary; it's a potential optimization that we can consider. We could also make a fixed-size allocation for each async function's frame and linked-list them together.

@adamkemp

To me the important traits I want for this feature are:

1. Composability:

• Can I call multiple async methods and then wait on them as a group?

2. Interoperability:

• Can I call async methods from non- async methods in a way that allows me to still handle their completion somehow?
• Can I easily await things that signal completion but aren't async methods?

3. Configurability:

• Can I influence how the completion of a call I just made gets scheduled? Ex: sometimes you want the completion to come back on the calling context (like the main queue) and sometimes you don't care. It's useful to be able to control that, and it can be bad for performance if the choice is forced on you by the compiler.

C# solved some of those by having the compiler support awaiting things based on API pattern matching (i.e., "does this type have a method with this name and signature?") and some of them by having a useful promise API that implements the expected pattern and then builds on it.

How will these problems be solved without having some kind of API to interact with?

You can still build these APIs even if coroutines are not primitively bound to a future API; the proposal gives an example of a Future wrapper that you could provide C#-like API on to do all these things.

[Chris_Lattner3 (Chris Lattner)]

@Joe_Groff

Sorry, I'm taking for granted that, because of the C interop and ABI concerns John has already delineated, that we aren't planning to move C code or existing Swift code off of a traditional C-like fixed preallocated system stack. Async functions would, in addition to having access to whatever C stack they're currently executing on, have access to a context object where they would store any local "stack" values that might persist across a suspend. Async functions however wouldn't be pinned to any particular C stack, since they could be suspended and potentially resumed on a different thread. I'm speaking only about the growth strategy we might use for that async context, not for all functions in general.

Ah, I definitely did not understand that.

Given such a design, why would you need to maintain a contiguous stack for your suspended state? It seems like it would be perfectly fine to keep that segmented and therefore allow separately allocated pieces. This would mean you don't need to impose movability (or extra heap boxing) on such types.

-Chris

[Joe_Groff]

@Chris_Lattner3

Given such a design, why would you need to maintain a contiguous stack for your suspended state? It seems like it would be perfectly fine to keep that segmented and therefore allow separately allocated pieces. This would mean you don't need to impose movability (or extra heap boxing) on such types.

Sure, you could go with segmented allocations. The performance tradeoffs for would be similar to contiguous vs. segmented stacks in "pure" coroutine worlds like Go's, albeit perhaps less severe because we could minimize our use of the segmented context to only state that needs to survive a suspend.

In either a segmented or movable stack implementation, we could probably also do call graph analysis in many cases to pre-compute a likely maximum stack depth for the context.

[John_McCall]

I think some of this implementation discussion is premature and maybe a little distracting, especially outside of the Development subforum. But yes, I think there’s a basic question of how applicable the lessons of general-purpose stack implementations are for narrower async-only use cases.

[adamkemp]

Well, the purpose of a callstack is store the locals being used by the execution of a function while other functions below it run.

I think it would be a good idea to stop thinking of calling async functions as being based on a stack at all. This is the wrong mental model for async functions. For non-async calls it makes sense to think of caller/callee chains as a stack because a given function can only call one other function at a time. When caller calls callee then caller is suspended until callee finishes.

Async functions don't have to work like this. Calling an async function starts its execution, but there's no reason the caller must be suspended until it has finished. When you think of the chain of callers/callees as a stack then that implies that any function marked async must be awaited directly by its caller. In fact, this is actually stated explicitly in the proposal:

...you are required to "mark" expressions that call async functions with the new await keyword...

Then later:

Finally, you are only allowed to invoke an async function from within another async function or closure.

These should not be requirements. It forces you to jump through unnecessary hoops in order to actually run multiple async calls in parallel, which is not an uncommon thing to do.

There's no such requirement in C#. In C# making a function async doesn't introduce any requirements for how the caller must invoke it (it's not part of the function's signature at all). async only tells the compiler that this function should be transformed into a state machine coroutine and allows the usage of the await keyword to control how it is transformed (where its suspension points are). A caller of a function implemented using the await keyword can choose to use await when calling it or just call it and handle completion some other way (including ignoring its completion entirely). The caller may or may not itself be marked as async. This is valid:

async Task<Image> GetImageAsync(string name) {
    var data = await LoadImageDataAsync(name);
    return Image(data);
}

Task<Image> GetBackButtonImageAsync() {
    // Call `async` method without using `await`
    return GetImageAsync("BackButton");
}

async Task LoadImages() {
    // Call non-`async` method using `await`
    var backButtonImage = await GetBackButtonImageAsync();
    // ...
}

There's nothing to gain by making GetBackButtonImageAsync use the async keyword or directly await its call to GetImageAsync. All it adds is filling in the name of the image. The caller of GetBackButtonImageAsync, however, may choose to use await when calling it.

Think of it as dealing with inside a function ("I want my function to be able to wait on the completion of asynchronous work in a clean way") rather than looking outside ("My function completes asynchronously so how should my callers wait on me?"). We want a solution to the first problem, not the second. Let the caller decide how to wait or whether to wait, with async/await being one option.

You can still build these APIs even if coroutines are not primitively bound to a future API

We don't need async/await to be a new way to build a futures API. We can already build futures APIs without async/await so that's not a problem that needs to be solved. What we need is for async/await to be a better way to interact with handling asynchronous completions and to play well with any existing or yet to be written futures implementations.

[Joe_Groff]

My own opinion is that futures aren't really great API at all; the best you can say about them is that they let you encode suspendable computations in an environment that doesn't directly support them. By having deeper support for suspendable contexts in the language, we can implement better APIs for running tasks in parallel and joining them than futures. The future example in the proposal is more intended to show you can still do that, if you really want, than intended to be an end in itself.

[anandabits (Matthew Johnson)]

@Joe_Groff

the best you can say about them is that they let you encode suspendable computations in an environment that doesn't directly support them.

Encoding a computation / effect as a value has quite a bit of value in and of itself.

[John_McCall]

Fundamentally, when async functions are just starting each other and waiting for each other to finish — which really is the dominant case, even though it is indeed sometimes useful to kick off multiple child tasks, or to kick off a child task and then wait for it in a different way, or various other use-patterns — they are behaving exactly like a single-threaded computation. It is not abstractly unreasonable to ask if the implementation should make this use-pattern more efficient by attempting to re-use memory associated with that computation rather than repeatedly allocating and deallocating structures with the normal allocator. Such an implementation does not change the high-level shape of the solution.

But this is a very implementation-focused conversation, which is why I think it's a distraction to have it in front of people who really only care about the interface.

[AlexanderM (Alexander Momchilov)]

Doesn't Erlang have lazy evaluation? In a sense, that's like everything being implicitly async

[ktoso (Konrad 'ktoso' Malawski 🐟🏴‍☠️)]

@AlexanderM

Doesn't Erlang have lazy evaluation? In a sense, that's like everything being implicitly async

No, erlang is not lazy evaluated by default (maybe you are thinking of Haskell here).

Worth mentioning as well, lazy does not mean or imply async; we can use an example from present day swift, a “lazy var” is not going to be computed asynchronously, but “by the caller (thread)” when it is being accessed. There’s also ways to make such lazy references thread safe, but again, that would usually mean “initialize once” (like scala’s lazy val), by locking around the initialization — that’s synchronous again, not async.

—

From the linked comment:

The BEAM VM (Erlang related?) languages don't have Async/Await from what I can tell.

That’s true, however BEAM’s uses a preemptive scheduling model, which goes “all the way” and the entire runtime speaks in terms of “processes” (also known as actors), and “blocking” on the BEAM is not “really blocking” but rather it is suspending the current process actor. That runtime performs the M:N scheduling as was mentioned a while when I mentioned Loom with it’s VirtualThread or other reactive frameworks and their user-land scheduling. The BEAM being a preemptive scheduler means that actors/processes can be interrupted at any time by the scheduler (e.g. by reduction counting, and exceeding the “quota” assigned for given process); in contrast to that, what async/await style systems are is cooperative scheduling — i.e. if I don’t write an await anywhere, I cant be forced to give up my thread. So the runtime and the actual code cooperate about figuring out where those points are where I can yield back control to the runtime scheduler.

Both scheduling strategies have their tradeoffs... but it does not seem that pulling off such preemptive scheduling in the swift runtime is really a realistic thing to aim for (though I could be wrong of course ).

[mattrips]

@John_McCall

But this is a very implementation-focused conversation, which is why I think it's a distraction to have it in front of people who really only care about the interface.

Actually, this is a great discussion. I’d like to think that many of us care about, and are interested in, how the thing gets built. Please continue.

[kiel (Kiel Gillard)]

Perhaps what John means is this. It seems what is being talked about is not the needs that asynchronous programming is a solution for. I hope that any first class asynchronous programming solution begins with and is grounded in a broader consideration of these needs. It’d be a tragedy if the language and its constructs mislead developers by inappropriately prescribing solutions or tools to problems or situations at hand.

[hartbit (David Hart)]

@Joe_Groff

My own opinion is that futures aren't really great API at all; the best you can say about them is that they let you encode suspendable computations in an environment that doesn't directly support them. By having deeper support for suspendable contexts in the language, we can implement better APIs for running tasks in parallel and joining them than futures. The future example in the proposal is more intended to show you can still do that, if you really want, than intended to be an end in itself.

Can you (or anybody else) explain to me in more detail why you think futures aren't a great API and what a better API we could build on top of a hypothetical Swift async/await model?

[Nevin]

I do not understand what is being proposed.

I have not used a language with async / await before, and the proposal does not explain what they mean in a way that I can understand.

To me, “async” should mean that something is done asynchronously, and “await” should mean that execution is blocked while waiting for something asynchronous to complete.

But the proposal text says otherwise.

The proposal text explains that “async” does not actually do anything asynchronous, as that is left to libraries like Dispatch.

And it states that “await does not block flow of execution”.

So if async does not kick off an asynchronous operation, and await does not block while waiting for it to complete, then I have no idea at all what they are supposed to do.

• • •

Furthermore, the proposal says that async functions can only be called from other async functions, which to me sounds like it defeats the purpose. The purpose of a keyword like async, in my mind, should be exactly to mark the departure from synchronous code into asynchronous code.

But again, the proposal claims that async is not actually asynchronous, so I really don’t comprehend its goal or purpose.

The proposal also says that regular programmers should never have to call beginAsync at all, which makes no sense to me. Beginning asynchronous operations from synchronous ones seems like it should be the primary objective of a proposal in this space.

And I cannot even begin to fathom what suspendAsync is for. The explanation and examples in the proposal do not enlighten me whatsoever. Its purpose is a complete mystery to me.

• • •

I have read the entire proposal from beginning to end, and I do not understand it.

The motivation section makes sense: I can understand that the pyramid-of-doom from nested callbacks is a problem worth addressing.

But I do not understand what async / await are intended to do, and I do not understand how they might eliminate the pyramid of doom.

I would like to request that the proposal should include much more basic explanations, for people who have never seen async / await before. To explain what those keywords are actually intended to do.