[Concurrency] Asynchronous functions

[Chris_Lattner3 (Chris Lattner)]

Thanks for the update @ktoso, as a hobbyist who participates in swift-evolution in spare time, I really appreciate it.

On this thread, I have serious concerns about the subtyping relationships between async and non-async functions and think we should follow the established model of try. Has there been any thoughts or comments on that?

It looks like the overloading rules aren't quite as bad as I was concerned about, but the protocol conformance, implicit conversion of function types, and other issues are still important to get nailed down. I can't tell if they were discussed and dismissed as unimportant, if there is an active discussion (if so, involving the community would be great), or if it was agreed to make some model change.

-Chris

[BigSur ({ @MainActor  in M1.Ultra }(Swift))]

@Chris_Lattner3

about the subtyping relationships between async and non-async functions and think we should follow the established model of try .

I also think so

[Nathan_Gray]

@Chris_Lattner3

On this thread, I have serious concerns about the subtyping relationships between async and non-async functions and think we should follow the established model of try.

I agree 100%.

It looks like the overloading rules aren't quite as bad as I was concerned about, but the protocol conformance, implicit conversion of function types, and other issues are still important to get nailed down.

Personally I’m still of the opinion that the feature would be better without overloading. Having overloading rules that behave differently in different contexts sounds overwrought and confusing, and I’m skeptical that the benefit is worth it.

[dabrahams (Dave Abrahams)]

My colleagues at Google (most notably @gribozavr, @saeta, and @pschuh) and I came up with these questions:

---

• Has any thought been given to an async analogue for rethrows?

Proposed solution: async/await

• “Because asynchronous functions must be able to abandon their thread, and synchronous functions don’t know how to abandon a thread”—My understanding is that the real reason synchronous functions can’t abandon threads is safety: they’ve been written with the expectation of non-reentrancy on blocking calls (and exclusive access to thread-local variables). This paragraph in general is hard to understand because it leads the reader down several obviously impossible paths and is vague. I think we can just say that synchronous functions have been written with the expectation of non-reentrancy on blocking calls, and calling an async function can cause it to be re-entered upon await. If there’s something more we are trying to say here, maybe clarify it?
• “When control returns to an asynchronous function, it picks up exactly where it was. That doesn’t necessarily mean that it’ll be running on the exact same thread it was before, …. However, many asynchronous functions are … associated with specific actors…. Swift does guarantee that such functions will in fact return to their actor to finish executing.”— It took making all of the elisions for me to come up with a theory of what this means; the “however” was confusing because actors aren’t necessarily associated with threads. Now I think what you’re saying is that there’s no (direct) way to associate an async function call with a given thread, but there is a way to associate it with a given actor (and then associating it with a global actor indirectly associates it with a thread). Is that right?
• “model those threads as actors in Swift”—it’s not clear what that means. Can you show an example?

Suspension Points

• “Execution context” is not defined anywhere in these proposals, but it’s used all over the place. It's probably a commonly-used term in general but I can't find an accepted definition that isn't tied to a specific language (mostly JavaScript), and the definitions for some languages (e.g. Scala) seem incompatible with what’s being said about it in the proposals. Can we please have a definition?

• “Note that suspension points are also called out explicitly in code using explicit callbacks: the suspension happens between the point where the outer function returns and the callback starts running.”—Is “code using explicit callbacks” referring to, e.g., libdispatch uses in existing code? If so, I don't think the point where the outer function returns is relevant. If not, maybe this needs some qualification, like “async code using explicit callbacks” or something. In general, an example would help.

• “Asynchronous programs that need to do intense computation should generally run it in a separate context.”—what does “context” mean in this, um, context?

• “When that’s not feasible, there will be library facilities to artificially suspend and allow other operations to be interleaved.”—facilities, plural? Isn't a single empty async function enough to do that?

• “This design currently provides no way to prevent the current context from interleaving code while an asynchronous function is waiting for an operation in a different context.”—The meaning of “context” is unclear again. Is this “execution context” or a different “context?”

  I think this sentence makes sense if we interpret "context" as "call stack." If that interpretation is correct maybe we should just use “call stack” everywhere.

Asynchronous calls

• “If the callee’s executor is different from the caller’s executor”—it took me a while to puzzle through this, thinking, “why would we want to forbid the suspension from occurring if the executors match, since it is semantically equivalent?” before I realized this wasn’t an “If and only if.” Still, it might simplify the model if we just say there’s a suspension at each async call.
• “If the callee’s executor is different from the caller’s executor, a suspension occurs”—Presumably in the caller's executor? It would be clearer if we always said which executor was suspending.
• “During the return, if the callee’s executor is different from the caller’s executor, a suspension occurs”—Presumably in the callee’s executor? It’s not clear why we need to say that, since the async function executed in the callee is finished and its bit of call stack goes away, there doesn’t seem to be anything to suspend. I don’t think it makes any semantic difference, so perhaps another opportunity to simplify the model?
• “From the caller's perspective, async calls behave similarly to synchronous calls, except that they may execute on a different executor, requiring the task to be briefly suspended.”—I think this glosses over an important point that must factor into the caller’s perspective: that during suspension other partial tasks may proceed on the same executor. That means data to which the callee has no access—such as the properties of the caller’s actor—may appear to mutate underneath the caller during the call.
• Could you provide more low-level details about the expected implementation? Where is the overhead? When is memory allocated? Does one partial task object in practice correspond to one function invocation (and reused over multiple suspensions), or one continuation (not reused across suspensions)? Are partial tasks always allocated on the heap? Where do reference counting operations on partial tasks happen? Are they contended?

Detailed Design — Asynchronous Functions

• “it is the ‘inner’ function type that is async, consistent with the usual rules for such references”—what does this mean? Could you show an example?
• The handling of overloading is pretty slick!

Detailed Design — Autoclosures

• “At first glance, the await expression implies to the programmer that there is a suspension point prior to the call to computeArgumentLater(_:)”—this is just the usual problem of autoclosures, is it not?
• “means that closure would be inferred to have async function type”—That conclusion isn’t obvious. Autoclosures mislead about execution order and whether something will be executed; that’s just how it is. The inference of whether something is async needn’t be done on a purely syntactic basis. The fact the await is in an autoclosure could change the inference.
• “An equivalent rewriting of the call should be…”—Yeah, again that assumes the usual correlation between syntax and semantics holds, but it doesn’t in the case of autoclosures. It’s not obvious that this is how it has to be.

[Jumhyn (Frederick Kellison-Linn)]

@dabrahams

Has any thought been given to an async analogue for rethrows ?

John's answer to my same question (which has since been buried in 100 posts):

@John_McCall

It's hard to say. I can certainly imagine situations where it might be useful, but I'm not sure there are enough to justify the complexity cost, which behind the scenes would be quite substantial — we'd essentially have to rig up a complete task just to call the function, then throw it away. The thing is that many higher-order functions probably need to think carefully about how they ought to work with an async function — guaranteeing sequential use is not necessarily what clients would actually want.

The most important use case would probably be withFoo functions that introduce a scoped value, and I'd really prefer to address those with a more targeted coroutine feature — they're problematic for a lot of other things besides async .

[rumnat (Muratov Anton)]

If I have 2 functions with the same declaration where one is sync and another async how do I call the sync one specifically inside another async function? As far as I understood the async one will be picked up automatically due to being called inside of the async scope.

[Lantua]

@rumnat

how do I call the sync one specifically inside another async function?

You can do this:

@Douglas_Gregor

wrap the call in a (non-async) closure, e.g.,

{ foo() }()

We could probably come up with a clearer way.

[Michael_Ilseman]

@dabrahams

Could you provide more low-level details about the expected implementation? Where is the overhead? When is memory allocated? Does one partial task object in practice correspond to one function invocation (and reused over multiple suspensions), or one continuation (not reused across suspensions)? Are partial tasks always allocated on the heap? Where do reference counting operations on partial tasks happen? Are they contended?

I'm also interested in more details here. The most salient for me is the overhead associated with a potential suspension point that does not suspend. I believe this is the scenario John was addressing with:

@John_McCall

Two things. The first is that the function is split, so there's extra low-level overhead on function suspend/resume; also, spilling values into an async frame is likely to have somewhat worse locality than spilling to the C stack. The second is that the function's frame needs to be allocated off the C stack, which will happen using the task-local allocator.

When I first read the proposal, I was confused by the narrow use of "suspension point" to refer to a place where the thread is definitely given up, and the broader use that includes potentially suspending points. I drafted a PR that I think clears this up:

Clarify definite vs potential suspension points

[dabrahams (Dave Abrahams)]

@Michael_Ilseman

When I first read the proposal, I was confused by the narrow use of "suspension point" to refer to a place where the thread is definitely given up, and the broader use that includes potentially suspending points. I drafted a PR that I think clears this up:

Interesting; this is going in a different direction than we are suggesting in our remarks on the “Asynchronous calls” section. We think there might be an opportunity to simplify the programmer-level semantic model significantly by thoroughly erasing the distinction between potential suspension and actual suspension. AFAICT there's never—or almost never—an observable difference to the programmer and the choice to actually suspend or not seems like it should be up to the implementation.

[kavon (Kavon Farvardin)]

@dabrahams

We think there might be an opportunity to simplify the programmer-level semantic model significantly by thoroughly erasing the distinction. AFAICT there's never—or almost never—an observable difference to the programmer and the choice to actually suspend or not seems like it should be up to the implementation.

I'm not quite sure what you mean by "erasing the distinction," but I wanted to emphasize that the reason for making a distinction between calls with respect to suspension, through the await keyword, is nicely stated in your earlier remarks on the "Asynchronous calls" section of the proposal:

@dabrahams

[...] during suspension other partial tasks may proceed on the same executor. That means data to which the callee has no access—such as the properties of the caller’s actor—may appear to mutate underneath the caller during the call.

Regardless, this point should probably be clarified, possibly with an example, in the proposal. Here's one example where the suspension is observable:

func g() async { /* ... */ }

actor class A {
  var i : Int
  
  func mutate(_ newVal : Int) async { i = newVal }

  func f() async {
    let currentI = i
    await g()
    assert(i == currentI) // not always true!
  }
}

For an individual instance of actor A, if we reach the call to g while in method f and suspend, that actor instance is free to process mutate method calls from other threads while g is executing. Thus, the assertion may fail. The purpose of await is a signal to programmers that they must be careful about reusing stale values in the function.

[ktoso (Konrad 'ktoso' Malawski 🐟🏴‍☠️)]

That touches the entire reentrance topic/“elephant in the room” which we really need to discuss more, not having non-reentrant actors is quite difficult to work with...

[dabrahams (Dave Abrahams)]

@kavon

I'm not quite sure what you mean by "erasing the distinction," but I wanted to emphasize that the reason for making a distinction between calls with respect to suspension, through the await keyword, is nicely stated in your earlier remarks on the "Asynchronous calls" section of the proposal:

I think you've misunderstood me. The distinction we're talking about is the one between potential suspension points and places where suspension is guaranteed to occur. Both are marked with await.

[John_McCall]

Erasing the distinction would mean allowing the scheduler to suspend at every async call and return, even when e.g. calling a function on the same actor? I’m not sure that’s a good idea; it seems to me that that would make certain kinds of abstraction very difficult to write and force a lot of awkward restructuring. I think programmers should be able to reason about what happens without a suspension across calls if they need to, and I don’t think they expect calls from an actor to itself to themselves cause a suspension.

[dabrahams (Dave Abrahams)]

@John_McCall

Erasing the distinction would mean allowing the scheduler to suspend at every async call and return, even when e.g. calling a function on the same actor?

Yep.

I’m not sure that’s a good idea; it seems to me that that would make certain kinds of abstraction very difficult to write and force a lot of awkward restructuring.

What abstraction and restructuring do you have in mind?

I think programmers should be able to reason about what happens without a suspension across calls if they need to, and I don’t think they expect calls from an actor to itself to themselves cause a suspension.

But the callee is still free to await a call to a different actor, so you can't exactly reason about much on that basis without breaking the abstraction barrier between functions, i.e. you have to know about the implementation of the callee to know that there's no suspension involved in the call. It seems to me that the way to know that there's no suspension is that the compiler hasn't forced you to await, and anything else makes for a really fragile programming model.

[numist (Scott Perry)]

From the proposal:

A call to a value of async function type (including a direct call to an async function) introduces a suspension point. Any suspension point must occur within an asynchronous context (e.g., an async function). Furthermore, it must occur within the operand of an await expression.

While I understand there may be implementation complexities, at a language level I don't understand why calling an async function without the await keyword wouldn't be a legitimate way to serialize a function (and its own internal async calls) in the current execution context. For things like filesystem operations that are commonly called synchronously (even though maybe they shouldn't ) it would allow a single function implementation to serve all callers.

[Lantua]

@Douglas_Gregor

[We are] Banning overloading of async and synchronous methods that otherwise have the same signature (but see the below about overload resolution).

[...]

We are not inclined to eliminate the overload-resolution rule that prefers async in asynchronous contexts and synchronous functions in synchronous contexts.

Just a quick question (without giving my opinion since the reasoning isn't available yet). What kind of scenario could invoke the second rule? Since it's no longer possible to have overload differing only by async-ness as per the first rule.

[John_McCall]

It’s pervasive in ObjC imports unless we rewrite method names, which just creates more problems.

[Douglas_Gregor]

@John_McCall

It’s pervasive in ObjC imports unless we rewrite method names, which just creates more problems.

It's not just Objective-C interoperability. It'll happen in Swift for the same reasons when someone adds async versions alongside existing completion-handler APIs (of which there are many):

func post(_ data: Data, to url: URL, completionHandler: ((Response) -> Void)? = nil) { ... }
func post(_ data: Data, to url: URL) async { ... }

Note that this is allowed even with the proposed change, because post(_:to:completionHandler:) and post(_:to:) have different method names and signatures even ignoring the async. Without overload resolution rules like the ones proposed, the expression post(data, to: url) will always resolve to the async version, breaking existing code.

I promise to write up a detailed discussion of this, because we went pretty far down the design and implementation path for removing overloading before deciding that we needed overloading still.

Doug

[Lantua]

@Douglas_Gregor

I promise to write up a detailed discussion of this, because we went pretty far down the design and implementation path for removing overloading before deciding that we needed overloading still.

No worries . I'm not trying to criticize the decision (esp. before the reasoning is provided). I'm just a little curious since I can see a lot of work is done to eliminate the overloading and as you said somehow some kinks still remain.

Thanks for the hard work!

[Douglas_Gregor]

@numist

While I understand there may be implementation complexities, at a language level I don't understand why calling an async function without the await keyword wouldn't be a legitimate way to serialize a function (and its own internal async calls) in the current execution context.

At some point, one of those async calls you make may have to suspend, and you'd be left with no choice but to block. Blocking goes against the goals of the asynchronous model.

Doug