[Concurrency] Asynchronous functions

I agree 100%.

Personally I’m still of the opinion that the feature would be better without overloading. Having overloading rules that behave differently in different contexts sounds overwrought and confusing, and I’m skeptical that the benefit is worth it.

My colleagues at Google (most notably @gribozavr, @saeta, and @pschuh) and I came up with these questions:


  • Has any thought been given to an async analogue for rethrows?

Proposed solution: async/await

  • “Because asynchronous functions must be able to abandon their thread, and synchronous functions don’t know how to abandon a thread”—My understanding is that the real reason synchronous functions can’t abandon threads is safety: they’ve been written with the expectation of non-reentrancy on blocking calls (and exclusive access to thread-local variables). This paragraph in general is hard to understand because it leads the reader down several obviously impossible paths and is vague. I think we can just say that synchronous functions have been written with the expectation of non-reentrancy on blocking calls, and calling an async function can cause it to be re-entered upon await. If there’s something more we are trying to say here, maybe clarify it?
  • “When control returns to an asynchronous function, it picks up exactly where it was. That doesn’t necessarily mean that it’ll be running on the exact same thread it was before, …. However, many asynchronous functions are … associated with specific actors…. Swift does guarantee that such functions will in fact return to their actor to finish executing.”— It took making all of the elisions for me to come up with a theory of what this means; the “however” was confusing because actors aren’t necessarily associated with threads. Now I think what you’re saying is that there’s no (direct) way to associate an async function call with a given thread, but there is a way to associate it with a given actor (and then associating it with a global actor indirectly associates it with a thread). Is that right?
  • “model those threads as actors in Swift”—it’s not clear what that means. Can you show an example?

Suspension Points

  • “Execution context” is not defined anywhere in these proposals, but it’s used all over the place. It's probably a commonly-used term in general but I can't find an accepted definition that isn't tied to a specific language (mostly JavaScript), and the definitions for some languages (e.g. Scala) seem incompatible with what’s being said about it in the proposals. Can we please have a definition?

  • “Note that suspension points are also called out explicitly in code using explicit callbacks: the suspension happens between the point where the outer function returns and the callback starts running.”—Is “code using explicit callbacks” referring to, e.g., libdispatch uses in existing code? If so, I don't think the point where the outer function returns is relevant. If not, maybe this needs some qualification, like “async code using explicit callbacks” or something. In general, an example would help.

  • “Asynchronous programs that need to do intense computation should generally run it in a separate context.”—what does “context” mean in this, um, context?

  • “When that’s not feasible, there will be library facilities to artificially suspend and allow other operations to be interleaved.”—facilities, plural? Isn't a single empty async function enough to do that?

  • “This design currently provides no way to prevent the current context from interleaving code while an asynchronous function is waiting for an operation in a different context.”—The meaning of “context” is unclear again. Is this “execution context” or a different “context?”

    I think this sentence makes sense if we interpret "context" as "call stack." If that interpretation is correct maybe we should just use “call stack” everywhere.

Asynchronous calls

  • “If the callee’s executor is different from the caller’s executor”—it took me a while to puzzle through this, thinking, “why would we want to forbid the suspension from occurring if the executors match, since it is semantically equivalent?” before I realized this wasn’t an “If and only if.” Still, it might simplify the model if we just say there’s a suspension at each async call.
  • “If the callee’s executor is different from the caller’s executor, a suspension occurs”—Presumably in the caller's executor? It would be clearer if we always said which executor was suspending.
  • “During the return, if the callee’s executor is different from the caller’s executor, a suspension occurs”—Presumably in the callee’s executor? It’s not clear why we need to say that, since the async function executed in the callee is finished and its bit of call stack goes away, there doesn’t seem to be anything to suspend. I don’t think it makes any semantic difference, so perhaps another opportunity to simplify the model?
  • “From the caller's perspective, async calls behave similarly to synchronous calls, except that they may execute on a different executor, requiring the task to be briefly suspended.”—I think this glosses over an important point that must factor into the caller’s perspective: that during suspension other partial tasks may proceed on the same executor. That means data to which the callee has no access—such as the properties of the caller’s actor—may appear to mutate underneath the caller during the call.
  • Could you provide more low-level details about the expected implementation? Where is the overhead? When is memory allocated? Does one partial task object in practice correspond to one function invocation (and reused over multiple suspensions), or one continuation (not reused across suspensions)? Are partial tasks always allocated on the heap? Where do reference counting operations on partial tasks happen? Are they contended?

Detailed Design — Asynchronous Functions

  • “it is the ‘inner’ function type that is async, consistent with the usual rules for such references”—what does this mean? Could you show an example?
  • The handling of overloading is pretty slick!

Detailed Design — Autoclosures

  • “At first glance, the await expression implies to the programmer that there is a suspension point prior to the call to computeArgumentLater(_:)”—this is just the usual problem of autoclosures, is it not?
  • “means that closure would be inferred to have async function type”—That conclusion isn’t obvious. Autoclosures mislead about execution order and whether something will be executed; that’s just how it is. The inference of whether something is async needn’t be done on a purely syntactic basis. The fact the await is in an autoclosure could change the inference.
  • “An equivalent rewriting of the call should be…”—Yeah, again that assumes the usual correlation between syntax and semantics holds, but it doesn’t in the case of autoclosures. It’s not obvious that this is how it has to be.
4 Likes

John's answer to my same question (which has since been buried in 100 posts):

2 Likes

If I have 2 functions with the same declaration where one is sync and another async how do I call the sync one specifically inside another async function? As far as I understood the async one will be picked up automatically due to being called inside of the async scope.

You can do this:

1 Like

I'm also interested in more details here. The most salient for me is the overhead associated with a potential suspension point that does not suspend. I believe this is the scenario John was addressing with:

When I first read the proposal, I was confused by the narrow use of "suspension point" to refer to a place where the thread is definitely given up, and the broader use that includes potentially suspending points. I drafted a PR that I think clears this up:

Interesting; this is going in a different direction than we are suggesting in our remarks on the “Asynchronous calls” section. We think there might be an opportunity to simplify the programmer-level semantic model significantly by thoroughly erasing the distinction between potential suspension and actual suspension. AFAICT there's never—or almost never—an observable difference to the programmer and the choice to actually suspend or not seems like it should be up to the implementation.

I'm not quite sure what you mean by "erasing the distinction," but I wanted to emphasize that the reason for making a distinction between calls with respect to suspension, through the await keyword, is nicely stated in your earlier remarks on the "Asynchronous calls" section of the proposal:

Regardless, this point should probably be clarified, possibly with an example, in the proposal. Here's one example where the suspension is observable:

func g() async { /* ... */ }

actor class A {
  var i : Int
  
  func mutate(_ newVal : Int) async { i = newVal }

  func f() async {
    let currentI = i
    await g()
    assert(i == currentI) // not always true!
  }
}

For an individual instance of actor A, if we reach the call to g while in method f and suspend, that actor instance is free to process mutate method calls from other threads while g is executing. Thus, the assertion may fail. The purpose of await is a signal to programmers that they must be careful about reusing stale values in the function.

That touches the entire reentrance topic/“elephant in the room” which we really need to discuss more, not having non-reentrant actors is quite difficult to work with...

3 Likes

I think you've misunderstood me. The distinction we're talking about is the one between potential suspension points and places where suspension is guaranteed to occur. Both are marked with await.

Erasing the distinction would mean allowing the scheduler to suspend at every async call and return, even when e.g. calling a function on the same actor? I’m not sure that’s a good idea; it seems to me that that would make certain kinds of abstraction very difficult to write and force a lot of awkward restructuring. I think programmers should be able to reason about what happens without a suspension across calls if they need to, and I don’t think they expect calls from an actor to itself to themselves cause a suspension.

Yep.

I’m not sure that’s a good idea; it seems to me that that would make certain kinds of abstraction very difficult to write and force a lot of awkward restructuring.

What abstraction and restructuring do you have in mind?

I think programmers should be able to reason about what happens without a suspension across calls if they need to, and I don’t think they expect calls from an actor to itself to themselves cause a suspension.

:thinking:
But the callee is still free to await a call to a different actor, so you can't exactly reason about much on that basis without breaking the abstraction barrier between functions, i.e. you have to know about the implementation of the callee to know that there's no suspension involved in the call. It seems to me that the way to know that there's no suspension is that the compiler hasn't forced you to await, and anything else makes for a really fragile programming model.

From the proposal:

A call to a value of async function type (including a direct call to an async function) introduces a suspension point. Any suspension point must occur within an asynchronous context (e.g., an async function). Furthermore, it must occur within the operand of an await expression.

While I understand there may be implementation complexities, at a language level I don't understand why calling an async function without the await keyword wouldn't be a legitimate way to serialize a function (and its own internal async calls) in the current execution context. For things like filesystem operations that are commonly called synchronously (even though maybe they shouldn't :roll_eyes:) it would allow a single function implementation to serve all callers.

Just a quick question (without giving my opinion since the reasoning isn't available yet). What kind of scenario could invoke the second rule? Since it's no longer possible to have overload differing only by async-ness as per the first rule.

It’s pervasive in ObjC imports unless we rewrite method names, which just creates more problems.

1 Like

It's not just Objective-C interoperability. It'll happen in Swift for the same reasons when someone adds async versions alongside existing completion-handler APIs (of which there are many):

func post(_ data: Data, to url: URL, completionHandler: ((Response) -> Void)? = nil) { ... }
func post(_ data: Data, to url: URL) async { ... }

Note that this is allowed even with the proposed change, because post(_:to:completionHandler:) and post(_:to:) have different method names and signatures even ignoring the async. Without overload resolution rules like the ones proposed, the expression post(data, to: url) will always resolve to the async version, breaking existing code.

I promise to write up a detailed discussion of this, because we went pretty far down the design and implementation path for removing overloading before deciding that we needed overloading still.

Doug

1 Like

No worries :grin:. I'm not trying to criticize the decision (esp. before the reasoning is provided). I'm just a little curious since I can see a lot of work is done to eliminate the overloading and as you said somehow some kinks still remain.

Thanks for the hard work!

At some point, one of those async calls you make may have to suspend, and you'd be left with no choice but to block. Blocking goes against the goals of the asynchronous model.

Doug

Composing synchronous functionality on top of asynchronous input. Earlier in the thread, I posted a TStream and ArduousTStream as an example of the diamond problem, where a library would like to encapsulate a control-flow diamond where one branch is asynchronous and the other synchronous. If that carries unacceptable performance overhead, then they might have to thrust the diamond upon their clients.

Here's a simplified version that synchronously parses values off a local buffer, which is asynchronous refilled:

struct TStream {
  // Returns nil when we've exhausted our source
  public mutating func next() async -> T? {
    if mightNeedRefill { await refillInternalBuffer() }
    // ... synchronously vend a T from our internal buffer
  }
}

struct ArduousTStream {
  public var mightNeedRefill: Bool
  public mutating func refillInternalBuffer() async {}

  // Returns nil when we've exhausted our source, if the client remembers
  // to refill the buffer when it needs refilling. Otherwise might return
  // an early nil
  public mutating func next() -> T? {
    // ... synchronously vend a T from our internal buffer
  }
}

If TStream.next() always suspends execution, even though refilling its internal buffer is an infrequent operation, I'm worried that will thrust many libraries into designing something akin to ArduousTStream. But, if the overhead is more on the order of shuffling around some registers, slightly worse data locality for the stack, etc., then this would be far more acceptable to vend TStream.next().

Furthermore, a definite suspension point won't disappear if TStream.next() is marked @inlinable, but (in theory) much of the other overhead could. (If we go the route where suspension points can be sunk into conditionally executed code, then we no longer have the rule that an await always suspends, rather await always suspends until inlining unblocks sinking.)

Michael, I think you're misunderstanding my point, which is not to constrain the implementation, but to simplify the semantic model for users. I'm suggesting that the semantic model is simpler if, at any await, the implementation is free to actually-suspend or not. If you want to write the implementation so that it only shuffles some registers in these particular cases, you would still be free to do so.

My point is simply that it complicates the user's semantic model to think about whether a particular await is actually going to suspend because it's crossing actors, and it doesn't seem to buy much in terms of reasoning about effects because the awaited function itself might await a call that runs on another actor, or be evolved to do so sometime in the future.

Yes, you can reason that calls to the same actor are going to perform better, but that could reasonably be considered an implementation artifact, just like creating a struct instance is going to be more efficient than creating the equivalent class instance. AFAICT it doesn't affect the correctness of the code.