[Pitch #2] Async/await

Hey all,

Here's a second pitch of the async/await proposal, which has been revised based on feedback from the first pitch thread.

This second document has the following changes from the first pitch:

  • One can no longer directly overload async and non-async functions. Overload resolution support remains, however, with additional justification.

  • Added an implicit conversion from a synchronous function to an asynchronous function.

  • Added await try ordering restriction to match the async throws restriction.

  • Added support for async initializers.

  • Added support for synchronous functions satisfying an async protocol requirement.

  • Added discussion of reasync.

  • Added justification for await not implying try.

  • Added justification for async following the function parameter list.

    Doug

28 Likes

This revision of the async/await proposal has made a very solid pitch even better. Fantastic, and congratulations.

I've got two pieces of feedback on closures:

A closure can have async function type. Such closures can be explicitly marked as async as follows:

{ () async -> Int in
  print("here")
  return await getInt()
}

Fully agree with this as-is, but it would be nice ™ to have a shorthand that doesn't require spelling out the rest of the type just to mark a closure as async; there is precedent for allowing certain shorthands for closure expressions only, and here it seems reasonable and a significant enough readability win to allow { async in ... }.

An anonymous closure is inferred to have async function type if it contains an await expression.

let closure = { await getInt() } // implicitly async

let closure2 = { () -> Int in     // implicitly async
  print("here")
  return await getInt()
}

Fully agree with inference of async when there is an await expression. However, if the argument for async being where it is is that it's part of the type, then it seems weird to allow inference of async in spite of a spelled-out type here where what's actually spelled is a valid but different type (i.e., () /* not async */ -> Int). I would suggest that, any time a return arrow appears before in, async should be required if the closure is in fact async.

16 Likes

+1

Seems to be the same with throws

let a = { () -> Int in
    throw NSError()
    return 3
}
5 Likes

Awesome, thank you for the update to this critical bedrock proposal!

Here are some more thoughts on this:

Awesome! I really love how you solved the overloading issue.

Thanks for keeping this split out as a separate proposal. I agree this is separable and can be added incrementally later.

While I agree that asynchrony should be orthogonal from error handling, I'm a bit concerned that we're going to get a bunch of await try keyword soup throughout the language (this also came up in Tony's for/in proposal).

Random idea on this aligned with your direction: Have you all considered adding a new word to represent the union of await try? I really have no idea what this should be, and it definitely shouldn't be awaitry :-) but if there were a single short word that could be used to mark these call sites, I think it could be valuable to reduce cognitive burden and make Swift code more beautiful.

In any case, such an idea is purely additive to the base proposal and is sugary and likely to draw bikeshedding, we could consider it at any time in the future if/when keyword soup becomes a problem in practice.


Random new thoughts:

  • Is inout a thing we want to support for async functions? Doesn't the problem identified in the actors proposal (where a very long 'access' to values are opened) apply to async functions? I supposed this would prohibit mutating async methods on structs, but I'm curious if this will be more of a bear trap than a useful thing in practice. Have you thought much about this?

  • The discussion about executors seems inappropriate for this proposal since actors are conceptually layered on top of this one. I'd recommend moving that to the actor proposal.

  • "From the caller's perspective, async calls behave similarly to synchronous calls, except that they may execute on a different executor, requiring the task to be briefly suspended. " Beyond the layering issue, I don't think this is technically true. The core difference here is that async functions can be suspended. Actors happen to suspend them for one reason, but there are other reasons as well. This can be fixed by hoisting this whole section to the actors proposal.

  • " Special functions like deinit and storage accessors cannot be async ." ==> what about subscripts? I'd recommend calling out your position explicitly, because they live somewhere between stored properties and normal functions.

Overall, this proposal is looking really really great. Nice work!

-Chris

8 Likes

subscript declarations are a kind of storage declaration, so their accessors are storage accessors. (A storage accessor doesn't have to be (directly) accessing an actual stored value, of course.) We don't have a spec, but that's a pretty standard use of terminology that's also in e.g. the ownership manifesto.

4 Likes

Cool, makes sense to me. I'd recommend writing this into the proposal a bit more explicitly, because I suspect I'm not the only one that isn't aware of this interpretation. Thx!

Attempt? That was the first word that popped up in my mind as a non-native speaker. It covers try and starts with “a” as well. That’s nice since both would pop-up in code completion from the first character.

Synonyms for await include expect - which doesn’t imply try to me - and anticipate - which is also a tad long.

My 2 humble cents.

3 Likes

tr-ait? :smiley:

4 Likes

I find it difficult to come up with such a word, after looking up on the thesaurus. And I think this difficulty strengthens the position that await shouldn't imply try.

1 Like

Hope?

20 Likes

Let’s not get sidetracked by this.

13 Likes

Unfortunately, { async in ... } already has a meaning as a closure that has a single parameter that we name async. It might not happen in practice, but another syntactic micro-optimization on closure syntax that conflicts with an existing syntactic micro-optimization doesn't seem worthwhile to me.

I also suspect the need to write async here will be greatly reduced by the introduction of the implicit conversion from synchronous functions to asynchronous functions.

As @Lantua notes, throws behaves this way. I don't think we should deviate from that.

I'm going to ignore this bike-shed for now ;)

I have thought about it a bit, and noted that inout parameters to async functions are still fine when working with local variables. That's very much in the spirit of value semantics being generally safe.

The problem identified in the actors proposal with long "access" to values passed inout is addressed by banning actor instance members from being passed inout to an async function. I expect we would do the same for local variables that have been captured by a closure/function that "may execute concurrently with" the place where they were defined.

You're absolutely right, I'll move the discussion. Thank you!

I'll clarify that this correct-but-clinical term "storage accessors" covers both properties and subscripts.

Doug

Warning about this is probably necessary QoI; I can see people reaching for this, and if it's parsed as an parameter they'll be very confused.

4 Likes

Would expect make sense as await try combo?

All make sense, thanks Doug

I guess I still don't really understand the difference here between async actor methods and normal async methods for the purpose of inout handling. Both are interruptable, and both have the same issue with local variable and closures. I think actors are actually a bit safer here because we can ban reference captures in "actor sendability of closures", wherease normal async methods can allow arbitrary recursion through closures without that check.

-Chris

I still think task cancellation should be automatically checked at every suspension point unless it has been disabled using something like Task.withoutInterruption. For the sake of discussion, let's assume that's the design. In that case every suspension point would potentially throw a cancellation error, thus making await naturally imply try. I know this isn't the thread to discuss cancellation policy but I wanted to point out this relationship.

There isn't a difference between async actor methods and normal async methods. Both can have inout parameters. The restrictions we need are on which arguments you can provide, to ensure that you have exclusive access to the argument to the inout even across a suspension. The actor proposal states the restriction for actor-isolated instance members:

Actor-isolated stored properties can be passed into synchronous functions via inout parameters, but it is ill-formed to pass them to asynchronous functions via inout parameters. For example:

func modifiesSynchronously(_: inout Double) { }
func modifiesAsynchronously(_: inout Double) async { }

extension BankAccount {
  func wildcardBalance() async {
    modifiesSynchronously(&balance)        // okay
    await modifiesAsynchronously(&balance) // error: actor-isolated property 'balance' cannot be passed 'inout' to an asynchronous function
  }
}  

This restriction prevents exclusivity violations where the modification of the actor-isolated balance is initiated by passing it as inout to a call that is then suspended, and another task executed on the same actor then fails with an exclusivity violation in trying to access balance itself.

It doesn't matter whether modifiesAsynchronously is on an actor or not; you still can't pass &self.balance to it.

However, it would be fine to pass &myLocalVariable to it so long as we've already ensured that myLocalVariable doesn't have captures that can run concurrently. That's partially implemented as a warning in the experimental concurrency mode, and I promised (somewhere these threads) to bring that into one of the proposals... but haven't done so yet.

Doug

2 Likes

One thing I'm missing in this proposal - await must be used in an asynchronous context, so how do you get into such a context? The only example I can find in the proposal is being inside an async function, which looks like a chicken and egg situation.

1 Like

I'm wondering if someone can confirm what the XCTest / unit testing story is like.

I'd like to call this out to be sure. I expect it is no different because it seems a synchronous function does not complete until all the asynchronous functions it calls have completed, too. But I'm not sure what effect suspension points and surrendering stacks and threads will have, either.

Riffing on the teacher/college example from the pitch, would this work as I expect with XCTest?

func testHireTeacherHappyPath() throws {
    let mockCollege = College(...)
    
    let teacher = await try Teacher(hiringFrom: mockCollege)
    
    XCTAssertEqual(teacher.faculty, "Faculty of Arts")
}

Specifically:

  1. Will the test be suspended for the initialiser to complete before the test completes?
  2. Will some asynchronously thrown error be caught by the test runner?

If not, what changes need to be made to XCTest to accomodate this?