[Concurrency] Asynchronous functions

At first blush, perhaps. But on reflection, this might be workable. After all, we can prefix a single try even if a chain has multiple throwing functions, and we don’t have try and retry just because we have throws and rethrows (for obvious reasons).

It seems natural to say that if you’re awaiting a task being completed, that task could fall: in real life, if I’m awaiting a delivery, that delivery could be damaged en route, lost in the postal system, returned to sender, etc.

To reiterate a point I made in a different thread about this:

7 Likes

I'd like to draw some attention towards and try to elaborate a bit on a part of this proposal that I don't think has been discussed yet: async initializers in classes.

Property Initialization

TL;DR = async property initializers should be disallowed.

While the proposal excludes the possibility that property getters and setters will be async, it does not specifically discuss default property values (i.e., property initializers) and whether those values are allowed to be evaluated asynchronously. Let's consider this example:

func databaseLookup() async -> String { /*...*/ }

class Teacher {
  var employeeID : String = await databaseLookup("DefaultID")
  // ...
}

The initializer for employeeID creates problems for all other explicit and implicit class initializers, because those initializers would have to be exclusively async. This is because the class's designated initializer is the one who will implicity invoke the async function databaseLookup to initialize the employeeID property. Any convenience initializers eventually must call a designated initializer, which is async and thus the convenience intializer must be async too.

So there's already an argument agianst async property initializers: they can lead to confusion and have significant knock-on effects when added to a class, so they're likely to be unpopular. Additionally, the reason for these knock-on effects that I just explained are likely to be confusing for programmers who are not intimately familiar with Swift's initialization procedures.

Furthermore, there is already a precedent in the language that property initializers cannot throw an error outside of its context, i.e., property initializers do not throw and thus its class initializers are not required to be throws. Instead, property initializers have to handle any errors that may arise either with a try!, try?, or do {} catch {} plus the {}() "closure application trick":

func throwingLookup() throws -> String { /*...*/ }

  var employeeID : String = {
    do {
      try databaseLookup("DefaultID")
    } catch {
      return "<error>"
    }
  }()

The closest analogue to await for error handling is try, but we cannot use the same closure application trick for await, because the closure itself will become async and applying it immediately puts us back where we started!

There are also additional problems when we consider lazy properties with an async initializer, since any use of that property might trigger an async call, though only the first use would actually do so. But, because all uses might be the first use, we would need to have all uses annotated with await, thus needlessy propagating async everywhere.

Due to these three factors, I think we can reasonably rule out all async property initializers.

Class Initialization

TL;DR = you must explicity write-out calls to an async super.init()

One of the key distinguishing features of initializers that are unlike ordinary functions is the requirement to call a super class's initializer, which in some instances is done for the programmer implicitly. Having async initializers in combination with implicit calls to them would create a conflict with the design goal of await. Specifically, that goal is to make explicit and obvious to the programmer that a suspension can occur within that expression.

Now, let's consider this set of class definitions:

struct Data {
  var x : Int = 0
  func setCurrentID(_ x : Int) { /* ... */ }
  func sendNetworkMessage() { /* ... */ }
}

class Animal {
  init () async { /* ... */}
}

class Zebra : Animal {
  var kind : Data = Data()
  var id : Int

  init(_ id : Int) async {
    kind.setCurrentID(id)
    self.id = id
    // PROBLEM: implicit async call to super.init here.
    kind.sendNetworkMessage()
  }
}

Note that Animal has a zero-argument designated initializer, so under the current rules, a call to super.init() happens just after the initialization of self.id during "Phase 1" of Zebra's init (according to my reading of the procedure). This implicit call could create problems for programmers who expect atomicity within the initializer's body to, say, send a network message immediately upon construction of a Zebra.

Thus, the simple and practical solution here is to adjust the rule to say that an implicit call to super.init will only happen if both of the following are true:

  1. The super class has a zero-argument, synchronous, designated initializer.
  2. The sub-class's initializer is declared synchronous.
7 Likes

Sounds good, no objection to the proposed rules.

Also, it's super important to allow async initializers I think, so glad we're discussing them slowly :slight_smile:

This is not addressed in the proposal, but if I'm understanding it correctly, then async functions can not be marked with @inliable or @_transparent, right?

There’s no such restriction, there doesn’t need to be, and it’s quite likely that we’ll have inlinable and transparent async functions in the standard library.

1 Like

Good point, I agree.

Good point, I agree.

-Chris

To be fully precise, this also needs to apply to global variables — really, anything except a local variable (which includes a global in a script).

Overall, these changes sound great; thanks for writing this up.

1 Like

The section Source compatibility says:

The positions of the new uses of async within the grammar (function declarations, function types, and as a prefix for let ) allows us to treat async as a contextual keyword without breaking source compatibility.

Have I overlooked something or has async as a prefix for let not been mentioned anywhere else? How can it be used and what does it do?

It's in the Structured concurrency proposal. Its thread in these forums is [Concurrency] Structured concurrency. Be sure to check out the other concurrency proposals and the concurrency roadmap too to get an overall idea of the full pitch.

2 Likes

Ah, thank you for the clarification! I was going to read the other proposals but have not had the time yet.

1 Like

My thought was that because the caller of an async function needs to save and restore its state at the suspension point, it seemed incompatible with inlining which removes the saving and restoring of a caller's state.

I think I have some fundamental misunderstanding in how inlining works.

If a function needs to save and restore state, it’s perfectly capable of doing that in the middle of its normal execution; it’s not constrained to only doing it during a call.

The main interesting interaction of inlining with the async model is that it means that our internal representation of functions can’t just assume that they run on a consistent actor, the way the language works: if it did, we wouldn’t be able to inline across executor boundaries. Instead we internally have to explicitly represent the executor-switching. But we want to do that anyway so that we can eliminate unnecessary executor-switching. All of that is just compiler technical design, though; it doesn’t shape the language.

7 Likes

Agreed. @kavon, do you mind raising these as a pull request against the proposal itself?

Doug

1 Like

I might want to report the progress of long running functions back to the user using a progress bar. How would I do this with async functions?

Speculation: a library provided progress actor for a task or task group that owns a unit value might be useful API.

I might want to report the progress of long running functions

That one is on my radar but we've not gotten to it yet. There exist some facilities to help with this with Foundation's Progress so we'd want to play nice with this most likely, but at the same time make it nicer to use with async functions.

Maybe that one deserves a separate thread actually -- progress monitoring with asynchronous functions?

I'd definitely welcome some more examples how people use these and what they'd want from such API.

8 Likes

Is there any plan to update the proposals (and start a new thread) after incorporating feedback from the discussions? It isn't clear how you're responding to things and whether you agree or disagree with much of it -- it would be helpful to see where things stand.

5 Likes

We should be updating the appropriate threads whenever we make a change, but please point out anything we've overlooked.

I'm not seeing any such updates. I'm not sure if that means the proposals aren't getting changed, or if the updates aren't getting propagated :slight_smile:

It would probably also be good to start a "pitch #2" thread if a major design change is accepted into any of the proposals.

-Chris

3 Likes

We since made some slight naming improvements in [Concurrency] Structured concurrency - #4 by Dante-Broggi in the actual's full text but since the first post is by yourself I was not able to update it.

I would suggest updating the first post in each of those 5 we started with just a link to the proposals' full texts.

We addressed many minor typos which are confusing people and early on took in feedback on renaming nurseries, but the initial post still uses that while the proposal (nor impl) does not. People look at the forums post which also misses some sections from the "full" writeup and then are confused, it'd really help to have the "full text" as the source of truth.

Agreed though that for a big refresh a noticable ping or new thread will be good... these have been small polish so far though.

4 Likes