Using async/await with existing non-concurrent code

[QuinceyMorris]

@adamkemp

It feels like you're arguing that "it's possible to write bugs with async versions of these methods", which...yeah.

Sure, if you change the intended semantics of a function — and that's what adding async does — then code that previously worked in that function stops working.

I don't think that changing those semantics is genuinely on the table, though. There is a long-standing and complex set of relationships amongst AppKit or UIKit functions, as well as a lot of other frameworks with similar synchronous behaviors, which all fall apart when you change their semantics. I can think of 2 immediate non-trivial examples of the larger problem:

1. viewWillAppear is provided so you can do things before a view appears, obviously. If you change it to run asynchronously, then your code in the function can't know whether it actually runs before the view appears.

2. Any function that returns a value — and there are plenty in this world of overrides, delegates, protocol conformances, etc — cannot be run asynchronously.

What you would need is a new system of events and intervention points, specifically designed for async usage. That's probably a great idea, but it's not going to happen as part of the current set of proposals, AFAICT.

(I'm not trying to prevent you from discussing these issues, but they weren't exactly the intended topic of this thread.)

[jayton (Jens Ayton)]

@adamkemp

When it comes to "should async void be allowed, and if so then what does that mean"

The elephant in the room here is structured concurrency. The structured concurrency pitch is still in a draft phase, but it’s clearly a central part of the core team’s thinking about the problem space. The pitch doesn’t really go into much depth on what “structured concurrency” is, let alone why it’s (perceived as) better; for that, one must look elsewhere.

The fact that scoped futures have lightweight syntax sugar in the pitch and runDetached doesn’t is hardly an oversight; async void is to structured concurrency as unconstrained goto is to structured programming. From this perspective, @asyncHandler is already a pretty big concession for compatibility with existing frameworks.

[lassejansen]

@QuinceyMorris

There is a long-standing and complex set of relationships amongst AppKit or UIKit functions, as well as a lot of other frameworks with similar synchronous behaviors, which all fall apart when you change their semantics.

I don't think @adamkemp is suggesting that all UIKit/AppKit methods should be converted to async. Instead I think the idea is that methods written by framework users (including overrides) can be marked as async entry points if it's a method that returns Void.

@QuinceyMorris

1. viewWillAppear is provided so you can do things before a view appears, obviously. If you change it to run asynchronously, then your code in the function can't know whether it actually runs before the view appears.

I think the idea would be that the method runs synchronously until the first occurrence of await. Up to await all code would run before the view appears.

@QuinceyMorris

2. Any function that returns a value — and there are plenty in this world of overrides, delegates, protocol conformances, etc — cannot be run asynchronously.

It's not required that every delegate/override etc must be able to run asynchronously. It would just be nice if less boilerplate would be required to launch from a synchronous context into asynchronous code.

Taking the earlier example it would be nice if we could get the semantics of this code:

extension ViewController {

  override func viewWillAppear(_ animated: Bool) {
    super.viewWillAppear(animated)
    Task.runDetached { @MainActor in
      do {
        let (signed, signature) = await try library.sign(data, using: pass)
        await passWasSigned(signed: signed, signature: signature)
      } catch {
        showError( error)
      }
    }
  }
}

by writing something like this:

extension ViewController {

  @someAsyncMarker
  override func viewWillAppear(_ animated: Bool) {
    super.viewWillAppear(animated)
    do {
      let (signed, signature) = await try library.sign(data, using: pass)
      await passWasSigned(signed: signed, signature: signature)
    } catch {
      showError( error)
    }
  }
}

The marker would do these things:

1. Run the function synchronously untill the first occurrence of await (possibly in different branches).
2. Insert a Task.runDetached at that point.
3. Use the "current" actor (@MainActor in this case) to run the closure.

@QuinceyMorris

What you would need is a new system of events and intervention points, specifically designed for async usage. That's probably a great idea, but it's not going to happen as part of the current set of proposals, AFAICT.

I think what @adamkemp is suggesting is that instead of using an @-annotation like in the example above, that such a transformation could happen automatically for every Void-returning async method that is called from synchronous code. That would not be a new system, just a quality of life improvement.

[Jean-Daniel]

@lassejansen

I think the idea would be that the method runs synchronously until the first occurrence of await . Up to await all code would run before the view appears.

You need a way to specify that when you call Task.runDetached to start an async scope, this call must block until the first await is reached and return at this point.

Is it possible with the current design ?

[QuinceyMorris]

@lassejansen

I don't think @adamkemp is suggesting that all UIKit/AppKit methods should be converted to async.

I don't think so either. I think @adamkemp is suggesting a strategy for callees that treats the caller as a kind of event or signal. The point I keep trying to make is that both the caller and the callee have wide-ranging expectations (semantics) regarding functions like the ones we're discussion, and we will not have any way of changing the caller's expectations.

The callee can certainly run some or all of its code asynchronously, but the asynchronous code can't necessarily run under the conditions (semantics) established by the caller.

@lassejansen

I think the idea would be that the method runs synchronously until the first occurrence of await . Up to await all code would run before the view appears.

That seems so dangerous. In code with if, switch and looping statements, it can be easy to make a mistake finding the "first" await. More subtly, the suspension point isn't where the await keyword is. The suspension point is inside a function call somewhere to the right of the await, of which there may be several, surrounded by sub-expressions evaluated before the suspension. The called function that will eventually suspend may itself execute some code synchronously before reaching the suspension point. It's going to be difficult to reason reliably about this.

I think it's fine to have syntax that says "this part of the code doesn't run synchronously or sequentially" with the rest of the function, but I think it should look non-sequential.

That's one reason why I suggested the attach { } syntax. I don't like the nesting that it introduces, but at least it's clear about what's happening and when.

[lassejansen]

@QuinceyMorris

That seems so dangerous. In code with if, switch and looping statements, it can be easy to make a mistake finding the "first" await.

The way this would probably need to work is that the compiler generates two versions for each of these "hybrid" functions: One that is used when called from a synchronous context and one for regular asynchronous use.

The function that is called from a synchronous context would essentially work as if each await was a return statement. At that point the function state would be saved and the corresponding async version of the function would be scheduled to resume in that state on the actor.

@QuinceyMorris

More subtly, the suspension point isn't where the await keyword is. The suspension point is inside a function call somewhere to the right of the await, of which there may be several, surrounded by sub-expressions evaluated before the suspension. The called function that will eventually suspend may itself execute some code synchronously before reaching the suspension point. It's going to be difficult to reason reliably about this.

I agree that this would be very hard to reason about. But if suspension points would be guaranteed to be inserted for each await if the function is called from a synchronous context (as would be the case with the approach described above) I think it would be quite simple to understand and reason about.

[adamkemp]

@QuinceyMorris you are speaking to me as if I don't know how to use UIKit or don't know how async/await works in practice. I have mentioned several times already that I in fact have years of experience using both together. I actually know what it is like to use these kinds of features with UIKit. I actually know for a fact that it works well. You're speculating that it doesn't, but you are demonstrably wrong.

[adamkemp]

@jayton

The elephant in the room here is structured concurrency. The structured concurrency pitch is still in a draft phase, but it’s clearly a central part of the core team’s thinking about the problem space.

Yes, and it is becoming increasingly clear that it was a mistake to propose async/await as a standalone feature when clearly so many decisions for async/await were made to suit this other pitch that hasn't even been fully fleshed out let alone properly explained to the community. If we can't have async void because of something in structured concurrency then that needs to be in the spec for async/await. It's an important consideration for how this feature will be used. I understand that this is a large and complex set of functionality, but even still it's not being communicated effectively.

[lassejansen]

@Jean-Daniel

Is it possible with the current design ?

I think so. Probably not by actually inserting calls to Task.runDetached, but see my other comment.

[michelf (Michel Fortin)]

@adamkemp

If we can't have async void because of something in structured concurrency...

At some point there's going to be a proposal with @asyncHandler in it, which to me sounds pretty much like what you want with async void. Perhaps that'll be a good time to discuss whether @asyncHandler should be made implicit in some circumstances (void returns in this case). There's also the possibility of making it a standalone proposal later on. I doubt very much the current implementation makes that feature impossible, although I'll admit I'm not familiar with it.

[Anders_Ha]

It's going to be difficult to reason reliably about this.

It can be difficult to reason. But it doesn’t preclude it being a useful utility, especially since it has great utility in UI programming, where synchronously responding to a message can be important to the UX.

Kotlin Coroutines offers this in two opt-in ways: CoroutineStart.UNDISPATCHED and Dispatchers.*.immediate. While having subtle differences in semantics, both deliver the same core behaviour as described by @adamkemp — run some parts of the work synchronously on the current call stack.

Explaining in the Task API terms:

The former means that the task body runs synchronously as part of the runDetached(), until nothing is left to process in the current call frame (e.g., it genuinely ended, or some of its inner async functions have switched execution context). In other words, an async dispatch is skipped for at least starting the detached task body.

The latter is a special version of an executor, which skips a dispatch if the caller is already on the same executor. This does however break FIFO dispatch order if the API guarantees that as default.

In any case, we have a similar problem today already in unidirectional data flows like TCA or RAC Loop. These are programmed in terms of asynchronous effects via Combine/ReactiveSwift. But there are scenarios of these asynchronous effects deliberately yielding conditionally synchronous results (as part of their on-subscribe/starting side effect). By resolving as many of these results as possible within the store/system initialiser, the later-bound UI will be able to start right at the desired populated state.

A example is an async sequence for UI consumption that:

1. Initially evaluate the database query synchronously on the MainActor; and then
2. Await for change notifications on a background thread, does the reevaluation upon reception, and resume on the MainActor with the result.
3. Repeat (2) until being cancelled.

Another being the staple example of an async image fetching routine:

1. If the request hits in-memory (or even disk) cache, serve the image synchronously; or otherwise
2. Fetch from remote asynchronously.

This is also relevant at the level of async publishers/streams. Both RxSwift and ReactiveSwift offers a special executor/scheduler (CurrentThreadScheduler / UIScheduler) to achieve such behaviour.