Async/await status?

One possibility is that queues could have a method to spawn a coroutine associated with that queue as its execution context, so you could do something like:

// We don't care where this outer context runs...
DispatchQueue.spawnDontCareContext {
  // but this should run in the background...
  let backgroundWork = backgroundQueue.spawn { ... }
  // and this should run on the main queue...
  let mainWork = DispatchQueue.main.spawn { ... }
  // then we want to wait for both of them to finish
  await backgroundWork.join()
  await mainWork.join()
}
1 Like

Yep, sorry for the confusion, typo. :slight_smile:

Sure, I agree that could be interesting to explore. The point I was trying to make was a bit different though: existing completion handler APIs in Apple frameworks are known to do queue hopping. I was just arguing that "autoupgrading" those APIs to behave correctly w.r.t. the new semantics (whatever they are) is not worth it anymore.

-Chris

1 Like

I don’t see how exactly the proposal fits in with the structured concurrency model and “requirements” outlined in the article. Don’t get me wrong, this is a great proposal that does combat callback-hell - and the others problems mentioned in the proposal, I just don’t see truly “structured” concurrency. Is there something that I’m missing here?

the more i read into this thread the stronger i realize that callback hell is not that bad thing (or maybe not even hell at all) :-)

yes, it adds indentation (that can be somewhat combat by splitting code into separate functions) and is a tad error prone (e.g. you have to not forget calling completion handlers in all code paths, etc) but at least conceptually it is so much simpler to what's been discussed here.

if we assume that this fragment:

func processImageData1() async -> Image {
  let dataResource  = await loadWebResource("dataprofile.txt")
  let imageResource = await loadWebResource("imagedata.dat")
  let imageTmp      = await decodeImage(dataResource, imageResource)
  let imageResult   =  await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

for all intents and purposes is equivalent to this fragment:

func processImageData1(completionBlock: (result: Image) -> Void) {
	loadWebResource("dataprofile.txt") { dataResource in
		loadWebResource("imagedata.dat") { imageResource in
			decodeImage(dataResource, imageResource) { imageTmp in
				dewarpAndCleanupImage(imageTmp) { imageResult in
					completionBlock(imageResult)
				}
			}
		}
	}
}

possibly even compiled to exactly the same code bit to bit, then the questions about stacks (continuous / segmented / moving), or queues, or threads, etc are clearly orthogonal, go out of scope, and can be resolved separately. treat it is a mere text transformation like a "preprocessor step" (ok, the intermediate representation transformation, but that doesn't change anything).

can we have this (hopefully not so hard to implement) transformation in the compiler sooner, like within a year? possibly refined later. i wouldn't even mind compilation errors "compiler wasn't able doing this in reasonable time, try to simplify it" in the first incarnations. maybe do it in a few simple, partial, "non-ideal" steps rather than do the whole thing "correctly" outright.

If that's all async/await did then it wouldn't be all that valuable. The value comes from simplifying the more complex cases, which look truly nasty using callbacks. Notice your example still doesn't handle any errors or ensuring you end up back on the main queue if you started there. Your proposed simplification is just not a reasonable option for this feature.

6 Likes

To expand on this, async/await’s primary advantage is that it allows you to use the pre-existing linear control flow primitives to manage control flow, rather than translating them to a callback model.

SwiftNIO users frequently struggle with conceptualising how to transform code that would have been a for loop into a callback chain, or how to handle logically branching execution. This is not a failure on their part, but because this is fundamentally hard. Bringing those linear control flow primitives back is enormously valuable for reducing the cognitive burden of asynchronous code.

15 Likes

Not only SwiftNIO, it's also widely applicable when working with Combine, where with linear control flow map is just a plain function application, filter is an if condition (or a ternary ?: expression if you will), collect could be expressed as a for loop, catch operator is just a plain catch block etc. The main barrier to learning Combine that I see (especially in beginners learning it) is translating all the linear control flow people are used to into a chain of "operators" to manage the asynchronous nature of streams/publishers.

3 Likes

Yeah, in addition to linear async/await, I think we'd also eventually want something like C#'s async for loops, which would facilitate working with Combine streams and other reactive libraries in a more imperative style.

10 Likes

How does async/await work with concurrent execution?

let a = await taskA()
let b = await taskB()

let c = taskC(a, b)

How do I run taskA and taskB in parallel? Insofar I only see examples where taskB starts after taskA returns the value (with continuation block), or maybe I'm misunderstanding the syntax.

1 Like

I would hope/expect you could do the following to await multiple tasks in parallel:

let (a, b) = await (taskA(), taskB())

let c = await taskC(a, b)

I would also hope/expect that you could await arrays of tasks.

1 Like

I do hope it could be more organic than constructing a compound types tuple, since it'd also be useful to mix with other control-flow (or even just if/else):

let a = await taskA()
let b: ...
if condition {
  b = await taskB1()
} else {
  b = await taskB2()
}

/// A and B1/B2 runs in parallel.

A library should provide primitives for forking and joining if you want to run tasks concurrency. There are a few possible models for doing this. You could do something with a future type:

let futureA = Future(taskA)
let futureB = Future(taskB)

let c = await taskC(a.get(), b.get())

With a structured concurrency library, scoped "nurseries" implicit wait for all their child tasks to finish on scope exit:

let a: A, b: B
withNursery { nursery in
  nursery.spawn { a = taskA() }
  nursery.spawn { b = taskB() }
}
taskC(a, b)
9 Likes

I would hope that would be written like this:

/// A and B1/B2 runs in parallel.
let a = async taskA()
let b: ...
if condition {
  b = async taskB1()
} else {
  b = async taskB2()
}

useAandB(await a, await b)

3 Likes

I think you're conflating concurrent programming with solving asynchronous programming issues. async/await is really about letting you write asynchronous code in a more imperative style, not really helping solve concurrent programming issues per se.

But as Joe points out above, an actual async/await implementation will provide ways for working with the primitives. I think what you're missing is that you don't have to await on every async function (at least I'm assuming that's a model still up for consideration). Not doing an await could in theory allow you to chain multiple async computaitons into a single await.

3 Likes

+1. possibly even without "async" to make it more dry as it can be inferred.

foo() {
	let a = taskA()
	let b = condition ? taskB1() : taskB2()
	useAandB(await a, await b) // awaits both arguments
}

and if useAandB() is async itself, then:

await useAandB(a, b)

and if foo() is async itself then:

useAandB(a, b)

i'm just suggesting we have something incomplete quicker (say within a year) than the whole thing at once in five year's time (by which time async/await will be obsolete by something new). the quick and dirty solution doesn't preclude further refinements.

here's my attempt of quick and dirty callback implementation of async/await.

with this approach the original fragment would be written almost as in the proposal:

func processImageData1() -> Async<UIImage> {
    let dataResource  = loadWebResource("dataprofile.txt")
    let imageResource = loadWebResource("imagedata.dat")
    let imageTmp      = decodeImage(dataResource, imageResource)
    let imageResult   =  dewarpAndCleanupImage(imageTmp)
    return imageResult
}

where the individual methods are typed as:

func loadWebResource(_ path: String) -> Async<Data>
func decodeImage(_ a: Async<Data>, _ b: Async<Data>) -> Async<UIImage>
func dewarpAndCleanupImage(_ a: Async<UIImage>) -> Async<UIImage>

in this approach Async is simulated as a callback that returns a thing and await is simulated as the callback call to get the thing. note how the concept of autoclosure intertwines with async here. showing example with some simple math to illustrate the gust of the concept.

typealias Async<T> = () -> T

func await<T>(_ x: Async<T>) -> T {
    x()
}

func const<T>(_ x: T) -> Async<T> { // see autoclosure example below
    { x }
}

func sin(_ x: @escaping Async<Double>) -> Async<Double> {
    { sin(await(x)) } // system sin is a sync function -> await
}

func cos(_ x: @escaping Async<Double>) -> Async<Double> {
    { cos(await(x)) }
}

func + (_ x: @escaping Async<Double>, _ y: @escaping Async<Double>) -> Async<Double> {
    { await(x) + await(y) } // built-in + is a sync function -> await
}

func - (_ x: @escaping Async<Double>, _ y: @escaping Async<Double>) -> Async<Double> {
    { await(x) - await(y) }
}

func ^ (_ x: @escaping Async<Double>, _ y: Double) -> Async<Double> {
    { pow(await(x), y) }
}

func ^ (_ x: @escaping Async<Double>, _ y: @escaping Async<Double>) -> Async<Double> {
    x ^ await(y) // await second argument and forward to another async function
}

func sqrt(_ x: @escaping Async<Double>) -> Async<Double> {
    { sqrt(await(x)) }
}

func someComplexThing(_ x: @escaping Async<Double>, _ y: @escaping Async<Double>) -> Async<Double> {
    sqrt(sin(x) ^ 2 + cos(x) ^ 2) // all what's called here is async function -> no await
}

func bar(_ x: @escaping @autoclosure Async<Double>, _ y: @escaping @autoclosure Async<Double>) -> Double {
    await(someComplexThing(x, y)) // bar is sync function -> await
}

func baz(_ x: @escaping @autoclosure Async<Double>, _ y: @escaping @autoclosure Async<Double>) -> Async<Double> {
    someComplexThing(x, y) // baz is async function -> no await
}

func poo(_ x: @escaping Async<Double>, _ y: @escaping Async<Double>) -> Async<Double> {
    someComplexThing(x, y) // poo is async function -> no await
}

func testFooBar() {
    let r = bar(1, 2)
    print(r)
    let z = await(baz(1, 2))
    print(z)
    let w = await(poo(const(1), const(2)))
    print(w)
}

note that this is more akin to a "future/promise" approach (returning a callback that can be used to get a thing instead of returning the thing itself) compared to that transformation with the chained callback calls where function result is moved to the last parameter as a closure. both approaches are valid, just one is easier to implement without language/compiler changes.

ps. i wish (and looking at the above fragment it is obvious why) it was possible to embed "@escaping" specification inside the typealias definition itself.

I hope we can avoid quick and dirty solutions. One could be considered if there's a crystal clear path of how issues with it are going to be resolved, and even that is risky. Since source compatibility is a very strict requirement right now, I really wish we don't end up with a half-baked feature that we won't be able to fix without breaking source compatibility. As we see with the trailing closures issue, realistically, those things might never be fixed.

I'd rather wait a couple of years to get a properly designed best of the breed concurrency in Swift. In the meantime we can use Combine and OpenCombine to make things easier. Speaking of, open-sourcing Combine would alleviate the pressure quite signifcantly as we'd have an "officially blessed" solution working across all platforms that doesn't require any language changes.

4 Likes

what issue?

I link to the main thread here, but I imagine there were many more threads about the inconsistencies in the trailing closure syntax. Essentially, because Swift went with a quick and dirty solution for the single trailing closure (STC) syntax, to avoid breaking source compatibility we now have suboptimal support for multiple trailing closures, while the original issue with STC is still not fixed.