Rethinking Async

I disagree with your entire premise. async/await, as implemented in other languages, solves a real problem. If you're brushing this off as "not really helpful" then you are either misunderstanding what it does or just not appreciating the benefits that you get from it. Other languages didn't go to so much trouble to implement this feature for a trivial improvement. It makes a significant difference in clarity and maintainability of asynchronous programming, which is a very common task in modern applications.

If you want to discuss new ideas to solve different problems then that's fine, but that's orthogonal to what async/await does, and it does not replace the need for async/await. Your proposal is not an alternative. It's a different feature to solve a different problem.

2 Likes

Hmm? This one looks much more like C#'s async/await that the ones in the other thread even.

If you want it to work like C# then you don't need async as a keyword at a call site or as a decorator for a type. That's trying to bake futures in as a language feature, which is not how C# does it, and not something I would recommend.

C#'s async/await works with any futures type that conforms to a certain pattern of available methods (including extension methods). It doesn't tie you to just one built-in type, which is what this design appears to do.

2 Likes

I'm not saying it's doing exactly what C# is doing, but there are certainly similarities like letting async functions return a checkpoint (Task<Int> & async Int) to be waited later with async, which is already a big departure from Chris' design.

Respectfully, I don't think you need two threads for this same conversation. Do you want me to close the last one?

2 Likes

Shouldn't we wrap this one up, and go back to the other thread instead?

PS

It think there are three-ish; Async/await status?, An extensible Syntax for effects like e.g. async-await, and this.

I'll let the three of you come to a consensus, but please pick one.

3 Likes

I think we could close this one, and we all can go back to Async/await status?, until the design ends up being more fleshed out.

Respectfully, I think @Jon_Hull has started a different conversation, and it's not the conversation happening in any other currently active thread. @Jon_Hull is proposing a different solution to the underlying problem, and it's a direction I very much want to discuss.

What would be good is if we could agree to keep discussion of the implementation details of other proposed solutions out of this thread. Those belong in their respective in-progress threads.

2 Likes

These are not the async/await discussion forums. You don't need separate threads for every new idea. When we have multiple threads for the same broad topic, time and time again what we see is that the same people end up saying similar things in all the threads and it becomes very difficult for other people to follow and participate. This is already a very high-traffic conversation; the participants are writing collectively writing tens of thousands of words a day. The sheer volume makes it unapproachable.

Therefore, there is going to be one active conversation thread from here on out. I am happy to merge threads or close them as the participants like, but if I don't get clear signal, I will just do what I think is best. This is your opportunity to shape the outcome.

14 Likes

FWIW, I agree with @QuinceyMorris. The reason I started a new thread is that although some of the terms are the same, I am actually talking about a completely different approach and way of thinking about the issue. I feel like if I had posted this there, it would have been rejected/ignored for being off-topic.

That said, I will defer to your judgment. I hope this much needed conversation can actually happen somewhere though...

I acknowledge that your position is reasonable, and I share your frustration about the nature of the discussion.

However, there are valid counterarguments to the rationale you've expressed:

  1. If you try to constrain multiple discussions to one thread, the effect is that any one discussion is constantly being derailed by others, and no one can follow the topic anyway.

  2. If a discussion gets too long, newcomers can't afford to invest the time to read the whole history, which result in a lot of repetition of ideas that have already been discussed.

So, I think a better model is to allow new discussions to start, and let them die away after a few days — which is what has actually happened to a number of these async/await topics, including ones that I've started.

Specifically, though, if you're asking for input about what threads to close, I say close those that have >70 posts (or somewhere around there). Experience has shown that longer threads don't really go anywhere.

8 Likes

Actually, I don't have a strong opinion which thread it's gonna be.

Okay. Since it seems to be people's preference, and since there are links to the earlier threads, we can continue in this one.

1 Like

OK, currently we have two categories of ideas for async/await to tackle both asynchronous and concurrency problems.

  1. async routine/function coroutine for async operation
  2. async T/value/result for concurrency - parallel async

I think 1) already well solved by traditional async/await pattern both in swift and other similar traits of lang C#/JS...

Maybe this thread Rethinking is focused on parallel async or concurrency representation by async/await pattern.

assuming func asyncFunc(...) async -> T

async let result1 : T = await asyncFunc(...) 
async let result2 : T = await asyncFunc(...) 
/* define async result but not call/branch async func immediately - 
`lazy async` or Future/Promise async representation */

await result1, result2 //actually call the asyncFunc and wait/suspend for return in parallel directly 

or call asyncFunc(...) async -> Void Actions in parallel

await asyncFunc(...), asyncFunc(...), ... //parallel async

Conclusion, await async Value for concurrency async, get async results in parallel; and await async Func for traditional async get async results in sequence.

Alternative pattern: using ViewBuilder equivalent AsyncBuilder {...} eDSL to build async calls in parallel but this is out of this topic scope.

1 Like

It’s certainly in async/await’s favour that it’s been adopted by so many languages. Having so many developers familiar with the concept makes it valuable in its own right.

That said, popularity alone is not enough to justify adding async/await to Swift. Many of those implementations have holes and sharp edges that I think we can improve on. See the discussion about modify accessors and how we’d like to express handling coroutine cancellation in the language, where many others simply acknowledge that it’s a dangerous trapdoor and continue.

There are other language designs for composing asynchronous tasks in to pipelines which I think could be just as powerful (if not more so) than async/await as we see it in popular languages today.

2 Likes

I see a lot of people mentioning designs that doesn’t immediately await function calls to facilitate concurrency. Fair enough.

What I don’t understand is the claim that it’d just work with coroutine. Could someone elaborate more on that. Or do they just treat async Int as native Future wrapper?

@John_McCall said:

With respect, I'd encourage people to think of Chris's old paper less as the "current design" and more as a conceptual starting point. It's trying to get us all familiar with some of the basic problems and make sure we're using the same terms to talk about them. If Chris's intrinsics don't let us express what we need to express, well, we need different intrinsics, that's all. In this case, beginAsync needs to also take some description of the current execution context that will get passed down to the async function so that it can resume itself there if need be. Generally this intrinsic would get used in the implementation of e.g. DispatchQueue.async , which can very easily describe the queue to beginAsync after moving execution there.

I agree. I think there are several different issues that are getting conflated into "async await":

  • There is a well understood effect system and state machine transformation that is enabled with the async and await keywords. There is general cross language and industry alignment about what these do and why, and LLVM has increasingly good support for this (thanks to John and others).

  • There are Swift standard library functions that are required for converting between the effect and other ways of modeling the same concepts. These would take low level forms and may have high level equivalents (like a "future" type). These are precedented in things like withoutActuallyEscaping(:do:) and Result for the @escaping and throws systems, respectively.

  • We probably want higher level APIs and patterns for dealing with cancelation and timeouts across async domains. There was a recent twitter thread with Joe Groff and others on it discussing this blog post and this library both of which are really well written and motivated, and have ideas we should be inspired by IMO.

  • There is a discussion about what is the underlying execution model for threads.

My point is that these are all separable discussions and orthogonal features - the results of which should compose in a beautiful way, but this is a "manifesto" level discussion just for async/await, not to mention actors and all the other stuff in the concurrency manifesto I wrote 3 years ago - time flies!

-Chris

22 Likes

Side note: The extensible syntax thread is actually not about async-await, async-await was just used as an example. Probably not the best move I ever made, the conversation I actually wanted to have never really materialized there. I'm not sure what to do with that now, maybe I should do an extra thread on algebraic effects? Or rename the above one another time to make the topic even clearer?

1 Like

I like the async/await syntax generally being proposed, modeled off of other languages, but don't find it to be as much of a developer pain point as the amount and the timeline of discussion here suggests. It just seems like a bit of sugar over a simple Future/Promise type that wraps a closure which you can write in 30 lines of Swift.

1 Like