Absolutely - this is one of the major pain points remaining in the ecosystem, especially for server side applications.
I think so - I particularly like the use of keywords like async and await instead of native Promise implementations like Javascript started with.
I've used futures/promises in Javascript/Typescript extensively and I'm excited to see that this proposal lines up with a lot of the syntax there. I think it reads clearly and is powerful for the user.
Quite a bit - I've been reading the proposals since they first went up a few weeks ago and participating in the discussions, and then of course did another review when this was posted.
Unless I’ve missed something, the proposal does not show any examples of interleaving.
The proposal does not show any examples of the original thread continuing to do other things during the await.
That is what I am asking for examples of.
That seems to be the main purpose and primary motivation for this feature.
And the proposal shows zero examples of it.
The proposal does not adequately communicate how this feature is intended to be used.
Even if the mechanism for actually using the feature in that way has been split off into a separate proposal, it should still be explained here as motivation for this feature, with examples, so that reviewers of this proposal can understand what the purpose is and how it is intended to be used.
The second of these is always called after the first, but the timing is asynchronous — there is a time gap (analogous to a suspension point) between the two, during which any other main thread code could execute.
The async/await proposal doesn't really change anything fundamental, except that it introduces suspension points inside functions alongside the existing suspension points between function invocations.
I think @Max_Desiatov is correct that you can't demonstrate the resulting interleaving solely in terms of base async/await syntax. However, there are lots of other ways of executing independent code on the same thread (e.g. using GCD as in the example I gave you over in the other thread).
The proposal under review in its text already links to the structured concurrency proposal. Reviewers needing more context can freely follow the link. What would be the benefit of duplicating examples in two separate, but related proposals?
The proposal presents async/await as an answer to the (numerous) problems identified with the "completion handler" pattern, and the interleaving/"gives up the thread" behavior is already commonplace in these situations. Indeed, that's precisely why you use a completion handler when scheduling UI work to be done after a long-running task: you want to allow other work items to execute on the main thread before the task completes.
Perhaps the proposal could draw this point out a little more explicitly, but I don't think that the concept is quite as foreign as you're contending.
I started off as broadly +1, but on more careful reading of the proposal and allowing things to stew a bit, I'm not so sure. It's quite hard to judge without also considering the planned executor/continuation API (which I haven't looked at in as much detail). But even considering the async/await model proposed in this document as a kind of abstract concept, I have concerns with the programming model that I think merit further discussion.
Read-only async properties and subscripts should be allowed. The proposal says:
Properties and subscripts that only have a getter could potentially be async ... Prohibiting async properties is a simpler rule than only allowing get-only async properties and subscripts.
My argument against this is similar to Doug's argument against banning inheritance for actors: why impose arbitrary restrictions? Also, currently, functions do not satisfy get-only property requirements in protocols, meaning synchronous types which want to use properties for a more natural interface will also be forced to use functions.
protocol AsyncCollection {
func startIndex() async -> Index
}
struct MySyncCollection: AsyncCollection {
// Does not conform to AsyncCollection:
var startIndex: Index { ... }
// Instead, they'd have to write this:
func startIndex() -> Index { ... }
}
let syncInstance = MySyncCollection()
syncInstance.startIndex() // function interface forced upon non-async code
Allowing async properties might just be the best interface for both synchronous and async conformers.
Cancellation.
The proposal tries to defer this discussion, but I believe it is a crucial element of the async/await programming model to say whether or not async functions can be cancelled. If they can be cancelled, it means every possible suspension point is also a possible return point, which is a critical detail that the proposal does not mention at all.
For example, consider the following code:
let lock = await getLock()
let otherData = await downloadOtherData()
lock.release()
If async functions could not be cancelled, we know that either lock.release() will be called, or that the function never got scheduled again and leaked its whole stack (lock.deinit will never be called). But if they can be cancelled, lock may be destroyed without us calling .release(). Is that obvious from the code? I don't think it is.
There are potential dragons here if you leave data in an invalid state across an await, and that data is somehow visible to others or (the more subtle case) relied upon in the type's deinitializer. For example, if lock was a buffer that we were filling, we might try to deinitialize invalid memory because we forgot that await meant the function could return in the middle of initializing. And to be fair, when the code looks like: let otherData = await downloadOtherData(), it's hard to spot that there's a hidden return in there. The only clue is on the right-hand side of an assignment operator.
As far as I can tell, this possible footgun is the same reason _modify still isn't an official language feature (because every yield is also a possible return point). It's important we don't ignore this very important consideration, which developers using async/await in other languages can struggle with.
Is the problem being addressed significant enough to warrant a change to Swift? Does this proposal fit well with the feel and direction of Swift?
It's fine, I guess. I'm a little concerned that we're doing this without a lot of the infrastructure other languages have (coroutines, generators).
If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
I've mostly been using Swift since async really started taking the world by storm, so I've missed a lot of that. That said, I have heard criticisms from other developers that async ends up spreading to all areas of your code, and I could see similar things happening here. For example, with synchronous functions able to satisfy async protocol requirements, I could imagine lots of developers will default to making everything async. I'm not sure there is any way to avoid that, or what a better approach would be.
Don't we have the exactly same problem with throws/try right now?
let lock = getLock()
let otherData = try throwingDataRetrieval()
lock.release()
If throwingDataRetrieval() throws, lock stays unreleased. Isn't it a responsibility of the developer to wrap lock.release() with a defer in both cases?
Every single usage of await is an example of this. If the function you're calling is async and you call it using await then you have to assume that the function will be suspended at that point, and the original thread will do other work.
I'm not sure what kind of example you're looking for, but if you see an await then that implies "this function may return control to the calling thread at this point". That's fundamental to the feature.
One thing I noticed when writing my last reply is that the proposal does not call for a compiler warning if you have an async function that doesn't use await. In C# that's a warning, and I would argue it should be a warning in Swift as well. Is there a rationale for not doing that?
When I call a function with a completion handler, that function may effectively “suspend” for a while before calling the completion handler.
When it does so, control returns to my code which called it. My code continues running while that function is “suspended”.
However in the proposal text, an async function can only be called from within another async function. And when such a call is made, if the inner function suspends, then the outer function also suspends.
IIUC (someone please correct me if I've misunderstood), this is essentially an implementation detail of the underlying operating system/scheduler/runtime. When a suspension point is reached, Swift will yield the thread back to the scheduler, and at some point in the future the rest of the async function will be scheduled to run.
In UIKit-land, this would (I assume) mean handing control back to the main RunLoop so that it can continue to service UI events until the rest of the async task is ready to run.
As for the chicken-egg problem with async function calls, there have been a couple solutions mentioned which will allow non-async functions to spawn async execution contexts.
This question may not be related to the proposal itself, but to the implementation on Apple platforms, not sure what's a more appropriate place to ask this.
Do I understand correctly that both async/await and structured concurrency primitives would require presence of certain runtime parts that can't be embedded within apps and libraries and can't be deployed to older versions of iOS? Thus, application and library authors would have to bump their deployment targets to iOS 15, or whatever OS version is going to ship with the new runtime and the new Concurrency module to be able to use async/await?
I am strongly in favor of this proposal if the structured concurrency proposal is also accepted in a form close to its current state.
In multiple discussions about this feature over the past few years I have stressed the importance of behaviors like await returning to the UI thread if it starts there and also the ability to detach tasks and compose them. Without those behaviors and features I don't think async/await should be added. In other words, if async/await were implemented with some other underlying behaviors then I think it would be bad for the language. So my endorsement of this proposal is conditional on those features being present from day one.
I understand the need for splitting out proposals to make them easier to digest and discuss independently, but in this case I just don't see any merit to accepting this proposal without a commitment on the other half of it.
Yes, that's correct. The suspension of an asynchronous task stashes any data it needs on resumption into task-allocated storage, puts the remaining work (a "partial task" in our nomenclature) on to a queue somewhere so it can be resumed later, and then does a synchronous return to the scheduler on that thread. The scheduler then looks for another partial task to run.
Speaking on behalf of the authors of these proposals, we consider the Swift Concurrency effort incompletion until we have all of async/await + structured concurrency + actors. This is why we started with the bundle of proposals under a unified roadmap. So, we're committed to following through with the full set of proposals.
For the Structured Concurrency proposal, it's undergoing some revision based on pitch feedback, but my read of it is that there is significant consensus on the basic model, and we're hammering out details. (And if anyone disagrees with that assessment, please let's discuss it on the structured concurrency thread or wait for the second pitch on structured concurrency), so I'm fairly confident that one will go through the review shortly after this one completes. The implementation of all of these is proceeding... err... concurrently.
That all makes sense. I would just say that if, for whatever reason, the structured concurrency spec is held up (maybe parts of it are controversial to some people), and it doesn't get accepted, then I would argue that async/await should...await it. (two can play at this game)
This feature is just entirely useless without at least the structured concurrency part to go with it so I don't see any value in proceeding with one but not the other.
I really can't comment on Apple products. The concurrency runtime support library is built as a separate library that builds upon existing APIs (e.g, libdispatch) that have been around for a while. If you grab the right PR and use a hack to start async code, it works fine on generally any macOS.
That's not the design of cancellation described in the Structured Concurrency proposal. A potential suspension point is not a return point. One manually checks for cancellation and can then choose to exit early via either of the normal mechanisms, whether it's return or throw. Cancellation is not a third kind of exit.
This is fine; lock.release() will always get called because await is a suspension point. It cannot return.
It's worth pointing out, though, that it may still be a different kind of bug because in some lock implementations the release must happen on the same thread as the acquisition. When you put the await in there then you may end up on a different thread, which may be a bug. That's another example of why it's important to see the suspension point so you might notice a bug like that.
That's not a flaw in the async/await proposal, IMO. It would be an argument for not providing a lock API that has such rules using separate acquire/release methods instead of a structured approach (like a method performWhileHoldingLock(_ fn: @autoclosure () -> Void). Then you couldn't possibly have an await in between the acquisition and release because that block is not async.
I see - very interesting! So once created, a coroutine’s detached stack can only be destroyed by resuming and running to completion? Well then, you can consider that concern as resolved.
I still think it is worth adding to the proposal as an important part of the programming model for await. Lots of implementations in other languages make suspension points implicit return points to support cancellation, IIUC.
I won’t pretend to know anything about designing a good lock API; the point was about objects which need some specific call between initialisation and destruction. It is straightforward to remember that throwing errors could aborting the function, but I think it would be much less obvious if await (which developers will associate with suspension) could do the same. Thankfully await cannot return from the function, so there’s no concern.
It’s quite interesting. Presumably it means we could use await inside synchronous non-escaping closures — if those closures are created in an async context (e.g. you should be able to await inside the Array and String unsafe initialisers, or inside Sequence’s map closure, if you call them from an async function).