Don't we have the exactly same problem with throws/try right now?
let lock = getLock()
let otherData = try throwingDataRetrieval()
lock.release()
If throwingDataRetrieval() throws, lock stays unreleased. Isn't it a responsibility of the developer to wrap lock.release() with a defer in both cases?
Every single usage of await is an example of this. If the function you're calling is async and you call it using await then you have to assume that the function will be suspended at that point, and the original thread will do other work.
I'm not sure what kind of example you're looking for, but if you see an await then that implies "this function may return control to the calling thread at this point". That's fundamental to the feature.
One thing I noticed when writing my last reply is that the proposal does not call for a compiler warning if you have an async function that doesn't use await. In C# that's a warning, and I would argue it should be a warning in Swift as well. Is there a rationale for not doing that?
When I call a function with a completion handler, that function may effectively “suspend” for a while before calling the completion handler.
When it does so, control returns to my code which called it. My code continues running while that function is “suspended”.
However in the proposal text, an async function can only be called from within another async function. And when such a call is made, if the inner function suspends, then the outer function also suspends.
So what code keeps running on that thread?
The proposal does not explain that.
IIUC (someone please correct me if I've misunderstood), this is essentially an implementation detail of the underlying operating system/scheduler/runtime. When a suspension point is reached, Swift will yield the thread back to the scheduler, and at some point in the future the rest of the async function will be scheduled to run.
In UIKit-land, this would (I assume) mean handing control back to the main RunLoop so that it can continue to service UI events until the rest of the async task is ready to run.
As for the chicken-egg problem with async function calls, there have been a couple solutions mentioned which will allow non-async functions to spawn async execution contexts.
This question may not be related to the proposal itself, but to the implementation on Apple platforms, not sure what's a more appropriate place to ask this.
Do I understand correctly that both async/await and structured concurrency primitives would require presence of certain runtime parts that can't be embedded within apps and libraries and can't be deployed to older versions of iOS? Thus, application and library authors would have to bump their deployment targets to iOS 15, or whatever OS version is going to ship with the new runtime and the new Concurrency module to be able to use async/await?
I am strongly in favor of this proposal if the structured concurrency proposal is also accepted in a form close to its current state.
In multiple discussions about this feature over the past few years I have stressed the importance of behaviors like await returning to the UI thread if it starts there and also the ability to detach tasks and compose them. Without those behaviors and features I don't think async/await should be added. In other words, if async/await were implemented with some other underlying behaviors then I think it would be bad for the language. So my endorsement of this proposal is conditional on those features being present from day one.
I understand the need for splitting out proposals to make them easier to digest and discuss independently, but in this case I just don't see any merit to accepting this proposal without a commitment on the other half of it.
Yes, that's correct. The suspension of an asynchronous task stashes any data it needs on resumption into task-allocated storage, puts the remaining work (a "partial task" in our nomenclature) on to a queue somewhere so it can be resumed later, and then does a synchronous return to the scheduler on that thread. The scheduler then looks for another partial task to run.
Doug
... snip ...
Speaking on behalf of the authors of these proposals, we consider the Swift Concurrency effort incompletion until we have all of async/await + structured concurrency + actors. This is why we started with the bundle of proposals under a unified roadmap. So, we're committed to following through with the full set of proposals.
For the Structured Concurrency proposal, it's undergoing some revision based on pitch feedback, but my read of it is that there is significant consensus on the basic model, and we're hammering out details. (And if anyone disagrees with that assessment, please let's discuss it on the structured concurrency thread or wait for the second pitch on structured concurrency), so I'm fairly confident that one will go through the review shortly after this one completes. The implementation of all of these is proceeding... err... concurrently.
Doug
That all makes sense. I would just say that if, for whatever reason, the structured concurrency spec is held up (maybe parts of it are controversial to some people), and it doesn't get accepted, then I would argue that async/await should...await it. (two can play at this game)
This feature is just entirely useless without at least the structured concurrency part to go with it so I don't see any value in proceeding with one but not the other.
I really can't comment on Apple products. The concurrency runtime support library is built as a separate library that builds upon existing APIs (e.g, libdispatch) that have been around for a while. If you grab the right PR and use a hack to start async code, it works fine on generally any macOS.
Doug
That's not the design of cancellation described in the Structured Concurrency proposal. A potential suspension point is not a return point. One manually checks for cancellation and can then choose to exit early via either of the normal mechanisms, whether it's return or throw. Cancellation is not a third kind of exit.
This is fine; lock.release() will always get called because await is a suspension point. It cannot return.
Doug
It's worth pointing out, though, that it may still be a different kind of bug because in some lock implementations the release must happen on the same thread as the acquisition. When you put the await in there then you may end up on a different thread, which may be a bug. That's another example of why it's important to see the suspension point so you might notice a bug like that.
That's not a flaw in the async/await proposal, IMO. It would be an argument for not providing a lock API that has such rules using separate acquire/release methods instead of a structured approach (like a method performWhileHoldingLock(_ fn: @autoclosure () -> Void). Then you couldn't possibly have an await in between the acquisition and release because that block is not async.
I see - very interesting! So once created, a coroutine’s detached stack can only be destroyed by resuming and running to completion? Well then, you can consider that concern as resolved.
I still think it is worth adding to the proposal as an important part of the programming model for await. Lots of implementations in other languages make suspension points implicit return points to support cancellation, IIUC.
I won’t pretend to know anything about designing a good lock API; the point was about objects which need some specific call between initialisation and destruction. It is straightforward to remember that throwing errors could aborting the function, but I think it would be much less obvious if await (which developers will associate with suspension) could do the same. Thankfully await cannot return from the function, so there’s no concern.
It’s quite interesting. Presumably it means we could use await inside synchronous non-escaping closures — if those closures are created in an async context (e.g. you should be able to await inside the Array and String unsafe initialisers, or inside Sequence’s map closure, if you call them from an async function).
I guess I’m still not understanding how async/await is intended to replace callbacks.
If I have a UI application where the user clicks a button to start downloading a file, then in today’s world the button-click function calls a download-file function which takes a completion handler. The button-click function then returns, and control goes back to the UI event loop.
In the proposed async world, where the download-file function is async, how does that work?
We can’t let an await suspend the UI event loop when the download-file function is called.
If I have to manually write Task.runDetached in all my button-click functions, that becomes onerous.
Does that mean all the button-click functions become async, whether or not they actually await anything, and the UI event loop calls Task.runDetached every time a button is clicked?
(Yes, I understand that Task.runDetached is part of another proposal. I am trying to understand how async and await from this proposal are intended to be used in the proposed world where all the concurrency features exist.)
I know it’s off topic for this thread, but I have asked for a more clear justification of this approach to cancellation and none has been provided thus far. Given that Structured Concurrency is a pitch and not an accepted proposal, I don’t think reviewers of async / await should be expected to accept its design as a given. There is more discussion to be had around cancellation before a final design is accepted.
My understanding is that cancellation is not a language feature (I.e., not built in to async/await at all). That makes it either up to the end developer or the standard library. It could be done by either returning early or by throwing.
The structured concurrency proposal is in part a standard library feature that includes mechanisms for handling cancellation more easily, including convenience utilities that make it easier to signal and check for cancellation. But you could also roll your own because it’s not tied to the language.
As proposed I think you would have to do something like that. In C# they have async void to cover this exact situation, but its use for anything other than event handlers is strongly discouraged. One C#-specific reason is that without a caller to await the result there is nowhere for exceptions to be handled, and so any exception thrown from a continuation (after the await) causes an immediate crash.
I think from the proposal authors’ perspective it just reduces the structuredness of the code, which makes some other things they want to do harder.
I do think it would be beneficial to have some kind of syntactic sugar for a use case like this, though, because it is so common. That is, some kind of way of marking a synchronous, non-throwing, void-returning function as “all of my code should be wrapped in a detached task”. The exception issue in C# doesn’t apply in that scenario, and I think this is going to be very common.
Cancellation has really two "parts" to it:
Task abstractions and their parent/child relationships matter for how cancellation is propagated; e.g. cancelling a parent implies cancelling all its children; This piece is part of the core runtime.The runtime bit is really part of the "Task" feature which we're planning to propose separately, and structured concurrency makes use of it.
Doug made a small diagram with all the proposal dependencies recently (though this will even keep growing a little bit 
):
This will be the "Asynchronous Handlers" proposal. For functions which can be called from an sync context, but really are async "inside"; they must return void indeed. This will be it's own complete proposal as well. Short version: @asyncHandler func buttonClicked() { ... actually async here }, but let's shelfe that and get to it in the Asynchronous Handlers proposal soon.
(I know it's annoying with all these references to "this will be done in X", but the proposals are very large already so we're trying to keep them focused and one-by-one)
My point is that this is still a pitch and not an accepted proposal. I think there are very good reasons for cancellation to be implicit at every suspension point by default. This topic deserves a more robust debate than has occurred thus far, including a more detailed rationale for the proposed design. It should not be assumed for the sake of this review that the proposed design for cancellation will be the final accepted design.