Right, but the proposals are essentially going to be accepted together or redesigned together.
In order to review this proposal (which I have not yet done), I need to understand how the features of this proposal are intended to be used.
I do not want to end up in a place where the community has accepted some of the proposals, then during review of a later proposal decides that a substantially different design would be preferable, requiring that we either revisit an accepted proposal, or settle for a model that we find inferior because we refuse to do that revisiting.
Right now people are saying, “That use-case is part of a different proposal which has not yet been reviewed.”
But if we accept this proposal and then move on to reviewing the others, when questions come up about the overall design people will say, “The basic behavior has already been accepted and is no longer under review.”
In order to review the basic behavior and decide if it’s what we want, it is necessary to understand and consider the full model that is being proposed.
I've already answered this meta-comment. That said, we don't need the details of async let to address your idea, nor do we need the exact spelling of Task.runDetached to be nailed down, which is why my I responded the way I did.
As I understand it, your idea boils down to allowing synchronous code to call an async function directly, and giving it "detached task" semantics. I replied why I consider that a poor design choice, but didn't receive a technical response. If you want to argue for your idea, you can and should address that technical objection.
Since the await happens possibly many lines removed from the defer block it would be difficult to ensure that the defer happens on the thread/queue you expect it to run. I think people who have such APIs are going to have to rethink their structure to work well in async code, and I don't feel like an async defer block changes that significantly.
Oh please no...that would be terrible UX. I believe conceptually a developer does not need to know that a try/throw/return statement might be a suspension point because the actual suspension point is in the defer block itself. The try/throw/return statement is control flow, and thus it results in jumping to some other place. That other place may happen to suspend, but that happens after the normal control flow.
If you required await on every one of those statements it would be very confusing, and introducing a single await in some far-removed code could affect many lines. I would almost rather not have the feature. But I still think await in defer can work, and I still think there's not quite a strong enough reason (that I've seen) that it shouldn't work.
I’m not so much arguing for a particular design, as I am exploring the space of possible designs. Whereas the proposal authors have already spent significant time considering these things, others (such as myself) have not.
It is not enough just to understand the proposal: in order to express meaningful support for it, I need a solid grasp of what the alternatives are.
One is as I described above.
Another would be to say that calling an async function is always implicitly awaited (so no await at the call-site), and you must explicitly write async at the call-site to get the detached behavior. That way concurrency locations are marked with async, and otherwise code runs sequentially.
In that model, async let works as proposed, but if you don’t want the return value (or there isn’t any) you could just write async foo() (which would be equivalent to async let _ = foo() as proposed).
This would still let sync functions call async ones directly, with the only ceremony being the presence of async before the call.
The proposal provides fairly significant rationale for needing await, so if you want to pursue this design, you're going to need to:
Argue that await is unnecessary boilerplate that should be removed
Show code examples where the code gets clearer/easier to reason about because of this change
Personally, I don't think this is a good technical direction. You're syntax-optimizing the least-important case (creating a detached task) at the cost of hiding potential suspension points. It's also a fairly significant divergence from the async/await designs of effectively every other language out there, so it's going to need some strong rationale.
I agree that I wouldn't want this to force us to have await return, await try, await throw, etc.
There's no technical reason that I can see, either. There are use cases (e.g., closing a file handle), and the reasons I can come up to not allow it---somewhat less-obvious potential suspension points (although there's still clearly an await in the source within the defer) and asymmetry with throw (which is banned in defer for legitimate reasons)---aren't very strong.
So let's chalk this up to "we should remove this restriction unless more arguments come up in favor of keeping it."
Again, I’m not pursuing any particular design (whereas the proposals are doing so).
I am trying to understand what alternative designs are possible, so that I can compare them with the proposed design.
If the proposal authors have strong reasons for preferring the proposed design over the alternatives (which one assumes they do), it would be useful to document those reasons in the proposal.
The “alternatives considered” section of the proposal at hand does not list any of these alternatives that we’re discussing. This makes it difficult for reviewers to understand why the specifics of the proposed design were selected over any given alternative.
I am confused by this.
The motivation section of the current proposal is dominated by (in fact I believe it consists exclusively of) discussing ergonomic problems with completion handlers. Async/await is presented as a clean replacement for completion handlers.
But completion handlers are (almost by definition) run in a “detached” manner. Some code calls a function which takes a completion handler, and then continues along sequentially. The function that took the handler, meanwhile, does its thing in the background then calls the completion handler later.
The entire purpose of functions that take a completion handler is to enable “detached” or concurrent operation.
It seems strange to propose a replacement for completion handlers, which does not solve the basic problem that completion handlers are designed for, namely running a detached task.
And it seems even stranger to categorize creating a detached task as “the least-important case”.
From my reading of the proposal, its entire motivation is to replace completion handlers, and the primary (sole?) purpose of functions that use a completion handler is to perform a detached task.
• • •
Furthermore, the alternative from my last post is syntax-optimized for the “await” case. The “detached” case requires an explicit “async” notation in that model.
The fact that completion handlers run "detached" is not a feature; it's a consequence of only having language facilities for wrapping up completion code in closures and then scheduling that for background work somewhere else. Let's consider the first example from the proposal:
func processImageData1(completionBlock: (_ result: Image) -> Void) {
loadWebResource("dataprofile.txt") { dataResource in
loadWebResource("imagedata.dat") { imageResource in
decodeImage(dataResource, imageResource) { imageTmp in
dewarpAndCleanupImage(imageTmp) { imageResult in
completionBlock(imageResult)
}
}
}
}
}
Note how all of the work of the function is done in that deeply-nested stack of closures. Yes, every one of those functions is effectively creating a detached task, because that's the only mechanism we have today.
The whole point of async/await is to take that work that's buried in those nested closures and linearize it, giving it normal control flow, but without losing the asynchrony. Your proposal to have a call to an asynchronous function create a detached task that runs concurrently with the next line of code---with no indication in the source that this is the case---is completely against the goals of async/await because you don't get linearization. In a sense, it's worse than the status quo, because at least today you have closures that help you reason about "this is different code."
Detached tasks are fine when they are needed, but you are assuming that the use of detached tasks in today's completion-handler world imply that they are the common case for async/await. They are not.
It is not possible to consider every objection or every design direction when writing a proposal. In this case, the design directions you're asking us to explore further seem to go against all of the prior art for async/await as well as its explicit design goals. As I noted before, I think this is the wrong direction, for reasons I've explained in several ways. If you wish to effect some change here, you are going to need specific suggestions and examples.
This might be the point where you need to go write some async/await code, whether in Swift or some other language, to get a stronger feel for the model we're going for.
Okay, this makes sense. It would have helped me understand if the proposal had spelled these things out.
That was the first alternative I described.
The second one did not have that issue, and would use async to demarcate the detached call.
The proposal as written did not impart that understanding to me. Similarly, it also did not impart an understanding of what the common cases for async/await are expected to be.
All the examples show one async function calling another. This is good as far as it goes, but it does not help me understand when and how the authors expect async functions to be used.
As I mentioned, the motivation section only talks about replacing callbacks, and callbacks always run detached.
So if the authors have a different vision in mind for how to use async, it would be helpful for the proposal to elucidate that so others can understand.
I am not asking for “every objection” to be addressed.
I am saying that the proposal should explain why the particular solution in it was chosen, and which possible alternatives were considered. This is standard practice for Swift Evolution.
I would love to write some async/await code.
I would especially love to write some code that uses async/await in the manner intended by the authors of the proposal. And that requires me to understand what that manner is.
I would love to try it in different languages, but I don’t know which languages have similar models to what is proposed. I do not see a section in the proposal which compares and contrasts the proposed implementation with that in other language.
It would be helpful if the authors could provide some information about what they have found works well or not so well in other languages, what other ideas they considered that may not already exist “in the wild”, and why they believe the design choices in the proposal are superior to the other options they considered.
• • •
Adopting a proposal like this is not simply saying “Yes I’d like to have async/await in Swift.”
Rather, adopting this proposal will make a fundamental decision about how asynchronicity will work in all future versions of Swift.
It is not enough to support the idea “in principle”. It is necessary to achieve high confidence that the chosen model is the best that we can come up with.
And frankly, it is not realistic to expect every reviewer to do all the research individually to reach that conclusion on their own.
It is a responsibility of the proposal authors to provide sufficient information, both to help reviewers understand the decisions involved, and to establish confidence that the authors themselves have done the research and reached the best possible solution.
I have no doubt that the authors have done their research, but the proposal in its current form does not adequately communicate what that research was and why it led to this design choice over any other.
The burden of proof rests with the proposal authors to demonstrate that this one specific design is the best that can be found.
Look, not to be blunt, but you write very long posts, the latest few of which indicate that you misunderstand what async/await is about and is trying to solve.
The burden of proof rests with the proposal authors to demonstrate that this one specific design is the best that can be found.
Well, to a certain extent. But it also falls upon the person who is confused to try to read up in the subject.
Maybe you could study how it works in the other languages where it’s implemented first, such as C# where I guess it originated? Or at least be more terse. This is quickly becoming a meta discussion.
We built some toolchains that you can use to try out this feature. There are still a bunch of rough edges, but you can use those to get a sense of how things fit together. Here are a few little sample programs to get you started:
Right now, there's an amusing-but-illustrative difference between macOS and Linux: the Linux toolchains are using a cooperative, single-threaded executor while the macOS one is using Dispatch for parallel execution. It's due to some temporarily layering issues in the implementation, but also shows how the model works both ways. We'll be fixing this bug soon on the Linux side.
Have fun!
Doug
[EDIT: Updated toolchains to versions that enable concurrency by default]
Go ahead. I posted links to toolchains and examples to get started with the Swift implementation.
For the questions you're asking, any language with async/await would suffice. C#, Rust, JavaScript, Python, or Kotlin all have the same basic approach that we're taking.
Another great example of use is this update replacing callback-based code in NetNewsWire with async/await. It is probably the closest to what an average developer for Apple platforms will experience in their codebase, that is removal of boilerplate and general improvements to code readability.
I have been following these discussions since before the pitch phase. I have read through all the proposals and the concurrency manifesto. I have asked multiple questions along the way.
And despite all of that, I still do not fully comprehend exactly what is being proposed, what problems it is intended to solve, and how it is meant to be used.
I have put in way more effort than any reviewer should have to, in an attempt to understand.
• • •
A proposal ought to be self-contained. It should explain to someone who has never heard of the idea what is being proposed, how it works, and why it should be added to Swift.
It should not assume any experience with the idea in other languages, and it should not expect people to have pre-existing knowledge of what certain keywords mean. That should all be fully explained in the proposal itself, as well as why the proposed design was selected, and what makes it superior to the alternatives.
• • •
Now, it’s possible that I’m just especially dense and bad at understanding things. But experience has shown that is generally not the case.
I may not be the best at coming up with completely new ideas, but when something is explained to me clearly, I’m usually pretty good at understanding it.
Honestly, I’m naturally rather prone to self-doubt, and it has taken a lot of intentional effort to be able to stand up and say, “No, the problem is not on my end. I am fully capable of recognizing when an explanation is incomplete, and the reason I have not grasped this yet is because it has not been explained sufficiently clearly.”
The proposal seems to assume that readers already know what the proposed feature does. And that is not how things should work. It is not fair to reviewers, and it is not fair to everyone who comes along in the future and wants to learn about the feature for the first time by reading the proposal which introduced it.
Again, it is not realistic to ask every reviewer to try a feature in other languages and do the research themselves.
All that stuff should be summarized and explained in the proposal. A reviewer should not need an in-depth understanding of what a keyword means or how a feature works elsewhere, in order to comprehend a proposal for Swift.
The proposal should communicate what the authors have found, and why they selected the proposed design over any other.
The design is rock solid and the proposal itself is well written and thorough. The proposal text, along with the other pending proposals in this space, will serve as an outstanding foundation for understanding Swift’s concurrency model.
I’d eventually like us to consider allowing get-only properties to be async, but I support deferring that consideration. (It’s a nuanced topic on its own given the intersection with lazy properties and property wrappers.)
Is the problem being addressed significant enough to warrant a change to Swift?
Most certainly.
Does this proposal fit well with the feel and direction of Swift?
The design feels exactly right to me. The analogies to try-throw provide a solid basis for understanding/teaching the syntactic and sub-typing rules here.
The ultimate test of the design won’t come until we’ve lived with the full Swift concurrency model for some time, but I think this proposal provides a solid foundation.
If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
I’m passingly familiar with the C# async/await model. This proposal aligns with the expectations I bring from there.
How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
Too much? I’ve read all of the pitches and threads for the full family of concurrency proposals. I’m familiar with the theory and practice of co-routine models.
It took me quite a bit thinking to grok how an async initializer would work in practice given Swift's very important constraint of full initialization. Possibly filling out the example might make it a bit clearer.
I have a hard time understanding what this means:
An async initializer of a class that has a superclass but lacks a call to a superclass initializer will get an implicit call to super.init() only if the superclass has has a zero-argument, synchronous, designated initializer.
I'm not sure I follow the section "Overloading and overload resolution" and I think it could be clarified as well.
Many such APIs are likely to be updated by adding an async form:
What is "updated" mean here? Adding a new overload or replacing the function outright?
They don't really match, because with the first one if you call doSomething() nothing gets "returned" in any sense.
Instead, we propose an overload-resolution rule to select the appropriate function based on the context of the call. (...later) func doSomething() -> String { /* ... */ } // synchronous, blocking
This kinda sounds dangerous, but also not sure if I'm reading it right. Does this imply that when someone updates to a new library all of a sudden a call to doSomething() might block when it wouldn't have previously?
What is your evaluation of the proposal?
As written, overall +.7. I love the approach that the author's are taking overall but I find the above pieces somewhat confusing.
Is the problem being addressed significant enough to warrant a change to Swift?
Yes.
Does this proposal fit well with the feel and direction of Swift?
Yes, and I very much appreciate the explicit nature of await and the cohesiveness of all the various proposals.
If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
I think when combined with Tasks and Actors this is (semantically) the best async/concurrency design I've come across. Experience with Kotlin and JS.
How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
I've been lurking and reading since the manifesto 1st edition. Read comments here and the proposal in full.
Normally, the compiler will insert super.init() for you automatically if you don't have any super.init. That won't be the case if super.init() is async.
I'm pretty sure most libraries would need to deprecate the old one first before removing it. So we should have both versions for a while.
You'd want to handle the string passed to the completionHandler though, in much the same way that you'd handle the string returned from the second version. So in that sense, the second version serves the same purpose as the first one, but the second version is in async form (which we deem desirable).
Those two are in different contexts, though.
The first part refers to when you have both the completion-handler version and async version. The compiler would need to infer the appropriate sync-async overload.
The second part refers to the fact that those two functions aren't allowed together. Doing so will be treated as redeclaration.
The first part refers to when you have both the completion-handler version and async version. The compiler would need to infer the appropriate sync-async overload.
Ah! I see where I got mixed up now. So that first section is really only comparing these 2 overloads