My take is that it adds complexity for the author and subsequent readers of code.
A small change in a defer block suddenly turns a whole lot of other keywords into suspension points.
That alone seems like it would be hard to grasp and could lead to bugs and oversights.
I'm not convinced that's a real problem. These are places where the control flow is already jumping elsewhere. They wouldn't immediately suspend. They would just jump to an existing defer block, which would follow the same rules as any other block of code, where suspension points are clearly marked with await. I can't think of a reason why someone would need to know that a try/throw/return statement might lead to a suspension in a defer block.
Can you give an example of where that would be confusing or misleading? Where someone might have a bug because they didn't understand what would happen?
If you're taking the behavior of @asyncHandler, the asynchronous block would run concurrently with the "below async block" code. For me, that's a strong reason not to introduce something like the asynchronous block, because it subverts the expectations set up by Structured Concurrency, which let you assume that the code executes linearly.
We're going into a lot of detail for an asyncHandler pitch that's not-yet-written. If it were written up in anything other than the prototype implementation, it'd say that (for example) @asyncHandler functions cannot return a value, cannot have inout parameters, and cannot throw, because all of their work happens on a separate task and you don't get to see or wait for that task.
Doug
This is my general feeling, but I don't feel strongly about it. Should we mark such defer blocks as async like we have to for functions, e.g.,
defer async {
await fileHandle.close()
}
Okay, here's an example that feels a little contrived, but perhaps is indicative of more-likely problems:
let lock = mutex.lock()
defer {
lock.unlock()
}
// The contrived part is what you would do in here
defer {
await fileHandle.close()
}
return // implicitly suspends while closing the file handle, so the unlock might happen on a different thread
Doug
To me that implies that any subsequent statement that would exit scope and trigger that defer would have to be called with await. Thus those hidden suspensions points are no longer hidden. I think it might not be a bad idea.
Perfect!
That’s exactly what I want: the body of asynchronous is executed asynchronously.
Heh, it’s interesting that we see the same code, share the same understanding of how it works, and draw such different conclusions. I think code using the asynchronous function is highly readable, and straightforward to understand.
Now, I’m not currently advocating that a function like asynchronous should be included in the standard library, but I find the code using it to be much nicer than the (otherwise identical) code using Task.runDetached.
• • •
Personally, if I were designing an async system, I would probably start by seeing if there’s a workable model where sync and async functions can both call each other directly.
The basic story would be that the result of an async function needs to be awaited before use, and a sync function cannot use await.
So you can call all the async functions you like from synchronous code, but you can never use their return values. They just do stuff in the background.
However, if async let were a first-class construct, then the result of an async call could be stored as such and passed into another async function which could then await it.
In my mind, that would greatly simplify the mental model: an async function is always asynchronous; you can call it from anywhere, without any ceremony; and if you want its return value then you need to await it.
I don’t know enough about the implementation side of things to say whether that can be made into a viable and efficient system, but from a naive perspective it’s how I would want to think about asynchronous functions.
It's worth noting that what you apparently mean by "asynchronously" is what this set of evolution proposals actually calls "concurrently".
In this proposal's usage, anything that's not synchronous is asynchronous, but not everything that's asynchronous is concurrent. In fact, specifically and importantly, this particular proposal (SE-0296) is only about non-concurrent asynchronicity.
Regarding @asyncHandler, I think what we really want is sequential asynchronicity, not concurrency, though — since there is no actual design yet — it's a bit early to have that discussion (and it's not really for this forum thread).
What I mean is that an @asyncHandler @IBAction function (for example) likely needs to run its body sequentially on the main thread. For this to be asynchronous, this means "return now but run this code later", i.e. something that behaves similarly to DispatchQueue.main.async.
Is there a reference which defines these (and other related terms) as they are being used in the proposals?
If it's down to the naming of Task.runDetached, let's please talk about this in the discussion of the task library.
That's what this proposal does.
That's effectively the semantics of Task.runDetached, but you want a non-awaited call to an async function to behave that way? Again, that breaks the mental model of structured concurrency. Consider:
func f() async { }
func g() async { }
func h() async {
f()
await g()
}
In the current proposal, the call to f() is an error. If calls to asynchronous functions without await became detached tasks, then this little error of omission with await now means that f and g are running asynchronously when you didn't mean them to.
That's a future. Task.runDetached returns one (called Task.Handle, subject to design discussion elsewhere). async let is intentionally hiding the future to keep the model simpler. async let is part of Structured Concurrency, not this proposal.
Doug
The nearest to a definition that I could see was the last paragraph of the introduction:
This proposal defines the semantics of asynchronous functions. However, it does not provide concurrency: that is covered by a separate proposal to introduce structured concurrency, which associates asynchronous functions with concurrently-executing tasks and provides APIs for creating, querying, and cancelling tasks.
Right, but the proposals are essentially going to be accepted together or redesigned together.
In order to review this proposal (which I have not yet done), I need to understand how the features of this proposal are intended to be used.
I do not want to end up in a place where the community has accepted some of the proposals, then during review of a later proposal decides that a substantially different design would be preferable, requiring that we either revisit an accepted proposal, or settle for a model that we find inferior because we refuse to do that revisiting.
Right now people are saying, “That use-case is part of a different proposal which has not yet been reviewed.”
But if we accept this proposal and then move on to reviewing the others, when questions come up about the overall design people will say, “The basic behavior has already been accepted and is no longer under review.”
In order to review the basic behavior and decide if it’s what we want, it is necessary to understand and consider the full model that is being proposed.
I've already answered this meta-comment. That said, we don't need the details of async let to address your idea, nor do we need the exact spelling of Task.runDetached to be nailed down, which is why my I responded the way I did.
As I understand it, your idea boils down to allowing synchronous code to call an async function directly, and giving it "detached task" semantics. I replied why I consider that a poor design choice, but didn't receive a technical response. If you want to argue for your idea, you can and should address that technical objection.
Doug
I feel like that example is more of an example of "code not to put in an async function period". It would be just as dangerous to write this:
let lock = mutex.lock()
defer {
lock.unlock()
}
// ...
await someFunc()
Since the await happens possibly many lines removed from the defer block it would be difficult to ensure that the defer happens on the thread/queue you expect it to run. I think people who have such APIs are going to have to rethink their structure to work well in async code, and I don't feel like an async defer block changes that significantly.
Oh please no...that would be terrible UX. I believe conceptually a developer does not need to know that a try/throw/return statement might be a suspension point because the actual suspension point is in the defer block itself. The try/throw/return statement is control flow, and thus it results in jumping to some other place. That other place may happen to suspend, but that happens after the normal control flow.
If you required await on every one of those statements it would be very confusing, and introducing a single await in some far-removed code could affect many lines. I would almost rather not have the feature. But I still think await in defer can work, and I still think there's not quite a strong enough reason (that I've seen) that it shouldn't work.
I’m not so much arguing for a particular design, as I am exploring the space of possible designs. Whereas the proposal authors have already spent significant time considering these things, others (such as myself) have not.
It is not enough just to understand the proposal: in order to express meaningful support for it, I need a solid grasp of what the alternatives are.
One is as I described above.
Another would be to say that calling an async function is always implicitly awaited (so no await at the call-site), and you must explicitly write async at the call-site to get the detached behavior. That way concurrency locations are marked with async, and otherwise code runs sequentially.
In that model, async let works as proposed, but if you don’t want the return value (or there isn’t any) you could just write async foo() (which would be equivalent to async let _ = foo() as proposed).
This would still let sync functions call async ones directly, with the only ceremony being the presence of async before the call.
The proposal provides fairly significant rationale for needing await, so if you want to pursue this design, you're going to need to:
await is unnecessary boilerplate that should be removedPersonally, I don't think this is a good technical direction. You're syntax-optimizing the least-important case (creating a detached task) at the cost of hiding potential suspension points. It's also a fairly significant divergence from the async/await designs of effectively every other language out there, so it's going to need some strong rationale.
Doug
I said it was contrived ;)
I agree that I wouldn't want this to force us to have await return, await try, await throw, etc.
There's no technical reason that I can see, either. There are use cases (e.g., closing a file handle), and the reasons I can come up to not allow it---somewhat less-obvious potential suspension points (although there's still clearly an await in the source within the defer) and asymmetry with throw (which is banned in defer for legitimate reasons)---aren't very strong.
So let's chalk this up to "we should remove this restriction unless more arguments come up in favor of keeping it."
Doug
Again, I’m not pursuing any particular design (whereas the proposals are doing so).
I am trying to understand what alternative designs are possible, so that I can compare them with the proposed design.
If the proposal authors have strong reasons for preferring the proposed design over the alternatives (which one assumes they do), it would be useful to document those reasons in the proposal.
The “alternatives considered” section of the proposal at hand does not list any of these alternatives that we’re discussing. This makes it difficult for reviewers to understand why the specifics of the proposed design were selected over any given alternative.
I am confused by this.
The motivation section of the current proposal is dominated by (in fact I believe it consists exclusively of) discussing ergonomic problems with completion handlers. Async/await is presented as a clean replacement for completion handlers.
But completion handlers are (almost by definition) run in a “detached” manner. Some code calls a function which takes a completion handler, and then continues along sequentially. The function that took the handler, meanwhile, does its thing in the background then calls the completion handler later.
The entire purpose of functions that take a completion handler is to enable “detached” or concurrent operation.
It seems strange to propose a replacement for completion handlers, which does not solve the basic problem that completion handlers are designed for, namely running a detached task.
And it seems even stranger to categorize creating a detached task as “the least-important case”.
From my reading of the proposal, its entire motivation is to replace completion handlers, and the primary (sole?) purpose of functions that use a completion handler is to perform a detached task.
• • •
Furthermore, the alternative from my last post is syntax-optimized for the “await” case. The “detached” case requires an explicit “async” notation in that model.
The fact that completion handlers run "detached" is not a feature; it's a consequence of only having language facilities for wrapping up completion code in closures and then scheduling that for background work somewhere else. Let's consider the first example from the proposal:
func processImageData1(completionBlock: (_ result: Image) -> Void) {
loadWebResource("dataprofile.txt") { dataResource in
loadWebResource("imagedata.dat") { imageResource in
decodeImage(dataResource, imageResource) { imageTmp in
dewarpAndCleanupImage(imageTmp) { imageResult in
completionBlock(imageResult)
}
}
}
}
}
Note how all of the work of the function is done in that deeply-nested stack of closures. Yes, every one of those functions is effectively creating a detached task, because that's the only mechanism we have today.
The whole point of async/await is to take that work that's buried in those nested closures and linearize it, giving it normal control flow, but without losing the asynchrony. Your proposal to have a call to an asynchronous function create a detached task that runs concurrently with the next line of code---with no indication in the source that this is the case---is completely against the goals of async/await because you don't get linearization. In a sense, it's worse than the status quo, because at least today you have closures that help you reason about "this is different code."
Detached tasks are fine when they are needed, but you are assuming that the use of detached tasks in today's completion-handler world imply that they are the common case for async/await. They are not.
It is not possible to consider every objection or every design direction when writing a proposal. In this case, the design directions you're asking us to explore further seem to go against all of the prior art for async/await as well as its explicit design goals. As I noted before, I think this is the wrong direction, for reasons I've explained in several ways. If you wish to effect some change here, you are going to need specific suggestions and examples.
This might be the point where you need to go write some async/await code, whether in Swift or some other language, to get a stronger feel for the model we're going for.
Doug
Okay, this makes sense. It would have helped me understand if the proposal had spelled these things out.
That was the first alternative I described.
The second one did not have that issue, and would use async to demarcate the detached call.
The proposal as written did not impart that understanding to me. Similarly, it also did not impart an understanding of what the common cases for async/await are expected to be.
All the examples show one async function calling another. This is good as far as it goes, but it does not help me understand when and how the authors expect async functions to be used.
As I mentioned, the motivation section only talks about replacing callbacks, and callbacks always run detached.
So if the authors have a different vision in mind for how to use async, it would be helpful for the proposal to elucidate that so others can understand.
I am not asking for “every objection” to be addressed.
I am saying that the proposal should explain why the particular solution in it was chosen, and which possible alternatives were considered. This is standard practice for Swift Evolution.
I would love to write some async/await code.
I would especially love to write some code that uses async/await in the manner intended by the authors of the proposal. And that requires me to understand what that manner is.
I would love to try it in different languages, but I don’t know which languages have similar models to what is proposed. I do not see a section in the proposal which compares and contrasts the proposed implementation with that in other language.
It would be helpful if the authors could provide some information about what they have found works well or not so well in other languages, what other ideas they considered that may not already exist “in the wild”, and why they believe the design choices in the proposal are superior to the other options they considered.
• • •
Adopting a proposal like this is not simply saying “Yes I’d like to have async/await in Swift.”
Rather, adopting this proposal will make a fundamental decision about how asynchronicity will work in all future versions of Swift.
It is not enough to support the idea “in principle”. It is necessary to achieve high confidence that the chosen model is the best that we can come up with.
And frankly, it is not realistic to expect every reviewer to do all the research individually to reach that conclusion on their own.
It is a responsibility of the proposal authors to provide sufficient information, both to help reviewers understand the decisions involved, and to establish confidence that the authors themselves have done the research and reached the best possible solution.
I have no doubt that the authors have done their research, but the proposal in its current form does not adequately communicate what that research was and why it led to this design choice over any other.
The burden of proof rests with the proposal authors to demonstrate that this one specific design is the best that can be found.
Look, not to be blunt, but you write very long posts, the latest few of which indicate that you misunderstand what async/await is about and is trying to solve.
The burden of proof rests with the proposal authors to demonstrate that this one specific design is the best that can be found.
Well, to a certain extent. But it also falls upon the person who is confused to try to read up in the subject.
Maybe you could study how it works in the other languages where it’s implemented first, such as C# where I guess it originated? Or at least be more terse. This is quickly becoming a meta discussion.