On the proliferation of try (and, soon, await)

[benjamin.g]

First, thanks for that very interesting topic. As a regular (aka: not PL) developer, i always take some keywords for granted because i've seen them used everywhere in other languages, without asking myself what they really bring to the table.

I'm clearly not qualified to oppose any counter-argument, however it seems to me that you're saying two very different things :
.we use "try" and "async" keywords without really understanding what value they bring (and many people are responding to this part, saying invariant-preservation isn't their only purpose anyway).

. the concept of invariant-preservation is never made visible, or understandable to the developer.

IIUC : wouldn't that second point imply that there would be so some sort of missing keyword to be added to the language, that would be completely orthogonal to both try and async, indicating that a given list of operations should be performed transactionaly (pardon my backend-developper background) ? Then the compiler would be able to infer which combination of throwing / async call should be allowed inside a given call ?

[dabrahams (Dave Abrahams)]

@adamkemp

@dabrahams

Are you saying that await signals code that would otherwise (in a non- async world) block, and thus provide an opportunity for the current thread to do other useful work?

await signals that you are potentially giving up your thread (and thus giving up the CPU) while waiting for some asynchronous response. Most likely the response you're waiting for won't be consuming any CPU itself while you're waiting so you are giving up control over the CPU to do nothing in the meantime. So the placement of that await relative to other code that you need to run is important information. Placing that await in the right place can make a significant difference in throughput. If the waiting were done implicitly then you wouldn't know to move it.

@dabrahams

IIUC, you're saying await serves as an important clue that async let can be used to improve performance

Yes!

@dabrahams

My point is that, if async let will dispatch to another thread when necessary/appropriate, then it could be used to improve performance even if the code were sync

That is not what async let is proposed to do. async let does not introduce concurrency.

OK, I'm really sorry to do this because I am sure you have a lot more experience with async than I do, but it sounds like you are mixing up concurrency and parallelism. My understanding is that the one thing async let is guaranteed to do is introduce concurrency. It was also my understanding that whether it introduces parallelism may depend on the current executor, although available information about the role of executors has been somewhat thin.

Also, according to [Concurrency] Structured concurrency,

@John_McCall

we can break down our recipe into different tasks that can happen in parallel.

Maybe that was just casual speaking but I guess that may have led me to believe things could happen in parallel. If that's not the case, it would seem to indicate these proposals aren't offering a replacement for DispatchQueue.concurrentPerform, which is an interesting twist to how I've been reading things.

Note that I'm not the only one who has suggested async will be used for parallelism, and nobody has corrected them so far:

[Concurrency] async/await + actors

Because async/await serve a different (but related) purpose than background threads. Remember that concurrency is not parallelism: running in parallel in the background is a useful construction, but not the *only* construction. If you have multiple tasks you’d like to kick off in parallel, async/await can support that by providing some kind of awaitable Future object. You’d then call multiple functions that return those futures and then await on the completion of all of the Futures (Future.gath…

[Concurrency] async/await + actors

Would it be possible to implement just the concurrency part without parallelism and GCD? That is, async functions called with await would always execute on the same thread they were called? My understanding is that this would be quite similar (if not the same conceptually) as generators/yield, is that correct?

@John_McCall, @Douglas_Gregor care to comment? Is parallelism in or out of scope for async/await?

[adamkemp]

@dabrahams

My understanding is that the one thing async let is guaranteed to do is introduce concurrency.

I have to word this very carefully because it gets confusing. async let does not by itself cause any code to run on a different thread that would not have otherwise run on a different thread. If you call a function using async let then that function itself will run exactly the same as if you had called it with a simple await. There's no difference in behavior for the function you're calling.

As I've said before, most asynchronous functions should in theory be asynchronous because they're waiting on something external. That means there's likely no thread in your current process doing any work to compute result of that async function. Your process is just waiting for a callback that ultimately gets triggered by the kernel somehow (like via a mach port).

If you use normal callbacks or a simple await then while you're waiting on that callback the call stack you were previously using will unwind back to whatever event loop or dispatch queue owns that thread, and that thread will move on to other things.

The one thing async let allows you to do is to unwind back to your function instead of all the way back to the run loop, and then your function can keep control of the thread and do other things while you're waiting for the callback. Then you can decide where you really need to wait for that callback, and at that point the stack will unwind back to the run loop, which can then handle the callback and resume your function on the other side of the await.

Is that "concurrency"? Not in the sense that there are different threads in your process both working at the same time on different CPU cores. async let doesn't cause that to happen. That is my understanding of what "concurrency" means, but it can be a very vague hand-wavy concept so discussing it leads to a lot of misunderstandings.

The reason I'm stressing this is that, again, if you start with a synchronous function then async let doesn't make it concurrent or even make it async at all because it doesn't introduce any behavior changes as far as which thread code runs on. You have to do that some other way.

All that together means the all-synchronous examples and the asynchronous examples may have very different things to consider, and having an explicit await in the asynchronous code has a lot of added value that is specific to asynchronous code. If you in theory had some way of marking synchronous functions that take a long time then it might make sense to force coders to decorate calls to those as well so that they could reason about the performance implications, but alas there's no such way to mark expensive synchronous calls. Asynchronous calls, on the other hand, should always be considered expensive (in terms of time), and that's why they're asynchronous.

[dabrahams (Dave Abrahams)]

@adamkemp

I have to word this very carefully because it gets confusing. async let does not by itself cause any code to run on a different thread that would not have otherwise run on a different thread. If you call a function using async let then that function itself will run exactly the same as if you had called it with a simple await .

…Which may itself be on a different thread, IIUC, because you might be calling into an actor with a different executor, but the current task doesn't resume until the call is finished… i.e. no parallelism between caller and callee is implied. Got it.

IIUC Task.runDetached is a way to get parallelism out of the system, and the global executor is effectively a thread pool.

The one thing async let allows you to do is to unwind back to your function instead of all the way back to the run loop, and then your function can keep control of the thread and do other things while you're waiting for the callback. Then you can decide where you really need to wait for that callback, and at that point the stack will unwind back to the run loop, which can then handle the callback and resume your function on the other side of the await .

Is that "concurrency"?

Yes.

Not in the sense that there are different threads in your process both working at the same time on different CPU cores.

That would be parallelism.

async let doesn't cause that to happen. That is my understanding of what "concurrency" means, but it can be a very vague hand-wavy concept so discussing it leads to a lot of misunderstandings.

IMO, it's not hand-wavy at all: there are really solid accepted definitions for concurrency and parallelism—you won't find contradicting definitions scattered all over the web—and as long as everybody uses those we can avoid misunderstandings. You just keep using the word “concurrency” to mean the accepted definition of “parallelism.” I can learn to translate, but it would save a lot of effort on both our parts if you'd just adopt the standard definitions.

Cheers,
Dave

[Douglas_Gregor]

@dabrahams

My understanding is that the one thing async let is guaranteed to do is introduce concurrency. It was also my understanding that whether it introduces parallelism may depend on the current executor, although available information about the role of executors has been somewhat thin.

I see how you came to that conclusion, but there is some subtlety here that is really important. When async let (or the add operation of a task group) creates a new child task, it isn't created on the current executor---it is added to the global concurrent executor.

Generally speaking, that global concurrent executor is expected to run the various tasks in parallel: the current implementation uses the Dispatch library's global concurrent queue (of the appropriate task priority) to provide that parallelism.

Now, we also have an implementation of a global concurrent executor that's effectively a run loop: it runs on the main thread and maintains a queue of partial async tasks that it uses to find its next bit of work. There's no parallelism at all, and all of the concurrency is effectively cooperative based on async functions.

From the perspective of the Structured Concurrency proposal, both should be valid implementations, and your choice is likely to depend on your environment. The most appropriate thing for most Swift developers is a global concurrent executor like the global concurrent Dispatch queue that manages a number of OS threads so you get actual parallelism, but @Max_Desiatov was recently asking about support for single-threaded environments as well.

I've taken a note to add some notion of the global concurrent executor into the Structured Concurrency proposal. Because you are right that all async let guarantees is concurrency, but that's because the parallelism is left to the global concurrent executor.

@dabrahams

Maybe that was just casual speaking but I guess that may have led me to believe things could happen in parallel. If that's not the case, it would seem to indicate these proposals aren't offering a replacement for DispatchQueue.concurrentPerform , which is an interesting twist to how I've been reading things.

You could certainly implement DispatchQueue.concurrentPerform on top of async let or (much better) task groups.

Just for fun, I went ahead and ported the merge sort implementation from the Swift Algorithm Club site over to use async let, so I could see the parallelism in action. My implementation is in this gist, but the only difference between mine and the original is the sprinkling of async and await in the mergeSort function:

func mergeSort<T: Comparable>(_ array: [T]) async -> [T] {
  guard array.count > 1 else { return array }

  let middleIndex = array.count / 2

  async let leftArray = await mergeSort(Array(array[0..<middleIndex]))
  async let rightArray = await mergeSort(Array(array[middleIndex..<array.count]))

  return merge(await leftArray, await rightArray)
}

(The await on the right-hand side of the async let will go away once we bring the implementation in line with the proposal)

I ran this on my Mac using Swift's main branch from today under Instruments, and the time profile shows that all of the cores are busy with different pieces of this parallel sort. If I were to switch over to the cooperative scheduler, there would be no parallelism but the result would be the same.

Doug

[sighoya]

+1 from me.

I even want to go further to say we assume that every function throws anything by default is a lot more stabilizing than totalize your world when something is thrown and when not by requiring the user to up-propagate this state to every function above until we catch the exception.
Wasn't that the idea of an exception at all to fit the role non-local control flow?

Toward await, why we even need to bifurcate the world in async and sync functions. Couldn't the caller decide when to call a function async or sync?
I don't even see the use of await when we have the distinction synced or asynced locals. If the left side is synced, then we block, if it is async then we are non block, that's it.

[adamkemp]

Fair enough. Clearly I’m the one mixing up the definitions.

My point, though, was that async let doesn’t make a synchronous function asynchronous. Maybe Douglas can confirm whether you can use async let to call a function that is not marked as async. My understanding was that you could not. Maybe I’m wrong.

Regardless, I maintain that there is a significant difference between a function calling a bunch of asynchronous functions and a function calling a bunch of synchronous functions, and that await adds significant value for understanding the throughput/performance when there are async calls.

[Douglas_Gregor]

@adamkemp

My point, though, was that async let doesn’t make a synchronous function asynchronous. Maybe Douglas can confirm whether you can use async let to call a function that is not marked as async . My understanding was that you could not. Maybe I’m wrong.

You can put any expression you want within an async let initializer, including synchronous code.

Doug

[adamkemp]

Interesting! I stand corrected, and I apologize for the confusion.

That is an interesting choice of behavior. I’ll have to think about it for when that proposal comes up.

[dabrahams (Dave Abrahams)]

So I think we're back to the problem of clearly identifying the value of await labels to the reader of code.

[adamkemp]

Again, I think everything I’ve said about the value of that signal still stands. An asynchronous function is nearly always going to result in your process doing nothing while it waits, so even if you have no invariants to worry about it still helps to know how it executes and where you might restructure it.

This whole tangent started because you asked whether the same applies to synchronous code. Generally it doesn’t. In the vast majority of cases synchronous code is best kept synchronous, while very commonly unnecessarily serializing sequential calls to asynchronous code by waiting for each to finish before starting the next is wasteful. So making the await implicit would be obscuring an important detail about the performance in a way that just doesn’t apply to synchronous code.

[dabrahams (Dave Abrahams)]

@adamkemp

Again, I think everything I’ve said about the value of that signal still stands. An asynchronous function is nearly always going to result in your process doing nothing while it waits, so even if you have no invariants to worry about it still helps to know how it executes and where you might restructure it.

I think your view is heavily influenced by the kind of programming that you do, which uses await to get some work done while synchronous code would otherwise block. Now that we know async is being used for both interleaved concurrency and parallelism, it seems to me that if I'm mostly using it to speed up compute-heavy tasks, it signals something else entirely (though I'm really not sure what!) And if you have an even balance between these two kinds of uses, the signal must be at best very weak.

This whole tangent started because you asked whether the same applies to synchronous code. Generally it doesn’t. In the vast majority of cases synchronous code is best kept synchronous,

Again, this really depends on your use case. I work in machine learning. We get a huge amount mileage out of parallelizing work, often across heterogeneous processing units. We do have some places where operations would block, and we generally use the same thread pools that provide our parallelism to keep cores busy while we wait for disks or network traffic, but I wouldn't say that kind of code dominates.

[Tino]

Imho this discussion would benefit from more examples — code where you can say that a certain occurrence of await is either useless or helpful.
Looking at

@Douglas_Gregor

func mergeSort<T: Comparable>(_ array: [T]) async -> [T] {
  guard array.count > 1 else { return array }

  let middleIndex = array.count / 2

  async let leftArray = await mergeSort(Array(array[0..<middleIndex]))
  async let rightArray = await mergeSort(Array(array[middleIndex..<array.count]))

  return merge(await leftArray, await rightArray)
}

I'd say that there is actually some noise: It just looks like regular code, sprinkled with lots of await (so I'm happy about the promise that some of them won't be needed anymore soon :-).
As far as I understand, the last line will stay as it is, so the question is wether those awaits indicate something interesting (like an opportunity for optimization), or if it merely emphasizes something that is obvious (I cannot use a result of a computation before that process did finish).

[Terje]

Personally I like how Swift deals with error handling. It’s the first language I have used that makes me want to deal with error handling. Swift made it easy and pleasant to deal with. Forced to use try makes it easy to see which calls can throw. The compiler helps a lot with taking error handling into account. And if I am trying out stuff I can use try? temporarily. It is a good example of progressive complexity. I don’t mind seeing try (and await) everywhere. It helps more than it hinders.

Ask yourself how many times being alerted to an error propagation point has saved you from making a programming mistake. For me the answer—and I'm not exaggerating—is zero.

A lot? With all due respect, not everyone is a rockstar programming CS-genius that works at a FAANG company. Some programmers require a bit more handholding. That doesn’t make those users bad programmers or stupid or whatever, it might mean that programming for them is just another tool to do their actual job.

[Douglas_Gregor]

@Tino

As far as I understand, the last line will stay as it is, so the question is wether those await s indicate something interesting (like an opportunity for optimization), or if it merely emphasizes something that is obvious (I cannot use a result of a computation before that process did finish).

The span between the async let and the await is where concurrent work happens. That's critically important to understanding the behavior of even this trivial example. And we need to be very careful of reasoning from simple examples.

I disagree with the entire premise of this thread. The case that await is unnecessary relies on programs having complete surety of the invariants of their system, which we know isn't the case. To accept that async calls would be unannotated means that try was such a problematic mis-step that we're willing to have an inconsistency in the language between async and throws.

I see no path for async/await without the await, and I do not believe it is worth the effort to go try to build a brand new constructive argument of why await is important given the precedent and general acceptance of try in Swift.

Doug

[dabrahams (Dave Abrahams)]

@Terje

Personally I like how Swift deals with error handling…

Ask yourself how many times being alerted to an error propagation point has saved you from making a programming mistake. For me the answer—and I'm not exaggerating—is zero.

A lot? With all due respect, not everyone is a rockstar programming CS-genius that works at a FAANG company. Some programmers require a bit more handholding. That doesn’t make those users bad programmers or stupid or whatever, it might mean that programming for them is just another tool to do their actual job.

@Terje, I'm sorry if you took that as some kind of insult; it certainly wasn't intended that way. I actually encouraged readers to ask themselves that question because I want to hear stories like yours, if they're out there. There's always a possibility I'm completely misguided, and it's good to have the data points.

Also, although you personally might not agree with this take, I don't think you have to be a rockstar-whatever to think that some trys—like those we end up repeating in Cod able conformances—aren't helping. The whole reason I've been pushing this topic (and actually the whole motivation behind most of my career, if I think about it) is that I agree with you that hand-holding is important.

Therefore, I think, how we do it makes a difference. My philosophy is, hand-holding should be done in a way that makes programmers more powerful and confident. I think forcing granular try everywhere perpetuates an illusion that error propagation is equally important/dangerous no matter where it occurs. Instead, I want try to give you a strong signal in the places where it matters, and give you the power—and a little incentive—to identify the places where it's not going to matter and get it out of the way. IMO that helps by cleaning up code and by cultivating the mental tools that ultimately help you handle errors correctly.

[Matt_McLaughlin]

@dabrahams

I think forcing granular try everywhere perpetuates an illusion that error propagation is equally important/dangerous no matter where it occurs. Instead, I want try to give you a strong signal in the places where it matters, and give you the power—and a little incentive—to identify the places where it's not going to matter and get it out of the way.

The great advantage of forcing is that it minimizes cognitive load. As the code author, there’s no nuanced decision to be made - you must use try. Now I understand that the anticipated advantage of your approach is to the reader - that it helps them focus on the places it matters the most and not the places it doesn’t. But in practice, absent a very very clear heuristic for when try is necessary and when it’s not, I think it would lead to a great deal of second-guessing code: “Did the original author of this code get it right in omitting try? Or did they make a mistake and I need to go back and add it in?”... and that second guessing adds cognitive load.

So I guess I would want to understand how you would provide clarity on where the bright line is between “must have try” and “can omit it”. How would you explain it to a novice coder such that they have minimal cognitive load making the decision? Similarly, if we accepted the idea that there was some subset of the current places marked with await where it was very important to so mark it and other places where it was not, how would you describe that line to a novice coder - how would minimize the cognitive load for the author when deciding to await or not?

[dabrahams (Dave Abrahams)]

@Douglas_Gregor

The span between the async let and the await is where concurrent work happens. That's critically important to understanding the behavior of even this trivial example.

Would you mind, please, explaining how it helps us understand the behavior of this trivial example? I'm sorry if you think it should be obvious, but it truly isn't to me. I started this thread because I want to understand this stuff better.

And we need to be very careful of reasoning from simple examples.

I'm also trying understand what your point might be here. If, as you say, even this trivial example demonstrates that await is helping, I personally will be very glad to reason from it that await is just as valuable in the parallel case as in the interleaved concurrent case, and that would leave very few questions on the table about why we have it (fair warning: I’ll probably ask whether it should be one keyword or two in that case, though).

I disagree with the entire premise of this thread. The case that await is unnecessary relies on programs having complete surety of the invariants of their system, which we know isn't the case.

Sorry to contradict, but that is not the premise of the thread. This is an exploration of some possible design trade-offs, and a search for the best balance in addressing what some of us see as a problem.

To accept that async calls would be unannotated means that try was such a problematic mis-step that we're willing to have an inconsistency in the language between async and throws .

Actually no. We've been talking in this thread about giving the programmer control over the granularity of try using various forms of try block: either try {...} or try do {...}. If async ends up having a similar signal value in the programming model to try, consistency could be maintained by allowing async to join or replace try in those syntaxes.

But my hunch that async and try have the similar signal value characteristics hasn't played out so far. In that case it is a totally legitimate question whether we'd be better off not solving the problems that motivated this thread, or breaking syntactic uniformity, and the right answer isn't obvious to me yet.

I see no path for async/await without the await, and I do not believe it is worth the effort to go try to build a brand new constructive argument of why await is important given the precedent and general acceptance of try in Swift.

I @-mentioned you to ask for some facts about the proposals, but that wasn't intended to drag you into a discussion that you consider to be a waste of time. I’m digging into these questions because I think it’s an important part of the language design process, and the process of learning how to use the language we ultimately design. Even if it were a foregone conclusion that we have granular await everywhere, I’d want to crisply identify its role in the programming model. That will help me use Swift more effectively, and more importantly, teach others to do so.

[dabrahams (Dave Abrahams)]

@Matt_McLaughlin

The great advantage of forcing is that it minimizes cognitive load. As the code author, there’s no nuanced decision to be made - you must use try. Now I understand that the anticipated advantage of your approach is to the reader - that it helps them focus on the places it matters the most and not the places it doesn’t. But in practice, absent a very very clear heuristic for when try is necessary and when it’s not, I think it would lead to a great deal of second-guessing code: “Did the original author of this code get it right in omitting try? Or did they make a mistake and I need to go back and add it in?”... and that second guessing adds cognitive load.

I do appreciate that argument. However, to be clear, we're not talking now about omitting it, simply making it possible to change its granularity so it applies to a block.

If we're gonna worry about this nuanced decision, I think we have to acknowledge that there's already an even more nuanced decision about granularity—where in the expression to place it—that nobody second-guesses. That's because it very seldom matters for exactly the same reasons that precisely locating the throwing statement seldom matters. Ultimately, some form of try block lets us replace many little nuanced decisions with one big one. The reader can quickly think, “is there any chance we're breaking an invariant here? No? then this is OK.

So I guess I would want to understand how you would provide clarity on where the bright line is between “must have try” and “can omit it”. How would you explain it to a novice coder such that they have minimal cognitive load making the decision?

In this post I gave some ideas about how tooling can help, but you're asking about guidelines, which is really a much more important question. So thanks for that

For the novice coder, I'd explain it this way:

• try helps us identify the places where control flow jumps out of a function, just like return. If you're unsure, just use it wherever the compiler tells you to, as close in the throwing expression to the actual propagation point as possible. But…

• Just making the compiler happy isn't enough to make your code correct. The whole point of writing try is to give you an opportunity to think about the consequences of that early return and make sure anything that needs to be cleaned up is taken care of (usually with defer). You can't leave this step out!

• If you use Swift errors enough, pretty soon you'll start to recognize patterns where you have to write several trys in a function, but there's nothing to clean up and there never will be.

  • The simplest examples are functions that compute new values and don't have side-effects (show example).
  • Also, functions like the encode and init(from:) methods of Codable types, whose effects are on a coder that is expected to be incomplete in case of error (the partial initialization of an instance is not an effect, since the compiler takes care of cleanup for you). (show example)

  In general, any function that doesn't need to take special clean-up actions in case of an early exit (including translation of errors thrown by its callees), has correct error handling regardless of exactly where the error is thrown. That's where you might consider using a try/try do block at the top level of your function. That will make your code more readable and signal to a reader that there's a quick way to validate that you've thought your error-handling through correctly: simply confirm that there are no clean-ups. (link to article about what kind of clean-ups make sense and which don't).

• Another guideline: avoid putting catch or defer statements inside try/try do blocks [if we don't have the compiler ban that outright, which I think we should]. If you find yourself wanting to do that, replace the block with expression-level try markers, and think about the consequences of early exit before and after the catch/defer to make sure you've got it right.

Similarly, if we accepted the idea that there was some subset of the current places marked with await where it was very important to so mark it and other places where it was not, how would you describe that line to a novice coder - how would minimize the cognitive load for the author when deciding to await or not?

I wouldn't presume to give guidelines about await yet, because a) I'm not at all sure there exist any awaits that are low-value, and b) more importantly, I don't understand what it means in the programming model yet. A big part of the point of challenging its value in this thread is to identify that meaning if it exists.

[dabrahams (Dave Abrahams)]

@benjamin.g

First, thanks for that very interesting topic.

You're most welcome!

As a regular (aka: not PL) developer, i always take some keywords for granted because i've seen them used everywhere in other languages, without asking myself what they really bring to the table.

I'm clearly not qualified to oppose any counter-argument, however it seems to me that you're saying two very different things :
.we use "try" and "async" keywords without really understanding what value they bring (and many people are responding to this part, saying invariant-preservation isn't their only purpose anyway).

[Aside: at @Matt_McLaughlin's prompting I came up with a way to describe the problem that (nicely, IMO) accounts for those other purposes, FWIW.]

. the concept of invariant-preservation is never made visible, or understandable to the developer.

Well, I wasn't actually coming out and saying that, but it's good you mentioned it because it's heavily tied into everything we've been discussing. We have some strong clues because of the way we use the language about where invariants are broken (e.g. mutating, access control levels) but it's true that it's not a first class concept in the language. Designing language features that express it directly would be an interesting exercise, but to me it looks like a very hard thing to do well.

IIUC : wouldn't that second point imply that there would be so some sort of missing keyword to be added to the language, that would be completely orthogonal to both try and async, indicating that a given list of operations should be performed transactionaly (pardon my backend-developper background) ? Then the compiler would be able to infer which combination of throwing / async call should be allowed inside a given call ?

I don't know; that's also an interesting feature idea, but I think it's different from the one that surfaces invariant-preservation as a first-class thing. There are lots of operations that are non-transactional in case of an error and yet preserve all knowable invariants (sorting an array, for example).