On the proliferation of try (and, soon, await)

[dabrahams (Dave Abrahams)]

@Chris_Lattner3

@dabrahams

That base is already covered by “throws.” The language prevents exceptions from being ignored by requiring “throws” on throwing functions. Either a function catches all the errors that may be thrown into its body, or it has to declare that it throws. You don't need try for that.

Right, I understand that try has no semantic meaning to the compiler and doesn't generate code.

That wasn't my point at all. My point was simply that even if there were no try in the language, the need to write throws on the caller's declaration (or catch and not propagate the error in its body) would still be forcing the programmer to acknowledge that a region of code can throw, and could be said to prevent exceptional control flow from being ignored, which was the deficiency you correctly ascribed to the C++ and Java exception handling model. Yes, throws is acknowledgement at a lower level of granularity than try in many cases, but it serves the purpose.

The purpose of try is to inform the writer of code that there is a nonlocal transfer of control going through the function (therefore, if you call malloc , you should think about defer ring that free ) and to provide a similar signal to people who come back around to maintain the code.

Yes, invariants like “every piece of memory you allocate is managed by something that will clean it up” are important to protect. As I've acknowledged, granular try really pays off in those situations where invariants are broken. But, as I've also said, those situations are surprisingly rare, and I argue there should be a way to avoid granular specification of try for the other cases.

This is one of the downsides of C++ exception safe logic - it often leads to the advice of writing "exception safe" code pervasively with RAII and other techniques, which are often massive overkill and sometimes obfuscate simple logic.

I don't know whose advice you've been reading, but the most prominent advice I've heard—and this isn't even advice I came up with—is to try to do all your throwing operations before invariants are broken or after they've been repaired. 99% of exception-safe programming in C++ is about, in decreasing criticality:

• picking the right order of operations
• documenting which things can throw—thank you for being a language construct, Swift throws annotation!
• occasionally documenting the extra guarantee that an operation has no effects in case it throws.

There are many approaches to using exceptions in C++, but the most successful ones I've seen are "-fno-exceptions" or "write code that assumes that anything can throw".

Yes, -fno-exceptions works out for a lot of people. I don't know who you think is following the latter strategy, but let me assure you it is demonstrably impossible to write exception-safe code under that assumption.

I think you're misinterpreting the most effective strategy, which is to mentally separate the kinds of code where it doesn't matter which things can throw, from the code where it matters, and only sweat the details in the latter cases (which is where I would apply try granularly). Because so much code falls into the former category, it would be easy to think of that as “write code that assumes that anything can throw,” but the other category is still crucial.

These two extremes are what I meant by binary. In practice most code doesn't throw even when using exceptions, and C++ isn't great at conveying that or providing that as a programming model.

Oh, that is very true. However, those issues are addressed in swift primarily by having the throws keyword and by our decision not to report errors for a pervasive yet typically un-recoverable condition: out of memory. The try requirement does help a bit, but its mandatory granularity has serious downsides.

@dabrahams

By “back pressure” I think you are referring to the idea that writing try is odious and the theory that therefore people will try to avoid creating APIs that throw. The premise here is that the language should have this error-handling feature but somehow, at the same time, we need to make it painful because we want to discourage its use.

Now I think your argument (which is reasonable) is that the backpressure is so odious that it makes error handling unpalatable to use in some cases

Really, I was saying the opposite (re-read? It should have been clear). I've only ever met one person who avoided using error handling because of the backpressure, and it was sad that they had to wrestle with that dilemma, because error handling was the appropriate mechanism for what they were doing (essentially, cancellation!).

I was saying that:

• try isn't actually odious enough to make most people think twice about throwing
• besides, I don't know why you would want to discourage the use of the first-class error reporting feature
• and this kind of discouragement is unprecedented in Swift (see below).

I think this is a fair concern, which is why I think we can improve things here. I don't think it makes the approach incorrect though, we just never got around to improving sugar to make this sweeter.

I never said it was incorrect. The only thing I ever objected to was the indiscriminately forcing granular marking of all trys, which is certainly a legitimate design choice but one that makes some very bad trade-offs, IMO.

@dabrahams

Well, I don't buy it as a language design strategy. First, it's punitive in a way that's inconsistent with the character of the rest of the language—thank goodness we haven't taken this kind of approach elsewhere, or Swift would be much less enjoyable to use.

I understand that this is something you don't philosophically agree with, but we do use this in multiple places in the language. For example, we require & at the caller side of inout parameters to make the mutation semantics more clear. We require the if let x = x pattern instead of making control flow implicitly rebind things, etc.

No, those are in fact completely different cases, because the measures are not designed to discourage use. I'm sure you would never characterize & as “backpressure against mutation.” I maintain that is in fact something that we have not done anywhere else in Swift.

I understand that you're opposed to the concept of try marking and don't find it valuable.

Again, that's a misunderstanding. I think it's extremely valuable in extremely few places. I am opposed to forcing granular try annotation in those places where it doesn't matter.

We can serve users better by teaching them where it does matter, and figuring out how to encourage it to be used there. As I've said elsewhere, I can easily think of ways to build that encouragement into tools.

@dabrahams

I don't think that scales. What happens when we add an impure effect (or whatever the next effect dimension is)?

I would expect that these stack, in the existing case, I would expect that await try do { ... } to work.

Context, my good, man; context is everything! What I was saying doesn't scale is the idea of “introducing a keyword that combines try and await into one keyword.” As we get more effect dimensions we then have an O(N^2) matrix of possible combination keywords, right?

This comes back to the philosophy of marking and whether it is a good idea to put syntactic burden into this. I agree that indentation isn't worth it for the "three try" case, but it would be worth it for the much larger case.

IMO that would make this a fairly weak intervention, probably not worth its weight in the language.

@dabrahams

Eliminate the need for do and allow try { ... } to mark an entire block as throwing.

The major concern I have with that it that it would try {...} catch (){} "work" and that would be confusing because it would look too much like C++ and Java and is not something we want people accidentally reaching for.

I thought you might bring something like that up, but how, exactly would that parse? I would expect catch still needs a preceding do, neh?

@dabrahams

Allow that at the top level of the function:

something like this doesn't have the same problem as I mention above, but has other challenges, I'm curious to know what others think about it.

What are the other challenges?

[David_Catmull]

@dabrahams

We can serve users better by teaching them where it does matter, and figuring out how to encourage it to be used there. As I've said elsewhere, I can easily think of ways to build that encouragement into tools.

Can you please provide an example or two? My concern with allowing something like a try {} block is that it will tend to be abused, and people would not be encouraged to think very carefully about error handling. If you already have a solution in mind for that, I'm very curious.

[adamkemp]

@dabrahams

Are you saying that await signals code that would otherwise (in a non- async world) block, and thus provide an opportunity for the current thread to do other useful work?

No, await doesn't add any new signaling behavior. We need to keep in mind that async/await is just new syntax for doing things you could already do before with callbacks. It's just easier now.

If you wrote the example without async/await (i.e., using callbacks) then you would probably also notice the inefficiencies because you could see (very clearly) which things are asynchronous and which are not. Unfortunately, doing things in the more efficient way with just callbacks gets rather complicated. Just think of how frustrating it is to write code that waits for N completion handlers to finish before proceeding. The structured concurrency spec combined with async/await makes that much easier to write, which is why it's such a good example.

The point of my explanation was to specifically compare async/await as proposed with your alternative suggestion that await should be optional. So we're talking about code that does exactly the same thing but without the await keyword being there (i.e., example 1 versus 2 in my post). I'm trying to explain why leaving out the await keyword obscures what's actually happening, which would in some cases lead people to write code that goes slower because it just ends up sitting idle sometimes.

@dabrahams

Hmm, but—correct me if I'm wrong—I got the impression that async let evaluation is allowed to happen on an arbitrary thread. So even if these functions were all synchronous (in which case there would have been no await s to clue us in), wouldn't this refactoring still make sense?

If all of these functions were synchronous then the calling thread would be busy the whole time. You could maybe improve performance by dispatching some of it to another thread and then waiting for it to complete (i.e., by adding concurrency). Whether that makes sense depends very much on the nature of that synchronous work.

The reality is that most of the time that an API is asynchronous it is because of something like I/O, where the CPU isn't necessarily doing much work at all. I would argue that in most cases if the CPU is busy then an API shouldn't be asynchronous (let the client decide how to schedule the work with knowledge of what else is going on in their process). So it only really makes sense to make an asynchronous API when it's asynchronous because it's truly waiting on something external (networking or disk I/O, for example).

When you have those kinds of asynchronous functions it very often makes sense to go on and do other work while you're waiting. Kick off more requests, maybe, or crunch some data you already have on hand. In fact, that's the whole point of asynchronous code in the first place because whether you're using callbacks or async/await what you're doing is allowing the original thread to move on and do other things. Sometimes you have other things to do right there in that function you're working on, and async/await can simplify the structure of your code to the point where you can more clearly understand how to restructure your code to improve the performance. When you can see clearly "this is just a sequence of kicking off some async requests plus some number crunching" then it's easier to write it as "go ahead and kick off all of the async requests up front, then crunch the numbers, then wait for the results of the async work, then compute the final answer and return". When you can't see what's async and what's not then you would likely end up writing it as "kick off and immediately wait for each request, then compute things and return the result", which leads to a bunch of wasted CPU cycles.

The non-async-await alternative is a bunch of very complex (often nested) callbacks that make it hard to understand what is even happening when and how data flows from one place to another. The async-without-explicit-await alternative is simpler to write (compared to non-async-await), but obscures where you might be sitting idle waiting for results. The actual proposal sits in between: it simplifies the structure of the code and makes it clear where you're waiting for results.

[David_Ungar2 (David Ungar)]

It feels to me as if the underlying issue is the idea that there is one way to see code that works for all use-cases. In other words, sometimes I want to see "try", or an inferred type, and sometimes I don't. Sometimes I want to see function bodies, and sometimes I don't. What I want (for me, your mileage may vary) is an IDE that unifies the idea of different perspectives on my program across all the different situations were the notion applies and that provides good affordances for switching perspectives, and viewing from perspectives. It's not a small ask, but I would argue that it helps to accept the underlying principle.

[QuinceyMorris]

The following is about await. I don't really have anything to say about the try part of this debate.

I have started to believe that the overarching principle here ought to be: Things should do what they appear to do.

After all, the guiding principle of the basic async/await proposal is that code that appears to be written to execute sequentially should execute sequentially. The fault in our current (completion handler) model is that it obscures the sequential execution order.

Now, refining that principle, within code that executes sequentially, code that looks synchronous should execute synchronously. Code that should execute asynchronously should look asynchronous.

Trying to be cleverer than that is IMO a mis-step. Outside the echo chamber that is evolution proposal discussion, I would say that code looking like this:

let dinner = makeDinner(ingredients)

"looks" synchronous, according to half a century or so of procedural language convention. Code that looks like this:

let dinner = await makeDinner(ingredients)

represents a clearly asynchronous appearance.

Rather than thinking of await as a mere syntactic marker that confirms the asynchronous-but-inexplicit nature of makeDinner, I think it's more useful to think of it as an await statement, like a switch or if statement, invoking this behavior rather than that behavior at this point in the code.

In principle, the language could give meaning to both of the above examples: synchronous when you want to block, asynchronous when you don't want to block. Of course, we aren't really going to offer the synchronous form, because that makes it too easy to write poorly-behaving code, but I would argue that this is what the await-less variant means.

Appropriating the form to mean the opposite is, as I've said, a mis-step IMO.

While I appreciate the argument that await could be omitted when nothing is at stake in the distinction between synchronous and asynchronous-sequential at that point, I would also argue that this overcomplexifies life for programmers who are merely human. It introduces a meta-question that must be applied at every use site, as to whether the keyword is "necessary" there.

As a mere human, I say: let it be boring rather than anxiety-producing. Let's let it look like what it does.

[Chris_Lattner3 (Chris Lattner)]

@dabrahams

That wasn't my point at all. My point was simply that even if there were no try in the language, the need to write throws on the caller's declaration (or catch and not propagate the error in its body) would still be forcing the programmer to acknowledge that a region of code can throw, and could be said to prevent exceptional control flow from being ignored, which was the deficiency you correctly ascribed to the C++ and Java exception handling model. Yes, throws is acknowledgement at a lower level of granularity than try in many cases, but it serves the purpose.

Right, I agree these are different things. throws isn't enough to inform readers about what in a scope throws, which is the purpose of the try marker. This is a tradeoff - the existing design of Swift decided that statement granularity is important enough to be worth the annotation burden, but decided that per-subexpression granularity would not be.

@dabrahams

As I've acknowledged, granular try really pays off in those situations where invariants are broken. But, as I've also said, those situations are surprisingly rare, and I argue there should be a way to avoid granular specification of try for the other cases.

Sure, I think we just have to agree to have a difference of opinion here. I agree with you that it depends a lot on the kind of code you end up writing.

@dabrahams

I think you're misinterpreting the most effective strategy, which is to mentally separate the kinds of code where it doesn't matter which things can throw, from the code where it matters, and only sweat the details in the latter cases (which is where I would apply try granularly).

While I agree this is possible, I haven't seen this widely accepted by the C++ community. Are there textbooks or prominent blog posts that explain this approach and advocate for it?

@dabrahams

No, those are in fact completely different cases, because the measures are not designed to discourage use. I'm sure you would never characterize & as “backpressure against mutation.” I maintain that is in fact something that we have not done anywhere else in Swift.

This is directly analogous to inout. It is syntactic burden that isn't necessary. Without this "backpressure" on call sites, people could sprinkle inout on function arguments. This isn't meant to discourage the use of either feature, it is meant to discourage needless use of it (i.e., people marking all functions as throws, or all arguments as inout). The back pressure makes the presence of these features more meaningful in practice.

@dabrahams

I don't think that scales. What happens when we add an impure effect (or whatever the next effect dimension is)?
[/quote]

I would expect that these stack, in the existing case, I would expect that await try do { ... } to work.

Context, my good, man; context is everything! What I was saying doesn't scale is the idea of “introducing a keyword that combines try and await into one keyword.” As we get more effect dimensions we then have an O(N^2) matrix of possible combination keywords, right?
[/quote]

I'm sorry but I really don't understand what you mean here.

@dabrahams

@dabrahams

Eliminate the need for do and allow try { ... } to mark an entire block as throwing.
[/quote]

The major concern I have with that it that it would try {...} catch (){} "work" and that would be confusing because it would look too much like C++ and Java and is not something we want people accidentally reaching for.

I thought you might bring something like that up, but how, exactly would that parse? I would expect catch still needs a preceding do , neh?

It depends entirely on the design of the feature, you could overload it in a bunch of different ways with different tradeoffs, but they all introduce try confusion in my opinion. Currently try is not a statement. Creating a try statement (no matter which specific design points you choose) muddies its role. Making it syntactically similar to things in other widely used languages is a bad thing to me.

The advantage of try do { } is that it continues the role of the keyword as a marker, with the explanation that "it applies the try marker to the entire scope of the do".

-Chris

[crusty]

@jrose

extension Array {
  mutating func mapInPlace(_ transform: (Element) throws -> Element) rethrows {
    for i in self.indices {
      self[i] = try transform(self[i])
    }
  }
}

No invariants are broken, but that’s just the baseline functionality. The thing is, a throwing function should have the option to make guarantees about postconditions in the failure case beyond just “no invariants have been broken”. In this case, the author of the function might want to restore the original contents of the Array on failure.

Focus, Bunky; focus! There's an invariant there that's broke; you just hain't learned to see it yet. Crusty'll do a little dance for the person who finds it, an' offer up a bonus challenge to boot!

[dabrahams (Dave Abrahams)]

@David_Catmull

Can you please provide an example or two? My concern with allowing something like a try {} block is that it will tend to be abused, and people would not be encouraged to think very carefully about error handling. If you already have a solution in mind for that, I'm very curious.

There's a spectrum, depending on how conservative you want to be and how easy you want implementation to be. The most conservative strategy encourages “precise try” (by which I mean placing try as close as possible to the actual error propagation point) in any code that does mutation. That would let you address part of the problem, making it convenient to use a try {} block in the code that uses lots of closures , but would still discourage it in the cod🐟e that [de-]serializes .

[Note that many strategies are going to encourage precise try in code that operates on class instances, since those APIs can mutate without anything declared in the type system.]

But there are ways to be less conservative, reducing the number of false-positive warnings, that would still be effective. For example, you can limit the encouragement of precise try:

• to those places where throwing can happen between mutations (a throw that happens before or after all mutations is harmless to invariants). That would add no risk but is a bit harder to implement.
• to those places that use non-public mutating API (adds a little risk; types that aren't well-encapsulated may expose invariant-breaking API publicly). Interpreted the right way, this covers [de-]serialization code.
• to those places that use mutating API that's less accessible than the type(s) it operates on. Adds a bit more risk than the last bullet, I suppose.

[later: in this post I added some rules that actually eliminate nearly all of the risk: forbid catch and defer inside these try blocks]

[dabrahams (Dave Abrahams)]

@adamkemp

@dabrahams

Are you saying that await signals code that would otherwise (in a non- async world) block, and thus provide an opportunity for the current thread to do other useful work?

No, await doesn't add any new signaling behavior. We need to keep in mind that async/await is just new syntax for doing things you could already do before with callbacks. It's just easier now.

Sorry, I should have been clearer; by “signals” I meant “indicates to the reader”, not ”raises a signal”. Please take my word for it that I really do understand the mechanics of async/await.

The point of my explanation was to specifically compare async/await as proposed with your alternative suggestion that await should be optional.

I hope it's clear that I'm not making the suggestion that all awaits should be optional! I was only suggesting that there may be kinds of code where precisely marking all async calls might not be optimal.

So we're talking about code that does exactly the same thing but without the await keyword being there (i.e., example 1 versus 2 in my post). I'm trying to explain why leaving out the await keyword obscures what's actually happening, which would in some cases lead people to write code that goes slower because it just ends up sitting idle sometimes.

Yes, I totally understood that.

@dabrahams

Hmm, but—correct me if I'm wrong—I got the impression that async let evaluation is allowed to happen on an arbitrary thread. So even if these functions were all synchronous (in which case there would have been no await s to clue us in), wouldn't this refactoring still make sense?

If all of these functions were synchronous then the calling thread would be busy the whole time. You could maybe improve performance by dispatching some of it to another thread and then waiting for it to complete (i.e., by adding concurrency). Whether that makes sense depends very much on the nature of that synchronous work.

IIUC, that last sentence is true—to different degrees—regardless of whether the work is sync or async. IIUC, you're saying await serves as an important clue that async let can be used to improve performance. My point is that, if async let will dispatch to another thread when necessary/appropriate, then it could be used to improve performance even if the code were sync (given the right workloads), where we wouldn't have the clue, and nobody is saying not having the clue is the end of the world in that case. Maybe this is just a matter of degree, i.e. because task interleaving is so much more efficient than crossing threads, async let is just a lot more likely to pay off in those cases? That would be a fine argument for await.

The reality is that most of the time that an API is asynchronous it is because of something like I/O, where the CPU isn't necessarily doing much work at all. I would argue that in most cases if the CPU is busy then an API shouldn't be asynchronous (let the client decide how to schedule the work with knowledge of what else is going on in their process).

Hmm. But isn't async being used for both purposes in these proposals? I thought “task groups” would be the thing to use if you had a bunch of compute that could be effectively parallelized. Maybe that's a complete misunderstanding on my part…

The non-async-await alternative is a bunch of very complex (often nested) callbacks that make it hard to understand what is even happening when and how data flows from one place to another. The async-without-explicit-await alternative is simpler to write (compared to non-async-await), but obscures where you might be sitting idle waiting for results. The actual proposal sits in between: it simplifies the structure of the code and makes it clear where you're waiting for results.

Really, really, truly you don't have to convince me that there's value in async/await, especially compared to the alternatives. I'm trying to understand what the programming model is like (which you seem to have a really good sense of) and the importance of labeling async calls; that's all.

[dabrahams (Dave Abrahams)]

@Chris_Lattner3

@dabrahams

I think you're misinterpreting the most effective strategy, which is to mentally separate the kinds of code where it doesn't matter which things can throw, from the code where it matters, and only sweat the details in the latter cases (which is where I would apply try granularly).

While I agree this is possible, I haven't seen this widely accepted by the C++ community. Are there textbooks or prominent blog posts that explain this approach and advocate for it?

It's implicit in the accepted definition of “exception safe.” I mean, I know you're not gonna like this reference, but my 1998 Dagstuhl paper which is widely cited by almost every authoritative source on C++ exception-safety clearly states that the basic requirement for exception-safety is the preservation of invariants. This definition is echoed by Wikipedia among many other places. It follows directly that when no invariants are violated there's nothing to worry about.

In the C++ world we focused on getting people to understand the critical issue (invariant preservation), not discussing the thought patterns they would end up following once the critical issue was clear. This framing did allow people to think powerfully about error handling, though.

@Chris_Lattner3

I'm sorry but I really don't understand what you mean here.

I could explain if you like, but I'm also happy to drop that part of the discussion if you are..

@Chris_Lattner3

Currently try is not a statement. Creating a try statement (no matter which specific design points you choose) muddies its role.

That's totally fair criticism.

Making it syntactically similar to things in other widely used languages is a bad thing to me.

Eh, we already crossed that bridge when we called it try.

The advantage of try do { } is that it continues the role of the keyword as a marker, with the explanation that "it applies the try marker to the entire scope of the do ".

Yeah, I get that, but I honestly don't think it's going to register that way for users in practice. I think people will be talking about “using a try do statement,” and at that point the fact that it's not a statement is just a technicality. Also, moving it in front of do will still be a mental shift for those of us who thought of try as an identity operator that could only be part of an expression. Maybe that's the wrong model, but it's the one I had.

[Chris_Lattner3 (Chris Lattner)]

Ok, I think we're squarely in a "disagree about how EH in C++ works in practice" mode here. That's understandable, it is used in many different ways, and different communities embrace different approaches.

I still believe that the current design of marking statements with try is the right approach and think that the logical extension to solve "overly verbose try for certain kinds of code" is to introduce a try do {} stmt. However, I can understand and appreciate that you have a different viewpoint.

-Chris

[hisekaldma (Jonathan Hise Kaldma)]

@jrose

The thing is, a throwing function should have the option to make guarantees about postconditions in the failure case beyond just “no invariants have been broken”. In this case, the author of the function might want to restore the original contents of the Array on failure.

”All elements in the array are transformed, or no elements are transformed” certainly feels like an invariant to me (but I'll admit my understanding of the term isn’t very precise).

[hartbit (David Hart)]

@dabrahams

Yes; but think about why . Having to stop moving forward because a postcondition can't be satisfied (a.k.a. dealing with an error) is difficult because you can easily make temporary invariant breakage permanent. You have to make sure you undo the breakage or make it irrelevant. There are no other hard problems in error handling.

I'm not sure it's only a problem of invariant. I also want to think hard where each error can appear if that error ought to be left to bubble up to the caller, handled gracefully, or entirely dismissed: not only to preserve invariants, but also because not all errors make sense to be returned to the caller.

@dabrahams

Can you tell the story of one or two of these instances? Did you find a bug because of try that you wouldn't have equally found because of the throws on the function containing the try , or, if the containing function didn't throw, that you wouldn't have found because of a surrounding catch ?

For example, if I know that a function should only throw specific errors because I don't want to pollute the end-user with irrelevant errors, it's nice to go over each try expression and asking myself the question if errors thrown there ought to be caught or wrapped. And once I've looked at all try expressions, I know I'm not missing any errors.

[crusty]

@hisekaldma

”All elements in the array are transformed, or no elements are transformed” certainly feels like an invariant to me (but I'll admit my understanding of the term isn’t very precise).

Good on ya!

Now yer bonus challenge, as promised: why do ya think adding that guarantee to this pertickuler function might just be a dang fool notion? I think the word might be “anti-pattern” if yer some kinda hipster, no offense.

[hisekaldma (Jonathan Hise Kaldma)]

@crusty

Now yer bonus challenge, as promised: why do ya think adding that guarantee to this pertickuler function might just be a dang fool notion? I think the word might be “anti-pattern” if yer some kinda hipster, no offense.

You’d have to remember to maintain the invariant wherever you mutate this pertickuler array. I would have wrapped it in a type where all inits and mutating methods make sure to maintain the invariant, so I can’t forget it. (That said, as an implementation detail of such a type, I don’t really see a problem with the method. Yarrr.)

[dabrahams (Dave Abrahams)]

@hartbit

I'm not sure it's only a problem of invariant. I also want to think hard where each error can appear if that error ought to be left to bubble up to the caller, handled gracefully, or entirely dismissed: not only to preserve invariants, but also because not all errors make sense to be returned to the caller.
…
For example, if I know that a function should only throw specific errors because I don't want to pollute the end-user with irrelevant errors, it's nice to go over each try expression and asking myself the question if errors thrown there ought to be caught or wrapped. And once I've looked at all try expressions, I know I'm not missing any errors.

Ah, yes… well you should ask yourself why you “want to think hard.”

Put less facetiously, I don't consider the problem of error translation/wrapping to be hard, just tedious. Because it's mostly tedious, if you're going to take it seriously, you should build library components to encapsulate the common wrapping patterns and use them at all the API boundaries so you don't have to “think hard” about every single call. If there's consensus that controlling which kinds of errors get thrown is an important problem to solve, you probably also want language support in the form of “typed throws,” but I have the same reservations @John_McCall has expressed about that feature, having seen how it works out in other languages.

Cheers,
Dave

[crusty]

@hisekaldma

@crusty

Now yer bonus challenge, as promised: why do ya think adding that guarantee to this pertickuler function might just be a dang fool notion? I think the word might be “anti-pattern” if yer some kinda hipster, no offense.

You’d have to remember to maintain the invariant wherever you mutate this pertickuler array.

Sorry, brother, but I think yer barkin' up the wrong tree there. Try writin' the code Bunky described, and put in the parts he left out (grumble).

I would have wrapped it in a type where all inits and mutating methods make sure to maintain the invariant, so I can’t forget it. (That said, as an implementation detail of such a type, I don’t really see a problem with the method. Yarrr.)

Heh, wrappin's great but it don't change the pickle yer in—jus' makes it harder to smell.

[jrose (Jordan Rose)]

@hisekaldma

”All elements in the array are transformed, or no elements are transformed” certainly feels like an invariant to me (but I'll admit my understanding of the term isn’t very precise).

This seems to be what Dave was going for, but I find it vacuous. As you (both!) point out, that’s not an invariant of the Array type, and it’s trivially true upon entry to the function. If you want to call all postconditions “invariants” then maybe this is just a terminology thing, but I wouldn’t normally say “my error postconditions aren’t satisfied” is a “broken invariant”, even if you can phrase them as “either I’m in the state where I entered the function or I’ve changed things in a certain way”.

But the larger point is “functions with non-returned outputs or side effects (including invoking callbacks) have to make decisions about error postconditions beyond ‘safety’ and type invariants”. And even functions without additional outputs or side effects should decide whether a particular failure should be recovered from or passed through.

[dabrahams (Dave Abrahams)]

@jrose

This seems to be what Dave was going for, but I find it vacuous. As you (both!) point out, that’s not an invariant of the Array type,

Tying an invariant to a type is a very useful thing to do, but type invariants are not the only invariants. This one is an invariant being maintained on behalf of the caller.

and it’s trivially true upon entry to the function.

That's not an argument against it as an invariant. That's the way invariants work, generally: before the mutation starts, it had better be true and your code can assume it's true.

If you want to call all postconditions “invariants” then maybe this is just a terminology thing, but I wouldn’t normally say “my error postconditions aren’t satisfied” is a “broken invariant”, even if you can phrase them as “either I’m in the state where I entered the function or I’ve changed things in a certain way”.

Eh, yes, you can call the state after an error is thrown a “postcondition,” if you want. It's logically sound, but it doesn't scale well, because it seriously complicates what you have to say to document most functions, often in a way that isn't useful to the caller, and this becomes especially obvious once you understand why that particular guarantee on that that particular function is, as @crusty says, “a dang fool notion.” It also makes it really complicated to talk about the postconditions of constructing an instance. This isn't something that has a provably right-or-wrong answer, especially when you look at one small example at a time, but becomes clearer when you consider the aggregate effects over a large codebase. Let me just say right now, I don't have much interest in having an argument about that, but if you want to understand my story better, I'm happy to discuss it.

What I've found works (for many reasons I'll be happy to detail if you like) is to describe the postconditions of a function as what happens when no error occurs, and to document any extra guarantees you may get in the case of an error as an additional guarantee.

But the larger point is “functions with non-returned outputs or side effects (including invoking callbacks) have to make decisions about error postconditions beyond ‘safety’ and type invariants”. And even functions without additional outputs or side effects should decide whether a particular failure should be recovered from or passed through.

All I can say is, I know exactly what you're talking about here, but I think if you gave some specific attention to studying the interactions of error handling, performance, and API documentation, I think you'd see it my way. Admittedly, it took me several years of intense engagement with the problem to get there, but then again I may not be the quickest study.

[adamkemp]

@dabrahams

Sorry, I should have been clearer; by “signals” I meant “indicates to the reader”, not ”raises a signal”. Please take my word for it that I really do understand the mechanics of async / await .

Oh, I'm sorry. I did misunderstand what you meant. Let me try again...

@dabrahams

Are you saying that await signals code that would otherwise (in a non- async world) block, and thus provide an opportunity for the current thread to do other useful work?

await signals that you are potentially giving up your thread (and thus giving up the CPU) while waiting for some asynchronous response. Most likely the response you're waiting for won't be consuming any CPU itself while you're waiting so you are giving up control over the CPU to do nothing in the meantime. So the placement of that await relative to other code that you need to run is important information. Placing that await in the right place can make a significant difference in throughput. If the waiting were done implicitly then you wouldn't know to move it.

@dabrahams

IIUC, you're saying await serves as an important clue that async let can be used to improve performance

Yes!

@dabrahams

My point is that, if async let will dispatch to another thread when necessary/appropriate, then it could be used to improve performance even if the code were sync

That is not what async let is proposed to do. async let does not introduce concurrency.

func longRunningComputeTask() {
// ...
}

func buttonClick() async {
// ...
longRunningComputeTask()
// ...
}

That code will block the UI, obviously. longRunningComputeTask is synchronous. You can't use await when calling it because it's not async. You can't use async let for the same reason. The only way to make longRunningComputeTask run concurrently so that you don't block the UI thread would be to use a detached task using some API (not yet fully proposed, as far as I know) to run on another queue. Then you would await that detached task. So something like this (made up API):

func buttonClick() async {
// ...
await Task.runDetached(onQueue: queue) {
    longRunningComputeTask()
}
// ...
}

Or:

func buttonClick() async {
// ...
async let computeTask = Task.runDetached(onQueue: queue) {
    longRunningComputeTask()
}
// ...
await computeTask
}

There's no good way for the compiler to automatically turn that synchronous code into the asynchronous and concurrent code because it has no idea how long that code will take to run so it doesn't know whether it will be worth it. It also can't know whether it would be safe to do so.