typed throws

[Chris_Lattner]

Splitting this off into its own thread:

One related topic that isn’t discussed is type errors. Many third party libraries use a Result type with typed errors. Moving to an async / await model without also introducing typed errors into Swift would require giving up something that is highly valued by many Swift developers. Maybe Swift 5 is the right time to tackle typed errors as well. I would be happy to help with design and drafting a proposal but would need collaborators on the implementation side.

Typed throws is something we need to settle one way or the other, and I agree it would be nice to do that in the Swift 5 cycle.

For the purposes of this sub-discussion, I think there are three kinds of code to think about:
1) large scale API like Cocoa which evolve (adding significant functionality) over the course of many years and can’t break clients.
2) the public API of shared swiftpm packages, whose lifecycle may rise and fall - being obsoleted and replaced by better packages if they encounter a design problem.
3) internal APIs and applications, which are easy to change because the implementations and clients of the APIs are owned by the same people.

These each have different sorts of concerns, and we hope that something can start out as #3 but work its way up the stack gracefully.

Here is where I think things stand on it:
- There is consensus that untyped throws is the right thing for a large scale API like Cocoa. NSError is effectively proven here. Even if typed throws is introduced, Apple is unlikely to adopt it in their APIs for this reason.
- There is consensus that untyped throws is the right default for people to reach for for public package (#2).
- There is consensus that Java and other systems that encourage lists of throws error types lead to problematic APIs for a variety of reasons.
- There is disagreement about whether internal APIs (#3) should use it. It seems perfect to be able to write exhaustive catches in this situation, since everything in knowable. OTOH, this could encourage abuse of error handling in cases where you really should return an enum instead of using throws.
- Some people are concerned that introducing typed throws would cause people to reach for it instead of using untyped throws for public package APIs.
- Some people think that while it might be useful in some narrow cases, the utility isn’t high enough to justify making the language more complex (complexity that would intrude on the APIs of result types, futures, etc)

I’m sure there are other points in the discussion that I’m forgetting.

One thing that I’m personally very concerned about is in the systems programming domain. Systems code is sort of the classic example of code that is low-level enough and finely specified enough that there are lots of knowable things, including the failure modes. Beyond expressivity though, our current model involves boxing thrown values into an Error existential, something that forces an implicit memory allocation when the value is large. Unless this is fixed, I’m very concerned that we’ll end up with a situation where certain kinds of systems code (i.e., that which cares about real time guarantees) will not be able to use error handling at all.

JohnMC has some ideas on how to change code generation for ‘throws’ to avoid this problem, but I don’t understand his ideas enough to know if they are practical and likely to happen or not.

-Chris

···

On Aug 17, 2017, at 7:39 PM, Matthew Johnson <matthew@anandabits.com> wrote:

[John_McCall]

Splitting this off into its own thread:

One related topic that isn’t discussed is type errors. Many third party libraries use a Result type with typed errors. Moving to an async / await model without also introducing typed errors into Swift would require giving up something that is highly valued by many Swift developers. Maybe Swift 5 is the right time to tackle typed errors as well. I would be happy to help with design and drafting a proposal but would need collaborators on the implementation side.

Typed throws is something we need to settle one way or the other, and I agree it would be nice to do that in the Swift 5 cycle.

For the purposes of this sub-discussion, I think there are three kinds of code to think about:
1) large scale API like Cocoa which evolve (adding significant functionality) over the course of many years and can’t break clients.
2) the public API of shared swiftpm packages, whose lifecycle may rise and fall - being obsoleted and replaced by better packages if they encounter a design problem.
3) internal APIs and applications, which are easy to change because the implementations and clients of the APIs are owned by the same people.

These each have different sorts of concerns, and we hope that something can start out as #3 but work its way up the stack gracefully.

Here is where I think things stand on it:
- There is consensus that untyped throws is the right thing for a large scale API like Cocoa. NSError is effectively proven here. Even if typed throws is introduced, Apple is unlikely to adopt it in their APIs for this reason.
- There is consensus that untyped throws is the right default for people to reach for for public package (#2).
- There is consensus that Java and other systems that encourage lists of throws error types lead to problematic APIs for a variety of reasons.
- There is disagreement about whether internal APIs (#3) should use it. It seems perfect to be able to write exhaustive catches in this situation, since everything in knowable. OTOH, this could encourage abuse of error handling in cases where you really should return an enum instead of using throws.
- Some people are concerned that introducing typed throws would cause people to reach for it instead of using untyped throws for public package APIs.

Even for non-public code. The only practical merit of typed throws I have ever seen someone demonstrate is that it would let them use contextual lookup in a throw or catch. People always say "I'll be able to exhaustively switch over my errors", and then I ask them to show me where they want to do that, and they show me something that just logs the error, which of course does not require typed throws. Every. Single. Time.

Sometimes we then go on to have a conversation about wrapping errors in other error types, and that can be interesting, but now we're talking about adding a big, messy feature just to get "safety" guarantees for a fairly minor need.

Programmers often have an instinct to obsess over error taxonomies that is very rarely directed at solving any real problem; it is just self-imposed busy-work.

- Some people think that while it might be useful in some narrow cases, the utility isn’t high enough to justify making the language more complex (complexity that would intrude on the APIs of result types, futures, etc)

I’m sure there are other points in the discussion that I’m forgetting.

One thing that I’m personally very concerned about is in the systems programming domain. Systems code is sort of the classic example of code that is low-level enough and finely specified enough that there are lots of knowable things, including the failure modes.

Here we are using "systems" to mean "embedded systems and kernels". And frankly even a kernel is a large enough system that they don't want to exhaustively switch over failures; they just want the static guarantees that go along with a constrained error type.

Beyond expressivity though, our current model involves boxing thrown values into an Error existential, something that forces an implicit memory allocation when the value is large. Unless this is fixed, I’m very concerned that we’ll end up with a situation where certain kinds of systems code (i.e., that which cares about real time guarantees) will not be able to use error handling at all.

JohnMC has some ideas on how to change code generation for ‘throws’ to avoid this problem, but I don’t understand his ideas enough to know if they are practical and likely to happen or not.

Essentially, you give Error a tagged-pointer representation to allow payload-less errors on non-generic error types to be allocated globally, and then you can (1) tell people to not throw errors that require allocation if it's vital to avoid allocation (just like we would tell them today not to construct classes or indirect enum cases) and (2) allow a special global payload-less error to be substituted if error allocation fails.

Of course, we could also say that systems code is required to use a typed-throws feature that we add down the line for their purposes. Or just tell them to not use payloads. Or force them to constrain their error types to fit within some given size. (Note that obsessive error taxonomies tend to end up with a bunch of indirect enum cases anyway, because they get recursive, so the allocation problem is very real whatever we do.)

John.

···

On Aug 18, 2017, at 12:58 AM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:

On Aug 17, 2017, at 7:39 PM, Matthew Johnson <matthew@anandabits.com> wrote:

[anandabits (Matthew Johnson)]

Splitting this off into its own thread:

Thanks. I considered starting a thread but decided to ask about it first in case it was considered out of scope for Swift 5.

One related topic that isn’t discussed is type errors. Many third party libraries use a Result type with typed errors. Moving to an async / await model without also introducing typed errors into Swift would require giving up something that is highly valued by many Swift developers. Maybe Swift 5 is the right time to tackle typed errors as well. I would be happy to help with design and drafting a proposal but would need collaborators on the implementation side.

Typed throws is something we need to settle one way or the other, and I agree it would be nice to do that in the Swift 5 cycle.

For the purposes of this sub-discussion, I think there are three kinds of code to think about:
1) large scale API like Cocoa which evolve (adding significant functionality) over the course of many years and can’t break clients.
2) the public API of shared swiftpm packages, whose lifecycle may rise and fall - being obsoleted and replaced by better packages if they encounter a design problem.
3) internal APIs and applications, which are easy to change because the implementations and clients of the APIs are owned by the same people.

These each have different sorts of concerns, and we hope that something can start out as #3 but work its way up the stack gracefully.

Here is where I think things stand on it:
- There is consensus that untyped throws is the right thing for a large scale API like Cocoa. NSError is effectively proven here. Even if typed throws is introduced, Apple is unlikely to adopt it in their APIs for this reason.
- There is consensus that untyped throws is the right default for people to reach for for public package (#2).
- There is consensus that Java and other systems that encourage lists of throws error types lead to problematic APIs for a variety of reasons.
- There is disagreement about whether internal APIs (#3) should use it. It seems perfect to be able to write exhaustive catches in this situation, since everything in knowable. OTOH, this could encourage abuse of error handling in cases where you really should return an enum instead of using throws.
- Some people are concerned that introducing typed throws would cause people to reach for it instead of using untyped throws for public package APIs.
- Some people think that while it might be useful in some narrow cases, the utility isn’t high enough to justify making the language more complex (complexity that would intrude on the APIs of result types, futures, etc)

I’m sure there are other points in the discussion that I’m forgetting.

To be clear, I don't think anyone is arguing that we should remove the ability to simply mark a function `throws` with an implicit error type of `Error`! :) This is obviously crucial and would continue to be used in many cases. One important variation of this use case is that with typed errors it is also possible to refine `Error` and throw a slightly more specific existential. Obviously Apple will continue to use untyped errors in most or all APIs regardless of what we do and unannotated `throws` will continue to be the shortest syntax.

The topic of Java errors has already been beat to death on the list in the past. I find it unfortunate that the counter-examples (notably Rust) are not usually mentioned. I haven't worked in Java, but my study of this topic in the past has convinced me the Java design has some serious flaws that are completely avoidable.

I would like to reiterate the point I made in the question that spawned this thread: there are *many* Swift libraries for writing async code of various sorts which are *already* using typed errors via `Result<Value, ErrorType: Error>`. We don't have to speculate about how this feature might be used and what kind of benefits might be realized. We can have a discussion about what people are already doing and determine whether this model should be supported in the async / await world or not. What do you think of putting a call out to the broader Swift community to bring us concrete examples of how they are benefiting from using typed errors in async code.

If we decide it should be, typed errors for synchronous code naturally follows. If we decide it shouldn't we have to be prepared for some noise about the unfortunate tradeoff developers will face between using async / await or continuing (pun not intended) to use explicit continuations and typed errors. I suspect most people would move to async / await and live with untyped errors (we don't see a lot of synchronous functions that return Result) but I also suspect there would be a lot of grumbling for a while.

···

Sent from my iPad

On Aug 17, 2017, at 11:58 PM, Chris Lattner <clattner@nondot.org> wrote:

On Aug 17, 2017, at 7:39 PM, Matthew Johnson <matthew@anandabits.com> wrote:

One thing that I’m personally very concerned about is in the systems programming domain. Systems code is sort of the classic example of code that is low-level enough and finely specified enough that there are lots of knowable things, including the failure modes. Beyond expressivity though, our current model involves boxing thrown values into an Error existential, something that forces an implicit memory allocation when the value is large. Unless this is fixed, I’m very concerned that we’ll end up with a situation where certain kinds of systems code (i.e., that which cares about real time guarantees) will not be able to use error handling at all.

JohnMC has some ideas on how to change code generation for ‘throws’ to avoid this problem, but I don’t understand his ideas enough to know if they are practical and likely to happen or not.

-Chris

[Joe_Groff]

My view of this is the opposite of Matthew's—the canonical examples of things for which untyped errors are the "right thing" due to unbounded failure modes, such as file IO, IPC, network communication, etc. are almost all things you also want to be 'async', so I don't think async makes typed 'throws' any more urgent to consider. As we've discussed in person, I feel like there's a strong argument to be made that 'async' should always imply untyped 'throws'.

-Joe

···

On Aug 17, 2017, at 9:58 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:

On Aug 17, 2017, at 7:39 PM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:
One related topic that isn’t discussed is type errors. Many third party libraries use a Result type with typed errors. Moving to an async / await model without also introducing typed errors into Swift would require giving up something that is highly valued by many Swift developers. Maybe Swift 5 is the right time to tackle typed errors as well. I would be happy to help with design and drafting a proposal but would need collaborators on the implementation side.

Typed throws is something we need to settle one way or the other, and I agree it would be nice to do that in the Swift 5 cycle.

[dwaite (David Waite)]

Typed throws is something we need to settle one way or the other, and I agree it would be nice to do that in the Swift 5 cycle.

For the purposes of this sub-discussion, I think there are three kinds of code to think about:
1) large scale API like Cocoa which evolve (adding significant functionality) over the course of many years and can’t break clients.
2) the public API of shared swiftpm packages, whose lifecycle may rise and fall - being obsoleted and replaced by better packages if they encounter a design problem.
3) internal APIs and applications, which are easy to change because the implementations and clients of the APIs are owned by the same people.

These each have different sorts of concerns, and we hope that something can start out as #3 but work its way up the stack gracefully.

Here is where I think things stand on it:
- There is consensus that untyped throws is the right thing for a large scale API like Cocoa. NSError is effectively proven here. Even if typed throws is introduced, Apple is unlikely to adopt it in their APIs for this reason.
- There is consensus that untyped throws is the right default for people to reach for for public package (#2).
- There is consensus that Java and other systems that encourage lists of throws error types lead to problematic APIs for a variety of reasons.
- There is disagreement about whether internal APIs (#3) should use it. It seems perfect to be able to write exhaustive catches in this situation, since everything in knowable. OTOH, this could encourage abuse of error handling in cases where you really should return an enum instead of using throws.
- Some people are concerned that introducing typed throws would cause people to reach for it instead of using untyped throws for public package APIs.
- Some people think that while it might be useful in some narrow cases, the utility isn’t high enough to justify making the language more complex (complexity that would intrude on the APIs of result types, futures, etc)

I actually can’t think of anything other than details to add to this synopsis.

I personally think even for internal usage, errors remain very hard to neatly categorize into an ontology, and it is very difficult to have a single ontology for all usage. For internal usage, you at least have the ability to evolve your usage when you discover the problems, but the weight of typed throws as part of function/method types doesn’t seem worth a compiler-enforced exhaustive catch to me.

Attempting to categorize errors using a concrete error type can’t help but push the complexity of errors down via nested errors (e.g. “The JSON failed to parse - but was it due to data corruption, or transient network failure?”). Java’s multi-page nested exception traces are a side effect of attempting to corral exceptional cases using the type system - I think Swift has benefit of retroactive protocol conformance which would keep it from getting Java-level bad (see jgroff’s example), but I still don’t think people who want clear documentation or exhaustive catches want to have to dive through a hierarchy of nested errors to try to determine the root cause.

-DW

[anandabits (Matthew Johnson)]

Splitting this off into its own thread:

One related topic that isn’t discussed is type errors. Many third party libraries use a Result type with typed errors. Moving to an async / await model without also introducing typed errors into Swift would require giving up something that is highly valued by many Swift developers. Maybe Swift 5 is the right time to tackle typed errors as well. I would be happy to help with design and drafting a proposal but would need collaborators on the implementation side.

Typed throws is something we need to settle one way or the other, and I agree it would be nice to do that in the Swift 5 cycle.

For the purposes of this sub-discussion, I think there are three kinds of code to think about:
1) large scale API like Cocoa which evolve (adding significant functionality) over the course of many years and can’t break clients.
2) the public API of shared swiftpm packages, whose lifecycle may rise and fall - being obsoleted and replaced by better packages if they encounter a design problem.
3) internal APIs and applications, which are easy to change because the implementations and clients of the APIs are owned by the same people.

These each have different sorts of concerns, and we hope that something can start out as #3 but work its way up the stack gracefully.

Here is where I think things stand on it:
- There is consensus that untyped throws is the right thing for a large scale API like Cocoa. NSError is effectively proven here. Even if typed throws is introduced, Apple is unlikely to adopt it in their APIs for this reason.
- There is consensus that untyped throws is the right default for people to reach for for public package (#2).
- There is consensus that Java and other systems that encourage lists of throws error types lead to problematic APIs for a variety of reasons.
- There is disagreement about whether internal APIs (#3) should use it. It seems perfect to be able to write exhaustive catches in this situation, since everything in knowable. OTOH, this could encourage abuse of error handling in cases where you really should return an enum instead of using throws.
- Some people are concerned that introducing typed throws would cause people to reach for it instead of using untyped throws for public package APIs.

Even for non-public code. The only practical merit of typed throws I have ever seen someone demonstrate is that it would let them use contextual lookup in a throw or catch. People always say "I'll be able to exhaustively switch over my errors", and then I ask them to show me where they want to do that, and they show me something that just logs the error, which of course does not require typed throws. Every. Single. Time.

I agree that exhaustive switching over errors is something that people are extremely likely to actually want to do. I also think it's a bit of a red herring. The value of typed errors is *not* in exhaustive switching. It is in categorization and verified documentation.

Here is a concrete example that applies to almost every app. When you make a network request there are many things that could go wrong to which you may want to respond differently:
* There might be no network available. You might recover by updating the UI to indicate that and start monitoring for a reachability change.
* There might have been a server error that should eventually be resolved (500). You might update the UI and provide the user the ability to retry.
* There might have been an unrecoverable server error (404). You will update the UI.
* There might have been a low level parsing error (bad JSON, etc). Recovery is perhaps similar in nature to #2, but the problem is less likely to be resolved quickly so you may not provide a retry option. You might also want to do something to notify your dev team that the server is returning JSON that can't be parsed.
* There might have been a higher-level parsing error (converting JSON to model types). This might be treated the same as bad JSON. On the other hand, depending on the specifics of the app, you might take an alternate path that only parses the most essential model data in hopes that the problem was somewhere else and this parse will succeed.

All of this can obviously be accomplished with untyped errors. That said, using types to categorize errors would significantly improve the clarity of such code. More importantly, I believe that by categorizing errors in ways that are most relevant to a specific domain a library (perhaps internal to an app) can encourage developers to think carefully about how to respond.

Bad error handling is pervasive. The fact that everyone shows you code that just logs the error is a prime example of this. It should be considered a symptom of a problem, not an acceptable status quo to be maintained. We need all the tools at our disposal to encourage better thinking about and handling of errors. Most importantly, I think we need a middle ground between completely untyped errors and an exhaustive list of every possible error that might happen. I believe a well designed mechanism for categorizing errors in a compiler-verified way can do exactly this.

In many respects, there are similarities to this in the design of `NSError` which provides categorization via the error domain. This categorization is a bit more broad than I think is useful in many cases, but it is the best example I'm aware of.

The primary difference between error domains and the kind of categorization I am proposing is that error domains categorize based on the source of an error whereas I am proposing categorization driven by likely recovery strategies. Recovery is obviously application dependent, but I think the example above demonstrates that there are some useful generalizations that can be made (especially in an app-specific library), even if they don't apply everywhere.

Sometimes we then go on to have a conversation about wrapping errors in other error types, and that can be interesting, but now we're talking about adding a big, messy feature just to get "safety" guarantees for a fairly minor need.

I think you're right that wrapping errors is tightly related to an effective use of typed errors. You can do a reasonable job without language support (as has been discussed on the list in the past). On the other hand, if we're going to introduce typed errors we should do it in a way that *encourages* effective use of them. My opinion is that encouraging effect use means categorizing (wrapping) errors without requiring any additional syntax beyond the simple `try` used by untyped errors. In practice, this means we should not need to catch and rethrow an error if all we want to do is categorize it. Rust provides good prior art in this area.

Programmers often have an instinct to obsess over error taxonomies that is very rarely directed at solving any real problem; it is just self-imposed busy-work.

I agree that obsessing over intricate taxonomies is counter-productive and should be discouraged. On the other hand, I hope the example I provided above can help to focus the discussion on a practical use of types to categorize errors in a way that helps guide *thinking* and therefore improves error handling in practice.

···

Sent from my iPad
On Aug 18, 2017, at 1:27 AM, John McCall <rjmccall@apple.com> wrote:

On Aug 18, 2017, at 12:58 AM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:

On Aug 17, 2017, at 7:39 PM, Matthew Johnson <matthew@anandabits.com> wrote:

- Some people think that while it might be useful in some narrow cases, the utility isn’t high enough to justify making the language more complex (complexity that would intrude on the APIs of result types, futures, etc)

I’m sure there are other points in the discussion that I’m forgetting.

One thing that I’m personally very concerned about is in the systems programming domain. Systems code is sort of the classic example of code that is low-level enough and finely specified enough that there are lots of knowable things, including the failure modes.

Here we are using "systems" to mean "embedded systems and kernels". And frankly even a kernel is a large enough system that they don't want to exhaustively switch over failures; they just want the static guarantees that go along with a constrained error type.

Beyond expressivity though, our current model involves boxing thrown values into an Error existential, something that forces an implicit memory allocation when the value is large. Unless this is fixed, I’m very concerned that we’ll end up with a situation where certain kinds of systems code (i.e., that which cares about real time guarantees) will not be able to use error handling at all.

JohnMC has some ideas on how to change code generation for ‘throws’ to avoid this problem, but I don’t understand his ideas enough to know if they are practical and likely to happen or not.

Essentially, you give Error a tagged-pointer representation to allow payload-less errors on non-generic error types to be allocated globally, and then you can (1) tell people to not throw errors that require allocation if it's vital to avoid allocation (just like we would tell them today not to construct classes or indirect enum cases) and (2) allow a special global payload-less error to be substituted if error allocation fails.

Of course, we could also say that systems code is required to use a typed-throws feature that we add down the line for their purposes. Or just tell them to not use payloads. Or force them to constrain their error types to fit within some given size. (Note that obsessive error taxonomies tend to end up with a bunch of indirect enum cases anyway, because they get recursive, so the allocation problem is very real whatever we do.)

John.

[Tino]

The only practical merit of typed throws I have ever seen someone demonstrate is that it would let them use contextual lookup in a throw or catch. People always say "I'll be able to exhaustively switch over my errors", and then I ask them to show me where they want to do that, and they show me something that just logs the error, which of course does not require typed throws. Every. Single. Time.

I think the aspect of documentation shouldn't be underestimated:
People want to know what might go wrong — even if they have only a single reaction to all cases at their disposal.

Typed throws could also help to lessen the tight coupling to Objective-C:
Being forced to declare conformance to a protocol without requirements or even public methods feels very awkward to me.
In some way, throws are typed already, but there's just one type allowed, and it can't be changed.

Imho the big problem of the whole topic is the bad experience people had with typed throws in Java; I don't see a problem in adding typed throws and keeping Error as default type to be thrown if nothing else is specified.

If people want to build huge lists of possible errors… why not? As long as I'm still able to write my own throwing functions as today, I'm fine with that.

- Tino

[Charlie_Monroe1 (Charlie Monroe)]

Splitting this off into its own thread:

One related topic that isn’t discussed is type errors. Many third party libraries use a Result type with typed errors. Moving to an async / await model without also introducing typed errors into Swift would require giving up something that is highly valued by many Swift developers. Maybe Swift 5 is the right time to tackle typed errors as well. I would be happy to help with design and drafting a proposal but would need collaborators on the implementation side.

Typed throws is something we need to settle one way or the other, and I agree it would be nice to do that in the Swift 5 cycle.

For the purposes of this sub-discussion, I think there are three kinds of code to think about:
1) large scale API like Cocoa which evolve (adding significant functionality) over the course of many years and can’t break clients.
2) the public API of shared swiftpm packages, whose lifecycle may rise and fall - being obsoleted and replaced by better packages if they encounter a design problem.
3) internal APIs and applications, which are easy to change because the implementations and clients of the APIs are owned by the same people.

These each have different sorts of concerns, and we hope that something can start out as #3 but work its way up the stack gracefully.

Here is where I think things stand on it:
- There is consensus that untyped throws is the right thing for a large scale API like Cocoa. NSError is effectively proven here. Even if typed throws is introduced, Apple is unlikely to adopt it in their APIs for this reason.
- There is consensus that untyped throws is the right default for people to reach for for public package (#2).
- There is consensus that Java and other systems that encourage lists of throws error types lead to problematic APIs for a variety of reasons.
- There is disagreement about whether internal APIs (#3) should use it. It seems perfect to be able to write exhaustive catches in this situation, since everything in knowable. OTOH, this could encourage abuse of error handling in cases where you really should return an enum instead of using throws.
- Some people are concerned that introducing typed throws would cause people to reach for it instead of using untyped throws for public package APIs.

Even for non-public code. The only practical merit of typed throws I have ever seen someone demonstrate is that it would let them use contextual lookup in a throw or catch. People always say "I'll be able to exhaustively switch over my errors", and then I ask them to show me where they want to do that, and they show me something that just logs the error, which of course does not require typed throws. Every. Single. Time.

The issue I see here with non-typed errors is that relying on documentation is very error-prone. I'll give an example where I've used exhaustive error catching (but then again, I was generally the only one using exhaustive enum switches when we discussed those). I've made a simple library for reporting purchases to a server. The report needs to be signed using a certificate and there are some validations to be made.

This generally divides the errors into three logical areas - initialization (e.g. errors when loading the certificate, etc.), validation (when the document doesn't pass validation) and sending (network error, error response from the server, etc.).

Instead of using a large error enum, I've split this into three enums. At this point, especially for a newcommer to the code, he may not realize which method can throw which of these error enums.

I've found that the app can take advantage of knowing what's wrong. For example, if some required information is missing e.g. Validation.subjectNameMissing is thrown. In such case the application can inform the user that name is missing and it can offer to open UI to enter this information (in the case of my app, the UI for sending is in the document view, while the mentioned "subject name" information is in Preferences).

This way I exhaustively switch over the error enums, suggesting to the user solution of the particular problem without dumbing down to a message "Oops, something went wrong, but I have no idea what because this kind of error is not handled.".

Alternatives I've considered:

- wrapping all the errors into an "Error" enum which would switch over type of the error, which is not a great solution as in some cases you only throw one type of error
- I could throw some error that only contains verbose description of the problem (generally a String), but I don't feel it's the library's job to stringify the error as it can be used for a command-line tools as well

Perhpas I'm missing something, but dealing with UI and presenting an adequate error dialog to the user can be a challenge in current state of things given that currently, in the error catching, you fallback to the basic Error type and generally don't have a lot of options but to display something like "unknown error" - which is terrible for the user.

···

On Aug 18, 2017, at 8:27 AM, John McCall via swift-evolution <swift-evolution@swift.org> wrote:

On Aug 18, 2017, at 12:58 AM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:

On Aug 17, 2017, at 7:39 PM, Matthew Johnson <matthew@anandabits.com> wrote:

Sometimes we then go on to have a conversation about wrapping errors in other error types, and that can be interesting, but now we're talking about adding a big, messy feature just to get "safety" guarantees for a fairly minor need.

Programmers often have an instinct to obsess over error taxonomies that is very rarely directed at solving any real problem; it is just self-imposed busy-work.

- Some people think that while it might be useful in some narrow cases, the utility isn’t high enough to justify making the language more complex (complexity that would intrude on the APIs of result types, futures, etc)

I’m sure there are other points in the discussion that I’m forgetting.

One thing that I’m personally very concerned about is in the systems programming domain. Systems code is sort of the classic example of code that is low-level enough and finely specified enough that there are lots of knowable things, including the failure modes.

Here we are using "systems" to mean "embedded systems and kernels". And frankly even a kernel is a large enough system that they don't want to exhaustively switch over failures; they just want the static guarantees that go along with a constrained error type.

Beyond expressivity though, our current model involves boxing thrown values into an Error existential, something that forces an implicit memory allocation when the value is large. Unless this is fixed, I’m very concerned that we’ll end up with a situation where certain kinds of systems code (i.e., that which cares about real time guarantees) will not be able to use error handling at all.

JohnMC has some ideas on how to change code generation for ‘throws’ to avoid this problem, but I don’t understand his ideas enough to know if they are practical and likely to happen or not.

Essentially, you give Error a tagged-pointer representation to allow payload-less errors on non-generic error types to be allocated globally, and then you can (1) tell people to not throw errors that require allocation if it's vital to avoid allocation (just like we would tell them today not to construct classes or indirect enum cases) and (2) allow a special global payload-less error to be substituted if error allocation fails.

Of course, we could also say that systems code is required to use a typed-throws feature that we add down the line for their purposes. Or just tell them to not use payloads. Or force them to constrain their error types to fit within some given size. (Note that obsessive error taxonomies tend to end up with a bunch of indirect enum cases anyway, because they get recursive, so the allocation problem is very real whatever we do.)

John.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[Joe_Groff]

Alternatively, with some LLVM work, we could have the thrower leave the error value on the stack when propagating an error, and make it the catcher's responsibility to consume the error value and pop the stack. We could make not only errors but existential returns in general more efficient and "systems code"-worthy with a technique like that.

-Joe

···

On Aug 17, 2017, at 11:27 PM, John McCall via swift-evolution <swift-evolution@swift.org> wrote:

Essentially, you give Error a tagged-pointer representation to allow payload-less errors on non-generic error types to be allocated globally, and then you can (1) tell people to not throw errors that require allocation if it's vital to avoid allocation (just like we would tell them today not to construct classes or indirect enum cases) and (2) allow a special global payload-less error to be substituted if error allocation fails.

Of course, we could also say that systems code is required to use a typed-throws feature that we add down the line for their purposes. Or just tell them to not use payloads. Or force them to constrain their error types to fit within some given size. (Note that obsessive error taxonomies tend to end up with a bunch of indirect enum cases anyway, because they get recursive, so the allocation problem is very real whatever we do.)

[Michael_Ilseman]

Wouldn’t switching from `async` to `async throws` be both a source and ABI break for libraries? If so, there is a library evolution argument to `async` also encompassing throws as a reasonable default. It's likely that the non-throwing-ness of many `async` operations is an artifact of the initial implementation rather than deliberate design. Using the more verbose `async(non-throwing)` (or even some other keyword `generator|yielding|…`) makes it a deliberate API decision rather than an accidental omission.

···

On Aug 18, 2017, at 10:05 AM, Joe Groff via swift-evolution <swift-evolution@swift.org> wrote:

On Aug 17, 2017, at 9:58 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Aug 17, 2017, at 7:39 PM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:
One related topic that isn’t discussed is type errors. Many third party libraries use a Result type with typed errors. Moving to an async / await model without also introducing typed errors into Swift would require giving up something that is highly valued by many Swift developers. Maybe Swift 5 is the right time to tackle typed errors as well. I would be happy to help with design and drafting a proposal but would need collaborators on the implementation side.

Typed throws is something we need to settle one way or the other, and I agree it would be nice to do that in the Swift 5 cycle.

My view of this is the opposite of Matthew's—the canonical examples of things for which untyped errors are the "right thing" due to unbounded failure modes, such as file IO, IPC, network communication, etc. are almost all things you also want to be 'async', so I don't think async makes typed 'throws' any more urgent to consider. As we've discussed in person, I feel like there's a strong argument to be made that 'async' should always imply untyped 'throws'.

-Joe
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[Christopher_Kornher1 (Christopher Kornher)]

I have found this “feature" has been painful to use. I have been forced to wrap hetrogenous errors in a custom error type just to satisfy these ill-conceived APIs. i have never found any benefit to ```ErrorType: Error``` generic parameter. … Any. Ever.

···

On Aug 18, 2017, at 7:38 AM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

I would like to reiterate the point I made in the question that spawned this thread: there are *many* Swift libraries for writing async code of various sorts which are *already* using typed errors via `Result<Value, ErrorType: Error>`. We don't have to speculate about how this feature might be used and what kind of benefits might be realized. We can have a discussion about what people are already doing and determine whether this model should be supported in the async / await world or not. What do you think of putting a call out to the broader Swift community to bring us concrete examples of how they are benefiting from using typed errors in async code.

[anandabits (Matthew Johnson)]

One related topic that isn’t discussed is type errors. Many third party libraries use a Result type with typed errors. Moving to an async / await model without also introducing typed errors into Swift would require giving up something that is highly valued by many Swift developers. Maybe Swift 5 is the right time to tackle typed errors as well. I would be happy to help with design and drafting a proposal but would need collaborators on the implementation side.

Typed throws is something we need to settle one way or the other, and I agree it would be nice to do that in the Swift 5 cycle.

My view of this is the opposite of Matthew's—the canonical examples of things for which untyped errors are the "right thing" due to unbounded failure modes, such as file IO, IPC, network communication, etc. are almost all things you also want to be 'async', so I don't think async makes typed 'throws' any more urgent to consider. As we've discussed in person, I feel like there's a strong argument to be made that 'async' should always imply untyped 'throws’.

That’s interesting. Have you looked at how people are using `Result<T, E: Error>` in current third party libraries that do async stuff?

Even when failure modes are unbounded it can still be very useful to categorize the kinds of errors that might happen and encourage thoughtfulness about recovery in a way that an untyped `Error` does not. Maybe types are not the best way to solve that problem but I haven’t seen any alternative that looks promising.

···

On Aug 18, 2017, at 12:05 PM, Joe Groff <jgroff@apple.com> wrote:

On Aug 17, 2017, at 9:58 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Aug 17, 2017, at 7:39 PM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

-Joe

[johannesweiss]

Hi John,

tl;dr I think throws should be optionally typed. Ie. `func someSyscall(...) throws(POSIXError) -> Int` should be just as legal as `func doSomeFancyCocoaOperation() throws -> Void`.

[...]

Here is where I think things stand on it:
- There is consensus that untyped throws is the right thing for a large scale API like Cocoa. NSError is effectively proven here. Even if typed throws is introduced, Apple is unlikely to adopt it in their APIs for this reason.
- There is consensus that untyped throws is the right default for people to reach for for public package (#2).
- There is consensus that Java and other systems that encourage lists of throws error types lead to problematic APIs for a variety of reasons.
- There is disagreement about whether internal APIs (#3) should use it. It seems perfect to be able to write exhaustive catches in this situation, since everything in knowable. OTOH, this could encourage abuse of error handling in cases where you really should return an enum instead of using throws.
- Some people are concerned that introducing typed throws would cause people to reach for it instead of using untyped throws for public package APIs.

Even for non-public code. The only practical merit of typed throws I have ever seen someone demonstrate is that it would let them use contextual lookup in a throw or catch. People always say "I'll be able to exhaustively switch over my errors", and then I ask them to show me where they want to do that, and they show me something that just logs the error, which of course does not require typed throws. Every. Single. Time.

I just grep'ed through some code and what I always see is this:

        } catch let e {
            fatalError("unexcepted error caught: \(e)")
        }

most of the code is fairly low-level and the only error thrown in there is a POSIXError which basically holds the errno value and a string describing the failed syscall. I find this a very common pattern. Also the IMHO quite widespread use of Result<E, T> types shows that there are valid uses of a typed error mechanism. On the other hand requiring a high-level Cocoa method to declare all errors doesn't seem right either. So why not optionally typing the throws?

[...]

-- Johannes

[Chris_Lattner]

Even for non-public code. The only practical merit of typed throws I have ever seen someone demonstrate is that it would let them use contextual lookup in a throw or catch. People always say "I'll be able to exhaustively switch over my errors", and then I ask them to show me where they want to do that, and they show me something that just logs the error, which of course does not require typed throws. Every. Single. Time.

I won’t make any claims about what people say to you, but there are other benefits of course. Even if you don’t *exhaustively* catch your errors, it is still useful to get code completion for the error cases that you do intend to handle (covering the rest with a "catch {“).

Throwing a resilient enum is actually a rather defensible way to handle this, since clients will be expected to have a default, and the implementation is allowed to add new cases.

Programmers often have an instinct to obsess over error taxonomies that is very rarely directed at solving any real problem; it is just self-imposed busy-work.

Indeed, no denying that.

One thing that I’m personally very concerned about is in the systems programming domain. Systems code is sort of the classic example of code that is low-level enough and finely specified enough that there are lots of knowable things, including the failure modes.

Here we are using "systems" to mean "embedded systems and kernels". And frankly even a kernel is a large enough system that they don't want to exhaustively switch over failures; they just want the static guarantees that go along with a constrained error type.

I think you’re over focusing on exhaustively handling cases. If you use errno as one poor proxy for what kernels care about, it is totally reasonable to handle a few of the errors produced by a call (EAGAIN, EINTR are two common examples), but not all of them. It is useful to get code completion on enum members, and it is also useful to get an indication from a compiler that something can/will never be thrown. You give up both with full type erasure.

JohnMC has some ideas on how to change code generation for ‘throws’ to avoid this problem, but I don’t understand his ideas enough to know if they are practical and likely to happen or not.

Essentially, you give Error a tagged-pointer representation to allow payload-less errors on non-generic error types to be allocated globally, and then you can (1) tell people to not throw errors that require allocation if it's vital to avoid allocation (just like we would tell them today not to construct classes or indirect enum cases) and (2) allow a special global payload-less error to be substituted if error allocation fails.

Of course, we could also say that systems code is required to use a typed-throws feature that we add down the line for their purposes. Or just tell them to not use payloads. Or force them to constrain their error types to fit within some given size. (Note that obsessive error taxonomies tend to end up with a bunch of indirect enum cases anyway, because they get recursive, so the allocation problem is very real whatever we do.)

The issue isn’t allocation that can fail, it is the fact that allocation runs in nondeterministic time even when it succeeds. This is highly problematic for realtime systems.

Your answer basically says to me that you’re ok with offering a vastly limited programming model to people who care about realtime guarantees, and that gives up performance needlessly even in use-cases that probably care about low level perf. I agree that it would not be the end of the world (that would be for systems programmers to have to keep using C :-), but it certainly falls short of the best possible system that we could build.

-Chris

···

On Aug 17, 2017, at 11:27 PM, John McCall <rjmccall@apple.com> wrote:

[Brent_Royal-Gordon]

Let me take this in a slightly different direction.

I largely agree with you that most code shouldn't limit the types of errors it can throw. But I still want still want typed throws because I want to address a completely different problem: generics. Other than the useful but limited abilities of `rethrows`, there is currently no way to sensibly abstract over the error behavior of a piece of code. In particular, there's no way for a single protocol to support both throwing and non-throwing conformances.

One very straightforward and sensible way to handle this would be to support typed `throws` and make `Never` a subtype of `Error`; then you could handle error behavior abstraction using generic parameters, associated types, `where` clauses, and all those nice things we've already invented and taught people how to use. The other alternative is to invent several generics features from whole cloth purely to handle `throws`.

If we want to support errors in generics, designing typed `throws` and its interactions with the generics system seems a lot easier than designing a novel mechanism. Perhaps there are complications I'm not aware of—the thought of figuring out how generics are implemented fills me with dread—but naïvely, I would guess that the typed-throws-based solution will be easier to implement, too. And of course typed `throws` also helps the people who think they want to, you know, specify the type that a function throws. And it might help with the systems programming challenges we've talked about in this thread. So we can solve several different problems with a single feature, even if we think some of those problems are kind of niche.

I don't think any of the individual reasons to support typed `throws` are home runs, but I think this thread has produced some pretty solid ground balls and a couple of well-timed bunts, and that might be enough to get it across home plate.

···

On Aug 17, 2017, at 11:27 PM, John McCall via swift-evolution <swift-evolution@swift.org> wrote:

The only practical merit of typed throws I have ever seen someone demonstrate is that it would let them use contextual lookup in a throw or catch.

--
Brent Royal-Gordon
Architechies

[Karl]

I’m really happy with the way this discussion has changed since last time. This is pretty much the same point I was advocating back then.

When people say they want typed-throws, I don’t think they’re talking about optimisation. Errors are really for exception cases - if you expect it to be thrown in normal program flow, it’s really a return value and you should be using an enum directly. Besides, if the compiler has enough knowledge about which errors are being thrown, there’s no reason it can’t eliminate or further optimise that boxing behind-the-scenes (at least within the same module). So I remain unconvinced that we absolutely need typed-throws for Error optimisation.

The key is that compiler knowledge. Within a single module, we should be able to trace the ‘throw X’ statements and annotate each function/closure with a list of potentially-thrown errors. We already do checking of throwing scopes to make sure every throw is handled; conceptually we could also produce a list of error-sources.

Once the compiler has this information, we could provide a lot of benefits for free:

- Within a single module, we could eliminate/optimise boxing and do exhaustive catching with no code changes.
- For lazy library authors (which we all, occasionally, are) we could automatically document the publicly-visible errors thrown by their public functions (with an opt-out switch).
- For diligent library authors, we could check their more descriptive error documentation and warn them if they missed any cases or invented ones that are never thrown.
- That documentation could be parsed by the compiler to provide autocomplete across libraries. We would extend our markdown documentation syntax a little to accommodate.

And that, I think, really encapsulates the core of what people want from a full strongly-typed throws. Better documentation about thrown errors, which the compiler by default ensures is reliable, and autocomplete when catching.

- Karl

P.S: We would still box Errors across modules, for resilience. I’m not sure that it’s worth allowing breaking that, because it locks your implementation in so much you might as well make the function @inlineable.

···

On 18. Aug 2017, at 08:27, John McCall via swift-evolution <swift-evolution@swift.org> wrote:

On Aug 18, 2017, at 12:58 AM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:
Splitting this off into its own thread:

On Aug 17, 2017, at 7:39 PM, Matthew Johnson <matthew@anandabits.com> wrote:
One related topic that isn’t discussed is type errors. Many third party libraries use a Result type with typed errors. Moving to an async / await model without also introducing typed errors into Swift would require giving up something that is highly valued by many Swift developers. Maybe Swift 5 is the right time to tackle typed errors as well. I would be happy to help with design and drafting a proposal but would need collaborators on the implementation side.

Typed throws is something we need to settle one way or the other, and I agree it would be nice to do that in the Swift 5 cycle.

For the purposes of this sub-discussion, I think there are three kinds of code to think about:
1) large scale API like Cocoa which evolve (adding significant functionality) over the course of many years and can’t break clients.
2) the public API of shared swiftpm packages, whose lifecycle may rise and fall - being obsoleted and replaced by better packages if they encounter a design problem.
3) internal APIs and applications, which are easy to change because the implementations and clients of the APIs are owned by the same people.

These each have different sorts of concerns, and we hope that something can start out as #3 but work its way up the stack gracefully.

Here is where I think things stand on it:
- There is consensus that untyped throws is the right thing for a large scale API like Cocoa. NSError is effectively proven here. Even if typed throws is introduced, Apple is unlikely to adopt it in their APIs for this reason.
- There is consensus that untyped throws is the right default for people to reach for for public package (#2).
- There is consensus that Java and other systems that encourage lists of throws error types lead to problematic APIs for a variety of reasons.
- There is disagreement about whether internal APIs (#3) should use it. It seems perfect to be able to write exhaustive catches in this situation, since everything in knowable. OTOH, this could encourage abuse of error handling in cases where you really should return an enum instead of using throws.
- Some people are concerned that introducing typed throws would cause people to reach for it instead of using untyped throws for public package APIs.

Even for non-public code. The only practical merit of typed throws I have ever seen someone demonstrate is that it would let them use contextual lookup in a throw or catch. People always say "I'll be able to exhaustively switch over my errors", and then I ask them to show me where they want to do that, and they show me something that just logs the error, which of course does not require typed throws. Every. Single. Time.

Sometimes we then go on to have a conversation about wrapping errors in other error types, and that can be interesting, but now we're talking about adding a big, messy feature just to get "safety" guarantees for a fairly minor need.

Programmers often have an instinct to obsess over error taxonomies that is very rarely directed at solving any real problem; it is just self-imposed busy-work.

- Some people think that while it might be useful in some narrow cases, the utility isn’t high enough to justify making the language more complex (complexity that would intrude on the APIs of result types, futures, etc)

I’m sure there are other points in the discussion that I’m forgetting.

One thing that I’m personally very concerned about is in the systems programming domain. Systems code is sort of the classic example of code that is low-level enough and finely specified enough that there are lots of knowable things, including the failure modes.

Here we are using "systems" to mean "embedded systems and kernels". And frankly even a kernel is a large enough system that they don't want to exhaustively switch over failures; they just want the static guarantees that go along with a constrained error type.

Beyond expressivity though, our current model involves boxing thrown values into an Error existential, something that forces an implicit memory allocation when the value is large. Unless this is fixed, I’m very concerned that we’ll end up with a situation where certain kinds of systems code (i.e., that which cares about real time guarantees) will not be able to use error handling at all.

JohnMC has some ideas on how to change code generation for ‘throws’ to avoid this problem, but I don’t understand his ideas enough to know if they are practical and likely to happen or not.

Essentially, you give Error a tagged-pointer representation to allow payload-less errors on non-generic error types to be allocated globally, and then you can (1) tell people to not throw errors that require allocation if it's vital to avoid allocation (just like we would tell them today not to construct classes or indirect enum cases) and (2) allow a special global payload-less error to be substituted if error allocation fails.

Of course, we could also say that systems code is required to use a typed-throws feature that we add down the line for their purposes. Or just tell them to not use payloads. Or force them to constrain their error types to fit within some given size. (Note that obsessive error taxonomies tend to end up with a bunch of indirect enum cases anyway, because they get recursive, so the allocation problem is very real whatever we do.)

John.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

[dabrahams (Dave Abrahams)]

+1; I couldn't have said this any better.

The problem is not merely that by implementing this we'd be expending
effort and adding language complexity; it's also that programmers love
to classify things and those working with statically-typed languages
have a natural inclination to think “more static type information is
better,” so they *will* use this feature whether it's good for code or
not.

···

on Thu Aug 17 2017, John McCall <swift-evolution@swift.org> wrote:

On Aug 18, 2017, at 12:58 AM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:
Splitting this off into its own thread:

On Aug 17, 2017, at 7:39 PM, Matthew Johnson <matthew@anandabits.com> wrote:
One related topic that isn’t discussed is type errors. Many third

party libraries use a Result type with typed errors. Moving to an
async / await model without also introducing typed errors into
Swift would require giving up something that is highly valued by
many Swift developers. Maybe Swift 5 is the right time to tackle
typed errors as well. I would be happy to help with design and
drafting a proposal but would need collaborators on the
implementation side.

Typed throws is something we need to settle one way or the other,
and I agree it would be nice to do that in the Swift 5 cycle.

For the purposes of this sub-discussion, I think there are three kinds of code to think about:
1) large scale API like Cocoa which evolve (adding significant
functionality) over the course of many years and can’t break
clients.
2) the public API of shared swiftpm packages, whose lifecycle may
rise and fall - being obsoleted and replaced by better packages if
they encounter a design problem.
3) internal APIs and applications, which are easy to change because
the implementations and clients of the APIs are owned by the same
people.

These each have different sorts of concerns, and we hope that
something can start out as #3 but work its way up the stack
gracefully.

Here is where I think things stand on it:
- There is consensus that untyped throws is the right thing for a
large scale API like Cocoa. NSError is effectively proven here.
Even if typed throws is introduced, Apple is unlikely to adopt it in
their APIs for this reason.
- There is consensus that untyped throws is the right default for people to reach for for public package (#2).
- There is consensus that Java and other systems that encourage
lists of throws error types lead to problematic APIs for a variety
of reasons.
- There is disagreement about whether internal APIs (#3) should use
it. It seems perfect to be able to write exhaustive catches in this
situation, since everything in knowable. OTOH, this could encourage
abuse of error handling in cases where you really should return an
enum instead of using throws.
- Some people are concerned that introducing typed throws would
cause people to reach for it instead of using untyped throws for
public package APIs.

Even for non-public code. The only practical merit of typed throws I
have ever seen someone demonstrate is that it would let them use
contextual lookup in a throw or catch. People always say "I'll be
able to exhaustively switch over my errors", and then I ask them to
show me where they want to do that, and they show me something that
just logs the error, which of course does not require typed throws.
Every. Single. Time.

Sometimes we then go on to have a conversation about wrapping errors
in other error types, and that can be interesting, but now we're
talking about adding a big, messy feature just to get "safety"
guarantees for a fairly minor need.

Programmers often have an instinct to obsess over error taxonomies
that is very rarely directed at solving any real problem; it is just
self-imposed busy-work.

--
-Dave

[John_McCall]

The only practical merit of typed throws I have ever seen someone demonstrate is that it would let them use contextual lookup in a throw or catch. People always say "I'll be able to exhaustively switch over my errors", and then I ask them to show me where they want to do that, and they show me something that just logs the error, which of course does not require typed throws. Every. Single. Time.

I think the aspect of documentation shouldn't be underestimated:
People want to know what might go wrong — even if they have only a single reaction to all cases at their disposal.

There's no reason we couldn't do some tooling work to expose emergent information about what kinds of errors are thrown by the current implementation of a function, and maybe even check that against the current documentation. Certainly, it should be possible to document particularly interesting errors that a function might throw. I'm just challenging the idea that this should be reflected and enforced in the type system.

Typed throws could also help to lessen the tight coupling to Objective-C:
Being forced to declare conformance to a protocol without requirements or even public methods feels very awkward to me.

Error is not about Objective-C interop; we could make the feature work without a protocol, and in fact the protocol alone isn't sufficient for what we try to do with it. The protocol's value is mostly communicative, a way of making it obvious that a type is meant to be thrown, as well as to verify that you aren't accidentally throwing something nonsensical.

It also gives us a hook to add defaulted requirements in a later release.

In some way, throws are typed already, but there's just one type allowed, and it can't be changed.

Imho the big problem of the whole topic is the bad experience people had with typed throws in Java; I don't see a problem in adding typed throws and keeping Error as default type to be thrown if nothing else is specified.

The problems people have with typed throws in Java aren't ultimately because of some specific failure of Java (although there are certainly ways in which Java's design could be better in my opinion). They pretty much all arise directly from conceptual problems with the idea of "restricted" sets of errors.

If people want to build huge lists of possible errors… why not? As long as I'm still able to write my own throwing functions as today, I'm fine with that.

Language features have to meet a higher bar than "why not?".

John.

···

On Aug 18, 2017, at 3:24 AM, Tino Heth <2th@gmx.de> wrote:

[John_McCall]

Splitting this off into its own thread:

One related topic that isn’t discussed is type errors. Many third party libraries use a Result type with typed errors. Moving to an async / await model without also introducing typed errors into Swift would require giving up something that is highly valued by many Swift developers. Maybe Swift 5 is the right time to tackle typed errors as well. I would be happy to help with design and drafting a proposal but would need collaborators on the implementation side.

Typed throws is something we need to settle one way or the other, and I agree it would be nice to do that in the Swift 5 cycle.

For the purposes of this sub-discussion, I think there are three kinds of code to think about:
1) large scale API like Cocoa which evolve (adding significant functionality) over the course of many years and can’t break clients.
2) the public API of shared swiftpm packages, whose lifecycle may rise and fall - being obsoleted and replaced by better packages if they encounter a design problem.
3) internal APIs and applications, which are easy to change because the implementations and clients of the APIs are owned by the same people.

These each have different sorts of concerns, and we hope that something can start out as #3 but work its way up the stack gracefully.

Here is where I think things stand on it:
- There is consensus that untyped throws is the right thing for a large scale API like Cocoa. NSError is effectively proven here. Even if typed throws is introduced, Apple is unlikely to adopt it in their APIs for this reason.
- There is consensus that untyped throws is the right default for people to reach for for public package (#2).
- There is consensus that Java and other systems that encourage lists of throws error types lead to problematic APIs for a variety of reasons.
- There is disagreement about whether internal APIs (#3) should use it. It seems perfect to be able to write exhaustive catches in this situation, since everything in knowable. OTOH, this could encourage abuse of error handling in cases where you really should return an enum instead of using throws.
- Some people are concerned that introducing typed throws would cause people to reach for it instead of using untyped throws for public package APIs.

Even for non-public code. The only practical merit of typed throws I have ever seen someone demonstrate is that it would let them use contextual lookup in a throw or catch. People always say "I'll be able to exhaustively switch over my errors", and then I ask them to show me where they want to do that, and they show me something that just logs the error, which of course does not require typed throws. Every. Single. Time.

The issue I see here with non-typed errors is that relying on documentation is very error-prone. I'll give an example where I've used exhaustive error catching (but then again, I was generally the only one using exhaustive enum switches when we discussed those). I've made a simple library for reporting purchases to a server. The report needs to be signed using a certificate and there are some validations to be made.

This generally divides the errors into three logical areas - initialization (e.g. errors when loading the certificate, etc.), validation (when the document doesn't pass validation) and sending (network error, error response from the server, etc.).

Instead of using a large error enum, I've split this into three enums. At this point, especially for a newcommer to the code, he may not realize which method can throw which of these error enums.

I've found that the app can take advantage of knowing what's wrong. For example, if some required information is missing e.g. Validation.subjectNameMissing is thrown. In such case the application can inform the user that name is missing and it can offer to open UI to enter this information (in the case of my app, the UI for sending is in the document view, while the mentioned "subject name" information is in Preferences).

This way I exhaustively switch over the error enums, suggesting to the user solution of the particular problem without dumbing down to a message "Oops, something went wrong, but I have no idea what because this kind of error is not handled.".

Surely you must have a message like that. You're transmitting over a network, so all sorts of things can go wrong that you're not going to explain in detail to the user or have specific recoveries for. I would guess that have a generic handler for errors, and it has carefully-considered responses for specific failures (validation errors, maybe initialization errors) but a default response for others. Maybe you've put effort into handling more errors intelligently, trying to let fewer and fewer things end up with the default response — that's great, but it must still be there.

That's one of the keys to my argument here: practically speaking, from the perspective of any specific bit of code, there will always be a default response, because errors naturally quickly tend towards complexity, far more complexity than any client can exhaustively handle. Typed throws just means that error types will all have catch-all cases like MyError.other(Error), which mostly seems counter-productive to me.

I totally agree that we could do a lot more for documentation. I might be able to be talked into a language design that expressly acknowledges that it's just providing documentation and usability hints and doesn't normally let you avoid the need for default cases. But I don't think there's a compelling need for such a feature to land in Swift 5.

Alternatives I've considered:

- wrapping all the errors into an "Error" enum which would switch over type of the error, which is not a great solution as in some cases you only throw one type of error

That's interesting. In what cases do you only throw one type of error? Does it not have a catch-all case?

- I could throw some error that only contains verbose description of the problem (generally a String), but I don't feel it's the library's job to stringify the error as it can be used for a command-line tools as well

Absolutely. Using enums is a much better way of structuring errors than using a string.

Perhpas I'm missing something, but dealing with UI and presenting an adequate error dialog to the user can be a challenge in current state of things given that currently, in the error catching, you fallback to the basic Error type and generally don't have a lot of options but to display something like "unknown error" - which is terrible for the user.

Again, it comes down to whether that's ever completely avoidable, and I don't think it is. Good error-handling means doing your best to handle common errors well.

John.

···

On Aug 18, 2017, at 3:28 AM, Charlie Monroe <charlie@charliemonroe.net> wrote:

On Aug 18, 2017, at 8:27 AM, John McCall via swift-evolution <swift-evolution@swift.org> wrote:

On Aug 18, 2017, at 12:58 AM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:

On Aug 17, 2017, at 7:39 PM, Matthew Johnson <matthew@anandabits.com> wrote:

Sometimes we then go on to have a conversation about wrapping errors in other error types, and that can be interesting, but now we're talking about adding a big, messy feature just to get "safety" guarantees for a fairly minor need.

Programmers often have an instinct to obsess over error taxonomies that is very rarely directed at solving any real problem; it is just self-imposed busy-work.

- Some people think that while it might be useful in some narrow cases, the utility isn’t high enough to justify making the language more complex (complexity that would intrude on the APIs of result types, futures, etc)

I’m sure there are other points in the discussion that I’m forgetting.

One thing that I’m personally very concerned about is in the systems programming domain. Systems code is sort of the classic example of code that is low-level enough and finely specified enough that there are lots of knowable things, including the failure modes.

Here we are using "systems" to mean "embedded systems and kernels". And frankly even a kernel is a large enough system that they don't want to exhaustively switch over failures; they just want the static guarantees that go along with a constrained error type.

Beyond expressivity though, our current model involves boxing thrown values into an Error existential, something that forces an implicit memory allocation when the value is large. Unless this is fixed, I’m very concerned that we’ll end up with a situation where certain kinds of systems code (i.e., that which cares about real time guarantees) will not be able to use error handling at all.

JohnMC has some ideas on how to change code generation for ‘throws’ to avoid this problem, but I don’t understand his ideas enough to know if they are practical and likely to happen or not.

Essentially, you give Error a tagged-pointer representation to allow payload-less errors on non-generic error types to be allocated globally, and then you can (1) tell people to not throw errors that require allocation if it's vital to avoid allocation (just like we would tell them today not to construct classes or indirect enum cases) and (2) allow a special global payload-less error to be substituted if error allocation fails.

Of course, we could also say that systems code is required to use a typed-throws feature that we add down the line for their purposes. Or just tell them to not use payloads. Or force them to constrain their error types to fit within some given size. (Note that obsessive error taxonomies tend to end up with a bunch of indirect enum cases anyway, because they get recursive, so the allocation problem is very real whatever we do.)

John.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[gwendal.roue (Gwendal Roué)]

Hello all,

I'm also on the "side" of untyped errors, but I can imagine how other developers may like a stricter error hierarchy. It surely fits some situations.

Enter Result<T> and Result<T, E>:

Since Swift "native" errors don't fit well with asynchronous APIs, various ways to encapsulate them have emerged, most of them eventually relying on some kind of variant of those `Result` type:

  // Untyped errors
  enum Result<T> {
    case success(T)
    case failure(Error)
  }
  
  // Typed errors
  enum Result<T, E: Error> {
    case success(T)
    case failure(E)
  }

The first Result<T> fits well people who like untyped errors. And Result<T, E> fits people who prefer typed errors. Result<T> is objectively closer to the "spirit" of Swift 2-4. Yet Result<T, E> has the right to live as well.

When Swift 5 brings sugar syntax around async/await/etc, most needs for Result<T> will naturally vanish.

However, the need for Result<T, E> will remain. The debate about "typed throws", for me, sums up to this question: will the typed folks be able to take profit from the syntax sugar brought by async/await/etc of Swift 5? Or will they have to keep on carrying Result<T, E> with them?

Gwendal Roué

···

Le 18 août 2017 à 10:23, John McCall via swift-evolution <swift-evolution@swift.org> a écrit :

On Aug 18, 2017, at 3:28 AM, Charlie Monroe <charlie@charliemonroe.net <mailto:charlie@charliemonroe.net>> wrote:

On Aug 18, 2017, at 8:27 AM, John McCall via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Aug 18, 2017, at 12:58 AM, Chris Lattner via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
Splitting this off into its own thread:

On Aug 17, 2017, at 7:39 PM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:
One related topic that isn’t discussed is type errors. Many third party libraries use a Result type with typed errors. Moving to an async / await model without also introducing typed errors into Swift would require giving up something that is highly valued by many Swift developers. Maybe Swift 5 is the right time to tackle typed errors as well. I would be happy to help with design and drafting a proposal but would need collaborators on the implementation side.

Typed throws is something we need to settle one way or the other, and I agree it would be nice to do that in the Swift 5 cycle.

For the purposes of this sub-discussion, I think there are three kinds of code to think about:
1) large scale API like Cocoa which evolve (adding significant functionality) over the course of many years and can’t break clients.
2) the public API of shared swiftpm packages, whose lifecycle may rise and fall - being obsoleted and replaced by better packages if they encounter a design problem.
3) internal APIs and applications, which are easy to change because the implementations and clients of the APIs are owned by the same people.

These each have different sorts of concerns, and we hope that something can start out as #3 but work its way up the stack gracefully.

Here is where I think things stand on it:
- There is consensus that untyped throws is the right thing for a large scale API like Cocoa. NSError is effectively proven here. Even if typed throws is introduced, Apple is unlikely to adopt it in their APIs for this reason.
- There is consensus that untyped throws is the right default for people to reach for for public package (#2).
- There is consensus that Java and other systems that encourage lists of throws error types lead to problematic APIs for a variety of reasons.
- There is disagreement about whether internal APIs (#3) should use it. It seems perfect to be able to write exhaustive catches in this situation, since everything in knowable. OTOH, this could encourage abuse of error handling in cases where you really should return an enum instead of using throws.
- Some people are concerned that introducing typed throws would cause people to reach for it instead of using untyped throws for public package APIs.

Even for non-public code. The only practical merit of typed throws I have ever seen someone demonstrate is that it would let them use contextual lookup in a throw or catch. People always say "I'll be able to exhaustively switch over my errors", and then I ask them to show me where they want to do that, and they show me something that just logs the error, which of course does not require typed throws. Every. Single. Time.

The issue I see here with non-typed errors is that relying on documentation is very error-prone. I'll give an example where I've used exhaustive error catching (but then again, I was generally the only one using exhaustive enum switches when we discussed those). I've made a simple library for reporting purchases to a server. The report needs to be signed using a certificate and there are some validations to be made.

This generally divides the errors into three logical areas - initialization (e.g. errors when loading the certificate, etc.), validation (when the document doesn't pass validation) and sending (network error, error response from the server, etc.).

Instead of using a large error enum, I've split this into three enums. At this point, especially for a newcommer to the code, he may not realize which method can throw which of these error enums.

I've found that the app can take advantage of knowing what's wrong. For example, if some required information is missing e.g. Validation.subjectNameMissing is thrown. In such case the application can inform the user that name is missing and it can offer to open UI to enter this information (in the case of my app, the UI for sending is in the document view, while the mentioned "subject name" information is in Preferences).

This way I exhaustively switch over the error enums, suggesting to the user solution of the particular problem without dumbing down to a message "Oops, something went wrong, but I have no idea what because this kind of error is not handled.".

Surely you must have a message like that. You're transmitting over a network, so all sorts of things can go wrong that you're not going to explain in detail to the user or have specific recoveries for. I would guess that have a generic handler for errors, and it has carefully-considered responses for specific failures (validation errors, maybe initialization errors) but a default response for others. Maybe you've put effort into handling more errors intelligently, trying to let fewer and fewer things end up with the default response — that's great, but it must still be there.

That's one of the keys to my argument here: practically speaking, from the perspective of any specific bit of code, there will always be a default response, because errors naturally quickly tend towards complexity, far more complexity than any client can exhaustively handle. Typed throws just means that error types will all have catch-all cases like MyError.other(Error), which mostly seems counter-productive to me.

I totally agree that we could do a lot more for documentation. I might be able to be talked into a language design that expressly acknowledges that it's just providing documentation and usability hints and doesn't normally let you avoid the need for default cases. But I don't think there's a compelling need for such a feature to land in Swift 5.

Alternatives I've considered:

- wrapping all the errors into an "Error" enum which would switch over type of the error, which is not a great solution as in some cases you only throw one type of error

That's interesting. In what cases do you only throw one type of error? Does it not have a catch-all case?

- I could throw some error that only contains verbose description of the problem (generally a String), but I don't feel it's the library's job to stringify the error as it can be used for a command-line tools as well

Absolutely. Using enums is a much better way of structuring errors than using a string.

Perhpas I'm missing something, but dealing with UI and presenting an adequate error dialog to the user can be a challenge in current state of things given that currently, in the error catching, you fallback to the basic Error type and generally don't have a lot of options but to display something like "unknown error" - which is terrible for the user.

Again, it comes down to whether that's ever completely avoidable, and I don't think it is. Good error-handling means doing your best to handle common errors well.

John.

Sometimes we then go on to have a conversation about wrapping errors in other error types, and that can be interesting, but now we're talking about adding a big, messy feature just to get "safety" guarantees for a fairly minor need.

Programmers often have an instinct to obsess over error taxonomies that is very rarely directed at solving any real problem; it is just self-imposed busy-work.

- Some people think that while it might be useful in some narrow cases, the utility isn’t high enough to justify making the language more complex (complexity that would intrude on the APIs of result types, futures, etc)

I’m sure there are other points in the discussion that I’m forgetting.

One thing that I’m personally very concerned about is in the systems programming domain. Systems code is sort of the classic example of code that is low-level enough and finely specified enough that there are lots of knowable things, including the failure modes.

Here we are using "systems" to mean "embedded systems and kernels". And frankly even a kernel is a large enough system that they don't want to exhaustively switch over failures; they just want the static guarantees that go along with a constrained error type.

Beyond expressivity though, our current model involves boxing thrown values into an Error existential, something that forces an implicit memory allocation when the value is large. Unless this is fixed, I’m very concerned that we’ll end up with a situation where certain kinds of systems code (i.e., that which cares about real time guarantees) will not be able to use error handling at all.

JohnMC has some ideas on how to change code generation for ‘throws’ to avoid this problem, but I don’t understand his ideas enough to know if they are practical and likely to happen or not.

Essentially, you give Error a tagged-pointer representation to allow payload-less errors on non-generic error types to be allocated globally, and then you can (1) tell people to not throw errors that require allocation if it's vital to avoid allocation (just like we would tell them today not to construct classes or indirect enum cases) and (2) allow a special global payload-less error to be substituted if error allocation fails.

Of course, we could also say that systems code is required to use a typed-throws feature that we add down the line for their purposes. Or just tell them to not use payloads. Or force them to constrain their error types to fit within some given size. (Note that obsessive error taxonomies tend to end up with a bunch of indirect enum cases anyway, because they get recursive, so the allocation problem is very real whatever we do.)

John.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution