[Pitch N+1] Typed Throws

It's actually much simpler than I thought, with that added, it’s just as @dmt originally wrote it:

struct WrappedError<Wrapped: Error>: Error {
  let wrapped: Wrapped
}

func f<E>(g: () throws(E) -> Void) throws(WrappedError<E>) {
  do {
    try g()
  } catch {
    throw WrappedError(wrapped: error)
  }
}

That's tidy.

How would this magic detection work if WrappedError were a type imported from a different, resilient module?

Also, how could struct Result be defined in a way that doesn’t cause Result<_, Never> to become uninhabited?

1 Like

Whether a type is uninhabited or not is derivable from its layout, so if it's frozen we can look at the layout. But more generally, if we had a layout constraint for it, then maybe you could explicitly declare extension WrappedError: Uninhabited where Wrapped: Uninhabited {} as an explicit guarantee that that's true (and have the compiler check that it is in fact true).

Result is an enum, and as long as an enum has one inhabited case, the enum is inhabited. When you hear functional programming nerds talk about structs and enums as "product types" and "sum types" this is one reason why: a struct has the product of all possible states of its fields, so if any field is uninhabited (zero) then zero * n == zero. An enum has the sum of all possible states of its payloads, so if any field is uninhabited that doesn't affect the habitation of its other payloads, because zero + n == n. If you were going to write an Uninhabited declaration for Result, it would have to be both where Success: Uninhabited, Failure: Uninhabited.

17 Likes

So you want something with a behavior like this?

// hypothetical syntax:
typealias SomeErrorHelper<E> = if E == Never { Never } else { SomeError }

func f<E>(_ g: () throws(E) -> ()) throws(SomeErrorHelper<E>) { ... }

Maybe we should look into enabling very simple type logic to work like that. Not in this pitch of course but as a general direction it could be interesting.

2 Likes

Thank you!

I've said it before, but I want to reiterate that this has been a particularly insightful and educational discussion. For me at least. But I suspect others also didn't know about a lot of these use cases, or a lot of the techniques people have developed.

The Original Error Handling Handler

I've been re-reading the original Error Handling Rationale documentation, by @John_McCall (I think…? Git history suggests so). From circa Swift 2.

I actually don't recall it being mentioned before in this thread, so for anyone who hasn't read it, it's well worth it. It's very well written and although nominally for the Swift team itself, I find it very approachable to the lay-programmer.

It's particularly amusing to read its section on "Typed propagation" where it lays out the argument for why Swift should require marking of throwing functions at all (the throws keyword in function declarations, to be clear). And similarly why try should be required. Of course now these are all just givens and obviously wise design choices, but keep in mind that at the time it was written no mainstream programming language adopted that approach (to my knowledge).

Specificity of Typed Throws

And there's one subset of that in particular that I think is relevant to the topic of typed throws:

The alternative is to say that, by default, functions are not being able to generate errors. This agrees with what I'm assuming is the most common case. In terms of resilience, it means expecting users to think more carefully about which functions can generate errors before publishing an API; but this is similar to how Swift already asks them to think carefully about types. Also, they'll have at least added the right set of annotations for their initial implementation. So I believe this is a reasonable alternative.

But then:

The last question is how specific the typing should be: should a function be able to state the specific classes of errors it produces, or should the annotation be merely boolean?

Experience with Java suggests that getting over-specific with exception types doesn't really work out for the best. It's useful to be able to recognize specific classes of error, but libraries generally want to reserve flexibility about the exact kind of error they produce…

I find it incongruent with itself. It uses essentially the same argument - "this requires forethought and mistakes may be locked in forever" - to argue both sides. In the first, that functions should have to opt into throwing if they ever might want to throw, but then in the second that functions should not specify what they actually throw because then they might be unable to change it.

I think it's fair to insist on consistency here. Either we err on the side of flexibility - removing the throws keyword so that functions may change their mind in future - or we err on the side of programmer empowerment - allowing them to trade that flexibility for specificity (or other benefits, like eliminating runtime exception-handling costs, or existentials).

In fairness to that original document, it does go on to present other arguments too (e.g. the problem with wrappings, that's also been raised in this thread). I'm not going to reiterate or re-debate those here, as I think they're already well-covered within this discussion thread.

Really, what I'm getting at in bringing this up again is that for written guidance on when to used typed throws and on what type to choose, I really encourage everyone to focus on providing rationale, not dogma. Just as in a good style guide, a persuasive argument for a guideline is much more useful than a blunt "rule".

The current proposal draft is pretty good in this respect - When to use typed throws and Effect on API resilience are both mostly well-written explanations of the trade-offs and pitfalls (if a little incomplete). But the former does lean a little bit into prescriptivism with statements like "The loadBytes(from:) function is not a good candidate for typed throws" and "Typed throws is potentially applicable in the following circumstances…" and then some bullet points which are somewhat but not entirely self-explanatory. I think it paints an overly-restrictive picture of when typed throws may be used. It also completely omits mention or consideration of domains like Embedded Swift where typed throws might be appropriate in a broader range of circumstances (like a loadBytes(from:) method).

I expect these sections of the proposal will also form the basis of updates to the Swift Language Guide, which is where the audience gets much bigger and this really matters.

I almost didn't bring this up, because it feels like it might appear a bit nitpicky, but I do think it's important to write with reader agency in mind and focus on education more than codification. Especially for things written by Swift team members or in official Swift documentation, because their position imbues a degree of authority whether intended or not.

11 Likes

With regards to when and when not to use typed throws there’s also a long and detailed historical post that drills down into specifics that largely arrives at the same conclusions reached in the proposal: in most cases it’s probably better to throw any Error, but, reluctantly, it’s needed for certain use cases.

I feel the current proposal does a good job of summarising the cases where it’s needed. i.e. generic expressivity, embedded/real-time applications. But maybe could benefit from examples where you shouldn’t use typed throws. i.e. most places.

One particular use case that stood out for me as something that may benefit from highlighting as an anti-pattern was using typed throws to encode a makeshift call stack, i.e. creating a breadcrumb trail up the call-stack through the use of typed, nested errors.

Now, while this is a perfectly valid use case (having something akin to a stack trace to diagnose exactly where something might have gone awry is a useful thing) perhaps providing a lower friction community-blessed alternative (i.e. consistent across third party modules) would go someways to eliminating this potential anti-pattern.

How this would manifest, I don’t know. One option could be to include a Foundation Error type which does include a underlyingError: any Error? property, or another could be something enabled by the runtime, maybe by some top-level annotation, which facilitates the accumulation of a stack trace equivalent up each of the ‘throw’ sites.

Something else that will help greatly with this is the approval of SE-0409: If a module imports a third-party module internally, they’ll be nudged when they accidentally ‘encode the dependency tree’ by including a third-party Error type as a part of their own public Error.

Now, all of this might sound a little prescriptive, but personally that’s something I appreciate in a language. It’s liberating to have one ‘blessed’ way of doing things rather than needing to learn dozens of equally clever alternatives. The smaller surface area is a boon to creativity.

1 Like

The way I've been thinking about typed throws is a a bit different: typed errors are great (and probably preferred) within a module. Like enums, they can aid you in enforcing exhaustivity and managing expectations within your own code. However also like enums, they're probably not the best when it comes to crossing a module boundary as public API. (Again, like enums, there are cases where it's the right call, but have the same sort of constraints as using enums for public API: dealing with a closed set of possible values).

9 Likes

I don't think this is necessarily contradictory, for a few reasons.

It's perfectly consistent to argue that it is more common for a throwing function to change the set of errors it wants to throw than it is for a function which initially has no error conditions to suddenly start wanting to throw an error. I don't have the empirical data on hand to argue that case, but it seems like a reasonable position.

There's also an angle from the use side—defaulting all functions to throwing would be an opinionated choice of the language that functions should assume they may throw unless one affirmatively commits otherwise. I suspect this would have the effect of making many more functions throwing than we have today. This in turn would effect the set of solutions that are acceptable on the handling side of things. We'd either be imposing a very high annotation burden by asking users to mark every potentially-throwing call with try, or else end up with a regime where error handling are passthrough-by-default (after all, every function is throwing-by-default anyway).

I think both snippets you've quoted point to a single coherent conclusion: the zero-to-one step in number of thrown errors is more reasonably viewed as a difference in kind, whereas the one-to-N step is more reasonably viewed as a difference in degree. While this conclusion is certainly arguable, I don't think it's inconsistent.

6 Likes

Hence an open set or non-exhaustive enums would aid here for the use of a "dependency" or imported module/package/library. You'd maintain the benefits of typed throws. Swift comes across this case with C enums and you have to use the @unknown default case. You can currently cheat and create enums like this in C but they do not have associated values like Swift enums.

From an API design standpoint: if you have a resilient layer for a public interface that is not inlined into the caller that can itself create some sort of failure that is a good layer to avoid concrete thrown types.

So if you have a function defined as:

public func frobulate() throws -> String 

If there is not @inline(__always) and/or it is not on a @frozen type (i.e. you want to be free to change the implementation) the fewer requirements on your implementation the better. With no typed throws it only remains that frobulate exists, that the signature is throwing and that the return type is a String. It could throw in 1.0 of a framework an NSError, then in 1.1 throw a FrobulationFailure and in 2.0 throw a FrobulationFailure or a CancellationError.

On the flip side; if you have the function completely visible for its implementation to the caller (to support inlining) then the signature is reasonable to consider if typed throws is appropriate. Likewise if the source of throwing is guaranteed to be known to exist from a given source; for example a closure that may have a distinct throwing type then it is also reasonable to make it have the mirrored throwing signature.

func withBruddle<E: Error>( _ apply: (Bruddle) throws(E) -> Void) throws(E) -> Void

There is a similarity in spirit to closed set enumerations versus open set enumerations. In the rethrows-like behavior of withBruddle it is a known closed set: the with* function can only throw what the apply can throw. The advantage here is that the information of what it does is not lost when traversing through an application of the throwing closure at the cost of enforcing strict behavior of what that with* function does.

Personal speculation: ABI/API stable APIs won't use typed throws for root level sources of failure, non-ABI/API stable APIs may use them. Furthermore, constrained environments that have discrete known failure modes and are sensitive to the existential costs might also consider typed throws at the root of actions. Things that rethrow (either by protocol conformances, or by closures) will likely re-emit those failures where possible. Likewise, I would expect that APIs that violate that expectation (throwing from something that ought to consider a source of rethrowing) might need some refinement.

In short I think that typed throws will be exceedingly useful in certain scenarios, but it will be something that will have to be considered carefully when designing for ABI/API stability. And perhaps those discussions are something that should be had when designing things; who uses the error type and how?

2 Likes

Precisely.

Good conclusion, but odd start: :slightly_smiling_face:

Better for whom? Certainly not the users of this API.

Every aspect of API design is trade-offs between forward-compatibility and specificity. There's nothing special about errors in this regard.

Each case should use whatever compromise suits it best, whether that be a frozen enum, an open enum, a struct, a protocol (and then either opaquely or existentially), etc. Same as for its non-error return type, its parameters (e.g. take Strings specifically or genericise over StringProtocol? Int or FixedWidthInteger, or BinaryInteger?), and whether it returns an optional or throws at all (maybe it doesn't now, but might want to in future?).

Not to mention the structural elements - should you use structs (and if so expose them directly, or opaquely?) or should you use classes? Enums or mimicry thereof with static member variables? And so on. Structs are directly exposed all the time in so-called stable APIs, and it works fine, because not everything is likely to change.

And "breaking" compatibility is not the bogeyman some make it out to be, anyway. Apple's "stable" APIs "break" all the time, intentionally (mostly). Functionality changes (generally improving), bugs are fixed, etc. Sometimes that's done within the existing API, sometimes it requires new overloads, sometimes new APIs entirely. Users adapt and progress and as long as the "breaks" have justification, everything moves forward with at worst a little griping (and then, mainly of the "ugh, reality is annoying", not "this was a bad choice").

3 Likes

Aside from the new ABI impact of having an error type in the function definition, it doesn't seem like there are any new considerations typed throws requires beyond what having a public error type in the first place already does. In fact, starting with typed throws seems superior in pretty much all cases to me.

  • With a public error type you already have to deal with breaking exhaustiveness when using an enum or users missing cases when using a struct.
  • With compiler-checked error types, if you use an API below your public surface that produces untyped errors, the compiler will tell you about it, requiring you to either untype your error or handle that fact in your typed error. Otherwise this is just a silent change you may miss, on both sides.
  • Developers can easily erase the public error, either automatically or by overloading.

Really the only thing the language needs is (the equivalent of) @frozen enums in non-stable modes and the API evolution concerns would be completely solved. In that case, even if you did change your underlying implementation to produce a new error and you didn't already have a fallback case in your typed error, you'd be able to easily add cases to your existing error.

In particular, I'm not sure this is really true:

Changing the underlying type of an error is a major API breaking change, as users of 1.0 who updated to 1.1 would lose any error handling they had for NSError. Currently Swift can't surface these changes in the language, but they're still breaking at runtime, which is why defaulting to typed errors would be the better start. At best the consuming devs would have an "unknown error" case to handle casting failures and catch, but that assumes a lot from them and generally means a degraded error handling experience anyway.

4 Likes

Yes, for users! If the possible thrown types are not documented and subject to change, it doesn't do users a service to pretend like there's a strong contract for the thrown error only to have it break down the line. I don't feel it should be taken for granted that more information is always better for the user and that trading against specificity in favor of flexibility is hurting the user in favor of the author—being transparent about what is fixed and what is subject to change is helpful, not harmful.

There's definitely a spectrum of how breaking a "breaking change" is (both in terms of raw magnitude, but also in terms of a more squishy 'reasonableness'). Any change in behavior has the potential to break someone who was relying on it. The same can be said for changing the underlying type of an existential or opaque return value—after all, someone could have been dynamically casting to observe the true type—and yet we don't generally expect API authors to vend entirely private, purpose built wrappers to eschew dynamic type information.

Similarly if someone is casting an undocumented error and observing specifics about the returned value, that feels like more reasonable break than someone who saw NSError declared in the function signature and opted to rely on that.

Of course, if someone is already documenting "this function always throws FrobulationFailure when it fails," then sure, getting a compiler error when that changes seems strictly better for both clients and authors, and people who want to get specific about their error types should make use of typed throws instead. But I don't think "people could already be relying on undocumented implementation details" is a great reason for defaulting to typed throws.

7 Likes

Sure, but the question is why they're not documentable?

Sometimes it's a challenge to exhaustively document the failure modes, given the possibility new ones will arise in future. But that's still very different from not documenting any error cases.

And if there truly is nothing useful the API can promise about the error type, then the benefit of throwing (versus e.g. returning an optional) diminishes a lot. Yes, it gives you some options for more conveniently failing up through the stack, but that's about it. Which is fine if that's just the reality of the situation, but it's not a good situation so one should try to avoid it if possible. It may also be a sign that the function in question doesn't understand its own failure modes (or at least hasn't explained them clearly to callers), which is never good anyway.

Even for functions that don't throw it's important that callers understand the function's failure modes. What inputs will make this function break? What preconditions are required? What environmental factors are relevant? If you're not throwing then the best you can currently do is put those into your human-readable documentation, but if you are throwing it's a good opportunity to document it in a way that's also machine-readable and can help you find bugs (e.g. the compiler won't let you just make up fictional enum cases at your throw site, they have to be properly defined and therefore at least minimally documented).

Indeed. But there's two aspects to this we should recognise:

  1. If callers subvert abstractions, then nominally it's on them if it breaks. It's no different to if they rely on some undocumented behaviour of the function - if no such behaviour was promised, then arguably it's fair game for the implementation to change or revoke it. But

  2. People have stuff to get done. If APIs are incomplete, poorly documented, buggy, etc, then you just have to be realistic about how people will react, like it or not. I don't think there's any benefit, let-alone moral high ground, to saying "well I never explicitly promised my API would do this thing [that you have no practical choice but to rely on]".

So it's a give-and-take, a compromise. You give me a bad API, I "abuse" it by your definition, but I say you started it with the bad API to begin with. It's not even about who's right or wrong - it's about everyone in this situation ending up sad or angry.

To put it another way: advising people to default to untyped errors is tacitly encouraging them to not think about failure modes. Whereas encouraging them to try to use more specific error types gets them thinking about that. We need to be clear that they might indeed want to use some level of abstraction - maybe even all the way up to any Error - but it should be with reason. Forward-compatibility can be a perfectly good reason (but also doesn't necessarily mean any Error, it might just mean an open enum).

1 Like

I think you're going too far towards argumentum ad absurdum here. There's no need to invoke XKCD here when the uses of Error I've outlined are not only how people are using it right now, but the expected and only way the feature is supposed to be used by design. Unlike opaque return types, which are an explicit choice to hide internal implementation details, you're supposed to cast Errors to concrete types to extract useful information. Aside from basic error messaging that's the only way to make them useful. Like many Swift features, typed errors eliminate an entire class of bugs (incorrect casting) and allow the compiler to verify proper handling.

2 Likes

At this point, typed throws seems inevitable and this seems like the most straightforward and reasonable approach. It does make me want sum types even more though.

4 Likes

The part of discussion about usecases of rethrows was remarkably useful and exhaustive. But now it's again about good API design practice and how good/bad (un)typed throws are. And the thing is whatever one might say will be opinionated somehow. Swift has always had the ability to express typed errors, even before Result<>. It was a bit inconvenient, but still. In this sense the proposal doesn't add something that was previously fundamentally impossible. It closes the gap between Result-like types and throws, and makes throws less opinionated. (Which is also an opinion, of course).
But from the practical perspective what this implies? What questions does it raise?

  • Union types - spawned another thread
  • rethrows incompatibility - discussed
  • Are there functions in stdlib that can't benefit from typed throws?
  • How to move towards typed throws while maintaining ABI (technically)?
  • What functions in stdlib would be good candidates to adopt typed throws (apart from rethrows ones).
  • Conditional types - too marginal to discuss (I guess?), but still an interesting topic.
  • Generalization of Uninhabitable types - wanted
  • Limited support for any Error in embedded for errors whose layout fits existential container.
  • And many more other topics I don't know about...

So maybe discussion about these topics will be more productive than opinion exchange about API design. Not because API design principles regarding errors are not important, it's just all the opinions are already well discussed several times.

3 Likes

I think the consensus here seems to be almost universally in support of typed throws; it’s more of a discussion on when it’s useful rather than if it’s useful.

Personally, I’ve been fairly convinced from the arguments here that many APIs will benefit from typing their Errors – not least of which because it’s the only way that embedded/performance-annotated Swift will be able to use these APIs.

However, It seems to me that there is a sentiment here that says ‘typed throws, always’ and it’s on that point I find myself unconvinced. And without understanding why we need something it’s hard to say what we need. (I'm open to being convinced.)

Specifically, the further up the call stack you get, the less utility typed throws seems to have, and the more potential for unnecessary complexity. I think this comes down to the fact that while the function of a returned type is unary between caller and callee, a thrown Error could come from anywhere down the call stack and can fill multiple roles from control-flow, system diagnostics, user diagnostics, to simple success/failure notification.

It seems there’s a continuum that runs from the point the Error was thrown, right up through potentially multiple application, and even system, levels. And it seems that depending on at what point on that continuum the Error is caught the more or less likely it is to fill a particular role; Errors caught closer to the point that they’re thrown are more likely to be recoverable and used for control-flow, Errors at the other end of the continuum are more likely to be in an unbounded, often unrecoverable, set of other Errors used to surface system or user diagnostics, or simply to signal failure.

I do think this is worth discussing this in some way, as I think it helps frame the rationale for the supporting features that are required to get the best from typed errors; for example, does a union type serve to simplify error-handling, or would it promote unwieldy attempts to shift a snowball of strongly typed errors up the call-stack?

Furthermore, documenting this will serve as a guide to developers on when and how they should use typed throws; whether or not a union (of a union, of a union) of Error types is necessary in their own applications, or if they're better off sticking with untyped throws.

That's a long way of saying I think it's hard to talk about the design of something without considering the bigger picture, and API design certainly fits within that.

1 Like

Documenting failure modes is different than documenting precise semantic constraints about the thrown value when those failure modes are triggered. I agree that if there are well-defined preconditions that must be satisfied to successfully complete an operation, the API author ought to note that in the docs, but I don't think it then follows that the correct thing to do is to document a specific error that is thrown in this case, and then require the user to dynamically inspect the error value to determine how to recover. Rather, if an error is being thrown that there's an clear precondition and/or recovery path for, that's an indication that the author should provide a separate API for verifying that precondition, and the client should make use of that API directly.

Concretely, I would personally discourage writing code that looks like this:

func writeData() {
  do {
    try fileWriter.write(data, to: url)
  } catch let e as NoSuchFileError {
    fileManager.createFile(at: url)
    retryWriteData()
  } catch { ... }
}

in favor of:

func writeData() {
  do {
    try fileWriter.write(data, to: url, createIfNotExists: true)
  } catch { ... }
}

And if I as the API vendor wasn't providing the createIfNotExists: API (or fileExists(at:) or something) so that the only way to catch the "file doesn't exist" case was dynamically inspect the thrown error, then sure, that's on me for not giving my clients the tools they need to get stuff done.

Yeah, I totally agree that error handling via casting is not equivalent to casting away opaque return types, I'm only trying to drive the point home that there's a spectrum of reasonableness when it comes to "what behavior might clients be relying upon?" If you're casting to well-documented error types that the API has agreed it will return in particular circumstances, then yeah, a responsible library vendor shouldn't be changing those error types/values without a major version bump.

OTOH, if you're casting to error types for which no contract is present and deriving information critical to your program's control flow based on implementation details of the thrown errors, I would be hard to pressed to blame the library vendor for 'breaking' clients if they change some of those implementation details.

Yeah, if we are going to introduce typed throws then IMO a necessary part of the proposal is strong, opinionated guidance on how and when we expect the feature to be used (which, of course, might then inform the feature design). Hopefully this guidance can make its way into the API design guidelines.

2 Likes

In practice, this approach leads to TOCTTOU (time-of-check-to-time-of-use) vulnerabilities. Sometimes the only way to see if an operation will succeed is to try the operation. Filesystem operations like creat are the canonical example of this.

Filesystem operations can also fail in myriad ways, some of which come about as new features like sandboxing are added to the OS. That’s why I also think they’re the strongest case for not strictly constraining errors to a closed set.

3 Likes