Adding Result to the Standard Library

[xwu (Xiaodi Wu)]

@allevato

If it makes sense to not require additional semantics provided via the type system for the failure case of Result, then why does it make sense to require that throw only apply to types conforming to Error?

Because, again, throwing was designed specifically for recoverable errors where the operation could fail in more than one meaningful way.

The question here is whether Result should be tied to that design or not; honestly, I could go either way, but I think there's a good argument to be made that if it's to hold independent weight (like both Optional and throwing does within the current design) then it ought not to be.

@allevato

I’m not sure that the ability to substitute Void there just because it happens to be a singular-valued type strengthens the case that other non-Error failure cases hold their weight.

Consider Double.signalingNaN. It's unambiguously a failure case; it doesn't even have its own type, but nonetheless it has no purpose other than to signify failure. Likewise, functions that return some positive integer on success but -1 on failure could be modeled as Result<T, T>.

[allevato (Tony Allevato)]

@xwu

Because, again, throwing was designed specifically for recoverable errors where the operation could fail in more than one meaningful way.

But recoverability is orthogonal to whether all the thrown types share a common conformance, right? Error is just an implementation detail imposed by the language.

@xwu

Consider Double.signalingNaN. It’s unambiguously a failure case; it doesn’t even have its own type, but nonetheless it has no purpose other than to signify failure. Likewise, functions that return some positive integer on success but -1 on failure could be modeled as Result<T, T>.

When you say it "could be modeled" do you mean it in the sense of "these two things are algebraically equivalent but you would never want to expose that API", or do you mean "that would be a reasonable API to expose to callers"?

If it's the former, sure—they're equivalent. But if some function returns –1 and only –1 in the case of an error, then why would we ever want to expose an API shaped like that? At this point, it feels like discussing type equivalences for the sake of type equivalences instead of focusing on what we would want an actual API to look like.

[lynch.sft (Adora Lynch)]

How much effort would it be to transition from Result<T> (untyped) to Result<T, E: Error> (typed)?

If and when the swift community decides to switch to typed errors, there will be non-trivial transition effort, particularly in the face of ABI stability. How much more effort would it be to also transition our Result type?

My reading of this list says the shortest edit path from Result<T> to Result<T, E: Error> is:

1. Make Error conform to itself with some compiler magic. (Unfortunate, but sometimes necessary.)
2. Introduce Default Generic Arguments (A generally useful feature.)

This would result in a final signature of ResultT, E: Error = Error>.

How hard are 1 and 2 to implement? How hard to deploy in the context of a bunch of existing Result<T>s? Are there easier migration paths?

If it can be deterred that there is a known-cost migration path from an untyped Result to a typed Result, then let's just add Result<T> now (because it matches swift's synchronous error handing as it exists today) and migrate as needed.

[xwu (Xiaodi Wu)]

@allevato

But recoverability is orthogonal to whether all the thrown types share a common conformance, right? Error is just an implementation detail imposed by the language.

Not in my understanding of Swift's error handling design, no. My understanding is that throwing is specifically designed for recoverable errors, which conform to the protocol Error.

If yours is a failure that isn't recoverable, don't conform it to Error, and don't throw it--which, conveniently, is enforced by the design as long as you don't conform it to Error.

@allevato

If it’s the former, sure—they’re equivalent. But if some function returns –1 and only –1 in the case of an error, then why would we ever want to expose an API shaped like that? At this point, it feels like discussing type equivalences for the sake of type equivalences instead of focusing on what we would want an actual API to look like.

A good question. It's pretty heavyweight to shoehorn any sort of failure into Swift throws; that's entirely the rationale of Optional being an alternative for simple domain errors. The same argument applies to other scenarios where we interact with other forms of failure. Some examples off the top of my head:

• We could build this into API note functionality, for instance, so that C functions that return T but have an error case instead return Result<T, T> in Swift; users would be spared the need to look up documentation as to which particular value (-1, or 0, or INT_MAX) is the arbitrarily designated failure case.

• Currently, users try to wrap HTTP status codes in an enum where each status code is a distinct case; it's a pretty horrible type to write and maintain, and it also erases the internal classifications that the numbers provide ("4XX" status codes are supposed to represent client errors, for example, but the equivalent pattern matching would be case .badRequest, .unauthorized, .paymentRequested, .forbidden, ...). Instead of shoving a round peg into a square hole, we could use a Result<T, Int> as a lightweight model for these errors.

[xwu (Xiaodi Wu)]

@lynch.sft

How much effort would it be to transition from Result<T> (untyped) to Result<T, E: Error> (typed)?

If and when the swift community decides to switch to typed errors, there will be non-trivial transition effort, particularly in the face of ABI stability. How much more effort would it be to also transition our Result type?

My reading of this list says the shortest edit path from Result<T> to Result<T, E: Error> is:

Make Error conform to itself with some compiler magic. (Unfortunate, but sometimes necessary.)
Introduce Default Generic Arguments (A generally useful feature.)

This would result in a final signature of ResultT, E: Error = Error>.

How hard are 1 and 2 to implement? How hard to deploy in the context of a bunch of existing Result<T>s? Are there easier migration paths?

If it can be deterred that there is a known-cost migration path from an untyped Result to a typed Result, then let’s just add Result<T> now (because it matches swift’s synchronous error handing as it exists today) and migrate as needed.

This is pragmatic, but it's entirely circumventing the salient design questions:

• Should Swift have typed throws?
• Should Result's failure case be constrained to associated values that are thrown?

The entire purpose of this forum is to answer such questions as we design features. It'd be supremely unfortunate to abdicate this responsibility.

[allevato (Tony Allevato)]

@xwu

We could build this into API note functionality, for instance, so that C functions that return T but have an error case always return Result<T, T>; users would be spared the need to look up documentation as to which particular value (-1, or 0, or INT_MAX) is the arbitrarily designated failure case.

Interesting idea. But if we continue with the language design philosophy that Optional is used for trivial failure cases, what is the advantage of returning Result<T, T> over transforming this to Optional<T> if there is a single sentinel value that can be returned?

@xwu

Currently, users try to wrap HTTP status codes in an enum where each status code is a distinct case; it’s a pretty horrible type to write and maintain, and it also erases the internal classifications that the numbers provide (“4XX” status codes are supposed to represent client errors, for example, but the equivalent pattern matching would be case .badRequest, .unauthorized, .paymentRequested, .forbidden, ...). Instead of shoving a round peg into a square hole, we could use a Result<T, Int> as a lightweight model for these errors.

HTTP statuses are a bit trickier to address as Error or Result because they're not mutually exclusive with the rest of the response. If we assume T is the response payload, Result<T, Int> may not make sense because, for example, even a 404 error can come back with valid custom error page content.

[xwu (Xiaodi Wu)]

@allevato

Interesting idea. But if we continue with the language design philosophy that Optional is used for trivial failure cases, what is the advantage of returning Result<T, T> over transforming this to Optional<T> if there is a single sentinel value that can be returned?

So that the interop isn't lossy--you may need to pass the sentinel value back to a C function.

(Is Result therefore being used here with the same semantics as an Optional with an associated value for none? Yes!)

@allevato

HTTP statuses are a bit trickier to address as Error or Result because they’re not mutually exclusive with the rest of the response.

Good point; that was perhaps not the best example. The point I want to make there is that--even if the failure case were to adopt a more elaborate design--a Result type that doesn't require its failure case to have an Error value permits alternative designs that are intermediate in "weight" between Optional and throwing.

[allevato (Tony Allevato)]

@xwu

So that the interop isn’t lossy–you may need to pass the sentinel value back to a C function.

That's a fair point! It would be an interesting exercise to try to dig up some motivating examples to see if that pattern is prevalent enough to design around.

Without any extensions, a Result<T, T> would still be a little awkward to use in that kind of situation because the user would have to still have two code paths to unwrap whichever case is set. But it would be reasonable to have an extension to add a var value: S to Result<S, F> where S == F, which would make that easier to interop with.

[dwaite (David Waite)]

@allevato

Optional has two main uses:

• Indicating the expected absence of a value, like Dictionary.subscript
• Indicating the inability to produce a value, like Int.init?(String, radix: Int), when there is only one possible failure case (the string is either a valid integer of the receiver’s size in that radix, or it’s not)

I kind of consider the second situation to be an improper application of Optional and think there would be value in having those initializers be throwing instead. The error could pass back additional information that must already be known anyway, like the index of the character that caused the conversion to fail.

The second is ironically one of the few where I think a typed throw may be appropriate - the API can only fail in a few, well-understood ways, and doesn't have extension points which may need to propagate other errors. The errors form a closed set of which all can be reasonably expected to be utilized to aid the calling code in recovery or intelligent user reporting for manual recovery (i.e. please retry entering in a valid number)

Compare this to say Decoder, where the errors could come from multiple sources (the underlying data I/O, the format of the data, illegal usage), and where even knowing you are using a JSON decoder doesn't give a reasonable expectation of either recovery logic or being able to instruct a user on appropriate action ("bad format on line 1" - from what? a JSON document? Where is it - in a plist? a database? from the network?)

That segue aside, I thought there were performance reasons to represent this particular call with no failure information. That may have been under the assumption that the error might return an Integer of the valid prefix data.

[Chris_Lattner3 (Chris Lattner)]

@mdiep

I don’t see why an E: Error constraint would be important. Does anyone else?

Because Result is a reified try call. If you can only throw an Error, then Result.failure should only take an Error.

Wherein we come back to the debate of whether Swift will support typed throws. In a typed throws world, Error is merely the default when you don't specify a type, but has no significance when you're throwing a typed value. Of course, that typed value may also conform to Error, but the try/throw syntax has no dependence on that.

I happen to be a very strong proponent for typed throws for a number of reasons (but, to be clear, I agree completely that the existing untyped throws is the right thing for large scale frameworks like Cocoa). However, my opinion isn't really relevant unless we're going to embark on the discussion to resolve whether typed throws is the right thing or not.

The premise of this discussion was to move forward with Result without having to resolve the typed throws debate. I see two options here: 1) address the debate head on and delay Result until it is decided, or 2) move forward with a desigh for Result that is open to supporting typed throws in the future.

Taking the approach of "defer the discussion but build a model that doesn't support typed throws" doesn't make sense to me as a path forward.

-Chris

[Chris_Lattner3 (Chris Lattner)]

@anandabits

I asked upthread about the rationale for restricting throws to types that conform to Error and haven’t gotten a response yet. There is no fundamental semantic reason for that restriction that I can think of either. It is fundamentally a control flow mechanism.

Why aren’t we allowed to say throw “error message” or throw 42?

There is nothing to prevent it. If we finally come to our senses :-) and support typed throws, this would be entirely possible:

func foo() throws Int -> String {
  if .. {
    return "foo"
  } else {
    throw 42
  }
}

[hooman (Hooman Mehr)]

Chris, while on the topic of Error protocol. This protocol seems to be special in some ways. For example, MemoryLayout<Error>.size returns 8 (on intel 64 bit), which is the same as an @objc protocol, but it works without Foundation. Is internals of Error protocol documented somewhere? (Even source code comment would be fine for me)

[hooman (Hooman Mehr)]

@Chris_Lattner3

There is nothing to prevent it. If we finally come to our senses :-) and support typed throws, this would be entirely possible:

Currently, I am a strong opponent of typed throws. Can you point me to some discussion or rationale on why you think typed throws is a good idea? How Swifts typed throws would address deficiencies of Java's checked exceptions which seems to be the closest existing implementation?

[nuclearace (Erik Little)]

I believe swift/ErrorObject.h at main · apple/swift · GitHub is the box type for the errors.

[hooman (Hooman Mehr)]

Thanks. It seems to be what I was looking for.

[anandabits (Matthew Johnson)]

@Chris_Lattner3

There is nothing to prevent it. If we finally come to our senses :-) and support typed throws, this would be entirely possible:

func foo() throws Int -> String {
if .. {
return "foo"
} else {
throw 42
}
}

Cool! With this possible future direction in mind, I agree that the unconstrained Result<T, E> is the way to go.

[Chris_Lattner3 (Chris Lattner)]

@hooman

Currently, I am a strong opponent of typed throws. Can you point me to some discussion or rationale on why you think typed throws is a good idea?

This has been discussed extensively in the past. Here are a few things that come to mind, but this isn't a complete coverage. I will start by restating that I agree with use of Error for the common case, and particularly for large scale frameworks like Cocoa.

Points in favor of typed throws:

1. Philosophical: Swift intentionally has strong dualities between static and dynamic features: struct/enum is very static, classes are very dynamic. Generics are static, existentials are dynamic. Untyped throws is dynamic, typed throws would be the static counterpart.

2. Practical: untyped throws requires memory allocation. In some domains this is a nonstarter (e.g. low level systems code and high performance subsystem), which means that currently there is no way to use swift error handling in those cases, so you have to resort to explicit code that is much more prone to bugs. Beyond the allocation, this sort of code is often exceptionally safety critical, and static analysis of it is hugely beneficial.

3. Interop: Swift has lots of features (e.g. UnsafePointer) that exist primarily because of interoperability. Many POSIX APIs "should" be typed as throwing a (non-@frozen) "errno" enum. They are never going to throw random objects. There are other languages, APIs, and error handling to interop with, right now our model only really supports NSError.

4. Convenience: Throwing untyped errors prevents exhaustive pattern matching in catch clauses, preventing its effective use in some cases. For example, I've written several parsers (including Clang and Swift's) which would benefit from throwing a locally defined closed/frozen enum (with things like "parse error" and "code completion") and being able to exhaustively switch on them. The absence of this and the performance consequences of untyped throws prevent swift error handling from being able to use this, forcing the use of buggier explicit techniques.

5. The primary motivations for untyped throws are guided by long lived public APIs. In the real world, some people work on large applications with lots of internal APIs, and have full control over them.

6. The only objections I've heard to including it are potential for misuse, which I find unmotivating. There are lots of ways to misuse lots of things, I don't see this one as being particularly risky, particularly given the obvious design.

@hooman

How Swifts typed throws would address deficiencies of Java’s checked exceptions which seems to be the closest existing implementation?

Java's design has tons of problems that this we would not be replicating. For example, we would only allow throwing at most a single type, and do not allow an equivalent of completely unchecked "runtime exceptions". The single type can be a non-frozen enum, and thus may be extended resiliently - completely unlike anything in the Java system.

These are fundamental differences. IMO, rooting your thinking about this from Java's design is not a very useful place to start from.

-Chris

[dwaite (David Waite)]

@hooman

Currently, I am a strong opponent of typed throws. Can you point me to some discussion or rationale on why you think typed throws is a good idea? How Swifts typed throws would address deficiencies of Java’s checked exceptions which seems to be the closest existing implementation?

There are three features to the Java exception model which might be referred to as checked exceptions IMHO:

1. Mandating that thrown exceptions are declared on method signatures, such that callers have to make allowances to handle them or pass them to their callers.
2. Declaring the types of exceptions thrown.
3. Escape hatching runtime exceptions and system/language errors (such as out of memory conditions, thread deaths, bad bytecode, etc) so that they don't have to be declared.

We could say swift already has 1, as errors are declared via a throws declaration. We could also say swift won't have 3, because the runtime and system error model is different (typically aborting the context of the running code completely - currently the process, but in the future maybe an AppDomain like in C# or an actor to support server processes).

So 2 is more the feature which is in question for Swift - today the throws declaration is untyped, similar AFAICT to most popular languages with exceptions and Go. Rust uses Result<T,E> (with enum values Ok and Err, and no constraints on E).

[hooman (Hooman Mehr)]

Thank you very much for your detailed and informative response. It is a very good summary of the key benefits of typed throws.

I have been ignorant about some of the key internal details. For example, I didn't know throwing currently always leads to memory allocation. I was assuming that if I do something like:

extension Int: Error {}

func f(x: Int) throws -> Int { throw -1 }

It would not allocate memory. (Also including cases where you would create a simple error enum or any case when the error type is smaller than the size of the error returning register)

This makes your point #2 totally convincing for a real need for typed throws for system programming.

Point #3 is also totally valid and important.

For point 4, I manually first downcast the error to my error enum and get exhaustiveness. Typed throws would make it nicer, but it wouldn't be a game changer for me. But I understand that avoiding a dynamic typecast is a game changer for systems programming and performance critical code.

Concerning point 5, I find it a bit weaker as the claim of complete control for me have repeatedly turned out to be a myth in practice.

@Chris_Lattner3

IMO, rooting your thinking about this from Java’s design is not a very useful place to start from.

You are correct. I have been looking at this mostly through the lens of my horrible experience with Java exceptions. I am sure we can find the right way to do this in Swift.

Still, I view the huge community of Java developers who are conditioned in a certain way towards error handling as a real threat to the success of Swift's future typed error support. I think we should work hard on educating correct use of Swift's error handling capabilities. Especially after we add Result and typed errors.

[dwaite (David Waite)]

@Chris_Lattner3

The only objections I’ve heard to including it are potential for misuse, which I find unmotivating. There are lots of ways to misuse lots of things, I don’t see this one as being particularly risky, particularly given the obvious design.

I would agree that my opposition to typed throws are their potential for misuse. However, this is because Swift being an opinionated language goes beyond just having an opinion on whether throws should be typed or untyped, but the purpose for throwing in the first place.

As a more practical example, I would point to Decodable - errors could come:

• From the data source, e.g. an I/O error, issue with the layout of a bundle, etc.
• From the decoder/archivers, e.g. data format corruption or error
• From the decodable types, e.g. retrieved data does not represent a valid state

All three of these could have their own custom errors to report. The question is how to best compose these options to maximize the ability for the calling code to recover from and/or report the issue.

Due to interfaces in java using the type of exceptions in their signatures, I feel from my experience that it is common to spend your error handling time massaging errors to fit into the type system, usually by wrapping the exception in a supported exception type, losing pattern-matching access to the details of the cause of the error. Time is not spent in considering how a caller would recover from an error, so until a need is identified to do otherwise most errors (for my area of expertise, server-side development) simply abort the current request and log a stack dump of the issue for a developer to (hopefully) diagnose later.

Swift's lack of (non-tagged) union types and the lack of attached stack dumps could make both massaging errors to fit into the type system imposed by callers harder, and the consequences of not designing around error recovery more severe.

@Chris_Lattner3

The single type can be a non-frozen enum, and thus may be extended resiliently - completely unlike anything in the Java system.

A final error type that is only extensible by a third party frameworks' author may mean functions and protocol implementations plugged into that framework may have serious contortions to fit into said type system. A framework can of course limit the types of errors at runtime (via a precondition); my concern is if language support for declaring static thrown types encourages people to craft code that limits the robustness of Swift code in general, the way it has done for Java code.