[Pitch] Typed throws

Daniel_Leping · February 28, 2017, 4:44pm

Mathew, I totally understand what you're talking about. And this is exactly
what I want to avoid.

If a library developer knows how to handle certain case (i.e. recover or
throw a special error) I'm ok with it. Though I'm not ok with completely
hiding errors from the underlying implementation. The library developer
might not handle EVERYTHING. I want to know what happened and where.
Otherwise I will have to explicitly debug his library to find out what the
issue is. In most cases I will not have time to wait developer fixing the
library. I will fix it myself and make PR. So I consider tranparent errors
rather a benefit.

Hiding errors under library errors might have worked for proprietary
middleware. Today the world changed. It's open source now.

My point is that I'm completely against typed throws. I'm sure there will
be a lot of developers who would try to use the feature to hide underlying
errors under their own. I can't consider it a good practice (reasons
above). This is how good intentions (caring about API and library users)
can lead to bad consequences. Sounds good in theory - works bad in practice
(in real, non ideal world).

···

On Tue, 28 Feb 2017 at 18:18 Matthew Johnson <matthew@anandabits.com> wrote:

On Feb 28, 2017, at 5:09 AM, Daniel Leping <daniel@crossroadlabs.xyz> > wrote:

When you're going to present a user with specific error message you don't
do it for all errors. First of all at the end of a call chain you will have
a couple of dozens of possible errors and only 2 or 3 will do really matter
for the end user. Would you list all the dozen in params? Probably not.
You'll just catch what matters and put the rest to the log with a default
"sorry" message.

What I mean is it's OK to be "unaware" why everything failed. What really
does matter is what you can deal with and what you can't (the rest). "The
rest" is usually a magnitude bigger, though.

I might agree, that listing some errors in the throws clause is useful
feature for documentation (even so, I think the docs should be in the
comments), but only and only if it can throw the listed errors + a bunch of
other stuff. In Java it's resolved with a terrible solution of RuntimeError
interface which I definitely suggest to avoid.

As a library developer I can say that even for a library it's a nightmare
and libraries will start wrapping system errors with their own if the trend
would become to list errors in middleware. My preference though is to have
all errors transparently passed in both roles, being a library developer
and a library user.

Whether to wrap or not is a judgment call.

If the purpose of your library is to simply provide a cleaner experience
for a system API that is a stable dependency that will not change. Users
might expect to have the errors from the system API forwarded directly and
that would be perfectly appropriate.

In other cases you might have dependencies that are considered
implementation details. In that case if you don’t wrap the errors you’re
doing users a disservice by giving them an unstable interface for error
handling. Changing out your dependency that is supposed to be an
implementation detail becomes a breaking change.

On Tue, 28 Feb 2017 at 8:29 Karl Wagner <razielim@gmail.com> wrote:

On 28 Feb 2017, at 02:43, Matthew Johnson via swift-evolution < > swift-evolution@swift.org> wrote:

Sent from my iPad

On Feb 27, 2017, at 7:19 PM, Daniel Leping <daniel@crossroadlabs.xyz> > wrote:

Well, as Dave pointed, you can very rarely recover from an error, which
IMO is absolutely true.

If your operation fails you don't really care unless you can recover. And
you know your cases, which you can recover from (in reality one usually
does it in optimization phase, though).

What else do you need the type of error for at the very end of your call
stack? In 90% you will tell the user "ah, sorry, something happened. Come
back later" and log the error (if you haven't forgot it).

In most cases the errors are not for recovering. They neither are to be
presented to users. They are for developers to read the log/crash
report/whatever else and analyze it. Most of the errors are for debugging
purposes.

I don't want to deal with cumbersome code the purpose of which is to just
"obey the language rules". Unless I know how to recover I would rethrow it.
Than catch at the top of the stack and log + show the user a nice "sorry"
message without getting techy.

In order to provide a helpful experience to end users an app needs to know
about what might have caused the error and therefore what might (or might
not) resolve it, allowing the operation to succeed on a subsequent
attempt. This can influence the strategy an app uses to avoid bothering
the user if it might be resolvable without user intervention and can also
influence the content of the message presented to users if that is
necessary.

Errors are certainly useful, but the key distinction (IMO) is that callers
should never rely on an error happening or not. They are typically highly
implementation and hence version-specific. I think that means specific
errors shouldn’t be part of the function signature.

That doesn’t mean you can’t handle the error. A function that throws is
one that reserves the right to fail; maybe you can resolve the problem, but
maybe you can’t. That means your enclosing operation also needs to be
prepared to fail for reasons beyond _its_ control, and so on. At some point
your entire UI-level sub-operation (like opening the file, downloading the
data from the network) just fails, and you decide whether or not the user
needs to be told of that and how to do so.

Indeed, many apps don't bother with this kind of detail and just treat all
errors the same. Personally, I find that to be an unfortunate state of
affairs both as a user and as a developer.

Types can be useful in conveying this kind of information and abstracting
low level details that are not helpful at higher levels in the stack. Of
course types are not the only way to convey this information. But my
experience is that untyped errors often result in libraries with poor
documentation of error cases and not enough consideration of the error
handling experience of users of the library in general. That makes it
very difficult to handle errors well. I have experiences like this in
several languages and on several platforms, including Apple's.

I’m not sure that types are really the best abstraction for “the list of
errors that can be thrown by this function”. They’re fine for error
themselves, but not for per-function error-lists IMO. Most of our types are
way too rigid for this to be easy to live with.

If I understand what you’re saying, the core problem can be summarised as:
"I don’t know/can’t easily communicate which errors this function throws”.
Really, it is a documentation problem; it's too onerous to document every
individual throwing function, even with our very pretty markdown-like
syntax. I’m just not sure that the relatively rigid type-system is the
solution.

Now, if we look at this from a documentation perspective: obviously, the
compiler _typically_ can't generate documentation for you, because it
doesn’t know what your types/functions are meant for. Error documentation,
though, is different: the compiler *can* often know the specific errors
which get thrown (or rethrown from a call to another function from whom it
can get that information); and even when it can’t know that, often it can
at least know the specific type of error. We could enhance the compiler
libraries to track that; I spent half a day modifying the existing
error-checker in Sema to prove the concept and it’s close.

Swift libraries don’t have header files, so third-parties will only ever
see your documentation as generated by some tool which integrates the
compiler. That means the compiler can “improve” the documentation when it
is lacking. For yourself, if you care about explicitly writing out the
errors which get thrown and you’re not working with an IDE, you could
enable compiler-checking of your documentation comments. We can invent new
shorthands to make it easier to list all of your various errors (individual
cases, entire types, and maybe some private “other errors”), isolated from
the rest of the language. For example:

/// - throws: MyError.{errorOne, errorThree}, AnotherError.*, *
func doSomething() throws

Maybe you can think of other ways we could help the documentation problem
from the compiler side?

Typed errors are certainly no panacea but I believe they can be an
important tool. Experience in other languages has shown that to be the
case for many people writing many kinds of software.

If you don't believe typed errors will improve your code or if you just
don't want to deal with typed errors, just don't use them! You will be
able to continue using untyped throws just as you do today. You can even
do this if you use libraries that throw typed errors.

If it’s the wrong solution to the problem, it does make the language worse
overall.

On Tue, 28 Feb 2017 at 1:12 Matthew Johnson via swift-evolution < > swift-evolution@swift.org> wrote:

> On Feb 27, 2017, at 5:01 PM, Dave Abrahams <dabrahams@apple.com> wrote:
>
>
> on Mon Feb 27 2017, Matthew Johnson <matthew-AT-anandabits.com > <http://matthew-at-anandabits.com/>> wrote:
>
>>> On Feb 27, 2017, at 4:20 PM, Dave Abrahams <dabrahams@apple.com> > wrote:
>>>
>>>
>>> I'm sorry, I don't see any substantive difference, based on what you've
>>> written here, between this feature and const.
>>
>> Let me give it one more shot and then I’ll drop it. :)
>>
>> Const is viral because if an API does not declare its arguments const
>> it cannot be used by a caller who has a const argument.
>
> Unless the caller casts away const, thus erasing information that was
> previously encoded in the type system.

Yes, of course.

>
>> It is required in order to make an API as generally useful as
>> possible.
>>
>> Typed errors are not viral in this way because no callers are
>> prevented from calling an API regardless of whether it declares error
>> types or just throws Error like we have today.
>
> Unless the caller can recover (which is *very* rare) or it catches and
> rethrows one of the errors *it* declares, thus erasing information that
> was previously encoded in the type system.

I view this as being fundamentally different than casting away const.
Casting away const says “I know better than the types”.

Converting an error to a different type is extremely different. It much
more similar to any other kind of value wrapper a library might create in
order to shield its users from being coupled to its implementation details
/ dependencies. This is not a way *around* the type system in the sense
that casting away const is. It is a way of *using* the type system
(hopefully) to your advantage.

>
>> Pressure to declare error types in your signature in order to make
>> your function as generally useful as possible does not exist. Each
>> function is free to declare error types or not according to the
>> contract it wishes to expose.
>>
>> An argument can be made that community expectations might develop that
>> good APIs should declare error types and they could be considered
>> viral in this sense because any API that is simply declared `throws`
>> is dropping type information. But I think this overstates the case.
>> The community appears to be very sensitive to the problems that can
>> arise when error types are too concrete, especially across module
>> boundaries. I think we can learn to use the tool where it works well
>> and to avoid it where it causes problems.
>>
>>>
>>> --
>>> -Dave
>>
>
> --
> -Dave

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

John_McCall · February 22, 2017, 2:32pm

As you know, I still think that adding typed throws is the right thing to do. I understand your concern about “the feature could be misused” but the same thing is true about many other language features.

That's fair, but I do think there's an important difference here. The way I see it, typed-throws is really something of an expert feature, not because it's at all difficult to use, but the reverse: because it's easy to use without really thinking about the consequences. (And the benefits are pretty subtle, too.) I'm not saying that we should design it to be hard to use, but I think maybe it shouldn't immediately suggest itself, and it especially shouldn't come across as just a more specific version of throws.

Yeah, I agree that it will be appealing to people who don’t know better, but here’s the thing: the (almost certain) Swift design will prevent the bad thing from happening in practice.

Consider the barriers Swift already puts in place to prevent the bad thing (declaring an inappropriately narrow explicitly-specified throw signature) from happening:

1) First of all, you need to declare a public API. If it isn’t public, then there is no concern at all, you can evolve the implementation and clients together.

2) The Second problem depends on the number of errors it can throw. If there is exactly one type of error, the most common way to handle it is by returning optional. If you have one obvious failure mode with a value, then you throw that value. The most common case is where you can throw more than one sort of error, and therefore have an enum to describe it.

3) Third, your enum needs to be declared fragile in order to allow clients to enumerate their cases specifically.

The third step (having to mark your enum fragile, however it is spelled) is the biggest sign that you’re opting into a commitment that you should think really hard about. If folks don’t know that this is a big API commitment, then we have bigger problems.

You're imagining somebody adding a throws annotation because their function itself has one or two explicit throw sites. In reality, it's probably because they're calling some other function that can throw. The low-impedence path in that case is to declare that you throw whatever that function throws; this is why people talk about exceptions encoding your library's dependency tree.

One thing you didn’t mention is that boxing thrown values in an existential requires allocation in the general case. This may be unacceptable for some classes of Swift application (in the embedded / deep systems space) or simply undesirable because of the performance implication.

So, the performance implication cuts both ways. We can design the ABI for typed-throws so that, say, the callee initializes some buffer that's passed into it. That's an ABI that will kill some potential allocations in deep systems code, no question about it.

Agreed.

But in non-deep-systems code, we generally expect that error types will be resilient, which means that there are non-zero dynamic costs for allocating space on the stack for the error.

Proposed solution: ABI is that the callee takes in a register which is either a buffer address to fill in or null. On error, the callee returns the error pointer in a specific register. If there was a buffer passed in, it uses it, otherwise it allocates.

In practice, this allows the compiler to only pre-allocate the buffer when it knows the fixed size, otherwise the caller allocates on the heap on demand.

AFAICT, the cost of this API is only a “li rN, 0” in the normal path.

If you're willing to give up on a no-malloc guarantee, even if the fact of resilience, yes, this and similar techniques would work. I'm not sure that all "deep systems" code is quite that deep.

John.

···

On Feb 21, 2017, at 11:50 PM, Chris Lattner <clattner@nondot.org> wrote:
On Feb 20, 2017, at 11:12 PM, John McCall <rjmccall@apple.com> wrote:

Joe_Groff · February 22, 2017, 8:13pm

If you really want zero mallocs, it seems to me like creative stack accounting could get you there. In cases where we concretely know we're working with an existential, we can "explode" the existential into an opened type variable and a value of that type, and avoid allocating the existential representation. In the case of an untyped or resilient error, the callee could leave the error value and its type metadata on the stack somewhere without resetting the stack pointer, stick the address of that payload somewhere, and code would then propagate up to the catcher, who's responsible for consuming the error value and popping the stack when it's done with it. We could potentially do this more generally with existential returns too, so that existentials are a viable abstraction tool even in predictable-performance mode.

-Joe

···

On Feb 21, 2017, at 8:50 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 20, 2017, at 11:12 PM, John McCall <rjmccall@apple.com> wrote:

As you know, I still think that adding typed throws is the right thing to do. I understand your concern about “the feature could be misused” but the same thing is true about many other language features.

That's fair, but I do think there's an important difference here. The way I see it, typed-throws is really something of an expert feature, not because it's at all difficult to use, but the reverse: because it's easy to use without really thinking about the consequences. (And the benefits are pretty subtle, too.) I'm not saying that we should design it to be hard to use, but I think maybe it shouldn't immediately suggest itself, and it especially shouldn't come across as just a more specific version of throws.

Yeah, I agree that it will be appealing to people who don’t know better, but here’s the thing: the (almost certain) Swift design will prevent the bad thing from happening in practice.

Consider the barriers Swift already puts in place to prevent the bad thing (declaring an inappropriately narrow explicitly-specified throw signature) from happening:

1) First of all, you need to declare a public API. If it isn’t public, then there is no concern at all, you can evolve the implementation and clients together.

2) The Second problem depends on the number of errors it can throw. If there is exactly one type of error, the most common way to handle it is by returning optional. If you have one obvious failure mode with a value, then you throw that value. The most common case is where you can throw more than one sort of error, and therefore have an enum to describe it.

3) Third, your enum needs to be declared fragile in order to allow clients to enumerate their cases specifically.

The third step (having to mark your enum fragile, however it is spelled) is the biggest sign that you’re opting into a commitment that you should think really hard about. If folks don’t know that this is a big API commitment, then we have bigger problems.

One thing you didn’t mention is that boxing thrown values in an existential requires allocation in the general case. This may be unacceptable for some classes of Swift application (in the embedded / deep systems space) or simply undesirable because of the performance implication.

So, the performance implication cuts both ways. We can design the ABI for typed-throws so that, say, the callee initializes some buffer that's passed into it. That's an ABI that will kill some potential allocations in deep systems code, no question about it.

Agreed.

But in non-deep-systems code, we generally expect that error types will be resilient, which means that there are non-zero dynamic costs for allocating space on the stack for the error.

Proposed solution: ABI is that the callee takes in a register which is either a buffer address to fill in or null. On error, the callee returns the error pointer in a specific register. If there was a buffer passed in, it uses it, otherwise it allocates.

In practice, this allows the compiler to only pre-allocate the buffer when it knows the fixed size, otherwise the caller allocates on the heap on demand.

AFAICT, the cost of this API is only a “li rN, 0” in the normal path.

Joe_Groff · February 24, 2017, 5:00pm

If `rethrows` can be turned into sugar for regular type parametricity, I think that'd be a win over our current model. The sugar is probably still useful since `<E> (() throws E -> ()) throws SomethingDerivedFrom<E> -> ()` is common enough; whether SomethingDerivedFrom<E> is E, Error, or MyModulesErrorWrapper<E> would have to be discussed.

-Joe

···

On Feb 23, 2017, at 10:36 PM, Russ Bishop <xenadu@gmail.com> wrote:

On Feb 21, 2017, at 8:18 AM, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:

Yeah but Rust has a macro system so error-chain makes re-wrapping trivial. A simple `chain_err(|| “my new error message”)` is enough to define and “throw” a new error that wraps the underlying error.

That was my point—they recognized error chaining/layering as a common phonemenon and provided affordances for it as part of their design. I was saying we should do the same if we investigate typed errors seriously. (Not sure why you mentioned macros—chain_err is just a regular method, and could be defined as an extension method on Error in swift today.)

-Joe

Sorry, I meant error_chain! handles the boilerplate. At the end of the day something has to create the new error types.

I agree with you on the chaining; if we really do want to go down this road we need an elegant solution. Seeing people talk about getting rid of rethrows makes me think this isn’t yet a serious discussion.

Zaid_Daghestani · February 28, 2017, 5:17pm

Matthew, Per api fragility, the only API fragility would likely be adding a new error to the signature, which isn’t that bad. Any significant error changes, would probably mirror a significant change in that libraries pattern in and of itself, which would lend to a Semver.Major version bump anyhow.

It looks the consensus here is:
Typed throws can be good, but wrapping kills dev cycles, and these types can pollute our space.

To which, I think the obvious answer here is:
Typed throws with inferred signatures.

For the compiler to infer throw signatures is easily accessible and trivial to accomplish. Xcode can even display the inferred signatures. The compiler only need to add any `throw err` and `try erringFunc()` results to the throw sig.

Implementing this gives us two gains at little cost:
1) Error sets are now ready available to use, but we can still ignore if we please!
2) No catch Error, unless we *explicitly* want a catch all

I honestly believe freedom is key when it comes to error implementation. Environment usually dictates whether we fully implement error flows or not. With these, we still maintain our wonderfully fast iteration cycles by ignoring/catchall, but when we do implement errors, we get to fully leverage it.

Everyone wins!

Cheers,
Z

P.S. Stack traces in errors would be awesome.

···

On Feb 28, 2017, at 9:00 AM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 28, 2017, at 10:44 AM, Daniel Leping <daniel@crossroadlabs.xyz <mailto:daniel@crossroadlabs.xyz>> wrote:

Mathew, I totally understand what you're talking about. And this is exactly what I want to avoid.

If a library developer knows how to handle certain case (i.e. recover or throw a special error) I'm ok with it. Though I'm not ok with completely hiding errors from the underlying implementation. The library developer might not handle EVERYTHING. I want to know what happened and where. Otherwise I will have to explicitly debug his library to find out what the issue is. In most cases I will not have time to wait developer fixing the library. I will fix it myself and make PR. So I consider tranparent errors rather a benefit.

Hiding errors under library errors might have worked for proprietary middleware. Today the world changed. It's open source now.

I agree with you that having *access* to the original error is important and that a library should not be expected to handle every case. I hope I have not said anything that suggests otherwise.

What I’m suggesting is that if the error originated in a dependency that might change it should be wrapped in a stable abstraction that the users of the library can depend on. The underlying error should still be available via a property or associated value exposed with an existential type. If you expect users to catch errors thrown by a dependency that might change you have a fragile API contract in the area of errors.

This can easily lead to unreliable software - when you change the dependency nothing alerts users to the fact that their error handling code is probably broken. Even if the compiler told them it was broken, they now have the burden of learning about the errors that might come from your new dependency. An API that could lead to this consequence is badly designed IMO.

I even suggested adding a new requirement to `Error` that includes a default implementation to standardize how we access wrapped errors, including the originating error underneath all wrapper layers.

My point is that I'm completely against typed throws. I'm sure there will be a lot of developers who would try to use the feature to hide underlying errors under their own. I can't consider it a good practice (reasons above). This is how good intentions (caring about API and library users) can lead to bad consequences. Sounds good in theory - works bad in practice (in real, non ideal world).

What is your opinion about the problem I describe above? Do you think this is unimportant?

On Tue, 28 Feb 2017 at 18:18 Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Feb 28, 2017, at 5:09 AM, Daniel Leping <daniel@crossroadlabs.xyz <mailto:daniel@crossroadlabs.xyz>> wrote:

When you're going to present a user with specific error message you don't do it for all errors. First of all at the end of a call chain you will have a couple of dozens of possible errors and only 2 or 3 will do really matter for the end user. Would you list all the dozen in params? Probably not. You'll just catch what matters and put the rest to the log with a default "sorry" message.

What I mean is it's OK to be "unaware" why everything failed. What really does matter is what you can deal with and what you can't (the rest). "The rest" is usually a magnitude bigger, though.

I might agree, that listing some errors in the throws clause is useful feature for documentation (even so, I think the docs should be in the comments), but only and only if it can throw the listed errors + a bunch of other stuff. In Java it's resolved with a terrible solution of RuntimeError interface which I definitely suggest to avoid.

As a library developer I can say that even for a library it's a nightmare and libraries will start wrapping system errors with their own if the trend would become to list errors in middleware. My preference though is to have all errors transparently passed in both roles, being a library developer and a library user.

Whether to wrap or not is a judgment call.

If the purpose of your library is to simply provide a cleaner experience for a system API that is a stable dependency that will not change. Users might expect to have the errors from the system API forwarded directly and that would be perfectly appropriate.

In other cases you might have dependencies that are considered implementation details. In that case if you don’t wrap the errors you’re doing users a disservice by giving them an unstable interface for error handling. Changing out your dependency that is supposed to be an implementation detail becomes a breaking change.

On Tue, 28 Feb 2017 at 8:29 Karl Wagner <razielim@gmail.com <mailto:razielim@gmail.com>> wrote:

On 28 Feb 2017, at 02:43, Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Sent from my iPad

On Feb 27, 2017, at 7:19 PM, Daniel Leping <daniel@crossroadlabs.xyz <mailto:daniel@crossroadlabs.xyz>> wrote:

Well, as Dave pointed, you can very rarely recover from an error, which IMO is absolutely true.

If your operation fails you don't really care unless you can recover. And you know your cases, which you can recover from (in reality one usually does it in optimization phase, though).

What else do you need the type of error for at the very end of your call stack? In 90% you will tell the user "ah, sorry, something happened. Come back later" and log the error (if you haven't forgot it).

In most cases the errors are not for recovering. They neither are to be presented to users. They are for developers to read the log/crash report/whatever else and analyze it. Most of the errors are for debugging purposes.

I don't want to deal with cumbersome code the purpose of which is to just "obey the language rules". Unless I know how to recover I would rethrow it. Than catch at the top of the stack and log + show the user a nice "sorry" message without getting techy.

In order to provide a helpful experience to end users an app needs to know about what might have caused the error and therefore what might (or might not) resolve it, allowing the operation to succeed on a subsequent attempt. This can influence the strategy an app uses to avoid bothering the user if it might be resolvable without user intervention and can also influence the content of the message presented to users if that is necessary.

Errors are certainly useful, but the key distinction (IMO) is that callers should never rely on an error happening or not. They are typically highly implementation and hence version-specific. I think that means specific errors shouldn’t be part of the function signature.

That doesn’t mean you can’t handle the error. A function that throws is one that reserves the right to fail; maybe you can resolve the problem, but maybe you can’t. That means your enclosing operation also needs to be prepared to fail for reasons beyond _its_ control, and so on. At some point your entire UI-level sub-operation (like opening the file, downloading the data from the network) just fails, and you decide whether or not the user needs to be told of that and how to do so.

Indeed, many apps don't bother with this kind of detail and just treat all errors the same. Personally, I find that to be an unfortunate state of affairs both as a user and as a developer.

Types can be useful in conveying this kind of information and abstracting low level details that are not helpful at higher levels in the stack. Of course types are not the only way to convey this information. But my experience is that untyped errors often result in libraries with poor documentation of error cases and not enough consideration of the error handling experience of users of the library in general. That makes it very difficult to handle errors well. I have experiences like this in several languages and on several platforms, including Apple's.

I’m not sure that types are really the best abstraction for “the list of errors that can be thrown by this function”. They’re fine for error themselves, but not for per-function error-lists IMO. Most of our types are way too rigid for this to be easy to live with.

If I understand what you’re saying, the core problem can be summarised as: "I don’t know/can’t easily communicate which errors this function throws”. Really, it is a documentation problem; it's too onerous to document every individual throwing function, even with our very pretty markdown-like syntax. I’m just not sure that the relatively rigid type-system is the solution.

Now, if we look at this from a documentation perspective: obviously, the compiler _typically_ can't generate documentation for you, because it doesn’t know what your types/functions are meant for. Error documentation, though, is different: the compiler *can* often know the specific errors which get thrown (or rethrown from a call to another function from whom it can get that information); and even when it can’t know that, often it can at least know the specific type of error. We could enhance the compiler libraries to track that; I spent half a day modifying the existing error-checker in Sema to prove the concept and it’s close.

Swift libraries don’t have header files, so third-parties will only ever see your documentation as generated by some tool which integrates the compiler. That means the compiler can “improve” the documentation when it is lacking. For yourself, if you care about explicitly writing out the errors which get thrown and you’re not working with an IDE, you could enable compiler-checking of your documentation comments. We can invent new shorthands to make it easier to list all of your various errors (individual cases, entire types, and maybe some private “other errors”), isolated from the rest of the language. For example:

/// - throws: MyError.{errorOne, errorThree}, AnotherError.*, *
func doSomething() throws

Maybe you can think of other ways we could help the documentation problem from the compiler side?

Typed errors are certainly no panacea but I believe they can be an important tool. Experience in other languages has shown that to be the case for many people writing many kinds of software.

If you don't believe typed errors will improve your code or if you just don't want to deal with typed errors, just don't use them! You will be able to continue using untyped throws just as you do today. You can even do this if you use libraries that throw typed errors.

If it’s the wrong solution to the problem, it does make the language worse overall.

On Tue, 28 Feb 2017 at 1:12 Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

> On Feb 27, 2017, at 5:01 PM, Dave Abrahams <dabrahams@apple.com <mailto:dabrahams@apple.com>> wrote:
>
>
> on Mon Feb 27 2017, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/>> wrote:
>
>>> On Feb 27, 2017, at 4:20 PM, Dave Abrahams <dabrahams@apple.com <mailto:dabrahams@apple.com>> wrote:
>>>
>>>
>>> I'm sorry, I don't see any substantive difference, based on what you've
>>> written here, between this feature and const.
>>
>> Let me give it one more shot and then I’ll drop it. :)
>>
>> Const is viral because if an API does not declare its arguments const
>> it cannot be used by a caller who has a const argument.
>
> Unless the caller casts away const, thus erasing information that was
> previously encoded in the type system.

Yes, of course.

>
>> It is required in order to make an API as generally useful as
>> possible.
>>
>> Typed errors are not viral in this way because no callers are
>> prevented from calling an API regardless of whether it declares error
>> types or just throws Error like we have today.
>
> Unless the caller can recover (which is *very* rare) or it catches and
> rethrows one of the errors *it* declares, thus erasing information that
> was previously encoded in the type system.

I view this as being fundamentally different than casting away const. Casting away const says “I know better than the types”.

Converting an error to a different type is extremely different. It much more similar to any other kind of value wrapper a library might create in order to shield its users from being coupled to its implementation details / dependencies. This is not a way *around* the type system in the sense that casting away const is. It is a way of *using* the type system (hopefully) to your advantage.

>
>> Pressure to declare error types in your signature in order to make
>> your function as generally useful as possible does not exist. Each
>> function is free to declare error types or not according to the
>> contract it wishes to expose.
>>
>> An argument can be made that community expectations might develop that
>> good APIs should declare error types and they could be considered
>> viral in this sense because any API that is simply declared `throws`
>> is dropping type information. But I think this overstates the case.
>> The community appears to be very sensitive to the problems that can
>> arise when error types are too concrete, especially across module
>> boundaries. I think we can learn to use the tool where it works well
>> and to avoid it where it causes problems.
>>
>>>
>>> --
>>> -Dave
>>
>
> --
> -Dave

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

anandabits · February 28, 2017, 5:00pm

Mathew, I totally understand what you're talking about. And this is exactly what I want to avoid.

If a library developer knows how to handle certain case (i.e. recover or throw a special error) I'm ok with it. Though I'm not ok with completely hiding errors from the underlying implementation. The library developer might not handle EVERYTHING. I want to know what happened and where. Otherwise I will have to explicitly debug his library to find out what the issue is. In most cases I will not have time to wait developer fixing the library. I will fix it myself and make PR. So I consider tranparent errors rather a benefit.

Hiding errors under library errors might have worked for proprietary middleware. Today the world changed. It's open source now.

I agree with you that having *access* to the original error is important and that a library should not be expected to handle every case. I hope I have not said anything that suggests otherwise.

What I’m suggesting is that if the error originated in a dependency that might change it should be wrapped in a stable abstraction that the users of the library can depend on. The underlying error should still be available via a property or associated value exposed with an existential type. If you expect users to catch errors thrown by a dependency that might change you have a fragile API contract in the area of errors.

This can easily lead to unreliable software - when you change the dependency nothing alerts users to the fact that their error handling code is probably broken. Even if the compiler told them it was broken, they now have the burden of learning about the errors that might come from your new dependency. An API that could lead to this consequence is badly designed IMO.

I even suggested adding a new requirement to `Error` that includes a default implementation to standardize how we access wrapped errors, including the originating error underneath all wrapper layers.

My point is that I'm completely against typed throws. I'm sure there will be a lot of developers who would try to use the feature to hide underlying errors under their own. I can't consider it a good practice (reasons above). This is how good intentions (caring about API and library users) can lead to bad consequences. Sounds good in theory - works bad in practice (in real, non ideal world).

What is your opinion about the problem I describe above? Do you think this is unimportant?

···

On Feb 28, 2017, at 10:44 AM, Daniel Leping <daniel@crossroadlabs.xyz> wrote:

On Tue, 28 Feb 2017 at 18:18 Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Feb 28, 2017, at 5:09 AM, Daniel Leping <daniel@crossroadlabs.xyz <mailto:daniel@crossroadlabs.xyz>> wrote:

When you're going to present a user with specific error message you don't do it for all errors. First of all at the end of a call chain you will have a couple of dozens of possible errors and only 2 or 3 will do really matter for the end user. Would you list all the dozen in params? Probably not. You'll just catch what matters and put the rest to the log with a default "sorry" message.

What I mean is it's OK to be "unaware" why everything failed. What really does matter is what you can deal with and what you can't (the rest). "The rest" is usually a magnitude bigger, though.

I might agree, that listing some errors in the throws clause is useful feature for documentation (even so, I think the docs should be in the comments), but only and only if it can throw the listed errors + a bunch of other stuff. In Java it's resolved with a terrible solution of RuntimeError interface which I definitely suggest to avoid.

As a library developer I can say that even for a library it's a nightmare and libraries will start wrapping system errors with their own if the trend would become to list errors in middleware. My preference though is to have all errors transparently passed in both roles, being a library developer and a library user.

Whether to wrap or not is a judgment call.

If the purpose of your library is to simply provide a cleaner experience for a system API that is a stable dependency that will not change. Users might expect to have the errors from the system API forwarded directly and that would be perfectly appropriate.

In other cases you might have dependencies that are considered implementation details. In that case if you don’t wrap the errors you’re doing users a disservice by giving them an unstable interface for error handling. Changing out your dependency that is supposed to be an implementation detail becomes a breaking change.

On Tue, 28 Feb 2017 at 8:29 Karl Wagner <razielim@gmail.com <mailto:razielim@gmail.com>> wrote:

On 28 Feb 2017, at 02:43, Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Sent from my iPad

On Feb 27, 2017, at 7:19 PM, Daniel Leping <daniel@crossroadlabs.xyz <mailto:daniel@crossroadlabs.xyz>> wrote:

Well, as Dave pointed, you can very rarely recover from an error, which IMO is absolutely true.

If your operation fails you don't really care unless you can recover. And you know your cases, which you can recover from (in reality one usually does it in optimization phase, though).

What else do you need the type of error for at the very end of your call stack? In 90% you will tell the user "ah, sorry, something happened. Come back later" and log the error (if you haven't forgot it).

In most cases the errors are not for recovering. They neither are to be presented to users. They are for developers to read the log/crash report/whatever else and analyze it. Most of the errors are for debugging purposes.

I don't want to deal with cumbersome code the purpose of which is to just "obey the language rules". Unless I know how to recover I would rethrow it. Than catch at the top of the stack and log + show the user a nice "sorry" message without getting techy.

In order to provide a helpful experience to end users an app needs to know about what might have caused the error and therefore what might (or might not) resolve it, allowing the operation to succeed on a subsequent attempt. This can influence the strategy an app uses to avoid bothering the user if it might be resolvable without user intervention and can also influence the content of the message presented to users if that is necessary.

Errors are certainly useful, but the key distinction (IMO) is that callers should never rely on an error happening or not. They are typically highly implementation and hence version-specific. I think that means specific errors shouldn’t be part of the function signature.

That doesn’t mean you can’t handle the error. A function that throws is one that reserves the right to fail; maybe you can resolve the problem, but maybe you can’t. That means your enclosing operation also needs to be prepared to fail for reasons beyond _its_ control, and so on. At some point your entire UI-level sub-operation (like opening the file, downloading the data from the network) just fails, and you decide whether or not the user needs to be told of that and how to do so.

Indeed, many apps don't bother with this kind of detail and just treat all errors the same. Personally, I find that to be an unfortunate state of affairs both as a user and as a developer.

Types can be useful in conveying this kind of information and abstracting low level details that are not helpful at higher levels in the stack. Of course types are not the only way to convey this information. But my experience is that untyped errors often result in libraries with poor documentation of error cases and not enough consideration of the error handling experience of users of the library in general. That makes it very difficult to handle errors well. I have experiences like this in several languages and on several platforms, including Apple's.

I’m not sure that types are really the best abstraction for “the list of errors that can be thrown by this function”. They’re fine for error themselves, but not for per-function error-lists IMO. Most of our types are way too rigid for this to be easy to live with.

If I understand what you’re saying, the core problem can be summarised as: "I don’t know/can’t easily communicate which errors this function throws”. Really, it is a documentation problem; it's too onerous to document every individual throwing function, even with our very pretty markdown-like syntax. I’m just not sure that the relatively rigid type-system is the solution.

Now, if we look at this from a documentation perspective: obviously, the compiler _typically_ can't generate documentation for you, because it doesn’t know what your types/functions are meant for. Error documentation, though, is different: the compiler *can* often know the specific errors which get thrown (or rethrown from a call to another function from whom it can get that information); and even when it can’t know that, often it can at least know the specific type of error. We could enhance the compiler libraries to track that; I spent half a day modifying the existing error-checker in Sema to prove the concept and it’s close.

Swift libraries don’t have header files, so third-parties will only ever see your documentation as generated by some tool which integrates the compiler. That means the compiler can “improve” the documentation when it is lacking. For yourself, if you care about explicitly writing out the errors which get thrown and you’re not working with an IDE, you could enable compiler-checking of your documentation comments. We can invent new shorthands to make it easier to list all of your various errors (individual cases, entire types, and maybe some private “other errors”), isolated from the rest of the language. For example:

/// - throws: MyError.{errorOne, errorThree}, AnotherError.*, *
func doSomething() throws

Maybe you can think of other ways we could help the documentation problem from the compiler side?

Typed errors are certainly no panacea but I believe they can be an important tool. Experience in other languages has shown that to be the case for many people writing many kinds of software.

If you don't believe typed errors will improve your code or if you just don't want to deal with typed errors, just don't use them! You will be able to continue using untyped throws just as you do today. You can even do this if you use libraries that throw typed errors.

If it’s the wrong solution to the problem, it does make the language worse overall.

On Tue, 28 Feb 2017 at 1:12 Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

> On Feb 27, 2017, at 5:01 PM, Dave Abrahams <dabrahams@apple.com <mailto:dabrahams@apple.com>> wrote:
>
>
> on Mon Feb 27 2017, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/>> wrote:
>
>>> On Feb 27, 2017, at 4:20 PM, Dave Abrahams <dabrahams@apple.com <mailto:dabrahams@apple.com>> wrote:
>>>
>>>
>>> I'm sorry, I don't see any substantive difference, based on what you've
>>> written here, between this feature and const.
>>
>> Let me give it one more shot and then I’ll drop it. :)
>>
>> Const is viral because if an API does not declare its arguments const
>> it cannot be used by a caller who has a const argument.
>
> Unless the caller casts away const, thus erasing information that was
> previously encoded in the type system.

Yes, of course.

>
>> It is required in order to make an API as generally useful as
>> possible.
>>
>> Typed errors are not viral in this way because no callers are
>> prevented from calling an API regardless of whether it declares error
>> types or just throws Error like we have today.
>
> Unless the caller can recover (which is *very* rare) or it catches and
> rethrows one of the errors *it* declares, thus erasing information that
> was previously encoded in the type system.

I view this as being fundamentally different than casting away const. Casting away const says “I know better than the types”.

Converting an error to a different type is extremely different. It much more similar to any other kind of value wrapper a library might create in order to shield its users from being coupled to its implementation details / dependencies. This is not a way *around* the type system in the sense that casting away const is. It is a way of *using* the type system (hopefully) to your advantage.

>
>> Pressure to declare error types in your signature in order to make
>> your function as generally useful as possible does not exist. Each
>> function is free to declare error types or not according to the
>> contract it wishes to expose.
>>
>> An argument can be made that community expectations might develop that
>> good APIs should declare error types and they could be considered
>> viral in this sense because any API that is simply declared `throws`
>> is dropping type information. But I think this overstates the case.
>> The community appears to be very sensitive to the problems that can
>> arise when error types are too concrete, especially across module
>> boundaries. I think we can learn to use the tool where it works well
>> and to avoid it where it causes problems.
>>
>>>
>>> --
>>> -Dave
>>
>
> --
> -Dave

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

anandabits · February 28, 2017, 5:42pm

Matthew, Per api fragility, the only API fragility would likely be adding a new error to the signature, which isn’t that bad. Any significant error changes, would probably mirror a significant change in that libraries pattern in and of itself, which would lend to a Semver.Major version bump anyhow.

Not at all. If I use library A to do something and switch to library B the errors reported may be significantly different. The point is not that you just need to add library B’s error types to your signature at all. The point is that it is better to allow callers to handle the errors based on the general semantics of the cause of failure and not the specific type that a dependency threw. You simply cannot do this without wrapping the errors.

It looks the consensus here is:
Typed throws can be good, but wrapping kills dev cycles, and these types can pollute our space.

The best answer to this is adding implicit conversion during propagation and enhancing the `Error` protocol to facilitate access to the underlying (wrapped) error.

To which, I think the obvious answer here is:
Typed throws with inferred signatures.

I disagree. Wrapping requires human judgement and intentional design. There is very little value in just saying “here’s all the concrete error types my current implementation might throw”.

For the compiler to infer throw signatures is easily accessible and trivial to accomplish. Xcode can even display the inferred signatures. The compiler only need to add any `throw err` and `try erringFunc()` results to the throw sig.

Implementing this gives us two gains at little cost:
1) Error sets are now ready available to use, but we can still ignore if we please!
2) No catch Error, unless we *explicitly* want a catch all

But this eliminates the human judgement that is required. A well designed error propagation mechanism is a much better solution to allowing us to avoid the need to manually catch, wrap, and throw the errors we are propagating. I *don’t want* the concrete error types exposed automatically. That would *increase* the kind of fragile coupling that everyone has pointed out is a problem.

···

On Feb 28, 2017, at 11:17 AM, Zaid Daghestani <zaid@arkverse.com> wrote:

I honestly believe freedom is key when it comes to error implementation. Environment usually dictates whether we fully implement error flows or not. With these, we still maintain our wonderfully fast iteration cycles by ignoring/catchall, but when we do implement errors, we get to fully leverage it.

Everyone wins!

Cheers,
Z

P.S. Stack traces in errors would be awesome.

On Feb 28, 2017, at 9:00 AM, Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Feb 28, 2017, at 10:44 AM, Daniel Leping <daniel@crossroadlabs.xyz <mailto:daniel@crossroadlabs.xyz>> wrote:

Mathew, I totally understand what you're talking about. And this is exactly what I want to avoid.

If a library developer knows how to handle certain case (i.e. recover or throw a special error) I'm ok with it. Though I'm not ok with completely hiding errors from the underlying implementation. The library developer might not handle EVERYTHING. I want to know what happened and where. Otherwise I will have to explicitly debug his library to find out what the issue is. In most cases I will not have time to wait developer fixing the library. I will fix it myself and make PR. So I consider tranparent errors rather a benefit.

Hiding errors under library errors might have worked for proprietary middleware. Today the world changed. It's open source now.

I agree with you that having *access* to the original error is important and that a library should not be expected to handle every case. I hope I have not said anything that suggests otherwise.

What I’m suggesting is that if the error originated in a dependency that might change it should be wrapped in a stable abstraction that the users of the library can depend on. The underlying error should still be available via a property or associated value exposed with an existential type. If you expect users to catch errors thrown by a dependency that might change you have a fragile API contract in the area of errors.

This can easily lead to unreliable software - when you change the dependency nothing alerts users to the fact that their error handling code is probably broken. Even if the compiler told them it was broken, they now have the burden of learning about the errors that might come from your new dependency. An API that could lead to this consequence is badly designed IMO.

I even suggested adding a new requirement to `Error` that includes a default implementation to standardize how we access wrapped errors, including the originating error underneath all wrapper layers.

My point is that I'm completely against typed throws. I'm sure there will be a lot of developers who would try to use the feature to hide underlying errors under their own. I can't consider it a good practice (reasons above). This is how good intentions (caring about API and library users) can lead to bad consequences. Sounds good in theory - works bad in practice (in real, non ideal world).

What is your opinion about the problem I describe above? Do you think this is unimportant?

On Tue, 28 Feb 2017 at 18:18 Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Feb 28, 2017, at 5:09 AM, Daniel Leping <daniel@crossroadlabs.xyz <mailto:daniel@crossroadlabs.xyz>> wrote:

When you're going to present a user with specific error message you don't do it for all errors. First of all at the end of a call chain you will have a couple of dozens of possible errors and only 2 or 3 will do really matter for the end user. Would you list all the dozen in params? Probably not. You'll just catch what matters and put the rest to the log with a default "sorry" message.

What I mean is it's OK to be "unaware" why everything failed. What really does matter is what you can deal with and what you can't (the rest). "The rest" is usually a magnitude bigger, though.

I might agree, that listing some errors in the throws clause is useful feature for documentation (even so, I think the docs should be in the comments), but only and only if it can throw the listed errors + a bunch of other stuff. In Java it's resolved with a terrible solution of RuntimeError interface which I definitely suggest to avoid.

As a library developer I can say that even for a library it's a nightmare and libraries will start wrapping system errors with their own if the trend would become to list errors in middleware. My preference though is to have all errors transparently passed in both roles, being a library developer and a library user.

Whether to wrap or not is a judgment call.

If the purpose of your library is to simply provide a cleaner experience for a system API that is a stable dependency that will not change. Users might expect to have the errors from the system API forwarded directly and that would be perfectly appropriate.

In other cases you might have dependencies that are considered implementation details. In that case if you don’t wrap the errors you’re doing users a disservice by giving them an unstable interface for error handling. Changing out your dependency that is supposed to be an implementation detail becomes a breaking change.

On Tue, 28 Feb 2017 at 8:29 Karl Wagner <razielim@gmail.com <mailto:razielim@gmail.com>> wrote:

On 28 Feb 2017, at 02:43, Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Sent from my iPad

On Feb 27, 2017, at 7:19 PM, Daniel Leping <daniel@crossroadlabs.xyz <mailto:daniel@crossroadlabs.xyz>> wrote:

Well, as Dave pointed, you can very rarely recover from an error, which IMO is absolutely true.

If your operation fails you don't really care unless you can recover. And you know your cases, which you can recover from (in reality one usually does it in optimization phase, though).

What else do you need the type of error for at the very end of your call stack? In 90% you will tell the user "ah, sorry, something happened. Come back later" and log the error (if you haven't forgot it).

In most cases the errors are not for recovering. They neither are to be presented to users. They are for developers to read the log/crash report/whatever else and analyze it. Most of the errors are for debugging purposes.

I don't want to deal with cumbersome code the purpose of which is to just "obey the language rules". Unless I know how to recover I would rethrow it. Than catch at the top of the stack and log + show the user a nice "sorry" message without getting techy.

In order to provide a helpful experience to end users an app needs to know about what might have caused the error and therefore what might (or might not) resolve it, allowing the operation to succeed on a subsequent attempt. This can influence the strategy an app uses to avoid bothering the user if it might be resolvable without user intervention and can also influence the content of the message presented to users if that is necessary.

Errors are certainly useful, but the key distinction (IMO) is that callers should never rely on an error happening or not. They are typically highly implementation and hence version-specific. I think that means specific errors shouldn’t be part of the function signature.

That doesn’t mean you can’t handle the error. A function that throws is one that reserves the right to fail; maybe you can resolve the problem, but maybe you can’t. That means your enclosing operation also needs to be prepared to fail for reasons beyond _its_ control, and so on. At some point your entire UI-level sub-operation (like opening the file, downloading the data from the network) just fails, and you decide whether or not the user needs to be told of that and how to do so.

Indeed, many apps don't bother with this kind of detail and just treat all errors the same. Personally, I find that to be an unfortunate state of affairs both as a user and as a developer.

Types can be useful in conveying this kind of information and abstracting low level details that are not helpful at higher levels in the stack. Of course types are not the only way to convey this information. But my experience is that untyped errors often result in libraries with poor documentation of error cases and not enough consideration of the error handling experience of users of the library in general. That makes it very difficult to handle errors well. I have experiences like this in several languages and on several platforms, including Apple's.

I’m not sure that types are really the best abstraction for “the list of errors that can be thrown by this function”. They’re fine for error themselves, but not for per-function error-lists IMO. Most of our types are way too rigid for this to be easy to live with.

If I understand what you’re saying, the core problem can be summarised as: "I don’t know/can’t easily communicate which errors this function throws”. Really, it is a documentation problem; it's too onerous to document every individual throwing function, even with our very pretty markdown-like syntax. I’m just not sure that the relatively rigid type-system is the solution.

Now, if we look at this from a documentation perspective: obviously, the compiler _typically_ can't generate documentation for you, because it doesn’t know what your types/functions are meant for. Error documentation, though, is different: the compiler *can* often know the specific errors which get thrown (or rethrown from a call to another function from whom it can get that information); and even when it can’t know that, often it can at least know the specific type of error. We could enhance the compiler libraries to track that; I spent half a day modifying the existing error-checker in Sema to prove the concept and it’s close.

Swift libraries don’t have header files, so third-parties will only ever see your documentation as generated by some tool which integrates the compiler. That means the compiler can “improve” the documentation when it is lacking. For yourself, if you care about explicitly writing out the errors which get thrown and you’re not working with an IDE, you could enable compiler-checking of your documentation comments. We can invent new shorthands to make it easier to list all of your various errors (individual cases, entire types, and maybe some private “other errors”), isolated from the rest of the language. For example:

/// - throws: MyError.{errorOne, errorThree}, AnotherError.*, *
func doSomething() throws

Maybe you can think of other ways we could help the documentation problem from the compiler side?

Typed errors are certainly no panacea but I believe they can be an important tool. Experience in other languages has shown that to be the case for many people writing many kinds of software.

If you don't believe typed errors will improve your code or if you just don't want to deal with typed errors, just don't use them! You will be able to continue using untyped throws just as you do today. You can even do this if you use libraries that throw typed errors.

If it’s the wrong solution to the problem, it does make the language worse overall.

On Tue, 28 Feb 2017 at 1:12 Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

> On Feb 27, 2017, at 5:01 PM, Dave Abrahams <dabrahams@apple.com <mailto:dabrahams@apple.com>> wrote:
>
>
> on Mon Feb 27 2017, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/>> wrote:
>
>>> On Feb 27, 2017, at 4:20 PM, Dave Abrahams <dabrahams@apple.com <mailto:dabrahams@apple.com>> wrote:
>>>
>>>
>>> I'm sorry, I don't see any substantive difference, based on what you've
>>> written here, between this feature and const.
>>
>> Let me give it one more shot and then I’ll drop it. :)
>>
>> Const is viral because if an API does not declare its arguments const
>> it cannot be used by a caller who has a const argument.
>
> Unless the caller casts away const, thus erasing information that was
> previously encoded in the type system.

Yes, of course.

>
>> It is required in order to make an API as generally useful as
>> possible.
>>
>> Typed errors are not viral in this way because no callers are
>> prevented from calling an API regardless of whether it declares error
>> types or just throws Error like we have today.
>
> Unless the caller can recover (which is *very* rare) or it catches and
> rethrows one of the errors *it* declares, thus erasing information that
> was previously encoded in the type system.

I view this as being fundamentally different than casting away const. Casting away const says “I know better than the types”.

Converting an error to a different type is extremely different. It much more similar to any other kind of value wrapper a library might create in order to shield its users from being coupled to its implementation details / dependencies. This is not a way *around* the type system in the sense that casting away const is. It is a way of *using* the type system (hopefully) to your advantage.

>
>> Pressure to declare error types in your signature in order to make
>> your function as generally useful as possible does not exist. Each
>> function is free to declare error types or not according to the
>> contract it wishes to expose.
>>
>> An argument can be made that community expectations might develop that
>> good APIs should declare error types and they could be considered
>> viral in this sense because any API that is simply declared `throws`
>> is dropping type information. But I think this overstates the case.
>> The community appears to be very sensitive to the problems that can
>> arise when error types are too concrete, especially across module
>> boundaries. I think we can learn to use the tool where it works well
>> and to avoid it where it causes problems.
>>
>>>
>>> --
>>> -Dave
>>
>
> --
> -Dave

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

Karl · February 28, 2017, 5:55pm

Mathew, I totally understand what you're talking about. And this is exactly what I want to avoid.

If a library developer knows how to handle certain case (i.e. recover or throw a special error) I'm ok with it. Though I'm not ok with completely hiding errors from the underlying implementation. The library developer might not handle EVERYTHING. I want to know what happened and where. Otherwise I will have to explicitly debug his library to find out what the issue is. In most cases I will not have time to wait developer fixing the library. I will fix it myself and make PR. So I consider tranparent errors rather a benefit.

Hiding errors under library errors might have worked for proprietary middleware. Today the world changed. It's open source now.

I agree with you that having *access* to the original error is important and that a library should not be expected to handle every case. I hope I have not said anything that suggests otherwise.

What I’m suggesting is that if the error originated in a dependency that might change it should be wrapped in a stable abstraction that the users of the library can depend on. The underlying error should still be available via a property or associated value exposed with an existential type. If you expect users to catch errors thrown by a dependency that might change you have a fragile API contract in the area of errors.

This can easily lead to unreliable software - when you change the dependency nothing alerts users to the fact that their error handling code is probably broken. Even if the compiler told them it was broken, they now have the burden of learning about the errors that might come from your new dependency. An API that could lead to this consequence is badly designed IMO.

I even suggested adding a new requirement to `Error` that includes a default implementation to standardize how we access wrapped errors, including the originating error underneath all wrapper layers.

I don’t see that it’s such a big problem. There are lots of ways to expose the information once we have it, and it could easily be limited to public errors from the current module (substituting private/unbound errors with an erasing “Error”). There are lots of small improvements we could make: such as having functions which re-throw errors from other, manually documented functions inherit their documentation (or at least appear to, when viewed). That would not be totally compiler-generated documentation, but would mean you can document your errors in one place and make doing so much more palatable. :

// Manual documentation in one place:

/// - throws: X, Y, Z
func doSomething() throws { … }

// Inherited by functions which call it and rethrow its errors:

/// [generated]: - throws: X, Y, X
func doSomethingExtended() throws {
try doSomething()
}

/// [generated]: - throws: X, Y, Z
func doSomethingMapped() throws {
// (call to rethrowing function ‘map’) -> (throwing closure expression) -> (call to throwing function “doSomething”, maybe others, too...) -> X, Y, Z
try [1, 2, 3].map { try doSomething() }
}

Personally, I don’t like wrapping. I like error-handling to be flat at each level because it’s much easier to reason about. You would basically need catch on "error as? MyError” and “error.underlyingError as? MyError”, duplicating code or forwarding them to a refactored handler function. Also, consider that an Error might be any type at all (including a class). If any instance of a thing which conforms to Error could potentially be holding a reference-counted pointer, that could limit optimisations. If it’s flat, we can potentially prove that it has an exact type (or at least is never reference-counted), and that could be valuable.

Error-contracts in libraries are only really fixed in the case of an implementation which is never going to change substantially. In that case, the function is a possible candidate for @inlineable and can be treated as though part of the client’s module, with full error-flow intelligence.

- Karl

···

On 28 Feb 2017, at 18:00, Matthew Johnson <matthew@anandabits.com> wrote:

On Feb 28, 2017, at 10:44 AM, Daniel Leping <daniel@crossroadlabs.xyz <mailto:daniel@crossroadlabs.xyz>> wrote:

My point is that I'm completely against typed throws. I'm sure there will be a lot of developers who would try to use the feature to hide underlying errors under their own. I can't consider it a good practice (reasons above). This is how good intentions (caring about API and library users) can lead to bad consequences. Sounds good in theory - works bad in practice (in real, non ideal world).

What is your opinion about the problem I describe above? Do you think this is unimportant?

On Tue, 28 Feb 2017 at 18:18 Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Feb 28, 2017, at 5:09 AM, Daniel Leping <daniel@crossroadlabs.xyz <mailto:daniel@crossroadlabs.xyz>> wrote:

When you're going to present a user with specific error message you don't do it for all errors. First of all at the end of a call chain you will have a couple of dozens of possible errors and only 2 or 3 will do really matter for the end user. Would you list all the dozen in params? Probably not. You'll just catch what matters and put the rest to the log with a default "sorry" message.

What I mean is it's OK to be "unaware" why everything failed. What really does matter is what you can deal with and what you can't (the rest). "The rest" is usually a magnitude bigger, though.

I might agree, that listing some errors in the throws clause is useful feature for documentation (even so, I think the docs should be in the comments), but only and only if it can throw the listed errors + a bunch of other stuff. In Java it's resolved with a terrible solution of RuntimeError interface which I definitely suggest to avoid.

As a library developer I can say that even for a library it's a nightmare and libraries will start wrapping system errors with their own if the trend would become to list errors in middleware. My preference though is to have all errors transparently passed in both roles, being a library developer and a library user.

Whether to wrap or not is a judgment call.

If the purpose of your library is to simply provide a cleaner experience for a system API that is a stable dependency that will not change. Users might expect to have the errors from the system API forwarded directly and that would be perfectly appropriate.

In other cases you might have dependencies that are considered implementation details. In that case if you don’t wrap the errors you’re doing users a disservice by giving them an unstable interface for error handling. Changing out your dependency that is supposed to be an implementation detail becomes a breaking change.

On Tue, 28 Feb 2017 at 8:29 Karl Wagner <razielim@gmail.com <mailto:razielim@gmail.com>> wrote:

On 28 Feb 2017, at 02:43, Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Sent from my iPad

On Feb 27, 2017, at 7:19 PM, Daniel Leping <daniel@crossroadlabs.xyz <mailto:daniel@crossroadlabs.xyz>> wrote:

Well, as Dave pointed, you can very rarely recover from an error, which IMO is absolutely true.

If your operation fails you don't really care unless you can recover. And you know your cases, which you can recover from (in reality one usually does it in optimization phase, though).

What else do you need the type of error for at the very end of your call stack? In 90% you will tell the user "ah, sorry, something happened. Come back later" and log the error (if you haven't forgot it).

In most cases the errors are not for recovering. They neither are to be presented to users. They are for developers to read the log/crash report/whatever else and analyze it. Most of the errors are for debugging purposes.

I don't want to deal with cumbersome code the purpose of which is to just "obey the language rules". Unless I know how to recover I would rethrow it. Than catch at the top of the stack and log + show the user a nice "sorry" message without getting techy.

In order to provide a helpful experience to end users an app needs to know about what might have caused the error and therefore what might (or might not) resolve it, allowing the operation to succeed on a subsequent attempt. This can influence the strategy an app uses to avoid bothering the user if it might be resolvable without user intervention and can also influence the content of the message presented to users if that is necessary.

Errors are certainly useful, but the key distinction (IMO) is that callers should never rely on an error happening or not. They are typically highly implementation and hence version-specific. I think that means specific errors shouldn’t be part of the function signature.

That doesn’t mean you can’t handle the error. A function that throws is one that reserves the right to fail; maybe you can resolve the problem, but maybe you can’t. That means your enclosing operation also needs to be prepared to fail for reasons beyond _its_ control, and so on. At some point your entire UI-level sub-operation (like opening the file, downloading the data from the network) just fails, and you decide whether or not the user needs to be told of that and how to do so.

Indeed, many apps don't bother with this kind of detail and just treat all errors the same. Personally, I find that to be an unfortunate state of affairs both as a user and as a developer.

Types can be useful in conveying this kind of information and abstracting low level details that are not helpful at higher levels in the stack. Of course types are not the only way to convey this information. But my experience is that untyped errors often result in libraries with poor documentation of error cases and not enough consideration of the error handling experience of users of the library in general. That makes it very difficult to handle errors well. I have experiences like this in several languages and on several platforms, including Apple's.

I’m not sure that types are really the best abstraction for “the list of errors that can be thrown by this function”. They’re fine for error themselves, but not for per-function error-lists IMO. Most of our types are way too rigid for this to be easy to live with.

If I understand what you’re saying, the core problem can be summarised as: "I don’t know/can’t easily communicate which errors this function throws”. Really, it is a documentation problem; it's too onerous to document every individual throwing function, even with our very pretty markdown-like syntax. I’m just not sure that the relatively rigid type-system is the solution.

Now, if we look at this from a documentation perspective: obviously, the compiler _typically_ can't generate documentation for you, because it doesn’t know what your types/functions are meant for. Error documentation, though, is different: the compiler *can* often know the specific errors which get thrown (or rethrown from a call to another function from whom it can get that information); and even when it can’t know that, often it can at least know the specific type of error. We could enhance the compiler libraries to track that; I spent half a day modifying the existing error-checker in Sema to prove the concept and it’s close.

Swift libraries don’t have header files, so third-parties will only ever see your documentation as generated by some tool which integrates the compiler. That means the compiler can “improve” the documentation when it is lacking. For yourself, if you care about explicitly writing out the errors which get thrown and you’re not working with an IDE, you could enable compiler-checking of your documentation comments. We can invent new shorthands to make it easier to list all of your various errors (individual cases, entire types, and maybe some private “other errors”), isolated from the rest of the language. For example:

/// - throws: MyError.{errorOne, errorThree}, AnotherError.*, *
func doSomething() throws

Maybe you can think of other ways we could help the documentation problem from the compiler side?

Typed errors are certainly no panacea but I believe they can be an important tool. Experience in other languages has shown that to be the case for many people writing many kinds of software.

If you don't believe typed errors will improve your code or if you just don't want to deal with typed errors, just don't use them! You will be able to continue using untyped throws just as you do today. You can even do this if you use libraries that throw typed errors.

If it’s the wrong solution to the problem, it does make the language worse overall.

On Tue, 28 Feb 2017 at 1:12 Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

> On Feb 27, 2017, at 5:01 PM, Dave Abrahams <dabrahams@apple.com <mailto:dabrahams@apple.com>> wrote:
>
>
> on Mon Feb 27 2017, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/>> wrote:
>
>>> On Feb 27, 2017, at 4:20 PM, Dave Abrahams <dabrahams@apple.com <mailto:dabrahams@apple.com>> wrote:
>>>
>>>
>>> I'm sorry, I don't see any substantive difference, based on what you've
>>> written here, between this feature and const.
>>
>> Let me give it one more shot and then I’ll drop it. :)
>>
>> Const is viral because if an API does not declare its arguments const
>> it cannot be used by a caller who has a const argument.
>
> Unless the caller casts away const, thus erasing information that was
> previously encoded in the type system.

Yes, of course.

>
>> It is required in order to make an API as generally useful as
>> possible.
>>
>> Typed errors are not viral in this way because no callers are
>> prevented from calling an API regardless of whether it declares error
>> types or just throws Error like we have today.
>
> Unless the caller can recover (which is *very* rare) or it catches and
> rethrows one of the errors *it* declares, thus erasing information that
> was previously encoded in the type system.

I view this as being fundamentally different than casting away const. Casting away const says “I know better than the types”.

Converting an error to a different type is extremely different. It much more similar to any other kind of value wrapper a library might create in order to shield its users from being coupled to its implementation details / dependencies. This is not a way *around* the type system in the sense that casting away const is. It is a way of *using* the type system (hopefully) to your advantage.

>
>> Pressure to declare error types in your signature in order to make
>> your function as generally useful as possible does not exist. Each
>> function is free to declare error types or not according to the
>> contract it wishes to expose.
>>
>> An argument can be made that community expectations might develop that
>> good APIs should declare error types and they could be considered
>> viral in this sense because any API that is simply declared `throws`
>> is dropping type information. But I think this overstates the case.
>> The community appears to be very sensitive to the problems that can
>> arise when error types are too concrete, especially across module
>> boundaries. I think we can learn to use the tool where it works well
>> and to avoid it where it causes problems.
>>
>>>
>>> --
>>> -Dave
>>
>
> --
> -Dave

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

Daniel_Leping · February 28, 2017, 6:21pm

Matthew, I understand your idea. Though IMO nested exceptions lead to
cluttering and to tons of boilerplate code.

What I think is that the important (logical) parts are to be provided by
custom errors (not wrapped, just custom). It's a good practice and you
don't need any typed throws for it. Anyways, though, the library developer
should change the major version (everybody is more or less bound to same
major version) and warn of the API change. It's not unreliable API, it's
sticking to the specific version. Anyways the major versions are exactly
the ones that break the API. What this means is that the logical part
remains the same, but some exceptions (all of which you will anyways treat
the same way as you don't know what to do with them) should be transparent.

It's not that I don't understand you. I do. Though I had quite some
experience in Java (it has typed throws) and I know what problems this
approach inherently has. How do you think why Scala (modern language based
on Java) dropped typed throws even being supposed having them inherently?

I think typed throws do more harm than good for all the reasons described
above.

TL;DR - typed throws is not a solution for the real world. It's good in
theory only.

···

On Tue, Feb 28, 2017 at 7:00 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Feb 28, 2017, at 10:44 AM, Daniel Leping <daniel@crossroadlabs.xyz> > wrote:

Mathew, I totally understand what you're talking about. And this is
exactly what I want to avoid.

If a library developer knows how to handle certain case (i.e. recover or
throw a special error) I'm ok with it. Though I'm not ok with completely
hiding errors from the underlying implementation. The library developer
might not handle EVERYTHING. I want to know what happened and where.
Otherwise I will have to explicitly debug his library to find out what the
issue is. In most cases I will not have time to wait developer fixing the
library. I will fix it myself and make PR. So I consider tranparent errors
rather a benefit.

Hiding errors under library errors might have worked for proprietary
middleware. Today the world changed. It's open source now.

I agree with you that having *access* to the original error is important
and that a library should not be expected to handle every case. I hope I
have not said anything that suggests otherwise.

What I’m suggesting is that if the error originated in a dependency that
might change it should be wrapped in a stable abstraction that the users of
the library can depend on. The underlying error should still be available
via a property or associated value exposed with an existential type. If
you expect users to catch errors thrown by a dependency that might change
you have a fragile API contract in the area of errors.

This can easily lead to unreliable software - when you change the
dependency nothing alerts users to the fact that their error handling code
is probably broken. Even if the compiler told them it was broken, they now
have the burden of learning about the errors that might come from your new
dependency. An API that could lead to this consequence is badly designed
IMO.

I even suggested adding a new requirement to `Error` that includes a
default implementation to standardize how we access wrapped errors,
including the originating error underneath all wrapper layers.

My point is that I'm completely against typed throws. I'm sure there will
be a lot of developers who would try to use the feature to hide underlying
errors under their own. I can't consider it a good practice (reasons
above). This is how good intentions (caring about API and library users)
can lead to bad consequences. Sounds good in theory - works bad in practice
(in real, non ideal world).

What is your opinion about the problem I describe above? Do you think
this is unimportant?

On Tue, 28 Feb 2017 at 18:18 Matthew Johnson <matthew@anandabits.com> > wrote:

On Feb 28, 2017, at 5:09 AM, Daniel Leping <daniel@crossroadlabs.xyz> >> wrote:

When you're going to present a user with specific error message you don't
do it for all errors. First of all at the end of a call chain you will have
a couple of dozens of possible errors and only 2 or 3 will do really matter
for the end user. Would you list all the dozen in params? Probably not.
You'll just catch what matters and put the rest to the log with a default
"sorry" message.

What I mean is it's OK to be "unaware" why everything failed. What really
does matter is what you can deal with and what you can't (the rest). "The
rest" is usually a magnitude bigger, though.

I might agree, that listing some errors in the throws clause is useful
feature for documentation (even so, I think the docs should be in the
comments), but only and only if it can throw the listed errors + a bunch of
other stuff. In Java it's resolved with a terrible solution of RuntimeError
interface which I definitely suggest to avoid.

As a library developer I can say that even for a library it's a nightmare
and libraries will start wrapping system errors with their own if the trend
would become to list errors in middleware. My preference though is to have
all errors transparently passed in both roles, being a library developer
and a library user.

Whether to wrap or not is a judgment call.

If the purpose of your library is to simply provide a cleaner experience
for a system API that is a stable dependency that will not change. Users
might expect to have the errors from the system API forwarded directly and
that would be perfectly appropriate.

In other cases you might have dependencies that are considered
implementation details. In that case if you don’t wrap the errors you’re
doing users a disservice by giving them an unstable interface for error
handling. Changing out your dependency that is supposed to be an
implementation detail becomes a breaking change.

On Tue, 28 Feb 2017 at 8:29 Karl Wagner <razielim@gmail.com> wrote:

On 28 Feb 2017, at 02:43, Matthew Johnson via swift-evolution < >> swift-evolution@swift.org> wrote:

Sent from my iPad

On Feb 27, 2017, at 7:19 PM, Daniel Leping <daniel@crossroadlabs.xyz> >> wrote:

Well, as Dave pointed, you can very rarely recover from an error, which
IMO is absolutely true.

If your operation fails you don't really care unless you can recover. And
you know your cases, which you can recover from (in reality one usually
does it in optimization phase, though).

What else do you need the type of error for at the very end of your call
stack? In 90% you will tell the user "ah, sorry, something happened. Come
back later" and log the error (if you haven't forgot it).

In most cases the errors are not for recovering. They neither are to be
presented to users. They are for developers to read the log/crash
report/whatever else and analyze it. Most of the errors are for debugging
purposes.

I don't want to deal with cumbersome code the purpose of which is to just
"obey the language rules". Unless I know how to recover I would rethrow it.
Than catch at the top of the stack and log + show the user a nice "sorry"
message without getting techy.

In order to provide a helpful experience to end users an app needs to
know about what might have caused the error and therefore what might (or
might not) resolve it, allowing the operation to succeed on a subsequent
attempt. This can influence the strategy an app uses to avoid bothering
the user if it might be resolvable without user intervention and can also
influence the content of the message presented to users if that is
necessary.

Errors are certainly useful, but the key distinction (IMO) is that
callers should never rely on an error happening or not. They are typically
highly implementation and hence version-specific. I think that means
specific errors shouldn’t be part of the function signature.

That doesn’t mean you can’t handle the error. A function that throws is
one that reserves the right to fail; maybe you can resolve the problem, but
maybe you can’t. That means your enclosing operation also needs to be
prepared to fail for reasons beyond _its_ control, and so on. At some point
your entire UI-level sub-operation (like opening the file, downloading the
data from the network) just fails, and you decide whether or not the user
needs to be told of that and how to do so.

Indeed, many apps don't bother with this kind of detail and just treat
all errors the same. Personally, I find that to be an unfortunate state of
affairs both as a user and as a developer.

Types can be useful in conveying this kind of information and abstracting
low level details that are not helpful at higher levels in the stack. Of
course types are not the only way to convey this information. But my
experience is that untyped errors often result in libraries with poor
documentation of error cases and not enough consideration of the error
handling experience of users of the library in general. That makes it
very difficult to handle errors well. I have experiences like this in
several languages and on several platforms, including Apple's.

I’m not sure that types are really the best abstraction for “the list of
errors that can be thrown by this function”. They’re fine for error
themselves, but not for per-function error-lists IMO. Most of our types are
way too rigid for this to be easy to live with.

If I understand what you’re saying, the core problem can be summarised
as: "I don’t know/can’t easily communicate which errors this function
throws”. Really, it is a documentation problem; it's too onerous to
document every individual throwing function, even with our very pretty
markdown-like syntax. I’m just not sure that the relatively rigid
type-system is the solution.

Now, if we look at this from a documentation perspective: obviously, the
compiler _typically_ can't generate documentation for you, because it
doesn’t know what your types/functions are meant for. Error documentation,
though, is different: the compiler *can* often know the specific errors
which get thrown (or rethrown from a call to another function from whom it
can get that information); and even when it can’t know that, often it can
at least know the specific type of error. We could enhance the compiler
libraries to track that; I spent half a day modifying the existing
error-checker in Sema to prove the concept and it’s close.

Swift libraries don’t have header files, so third-parties will only ever
see your documentation as generated by some tool which integrates the
compiler. That means the compiler can “improve” the documentation when it
is lacking. For yourself, if you care about explicitly writing out the
errors which get thrown and you’re not working with an IDE, you could
enable compiler-checking of your documentation comments. We can invent new
shorthands to make it easier to list all of your various errors (individual
cases, entire types, and maybe some private “other errors”), isolated from
the rest of the language. For example:

/// - throws: MyError.{errorOne, errorThree}, AnotherError.*, *
func doSomething() throws

Maybe you can think of other ways we could help the documentation problem
from the compiler side?

Typed errors are certainly no panacea but I believe they can be an
important tool. Experience in other languages has shown that to be the
case for many people writing many kinds of software.

If you don't believe typed errors will improve your code or if you just
don't want to deal with typed errors, just don't use them! You will be
able to continue using untyped throws just as you do today. You can even
do this if you use libraries that throw typed errors.

If it’s the wrong solution to the problem, it does make the language
worse overall.

On Tue, 28 Feb 2017 at 1:12 Matthew Johnson via swift-evolution < >> swift-evolution@swift.org> wrote:

> On Feb 27, 2017, at 5:01 PM, Dave Abrahams <dabrahams@apple.com> wrote:
>
>
> on Mon Feb 27 2017, Matthew Johnson <matthew-AT-anandabits.com >> <http://matthew-at-anandabits.com/>> wrote:
>
>>> On Feb 27, 2017, at 4:20 PM, Dave Abrahams <dabrahams@apple.com> >> wrote:
>>>
>>>
>>> I'm sorry, I don't see any substantive difference, based on what
you've
>>> written here, between this feature and const.
>>
>> Let me give it one more shot and then I’ll drop it. :)
>>
>> Const is viral because if an API does not declare its arguments const
>> it cannot be used by a caller who has a const argument.
>
> Unless the caller casts away const, thus erasing information that was
> previously encoded in the type system.

Yes, of course.

>
>> It is required in order to make an API as generally useful as
>> possible.
>>
>> Typed errors are not viral in this way because no callers are
>> prevented from calling an API regardless of whether it declares error
>> types or just throws Error like we have today.
>
> Unless the caller can recover (which is *very* rare) or it catches and
> rethrows one of the errors *it* declares, thus erasing information that
> was previously encoded in the type system.

I view this as being fundamentally different than casting away const.
Casting away const says “I know better than the types”.

Converting an error to a different type is extremely different. It much
more similar to any other kind of value wrapper a library might create in
order to shield its users from being coupled to its implementation details
/ dependencies. This is not a way *around* the type system in the sense
that casting away const is. It is a way of *using* the type system
(hopefully) to your advantage.

>
>> Pressure to declare error types in your signature in order to make
>> your function as generally useful as possible does not exist. Each
>> function is free to declare error types or not according to the
>> contract it wishes to expose.
>>
>> An argument can be made that community expectations might develop that
>> good APIs should declare error types and they could be considered
>> viral in this sense because any API that is simply declared `throws`
>> is dropping type information. But I think this overstates the case.
>> The community appears to be very sensitive to the problems that can
>> arise when error types are too concrete, especially across module
>> boundaries. I think we can learn to use the tool where it works well
>> and to avoid it where it causes problems.
>>
>>>
>>> --
>>> -Dave
>>
>
> --
> -Dave

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Chris_Lattner · February 22, 2017, 4:55pm

3) Third, your enum needs to be declared fragile in order to allow clients to enumerate their cases specifically.

The third step (having to mark your enum fragile, however it is spelled) is the biggest sign that you’re opting into a commitment that you should think really hard about. If folks don’t know that this is a big API commitment, then we have bigger problems.

You're imagining somebody adding a throws annotation because their function itself has one or two explicit throw sites.

I was trying to imagine what the “bad thing” was that you were imagining, I guess I missed the mark :)

In reality, it's probably because they're calling some other function that can throw. The low-impedence path in that case is to declare that you throw whatever that function throws; this is why people talk about exceptions encoding your library's dependency tree.

Yeah, I’ve totally seen that in Java and other languages, so I can see your concern. I think that that concern is completely addressed by:

1) Only allow one thrown type.
2) Never add T | U | V syntax for anonymous enums.

Both of these seem strongly motivated by other design points in swift, so I took those for granted.

With the two of them together, Swift would seriously reduce the odds of your "bad thing” from happening, because people who have to write out new enum types to encode their dependency tree.

AFAICT, the cost of this API is only a “li rN, 0” in the normal path.

If you're willing to give up on a no-malloc guarantee, even if the fact of resilience, yes, this and similar techniques would work. I'm not sure that all "deep systems" code is quite that deep.

I’m not claiming that all “deep systems” code (whatever that means) is quite "that deep", only that an interesting communities who consider themselves purveyors of systems code also care about controlling allocations for a variety of reasons.

The sort of systems code I’m imagining doesn’t care about resilience at all. They generally have a closed world assumption anyway.

-Chris

···

On Feb 22, 2017, at 6:32 AM, John McCall <rjmccall@apple.com> wrote:

anandabits · February 22, 2017, 3:51pm

As you know, I still think that adding typed throws is the right thing to do. I understand your concern about “the feature could be misused” but the same thing is true about many other language features.

That's fair, but I do think there's an important difference here. The way I see it, typed-throws is really something of an expert feature, not because it's at all difficult to use, but the reverse: because it's easy to use without really thinking about the consequences. (And the benefits are pretty subtle, too.) I'm not saying that we should design it to be hard to use, but I think maybe it shouldn't immediately suggest itself, and it especially shouldn't come across as just a more specific version of throws.

Yeah, I agree that it will be appealing to people who don’t know better, but here’s the thing: the (almost certain) Swift design will prevent the bad thing from happening in practice.

Consider the barriers Swift already puts in place to prevent the bad thing (declaring an inappropriately narrow explicitly-specified throw signature) from happening:

1) First of all, you need to declare a public API. If it isn’t public, then there is no concern at all, you can evolve the implementation and clients together.

2) The Second problem depends on the number of errors it can throw. If there is exactly one type of error, the most common way to handle it is by returning optional. If you have one obvious failure mode with a value, then you throw that value. The most common case is where you can throw more than one sort of error, and therefore have an enum to describe it.

3) Third, your enum needs to be declared fragile in order to allow clients to enumerate their cases specifically.

The third step (having to mark your enum fragile, however it is spelled) is the biggest sign that you’re opting into a commitment that you should think really hard about. If folks don’t know that this is a big API commitment, then we have bigger problems.

You're imagining somebody adding a throws annotation because their function itself has one or two explicit throw sites. In reality, it's probably because they're calling some other function that can throw. The low-impedence path in that case is to declare that you throw whatever that function throws; this is why people talk about exceptions encoding your library's dependency tree.

A function should absolutely consider carefully how it propagates errors. That doesn’t mean that it shouldn’t be allowed to provide an appropriate abstraction that bounds the errors it might throw. We should try to make this easy to do. Sometimes `Error` will be the appropriate bound and other times it will be very useful to provide a more granular bound that is not simply “here’s everything my dependencies can throw”.

Of course this power can be abused. But by designing a system that makes doing the right thing ergonomic and educating the community about how to use it I think we can avoid the worst problems people have faced in other languages like Java.

···

On Feb 22, 2017, at 8:32 AM, John McCall via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 21, 2017, at 11:50 PM, Chris Lattner <clattner@nondot.org> wrote:
On Feb 20, 2017, at 11:12 PM, John McCall <rjmccall@apple.com> wrote:

One thing you didn’t mention is that boxing thrown values in an existential requires allocation in the general case. This may be unacceptable for some classes of Swift application (in the embedded / deep systems space) or simply undesirable because of the performance implication.

So, the performance implication cuts both ways. We can design the ABI for typed-throws so that, say, the callee initializes some buffer that's passed into it. That's an ABI that will kill some potential allocations in deep systems code, no question about it.

Agreed.

But in non-deep-systems code, we generally expect that error types will be resilient, which means that there are non-zero dynamic costs for allocating space on the stack for the error.

Proposed solution: ABI is that the callee takes in a register which is either a buffer address to fill in or null. On error, the callee returns the error pointer in a specific register. If there was a buffer passed in, it uses it, otherwise it allocates.

In practice, this allows the compiler to only pre-allocate the buffer when it knows the fixed size, otherwise the caller allocates on the heap on demand.

AFAICT, the cost of this API is only a “li rN, 0” in the normal path.

If you're willing to give up on a no-malloc guarantee, even if the fact of resilience, yes, this and similar techniques would work. I'm not sure that all "deep systems" code is quite that deep.

John.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Karl · February 23, 2017, 8:06am

Surely the compiler could already do that within a single module? If it knows it’s going to get a bunch of cases from the same enum, it can have an optimised internal call. We could possibly allow exhaustive catching within the same module this way.

Most libraries will need the existential wrapper due to resilience concerns. Libraries that want to get this optimised, non-resilient, exhaustive error handling could opt-in somehow. I think it’s a pretty small group that really need it - perhaps they could be satisfied with throwing a single fixed enum.

For most practical purposes, the best thing we could do to improve our error-handling experience is have the compiler generate some minimal documentation of the public Errors which a function throws. That means you only need to document the error cases themselves, rather than documenting every function. App developers and library authors would get substantially better diagnostics and documentation with very little effort. Again, considering resilience, I think it’s a bigger win than strongly-typed throws.

The only language change would be that you could exhaustively catch errors within your own module. The rest is QOI.

- Karl

···

On 22 Feb 2017, at 21:13, Joe Groff <jgroff@apple.com> wrote:

On Feb 21, 2017, at 8:50 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 20, 2017, at 11:12 PM, John McCall <rjmccall@apple.com> wrote:

As you know, I still think that adding typed throws is the right thing to do. I understand your concern about “the feature could be misused” but the same thing is true about many other language features.

That's fair, but I do think there's an important difference here. The way I see it, typed-throws is really something of an expert feature, not because it's at all difficult to use, but the reverse: because it's easy to use without really thinking about the consequences. (And the benefits are pretty subtle, too.) I'm not saying that we should design it to be hard to use, but I think maybe it shouldn't immediately suggest itself, and it especially shouldn't come across as just a more specific version of throws.

Yeah, I agree that it will be appealing to people who don’t know better, but here’s the thing: the (almost certain) Swift design will prevent the bad thing from happening in practice.

Consider the barriers Swift already puts in place to prevent the bad thing (declaring an inappropriately narrow explicitly-specified throw signature) from happening:

1) First of all, you need to declare a public API. If it isn’t public, then there is no concern at all, you can evolve the implementation and clients together.

2) The Second problem depends on the number of errors it can throw. If there is exactly one type of error, the most common way to handle it is by returning optional. If you have one obvious failure mode with a value, then you throw that value. The most common case is where you can throw more than one sort of error, and therefore have an enum to describe it.

3) Third, your enum needs to be declared fragile in order to allow clients to enumerate their cases specifically.

The third step (having to mark your enum fragile, however it is spelled) is the biggest sign that you’re opting into a commitment that you should think really hard about. If folks don’t know that this is a big API commitment, then we have bigger problems.

One thing you didn’t mention is that boxing thrown values in an existential requires allocation in the general case. This may be unacceptable for some classes of Swift application (in the embedded / deep systems space) or simply undesirable because of the performance implication.

So, the performance implication cuts both ways. We can design the ABI for typed-throws so that, say, the callee initializes some buffer that's passed into it. That's an ABI that will kill some potential allocations in deep systems code, no question about it.

Agreed.

But in non-deep-systems code, we generally expect that error types will be resilient, which means that there are non-zero dynamic costs for allocating space on the stack for the error.

Proposed solution: ABI is that the callee takes in a register which is either a buffer address to fill in or null. On error, the callee returns the error pointer in a specific register. If there was a buffer passed in, it uses it, otherwise it allocates.

In practice, this allows the compiler to only pre-allocate the buffer when it knows the fixed size, otherwise the caller allocates on the heap on demand.

AFAICT, the cost of this API is only a “li rN, 0” in the normal path.

If you really want zero mallocs, it seems to me like creative stack accounting could get you there. In cases where we concretely know we're working with an existential, we can "explode" the existential into an opened type variable and a value of that type, and avoid allocating the existential representation. In the case of an untyped or resilient error, the callee could leave the error value and its type metadata on the stack somewhere without resetting the stack pointer, stick the address of that payload somewhere, and code would then propagate up to the catcher, who's responsible for consuming the error value and popping the stack when it's done with it. We could potentially do this more generally with existential returns too, so that existentials are a viable abstraction tool even in predictable-performance mode.

-Joe

anandabits · February 24, 2017, 7:32pm

Yeah but Rust has a macro system so error-chain makes re-wrapping trivial. A simple `chain_err(|| “my new error message”)` is enough to define and “throw” a new error that wraps the underlying error.

That was my point—they recognized error chaining/layering as a common phonemenon and provided affordances for it as part of their design. I was saying we should do the same if we investigate typed errors seriously. (Not sure why you mentioned macros—chain_err is just a regular method, and could be defined as an extension method on Error in swift today.)

-Joe

Sorry, I meant error_chain! handles the boilerplate. At the end of the day something has to create the new error types.

I agree with you on the chaining; if we really do want to go down this road we need an elegant solution. Seeing people talk about getting rid of rethrows makes me think this isn’t yet a serious discussion.

Is it just the syntactic sugar you would miss or is there more to it? Did you see the analysis I sent out showing a case rethrows can’t handle and a problem the current proposal has in handling typed rethrows?

If `rethrows` can be turned into sugar for regular type parametricity, I think that'd be a win over our current model. The sugar is probably still useful since `<E> (() throws E -> ()) throws SomethingDerivedFrom<E> -> ()` is common enough; whether SomethingDerivedFrom<E> is E, Error, or MyModulesErrorWrapper<E> would have to be discussed.

I don’t mind keeping rethrows as syntactic sugar but I would like to have a solution that can handle all cases including the one it isn’t able to handle.

I almost included something like this in the analysis I sent out. I thought maybe injecting implicit generic parameters would be a little bit too magical. It would be super convenient though!

···

On Feb 24, 2017, at 11:00 AM, Joe Groff via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 23, 2017, at 10:36 PM, Russ Bishop <xenadu@gmail.com <mailto:xenadu@gmail.com>> wrote:

On Feb 21, 2017, at 8:18 AM, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

dabrahams · February 28, 2017, 6:44pm

I just want to point at some prior art in this area,
http://boost.org/libs/exception, which deals with the issues of wrapping
errors at library boundaries and the dynamic construction of a stack of
useful information. This library was designed about 10 years ago, but I
still remember the design discussions as compelling. The formal review
thread is here
http://boost.2283326.n4.nabble.com/review-Review-of-Exception-begins-today-tt2633630.html#none
but I think the most interesting discussion is probably in the foregoing
thread
http://boost.2283326.n4.nabble.com/Review-Request-Boost-Exception-tt2616607.html#a2616624
(among which my comments are probably the *least* useful).

Anyone interested in pursuing features in this area should probably
review those discussions. Even though the context is C++, I think
there's very little in it that's language-specific.

···

on Tue Feb 28 2017, Karl Wagner <razielim-AT-gmail.com> wrote:

On 28 Feb 2017, at 18:00, Matthew Johnson <matthew@anandabits.com> wrote:

On Feb 28, 2017, at 10:44 AM, Daniel Leping <daniel@crossroadlabs.xyz > <mailto:daniel@crossroadlabs.xyz>> wrote:

Mathew, I totally understand what you're talking about. And this is exactly what I want to avoid.

If a library developer knows how to handle certain case
(i.e. recover or throw a special error) I'm ok with it. Though I'm
not ok with completely hiding errors from the underlying
implementation. The library developer might not handle
EVERYTHING. I want to know what happened and where. Otherwise I
will have to explicitly debug his library to find out what the
issue is. In most cases I will not have time to wait developer
fixing the library. I will fix it myself and make PR. So I consider
tranparent errors rather a benefit.

Hiding errors under library errors might have worked for
proprietary middleware. Today the world changed. It's open source
now.

I agree with you that having *access* to the original error is
important and that a library should not be expected to handle every
case. I hope I have not said anything that suggests otherwise.

What I’m suggesting is that if the error originated in a dependency
that might change it should be wrapped in a stable abstraction that
the users of the library can depend on. The underlying error should
still be available via a property or associated value exposed with
an existential type. If you expect users to catch errors thrown by
a dependency that might change you have a fragile API contract in
the area of errors.

This can easily lead to unreliable software - when you change the
dependency nothing alerts users to the fact that their error
handling code is probably broken. Even if the compiler told them it
was broken, they now have the burden of learning about the errors
that might come from your new dependency. An API that could lead to
this consequence is badly designed IMO.

I even suggested adding a new requirement to `Error` that includes a
default implementation to standardize how we access wrapped errors,
including the originating error underneath all wrapper layers.

I don’t see that it’s such a big problem. There are lots of ways to
expose the information once we have it, and it could easily be limited
to public errors from the current module (substituting private/unbound
errors with an erasing “Error”). There are lots of small improvements
we could make: such as having functions which re-throw errors from
other, manually documented functions inherit their documentation (or
at least appear to, when viewed). That would not be totally
compiler-generated documentation, but would mean you can document your
errors in one place and make doing so much more palatable. :

--
-Dave

anandabits · February 28, 2017, 6:36pm

Mathew, I totally understand what you're talking about. And this is exactly what I want to avoid.

If a library developer knows how to handle certain case (i.e. recover or throw a special error) I'm ok with it. Though I'm not ok with completely hiding errors from the underlying implementation. The library developer might not handle EVERYTHING. I want to know what happened and where. Otherwise I will have to explicitly debug his library to find out what the issue is. In most cases I will not have time to wait developer fixing the library. I will fix it myself and make PR. So I consider tranparent errors rather a benefit.

Hiding errors under library errors might have worked for proprietary middleware. Today the world changed. It's open source now.

I agree with you that having *access* to the original error is important and that a library should not be expected to handle every case. I hope I have not said anything that suggests otherwise.

What I’m suggesting is that if the error originated in a dependency that might change it should be wrapped in a stable abstraction that the users of the library can depend on. The underlying error should still be available via a property or associated value exposed with an existential type. If you expect users to catch errors thrown by a dependency that might change you have a fragile API contract in the area of errors.

This can easily lead to unreliable software - when you change the dependency nothing alerts users to the fact that their error handling code is probably broken. Even if the compiler told them it was broken, they now have the burden of learning about the errors that might come from your new dependency. An API that could lead to this consequence is badly designed IMO.

I even suggested adding a new requirement to `Error` that includes a default implementation to standardize how we access wrapped errors, including the originating error underneath all wrapper layers.

I don’t see that it’s such a big problem. There are lots of ways to expose the information once we have it, and it could easily be limited to public errors from the current module (substituting private/unbound errors with an erasing “Error”).

If you restrict it in this way then it will require wrapping if the caller has any meaningful way to handle errors thrown by your dependencies that is not “something went wrong (but I have no idea what it is or how to fix it)”.

There are lots of small improvements we could make: such as having functions which re-throw errors from other, manually documented functions inherit their documentation (or at least appear to, when viewed). That would not be totally compiler-generated documentation, but would mean you can document your errors in one place and make doing so much more palatable. :

// Manual documentation in one place:

/// - throws: X, Y, Z
func doSomething() throws { … }

// Inherited by functions which call it and rethrow its errors:

/// [generated]: - throws: X, Y, X
func doSomethingExtended() throws {
    try doSomething()
}

/// [generated]: - throws: X, Y, Z
func doSomethingMapped() throws {
    // (call to rethrowing function ‘map’) -> (throwing closure expression) -> (call to throwing function “doSomething”, maybe others, too...) -> X, Y, Z
    try [1, 2, 3].map { try doSomething() }
}

Personally, I don’t like wrapping.

I don’t like the boilerplate wrapping can lead to but I think the semantics are often the right semantics. If you don’t do this you either prevent callers from handling some errors with a more specific error path or you create unwanted coupling between your callers and your dependencies.

The right way to solve this (IMO) is to find ways to eliminate the boilerplate.

I like error-handling to be flat at each level because it’s much easier to reason about.

I like wrapping because it is often easier to reason about high-level groupings of underlying errors that have been thoughtfully designed than it is to reason about everything that can go wrong. Often this will still result in flat error handling because the grouping is sufficient and the underlying error is irrelevant to how the error is handled.

You would basically need catch on "error as? MyError” and “error.underlyingError as? MyError”, duplicating code or forwarding them to a refactored handler function.

Or we could introduce some syntax to allow you to ignore the wrapper when you really need to and catch the underlying error directly. Wrapping and unwrapping is a problem that can be solved by a language feature.

Also, consider that an Error might be any type at all (including a class). If any instance of a thing which conforms to Error could potentially be holding a reference-counted pointer, that could limit optimisations. If it’s flat, we can potentially prove that it has an exact type (or at least is never reference-counted), and that could be valuable.

I think performance is the last thing we should be concerned about in the vast majority of error handling use cases. Providing a good experience to users which includes robust and helpful recovery and / or reporting is much more important. In cases where there is a performance reason, it is easy enough to avoid wrapping (and typed errors) if necessary.

Error-contracts in libraries are only really fixed in the case of an implementation which is never going to change substantially. In that case, the function is a possible candidate for @inlineable and can be treated as though part of the client’s module, with full error-flow intelligence.

I’m not suggesting they be fixed at all. Resilient enums will often be a good way to provide a wrapper. In fact, they allow users to handle errors in a stable manner *despite* underlying errors changing over time as the implementation changes.

What I am saying is that thoughtful categorization of underlying errors is an important API design technique. It is true that this technique is currently burdened by unfortunate boilerplate. This is a problem that can be solved if we decide we want to solve it.

I think we should be encouraging thoughtful design of the error contracts our APIs have rather than encouraging mindless propagation of errors thrown by our dependencies.

···

On Feb 28, 2017, at 11:55 AM, Karl Wagner <razielim@gmail.com> wrote:

On 28 Feb 2017, at 18:00, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Feb 28, 2017, at 10:44 AM, Daniel Leping <daniel@crossroadlabs.xyz <mailto:daniel@crossroadlabs.xyz>> wrote:

- Karl

My point is that I'm completely against typed throws. I'm sure there will be a lot of developers who would try to use the feature to hide underlying errors under their own. I can't consider it a good practice (reasons above). This is how good intentions (caring about API and library users) can lead to bad consequences. Sounds good in theory - works bad in practice (in real, non ideal world).

What is your opinion about the problem I describe above? Do you think this is unimportant?

On Tue, 28 Feb 2017 at 18:18 Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Feb 28, 2017, at 5:09 AM, Daniel Leping <daniel@crossroadlabs.xyz <mailto:daniel@crossroadlabs.xyz>> wrote:

When you're going to present a user with specific error message you don't do it for all errors. First of all at the end of a call chain you will have a couple of dozens of possible errors and only 2 or 3 will do really matter for the end user. Would you list all the dozen in params? Probably not. You'll just catch what matters and put the rest to the log with a default "sorry" message.

What I mean is it's OK to be "unaware" why everything failed. What really does matter is what you can deal with and what you can't (the rest). "The rest" is usually a magnitude bigger, though.

I might agree, that listing some errors in the throws clause is useful feature for documentation (even so, I think the docs should be in the comments), but only and only if it can throw the listed errors + a bunch of other stuff. In Java it's resolved with a terrible solution of RuntimeError interface which I definitely suggest to avoid.

As a library developer I can say that even for a library it's a nightmare and libraries will start wrapping system errors with their own if the trend would become to list errors in middleware. My preference though is to have all errors transparently passed in both roles, being a library developer and a library user.

Whether to wrap or not is a judgment call.

If the purpose of your library is to simply provide a cleaner experience for a system API that is a stable dependency that will not change. Users might expect to have the errors from the system API forwarded directly and that would be perfectly appropriate.

In other cases you might have dependencies that are considered implementation details. In that case if you don’t wrap the errors you’re doing users a disservice by giving them an unstable interface for error handling. Changing out your dependency that is supposed to be an implementation detail becomes a breaking change.

On Tue, 28 Feb 2017 at 8:29 Karl Wagner <razielim@gmail.com <mailto:razielim@gmail.com>> wrote:

On 28 Feb 2017, at 02:43, Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Sent from my iPad

On Feb 27, 2017, at 7:19 PM, Daniel Leping <daniel@crossroadlabs.xyz <mailto:daniel@crossroadlabs.xyz>> wrote:

Well, as Dave pointed, you can very rarely recover from an error, which IMO is absolutely true.

If your operation fails you don't really care unless you can recover. And you know your cases, which you can recover from (in reality one usually does it in optimization phase, though).

What else do you need the type of error for at the very end of your call stack? In 90% you will tell the user "ah, sorry, something happened. Come back later" and log the error (if you haven't forgot it).

In most cases the errors are not for recovering. They neither are to be presented to users. They are for developers to read the log/crash report/whatever else and analyze it. Most of the errors are for debugging purposes.

I don't want to deal with cumbersome code the purpose of which is to just "obey the language rules". Unless I know how to recover I would rethrow it. Than catch at the top of the stack and log + show the user a nice "sorry" message without getting techy.

In order to provide a helpful experience to end users an app needs to know about what might have caused the error and therefore what might (or might not) resolve it, allowing the operation to succeed on a subsequent attempt. This can influence the strategy an app uses to avoid bothering the user if it might be resolvable without user intervention and can also influence the content of the message presented to users if that is necessary.

Errors are certainly useful, but the key distinction (IMO) is that callers should never rely on an error happening or not. They are typically highly implementation and hence version-specific. I think that means specific errors shouldn’t be part of the function signature.

That doesn’t mean you can’t handle the error. A function that throws is one that reserves the right to fail; maybe you can resolve the problem, but maybe you can’t. That means your enclosing operation also needs to be prepared to fail for reasons beyond _its_ control, and so on. At some point your entire UI-level sub-operation (like opening the file, downloading the data from the network) just fails, and you decide whether or not the user needs to be told of that and how to do so.

Indeed, many apps don't bother with this kind of detail and just treat all errors the same. Personally, I find that to be an unfortunate state of affairs both as a user and as a developer.

Types can be useful in conveying this kind of information and abstracting low level details that are not helpful at higher levels in the stack. Of course types are not the only way to convey this information. But my experience is that untyped errors often result in libraries with poor documentation of error cases and not enough consideration of the error handling experience of users of the library in general. That makes it very difficult to handle errors well. I have experiences like this in several languages and on several platforms, including Apple's.

I’m not sure that types are really the best abstraction for “the list of errors that can be thrown by this function”. They’re fine for error themselves, but not for per-function error-lists IMO. Most of our types are way too rigid for this to be easy to live with.

If I understand what you’re saying, the core problem can be summarised as: "I don’t know/can’t easily communicate which errors this function throws”. Really, it is a documentation problem; it's too onerous to document every individual throwing function, even with our very pretty markdown-like syntax. I’m just not sure that the relatively rigid type-system is the solution.

Now, if we look at this from a documentation perspective: obviously, the compiler _typically_ can't generate documentation for you, because it doesn’t know what your types/functions are meant for. Error documentation, though, is different: the compiler *can* often know the specific errors which get thrown (or rethrown from a call to another function from whom it can get that information); and even when it can’t know that, often it can at least know the specific type of error. We could enhance the compiler libraries to track that; I spent half a day modifying the existing error-checker in Sema to prove the concept and it’s close.

Swift libraries don’t have header files, so third-parties will only ever see your documentation as generated by some tool which integrates the compiler. That means the compiler can “improve” the documentation when it is lacking. For yourself, if you care about explicitly writing out the errors which get thrown and you’re not working with an IDE, you could enable compiler-checking of your documentation comments. We can invent new shorthands to make it easier to list all of your various errors (individual cases, entire types, and maybe some private “other errors”), isolated from the rest of the language. For example:

/// - throws: MyError.{errorOne, errorThree}, AnotherError.*, *
func doSomething() throws

Maybe you can think of other ways we could help the documentation problem from the compiler side?

Typed errors are certainly no panacea but I believe they can be an important tool. Experience in other languages has shown that to be the case for many people writing many kinds of software.

If you don't believe typed errors will improve your code or if you just don't want to deal with typed errors, just don't use them! You will be able to continue using untyped throws just as you do today. You can even do this if you use libraries that throw typed errors.

If it’s the wrong solution to the problem, it does make the language worse overall.

On Tue, 28 Feb 2017 at 1:12 Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

> On Feb 27, 2017, at 5:01 PM, Dave Abrahams <dabrahams@apple.com <mailto:dabrahams@apple.com>> wrote:
>
>
> on Mon Feb 27 2017, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/>> wrote:
>
>>> On Feb 27, 2017, at 4:20 PM, Dave Abrahams <dabrahams@apple.com <mailto:dabrahams@apple.com>> wrote:
>>>
>>>
>>> I'm sorry, I don't see any substantive difference, based on what you've
>>> written here, between this feature and const.
>>
>> Let me give it one more shot and then I’ll drop it. :)
>>
>> Const is viral because if an API does not declare its arguments const
>> it cannot be used by a caller who has a const argument.
>
> Unless the caller casts away const, thus erasing information that was
> previously encoded in the type system.

Yes, of course.

>
>> It is required in order to make an API as generally useful as
>> possible.
>>
>> Typed errors are not viral in this way because no callers are
>> prevented from calling an API regardless of whether it declares error
>> types or just throws Error like we have today.
>
> Unless the caller can recover (which is *very* rare) or it catches and
> rethrows one of the errors *it* declares, thus erasing information that
> was previously encoded in the type system.

I view this as being fundamentally different than casting away const. Casting away const says “I know better than the types”.

Converting an error to a different type is extremely different. It much more similar to any other kind of value wrapper a library might create in order to shield its users from being coupled to its implementation details / dependencies. This is not a way *around* the type system in the sense that casting away const is. It is a way of *using* the type system (hopefully) to your advantage.

>
>> Pressure to declare error types in your signature in order to make
>> your function as generally useful as possible does not exist. Each
>> function is free to declare error types or not according to the
>> contract it wishes to expose.
>>
>> An argument can be made that community expectations might develop that
>> good APIs should declare error types and they could be considered
>> viral in this sense because any API that is simply declared `throws`
>> is dropping type information. But I think this overstates the case.
>> The community appears to be very sensitive to the problems that can
>> arise when error types are too concrete, especially across module
>> boundaries. I think we can learn to use the tool where it works well
>> and to avoid it where it causes problems.
>>
>>>
>>> --
>>> -Dave
>>
>
> --
> -Dave

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

anandabits · February 28, 2017, 6:49pm

Matthew, I understand your idea. Though IMO nested exceptions lead to cluttering and to tons of boilerplate code.

I understand this and am suggesting that this boilerplate is a problem that can be addressed by language features. We should not disregard the value of a stable error contract just because it currently requires boilerplate that could be eliminated.

What I think is that the important (logical) parts are to be provided by custom errors (not wrapped, just custom). It's a good practice and you don't need any typed throws for it.

When the error is a result of a condition directly detected by the library this is true. But if it was originally produced by an underlying error that information should not be thrown away. It can be very useful, if for no other purpose than logging / debugging / troubleshooting.

You don’t need typed throws for wrappers either, it just allows you to communicate the wrapped error type to users when you decide that is appropriate. When you decide that is providing too much concrete type information to callers you can still just throw `Error`. In the latter case, you don’t need to wrap the errors even if you call other functions that can throw different error types. `Error` is a supertype of all errors.

Anyways, though, the library developer should change the major version (everybody is more or less bound to same major version) and warn of the API change. It's not unreliable API, it's sticking to the specific version. Anyways the major versions are exactly the ones that break the API. What this means is that the logical part remains the same, but some exceptions (all of which you will anyways treat the same way as you don't know what to do with them) should be transparent.

It’s best to minimize source-breaking changes even across major versions. If you change a dependency and you are directly propagating errors from your dependencies it is source breaking for all callers who try to do anything more than totally generic “unknown error” handling in their code. It is a bad idea to put users in the position of a) fragile error handling or b) totally generic error handling.

It's not that I don't understand you. I do. Though I had quite some experience in Java (it has typed throws) and I know what problems this approach inherently has. How do you think why Scala (modern language based on Java) dropped typed throws even being supposed having them inherently?

I am not very familiar with Java and Scala but this topic has come up several times in the past and there have been several people who have commented on differences between the design in Java and the design in languages where it works better.

···

On Feb 28, 2017, at 12:21 PM, Daniel Leping <daniel@crossroadlabs.xyz> wrote:

I think typed throws do more harm than good for all the reasons described above.

TL;DR - typed throws is not a solution for the real world. It's good in theory only.

On Tue, Feb 28, 2017 at 7:00 PM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Feb 28, 2017, at 10:44 AM, Daniel Leping <daniel@crossroadlabs.xyz <mailto:daniel@crossroadlabs.xyz>> wrote:

Mathew, I totally understand what you're talking about. And this is exactly what I want to avoid.

If a library developer knows how to handle certain case (i.e. recover or throw a special error) I'm ok with it. Though I'm not ok with completely hiding errors from the underlying implementation. The library developer might not handle EVERYTHING. I want to know what happened and where. Otherwise I will have to explicitly debug his library to find out what the issue is. In most cases I will not have time to wait developer fixing the library. I will fix it myself and make PR. So I consider tranparent errors rather a benefit.

Hiding errors under library errors might have worked for proprietary middleware. Today the world changed. It's open source now.

I agree with you that having *access* to the original error is important and that a library should not be expected to handle every case. I hope I have not said anything that suggests otherwise.

What I’m suggesting is that if the error originated in a dependency that might change it should be wrapped in a stable abstraction that the users of the library can depend on. The underlying error should still be available via a property or associated value exposed with an existential type. If you expect users to catch errors thrown by a dependency that might change you have a fragile API contract in the area of errors.

This can easily lead to unreliable software - when you change the dependency nothing alerts users to the fact that their error handling code is probably broken. Even if the compiler told them it was broken, they now have the burden of learning about the errors that might come from your new dependency. An API that could lead to this consequence is badly designed IMO.

I even suggested adding a new requirement to `Error` that includes a default implementation to standardize how we access wrapped errors, including the originating error underneath all wrapper layers.

My point is that I'm completely against typed throws. I'm sure there will be a lot of developers who would try to use the feature to hide underlying errors under their own. I can't consider it a good practice (reasons above). This is how good intentions (caring about API and library users) can lead to bad consequences. Sounds good in theory - works bad in practice (in real, non ideal world).

What is your opinion about the problem I describe above? Do you think this is unimportant?

On Tue, 28 Feb 2017 at 18:18 Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Feb 28, 2017, at 5:09 AM, Daniel Leping <daniel@crossroadlabs.xyz <mailto:daniel@crossroadlabs.xyz>> wrote:

When you're going to present a user with specific error message you don't do it for all errors. First of all at the end of a call chain you will have a couple of dozens of possible errors and only 2 or 3 will do really matter for the end user. Would you list all the dozen in params? Probably not. You'll just catch what matters and put the rest to the log with a default "sorry" message.

What I mean is it's OK to be "unaware" why everything failed. What really does matter is what you can deal with and what you can't (the rest). "The rest" is usually a magnitude bigger, though.

I might agree, that listing some errors in the throws clause is useful feature for documentation (even so, I think the docs should be in the comments), but only and only if it can throw the listed errors + a bunch of other stuff. In Java it's resolved with a terrible solution of RuntimeError interface which I definitely suggest to avoid.

As a library developer I can say that even for a library it's a nightmare and libraries will start wrapping system errors with their own if the trend would become to list errors in middleware. My preference though is to have all errors transparently passed in both roles, being a library developer and a library user.

Whether to wrap or not is a judgment call.

If the purpose of your library is to simply provide a cleaner experience for a system API that is a stable dependency that will not change. Users might expect to have the errors from the system API forwarded directly and that would be perfectly appropriate.

In other cases you might have dependencies that are considered implementation details. In that case if you don’t wrap the errors you’re doing users a disservice by giving them an unstable interface for error handling. Changing out your dependency that is supposed to be an implementation detail becomes a breaking change.

On Tue, 28 Feb 2017 at 8:29 Karl Wagner <razielim@gmail.com <mailto:razielim@gmail.com>> wrote:

On 28 Feb 2017, at 02:43, Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Sent from my iPad

On Feb 27, 2017, at 7:19 PM, Daniel Leping <daniel@crossroadlabs.xyz <mailto:daniel@crossroadlabs.xyz>> wrote:

Well, as Dave pointed, you can very rarely recover from an error, which IMO is absolutely true.

If your operation fails you don't really care unless you can recover. And you know your cases, which you can recover from (in reality one usually does it in optimization phase, though).

What else do you need the type of error for at the very end of your call stack? In 90% you will tell the user "ah, sorry, something happened. Come back later" and log the error (if you haven't forgot it).

In most cases the errors are not for recovering. They neither are to be presented to users. They are for developers to read the log/crash report/whatever else and analyze it. Most of the errors are for debugging purposes.

I don't want to deal with cumbersome code the purpose of which is to just "obey the language rules". Unless I know how to recover I would rethrow it. Than catch at the top of the stack and log + show the user a nice "sorry" message without getting techy.

In order to provide a helpful experience to end users an app needs to know about what might have caused the error and therefore what might (or might not) resolve it, allowing the operation to succeed on a subsequent attempt. This can influence the strategy an app uses to avoid bothering the user if it might be resolvable without user intervention and can also influence the content of the message presented to users if that is necessary.

Errors are certainly useful, but the key distinction (IMO) is that callers should never rely on an error happening or not. They are typically highly implementation and hence version-specific. I think that means specific errors shouldn’t be part of the function signature.

That doesn’t mean you can’t handle the error. A function that throws is one that reserves the right to fail; maybe you can resolve the problem, but maybe you can’t. That means your enclosing operation also needs to be prepared to fail for reasons beyond _its_ control, and so on. At some point your entire UI-level sub-operation (like opening the file, downloading the data from the network) just fails, and you decide whether or not the user needs to be told of that and how to do so.

Indeed, many apps don't bother with this kind of detail and just treat all errors the same. Personally, I find that to be an unfortunate state of affairs both as a user and as a developer.

Types can be useful in conveying this kind of information and abstracting low level details that are not helpful at higher levels in the stack. Of course types are not the only way to convey this information. But my experience is that untyped errors often result in libraries with poor documentation of error cases and not enough consideration of the error handling experience of users of the library in general. That makes it very difficult to handle errors well. I have experiences like this in several languages and on several platforms, including Apple's.

I’m not sure that types are really the best abstraction for “the list of errors that can be thrown by this function”. They’re fine for error themselves, but not for per-function error-lists IMO. Most of our types are way too rigid for this to be easy to live with.

If I understand what you’re saying, the core problem can be summarised as: "I don’t know/can’t easily communicate which errors this function throws”. Really, it is a documentation problem; it's too onerous to document every individual throwing function, even with our very pretty markdown-like syntax. I’m just not sure that the relatively rigid type-system is the solution.

Now, if we look at this from a documentation perspective: obviously, the compiler _typically_ can't generate documentation for you, because it doesn’t know what your types/functions are meant for. Error documentation, though, is different: the compiler *can* often know the specific errors which get thrown (or rethrown from a call to another function from whom it can get that information); and even when it can’t know that, often it can at least know the specific type of error. We could enhance the compiler libraries to track that; I spent half a day modifying the existing error-checker in Sema to prove the concept and it’s close.

Swift libraries don’t have header files, so third-parties will only ever see your documentation as generated by some tool which integrates the compiler. That means the compiler can “improve” the documentation when it is lacking. For yourself, if you care about explicitly writing out the errors which get thrown and you’re not working with an IDE, you could enable compiler-checking of your documentation comments. We can invent new shorthands to make it easier to list all of your various errors (individual cases, entire types, and maybe some private “other errors”), isolated from the rest of the language. For example:

/// - throws: MyError.{errorOne, errorThree}, AnotherError.*, *
func doSomething() throws

Maybe you can think of other ways we could help the documentation problem from the compiler side?

Typed errors are certainly no panacea but I believe they can be an important tool. Experience in other languages has shown that to be the case for many people writing many kinds of software.

If you don't believe typed errors will improve your code or if you just don't want to deal with typed errors, just don't use them! You will be able to continue using untyped throws just as you do today. You can even do this if you use libraries that throw typed errors.

If it’s the wrong solution to the problem, it does make the language worse overall.

On Tue, 28 Feb 2017 at 1:12 Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

> On Feb 27, 2017, at 5:01 PM, Dave Abrahams <dabrahams@apple.com <mailto:dabrahams@apple.com>> wrote:
>
>
> on Mon Feb 27 2017, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/>> wrote:
>
>>> On Feb 27, 2017, at 4:20 PM, Dave Abrahams <dabrahams@apple.com <mailto:dabrahams@apple.com>> wrote:
>>>
>>>
>>> I'm sorry, I don't see any substantive difference, based on what you've
>>> written here, between this feature and const.
>>
>> Let me give it one more shot and then I’ll drop it. :)
>>
>> Const is viral because if an API does not declare its arguments const
>> it cannot be used by a caller who has a const argument.
>
> Unless the caller casts away const, thus erasing information that was
> previously encoded in the type system.

Yes, of course.

>
>> It is required in order to make an API as generally useful as
>> possible.
>>
>> Typed errors are not viral in this way because no callers are
>> prevented from calling an API regardless of whether it declares error
>> types or just throws Error like we have today.
>
> Unless the caller can recover (which is *very* rare) or it catches and
> rethrows one of the errors *it* declares, thus erasing information that
> was previously encoded in the type system.

I view this as being fundamentally different than casting away const. Casting away const says “I know better than the types”.

Converting an error to a different type is extremely different. It much more similar to any other kind of value wrapper a library might create in order to shield its users from being coupled to its implementation details / dependencies. This is not a way *around* the type system in the sense that casting away const is. It is a way of *using* the type system (hopefully) to your advantage.

>
>> Pressure to declare error types in your signature in order to make
>> your function as generally useful as possible does not exist. Each
>> function is free to declare error types or not according to the
>> contract it wishes to expose.
>>
>> An argument can be made that community expectations might develop that
>> good APIs should declare error types and they could be considered
>> viral in this sense because any API that is simply declared `throws`
>> is dropping type information. But I think this overstates the case.
>> The community appears to be very sensitive to the problems that can
>> arise when error types are too concrete, especially across module
>> boundaries. I think we can learn to use the tool where it works well
>> and to avoid it where it causes problems.
>>
>>>
>>> --
>>> -Dave
>>
>
> --
> -Dave

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

anandabits · February 28, 2017, 6:52pm

Mathew, I totally understand what you're talking about. And this is exactly what I want to avoid.

If a library developer knows how to handle certain case
(i.e. recover or throw a special error) I'm ok with it. Though I'm
not ok with completely hiding errors from the underlying
implementation. The library developer might not handle
EVERYTHING. I want to know what happened and where. Otherwise I
will have to explicitly debug his library to find out what the
issue is. In most cases I will not have time to wait developer
fixing the library. I will fix it myself and make PR. So I consider
tranparent errors rather a benefit.

Hiding errors under library errors might have worked for
proprietary middleware. Today the world changed. It's open source
now.

I agree with you that having *access* to the original error is
important and that a library should not be expected to handle every
case. I hope I have not said anything that suggests otherwise.

What I’m suggesting is that if the error originated in a dependency
that might change it should be wrapped in a stable abstraction that
the users of the library can depend on. The underlying error should
still be available via a property or associated value exposed with
an existential type. If you expect users to catch errors thrown by
a dependency that might change you have a fragile API contract in
the area of errors.

This can easily lead to unreliable software - when you change the
dependency nothing alerts users to the fact that their error
handling code is probably broken. Even if the compiler told them it
was broken, they now have the burden of learning about the errors
that might come from your new dependency. An API that could lead to
this consequence is badly designed IMO.

I even suggested adding a new requirement to `Error` that includes a
default implementation to standardize how we access wrapped errors,
including the originating error underneath all wrapper layers.

I don’t see that it’s such a big problem. There are lots of ways to
expose the information once we have it, and it could easily be limited
to public errors from the current module (substituting private/unbound
errors with an erasing “Error”). There are lots of small improvements
we could make: such as having functions which re-throw errors from
other, manually documented functions inherit their documentation (or
at least appear to, when viewed). That would not be totally
compiler-generated documentation, but would mean you can document your
errors in one place and make doing so much more palatable. :

I just want to point at some prior art in this area,
http://boost.org/libs/exception, which deals with the issues of wrapping
errors at library boundaries and the dynamic construction of a stack of
useful information. This library was designed about 10 years ago, but I
still remember the design discussions as compelling. The formal review
thread is here
http://boost.2283326.n4.nabble.com/review-Review-of-Exception-begins-today-tt2633630.html#none
but I think the most interesting discussion is probably in the foregoing
thread
http://boost.2283326.n4.nabble.com/Review-Request-Boost-Exception-tt2616607.html#a2616624
(among which my comments are probably the *least* useful).

Anyone interested in pursuing features in this area should probably
review those discussions. Even though the context is C++, I think
there's very little in it that's language-specific.

Thanks for the links Dave! I will add this to my list of things to investigate further.

···

On Feb 28, 2017, at 12:44 PM, Dave Abrahams <dabrahams@apple.com> wrote:
on Tue Feb 28 2017, Karl Wagner <razielim-AT-gmail.com> wrote:

On 28 Feb 2017, at 18:00, Matthew Johnson <matthew@anandabits.com> wrote:

On Feb 28, 2017, at 10:44 AM, Daniel Leping <daniel@crossroadlabs.xyz >> <mailto:daniel@crossroadlabs.xyz>> wrote:

--
-Dave

Joe_Groff · February 23, 2017, 4:19pm

Even across resilience boundaries, values of existential type don't fundamentally *need* to use the wrapper representation when we're know we're statically working with an existential, only when we're working with one as an unspecialized generic T == SomeExistentialType. An existential function argument can be exploded into a generic argument; existential returns and errors would require out-of-line allocation if shoehorned into traditional callstack discipline due to the variable size of the payload, but an implementation that avoids that is still possible. You're right that, for a protocol whose conformers are all known, there are further refinements possible, such as discriminating the type by a enum-like tag rather than an arbitrary type metadata pointer, and potentially giving the existential box a specialized representation with a known maximum size. (In deep enough systems code that isn't resilient at all and is compiled all at once, you could also conceivably attack the untyped-Error problem by determining the max size an Error buffer needs at compile time this way.)

-Joe

···

On Feb 23, 2017, at 12:06 AM, Karl Wagner <razielim@gmail.com> wrote:

On 22 Feb 2017, at 21:13, Joe Groff <jgroff@apple.com> wrote:

On Feb 21, 2017, at 8:50 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 20, 2017, at 11:12 PM, John McCall <rjmccall@apple.com> wrote:

As you know, I still think that adding typed throws is the right thing to do. I understand your concern about “the feature could be misused” but the same thing is true about many other language features.

That's fair, but I do think there's an important difference here. The way I see it, typed-throws is really something of an expert feature, not because it's at all difficult to use, but the reverse: because it's easy to use without really thinking about the consequences. (And the benefits are pretty subtle, too.) I'm not saying that we should design it to be hard to use, but I think maybe it shouldn't immediately suggest itself, and it especially shouldn't come across as just a more specific version of throws.

Yeah, I agree that it will be appealing to people who don’t know better, but here’s the thing: the (almost certain) Swift design will prevent the bad thing from happening in practice.

Consider the barriers Swift already puts in place to prevent the bad thing (declaring an inappropriately narrow explicitly-specified throw signature) from happening:

1) First of all, you need to declare a public API. If it isn’t public, then there is no concern at all, you can evolve the implementation and clients together.

2) The Second problem depends on the number of errors it can throw. If there is exactly one type of error, the most common way to handle it is by returning optional. If you have one obvious failure mode with a value, then you throw that value. The most common case is where you can throw more than one sort of error, and therefore have an enum to describe it.

3) Third, your enum needs to be declared fragile in order to allow clients to enumerate their cases specifically.

The third step (having to mark your enum fragile, however it is spelled) is the biggest sign that you’re opting into a commitment that you should think really hard about. If folks don’t know that this is a big API commitment, then we have bigger problems.

One thing you didn’t mention is that boxing thrown values in an existential requires allocation in the general case. This may be unacceptable for some classes of Swift application (in the embedded / deep systems space) or simply undesirable because of the performance implication.

So, the performance implication cuts both ways. We can design the ABI for typed-throws so that, say, the callee initializes some buffer that's passed into it. That's an ABI that will kill some potential allocations in deep systems code, no question about it.

Agreed.

But in non-deep-systems code, we generally expect that error types will be resilient, which means that there are non-zero dynamic costs for allocating space on the stack for the error.

Proposed solution: ABI is that the callee takes in a register which is either a buffer address to fill in or null. On error, the callee returns the error pointer in a specific register. If there was a buffer passed in, it uses it, otherwise it allocates.

In practice, this allows the compiler to only pre-allocate the buffer when it knows the fixed size, otherwise the caller allocates on the heap on demand.

AFAICT, the cost of this API is only a “li rN, 0” in the normal path.

If you really want zero mallocs, it seems to me like creative stack accounting could get you there. In cases where we concretely know we're working with an existential, we can "explode" the existential into an opened type variable and a value of that type, and avoid allocating the existential representation. In the case of an untyped or resilient error, the callee could leave the error value and its type metadata on the stack somewhere without resetting the stack pointer, stick the address of that payload somewhere, and code would then propagate up to the catcher, who's responsible for consuming the error value and popping the stack when it's done with it. We could potentially do this more generally with existential returns too, so that existentials are a viable abstraction tool even in predictable-performance mode.

-Joe

Surely the compiler could already do that within a single module? If it knows it’s going to get a bunch of cases from the same enum, it can have an optimised internal call. We could possibly allow exhaustive catching within the same module this way.

Most libraries will need the existential wrapper due to resilience concerns.

Karl · February 24, 2017, 4:54pm

Yeah that sounds like a much better solution.

I’m having a go at making the compiler tag all of the errors a function throws - theoretically I figure it should be similar to the CaptureWalker, just tagging a list of thrown Errors rather than captured variables, with some special annotation for masked errors or errors rethrown from another function or closure (if that function or closure comes from another module, it will always require a catch-all). That should allow later stages to figure out an envelope size or generate a private enum to have optimised calls, or serialise it with the module as a form of basic documentation. Within a single module, we could re-walk that list and warn about redundant catches or prove exhaustiveness, without the programmer having to write anything or changing our current model in any way.

I think we should take the softest possible course before embarking on something like typed-throws. Errors just _are_ version and implementation-specific; it’s in their nature. You shouldn’t rely on them happening or not. My understanding of the arguments for typed-throws is that the primary motivation is performance and exhaustive catching across modules (since we can do both for intra-modular errors regardless). The cost is that you will have to shuffle around your errors to essentially become per-function, making them harder to work with or re-throw (e.g. my function “read” now needs its own enum to include the errors from “open” + its own actual, unique errors), and the performance and exhaustive catching will only be available for @fixed enums, fixing your function _implementation_ in ways beyond what just adding @versioned does. Maybe you should just use an Either<T, Error> enum?

In practice, the benefits just aren’t there. Again, intra-modular errors can be made easier to live with, just with some compiler improvements. The user-facing side (exhaustive catching, redundant catching warnings), realistically, aren’t *that* hard. Optimisations to take advantage of that information can just happen when they happen.

- Karl

···

On 23 Feb 2017, at 17:19, Joe Groff <jgroff@apple.com> wrote:

On Feb 23, 2017, at 12:06 AM, Karl Wagner <razielim@gmail.com> wrote:

On 22 Feb 2017, at 21:13, Joe Groff <jgroff@apple.com> wrote:

On Feb 21, 2017, at 8:50 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 20, 2017, at 11:12 PM, John McCall <rjmccall@apple.com> wrote:

As you know, I still think that adding typed throws is the right thing to do. I understand your concern about “the feature could be misused” but the same thing is true about many other language features.

That's fair, but I do think there's an important difference here. The way I see it, typed-throws is really something of an expert feature, not because it's at all difficult to use, but the reverse: because it's easy to use without really thinking about the consequences. (And the benefits are pretty subtle, too.) I'm not saying that we should design it to be hard to use, but I think maybe it shouldn't immediately suggest itself, and it especially shouldn't come across as just a more specific version of throws.

Yeah, I agree that it will be appealing to people who don’t know better, but here’s the thing: the (almost certain) Swift design will prevent the bad thing from happening in practice.

Consider the barriers Swift already puts in place to prevent the bad thing (declaring an inappropriately narrow explicitly-specified throw signature) from happening:

1) First of all, you need to declare a public API. If it isn’t public, then there is no concern at all, you can evolve the implementation and clients together.

2) The Second problem depends on the number of errors it can throw. If there is exactly one type of error, the most common way to handle it is by returning optional. If you have one obvious failure mode with a value, then you throw that value. The most common case is where you can throw more than one sort of error, and therefore have an enum to describe it.

3) Third, your enum needs to be declared fragile in order to allow clients to enumerate their cases specifically.

The third step (having to mark your enum fragile, however it is spelled) is the biggest sign that you’re opting into a commitment that you should think really hard about. If folks don’t know that this is a big API commitment, then we have bigger problems.

One thing you didn’t mention is that boxing thrown values in an existential requires allocation in the general case. This may be unacceptable for some classes of Swift application (in the embedded / deep systems space) or simply undesirable because of the performance implication.

So, the performance implication cuts both ways. We can design the ABI for typed-throws so that, say, the callee initializes some buffer that's passed into it. That's an ABI that will kill some potential allocations in deep systems code, no question about it.

Agreed.

But in non-deep-systems code, we generally expect that error types will be resilient, which means that there are non-zero dynamic costs for allocating space on the stack for the error.

Proposed solution: ABI is that the callee takes in a register which is either a buffer address to fill in or null. On error, the callee returns the error pointer in a specific register. If there was a buffer passed in, it uses it, otherwise it allocates.

In practice, this allows the compiler to only pre-allocate the buffer when it knows the fixed size, otherwise the caller allocates on the heap on demand.

AFAICT, the cost of this API is only a “li rN, 0” in the normal path.

If you really want zero mallocs, it seems to me like creative stack accounting could get you there. In cases where we concretely know we're working with an existential, we can "explode" the existential into an opened type variable and a value of that type, and avoid allocating the existential representation. In the case of an untyped or resilient error, the callee could leave the error value and its type metadata on the stack somewhere without resetting the stack pointer, stick the address of that payload somewhere, and code would then propagate up to the catcher, who's responsible for consuming the error value and popping the stack when it's done with it. We could potentially do this more generally with existential returns too, so that existentials are a viable abstraction tool even in predictable-performance mode.

-Joe

Surely the compiler could already do that within a single module? If it knows it’s going to get a bunch of cases from the same enum, it can have an optimised internal call. We could possibly allow exhaustive catching within the same module this way.

Most libraries will need the existential wrapper due to resilience concerns.

Even across resilience boundaries, values of existential type don't fundamentally *need* to use the wrapper representation when we're know we're statically working with an existential, only when we're working with one as an unspecialized generic T == SomeExistentialType. An existential function argument can be exploded into a generic argument; existential returns and errors would require out-of-line allocation if shoehorned into traditional callstack discipline due to the variable size of the payload, but an implementation that avoids that is still possible. You're right that, for a protocol whose conformers are all known, there are further refinements possible, such as discriminating the type by a enum-like tag rather than an arbitrary type metadata pointer, and potentially giving the existential box a specialized representation with a known maximum size. (In deep enough systems code that isn't resilient at all and is compiled all at once, you could also conceivably attack the untyped-Error problem by determining the max size an Error buffer needs at compile time this way.)

-Joe