Throws? and throws!

The main problem with the Java approach is that, by giving programmer errors precise, observable semantics in your program, they no longer become purely programmer errors. Writing a bunch of code in a `try { ... } catch (NullPointerException *) { ... }` block to swallow null checks gives you a well-defined Java program, so static analysis tooling that tries to flag these problems has to contend with false positive "intended" uses of runtime errors. It also means error checks can never be optimized around or removed without breaking valid programs.

-Joe

···

On Jan 17, 2017, at 3:53 AM, Alex Blewitt via swift-evolution <swift-evolution@swift.org> wrote:

On 17 Jan 2017, at 11:46, Jeremy Pereira <jeremy.j.pereira@googlemail.com <mailto:jeremy.j.pereira@googlemail.com>> wrote:

On 17 Jan 2017, at 11:28, Alex Blewitt <alblue@apple.com <mailto:alblue@apple.com>> wrote:

On 17 Jan 2017, at 11:10, Jeremy Pereira via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 17 Jan 2017, at 02:38, thislooksfun via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I really hate to bring Java up, but I really do think it got at least one thing right with its error system, namely that one subset of error (namely `RuntimeException`), wouldn't be enforced by the compiler, but could optionally be caught.

I don’t entirely agree for two reasons:

1. If a runtime error is thrown and caught, there is no way to guarantee the logical consistency of the state of the running process because who knows which stack frames were unwound without cleaning up properly. There is no way to safely catch a run time exception and recover.

From a Java perspective, clearly that's not true.

It absolutely is true.

    int someMethod()
    {
        aquireSomeLock();
        doSomethingThatIsNotDeclaredToThrowAnException();
        releaseSomeLock();
    }

The general way to perform locking in Java is to use a synchronised block around a particular resource, and those are cleaned up by the Java VM when unwinding the stack, or to use a finally block to do the same. You can certainly write incorrect code in Java; the correct code would look like:

int someMethod() {
  try {
    acquireSomeLock();
    doSomethingThatIsNotDeclaredToThrowAnException();
  } finally {
    releaseSomeLock();
  }
}

It's more likely to be written as:

int someMethod() {
  synchronized(lockObject) {
    doSomethingThatIsNotDeclaredToThrowAnException();
  }
}

and where 'lockObject' is 'this' then the shorter

synchronized int someMethod() {
   doSomethingThatIsNotDeclaredToThrowAnException();
}

It may well be true for the current implementation of Swift, and there's questions about how to clean up objects with reference counts outstanding (since there's no garbage collector). It isn't necessarily the case that it isn't possible, but it does require some additional stack unwinding; and the implementation of that may be too cumbersome/impractical/undesirable to occur.

Not all resources are reference counts or memory allocations.

True; file descriptors are one such example. Again, the language has the ability to recover instead of blowing up when that occurs:

try(FileInputStream in = new FileInputStream("theFIle")) {
  // do stuff with input stream
} // input stream is closed

Compilers and tools can generate warnings for such cases, in the same way that other languages can warn about not freeing memory.

2. People abuse RuntimeException to simplify their error handling code: “if I throw a RuntimeException I don’t need a zillion catch clauses or throws declarations”. Furthermore, if a library uses RuntimeExceptions where it should be using Exceptions there is no way to know if its API has changed except by reading the (hopefully up to date) documentation.

Problem 2 makes me particularly bitter because JEE programmers seem to have learned that allowing code to throw null pointer exceptions has no real consequences for them so they become very cavalier about doing their null checks. The user sees an 500 error page, the sys admin gets a 200 line stack trace in the log, but the system carries on. If you are lucky enough to have the source code to diagnose the problem, it usually turns out that the exception was thrown on a line with eight chained method calls. When you do track the problem down, it turns out you forgot a line in the properties file or something similar but the programmer couldn’t be bothered to help you out because it was easier just to let the null pointer exception happen.

That's a pretty poor example, and in any case, the individual cause would allow the system to continue on processing subsequent requests, which is generally what's wanted. When you're working on large systems and with large data sets, there are generally always problematic items like this which have to be diagnosed sufficiently in order to be retried, or even handled manually. It has very little to do with the language and more to do with the quality of data which isn't something you always have control over.

No it’s not a poor example. I’ve seen it happen with real software in real production scenarios.

And if we're talking anecdata, I've seen large production systems where a particular message has been unable to be processed due to some incorrect data formatting (in some cases, an invalid UTF-8 sequence of bytes being passed in and treated as UTF-8). The general way is not to crash and return that and let it be processed a subsequent time, but ignored, put into a manual resolve queued, and keep going.

I like Swift’s error handling because programming errors (like force unwrapping nil) are punished mercilessly by process termination and errors caused by external factors cannot be completely ignored. You have to at least put an empty catch block somewhere.

This is one of the significant problems in Swift at the moment for server-side logic. It may make sense to do this from a single-user application, but writing a server which is designed to handle hundreds or thousands of simultaneous clients can suffer from the fact that one bad client request can take out everyone's connection. In the server working group and in existing tools like Kitura/Vapor/Perfect etc. it's a non-trivial problem to solve, other than using a CGI like model where each request is handled by a single worker process that can terminate independently of the other requests in flight.

I agree it’s a non trivial problem to resolve. I’m saying that the Java “solution” of RuntimeExceptions doesn’t resolve it.

Except you've not said /why/ the Java 'solution' of RuntimeExceptions doesn't resolve it. You've just used a couple of examples to prove it's possible to write bad Java code; but you can write bad code in any language.

It also means, from a purely pragmatic view, that control flow for
things that are really not reliably recoverable conditions (like index
out-of-range) is not cleanly separated from control flow for handling
recoverable things like dropped network conditions, so it can be really
hard to know that you're reliably shutting down the program when you
should, and that's not being subverted by some general “recovery” code.

···

on Tue Jan 17 2017, Joe Groff <swift-evolution@swift.org> wrote:

On Jan 17, 2017, at 3:53 AM, Alex Blewitt via swift-evolution > <swift-evolution@swift.org> wrote:

On 17 Jan 2017, at 11:46, Jeremy Pereira >>> <jeremy.j.pereira@googlemail.com > >>> <mailto:jeremy.j.pereira@googlemail.com>> >>> wrote:

On 17 Jan 2017, at 11:28, Alex Blewitt <alblue@apple.com <mailto:alblue@apple.com>> wrote:

On 17 Jan 2017, at 11:10, Jeremy Pereira via swift-evolution >>>>> <swift-evolution@swift.org >>>>> <mailto:swift-evolution@swift.org>> >>>>> wrote:

On 17 Jan 2017, at 02:38, thislooksfun via swift-evolution >>>>>> <swift-evolution@swift.org >>>>>> <mailto:swift-evolution@swift.org>> >>>>>> wrote:

I really hate to bring Java up, but I really do think it got at
least one thing right with its error system, namely that one
subset of error (namely `RuntimeException`), wouldn't be
enforced by the compiler, but could optionally be caught.

I don’t entirely agree for two reasons:

1. If a runtime error is thrown and caught, there is no way to
guarantee the logical consistency of the state of the running
process because who knows which stack frames were unwound without
cleaning up properly. There is no way to safely catch a run time
exception and recover.

From a Java perspective, clearly that's not true.

It absolutely is true.

    int someMethod()
    {
        aquireSomeLock();
        doSomethingThatIsNotDeclaredToThrowAnException();
        releaseSomeLock();
    }

The general way to perform locking in Java is to use a synchronised block around a particular resource, and those are cleaned up by the Java VM when unwinding the stack, or to use a finally block to do the same. You can certainly write incorrect code in Java; the correct code would look like:

int someMethod() {
  try {
    acquireSomeLock();
    doSomethingThatIsNotDeclaredToThrowAnException();
  } finally {
    releaseSomeLock();
  }
}

It's more likely to be written as:

int someMethod() {
  synchronized(lockObject) {
    doSomethingThatIsNotDeclaredToThrowAnException();
  }
}

and where 'lockObject' is 'this' then the shorter

synchronized int someMethod() {
   doSomethingThatIsNotDeclaredToThrowAnException();
}

It may well be true for the current implementation of Swift, and
there's questions about how to clean up objects with reference
counts outstanding (since there's no garbage collector). It isn't
necessarily the case that it isn't possible, but it does require
some additional stack unwinding; and the implementation of that
may be too cumbersome/impractical/undesirable to occur.

Not all resources are reference counts or memory allocations.

True; file descriptors are one such example. Again, the language has the ability to recover instead of blowing up when that occurs:

try(FileInputStream in = new FileInputStream("theFIle")) {
  // do stuff with input stream
} // input stream is closed

Compilers and tools can generate warnings for such cases, in the same way that other languages can warn about not freeing memory.

2. People abuse RuntimeException to simplify their error handling
code: “if I throw a RuntimeException I don’t need a zillion catch
clauses or throws declarations”. Furthermore, if a library uses
RuntimeExceptions where it should be using Exceptions there is no
way to know if its API has changed except by reading the
(hopefully up to date) documentation.

Problem 2 makes me particularly bitter because JEE programmers
seem to have learned that allowing code to throw null pointer
exceptions has no real consequences for them so they become very
cavalier about doing their null checks. The user sees an 500
error page, the sys admin gets a 200 line stack trace in the log,
but the system carries on. If you are lucky enough to have the
source code to diagnose the problem, it usually turns out that
the exception was thrown on a line with eight chained method
calls. When you do track the problem down, it turns out you
forgot a line in the properties file or something similar but the
programmer couldn’t be bothered to help you out because it was
easier just to let the null pointer exception happen.

That's a pretty poor example, and in any case, the individual
cause would allow the system to continue on processing subsequent
requests, which is generally what's wanted. When you're working on
large systems and with large data sets, there are generally always
problematic items like this which have to be diagnosed
sufficiently in order to be retried, or even handled manually. It
has very little to do with the language and more to do with the
quality of data which isn't something you always have control
over.

No it’s not a poor example. I’ve seen it happen with real software in real production scenarios.

And if we're talking anecdata, I've seen large production systems
where a particular message has been unable to be processed due to
some incorrect data formatting (in some cases, an invalid UTF-8
sequence of bytes being passed in and treated as UTF-8). The general
way is not to crash and return that and let it be processed a
subsequent time, but ignored, put into a manual resolve queued, and
keep going.

I like Swift’s error handling because programming errors (like
force unwrapping nil) are punished mercilessly by process
termination and errors caused by external factors cannot be
completely ignored. You have to at least put an empty catch block
somewhere.

This is one of the significant problems in Swift at the moment for
server-side logic. It may make sense to do this from a single-user
application, but writing a server which is designed to handle
hundreds or thousands of simultaneous clients can suffer from the
fact that one bad client request can take out everyone's
connection. In the server working group and in existing tools like
Kitura/Vapor/Perfect etc. it's a non-trivial problem to solve,
other than using a CGI like model where each request is handled by
a single worker process that can terminate independently of the
other requests in flight.

I agree it’s a non trivial problem to resolve. I’m saying that the Java “solution” of

RuntimeExceptions doesn’t resolve it.

Except you've not said /why/ the Java 'solution' of
RuntimeExceptions doesn't resolve it. You've just used a couple of
examples to prove it's possible to write bad Java code; but you can
write bad code in any language.

The main problem with the Java approach is that, by giving programmer
errors precise, observable semantics in your program, they no longer
become purely programmer errors. Writing a bunch of code in a `try {
... } catch (NullPointerException *) { ... }` block to swallow null
checks gives you a well-defined Java program, so static analysis
tooling that tries to flag these problems has to contend with false
positive "intended" uses of runtime errors. It also means error checks
can never be optimized around or removed without breaking valid
programs.

--
-Dave

Bringing it back towards the initial post, what if there was a
separation from true needs-to-take-down-the-entire-system trapping and
things like out-of-bounds and overflow errors which could stop at
thread/actor bounds (or in some cases even be recovered)?

The latter were the ones I was targeting with my proposal. They live
in this grey area, because honestly, they should be throwing errors if
not for the performance overhead and usability issues.

I fundamentally disagree with that statement. There is value in
declaring certain program behaviors illegal, and in general for things
like out-of-bounds access and overflow no sensible recovery (where
“recovery” means something that would allow the program to continue
reliably) is possible.

I think we do fundamentally disagree. I know I come from a very different background (Human-Computer Interaction & Human Factors) than most people here, and I am kind of the odd man out, but I have never understood this viewpoint for anything but the most severe cases where the system itself is in danger of being compromised (certainly not for an index out of bounds). In my mind “fail fast” is great for iterating in development builds, but once you are deploying, the user’s needs should come ahead of the programmer’s.

Shouldn’t a system be as robust as possible

Yes

and try to minimize the fallout from any failure point?

That is in direct conflict with the robustness
Once an error is detected that is not handled by the immediate code, it must be assumed that a worst-case scenario happened. And further damage to the user can only be prevent by bringing down the app. Even if that means losing all work in progress.

A compromised system must be prevent from accessing any resources. Once a system is compromised the risk to the user is *much* higher than simply loosing work in progress. He might loose his job, career, etc.

Rien.

···

On 18 Jan 2017, at 08:54, Jonathan Hull via swift-evolution <swift-evolution@swift.org> wrote:

On Jan 17, 2017, at 7:13 PM, Dave Abrahams <dabrahams@apple.com> wrote:
on Tue Jan 17 2017, Jonathan Hull <jhull-AT-gbis.com> wrote:

I don’t consider crashing and losing all of the user’s data minimal. It used to be that something like dividing by zero took down the entire machine. Now we mimic that by crashing the application, even though it isn’t strictly necessary. Wouldn’t it be even better if we only took down the current operation, notified the user about what happened and continue on?

Swift does a great job of using forcing functions (like optionals) to make some errors impossible, and this is literally the opposite of that. This requires the programmer to remember to add a check that the number is within certain bounds, but there is no reminder for them to do that. The failure is silent (i.e. there isn’t a ‘!’ or ’try' to mark that it is a possibility), at runtime, under certain conditions and not others. It is a recipe for bugs which cause a crash for the user.

If we wanted fool-proof arrays, then the subscript would return an optional, forcing the programmer to think about and deal with the possibility of out-of-bounds failure (that is how we handle dictionary lookups after all). If I remember correctly, we don’t do that because of performance. Instead we ask the programmer to remember to check against the array count first, just like we used to ask the programmer to remember to check for nil pointers in C++.

The idea that a programmer can or should be perfect is a lovely fantasy… My CS teachers always talked about how it would encourage bad programming if there wasn’t punishment for making these types of mistakes.

I don’t see the value in punishing the user for the programmer’s mistake, and I have rarely seen a case where sensible recovery wouldn’t be possible (given Swift’s error handling). In most real-world applications, you would just end up cancelling whatever operation was happening, reverting to the state before it, and notifying the user of the problem. The programming languages which get into trouble are the ones which treat everything as valid, so they just keep barreling on, overwriting data or something like that. We don’t have that problem though, since we have sensible error handling that can be used to fail an operation and get things back to a previous state (or better yet, avoiding overwriting the state until after the operation has succeeded). We should aim for robustness, and crashing isn’t robust.

Please note: I am not saying we allow memory access out of bounds. We are still triggering an error state when these things happen (either returning nil or throwing an error), we’re just not crashing the entire program because of it.

My solution was to give the compiler a way to know that this was the
desired behavior and optimize the throwing away unless it was
explicitly requested.

I guess another option would be to introduce a new concept for this
grey type of error. Maybe instead of ‘fatalError’ you have something
with a different name saying “this should only take down the current
actor”… and then you add a well defined process for cleanup.

I would still really like to see the ability to turn this type of
thing into normal throwing error handling, so maybe something like
‘fatalThrow’ which takes the same information as ‘throw’, so that it
can be converted to a standard throw by the caller, but otherwise
traps and takes down the actor. That would make certain types of
algorithms much simpler for me.

Thanks,
Jon

On Jan 17, 2017, at 11:49 AM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:

on Mon Jan 16 2017, Chris Lattner <swift-evolution@swift.org >>>> <mailto:swift-evolution@swift.org>> wrote:

On Jan 16, 2017, at 3:57 PM, David Waite via swift-evolution >>>>> <swift-evolution@swift.org> wrote:

My interpretation is that he was advocating a future where a
precondition’s failure killed less than the entire process. Instead,
shut down some smaller portion like a thread, actor, or container

like .Net's app domains (which for those more familiar with
Javascript could be loosely compared with Web Workers).

Today - if you wanted a Swift server where overflowing addition
didn’t interrupt your service for multiple users, you would need to
use something like a pre-fork model (with each request handled by a
separate swift process)

That's the difference between CLI and desktop apps where the process
is providing services for a single user, and a server where it may
be providing a service for thousands or millions of users.

Agreed, I’d also really like to see this some day. It seems like a
natural outgrowth of the concurrency model, if it goes the direction
of actors.

If you’re interested, I speculated on this direction in this talk:
http://researcher.watson.ibm.com/researcher/files/us-lmandel/lattner.pdf
<http://researcher.watson.ibm.com/researcher/files/us-lmandel/lattner.pdf
<http://researcher.watson.ibm.com/researcher/files/us-lmandel/lattner.pdf&gt;&gt;

I totally support the idea of emergency shutown measures
(e.g. save document for recovery),

In general, though, when a precondition is violated, it means your
program state is compromised in an arbitrarily bad way. Unfortunately,
that applies equally across process boundaries as it does across thread
boundaries, if there's some kind of static guarantee of safety as might
be provided by actors. This means you need a way to make decisions
about which kinds of precondition violations should be considered
recoverable as long as you're willing to abandon the job, and which
really do need to be fatal for the whole process... and I don't know if
anyone's really ever figured that problem out. It'd be cool if Swift
could solve it.

--
-Dave

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution
<https://lists.swift.org/mailman/listinfo/swift-evolution&gt;

--
-Dave

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Bringing it back towards the initial post, what if there was a
separation from true needs-to-take-down-the-entire-system trapping and
things like out-of-bounds and overflow errors which could stop at
thread/actor bounds (or in some cases even be recovered)?

The latter were the ones I was targeting with my proposal. They live
in this grey area, because honestly, they should be throwing errors if
not for the performance overhead and usability issues.

I fundamentally disagree with that statement. There is value in
declaring certain program behaviors illegal, and in general for things
like out-of-bounds access and overflow no sensible recovery (where
“recovery” means something that would allow the program to continue
reliably) is possible.

I think we do fundamentally disagree. I know I come from a very
different background (Human-Computer Interaction & Human Factors) than
most people here, and I am kind of the odd man out, but I have never
understood this viewpoint for anything but the most severe cases where
the system itself is in danger of being compromised (certainly not for
an index out of bounds). In my mind “fail fast” is great for
iterating in development builds, but once you are deploying, the
user’s needs should come ahead of the programmer’s.

The question is really whether continuing when the program no longer
understands its own state is in the user's best interest. Bad
consequences are just as easy to imagine as benign ones.

Shouldn’t a system be as robust as possible and try to minimize the
fallout from any failure point? I don’t consider crashing and losing
all of the user’s data minimal.

That's why we need emergency shutdown facilities.

···

on Tue Jan 17 2017, Jonathan Hull <jhull-AT-gbis.com> wrote:

On Jan 17, 2017, at 7:13 PM, Dave Abrahams <dabrahams@apple.com> wrote:
on Tue Jan 17 2017, Jonathan Hull <jhull-AT-gbis.com <http://jhull-at-gbis.com/&gt;&gt; wrote:

--
-Dave

I really hate to bring Java up, but I really do think it got at least one thing right with its error system, namely that one subset of error (namely `RuntimeException`), wouldn't be enforced by the compiler, but could optionally be caught.

I don’t entirely agree for two reasons:

1. If a runtime error is thrown and caught, there is no way to guarantee the logical consistency of the state of the running process because who knows which stack frames were unwound without cleaning up properly. There is no way to safely catch a run time exception and recover.

From a Java perspective, clearly that's not true.

It absolutely is true.

    int someMethod()
    {
        aquireSomeLock();
        doSomethingThatIsNotDeclaredToThrowAnException();
        releaseSomeLock();
    }

The general way to perform locking in Java is to use a synchronised block around a particular resource, and those are cleaned up by the Java VM when unwinding the stack, or to use a finally block to do the same. You can certainly write incorrect code in Java; the correct code would look like:

int someMethod() {
  try {
    acquireSomeLock();
    doSomethingThatIsNotDeclaredToThrowAnException();
  } finally {
    releaseSomeLock();
  }
}

Yes, which is why catching runtime errors is a problem. You need all the boilerplate of try { … } finally { … } every time you have a resource like this just in case any of the methods you call while you have the resource throw a run time exception. If you have more than one such resource in a scope you start to need pyramids of try … finally structures. It gets quite onerous, so people don’t do it or they don’t do it properly.

And also people often make mistakes when writing their try finally blocks as you did. The correct way to code my example is

int someMethod()
{
  acquireSomeLock();
  try
  {
    doSomethingThatIsNotDeclaredToThrowAnException();
  }
  finally
  {
    releaseSomeLock();
  }
}

(you don’t want to call releaseSomeLock() if the exception is throw while acquiring it - assuming acquireSomeLock() was properly designed).

It's more likely to be written as:

int someMethod() {
  synchronized(lockObject) {
    doSomethingThatIsNotDeclaredToThrowAnException();
  }
}

Here you are assuming all resources that need to be acquired/released can be replaced with Java locks. You are also assuming that all cases where logical consistency is needed are locks or memory. What if you have a sequence of operations that must appear to be performed atomically in order to keep the model consistent?

synchronized int someMethodThatNeedsToBeAtomic()
{
    doOperationOne();
    doOperationTwo();
    doOperationThree();
    doOperationFour();
}

If I want to be able to recover from RunTimeExceptions, I need to catch them here and roll back any work already done. That means multiple try catch blocks or some other logic so I know how much I need to roll back. With Swift, if a method is not declared to throw, you know you don’t need write code to handle errors. The same applies to Java but only if you do not try to recover from RuntimeExceptions.

True; file descriptors are one such example. Again, the language has the ability to recover instead of blowing up when that occurs:

try(FileInputStream in = new FileInputStream("theFIle")) {
  // do stuff with input stream
} // input stream is closed

This is a recent feature with Java and it’s certainly a step in the right direction.

No it’s not a poor example. I’ve seen it happen with real software in real production scenarios.

And if we're talking anecdata, I've seen large production systems where a particular message has been unable to be processed due to some incorrect data formatting (in some cases, an invalid UTF-8 sequence of bytes being passed in and treated as UTF-8). The general way is not to crash and return that and let it be processed a subsequent time, but ignored, put into a manual resolve queued, and keep going.

No, the correct way to handle errors in input data is validate it and/or throw a checked exception which can be handled and recovered safely in Java. In your case, you don’t want an exception in the log, you want a message saying what went wrong, where and how to fix it

I like Swift’s error handling because programming errors (like force unwrapping nil) are punished mercilessly by process termination and errors caused by external factors cannot be completely ignored. You have to at least put an empty catch block somewhere.

This is one of the significant problems in Swift at the moment for server-side logic. It may make sense to do this from a single-user application, but writing a server which is designed to handle hundreds or thousands of simultaneous clients can suffer from the fact that one bad client request can take out everyone's connection. In the server working group and in existing tools like Kitura/Vapor/Perfect etc. it's a non-trivial problem to solve, other than using a CGI like model where each request is handled by a single worker process that can terminate independently of the other requests in flight.

I agree it’s a non trivial problem to resolve. I’m saying that the Java “solution” of RuntimeExceptions doesn’t resolve it.

Except you've not said /why/ the Java 'solution' of RuntimeExceptions doesn't resolve it. You've just used a couple of examples to prove it's possible to write bad Java code; but you can write bad code in any language.

I’ve already said: unchecked exceptions cannot be recovered safely (see everything above).

···

On 17 Jan 2017, at 11:53, Alex Blewitt <alblue@apple.com> wrote:

On 17 Jan 2017, at 11:46, Jeremy Pereira <jeremy.j.pereira@googlemail.com> wrote:

On 17 Jan 2017, at 11:28, Alex Blewitt <alblue@apple.com> wrote:

On 17 Jan 2017, at 11:10, Jeremy Pereira via swift-evolution <swift-evolution@swift.org> wrote:

On 17 Jan 2017, at 02:38, thislooksfun via swift-evolution <swift-evolution@swift.org> wrote:

Agreed. The “try” syntax is part of a decision to use errors to represent conditions recoverable by business logic. This might be a network connection closed, but shouldn’t be used for conditions like memory allocation failed, a KILL signal, or detection of corrupted data within an application bundle.

In the middle are avoidable runtime errors like array indexing out-of-bounds and math overflow. Some business logic might care about these, in particular logic which deals with arbitrary user input. However, using ‘try’ syntax would result in both an explosion of needing to specify try in expressions, and in errors which can’t really be recovered in typical business logic.

I see this as three regions of error conditions - system level errors that the developer isn’t expected to try and recover from, application-level errors that the developer is likely going to want to control the behavior of, and in between the runtime errors which some logic might care about, but which in most cases should be treated as a system level error.

Java roughly has Error, Exception, and RuntimeException to correspond to these, with a few differences which I generally consider to be flaws in the Java language (RuntimeException is under Exception, RuntimeExceptions percolate up the entire stack rather than being escalated to Errors if unhandled).

I can envision a future state where:
- system level errors can have atexit/terminate-style shutdown hooks. These do not support recovery, but can be used to attempt a cleaner shutdown
- functions which emit runtime errors (such as math overflow) can support a call form which returns an error rather than precondition failure, but it is opt in and optional for callers. The runtime errors do not propagate automatically, and callers which do not request it get precondition failures
- the possibility of containerization so that runtime errors and certain system level errors terminate a thread/actor/app domain rather than the whole process

-DW

···

On Jan 17, 2017, at 1:38 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:

It also means, from a purely pragmatic view, that control flow for
things that are really not reliably recoverable conditions (like index
out-of-range) is not cleanly separated from control flow for handling
recoverable things like dropped network conditions, so it can be really
hard to know that you're reliably shutting down the program when you
should, and that's not being subverted by some general “recovery” code.

That's certainly true of code that makes unaudited use of `unsafe` constructs that can violate safety without any checking. It's my hope that our normal safety checks are thorough and fire early enough that your subprocess would crash before wide-system compromise happens. In an "actor" or similar model, even if we decide we don't want to pay for unwinding to fully clean up after the crashed actor, that crash could still at least be noted by a coordinator actor, which in your server situation could handle the problem by not accepting any new connections and letting its existing connections finish before restarting the process, or in an iOS-like mobile situation could trigger serialization of the current user state so that the process can be transparently killed and restarted. In either situation, perhaps we'd want to "taint" actors that use unsafe constructs so that their failure can't be recovered at all.

-Joe

···

On Jan 18, 2017, at 12:04 AM, Rien via swift-evolution <swift-evolution@swift.org> wrote:

On 18 Jan 2017, at 08:54, Jonathan Hull via swift-evolution <swift-evolution@swift.org> wrote:

On Jan 17, 2017, at 7:13 PM, Dave Abrahams <dabrahams@apple.com> wrote:

on Tue Jan 17 2017, Jonathan Hull <jhull-AT-gbis.com> wrote:

Bringing it back towards the initial post, what if there was a
separation from true needs-to-take-down-the-entire-system trapping and
things like out-of-bounds and overflow errors which could stop at
thread/actor bounds (or in some cases even be recovered)?

The latter were the ones I was targeting with my proposal. They live
in this grey area, because honestly, they should be throwing errors if
not for the performance overhead and usability issues.

I fundamentally disagree with that statement. There is value in
declaring certain program behaviors illegal, and in general for things
like out-of-bounds access and overflow no sensible recovery (where
“recovery” means something that would allow the program to continue
reliably) is possible.

I think we do fundamentally disagree. I know I come from a very different background (Human-Computer Interaction & Human Factors) than most people here, and I am kind of the odd man out, but I have never understood this viewpoint for anything but the most severe cases where the system itself is in danger of being compromised (certainly not for an index out of bounds). In my mind “fail fast” is great for iterating in development builds, but once you are deploying, the user’s needs should come ahead of the programmer’s.

Shouldn’t a system be as robust as possible

Yes

and try to minimize the fallout from any failure point?

That is in direct conflict with the robustness
Once an error is detected that is not handled by the immediate code, it must be assumed that a worst-case scenario happened. And further damage to the user can only be prevent by bringing down the app. Even if that means losing all work in progress.

A compromised system must be prevent from accessing any resources. Once a system is compromised the risk to the user is *much* higher than simply loosing work in progress. He might loose his job, career, etc.

We already have three ways of handling integer overflow:

+ traps
&+ overflows
addWithOverflow overflows and sets a flag if it does so

This supports all possible ways of handling the issue: you can trap, you
can get a result and not care about the overflow flag, or you can get the
result *and* the overflow flag. You can even discard the result and work
with only the flag.

I don't see how this doesn't meet exactly what you and Jonathan Hull
describe: it is opt-in, optional for callers, and does not propagate. It
doesn't require try or any other cumbersome syntax, and it allows you to
just compute the operation first and handle the overflow later only if it
occurs.

For array indexing, there is an outstanding proposal for more lenient
subscripts. In the meantime, you can trivially implement your own. Such a
facility would also behave according to your stated requirements.

···

On Tue, Jan 17, 2017 at 18:03 David Waite via swift-evolution < swift-evolution@swift.org> wrote:

On Jan 17, 2017, at 1:38 PM, Dave Abrahams via swift-evolution < > swift-evolution@swift.org> wrote:

It also means, from a purely pragmatic view, that control flow for
things that are really not reliably recoverable conditions (like index
out-of-range) is not cleanly separated from control flow for handling
recoverable things like dropped network conditions, so it can be really
hard to know that you're reliably shutting down the program when you
should, and that's not being subverted by some general “recovery” code.

Agreed. The “try” syntax is part of a decision to use errors to represent
conditions recoverable by business logic. This might be a network
connection closed, but shouldn’t be used for conditions like memory
allocation failed, a KILL signal, or detection of corrupted data within an
application bundle.

In the middle are avoidable runtime errors like array indexing
out-of-bounds and math overflow. Some business logic might care about
these, in particular logic which deals with arbitrary user input. However,
using ‘try’ syntax would result in both an explosion of needing to specify
try in expressions, and in errors which can’t really be recovered in
typical business logic.

I see this as three regions of error conditions - system level errors that
the developer isn’t expected to try and recover from, application-level
errors that the developer is likely going to want to control the behavior
of, and in between the runtime errors which some logic might care about,
but which in most cases should be treated as a system level error.

Java roughly has Error, Exception, and RuntimeException to correspond to
these, with a few differences which I generally consider to be flaws in the
Java language (RuntimeException is under Exception, RuntimeExceptions
percolate up the entire stack rather than being escalated to Errors if
unhandled).

I can envision a future state where:
- system level errors can have atexit/terminate-style shutdown hooks.
These do not support recovery, but can be used to attempt a cleaner shutdown
- functions which emit runtime errors (such as math overflow) can support
a call form which returns an error rather than precondition failure, but it
is opt in and optional for callers. The runtime errors do not propagate
automatically, and callers which do not request it get precondition failures
- the possibility of containerization so that runtime errors and certain
system level errors terminate a thread/actor/app domain rather than the
whole process

-DW
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

This seems like basically the right approach to me. It means we don't make any effort to "clean up" the failing actor — essentially, it's treated as if it were just deadlocked — which means we don't pay the pervasive code-size costs of unwinding. That's even fairly likely to leave the process in a state that can still be usefully debugged (as opposed to unwinding stacks, which completely destroys the execution context). But there's still an opportunity to react and try to wind up other tasks.

I'm not sure it makes any sense to call out actors that have used unsafe constructs as somehow specially unrecoverable. If the concern is that the unsafe code may corrupt the other actors, well, that's true, but (1) that implies that you have to forbid recovery if *any* actor has used unsafe constructs, since low-level corruption can be passed between actors when they communicate normally, and (2) that's equally true of all sorts of high-level corruption that don't depend on unsafe constructs, and which the failing assertion may be the first indication of.

John.

···

On Jan 18, 2017, at 9:06 AM, Joe Groff via swift-evolution <swift-evolution@swift.org> wrote:

On Jan 18, 2017, at 12:04 AM, Rien via swift-evolution <swift-evolution@swift.org> wrote:

On 18 Jan 2017, at 08:54, Jonathan Hull via swift-evolution <swift-evolution@swift.org> wrote:

On Jan 17, 2017, at 7:13 PM, Dave Abrahams <dabrahams@apple.com> wrote:

on Tue Jan 17 2017, Jonathan Hull <jhull-AT-gbis.com> wrote:

Bringing it back towards the initial post, what if there was a
separation from true needs-to-take-down-the-entire-system trapping and
things like out-of-bounds and overflow errors which could stop at
thread/actor bounds (or in some cases even be recovered)?

The latter were the ones I was targeting with my proposal. They live
in this grey area, because honestly, they should be throwing errors if
not for the performance overhead and usability issues.

I fundamentally disagree with that statement. There is value in
declaring certain program behaviors illegal, and in general for things
like out-of-bounds access and overflow no sensible recovery (where
“recovery” means something that would allow the program to continue
reliably) is possible.

I think we do fundamentally disagree. I know I come from a very different background (Human-Computer Interaction & Human Factors) than most people here, and I am kind of the odd man out, but I have never understood this viewpoint for anything but the most severe cases where the system itself is in danger of being compromised (certainly not for an index out of bounds). In my mind “fail fast” is great for iterating in development builds, but once you are deploying, the user’s needs should come ahead of the programmer’s.

Shouldn’t a system be as robust as possible

Yes

and try to minimize the fallout from any failure point?

That is in direct conflict with the robustness
Once an error is detected that is not handled by the immediate code, it must be assumed that a worst-case scenario happened. And further damage to the user can only be prevent by bringing down the app. Even if that means losing all work in progress.

A compromised system must be prevent from accessing any resources. Once a system is compromised the risk to the user is *much* higher than simply loosing work in progress. He might loose his job, career, etc.

That's certainly true of code that makes unaudited use of `unsafe` constructs that can violate safety without any checking. It's my hope that our normal safety checks are thorough and fire early enough that your subprocess would crash before wide-system compromise happens. In an "actor" or similar model, even if we decide we don't want to pay for unwinding to fully clean up after the crashed actor, that crash could still at least be noted by a coordinator actor, which in your server situation could handle the problem by not accepting any new connections and letting its existing connections finish before restarting the process, or in an iOS-like mobile situation could trigger serialization of the current user state so that the process can be transparently killed and restarted. In either situation, perhaps we'd want to "taint" actors that use unsafe constructs so that their failure can't be recovered at all.

I rewrote some code which used NSDecimalNumber to use Decimal and it went from 20 lines of hard to follow code to 2 lines of easy to follow expressions. ‘2+2’ is much easier to understand at a glance than having a sentence for each operation.

Also the control flow is completely different. With one, you have try and then a full equation, and you catch and handle different cases if something went wrong (overflow, underflow, divide by zero). With the other you have to have a guard AFTER each line which uses a previous result (or a pyramid of doom), potentially repeating the same error handling code over and over:

  let (a,o1) = Int.addWithOverflow(x,y)
  guard o1 == false else {throw MyError}
  let (b,o2) = Int.multiplyWithOverflow(a,z)
  guard o2 == false else {throw MyError}
  let (c,o3) = Int.subtractWithOverflow(b,z)
  guard o3 == false else {throw MyError}
  return c

vs:

  do{
      return try ((x+y)*z)-z
  }catch _ {
      throw MyError
  }

Thanks,
Jon

···

On Jan 17, 2017, at 4:22 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:

We already have three ways of handling integer overflow:

+ traps
&+ overflows
addWithOverflow overflows and sets a flag if it does so

This supports all possible ways of handling the issue: you can trap, you can get a result and not care about the overflow flag, or you can get the result *and* the overflow flag. You can even discard the result and work with only the flag.

I don't see how this doesn't meet exactly what you and Jonathan Hull describe: it is opt-in, optional for callers, and does not propagate. It doesn't require try or any other cumbersome syntax, and it allows you to just compute the operation first and handle the overflow later only if it occurs.

For array indexing, there is an outstanding proposal for more lenient subscripts. In the meantime, you can trivially implement your own. Such a facility would also behave according to your stated requirements.

On Tue, Jan 17, 2017 at 18:03 David Waite via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Jan 17, 2017, at 1:38 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

It also means, from a purely pragmatic view, that control flow for
things that are really not reliably recoverable conditions (like index
out-of-range) is not cleanly separated from control flow for handling
recoverable things like dropped network conditions, so it can be really
hard to know that you're reliably shutting down the program when you
should, and that's not being subverted by some general “recovery” code.

Agreed. The “try” syntax is part of a decision to use errors to represent conditions recoverable by business logic. This might be a network connection closed, but shouldn’t be used for conditions like memory allocation failed, a KILL signal, or detection of corrupted data within an application bundle.

In the middle are avoidable runtime errors like array indexing out-of-bounds and math overflow. Some business logic might care about these, in particular logic which deals with arbitrary user input. However, using ‘try’ syntax would result in both an explosion of needing to specify try in expressions, and in errors which can’t really be recovered in typical business logic.

I see this as three regions of error conditions - system level errors that the developer isn’t expected to try and recover from, application-level errors that the developer is likely going to want to control the behavior of, and in between the runtime errors which some logic might care about, but which in most cases should be treated as a system level error.

Java roughly has Error, Exception, and RuntimeException to correspond to these, with a few differences which I generally consider to be flaws in the Java language (RuntimeException is under Exception, RuntimeExceptions percolate up the entire stack rather than being escalated to Errors if unhandled).

I can envision a future state where:
- system level errors can have atexit/terminate-style shutdown hooks. These do not support recovery, but can be used to attempt a cleaner shutdown
- functions which emit runtime errors (such as math overflow) can support a call form which returns an error rather than precondition failure, but it is opt in and optional for callers. The runtime errors do not propagate automatically, and callers which do not request it get precondition failures
- the possibility of containerization so that runtime errors and certain system level errors terminate a thread/actor/app domain rather than the whole process

-DW
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I rewrote some code which used NSDecimalNumber to use Decimal and it went
from 20 lines of hard to follow code to 2 lines of easy to follow
expressions. ‘2+2’ is much easier to understand at a glance than having a
sentence for each operation.

So, again, your particular problem isn't about whether Swift lets you
handle errors in a certain way--it already does. Your objection is about
the spelling.

Also the control flow is completely different. With one, you have try and
then a full equation, and you catch and handle different cases if something
went wrong (overflow, underflow, divide by zero). With the other you have
to have a guard AFTER each line which uses a previous result (or a pyramid
of doom), potentially repeating the same error handling code over and over:

You already have the option of making this work for you _exactly_ as you
like:


enum ArithmeticError : Error {

  case add, multiply, subtract, divide

}

infix operator &&+ : AdditionPrecedence

infix operator &&- : AdditionPrecedence

infix operator &&* : MultiplicationPrecedence

infix operator &&/ : MultiplicationPrecedence

extension Int {

  static func &&+ (lhs: Int, rhs: Int) throws -> Int {

    let (result, flag) = Int.addWithOverflow(lhs, rhs)

    guard !flag else { throw ArithmeticError.add }

    return result

  }

}

This requires absolutely no redesign of Swift error handling. You can even
customize how fine-grained the error messages are.

By contrast, I think the present topic of this thread--identifying ways in
which a logic error might bring down only a single process, or whether
that's even a desirable thing--is a sorely needed discussion which will
require deep support baked into the compiler.

let (a,o1) = Int.addWithOverflow(x,y)

···

On Tue, Jan 17, 2017 at 6:50 PM, Jonathan Hull <jhull@gbis.com> wrote:

guard o1 == false else {throw MyError}
let (b,o2) = Int.multiplyWithOverflow(a,z)
guard o2 == false else {throw MyError}
let (c,o3) = Int.subtractWithOverflow(b,z)
guard o3 == false else {throw MyError}
return c

vs:

do{
    return try ((x+y)*z)-z
}catch _ {
    throw MyError
}

Thanks,
Jon

On Jan 17, 2017, at 4:22 PM, Xiaodi Wu via swift-evolution < > swift-evolution@swift.org> wrote:

We already have three ways of handling integer overflow:

+ traps
&+ overflows
addWithOverflow overflows and sets a flag if it does so

This supports all possible ways of handling the issue: you can trap, you
can get a result and not care about the overflow flag, or you can get the
result *and* the overflow flag. You can even discard the result and work
with only the flag.

I don't see how this doesn't meet exactly what you and Jonathan Hull
describe: it is opt-in, optional for callers, and does not propagate. It
doesn't require try or any other cumbersome syntax, and it allows you to
just compute the operation first and handle the overflow later only if it
occurs.

For array indexing, there is an outstanding proposal for more lenient
subscripts. In the meantime, you can trivially implement your own. Such a
facility would also behave according to your stated requirements.

On Tue, Jan 17, 2017 at 18:03 David Waite via swift-evolution < > swift-evolution@swift.org> wrote:

On Jan 17, 2017, at 1:38 PM, Dave Abrahams via swift-evolution < >> swift-evolution@swift.org> wrote:

It also means, from a purely pragmatic view, that control flow for
things that are really not reliably recoverable conditions (like index
out-of-range) is not cleanly separated from control flow for handling
recoverable things like dropped network conditions, so it can be really
hard to know that you're reliably shutting down the program when you
should, and that's not being subverted by some general “recovery” code.

Agreed. The “try” syntax is part of a decision to use errors to represent
conditions recoverable by business logic. This might be a network
connection closed, but shouldn’t be used for conditions like memory
allocation failed, a KILL signal, or detection of corrupted data within an
application bundle.

In the middle are avoidable runtime errors like array indexing
out-of-bounds and math overflow. Some business logic might care about
these, in particular logic which deals with arbitrary user input. However,
using ‘try’ syntax would result in both an explosion of needing to specify
try in expressions, and in errors which can’t really be recovered in
typical business logic.

I see this as three regions of error conditions - system level errors
that the developer isn’t expected to try and recover from,
application-level errors that the developer is likely going to want to
control the behavior of, and in between the runtime errors which some logic
might care about, but which in most cases should be treated as a system
level error.

Java roughly has Error, Exception, and RuntimeException to correspond to
these, with a few differences which I generally consider to be flaws in the
Java language (RuntimeException is under Exception, RuntimeExceptions
percolate up the entire stack rather than being escalated to Errors if
unhandled).

I can envision a future state where:
- system level errors can have atexit/terminate-style shutdown hooks.
These do not support recovery, but can be used to attempt a cleaner shutdown
- functions which emit runtime errors (such as math overflow) can support
a call form which returns an error rather than precondition failure, but it
is opt in and optional for callers. The runtime errors do not propagate
automatically, and callers which do not request it get precondition failures
- the possibility of containerization so that runtime errors and certain
system level errors terminate a thread/actor/app domain rather than the
whole process

-DW
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I agree with John on both points. To the second, keep in mind that lots of safe constructs are built in terms of unsafe constructs (e.g. Array).

-Chris

···

On Jan 18, 2017, at 10:45 AM, John McCall via swift-evolution <swift-evolution@swift.org> wrote:

That's certainly true of code that makes unaudited use of `unsafe` constructs that can violate safety without any checking. It's my hope that our normal safety checks are thorough and fire early enough that your subprocess would crash before wide-system compromise happens. In an "actor" or similar model, even if we decide we don't want to pay for unwinding to fully clean up after the crashed actor, that crash could still at least be noted by a coordinator actor, which in your server situation could handle the problem by not accepting any new connections and letting its existing connections finish before restarting the process, or in an iOS-like mobile situation could trigger serialization of the current user state so that the process can be transparently killed and restarted. In either situation, perhaps we'd want to "taint" actors that use unsafe constructs so that their failure can't be recovered at all.

This seems like basically the right approach to me. It means we don't make any effort to "clean up" the failing actor — essentially, it's treated as if it were just deadlocked — which means we don't pay the pervasive code-size costs of unwinding. That's even fairly likely to leave the process in a state that can still be usefully debugged (as opposed to unwinding stacks, which completely destroys the execution context). But there's still an opportunity to react and try to wind up other tasks.

I'm not sure it makes any sense to call out actors that have used unsafe constructs as somehow specially unrecoverable. If the concern is that the unsafe code may corrupt the other actors, well, that's true, but (1) that implies that you have to forbid recovery if *any* actor has used unsafe constructs, since low-level corruption can be passed between actors when they communicate normally, and (2) that's equally true of all sorts of high-level corruption that don't depend on unsafe constructs, and which the failing assertion may be the first indication of.