Currently, it is impossible for XCTest to unit test assert and precondition failures because they kill the process. And those are important to unit test: how would you test NSArray's objectAtIndex bound conditions? Currently, the standard library tests those with StdlibUnittest, a small piece of code which runs those tests in a forked process.
I want to start a proposal but I'm not sure if it should come as a XCTest improvement which spawns a process or if the language should implement assert and precondition as a special kind of throw?
Currently, it is impossible for XCTest to unit test assert and precondition failures because they kill the process. And those are important to unit test: how would you test NSArray's objectAtIndex bound conditions? Currently, the standard library tests those with StdlibUnittest, a small piece of code which runs those tests in a forked process.
I want to start a proposal but I'm not sure if it should come as a XCTest improvement which spawns a process or if the language should implement assert and precondition as a special kind of throw?
One way to start a proposal is to try and work through what the implications of each approach are, decide which is best, and then write that up in a proposal explaining why you chose the approach you did.
I suspect that spawning a process is the best approach, because it can cover more cases than the language built in features, and doesn't require language support, but I would be interested in seeing an in depth study of what this would mean for the framework and the APIs.
- Daniel
···
On Dec 13, 2015, at 2:03 AM, David Hart via swift-evolution <swift-evolution@swift.org> wrote:
+1 from me. I'm really annoyed by the fact I can't reach 100% code coverage
due to this :)
Being able to test them is very valuable. Take Swift's promise to be safe
for example. If you can't test the behavior in cases where an array index
was used which is out of bounds then you can't test even the promise to be
safe.
···
On Sun, Dec 13, 2015 at 11:03 AM, David Hart via swift-evolution < swift-evolution@swift.org> wrote:
Currently, it is impossible for XCTest to unit test assert and
precondition failures because they kill the process. And those are
important to unit test: how would you test NSArray's objectAtIndex bound
conditions? Currently, the standard library tests those with
StdlibUnittest, a small piece of code which runs those tests in a forked
process.
I want to start a proposal but I'm not sure if it should come as a XCTest
improvement which spawns a process or if the language should implement
assert and precondition as a special kind of throw?
As it stands today, having the test harness run the test under a separate process, the way StdlibUnittest does, is the best approach IMO. Making failure traps even conditionally recoverable would be a fairly large semantic design and codegen complication. In the bright distant future where we have an actor model, then actor failures might be isolatable.
-Joe
···
On Dec 13, 2015, at 2:03 AM, David Hart via swift-evolution <swift-evolution@swift.org> wrote:
Currently, it is impossible for XCTest to unit test assert and precondition failures because they kill the process. And those are important to unit test: how would you test NSArray's objectAtIndex bound conditions? Currently, the standard library tests those with StdlibUnittest, a small piece of code which runs those tests in a forked process.
I want to start a proposal but I'm not sure if it should come as a XCTest improvement which spawns a process or if the language should implement assert and precondition as a special kind of throw?
I would like to test both asserts and preconditions too.
···
On 13 Dec 2015, at 15:18, Marc Knaup via swift-corelibs-dev <swift-corelibs-dev@swift.org> wrote:
+1 from me. I'm really annoyed by the fact I can't reach 100% code coverage due to this :)
Being able to test them is very valuable. Take Swift's promise to be safe for example. If you can't test the behavior in cases where an array index was used which is out of bounds then you can't test even the promise to be safe.
On Sun, Dec 13, 2015 at 11:03 AM, David Hart via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
Currently, it is impossible for XCTest to unit test assert and precondition failures because they kill the process. And those are important to unit test: how would you test NSArray's objectAtIndex bound conditions? Currently, the standard library tests those with StdlibUnittest, a small piece of code which runs those tests in a forked process.
I want to start a proposal but I'm not sure if it should come as a XCTest improvement which spawns a process or if the language should implement assert and precondition as a special kind of throw?