… Ah OK, just forgetting part of the stack (= exceptions as in Java) is not possible because Swift does not have a tracing garbage collector, so failing is the only option so to speak: You try to bring together in Swift a) safeness (being “harmless”), b) efficiency, c) correctness, and d) having to “respect the stack” because of reference counting. Is this the “full picture”?
I would rather dream of a well written, educating, and illuminating official Swift Language Programming Book. 
1 Like
There are languages out there that aim towards this. You might want to look into Haskell for example. However, none is fully "safe". If you want to have more than a toy language, you'll want to offer foreign interface calls towards C or C++, and once you do that, it's over.
Also, while good on paper (i.e. it compiles, it works) it does affect developer friendliness and significantly rises the entry level (because of the constructs you'd need to understand in order to write code in such a language). Simply not a fit for Swift.
2 Likes
I think the official book is already quite good, and you can participate.
(...I was also thinking of contributing some of the bits I learned in this topic to the book, but maybe this is more for someone who knows more about the Swift internals and the intentions. In my last comment I speculated about reference counting — which I think is a very good thing — hindering an “isolation mechanism” akin to Java-like exceptions, but honestly I do not know enough about these issues to be in the positions to inform other people about it.)
Well, I know Haskell and today I am very much opposed to its strict functional idioms. This is a bit another topic than safeness, to my knowledge the addition of two numbers in Haskell overflows silently as in Java. And Haskell suffers a lot from unexpected changes in efficiency when changing the code, that gives you another kind of unsafeness; from what I have read the Swift people work very hard to avoid those traps. What is interesting about Haskell are its variants with dependent types.
Even if you don't know about the compiler internals, you can still contribute to TSPL. Sometimes I specifically avoid reading the compiler source code before I start writing, because I want to document the way something is intended to behave, rather than describing incidental details about how the implementation currently behaves.
6 Likes
consider this evil test case from Oliver Hallam:
adjust(date: "-25252734927766555-06-07+02:00", to: "UTC")
- ok since date is in
+02:00timezone, roll back -2 hours - but since no time is given, 0:00:00 is implied
- so you gotta roll back into the previous day, which is
-25252734927766555-06-06 - so now you gotta do real calendar arithmetic...:
- most ppl use Hinnant's fast "branch-free proleptic‑Gregorian algorithm" for this
- but consider a naive Swift version of it:
static func daysFromCivil(_ y0: Int, _ m: Int, _ d: Int) -> Int? {
let y = m <= 2 ? y0 - 1 : y0
let era = (y >= 0 ? y : y - 399) / 400
let yoe = y - era * 400
let doy = (153 * (m > 2 ? m - 3 : m + 9) + 2) / 5 + d - 1
let doe = yoe * 365 + yoe / 4 - yoe / 100 + doy
let eraDays = era * 146097
let sum = eraDays + doe - 719468
return sum
}
- so walk with me...
let y = -25252734927766555
let era = (y >= 0 ? y : y-399) / 400 // -63131837319418 (days per 400-year era)
let eraDays = era * 146097 // -9223372036855011546
// BUT Int64.min == -9223372036854775808
// so eraDays overflows by: 235,738 == SIGTRAP in swift
...
fortunately we have these in swift:
multipliedReportingOverflowaddingReportingOverflow
so the fix:
let (eraDays, o1) = era.multipliedReportingOverflow(by: 146097) // ← was era * 146097
guard !o1 else { return nil }
let (sum, o2) = eraDays.addingReportingOverflow(doe - 719468)
return o2 ? nil : sum
i've thought about how you'd ever protect against this kind of thing statically...
not sure there's really a way...
1 Like
I'd say something like this:
// -experimental-performance-annotations flag required
@_noAllocation func foo() {
class C {}
_ = C() // ❌ Compilation error: can cause metadata allocation or locks
}
// -experimental-trap_annotations flag required
@_noTraps func bar(_ x: Int) -> Int {
x + 1 // ❌ Compilation error: can trap
}
I was kind of thinking about a Java-esque conversion of anything that currently crashes to things that throw exceptions, and adding a “special” set of errors that can be thrown even by functions not declared to be throwing. In that mode accessing an Array out of bounds throws a MachineCheck.IndexOutOfBounds exception. Division by zero? MachineCheck.DivisionByZero! Precondition failure? MachineCheck.PreconditionFailure(String)!
If you really want to “never crash” you compile in this mode, and make sure you wrap the “critical never crash” sections with try/catch that does “something” appropriate.
I expect this to be far from performance neutral because I think calling a function that can throw has overhead, and in this mode basically anything can throw. In addition to overhead it may also make a lot of compiler optimizations far less useful because they can’t intermix state from around a possible exception boundary (i.e. a lot more ordering becomes program visable state when a lot more things have the potential to throw).
It would make the “crashless” mode more similar to the “full” mode in that the only real change is in crashless mode you can catch MachineCheck exceptions that “full” mode knows nothing about, and you can throw a MachineCheck even from non-throwable functions.
(Or maybe you have to declare throw(MachineCheck), but callers get a default of silently rethrowing them, and get implicitly marked as rethrowing them? That might make the two modes more similar)
Without knowing the actual cost of “way more things might throw” I can’t really say if this is a valuable way to say write Vapor servers that “never crash”, or if it would just be frittering away too much of Swift’s performance to be worth it.
In a model like this you’d inevitably have cleanup code (catch blocks, deinit functions) running after the MachineException is thrown. That code could throw other exceptions — a nightmare scenario for error handling where even C++ throws up its hands and aborts the process. Or it could deadlock.
if you disallow cleanup, you very easily end up with nasty state corruption like stuck mutexes. This is the reason Java 1.2(?) gave up on allowing threads to be killed pre-emptively.
I think code compiled this way would have to be sandboxed away from the rest of the process, on separate threads, with absolutely no way to share state. Maybe even a separate heap so leaks can be cleaned up. At this point it really starts to feel like a separate process that you communicate with via distributed actors.
5 Likes
Here was an idea of how to have unchecked exceptions implementation (also see a couple more refining posts down that thread) that won't have too much performance implications. But that's different to what OP has in mind (compile time checks).
How would you apply @_noAllocation or @_noTraps to the example that I gave? Would it simply just not compile let eraDays = era * 146097 because it might trap?
Yeha, it's the same idea as with @_noAllocation (which won't compile if you call something that might allocate – although not necessarily will allocate). So a hypothetical @noTraps will fail to compile era * 146097 because it might overflow – not necessarily will overflow on every occasion.
I like this idea. It would have caught the possibility of our code maybe crashing instead of us having to wait until it crashed to realize we should use the safe types.
And believe me the W3C test suite has thought of every freaking possible way to break you
NB: "safe" in Swift parlance includes termination, e.g. "terminate the app safely instead of continuing proceeding after UB".
I wonder if they have tests for:
- stack overflows
- infinite loops
- finite loops that take millennium to complete
In general case you can't check for those things at compile time... You might check for those conditions at runtime, e.g.:
try callFunction(foo, arguments) // might fail when not enough stack
try forLoop(arguments, timeout: 10sec) { closure } // might fail if takes too long
2 Likes
I like that about it actually. A lot of people hate crashes but I think they're fine. I'd much rather have my app crash during QA than silently pretend to "work" until our backend DBs are corrupted and someone loses $43 million. If my app's not crashing it's because it's following all the happy code paths and not hitting my asserts.
But the compiler should always make it clear when a given statement can potentially crash. Ideally they'd all throw or require an "unsafe" method call. And a mode where things that fail to warn you simply cannot compile would be great.
If my app crashes I want it to be because I intentionally walled off an unhappy path with termination, not because somone assigned the year -474929575894929585738846 from parsed YAML into a date.
Yes they thought of literally everything (and more). It's the most diabolical test suite I've ever seen. These guys are geniuses.
1 Like
It's possible to have a system where "traps" is another function effect, like "throws". The problem then becomes that almost all functions will have this "traps" effect in practice--anything that performs integer arithmetic, array indexing, etc.
Also, compiler-generated traps are not the only way in which a safe program might fail to terminate. For example, you might exhaust the heap or stack. Of course your program might enter an infinite loop as well, or a finite but very long loop that may as well be infinite, if you have an exponential-time algorithm in there. If your program is a long-lived server process, it would then need to be killed and restarted, which is effectively the same as if you hit a trap.
What you're looking for is essentially some way to guarantee successful termination with a reasonable bound on time and space usage, which is a far stronger statement than to say that a program is memory-safe.
5 Likes
True, although it's not unimaginable to have:
// not current Swift:
try x + 1 // could throw on overflow
try items[100] // could throw on out of bounds access
func foo() {} // NOTE: non throwing
try foo() // any function call could throw because of stack overflow
class C {} // NOTE: no throwing init
let x = try C() // any class instance allocation could throw because of out of memory
Oh yes, I'm quite familiar Sig Segv, aka Mr. Exception, Bad Access.
Funny story about that:
While researching bizarre "grammars" hidden in Collatz orbits for extremely large integers, I kept hitting SIGSEGV after recursing 15,000-25,000 times.
Then I hit upon a weird Swift Concurrency trick (full git repo here) to get recursion only limited by available memory, without hitting SIGSEGV (see below).
let evaluator = Evaluator.shared
extension Evaluator {
static func evaluate(_ arg: BigUInt) {
let group = DispatchGroup()
// STEP 1: Enter a DispatchGroup.
group.enter()
// STEP 2: Run a task in the Dispatch Group.
Task {
// STEP 4: await an async method.
await evaluator.evaluate(arg)
// ... TWO HOURS LATER ...
// STEP 9: leave the group and terminate the program.
group.leave()
}
// STEP 3: wait on the DispatchGroup.
group.wait()
}
}
class Evaluator {
static let shared: Evaluator = Evaluator()
func evaluate(_ i: BigUInt) async {
// STEP 5: await the asynchronous recursive function
await isHailstoneTo421(i) { result in
// output the result, etc.
}
}
}
func isHailstoneTo421(
_ x: BigUInt,
_ y: [BigUInt] = [],
_ thetas: [BigInt] = [],
_ sums: [BigInt] = [],
_ evens: [UInt8] = [],
_ completion: @escaping ((
Bool,
[BigUInt],
[BigInt],
[BigInt],
[UInt8])) -> Void)
async {
// STEP 6: do one cycle of the recursive work
// calculate the next number in the Collatz orbit
// STEP 8: call completion if we've reached a cycle
guard !y.contains(x) else {
completion((false, y, thetas, sums, evens))
return
}
// STEP 7: await the asynchronous recursive function
await isHailstoneTo421(...)
}
}
Evaluator.evaluate(arg)
For toy examples like this one where all the numeric constants are statically known, yes, the compiler can just do the math at compile time (and usually does!) and can figure out that its result will be invalid and emit an error.
But if the numbers are only known at runtime? Then… no, not in our universe. You're effectively asking the compiler to statically determine if a program will terminate under certain conditions. Which, for non-trivial programs, is of course undecidable.
3 Likes
Well, type systems can prove properties of programs that would otherwise be undecidable, because they reject programs that are valid but "too complex" to reason about. That's sort of how you work around the halting problem.
Eg, you can do this in a dynamic language, and it might be correct:
func f() -> Bool { ... some complicated computation that happens to always return true ... }
let x = f() ? 3 : "hi"
let y = f() ? 2 : "bye"
return x * y // no type error
However a static type checker will probably reject this program, because it cannot prove that f() always returns true.
So you could design a type system that can guarantee that your program is free from integer overflow for example, and basically each function's type annotation becomes a proof of this fact. It just wouldn't be able to prove that this property holds in every program, because you might not be able to express the proof within the type system if the logic in your function is too complicated. And the burden of writing these annotations everywhere is probably too much for day-to-day development.
3 Likes