Typed throws in swift-json: a case study

Using Swift
1

as is well known, a regular untyped Swift Error calls into the Swift runtime when it is thrown, which adds a very small but nonzero overhead to returning by throws when compared to returning by return.

this behavior is potentially very useful for debugging, as the runtime call can be associated with a debugging hook, and enabled and disabled without recompiling applications. on the other hand, it can cause code that naturally throws a lot of errors internally (perhaps in the course of its expected control flow) to become dramatically slower if something is registered to that runtime hook, such as when code is running in a test suite.

anyway, i was never really sure how much those untyped throws cost when a runtime hook is not present, so i did an experiment tonight refactoring the swift-json library to use typed throws and it turns out, the answer is a lot.

the version of the JSON parser that uses typed throws is much faster than the original version, approaching a 2x speedup on aarch64 linux:

old:

architecture subject benchmark duration
x86_64 Swift.symbols.json 1.901695668 seconds
aarch64 Swift.symbols.json 1.746343872 seconds

new:

architecture subject benchmark duration
x86_64 Swift.symbols.json 1.180054329 seconds
aarch64 Swift.symbols.json 0.932082752 seconds

(i had used a swift symbolgraph-extract symbol dump as a convenient test subject)

this is notable because the swift-json parser was already heavily optimized, so i had long thought i had already maxed out its performance ceiling. but it turns out adopting typed throws unblocks considerable optimization opportunities.

i will say, except for one problem, it was really easy to port the JSON parser to typed throws. function coloring does all the heavy lifting, so it’s as mechanical as just copy-pasting throws(PatternMatchingError) everywhere throws used to be. it’s one of those wonderful instances where you invest very little effort in a refactor and the compiler just makes all your code faster for you.

the problem that i had was a known one - typed throws just do not get along with closures at all - so it’s very tedious to have to expand all the closures so that they don’t use anonymous parameters anymore.

but anyway i’m quite suprised how profitable it is to add typed throws to internal APIs, i shipped the newly-optimized swift-json in v3.0. you won’t see many typed throws in the library’s public API, as i’m still lukewarm on typed throws on public surfaces (it just feels like it defeats the purpose of any Error) but it’s really really great in interiors, and unlocks a lot of powerful declarative control flow patterns. i see a lot of people reaching for macros when what they probably really want is typed throws.

15 Likes
2

Thanks for the writeup and congrats on the performance win!

I wasn't fully aware of this, so I did some research. To clarify, the call into the runtime isn't specific to untyped throws nor is the debugging hook. As far as I can tell, there is no meaningful difference between untyped and typed throws in this regard. (I'm not claiming you said there was, but I also wasn't sure you didn't want to say that. Apologies if I didn't get your meaning.)

What I found out:

  • In an untyped throws context, the generated code calls swift_allocError to put the error in a heap-allocated box and then calls the swift_willThrow runtime function.
  • In a typed throws context, the generated code calls the swift_willThrowTypedImpl runtime function.
  • The debugging hook for catching errors exists for both untyped and typed throws in a similar form. There are separate hooks for untyped and typed errors. The default implementation for the typed debugging hook boxes the error and forwards to the untyped hook. But swift-testing assigns separate untyped and typed handlers.

Sources:

I'm sure you know this already, but to spell it out explicitly: Given the similarities in the swift_willThrow behavior for untyped and typed throws, I suspect most of the speedup you observed can be attributed to the elimination of heap allocations for the error existential.

7 Likes
3

one thing i am curious about is if there is any way to achieve the success-or-exit behavior of try without calling into any runtime hooks at all. if so, it would suggest the performance ceiling of semi-declarative parsers has not yet been reached, even after typed throws.

4

And this somehow slows down release builds as well?

(I assume it's only the actual throws that are affected, if my code doesn't throw often it should not be affected as dramatically).

5

yes, quite dramatically, i’ve seen differences of up to 300x anecotally, with swift test -c release. this happens if code throws a lot internally, as a declarative BNF parser does when it backtracks.

6

I see. Coming from C++ I'd never consider implementing parser backtracking via throwing – exception throwing is quite heavy in C++. Perhaps not so heavy in Swift, but... old habits die hard. Must be a very good test case, though!

+1

7

This is the important distinction between exceptions in languages like C++ and errors in Swift. The former involves expensive stack rewinding, but Swift errors are just a value stored in an extra register. When an error is thrown, the register is populated and the function is returned, and the caller checks that register.

This article by Mike Ash is a great read on the internals.

1 Like
8

Sure. Throws could be much less heavy than stack unwinding in those other languages but still heavy enough for the task at hand and much heavier than, say, returning the error via inout parameter or a return value. I made a quick and dirty microbenchmark just now to test this claim (totally unscientific of course); relative timing of untyped throws vs typed throws vs no throws happened to be some 25:9:1.

1 Like
9

when you write a parser you have to balance speed, correctness, and debuggability. a semi-declarative try-based parser really shines with the second two axes – there’s just tremendous value in having a parser that looks just like the formal BNF grammar of the thing you’re trying to parse, and it also gives you a straightforward way to collect a “nice” error when an input fails to parse, a trace that outlines exactly how the parser ended up where it did.

and even in the Swift 5.x days, before typed throws, i still found the speed to be quite good, at least compared to what passed for “heavily optimized” imperative parsers back then. a single Collection.split will easily eat up any savings you get by avoiding try catch, so BNF parsers were only “slow” if you were comparing them to byte-level state machines.

2 Likes
10
Deleted [quote="ole, post:2, topic:86100"]

I suspect most of the speedup you observed can be attributed to the elimination of heap allocations for the error existential.

[/quote]
I have a basic question. I think swift symbolgraph-extract generates a valid json file, right? (I did a quick experiment and I think so). So I expect there are almost no errors generated during the benchmark run (there might be some if swift-json code uses internal errors for logic control). If so, I think the code to create existential values and allocate heap aren't executed.

EDIT: on a second thought, perhaps parsing inherently involves "trial and error" so it indeed generates many errors?

11

Taking my experiment from above as an example (Compiler Explorer link), we can add up what happens when a function throws.

This Swift code:

struct MyError: Error {}

func myFuncTyped() throws(MyError) -> Int {
    throw MyError()
}

generates this assembly (I omitted the function prologue and epilogue as they're irrelevant) (edit: on Linux):

bl      lazy protocol witness table accessor for type output.MyError and conformance output.MyError : Swift.Error in output
adrp    x1, full type metadata for output.MyError+16
add     x1, x1, :lo12:full type metadata for output.MyError+16
mov     x2, x0
bl      swift_willThrowTypedImpl
mov     w21, #1                         // =0x1

What this is doing:

  • Load the type metadata and conformance witness table for our error type. This is necessary because the debugging hook in the Swift runtime is a quasi-generic function over the error type, so we must pass it these values in addition to the actual error value.
  • Call the debugging hook in the Swift runtime swift_willThrowTypedImpl
  • Set register x21 = 1 (w21 in the assembly, but both refer to the same register). This tells the caller that an error occurred. The actual error value is returned in x0.

The implementation of swift_willThrowTypedImpl is here: https://github.com/swiftlang/swift/blob/cc6c833d6cd5ee8d11b9b072bb550001a18d9bfb/stdlib/public/runtime/ErrorObjectCommon.cpp. In the common case where no debugging hook is installed (unless the code is running in a test suite), it boils down to 2 atomic loads, one for the typed throws debugging hook and one for the fallback untyped throws debugging hook. If a debugging hook is installed, the function will execute it, otherwise (the common case) it does nothing else.

So, in terms of overhead vs. something like Result we have by my count:

  • Getting the pointers to the error type's type metadata and Error conformance witness table.
  • One function call
  • 2 atomic loads

(Apologies for the multiple edits. I clicked Send too soon.)

3 Likes
12

Sort of a tangential question, but is there no way to disable the builtin that calls that hook? It seems like the lowering code supports conditionally emitting it, but most callers enable it unconditionally. If it's primarily for debugging/instrumentation purposes, would it make sense to support a flag or something that would suppress it? I assume that could break some IDE/debugger functionality, etc, but perhaps that would be a reasonable tradeoff for certain high-performance use cases?

13

Thanks for the analysis!

I stepped through the code, instruction by instruction to see the actual overhead.

For untyped throws the overhead was around 35 instructions plus what's inside swift_allocError.

For typed throws it was around 40 instructions plus what's inside _stdlib_is0SVersionAtLeastOrVariantVersionAtLeast.

The two mentioned calls were very heavy, perhaps 100s if not 1000s of instructions, I was not able stepping through them completely line by line.

To @rayx

Traditionally with BNF parsing you look at the next symbol and that limits what you could see next (in a valid formatted syntax) so there is no need for backtracking, and the only errors that occur are due to incorrectly formatted syntax tree.

14

Is there a compelling reason to have swift_willThrow(TypedImpl) be called at all in -O builds? I guess there are some folks who still need to debug optimized builds but I'd love to know if anyone is relying on that specific handler there in practice.

If it's still needed, adding a flag to control emitting the hook seems super straightforward. (Or an attribute on specific declarations, but that might be overkill and would be pretty noisy in code, too.)

4 Likes
15

Xcode's "break on Swift throws" feature puts breakpoints on swift_willThrow and swift_willThrowTypedImpl (in Release as well) but I guess those could be some tiny stubs if that's just for that.

16

as i understand it, Embedded Mode can be used as a sledgehammer to disable it, but that pushes a lot of complexity into the build-and-link stage of the process.

i actually think an attribute on the thrown error type is the right way to go here. something like

enum PatternMatchingError: @untraceable Error {}

would completely address this use case. it would not even need to be respected in the general case. it would be enough if it were to only go into effect if the thrown error type were statically known.