More informative compiler assertions in C++

By default, Swift assertions are much more informative than C++ assertions:

// (1) C++: we get a non-interpolatable assertion message, by default.
assert(type->isEqual(otherType) && "Types are not equal")
// (2) Swift: we can interpolate anything easily into the assertion message.
precondition(type == otherType, "Types \(type) and \(otherType) are not equal")

Can we add something like (2) as a utility/mini-library within the Swift compiler implementation?

SILVerifier has good but limited infrastructure (requireSameType, requireABICompatibleFunctionTypes) for more informative assertions.

I think refactoring to a mini-library would be great. I've wanted to use informative assertions basically any time I hack on the compiler. Designing performant informative assertions and measuring their performance would be interesting.

@Michael_Gottesman @beccadax @typesanitizer

2 Likes

Perhaps one perspective is: informative assertions are slow and unnecessary, just use a debugger.

That perspective makes sense. I had a bit of trouble using lldb in the past, so I have some questions there too. Informative assertions can have messages that provide more insight to users as well.

I am not against this in principle. I also think that it would be good to get @atrick's thoughts.

I am against going through and changing the entire compiler though to use these. If this is a utility that people want, I don't have a problem with it being additive.

1 Like

I don't believe the ping to Andy worked, so I'll try again: @Andrew_Trick

I'm strongly in favor of anything that makes a compiler engineer's life easier, including informative asserts in the release build. One of my biggest complaints with LLVM is lack of a proper assert package. Ideally, this would be an LLVM addition. If that's a hard sell, I'm not against adding something to the Swift repo. Either way, this is something you want to be sure to get right from the outset, consider many of the good C++ asserts packages that are already out there, and get plenty of review feedback. Having resigned myself to what LLVM provides, I haven't looked into it in many years, so I don't have any concrete suggestions.

2 Likes

This has been one of the things on my wishlist but I haven't had time to work on. :sob:

There are several papercuts in this area IMO (in no particular order):

  1. The PrettyStackTrace machinery is a bit cumbersome to use (you need to create a new subclass for new type combinations).
  2. Writing assertions/crash messages which are informative is cumbersome.
  3. Many types don't have dump()/dump(llvm::raw_ostream &) methods, this makes debugging unnecessarily annoying.

The problem with "just use a debugger" is that building the stdlib with a debug compiler is painfully slow, so if you're hitting an issue in compiling the stdlib, then you probably want to use a release build which means you miss out on many variables and even methods.

You mentioned performance: are you suggesting that the new assertions be both more informative and on-by-default? Or would they be only enabled in asserts builds, but you just mean that they shouldn't be super slow?

2 Likes

I don't think I can make this decision, but I would prefer that informative assertions be enabled in (1) the build mode of Swift.org - Download Swift toolchains and (2) the build mode of compiler developers.

(1) would be nice so users see informative crash messages. If (1) is unacceptable due to performance costs, (2) would still be great.

The development snapshot toolchains on swift.org have assertions enabled, but the other ones not.

Perf impact is... I suspect, you're not going to be able to measure much at the beginning, but once usage grows, it will make things slower. The problem with that is... if people are wary of using the API because they are worried about making the compiler slower, then they don't use it, and you don't get an informative message.

Personally, I think it's worth just tackling (2) first and we can have a separate conversation about (1) holistically (maybe we enable some verifier specifically which doesn't penalize compile times too much).


Wrt the API, I think it would be nice to have something like absl::StrCat that works with compiler-specific types, say (strawman name) swift::debug_message. This can be used separately for print debugging, and one could build the informative assert on top of that.

llvm::errs() << swift::debug_message("Expected ", type1, " to be equal to ", type2);

// strawman name
#define swift_assert(check, ...) \
#ifndef NDEBUG \
if (!check) { \
  llvm::errs() << swift::debug_message("assertion " #check " failed: ", __VA_ARGS__); \
  fatal_error();
} \
#endif

Also, happy to talk over DM if you want to bounce ideas.

1 Like

Just curious, but do you (or anybody else) have figures on how much of a performance hit is caused by enabling assertions in a release compiler?

Intuitively, I would expect that it has barely any impact in the context of everything the compiler does (reading files, parsing, building the ast, sema, silgen, sil optimisation, irgen, llvm optimisations, codegen, etc).

1 Like

Does adding [[clang::optnone]] to the crashing function (while still otherwise doing a release build) help? It's worked out pretty well for me on projects I've worked on

This might work for some cases better than others. In my experience, when debugging the compiler, often there is a cross-cutting issue across several components and you're jumping between multiple breakpoints and inspecting values in many different places, so marking lots of things with attributes is not really convenient. Worse if the method's implementation is in a header. At that point, the feedback loop is close to that of print debugging because you keep mixing debugging with rebuilds.

1 Like

Times reported are wall clock time averages ± stdev over 5 runs after "warm-up" (few discarded builds) and in seconds.

Key: debug = debug build of Swift code (i.e. with swift build), release = release build of Swift code (i.e. with swift build -c release), assert = compiler in ReleaseAssert (build-script --release) (EDIT: there is no --assertions here), noassert = compiler in Release (build-script --release --no-assertions).

Package debug/noassert debug/assert release/noassert release/assert
Alamofire 10.79 ± 0.09 12.60 ± 0.14 17.16 ± 0.10 29.27 ± 0.14
SwiftNIO 15.86 ± 0.17 19.99 ± 0.23 33.40 ± 0.17 57.92 ± 0.25
SwiftSyntax 24.00 ± 0.44 30.82 ± 0.22 139.7 ± 6.6 242.8 ± 4.0

(The absolute average times are off by ~0.2s since this includes times to delete the build directory.)

So roughly, assertions slow down the compiler by 15%~25% when compiling Swift code without optimizations, and 70%~75% when compiling Swift code with optimizations.

Both of these numbers are smaller than what I expected, I hope I didn't make a blunder is measuring things. It would be nice if someone could reproduce the numbers.

Configuration: 2017 10-core iMac Pro. Swift compiler built with Clang in Xcode 12.

  • Alamofire @ 9e0328127dfb801cefe8ac53a13c0c90a7770448
  • SwiftNIO @ 076fda15213a9cc1da26b1e3467f1daba2407391
  • Swift @ 887464b7b67d5202bfa7adc4e3f045ff1027a5a7
  • swift-syntax @ 35eba3fafc92812b997957a437b2e597613f0e01
3 Likes

build-script --assertions changes the compiler input by virtue of changing the stdlib (the build script options make NO sense, and everyone gets this wrong all the time).

Beyond that, I don't know if you're measuring the cost of assertions or the cost of verifying SIL, which happens to be enabled by assertions. I'd be shocked if assertions themselves have such a significant affect on compile time.

1 Like

Ah, I made a mistake there. I didn't explicitly pass --assertions, I was assuming that was equivalent to the default (but that's not true). Fixed the comment.

That's a good point; I don't have a breakdown at this stage. I can make more detailed measurements when I have some time to spare.

On using assertions + -sil-verify-none (to isolate the cost of the verifier), the overhead over a non-asserts build becomes ~18% instead of ~23% for debug compilation, and ~60% instead of ~75% for release.

I'm going to spend some time trying out other combinations (e.g. turn off assertions only in LLVM) and create a separate post with all the numbers together, to avoid derailing this thread.