-disable-reflection-metadata flag causes compiler crash (5.3)

I recently realised I'm getting huge, pointless code bloat in my libraries from reflection metadata. I never use reflection on my platform (and have removed all stdlib support for it).

I stumbled across the flag "-Xfrontend" "-disable-reflection-metadata" and added that to my build, however, now I'm getting a crash in IRGen...

Assertion failed: (castIsValid(op, S, Ty) && "Invalid cast!"), function Create, file /Users/carlpeto/compilers/avr-swift/llvm-project/llvm/lib/IR/Instructions.cpp, line 2741.
Stack dump:
0.	Program arguments: /Users/carlpeto/compilers/avr-swift/build/Ninja-ReleaseAssert+swift-DebugAssert/swift-macosx-x86_64/bin/swift -frontend -emit-bc CoreOperators.swift CoreAliases.swift RawRepresentable.swift LiteralProtocols.swift TopLevelFunctions.swift CoreProtocols.swift CoreFloatingPoint.swift CoreBinaryFloatingPoint.swift Float.swift Float16.swift CoreFloatingPointFunctions.swift Optional.swift Bridging.swift CoreNumericProtocols.swift BinaryInteger.swift CoreIntegers.swift ErrorType.swift Bool.swift Integers.swift Ranges.swift Sequence.swift Stride.swift Slice.swift Collection.swift BidirectionalCollection.swift RandomAccessCollection.swift ClosedRange.swift MutableCollection.swift Hash.swift Pointer.swift UnsafeBufferPointer.swift UnsafeRawBufferPointer.swift UnsafeRawPointer.swift Indices.swift Existential.swift Algorithm.swift FixedWidth.swift IntegerMath.swift CTypes.swift UnsafePointer.swift ObjectIdentifier.swift CollectionAlgorithms.swift WriteBackMutableSlice.swift Random.swift RangeReplaceableCollection.swift MemoryLayout.swift Tuple.swift SequenceAlgorithms.swift LifetimeManager.swift Repeat.swift EmptyCollection.swift CollectionOfOne.swift StringLiterals.swift StaticString.swift Unicode.swift UnicodeScalar.swift UnicodeEncoding.swift UTF8.swift UTF16.swift ValidUTF8Buffer.swift UnicodeParser.swift UIntBuffer.swift UTFEncoding.swift UTF32.swift ArrayType.swift ArrayBufferProtocol.swift ArrayLiterals.swift ArrayShared.swift ContiguousArray.swift SliceBuffer.swift ArraySlice.swift Array.swift ArrayBody.swift ArrayCast.swift AnyHashable.swift ManagedBuffer.swift AVRArrayBuffer.swift Reverse.swift Map.swift Zip.swift LazySequence.swift LazyCollection.swift Filter.swift FlatMap.swift Flatten.swift DropWhile.swift Volatile.swift Integer-16.swift IntegerMath-16.swift Progmem.swift -supplementary-output-file-map /var/folders/1q/fy3cg3j92fgb11hj50dtzf900000gn/T/supplementaryOutputs-4fb296 -disable-objc-attr-requires-foundation-module -target avr-atmel-linux-gnueabihf -disable-objc-interop -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -color-diagnostics -I uSwiftShims -I /Applications/Swift For Arduino.app/Contents/XPCServices/S4ABuildEngine.xpc/Contents/Resources/gpl-tools-avr/lib/avr-libgcc/include -I /Applications/Swift For Arduino.app/Contents/XPCServices/S4ABuildEngine.xpc/Contents/Resources/gpl-tools-avr/lib/avr-libc/include -I AVR -enable-library-evolution -nostdimport -parse-stdlib -Osize -D AVR_LIBC_DEFINED_SWIFT -disable-reflection-metadata -Xcc -DAVR_LIBC_DEFINED -DLIBC_DEFINED -parse-as-library -module-name Swift -num-threads 4 -o bin/AVR/CoreOperators.bc -o bin/AVR/CoreAliases.bc -o bin/AVR/RawRepresentable.bc -o bin/AVR/LiteralProtocols.bc -o bin/AVR/TopLevelFunctions.bc -o bin/AVR/CoreProtocols.bc -o bin/AVR/CoreFloatingPoint.bc -o bin/AVR/CoreBinaryFloatingPoint.bc -o bin/AVR/Float.bc -o bin/AVR/Float16.bc -o bin/AVR/CoreFloatingPointFunctions.bc -o bin/AVR/Optional.bc -o bin/AVR/Bridging.bc -o bin/AVR/CoreNumericProtocols.bc -o bin/AVR/BinaryInteger.bc -o bin/AVR/CoreIntegers.bc -o bin/AVR/ErrorType.bc -o bin/AVR/Bool.bc -o bin/AVR/Integers.bc -o bin/AVR/Ranges.bc -o bin/AVR/Sequence.bc -o bin/AVR/Stride.bc -o bin/AVR/Slice.bc -o bin/AVR/Collection.bc -o bin/AVR/BidirectionalCollection.bc -o bin/AVR/RandomAccessCollection.bc -o bin/AVR/ClosedRange.bc -o bin/AVR/MutableCollection.bc -o bin/AVR/Hash.bc -o bin/AVR/Pointer.bc -o bin/AVR/UnsafeBufferPointer.bc -o bin/AVR/UnsafeRawBufferPointer.bc -o bin/AVR/UnsafeRawPointer.bc -o bin/AVR/Indices.bc -o bin/AVR/Existential.bc -o bin/AVR/Algorithm.bc -o bin/AVR/FixedWidth.bc -o bin/AVR/IntegerMath.bc -o bin/AVR/CTypes.bc -o bin/AVR/UnsafePointer.bc -o bin/AVR/ObjectIdentifier.bc -o bin/AVR/CollectionAlgorithms.bc -o bin/AVR/WriteBackMutableSlice.bc -o bin/AVR/Random.bc -o bin/AVR/RangeReplaceableCollection.bc -o bin/AVR/MemoryLayout.bc -o bin/AVR/Tuple.bc -o bin/AVR/SequenceAlgorithms.bc -o bin/AVR/LifetimeManager.bc -o bin/AVR/Repeat.bc -o bin/AVR/EmptyCollection.bc -o bin/AVR/CollectionOfOne.bc -o bin/AVR/StringLiterals.bc -o bin/AVR/StaticString.bc -o bin/AVR/Unicode.bc -o bin/AVR/UnicodeScalar.bc -o bin/AVR/UnicodeEncoding.bc -o bin/AVR/UTF8.bc -o bin/AVR/UTF16.bc -o bin/AVR/ValidUTF8Buffer.bc -o bin/AVR/UnicodeParser.bc -o bin/AVR/UIntBuffer.bc -o bin/AVR/UTFEncoding.bc -o bin/AVR/UTF32.bc -o bin/AVR/ArrayType.bc -o bin/AVR/ArrayBufferProtocol.bc -o bin/AVR/ArrayLiterals.bc -o bin/AVR/ArrayShared.bc -o bin/AVR/ContiguousArray.bc -o bin/AVR/SliceBuffer.bc -o bin/AVR/ArraySlice.bc -o bin/AVR/Array.bc -o bin/AVR/ArrayBody.bc -o bin/AVR/ArrayCast.bc -o bin/AVR/AnyHashable.bc -o bin/AVR/ManagedBuffer.bc -o bin/AVR/AVRArrayBuffer.bc -o bin/AVR/Reverse.bc -o bin/AVR/Map.bc -o bin/AVR/Zip.bc -o bin/AVR/LazySequence.bc -o bin/AVR/LazyCollection.bc -o bin/AVR/Filter.bc -o bin/AVR/FlatMap.bc -o bin/AVR/Flatten.bc -o bin/AVR/DropWhile.bc -o bin/AVR/Volatile.bc -o bin/AVR/Integer-16.bc -o bin/AVR/IntegerMath-16.bc -o bin/AVR/Progmem.bc
1.	Swift version 5.3-dev (LLVM 9bbbe0cda8, Swift 3675d8bbac)
2.	While evaluating request IRGenWholeModuleRequest(IR Generation for module Swift)
3.	While emitting IR SIL function "@$sSxsE5_step5after4from2bySiSg5index_x5valuetAeF_xAGt_x6StrideQztFZ".
 for '_step(after:from:by:)' (at Stride.swift:28:10)
0  swift                    0x0000000106841c95 llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 37
1  swift                    0x0000000106840c48 llvm::sys::RunSignalHandlers() + 248
2  swift                    0x0000000106842296 SignalHandler(int) + 262
3  libsystem_platform.dylib 0x00007fff6f0565fd _sigtramp + 29
4  swift                    0x00000001092f8f86 cmark_strbuf__initbuf + 1266817
5  libsystem_c.dylib        0x00007fff6ef2c808 abort + 120
6  libsystem_c.dylib        0x00007fff6ef2bac6 err + 0
7  swift                    0x0000000107bebb23 llvm::CastInst::Create(llvm::Instruction::CastOps, llvm::Value*, llvm::Type*, llvm::Twine const&, llvm::Instruction*) (.cold.14) + 35
8  swift                    0x0000000106682f69 llvm::CastInst::Create(llvm::Instruction::CastOps, llvm::Value*, llvm::Type*, llvm::Twine const&, llvm::Instruction*) + 1145
9  swift                    0x0000000100a29844 llvm::IRBuilder<llvm::ConstantFolder, llvm::IRBuilderDefaultInserter>::CreateSExtOrBitCast(llvm::Value*, llvm::Type*, llvm::Twine const&) + 196
10 swift                    0x0000000100a28907 emitMetadataAccessByMangledName(swift::irgen::IRGenFunction&, swift::CanType, swift::irgen::DynamicMetadataRequest)::$_10::operator()() const + 1655
11 swift                    0x0000000100a2789e emitMetadataAccessByMangledName(swift::irgen::IRGenFunction&, swift::CanType, swift::irgen::DynamicMetadataRequest) + 1246
12 swift                    0x0000000100a189e6 emitCallToTypeMetadataAccessFunction(swift::irgen::IRGenFunction&, swift::CanType, swift::irgen::DynamicMetadataRequest) + 358
13 swift                    0x0000000100a186d1 swift::irgen::IRGenFunction::emitTypeMetadataRef(swift::CanType, swift::irgen::DynamicMetadataRequest) + 481
14 swift                    0x0000000100a249ba (anonymous namespace)::EmitTypeMetadataRef::visitTupleType(swift::CanTypeWrapper<swift::TupleType>, swift::irgen::DynamicMetadataRequest)::'lambda'(swift::CanType, swift::irgen::DynamicMetadataRequest)::operator()(swift::CanType, swift::irgen::DynamicMetadataRequest) const + 106
15 swift                    0x0000000100a2493c swift::irgen::MetadataResponse llvm::function_ref<swift::irgen::MetadataResponse (swift::CanType, swift::irgen::DynamicMetadataRequest)>::callback_fn<(anonymous namespace)::EmitTypeMetadataRef::visitTupleType(swift::CanTypeWrapper<swift::TupleType>, swift::irgen::DynamicMetadataRequest)::'lambda'(swift::CanType, swift::irgen::DynamicMetadataRequest)>(long, swift::CanType, swift::irgen::DynamicMetadataRequest) + 140
16 swift                    0x0000000100a24438 llvm::function_ref<swift::irgen::MetadataResponse (swift::CanType, swift::irgen::DynamicMetadataRequest)>::operator()(swift::CanType, swift::irgen::DynamicMetadataRequest) const + 152
17 swift                    0x0000000100a244b2 emitTupleTypeMetadataRef(swift::irgen::IRGenFunction&, swift::CanTypeWrapper<swift::TupleType>, swift::irgen::DynamicMetadataRequest, bool, llvm::function_ref<swift::irgen::MetadataResponse (swift::CanType, swift::irgen::DynamicMetadataRequest)>)::$_9::operator()(swift::CanType) const + 98
18 swift                    0x0000000100a237f1 emitTupleTypeMetadataRef(swift::irgen::IRGenFunction&, swift::CanTypeWrapper<swift::TupleType>, swift::irgen::DynamicMetadataRequest, bool, llvm::function_ref<swift::irgen::MetadataResponse (swift::CanType, swift::irgen::DynamicMetadataRequest)>) + 369
19 swift                    0x0000000100a21d7e (anonymous namespace)::EmitTypeMetadataRef::visitTupleType(swift::CanTypeWrapper<swift::TupleType>, swift::irgen::DynamicMetadataRequest) + 318
20 swift                    0x0000000100a205e9 swift::CanTypeVisitor<(anonymous namespace)::EmitTypeMetadataRef, swift::irgen::MetadataResponse, swift::irgen::DynamicMetadataRequest>::visit(swift::CanType, swift::irgen::DynamicMetadataRequest) + 1737
21 swift                    0x0000000100a18867 emitDirectTypeMetadataRef(swift::irgen::IRGenFunction&, swift::CanType, swift::irgen::DynamicMetadataRequest) + 135
22 swift                    0x0000000100a18672 swift::irgen::IRGenFunction::emitTypeMetadataRef(swift::CanType, swift::irgen::DynamicMetadataRequest) + 386
23 swift                    0x0000000100a18f7c swift::irgen::IRGenFunction::emitTypeMetadataRefForLayout(swift::SILType, swift::irgen::DynamicMetadataRequest) + 492
24 swift                    0x00000001007e8b63 swift::irgen::IRGenFunction::emitValueWitnessTableRef(swift::SILType, swift::irgen::DynamicMetadataRequest, llvm::Value**) + 451
25 swift                    0x00000001007e8995 swift::irgen::IRGenFunction::emitValueWitnessTableRef(swift::SILType, llvm::Value**) + 101
26 swift                    0x000000010082196a swift::irgen::IRGenFunction::emitValueWitnessValue(swift::SILType, swift::irgen::ValueWitness) + 186
27 swift                    0x000000010082241a swift::irgen::emitLoadOfSize(swift::irgen::IRGenFunction&, swift::SILType) + 42
28 swift                    0x000000010082239b swift::irgen::IRGenFunction::emitDynamicAlloca(swift::SILType, llvm::Twine const&) + 59
29 swift                    0x000000010089cbfc swift::irgen::WitnessSizedTypeInfo<(anonymous namespace)::NonFixedTupleTypeInfo>::allocateStack(swift::irgen::IRGenFunction&, swift::SILType, llvm::Twine const&) const + 92
30 swift                    0x0000000100989530 (anonymous namespace)::IRGenSILFunction::visitAllocStackInst(swift::AllocStackInst*) + 256
31 swift                    0x00000001009875d7 swift::SILInstructionVisitor<(anonymous namespace)::IRGenSILFunction, void>::visit(swift::SILInstruction*) + 135
32 swift                    0x000000010097c0e7 (anonymous namespace)::IRGenSILFunction::visitSILBasicBlock(swift::SILBasicBlock*) + 1415
33 swift                    0x000000010097484f (anonymous namespace)::IRGenSILFunction::emitSILFunction() + 1695
34 swift                    0x0000000100974132 swift::irgen::IRGenModule::emitSILFunction(swift::SILFunction*) + 210
35 swift                    0x00000001007059bf swift::irgen::IRGenerator::emitGlobalTopLevel(llvm::StringSet<llvm::MallocAllocator>*) + 1503
36 swift                    0x00000001008d54ed performParallelIRGeneration(swift::IRGenOptions const&, swift::ModuleDecl*, std::__1::unique_ptr<swift::SILModule, std::__1::default_delete<swift::SILModule> >, llvm::StringRef, int, llvm::ArrayRef<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, llvm::StringSet<llvm::MallocAllocator>*) + 1213
37 swift                    0x00000001008d4e9e swift::IRGenWholeModuleRequest::evaluate(swift::Evaluator&, swift::IRGenDescriptor) const + 334
38 swift                    0x000000010097378c swift::GeneratedModule swift::SimpleRequest<swift::IRGenWholeModuleRequest, swift::GeneratedModule (swift::IRGenDescriptor), (swift::RequestFlags)1>::callDerived<0ul>(swift::Evaluator&, std::__1::integer_sequence<unsigned long, 0ul>) const + 156
39 swift                    0x000000010097362e swift::SimpleRequest<swift::IRGenWholeModuleRequest, swift::GeneratedModule (swift::IRGenDescriptor), (swift::RequestFlags)1>::evaluateRequest(swift::IRGenWholeModuleRequest const&, swift::Evaluator&) + 46
40 swift                    0x00000001008f87dd llvm::Expected<swift::IRGenWholeModuleRequest::OutputType> swift::Evaluator::getResultUncached<swift::IRGenWholeModuleRequest>(swift::IRGenWholeModuleRequest const&) + 413
41 swift                    0x00000001008d4cee llvm::Expected<swift::IRGenWholeModuleRequest::OutputType> swift::Evaluator::operator()<swift::IRGenWholeModuleRequest, (void*)0>(swift::IRGenWholeModuleRequest const&) + 78
42 swift                    0x00000001008d4a3a swift::performIRGeneration(swift::IRGenOptions const&, swift::ModuleDecl*, std::__1::unique_ptr<swift::SILModule, std::__1::default_delete<swift::SILModule> >, llvm::StringRef, swift::PrimarySpecificPaths const&, llvm::ArrayRef<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, llvm::GlobalVariable**, llvm::StringSet<llvm::MallocAllocator>*) + 410
43 swift                    0x000000010018a8ec generateIR(swift::IRGenOptions const&, std::__1::unique_ptr<swift::SILModule, std::__1::default_delete<swift::SILModule> >, swift::PrimarySpecificPaths const&, llvm::StringRef, llvm::PointerUnion<swift::ModuleDecl*, swift::SourceFile*>, llvm::GlobalVariable*&, llvm::ArrayRef<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, llvm::StringSet<llvm::MallocAllocator>&) + 588
44 swift                    0x0000000100189c9d performCompileStepsPostSILGen(swift::CompilerInstance&, swift::CompilerInvocation const&, std::__1::unique_ptr<swift::SILModule, std::__1::default_delete<swift::SILModule> >, llvm::PointerUnion<swift::ModuleDecl*, swift::SourceFile*>, swift::PrimarySpecificPaths const&, int&, swift::FrontendObserver*) + 2125
45 swift                    0x000000010016dbe4 performCompileStepsPostSema(swift::CompilerInvocation const&, swift::CompilerInstance&, int&, swift::FrontendObserver*) + 548
46 swift                    0x0000000100162df5 performCompile(swift::CompilerInstance&, swift::CompilerInvocation const&, llvm::ArrayRef<char const*>, int&, swift::FrontendObserver*) + 1525
47 swift                    0x0000000100161952 swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*) + 2226
48 swift                    0x0000000100004427 run_driver(llvm::StringRef, llvm::ArrayRef<char const*>) + 263
49 swift                    0x00000001000036e4 main + 2084
50 libdyld.dylib            0x00007fff6ee5dcc9 start + 1
51 libdyld.dylib            0x0000000000000130 start + 18446603338655605864
<unknown>:0: error: unable to execute command: Abort trap: 6
<unknown>:0: error: compile command failed due to signal 6 (use -v to see invocation)
Process 49672 exited with status = 254 (0x000000fe)

SIL function being emitted...

// static Strideable._step(after:from:by:)
sil @$sSxsE5_step5after4from2bySiSg5index_x5valuetAeF_xAGt_x6StrideQztFZ : $@convention(method) <Self where Self : Strideable> (Optional<Int>, @in_guaranteed Self, @in_guaranteed Self, @in_guaranteed Self.Stride, @thick Self.Type) -> (Optional<Int>, @out Self) {
// %0 "$return_value"                             // user: %14
// %1 "current"                                   // user: %8
// %2 "current"                                   // user: %10
// %3 "start"
// %4 "distance"                                  // users: %14, %11
// %5 "self"
bb0(%0 : $*Self, %1 : $Optional<Int>, %2 : $*Self, %3 : $*Self, %4 : $*Self.Stride, %5 : $@thick Self.Type):
  %6 = alloc_stack $(index: Optional<Int>, value: Self), let, name "current", argno 1 // users: %16, %15, %9, %7
  %7 = tuple_element_addr %6 : $*(index: Optional<Int>, value: Self), 0 // user: %8
  store %1 to %7 : $*Optional<Int>                // id: %8
  %9 = tuple_element_addr %6 : $*(index: Optional<Int>, value: Self), 1 // users: %14, %10
  copy_addr %2 to [initialization] %9 : $*Self    // id: %10
  debug_value_addr %4 : $*Self.Stride, let, name "distance", argno 3 // id: %11
  %12 = enum $Optional<Int>, #Optional.none!enumelt // user: %17
  %13 = witness_method $Self, #Strideable.advanced : <Self where Self : Strideable> (Self) -> (Self.Stride) -> Self : $@convention(witness_method: Strideable) <τ_0_0 where τ_0_0 : Strideable> (@in_guaranteed τ_0_0.Stride, @in_guaranteed τ_0_0) -> @out τ_0_0 // user: %14
  %14 = apply %13<Self>(%0, %4, %9) : $@convention(witness_method: Strideable) <τ_0_0 where τ_0_0 : Strideable> (@in_guaranteed τ_0_0.Stride, @in_guaranteed τ_0_0) -> @out τ_0_0
  destroy_addr %6 : $*(index: Optional<Int>, value: Self) // id: %15
  dealloc_stack %6 : $*(index: Optional<Int>, value: Self) // id: %16
  return %12 : $Optional<Int>                     // id: %17
} // end sil function '$sSxsE5_step5after4from2bySiSg5index_x5valuetAeF_xAGt_x6StrideQztFZ'

This SIL instruction...
%6 = alloc_stack $(index: Optional<Int>, value: Self), let, name "current", argno 1 // users: %16, %15, %9, %7

Can anyone advise what I'm doing wrong? I sort of expected this just to stop emitting reflection metadata only?

addendum: my best guess is this is another 16 bit addressing problem, one that maybe I haven't hit up to now because somehow the existence of reflection metadata had been "masking" this code path?

I'm guessing frame 10 is where the issue comes from (emitMetadataAccessByMangledName).

Specifically this code:

   2516	      // Break up the loaded value into size and relative address to the
   2517	      // string.
   2518	      auto size = subIGF.Builder.CreateAShr(load, 32);
   2519	      size = subIGF.Builder.CreateTruncOrBitCast(size, IGM.SizeTy);
   2520	      size = subIGF.Builder.CreateNeg(size);
   2521
   2522	      auto stringAddrOffset = subIGF.Builder.CreateTrunc(load,
   2523	                                                         IGM.Int32Ty);
-> 2524	      stringAddrOffset = subIGF.Builder.CreateSExtOrBitCast(stringAddrOffset,
   2525	                                                            IGM.SizeTy);
   2526	      auto stringAddrBase = subIGF.Builder.CreatePtrToInt(cache, IGM.SizeTy);
   2527	      if (IGM.getModule()->getDataLayout().isBigEndian()) {
   2528	        stringAddrBase = subIGF.Builder.CreateAdd(stringAddrBase,
   2529	                                        llvm::ConstantInt::get(IGM.SizeTy, 4));
   2530	      }
   2531	      auto stringAddr = subIGF.Builder.CreateAdd(stringAddrBase,
   2532	                                                 stringAddrOffset);
   2533	      stringAddr = subIGF.Builder.CreateIntToPtr(stringAddr, IGM.Int8PtrTy);

In my case, I'm building for AVR, with 16 bit address pointers, so subIGF.Builder.CreateSExtOrBitCast(stringAddrOffset, IGM.SizeTy) is impossible because you're trying to sign extend or bit cast from an i32 to an i16, so llvm aborts.

Does it make sense to just change this to CreateSExtOrTrunc?

That didn't work... now I'm getting the error Assertion failed: ((i >= FTy->getNumParams() || FTy->getParamType(i) == Args[i]->getType()) && "Calling a function with a bad signature!"), function init, file /Users/carlpeto/compilers/avr-swift/llvm-project/llvm/lib/IR/Instructions.cpp, line 400.

...when it hits the line auto call = IGF.Builder.CreateCall(instantiationFn, cache);

Looking at instantiationFn and cache gives...

(lldb) e cache->dump()
{ i32, i32 }* bitcast ({ i16, i32 }* @"$sSiSgMD" to { i32, i32 }*)

and

(lldb) e instantiationFn->dump()
; Function Attrs: noinline nounwind readnone
define linkonce_odr hidden %swift.type* @__swift_instantiateConcreteTypeFromMangledNameAbstract({ i16, i32 }* %0) #1 {
  %2 = bitcast { i16, i32 }* %0 to i64*
  %3 = load atomic i64, i64* %2 monotonic, align 8
  %4 = icmp slt i64 %3, 0
  %5 = call i1 @llvm.expect.i1(i1 %4, i1 false)
  br i1 %5, label %10, label %6

6:                                                ; preds = %10, %1
  %7 = phi i64 [ %3, %1 ], [ %21, %10 ]
  %8 = trunc i64 %7 to i16
  %9 = inttoptr i16 %8 to %swift.type*
  ret %swift.type* %9

10:                                               ; preds = %1
  %11 = ashr i64 %3, 32
  %12 = trunc i64 %11 to i16
  %13 = sub i16 0, %12
  %14 = trunc i64 %3 to i32
  %15 = trunc i32 %14 to i16
  %16 = ptrtoint { i16, i32 }* %0 to i16
  %17 = add i16 %16, %15
  %18 = inttoptr i16 %17 to i8*
  %19 = call swiftcc %swift.type* @swift_getTypeByMangledNameInContextInMetadataState(i16 255, i8* %18, i16 %13, %swift.type_descriptor* null, i8** null) #7
  %20 = ptrtoint %swift.type* %19 to i16
  %21 = zext i16 %20 to i64
  store atomic i64 %21, i64* %2 monotonic, align 8
  br label %6
}

(lldb)

...so I guess cache contains code being built that produces a pointer to {i32, i32} but the function we are building in instantiationFn accepts a pointer to {i16, i32}, which doesn't work right?

I'm getting a little out of my depth here. Can anyone offer any advice?

Carl

Stepping through, debugging the compiler with lldb, I'm confused about the behaviour of llvm.

emitMetadataAccessByMangledName is called in multiple places as compilation proceeds. Breakpointing in it and stepping, this is the bit of llvm that's not behaving as I would expect:

   2452	  auto instantiationFn = cast<llvm::Function>(
   2453	      IGM.getModule()
   2454	          ->getOrInsertFunction(instantiationFnName, IGF.IGM.TypeMetadataPtrTy,
   2455	                                cache->getType())
   2456	          .getCallee()
   2457	          ->stripPointerCasts());

On my first pass through, cache->getType() is { i16, i32 }* and getOrInsertFunction creates a new function in the module, instantiationFn->empty() is true and the code builds the function body.

2583 auto call = IGF.Builder.CreateCall(instantiationFn, cache); works fine as the function signature and arguments match.

Next pass, cache->getType() is { i32, i32 }*... now when swift calls...

   2452	  auto instantiationFn = cast<llvm::Function>(
   2453	      IGM.getModule()
   2454	          ->getOrInsertFunction(instantiationFnName, IGF.IGM.TypeMetadataPtrTy,
   2455	                                cache->getType())
   2456	          .getCallee()
   2457	          ->stripPointerCasts());

...it seems to return the previously created function, even though the argument types don't match.

This then causes the crash because auto call = IGF.Builder.CreateCall(instantiationFn, cache); asserts that the function signature and the argument call site attempting to be constructed don't match ({i16, i32}* versus {i32, i32}*).

At first I was naively expecting llvm to create a new function with the same name and different parameters, but thinking about that, it wouldn't make sense for llvm to do that. llvm IR presumably doesn't have function overloading, that's a high level language construct (implemented partly using name mangling) and function names will be lowered to symbols in the object file, so they must be unique.

So after thinking, I sort of half expect an assert in llvm when you try to create a function with the same name but different parameters from an existing function in the model?

The llvm documentation (LLVM: llvm::Module Class Reference) seems to suggest...

Look up the specified function in the module symbol table.

Four possibilities:

If it does not exist, add a prototype for the function and return it.
Otherwise, if the existing function has the correct prototype, return the existing function.
Finally, the function exists but has the wrong prototype: return the function with a constantexpr cast to the right prototype.

...a bit strange and not clearly written. I'm left puzzled.

Anyway, perhaps the root cause is that in 32 bit and 64 bit builds, getAddrOfTypeMetadataDemanglingCacheVariable somehow always returns something with the same signature, either {i32, i32}* or {i64, i32}*... but always the same in all cases for the same platform. For some reason on 16 bit builds, it is varying and that's causing the crash?

Can anyone add more colour?

OK, after some debugging and clearing out the cruft in my brain I more or less realised what's happening. In emitMetadataAccessByMangledName, the call to getAddrOfTypeMetadataDemanglingCacheVariable returns a builder expression that resolves to a {i32, i32}*. This is hard coded from within Linking.cpp by the following lines...

  case Kind::TypeMetadataDemanglingCacheVariable:
    return llvm::StructType::get(IGM.Int32Ty, IGM.Int32Ty);

...based on @Joe_Groff and @Douglas_Gregor 's talk about metadata (2018 LLVM Developers’ Meeting: J. Groff & D. Gregor “Efficiently Implementing Runtime Metadata... ” - YouTube), this is a relative pointer to the metadata and size. The relative pointer is hard coded to 32 bit, because on the platforms of interest until now, 32 bit and 64 bit addresses were all that existed and 32 bit was more efficient for relative offsets (see talk).

The problem comes in emitMetadataAccessByMangledName. When this code runs...

  if (cast<llvm::GlobalVariable>(cache->stripPointerCasts())->isDeclaration()) {
    ConstantInitBuilder builder(IGM);
    auto structBuilder = builder.beginStruct();
    
    // A "negative" 64-bit value in the cache indicates the uninitialized state.
    // Which word has that bit in the {i32, i32} layout depends on endianness.

    if (IGM.getModule()->getDataLayout().isBigEndian()) {
      structBuilder.addInt32(-mangledStringSize);
      structBuilder.addRelativeAddress(mangledString);
      //structBuilder.addRelativeAddress(mangledString, IGM.Int32Ty);
    } else {
      structBuilder.addRelativeAddress(mangledString);
      //structBuilder.addRelativeAddress(mangledString, IGM.Int32Ty);
      structBuilder.addInt32(-mangledStringSize);
    }

    auto init = structBuilder.finishAndCreateFuture();
    cache = IGM.getAddrOfTypeMetadataDemanglingCacheVariable(type, init);
  }

...it changes cache to a builder expression that resolves to {i16, i32}* on my platform (because addRelativeAddress adds a IGM().RelativeAddressTy which is 16 bit on AVR rather than 32 bit on all other platforms... due to one of my earlier bug-fixes).

This causes the crash.

To resolve it a few ideas spring to mind...

  1. change...
  case Kind::TypeMetadataDemanglingCacheVariable:
    return llvm::StructType::get(IGM.Int32Ty, IGM.Int32Ty);

...to...

  case Kind::TypeMetadataDemanglingCacheVariable:
    return llvm::StructType::get(IGM.RelativeAddressTy, IGM.Int32Ty);

(but fixed for endian-ness)

or

  1. change...
    if (IGM.getModule()->getDataLayout().isBigEndian()) {
      structBuilder.addInt32(-mangledStringSize);
      structBuilder.addRelativeAddress(mangledString);
    } else {
      structBuilder.addRelativeAddress(mangledString);
      structBuilder.addInt32(-mangledStringSize);
    }

...to add 32 bit integers for relative addresses.

@Joe_Groff and @Douglas_Gregor ... I wondered on your thoughts? Which do you think I should do?

Cheers,
Carl

I'm surprised that the runtime demangler would fit the constraints of an AVR device at all! You might just want to disable it entirely.

I'm definitely planning to disable most metadata that's not needed.

What's of critical importance at this time is getting the size of the built executable down sufficiently in simple programs.

So the issue is actually static metadata that's getting linked in to statically linked executables.

(Swift) Metadata is emitted in lots of places. In a sample I'm testing, local types (such as structs) emit metadata but it doesn't cause a problem because when the final link is done, if it's not used, it doesn't bloat the binary.

In one example program, just a few lines of swift, this one line of IR changes the statically linke executable from 3kb in size to about 250kb: br i1 icmp eq (i16 ptrtoint (i16* getelementptr inbounds (<{ i8**, i16, <{ i32, i16, i16, i16, i16, i32, i32 }>*, i32, i32, i32 }>, <{ i8**, i16, <{ i32, i16, i16, i16, i16, i32, i32 }>*, i32, i32, i32 }>* @"$s4main5TimerVMf", i16 0, i32 1) to i16), i16 ptrtoint (%swift.type* @"$ss5UInt8VN" to i16)), label %8, label %13, !dbg !81 (even with gc-sections enabled in the linker)

The symbol ss5UInt8VN is undefined in main.o but is found in libSwift.a...
the chain of linker dependencies from there (I think) is this...
ss5UInt8VN references ss5UInt8VMn
ss5UInt8VMn references ss5UInt8VMa
ss5UInt8VMa references ss5UInt8VMf
ss5UInt8VMf references symbolic Bi8_

symbolic Bi8_ is not in its own object file section, but is instead found in swift5_typeref.

swift5_typeref is a huge linker section with loads of undefined symbols right across the standard library, things like ss6UInt64Vs17FixedWidthIntegers9MagnitudeSj_SUWT, ss6UInt64Vs27ExpressibleByIntegerLiterals0dE4TypesACP_s01_bc7BuiltindE0PWT, ss5Int32Vs17FixedWidthIntegers6StrideSx_SZWT... these cause a cascade of code to be added to the final linked binary.

My best guess (without understanding fully) is that swift5_typeref contains reflection metadata. My understanding is reflection metadata isn't needed for anything except reflection, and you can ask the compiler not to emit it by passing the disable-reflection-metadata switch to the front end. So I thought I would try that...

But doing so seems to cause a cascade of compiler crashes to do with this area of emitMetadataAccessByMangledName.

My hope is that if I can fix these crashes, I will end up with static object files and libraries without swift5_typeref sections and the chain of linked unwanted code will be broken.

To be clear, the issue is loads of code is getting linked that as far as I can see is not called or used anywhere, this is probably an artifact of so much stuff getting put into swift5_typeref, so the minute anything references a symbol defined in that section, you get a huge explosion in binary size due to loads of unwanted, unused code being linked in.

Does that explain the root cause of what I was trying to do when I got these compiler crashes?

So I think I've fixed the crashes in my build and managed to stop the explosion of unused code in the binary (described above).

Firstly, to fix the crashes...

Carls-MacBook-Pro:swift carlpeto$ git diff
diff --git a/lib/IRGen/Linking.cpp b/lib/IRGen/Linking.cpp
index 8d8f898192d..1238e141a46 100644
--- a/lib/IRGen/Linking.cpp
+++ b/lib/IRGen/Linking.cpp
@@ -768,7 +768,7 @@ llvm::Type *LinkEntity::getDefaultDeclarationType(IRGenModule &IGM) const {
   case Kind::TypeMetadataLazyCacheVariable:
     return IGM.TypeMetadataPtrTy;
   case Kind::TypeMetadataDemanglingCacheVariable:
-    return llvm::StructType::get(IGM.Int32Ty, IGM.Int32Ty);
+    return llvm::StructType::get(IGM.RelativeAddressTy, IGM.Int32Ty);
   case Kind::TypeMetadataSingletonInitializationCache:
     // TODO: put a cache variable on IGM
     return llvm::StructType::get(IGM.getLLVMContext(),
diff --git a/lib/IRGen/MetadataRequest.cpp b/lib/IRGen/MetadataRequest.cpp
index 8d89f64ac4b..75540f313d9 100644
--- a/lib/IRGen/MetadataRequest.cpp
+++ b/lib/IRGen/MetadataRequest.cpp
@@ -2521,7 +2521,7 @@ emitMetadataAccessByMangledName(IRGenFunction &IGF, CanType type,

       auto stringAddrOffset = subIGF.Builder.CreateTrunc(load,
                                                          IGM.Int32Ty);
-      stringAddrOffset = subIGF.Builder.CreateSExtOrBitCast(stringAddrOffset,
+      stringAddrOffset = subIGF.Builder.CreateSExtOrTrunc(stringAddrOffset,
                                                             IGM.SizeTy);
       auto stringAddrBase = subIGF.Builder.CreatePtrToInt(cache, IGM.SizeTy);
       if (IGM.getModule()->getDataLayout().isBigEndian()) {

(That's not a proper patch. Hard coding to relative address type will break big endian target builds.)

Now those patches were applied, I can use the flag -Xfrontend -disable-reflection-metadata, which fixes my 'explosion' issues in my build.

The nominal type descriptor for UInt8 (for example) changes from...

00000000 <$ss5UInt8VMn>:
   0:	51 00       	.word	0x0051	; ????
   2:	00 00       	nop
   4:	00 00       	nop
			4: R_AVR_16	$ssMXM
   6:	00 00       	nop
			6: R_AVR_16	.rodata.__unnamed_11
   8:	00 00       	nop
			8: R_AVR_16	$ss5UInt8VMa
   a:	00 00       	nop
			a: R_AVR_16	$ss5UInt8VMF
   c:	01 00       	.word	0x0001	; ????
   e:	00 00       	nop
  10:	02 00       	.word	0x0002	; ????
  12:	00 00       	nop

...to...

00000000 <$ss5UInt8VMn>:
   0:	51 00       	.word	0x0051	; ????
   2:	00 00       	nop
   4:	00 00       	nop
			4: R_AVR_16	$ssMXM
   6:	00 00       	nop
			6: R_AVR_16	.rodata.__unnamed_10
   8:	00 00       	nop
			8: R_AVR_16	$ss5UInt8VMa
   a:	00 00       	nop
   c:	00 00       	nop
   e:	01 00       	.word	0x0001	; ????
  10:	00 00       	nop
  12:	02 00       	.word	0x0002	; ????
  14:	00 00       	nop

i.e. there is no pointer to reflection metadata. This stops the undefined symbol ss5UInt8VN being needed in the link, which in turn stops the undefined symbol symbolic Bi8_ being needed, which means that the swift5_typeref section is garbage collected out of the binary. This fixes the problem.

I'm still a bit puzzled why these crashes in emitMetadataAccessByMangledName only start to show up if I use the flag disable-reflection-metadata. Anyone got any ideas?