[Embedded] Assertion failed: (!f->isGeneric()) when compiling my seemingly non generic(?) function

Development Compiler
1

Hi all, I finally rebased my compiler patches on top of recent swift (commit 2544509). It works fine until I turn on -enable-experimental-feature Embedded. Then I start to get assertion failures in IRGen when compiling some of my otherwise stable library code.

Assertion failed: (!f->isGeneric()), function addLazyFunction, file GenDecl.cpp, line 1577.
....
1.	Swift version 5.11-dev (LLVM 30fc8acdf272b61, Swift 28672100783c417)
2.	Compiling with the current language version
3.	While emitting IR SIL function "@$s3AVR5print6buffer10addNewlineyAA15AVRStringBufferV_SbtF".
 for 'print(buffer:addNewline:)' (at serials.swift:88:8)

The crash is in IRGenerator::addLazyFunction...

void IRGenerator::addLazyFunction(SILFunction *f) {
  // Add it to the queue if it hasn't already been put there.
  if (!LazilyEmittedFunctions.insert(f).second)
    return;

  // Embedded Swift doesn't expect any generic functions to be referenced.
  if (SIL.getASTContext().LangOpts.hasFeature(Feature::Embedded)) {
    assert(!f->isGeneric());
  }
...

The function being compiled doesn't seem to have any generic parameters?

@inlinable
public func print(buffer stringBuffer: AVRStringBuffer, addNewline: Bool = true) {
...
}

AVRStringBuffer is a non generic struct (unless I'm going mad):

@frozen
public struct AVRStringBuffer {
	@usableFromInline internal var buffer: UnsafeMutableBufferPointer<Int8>
	@usableFromInline internal var currentIndex: UnsafeMutableBufferPointer<Int8>.Index
...
}

Am I doing something stupid or is this a bug?

Thanks for any advice and help anyone can give.

Carl

1 Like
2

I think the problem here is a symbol referenced from print() and not print() itself. Can you share the implementation?

Also if you attach a debugger to the frontend process that crashed, you can ‘print f->dump();’ to see what the referenced generic function is.

3

np…

It’s not very complex…

@inlinable
public func print(buffer stringBuffer: AVRStringBuffer, addNewline: Bool = true) {
	guard let buffer = stringBuffer.currentValue else {
		return
	}

	_sendBuffer(UInt16(stringBuffer.currentLength), buffer, addNewline, false)
}

Current value and length are…


public typealias CString = Optional<UnsafePointer<Int8>>

extension AVRStringBuffer {
	@_transparent
	public var currentValue: CString {
		return stringCurrentValue(stringBuffer: self)
	}

	@_transparent
	public var currentLength: UInt {
		return stringCurrentLength(stringBuffer: self)
	}
}

@_transparent
public func stringCurrentValue(stringBuffer: AVRStringBuffer) -> UnsafePointer<Int8>? {
	return UnsafePointer<Int8>(stringBuffer.buffer.baseAddress)
}

@_transparent
public func stringCurrentLength(stringBuffer: AVRStringBuffer) -> UInt {
	return UInt(bitPattern: stringBuffer.currentIndex)
}

And _sendBuffer is an imported C function that streams the buffer it’s passed out to a UART peripheral…

// send a buffer from RAM (do not mix these two up!)
void _sendBuffer(unsigned short maxMsgLen, const char * __nonnull buffer, _Bool addNewline, _Bool isString); // must be NULL terminated unless isString = false

The results of e f->dump() were illuminating...

(lldb) e f->dump()
// UnsafeMutableBufferPointer.baseAddress.getter
sil @$sSr11baseAddressSpyxGSgvg : $@convention(method) <Element> (UnsafeMutableBufferPointer<Element>) -> Optional<UnsafeMutablePointer<Element>> {
// %0                                             // user: %1
bb0(%0 : $UnsafeMutableBufferPointer<Element>):
  %1 = struct_extract %0 : $UnsafeMutableBufferPointer<Element>, #UnsafeMutableBufferPointer._position // user: %2
  return %1 : $Optional<UnsafeMutablePointer<Element>> // id: %2
} // end sil function '$sSr11baseAddressSpyxGSgvg'

demangling this name...
$sSr11baseAddressSpyxGSgvg ---> Swift.UnsafeMutableBufferPointer.baseAddress.getter : Swift.UnsafeMutablePointer<A>?

So it looks like the issue is coming at root from the baseAddress getter being seen as generic. This is all defined the same as in the normal standard library...

@frozen // unsafe-performance
public struct UnsafeMutableBufferPointer<Element> {

  @usableFromInline
  let _position: UnsafeMutablePointer<Element>?

  public let count: Int
}

extension UnsafeMutableBufferPointer {
...
  @inlinable // unsafe-performance
  public var baseAddress: UnsafeMutablePointer<Element>? {
    return _position
  }
}

I must admit I'm pretty stumped how I might go about fixing this? I suppose I expected this accessor function would have been inlined away into the calling function before the SIL was lowered through IRGen?

Am I missing something or is there an optimisation I need to add? I don't think I need to add @inline(__always) to the baseAddress getter, because I thought that the Embedded mode effectively does that everywhere?

Grateful of any pointers...

Carl

1 Like
4

A bit further update... For the sake of trying it, I changed my standard library to really force inlining on the method that was being lowered and causing the assertion failure...

  @_transparent // unsafe-performance
  public var baseAddress: UnsafeMutablePointer<Element>? {
    return _position
  }

Changed from @inlinable to @_transparent, which shouldn't make any difference in Swift Embedded mode but it stopped the crash! And instead it asserted somewhere else...

0.	Program arguments: /Users/petoc01/Documents/Code/swift/build/Ninja-DebugAssert+stdlib-ReleaseAssert/swift-macosx-x86_64/bin/swift-frontend -frontend -emit-bc serials.swift utils.swift i2c.swift spi.swift asyncs.swift timers.swift stringBuffer.swift stringOther.swift version.swift -supplementary-output-file-map /var/folders/p9/dy3nmf315svbkqpz9z78jxmc0000gn/T/supplementaryOutputs-df6284 -target avr-atmel-linux-gnueabihf -disable-objc-interop -color-diagnostics -I ../uSwift/bin/AVR -I ../uSwift -I . -I AVR/bin -I "/Applications/Swift For Arduino.app/Contents/XPCServices/BuildEngine.xpc/Contents/Resources/gpl-tools-avr/lib/avr-libgcc/include" -I "/Applications/Swift For Arduino.app/Contents/XPCServices/BuildEngine.xpc/Contents/Resources/gpl-tools-avr/lib/avr-libc/include" -enable-experimental-feature Embedded -import-underlying-module -nostdimport -enforce-exclusivity=unchecked -Osize -D AVR_LIBC_DEFINED_SWIFT -disable-reflection-metadata -plugin-path /Users/petoc01/Documents/Code/swift/build/Ninja-DebugAssert+stdlib-ReleaseAssert/swift-macosx-x86_64/lib/swift/host/plugins -plugin-path /Users/petoc01/Documents/Code/swift/build/Ninja-DebugAssert+stdlib-ReleaseAssert/swift-macosx-x86_64/local/lib/swift/host/plugins -Xcc -DAVR_LIBC_DEFINED -Xcc -DLIBC_DEFINED -parse-as-library -module-name AVR -num-threads 4 -o bin/serials.bc -o bin/utils.bc -o bin/i2c.bc -o bin/spi.bc -o bin/asyncs.bc -o bin/timers.bc -o bin/stringBuffer.bc -o bin/stringOther.bc -o bin/version.bc
1.	Swift version 5.11-dev (LLVM 30fc8acdf272b61, Swift 28672100783c417)
2.	Compiling with the current language version
3.	While emitting IR SIL function "@$s3AVR5print6buffer10addNewlineyAA15AVRStringBufferV_SbtF".
 for 'print(buffer:addNewline:)' (at serials.swift:88:8)
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0  swift-frontend           0x000000010c037ffd llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 61
1  swift-frontend           0x000000010c0384cb PrintStackTraceSignalHandler(void*) + 27
2  swift-frontend           0x000000010c036746 llvm::sys::RunSignalHandlers() + 134
3  swift-frontend           0x000000010c0390ff SignalHandler(int) + 223
4  libsystem_platform.dylib 0x00007ff814e245ed _sigtramp + 29
5  libsystem_platform.dylib 000000000000000000 _sigtramp + 18446603370230561328
6  libsystem_c.dylib        0x00007ff814d1db45 abort + 123
7  libsystem_c.dylib        0x00007ff814d1ce5e err + 0
8  swift-frontend           0x0000000100abdf4d swift::irgen::IRGenerator::addLazyFunction(swift::SILFunction*) + 173
9  swift-frontend           0x0000000100ac1971 swift::irgen::IRGenModule::getAddrOfSILFunction(swift::SILFunction*, swift::ForDefinition_t, bool, bool) + 1809
10 swift-frontend           0x0000000100e578fe (anonymous namespace)::IRGenSILFunction::visitFunctionRefBaseInst(swift::FunctionRefBaseInst*) + 254
11 swift-frontend           0x0000000100e355cd (anonymous namespace)::IRGenSILFunction::visitFunctionRefInst(swift::FunctionRefInst*) + 29
12 swift-frontend           0x0000000100e30587 swift::SILInstructionVisitor<(anonymous namespace)::IRGenSILFunction, void>::visit(swift::SILInstruction*) + 391
13 swift-frontend           0x0000000100e1958e (anonymous namespace)::IRGenSILFunction::visitSILBasicBlock(swift::SILBasicBlock*) + 1422
14 swift-frontend           0x0000000100e0e457 (anonymous namespace)::IRGenSILFunction::emitSILFunction() + 2791
15 swift-frontend           0x0000000100e0d898 swift::irgen::IRGenModule::emitSILFunction(swift::SILFunction*) + 328
16 swift-frontend           0x0000000100aba805 swift::irgen::IRGenerator::emitGlobalTopLevel(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&) + 1397
17 swift-frontend           0x0000000100d09782 performParallelIRGeneration(swift::IRGenDescriptor) + 1218
18 swift-frontend           0x0000000100d0917a swift::performIRGeneration(swift::ModuleDecl*, swift::IRGenOptions const&, swift::TBDGenOptions const&, std::__1::unique_ptr<swift::SILModule, std::__1::default_delete<swift::SILModule>>, llvm::StringRef, swift::PrimarySpecificPaths const&, llvm::ArrayRef<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, llvm::GlobalVariable**) + 554
19 swift-frontend           0x0000000100178b2e generateIR(swift::IRGenOptions const&, swift::TBDGenOptions const&, std::__1::unique_ptr<swift::SILModule, std::__1::default_delete<swift::SILModule>>, swift::PrimarySpecificPaths const&, llvm::StringRef, llvm::PointerUnion<swift::ModuleDecl*, swift::SourceFile*>, llvm::GlobalVariable*&, llvm::ArrayRef<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>) + 542
20 swift-frontend           0x0000000100172bb3 performCompileStepsPostSILGen(swift::CompilerInstance&, std::__1::unique_ptr<swift::SILModule, std::__1::default_delete<swift::SILModule>>, llvm::PointerUnion<swift::ModuleDecl*, swift::SourceFile*>, swift::PrimarySpecificPaths const&, int&, swift::FrontendObserver*) + 2179
21 swift-frontend           0x0000000100171c64 swift::performCompileStepsPostSema(swift::CompilerInstance&, int&, swift::FrontendObserver*) + 388
22 swift-frontend           0x00000001001a1413 performAction(swift::CompilerInstance&, int&, swift::FrontendObserver*)::$_30::operator()(swift::CompilerInstance&) const + 147
23 swift-frontend           0x00000001001a136d bool llvm::function_ref<bool (swift::CompilerInstance&)>::callback_fn<performAction(swift::CompilerInstance&, int&, swift::FrontendObserver*)::$_30>(long, swift::CompilerInstance&) + 29
24 swift-frontend           0x00000001001a0731 llvm::function_ref<bool (swift::CompilerInstance&)>::operator()(swift::CompilerInstance&) const + 33
25 swift-frontend           0x000000010019f798 withSemanticAnalysis(swift::CompilerInstance&, swift::FrontendObserver*, llvm::function_ref<bool (swift::CompilerInstance&)>, bool) + 392
26 swift-frontend           0x0000000100198105 performAction(swift::CompilerInstance&, int&, swift::FrontendObserver*) + 1349
27 swift-frontend           0x0000000100175086 performCompile(swift::CompilerInstance&, int&, swift::FrontendObserver*) + 262
28 swift-frontend           0x0000000100173a66 swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*) + 2822
29 swift-frontend           0x0000000100046324 run_driver(llvm::StringRef, llvm::ArrayRef<char const*>, llvm::ArrayRef<char const*>) + 388

This time the function being lowered is...

(lldb) e fn->dump()
// FixedWidthInteger.bitWidth.getter
sil @$ss17FixedWidthIntegerPsE03bitB0Sivg : $@convention(method) <Self where Self : FixedWidthInteger> (@in_guaranteed Self) -> Int {
bb0(%0 : $*Self):
  %1 = metatype $@thick Self.Type                 // user: %3
  %2 = witness_method $Self, #FixedWidthInteger.bitWidth!getter : <Self where Self : FixedWidthInteger> (Self.Type) -> () -> Int : $@convention(witness_method: FixedWidthInteger) <τ_0_0 where τ_0_0 : FixedWidthInteger> (@thick τ_0_0.Type) -> Int // user: %3
  %3 = apply %2<Self>(%1) : $@convention(witness_method: FixedWidthInteger) <τ_0_0 where τ_0_0 : FixedWidthInteger> (@thick τ_0_0.Type) -> Int // user: %4
  return %3 : $Int                                // id: %4
} // end sil function '$ss17FixedWidthIntegerPsE03bitB0Sivg'

I moved up a few frames to try and get some context. this is the SIL that's being lowered in IRGen and causing the crash. Again, it seems wrong to me because the bitWidth getter should have been inlined by the SIL Optimiser long before it reached IRGen shouldn't it?

SWITCH TO FRAME 15

(lldb) e f->dump()
// print(buffer:addNewline:)
sil @$s3AVR5print6buffer10addNewlineyAA15AVRStringBufferV_SbtF : $@convention(thin) (AVRStringBuffer, Bool) -> () {
// %0 "stringBuffer"                              // users: %4, %58, %2
// %1 "addNewline"                                // users: %70, %3
bb0(%0 : $AVRStringBuffer, %1 : $Bool):
  debug_value %0 : $AVRStringBuffer, let, name "stringBuffer", argno 1 // id: %2
  debug_value %1 : $Bool, let, name "addNewline", argno 2 // id: %3
  %4 = struct_extract %0 : $AVRStringBuffer, #AVRStringBuffer.buffer // user: %5
  %5 = struct_extract %4 : $UnsafeMutableBufferPointer<Int8>, #UnsafeMutableBufferPointer._position // user: %6
  switch_enum %5 : $Optional<UnsafeMutablePointer<Int8>>, case #Optional.some!enumelt: bb2, case #Optional.none!enumelt: bb1 // id: %6

bb1:                                              // Preds: bb0
  br bb13                                         // id: %7

// %8                                             // user: %9
bb2(%8 : $UnsafeMutablePointer<Int8>):            // Preds: bb0
  %9 = struct_extract %8 : $UnsafeMutablePointer<Int8>, #UnsafeMutablePointer._rawValue // users: %10, %56
  %10 = builtin "ptrtoint_Word"(%9 : $Builtin.RawPointer) : $Builtin.Word // user: %11
  %11 = builtin "truncOrBitCast_Word_Int16"(%10 : $Builtin.Word) : $Builtin.Int16 // users: %53, %34, %51, %13, %15
  %12 = integer_literal $Builtin.Int16, 0         // users: %51, %13
  %13 = builtin "cmp_slt_Int16"(%11 : $Builtin.Int16, %12 : $Builtin.Int16) : $Builtin.Int1 // user: %14
  cond_br %13, bb4, bb3                           // id: %14

bb3:                                              // Preds: bb2
  %15 = struct $Int (%11 : $Builtin.Int16)        // users: %38, %19
  %16 = integer_literal $Builtin.Int8, 0          // users: %54, %35, %17
  %17 = struct $UInt8 (%16 : $Builtin.Int8)       // users: %41, %23
  %18 = alloc_stack [lexical] $Int                // users: %19, %29, %21
  store %15 to %18 : $*Int                        // id: %19
  // function_ref FixedWidthInteger.bitWidth.getter
  %20 = function_ref @$ss17FixedWidthIntegerPsE03bitB0Sivg : $@convention(method) <τ_0_0 where τ_0_0 : FixedWidthInteger> (@in_guaranteed τ_0_0) -> Int // users: %42, %39, %24, %21
  %21 = apply %20<Int>(%18) : $@convention(method) <τ_0_0 where τ_0_0 : FixedWidthInteger> (@in_guaranteed τ_0_0) -> Int // user: %25
  %22 = alloc_stack [lexical] $UInt8              // users: %23, %28, %24
  store %17 to %22 : $*UInt8                      // id: %23
  %24 = apply %20<UInt8>(%22) : $@convention(method) <τ_0_0 where τ_0_0 : FixedWidthInteger> (@in_guaranteed τ_0_0) -> Int // user: %26
  %25 = struct_extract %21 : $Int, #Int._value    // user: %27
  %26 = struct_extract %24 : $Int, #Int._value    // user: %27
  %27 = builtin "cmp_slt_Int16"(%25 : $Builtin.Int16, %26 : $Builtin.Int16) : $Builtin.Int1 // user: %30
  dealloc_stack %22 : $*UInt8                     // id: %28
  dealloc_stack %18 : $*Int                       // id: %29
  cond_br %27, bb7, bb8                           // id: %30

bb4:                                              // Preds: bb2
  br bb12                                         // id: %31

bb5:                                              // Preds: bb9
  br bb12                                         // id: %32

bb6:                                              // Preds: bb9
  br bb13                                         // id: %33

bb7:                                              // Preds: bb3
  %34 = builtin "truncOrBitCast_Int16_Int8"(%11 : $Builtin.Int16) : $Builtin.Int8 // user: %35
  %35 = builtin "cmp_eq_Int8"(%34 : $Builtin.Int8, %16 : $Builtin.Int8) : $Builtin.Int1 // user: %36
  br bb9(%35 : $Builtin.Int1)                     // id: %36

bb8:                                              // Preds: bb3
  %37 = alloc_stack [lexical] $Int                // users: %38, %47, %39
  store %15 to %37 : $*Int                        // id: %38
  %39 = apply %20<Int>(%37) : $@convention(method) <τ_0_0 where τ_0_0 : FixedWidthInteger> (@in_guaranteed τ_0_0) -> Int // user: %44
  %40 = alloc_stack [lexical] $UInt8              // users: %41, %46, %42
  store %17 to %40 : $*UInt8                      // id: %41
  %42 = apply %20<UInt8>(%40) : $@convention(method) <τ_0_0 where τ_0_0 : FixedWidthInteger> (@in_guaranteed τ_0_0) -> Int // user: %43
  %43 = struct_extract %42 : $Int, #Int._value    // user: %45
  %44 = struct_extract %39 : $Int, #Int._value    // user: %45
  %45 = builtin "cmp_slt_Int16"(%43 : $Builtin.Int16, %44 : $Builtin.Int16) : $Builtin.Int1 // user: %48
  dealloc_stack %40 : $*UInt8                     // id: %46
  dealloc_stack %37 : $*Int                       // id: %47
  cond_br %45, bb10, bb11                         // id: %48

// %49                                            // user: %50
bb9(%49 : $Builtin.Int1):                         // Preds: bb11 bb10 bb7
  cond_br %49, bb6, bb5                           // id: %50

bb10:                                             // Preds: bb8
  %51 = builtin "cmp_eq_Int16"(%11 : $Builtin.Int16, %12 : $Builtin.Int16) : $Builtin.Int1 // user: %52
  br bb9(%51 : $Builtin.Int1)                     // id: %52

bb11:                                             // Preds: bb8
  %53 = builtin "truncOrBitCast_Int16_Int8"(%11 : $Builtin.Int16) : $Builtin.Int8 // user: %54
  %54 = builtin "cmp_eq_Int8"(%53 : $Builtin.Int8, %16 : $Builtin.Int8) : $Builtin.Int1 // user: %55
  br bb9(%54 : $Builtin.Int1)                     // id: %55

bb12:                                             // Preds: bb5 bb4
  %56 = struct $UnsafePointer<Int8> (%9 : $Builtin.RawPointer) // users: %57, %70
  debug_value %56 : $UnsafePointer<Int8>, let, name "buffer" // id: %57
  %58 = struct_extract %0 : $AVRStringBuffer, #AVRStringBuffer.currentIndex // user: %59
  %59 = struct_extract %58 : $Int, #Int._value    // users: %66, %61
  %60 = alloc_stack $UInt                         // users: %64, %62, %65
  %61 = struct $UInt (%59 : $Builtin.Int16)       // user: %62
  store %61 to %60 : $*UInt                       // id: %62
  // function_ref FixedWidthInteger.bitWidth.getter
  %63 = function_ref @$ss17FixedWidthIntegerPsE03bitB0Sivg : $@convention(method) <τ_0_0 where τ_0_0 : FixedWidthInteger> (@in_guaranteed τ_0_0) -> Int // user: %64
  %64 = apply %63<UInt>(%60) : $@convention(method) <τ_0_0 where τ_0_0 : FixedWidthInteger> (@in_guaranteed τ_0_0) -> Int
  dealloc_stack %60 : $*UInt                      // id: %65
  %66 = struct $UInt16 (%59 : $Builtin.Int16)     // user: %70
  %67 = integer_literal $Builtin.Int1, 0          // user: %68
  %68 = struct $Bool (%67 : $Builtin.Int1)        // user: %70
  // function_ref _sendBuffer
  %69 = function_ref @_sendBuffer : $@convention(c) (UInt16, UnsafePointer<Int8>, Bool, Bool) -> () // user: %70
  %70 = apply %69(%66, %56, %1, %68) : $@convention(c) (UInt16, UnsafePointer<Int8>, Bool, Bool) -> ()
  br bb13                                         // id: %71

bb13:                                             // Preds: bb1 bb6 bb12
  %72 = tuple ()                                  // user: %73
  return %72 : $()                                // id: %73
} // end sil function '$s3AVR5print6buffer10addNewlineyAA15AVRStringBufferV_SbtF'

Hoping this gives a bit more context on what I'm doing wrong?

Might -Osize be contributing to the problem?

Tagging @kubamracek to see if he's got any ideas too?

Thanks for any help you can give.

Regards,
Carl

1 Like
5

Do you have details about the compiler being used here? I guess it's locally built and not downloaded from swift.org? Somehow, in the last snippet you posted, there is a use of an unspecialized function through a wtable (FixedWidthInteger.bitWidth.getter) and that should never be present in SIL in Embedded Swift. Somehow the mandatory specializer didn't handle it.

Could you try (1) a newer commit on the main branch, and (2) check if you're by any chance building the compiler without SwiftCompilerSources (e.g. by having BOOTSTRAPPING_MODE=OFF)? Embedded Swift won't work without those.

6

It sounds like it's bootstrapping (which I have turned off) that's the problem.

--
Our compiler is locally built on our CI from our fork it's not public but I can give you access to our repo on a private channel if you want it, we are planning to try and upstream as much of it as possible "soon". I don't think our fork has very much of interest to anyone else, mostly it's things like handling llvm addresspace(1) pointers and 16 bit pointers better, which has zero impact on any target other than AVR.

Our base is 254450981a5 which is from 8th October, so it's fairly recent, it's very easy to rebase up to today if that will help but it sounds like that's not the problem. We have a number of essential (for us) patches on top of that. (Our llvm-project is based on cdb60493231c0e1acdf34f1eb70d47b77c0b2b4c with some very small patches.)

I use the checkout and build scripts to run CMake, but I turn off most things because they're not applicable (yet) in our constrained environment.

utils/update-checkout --clone-with-ssh --skip-repository swift-nio \
--skip-repository swift-nio-ssh --skip-repository swift-lmdb --skip-repository swift-docc \
--skip-repository swift-docc-render-artifact --skip-repository swift-docc-symbolkit \
--skip-repository swift-markdown --skip-repository swift-experimental-string-processing \
--skip-repository swift-llvm-bindings --skip-repository swift-xcode-playground-support \
--skip-repository swift-corelibs-libdispatch --skip-repository swift-corelibs-foundation \
--skip-repository swift-corelibs-xctest --skip-repository swift-stress-tester \
--skip-repository swift-crypto --skip-repository swift-atomics \
--skip-repository swift-nio-ssl --skip-repository sourcekit-lsp \
--skip-repository indexstore-db --skip-repository swiftpm \
--skip-repository swift-numerics --skip-repository swift

...fetch our local fork of llvm-project (not very different, just Swift added to the AVR ABI pretty much)...

utils/build-script -R -S --clean --extra-cmake-options="-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=AVR;ARM"  --extra-cmake-options="-DLLVM_ENABLE_PROJECTS='clang'" \
--skip-build-benchmarks --skip-ios --skip-watchos --skip-tvos --darwin-deployment-version-osx 10.15 \
--skip-early-swift-driver=true --skip-early-swiftsyntax=true \
--swift-driver=false --swift-disable-dead-stripping --bootstrapping=off

... then go into the build directories and build cmark with `ninja`, llvm with `ninja` and just the swift frontend with `ninja swift-frontend`...

Our compiler doesn't necessarily actually work on Darwin, just because it wasn't a priority.

The reason ours is like this? ... We were on 5.3 base until recently. When I upgraded to Swift 5.8 / Swift 5.9 base, I ended up disabling a lot of "the fancy new stuff" that I didn't really understand, which seemingly includes SwiftCompilerSources.

So it looks like you've got some of the compiler pipeline done in swift itself now? So the compiler is a sort of Frankenstein of some bits are Swift and some are still C++? I did vaguely know about this, but I thought it was "only the swift driver" so it wouldn't really make any material difference to the compile. Having looked, it seems like quite a lot of the meat of compilation is being done in the new pure swift compiler?

And am I right that you generally did most of the work to add Embedded Swift features in the Swift part of the compiler (which I have disabled!) so that's why my Embedded Swift is not behaving like Embedded Swift?

Regards,
Carl

7

Some major parts of Embedded Swift are relying on features in the Swift parts of the compiler, yes. Also I think it's likely that building with SwiftCompilerSources will become mandatory (and the bootstrapping=off option would be removed) because there are more and more load-bearing features in there (MandatoryPerformanceOptimizations used in Embedded Swift are just one such example).

1 Like
8

Alright. I’ll try turning on the bootstrapping and see if the compiler still builds and runs. From memory the last time I tried to do that the compiler kept crashing saying various shared libraries were missing, which is why I gave up. But I’ll try again!

2 Likes
9

So this is what happens when I try to compile swift-frontend with ninja swift-frontend (which is what I normally do).

[78/88] Linking CXX executable bootstrapping1/bin/swift-frontend
FAILED: bootstrapping1/bin/swift-frontend 
: && /Applications/Xcode-15.0.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -Wno-unknown-warning-option -Werror=unguarded-availability-new -fno-stack-protector -fPIC -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wimplicit-fallthrough -Wcovered-switch-default -Wno-class-memaccess -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -Werror=switch -Wdocumentation -Wimplicit-fallthrough -Wunreachable-code -Woverloaded-virtual -DOBJC_OLD_DISPATCH_PROTOTYPES=0 -O3 -DNDEBUG -arch x86_64 -isysroot /Applications/Xcode-15.0.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names -Wl,-add_ast_path,/Users/petoc01/Documents/Code/swift/build/Ninja-ReleaseAssert/swift-macosx-x86_64/SwiftCompilerSources/Basic.swiftmodule,-add_ast_path,/Users/petoc01/Documents/Code/swift/build/Ninja-ReleaseAssert/swift-macosx-x86_64/SwiftCompilerSources/AST.swiftmodule,-add_ast_path
...
Code/swift/build/Ninja-ReleaseAssert/llvm-macosx-x86_64/lib/libLLVMBinaryFormat.a  /Users/petoc01/Documents/Code/swift/build/Ninja-ReleaseAssert/llvm-macosx-x86_64/lib/libclangLex.a  /Users/petoc01/Documents/Code/swift/build/Ninja-ReleaseAssert/llvm-macosx-x86_64/lib/libclangBasic.a  lib/libswiftMarkup.a  /Users/petoc01/Documents/Code/swift/build/Ninja-ReleaseAssert/cmark-macosx-x86_64/src/libcmark-gfm.a  /Users/petoc01/Documents/Code/swift/build/Ninja-ReleaseAssert/llvm-macosx-x86_64/lib/libLLVMBitstreamReader.a  /Users/petoc01/Documents/Code/swift/build/Ninja-ReleaseAssert/llvm-macosx-x86_64/lib/libLLVMSupport.a  -lm && cd /Users/petoc01/Documents/Code/swift/build/Ninja-ReleaseAssert/swift-macosx-x86_64/bootstrapping1/bin && /Applications/CMake.app/Contents/bin/cmake -E create_symlink swift-frontend swiftc
ld: warning: ignoring duplicate libraries: '/Users/petoc01/Documents/Code/swift/build/Ninja-ReleaseAssert/llvm-macosx-x86_64/lib/libLLVMDemangle.a', 'lib/libswiftSIL.a', 'lib/libswiftSema.a', 'lib/libswiftSerialization.a'
ld: warning: Could not find or use auto-linked library 'swiftCompatibility51': library 'swiftCompatibility51' not found
ld: warning: Could not find or use auto-linked library 'swiftCompatibility56': library 'swiftCompatibility56' not found
ld: warning: Could not find or use auto-linked library 'swiftCompatibilityConcurrency': library 'swiftCompatibilityConcurrency' not found
ld: warning: Could not find or use auto-linked library 'swiftCompatibilityPacks': library 'swiftCompatibilityPacks' not found
ld: warning: Could not find or use auto-linked library 'swiftCxx': library 'swiftCxx' not found
ld: warning: Could not find or use auto-linked library 'swiftCxxStdlib': library 'swiftCxxStdlib' not found
ld: Undefined symbols:
  __swift_FORCE_LOAD_$_swiftCompatibility51, referenced from:
      __swift_FORCE_LOAD_$_swiftCompatibility51_$_Basic in libswiftCompilerModules-bootstrapping1.a[2](Basic.o)
      __swift_FORCE_LOAD_$_swiftCompatibility51_$_SIL in libswiftCompilerModules-bootstrapping1.a[5](SIL.o)
      __swift_FORCE_LOAD_$_swiftCompatibility51_$_Optimizer in libswiftCompilerModules-bootstrapping1.a[6](Optimizer.o)
  __swift_FORCE_LOAD_$_swiftCompatibility56, referenced from:
      __swift_FORCE_LOAD_$_swiftCompatibility56_$_Basic in libswiftCompilerModules-bootstrapping1.a[2](Basic.o)
      __swift_FORCE_LOAD_$_swiftCompatibility56_$_SIL in libswiftCompilerModules-bootstrapping1.a[5](SIL.o)
      __swift_FORCE_LOAD_$_swiftCompatibility56_$_Optimizer in libswiftCompilerModules-bootstrapping1.a[6](Optimizer.o)
  __swift_FORCE_LOAD_$_swiftCompatibilityConcurrency, referenced from:
      __swift_FORCE_LOAD_$_swiftCompatibilityConcurrency_$_Basic in libswiftCompilerModules-bootstrapping1.a[2](Basic.o)
      __swift_FORCE_LOAD_$_swiftCompatibilityConcurrency_$_SIL in libswiftCompilerModules-bootstrapping1.a[5](SIL.o)
      __swift_FORCE_LOAD_$_swiftCompatibilityConcurrency_$_Optimizer in libswiftCompilerModules-bootstrapping1.a[6](Optimizer.o)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

Not sure if it's something obvious? I normally use Xcode 14 but it doesn't look like that's the issue?

10

Hi,

@kubamracek @rauhul

It took a while, but I got a version of our compiler built using bootstrapping (with host tools). But it's still crashing when compiling our stdlib when embedded mode is turned on.

The assertion is...
Assertion failed: (!shouldSkipDecl(fd)), function emitFunction, file SILGen.cpp, line 1437.

The compiler is...
1. Swift version 5.11-dev (LLVM d50917983d84235, Swift 6814217eeb3b1fc)
(GitHub - carlos4242/swift at avr-mods-1)

Here's the full backtrace...

* thread #1, queue = 'com.apple.main-thread', stop reason = hit program assert
    frame #0: 0x00007ff802c847ce libsystem_kernel.dylib`__pthread_kill + 10
    frame #1: 0x00007ff802cbcf30 libsystem_pthread.dylib`pthread_kill + 262
    frame #2: 0x00007ff802bdba49 libsystem_c.dylib`abort + 126
    frame #3: 0x00007ff802bdad30 libsystem_c.dylib`__assert_rtn + 314
  * frame #4: 0x00000001010268ca swift-frontend`swift::Lowering::SILGenModule::emitFunction(this=0x000000031b327298, fd=0x00007fc6cb883a38) at SILGen.cpp:1437:3
    frame #5: 0x00000001012a13e0 swift-frontend`SILGenExtension::visitFuncDecl(this=0x000000031b326c58, fd=0x00007fc6cb883a38) at SILGenType.cpp:1346:9
    frame #6: 0x00000001012a1990 swift-frontend`SILGenExtension::visitAccessors(this=0x000000031b326928, accessor=0x00007fc6cb883a38)::'lambda'(swift::AccessorDecl*)::operator()(swift::AccessorDecl*) const at SILGenType.cpp:1426:7
    frame #7: 0x00000001012a195d swift-frontend`void llvm::function_ref<void (swift::AccessorDecl*)>::callback_fn<SILGenExtension::visitAccessors(swift::AbstractStorageDecl*)::'lambda'(swift::AccessorDecl*)>(callable=13341190440, params=0x00007fc6cb883a38) at STLFunctionalExtras.h:45:12
    frame #8: 0x000000010330b211 swift-frontend`llvm::function_ref<void (swift::AccessorDecl*)>::operator()(this=0x000000031b3268a0, params=0x00007fc6cb883a38) const at STLFunctionalExtras.h:68:12
    frame #9: 0x0000000103718582 swift-frontend`swift::AbstractStorageDecl::visitParsedAccessors(this=0x00007fc6cb8833c8, visit=function_ref<void (swift::AccessorDecl *)> @ 0x000000031b3268a0) const at Decl.cpp:2897:7
    frame #10: 0x00000001037185d9 swift-frontend`swift::AbstractStorageDecl::visitEmittedAccessors(this=0x00007fc6cb8833c8, visit=function_ref<void (swift::AccessorDecl *)> @ 0x000000031b326900) const at Decl.cpp:2902:3
    frame #11: 0x00000001012a1855 swift-frontend`SILGenExtension::visitAccessors(this=0x000000031b326c58, asd=0x00007fc6cb8833c8) at SILGenType.cpp:1425:10
    frame #12: 0x00000001012a18bb swift-frontend`SILGenExtension::visitAbstractStorageDecl(this=0x000000031b326c58, asd=0x00007fc6cb8833c8) at SILGenType.cpp:1421:5
    frame #13: 0x00000001012a11d8 swift-frontend`SILGenExtension::visitSubscriptDecl(this=0x000000031b326c58, sd=0x00007fc6cb8833a8) at SILGenType.cpp:1408:5
    frame #14: 0x00000001012a0a07 swift-frontend`swift::ASTVisitor<SILGenExtension, void, void, void, void, void, void>::visit(this=0x000000031b326c58, D=0x00007fc6cb8833c8) at DeclNodes.def:173:5
    frame #15: 0x00000001012a06e3 swift-frontend`SILGenExtension::visit(this=0x000000031b326c58, D=0x00007fc6cb8833c8) at SILGenType.cpp:1310:24
    frame #16: 0x0000000101290d4c swift-frontend`SILGenExtension::emitExtension(this=0x000000031b326c58, e=0x00007fc6cb882408) at SILGenType.cpp:1281:7
    frame #17: 0x0000000101290c8a swift-frontend`swift::Lowering::SILGenModule::visitExtensionDecl(this=0x000000031b327298, ed=0x00007fc6cb882408) at SILGenType.cpp:1440:26
    frame #18: 0x0000000101026630 swift-frontend`swift::ASTVisitor<swift::Lowering::SILGenModule, void, void, void, void, void, void>::visit(this=0x000000031b327298, D=0x00007fc6cb882428) at DeclNodes.def:185:1
    frame #19: 0x0000000101026100 swift-frontend`swift::Lowering::SILGenModule::visit(this=0x000000031b327298, D=0x00007fc6cb882428) at SILGen.cpp:832:15
    frame #20: 0x00000001010301a4 swift-frontend`(anonymous namespace)::SILGenModuleRAII::emitSourceFile(this=0x000000031b327298, sf=0x00007fc6cd03e020) at SILGen.cpp:2000:11
    frame #21: 0x000000010102fd1e swift-frontend`swift::ASTLoweringRequest::evaluate(this=0x000000031b327b10, evaluator=0x00007fc6cd834270, desc=ASTLoweringDescriptor @ 0x000000031b3278e0) const at SILGen.cpp:2138:13
    frame #22: 0x000000010126ff6e swift-frontend`std::__1::unique_ptr<swift::SILModule, std::__1::default_delete<swift::SILModule>> swift::SimpleRequest<swift::ASTLoweringRequest, std::__1::unique_ptr<swift::SILModule, std::__1::default_delete<swift::SILModule>> (swift::ASTLoweringDescriptor), (swift::RequestFlags)9>::callDerived<0ul>(this=0x000000031b327b10, evaluator=0x00007fc6cd834270, (null)=std::__1::index_sequence<0UL> @ 0x000000031b3278d0) const at SimpleRequest.h:267:24
    frame #23: 0x000000010126fe88 swift-frontend`swift::SimpleRequest<swift::ASTLoweringRequest, std::__1::unique_ptr<swift::SILModule, std::__1::default_delete<swift::SILModule>> (swift::ASTLoweringDescriptor), (swift::RequestFlags)9>::evaluateRequest(request=0x000000031b327b10, evaluator=0x00007fc6cd834270) at SimpleRequest.h:290:20
    frame #24: 0x0000000101043dba swift-frontend`llvm::Expected<swift::ASTLoweringRequest::OutputType> swift::Evaluator::getResultUncached<swift::ASTLoweringRequest>(this=0x00007fc6cd834270, request=0x000000031b327b10) at Evaluator.h:368:21
    frame #25: 0x0000000101030978 swift-frontend`llvm::Expected<swift::ASTLoweringRequest::OutputType> swift::Evaluator::operator()<swift::ASTLoweringRequest, (void*)0>(this=0x00007fc6cd834270, request=0x000000031b327b10) at Evaluator.h:273:12
    frame #26: 0x0000000101030870 swift-frontend`swift::performASTLowering(mod=0x00007fc6cb83e850, tc=0x00007fc67b912870, options=0x000000031b328100, irgenOptions=0x000000031b3288d8) at SILGen.cpp:2162:7
    frame #27: 0x00000001002c05c1 swift-frontend`swift::performCompileStepsPostSema(Instance=0x00007fc6cc80c200, ReturnValue=0x000000031b32937c, observer=0x0000000000000000) at FrontendTool.cpp:872:15
    frame #28: 0x00000001002f0863 swift-frontend`performAction(swift::CompilerInstance&, int&, swift::FrontendObserver*)::$_29::operator()(this=0x000000031b328f00, Instance=0x00007fc6cc80c200) const at FrontendTool.cpp:1423:18
    frame #29: 0x00000001002f07bd swift-frontend`bool llvm::function_ref<bool (swift::CompilerInstance&)>::callback_fn<performAction(swift::CompilerInstance&, int&, swift::FrontendObserver*)::$_29>(callable=13341200128, params=0x00007fc6cc80c200) at STLFunctionalExtras.h:45:12
    frame #30: 0x00000001002efb81 swift-frontend`llvm::function_ref<bool (swift::CompilerInstance&)>::operator()(this=0x000000031b328e38, params=0x00007fc6cc80c200) const at STLFunctionalExtras.h:68:12
    frame #31: 0x00000001002eeb18 swift-frontend`withSemanticAnalysis(Instance=0x00007fc6cc80c200, observer=0x0000000000000000, cont=function_ref<bool (swift::CompilerInstance &)> @ 0x000000031b328e38, runDespiteErrors=false) at FrontendTool.cpp:1283:10
    frame #32: 0x00000001002e74d5 swift-frontend`performAction(Instance=0x00007fc6cc80c200, ReturnValue=0x000000031b32937c, observer=0x0000000000000000) at FrontendTool.cpp:1419:12
    frame #33: 0x00000001002c3a86 swift-frontend`performCompile(Instance=0x00007fc6cc80c200, ReturnValue=0x000000031b32937c, observer=0x0000000000000000) at FrontendTool.cpp:1494:19
    frame #34: 0x00000001002c246c swift-frontend`swift::performFrontend(Args=ArrayRef<const char *> @ 0x000000031b3295c0, Argv0="/Users/carlpeto/Code/avr-swift/build/Ninja-ReleaseAssert+swift-DebugAssert/swift-macosx-x86_64/bin/swift-frontend", MainAddr=0x0000000100036aa0, observer=0x0000000000000000) at FrontendTool.cpp:2452:19
    frame #35: 0x0000000100037c84 swift-frontend`run_driver(ExecName=(Data = "swift-frontend", Length = 14), argv=ArrayRef<const char *> @ 0x000000031b32b7f0, originalArgv=const llvm::ArrayRef<const char *> @ 0x000000031b32b7e0) at driver.cpp:239:14
    frame #36: 0x00000001000370ed swift-frontend`swift::mainEntry(argc_=133, argv_=0x000000031b32e4a8) at driver.cpp:505:10
    frame #37: 0x0000000100036732 swift-frontend`main(argc_=133, argv_=0x000000031b32e4a8) at driver.cpp:20:10
    frame #38: 0x000000021722e3a6 dyld`start + 1942

Trying to dump the function from AST that it is attempting to emit...

(lldb) e fd->dump()
(accessor_decl range=[Ranges.swift:391:5 - line:393:5] <anonymous @ 0x7fc6cb883a58> interface type="<Self where Self : MutableCollection> (inout Self) -> (Self.SubSequence, (UnboundedRange_) -> ()) -> ()" access=public captures=(<generic> ) set for="subscript(_:)"
  (parameter "self")
  (parameter_list range=[Ranges.swift:391:5 - line:391:5]
    (parameter "newValue" interface type="Self.SubSequence")
    (parameter "x" interface type="(UnboundedRange_) -> ()"))
  (brace_stmt range=[Ranges.swift:391:9 - line:393:5]))

And our standard :books: source code at the location referred to is...

extension MutableCollection {
  @inlinable
  public subscript<R: RangeExpression>(r: R) -> SubSequence
  where R.Bound == Index {
    get {
      return self[r.relative(to: self)]
    }
    set {
      self[r.relative(to: self)] = newValue
    }
  }

  @inlinable
  public subscript(x: UnboundedRange) -> SubSequence {
    get {
      return self[startIndex...]
    }
    @available(*, unavailable, message: "AVR collections cannot have unbounded set")
    set {
      // self[startIndex...] = newValue
    }
  }
}

Any thoughts or ideas what might be going wrong here?

Thanks for any help you can give!

Regards,
Carl

p.s. FYI... The compiler was built with a build script using host tools (from Xcode 15.0.1 on macOS Sonoma). Here's an excerpt from the build script running CMake...

~/Code/avr-swift/build/Ninja-ReleaseAssert+swift-DebugAssert/swift-macosx-x86_64 ~/Code/avr-swift/swift
+ env /Applications/CMake.app/Contents/bin/cmake -G Ninja -DCMAKE_C_COMPILER:PATH=/Applications/Xcode-15.0.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -DCMAKE_CXX_COMPILER:PATH=/Applications/Xcode-15.0.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -DCMAKE_Swift_COMPILER:PATH=/Applications/Xcode-15.0.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -DCMAKE_LIBTOOL:PATH=/Applications/Xcode-15.0.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool -DCMAKE_AR:PATH=/Applications/Xcode-15.0.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ar -DCMAKE_RANLIB:PATH=/Applications/Xcode-15.0.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib -DLLVM_VERSION_MAJOR:STRING=17 -DLLVM_VERSION_MINOR:STRING=0 -DLLVM_VERSION_PATCH:STRING=0 -DCLANG_VERSION_MAJOR:STRING=17 -DCLANG_VERSION_MINOR:STRING=0 -DCLANG_VERSION_PATCH:STRING=0 -DCMAKE_MAKE_PROGRAM=/opt/homebrew/bin/ninja -DCMAKE_EXPORT_COMPILE_COMMANDS=TRUE -DSWIFT_FORCE_OPTIMIZED_TYPECHECKER:BOOL=FALSE -DSWIFT_STDLIB_ENABLE_STDLIBCORE_EXCLUSIVITY_CHECKING:BOOL=FALSE -DSWIFT_ENABLE_EXPERIMENTAL_DIFFERENTIABLE_PROGRAMMING:BOOL=TRUE -DSWIFT_ENABLE_EXPERIMENTAL_CONCURRENCY:BOOL=TRUE -DSWIFT_ENABLE_EXPERIMENTAL_CXX_INTEROP:BOOL=TRUE -DSWIFT_ENABLE_CXX_INTEROP_SWIFT_BRIDGING_HEADER:BOOL=TRUE -DSWIFT_ENABLE_EXPERIMENTAL_DISTRIBUTED:BOOL=TRUE -DSWIFT_ENABLE_BACKTRACING:BOOL=TRUE -DSWIFT_ENABLE_EXPERIMENTAL_OBSERVATION:BOOL=TRUE -DSWIFT_STDLIB_STATIC_PRINT=FALSE -DSWIFT_FREESTANDING_IS_DARWIN:BOOL=FALSE -DSWIFT_STDLIB_BUILD_PRIVATE:BOOL=TRUE -DSWIFT_STDLIB_ENABLE_UNICODE_DATA=TRUE -DSWIFT_TOOLS_LD64_LTO_CODEGEN_ONLY_FOR_SUPPORTING_TARGETS:BOOL=FALSE -DSWIFT_WASI_SYSROOT_PATH:STRING= -DSWIFT_LIPO:PATH=/Applications/Xcode-15.0.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/lipo '-DCMAKE_C_FLAGS= -Wno-unknown-warning-option -Werror=unguarded-availability-new' '-DCMAKE_CXX_FLAGS= -Wno-unknown-warning-option -Werror=unguarded-availability-new' '-DCMAKE_C_FLAGS_RELWITHDEBINFO=-O2 -DNDEBUG' '-DCMAKE_CXX_FLAGS_RELWITHDEBINFO=-O2 -DNDEBUG' -DCMAKE_BUILD_TYPE:STRING=Debug -DLLVM_ENABLE_ASSERTIONS:BOOL=TRUE -DBOOTSTRAPPING_MODE:STRING=HOSTTOOLS -DSWIFT_ANALYZE_CODE_COVERAGE:STRING=FALSE -DSWIFT_STDLIB_BUILD_TYPE:STRING=Release -DSWIFT_STDLIB_ASSERTIONS:BOOL=TRUE -DSWIFT_STDLIB_ENABLE_DEBUG_PRECONDITIONS_IN_RELEASE=FALSE -DSWIFT_ENABLE_DISPATCH:BOOL=TRUE -DSWIFT_IMPLICIT_CONCURRENCY_IMPORT:BOOL=TRUE -DSWIFT_STDLIB_SUPPORT_BACK_DEPLOYMENT:BOOL=TRUE -DSWIFT_STDLIB_SINGLE_THREADED_CONCURRENCY:BOOL=FALSE -DSWIFT_ENABLE_RUNTIME_FUNCTION_COUNTERS:BOOL=TRUE -DSWIFT_STDLIB_HAS_DLADDR:BOOL=TRUE -DSWIFT_STDLIB_HAS_DLSYM:BOOL=TRUE -DSWIFT_STDLIB_HAS_FILESYSTEM:BOOL=TRUE -DSWIFT_RUNTIME_STATIC_IMAGE_INSPECTION:BOOL=FALSE -DSWIFT_STDLIB_OS_VERSIONING:BOOL=TRUE -DSWIFT_STDLIB_HAS_COMMANDLINE:BOOL=TRUE -DSWIFT_STDLIB_HAS_DARWIN_LIBMALLOC:BOOL=TRUE -DSWIFT_STDLIB_HAS_STDIN:BOOL=TRUE -DSWIFT_STDLIB_HAS_ENVIRON:BOOL=TRUE -DSWIFT_STDLIB_ENABLE_LTO:STRING= -DSWIFT_STDLIB_PASSTHROUGH_METADATA_ALLOCATOR:BOOL=FALSE -DSWIFT_STDLIB_SHORT_MANGLING_LOOKUPS:BOOL=TRUE -DSWIFT_STDLIB_ENABLE_VECTOR_TYPES:BOOL=TRUE -DSWIFT_STDLIB_HAS_TYPE_PRINTING:BOOL=TRUE -DSWIFT_STDLIB_TRAP_FUNCTION:STRING= -DSWIFT_STDLIB_EXPERIMENTAL_HERMETIC_SEAL_AT_LINK:BOOL=FALSE -DSWIFT_STDLIB_EXPERIMENTAL_NONCOPYABLE_GENERICS:BOOL=FALSE -DSWIFT_STDLIB_DISABLE_INSTANTIATION_CACHES:BOOL=FALSE -DSWIFT_STDLIB_REFLECTION_METADATA:STRING=enabled -DSWIFT_NATIVE_LLVM_TOOLS_PATH:STRING= -DSWIFT_NATIVE_CLANG_TOOLS_PATH:STRING= -DSWIFT_NATIVE_SWIFT_TOOLS_PATH:STRING= -DSWIFT_INCLUDE_TOOLS:BOOL=TRUE -DSWIFT_BUILD_REMOTE_MIRROR:BOOL=TRUE -DSWIFT_STDLIB_SIL_DEBUGGING:BOOL=FALSE -DSWIFT_CHECK_INCREMENTAL_COMPILATION:BOOL=FALSE -DSWIFT_ENABLE_ARRAY_COW_CHECKS:BOOL=FALSE -DSWIFT_REPORT_STATISTICS:BOOL=FALSE -DSWIFT_BUILD_DYNAMIC_STDLIB:BOOL=TRUE -DSWIFT_BUILD_STATIC_STDLIB:BOOL=FALSE -DSWIFT_BUILD_DYNAMIC_SDK_OVERLAY:BOOL=TRUE -DSWIFT_BUILD_STATIC_SDK_OVERLAY:BOOL=FALSE -DSWIFT_BUILD_PERF_TESTSUITE:BOOL=FALSE -DSWIFT_BUILD_EXAMPLES:BOOL=TRUE -DSWIFT_BUILD_LIBEXEC:BOOL=TRUE -DSWIFT_INCLUDE_TESTS:BOOL=TRUE -DSWIFT_EMBED_BITCODE_SECTION:BOOL=FALSE -DSWIFT_TOOLS_ENABLE_LTO:STRING= -DSWIFT_BUILD_RUNTIME_WITH_HOST_COMPILER:BOOL=FALSE -DLIBDISPATCH_CMAKE_BUILD_TYPE:STRING=Release -DSWIFT_PATH_TO_SWIFT_SYNTAX_SOURCE:PATH=/Users/carlpeto/Code/avr-swift/swift-syntax -DSWIFT_ENABLE_BACKTRACING:BOOL=TRUE -DSWIFT_STDLIB_OVERRIDABLE_RETAIN_RELEASE:BOOL=TRUE -DPython3_EXECUTABLE=/Applications/Xcode-15.0.1.app/Contents/Developer/usr/bin/python3 -DSWIFTLIB_DEPLOYMENT_VERSION_XCTEST_IOS=8.0 -DSWIFT_DARWIN_DEPLOYMENT_VERSION_OSX=10.15 -DSWIFT_DARWIN_DEPLOYMENT_VERSION_IOS=11.0 -DSWIFT_DARWIN_DEPLOYMENT_VERSION_TVOS=11.0 -DSWIFT_DARWIN_DEPLOYMENT_VERSION_WATCHOS=4.0 -DCMAKE_OSX_SYSROOT:PATH=/Applications/Xcode-15.0.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk -DCMAKE_OSX_ARCHITECTURES=x86_64 -DSWIFT_HOST_TRIPLE:STRING=x86_64-apple-macosx10.15 -DSWIFT_HOST_VARIANT=macosx -DSWIFT_HOST_VARIANT_SDK=OSX -DSWIFT_HOST_VARIANT_ARCH=x86_64 '-DLLVM_LIT_ARGS=-sv -j 8' -DCOVERAGE_DB= -DSWIFT_DARWIN_XCRUN_TOOLCHAIN:STRING=default -DSWIFT_AST_VERIFIER:BOOL=TRUE -DSWIFT_SIL_VERIFY_ALL:BOOL=FALSE -DSWIFT_SIL_VERIFY_ALL_MACOS_ONLY:BOOL=FALSE -DSWIFT_RUNTIME_ENABLE_LEAK_CHECKER:BOOL=FALSE -DCMAKE_INSTALL_PREFIX:PATH=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/ -DClang_DIR:PATH=/Users/carlpeto/Code/avr-swift/build/Ninja-ReleaseAssert+swift-DebugAssert/llvm-macosx-x86_64/lib/cmake/clang -DLLVM_DIR:PATH=/Users/carlpeto/Code/avr-swift/build/Ninja-ReleaseAssert+swift-DebugAssert/llvm-macosx-x86_64/lib/cmake/llvm -DSWIFT_PATH_TO_CMARK_SOURCE:PATH=/Users/carlpeto/Code/avr-swift/cmark -DSWIFT_PATH_TO_CMARK_BUILD:PATH=/Users/carlpeto/Code/avr-swift/build/Ninja-ReleaseAssert+swift-DebugAssert/cmark-macosx-x86_64 -DSWIFT_PATH_TO_LIBDISPATCH_SOURCE:PATH=/Users/carlpeto/Code/avr-swift/swift-corelibs-libdispatch -DSWIFT_PATH_TO_LIBDISPATCH_BUILD:PATH=/Users/carlpeto/Code/avr-swift/build/Ninja-ReleaseAssert+swift-DebugAssert/libdispatch-macosx-x86_64 -DSWIFT_PATH_TO_LIBDISPATCH_STATIC_BUILD:PATH=/Users/carlpeto/Code/avr-swift/build/Ninja-ReleaseAssert+swift-DebugAssert/swift-macosx-x86_64/libdispatch-macosx-x86_64-static-prefix -DSWIFT_SDKS:STRING=OSX -DSWIFT_BUILD_REGEX_PARSER_IN_COMPILER:BOOL=FALSE -DSWIFT_STDLIB_USE_RELATIVE_PROTOCOL_WITNESS_TABLES:BOOL=FALSE '-DLLVM_TARGETS_TO_BUILD=AVR;ARM' -DLLVM_ENABLE_PROJECTS=clang -USWIFT_DARWIN_SUPPORTED_ARCHS -DSWIFT_DISABLE_DEAD_STRIPPING:BOOL=TRUE '-DCMAKE_IGNORE_PATH=/usr/lib;/usr/local/lib;/lib' -DPKG_CONFIG_EXECUTABLE=/usr/bin/false /Users/carlpeto/Code/avr-swift/swift

Which looks OK to me?

11

Just checking: Are you saying it doesn't crash and builds fine with embedded mode turned off?

Can you post the full compilation command you're using? Any unusual flags?

Just as an experiment, what if you remove the unavailability annotation?

12

I’ll check…

So here's the command line (when I'm running it from lldb... just strip off the lldb -- to see the debugger free command line, which crashes the same)...

lldb -- /Users/carlpeto/Code/avr-swift/build/Ninja-ReleaseAssert+swift-DebugAssert/swift-macosx-x86_64/bin/swift-frontend -frontend -emit-module CoreOperators.swift CoreAliases.swift RawRepresentable.swift LiteralProtocols.swift TopLevelFunctions.swift CoreProtocols.swift CoreFloatingPoint.swift CoreBinaryFloatingPoint.swift Float.swift Float16.swift CoreFloatingPointFunctions.swift Optional.swift Bridging.swift CoreNumericProtocols.swift BinaryInteger.swift CoreIntegers.swift ErrorType.swift Bool.swift Integers.swift Ranges.swift Sequence.swift Stride.swift Slice.swift Collection.swift BidirectionalCollection.swift RandomAccessCollection.swift ClosedRange.swift MutableCollection.swift Hash.swift Pointer.swift UnsafeBufferPointer.swift UnsafeRawBufferPointer.swift UnsafeRawPointer.swift Indices.swift Existential.swift Algorithm.swift FixedWidth.swift IntegerMath.swift CTypes.swift UnsafePointer.swift ObjectIdentifier.swift CollectionAlgorithms.swift WriteBackMutableSlice.swift Random.swift RangeReplaceableCollection.swift MemoryLayout.swift Tuple.swift SequenceAlgorithms.swift LifetimeManager.swift Repeat.swift EmptyCollection.swift CollectionOfOne.swift StringLiterals.swift StaticString.swift StringInterpolation.swift Unicode.swift UnicodeScalar.swift UnicodeEncoding.swift UTF8.swift UTF16.swift ValidUTF8Buffer.swift UnicodeParser.swift UIntBuffer.swift UTFEncoding.swift UTF32.swift ArrayType.swift ArrayBufferProtocol.swift ArrayLiterals.swift ArrayShared.swift ContiguousArray.swift SliceBuffer.swift ArraySlice.swift Array.swift ArrayBody.swift ArrayCast.swift AnyHashable.swift ManagedBuffer.swift AVRArrayBuffer.swift Reverse.swift Map.swift Zip.swift LazySequence.swift LazyCollection.swift Filter.swift FlatMap.swift Flatten.swift DropWhile.swift Volatile.swift uSwift.swift Integer-16.swift IntegerMath-16.swift CTypes-16.swift Progmem.swift version.swift -supplementary-output-file-map /var/folders/dj/1dfks4bd5l598f4myn5gz2_w0000gn/T/supplementaryOutputs-091603 -disable-objc-attr-requires-foundation-module -target avr-atmel-linux-gnueabihf -disable-objc-interop -color-diagnostics -I uSwiftShims -I AVR/bin -I "/Applications/Swift For Arduino.app/Contents/XPCServices/BuildEngine.xpc/Contents/Resources/gpl-tools-avr/lib/avr-libgcc/include" -I "/Applications/Swift For Arduino.app/Contents/XPCServices/BuildEngine.xpc/Contents/Resources/gpl-tools-avr/lib/avr-libc/include" -enable-experimental-feature Embedded -nostdimport -parse-stdlib -Osize -D AVR_LIBC_DEFINED_SWIFT -disable-reflection-metadata -plugin-path /Users/carlpeto/Code/avr-swift/build/Ninja-ReleaseAssert+swift-DebugAssert/swift-macosx-x86_64/lib/swift/host/plugins -plugin-path /Users/carlpeto/Code/avr-swift/build/Ninja-ReleaseAssert+swift-DebugAssert/swift-macosx-x86_64/local/lib/swift/host/plugins -Xcc -DAVR_LIBC_DEFINED -Xcc -DLIBC_DEFINED -parse-as-library -module-name Swift -o bin/AVR/Swift.swiftmodule

... if I simply remove the embedded flag then it compiles just fine...

lldb -- /Users/carlpeto/Code/avr-swift/build/Ninja-ReleaseAssert+swift-DebugAssert/swift-macosx-x86_64/bin/swift-frontend -frontend -emit-module CoreOperators.swift CoreAliases.swift RawRepresentable.swift LiteralProtocols.swift TopLevelFunctions.swift CoreProtocols.swift CoreFloatingPoint.swift CoreBinaryFloatingPoint.swift Float.swift Float16.swift CoreFloatingPointFunctions.swift Optional.swift Bridging.swift CoreNumericProtocols.swift BinaryInteger.swift CoreIntegers.swift ErrorType.swift Bool.swift Integers.swift Ranges.swift Sequence.swift Stride.swift Slice.swift Collection.swift BidirectionalCollection.swift RandomAccessCollection.swift ClosedRange.swift MutableCollection.swift Hash.swift Pointer.swift UnsafeBufferPointer.swift UnsafeRawBufferPointer.swift UnsafeRawPointer.swift Indices.swift Existential.swift Algorithm.swift FixedWidth.swift IntegerMath.swift CTypes.swift UnsafePointer.swift ObjectIdentifier.swift CollectionAlgorithms.swift WriteBackMutableSlice.swift Random.swift RangeReplaceableCollection.swift MemoryLayout.swift Tuple.swift SequenceAlgorithms.swift LifetimeManager.swift Repeat.swift EmptyCollection.swift CollectionOfOne.swift StringLiterals.swift StaticString.swift StringInterpolation.swift Unicode.swift UnicodeScalar.swift UnicodeEncoding.swift UTF8.swift UTF16.swift ValidUTF8Buffer.swift UnicodeParser.swift UIntBuffer.swift UTFEncoding.swift UTF32.swift ArrayType.swift ArrayBufferProtocol.swift ArrayLiterals.swift ArrayShared.swift ContiguousArray.swift SliceBuffer.swift ArraySlice.swift Array.swift ArrayBody.swift ArrayCast.swift AnyHashable.swift ManagedBuffer.swift AVRArrayBuffer.swift Reverse.swift Map.swift Zip.swift LazySequence.swift LazyCollection.swift Filter.swift FlatMap.swift Flatten.swift DropWhile.swift Volatile.swift uSwift.swift Integer-16.swift IntegerMath-16.swift CTypes-16.swift Progmem.swift version.swift -supplementary-output-file-map /var/folders/dj/1dfks4bd5l598f4myn5gz2_w0000gn/T/supplementaryOutputs-091603 -disable-objc-attr-requires-foundation-module -target avr-atmel-linux-gnueabihf -disable-objc-interop -color-diagnostics -I uSwiftShims -I AVR/bin -I "/Applications/Swift For Arduino.app/Contents/XPCServices/BuildEngine.xpc/Contents/Resources/gpl-tools-avr/lib/avr-libgcc/include" -I "/Applications/Swift For Arduino.app/Contents/XPCServices/BuildEngine.xpc/Contents/Resources/gpl-tools-avr/lib/avr-libc/include" -nostdimport -parse-stdlib -Osize -D AVR_LIBC_DEFINED_SWIFT -disable-reflection-metadata -plugin-path /Users/carlpeto/Code/avr-swift/build/Ninja-ReleaseAssert+swift-DebugAssert/swift-macosx-x86_64/lib/swift/host/plugins -plugin-path /Users/carlpeto/Code/avr-swift/build/Ninja-ReleaseAssert+swift-DebugAssert/swift-macosx-x86_64/local/lib/swift/host/plugins -Xcc -DAVR_LIBC_DEFINED -Xcc -DLIBC_DEFINED -parse-as-library -module-name Swift -o bin/AVR/Swift.swiftmodule

p.s. I tried changing -Osize to -O in case that made a difference but it didn't make any difference. Still crashes in the same place if and only if the embedded mode is enabled.

13

@kubamracek I ran the check you suggested ^^

14

And what if you remove the @available(*, unavailable, ...) line? Not saying that's a solution, just something that could help narrow down the problem.

15

Actually, the source code snippet you posted reproduces the compiler crash in isolation (just building that snippet without anything else). Would you mind filing a github issue for it and assigning to me? (And yes, it seems to be triggered by the availability annotation somehow.)

16

Ok done!

The issue probably isn’t as good as it should be because the code snippet I added won’t compile. My laptop got another hardware fault so I’m writing this with my phone!

When I get a chance I’ll try removing the unavailable attribute and seeing if that fixes it.

17

Fix here: [embedded] Consider 'skipped' decls when SILGen-ing accessors, fix compiler crash on unavailable accessors by kubamracek · Pull Request #70042 · apple/swift · GitHub

18

Cool! Thank you so much! Reading it, understanding it and testing it my end...

—-

The quick update is that patch seems to work. I have errors in my stdlib now but they are genuine issues that need addressing to make my source code compatible with the embedded mode. So progress!

Thanks!