Unable to build with bootstrapping enabled

carlos42421 · November 16, 2023, 11:33am

I'm having trouble building my fork of swift when I turn bootstrapping back on.

The public fork is here: GitHub - carlos4242/swift: The Swift Programming Language and my changes are on the branch avr-mods-1.

It works if I check out a fresh copy of the above into a clean folder on my Mac Mini M1 (macOS 11.7.10, Xcode 13.2.1), then run update checkout like so...

utils/update-checkout --clone-with-ssh --skip-repository swift-nio \
--skip-repository swift-nio-ssh --skip-repository swift-lmdb --skip-repository swift-docc \
--skip-repository swift-docc-render-artifact --skip-repository swift-docc-symbolkit \
--skip-repository swift-markdown --skip-repository swift-experimental-string-processing \
--skip-repository swift-llvm-bindings --skip-repository swift-xcode-playground-support \
--skip-repository swift-corelibs-libdispatch --skip-repository swift-corelibs-foundation \
--skip-repository swift-corelibs-xctest --skip-repository swift-stress-tester \
--skip-repository swift-crypto --skip-repository swift-atomics \
--skip-repository swift-nio-ssl --skip-repository sourcekit-lsp \
--skip-repository indexstore-db --skip-repository swiftpm \
--skip-repository swift-numerics --skip-repository swift

and then run the build script to set up makefiles like so...

utils/build-script -R -S --clean ${DEBUG_SWIFT_OPT} --extra-cmake-options="-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=AVR;ARM"  --extra-cmake-options="-DLLVM_ENABLE_PROJECTS='clang'" \
--skip-build-benchmarks --skip-ios --skip-watchos --skip-tvos --darwin-deployment-version-osx 10.15 \
--skip-early-swift-driver=true --skip-early-swiftsyntax=true \
--swift-driver=false --swift-disable-dead-stripping --bootstrapping=off

Then finally run ninja in each sub folder of cmark, llvm and ninja swift-frontend in the swift folder.

This builds 100% fine normally and I can make my working compiler tools... this has worked for us for years.

If I remove --bootstrapping=off on the build script ninja compiles cmark and llvm fine but fails almost immediately when I run ninja swift-frontend to build the swift frontend...

ninja swift-frontend
[1/995] Building CXX object stdlib/toolchain/Compatibility56/CMakeFiles/swiftCompatibility56-macosx-x86_64.dir/Concurrency/Error.cpp.o
FAILED: stdlib/toolchain/Compatibility56/CMakeFiles/swiftCompatibility56-macosx-x86_64.dir/Concurrency/Error.cpp.o
/Users/carlpeto/Code/avr-swift/build/Ninja-ReleaseAssert/llvm-macosx-x86_64/./bin/clang++ -DCMARK_STATIC_DEFINE -DGTEST_HAS_RTTI=0 -DLLVM_DISABLE_ABI_BREAKING_CHECKS_ENFORCING=1 -DSWIFT_COMPATIBILITY56 -DSWIFT_INLINE_NAMESPACE=__runtime -D_DEBUG -D_GLIBCXX_ASSERTIONS -D_LIBCPP_ENABLE_HARDENED_MODE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Istdlib/toolchain/Compatibility56 -I/Users/carlpeto/Code/avr-swift/swift/stdlib/toolchain/Compatibility56 -I/Users/carlpeto/Code/avr-swift/swift/stdlib/include -I/Users/carlpeto/Code/avr-swift/swift/stdlib/public/SwiftShims -Iinclude -I/Users/carlpeto/Code/avr-swift/swift/include -I/Users/carlpeto/Code/avr-swift/build/Ninja-ReleaseAssert/llvm-macosx-x86_64/include -I/Users/carlpeto/Code/avr-swift/llvm-project/clang/include -I/Users/carlpeto/Code/avr-swift/build/Ninja-ReleaseAssert/llvm-macosx-x86_64/tools/clang/include -I/Users/carlpeto/Code/avr-swift/cmark/src/include -I/Users/carlpeto/Code/avr-swift/build/Ninja-ReleaseAssert/cmark-macosx-x86_64/src -I/Users/carlpeto/Code/avr-swift/swift/stdlib/toolchain/Compatibility56/include -I/Users/carlpeto/Code/avr-swift/swift/stdlib -Wno-unknown-warning-option -Werror=unguarded-availability-new -fno-stack-protector -fPIC -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wimplicit-fallthrough -Wcovered-switch-default -Wno-class-memaccess -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -Werror=switch -Wdocumentation -Wimplicit-fallthrough -Wunreachable-code -Woverloaded-virtual -DOBJC_OLD_DISPATCH_PROTOTYPES=0 -DPURE_BRIDGING_MODE -O3 -DNDEBUG -arch x86_64 -isysroot /Applications/Xcode-13.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk -fvisibility=hidden -fvisibility-inlines-hidden  -fno-exceptions -funwind-tables -fno-rtti -Werror=gnu -Werror=c++98-compat-extra-semi -UNDEBUG -fno-sanitize=all -Wall -Wglobal-constructors -Wexit-time-destructors -DSWIFT_COMPATIBILITY_LIBRARY=1 -D__STDC_WANT_LIB_EXT1__=1 -DSWIFT_THREADING_IS_COMPATIBILITY_LIBRARY -DSWIFT_TARGET_LIBRARY_NAME=swiftCompatibility56 -DSWIFT_RUNTIME -DSWIFT_LIB_SUBDIR=\"macosx\" -DSWIFT_ARCH=\"x86_64\" -target x86_64-apple-macosx10.9 -isysroot /Applications/Xcode-13.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk -F/Applications/Xcode-13.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/../../../Developer/Library/Frameworks -O2 -g0 -mcx16 -DSWIFT_ENABLE_RUNTIME_FUNCTION_COUNTERS -DSWIFT_LIBRARY_EVOLUTION=1 -DSWIFT_STDLIB_SUPPORT_BACK_DEPLOYMENT -DSWIFT_ENABLE_REFLECTION -DSWIFT_STDLIB_HAS_DLADDR -DSWIFT_STDLIB_HAS_DLSYM=1 -DSWIFT_STDLIB_HAS_FILESYSTEM -DSWIFT_STDLIB_HAS_DARWIN_LIBMALLOC=1 -DSWIFT_STDLIB_HAS_ASL -DSWIFT_STDLIB_HAS_STDIN -DSWIFT_STDLIB_HAS_ENVIRON -DSWIFT_STDLIB_HAS_LOCALE -DSWIFT_THREADING_DARWIN -DSWIFT_RUNTIME_OS_VERSIONING -DSWIFT_STDLIB_SHORT_MANGLING_LOOKUPS -DSWIFT_STDLIB_ENABLE_VECTOR_TYPES -DSWIFT_STDLIB_HAS_TYPE_PRINTING -DSWIFT_STDLIB_SUPPORTS_BACKTRACE_REPORTING -DSWIFT_STDLIB_ENABLE_UNICODE_DATA -DSWIFT_STDLIB_TRACING -DSWIFT_STDLIB_CONCURRENCY_TRACING -DSWIFT_STDLIB_OVERRIDABLE_RETAIN_RELEASE -std=c++17 -MD -MT stdlib/toolchain/Compatibility56/CMakeFiles/swiftCompatibility56-macosx-x86_64.dir/Concurrency/Error.cpp.o -MF stdlib/toolchain/Compatibility56/CMakeFiles/swiftCompatibility56-macosx-x86_64.dir/Concurrency/Error.cpp.o.d -o stdlib/toolchain/Compatibility56/CMakeFiles/swiftCompatibility56-macosx-x86_64.dir/Concurrency/Error.cpp.o -c /Users/carlpeto/Code/avr-swift/swift/stdlib/toolchain/Compatibility56/Concurrency/Error.cpp
error: unable to create target: 'No available targets are compatible with triple "x86_64-apple-macosx10.9.0"'
1 error generated.

Can anyone shed light on what is happening here? Is this because it's trying to build the standard library? I don't want it to build stdlib.

Thanks for any advice or help.

Carl

Andrew_Trick · November 16, 2023, 5:55pm

Thank you for using --skip-build and reporting issues. This is one of the most important build-script options because it's the only way to decompose/recompose the build process
(I need a -O2 LLVM with a -O0 Swift and don't know how else to get that). But it's untested and routinely breaks for me. Part of the problem is that build-script options have been added to control the build rather than configure the build.

--skip-early-swiftsyntax=true was broken recently but may be fixed by now. At any rate, the option may not be relevant anymore.

You may need ninja bin/swift-frontend. But parts of the compiler will be missing without bootstrapping. I try to stick with --bootstrapping=hosttools, which works with Xcode 15.0 (but not 15.1 last I tried).

I always see a handful of lit test failures, but at least I can build the compiler.

carlos42421 · November 16, 2023, 6:32pm

Ok sounds worth a try.

I’m struggling a bit to find documentation but that might just be blindness on my part.

My sort of guesswork/assumptions are:

“bootstrapping” generally means “building part of the swift compiler executables in swift itself”
when it is turned off, the whole swift-frontend executable is basically built from C++ code, like it used to be, linked to LLVM and cmark libraries
currently both versions are being maintained for now, although in the long run the “bootstrapped” version will probably take over and the pure C++ version will become deprecated. Over time, some features will only be implemented only in the bootstrapped version.
the C++ code in either version will be compiled by the system installed C++ compiler, not any version of clang that was just built in the LLVM compile step
by contrast, currently it looks like bootstrapping usually tries to build at least a basic version of swift first, (maybe with pure C++), then uses that version to “bootstrap” the hybrid compiler (the one made half of swift and half of C++)
if you specify —bootstrap=hosttools then building swift-frontend is done using installed swift on your machine. In my case that would be swift inside Xcode 13, for a Linux user it would probably be swift from an installed toolchain

I’ll try changing the build as suggested. Thanks!

UPDATE2: I had to upgrade to Sonoma and Xcode 15 to get a modern enough swift compiler, but —bootstrap=hosttools worked! I need to do some more checks and will get back if any more issues. But so far, it looks good, thanks!