Can’t build libswiftDispatch.so

taylorswift · December 16, 2023, 11:35pm

my swift toolchain build keeps failing while

Linking Swift shared library libswiftDispatch.so

.

the build output doesn’t indicate why the build is failing, although some of the “Looking for” failures seem suspicious.

Cleaning the libdispatch build directory
+ rm -rf /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64
+ mkdir -p /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64
+ mkdir -p /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/.cmake/api/v1/query
+ touch /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/.cmake/api/v1/query/codemodel-v2 /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/.cmake/api/v1/query/cache-v2
+ pushd /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64
/swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64 /swift/swift-project/swift
+ env /swift/swift-project/build/cmake-linux-x86_64/bin/cmake -DENABLE_SWIFT=YES -G Ninja -DCMAKE_C_COMPILER:PATH=/opt/swift/5.9.2/usr/bin/clang -DCMAKE_CXX_COMPILER:PATH=/opt/swift/5.9.2/usr/bin/clang++ -DCMAKE_Swift_COMPILER:PATH=/opt/swift/5.9.2/usr/bin/swiftc -DCMAKE_LIBTOOL:PATH=/usr/bin/libtool -DCMAKE_AR:PATH=/usr/bin/ar -DCMAKE_RANLIB:PATH=/usr/bin/ranlib -DLLVM_VERSION_MAJOR:STRING=17 -DLLVM_VERSION_MINOR:STRING=0 -DLLVM_VERSION_PATCH:STRING=0 -DCLANG_VERSION_MAJOR:STRING=17 -DCLANG_VERSION_MINOR:STRING=0 -DCLANG_VERSION_PATCH:STRING=0 -DCMAKE_MAKE_PROGRAM=/swift/swift-project/build/buildbot_linux/ninja-build/ninja -DCMAKE_BUILD_TYPE:STRING=Release -DCMAKE_C_COMPILER:PATH=/swift/swift-project/build/buildbot_linux/llvm-linux-x86_64/bin/clang -DCMAKE_CXX_COMPILER:PATH=/swift/swift-project/build/buildbot_linux/llvm-linux-x86_64/bin/clang++ -DCMAKE_SWIFT_COMPILER:PATH=/swift/swift-project/build/buildbot_linux/swift-linux-x86_64/bin/swiftc -DCMAKE_Swift_COMPILER:PATH=/swift/swift-project/build/buildbot_linux/swift-linux-x86_64/bin/swiftc '-DCMAKE_Swift_FLAGS:STRING=  -module-cache-path "/swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/module-cache" ' -DCMAKE_INSTALL_PREFIX:PATH=/usr/ -DCMAKE_INSTALL_LIBDIR:PATH=lib -DSwift_DIR=/swift/swift-project/build/buildbot_linux/swift-linux-x86_64/lib/cmake/swift -DENABLE_TESTING=YES -DBUILD_SHARED_LIBS=YES -USWIFT_DARWIN_SUPPORTED_ARCHS -DSWIFT_PATH_TO_SWIFT_SYNTAX_SOURCE:PATH=/swift/swift-project/swift-syntax -DSWIFT_BUILD_SWIFT_SYNTAX:BOOL=TRUE -DSWIFTSYNTAX_ENABLE_ASSERTIONS:BOOL=TRUE /swift/swift-project/swift-corelibs-libdispatch
-- The C compiler identification is Clang 17.0.0
-- The CXX compiler identification is Clang 17.0.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /swift/swift-project/build/buildbot_linux/llvm-linux-x86_64/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /swift/swift-project/build/buildbot_linux/llvm-linux-x86_64/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- The Swift compiler identification is Apple 5.11
-- Check for working Swift compiler: /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/bin/swiftc
-- Check for working Swift compiler: /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/bin/swiftc - works
-- Looking for __GNU_LIBRARY__
-- Looking for __GNU_LIBRARY__ - found
-- Performing Test __BUILTIN_TRAP
-- Performing Test __BUILTIN_TRAP - Success
-- Found LibRT: /usr/lib64/librt.a  
-- Looking for _pthread_workqueue_init
-- Looking for _pthread_workqueue_init - not found
-- Looking for getprogname
-- Looking for getprogname - not found
-- Looking for mach_absolute_time
-- Looking for mach_absolute_time - not found
-- Looking for mach_approximate_time
-- Looking for mach_approximate_time - not found
-- Looking for mach_port_construct
-- Looking for mach_port_construct - not found
-- Looking for malloc_create_zone
-- Looking for malloc_create_zone - not found
-- Looking for posix_fadvise
-- Looking for posix_fadvise - found
-- Looking for posix_spawnp
-- Looking for posix_spawnp - found
-- Looking for pthread_key_init_np
-- Looking for pthread_key_init_np - not found
-- Looking for pthread_attr_setcpupercent_np
-- Looking for pthread_attr_setcpupercent_np - not found
-- Looking for pthread_yield_np
-- Looking for pthread_yield_np - not found
-- Looking for pthread_main_np
-- Looking for pthread_main_np - not found
-- Looking for pthread_workqueue_setdispatch_np
-- Looking for pthread_workqueue_setdispatch_np - not found
-- Looking for strlcpy
-- Looking for strlcpy - not found
-- Looking for sysconf
-- Looking for sysconf - found
-- Looking for arc4random
-- Looking for arc4random - not found
-- Looking for include file TargetConditionals.h
-- Looking for include file TargetConditionals.h - not found
-- Looking for include file dlfcn.h
-- Looking for include file dlfcn.h - found
-- Looking for include file fcntl.h
-- Looking for include file fcntl.h - found
-- Looking for include file inttypes.h
-- Looking for include file inttypes.h - found
-- Looking for include file libkern/OSAtomic.h
-- Looking for include file libkern/OSAtomic.h - not found
-- Looking for include file libkern/OSCrossEndian.h
-- Looking for include file libkern/OSCrossEndian.h - not found
-- Looking for include file libproc_internal.h
-- Looking for include file libproc_internal.h - not found
-- Looking for include file mach/mach.h
-- Looking for include file mach/mach.h - not found
-- Looking for include file malloc/malloc.h
-- Looking for include file malloc/malloc.h - not found
-- Looking for include file memory.h
-- Looking for include file memory.h - found
-- Looking for include file pthread/qos.h
-- Looking for include file pthread/qos.h - not found
-- Looking for include file pthread/workqueue_private.h
-- Looking for include file pthread/workqueue_private.h - not found
-- Looking for include file pthread_machdep.h
-- Looking for include file pthread_machdep.h - not found
-- Looking for include file pthread_np.h
-- Looking for include file pthread_np.h - not found
-- Looking for include file pthread_workqueue.h
-- Looking for include file pthread_workqueue.h - not found
-- Looking for include file stdint.h
-- Looking for include file stdint.h - found
-- Looking for include file stdlib.h
-- Looking for include file stdlib.h - found
-- Looking for include file string.h
-- Looking for include file string.h - found
-- Looking for include file strings.h
-- Looking for include file strings.h - found
-- Looking for include file sys/guarded.h
-- Looking for include file sys/guarded.h - not found
-- Looking for include file sys/stat.h
-- Looking for include file sys/stat.h - found
-- Looking for include file sys/types.h
-- Looking for include file sys/types.h - found
-- Looking for include file objc/objc-internal.h
-- Looking for include file objc/objc-internal.h - not found
-- Looking for sem_init in pthread
-- Looking for sem_init in pthread - found
-- Looking for CLOCK_UPTIME
-- Looking for CLOCK_UPTIME - not found
-- Looking for CLOCK_UPTIME_FAST
-- Looking for CLOCK_UPTIME_FAST - not found
-- Looking for CLOCK_MONOTONIC
-- Looking for CLOCK_MONOTONIC - found
-- Looking for CLOCK_REALTIME
-- Looking for CLOCK_REALTIME - found
-- Looking for CLOCK_MONOTONIC_COARSE
-- Looking for CLOCK_MONOTONIC_COARSE - found
-- Looking for FD_COPY
-- Looking for FD_COPY - not found
-- Looking for NOTE_LOWAT
-- Looking for NOTE_LOWAT - not found
-- Looking for NOTE_NONE
-- Looking for NOTE_NONE - not found
-- Looking for NOTE_REAP
-- Looking for NOTE_REAP - not found
-- Looking for NOTE_REVOKE
-- Looking for NOTE_REVOKE - not found
-- Looking for NOTE_SIGNAL
-- Looking for NOTE_SIGNAL - not found
-- Looking for POSIX_SPAWN_START_SUSPENDED
-- Looking for POSIX_SPAWN_START_SUSPENDED - not found
-- Looking for SIGEMT
-- Looking for SIGEMT - not found
-- Looking for VQ_DESIRED_DISK
-- Looking for VQ_DESIRED_DISK - not found
-- Looking for VQ_NEARLOWDISK
-- Looking for VQ_NEARLOWDISK - not found
-- Looking for VQ_QUOTA
-- Looking for VQ_QUOTA - not found
-- Looking for VQ_UPDATE
-- Looking for VQ_UPDATE - not found
-- Looking for VQ_VERYLOWDISK
-- Looking for VQ_VERYLOWDISK - not found
-- Looking for VQ_FREE_SPACE_CHANGE
-- Looking for VQ_FREE_SPACE_CHANGE - not found
-- Looking for program_invocation_name
-- Looking for program_invocation_name - found
-- Looking for __printflike
-- Looking for __printflike - found
-- Performing Test C_SUPPORTS_OMIT_LEAF_FRAME_POINTER
-- Performing Test C_SUPPORTS_OMIT_LEAF_FRAME_POINTER - Success
-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    CLANG_VERSION_MAJOR
    CLANG_VERSION_MINOR
    CLANG_VERSION_PATCH
    CMAKE_LIBTOOL
    CMAKE_SWIFT_COMPILER
    ENABLE_TESTING
    LLVM_VERSION_MAJOR
    LLVM_VERSION_MINOR
    LLVM_VERSION_PATCH
    SWIFTSYNTAX_ENABLE_ASSERTIONS
    SWIFT_BUILD_SWIFT_SYNTAX
    SWIFT_PATH_TO_SWIFT_SYNTAX_SOURCE
    Swift_DIR


-- Build files have been written to: /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64
+ popd
/swift/swift-project/swift
+ /swift/swift-project/build/cmake-linux-x86_64/bin/cmake --build /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64 -- -j12 all
[81/81][100%][5.014s] Linking Swift shared library libswiftDispatch.so
FAILED: libswiftDispatch.so src/swift/CMakeFiles/swiftDispatch.dir/Block.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Data.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Dispatch.swift.o src/swift/CMakeFiles/swiftDispatch.dir/IO.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Private.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Queue.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Source.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Time.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Wrapper.swift.o src/swift/swift/Dispatch.swiftmodule 
: && /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/bin/swiftc -output-file-map src/swift/CMakeFiles/swiftDispatch.dir/Release/output-file-map.json -incremental -j 12 -emit-library -o libswiftDispatch.so -module-name Dispatch -module-link-name swiftDispatch -emit-module -emit-module-path src/swift/swift/Dispatch.swiftmodule -emit-dependencies -DswiftDispatch_EXPORTS -module-cache-path "/swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/module-cache" -O -Xcc -fblocks -Xcc -fmodule-map-file=/swift/swift-project/swift-corelibs-libdispatch/dispatch/module.modulemap -Xcc -I/swift/swift-project/swift-corelibs-libdispatch -Xcc -I/swift/swift-project/swift-corelibs-libdispatch/src/swift/shims -vfsoverlay /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/dispatch-vfs-overlay.yaml -I /swift/swift-project/swift-corelibs-libdispatch/src/BlocksRuntime -I /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64 -I /swift/swift-project/swift-corelibs-libdispatch -I /swift/swift-project/swift-corelibs-libdispatch/src -I /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/src /swift/swift-project/swift-corelibs-libdispatch/src/swift/Block.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Data.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Dispatch.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/IO.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Private.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Queue.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Source.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Time.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Wrapper.swift -no-toolchain-stdlib-rpath -Xlinker -soname -Xlinker libswiftDispatch.so  -L /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/src/swift  -L /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64  -L /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64   -L /usr/lib/gcc/x86_64-amazon-linux/11 -Xlinker -rpath -Xlinker /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64:  src/swift/libDispatchStubs.a  libdispatch.so  libBlocksRuntime.so  -lstdc++  -lm  -lgcc_s  -lgcc  -lc  -lgcc_s  -lgcc && :
/swift/swift-project/swift-corelibs-libdispatch/src/swift/Dispatch.swift:99:4: warning: default will never be executed
                        default: return nil
                        ^
error: fatalError
ninja: build stopped: subcommand failed.
ERROR: command terminated with a non-zero exit status 1, aborting

ERROR: command terminated with a non-zero exit status 1, aborting

any idea what’s going on here?

Finagolfin · December 17, 2023, 3:47am

You can compare that output to the log from a recent passing linux CI run.

CMake has a bad habit of sometimes swallowing build errors, but it does output the failing command. Try going into the libdispatch build directory with cd /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/ and running that last command manually, perhaps with -v appended, ie the swiftc line after FAILED in your paste above.

That should give you the actual error.

taylorswift · December 17, 2023, 4:57am

running it in -v mode only yields an error: fatalError.

$ /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/bin/swiftc -v -output-file-map src/swift/CMakeFiles/swiftDispatch.dir/Release/output-file-map.json -incremental -j 12 -emit-library -o libswiftDispatch.so -module-name Dispatch -module-link-name swiftDispatch -emit-module -emit-module-path src/swift/swift/Dispatch.swiftmodule -emit-dependencies -DswiftDispatch_EXPORTS -module-cache-path "/swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/module-cache" -O -Xcc -fblocks -Xcc -fmodule-map-file=/swift/swift-project/swift-corelibs-libdispatch/dispatch/module.modulemap -Xcc -I/swift/swift-project/swift-corelibs-libdispatch -Xcc -I/swift/swift-project/swift-corelibs-libdispatch/src/swift/shims -vfsoverlay /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/dispatch-vfs-overlay.yaml -I /swift/swift-project/swift-corelibs-libdispatch/src/BlocksRuntime -I /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64 -I /swift/swift-project/swift-corelibs-libdispatch -I /swift/swift-project/swift-corelibs-libdispatch/src -I /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/src /swift/swift-project/swift-corelibs-libdispatch/src/swift/Block.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Data.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Dispatch.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/IO.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Private.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Queue.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Source.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Time.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Wrapper.swift -no-toolchain-stdlib-rpath -Xlinker -soname -Xlinker libswiftDispatch.so  -L /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/src/swift  -L /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64  -L /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64   -L /usr/lib/gcc/x86_64-amazon-linux/11 -Xlinker -rpath -Xlinker /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64:  src/swift/libDispatchStubs.a  libdispatch.so  libBlocksRuntime.so  -lstdc++  -lm  -lgcc_s  -lgcc  -lc  -lgcc_s  -lgcc

Swift version 5.11-dev (LLVM 2a31eae409ff4ff, Swift f68e0c90f9784ab)
Target: x86_64-unknown-linux-gnu
/swift/swift-project/build/buildbot_linux/swift-linux-x86_64/bin/swift-frontend -frontend -c -primary-file /swift/swift-project/swift-corelibs-libdispatch/src/swift/Block.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Data.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Dispatch.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/IO.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Private.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Queue.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Source.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Time.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Wrapper.swift -emit-module-path src/swift/CMakeFiles/swiftDispatch.dir/Block.swift.swiftmodule -emit-module-doc-path src/swift/CMakeFiles/swiftDispatch.dir/Block.swift.swiftdoc -emit-module-source-info-path src/swift/CMakeFiles/swiftDispatch.dir/Block.swift.swiftsourceinfo -emit-dependencies-path src/swift/CMakeFiles/swiftDispatch.dir/Block.swift.o.d -emit-reference-dependencies-path src/swift/CMakeFiles/swiftDispatch.dir/Block.swift.o.swiftdeps -target x86_64-unknown-linux-gnu -disable-objc-interop -I /swift/swift-project/swift-corelibs-libdispatch/src/BlocksRuntime -I /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64 -I /swift/swift-project/swift-corelibs-libdispatch -I /swift/swift-project/swift-corelibs-libdispatch/src -I /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/src -vfsoverlay /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/dispatch-vfs-overlay.yaml -color-diagnostics -module-cache-path /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/module-cache -module-link-name swiftDispatch -O -D swiftDispatch_EXPORTS -new-driver-path /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/bin/swift-driver -empty-abi-descriptor -resource-dir /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/lib/swift -Xcc -fblocks -Xcc -fmodule-map-file=/swift/swift-project/swift-corelibs-libdispatch/dispatch/module.modulemap -Xcc -I/swift/swift-project/swift-corelibs-libdispatch -Xcc -I/swift/swift-project/swift-corelibs-libdispatch/src/swift/shims -module-name Dispatch -plugin-path /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/lib/swift/host/plugins -plugin-path /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/local/lib/swift/host/plugins -enable-default-cmo -parse-as-library -o src/swift/CMakeFiles/swiftDispatch.dir/Block.swift.o
/swift/swift-project/build/buildbot_linux/swift-linux-x86_64/bin/swift-frontend -frontend -c /swift/swift-project/swift-corelibs-libdispatch/src/swift/Block.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Data.swift -primary-file /swift/swift-project/swift-corelibs-libdispatch/src/swift/Dispatch.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/IO.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Private.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Queue.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Source.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Time.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Wrapper.swift -emit-module-path src/swift/CMakeFiles/swiftDispatch.dir/Dispatch.swift.swiftmodule -emit-module-doc-path src/swift/CMakeFiles/swiftDispatch.dir/Dispatch.swift.swiftdoc -emit-module-source-info-path src/swift/CMakeFiles/swiftDispatch.dir/Dispatch.swift.swiftsourceinfo -emit-dependencies-path src/swift/CMakeFiles/swiftDispatch.dir/Dispatch.swift.o.d -emit-reference-dependencies-path src/swift/CMakeFiles/swiftDispatch.dir/Dispatch.swift.o.swiftdeps -target x86_64-unknown-linux-gnu -disable-objc-interop -I /swift/swift-project/swift-corelibs-libdispatch/src/BlocksRuntime -I /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64 -I /swift/swift-project/swift-corelibs-libdispatch -I /swift/swift-project/swift-corelibs-libdispatch/src -I /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/src -vfsoverlay /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/dispatch-vfs-overlay.yaml -color-diagnostics -module-cache-path /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/module-cache -module-link-name swiftDispatch -O -D swiftDispatch_EXPORTS -new-driver-path /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/bin/swift-driver -empty-abi-descriptor -resource-dir /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/lib/swift -Xcc -fblocks -Xcc -fmodule-map-file=/swift/swift-project/swift-corelibs-libdispatch/dispatch/module.modulemap -Xcc -I/swift/swift-project/swift-corelibs-libdispatch -Xcc -I/swift/swift-project/swift-corelibs-libdispatch/src/swift/shims -module-name Dispatch -plugin-path /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/lib/swift/host/plugins -plugin-path /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/local/lib/swift/host/plugins -enable-default-cmo -parse-as-library -o src/swift/CMakeFiles/swiftDispatch.dir/Dispatch.swift.o
/swift/swift-project/build/buildbot_linux/swift-linux-x86_64/bin/swift-frontend -frontend -c /swift/swift-project/swift-corelibs-libdispatch/src/swift/Block.swift -primary-file /swift/swift-project/swift-corelibs-libdispatch/src/swift/Data.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Dispatch.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/IO.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Private.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Queue.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Source.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Time.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Wrapper.swift -emit-module-path src/swift/CMakeFiles/swiftDispatch.dir/Data.swift.swiftmodule -emit-module-doc-path src/swift/CMakeFiles/swiftDispatch.dir/Data.swift.swiftdoc -emit-module-source-info-path src/swift/CMakeFiles/swiftDispatch.dir/Data.swift.swiftsourceinfo -emit-dependencies-path src/swift/CMakeFiles/swiftDispatch.dir/Data.swift.o.d -emit-reference-dependencies-path src/swift/CMakeFiles/swiftDispatch.dir/Data.swift.o.swiftdeps -target x86_64-unknown-linux-gnu -disable-objc-interop -I /swift/swift-project/swift-corelibs-libdispatch/src/BlocksRuntime -I /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64 -I /swift/swift-project/swift-corelibs-libdispatch -I /swift/swift-project/swift-corelibs-libdispatch/src -I /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/src -vfsoverlay /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/dispatch-vfs-overlay.yaml -color-diagnostics -module-cache-path /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/module-cache -module-link-name swiftDispatch -O -D swiftDispatch_EXPORTS -new-driver-path /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/bin/swift-driver -empty-abi-descriptor -resource-dir /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/lib/swift -Xcc -fblocks -Xcc -fmodule-map-file=/swift/swift-project/swift-corelibs-libdispatch/dispatch/module.modulemap -Xcc -I/swift/swift-project/swift-corelibs-libdispatch -Xcc -I/swift/swift-project/swift-corelibs-libdispatch/src/swift/shims -module-name Dispatch -plugin-path /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/lib/swift/host/plugins -plugin-path /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/local/lib/swift/host/plugins -enable-default-cmo -parse-as-library -o src/swift/CMakeFiles/swiftDispatch.dir/Data.swift.o
/swift/swift-project/build/buildbot_linux/swift-linux-x86_64/bin/swift-frontend -frontend -c /swift/swift-project/swift-corelibs-libdispatch/src/swift/Block.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Data.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Dispatch.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/IO.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Private.swift -primary-file /swift/swift-project/swift-corelibs-libdispatch/src/swift/Queue.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Source.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Time.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Wrapper.swift -emit-module-path src/swift/CMakeFiles/swiftDispatch.dir/Queue.swift.swiftmodule -emit-module-doc-path src/swift/CMakeFiles/swiftDispatch.dir/Queue.swift.swiftdoc -emit-module-source-info-path src/swift/CMakeFiles/swiftDispatch.dir/Queue.swift.swiftsourceinfo -emit-dependencies-path src/swift/CMakeFiles/swiftDispatch.dir/Queue.swift.o.d -emit-reference-dependencies-path src/swift/CMakeFiles/swiftDispatch.dir/Queue.swift.o.swiftdeps -target x86_64-unknown-linux-gnu -disable-objc-interop -I /swift/swift-project/swift-corelibs-libdispatch/src/BlocksRuntime -I /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64 -I /swift/swift-project/swift-corelibs-libdispatch -I /swift/swift-project/swift-corelibs-libdispatch/src -I /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/src -vfsoverlay /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/dispatch-vfs-overlay.yaml -color-diagnostics -module-cache-path /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/module-cache -module-link-name swiftDispatch -O -D swiftDispatch_EXPORTS -new-driver-path /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/bin/swift-driver -empty-abi-descriptor -resource-dir /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/lib/swift -Xcc -fblocks -Xcc -fmodule-map-file=/swift/swift-project/swift-corelibs-libdispatch/dispatch/module.modulemap -Xcc -I/swift/swift-project/swift-corelibs-libdispatch -Xcc -I/swift/swift-project/swift-corelibs-libdispatch/src/swift/shims -module-name Dispatch -plugin-path /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/lib/swift/host/plugins -plugin-path /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/local/lib/swift/host/plugins -enable-default-cmo -parse-as-library -o src/swift/CMakeFiles/swiftDispatch.dir/Queue.swift.o
/swift/swift-project/build/buildbot_linux/swift-linux-x86_64/bin/swift-frontend -frontend -c /swift/swift-project/swift-corelibs-libdispatch/src/swift/Block.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Data.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Dispatch.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/IO.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Private.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Queue.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Source.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Time.swift -primary-file /swift/swift-project/swift-corelibs-libdispatch/src/swift/Wrapper.swift -emit-module-path src/swift/CMakeFiles/swiftDispatch.dir/Wrapper.swift.swiftmodule -emit-module-doc-path src/swift/CMakeFiles/swiftDispatch.dir/Wrapper.swift.swiftdoc -emit-module-source-info-path src/swift/CMakeFiles/swiftDispatch.dir/Wrapper.swift.swiftsourceinfo -emit-dependencies-path src/swift/CMakeFiles/swiftDispatch.dir/Wrapper.swift.o.d -emit-reference-dependencies-path src/swift/CMakeFiles/swiftDispatch.dir/Wrapper.swift.o.swiftdeps -target x86_64-unknown-linux-gnu -disable-objc-interop -I /swift/swift-project/swift-corelibs-libdispatch/src/BlocksRuntime -I /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64 -I /swift/swift-project/swift-corelibs-libdispatch -I /swift/swift-project/swift-corelibs-libdispatch/src -I /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/src -vfsoverlay /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/dispatch-vfs-overlay.yaml -color-diagnostics -module-cache-path /swift/swift-project/build/buildbot_linux/libdispatch-linux-x86_64/module-cache -module-link-name swiftDispatch -O -D swiftDispatch_EXPORTS -new-driver-path /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/bin/swift-driver -empty-abi-descriptor -resource-dir /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/lib/swift -Xcc -fblocks -Xcc -fmodule-map-file=/swift/swift-project/swift-corelibs-libdispatch/dispatch/module.modulemap -Xcc -I/swift/swift-project/swift-corelibs-libdispatch -Xcc -I/swift/swift-project/swift-corelibs-libdispatch/src/swift/shims -module-name Dispatch -plugin-path /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/lib/swift/host/plugins -plugin-path /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/local/lib/swift/host/plugins -enable-default-cmo -parse-as-library -o src/swift/CMakeFiles/swiftDispatch.dir/Wrapper.swift.o
/swift/swift-project/swift-corelibs-libdispatch/src/swift/Dispatch.swift:99:4: warning: default will never be executed
                        default: return nil
                        ^
error: fatalError

Finagolfin · December 18, 2023, 3:57am

Did comparing the CMake "Looking for" log turn up anything?

Looks like the problem is on our end, not CMake: I would guess that the new swift-syntax parser or some other compiler component written in Swift is crashing somewhere.

Have you run the compiler validation suite on this trunk compiler? If not, do that first, either by passing --skip-build-libdispatch to build-script, as I recently rewired libdispatch as a dependency of the validation suite on linux, or by also turning --bootstrapping=off if that doesn't work, to remove all components of the Swift compiler written in Swift.

Once you are sure the compiler validation suite is working well, you can turn those two back on and hopefully the problem won't be there by then.

taylorswift · December 18, 2023, 8:59pm

it looks like --skip-build-libdispatch isn’t a valid build-script flag:

build-script: error: unrecognized arguments: --skip-build-libdispatch

  --skip-build-linux [BOOL]
                        skip building Swift stdlibs for Linux
  --skip-build-freebsd [BOOL]
                        skip building Swift stdlibs for FreeBSD
  --skip-build-cygwin [BOOL]
                        skip building Swift stdlibs for Cygwin
  --skip-build-osx [BOOL]
                        skip building Swift stdlibs for MacOSX
  --skip-build-ios [BOOL]
                        skip building Swift stdlibs for iOS
  --skip-build-ios-device [BOOL]
                        skip building Swift stdlibs for iOS devices (i.e. build simulators
                        only)
  --skip-build-ios-simulator [BOOL]
                        skip building Swift stdlibs for iOS simulator (i.e. build devices
                        only)
  --skip-build-tvos [BOOL]
                        skip building Swift stdlibs for tvOS
  --skip-build-tvos-device [BOOL]
                        skip building Swift stdlibs for tvOS devices (i.e. build simulators
                        only)
  --skip-build-tvos-simulator [BOOL]
                        skip building Swift stdlibs for tvOS simulator (i.e. build devices
                        only)
  --skip-build-watchos [BOOL]
                        skip building Swift stdlibs for watchOS
  --skip-build-watchos-device [BOOL]
                        skip building Swift stdlibs for watchOS devices (i.e. build simulators
                        only)
  --skip-build-watchos-simulator [BOOL]
                        skip building Swift stdlibs for watchOS simulator (i.e. build devices
                        only)
  --skip-build-android [BOOL]
                        skip building Swift stdlibs for Android
  --skip-build-benchmarks [BOOL]
                        skip building Swift Benchmark Suite

Finagolfin · December 19, 2023, 7:51am

I typed that from memory and misremembered: the correct flag is --libdispatch=False. If you are running a preset that also builds Foundation, you will have to patch this non-Darwin override too.

You never answered my questions about the CMake log and compiler validation suite. It is frustrating trying to figure things out remotely when salient questions are randomly unanswered.

taylorswift · December 19, 2023, 11:22pm

sorry for missing that, here’s a diff of my log against that of the CI, lightly edited:

5c5
- Check for working C compiler: /home/build-user/build/buildbot_linux/llvm-linux-x86_64/bin/clang - skipped
---
+ Check for working C compiler: /swift/swift-project/build/buildbot_linux/llvm-linux-x86_64/bin/clang - skipped
10c10
- Check for working CXX compiler: /home/build-user/build/buildbot_linux/llvm-linux-x86_64/bin/clang++ - skipped
---
+ Check for working CXX compiler: /swift/swift-project/build/buildbot_linux/llvm-linux-x86_64/bin/clang++ - skipped
14,16c14
- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
- Check if compiler accepts -pthread
- Check if compiler accepts -pthread - yes
---
+ Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
19,20c17,18
- Check for working Swift compiler: /home/build-user/build/buildbot_linux/swift-linux-x86_64/bin/swiftc
- Check for working Swift compiler: /home/build-user/build/buildbot_linux/swift-linux-x86_64/bin/swiftc - works
---
+ Check for working Swift compiler: /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/bin/swiftc
+ Check for working Swift compiler: /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/bin/swiftc - works
25c23
- Found LibRT: /usr/lib64/librt.so
---
+ Found LibRT: /usr/lib64/librt.a

interestingly, CMAKE_HAVE_LIBC_PTHREAD succeeds for me, but not the CI.

as for running the validation suite, i’m assuming you mean utils/build-script --validation-test, so i tried doing that; this time it is SwiftSyntax that fails to build:

Linking Swift shared library lib/swift/host/libSwiftSyntax.so
FAILED: lib/swift/host/libSwiftSyntax.so ...

...

error: unknown argument: '-gsplit-dwarf'

the step that seems to be failing is

/opt/swift/5.9.2/usr/bin/swiftc -target x86_64-unknown-linux-gnu -output-file-map _deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/Debug/output-file-map.json -incremental -j 12 -emit-library -o lib/swift/host/libSwiftSyntax.so -module-name SwiftSyntax -module-link-name SwiftSyntax -emit-module -emit-module-path lib/swift/host/SwiftSyntax.swiftmodule -emit-dependencies -DCMARK_STATIC_DEFINE -DSWIFTSYNTAX_ENABLE_ASSERTIONS -DSWIFT_BUILD_SWIFT_SYNTAX -DSWIFT_TARGET -DSwiftSyntax_EXPORTS -D_DEBUG -D_GLIBCXX_ASSERTIONS -D_LIBCPP_ENABLE_HARDENED_MODE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -g -swift-version 5 -gsplit-dwarf -Xfrontend -disable-implicit-string-processing-module-import -module-name SwiftSyntax -enable-library-evolution -emit-module-path /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/lib/swift/host/SwiftSyntax.swiftmodule/x86_64-unknown-linux-gnu.swiftmodule -emit-module-source-info-path /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/lib/swift/host/SwiftSyntax.swiftmodule/x86_64-unknown-linux-gnu.swiftsourceinfo -emit-module-interface-path /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/lib/swift/host/SwiftSyntax.swiftmodule/x86_64-unknown-linux-gnu.swiftinterface -Xfrontend -module-abi-name -Xfrontend CompilerSwiftSyntax -color-diagnostics -no-toolchain-stdlib-rpath -I /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax -I /swift/swift-project/swift-syntax/Sources/SwiftSyntax -I /swift/swift-project/swift/stdlib/public/SwiftShims -I /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/include -I /swift/swift-project/swift/include -I /swift/swift-project/llvm-project/llvm/include -I /swift/swift-project/build/Ninja-DebugAssert/llvm-linux-x86_64/include -I /swift/swift-project/llvm-project/clang/include -I /swift/swift-project/build/Ninja-DebugAssert/llvm-linux-x86_64/tools/clang/include -I /swift/swift-project/cmark/src/include -I /swift/swift-project/build/Ninja-DebugAssert/cmark-linux-x86_64/src -I /swift/swift-project/swift-corelibs-libdispatch/src/BlocksRuntime -I /swift/swift-project/swift-corelibs-libdispatch /swift/swift-project/swift-syntax/Sources/SwiftSyntax/AbsolutePosition.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/AbsoluteRawSyntax.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/AbsoluteSyntaxInfo.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/Assert.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/BumpPtrAllocator.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/CommonAncestor.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/Convenience.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/MemoryLayout.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/MissingNodeInitializers.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/SourceEdit.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/SourceLength.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/SourceLocation.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/SourcePresence.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/SwiftSyntaxCompatibility.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/Syntax.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/SyntaxArena.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/SyntaxChildren.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/SyntaxCollection.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/SyntaxHashable.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/SyntaxIdentifier.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/SyntaxNodeStructure.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/SyntaxProtocol.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/SyntaxText.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/SyntaxTreeViewMode.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/TokenDiagnostic.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/TokenSequence.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/TokenSyntax.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/Trivia.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/Utils.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/Raw/RawSyntax.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/Raw/RawSyntaxLayoutView.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/Raw/RawSyntaxNodeProtocol.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/Raw/RawSyntaxTokenView.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/raw/RawSyntaxNodesAB.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/raw/RawSyntaxNodesC.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/raw/RawSyntaxNodesD.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/raw/RawSyntaxNodesEF.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/raw/RawSyntaxNodesGHI.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/raw/RawSyntaxNodesJKLMN.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/raw/RawSyntaxNodesOP.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/raw/RawSyntaxNodesQRS.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/raw/RawSyntaxNodesTUVWXYZ.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/raw/RawSyntaxValidation.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/ChildNameForKeyPath.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/Keyword.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/RenamedChildrenCompatibility.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/RenamedNodesCompatibility.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/SyntaxAnyVisitor.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/SyntaxBaseNodes.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/SyntaxCollections.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/SyntaxEnum.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/SyntaxKind.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/SyntaxRewriter.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/SyntaxTraits.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/SyntaxVisitor.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/TokenKind.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/Tokens.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/TriviaPieces.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/syntaxNodes/SyntaxNodesAB.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/syntaxNodes/SyntaxNodesC.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/syntaxNodes/SyntaxNodesD.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/syntaxNodes/SyntaxNodesEF.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/syntaxNodes/SyntaxNodesGHI.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/syntaxNodes/SyntaxNodesJKLMN.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/syntaxNodes/SyntaxNodesOP.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/syntaxNodes/SyntaxNodesQRS.swift /swift/swift-project/swift-syntax/Sources/SwiftSyntax/generated/syntaxNodes/SyntaxNodesTUVWXYZ.swift  -Xlinker -soname -Xlinker libSwiftSyntax.so  -L /swift/swift-project/build/Ninja-DebugAssert/llvm-linux-x86_64/./lib -Xlinker -rpath -Xlinker "\$ORIGIN:\$ORIGIN/../linux" && cd /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax && /swift/swift-project/build/cmake-linux-x86_64/bin/cmake -E touch_nocreate /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/lib/swift/host/libSwiftSyntax.so /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./AbsolutePosition.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./AbsoluteRawSyntax.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./AbsoluteSyntaxInfo.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./Assert.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./BumpPtrAllocator.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./CommonAncestor.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./Convenience.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./MemoryLayout.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./MissingNodeInitializers.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./SourceEdit.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./SourceLength.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./SourceLocation.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./SourcePresence.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./SwiftSyntaxCompatibility.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./Syntax.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./SyntaxArena.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./SyntaxChildren.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./SyntaxCollection.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./SyntaxHashable.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./SyntaxIdentifier.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./SyntaxNodeStructure.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./SyntaxProtocol.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./SyntaxText.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./SyntaxTreeViewMode.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./TokenDiagnostic.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./TokenSequence.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./TokenSyntax.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./Trivia.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./Utils.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./Raw/RawSyntax.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./Raw/RawSyntaxLayoutView.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./Raw/RawSyntaxNodeProtocol.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./Raw/RawSyntaxTokenView.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/raw/RawSyntaxNodesAB.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/raw/RawSyntaxNodesC.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/raw/RawSyntaxNodesD.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/raw/RawSyntaxNodesEF.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/raw/RawSyntaxNodesGHI.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/raw/RawSyntaxNodesJKLMN.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/raw/RawSyntaxNodesOP.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/raw/RawSyntaxNodesQRS.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/raw/RawSyntaxNodesTUVWXYZ.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/raw/RawSyntaxValidation.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/ChildNameForKeyPath.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/Keyword.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/RenamedChildrenCompatibility.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/RenamedNodesCompatibility.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/SyntaxAnyVisitor.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/SyntaxBaseNodes.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/SyntaxCollections.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/SyntaxEnum.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/SyntaxKind.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/SyntaxRewriter.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/SyntaxTraits.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/SyntaxVisitor.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/TokenKind.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/Tokens.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/TriviaPieces.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/syntaxNodes/SyntaxNodesAB.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/syntaxNodes/SyntaxNodesC.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/syntaxNodes/SyntaxNodesD.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/syntaxNodes/SyntaxNodesEF.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/syntaxNodes/SyntaxNodesGHI.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/syntaxNodes/SyntaxNodesJKLMN.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/syntaxNodes/SyntaxNodesOP.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/syntaxNodes/SyntaxNodesQRS.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/_deps/swiftsyntax-build/Sources/SwiftSyntax/CMakeFiles/SwiftSyntax.dir/./generated/syntaxNodes/SyntaxNodesTUVWXYZ.swift.o /swift/swift-project/build/Ninja-DebugAssert/swift-linux-x86_64/lib/swift/host/SwiftSyntax.swiftmodule

i’m not sure why cmake is passing that flag to swiftc; google tells me -gsplit-dwarf is a clang flag. adding a --bootstrapping=off didn’t change anything.

i wonder if this is because the --validation-test mode runs in debug mode, as i don’t remember this happening with --release-debuginfo. unfortunately i had to delete Ninja-RelWithDebInfoAssert to make room on disk for the validation test build, so i will have to do that tomorrow.

Finagolfin · December 20, 2023, 6:22pm

OK, so that's not the problem.

You were running one of the presets so far: is it one that runs the tests or not? If not, simply adding -T to that preset will run the tests for most Swift repos.

Did you add a --reconfigure too? Check the CMake config log and if adding that flag doesn't change the bootstrapping mode either, deleting the Swift frontend build directory swift-linux-x86_64/ should do it.

Don't start over with a whole new command, simply add -T if needed to your original build command instead.

taylorswift · December 20, 2023, 11:34pm

okay, so i ran

$ utils/build-script --release-debuginfo --validation-test

and the first time, the swift driver crashed with a segmentation fault superficially resembling one i’ve encountered in the past:

FAILED: lib/swift/embedded/Swift.swiftmodule/armv7em-none-none-eabi.swiftmodule lib/swift/embedded/Swift.swiftmodule/armv7em-none-none-eabi.swiftdoc /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/lib/swift/embedded/Swift.swiftmodule/armv7em-none-none-eabi.swiftmodule /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/lib/swift/embedded/Swift.swiftmodule/armv7em-none-none-eabi.swiftdoc 
cd /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/stdlib/public/core && /swift/swift-project/build/cmake-linux-x86_64/bin/cmake -E make_directory /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/stdlib/public/core/embedded/armv7em && /swift/swift-project/build/cmake-linux-x86_64/bin/cmake -E remove -f /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/lib/swift/embedded/Swift.swiftmodule/armv7em-none-none-eabi.swiftmodule /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/lib/swift/embedded/Swift.swiftmodule/armv7em-none-none-eabi.swiftdoc && /swift/swift-project/build/cmake-linux-x86_64/bin/cmake -E make_directory /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/lib/swift/embedded /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/lib/swift/embedded/Swift.swiftmodule && /swift/swift-project/build/cmake-linux-x86_64/bin/cmake -E env PYTHONIOENCODING=UTF8 LD_LIBRARY_PATH=/opt/swift/5.9.2/usr/lib/swift/linux /usr/bin/python3.9 /swift/swift-project/swift/utils/line-directive @/swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/stdlib/public/core/81eed2b71b4043f7a19f80514872f19668ede62c.txt -- /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/./bin/swiftc -emit-module -o /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/lib/swift/embedded/Swift.swiftmodule/armv7em-none-none-eabi.swiftmodule -avoid-emit-module-source-info -target armv7em-none-none-eabi -resource-dir /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/./lib/swift -O -g -D INTERNAL_CHECKS_ENABLED -D SWIFT_ENABLE_EXPERIMENTAL_CONCURRENCY -D SWIFT_ENABLE_EXPERIMENTAL_DISTRIBUTED -D SWIFT_ENABLE_RUNTIME_FUNCTION_COUNTERS -D SWIFT_ENABLE_EXPERIMENTAL_DIFFERENTIABLE_PROGRAMMING -D SWIFT_ENABLE_EXPERIMENTAL_STRING_PROCESSING -D SWIFT_ENABLE_EXPERIMENTAL_OBSERVATION -D SWIFT_RUNTIME_OS_VERSIONING -D SWIFT_STDLIB_ENABLE_UNICODE_DATA -D SWIFT_STDLIB_ENABLE_VECTOR_TYPES -D SWIFT_STDLIB_HAS_COMMANDLINE -D SWIFT_STDLIB_HAS_STDIN -D SWIFT_STDLIB_HAS_ENVIRON -Xcc -DSWIFT_STDLIB_HAS_ENVIRON -D SWIFT_CONCURRENCY_USES_DISPATCH -D SWIFT_STDLIB_OVERRIDABLE_RETAIN_RELEASE -D SWIFT_THREADING_ -tools-directory /swift/swift-project/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/./bin -module-cache-path /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/./module-cache -no-link-objc-runtime -Xfrontend -enforce-exclusivity=unchecked -nostdimport -parse-stdlib -module-name Swift -Xfrontend -group-info-path -Xfrontend /swift/swift-project/swift/stdlib/public/core/GroupInfo.json -swift-version 5 -runtime-compatibility-version none -disable-autolinking-runtime-compatibility-dynamic-replacements -Xfrontend -disable-autolinking-runtime-compatibility-concurrency -warn-swift3-objc-inference-complete -Xfrontend -disable-objc-interop -Xfrontend -enable-experimental-associated-type-inference -diagnostic-style swift -Xllvm -sil-inline-generics -Xllvm -sil-partial-specialization -Xfrontend -enable-experimental-concise-pound-file -enable-experimental-feature Macros -enable-experimental-feature FreestandingMacros -enable-experimental-feature Extern -enable-experimental-feature BitwiseCopyable -Xcc -D__MACH__ -Xcc -D__APPLE__ -Xcc -ffreestanding -enable-experimental-feature Embedded -Xfrontend -enable-ossa-modules -Xfrontend -define-availability -Xfrontend SwiftStdlib\ 9999:macOS\ 9999,\ iOS\ 9999,\ watchOS\ 9999,\ tvOS\ 9999 -Xfrontend -define-availability -Xfrontend SwiftStdlib\ 5.0:macOS\ 10.14.4,\ iOS\ 12.2,\ watchOS\ 5.2,\ tvOS\ 12.2 -Xfrontend -define-availability -Xfrontend SwiftStdlib\ 5.1:macOS\ 10.15,\ iOS\ 13.0,\ watchOS\ 6.0,\ tvOS\ 13.0 -Xfrontend -define-availability -Xfrontend SwiftStdlib\ 5.2:macOS\ 10.15.4,\ iOS\ 13.4,\ watchOS\ 6.2,\ tvOS\ 13.4 -Xfrontend -define-availability -Xfrontend SwiftStdlib\ 5.3:macOS\ 11.0,\ iOS\ 14.0,\ watchOS\ 7.0,\ tvOS\ 14.0 -Xfrontend -define-availability -Xfrontend SwiftStdlib\ 5.4:macOS\ 11.3,\ iOS\ 14.5,\ watchOS\ 7.4,\ tvOS\ 14.5 -Xfrontend -define-availability -Xfrontend SwiftStdlib\ 5.5:macOS\ 12.0,\ iOS\ 15.0,\ watchOS\ 8.0,\ tvOS\ 15.0 -Xfrontend -define-availability -Xfrontend SwiftStdlib\ 5.6:macOS\ 12.3,\ iOS\ 15.4,\ watchOS\ 8.5,\ tvOS\ 15.4 -Xfrontend -define-availability -Xfrontend SwiftStdlib\ 5.7:macOS\ 13.0,\ iOS\ 16.0,\ watchOS\ 9.0,\ tvOS\ 16.0 -Xfrontend -define-availability -Xfrontend SwiftStdlib\ 5.8:macOS\ 13.3,\ iOS\ 16.4,\ watchOS\ 9.4,\ tvOS\ 16.4 -Xfrontend -define-availability -Xfrontend SwiftStdlib\ 5.9:macOS\ 14.0,\ iOS\ 17.0,\ watchOS\ 10.0,\ tvOS\ 17.0 -Xfrontend -define-availability -Xfrontend SwiftStdlib\ 5.10:macOS\ 9999,\ iOS\ 9999,\ watchOS\ 9999,\ tvOS\ 9999 -Xfrontend -define-availability -Xfrontend SwiftStdlib\ 5.11:macOS\ 9999,\ iOS\ 9999,\ watchOS\ 9999,\ tvOS\ 9999 -Xfrontend -target-min-inlining-version -Xfrontend min -module-link-name swiftCore -whole-module-optimization -color-diagnostics -parse-as-library -resource-dir /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/./lib/swift -I /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/./lib/swift/embedded @/swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/stdlib/public/core/81eed2b71b4043f7a19f80514872f19668ede62c.txt

*** Signal 11: Backtracing from 0x7fe49b8a7fa0... done ***

*** Program crashed: Bad pointer dereference at 0x0000000000000066 ***

Thread 0 "swift-driver" crashed:

0  0x00007fe49b8a7fa0 <unknown> in lib_InternalSwiftScan.so

Thread 1:

0  0x00007fe4a927586e epoll_wait + 94 in libc.so.6


Registers:

rax 0x00007fe49fe6db30  80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ················
rdx 0x00007fe49fe6dab0  60 f0 c7 9c e4 7f 00 00 00 00 00 00 00 00 00 00  `ðÇ·ä···········
rcx 0x0000000000000000  0
rbx 0x0000000000000000  0
rsi 0x00007fe49fe6dab0  60 f0 c7 9c e4 7f 00 00 00 00 00 00 00 00 00 00  `ðÇ·ä···········
rdi 0x0000000000000000  0
rbp 0x00000000000000fa  250
rsp 0x00007ffe48147428  fe d8 6e 95 e4 7f 00 00 28 66 64 9c e4 7f 00 00  þØn·ä···(fd·ä···
 r8 0x00005556f4a3bb50  04 00 00 00 00 00 00 00 76 82 77 13 36 4f 27 93  ········v·w·6O'·
 r9 0x0000000000000000  0
r10 0x00007fe4a9136410  f7 09 00 00 12 00 0e 00 60 6b 05 00 00 00 00 00  ÷·······`k······
r11 0x00007fe4a917db60  f3 0f 1e fa 41 57 41 56 41 55 41 54 55 53 48 83  ó··úAWAVAUATUSH·
r12 0x00007ffe4814bad8  02 e4 14 48 fe 7f 00 00 5c e4 14 48 fe 7f 00 00  ·ä·Hþ···\ä·Hþ···
r13 0x00007ffe4814c2b0  92 09 15 48 fe 7f 00 00 a7 09 15 48 fe 7f 00 00  ···Hþ···§··Hþ···
r14 0x00007fe49fe6db00  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ················
r15 0x0000000000000000  0
rip 0x00007fe49b8a7fa0  00 66 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00  ·ffffff.········

rflags 0x0000000000010a82  SF

cs 0x0033  fs 0x0000  gs 0x0000


Images (44 omitted):

0x00007fe494c00000–0x00007fe49c644b30 <no build ID>                            lib_InternalSwiftScan.so /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/lib/lib_InternalSwiftScan.so
0x00007fe4a9127000–0x00007fe4a92c304c a84b0b221620b4944c5940d3bed62056c7d7baa4 libc.so.6                /usr/lib64/libc.so.6

Backtrace took 0.02s

[122/283][ 43%][419.361s] Generating /swift/s...ftmodule/x86_64-unknown-linux-gnu.swiftmodule
warning: Could not read SDKSettings.json for SDK at: /
[123/283][ 43%][510.091s] Compiling /swift/sw...6_64/stdlib/public/core//LINUX/x86_64/Swift.o
warning: Could not read SDKSettings.json for SDK at: /
ninja: build stopped: subcommand failed.
ERROR: command terminated with a non-zero exit status 1, aborting

note that swift-driver already segfaults whenever it can’t find one of its input files, and this is not specific to building the compiler.

oddly enough, when i retried the same build, it succeeded and began running tests. some of the tests failed, but i’m told that’s normal.

--- Running tests for swift ---
--- check-swift-only_early_swiftdriver-linux-x86_64 ---
+ /swift/swift-project/build/cmake-linux-x86_64/bin/cmake --build /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64 -- -j12 check-swift-only_early_swiftdriver-linux-x86_64
[1/33][  3%][0.195s] Performing build step for 'libdispatch-linux-x86_64'
ninja: no work to do.
[3/4][ 75%][0.234s] Running only_early_swiftdriver Swift tests for x86_64-unknown-linux-gnu
...
lit.py: /swift/swift-project/swift/test/lit.cfg:2873: note: Available features: CMAKE_GENERATOR=Ninja, CODEGENERATOR=AArch64, CODEGENERATOR=ARM, CODEGENERATOR=Mips, CODEGENERATOR=PowerPC, CODEGENERATOR=RISCV, CODEGENERATOR=SystemZ, CODEGENERATOR=WebAssembly, CODEGENERATOR=X86, CPU=x86_64, OS=linux-gnu, PTRSIZE=64, STDLIB_VARIANT=linux-x86_64, SWIFT_VERSION=4, VENDOR=unknown, asan_runtime, asserts, autolink-extract, backtracing, concurrency, concurrency_runtime, cplusplus_driver, crash-recovery, differentiable_programming, distributed, executable_test, fuzzer_runtime, hosttools_mode, libxml2, no_asan, no_lto, nonexecutable_test, objc_codegen, observation, optimized_stdlib, optimized_stdlib_x86_64, reflection, reflection_test_support, runtime_function_counters, scudo_runtime, shell, sourcekit, string_processing, swift-remoteast-test, swift_in_compiler, swift_interpreter, swift_only_stable_abi, swift_stdlib_asserts, swift_swift_parser, swift_test_mode_optimize_none, swift_test_mode_optimize_none_x86_64, swift_tools_extra, thread_safe_runtime, threading_linux, ubsan_runtime

Testing Time: 10.06s
  Unsupported: 17
  Passed     : 33

2 warning(s) in tests
-- check-swift-only_early_swiftdriver-linux-x86_64 finished --
--- check-swift-validation-linux-x86_64 ---
+ /swift/swift-project/build/cmake-linux-x86_64/bin/cmake --build /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64 -- -j12 check-swift-validation-linux-x86_64
[1/31][  3%][0.178s] Performing build step for 'libdispatch-linux-x86_64'
ninja: no work to do.
[1/2][ 50%][0.179s] Running validation Swift tests for x86_64-unknown-linux-gnu

...

Failed Tests (1):
  Swift-validation(linux-x86_64) :: Reflection/reflect_Enum_MultiPayload_bulky.swift


Testing Time: 2435.12s
  Unsupported      :  2250
  Passed           : 15253
  Expectedly Failed:    37
  Failed           :     1

4 warning(s) in tests
FAILED: test/CMakeFiles/check-swift-validation-linux-x86_64 /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/test/CMakeFiles/check-swift-validation-linux-x86_64 
cd /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/test && /swift/swift-project/build/cmake-linux-x86_64/bin/cmake -E remove_directory /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/./swift-test-results/x86_64-unknown-linux-gnu && /swift/swift-project/build/cmake-linux-x86_64/bin/cmake -E make_directory /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/./swift-test-results/x86_64-unknown-linux-gnu && /swift/swift-project/build/cmake-linux-x86_64/bin/cmake -E env /usr/bin/python3.9 /swift/swift-project/llvm-project/llvm/utils/lit/lit.py -sv -j 12 --xunit-xml-output=/swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/./swift-test-results/x86_64-unknown-linux-gnu/lit-tests.xml --param differentiable_programming --param concurrency --param distributed --param string_processing --param backtracing --param observation --param threading=linux --param backtrace_on_crash --param swift_test_subset=validation --param swift_test_mode=optimize_none /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/test-linux-x86_64 /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/test/../validation-test-linux-x86_64
ninja: build stopped: subcommand failed.
ERROR: command terminated with a non-zero exit status 1, aborting

if i instead run

$ utils/build-script --release-debuginfo -T

it runs far fewer tests, but one of the failing tests is related to swift-driver:

FAIL: Swift(linux-x86_64) :: one-way-parallel-fine.swift (42 of 50)
******************** TEST 'Swift(linux-x86_64) :: one-way-parallel-fine.swift' FAILED ********************
Script:
--
: 'RUN: at line 3';   rm -rf "/swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/test-linux-x86_64/Output/one-way-parallel-fine.swift.tmp" && mkdir -p "/swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/test-linux-x86_64/Output/one-way-parallel-fine.swift.tmp"
: 'RUN: at line 4';   cp -r /swift/swift-project/swift/test/Driver/Dependencies/Inputs/one-way-fine/* /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/test-linux-x86_64/Output/one-way-parallel-fine.swift.tmp
: 'RUN: at line 5';   touch -t 201401240005 /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/test-linux-x86_64/Output/one-way-parallel-fine.swift.tmp/*
: 'RUN: at line 7';   cd /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/test-linux-x86_64/Output/one-way-parallel-fine.swift.tmp && '/swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/bin/swiftc' -toolchain-stdlib-rpath -module-cache-path /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/swift-test-results/x86_64-unknown-linux-gnu/clang-module-cache -swift-version 4  -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 9999:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.0:macOS 10.14.4, iOS 12.2, watchOS 5.2, tvOS 12.2' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.1:macOS 10.15, iOS 13.0, watchOS 6.0, tvOS 13.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.2:macOS 10.15.4, iOS 13.4, watchOS 6.2, tvOS 13.4' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.3:macOS 11.0, iOS 14.0, watchOS 7.0, tvOS 14.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.4:macOS 11.3, iOS 14.5, watchOS 7.4, tvOS 14.5' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.5:macOS 12.0, iOS 15.0, watchOS 8.0, tvOS 15.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.6:macOS 12.3, iOS 15.4, watchOS 8.5, tvOS 15.4' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.7:macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.8:macOS 13.3, iOS 16.4, watchOS 9.4, tvOS 16.4' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.9:macOS 14.0, iOS 17.0, watchOS 10.0, tvOS 17.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.10:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.11:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999' -c -driver-use-frontend-path "/usr/bin/python3.9;/swift/swift-project/swift/test/Driver/Dependencies/Inputs/update-dependencies.py;/swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/bin/swift-dependency-tool" -output-file-map /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/test-linux-x86_64/Output/one-way-parallel-fine.swift.tmp/output.json -incremental -driver-always-rebuild-dependents ./main.swift ./other.swift -module-name main -j1 -parseable-output 2>&1 | /usr/bin/python3.9 /swift/swift-project/swift/utils/PathSanitizingFileCheck --allow-unused-prefixes --sanitize BUILD_DIR=/swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64 --sanitize SOURCE_DIR=/swift/swift-project/swift --use-filecheck /swift/swift-project/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/bin/FileCheck  -check-prefix=CHECK-FIRST /swift/swift-project/swift/test/Driver/Dependencies/one-way-parallel-fine.swift
: 'RUN: at line 34';   touch -t 201401240006 /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/test-linux-x86_64/Output/one-way-parallel-fine.swift.tmp/other.swift
: 'RUN: at line 35';   cd /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/test-linux-x86_64/Output/one-way-parallel-fine.swift.tmp && '/swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/bin/swiftc' -toolchain-stdlib-rpath -module-cache-path /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/swift-test-results/x86_64-unknown-linux-gnu/clang-module-cache -swift-version 4  -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 9999:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.0:macOS 10.14.4, iOS 12.2, watchOS 5.2, tvOS 12.2' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.1:macOS 10.15, iOS 13.0, watchOS 6.0, tvOS 13.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.2:macOS 10.15.4, iOS 13.4, watchOS 6.2, tvOS 13.4' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.3:macOS 11.0, iOS 14.0, watchOS 7.0, tvOS 14.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.4:macOS 11.3, iOS 14.5, watchOS 7.4, tvOS 14.5' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.5:macOS 12.0, iOS 15.0, watchOS 8.0, tvOS 15.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.6:macOS 12.3, iOS 15.4, watchOS 8.5, tvOS 15.4' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.7:macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.8:macOS 13.3, iOS 16.4, watchOS 9.4, tvOS 16.4' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.9:macOS 14.0, iOS 17.0, watchOS 10.0, tvOS 17.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.10:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.11:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999' -c -driver-use-frontend-path "/usr/bin/python3.9;/swift/swift-project/swift/test/Driver/Dependencies/Inputs/update-dependencies.py;/swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/bin/swift-dependency-tool" -output-file-map /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/test-linux-x86_64/Output/one-way-parallel-fine.swift.tmp/output.json -incremental -driver-always-rebuild-dependents ./main.swift ./other.swift -module-name main -j2 -parseable-output 2>&1 | /usr/bin/python3.9 /swift/swift-project/swift/utils/PathSanitizingFileCheck --allow-unused-prefixes --sanitize BUILD_DIR=/swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64 --sanitize SOURCE_DIR=/swift/swift-project/swift --use-filecheck /swift/swift-project/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/bin/FileCheck  -check-prefix=CHECK-SECOND /swift/swift-project/swift/test/Driver/Dependencies/one-way-parallel-fine.swift
--
Exit Code: 1

Command Output (stderr):
--
/swift/swift-project/swift/test/Driver/Dependencies/one-way-parallel-fine.swift:44:22: error: CHECK-SECOND-DAG: expected string not found in input
// CHECK-SECOND-DAG: "kind"{{ *}}: "began"
                     ^
<stdin>:82:2: note: scanning from here
{
 ^
<stdin>:84:2: note: possible intended match here
 "kind" : "finished",
 ^

Input file: <stdin>
Check file: /swift/swift-project/swift/test/Driver/Dependencies/one-way-parallel-fine.swift

-dump-input=help explains the following input dump.

Input was:
<<<<<<
          .
          .
          .
         77:  "process" : { 
         78:  "real_pid" : 200858 
         79:  } 
         80: } 
         81: 168 
         82: { 
dag:44'0      X error: no match found
         83:  "exit-status" : 0, 
dag:44'0     ~~~~~~~~~~~~~~~~~~~~
         84:  "kind" : "finished", 
dag:44'0     ~~~~~~~~~~~~~~~~~~~~~~
dag:44'1      ?                     possible intended match
         85:  "name" : "compile", 
dag:44'0     ~~~~~~~~~~~~~~~~~~~~~
         86:  "output" : "Handled main.swift\n", 
dag:44'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         87:  "pid" : 200858, 
dag:44'0     ~~~~~~~~~~~~~~~~~
         88:  "process" : { 
dag:44'0     ~~~~~~~~~~~~~~~
         89:  "real_pid" : 200858 
dag:44'0     ~~~~~~~~~~~~~~~~~~~~~
          .
          .
          .
>>>>>>

--

********************
********************
Failed Tests (1):
  Swift(linux-x86_64) :: one-way-parallel-fine.swift


Testing Time: 8.32s
  Unsupported: 17
  Passed     : 32
  Failed     :  1

2 warning(s) in tests
FAILED: test/CMakeFiles/check-swift-only_early_swiftdriver-linux-x86_64 /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/test/CMakeFiles/check-swift-only_early_swiftdriver-linux-x86_64 
cd /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/test && /swift/swift-project/build/cmake-linux-x86_64/bin/cmake -E remove_directory /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/./swift-test-results/x86_64-unknown-linux-gnu && /swift/swift-project/build/cmake-linux-x86_64/bin/cmake -E make_directory /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/./swift-test-results/x86_64-unknown-linux-gnu && /swift/swift-project/build/cmake-linux-x86_64/bin/cmake -E env /usr/bin/python3.9 /swift/swift-project/llvm-project/llvm/utils/lit/lit.py -sv -j 12 --xunit-xml-output=/swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/./swift-test-results/x86_64-unknown-linux-gnu/lit-tests.xml --param differentiable_programming --param concurrency --param distributed --param string_processing --param backtracing --param observation --param threading=linux --param backtrace_on_crash --param swift_test_subset=only_early_swiftdriver --param swift_test_mode=optimize_none /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/test-linux-x86_64
ninja: build stopped: subcommand failed.
ERROR: command terminated with a non-zero exit status 1, aborting

again, not sure if this is normal for the environment (amazonlinux:2023) in which i am building.

Finagolfin · December 21, 2023, 2:12am

OK, that is very good, the compiler validation tests are running well. It looks like you have some flakiness from the early swift-driver built by the prebuilt Swift 5.9.2 compiler though, as it passed the early swift-driver tests once and failed the second time.

Now try running utils/build-script --release-debuginfo -T --xctest --skip-test-swift, which then builds the Swift corelibs with this tested compiler and runs their tests, while skipping the validation suite because of the one failing test (remove that libdispatch patch for utils/build-script that I suggested earlier, as you want to build libdispatch now, and add a --skip-test-early-swift-driver if that keeps flaking on you).

If that works well, go back and run the validation suite with libdispatch, utils/build-script --release-debuginfo -T --libdispatch, as your first run of the compiler validation suite would have disabled several concurrency-related tests that depend on libdispatch.

Since you're seeing flakiness with the early swift-driver, try any failing run again a handful of times and skip those early swift-driver tests whenever wanted.

taylorswift · December 23, 2023, 11:40pm

okay, i think i’ve gotten a bit too far ahead of myself, because when i tried running utils/build-script --release-debuginfo -T --xctest --skip-test-swift, the earlyswiftdriver build step complained about:

CMake Error in Sources/Yams/CMakeLists.txt:
  Imported target "Foundation" includes non-existent path

    "/swift/swift-project/build/Ninja-RelWithDebInfoAssert/foundation-linux-x86_64/swift"

  in its INTERFACE_INCLUDE_DIRECTORIES.  Possible reasons include:

  * The path was deleted, renamed, or moved to another location.

  * An install or uninstall procedure did not complete successfully.

  * The installation package was faulty and references files it does not
  provide.

the error spoke the truth, the ../build/Ninja-RelWithDebInfoAssert/foundation-linux-x86_64/ directory exists but there is no swift directory inside of it. thinking this was a stale configuration issue, i tried adding --reconfigure but got the same result. same with --skip-test-early-swift-driver.

i then tried deleting the entire foundation-linux-x86_64 parent directory, but that gave me an:

ninja: error: '/swift/swift-project/build/Ninja-RelWithDebInfoAssert/libdispatch-linux-x86_64/libswiftDispatch.so', needed by 'lib/libFoundation.so', missing and no known rule to make it

this wasn’t the end of the world, i just cleared the entire build directory and started another attempt. the first time i got another crash just like the one here. the second time i got:

-- The C compiler identification is Clang 17.0.0
-- The Swift compiler identification is Apple 5.11
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /swift/swift-project/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working Swift compiler: /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/bin/swiftc
-- Check for working Swift compiler: /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/bin/swiftc - works

CMake Error at CMakeLists.txt:47 (find_package):
  Could not find a package configuration file provided by "dispatch" with any
  of the following names:

    dispatchConfig.cmake
    dispatch-config.cmake

  Add the installation prefix of "dispatch" to CMAKE_PREFIX_PATH or set
  "dispatch_DIR" to a directory containing one of the above files.  If
  "dispatch" provides a separate development package or SDK, be sure it has
  been installed.

i don’t know why i can’t get it to build consistently. (it has failed three times in a row.) here is the exact command i was running:

$ utils/build-script --release-debuginfo -T --xctest --skip-test-swift --reconfigure --skip-test-early-swift-driver --bootstrapping=off

Finagolfin · December 24, 2023, 2:18am

This is a known issue with the current build, that if you try to start the build over, the early swift-driver build picks up the newly-built foundation instead and fails.

I work around it in two ways:

After the first time building the compiler, pass in --skip-early-swift-driver for subsequent builds, as the early swift-driver is already copied next to the compiler and there is no need to rebuild it each time.
Or similarly, delete foundation-linux-x86_64/ every time you build it and need to start over.

Does that libdispatch file and directory exist after that error? If not, maybe you forgot to "remove that libdispatch patch for utils/build-script that I suggested earlier, as you want to build libdispatch now," ie enable this override I mentioned to you earlier in the week again.

Probably that same libdispatch error from removing the non-Darwin override?

Any particular reason you are passing this in? If the hosttools bootstrap worked before and you're hoping this flag will help with these Swift corelibs issues, it won't. You are better off switching it back to hosttools.

art-divin · January 4, 2024, 3:45pm

Hello @Finagolfin ,

thank you for this thread and thanks @taylorswift for raising the issue in the first place!

For the last couple of days I have been trying to reproduce this crash whilst having a debugger attached.

I have compiled the compiler using utils/build-script --release-debuginfo --skip-test-early-swift-driver --skip-test-swift -T - at least this was my last attempt after cleaning the build folder. Before that I have tried all the other configurations for utils/build-script.
Next, I have tried to use VSCode to run swift test where the subject is located (working directory), but SPM would stuck and I would not be able to continue to compilation process of the Sourcery, for whatever reason.

Good thing is that I was able to reproduce the crash in CLion instead! I need to update HowToGuides/GettingStarted.md for CLion configuration, the one written there is insufficient (i.e. does not mention that clang and clang++ must be set as compilers in CLion environment settings etc.).

Having crash reproduced, the next step is to attach to a debuggable libSwiftCore.so, but this is really unclear how to do for a beginner like me. Maybe if I have a source code and a debuggable compiled library, I would magically be able to step over inside of the IDE if all paths are set correctly, this is something for me to learn still.

BUT running the utils/build-toolchain according to the DevelopmentTips.md to build a debuggable libSwiftCore brings the error in the OP mentioned by @taylorswift -

[81/81][100%][2.511s] Linking Swift shared library libswiftDispatch.so
FAILED: libswiftDispatch.so

The crash report that I have got from reproducing locally contains libSwiftCore path pointing to my local installation of Swift toolchain rather than the one compiled in /build/Ninja-RelWithDebInfoAssert subfolder. And I am unsure how to make IDE link against libSwiftCore built along with the compiler, rather than the one installed in the system.

I hope you could guide me how to debug this crash so I can, eventually, resolve it and enable 100% Linux support in Sourcery

taylorswift · January 4, 2024, 9:38pm

building libdispatch has never worked (for me), so if i reenable libdispatch, the build once again fails with the same error: fatalError as in this attempt.

the exact output, if it helps

/swift/swift-project/swift$ /swift/swift-project/build/cmake-linux-x86_64/bin/cmake --build /swift/swift-project/build/Ninja-RelWithDebInfoAssert/libdispatch-linux-x86_64 -- -j12 all
[1/1] Linking Swift shared library libswiftDispatch.so
FAILED: libswiftDispatch.so src/swift/CMakeFiles/swiftDispatch.dir/Block.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Data.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Dispatch.swift.o src/swift/CMakeFiles/swiftDispatch.dir/IO.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Private.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Queue.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Source.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Time.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Wrapper.swift.o src/swift/swift/Dispatch.swiftmodule 
: && /swift/swift-project/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/bin/swiftc -output-file-map src/swift/CMakeFiles/swiftDispatch.dir/RelWithDebInfo/output-file-map.json -incremental -j 12 -emit-library -o libswiftDispatch.so -module-name Dispatch -module-link-name swiftDispatch -emit-module -emit-module-path src/swift/swift/Dispatch.swiftmodule -emit-dependencies -DswiftDispatch_EXPORTS -module-cache-path "/swift/swift-project/build/Ninja-RelWithDebInfoAssert/libdispatch-linux-x86_64/module-cache" -O -g -Xcc -fblocks -Xcc -fmodule-map-file=/swift/swift-project/swift-corelibs-libdispatch/dispatch/module.modulemap -Xcc -I/swift/swift-project/swift-corelibs-libdispatch -Xcc -I/swift/swift-project/swift-corelibs-libdispatch/src/swift/shims -vfsoverlay /swift/swift-project/build/Ninja-RelWithDebInfoAssert/libdispatch-linux-x86_64/dispatch-vfs-overlay.yaml -I /swift/swift-project/swift-corelibs-libdispatch/src/BlocksRuntime -I /swift/swift-project/build/Ninja-RelWithDebInfoAssert/libdispatch-linux-x86_64 -I /swift/swift-project/swift-corelibs-libdispatch -I /swift/swift-project/swift-corelibs-libdispatch/src -I /swift/swift-project/build/Ninja-RelWithDebInfoAssert/libdispatch-linux-x86_64/src /swift/swift-project/swift-corelibs-libdispatch/src/swift/Block.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Data.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Dispatch.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/IO.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Private.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Queue.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Source.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Time.swift /swift/swift-project/swift-corelibs-libdispatch/src/swift/Wrapper.swift -no-toolchain-stdlib-rpath -Xlinker -soname -Xlinker libswiftDispatch.so  -L /swift/swift-project/build/Ninja-RelWithDebInfoAssert/libdispatch-linux-x86_64/src/swift  -L /swift/swift-project/build/Ninja-RelWithDebInfoAssert/libdispatch-linux-x86_64  -L /swift/swift-project/build/Ninja-RelWithDebInfoAssert/libdispatch-linux-x86_64   -L /usr/lib/gcc/x86_64-amazon-linux/11 -Xlinker -rpath -Xlinker /swift/swift-project/build/Ninja-RelWithDebInfoAssert/libdispatch-linux-x86_64:  src/swift/libDispatchStubs.a  libdispatch.so  libBlocksRuntime.so  -lstdc++  -lm  -lgcc_s  -lgcc  -lc  -lgcc_s  -lgcc && :
/swift/swift-project/swift-corelibs-libdispatch/src/swift/Dispatch.swift:99:4: warning: default will never be executed
                        default: return nil
                        ^
error: fatalError

art-divin · January 5, 2024, 1:24pm

I see, thanks!

My current progress is by guessing I have built somewhat a debuggable version of libSwiftCore running utils/build-script --release-debuginfo --skip-test-early-swift-driver --skip-test-swift --debug-swift-stdlib --bootstrapping=hosttools --skip-test-swiftdocc (had to disable swiftdocc because it was failing during Generating ... phase).

Now I do have libSwiftCore that I needed in the first place to debug the crash, but the crash itself is linked against system (host) Swift toolchain. Now my task is to link against the debuggable version of libSwiftCore.so somehow tweaking the build script

Registers:
x0 0x0000ffffe9355150  02 00 00 00 00 00 00 00 e0 52 35 e9 ff ff 00 00  ········àR5éÿÿ··

x21 0x0000ffffb8cec45c  48 00 00 00 20 cd d7 ee aa aa 00 00 11 61 00 a0  H··· Í×îªª···a· 
x22 0x0000ffffb8cec458  01 00 00 00 48 00 00 00 20 cd d7 ee aa aa 00 00  ····H··· Í×îªª··
x23 0x0000ffffe9355280  00 00 00 00 00 00 00 00 8c ab aa b8 ff ff 00 00  ·········«ª¸ÿÿ··
x24 0x0000ffffb8be1b68  00 02 00 00 00 00 00 00 a8 bf c0 b8 ff ff 00 00  ········¨¿À¸ÿÿ··
x25 0x0000aaaaeed96a70  02 02 00 00 00 00 00 00 04 d4 c0 b8 ff ff 00 00  ·········ÔÀ¸ÿÿ··
x26 0x0000aaaaeed969f8  01 03 00 00 00 00 00 00 02 00 00 00 00 00 00 00  ················
x27 0x0000ffffb8be3188  00 02 00 00 00 00 00 00 68 ed c0 b8 ff ff 00 00  ········híÀ¸ÿÿ··
x28 0x0000ffffe93553a0  40 ef be b8 ff ff 00 00 02 00 00 00 00 00 00 00  @ï¾¸ÿÿ··········
fp 0x0000ffffe93551e0  a0 52 35 e9 ff ff 00 00 28 fb a9 b8 ff ff 00 00   R5éÿÿ··(û©¸ÿÿ··
lr 0x0000ffffb8a9fe58  00 01 00 b4 08 00 40 f9 e1 03 16 aa c8 2a 00 f9  ···´··@ùá··ªÈ*·ù
sp 0x0000ffffe9354fc0  281474594328512
pc 0x0000ffffb8ab10b8  fd 7b 02 a9 fc 6f 03 a9 fa 67 04 a9 f8 5f 05 a9  ý{·©üo·©úg·©ø_·©
Images (54 omitted):
0x0000aaaabb830000–0x0000aaaac0459a38 <no build ID>                            SourceryPackageTests.xctest /home/tart/Documents/projects/Sourcery-master/.build/aarch64-unknown-linux-gnu/debug/SourceryPackageTests.xctest
0x0000ffffb7320000–0x0000ffffb74a8214 b2eb52eb317bf12f124bc94b4e32643c450e6c9a libc.so.6                   /usr/lib/aarch64-linux-gnu/libc.so.6
0x0000ffffb7860000–0x0000ffffb78b9a10 <no build ID>                            libXCTest.so                /home/tart/.local/share/swiftly/toolchains/5.9.2/usr/lib/swift/linux/libXCTest.so
0x0000ffffb7ab0000–0x0000ffffb81afd90 <no build ID>                            libFoundation.so            /home/tart/.local/share/swiftly/toolchains/5.9.2/usr/lib/swift/linux/libFoundation.so
0x0000ffffb86d0000–0x0000ffffb8bbd230 <no build ID>                            libswiftCore.so             /home/tart/.local/share/swiftly/toolchains/5.9.2/usr/lib/swift/linux/libswiftCore.so

Backtrace took 8.77s
error: Exited with signal code 11

Finagolfin · January 6, 2024, 3:07am

A lot of redundant flags here, just utils/build-script --release-debuginfo will mostly do the same.

What distro are you running on? @taylorswift says they're running on amazonlinux:2023, which is not currently officially supported.

This is the crash report from building libswiftDispatch.so or running the Sourcery tests?

If you're unable to build libswiftDispatch.so, that means you don't have a full freshly-built toolchain, so that would mean mixing freshly-built components with a prebuilt toolchain. You have not specified how you are doing this.

Instead of building with --release-debuginfo, which builds everything including the compiler with debug info, you may want to build --release --debug-swift-stdlib instead, since your problem is in the stdlib.

You have never specified exactly what build-script command gives you that error: please detail that and list any patches you have applied.

It is possible you have found a genuine build issue, as I notice that most CI presets run with --release --assertions instead, though I do see some that set --release-debuginfo. Try swapping in the former and see if that makes a difference.

This command freshly builds only the compiler and stdlib, not the corelibs like Foundation or the SwiftPM build tool, so you won't be able to build Sourcery with these components alone. You could try swapping the freshly-built debug stdlib into a prebuilt release toolchain, but you'd have to make sure the versions matched, eg you can't swap a trunk debug stdlib into a 5.9 release toolchain.

art-divin · January 6, 2024, 6:15am

I am running on Ubuntu 22.04.3 LTS codename Jammy. This is run under VM on M1, meaning architecture is aarch64. Is there a list of supported linux OSes somewhere, if you will?

It is a crash from Sourcery. It is connected to this topic - i.e. unable to build libswiftDispatch.so, only in terms of me not being able to build a debuggable toolchain (using utils/build-toolchain) to point to it when compiling Swift compiler, so that I'd be able to debug NSCoder-related routine, why it crashes.

Did that, still the crash would point to libSwiftCore.so installed from swift.org rather the one compiled and located in Ninja-RelWithDebInfoAssert+stdlib-DebugAssert. This is why I am trying to build the toolchain.

Oh, wow, thanks for the insight! In order to build SPM and foundation, I'd set these flags as well - --debug-foundation - in order to be able to use only what's under build/Ninja... to reproduce the crash, right?

I am not clear about why exactly when I run Build/Ninja-RelWithDebInfoAssert+stdlib-DebugAssert/swift/bin/swift test in Sourcery working directory, crashing links against libSwiftCore which is located in a system-wide-installed toolchain (as far as I tried, there's no way to build Swift compiler without having system-wide installed Swift toolchain).
In fact, I have this many libSwiftCore.so under Ninja... build folder, which is quite confusing

Finagolfin · January 7, 2024, 4:09pm

The README lists most of the officially supported OS's, with only Windows missing for some reason. If you are able to reproduce @taylorswift's crash on Ubuntu, that implies this is a genuine build issue.

I am not sure how that runs for you at all, as you can't have a freshly-built swift-test without building libswiftDispatch.so first. Maybe the new swift-driver just invokes the system swift-test if it can't find one next to Build/Ninja-RelWithDebInfoAssert+stdlib-DebugAssert/swift/bin/swift, which would explain why it's still using the non-debug stdlib.

What I suggest to you is to simply build the debug stdlib for the prebuilt toolchain you're using, by first checking out that source tag then running utils/build-script --release-debuginfo --skip-build-cmark --build-llvm=0 --build-swift-tools=0 to only build the debug stdlib. Finally, copy over that debug stdlib into your prebuilt toolchain with something like cp Build/Ninja-RelWithDebInfoAssert+stdlib-DebugAssert/swift/lib/swift/linux/libswift*.so swift-5.9.2-RELEASE/usr/lib/swift/linux/, use it to build the Sourcery tests with cd Sourcery; ../swift-5.9.2-RELEASE/usr/bin/swift build --build-tests, then run the test runner in the debugger to get a proper backtrace and debug it, ../swift-5.9.2-RELEASE/usr/bin/lldb .build/x86_64-unknown-linux-gnu/debug/SourceryPackageTests.xctest.

Given the build issue you're seeing, this may be the best way for you to debug your Sourcery issue.

taylorswift · January 13, 2024, 2:18am

i was using the last full invocation you suggested, which was:

i no longer have any patches applied, as i am now attempting to build libdispatch.

the build usually fails for one of two reasons:

the swift-driver segfaults, as described here, or
building libswiftDispatch.so fails with the fatalError, as described here.

the second failure of course didn’t occur when i manually disabled libdispatch as you suggested. but as i understand it, i need to build libdispatch in order to build the toolchain.

i have tried swapping out the original command with:

$ utils/build-script --release --assertions -T --xctest --skip-test-swift

but i get the exact same results.

Finagolfin · January 13, 2024, 5:10am

Huh, this suggests the problem is in your environment, as that works on the linux CI, at least when the trunk compiler is built with Swift 5.8.

Can you try running the failing release-debuginfo compilation command through the debugger to get a backtrace?

lldb /swift/swift-project/build/buildbot_linux/swift-linux-x86_64/bin/swift-frontend -- -frontend -c /swift/swift-project/..

That may just exit, so you might have to set a breakpoint on fatalError() or some other method like that.