Swift 6.0.1 for Gentoo on GURU

Community Showcase
,
1

Although it's not quite live yet at the time of posting (merged to dev but not yet promoted to master), I'm happy to announce the upcoming availability of Swift 6.0.1 for Gentoo via GURU. As with Swift 5.10.1 for Gentoo on GURU, this is an unofficial package not supported by either the Swift project or Gentoo developers.

tl;dr:

If you haven't previously installed Swift via GURU, you can install Swift 6.0.1 by:

  1. emerge --ask app-eselect/eselect-repository to enable adding additional ebuild repositories using eselect
  2. eselect repository enable guru to enable the GURU package repo
  3. emerge --sync guru
  4. Add dev/lang-swift ~amd64 to /etc/portage/package.accept_keywords
    • Because GURU packages are considered unstable, you need to explicitly accept packages via the "accept keywords" system. An alternative to doing this manually is running the following command with --autounmask --autounmask-write to accept the necessary keyword changes, then run etc-update to write the changes to the right file (but doing it manually makes it easier to permanently whitelist all Swift versions)
  5. emerge --ask dev-lang/swift

If you currently have the swift-5.10.1 package installed, you'll need to:

  1. emerge -uUD dev-lang/swift to do a deep update to the latest version
  2. eselect swift update afterwards to ensure /usr/bin/swift & co are pointing to the latest Swift version (just this once; it's otherwise automatic)

I'll update this thread when these updates should be generally available; if you get the opportunity to update and run into issues, please don't hesitate to let me know! The easiest way to capture as much info as possible during the installation would be to

emerge --ask dev-lang/swift 2>&1 | tee emerge-swift.log

or

emerge -uUD dev-lang/swift 2>&1 | tee emerge-swift.log

so we can capture standard error.

Details

Background: RFC: Requiring Swift to build the Swift toolchain

The Swift 6 compiler and toolchain require an existing Swift compiler in order to build; for a source-based distribution like Gentoo, this means that you need to bootstrap using an earlier compiler that can build without Swift. The Swift 5.10.x line is intended to be the last Swift release capable of being built this way, with 5.10.1 being the latest compiler release in this line at the time of writing.

This means that it must be possible to have multiple Swift compilers installed simultaneously, which required some changes to how I wrote the existing Swift 5.10.1 package:

  1. I had to change package slotting in order to allow simultaneous compilers to be installed. The existing package defaulted to slot 0, and if the new Swift 6.0.1 package also used slot 0, it would be impossible for them to coexist. Slot numbers don't have an inherent meaning, and I could have picked slot 1 for Swift 6.x.y, slot 2 for Swift 7.x.y, etc. Instead, I opted to use sub-slots to allow for multiple compilers in the same line to also be installed: Swift 5.10.1 now uses slot 5/10, Swift 6.0.1 uses slot 6/0, and so on. This scheme will allow you to have one compiler version installed per major.minor release, if you desire
    • The benefit of going through the pain of changing the slot number for 5.10.1 right now is that it's trivial to keep the mapping in the future, and it enables requesting specific Swift versions via slots. (emerge doesn't otherwise support depending on specific version ranges, but by requesting, e.g., slot 5/10, you'll get a compiler in the 5.10.x range)
  2. Because you'll now likely have at least two compilers installed at a time, I also wrote app-eselect/eselect-swift to help manage the "currently active" Swift compiler at /usr/bin/swift with eselect swift set. (eselect swift list will list available Swift versions, eselect swift show will show the currently-set compiler, eselect swift update will switch to the newest-installed version, etc.)
    • This works because the updated packages write versioned symlinks for swift, swiftc, and sourcekit-lsp to /usr/bin (e.g., /usr/bin/swift-5.10.1, /usr/bin/swift-6.0.1) which the eselect module can then link to as /usr/bin/swift, /usr/bin/swiftc, etc.) The mechanism is written flexibly so if future Swift packages wanted to symlink more versioned tools into /usr/bin, they'll get picked up automatically and things should Just Work™

Unfortunately, these two changes required a significant update to the swift-5.10.1 package, so I've also released swift-5.10.1-r1 which will require a rebuild:

  • It's not possible to update a package's slot without reinstalling
  • The original Swift package wasn't written with the eselect symlinking scheme in mind, so unconditionally symlinked to /usr/bin/swift, and did not write versioned symlinks
    • This also means that eselect would technically be messing with files installed by a package, which is iffy
  • There were a few unrelated fixes I wanted to make to the package that wouldn't have been picked up if you already had it installed, such as compiling without assertions (:see_no_evil:)

Because these first two changes are pretty significant, I've made swift-5.10.1-r1 and swift-6.0.1 intentionally incompatible with swift-5.10.1:0 (the original package in slot 0) — I effectively want to phase out the slot 0 version of the original package, and offer folks the benefits of the new slotting scheme (without having to worry about compatibility with a slot 0 in the future).

If you just try to emerge --ask dev-lang/swift, emerge will complain that the new packages are incompatible with what you've got installed, and you'll need to emerge --depclean dev-lang/swift before you can install the newer versions; if you instead do a deep update with emerge -uUD, emerge will automatically uninstall the existing package.

  • Annoying nit: from my experience, emerge will uninstall the existing package after installing the newer packages. I wrote these new packages to automatically eselect swift update after installation so you're typically pointing at the latest Swift version after you update. Since the slot 0 package technically owns /usr/bin/swift, /usr/bin/swiftc, and /usr/bin/sourcekit-lsp, uninstalling the package will likely remove those symlinks. Just run eselect swift update again if this happens and you should have the latest links. This shouldn't be necessary again in the future!

Sorry for the hassle of having to reinstall, but hopefully, future updates will be a heck of a lot smoother.

7 Likes
2

Swift 6.0.1 is now generally available via GURU — please let me know if you run into any issues building and installing, and I'll do my best to help!

2 Likes
3

The REPL seems to be broken in Swift 6.0.1. I don't know if this is a general Linux thing or if it's specific to the ebuild:

Welcome to Swift version 6.0.1 (swift-6.0.1-RELEASE).
Type :help for assistance.
  1> var n = 1;
Optimizer/Verifier.swift:22: Fatal error: instruction   %5 = struct $Int (%4 : $Builtin.Int64)          // user: %6
should conform to ForwardingInstruction
LLDB diagnostics will be written to /tmp/diagnostics-2db35e
Please include the directory content when filing a bug report
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
 #0 0x00005585fc979008 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/usr/lib64/swift-6.0.1/usr/bin/lldb+0x30008)
 #1 0x00005585fc97739e llvm::sys::RunSignalHandlers() (/usr/lib64/swift-6.0.1/usr/bin/lldb+0x2e39e)
 #2 0x00005585fc979938 SignalHandler(int) Signals.cpp:0:0
 #3 0x00007f1fe5254cb0 (/lib64/libc.so.6+0x3bcb0)
 #4 0x00007f1fe4d3d13f $ss17_assertionFailure__4file4line5flagss5NeverOs12StaticStringV_SSAHSus6UInt32VtF (/usr/lib64/swift-6.0.1/usr/bin/../lib/../lib/swift/linux/libswiftCore.so+0x33d13f)
 #5 0x00007f1fe735352f $s3SIL8FunctionC9OptimizerE6verifyyyAD0B11PassContextVF Optimizer.o:0:0
 #6 0x00007f1fe735355a $s9Optimizer16registerVerifieryyFySo18BridgedPassContextV_So0D8FunctionVtcfU_To Optimizer.o:0:0
 #7 0x00007f1fe7769eec swift::SILPassManager::runSwiftFunctionVerification(swift::SILFunction*) PassManager.cpp:0:0
 #8 0x00007f1fe795eafb (anonymous namespace)::OwnershipModelEliminator::run() OwnershipModelEliminator.cpp:0:0
 #9 0x00007f1fe776931e swift::SILPassManager::runPassOnFunction(unsigned int, swift::SILFunction*) PassManager.cpp:0:0
#10 0x00007f1fe776a33f swift::SILPassManager::runFunctionPasses(unsigned int, unsigned int) PassManager.cpp:0:0
#11 0x00007f1fe776befb swift::SILPassManager::execute() PassManager.cpp:0:0
#12 0x00007f1fe7767778 swift::SILPassManager::executePassPipelinePlan(swift::SILPassPipelinePlan const&) PassManager.cpp:0:0
#13 0x00007f1fe776772d swift::ExecuteSILPipelineRequest::evaluate(swift::Evaluator&, swift::SILPipelineExecutionDescriptor) const PassManager.cpp:0:0
#14 0x00007f1fe7a2b27a swift::SimpleRequest<swift::ExecuteSILPipelineRequest, std::tuple<> (swift::SILPipelineExecutionDescriptor), (swift::RequestFlags)1>::evaluateRequest(swift::ExecuteSILPipelineRequest const&, swift::Evaluator&) SILOptimizerRequests.cpp:0:0
#15 0x00007f1fe7785b4d swift::ExecuteSILPipelineRequest::OutputType swift::Evaluator::getResultUncached<swift::ExecuteSILPipelineRequest, swift::ExecuteSILPipelineRequest::OutputType swift::evaluateOrFatal<swift::ExecuteSILPipelineRequest>(swift::Evaluator&, swift::ExecuteSILPipelineRequest)::'lambda'()>(swift::ExecuteSILPipelineRequest const&, swift::ExecuteSILPipelineRequest::OutputType swift::evaluateOrFatal<swift::ExecuteSILPipelineRequest>(swift::Evaluator&, swift::ExecuteSILPipelineRequest)::'lambda'()) PassManager.cpp:0:0
#16 0x00007f1fe776788c swift::executePassPipelinePlan(swift::SILModule*, swift::SILPassPipelinePlan const&, bool, swift::irgen::IRGenModule*) PassManager.cpp:0:0
#17 0x00007f1fe7a0ee0e swift::runSILLoweringPasses(swift::SILModule&) Passes.cpp:0:0
#18 0x00007f1fe6dfeb54 lldb_private::SwiftExpressionParser::Parse(lldb_private::DiagnosticManager&, unsigned int, unsigned int) SwiftExpressionParser.cpp:0:0
#19 0x00007f1fe6dddb29 lldb_private::SwiftUserExpression::GetTextAndSetExpressionParser(lldb_private::DiagnosticManager&, std::unique_ptr<lldb_private::SwiftExpressionSourceCode, std::default_delete<lldb_private::SwiftExpressionSourceCode>>&, lldb_private::ExecutionContext&, lldb_private::ExecutionContextScope*) SwiftUserExpression.cpp:0:0
#20 0x00007f1fe6ddea40 lldb_private::SwiftUserExpression::Parse(lldb_private::DiagnosticManager&, lldb_private::ExecutionContext&, lldb_private::ExecutionPolicy, bool, bool) SwiftUserExpression.cpp:0:0
#21 0x00007f1fe678a615 lldb_private::UserExpression::Evaluate(lldb_private::ExecutionContext&, lldb_private::EvaluateExpressionOptions const&, llvm::StringRef, llvm::StringRef, std::shared_ptr<lldb_private::ValueObject>&, lldb_private::Status&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, lldb_private::ValueObject*) UserExpression.cpp:0:0
#22 0x00007f1fe678716b lldb_private::REPL::IOHandlerInputComplete(lldb_private::IOHandler&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>&) REPL.cpp:0:0
#23 0x00007f1fe66fd4b9 lldb_private::IOHandlerEditline::Run() IOHandler.cpp:0:0
#24 0x00007f1fe66db3f6 lldb_private::Debugger::RunIOHandlers() Debugger.cpp:0:0
#25 0x00007f1fe66e3962 std::_Function_handler<void* (), lldb_private::Debugger::StartIOHandlerThread()::$_5>::_M_invoke(std::_Any_data const&) Debugger.cpp:0:0
#26 0x00007f1fe67943b9 lldb_private::HostNativeThreadBase::ThreadCreateTrampoline(void*) HostNativeThreadBase.cpp:0:0
#27 0x00007f1fe52a1701 start_thread /var/tmp/portage/portage/sys-libs/glibc-2.39-r6/work/glibc-2.39/nptl/pthread_create.c:447:8
#28 0x00007f1fe531b71c clone3 /var/tmp/portage/portage/sys-libs/glibc-2.39-r6/work/glibc-2.39/misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:80:0
zsh: illegal hardware instruction  /usr/lib64/swift-6.0.1/usr/bin/swift repl

The diagnostics file (/tmp/diagnostics-2db35e/diagnostics.log) is empty.

Compiling some simple programs with swiftc worked fine.

The REPL from swift-5.10.1-r1.ebuild also works fine.

my USE flags:

 % equery u dev-lang/swift
[ Legend : U - final flag setting for installation]
[        : I - package is installed with flag     ]
[ Colors : set, unset                             ]
 * Found these USE flags for dev-lang/swift-6.0.1:
 U I
 - - llvm_slot_15                    : Use LLVM 15.
 - - llvm_slot_16                    : Use LLVM 16.
 - - llvm_slot_17                    : Use LLVM 17.
 + + llvm_slot_18                    : Use LLVM 18.
 - - python_single_target_python3_10 : Build for Python 3.10 only
 - - python_single_target_python3_11 : Build for Python 3.11 only
 + + python_single_target_python3_12 : Build for Python 3.12 only
 - - python_single_target_python3_13 : Build for Python 3.13 only

The output of emerge --info:

Portage 3.0.65 (python 3.12.6-final-0, default/linux/amd64/23.0/split-usr, gcc-13, glibc-2.39-r6, 6.4.3-gentoo x86_64)
=================================================================
System uname: Linux-6.4.3-gentoo-x86_64-AMD_Opteron-TM-_Processor_6276-with-glibc2.39
KiB Mem:   528360740 total, 112319196 free
KiB Swap:   67108860 total,  67010136 free
Timestamp of repository gentoo: Mon, 14 Oct 2024 10:30:00 +0000
Head commit of repository gentoo: 81a3b93870aa79ebb802447b2fcb36d125eb2e6b
Timestamp of repository guru: Mon, 14 Oct 2024 09:33:24 +0000
Head commit of repository guru: 9ea962c30c2d4d5567182df75247581e22023f3f

Timestamp of repository science: Thu, 10 Oct 2024 18:19:11 +0000
Head commit of repository science: 325d4b8ee78e56c1aefbcb42c118e56b11e41c91

Timestamp of repository haskell: Sun, 13 Oct 2024 03:03:16 +0000
Head commit of repository haskell: 822a24acb3b8b2cd1eea2ea7a5a2d81da20b3890

sh bash 5.2_p37
ld GNU ld (Gentoo 2.42 p6) 2.42.0
ccache version 4.10.1 [disabled]
app-misc/pax-utils:        1.3.7::gentoo
app-shells/bash:           5.2_p37::gentoo
dev-build/autoconf:        2.71-r7::gentoo
dev-build/automake:        1.16.5-r2::gentoo
dev-build/cmake:           3.30.2::gentoo
dev-build/libtool:         2.4.7-r4::gentoo
dev-build/make:            4.4.1-r1::gentoo
dev-build/meson:           1.5.1::gentoo
dev-java/java-config:      2.3.4::gentoo
dev-lang/perl:             5.40.0::gentoo
dev-lang/python:           2.7.18_p16-r2::gentoo, 3.10.15_p1::gentoo, 3.11.10_p1::gentoo, 3.12.6_p4::gentoo
dev-lang/rust:             1.80.1::gentoo
dev-util/ccache:           4.10.1::gentoo
sys-apps/baselayout:       2.15::gentoo
sys-apps/openrc:           0.54.2::gentoo
sys-apps/sandbox:          2.39::gentoo
sys-devel/binutils:        2.42-r2::gentoo
sys-devel/binutils-config: 5.5.2::gentoo
sys-devel/clang:           16.0.6::gentoo, 17.0.6::gentoo, 18.1.8::gentoo
sys-devel/gcc:             12.4.0::gentoo, 13.3.1_p20240614::gentoo
sys-devel/gcc-config:      2.11::gentoo
sys-devel/lld:             18.1.8::gentoo
sys-devel/llvm:            16.0.6::gentoo, 17.0.6::gentoo, 18.1.8-r1::gentoo
sys-kernel/linux-headers:  6.6-r1::gentoo (virtual/os-headers)
sys-libs/glibc:            2.39-r6::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: rsync
    sync-uri: rsync://rsync.gentoo.org/gentoo-portage
    priority: -1000
    volatile: True
    sync-rsync-extra-opts: 
    sync-rsync-verify-jobs: 1
    sync-rsync-verify-metamanifest: yes
    sync-rsync-verify-max-age: 24

HMC-CS
    location: /mnt/local/portage
    masters: gentoo
    priority: 0
    volatile: True

guru
    location: /var/db/repos/guru
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/guru.git
    masters: gentoo
    volatile: False

science
    location: /var/db/repos/science
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/science.git
    masters: gentoo
    volatile: False

haskell
    location: /var/db/repos/haskell
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/haskell.git
    masters: gentoo
    priority: 50
    volatile: False

ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /usr/share/maven-bin-3.9/conf"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php8.2/ext-active/ /etc/php/cgi-php8.2/ext-active/ /etc/php/cli-php8.2/ext-active/ /etc/php/fpm-php8.2/ext-active/ /etc/php/phpdbg-php8.2/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-march=native -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--jobs=32 --load-average=48.0 --autounmask"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync merge-wait multilib-strict network-sandbox news parallel-fetch pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="https://gentoo.osuosl.org/ https://mirrors.rit.edu/gentoo/"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,-z,pack-relative-relocs"
LEX="flex"
MAKEOPTS="-j56 -l 80.0"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp/portage"
SHELL="/bin/zsh"
USE="acl amd64 bzip2 cet crypt eselect-ldso gdbm iconv ipv6 libtirpc multilib ncurses nls openmp pam pcre readline seccomp split-usr ssl test-rust unicode xattr zlib" ABI_X86="64" ADA_TARGET="gcc_12" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx fma4 mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 xop" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 ntrip navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" GUILE_SINGLE_TARGET="3-0" GUILE_TARGETS="3-0" INPUT_DEVICES="mouse keyboard evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-2" POSTGRES_TARGETS="postgres16" PYTHON_SINGLE_TARGET="python3_12" PYTHON_TARGETS="python3_12 python3_11" RUBY_TARGETS="ruby32" VIDEO_CARDS="virgl" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account"
Unset:  ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
4

Thanks for this! I can confirm I'm seeing the same thing — I didn't catch it during testing because I don't use the repl much. The built Swift 6.0.1 release for Fedora 39 doesn't exhibit this behavior for me on the same system, so it looks like something about how this was built.

I'll try to dig in to see if I can figure anything out.

1 Like
5

@Erik_Eckstein, it looks like you added the SIL verification that's failing in the repl here back in SwiftCompilerSources: refactor and verify conformances to ForwardingInstruction by eeckstein · Pull Request #71723 · swiftlang/swift · GitHub, so I figure you might be someone to check in with: any idea why SIL verification could fail this way, and only in the Swift repl? (Any expression executed in the repl dies with the same error: <instruction> should conform to ForwardingInstruction, whereas the same exact code builds fine outside of the repl)

I'm fairly unfamiliar with the SIL layer, so any pointers on how I could approach tracking down the cause would be super helpful!

For reference, we're building Swift 6.0.1 from source here (with an available Swift 5.10.1 compiler), and it appears that something about the compilation environment itself leads to the failure here: pulling in a prebuilt Swift 6.0.1 compiler (built for Fedora 39) works just fine on the same machine. (So it's at least not purely a runtime issue.)

In case it helps, here's the build preset I'm using for compilation:

[preset: gentoo]
mixin-preset=buildbot_linux,no_assertions,no_test
build-ninja=0
extra-cmake-options=
       -DLLVM_USE_LINKER=lld
       -DBUILD_TESTING:BOOL=NO
       -DSWIFT_INCLUDE_TESTS:BOOL=NO
       -DSWIFT_INCLUDE_TEST_BINARIES:BOOL=NO
       -DCOMPILER_RT_BUILD_ORC:BOOL=NO
       -DPython3_FIND_UNVERSIONED_NAMES=FIRST
libicu=0
llvm-targets-to-build=host
skip-build-curl
skip-build-zlib

Happy to offer any additional details that might help!

6

To be clearer, this also reproduces in the built lldb directly, it's just easier to hit in the repl as a wrapper for lldb:

$ echo "while true {}" > Test.swift
$ swiftc Test.swift
$ /usr/lib64/swift-6.0.1/usr/bin/lldb -- ./Test
$ /usr/lib64/swift-6.0.1/usr/bin/lldb -- ./Test
(lldb) target create "./Test"
Current executable set to '/home/itai/Desktop/Test' (x86_64).
(lldb) run
Process 12301 launched: '/home/itai/Desktop/Test' (x86_64)
(^C)
Process 12301 stopped
* thread #1, name = 'Test', stop reason = signal SIGSTOP
    frame #0: 0x0000555555555a86 Test`main + 6
Test`main:
->  0x555555555a86 <+6>: jmp    0x555555555a86 ; <+6>

Test`_init:
    0x555555555a88 <+0>: endbr64
    0x555555555a8c <+4>: subq   $0x8, %rsp
    0x555555555a90 <+8>: movq   0x1279(%rip), %rax
Target 0: (Test) stopped.
(lldb) settings set target.language swift
(lldb) p "Hi"
Optimizer/Verifier.swift:22: Fatal error: instruction   %2 = struct_extract %0 : $UnsafeMutablePointer<Any>, #UnsafeMutablePointer._rawValue // user: %4
should conform to ForwardingInstruction
LLDB diagnostics will be written to /tmp/diagnostics-5eca5d
Please include the directory content when filing a bug report
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.      Program arguments: /usr/lib64/swift-6.0.1/usr/bin/lldb -- ./Test
 #0 0x000055b136a2f028 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/usr/lib64/swift-6.0.1/usr/bin/lldb+0x30028)
 #1 0x000055b136a2d3be llvm::sys::RunSignalHandlers() (/usr/lib64/swift-6.0.1/usr/bin/lldb+0x2e3be)
 #2 0x000055b136a2f958 SignalHandler(int) Signals.cpp:0:0
 #3 0x00007f19ae46f960 (/usr/lib64/libc.so.6+0x3c960)
 #4 0x00007f19adf3d16f $ss17_assertionFailure__4file4line5flagss5NeverOs12StaticStringV_SSAHSus6UInt32VtF (/usr/lib64/swift-6.0.1/usr/bin/../lib/../lib/swift/linux/libswiftCore.so+0x33d16f)
 #5 0x00007f19b055356f $s3SIL8FunctionC9OptimizerE6verifyyyAD0B11PassContextVF Optimizer.o:0:0
 #6 0x00007f19b055359a $s9Optimizer16registerVerifieryyFySo18BridgedPassContextV_So0D8FunctionVtcfU_To Optimizer.o:0:0
 #7 0x00007f19b0969f2c swift::SILPassManager::runSwiftFunctionVerification(swift::SILFunction*) PassManager.cpp:0:0
 #8 0x00007f19b0b5eb3b (anonymous namespace)::OwnershipModelEliminator::run() OwnershipModelEliminator.cpp:0:0
 #9 0x00007f19b096935e swift::SILPassManager::runPassOnFunction(unsigned int, swift::SILFunction*) PassManager.cpp:0:0
#10 0x00007f19b096a37f swift::SILPassManager::runFunctionPasses(unsigned int, unsigned int) PassManager.cpp:0:0
#11 0x00007f19b096bf3b swift::SILPassManager::execute() PassManager.cpp:0:0
#12 0x00007f19b09677b8 swift::SILPassManager::executePassPipelinePlan(swift::SILPassPipelinePlan const&) PassManager.cpp:0:0
#13 0x00007f19b096776d swift::ExecuteSILPipelineRequest::evaluate(swift::Evaluator&, swift::SILPipelineExecutionDescriptor) const PassManager.cpp:0:0
#14 0x00007f19b0c2b2ba swift::SimpleRequest<swift::ExecuteSILPipelineRequest, std::tuple<> (swift::SILPipelineExecutionDescriptor), (swift::RequestFlags)1>::evaluateRequest(swift::ExecuteSILPipelineRequest const&, swift::Evaluator&) SILOptimizerRequests.cpp:0:0
#15 0x00007f19b0985b8d swift::ExecuteSILPipelineRequest::OutputType swift::Evaluator::getResultUncached<swift::ExecuteSILPipelineRequest, swift::ExecuteSILPipelineRequest::OutputType swift::evaluateOrFatal<swift::ExecuteSILPipelineRequest>(swift::Evaluator&, swift::ExecuteSILPipelineRequest)::'lambda'()>(swift::ExecuteSILPipelineRequest const&, swift::ExecuteSILPipelineRequest::OutputType swift::evaluateOrFatal<swift::ExecuteSILPipelineRequest>(swift::Evaluator&, swift::ExecuteSILPipelineRequest)::'lambda'()) PassManager.cpp:0:0
#16 0x00007f19b09678cc swift::executePassPipelinePlan(swift::SILModule*, swift::SILPassPipelinePlan const&, bool, swift::irgen::IRGenModule*) PassManager.cpp:0:0
#17 0x00007f19b0c0ee4e swift::runSILLoweringPasses(swift::SILModule&) Passes.cpp:0:0
#18 0x00007f19afffeb84 lldb_private::SwiftExpressionParser::Parse(lldb_private::DiagnosticManager&, unsigned int, unsigned int) SwiftExpressionParser.cpp:0:0
#19 0x00007f19affddb59 lldb_private::SwiftUserExpression::GetTextAndSetExpressionParser(lldb_private::DiagnosticManager&, std::unique_ptr<lldb_private::SwiftExpressionSourceCode, std::default_delete<lldb_private::SwiftExpressionSourceCode>>&, lldb_private::ExecutionContext&, lldb_private::ExecutionContextScope*) SwiftUserExpression.cpp:0:0
#20 0x00007f19affdea70 lldb_private::SwiftUserExpression::Parse(lldb_private::DiagnosticManager&, lldb_private::ExecutionContext&, lldb_private::ExecutionPolicy, bool, bool) SwiftUserExpression.cpp:0:0
#21 0x00007f19af98a645 lldb_private::UserExpression::Evaluate(lldb_private::ExecutionContext&, lldb_private::EvaluateExpressionOptions const&, llvm::StringRef, llvm::StringRef, std::shared_ptr<lldb_private::ValueObject>&, lldb_private::Status&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, lldb_private::ValueObject*) UserExpression.cpp:0:0
#22 0x00007f19afae00ea lldb_private::Target::EvaluateExpression(llvm::StringRef, lldb_private::ExecutionContextScope*, std::shared_ptr<lldb_private::ValueObject>&, lldb_private::EvaluateExpressionOptions const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, lldb_private::ValueObject*) Target.cpp:0:0
#23 0x00007f19b015276c lldb_private::CommandObjectDWIMPrint::DoExecute(llvm::StringRef, lldb_private::CommandReturnObject&) CommandObjectDWIMPrint.cpp:0:0
#24 0x00007f19af9d16f1 lldb_private::CommandObjectRaw::Execute(char const*, lldb_private::CommandReturnObject&) CommandObject.cpp:0:0
#25 0x00007f19af9c2a21 lldb_private::CommandInterpreter::HandleCommand(char const*, lldb_private::LazyBool, lldb_private::CommandReturnObject&, bool) CommandInterpreter.cpp:0:0
#26 0x00007f19af9c673e lldb_private::CommandInterpreter::IOHandlerInputComplete(lldb_private::IOHandler&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>&) CommandInterpreter.cpp:0:0
#27 0x00007f19af8fd45a lldb_private::IOHandlerEditline::Run() IOHandler.cpp:0:0
#28 0x00007f19af8db426 lldb_private::Debugger::RunIOHandlers() Debugger.cpp:0:0
#29 0x00007f19af9c85e8 lldb_private::CommandInterpreter::RunCommandInterpreter(lldb_private::CommandInterpreterRunOptions&) CommandInterpreter.cpp:0:0
#30 0x00007f19af6f014c lldb::SBDebugger::RunCommandInterpreter(bool, bool) (/usr/lib64/swift-6.0.1/usr/bin/../lib/liblldb.so.17+0xcf014c)
#31 0x000055b136a1585e Driver::MainLoop() (/usr/lib64/swift-6.0.1/usr/bin/lldb+0x1685e)
#32 0x000055b136a164c8 main (/usr/lib64/swift-6.0.1/usr/bin/lldb+0x174c8)
#33 0x00007f19ae4592e0 (/usr/lib64/libc.so.6+0x262e0)
#34 0x00007f19ae459399 __libc_start_main (/usr/lib64/libc.so.6+0x26399)
#35 0x000055b136a11b85 _start (/usr/lib64/swift-6.0.1/usr/bin/lldb+0x12b85)

💣 Program crashed: Illegal instruction at 0x00007f19adf3d16f
7

I've spent a bit more time investigating this, and it appears that Swift built outside of the emerge environment (with the same exact setup) doesn't have this issue, so this is definitely caused by something about the build environment itself. I'll try to narrow this down further.

Edit: This does not appear to be related to

  • CFLAGS (-march=native -O2 -pipe)
  • CXXFLAGS (-march=native -O2 -pipe)
  • LDFLAGS (-Xlinker -O1 -Xlinker --as-needed -Xlinker -z -Xlinker pack-relative-relocs)

passed in to the build environment, so there's something else getting picked up that causes this.

The full build environment for a package can be found in /var/db/pkg/<pkg_category>/<pkg>, so I'll try to rebuild with the same environment that emerge used on my system.

Edit 2: something else appears to be going on here; compiling from scratch after sourcing the full environment file from the built Swift leads to a working repl. Will try to figure out what else could be different.

Edit 3: It appears that this has to do with how some of the Swift libs are being built. If I copy over /usr/lib64/swift-6.0.1/usr/bin/swift or /usr/lib64/swift-6.0.1/usr/bin/lldb into the bin directory that I've got locally built, both work as expected. Running tree on both my local dir and /usr/lib64/swift-6.0.1 shows an identical directory structure with exactly the same files, so nothing is missing, but there is some difference I need to track down.

Edit 4: I made a local rev of the swift-6.0.1 package on my system with some additional logging and installed, and now the repl appears to work. The only difference that I can tell is that because I already had Swift 6 on my system, this version built with Swift 6 (and I had also been using Swift 6 to build the local lldb and repl that did work). I'll try again by building with swift-5.10.1

1 Like
8

Okay, I've confirmed that this appears to be a difference between building Swift 6.0.1 using Swift 5.10.1 or 6.0.1 itself. With the same exact settings, in the same exact environment, building Swift 6 using 5.10 leads to a broken repl and lldb, whereas building with Swift 6 leads to working tools. :thinking:

It's not yet clear why this would be the case, but I don't believe this to be intentional; based on discussion from RFC: Requiring Swift to build the Swift toolchain, I would expect it to be possible to bootstrap a Swift 6 setup from Swift 5.10 directly, without needing a multi-stage compilation process (e.g., build an initial Swift 6 compiler using Swift 5.10, then use that compiler to rebuild another Swift 6 compiler).

I'll see what I can figure out from here.

(This means that at the moment, the first time you install Swift 6 and bootstrap using Swift 5.10.1, you'll end up with a broken repl; upon the first update to Swift 6, you'll build using Swift 6, and it'll work again.)

By the way, the failure mode appears to be that whatever you attempt to run in the repl causes a StructInst to fail validation as a ForwardingInstruction:

$ ./swift repl
Welcome to Swift version 6.0.1-dev (LLVM 43d73cb0cc589f7, Swift 15672c560c7fd09).
Type :help for assistance.
  1> "Hello"
Optimizer/Verifier.swift:22: Fatal error: instruction <StructInst>   %14 = struct $Int32 (%13 : $Builtin.Int32)      // user: %15
should conform to ForwardingInstruction

This is interesting, because extension StructInst : ForwardingInstruction { conformance is declared in ForwardingInstruction.swift. :thinking:

Edit:

In both the success case and the failure case, I see hits for symbols containing ForwardingInstruction in the following files:

$ find . -mindepth 1 -type f -executable -exec sh -c 'if nm "{}" 2>/dev/null | grep -q ForwardingInstruction &>/dev/null; then echo "{}"; fi' ';'
./usr/bin/swift-frontend
./usr/bin/lldb-server
./usr/lib/swift/host/lib_InternalSwiftScan.so
./usr/lib/libsourcekitdInProc.so
./usr/lib/liblldb.so.17.0.0

All of these files have almost the same symbols (at different offsets, of course), and all have the StructInst conformance to ForwardingInstruction:

0000000006828a10 t $s3SIL10StructInstCAA21ForwardingInstructionA2aDP17preservesIdentitySbvgTW
0000000006828a40 t $s3SIL10StructInstCAA21ForwardingInstructionA2aDP21canForwardOwnedValuesSbvgTW
0000000006828a00 t $s3SIL10StructInstCAA21ForwardingInstructionA2aDP22singleForwardedOperandAA0H0VSgvgTW
0000000006828a20 t $s3SIL10StructInstCAA21ForwardingInstructionA2aDP23preservesRepresentationSbvgTW
0000000006828a30 t $s3SIL10StructInstCAA21ForwardingInstructionA2aDP26canForwardGuaranteedValuesSbvgTW
00000000078ddcc0 d $s3SIL10StructInstCAA21ForwardingInstructionAAMc
0000000006bf72c8 d $s3SIL10StructInstCAA21ForwardingInstructionAAWP

The only outlier is lldb-server from the successful build, which lacks almost all of these symbols — but replacing it with the lldb-server binary from the failure case doesn't appear to make a difference.

My next guess, then, is that it's not that the conformance is missing, but that something about the runtime is failing to pick it up.

Edit 2: Okay, I've bisected the failing lib down to liblldb.so.17.0.0: running the successful Gentoo build with the failing build's liblldb.so.17.0.0 reproduces the failure; no other library change does.

Now I need to see if I can figure out what's changed between these versions.

1 Like
9

Okay, this appears to somehow be a consequence of building and linking with lld. In a swift:5.10.1-fedora39 container, I built Swift 6.0.1 with the following preset:

[preset: fedora]
mixin-preset=buildbot_linux,no_assertions,no_test
build-ninja=0
llvm-targets-to-build=host
skip-build-curl
skip-build-zlib

and build command:

./swift/utils/build-script --preset=fedora install_destdir="$(pwd)/swift-6.0.1" installable_package=""

(I had to apply the patch from [ClangImporter] Make sure the `-resource-dir` is checked before the `-sdk`, as done everywhere else in the compiler by finagolfin · Pull Request #74814 · swiftlang/swift · GitHub to get things to build)

This yielded a working lldb:

[root@9e7ec6e19f12 bin]# echo "while true {}" >> Test.swift
[root@9e7ec6e19f12 bin]# ./swiftc Test.swift
[root@9e7ec6e19f12 bin]# ./lldb -- ./Test
(lldb) target create "./Test"
Current executable set to '/root/swift/swift-6.0.1/usr/bin/Test' (x86_64).
(lldb) settings set target.disable-aslr false
(lldb) run
Process 31240 launched: '/root/swift/swift-6.0.1/usr/bin/Test' (x86_64)
warning: (x86_64) /lib64/libm.so.6 No LZMA support found for reading .gnu_debugdata section
warning: (x86_64) /lib64/libgcc_s.so.1 No LZMA support found for reading .gnu_debugdata section
Process 31240 stopped
* thread #1, name = 'Test', stop reason = signal SIGSTOP
    frame #0: 0x00005560b3d299e6 Test`main + 6
Test`main:
->  0x5560b3d299e6 <+6>: jmp    0x5560b3d299e6 ; <+6>

Test`_fini:
    0x5560b3d299e8 <+0>: endbr64
    0x5560b3d299ec <+4>: subq   $0x8, %rsp
    0x5560b3d299f0 <+8>: addq   $0x8, %rsp
Target 0: (Test) stopped.
(lldb) settings set target.language swift
(lldb) expr import Foundation
(lldb)

Updating the preset to

[preset: fedora]
mixin-preset=buildbot_linux,no_assertions,no_test
build-ninja=0
extra-cmake-flags=-DLLVM_USE_LINKER:STRING=lld
llvm-targets-to-build=host
skip-build-curl
skip-build-zlib

yields a broken lldb:

[root@9e7ec6e19f12 bin]# echo "while true {}" >> Test.swift
[root@9e7ec6e19f12 bin]# ./swiftc Test.swift
[root@9e7ec6e19f12 bin]# ./lldb -- ./Test
(lldb) target create "./Test"
Current executable set to '/root/swift/swift-6.0.1/usr/bin/Test' (x86_64).
(lldb) settings set target.disable-aslr false
(lldb) run
Process 30380 launched: '/root/swift/swift-6.0.1/usr/bin/Test' (x86_64)
warning: (x86_64) /lib64/libm.so.6 No LZMA support found for reading .gnu_debugdata section
warning: (x86_64) /lib64/libgcc_s.so.1 No LZMA support found for reading .gnu_debugdata section
Process 30380 stopped
* thread #1, name = 'Test', stop reason = signal SIGSTOP
    frame #0: 0x00005560b3d299e6 Test`main + 6
Test`main:
->  0x5560b3d299e6 <+6>: jmp    0x5560b3d299e6 ; <+6>

Test`_fini:
    0x5560b3d299e8 <+0>: endbr64
    0x5560b3d299ec <+4>: subq   $0x8, %rsp
    0x5560b3d299f0 <+8>: addq   $0x8, %rsp
Target 0: (Test) stopped.
(lldb) settings set target.language swift
(lldb) expr import Foundation
Optimizer/Verifier.swift:22: Fatal error: instruction   %2 = tuple ()                                   // user: %3
should conform to ForwardingInstruction
LLDB diagnostics will be written to /tmp/diagnostics-4e1682
Please include the directory content when filing a bug report
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.      Program arguments: ./lldb -- ./Test
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0  lldb            0x00005642a5aca028
1  lldb            0x00005642a5ac83be
2  lldb            0x00005642a5aca958
3  libc.so.6       0x00007fade1f449a0
4  libswiftCore.so 0x00007fade1bb26bf $ss17_assertionFailure__4file4line5flagss5NeverOs12StaticStringV_SSAHSus6UInt32VtF + 351
5  liblldb.so.17   0x00007fade44afd26
6  liblldb.so.17   0x00007fade44afd3a
7  liblldb.so.17   0x00007fade489ac8c
8  liblldb.so.17   0x00007fade4a8f89b
9  liblldb.so.17   0x00007fade489a0be
10 liblldb.so.17   0x00007fade489b0df
11 liblldb.so.17   0x00007fade489cc9b
12 liblldb.so.17   0x00007fade4898518
13 liblldb.so.17   0x00007fade48984cd
14 liblldb.so.17   0x00007fade4b5d72a
15 liblldb.so.17   0x00007fade48b68ed
16 liblldb.so.17   0x00007fade489862c
17 liblldb.so.17   0x00007fade4b412be
18 liblldb.so.17   0x00007fade3f92a34
19 liblldb.so.17   0x00007fade3f71a09
20 liblldb.so.17   0x00007fade3f72920
21 liblldb.so.17   0x00007fade391e7e5
22 liblldb.so.17   0x00007fade3a73caa
23 liblldb.so.17   0x00007fade40fa5c8
24 liblldb.so.17   0x00007fade40fb795
25 liblldb.so.17   0x00007fade3965891
26 liblldb.so.17   0x00007fade3956bc1
27 liblldb.so.17   0x00007fade395a8de
28 liblldb.so.17   0x00007fade389162a
29 liblldb.so.17   0x00007fade386f5f6
30 liblldb.so.17   0x00007fade395c788
31 liblldb.so.17   0x00007fade368431c lldb::SBDebugger::RunCommandInterpreter(bool, bool) + 156
32 lldb            0x00005642a5ab085e
33 lldb            0x00005642a5ab14c8
34 libc.so.6       0x00007fade1f2e14a
35 libc.so.6       0x00007fade1f2e20b __libc_start_main + 139
36 lldb            0x00005642a5aacbb5

💣 Program crashed: Illegal instruction at 0x00007fade1bb26bf

Thread 0 "lldb" crashed:

0 0x00007fade1bb26bf

Backtrace took 0.36s

Illegal instruction

I'm going to confirm by building 6.0.1 in a swift:6.0.1-fedora39 container: if it works out-of-the-box with lld, then this is some combination of 5.10.1 and lld that doesn't work, which is unfortunate.

Edit: I took a random guess at building with -DLLDB_EXPORT_ALL_SYMBOLS:BOOL=YES but it doesn't appear to have made a difference

Edit 2: Can confirm that the exact same build that was producing a broken lldb in swift:5.10.1-fedora39 builds a working one in swift:6.0.1-fedora39 with LLVM_USE_LINKER=lld. So, this isn't unique to Gentoo

1 Like
10

I've gone ahead and filed Linux: Swift 6 LLDB/repl built with 5.10.1+lld crashes on evaluating Swift code · Issue #77243 · swiftlang/swift · GitHub for this: I'm able to reproduce this issue in the official Swift Docker images for both Ubuntu 24.04 and Fedora 39.

tl;dr:

Linker Swift 5.10.1 Swift 6.0.1
lld :boom: :white_check_mark:
gold :white_check_mark: :white_check_mark:
3 Likes