Hi folks! To anyone out there using Gentoo (there are dozens of us!), you might be interested to know that Swift 5.10.1 is now available for installation as an unofficial package via GURU. If you're not aware, GURU is the official repository for unofficial packages, following the standard package manager build and installation process. Project:GURU/Information for End Users - Gentoo wiki has more information, but tl;dr:
eselect repository enable guru to add the GURU repo for packages through emerge
emerge --sync guru to sync
emerge --ask dev-lang/swift
Again, this is an unofficial package, not supported by either the Swift project or the Gentoo developers; I put this together for my own use and figured it might be helpful to others. If you run into trouble installing or running, please let me know and I'll do my best to help.
Although Gentoo is a very build-from-source-locally-focused distro, I went with the prebuilt binaries because:
I can't for the life of me get either release/5.10.1 or release/6.0 to build on my machine (with all of the required dependencies installed, with the same build presets as the Linux buildbots); I plan on following up on this when I have the time so I can explore what it might take to get source builds going
Building Swift from source can be somewhat painful for end-users, and the process differs from what Gentoo users might expect:
Swift optimizes for building self-contained toolchains: the way that Swift is built optimizes for having potentially multiple isolated versions of Swift installed simultaneously — great for most end-users, but somewhat painful for building because it bundles its own copies of LLVM, Clang, LLDB, ICU, etc. Not only does this mean that building a release from source requires building each of these projects, the built result can't be installed in a non-isolated way because the toolchain contains files which would conflict with existing system dependencies (e.g., a typical Gentoo system already has LLVM, Clang, LLDB, ICU, and more). This approach lends itself well to "download a built toolchain and point your $PATH at it", but this is very different from the typical Gentoo process, which would have the installation be system-wide, and depend on already-installed system dependencies
There is no single tarball containing all Swift sources for a release: this is more of a minor pain point, but it's a bit difficult to actually get all of the Swift source code needed for a build downloaded in one go. In order to get all of a release's dependencies, you either need to fetch the swiftlang/swift source for a release, then run update-checkout --clone to get the dependencies (which even with the flags to skip tags + history takes many, many minutes on my machine), or maintain a list of tarballs to fetch containing sources directly. The latter makes more sense, but the mapping from Swift release ↔︎ dependency versions feels better maintained in some "official" way than copied into the Gentoo ebuild
I chose the Fedora package because its required dependencies very closely match what can be found or installed on an up-to-date Gentoo system
Because this is a self-contained toolchain, it can't be splatted onto the filesystem in the usual locations; instead, it installs into /usr/lib64/swift-<version>, with swift, swift-*, and sourcekit_lsp symlinked into /usr/bin
Theoretically, if the tools are symlinked as /usr/bin/swift-*-<version>, you could have multiple Swift versions installed simultaneously in various slots. I didn't know whether anyone would find this actually useful, but this is possible to adjust in the future. (I'm not sure whether there's an existing USE flag we could use to say "please build this versioned", but theoretically we can add a package-local one. I figured being able to just run swift as opposed to swift-5.10.1 would be more useful)
Because I'm relying on the prebuilt Fedora toolchain and there aren't yet any 6.0 builds, only 5.10.1 is available at the moment; when 6.0 Fedora toolchains are available, I can add that too
Ideally, it'd be interesting to explore contributing to swiftlang/swift-installer-scripts to better spin up infrastructure for producing better builds tailored for Gentoo, but I'm not sure whether there'd be enough demand to actually justify the infrastructure (and hosting), so for now, I'm happy to piggyback off of existing work
If you have any comments, questions, suggestions, or more, I'd be happy to hear them!
I haven't used Gentoo in a long while, but this is awesome nonetheless! I would like to try this package out later in my VM playground where I get many different versions of Linux and try to run Swift on them...
That's great! Unfortunately, at the moment, you may run into some trouble installing Swift because of some new restrictions on Swift's gold dependency, but I'm working on getting source builds set up; hoping this will be a non-issue within the next week or so.
Okay, I've finally managed to get Swift source builds set up, and the changes are now live on GURU.
Edit: I guess the original post in this thread is too old for me to be allowed to edit, but the original "More Info" is no longer correct and should be disregarded.
I'm building this right now. This has been a dream of mine for the 10+ years Swift has been around. Thank you so much for putting so much work into this :D
I really appreciate all the effort you put into this ebuild! It was so helpful for me to start writing my first ever program in Swift :)
I just wanted to share 2 things here.
Even if I specify a Python version via PYTHON_SINGLE_TARGET (e.g. python3_10), my latest Python installation (3.13) was unexpectedly used to build the bundled lldb via ninja.
Regarding to the use of Python >= 3.12, I also had to add setuptools (and its Python module dependencies) as a dependency, otherwise I got an error like "ModuleNotFoundError: No module named 'distutils'" because distutils was removed in Python 3.12. So, it might be better to include it as one of the dependencies for Python >= 3.12.
By using Python 3.13 and adding setuptools, I was able to build Swift using your ebuild without any problems. Thank you very much!
Really glad you managed to get up and running, but even more, thanks for taking the time to write this up! This is extremely helpful; I've found it very difficult to capture the necessary dependencies, so the additional testing is greatly appreciated.
Ah, I need to test, but I believe this should be an easy fix — I think I'm missing a call to python_setup prior to compilation, which should set up the right Python env and respect the PYTHON_SINGLE_TARGET you have set.
Great catch! This should also be easy to add; I run Python 3.12 on my system and likely already have setuptools installed, which is why I missed this.
I hope to have these fixes pushed out some time in the next few days!
I know you're already set up for now (and certainly no expectations on my end), but if you find yourself in a position to test either of these, I'd be happy to know whether they resolved the issues you've run into! I managed to do some minimal testing in a Docker container, but real-world confirmation would be even better.
Hello @itaiferber,
Thank you for the updates! After removing setuptools for target Python and Swift itself, I tried doing re-emerge it on my local machine again
1. Fix for PYTHON_SINGLE_TARGET
As you said, another version of python unfortunately was used also at this time and got an error like below (It's same as before, but I paste it here just in case):
# got an error for building the bundled lldb (python 3.13 was used)
% sudo PYTHON_SINGLE_TARGET="python3_10" emerge -av dev-lang/swift
These are the packages that would be merged, in order:
Calculating dependencies... done!
Dependency resolution took 2.75 s (backtrack: 0/20).
[ebuild N ] dev-lang/swift-5.10.1::guru PYTHON_SINGLE_TARGET="python3_10 -python3_11 -python3_12 -python3_13" 0 KiB
Total: 1 package (1 new), Size of downloads: 0 KiB
...
FAILED: bindings/python/LLDBWrapPython.cpp bindings/python/lldb.py /var/tmp/portage/dev-lang/swift-5.10.1/work/build/Ninja-ReleaseAssert/lldb-linux-x86_64/bindings/python/LLDBWrapPython.cpp /var/tmp/portage/dev-lang/swift-5.10.1/work/build/Ninja-ReleaseAssert/lldb-linux-x86_64/bindings/python/lldb.py
cd /var/tmp/portage/dev-lang/swift-5.10.1/work/build/Ninja-ReleaseAssert/lldb-linux-x86_64/bindings/python && /usr/bin/python3.13 /var/tmp/portage/dev-lang/swift-5.10.1/work/llvm-project/lldb/bindings/prepare_bindings.py --srcRoot=/var/tmp/portage/dev-lang/swift-5.10.1/work/llvm-project/lldb --targetDir=/var/tmp/portage/dev-lang/swift-5.10.1/work/build/Ninja-ReleaseAssert/lldb-linux-x86_64/bindings/python --cfgBldDir=/var/tmp/portage/dev-lang/swift-5.10.1/work/build/Ninja-ReleaseAssert/lldb-linux-x86_64/bindings/python --prefix=/var/tmp/portage/dev-lang/swift-5.10.1/work/build/Ninja-ReleaseAssert/lldb-linux-x86_64 --use-static-binding
Traceback (most recent call last):
File "/var/tmp/portage/dev-lang/swift-5.10.1/work/llvm-project/lldb/bindings/prepare_bindings.py", line 285, in <module>
main(sys.argv[1:])
~~~~^^^^^^^^^^^^^^
File "/var/tmp/portage/dev-lang/swift-5.10.1/work/llvm-project/lldb/bindings/prepare_bindings.py", line 280, in main
prepare_all_bindings(options)
~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
File "/var/tmp/portage/dev-lang/swift-5.10.1/work/llvm-project/lldb/bindings/prepare_bindings.py", line 74, in prepare_all_bindings
prepare_binding_for_language(bindings_dir, script_lang, options)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/tmp/portage/dev-lang/swift-5.10.1/work/llvm-project/lldb/bindings/prepare_bindings.py", line 49, in prepare_binding_for_language
module.main(options)
~~~~~~~~~~~^^^^^^^^^
File "/var/tmp/portage/dev-lang/swift-5.10.1/work/llvm-project/lldb/bindings/python/prepare_binding_python.py", line 443, in main
python_module_path = get_python_module_path(options)
File "/var/tmp/portage/dev-lang/swift-5.10.1/work/llvm-project/lldb/bindings/python/prepare_binding_python.py", line 369, in get_python_module_path
from distutils.sysconfig import get_python_lib
ModuleNotFoundError: No module named 'distutils'
2. Conditional dependency for dev-python/setuptools
It worked like a charm! I successfully could build Swift again
Thanks again for testing and reporting back! I know the process of recompiling is time-consuming and resource-intensive...
Argh, frustrating! Just to double-check: when have you last emerge --sync'd, and when did you try installing? I ask only because the fix I tested only landed on the master branch of GURU within the last 24 hours, so it's possible that if you haven't sync'd since then, the fix wasn't picked up.
If you sync'd more recently, then I'll need to keep digging to figure out what's going on.
Ahh, at my last sync, somehow I just missed the changes for the fix. You've already fixed it. Sorry! With your last commit, I was able to rebuild Swift successfully again also with any specific Python versions using PYTHON_SINGLE_TARGET! That's awesome
Hey @srbok, thanks for giving this a shot, and for reporting the error! Out of curiosity, what was the invocation you used to collect the error log? Unfortunately, I can't tell from the log what the error was; normally there's more information when compilation fails (which prints to stderr instead of stdout) and that seems to be missing here — it's possible that your invocation only captured stdout.
When testing out installation, I typically run something like
emerge -a dev-lang/swift 2>&1 | tee emerge-swift.log
tee writes whatever data is input into out both to the terminal and to the given file simultaneously (so I can see it live, and also have it stored)
2>&1 redirects stderr to stdout, so everything gets captured together
If you wanted to only write the log to a file without tee, you can run something like
emerge -a dev-lang/swift >emerge-log.log 2>&1
(The order of the redirections matters here, because you want to point stdout to your log file first, then stderr (2) to wherever stdout (1) is now pointing)
If you get the chance to try again, could you see if you manage to capture more error output this way? Thanks!
Thanks for trying again! Unfortunately, I'm not seeing any sort of explicit cause for compilation failure, but I might have a guess:
Normally, a compilation failure ends up printing a long FAILED: ... line with the error messages produced by the compiler, and there isn't any info like that here; it appears that compilation is just stopping
It appears that both compilation attempts failed at around the same time:
Your latest log shows a failure at [3340/4412] in building /var/tmp/portage/dev-lang/swift-5.10.1/work/llvm-project/clang-tools-extra/clang-tidy/performance/TypePromotionInMathFnCheck.cpp
Your previous log shows a failure at [3341/4412]just afterwards, having successfully compiled TypePromotionInMathFnCheck.cpp
Both compilation attempts have failed relatively early on in the process, at building the stage0 LLVM, somewhere that doesn't indicate any sort of configuration error
This points me toward potential resource exhaustion: is it possible that your machine has run out of disk space or memory during compilation? Building LLVM (and the Swift compiler at large) has pretty heavy requirements for both, so I'm wondering if the process is just being killed. More info on your build setup might reveal more!
Edit: for reference, on my system, the final size of the swift-5.10.1 build dir is 14.5 GiB, and swift-6.0.1 is 16.2 GiB
Hmm I wonder if you're running into any other resource limits, like number of open file descriptors, or number of processes available to you. The output of ulimit --all may help.
Other things that you could try:
See if you can manually resume LLVM compilation with sudo ninja -C /var/tmp/portage/dev-lang/swift-5.10.1/work/build/Ninja-ReleaseAssert/llvm-linux_x86_64 build (please double-check this path; this is from memory) — a failed emerge will leave behind the temporary work directory, so this should continue where the build left off. If this succeeds, this points more clearly at some form of resource exhaustion
Copy the contents of /var/tmp/portage/dev-lang/swift-5.10.1/work to a directory you own, and trigger the build script manually with the same arguments as the ebuild:
If this succeeds, it's indicative of some form of configuration difference inside of the emerge build environment that we can try to track down
Edit /var/db/repos/guru/dev-lang/swift/swift-5.10.1.ebuild to insert set -x (and possibly additional prints to stderr, for sanity checking) to see if there's additional info we're missing somehow
I assume, too, that you're not doing much on the machine while it's compiling, but if manage to ensure this builds when the machine is under minimal load, I wonder if it'll get any further
Hope we manage to track this down one way or another!
Edit: the above paths assume you're still working with the original swift-5.10.1 ebuild as opposed to the updated swift-5.10.1-r1 version; you've --sync'd the changes down and are compiling the newer version, you'll need to sub in swift-5.10.1-r1
I wonder if you're running into any other resource limits, like number of open file descriptors, or number of processes available to you. The output of