Separate Packages for each Linux Distribution?

I am trying to use Swift on Linux, specifically on Fedora 38. However, I've noticed that Swift currently only officially supports Ubuntu variants, CentOS 7, Amazon Linux 2, and RHEL UBI 9.

I'm curious about the reasons behind having separate packages for each Linux distribution. Could you please explain the technical rationale behind this decision?

Additionally, I'm curious about why the Swift SDK includes its own clang/clang++ for each distribution. How are these components utilized in the build process?

1 Like

My understanding is that different linux distros do things slightly differently, eg naming the shared system libraries that the toolchain links against differently, so this is done to make sure the Swift toolchain works. Either the RHEL or CentOS builds should work for you on Fedora.

The Swift Package Manager supports building C and C++ too, for example, the swift-crypto package simply builds BoringSSL on linux and provides a Swift wrapper for that C and assembly code. The latest Swift 5.9 release adds experimental C++ interoperability, so that Swift and C++ code can directly call each other.

@bclee Swift is actually officially supported by Fedora 38 and is available in their official package repository.
To install simply run sudo dnf install swiftlang.
Currently Swift 5.8.1 is the latest version available but I am sure 5.9 will be coming soon.
Some more information here - https://developer.fedoraproject.org/tech/languages/swift/swift_installation.html

https://src.fedoraproject.org/rpms/swift-lang

1 Like

What I want to know is whether it is possible to distribute a single Swift SDK package that can be used on both ancient distributions like CentOS 7 and modern distributions like Ubuntu 22.04, Fedora 38, etc.

I've created a simple script to check the differences between two directories containing ELF files. The script is available here: Compares ELF files in two directories · GitHub

The results of comparing swift-5.9-RELEASE-centos7 and swift-5.9-RELEASE-ubuntu22.04 are available here: Compares ELF files in two directories · GitHub

Notable differences are found in the following libraries:

libedit.so.0 vs. libedit.so.2
libpython3.6m.so.1.0 vs. libpython3.10.so.1.0
libncurses.so.5 vs. libncurses.so.6 (ncurses)

All of these dependencies are documented in llvm.

Here are some details about these libraries:

libedit: It is only used in lldb, is BSD licensed, and can be statically linked.
libpython: It is only used in lldb.
libncurses: It is used in various places. Most distributions have an older version of ncurses for compatibility reasons (libncurses5 for Debian, ncurses-compat-libs for Fedora).

Except for libpython, if we make slight modifications to the Centos 7 build, we can use the same Swift SDK on both Centos 7 and Ubuntu 22.04.

For Python, I believe that's why Xcode ships its own Python interpreter. We can do the same for Swift, as it is only used in lldb.

What do you think?

It’s about ABI. The libraries that things like corelibs foundation, dispatch, xctest, and even the main runtimes themselves use aren’t guaranteed to have the same layout across versions or across distributions. Copying libraries and executables built for one distro to another and running it might look like it works, but it’s certainly a happens-to-work situation and not a guaranteed to work situation. If you’re okay foregoing CXX/C interop with any existing libraries (read no shared objects that you didn’t build and no dlopen to thing you didn’t build) you could build an SDK that was isolated enough to maybe work this way. You would have to provide your own icu, libxml2, zlib, etc, and make very certain that you, nor your dependencies pulled in anything from the system or you’d end up with undefined behavior. It’s true that some of the libraries try to maintain ABI stability, but then we’d need to figure out what version started doing that and make sure that at least that version is used on your minimum deployment target.

2 Likes

How does Rust get away with this (a single x86_64-unknown-linux-gnu target)? Is it just that the Rust stdlib restricts itself to glibc, which has stronger promises about cross-vendor behavior?

1 Like

Yes, Rust doesn't have as many dependencies as Swift does, especially in the core libraries (primarily Foundation). And it's not just Glibc, I don't think it has a strong dependency on lib(std)c++ as we do. Although Rust does have a Musl distribution too IIRC, which means you may not even need Glibc installed, or even a specific version of it.

3 Likes

Pretty much. Everything is completely statically linked with the exception of glibc and/or takes part in the cargo ecosystem. Take SQLite and the sqlite-sys package for example. Cargo will use the system headers and system library if available and build against that for local builds. Otherwise it will use the sources from its own package. That’s maybe okay for pure package content, but corelibs foundation/dispatch and friends don’t participate in that and can share memory objects across boundaries.

3 Likes

Yeah, from what I got from reading the sources, it looked like the main glibc dependency was on malloc. Otherwise everything else in std looked to be implemented in rust itself. I may have missed something, but I saw no C++.

1 Like

At the very least it also gets filesystem, networking, and threading support from glibc. But it can probably confine itself to those.

1 Like

Yeah, they have a few things. Was on phone.

Looks like a fair bit of platform-specifics handling the differences between OS's in library/std/src/sys
and library/std/src/os, which makes sense.

Though even looking through those, it looks like a lot of the implementation is coming from the rust runtimes themselves too. I still don't see a lot of libc::.

1 Like

Aha, I understand now. So there are much larger issues in runtime libraries required by Swift Foundations than the compiler itself. It is unfortunate that Swift cannot be distributed as a single binary like Go or Rust.
I think I need to find a way to distribute using flatpak or something like that to ensure that libicu and other dependencies are fixed.