Why so few Linux distros supported by swift.org?

I'm a little surprised about the limited options for Linux images / tarballs on Swift.org - Download Swift. i.e. it's just Ubuntu. No Debian, no RHEL, no CentOS, no Raspbian, etc.

Is there any particular reason for this? Or, more pointedly, what needs to happen to add additional distros?

(I'm aware there are 3rd parties providing tarballs etc for other distros, but it would of course be better if it was all centralised on the official website)

1 Like

Are you asking why the 3rd-party distributions can't be housed on swift.org, or are you asking why Apple only funds Ubuntu distributions?

…either? Both? :)

Can I use apt-get install swift to install Swift?

On Debian based systems (we're using Stretch/Buster/Bullseye), you should be able to do the following:

mv ~/Downloads/swift-5.0-RELEASE-ubuntu18.04.tar.gz /someFolder
cd /someFolder
tar xzf /someFolder/swift-5.0-RELEASE-ubuntu18.04.tar.gz
rm /someFolder/swift-5.0-RELEASE-ubuntu18.04.tar.gz
mv /someFolder/swift-5.0-RELEASE-ubuntu18.04 /someFolder/.swift-5.0-RELEASE-ubuntu18.04
ln -s /someFolder/.swift-5.0-RELEASE-ubuntu18.04 ./swift

The reason for a symbolically linking swift to .swift-5.0-RELEASE-ubuntu18.04 is that you can then add /someFolder/swift to your path, but continue to upgrade Swift by changing out the releases over time and simply re-linking as you go.

Speaking on behalf of the Core team, this is an area we hope to see some progress on over the next year. It mostly boils down to identifying how to support more distributions with infrastructure and people who will help keep builds of Swift working on various distributions.

9 Likes

Totally uninformed guy here: what's the difference between the distributions (as far as Swift is concerned)?

What dependencies does Swift have that these distros need to provide? My naive assumption is that as long as you have the basic sys calls to read/write/alloc memory, you're good to go. What am I missing?

1 Like

I think that my experiences packaging and porting software for Gentoo and Exherbo have given me a different perspective. There are subtle differences, there is no stable ABI for Linux. Each distribution is built differently with a different version of the libraries, and different compiler.

The C++ runtime is not ABI stable, the libstdc++ version must match exactly. If this is not done, you can find odd crashes at various points.

Similarly, the C library has subtle configuration differences and can result in similar failures as well.

The file system layout is "standard", so clearly, everyone has their own. Those differences matter because the package is going to be just completely ignored by the distro maintainers unless it conforms to the distribution's policies.

Integrating into the package manager is generally a good idea, not doing so means that any changes to the system configuration can accidentally render your tools broken. By integrating into the package manager, you ensure that the changes to the system are aware and can flag the problems in the first place.

These may seem like nice to have little details and a way to stop Swift from being available on Linux distributions, but, at the scale of distributing the package to distribution users, these become extremely important. More so, making it broadly available with a poor experience with the toolchain just not working the first time around is likely to be more detrimental to the project than not having it packaged.

I'm happy to go into the litany of the nuances associated with wider distribution of Swift on Linux, but, I don't think that it would be complete nor useful.

10 Likes

That's good to hear. Having e.g. RHEL & CentOS onboard would make server-side Swift a notably easier sell at my work.

In the meantime, would it be possible to link to some of the 3rd party efforts supporting other distros right now? e.g. GitHub - uraimo/buildSwiftOnARM: All you need to build Swift on a RaspberryPi or other ARM boards, updated to Swift 5.1.5, swift-arm/release - Packages · packagecloud, etc? I for a long time was mislead into thinking that only Ubuntu was possible, because of Swift.org - Download Swift.

How do other languages solve this problem though? Is this an issue for every compiled language? I've rarely had any sort of similar issue with Ruby or Python, for example, you can just use any distro-independent version manager (like rbenv, pyenv) for it. Similarly, I hear that the problem appears to be solved for Rust with rustup. Do they just have a lot of distro-specific logic within these tools?

1 Like

The other languages solve the problem by being distributed by the distribution itself. C/C++ compilers are part of the distribution. For Python/Ruby, again you are using distribution provided packages. The python packages themselves are not compiled to native code - they are distributing the sources and the code will be interpreted by the distributions' interpreter. Rust solves the problem the same way - you distribute sources and build locally.

However, if you are just looking to build Swift from source, that use case is already covered with extensive documentation :-).

3 Likes

It is, and blessedly it actually works in my experience (on Raspbian), but it doesn't appear to be actually documented anywhere on swift.org, leaving it to the mercy of third parties like buildSwiftOnARM, whom are doing an admirable job but are still quite some ways away from sudo apt install swift - which is really the minimal level it has to get to before the broader developer community will even consider Swift, as opposed to just enthusiasts as today.

If this sounds strange to you, imagine that you bought a new Mac and/or iDevice and were told that while it comes with Objective-C, C, C++, & Python, if you want to code in Swift you'll need to search the web for discussions about installing it, hopefully eventually find a random Github repo or four year old blog post describing how, which involves downloading gigabytes of source code, dozens of steps, many hours of waiting, and probably doesn't go smoothly much of the time.

There's no way 99% of people would bother even trying Swift, let-alone commit to it to build their production software. And that's the situation today, on Linux (and, from what I hear, Windows).

I expect I'm preaching to the choir here in these forums, to a large degree, but I wanted to lay out the situation in case some readers just aren't familiar with it (most Swift developers today being exclusively Mac or iDevice developers, after all, who simply never have to deal with any of this).

I wish I had any kind of 'in' or expertise with Linux package managers, but alas I don't. I don't have a lot of free time, but I'd at least like to know how I might be able to help with all this in a concrete way.

5 Likes

@wadetregaskis i see your point, i am a Linux user myself, and don't use Ubuntu, so i need to compile Swift everytime a new release comes out, from time to time it does require some changes to make it work, but it's possible.
However getting Swift to run on different distros would be a huge step forward for Swift :slight_smile:

Unfortunately my time currently don't allow me to contribute to help getting Swift more stable on Linux (Even though i really would love to), but if you ever need help with something specific, feel free to write to me, either here or in the Vapor discord :slight_smile:

Windows does have nightly builds that you can download and use. Because there is somewhat more stability on Windows due to the way that Microsoft distributes software, it is possible to easily create a distributable image.

FWIW, you are not preaching to the choir. I am a Linux user, and so, that is something that I need to deal with. In fact my Linux distribution is different enough from Ubuntu to require special support in Swift itself.

But that is exactly the thing - bringing Swift to any platform requires expertise of that platform and in Swift. I would say that there are a few ways to help: you can donate hardware to run CI, you can fix issues for other targets/platforms, you can package Swift for other distributions. Getting the packages pushed into the distribution repositories would certainly help.

2 Likes

Great to hear that! I hope that a Windows installer is also coming to swift.org.

I need to finish restoring the builds, but, there has been a Windows installer on Azure for quite some time.

What do you mean by "donate"? A while ago I offered here to install a CI on my Odroid U2:

but I never got an answer to this offer.

regards,

Lars

P.S.: I would need some advice to set up all the stuff on that Odroid.

Interesting, I never saw that offer. Unfortunately, I dont think that building on the odroid u2 is really that great of an experience. Especially at 2GB, you would be unable to link the toolchain itself with debug information.

That said, it may be valuable to consider using that for running just the test suite, cross-compiling for the device from x86_64. That way, we have additional coverage of the runtime aspects on different architectures.

What type of advice are you looking for in terms of setting that up. Im certain if there are specifics, the community collectively would be able to help.

I was afraid that the hardware of the Odroid U2 would not be sufficient to build the toolchain. However, I find the idea of just running the test suite on this hardware interesting.

How do we proceed from here? Who is taking care of the cross compilation (I don't have hardware available for this)? How would the cross compiled toolchain get to the Odroid (I am on a ADSL Link with IPv4/IPv6 but changing IP-Adresses — I guess some DynDNS Setup would help here) … those are the first questions coming to my mind.

That is exactly the problem - in order to support these environments, someone needs to be willing to support it, and one of the areas that involves is funding. Someone needs to be willing to provide the hardware. In this case, the single host is not sufficient for this testing, something else would be needed to build it as well as both machines need to be colocated to allow the content to be executed (trying to copy the artifacts over does complicate things).

In order to have something like this in the community CI, you would need a static IP and the ability to expose ports to the internet.

This is part of why it is difficult to add hosts to the CI - it involves a real commitment and I don't think that there have been very many participants willing to make that level of commitment.

As to the cross-compilation, the infrastructure for that is more or less present currently. I did that work as part of the Windows efforts to allow me to quickly be able to work on the project.