Swift on Linux: OpenGL, CUDA bindings, and IDE (IntelliJ?)

Background:

I have a 20k-30k Rust code base that I am considering rewriting in Swift. This is because I want to see what development is like in a language with RAII + Static Types + a builtin REPL.

I have a very weak (13" mbp) Mac and a very beefy Linux server (24 core, 100+ GB RAM, 2 1080Ti).

The host Linux machine is Fedora. Using GitHub - apple/swift-docker: Docker Official Image packaging for Swift , I am able to copy the docker image and run a swift repl, doing "2 + 3 => 5".

Questions:

  1. Does Swift-in-Docker support OpenGL ?
  2. Does Swift-in-Docker support CUDA ?
  3. Can I use IntelliJ with Swift-in-Docker ?

Thanks!

EDIT: One of the main issues here is that most Swift resources I find are XCode/OSX geared -- but I have a very weak mbp and a very beefy Linux server, and I'd prefer to do my development on Linux.

EDIT2: By "Cuda", I mean "CUDA enabled Tensorflow/PyTorch on Linux"

CLion has a Swift plugin, which gives you an IntelliJ-based IDE for Swift on Linux. That's probably your best choice on Linux at the moment. See Swift | CLion

Also, you can install Swift on Fedora via sudo dnf install swift-lang (thanks to @Ron_Olson).

This is just FYI (I use Ubuntu), so do let us know your experiences :slight_smile:

  1. yum install swift-lang got me a swift 5.1.2 repl (on Fedora 29)

  2. CLion: "your version of swift & swift plugin are incompatible"

  3. I suspect this is because I need 5.1.3

  4. swift-5.1.3 is going to require Fedora 30: https://pkgs.org/download/swift-lang

  5. Cuda 10 (without a downgraded GCC) does not support Fedora 30: Cuda 10 installation on Fedora 30 with gcc 9.1.1 20190503 - CUDA Setup and Installation - NVIDIA Developer Forums [got burned by this once already, lol]

It turns out trying to pull in swift-lang 5.1.3 from fedora/rawhide (latest dev branch) has libsgtdc++/gcc-c++ conflicts -- likely the same issue that breaks Cuda 10.

I manually installed an earlier version of the Swift Plugin (from before 5.1.3 release) from Swift - IntelliJ IDEs Plugin | Marketplace

Still mismatch.

Looks like might be time to try Ubuntu 18 again.

Hey, can you give me a little more info on the conflicts you mentioned? I've run into issues with libstd before but thought I had fixed all of them.

sudo yum install --enablerepo rawhide swift-lang
Fedora - Rawhide - Developme  36 kB/s |  12 kB     00:00    
Fedora - Rawhide - Developmental packages for the next Fedora release 2.6 MB/s |  60 MB     00:22    
Fedora Modular 29 - x86_64 - Updates                                   74 kB/s |  16 kB     00:00    
Fedora 29 - x86_64 - Updates                                           45 kB/s |  17 kB     00:00    
Fedora 29 - x86_64                                                     69 kB/s |  17 kB     00:00    
google-chrome                                                          14 kB/s | 1.3 kB     00:00    
Fedora 29 - x86_64 - VirtualBox                                       980  B/s | 181  B     00:00    
Visual Studio Code                                                    7.9 kB/s | 2.9 kB     00:00    
Modular dependency problems:

 Problem 1: conflicting requests
  - nothing provides module(platform:f29) needed by module stratis:1:20180927214347:7a127764-0.x86_64
 Problem 2: conflicting requests
  - nothing provides module(platform:f29) needed by module dwm:6.1:20180813122714:6c81f848-0.x86_64
 Problem 3: conflicting requests
  - nothing provides module(platform:f29) needed by module afterburn:rolling:2920191015163839:c5ebb199-0.x86_64
 Problem 4: conflicting requests
  - nothing provides module(platform:f29) needed by module askalono-cli:rolling:2920190721090701:c5ebb199-0.x86_64
 Problem 5: conflicting requests
  - nothing provides module(platform:f29) needed by module avocado:52lts:2920190514181108:6c81f848-0.x86_64
 Problem 6: conflicting requests
  - nothing provides module(platform:f29) needed by module bat:latest:2920190714171319:c5ebb199-0.x86_64
 Problem 7: conflicting requests
  - nothing provides module(platform:f29) needed by module cbindgen:rolling:2920190515070043:b8a06959-0.x86_64
 Problem 8: conflicting requests
  - nothing provides module(platform:f29) needed by module dutree:rolling:2920190926112243:c5ebb199-0.x86_64
 Problem 9: conflicting requests
  - nothing provides module(platform:f29) needed by module fd-find:rolling:2920190722174030:c5ebb199-0.x86_64
 Problem 10: conflicting requests
  - nothing provides module(platform:f29) needed by module ffsend:latest:2920190630105436:c5ebb199-0.x86_64
 Problem 11: conflicting requests
  - nothing provides module(platform:f29) needed by module heatseeker:latest:2920190921124230:c5ebb199-0.x86_64
 Problem 12: conflicting requests
  - nothing provides module(platform:f29) needed by module hyperfine:latest:2920190721071357:c5ebb199-0.x86_64
 Problem 13: conflicting requests
  - nothing provides module(platform:f29) needed by module lsd:rolling:2920190803124134:c5ebb199-0.x86_64
 Problem 14: conflicting requests
  - nothing provides module(platform:f29) needed by module pretty-git-prompt:rolling:2920190714104357:c5ebb199-0.x86_64
 Problem 15: conflicting requests
  - nothing provides module(platform:f29) needed by module sd:rolling:2920190921120742:c5ebb199-0.x86_64
 Problem 16: conflicting requests
  - nothing provides module(platform:f29) needed by module silver:rolling:2920190912064049:c5ebb199-0.x86_64
 Problem 17: conflicting requests
  - nothing provides module(platform:f29) needed by module skim:rolling:2920190921123929:c5ebb199-0.x86_64
 Problem 18: conflicting requests
  - nothing provides module(platform:f29) needed by module stratis:1:2920190907214611:c5ebb199-0.x86_64
 Problem 19: conflicting requests
  - nothing provides module(platform:f29) needed by module tokei:rolling:2920190921122652:c5ebb199-0.x86_64
 Problem 20: conflicting requests
  - nothing provides module(platform:f29) needed by module zola:rolling:2920190922161148:c5ebb199-0.x86_64
Package swift-lang-5.1.2-0.1.20191107git71def56.fc29.x86_64 is already installed.
Dependencies resolved.

 Problem: problem with installed package docker-2:1.13.1-68.git47e2230.fc29.x86_64
  - package docker-2:1.13.1-68.git47e2230.fc29.x86_64 requires atomic-registries >= 1.19.1-6, but none of the providers can be installed
  - package docker-2:1.13.1-62.git9cb56fd.fc29.x86_64 requires atomic-registries >= 1.19.1-6, but none of the providers can be installed
  - package atomic-registries-1.22.1-27.gitb507039.fc29.x86_64 requires python(abi) = 3.7, but none of the providers can be installed
  - python3-3.7.5-1.fc29.i686 has inferior architecture
  - python3-3.7.0-9.fc29.i686 has inferior architecture
  - package python3-3.7.5-1.fc29.x86_64 requires python3-libs(x86-64) = 3.7.5-1.fc29, but none of the providers can be installed
  - package python3-3.7.0-9.fc29.x86_64 requires python3-libs(x86-64) = 3.7.0-9.fc29, but none of the providers can be installed
  - package python3-libs-3.7.5-1.fc29.x86_64 requires libreadline.so.7()(64bit), but none of the providers can be installed
  - package python3-libs-3.7.0-9.fc29.x86_64 requires libreadline.so.7()(64bit), but none of the providers can be installed
  - cannot install both readline-8.0-3.fc31.x86_64 and readline-7.0-12.fc29.x86_64
  - cannot install both readline-7.0-12.fc29.x86_64 and readline-8.0-3.fc31.x86_64
  - package gdb-headless-9.0.50.20191119-2.fc32.x86_64 requires libreadline.so.8()(64bit), but none of the providers can be installed
  - problem with installed package gdb-headless-8.2-7.fc29.x86_64
  - package gdb-headless-8.2-7.fc29.x86_64 requires libmpfr.so.4()(64bit), but none of the providers can be installed
  - package gdb-headless-8.2-3.fc29.x86_64 requires libmpfr.so.4()(64bit), but none of the providers can be installed
  - cannot install both mpfr-4.0.2-2.fc32.x86_64 and mpfr-3.1.6-2.fc29.x86_64
  - cannot install both mpfr-3.1.6-2.fc29.x86_64 and mpfr-4.0.2-2.fc32.x86_64
  - package gcc-c++-9.2.1-1.fc32.3.x86_64 requires libmpfr.so.6()(64bit), but none of the providers can be installed
  - problem with installed package gcc-c++-8.3.1-2.fc29.x86_64
  - package gcc-c++-8.3.1-2.fc29.x86_64 requires libstdc++ = 8.3.1-2.fc29, but none of the providers can be installed
  - package gcc-c++-8.2.1-2.fc29.x86_64 requires libstdc++ = 8.2.1-2.fc29, but none of the providers can be installed
  - libstdc++-8.3.1-2.fc29.i686 has inferior architecture
  - libstdc++-8.2.1-2.fc29.i686 has inferior architecture
  - cannot install both libstdc++-9.2.1-1.fc32.3.x86_64 and libstdc++-8.3.1-2.fc29.x86_64
  - cannot install both libstdc++-8.3.1-2.fc29.x86_64 and libstdc++-9.2.1-1.fc32.3.x86_64
  - cannot install both libstdc++-8.2.1-2.fc29.x86_64 and libstdc++-9.2.1-1.fc32.3.x86_64
  - package swift-lang-5.1.3-0.1.20191213git005fc1f.fc32.x86_64 requires libstdc++.so.6(GLIBCXX_3.4.26)(64bit), but none of the providers can be installed
  - cannot install the best candidate for the job
  - package atomic-registries-1.22.1-2.module_1637+1872e86a.x86_64 is excluded
  - nothing provides python(abi) = 3.6 needed by atomic-registries-1.22.1-2.module_1637+1872e86a.x86_64
  - package atomic-registries-1.22.1-2.module_1816+29c6bc78.x86_64 is excluded
  - nothing provides python(abi) = 3.6 needed by atomic-registries-1.22.1-2.module_1816+29c6bc78.x86_64
  - package docker-2:1.13.1-61.git9cb56fd.module_2109+7c83ead1.x86_64 is excluded

======================================================================================================
 Package              Architecture   Version                                    Repository       Size
======================================================================================================
Skipping packages with conflicts:
(add '--best --allowerasing' to command line to force their upgrade):
 libstdc++            x86_64         8.2.1-2.fc29                               fedora          452 k
 libstdc++            x86_64         9.2.1-1.fc32.3                             rawhide         627 k
 mpfr                 x86_64         4.0.2-2.fc32                               rawhide         257 k
 readline             x86_64         8.0-3.fc31                                 rawhide         206 k
Skipping packages with broken dependencies:
 docker               x86_64         2:1.13.1-62.git9cb56fd.fc29                fedora           24 M
 gcc-c++              x86_64         8.2.1-2.fc29                               fedora           12 M
 gcc-c++              x86_64         9.2.1-1.fc32.3                             rawhide          12 M
 gdb-headless         x86_64         8.2-3.fc29                                 fedora          3.3 M
 gdb-headless         x86_64         9.0.50.20191119-2.fc32                     rawhide         3.8 M
 python3              x86_64         3.7.0-9.fc29                               fedora           39 k
 python3-libs         x86_64         3.7.0-9.fc29                               fedora          7.8 M
 swift-lang           x86_64         5.1.3-0.1.20191213git005fc1f.fc32          rawhide         191 M

Transaction Summary
======================================================================================================
Skip  12 Packages

Nothing to do.
Complete!

I'm not sure there's anything you can do about this.

Because of Cuda 10's gcc requirements, I'm stuck on FC29 instead of using FC30.

However, FC29 only has Swift 5.1.2, while Swift 5.1.3 requires FC30+.

In particular, Cuda10 = wants older gcc, Swift 5.1.3 = wants newer gcc. I don't see a way to use both at once.

Yeah, when F31 came out Fedora stopped accepting F29 packages. That's standard policy regarding maintaining versions; I couldn't push a new version even I wanted to.

However I may still be able to build it and provide it under my COPR repo: tachoknight/swift-lang Copr. The last version I built was 5.1.1; I'll invoke a 5.1.3 build and if it builds successfully, will show up there.
No promises it will, and COPR builds can take a day or more to build (I get the impression the builders run on the 'meh, whenever I get around to it' schedule), but hopefully you'll have a F29 5.1.3 by early next week.

Not blaming you for this -- can you offer insights on why it seems to take forever to build Swift on Linux ? I was reading about building Swift from source on Linux -- and the time / cpu resources seemed quite heavy. Is there some particular design decision of Swift that causes the compiler / runtime to be slow to build?

I am extremely familiar with how long it takes to build Swift on Linux. :upside_down_face: The biggest issue is CPU type and how many cores you can give it; Ninja will use all the cores it can unless you explicitly tell it to use fewer (though why would you?) and in my case I do not have any machines that offer more than 8; I tried building on a 16-core box but the Xeon chip was so old and slow that it performed far, far worse than the newer-ish 8-core boxes.

Essentially you're bootstrapping the entire thing, soup-to-nuts. LLVM has to be built, then from there Clang (with its accompanying Swift-specific extensions), so that it can build LLDB, for the REPL, then it goes and builds the Swift compiler and accompanying libs, then uses that to build even more stuff....it's actually an extremely impressive how everything works together using Python as the glue between the various components (GYB was not something I was familiar with until Swift).

So, ultimately you're building a lot of stuff. The fastest I've seen it compile is an hour, and that was on an AWS EC2 instance (forgot the specs, but it was a lot of cores). If I'm feeling like I have the money, I may spin up some ridiculous EC2 instance and see just how quickly it can be built (and then quickly shut it down and terminate it :slight_smile:).

I applaud the use of Ninja; at some point I had modified the CMake files to use make instead and, well, I stopped the job after a week.

2 Likes

LOL. I guess when they say "Swift Toolchain", they literally mean the entire toolchain -- everything above the kernel.

Thanks for the insights. I'm going to go ahead and install Ubuntu 18 (I think Cuda 10) also works there. I have 24 cores + 100+ GB of RAM -- but it's an refurbished end of life server from ebay -- so probably not good for building Swift.

Heh, your one server runs rings around mine. :). That's what I use to build Swift for Ubuntu, Fedora, RHEL, CentOS, along with CoreOS for containers for testing. Those things run 24x7, and yes, the power bill is quite high. :upside_down_face:

FYI, the F29 build was successful, you can get it here.

1 Like