The current state of Swift for Server and Linux

I'd also want to point out GitHub - swift-server/guides: Guides for building, debugging and deploying Swift Server applications which I had missed before, provides a number of useful pointers and is a community effort to capture Swift-server knowledge.

3 Likes

This should have been on swift.org, though.

Agree with the assessment and sentiment. I am not doing Swift server development, but am extensively working on Swift with Linux. I've wrote it in Data Science Setup with Swift in 2021.

  1. Visual Studio Code kinda works, but I need to recompile lldb to fix some symbolication issues (it is a Ubuntu 20.04 LTS, pretty standard distro). SourceKit-LSP works great with SPM, but if you use alternatives (Bazel is much much better for monorepo and cross-language dependencies), you are on yourself. The Build Server Protocol support is idling for 2 years without much movement.

  2. Most open-source projects are very much iOS / Apple centric. Combine should be open-source and cross-platform, but it is not. (SwiftUI would be great to do the same, but maintaining a GUI kit is much much harder, so this is understandable).

It certainly doesn't help for 2 when some of the foundational components are Apple-only (like the CryptoKit).

But I do see things are changing. There are bunch of Apple-published Swift libraries that in semi-standard status: (crypto, argument-parser, atomic, algorithm, data structure etc), these are all Linux-ready. To turn open-source community to be more Linux-friendly, that will be a bit uphill (and if you have a good amount of open-source libraries to smooth out the #if #else macro dances, it is not that bad to import Glibc and Darwin differently).

As for me, I am currently working on enabling Apollo-GraphQL client on Linux, want to pool a small amount of money to enable Swift for Apache Arrow support among other things.

Concurrency & Actor mainlined will be a major event down the road for Swift on Linux.

5 Likes

I'll try to answer this while balancing all my different hats! Being someone who works full time on server-side Swift, on the core team for Vapor and part of the SSWG I have a very biased view, but one I feel that has a lot of experience (without trying to sound too big headed here :joy:)

To start - the current state of building server-side apps on Linux in Swift is very good. You can build large, distributed, performant apps that serve thousands of concurrent users. I know because I am. The libraries are there for 95% of people to do whatever you want. And sure I've ended up building a lot of stuff from scratch myself, but a) it's possible to build all that stuff and b) anyone starting now has access to full web frameworks, SDKs for AWS and GCP etc. There are a few areas where this falls short, specifically alternative databases like Cassandra and any kind of messaging queue driver (though I know @adam-fowler has his MQTT implemenation and I hope this gets pitched one day!).

In my personal opinion there are three things that need to be worked on to get Swift to somewhere like the likes of Go:

  • tooling
  • documentation
  • messaging

Tooling

I think everyone here is in agreement that whilst it's possible to develop apps on Linux, it's not the most pleasant experience. Installing Swift is fairly painless these days but still quite involved. And getting an IDE set up involves lots of hoops. Everything mentioned in here and on other threads in definitely valid and I would love to see Swift bundled in apt/yum/pick your package manager of choice. As well as awesome tools to manage different versions and extensions to make setting up VSCode or anything else seamless.

Not only would this make the experience for anyone developing on Linux, it would also help both Windows and macOS. Not every developer needs (or wants) Xcode for developing Swift and having the option would be great.

Documentation

This is something the SSWG are actively working on, but building great documentation for both beginners and packages, guides and tutorials is important. The earlier example of CryptoKit is a good one - there's clearly some confusion there about which one to use and what it's for, where in reality for 99% of people you can just do import Crypto and your code will work on all platforms.

Another issue is the duplication of documentation on developer.apple.com and swift.org - hopefully swift.org can be expanded and developer.apple.com focused on documentation for writing specifically for Apple platforms.

Messaging

It's clear that there's certainly a perception difference in how some view Swift and what's it's capable. Plenty of examples from Twitter, here and Hackernews show that plenty of people are unaware of the maturity of Swift on Linux and all the packages available. Most of these should hopefully be solved by by improving the two points above.

One aspect that will really help IMO is people sharing examples of how they are using it in production. Real world use cases of actual system built and running in Swift will go a long way to proving that it is viable on other platforms. As you can imagine, I'm actively working on this part!

In conclusion...

To sum up, there are a lot of very clever people working on Swift on the server and building awesome stuff. The change of pace and improvements of Swift on the server only continues to speed up and I'm excited to see what the next year brings!

37 Likes

I fully agree. I think it is better to have most of the swift documentation available in swift.org. One piece of documentation that I couldn't find in swift.org, that I wish was there, is the standard library reference. That same documentation is available on developer.apple.com, but it is buried somewhere less accessible to people who are not familiar with developing for Apple's platforms.

In summary, the documentation for Swift is scattered across two websites, with different levels of discoverability. In my opinion Swift - Resources - Apple Developer should avoid stepping on swift.org 's toes and do a better job of guiding people there, and swift.org should have more of the documentation that is currently only available elsewhere.

Another painful point is that, as far as I can tell, there is no way of performing a search in any of the documentation in swift.org, it just isn't ergonomic to use and I personally always fallback to using my preferred search engine and hoping the right documentation page appears from either swift.org or developer.apple.com. The search functionality on Apple's website also does not allow searching within just the standard library reference for example, so you end up having to look at all the other documentation for Apple platforms before you find what you want; the fallback, once more, is to use your search engine of choice. All in all, not an ideal situation in my eyes.

12 Likes

I wanted to document how easy it is to set up a working swift environment. To give it the most chances, I used a brand new virtual machine with pristine ubuntu, so that there are no problems with old versions of programs, conflicting versions of programs, any hardware issues. Here's how it went:

  1. virutalbox said that install failed when I tried to run the VM the first time. I had to uninstall virutalbox and install it again.

  2. During Ubuntu installation I clicked cancel on the prompt warning me about my disk being overwritten. Disk formatting wouldn't work after that. I rebooted the VM, and it worked after that.

  3. apt search swift doesn't find swiftlang, let's go to the swift.org to download the toolchain there

  4. "Getting started" page has a section "Installing" with subsection "on linux". I assumed it contained instructions how to get started and install swift on linux. It doesn't. Instructions on how to install swift are on the "Download" page. :/ You have to read very carefully to notice that. (you can find scraps of the instructions here, but the complete version is on the Download page) SR-14565

  5. "Installation" section on "Download" page has two subsections number 1 (installing dependencies, and downloading the binary) SR-14566

  6. Why is Ubuntu 16.04 at the top of the list, even though it's older? I almost downloaded that one by accident. SR-14568

  7. Is there a difference between clicking on a "Toolchain" link and "Ubuntu 20.04"? No, they lead to the same url. Why are there two links if there is no difference? Very confusing, especially when for macos version it matters which one you click. SR-14569

  8. Verifying the signature went well! First step that I had no problems with!

  9. "This creates a usr/ directory in the location of the archive." It creates a directory named after the archive, not usr/ but that's just nitpicking SR-14570

  10. "You can now execute the swift command to run the REPL or build Swift projects." This doesn't mention anything about the fact that it will stop working after you close your terminal. It should tell you to add a command to your bashrc SR-14571

  11. Should there be a link to the Getting Started page at the end, to teach people how to use compiler, package manager and repl?

  12. I cannot see any mention of sourcekit-lsp. Let's assume that I know that such tool exists, and I can google the git repo.

  13. "Getting started" section in the README.md is very good. Short, informative, easily found. Good job!

  14. Installing vscode is easy. It has a big button for downloading .deb version on the front page, and it says that it's for Ubuntu.

  15. Why do I have to install the extension from source?:frowning: I don't want to install node and download git repos. I hope sourcekit-lsp won't hate me if I install npm from apt instead of downloading a tar from the website as it recommends.

-- slight intermission for resizing disk on virtual machine --

  1. I compiled and installed the extension without any problems, enabled it, and opened the sourcekit-lsp repository to test it. I got greeted by a giant wall of sourcekit-lsp errors after opening main.swift :/

  2. Maybe the problem is that I never executed swift build before opening the project? Let's try that. Should there be three empty lines and a single full stop in the output? It doesn't look normal. Let's ignore it. Hooray! I have working sourcekit-lsp now!

  3. ...except for the fact that there are errors on importing of some modules... for example "no such module 'SKCore'" in main.swift. Let's ignore that, and get ourselves a debugger.

  4. I'm going to install CodeLLDB because it seems to be popular? and let's make a new, simple project to test the debugger on.

  5. Enabling debugging is pretty easy. When you choose lldb as your debugger and try to start debugging, it prompts you to make a new configuration. It hints where you should type the path to your executable. Slightly annoying that you have to do that, but it works.

  6. Swift doesn't work perfectly with the default debugger. Instead of nice names in the call stack, you get mangled names full of numbers and dollar signs. I know that the solution is to use lldb that is bundled with the toolchain, but someone new might not know that.

  7. Settings page for that extension doesn't have anything that would help with changing the lldb binary. Thankfully CodeLLDB github page tells us the solution (use a special command) that explicitly mentions swift as the use case.

  8. CodeLLDB yells warnings like [2021-05-03T00:23:27.408Z ERROR codelldb::debug_session] name 'equal' is not defined when you hover over anything.

  9. Next day as I'm filing bug reports sourcekit-lsp stops compiling


Resolving https://github.com/apple/indexstore-db.git at main

:28:10: fatal error: 'sqlite3.h' file not found

#include <sqlite3.h>

         ^~~~~~~~~~~

-.- I give up. Swift tools are awful. It's a great language, but pain in the neck to set up on linux. The whole experience was infuriating, even though I did it many, many times already. Every time I find a new pain point. The only straightforward part of this whole ordeal was installing vscode, which is totally unrelated to swift project.

46 Likes
#include <sqlite3.h> 

is a system dependency and needs to be installed separately

sudo apt-get install sqlite3 libsqlite3-dev

should give it to you

2 Likes

While I'm still new to Swift I have a LOT of experience with Macs, Xcode, Objective-C, etc. I also have a LOT of experience with development in the Linux space with C/C++, Objective-C (via GNUstep), PHP, and Java.

Personally I am VERY excited about Swift in the Linux space. Yes, there are places where code I write on my Mac has to be tweaked to build on Linux but given the brief amount of time that Swift has been Open Source it has come a long way!

The future is bright!

9 Likes

:clap: well done. Thank you. Clearly a lot of work to do to make this actually good for programmers new to Swift on the server.

1 Like

The 14th April SSWG meeting has already set a goal of better tooling (basically, better toolchain installation experience) for 2021. I鈥檓 really looking forward to official tools like rustup for Swift toolchains so I can retire my swiftbox :)

Also, cross-compilation is useful (not necessary because we have Docker) for server side. The current toolchain layout is not doing very well with such use case (the Windows one is better than Linux and Darwin, thanks to @compnerd). I would expect a unified layout (and build script) for all platforms, which has a place for SDKs of different OS versions or completely different targets; and also standalone runtime distributions for a larger range of OSes.

6 Likes

While the concept of Sourcekit-LSP is great, in practice, I think there is room to criticise the way it has been maintained. I've seen quite a few PRs from community members that would really improve the experience with VSCode, but they seem to go unreviewed until the authors give up hope and abandons them.

Some examples:

VSCode task providers: Add vscode extension task provider by satishbabariya 路 Pull Request #159 路 apple/sourcekit-lsp 路 GitHub

This would give you easy access to common commands like building a project or running tests. Currently, you need to run the command line manually in the built-in terminal or manually configure a bunch of tasks (basically, a JSON file with the command lines) for each and every project. I mean, come on... that's a pretty awful experience. There is no way Apple would accept that kind of developer experience on their platform.

It's disheartening that somebody took the time to work on it, and their work was just left to rot. I think we should be ashamed and embarrassed at how that was handled. I mean, look at this:

Anybody who has tried to use Swift with VSCode knows what an enormous improvement this would have been.

Another one, document formatting: implement DocumentFormattingRequest by Trzyipolkostkicukru 路 Pull Request #361 路 apple/sourcekit-lsp 路 GitHub

The original PR is from January 2020. The author even said at that time "I am used to waiting weeks and months" :frowning_face: They took the time to rebase it, but even that new PR has been sitting around since January (it is now May), and nobody has taken the time to review it.

Perhaps SourceKit-LSP needs a new maintainer, because the current situation really should not be considered acceptable. Lots of great engineers are doing fantastic work to take Swift forward, but this project is just not pulling its weight. And it's not because the community aren't interested in improving it.

29 Likes

Hi @Karl

It's fair to say the communication from me on the two PRs you brought up was poor. Thanks for bringing it up. Taking them in reverse order,

I'll follow up on the PR to summarize what I think is blocking this integration. Part of that information was on the original PR (the situation with testing swift-format). But looking back at it, I also see that I didn't write down all the issues that I had in my head. I'll do that now.

VSCode task providers: Add vscode extension task provider by satishbabariya 路 Pull Request #159 路 apple/sourcekit-lsp 路 GitHub

This is more a question of VSCode integration for SwiftPM than about sourcekit-lsp itself. If someone is interested in moving this forward, I think it probably needs to be a separate thread on the forums. I would want to have feedback from the SwiftPM folks on what they think of the integration itself, and whether SourceKit-LSP's VSCode plugin is an appropriate home for it. I apologize for not following up on that PR.

Ben

14 Likes

@blangmuir thanks for bringing it up, in my opinion we need one separate repo for VSCode extension, separate from sourcekit-lsp, where contributors can work and improve the VS Extension primarily. and can be distributed to marketplace.

for example: GitHub - rust-lang/vscode-rust: Rust extension for Visual Studio Code which is separate form GitHub - rust-lang/rls: Repository for the Rust Language Server (aka RLS) and has great feature integration with vscode.

6 Likes

+1 to separate repos for SourceKit-LSP and a VSCode integration.

As an FYI to SSWG incubation process covers tools (like a VSCode plugin) as well as libraries and we'd be very happy to see something like that pitched!

10 Likes

Thanks, @blangmuir. This topic of Swift having a poor developer experience on Linux comes up time and time again, and the VSCode extension is a big part of it. If it could move faster, and be broadened to include more general Swift/SwiftPM-based features which don't relate to LSP by being organised as a separate repository, I think it sounds sensible to do that.

7 Likes

Thanks! Filed a bug about requirements missing from the documentation. Unfortunately it still doesn't compile, because on the snapshot compiler crashes (filed another bug), and on release toolchain it just stops without any diagnostics. :frowning:

Also the package manager sometimes flashes the text * Build Completed! at the start of the build. Filed another bug.

With regards to toolchains鈥 it's also a bit sad that Swift support for armv6 (i.e.: Raspberry Pi Zero) seems to be stuck at 5.1.5 for the time being. I wonder whether the core team would be interested in accepting patches to make this less of a hassle with every new release.

8 Likes

I don't understand 'swift already works great in Linux' argument. No it is not. Compare it to Rust and Go; Swift support on Linux is a joke.

Some volunteers putting a lot of effort but the trillion dollar company behind this beautiful language barely support any non-apple platform.

This 'Steve Ballmer' approach will ruin the great potential of Swift on Linux if the apple management do something about it.

2 Likes

There are many people working on Swift on Linux, both at Apple and in the broader community. I understand your annoyance, but please try to be constructive about things you'd like to see improved instead of broadly disparaging their work.

25 Likes

As someone who works full time on server-side Swift I disagree with this statement. If you feel like there are missing pieces, the SSWG would love to know your (constructive) thoughts, e.g. SSWG Annual Update 2020

14 Likes