The current state of Swift for Server and Linux

Hi there! I would like to start a discussion about the current state of Swift for Server. The real "for server" - No Darwin.

Why is it advertised as a language for servers if it barely has any tooling for actual server development? Let me show you the main issues.

  • Swift IDEs for Linux is a piece of work.
    There aren't any Swift IDEs for Linux, so you have to use plugins. CLion, a very good IDE that works on Linux has a Swift plugin. The problem? It's not very good. A project of mine didn't even load at all, not even syntax highlighting. SourceKit-LSP is a thing, but it's still a WIP. There's no highlighting, no formatting. If you need any of those things, you'll have to download even more stuff (if it's available for platforms other than Darwin).

  • Swift depends way too much on XCode.
    Most Swift examples and packages depend on XCode. There's no XCode for Linux. This means that if you want to learn Swift and you don't own a device running iPadOS or macOS, you'll need to figure out everything by yourself. Reading tons of docs and code, even though you haven't ever used Swift before.

  • There's no consistency with dependencies across platforms.
    If you need a crypto module you can just use import CryptoKit on Darwin. However, if you're using Linux, you'll have to add apple/swift-crypto to your dependencies and import it as Crypto. That's not very "wow", but if at some point you add support for both Darwin and Linux without compromising speed, you'll probably have to do something like this:

#if os(Linux)
import Crypto
import Glibc
#else
import CryptoKit
import Darwin
#endif

That's not very good.

Those issues (and others that I didn't talk about) prevent Swift's ecosystem from actually growing and they also break one of the basic purposes of the language. Let's take a look at Swift.org - About Swift.

Swift is a general-purpose programming language built using a modern approach to safety, performance, and software design patterns.

...

Fast. Swift is intended as a replacement for C-based languages (C, C++, and Objective-C).

I know it is supposed to be an alternative to C-based languages but that's not possible, not while these issues are a thing. Swift is supposed to be modern, but this isn't modern at all. You can use C just fine on both Linux and Darwin. This shouldn't be an exception for Swift.

I hope this topic finally starts a discussion about bringing Swift frameworks and tools that are currently only available for Apple platforms to Linux. Swift is such an amazing language, beautiful and fast, but you can't use it for server-side programming if you're a Linux user. Period.
(also, I'm sorry if I sound offensive. It truly isn't my intention. English is not my first language and Grammarly's tone detector says it sounds just fine LOL :sob:)

18 Likes

Without commenting on your other points: you can actually just import Crypto and it will use CryptoKit on macOS and Crypto/BoringSSL on Linux.

Source:

Swift Crypto compiles in two distinct modes depending on the platform for which it is being built.

When building Swift Crypto for use on an Apple platform where CryptoKit is already available, Swift Crypto compiles its entire API surface down to nothing and simply re-exports the API of CryptoKit. This means that when using Apple platforms Swift Crypto simply delegates all work to the core implementation of CryptoKit, as though Swift Crypto was not even there.

6 Likes

Hello.

I respectfully disagree with some of your assessments.

  • As a person who uses Swift on both macOS (as a profession) and on Linux (for purposes of college thesis and contributing to open source) I have had really good experience using Sourcekit-LSP. I always use the latest release (and sometimes development) versions of the toolchan and I have been able to comfortably use Sourcekit-LSP via VSCode. I have been working on project using autogenerated code, that had more than 600.000 lines in total (taking to account parsed documentation). At this point, the stability of the LSP was at least comparable with my experiences on Xcode, where I have to restart or rebuild project on occasion too. My setup (complete with LLDB) consits of 3 vscode plugins.

  • I do not know what do you mean by “most Swift examples and packages depend on Xcode”. There is a complete Swift guide available on swift.org which is in my opinion more then adequate for people getting into Swift. In case you want to expand your knowledge beyond that, you can buy books from objc.io or other sources. If the “examples” you’re talking about are from developer.apple.com, then such an example certainly demontrates an Apple API or Apple specific feature and as such is not compatible with non-Apple platform. For more advanced developers, I would also recommend to check out the Swift Evolution repository.

——
As for “bringing frameworks and tools that are currently only available for Apple platforms to Linux” I am highly skeptical. We already have things like Dispatch (GCD, libdispatch), Foundation and FoundationNetworking (which are improving a lot but still somewhat buggy) and thats probably all we will ever get. Low-level things like Core[anything] will never get ported. However various “frameworks” are sometimes re-implemented (like Pulse Audio vs. Core Audio, Tokamak vs. SwiftUI, GNUStep vs NeXTSTEP).

10 Likes

Hi there! Disagreeing with what I think is fine; you just have a different vision. Different visions help evolving projects while keeping them accessible for everyone.

Let me go over some of the points you made:

  • "I do not know what do you mean by most Swift examples and packages depend on Xcode"

I am talking about developer.apple.com too. Some of the examples in there don't necessarily depend on Apple-specific features. The Swift Standard Library documentation @ Apple Developer, for example. The standard platform is not specific to Apple platforms but the first thing you see when you open the website is an XCode playground, and that happens with a lot of topics that aren't related to Apple devices at all. That's not very good for people who well, don't have XCode. You'll either have to check the examples at swift.org (the website is very good, but it's not the first thing you'll see when you search for "swift docs") or going through tons of docs to find what you're looking for at Apple Developer.

But it's not only them. An example of other Swift stuff being way too dependant on XCode (and Apple in general) is the Swift blog. The release note for Swift 5.4, for example. If you take a look, you only have 1 way to "quickly" see what will change on your code - and that is by using an XCode playground. You can also either read the Swift book again and find what changed compared to an older version of the book or read every proposal listed in the release notes. (And no, this is not an isolated case. The same thing happens with the release notes of 5.3, 5.2, 5.1, and 5.0. It probably happens with every version, but I didn't check all of them to claim that).

Now, you may think that this isn't that big of an issue, but it is. If you take a look at swift.org, it says that one of the goals is "to make programming simple things easy, and difficult things possible". In the current state, it's not simple at all. The standard lib, for example, is very easy to use, but yet you have to scroll through dozens of unrelated functions to find what you're looking for if you can't use the playgrounds.

  • However various “frameworks” are sometimes re-implemented

This will probably be very troublesome in the future. It will divide the Swift ecosystem between Linux-specific packages and Darwin-specific. This is not good considering how quality between packages may differ across platforms and how new Swift developers will be confused when they first meet the whole thing. It's not just a good solution in the long run.

3 Likes

What do you consider to be the Swift ecosystem? All the frameworks mentioned previously that are highly unlikely of being ported to Linux, Windows (Core*, AppKit/NextSTEP,etc.) are all part of the Apple ecosystem that are accessible by Swift via a Swift-compatible API, but, they also are accessible via ObjC/ObjC++/C/C++. They are not part of what is considered to be the Swift ecosystem.

SwiftUI and Combine are also components of the Apple ecosystem even though they are written in Swift and usable only in a Swift program. However, language upgrades developed by Apple needed by SwiftUI and Combine (e.g., property wrappers, result builders) have been incorporated into the Swift ecosystem via the Swift evolution process.

It's really not any different than writing a C program to work on Darwin, Linux, FreeBSD, Windows.

3 Likes

@lyricalsoul I can understand your frustration, I'm in your boat as well, currently fighting against CFSocketStreams not being implemented on Linux, #selector and objc_setAssociatedObject not working due to the missing Objective-C runtime, and so on. A lot of little things which make cross-platform projects very hard – certainly harder than with other languages. And while I'm also using CLion, I'm not a fan.

Then again, I'm afraid there's no-one to complain about that, really. The fact that Apple has released the language and a lot of frameworks (and pays people to continue developing it) does not mean that they have any obligation whatsoever to make it work as well as on their platforms – hence, they can't be blamed about the lack of nice and snappy native IDEs.

Also, the documentation is in pretty good shape, if you ask me. Swift.org - Documentation contains a wealth of information, most of it well written and understandable.

You really have to distinguish between Swift, the language, and Swift, the way to write apps for Apple platforms. They happen to share the same language and some of the tools, but at the end of the day they are very different things.

Of course this will always lead to some friction, as it did in the past, but still, there is so much to love in Swift, the language and tools, and this community, that it is worth going the way.

7 Likes

Wrt CLion/XCode alternatives, did you try Visual Studio Code + Maintained Swift Development Environment - Visual Studio Marketplace ?

1 Like

An example of other Swift stuff being way too dependant on XCode (and Apple in general) is the Swift blog. The release note for Swift 5.4, for example

I think that was a bit unfair, they linked to the playground which was a nice way for a large part of the audience to learn more hands on - if you actually followed that link there was directly a link to this article by Paul Hudson: https://www.hackingwithswift.com/articles/228/whats-new-in-swift-5-4

1 Like

I can also say that coming from just looking at what's available for doing actual server development, I was pleasantly surprised how many frameworks were out there already supporting a lot of the basics for it.

E.g.

Swift NIO
Swift Argument Parser
Swift Service Lifecycle
Swift Log API
Swift Log backends like Puppy
Swift Distributed Tracing
Swift Metrics
Swift Metrics Extras
Swift Statsd client
Swift Service Discovery
Swift Protobuf parser
Swift JSON parser
Swift YAML parser
Google flatbuffers parser
Swift AWS Lambda function execution
Swift integrated span tracing for Jaeger
Swift integrated span tracing for OpenTelemetry

etc

9 Likes

I too am skeptical about the future of Swift on Linux. I’ve managed to install the latest toolset onto Ubuntu 20.10 only to run into some of the same basic development support issues.
The first encountered was is there even a Swift aware editor. Turning to my old friend emacs, after an hour of searching I found an ancient, abandoned Apple swift-mode.el (essentially useless now) and one other attempt on github that was several versions out-of-date with v5+ Swift.
I’ve looked at some of the more omnibus environments such as Anaconda but have found in every case so far no Swift support for editing let alone an ide. I’ll look into Clion and if there is some Visual Studio plug-in support but from what I’ve read here, I’m not expecting much.
Xcode would be great to have but I believe you will see a port to iOS and a merging of macOS on iPads long before any port to any Xnix platform.
I’ve developed on Apple machines since the Lisa ran the Macintosh OS from diskette and feel fairly confident that Apple only cooperated with OS groups when it will expand Apple’s application base significantly and then only in the hopes of getting converts to Apple’s platforms. Read that as there will only be a Swift environment major push to general use if there a need on the part of Apple. I don’t blame them since it’s their money but the pretense of Swift becoming a widely used language is a bit irritating.
My experience since 1982, tells me that REPL is the closest we will likely see to multi platform support.
The next thing I ran into was, just what are the functions, data types, etc. that survived the voyage to Linux? In writing my first simple program, I needed to know if a random number generator made the crossing. Thanks to Google (yeck), and a tutorial program on my iPhone I was able to find one without digging through the volumes of documentation available or having to buy a collection of outdated “how to” books.
Thanks to the strong data typing (which I’ve advocated for decades since Modula 2) I didn’t have to discover how to use the debugging capabilities yet but don’t look forward to the first time.
All this and more are obstacles that I just don’t see new programmers or people trying to sample the choices of “best language” for their dream projects (yes, we all have them). I’ll keep exploring Swift and other languages in my retirement because I have time and curiosity but I don’t see the vast majority of non-Macintosh users giving Swift more than a glance and maybe a sigh as it is too much trouble for them to bother with.

I gather this is unavoidable with regards to private Apple frameworks. Without Apple doing it, the best other developers could do is attempt to match the full API of Apple specific frameworks and playing catch trying to sync with the latest updates. As far as I can tell, unless Apple open sources their private frameworks, the best that could be done is for Apple to do like they did with Swift Crypto, where it automatically forwards to Apple's internal framework on macOS and iOS and provides a native implementation on other platforms, all while being maintained by them. Nevertheless, I feel like Apple is increasingly expanding the number of packages available for Swift that work independently of the operating system, which should subsume this specific issue. I should say though that most of the packages that don't depend on Apple's internal frameworks should work on all OSs supported by Swift, especially the server oriented ones! What frameworks specific to Apple's platforms would you like/expect to see available on Linux that would currently require the if #else dance?

I checked out the article and it does seem very reliant on SwiftUI for the examples and explanations though, which would be a set back to anyone not familiar with it such as people coming to Swift from outside the Apple ecosystem. I am not saying I expect Paul Hudson to write articles that have no mention of Apple specific APIs, just that the OP does still have a very valid point in my opinion.

I haven't experimented with Swift on Linux in some time, but if you read some of the other replies above, it seems that Visual Studio does have plug-ins available for Swift development and that apparently they work pretty well. Sourcekit-LSP is a language server being developed for exactly this use case of enabling, as much as possible, the same experience that you have with Swift in Xcode, but for any other IDEs/editors in whatever operating system it can be compiled to (I may be wrong but I think Xcode uses it internally? or at least I think that is the plan).
You say you are not expecting much from Visual Studio support based on what you read here, is there a reason for that? Like, are there features you expect not to work because they weren't mentioned in the previous replies? This could be really good feedback to give, in case those features are not available already.

The rest of what you wrote about Apple not really being that interested in Swift working well on the server I think is misguided. I think they are investing a lot in it, it might just not be visible enough in this forum or elsewhere. I'm pretty sure I read somewhere they are using Swift on the server internally, so they should be invested in it. Last year they announced support for several new Linux based operating systems for Swift. There is the Swift Server Working Group (SSWG) that is trying to manage this space, SwiftNIO and a bunch of other packages developed specifically for use in server environments as mentioned by @hassila above and I think Amazon is supporting it too (they were (partially?) behind bringing swift support to Amazon Linux 2 if I'm not wrong, and the Swift AWS Lambda Runtime package). I think the folks who work with Swift on the server the most usually hangout in other channels where they talk more casually, like on a Discord or Slack channel. Maybe someone can share a link to those channels here.

Overall I think Swift on the server is on a good path. Note though that I'm not saying that there is nothing to improve — if anything the projects mentioned here could become more visible, "official" tutorials and guides should become available as well (I'm pretty sure the SSWG is in the midst of tackling this) and other improvements should definitely be made.

I think (almost?) everything in the Swift standard library is available on Linux. Foundation is where there are some unimplemented APIs or diferences in behaviour. A random number generator is also one of the APIs from the standard library, so it is available on all supported platforms. Hopefully this was the one you found, although I would have thought searching in the documentation first would have yielded results sooner and with less hassle than looking for it elsewhere on the internet or through a tutorial program for the iPhone.

I still think it is important to have these conversations so we can better understand where people are coming from and how we can make it easier for them to get started with Swift whether it is on the server, on macOS, on iOS, on Android, on Windows, on the web (SwiftWASM), on embedded devices, or on any future platforms that become available.

7 Likes

Sourcekit-LSP is a language server being developed for exactly this use case of enabling, as much as possible,

They also have an extensive list of editors that supports it and instructions on how to get Swift completion in your favourite editor.

3 Likes

I'd also want to point out GitHub - swift-server/guides: Guides for building, debugging and deploying Swift Server applications which I had missed before, provides a number of useful pointers and is a community effort to capture Swift-server knowledge.

3 Likes

This should have been on swift.org, though.

Agree with the assessment and sentiment. I am not doing Swift server development, but am extensively working on Swift with Linux. I've wrote it in Data Science Setup with Swift in 2021.

  1. Visual Studio Code kinda works, but I need to recompile lldb to fix some symbolication issues (it is a Ubuntu 20.04 LTS, pretty standard distro). SourceKit-LSP works great with SPM, but if you use alternatives (Bazel is much much better for monorepo and cross-language dependencies), you are on yourself. The Build Server Protocol support is idling for 2 years without much movement.

  2. Most open-source projects are very much iOS / Apple centric. Combine should be open-source and cross-platform, but it is not. (SwiftUI would be great to do the same, but maintaining a GUI kit is much much harder, so this is understandable).

It certainly doesn't help for 2 when some of the foundational components are Apple-only (like the CryptoKit).

But I do see things are changing. There are bunch of Apple-published Swift libraries that in semi-standard status: (crypto, argument-parser, atomic, algorithm, data structure etc), these are all Linux-ready. To turn open-source community to be more Linux-friendly, that will be a bit uphill (and if you have a good amount of open-source libraries to smooth out the #if #else macro dances, it is not that bad to import Glibc and Darwin differently).

As for me, I am currently working on enabling Apollo-GraphQL client on Linux, want to pool a small amount of money to enable Swift for Apache Arrow support among other things.

Concurrency & Actor mainlined will be a major event down the road for Swift on Linux.

5 Likes

I'll try to answer this while balancing all my different hats! Being someone who works full time on server-side Swift, on the core team for Vapor and part of the SSWG I have a very biased view, but one I feel that has a lot of experience (without trying to sound too big headed here :joy:)

To start - the current state of building server-side apps on Linux in Swift is very good. You can build large, distributed, performant apps that serve thousands of concurrent users. I know because I am. The libraries are there for 95% of people to do whatever you want. And sure I've ended up building a lot of stuff from scratch myself, but a) it's possible to build all that stuff and b) anyone starting now has access to full web frameworks, SDKs for AWS and GCP etc. There are a few areas where this falls short, specifically alternative databases like Cassandra and any kind of messaging queue driver (though I know @adam-fowler has his MQTT implemenation and I hope this gets pitched one day!).

In my personal opinion there are three things that need to be worked on to get Swift to somewhere like the likes of Go:

  • tooling
  • documentation
  • messaging

Tooling

I think everyone here is in agreement that whilst it's possible to develop apps on Linux, it's not the most pleasant experience. Installing Swift is fairly painless these days but still quite involved. And getting an IDE set up involves lots of hoops. Everything mentioned in here and on other threads in definitely valid and I would love to see Swift bundled in apt/yum/pick your package manager of choice. As well as awesome tools to manage different versions and extensions to make setting up VSCode or anything else seamless.

Not only would this make the experience for anyone developing on Linux, it would also help both Windows and macOS. Not every developer needs (or wants) Xcode for developing Swift and having the option would be great.

Documentation

This is something the SSWG are actively working on, but building great documentation for both beginners and packages, guides and tutorials is important. The earlier example of CryptoKit is a good one - there's clearly some confusion there about which one to use and what it's for, where in reality for 99% of people you can just do import Crypto and your code will work on all platforms.

Another issue is the duplication of documentation on developer.apple.com and swift.org - hopefully swift.org can be expanded and developer.apple.com focused on documentation for writing specifically for Apple platforms.

Messaging

It's clear that there's certainly a perception difference in how some view Swift and what's it's capable. Plenty of examples from Twitter, here and Hackernews show that plenty of people are unaware of the maturity of Swift on Linux and all the packages available. Most of these should hopefully be solved by by improving the two points above.

One aspect that will really help IMO is people sharing examples of how they are using it in production. Real world use cases of actual system built and running in Swift will go a long way to proving that it is viable on other platforms. As you can imagine, I'm actively working on this part!

In conclusion...

To sum up, there are a lot of very clever people working on Swift on the server and building awesome stuff. The change of pace and improvements of Swift on the server only continues to speed up and I'm excited to see what the next year brings!

37 Likes

I fully agree. I think it is better to have most of the swift documentation available in swift.org. One piece of documentation that I couldn't find in swift.org, that I wish was there, is the standard library reference. That same documentation is available on developer.apple.com, but it is buried somewhere less accessible to people who are not familiar with developing for Apple's platforms.

In summary, the documentation for Swift is scattered across two websites, with different levels of discoverability. In my opinion Swift - Resources - Apple Developer should avoid stepping on swift.org 's toes and do a better job of guiding people there, and swift.org should have more of the documentation that is currently only available elsewhere.

Another painful point is that, as far as I can tell, there is no way of performing a search in any of the documentation in swift.org, it just isn't ergonomic to use and I personally always fallback to using my preferred search engine and hoping the right documentation page appears from either swift.org or developer.apple.com. The search functionality on Apple's website also does not allow searching within just the standard library reference for example, so you end up having to look at all the other documentation for Apple platforms before you find what you want; the fallback, once more, is to use your search engine of choice. All in all, not an ideal situation in my eyes.

12 Likes

I wanted to document how easy it is to set up a working swift environment. To give it the most chances, I used a brand new virtual machine with pristine ubuntu, so that there are no problems with old versions of programs, conflicting versions of programs, any hardware issues. Here's how it went:

  1. virutalbox said that install failed when I tried to run the VM the first time. I had to uninstall virutalbox and install it again.

  2. During Ubuntu installation I clicked cancel on the prompt warning me about my disk being overwritten. Disk formatting wouldn't work after that. I rebooted the VM, and it worked after that.

  3. apt search swift doesn't find swiftlang, let's go to the swift.org to download the toolchain there

  4. "Getting started" page has a section "Installing" with subsection "on linux". I assumed it contained instructions how to get started and install swift on linux. It doesn't. Instructions on how to install swift are on the "Download" page. :/ You have to read very carefully to notice that. (you can find scraps of the instructions here, but the complete version is on the Download page) SR-14565

  5. "Installation" section on "Download" page has two subsections number 1 (installing dependencies, and downloading the binary) SR-14566

  6. Why is Ubuntu 16.04 at the top of the list, even though it's older? I almost downloaded that one by accident. SR-14568

  7. Is there a difference between clicking on a "Toolchain" link and "Ubuntu 20.04"? No, they lead to the same url. Why are there two links if there is no difference? Very confusing, especially when for macos version it matters which one you click. SR-14569

  8. Verifying the signature went well! First step that I had no problems with!

  9. "This creates a usr/ directory in the location of the archive." It creates a directory named after the archive, not usr/ but that's just nitpicking SR-14570

  10. "You can now execute the swift command to run the REPL or build Swift projects." This doesn't mention anything about the fact that it will stop working after you close your terminal. It should tell you to add a command to your bashrc SR-14571

  11. Should there be a link to the Getting Started page at the end, to teach people how to use compiler, package manager and repl?

  12. I cannot see any mention of sourcekit-lsp. Let's assume that I know that such tool exists, and I can google the git repo.

  13. "Getting started" section in the README.md is very good. Short, informative, easily found. Good job!

  14. Installing vscode is easy. It has a big button for downloading .deb version on the front page, and it says that it's for Ubuntu.

  15. Why do I have to install the extension from source?:slightly_frowning_face: I don't want to install node and download git repos. I hope sourcekit-lsp won't hate me if I install npm from apt instead of downloading a tar from the website as it recommends.

-- slight intermission for resizing disk on virtual machine --

  1. I compiled and installed the extension without any problems, enabled it, and opened the sourcekit-lsp repository to test it. I got greeted by a giant wall of sourcekit-lsp errors after opening main.swift :/

  2. Maybe the problem is that I never executed swift build before opening the project? Let's try that. Should there be three empty lines and a single full stop in the output? It doesn't look normal. Let's ignore it. Hooray! I have working sourcekit-lsp now!

  3. ...except for the fact that there are errors on importing of some modules... for example "no such module 'SKCore'" in main.swift. Let's ignore that, and get ourselves a debugger.

  4. I'm going to install CodeLLDB because it seems to be popular? and let's make a new, simple project to test the debugger on.

  5. Enabling debugging is pretty easy. When you choose lldb as your debugger and try to start debugging, it prompts you to make a new configuration. It hints where you should type the path to your executable. Slightly annoying that you have to do that, but it works.

  6. Swift doesn't work perfectly with the default debugger. Instead of nice names in the call stack, you get mangled names full of numbers and dollar signs. I know that the solution is to use lldb that is bundled with the toolchain, but someone new might not know that.

  7. Settings page for that extension doesn't have anything that would help with changing the lldb binary. Thankfully CodeLLDB github page tells us the solution (use a special command) that explicitly mentions swift as the use case.

  8. CodeLLDB yells warnings like [2021-05-03T00:23:27.408Z ERROR codelldb::debug_session] name 'equal' is not defined when you hover over anything.

  9. Next day as I'm filing bug reports sourcekit-lsp stops compiling


Resolving https://github.com/apple/indexstore-db.git at main

:28:10: fatal error: 'sqlite3.h' file not found

#include <sqlite3.h>

         ^~~~~~~~~~~

-.- I give up. Swift tools are awful. It's a great language, but pain in the neck to set up on linux. The whole experience was infuriating, even though I did it many, many times already. Every time I find a new pain point. The only straightforward part of this whole ordeal was installing vscode, which is totally unrelated to swift project.

46 Likes
#include <sqlite3.h> 

is a system dependency and needs to be installed separately

sudo apt-get install sqlite3 libsqlite3-dev

should give it to you

2 Likes

While I'm still new to Swift I have a LOT of experience with Macs, Xcode, Objective-C, etc. I also have a LOT of experience with development in the Linux space with C/C++, Objective-C (via GNUstep), PHP, and Java.

Personally I am VERY excited about Swift in the Linux space. Yes, there are places where code I write on my Mac has to be tweaked to build on Linux but given the brief amount of time that Swift has been Open Source it has come a long way!

The future is bright!

9 Likes