Swift 5.7 on macOS Catalina and a toolchain for macOS

I am writing a small Swift project to learn OpenGL and I have an issue with swiftc forcing me to dynamically link all libraries. I understand the reasoning, Apple can then change the implementation of these libraries and it reduces the size of applications. However, Swift is trying to be a general purpose language, not exclusively an AppKit programming language - not everything is an app.

I first ran into this trying to use Task.sleep where using most of its functionality would stop my code from working on older versions of macOS. Ok.
But I actually got upset when I decided to use a distributed actor system for state, so that I could more easily implement multiplayer without completely rewriting my code. Now my code can't run on anything but the latest version of macOS. :eyes:

Why is this limitation still present in the language outside of apps? There seems to be some progress with removing darwin specific features from the language, but no progress on making Swift a language independent of the OS. (again, not everything is an app and needs to be linking against the objective C runtime and friends)

Another issue is that I'm just tired of my cursor stuttering, oversized UI, spotlight removing previews etc, so I would prefer to use macOS Catalina. And I can't find a way to get the latest toolchain and all the features without the latest Xcode (you can't even compile it apparently), which obviously doesn't support any older versions of macOS.
How can I get a proper Swift toolchain on macOS the same way I can on Linux? :slight_smile:

1 Like

so I would prefer to use macOS Catalina
How can I get a proper Swift toolchain on macOS the same way I can on Linux?

It looks like based on this article that macOS Catalina 10.15.4 only supports up to Swift 5.3 with Xcode 12.4. So to get to a macOS version that supports Swift 5.7 it looks like you are going to need to bump to Monterey.

I am aware of that, and I'm on Ventura, that's not the issue. I just want to run Swift code independently of the platform like you can do with Swift on Linux. The toolchain is specifically designed not to let anyone do that. (without manually linking the library and bothering with makefiles)

Things like this make the wider developer community see Swift as just a macOS/iOS programming language.

Why not require dynamic stdlib linking (and the obj-c runtime) only when using Cocoa and other Apple apis, which are the reason for this arbitrary limitation in the first place. I'm not using any Apple frameworks in my project and it still does it...

It's weird that macOS users have a worse Swift experience.

What I mean by the title with "toolchain for macOS" is that the existing toolchains are for Xcode, not macOS. Swift is perfectly fine with nvim or VS Code.

It's unclear what you're asking then. Are you asking how to deploy to Catalina? Are you trying to build a toolchain to run on Catalina?

This is not only about Cocoa APIs. Either you're using libSystem under the hood, which can be only linked dynamically, or you're making syscalls directly, which aren't stable across macOS versions, which is one of the reason why libSystem can only be linked dynamically.

This cascades up to almost every other system library on a higher level. If any of system frameworks need to expose C++ API, parts of libc++ also need to be linked dynamically, and Swift runtime is written in C++.


I started writing a long post about static vs. dynamic linking, but this gets at the heart of what I wanted to say much more succinctly. In general, to get the benefits of static linking, you need to statically link all of the code your executable is going to use — and if on a given platform, the Swift stdlib depends on any system library/framework, you're going to need to statically link against all of those too.

On a platform which has no system frameworks for you to depend on, this is pretty simple: all of your code needs to be self-contained anyway; on a platform which requires you to use system frameworks to interact with the OS, you can't get away from linking against those libraries, which means that you're really tied to the OS.

1 Like

I want to avoid unnecessary dynamic -lobjc and link the stdlib statically like on Linux, and for example have distributed actors work on Catalina, why shouldn't they.

Not being able to write that code on Catalina is a different issue which I just mentioned as a side note.

Considering how other languages like C, C++ or Rust figured it out (apparently Rust can run on 10.7) I see no reason why Swift couldn't work on a 4 year old version of macOS.

I am dynamically linking SDL which handles all the macOS system specific things, my code does not need to do that. All that is required is for SDL to be installed on a system, since its api is stable across versions

I don't think that's true. If you need to use a libc or libc++ API that's only available on a newer OS, you want be able to deploy to an older one. This has been discussed at length on the Forums previously, I recommend checking out those topics.

While SDL's API may be stable, system APIs that SDL relies on may change with OS versions. I doubt you'll be able to deploy an app built with a fresh version of SDL back to MacOS X 10.2. While true, 10.2 and 10.15 are apart time-wise, as said in the linked post it's due to how conservative C and C++ are in their use of system APIs. You either use a language that stays fresh and drops support for older OS versions when needed, or use a stale language that can be deployed to somewhat older OS versions as a consequence of staleness.

I don't disagree that system apis change, I'm not saying I should be able to use SwiftUI on Catalina, just that language features themselves shouldn't be locked like this. Is there really something that platform specific about Task.sleep() that makes it unusable on an older version?

That's true, but even touching distributed actors locking out Monterey is a bit different than a version from decades ago :slight_smile:

You can definitely backport code with a small enough surface area which is stable across previous system versions that it is portable — but if you need a slightly larger surface area, it's eventually going to stop working. (e.g., why can Rust backport to 10.7 but not 10.6, 10.5, or macOS 9? Likely, there are APIs it requires that at some point, did not exist, or did not behave in a compatible way.)

The specific feature you're looking to use on older OSes does not have a small surface area, and requires frameworks which themselves ship as part of the OS. e.g., on Darwin platforms, the Swift stdlib uses the closed-source Darwin version of libdispatch for Concurrency, which itself might rely on libsystem for OS-specific integration. In order to do this, you'd need OS independent versions of all of the libraries themselves — and again, this makes total sense if you're writing with portability in mind from the get-go: but libraries on Apple platforms have historically not had a reason to be written with portability in mind because they're written for Apple platforms specifically (and in doing so, can benefit from OS-specific optimizations, tight integration with other system components, etc.).

Is it possible to write a toolchain which uses all of the open-source components on Darwin platforms in order for the final executable to be portable even on Darwin? Absolutely. But: that executable couldn't then rely on any system frameworks, and runs the risk of behaving differently from the same code compiled and dynamically linked against what's actually running on the system. In all, it's just a somewhat niche use-case that simply isn't prioritized — but it could be made to work if you really want, and are willing to behave differently from the rest of the OS.

1 Like

That's fair, and if that's a priority, sure.
I don't understand why new additions to Swift, now that it works on Linux and Windows, still choose to depend so heavily on Apple frameworks?
If Swift is to expand to more use cases, such as embedded systems, it's going to be more difficult than just using another language if it requires reimplementing a large portion of the library.
But if that's the direction Swift chooses, it would at least be nice if Swift packages could choose a custom implementation of the library, without recompiling the entire 80gb toolchain.

Would you prefer Swift on macOS instead of relying on system frameworks to reimplement those frameworks from scratch and ship them as a part of Swift distribution and apps built with Swift, increasing their size and duplicating this functionality on user's storage devices?

To be clear, are you asking why Swift uses Darwin frameworks when running on Darwin platforms? Because the answer is pretty simple: they're available, performant, and save a lot of engineering effort that's already so limited.

Swift doesn't duplicate efforts on other platforms because it wants to, but does so in places where it has to. There's been plenty of past talk on the forums about trying to reduce the amount of work Swift has to do on other platforms, and integrate more closely with the OS for this reason. It's a lot of work to maintain various implementations of libraries so there's a consistent way to implement Swift on various platforms.

I'm not quite sure what you mean by this. On any platform which offers system libraries, Swift can be ported to use those system libraries; if the platform doesn't offer usable system libraries (and thus requires portability), then the existing libraries need to be ported to work on that system.

An embedded system is more likely to have stable libraries written specifically for that system that Swift could rely on, allowing it to avoid needing to statically link anything.

1 Like

I would argue a statically linked binary is still a lot smaller than bundling chromium with every app, which the majority of apps, which are written in cross platform frameworks do anyway.

Why does it have to be one or the other? :slight_smile:
Especially now that even iPhones come with 128gb as standard, saving megabytes per app is not as important as it used to be.

I'm saying the time saved by implementing things just leaning on existing, non portable frameworks will have to be wasted every time a new platform is to be added, which is somewhat easy to see in the number of platforms Swift compiles to.

I guess that's fine, just not something I see very many people wanting to do when there's existing languages that consider portability more critical. Why port Swift to a new platform if one can get most of the features with Rust or C++ without investing in anything?

I find this sad because Swift is so much nicer to use than competing languages, but (currently) no more portable than Java.

There's give and take, sure. If every platform has platform-specific frameworks, then yes, you have to adapt to those frameworks every time — there's a cost to it. But there's also a cost to having to port some platform-agnostic library which offers the same features, and a benefit to be gained by integrating with the platform closely: you're a good platform citizen, behave like other binaries on that system, and are more likely to be forward-compatible by default.

Porting features backwards is always going to be a pain, but something you have to keep in mind is that most systems want to be backwards-compatible, so that keeping older binaries working is much easier. The Go programming language somewhat famously learned this the hard way when they tried to statically link everything into their binaries, and entirely avoided linking against anything from the system on macOS. This included making syscalls directly, which Apple makes no guarantees about, and this meant that after certain OS upgrades, Go binaries broke entirely because they were relying on intentionally-unstable platform features. Go now makes syscalls by dynamically linking against the system libc, to avoid this sort of thing.

I think you're straying further away from the original intention of this thread, and I think this is also simply not true: Rust and C++ also have to get ported to new platforms, just the same. There's nothing magical there, and nothing comes for free. Same with Java, same with Swift, same with all languages.

1 Like

I think you're conflating Swift's general cross platform support with Apple's support of their own platforms. While it's somewhat ironic that Apple's platforms are actually less supported than Linux, it's Apple's choice. Technically, it may be possible to port Swift on Linux back to macOS (using POSIX libraries for threading, etc.), no one's done it since Apple would never allow its use on the App Stores. But this is really unique to Apple's platforms and not an issue with Swift generally.

1 Like

I think you misunderstand what I meant. The point was that Rust is much easier to port and C++ existed for decades, both languages have the advantage of already working on most platforms. Swift has existed for roughly the same amount of time Rust has, and it supports Darwin, a few distributions of Linux (half of which are Ubuntu) and to some extent Android and Windows. Rust can compile even to useless platforms, like a GBA.

If I wanted to write a game, and I used Swift because it's a really nice language, and then at some point decided to release it on the Switch or some other console:
Do I waste time I could be spending on the game making a new toolchain with no documentation or interest from anyone? or do I just use C++?

And yes, this thread is going in a different direction than intended, the issue was that my code can't even run on Monterey, a version of macOS from a year ago, not that it can't run on a Switch.

I find it fascinating that a more portable version of Swift that supports more devices would be banned from the store, but when if I click "show package contents" on the AppStore version of WhatsApp I find 200mb of Electron :grin:

Strange priorities I must say.

Sorry, you’re right, it’s more the friction of supporting such a thing in general, not that it would actually be banned.