Why the new features in Swift 5.7 needs the latest OS version support?

I found that most features of Swift 5.7 are needs the latest OS support, e.g. Regex and extensional any. The language-level features need OS-level support, I can't understand... :upside_down_face:

Runtime support for parameterized protocol types is only available in iOS 16.0.0 or newer
'Regex' is only available in iOS 16.0 or newer

The error message tells you; the feature requires runtime support and Apple doesn’t back deploy new runtimes.

3 Likes

The error message doesn't really answer the "why" they need runtime support.

It's particularly strange for Regex, but I'm guessing it's because there some bytecode interpretter that's built into the runtime. I found this mentioned in SE-0351 Regex builder DSL:

It is worth noting that the internal representation of a Regex is not textual regex, but an efficient pattern matching bytecode compiled from an abstract syntax tree."

3 Likes

Regex lives in the standard library, not in your application. The standard library is part of the OS.

It is quite normal that additions to the standard library require a particular OS version on Apple platforms. There are some exceptions (such as Concurrency), but otherwise it's been that way for a number of years now.

11 Likes

i think the idea is that this saves memory usage via shared libraries?

2 Likes

More importantly, it allows system frameworks to depend on the standard library as well, enabling them to use Swift internally, and in APIs.

Because processes can't reasonably link multiple versions of the same library simultaneously (technically possible with symbol versioning, but fraught with peril for many reasons), if you use any system frameworks (which pretty much all apps do), you have to use the same version of the standard library that those frameworks use — i.e., the one provided by the OS. And on an older OS without the features you want, well, you can't use those features. (Certain features can be backported, but at significant engineering cost.)

13 Likes

may be...

For constrained any types, there are 2 components to the runtime support:

  1. new type metadata, so that constrained any types can be used as generic arguments
  2. new dynamic casting support, so you can cast types to a constrained any type

For many use cases that need to back deploy to a pre Swift 5.7 runtime, you can re-structure the code to not directly need either of these new runtime additions. For example, instead of using any P<T> directly, you can write an AnyP<T> wrapper struct that stores any P<T> internally, but generic arguments and dynamic casting use AnyP<T>. It's still annoying boilerplate, but it's a big step above what you would've had to do to implement struct AnyP<T> before constrained any types existed (see also: the 2500 lines of boilerplate that implement AnyCollection :slightly_smiling_face:).

8 Likes

In my opinion, If it depends on runtime, the runtime should be auto-updated by the system if the runtime is not the latest one and does not include in the app. The runtime should not bind with the OS.

6 Likes

While this is nice in theory, I don't think this would be quite tenable in practice:

  • "should be auto-updated by the system" is easy to say, but may not work the way you want, for at least two reasons:

    1. The security model of Darwin platforms for a few years now (well, on macOS; iOS-based platforms have always been locked down) has had the OS and its libraries stored in an immutable, read-only manner, and updating those libraries requires a full software update in order to change them. Having the standard library or runtime live in a writable location on disk for updating in a different manner means either defeating the entire security model (if OS libraries can link against a writable framework that someone with admin privileges can replace with their own means you have an easily target-able way to inject code into the entire system), or that OS libraries can't link against this library, which leaves us where we are right now
    2. When and how do updates happen? Does launching any process that uses Swift immediately check the web for a new Swift version, or does this happen periodically in the background, like a software update?

    In either case here, a regular software update seems like the most reasonable approach, which leaves us in the same place as updating Swift with the OS — because it's just an OS update, like anything else. (And we know how users love to install those all the time, right? :smiling_imp:)

  • How do we deal with the complexity of versioning, and behavior changes in the standard library? What happens if a system framework relies on the behavior of a version of the standard library that changes from underneath it, introducing a bug? What happens if it's your app that can't deal with a change in that behavior? What happens across major Swift versions, which might introduce breaking changes?

    There is a significant cost to turning an <app version> × <os version> matrix of possible behaviors you need to account for into an <app version> × <os version> × <swift version> one

In all, there are possible ways to make this happen, but the sheer complexity of supporting something like this, in my opinion, will leave us with an experience that is far from the ideal of "wouldn't it be nice if things just updated automatically?"

12 Likes

I appreciate the nuances involved, but there's still something ultimately dissatisfying with the state we're in.

It's ironic that my old macs can run code that depends on the latest versions of NodeJS, Microsoft's .NET, Google's Go, but not Apple's own Swift stdlib.

8 Likes

I don't disagree that it's dissatisfying, for sure, but if OS libraries used NodeJS, .NET, or Go, we'd be exactly in the same spot with those languages for exactly the same reasons. That's the tradeoff that's been made in order to allow system frameworks to offer nuanced APIs with strong support for features in the language we want to use.

8 Likes

I have an immature idea (of course I'm a layman). Maybe the runtime could be included in the app, the runtime in the app will auto-trimmed while installation if the version is less than the system one. It will trigger system runtime upgrading if the runtime version is great than the system one, and apply the new runtime version in the next launch. After the new runtime version is applied, the system will automatically trim all the runtime in the apps that are less than the system version.

I think there are plenty of compelling schemes that could work from the technical side, but at the end of the day, you have to keep in mind the user experience:

  1. Users typically already feel burdened by software updates: they're disruptive, time consuming, and represent risk ("my device works fine right now and who knows what bugs are in the latest version"). It's already difficult to convince users to upgrade software versions when updates involve new functionality, but an "invisible" update that does nothing for them (and may introduce new problems) is a tough sell
  2. Similarly, going back to a spot where all apps bundled their own version of the Swift libraries inflates the size of apps back up again. This was problematic for developers, but even more so for users, wasting bandwidth and making app downloads larger and more time consuming
    • Theoretically, the App Store could have two versions of the app, one bundled with the standard library and one without, and when a user goes to download the app, they can get a pre-stripped version of the app to decrease the size. This would complicate things for the App Store, and for users, who may not know how large an app might actually be

At the end of the day, users don't know or care about Swift; it (rightly) doesn't matter to them whether an app is able to use the latest runtime or not. Apple, as an organization, cares immensely about the user experience, even if it makes life more difficult for developers, including its own — between the status quo, and investing a ton of time and effort to enable developers to use the latest Swift version across all OS versions at the cost of enormous complexity and a compromised user experience, one is clearly a tougher sell. You don't need to convince me that the status quo makes our lives more difficult; you'd need to convince Apple that the tradeoff being made is the wrong one. (And I'd bet a lot of money on what decision it would make as an organization 100% of the time.)

2 Likes

In my experience, the issue is less that users don't want to upgrade, and more that they are not able to, because the OS vendor is quite aggressive about labelling their hardware as obsolete.

But that's not something we can change.

4 Likes

Come on now, we can all point to any number of situations where Apple has made user experience worse for their own benefit. Generally Apple values itself and the money it can make far more than user experience or, especially, developer experience. It just so happens those priorities align every so often. So any argument that Apple's following some sort of purity of experience here just doesn't make sense.

As for download size, one need only look at macOS update sizes to see that Apple doesn't care about user download sizes at all. Or look at how frequently they allow Facebook and other companies to ship multi-hundred megabyte app updates while doing nothing to support binary diffing that could apply those updates in a fraction of the space.

In any case, stripping Swift libraries is how the App Store already works. It strips the concurrency library from downloads on newer OSes, and it strips the Swift runtime library when deploying to OSes that include it.

Ultimately there a number of ways Apple could support better Swift deployment. They just don't want to.

10 Likes

I think it’s a bit harsh to say that someone doesn’t want to do something just because they are not doing it. I’m sure most of us can relate to many things we’d like to do, but aren’t able to for various reasons. When it comes to the real world there’s always a tension between priorities, resources, alternate opportunity cost and other things complicating the pure wishes for something.

6 Likes

You might have a point if I were talking about individuals. I'm not. An organization with the resources of Apple isn't usually bound by resources the way an individual might be, so I see no need for that consideration. Priorities or alternate opportunity costs are both choices. I'm quite sure there are people within various teams at Apple who would like to improve the Swift deployment story on Apple platforms but are prevented from doing so due to a lack of support or leadership directives. So it's not their choice or their fault. But that doesn't mean it's not the choice of someone in the org.

1 Like

This thread feels like it is starting to stray off-topic for these forums...

12 Likes

I think this is a disingenuous interpretation of what I'm trying to say. I'm not claiming any sort of "purity of experience", nor am I trying to say that Apple is perfect. I'm saying that there's a very high bar for compromising the user experience in favor of the developer experience, and given the option, Apple will prefer to benefit the user experience, full stop.


I was going to respond to some of your other criticisms, but I don't think this is a productive avenue of discussion.

Could Apple make decisions that change the outcome here? Sure. Are their priorities, resource allocations, long-term plans, and opportunity costs aligned with what you might think is best? That's not our call. Either way, assuming intent/desire/lack thereof isn't what this thread is about.

1 Like