Justified concerns about the consequences of ABI stability?

Or have two code paths, just like we actually do for any other API on the platform (which have a stable ABI from the beginning).

That’s ok for trivial features, not so great otherwise. Consider the new string interpolation API in Swift 5. Would you really have time to implement, test and maintain things like this twice knowing how cumbersome it’d be in Swift 4?

1 Like

No more than I had time to reimplement NSCollectionView new behaviour (introduced in 10.11 IIRC), or to backport libdispatch on 10.5. This is not a problem specific to Swift. Any library has this very same issue.

The same apply to languages features, like thread local in C++ which was introduces rather late in macOS and required an updated version of dyld, so a major system upgrade.

It will be frustrating when things like property behaviours or coroutines are introduced in Swift, and it may take some times before we can use them widely.
Fortunately Apple is pretty good at pushing updates on user system, so it will not take too much time before new features are available to enough people in your audience to be able to use them.

2 Likes

Has the core team taken the dev community pulse if ABI stability is something desired at this stage of Swift? I would think most developers would rather be able to use all new language features immediately with a language capable of shedding bad decisions than care about binary compatibility.

I also don't recall any ABI evolution proposal being put forth for the community to discuss and weigh in on.

Yeah developers will always prefer that, but there are multiple points of view to consider.

I think the point mentioned previously about system frameworks being written in Swift is more interesting. We've heard at previous WWDC talks about certain internal components (like the macOS Dock) being partially rewritten in Swift, and the advantages to doing so. I can imagine Apple would like to bring those gains to public frameworks, too.

The more Apple are able to do that, the more they commit to Swift. That's great for the project's health. It's no small thing to be part of a major commercial operating system.

10 Likes

Even though iOS users are generally pretty quick to upgrade looking at the current iOS adoption stats it looks like we’re headed for a 15% tax on users if your app wants to user the latest evolved feature and a lag on when this can be rolled out until sufficient users have loaded the latest OS image. Worse, an application that previously worked will suddenly not work or be un-upgradable or previous versions of the app need to be supported if the user does not/can not upgrade their OS.

https://data.apteligent.com/ios/

I wrote about this in a digression off piste in Prepitch: Character integer literals

The tl;dr; it is the idea of shipping the Swift shared libraries with the operating system image seems to be flawed to me and they are better downloaded on demand by the app store app and installed to a central place in the file system where they can be versioned. This allows for all the advantages of shared distribution including saved resources on the device and allowing Apple to ship system frameworks written in Swift without tying Swift Evolution’s or developer’s hands or somehow hindering Apple’s commitment.

This said, unfortunately the ship has rather sailed on this now with Xcode 10.2 so mentioning this again may not be viewed as being particularly helpful but I have reservations about whether the course we have set is going to pan out - If it doesn’t this idea could always be introduced at a later date.

2 Likes

There was some discussion last year about the possibility of having two components to the standard library: the existing “non-ABI stable” portion that gets bundled with apps, and a new “ABI stable” portion that can be distributed as a system library.

Then over time, as each individual piece of the standard library gets rigorously evaluated, optimized, and ossified, it can be moved from the non-stable component into the stable component.

That way we wouldn’t have this rush where everything gets permanently baked into its current shape all at once, and it would still be possible to modify (and potentially remove) parts that never made the jump to ABI stability.

It also means that newly-proposed features could begin with any implementation that works, without having to worry about getting the exact optimal implementation in the very first attempt. Instead new features would start in the non-stable component, and eventually make their way into the stable component after being optimized.

2 Likes

I remember, but it seemed neither here nor there. Why not just version the whole set of core libraries and Swift system libraries for a particular version of Xcode/Swift in a shared directory managed by the App Store app as the middle road between shipping with the OS and shipping with the app - far simpler to implement and test. Let’s not talk of “ossifying" anything to do with Swift just yet!

1 Like

That does not solve the main problem that ABI stability is supposed to fix: being able to expose Swift API from system framework. If you have 2 incompatibles String types (one used by the main app, one by the frameworks), how is it supposed to work ?

3 Likes

There will be different versions of the system library written in Swift for each Swift version so they run with applications compiled with that Swift version. These would be produced for a version of Xcode then remain static for the future as released applications will not be expecting the API to evolve. If new version of the application are they would use the new version of the Swift framework/core libraries with the version of Xcode they are developed under. The alternative is never being able to change the String implementation in future. If the OS deprecates the underlying C api they use then that’s when you get a problem. It’s a grey area whether frameworks that happen to be written in Swift are part of the OS or part of the Swift version but I’m proposing they are strongly bound to the Swift version people will be coding against when they release their app.

/System
	Library
		Swift
			4_2
				libSwiftCore.dylib
				HealthKit.framework
			5_0
				libSwiftCore.dylib
				HealthKit.framework

The cost of such feature is very high.

First in disk space, as it would requires to have a whole copy of all the Frameworks using Swift.

Second in RAM usage, the system will have to map all variants the frameworks instead of having a single copy.

And also in update size, as each update will have to replace all the versions of a framework. Why fixing bug for a framework using Swift 5 and not fixing it on the same framework running Swift 6 ?

1 Like

It’s high but considerably lower than the cost of all the separate copies of the Swift core libs per app that is the case today. The question is how many pure Swift frameworks there are going to be (N) and how many live versions of Swift (V) there are going to be on a phone at the same time. Admittedly that is two increasing numbers multiplied N * V i.e Quadratic which is not good in the long term but the amount of resources is only ever V times what would be used otherwise if a user has a large number of old apps. Whether old versions of frameworks would (need to) be updated is an open question. But the other side of the equation, trying to fix the ABI, has a cost too in the agility of Swift and its future reliability.

I'm concerned many developers will be caught off guard when upgrading to Swift 5 because their app suddenly again another OS versions dependency. While this probably isn't a big deal ("minimum Base SDK < Swift 5 OS requirement" alert), the expectation from past experience is that migrating to Swift 5 has no other side effects than what one might expect from migrating to previous versions.

I would think the better experience for the developer would be, at the very minimum, to opt into using Swift provided by the system. For example, changing a build setting in Xcode labelled "Links against system Swift runtime" (where other state implies the Swift runtime is bundled in the build product as it in Swift 4.2 and earlier).

I would also think the better experience for our users would be to allow the developer to opt in, as well. Improvements are made so quickly and Swift is very young. For example, crashes fixed in the compiled product between Swift 4 and 5. For another, String performance looks to be improving in leaps and abounds. Closures on the stack. Surely we'd want our users to have these with the least amount of friction as possible?

I don't think I'm alone in suggesting I'd rather spend time building features for users with the APIs out of the box made available by people much smarter than I am. Backporting or reverse engineering APIs strikes me as a luxury (or skill) many developers do not have and sure way of introducing security, performance and usability issues. Also, some platforms may not allow the inclusion of custom or updated versions of libraries like dyld.

Out of my naivety, I'm not sure I buy the argument that somehow system Swift is needed for Apple to offer Swift APIs, either. For one, there's nothing stopping them from offering a package. For another, I remember hearing (no link, sorry!) they're already statically linking Swift for the apps bundled in iOS 12 (eg: Music.app). The only real advantage to date for ABI stability I've heard is that app sizes will be smaller. I realise I'm arguing this is an acceptable cost to the consequences of the alternative.

Generally, iOS users update their devices. But business don't look at the general population for deciding to drop iOS versions. They look at their user base. Some user bases don't update very much or at all. For example, the IT departments of enterprise clients might not appreciate forking out however much it costs to pay their staff to update all their iPad kiosks to iOS whatever just so the developers can write in latest Swift. Even in terms of the general population, consider Instagram. It couldn't be optimised on iPhone XS because Xcode 10 couldn't build asset catalogs for iOS 9. Do we really want developers not to use Swift 5 so we can get products to the widest audience reasonable?

1 Like

The entire thread in Kiel’s first link is an interesting read: A case for postponing ABI stability - #46 by Chris_Lattner. It seems many of these issues were thrashed out two years ago and to an extent the arguments still apply today on both sides. I’m putting forward more of a third way that gives you the best of both worlds but it’ll be difficult to change course now. As I say, it can be adopted later if need be.

It seems entirely likely Apple can fall back to the current status quo of shipping the Swift libraries with apps in order to deploy back to older OS versions that either don't have any system Swift library or have one that's out of date. Then, due to app thinning, only users on those older OSes suffer the overhead of those libraries, and not the vast majority of users who are on the latest OS. Apple, since they're only concerned with one major version of an OS at a time, can ship their code in such a way as to work around changes in Swift, just as they do for Obj-C right now. Their libraries will be the most impacted by incompatible changes to Swift, just as they are in Obj-C (though hopefully it'll be easier for them to implement workarounds with fewer bugs in Swift). So nearly all end users win, devs won't see a different, and Apple continues their library development into Swift.

6 Likes

Thanks for writing so well what I was expecting (hoping).

I’m just speculating. I’m hoping Apple will be more forthcoming with their plans once Swift 5 ships, but who knows.

1 Like

The idea is to improve on the current situation, which is pretty bad, not to keep it the same.

For most iOS/macOS developers, the OS you ship your first version is more or less frozen for ever (or at least can not be changed for the next 4-5 years).

Compare the current Swift workflow:

  • You (or someone else) submits a bug/feature request
  • A couple months later (possibly less) it is implemented
  • A couple months later a new Swift version is released
  • You can now use this new feature (or the bug is fixed)

The time it takes from submition to actually using it is around 6 months, possibly less.

And here is what happens with an iOS/macOS SDK:

  • You submit a Radar
  • Months (possibly) years pass, without any feedback. Let's say you get lucky and it gets implemented 6 months later
  • Depending when we are on the release cycle, it can take anything from 6 to 18 months to actually get released on an OS update.
  • Unlike Apple, you can't really dump older OSes, you need to support at least 2-3 versions back, possibly more. Depending what was your original deployment version, it can be 3-4 years of waiting.

The time it takes from submission to actually using this feature is 4-5 years, possibly more.

6 Likes

It will not be possible to drop in a bundled Swift runtime once programs are compiled against the version in the OS, for a number of technical and practical reasons. In cases where we are able to backward deploy new runtime or standard library features, it will more likely look like the "arclite" mechanism for Objective-C, where a backward deployment static library is linked into the executable and used if the OS it runs on does not have that functionality in the runtime.

8 Likes

I feel that that should cover most things, shouldn't it? For example, there doesn't seem to be a reason why language-level concurrency couldn't be bridged using a shim. Would that even be a design requirement?

There is one thing we can see coming (ish) that such a mechanism couldn't accommodate: move-only types. But that will probably break half the standard library, too.