We need long-term-support” (“LTS”) releases

overview

for those who don’t follow the Development topic, @mickeyl, @timdecode, and i have recently uncovered an alarming number of highly dangerous stack corruption bugs in the Swift 5.5 release toolchain.

setting aside the technical aspects of the stack corruption issues in Swift 5.5, which are detailed in the corresponding thread, i would like to kickoff discussions on whether it would be valuable for us to adopt some form of the concept of a “Long-Term Support” (LTS) release in our release cycle, much like Ubuntu has been doing for some time.

what went wrong with Swift 5.5?

to summarize, binaries compiled with the Swift 5.5 toolchain currently suffer from a variety of stack corruption-related undefined-behavior. these issues have affected both MacOS and Linux users, and range from merely disruptive (e.g. users experiencing random crashes) to highly dangerous (e.g. callers silently receiving return values that are different from the values actually returned by the callee).

based on my experiences testing the 5.5 toolchain, i estimate that the average Swift application probably now suffers from dozens of instances of undefined behavior, nearly all of which go unnoticed since the data being corrupted is often “unimportant” (e.g. UTF-8 contents of a string), or gets corrupted to a value that is incorrect, but still considered valid by the application (e.g., an account balance getting set to zero.) this problem is compounded by the fact that many of the bugs are highly chaotic — a code change as minor as moving a print(_:separator:terminator:) call up a line could turn a benign stack-corruption bug malignant, and moreover, these bugs often manifest only in release builds, and not debug builds, or vice-versa. this greatly reduces the chance that developers would notice these bugs before releasing applications to the public.

why we need LTS releases

as @Karl has pointed out, the Swift 5.5 release has exposed some very troubling deficiencies in our development-to-release pipeline, which probably requires a revamp of our testing and verification process. this pitch is not meant to detract or minimize the importance of those efforts.

however, we need to recognize the fact that bugs, including severe bugs, are a fact of life in any sufficiently complex piece of software, like the Swift compiler. beyond shoring up our testing and release procedures, we also need to be looking at how we can better serve safety-critical use cases, which require stronger guarantees about the stability of the toolchains they rely on.

async/await in particular, tends to be used in the sort of real-time applications where a subtle stack-corruption bug could have the potential to inflict significant real-world harm. it is not enough to simply fix the issues at hand, and hope it will never happen again. bugs happen. already, the fact that the 5.5 toolchain made it to a public release, and is still available to download on swift.org, is likely to harm the reputation of the Swift language, and i fear that if we do not prioritize release stability, users will come to view Swift as unreliable.

i feel that it would be beneficial for the Swift project to follow the example of the Ubuntu project, and adopt a sort of tiered-release schedule, where every few regular releases is designated as a “Long-Term Support” release, which would receive critical fixes from the mainline, but nothing significant enough to destabilize the product. i think that this is important and necessary to shore up Swift’s credibility as a safe and secure language. what do you all think?

21 Likes

Let me make sure I’m understanding you correctly… Are you advocating for, say, a “Swift N LTS” release that freezes the feature set at whatever it was when Swift N came out, but gets bug fixes from the Swift N.x releases?

If so, isn’t that more or less what setting the language mode flag is supposed to do?

IIUC (and there’s a fair chance I don’t), the Ubuntu LTS releases are more about guaranteeing compatibility and providing a stable “target” combined with security patches, than they are about providing releases with lower bug counts/severity. Do our source & binary compatibility requirements mostly already us give that?

Edit: I agree with your concerns, though. I’m just not sure I understand how the solution I think you’re proposing would interact with our existing compatibility mechanisms.

6 Likes

What if swift5.4 is limited?

1 Like

Swift 5.4 is still available, and anyone on Linux or Windows can use it for whatever they want. It’s only for Apple platforms that 5.4 isn’t good enough, because Apple ships 5.5 with Xcode 13, and you can’t build for the latest iOS without Xcode 13. (And even then it’s possible that a downloadable 5.4 toolchain would work with Xcode 13, though I don’t remember the rules on submitting with that to the app store.)

So this sounds like an Apple policy request, which is perhaps fair and something the community can throw their weight behind, but ultimately not something the community can do much about directly.

EDIT: I’m pretty upset about the stack corruption too. I just don’t see an LTS branch as helping with that problem.

25 Likes

I think that a separate branch would do a lot help maintain adoption of the Swift Language, from my experiences... We are a small team (3 developers + 1 intern) where the 3 of us were able to convince our company to port our web service (from C#) to Swift earlier this year. We use Swift in our server-side implementation, so we aren't limited by App Store restrictions, but up to this point we had just followed the policy of adopting whatever the latest release of the Swift Compiler was that was available... A lot of us are excited about the new Structured Concurrency feature and have been following the previews of it, so when Swift 5.5 was released we began porting our stack to Swift 5.5 + Structured Concurrency. Of course we started seeing strange behavior exactly like what @taylorswift is describing, and when I found out other people were experiencing the same behavior we were of course glad to know we are not crazy after all, but at the same time we are in the difficult position of having to explain why we have to revert most of our work from the past 1.5 weeks and why Swift 5.5 is not okay to use when we campaigned so hard for Swift 5.3... To me, blanket rules such as "subtract 1 minor release" are not helpful for those like us who just want a chance to prove that Swift is a reliable alternative to C#, and do not want to have to account for unreliability around every release cycle. LTS releases sound like a good idea to me because they give more predictability and we would not have to deal with version migrations so frequently. Don't get me wrong, I am all in support of Structured Concurrency, I just wish Apple’s public communication had been clearer and did not lead us to believe that the feature is stable and we would not have wasted so much time.

4 Likes

How do you know the 1st version of the latest LTS release isn’t buggy? Without knowing ahead of time that 5.5 is buggy, what would’ve stopped the Swift team from declaring 5.5 to be the first LTS release? And with knowing ahead of time that a release isn’t ready, why is having a separate LTS release better than delaying the normal release to fix the bugs?

5 Likes

The LTS doesn't mean that it will have less bugs. I personally view it also as a commitment from developers and maintainers to address high-priority issues for that specific release and not for the latest bells'n'whistles. It's the commitment and promise that makes me feel safe to use LTS releases over latest and greatest ones (aforementioned Ubuntu as well as Node.js come to mind).

4 Likes

I think releasing Swift 5.5 as-is was irresponsible.

I also think that it is a serious problem to have language features that are not actually available on every platform running the language. Everything in Swift and its standard library should be available on every platform running the relevant version of Swift.

In light of that, I’d much prefer that older versions of Swift remain the norm on platforms that do not support every bit of new versions.

7 Likes

I’m still not sure I’m understanding this correctly. I’ve never used Node.js, so I don’t know what that term means in that context. My only substantial relevant Linux experience comes from CentOS, where, as I understand it, each major release (“CentOS 6”, “CentOS 7”, etc) would get security fixes and some/all* bug fixes back-ported to whatever version of all the tools and libraries that release originally shipped with, but otherwise nothing that might change the behavior of the system would get updated. As I understand it, this is done to give developers and sys admins a “stable” target. It’s telling them (more or less) “oh, yes, you can just install all the updates without worrying that Python 2, which this release originally shipped with, will get updated to Python 3 and break half your stuff”. It’s about prioritizing compatibility & reliability over “latest & greatest”, which makes a ton of sense for a lot of use cases. When I hear “LTS”, that’s what I think of. Am I understanding the term correctly?

If so, I’m not sure how it helps. The idea is “prioritizing compatibility & reliability”… WRT the “compatibility” part, we already have source and binary compatibility requirements. WRT the “reliability” part, I’m sure the Swift team is trying to fix this bug now that they’re aware of it.

Given that bugs happen, it seems to me that the core of the issue is that testing didn’t reveal the problem ahead of time. So wouldn’t the solution going forward be to improve the testing methodologies? If so, why wouldn’t the same improved testing get applied to non-LTS releases? I don’t think you’re suggesting that regular, non-LTS releases should ship with major bugs, but I’m not sure how that kind of situation isn’t an inescapable consequence of having LTS releases with more stringent testing than the regular releases.

* Probably Off-topic

It wasn’t ever clear to me if “load-bearing” bugs that didn’t have security implications would get fixed, but that’s probably not an issue here.

3 Likes

I agree with David, I'm not seeing the translation of LTS => 5.5 bug fixes.

I think the real request is that we get a patch for Swift 5.5 on Apple platforms / Xcode 13, like non-Apple platforms get. ASAP

I 100% agree that these bugs cannot be allowed to stay until 5.6 or later. There needs to be a patch release to resolve this.

2 Likes

I noted in the other thread that there's a lot of hyperbole going on, and it's counterproductive. You found a bug in an optimization pass for a brand new feature in its first release, and it had a significant impact on you personally. We're sorry that you ran into this problem, appreciate the bug report that made it easy to reproduce and diagnose the issue, and have a fix on hand.

As David has noted, a long-term-stable wouldn't actually help you at all. Either you wouldn't have had access to async functions because you were sticking with an old release, and a new version with new features can always have bugs.

Swift has dot releases that address bugs between major versions. You can see, for example, the timeline for 5.4.3 that picked up a few fixes and finished up around the same time 5.5 was released. For a serious issue such as the one in the thread that spawned this discussion, we'll put it in a dot release.

Doug

41 Likes