Is the Swift Source Compatibility Suite still maintained?

Hello,

Recent Swift 6.1 snapshots can't build groue/GRDB.swift, so I rushed to swiftlang/swift-source-compat-suite in order to register a Swift 6.0 compatible commit.

Surprises:

  1. swiftlang/swift-source-compat-suite is currently unable to accept Swift 6.0-compatible submissions: the pul-request checklist can't pass due to lack of support for Swift 6.0. Reported as #963.
  2. I don't see any repository in the compatibility suite which has registered a Swift-6.0 compatible commit.

The last point is very, very, surprising. Did the compatibility suite move to some other place? If not... does this mean that Swift 6.1 is developed without any safety net?

12 Likes

Some off-topic work I'd like to state:

The recent Swift 6 compiler has introduced several accident bug which make it source breaking. It might be caught early in the development process if the compat-suite is still alive.

eg.

1 Like

this also happened during the Swift 6 release cycle:

The Swift Source Compatibility Suite helps the compiler developers ship compilers that do not introduce unexpected breaking changes.

  • Major versions are made for introducing breaking changes. It is thus expected that a Swift 6 compiler (in Swift 6 language mode) no longer compiles some code that was compiling in a Swift 5 compiler. See Source Compatibility for some examples of Swift 6 features that can "potentially affect source compatibility". We can report issues about code that stops compiling, but maybe this was perfectly intended.

    The Compatibility Suite is not intended to help Swift 6 behave as Swift 5.

  • Development snapshots available from Swift.org - Install Swift do not pass the full test suite:

    These snapshots are not official releases. They have gone through automated unit testing, but they have not gone through the full testing that is performed for official releases.

    I'm not sure the Compatibility Suite is run for all development snapshots, or even if a failure reported by the Compatibility Suite can block the release of a snapshot. Very probably not. It is thus expected that those snapshots do not compile code that they should accept. We report an issue when this happens. This may apply to Xcode beta releases as well.

What we do not expect is an official release version of Swift 6.1 that no longer compiles code that Swift 6.0 does (there may be exceptions for rare bug fixes). That's where the Compatibility Suite is supposed to help. In the ideal world, we developers do not even have to report any compatibility issue, because the suite has reported problems before the compiler is released. And this is the reason for this thread: I can't see how it can help if no one can register code that is known to compile with Swift 6.0.

2 Likes

If you're talking about compiler versions (as opposed to language modes), I don't think this is true.

The Swift 6.0 compiler is fully expected to compile all code that was compiling in a Swift 5.x compiler, as long as you stay in the Swift 5 language mode and you don't enable any upcoming features. This is exactly the meaning of source compatibility.

(Upcoming features were introduced to allow people to selectively opt into specific source-breaking features before the new language mode was available, and to ease the migration to the new language mode by picking-and-choosing which breaking changes you want to adopt.)

4 Likes

You are right, @ole, thank you.

I indeed suppose that the Compatibility Suite was run, with a Swift 6 compiler in Swift 5 language mode, on code known to compile with a Swift 5 compiler, before Swift 6 was officially released.

My concern is that the Compatibility Suite does not contain any code known to compile with a Swift 6.0 compiler, jeopardizing the Swift 6.1 release.

EDIT: I have amended my previous message, according to your input, for future readers

4 Likes

the regression in tuple label matching was not an intentional source break introduced in the Swift 6 languge mode (that would have had to go through Evolution). it was just an ordinary type checker bug that went unnoticed for several weeks, and could potentially have ended up in the 6.0 release had swift-png users not noticed it.

1 Like

Yes. That's the kind of regression the Swift Source Compatibility Suite is supposed to prevent (if it were maintained). I wish I did not have to open my latest issue Swift 6.1 regression in Sendable checking · Issue #78635 · swiftlang/swift · GitHub, but I have to, due to the current, non-maintained, state of the suite. I'm always glad when I can help, but the reality is that this is useless work that should have been performed by the compatibility suite.

I really wish someone in charge, from the Steering Group, would come in this thread with some update.

2 Likes

yes, let us hope someone from the Steering Group provides some clarity on the situation. the Source Compatibility Suite is an incredibly important safeguard and it would be very concerning if it were no longer operational.

1 Like

There's a CI builder that continually monitors the status of the suite, so yes, it is maintained. It's common for there to be failures on it for a few weeks, and it does look like we have a bunch of active failures right now.

It looks like there hasn't been a concerted effort to add projects using the Swift 6 language mode, but that would obviously be a good thing. There is an open-source process for adding projects to the suite that's documented on the repository. If there's a reason the current suite doesn't support building under the Swift 6 mode, I'm sure we can get that fixed promptly.

7 Likes

I should note that the suite, by its nature, tests specific commits of projects. We are not going to automatically pull and build the latest commit of your project, for the fairly obvious reason that project maintainers accidentally break their own builds all the time, which is not something Swift maintainers should have to spend their time debugging. If you want to update the commit that's tested for your project, I assume that's a pretty straightforward change.

1 Like

Thanks for your feedback.

That's the assumptions that does not match the reality, as reported in Support for Swift 6.0? · Issue #963 · swiftlang/swift-source-compat-suite · GitHub. The README of the repo is pretty out of date also (not a real problem, but not a good signal).

1 Like

We're almost there! The Swift compatibility suite does not yet ship with the Swift 6 toolchain used by people who submit code, so this triggers false positive concurrency errors: