Pitch: Standard Library Preview Package

This sounds great! I'm a big fan of the toolchains but it's true that they're still not the ideal, this will make it so much easier to try new stdlib features.

Should we go the extra mile and consider this as part of SE for stdlib additions? Something like: Pitch (sandalone package) -> Proposal -> stdlib preview -> stlib.

Could those limitations eventually be lifted in the future? Have the SwiftPreview package made special, able to activate feature flags not only in the regular stdlib, but down to the compiler itself? I think of Python's from future import ... statement, which is very powerful.

2 Likes

No, I don't think so. Or rather, we shouldn't formalize that workflow. Whether a proposal deserves time in a standalone package really depends on a number of factors:

  • The complexity of the proposal. We don't need toggle to sit in a package for any length of time. But maybe SortedSet should.
  • The appetite for a standalone package. It doesn't help if no-one actually uses that package.
  • The urgency of the need. For example, the introduction of Identifiable was driven by community members seeing the protocol in one of Apple's frameworks and thinking it would be good as a standard library addition. This meant it had to go into the standard library immediately to avoid a situation where Apple introduced its own version. This is also one reason why proposals may skip the preview package stage, too.
2 Likes

Maybe, but let's learn to run before we pole vault :). We can see over time how limiting those restrictions become, and then we'll have data to decide whether it's worth trying to lift them.

The main limitation is always going to be the ABI. We cannot easily introduce types into the standard library and integrate them into the compiler without making them ABI, which then means they can't be changed. Hiding them behind a feature flag doesn't necessarily help if you can ship an app that needs to be backward/forward deployable.

I don't think "serious" is the right word to use here, but there's certainly a limitation for packages that need to preserve source compatibility. If all packages need to work with the same version of a dependency, then a package that frequently breaks source compatibility will implicitly cause packages that use it to break compatibility too.

Version pinning doesn't help though, in fact it arguably makes it worse, since it prevents users of the package from all moving forward together even if the changes haven't actually broken anyone.

But this doesn't mean packages shouldn't use this package. It means packages that value source compatibility highly shouldn't use it. But it would be valid for other "version zero" packages to use the preview package. These packages can still be "serious" though: they just impose a requirement on their importers that they too need to stay up to date if they want to use the preview package too.

+1 from me.

Sure and that'll still be possible with branch-based dependencies. I think I didn't mention that branch-based dependencies can be transitive if they're reachable via other branch-based dependencies.

Valid:
MyApp -> MyDep (master) -> PreviewPackage (master)

Invalid:
MyApp -> MyDep (1.0.0) -> PreviewPackage (master)

With the currently proposed zero-based semver approach, I worry a lot of packages in the ecosystem might end up becoming incompatible because the preview package is never going to be source stable. I think it makes sense to have a little bit of barrier since it's not a good idea to rely on this package if you care about source compatibility. In worst case, one of your dependency might adopt this package and now you suddenly end up with a transitive dependency on the preview package.

5 Likes

I think if you release no semantic versions, people (I for one) are likely to just fork and tag their own, due to the nuisance branch dependencies are when they are out of your control. Multiple forks will cause even worse issues if they show up in the same package graph.

I guess it comes down to how you want it to be used. If the preview package were just intended for experimentation like a playground, then discouraging its use as a dependency by having no versioned releases might be a good idea. But the proposal draft says,

Even though all additions go through a thorough review, there is no substitute for feedback from real-world usage in production.

If you want it used in production, then versioned releases seem pretty much necessary, even if the numbers skyrocket due to frequent breaking changes.

2 Likes

Maybe making it follow the minimal major version of Swift necessary to using the package and resetting the minor/bug-fix part of it each time it is changed ?

Quick example:

Let's say this package comes out at the same time as Swift 5.2. Its first public version would then be 1.0.0 (or maybe 5.0.0 though that may be too confusing).
Then a source-breaking change/a new feature to test would be included and it would move up to 1.1.0 an so on.

Then, when Swift 6 comes out and the package starts to depend on it, its version changes from 1.x.y to 2.0.0, restarting the cycle.

To be honest I don't see why semvar can't be used here. Yes, source breaking changes will be common. So what? It just means we'll see frequent changes in major version number. Again, why not? Is it because it "feels" funny?

2 Likes

I'm really happy to see this happening. I think it will be very positive for encouraging community contributions and letting designs settle before the get locked into the Standard Library's ABI.

I think this lag for the main releases strikes a good balance that allows people to use the current version of the package without needing to immediately upgrade Swift. That will encourage use of the package and early adoption of new library features which are important goals.

I wonder though if it would make sense to also support a separate stream of releases that always requires the latest released version of Swift (or later) and drops any symbols that have moved to the library as of that Swift release. This would provide a better alternative for users who do want to update quickly to the latest version of Swift. It would also ensure more immediate usage of the library implementations because users of the "edge" release would not need to modify code to switch to the library's implementation.

I'm really happy to see this choice! If there are features required by standard library tests that are not available in XCTest they should be added, as you mentioned. We will benefit from having the strongest shared testing infrastructure we and this would be a good step in that direction.

1 Like

Yeah, this is the part that I think needs to be clarified. I don't really understand what fraction of changes in this package are expected to be major, minor and patch changes, and also who the intended audience is. I can see that there are arguments for semantic versioning, permanent 0.x.x versioning, and using it as branch-based dependency, but it's hard to understand the tradeoff in terms of usage/adoption, how frequently it will break or cause dependency chaos, etc. These options are also not mutually exclusive, e.g. I think one option would be to use semantic versioning but recommend using it as a branch-based dependency for some subset of users, which would have the benefit of keeping them on the latest version.

Just a note: as has been pointed out already, the “permanent 0.x.x” strategy is not a violation of semantic versioning, since major version 0 is never considered to be stable (and so the choices are not semantic versioning versus non-semantic-versioning current proposal, but rather two different strategies of semantic versioning). I personally think using this strategy of semantic versioning fits better with the spirit of this package as a “staging area” for the stdlib.

6 Likes

I don't think there is anybody who literally does not care at all about source compatibility (even for a preview library) - the difference is how often they are prepared to update their code for breakages.

Nobody wants code that breaks and has to be fixed every single day. That would be completely pointless. One a week or month may be acceptable to some. Others will only tolerate it once or twice a year, or less.

I wouldn’t expect source-breaking changes all that often. This library just makes it possible, it doesn’t make it easy. Just the fact that changes are still beholden to the SE process ensures things are not changing very rapidly.

+1 from me and I think the versioning strategy is good as written.

2 Likes

0.x.y not a violation of semvar, but also not giving us the benefit of semvar. the 0 major version in semvar is there assuming there's an eventual convergence of features which leads to a version 1.

I think if we simply increase the major version number whenever a source breaking change occurs, it not only gives us the benefit of semvar (should I upgrade? oh, I can because major version didn't change!), but also tells a clear story about the nature of the library (oh, it went from major version x to x + 12 in a year!)

Based on the proposal, it sounds like we'll have that:

So, if you peg to a minor version, you'll be able to upgrade across patches without any hassle. The constant addition and one-year removal timeline of API from this package makes fully-fledged semver somewhat inappropriate. Major version N would almost certainly be disjoint from major version N+M for even moderately large values of M.

The semver rules for version 0 make no mention of convergence:

  1. Major version zero (0.y.z) is for initial development. Anything may change at any time. The public API should not be considered stable.

This package will be in a constant stage of "initial development". To me, increasing major version numbers tells a story that much more strongly implies some sort of convergence.

Sure, I think everyone understands the facts here. It's just not clear to me what the benefit is of effectively making the minor version equivalent to a major version, and the patch version equivalent to both minor & patch versions. Is that better than just versioning it properly, and maybe additionally recommending people use it as a branch-based dependency if they want to stick to the latest version? Why? As I said previously, I don't believe the package manager has any special treatment of 0.y.z versions, so it is not going to know anything useful about the distinction the proposal draws between minor and patch versions.

1 Like

Totally agree with this one, many features of Swift have to be delayed until version bumps and later on integrated in projects. I'm totally in with this one.

Big +1

Only thing I’m not sure about is keeping functionality a whole year after it appears in the released library. Some might start treating it as a backport package instead of a preview. Then there may be backlash against removing certain features even after the year is over.

I think it makes sense to say that this package is appropriate for users who are committed to upgrading to the next version of Swift as soon as it comes out.

People can always pin their dependencies to the last version supported by their Swift version or copy and paste the functionality they need.

2 Likes
Terms of Service

Privacy Policy

Cookie Policy