SE-0264 — Standard Library Preview Package

+1 for me as well.

As I said in the pitch thread I'm generally in favor of this proposal. However, as I mentioned there (with no reply), I am not entirely satisfied with the one year cycle for removing API that has migrated to the standard library.

I think we should seriously consider a separate stream of releases that always requires the latest released version of Swift (or later) and drops any symbols that have moved to the standard library as of that Swift release. This would provide a better alternative for users who do want to update quickly to the latest version of Swift. It would also ensure more immediate usage of the library implementations because users of the "edge" release would not need to modify code to switch to the library's implementation. In order to avoid confusion this version of the preview could have a different module name (i.e. SwiftPreviewEdge or something like that).

Supporting two streams of preview package releases is obviously more work for the package authors. However, I can speak from experience in saying that with appropriate planning it isn't terribly burdensome to support multiple build products from the same Swift sources.

One advantage having two streams of releases from the same source is that the Swift project contributors will gain experience in doing exactly that. It is possible this could motivate language features that would improve Swift's ability to fulfill this kind of requirement. (In some cases more boilerplate than is ideal)

If the core team decides it doesn't want to support two streams of release I think the one year timeline for staging features out of the preview package is much too long. When a new version of Swift is released it should be accompanied by a release of the preview package that does not include any symbols shipping in the standard library. This will ease adoption for those of who prefer to live on the bleeding edge. Users with slower update cycles are less likely to want to use the preview package anyway.

14 Likes

I like this proposal very much and think it will be a great addition to the ecosystem. But I was going to propose a modification similar to @anandabits

Specifically, I'm concerned about the proliferation and interweaving of arbitrary release numbers from the proposed scheme. We might have 0.134 that drops item X on schedule, 0.135 that adds item Y, and then 0.136 dropping Z. A user of the package has to keep track mentally of what they have access to at any given moment.

As an alternative, what about coalescing removal? Something along these lines:

  • As in the proposal, changes may be added freely to SwiftPreview, each with a minor version bump
  • As in the proposal, they may be added to the stdlib when ready/as time allows

BUT when a new release of the stdlib is made (yearly, I suppose, with Apple's release cycle):

  • All preview items that have moved to the stdlib are dropped from SwiftPreview at that time, AND
  • The major version of SwiftPreview is incremented

This provides us with a clear and easily describable progression for SwiftPreview: "Oh, yeah, FrobozzMagicSequence was moved to stdlib in v2, you need to update".

7 Likes

I support the idea, but I'm still not sure about the versioning part. As far as I remember, these sections of the proposal don't seem to have changed much since the discussion about this in pitch thread, and I find the reasoning here unclear. The text says that fixing the major version at 0 will act as a signal that it isn't source stable, but when I checked during the pitch thread it seemed that there is no special treatment major version 0 in the Swift package manager. What kind of chaos is this going to cause in dependency resolution? How does this signal anything to people who pick up this dependency transitively? Why is “proper” semantic versioning not even considered as an alternative?

The reason for keeping the zero-based major version was because it clearly signals the ongoing instability of the package, which will never converge as long as we continue to preview then land and release new features in the library.

Bumping the major version number would be contrary to this goal, without having a significant benefit (because minor versions would continue to add new features, which is in theory source breaking if not commonly so, or evolve APIs while still in preview, which is definitively so).

Indeed. It is a signal for humans, not for the package manager itself.

"Proper" semantic versioning would not bring any benefits, because when two packages both depend on an unstable package, they would have to agree on a version to use. When the major version is bumped constantly – because the goal of the package is to break source compatibility constantly (if only to add new API, which is technically source breaking) – then this would cause a logjam by giving the illusion of being useable by source-stable packages without actually being so in practice.

The fact is this package should not be pulled in by other packages that want to remain source stable. It would be nice to find a solution for these use cases too, but I've yet to see one, and holding up the preview package while we search for one, preventing apps and their supporting frameworks that could benefit from it. Once we do resolve this issue I hope we can adapt the process to use it.

  • What is your evaluation of the proposal?
    +1

  • Is the problem being addressed significant enough to warrant a change to Swift?
    Yes.

  • Does this proposal fit well with the feel and direction of Swift?
    Yes! It lowers the barrier for Standard Library Contributors in an ABI-Stable world, and will also lead to better real-world tested code landing into the Standard Library.

  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
    N/A

  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
    In-depth study.


Notes

  1. About the 1-year period for feature removal: Since the goal for this package is to be used in real-production code, I think the 1 year time period for features removal is fine; if we really want to push it down it could probably be a six months period.

  2. Something that I haven’t seen but I think is very important for this package to achieve its goal: how is this package going to be exposed to the community?

We need to push the SwiftPreview package so it can be used not only by the folks who participates in the forums.

I think this package deserves a spot or at least a mention in swift.org, it can be mentioned under the Standard Library Design section, or if it becomes important enough in the Swift community, it can be under the Projects section in the sidebar.

Also, the release of this package deserves a blog post in the Swift blog.

@Ben_Cohen can you clarify the implication of this sentence for me please?

The main difference from the existing process is that the final result will become available for general use immediately in a new version of the preview package.

Does this imply that unless the implementation of a particular feature requires some specific API from a stdlib version which is version locked, there will be no version locking? In that sense if diffing API's were implemented in the future and not in 5.1 then SwiftPreview package would have had this API's without @availabe(...) attributes, until it would officially move into stdlib?

1 Like

Is the “official” position that any new API addition in Swift is considered source breaking, and therefore requires a major version bump? This doesn't seem to square with the Swift Evolution definition of source stability, or with how semantic versioning is used by Swift packages today. I understand that it technically could be, because of overload resolution, etc. If you take a more reasonable (or lax, depending on your opinion) view here, then the major version would only need to be bumped when API is removed or changed in a source-incompatible way.

1 Like

The significance of this is that a new API with no @available(...) attached would be usable on an older OS in its SwiftPreview iteration, but not in its Standard Library iteration.


I was wondering this too. The semantic versioning standard says:

MINOR version when you add functionality in a backwards compatible manner, and
PATCH version when you make backwards compatible bug fixes.

What would possibly fall under a minor version, if not new APIs? Is Swift more vulnerable to creating new name clashes than the myriad other languages using the semantic version standard?

(I’m not arguing that we should or shouldn’t use version zero. I’m asking a broader question about how packages should be versioned in general.)

1 Like

Correct, that‘s what I hope, because it also has been mentioned that a user can (should) rip out the code from the package if he really needs it but the package itself removes it when it gets merged into stdlib.

I just want a clear clarification of that, otherwise I don‘t see the usefulness of this package besides that it will 'workaround' the slow release duration that we‘re used to from apple OS‘s.

I continue to feel that the best solution to the preview problem is to curate a collection of packages, one for each proposal. The single-minded pursuit of discoverability in this proposal creates a number of completely unnecessary problems around stability and deployment. Put together, these problems ironically undermine the discoverability goals by discouraging use of the preview package altogether. Using functionality from the unified preview package creates a ticking time bomb where the programmer is likely to be forced to make source changes across their codebase whenever the package version needs to be rolled forward for any unrelated reason, and it cannot be used in good conscience in any sort of open-source library. In contrast, with a collection of independent packages, programmers can independently manage the versioning of different preview libraries, and programmers can easily solve deployment issues by including the package as long as it is still necessary.

32 Likes

Can we use a negative major version number? We have to increase the value (i.e. towards positive infinity) each time, so we need to pick an anti-ceiling (for example, -1,000,000) to start from, and hope we don't get that many changes.

2 Likes

Negative numbers are not valid semantic versions. From the specification:

  1. A normal version number MUST take the form X.Y.Z where X, Y, and Z are non-negative integers, and MUST NOT contain leading zeroes.
<numeric identifier> ::= "0"
                       | <positive digit>
                       | <positive digit> <digits>

[...]

<digits> ::= <digit>
           | <digit> <digits>

<digit> ::= "0"
          | <positive digit>

<positive digit> ::= "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9"
1 Like

Yes, but it says that for versions with MAJOR 0 any update (even a PATCH one) can be breaking

Yes. Version 0 is a free‐for‐all. (Did I accidentally say otherwise somewhere? Discourse says your comment is directed at me.)

1 Like

No I don’t think you said that, but I did it reply because I think that the semantics of MINOR and PATCH are not that important given where’re working under a 0 MAJOR (or at least that was something I understood from previous discussions...)

I do like the idea of having different packages for this. Moreover, I think we can convince people to stop using these packages with clever deprecations messages. For instance this:

// package SwiftPreviewSubrangeSort
extension RangeReplacableCollection {
   @available(macOS, deprecated: 9999, *)
   @available(iOS, deprecated: 9999, *)
   @available(watchOS, deprecated: 9999, *)
   @available(tvOS, deprecated: 9999, *)
   func sort(subrange: Range<Index>)
}

could be used to alert users that usage of the preview package should be migrated to using the stdlib instead, but only when it makes sense based on the chosen minimum deployment target. (9999 here is the os version where the feature landed in the stdlib.) You can keep each package available indefinitely (or as long as it still compiles) and let the availability system intelligently annoy users into migrating to the stdlib version when they can.

Note: Maybe the import itself should show the deprecation warning (even though I don't think that's currently possible). That'd be better because you'd remove the import SomePreviewPackage at the top and most things would still work, relying on what's in the implicitly imported standard library.

1 Like

Part of this discussion has been whether we should stick to version 0 or should rapidly increment a non‐zero major version. (I don’t care which.) @Ben_Cohen seemed to say (here) that one reason not to go the non‐zero route is that in that case every API addition would force a major version bump. That is the statement @DevAndArtist @jawbroken and I are hoping he will clarify, since it has significant implications beyond just this review that affect anyone publishing any package.

(Edit: I guess I looked at the wrong quotation when I checked who wrote it. Sorry for the false attribution.)

1 Like

I think there’s a lot to like about this approach. It certainly addresses the issues I have raised regarding the proposed one year lag for removal from the preview package. Users who live on the latest version of Swift would simply drop dependency on preview packages as they are added to the standard library. Users who are on older versions of Swift would just continue to maintain a dependency on the older preview packages as long as necessary.

I think this approach would also have some appeal to larger clients I’ve worked with. Much of the time they may choose not to take a dependency on a preview package, but occasionally it may be very useful to do so. As a recent example, we have pulled the collection diffing algorithms into our codebase until we are able to migrate to Swift 5.1. If a preview package allowed us to pull in just that one dependency instead, we probably would have done so. On the other hand, introducing a dependency on a highly unstable preview package is unlikely to be something we would choose to do.

The obvious downside of this approach is that it requires more effort on the part users to keep track of all the preview packages that are available, choose the ones they wish to use, and remove the dependency on them when updating to a Swift release that includes the feature. For myself and typical clients this tradeoff is probably acceptable. Others may prefer the “simpler” solution of a monolithic preview package.

Either way, I continue to think it’s important to provide a good experience for those willing and able to live on the bleeding edge. This is the audience that is going to spend the most time with the new library features, gaining experience with them and providing the feedback we’re seeking before they land in the standard library proper.

5 Likes
Terms of Service

Privacy Policy

Cookie Policy