SE-0264 — Standard Library Preview Package

Indeed. It is a signal for humans, not for the package manager itself.

"Proper" semantic versioning would not bring any benefits, because when two packages both depend on an unstable package, they would have to agree on a version to use. When the major version is bumped constantly – because the goal of the package is to break source compatibility constantly (if only to add new API, which is technically source breaking) – then this would cause a logjam by giving the illusion of being useable by source-stable packages without actually being so in practice.

The fact is this package should not be pulled in by other packages that want to remain source stable. It would be nice to find a solution for these use cases too, but I've yet to see one, and holding up the preview package while we search for one, preventing apps and their supporting frameworks that could benefit from it. Once we do resolve this issue I hope we can adapt the process to use it.

  • What is your evaluation of the proposal?
    +1

  • Is the problem being addressed significant enough to warrant a change to Swift?
    Yes.

  • Does this proposal fit well with the feel and direction of Swift?
    Yes! It lowers the barrier for Standard Library Contributors in an ABI-Stable world, and will also lead to better real-world tested code landing into the Standard Library.

  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
    N/A

  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
    In-depth study.


Notes

  1. About the 1-year period for feature removal: Since the goal for this package is to be used in real-production code, I think the 1 year time period for features removal is fine; if we really want to push it down it could probably be a six months period.

  2. Something that I haven’t seen but I think is very important for this package to achieve its goal: how is this package going to be exposed to the community?

We need to push the SwiftPreview package so it can be used not only by the folks who participates in the forums.

I think this package deserves a spot or at least a mention in swift.org, it can be mentioned under the Standard Library Design section, or if it becomes important enough in the Swift community, it can be under the Projects section in the sidebar.

Also, the release of this package deserves a blog post in the Swift blog.

@Ben_Cohen can you clarify the implication of this sentence for me please?

The main difference from the existing process is that the final result will become available for general use immediately in a new version of the preview package.

Does this imply that unless the implementation of a particular feature requires some specific API from a stdlib version which is version locked, there will be no version locking? In that sense if diffing API's were implemented in the future and not in 5.1 then SwiftPreview package would have had this API's without @availabe(...) attributes, until it would officially move into stdlib?

1 Like

Is the “official” position that any new API addition in Swift is considered source breaking, and therefore requires a major version bump? This doesn't seem to square with the Swift Evolution definition of source stability, or with how semantic versioning is used by Swift packages today. I understand that it technically could be, because of overload resolution, etc. If you take a more reasonable (or lax, depending on your opinion) view here, then the major version would only need to be bumped when API is removed or changed in a source-incompatible way.

1 Like

The significance of this is that a new API with no @available(...) attached would be usable on an older OS in its SwiftPreview iteration, but not in its Standard Library iteration.


I was wondering this too. The semantic versioning standard says:

MINOR version when you add functionality in a backwards compatible manner, and
PATCH version when you make backwards compatible bug fixes.

What would possibly fall under a minor version, if not new APIs? Is Swift more vulnerable to creating new name clashes than the myriad other languages using the semantic version standard?

(I’m not arguing that we should or shouldn’t use version zero. I’m asking a broader question about how packages should be versioned in general.)

1 Like

Correct, that‘s what I hope, because it also has been mentioned that a user can (should) rip out the code from the package if he really needs it but the package itself removes it when it gets merged into stdlib.

I just want a clear clarification of that, otherwise I don‘t see the usefulness of this package besides that it will 'workaround' the slow release duration that we‘re used to from apple OS‘s.

I continue to feel that the best solution to the preview problem is to curate a collection of packages, one for each proposal. The single-minded pursuit of discoverability in this proposal creates a number of completely unnecessary problems around stability and deployment. Put together, these problems ironically undermine the discoverability goals by discouraging use of the preview package altogether. Using functionality from the unified preview package creates a ticking time bomb where the programmer is likely to be forced to make source changes across their codebase whenever the package version needs to be rolled forward for any unrelated reason, and it cannot be used in good conscience in any sort of open-source library. In contrast, with a collection of independent packages, programmers can independently manage the versioning of different preview libraries, and programmers can easily solve deployment issues by including the package as long as it is still necessary.

32 Likes

Can we use a negative major version number? We have to increase the value (i.e. towards positive infinity) each time, so we need to pick an anti-ceiling (for example, -1,000,000) to start from, and hope we don't get that many changes.

2 Likes

Negative numbers are not valid semantic versions. From the specification:

  1. A normal version number MUST take the form X.Y.Z where X, Y, and Z are non-negative integers, and MUST NOT contain leading zeroes.
<numeric identifier> ::= "0"
                       | <positive digit>
                       | <positive digit> <digits>

[...]

<digits> ::= <digit>
           | <digit> <digits>

<digit> ::= "0"
          | <positive digit>

<positive digit> ::= "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9"
1 Like

Yes, but it says that for versions with MAJOR 0 any update (even a PATCH one) can be breaking

Yes. Version 0 is a free‐for‐all. (Did I accidentally say otherwise somewhere? Discourse says your comment is directed at me.)

1 Like

No I don’t think you said that, but I did it reply because I think that the semantics of MINOR and PATCH are not that important given where’re working under a 0 MAJOR (or at least that was something I understood from previous discussions...)

I do like the idea of having different packages for this. Moreover, I think we can convince people to stop using these packages with clever deprecations messages. For instance this:

// package SwiftPreviewSubrangeSort
extension RangeReplacableCollection {
   @available(macOS, deprecated: 9999, *)
   @available(iOS, deprecated: 9999, *)
   @available(watchOS, deprecated: 9999, *)
   @available(tvOS, deprecated: 9999, *)
   func sort(subrange: Range<Index>)
}

could be used to alert users that usage of the preview package should be migrated to using the stdlib instead, but only when it makes sense based on the chosen minimum deployment target. (9999 here is the os version where the feature landed in the stdlib.) You can keep each package available indefinitely (or as long as it still compiles) and let the availability system intelligently annoy users into migrating to the stdlib version when they can.

Note: Maybe the import itself should show the deprecation warning (even though I don't think that's currently possible). That'd be better because you'd remove the import SomePreviewPackage at the top and most things would still work, relying on what's in the implicitly imported standard library.

1 Like

Part of this discussion has been whether we should stick to version 0 or should rapidly increment a non‐zero major version. (I don’t care which.) @Ben_Cohen seemed to say (here) that one reason not to go the non‐zero route is that in that case every API addition would force a major version bump. That is the statement @DevAndArtist @jawbroken and I are hoping he will clarify, since it has significant implications beyond just this review that affect anyone publishing any package.

(Edit: I guess I looked at the wrong quotation when I checked who wrote it. Sorry for the false attribution.)

1 Like

I think there’s a lot to like about this approach. It certainly addresses the issues I have raised regarding the proposed one year lag for removal from the preview package. Users who live on the latest version of Swift would simply drop dependency on preview packages as they are added to the standard library. Users who are on older versions of Swift would just continue to maintain a dependency on the older preview packages as long as necessary.

I think this approach would also have some appeal to larger clients I’ve worked with. Much of the time they may choose not to take a dependency on a preview package, but occasionally it may be very useful to do so. As a recent example, we have pulled the collection diffing algorithms into our codebase until we are able to migrate to Swift 5.1. If a preview package allowed us to pull in just that one dependency instead, we probably would have done so. On the other hand, introducing a dependency on a highly unstable preview package is unlikely to be something we would choose to do.

The obvious downside of this approach is that it requires more effort on the part users to keep track of all the preview packages that are available, choose the ones they wish to use, and remove the dependency on them when updating to a Swift release that includes the feature. For myself and typical clients this tradeoff is probably acceptable. Others may prefer the “simpler” solution of a monolithic preview package.

Either way, I continue to think it’s important to provide a good experience for those willing and able to live on the bleeding edge. This is the audience that is going to spend the most time with the new library features, gaining experience with them and providing the feedback we’re seeking before they land in the standard library proper.

5 Likes

Actually this is an interesting idea. To solve the naming problem I would propose to call such packages with an SE## suffix. Let‘s say we get count(where:). Then the package could be called PreviewSE220.

Two good parts about this approach are that we won‘t need to version lock the APIs if they don‘t depend on already version locked stdlib APIs, and last but not least we don‘t have to delete these packages after the APIs are merged into stdlib, a deprecation attribute should do the trick to notify the users.

9 Likes

Couldn‘t this issue be solve if the umbrella module SwiftPreview would stop to import some of the standalone modules?

If the user don‘t want to track all the modules he can import them all and then at some point of upgrade he‘ll get compile time errors that some APi is no longer available because the umbrella module dropped a dependency on one particular module he was using.

2 Likes

Apple wants that you update Swift, like it wants that you update to the newest OSs. It has many benefits if the majority is on the newest version. This will reduce the motivation to upgrade.

It should be considered, that it could make it harder for people not so familiar with Swift in the long run.
Companies could keep preview packages installed even though it has been declined or already published in their used Swift version. Do we really want to have apps with declined functionality? 5 years later nobody knows why there is a preview package with a fork and a different name of the package in the codebase and only one new developer left to fix bugs (this happens). But developers are wondering why feature A is not working like in the documentation. Lazy developers could just take the first hit on google when they search for a feature and instead of using the already installed feature in their Swift version they just install the preview package with a slightly off functionality.
Programmers are not always as motivated and interested in the topic of Swift as the members of this forum. They may use JavaScript, Swift and Kotlin at the same time at work. Getting a solution if everything they need, not understanding it, even though there are better solutions at hand. They are users of Swift too and influence code that other developers have to code with.

My point is: Make it very obvious and not to easy to use preview packages, even though this may be counter-intuitive for people on the forums.

3 Likes

I strongly agree, albeit from a different perspective. I think "Propose as a Package", where one proposes an individual self-contained package that reviewers can check out and depend on is a superior approach for swift-evolution proposals, in addition to the benefits you highlighted. After acceptance, it can live as an orphaned branch on a "swift-evolution-packages" repo or something similar.

If we really want to, we can reproduce what is proposed here as a meta-package that reexports a rolling window of proposals. That catalogue-like package could be simpler to maintain as well since there's no untangling of code, only reexports to drop.

A fresh self-contained package is easier to propose than a PR against (what could be) a larger body of code. Especially if that body of code is changing (rolling release window) or versioned (availability annotations to drop the decl depending on OS version).

Pitches can evolve over a long period of time without force-rebasing or other PR synchronization. There are so many more options available for collaborative workflows and building upon prior pitches or prototypes.

Reviewers can evaluate just the functionality under consideration, especially if they are already using some preview functionality. No need to swap out their version of a preview package with that from a PR. Independent packages are independently versioned, and they go stale much slower than PRs.

Any dependencies on prior proposals can be modeled with explicit package dependencies, rather than implicit calls in code. Dropping from a meta-package is cleaner because the code never got tangled up in the first place.

20 Likes

After following the thread and the different ideas proposed, I fell like I must change my first feedback.

I'm still strongly in favor of having a simplified system to try out new additions to the standard library

But after reading @John_McCall & @Michael_Ilseman ideas, I really do like the idea of having a collection of packages for each proposal.

Terms of Service

Privacy Policy

Cookie Policy