SE-0192 — Non-Exhaustive Enums

Hey, everyone. SE-0192 is not dead! Here's what's been going on.

Over the last few weeks I've been discussing this with the core team, who pretty much all agreed that this is not the right direction for third-party libraries. That is, for this table in the proposal:

Use Case Frozen Non-frozen
Multi-module app The desired behavior. Compiler can find all clients if the enum becomes non-frozen. Compiler can find all clients if the enum becomes frozen.
Open-source library (SwiftPM) Changing to non-frozen is a source-breaking change; it produces errors in any clients. Changing to frozen produces warnings in any clients.
ABI-stable library (Apple OSs) Cannot change to non-frozen; it would break binary compatibility. Changing to frozen produces warnings in clients (probably dependent on deployment target).

the first two rows are being unnecessarily inconvenienced by the frozen/non-frozen distinction.

This was always a controversial part of the proposal, since it meant that all enum that anyone had ever written in Swift would be affected. For C enums and the standard library, the benefit is clear, but for third-party libraries it was merely about whether a particular change (adding a case) would break source compatibility, and for an app that had just been broken up into multiple modules, there was almost no point at all. The core team, unlike me, did not feel that this trade-off was worth it. So, the frozen/non-frozen distinction will only apply to (1) C enums and (2) enums from the standard library and overlays.

This was the biggest part of the proposal, but the rest of it is mostly still relevant. In particular:

  • Switching over a C enum will require a catch-all clause, unless the enum is marked specially. (The current Clang attribute that controls that is spelled __attribute__((enum_extensibility(closed))).) This will be an error in Swift 5 mode, but we're still deciding whether it should be a warning in Swift 4 mode or just left un-diagnosed. (In either case, you'll be able to add the catch-all clause in Swift 4 mode without the compiler complaining.)

  • Switching over a Swift enum in the standard library or overlays will require a catch-all clause, unless that enum is marked specially. The attribute that distinguishes frozen enums from non-frozen enums will be underscored (something like @_frozen), indicating that it is not ready for general use yet; in order to discuss it properly, we need to be able to talk about the differences between "libraries with a stable binary interface" and "libraries without a stable binary interface", and SE-0192 is big enough already.

  • If an enum is explicitly marked frozen, using any sort of catch-all case will get an unreachable code warning, as it does today.

  • The community has made it clear that they want some form of catch-all case that will still result in compiler warnings whenever it's reachable with a known enum case. This is what I've been calling unknown case and what's being discussed over in Handling unknown cases in enums [RE: SE-0192] as well.

I'm working on cutting down the proposal text (and the implementation) to match this feedback from the core team, but there's also one more issue that needs to be worked out about unknown case—and no, I'm not just talking about the name. I'm going to bring that up in Handling unknown cases in enums [RE: SE-0192].

Once that's all done, the plan is to do another formal round of review (though likely a short one), since much of the community may not have had the stamina to keep up with these very long discussions. I may also start landing some of the implementation ahead of that just to keep from having to rebase it all the time (with approval from Ted), but all the checking will be behind a flag.

Thanks as always to everyone for their participation, patience, and thoughtful feedback.

18 Likes

Thanks very much for taking the time to work to this position. I very much appreciate the recognition that the initial approach would have caused much undue pain for many developers and the course correction to this approach.

3 Likes

I think this is a great call. I would suggest going further. The default enum is frozen and C enums and Apple standard library enums are annotated @NotFrozen.

2 Likes

I agree that having frozen as a default status would be nice: it means the safest, more controlled behaviour is default, and the less strict one is opt-in.
It also means that the change is less dramatic, as the old behaviour is stil the default one, and that the default behaviour is closer to the most commonly seen in other languages. That can make the language easier to learn for newcomers

1 Like

This is a positive change, thanks to the core team and everyone participating for coming to such a sensible conclusion. I look forward to the discussion around unknown case as well. -C

To be clear, I don't think "the default behavior is frozen" is what's happening. Normal code will not be able to mark enums as non-frozen at all.

I still very strongly feel that the default behavior should be non-frozen for "libaries with binary compatibility concerns". However, if you are making such a library, you already have plenty of other things to think about that you wouldn't for a source package. (Again, I don't think we have the vocabulary to talk about this yet, and trying to do so as part of SE-0192 would be counterproductive.)

Is the way to interpret the result of your discussions that you're going to hack this into the compiler just for the standard library because you need to hit ABI stability goals, but perhaps in future you will generalise this in whatever way that makes sense (possibly involving the compiler understanding your binary compatibility requirements)? But you still need to design all the user-facing features for interacting with such enums, even if the enums themselves can't be written by users in Swift (without using undocumented compiler modes and underscored attributes).

In essence, you're going to design half the feature, how to use non-frozen enums, while punting on the question of how they are declared. That makes sense, given your goals for the next release, but it definitely feels strange.

That seems 90% correct, though you should remember the primary motivation is not Swift enums but C enums from Apple's SDKs.

The last 10% is that the core team made it pretty clear that this would probably never be supported for source packages, so a library author would only start dealing with the complexity of non-frozen enums when they decide to make a "library with binary compatibility concerns". I really want to get to a place where we can talk about that soon (it doesn't just cover @_frozen but also the @_fixed_layout / "fixed-contents" attribute we've been using in the standard library), but it's not something we expect anyone to care about outside of the stdlib-and-overlays until we have module stability anyway.

The last 10% is that the core team made it pretty clear that this would probably never be supported for source packages, so a library author would only start dealing with the complexity of non-frozen enums when they decide to make a “library with binary compatibility concerns”.

I don't think that was the consensus.

Frozen types and non-exhaustive switches are an important concern when maintaining source and binary compatibility. However, that concern is largely limited to library authors, not clients. Programmers should be able to use third-party libraries without being forced to do work eagerly to maintain compatibility in the event that they upgrade their dependencies. (They might choose to do this work eagerly, and we should provide tools for that; but it shouldn't be forced on them.)

The only time when this work should be forced on programmers eagerly is when a single version of their code needs to be able to work with multiple versions of the dependency. The most obvious example of this is when the dependency is a binary package distributed separately from their own, for example in the OS; but another important example is when both their code and the dependency are separately-distributed source packages, because it should be possible for clients of the downstream library to update the dependency without waiting for an update for everything that depends on it. The reason that we're only talking about OS dependencies for now is just that we lack any mechanism for expressing (either in the language or the build system(s)) that the code currently being compiled is actually in the second case. Adding that is a large project that needs careful design. In the meantime, source packages can still use features like unknown case to achieve source compatibility, even if that work isn't assisted by the compiler.

2 Likes

I'm curious, where did this notion that the community wants "clients of the downstream library to update the dependency without waiting for an update for everything that depends on it"? Because it's certainly not something I've ever wanted, even in the dark days of building an app during the Xcode 8 betas. And it's not something I've ever heard anyone in the community ask for, despite rather constant complaining about source breakage during Swift's development. Regardless of Apple's internal development practices, the Apple programming community has been using dependency managers for nearly 5 years now, so I don't think it's worthwhile for Swift to change in ways that try to workaround the fact that there isn't an official solution to the dependency problem yet (for anyone developing more than command line utilities at the moment). Changes in enums are really no different than any other breaking change in a major new version of a dependency. Some libraries offer availability statements to help automate the upgrade process, but that's not always possible, and not every project has the time or energy to generate them in the first place. But because we can control when we upgrade to those new, breaking versions, we know we may need to spend time updating our function calls, our type names, and yes, our enums. Frankly, I think the source compatibility angle can be dropped entirely, which would make this whole issue solely ABI focused and likely much easier to get done.

You've never needed to update a dependency and had it break another library that you were using (that shared the same dependency) because it happened to be too tightly tied to that dependency, and then found yourself either blocked waiting for an update for the second library or manually hacking their code yourself? All I'm talking about is encouraging libraries to allow their dependencies to float a little more than that. We think that, ultimately, we will be able to provide a pretty solid set of tools for library authors to check source compatibility for their clients, and this will be part of that story. But library authors will of course remain free to version-lock their code to all their dependencies and ignore all of those tools if that's their choice.

Regardless, nothing is being held up by the source-compatibility design here.

1 Like

Or, to put it another way:

But because we can control when we upgrade to those new, breaking versions, we know we may need to spend time updating our function calls, our type names, and yes, our enums.

This is absolutely true of leaf clients, like applications and their embedded libraries. However, if intermediate open-source libraries try to follow it, it creates a really brittle package ecosystem. Some of that is inescapable — even perfect source compatibility doesn't really guarantee semantic compatibility, after all — but there are parts of it that the language can help with, and we'd like to at least take those steps.

Unless I’m using a sub dependency directly as well as indirectly, that’s never a need. And in the rare cases where that happens, the dependency manger tells me I can’t. And I’ve never had a sub dependency drive my update schedule. So like I said, this is a solved, or at the very least extremely minor, issue. So my question remains, where did this requirement come from? Was it just some thought of as a nice to have, not knowing how minor it is to the community? Or was it driven by actual feedback from some segment of the Swift community which has never posted a request for such a feature to the Swift dev lists?

And without the source compatibility requirement, the discussion around #unknown becomes vastly simpler, since it would only be used at runtime when binary dependencies change. To me, that would greatly simplify the mental model requires to understand and design the feature.

Dependency mangers make this theoretical brittleness a nonissue, like in every other language. I could understand the value if we were all operating without one, but we are and have been for years, and it works pretty well, all without having to change Swift (or Objective-C) at all.

At the very least, I’m saying the source compatibility angle could be dropped to more easily get community buy in and a resolution to the unknown issue.

To be clear here, the intent here is not to somehow eliminate the need for dependency management, but to reduce the number of times that a dependency manager has to pull an older version of a library or report an unresolvable conflict.

Regardless, I don't think source-compatibility is actually blocking community acceptance in any way. It is not what the discussion about the unknown feature has centered around.

But is that helping or hiding complexity and risk under the carpet (no offence meant)?

Jordan, thank you for the update and for all your work!

A couple of questions to clarify new suggestions:

  1. Do I understand correctly that we'll have no allowed way (at least during some time after Swift 5 release) to mark
    our own enums as 'non-frozen' ? I.e. enums declared in Swift (inside module or in separate module/framework compiled
    from Swift code) will have the same rules as in Swift4 - i.e. switch over such enum must be exhaustive or contains
    'default' branch ?
    I.e. only C enums and Apple's SDK enums could be 'non-frozen' enums for now?

  2. Can we have 'unknown case' in switch for 'standard'(frozen) enum?
    For example I want to write 'switch' code in a way it will not generate error (but will generate warning) in case my
    binary framework,I use in my app, added a new case?
    I'm about situation when I'm compiling my app with 3rd party built framework(for example by using Carthage), and this
    framework introduced new enum case. And I want a solution to not stop right now to add new case into code, but just
    compile the app and let 'unknown case' work for now in my code for that new case.
    IMO this should be allowed to make a code more flexible, in case developer expects new cases can appear later in the
    switch, and want to be notified, but want to provide 'default' processing for future cases.

Thank you.

That is correct. I'm not sure I quite agree with John's take on what we might do in the future (although I admit to simplifying and leaving out some of the less common possible use cases), but it will not be supported in Swift 5. Or at least, it is no longer being proposed as part of SE-0192, and I don't intend to bring any such proposal myself, and I suspect it would be very unlikely that the core team would accept such a proposal for Swift 5.

Yes, per suggestion by someone earlier (possibly you?) it will be okay to write unknown case-or-whatever for a switch over an enum that has not been explicitly marked frozen, including all non-stdlib-non-overlay Swift enums. (The compiler will not require it but it will allow it.) I'll make sure that's clear in the revised proposal.

Important Update

A second review of this proposal has launched, based on feedback on this review thread and from the Core Team. Please put feedback on the updated proposal in that review thread.

Ted Kremenek
Review Manager

1 Like