[Pitch] Non-Frozen Enumerations

PR for editorial suggestions

Non-Frozen Enumerations

  • Proposal: SE-NNNN
  • Authors: Karl Wagner
  • Review Manager: TBD
  • Status: Implementation In Progress
  • Previous Pitch: pitch

Introduction

Swift's enumerations provide powerful expressive capabilities for developers. However, it is currently not possible for source libraries (such as Swift Packages) to expose enums that can evolve while maintaining source compatibility. This proposal would allow them to opt-in to that stability.

Motivation

In some contexts, knowing all of an enum's cases is highly desirable; unhandled cases in a switch statement lead to a compilation error, requiring developers to examine every switch and proactively consider how the omitted cases apply to their operation. This helps ensure robustness in systems which evolve together (for instance, because they are part of the same module).

In other contexts, we want to leave room for an enum to grow and add new cases. This is particularly useful for enums which are exposed as part of a library's public interface, as it allows the library to evolve without breaking existing clients. When writing a switch statement involving an enum from a foreign library, clients should have to consider future cases so their code continues to compile as their dependencies evolve.

To illustrate, consider a library with a public data type. The type exposes a formatting function, taking as a parameter an enum describing the desired output:

public struct MyDataType {

  // ...

  public enum FormatStyle {
    case short
    case verbose
  }

  public func format(style: FormatStyle) -> String {
    switch style {
    case .short:
      // ...
    case .verbose:
      // ...
    }
  }
}

Because MyDataType.FormatStyle is a public enum, it is possible that some client library has written an exhaustive switch over it. If the library were to add another case - say, .ultraCompact, .medium, or .extraVerbose, that would technically be a source-breaking change, and require incrementing the library's major version (e.g. 2.x.y -> 3.x.y). Incrementing a library's major version is a highly disruptive process that requires extensive coordination with downstream packages, and is entirely disproportionate to the modest change being made here.

This is not a novel insight. Such changes could also alter an enum's ABI, which is why SE-0192 - Handling Future Enum Cases introduced the idea of frozen and non-frozen enums to Swift, and established that enums compiled in library-evolution mode would be non-frozen by default. When compiling in this mode, enums may be marked @frozen to opt-in to allowing exhaustive switching by clients.

SE-0192 did not address enums outside of library-evolution mode, instead leaving it for future discussion. However, the fragility of enums is not only a concern for ABI-stable libraries -- as discussed above, it is also a major concern for libraries distributed as source packages. Without the ability to add cases to an enum and preserve source compatibility, major libraries have decided to not expose enums in their interfaces, even when they would be the best, most expressive tool for the task.

Proposed solution, Detailed design

A new attribute will be introduced to the language, @nonfrozen.

  • Only public enums may be marked @nonfrozen
  • An enum may not be marked both @frozen and @nonfrozen
  • When compiling with library-evolution mode enabled, the @nonfrozen attribute has no effect

An enum with the @nonfrozen attribute is formally non-exhaustive. That means switch statements in other modules (including @inlinable code exposed to such modules) which involve the enum must include a 'catch-all' clause to handle future cases.

When library-evolution mode is not enabled, @nonfrozen enums remain effectively exhaustive to later stages of the compiler. This means source packages do not incur any performance penalty for marking an enum @nonfrozen; they should perform identically to unannotated (implicitly @frozen) enums. A @nonfrozen public enum compiled without library-evolution mode is not ABI stable.

A @nonfrozen enum behaves as the non-frozen enums described by SE-0192 do, with two minor alterations:

1. Switch statements MUST contain a 'catch-all'

To ease the rollout of SE-0192, it was softened so that omitting the 'catch-all' clause which handles future values only prompts a warning from the compiler, rather than an error. If an unknown value is encountered at runtime, the program reliably traps.

// Note that only warnings are produced here.
// The program still compiles successfully.

func test(_ x: FloatingPointRoundingRule) {
    switch x {
    // ^ warning: switch covers known cases, but 'FloatingPointRoundingRule' may have additional unknown values, possibly added in future versions
    // ^ note: handle unknown values using "@unknown default"
    case .up,
         .down,
         .toNearestOrEven,
         .toNearestOrAwayFromZero,
         .towardZero,
         .awayFromZero:
      print("...")
    }
}

However, this only applies to switch statements which are exhaustive. If we omit a case (simulating code that was written against some version of the standard library where .towardZero had not yet been introduced), we find the compiler is no longer willing to synthesise a catch-all clause for us, and instead refuses to compile the code:

func test(_ x: FloatingPointRoundingRule) {
    switch x {
    // ^ error: switch must be exhaustive
    // ^ note: add missing case: '.towardZero'
    // ^ note: handle unknown values using "@unknown default"
    case .up,
         .down,
         .toNearestOrEven,
         .toNearestOrAwayFromZero,
         .awayFromZero:
      print("...")
    }
}

This serves the narrow goal of providing ABI stability to switch statements which were exhaustive when they were compiled, but does not provide source stability because the addition of an enum case means our client code no longer compiles. For source stability, we must include a catch-all clause, such as an @unknown default. Once we do so, the switch exhaustiveness error is downgraded to a warning.

func test(_ x: FloatingPointRoundingRule) {
    switch x {
    // ^ warning: switch must be exhaustive
    // note: add missing case: '.towardZero'
    case .up,
         .down,
         .toNearestOrEven,
         .toNearestOrAwayFromZero,
         .awayFromZero:
      print("...")
    @unknown default:
      print("???")
    }
}

The @nonfrozen enums being discussed in this proposal are motivated by source stability, therefore we will insist that all switch statements involving them include a catch-all clause. Failure to do so will be an error.

2. Modules in the same package may continue to treat the enum as frozen.

As previously mentioned, it is often desirable to treat enums as frozen and exhaustively switch over them. The line where it becomes desirable or undesirable can approximately be described as "things which evolve together"; if a usage site evolves together with the enum's declaration (e.g. because they are in the same module) we can ensure they are always in sync, but if they evolve separately the usage site needs to consider that evolution.

SE-0386 New access modifier: package introduced the concept of packages in to the language. Packages are a unit of code distribution which may encompass several modules, and the modules inside a package indeed evolve together and are version-locked with respect to each other.

Therefore, when switching over a @nonfrozen public enum, if the declaration and usage modules belong to the same package, no catch-all is required and the enum's cases may be reasoned about exhaustively.

Source compatibility

  • If library-evolution mode is enabled:

    • Adding/removing the @nonfrozen attribute has no effect, since it is already the default and is mutually-exclusive with @frozen.
  • If library-evolution mode is disabled:

    • Adding the @nonfrozen attribute to an existing public enum is a source-breaking change.
    • Removing the @nonfrozen attribute from a public enum when the set of cases stabilise is a source-compatible change.

ABI compatibility

  • If library-evolution mode is enabled:

    • Adding/removing the @nonfrozen attribute has no effect, since it is already the default and is mutually-exclusive with @frozen.
  • If library-evolution mode is disabled:

    • Adding/removing the @nonfrozen attribute has no effect on the enum's ABI. Importantly, it does not confer ABI stability.

Implications on adoption

This is an additive language feature which does not require support from the runtime or standard library.

Future directions

Version-locked dependencies

If a package is a collection of version-locked modules, perhaps there is room to introduce another organisational unit for a collection of version-locked packages. For instance, an App developer might split their project up in to a number of packages, for reuse in various internal projects:

  • MyApp
  • SharedUtilityViews
  • SharedNetworkRequests
  • (Possibly also 3rd-party packages which the developer updates manually)
  • ...etc

The reason it would be attractive to model this is that @nonfrozen enums declared anywhere in this collection could be treated as exhaustive by every other package in the collection. This may allow us to make enums @nonfrozen by default even when library-evolution is disabled, with minimal source breakage and inconvenience.

This is a complex mix of several related features, and deserves extensive investigation. It is separable from the idea of giving source packages the ability to express non-frozen enums.

Alternatives considered

  • Do nothing.

    Package developers are avoiding exposing public enums because it is not possible to evolve them. That's not great.

  • Wait for version-locked dependencies.

    The only way version-locked dependencies would satisfy the evolution requirements of source packages is if we also switched the default behaviour of enums to be @nonfrozen.

    If that ever happens (which isn't clear), it's going to be a significant undertaking and deinfitely, massively source-breaking. It's definitely interesting but it's also unreasonable to make package developers wait for such an enormous change to maybe happen one day.

Acknowledgments

@lukasa pitched a version of this feature before.

34 Likes

I think this is very reasonable functionality to provide for library authors. I've never liked that the non-exhaustive behavior is restricted to library evolution mode.

One minor gripe I have is that I would prefer that the 'frozen' terminology remain relegated purely to library evolution mode, and that we use something like @nonExhaustive here if we want to go with an attribute on the type definition.

Another spelling for this that I like is to mirror the switch syntax and have library authors insert @unknown case in the enum body to signify that there may be additional cases in the future. Though that does mean you wouldn't be able to see whether an enum is exhaustive just from the type declaration preamble. Not sure if that's so important, as I expect most people will discover whether enums have this property as guided by diagnostics.

11 Likes

I actually think that "library evolution mode" is the thing that's poorly-named -- all libraries need to care about evolution, and the 'frozen' terminology, meaning "can this thing evolve or not?", is useful to all of them.

There are some other benefits:

  1. There are libraries which can be used either as source packages or binaries. swift-system is one example, but it also applies to user libraries that may sometimes be used as xcframeworks.

    If these libraries annotate all enums which can't evolve with @frozen, and those that can with @nonfrozen, the source and binary distributions would behave exactly the same for clients.

  2. There is symmetry with partial consumption of non-copyable types in non-resilient libraries:

    In that proposal, a @frozen struct promises that it will not gain stored members, allowing clients to totally de-structure the struct without fear of source incompatibility as the library evolves.

    It's very similar to what we're talking about here - where even without library-evolution mode enabled, @frozen means that clients may totally destructure the enum because it will not evolve new cases.

    That said, I see you also mentioned in response to that proposal that you weren't happy with @frozen being used in that way (so +10 for consistency). I guess I'd lean more towards Michael and Xiaodi's interpretation in that thread that it's valuable to have less divergence. I like the symmetry between frozen and non-frozen, but if there's consensus on something different of course I'd be happy to adopt that.

6 Likes

I'd heartily echo all of these comments as well. I'm also concerned about the divergence from @lukasa's prior pitch with respect to different treatment when a type is defined within the package.

I agree with the point that we need a holistic review of how 'resilience domains' are defined and how language features behave in light of these domains (with or without library evolution mode). However, creating ad hoc rules for one very specific feature isn't the place to start, in my view: indeed, creating ad hoc rules that treated enums differently based on an incomplete vision of how libraries evolve over time was how we got to this point in the first place.

The way I see it, either this feature can be orthogonal to how resilience domains are defined—for instance, by having the same behavior regardless of where a non-exhaustive enum is defined but relaxing the requirement for @unknown default to a warning which can be silenced or upgraded later—or we should sequence this proposal to be reviewed after a vision document (or miniature version thereof) has been approved that holistically reviews how the language should evolve with respect to resilience domains.

3 Likes

+9000 to this capability. Will coincide very nicely with Typed Throws for library errors.

2 Likes

Agreed. Even putting aside my concerns about overloading 'frozen', the @nonfrozen spelling is, like @frozen, quite general and plausibly promises more than just 'you can add enum cases.' For a general "maximum source compatibility" feature we might additionally want to be able to, say, add more associated values to an enum case, which users would have to account for with a case .enumCase(let val1, _, *) syntax, or something.

So absent a desire to explore the space of what that "maximum source compatibility" feature should or shouldn't look like, I would lean towards a more narrow spelling that covers "this enum may add more cases in the future" without overpromising. (Or, alternatively, want to see an argument that if we ever did get a more general @nonfrozen feature, "you can add an enum case" is the only thing we should support for enums).

2 Likes

Do we have a handle on how breaking this would actually be?

to answer that question we'd first have to define what a "grouping" of independently versioned packages/repositories is. the closest thing we have to such a concept today is SPM package collections, but those are just a marketing concept and don’t provide any guarantees for compiling code.

It seems to me that as long as a missing @unknwon default is a warning (so not making it an error), we could change the default with no actual breakage.

2 Likes

Do I understand correctly that in the case that a public enum in a library I use gets additional cases and my current code does not contain a “catch all”, but contains all cases as defined in the previous library’s version of the enum, the compiler will refuse to compile?

And do I understand correctly that the behavior of these modifiers changes depending on the mode we work in (evolution or non-evolution mode)?

switch is required to be exhaustive. If your code achieves that by listing all the cases of an enum from a different library, and then the library adds another case, your switch will no longer be exhaustive, and it will fail to compile.

2 Likes

Thank you for your clarification. For me it’s very important that this behavior does not change.

1 Like

Changing the default to non-frozen makes the behavior coherent with evolution-enabled libraries and it forces library authors to think about it before promising exhaustiveness. I think this is what we should strive for. (Lengthy explanation of how below.)

Not today in the presence of @unknown default: you only get a warning. For instance:

// warning: Switch must be exhaustive
switch NSUserInterfaceLayoutOrientation.vertical {
case .horizontal: break
@unknown default: break
}

It's also a warning for a missing @unknown default:

// warning: Switch covers known cases, but 'NSUserInterfaceLayoutOrientation' 
//  may have additional unknown values, possibly added in future versions
switch NSUserInterfaceLayoutOrientation.vertical {
case .horizontal: break
case .vertical: break
}

What I'm leaning at is that it does not have to be source-breaking, by itself, to make an enum non-frozen. It only becomes source-breaking once you add a case to the enum, and only if client code was written while ignoring the warning or if the author was using an older compiler that didn't emit a warning.

In short: regardless of whether we change the default or not adding a case for an enum existing today remains source breaking.

This gives us some room to choose the default correctly.


The best solution in my opinion would be:

  • Change the default to non-frozen.

  • When a library is compiled in Swift 5.x mode: clients in both 5.x and 6 modes get warnings about a missing @unknown default for a switch over an non-frozen enum (second example above).

  • When a library is compiled in Swift 6 mode: clients in both 5.x and 6 modes get errors about a missing @unknown default for a switch over an non-frozen enum.

And some guidance for library authors:

  • Regardless of whether enums are @frozen or not, it is a breaking change to add a case to an enum in a library compiled in Swift 5.x mode (because someone can ignore the warnings). But it is not a breaking change for an non-frozen enum in a library compiled in Swift 6 mode (can't ignore an error).

  • When migrating a library to Swift 6, you need to either bump the major version of your library, or you need to add @frozen to your enums. If you do neither, clients might start to get errors (instead of warnings) about missing @unknown default: in switch, which could cause source breakage for clients who didn't fix their warnings in Swift 5.10 (or whatever version it appears in).

This way we get the correct default and avoid the confusion of library-evolution mode having an opposite behavior.

I realize there is still some room for breakage with this plan, and a lot of room for annoying warnings appearing everywhere. It just seems less problematic in the long run than having a situation where copying code between two libraries can change its meaning if one of them is in library-evolution mode. Do we really want to have two dialects?

5 Likes

Having a ternary for this property ("amount of frozeness") seems over-complicated. In practical terms a structure (enum or otherwise) is either frozen or it's not. You can either rely on it not changing, or you can't.

Technicalities like "it's not a breaking change until you add a case to the enum" seem irrelevant to me, since the important moment in time is when the code using the enum was written; that's when it's either written to be forward-compatible or not.

I'm not sure this needs to be compiler enforced, in the sense of erroring out if an @unknown default is missing. You can provide an @unknown default today, if you want. It's just not required by the compiler.

It comes down to the question of when you have to deal with the change, and who's responsible for deciding that - users or enum authors? If a user cares about not having source incompatibilities when upgrading a library, they can choose to proactively use @unknown default (although it only prevents outright build breaks - it doesn't necessarily prevent any actual necessary work to accomodate new cases). If they prefer to just deal with the change if and when it occurs, they can omit @unknown default. It's not apparent to me that this decision should be made exclusively by enum authors.

I recognise that there are practical concerns, currently, with bumping package major versions - down to a fundamental level, such as Xcode et al not even notifying you when a new major version of a dependency is available. And with chains of dependencies and the sort of 'chicken and egg' problems. But those concerns exist irrespective of this particular aspect. So perhaps it'd be more fruitful to find ways to generally alleviate those issues, than merely work around them in specific cases like this.

First off, I am very much in favor of addressing this topic:

Looking at the growing landscape of "load-bearing" swift packages (potentially combined with typed throws), the problem of public enums in APIs and the semver dance connected to it is quite unfortunate.

I initially found myself drawn in the direction of "now is the chance, we can 'fix' the disconnect between library-evolution-mode and 'normal' mode" - but the more I thought about it the more I came out on the other side.

tl;dr: I would love to just see this done as described, but with a spelling that is not connected to @frozen at all. (I liked @extensible of the original pitch by Cory.)

Here are a few thoughts:

  • @frozen means a lot more than just "you should add @unknown to you switch or the compiler will annoy you"
    It is established and clearly linked to library-evolution mode and ABI-stability. I think we should leave @frozen out of this.

  • All we want is adding cases to a public enum without without having to declare it a breaking change.
    At least I assume that we can all agree that this is what it ultimately boils down to: minor vs major version of the release tag if you will.

  • People who write modules in library-evolution mode will hardly be confused by this. People who don't will not need to fall down the @frozen rabbit hole, just add @unknown default when an enum is @extensible - no memory layouts or witness tables in sight.

  • Sometimes, it's just about fixing the problem, and not every minor inconsistency around it.

I never understood the rationale behind "@unknown default" properly, IMHO it just doesn't work in general case.


Consider the client networking application that accepts a stream of drawing commands sent over the network by the server and renders them to the screen accordingly.
The client and the server are both v1.0 and all is good.

Then there was a change to one of the enumerations that both client and server use: a new case was added. The client usage of that enumeration contained "@unknown default" clause so when the client was recompiled with the new v1.1 library there were no compilation errors. No one alarmed, yet we could have one of the following outcomes:

1. some shapes are missing or not drawn correctly (e.g. colours are off)
// common enumeration
enum PayloadType {
    case string
    case point
    case rect
    case polygon // new case in v1.1
}
// client code
switch payloadType {
	case .string: drawString()
	case .point: drawPoint()
	case .rect: drawPoint()
	@unknown default: drawPlaceholder()
}
2a. the app stops rendering some streams, and you have to restart the stream, but the issue repeats.
// stream format:
// <lenType> <len> <payloadType> <payload> ... repeat

// common enumeration
enum LengthType {
    case byte
    case short
    case word
    case long // new case in v1.1
}

// client code
switch lengthType {
	case .byte: len = readByte();   let payload = readBytes(len); renderPayload(payload)
	case .short: len = readShort(); let payload = readBytes(len); renderPayload(payload)
	case .word: len = readWord();   let payload = readBytes(len); renderPayload(payload)
	@unknown default: print("can't really proceed"); abortStream()
	
}
2b. there are crashes when rendering some streams
// same as above, plus:
// client code
switch lengthType {
    ....
    @unknown default: fatalError("can't really proceed")
}

Note that the library change was "minor" yet we could still have the outcome #2, which is hardly minor from the end users POV and doesn't feel a justifiable behaviour when going from library v1.0 to v1.1


IMHO, this is better (from the end user point of view at least):

switch payloadType {
    case .string: drawString()
    case .point: drawPoint()
    case .rect: drawPoint()

    // Option A: "lazy developer":
    default: drawPlaceholder() // normal default clause

    // Option B: "hard working developer":
    // no default statement here
    // no warning when compiling with library v1.0 (IMHO how it should be)
    // error with compiling with library v1.1 (and a chance to improve)
}

switch lengthType {
    case .byte: len = readByte();   let payload = readBytes(len); renderPayload(payload)
    case .short: len = readShort(); let payload = readBytes(len); renderPayload(payload)
    case .word: len = readWord();   let payload = readBytes(len); renderPayload(payload)
    // no default statement here
    // no warning when compiling with library v1.0 (IMHO how it should be)
    // error with compiling with library v1.1 (and a chance to improve)
}

How do I opt-opt of getting a warning:

Say, I have a zero warning tolerance policy by making all warnings errors... So I can't have a warning there. Yet, I do not want to add "@unknown default" (or a normal default) as if I do that – I will not notice the change when the upgraded library gets a new case! Any way to opt-out of SE-0192 ?

1 Like

@unknown default should not be suppressing warnings or errors about missed known cases; if it is, that's an implementation bug, since it defeats the purpose of having @unknown default as distinct from just default. SE-0192 explicitly states:

the compiler will produce a warning if all known elements of the enum have not already been matched.

Are you saying that there is in fact an implementation bug, or is there something else I don't understand about why you'd like to opt out of SE-0192? The language steering group plans to make missing @unknown default a hard error in Swift 6 (for resilient types).

3 Likes

Thank you, and please bear with me as I wasn't following enum topics closely.

is the fatalError() here unreachable?

@unknown default : fatalError("can never happen") 

or could it happen, if so under what circumstances?
(assuming all my warnings are errors, all known cases of the enum are handled and there is no normal "default" statement.

Assuming as you state that you have handled all known cases and aren't ignoring a warning about a missing known case, this has equivalent behavior to omitting @unknown default entirely: if you switch over a value that's a future enum case, there will be a runtime trap. This can only happen if the enum is a type defined in a resilient library where at runtime you have a later version.

1 Like

Does that mean that this code:

// -warnings-as-errors
// no warnings, all known cases of v1.0 enum are handled
switch lengthType {
    case .byte: break
    case .short: break
    case .word: break
    @unknown default:
        print("this won't be printed")
        fatalError("won't happen")
}

when at runtime the new v1.1 of the resilient library is used (that has a new case in that enumeration), will trap (crash), and there will be no "this won't be printed" line printed? Is that alright? We are talking about a "minor" version change.