Hey folks. Over the weekend I put together a quick draft patch that would enable non-resilient Swift libraries to provide "extensible" enums, where adding a new case would not be a Semver major breaking change. This patch does the absolute minimum required to enable the functionality, so before I go any further with it I wanted to get the community's opinion on whether this is a problem worth addressing.
Below is a draft proposal that covers the issue. Please give it a read and let me know your thoughts.
Extensible Enumerations for Non-Resilient Libraries
- Authors: Cory Benfield
- Implementation: apple/swift#31336
Introduction
This proposal adds an attribute to allow Swift enumerations to opt-in to an extensible behaviour. This reconciles a feature mismatch between resilient and non-resilient dialects of Swift, and makes Swift enumerations vastly more useful in the public API of non-resilient Swift libraries.
Motivation
When Swift was enhanced to add support for "library evolution" mode (henceforth called "resilient" mode), the Swift project had to make a number of changes to support a movable scale between "maximally evolveable" and "maximally performant". This is because it is necessary for an ABI stable library to be able to add new features and API surface without breaking pre-existing compiled binaries. While by-and-large this was done without introducing feature mismatches between the "resilient" and default "non-resilient" language dialects, the @frozen attribute when applied to enumerations managed to introduce a difference. This difference was introduced late in the process of evolving SE-0192, and this pitch would aim to address it.
@frozen is a very powerful attribute. It can be applied to both structures and enumerations. It has a wide ranging number of effects, including exposing their size directly as part of the ABI and providing direct access to stored properties. However, on enumerations it happens to also exert effects on the behaviour of switch statements.
Consider the following simple library, an SDK to your favourite pizza place:
public enum PizzaFlavor {
case hawaiian
case pepperoni
case cheese
}
public func bakePizza(_ flavor: PizzaFlavor)
Depending on whether the framework is compiled with library evolution mode enabled, what the caller can do with the PizzaFlavor enum varies. Specifically, the behaviour in switch statements changes.
In the "standard", "non-resilient" mode (with library evolution disabled), users of the PizzaSDK can write exhaustive switch statements over the enums vended by PizzaSDK:
switch pizzaFlavor {
case .hawaiian:
throw BadFlavorError()
case .pepperoni:
try validateNoVegetariansEating()
case .cheese:
return .delicious
}
This enumeration will happily compile. If the author of the above switch statement was missing a case (perhaps they forgot .hawaiian is a flavor), the compiler will error, and force the user to either add a default: clause, or to express a behaviour for the missing case. The term for this is "exhaustiveness": in the default non-resilient dialect, the Swift compiler will ensure that all switch statements over enumerations cover every case that is present.
There is a downside to this mode. If PizzaSDK wants to add a new flavour (maybe .veggieSupreme), they are in a bind. If any user anywhere has written an exhaustive switch over PizzaFlavor, adding this flavor will be an API and ABI breaking change, as the compiler will error due to the missing case statement for the new enum case.
Because of the implications on ABI and the requirement to be able to evolve libraries with public enumerations in their API, the resilient language dialect behaves differently. If PizzaSDK was compiled with enable-library-evolution turned on, when a user attempts to exhaustively switch over the PizzaFlavor enum the compiler will emit a warning, encouraging users to add an @unknown default: clause. Thus, to avoid the warning the user would be forced to consider how new enumeration cases should be treated. They may arrive at something like this:
switch pizzaFlavor {
case .hawaiian:
throw BadFlavorError()
case .pepperoni:
try validateNoVegetariansEating()
return .delicious
case .cheese:
return .delicious
@unknown default:
try validateNoVegetariansEating()
return .delicious
}
When a resilient library knows that an enumeration will not be extended, and wants to improve the performance of using it, the author can annotate the enum with @frozen. This annotation has a wide range of effects, but one of its effects is to enable callers to perform exhaustive switches over the frozen enumeration. Thus, resilient library authors that are interested in the exhaustive switching behaviour are able to opt-into it.
However, in Swift today it is not possible for the default, non-resilient dialect to opt-in to the extensible enumeration behaviour. That is, there is no way for a Swift Package Manager package to be able to evolve a public enumeration without breaking the API. This is a substantial limitation, and greatly reduces the utility of enumerations in non-resilient Swift. As a non-exhaustive list of problems this can cause:
- Using enumerations to represent
Errors is inadvisable, as if new errors need to be introduced they cannot be added to existing enumerations. This leads to a proliferation ofErrorenumerations. "Fake" enumerations can be made usingstructs andstatic lets, but these do not work with the niceErrorpattern-match logic incatchblocks, requiring type casts. - Using enumerations as tagged unions to allow multiple types to be passed to a single argument slot is not a good idea unless the library author is confident that no new types will be added. This forces developers into the pattern of overloads instead. Doing this on functions is ok, but doing it with properties or getters is very painful, as this can lead to confusion in the type system.
- Using an enumeration to refer to a group of possible ideas without entirely exhaustively evaluating the set is potentially dangerous, requiring a deprecate-and-replace if any new elements appear.
- Using an enumeration to represent any concept that is inherently extensible is tricky. For example, SwiftNIO uses an enumeration to represent HTTP status codes. If new status codes are added, SwiftNIO needs to either mint new enumerations and do a deprecate-and-replace, or it needs to force these new status codes through the
.customenum case.
This proposal plans to address these limitations on enumerations in non-resilient Swift.
Proposed solution
This solution would add a new attribute to Swift, analogous to @frozen. We propose to name this attribute @extensible. When applied to a public enum, this will cause the behaviour of the switch statement in calling code to be the same as if the module that provided the enum was compiled in library evolution mode. Specifically, this will issue warnings if there is no @unknown default clause provided, and will not lead to compile failures if new clauses are added to the enumeration in the presence of such a clause.
In essence, this attribute will cause the compiler to treat an enum annotated with @extensible to be as though it were defined in a resilient library for the purposes of switch statements. It will change nothing else about its representation: these enumerations will continue not to be ABI stable in the absence of -enable-library-evolution.
Source compatibility
Adding new attributes is not a source breaking change. This proposal does enable the ability to make more non-source-breaking changes for non-resilient libraries.
Effect on ABI stability
This attribute does not affect the ABI, as it is a no-op when used in a resilient library.
Effect on API resilience
This proposal only affects API resilience of non-resilient libraries, by enabling more changes to be made without API breakage.
Alternatives considered
Resolving the dialect issue
The Core Team has made it clear that they are opposed to language dialects in general. One may then conclude that the Core Team would ideally like for the current language dialect situation to go away. In order to do this, one of the two language dialects must undergo a source-breaking change, as their default enum exhaustibility behaviour will have to change.
Source-breaking changes are extremely heavyweight. It is likely that no further source-breaking changes will be accepted until Swift 6, which has no concrete timetable. For that reason, attempting to resolve this discrepancy by resolving the dialect issue cannot happen until Swift 6 until the very earliest: a time that does not currently concretely exist.
Additionally, it's not clear to the authors exactly which direction the language dialect would be resolved in. Will resilient libraries have enums default to exhaustive? This seems like a deeply problematic outcome, as it's non-obvious to library authors (who always exhaustively switch over their own enums) and so will likely not be noticed until users complain about source breaking changes. If this did happen, of course, an attribute would be needed to mark an enum as extensible: we could therefore add that attribute now.
Alternatively, the non-resilient dialect could be changed to make all enums extensible unless marked otherwise. This is easier to motivate, as the downsides of forgetting to mark your enum @frozen is user-annoyance, not user-breakage. Additionally, making a non-frozen enum @frozen is not source-breaking. This is a plausible alternative to this proposal, and could be pursued by the community in lieu of adopting this work.
Versioning
SE-0192 notes that:
Earlier versions of this proposal included syntax that allowed all public Swift enums to have a frozen/non-frozen distinction, rather than just those in the standard library and overlays. This is still something we want to support, but the core team has made it clear that such a distinction is only worth it for libraries that have binary compatibility concerns (such as those installed into a standard location and used by multiple clients), at least without a more developed notion of versioning and version-locking. Exactly what it means to be a "library with binary compatibility concerns" is a large topic that deserves its own proposal.
This is addressing a specific problem with having extensible enumerations. When switching over all cases, even with @unknown default, the compiler will warn if a known case is not explicitly handled. This means that code written against library version X.Y will emit warnings when compiled against X.Y+1 if X.Y+1 introduces new enum cases. For users that compile with warnings-as-errors, this is annoying. When paired with the absence of @available guards for non-OS Swift modules, this essentially makes it impossible to use warnings-as-errors if you are depending on Swift libraries that use extensible enums.
This problem currently exists today for all non-@frozen public enums defined in non-OS libraries compiled with -enable-library-evolution. However, making extensible enumerations available to non-resilient libraries without supporting version guards will definitely make this problem more pervasive. It's the opinion of the author of this pitch that this is an acceptable trade-off, but the community may well disagree.
To avoid this issue would require a much more substantial change than this proposal envisions, requiring extensions to the Swift module system to include a notion of versioning, as well as extensions to the availability system and the Swift Package Manager. Such a wide-ranging change is not to be entered into lightly!
Other than that I think itās a good idea to support both variants to help library authors.