[Pitch] Extensible enums for non-resilient modules

[FranzBusch]

@tshortli

Libraries built with library evolution. A migration to add @frozen must not be offered when building these modules. I assume it should be straightforward to make sure that the migration tooling takes the presence of -enable-library-evolution into account (though there is some code out there that is built both as a package and also as a resilient library, depending on context, and I don't know how the tooling would detect that).

I think this makes sense. We should be able to disable the migration fix-it if the module is build with library evolution. Side note (the proposal is not updated yet with any of the migration tooling since the migration tooling feature itself hasn't gone through review yet)

@tshortli

Application code, or any other kind of leaf-node module. The only clients of a module like this are tests, and therefore source compatibility does not seem relevant.

This is an interesting one. For executable targets either via SwiftPM or inside Xcode this seems sensible. To not make this overly complicated I'm wondering if appropriately phrasing the migration note + suggested fix-it might be enough. Something along those lines:

This enumeration was previously treated as non-extensible. With the ExtensibleEnum feature this is now treated as extensible. If your module is consumed by others this change in behaviour might be source breaking. You can either accept the break in behaviour or apply @frozen to mark this enum as non-extensible.

What do you think about solving this via a more length migration note?

@tshortli

Packages that aren't open source. It's hard to state anything concrete about the requirements of this kind of code. For some of these packages, maybe source compatibility is important to the organization that maintains them. For others, source compatibility may not really be relevant at all.

I personally don't think there is anything special for internal packages that we need to consider here. From my experience they often have the same API concerns as public packages but are just way more open to break their API and release a new major. I would suggest that a note like the above would cover those cases as well.

@tshortli

I think it's fine to implement a migration tool that can put @frozen on every public enum as a convenience, but I'd prefer to see it reserved as a tool that is available to experts who know they need it, rather than something that is promoted to everyone adopting the latest Swift language mode that includes this proposal.

I agree with this. I don't want everyone to just spray @frozen everywhere and I hope the proposal for adoption tooling for upcoming features land so that we can leverage it here. I agree a good note explaining the choices developers have is needed.

[John_McCall]

I'm broadly happy with this proposal and think it's very important to address. I agree that we should flip the default for cross-package references. I also agree that we will need to ensure that common environments like Xcode have a better default story for packages before we can do that. Xcode is not under Swift Evolution, of course, but this is still a practical necessity.

I do think it would be reasonable to allow programmers to explicitly vendor other packages for the purposes of diagnostics like this. Basically, we would add a command line flag to tell the compiler that the current module considers package X to be source-stable despite how it might have been built. (This would not be allowed with binary-stable libraries, of course.) This would create an out for programmers who don't frequently update their dependencies and would rather just deal with source breaks as they come. It would be strongly discouraged in library packages, of course.

The root cause of the source compatibility issue here is that exhaustive switches are capable of observing what is otherwise an implementation detail of an enum, making the complete set of cases part of the enum's interface. This is really a more general problem that can arise with other potential language features. Some of these features are either theoretical or debatable^[1], but I'm currently working on one that's quite real. We want to add a borrow operator that allows programmers to avoid copies, and we want it to have force: Swift should never implicitly copy an operand that was written with borrow. Unfortunately, this has exactly the same source-compatibility problem as exhaustive switches: it observes whether you can borrow from a declaration, and therefore distinguishes a stored property from one written with get. So I would like a design that addresses this problem that other features can build on; as a result, I would encourage the design to prefer general names like frozen rather than names like extensible that only seem to apply to the enum problem.

---

1. An example that's come up before: suppose that Swift had a case pattern that could decompose a struct, like { x: <pattern>, y: <pattern> }. Arguably that pattern should only be allowed if x and y comprise the full set of stored properties, and incomplete sets of patterns should have to be written something like { x: <pattern>, y: <pattern>, ... }. This would make the full set of stored properties part of the interface. However, there's enough to debate with this idea that it tends to side-track the discussion. ↩︎

[johannesweiss]

+100, an easy, enthusiastic and obvious YES PLEASE, can I have it yesterday? :).

[Ben_Cohen]

In reading this thread, I am very sympathetic to the voices who expressed concern about the impact this (most recently update) proposal would have on the current common practice of breaking up code into modules, but where all the modules are treated as one codebase, and that they would always want the frozen behavior for all public types. The expectation that huge bodies of code that follow this practice would either have to splatter @frozen on all their enums to keep the current behavior, or embark on a move from public to package-level visibility, seems a high bar to clear to me.

At the same time, this change is clearly very important for the future evolution of the ecosystem. But I think this leads me to consider the alternative of an @extensible option.

The explanation for the alternative considered here is:

We believe that the default behavior in both language dialects should be that
public enumerations are extensible. ne of Swift's goals, is safe defaults and
the current non-extensible default in non-resilient modules doesn't achieve that
goal.

And this is fair. But another goal of Swift is to avoid ceremony, and yet another is to avoid source churn between language versions unless strongly justified, and this change will hit up against both of these. I am pretty skeptical the bar is cleared, and therefore @extensible may be preferable.

I believe there is another reason why @extensible may be worth considering. In talking to package authors, a theme has come up which is that in some cases when authoring a public API, a default either way is unfortunate. Other examples are defaulting to non-Sendable (maybe even non-Hashable). In these cases, it may be desirable to be able to put the compiler in a mode where it requires you to state either that a type is Sendable or not (via an unavailable conformance, or perhaps in the future a definition of ~Sendable).^[1] The same would work for enums – when authoring a package, every public enum would need to be either @extensible or @frozen, indicating a choice has been made.

Package authors could enable this "pedantic mode" if desired, while developers using the current idiom would be unaffected. It would also allow developers who want internal- or package-only enums to also require extensibility to enable that. It does mean we continue with two dialects – but that is the status quo, and I'm not seeing a strong enough case to change it, relative to this alternative.

---

1. There is already an experimental feature to do this in fact, via -require-explicit-sendable. ↩︎

[johannesweiss]

@Ben_Cohen

But I think this leads me to consider the alternative of an @extensible option.

[...]

@Ben_Cohen

Package authors could enable this "pedantic mode" if desired, while developers using the current idiom would be unaffected.

I can only speak for myself but I could live with allowing @extensible whilst offering a pedantic mode which forces authors to choose one or the other explicitly.

It's a little unfortunate that we're violating progressive disclosure with this amended suggestion though: Even in new language versions will still be to have a default which will remain @frozen. And @frozen is IMHO a bad default because changing to @extensible is source breaking. But then, we can't erase history and having to add @frozen everywhere just to retain existing behaviour isn't ideal either. Hence: Either the original suggestion or Ben's amendment would work for me, the important bit is that there will be an option for open enums in package land .

[FranzBusch]

Thanks for the feedback. I'm also very sympathetic to all the voices that expressed concerns regarding the migration. From what I can see we have two directions to choose from:

• Align the default behaviour of enums which means existing library authors have to annotate all existing public enums with @frozen
• Don't align the behaviour of enums and introduce a new attribute to mark enums as @extensible leaving the default a pitfall in API design

@Ben_Cohen

...but where all the modules are treated as one codebase

This seems to me like the biggest open problem. What do you and others think about exposing an option in the package manifest to set the package access identifier similar to what Xcode allows on a per target level via the Package Access Identifier setting? This would allow developers that break up their code base into multiple packages and treat them all as the same package access domain.

@Ben_Cohen

or embark on a move from public to package-level visibility,

This shouldn't be necessary if we provide the above proposed package access identifier option. Matching over public enums in the same package domain must be exhaustive which matches the current behaviour.

@Ben_Cohen

And this is fair. But another goal of Swift is to avoid ceremony, and yet another is to avoid source churn between language versions unless strongly justified, and this change will hit up against both of these. I am pretty skeptical the bar is cleared, and therefore @extensible may be preferable.

I agree that this will cause churn when adopting the new language mode but I don't yet see where it will cause ceremony. I would actually argue that the current proposed solution by aligning the behaviour will cause less ceremony long term since extensible enums are from my experience the better default.

@Ben_Cohen

In talking to package authors, a theme has come up which is that in some cases when authoring a public API, a default either way is unfortunate. Other examples are defaulting to non-Sendable (maybe even non-Hashable). In these cases, it may be desirable to be able to put the compiler in a mode where it requires you to state either that a type is Sendable or not (via an unavailable conformance, or perhaps in the future a definition of ~Sendable).[1]

With all the examples you mentioned here, e.g. Hashable or Sendable, the compiler defaults to the safe option. It doesn't implicitly commit to public API and leaves the module author the choice about those conformances later on. In non-resilient modules enums are exactly the opposite right now and lock the module author in by implicitly freezing them. This is for me is the main reason why I strongly prefer us aligning the defaults since I think the safe default goal of Swift is more important long term than the one-time churn between language modes.

@Ben_Cohen

Package authors could enable this "pedantic mode" if desired, while developers using the current idiom would be unaffected. It would also allow developers who want internal- or package-only enums to also require extensibility to enable that. It does mean we continue with two dialects – but that is the status quo, and I'm not seeing a strong enough case to change it, relative to this alternative.

I think this mode is good regardless of what approach we choose. Similar to the -require-explicit-sendable flag a -require-explicit-enum-extensibility flag makes sense to me.

[FranzBusch]

Thanks everyone for the healthy discussion and all the feedback so far. Many people have raised their concerns about the migration story for this new language feature. Especially in setups that are not the semantically versioned package world. We agree that it is important that this new feature can be easily adopted in every setup and doesn't result in a lot of churn. I just pushed a new commit that includes our proposed migration paths. I would love to hear from everyone if the proposed changes would solve their concerns around migrating code to this new feature. Below is a copy of the new migration paths section from the proposal.

Migration paths

The following section is outlining the migration paths and tools we propose to provide for different kinds of projects to adopt the proposed feature. The goal is to reduce churn across the ecosystem while still allowing us to align the default behavior of enums. There are many scenarios why these migration paths must exist such as:

• Projects split up into multiple packages
• Projects build with other tools than Swift PM
• Projects explicitly vendoring packages without wanting to modify the original
  source
• Projects that prefer to deal with source breaks as they come up rather than
  writing source-stable code

Semantically versioned packages

Semantically versioned packages are the primary reason for this proposal. The
expected migration path for packages when adopting the proposed feature is one
of the two:

• API stable adoption by turning on the feature and marking all existing public
  enums with @frozen
• API breaking adoption by turning on the feature and tagging a new major if the public API contains enums

Projects with multiple non-semantically versioned packages

A common project setup is splitting the code base into multiple packages that
are not semantically versioned. This can either be done by using local packages or by using revision locked dependencies. The packages in such a setup are often considered part of the same logical collection of code and would like to follow the same source stability rules as same module or same package code. We propose to extend the package manifest to allow overriding the package name used by a target.

extension SwiftSetting {
    /// Defines the package name used by the target.
    ///
    /// This setting is passed as the `-package-name` flag
    /// to the compiler. It allows overriding the package name on a
    /// per target basis. The default package name is the package identity.
    ///
    /// - Important: Package names should only be aligned across co-developed and
    ///  co-released packages.
    ///
    /// - Parameters:
    ///   - name: The package name to use.
    ///   - condition: A condition that restricts the application of the build
    /// setting.
    public static func packageName(_ name: String, _ condition: PackageDescription.BuildSettingCondition? = nil) -> PackageDescription.SwiftSetting
}

This allows to construct arbitrary package domains across multiple targets
inside a single package or across multiple packages. When adopting the
ExtensibleEnums feature across multiple packages the new Swift setting can be used to continue allowing exhaustive matching. While this setting allows treating multiple targets as part of the same package. This setting should only be used across packages when the packages are both co-developed and co-released.

Other build systems

Swift PM isn't the only system used to create and build Swift projects. Build
systems and IDEs such as Bazel or Xcode offer support for Swift projects as
well. When using such tools it is common to split a project into multiple
targets/modules. Since those targets/modules are by default not considered to be part of the package, when adopting the ExtensibleEnums feature it would
require to either add an @unknown default when switching over enums defined in other targets/modules or marking all public enums as @frozen. Similarly, to the above to avoid this churn we recommend specifying the -package-name flag to the compiler for all targets/modules that should be considered as part of the
same unit.

Escape hatch

There might still be cases where developers need to consume a module that is
outside of their control which adopts the ExtensibleEnums feature. For such
cases we propose to introduce a new flag --assume-source-stable-package that allows assuming modules of a package as source stable. When checking if a switch needs to be exhaustive we will check if the code is either in the same module, the same package, or if the defining package is assumed to be source stable. This flag can be passed multiple times to define a set of assumed-source-stable packages.

// a.swift inside Package A
public enum MyEnum {
    case foo
    case bar
}

// b.swift inside Package B compiled with `--assume-source-stable-package A`

switch myEnum { // No @unknown default case needed
case .foo:
    print("foo")
case .bar:
    print("bar")
}

In general, we recommend to avoid using this flag but it provides an important
escape hatch to the ecosystem.

[jameesbrown (James Brown)]

I like this proposal and agree that frozen is a better approach than introducing extensible . My concern with --assume-source-stable-package is that if a module is truly outside of your control, using this flag runs counter to part of the intent of the proposal and might not be the best practice. Would the escape hatch not simply be avoiding enabling the language mode in the package that consumes the module, similar to how Swift 6 mode can be adopted on a per-module basis?

[FranzBusch]

@jameesbrown

I like this proposal and agree that frozen is a better approach than introducing extensible . My concern with --assume-source-stable-package is that if a module is truly outside of your control, using this flag runs counter to part of the intent of the proposal and might not be the best practice. Would the escape hatch not simply be avoiding enabling the language mode in the package that consumes the module, similar to how Swift 6 mode can be adopted on a per-module basis?

This feature is different than the Swift 6 language mode since the change in behavior happens when the module declaring the enums adopts the feature or the new language mode. In practice, this means a dependency of yours might choose to adopt this feature and cause your code to start emitting errors about exhaustively matching an extensible enum requiring you to add an @unknown default case. In these scenarios, developers might choose to use the proposed --assume-source-stable-package flag if they would rather deal with the potential source breaks when they update the dependency next time.
This flag is really intended to be an escape hatch and shouldn't be used broadly. In fact, when using it in a package it would need to be declared as an unsafe flag; hence, only really useable by leaf packages.

[jameesbrown (James Brown)]

Thanks for the clarification. I understand the feature overall (I’ve read the proposal and the discussion here), but I initially envisioned the flag being used in a broader context. My thought was whether the language mode itself could implicitly carry that flag.

This flag is really intended to be an escape hatch and shouldn't be used broadly. In fact, when using it in a package it would need to be declared as an unsafe flag; hence, only really useable by leaf packages.

I now see why my original concern doesn’t quite apply and retract my earlier criticism.

[hborla (Holly Borla)]

Thanks for adding a detailed section on migration paths. I've read through it a few times and I do not think it's sufficient.

First, I do not agree with the decision to remove the explicit @extensible attribute on enums. I believe that undertaking any transition that changes what existing, valid code means must have an explicit spelling for both behaviors so that you can write code that has consistent behavior independent of upcoming features / language mode, even if we do not think the attribute will exist in the long term. We learned this with SE-0285: Ease the transition to concise magic file strings, and again with SE-0461: Run nonisolated async functions on the caller's actor by default. An explicit, transitory attribute is valuable because there will be a period of time where it is not immediately clear from source what kind of enum a programmer is working with. It's necessary to be able to discover that information from source, such as by showing an inferred attribute explicitly in SourceKit's cursor info request (surfaced by "Quick Help" in Xcode and "Hover" in LSP / VSCode). An explicit spelling that has consistent behavior independent of language mode is also valuable for code generation tools like macros, so that they do not have to consider build settings to determine the right code to generate, it's valuable for posting code snippets on the forums during the transition period, etc. I strongly believe that the @extensible spelling is both necessary and separable from changing the default in a future language mode.

Second, I believe that it's critical for the migration path to have a mechanism for libraries to stage in changing an enum from @frozen to @extensible as warnings for the client. The proposal has this justification for not including a @preconcurrency-like annotation for accomplishing this:

We considered introducing an annotation that allows developers to mark enumerations as pre-existing to the new language feature similar to how @preconcurrency works. Such an annotation seems to work initially when existing public enumerations are marked as @preEnumExtensibility instead of @frozen . It would result in the error about the missing @unknown default case to be downgraded as a warning. However, such an annotation still doesn't allow new cases to be added since there is no safe default at runtime when encountering an unknown case.

I don't find this argument convincing, because the diagnostic for omitting an @unknown default case for resilient enums was a warning by default for years, and only promoted to an error by default in the Swift 6 language mode. This means that the vast majority of people are writing code in a set up where this is already a warning that will result in an assertion failure if the unhandled enum case is actually encountered at runtime, and I haven't seen evidence that this is a serious problem in practice. I also don't see the necessity to be judicious about when to start introducing new enum cases as a reason to not have the ability to stage in the diagnostic to add an @unknown default case as a warning for clients before the library starts to add new cases. One reasonable path that a library might take is to mark a public enum that should be extensible as @preExtensibleEnums @extensible a release prior to adding new enum cases. This way, clients make the code change to support an unknown enum case before they make the code change to support specific new enum cases, which helps ease the transition.

In particular, I think it's a problem that the only migration path offered for non-resilient libraries that do not use semantic versioning and are not part of the same package as their clients is to use the escape hatch on the client side. This leaves the library author with no good options to facilitate the migration. Either they need to know about all of their clients and get them to enable the --assume-source-stable-package flag before making their enums extensible on the library side, or they simply make the breaking change and let clients deal with the fallout. I believe it's important to provide library authors with a mechanism for easing the transition for their clients that does not involve blocking the library on the client making a change.