SE-0192 — Non-Exhaustive Enums

John_McCall · January 1, 2018, 6:40am

Well, this is both true, and false, and true again.

It's true that frozen/inlineable on a function leaves the choice of whether to actually perform inlining (or other interprocedural optimizations like specialization) up to the compiler, whereas the impact of frozen on a type has a direct and reliably-observable impact in the sense of lifting certain semantic restrictions.

But from another perspective, the meaning of the attribute to the implementor is the same in both cases: it means that the implementation is the same across versions. frozen on a type means that the basic storage structure of the type won't change (i.e. it has exactly the same stored properties/cases), and frozen on a function means that the behavior of the function won't change.

But then again, while we can reliably check the correctness of frozen on a type, we can't really check that for a function. We can check frozen-correctness on a type because we don't mind forbidding non-trivial changes to the storage structure: i.e. the set of stored properties/cases (and their types) have to remain exactly the same, meaning you can't even do "obviously equivalent" changes like combining two stored properties into a single property of tuple type. But we don't want to be that strict with functions because we do want to allow some non-trivial differences: it's frequently possible to improve the implementation of a function without changing its behavior, and we don't want to make that impossible, but we also don't really want to get into the business of requiring library authors to prove the semantic equivalence of two different function bodies. That means we just have to take implementors at their word that using the new function body instead of the old isn't too annoying a difference. I know Joe Groff has proposed in the past that we use a somewhat different semantic model for inlineable functions, one that promises that we use the newest available functionality, and I think there's some merit to that.

To me, the important questions around keyword choice are:
  - whether the operations feel sufficiently different that using the same keyword is going to feel awkward;
  - whether we might be preventing useful expressiveness by using the same keyword (the thing that occurs to me is that having a separate attribute for inlineable might let us easily mass-annotate an extension, which seems like a common use-case); and
  - whether we can get community agreement about it, which might seem "meta", but sometimes one must pick one's battles.

John.

···

On Dec 31, 2017, at 1:21 PM, Cheyo Jimenez <cheyo@masters3d.com> wrote:

On Dec 31, 2017, at 8:59 AM, Ben Rimmington via swift-evolution <swift-evolution@swift.org> wrote:
On 21 Dec 2017, at 03:32, John McCall wrote:

On Dec 20, 2017, at 10:16 PM, Brent Royal-Gordon wrote:

On Dec 19, 2017, at 2:58 PM, Ted Kremenek wrote:

• What is your evaluation of the proposal?

I am pleased with the broad strokes of this design. I have quibbles with three areas:

1. The `@exhaustive` attribute may be confusing because the term doesn't suggest versioning. My best alternative suggestion is `@frozen`, which matches existing programming terminology: something that has been frozen will not be changed in the future.

I rather like @frozen. We could use that across language features, so that we don't end up with a keyword per kind of declaration.

Could this also be used on functions to make them inlinable?
i.e. The body of the function has been frozen.
@frozen
public func a()

@available(*, frozen)
public func b()

@available(swift, introduced: 4.1, frozen: 5.0)
public func c()
My understanding is that frozen / exhaustible is guaranteed by the compiler while inlineable is more of a strong suggestion to the compiler. It would be confusing to use the same word for both.

xwu · January 2, 2018, 6:48pm

Sent from my iPad

I agree that we need a solution to the problem described. I also agree
that non-exhaustive is most in keeping with the overall design of Swift at
module boundaries. However, I believe this proposal should be modified
before being accepted

Thanks for writing this up - you’ve explained a common concern in an
interesting way:

This is likely to be a relatively rare need mostly encountered by 3rd
party libraries but it will happen. When it does happen it would be really
unfortunate to be forced to use a `default` clause rather than something
like a `future` clause which will produce an error when compiled against an
SDK where the enum includes cases that are not covered. I can imagine
cases where this catch-all case would need to do something *other than *abort
the program so I do not like the `switch!` suggestion that has been
discussed. The programmer should still be responsible for determining the
behavior of unknown cases.

..

While library authors have a legitimate need to reserve the right to
introduce new cases for some enums this need can be met without taking away
a useful tool for generating static compiler errors when code does not
align with intent (in this case, the intent being to cover all known
cases). Switch statements working with these kinds of enums should be
required to cover unknown cases but should be able to do so while still
being statically checked with regards to known cases.

I think that this could be the crux of some major confusion, the root of
which is the difference between source packages and binary packages that
are updated outside your control (e.g. the OS, or a dynamic library that is
updated independently of your app like a 3rd party plugin). Consider:

1) When dealing with independently updated binary packages, your code
*has* to implement some behavior for unexpected cases if the enum is
non-exhaustive. It isn’t acceptable to not handle that case, and it isn’t
acceptable to abort because then your app will start crashing when a new OS
comes out. You have to build some sort of fallback into your app.

2) When dealing with a source package that contributes to your app (e.g.
through SwiftPM), *YOU* control when you update that package, and therefore
it is entirely reasonable to exhaustively handle enums even if that package
owner didn’t “intend” for them to be exhaustive. When *you* chose to
update the package, you get the “unhandled case” error, and you have
maximal “knowability” about the package’s behavior.

It seems that your concern stems from the fact that the feature as
proposed is aligned around module boundaries, and therefore overly punishes
source packages like #2. I hope you agree that in case #1, that the
feature as proposed is the right and only thing we can do: you really do
have to handle unknown future cases somehow.

If I’m getting this right, then maybe there is a variant of the proposal
that ties the error/warning behavior to whether or not a module is a source
module vs a binary module. The problem with that right now is that we have
no infrastructure in the language to know this…

Hi Chris, thanks for your reply.

The concern you describe isn’t exactly what I was describing but it is
related. John McCall recently posted a sketch of a solution to the concern
you describe which looked great to me. I don’t have time to look up the
link this morning but I think it was in this review thread.

The actual concern I am describing is where a 3rd party library (or app)
wants to switch over a non-exhaustive enum provided by a module that is a
binary (not source) dependency. The author of the 3rd party library may
have a legitimate reason to switch over an enum despite the author of the
binary module reserving the right to add additional cases.

When this circumstance arises they will do it using the tools provided by
the language. Regardless of the final language solution they obviously
need to cover unknown cases - their library could be shipping on a device
which receives an update to the binary dependency that contains a new
case. I agree with you that a language-defined crash is not appropriate.
The author of the switch must take responsibility for the behavior of
unknown cases.

I am arguing that these “pseudo-exhaustive” switch statements *will*
exist in the wild. The crucial point of contention is whether or not the
language provides assistance to the author of the 3rd party library in
updating their library when the enum provided by the binary dependency
changes. Is the author forced to use a `default` case which turns of
exhaustiveness checking? Or are they able to use an alternative mechanism
for handling unknown cases which does not turn off exhaustiveness checking
- all *statically known* cases must be covered. The most common example
of such a mechanism is the `future` (or perhaps `unknown`) case which would
only be used for cases that *are not* statically known.

This facility will of course help authors of these switch statements make
the necessary updates as the enum vended by the binary dependency changes.
It *will also *help alert authors of apps that depend on that 3rd party
library (which will usually be a source dependency). If the author of the
app attempts to rebuild the dependency against a new SDK with added cases
the library will fail to build, alerting the user that they should update
the 3rd party library.

My position is that if there are *reasonable* use cases for these kinds
of “pseudo-exhaustive” switches then the language should provide
exhaustiveness checking of statically known cases via some mechanism that
authors can opt-in to using. It’s ok with me if this is a relatively
esoteric feature. It won’t be *commonly* needed, but when it is
necessary it will provide significant value.

IIRC there were some reasonable examples of these kinds of switches posted
in earlier threads on this topic. I don’t have time to look those up right
now either but it would be good for the core team to be aware of them
before making a final decision.

I am only aware of two arguments *against *this kind of
“pseudo-exhaustive” switch. One is that users should not attempt to switch
over an enum that a library author does not intend to be exhaustive (i.e.
it is an “input-only” enum). The other is that a `future` or `unknown`
case is not testable.

The first argument is a moral one which I believe should not carry much
weight relative to concrete, pragmatic counter-examples such as those that
(IIRC) were provided on this list in the past.

I think the discussion is converging on the defect here, but I disagree
that the first argument is merely a "moral one"; it is an epistemological
one.

The epistemological dilemma is: how can the user of the library "know" all
the cases when the library "author" has made it clear that even _they_
don't know all the cases? Obviously, the library user can only switch over
the cases that _he or she_ knows about. And if the user has forgotten about
some cases that are documented, then clearly, he or she doesn't know about
it.

The argument here is rather that it should be possible to ensure that the
user has switched over all the cases that _the compiler_ knows about.
Suppose we accept the argument that we want to enable the user to deal with
the largest possible set of known cases: why does it have to the compiler
that does the warning? why is this not a task for a linter if so desired?
in what other circumstances do we insist that the compiler inform the end
user about future additions to the API at compile time?

The second argument doesn’t make sense to me: as I noted in my review post,

···

On Tue, Jan 2, 2018 at 9:38 AM, Matthew Johnson via swift-evolution < swift-evolution@swift.org> wrote:

On Jan 1, 2018, at 11:47 PM, Chris Lattner <clattner@nondot.org> wrote:
On Dec 31, 2017, at 12:14 PM, Matthew Johnson via swift-evolution < > swift-evolution@swift.org> wrote:
the code path is equally untestable when a `default` case is used, but all *statically
known* cases are matched in earlier patterns. The testability problem of
non-exhaustive enums is orthogonal to the issue of language support for
“pseudo-exhaustive” switches.

This is the line of reasoning which leads me to conclude that we should
dig up the concrete examples which have been provided and evaluate them for
merit. If we can’t discard them as a bad coding practice for which a
better solution is available then we should strongly consider providing
language support for these use cases. The existence of such use cases
should also motivate a solution to the testability problem.

- Matthew

-Chris

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

xwu · January 3, 2018, 3:03am

Indeed, you're quite right. And the whole notion is actually half-baked
because a @testable import should treat the enum as though internal and
therefore exhaustive.

It would appear, however, that testability would require the library itself
to handle unknown cases. That seems annoying but potentially justifiable.

···

On Tue, Jan 2, 2018 at 8:49 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Jan 2, 2018, at 8:45 PM, Xiaodi Wu via swift-evolution < > swift-evolution@swift.org> wrote:

On Tue, Jan 2, 2018 at 8:07 PM, Jordan Rose via swift-evolution < > swift-evolution@swift.org> wrote:

[Proposal: https://github.com/apple/swift-evolution/blob/mas
ter/proposals/0192-non-exhaustive-enums.md]

Whew! Thanks for your feedback, everyone. On the lighter side of
feedback—naming things—it seems that most people seem to like '*@frozen*',
and that does in fact have the connotations we want it to have. I like it
too.

More seriously, this discussion has convinced me that it's worth
including what the proposal discusses as a *'future' case*. The key
point that swayed me is that this can produce a *warning* when the
switch is missing a case rather than an *error,* which both provides the
necessary compiler feedback to update your code and allows your
dependencies to continue compiling when you update to a newer SDK. I know
people on both sides won't be 100% satisfied with this, but does it seem
like a reasonable compromise?

The next question is how to spell it. I'm leaning towards `unexpected
case:`, which (a) is backwards-compatible, and (b) also handles "private
cases", either the fake kind that you can do in C (as described in the
proposal), or some real feature we might add to Swift some day. `unknown
case:` isn't bad either.

I too would like to just do `unknown:` or `unexpected:` but that's
technically a source-breaking change:

switch foo {
case bar:
  unknown:
  while baz() {
    while garply() {
      if quux() {
        break unknown
      }
    }
  }
}

Another downside of the `unexpected case:` spelling is that it doesn't
work as part of a larger pattern. I don't have a good answer for that one,
but perhaps it's acceptable for now.

I'll write up a revision of the proposal soon and make sure the core team
gets my recommendation when they discuss the results of the review.

---

I'll respond to a few of the more intricate discussions tomorrow,
including the syntax of putting a new declaration inside the enum rather
than outside. Thank you again, everyone, and happy new year!

I do like this spelling of `@frozen`, and `unknown case` looks perfectly
cromulent to me. If this is the path to go down, I'd urge more explicit
design as to what happens when `unknown case` and `default` are mixed. I
would imagine the most consistent design would be:

`unknown case` should allow `default` to be omitted if the switch is
otherwise exhaustive, obviously.
`default` should allow `unknown case` to be omitted, just like any other
case may then be omitted.
`unknown case` before `default` should be allowed, just like any other
case before `default`; in that case, only known cases not otherwise matched
reach the `default`.
`default` before `unknown case` makes the latter unreachable, just like
any other case after `default`.

The issue here remains that of testability. I wonder if, for such
purposes, unknown case should be instantiable when testably imported, with
some grammar. In its simplest and yet most exotic form, we could imagine
code that testably imports the enum to be allowed to instantiate any
made-up case whatsoever (e.g., `@testable import Foo.MyEnum; let x = MyEnum.
asdfasdfasdfNonexistent`).

What should happen when an unknown case instantiated via a testability
mechanism is passed to the library that vended the enum (which is able to
truly exhaustively switch over the enum)? I would like to see a solution
to the testability problem and answering this question seems to be the most
difficult part of finding a solution. The best answer is not obvious to me.

Karl · January 2, 2018, 8:35pm

So if I can summarise the problem (for myself), what you’re saying is there is an enum in some OS library...

enum OSButtonState {
case normal, selected
}

And in my App, I switch its values. As far as this proposal goes, there is only one way to write it (with a “default”):

switch buttonState {
    case .selected: …
    case .normal: ...
    default: // reset state, do not react… I suppose?
}

Then, an update happens. Some new functionality is added, and the enum gains a new case to expose that:

enum OSButtonState {
    case normal, selected
    @available(1.1)
    case focussed
}

Now, on the one hand: yes, my original “switch” statement is still valid Swift code (from the perspective of “it will compile”), but it may no longer be semantically correct.
Those new enum cases may be reflecting important state transitions which my App needs to be aware of to maintain its own internal state. There are a couple of points that I take from this:

1) OK, fair point. “Default” is not expressive enough, and I’m starting to see the point in a “future” case. I think it’s useful to have a way to write a switch which tracks the latest version of the enum that the compiler can see. It would behave the same as a “default” in practice, but wouldn’t short-circuit the compiler checking that all currently-known cases are handled. Of course, it’s not the same as the strict guarantee given by an @exhaustive enum, and I have some concerns in general with our “catch-all” approach to dealing with enum evolution, but if we go that way I see value in annotating a best-attempt to stay in sync with the library.

2) As for apps supporting "pseudo-exhaustive” switching against multiple versions of the library, I assume something like this would be possible:

if @available(OSKit 1.1) {
    switch buttonState {
        case .selected: …
        case .focussed: ...
        case .normal: …
        future: ...
    }
} else {
    switch buttonState {
        case .selected: …
        case .normal: …
        future: ...
    }
}

This could be useful if you have complex state-machines which would be awkward when some cases were @available and others not. So you could hoist the availability check over the entire switch.

3) This proposal is predicated on the assumption that App developers are able to write non-fatal “default” execution paths to keep their internal state consistent, even if they can’t predict which new features or options or state-transitions the enum will later expose. I’m not sure that’s entirely realistic. Even if the compiler was going to help us out with exhaustiveness in switching, App developers would still require excellent documentation, API design, support and testing from library authors in order to make this work in practice. Perhaps in the end it won’t actually make anybody’s life easier, just less certain?

I see an analogue to protocols. Whenever people propose new protocols (or additions to existing protocols), we ask: "which useful generic algorithms can you write with this?”. Similarly, for this new control flow event (an unknown value in an enum), we need to ask: “how can you reasonably handle this event?”. Is it really the correct approach to punt all future enum values to some generic “catch-all” event in the client?

The App has absolutely no idea what value the enum has, which semantic implications that might have (if any), or if the library has any unique requirements when these events are triggered. These are all easy-to-miss details that lie behind simply which cases are available; typically they will require that your library author has carefully planned their API to avoid such unique handling requirements and has documented to you exactly how you should handle these future cases.

I think that’s what Dave DeLong has been saying; that we should be focussing on tools to help library authors make backwards-compatible changes, rather than forcing app developers to make their logic forwards-compatible to account for changes they cannot influence or foresee.

Maybe we are focussing too much on the language impact, and ensuring code continues to compile. The devil is in the details, as they say. In an updated-library situation, it’s the library which knows everything, and the App has absolutely no idea what changed. I’m not sure it makes sense to insist that the App handle this situation.

I don’t have a concrete counter-proposal in that respect, but then, I don’t have the experience dealing with these issues that the former/current Apple folks have. I would naively guess libraries would need a runtime-defined compatibility version, and would be able to inspect it in a way similar to @availability checking, returning only values that are consistent with the semantics the client expects (and allowing them to define their own “unknown” cases with documented handling behaviour, if appropriate for that enum). I could see how something like that could be more code-heavy and burdensome for library authors, though.

- Karl

···

On 2. Jan 2018, at 16:38, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

Sent from my iPad

On Jan 1, 2018, at 11:47 PM, Chris Lattner <clattner@nondot.org <mailto:clattner@nondot.org>> wrote:

On Dec 31, 2017, at 12:14 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I agree that we need a solution to the problem described. I also agree that non-exhaustive is most in keeping with the overall design of Swift at module boundaries. However, I believe this proposal should be modified before being accepted

Thanks for writing this up - you’ve explained a common concern in an interesting way:

This is likely to be a relatively rare need mostly encountered by 3rd party libraries but it will happen. When it does happen it would be really unfortunate to be forced to use a `default` clause rather than something like a `future` clause which will produce an error when compiled against an SDK where the enum includes cases that are not covered. I can imagine cases where this catch-all case would need to do something other than abort the program so I do not like the `switch!` suggestion that has been discussed. The programmer should still be responsible for determining the behavior of unknown cases.

..

While library authors have a legitimate need to reserve the right to introduce new cases for some enums this need can be met without taking away a useful tool for generating static compiler errors when code does not align with intent (in this case, the intent being to cover all known cases). Switch statements working with these kinds of enums should be required to cover unknown cases but should be able to do so while still being statically checked with regards to known cases.

I think that this could be the crux of some major confusion, the root of which is the difference between source packages and binary packages that are updated outside your control (e.g. the OS, or a dynamic library that is updated independently of your app like a 3rd party plugin). Consider:

1) When dealing with independently updated binary packages, your code *has* to implement some behavior for unexpected cases if the enum is non-exhaustive. It isn’t acceptable to not handle that case, and it isn’t acceptable to abort because then your app will start crashing when a new OS comes out. You have to build some sort of fallback into your app.

2) When dealing with a source package that contributes to your app (e.g. through SwiftPM), *YOU* control when you update that package, and therefore it is entirely reasonable to exhaustively handle enums even if that package owner didn’t “intend” for them to be exhaustive. When *you* chose to update the package, you get the “unhandled case” error, and you have maximal “knowability” about the package’s behavior.

It seems that your concern stems from the fact that the feature as proposed is aligned around module boundaries, and therefore overly punishes source packages like #2. I hope you agree that in case #1, that the feature as proposed is the right and only thing we can do: you really do have to handle unknown future cases somehow.

If I’m getting this right, then maybe there is a variant of the proposal that ties the error/warning behavior to whether or not a module is a source module vs a binary module. The problem with that right now is that we have no infrastructure in the language to know this…

Hi Chris, thanks for your reply.

The concern you describe isn’t exactly what I was describing but it is related. John McCall recently posted a sketch of a solution to the concern you describe which looked great to me. I don’t have time to look up the link this morning but I think it was in this review thread.

The actual concern I am describing is where a 3rd party library (or app) wants to switch over a non-exhaustive enum provided by a module that is a binary (not source) dependency. The author of the 3rd party library may have a legitimate reason to switch over an enum despite the author of the binary module reserving the right to add additional cases.

When this circumstance arises they will do it using the tools provided by the language. Regardless of the final language solution they obviously need to cover unknown cases - their library could be shipping on a device which receives an update to the binary dependency that contains a new case. I agree with you that a language-defined crash is not appropriate. The author of the switch must take responsibility for the behavior of unknown cases.

I am arguing that these “pseudo-exhaustive” switch statements will exist in the wild. The crucial point of contention is whether or not the language provides assistance to the author of the 3rd party library in updating their library when the enum provided by the binary dependency changes. Is the author forced to use a `default` case which turns of exhaustiveness checking? Or are they able to use an alternative mechanism for handling unknown cases which does not turn off exhaustiveness checking - all statically known cases must be covered. The most common example of such a mechanism is the `future` (or perhaps `unknown`) case which would only be used for cases that are not statically known.

This facility will of course help authors of these switch statements make the necessary updates as the enum vended by the binary dependency changes. It will also help alert authors of apps that depend on that 3rd party library (which will usually be a source dependency). If the author of the app attempts to rebuild the dependency against a new SDK with added cases the library will fail to build, alerting the user that they should update the 3rd party library.

My position is that if there are reasonable use cases for these kinds of “pseudo-exhaustive” switches then the language should provide exhaustiveness checking of statically known cases via some mechanism that authors can opt-in to using. It’s ok with me if this is a relatively esoteric feature. It won’t be commonly needed, but when it is necessary it will provide significant value.

IIRC there were some reasonable examples of these kinds of switches posted in earlier threads on this topic. I don’t have time to look those up right now either but it would be good for the core team to be aware of them before making a final decision.

I am only aware of two arguments against this kind of “pseudo-exhaustive” switch. One is that users should not attempt to switch over an enum that a library author does not intend to be exhaustive (i.e. it is an “input-only” enum). The other is that a `future` or `unknown` case is not testable.

The first argument is a moral one which I believe should not carry much weight relative to concrete, pragmatic counter-examples such as those that (IIRC) were provided on this list in the past.

The second argument doesn’t make sense to me: as I noted in my review post, the code path is equally untestable when a `default` case is used, but all statically known cases are matched in earlier patterns. The testability problem of non-exhaustive enums is orthogonal to the issue of language support for “pseudo-exhaustive” switches.

This is the line of reasoning which leads me to conclude that we should dig up the concrete examples which have been provided and evaluate them for merit. If we can’t discard them as a bad coding practice for which a better solution is available then we should strongly consider providing language support for these use cases. The existence of such use cases should also motivate a solution to the testability problem.

- Matthew

-Chris

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

hlovatt · January 3, 2018, 6:22am

I am not convinced by these arguments, they seem to be a ‘poor man’s’ versioning system. For example consider:

    // In module.
    public enum E {
        case A, B, C
    }

    // In application.
    switch e {
    case A: a()
    default: d()
    unknown case: u()
    }

When e == B or C is u() or d() called? I would expect d() since the application programmer obviously intends to handle unexpected differently than default.

Now when E is modified and case D added by the module programmer I would expect B and C to still call d() and D to call u().

To achieve the above behaviour the switch encodes that when it compiled default was for B and C and therefore D is the new case and therefore it calls u().

When the code is recompiled against the new module the behaviour changes. D will now call d(). This will be without a warning. Hence I am classing this as a ‘poor man’s’ module system.

Possible solutions include:

1. You can’t have a default with an extensible enum, but you must have a unknown case. This prevents handling default cases at all, you have to list all the existing cases separately.

2. As described above in 1 the unknown case does very little. Instead just use default and don’t introduce unknown.

3. Have a versioned module system that requires enum cases and matching switch statements to be versioned. EG:

    // In module.
    @version(1) public enum E {
        case A, B, C
    }
    @version(1.6) public enum E {
        case A, C, D
    }

    // In application.
    @version(1.5) switch e {
    case A: a()
    default: d()
    unknown case: u()
    }

The module system would have to publish which enum cases were available for each version including all old versions. Note how the above notation allows removal and addition of cases.

-- Howard.

···

On 3 Jan 2018, at 12:26 am, Kelvin Ma via swift-evolution <swift-evolution@swift.org> wrote:

On Tue, Jan 2, 2018 at 11:45 PM, Nevin Brackett-Rozinsky via swift-evolution <swift-evolution@swift.org> wrote:

On Tue, Jan 2, 2018 at 9:07 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org> wrote:
[Proposal: https://github.com/apple/swift-evolution/blob/master/proposals/0192-non-exhaustive-enums.md\]

Whew! Thanks for your feedback, everyone. On the lighter side of feedback—naming things—it seems that most people seem to like '@frozen', and that does in fact have the connotations we want it to have. I like it too.

More seriously, this discussion has convinced me that it's worth including what the proposal discusses as a 'future' case. The key point that swayed me is that this can produce a warning when the switch is missing a case rather than an error, which both provides the necessary compiler feedback to update your code and allows your dependencies to continue compiling when you update to a newer SDK. I know people on both sides won't be 100% satisfied with this, but does it seem like a reasonable compromise?

The next question is how to spell it. I'm leaning towards `unexpected case:`, which (a) is backwards-compatible, and (b) also handles "private cases", either the fake kind that you can do in C (as described in the proposal), or some real feature we might add to Swift some day. `unknown case:` isn't bad either.

I too would like to just do `unknown:` or `unexpected:` but that's technically a source-breaking change:

switch foo {
case bar:
  unknown:
  while baz() {
    while garply() {
      if quux() {
        break unknown
      }
    }
  }
}

Another downside of the `unexpected case:` spelling is that it doesn't work as part of a larger pattern. I don't have a good answer for that one, but perhaps it's acceptable for now.

I'll write up a revision of the proposal soon and make sure the core team gets my recommendation when they discuss the results of the review.

---

I'll respond to a few of the more intricate discussions tomorrow, including the syntax of putting a new declaration inside the enum rather than outside. Thank you again, everyone, and happy new year!

Jordan

+1 to warning instead of error
+1 to unknown/unexpected case
+1 to “@frozen” or any other reasonable spelling, they are all fine by me.

+1 to “@tangled” because abi is complicated

The one remaining problem to solve is making sure multi-module apps can leave out the unknown/unexpected case on enums from modules which are part of the app itself and thus cannot be updated independently of it. John McCall’s version-locking plan sounds promising, though we should explore the available options before finalizing a course.

Perhaps we need a concept of submodules, or supermodules, or some other way to demarcate the boundaries of a resilience domain.

Nevin

i would support a proper submodule system over some verson-locking system that only the most advanced users will probably know about. i think modules should be one level higher than what they’re currently being used for right now for lack of a better alternative (one application should never have to define more than one capital M Module). submodules shouldn’t be that hard to implement, though the submodule names should be part of ABI to avoid name mangling problems

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

davedelong · January 3, 2018, 4:38pm

[Proposal: https://github.com/apple/swift-evolution/blob/master/proposals/0192-non-exhaustive-enums.md\]

Whew! Thanks for your feedback, everyone. On the lighter side of feedback—naming things—it seems that most people seem to like '@frozen', and that does in fact have the connotations we want it to have. I like it too.

More seriously, this discussion has convinced me that it's worth including what the proposal discusses as a 'future' case. The key point that swayed me is that this can produce a warning when the switch is missing a case rather than an error, which both provides the necessary compiler feedback to update your code and allows your dependencies to continue compiling when you update to a newer SDK. I know people on both sides won't be 100% satisfied with this, but does it seem like a reasonable compromise?

The next question is how to spell it. I'm leaning towards `unexpected case:`, which (a) is backwards-compatible, and (b) also handles "private cases", either the fake kind that you can do in C (as described in the proposal), or some real feature we might add to Swift some day. `unknown case:` isn't bad either.

I too would like to just do `unknown:` or `unexpected:` but that's technically a source-breaking change:

switch foo {
case bar:
  unknown:
  while baz() {
    while garply() {
      if quux() {
        break unknown
      }
    }
  }
}

Another downside of the `unexpected case:` spelling is that it doesn't work as part of a larger pattern. I don't have a good answer for that one, but perhaps it's acceptable for now.

I'll write up a revision of the proposal soon and make sure the core team gets my recommendation when they discuss the results of the review.

---

I'll respond to a few of the more intricate discussions tomorrow, including the syntax of putting a new declaration inside the enum rather than outside. Thank you again, everyone, and happy new year!

Jordan

+1 to warning instead of error
+1 to unknown/unexpected case
+1 to “@frozen” or any other reasonable spelling, they are all fine by me.

+1 to “@tangled” because abi is complicated

+1 to “@moana” because these are all just Disney movies

···

On Jan 2, 2018, at 10:26 PM, Kelvin Ma via swift-evolution <swift-evolution@swift.org> wrote:
On Tue, Jan 2, 2018 at 11:45 PM, Nevin Brackett-Rozinsky via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
On Tue, Jan 2, 2018 at 9:07 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

The one remaining problem to solve is making sure multi-module apps can leave out the unknown/unexpected case on enums from modules which are part of the app itself and thus cannot be updated independently of it. John McCall’s version-locking plan sounds promising, though we should explore the available options before finalizing a course.

Perhaps we need a concept of submodules, or supermodules, or some other way to demarcate the boundaries of a resilience domain.

Nevin

i would support a proper submodule system over some verson-locking system that only the most advanced users will probably know about. i think modules should be one level higher than what they’re currently being used for right now for lack of a better alternative (one application should never have to define more than one capital M Module). submodules shouldn’t be that hard to implement, though the submodule names should be part of ABI to avoid name mangling problems

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

knatt · January 3, 2018, 5:58pm

[...]
2️⃣ If the enum is NOT decorated with @frozen, then I, as an app author, have to account for the possibility that the module may update from underneath my app, and I have to handle an unknown case. This is simple: the compiler should require me to add a “default:” case to my switch statement. This warning is produced IFF: the enum is coming from an external module, and the enum is not decorated with @frozen.

This does not help people who need to write a switch statement over an enum vended by a module that ships with the OS keep their code up to date as the module adds new cases. I find the example of `SKPaymentTransactionState` provided by Brent Royal-Gordon here: [swift-evolution] Enums and Source Compatibility to be compelling. There are rare but legitimate reasons to switch over all known cases of a non-@frozen enum that ship with the OS. These use cases deserve proper language support. I think Jordan’s solution strikes a good balance.

I disagree that more is needed. In the case of the transaction state, it should not be marked as @moana, and so the compiler would force you to add a “default” case to your switch statements. The switch statements would still be exhaustive with all known cases (if you choose to handle all known cases), but you’d still need a default case because there might be new transaction states in the future.

And then you don't get the compiler error/warning when you start compiling against the next OS update that changes the enum. That is an absolutely unacceptable loss of compile-time checks.

···

In those cases, your app could decide what to do, if that’s possible at all. Maybe there’s other transaction information you could introspect to determine if it succeeded or is still pending or whatever, and then your app could respond as you see fit.

Dave
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

bjhomer · January 3, 2018, 5:47pm

I agree that the SKPaymentTransactionState example is compelling. However, I don’t understand what actual code a developer would put in the “unexpected case:” clause. That enum is related to handling in-app purchases; it’s expected that the app would react in some way to a transaction changing state. What should the app do if the transaction enters an “unexpected” state? Pop up an error to the user that the user can do nothing about? It’s great to get a compiler warning about a case like when the developer builds with the new SDK, but I don’t understand what the already-shipped app is supposed to do in a case like this.

For the SKPaymentTransactionState case, at least, it seems like the library needs to be responsible to only send known values to the client. Otherwise, the app can have no idea what to do.

-BJ

···

On Jan 3, 2018, at 10:36 AM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

This does not help people who need to write a switch statement over an enum vended by a module that ships with the OS keep their code up to date as the module adds new cases. I find the example of `SKPaymentTransactionState` provided by Brent Royal-Gordon here: [swift-evolution] Enums and Source Compatibility to be compelling. There are rare but legitimate reasons to switch over all known cases of a non-@frozen enum that ship with the OS. These use cases deserve proper language support. I think Jordan’s solution strikes a good balance.

davedelong · January 3, 2018, 5:49pm

I disagree that more is needed. In the case of the transaction state, it should not be marked as @moana, and so the compiler would force you to add a “default” case to your switch statements. The switch statements would still be exhaustive with all known cases (if you choose to handle all known cases), but you’d still need a default case because there might be new transaction states in the future.

In those cases, your app could decide what to do, if that’s possible at all. Maybe there’s other transaction information you could introspect to determine if it succeeded or is still pending or whatever, and then your app could respond as you see fit.

Dave

···

On Jan 3, 2018, at 10:36 AM, Matthew Johnson <matthew@anandabits.com> wrote:

On Jan 3, 2018, at 11:07 AM, Dave DeLong via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

IMO this is still too large of a hammer for this problem.

This whole “unexpected case” thing is only a problem when you’re linking libraries that are external to/shipped independently of your app. Right now, the *only* case where this might exist is Swift on the server. We *might* run in to this in the future once the ABI stabilizes and we have the Swift libraries shipping as part of iOS/macOS/Linux. Other than this, unexpected enum cases won’t really be a problem developers have to deal with.

Because this will be such a relatively rare problem, I feel like a syntax change like what’s being proposed is a too-massive hammer for such a small nail.

What feels far more appropriate is:

Teaching the compiler/checker/whatever about the linking semantics of modules. For modules that are packaged inside the final built product, there is no need to deal with any unexpected cases, because we already have the exhaustiveness check appropriate for that scenario (regardless of whether the module is shipped as a binary or compiled from source). The app author decides when to update their dependencies, and updating those dependencies will produce new warnings/errors as the compiler notices new or deprecated cases. This is the current state of things and is completely orthogonal to the entire discussion.

John McCall sketched out a vision of what a solution to this might look like here: https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20171218/042333.html\.

and

Adding an attribute (@frozen, @tangled, @moana, @whatever) that can be used to decorate an enum declaration. This attribute would only need to be consulted on enums where the compiler can determine that the module will *not* be part of the final built product. (Ie, it’s an “external” module, in my nomenclature). This, then, is a module that can update independently of the final app, and therefore there are two possible cases:

1️⃣ If the enum is decorated with @frozen, then I, as an app author, have the assurance that the enum case will not change in future releases of the library, and I can safely switch on all known cases and not have to provide a default case.

2️⃣ If the enum is NOT decorated with @frozen, then I, as an app author, have to account for the possibility that the module may update from underneath my app, and I have to handle an unknown case. This is simple: the compiler should require me to add a “default:” case to my switch statement. This warning is produced IFF: the enum is coming from an external module, and the enum is not decorated with @frozen.

This does not help people who need to write a switch statement over an enum vended by a module that ships with the OS keep their code up to date as the module adds new cases. I find the example of `SKPaymentTransactionState` provided by Brent Royal-Gordon here: [swift-evolution] Enums and Source Compatibility to be compelling. There are rare but legitimate reasons to switch over all known cases of a non-@frozen enum that ship with the OS. These use cases deserve proper language support. I think Jordan’s solution strikes a good balance.

davedelong · January 3, 2018, 6:02pm

[...]
2️⃣ If the enum is NOT decorated with @frozen, then I, as an app author, have to account for the possibility that the module may update from underneath my app, and I have to handle an unknown case. This is simple: the compiler should require me to add a “default:” case to my switch statement. This warning is produced IFF: the enum is coming from an external module, and the enum is not decorated with @frozen.

This does not help people who need to write a switch statement over an enum vended by a module that ships with the OS keep their code up to date as the module adds new cases. I find the example of `SKPaymentTransactionState` provided by Brent Royal-Gordon here: [swift-evolution] Enums and Source Compatibility to be compelling. There are rare but legitimate reasons to switch over all known cases of a non-@frozen enum that ship with the OS. These use cases deserve proper language support. I think Jordan’s solution strikes a good balance.

I disagree that more is needed. In the case of the transaction state, it should not be marked as @moana, and so the compiler would force you to add a “default” case to your switch statements. The switch statements would still be exhaustive with all known cases (if you choose to handle all known cases), but you’d still need a default case because there might be new transaction states in the future.

And then you don't get the compiler error/warning when you start compiling against the next OS update that changes the enum. That is an absolutely unacceptable loss of compile-time checks.

Ah, that’s an excellent point! Thanks for pointing that out.

In that case, I revise my proposal:

a @frozen/@tangled/@moana attribute for enums that’s only consulted on externally-linked modules
a “future case:” statement that’s only required on externally-linked, non-@moana enums. That way, if the enum adds a new case in the future, you’ll still get a switch warning about handling all the cases. And then if you want, you could “fallthrough” from this to a “default” case, or vice-versa, or have separate implementations.

Dave

···

On Jan 3, 2018, at 10:58 AM, Kevin Nattinger <swift@nattinger.net> wrote:

In those cases, your app could decide what to do, if that’s possible at all. Maybe there’s other transaction information you could introspect to determine if it succeeded or is still pending or whatever, and then your app could respond as you see fit.

Dave
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

anandabits · January 3, 2018, 5:57pm

This does not help people who need to write a switch statement over an enum vended by a module that ships with the OS keep their code up to date as the module adds new cases. I find the example of `SKPaymentTransactionState` provided by Brent Royal-Gordon here: [swift-evolution] Enums and Source Compatibility to be compelling. There are rare but legitimate reasons to switch over all known cases of a non-@frozen enum that ship with the OS. These use cases deserve proper language support. I think Jordan’s solution strikes a good balance.

I agree that the SKPaymentTransactionState example is compelling. However, I don’t understand what actual code a developer would put in the “unexpected case:” clause. That enum is related to handling in-app purchases; it’s expected that the app would react in some way to a transaction changing state. What should the app do if the transaction enters an “unexpected” state? Pop up an error to the user that the user can do nothing about? It’s great to get a compiler warning about a case like when the developer builds with the new SDK, but I don’t understand what the already-shipped app is supposed to do in a case like this.

For the SKPaymentTransactionState case, at least, it seems like the library needs to be responsible to only send known values to the client. Otherwise, the app can have no idea what to do.

There is definitely not a good answer here. Making the library responsible to only send known values is not a solution. This type represents the state of an entity and it should do so as accurately as possible. Sometimes a new state cannot be mapped to one of the previous states. How would you have StoreKit handle the deferred state for old clients?

My position is that the situation is much worse for an app developer when they don’t even get the assistance of a compiler warning when updating to the new SDK. App developers are sometimes forced to deal with unfortunate cases like this. Any help the language can provide is appreciated.

···

On Jan 3, 2018, at 11:47 AM, BJ Homer <bjhomer@gmail.com> wrote:

On Jan 3, 2018, at 10:36 AM, Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

-BJ

anandabits · January 3, 2018, 6:06pm

IMO this is still too large of a hammer for this problem.

This whole “unexpected case” thing is only a problem when you’re linking libraries that are external to/shipped independently of your app. Right now, the *only* case where this might exist is Swift on the server. We *might* run in to this in the future once the ABI stabilizes and we have the Swift libraries shipping as part of iOS/macOS/Linux. Other than this, unexpected enum cases won’t really be a problem developers have to deal with.

Because this will be such a relatively rare problem, I feel like a syntax change like what’s being proposed is a too-massive hammer for such a small nail.

What feels far more appropriate is:

Teaching the compiler/checker/whatever about the linking semantics of modules. For modules that are packaged inside the final built product, there is no need to deal with any unexpected cases, because we already have the exhaustiveness check appropriate for that scenario (regardless of whether the module is shipped as a binary or compiled from source). The app author decides when to update their dependencies, and updating those dependencies will produce new warnings/errors as the compiler notices new or deprecated cases. This is the current state of things and is completely orthogonal to the entire discussion.

John McCall sketched out a vision of what a solution to this might look like here: https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20171218/042333.html\.

and

Adding an attribute (@frozen, @tangled, @moana, @whatever) that can be used to decorate an enum declaration. This attribute would only need to be consulted on enums where the compiler can determine that the module will *not* be part of the final built product. (Ie, it’s an “external” module, in my nomenclature). This, then, is a module that can update independently of the final app, and therefore there are two possible cases:

1️⃣ If the enum is decorated with @frozen, then I, as an app author, have the assurance that the enum case will not change in future releases of the library, and I can safely switch on all known cases and not have to provide a default case.

2️⃣ If the enum is NOT decorated with @frozen, then I, as an app author, have to account for the possibility that the module may update from underneath my app, and I have to handle an unknown case. This is simple: the compiler should require me to add a “default:” case to my switch statement. This warning is produced IFF: the enum is coming from an external module, and the enum is not decorated with @frozen.

This does not help people who need to write a switch statement over an enum vended by a module that ships with the OS keep their code up to date as the module adds new cases. I find the example of `SKPaymentTransactionState` provided by Brent Royal-Gordon here: [swift-evolution] Enums and Source Compatibility to be compelling. There are rare but legitimate reasons to switch over all known cases of a non-@frozen enum that ship with the OS. These use cases deserve proper language support. I think Jordan’s solution strikes a good balance.

I disagree that more is needed. In the case of the transaction state, it should not be marked as @moana, and so the compiler would force you to add a “default” case to your switch statements. The switch statements would still be exhaustive with all known cases (if you choose to handle all known cases), but you’d still need a default case because there might be new transaction states in the future.

In those cases, your app could decide what to do, if that’s possible at all. Maybe there’s other transaction information you could introspect to determine if it succeeded or is still pending or whatever, and then your app could respond as you see fit.

SKPaymentTransactionState is an excellent motivating example for a better solution because it is crucial to update code to handle any new states accurately and promptly. This is much easier to do with compiler assistance than without it.

It is also an excellent motivating example because the API change is driven by a business change and for which all states should be reflected accurately in the UX. Apps need to switch over this enum as exhaustively as possible while the StoreKit team also needs to reserve the right to add cases as business requirements change.

It is an evolution problem without a great answer. I think Jordan has landed on a pretty good compromise. If you don’t want a warning when modules change you’re still free to use `default`. Nobody is talking about forcing you to use `unknown case`. If you really dislike it you could even ban it with a linter.

···

On Jan 3, 2018, at 11:49 AM, Dave DeLong <swift@davedelong.com> wrote:

On Jan 3, 2018, at 10:36 AM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Jan 3, 2018, at 11:07 AM, Dave DeLong via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Dave

anandabits · January 3, 2018, 6:09pm

[...]
2️⃣ If the enum is NOT decorated with @frozen, then I, as an app author, have to account for the possibility that the module may update from underneath my app, and I have to handle an unknown case. This is simple: the compiler should require me to add a “default:” case to my switch statement. This warning is produced IFF: the enum is coming from an external module, and the enum is not decorated with @frozen.

This does not help people who need to write a switch statement over an enum vended by a module that ships with the OS keep their code up to date as the module adds new cases. I find the example of `SKPaymentTransactionState` provided by Brent Royal-Gordon here: [swift-evolution] Enums and Source Compatibility to be compelling. There are rare but legitimate reasons to switch over all known cases of a non-@frozen enum that ship with the OS. These use cases deserve proper language support. I think Jordan’s solution strikes a good balance.

I disagree that more is needed. In the case of the transaction state, it should not be marked as @moana, and so the compiler would force you to add a “default” case to your switch statements. The switch statements would still be exhaustive with all known cases (if you choose to handle all known cases), but you’d still need a default case because there might be new transaction states in the future.

And then you don't get the compiler error/warning when you start compiling against the next OS update that changes the enum. That is an absolutely unacceptable loss of compile-time checks.

Ah, that’s an excellent point! Thanks for pointing that out.

In that case, I revise my proposal:

a @frozen/@tangled/@moana attribute for enums that’s only consulted on externally-linked modules
a “future case:” statement that’s only required on externally-linked, non-@moana enums. That way, if the enum adds a new case in the future, you’ll still get a switch warning about handling all the cases. And then if you want, you could “fallthrough” from this to a “default” case, or vice-versa, or have separate implementations.

It sounds to me like the main thing you’re unhappy about is having to deal with unknown cases when you have a source dependency that you always build and ship with an app. I don’t think anyone is happy with that situation. Did you take a look at the sketch of a solution provided by John McCall? [swift-evolution] [Review] SE 0192 - Non-Exhaustive Enums

···

On Jan 3, 2018, at 12:02 PM, Dave DeLong <swift@davedelong.com> wrote:

On Jan 3, 2018, at 10:58 AM, Kevin Nattinger <swift@nattinger.net <mailto:swift@nattinger.net>> wrote:

Dave

In those cases, your app could decide what to do, if that’s possible at all. Maybe there’s other transaction information you could introspect to determine if it succeeded or is still pending or whatever, and then your app could respond as you see fit.

Dave
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

Karl · January 3, 2018, 7:20pm

So here’s something that I’m confused about - you said before that when the library updates, the application behaviour should remain the same. Then why is it up to the App developers to try and develop forwards-compatible logic instead of the library developers to ensure backwards-compatibility?

SKPaymentTransactionState is a great example - it’s exactly what this proposal is about. We are all looking at the superficial side of keeping App code compiling, when really we should be thinking about the semantics that those enums represent. Enums usually mean state, and it’s often difficult to design good state-machines which can deal with completely unpredictable values popping up. Most of the the time, your default handler is just garbage which you expect and hope never gets called IRL.

I’m not convinced that it is possible to write a good default handler for SKPaymentTransactionState, or really most enums. If there were other properties you could inspect to perform your App logic, you should clearly prefer those since they are more robust than an enum which may give you unpredictable values in the future. How would users even know about this recommended practice of ignoring the enum and looking for ad-hoc state properties? Again, getting this stuff right requires careful design and documentation from the library developers, regardless if enums are exhaustive or not.

I think the StoreKit library authors got it wrong there; they added a new enum case which required special handling; that’s an API-breaking change. They should have defined backwards-compatibility behaviour (if that was possible), or disabled the feature for Apps which didn’t support it. Just throwing out values which you know the App doesn’t understand puts you in dangerous territory, even if it doesn’t necessarily crash.

- Karl

···

On 3. Jan 2018, at 18:49, Dave DeLong via swift-evolution <swift-evolution@swift.org> wrote:

On Jan 3, 2018, at 10:36 AM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Jan 3, 2018, at 11:07 AM, Dave DeLong via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

IMO this is still too large of a hammer for this problem.

This whole “unexpected case” thing is only a problem when you’re linking libraries that are external to/shipped independently of your app. Right now, the *only* case where this might exist is Swift on the server. We *might* run in to this in the future once the ABI stabilizes and we have the Swift libraries shipping as part of iOS/macOS/Linux. Other than this, unexpected enum cases won’t really be a problem developers have to deal with.

Because this will be such a relatively rare problem, I feel like a syntax change like what’s being proposed is a too-massive hammer for such a small nail.

What feels far more appropriate is:

Teaching the compiler/checker/whatever about the linking semantics of modules. For modules that are packaged inside the final built product, there is no need to deal with any unexpected cases, because we already have the exhaustiveness check appropriate for that scenario (regardless of whether the module is shipped as a binary or compiled from source). The app author decides when to update their dependencies, and updating those dependencies will produce new warnings/errors as the compiler notices new or deprecated cases. This is the current state of things and is completely orthogonal to the entire discussion.

John McCall sketched out a vision of what a solution to this might look like here: https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20171218/042333.html\.

and

Adding an attribute (@frozen, @tangled, @moana, @whatever) that can be used to decorate an enum declaration. This attribute would only need to be consulted on enums where the compiler can determine that the module will *not* be part of the final built product. (Ie, it’s an “external” module, in my nomenclature). This, then, is a module that can update independently of the final app, and therefore there are two possible cases:

1️⃣ If the enum is decorated with @frozen, then I, as an app author, have the assurance that the enum case will not change in future releases of the library, and I can safely switch on all known cases and not have to provide a default case.

2️⃣ If the enum is NOT decorated with @frozen, then I, as an app author, have to account for the possibility that the module may update from underneath my app, and I have to handle an unknown case. This is simple: the compiler should require me to add a “default:” case to my switch statement. This warning is produced IFF: the enum is coming from an external module, and the enum is not decorated with @frozen.

This does not help people who need to write a switch statement over an enum vended by a module that ships with the OS keep their code up to date as the module adds new cases. I find the example of `SKPaymentTransactionState` provided by Brent Royal-Gordon here: [swift-evolution] Enums and Source Compatibility to be compelling. There are rare but legitimate reasons to switch over all known cases of a non-@frozen enum that ship with the OS. These use cases deserve proper language support. I think Jordan’s solution strikes a good balance.

I disagree that more is needed. In the case of the transaction state, it should not be marked as @moana, and so the compiler would force you to add a “default” case to your switch statements. The switch statements would still be exhaustive with all known cases (if you choose to handle all known cases), but you’d still need a default case because there might be new transaction states in the future.

In those cases, your app could decide what to do, if that’s possible at all. Maybe there’s other transaction information you could introspect to determine if it succeeded or is still pending or whatever, and then your app could respond as you see fit.

Dave
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

masters3d · January 4, 2018, 5:57am

This is a very nice revision. One bikeshedding thought:

Since "unknown case" is presented as a special kind of "default", can't be mixed with "default", and can't be used in case patterns, why not "default unknown" (or "unknown default") instead of "unknown case"?

`case _ :` is already a special case of default.
I’d rather have `case unknown :`
`unknown case :` is weird because of the order of `case`.

Another alternative is `case \unknown :`
`\unknown` would also allow pattern matching.

···

On Jan 3, 2018, at 6:52 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:

On Wed, Jan 3, 2018 at 8:05 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org> wrote:

On Jan 2, 2018, at 18:07, Jordan Rose <jordan_rose@apple.com> wrote:

[Proposal: https://github.com/apple/swift-evolution/blob/master/proposals/0192-non-exhaustive-enums.md\]

Whew! Thanks for your feedback, everyone. On the lighter side of feedback—naming things—it seems that most people seem to like '@frozen', and that does in fact have the connotations we want it to have. I like it too.

More seriously, this discussion has convinced me that it's worth including what the proposal discusses as a 'future' case. The key point that swayed me is that this can produce a warning when the switch is missing a case rather than an error, which both provides the necessary compiler feedback to update your code and allows your dependencies to continue compiling when you update to a newer SDK. I know people on both sides won't be 100% satisfied with this, but does it seem like a reasonable compromise?

The next question is how to spell it. I'm leaning towards `unexpected case:`, which (a) is backwards-compatible, and (b) also handles "private cases", either the fake kind that you can do in C (as described in the proposal), or some real feature we might add to Swift some day. `unknown case:` isn't bad either.

I too would like to just do `unknown:` or `unexpected:` but that's technically a source-breaking change:

switch foo {
case bar:
  unknown:
  while baz() {
    while garply() {
      if quux() {
        break unknown
      }
    }
  }
}

Another downside of the `unexpected case:` spelling is that it doesn't work as part of a larger pattern. I don't have a good answer for that one, but perhaps it's acceptable for now.

I'll write up a revision of the proposal soon and make sure the core team gets my recommendation when they discuss the results of the review.

---

I'll respond to a few of the more intricate discussions tomorrow, including the syntax of putting a new declaration inside the enum rather than outside. Thank you again, everyone, and happy new year!

I ended up doing these in the opposite order, writing up the new proposal first and not yet responding to the discussion that's further out. You can read my revisions at https://github.com/apple/swift-evolution/pull/777\.

In particular, I want to at least address:
- Dave D and Drew C's points about versioned libraries / linking semantics of modules.
- Jason M's point about migration
and I'll do one more pass over the thread to see if there's anything else I didn't address directly. (That doesn't mean everyone who disagrees, just messages where I think there's more I can do to explain why the proposal is the way it is.)

Jordan

P.S. Enjoying the Disney references. Thanks, Nevin and Dave. :-)

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Brent_Royal-Gordon · January 4, 2018, 1:23pm

Naming is serious business, Dave. Let it go.

*ducks*

···

On Jan 3, 2018, at 10:02 AM, Dave DeLong via swift-evolution <swift-evolution@swift.org> wrote:

a @frozen/@tangled/@moana attribute for enums that’s only consulted on externally-linked modules

--
Brent Royal-Gordon
Architechies

gwendal.roue · January 5, 2018, 4:19am

Thanks for this extended rationale.

I would pin this post if I could. Or extend the "motivation" section of the proposal with it.

Gwendal

···

Le 5 janv. 2018 à 01:38, Jordan Rose via swift-evolution <swift-evolution@swift.org> a écrit :

Hi, Dave. You're right, all these points are worth addressing. I'm going to go in sections.

This whole “unexpected case” thing is only a problem when you’re linking libraries that are external to/shipped independently of your app. Right now, the *only* case where this might exist is Swift on the server. We *might* run in to this in the future once the ABI stabilizes and we have the Swift libraries shipping as part of iOS/macOS/Linux. Other than this, unexpected enum cases won’t really be a problem developers have to deal with.

I wish this were the case, but it is not. Regardless of what we do for Swift enums, we are in dire need of a fix for C enums. Today, if a C enum doesn't have one of the expected values, the behavior is undefined in the C sense (as in, type-unsafe, memory-unsafe, may invoke functions that shouldn't be invoked, may not invoke functions that should be invoked, etc).

Obviously that's an unacceptable state of affairs; even without this proposal we would fix it so that the program will deterministically trap instead. This isn't perfect because it results in a (tiny) performance and code size hit compared to C, but it's better than leaving such a massive hole in Swift's safety story.

The trouble is that many enums—maybe even most enums—in the Apple SDK really are expected to grow new cases, and the Apple API authors rely on this. Many of those—probably most of them—are the ones that Brent Royal-Gordon described as "opaque inputs", like UIViewAnimationTransition, which you're unlikely to switch over but which the compiler should handle correctly if you do. Then there are the murkier ones like SKPaymentTransactionState.

I'm going to come dangerously close to criticizing Apple and say I have a lot of sympathy for third-party developers in the SKPaymentTransactionState case. As Karl Wagner said, there wasn't really any way an existing app could handle that case well, even if they had written an 'unknown case' handler. So what could the StoreKit folks have done instead? They can't tell themselves whether your app supports the new case, other than the heavy-handed "check what SDK they compiled against" that ignores the possibility of embedded binary frameworks. So maybe they should have added a property "supportsDeferredState" or something that would have to be set before the new state was returned.

(I'll pause to say I don't know what consideration went into this API and I'm going to avoid looking it up to avoid perjury. This is all hypothetical, for the next API that needs to add a case.)

Let's say we go with that, a property that controls whether the new case is ever passed to third-party code. Now the new case exists, and new code needs to switch over it. At the same time, old code needs to continue working. The new enum case exists, and so even if it shouldn't escape into old code that doesn't know how to handle it, the behavior needs to be defined if it does. Furthermore, the old code needs to continue working without source changes, because updating to a new SDK must not break existing code. (It can introduce new warnings, but even that is something that should be considered carefully.)

So: this proposal is designed to handle the use cases both for Swift library authors to come and for C APIs today, and in particular Apple's Objective-C SDKs and how they've evolved historically.

There's another really interesting point in your message, which Karl, Drew Crawford, and others also touched on.

Teaching the compiler/checker/whatever about the linking semantics of modules. For modules that are packaged inside the final built product, there is no need to deal with any unexpected cases, because we already have the exhaustiveness check appropriate for that scenario (regardless of whether the module is shipped as a binary or compiled from source). The app author decides when to update their dependencies, and updating those dependencies will produce new warnings/errors as the compiler notices new or deprecated cases. This is the current state of things and is completely orthogonal to the entire discussion.

This keeps sneaking into discussions and I hope to have it formalized in a proposal soon. On the library side, we do want to make a distinction between "needs binary compatibility" and "does not need binary compatibility". Why? Because we can get much better performance if we know a library is never going to change. A class will not acquire new dynamic-dispatch members; a stored property will not turn into a computed property; a struct will not gain new stored properties. None of those things affect how client code is written, but they do affect what happens at run-time.

Okay, so should we use this as an indicator of whether an enum can grow new cases? (I'm going to ignore C libraries in this section, both because they don't have this distinction and because they can always lie anyway.)

- If a library really is shipped separately from the app, enums can grow new cases, except for the ones that can't. So we need some kind of annotation here. This is your "B" in the original email, so we're all agreed here.

- If a library is shipped with the app, there's no chance of the enum growing a new case at run time. Does that mean we don't need a default case? (Or "unknown case" now.)

The answer here is most easily understood in terms of semantic versioning. If adding a new enum case is a source-breaking change, then it's a source-breaking change, requiring a major version update. The app author decides when to update their dependencies, and might hold off on getting a newer version of a library because it's not compatible with what they have.

If adding a new enum case is not a source-breaking change, then it can be done in a minor version release of a library. Like deprecations, this can produce new warnings, but not new errors, and it should not (if done carefully) break existing code. This isn't a critical feature for a language to have, but I would argue (and have argued) that it's a useful one for library developers. Major releases still exist; this just makes one particular kind of change valid for minor releases as well.

(It also feels very subtle to me that 'switch' behaves differently based on where the enum came from. I know this whole proposal adds complexity to the language, and I'd like to keep it as consistent as possible.)

Okay, so what if we did this based on the 'import' rather than on how the module was compiled—Karl's `@static import`? That feels a little better to me because you can see it in your code. (Let's ignore re-exported modules for now.) But now we have two types of 'import', only one of which can be used with system libraries. That also makes me uncomfortable. (And to be fair, it's also something that can be added after the fact without disturbing the rest of the language.)

Finally, it's very important that whatever you do in your code doesn't necessarily apply to your dependencies. We've seen in practice that people are not willing to edit their dependencies, even to handle simple SDK changes or language syntax changes (of which there are hopefully no more). That's why I'm pushing the source compatibility aspect so hard, even for libraries that won't be shipped separately from an app.

Overall, I think we're really trying to keep from breaking Swift into different dialects, and making this feature dependent on whether or not the library is embedded in the app would work at cross-purposes to that. Everyone would still be forced to learn about the feature if they used C enums anyway, so we're not even helping out average developers. Instead, it's better that we have one, good model for dealing with other people's enums, which in practice can and do grow new cases regardless of how they are linked.

Jordan

On Jan 3, 2018, at 09:07, Dave DeLong <swift@davedelong.com> wrote:

IMO this is still too large of a hammer for this problem.

This whole “unexpected case” thing is only a problem when you’re linking libraries that are external to/shipped independently of your app. Right now, the *only* case where this might exist is Swift on the server. We *might* run in to this in the future once the ABI stabilizes and we have the Swift libraries shipping as part of iOS/macOS/Linux. Other than this, unexpected enum cases won’t really be a problem developers have to deal with.

Because this will be such a relatively rare problem, I feel like a syntax change like what’s being proposed is a too-massive hammer for such a small nail.

What feels far more appropriate is:

Teaching the compiler/checker/whatever about the linking semantics of modules. For modules that are packaged inside the final built product, there is no need to deal with any unexpected cases, because we already have the exhaustiveness check appropriate for that scenario (regardless of whether the module is shipped as a binary or compiled from source). The app author decides when to update their dependencies, and updating those dependencies will produce new warnings/errors as the compiler notices new or deprecated cases. This is the current state of things and is completely orthogonal to the entire discussion.

and

Adding an attribute (@frozen, @tangled, @moana, @whatever) that can be used to decorate an enum declaration. This attribute would only need to be consulted on enums where the compiler can determine that the module will *not* be part of the final built product. (Ie, it’s an “external” module, in my nomenclature). This, then, is a module that can update independently of the final app, and therefore there are two possible cases:

  1️⃣ If the enum is decorated with @frozen, then I, as an app author, have the assurance that the enum case will not change in future releases of the library, and I can safely switch on all known cases and not have to provide a default case.

  2️⃣ If the enum is NOT decorated with @frozen, then I, as an app author, have to account for the possibility that the module may update from underneath my app, and I have to handle an unknown case. This is simple: the compiler should require me to add a “default:” case to my switch statement. This warning is produced IFF: the enum is coming from an external module, and the enum is not decorated with @frozen.

==========

With this proposal, we only have one thing to consider: the spelling of @frozen/@moana/@whatever that we decorate enums in external modules with. Other than that, the existing behavior we currently have is completely capable of covering the possibilities: we just keep using a “default:” case whenever the compiler can’t guarantee that we can be exhaustive in our switching.

Where the real work would be is teaching the compiler about internally-vs-externally linked modules.

Dave

On Jan 2, 2018, at 7:07 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org> wrote:

[Proposal: https://github.com/apple/swift-evolution/blob/master/proposals/0192-non-exhaustive-enums.md\]

Whew! Thanks for your feedback, everyone. On the lighter side of feedback—naming things—it seems that most people seem to like '@frozen', and that does in fact have the connotations we want it to have. I like it too.

More seriously, this discussion has convinced me that it's worth including what the proposal discusses as a 'future' case. The key point that swayed me is that this can produce a warning when the switch is missing a case rather than an error, which both provides the necessary compiler feedback to update your code and allows your dependencies to continue compiling when you update to a newer SDK. I know people on both sides won't be 100% satisfied with this, but does it seem like a reasonable compromise?

The next question is how to spell it. I'm leaning towards `unexpected case:`, which (a) is backwards-compatible, and (b) also handles "private cases", either the fake kind that you can do in C (as described in the proposal), or some real feature we might add to Swift some day. `unknown case:` isn't bad either.

I too would like to just do `unknown:` or `unexpected:` but that's technically a source-breaking change:

switch foo {
case bar:
  unknown:
  while baz() {
    while garply() {
      if quux() {
        break unknown
      }
    }
  }
}

Another downside of the `unexpected case:` spelling is that it doesn't work as part of a larger pattern. I don't have a good answer for that one, but perhaps it's acceptable for now.

I'll write up a revision of the proposal soon and make sure the core team gets my recommendation when they discuss the results of the review.

---

I'll respond to a few of the more intricate discussions tomorrow, including the syntax of putting a new declaration inside the enum rather than outside. Thank you again, everyone, and happy new year!

Jordan

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

gwendal.roue · January 5, 2018, 4:42am

May I ask two questions?

Would StoreKit be written in Swift 5+, do you think that SKPaymentTransactionState should have been introduced as a @closed enum?

If so, what would have then been the consequences of adding the new .deferred state (assuming this would even be possible)?

Gwendal

···

Le 5 janv. 2018 à 01:38, Jordan Rose via swift-evolution <swift-evolution@swift.org> a écrit :

Hi, Dave. You're right, all these points are worth addressing. I'm going to go in sections.

This whole “unexpected case” thing is only a problem when you’re linking libraries that are external to/shipped independently of your app. Right now, the *only* case where this might exist is Swift on the server. We *might* run in to this in the future once the ABI stabilizes and we have the Swift libraries shipping as part of iOS/macOS/Linux. Other than this, unexpected enum cases won’t really be a problem developers have to deal with.

I wish this were the case, but it is not. Regardless of what we do for Swift enums, we are in dire need of a fix for C enums. Today, if a C enum doesn't have one of the expected values, the behavior is undefined in the C sense (as in, type-unsafe, memory-unsafe, may invoke functions that shouldn't be invoked, may not invoke functions that should be invoked, etc).

Obviously that's an unacceptable state of affairs; even without this proposal we would fix it so that the program will deterministically trap instead. This isn't perfect because it results in a (tiny) performance and code size hit compared to C, but it's better than leaving such a massive hole in Swift's safety story.

The trouble is that many enums—maybe even most enums—in the Apple SDK really are expected to grow new cases, and the Apple API authors rely on this. Many of those—probably most of them—are the ones that Brent Royal-Gordon described as "opaque inputs", like UIViewAnimationTransition, which you're unlikely to switch over but which the compiler should handle correctly if you do. Then there are the murkier ones like SKPaymentTransactionState.

I'm going to come dangerously close to criticizing Apple and say I have a lot of sympathy for third-party developers in the SKPaymentTransactionState case. As Karl Wagner said, there wasn't really any way an existing app could handle that case well, even if they had written an 'unknown case' handler. So what could the StoreKit folks have done instead? They can't tell themselves whether your app supports the new case, other than the heavy-handed "check what SDK they compiled against" that ignores the possibility of embedded binary frameworks. So maybe they should have added a property "supportsDeferredState" or something that would have to be set before the new state was returned.

(I'll pause to say I don't know what consideration went into this API and I'm going to avoid looking it up to avoid perjury. This is all hypothetical, for the next API that needs to add a case.)

Let's say we go with that, a property that controls whether the new case is ever passed to third-party code. Now the new case exists, and new code needs to switch over it. At the same time, old code needs to continue working. The new enum case exists, and so even if it shouldn't escape into old code that doesn't know how to handle it, the behavior needs to be defined if it does. Furthermore, the old code needs to continue working without source changes, because updating to a new SDK must not break existing code. (It can introduce new warnings, but even that is something that should be considered carefully.)

So: this proposal is designed to handle the use cases both for Swift library authors to come and for C APIs today, and in particular Apple's Objective-C SDKs and how they've evolved historically.

There's another really interesting point in your message, which Karl, Drew Crawford, and others also touched on.

Teaching the compiler/checker/whatever about the linking semantics of modules. For modules that are packaged inside the final built product, there is no need to deal with any unexpected cases, because we already have the exhaustiveness check appropriate for that scenario (regardless of whether the module is shipped as a binary or compiled from source). The app author decides when to update their dependencies, and updating those dependencies will produce new warnings/errors as the compiler notices new or deprecated cases. This is the current state of things and is completely orthogonal to the entire discussion.

This keeps sneaking into discussions and I hope to have it formalized in a proposal soon. On the library side, we do want to make a distinction between "needs binary compatibility" and "does not need binary compatibility". Why? Because we can get much better performance if we know a library is never going to change. A class will not acquire new dynamic-dispatch members; a stored property will not turn into a computed property; a struct will not gain new stored properties. None of those things affect how client code is written, but they do affect what happens at run-time.

Okay, so should we use this as an indicator of whether an enum can grow new cases? (I'm going to ignore C libraries in this section, both because they don't have this distinction and because they can always lie anyway.)

- If a library really is shipped separately from the app, enums can grow new cases, except for the ones that can't. So we need some kind of annotation here. This is your "B" in the original email, so we're all agreed here.

- If a library is shipped with the app, there's no chance of the enum growing a new case at run time. Does that mean we don't need a default case? (Or "unknown case" now.)

The answer here is most easily understood in terms of semantic versioning. If adding a new enum case is a source-breaking change, then it's a source-breaking change, requiring a major version update. The app author decides when to update their dependencies, and might hold off on getting a newer version of a library because it's not compatible with what they have.

If adding a new enum case is not a source-breaking change, then it can be done in a minor version release of a library. Like deprecations, this can produce new warnings, but not new errors, and it should not (if done carefully) break existing code. This isn't a critical feature for a language to have, but I would argue (and have argued) that it's a useful one for library developers. Major releases still exist; this just makes one particular kind of change valid for minor releases as well.

(It also feels very subtle to me that 'switch' behaves differently based on where the enum came from. I know this whole proposal adds complexity to the language, and I'd like to keep it as consistent as possible.)

Okay, so what if we did this based on the 'import' rather than on how the module was compiled—Karl's `@static import`? That feels a little better to me because you can see it in your code. (Let's ignore re-exported modules for now.) But now we have two types of 'import', only one of which can be used with system libraries. That also makes me uncomfortable. (And to be fair, it's also something that can be added after the fact without disturbing the rest of the language.)

Finally, it's very important that whatever you do in your code doesn't necessarily apply to your dependencies. We've seen in practice that people are not willing to edit their dependencies, even to handle simple SDK changes or language syntax changes (of which there are hopefully no more). That's why I'm pushing the source compatibility aspect so hard, even for libraries that won't be shipped separately from an app.

Overall, I think we're really trying to keep from breaking Swift into different dialects, and making this feature dependent on whether or not the library is embedded in the app would work at cross-purposes to that. Everyone would still be forced to learn about the feature if they used C enums anyway, so we're not even helping out average developers. Instead, it's better that we have one, good model for dealing with other people's enums, which in practice can and do grow new cases regardless of how they are linked.

Jordan

On Jan 3, 2018, at 09:07, Dave DeLong <swift@davedelong.com> wrote:

IMO this is still too large of a hammer for this problem.

This whole “unexpected case” thing is only a problem when you’re linking libraries that are external to/shipped independently of your app. Right now, the *only* case where this might exist is Swift on the server. We *might* run in to this in the future once the ABI stabilizes and we have the Swift libraries shipping as part of iOS/macOS/Linux. Other than this, unexpected enum cases won’t really be a problem developers have to deal with.

Because this will be such a relatively rare problem, I feel like a syntax change like what’s being proposed is a too-massive hammer for such a small nail.

What feels far more appropriate is:

Teaching the compiler/checker/whatever about the linking semantics of modules. For modules that are packaged inside the final built product, there is no need to deal with any unexpected cases, because we already have the exhaustiveness check appropriate for that scenario (regardless of whether the module is shipped as a binary or compiled from source). The app author decides when to update their dependencies, and updating those dependencies will produce new warnings/errors as the compiler notices new or deprecated cases. This is the current state of things and is completely orthogonal to the entire discussion.

and

Adding an attribute (@frozen, @tangled, @moana, @whatever) that can be used to decorate an enum declaration. This attribute would only need to be consulted on enums where the compiler can determine that the module will *not* be part of the final built product. (Ie, it’s an “external” module, in my nomenclature). This, then, is a module that can update independently of the final app, and therefore there are two possible cases:

  1️⃣ If the enum is decorated with @frozen, then I, as an app author, have the assurance that the enum case will not change in future releases of the library, and I can safely switch on all known cases and not have to provide a default case.

  2️⃣ If the enum is NOT decorated with @frozen, then I, as an app author, have to account for the possibility that the module may update from underneath my app, and I have to handle an unknown case. This is simple: the compiler should require me to add a “default:” case to my switch statement. This warning is produced IFF: the enum is coming from an external module, and the enum is not decorated with @frozen.

==========

With this proposal, we only have one thing to consider: the spelling of @frozen/@moana/@whatever that we decorate enums in external modules with. Other than that, the existing behavior we currently have is completely capable of covering the possibilities: we just keep using a “default:” case whenever the compiler can’t guarantee that we can be exhaustive in our switching.

Where the real work would be is teaching the compiler about internally-vs-externally linked modules.

Dave

On Jan 2, 2018, at 7:07 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org> wrote:

[Proposal: https://github.com/apple/swift-evolution/blob/master/proposals/0192-non-exhaustive-enums.md\]

Whew! Thanks for your feedback, everyone. On the lighter side of feedback—naming things—it seems that most people seem to like '@frozen', and that does in fact have the connotations we want it to have. I like it too.

More seriously, this discussion has convinced me that it's worth including what the proposal discusses as a 'future' case. The key point that swayed me is that this can produce a warning when the switch is missing a case rather than an error, which both provides the necessary compiler feedback to update your code and allows your dependencies to continue compiling when you update to a newer SDK. I know people on both sides won't be 100% satisfied with this, but does it seem like a reasonable compromise?

The next question is how to spell it. I'm leaning towards `unexpected case:`, which (a) is backwards-compatible, and (b) also handles "private cases", either the fake kind that you can do in C (as described in the proposal), or some real feature we might add to Swift some day. `unknown case:` isn't bad either.

I too would like to just do `unknown:` or `unexpected:` but that's technically a source-breaking change:

switch foo {
case bar:
  unknown:
  while baz() {
    while garply() {
      if quux() {
        break unknown
      }
    }
  }
}

Another downside of the `unexpected case:` spelling is that it doesn't work as part of a larger pattern. I don't have a good answer for that one, but perhaps it's acceptable for now.

I'll write up a revision of the proposal soon and make sure the core team gets my recommendation when they discuss the results of the review.

---

I'll respond to a few of the more intricate discussions tomorrow, including the syntax of putting a new declaration inside the enum rather than outside. Thank you again, everyone, and happy new year!

Jordan

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

davedelong · January 5, 2018, 2:52pm

Hi Jordan,

Thanks for your thoughtful reply. Comments inline...

Hi, Dave. You're right, all these points are worth addressing. I'm going to go in sections.

This whole “unexpected case” thing is only a problem when you’re linking libraries that are external to/shipped independently of your app. Right now, the *only* case where this might exist is Swift on the server. We *might* run in to this in the future once the ABI stabilizes and we have the Swift libraries shipping as part of iOS/macOS/Linux. Other than this, unexpected enum cases won’t really be a problem developers have to deal with.

I wish this were the case, but it is not. Regardless of what we do for Swift enums, we are in dire need of a fix for C enums. Today, if a C enum doesn't have one of the expected values, the behavior is undefined in the C sense (as in, type-unsafe, memory-unsafe, may invoke functions that shouldn't be invoked, may not invoke functions that should be invoked, etc).

That’s a fair point, and one I had forgotten about. Thanks for reminding me.

Obviously that's an unacceptable state of affairs; even without this proposal we would fix it so that the program will deterministically trap instead. This isn't perfect because it results in a (tiny) performance and code size hit compared to C, but it's better than leaving such a massive hole in Swift's safety story.

The trouble is that many enums—maybe even most enums—in the Apple SDK really are expected to grow new cases, and the Apple API authors rely on this. Many of those—probably most of them—are the ones that Brent Royal-Gordon described as "opaque inputs", like UIViewAnimationTransition, which you're unlikely to switch over but which the compiler should handle correctly if you do. Then there are the murkier ones like SKPaymentTransactionState.

I'm going to come dangerously close to criticizing Apple and say I have a lot of sympathy for third-party developers in the SKPaymentTransactionState case.

This isn’t criticism.

<former evangelist hat>
You’re acknowledging the complexities that exist when writing software, from the point-of-view of someone who has no insight in to the intricacies and challenges faced by the StoreKit team. Our takeaway from this is that no one writes perfect code.
</former evangelist hat>

As Karl Wagner said, there wasn't really any way an existing app could handle that case well, even if they had written an 'unknown case' handler. So what could the StoreKit folks have done instead?

Well, the converse is also true: what reasonable logic exists that an app developer could do to handle a new transaction state for which they’re unprepared? The only sensical thing would be to abort the transaction, apologize to the user, and quickly release an update to your app.

They can't tell themselves whether your app supports the new case, other than the heavy-handed "check what SDK they compiled against" that ignores the possibility of embedded binary frameworks.

To be fair, a “linked on or after” check isn’t that heavy-handed. It’s a single if-statement. Yes, they become unwieldy if you’ve got them *everywhere* in your library code, which is why I brought up the point earlier that I’d really love to see more in the way of facilitating this sort of link-dependent behavior that apps are expecting.

The software-level solution is to make everything require a configuration object (as you allude to below), but a lower-level solution would (hopefully) be even nicer.

So maybe they should have added a property "supportsDeferredState" or something that would have to be set before the new state was returned.

(I'll pause to say I don't know what consideration went into this API and I'm going to avoid looking it up to avoid perjury. This is all hypothetical, for the next API that needs to add a case.)

Let's say we go with that, a property that controls whether the new case is ever passed to third-party code. Now the new case exists, and new code needs to switch over it. At the same time, old code needs to continue working. The new enum case exists, and so even if it shouldn't escape into old code that doesn't know how to handle it, the behavior needs to be defined if it does. Furthermore, the old code needs to continue working without source changes, because updating to a new SDK must not break existing code. (It can introduce new warnings, but even that is something that should be considered carefully.)

So: this proposal is designed to handle the use cases both for Swift library authors to come and for C APIs today, and in particular Apple's Objective-C SDKs and how they've evolved historically.

There's another really interesting point in your message, which Karl, Drew Crawford, and others also touched on.

Teaching the compiler/checker/whatever about the linking semantics of modules. For modules that are packaged inside the final built product, there is no need to deal with any unexpected cases, because we already have the exhaustiveness check appropriate for that scenario (regardless of whether the module is shipped as a binary or compiled from source). The app author decides when to update their dependencies, and updating those dependencies will produce new warnings/errors as the compiler notices new or deprecated cases. This is the current state of things and is completely orthogonal to the entire discussion.

This keeps sneaking into discussions and I hope to have it formalized in a proposal soon. On the library side, we do want to make a distinction between "needs binary compatibility" and "does not need binary compatibility". Why? Because we can get much better performance if we know a library is never going to change. A class will not acquire new dynamic-dispatch members; a stored property will not turn into a computed property; a struct will not gain new stored properties. None of those things affect how client code is written, but they do affect what happens at run-time.

This is incorrect; binary compatibility absolutely affects how code is written. No swift library available today is written with binary compatibility in mind, because it’s a non-issue. If it were a pervasive issue, then you’d see a lot more diligence in libraries about not straight-up breaking things between releases.

Other responses on this thread have mentioned removing implementations between releases, which is an excellent illustration of this point: when writing a binary-compatible library, *you cannot remove implementations*. So the very fact that people think that it’s ok to do so is an explicit refutation of the assertion that the manner in which client code is written is independent of the consideration of a library needing binary compatibility.

Okay, so should we use this as an indicator of whether an enum can grow new cases? (I'm going to ignore C libraries in this section, both because they don't have this distinction and because they can always lie anyway.)

- If a library really is shipped separately from the app, enums can grow new cases, except for the ones that can't. So we need some kind of annotation here. This is your "B" in the original email, so we're all agreed here.

Yes, I think so.

- If a library is shipped with the app, there's no chance of the enum growing a new case at run time. Does that mean we don't need a default case? (Or "unknown case" now.)

I don’t see why you would need a default case. While the enum’s “rawValue” (or whatever) may come from an external library that could change independently, the initializer of the enum (within the library) wouldn’t have a case to handle that. This would be up to the library author to deal with (not the app author), so you shouldn’t need “unknown case:” on your switch statement, because the library has declared the full list of enums, and that *cannot change* unless the app author decides to update his/her dependency.

The answer here is most easily understood in terms of semantic versioning. If adding a new enum case is a source-breaking change, then it's a source-breaking change, requiring a major version update. The app author decides when to update their dependencies, and might hold off on getting a newer version of a library because it's not compatible with what they have.

If adding a new enum case is not a source-breaking change, then it can be done in a minor version release of a library. Like deprecations, this can produce new warnings, but not new errors, and it should not (if done carefully) break existing code. This isn't a critical feature for a language to have, but I would argue (and have argued) that it's a useful one for library developers. Major releases still exist; this just makes one particular kind of change valid for minor releases as well.

I feel that this is a HUGE cognitive burden to place on library and app authors. And the intricacies of SemVer are, in my opinion, things that your typical app (and library) author really just doesn’t care about. They’re focused on writing their apps. SemVer also breaks when you deal with the common scenario of having your marketing department pick version numbers.

Will the compiler require me to update my CFBundleVersion because I added a new case to an enum? If not, then this is all just hand-wavy and we’re not really making anything safer at all.

(It also feels very subtle to me that 'switch' behaves differently based on where the enum came from. I know this whole proposal adds complexity to the language, and I'd like to keep it as consistent as possible.)

Okay, so what if we did this based on the 'import' rather than on how the module was compiled—Karl's `@static import`? That feels a little better to me because you can see it in your code. (Let's ignore re-exported modules for now.) But now we have two types of 'import', only one of which can be used with system libraries. That also makes me uncomfortable. (And to be fair, it's also something that can be added after the fact without disturbing the rest of the language.)

It’s not about how the module was compiled; it’s about whether or not the module is copied in to your final built product or not.

Maybe instead of “@static import”, it’s more like this off-the-cuff idea:

import Module1
import Module2 @ 12.13.2

I still feel like we’re missing a huge piece of the puzzle, which is how libraries can deal with the linking expectations of clients.

Do library versions need to be part of a mangled name, and then I can create a “multi-fat” library that has not only multiple architectures, but multiple versions per architecture?

Dave

···

Sent from my iPad

On Jan 4, 2018, at 5:37 PM, Jordan Rose <jordan_rose@apple.com> wrote:

Finally, it's very important that whatever you do in your code doesn't necessarily apply to your dependencies. We've seen in practice that people are not willing to edit their dependencies, even to handle simple SDK changes or language syntax changes (of which there are hopefully no more). That's why I'm pushing the source compatibility aspect so hard, even for libraries that won't be shipped separately from an app.

Overall, I think we're really trying to keep from breaking Swift into different dialects, and making this feature dependent on whether or not the library is embedded in the app would work at cross-purposes to that. Everyone would still be forced to learn about the feature if they used C enums anyway, so we're not even helping out average developers. Instead, it's better that we have one, good model for dealing with other people's enums, which in practice can and do grow new cases regardless of how they are linked.

Jordan

On Jan 3, 2018, at 09:07, Dave DeLong <swift@davedelong.com> wrote:

IMO this is still too large of a hammer for this problem.

This whole “unexpected case” thing is only a problem when you’re linking libraries that are external to/shipped independently of your app. Right now, the *only* case where this might exist is Swift on the server. We *might* run in to this in the future once the ABI stabilizes and we have the Swift libraries shipping as part of iOS/macOS/Linux. Other than this, unexpected enum cases won’t really be a problem developers have to deal with.

Because this will be such a relatively rare problem, I feel like a syntax change like what’s being proposed is a too-massive hammer for such a small nail.

What feels far more appropriate is:

Teaching the compiler/checker/whatever about the linking semantics of modules. For modules that are packaged inside the final built product, there is no need to deal with any unexpected cases, because we already have the exhaustiveness check appropriate for that scenario (regardless of whether the module is shipped as a binary or compiled from source). The app author decides when to update their dependencies, and updating those dependencies will produce new warnings/errors as the compiler notices new or deprecated cases. This is the current state of things and is completely orthogonal to the entire discussion.

and

Adding an attribute (@frozen, @tangled, @moana, @whatever) that can be used to decorate an enum declaration. This attribute would only need to be consulted on enums where the compiler can determine that the module will *not* be part of the final built product. (Ie, it’s an “external” module, in my nomenclature). This, then, is a module that can update independently of the final app, and therefore there are two possible cases:

  1️⃣ If the enum is decorated with @frozen, then I, as an app author, have the assurance that the enum case will not change in future releases of the library, and I can safely switch on all known cases and not have to provide a default case.

  2️⃣ If the enum is NOT decorated with @frozen, then I, as an app author, have to account for the possibility that the module may update from underneath my app, and I have to handle an unknown case. This is simple: the compiler should require me to add a “default:” case to my switch statement. This warning is produced IFF: the enum is coming from an external module, and the enum is not decorated with @frozen.

==========

With this proposal, we only have one thing to consider: the spelling of @frozen/@moana/@whatever that we decorate enums in external modules with. Other than that, the existing behavior we currently have is completely capable of covering the possibilities: we just keep using a “default:” case whenever the compiler can’t guarantee that we can be exhaustive in our switching.

Where the real work would be is teaching the compiler about internally-vs-externally linked modules.

Dave

On Jan 2, 2018, at 7:07 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org> wrote:

[Proposal: https://github.com/apple/swift-evolution/blob/master/proposals/0192-non-exhaustive-enums.md\]

Whew! Thanks for your feedback, everyone. On the lighter side of feedback—naming things—it seems that most people seem to like '@frozen', and that does in fact have the connotations we want it to have. I like it too.

More seriously, this discussion has convinced me that it's worth including what the proposal discusses as a 'future' case. The key point that swayed me is that this can produce a warning when the switch is missing a case rather than an error, which both provides the necessary compiler feedback to update your code and allows your dependencies to continue compiling when you update to a newer SDK. I know people on both sides won't be 100% satisfied with this, but does it seem like a reasonable compromise?

The next question is how to spell it. I'm leaning towards `unexpected case:`, which (a) is backwards-compatible, and (b) also handles "private cases", either the fake kind that you can do in C (as described in the proposal), or some real feature we might add to Swift some day. `unknown case:` isn't bad either.

I too would like to just do `unknown:` or `unexpected:` but that's technically a source-breaking change:

switch foo {
case bar:
  unknown:
  while baz() {
    while garply() {
      if quux() {
        break unknown
      }
    }
  }
}

Another downside of the `unexpected case:` spelling is that it doesn't work as part of a larger pattern. I don't have a good answer for that one, but perhaps it's acceptable for now.

I'll write up a revision of the proposal soon and make sure the core team gets my recommendation when they discuss the results of the review.

---

I'll respond to a few of the more intricate discussions tomorrow, including the syntax of putting a new declaration inside the enum rather than outside. Thank you again, everyone, and happy new year!

Jordan

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Panajev · January 5, 2018, 8:11am

I will let your very comprehensive rationale brew a little bit more in my mind and reread it, but I still think this is a bit leaning too much over to library authors than library consumers (not unlike the closed by default discussion we all had a while ago).

I remain very unconvinced by the “updating library/SDK” must be source compatible. Especially if it is major version change and a deprecated method has been removed, methods have changed signature, etc... Backward compatible runtime with code compiled against the previous SDK yes, source compatible no.

I feel like this change will make the life easier for library authors, but risks making it easier and easier to make mistakes in apps using the libraries: exhaustive switching over enums is a good important feature that risks going away (the default matters) for not a huge effective benefit to app developers.

···

On 5 Jan 2018, at 00:38, Jordan Rose via swift-evolution <swift-evolution@swift.org> wrote:

Furthermore, the old code needs to continue working without source changes, because updating to a new SDK must not break existing code. (It can introduce new warnings, but even that is something that should be considered carefully.)

So: this proposal is designed to handle the use cases both for Swift library authors to come and for C APIs today, and in particular Apple's Objective-C SDKs and how they've evolved historically.