SE-0192 — Non-Exhaustive Enums

In regards of A, doesn’t this code cover al cases?

@incomplete enum {
  case pancake
  case waffle
  case juice
}

When the @incomplete tag is present, the compiler enforces (with an error) that all switches handle a default case:

switch breakfast {
  case .pancake:
  case .waffle:
  case .juice:
  default: // <— default case must be present to compile
    break
}

This is also allowed:

switch breakfast {
  case .pancake:
    // only like pancakes and nothing else!
  default: // <— default case must be present to compile
    break
}

I think it is safe for the compiler not to warn users when new cases are introduced (by the new OS, for instance), in similar way as users are not warned when new methods are added to a class, or new classes added to a framework. For instance, if a new case is added for UILabel text alignment, I don’t _really_ need to know unless I wanted my app to support that case. Users would be able to get that information from the documentation.

In regards of B (select one of the choices to be chosen as the normal choice if no choice is made by the user), sounds like an edge case and should be left for a separate proposal.

Thank you,
Eneko

···

On Jan 2, 2018, at 10:11 AM, Jason Merchant via swift-evolution <swift-evolution@swift.org> wrote:

I think this whole thing has been unnecessarily convoluted. As a result, the majority of the replies are rabbit holes.

In my opinion, the true root of the concept in question is as follows:

A list of something is desired:
1 - Pancake
2 - Waffle
3 - Juice

Developer wishes to be able to:
A) Add new things to the list of choices in the future as they come up with new ideas
B) Sometimes select one of the choices to be chosen as the normal choice if no choice is made by the user

A and B are separate desires. In some circumstances a developer may want to add a new choice and make it the normal choice when there was no normal choice was clarified before.

____________________

Part 2:

After this simple desire is clear, there should be two discussions:
A) In a text only coding language, what would we like the syntax to look like? (Without regard to past-bias. What should it really be, forget what mistaken design choices were made in Swift in the past)
B) How do we approach making this happen behind the scenes?

Bonus: Given that some of us have changed our approach to programming significantly beyond text based coding, and into more dynamic mediums of programming in other niches, and even here and there in Xcode - I would recommend considering how the IDE would show a modern version of this concept. I feel too often that Swift design syntax has a lack of awareness between the distinctions of what the IDE should do, as opposed to what the syntax of the language should be, and what should be handled behind the scenes by automated tooling.

_____________________

My opinion, in answering the above questions is in preference to a simple easy to read and write syntax, something like the following:

choices Breakfast {
    Pancake, Waffle, Juice
}

If a "default" choice is desired, it is obvious to me that I would select the choice from the IDE, and it would be visually indicated that it was the default.

When changes occur, whether new choices are added, old ones are removed or changed, or a default is added, changed, or removed - a behind the scenes automated tool analyzes the changes and presents migration options through the IDE.

_____________________

Sincerely,
Jason

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I'll admit I hadn't thought of using "unknown default" (or "default unknown"). I don't think that's terrible, but I mildly prefer `unknown case` because it builds on the "pun" that enum elements are also defined using 'case'. If anything hits this part of the switch, it really will be an "unknown case", i.e. a statically-unknown enum element.

To Cheyo's point, if this were to be a single token I'd probably spell it #unknown, like #available. Then we'd have `case #unknown:` and something that naturally expands to other pattern positions. I found that less aesthetically pleasing, though, and so a context-sensitive keyword seemed like the way to go.

(For the record, though, I wouldn't describe `case _` as a special case of `default`. They do exactly the same thing, and `_` is a useful pattern in other contexts, so if anything the current `default` should be thought of as syntactic sugar for `case _`.)

I'll add these points to the "Alternatives Considered" section in the PR later today.

Jordan

···

On Jan 3, 2018, at 22:56, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

As has already been said, “case unknown” is source-breaking because it conflicts with any real cases named “unknown”; “\unknown” looks like a key path but isn’t, and I wonder if it would potentially conflict with existing key paths.

In any case, my point was not to bikeshed the “unknown” part, but to ask whether any consideration had been made to have the feature presented as a flavor of default instead of a flavor of case.

On Wed, Jan 3, 2018 at 23:57 Cheyo Jimenez <cheyo@masters3d.com <mailto:cheyo@masters3d.com>> wrote:

On Jan 3, 2018, at 6:52 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

This is a very nice revision. One bikeshedding thought:

Since "unknown case" is presented as a special kind of "default", can't be mixed with "default", and can't be used in case patterns, why not "default unknown" (or "unknown default") instead of "unknown case"?

`case _ :` is already a special case of default.
I’d rather have `case unknown :`
`unknown case :` is weird because of the order of `case`.

Another alternative is `case \unknown :`
`\unknown` would also allow pattern matching.

On Wed, Jan 3, 2018 at 8:05 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Jan 2, 2018, at 18:07, Jordan Rose <jordan_rose@apple.com <mailto:jordan_rose@apple.com>> wrote:

[Proposal: https://github.com/apple/swift-evolution/blob/master/proposals/0192-non-exhaustive-enums.md\]

Whew! Thanks for your feedback, everyone. On the lighter side of feedback—naming things—it seems that most people seem to like '@frozen', and that does in fact have the connotations we want it to have. I like it too.

More seriously, this discussion has convinced me that it's worth including what the proposal discusses as a 'future' case. The key point that swayed me is that this can produce a warning when the switch is missing a case rather than an error, which both provides the necessary compiler feedback to update your code and allows your dependencies to continue compiling when you update to a newer SDK. I know people on both sides won't be 100% satisfied with this, but does it seem like a reasonable compromise?

The next question is how to spell it. I'm leaning towards `unexpected case:`, which (a) is backwards-compatible, and (b) also handles "private cases", either the fake kind that you can do in C (as described in the proposal), or some real feature we might add to Swift some day. `unknown case:` isn't bad either.

I too would like to just do `unknown:` or `unexpected:` but that's technically a source-breaking change:

switch foo {
case bar:
  unknown:
  while baz() {
    while garply() {
      if quux() {
        break unknown
      }
    }
  }
}

Another downside of the `unexpected case:` spelling is that it doesn't work as part of a larger pattern. I don't have a good answer for that one, but perhaps it's acceptable for now.

I'll write up a revision of the proposal soon and make sure the core team gets my recommendation when they discuss the results of the review.

---

I'll respond to a few of the more intricate discussions tomorrow, including the syntax of putting a new declaration inside the enum rather than outside. Thank you again, everyone, and happy new year!

I ended up doing these in the opposite order, writing up the new proposal first and not yet responding to the discussion that's further out. You can read my revisions at https://github.com/apple/swift-evolution/pull/777\.

In particular, I want to at least address:
- Dave D and Drew C's points about versioned libraries / linking semantics of modules.
- Jason M's point about migration
and I'll do one more pass over the thread to see if there's anything else I didn't address directly. (That doesn't mean everyone who disagrees, just messages where I think there's more I can do to explain why the proposal is the way it is.)

Jordan

P.S. Enjoying the Disney references. Thanks, Nevin and Dave. :-)

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

Fair concerns Gwendal, but why can’t the default in these cases be just exhaustive / frozen unless the library developer explicitly marks it as “unfrozen/non exhaustive” and the compiler can warn the users when they switch over it instead of throwing an error by default (the user can still treat warnings as errors if they want and suppress this warning if they wanted to in this vision)?

I think, sorry if I am being presumptions, that the onus in this case should be on the library author to mark enums and opt in in this non exhaustive behaviour. @Jordan what would the issue in this be?

···

Sent from my iPhone

On 5 Jan 2018, at 09:11, Gwendal Roué <gwendal.roue@gmail.com> wrote:

Le 5 janv. 2018 à 09:11, Goffredo Marocchi via swift-evolution <swift-evolution@swift.org> a écrit :

I feel like this change will make the life easier for library authors, but risks making it easier and easier to make mistakes in apps using the libraries: exhaustive switching over enums is a good important feature that risks going away (the default matters) for not a huge effective benefit to app developers.

I agree that life is made worse for a library user that meets a non-frozen enum. And I don't know if it makes the life easier for library authors, because library authors not only throw code in the wild, but also try to prevent library users from making the mistakes you talk about.

To take GRDB.swift as an example, it currently has 15 public enums. What are my options?

More than half of those enums mirror SQLite C enums, or sets of related SQLite values. SQLite is still a living piece of sofware, and it may adds new cases or values in the future. It is then likely that GRDB has to be updated as well. It thus looks like I can make those enums @frozen. But then it looks like I betray the proposal's goal. What's the correct choice?

Some enums are like SKPaymentTransactionState: there's no way the user could infer a proper behavior in a mandatory default/unknown switch case. I thus also want to make them @frozen, and avoid headaches to the library users. Will I pay for it later?

Some other enums are positively non-frozen. I may want to replace some of them with a struct with a limited set of pre-built values. This would, IMHO, improve the library UX because it would again prevent headaches for the library users about scary "future" cases. In the struct with a limited set of values, there is no future cases. Instead, there are a limited set of named values. Which is a very different way to say the same thing.

This last paragraph draws a strong parallel between non-frozen enums and structs:

    // non-frozen enum:
    public enum E {
        case a, b
    }
    
    // equivalent struct:
    public struct S: Equatable {
        private let value: Int
        private init(_ value: Int) { self.value = value }
        public static let a = S(1)
        public static let b = S(2)
        public static func == (lhs: S, rhs: S) -> Bool {
            return lhs.value == rhs.value
        }
    }

    func f(_ s: S) {
        // S can be switched over just like a struct:
        switch s {
        case .a: print("a")
        case .b: print("b")
        default: print("unknown")
        }
    }

BTW, did anybody think about importing C enums as such structs?

Gwendal

The proposal suggests default non-frozen enums. If I remember well, that's because it's easier to switch from non-frozen to frozen than the opposite.

I buy this ABI-based argument very well, since 1. I'm not an ABI expert, and 2. as a library author I will scratch my head for each of my public enums anyway.

Now I still think that the choice is really uneasy. I've given some GRDB examples. And I'd also like to know how ABI experts would have introduced and evolved SKPaymentTransactionState in a hypothetic Swift 5+ world.

Gwendal

···

Le 5 janv. 2018 à 10:33, Goffredo Marocchi <panajev@gmail.com> a écrit :

Fair concerns Gwendal, but why can’t the default in these cases be just exhaustive / frozen unless the library developer explicitly marks it as “unfrozen/non exhaustive” and the compiler can warn the users when they switch over it instead of throwing an error by default (the user can still treat warnings as errors if they want and suppress this warning if they wanted to in this vision)?

The review of "SE 0192 - Non-Exhaustive Enums" begins now and runs through January 3, 2018.

The proposal is available here:

https://github.com/apple/swift-evolution/blob/master/proposals/0192-non-exhaustive-enums.md+1, it needs to happen (and ASAP, since it _will_ introduce source-breaking changes one way or the other).

I think non-exhaustive is the correct default. However, does this not mean that, by default, enums will be boxed because the receiver doesn’t know their potential size?

It's not always boxing, but yes, there will be more indirection if the compiler can't see the contents of the enum. (More on that below.)

That would mean that the best transition path for multi-module Apps would be to make your enums @exhaustive, rather than adding “default” statements (which is unfortunate, because I imagine when this change hits, the way you’ll notice will be complaints about missing “default” statements).

Yep, that's going to be the recommendation. The current minimal-for-review implementation does not do this but I'd like to figure out how to improve that; at the very least it might be a sensible thing to do in the migrator.

I do have some thoughts about how we could ease the transition (for this and other resilience-related changes), but it’s best to leave that to a separate discussion.

The one thing I’m still not overly fond of is the name - I would like us to keep the set of resilience/optimisation related keywords to a minimum. “exhaustive” for enums feels an awful lot like “fixed_contents” for structs - couldn’t we come up with a single name which could be used for both? I don’t think anybody’s going to want to use “exhaustive” for structs.

The core team was very focused on this too, but I contend that "exhaustive" is not about optimization and really isn't even about "resilience" (i.e. the ability to evolve a library's API while preserving binary compatibility). It's a semantic feature of an enum, much like 'open' or 'final' is for classes, and it affects what a client can or can't do with an enum. For libaries compiled from source, it won't affect performance at all—the compiler still knows the full set of cases in the current version of the library even if the programmer is forced to consider future versions.

I'm working on the fixed-contents proposal now, though it won't be ready for a while, and the same thing applies there: for structs compiled from source, the compiler can still do all the same optimizations. It's only when the library has binary compatibility concerns that we need to use extra indirection, and then "fixed-contents" becomes important. (As currently designed, it doesn't affect what clients can do with the struct at all.) This means that I don't expect a "normal" package author to write "fixed-contents" at all (however it ends up being spelled), whereas "exhaustive" is a fairly normal thing to consider whenever you make an enum public.

I hope that convinces you that "fixed-contents" and "exhaustive" don't need to have the same name. I don't think anyone loves the particular name "exhaustive", but as you see in the "Alternatives considered" we didn't manage to come up with anything significantly better. If reviewers all prefer something else we'd consider changing it.

Thanks for responding!
Jordan

When you say “libraries compiled from source”, what do you mean?

- Other targets in your project
- Source packages built through SwiftPM / CocoaPods / Carthage / other

And I was being imprecise with the terminology, but also

- Libraries built by someone else but designed to be embedded into an app, so that there's no chance of a different version showing up at run-time.

As for whether its a resilience feature: actually it is completely a resilience feature. The effects on switching are only side-effects; really what “exhaustive” or “nonexhaustive” are saying is literally that cases may be added later. Even if we added private cases, you wouldn’t need to mark those enums as specially exhaustive or not; that would be implied. It’s an accommodation for things which don’t exist yet, so really, it is all about resilience IMO.

"Resilience", as an admittedly fuzzily-defined term in the Swift project, specifically refers to what changes can be made without breaking binary compatibility <https://github.com/apple/swift/blob/master/docs/Lexicon.rst&gt;\. It does not refer to every change you can make to a library. (For comparison, adding a field to a struct is not source-breaking in Swift. We would like to make it not ABI-breaking either; that proposal's coming soon.)

Anyway, as I see it, library authors in general ought to be happy about this:
+ Their libraries become safer by default, so they can make changes in the future without having to worry about breakage
+ It doesn’t affect your code inside of a module, so it only affects types they already explicitly marked “public”

That's the intent.

The only people who lose are multi-module App developers, because they are “library authors” who don’t need to care about evolution, and now need to add attributes to things they wouldn’t have to before, or suffer language and performance penalties. Their libraries become less reusable and not resilient-by-default.

For example, I have an App for which I wrote a cross-platform model framework in Swift. When I compile it as a framework inside my App, it is bundled there forever. However, I use the same code to build libraries for Linux, which I would like to ship in binary form to 3rd-parties. Am I supposed to litter my code with annotations to mark those types as final, just to make the App fast and convenient to code? What happens when I need to fix a bug and distribute an updated copy, this means the 3rd-parties need to recompile (which they won’t do…).

Typically, for such a problem, I would recommend using a static library instead. But we don’t have those, and anyway they’re not always the best thing these days. So that’s why I started a new thread about creating a “@static” import, so App developers can go back to all the conveniences they had before.

There won't be a perf penalty, but yes, I do expect multi-module apps to use 'exhaustive' on most of their enums, because they don't need the futureproofing. Maybe this should have been mentioned more explicitly in the proposal.

As a perhaps more long-term design note, I think modules ought to have the ability to version-lock themselves to one or more of their dependencies. They would still be required to obey access control as if they were outside those dependencies, but we would suppress some of the semantic consequences of being outside the module, such as the need to assume non-exhaustiveness by default.

That is, there would be two independent axes of library dependency: source vs. binary and version-compatible vs. version-locked:
  - a source dependency allows the compiler to take advantage of the implementation of public entities when generating code
  - a version-locked dependency allows the compiler to take advantage of the implementation of public entities when enforcing semantics

Apps would generally elect to primarily use version-locked source dependencies because they're just pulling down source libraries (e.g. from github) and are comfortable with updating their code if the library changes.

Source libraries on github would generally want to use version-compatible source dependencies because version-locking would put their clients in "library hell" if the locking didn't all agree.

Binary dependencies could reasonably use either.

This model aligns pretty well with what I would like to see. It prevents us from paying a penalty when we don’t need the benefits provided by a restriction.

Relating this back to the current proposal, would you expect an app to have the ability to switch over an enum provided by a version-locked dependency that is not annotated with @exhaustive without requiring a default clause?

Yes, and as we find other places where program semantics depend on knowing the implementation, I would expect them to follow suit.

My guess is that enum exhaustiveness is probably more prominent than any other such feature, and maybe even more prominent than all of them put together, but possible examples include:
  - automatically deriving protocol conformances, which we hope will eventually be something you can do for an arbitrary protocol
  - any other kind of structural metaprogramming we might add
  - maybe memberwise struct initialization if there are no explicit initializers, although this is arguably an access control question (just as public/open is)
  - ownership-related features that might make sense to restrict to stored properties, like expecting a struct property to have a stable address, or destructuring a struct with pattern-matching

Now, some of these things might be nice to allow even for resilient types. I know Joe has suggested adding some way of resiliently describing a structural decomposition of a type, which you could then use to derive conformances, etc. But since the basic motivation for restricting any of them is source/binary compatibility, and since version-locking would tell us that the programmer doesn't care about that, it seems sensible that version-locking ought to suppress the restrictions.

Relating to @inlinable proposal also under review, would everything in a source dependency be automatically inlinable whether they were annotated as such or not (at least when version-locked)?

Yes, and regardless of being version-locked. Inlining isn't semantically visible: it's observable in various low-level ways, but it's not supposed to affect basic program semantics except in incidental ways, e.g. by lowering the memory requirements so that programs start working that didn't before. So the compiler is generally always allowed to inline when the call is direct and the callee has a known implementation; that's just standard "as if" behavior. The fact that we can't do this today is just an unfortunate consequence of our current build model.

This is all exciting to hear (as a long term direction)! Thank you for the elaboration.

To be clear, I'm laying out my own vision for this, not an accepted core-team direction. But if you think it's exciting, that's certainly helpful!

Of course. :) This model would eliminate some of the tradeoffs app developers are currently required to make so it’s a huge win if we can get there. I hope the rest of the core team will be on board when the time is right.

···

On Dec 21, 2017, at 2:17 PM, John McCall <rjmccall@apple.com> wrote:

On Dec 21, 2017, at 3:10 PM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Dec 21, 2017, at 2:06 PM, John McCall <rjmccall@apple.com <mailto:rjmccall@apple.com>> wrote:

On Dec 21, 2017, at 2:41 PM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Dec 21, 2017, at 1:26 PM, John McCall via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Dec 21, 2017, at 2:03 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Dec 20, 2017, at 12:35, Karl Wagner <razielim@gmail.com <mailto:razielim@gmail.com>> wrote:

On 20. Dec 2017, at 19:54, Jordan Rose <jordan_rose@apple.com <mailto:jordan_rose@apple.com>> wrote:

On Dec 20, 2017, at 05:36, Karl Wagner via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 19. Dec 2017, at 23:58, Ted Kremenek via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

John.

[...]

in what other circumstances do we insist that the compiler inform the end user about future additions to the API at compile time?

This isn’t a request for the compiler to inform the user about future additions to an API. It is a request to validate the compiler’s knowledge of the current state of an API with the current state of the source code.

Well, it's of course impossible to inform the user about future additions, so that's poorly phrased on my part. It's about the compiler informing the end user about *new* additions, part of the *current* state of the API, that have cropped up since the user last revised the code when the API was in a *previous* state (or, indistinguishably, members of which a user is unaware regardless of the temporal sequence of when such members were added). In what other circumstances do we insist that the compiler perform this service?

Enums. That's literally how they work today. You are arguing in favor of actively removing compiler-aided correctness.

There's also protocol requirements and, arguably, deprecated methods with a proper message ("use foo instead").

[...]

in what other circumstances do we insist that the compiler inform the end

user about future additions to the API at compile time?

This isn’t a request for the compiler to inform the user about future
additions to an API. It is a request to validate the compiler’s knowledge
of the *current* state of an API with the *current* state of the source
code.

Well, it's of course impossible to inform the user about future additions,
so that's poorly phrased on my part. It's about the compiler informing the
end user about *new* additions, part of the *current* state of the API,
that have cropped up since the user last revised the code when the API was
in a *previous* state (or, indistinguishably, members of which a user is
unaware regardless of the temporal sequence of when such members were
added). In what other circumstances do we insist that the compiler perform
this service?

Enums. That's literally how they work today. You are arguing in favor of
actively removing compiler-aided correctness.

There's also protocol requirements

No, that's now how enums work today, and it's not how protocol requirements
work today. Enums today are all semantically exhaustive; if a case is added
in a later version of a library, it's semantically a *different* enum type
that happens to share the same name. Not considering all the cases of an
exhaustive enum is an _error_, not a _warning_, because there is no basis
on which to proceed. This will not change with the proposal. Likewise,
adding a protocol requirement without a default implementation is
source-breaking. The result is a compiler _error_.

The question is, what non-source breaking API additions today cause the
compiler to inform the end user of such additions? The answer is: none
whatsoever. Not new methods or properties on a type, not even new protocol
requirements that have a default implementation.

and, arguably, deprecated methods with a proper message ("use foo instead").

···

On Tue, Jan 2, 2018 at 3:27 PM, Kevin Nattinger <swift@nattinger.net> wrote:

Hello all,

I think this whole thing has been unnecessarily convoluted. As a result, the majority of the replies are rabbit holes.

In my opinion, the true root of the concept in question is as follows:

A list of something is desired:
1 - Pancake
2 - Waffle
3 - Juice

Developer wishes to be able to:
A) Add new things to the list of choices in the future as they come up with new ideas
B) Sometimes select one of the choices to be chosen as the normal choice if no choice is made by the user

A and B are separate desires. In some circumstances a developer may want to add a new choice and make it the normal choice when there was no normal choice was clarified before.

I don't think this is an accurate summary of the problem being tackled here. Rather, we are how to enable the vendor of a nonexhaustive enum to add new cases without breaking binaries compiled against previous versions. There is little here to do with what a "default" should be. Indeed, it is an explicit design decision of Swift not to support types having an implicit default value.

There is no way a library developer of libraries bundled in the app can break the app by adding new cases. They may cause compiler issues when the app author tries to update the library, but it will not break existing apps.

The concern for updating enums is mostly an Apple / OS related concern for libraries/dynamic frameworks the app does not ship with, but links to at runtime and we should have an exception for that. We should not use the same solution for both and lose exhaustiveness checks when we do not need to. It would be wrong.

Those dynamic frameworks should be the one that have to opt-in (even better if it is done automatically for them) in non-exhaustive extra resilient behaviour, not libraries you ship in the app.

The app developer should be informed and have to address the new cases or the removal of old cases.

···

On 2 Jan 2018, at 18:36, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:

On Tue, Jan 2, 2018 at 12:11 PM, Jason Merchant via swift-evolution <swift-evolution@swift.org> wrote:

____________________

Part 2:

After this simple desire is clear, there should be two discussions:
A) In a text only coding language, what would we like the syntax to look like? (Without regard to past-bias. What should it really be, forget what mistaken design choices were made in Swift in the past)
B) How do we approach making this happen behind the scenes?

Bonus: Given that some of us have changed our approach to programming significantly beyond text based coding, and into more dynamic mediums of programming in other niches, and even here and there in Xcode - I would recommend considering how the IDE would show a modern version of this concept. I feel too often that Swift design syntax has a lack of awareness between the distinctions of what the IDE should do, as opposed to what the syntax of the language should be, and what should be handled behind the scenes by automated tooling.

_____________________

My opinion, in answering the above questions is in preference to a simple easy to read and write syntax, something like the following:

choices Breakfast {
    Pancake, Waffle, Juice
}

If a "default" choice is desired, it is obvious to me that I would select the choice from the IDE, and it would be visually indicated that it was the default.

When changes occur, whether new choices are added, old ones are removed or changed, or a default is added, changed, or removed - a behind the scenes automated tool analyzes the changes and presents migration options through the IDE.

_____________________

Sincerely,
Jason

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Sent from my iPad

Sent from my iPad

I agree that we need a solution to the problem described. I also agree that non-exhaustive is most in keeping with the overall design of Swift at module boundaries. However, I believe this proposal should be modified before being accepted

Thanks for writing this up - you’ve explained a common concern in an interesting way:

This is likely to be a relatively rare need mostly encountered by 3rd party libraries but it will happen. When it does happen it would be really unfortunate to be forced to use a `default` clause rather than something like a `future` clause which will produce an error when compiled against an SDK where the enum includes cases that are not covered. I can imagine cases where this catch-all case would need to do something other than abort the program so I do not like the `switch!` suggestion that has been discussed. The programmer should still be responsible for determining the behavior of unknown cases.

..

While library authors have a legitimate need to reserve the right to introduce new cases for some enums this need can be met without taking away a useful tool for generating static compiler errors when code does not align with intent (in this case, the intent being to cover all known cases). Switch statements working with these kinds of enums should be required to cover unknown cases but should be able to do so while still being statically checked with regards to known cases.

I think that this could be the crux of some major confusion, the root of which is the difference between source packages and binary packages that are updated outside your control (e.g. the OS, or a dynamic library that is updated independently of your app like a 3rd party plugin). Consider:

1) When dealing with independently updated binary packages, your code *has* to implement some behavior for unexpected cases if the enum is non-exhaustive. It isn’t acceptable to not handle that case, and it isn’t acceptable to abort because then your app will start crashing when a new OS comes out. You have to build some sort of fallback into your app.

2) When dealing with a source package that contributes to your app (e.g. through SwiftPM), *YOU* control when you update that package, and therefore it is entirely reasonable to exhaustively handle enums even if that package owner didn’t “intend” for them to be exhaustive. When *you* chose to update the package, you get the “unhandled case” error, and you have maximal “knowability” about the package’s behavior.

It seems that your concern stems from the fact that the feature as proposed is aligned around module boundaries, and therefore overly punishes source packages like #2. I hope you agree that in case #1, that the feature as proposed is the right and only thing we can do: you really do have to handle unknown future cases somehow.

If I’m getting this right, then maybe there is a variant of the proposal that ties the error/warning behavior to whether or not a module is a source module vs a binary module. The problem with that right now is that we have no infrastructure in the language to know this…

Hi Chris, thanks for your reply.

The concern you describe isn’t exactly what I was describing but it is related. John McCall recently posted a sketch of a solution to the concern you describe which looked great to me. I don’t have time to look up the link this morning but I think it was in this review thread.

The actual concern I am describing is where a 3rd party library (or app) wants to switch over a non-exhaustive enum provided by a module that is a binary (not source) dependency. The author of the 3rd party library may have a legitimate reason to switch over an enum despite the author of the binary module reserving the right to add additional cases.

When this circumstance arises they will do it using the tools provided by the language. Regardless of the final language solution they obviously need to cover unknown cases - their library could be shipping on a device which receives an update to the binary dependency that contains a new case. I agree with you that a language-defined crash is not appropriate. The author of the switch must take responsibility for the behavior of unknown cases.

I am arguing that these “pseudo-exhaustive” switch statements will exist in the wild. The crucial point of contention is whether or not the language provides assistance to the author of the 3rd party library in updating their library when the enum provided by the binary dependency changes. Is the author forced to use a `default` case which turns of exhaustiveness checking? Or are they able to use an alternative mechanism for handling unknown cases which does not turn off exhaustiveness checking - all statically known cases must be covered. The most common example of such a mechanism is the `future` (or perhaps `unknown`) case which would only be used for cases that are not statically known.

This facility will of course help authors of these switch statements make the necessary updates as the enum vended by the binary dependency changes. It will also help alert authors of apps that depend on that 3rd party library (which will usually be a source dependency). If the author of the app attempts to rebuild the dependency against a new SDK with added cases the library will fail to build, alerting the user that they should update the 3rd party library.

My position is that if there are reasonable use cases for these kinds of “pseudo-exhaustive” switches then the language should provide exhaustiveness checking of statically known cases via some mechanism that authors can opt-in to using. It’s ok with me if this is a relatively esoteric feature. It won’t be commonly needed, but when it is necessary it will provide significant value.

IIRC there were some reasonable examples of these kinds of switches posted in earlier threads on this topic. I don’t have time to look those up right now either but it would be good for the core team to be aware of them before making a final decision.

I am only aware of two arguments against this kind of “pseudo-exhaustive” switch. One is that users should not attempt to switch over an enum that a library author does not intend to be exhaustive (i.e. it is an “input-only” enum). The other is that a `future` or `unknown` case is not testable.

The first argument is a moral one which I believe should not carry much weight relative to concrete, pragmatic counter-examples such as those that (IIRC) were provided on this list in the past.

I think the discussion is converging on the defect here, but I disagree that the first argument is merely a "moral one"; it is an epistemological one.

The epistemological dilemma is: how can the user of the library "know" all the cases when the library "author" has made it clear that even _they_ don't know all the cases? Obviously, the library user can only switch over the cases that _he or she_ knows about. And if the user has forgotten about some cases that are documented, then clearly, he or she doesn't know about it.

The purpose of the requested modification is to ensure that the user’s knowledge of cases matches the compiler’s knowledge of cases.

I disagree with your framing of the argument. I don’t mean to split hairs, but I view it as a moral argument with an epistemological basis. The argument says that because of the epistemological uncertainty introduced by the possibility of unknown future cases the language should not assist a developer who attempts to cover all cases that are knowable by the compiler at compilation time. It is a judgement that such a language feature is undesirable for one reason or another. This is fundamentally a value judgement.

I would disagree with you here. It's not a value judgment to say that, semantically, there is no meaningful distinction between cases the compiler knows about that you don't know about, and future cases neither you nor the compiler knows about.

I don’t understand what you intend to distinguish by "cases the compiler knows about that you don’t know about”. The compiler should behave consistent with the documentation and code completion for the SDK in use therefore such knowledge should always be equal. If you’re referring to a change when updating the compiler and SDK, this is exactly what we want the compiler to help us detect.

The argument here is rather that it should be possible to ensure that the user has switched over all the cases that _the compiler_ knows about. Suppose we accept the argument that we want to enable the user to deal with the largest possible set of known cases: why does it have to the compiler that does the warning? why is this not a task for a linter if so desired?

Only the compiler is able to do this with 100% accuracy. Even if we assume that a linter is able to correctly validate this condition, the source will not have the necessary information. How should a linter distinguish intentional use of `default` from one which is only intended to be used when a value that is not known at compile time is encountered? This will require a pseudo-linguistic keyword comment such as `/* unknown */ default`. I hope we can agree that encouraging linters to introduce language variations like this is undesirable.

There are many ways to skin the cat, as it were. I do not agree that comments for linters is undesirable, for one. But seeing as we clearly have a difference of opinion here, this reinforces the idea that this is a great task for a linter. *My* linter of choice could read the comments. *Yours* could, for example, replace every `default` which doesn't cover cases known to the linter with

case knownToLinter1, knownToLinter2, knownToLinter3:
  fallthrough
default:
  // ...

I think we have a strong difference of opinion here. I don’t mind linter comments for pragmas to disable or modify settings but I would strongly dislike any linter that added pseudo-linguistic features like this.

in what other circumstances do we insist that the compiler inform the end user about future additions to the API at compile time?

This isn’t a request for the compiler to inform the user about future additions to an API. It is a request to validate the compiler’s knowledge of the current state of an API with the current state of the source code.

Well, it's of course impossible to inform the user about future additions, so that's poorly phrased on my part. It's about the compiler informing the end user about *new* additions, part of the *current* state of the API, that have cropped up since the user last revised the code when the API was in a *previous* state (or, indistinguishably, members of which a user is unaware regardless of the temporal sequence of when such members were added). In what other circumstances do we insist that the compiler perform this service?

The compiler performs this service all the time when the code is incompatible with the addition to the API. I think you specifically mean non-source breaking additions. See my reply to your later email for my thoughts on this.

···

On Jan 2, 2018, at 2:09 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Tue, Jan 2, 2018 at 1:46 PM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:
On Jan 2, 2018, at 12:48 PM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:

On Tue, Jan 2, 2018 at 9:38 AM, Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
On Jan 1, 2018, at 11:47 PM, Chris Lattner <clattner@nondot.org <mailto:clattner@nondot.org>> wrote:

On Dec 31, 2017, at 12:14 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

The second argument doesn’t make sense to me: as I noted in my review post, the code path is equally untestable when a `default` case is used, but all statically known cases are matched in earlier patterns. The testability problem of non-exhaustive enums is orthogonal to the issue of language support for “pseudo-exhaustive” switches.

This is the line of reasoning which leads me to conclude that we should dig up the concrete examples which have been provided and evaluate them for merit. If we can’t discard them as a bad coding practice for which a better solution is available then we should strongly consider providing language support for these use cases. The existence of such use cases should also motivate a solution to the testability problem.

- Matthew

-Chris

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

Is it hard to imagine that most everyone can get what they want and keep
the syntax clean and streamlined at the same time? Without any "@" signs or
other compiler hints?

"Rather, we are how to enable the vendor of a nonexhaustive enum to add new

cases without breaking binaries compiled against previous versions"

When an enum changes, and the change causes the code to break, the user can
be presented with migration options from an automated IDE tool. In what
specific way does this not solve the issue about having to upgrade your
code when using someone else's code library? This very notion implies your
disgruntled about doing work when things are upgraded, is that really what
this fuss is all about?

A well written language interpreter and auto-tooling IDE would not need
hints embedded in the code syntax itself. Migration hints from version to
version should not be a part of either the past or future version of the
code library.

...

I don't expect the community to agree on language grammar, but the common
sense here on how to achieve the intended goals seems to be out of wack.

If someone can present a clear logical statement as to how an automated
migration tool behind the scenes in the IDE to handle all your versioning
worries, does not make this whole discussion about adding more convoluted
syntax additions irrelevant, I'd love to hear it.

···

___________________

Sincerely,
Jason

On Tue, Jan 2, 2018 at 12:36 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Tue, Jan 2, 2018 at 12:11 PM, Jason Merchant via swift-evolution < > swift-evolution@swift.org> wrote:

I think this whole thing has been unnecessarily convoluted. As a result,
the majority of the replies are rabbit holes.

In my opinion, the true root of the concept in question is as follows:

*A list of something is desired:*
1 - Pancake
2 - Waffle
3 - Juice

*Developer wishes to be able to:*
*A)* Add new things to the list of choices in the future as they come up
with new ideas
*B)* Sometimes select one of the choices to be chosen as the normal
choice if no choice is made by the user

A and B are *separate desires*. In some circumstances a developer may
want to add a new choice and make it the normal choice when there was no
normal choice was clarified before.

I don't think this is an accurate summary of the problem being tackled
here. Rather, we are how to enable the vendor of a nonexhaustive enum to
add new cases without breaking binaries compiled against previous versions.
There is little here to do with what a "default" should be. Indeed, it is
an explicit design decision of Swift not to support types having an
implicit default value.

____________________

*Part 2:*

After this simple desire is clear, there should be two discussions:
*A)* In a text only coding language, what would we like the syntax to
look like? (Without regard to past-bias. What should it really be, forget
what mistaken design choices were made in Swift in the past)
*B)* How do we approach making this happen behind the scenes?

*Bonus:* Given that some of us have changed our approach to programming
significantly beyond text based coding, and into more dynamic mediums of
programming in other niches, and even here and there in Xcode - I would
recommend considering how the IDE would show a modern version of this
concept. I feel too often that Swift design syntax has a *lack of
awareness between the distinctions of what the IDE should do, as opposed to
what the syntax of the language should be*, and what should be handled
behind the scenes by automated tooling.

_____________________

*My opinion*, in answering the above questions is in preference to a
simple easy to read and write syntax, something like the following:

choices Breakfast {
    Pancake, *Waffle*, Juice
}

If a "default" choice is desired, it is obvious to me that I would select
the choice from the IDE, and it would be visually indicated that it was the
default.

When changes occur, whether new choices are added, old ones are removed
or changed, or a default is added, changed, or removed - a behind the
scenes automated tool analyzes the changes and presents migration options
through the IDE.

_____________________

Sincerely,
Jason

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I'll admit I hadn't thought of using "unknown default" (or "default unknown"). I don't think that's terrible, but I mildly prefer `unknown case` because it builds on the "pun" that enum elements are also defined using 'case'. If anything hits this part of the switch, it really will be an "unknown case", i.e. a statically-unknown enum element.

To Cheyo's point, if this were to be a single token I'd probably spell it #unknown, like #available. Then we'd have `case #unknown:` and something that naturally expands to other pattern positions. I found that less aesthetically pleasing, though, and so a context-sensitive keyword seemed like the way to go.

(For the record, though, I wouldn't describe `case _` as a special case of `default`. They do exactly the same thing, and `_` is a useful pattern in other contexts, so if anything the current `default` should be thought of as syntactic sugar for `case _`.)

Can case _ be mixed with unknown case? How can we match all compile time known cases but exclude future cases? Should be something like `case *` that would capture all currently known cases during compile time? case * and case _ would be the same in exhaustive enums.

···

On Jan 4, 2018, at 10:49 AM, Jordan Rose <jordan_rose@apple.com> wrote:

I'll add these points to the "Alternatives Considered" section in the PR later today.

Jordan

On Jan 3, 2018, at 22:56, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

As has already been said, “case unknown” is source-breaking because it conflicts with any real cases named “unknown”; “\unknown” looks like a key path but isn’t, and I wonder if it would potentially conflict with existing key paths.

In any case, my point was not to bikeshed the “unknown” part, but to ask whether any consideration had been made to have the feature presented as a flavor of default instead of a flavor of case.

On Wed, Jan 3, 2018 at 23:57 Cheyo Jimenez <cheyo@masters3d.com> wrote:

On Jan 3, 2018, at 6:52 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:

This is a very nice revision. One bikeshedding thought:

Since "unknown case" is presented as a special kind of "default", can't be mixed with "default", and can't be used in case patterns, why not "default unknown" (or "unknown default") instead of "unknown case"?

`case _ :` is already a special case of default.
I’d rather have `case unknown :`
`unknown case :` is weird because of the order of `case`.

Another alternative is `case \unknown :`
`\unknown` would also allow pattern matching.

On Wed, Jan 3, 2018 at 8:05 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org> wrote:

On Jan 2, 2018, at 18:07, Jordan Rose <jordan_rose@apple.com> wrote:

[Proposal: https://github.com/apple/swift-evolution/blob/master/proposals/0192-non-exhaustive-enums.md\]

Whew! Thanks for your feedback, everyone. On the lighter side of feedback—naming things—it seems that most people seem to like '@frozen', and that does in fact have the connotations we want it to have. I like it too.

More seriously, this discussion has convinced me that it's worth including what the proposal discusses as a 'future' case. The key point that swayed me is that this can produce a warning when the switch is missing a case rather than an error, which both provides the necessary compiler feedback to update your code and allows your dependencies to continue compiling when you update to a newer SDK. I know people on both sides won't be 100% satisfied with this, but does it seem like a reasonable compromise?

The next question is how to spell it. I'm leaning towards `unexpected case:`, which (a) is backwards-compatible, and (b) also handles "private cases", either the fake kind that you can do in C (as described in the proposal), or some real feature we might add to Swift some day. `unknown case:` isn't bad either.

I too would like to just do `unknown:` or `unexpected:` but that's technically a source-breaking change:

switch foo {
case bar:
  unknown:
  while baz() {
    while garply() {
      if quux() {
        break unknown
      }
    }
  }
}

Another downside of the `unexpected case:` spelling is that it doesn't work as part of a larger pattern. I don't have a good answer for that one, but perhaps it's acceptable for now.

I'll write up a revision of the proposal soon and make sure the core team gets my recommendation when they discuss the results of the review.

---

I'll respond to a few of the more intricate discussions tomorrow, including the syntax of putting a new declaration inside the enum rather than outside. Thank you again, everyone, and happy new year!

I ended up doing these in the opposite order, writing up the new proposal first and not yet responding to the discussion that's further out. You can read my revisions at https://github.com/apple/swift-evolution/pull/777\.

In particular, I want to at least address:
- Dave D and Drew C's points about versioned libraries / linking semantics of modules.
- Jason M's point about migration
and I'll do one more pass over the thread to see if there's anything else I didn't address directly. (That doesn't mean everyone who disagrees, just messages where I think there's more I can do to explain why the proposal is the way it is.)

Jordan

P.S. Enjoying the Disney references. Thanks, Nevin and Dave. :-)

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Hello all,

I think this whole thing has been unnecessarily convoluted. As a result,
the majority of the replies are rabbit holes.

In my opinion, the true root of the concept in question is as follows:

*A list of something is desired:*
1 - Pancake
2 - Waffle
3 - Juice

*Developer wishes to be able to:*
*A)* Add new things to the list of choices in the future as they come up
with new ideas
*B)* Sometimes select one of the choices to be chosen as the normal
choice if no choice is made by the user

A and B are *separate desires*. In some circumstances a developer may
want to add a new choice and make it the normal choice when there was no
normal choice was clarified before.

I don't think this is an accurate summary of the problem being tackled
here. Rather, we are how to enable the vendor of a nonexhaustive enum to
add new cases without breaking binaries compiled against previous versions.
There is little here to do with what a "default" should be. Indeed, it is
an explicit design decision of Swift not to support types having an
implicit default value.

There is no way a library developer of libraries bundled in the app can
break the app by adding new cases. They may cause compiler issues when the
app author tries to update the library, but it will not break existing apps.

The concern for updating enums is mostly an Apple / OS related concern for
libraries/dynamic frameworks the app does not ship with, but links to at
runtime and we should have an exception for that. We should not use the
same solution for both and lose exhaustiveness checks when we do not need
to. It would be wrong.

Right, this proposal is about enabling ABI stability and is about libraries
that don't ship with the app.

However, I disagree strongly with your point above. There should not be
dialects of Swift depending on how you link a framework. The point made
above is salient that there are, semantically, certain enums that are
exhaustive (for example, Optional, which can only have two cases), and
others that are nonexhaustive (for example, a list of foods, which will
never be complete).

Those dynamic frameworks should be the one that have to opt-in (even better

···

On Tue, Jan 2, 2018 at 1:27 PM, Goffredo Marocchi <panajev@gmail.com> wrote:

On 2 Jan 2018, at 18:36, Xiaodi Wu via swift-evolution < > swift-evolution@swift.org> wrote:
On Tue, Jan 2, 2018 at 12:11 PM, Jason Merchant via swift-evolution < > swift-evolution@swift.org> wrote:
if it is done automatically for them) in non-exhaustive extra resilient
behaviour, not libraries you ship in the app.

The app developer should be informed and have to address the new cases or
the removal of old cases.

____________________

*Part 2:*

After this simple desire is clear, there should be two discussions:
*A)* In a text only coding language, what would we like the syntax to
look like? (Without regard to past-bias. What should it really be, forget
what mistaken design choices were made in Swift in the past)
*B)* How do we approach making this happen behind the scenes?

*Bonus:* Given that some of us have changed our approach to programming
significantly beyond text based coding, and into more dynamic mediums of
programming in other niches, and even here and there in Xcode - I would
recommend considering how the IDE would show a modern version of this
concept. I feel too often that Swift design syntax has a *lack of
awareness between the distinctions of what the IDE should do, as opposed to
what the syntax of the language should be*, and what should be handled
behind the scenes by automated tooling.

_____________________

*My opinion*, in answering the above questions is in preference to a
simple easy to read and write syntax, something like the following:

choices Breakfast {
    Pancake, *Waffle*, Juice
}

If a "default" choice is desired, it is obvious to me that I would select
the choice from the IDE, and it would be visually indicated that it was the
default.

When changes occur, whether new choices are added, old ones are removed
or changed, or a default is added, changed, or removed - a behind the
scenes automated tool analyzes the changes and presents migration options
through the IDE.

_____________________

Sincerely,
Jason

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I think there are a couple of different definitions running around, and that is confusing things.

In my mind, ‘unexpected:’ catches only cases which were unknown at compile time. Adding cases to an enum *should* be a source-breaking change. That is the whole point of this. We should have to update the switch (either by handling new case explicitly, or by adding default) before successfully compiling. What ‘unexpected:’ protects against are changes to a linked binary (e.g. iOS) that are now vending cases we didn’t know about when we were compiled.

I’ll say it again… framing this idea as one of exhaustiveness is really confusing. Enums should just always be exhaustive in swift. There may be cases where we need to use ‘unexpected:’ to handle unexpected/future cases exhaustively. If we annotate an enum as @frozen, then we won’t need to do that to be exhaustive because we know it won’t change out from under us. Always exhaustive. Much less confusing…

Thanks,
Jon

···

On Jan 2, 2018, at 1:41 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:

On Tue, Jan 2, 2018 at 3:27 PM, Kevin Nattinger <swift@nattinger.net <mailto:swift@nattinger.net>> wrote:
[...]

in what other circumstances do we insist that the compiler inform the end user about future additions to the API at compile time?

This isn’t a request for the compiler to inform the user about future additions to an API. It is a request to validate the compiler’s knowledge of the current state of an API with the current state of the source code.

Well, it's of course impossible to inform the user about future additions, so that's poorly phrased on my part. It's about the compiler informing the end user about *new* additions, part of the *current* state of the API, that have cropped up since the user last revised the code when the API was in a *previous* state (or, indistinguishably, members of which a user is unaware regardless of the temporal sequence of when such members were added). In what other circumstances do we insist that the compiler perform this service?

Enums. That's literally how they work today. You are arguing in favor of actively removing compiler-aided correctness.

There's also protocol requirements

No, that's now how enums work today, and it's not how protocol requirements work today. Enums today are all semantically exhaustive; if a case is added in a later version of a library, it's semantically a *different* enum type that happens to share the same name. Not considering all the cases of an exhaustive enum is an _error_, not a _warning_, because there is no basis on which to proceed. This will not change with the proposal. Likewise, adding a protocol requirement without a default implementation is source-breaking. The result is a compiler _error_.

The question is, what non-source breaking API additions today cause the compiler to inform the end user of such additions? The answer is: none whatsoever. Not new methods or properties on a type, not even new protocol requirements that have a default implementation.

and, arguably, deprecated methods with a proper message ("use foo instead").

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I think there are a couple of different definitions running around, and
that is confusing things.

In my mind, ‘unexpected:’ catches only cases which were unknown at compile
time. Adding cases to an enum *should* be a source-breaking change. That is
the whole point of this. We should have to update the switch (either by
handling new case explicitly, or by adding default) before successfully
compiling. What ‘unexpected:’ protects against are changes to a linked
binary (e.g. iOS) that are now vending cases we didn’t know about when we
were compiled.

I’ll say it again… framing this idea as one of exhaustiveness is really
confusing. Enums should just always be exhaustive in swift. There may be
cases where we need to use ‘unexpected:’ to handle unexpected/future cases
exhaustively. If we annotate an enum as @frozen, then we won’t need to do
that to be exhaustive because we know it won’t change out from under us.
Always exhaustive. Much less confusing…

Thanks,
Jon

I think, then, you fundamentally disagree with the starting premise of the
proposal, which is specifically the addition of nonexhaustive enums to the
language, and making them the default sort of enum so that adding cases *is
not* a source-breaking change. If your whole purpose is to change the
proposal so that adding cases will _always_ be a source-breaking change and
enums are _never_ nonexhaustive, then I'm not sure how to proceed in the
discussion as we're working towards diametrically opposite goals.

···

On Tue, Jan 2, 2018 at 4:31 PM, Jonathan Hull <jhull@gbis.com> wrote:

On Jan 2, 2018, at 1:41 PM, Xiaodi Wu via swift-evolution < > swift-evolution@swift.org> wrote:

On Tue, Jan 2, 2018 at 3:27 PM, Kevin Nattinger <swift@nattinger.net> > wrote:

[...]

in what other circumstances do we insist that the compiler inform the end

user about future additions to the API at compile time?

This isn’t a request for the compiler to inform the user about future
additions to an API. It is a request to validate the compiler’s knowledge
of the *current* state of an API with the *current* state of the source
code.

Well, it's of course impossible to inform the user about future
additions, so that's poorly phrased on my part. It's about the compiler
informing the end user about *new* additions, part of the *current* state
of the API, that have cropped up since the user last revised the code when
the API was in a *previous* state (or, indistinguishably, members of which
a user is unaware regardless of the temporal sequence of when such members
were added). In what other circumstances do we insist that the compiler
perform this service?

Enums. That's literally how they work today. You are arguing in favor of
actively removing compiler-aided correctness.

There's also protocol requirements

No, that's now how enums work today, and it's not how protocol
requirements work today. Enums today are all semantically exhaustive; if a
case is added in a later version of a library, it's semantically a
*different* enum type that happens to share the same name. Not considering
all the cases of an exhaustive enum is an _error_, not a _warning_, because
there is no basis on which to proceed. This will not change with the
proposal. Likewise, adding a protocol requirement without a default
implementation is source-breaking. The result is a compiler _error_.

The question is, what non-source breaking API additions today cause the
compiler to inform the end user of such additions? The answer is: none
whatsoever. Not new methods or properties on a type, not even new protocol
requirements that have a default implementation.

and, arguably, deprecated methods with a proper message ("use foo

instead").

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[...]

in what other circumstances do we insist that the compiler inform the end user about future additions to the API at compile time?

This isn’t a request for the compiler to inform the user about future additions to an API. It is a request to validate the compiler’s knowledge of the current state of an API with the current state of the source code.

Well, it's of course impossible to inform the user about future additions, so that's poorly phrased on my part. It's about the compiler informing the end user about *new* additions, part of the *current* state of the API, that have cropped up since the user last revised the code when the API was in a *previous* state (or, indistinguishably, members of which a user is unaware regardless of the temporal sequence of when such members were added). In what other circumstances do we insist that the compiler perform this service?

Enums. That's literally how they work today. You are arguing in favor of actively removing compiler-aided correctness.

There's also protocol requirements

No, that's now how enums work today, and it's not how protocol requirements work today. Enums today are all semantically exhaustive; if a case is added in a later version of a library, it's semantically a *different* enum type that happens to share the same name. Not considering all the cases of an exhaustive enum is an _error_, not a _warning_, because there is no basis on which to proceed. This will not change with the proposal. Likewise, adding a protocol requirement without a default implementation is source-breaking. The result is a compiler _error_.

The question is, what non-source breaking API additions today cause the compiler to inform the end user of such additions?

Posing the question this way takes it as a given that adding a case to a resilient enum is non-source breaking with a full stop. The position of everyone asking for something like `future` / `unknown` as an alternative to `default` is exactly that this should not be the case. Instead, adding a case should always be binary compatible and should be source compatible by default, but authors should have the ability to opt-in to making case additions be source-breaking for individual switch statements.

When you view it this way we are not asking the compiler to inform us of a non-source breaking addition. We are asking the compiler to treat an addition as source breaking in a specific context.

···

On Jan 2, 2018, at 3:41 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Tue, Jan 2, 2018 at 3:27 PM, Kevin Nattinger <swift@nattinger.net <mailto:swift@nattinger.net>> wrote:

The answer is: none whatsoever. Not new methods or properties on a type, not even new protocol requirements that have a default implementation.

and, arguably, deprecated methods with a proper message ("use foo instead").

Despite having suggested the @version syntax I do agree with people that have said any @annotation is both a burden on the programmer and severely reduces readability.

Therefore a 4th option, carrying on from the numbering in my previous email is:

  4. The compiler issues a description of the public enums in a module. Application code that uses the enums knows what version of the enum they were compiled against. When the application launches the runtime writes a translation between the module enum and the application enum and vice versa. When an enum goes into or comes out of the application code to/from the module code it goes through the translation functions. EG:

    // Module version 1.
    public enum E {
        case A, B, C
    }
    ...
    switch e {
    case A: a()
    default: d() // Could alternatively have cases B and C instead of default.
    unknown case: u() // Must have unknown case for a public enum.
    }

    // Application code compiled against module version 1.
    // Stored in the code is the description of E having cases A, B, and C.
    switch e {
    case A: a()
    default: d() // Could alternatively have cases B and C instead of default.
    unknown case: u() // Must have unknown case for a public enum.
    }

The translation functions in each direction are trivial since the two enums match and therefore have the same numbering and the same number range.

Now the module is changed.

    // Module version 2.
    public enum E {
        case A, C, D // B deleted and D added.
    }
    ...
    // switch as before but now D calls d() and B calls u(). Previously B called d() but since B is deleted it is now an unknown case.

The translation between module 2 code and application compiled against 1 code is:

  Module 2 <—> Application 1
  A <—> A
  U <— B
  C <—> C
  D —> U

IE all the old cases, from the application, are translated into unknown on the module side and all the new cases, from the module, are translated to unknown on the application side.

This way neither the module programmer nor the application programmer has to version their code since the runtime provides the translation.

It is source breaking however, since all switches of public enums require an unknown case both in the module code and application code.

-- Howard.

···

On 3 Jan 2018, at 3:54 am, Jason Merchant via swift-evolution <swift-evolution@swift.org> wrote:

Is it hard to imagine that most everyone can get what they want and keep the syntax clean and streamlined at the same time? Without any "@" signs or other compiler hints?

"Rather, we are how to enable the vendor of a nonexhaustive enum to add new cases without breaking binaries compiled against previous versions"

When an enum changes, and the change causes the code to break, the user can be presented with migration options from an automated IDE tool. In what specific way does this not solve the issue about having to upgrade your code when using someone else's code library? This very notion implies your disgruntled about doing work when things are upgraded, is that really what this fuss is all about?

A well written language interpreter and auto-tooling IDE would not need hints embedded in the code syntax itself. Migration hints from version to version should not be a part of either the past or future version of the code library.

...

I don't expect the community to agree on language grammar, but the common sense here on how to achieve the intended goals seems to be out of wack.

If someone can present a clear logical statement as to how an automated migration tool behind the scenes in the IDE to handle all your versioning worries, does not make this whole discussion about adding more convoluted syntax additions irrelevant, I'd love to hear it.

___________________

Sincerely,
Jason

On Tue, Jan 2, 2018 at 12:36 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Tue, Jan 2, 2018 at 12:11 PM, Jason Merchant via swift-evolution <swift-evolution@swift.org> wrote:

I think this whole thing has been unnecessarily convoluted. As a result, the majority of the replies are rabbit holes.

In my opinion, the true root of the concept in question is as follows:

A list of something is desired:
1 - Pancake
2 - Waffle
3 - Juice

Developer wishes to be able to:
A) Add new things to the list of choices in the future as they come up with new ideas
B) Sometimes select one of the choices to be chosen as the normal choice if no choice is made by the user

A and B are separate desires. In some circumstances a developer may want to add a new choice and make it the normal choice when there was no normal choice was clarified before.

I don't think this is an accurate summary of the problem being tackled here. Rather, we are how to enable the vendor of a nonexhaustive enum to add new cases without breaking binaries compiled against previous versions. There is little here to do with what a "default" should be. Indeed, it is an explicit design decision of Swift not to support types having an implicit default value.

____________________

Part 2:

After this simple desire is clear, there should be two discussions:
A) In a text only coding language, what would we like the syntax to look like? (Without regard to past-bias. What should it really be, forget what mistaken design choices were made in Swift in the past)
B) How do we approach making this happen behind the scenes?

Bonus: Given that some of us have changed our approach to programming significantly beyond text based coding, and into more dynamic mediums of programming in other niches, and even here and there in Xcode - I would recommend considering how the IDE would show a modern version of this concept. I feel too often that Swift design syntax has a lack of awareness between the distinctions of what the IDE should do, as opposed to what the syntax of the language should be, and what should be handled behind the scenes by automated tooling.

_____________________

My opinion, in answering the above questions is in preference to a simple easy to read and write syntax, something like the following:

choices Breakfast {
    Pancake, Waffle, Juice
}

If a "default" choice is desired, it is obvious to me that I would select the choice from the IDE, and it would be visually indicated that it was the default.

When changes occur, whether new choices are added, old ones are removed or changed, or a default is added, changed, or removed - a behind the scenes automated tool analyzes the changes and presents migration options through the IDE.

_____________________

Sincerely,
Jason

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I thought about this, but I couldn’t think of a use case for it. You won’t get any exhaustivity checks, and if you recompile the app you silently get different behavior. That didn’t seem like a good idea to me.

Jordan

···

On Jan 4, 2018, at 11:45, Cheyo Jimenez <cheyo@masters3d.com> wrote:

On Jan 4, 2018, at 10:49 AM, Jordan Rose <jordan_rose@apple.com <mailto:jordan_rose@apple.com>> wrote:

I'll admit I hadn't thought of using "unknown default" (or "default unknown"). I don't think that's terrible, but I mildly prefer `unknown case` because it builds on the "pun" that enum elements are also defined using 'case'. If anything hits this part of the switch, it really will be an "unknown case", i.e. a statically-unknown enum element.

To Cheyo's point, if this were to be a single token I'd probably spell it #unknown, like #available. Then we'd have `case #unknown:` and something that naturally expands to other pattern positions. I found that less aesthetically pleasing, though, and so a context-sensitive keyword seemed like the way to go.

(For the record, though, I wouldn't describe `case _` as a special case of `default`. They do exactly the same thing, and `_` is a useful pattern in other contexts, so if anything the current `default` should be thought of as syntactic sugar for `case _`.)

Can case _ be mixed with unknown case? How can we match all compile time known cases but exclude future cases? Should be something like `case *` that would capture all currently known cases during compile time? case * and case _ would be the same in exhaustive enums.

I'll admit I hadn't thought of using "unknown default" (or "default
unknown"). I don't think that's terrible, but I mildly prefer `unknown
case` because it builds on the "pun" that enum elements are also defined
using 'case'. If anything hits this part of the switch, it really will be
an "unknown case", i.e. a statically-unknown enum element.

To Cheyo's point, if this *were* to be a single token I'd probably spell
it #unknown, like #available. Then we'd have `case #unknown:` and something
that naturally expands to other pattern positions. I found that less
aesthetically pleasing, though, and so a context-sensitive keyword seemed
like the way to go.

(For the record, though, I wouldn't describe `case _` as a special case of
`default`. They do exactly the same thing, and `_` is a useful pattern in
other contexts, so if anything the current `default` should be thought of
as syntactic sugar for `case _`.)

Can case _ be mixed with unknown case? How can we match all compile time
known cases but exclude future cases?

What’s your use case for that? That eliminates the possibility of “unknown
case” giving you compile-time warnings for subsequently added cases, which
was the entire purpose of adding the syntax in the first place.

Should be something like `case *` that would capture all currently known

···

On Thu, Jan 4, 2018 at 13:46 Cheyo Jimenez <cheyo@masters3d.com> wrote:

On Jan 4, 2018, at 10:49 AM, Jordan Rose <jordan_rose@apple.com> wrote:
cases during compile time? case * and case _ would be the same in
exhaustive enums.

I'll add these points to the "Alternatives Considered" section in the PR
later today.

Jordan

On Jan 3, 2018, at 22:56, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

As has already been said, “case unknown” is source-breaking because it
conflicts with any real cases named “unknown”; “\unknown” looks like a key
path but isn’t, and I wonder if it would potentially conflict with existing
key paths.

In any case, my point was not to bikeshed the “unknown” part, but to ask
whether any consideration had been made to have the feature presented as a
flavor of default instead of a flavor of case.

On Wed, Jan 3, 2018 at 23:57 Cheyo Jimenez <cheyo@masters3d.com> wrote:

On Jan 3, 2018, at 6:52 PM, Xiaodi Wu via swift-evolution < >> swift-evolution@swift.org> wrote:

This is a very nice revision. One bikeshedding thought:

Since "unknown case" is presented as a special kind of "default", can't
be mixed with "default", and can't be used in case patterns, why not
"default unknown" (or "unknown default") instead of "unknown case"?

`case _ :` is already a special case of default.
I’d rather have `case unknown :`
`unknown case :` is weird because of the order of `case`.

Another alternative is `case \unknown :`
`\unknown` would also allow pattern matching.

On Wed, Jan 3, 2018 at 8:05 PM, Jordan Rose via swift-evolution < >> swift-evolution@swift.org> wrote:

On Jan 2, 2018, at 18:07, Jordan Rose <jordan_rose@apple.com> wrote:

[Proposal:
https://github.com/apple/swift-evolution/blob/master/proposals/0192-non-exhaustive-enums.md
]

Whew! Thanks for your feedback, everyone. On the lighter side of
feedback—naming things—it seems that most people seem to like '*@frozen*',
and that does in fact have the connotations we want it to have. I like it
too.

More seriously, this discussion has convinced me that it's worth
including what the proposal discusses as a *'future' case*. The key
point that swayed me is that this can produce a *warning* when the
switch is missing a case rather than an *error,* which both provides
the necessary compiler feedback to update your code and allows your
dependencies to continue compiling when you update to a newer SDK. I know
people on both sides won't be 100% satisfied with this, but does it seem
like a reasonable compromise?

The next question is how to spell it. I'm leaning towards `unexpected
case:`, which (a) is backwards-compatible, and (b) also handles "private
cases", either the fake kind that you can do in C (as described in the
proposal), or some real feature we might add to Swift some day. `unknown
case:` isn't bad either.

I too would like to just do `unknown:` or `unexpected:` but that's
technically a source-breaking change:

switch foo {
case bar:
  unknown:
  while baz() {
    while garply() {
      if quux() {
        break unknown
      }
    }
  }
}

Another downside of the `unexpected case:` spelling is that it doesn't
work as part of a larger pattern. I don't have a good answer for that one,
but perhaps it's acceptable for now.

I'll write up a revision of the proposal soon and make sure the core
team gets my recommendation when they discuss the results of the review.

---

I'll respond to a few of the more intricate discussions tomorrow,
including the syntax of putting a new declaration inside the enum rather
than outside. Thank you again, everyone, and happy new year!

I ended up doing these in the opposite order, writing up the new
proposal first and not yet responding to the discussion that's further out.
You can read my revisions at
https://github.com/apple/swift-evolution/pull/777\.

In particular, I want to at least address:
- Dave D and Drew C's points about versioned libraries / linking
semantics of modules.
- Jason M's point about migration
and I'll do one more pass over the thread to see if there's anything
else I didn't address directly. (That doesn't mean everyone who disagrees,
just messages where I think there's more I can do to explain why the
proposal is the way it is.)

Jordan

P.S. Enjoying the Disney references. Thanks, Nevin and Dave. :-)

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Is it hard to imagine that most everyone can get what they want and keep the syntax clean and streamlined at the same time? Without any "@" signs or other compiler hints?

For what it's worth, the original version of the proposal started with a modifier (a context-sensitive keyword, like 'final'), but the core team felt that there were a lot of modifiers in the language already, and this didn't meet the bar.

"Rather, we are how to enable the vendor of a nonexhaustive enum to add new cases without breaking binaries compiled against previous versions"

When an enum changes, and the change causes the code to break, the user can be presented with migration options from an automated IDE tool. In what specific way does this not solve the issue about having to upgrade your code when using someone else's code library? This very notion implies your disgruntled about doing work when things are upgraded, is that really what this fuss is all about?

A well written language interpreter and auto-tooling IDE would not need hints embedded in the code syntax itself. Migration hints from version to version should not be a part of either the past or future version of the code library.

Thanks for bringing this up! Unfortunately, it falls down in practice, because if there's a new enum case, it's unclear what you want to do with it. If you're handling errors, it's not obvious that the way you've handled any of the other errors is appropriate. In the (admittedly controversial) SKPaymentTransactionState case, none of the existing code would be appropriate to handle the newly-introduced "deferred" case, and nor could StoreKit provide "template" code that would be appropriate to the client app.

In any case, though, the key point on this particular quoted sentence is "without breaking binaries". Any such change must be valid without recompilation, and indeed without any intervention from the developer or an IDE, because that's what happens when the user updates their OS.

Jordan

···

On Jan 3, 2018, at 00:54, Jason Merchant via swift-evolution <swift-evolution@swift.org> wrote:

...

I don't expect the community to agree on language grammar, but the common sense here on how to achieve the intended goals seems to be out of wack.

If someone can present a clear logical statement as to how an automated migration tool behind the scenes in the IDE to handle all your versioning worries, does not make this whole discussion about adding more convoluted syntax additions irrelevant, I'd love to hear it.

___________________

Sincerely,
Jason

On Tue, Jan 2, 2018 at 12:36 PM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:
On Tue, Jan 2, 2018 at 12:11 PM, Jason Merchant via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
I think this whole thing has been unnecessarily convoluted. As a result, the majority of the replies are rabbit holes.

In my opinion, the true root of the concept in question is as follows:

A list of something is desired:
1 - Pancake
2 - Waffle
3 - Juice

Developer wishes to be able to:
A) Add new things to the list of choices in the future as they come up with new ideas
B) Sometimes select one of the choices to be chosen as the normal choice if no choice is made by the user

A and B are separate desires. In some circumstances a developer may want to add a new choice and make it the normal choice when there was no normal choice was clarified before.

I don't think this is an accurate summary of the problem being tackled here. Rather, we are how to enable the vendor of a nonexhaustive enum to add new cases without breaking binaries compiled against previous versions. There is little here to do with what a "default" should be. Indeed, it is an explicit design decision of Swift not to support types having an implicit default value.

____________________

Part 2:

After this simple desire is clear, there should be two discussions:
A) In a text only coding language, what would we like the syntax to look like? (Without regard to past-bias. What should it really be, forget what mistaken design choices were made in Swift in the past)
B) How do we approach making this happen behind the scenes?

Bonus: Given that some of us have changed our approach to programming significantly beyond text based coding, and into more dynamic mediums of programming in other niches, and even here and there in Xcode - I would recommend considering how the IDE would show a modern version of this concept. I feel too often that Swift design syntax has a lack of awareness between the distinctions of what the IDE should do, as opposed to what the syntax of the language should be, and what should be handled behind the scenes by automated tooling.

_____________________

My opinion, in answering the above questions is in preference to a simple easy to read and write syntax, something like the following:

choices Breakfast {
    Pancake, Waffle, Juice
}

If a "default" choice is desired, it is obvious to me that I would select the choice from the IDE, and it would be visually indicated that it was the default.

When changes occur, whether new choices are added, old ones are removed or changed, or a default is added, changed, or removed - a behind the scenes automated tool analyzes the changes and presents migration options through the IDE.

_____________________

Sincerely,
Jason

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[...]

in what other circumstances do we insist that the compiler inform the end

user about future additions to the API at compile time?

This isn’t a request for the compiler to inform the user about future
additions to an API. It is a request to validate the compiler’s knowledge
of the *current* state of an API with the *current* state of the source
code.

Well, it's of course impossible to inform the user about future
additions, so that's poorly phrased on my part. It's about the compiler
informing the end user about *new* additions, part of the *current* state
of the API, that have cropped up since the user last revised the code when
the API was in a *previous* state (or, indistinguishably, members of which
a user is unaware regardless of the temporal sequence of when such members
were added). In what other circumstances do we insist that the compiler
perform this service?

Enums. That's literally how they work today. You are arguing in favor of
actively removing compiler-aided correctness.

There's also protocol requirements

No, that's now how enums work today, and it's not how protocol
requirements work today. Enums today are all semantically exhaustive; if a
case is added in a later version of a library, it's semantically a
*different* enum type that happens to share the same name. Not considering
all the cases of an exhaustive enum is an _error_, not a _warning_, because
there is no basis on which to proceed. This will not change with the
proposal. Likewise, adding a protocol requirement without a default
implementation is source-breaking. The result is a compiler _error_.

The question is, what non-source breaking API additions today cause the
compiler to inform the end user of such additions?

Posing the question this way takes it as a given that adding a case to a
resilient enum is non-source breaking with a full stop.

Ah, yes. As I wrote in an earlier email in reply to Jon and others, it
seems we have a fundamental difference here. I take it as a given, full
stop, that this is what we are setting out to do. If it is not, then we are
trying to design with different end goals in mind.

The position of everyone asking for something like `future` / `unknown` as

···

On Tue, Jan 2, 2018 at 8:25 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Jan 2, 2018, at 3:41 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Tue, Jan 2, 2018 at 3:27 PM, Kevin Nattinger <swift@nattinger.net> > wrote:
an alternative to `default` is exactly that this should not be the case.
Instead, adding a case should always be binary compatible and should be
source compatible by default, but authors should have the ability to opt-in
to making case additions be source-breaking for individual switch
statements.

When you view it this way we are not asking the compiler to inform us of a
non-source breaking addition. We are asking the compiler to treat an
addition as source breaking in a specific context.

The answer is: none whatsoever. Not new methods or properties on a type,
not even new protocol requirements that have a default implementation.

and, arguably, deprecated methods with a proper message ("use foo

instead").