SE-0192 — Non-Exhaustive Enums

[anandabits (Matthew Johnson)]

The review of "SE 0192 - Non-Exhaustive Enums" begins now and runs through January 3, 2018.

The proposal is available here:

https://github.com/apple/swift-evolution/blob/master/proposals/0192-non-exhaustive-enums.md+1, it needs to happen (and ASAP, since it _will_ introduce source-breaking changes one way or the other).

I think non-exhaustive is the correct default. However, does this not mean that, by default, enums will be boxed because the receiver doesn’t know their potential size?

It's not always boxing, but yes, there will be more indirection if the compiler can't see the contents of the enum. (More on that below.)

That would mean that the best transition path for multi-module Apps would be to make your enums @exhaustive, rather than adding “default” statements (which is unfortunate, because I imagine when this change hits, the way you’ll notice will be complaints about missing “default” statements).

Yep, that's going to be the recommendation. The current minimal-for-review implementation does not do this but I'd like to figure out how to improve that; at the very least it might be a sensible thing to do in the migrator.

I do have some thoughts about how we could ease the transition (for this and other resilience-related changes), but it’s best to leave that to a separate discussion.

The one thing I’m still not overly fond of is the name - I would like us to keep the set of resilience/optimisation related keywords to a minimum. “exhaustive” for enums feels an awful lot like “fixed_contents” for structs - couldn’t we come up with a single name which could be used for both? I don’t think anybody’s going to want to use “exhaustive” for structs.

The core team was very focused on this too, but I contend that "exhaustive" is not about optimization and really isn't even about "resilience" (i.e. the ability to evolve a library's API while preserving binary compatibility). It's a semantic feature of an enum, much like 'open' or 'final' is for classes, and it affects what a client can or can't do with an enum. For libaries compiled from source, it won't affect performance at all—the compiler still knows the full set of cases in the current version of the library even if the programmer is forced to consider future versions.

I'm working on the fixed-contents proposal now, though it won't be ready for a while, and the same thing applies there: for structs compiled from source, the compiler can still do all the same optimizations. It's only when the library has binary compatibility concerns that we need to use extra indirection, and then "fixed-contents" becomes important. (As currently designed, it doesn't affect what clients can do with the struct at all.) This means that I don't expect a "normal" package author to write "fixed-contents" at all (however it ends up being spelled), whereas "exhaustive" is a fairly normal thing to consider whenever you make an enum public.

I hope that convinces you that "fixed-contents" and "exhaustive" don't need to have the same name. I don't think anyone loves the particular name "exhaustive", but as you see in the "Alternatives considered" we didn't manage to come up with anything significantly better. If reviewers all prefer something else we'd consider changing it.

Thanks for responding!
Jordan

When you say “libraries compiled from source”, what do you mean?

- Other targets in your project
- Source packages built through SwiftPM / CocoaPods / Carthage / other

And I was being imprecise with the terminology, but also

- Libraries built by someone else but designed to be embedded into an app, so that there's no chance of a different version showing up at run-time.

As for whether its a resilience feature: actually it is completely a resilience feature. The effects on switching are only side-effects; really what “exhaustive” or “nonexhaustive” are saying is literally that cases may be added later. Even if we added private cases, you wouldn’t need to mark those enums as specially exhaustive or not; that would be implied. It’s an accommodation for things which don’t exist yet, so really, it is all about resilience IMO.

"Resilience", as an admittedly fuzzily-defined term in the Swift project, specifically refers to what changes can be made without breaking binary compatibility <https://github.com/apple/swift/blob/master/docs/Lexicon.rst&gt;\. It does not refer to every change you can make to a library. (For comparison, adding a field to a struct is not source-breaking in Swift. We would like to make it not ABI-breaking either; that proposal's coming soon.)

Anyway, as I see it, library authors in general ought to be happy about this:
+ Their libraries become safer by default, so they can make changes in the future without having to worry about breakage
+ It doesn’t affect your code inside of a module, so it only affects types they already explicitly marked “public”

That's the intent.

The only people who lose are multi-module App developers, because they are “library authors” who don’t need to care about evolution, and now need to add attributes to things they wouldn’t have to before, or suffer language and performance penalties. Their libraries become less reusable and not resilient-by-default.

For example, I have an App for which I wrote a cross-platform model framework in Swift. When I compile it as a framework inside my App, it is bundled there forever. However, I use the same code to build libraries for Linux, which I would like to ship in binary form to 3rd-parties. Am I supposed to litter my code with annotations to mark those types as final, just to make the App fast and convenient to code? What happens when I need to fix a bug and distribute an updated copy, this means the 3rd-parties need to recompile (which they won’t do…).

Typically, for such a problem, I would recommend using a static library instead. But we don’t have those, and anyway they’re not always the best thing these days. So that’s why I started a new thread about creating a “@static” import, so App developers can go back to all the conveniences they had before.

There won't be a perf penalty, but yes, I do expect multi-module apps to use 'exhaustive' on most of their enums, because they don't need the futureproofing. Maybe this should have been mentioned more explicitly in the proposal.

As a perhaps more long-term design note, I think modules ought to have the ability to version-lock themselves to one or more of their dependencies. They would still be required to obey access control as if they were outside those dependencies, but we would suppress some of the semantic consequences of being outside the module, such as the need to assume non-exhaustiveness by default.

That is, there would be two independent axes of library dependency: source vs. binary and version-compatible vs. version-locked:
  - a source dependency allows the compiler to take advantage of the implementation of public entities when generating code
  - a version-locked dependency allows the compiler to take advantage of the implementation of public entities when enforcing semantics

Apps would generally elect to primarily use version-locked source dependencies because they're just pulling down source libraries (e.g. from github) and are comfortable with updating their code if the library changes.

Source libraries on github would generally want to use version-compatible source dependencies because version-locking would put their clients in "library hell" if the locking didn't all agree.

Binary dependencies could reasonably use either.

This model aligns pretty well with what I would like to see. It prevents us from paying a penalty when we don’t need the benefits provided by a restriction.

Relating this back to the current proposal, would you expect an app to have the ability to switch over an enum provided by a version-locked dependency that is not annotated with @exhaustive without requiring a default clause?

Relating to @inlinable proposal also under review, would everything in a source dependency be automatically inlinable whether they were annotated as such or not (at least when version-locked)?

···

On Dec 21, 2017, at 1:26 PM, John McCall via swift-evolution <swift-evolution@swift.org> wrote:

On Dec 21, 2017, at 2:03 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Dec 20, 2017, at 12:35, Karl Wagner <razielim@gmail.com <mailto:razielim@gmail.com>> wrote:

On 20. Dec 2017, at 19:54, Jordan Rose <jordan_rose@apple.com <mailto:jordan_rose@apple.com>> wrote:

On Dec 20, 2017, at 05:36, Karl Wagner via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 19. Dec 2017, at 23:58, Ted Kremenek via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

John.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

[John_McCall]

The review of "SE 0192 - Non-Exhaustive Enums" begins now and runs through January 3, 2018.

The proposal is available here:

https://github.com/apple/swift-evolution/blob/master/proposals/0192-non-exhaustive-enums.md+1, it needs to happen (and ASAP, since it _will_ introduce source-breaking changes one way or the other).

I think non-exhaustive is the correct default. However, does this not mean that, by default, enums will be boxed because the receiver doesn’t know their potential size?

It's not always boxing, but yes, there will be more indirection if the compiler can't see the contents of the enum. (More on that below.)

That would mean that the best transition path for multi-module Apps would be to make your enums @exhaustive, rather than adding “default” statements (which is unfortunate, because I imagine when this change hits, the way you’ll notice will be complaints about missing “default” statements).

Yep, that's going to be the recommendation. The current minimal-for-review implementation does not do this but I'd like to figure out how to improve that; at the very least it might be a sensible thing to do in the migrator.

I do have some thoughts about how we could ease the transition (for this and other resilience-related changes), but it’s best to leave that to a separate discussion.

The one thing I’m still not overly fond of is the name - I would like us to keep the set of resilience/optimisation related keywords to a minimum. “exhaustive” for enums feels an awful lot like “fixed_contents” for structs - couldn’t we come up with a single name which could be used for both? I don’t think anybody’s going to want to use “exhaustive” for structs.

The core team was very focused on this too, but I contend that "exhaustive" is not about optimization and really isn't even about "resilience" (i.e. the ability to evolve a library's API while preserving binary compatibility). It's a semantic feature of an enum, much like 'open' or 'final' is for classes, and it affects what a client can or can't do with an enum. For libaries compiled from source, it won't affect performance at all—the compiler still knows the full set of cases in the current version of the library even if the programmer is forced to consider future versions.

I'm working on the fixed-contents proposal now, though it won't be ready for a while, and the same thing applies there: for structs compiled from source, the compiler can still do all the same optimizations. It's only when the library has binary compatibility concerns that we need to use extra indirection, and then "fixed-contents" becomes important. (As currently designed, it doesn't affect what clients can do with the struct at all.) This means that I don't expect a "normal" package author to write "fixed-contents" at all (however it ends up being spelled), whereas "exhaustive" is a fairly normal thing to consider whenever you make an enum public.

I hope that convinces you that "fixed-contents" and "exhaustive" don't need to have the same name. I don't think anyone loves the particular name "exhaustive", but as you see in the "Alternatives considered" we didn't manage to come up with anything significantly better. If reviewers all prefer something else we'd consider changing it.

Thanks for responding!
Jordan

When you say “libraries compiled from source”, what do you mean?

- Other targets in your project
- Source packages built through SwiftPM / CocoaPods / Carthage / other

And I was being imprecise with the terminology, but also

- Libraries built by someone else but designed to be embedded into an app, so that there's no chance of a different version showing up at run-time.

As for whether its a resilience feature: actually it is completely a resilience feature. The effects on switching are only side-effects; really what “exhaustive” or “nonexhaustive” are saying is literally that cases may be added later. Even if we added private cases, you wouldn’t need to mark those enums as specially exhaustive or not; that would be implied. It’s an accommodation for things which don’t exist yet, so really, it is all about resilience IMO.

"Resilience", as an admittedly fuzzily-defined term in the Swift project, specifically refers to what changes can be made without breaking binary compatibility <https://github.com/apple/swift/blob/master/docs/Lexicon.rst&gt;\. It does not refer to every change you can make to a library. (For comparison, adding a field to a struct is not source-breaking in Swift. We would like to make it not ABI-breaking either; that proposal's coming soon.)

Anyway, as I see it, library authors in general ought to be happy about this:
+ Their libraries become safer by default, so they can make changes in the future without having to worry about breakage
+ It doesn’t affect your code inside of a module, so it only affects types they already explicitly marked “public”

That's the intent.

The only people who lose are multi-module App developers, because they are “library authors” who don’t need to care about evolution, and now need to add attributes to things they wouldn’t have to before, or suffer language and performance penalties. Their libraries become less reusable and not resilient-by-default.

For example, I have an App for which I wrote a cross-platform model framework in Swift. When I compile it as a framework inside my App, it is bundled there forever. However, I use the same code to build libraries for Linux, which I would like to ship in binary form to 3rd-parties. Am I supposed to litter my code with annotations to mark those types as final, just to make the App fast and convenient to code? What happens when I need to fix a bug and distribute an updated copy, this means the 3rd-parties need to recompile (which they won’t do…).

Typically, for such a problem, I would recommend using a static library instead. But we don’t have those, and anyway they’re not always the best thing these days. So that’s why I started a new thread about creating a “@static” import, so App developers can go back to all the conveniences they had before.

There won't be a perf penalty, but yes, I do expect multi-module apps to use 'exhaustive' on most of their enums, because they don't need the futureproofing. Maybe this should have been mentioned more explicitly in the proposal.

As a perhaps more long-term design note, I think modules ought to have the ability to version-lock themselves to one or more of their dependencies. They would still be required to obey access control as if they were outside those dependencies, but we would suppress some of the semantic consequences of being outside the module, such as the need to assume non-exhaustiveness by default.

That is, there would be two independent axes of library dependency: source vs. binary and version-compatible vs. version-locked:
  - a source dependency allows the compiler to take advantage of the implementation of public entities when generating code
  - a version-locked dependency allows the compiler to take advantage of the implementation of public entities when enforcing semantics

Apps would generally elect to primarily use version-locked source dependencies because they're just pulling down source libraries (e.g. from github) and are comfortable with updating their code if the library changes.

Source libraries on github would generally want to use version-compatible source dependencies because version-locking would put their clients in "library hell" if the locking didn't all agree.

Binary dependencies could reasonably use either.

This model aligns pretty well with what I would like to see. It prevents us from paying a penalty when we don’t need the benefits provided by a restriction.

Relating this back to the current proposal, would you expect an app to have the ability to switch over an enum provided by a version-locked dependency that is not annotated with @exhaustive without requiring a default clause?

Yes, and as we find other places where program semantics depend on knowing the implementation, I would expect them to follow suit.

My guess is that enum exhaustiveness is probably more prominent than any other such feature, and maybe even more prominent than all of them put together, but possible examples include:
  - automatically deriving protocol conformances, which we hope will eventually be something you can do for an arbitrary protocol
  - any other kind of structural metaprogramming we might add
  - maybe memberwise struct initialization if there are no explicit initializers, although this is arguably an access control question (just as public/open is)
  - ownership-related features that might make sense to restrict to stored properties, like expecting a struct property to have a stable address, or destructuring a struct with pattern-matching

Now, some of these things might be nice to allow even for resilient types. I know Joe has suggested adding some way of resiliently describing a structural decomposition of a type, which you could then use to derive conformances, etc. But since the basic motivation for restricting any of them is source/binary compatibility, and since version-locking would tell us that the programmer doesn't care about that, it seems sensible that version-locking ought to suppress the restrictions.

Relating to @inlinable proposal also under review, would everything in a source dependency be automatically inlinable whether they were annotated as such or not (at least when version-locked)?

Yes, and regardless of being version-locked. Inlining isn't semantically visible: it's observable in various low-level ways, but it's not supposed to affect basic program semantics except in incidental ways, e.g. by lowering the memory requirements so that programs start working that didn't before. So the compiler is generally always allowed to inline when the call is direct and the callee has a known implementation; that's just standard "as if" behavior. The fact that we can't do this today is just an unfortunate consequence of our current build model.

John.

···

On Dec 21, 2017, at 2:41 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Dec 21, 2017, at 1:26 PM, John McCall via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Dec 21, 2017, at 2:03 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Dec 20, 2017, at 12:35, Karl Wagner <razielim@gmail.com <mailto:razielim@gmail.com>> wrote:

On 20. Dec 2017, at 19:54, Jordan Rose <jordan_rose@apple.com <mailto:jordan_rose@apple.com>> wrote:

On Dec 20, 2017, at 05:36, Karl Wagner via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 19. Dec 2017, at 23:58, Ted Kremenek via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

[Nevin]

Making enums non-exhaustive solves a problem for authors of precompiled
frameworks which apps can dynamically link against. These libraries need to
preserve binary compatibility, in the sense that apps should continue to
work with newer versions of the library.

Now, that creates a problem for app developers, wherein the compiler no
longer informs them when their switch statements don’t cover all known enum
cases. This can be solved by introducing “future” for switch statements,
which acts like “default” but will warn if it is provably reachable.

Perhaps the most common scenario, however, is developers who write
multi-module apps. Such apps could include modules with the same author,
modules included as source, and even precompiled binary modules. As long as
these are all bundled with the app, they cannot possibly change out from
under it at runtime, so binary-compatibility is not a concern.

I think it is important that any solution to the problems of dynamically
linked libraries should not adversely impact authors of multi-module apps.

It sounds like John McCall has an approach to handle this with version
locking of dependencies.

Another possibility is to say that public enums are non-exhaustive by
default if the module is built with resilience enabled, but if resilience
is not enabled then enums are exhaustive.

Alternatively, we could introduce a concept of “the entire thing I am
building, including all its statically-linked modules”, and let enums be
exhaustive within that domain while still defaulting to non-exhaustive
outside it.

I am glad to see vigorous debate and brainstorming on this thread, and I am
confident that we will find a good solution.

Nevin

···

On Thu, Dec 21, 2017 at 2:26 PM, John McCall via swift-evolution < swift-evolution@swift.org> wrote:

On Dec 21, 2017, at 2:03 PM, Jordan Rose via swift-evolution < > swift-evolution@swift.org> wrote:

On Dec 20, 2017, at 12:35, Karl Wagner <razielim@gmail.com> wrote:

On 20. Dec 2017, at 19:54, Jordan Rose <jordan_rose@apple.com> wrote:

On Dec 20, 2017, at 05:36, Karl Wagner via swift-evolution < > swift-evolution@swift.org> wrote:

On 19. Dec 2017, at 23:58, Ted Kremenek via swift-evolution < > swift-evolution@swift.org> wrote:

The review of "SE 0192 - Non-Exhaustive Enums" begins now and runs through *January
3, 2018*.

The proposal is available here:

GitHub - apple/swift-evolution: This maintains proposals for changes and user-visible enhancements to the Swift Programming Language.
proposals/0192-non-exhaustive-enums.md

+1, it needs to happen (and ASAP, since it _will_ introduce
source-breaking changes one way or the other).

I think non-exhaustive is the correct default. However, does this not mean
that, by default, enums will be boxed because the receiver doesn’t know
their potential size?

It's not always boxing, but yes, there will be more indirection if the
*compiler* can't see the contents of the enum. (More on that below.)

That would mean that the best transition path for multi-module Apps would
be to make your enums @exhaustive, rather than adding “default” statements
(which is unfortunate, because I imagine when this change hits, the way
you’ll notice will be complaints about missing “default” statements).

Yep, that's going to be the recommendation. The current minimal-for-review
implementation does not do this but I'd like to figure out how to improve
that; at the very least it might be a sensible thing to do in the migrator.

I do have some thoughts about how we could ease the transition (for this
and other resilience-related changes), but it’s best to leave that to a
separate discussion.

The one thing I’m still not overly fond of is the name - I would like us
to keep the set of resilience/optimisation related keywords to a minimum.
“exhaustive” for enums feels an awful lot like “fixed_contents” for structs
- couldn’t we come up with a single name which could be used for both? I
don’t think anybody’s going to want to use “exhaustive” for structs.

The core team was very focused on this too, but I contend that
"exhaustive" is not about optimization and really isn't even about
"resilience" (i.e. the ability to evolve a library's API while preserving
binary compatibility). It's a semantic feature of an enum, much like 'open'
or 'final' is for classes, and it affects what a client can or can't do
with an enum. For libaries compiled from source, it won't affect
performance at all—the *compiler* still knows the full set of cases in the
*current* version of the library even if the programmer is forced to
consider future versions.

I'm working on the fixed-contents proposal now, though it won't be ready
for a while, and the same thing applies there: for structs compiled from
source, the compiler can still do all the same optimizations. It's only
when the library has binary compatibility concerns that we need to use
extra indirection, and then "fixed-contents" becomes important. (As
currently designed, it doesn't affect what clients can do with the struct
at all.) This means that I don't expect a "normal" package author to write
"fixed-contents" at all (however it ends up being spelled), whereas
"exhaustive" is a fairly normal thing to consider whenever you make an enum
public.

I hope that convinces you that "fixed-contents" and "exhaustive" don't
need to have the same name. I don't think anyone loves the *particular* name
"exhaustive", but as you see in the "Alternatives considered" we didn't
manage to come up with anything significantly better. If reviewers all
prefer something else we'd consider changing it.

Thanks for responding!
Jordan

When you say “libraries compiled from source”, what do you mean?

- Other targets in your project
- Source packages built through SwiftPM / CocoaPods / Carthage / other

And I was being imprecise with the terminology, but also

- Libraries built by someone else but designed to be embedded into an app,
so that there's no chance of a different version showing up at run-time.

As for whether its a resilience feature: actually it is completely a
resilience feature. The effects on switching are only side-effects; really
what “exhaustive” or “nonexhaustive” are saying is literally that cases may
be added later. Even if we added private cases, you wouldn’t need to mark
those enums as specially exhaustive or not; that would be implied. It’s an
accommodation for things which don’t exist yet, so really, it is all about
resilience IMO.

"Resilience", as an admittedly fuzzily-defined term in the Swift project, specifically
refers to what changes can be made without breaking binary compatibility
<https://github.com/apple/swift/blob/master/docs/Lexicon.rst&gt;\. It does
not refer to *every* change you can make to a library. (For comparison,
adding a field to a struct is not source-breaking in Swift. We would like
to make it not ABI-breaking either; that proposal's coming soon.)

Anyway, as I see it, library authors in general ought to be happy about
this:
+ Their libraries become safer by default, so they can make changes in the
future without having to worry about breakage
+ It doesn’t affect your code inside of a module, so it only affects types
they already explicitly marked “public”

That's the intent.

The only people who lose are multi-module App developers, because they are
“library authors” who don’t need to care about evolution, and now need to
add attributes to things they wouldn’t have to before, or suffer language
and performance penalties. Their libraries become less reusable and not
resilient-by-default.

For example, I have an App for which I wrote a cross-platform model
framework in Swift. When I compile it as a framework inside my App, it is
bundled there forever. However, I use the same code to build libraries for
Linux, which I would like to ship in binary form to 3rd-parties. Am I
supposed to litter my code with annotations to mark those types as final,
just to make the App fast and convenient to code? What happens when I need
to fix a bug and distribute an updated copy, this means the 3rd-parties
need to recompile (which they won’t do…).

Typically, for such a problem, I would recommend using a static library
instead. But we don’t have those, and anyway they’re not always the best
thing these days. So that’s why I started a new thread about creating a
“@static” import, so App developers can go back to all the conveniences
they had before.

There won't be a perf penalty, but yes, I do expect multi-module apps to
use 'exhaustive' on most of their enums, because they don't need the
futureproofing. Maybe this should have been mentioned more explicitly in
the proposal.

As a perhaps more long-term design note, I think modules ought to have the
ability to version-lock themselves to one or more of their dependencies.
They would still be required to obey access control as if they were outside
those dependencies, but we would suppress some of the *semantic*
consequences of being outside the module, such as the need to assume
non-exhaustiveness by default.

That is, there would be two independent axes of library dependency: source
vs. binary and version-compatible vs. version-locked:
  - a source dependency allows the compiler to take advantage of the
implementation of public entities when generating code
  - a version-locked dependency allows the compiler to take advantage of
the implementation of public entities when enforcing semantics

Apps would generally elect to primarily use version-locked source
dependencies because they're just pulling down source libraries (e.g. from
github) and are comfortable with updating their code if the library changes.

Source libraries on github would generally want to use version-compatible
source dependencies because version-locking would put their clients in
"library hell" if the locking didn't all agree.

Binary dependencies could reasonably use either.

John.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[Charlie_Monroe1 (Charlie Monroe)]

I think that the main confusion here stems from the word library as we are addressing something that can be divided further (and this is IMHO as many macOS/iOS devs see it):

- libraries that come with the OS - here, it absolutely makes sense to make the enums non-exhaustive as the apps are linked against these libraries and the user installs a binary that will load these at launch and they are not bundled with the app - the developer can't control future OS releases and he wants the app to run on a future OS release.
- libraries that are bundled with the app - be it PM, CocoaPods or something else - you typically update your dependencies once in a while and they change. And you want to be notified by the compiler about possible changes - extended enums, in this case. Because let's be honest - if your app has a dozen dependencies and you come to the app after a year of no development, Swift 5 came along during that period, you want to update these libraries to Swift-5-compatible versions. And no one has the time to go through all change logs - even if they were kept up-to-date and thorough, which I can't say that I've seen in many instances.

I know that this is a limited view from the perspective of an app developer and that potentially, e.g. on Linux, there may be libraries written in Swift that you may want to install via package managers and depend on them once the ABI is stable, but the choice to make them non-exhaustive by default is not in line with everything else in Swift - everything else is generally closed by default - public (-> final in other modules), no access modified (-> internal), ...

For me, it's a -1 as it is now. I'd prefer exhaustive-by-default, ObjC/C-import non-exhaustive by default (the way ObjC classes are open by default vs. public). When it comes to the switch statement, there definitely needs to be an option to make an exhaustive switch over all compile-time-known values with a warning shall a new one be added. Without that, the code will become incredibly prone to errors and hard to maintain.

This does bring up another option, which is to differentiate Apple-provided libraries (and in general, libraries with binary compatibility concerns, i.e. libraries that may be different at run-time from what you compiled against) from bundled / built-from-source libraries. Slava and I have been very leery of this because it fragments the language into dialects, and has the potential to confuse people when they try to write code that behaves like an Apple framework, but I suppose it is an option.

I wouldn't differentiate, I would just try to think which should be the default. Which is more common? Which will be the more common scenario? I would dare say that vast majority (currently) falls under b) - and in these cases, it makes more sense to make the enum exhaustive by default.

···

On Dec 21, 2017, at 8:17 PM, Jordan Rose <jordan_rose@apple.com> wrote:

On Dec 20, 2017, at 12:54, Charlie Monroe <charlie@charliemonroe.net <mailto:charlie@charliemonroe.net>> wrote:

Jordan

On Dec 20, 2017, at 9:35 PM, Karl Wagner via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 20. Dec 2017, at 19:54, Jordan Rose <jordan_rose@apple.com <mailto:jordan_rose@apple.com>> wrote:

On Dec 20, 2017, at 05:36, Karl Wagner via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 19. Dec 2017, at 23:58, Ted Kremenek via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

The review of "SE 0192 - Non-Exhaustive Enums" begins now and runs through January 3, 2018.

The proposal is available here:

https://github.com/apple/swift-evolution/blob/master/proposals/0192-non-exhaustive-enums.md+1, it needs to happen (and ASAP, since it _will_ introduce source-breaking changes one way or the other).

I think non-exhaustive is the correct default. However, does this not mean that, by default, enums will be boxed because the receiver doesn’t know their potential size?

It's not always boxing, but yes, there will be more indirection if the compiler can't see the contents of the enum. (More on that below.)

That would mean that the best transition path for multi-module Apps would be to make your enums @exhaustive, rather than adding “default” statements (which is unfortunate, because I imagine when this change hits, the way you’ll notice will be complaints about missing “default” statements).

Yep, that's going to be the recommendation. The current minimal-for-review implementation does not do this but I'd like to figure out how to improve that; at the very least it might be a sensible thing to do in the migrator.

I do have some thoughts about how we could ease the transition (for this and other resilience-related changes), but it’s best to leave that to a separate discussion.

The one thing I’m still not overly fond of is the name - I would like us to keep the set of resilience/optimisation related keywords to a minimum. “exhaustive” for enums feels an awful lot like “fixed_contents” for structs - couldn’t we come up with a single name which could be used for both? I don’t think anybody’s going to want to use “exhaustive” for structs.

The core team was very focused on this too, but I contend that "exhaustive" is not about optimization and really isn't even about "resilience" (i.e. the ability to evolve a library's API while preserving binary compatibility). It's a semantic feature of an enum, much like 'open' or 'final' is for classes, and it affects what a client can or can't do with an enum. For libaries compiled from source, it won't affect performance at all—the compiler still knows the full set of cases in the current version of the library even if the programmer is forced to consider future versions.

I'm working on the fixed-contents proposal now, though it won't be ready for a while, and the same thing applies there: for structs compiled from source, the compiler can still do all the same optimizations. It's only when the library has binary compatibility concerns that we need to use extra indirection, and then "fixed-contents" becomes important. (As currently designed, it doesn't affect what clients can do with the struct at all.) This means that I don't expect a "normal" package author to write "fixed-contents" at all (however it ends up being spelled), whereas "exhaustive" is a fairly normal thing to consider whenever you make an enum public.

I hope that convinces you that "fixed-contents" and "exhaustive" don't need to have the same name. I don't think anyone loves the particular name "exhaustive", but as you see in the "Alternatives considered" we didn't manage to come up with anything significantly better. If reviewers all prefer something else we'd consider changing it.

Thanks for responding!
Jordan

When you say “libraries compiled from source”, what do you mean?

As for whether its a resilience feature: actually it is completely a resilience feature. The effects on switching are only side-effects; really what “exhaustive” or “nonexhaustive” are saying is literally that cases may be added later. Even if we added private cases, you wouldn’t need to mark those enums as specially exhaustive or not; that would be implied. It’s an accommodation for things which don’t exist yet, so really, it is all about resilience IMO.

Anyway, as I see it, library authors in general ought to be happy about this:
+ Their libraries become safer by default, so they can make changes in the future without having to worry about breakage
+ It doesn’t affect your code inside of a module, so it only affects types they already explicitly marked “public”

The only people who lose are multi-module App developers, because they are “library authors” who don’t need to care about evolution, and now need to add attributes to things they wouldn’t have to before, or suffer language and performance penalties. Their libraries become less reusable and not resilient-by-default.

For example, I have an App for which I wrote a cross-platform model framework in Swift. When I compile it as a framework inside my App, it is bundled there forever. However, I use the same code to build libraries for Linux, which I would like to ship in binary form to 3rd-parties. Am I supposed to litter my code with annotations to mark those types as final, just to make the App fast and convenient to code? What happens when I need to fix a bug and distribute an updated copy, this means the 3rd-parties need to recompile (which they won’t do…).

Typically, for such a problem, I would recommend using a static library instead. But we don’t have those, and anyway they’re not always the best thing these days. So that’s why I started a new thread about creating a “@static” import, so App developers can go back to all the conveniences they had before.

- Karl

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

[Rod_Brown]

Thanks for this response, John.

I think you’re right. This would be very helpful. It a) helps those who are source compiling with their app, and b) avoids fragmenting the language because it would be a core understanding of the code you are compiling, rather than creating a new “variant” of Swift for the binary compatible framework vs client code.

- Rod

···

On 22 Dec 2017, at 6:26 am, John McCall via swift-evolution <swift-evolution@swift.org> wrote:

On Dec 21, 2017, at 2:03 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org> wrote:

On Dec 20, 2017, at 12:35, Karl Wagner <razielim@gmail.com> wrote:

On 20. Dec 2017, at 19:54, Jordan Rose <jordan_rose@apple.com> wrote:

On Dec 20, 2017, at 05:36, Karl Wagner via swift-evolution <swift-evolution@swift.org> wrote:

On 19. Dec 2017, at 23:58, Ted Kremenek via swift-evolution <swift-evolution@swift.org> wrote:

The review of "SE 0192 - Non-Exhaustive Enums" begins now and runs through January 3, 2018.

The proposal is available here:

https://github.com/apple/swift-evolution/blob/master/proposals/0192-non-exhaustive-enums.md

+1, it needs to happen (and ASAP, since it _will_ introduce source-breaking changes one way or the other).

I think non-exhaustive is the correct default. However, does this not mean that, by default, enums will be boxed because the receiver doesn’t know their potential size?

It's not always boxing, but yes, there will be more indirection if the compiler can't see the contents of the enum. (More on that below.)

That would mean that the best transition path for multi-module Apps would be to make your enums @exhaustive, rather than adding “default” statements (which is unfortunate, because I imagine when this change hits, the way you’ll notice will be complaints about missing “default” statements).

Yep, that's going to be the recommendation. The current minimal-for-review implementation does not do this but I'd like to figure out how to improve that; at the very least it might be a sensible thing to do in the migrator.

I do have some thoughts about how we could ease the transition (for this and other resilience-related changes), but it’s best to leave that to a separate discussion.

The one thing I’m still not overly fond of is the name - I would like us to keep the set of resilience/optimisation related keywords to a minimum. “exhaustive” for enums feels an awful lot like “fixed_contents” for structs - couldn’t we come up with a single name which could be used for both? I don’t think anybody’s going to want to use “exhaustive” for structs.

The core team was very focused on this too, but I contend that "exhaustive" is not about optimization and really isn't even about "resilience" (i.e. the ability to evolve a library's API while preserving binary compatibility). It's a semantic feature of an enum, much like 'open' or 'final' is for classes, and it affects what a client can or can't do with an enum. For libaries compiled from source, it won't affect performance at all—the compiler still knows the full set of cases in the current version of the library even if the programmer is forced to consider future versions.

I'm working on the fixed-contents proposal now, though it won't be ready for a while, and the same thing applies there: for structs compiled from source, the compiler can still do all the same optimizations. It's only when the library has binary compatibility concerns that we need to use extra indirection, and then "fixed-contents" becomes important. (As currently designed, it doesn't affect what clients can do with the struct at all.) This means that I don't expect a "normal" package author to write "fixed-contents" at all (however it ends up being spelled), whereas "exhaustive" is a fairly normal thing to consider whenever you make an enum public.

I hope that convinces you that "fixed-contents" and "exhaustive" don't need to have the same name. I don't think anyone loves the particular name "exhaustive", but as you see in the "Alternatives considered" we didn't manage to come up with anything significantly better. If reviewers all prefer something else we'd consider changing it.

Thanks for responding!
Jordan

When you say “libraries compiled from source”, what do you mean?

- Other targets in your project
- Source packages built through SwiftPM / CocoaPods / Carthage / other

And I was being imprecise with the terminology, but also

- Libraries built by someone else but designed to be embedded into an app, so that there's no chance of a different version showing up at run-time.

As for whether its a resilience feature: actually it is completely a resilience feature. The effects on switching are only side-effects; really what “exhaustive” or “nonexhaustive” are saying is literally that cases may be added later. Even if we added private cases, you wouldn’t need to mark those enums as specially exhaustive or not; that would be implied. It’s an accommodation for things which don’t exist yet, so really, it is all about resilience IMO.

"Resilience", as an admittedly fuzzily-defined term in the Swift project, specifically refers to what changes can be made without breaking binary compatibility. It does not refer to every change you can make to a library. (For comparison, adding a field to a struct is not source-breaking in Swift. We would like to make it not ABI-breaking either; that proposal's coming soon.)

Anyway, as I see it, library authors in general ought to be happy about this:
+ Their libraries become safer by default, so they can make changes in the future without having to worry about breakage
+ It doesn’t affect your code inside of a module, so it only affects types they already explicitly marked “public”

That's the intent.

The only people who lose are multi-module App developers, because they are “library authors” who don’t need to care about evolution, and now need to add attributes to things they wouldn’t have to before, or suffer language and performance penalties. Their libraries become less reusable and not resilient-by-default.

For example, I have an App for which I wrote a cross-platform model framework in Swift. When I compile it as a framework inside my App, it is bundled there forever. However, I use the same code to build libraries for Linux, which I would like to ship in binary form to 3rd-parties. Am I supposed to litter my code with annotations to mark those types as final, just to make the App fast and convenient to code? What happens when I need to fix a bug and distribute an updated copy, this means the 3rd-parties need to recompile (which they won’t do…).

Typically, for such a problem, I would recommend using a static library instead. But we don’t have those, and anyway they’re not always the best thing these days. So that’s why I started a new thread about creating a “@static” import, so App developers can go back to all the conveniences they had before.

There won't be a perf penalty, but yes, I do expect multi-module apps to use 'exhaustive' on most of their enums, because they don't need the futureproofing. Maybe this should have been mentioned more explicitly in the proposal.

As a perhaps more long-term design note, I think modules ought to have the ability to version-lock themselves to one or more of their dependencies. They would still be required to obey access control as if they were outside those dependencies, but we would suppress some of the semantic consequences of being outside the module, such as the need to assume non-exhaustiveness by default.

That is, there would be two independent axes of library dependency: source vs. binary and version-compatible vs. version-locked:
  - a source dependency allows the compiler to take advantage of the implementation of public entities when generating code
  - a version-locked dependency allows the compiler to take advantage of the implementation of public entities when enforcing semantics

Apps would generally elect to primarily use version-locked source dependencies because they're just pulling down source libraries (e.g. from github) and are comfortable with updating their code if the library changes.

Source libraries on github would generally want to use version-compatible source dependencies because version-locking would put their clients in "library hell" if the locking didn't all agree.

Binary dependencies could reasonably use either.

John.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[Jean-Daniel1 (Jean-Daniel)]

Hello everyone,

My name is Eneko Alonso, iOS developer, new here on the list.

Is there a good summary anywhere that condenses the pros and cons of this new feature that have been discussed so far?

It is not clear to me why non-exhaustive would be the default, requiring adding `@exhaustive` otherwise. Has anyone discussed doing it the other way around, this is, defaulting to exhaustive (no changes with prior Swift versions) and adding a `@nonExhaustive` tag instead as needed?

IIUC, this is to help binary stability.
If a library developer write an enum without knowing about « @exhaustive », it can safely add another enum in a v2 of the library without breaking all the clients.

···

Le 28 déc. 2017 à 21:17, Eneko Alonso via swift-evolution <swift-evolution@swift.org> a écrit :

Apologies if this has been covered already.

Regards and thank you everyone for making Swift better!
Eneko

On Dec 27, 2017, at 10:26 PM, Riley Testut via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Actually, from the other email thread about this same topic (thank god forums are almost here), I see the proposed syntax “final switch” for what I referred to as “switch!”, which I prefer.

On Dec 28, 2017, at 12:17 AM, Riley Testut via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

-1.

I agree this is a problem, but I think this is the wrong solution. I think the solution should be on the client side, not on the framework author’s side.

I would be fine if enums from imported modules are non-exhaustive, as long as I can choose to treat them as exhaustive if I want to. And in that case, if a new case is introduced, I think a fatal error is a reasonable result.

The proposed “switch!” command would do just this, and I think that is the better answer for this. Adding an @exhaustive attribute doesn’t actually prevent someone from adding a case anyway, which I think is a big (and not really solvable) issue

I know much has been said about this, but it’s just my 2c.

On Dec 27, 2017, at 9:42 AM, Thorsten Seitz via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

The proposal is available here:
https://github.com/apple/swift-evolution/blob/master/proposals/0192-non-exhaustive-enums.md
What is your evaluation of the proposal?

-1

I would much prefer the solution proposed by Andrew Bennett in another thread which solves all problems very nicely including the testability of future cases by giving them a placeholder name:

From Andrew’s mail:

public enum HomeworkExcuse {
  case eatenByPet
  case thoughtItWasDueNextWeek
  fallback unknown // NEW
}

Then I believe you would be able to have an exhaustive switch like this:

switch thing {
  case eatenByPet: break
  case thoughtItWasDueNextWeek: break
  case unknown: break
}

Which would still allow compile-time errors if new cases are introduced, while providing a concise way to show something is not exhaustible.

This would also support existing enums with "unknown" equivalent cases would be able to explicitly label those fields as fallback without needing to make large code changes.

I see no reason why you shouldn't be able to use ".unknown", which should still allow this to be testable.

i.e. Andrew’s idea is to introduce a placeholder case instead of marking the enum as exhaustive/non-exhaustive. This gives the future cases a handle to be switched on and to be tested against. Very elegant.

Is the problem being addressed significant enough to warrant a change to Swift?

Yes, but the proposed solution is not as good as it should be, neglecting to provide compile-time errors if new cases are introduced.

Does this proposal fit well with the feel and direction of Swift?

No, due to its shortcomings.

If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

None, but see Andrew Bennett’s idea above.

How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

Followed most of the discussion and review threads.

-Thorsten
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[anandabits (Matthew Johnson)]

Sent from my iPad

I agree that we need a solution to the problem described. I also agree that non-exhaustive is most in keeping with the overall design of Swift at module boundaries. However, I believe this proposal should be modified before being accepted

Thanks for writing this up - you’ve explained a common concern in an interesting way:

This is likely to be a relatively rare need mostly encountered by 3rd party libraries but it will happen. When it does happen it would be really unfortunate to be forced to use a `default` clause rather than something like a `future` clause which will produce an error when compiled against an SDK where the enum includes cases that are not covered. I can imagine cases where this catch-all case would need to do something other than abort the program so I do not like the `switch!` suggestion that has been discussed. The programmer should still be responsible for determining the behavior of unknown cases.

..

While library authors have a legitimate need to reserve the right to introduce new cases for some enums this need can be met without taking away a useful tool for generating static compiler errors when code does not align with intent (in this case, the intent being to cover all known cases). Switch statements working with these kinds of enums should be required to cover unknown cases but should be able to do so while still being statically checked with regards to known cases.

I think that this could be the crux of some major confusion, the root of which is the difference between source packages and binary packages that are updated outside your control (e.g. the OS, or a dynamic library that is updated independently of your app like a 3rd party plugin). Consider:

1) When dealing with independently updated binary packages, your code *has* to implement some behavior for unexpected cases if the enum is non-exhaustive. It isn’t acceptable to not handle that case, and it isn’t acceptable to abort because then your app will start crashing when a new OS comes out. You have to build some sort of fallback into your app.

2) When dealing with a source package that contributes to your app (e.g. through SwiftPM), *YOU* control when you update that package, and therefore it is entirely reasonable to exhaustively handle enums even if that package owner didn’t “intend” for them to be exhaustive. When *you* chose to update the package, you get the “unhandled case” error, and you have maximal “knowability” about the package’s behavior.

It seems that your concern stems from the fact that the feature as proposed is aligned around module boundaries, and therefore overly punishes source packages like #2. I hope you agree that in case #1, that the feature as proposed is the right and only thing we can do: you really do have to handle unknown future cases somehow.

If I’m getting this right, then maybe there is a variant of the proposal that ties the error/warning behavior to whether or not a module is a source module vs a binary module. The problem with that right now is that we have no infrastructure in the language to know this…

Hi Chris, thanks for your reply.

The concern you describe isn’t exactly what I was describing but it is related. John McCall recently posted a sketch of a solution to the concern you describe which looked great to me. I don’t have time to look up the link this morning but I think it was in this review thread.

The actual concern I am describing is where a 3rd party library (or app) wants to switch over a non-exhaustive enum provided by a module that is a binary (not source) dependency. The author of the 3rd party library may have a legitimate reason to switch over an enum despite the author of the binary module reserving the right to add additional cases.

When this circumstance arises they will do it using the tools provided by the language. Regardless of the final language solution they obviously need to cover unknown cases - their library could be shipping on a device which receives an update to the binary dependency that contains a new case. I agree with you that a language-defined crash is not appropriate. The author of the switch must take responsibility for the behavior of unknown cases.

I am arguing that these “pseudo-exhaustive” switch statements will exist in the wild. The crucial point of contention is whether or not the language provides assistance to the author of the 3rd party library in updating their library when the enum provided by the binary dependency changes. Is the author forced to use a `default` case which turns of exhaustiveness checking? Or are they able to use an alternative mechanism for handling unknown cases which does not turn off exhaustiveness checking - all statically known cases must be covered. The most common example of such a mechanism is the `future` (or perhaps `unknown`) case which would only be used for cases that are not statically known.

This facility will of course help authors of these switch statements make the necessary updates as the enum vended by the binary dependency changes. It will also help alert authors of apps that depend on that 3rd party library (which will usually be a source dependency). If the author of the app attempts to rebuild the dependency against a new SDK with added cases the library will fail to build, alerting the user that they should update the 3rd party library.

My position is that if there are reasonable use cases for these kinds of “pseudo-exhaustive” switches then the language should provide exhaustiveness checking of statically known cases via some mechanism that authors can opt-in to using. It’s ok with me if this is a relatively esoteric feature. It won’t be commonly needed, but when it is necessary it will provide significant value.

IIRC there were some reasonable examples of these kinds of switches posted in earlier threads on this topic. I don’t have time to look those up right now either but it would be good for the core team to be aware of them before making a final decision.

I am only aware of two arguments against this kind of “pseudo-exhaustive” switch. One is that users should not attempt to switch over an enum that a library author does not intend to be exhaustive (i.e. it is an “input-only” enum). The other is that a `future` or `unknown` case is not testable.

The first argument is a moral one which I believe should not carry much weight relative to concrete, pragmatic counter-examples such as those that (IIRC) were provided on this list in the past.

I think the discussion is converging on the defect here, but I disagree that the first argument is merely a "moral one"; it is an epistemological one.

The epistemological dilemma is: how can the user of the library "know" all the cases when the library "author" has made it clear that even _they_ don't know all the cases? Obviously, the library user can only switch over the cases that _he or she_ knows about. And if the user has forgotten about some cases that are documented, then clearly, he or she doesn't know about it.

The purpose of the requested modification is to ensure that the user’s knowledge of cases matches the compiler’s knowledge of cases.

I disagree with your framing of the argument. I don’t mean to split hairs, but I view it as a moral argument with an epistemological basis. The argument says that because of the epistemological uncertainty introduced by the possibility of unknown future cases the language should not assist a developer who attempts to cover all cases that are knowable by the compiler at compilation time. It is a judgement that such a language feature is undesirable for one reason or another. This is fundamentally a value judgement.

The argument here is rather that it should be possible to ensure that the user has switched over all the cases that _the compiler_ knows about. Suppose we accept the argument that we want to enable the user to deal with the largest possible set of known cases: why does it have to the compiler that does the warning? why is this not a task for a linter if so desired?

Only the compiler is able to do this with 100% accuracy. Even if we assume that a linter is able to correctly validate this condition, the source will not have the necessary information. How should a linter distinguish intentional use of `default` from one which is only intended to be used when a value that is not known at compile time is encountered? This will require a pseudo-linguistic keyword comment such as `/* unknown */ default`. I hope we can agree that encouraging linters to introduce language variations like this is undesirable.

in what other circumstances do we insist that the compiler inform the end user about future additions to the API at compile time?

This isn’t a request for the compiler to inform the user about future additions to an API. It is a request to validate the compiler’s knowledge of the current state of an API with the current state of the source code.

···

Sent from my iPad

On Jan 2, 2018, at 12:48 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Tue, Jan 2, 2018 at 9:38 AM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:
On Jan 1, 2018, at 11:47 PM, Chris Lattner <clattner@nondot.org> wrote:

On Dec 31, 2017, at 12:14 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

The second argument doesn’t make sense to me: as I noted in my review post, the code path is equally untestable when a `default` case is used, but all statically known cases are matched in earlier patterns. The testability problem of non-exhaustive enums is orthogonal to the issue of language support for “pseudo-exhaustive” switches.

This is the line of reasoning which leads me to conclude that we should dig up the concrete examples which have been provided and evaluate them for merit. If we can’t discard them as a bad coding practice for which a better solution is available then we should strongly consider providing language support for these use cases. The existence of such use cases should also motivate a solution to the testability problem.

- Matthew

-Chris

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[davedelong]

[...]
  2️⃣ If the enum is NOT decorated with @frozen, then I, as an app author, have to account for the possibility that the module may update from underneath my app, and I have to handle an unknown case. This is simple: the compiler should require me to add a “default:” case to my switch statement. This warning is produced IFF: the enum is coming from an external module, and the enum is not decorated with @frozen.

This does not help people who need to write a switch statement over an enum vended by a module that ships with the OS keep their code up to date as the module adds new cases. I find the example of `SKPaymentTransactionState` provided by Brent Royal-Gordon here: [swift-evolution] Enums and Source Compatibility to be compelling. There are rare but legitimate reasons to switch over all known cases of a non-@frozen enum that ship with the OS. These use cases deserve proper language support. I think Jordan’s solution strikes a good balance.

I disagree that more is needed. In the case of the transaction state, it should not be marked as @moana, and so the compiler would force you to add a “default” case to your switch statements. The switch statements would still be exhaustive with all known cases (if you choose to handle all known cases), but you’d still need a default case because there might be new transaction states in the future.

And then you don't get the compiler error/warning when you start compiling against the next OS update that changes the enum. That is an absolutely unacceptable loss of compile-time checks.

Ah, that’s an excellent point! Thanks for pointing that out.

In that case, I revise my proposal:

a @frozen/@tangled/@moana attribute for enums that’s only consulted on externally-linked modules
a “future case:” statement that’s only required on externally-linked, non-@moana enums. That way, if the enum adds a new case in the future, you’ll still get a switch warning about handling all the cases. And then if you want, you could “fallthrough” from this to a “default” case, or vice-versa, or have separate implementations.

It sounds to me like the main thing you’re unhappy about is having to deal with unknown cases when you have a source dependency that you always build and ship with an app. I don’t think anyone is happy with that situation.

That’s definitely part of it. The other main part is that I’m hugely resistant to adding extraneous syntax when a solution that’s simpler for users exists.

Did you take a look at the sketch of a solution provided by John McCall? [swift-evolution] [Review] SE 0192 - Non-Exhaustive Enums

I’ve read that email several times and don’t understand how it relates.

Dave

···

On Jan 3, 2018, at 11:09 AM, Matthew Johnson <matthew@anandabits.com> wrote:

On Jan 3, 2018, at 12:02 PM, Dave DeLong <swift@davedelong.com <mailto:swift@davedelong.com>> wrote:

On Jan 3, 2018, at 10:58 AM, Kevin Nattinger <swift@nattinger.net <mailto:swift@nattinger.net>> wrote:

[anandabits (Matthew Johnson)]

[...]
  2️⃣ If the enum is NOT decorated with @frozen, then I, as an app author, have to account for the possibility that the module may update from underneath my app, and I have to handle an unknown case. This is simple: the compiler should require me to add a “default:” case to my switch statement. This warning is produced IFF: the enum is coming from an external module, and the enum is not decorated with @frozen.

This does not help people who need to write a switch statement over an enum vended by a module that ships with the OS keep their code up to date as the module adds new cases. I find the example of `SKPaymentTransactionState` provided by Brent Royal-Gordon here: [swift-evolution] Enums and Source Compatibility to be compelling. There are rare but legitimate reasons to switch over all known cases of a non-@frozen enum that ship with the OS. These use cases deserve proper language support. I think Jordan’s solution strikes a good balance.

I disagree that more is needed. In the case of the transaction state, it should not be marked as @moana, and so the compiler would force you to add a “default” case to your switch statements. The switch statements would still be exhaustive with all known cases (if you choose to handle all known cases), but you’d still need a default case because there might be new transaction states in the future.

And then you don't get the compiler error/warning when you start compiling against the next OS update that changes the enum. That is an absolutely unacceptable loss of compile-time checks.

Ah, that’s an excellent point! Thanks for pointing that out.

In that case, I revise my proposal:

a @frozen/@tangled/@moana attribute for enums that’s only consulted on externally-linked modules
a “future case:” statement that’s only required on externally-linked, non-@moana enums. That way, if the enum adds a new case in the future, you’ll still get a switch warning about handling all the cases. And then if you want, you could “fallthrough” from this to a “default” case, or vice-versa, or have separate implementations.

It sounds to me like the main thing you’re unhappy about is having to deal with unknown cases when you have a source dependency that you always build and ship with an app. I don’t think anyone is happy with that situation.

That’s definitely part of it. The other main part is that I’m hugely resistant to adding extraneous syntax when a solution that’s simpler for users exists.

That’s fair. If we didn’t have a compelling motivating example I would be more reluctant about the need to add syntax here, but we do have such an example.

Did you take a look at the sketch of a solution provided by John McCall? [swift-evolution] [Review] SE 0192 - Non-Exhaustive Enums

I’ve read that email several times and don’t understand how it relates.

I should have also linked to his previous email. Does this one help clarify what he has in mind? [swift-evolution] [Review] SE 0192 - Non-Exhaustive Enums

···

Sent from my iPad

On Jan 3, 2018, at 12:20 PM, Dave DeLong <swift@davedelong.com> wrote:

On Jan 3, 2018, at 11:09 AM, Matthew Johnson <matthew@anandabits.com> wrote:

On Jan 3, 2018, at 12:02 PM, Dave DeLong <swift@davedelong.com> wrote:
On Jan 3, 2018, at 10:58 AM, Kevin Nattinger <swift@nattinger.net> wrote:

Dave

[xwu (Xiaodi Wu)]

As has already been said, “case unknown” is source-breaking because it
conflicts with any real cases named “unknown”; “\unknown” looks like a key
path but isn’t, and I wonder if it would potentially conflict with existing
key paths.

In any case, my point was not to bikeshed the “unknown” part, but to ask
whether any consideration had been made to have the feature presented as a
flavor of default instead of a flavor of case.

···

On Wed, Jan 3, 2018 at 23:57 Cheyo Jimenez <cheyo@masters3d.com> wrote:

On Jan 3, 2018, at 6:52 PM, Xiaodi Wu via swift-evolution < > swift-evolution@swift.org> wrote:

This is a very nice revision. One bikeshedding thought:

Since "unknown case" is presented as a special kind of "default", can't be
mixed with "default", and can't be used in case patterns, why not "default
unknown" (or "unknown default") instead of "unknown case"?

`case _ :` is already a special case of default.
I’d rather have `case unknown :`
`unknown case :` is weird because of the order of `case`.

Another alternative is `case \unknown :`
`\unknown` would also allow pattern matching.

On Wed, Jan 3, 2018 at 8:05 PM, Jordan Rose via swift-evolution < > swift-evolution@swift.org> wrote:

On Jan 2, 2018, at 18:07, Jordan Rose <jordan_rose@apple.com> wrote:

[Proposal:
https://github.com/apple/swift-evolution/blob/master/proposals/0192-non-exhaustive-enums.md
]

Whew! Thanks for your feedback, everyone. On the lighter side of
feedback—naming things—it seems that most people seem to like '*@frozen*',
and that does in fact have the connotations we want it to have. I like it
too.

More seriously, this discussion has convinced me that it's worth
including what the proposal discusses as a *'future' case*. The key
point that swayed me is that this can produce a *warning* when the
switch is missing a case rather than an *error,* which both provides the
necessary compiler feedback to update your code and allows your
dependencies to continue compiling when you update to a newer SDK. I know
people on both sides won't be 100% satisfied with this, but does it seem
like a reasonable compromise?

The next question is how to spell it. I'm leaning towards `unexpected
case:`, which (a) is backwards-compatible, and (b) also handles "private
cases", either the fake kind that you can do in C (as described in the
proposal), or some real feature we might add to Swift some day. `unknown
case:` isn't bad either.

I too would like to just do `unknown:` or `unexpected:` but that's
technically a source-breaking change:

switch foo {
case bar:
  unknown:
  while baz() {
    while garply() {
      if quux() {
        break unknown
      }
    }
  }
}

Another downside of the `unexpected case:` spelling is that it doesn't
work as part of a larger pattern. I don't have a good answer for that one,
but perhaps it's acceptable for now.

I'll write up a revision of the proposal soon and make sure the core team
gets my recommendation when they discuss the results of the review.

---

I'll respond to a few of the more intricate discussions tomorrow,
including the syntax of putting a new declaration inside the enum rather
than outside. Thank you again, everyone, and happy new year!

I ended up doing these in the opposite order, writing up the new proposal
first and not yet responding to the discussion that's further out. You can
read my revisions at https://github.com/apple/swift-evolution/pull/777\.

In particular, I want to at least address:
- Dave D and Drew C's points about versioned libraries / linking
semantics of modules.
- Jason M's point about migration
and I'll do one more pass over the thread to see if there's anything else
I didn't address directly. (That doesn't mean everyone who disagrees, just
messages where I think there's more I can do to explain why the proposal is
the way it is.)

Jordan

P.S. Enjoying the Disney references. Thanks, Nevin and Dave. :-)

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[Nevin]

Since the attribute means, “Don’t worry about new cases, this enum won’t
change for the rest of its days,” the optimal spelling is clearly
@hakunaMatata

We do still need a way for multi-module apps to work problem-free though.

Nevin

···

On Thu, Jan 4, 2018 at 8:23 AM, Brent Royal-Gordon via swift-evolution < swift-evolution@swift.org> wrote:

On Jan 3, 2018, at 10:02 AM, Dave DeLong via swift-evolution < > swift-evolution@swift.org> wrote:

a @frozen/@tangled/@moana attribute for enums that’s only consulted on
externally-linked modules

Naming is serious business, Dave. Let it go.

[Panajev (Goffredo Marocchi)]

Code that ends up running in a default case somewhere in one of the several places you switch over that enum that you forgot about... well, it may not break source code compatibility, but it may introduce subtle bugs... submarine bugs.

···

Sent from my iPhone

On 5 Jan 2018, at 08:11, Goffredo Marocchi via swift-evolution <swift-evolution@swift.org> wrote:

On 5 Jan 2018, at 00:38, Jordan Rose via swift-evolution <swift-evolution@swift.org> wrote:

Furthermore, the old code needs to continue working without source changes, because updating to a new SDK must not break existing code. (It can introduce new warnings, but even that is something that should be considered carefully.)

So: this proposal is designed to handle the use cases both for Swift library authors to come and for C APIs today, and in particular Apple's Objective-C SDKs and how they've evolved historically.

I will let your very comprehensive rationale brew a little bit more in my mind and reread it, but I still think this is a bit leaning too much over to library authors than library consumers (not unlike the closed by default discussion we all had a while ago).

I remain very unconvinced by the “updating library/SDK” must be source compatible. Especially if it is major version change and a deprecated method has been removed, methods have changed signature, etc... Backward compatible runtime with code compiled against the previous SDK yes, source compatible no.

I feel like this change will make the life easier for library authors, but risks making it easier and easier to make mistakes in apps using the libraries: exhaustive switching over enums is a good important feature that risks going away (the default matters) for not a huge effective benefit to app developers.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[gwendal.roue (Gwendal Roué)]

I agree that life is made worse for a library user that meets a non-frozen enum. And I don't know if it makes the life easier for library authors, because library authors not only throw code in the wild, but also try to prevent library users from making the mistakes you talk about.

To take GRDB.swift as an example, it currently has 15 public enums. What are my options?

More than half of those enums mirror SQLite C enums, or sets of related SQLite values. SQLite is still a living piece of sofware, and it may adds new cases or values in the future. It is then likely that GRDB has to be updated as well. It thus looks like I can make those enums @frozen. But then it looks like I betray the proposal's goal. What's the correct choice?

Some enums are like SKPaymentTransactionState: there's no way the user could infer a proper behavior in a mandatory default/unknown switch case. I thus also want to make them @frozen, and avoid headaches to the library users. Will I pay for it later?

Some other enums are positively non-frozen. I may want to replace some of them with a struct with a limited set of pre-built values. This would, IMHO, improve the library UX because it would again prevent headaches for the library users about scary "future" cases. In the struct with a limited set of values, there is no future cases. Instead, there are a limited set of named values. Which is a very different way to say the same thing.

This last paragraph draws a strong parallel between non-frozen enums and structs:

    // non-frozen enum:
    public enum E {
        case a, b
    }
    
    // equivalent struct:
    public struct S: Equatable {
        private let value: Int
        private init(_ value: Int) { self.value = value }
        public static let a = S(1)
        public static let b = S(2)
        public static func == (lhs: S, rhs: S) -> Bool {
            return lhs.value == rhs.value
        }
    }

    func f(_ s: S) {
        // S can be switched over just like a struct:
        switch s {
        case .a: print("a")
        case .b: print("b")
        default: print("unknown")
        }
    }

BTW, did anybody think about importing C enums as such structs?

Gwendal

···

Le 5 janv. 2018 à 09:11, Goffredo Marocchi via swift-evolution <swift-evolution@swift.org> a écrit :

I feel like this change will make the life easier for library authors, but risks making it easier and easier to make mistakes in apps using the libraries: exhaustive switching over enums is a good important feature that risks going away (the default matters) for not a huge effective benefit to app developers.

[jrose (Jordan Rose)]

Clarifications here only! I don't think I have too much more to convince you; at this point we've looked at the same landscape and come to different conclusions. Although I think I'm more optimistic than you are about being able to build a binary compatibility checker.

They can't tell themselves whether your app supports the new case, other than the heavy-handed "check what SDK they compiled against" that ignores the possibility of embedded binary frameworks.

To be fair, a “linked on or after” check isn’t that heavy-handed. It’s a single if-statement. Yes, they become unwieldy if you’ve got them *everywhere* in your library code, which is why I brought up the point earlier that I’d really love to see more in the way of facilitating this sort of link-dependent behavior that apps are expecting.

Just to clear up this point, I meant "heavy-handed" in that it's just an approximation of whether the client is actually ready to deal with new complexity. It's not an unreasonable one, but it still forces clients to potentially think about all the new things if they just want to update to use one of them.

Maybe "coarse-grained" would have been better phrasing. That also covers the "on or off for the entire app" aspect.

Teaching the compiler/checker/whatever about the linking semantics of modules. For modules that are packaged inside the final built product, there is no need to deal with any unexpected cases, because we already have the exhaustiveness check appropriate for that scenario (regardless of whether the module is shipped as a binary or compiled from source). The app author decides when to update their dependencies, and updating those dependencies will produce new warnings/errors as the compiler notices new or deprecated cases. This is the current state of things and is completely orthogonal to the entire discussion.

This keeps sneaking into discussions and I hope to have it formalized in a proposal soon. On the library side, we do want to make a distinction between "needs binary compatibility" and "does not need binary compatibility". Why? Because we can get much better performance if we know a library is never going to change. A class will not acquire new dynamic-dispatch members; a stored property will not turn into a computed property; a struct will not gain new stored properties. None of those things affect how client code is written, but they do affect what happens at run-time.

This is incorrect; binary compatibility absolutely affects how code is written. No swift library available today is written with binary compatibility in mind, because it’s a non-issue. If it were a pervasive issue, then you’d see a lot more diligence in libraries about not straight-up breaking things between releases.

Other responses on this thread have mentioned removing implementations between releases, which is an excellent illustration of this point: when writing a binary-compatible library, *you cannot remove implementations*. So the very fact that people think that it’s ok to do so is an explicit refutation of the assertion that the manner in which client code is written is independent of the consideration of a library needing binary compatibility.

I very carefully said "client", and then Ted reminded me yesterday in another context that that's not a standard term. So to be clear, I meant "none of these performance-related things would change what an app can and can't do with the API of the library", not anything broader about the design of the library or even about the design of the app.

Jordan

···

On Jan 5, 2018, at 06:52, Swift <swift@davedelong.com> wrote:

[Karl]

Hi Jordan,

Thanks for your thoughtful reply. Comments inline...

Sent from my iPad

Hi, Dave. You're right, all these points are worth addressing. I'm going to go in sections.

This whole “unexpected case” thing is only a problem when you’re linking libraries that are external to/shipped independently of your app. Right now, the *only* case where this might exist is Swift on the server. We *might* run in to this in the future once the ABI stabilizes and we have the Swift libraries shipping as part of iOS/macOS/Linux. Other than this, unexpected enum cases won’t really be a problem developers have to deal with.

I wish this were the case, but it is not. Regardless of what we do for Swift enums, we are in dire need of a fix for C enums. Today, if a C enum doesn't have one of the expected values, the behavior is undefined in the C sense (as in, type-unsafe, memory-unsafe, may invoke functions that shouldn't be invoked, may not invoke functions that should be invoked, etc).

That’s a fair point, and one I had forgotten about. Thanks for reminding me.

Obviously that's an unacceptable state of affairs; even without this proposal we would fix it so that the program will deterministically trap instead. This isn't perfect because it results in a (tiny) performance and code size hit compared to C, but it's better than leaving such a massive hole in Swift's safety story.

The trouble is that many enums—maybe even most enums—in the Apple SDK really are expected to grow new cases, and the Apple API authors rely on this. Many of those—probably most of them—are the ones that Brent Royal-Gordon described as "opaque inputs", like UIViewAnimationTransition, which you're unlikely to switch over but which the compiler should handle correctly if you do. Then there are the murkier ones like SKPaymentTransactionState.

I'm going to come dangerously close to criticizing Apple and say I have a lot of sympathy for third-party developers in the SKPaymentTransactionState case.

This isn’t criticism.

<former evangelist hat>
You’re acknowledging the complexities that exist when writing software, from the point-of-view of someone who has no insight in to the intricacies and challenges faced by the StoreKit team. Our takeaway from this is that no one writes perfect code.
</former evangelist hat>

As Karl Wagner said, there wasn't really any way an existing app could handle that case well, even if they had written an 'unknown case' handler. So what could the StoreKit folks have done instead?

Well, the converse is also true: what reasonable logic exists that an app developer could do to handle a new transaction state for which they’re unprepared? The only sensical thing would be to abort the transaction, apologize to the user, and quickly release an update to your app.

They can't tell themselves whether your app supports the new case, other than the heavy-handed "check what SDK they compiled against" that ignores the possibility of embedded binary frameworks.

To be fair, a “linked on or after” check isn’t that heavy-handed. It’s a single if-statement. Yes, they become unwieldy if you’ve got them *everywhere* in your library code, which is why I brought up the point earlier that I’d really love to see more in the way of facilitating this sort of link-dependent behavior that apps are expecting.

The software-level solution is to make everything require a configuration object (as you allude to below), but a lower-level solution would (hopefully) be even nicer.

So maybe they should have added a property "supportsDeferredState" or something that would have to be set before the new state was returned.

(I'll pause to say I don't know what consideration went into this API and I'm going to avoid looking it up to avoid perjury. This is all hypothetical, for the next API that needs to add a case.)

Let's say we go with that, a property that controls whether the new case is ever passed to third-party code. Now the new case exists, and new code needs to switch over it. At the same time, old code needs to continue working. The new enum case exists, and so even if it shouldn't escape into old code that doesn't know how to handle it, the behavior needs to be defined if it does. Furthermore, the old code needs to continue working without source changes, because updating to a new SDK must not break existing code. (It can introduce new warnings, but even that is something that should be considered carefully.)

So: this proposal is designed to handle the use cases both for Swift library authors to come and for C APIs today, and in particular Apple's Objective-C SDKs and how they've evolved historically.

There's another really interesting point in your message, which Karl, Drew Crawford, and others also touched on.

Teaching the compiler/checker/whatever about the linking semantics of modules. For modules that are packaged inside the final built product, there is no need to deal with any unexpected cases, because we already have the exhaustiveness check appropriate for that scenario (regardless of whether the module is shipped as a binary or compiled from source). The app author decides when to update their dependencies, and updating those dependencies will produce new warnings/errors as the compiler notices new or deprecated cases. This is the current state of things and is completely orthogonal to the entire discussion.

This keeps sneaking into discussions and I hope to have it formalized in a proposal soon. On the library side, we do want to make a distinction between "needs binary compatibility" and "does not need binary compatibility". Why? Because we can get much better performance if we know a library is never going to change. A class will not acquire new dynamic-dispatch members; a stored property will not turn into a computed property; a struct will not gain new stored properties. None of those things affect how client code is written, but they do affect what happens at run-time.

This is incorrect; binary compatibility absolutely affects how code is written. No swift library available today is written with binary compatibility in mind, because it’s a non-issue. If it were a pervasive issue, then you’d see a lot more diligence in libraries about not straight-up breaking things between releases.

Other responses on this thread have mentioned removing implementations between releases, which is an excellent illustration of this point: when writing a binary-compatible library, *you cannot remove implementations*. So the very fact that people think that it’s ok to do so is an explicit refutation of the assertion that the manner in which client code is written is independent of the consideration of a library needing binary compatibility.

Okay, so should we use this as an indicator of whether an enum can grow new cases? (I'm going to ignore C libraries in this section, both because they don't have this distinction and because they can always lie anyway.)

- If a library really is shipped separately from the app, enums can grow new cases, except for the ones that can't. So we need some kind of annotation here. This is your "B" in the original email, so we're all agreed here.

Yes, I think so.

- If a library is shipped with the app, there's no chance of the enum growing a new case at run time. Does that mean we don't need a default case? (Or "unknown case" now.)

I don’t see why you would need a default case. While the enum’s “rawValue” (or whatever) may come from an external library that could change independently, the initializer of the enum (within the library) wouldn’t have a case to handle that. This would be up to the library author to deal with (not the app author), so you shouldn’t need “unknown case:” on your switch statement, because the library has declared the full list of enums, and that *cannot change* unless the app author decides to update his/her dependency.

The answer here is most easily understood in terms of semantic versioning <https://semver.org/&gt;\. If adding a new enum case is a source-breaking change, then it's a source-breaking change, requiring a major version update. The app author decides when to update their dependencies, and might hold off on getting a newer version of a library because it's not compatible with what they have.

If adding a new enum case is not a source-breaking change, then it can be done in a minor version release of a library. Like deprecations, this can produce new warnings, but not new errors, and it should not (if done carefully) break existing code. This isn't a critical feature for a language to have, but I would argue (and have argued) that it's a useful one for library developers. Major releases still exist; this just makes one particular kind of change valid for minor releases as well.

I feel that this is a HUGE cognitive burden to place on library and app authors. And the intricacies of SemVer are, in my opinion, things that your typical app (and library) author really just doesn’t care about. They’re focused on writing their apps. SemVer also breaks when you deal with the common scenario of having your marketing department pick version numbers.

Will the compiler require me to update my CFBundleVersion because I added a new case to an enum? If not, then this is all just hand-wavy and we’re not really making anything safer at all.

(It also feels very subtle to me that 'switch' behaves differently based on where the enum came from. I know this whole proposal adds complexity to the language, and I'd like to keep it as consistent as possible.)

Okay, so what if we did this based on the 'import' rather than on how the module was compiled—Karl's `@static import`? That feels a little better to me because you can see it in your code. (Let's ignore re-exported modules for now.) But now we have two types of 'import', only one of which can be used with system libraries. That also makes me uncomfortable. (And to be fair, it's also something that can be added after the fact without disturbing the rest of the language.)

It’s not about how the module was compiled; it’s about whether or not the module is copied in to your final built product or not.

Maybe instead of “@static import”, it’s more like this off-the-cuff idea:

import Module1
import Module2 @ 12.13.2

I still feel like we’re missing a huge piece of the puzzle, which is how libraries can deal with the linking expectations of clients.

Do library versions need to be part of a mangled name, and then I can create a “multi-fat” library that has not only multiple architectures, but multiple versions per architecture?

Dave

+1 to everything.

I just have two comments:

- As for @static import, we already have two kinds of imports, because we also have @testable imports. It also doesn’t work for system modules, because you can’t compile them with testability. Also, the @testable attribute definitely changes language semantics; if you remove it, your code will lose visibility to many symbols and will possibly fail to compile. Why should it be so outrageous that adding/removing a @static attribute relaxes/enforces handling of unknown cases in switch statements?

Some people might say, “oh, well it’s only test code”. I don’t buy that; there’s nothing trivial about test code. In many large projects, test code can be at least half the volume of code in your repository. It may contain large and complex helpers, mocking capabilities and harnesses, and you still need to maintain all of that code. So actually, it makes up quite a lot of the Swift code floating around out there (or will, once Swift projects get that big), and people seem to be able to understand the slightly different language semantics of @testable imports just fine.

- As for pieces of the puzzle that we are missing - how about module versioning at all? There isn’t even a “-module-version” compiler flag. Only the system modules can version their APIs right now, because we can infer their version by the SDK version.

Also, I realised while writing this that Foundation version numbers appear to be Doubles (Foundation Framework Version Numbers | Apple Developer Documentation — my MBP says 1450.1600000000001). Aren’t they going to have to switch to a consistent SemVer for both platforms at some point? I should be able to write something like “if @available(Foundation 3.1)” and it should compile on Linux and Apple platforms.

Anyway, interesting discussion.

- Karl

···

On 5. Jan 2018, at 15:52, Swift via swift-evolution <swift-evolution@swift.org> wrote:
On Jan 4, 2018, at 5:37 PM, Jordan Rose <jordan_rose@apple.com <mailto:jordan_rose@apple.com>> wrote:

Finally, it's very important that whatever you do in your code doesn't necessarily apply to your dependencies. We've seen in practice that people are not willing to edit their dependencies, even to handle simple SDK changes or language syntax changes (of which there are hopefully no more). That's why I'm pushing the source compatibility aspect so hard, even for libraries that won't be shipped separately from an app.

Overall, I think we're really trying to keep from breaking Swift into different dialects, and making this feature dependent on whether or not the library is embedded in the app would work at cross-purposes to that. Everyone would still be forced to learn about the feature if they used C enums anyway, so we're not even helping out average developers. Instead, it's better that we have one, good model for dealing with other people's enums, which in practice can and do grow new cases regardless of how they are linked.

Jordan

On Jan 3, 2018, at 09:07, Dave DeLong <swift@davedelong.com <mailto:swift@davedelong.com>> wrote:

IMO this is still too large of a hammer for this problem.

This whole “unexpected case” thing is only a problem when you’re linking libraries that are external to/shipped independently of your app. Right now, the *only* case where this might exist is Swift on the server. We *might* run in to this in the future once the ABI stabilizes and we have the Swift libraries shipping as part of iOS/macOS/Linux. Other than this, unexpected enum cases won’t really be a problem developers have to deal with.

Because this will be such a relatively rare problem, I feel like a syntax change like what’s being proposed is a too-massive hammer for such a small nail.

What feels far more appropriate is:

Teaching the compiler/checker/whatever about the linking semantics of modules. For modules that are packaged inside the final built product, there is no need to deal with any unexpected cases, because we already have the exhaustiveness check appropriate for that scenario (regardless of whether the module is shipped as a binary or compiled from source). The app author decides when to update their dependencies, and updating those dependencies will produce new warnings/errors as the compiler notices new or deprecated cases. This is the current state of things and is completely orthogonal to the entire discussion.

and

Adding an attribute (@frozen, @tangled, @moana, @whatever) that can be used to decorate an enum declaration. This attribute would only need to be consulted on enums where the compiler can determine that the module will *not* be part of the final built product. (Ie, it’s an “external” module, in my nomenclature). This, then, is a module that can update independently of the final app, and therefore there are two possible cases:

  1️⃣ If the enum is decorated with @frozen, then I, as an app author, have the assurance that the enum case will not change in future releases of the library, and I can safely switch on all known cases and not have to provide a default case.

  2️⃣ If the enum is NOT decorated with @frozen, then I, as an app author, have to account for the possibility that the module may update from underneath my app, and I have to handle an unknown case. This is simple: the compiler should require me to add a “default:” case to my switch statement. This warning is produced IFF: the enum is coming from an external module, and the enum is not decorated with @frozen.

==========

With this proposal, we only have one thing to consider: the spelling of @frozen/@moana/@whatever that we decorate enums in external modules with. Other than that, the existing behavior we currently have is completely capable of covering the possibilities: we just keep using a “default:” case whenever the compiler can’t guarantee that we can be exhaustive in our switching.

Where the real work would be is teaching the compiler about internally-vs-externally linked modules.

Dave

On Jan 2, 2018, at 7:07 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

[Proposal: https://github.com/apple/swift-evolution/blob/master/proposals/0192-non-exhaustive-enums.md\]

Whew! Thanks for your feedback, everyone. On the lighter side of feedback—naming things—it seems that most people seem to like '@frozen', and that does in fact have the connotations we want it to have. I like it too.

More seriously, this discussion has convinced me that it's worth including what the proposal discusses as a 'future' case. The key point that swayed me is that this can produce a warning when the switch is missing a case rather than an error, which both provides the necessary compiler feedback to update your code and allows your dependencies to continue compiling when you update to a newer SDK. I know people on both sides won't be 100% satisfied with this, but does it seem like a reasonable compromise?

The next question is how to spell it. I'm leaning towards `unexpected case:`, which (a) is backwards-compatible, and (b) also handles "private cases", either the fake kind that you can do in C (as described in the proposal), or some real feature we might add to Swift some day. `unknown case:` isn't bad either.

I too would like to just do `unknown:` or `unexpected:` but that's technically a source-breaking change:

switch foo {
case bar:
  unknown:
  while baz() {
    while garply() {
      if quux() {
        break unknown
      }
    }
  }
}

Another downside of the `unexpected case:` spelling is that it doesn't work as part of a larger pattern. I don't have a good answer for that one, but perhaps it's acceptable for now.

I'll write up a revision of the proposal soon and make sure the core team gets my recommendation when they discuss the results of the review.

---

I'll respond to a few of the more intricate discussions tomorrow, including the syntax of putting a new declaration inside the enum rather than outside. Thank you again, everyone, and happy new year!

Jordan

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[anandabits (Matthew Johnson)]

The review of "SE 0192 - Non-Exhaustive Enums" begins now and runs through January 3, 2018.

The proposal is available here:

https://github.com/apple/swift-evolution/blob/master/proposals/0192-non-exhaustive-enums.md+1, it needs to happen (and ASAP, since it _will_ introduce source-breaking changes one way or the other).

I think non-exhaustive is the correct default. However, does this not mean that, by default, enums will be boxed because the receiver doesn’t know their potential size?

It's not always boxing, but yes, there will be more indirection if the compiler can't see the contents of the enum. (More on that below.)

That would mean that the best transition path for multi-module Apps would be to make your enums @exhaustive, rather than adding “default” statements (which is unfortunate, because I imagine when this change hits, the way you’ll notice will be complaints about missing “default” statements).

Yep, that's going to be the recommendation. The current minimal-for-review implementation does not do this but I'd like to figure out how to improve that; at the very least it might be a sensible thing to do in the migrator.

I do have some thoughts about how we could ease the transition (for this and other resilience-related changes), but it’s best to leave that to a separate discussion.

The one thing I’m still not overly fond of is the name - I would like us to keep the set of resilience/optimisation related keywords to a minimum. “exhaustive” for enums feels an awful lot like “fixed_contents” for structs - couldn’t we come up with a single name which could be used for both? I don’t think anybody’s going to want to use “exhaustive” for structs.

The core team was very focused on this too, but I contend that "exhaustive" is not about optimization and really isn't even about "resilience" (i.e. the ability to evolve a library's API while preserving binary compatibility). It's a semantic feature of an enum, much like 'open' or 'final' is for classes, and it affects what a client can or can't do with an enum. For libaries compiled from source, it won't affect performance at all—the compiler still knows the full set of cases in the current version of the library even if the programmer is forced to consider future versions.

I'm working on the fixed-contents proposal now, though it won't be ready for a while, and the same thing applies there: for structs compiled from source, the compiler can still do all the same optimizations. It's only when the library has binary compatibility concerns that we need to use extra indirection, and then "fixed-contents" becomes important. (As currently designed, it doesn't affect what clients can do with the struct at all.) This means that I don't expect a "normal" package author to write "fixed-contents" at all (however it ends up being spelled), whereas "exhaustive" is a fairly normal thing to consider whenever you make an enum public.

I hope that convinces you that "fixed-contents" and "exhaustive" don't need to have the same name. I don't think anyone loves the particular name "exhaustive", but as you see in the "Alternatives considered" we didn't manage to come up with anything significantly better. If reviewers all prefer something else we'd consider changing it.

Thanks for responding!
Jordan

When you say “libraries compiled from source”, what do you mean?

- Other targets in your project
- Source packages built through SwiftPM / CocoaPods / Carthage / other

And I was being imprecise with the terminology, but also

- Libraries built by someone else but designed to be embedded into an app, so that there's no chance of a different version showing up at run-time.

As for whether its a resilience feature: actually it is completely a resilience feature. The effects on switching are only side-effects; really what “exhaustive” or “nonexhaustive” are saying is literally that cases may be added later. Even if we added private cases, you wouldn’t need to mark those enums as specially exhaustive or not; that would be implied. It’s an accommodation for things which don’t exist yet, so really, it is all about resilience IMO.

"Resilience", as an admittedly fuzzily-defined term in the Swift project, specifically refers to what changes can be made without breaking binary compatibility <https://github.com/apple/swift/blob/master/docs/Lexicon.rst&gt;\. It does not refer to every change you can make to a library. (For comparison, adding a field to a struct is not source-breaking in Swift. We would like to make it not ABI-breaking either; that proposal's coming soon.)

Anyway, as I see it, library authors in general ought to be happy about this:
+ Their libraries become safer by default, so they can make changes in the future without having to worry about breakage
+ It doesn’t affect your code inside of a module, so it only affects types they already explicitly marked “public”

That's the intent.

The only people who lose are multi-module App developers, because they are “library authors” who don’t need to care about evolution, and now need to add attributes to things they wouldn’t have to before, or suffer language and performance penalties. Their libraries become less reusable and not resilient-by-default.

For example, I have an App for which I wrote a cross-platform model framework in Swift. When I compile it as a framework inside my App, it is bundled there forever. However, I use the same code to build libraries for Linux, which I would like to ship in binary form to 3rd-parties. Am I supposed to litter my code with annotations to mark those types as final, just to make the App fast and convenient to code? What happens when I need to fix a bug and distribute an updated copy, this means the 3rd-parties need to recompile (which they won’t do…).

Typically, for such a problem, I would recommend using a static library instead. But we don’t have those, and anyway they’re not always the best thing these days. So that’s why I started a new thread about creating a “@static” import, so App developers can go back to all the conveniences they had before.

There won't be a perf penalty, but yes, I do expect multi-module apps to use 'exhaustive' on most of their enums, because they don't need the futureproofing. Maybe this should have been mentioned more explicitly in the proposal.

As a perhaps more long-term design note, I think modules ought to have the ability to version-lock themselves to one or more of their dependencies. They would still be required to obey access control as if they were outside those dependencies, but we would suppress some of the semantic consequences of being outside the module, such as the need to assume non-exhaustiveness by default.

That is, there would be two independent axes of library dependency: source vs. binary and version-compatible vs. version-locked:
  - a source dependency allows the compiler to take advantage of the implementation of public entities when generating code
  - a version-locked dependency allows the compiler to take advantage of the implementation of public entities when enforcing semantics

Apps would generally elect to primarily use version-locked source dependencies because they're just pulling down source libraries (e.g. from github) and are comfortable with updating their code if the library changes.

Source libraries on github would generally want to use version-compatible source dependencies because version-locking would put their clients in "library hell" if the locking didn't all agree.

Binary dependencies could reasonably use either.

This model aligns pretty well with what I would like to see. It prevents us from paying a penalty when we don’t need the benefits provided by a restriction.

Relating this back to the current proposal, would you expect an app to have the ability to switch over an enum provided by a version-locked dependency that is not annotated with @exhaustive without requiring a default clause?

Yes, and as we find other places where program semantics depend on knowing the implementation, I would expect them to follow suit.

My guess is that enum exhaustiveness is probably more prominent than any other such feature, and maybe even more prominent than all of them put together, but possible examples include:
  - automatically deriving protocol conformances, which we hope will eventually be something you can do for an arbitrary protocol
  - any other kind of structural metaprogramming we might add
  - maybe memberwise struct initialization if there are no explicit initializers, although this is arguably an access control question (just as public/open is)
  - ownership-related features that might make sense to restrict to stored properties, like expecting a struct property to have a stable address, or destructuring a struct with pattern-matching

Now, some of these things might be nice to allow even for resilient types. I know Joe has suggested adding some way of resiliently describing a structural decomposition of a type, which you could then use to derive conformances, etc. But since the basic motivation for restricting any of them is source/binary compatibility, and since version-locking would tell us that the programmer doesn't care about that, it seems sensible that version-locking ought to suppress the restrictions.

Relating to @inlinable proposal also under review, would everything in a source dependency be automatically inlinable whether they were annotated as such or not (at least when version-locked)?

Yes, and regardless of being version-locked. Inlining isn't semantically visible: it's observable in various low-level ways, but it's not supposed to affect basic program semantics except in incidental ways, e.g. by lowering the memory requirements so that programs start working that didn't before. So the compiler is generally always allowed to inline when the call is direct and the callee has a known implementation; that's just standard "as if" behavior. The fact that we can't do this today is just an unfortunate consequence of our current build model.

This is all exciting to hear (as a long term direction)! Thank you for the elaboration.

···

On Dec 21, 2017, at 2:06 PM, John McCall <rjmccall@apple.com> wrote:

On Dec 21, 2017, at 2:41 PM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Dec 21, 2017, at 1:26 PM, John McCall via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Dec 21, 2017, at 2:03 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Dec 20, 2017, at 12:35, Karl Wagner <razielim@gmail.com <mailto:razielim@gmail.com>> wrote:

On 20. Dec 2017, at 19:54, Jordan Rose <jordan_rose@apple.com <mailto:jordan_rose@apple.com>> wrote:

On Dec 20, 2017, at 05:36, Karl Wagner via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 19. Dec 2017, at 23:58, Ted Kremenek via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

John.

[John_McCall]

To be clear, I'm laying out my own vision for this, not an accepted core-team direction. But if you think it's exciting, that's certainly helpful!

John.

···

On Dec 21, 2017, at 3:10 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Dec 21, 2017, at 2:06 PM, John McCall <rjmccall@apple.com <mailto:rjmccall@apple.com>> wrote:

On Dec 21, 2017, at 2:41 PM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Dec 21, 2017, at 1:26 PM, John McCall via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Dec 21, 2017, at 2:03 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Dec 20, 2017, at 12:35, Karl Wagner <razielim@gmail.com <mailto:razielim@gmail.com>> wrote:

On 20. Dec 2017, at 19:54, Jordan Rose <jordan_rose@apple.com <mailto:jordan_rose@apple.com>> wrote:

On Dec 20, 2017, at 05:36, Karl Wagner via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 19. Dec 2017, at 23:58, Ted Kremenek via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

The review of "SE 0192 - Non-Exhaustive Enums" begins now and runs through January 3, 2018.

The proposal is available here:

https://github.com/apple/swift-evolution/blob/master/proposals/0192-non-exhaustive-enums.md+1, it needs to happen (and ASAP, since it _will_ introduce source-breaking changes one way or the other).

I think non-exhaustive is the correct default. However, does this not mean that, by default, enums will be boxed because the receiver doesn’t know their potential size?

It's not always boxing, but yes, there will be more indirection if the compiler can't see the contents of the enum. (More on that below.)

That would mean that the best transition path for multi-module Apps would be to make your enums @exhaustive, rather than adding “default” statements (which is unfortunate, because I imagine when this change hits, the way you’ll notice will be complaints about missing “default” statements).

Yep, that's going to be the recommendation. The current minimal-for-review implementation does not do this but I'd like to figure out how to improve that; at the very least it might be a sensible thing to do in the migrator.

I do have some thoughts about how we could ease the transition (for this and other resilience-related changes), but it’s best to leave that to a separate discussion.

The one thing I’m still not overly fond of is the name - I would like us to keep the set of resilience/optimisation related keywords to a minimum. “exhaustive” for enums feels an awful lot like “fixed_contents” for structs - couldn’t we come up with a single name which could be used for both? I don’t think anybody’s going to want to use “exhaustive” for structs.

The core team was very focused on this too, but I contend that "exhaustive" is not about optimization and really isn't even about "resilience" (i.e. the ability to evolve a library's API while preserving binary compatibility). It's a semantic feature of an enum, much like 'open' or 'final' is for classes, and it affects what a client can or can't do with an enum. For libaries compiled from source, it won't affect performance at all—the compiler still knows the full set of cases in the current version of the library even if the programmer is forced to consider future versions.

I'm working on the fixed-contents proposal now, though it won't be ready for a while, and the same thing applies there: for structs compiled from source, the compiler can still do all the same optimizations. It's only when the library has binary compatibility concerns that we need to use extra indirection, and then "fixed-contents" becomes important. (As currently designed, it doesn't affect what clients can do with the struct at all.) This means that I don't expect a "normal" package author to write "fixed-contents" at all (however it ends up being spelled), whereas "exhaustive" is a fairly normal thing to consider whenever you make an enum public.

I hope that convinces you that "fixed-contents" and "exhaustive" don't need to have the same name. I don't think anyone loves the particular name "exhaustive", but as you see in the "Alternatives considered" we didn't manage to come up with anything significantly better. If reviewers all prefer something else we'd consider changing it.

Thanks for responding!
Jordan

When you say “libraries compiled from source”, what do you mean?

- Other targets in your project
- Source packages built through SwiftPM / CocoaPods / Carthage / other

And I was being imprecise with the terminology, but also

- Libraries built by someone else but designed to be embedded into an app, so that there's no chance of a different version showing up at run-time.

As for whether its a resilience feature: actually it is completely a resilience feature. The effects on switching are only side-effects; really what “exhaustive” or “nonexhaustive” are saying is literally that cases may be added later. Even if we added private cases, you wouldn’t need to mark those enums as specially exhaustive or not; that would be implied. It’s an accommodation for things which don’t exist yet, so really, it is all about resilience IMO.

"Resilience", as an admittedly fuzzily-defined term in the Swift project, specifically refers to what changes can be made without breaking binary compatibility <https://github.com/apple/swift/blob/master/docs/Lexicon.rst&gt;\. It does not refer to every change you can make to a library. (For comparison, adding a field to a struct is not source-breaking in Swift. We would like to make it not ABI-breaking either; that proposal's coming soon.)

Anyway, as I see it, library authors in general ought to be happy about this:
+ Their libraries become safer by default, so they can make changes in the future without having to worry about breakage
+ It doesn’t affect your code inside of a module, so it only affects types they already explicitly marked “public”

That's the intent.

The only people who lose are multi-module App developers, because they are “library authors” who don’t need to care about evolution, and now need to add attributes to things they wouldn’t have to before, or suffer language and performance penalties. Their libraries become less reusable and not resilient-by-default.

For example, I have an App for which I wrote a cross-platform model framework in Swift. When I compile it as a framework inside my App, it is bundled there forever. However, I use the same code to build libraries for Linux, which I would like to ship in binary form to 3rd-parties. Am I supposed to litter my code with annotations to mark those types as final, just to make the App fast and convenient to code? What happens when I need to fix a bug and distribute an updated copy, this means the 3rd-parties need to recompile (which they won’t do…).

Typically, for such a problem, I would recommend using a static library instead. But we don’t have those, and anyway they’re not always the best thing these days. So that’s why I started a new thread about creating a “@static” import, so App developers can go back to all the conveniences they had before.

There won't be a perf penalty, but yes, I do expect multi-module apps to use 'exhaustive' on most of their enums, because they don't need the futureproofing. Maybe this should have been mentioned more explicitly in the proposal.

As a perhaps more long-term design note, I think modules ought to have the ability to version-lock themselves to one or more of their dependencies. They would still be required to obey access control as if they were outside those dependencies, but we would suppress some of the semantic consequences of being outside the module, such as the need to assume non-exhaustiveness by default.

That is, there would be two independent axes of library dependency: source vs. binary and version-compatible vs. version-locked:
  - a source dependency allows the compiler to take advantage of the implementation of public entities when generating code
  - a version-locked dependency allows the compiler to take advantage of the implementation of public entities when enforcing semantics

Apps would generally elect to primarily use version-locked source dependencies because they're just pulling down source libraries (e.g. from github) and are comfortable with updating their code if the library changes.

Source libraries on github would generally want to use version-compatible source dependencies because version-locking would put their clients in "library hell" if the locking didn't all agree.

Binary dependencies could reasonably use either.

This model aligns pretty well with what I would like to see. It prevents us from paying a penalty when we don’t need the benefits provided by a restriction.

Relating this back to the current proposal, would you expect an app to have the ability to switch over an enum provided by a version-locked dependency that is not annotated with @exhaustive without requiring a default clause?

Yes, and as we find other places where program semantics depend on knowing the implementation, I would expect them to follow suit.

My guess is that enum exhaustiveness is probably more prominent than any other such feature, and maybe even more prominent than all of them put together, but possible examples include:
  - automatically deriving protocol conformances, which we hope will eventually be something you can do for an arbitrary protocol
  - any other kind of structural metaprogramming we might add
  - maybe memberwise struct initialization if there are no explicit initializers, although this is arguably an access control question (just as public/open is)
  - ownership-related features that might make sense to restrict to stored properties, like expecting a struct property to have a stable address, or destructuring a struct with pattern-matching

Now, some of these things might be nice to allow even for resilient types. I know Joe has suggested adding some way of resiliently describing a structural decomposition of a type, which you could then use to derive conformances, etc. But since the basic motivation for restricting any of them is source/binary compatibility, and since version-locking would tell us that the programmer doesn't care about that, it seems sensible that version-locking ought to suppress the restrictions.

Relating to @inlinable proposal also under review, would everything in a source dependency be automatically inlinable whether they were annotated as such or not (at least when version-locked)?

Yes, and regardless of being version-locked. Inlining isn't semantically visible: it's observable in various low-level ways, but it's not supposed to affect basic program semantics except in incidental ways, e.g. by lowering the memory requirements so that programs start working that didn't before. So the compiler is generally always allowed to inline when the call is direct and the callee has a known implementation; that's just standard "as if" behavior. The fact that we can't do this today is just an unfortunate consequence of our current build model.

This is all exciting to hear (as a long term direction)! Thank you for the elaboration.

[Jason_Merchant]

I think this whole thing has been unnecessarily convoluted. As a result,
the majority of the replies are rabbit holes.

In my opinion, the true root of the concept in question is as follows:

*A list of something is desired:*
1 - Pancake
2 - Waffle
3 - Juice

*Developer wishes to be able to:*
*A)* Add new things to the list of choices in the future as they come up
with new ideas
*B)* Sometimes select one of the choices to be chosen as the normal choice
if no choice is made by the user

A and B are *separate desires*. In some circumstances a developer may want
to add a new choice and make it the normal choice when there was no normal
choice was clarified before.

···

____________________

*Part 2:*

After this simple desire is clear, there should be two discussions:
*A)* In a text only coding language, what would we like the syntax to look
like? (Without regard to past-bias. What should it really be, forget what
mistaken design choices were made in Swift in the past)
*B)* How do we approach making this happen behind the scenes?

*Bonus:* Given that some of us have changed our approach to programming
significantly beyond text based coding, and into more dynamic mediums of
programming in other niches, and even here and there in Xcode - I would
recommend considering how the IDE would show a modern version of this
concept. I feel too often that Swift design syntax has a *lack of awareness
between the distinctions of what the IDE should do, as opposed to what the
syntax of the language should be*, and what should be handled behind the
scenes by automated tooling.

_____________________

*My opinion*, in answering the above questions is in preference to a simple
easy to read and write syntax, something like the following:

choices Breakfast {
    Pancake, *Waffle*, Juice
}

If a "default" choice is desired, it is obvious to me that I would select
the choice from the IDE, and it would be visually indicated that it was the
default.

When changes occur, whether new choices are added, old ones are removed or
changed, or a default is added, changed, or removed - a behind the scenes
automated tool analyzes the changes and presents migration options through
the IDE.

_____________________

Sincerely,
Jason

[xwu (Xiaodi Wu)]

I think this whole thing has been unnecessarily convoluted. As a result,
the majority of the replies are rabbit holes.

In my opinion, the true root of the concept in question is as follows:

*A list of something is desired:*
1 - Pancake
2 - Waffle
3 - Juice

*Developer wishes to be able to:*
*A)* Add new things to the list of choices in the future as they come up
with new ideas
*B)* Sometimes select one of the choices to be chosen as the normal
choice if no choice is made by the user

A and B are *separate desires*. In some circumstances a developer may
want to add a new choice and make it the normal choice when there was no
normal choice was clarified before.

I don't think this is an accurate summary of the problem being tackled
here. Rather, we are how to enable the vendor of a nonexhaustive enum to
add new cases without breaking binaries compiled against previous versions.
There is little here to do with what a "default" should be. Indeed, it is
an explicit design decision of Swift not to support types having an
implicit default value.

···

On Tue, Jan 2, 2018 at 12:11 PM, Jason Merchant via swift-evolution < swift-evolution@swift.org> wrote:

____________________

*Part 2:*

After this simple desire is clear, there should be two discussions:
*A)* In a text only coding language, what would we like the syntax to
look like? (Without regard to past-bias. What should it really be, forget
what mistaken design choices were made in Swift in the past)
*B)* How do we approach making this happen behind the scenes?

*Bonus:* Given that some of us have changed our approach to programming
significantly beyond text based coding, and into more dynamic mediums of
programming in other niches, and even here and there in Xcode - I would
recommend considering how the IDE would show a modern version of this
concept. I feel too often that Swift design syntax has a *lack of
awareness between the distinctions of what the IDE should do, as opposed to
what the syntax of the language should be*, and what should be handled
behind the scenes by automated tooling.

_____________________

*My opinion*, in answering the above questions is in preference to a
simple easy to read and write syntax, something like the following:

choices Breakfast {
    Pancake, *Waffle*, Juice
}

If a "default" choice is desired, it is obvious to me that I would select
the choice from the IDE, and it would be visually indicated that it was the
default.

When changes occur, whether new choices are added, old ones are removed or
changed, or a default is added, changed, or removed - a behind the scenes
automated tool analyzes the changes and presents migration options through
the IDE.

_____________________

Sincerely,
Jason

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[xwu (Xiaodi Wu)]

Sent from my iPad

Sent from my iPad

I agree that we need a solution to the problem described. I also agree
that non-exhaustive is most in keeping with the overall design of Swift at
module boundaries. However, I believe this proposal should be modified
before being accepted

Thanks for writing this up - you’ve explained a common concern in an
interesting way:

This is likely to be a relatively rare need mostly encountered by 3rd
party libraries but it will happen. When it does happen it would be really
unfortunate to be forced to use a `default` clause rather than something
like a `future` clause which will produce an error when compiled against an
SDK where the enum includes cases that are not covered. I can imagine
cases where this catch-all case would need to do something *other than *abort
the program so I do not like the `switch!` suggestion that has been
discussed. The programmer should still be responsible for determining the
behavior of unknown cases.

..

While library authors have a legitimate need to reserve the right to
introduce new cases for some enums this need can be met without taking away
a useful tool for generating static compiler errors when code does not
align with intent (in this case, the intent being to cover all known
cases). Switch statements working with these kinds of enums should be
required to cover unknown cases but should be able to do so while still
being statically checked with regards to known cases.

I think that this could be the crux of some major confusion, the root of
which is the difference between source packages and binary packages that
are updated outside your control (e.g. the OS, or a dynamic library that is
updated independently of your app like a 3rd party plugin). Consider:

1) When dealing with independently updated binary packages, your code
*has* to implement some behavior for unexpected cases if the enum is
non-exhaustive. It isn’t acceptable to not handle that case, and it isn’t
acceptable to abort because then your app will start crashing when a new OS
comes out. You have to build some sort of fallback into your app.

2) When dealing with a source package that contributes to your app (e.g.
through SwiftPM), *YOU* control when you update that package, and therefore
it is entirely reasonable to exhaustively handle enums even if that package
owner didn’t “intend” for them to be exhaustive. When *you* chose to
update the package, you get the “unhandled case” error, and you have
maximal “knowability” about the package’s behavior.

It seems that your concern stems from the fact that the feature as
proposed is aligned around module boundaries, and therefore overly punishes
source packages like #2. I hope you agree that in case #1, that the
feature as proposed is the right and only thing we can do: you really do
have to handle unknown future cases somehow.

If I’m getting this right, then maybe there is a variant of the proposal
that ties the error/warning behavior to whether or not a module is a source
module vs a binary module. The problem with that right now is that we have
no infrastructure in the language to know this…

Hi Chris, thanks for your reply.

The concern you describe isn’t exactly what I was describing but it is
related. John McCall recently posted a sketch of a solution to the concern
you describe which looked great to me. I don’t have time to look up the
link this morning but I think it was in this review thread.

The actual concern I am describing is where a 3rd party library (or app)
wants to switch over a non-exhaustive enum provided by a module that is a
binary (not source) dependency. The author of the 3rd party library may
have a legitimate reason to switch over an enum despite the author of the
binary module reserving the right to add additional cases.

When this circumstance arises they will do it using the tools provided by
the language. Regardless of the final language solution they obviously
need to cover unknown cases - their library could be shipping on a device
which receives an update to the binary dependency that contains a new
case. I agree with you that a language-defined crash is not appropriate.
The author of the switch must take responsibility for the behavior of
unknown cases.

I am arguing that these “pseudo-exhaustive” switch statements *will*
exist in the wild. The crucial point of contention is whether or not the
language provides assistance to the author of the 3rd party library in
updating their library when the enum provided by the binary dependency
changes. Is the author forced to use a `default` case which turns of
exhaustiveness checking? Or are they able to use an alternative mechanism
for handling unknown cases which does not turn off exhaustiveness checking
- all *statically known* cases must be covered. The most common example
of such a mechanism is the `future` (or perhaps `unknown`) case which would
only be used for cases that *are not* statically known.

This facility will of course help authors of these switch statements make
the necessary updates as the enum vended by the binary dependency changes.
It *will also *help alert authors of apps that depend on that 3rd party
library (which will usually be a source dependency). If the author of the
app attempts to rebuild the dependency against a new SDK with added cases
the library will fail to build, alerting the user that they should update
the 3rd party library.

My position is that if there are *reasonable* use cases for these kinds
of “pseudo-exhaustive” switches then the language should provide
exhaustiveness checking of statically known cases via some mechanism that
authors can opt-in to using. It’s ok with me if this is a relatively
esoteric feature. It won’t be *commonly* needed, but when it is
necessary it will provide significant value.

IIRC there were some reasonable examples of these kinds of switches
posted in earlier threads on this topic. I don’t have time to look those
up right now either but it would be good for the core team to be aware of
them before making a final decision.

I am only aware of two arguments *against *this kind of
“pseudo-exhaustive” switch. One is that users should not attempt to switch
over an enum that a library author does not intend to be exhaustive (i.e.
it is an “input-only” enum). The other is that a `future` or `unknown`
case is not testable.

The first argument is a moral one which I believe should not carry much
weight relative to concrete, pragmatic counter-examples such as those that
(IIRC) were provided on this list in the past.

I think the discussion is converging on the defect here, but I disagree
that the first argument is merely a "moral one"; it is an epistemological
one.

The epistemological dilemma is: how can the user of the library "know" all
the cases when the library "author" has made it clear that even _they_
don't know all the cases? Obviously, the library user can only switch over
the cases that _he or she_ knows about. And if the user has forgotten about
some cases that are documented, then clearly, he or she doesn't know about
it.

The purpose of the requested modification is to ensure that the user’s
knowledge of cases matches the compiler’s knowledge of cases.

I disagree with your framing of the argument. I don’t mean to split
hairs, but I view it as a moral argument with an epistemological basis.
The argument says that because of the epistemological uncertainty
introduced by the possibility of unknown future cases the language should
not assist a developer who attempts to cover all cases that are knowable by
the compiler at compilation time. It is a judgement that such a language
feature is undesirable for one reason or another. This is fundamentally a
value judgement.

I would disagree with you here. It's not a value judgment to say that,
semantically, there is no meaningful distinction between cases the compiler
knows about that you don't know about, and future cases neither you nor the
compiler knows about.

The argument here is rather that it should be possible to ensure that the

user has switched over all the cases that _the compiler_ knows about.
Suppose we accept the argument that we want to enable the user to deal with
the largest possible set of known cases: why does it have to the compiler
that does the warning? why is this not a task for a linter if so desired?

Only the compiler is able to do this with 100% accuracy. Even if we
assume that a linter is able to correctly validate this condition, the
source will not have the necessary information. How should a linter
distinguish intentional use of `default` from one which is *only* intended
to be used when a value that is not known at compile time is encountered?
This will require a pseudo-linguistic keyword comment such as `/* unknown
*/ default`. I hope we can agree that encouraging linters to introduce
language variations like this is undesirable.

There are many ways to skin the cat, as it were. I do not agree that
comments for linters is undesirable, for one. But seeing as we clearly have
a difference of opinion here, this reinforces the idea that this is a great
task for a linter. *My* linter of choice could read the comments. *Yours*
could, for example, replace every `default` which doesn't cover cases known
to the linter with

case knownToLinter1, knownToLinter2, knownToLinter3:
  fallthrough
default:
  // ...

in what other circumstances do we insist that the compiler inform the end

user about future additions to the API at compile time?

This isn’t a request for the compiler to inform the user about future
additions to an API. It is a request to validate the compiler’s knowledge
of the *current* state of an API with the *current* state of the source
code.

Well, it's of course impossible to inform the user about future additions,
so that's poorly phrased on my part. It's about the compiler informing the
end user about *new* additions, part of the *current* state of the API,
that have cropped up since the user last revised the code when the API was
in a *previous* state (or, indistinguishably, members of which a user is
unaware regardless of the temporal sequence of when such members were
added). In what other circumstances do we insist that the compiler perform
this service?

···

On Tue, Jan 2, 2018 at 1:46 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Jan 2, 2018, at 12:48 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Tue, Jan 2, 2018 at 9:38 AM, Matthew Johnson via swift-evolution < > swift-evolution@swift.org> wrote:

On Jan 1, 2018, at 11:47 PM, Chris Lattner <clattner@nondot.org> wrote:
On Dec 31, 2017, at 12:14 PM, Matthew Johnson via swift-evolution < >> swift-evolution@swift.org> wrote:

The second argument doesn’t make sense to me: as I noted in my review

post, the code path is equally untestable when a `default` case is used,
but all *statically known* cases are matched in earlier patterns. The
testability problem of non-exhaustive enums is orthogonal to the issue of
language support for “pseudo-exhaustive” switches.

This is the line of reasoning which leads me to conclude that we should
dig up the concrete examples which have been provided and evaluate them for
merit. If we can’t discard them as a bad coding practice for which a
better solution is available then we should strongly consider providing
language support for these use cases. The existence of such use cases
should also motivate a solution to the testability problem.

- Matthew

-Chris

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution