SE-0194: Derived Collection of Enum Cases

Paul_Cantrell · January 12, 2018, 3:58pm

This raises a question related to Chris’s: what is the utility of having Limb conform to a protocol instead of just providing allValues ad hoc? Does CaseEnumerable / ValueEnumerable serve any purpose other than triggering special behavior in the compiler? Would the protocol ever be used as the type of something in code?

My answers, admittedly weak ones, are: (1) conventions are nice and consistency is nice, and (2) you never know how an abstraction might be used, but you do know that people will be angry when it should fit but doesn’t. I can’t come up with a more compelling or specific argument than those.

Here's a place where you might want to use the protocol: Suppose you're writing a table view data source that displays editable controls for a form. You support several different types of controls, one of which is a list of choices for a picker controller.

  enum Control<Value> {
    case textField
    case picker(choices: [Value])
    …

    func makeView() -> UIView { … }
    subscript(valueOf view: UIView) -> Value { get { … } set { … } }
  }

Presumably you end up writing a schema which looks something like:

  formDataSource = FormDataSource(value: person)
  formDataSource.fields = [
    Section(title: nil, fields: [
      Field(title: "Name", keyPath: \.name, control: .textField),
      Field(title: "Gender", keyPath: \.gender, control: . picker(choices: Array(Gender.allValues))),
      …
    ])
  ]
  tableView.dataSource = formDataSource

The `Array(Gender.allValues)` here is clutter; it'd be nice if we didn't have to write it explicitly. After all, if you're choosing a value of something you know is an enum, it's sensible to assume that you want to choose from all values if it doesn't specify anything more specific. If `ValueEnumerable` is a formalized protocol, you can do that with a constrained extension:

  extension Control where Value: ValueEnumerable {
    static var picker: Control {
      return .picker(choices: Array(Value.allValues))
    }
  }

And now you just need to write:

      Field(title: "Gender", keyPath: \.gender, control: . picker)

One magic future day, the protocol could be Chris’s narrower CaseEnumerable, Brent’s hypothetical picker could use a more general ValueEnumerable, and we could short-circuit the whole debate with:

    extension CaseEnumerable: ValueEnumerable {
        var allValues: WhateverThisTypeIsSupposedToBe {
            return allCases
        }
    }

…but I recall a particularly thorny “alternatives considered” section about the implications of allowing extensions to add protocol conformance to other protocols!

Absent that language feature, I do think Brent has a point.

I tend to agree with Chris’s assessment here, which to my eyes doesn’t contradict Brent’s example:

While we generally generally steer protocols towards being a “bag of semantics” instead of a “bag of syntax”, there are definitely exceptions to that rule, including the ExpressibleBy and other compiler intrinsic protocols which really are *all about* defining syntax. These protocols are not particularly useful for generic algorithms.

The next level down are protocols like hashable/comparable that are useful for generic algorithms, but are also particularly interesting because of compiler synthesized conformances. They are unique because they are both sometimes interesting for generic algorithms, but also sometimes interesting just because you want the synthesized members on *concrete* types for non-generic uses. IMO, this is the bucket that this proposal falls into.

That seems right: this CaseEnumerable / ValueEnumerable protocol will usually be about synthesized conformance, but occasionally be about semantics that are useful for generic algorithms.

I wouldn’t want to touch the “all possible values for all types” idea with an ℵ₀-foot pole.[1] However, I don’t see the harm in going Brent’s direction: a protocol whose “bag of semantics” is along the lines of “has a limited, known set of possible values that does not change during execution, and that might reasonably be iterated over or displayed to the user as a set of choices.”

That suggests to me the name ValueEnumerable. As with Equatable, it is named for the generic use, but synthesized by the compiler in a specific situation where it makes obvious sense to do so.

Cheers, P

[1] Surgeon general warning: cardinalities are not numbers. Do not use the Cantor hierarchy for measurement. Do not attempt to count to infinity. Do not shake hands with infinity. Do not approach an undomesticated infinity without approved protective gear. Stay safe out there.

···

On Jan 11, 2018, at 11:42 PM, Brent Royal-Gordon <brent@architechies.com> wrote:

On Jan 11, 2018, at 4:21 PM, Paul Cantrell <cantrell@pobox.com <mailto:cantrell@pobox.com>> wrote:

On Jan 12, 2018, at 1:28 AM, Chris Lattner <clattner@nondot.org> wrote:

Basically, having `ValueEnumerable` be a formal protocol means we can extend this small automatic behavior into larger automatic behaviors. I can imagine, for instance, building a fuzzer which, given a list of `WritableKeyPath`s whose types are all `ValueEnumerable`, automatically generates random instances of a type and feeds them into a test function. If we get read-write reflection, we might be able to do this automatically and recursively. That'd be pretty cool.

--
Brent Royal-Gordon
Architechies

hlovatt · January 12, 2018, 1:51am

I missed that sorry.

I think it will find other uses and therefore should be ValueEnumerable.

-- Howard.

···

On 11 Jan 2018, at 6:36 pm, Paul Cantrell <cantrell@pobox.com> wrote:

On Jan 11, 2018, at 7:28 PM, Howard Lovatt <howard.lovatt@gmail.com> wrote:

I am in favour of a protocol that you have to explicitly declare, it feels much more like Swift to me. For example you have to say Equatable explicitly.

Howard — Either you’ve missed something or I have. I didn’t view requiring types to explicitly declare conformance as being up for debate at all.

The question I was weighing on it — what I thought Chris and Brent were discussing — was whether this new protocol should be used narrowly for cases of enums without associated types (which Chris favors), or whether it should find more broad use to mean “type which can enumerate all of its possible values” (which Brent finds interesting with caveats). This question has a bearing on whether the protocol’s name should be CaseEnumerable or ValueEnumerable.

In either case, the conformance is always explicitly declared; the compiler merely synthesizes the implementation for enums without associated types.

I think?

As a contra example in Java it feels natural that the compiler just provides the functionality because that is consistent across the language, you don’t declare something as equatable and you don’t tell the compiler that you want the values array generating.

‘Horses for courses’, this is what Swift does.

As a more hard-core example, suppose you want to use statics as an enum like construct, e.g.:

    struct Ex: ValueEnumerable {
        let x1: Int
        let x2: Int
        init(x1: Int, x2: Int) { self.x1 = x1, self.x2 = x2 }
        static let allValues = [x1, x2]
    }

Perhaps the above Ex started as an enum but was changed to a struct during enhancements to the program because the values of x1 and x2 are now read in rather than constants.

-- Howard.

On 11 Jan 2018, at 5:21 pm, Paul Cantrell via swift-evolution <swift-evolution@swift.org> wrote:

On Jan 10, 2018, at 10:21 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:

Brent, thanks for the detailed response, one question about it:

On Jan 10, 2018, at 3:06 AM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

But that's beside the point. What I think the "`allValues` should be allowed to be infinite" suggestion misses is that one of `ValueEnumerable`'s semantics is that it's not only theoretically *possible* to enumerate all the values, but actually *reasonable* to do so.

...

Some types, of course, fall into a gray area. `Int8` is fairly reasonable, but larger integer types get increasingly unreasonable until, by `Int64`, we reach types that would take decades to enumerate. Where the line should be drawn is a matter of opinion. (My opinion, to be clear, is that we shouldn't conform any of them; if someone really wants to do it, they can add a retroactive conformance.)

I’m not sure which way you’re arguing here, but that’s ok. :-)

In my opinion, while I can see where you are coming from (that it could be “reasonable” to allow random types to be ValueEnumerable) I don’t see what the *utility* or *benefit* that would provide.

If we went with a simpler design - one that named this CaseEnumerable and .allCases - we would be heavily biasing the design of the feature towards enum-like applications that do not have associated types. This is “the” problem to be solved in my opinion, and would lead to a more clear and consistently understood feature that doesn’t have the ambiguity and “gray areas” that you discuss above. Given this bias, it is clear that infinite sequences are not interesting.

Of course it would certainly be *possible* for someone to conform a non-enum-like type to CaseEnumerable, but that would be an abuse of the feature, and not a "gray area”.

Is there some specific *utility* and *benefit* from creeping this feature beyond “enumerating cases in enums that don’t have associated types”? Is that utility and benefit large enough to make it worthwhile to water down the semantics of this protocol by making it so abstract?

I gave an example in my review of an enumerable thing where the items are analogous to cases but are not actually cases, and where I thought `allCases` would cause confusion but `allValues` would make sense:

On Jan 10, 2018, at 10:22 AM, Paul Cantrell via swift-evolution <swift-evolution@swift.org> wrote:

Contra Chris, I slightly prefer ValueEnumerable, because it extends to situations where we still want to enumerate a fixed set of possibilities which don’t strictly correspond to enum cases but still have that sort of flavor. For example, one might want:

    enum SideOfBody
      {
      case left
      case right
      }

    enum Limb: ValueEnumerable
      {
      case arm(SideOfBody)
      case leg(SideOfBody)

      static let allValues =
        [
        arm(.left),
        arm(.right),
        leg(.left),
        leg(.right)
        ]
      }

To my eyes, this code reads better than it would with CaseEnumerable / allCases.

This raises a question related to Chris’s: what is the utility of having Limb conform to a protocol instead of just providing allValues ad hoc? Does CaseEnumerable / ValueEnumerable serve any purpose other than triggering special behavior in the compiler? Would the protocol ever be used as the type of something in code?

My answers, admittedly weak ones, are: (1) conventions are nice and consistency is nice, and (2) you never know how an abstraction might be used, but you do know that people will be angry when it should fit but doesn’t. I can’t come up with a more compelling or specific argument than those.

Cheers,

Paul

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

gwendal.roue · January 12, 2018, 5:26am

Hello,

···

Le 12 janv. 2018 à 02:36, Paul Cantrell via swift-evolution <swift-evolution@swift.org> a écrit :

The question I was weighing on it — what I thought Chris and Brent were discussing — was whether this new protocol should be used narrowly for cases of enums without associated types (which Chris favors), or whether it should find more broad use to mean “type which can enumerate all of its possible values” (which Brent finds interesting with caveats). This question has a bearing on whether the protocol’s name should be CaseEnumerable or ValueEnumerable.

A classic mantra on this mailing list is that stdlib protocols ought to come with generic use cases and algorithms.

Is there any such use case, for either CaseEnumerable or ValueEnumerable, that are not already covered by sequence and the various collection protocols?

Did I miss them in the proposal? Did I miss Xiaodi's post about the lack of such use cases?

Gwendal

Chris_Lattner · January 12, 2018, 7:28am

While this is a good general goal, I don’t think this is an absolute requirement. As usual, I think the reality of the situation is more complicated:

While we generally generally steer protocols towards being a “bag of semantics” instead of a “bag of syntax”, there are definitely exceptions to that rule, including the ExpressibleBy and other compiler intrinsic protocols which really are *all about* defining syntax. These protocols are not particularly useful for generic algorithms.

The next level down are protocols like hashable/comparable that are useful for generic algorithms, but are also particularly interesting because of compiler synthesized conformances. They are unique because they are both sometimes interesting for generic algorithms, but also sometimes interesting just because you want the synthesized members on *concrete* types for non-generic uses. IMO, this is the bucket that this proposal falls into.

Finally, there is the “normal” user defined protocol case, which are clearly about defining the semantics of types and nothing more. Duck typing and “defined by syntax” conformance is clearly out of bounds for these.

-Chris

···

On Jan 11, 2018, at 9:26 PM, Gwendal Roué via swift-evolution <swift-evolution@swift.org> wrote:

Hello,

Le 12 janv. 2018 à 02:36, Paul Cantrell via swift-evolution <swift-evolution@swift.org> a écrit :

The question I was weighing on it — what I thought Chris and Brent were discussing — was whether this new protocol should be used narrowly for cases of enums without associated types (which Chris favors), or whether it should find more broad use to mean “type which can enumerate all of its possible values” (which Brent finds interesting with caveats). This question has a bearing on whether the protocol’s name should be CaseEnumerable or ValueEnumerable.

A classic mantra on this mailing list is that stdlib protocols ought to come with generic use cases and algorithms.

davbeck · January 25, 2018, 5:59pm

What is your evaluation of the proposal?

Strong +1. This seems like a no brainer to me, including the naming.

Is the problem being addressed significant enough to warrant a change to Swift?

I think the proposal lays this out pretty clearly. It's a common need or desire, with many suboptimal solutions that are limited by not being part of the compiler. I can think of several places in my own projects that would benefit from this.

Does this proposal fit well with the feel and direction of Swift?

Yes. This is right in line with Codable and now Equatable default conformances. Swift always prefers being explicit over implicit in it's implementations which this follows.

If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

I've experimented with Sourcery (although decided not to ship with it to avoid extra build dependencies) as well as go the manual route.

In general, I really love how Sourcery has started to be used to experiment with new language features similar to how babel works for the node community. It allows us to see how a feature will impact existing code without committing to long term support of a feature.

jtbandes · February 5, 2018, 7:01pm

Hi @Douglas_Gregor — any update from the core team on this?

mackoj · February 12, 2018, 3:22pm

Is this still in review ?

griotspeak · February 12, 2018, 7:50pm

+1 from me if it is still in review.

I disagree with characterizing use of this for non-enums as abuse. If I have a struct composed solely of enums… it seems fairly natural to have an allValues static member.

hooman · February 13, 2018, 12:30am

It is +1 from me for addressing this need but -1 for the proposed solution.

I agree with your literal idea in general, but in a slightly different way. I have previously responded to this message on the old mailing list. I am repeating my counter-proposal again, just in case this review is still open and someone may notice it this time around:

Lets have a literal/macro that expands to a literal array. Lets call it #allEnumValues and make it contextual/scoped as follows:

enum Answer {

    case yes
    case no
    case maybe
    case unknown // Pun intended! (SE-0192 `unknown case`)

    static let allValues = #allEnumValues // Expands to: [Answer.yes, .no, .maybe, .unknown]
}

We could also support arguments #allEnumValues(...) to support filtering based on availability (or other) attributes. This will let developer to be able to have something like this:

enum Answer {
    //...
    static let legacyValues = #allEnumValues(deprecated)
    static let allValues = #allEnumValues(!deprecated)
}

It also eliminates the question of collection type of allValues. The developer can use any type:

enum Answer {
    //...
    static let allValues = MyCollection(#allEnumValues)
}

benrimmington · February 13, 2018, 1:32am

The counter-proposal might have been lost while the mailing lists were in suspend mode.

(Your last message was a few days before the transition to these forums.)

Would #allEnumValues be allowed in a Swift extension of an Objective-C enum?

hooman · February 13, 2018, 1:48am

I am not knowledgeable enough to know if that will be implementable. But for me, the basic implementation would work when the compiler readily "sees" the enum in context and can itself enumerate it. This approach just eliminates some boilerplate and the risk of bugs from manually maintained allValues going out of sync with the actual enum (developer forgetting to update allValues after adding a case). It does not try to be anything more than that. But I think in this case, less is more.

jawbroken · February 13, 2018, 2:07am

I'm having trouble understanding the advantage of this alternative, or how it is a “less is more” situation. The proposed solution is conceptually very simple: there is a protocol with a default implementation for a property. Okay, the default implementation happens to require some compiler help at this point, but this matches similar features like Hashable conformance.

If I understand your proposal, you instead want a contextual literal that expands to an array (or an array literal?) of all enum cases. I think this is somewhat unprecedented in the language, unless you count #line and similar which are weakly contextual (don't require knowledge of scope), and I'm not sure in what ways it is an improvement. Is it mostly because it doesn't require a specific collection type to be specified?

As an aside, throughout these discussions, some people have wondered if there are any useful generic algorithms that can be written uses this proposed protocol. I can think of at least one: a function for use in unit testing that verifies that a dictionary with a ValueEnumerable key contains an entry for every possible element of allValues. This is something that I've often wanted to verify somehow.

hooman · February 13, 2018, 2:56am

The result of the literal will be array literal, not Array. (Heap memory allocation is not needed for array literal). Whether the proposed literal is contextual or accepts the enum as parameter is not a key concern of my proposal. The point is: Compiler does have this information in a given context. It uses it to enforce switch exhaustiveness for example.

My proposal basically reduces the solution to just making the required compiler magic explicit. It does not preclude anything to be added later. It provides a reliable and bug-free means of generating all values of an enum without enforcing anything else. In my proposal the name or type of the static property (or even if it is a static property at all) is not constrained. Something like ValueEnumerable becomes a protocol to facilitate helper functions that we may later decide are generally helpful.

You provided a good example of "less is more" yourself: The above test would not be necessary with my proposal. "Less is more" by providing a clear and obvious path for supporting stuff like deprecation which cannot be handled with the existing mirror/reflection facilities and is not touched in the current proposal.

xwu · February 13, 2018, 3:00am

This sounds like you're proposing macro-like text replacement at compile time, distinct from having a runtime member that is a collection of instantiated values.

I suppose that would solve the motivating problem. It does raise the question of whether what we want after all really is just macros...

hooman · February 13, 2018, 3:04am

One of my first feature requests in pre Swift 1.0 days was hygienic macros. I really liked how Dylan worked. Yes, basically I am asking to get compile-time macro expansion to eliminate the burden and potential for bug in maintaining the list of values manually. Everything else is far less important.

jawbroken · February 13, 2018, 3:37am

I'm not sure what “not necessary” means. I still would like to be able to confirm that e.g. a Dictionary covers all possible enum cases regardless of the outcome of this proposal.

It's still not clear to me why this would be better than having a protocol, which is how similar features have been provided in Swift. They could have made an #equals “macro” that would expand to a member-wise equality check and people could use to manually implement their own Equatable conformance, for example, instead of just conforming to the protocol. Would that have been better? Why?

hooman · February 13, 2018, 3:48am

That one would not be better. As I mentioned to @xwu, I would love to have real macros in the language for such cases, but I understand that it is not going to happen any time soon.

The reason I think this particular one is better served with a macro (disguised as a compiler literal) is evident from the discussions of this thread. Many choices are not obvious and come with significant drawbacks and a single solution does not fit many reasonable requirements and expectations well. We need a more flexible approach and dropping to a lower level (just providing the compiler magic) provides that flexibility without making developers work harder.

dabrahams · February 13, 2018, 6:14am

I think you are mistaken. There are lots of interesting data structures (and other multipass sequences) we want to expose as Collections that simply can't be efficiently indexed by consecutive Ints, but that can be operated on efficiently by code that doesn't assume Int indices. Swift would be a much poorer language if you couldn't make a Collection out of a Set, Dictionary, String, B-tree, or a mathematical series expansion, for example.

benrimmington · February 13, 2018, 10:38am

@jawbroken One advantage of #allEnumValues is that your static property can have any name (e.g. allAnswers, allGregorianWeekdays, allHomeworkExcuses) instead of the proposed allValues. But you could also have your own computed property (with any name) which returns allValues, so this isn't a big deal.

Another possible advantage is that the macro could work with enums imported from Objective-C. But then it wouldn't include private cases defined in ".m" files, or cases added since the macro was compiled, so #allEnumValues would be a misnomer.

Douglas_Gregor · February 13, 2018, 4:59pm

The major problem with something like #allEnumValues is that it doesn't work with resilient libraries, because you would only see the set of cases known at compile time. This is discussed in the proposal at

swift-evolution/0194-derived-collection-of-enum-cases.md at master · apple/swift-evolution · GitHub

Personally, I consider the resilience implications to be significant enough that #allEnumValues is the wrong solution to this problem.

Doug