Re-pitch: Deriving collections of enum cases

Over a year ago, we discussed adding a magic "allValues"/"allCases" static
property on enums with a compiler-derived implementation. The original proposal
PR <https://github.com/apple/swift-evolution/pull/114&gt; has been reopened
for Swift 5 after languishing for a while, and I'd like to revisit it and
make some changes before it goes up for formal review.

Prior discussion:
https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160411/015098.html
(good luck finding the rest of the thread if you weren't on the list at the
time...)

[cc'd swift-dev for importer/availability-related topics below.]

***Naming***

Given the complexity gap between a simple enumeration of cases and full
support for non-enum types and associated values (which we don't intend to
support with this proposal), I think it might be a good idea to adopt the
names *CaseEnumerable/allCases* instead of ValueEnumerable/allValues.

The original proposal didn't expose allValues as a requirement, for fear of
unduly restricting its type. However, if the protocol's scope is more
limited, *static var allCases* can be exposed as a requirement since the
implementations are not likely to be complex. Furthermore...

***Generics***

Since SE-0142
<https://github.com/apple/swift-evolution/blob/master/proposals/0142-associated-types-constraints.md&gt;
was implemented in Swift 4, we now have more expressive options for the
protocol requirements:

  // 1 - array only
  protocol CaseEnumerable {
    static var allCases: [Self] { get }
  }

  // 2 - any sequence
  protocol CaseEnumerable {
    associatedtype *CaseSequence*: Sequence where CaseSequence.Element ==
Self
    static var *allCases*: CaseSequence { get }
  }

  // 3 - any collection
  protocol CaseEnumerable {
    associatedtype *CaseCollection*: Collection where
CaseCollection.Element == Self
    static var *allCases*: CaseCollection { get }
  }

This restricts the CaseEnumerable protocol to be used as a generic
constraint, but that'd be true even with a plain array because of the Self
type.

Personally I like the flexibility provided by the associatedtype, but I
also recognize it won't be incredibly useful for enums ‚ÄĒ more so if we
wanted to provide e.g. UInt8.allValues, whose ideal implementation might be
"return 0...UInt8.max". So I could see allowing allValues to be any
sequence or collection, but flexibility for allCases might be less
important. Others should weigh in here.

***Implementation strategy and edge cases***

Last year <https://twitter.com/CodaFi_/status/920132464001024001&gt;, Robert
Widmann put together an implementation of CaseEnumerable:
https://github.com/apple/swift/compare/master...CodaFi:ace-attorney
I'd love to hear from anyone more familiar with the code whether there's
anything we'd want to change about this approach.

A few tricky situations have been brought to my attention:

- Enums *imported from C/Obj-C* headers. Doug Gregor writes: *"The
autogenerated allValues would only be able to list the enum cases it knows
about from the header it was compiled with. If the library changes to
add cases in the future (which, for example, Apple frameworks tend to do),
those wouldn’t be captured in allValues."*

My understanding of the runtime/importer is very shallow, but with the
current metadata-based strategy, I suspect imported enums couldn't be
supported at all, or if they could, the metadata would be generated at
import time rather than loaded dynamically from the library, which
naturally wouldn't behave the same way when you drop in an upgraded version
of the library. Is that correct?

(Nonetheless, if a user really wanted this auto-generation, it would be
nice to allow it somehow. Personally, I have had enums whose "source of
truth" was an Obj-C header file, but since it was compiled in with the rest
of the application, we didn't care at all about library upgrades. Maybe an
internal extension adding a conformance can be allowed to participate in
auto-generation?)

- Enums with *availability* annotations on some cases. Doug Gregor writes: *"if
I have a case that’s only available on macOS 10.12 and newer, it probably
shouldn’t show up if I use allValues when running on macOS 10.11."*

If we fetch cases from the enum metadata, does this "just work" since the
metadata will be coming from whichever version of the library is loaded at
runtime? If not, is it at least *possible* to extract availability info
from the metadata? Finally, if not, should we try to synthesize an
implementation that uses #available checks, or just refuse to synthesize
allCases?

- Should it be possible to add a CaseEnumerable conformance in an
*extension*? My thinking is: we want to make sure the metadata is coming
from the module that defines the enum, so we could restrict autogeneration
of allCases to that same module. (That is, it wouldn't be possible to
synthesize allCases for a CaseEnumerable extension on an enum from another
module.) Although, it may be that I am missing something and this
restriction isn't actually necessary. The question to answer is: in exactly
which circumstances can the implementation be synthesized?

Looking forward to hearing everyone's thoughts,
Jacob

Over a year ago, we discussed adding a magic "allValues"/"allCases" static
property on enums with a compiler-derived implementation. The original proposal
PR <https://github.com/apple/swift-evolution/pull/114&gt; has been reopened
for Swift 5 after languishing for a while, and I'd like to revisit it and
make some changes before it goes up for formal review.

Thanks for bringing this one back up!

Prior discussion:
[swift-evolution] ValueEnumerable protocol with derived implementation for enums
(good luck finding the rest of the thread if you weren't on the list at the
time...)

[cc'd swift-dev for importer/availability-related topics below.]

***Naming***

Given the complexity gap between a simple enumeration of cases and full
support for non-enum types and associated values (which we don't intend to
support with this proposal), I think it might be a good idea to adopt the
names *CaseEnumerable/allCases* instead of ValueEnumerable/allValues.

Naming the protocol CaseEnumerable/allCases seems like an unnecessary
restriction. There may be a complexity gap today in synthesizing the
requirement for types other than basic enums, but that doesn't mean that it
will always exist. What's the value of locking ourselves into the more
restrictive name? Protocols act as contracts that can be used for generic
programming and should thus be as generalized as possible. Any type could
implement ValueEnumerable and implement allValues by hand if they wanted.
If you don't intend to forbid this, then the name CaseEnumerable/allCases
is not an improvement over ValueEnumerable/allValues.

The original proposal didn't expose allValues as a requirement, for fear
of unduly restricting its type. However, if the protocol's scope is more
limited, *static var allCases* can be exposed as a requirement since the
implementations are not likely to be complex. Furthermore...

***Generics***

Since SE-0142
<https://github.com/apple/swift-evolution/blob/master/proposals/0142-associated-types-constraints.md&gt;
was implemented in Swift 4, we now have more expressive options for the
protocol requirements:

  // 1 - array only
  protocol CaseEnumerable {
    static var allCases: [Self] { get }
  }

  // 2 - any sequence
  protocol CaseEnumerable {
    associatedtype *CaseSequence*: Sequence where CaseSequence.Element ==
Self
    static var *allCases*: CaseSequence { get }
  }

  // 3 - any collection
  protocol CaseEnumerable {
    associatedtype *CaseCollection*: Collection where
CaseCollection.Element == Self
    static var *allCases*: CaseCollection { get }
  }

This restricts the CaseEnumerable protocol to be used as a generic
constraint, but that'd be true even with a plain array because of the Self
type.

Personally I like the flexibility provided by the associatedtype, but I
also recognize it won't be incredibly useful for enums ‚ÄĒ more so if we
wanted to provide e.g. UInt8.allValues, whose ideal implementation might be
"return 0...UInt8.max". So I could see allowing allValues to be any
sequence or collection, but flexibility for allCases might be less
important. Others should weigh in here.

This goes back to the point above about generality in protocol design and
future-proofing‚ÄĒSequence would be the most forward thinking way to specify
the allValues requirement, even if the synthesized version uses something
more refined, like Collection.

I would strongly prefer if the synthesized allValues for simple enums was
*not* an array, but rather a special Int-indexable RandomAccessCollection
(which could be wrapped in an AnyRandomAccessCollection to prevent leaking
implementation details in the API). The rationale for this is that the
information about the cases of an enum is already part of the static
metadata of a type, so we shouldn't force callers of allValues to incur a
heap allocation that is both (1) relatively slow and (2) moves information
we already have into a new copy in heap memory.

The idea that popped into my head for synthesizing the implementation is
something like this, where we map ordinals to cases. The optimizer will
compress this very nicely, I believe, so iterating the cases of an enum
would be extremely fast:

enum MyEnum: ValueEnumerable {
  case zero, one, two, three

  static var allValues: AnyRandomAccessCollection<MyEnum> {
    return AnyRandomAccessCollection((0..<4).lazy.map {
      switch $0 {
      case 0: return zero
      case 1: return one
      case 2: return two
      case 3: return three
      default: fatalError("unreachable")
      }
    })
  }
}

Some might argue that the cases where this performance difference is a
concern are rare, but IMO an implementation synthesized by the compiler
should attempt to be as optimal as possible and not admit obvious
performance penalties if we can predict and avoid them.

There's also a possible optimization to the code above if you special case
RawRepresentable enums where all cases are in a contiguous range of
integral raw values‚ÄĒyou can just have your `map` function call
init?(rawValue:) instead.

***Implementation strategy and edge cases***

Last year <https://twitter.com/CodaFi_/status/920132464001024001&gt;, Robert
Widmann put together an implementation of CaseEnumerable:
https://github.com/apple/swift/compare/master...CodaFi:ace-attorney
I'd love to hear from anyone more familiar with the code whether there's
anything we'd want to change about this approach.

A few tricky situations have been brought to my attention:

- Enums *imported from C/Obj-C* headers. Doug Gregor writes: *"The
autogenerated allValues would only be able to list the enum cases it knows
about from the header it was compiled with. If the library changes to
add cases in the future (which, for example, Apple frameworks tend to do),
those wouldn’t be captured in allValues."*

My understanding of the runtime/importer is very shallow, but with the
current metadata-based strategy, I suspect imported enums couldn't be
supported at all, or if they could, the metadata would be generated at
import time rather than loaded dynamically from the library, which
naturally wouldn't behave the same way when you drop in an upgraded version
of the library. Is that correct?

(Nonetheless, if a user really wanted this auto-generation, it would be
nice to allow it somehow. Personally, I have had enums whose "source of
truth" was an Obj-C header file, but since it was compiled in with the rest
of the application, we didn't care at all about library upgrades. Maybe an
internal extension adding a conformance can be allowed to participate in
auto-generation?)

- Enums with *availability* annotations on some cases. Doug Gregor
writes: *"if I have a case that’s only available on macOS 10.12 and
newer, it probably shouldn’t show up if I use allValues when running on
macOS 10.11."*

If we fetch cases from the enum metadata, does this "just work" since the
metadata will be coming from whichever version of the library is loaded at
runtime? If not, is it at least *possible* to extract availability info
from the metadata? Finally, if not, should we try to synthesize an
implementation that uses #available checks, or just refuse to synthesize
allCases?

- Should it be possible to add a CaseEnumerable conformance in an
*extension*? My thinking is: we want to make sure the metadata is coming
from the module that defines the enum, so we could restrict autogeneration
of allCases to that same module. (That is, it wouldn't be possible to
synthesize allCases for a CaseEnumerable extension on an enum from another
module.) Although, it may be that I am missing something and this
restriction isn't actually necessary. The question to answer is: in exactly
which circumstances can the implementation be synthesized?

If we want to be consistent with Codable and Equatable/Hashable
conformance, then conformance should not be synthesized in an extension.
However, I personally want to remove that restriction for Eq/Hash so that
it can be added in same-file extensions, and Itai Ferber wants to do the
same for Codable (https://github.com/apple/swift/pull/11735\).

In those cases, you would have a resilience problem if you could synthesize
in an extension outside the same file‚ÄĒCodable and Equatable/Hashable need
access to the private stored members of the type, so same-file extensions
are the only place where that would still work. For ValueEnumerable, you
don't have that problem, because cases are always public, so it could be
argued that synthesizing it anywhere is safe. So, in my mind, the only
issue becomes where we want to be consistent with the other synthesized
protocols or not.

···

On Sun, Nov 5, 2017 at 11:54 PM Jacob Bandes-Storch via swift-evolution < swift-evolution@swift.org> wrote:

Looking forward to hearing everyone's thoughts,
Jacob
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Over a year ago, we discussed adding a magic "allValues"/"allCases" static property on enums with a compiler-derived implementation. The original proposal PR <https://github.com/apple/swift-evolution/pull/114&gt; has been reopened for Swift 5 after languishing for a while, and I'd like to revisit it and make some changes before it goes up for formal review.

Prior discussion: [swift-evolution] ValueEnumerable protocol with derived implementation for enums (good luck finding the rest of the thread if you weren't on the list at the time...)

[cc'd swift-dev for importer/availability-related topics below.]

**Naming**

Given the complexity gap between a simple enumeration of cases and full support for non-enum types and associated values (which we don't intend to support with this proposal), I think it might be a good idea to adopt the names CaseEnumerable/allCases instead of ValueEnumerable/allValues.

The original proposal didn't expose allValues as a requirement, for fear of unduly restricting its type. However, if the protocol's scope is more limited, static var allCases can be exposed as a requirement since the implementations are not likely to be complex. Furthermore...

**Generics**

Since SE-0142 <https://github.com/apple/swift-evolution/blob/master/proposals/0142-associated-types-constraints.md&gt; was implemented in Swift 4, we now have more expressive options for the protocol requirements:

  // 1 - array only
  protocol CaseEnumerable {
    static var allCases: [Self] { get }
  }

  // 2 - any sequence
  protocol CaseEnumerable {
    associatedtype CaseSequence: Sequence where CaseSequence.Element == Self
    static var allCases: CaseSequence { get }
  }

  // 3 - any collection
  protocol CaseEnumerable {
    associatedtype CaseCollection: Collection where CaseCollection.Element == Self
    static var allCases: CaseCollection { get }
  }

This restricts the CaseEnumerable protocol to be used as a generic constraint, but that'd be true even with a plain array because of the Self type.

Personally I like the flexibility provided by the associatedtype, but I also recognize it won't be incredibly useful for enums ‚ÄĒ more so if we wanted to provide e.g. UInt8.allValues, whose ideal implementation might be "return 0...UInt8.max". So I could see allowing allValues to be any sequence or collection, but flexibility for allCases might be less important. Others should weigh in here.

Generalizing the result type is probably a good thing to do before we finalize the user-facing component of this. I wrote the interface with an Array at the time because it was easy and because the first implementation of this tried to directly synthesize an array literal expression.

**Implementation strategy and edge cases**

Last year <https://twitter.com/CodaFi_/status/920132464001024001&gt;, Robert Widmann put together an implementation of CaseEnumerable: https://github.com/apple/swift/compare/master...CodaFi:ace-attorney
I'd love to hear from anyone more familiar with the code whether there's anything we'd want to change about this approach.

A few tricky situations have been brought to my attention:

- Enums imported from C/Obj-C headers. Doug Gregor writes: "The autogenerated allValues would only be able to list the enum cases it knows about from the header it was compiled with. If the library changes to add cases in the future (which, for example, Apple frameworks tend to do), those wouldn’t be captured in allValues."

My understanding of the runtime/importer is very shallow, but with the current metadata-based strategy, I suspect imported enums couldn't be supported at all, or if they could, the metadata would be generated at import time rather than loaded dynamically from the library, which naturally wouldn't behave the same way when you drop in an upgraded version of the library. Is that correct?

Exactly. Today, we don’t write out metadata for foreign enumerations and it would be interesting to see how that would look if we decide to do so in the future. Either way, leaving it up to the authors of Objective-C/C to create a Swift wrapper to get access to this protocol is a little unfortunate, but not that big a deal ultimately.

- Enums with availability annotations on some cases. Doug Gregor writes: "if I have a case that’s only available on macOS 10.12 and newer, it probably shouldn’t show up if I use allValues when running on macOS 10.11."

If we fetch cases from the enum metadata, does this "just work" since the metadata will be coming from whichever version of the library is loaded at runtime? If not, is it at least possible to extract availability info from the metadata? Finally, if not, should we try to synthesize an implementation that uses #available checks, or just refuse to synthesize allCases?

Up front, I want to point out that using a metadata-based approach will still enumerate unavailable cases.

enum Foo {
  case bar
  case baz
  @available(*, unavailable)
  case quux
}

print(unsafeBitCast(Int8(2), to: Foo.self))

Whether or not this is desirable is a point that will need to be fleshed out. My original implementation did take conditional availability into account and tried to group together similarly available cases - but (modulo how weird that was) that implementation also assumed that ‚ÄúallCases‚ÄĚ meant that the resulting collection excluded unavailable cases. I don‚Äôt lean one way or the other on this issue but I would like it settled in the proposal. On the one hand, it will now be possible to synthesize unavailable enum cases, on the other things like switches over enums with unavailable cases must still handle the unavailable case.

- Should it be possible to add a CaseEnumerable conformance in an extension? My thinking is: we want to make sure the metadata is coming from the module that defines the enum, so we could restrict autogeneration of allCases to that same module. (That is, it wouldn't be possible to synthesize allCases for a CaseEnumerable extension on an enum from another module.) Although, it may be that I am missing something and this restriction isn't actually necessary. The question to answer is: in exactly which circumstances can the implementation be synthesized?

The metadata is there across framework boundaries.

From an aesthetic POV, I‚Äôm partial to Slava‚Äôs idea that it shouldn‚Äôt ever be synthesized (because I‚Äôm lazy, go figure) which makes the protocol opt-in. But this also means users may have to go through multiple rounds of ‚Äúwhy doesn't X framework/overlay export CaseEnumerable implementations‚ÄĚ with authors. I think a good middle ground is to make conformance opt-in then teach Mirror to recognize when it‚Äôs reflecting a metatype for a simple enumeration to provide a getter for its cases.

~Robert Widmann

···

On Nov 6, 2017, at 2:54 AM, Jacob Bandes-Storch via swift-dev <swift-dev@swift.org> wrote:

Looking forward to hearing everyone's thoughts,
Jacob
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Personally I like the flexibility provided by the associatedtype, but I also recognize it won't be incredibly useful for enums ‚ÄĒ more so if we wanted to provide e.g. UInt8.allValues, whose ideal implementation might be "return 0...UInt8.max". So I could see allowing allValues to be any sequence or collection, but flexibility for allCases might be less important. Others should weigh in here.

I think we should allow any `Collection` (option 3), and I think we should have a `DefaultCaseCollection<Enum>` type in the standard library which encapsulates the interaction with the runtime.

A `Collection` is good because `Sequence` doesn't provide all of the guarantees we want‚ÄĒ`allValues` should never be single-pass and it should always be possible to resume iteration at an earlier point, which `Sequence` doesn't guarantee. (It probably makes sense to use `BidirectionalCollection`, actually; I'm less sure about `RandomAccessCollection`.) At the same time, an `Array` ties us to all sorts of things that are unnecessary at best and harmful at worst, like a heap allocation. It also forces us into integer indices, which may be suboptimal for certain use cases (particularly if we later want to support associated values). And it prevents us from making types like integers conform, which I think would be a good idea.

Meanwhile, encapsulating the runtime machinery in `DefaultCaseCollection` gives users an escape hatch: If the enum you want to use doesn't conform to `ValueEnumerable`, but you're certain it's compatible, you can construct a `DefaultCaseCollection` for it. `DefaultCaseCollection` can be a `RandomAccessCollection` with `Int` indices, making it convenient to use, but at the same time, it's *not* an array, so it doesn't have to allocate storage or think about `NSArray` bridging. And it minimizes the complexity of what the compiler needs to synthesize.

  public protocol ValueEnumerable {
    associatedtype AllValues: BidirectionalCollection where AllValues.Element == Self
    static var allValues: AllValues { get }
  }

  // The compiler automatically does `typealias AllValues = DefaultCaseCollection<Self>` if the
  // conformance is on the original declaration, the type is compatible (e.g. no associated values),
  // and a different type is neither explicitly specified nor inferred. That will cause this default
  // implementation to be used:
  extension ValueEnumerable where AllValues == DefaultCaseCollection<Self> {
    public static var allValues: DefaultCaseCollection<Self> {
      return DefaultCaseCollection(unsafeForEnum: Self.self)
    }
  }

  public struct DefaultCaseCollection<Enum>: RandomAccessCollection {
    public var startIndex: Int { return 0 }
    public let endIndex: Int
    
    public init(unsafeForEnum _: Enum.Type) {
      endIndex = _countCaseValues(Enum.self)
    }
    
    public subscript(i: Int) -> Enum {
      precondition(indices.contains(i), "Case index out of range")
      return Builtin.reinterpretCast(i) as Enum
    }
  }

···

--
Brent Royal-Gordon
Architechies

Over a year ago, we discussed adding a magic "allValues"/"allCases" static
property on enums with a compiler-derived implementation. The original proposal
PR <https://github.com/apple/swift-evolution/pull/114&gt; has been reopened
for Swift 5 after languishing for a while, and I'd like to revisit it and
make some changes before it goes up for formal review.

Prior discussion: https://lists.swift.org/pipermail/swift-
evolution/Week-of-Mon-20160411/015098.html (good luck finding the rest of
the thread if you weren't on the list at the time...)

[cc'd swift-dev for importer/availability-related topics below.]

***Naming***

Given the complexity gap between a simple enumeration of cases and full
support for non-enum types and associated values (which we don't intend to
support with this proposal), I think it might be a good idea to adopt the
names *CaseEnumerable/allCases* instead of ValueEnumerable/allValues.

The original proposal didn't expose allValues as a requirement, for fear
of unduly restricting its type. However, if the protocol's scope is more
limited, *static var allCases* can be exposed as a requirement since the
implementations are not likely to be complex. Furthermore...

***Generics***

Since SE-0142
<https://github.com/apple/swift-evolution/blob/master/proposals/0142-associated-types-constraints.md&gt;
was implemented in Swift 4, we now have more expressive options for the
protocol requirements:

  // 1 - array only
  protocol CaseEnumerable {
    static var allCases: [Self] { get }
  }

  // 2 - any sequence
  protocol CaseEnumerable {
    associatedtype *CaseSequence*: Sequence where CaseSequence.Element ==
Self
    static var *allCases*: CaseSequence { get }
  }

  // 3 - any collection
  protocol CaseEnumerable {
    associatedtype *CaseCollection*: Collection where
CaseCollection.Element == Self
    static var *allCases*: CaseCollection { get }
  }

This restricts the CaseEnumerable protocol to be used as a generic
constraint, but that'd be true even with a plain array because of the Self
type.

Personally I like the flexibility provided by the associatedtype, but I
also recognize it won't be incredibly useful for enums ‚ÄĒ more so if we
wanted to provide e.g. UInt8.allValues, whose ideal implementation might be
"return 0...UInt8.max". So I could see allowing allValues to be any
sequence or collection, but flexibility for allCases might be less
important. Others should weigh in here.

***Implementation strategy and edge cases***

Last year <https://twitter.com/CodaFi_/status/920132464001024001&gt;, Robert
Widmann put together an implementation of CaseEnumerable: https://
github.com/apple/swift/compare/master...CodaFi:ace-attorney
I'd love to hear from anyone more familiar with the code whether there's
anything we'd want to change about this approach.

A few tricky situations have been brought to my attention:

- Enums *imported from C/Obj-C* headers. Doug Gregor writes: *"The
autogenerated allValues would only be able to list the enum cases it knows
about from the header it was compiled with. If the library changes to
add cases in the future (which, for example, Apple frameworks tend to do),
those wouldn’t be captured in allValues."*

This proposal would be very useful in reducing boiler plate code in many
cases!

With "Enums imported from C/Obj-C headers", it would be nice to
automatically bridge it by using an appropriately namespaced (and
dynamically loaded) ObjC equivalent values(s), if one is available. I think
this addresses Doug's concerns.

For example:

//

// Foundation NSObjCRuntime.h

//

#define NS_VALUE_ENUMERABLE(__TYPE__) \

    FOUNDATION_EXPORT NSInteger const __TYPE__ ##ValueCount; \

    FOUNDATION_EXPORT __TYPE__ __TYPE__##ValueAtIndex(NSInteger index);

//

// UITableViewCellStyle.h

//

...

NS_VALUE_ENUMERABLE(UITableViewCellStyle);

//

// UITableViewCellStyle.m

//

...

NSInteger const UITableViewCellStyleValueCount = 4;

UITableViewCellStyle UITableViewCellStyleValueAtIndex(NSInteger index) {

    // Can be a switch statement in more complicated cases

    return (UITableViewCellStyle) index;

}

//

// UITableViewCellStyle-Generated.swift

//

extension UITableViewCellStyle: ValueEnumerable {

    typealias AllValuesCollection = _NSValueEnumerableCollection<

    static var allValues: AllValuesCollection {

        return AllValuesCollection(

            count: UITableViewCellStyleValueCount,

            valueAtIndex: UITableViewCellStyleValueAtIndex)

    }

}

//

// Swift Standard Library

//

public struct _NSValueEnumerableCollection<T>: Collection {

    private let valueAtIndex: (Int) -> T

    public let count: Int

    public var startIndex: Int { return 0 }

    public var endIndex: Int { return count }

    public init(count: Int, valueAtIndex: @escaping (Int) -> T) {

        self.count = count

        self.valueAtIndex = valueAtIndex

    }

    public subscript(index: Int) -> T {

        // Potentially valueAtIndex could be @convention(c) returning

        // an integer type. T could be RawRepresentable, and conversion

        // errors could be handled here.

        return self.valueAtIndex(index)

    }

    public func index(after i: Int) -> Int {

        return i + 1

    }

}

Thanks,
Andrew Bennett

My understanding of the runtime/importer is very shallow, but with the

···

On Mon, Nov 6, 2017 at 6:54 PM, Jacob Bandes-Storch via swift-evolution < swift-evolution@swift.org> wrote:

current metadata-based strategy, I suspect imported enums couldn't be
supported at all, or if they could, the metadata would be generated at
import time rather than loaded dynamically from the library, which
naturally wouldn't behave the same way when you drop in an upgraded version
of the library. Is that correct?

(Nonetheless, if a user really wanted this auto-generation, it would be
nice to allow it somehow. Personally, I have had enums whose "source of
truth" was an Obj-C header file, but since it was compiled in with the rest
of the application, we didn't care at all about library upgrades. Maybe an
internal extension adding a conformance can be allowed to participate in
auto-generation?)

- Enums with *availability* annotations on some cases. Doug Gregor
writes: *"if I have a case that’s only available on macOS 10.12 and
newer, it probably shouldn’t show up if I use allValues when running on
macOS 10.11."*

If we fetch cases from the enum metadata, does this "just work" since the
metadata will be coming from whichever version of the library is loaded at
runtime? If not, is it at least *possible* to extract availability info
from the metadata? Finally, if not, should we try to synthesize an
implementation that uses #available checks, or just refuse to synthesize
allCases?

- Should it be possible to add a CaseEnumerable conformance in an
*extension*? My thinking is: we want to make sure the metadata is coming
from the module that defines the enum, so we could restrict autogeneration
of allCases to that same module. (That is, it wouldn't be possible to
synthesize allCases for a CaseEnumerable extension on an enum from another
module.) Although, it may be that I am missing something and this
restriction isn't actually necessary. The question to answer is: in exactly
which circumstances can the implementation be synthesized?

Looking forward to hearing everyone's thoughts,
Jacob

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Nit: if you want to call it `ValueEnumerable`, then this should be
`DefaultValueCollection`.

More generally though, I can see the appeal of allowing `Int` to conform to
`ValueEnumerable`, but I've got a feeling that this is headed rapidly in
the direction of overengineering a feature without a good rationale for the
additional complexity. The stated use case is to enumerate the cases of an
enum, and any additional complexity above that should stand on its own
merit. I disagree quite vehemently that protocols should be "as general as
possible"; rather, they exist to enable useful generic algorithms and
should be as _useful_ as possible. There is a happy medium beyond which
overengineering the design makes a protocol markedly less
usable/approachable for the sake of enabling rare functionality.

Along the lines of user ergonomics, I would advocate for as many enums as
possible to conform without explicit opt-in. It's true that we are moving
away from such magical designs, and for good reason, but the gain here of
enums Just Working(TM) for such a long-demanded feature has, I would argue,
more benefits than drawbacks. To my mind, the feature is a lot like
`RawRepresentable` in several ways, and it would be defensible for an equal
amount of magic to be enabled for it.

···

On Sat, Nov 11, 2017 at 12:15 AM, Brent Royal-Gordon via swift-evolution < swift-evolution@swift.org> wrote:

> Personally I like the flexibility provided by the associatedtype, but I
also recognize it won't be incredibly useful for enums ‚ÄĒ more so if we
wanted to provide e.g. UInt8.allValues, whose ideal implementation might be
"return 0...UInt8.max". So I could see allowing allValues to be any
sequence or collection, but flexibility for allCases might be less
important. Others should weigh in here.

I think we should allow any `Collection` (option 3), and I think we should
have a `DefaultCaseCollection<Enum>` type in the standard library which
encapsulates the interaction with the runtime.

A `Collection` is good because `Sequence` doesn't provide all of the
guarantees we want‚ÄĒ`allValues` should never be single-pass and it should
always be possible to resume iteration at an earlier point, which
`Sequence` doesn't guarantee. (It probably makes sense to use
`BidirectionalCollection`, actually; I'm less sure about
`RandomAccessCollection`.) At the same time, an `Array` ties us to all
sorts of things that are unnecessary at best and harmful at worst, like a
heap allocation. It also forces us into integer indices, which may be
suboptimal for certain use cases (particularly if we later want to support
associated values). And it prevents us from making types like integers
conform, which I think would be a good idea.

Meanwhile, encapsulating the runtime machinery in `DefaultCaseCollection`
gives users an escape hatch: If the enum you want to use doesn't conform to
`ValueEnumerable`, but you're certain it's compatible, you can construct a
`DefaultCaseCollection` for it. `DefaultCaseCollection` can be a
`RandomAccessCollection` with `Int` indices, making it convenient to use,
but at the same time, it's *not* an array, so it doesn't have to allocate
storage or think about `NSArray` bridging. And it minimizes the complexity
of what the compiler needs to synthesize.

        public protocol ValueEnumerable {
                associatedtype AllValues: BidirectionalCollection where
AllValues.Element == Self
                static var allValues: AllValues { get }
        }

        // The compiler automatically does `typealias AllValues =
DefaultCaseCollection<Self>` if the
        // conformance is on the original declaration, the type is
compatible (e.g. no associated values),
        // and a different type is neither explicitly specified nor
inferred. That will cause this default
        // implementation to be used:
        extension ValueEnumerable where AllValues ==
DefaultCaseCollection<Self> {
                public static var allValues: DefaultCaseCollection<Self> {
                        return DefaultCaseCollection(unsafeForEnum:
Self.self)
                }
        }

        public struct DefaultCaseCollection<Enum>: RandomAccessCollection {
                public var startIndex: Int { return 0 }
                public let endIndex: Int

                public init(unsafeForEnum _: Enum.Type) {
                        endIndex = _countCaseValues(Enum.self)
                }

                public subscript(i: Int) -> Enum {
                        precondition(indices.contains(i), "Case index out
of range")
                        return Builtin.reinterpretCast(i) as Enum
                }
        }

> Personally I like the flexibility provided by the associatedtype, but I
also recognize it won't be incredibly useful for enums ‚ÄĒ more so if we
wanted to provide e.g. UInt8.allValues, whose ideal implementation might be
"return 0...UInt8.max". So I could see allowing allValues to be any
sequence or collection, but flexibility for allCases might be less
important. Others should weigh in here.

I think we should allow any `Collection` (option 3), and I think we
should have a `DefaultCaseCollection<Enum>` type in the standard library
which encapsulates the interaction with the runtime.

A `Collection` is good because `Sequence` doesn't provide all of the
guarantees we want‚ÄĒ`allValues` should never be single-pass and it should
always be possible to resume iteration at an earlier point, which
`Sequence` doesn't guarantee. (It probably makes sense to use
`BidirectionalCollection`, actually; I'm less sure about
`RandomAccessCollection`.) At the same time, an `Array` ties us to all
sorts of things that are unnecessary at best and harmful at worst, like a
heap allocation. It also forces us into integer indices, which may be
suboptimal for certain use cases (particularly if we later want to support
associated values). And it prevents us from making types like integers
conform, which I think would be a good idea.

Meanwhile, encapsulating the runtime machinery in `DefaultCaseCollection`
gives users an escape hatch: If the enum you want to use doesn't conform to
`ValueEnumerable`, but you're certain it's compatible, you can construct a
`DefaultCaseCollection` for it. `DefaultCaseCollection` can be a
`RandomAccessCollection` with `Int` indices, making it convenient to use,
but at the same time, it's *not* an array, so it doesn't have to allocate
storage or think about `NSArray` bridging. And it minimizes the complexity
of what the compiler needs to synthesize.

        public protocol ValueEnumerable {
                associatedtype AllValues: BidirectionalCollection where
AllValues.Element == Self
                static var allValues: AllValues { get }
        }

        // The compiler automatically does `typealias AllValues =
DefaultCaseCollection<Self>` if the
        // conformance is on the original declaration, the type is
compatible (e.g. no associated values),
        // and a different type is neither explicitly specified nor
inferred. That will cause this default
        // implementation to be used:
        extension ValueEnumerable where AllValues ==
DefaultCaseCollection<Self> {
                public static var allValues: DefaultCaseCollection<Self> {
                        return DefaultCaseCollection(unsafeForEnum:
Self.self)
                }
        }

        public struct DefaultCaseCollection<Enum>: RandomAccessCollection
{
                public var startIndex: Int { return 0 }
                public let endIndex: Int

                public init(unsafeForEnum _: Enum.Type) {
                        endIndex = _countCaseValues(Enum.self)
                }

                public subscript(i: Int) -> Enum {
                        precondition(indices.contains(i), "Case index out
of range")
                        return Builtin.reinterpretCast(i) as Enum
                }
        }

Nit: if you want to call it `ValueEnumerable`, then this should be
`DefaultValueCollection`.

More generally though, I can see the appeal of allowing `Int` to conform
to `ValueEnumerable`, but I've got a feeling that this is headed rapidly in
the direction of overengineering a feature without a good rationale for the
additional complexity. The stated use case is to enumerate the cases of an
enum, and any additional complexity above that should stand on its own
merit. I disagree quite vehemently that protocols should be "as general as
possible"; rather, they exist to enable useful generic algorithms and
should be as _useful_ as possible. There is a happy medium beyond which
overengineering the design makes a protocol markedly less
usable/approachable for the sake of enabling rare functionality.

Perhaps "no more specific than they need to be" would have been a better
choice of words on my part than "as general as possible".

I'm not sure why you think this decision makes the protocol "markedly less
usable/approachable". and I think you're seeing a lot of complexity that
isn't there. Regarding good rationale, I've given that in this thread above
and in the previous discussion thread, but I'll summarize it again here:

1) "The stated use case is to enumerate the cases of an enum" isn't
sufficient for designing a protocol because Swift does not have protocols
that are restricted only to enums. Just as anyone can conform their own
types to RawRepresentable, anyone should be able to conform their own types
to ValueEnumerable. And once that's allowed, they should be able to do so
without imposing an overly restrictive API. What if a custom type has
thousands of elements that are easily computed sequentially or from an
index? Forcing an Array on the implementor is an unnecessary restriction
and a burden. (More on this below.)

2) There is *no decrease in usability for the common use case *by choosing
to make the protocol associated type a Sequence instead of a Collection or
Array because the concrete type synthesized by the compiler *would be a
more refined concrete type anyway.* In other words, anyone who writes
"MyEnum.allValues" would get back an integer-indexable random access
collection. I can imagine useful algorithms (more of an
algebraic/set-theoretical nature) that could be written for types with
countably infinite value sets. Why require breaking source/ABI
compatibility later if we can avoid it now, at no cost for the common user?

2.1) Anyone who writes a generic algorithm constrained over ValueEnumerable
*would* have to add an additional constraint of "where T:
RandomAccessCollection, T.Index == Int". This is a slight increase in
complexity, but I would argue that if someone is writing generic algorithms
over collections (and that's what this is), then this is something they
should already familiar with. I argue that that trade-off is
acceptable‚ÄĒothers might disagree.

3) There is *no good reason* for the compiler to perform slow memory
allocation operations to shuffle around information that it *already
knows* when
it can just as easily provide a specialized compact value type that is far
more time/space efficient. I should finish the prototype I was working
on‚ÄĒthe code synthesis is not markedly more difficult than it would be to
write code that constructs an array.

4) Most users *think* they need an array when what they really want is to
be able to get the count of elements, iterate over them, and possibly
integer-index into them. The specialized collection type can do all of
those things more efficiently.

4.1) If a user *truly* needs an array, they just write
"Array(MyEnum.allValues)".

Along the lines of user ergonomics, I would advocate for as many enums as
possible to conform without explicit opt-in. It's true that we are moving
away from such magical designs, and for good reason, but the gain here of
enums Just Working(TM) for such a long-demanded feature has, I would argue,
more benefits than drawbacks. To my mind, the feature is a lot like
`RawRepresentable` in several ways, and it would be defensible for an equal
amount of magic to be enabled for it.

I tend to agree with this philosophically, but the same argument about a
long-demanded feature Just Working‚ĄĘ could be/was made for
Equatable/Hashable being implicit and we decided to make it opt-in instead.
I'm worried that we're going down a road where "this set of protocols is
opt-in and this set of protocols is always-on" and users have to remember
which are which. (And in some cases, it's even more complex! Enums are
always Eq/Hash if they have raw values or no associated values, for
historical reasons but they're opt-in with associated values.)

···

On Fri, Nov 10, 2017 at 11:01 PM Xiaodi Wu via swift-evolution < swift-evolution@swift.org> wrote:

On Sat, Nov 11, 2017 at 12:15 AM, Brent Royal-Gordon via swift-evolution < > swift-evolution@swift.org> wrote:

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Nit: if you want to call it `ValueEnumerable`, then this should be `DefaultValueCollection`.

I used `DefaultCaseCollection` because, although the protocol can work with any type to return its values, this type can only work with enums and only returns their cases. `ValueEnumerable` could be sensibly applied to `Int`; `DefaultCaseCollection` could not.

More generally though, I can see the appeal of allowing `Int` to conform to `ValueEnumerable`, but I've got a feeling that this is headed rapidly in the direction of overengineering a feature without a good rationale for the additional complexity. The stated use case is to enumerate the cases of an enum, and any additional complexity above that should stand on its own merit. I disagree quite vehemently that protocols should be "as general as possible"; rather, they exist to enable useful generic algorithms and should be as _useful_ as possible. There is a happy medium beyond which overengineering the design makes a protocol markedly less usable/approachable for the sake of enabling rare functionality.

Tony's "no more specific than they need to" language applies here. The way I see it is this:

* `Bool` is not an enum, but it could be usefully conformed to `ValueEnumerable`. Why should we prevent that?
* A type with two independently-switchable `Bool`s‚ÄĒsay, `isMirrored` and `isFlipped`‚ÄĒcould be usefully conformed to `ValueEnumerable`. Why should we prevent that?
* Having integer types conform to `ValueEnumerable` with `static let allValues = Self.min...Self.max` could be useful. Why should we prevent that?

And at the same time, a small, specialized collection type _also_ helps with our intended use case in some ways (while admittedly making things more difficult in others). So I think the more general design, which also works better for our intended use case, is the superior option.

Along the lines of user ergonomics, I would advocate for as many enums as possible to conform without explicit opt-in. It's true that we are moving away from such magical designs, and for good reason, but the gain here of enums Just Working(TM) for such a long-demanded feature has, I would argue, more benefits than drawbacks. To my mind, the feature is a lot like `RawRepresentable` in several ways, and it would be defensible for an equal amount of magic to be enabled for it.

But `RawRepresentable` *doesn't* get automatically added to all enums‚ÄĒyou explicitly opt in, albeit using a special sugar syntax. No, I think opt-in is the right answer here. We might be able to justify adding sugar to opt-in, but I can't actually think of a way to make opting in easier than conforming to a protocol and letting the complier synthesize the requirements.

(This is also important for library evolution. A public enum may not want to promise that its values will always be enumerable‚ÄĒfor instance, it might add a case with a non-enumerable associated type in the future. Since conformances are always as public as the type itself, we can't have this feature be opt-out without moving away from using a protocol.)

···

On Nov 10, 2017, at 11:01 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

--
Brent Royal-Gordon
Architechies

> Personally I like the flexibility provided by the associatedtype, but
I also recognize it won't be incredibly useful for enums ‚ÄĒ more so if we
wanted to provide e.g. UInt8.allValues, whose ideal implementation might be
"return 0...UInt8.max". So I could see allowing allValues to be any
sequence or collection, but flexibility for allCases might be less
important. Others should weigh in here.

I think we should allow any `Collection` (option 3), and I think we
should have a `DefaultCaseCollection<Enum>` type in the standard library
which encapsulates the interaction with the runtime.

A `Collection` is good because `Sequence` doesn't provide all of the
guarantees we want‚ÄĒ`allValues` should never be single-pass and it should
always be possible to resume iteration at an earlier point, which
`Sequence` doesn't guarantee. (It probably makes sense to use
`BidirectionalCollection`, actually; I'm less sure about
`RandomAccessCollection`.) At the same time, an `Array` ties us to all
sorts of things that are unnecessary at best and harmful at worst, like a
heap allocation. It also forces us into integer indices, which may be
suboptimal for certain use cases (particularly if we later want to support
associated values). And it prevents us from making types like integers
conform, which I think would be a good idea.

Meanwhile, encapsulating the runtime machinery in
`DefaultCaseCollection` gives users an escape hatch: If the enum you want
to use doesn't conform to `ValueEnumerable`, but you're certain it's
compatible, you can construct a `DefaultCaseCollection` for it.
`DefaultCaseCollection` can be a `RandomAccessCollection` with `Int`
indices, making it convenient to use, but at the same time, it's *not* an
array, so it doesn't have to allocate storage or think about `NSArray`
bridging. And it minimizes the complexity of what the compiler needs to
synthesize.

        public protocol ValueEnumerable {
                associatedtype AllValues: BidirectionalCollection where
AllValues.Element == Self
                static var allValues: AllValues { get }
        }

        // The compiler automatically does `typealias AllValues =
DefaultCaseCollection<Self>` if the
        // conformance is on the original declaration, the type is
compatible (e.g. no associated values),
        // and a different type is neither explicitly specified nor
inferred. That will cause this default
        // implementation to be used:
        extension ValueEnumerable where AllValues ==
DefaultCaseCollection<Self> {
                public static var allValues: DefaultCaseCollection<Self>
{
                        return DefaultCaseCollection(unsafeForEnum:
Self.self)
                }
        }

        public struct DefaultCaseCollection<Enum>:
RandomAccessCollection {
                public var startIndex: Int { return 0 }
                public let endIndex: Int

                public init(unsafeForEnum _: Enum.Type) {
                        endIndex = _countCaseValues(Enum.self)
                }

                public subscript(i: Int) -> Enum {
                        precondition(indices.contains(i), "Case index
out of range")
                        return Builtin.reinterpretCast(i) as Enum
                }
        }

Nit: if you want to call it `ValueEnumerable`, then this should be
`DefaultValueCollection`.

More generally though, I can see the appeal of allowing `Int` to conform
to `ValueEnumerable`, but I've got a feeling that this is headed rapidly in
the direction of overengineering a feature without a good rationale for the
additional complexity. The stated use case is to enumerate the cases of an
enum, and any additional complexity above that should stand on its own
merit. I disagree quite vehemently that protocols should be "as general as
possible"; rather, they exist to enable useful generic algorithms and
should be as _useful_ as possible. There is a happy medium beyond which
overengineering the design makes a protocol markedly less
usable/approachable for the sake of enabling rare functionality.

Perhaps "no more specific than they need to be" would have been a better
choice of words on my part than "as general as possible".

I'm not sure why you think this decision makes the protocol "markedly less
usable/approachable". and I think you're seeing a lot of complexity that
isn't there. Regarding good rationale, I've given that in this thread above
and in the previous discussion thread, but I'll summarize it again here:

1) "The stated use case is to enumerate the cases of an enum" isn't
sufficient for designing a protocol because Swift does not have protocols
that are restricted only to enums. Just as anyone can conform their own
types to RawRepresentable, anyone should be able to conform their own types
to ValueEnumerable.

This is the part of the argument that I am questioning. If the desired use
case is to enumerate the cases of an enum, is it much of a gain to allow
anyone to be able to conform their own non-enum types to this protocol?
Yes, if you buy that, then much of the rest follows. But make no mistake
that it is not necessary to serve the desired use case and dramatically
increases the complexity of the design (see below for just some
considerations).

And once that's allowed, they should be able to do so without imposing an
overly restrictive API. What if a custom type has thousands of elements
that are easily computed sequentially or from an index? Forcing an Array on
the implementor is an unnecessary restriction and a burden. (More on this
below.)

2) There is *no decrease in usability for the common use case *by
choosing to make the protocol associated type a Sequence instead of a
Collection or Array because the concrete type synthesized by the compiler *would
be a more refined concrete type anyway.* In other words, anyone who
writes "MyEnum.allValues" would get back an integer-indexable random access
collection. I can imagine useful algorithms (more of an
algebraic/set-theoretical nature) that could be written for types with
countably infinite value sets. Why require breaking source/ABI
compatibility later if we can avoid it now, at no cost for the common user?

2.1) Anyone who writes a generic algorithm constrained over
ValueEnumerable *would* have to add an additional constraint of "where T:
RandomAccessCollection, T.Index == Int". This is a slight increase in
complexity, but I would argue that if someone is writing generic algorithms
over collections (and that's what this is), then this is something they
should already familiar with. I argue that that trade-off is
acceptable‚ÄĒothers might disagree.

3) There is *no good reason* for the compiler to perform slow memory
allocation operations to shuffle around information that it *already
knows* when it can just as easily provide a specialized compact value
type that is far more time/space efficient. I should finish the prototype I
was working on‚ÄĒthe code synthesis is not markedly more difficult than it
would be to write code that constructs an array.

4) Most users *think* they need an array when what they really want is to
be able to get the count of elements, iterate over them, and possibly
integer-index into them. The specialized collection type can do all of
those things more efficiently.

4.1) If a user *truly* needs an array, they just write
"Array(MyEnum.allValues)".

Along the lines of user ergonomics, I would advocate for as many enums as
possible to conform without explicit opt-in. It's true that we are moving
away from such magical designs, and for good reason, but the gain here of
enums Just Working(TM) for such a long-demanded feature has, I would argue,
more benefits than drawbacks. To my mind, the feature is a lot like
`RawRepresentable` in several ways, and it would be defensible for an equal
amount of magic to be enabled for it.

I tend to agree with this philosophically, but the same argument about a
long-demanded feature Just Working‚ĄĘ could be/was made for
Equatable/Hashable being implicit and we decided to make it opt-in instead.
I'm worried that we're going down a road where "this set of protocols is
opt-in and this set of protocols is always-on" and users have to remember
which are which. (And in some cases, it's even more complex! Enums are
always Eq/Hash if they have raw values or no associated values, for
historical reasons but they're opt-in with associated values.)

There are key differences here. Equatable and Hashable have extensive
semantic guarantees, and it is certainly not the case that a type with
members that are all Equatable fulfills those guarantees itself. It would
be undesirable to have implicit Equatable conformance because the compiler
cannot guarantee semantics and must not make such assumptions. (The
implicit conformance of certain enums to Equatable and Hashable is
interesting yet defensible because it relies on an aspect of those enums
not shared with other types, an aspect that can be profitably relied
upon--see below.) All enums that have one or more cases with no associated
values, by contrast, have an enumerable set of cases. This is something
knowable to the compiler.

This goes to the issue above of whether it is desirable in the first place
to conflate enums that have enumerable cases with a more general
"ValueEnumerable"; the requested feature is to enumerate enum cases, and
having a set of unique cases is a feature of enums not shared with other
types. Inevitably, when you generalize, then you must ask what it means for
a non-enum type to have a set of enumerable case-like values. For example,
note that there is no particular need for an ordered collection if we are
considering enums (cf. previous conversations about supporting resilient
reordering of enum cases); by contrast, such a design would be highly
unintuitive for a range, which has an implicitly ordered set of values.
Since each enum case is guaranteed to be unique, and since Equatable and
Hashable conformance is implicitly synthesized in such a way that all cases
compare not equal to other cases, we could even use a Set for an enum's
`allCases`. Of course, to eliminate the potential need for allocating
memory, this could be a custom type with set-like semantics, but
nonetheless it allows us to guarantee that `Set(T.allCases).count ==
T.allCases.count`, which would be a nontrivial semantic requirement in the
more general case of custom `ValueEnumerable` conformance that the author
of the conforming type would need to consider--if we want to insist on that
semantic requirement at all.

···

On Sat, Nov 11, 2017 at 11:23 AM, Tony Allevato <tony.allevato@gmail.com> wrote:

On Fri, Nov 10, 2017 at 11:01 PM Xiaodi Wu via swift-evolution < > swift-evolution@swift.org> wrote:

On Sat, Nov 11, 2017 at 12:15 AM, Brent Royal-Gordon via swift-evolution >> <swift-evolution@swift.org> wrote:

Nit: if you want to call it `ValueEnumerable`, then this should be
`DefaultValueCollection`.

More generally though, I can see the appeal of allowing `Int` to conform
to `ValueEnumerable`, but I've got a feeling that this is headed rapidly in
the direction of overengineering a feature without a good rationale for the
additional complexity. The stated use case is to enumerate the cases of an
enum, and any additional complexity above that should stand on its own
merit. I disagree quite vehemently that protocols should be "as general as
possible"; rather, they exist to enable useful generic algorithms and
should be as _useful_ as possible. There is a happy medium beyond which
overengineering the design makes a protocol markedly less
usable/approachable for the sake of enabling rare functionality.

Perhaps "no more specific than they need to be" would have been a better
choice of words on my part than "as general as possible".

I'm not sure why you think this decision makes the protocol "markedly
less usable/approachable". and I think you're seeing a lot of complexity
that isn't there. Regarding good rationale, I've given that in this thread
above and in the previous discussion thread, but I'll summarize it again
here:

1) "The stated use case is to enumerate the cases of an enum" isn't
sufficient for designing a protocol because Swift does not have protocols
that are restricted only to enums. Just as anyone can conform their own
types to RawRepresentable, anyone should be able to conform their own types
to ValueEnumerable.

This is the part of the argument that I am questioning. If the desired use
case is to enumerate the cases of an enum, is it much of a gain to allow
anyone to be able to conform their own non-enum types to this protocol?
Yes, if you buy that, then much of the rest follows. But make no mistake
that it is not necessary to serve the desired use case and dramatically
increases the complexity of the design (see below for just some
considerations).

Some increase in complexity is certainly going to be introduced by
generalizing a design to make it useful to more than just a single use
case, but I hope you understand that using unquantifiable and subjective
qualifiers like "markedly less usable" or "dramatically increases the
complexity" is a bit unhelpful when trying to debate specifics. You clearly
put a great deal of thought into design problems like this and I always
appreciate your points of view, so I want to focus on the objective parts.

In prior threads on this topic, I believe I recall you saying that a static
property that returns an array is fine. (I don't remember your exact words,
so please correct me if I'm wrong. I don't want to misstate your position.)
Given the concerns I've pointed out about why that is both a time and
memory performance problem, and given your focus on the common use case
above, I'd appreciate your opinion on these points:

* Are you advocating for a magic protocol that *only* enums can conform to?
(This is a different issue than whether the protocol is just a
compiler-known protocol that it can synthesize.)
    * If yes, why intentionally restrict it others from implementing the
protocol? That's actually *more* complexity than allowing it, like we do
with RawRepresentable. There's also no precedent for "a protocol that can
only be applied to enums" in the Swift language. The ramifications of that
decision introduce their own design complexity‚ÄĒyou're just shifting it
around, not avoiding it.
    * If no, then I don't think we differ on this point.

* Do you think that the type of this property should still be an array,
with all of the performance problems that it brings, or would you find a
sufficiently array-like but more performant type to be fine?

And once that's allowed, they should be able to do so without imposing an
overly restrictive API. What if a custom type has thousands of elements
that are easily computed sequentially or from an index? Forcing an Array on
the implementor is an unnecessary restriction and a burden. (More on this
below.)

2) There is *no decrease in usability for the common use case *by
choosing to make the protocol associated type a Sequence instead of a
Collection or Array because the concrete type synthesized by the compiler *would
be a more refined concrete type anyway.* In other words, anyone who
writes "MyEnum.allValues" would get back an integer-indexable random access
collection. I can imagine useful algorithms (more of an
algebraic/set-theoretical nature) that could be written for types with
countably infinite value sets. Why require breaking source/ABI
compatibility later if we can avoid it now, at no cost for the common user?

2.1) Anyone who writes a generic algorithm constrained over
ValueEnumerable *would* have to add an additional constraint of "where
T: RandomAccessCollection, T.Index == Int". This is a slight increase in
complexity, but I would argue that if someone is writing generic algorithms
over collections (and that's what this is), then this is something they
should already familiar with. I argue that that trade-off is
acceptable‚ÄĒothers might disagree.

3) There is *no good reason* for the compiler to perform slow memory
allocation operations to shuffle around information that it *already
knows* when it can just as easily provide a specialized compact value
type that is far more time/space efficient. I should finish the prototype I
was working on‚ÄĒthe code synthesis is not markedly more difficult than it
would be to write code that constructs an array.

4) Most users *think* they need an array when what they really want is
to be able to get the count of elements, iterate over them, and possibly
integer-index into them. The specialized collection type can do all of
those things more efficiently.

4.1) If a user *truly* needs an array, they just write
"Array(MyEnum.allValues)".

Along the lines of user ergonomics, I would advocate for as many enums
as possible to conform without explicit opt-in. It's true that we are
moving away from such magical designs, and for good reason, but the gain
here of enums Just Working(TM) for such a long-demanded feature has, I
would argue, more benefits than drawbacks. To my mind, the feature is a lot
like `RawRepresentable` in several ways, and it would be defensible for an
equal amount of magic to be enabled for it.

I tend to agree with this philosophically, but the same argument about a
long-demanded feature Just Working‚ĄĘ could be/was made for
Equatable/Hashable being implicit and we decided to make it opt-in instead.
I'm worried that we're going down a road where "this set of protocols is
opt-in and this set of protocols is always-on" and users have to remember
which are which. (And in some cases, it's even more complex! Enums are
always Eq/Hash if they have raw values or no associated values, for
historical reasons but they're opt-in with associated values.)

There are key differences here. Equatable and Hashable have extensive
semantic guarantees, and it is certainly not the case that a type with
members that are all Equatable fulfills those guarantees itself. It would
be undesirable to have implicit Equatable conformance because the compiler
cannot guarantee semantics and must not make such assumptions. (The
implicit conformance of certain enums to Equatable and Hashable is
interesting yet defensible because it relies on an aspect of those enums
not shared with other types, an aspect that can be profitably relied
upon--see below.) All enums that have one or more cases with no associated
values, by contrast, have an enumerable set of cases. This is something
knowable to the compiler.

I agree with this. I think an argument could be successfully made that
since enums without associated values already conform to Equatable/Hashable
implicitly, and enums with raw values conform to RawRepresentable
implicitly, the same could be done for ValueEnumerable. But, I brought it
up because it's easy to imagine that there would be concerns raised about
adding more of these special cases to the language (which does add design
complexity, right?).

This goes to the issue above of whether it is desirable in the first place

to conflate enums that have enumerable cases with a more general
"ValueEnumerable"; the requested feature is to enumerate enum cases, and
having a set of unique cases is a feature of enums not shared with other
types. Inevitably, when you generalize, then you must ask what it means for
a non-enum type to have a set of enumerable case-like values. For example,
note that there is no particular need for an ordered collection if we are
considering enums (cf. previous conversations about supporting resilient
reordering of enum cases); by contrast, such a design would be highly
unintuitive for a range, which has an implicitly ordered set of values.
Since each enum case is guaranteed to be unique, and since Equatable and
Hashable conformance is implicitly synthesized in such a way that all cases
compare not equal to other cases, we could even use a Set for an enum's
`allCases`. Of course, to eliminate the potential need for allocating
memory, this could be a custom type with set-like semantics, but
nonetheless it allows us to guarantee that `Set(T.allCases).count ==
T.allCases.count`, which would be a nontrivial semantic requirement in the
more general case of custom `ValueEnumerable` conformance that the author
of the conforming type would need to consider--if we want to insist on that
semantic requirement at all.

These are great points! Regarding the ordering question‚ÄĒI agree that purely
theoretically, the cases of an enum are just an unordered set of values.
But, just as you are interested in meeting the needs of end users, so am
I‚ÄĒand one of the situations where I commonly see this feature requested is
to build UIs (menus, table view sections, etc.) from the cases of an enum.
Any design of this feature that would require users to specify the ordering
of their elements secondarily to the order they appear in source code would
make this feature useless for them, and those are the people we're trying
to help.

So, I do accept an ordered collection as a compromise that makes the
feature usable even at the cost of losing some amount of generality. But
that's fine‚ÄĒall of these design problems are on a spectrum, from highly
specific on one end to highly general on the other. The main thing I want
to point out with this discussion is that we *can* make the design a bit
more general without harming usability for the common use case *and* make
it future-proof so we're not shutting out other potential use cases that
are harder to fix due to source/ABI compatibility or adding odd special
cases to the language.

···

On Sat, Nov 11, 2017 at 10:28 AM Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Sat, Nov 11, 2017 at 11:23 AM, Tony Allevato <tony.allevato@gmail.com> > wrote:

On Fri, Nov 10, 2017 at 11:01 PM Xiaodi Wu via swift-evolution < >> swift-evolution@swift.org> wrote:

Nit: if you want to call it `ValueEnumerable`, then this should be
`DefaultValueCollection`.

I used `DefaultCaseCollection` because, although the protocol can work
with any type to return its values, this type can only work with enums and
only returns their cases. `ValueEnumerable` could be sensibly applied to
`Int`; `DefaultCaseCollection` could not.

Because of how you've chosen to implement `DefaultCaseCollection`, or do
you mean to say that you deliberately want a design where types other than
enums do not share the default return type?

More generally though, I can see the appeal of allowing `Int` to conform
to `ValueEnumerable`, but I've got a feeling that this is headed rapidly in
the direction of overengineering a feature without a good rationale for the
additional complexity. The stated use case is to enumerate the cases of an
enum, and any additional complexity above that should stand on its own
merit. I disagree quite vehemently that protocols should be "as general as
possible"; rather, they exist to enable useful generic algorithms and
should be as _useful_ as possible. There is a happy medium beyond which
overengineering the design makes a protocol markedly less
usable/approachable for the sake of enabling rare functionality.

Tony's "no more specific than they need to" language applies here. The way
I see it is this:

* `Bool` is not an enum, but it could be usefully conformed to
`ValueEnumerable`. Why should we prevent that?
* A type with two independently-switchable `Bool`s‚ÄĒsay, `isMirrored` and
`isFlipped`‚ÄĒcould be usefully conformed to `ValueEnumerable`. Why should we
prevent that?
* Having integer types conform to `ValueEnumerable` with `static let
allValues = Self.min...Self.max` could be useful. Why should we prevent
that?

I'd say you're looking at it the wrong way. We're not *preventing*
anything. We're adding a feature, and the question is, why should we *add*
more than is justified by the use case?

Is there a clamor for enumerating the two possible values of `Bool`? If so,
is that not an argument instead to make `Bool` an enum and not a struct
under the hood?
Is there a clamor for enumerating the possible values of a type with two
independently switchable `Bool`s? If so, is that not an argument to make
`Bool` a valid raw value type? (There is already a bug report to make
tuples of raw value types valid raw value types themselves.)
Why is it useful for (fixed-width) integer types to conform to
`ValueEnumerable`? What use cases, exactly, would that enable that are not
possible now?

And at the same time, a small, specialized collection type _also_ helps
with our intended use case in some ways (while admittedly making things
more difficult in others). So I think the more general design, which also
works better for our intended use case, is the superior option.

Along the lines of user ergonomics, I would advocate for as many enums as
possible to conform without explicit opt-in. It's true that we are moving
away from such magical designs, and for good reason, but the gain here of
enums Just Working(TM) for such a long-demanded feature has, I would argue,
more benefits than drawbacks. To my mind, the feature is a lot like
`RawRepresentable` in several ways, and it would be defensible for an equal
amount of magic to be enabled for it.

But `RawRepresentable` *doesn't* get automatically added to all enums‚ÄĒyou
explicitly opt in, albeit using a special sugar syntax. No, I think opt-in
is the right answer here. We might be able to justify adding sugar to
opt-in, but I can't actually think of a way to make opting in easier than
conforming to a protocol and letting the complier synthesize the
requirements.

Yes, you're right that `RawRepresentable` conformance *doesn't* get
automatically added in, but there exists special sugar which makes the end
result indistinguishable. By this I mean that the user gets
`RawRepresentable` conformance without ever writing `Foo :
RawRepresentable` anywhere (and neither do they write `Foo : Bar` where
`Bar` is in turn `RawRepresentable`).

(This is also important for library evolution. A public enum may not want

to promise that its values will always be enumerable‚ÄĒfor instance, it might
add a case with a non-enumerable associated type in the future. Since
conformances are always as public as the type itself, we can't have this
feature be opt-out without moving away from using a protocol.)

This is, in fact, a perfect opportunity to bring up a question I've been
leaving implicit. Why not explore moving away from using a protocol? The
proposed protocol has no syntactic requirements, and when constrained only
the use case of enums, it has essentially no semantic requirements either.
It seems that it's only because we've committed to using a protocol that
we've opened up this exploration of what it means semantically to be
`ValueEnumerable`, and how to generalize it to other types, and how to
design the return type of the synthesized function, etc.

What if some attribute would simply make the metatype conform to `Sequence`
(or, for that matter, `BidirectionalCollection`)?

···

On Sun, Nov 12, 2017 at 4:54 AM, Brent Royal-Gordon <brent@architechies.com> wrote:

On Nov 10, 2017, at 11:01 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

Nit: if you want to call it `ValueEnumerable`, then this should be
`DefaultValueCollection`.

More generally though, I can see the appeal of allowing `Int` to
conform to `ValueEnumerable`, but I've got a feeling that this is headed
rapidly in the direction of overengineering a feature without a good
rationale for the additional complexity. The stated use case is to
enumerate the cases of an enum, and any additional complexity above that
should stand on its own merit. I disagree quite vehemently that protocols
should be "as general as possible"; rather, they exist to enable useful
generic algorithms and should be as _useful_ as possible. There is a happy
medium beyond which overengineering the design makes a protocol markedly
less usable/approachable for the sake of enabling rare functionality.

Perhaps "no more specific than they need to be" would have been a better
choice of words on my part than "as general as possible".

I'm not sure why you think this decision makes the protocol "markedly
less usable/approachable". and I think you're seeing a lot of complexity
that isn't there. Regarding good rationale, I've given that in this thread
above and in the previous discussion thread, but I'll summarize it again
here:

1) "The stated use case is to enumerate the cases of an enum" isn't
sufficient for designing a protocol because Swift does not have protocols
that are restricted only to enums. Just as anyone can conform their own
types to RawRepresentable, anyone should be able to conform their own types
to ValueEnumerable.

This is the part of the argument that I am questioning. If the desired
use case is to enumerate the cases of an enum, is it much of a gain to
allow anyone to be able to conform their own non-enum types to this
protocol? Yes, if you buy that, then much of the rest follows. But make no
mistake that it is not necessary to serve the desired use case and
dramatically increases the complexity of the design (see below for just
some considerations).

Some increase in complexity is certainly going to be introduced by
generalizing a design to make it useful to more than just a single use
case, but I hope you understand that using unquantifiable and subjective
qualifiers like "markedly less usable" or "dramatically increases the
complexity" is a bit unhelpful when trying to debate specifics. You clearly
put a great deal of thought into design problems like this and I always
appreciate your points of view, so I want to focus on the objective parts.

In prior threads on this topic, I believe I recall you saying that a
static property that returns an array is fine. (I don't remember your exact
words, so please correct me if I'm wrong. I don't want to misstate your
position.) Given the concerns I've pointed out about why that is both a
time and memory performance problem, and given your focus on the common use
case above, I'd appreciate your opinion on these points:

* Are you advocating for a magic protocol that *only* enums can conform
to? (This is a different issue than whether the protocol is just a
compiler-known protocol that it can synthesize.)
    * If yes, why intentionally restrict it others from implementing the
protocol? That's actually *more* complexity than allowing it, like we do
with RawRepresentable. There's also no precedent for "a protocol that can
only be applied to enums" in the Swift language. The ramifications of that
decision introduce their own design complexity‚ÄĒyou're just shifting it
around, not avoiding it.
    * If no, then I don't think we differ on this point.

No magic. Consider: Does the compiler stop me from conforming `UIButton` to
`Error`? No. Can `UIButton` conform to `Error`? Semantically, no.

I am arguing that the use case justifies a protocol called "CaseEnumerable"
with documented semantics such that only types with cases (i.e., enums with
one or more cases) would fulfill those semantics. Any more generalization
and we are designing without a justification. I'm unsure what design
complexity you anticipate with such a definition of "CaseEnumerable."

* Do you think that the type of this property should still be an array,

with all of the performance problems that it brings, or would you find a
sufficiently array-like but more performant type to be fine?

I have no problem with any suitable type if it can be justified, say, on
the grounds of performance or memory efficiency. I do have a problem if the
justification for a design is that it might be necessary to accommodate
some unspecified future use case with a currently non-existent type.

And once that's allowed, they should be able to do so without imposing an

overly restrictive API. What if a custom type has thousands of elements
that are easily computed sequentially or from an index? Forcing an Array on
the implementor is an unnecessary restriction and a burden. (More on this
below.)

2) There is *no decrease in usability for the common use case *by
choosing to make the protocol associated type a Sequence instead of a
Collection or Array because the concrete type synthesized by the compiler *would
be a more refined concrete type anyway.* In other words, anyone who
writes "MyEnum.allValues" would get back an integer-indexable random access
collection. I can imagine useful algorithms (more of an
algebraic/set-theoretical nature) that could be written for types with
countably infinite value sets. Why require breaking source/ABI
compatibility later if we can avoid it now, at no cost for the common user?

2.1) Anyone who writes a generic algorithm constrained over
ValueEnumerable *would* have to add an additional constraint of "where
T: RandomAccessCollection, T.Index == Int". This is a slight increase in
complexity, but I would argue that if someone is writing generic algorithms
over collections (and that's what this is), then this is something they
should already familiar with. I argue that that trade-off is
acceptable‚ÄĒothers might disagree.

3) There is *no good reason* for the compiler to perform slow memory
allocation operations to shuffle around information that it *already
knows* when it can just as easily provide a specialized compact value
type that is far more time/space efficient. I should finish the prototype I
was working on‚ÄĒthe code synthesis is not markedly more difficult than it
would be to write code that constructs an array.

4) Most users *think* they need an array when what they really want is
to be able to get the count of elements, iterate over them, and possibly
integer-index into them. The specialized collection type can do all of
those things more efficiently.

4.1) If a user *truly* needs an array, they just write
"Array(MyEnum.allValues)".

Along the lines of user ergonomics, I would advocate for as many enums
as possible to conform without explicit opt-in. It's true that we are
moving away from such magical designs, and for good reason, but the gain
here of enums Just Working(TM) for such a long-demanded feature has, I
would argue, more benefits than drawbacks. To my mind, the feature is a lot
like `RawRepresentable` in several ways, and it would be defensible for an
equal amount of magic to be enabled for it.

I tend to agree with this philosophically, but the same argument about a
long-demanded feature Just Working‚ĄĘ could be/was made for
Equatable/Hashable being implicit and we decided to make it opt-in instead.
I'm worried that we're going down a road where "this set of protocols is
opt-in and this set of protocols is always-on" and users have to remember
which are which. (And in some cases, it's even more complex! Enums are
always Eq/Hash if they have raw values or no associated values, for
historical reasons but they're opt-in with associated values.)

There are key differences here. Equatable and Hashable have extensive
semantic guarantees, and it is certainly not the case that a type with
members that are all Equatable fulfills those guarantees itself. It would
be undesirable to have implicit Equatable conformance because the compiler
cannot guarantee semantics and must not make such assumptions. (The
implicit conformance of certain enums to Equatable and Hashable is
interesting yet defensible because it relies on an aspect of those enums
not shared with other types, an aspect that can be profitably relied
upon--see below.) All enums that have one or more cases with no associated
values, by contrast, have an enumerable set of cases. This is something
knowable to the compiler.

I agree with this. I think an argument could be successfully made that
since enums without associated values already conform to Equatable/Hashable
implicitly, and enums with raw values conform to RawRepresentable
implicitly, the same could be done for ValueEnumerable. But, I brought it
up because it's easy to imagine that there would be concerns raised about
adding more of these special cases to the language (which does add design
complexity, right?).

This goes to the issue above of whether it is desirable in the first place

to conflate enums that have enumerable cases with a more general
"ValueEnumerable"; the requested feature is to enumerate enum cases, and
having a set of unique cases is a feature of enums not shared with other
types. Inevitably, when you generalize, then you must ask what it means for
a non-enum type to have a set of enumerable case-like values. For example,
note that there is no particular need for an ordered collection if we are
considering enums (cf. previous conversations about supporting resilient
reordering of enum cases); by contrast, such a design would be highly
unintuitive for a range, which has an implicitly ordered set of values.
Since each enum case is guaranteed to be unique, and since Equatable and
Hashable conformance is implicitly synthesized in such a way that all cases
compare not equal to other cases, we could even use a Set for an enum's
`allCases`. Of course, to eliminate the potential need for allocating
memory, this could be a custom type with set-like semantics, but
nonetheless it allows us to guarantee that `Set(T.allCases).count ==
T.allCases.count`, which would be a nontrivial semantic requirement in the
more general case of custom `ValueEnumerable` conformance that the author
of the conforming type would need to consider--if we want to insist on that
semantic requirement at all.

These are great points! Regarding the ordering question‚ÄĒI agree that
purely theoretically, the cases of an enum are just an unordered set of
values. But, just as you are interested in meeting the needs of end users,
so am I‚ÄĒand one of the situations where I commonly see this feature
requested is to build UIs (menus, table view sections, etc.) from the cases
of an enum. Any design of this feature that would require users to specify
the ordering of their elements secondarily to the order they appear in
source code would make this feature useless for them, and those are the
people we're trying to help.

I'm not sure I understand this use case. Are you saying that it is
desirable *not* to require users to specify an order for the options in
their UI? On the contrary, that would seem to be extremely brittle; as a
library author, I might rearrange the order of cases in source code--which
should be a completely transparent change--and this would break end users'
UI. If the final design for ABI stability accommodates resilient reordering
of cases, then the UI would change depending on which version of the
library is installed at any particular moment. I would imagine that we
should explicitly discourage any reliance on source code order of cases. If
a user wants to get an array, then that is simple enough, as you say.

So, I do accept an ordered collection as a compromise that makes the

···

On Sat, Nov 11, 2017 at 3:15 PM, Tony Allevato <tony.allevato@gmail.com> wrote:

On Sat, Nov 11, 2017 at 10:28 AM Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Sat, Nov 11, 2017 at 11:23 AM, Tony Allevato <tony.allevato@gmail.com> >> wrote:

On Fri, Nov 10, 2017 at 11:01 PM Xiaodi Wu via swift-evolution < >>> swift-evolution@swift.org> wrote:

feature usable even at the cost of losing some amount of generality. But
that's fine‚ÄĒall of these design problems are on a spectrum, from highly
specific on one end to highly general on the other. The main thing I want
to point out with this discussion is that we *can* make the design a bit
more general without harming usability for the common use case *and* make
it future-proof so we're not shutting out other potential use cases that
are harder to fix due to source/ABI compatibility or adding odd special
cases to the language.

Nit: if you want to call it `ValueEnumerable`, then this should be
`DefaultValueCollection`.

More generally though, I can see the appeal of allowing `Int` to
conform to `ValueEnumerable`, but I've got a feeling that this is headed
rapidly in the direction of overengineering a feature without a good
rationale for the additional complexity. The stated use case is to
enumerate the cases of an enum, and any additional complexity above that
should stand on its own merit. I disagree quite vehemently that protocols
should be "as general as possible"; rather, they exist to enable useful
generic algorithms and should be as _useful_ as possible. There is a happy
medium beyond which overengineering the design makes a protocol markedly
less usable/approachable for the sake of enabling rare functionality.

Perhaps "no more specific than they need to be" would have been a
better choice of words on my part than "as general as possible".

I'm not sure why you think this decision makes the protocol "markedly
less usable/approachable". and I think you're seeing a lot of complexity
that isn't there. Regarding good rationale, I've given that in this thread
above and in the previous discussion thread, but I'll summarize it again
here:

1) "The stated use case is to enumerate the cases of an enum" isn't
sufficient for designing a protocol because Swift does not have protocols
that are restricted only to enums. Just as anyone can conform their own
types to RawRepresentable, anyone should be able to conform their own types
to ValueEnumerable.

This is the part of the argument that I am questioning. If the desired
use case is to enumerate the cases of an enum, is it much of a gain to
allow anyone to be able to conform their own non-enum types to this
protocol? Yes, if you buy that, then much of the rest follows. But make no
mistake that it is not necessary to serve the desired use case and
dramatically increases the complexity of the design (see below for just
some considerations).

Some increase in complexity is certainly going to be introduced by
generalizing a design to make it useful to more than just a single use
case, but I hope you understand that using unquantifiable and subjective
qualifiers like "markedly less usable" or "dramatically increases the
complexity" is a bit unhelpful when trying to debate specifics. You clearly
put a great deal of thought into design problems like this and I always
appreciate your points of view, so I want to focus on the objective parts.

In prior threads on this topic, I believe I recall you saying that a
static property that returns an array is fine. (I don't remember your exact
words, so please correct me if I'm wrong. I don't want to misstate your
position.) Given the concerns I've pointed out about why that is both a
time and memory performance problem, and given your focus on the common use
case above, I'd appreciate your opinion on these points:

* Are you advocating for a magic protocol that *only* enums can conform
to? (This is a different issue than whether the protocol is just a
compiler-known protocol that it can synthesize.)
    * If yes, why intentionally restrict it others from implementing the
protocol? That's actually *more* complexity than allowing it, like we do
with RawRepresentable. There's also no precedent for "a protocol that can
only be applied to enums" in the Swift language. The ramifications of that
decision introduce their own design complexity‚ÄĒyou're just shifting it
around, not avoiding it.
    * If no, then I don't think we differ on this point.

No magic. Consider: Does the compiler stop me from conforming `UIButton`
to `Error`? No. Can `UIButton` conform to `Error`? Semantically, no.

I am arguing that the use case justifies a protocol called
"CaseEnumerable" with documented semantics such that only types with cases
(i.e., enums with one or more cases) would fulfill those semantics. Any
more generalization and we are designing without a justification. I'm
unsure what design complexity you anticipate with such a definition of
"CaseEnumerable."

I don't anticipate any design complexity with that definition. I also don't
see where the design complexity allegedly exists with the more general
version‚ÄĒyou've claimed that it exists but haven't shown what it is or
proved why it's harmful. The enum's implementation of the protocol would do
the same thing you're describing regardless of the associated type‚ÄĒonly the
protocol would be more general, which would allow other non-enum types to
represent their collection or sequence of (possibly infinite) values. You
keep claiming that this is harmful, but I haven't seen any concrete reasons
why‚ÄĒjust that there is "no justification". Why is "no justification"
sufficient for arguing against over-generalizing something but it's not
sufficient for arguing against over-specializing something?

For what it's worth, if enough people in the community think that
constraining the associated type of this hypothetical protocol to a
Collection type instead of Sequence is the way it should be, fine‚ÄĒI'd
disagree that it's a necessary restriction (because I feel I've shown
sufficiently that it doesn't harm the design), but it's not something that
I would want to stand in the way of the feature as a whole.

* Do you think that the type of this property should still be an array,

with all of the performance problems that it brings, or would you find a
sufficiently array-like but more performant type to be fine?

I have no problem with any suitable type if it can be justified, say, on
the grounds of performance or memory efficiency. I do have a problem if the
justification for a design is that it might be necessary to accommodate
some unspecified future use case with a currently non-existent type.

Then hopefully I've already shown that using a custom no-heap-allocation
collection type in my previous messages on this topic is a performance win
vs. allocating and populating an array.

And once that's allowed, they should be able to do so without imposing an

overly restrictive API. What if a custom type has thousands of elements
that are easily computed sequentially or from an index? Forcing an Array on
the implementor is an unnecessary restriction and a burden. (More on this
below.)

2) There is *no decrease in usability for the common use case *by
choosing to make the protocol associated type a Sequence instead of a
Collection or Array because the concrete type synthesized by the compiler *would
be a more refined concrete type anyway.* In other words, anyone who
writes "MyEnum.allValues" would get back an integer-indexable random access
collection. I can imagine useful algorithms (more of an
algebraic/set-theoretical nature) that could be written for types with
countably infinite value sets. Why require breaking source/ABI
compatibility later if we can avoid it now, at no cost for the common user?

2.1) Anyone who writes a generic algorithm constrained over
ValueEnumerable *would* have to add an additional constraint of "where
T: RandomAccessCollection, T.Index == Int". This is a slight increase in
complexity, but I would argue that if someone is writing generic algorithms
over collections (and that's what this is), then this is something they
should already familiar with. I argue that that trade-off is
acceptable‚ÄĒothers might disagree.

3) There is *no good reason* for the compiler to perform slow memory
allocation operations to shuffle around information that it *already
knows* when it can just as easily provide a specialized compact value
type that is far more time/space efficient. I should finish the prototype I
was working on‚ÄĒthe code synthesis is not markedly more difficult than it
would be to write code that constructs an array.

4) Most users *think* they need an array when what they really want is
to be able to get the count of elements, iterate over them, and possibly
integer-index into them. The specialized collection type can do all of
those things more efficiently.

4.1) If a user *truly* needs an array, they just write
"Array(MyEnum.allValues)".

Along the lines of user ergonomics, I would advocate for as many enums
as possible to conform without explicit opt-in. It's true that we are
moving away from such magical designs, and for good reason, but the gain
here of enums Just Working(TM) for such a long-demanded feature has, I
would argue, more benefits than drawbacks. To my mind, the feature is a lot
like `RawRepresentable` in several ways, and it would be defensible for an
equal amount of magic to be enabled for it.

I tend to agree with this philosophically, but the same argument about
a long-demanded feature Just Working‚ĄĘ could be/was made for
Equatable/Hashable being implicit and we decided to make it opt-in instead.
I'm worried that we're going down a road where "this set of protocols is
opt-in and this set of protocols is always-on" and users have to remember
which are which. (And in some cases, it's even more complex! Enums are
always Eq/Hash if they have raw values or no associated values, for
historical reasons but they're opt-in with associated values.)

There are key differences here. Equatable and Hashable have extensive
semantic guarantees, and it is certainly not the case that a type with
members that are all Equatable fulfills those guarantees itself. It would
be undesirable to have implicit Equatable conformance because the compiler
cannot guarantee semantics and must not make such assumptions. (The
implicit conformance of certain enums to Equatable and Hashable is
interesting yet defensible because it relies on an aspect of those enums
not shared with other types, an aspect that can be profitably relied
upon--see below.) All enums that have one or more cases with no associated
values, by contrast, have an enumerable set of cases. This is something
knowable to the compiler.

I agree with this. I think an argument could be successfully made that
since enums without associated values already conform to Equatable/Hashable
implicitly, and enums with raw values conform to RawRepresentable
implicitly, the same could be done for ValueEnumerable. But, I brought it
up because it's easy to imagine that there would be concerns raised about
adding more of these special cases to the language (which does add design
complexity, right?).

This goes to the issue above of whether it is desirable in the first

place to conflate enums that have enumerable cases with a more general
"ValueEnumerable"; the requested feature is to enumerate enum cases, and
having a set of unique cases is a feature of enums not shared with other
types. Inevitably, when you generalize, then you must ask what it means for
a non-enum type to have a set of enumerable case-like values. For example,
note that there is no particular need for an ordered collection if we are
considering enums (cf. previous conversations about supporting resilient
reordering of enum cases); by contrast, such a design would be highly
unintuitive for a range, which has an implicitly ordered set of values.
Since each enum case is guaranteed to be unique, and since Equatable and
Hashable conformance is implicitly synthesized in such a way that all cases
compare not equal to other cases, we could even use a Set for an enum's
`allCases`. Of course, to eliminate the potential need for allocating
memory, this could be a custom type with set-like semantics, but
nonetheless it allows us to guarantee that `Set(T.allCases).count ==
T.allCases.count`, which would be a nontrivial semantic requirement in the
more general case of custom `ValueEnumerable` conformance that the author
of the conforming type would need to consider--if we want to insist on that
semantic requirement at all.

These are great points! Regarding the ordering question‚ÄĒI agree that
purely theoretically, the cases of an enum are just an unordered set of
values. But, just as you are interested in meeting the needs of end users,
so am I‚ÄĒand one of the situations where I commonly see this feature
requested is to build UIs (menus, table view sections, etc.) from the cases
of an enum. Any design of this feature that would require users to specify
the ordering of their elements secondarily to the order they appear in
source code would make this feature useless for them, and those are the
people we're trying to help.

I'm not sure I understand this use case. Are you saying that it is
desirable *not* to require users to specify an order for the options in
their UI? On the contrary, that would seem to be extremely brittle; as a
library author, I might rearrange the order of cases in source code--which
should be a completely transparent change--and this would break end users'
UI. If the final design for ABI stability accommodates resilient reordering
of cases, then the UI would change depending on which version of the
library is installed at any particular moment. I would imagine that we
should explicitly discourage any reliance on source code order of cases. If
a user wants to get an array, then that is simple enough, as you say.

What I'm saying is that users are working with cases that have an inherent
order, and they encode that order in their source code so they don't want
to repeat it. It's only brittle if you believe that users are likely to
take information that has an inherent order based on the concepts they
represent and randomly shuffle them in their source code‚ÄĒlike "case
thursday, monday, friday, sunday, ...".

You've cited "the use case being requested" as a motivating factor for the
design, so that means you have to consider *why* people are requesting it
and *what* they want to do with it‚ÄĒnot just "enumerate the cases of an
enum", but the specific problem they're trying to solve when they ask for
that. I just did a spot check of web search results, and combined with
conversations I've had with colleagues on the same topic, it appears the
people asking for this feature are primarily trying to do one of two things:

1) Enumerate cases that have a well-defined order for display in a UI.
2) Enumerate cases that don't have an inherent order, but where stability
and predictability of the enumeration order is important (like generating
the cartesian product of (suits x face-values) for a deck of cards).

In both cases, users want to avoid writing boilerplate that maintains a
separate collection of those values (which itself is also brittle if cases
are added in the future, because even if usage sites like switches change,
there is no compile-time checking that a manually maintained array of cases
is exhaustive).

I suppose I'm having difficulty understanding where you stand on this topic
because in one place you say "the feature should be designed this way
because the intended use case is X" and then when we drill deeper into that
use case, you say "people should not do X because it's brittle". It can't
be both.

···

On Sat, Nov 11, 2017 at 1:53 PM Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Sat, Nov 11, 2017 at 3:15 PM, Tony Allevato <tony.allevato@gmail.com> > wrote:

On Sat, Nov 11, 2017 at 10:28 AM Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Sat, Nov 11, 2017 at 11:23 AM, Tony Allevato <tony.allevato@gmail.com >>> > wrote:

On Fri, Nov 10, 2017 at 11:01 PM Xiaodi Wu via swift-evolution < >>>> swift-evolution@swift.org> wrote:

So, I do accept an ordered collection as a compromise that makes the

feature usable even at the cost of losing some amount of generality. But
that's fine‚ÄĒall of these design problems are on a spectrum, from highly
specific on one end to highly general on the other. The main thing I want
to point out with this discussion is that we *can* make the design a bit
more general without harming usability for the common use case *and* make
it future-proof so we're not shutting out other potential use cases that
are harder to fix due to source/ABI compatibility or adding odd special
cases to the language.

Nit: if you want to call it `ValueEnumerable`, then this should be `DefaultValueCollection`.

I used `DefaultCaseCollection` because, although the protocol can work with any type to return its values, this type can only work with enums and only returns their cases. `ValueEnumerable` could be sensibly applied to `Int`; `DefaultCaseCollection` could not.

Because of how you've chosen to implement `DefaultCaseCollection`, or do you mean to say that you deliberately want a design where types other than enums do not share the default return type?

Because the way `DefaultCaseCollection` works is that it queries runtime metadata that can only exist for enums.

In theory, we might be able to write a `DefaultValueCollection` which would work for structs with `ValueEnumerable` properties by generating all possible permutations of those fields. In practice, I suspect that this would only rarely be useful. Structs' types are rarely specified so tightly that all permutations are valid; for instance, a `struct PlayingCard` with an integer `rank` property would only be *valid* with a rank between 1 and 13, even though `Int`'s range is much wider. So I don't think we'll ever want this type to support structs, and it would therefore be clearer to bake its enum-only nature into its name.

(If your response is that "your argument against permuting all possible struct values is just as true with an integer associated value"…well, you're not wrong, and that might be an argument against making integer types `ValueEnumerable`. But we don't propose conforming `Int` to `ValueEnumerable` immediately, just adopting a design flexible enough to permit it.)

Tony's "no more specific than they need to" language applies here. The way I see it is this:

* `Bool` is not an enum, but it could be usefully conformed to `ValueEnumerable`. Why should we prevent that?
* A type with two independently-switchable `Bool`s‚ÄĒsay, `isMirrored` and `isFlipped`‚ÄĒcould be usefully conformed to `ValueEnumerable`. Why should we prevent that?
* Having integer types conform to `ValueEnumerable` with `static let allValues = Self.min...Self.max` could be useful. Why should we prevent that?

I'd say you're looking at it the wrong way. We're not *preventing* anything. We're adding a feature, and the question is, why should we *add* more than is justified by the use case?

Okay, here's the positive justification: The choice of an enum vs. a struct ought, to some degree, to be an implementation detail. As a general example of this, `__attribute__((swift_wrapper(enum)) ` types in Objective-C are actually imported into Swift as structs, but this detail rarely matters to users. A little closer to home, `Bool` could be an enum, but is implemented as a struct instead. `EncodingError` and `DecodingError` could be structs, but are implemented as enums instead.

To allow this flexibility, Swift rarely creates features for enums which are completely closed off to the structs (or vice versa), though they may have convenience features on only one of them. For example, both can have initializers, but only structs have them created implicitly; both can be RawRepresentable, but only enums get the sugar syntax. (The big exceptions are enum's pattern matching and struct's ability to encapsulate implementation details, but we've talked about bringing both of these features to the other side in some fashion.)

Therefore, I think this feature should follow the general Swift pattern and not be completely closed off to structs. It may not be as convenient to use there, but it should be possible. This preserves flexibility for type designers so they aren't forced to use enums merely because they want to use the standard mechanism for publishing the possible values of a type.

Is there a clamor for enumerating the possible values of a type with two independently switchable `Bool`s? If so, is that not an argument to make `Bool` a valid raw value type? (There is already a bug report to make tuples of raw value types valid raw value types themselves.)

FWIW, I'm surprised Swift thinks "raw type 'Bool' is not expressible by any literal" when Bool is `ExpressibleByBooleanLiteral`. I'm not sure whether this is an oversight or if there's a specific reason for it.

Why is it useful for (fixed-width) integer types to conform to `ValueEnumerable`? What use cases, exactly, would that enable that are not possible now?

It might permit advanced `ValueEnumerable` synthesis, for one thing. (But again, see my misgivings above about the usefulness of generating all possible permutations.)

And at the same time, a small, specialized collection type _also_ helps with our intended use case in some ways (while admittedly making things more difficult in others). So I think the more general design, which also works better for our intended use case, is the superior option.

Along the lines of user ergonomics, I would advocate for as many enums as possible to conform without explicit opt-in. It's true that we are moving away from such magical designs, and for good reason, but the gain here of enums Just Working(TM) for such a long-demanded feature has, I would argue, more benefits than drawbacks. To my mind, the feature is a lot like `RawRepresentable` in several ways, and it would be defensible for an equal amount of magic to be enabled for it.

But `RawRepresentable` *doesn't* get automatically added to all enums‚ÄĒyou explicitly opt in, albeit using a special sugar syntax. No, I think opt-in is the right answer here. We might be able to justify adding sugar to opt-in, but I can't actually think of a way to make opting in easier than conforming to a protocol and letting the complier synthesize the requirements.

Yes, you're right that `RawRepresentable` conformance *doesn't* get automatically added in, but there exists special sugar which makes the end result indistinguishable. By this I mean that the user gets `RawRepresentable` conformance without ever writing `Foo : RawRepresentable` anywhere (and neither do they write `Foo : Bar` where `Bar` is in turn `RawRepresentable`).

That is *not* getting conformance "without explicit opt-in". That is explicitly opting in through a sugar feature.

If you want to suggest a form of sugar which is substantially easier for users than adding a `ValueEnumerable` conformance clause, we're all ears. But I don't think there really is one. I don't think `@allValues enum Foo` is really enough of a win over `enum Foo: ValueEnumerable` to justify the additional language surface area.

This is, in fact, a perfect opportunity to bring up a question I've been leaving implicit. Why not explore moving away from using a protocol? The proposed protocol has no syntactic requirements, and when constrained only the use case of enums, it has essentially no semantic requirements either. It seems that it's only because we've committed to using a protocol that we've opened up this exploration of what it means semantically to be `ValueEnumerable`, and how to generalize it to other types, and how to design the return type of the synthesized function, etc.

What if some attribute would simply make the metatype conform to `Sequence` (or, for that matter, `BidirectionalCollection`)?

I would absolutely *adore* being able to conform metatypes to protocols, but I assume this would require major surgery to the type system. I also assume that the code generation needed to fulfill `RandomAccessCollection`'s requirements on `Foo.Type` would be much more complicated than generating a conformance. And there's also the problem that subscripts are currently not allowed as class/static members.

If it's actually feasible to modify the compiler with the features necessary to support this in the Swift 5 timeframe, I am totally willing to consider using `Foo.self` as the collection instead of `Foo.allValues`. But "give me a collection of all the cases" is a major convenience that users have been requesting for four years, and I really don't want to keep them waiting any longer just so we can deliver a Platonically ideal design.`Array(Foo.self)` would be pretty cool, but it's not *so* cool that we should delay something users will be ecstatic to have for several years just to get it.

···

On Nov 12, 2017, at 10:16 AM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Sun, Nov 12, 2017 at 4:54 AM, Brent Royal-Gordon <brent@architechies.com <mailto:brent@architechies.com>> wrote:

On Nov 10, 2017, at 11:01 PM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:

--
Brent Royal-Gordon
Architechies

Nit: if you want to call it `ValueEnumerable`, then this should be
`DefaultValueCollection`.

More generally though, I can see the appeal of allowing `Int` to
conform to `ValueEnumerable`, but I've got a feeling that this is headed
rapidly in the direction of overengineering a feature without a good
rationale for the additional complexity. The stated use case is to
enumerate the cases of an enum, and any additional complexity above that
should stand on its own merit. I disagree quite vehemently that protocols
should be "as general as possible"; rather, they exist to enable useful
generic algorithms and should be as _useful_ as possible. There is a happy
medium beyond which overengineering the design makes a protocol markedly
less usable/approachable for the sake of enabling rare functionality.

Perhaps "no more specific than they need to be" would have been a
better choice of words on my part than "as general as possible".

I'm not sure why you think this decision makes the protocol "markedly
less usable/approachable". and I think you're seeing a lot of complexity
that isn't there. Regarding good rationale, I've given that in this thread
above and in the previous discussion thread, but I'll summarize it again
here:

1) "The stated use case is to enumerate the cases of an enum" isn't
sufficient for designing a protocol because Swift does not have protocols
that are restricted only to enums. Just as anyone can conform their own
types to RawRepresentable, anyone should be able to conform their own types
to ValueEnumerable.

This is the part of the argument that I am questioning. If the desired
use case is to enumerate the cases of an enum, is it much of a gain to
allow anyone to be able to conform their own non-enum types to this
protocol? Yes, if you buy that, then much of the rest follows. But make no
mistake that it is not necessary to serve the desired use case and
dramatically increases the complexity of the design (see below for just
some considerations).

Some increase in complexity is certainly going to be introduced by
generalizing a design to make it useful to more than just a single use
case, but I hope you understand that using unquantifiable and subjective
qualifiers like "markedly less usable" or "dramatically increases the
complexity" is a bit unhelpful when trying to debate specifics. You clearly
put a great deal of thought into design problems like this and I always
appreciate your points of view, so I want to focus on the objective parts.

In prior threads on this topic, I believe I recall you saying that a
static property that returns an array is fine. (I don't remember your exact
words, so please correct me if I'm wrong. I don't want to misstate your
position.) Given the concerns I've pointed out about why that is both a
time and memory performance problem, and given your focus on the common use
case above, I'd appreciate your opinion on these points:

* Are you advocating for a magic protocol that *only* enums can conform
to? (This is a different issue than whether the protocol is just a
compiler-known protocol that it can synthesize.)
    * If yes, why intentionally restrict it others from implementing the
protocol? That's actually *more* complexity than allowing it, like we do
with RawRepresentable. There's also no precedent for "a protocol that can
only be applied to enums" in the Swift language. The ramifications of that
decision introduce their own design complexity‚ÄĒyou're just shifting it
around, not avoiding it.
    * If no, then I don't think we differ on this point.

No magic. Consider: Does the compiler stop me from conforming `UIButton`
to `Error`? No. Can `UIButton` conform to `Error`? Semantically, no.

I am arguing that the use case justifies a protocol called
"CaseEnumerable" with documented semantics such that only types with cases
(i.e., enums with one or more cases) would fulfill those semantics. Any
more generalization and we are designing without a justification. I'm
unsure what design complexity you anticipate with such a definition of
"CaseEnumerable."

I don't anticipate any design complexity with that definition. I also
don't see where the design complexity allegedly exists with the more
general version‚ÄĒyou've claimed that it exists but haven't shown what it is
or proved why it's harmful. The enum's implementation of the protocol would
do the same thing you're describing regardless of the associated type‚ÄĒonly
the protocol would be more general, which would allow other non-enum types
to represent their collection or sequence of (possibly infinite) values.

The complexity stems from exactly questions such as this; it is the
complexity of defining what the semantics should be such that types other
than enums may conform to the proposed protocol, and then the complexity of
generalizing return types, etc., to accommodate use cases such as possibly
infinite values, etc. Why are we doing this?

You keep claiming that this is harmful, but I haven't seen any concrete
reasons why‚ÄĒjust that there is "no justification". Why is "no
justification" sufficient for arguing against over-generalizing something
but it's not sufficient for arguing against over-specializing something?

Every addition to the language needs to be justified; every design needs to
serve a purpose, solving real problems. It's not "generalizing" versus
"specializing." The question is, what is the simplest, most elegant, most
teachable change that can make possible the stated use case.

For what it's worth, if enough people in the community think that
constraining the associated type of this hypothetical protocol to a
Collection type instead of Sequence is the way it should be, fine‚ÄĒI'd
disagree that it's a necessary restriction (because I feel I've shown
sufficiently that it doesn't harm the design), but it's not something that
I would want to stand in the way of the feature as a whole.

* Do you think that the type of this property should still be an array,

with all of the performance problems that it brings, or would you find a
sufficiently array-like but more performant type to be fine?

I have no problem with any suitable type if it can be justified, say, on
the grounds of performance or memory efficiency. I do have a problem if the
justification for a design is that it might be necessary to accommodate
some unspecified future use case with a currently non-existent type.

Then hopefully I've already shown that using a custom no-heap-allocation
collection type in my previous messages on this topic is a performance win
vs. allocating and populating an array.

I've got no problem with that.

And once that's allowed, they should be able to do so without imposing an

overly restrictive API. What if a custom type has thousands of elements
that are easily computed sequentially or from an index? Forcing an Array on
the implementor is an unnecessary restriction and a burden. (More on this
below.)

2) There is *no decrease in usability for the common use case *by
choosing to make the protocol associated type a Sequence instead of a
Collection or Array because the concrete type synthesized by the compiler *would
be a more refined concrete type anyway.* In other words, anyone who
writes "MyEnum.allValues" would get back an integer-indexable random access
collection. I can imagine useful algorithms (more of an
algebraic/set-theoretical nature) that could be written for types with
countably infinite value sets. Why require breaking source/ABI
compatibility later if we can avoid it now, at no cost for the common user?

2.1) Anyone who writes a generic algorithm constrained over
ValueEnumerable *would* have to add an additional constraint of
"where T: RandomAccessCollection, T.Index == Int". This is a slight
increase in complexity, but I would argue that if someone is writing
generic algorithms over collections (and that's what this is), then this is
something they should already familiar with. I argue that that trade-off is
acceptable‚ÄĒothers might disagree.

3) There is *no good reason* for the compiler to perform slow memory
allocation operations to shuffle around information that it *already
knows* when it can just as easily provide a specialized compact value
type that is far more time/space efficient. I should finish the prototype I
was working on‚ÄĒthe code synthesis is not markedly more difficult than it
would be to write code that constructs an array.

4) Most users *think* they need an array when what they really want
is to be able to get the count of elements, iterate over them, and possibly
integer-index into them. The specialized collection type can do all of
those things more efficiently.

4.1) If a user *truly* needs an array, they just write
"Array(MyEnum.allValues)".

Along the lines of user ergonomics, I would advocate for as many
enums as possible to conform without explicit opt-in. It's true that we are
moving away from such magical designs, and for good reason, but the gain
here of enums Just Working(TM) for such a long-demanded feature has, I
would argue, more benefits than drawbacks. To my mind, the feature is a lot
like `RawRepresentable` in several ways, and it would be defensible for an
equal amount of magic to be enabled for it.

I tend to agree with this philosophically, but the same argument about
a long-demanded feature Just Working‚ĄĘ could be/was made for
Equatable/Hashable being implicit and we decided to make it opt-in instead.
I'm worried that we're going down a road where "this set of protocols is
opt-in and this set of protocols is always-on" and users have to remember
which are which. (And in some cases, it's even more complex! Enums are
always Eq/Hash if they have raw values or no associated values, for
historical reasons but they're opt-in with associated values.)

There are key differences here. Equatable and Hashable have extensive
semantic guarantees, and it is certainly not the case that a type with
members that are all Equatable fulfills those guarantees itself. It would
be undesirable to have implicit Equatable conformance because the compiler
cannot guarantee semantics and must not make such assumptions. (The
implicit conformance of certain enums to Equatable and Hashable is
interesting yet defensible because it relies on an aspect of those enums
not shared with other types, an aspect that can be profitably relied
upon--see below.) All enums that have one or more cases with no associated
values, by contrast, have an enumerable set of cases. This is something
knowable to the compiler.

I agree with this. I think an argument could be successfully made that
since enums without associated values already conform to Equatable/Hashable
implicitly, and enums with raw values conform to RawRepresentable
implicitly, the same could be done for ValueEnumerable. But, I brought it
up because it's easy to imagine that there would be concerns raised about
adding more of these special cases to the language (which does add design
complexity, right?).

This goes to the issue above of whether it is desirable in the first

place to conflate enums that have enumerable cases with a more general
"ValueEnumerable"; the requested feature is to enumerate enum cases, and
having a set of unique cases is a feature of enums not shared with other
types. Inevitably, when you generalize, then you must ask what it means for
a non-enum type to have a set of enumerable case-like values. For example,
note that there is no particular need for an ordered collection if we are
considering enums (cf. previous conversations about supporting resilient
reordering of enum cases); by contrast, such a design would be highly
unintuitive for a range, which has an implicitly ordered set of values.
Since each enum case is guaranteed to be unique, and since Equatable and
Hashable conformance is implicitly synthesized in such a way that all cases
compare not equal to other cases, we could even use a Set for an enum's
`allCases`. Of course, to eliminate the potential need for allocating
memory, this could be a custom type with set-like semantics, but
nonetheless it allows us to guarantee that `Set(T.allCases).count ==
T.allCases.count`, which would be a nontrivial semantic requirement in the
more general case of custom `ValueEnumerable` conformance that the author
of the conforming type would need to consider--if we want to insist on that
semantic requirement at all.

These are great points! Regarding the ordering question‚ÄĒI agree that
purely theoretically, the cases of an enum are just an unordered set of
values. But, just as you are interested in meeting the needs of end users,
so am I‚ÄĒand one of the situations where I commonly see this feature
requested is to build UIs (menus, table view sections, etc.) from the cases
of an enum. Any design of this feature that would require users to specify
the ordering of their elements secondarily to the order they appear in
source code would make this feature useless for them, and those are the
people we're trying to help.

I'm not sure I understand this use case. Are you saying that it is
desirable *not* to require users to specify an order for the options in
their UI? On the contrary, that would seem to be extremely brittle; as a
library author, I might rearrange the order of cases in source code--which
should be a completely transparent change--and this would break end users'
UI. If the final design for ABI stability accommodates resilient reordering
of cases, then the UI would change depending on which version of the
library is installed at any particular moment. I would imagine that we
should explicitly discourage any reliance on source code order of cases. If
a user wants to get an array, then that is simple enough, as you say.

What I'm saying is that users are working with cases that have an inherent
order, and they encode that order in their source code so they don't want
to repeat it. It's only brittle if you believe that users are likely to
take information that has an inherent order based on the concepts they
represent and randomly shuffle them in their source code‚ÄĒlike "case
thursday, monday, friday, sunday, ...".

You've cited "the use case being requested" as a motivating factor for the
design, so that means you have to consider *why* people are requesting it
and *what* they want to do with it‚ÄĒnot just "enumerate the cases of an
enum", but the specific problem they're trying to solve when they ask for
that. I just did a spot check of web search results, and combined with
conversations I've had with colleagues on the same topic, it appears the
people asking for this feature are primarily trying to do one of two things:

1) Enumerate cases that have a well-defined order for display in a UI.
2) Enumerate cases that don't have an inherent order, but where stability
and predictability of the enumeration order is important (like generating
the cartesian product of (suits x face-values) for a deck of cards).

In both cases, users want to avoid writing boilerplate that maintains a
separate collection of those values (which itself is also brittle if cases
are added in the future, because even if usage sites like switches change,
there is no compile-time checking that a manually maintained array of cases
is exhaustive).

If the idea is to have a custom collection for memory and performance
reasons, then I take it as given that cases will be enumerated in ABI
order. In the ABI stability discussions, the question has arisen whether
_reordering_ of cases should be an ABI-breaking change, or whether
_renaming_ of cases should be an ABI-breaking change. If cases are stored
in source order, then reordering cases breaks ABI but renaming does not; if
cases are stored in alphabetical order, then renaming cases breaks ABI but
reordering does not. If I recall correctly, a greater number of people
thought that changing declaration order should not be an ABI-breaking
phenomenon, and that the compiler should disregard the order in which cases
are written in the source code. Suffice it to say that, should such a
design be adopted, then it would be inconsistent (and suboptimal for
performance) for this protocol to vend an API that leaks the source order.

I suppose I'm having difficulty understanding where you stand on this
topic because in one place you say "the feature should be designed this way
because the intended use case is X" and then when we drill deeper into that
use case, you say "people should not do X because it's brittle". It can't
be both.

The stated use case in the proposal is, in its entirety, that enum cases
should be enumerable; I agree that it is a legitimate use case and am
exploring options for making that possible. The users you've polled seem to
have additional demands unstated in the proposal which actually do not jive
with those users who've weighed in on the ABI stability conversation. Put
another way, it seems like there is disagreement as to whether the source
order of enum cases has semantic significance. I am proceeding as though it
does not, but it appears that you think that it does. This would need to be
settled--probably as part of the ABI stability discussion--before we can
proceed here.

···

On Sun, Nov 12, 2017 at 1:37 PM, Tony Allevato <tony.allevato@gmail.com> wrote:

On Sat, Nov 11, 2017 at 1:53 PM Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Sat, Nov 11, 2017 at 3:15 PM, Tony Allevato <tony.allevato@gmail.com> >> wrote:

On Sat, Nov 11, 2017 at 10:28 AM Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Sat, Nov 11, 2017 at 11:23 AM, Tony Allevato < >>>> tony.allevato@gmail.com> wrote:

On Fri, Nov 10, 2017 at 11:01 PM Xiaodi Wu via swift-evolution < >>>>> swift-evolution@swift.org> wrote:

So, I do accept an ordered collection as a compromise that makes the

feature usable even at the cost of losing some amount of generality. But
that's fine‚ÄĒall of these design problems are on a spectrum, from highly
specific on one end to highly general on the other. The main thing I want
to point out with this discussion is that we *can* make the design a bit
more general without harming usability for the common use case *and* make
it future-proof so we're not shutting out other potential use cases that
are harder to fix due to source/ABI compatibility or adding odd special
cases to the language.

1 Like

Nit: if you want to call it `ValueEnumerable`, then this should be
`DefaultValueCollection`.

I used `DefaultCaseCollection` because, although the protocol can work
with any type to return its values, this type can only work with enums and
only returns their cases. `ValueEnumerable` could be sensibly applied to
`Int`; `DefaultCaseCollection` could not.

Because of how you've chosen to implement `DefaultCaseCollection`, or do
you mean to say that you deliberately want a design where types other than
enums do not share the default return type?

Because the way `DefaultCaseCollection` works is that it queries runtime
metadata that can only exist for enums.

In theory, we might be able to write a `DefaultValueCollection` which
would work for structs with `ValueEnumerable` properties by generating all
possible permutations of those fields. In practice, I suspect that this
would only rarely be useful. Structs' types are rarely specified so tightly
that all permutations are valid; for instance, a `struct PlayingCard` with
an integer `rank` property would only be *valid* with a rank between 1 and
13, even though `Int`'s range is much wider. So I don't think we'll ever
want this type to support structs, and it would therefore be clearer to
bake its enum-only nature into its name.

(If your response is that "your argument against permuting all possible
struct values is just as true with an integer associated value"…well,
you're not wrong, and that might be an argument against making integer
types `ValueEnumerable`.

Not only is it an argument against making integer types `ValueEnumerable`,
it's a pretty darn good argument against any design that makes such an
eventuality possible--in the absence of a compelling use case that would
make that design desirable for other reasons.

But we don't propose conforming `Int` to `ValueEnumerable` immediately,
just adopting a design flexible enough to permit it.)

Tony's "no more specific than they need to" language applies here. The way

I see it is this:

* `Bool` is not an enum, but it could be usefully conformed to
`ValueEnumerable`. Why should we prevent that?
* A type with two independently-switchable `Bool`s‚ÄĒsay, `isMirrored` and
`isFlipped`‚ÄĒcould be usefully conformed to `ValueEnumerable`. Why should we
prevent that?
* Having integer types conform to `ValueEnumerable` with `static let
allValues = Self.min...Self.max` could be useful. Why should we prevent
that?

I'd say you're looking at it the wrong way. We're not *preventing*
anything. We're adding a feature, and the question is, why should we *add*
more than is justified by the use case?

Okay, here's the positive justification: The choice of an enum vs. a
struct ought, to some degree, to be an implementation detail. As a general
example of this, `__attribute__((swift_wrapper(enum)) ` types in
Objective-C are actually imported into Swift as structs, but this detail
rarely matters to users. A little closer to home, `Bool` could be an enum,
but is implemented as a struct instead. `EncodingError` and `DecodingError`
could be structs, but are implemented as enums instead.

This is not a terrible argument from a practical standpoint, but I'd turn
it around: based on your examples, it's a convincing argument for improving
Objective-C import so that all types that users would want to be enums can
actually be bridged as enums, and for making `Bool` into an enum. I don't
see what the problem is with `EncodingError` or `DecodingError` being
enums, so I can't comment on that.

To allow this flexibility, Swift rarely creates features for enums which
are completely closed off to the structs (or vice versa), though they may
have convenience features on only one of them. For example, both can have
initializers, but only structs have them created implicitly; both can be
RawRepresentable, but only enums get the sugar syntax. (The big exceptions
are enum's pattern matching and struct's ability to encapsulate
implementation details, but we've talked about bringing both of these
features to the other side in some fashion.)

Therefore, I think this feature should follow the general Swift pattern
and not be completely closed off to structs. It may not be as convenient to
use there, but it should be possible. This preserves flexibility for type
designers so they aren't forced to use enums merely because they want to
use the standard mechanism for publishing the possible values of a type.

Or, we could fix the edge cases so that no user needs to use a `struct`
where an `enum` would be most suitable so that the choice of enum vs.
struct is never an implementation artifact but consistently holds some
semantic significance. In the meantime, I'd argue we shouldn't weaken what
distinctions do exist between the two.

Is there a clamor for enumerating the possible values of a type with two

independently switchable `Bool`s? If so, is that not an argument to make
`Bool` a valid raw value type? (There is already a bug report to make
tuples of raw value types valid raw value types themselves.)

FWIW, I'm surprised Swift thinks "raw type 'Bool' is not expressible by
any literal" when Bool is `ExpressibleByBooleanLiteral`. I'm not sure
whether this is an oversight or if there's a specific reason for it.

Yes, I agree that this should be fixed.

Why is it useful for (fixed-width) integer types to conform to

`ValueEnumerable`? What use cases, exactly, would that enable that are not
possible now?

It might permit advanced `ValueEnumerable` synthesis, for one thing. (But
again, see my misgivings above about the usefulness of generating all
possible permutations.)

And at the same time, a small, specialized collection type _also_ helps

with our intended use case in some ways (while admittedly making things
more difficult in others). So I think the more general design, which also
works better for our intended use case, is the superior option.

Along the lines of user ergonomics, I would advocate for as many enums as
possible to conform without explicit opt-in. It's true that we are moving
away from such magical designs, and for good reason, but the gain here of
enums Just Working(TM) for such a long-demanded feature has, I would argue,
more benefits than drawbacks. To my mind, the feature is a lot like
`RawRepresentable` in several ways, and it would be defensible for an equal
amount of magic to be enabled for it.

But `RawRepresentable` *doesn't* get automatically added to all enums‚ÄĒyou
explicitly opt in, albeit using a special sugar syntax. No, I think opt-in
is the right answer here. We might be able to justify adding sugar to
opt-in, but I can't actually think of a way to make opting in easier than
conforming to a protocol and letting the complier synthesize the
requirements.

Yes, you're right that `RawRepresentable` conformance *doesn't* get
automatically added in, but there exists special sugar which makes the end
result indistinguishable. By this I mean that the user gets
`RawRepresentable` conformance without ever writing `Foo :
RawRepresentable` anywhere (and neither do they write `Foo : Bar` where
`Bar` is in turn `RawRepresentable`).

That is *not* getting conformance "without explicit opt-in". That is
explicitly opting in through a sugar feature.

If you want to suggest a form of sugar which is substantially easier for
users than adding a `ValueEnumerable` conformance clause, we're all ears.
But I don't think there really is one. I don't think `@allValues enum Foo`
is really enough of a win over `enum Foo: ValueEnumerable` to justify the
additional language surface area.

As a thought experiment, let's suppose we wanted to find an attribute that
would be suitable. What would such an attribute be named? Probably not
`@allValues enum`, which is kind of cryptic. (What, after all, is an enum
that isn't "all values"? Would there be "some references and some values"?
Absurd!) How about simply `@valueEnumerable enum`? But enumerations are a
value type, so clearly its cases are values, making `value` redundant.
Likewise, similar reasoning would apply to `@caseEnumerable`. Now we're
left with `@enumerable enum`. See the problem there?

No, I'm really suggesting automatic conformance. Enums (aka enumerations),
after all, should be enumerable. It says so right in the name. (Yes, Swift
extends enums to support features such as associated values which can make
their enumeration a tricky issue, but that's part and parcel of _extending
a paradigm_; it is still important, however, to reckon with what a
"bread-and-butter" enum is and what it is capable of.) To say that we have
an enumerable enum should be a redundant redundancy; we have truly diluted
away any semblance of what an enum is if we have to opt into enums being
enumerable.

This is, in fact, a perfect opportunity to bring up a question I've been

leaving implicit. Why not explore moving away from using a protocol? The
proposed protocol has no syntactic requirements, and when constrained only
the use case of enums, it has essentially no semantic requirements either.
It seems that it's only because we've committed to using a protocol that
we've opened up this exploration of what it means semantically to be
`ValueEnumerable`, and how to generalize it to other types, and how to
design the return type of the synthesized function, etc.

What if some attribute would simply make the metatype conform to
`Sequence` (or, for that matter, `BidirectionalCollection`)?

I would absolutely *adore* being able to conform metatypes to protocols,
but I assume this would require major surgery to the type system. I also
assume that the code generation needed to fulfill
`RandomAccessCollection`'s requirements on `Foo.Type` would be much more
complicated than generating a conformance. And there's also the problem
that subscripts are currently not allowed as class/static members.

If it's actually feasible to modify the compiler with the features
necessary to support this in the Swift 5 timeframe, I am totally willing to
consider using `Foo.self` as the collection instead of `Foo.allValues`. But
"give me a collection of all the cases" is a major convenience that users
have been requesting for four years, and I really don't want to keep them
waiting any longer just so we can deliver a Platonically ideal
design.`Array(Foo.self)` would be pretty cool, but it's not *so* cool that
we should delay something users will be ecstatic to have for several years
just to get it.

Just for the record, let it be recorded that you and I agree that the
_perfect_ solution would be having the ability to write `for case in
Foo.self`.

Now, here's the thing: we don't need to be able to conform any metatype to
any protocol in order to deliver what users have asked for. As a first
pass, we need only to allow enums to magically conform to `Collection` (not
even `RandomAccessCollection`, just `Collection`). You'd be able to iterate
over the cases in a for...in loop, you'd be able to write
`Array(Foo.self)`, and that comfortably gets us 80% of the way there _while
adding nothing to the API surface area that would not later be subsumed_ by
a more complete implementation of the ideal solution.

There is precedent for such a solution. Recall how tuples of Equatable
types have `==` defined up to arity 6 by a manual implementation. We still
haven't implemented the general solution, but when we do, the interim
solution can fade away transparently.

···

On Mon, Nov 13, 2017 at 8:16 PM, Brent Royal-Gordon <brent@architechies.com> wrote:

On Nov 12, 2017, at 10:16 AM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Sun, Nov 12, 2017 at 4:54 AM, Brent Royal-Gordon < > brent@architechies.com> wrote:

On Nov 10, 2017, at 11:01 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

Nit: if you want to call it `ValueEnumerable`, then this should be
`DefaultValueCollection`.

I used `DefaultCaseCollection` because, although the protocol can work
with any type to return its values, this type can only work with enums and
only returns their cases. `ValueEnumerable` could be sensibly applied to
`Int`; `DefaultCaseCollection` could not.

Because of how you've chosen to implement `DefaultCaseCollection`, or do
you mean to say that you deliberately want a design where types other than
enums do not share the default return type?

Because the way `DefaultCaseCollection` works is that it queries runtime
metadata that can only exist for enums.

In theory, we might be able to write a `DefaultValueCollection` which
would work for structs with `ValueEnumerable` properties by generating all
possible permutations of those fields. In practice, I suspect that this
would only rarely be useful. Structs' types are rarely specified so tightly
that all permutations are valid; for instance, a `struct PlayingCard` with
an integer `rank` property would only be *valid* with a rank between 1 and
13, even though `Int`'s range is much wider. So I don't think we'll ever
want this type to support structs, and it would therefore be clearer to
bake its enum-only nature into its name.

(If your response is that "your argument against permuting all possible
struct values is just as true with an integer associated value"…well,
you're not wrong, and that might be an argument against making integer
types `ValueEnumerable`.

Not only is it an argument against making integer types `ValueEnumerable`,
it's a pretty darn good argument against any design that makes such an
eventuality possible--in the absence of a compelling use case that would
make that design desirable for other reasons.

But we don't propose conforming `Int` to `ValueEnumerable` immediately,
just adopting a design flexible enough to permit it.)

Tony's "no more specific than they need to" language applies here. The

way I see it is this:

* `Bool` is not an enum, but it could be usefully conformed to
`ValueEnumerable`. Why should we prevent that?
* A type with two independently-switchable `Bool`s‚ÄĒsay, `isMirrored` and
`isFlipped`‚ÄĒcould be usefully conformed to `ValueEnumerable`. Why should we
prevent that?
* Having integer types conform to `ValueEnumerable` with `static let
allValues = Self.min...Self.max` could be useful. Why should we prevent
that?

I'd say you're looking at it the wrong way. We're not *preventing*
anything. We're adding a feature, and the question is, why should we *add*
more than is justified by the use case?

Okay, here's the positive justification: The choice of an enum vs. a
struct ought, to some degree, to be an implementation detail. As a general
example of this, `__attribute__((swift_wrapper(enum)) ` types in
Objective-C are actually imported into Swift as structs, but this detail
rarely matters to users. A little closer to home, `Bool` could be an enum,
but is implemented as a struct instead. `EncodingError` and `DecodingError`
could be structs, but are implemented as enums instead.

This is not a terrible argument from a practical standpoint, but I'd turn
it around: based on your examples, it's a convincing argument for improving
Objective-C import so that all types that users would want to be enums can
actually be bridged as enums, and for making `Bool` into an enum. I don't
see what the problem is with `EncodingError` or `DecodingError` being
enums, so I can't comment on that.

To allow this flexibility, Swift rarely creates features for enums which
are completely closed off to the structs (or vice versa), though they may
have convenience features on only one of them. For example, both can have
initializers, but only structs have them created implicitly; both can be
RawRepresentable, but only enums get the sugar syntax. (The big exceptions
are enum's pattern matching and struct's ability to encapsulate
implementation details, but we've talked about bringing both of these
features to the other side in some fashion.)

Therefore, I think this feature should follow the general Swift pattern
and not be completely closed off to structs. It may not be as convenient to
use there, but it should be possible. This preserves flexibility for type
designers so they aren't forced to use enums merely because they want to
use the standard mechanism for publishing the possible values of a type.

Or, we could fix the edge cases so that no user needs to use a `struct`
where an `enum` would be most suitable so that the choice of enum vs.
struct is never an implementation artifact but consistently holds some
semantic significance. In the meantime, I'd argue we shouldn't weaken what
distinctions do exist between the two.

Is there a clamor for enumerating the possible values of a type with two

independently switchable `Bool`s? If so, is that not an argument to make
`Bool` a valid raw value type? (There is already a bug report to make
tuples of raw value types valid raw value types themselves.)

FWIW, I'm surprised Swift thinks "raw type 'Bool' is not expressible by
any literal" when Bool is `ExpressibleByBooleanLiteral`. I'm not sure
whether this is an oversight or if there's a specific reason for it.

Yes, I agree that this should be fixed.

Why is it useful for (fixed-width) integer types to conform to

`ValueEnumerable`? What use cases, exactly, would that enable that are not
possible now?

It might permit advanced `ValueEnumerable` synthesis, for one thing. (But
again, see my misgivings above about the usefulness of generating all
possible permutations.)

And at the same time, a small, specialized collection type _also_ helps

with our intended use case in some ways (while admittedly making things
more difficult in others). So I think the more general design, which also
works better for our intended use case, is the superior option.

Along the lines of user ergonomics, I would advocate for as many enums
as possible to conform without explicit opt-in. It's true that we are
moving away from such magical designs, and for good reason, but the gain
here of enums Just Working(TM) for such a long-demanded feature has, I
would argue, more benefits than drawbacks. To my mind, the feature is a lot
like `RawRepresentable` in several ways, and it would be defensible for an
equal amount of magic to be enabled for it.

But `RawRepresentable` *doesn't* get automatically added to all
enums‚ÄĒyou explicitly opt in, albeit using a special sugar syntax. No, I
think opt-in is the right answer here. We might be able to justify adding
sugar to opt-in, but I can't actually think of a way to make opting in
easier than conforming to a protocol and letting the complier synthesize
the requirements.

Yes, you're right that `RawRepresentable` conformance *doesn't* get
automatically added in, but there exists special sugar which makes the end
result indistinguishable. By this I mean that the user gets
`RawRepresentable` conformance without ever writing `Foo :
RawRepresentable` anywhere (and neither do they write `Foo : Bar` where
`Bar` is in turn `RawRepresentable`).

That is *not* getting conformance "without explicit opt-in". That is
explicitly opting in through a sugar feature.

If you want to suggest a form of sugar which is substantially easier for
users than adding a `ValueEnumerable` conformance clause, we're all ears.
But I don't think there really is one. I don't think `@allValues enum Foo`
is really enough of a win over `enum Foo: ValueEnumerable` to justify the
additional language surface area.

As a thought experiment, let's suppose we wanted to find an attribute that
would be suitable. What would such an attribute be named? Probably not
`@allValues enum`, which is kind of cryptic. (What, after all, is an enum
that isn't "all values"? Would there be "some references and some values"?
Absurd!) How about simply `@valueEnumerable enum`? But enumerations are a
value type, so clearly its cases are values, making `value` redundant.
Likewise, similar reasoning would apply to `@caseEnumerable`. Now we're
left with `@enumerable enum`. See the problem there?

No, I'm really suggesting automatic conformance. Enums (aka enumerations),
after all, should be enumerable. It says so right in the name. (Yes, Swift
extends enums to support features such as associated values which can make
their enumeration a tricky issue, but that's part and parcel of _extending
a paradigm_; it is still important, however, to reckon with what a
"bread-and-butter" enum is and what it is capable of.) To say that we have
an enumerable enum should be a redundant redundancy; we have truly diluted
away any semblance of what an enum is if we have to opt into enums being
enumerable.

This is, in fact, a perfect opportunity to bring up a question I've been

leaving implicit. Why not explore moving away from using a protocol? The
proposed protocol has no syntactic requirements, and when constrained only
the use case of enums, it has essentially no semantic requirements either.
It seems that it's only because we've committed to using a protocol that
we've opened up this exploration of what it means semantically to be
`ValueEnumerable`, and how to generalize it to other types, and how to
design the return type of the synthesized function, etc.

What if some attribute would simply make the metatype conform to
`Sequence` (or, for that matter, `BidirectionalCollection`)?

I would absolutely *adore* being able to conform metatypes to protocols,
but I assume this would require major surgery to the type system. I also
assume that the code generation needed to fulfill
`RandomAccessCollection`'s requirements on `Foo.Type` would be much more
complicated than generating a conformance. And there's also the problem
that subscripts are currently not allowed as class/static members.

If it's actually feasible to modify the compiler with the features
necessary to support this in the Swift 5 timeframe, I am totally willing to
consider using `Foo.self` as the collection instead of `Foo.allValues`. But
"give me a collection of all the cases" is a major convenience that users
have been requesting for four years, and I really don't want to keep them
waiting any longer just so we can deliver a Platonically ideal
design.`Array(Foo.self)` would be pretty cool, but it's not *so* cool that
we should delay something users will be ecstatic to have for several years
just to get it.

Just for the record, let it be recorded that you and I agree that the
_perfect_ solution would be having the ability to write `for case in
Foo.self`.

Now, here's the thing: we don't need to be able to conform any metatype to
any protocol in order to deliver what users have asked for. As a first
pass, we need only to allow enums to magically conform to `Collection` (not
even `RandomAccessCollection`, just `Collection`). You'd be able to iterate
over the cases in a for...in loop, you'd be able to write
`Array(Foo.self)`, and that comfortably gets us 80% of the way there _while
adding nothing to the API surface area that would not later be subsumed_ by
a more complete implementation of the ideal solution.

...I should add, if full conformance to `Collection` is still too much to
ask, enabling "for `case` in Foo.self" by magic would itself address the
entirety of the proposal's use case, adding no API surface area.
Essentially all other uses for enumeration of enum cases can be trivially
recreated based on just that.

···

On Mon, Nov 13, 2017 at 11:15 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Mon, Nov 13, 2017 at 8:16 PM, Brent Royal-Gordon < > brent@architechies.com> wrote:

On Nov 12, 2017, at 10:16 AM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Sun, Nov 12, 2017 at 4:54 AM, Brent Royal-Gordon < >> brent@architechies.com> wrote:

On Nov 10, 2017, at 11:01 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

There is precedent for such a solution. Recall how tuples of Equatable
types have `==` defined up to arity 6 by a manual implementation. We still
haven't implemented the general solution, but when we do, the interim
solution can fade away transparently.

...I should add, if full conformance to `Collection` is still too much to ask, enabling "for `case` in Foo.self" by magic would itself address the entirety of the proposal's use case, adding no API surface area.

No, Xiaodi. No, it would not.

Okay, thus far we've talked vaguely about "accessing the cases of an enum", but let's talk a little more concretely about what that means. I think that, especially on Apple platforms, this is the most important concrete use case:

PROBLEM:

You have an enum like:

  enum BugStatus {
    case open, inProgress, resolved, closed, reopened
    var localizedName: String { … }
  }

You wish to present these options in a user interface so a user can select one of them. For instance, if you're on iOS and using UITableView, you might want to present the enum's values through `UITableViewDataSource` and allow selection through `UITableViewDelegate`.

REQUIREMENTS:

1. It must be possible to easily access the count of values, and to access any particular value using contiguous `Int` indices. This could be achieved either by directly accessing elements in the list of values through an Int subscript, or by constructing an Array from the list of values.

2. It must be possible to control the order of values in the list of values, either by using source order or through some other simple, straightforward mechanism.

PROPOSED SOLUTION:

You conform `BugStatus` to `ValueEnumerable`:

  enum BugStatus: ValueEnumerable {
    case open, inProgress, resolved, closed, reopened
    var localizedName: String { … }
  }

And then write the table view data source to present the elements of `BugStatus.allValues`:

  class BugStatusDataSource: NSObject, UITableViewDataSource, UITableViewDelegate {
    @IBOutlet var tableView: UITableView?
    
    @objc dynamic var selected: BugStatus? { // Observable via KVO
      didSet { tableView.reloadData() }
    }
    
    func status(at indexPath: IndexPath) -> Status {
      BugStatus.allValues[indexPath.row]
    }
    
    func tableView(_: UITableView, numberOfRowsInSection section: Int) -> Int {
      return BugStatus.allValues.count
    }
    
    func tableView(_: UITableView, cellForRowAt indexPath: IndexPath) -> UITableViewCell {
      let status = self.status(at: indexPath)
      let identifier = (status == selected) ? "SelectedCell" : "RegularCell"
      
      let cell = tableView.dequeueReusableCell(withIdentifier: identifier, for: indexPath)
      cell.titleLabel.text = status.localizedName
      return cell
    }
    
    func tableView(_: UITableView, didSelectRowAt indexPath: IndexPath) {
      selected = status(at: indexPath)
    }
  }

This is the most direct solution; a more sophisticated version might inject the list as an Array so that you can show a subset of the full set of values.

EXTENSIONS:

Now, let's quickly talk about a couple extensions of this use case:

* The values, and the table view cells, are grouped into sections. This suggests some sort of two-level, nested structure, which may not use `Int` indices.

* You want to write a *generic* data source which can present all the values of *any* ValueEnumerable type (or at least any conforming to a protocol that allows us to fill in their cells). For that purpose, it's helpful to have the type conform to *some* sort of protocol and expose the value list through that protocol.

You say that:

Essentially all other uses for enumeration of enum cases can be trivially recreated based on just that.

But with this use case in mind, we can see that it is "trivial" in the sense that the annoying boilerplate you need to bridge the significant impedance mismatch is easy to come up with. Yes, you could construct an array using the magic `for` loop, but that would be a serious pain. (And there would be no ergonomic, mistake-resistant way to hide that pain behind a function/initializer call, because there's no way to say that a parameter must be a metatype for an enum.) What you really want is a way to access or construct an `Array` or array-like type containing the type's values.

*Actually* conforming the metatype to `Sequence` or `Collection` would be a different story. There, you could construct `Array`s or access elements using ordinary APIs and type system features. And you could write generic algorithms which used the set of all types: they would require conformance to `Sequence` or `Collection`, and users would specify `Foo.Type` as the generic parameter. But I suspect that would require deeper compiler changes than we can be certain to get in Swift 5 or really at any specific point on the roadmap, and I don't think we should delay this feature indefinitely to get a design whose only real benefit is elegance.

···

On Nov 13, 2017, at 9:21 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

--
Brent Royal-Gordon
Architechies

...I should add, if full conformance to `Collection` is still too much to
ask, enabling "for `case` in Foo.self" by magic would itself address the
entirety of the proposal's use case, adding no API surface area.

No, Xiaodi. No, it would not.

Okay, thus far we've talked vaguely about "accessing the cases of an
enum", but let's talk a little more concretely about what that means. I
think that, especially on Apple platforms, this is the most important
concrete use case:

PROBLEM:

You have an enum like:

enum BugStatus {
case open, inProgress, resolved, closed, reopened
var localizedName: String { … }
}

You wish to present these options in a user interface so a user can select
one of them. For instance, if you're on iOS and using UITableView, you
might want to present the enum's values through `UITableViewDataSource` and
allow selection through `UITableViewDelegate`.

REQUIREMENTS:

1. It must be possible to easily access the count of values, and to access
any particular value using contiguous `Int` indices. This could be achieved
either by directly accessing elements in the list of values through an Int
subscript, or by constructing an Array from the list of values.

2. It must be possible to control the order of values in the list of
values, either by using source order or through some other simple,
straightforward mechanism.

OK, first of all, nowhere in the proposal text are these requirements
stated as part of the use case. You're free to put forward new use cases,
but here I am trying to design the most elegant way to fulfill a stated
need and you're telling me that it's something other than what's written.
But sure, let's proceed on this basis.

PROPOSED SOLUTION:

You conform `BugStatus` to `ValueEnumerable`:

enum BugStatus: ValueEnumerable {
case open, inProgress, resolved, closed, reopened
var localizedName: String { … }
}

And then write the table view data source to present the elements of
`BugStatus.allValues`:

class BugStatusDataSource: NSObject, UITableViewDataSource,
UITableViewDelegate {
@IBOutlet var tableView: UITableView?
@objc dynamic var selected: BugStatus? { // Observable via KVO
didSet { tableView.reloadData() }
}
func status(at indexPath: IndexPath) -> Status {
BugStatus.allValues[indexPath.row]
}
func tableView(_: UITableView, numberOfRowsInSection section: Int) -> Int {
return BugStatus.allValues.count
}
func tableView(_: UITableView, cellForRowAt indexPath: IndexPath) ->
UITableViewCell {
let status = self.status(at: indexPath)
let identifier = (status == selected) ? "SelectedCell" : "RegularCell"
let cell = tableView.dequeueReusableCell(withIdentifier: identifier, for:
indexPath)
cell.titleLabel.text = status.localizedName
return cell
}
func tableView(_: UITableView, didSelectRowAt indexPath: IndexPath) {
selected = status(at: indexPath)
}
}

This is the most direct solution; a more sophisticated version might
inject the list as an Array so that you can show a subset of the full set
of values.

EXTENSIONS:

Now, let's quickly talk about a couple extensions of this use case:

* The values, and the table view cells, are grouped into sections. This
suggests some sort of two-level, nested structure, which may not use `Int`
indices.

* You want to write a *generic* data source which can present all the
values of *any* ValueEnumerable type (or at least any conforming to a
protocol that allows us to fill in their cells). For that purpose, it's
helpful to have the type conform to *some* sort of protocol and expose the
value list through that protocol.

You say that:

Essentially all other uses for enumeration of enum cases can be
trivially recreated based on just that.

But with this use case in mind, we can see that it is "trivial" in the
sense that the annoying boilerplate you need to bridge the significant
impedance mismatch is easy to come up with. Yes, you could construct an
array using the magic `for` loop, but that would be a serious pain. (And
there would be no ergonomic, mistake-resistant way to hide that pain behind
a function/initializer call, because there's no way to say that a parameter
must be a metatype for an enum.) What you really want is a way to access or
construct an `Array` or array-like type containing the type's values.

You cannot truly believe that

var cases = [BugStatus]()
for c in BugStatus.self { cases.append(c) }

is "serious pain." Yes, part of being an incomplete implementation is that
it lacks the ergonomics of a fleshed-out conformance to `Collection`. But
"serious pain"?

The point here is that even a minimal step towards what we agree is the
ideal design would make _possible_ what today is _impossible_: namely,
future-proof enumeration of all the cases of an enum type without
associated values.

*Actually* conforming the metatype to `Sequence` or `Collection` would be a

different story. There, you could construct `Array`s or access elements
using ordinary APIs and type system features. And you could write generic
algorithms which used the set of all types: they would require conformance
to `Sequence` or `Collection`, and users would specify `Foo.Type` as the
generic parameter.

Indeed. The point here is that we don't need a name for this protocol.
You'd be able to write useful generic algorithms by using functions such as
`map` on `T.self` with intuitive constraints such as `T where T.Type :
Collection`. Isn't that a sublime way of expressing exactly what we mean?

But I suspect that would require deeper compiler changes than we can be
certain to get in Swift 5 or really at any specific point on the roadmap,
and I don't think we should delay this feature indefinitely to get a design
whose only real benefit is elegance.

We may (almost certainly) need more time to realize the full design. But we
don't need much to take the first steps towards it, which--as I write
above--would already make possible the currently impossible. It seems you'd
rather ship a complete but inelegant design forever than an incomplete but
useful part of an elegant design now. I wouldn't make that trade-off.

···

On Tue, Nov 14, 2017 at 5:49 AM, Brent Royal-Gordon <brent@architechies.com> wrote:

On Nov 13, 2017, at 9:21 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

1. It must be possible to easily access the count of values, and to access any particular value using contiguous `Int` indices. This could be achieved either by directly accessing elements in the list of values through an Int subscript, or by constructing an Array from the list of values.

2. It must be possible to control the order of values in the list of values, either by using source order or through some other simple, straightforward mechanism.

OK, first of all, nowhere in the proposal text are these requirements stated as part of the use case. You're free to put forward new use cases, but here I am trying to design the most elegant way to fulfill a stated need and you're telling me that it's something other than what's written.

Honestly, re-reading the proposal, it never cites a fully-formed use case. Instead, it cites several blog posts, Stack Overflow questions, and small code samples without digging in to the underlying reasons why developers are doing what they're doing. Most of the people discussing it so far seem to have had a tacit understanding that we wanted roughly Array-like access, but we haven't explicitly dug into which properties of an Array are important.

(If anyone involved feels like they had a different understanding of the use case, please speak up.)

I think this is a place where the proposal can be improved, and I'm willing to do some writing to improve it.

You say that:

Essentially all other uses for enumeration of enum cases can be trivially recreated based on just that.

But with this use case in mind, we can see that it is "trivial" in the sense that the annoying boilerplate you need to bridge the significant impedance mismatch is easy to come up with. Yes, you could construct an array using the magic `for` loop, but that would be a serious pain. (And there would be no ergonomic, mistake-resistant way to hide that pain behind a function/initializer call, because there's no way to say that a parameter must be a metatype for an enum.) What you really want is a way to access or construct an `Array` or array-like type containing the type's values.

You cannot truly believe that

var cases = [BugStatus]()
for c in BugStatus.self { cases.append(c) }

is "serious pain." Yes, part of being an incomplete implementation is that it lacks the ergonomics of a fleshed-out conformance to `Collection`. But "serious pain"?

Yes, I'll stand by "serious pain". This is fundamentally a convenience feature, so it needs to actually *be* convenient‚ÄĒmore convenient than writing out the enum cases in an array literal. Forcing users to write imperative-style code that can't be encapsulated is not convenient.

The point here is that even a minimal step towards what we agree is the ideal design would make _possible_ what today is _impossible_: namely, future-proof enumeration of all the cases of an enum type without associated values.

We can take a minimal step towards having the feature…or we can just have the feature.

*Actually* conforming the metatype to `Sequence` or `Collection` would be a different story. There, you could construct `Array`s or access elements using ordinary APIs and type system features. And you could write generic algorithms which used the set of all types: they would require conformance to `Sequence` or `Collection`, and users would specify `Foo.Type` as the generic parameter.

Indeed. The point here is that we don't need a name for this protocol. You'd be able to write useful generic algorithms by using functions such as `map` on `T.self` with intuitive constraints such as `T where T.Type : Collection`. Isn't that a sublime way of expressing exactly what we mean?

It is! As I said, I love the idea of conforming the metatype to `Collection`‚ÄĒit's very elegant. But the only advantage I can identify over this proposal is a slight gain in elegance, while its *disadvantages*‚ÄĒrequiring several nontrivial enhancements to the language, and therefore deferring large amounts of very desirable functionality to an unspecified future release‚ÄĒare significant.

Basically, I don't think it's worth waiting for the "sublime way".

(There's also the disadvantage that the meaning of `Foo.self[i]` is not immediately obvious in the way `Foo.allValues[i]` is. As it is, I'm not totally convinced that `ValueEnumerable` is an obvious enough name; `Foo.self` would be much more problematic in that sense.)

(And there's the opt-in question. Public types may not *want* to participate in this feature, but you seem to suggest they should have to.)

But I suspect that would require deeper compiler changes than we can be certain to get in Swift 5 or really at any specific point on the roadmap, and I don't think we should delay this feature indefinitely to get a design whose only real benefit is elegance.

We may (almost certainly) need more time to realize the full design. But we don't need much to take the first steps towards it, which--as I write above--would already make possible the currently impossible. It seems you'd rather ship a complete but inelegant design forever than an incomplete but useful part of an elegant design now. I wouldn't make that trade-off.

This is a feature people have wanted‚ÄĒasked for constantly‚ÄĒsince Swift was released. SR-30 is the oldest bug labeled "LanguageFeatureRequest" in the Swift.org bug tracker. We've had several different threads, posted by several different people, independently proposing it. Outside the evolution process, questions and answers about how to do this (and related questions, like "how do I get the number of cases?", which is usually a backdoor version of it) are extremely common.

Apple deferred it beyond Swift 2, and Swift open source deferred it beyond Swift 3 and 4, in part because we saw that future associated type features and finalized type metadata formats would make the implementation significantly better if we waited. The type features are now here, and the metadata format will be frozen in the next release. We have everything we need right now to make a very good version of this feature. You propose deferring it again, not because the functionality would be appreciably improved by the delay, but because it could be done more cleverly.

So let me turn the question around: Why *should* we wait? What appreciable advantages, *besides* elegance, do conforming the metatype to `Collection` offer over a static constant? If someone buttonholes you at a conference and asks, "Why did you choose a magic `for` loop, instead of just giving us an array of cases?", what is your answer, and is it one that will satisfy them?

Personally, I don't think I could give the answer "We thought of a design that offered no additional functionality, but was much cooler" with a straight face.

···

On Nov 14, 2017, at 5:21 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

--
Brent Royal-Gordon
Architechies