[swift-evolution-announce] [Review] SE-0089: Replace protocol<P1, P2> syntax with Any<P1, P2>


(Thorsten Seitz) #1

I am not going to comment on the proposal (conflict of interest etc). I do want to speak up in support of Brent's points, though.

···

Am 25. Mai 2016 um 10:01 schrieb Austin Zheng via swift-evolution <swift-evolution@swift.org>:

On May 25, 2016, at 12:34 AM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

* What is your evaluation of the proposal?

I am in favor. This is a necessary step towards many future features: class-plus-protocol types, the replacement/reimplementation of AnyObject with Any<class>, existentials with associated types, etc.

One reason to prefer `Any` over `any` which is not listed in the proposal is confusion with the unparameterized `Any` type. Having an uppercase `Any` and a lowercase `any<…>` is going to lead to a lot of confusion; people aren't going to remember whether they need the capitalized form or the lowercase one for any particular use. I don't think we can have `any<...>` unless we're also willing to have an unparameterized `any`, and I think `any` is 100% wrong, because it is absolutely a type but is lowercase.

Since we are trying to cram as many breaking changes as possible into Swift 3, I also think we should consider now, or soon, whether or not we want to draw a strong syntactic line between protocols-as-existentials and protocols-as-constraints by requiring the use of `Any<…>` on all existentials and forbidding its use in constraints. That would mean, for instance, that code like this:

 let printable: CustomStringConvertible = foo

Would now be written:

let printable: Any<CustomStringConvertible> = foo

I'm sure this will be controversial, but I like the idea of marking all existential types using Any-syntax. It makes the distinction between concrete and existential types in code completely clear to the reader. Given that there are some subtle differences in how concrete and existential types can be used (for example, used as the types of values passed to generic functions), I think this is definitely worth considering.

AFAIK an existential type is a type T with type parameters that are still abstract (see for example https://en.wikipedia.org/wiki/Type_system#Existential_types), i.e. have not been assigned concrete values.

In Swift these are protocols with Self type requirements (which are always varying) or with associated types that haven't been assigned, yet.

These currently cannot be used in type declarations but will be usable with Any<...>

Other protocols which do not have Self type requirements or associated types are *not* existential types.

`CustomStringConvertible` in particular which is used in the example is *not* an existential type and therefore should still be usable as:

let printable: CustomStringConvertible = foo

i.e. it would *not* be required to write Any<CustomStringConvertible> because CustomStringConvertible is not an existential type.

While I would love making existential types available at last in Swift with this proposal I will not support requiring all protocols to be written as existentials. This makes no sense.

-Thorsten


Joining Old Orphaned Posts to the Main Threads
(Brent Royal-Gordon) #2

AFAIK an existential type is a type T with type parameters that are still abstract (see for example https://en.wikipedia.org/wiki/Type_system#Existential_types), i.e. have not been assigned concrete values.

My understanding is that, in Swift, the instance used to store something whose concrete type is unknown (i.e. is still abstract), but which is known to conform to some protocol, is called an "existential". Protocols with associated values cannot be packed into normal existentials because, even though we know that the concrete type conforms to some protocol, the associated types represent additional unknowns, and Swift cannot be sure how to translate uses of those unknown types into callable members. Hence, protocols with associated types are sometimes called "non-existential".

If I am misusing the terminology in this area, please understand that that's what I mean when I use that word.

···

--
Brent Royal-Gordon
Architechies


(Jordan Rose) #3

We’re not consistent about it, but an “existential value” is a value with protocol or protocol composition type. My mnemonic for this is that all we know is that certain operations exist (unlike a generic value, where we also have access to the type). John could explain it more formally. We sometimes use “existentials” as a (noun) shorthand for “existential value”.

In the compiler source, all protocols and protocol compositions are referred to as “existential types”, whether they have associated types or not. Again, a protocol asserts the existence (and semantics) of various operations, but nothing else about the conforming type. (Except perhaps that it’s a class.) All protocols are thus “existential types” whether or not the language supports values having that type.

It is incorrect to say that protocols with associated types (or requirements involving Self) are “non-existential”.

Jordan

···

On May 25, 2016, at 05:27, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

AFAIK an existential type is a type T with type parameters that are still abstract (see for example https://en.wikipedia.org/wiki/Type_system#Existential_types), i.e. have not been assigned concrete values.

My understanding is that, in Swift, the instance used to store something whose concrete type is unknown (i.e. is still abstract), but which is known to conform to some protocol, is called an "existential". Protocols with associated values cannot be packed into normal existentials because, even though we know that the concrete type conforms to some protocol, the associated types represent additional unknowns, and Swift cannot be sure how to translate uses of those unknown types into callable members. Hence, protocols with associated types are sometimes called "non-existential".

If I am misusing the terminology in this area, please understand that that's what I mean when I use that word.


(Matthew Johnson) #4

AFAIK an existential type is a type T with type parameters that are still abstract (see for example https://en.wikipedia.org/wiki/Type_system#Existential_types), i.e. have not been assigned concrete values.

My understanding is that, in Swift, the instance used to store something whose concrete type is unknown (i.e. is still abstract), but which is known to conform to some protocol, is called an "existential". Protocols with associated values cannot be packed into normal existentials because, even though we know that the concrete type conforms to some protocol, the associated types represent additional unknowns, and Swift cannot be sure how to translate uses of those unknown types into callable members. Hence, protocols with associated types are sometimes called "non-existential".

If I am misusing the terminology in this area, please understand that that's what I mean when I use that word.

We’re not consistent about it, but an “existential value” is a value with protocol or protocol composition type. My mnemonic for this is that all we know is that certain operations exist (unlike a generic value, where we also have access to the type). John could explain it more formally. We sometimes use “existentials” as a (noun) shorthand for “existential value”.

In the compiler source, all protocols and protocol compositions are referred to as “existential types”, whether they have associated types or not. Again, a protocol asserts the existence (and semantics) of various operations, but nothing else about the conforming type. (Except perhaps that it’s a class.) All protocols are thus “existential types” whether or not the language supports values having that type.

It is incorrect to say that protocols with associated types (or requirements involving Self) are “non-existential”.

I haven't heard people using this term myself, but I imagine they probably mean "can't form an existential value with the protocol". There certainly appears to be a lot of confusion in the community with many not realizing that this is a temporary limitation of the implementation, not a necessary fact.

···

Sent from my iPad

On May 25, 2016, at 12:10 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org> wrote:

On May 25, 2016, at 05:27, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

Jordan
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Jordan Rose) #5

This sort of confusion is why we try to keep the word “existential” out of diagnostics. (We could have also had success going the other way, and consistently referring to protocols as existential even in non-existential-value cases, but originally we decided it was better to stick with Objective-C’s terms even if it was a bit limiting and awkward.)

Jordan

···

On May 25, 2016, at 10:43, Matthew Johnson <matthew@anandabits.com> wrote:

Sent from my iPad

On May 25, 2016, at 12:10 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org> wrote:

On May 25, 2016, at 05:27, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

AFAIK an existential type is a type T with type parameters that are still abstract (see for example https://en.wikipedia.org/wiki/Type_system#Existential_types), i.e. have not been assigned concrete values.

My understanding is that, in Swift, the instance used to store something whose concrete type is unknown (i.e. is still abstract), but which is known to conform to some protocol, is called an "existential". Protocols with associated values cannot be packed into normal existentials because, even though we know that the concrete type conforms to some protocol, the associated types represent additional unknowns, and Swift cannot be sure how to translate uses of those unknown types into callable members. Hence, protocols with associated types are sometimes called "non-existential".

If I am misusing the terminology in this area, please understand that that's what I mean when I use that word.

We’re not consistent about it, but an “existential value” is a value with protocol or protocol composition type. My mnemonic for this is that all we know is that certain operations exist (unlike a generic value, where we also have access to the type). John could explain it more formally. We sometimes use “existentials” as a (noun) shorthand for “existential value”.

In the compiler source, all protocols and protocol compositions are referred to as “existential types”, whether they have associated types or not. Again, a protocol asserts the existence (and semantics) of various operations, but nothing else about the conforming type. (Except perhaps that it’s a class.) All protocols are thus “existential types” whether or not the language supports values having that type.

It is incorrect to say that protocols with associated types (or requirements involving Self) are “non-existential”.

I haven't heard people using this term myself, but I imagine they probably mean "can't form an existential value with the protocol". There certainly appears to be a lot of confusion in the community with many not realizing that this is a temporary limitation of the implementation, not a necessary fact.


(Dave Abrahams) #6

As far as I know there is no known way to make protocols with Self
requirements usefully “existentiable,” or whatever you want to call it.
So unless I'm missing something, in this respect, the limitation is not
temporary at all.

···

on Wed May 25 2016, Matthew Johnson <swift-evolution@swift.org> wrote:

Sent from my iPad

On May 25, 2016, at 12:10 PM, Jordan Rose via swift-evolution >> <swift-evolution@swift.org> wrote:

On May 25, 2016, at 05:27, Brent Royal-Gordon via swift-evolution >>>> <swift-evolution@swift.org> wrote:

AFAIK an existential type is a type T with type parameters that
are still abstract (see for example
https://en.wikipedia.org/wiki/Type_system#Existential_types),
i.e. have not been assigned concrete values.

My understanding is that, in Swift, the instance used to store
something whose concrete type is unknown (i.e. is still abstract),
but which is known to conform to some protocol, is called an
"existential". Protocols with associated values cannot be packed
into normal existentials because, even though we know that the
concrete type conforms to some protocol, the associated types
represent additional unknowns, and Swift cannot be sure how to
translate uses of those unknown types into callable members. Hence,
protocols with associated types are sometimes called
"non-existential".

If I am misusing the terminology in this area, please understand
that that's what I mean when I use that word.

We’re not consistent about it, but an “existential value” is a value
with protocol or protocol composition type. My mnemonic for this is
that all we know is that certain operations exist (unlike a generic
value, where we also have access to the type). John could explain it
more formally. We sometimes use “existentials” as a (noun) shorthand
for “existential value”.

In the compiler source, all protocols and protocol compositions are
referred to as “existential types”, whether they have associated
types or not. Again, a protocol asserts the existence (and
semantics) of various operations, but nothing else about the
conforming type. (Except perhaps that it’s a class.) All protocols
are thus “existential types” whether or not the language supports
values having that type.

It is incorrect to say that protocols with associated types (or
requirements involving Self) are “non-existential”.

I haven't heard people using this term myself, but I imagine they
probably mean "can't form an existential value with the protocol".
There certainly appears to be a lot of confusion in the community with
many not realizing that this is a temporary limitation of the
implementation, not a necessary fact.

--
-Dave


(Matthew Johnson) #7

Sent from my iPhone

Sent from my iPad

AFAIK an existential type is a type T with type parameters that
are still abstract (see for example
https://en.wikipedia.org/wiki/Type_system#Existential_types),
i.e. have not been assigned concrete values.

My understanding is that, in Swift, the instance used to store
something whose concrete type is unknown (i.e. is still abstract),
but which is known to conform to some protocol, is called an
"existential". Protocols with associated values cannot be packed
into normal existentials because, even though we know that the
concrete type conforms to some protocol, the associated types
represent additional unknowns, and Swift cannot be sure how to
translate uses of those unknown types into callable members. Hence,
protocols with associated types are sometimes called
"non-existential".

If I am misusing the terminology in this area, please understand
that that's what I mean when I use that word.

We’re not consistent about it, but an “existential value” is a value
with protocol or protocol composition type. My mnemonic for this is
that all we know is that certain operations exist (unlike a generic
value, where we also have access to the type). John could explain it
more formally. We sometimes use “existentials” as a (noun) shorthand
for “existential value”.

In the compiler source, all protocols and protocol compositions are
referred to as “existential types”, whether they have associated
types or not. Again, a protocol asserts the existence (and
semantics) of various operations, but nothing else about the
conforming type. (Except perhaps that it’s a class.) All protocols
are thus “existential types” whether or not the language supports
values having that type.

It is incorrect to say that protocols with associated types (or
requirements involving Self) are “non-existential”.

I haven't heard people using this term myself, but I imagine they
probably mean "can't form an existential value with the protocol".
There certainly appears to be a lot of confusion in the community with
many not realizing that this is a temporary limitation of the
implementation, not a necessary fact.

As far as I know there is no known way to make protocols with Self
requirements usefully “existentiable,” or whatever you want to call it.
So unless I'm missing something, in this respect, the limitation is not
temporary at all.

Take a look at the Equatable example in the opening existentials
section of Doug's manifesto:
https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160229/011666.html

Yes, note that I said *usefully* “existential.”

Fair enough. But note that I was only talking about the inability to form and open such an existential which appears likely to be a temporary limitation given the generics manifesto (of course things could change).

While we can of
course downcast in this way, you have to handle the failure case and
it's not like you can use this to make a heterogeneous Set<Hashable>.
AFAICT, this is not at all like what happens with associated types,
where Collection<Element: Int> has obvious uses.

We can’t use this to form Set<Hashable> because existentials don’t conform to the protocol. I know there is complexity in implementing this and it is not possible to synthesize conformance of the existential for all protocols, but AFAIK it isn’t a settled point that we won’t try to improve the situation in some way. Maybe we can make progress here somehow.

In the meantime, we can make a simple wrapper type to provide the required conformance and make a Set<HashableWrapper> that allows us to put Hashable into a Set. This isn’t ideal but it is a lot less boilerplate than is involved in manual type erasure today where you need to define a struct, base class, and wrapper class. I’d say that’s a win even if we’d like to do better in the future.

struct HashableWrapper: Hashable {
    var value: Hashable
    public var hashValue: Int { return base.hashValue }
}

public func ==(lhs: HashableWrapper, res: HashableWrapper) -> Bool {
    if let lhsValue = lhs.value openas T { // T is a the type of lhsValue, a copy of the value stored in lhs
        if let rhsValue = rhs.value as? T { // is res also a T?
            // okay: lhsValue and rhsValue are both of type T, which we know is Equatable
            if lhsValue == rhsValue {
                return true
            }
        }
    }
    return false
}

(We could also do this with a generic Wrapper<T> and conditional conformance in an `extension Wrapper: Hashable where T == Hashable` if we don’t want a bunch of wrapper types laying around)

-Matthew

···

On Jun 6, 2016, at 12:22 AM, Dave Abrahams <dabrahams@apple.com> wrote:
on Sun Jun 05 2016, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/>> wrote:

On Jun 5, 2016, at 3:51 PM, Dave Abrahams via swift-evolution >>> <swift-evolution@swift.org> wrote:

on Wed May 25 2016, Matthew Johnson <swift-evolution@swift.org> wrote:

On May 25, 2016, at 12:10 PM, Jordan Rose via swift-evolution >>>>> <swift-evolution@swift.org> wrote:

On May 25, 2016, at 05:27, Brent Royal-Gordon via swift-evolution >>>>>>> <swift-evolution@swift.org> wrote:

--
-Dave


(Dave Abrahams) #8

Sent from my iPhone

Sent from my iPad

AFAIK an existential type is a type T with type parameters that
are still abstract (see for example
https://en.wikipedia.org/wiki/Type_system#Existential_types),
i.e. have not been assigned concrete values.

My understanding is that, in Swift, the instance used to store
something whose concrete type is unknown (i.e. is still abstract),
but which is known to conform to some protocol, is called an
"existential". Protocols with associated values cannot be packed
into normal existentials because, even though we know that the
concrete type conforms to some protocol, the associated types
represent additional unknowns, and Swift cannot be sure how to
translate uses of those unknown types into callable members. Hence,
protocols with associated types are sometimes called
"non-existential".

If I am misusing the terminology in this area, please understand
that that's what I mean when I use that word.

We’re not consistent about it, but an “existential value” is a value
with protocol or protocol composition type. My mnemonic for this is
that all we know is that certain operations exist (unlike a generic
value, where we also have access to the type). John could explain it
more formally. We sometimes use “existentials” as a (noun) shorthand
for “existential value”.

In the compiler source, all protocols and protocol compositions are
referred to as “existential types”, whether they have associated
types or not. Again, a protocol asserts the existence (and
semantics) of various operations, but nothing else about the
conforming type. (Except perhaps that it’s a class.) All protocols
are thus “existential types” whether or not the language supports
values having that type.

It is incorrect to say that protocols with associated types (or
requirements involving Self) are “non-existential”.

I haven't heard people using this term myself, but I imagine they
probably mean "can't form an existential value with the protocol".
There certainly appears to be a lot of confusion in the community with
many not realizing that this is a temporary limitation of the
implementation, not a necessary fact.

As far as I know there is no known way to make protocols with Self
requirements usefully “existentiable,” or whatever you want to

call it.

So unless I'm missing something, in this respect, the limitation

is not

temporary at all.

Take a look at the Equatable example in the opening existentials
section of Doug's manifesto:

https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160229/011666.html

Yes, note that I said *usefully* “existential.”

Fair enough. But note that I was only talking about the inability to
form and open such an existential which appears likely to be a
temporary limitation given the generics manifesto (of course things
could change).

While we can of course downcast in this way, you have to handle the
failure case and it's not like you can use this to make a
heterogeneous Set<Hashable>. AFAICT, this is not at all like what
happens with associated types, where Collection<Element: Int> has
obvious uses.

We can’t use this to form Set<Hashable> because existentials don’t
conform to the protocol. I know there is complexity in implementing
this and it is not possible to synthesize conformance of the
existential for all protocols, but AFAIK it isn’t a settled point that
we won’t try to improve the situation in some way.

Of course. I'm just trying to point out that such existentials are
likely to be a whole lot less useful than many people might think.

Maybe we can make progress here somehow.

Maybe. It's a research project.

In the meantime, we can make a simple wrapper type to provide the
required conformance and make a Set<HashableWrapper> that allows us to
put Hashable into a Set. This isn’t ideal but it is a lot less
boilerplate than is involved in manual type erasure today where you
need to define a struct, base class, and wrapper class. I’d say
that’s a win even if we’d like to do better in the future.

struct HashableWrapper: Hashable {
    var value: Hashable
    public var hashValue: Int { return base.hashValue }
}

public func ==(lhs: HashableWrapper, res: HashableWrapper) -> Bool {
    if let lhsValue = lhs.value openas T { // T is a the type of
lhsValue, a copy of the value stored in lhs
        if let rhsValue = rhs.value as? T { // is res also a T?
            // okay: lhsValue and rhsValue are both of type T, which
we know is Equatable
            if lhsValue == rhsValue {
                return true
            }
        }
    }
    return false
}

(We could also do this with a generic Wrapper<T> and conditional
conformance in an `extension Wrapper: Hashable where T == Hashable` if
we don’t want a bunch of wrapper types laying around)

I don't think I"ve made my point very well. Equatable (the
Self-requirement part of Hashable) has a simple answer when the types
don't match, as I noted in the POP talk. Let me put it differently:
existentializing a protocol with Self requirements comes with
limitations that I'm betting most people haven't recognized. For
example, you won't be able to add two arbitrary FloatingPoint
existentials. I think many people view working with existentials as
“easier” or “cleaner” than working with generics, but haven't realized
that if you step around the type relationships encoded in Self
requirements and associated types you end up with types that appear to
interoperate but in fact trap at runtime unless used in exactly the
right way.

···

on Tue Jun 07 2016, Matthew Johnson <matthew-AT-anandabits.com> wrote:

On Jun 6, 2016, at 12:22 AM, Dave Abrahams <dabrahams@apple.com> > wrote:
on Sun Jun 05 2016, Matthew Johnson <matthew-AT-anandabits.com > <http://matthew-at-anandabits.com/>> wrote:

On Jun 5, 2016, at 3:51 PM, Dave Abrahams via swift-evolution >>>> <swift-evolution@swift.org> wrote:

on Wed May 25 2016, Matthew Johnson <swift-evolution@swift.org> wrote:

On May 25, 2016, at 12:10 PM, Jordan Rose via swift-evolution >>>>>> <swift-evolution@swift.org> wrote:

On May 25, 2016, at 05:27, Brent Royal-Gordon via swift-evolution >>>>>>>> <swift-evolution@swift.org> wrote:

--
Dave


(Matthew Johnson) #9

Sent from my iPhone

Sent from my iPad

AFAIK an existential type is a type T with type parameters that
are still abstract (see for example
https://en.wikipedia.org/wiki/Type_system#Existential_types),
i.e. have not been assigned concrete values.

My understanding is that, in Swift, the instance used to store
something whose concrete type is unknown (i.e. is still abstract),
but which is known to conform to some protocol, is called an
"existential". Protocols with associated values cannot be packed
into normal existentials because, even though we know that the
concrete type conforms to some protocol, the associated types
represent additional unknowns, and Swift cannot be sure how to
translate uses of those unknown types into callable members. Hence,
protocols with associated types are sometimes called
"non-existential".

If I am misusing the terminology in this area, please understand
that that's what I mean when I use that word.

We’re not consistent about it, but an “existential value” is a value
with protocol or protocol composition type. My mnemonic for this is
that all we know is that certain operations exist (unlike a generic
value, where we also have access to the type). John could explain it
more formally. We sometimes use “existentials” as a (noun) shorthand
for “existential value”.

In the compiler source, all protocols and protocol compositions are
referred to as “existential types”, whether they have associated
types or not. Again, a protocol asserts the existence (and
semantics) of various operations, but nothing else about the
conforming type. (Except perhaps that it’s a class.) All protocols
are thus “existential types” whether or not the language supports
values having that type.

It is incorrect to say that protocols with associated types (or
requirements involving Self) are “non-existential”.

I haven't heard people using this term myself, but I imagine they
probably mean "can't form an existential value with the protocol".
There certainly appears to be a lot of confusion in the community with
many not realizing that this is a temporary limitation of the
implementation, not a necessary fact.

As far as I know there is no known way to make protocols with Self
requirements usefully “existentiable,” or whatever you want to

call it.

So unless I'm missing something, in this respect, the limitation

is not

temporary at all.

Take a look at the Equatable example in the opening existentials
section of Doug's manifesto:

https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160229/011666.html

Yes, note that I said *usefully* “existential.”

Fair enough. But note that I was only talking about the inability to
form and open such an existential which appears likely to be a
temporary limitation given the generics manifesto (of course things
could change).

While we can of course downcast in this way, you have to handle the
failure case and it's not like you can use this to make a
heterogeneous Set<Hashable>. AFAICT, this is not at all like what
happens with associated types, where Collection<Element: Int> has
obvious uses.

We can’t use this to form Set<Hashable> because existentials don’t
conform to the protocol. I know there is complexity in implementing
this and it is not possible to synthesize conformance of the
existential for all protocols, but AFAIK it isn’t a settled point that
we won’t try to improve the situation in some way.

Of course. I'm just trying to point out that such existentials are
likely to be a whole lot less useful than many people might think.

Maybe we can make progress here somehow.

Maybe. It's a research project.

In the meantime, we can make a simple wrapper type to provide the
required conformance and make a Set<HashableWrapper> that allows us to
put Hashable into a Set. This isn’t ideal but it is a lot less
boilerplate than is involved in manual type erasure today where you
need to define a struct, base class, and wrapper class. I’d say
that’s a win even if we’d like to do better in the future.

struct HashableWrapper: Hashable {
   var value: Hashable
   public var hashValue: Int { return base.hashValue }
}

public func ==(lhs: HashableWrapper, res: HashableWrapper) -> Bool {
   if let lhsValue = lhs.value openas T { // T is a the type of
lhsValue, a copy of the value stored in lhs
       if let rhsValue = rhs.value as? T { // is res also a T?
           // okay: lhsValue and rhsValue are both of type T, which
we know is Equatable
           if lhsValue == rhsValue {
               return true
           }
       }
   }
   return false
}

(We could also do this with a generic Wrapper<T> and conditional
conformance in an `extension Wrapper: Hashable where T == Hashable` if
we don’t want a bunch of wrapper types laying around)

I don't think I"ve made my point very well. Equatable (the
Self-requirement part of Hashable) has a simple answer when the types
don't match, as I noted in the POP talk.

I agree this is a simple case with an obvious answer.

But there are other cases where existentials formed from protocols with Self requirements will be useful (or more precisely, specific members of those existentials). For example, a member with a Self return type will be exposed on the existential as returning another existential with the same constraints.

Let me put it differently:
existentializing a protocol with Self requirements comes with
limitations that I'm betting most people haven't recognized. For
example, you won't be able to add two arbitrary FloatingPoint
existentials. I think many people view working with existentials as
“easier” or “cleaner” than working with generics

I agree with this. One thing generalized existentials will hopefully do is help to clarify people’s thinking. They will be able to form the existential they want. It just may not expose the operations they expect in a way they expect. Hopefully that will prompt them to think more carefully about what they are trying to do and realize that it may not actually make sense.

I definitely don’t agree with those who would say existentials are “easier” or “cleaner” than generics in the general case. I think that perspective might arise from experience working with simple protocols / interfaces in OO languages where you can’t encode more sophisticated type relationships at all. I can see how in those simple cases some people might view generics as unnecessarily noisy or complex syntactically.

, but haven't realized
that if you step around the type relationships encoded in Self
requirements and associated types you end up with types that appear to
interoperate but in fact trap at runtime unless used in exactly the
right way.

Trap at runtime? How so? Generalized existentials should still be type-safe. Or are you talking about the hypothetical types / behaviors people think they want when they don’t fully understand what is happening...

···

On Jun 7, 2016, at 12:51 PM, Dave Abrahams <dabrahams@apple.com> wrote:
on Tue Jun 07 2016, Matthew Johnson <matthew-AT-anandabits.com> wrote:

On Jun 6, 2016, at 12:22 AM, Dave Abrahams <dabrahams@apple.com> >> wrote:
on Sun Jun 05 2016, Matthew Johnson <matthew-AT-anandabits.com >> <http://matthew-at-anandabits.com/>> wrote:

On Jun 5, 2016, at 3:51 PM, Dave Abrahams via swift-evolution >>>>> <swift-evolution@swift.org> wrote:

on Wed May 25 2016, Matthew Johnson <swift-evolution@swift.org> wrote:

On May 25, 2016, at 12:10 PM, Jordan Rose via swift-evolution >>>>>>> <swift-evolution@swift.org> wrote:

On May 25, 2016, at 05:27, Brent Royal-Gordon via swift-evolution >>>>>>>>> <swift-evolution@swift.org> wrote:

--
Dave


(L Mihalkovic) #10

Sent from my iPhone

Sent from my iPad

AFAIK an existential type is a type T with type parameters that
are still abstract (see for example
https://en.wikipedia.org/wiki/Type_system#Existential_types),
i.e. have not been assigned concrete values.

My understanding is that, in Swift, the instance used to store
something whose concrete type is unknown (i.e. is still abstract),
but which is known to conform to some protocol, is called an
"existential". Protocols with associated values cannot be packed
into normal existentials because, even though we know that the
concrete type conforms to some protocol, the associated types
represent additional unknowns, and Swift cannot be sure how to
translate uses of those unknown types into callable members. Hence,
protocols with associated types are sometimes called
"non-existential".

If I am misusing the terminology in this area, please understand
that that's what I mean when I use that word.

We’re not consistent about it, but an “existential value” is a value
with protocol or protocol composition type. My mnemonic for this is
that all we know is that certain operations exist (unlike a generic
value, where we also have access to the type). John could explain it
more formally. We sometimes use “existentials” as a (noun) shorthand
for “existential value”.

In the compiler source, all protocols and protocol compositions are
referred to as “existential types”, whether they have associated
types or not. Again, a protocol asserts the existence (and
semantics) of various operations, but nothing else about the
conforming type. (Except perhaps that it’s a class.) All protocols
are thus “existential types” whether or not the language supports
values having that type.

It is incorrect to say that protocols with associated types (or
requirements involving Self) are “non-existential”.

I haven't heard people using this term myself, but I imagine they
probably mean "can't form an existential value with the protocol".
There certainly appears to be a lot of confusion in the community with
many not realizing that this is a temporary limitation of the
implementation, not a necessary fact.

As far as I know there is no known way to make protocols with Self
requirements usefully “existentiable,” or whatever you want to

call it.

So unless I'm missing something, in this respect, the limitation

is not

temporary at all.

Take a look at the Equatable example in the opening existentials
section of Doug's manifesto:

https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160229/011666.html

Yes, note that I said *usefully* “existential.”

Fair enough. But note that I was only talking about the inability to
form and open such an existential which appears likely to be a
temporary limitation given the generics manifesto (of course things
could change).

While we can of course downcast in this way, you have to handle the
failure case and it's not like you can use this to make a
heterogeneous Set<Hashable>. AFAICT, this is not at all like what
happens with associated types, where Collection<Element: Int> has
obvious uses.

We can’t use this to form Set<Hashable> because existentials don’t
conform to the protocol. I know there is complexity in implementing
this and it is not possible to synthesize conformance of the
existential for all protocols, but AFAIK it isn’t a settled point that
we won’t try to improve the situation in some way.

Of course. I'm just trying to point out that such existentials are
likely to be a whole lot less useful than many people might think.

Maybe we can make progress here somehow.

Maybe. It's a research project.

In the meantime, we can make a simple wrapper type to provide the
required conformance and make a Set<HashableWrapper> that allows us to
put Hashable into a Set. This isn’t ideal but it is a lot less
boilerplate than is involved in manual type erasure today where you
need to define a struct, base class, and wrapper class. I’d say
that’s a win even if we’d like to do better in the future.

struct HashableWrapper: Hashable {
   var value: Hashable
   public var hashValue: Int { return base.hashValue }
}

public func ==(lhs: HashableWrapper, res: HashableWrapper) -> Bool {
   if let lhsValue = lhs.value openas T { // T is a the type of
lhsValue, a copy of the value stored in lhs
       if let rhsValue = rhs.value as? T { // is res also a T?
           // okay: lhsValue and rhsValue are both of type T, which
we know is Equatable
           if lhsValue == rhsValue {
               return true
           }
       }
   }
   return false
}

(We could also do this with a generic Wrapper<T> and conditional
conformance in an `extension Wrapper: Hashable where T == Hashable` if
we don’t want a bunch of wrapper types laying around)

I don't think I"ve made my point very well. Equatable (the
Self-requirement part of Hashable) has a simple answer when the types
don't match, as I noted in the POP talk. Let me put it differently:
existentializing a protocol with Self requirements comes with
limitations that I'm betting most people haven't recognized. For
example, you won't be able to add two arbitrary FloatingPoint
existentials. I think many people view working with existentials as
“easier” or “cleaner” than working with generics, but haven't realized
that if you step around the type relationships encoded in Self
requirements and associated types you end up with types that appear to
interoperate but in fact trap at runtime unless used in exactly the
right way.

I was thinking more about that, and it seems that of all the possible generics extensions, the least ‘generic’ might be the most interesting in the short term: a type safe way to open existentials

But rather than Doug, strawman suggestion:
if let storedInE1 = e1 openas T { // T is a the type of storedInE1, a copy of the value stored in e1
  if let storedInE2 = e2 as? T { // is e2 also a T?
    if storedInE1 == storedInE2 { … } // okay: storedInT1 and storedInE2 are both of type T, which we know is Equatable
  }
}
I’d rather have something like this (using the new extended ‘if/where’)
if let storedInE1 = e1 as? T;
   let storedInE2 = e2 as? T;
       storedInE1 == storedInE2 { … } // okay: storedInT1 and storedInE2 are both of type T, which we know is Equatable
  }
}
which with my ongoing little pet-project might go down to
if let! e1 as? T;
   let! e2 as? T;
       e1 == e2 { … } // okay: e1 and e2 are both of type T, which we know is Equatable
  }
}

···

On Jun 7, 2016, at 7:51 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:
on Tue Jun 07 2016, Matthew Johnson <matthew-AT-anandabits.com> wrote:

On Jun 6, 2016, at 12:22 AM, Dave Abrahams <dabrahams@apple.com> >> wrote:
on Sun Jun 05 2016, Matthew Johnson <matthew-AT-anandabits.com >> <http://matthew-at-anandabits.com/>> wrote:

On Jun 5, 2016, at 3:51 PM, Dave Abrahams via swift-evolution >>>>> <swift-evolution@swift.org> wrote:

on Wed May 25 2016, Matthew Johnson <swift-evolution@swift.org> wrote:

On May 25, 2016, at 12:10 PM, Jordan Rose via swift-evolution >>>>>>> <swift-evolution@swift.org> wrote:

On May 25, 2016, at 05:27, Brent Royal-Gordon via swift-evolution >>>>>>>>> <swift-evolution@swift.org> wrote:

--
Dave
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Dave Abrahams) #11

, but haven't realized
that if you step around the type relationships encoded in Self
requirements and associated types you end up with types that appear to
interoperate but in fact trap at runtime unless used in exactly the
right way.

Trap at runtime? How so? Generalized existentials should still be
type-safe.

There are two choices when you erase static type relationships:

1. Acheive type-safety by trapping at runtime

  FloatingPoint(3.0 as Float) + FloatingPoint(3.0 as Double) // trap

2. Don't expose protocol requirements that involve these relationships,
   which would prevent the code above from compiling and prevent
   FloatingPoint from conforming to itself.

Or are you talking about the hypothetical types / behaviors people
think they want when they don’t fully understand what is happening...

I don't know what you mean here. I think generalized existentials will
be nice to have, but I think most people will want them to do something
they can't possibly do.

···

on Tue Jun 07 2016, Matthew Johnson <swift-evolution@swift.org> wrote:

--
-Dave


(Matthew Johnson) #12

, but haven't realized
that if you step around the type relationships encoded in Self
requirements and associated types you end up with types that appear to
interoperate but in fact trap at runtime unless used in exactly the
right way.

Trap at runtime? How so? Generalized existentials should still be
type-safe.

There are two choices when you erase static type relationships:

1. Acheive type-safety by trapping at runtime

FloatingPoint(3.0 as Float) + FloatingPoint(3.0 as Double) // trap

2. Don't expose protocol requirements that involve these relationships,
  which would prevent the code above from compiling and prevent
  FloatingPoint from conforming to itself.

Or are you talking about the hypothetical types / behaviors people
think they want when they don’t fully understand what is happening...

I don't know what you mean here. I think generalized existentials will
be nice to have, but I think most people will want them to do something
they can't possibly do.

Exactly. What I meant is that people think they want that expression to compile because they don’t understand that the only thing it can do is trap. I said “hypothetical” because producing a compile time error rather than a runtime trap is the only sane thing to do. Your comment surprised me because I can’t imagine we would move forward in Swift with the approach of trapping.

The low hanging fruit in the “protocols whose existentials conform to the protocol” space is simple protocols that can already by existentials today (like CustomStringConvertible). I don’t know enough about Swift’s implementation to comment on how complex it is there, but there aren’t any theoretical problems with making their existentials conform.

···

On Jun 7, 2016, at 4:13 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:
on Tue Jun 07 2016, Matthew Johnson <swift-evolution@swift.org> wrote:

--
-Dave

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Dave Abrahams) #13

, but haven't realized
that if you step around the type relationships encoded in Self
requirements and associated types you end up with types that appear to
interoperate but in fact trap at runtime unless used in exactly the
right way.

Trap at runtime? How so? Generalized existentials should still be
type-safe.

There are two choices when you erase static type relationships:

1. Acheive type-safety by trapping at runtime

FloatingPoint(3.0 as Float) + FloatingPoint(3.0 as Double) // trap

2. Don't expose protocol requirements that involve these relationships,
  which would prevent the code above from compiling and prevent
  FloatingPoint from conforming to itself.

Or are you talking about the hypothetical types / behaviors people
think they want when they don’t fully understand what is happening...

I don't know what you mean here. I think generalized existentials will
be nice to have, but I think most people will want them to do something
they can't possibly do.

Exactly. What I meant is that people think they want that expression
to compile because they don’t understand that the only thing it can do
is trap. I said “hypothetical” because producing a compile time error
rather than a runtime trap is the only sane thing to do. Your comment
surprised me because I can’t imagine we would move forward in Swift
with the approach of trapping.

I would very much like to be able to create instances of “Collection
where Element == Int” so we can throw away the wrappers in the stdlib.
That will require some type mismatches to be caught at runtime via
trapping.

···

on Tue Jun 07 2016, Matthew Johnson <matthew-AT-anandabits.com> wrote:

On Jun 7, 2016, at 4:13 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:
on Tue Jun 07 2016, Matthew Johnson <swift-evolution@swift.org> wrote:

The low hanging fruit in the “protocols whose existentials conform to
the protocol” space is simple protocols that can already by
existentials today (like CustomStringConvertible). I don’t know
enough about Swift’s implementation to comment on how complex it is
there, but there aren’t any theoretical problems with making their
existentials conform.

--
-Dave

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
Dave


(Matthew Johnson) #14

, but haven't realized
that if you step around the type relationships encoded in Self
requirements and associated types you end up with types that appear to
interoperate but in fact trap at runtime unless used in exactly the
right way.

Trap at runtime? How so? Generalized existentials should still be
type-safe.

There are two choices when you erase static type relationships:

1. Acheive type-safety by trapping at runtime

FloatingPoint(3.0 as Float) + FloatingPoint(3.0 as Double) // trap

2. Don't expose protocol requirements that involve these relationships,
which would prevent the code above from compiling and prevent
FloatingPoint from conforming to itself.

Or are you talking about the hypothetical types / behaviors people
think they want when they don’t fully understand what is happening...

I don't know what you mean here. I think generalized existentials will
be nice to have, but I think most people will want them to do something
they can't possibly do.

Exactly. What I meant is that people think they want that expression
to compile because they don’t understand that the only thing it can do
is trap. I said “hypothetical” because producing a compile time error
rather than a runtime trap is the only sane thing to do. Your comment
surprised me because I can’t imagine we would move forward in Swift
with the approach of trapping.

I would very much like to be able to create instances of “Collection
where Element == Int” so we can throw away the wrappers in the stdlib.
That will require some type mismatches to be caught at runtime via
trapping.

For invalid index because the existential accepts a type erased index?

How do you decide where to draw the line here? It feels like a very slippery slope for a language where safety is a stated priority to start adopting a strategy of runtime trapping for something as fundamental as how you expose members on an existential.

IMO you should *have* to introduce unsafe behavior like that manually. Collection indices are already something that isn’t fully statically safe so I understand why you might want to allow this. But I don’t think having the language's existentials do this automatically is the right approach. Maybe there is another approach that could be used in targeted use cases where the less safe behavior makes sense and is carefully designed.

···

On Jun 7, 2016, at 9:15 PM, Dave Abrahams <dabrahams@apple.com> wrote:
on Tue Jun 07 2016, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/>> wrote:

On Jun 7, 2016, at 4:13 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:
on Tue Jun 07 2016, Matthew Johnson <swift-evolution@swift.org> wrote:

The low hanging fruit in the “protocols whose existentials conform to
the protocol” space is simple protocols that can already by
existentials today (like CustomStringConvertible). I don’t know
enough about Swift’s implementation to comment on how complex it is
there, but there aren’t any theoretical problems with making their
existentials conform.

--
-Dave

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
Dave


(Matthew Johnson) #15

, but haven't realized
that if you step around the type relationships encoded in Self
requirements and associated types you end up with types that appear to
interoperate but in fact trap at runtime unless used in exactly the
right way.

Trap at runtime? How so? Generalized existentials should still be
type-safe.

There are two choices when you erase static type relationships:

1. Acheive type-safety by trapping at runtime

FloatingPoint(3.0 as Float) + FloatingPoint(3.0 as Double) // trap

2. Don't expose protocol requirements that involve these relationships,
which would prevent the code above from compiling and prevent
FloatingPoint from conforming to itself.

Or are you talking about the hypothetical types / behaviors people
think they want when they don’t fully understand what is happening...

I don't know what you mean here. I think generalized existentials will
be nice to have, but I think most people will want them to do something
they can't possibly do.

Exactly. What I meant is that people think they want that expression
to compile because they don’t understand that the only thing it can do
is trap. I said “hypothetical” because producing a compile time error
rather than a runtime trap is the only sane thing to do. Your comment
surprised me because I can’t imagine we would move forward in Swift
with the approach of trapping.

I would very much like to be able to create instances of “Collection
where Element == Int” so we can throw away the wrappers in the stdlib.
That will require some type mismatches to be caught at runtime via
trapping.

BTW, Austin’s generalized existentials proposal has several examples of how existential collections would work: https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#protocol-apis-and-associated-types.

In what ways do you find this functionality insufficient? What operations do you want to be able to do that this doesn’t support (and can only be supported in an unsafe way that must trap for invalid input)?

···

On Jun 7, 2016, at 9:27 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Jun 7, 2016, at 9:15 PM, Dave Abrahams <dabrahams@apple.com <mailto:dabrahams@apple.com>> wrote:
on Tue Jun 07 2016, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/>> wrote:

On Jun 7, 2016, at 4:13 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
on Tue Jun 07 2016, Matthew Johnson <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

For invalid index because the existential accepts a type erased index?

How do you decide where to draw the line here? It feels like a very slippery slope for a language where safety is a stated priority to start adopting a strategy of runtime trapping for something as fundamental as how you expose members on an existential.

IMO you should *have* to introduce unsafe behavior like that manually. Collection indices are already something that isn’t fully statically safe so I understand why you might want to allow this. But I don’t think having the language's existentials do this automatically is the right approach. Maybe there is another approach that could be used in targeted use cases where the less safe behavior makes sense and is carefully designed.

The low hanging fruit in the “protocols whose existentials conform to
the protocol” space is simple protocols that can already by
existentials today (like CustomStringConvertible). I don’t know
enough about Swift’s implementation to comment on how complex it is
there, but there aren’t any theoretical problems with making their
existentials conform.

--
-Dave

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

--
Dave


(Dave Abrahams) #16

, but haven't realized
that if you step around the type relationships encoded in Self
requirements and associated types you end up with types that appear to
interoperate but in fact trap at runtime unless used in exactly the
right way.

Trap at runtime? How so? Generalized existentials should still be
type-safe.

There are two choices when you erase static type relationships:

1. Acheive type-safety by trapping at runtime

FloatingPoint(3.0 as Float) + FloatingPoint(3.0 as Double) // trap

2. Don't expose protocol requirements that involve these relationships,
which would prevent the code above from compiling and prevent
FloatingPoint from conforming to itself.

Or are you talking about the hypothetical types / behaviors people
think they want when they don’t fully understand what is happening...

I don't know what you mean here. I think generalized existentials will
be nice to have, but I think most people will want them to do something
they can't possibly do.

Exactly. What I meant is that people think they want that expression
to compile because they don’t understand that the only thing it can do
is trap. I said “hypothetical” because producing a compile time error
rather than a runtime trap is the only sane thing to do. Your comment
surprised me because I can’t imagine we would move forward in Swift
with the approach of trapping.

I would very much like to be able to create instances of “Collection
where Element == Int” so we can throw away the wrappers in the stdlib.
That will require some type mismatches to be caught at runtime via
trapping.

For invalid index because the existential accepts a type erased index?

Exactly.

How do you decide where to draw the line here? It feels like a very
slippery slope for a language where safety is a stated priority to
start adopting a strategy of runtime trapping for something as
fundamental as how you expose members on an existential.

If you don't do this, the alternative is that “Collection where Element
== Int” does not conform to Collection. That's weird and not very
useful. You could expose all the methods that were on protocol
extensions of Collection on this existential, unless they used
associated types other than the element type. But you couldn't pass the
existential to a generic function like

   func scrambled<C: Collection>(_ c: C) -> [C.Element]

IMO you should *have* to introduce unsafe behavior like that manually.

  Collection where Element == Int & Index == *

?

Collection indices are already something that isn’t fully statically
safe so I understand why you might want to allow this.

By the same measure, so are Ints :slight_smile:

The fact that a type's methods have preconditions does *not* make it
“statically unsafe.”

But I don’t think having the language's existentials do this
automatically is the right approach. Maybe there is another approach
that could be used in targeted use cases where the less safe behavior
makes sense and is carefully designed.

Whether it makes sense or not really depends on the use-cases. There's
little point in generalizing existentials if the result isn't very useful.
The way to find out is to take a look at the examples we currently have
of protocols with associated types or Self requirements and consider
what you'd be able to do with their existentials if type relationships
couldn't be erased.

We have known use-cases, currently emulated in the standard library, for
existentials with erased type relationships. *If* these represent the
predominant use cases for something like generalized existentials, it
seems to me that the language feature should support that. Note: I have
not seen anyone build an emulation of the other kind of generalized
existential. My theory: there's a good reason for that :-).

···

on Tue Jun 07 2016, Matthew Johnson <matthew-AT-anandabits.com> wrote:

On Jun 7, 2016, at 9:15 PM, Dave Abrahams <dabrahams@apple.com> wrote:
on Tue Jun 07 2016, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/>> wrote:

On Jun 7, 2016, at 4:13 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:
on Tue Jun 07 2016, Matthew Johnson <swift-evolution@swift.org> wrote:

--
Dave


(Matthew Johnson) #17

, but haven't realized
that if you step around the type relationships encoded in Self
requirements and associated types you end up with types that appear to
interoperate but in fact trap at runtime unless used in exactly the
right way.

Trap at runtime? How so? Generalized existentials should still be
type-safe.

There are two choices when you erase static type relationships:

1. Acheive type-safety by trapping at runtime

FloatingPoint(3.0 as Float) + FloatingPoint(3.0 as Double) // trap

2. Don't expose protocol requirements that involve these relationships,
which would prevent the code above from compiling and prevent
FloatingPoint from conforming to itself.

Or are you talking about the hypothetical types / behaviors people
think they want when they don’t fully understand what is happening...

I don't know what you mean here. I think generalized existentials will
be nice to have, but I think most people will want them to do something
they can't possibly do.

Exactly. What I meant is that people think they want that expression
to compile because they don’t understand that the only thing it can do
is trap. I said “hypothetical” because producing a compile time error
rather than a runtime trap is the only sane thing to do. Your comment
surprised me because I can’t imagine we would move forward in Swift
with the approach of trapping.

I would very much like to be able to create instances of “Collection
where Element == Int” so we can throw away the wrappers in the stdlib.
That will require some type mismatches to be caught at runtime via
trapping.

For invalid index because the existential accepts a type erased index?

Exactly.

How do you decide where to draw the line here? It feels like a very
slippery slope for a language where safety is a stated priority to
start adopting a strategy of runtime trapping for something as
fundamental as how you expose members on an existential.

If you don't do this, the alternative is that “Collection where Element
== Int” does not conform to Collection.

This isn’t directly related to having self or associated type requirements. It is true of all existentials. If that changes for simple existentials and generalized existentials expose all members (as in the latest draft of the proposal) maybe it will be possible for all existentials to conform to their protocol.

That's weird and not very
useful. You could expose all the methods that were on protocol
extensions of Collection on this existential, unless they used
associated types other than the element type. But you couldn't pass the
existential to a generic function like

  func scrambled<C: Collection>(_ c: C) -> [C.Element]

IMO you should *have* to introduce unsafe behavior like that manually.

Collection where Element == Int & Index == *

?

I didn’t mean directly through the type of the existential.

One obvious mechanism for introducing unsafe behavior is to write manual type erasure wrappers like we do today.

Another possibility would be to allow extending the existential type (not the protocol). This would allow you to write overloads on the Collection existential that takes some kind of type erased index if that is what you want and either trap if you receive an invalid index or better (IMO) return an `Element?`. I’m not sure how extensions on existentials might be implemented, but this is an example of the kind of operation you might want available on it that you wouldn’t want available on all Collection types.

Collection indices are already something that isn’t fully statically
safe so I understand why you might want to allow this.

By the same measure, so are Ints :slight_smile:

The fact that a type's methods have preconditions does *not* make it
“statically unsafe.”

That depends on what you mean by safe. Sure, those methods aren’t going corrupt memory, but they *are* going to explicitly and intentionally crash for some inputs. That doesn’t qualify as “fully safe” IMO.

But I don’t think having the language's existentials do this
automatically is the right approach. Maybe there is another approach
that could be used in targeted use cases where the less safe behavior
makes sense and is carefully designed.

Whether it makes sense or not really depends on the use-cases. There's
little point in generalizing existentials if the result isn't very useful.

Usefulness depends on your perspective. I have run into several scenarios where they would be very useful without needing to be prone to crashes when used incorrectly. One obvious basic use case is storing things in a heterogenous collection where you bind .

The way to find out is to take a look at the examples we currently have
of protocols with associated types or Self requirements and consider
what you'd be able to do with their existentials if type relationships
couldn't be erased.

We have known use-cases, currently emulated in the standard library, for
existentials with erased type relationships. *If* these represent the
predominant use cases for something like generalized existentials, it
seems to me that the language feature should support that. Note: I have
not seen anyone build an emulation of the other kind of generalized
existential. My theory: there's a good reason for that :-).

AFAIK (and I could be wrong) the only rules in the language that require the compiler to synthesize a trap except using a nil IUO, `!` on a nil Optional, and an invalid `as` cast . These are all syntactically explicit unsafe / dangerous operations. All other traps are in the standard library (array index, overflow, etc). Most important about all of these cases is that they have received direct human consideration.

Introducing a language (not library) mechanism that exposes members on generalized existentials in a way that relies on runtime traps for type safety feels to me like a pretty dramatic turn agains the stated priority of safety. It will mean you must understand exactly what is going on and be extremely careful to use generalized existentials without causing crashes. This will either make Swift code much more crashy or will scare people away from using generalized existentials (and maybe both). Neither of those outcomes is good.

Collection indices are a somewhat special case as there is already a strong precondition that people are familiar with because it would be too costly to performance and arguably too annoying to deal with an Optional result in every array lookup. IMO that is why the library is able to get away with it in the current type erased AnyCollection. But this is not a good model for exposing any members on an existential that do not already have a strong precondition that causes a trap when violated.

I think a big reason why you maybe haven’t seen a lot of examples of people writing type erased “existentials" is because it is a huge pain in the neck to write this stuff manually today. People may be designing around the need for them. I haven’t seen a huge sampling of type erased “existentials" other people are writing but I haven’t written any that introduce a trap like this. The only traps are in the “abstract" base class whose methods will never be called (and wouldn’t even be implemented if they could be marked abstract).

What specific things do you think we need to be able to do that rely on the compiler synthesizing a trap in the way it exposes the members of the existential?

Here are a few examples from Austin’s proposal that safely use existential collections. I don’t understand why you think this approach is insufficient. Maybe you could supply a concrete example of a use case that can’t be written with the mechanism in Austin’s proposal.

https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#associated-types-and-member-exposure

let a : Any<Collection>

// A variable whose type is the Index associated type of the underlying
// concrete type of 'a'.
let theIndex : a.Index = ...

// A variable whose type is the Element associated type of the underlying
// concrete type of 'a'.
let theElement : a.Element = ...

// Given a mutable collection, swap its first and last items.
// Not a generic function.
func swapFirstAndLast(inout collection: Any<BidirectionalMutableCollection>) {
    // firstIndex and lastIndex both have type "collection.Index"
    guard let firstIndex = collection.startIndex,
        lastIndex = collection.endIndex?.predecessor(collection) where lastIndex != firstIndex else {
            print("Nothing to do")
            return
    }

    // oldFirstItem has type "collection.Element"
    let oldFirstItem = collection[firstIndex]

    collection[firstIndex] = collection[lastIndex]
    collection[lastIndex] = oldFirstItem
}

var a : Any<BidirectionalMutableCollection where .Element == String> = ...

let input = "West Meoley"

// Not actually necessary, since the compiler knows "a.Element" is String.
// A fully constrained anonymous associated type is synonymous with the concrete
// type it's forced to take on, and the two are interchangeable.
// However, 'as' casting is still available if desired.
let anonymousInput = input as a.Element

a[a.startIndex] = anonymousInput

// as mentioned, this also works:
a[a.startIndex] = input

// If the collection allows it, set the first element in the collection to a given string.
func setFirstElementIn(inout collection: Any<Collection> toString string: String) {
    if let element = string as? collection.Element {
        // At this point, 'element' is of type "collection.Element"
        collection[collection.startIndex] = element
    }
}

···

On Jun 8, 2016, at 1:33 PM, Dave Abrahams <dabrahams@apple.com> wrote:
on Tue Jun 07 2016, Matthew Johnson <matthew-AT-anandabits.com> wrote:

On Jun 7, 2016, at 9:15 PM, Dave Abrahams <dabrahams@apple.com> wrote:
on Tue Jun 07 2016, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/>> wrote:

On Jun 7, 2016, at 4:13 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:
on Tue Jun 07 2016, Matthew Johnson <swift-evolution@swift.org> wrote:

--
Dave


(Thorsten Seitz) #18

, but haven't realized
that if you step around the type relationships encoded in Self
requirements and associated types you end up with types that appear to
interoperate but in fact trap at runtime unless used in exactly the
right way.

Trap at runtime? How so? Generalized existentials should still be
type-safe.

There are two choices when you erase static type relationships:

1. Acheive type-safety by trapping at runtime

FloatingPoint(3.0 as Float) + FloatingPoint(3.0 as Double) // trap

2. Don't expose protocol requirements that involve these relationships,
which would prevent the code above from compiling and prevent
FloatingPoint from conforming to itself.

Or are you talking about the hypothetical types / behaviors people
think they want when they don’t fully understand what is happening...

I don't know what you mean here. I think generalized existentials will
be nice to have, but I think most people will want them to do something
they can't possibly do.

Exactly. What I meant is that people think they want that expression
to compile because they don’t understand that the only thing it can do
is trap. I said “hypothetical” because producing a compile time error
rather than a runtime trap is the only sane thing to do. Your comment
surprised me because I can’t imagine we would move forward in Swift
with the approach of trapping.

I would very much like to be able to create instances of “Collection
where Element == Int” so we can throw away the wrappers in the stdlib.
That will require some type mismatches to be caught at runtime via
trapping.

For invalid index because the existential accepts a type erased index?

Exactly.

How do you decide where to draw the line here? It feels like a very
slippery slope for a language where safety is a stated priority to
start adopting a strategy of runtime trapping for something as
fundamental as how you expose members on an existential.

If you don't do this, the alternative is that “Collection where Element
== Int” does not conform to Collection. That's weird and not very
useful. You could expose all the methods that were on protocol
extensions of Collection on this existential, unless they used
associated types other than the element type. But you couldn't pass the
existential to a generic function like

  func scrambled<C: Collection>(_ c: C) -> [C.Element]

I don’t understand. Why couldn’t an existential be passed to that function?

-Thorsten

···

Am 08.06.2016 um 20:33 schrieb Dave Abrahams via swift-evolution <swift-evolution@swift.org>:
on Tue Jun 07 2016, Matthew Johnson <matthew-AT-anandabits.com> wrote:

On Jun 7, 2016, at 9:15 PM, Dave Abrahams <dabrahams@apple.com> wrote:
on Tue Jun 07 2016, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/>> wrote:

On Jun 7, 2016, at 4:13 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:
on Tue Jun 07 2016, Matthew Johnson <swift-evolution@swift.org> wrote:

IMO you should *have* to introduce unsafe behavior like that manually.

Collection where Element == Int & Index == *

?

Collection indices are already something that isn’t fully statically
safe so I understand why you might want to allow this.

By the same measure, so are Ints :slight_smile:

The fact that a type's methods have preconditions does *not* make it
“statically unsafe.”

But I don’t think having the language's existentials do this
automatically is the right approach. Maybe there is another approach
that could be used in targeted use cases where the less safe behavior
makes sense and is carefully designed.

Whether it makes sense or not really depends on the use-cases. There's
little point in generalizing existentials if the result isn't very useful.
The way to find out is to take a look at the examples we currently have
of protocols with associated types or Self requirements and consider
what you'd be able to do with their existentials if type relationships
couldn't be erased.

We have known use-cases, currently emulated in the standard library, for
existentials with erased type relationships. *If* these represent the
predominant use cases for something like generalized existentials, it
seems to me that the language feature should support that. Note: I have
not seen anyone build an emulation of the other kind of generalized
existential. My theory: there's a good reason for that :-).

--
Dave
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Austin Zheng) #19

It's not possible, even with Swift's current implementation of
existentials. A protocol type P isn't considered to conform to itself, thus
the following is rejected:

let a : MyProtocol = // ...
func myFunc<T : MyProtocol>(x: T) {
  // ....
}
myFunc(a) // "Cannot invoke 'myFunc' with an argument list of type
MyProtocol"

Changing how this works is probably worth a proposal by itself.

Austin

···

On Wed, Jun 8, 2016 at 12:34 PM, Thorsten Seitz via swift-evolution < swift-evolution@swift.org> wrote:

> Am 08.06.2016 um 20:33 schrieb Dave Abrahams via swift-evolution < > swift-evolution@swift.org>:
>
>
> on Tue Jun 07 2016, Matthew Johnson <matthew-AT-anandabits.com> wrote:
>
>>> On Jun 7, 2016, at 9:15 PM, Dave Abrahams <dabrahams@apple.com> wrote:
>>>
>>>
>>> on Tue Jun 07 2016, Matthew Johnson <matthew-AT-anandabits.com < > http://matthew-at-anandabits.com/>> wrote:
>>>
>>
>>>>> On Jun 7, 2016, at 4:13 PM, Dave Abrahams via swift-evolution < > swift-evolution@swift.org> wrote:
>>>>>
>>>>>
>>>>> on Tue Jun 07 2016, Matthew Johnson <swift-evolution@swift.org> > wrote:
>>>>>
>>>>
>>>>>>> , but haven't realized
>>>>>>> that if you step around the type relationships encoded in Self
>>>>>>> requirements and associated types you end up with types that
appear to
>>>>>>> interoperate but in fact trap at runtime unless used in exactly the
>>>>>>> right way.
>>>>>>
>>>>>> Trap at runtime? How so? Generalized existentials should still be
>>>>>> type-safe.
>>>>>
>>>>> There are two choices when you erase static type relationships:
>>>>>
>>>>> 1. Acheive type-safety by trapping at runtime
>>>>>
>>>>> FloatingPoint(3.0 as Float) + FloatingPoint(3.0 as Double) // trap
>>>>>
>>>>> 2. Don't expose protocol requirements that involve these
relationships,
>>>>> which would prevent the code above from compiling and prevent
>>>>> FloatingPoint from conforming to itself.
>>>>>
>>>>>> Or are you talking about the hypothetical types / behaviors people
>>>>>> think they want when they don’t fully understand what is
happening...
>>>>>
>>>>> I don't know what you mean here. I think generalized existentials
will
>>>>> be nice to have, but I think most people will want them to do
something
>>>>> they can't possibly do.
>>>>
>>>> Exactly. What I meant is that people think they want that expression
>>>> to compile because they don’t understand that the only thing it can do
>>>> is trap. I said “hypothetical” because producing a compile time error
>>>> rather than a runtime trap is the only sane thing to do. Your comment
>>>> surprised me because I can’t imagine we would move forward in Swift
>>>> with the approach of trapping.
>>>
>>> I would very much like to be able to create instances of “Collection
>>> where Element == Int” so we can throw away the wrappers in the stdlib.
>>> That will require some type mismatches to be caught at runtime via
>>> trapping.
>>
>> For invalid index because the existential accepts a type erased index?
>
> Exactly.
>
>> How do you decide where to draw the line here? It feels like a very
>> slippery slope for a language where safety is a stated priority to
>> start adopting a strategy of runtime trapping for something as
>> fundamental as how you expose members on an existential.
>
> If you don't do this, the alternative is that “Collection where Element
> == Int” does not conform to Collection. That's weird and not very
> useful. You could expose all the methods that were on protocol
> extensions of Collection on this existential, unless they used
> associated types other than the element type. But you couldn't pass the
> existential to a generic function like
>
> func scrambled<C: Collection>(_ c: C) -> [C.Element]

I don’t understand. Why couldn’t an existential be passed to that function?

-Thorsten

>
>> IMO you should *have* to introduce unsafe behavior like that manually.
>
> Collection where Element == Int & Index == *
>
> ?
>
>> Collection indices are already something that isn’t fully statically
>> safe so I understand why you might want to allow this.
>
> By the same measure, so are Ints :slight_smile:
>
> The fact that a type's methods have preconditions does *not* make it
> “statically unsafe.”
>
>> But I don’t think having the language's existentials do this
>> automatically is the right approach. Maybe there is another approach
>> that could be used in targeted use cases where the less safe behavior
>> makes sense and is carefully designed.
>
> Whether it makes sense or not really depends on the use-cases. There's
> little point in generalizing existentials if the result isn't very
useful.
> The way to find out is to take a look at the examples we currently have
> of protocols with associated types or Self requirements and consider
> what you'd be able to do with their existentials if type relationships
> couldn't be erased.
>
> We have known use-cases, currently emulated in the standard library, for
> existentials with erased type relationships. *If* these represent the
> predominant use cases for something like generalized existentials, it
> seems to me that the language feature should support that. Note: I have
> not seen anyone build an emulation of the other kind of generalized
> existential. My theory: there's a good reason for that :-).
>
> --
> Dave
> _______________________________________________
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Thorsten Seitz) #20

Ah, thanks, I forgot!
I still consider this a bug, though (will have to read up again what the reasons are for that behavior).

-Thorsten

···

Am 08.06.2016 um 21:43 schrieb Austin Zheng <austinzheng@gmail.com>:

It's not possible, even with Swift's current implementation of existentials. A protocol type P isn't considered to conform to itself, thus the following is rejected:

let a : MyProtocol = // ...
func myFunc<T : MyProtocol>(x: T) {
  // ....
}
myFunc(a) // "Cannot invoke 'myFunc' with an argument list of type MyProtocol"

Changing how this works is probably worth a proposal by itself.

Austin

On Wed, Jun 8, 2016 at 12:34 PM, Thorsten Seitz via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

> Am 08.06.2016 um 20:33 schrieb Dave Abrahams via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>>:
>
>
> on Tue Jun 07 2016, Matthew Johnson <matthew-AT-anandabits.com> wrote:
>
>>> On Jun 7, 2016, at 9:15 PM, Dave Abrahams <dabrahams@apple.com <mailto:dabrahams@apple.com>> wrote:
>>>
>>>
>>> on Tue Jun 07 2016, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/>> wrote:
>>>
>>
>>>>> On Jun 7, 2016, at 4:13 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
>>>>>
>>>>>
>>>>> on Tue Jun 07 2016, Matthew Johnson <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
>>>>>
>>>>
>>>>>>> , but haven't realized
>>>>>>> that if you step around the type relationships encoded in Self
>>>>>>> requirements and associated types you end up with types that appear to
>>>>>>> interoperate but in fact trap at runtime unless used in exactly the
>>>>>>> right way.
>>>>>>
>>>>>> Trap at runtime? How so? Generalized existentials should still be
>>>>>> type-safe.
>>>>>
>>>>> There are two choices when you erase static type relationships:
>>>>>
>>>>> 1. Acheive type-safety by trapping at runtime
>>>>>
>>>>> FloatingPoint(3.0 as Float) + FloatingPoint(3.0 as Double) // trap
>>>>>
>>>>> 2. Don't expose protocol requirements that involve these relationships,
>>>>> which would prevent the code above from compiling and prevent
>>>>> FloatingPoint from conforming to itself.
>>>>>
>>>>>> Or are you talking about the hypothetical types / behaviors people
>>>>>> think they want when they don’t fully understand what is happening...
>>>>>
>>>>> I don't know what you mean here. I think generalized existentials will
>>>>> be nice to have, but I think most people will want them to do something
>>>>> they can't possibly do.
>>>>
>>>> Exactly. What I meant is that people think they want that expression
>>>> to compile because they don’t understand that the only thing it can do
>>>> is trap. I said “hypothetical” because producing a compile time error
>>>> rather than a runtime trap is the only sane thing to do. Your comment
>>>> surprised me because I can’t imagine we would move forward in Swift
>>>> with the approach of trapping.
>>>
>>> I would very much like to be able to create instances of “Collection
>>> where Element == Int” so we can throw away the wrappers in the stdlib.
>>> That will require some type mismatches to be caught at runtime via
>>> trapping.
>>
>> For invalid index because the existential accepts a type erased index?
>
> Exactly.
>
>> How do you decide where to draw the line here? It feels like a very
>> slippery slope for a language where safety is a stated priority to
>> start adopting a strategy of runtime trapping for something as
>> fundamental as how you expose members on an existential.
>
> If you don't do this, the alternative is that “Collection where Element
> == Int” does not conform to Collection. That's weird and not very
> useful. You could expose all the methods that were on protocol
> extensions of Collection on this existential, unless they used
> associated types other than the element type. But you couldn't pass the
> existential to a generic function like
>
> func scrambled<C: Collection>(_ c: C) -> [C.Element]

I don’t understand. Why couldn’t an existential be passed to that function?

-Thorsten

>
>> IMO you should *have* to introduce unsafe behavior like that manually.
>
> Collection where Element == Int & Index == *
>
> ?
>
>> Collection indices are already something that isn’t fully statically
>> safe so I understand why you might want to allow this.
>
> By the same measure, so are Ints :slight_smile:
>
> The fact that a type's methods have preconditions does *not* make it
> “statically unsafe.”
>
>> But I don’t think having the language's existentials do this
>> automatically is the right approach. Maybe there is another approach
>> that could be used in targeted use cases where the less safe behavior
>> makes sense and is carefully designed.
>
> Whether it makes sense or not really depends on the use-cases. There's
> little point in generalizing existentials if the result isn't very useful.
> The way to find out is to take a look at the examples we currently have
> of protocols with associated types or Self requirements and consider
> what you'd be able to do with their existentials if type relationships
> couldn't be erased.
>
> We have known use-cases, currently emulated in the standard library, for
> existentials with erased type relationships. *If* these represent the
> predominant use cases for something like generalized existentials, it
> seems to me that the language feature should support that. Note: I have
> not seen anyone build an emulation of the other kind of generalized
> existential. My theory: there's a good reason for that :-).
>
> --
> Dave
> _______________________________________________
> swift-evolution mailing list
> swift-evolution@swift.org <mailto:swift-evolution@swift.org>
> https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution