[Proposal] Protocols on Steroids


(Howard Lovatt) #1

Proposal: Protocols on Steroids

···

=======================

Change the way protocols and generics work, in particular:
Generic protocol with type parameters inside `<>`, like classes and structs
Allow covariant return types including for generic types
Allow covariant generic argument types with a runtime check that it is of the correct type
Treat Self as a shorthand for the type name; in particular as though there was an extra genetic type, `Type<Self: Type>` and everywhere `Self` appeared in the body the compiler substitutes `Type`
Allow `GenericTypeName.dynamicType` to return the metatype of a generic type
Treat typealias as introducing a generic type parameter that is not part of the type's type signature (or whatever it is renamed to)
Allow implementations in protocols as well as in extensions to protocols (I think this is on the cards already)
Allow default stored properties and default inits in protocol, see `Holder` example below
Disallow overload of a function with a generically typed argument with a function whose argument is derived from the generic type, must be an override.
Implications
------------
Arrays and a like become covariant (with runtime type check for write) - like Java arrays but not Java Lists
`Equatable` and co would not be a special type, could have arrays of `Equatable`s
No need for `AnyXXX` types, the protocol does this directly
No need for `CollectionType` and `Array`, `Array` would become a `protocol` and a `struct`

Example
--------
A holder of 1 value (chosen to keep the example short - think of it as an array):

    protocol Holder<T> {
        default var value: T // See implementation below for what default does
    }

In use it could be used like:

    var holder = Holder(value: 1) // Holder<Int>
    func printout(value: Holder<Any>) {
        print("\(value.value)")
    }
    printout(holder) // Passing a Holder<Int> to a Holder<Any> OK

Implementation
----------------
The above example would be translated by the compiler to something like (though the names would be changed - names chosen to spell out what is happening):

    protocol Holder<T> { // Retains generic type because protocols are types
        var value: T { get set }
    }

    struct DefaultHolder<T>: Holder<T> { // Written because protocol had a default implementation
        var value: T
    }

    struct HolderInt: DefaultHolder<Int> { // Written because a `Holder<Int>` was created
        var _value: Int
        var value: Int {
            get {
                return _value
            }
            set {
                let temp = newValue as! Int // Runtime type check for write
                _value = temp
            }
    }

Other languages
-----------------
This is how Java arrays work, but not Java generics which are cumbersome to use in practice but do offer static type checking on write which neither this proposal or Java arrays offer. Java arrays use a runtime type check.
Dynamic languages and Obj-C normally allow you to store anything in an array (NSArray). This is different than the behaviour proposed, an array would be covariant, i.e. a `[Hashable]` could store anything derived from `Hashable` but could not store an `Equatable` or an `Any` that were not `Hashable`. But the array could be passed to a function expecting an `[Any]`, for example, and that function could read from the array.


(Félix Cloutier) #2

Please split this in smaller pieces. Nobody will want to discuss all of it at once.

Félix

···

Le 30 déc. 2015 à 17:50:44, Howard Lovatt via swift-evolution <swift-evolution@swift.org> a écrit :

Proposal: Protocols on Steroids

Change the way protocols and generics work, in particular:
Generic protocol with type parameters inside `<>`, like classes and structs
Allow covariant return types including for generic types
Allow covariant generic argument types with a runtime check that it is of the correct type
Treat Self as a shorthand for the type name; in particular as though there was an extra genetic type, `Type<Self: Type>` and everywhere `Self` appeared in the body the compiler substitutes `Type`
Allow `GenericTypeName.dynamicType` to return the metatype of a generic type
Treat typealias as introducing a generic type parameter that is not part of the type's type signature (or whatever it is renamed to)
Allow implementations in protocols as well as in extensions to protocols (I think this is on the cards already)
Allow default stored properties and default inits in protocol, see `Holder` example below
Disallow overload of a function with a generically typed argument with a function whose argument is derived from the generic type, must be an override.
Implications
------------
Arrays and a like become covariant (with runtime type check for write) - like Java arrays but not Java Lists
`Equatable` and co would not be a special type, could have arrays of `Equatable`s
No need for `AnyXXX` types, the protocol does this directly
No need for `CollectionType` and `Array`, `Array` would become a `protocol` and a `struct`

Example
--------
A holder of 1 value (chosen to keep the example short - think of it as an array):

    protocol Holder<T> {
        default var value: T // See implementation below for what default does
    }

In use it could be used like:

    var holder = Holder(value: 1) // Holder<Int>
    func printout(value: Holder<Any>) {
        print("\(value.value)")
    }
    printout(holder) // Passing a Holder<Int> to a Holder<Any> OK

Implementation
----------------
The above example would be translated by the compiler to something like (though the names would be changed - names chosen to spell out what is happening):

    protocol Holder<T> { // Retains generic type because protocols are types
        var value: T { get set }
    }

    struct DefaultHolder<T>: Holder<T> { // Written because protocol had a default implementation
        var value: T
    }

    struct HolderInt: DefaultHolder<Int> { // Written because a `Holder<Int>` was created
        var _value: Int
        var value: Int {
            get {
                return _value
            }
            set {
                let temp = newValue as! Int // Runtime type check for write
                _value = temp
            }
    }

Other languages
-----------------
This is how Java arrays work, but not Java generics which are cumbersome to use in practice but do offer static type checking on write which neither this proposal or Java arrays offer. Java arrays use a runtime type check.
Dynamic languages and Obj-C normally allow you to store anything in an array (NSArray). This is different than the behaviour proposed, an array would be covariant, i.e. a `[Hashable]` could store anything derived from `Hashable` but could not store an `Equatable` or an `Any` that were not `Hashable`. But the array could be passed to a function expecting an `[Any]`, for example, and that function could read from the array.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Lily Ballard) #3

As Félix said this is a lot of stuff to cram into one proposal, so much so that I admit I haven't even read it. But skimming it very briefly I found the two following suggestions:

1. Allow covariant generic argument types with a runtime check that
    it is of the correct type

...

1. Arrays and a like become covariant (with runtime type check for
    write) - like Java arrays but not Java Lists

And this makes no sense. Why would you break variance? The only
justification I can see from your email is "because Java Arrays behave
this way", but if anything that's an argument not to do it. Java Arrays
predate Java Generics, and so the only way to write polymorphic
functions that operated on Arrays was to make Array covariant. But this
is generally regarded as a mistake (although I suspect a necessary one).
As you mentioned Java Lists don't behave this way, and that's because
they learned from their mistake (also, with Generics the type could be
safely invariant and functions that operate on it could express the
variance directly).

FWIW, Swift Arrays actually _are_ covariant anyway (just try passing a
[SubClass] to a function that expects [BaseClass]). But not in the sense
that Java Arrays are. Swift's Array is a value type, which means that if
that function then appends a BaseClass instance to the array it got,
that's perfectly safe as it's really just mutating a copy (whereas Java
Arrays are like Obj-C's NSMutableArray i.e. a reference type). I believe
this is modeled internally as simply being an implicit coercion from [U]
to [T] whenever U <: T (but I'm not sure where this is actually defined
in the code). And of course because this is a coercion, it produces a
temporary, and you can't use temporaries with inout parameters, so that
preserves the invariance of arrays passed as inout parameters such as
mutating methods (although if you could pass a temporary it would still
be safe because it would write back to that temporary instead of the
original array; this would be very confusing though which is why it's
disallowed).

-Kevin Ballard


(Lily Ballard) #4

Skimming it again, here's some brief commentary on your other suggestions:

1. Generic protocol with type parameters inside `<>`, like classes
    and structs

I believe this has already been proposed.

1. Allow covariant return types including for generic types

I'm not sure what you mean by this. The return type of a function is
already covariant. As for generics, I'm not sure how the compiler is
supposed to know whether any other type U is a subtype of a generic type
T. In theory, we could modify Swift to allow you to say something like

func foo<T: AnyObject, U: T>(_: T.Type, _ x: U) -> T { return x }

(right now this fails on the `U: T` because it doesn't know that T is a
class type despite the previous bound of `T: AnyObject`)

But I can't think of where this would actually be useful in practice to
do (which may be why Swift doesn't bother to support it).

1. Treat Self as a shorthand for the type name; in particular as
    though there was an extra genetic type, `Type<Self: Type>` and
    everywhere `Self` appeared in the body the compiler substitutes
    `Type`

I suspect this has been proposed before; I know I filed a radar for this
a long time ago. Although just to be clear, what I'm thinking of is
simply the ability to use the token `Self` anywhere inside a type
definition as a convenient way to refer to the enclosing type, and not
actually changing anything about generics or type parameters.

1. Allow `GenericTypeName.dynamicType` to return the metatype of a
    generic type

You can already say `T.self.dynamicType` to get the metatype.

1. Treat typealias as introducing a generic type parameter that is
    not part of the type's type signature (or whatever it is
    renamed to)

I have no idea what you mean by this. How can you have a type parameter
that's not part of the type parameters list?

1. Allow implementations in protocols as well as in extensions to
    protocols (I think this is on the cards already)

This sounds not-unreasonable. Has it been proposed before?

1. Allow default stored properties and default inits in protocol, see
    `Holder` example below

Your example below implies the addition of struct inheritance, which
isn't really something we can support. Inheritance implies a subtyping
relationship, but you cannot have some struct Foo be a subtype of some
other struct Bar if for no other reason than the fact that structs are
value types and so you cannot possibly substitute a value of type Foo in
any code that expects a Bar as it would be the wrong size. You could add
implicit coercions that _slice_ the value (similar to C++), but that's a
pretty nasty route to go down. Note that your example already violates
this; you can't call `printout(holder)` because a Holder<Int> is not the
same size as as Holder<Any>. Besides, you can already define that
function pretty easily with generics as `func printout<T>(value:
Holder<T>)`. I feel like all you're really trying to accomplish here is
the removal of generics from functions (since this looks awfully similar
to what you'd get with covariant mutable Arrays).

I suspect that what you really want is just a way to define common
behavior that's included in other structs, which is basically protocols
except you want to add stored properties too. So in a sense what you
really want is just a way to "embed" one struct in another (a stored
property containing a struct is a form of embedding, but I'm talking
here about embedding all its members directly without going through a
stored property). This is not a true subtyping relationship (see
previous paragraph) but accomplishes what you want.

If you want to propose such a thing, I'd suggest maybe defining it like

struct HolderInt: embeds DefaultHolder<Int> { // ... }

to make it clear that this isn't a subtyping thing (you could make
"embeds" into a member, but that becomes a little confusing then when
you realize that the new struct acquires all of the embedded structs
methods/properties).

That said, a fair amount of thought would have to go into doing this and
making sure it's compatible with any future changes that we want to make
to the language (it's definitely worth a proposal all on its own if you
really want to do it).

Of course, in the end your original suggestion of having a protocol with
stored properties still doesn't work, because it would be impossible to
declare protocol conformance for a type in an extension (which is a
pretty serious limitation). And I'm not really sure what benefit you
have with this over the current approach of simply having to declare the
stored property in the actual struct.

1. Disallow overload of a function with a generically typed argument
    with a function whose argument is derived from the generic type,
    must be an override.

I'm not sure what you mean by this. Can you elaborate? When you say
"derived from the generic type", it makes me think you're talking about
something like

class Foo { func foo<T: SequenceType>(x: T) }

class Bar : Foo { func foo<T: SequenceType>(x: T.Generator) }

But I don't understand why you'd want that second function to be marked
as an override; it's clearly _not_ an override, as it doesn't have the
same type! Note that all overridden functions can call super.func(), but
if you required the "override" keyword here you clearly can't call
super.foo() as the "overridden" function has a different type signature.

1. `Equatable` and co would not be a special type, could have arrays
    of `Equatable`s

It's not a special type already, it's just a protocol. And as far as I
can tell, nothing that you've described would allow you to to have an
array of type `[Equatable]`. I suspect you're thinking that protocol
type parameters + typealiases somehow being implicit type parameters
would do this, except the type `[Equatable]` still doesn't declare the
type. Or with the typealias thing did you actually intend to have
`Equatable` look something like `Equatable<Self=T>`, thus allowing you
to say `[Equatable<Self=Int>]`? But that doesn't actually work, because
if you know that Self is an Int, then you'd just say `[Int]` instead.

An argument can be made for allowing one to specify typealiases on
existential protocol values, e.g. `GeneratorType<Element=Int>` (although
that would get pretty unwieldy when you try to figure out how to specify
a SequenceType or a CollectionType), but there's no sense in attempting
to do that for Self, because if you set Self=T then you should just use
a T directly instead of an existential protocol value.

1. No need for `AnyXXX` types, the protocol does this directly

No it doesn't. Barring the above-mentioned issues with specifying
typealiases on existential protocol values, types like AnySequence also
erase a lot of the specifics of the typealias. Even with your suggested
changes, there'd be no way to say "SequenceType<Element=Int>" because
that leaves the Generator and SubSequence types undefined, which means
you can't actually do much at all with the existential protocol value.
But AnySequence only has a parameter for the element type, the details
of the generator and subsequence are erased.

1. No need for `CollectionType` and `Array`, `Array` would become a
    `protocol` and a `struct`

What? That doesn't make any sense. Even with your struct inheritance
idea, Array can't possibly be the "base" for all collections. If you
want a collection with the stored properties and behavior of Array, you
just use an Array! The whole point of using other collections is because
you want different behavior.

-Kevin Ballard

···

On Wed, Dec 30, 2015, at 02:50 PM, Howard Lovatt via swift-evolution wrote:


(Howard Lovatt) #5

Yeah I can see that "it is too big" is a valid criticism, I will try and split it up. The reason that I didn't split it before was that the proposals work well together. "Sum greater than the parts". Anyway now that there is a place marker for the whole it can be split and reference made to the whole.

This problem of discussing multiple proposals that work well together seems to be a problem with this type of evolution that requires small chunks. Not sure what can be done :frowning:

···

Sent from my iPad

On 31 Dec 2015, at 10:34 AM, Félix Cloutier <felixcca@yahoo.ca> wrote:

Please split this in smaller pieces. Nobody will want to discuss all of it at once.

Félix

Le 30 déc. 2015 à 17:50:44, Howard Lovatt via swift-evolution <swift-evolution@swift.org> a écrit :

Proposal: Protocols on Steroids

Change the way protocols and generics work, in particular:
Generic protocol with type parameters inside `<>`, like classes and structs
Allow covariant return types including for generic types
Allow covariant generic argument types with a runtime check that it is of the correct type
Treat Self as a shorthand for the type name; in particular as though there was an extra genetic type, `Type<Self: Type>` and everywhere `Self` appeared in the body the compiler substitutes `Type`
Allow `GenericTypeName.dynamicType` to return the metatype of a generic type
Treat typealias as introducing a generic type parameter that is not part of the type's type signature (or whatever it is renamed to)
Allow implementations in protocols as well as in extensions to protocols (I think this is on the cards already)
Allow default stored properties and default inits in protocol, see `Holder` example below
Disallow overload of a function with a generically typed argument with a function whose argument is derived from the generic type, must be an override.
Implications
------------
Arrays and a like become covariant (with runtime type check for write) - like Java arrays but not Java Lists
`Equatable` and co would not be a special type, could have arrays of `Equatable`s
No need for `AnyXXX` types, the protocol does this directly
No need for `CollectionType` and `Array`, `Array` would become a `protocol` and a `struct`

Example
--------
A holder of 1 value (chosen to keep the example short - think of it as an array):

    protocol Holder<T> {
        default var value: T // See implementation below for what default does
    }

In use it could be used like:

    var holder = Holder(value: 1) // Holder<Int>
    func printout(value: Holder<Any>) {
        print("\(value.value)")
    }
    printout(holder) // Passing a Holder<Int> to a Holder<Any> OK

Implementation
----------------
The above example would be translated by the compiler to something like (though the names would be changed - names chosen to spell out what is happening):

    protocol Holder<T> { // Retains generic type because protocols are types
        var value: T { get set }
    }

    struct DefaultHolder<T>: Holder<T> { // Written because protocol had a default implementation
        var value: T
    }

    struct HolderInt: DefaultHolder<Int> { // Written because a `Holder<Int>` was created
        var _value: Int
        var value: Int {
            get {
                return _value
            }
            set {
                let temp = newValue as! Int // Runtime type check for write
                _value = temp
            }
    }

Other languages
-----------------
This is how Java arrays work, but not Java generics which are cumbersome to use in practice but do offer static type checking on write which neither this proposal or Java arrays offer. Java arrays use a runtime type check.
Dynamic languages and Obj-C normally allow you to store anything in an array (NSArray). This is different than the behaviour proposed, an array would be covariant, i.e. a `[Hashable]` could store anything derived from `Hashable` but could not store an `Equatable` or an `Any` that were not `Hashable`. But the array could be passed to a function expecting an `[Any]`, for example, and that function could read from the array.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Howard Lovatt) #6

There is a significant downside to variance in Java and Scala, you have to annotate your code all over the place. This annotation completely clutters your code, much like Swift is a lot 'cleaner' than Java, all the annotations detract. You see the same effect in code that uses Java arrays which much 'cleaner' that code that uses `List` (which is the generic equivalent of an array and hence requires variance annotations).

···

Sent from my iPad

On 31 Dec 2015, at 5:00 PM, Kevin Ballard via swift-evolution <swift-evolution@swift.org> wrote:

As Félix said this is a lot of stuff to cram into one proposal, so much so that I admit I haven't even read it. But skimming it very briefly I found the two following suggestions:

Allow covariant generic argument types with a runtime check that it is of the correct type

...

Arrays and a like become covariant (with runtime type check for write) - like Java arrays but not Java Lists

And this makes no sense. Why would you break variance? The only justification I can see from your email is "because Java Arrays behave this way", but if anything that's an argument not to do it. Java Arrays predate Java Generics, and so the only way to write polymorphic functions that operated on Arrays was to make Array covariant. But this is generally regarded as a mistake (although I suspect a necessary one). As you mentioned Java Lists don't behave this way, and that's because they learned from their mistake (also, with Generics the type could be safely invariant and functions that operate on it could express the variance directly).

FWIW, Swift Arrays actually _are_ covariant anyway (just try passing a [SubClass] to a function that expects [BaseClass]). But not in the sense that Java Arrays are. Swift's Array is a value type, which means that if that function then appends a BaseClass instance to the array it got, that's perfectly safe as it's really just mutating a copy (whereas Java Arrays are like Obj-C's NSMutableArray i.e. a reference type). I believe this is modeled internally as simply being an implicit coercion from [U] to [T] whenever U <: T (but I'm not sure where this is actually defined in the code). And of course because this is a coercion, it produces a temporary, and you can't use temporaries with inout parameters, so that preserves the invariance of arrays passed as inout parameters such as mutating methods (although if you could pass a temporary it would still be safe because it would write back to that temporary instead of the original array; this would be very confusing though which is why it's disallowed).

-Kevin Ballard

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Austin Zheng) #7

I would much rather have proper support for covariance/contravariance than pretty but unsound code. It's been stated in other threads that making things pretty is not, in and of itself, a Swift project goal.

I like most of the other proposals, although I think most of them are covered by the expanded Swift 3 generics system.

Austin

···

On Dec 31, 2015, at 5:32 PM, Howard Lovatt via swift-evolution <swift-evolution@swift.org> wrote:

There is a significant downside to variance in Java and Scala, you have to annotate your code all over the place. This annotation completely clutters your code, much like Swift is a lot 'cleaner' than Java, all the annotations detract. You see the same effect in code that uses Java arrays which much 'cleaner' that code that uses `List` (which is the generic equivalent of an array and hence requires variance annotations).

Sent from my iPad

On 31 Dec 2015, at 5:00 PM, Kevin Ballard via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

As Félix said this is a lot of stuff to cram into one proposal, so much so that I admit I haven't even read it. But skimming it very briefly I found the two following suggestions:

Allow covariant generic argument types with a runtime check that it is of the correct type

...

Arrays and a like become covariant (with runtime type check for write) - like Java arrays but not Java Lists

And this makes no sense. Why would you break variance? The only justification I can see from your email is "because Java Arrays behave this way", but if anything that's an argument not to do it. Java Arrays predate Java Generics, and so the only way to write polymorphic functions that operated on Arrays was to make Array covariant. But this is generally regarded as a mistake (although I suspect a necessary one). As you mentioned Java Lists don't behave this way, and that's because they learned from their mistake (also, with Generics the type could be safely invariant and functions that operate on it could express the variance directly).

FWIW, Swift Arrays actually _are_ covariant anyway (just try passing a [SubClass] to a function that expects [BaseClass]). But not in the sense that Java Arrays are. Swift's Array is a value type, which means that if that function then appends a BaseClass instance to the array it got, that's perfectly safe as it's really just mutating a copy (whereas Java Arrays are like Obj-C's NSMutableArray i.e. a reference type). I believe this is modeled internally as simply being an implicit coercion from [U] to [T] whenever U <: T (but I'm not sure where this is actually defined in the code). And of course because this is a coercion, it produces a temporary, and you can't use temporaries with inout parameters, so that preserves the invariance of arrays passed as inout parameters such as mutating methods (although if you could pass a temporary it would still be safe because it would write back to that temporary instead of the original array; this would be very confusing though which is why it's disallowed).

-Kevin Ballard

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution