Proposal for generic protocols

Evolution Discussion
1

With the awesome expansion of protocol oriented programming that swift has
allowed, the lack of generic protocols has felt noticeably lacking and
painful in some cases. I made an in depth proposal here:

But the tl;dr is this:

protocol Validator<TypeToValidate> {
  var value: TypeToValidate { get set }
  var valueIfValid: TypeToValidate? { get }
}

struct FooStringValidator: Validator<String> {
  //... implementation
}

let stringValidator: Validator<String>

Look forward to hearing some feedback.

2 Likes
2

The lack of protocol type erasure when associated types are involved is definitely a big problem, and one we'd like to address. However, I don't think moving wholesale to modeling associated types this way is feasible. Many of the standard library protocols have a lot of associated types. 'CollectionType' for instance has its 'Index' type while also inheriting a 'Generator' from SequenceType, and neither of these is what you typically want to parameterize a collection on—you'd want 'CollectionType<Int>' ideally to refer to a collection whose *Element* is Int without having to fully specify the generator and index if you don't care about them. Swift's protocols also support type system features that make some erased protocol types not actual models of their own protocols; `Equatable` is notorious for this, since a type being `Equatable` to its own values does not mean it can be equated to arbitrary other types; `1 == "1"` is nonsense for instance. We have a number of issues that need to be considered here; don't worry, we are considering them!

-Joe

···

On Dec 3, 2015, at 2:12 PM, Tal Atlas <me@tal.by> wrote:

With the awesome expansion of protocol oriented programming that swift has allowed, the lack of generic protocols has felt noticeably lacking and painful in some cases. I made an in depth proposal here: https://github.com/tal/swift-evolution/blob/tal/generic-protocol-proposal/proposals/NNNN-add-generic-protocols.md

But the tl;dr is this:

protocol Validator<TypeToValidate> {
  var value: TypeToValidate { get set }
  var valueIfValid: TypeToValidate? { get }
}

struct FooStringValidator: Validator<String> {
  //... implementation
}

let stringValidator: Validator<String>

6 Likes
3

I have been waiting a long time for something like this. I’m 100% behind this.

···

On 03 Dec 2015, at 23:12, Tal Atlas <me@tal.by> wrote:

With the awesome expansion of protocol oriented programming that swift has allowed, the lack of generic protocols has felt noticeably lacking and painful in some cases. I made an in depth proposal here: https://github.com/tal/swift-evolution/blob/tal/generic-protocol-proposal/proposals/NNNN-add-generic-protocols.md

But the tl;dr is this:

protocol Validator<TypeToValidate> {
  var value: TypeToValidate { get set }
  var valueIfValid: TypeToValidate? { get }
}

struct FooStringValidator: Validator<String> {
  //... implementation
}

let stringValidator: Validator<String>

Look forward to hearing some feedback.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

4

Thanks for the great info. I made this because I didn't see anything on the
current plan and I really hope that whatever comes out has the power and
ease of the proposal.

Thanks again for the great responses.

···

On Thu, Dec 3, 2015 at 5:43 PM Joe Groff <jgroff@apple.com> wrote:

On Dec 3, 2015, at 2:12 PM, Tal Atlas <me@tal.by> wrote:

With the awesome expansion of protocol oriented programming that swift has
allowed, the lack of generic protocols has felt noticeably lacking and
painful in some cases. I made an in depth proposal here:
https://github.com/tal/swift-evolution/blob/tal/generic-protocol-proposal/proposals/NNNN-add-generic-protocols.md

But the tl;dr is this:

protocol Validator<TypeToValidate> {
  var value: TypeToValidate { get set }
  var valueIfValid: TypeToValidate? { get }
}

struct FooStringValidator: Validator<String> {
  //... implementation
}

let stringValidator: Validator<String>

The lack of protocol type erasure when associated types are involved is
definitely a big problem, and one we'd like to address. However, I don't
think moving wholesale to modeling associated types this way is feasible.
Many of the standard library protocols have a lot of associated types.
'CollectionType' for instance has its 'Index' type while also inheriting a
'Generator' from SequenceType, and neither of these is what you typically
want to parameterize a collection on—you'd want 'CollectionType<Int>'
ideally to refer to a collection whose *Element* is Int without having to
fully specify the generator and index if you don't care about them. Swift's
protocols also support type system features that make some erased protocol
types not actual models of their own protocols; `Equatable` is notorious
for this, since a type being `Equatable` to its own values does not mean it
can be equated to arbitrary other types; `1 == "1"` is nonsense for
instance. We have a number of issues that need to be considered here; don't
worry, we are considering them!

-Joe

5

Changing all typealiases in protocols to generic arguments is a pretty
significant change to the language, with a very large impact on existing
code. It also opens the door to implementing the same protocol with
different types, which is something that Swift does not currently allow.

I think Rust's trait system is a good example of the right way to do
this. Rust traits are like Swift protocols, but they started out with
only generics, no associated types. Later on they gained associated
types as well (with the same limitation that Swift protocols have,
where any trait with an associated type cannot be used as a "trait
object"). The end result is Rust traits can have both generics and
associated types, and the choice of which to use depends on what it's
for. Also, a type in Rust can implement the same trait with different
generic parameters (but for any parameterized trait, it can only have
one implementation regardless of associated types). This is also how
Rust has implemented multi-dispatch (Rust does not have method
overloading in general). And the way you're supposed to think about
this is generic type parameters to a trait are "input types", and
associated types are "output types". So any given type can implement
the same protocol as many times as it wants with distinct input types,
but for every set of input types, there is only one set of output
types. And this works very well.

An example of how this is used is the trait that powers the + operator,
called std::ops::Add. It's defined as

pub trait Add<RHS = Self> { type Output; fn add[1](self, rhs: RHS)
-> Self::Output[2]; }

(the `= Self` bit is a defaulted type parameter)

This means that for any given type T, it can support addition with any
number of other types, but for every pair of types (T,U), the expression
`T + U` can only ever have one return value. To demonstrate how this
would work in Swift, you can imagine supporting `+` with NSNumber
against different numeric types:

extension NSNumber: Add<Int> { typealias Output = Int func
add(rhs: Int) -> Int { return integerValue + rhs } }

extension NSNumber: Add<UInt> { typealias Output = UInt func
add(rhs: UInt) -> UInt { return unsignedIntegerValue + rhs } }

// etc...

Besides the clean distinction between "input" and "output" types, this
also allows various traits to have only one or the other. For example,
Rust's equivalent to Swift's Generator is std::iter::Iterator, which has
an associated type for the iterated element. And it makes a lot of sense
for Iterator to use an associated type for this instead of a type
parameter, because it's confusing to have a sequence that can yield
multiple different element types from a call to `seq.generate()` (or in
Rust's case, `seq.iter()`) which would require an explicit type
annotation. And it's even worse when you realize that most code that
iterates over sequences doesn't even care about the concrete generator
type, but the type annotation requires declaring that concrete type (the
alternative, declaring it as `GeneratorType<T>`, will wrap the concrete
type in a protocol object and incur the overhead of extra allocation +
dynamic function dispatch if the optimizer can't remove it).

tl;dr: I want both type parameters and associated types for protocols

-Kevin Ballard

···

On Thu, Dec 3, 2015, at 02:39 PM, David Hart wrote:

I have been waiting a long time for something like this. I’m 100%
behind this.

On 03 Dec 2015, at 23:12, Tal Atlas <me@tal.by> wrote:

With the awesome expansion of protocol oriented programming that
swift has allowed, the lack of generic protocols has felt noticeably
lacking and painful in some cases. I made an in depth proposal here:
https://github.com/tal/swift-evolution/blob/tal/generic-protocol-proposal/proposals/NNNN-add-generic-protocols.md

But the tl;dr is this:

protocol Validator<TypeToValidate> { var value: TypeToValidate { get
set } var valueIfValid: TypeToValidate? { get } }

struct FooStringValidator: Validator<String> { //...
implementation }

let stringValidator: Validator<String>

Look forward to hearing some feedback.

_______________________________________________

swift-evolution mailing list swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_________________________________________________
swift-evolution mailing list swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Links:

  1. Add in std::ops - Rust
  2. Add in std::ops - Rust

3 Likes
6

No problem, thank you for writing this up! If you want to think about this more, some things you might consider:

- What changes would be necessary to the standard library to make the most of this feature? Some particular problems we'd like to solve are to eliminate the need for the AnyGenerator/AnySequence/AnyCollection wrapper types, and to provide a solution for heterogeneous equality, so that protocols can inherit Equatable and Hashable without forfeiting the ability to be used dynamically. See Brent Simmons' Swift diary posts at inessential: Swift Diary for an example of why the latter is important.
- A hybrid approach that allows for both generic parameters and associated types like Rust, as Kevin pointed out, or one where generic parameters are sugar for associated types, might be worth considering.

Thanks again!

-Joe

···

On Dec 3, 2015, at 6:10 PM, Tal Atlas <me@tal.by> wrote:

Thanks for the great info. I made this because I didn't see anything on the current plan and I really hope that whatever comes out has the power and ease of the proposal.

Thanks again for the great responses.

7

How about something along the lines of

  protocol Foo {
      func doThing() -> <C:CollectionType, C.Generator.Element == Int>C
  }

  let coll : <MyColl>MyColl = foo.doThing()
  print("got \(coll)!")

which compiles to be equivalent to

  protocol Foo {
      func doThing(callback:FooDoThingCallback)
  }

  protocol FooDoThingCallback {
      func callback<C : CollectionType where C.Generator.Element == Int>(coll:C)
  }

      foo.doThing(MyCallback())
    struct MyCallback : FooDoThingCallback {
            func callback<C : CollectionType where C.Generator.Element == Int>(coll:C) {
          print("got \(coll)!")
      }
  }

-ken

···

On Dec 3, 2015, at 6:19 PM, Joe Groff <jgroff@apple.com> wrote:

On Dec 3, 2015, at 6:10 PM, Tal Atlas <me@tal.by> wrote:

Thanks for the great info. I made this because I didn't see anything on the current plan and I really hope that whatever comes out has the power and ease of the proposal.

Thanks again for the great responses.

No problem, thank you for writing this up! If you want to think about this more, some things you might consider:

- What changes would be necessary to the standard library to make the most of this feature? Some particular problems we'd like to solve are to eliminate the need for the AnyGenerator/AnySequence/AnyCollection wrapper types, and to provide a solution for heterogeneous equality, so that protocols can inherit Equatable and Hashable without forfeiting the ability to be used dynamically. See Brent Simmons' Swift diary posts at inessential: Swift Diary for an example of why the latter is important.
- A hybrid approach that allows for both generic parameters and associated types like Rust, as Kevin pointed out, or one where generic parameters are sugar for associated types, might be worth considering.

Thanks again!

-Joe
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

8

How about something along the lines of

  protocol Foo {
      func doThing() -> <C:CollectionType, C.Generator.Element == Int>C
  }

  let coll : <MyColl>MyColl = foo.doThing()
  print("got \(coll)!")

which compiles to be equivalent to

  protocol Foo {
      func doThing(callback:FooDoThingCallback)
  }

  protocol FooDoThingCallback {
      func callback<C : CollectionType where C.Generator.Element == Int>(coll:C)
  }

      foo.doThing(MyCallback())
    struct MyCallback : FooDoThingCallback {
            func callback<C : CollectionType where C.Generator.Element == Int>(coll:C) {
          print("got \(coll)!")
      }
  }

-ken

···

On Dec 3, 2015, at 6:19 PM, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:

On Dec 3, 2015, at 6:10 PM, Tal Atlas <me@tal.by <mailto:me@tal.by>> wrote:

Thanks for the great info. I made this because I didn't see anything on the current plan and I really hope that whatever comes out has the power and ease of the proposal.

Thanks again for the great responses.

No problem, thank you for writing this up! If you want to think about this more, some things you might consider:

- What changes would be necessary to the standard library to make the most of this feature? Some particular problems we'd like to solve are to eliminate the need for the AnyGenerator/AnySequence/AnyCollection wrapper types, and to provide a solution for heterogeneous equality, so that protocols can inherit Equatable and Hashable without forfeiting the ability to be used dynamically. See Brent Simmons' Swift diary posts at inessential: Swift Diary for an example of why the latter is important.
- A hybrid approach that allows for both generic parameters and associated types like Rust, as Kevin pointed out, or one where generic parameters are sugar for associated types, might be worth considering.

Thanks again!

-Joe
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

9

Oops, I messed that up a bit. I meant equivalent codegen to

protocol Foo {
    func doThing<CB : FooDoThingCallback>(callback:CB)
}

protocol FooDoThingCallback {
    func callback<C : CollectionType where C.Generator.Element == Int>(coll:C)
}

foo.doThing(Callback())
struct Callback : FooDoThingCallback {
    func callback<C : CollectionType where C.Generator.Element == Int>(coll:C) {
        print("got \(coll)!")
    }
}

···

On Dec 3, 2015, at 6:59 PM, Ken Ferry <kenferry@gmail.com> wrote:

How about something along the lines of

  protocol Foo {
      func doThing() -> <C:CollectionType, C.Generator.Element == Int>C
  }

  let coll : <MyColl>MyColl = foo.doThing()
  print("got \(coll)!")

which compiles to be equivalent to

  protocol Foo {
      func doThing(callback:FooDoThingCallback)
  }

  protocol FooDoThingCallback {
      func callback<C : CollectionType where C.Generator.Element == Int>(coll:C)
  }

      foo.doThing(MyCallback())
    struct MyCallback : FooDoThingCallback {
            func callback<C : CollectionType where C.Generator.Element == Int>(coll:C) {
          print("got \(coll)!")
      }
  }

-ken

On Dec 3, 2015, at 6:19 PM, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:

On Dec 3, 2015, at 6:10 PM, Tal Atlas <me@tal.by <mailto:me@tal.by>> wrote:

Thanks for the great info. I made this because I didn't see anything on the current plan and I really hope that whatever comes out has the power and ease of the proposal.

Thanks again for the great responses.

No problem, thank you for writing this up! If you want to think about this more, some things you might consider:

- What changes would be necessary to the standard library to make the most of this feature? Some particular problems we'd like to solve are to eliminate the need for the AnyGenerator/AnySequence/AnyCollection wrapper types, and to provide a solution for heterogeneous equality, so that protocols can inherit Equatable and Hashable without forfeiting the ability to be used dynamically. See Brent Simmons' Swift diary posts at inessential: Swift Diary for an example of why the latter is important.
- A hybrid approach that allows for both generic parameters and associated types like Rust, as Kevin pointed out, or one where generic parameters are sugar for associated types, might be worth considering.

Thanks again!

-Joe
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

10

Joe, I had an idea to replace _ObjectiveCBridgeable with two protocols in order to pave the way for bridging using as or as? operators from other language types like C++'s map or Python's dict. But I see there is no support for generic protocols. The current design which uses associated types does not fit for this application. We can also use this design to replace protocols like CustomStringConvertible.

See this:

protocol LoselessConvertible<T> {
    func cast() -> T
}

protocol Convertible<T> {
    func cast() -> T?
}

We can use these protocols to replace _ObjectiveCBridgeable internal protocol. For example, the bridiging mechanism for NSData and Data will be:

extension Data: LoselessConvertible<NSData> {
    func cast() -> NSData {
        return NSData(data: self)
        // or 
        // return _backing.bridgedReference(_sliceRange)
    }
}

extension Data: LoselessConvertible<[UInt8]> {
    func cast() -> [UInt8] {
        return [UInt8](self)
    }
}

extension NSData: LoselessConvertible<Data> {
    func cast() -> Data {
        return Data(referencing: src)
    }
}

extension Array: LoselessConvertible<Data> where Element == UInt8 {
    func cast() -> Data {
        return Data(bytes: self)
    }
}

Another use can be casting from C types:

extension UnsafeMutablePointer<UInt8>: LoselessConvertible<String> {
    func cast() -> String {
        return String(cString: self)
    }
}
1 Like
11

If you had generic protocols that would happen naturally since you would have Generator<Int> and Index<Int>. Therefore no need for the associated types. See Collection libries in Java/Scala/Kotlin etc.

12

Having protocols like this opens the door to one type conforming to e.g. both Generator<Int> and Generator<String>, which is a problem.

Rust I think is a good model here. Rust's traits allow for both generic bounds and associated types, and it uses them separately. It uses generic bounds on a trait for "input" types and associated types for "output" types. So for example Rust's Iterator trait uses associated types for the Item element type, so any given type can only conform to Iterator once. And its From<T> trait is generic, so any given type can conform to From multiple times if there are multiple types it can be converted from (e.g. a type could conform to both From<Int> and From<String> if there were reasonable ways to convert from both Int and String to the type). These can also be combined in the same trait; the Add trait, which is used to define the behavior of +, has a generic bound for the RHS, and an associated type for the result. This means for a given type T you could define both Add<Int> and Add<String> in order to support both T + Int and T + String, but for either definition, you can only define a single output type.

8 Likes
13

I think Rust's model is good and I also like Scala, which also has both generics and associated types. If I could only have one though I would prefer generic types over associated; lets hope Swift gets both.

As an aside w.r.t. Add, I have often wanted Int.Add<Int> -> Int and Int.Add<Double> -> Double, or similar, which are versions of Add on Int with different return types (i.e. automatic promotion as part of the type system rather than a compiler hack).

14

I’m not convinced that being able to conform to both Iterator and Iterator should be a deal-breaker... we could just make it an error to write ambiguous code.

func foo<T: Iterator>(...) {
... // This is fine, no ambiguity
}
func foo<T: Iterator>(...) {
... // This is still fine, no ambiguity
}
func foo<T: Iterator & Iterator>(...) {
... // This would require disambiguation... maybe x as : Iterator<Int>
}

15

It's a problem because it's an ergonomics issue. Having one type conform to Iterator twice would make for x in foo be ambiguous (and Array(foo) and foo.map(String.init(describing:)) and anything else like that). There's no technical reason why you can't do it, it just makes working with the type very awkward.

16

Yeah, but that awkwardness would discourage people from doing it without a reason, no? Non-awkward use cases like that From< > example you gave seem pretty compelling to me.

I’m not claiming that having a type conform to Iterator 20 different ways is a good thing, I’m claiming that, IMHO, the functionality it’d enable outweighs the potential for awkwardness.

3 Likes
17

JVM and .NET languages benefit from runtime JIT specialization, so type-erasing Generator and Index is still acceptable once the JIT warms up. There's also no benefit to compile-time specialization in JVM because generics are type erased. Maintaining the 1:1 correspondence between Collections and their associated types is important for AOT performance, and improves the type safety of these APIs, since you can't use a mismatched Index<T> with the wrong kind of Collection. To express that correspondence in Java or C# using generic interfaces, the interface would have to be parameterized on those, Collection<Generator<Int>, Index<Int>>, rather than on the element, which is what you more commonly want.

1 Like
18

The Objective-C bridging behavior of as? is somethnig we're stuck with for compatibility, but not really something I'd want us to extend further. What this kind of protocol requires more specifically is multi-parameter protocols. Generic notation is one way to express that, and Rust does this, but I'm not convinced it's the right design for Swift. As others noted, it doesn't eliminate the need for associated types because of type inference ergonomics, and the difference is very subtle. Since the Protocol<T> syntax is very desirable, I can see it easily being misused to declare multi-parameter protocols where associated types would be more appropriate.

1 Like
19

I think Rust's key insight here was the distinction between "input types" and "output types", and the recognition that a given type T should be able to conform to the same trait multiple times with different "input types", but for any given combination of "input types" there should only be one set of "output types". This is analogous to a function (InputTypes) -> OutputTypes that's polymorphic on the arguments but not on the return type.

As for the syntax, it's desirable, but any syntax that lets you parameterize protocols is desirable. If we have the ability to parameterize a protocol with "input types" then, no matter what the syntax, someone somewhere is going to abuse it to declare a protocol that should really be using associated types. That said, I think one of the reasons Protocol<T> is desirable is because it suggests that you can have parameterized protocol existentials. This is of course something that a lot of us would love to have, so if we gain the ability to write e.g. IteratorProtocol<where Element=String> then that would eliminate the desire to have a parameterized IteratorProtocol<T>.

20

Yeah, by far the most common thing I see people asking for is the parameterization of existentials, which can be applied to associated types or multiple input types equally. I feel like Rust gives the privileged syntax to the more marginal use case—the applications for protocols with multiple peer input types are fairly limited compared to associated types. Rust's approach further introduces a syntactic asymmetry where none is really necessary. For conversion protocols in particular, which seem to be the most commonly raised use case for multiple-input protocols, the relationship is bidirectional, and it's arbitrary which one is "Self" and which one is "T". You could likewise want to type-erase either Self or T relative to the other input type.

5 Likes