[Idea] Generic associated types

I see. Makes sense now. I'm out of my depth here, but that does sound like
what you want here is higher kinded types.

···

On Sun, Mar 12, 2017 at 3:23 AM, Karl Wagner <razielim@gmail.com> wrote:

On 12 Mar 2017, at 09:11, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Sun, Mar 12, 2017 at 3:02 AM, Karl Wagner <razielim@gmail.com> wrote:

On 12 Mar 2017, at 08:50, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Sun, Mar 12, 2017 at 1:39 AM, Karl Wagner <razielim@gmail.com> wrote:

On 12 Mar 2017, at 08:21, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

Sorry, I'm confused. The following works:

protocol Promise {
  associatedtype Result
}

protocol Scanner {
  associatedtype ScannerPromise : Promise
  func frobnicate<T>(_: T) -> ScannerPromise
    where ScannerPromise.Result == T
}

Why does it matter if `ScannerPromise` is generic or not?

That’s some pretty strange syntax. I admit I didn’t even try that.
ScannerPromise would necessarily have to be generic in this context,
because I want to bind one of its associated types to a generic parameter
from a function. There is no way a non-generic type could do that.

That said, even though the compiler accepts your code, it doesn’t seem
to actually work:

protocol Promise {
  associatedtype Result
  func await() -> Result
}

protocol Scanner {
  associatedtype ScannerPromise : Promise
  func frobnicate<T>(_: T) -> ScannerPromise
    where ScannerPromise.Result == T
}

func use<S: Scanner, T>(_ s: S, _ t: T) -> T {
return s.frobnicate(t).await()
}

3.0.2: Segfault

3.1:

error: repl.swift:13:14: error: cannot invoke 'frobnicate' with an
argument list of type '(T)'
    return s.frobnicate(t).await()
             ^

repl.swift:13:14: note: expected an argument list of type '(T)'
    return s.frobnicate(t).await()

That's because your `T` in `use` has no relationship with your `T` in
`protocol Scanner`; you just happen to have chosen the same letter of the
alphabet. This becomes clear if you rename:

func use<S: Scanner, U>(_ s: S, _ t: U) -> U {
  return s.frobnicate(t).await()
}

*// cannot invoke 'frobnicate' with an argument list of type '(U)'*
*// expected an argument list of type '(T)'*

However, this works:

func use<S: Scanner, T>(_ s: S, _ t: T) -> T
  where S.ScannerPromise.Result == T {
  return s.frobnicate(t).await()
}

...or just this:

func use<S: Scanner>(
  _ s: S, _ t: S.ScannerPromise.Result
) -> S.ScannerPromise.Result {
  return s.frobnicate(t).await()
}

- Karl

No, what you’ve done is something different. “frobnicate” is itself a
generic function, so it should work if you rename the parameter to “U".
You’ve just punted the constraint down the abstraction hierarchy.

This becomes clear if you try to continue working at the protocol-level:

extension Scanner {
func synchronised_frob<T>(_ t: T) -> T {
    return frobnicate(t).await()
}
}

error: repl.swift:14:16: error: cannot invoke 'frobnicate' with an
argument list of type '(T)'
        return frobnicate(t).await()
               ^

repl.swift:14:16: note: expected an argument list of type '(T)'
        return frobnicate(t).await()

Hmm, I'm not sure what you're trying to accomplish here. If I have an
instance `s` that conforms to `Scanner`, then it must have a concrete
associated type `ScannerPromise.Result`. `s.frobnicate()` can only take an
argument of type `ScannerPromise.Result`; you want to pass an argument of
arbitrary type `T` and have an existing instance conforming to `Scanner`
retroactively change the type of `ScannerPromise.Result`?

Yes, but I may want ScannerPromise.Result to be a generic parameter. I
refer you to my original email:

Even with SE-0142, this kind of constraint would not be possible. What I
would like to write is something like this:

protocol Promise {
    associatedtype Result
    func await() -> Result
}

protocol Scanner {
    associatedtype ScanPromise<X>: Promise where Result == X // (incl.
SE-0142)

    func promiseScan<T>(from: Offset, until: @escaping (Offset, Item) ->
T?) -> ScanPromise<T?>
}

So, for example, I could write something like this (allowing my
asynchronous Scanners to all automatically implement a synchronous API):

extension Scanner {
    func scan(from f: Offset, until u: (Offset, Item) -> T?) -> T? {
        return promiseScan(from: f, until: u).await()
    }
}

A conforming Promise would look like this:

class MyPromise<Result>: Promise {
    func await() -> Result { … }
}

And a conforming scanner would look like this:

class Scanner {
    typealias ScanPromise<X> = MyPromise<X> // compiler checks where
clause: Result == X

    public func promiseScan<T>(from f: Offset, until u: @escaping (Offset,
Item) -> T?) -> MyPromise<T?> {
       return MyPromise(from: f, until: u)
    }
}

I'm totally +1 with this pitch and was going to draft it myself. Though,
since it's here let's pull it off.

Let's bring the same Functor (event considering it's not a _real_ functor)
example from higher-kinded types:

protocol Functor {
  associatedtype A
  func fmap<FB where FB ~= Self>(f: A -> FB.A) -> FB
}

And make it a functor:

protocol Functor {
  associatedtype A

  associatedtype F<B> : Functor where Functor.A == B

  func map<B>(f: (A) -> B) -> F<B>
}

Now we can implement it in a concrete class the way we want:

protocol Array {
  typealias A = Element

  typealis F<B> = Array<B> //Our where clause fits ok: Functor.A == B;

//Keep in mind we are free to use another class instead of Array and
it still will work.

//This is the difference with higher-kinded.

//I'm not saying higher-kinded is bad, I'm saying I would love to have both :)

  func map<B>(f: (A) -> B) -> F<B> {

  //implementation details

  }
}

The benefit here is that we can use the protocol, use its map in other
places. I.e.

protocol Monad : Functor {

  func flatMap<M : Monad>(f: (A) -> M) -> M {

     //pseudo code

     return flatten(map(f: f))

  }
}

and further... extend Monad, create Applicative, etc.

Does it make sense now?

···

On Sun, Mar 12, 2017 at 10:39 AM, Xiaodi Wu via swift-evolution < swift-evolution@swift.org> wrote:

On Sun, Mar 12, 2017 at 3:23 AM, Karl Wagner <razielim@gmail.com> wrote:

On 12 Mar 2017, at 09:11, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Sun, Mar 12, 2017 at 3:02 AM, Karl Wagner <razielim@gmail.com> wrote:

On 12 Mar 2017, at 08:50, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Sun, Mar 12, 2017 at 1:39 AM, Karl Wagner <razielim@gmail.com> wrote:

On 12 Mar 2017, at 08:21, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

Sorry, I'm confused. The following works:

protocol Promise {
  associatedtype Result
}

protocol Scanner {
  associatedtype ScannerPromise : Promise
  func frobnicate<T>(_: T) -> ScannerPromise
    where ScannerPromise.Result == T
}

Why does it matter if `ScannerPromise` is generic or not?

That’s some pretty strange syntax. I admit I didn’t even try that.
ScannerPromise would necessarily have to be generic in this context,
because I want to bind one of its associated types to a generic parameter
from a function. There is no way a non-generic type could do that.

That said, even though the compiler accepts your code, it doesn’t seem
to actually work:

protocol Promise {
  associatedtype Result
  func await() -> Result
}

protocol Scanner {
  associatedtype ScannerPromise : Promise
  func frobnicate<T>(_: T) -> ScannerPromise
    where ScannerPromise.Result == T
}

func use<S: Scanner, T>(_ s: S, _ t: T) -> T {
return s.frobnicate(t).await()
}

3.0.2: Segfault

3.1:

error: repl.swift:13:14: error: cannot invoke 'frobnicate' with an
argument list of type '(T)'
    return s.frobnicate(t).await()
             ^

repl.swift:13:14: note: expected an argument list of type '(T)'
    return s.frobnicate(t).await()

That's because your `T` in `use` has no relationship with your `T` in
`protocol Scanner`; you just happen to have chosen the same letter of the
alphabet. This becomes clear if you rename:

func use<S: Scanner, U>(_ s: S, _ t: U) -> U {
  return s.frobnicate(t).await()
}

*// cannot invoke 'frobnicate' with an argument list of type '(U)'*
*// expected an argument list of type '(T)'*

However, this works:

func use<S: Scanner, T>(_ s: S, _ t: T) -> T
  where S.ScannerPromise.Result == T {
  return s.frobnicate(t).await()
}

...or just this:

func use<S: Scanner>(
  _ s: S, _ t: S.ScannerPromise.Result
) -> S.ScannerPromise.Result {
  return s.frobnicate(t).await()
}

- Karl

No, what you’ve done is something different. “frobnicate” is itself a
generic function, so it should work if you rename the parameter to “U".
You’ve just punted the constraint down the abstraction hierarchy.

This becomes clear if you try to continue working at the protocol-level:

extension Scanner {
func synchronised_frob<T>(_ t: T) -> T {
    return frobnicate(t).await()
}
}

error: repl.swift:14:16: error: cannot invoke 'frobnicate' with an
argument list of type '(T)'
        return frobnicate(t).await()
               ^

repl.swift:14:16: note: expected an argument list of type '(T)'
        return frobnicate(t).await()

Hmm, I'm not sure what you're trying to accomplish here. If I have an
instance `s` that conforms to `Scanner`, then it must have a concrete
associated type `ScannerPromise.Result`. `s.frobnicate()` can only take an
argument of type `ScannerPromise.Result`; you want to pass an argument of
arbitrary type `T` and have an existing instance conforming to `Scanner`
retroactively change the type of `ScannerPromise.Result`?

Yes, but I may want ScannerPromise.Result to be a generic parameter. I
refer you to my original email:

Even with SE-0142, this kind of constraint would not be possible. What I
would like to write is something like this:

protocol Promise {
    associatedtype Result
    func await() -> Result
}

protocol Scanner {
    associatedtype ScanPromise<X>: Promise where Result == X // (incl.
SE-0142)

    func promiseScan<T>(from: Offset, until: @escaping (Offset, Item) ->
T?) -> ScanPromise<T?>
}

So, for example, I could write something like this (allowing my
asynchronous Scanners to all automatically implement a synchronous API):

extension Scanner {
    func scan(from f: Offset, until u: (Offset, Item) -> T?) -> T? {
        return promiseScan(from: f, until: u).await()
    }
}

A conforming Promise would look like this:

class MyPromise<Result>: Promise {
    func await() -> Result { … }
}

And a conforming scanner would look like this:

class Scanner {
    typealias ScanPromise<X> = MyPromise<X> // compiler checks where
clause: Result == X

    public func promiseScan<T>(from f: Offset, until u: @escaping
(Offset, Item) -> T?) -> MyPromise<T?> {
       return MyPromise(from: f, until: u)
    }
}

I see. Makes sense now. I'm out of my depth here, but that does sound like
what you want here is higher kinded types.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I was thinking of making a new thread for this, but since this discussion’s basically over I thought I’d ask a quick question about generic closures.

Basically, I would like to express something like the following (without type-erasing via AnyCollection):

func vendsGenericBytes(callback: <T: Collection>(_: T) -> Void where T.Iterator.Element == UInt8) {
  callback([1, 2, 3, 4, 5])
  callback(DispatchData.empty)
  callback(UnsafeRawBufferPointer(start: nil, count: 0))
}

// type of vendsGenericBytes is "(<T>(T)->Void where ...)->Void"

func takesGenericBytes<T: Collection>(_: T) where T.Iterator.Element == UInt8 {
  // ...
}
// type of takesGenericBytes is: "<T>(T)->Void where …”
vendsGenericBytes(takesGenericBytes)

vendsGenericBytes {
    if let idx = index(where: { $0 == 0x04 }) {
        //...
    }
}

For functions and closures, you sometimes want to refer to them as an unbound generic because you intend to invoke them with a multitude of types within the same scope, and you want to invoke a specialised function/closure for each type of argument.

For example, vendsGenericBytes might decide to call the callback with a ReversedCollection (if the bytes are coming in reverse), or a LazyMapCollection, FilterCollection or any other kind of collection of UInt8s. You can write a generic function which handles that, but you can’t use it as a callback.

Would this kind of thing ever be supported in Swift? If so, could I (or somebody) add it to the generics manifesto so we don't forget?

"Higher-rank polymorphism". You can already express something similar via generic methods, e.g.

protocol GenericBytesFunction {
  func invoke<T: Collection>(collection: T) where T.Iterator.Element == UInt8
}

You just don't get the convenience of using a closure.

I'd be interested in supporting higher-rank generics directly, but it's one of those "you have to be very cautious" things because (IIRC) type reconstruction is not fully decidable.

John.

···

On Mar 14, 2017, at 2:42 PM, Karl Wagner <razielim@gmail.com> wrote:

- Karl

On 13 Mar 2017, at 17:50, John McCall <rjmccall@apple.com <mailto:rjmccall@apple.com>> wrote:

On Mar 12, 2017, at 5:00 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Mar 12, 2017, at 3:23 PM, Karl Wagner <razielim@gmail.com <mailto:razielim@gmail.com>> wrote:

On 12 Mar 2017, at 14:32, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

This is a really important feature IMO, but as others have pointed out it basically amounts to higher-kinded types. I would love to be wrong about this but I am reasonably sure this is out of scope for Swift 4 (otherwise I would be working on a proposal already).

Sent from my iPad

I’m not an expert on this stuff, but are they still higher-kinded types if we don’t express a relationship between Self and the associated type? I don’t think it’s quite the same conceptual leap as HKT.

I’m no expert either but it sure seems to me like it enables the things usually discussed in the context of higher-kinder types. Maybe someone from the core team can comment on whether there is a meaningful difference

Yes, it's a way of getting some of the behavior of higher-kinded types. Kind of a well-known trick in a number of languages. It's significantly simpler to handle in the type system because the higher-kinded entities stay "second class" — you don't necessarily have to deal with, say, higher-kinded type variables in the constraint solver or in type inference. Of course, that limits some of the code you can write, or at least the simplicity of that code.

and whether this is something that could fit into Swift 4.

No.

John.

Consider that every associated type must be backed by a typealias (explicit or inferred) in the conforming type. We can already have generic typealiases. This would be a more targeted thing which required those associatedtype-implementing-typealiases to contain generic parameters. It would also extend the constraints from SE-0142 to allow constraints to refer to those parameters and bind them to other associated types.

The workaround is basically to erase and dynamic-cast your way out:

Yes, there are workarounds, none of which are desirable.

I ran into a case last year where there was a significant performance impact caused by the need to perform type erasure as a workaround. The type erasing wrapper required an allocation and type information that could have been used by the optimizer was lost. This was frustrating and convinced me that we definitely need HKT in Swift eventually. There are very useful generic libraries that cannot be implemented efficiently without them.

      //NOTE: dynamic type of ScanPromise.Result *must* be same as closure result. No static enforcement though :(

extension Scanner where ScanPromise.Result == Any? {
    func scan<T>(from f: Offset, until u: (Offset, Item) -> T?) -> T? {
        return withoutActuallyEscaping(u) { _u -> T? in
          return promiseScan(from: f, until: _u).await() as? T // downcast from Any? to T?
        }
    }
}

class MyPromise<R>: Promise {
    typealias Result = R?
    let offset: Offset
    let block: (Offset, Item) -> R?
}

class MyScanner: Scanner {
    typealias ScanPromise = MyPromise<Any> // want this to be “typealias ScanPromise<X> = MyPromise<X>"

    func promiseScan<T>(from: Offset, until: @escaping (Offset, Item) -> T?) -> ScanPromise {
        return MyPromise(offset: from, block: until) // upcast from T? to Any?
    }
}

- Karl

On Mar 11, 2017, at 11:49 PM, Karl Wagner via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I have a model like this:

protocol Promise {
    associatedtype Result
}

protocol Scanner {
    associatedtype ScanPromise: Promise

    func promiseScan<T>(from: Offset, until: (Offset, Item) -> T?) -> ScanPromise // where Result == T?
}

The thing that I’m trying to express is: whichever type implements the associated type ‘ScanPromise’ must be generic, and that parameter must be its result (i.e. something it got as a result of calling the “until” closure).

Even with SE-0142, this kind of constraint would not be possible. What I would like to write is something like this:

protocol Promise {
    associatedtype Result
}

protocol Scanner {
    associatedtype ScanPromise<T>: Promise // now generic. [SE-0142]: where Result == T

    func promiseScan<T>(from: Offset, until: (Offset, Item) -> T?) -> ScanPromise<T>
}

Thoughts?

- Karl
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution