Improving the UI of generics

A "generic protocol" as Rust implements it, and many people argue for it, is a protocol with multiple independent conforming types, by contrast with associated types, which model a functional dependency between the Self type and the associated types (meaning that for any conforming Self type, you can infer there's one associated type). The most common examples of these I've seen are conversion-style protocols, where you want to be able to say that there's a relationship between arbitrary pairs of types, like (Int, Float), (Int, Double), etc. With only associated types, you wouldn't be able to express this without multiple conformances on Int.

In your example, it looks like really existentials with associated type constraints would be sufficient:

protocol P {
  associatedtype A, B
  init(a: A, b: B)
}
​
extension P {
  typealias Next<T> = any P<.A == T, .B == Self>
  func next<U>(value: U) -> Next<U> {
    .init(a: value, b: self)
  }
}
6 Likes

I have a couple of comments on this:

First, minor one. I'm reading these keywords as prefix operators, so I would expect last one to be meta any P, because it is a [metatype of]-[existential of]-protocol.

Second, major one. While I like the idea of existential abstracting generic types, the idea of associated existential seems fundamentally wrong to me.

Associated types associate types with types. Existential types wrap non-type entities as types by abstraction.

Given

protocol P { associatedtype A }
struct S: P { typealias A = Int }
struct Z<T: P> { ... }
  • Array<S> is a type
  • any Array is a type
  • Array is not, at least not a rank-1 type
  • any P is a type (where P is a protocol)
  • P is not
  • S.A is a type
  • P.A is not a type
  • T is a type
  • T.A is a type

Also for the same reason generic syntax Type<P> cannot be used to express P.Type. Currently all generic parameters are types, and P is not a type. You can have Type<any P>, but that's a P.Protocol.

You're right, this is a much better ordering. I had it backwards.

The idea here is that each type has an "existential" type implicitly associated with it, similar to how the type of the metatype is associated with it via .Type. For existentials and non-generic types this would just be an identity association, but for bound generics types (i.e. Array<Int>) and protocols metatypes this would reference the related existential type. The first post I made in this sub-thread on this topic (which used an earlier version of the syntax) contains some details that weren't stated in the post you quoted.

I hope this helps to clarify what is intended by the term "associated existential".

So association would be between Array<Int> and any Array? And between meta P and any P? I can imagine how the later would be useful, but first one really confuses me.

The association between Array<Int> and any Array can be used to make an encoding of higher-kindred types both more robust and more convenient. I’m not sure if there are any other use cases for it. That’s the one where I stumbled across it.

In any case, this is a relatively minor aspect of the language changes discussed in the related sub-thread. Changes along the lines discussed there would be nice to have for many much more practical reasons.

With the any modifier, there’s an opportunity to revise the current meta type syntax to be more obvious. P.Protocol, the type of P.self, would be (any P).Type, whereas the type of all T.self where T: P would be any P.Type if we say that P.Type is a generic constraint for a meta type conforming to P.

6 Likes

@Joe_Groff Do you see no chance to free up the .Type namespace in Swift?


Btw. @Nickolas_Pohilets here is the translation from Joe's notation to the other meta keyword I was using.

typealias Meta<T> = meta T

P.Protocol == (any P).Type == Meta<any P> == Meta<P> == meta P
    P.Type ==   any P.Type == any Meta<P> == any meta P

Why is there any in the last line?

Right now there are two kind of meta types which are merged into one type, which makes it really hard to work with and which also explains why there is no pure Swift implementation of type(of:) function.

Around Swift 3 end we were working on a proposal to try to push a meta type revamp though while you didn't need to provide an implementation as a proposal author.

Here is that document: https://github.com/DevAndArtist/swift-evolution/blob/refactor_existential_metatypes/proposals/0126-refactor-metatypes.md

Back then we sliced .Type and .Protocol into Type<T> and AnyType<T> where the last type is an existential like type but for meta types.

If we now rename Type<T> to Meta<T> exchange Any prefix by the any keyword, we'll get the exact same types as above:

  • AnyType<P> == any Meta<P> == any P.Type
  • Type<P> == Meta<P> == (any P).Type

Using Joe's syntax version type(of:) would probably look something like this:

func type<T>(of value: T) -> any T.Type

And a 'potential' subtype(of:named:) like so:

func subtype<T>(of type: T.Type, named name: String) -> (any T.Type)?
1 Like

What advantage do you perceive in Type<P> or Meta<P> over meta P? In that syntax, wouldn't your any Meta<P> just be any meta P?

None, that is an old proposal and I like your idea of the meta keyword a lot more. ;) The syntax is different, but the behavior remains exactly the same.

2 Likes

I thought I would be bumping an old thread, but apparently this one is still (kind of) alive!

I didn't get a chance to comment while this was fresh, so I'll leave my thoughts now. While the post is factually/technically accurate, of course, I feel that it's a bit too diminutive towards existentials and almost implies that they are useless or mistakes, or that they're not in the language's future plans. I very much hope that isn't true, and that improving existentials is still on the roadmap somewhere.

I think it's important to argue the case for existentials. Sometimes the post (document?) draws a sharp, fundamental line, and clearly acknowledges that they are very different things, meant for different purposes; and at other times it directly compares them, as though they were interchangeable, and unsurprisingly finds existentials coming up short.

Indeed.

This isn't a brilliant example, IMO - as the rest of the post explains, existentials and generics are entirely different things. I wouldn't say that writing the function this way "loses" type information - it's a different thing entirely.

Maybe the difference isn't obvious enough in the syntax, or perhaps this is the first thing users would try to write and it wouldn't have the behaviour they expect. That's a notation question, and I wouldn't presume to know how others learn to code.

That's a bit of a loaded statement. It's true that existentials can't provide the same type-level guarantees as generic parameters, but that's because that's not what they do. As you said, they are value-level constraints/abstractions. That is the critical thing that makes existentials so useful in the first place. You could equally say that generics won't ever quite reach the flexibility of existentials.

Neither has inherently more "power" than the other.

I really dislike this idea. It's pitched as a solution to accessing associated types from existentials, but I think it is entirely the wrong solution to that problem.

The Collection indexing example proves it - all it does it force-cast. If you consider the various ways this could be used, you'll discover they all amount to force-downcasting. It's no different to saying:

extension Collection {
  subscript(idx: Any) -> Any { self[idx as! Index] }
}

... which you could do today. But I think we can all agree that it's awful.

So why not hoist the casting up a level? Why not have the caller guarantee that the index really is of type (dynamic type of existential 'c').Index? That could be done via conditional casting, or by tracking the provenance of returned values somehow. Then you could call the original Collection method directly and there would be no need for any of this "existential self-conformance" malarkey.

And as it just so happens, the very next point gives us a way to do that:

This is what we should focus on IMO, because it so precisely addresses the issue. If we had a way to talk about the specific type inside an existential, issues with associated types and uses of Self pretty-much melt away. This is a big hole in the type-system anyway: while you can box a value of any type (including a generic type) in an existential box, and transfer it between different boxes (sometimes), you can't actually, truly un-box the existential unless you know the specific type it contains (which defeats much of the purpose of using existentials in the fist place).

I still feel that the post could be kinder towards this approach, though. There are multiple possible interpretations for what "computations derived from a single existential value" could mean:

  1. Does it mean this approach wouldn't scale to multiple values?

    That's not true - we could support conditionally-downcasting other values to type X:

    let <X: Collection> openedX = x // X is now bound to the dynamic type of x
    var start = openedX.startIndex // type: X.Index
    if let openedOther = other as? X {
      // 'openedOther' is also of type X
      start = openedOther.startIndex // type-safe.
    }
    

    Or unboxing them to their own types, with constraints based on X:

    var objects: Collection = ...
    var openedObjects: <X: Collection> = objects
    
    var destination: Collection = ...
    if var rrc = destination as? <R> where R: RangeReplaceableCollection, R.Element == X.Element {
      rrc.append(contentsOf: openedObjects)
      destination = rrc
    }
    
  2. Does it mean that it wouldn't support writing func foo<T>(a: T, b: T) -> [T]? (i.e. binding multiple parameters to the same type).

    Because that seems obvious. Of course a value-level abstraction is not the right thing for expressing constraints across values. That's not what it's for. Just like opaque types have difficulty expressing constraints across different functions. You need a lexically-higher scope to define a single thing that the various abstractions can reference in their own constraints.

What's more - this idea of introducing a local type that we use for unboxing would be great for code that doesn't even use existentials, too. For example, it could allow us to up/downcast protocols with associated types (e.g. casting from Collection -> RandomAccessCollection).

extension Collection {
  func myAlgorithm() { print("Collection default") }
}
extension RandomAccessCollection {
  func myAlgorithm() { print("RAC default") }
}

func doSomething<C: Collection>(_ objects: C) {
  if let rac_objects = objects as? <R: RandomAccessCollection> {
    rac_objects.myAlgorithm() // "RAC default"
  } else {
    objects.myAlgorithm() // "Collection default"
  }
}

(with the compiler inferring that R.Element == C.Element, R.Index == C.Index, as its a downcast)


Anyway, those are my thoughts. I hope existentials haven't been forgotten about - there is clearly some design work to do, but I don't see anything fundamentally flawed.

9 Likes

No one's forgotten about existentials, don't worry. I don't think there's any contradiction in what I wrote and what you said—existentials do lose static information, and you'd have to use casts to recover it. We should certainly make it possible to write those casts, but we should also make it possible to express statically type-safe APIs that don't fundamentally rely on casting.

14 Likes

As discussed in this thread last year, using new syntax to refer to existentials as any TypeName and generics as some TypeName would make these two features more equally represented. Also it could help in understanding the differences and when to use which.

4 Likes

Absolutely - I even said "the post is factually/technically accurate". I just feel that in trying to make the case to the community about the value of opaque types (which it clearly succeeded in doing), it ends up reading a bit unfair towards existentials. That's just my impression, and I wanted to clarify whether existentials have a future in the language and to make the case that they should.

I'm very, very happy they haven't been forgotten about :slight_smile: .

I've always been a massive fan of using the word "any" for existentials. I think it makes the whole model a lot simpler to teach, to learn, to understand and use.

There were some fears in the thread that SIMD's any(...) free-function might make it impractical to use that word for something else, but I really hope we can find a way around it. I can't think of another word that's as accurate and concise as any.

6 Likes

Anything new on this?

4 Likes

Wouldn't the fact that any ProtocolName is only used in a type position stop there from being conflict between it and the any(...) function? Or are we worried about humans being confused by the reuse?

Yes, I'm working on an implementation of general opaque result types, i.e. func foo() -> <T> T where T.U == Int.

17 Likes

Just reading this thread and thought I'd chime in on this. I've done a lot of C# programming over the last couple of years and the main conceptual difference between protocols in Swift and interfaces in C# is that in C# an interface is a reference type, effectively a specialised abstract class. So old concepts are re-used. "Conforming to an interface" = "inheriting the interface". "Interface as a type" vs "interface as a constraint" doesn't arise because the interface is essentially a class and its use as an "existential type" is just plain old inheritance.

I'm not saying things don't get more complicated under the hood or that the above holds perfectly when conforming structs to interfaces (I hardly ever use structs in C#) but this is how things are presented to the programmer and it works well. If you use an interface the way you would an abstract class you virtually never go wrong.

I think where Swift gets complicated by comparison is because protocols are a completely different thing compared to classes/structs and deviate from traditional ideas of inheritance (whilst retaining the same syntax). I'm sure there are reasons for this, just pointing out how it feels to the programmer.

Having said this, associated types in Swift protocols are fantastic compared to generic parameter types in C# interfaces. The amount of boilerplate in C# generics due to the absence of associated types is unbelievable. I wrote a comment in support of a proposal to add associated types to C# interfaces (terminology note: "existential types" means associated types in proposal). Personally I have come to the view that associated type and generic type parameters complement each other well and I am pretty confident the C# team will add associated types (including the Self type) to C# interfaces after they finish their work on static interface members.

6 Likes

That doesn't really clarify much in my mind. Interfaces in C# don't seem to me any more like inheritance than protocols in Swift. In C# you can only inherit from a single base class, but you can conform to multiple interfaces (even with overlapping methods). Calling C#'s interface semantics "inheritance" doesn't help me understand the distinction at all.

As far as I can tell the answer to "why was Swift designed this way instead" is "so that it can be better optimized", but I guess I'm still not convinced that was a worthwhile tradeoff. It just feels like at this point you need to be an expert in type systems to make use of Swift generics with ease, and I never felt that way when working with C#. For sure there were things I couldn't always express in C# generics that maybe I wished I could, but it seemed like the simple cases were simple, and in Swift I don't feel like that's true today.

So far in my career I've used C++ templates, C# generics, and now Swift generics. I remember when learning C++ templates the syntax sometimes was confusing, but overall it was relatively simple to understand how they worked and how to use them. For C# I remember feeling limited at times in what I could do (what kinds of constraints I could use), but otherwise it felt very straightforward. In Swift I often feel like I have no clue what I'm doing, and it feels like even simple things are just overly difficult. I don't feel more productive. I feel like "if I'm not writing a library for a large audience then maybe I just shouldn't even bother because it's probably not worth the effort". To me that just feels like a miss...

Even so, I think the recent proposals to allow existential to be used in more places will help a lot. Other proposals, though, just feel like they require so much more conceptual understanding to even get started, which, again, I never really felt when using C++ or C#.

Terms of Service

Privacy Policy

Cookie Policy