[Pitch] Introduce existential `any`

If we all agree on the fact that metatypes have metatype existentials and we spelled both of them as T.Type and any T.Type, the addition of type aliases would further visualize the direction I was pushing all along.

typealias Type<T> = T.Type
typealias AnyType<T> = any T.Type

protocol P {}

// if `P` becomes `any P` then

// means `(any P).Type` metatype of the existential P
let _: Type<any P> = ... 

// means any (any P).Type the existential metattype of the existential P
let _: AnyType<any P> = ... 

I do not understand this at all, so I cannot agree with it. More generally I do not understand what you mean when you prefix non-protocols with any and, since there is no such thing suggested in this pitch, I surmise it’s totally separate from the topic of protocol existentials and would strongly suggest leaving that aside for the purposes of this discussion.

Like a protocol, there is also a protocol-as-a-type (protocol existential). In Swift you can ask for an instance of a protocol existential only (e.g. let p: P = ... or let p: any P = ... after this proposal), you cannot ask just for the protocol. In the context of a metatype, you can ask for TWO kinds of metatypes, the final metatype (which is always the result of some T.self) or you can ask for the "existential metatype". The problem is that both are merged into the T.Type spelling where in context of protocols-as-a-type it transforms into P.Protocol (the final metatype). There is not way to express true P.Type, the non P.Protocol version of it.

Sorry it might still be confusing, but this is the status quo. Since the any keyword is meant for existentials, it implies to me that it also has to cover "existential metatypes".

The introduction of the any keyword allows us to finally properly split both kinds of meta types into two separate spellings.

3 Likes

Ahhh, I see now what you mean. And I got myself twisted in knots before. I do not think, actually, that any (any P).Type is the right spelling now that I have unconfused myself.

This is because I agree with the overall direction that any is meant to distinguish protocols from their existential types, not to indicate any supertype that can at runtime store a value of a subtype. That is, I absolutely would not want to see a version of Swift where the superclass Base has to be spelled any Base to indicate that a value could have dynamic type of subclass Derived.

I get your point that you think of the container that can store any metatype as an existential, but I fail to see why it has to be thought of that way. Why not just think of this as another (custom) subtyping relationship, like T? and T, etc.? If it were conceived of in that way, then we could define away a lot of the confusion here by making it so that the metatype of any conforming T: P would just itself be a subtype of the metatype (any P).Type. In that way, (any P).Type would be the “type that can store any concrete metatype conforming to the protocol” simply by being the “metatype of the existential type,” by virtue of the latter being a supertype of all concrete metatypes conforming to the protocol.

6 Likes

This is what most closely matches my own mental model of protocol existentials and the corresponding subtyping relationships—thanks for verbalizing it!

I appreciate your greater point, but this seems like a mischaracterization. Any<P> has been considered to the extent of even getting a full Swift Evolution proposal, SE-0089

It was pitched again by @bjhomer last August.

And my memory of the state of conversation prior to @Joe_Groff’s codification in the “Improving UI” thread was mostly ambivalence toward Any<P> vs. any P.

Thanks! As a sanity check, I refreshed my memory empirically that this is what we do with class metatypes. That is, I can assign a value of type Derived.Type to a variable of type Base.Type. Although existential types do not and cannot self-conform to their corresponding protocols, I am not aware of any reason why their metatypes cannot be supertypes of conforming types’ metatypes—but maybe I am missing something.

This is indeed my understanding as well. This is what I meant above when I said (any P).Type made sense to me because of the way subtyping works with metatypes, and you've clarified it nicely - thank you! The proposal definitely needs more explanation on this topic.

3 Likes

Where should discussions about constrained existentials happen, this thread or the “same-type constraints” thread?

Here’s what I envision the logical extension of this syntax to be, absent the sugar pitched in the other thread:

let p: any Proto
let p_array: [any Proto]
let p_coll: any (Collection where .Element: Proto)

The biggest concern I have about this direction is that it does away with the highly visible angle bracket syntax. Brent complained about angle bracket blindness, but this feels like the opposite—angle bracket erasure, if you will. Whereas creating a special Any<...> syntax clearly indicates the type-level shenanigans:

let p_coll: Any<Collection where .Element: Proto>

While you're not wrong, I view all this with two possible outcomes.

  1. We either keep the status quo and just add any in the context of protocols and simply miss the opportunity to improve something else, just like we missed the opportunity to do more with the open keyword in context of protocols.

  2. We challenge the status quo by fixing a really old issue with metatypes, while not pushing the four extra characters on any other sub-typing relation in Swift. This would at least enable us to express new, more complex types of generic constraints, fix type(of:) and open the door for other reflection like APIs as I already mentioned a couple of times (e.g. the hypothetical subtype(of:named:) function). This option would likely also deprecate the P.Protocol notation, which I can only assume that it's not widely used anyways. It will become obsolete.

Personally I wouldn't mind to expand the notion of any to sub-typing, but hey this evolution process isn't a democracy after all. All I can do is share my 2 cents.

1 Like

+1 for this proposal, have been hoping for a long time for this to happen. Source breaking at Swift 6 sounds good.

Also, in future directions the Extending existential types looks very intuitive and useful.

Whatever happens here, the meaning of .self of a protocol should not change again, and today's .Protocol and .Type must be accessible via the .selfs of two different spellings.

For example, this function call returned true, early last year, and now returns false.

func `is`<Derived, Base>(_: Derived.Type, _: Base.Type) -> Bool {
  Derived.self is Base.Type
}

protocol Protocol { }
enum Concrete: Protocol { }

`is`(Concrete.self, Protocol.self)

Which is to say, the result now matches…

Concrete.self is Protocol.Protocol // false

…instead of

Concrete.self is Protocol.Type // true

There is no way to get the old result.

I agree with adding some keyword for existentials!!

I wonder why there is no section that explains its relationship to some, which is obviously relevant keyword. IIRC, some keyword was intended to make contrast with this any. I want to read more detailed explanation of the relationship.

1 Like

That's a step in the right direction, but a really tiny one. I see a bigger opportunity here.

There is a fundamental connection between generic types and existentials. Existential type, is just a value of generic type boxed together with its type arguments.

I think making this clear in the language will help people to grasp the difference between existentials and generics.

But, currently there is a major gap in functionality of the generics and existentials. Only generic types of the form T: P can be expressed as existentials, while there are many generic types that cannot be expressed as existentials. Consider the following example:

protocol P {
    mutating func increment()
}

func bump<T: P>(getter: () -> T, setter: (T) -> Void) {
    var value = getter()
    value.increment()
    setter(value)
}

typealias Lens<T: P> = (getter: () -> T, setter: (T) -> Void)
let lenses: [any Lens] = [ ... ] // Not possible :(

To make this happen we need:
a) a syntax that indicates that existential is a boxed generic type;
b) a method for opening the box:

/// example of (a) - array of boxed generic tuples
func bumpAll(_ lenses: [any<T: P> (getter: () -> T, setter: (T) -> Void)]) {
    for boxedLens in lenses {
        // example of (b) - opening an existential
        let<T> lens = boxedLens
        print("Bumping a value of type \(T.self)")
        dump(getter: lens.getter, setter: lens.setter)
    }
}

With generalised existentials, any P is a shortcut for a canonical type any<T> T where T: P.

Two currently confusing spellings P.Type and P.Protocol can be written any<T> T.Type where T: P and (any P).Type respectively.

Any becomes a typealias for any<T> T. Note that this is an existential without a protocol.

Would this proposal allow reusing type names between protocols and concrete types as in Objective-C?

protocol SomeType {}
struct SomeType: SomeType {}

I don't think we can replace both P.Type and P.Protocol with (any P).Type, because there is fundamental different between the two - P.Type allows to call static protocol methods, while P.Protocol does not.

protocol P {
     static func generate() -> Self
}

extension Int: P {
    static func generate() -> Int { 42 }
}

let t1: (any P).Type = (any P).self
t1.generate() // error

let t2: P.Type = Int.self
t2.generate() // ok

So, (any P).Type is good as replacement for P.Protocol, but P.Type needs to stay, unless we adopt generalised syntax any<T> T.Type where T: P. But even with generalised syntax, IMO, that's a common case that deserves a convenience spelling, so I guess P.Type is here to stay.

When any keyword becomes mandatory P.self is no longer a valid syntax, only (any P).self. I think already with optional any keyword, P.self should produce a warning with a fix-it to turn it into (any P).self.

It is compelling to add subtyping relationship between P.Type and (any P).Type - but there are gotchas with self-conforming existentials - see Why there is no subtyping relation between existential meta types?.

I think, it is possible to avoid self-conforming existentials using universal hashability and existential subtyping/convertibility as a constraint.

For example, we would not need self-conformance for Error if everywhere conformance to Error is replaced with convertible to (any Error) constraint.

protocol MyError: Error {
    var myStatusCode: Int { get }
}
enum FooError: MyError { ... }
enum BarError: MyError { ... }
enum OldResult<Success, Failure> where Failure : Error { ... }
enum NewResult<Success, Failure> where Failure : (any Error) { ... }

OldResult<Void, any Error> // only works with self-conformance
OldResult<Void, any MyError> // does not work even with self-conformance
OldResult<Void, FooError> // works

NewResult<Void, any Error> // works
NewResult<Void, any MyError> // works
NewResult<Void, FooError> // works

My understanding is that (any P).Type only replaces P.Type, not P.Protocol. The latter would remain, and continue to be the type of P.self. Presumably, since P in P.self does not refer to the existential any P type, P.self would also continue to be valid.

We would also gain the (any P).self spelling for accessing the existential type instance (of type, under this proposal, (any P).Type). I don't know if there's an existing spelling for this value. :thinking:

@hborla, does this sound correct to you?

2 Likes

This is the first time I'm seeing someone bring up some P. This proposal confuses me a bit, and I was scratching my head wondering what is the difference between some P and any P. Can someone help me understand this?

some ("opaque types") was introduced in Swift 5.1 and is used extensively in SwiftUI: Opaque Types — The Swift Programming Language (Swift 5.7)

I understand and have extensively used opaque types. I'm just finding it difficult to understand the difference between opaque types and existentials.