[Pitch] Introduce existential `any`

Heh, perhaps the (any P).Type win was not as clear as I initially thought. Between this proposal, @jrose's suggestion that any P.Type and (any P).Type be the same, and @Joe_Groff's earlier suggestion that P.Protocol become (any P).Type, we have three different (and incompatible) ideas of how any syntax should apply to protocol metatypes. IMO, having P.Type become (any P).Type makes sense, since to me that syntax naturally reads as "the metatype of all types which conform to P", and I'm ambivalent-to-negative on having any P.Type mean the same thing. I think I big plus of (any P).Type is that it puts some visual distance and a bit of syntactic clarification between the former P.Type and P.Protocol. If we continued to allow P.Type as a syntactic form (even if it were required to prefix with any, we'd lose that benefit.

In the same vein, I'd be strongly against allowing both any P.Type and (any P).Type and having those mean different things. I suspect that would lead to even more confusion than the Type-Protocol issue we have today.


On another note, I'd like to discuss a bit more about the source break here. As I mentioned previously, this is going to be a big break if adopted as is. Hopefully it will be as simple as prepending any to all existing existential type names, but I do wonder whether there are ways that we could mitigate the amount of churn that will be required.

As came up in a previous thread, some uses of existentials are isomorphic to generics thanks to Swift not requiring monomorphization for generic functions. That is, if we remove the associated type from the proposal's example, there's not a fundamental issue with:

protocol P {
  func test()
}

func generic<ConcreteP: P>(p: ConcreteP) {
  p.test()
}

func useExistential(p: P) {
  generic(p: p)
}

In fact, we can already call generic from within useExistential today, with _openExistential:

func useExistential(p: P) {
  _openExistential(p) { concrete in
    generic(p: concrete)
  }
}

I suspect that we could come up with a reasonable list of non-problematic uses of existentials that maybe don't need to require any syntax. In the above situation, we could auto-open the existential so that the generic(p: p) call succeeds without any dance.

The proposal also calls out concerns that "the type of value stored can change dynamically" but this is only true insofar as the value can change at all—if the value of existential type is a let (as with non-inout function parameters), then we needn't worry about the concrete type changing dynamically.

I think we could possibly also get away with allowing bare existentials for @objc protocols.

Together with auto-opening of existentials, I do wonder if a rule like the following would get us most (all?) of the benefit while reducing the code churn required:

An existential type must be annotated with any if the protocol is not @objc and either of the following is true:

  • The protocol has Self or associatedtype requirements
  • The value of existential type is mutable

Since SE-0306 hasn't shipped with an official version of Swift yet (right?), that would allow us to only require updating the uses of mutable values of existential type (which are not @objc). This would likely include any Swift protocol delegates, since most of those will be weak vars, but it might carve out enough exceptions to make the source break more palatable.

Of course, I have a generally higher tolerance for source breaks, so if this problem is deemed big enough for the big, hard break in Swift 6, then I'm all for it. I'd rather end up in a world with fewer complex rules for type syntax, so a rule of "existential types are prefixed with any" is preferable to me in that regard. Just wanted to raise a couple of alternative paths forward in the event that the widest-reaching version of this proposal is unpalatable to some. :slight_smile:

I agree with you that those things are extremely close to each other, yet they mean two totally different things. Metatatypes have two kinds merged into them today, this is rather awkward because not only does it limit our way into expressing a specific type constraint, but it's also the fact on why type(of:) remains compiler magic instead of having a pure swift function signature.

  • any P.Type is the existential that accepts all meta types, which inherit from P.Type (e.g. T.Type from T.self where T: P which also means that T.Type: P.Type)

  • (any P).Type is a static metatype (not an existential itself) of an existential for P (e.g. the result of type(of: existentialP) or P.self)

A small side note: The "true" P.Type is not expressible in Swift. Let this sink in for a bit. ;)

2 Likes

Hm—I'm not sure I understand this distinction fully. In Swift today, I thought that the result of type(of: existentialP) would be a P.Type (i.e., the metatype of some type conforming to P) while P.self is of type P.Protocol.

Even if I can bring myself to grasp this distinction fully, I don't think we should expose it to users as (any P).Type vs. any P.Type. We should come up with a better way of spelling those two (?) types (perhaps with the suggested meta keyword). Regardless, I don't think it's a problem that this proposal really needs to solve, as long as we're not boxing ourselves out of a potential future.

1 Like

One more thing, about 5 years ago we already tried to move this topic forward, but we (the original proposal authors) failed and withdrew our proposal. Later on @beccadax helped us to clear things a bit, but the proposal never seen any daylight. I still have it dusting in my outdated swift-evolution repository.

If anyone is interested in reading it and get a feeling on the issues around meta types, here's the link to it: Refactor Metatypes

The introduction of any keyword is probably our (second to) last chance to fix this in the language.

1 Like

What exactly do you mean by this? Is this proposal consuming design space that would be required for a (separate) metatype-focused proposal, or is your concern that metatype issues won't on their own justify a source break and so would have to 'ride along' with the any proposal if the fixes require their own source breaks?

Well, as outlined before we have to be careful on how we involve any in the context out meta types. The proposed (any P).Type does already break the ideas from the UX discussion, as any P.Type was meant to become the "existential of the protocol P metatype".

If on the other hand we'd design it in an additive manner, then I'm fine deferring this topic. I have a fear that if we don't solve it with any at all, or if we go the with the pitched direction in the context of meta types, the chances on when we can resolve the issue with meta types might shrink down to zero.

Personally I would love to see a meta keyword and finally free up the .Type space for custom nested types called Type, however this is such a massive source break. :(

One future direction from our old proposal was this:

func subtype<T>(of type: Type<T>, named: String) -> AnyType<T>? { ... }

// with the idea of a `meta` keyword this would be
func subtype<T>(of type: meta T, named: String) -> (any meta T)? { ... }

// with the spelling from Joe Groff this would become
func subtype<T>(of type: (any T).Type, named: String) -> (any P.Type)? { ... }

I wonder if it could be as simple as allotting an entire major version (Swift 6) to be a transitional period where bare P-as-existential leads to only a warning with a fix-it.

There may be value in studying in detail how Rust made the transition also (I have not yet had opportunity to read the linked document that is undoubtedly illuminating).

3 Likes

Sorry, I probably should have written this in alternatives considered - this is a really enticing idea, and I tried really hard to make it work, but it leads to a pretty confusing programming model because you need to have a really deep understanding to know when you can write P and when you need to write any P.

This is true for function parameters, but it is not for stored properties. A stored let-constant with existential type cannot change dynamically for a single instance of that type, but the underlying type can certainly change across instantiations of that type, which means the type must be parameterized on that underlying type in order to preserve concrete type information, which is a much bigger semantic difference than using an existential type. These semantic changes also necessitate an ABI change.

That said, I really like the idea when I think about it with a different framing. What you're suggesting is effectively to be able to use a plain protocol name to mean an implicit type parameter conforming to the protocol. I don't think we should encourage people to think of this thing as a "special" kind of existential - people should think of it as a type parameter. This is what lead me to the future direction to allow this. It's effectively what we've been talking about doing with some, but not needed the some keyword.

I agree that a period of time when this is a warning with a fix-it would help with the transition.

5 Likes

Yeah, this is really only appealing to me if the "require any for all existentials" is considered too large of a break.

Ah, good point.

Cool, I like that a lot. If we can go the route of completely deprecating bare existential syntax, then reintroducing the bare protocol name as shorthand for an implicit, anonymous type parameter in Swift X1 would be great.

1: pronounced "Swift ten" :wink:

4 Likes

I've actually updated my previous message as I had things wrong interpreted by the pitch. I'm against the meaning of (any P).Type as proposed. This goes against:

And for more context on this issue here are a few examples:

protocol P {}
struct S: P {}

type(of: P.self)      // P.Protocol.Type

let p: P = S()
type(of: p)           // S.Type
type(of: p) is S.Type // true
type(of: p) == S.self // true

P.self is P.Type      // false
P.self is P.Protocol  // true

func whatAmI<T>(_ t: T.Type) {
  print(type(of: t))
}

whatAmI(P.self)              // P.Protocol
whatAmI(P.Type.self)         // P.Type.Protocol
whatAmI(P.Protocol.self)     // P.Protocol.Type
whatAmI(Any.Type.self)       // Any.Type.Protocol
whatAmI(AnyObject.Type.self) // AnyObject.Type.Protocol

// True `P.Type` is not accessible today.

Here are some sub-type relations between existential types and also normal and existential metatypes. Let's say T is a non-protocol type and P is a protocol.

// For non-protocol types those types are sub-types of themselves.
       any T :        any T
(any T).Type : (any T).Type
  any T.Type :   any T.Type

// In case of protocols this is a bit different.
       any P :        any Any
(any P).Type : (any Any).Type
  any P.Type :     any P.Type

I know this topic is mind bending, but it is what it is and we eventually have to talk about it.

2 Likes

I’m of the understanding that the pitched meaning for (any P).Type is exactly what @Joe_Groff describes in the text you quote (“the type of the existential itself”). Indeed I cannot contemplate that spelling meaning anything else.

So long as that is agreed, everything else about changing metatype syntax can be a separate proposal—I would be strongly opposed to tackling any of that in one go as it’s an extensive topic of its own, as you well know.

A different quote a bit further up better illustrates the difference:

This proposal suggests that P.Type would become (any P).Type. Joe's previous suggestion was that P.Type becomes any (P.Type).

1 Like

@xwu "existential metatype" is not the same as "the (meta)type of an existential". The second and third quotes emphasis this more specific.

protocol P {}

// Joe's suggestion: `(any P).Type` vs. this proposal `P.Protocol`
let _ = P.self // "static" metatype

// Joe's suggestion: `any P.Type` vs. this proposal `(any P).Type`
let _: P.Type = ...
       ^~~~~~ // "existential" metatype

I am now thoroughly confused, but my read is that Joe has made several distinct suggestions, and what you quote is one that could be “if we say that P.Type is a generic constraint…” I do not think we need to say anything about that necessarily as part of this proposal.

Here is the only scheme my tiny mind can understand; I had thought that this was what was pitched:

  1. (any P).Type is the (meta)type of any P, the existential type.
  2. P.Type is an existing spelling; we do nothing with it.
  3. any P.Type is not part of this proposal, because P.Type is not itself an existential type (per item 2).

This would not preclude or limit the design space for any other examination or overhaul of metatypes so long as we are agreed on item 1, since the rest is just the status quo.


protocol P {}

// Joe's suggestion: `(any P).Type` vs. this proposal `P.Protocol`
let _ = P.self // "static" metatype

// Joe's suggestion: `any P.Type` vs. this proposal `(any P).Type`
let _: P.Type = ...
       ^~~~~~ // "existential" metatype

Sorry, I do not have the faintest idea what any part of this example means.

1 Like

I don't want this to sound like a drama, the proposal authors have to clarify this on us. It could just be a wording issue in the proposal where they actually meant to say "(any P).Type is the (meta)type of the protocol existential" instead of the "metatype existential" which are two different things.

In fact the "existential metatype" is probably better spelled as any (any P).Type.

I'm fine with the former, but against the latter.

If we're deprecating P as a type (for the time being) as part of this proposal, IMO we should also deprecate P.Type. It doesn't make much sense to me to have a P.Type when, indeed, P is not a type!

3 Likes

I had suggested in the pitch for the "existential metatype", i.e. the type that can store any concrete metatype conforming to the protocol, be spelled as (any P).Type. It made sense to me because of the way subtyping works with metatypes, but I can understand why folks would want this to be spelled as any P.Type. I agree with @Jumhyn that (any P).Type and any P.Type should not mean two different things because the syntactic distinction is far too subtle. In any case, I do not think that this proposal should change the .Protocol syntax, aside from perhaps making it (any P).Protocol.

1 Like

I agree with @xwu that this is the only meaning that makes sense to me for the (any P).Type spelling.

One alternative spelling that has been mooted in the past is Any<P>. Can we spare a few words in the Alternatives Considered section as to why any P is superior? Any<P> has the benefit of more clearly connoting the “existential box”.

2 Likes

Well I remain in a strong disagreement. This proposal says that "existentials" should receive the any keyword, however it mixes up what an "existential metatype" is and does not add the promised any keyword to metatype existentials themselves.

As I already said, I understand that this area of the language and discussion is extremely confusing to most of the readers. I spent days figuring out these tiny details. So I apologize for the caused disruption and confusing to everyone. However, it's on the table and I speak up, because I care about this topic, as I already did 5 years ago.

That said I would like to pivot the direction of this proposal to this:

// `T` is a non-protocol type, and `P` is aprotocol
let _: T = ... 
let _: P = ...
let _: any P = ... // already proposed


let _: T.Type = ...
let _: T.Type = ... // okay as it request `T.self`
let _: any T.Type = ... // explicitly requests the "existential metatype for T"

let _: P.Type = ...
let _: any (any P).Type = ... // the existential metatype of a protocol existential
1 Like