[Pitch] Introduce existential `any`

Hi, very excited about this changes, thanks for summarizing all the points.
Just one question: how self-conforming protocols will be expressed?

let error: Error = NetworkError() // or `let error: any Error = ...` ?

generic(error)

func generic<SomeError: Error>(error: SomeError) {
  let errorCopy: SomeError // or `let errorCopy: any SomeError` if argument of Error existential type is passed?
}

Huge +1 on this, I'm so glad it's getting a pitch.

+1 on the observation that the current syntax causes active harm. I wager a majority of working Swift users never properly learned the concept of existential due to confusion caused by it.

8 Likes

Isn’t Any considered an existential here? If so, then any Any seems more consistent to me. Yes, it’s not great, but we can introduce Value and deprecate Any in Swift 6 for a less disruptive source break.

This is a great proposal and I welcome the introduction of the any keyword. There is one thing that I'd like to push for reconsideration. Why is it a requirement to disallow any S where S a value type or a class, etc.?

Can't any not only represent existentials but also sub-types? While any S won't have any effect as we don't have sub typing for value types now, Never as a bottom type will/could fall under this umbrella in the future. any C for a class could already make sense and it would be equivalent to the today's C.

class C {} 
class D: C {}

// today
let c: C = D() // okay
// after the proposal
let anyC: any C = D() // theoretically the same as above

I personally think that some C would equally make sense where one could express that there is an opaque class involved, especially when we discuss the possibility of the generic footprint reduction via the some keyword in a context of generic type parameter T constrained by a class type.

<T: C>(_: T) ---- becomes ----> (_: some C)

If there is some C, then there should be any C as well.


TLDR; can't we make those errors as warnings instead?

Of course we could relax this rule in the future, but I'd like to know if it would be okay to do it with this proposal already.

Thank you in advance.

1 Like

I'm also going to bump the ideas regarding the issues with the current metatype spelling. There are meta types which are static and the result of Type.self and there are meta type existentials. The current state of the proposal pitch does not cover this whole area, which can be improved with the introduction of the any keyword.


Without derailing the current topic from the any keyword, in the UX related thread we also briefly discussed the idea of another keyword named meta to improve metatypes. Here's a short explanation of the types it would cover compared against the any syntax:

typealias Meta<T> = meta T

P.Protocol == (any P).Type == Meta<any P> == Meta<P> == meta P
    P.Type ==   any P.Type == any Meta<P> == any meta P
              ~~~~~~~~~~~~

I'm not asking for an additional introduction of that keyword, I just mention it to further clarify what any keyword can already cover and why.

This also further emphasis why the any keyword should support not only existentials (e.g. protocol conformances) but sub-typing (e.g. inheritance) as well!

+1 for this very nice proposal.

I like Any->Value and AnyObject->Object transition, maybe it's time to make this change through 5.6 release.

Any protocol means any value by literal in Swift, now we have any P syntax to support this subtle conception, formalize Value and any Value would be much better.

IIRC, AnyObject naming problem been raised and discussed several times in forum, it should be Object by initial design. Now finally we have a chance to fix this problem in this proposal. any Object is the right design by nature, and furthermore I really hate protocol SomeP: AnyObject class bound protocol declaration. So weird to look and read~

any Value/Object change is the right thing to do, and it's the right time to do this right thing. Please fixing it!

7 Likes

Heh, perhaps the (any P).Type win was not as clear as I initially thought. Between this proposal, @jrose's suggestion that any P.Type and (any P).Type be the same, and @Joe_Groff's earlier suggestion that P.Protocol become (any P).Type, we have three different (and incompatible) ideas of how any syntax should apply to protocol metatypes. IMO, having P.Type become (any P).Type makes sense, since to me that syntax naturally reads as "the metatype of all types which conform to P", and I'm ambivalent-to-negative on having any P.Type mean the same thing. I think I big plus of (any P).Type is that it puts some visual distance and a bit of syntactic clarification between the former P.Type and P.Protocol. If we continued to allow P.Type as a syntactic form (even if it were required to prefix with any, we'd lose that benefit.

In the same vein, I'd be strongly against allowing both any P.Type and (any P).Type and having those mean different things. I suspect that would lead to even more confusion than the Type-Protocol issue we have today.


On another note, I'd like to discuss a bit more about the source break here. As I mentioned previously, this is going to be a big break if adopted as is. Hopefully it will be as simple as prepending any to all existing existential type names, but I do wonder whether there are ways that we could mitigate the amount of churn that will be required.

As came up in a previous thread, some uses of existentials are isomorphic to generics thanks to Swift not requiring monomorphization for generic functions. That is, if we remove the associated type from the proposal's example, there's not a fundamental issue with:

protocol P {
  func test()
}

func generic<ConcreteP: P>(p: ConcreteP) {
  p.test()
}

func useExistential(p: P) {
  generic(p: p)
}

In fact, we can already call generic from within useExistential today, with _openExistential:

func useExistential(p: P) {
  _openExistential(p) { concrete in
    generic(p: concrete)
  }
}

I suspect that we could come up with a reasonable list of non-problematic uses of existentials that maybe don't need to require any syntax. In the above situation, we could auto-open the existential so that the generic(p: p) call succeeds without any dance.

The proposal also calls out concerns that "the type of value stored can change dynamically" but this is only true insofar as the value can change at all—if the value of existential type is a let (as with non-inout function parameters), then we needn't worry about the concrete type changing dynamically.

I think we could possibly also get away with allowing bare existentials for @objc protocols.

Together with auto-opening of existentials, I do wonder if a rule like the following would get us most (all?) of the benefit while reducing the code churn required:

An existential type must be annotated with any if the protocol is not @objc and either of the following is true:

  • The protocol has Self or associatedtype requirements
  • The value of existential type is mutable

Since SE-0306 hasn't shipped with an official version of Swift yet (right?), that would allow us to only require updating the uses of mutable values of existential type (which are not @objc). This would likely include any Swift protocol delegates, since most of those will be weak vars, but it might carve out enough exceptions to make the source break more palatable.

Of course, I have a generally higher tolerance for source breaks, so if this problem is deemed big enough for the big, hard break in Swift 6, then I'm all for it. I'd rather end up in a world with fewer complex rules for type syntax, so a rule of "existential types are prefixed with any" is preferable to me in that regard. Just wanted to raise a couple of alternative paths forward in the event that the widest-reaching version of this proposal is unpalatable to some. :slight_smile:

I agree with you that those things are extremely close to each other, yet they mean two totally different things. Metatatypes have two kinds merged into them today, this is rather awkward because not only does it limit our way into expressing a specific type constraint, but it's also the fact on why type(of:) remains compiler magic instead of having a pure swift function signature.

  • any P.Type is the existential that accepts all meta types, which inherit from P.Type (e.g. T.Type from T.self where T: P which also means that T.Type: P.Type)

  • (any P).Type is a static metatype (not an existential itself) of an existential for P (e.g. the result of type(of: existentialP) or P.self)

A small side note: The "true" P.Type is not expressible in Swift. Let this sink in for a bit. ;)

2 Likes

Hm—I'm not sure I understand this distinction fully. In Swift today, I thought that the result of type(of: existentialP) would be a P.Type (i.e., the metatype of some type conforming to P) while P.self is of type P.Protocol.

Even if I can bring myself to grasp this distinction fully, I don't think we should expose it to users as (any P).Type vs. any P.Type. We should come up with a better way of spelling those two (?) types (perhaps with the suggested meta keyword). Regardless, I don't think it's a problem that this proposal really needs to solve, as long as we're not boxing ourselves out of a potential future.

1 Like

One more thing, about 5 years ago we already tried to move this topic forward, but we (the original proposal authors) failed and withdrew our proposal. Later on @beccadax helped us to clear things a bit, but the proposal never seen any daylight. I still have it dusting in my outdated swift-evolution repository.

If anyone is interested in reading it and get a feeling on the issues around meta types, here's the link to it: Refactor Metatypes

The introduction of any keyword is probably our (second to) last chance to fix this in the language.

1 Like

What exactly do you mean by this? Is this proposal consuming design space that would be required for a (separate) metatype-focused proposal, or is your concern that metatype issues won't on their own justify a source break and so would have to 'ride along' with the any proposal if the fixes require their own source breaks?

Well, as outlined before we have to be careful on how we involve any in the context out meta types. The proposed (any P).Type does already break the ideas from the UX discussion, as any P.Type was meant to become the "existential of the protocol P metatype".

If on the other hand we'd design it in an additive manner, then I'm fine deferring this topic. I have a fear that if we don't solve it with any at all, or if we go the with the pitched direction in the context of meta types, the chances on when we can resolve the issue with meta types might shrink down to zero.

Personally I would love to see a meta keyword and finally free up the .Type space for custom nested types called Type, however this is such a massive source break. :(

One future direction from our old proposal was this:

func subtype<T>(of type: Type<T>, named: String) -> AnyType<T>? { ... }

// with the idea of a `meta` keyword this would be
func subtype<T>(of type: meta T, named: String) -> (any meta T)? { ... }

// with the spelling from Joe Groff this would become
func subtype<T>(of type: (any T).Type, named: String) -> (any P.Type)? { ... }

I wonder if it could be as simple as allotting an entire major version (Swift 6) to be a transitional period where bare P-as-existential leads to only a warning with a fix-it.

There may be value in studying in detail how Rust made the transition also (I have not yet had opportunity to read the linked document that is undoubtedly illuminating).

3 Likes

Sorry, I probably should have written this in alternatives considered - this is a really enticing idea, and I tried really hard to make it work, but it leads to a pretty confusing programming model because you need to have a really deep understanding to know when you can write P and when you need to write any P.

This is true for function parameters, but it is not for stored properties. A stored let-constant with existential type cannot change dynamically for a single instance of that type, but the underlying type can certainly change across instantiations of that type, which means the type must be parameterized on that underlying type in order to preserve concrete type information, which is a much bigger semantic difference than using an existential type. These semantic changes also necessitate an ABI change.

That said, I really like the idea when I think about it with a different framing. What you're suggesting is effectively to be able to use a plain protocol name to mean an implicit type parameter conforming to the protocol. I don't think we should encourage people to think of this thing as a "special" kind of existential - people should think of it as a type parameter. This is what lead me to the future direction to allow this. It's effectively what we've been talking about doing with some, but not needed the some keyword.

I agree that a period of time when this is a warning with a fix-it would help with the transition.

5 Likes

Yeah, this is really only appealing to me if the "require any for all existentials" is considered too large of a break.

Ah, good point.

Cool, I like that a lot. If we can go the route of completely deprecating bare existential syntax, then reintroducing the bare protocol name as shorthand for an implicit, anonymous type parameter in Swift X1 would be great.

1: pronounced "Swift ten" :wink:

4 Likes

I've actually updated my previous message as I had things wrong interpreted by the pitch. I'm against the meaning of (any P).Type as proposed. This goes against:

And for more context on this issue here are a few examples:

protocol P {}
struct S: P {}

type(of: P.self)      // P.Protocol.Type

let p: P = S()
type(of: p)           // S.Type
type(of: p) is S.Type // true
type(of: p) == S.self // true

P.self is P.Type      // false
P.self is P.Protocol  // true

func whatAmI<T>(_ t: T.Type) {
  print(type(of: t))
}

whatAmI(P.self)              // P.Protocol
whatAmI(P.Type.self)         // P.Type.Protocol
whatAmI(P.Protocol.self)     // P.Protocol.Type
whatAmI(Any.Type.self)       // Any.Type.Protocol
whatAmI(AnyObject.Type.self) // AnyObject.Type.Protocol

// True `P.Type` is not accessible today.

Here are some sub-type relations between existential types and also normal and existential metatypes. Let's say T is a non-protocol type and P is a protocol.

// For non-protocol types those types are sub-types of themselves.
       any T :        any T
(any T).Type : (any T).Type
  any T.Type :   any T.Type

// In case of protocols this is a bit different.
       any P :        any Any
(any P).Type : (any Any).Type
  any P.Type :     any P.Type

I know this topic is mind bending, but it is what it is and we eventually have to talk about it.

2 Likes

I’m of the understanding that the pitched meaning for (any P).Type is exactly what @Joe_Groff describes in the text you quote (“the type of the existential itself”). Indeed I cannot contemplate that spelling meaning anything else.

So long as that is agreed, everything else about changing metatype syntax can be a separate proposal—I would be strongly opposed to tackling any of that in one go as it’s an extensive topic of its own, as you well know.

A different quote a bit further up better illustrates the difference:

This proposal suggests that P.Type would become (any P).Type. Joe's previous suggestion was that P.Type becomes any (P.Type).

1 Like

@xwu "existential metatype" is not the same as "the (meta)type of an existential". The second and third quotes emphasis this more specific.

protocol P {}

// Joe's suggestion: `(any P).Type` vs. this proposal `P.Protocol`
let _ = P.self // "static" metatype

// Joe's suggestion: `any P.Type` vs. this proposal `(any P).Type`
let _: P.Type = ...
       ^~~~~~ // "existential" metatype