[Pitch] Introduce existential `any`

+1 for this proposal, have been hoping for a long time for this to happen. Source breaking at Swift 6 sounds good.

Also, in future directions the Extending existential types looks very intuitive and useful.

Whatever happens here, the meaning of .self of a protocol should not change again, and today's .Protocol and .Type must be accessible via the .selfs of two different spellings.

For example, this function call returned true, early last year, and now returns false.

func `is`<Derived, Base>(_: Derived.Type, _: Base.Type) -> Bool {
  Derived.self is Base.Type
}

protocol Protocol { }
enum Concrete: Protocol { }

`is`(Concrete.self, Protocol.self)

Which is to say, the result now matches…

Concrete.self is Protocol.Protocol // false

…instead of

Concrete.self is Protocol.Type // true

There is no way to get the old result.

I agree with adding some keyword for existentials!!

I wonder why there is no section that explains its relationship to some, which is obviously relevant keyword. IIRC, some keyword was intended to make contrast with this any. I want to read more detailed explanation of the relationship.

1 Like

That's a step in the right direction, but a really tiny one. I see a bigger opportunity here.

There is a fundamental connection between generic types and existentials. Existential type, is just a value of generic type boxed together with its type arguments.

I think making this clear in the language will help people to grasp the difference between existentials and generics.

But, currently there is a major gap in functionality of the generics and existentials. Only generic types of the form T: P can be expressed as existentials, while there are many generic types that cannot be expressed as existentials. Consider the following example:

protocol P {
    mutating func increment()
}

func bump<T: P>(getter: () -> T, setter: (T) -> Void) {
    var value = getter()
    value.increment()
    setter(value)
}

typealias Lens<T: P> = (getter: () -> T, setter: (T) -> Void)
let lenses: [any Lens] = [ ... ] // Not possible :(

To make this happen we need:
a) a syntax that indicates that existential is a boxed generic type;
b) a method for opening the box:

/// example of (a) - array of boxed generic tuples
func bumpAll(_ lenses: [any<T: P> (getter: () -> T, setter: (T) -> Void)]) {
    for boxedLens in lenses {
        // example of (b) - opening an existential
        let<T> lens = boxedLens
        print("Bumping a value of type \(T.self)")
        dump(getter: lens.getter, setter: lens.setter)
    }
}

With generalised existentials, any P is a shortcut for a canonical type any<T> T where T: P.

Two currently confusing spellings P.Type and P.Protocol can be written any<T> T.Type where T: P and (any P).Type respectively.

Any becomes a typealias for any<T> T. Note that this is an existential without a protocol.

Would this proposal allow reusing type names between protocols and concrete types as in Objective-C?

protocol SomeType {}
struct SomeType: SomeType {}

I don't think we can replace both P.Type and P.Protocol with (any P).Type, because there is fundamental different between the two - P.Type allows to call static protocol methods, while P.Protocol does not.

protocol P {
     static func generate() -> Self
}

extension Int: P {
    static func generate() -> Int { 42 }
}

let t1: (any P).Type = (any P).self
t1.generate() // error

let t2: P.Type = Int.self
t2.generate() // ok

So, (any P).Type is good as replacement for P.Protocol, but P.Type needs to stay, unless we adopt generalised syntax any<T> T.Type where T: P. But even with generalised syntax, IMO, that's a common case that deserves a convenience spelling, so I guess P.Type is here to stay.

When any keyword becomes mandatory P.self is no longer a valid syntax, only (any P).self. I think already with optional any keyword, P.self should produce a warning with a fix-it to turn it into (any P).self.

It is compelling to add subtyping relationship between P.Type and (any P).Type - but there are gotchas with self-conforming existentials - see Why there is no subtyping relation between existential meta types?.

I think, it is possible to avoid self-conforming existentials using universal hashability and existential subtyping/convertibility as a constraint.

For example, we would not need self-conformance for Error if everywhere conformance to Error is replaced with convertible to (any Error) constraint.

protocol MyError: Error {
    var myStatusCode: Int { get }
}
enum FooError: MyError { ... }
enum BarError: MyError { ... }
enum OldResult<Success, Failure> where Failure : Error { ... }
enum NewResult<Success, Failure> where Failure : (any Error) { ... }

OldResult<Void, any Error> // only works with self-conformance
OldResult<Void, any MyError> // does not work even with self-conformance
OldResult<Void, FooError> // works

NewResult<Void, any Error> // works
NewResult<Void, any MyError> // works
NewResult<Void, FooError> // works

My understanding is that (any P).Type only replaces P.Type, not P.Protocol. The latter would remain, and continue to be the type of P.self. Presumably, since P in P.self does not refer to the existential any P type, P.self would also continue to be valid.

We would also gain the (any P).self spelling for accessing the existential type instance (of type, under this proposal, (any P).Type). I don't know if there's an existing spelling for this value. :thinking:

@hborla, does this sound correct to you?

2 Likes

This is the first time I'm seeing someone bring up some P. This proposal confuses me a bit, and I was scratching my head wondering what is the difference between some P and any P. Can someone help me understand this?

some ("opaque types") was introduced in Swift 5.1 and is used extensively in SwiftUI: Opaque Types — The Swift Programming Language (Swift 5.5)

I understand and have extensively used opaque types. I'm just finding it difficult to understand the difference between opaque types and existentials.

I half-jokingly proposed once that instead of "some" we should have "a specific" and instead of "any" we should have "an arbitrary". Here's an example that might clarify:

protocol Example {}

extension String: Example {}
extension Int: Example {}

/*
error: repl.swift:6:6: error: function declares an opaque return type, but the return statements in its body do not have matching underlying types
func illegal(_ x: Int) -> some Example {
     ^

repl.swift:7:22: note: return statement has underlying type 'Int'
  if x == 5 { return x }
                     ^

repl.swift:8:10: note: return statement has underlying type 'String'
  return "X wasn't 5"
*/
func illegal(_ x: Int) -> some Example {
  if x == 5 { return x }
  return "X wasn't 5"
}

// compiles successfully
func legal(_ x: Int) -> some Example {
  return x
}

// compiles successfully
func legal2(_ x: Int) -> any Example { // in today's Swift this would be func legal2(_ x: Int) -> Example
  if x == 5 { return x }
  return "X wasn't 5"
}
4 Likes

An opaque type instance has a fixed and "obscured" type, its underlying type. It cannot change dynamically, and it conforms to the protocol used in the type signature, i.e. all the protocol requirements are available.

An existential type instance has a type which is a box capable of containing any type conforming to the protocol used in the signature. It can change dynamically (e.g. if x: any Equatable, you can assign an Int or a String, and change the underlying type of x freely). An existential type, depending on the circumstances, may not give you all the protocol requirements (e.g. if x: any Equatable and y: any Equatable, you cannot do x == y, even though the == is included in the Equatable protocol, because == requires the same underlying type for both the left and the right hand sides and x and y may have different types under the hood).

6 Likes

This is helpful. So, I can think of an opaque type as a more constrained version of an existential type. Right?

The real discriminant between an existential and an opaque type is not the restriction on the former to forbid changes on its underlying type (after all, having a let existential instance would be the same as having an opaque type instance). It's the fact that opaque types are known by the compiler at compile time (and thus, they can be optimized by means of inlining code), while existentials are dynamic and known only at run time. @David_Smith's example is great to better grasp that difference:

func illegal(_ x: Int) -> some Example {
  if x == 5 { return x }
  return "X wasn't 5"
}

This function is illegal because it can return either an Int or a String dynamically. Maybe this one is even better:

func illegal() -> some Example {
  if .random() {
    return 3
  } else {
    return "three"
  }
}

There's no way for the compiler to know in advance what type will be returned. It's something that can only be known when you run the script. Opaque types lack this kind of dynamisms.

1 Like

For teaching purposes we may phrase them like this:

"some P means some specific type that conforms to P, while any P means an instance of any arbitrary type that conforms to P." We can further clarify that any P is a box/container with a special type (called Existential in CS lingo) that can hold any instance of any type as long as the type of the instance conforms to P.

2 Likes

I’ve always been unsatisfied with defining opaque types in terms of what they can’t do relative to existential types.

I like to think of it like this: for every concrete type T, there exists an existential type any T. Values of any T are wrappers around values of T. Values of type T are convertible to type any T, as are values of every type S that is a subtype of T.

However, there is no such thing as a type named some T. Rather, some T is a syntax that is allowed in function return position. The actual return type of the function is a unique alias for another type. Given a function declaration func f() -> some T, the type of f is in fact Void -> f’s opaque return type, which is either T or a subtype of T. Given another function declaration func g() -> some T, g’s return type is entirely distinct from f’s return type, even though both are known to alias T or some subtype of T, and even if f and g return a value of the exact same concrete type.

This distinction is why I have been keen on spelling existentials as Any<T> instead of any T. While @Ben_Cohen is right that you can’t write a generic type Any<T>, it is conceptually relatable to any sort of user-authored wrapper type. Whereas some T doesn’t name a type at all. Plus, there are plenty of things you can write that look like native Swift but can’t be written in native Swift, such as the “function” type(of:).

2 Likes

That's right. I am not proposing to change the .Protocol syntax.

I suppose it depends on how you're thinking about it, but my mental model is that the P in P.self does refer to the existential type any P. This is where metatypes are different from instances of existential types - you cannot actually create an instance of existential type that does not have an underlying concrete type. With metatypes, you can. That's why there's a difference between P.Type and P.Protocol - the former must have an underlying concrete type, and the latter does not. This is also why P.Protocol is not very useful - you don't actually have an underlying conformance, so you can't really do much with this type since there is no facility to add static methods to an existential type. So, I don't think there's a difference between "the metatype of the protocol" and "the metatype of the existential type itself" (but I'll admit that I find the terminology here very subtle and confusing). If you have the metatype of the existential type itself, i.e. (any P).self, you cannot call static protocol requirements because you don't actually have an underlying conformance with a witness. And because extension P { ... } really means "extend every type that conforms to P", you can't even call static methods that are implemented in protocol extensions. I assume that if/when we add extensions on existential types and can declare static methods there, you would be able to call those methods on an instance of (today's) P.Protocol.

Part of the reason why I feel it makes sense to have (any P).Type replace P.Type and not P.Protocol is because I think it would be confusing if this didn't work:

let x: (any P).Type = ConcreteP.self

This is totally inline with the subtype relationship of metatypes. ConcreteP is a subtype of any P, so why would this not work?

(That said, I'm definitely open to spelling P.Type as any P.Type instead if that is more clear, and I have a lingering feeling that this is the case. I haven't come to a conclusion yet, and I need to sit down and sort through my thoughts in the proposal.)

Personally, I think the subtlety of what it means to have a "metatype of the existential type itself" deserves a more obscure syntax. The "metatype of the existential type itself" is the only case where you don't have witnesses that you can call, and if/when we have extensions on existential types, I still anticipate the use case for these metatypes being much more rare than needing metatypes of concrete conforming types.

Please let me know if this made sense. I'm going to sit down later today and attempt to clarify all of this in the proposal :slightly_smiling_face:

Everywhere else in the language, the angle bracket notation signals that concrete type information is preserved in the type system by statically substituting the type parameters, and therefore those concrete types are known by the compiler. This isn't true for any P, so I think describing it as sugar for any<T> T where T: P is misleading.

2 Likes

Ah, yes, you're totally right. Agree that the terminology here is confusing. :sweat_smile:

So, then, under this proposal, would we replace P.self with (any P).self? Or, would we keep P.self as the spelling of the protocol metatype instance?

Without thinking too hard, I think there might be a case for maintaining the P.self spelling—otherwise we'd be perpetuating some of today's confusion in that (any P).self would not have type (any P).Type. Maybe this is another argument for spelling it as any P.Type. :thinking:

1 Like

I agree with this. It's not going to be obvious to many users that any creates a distinct type, while some does not.

protocol Food {}

// 'some Food' is not a distinct type.
// It is just an anonymization of some known type
var food1: some Food { Pizza() }

// 'any Food' is a distinct type, despite
// syntactic similarity with 'some Food'
var food2: any Food  { Pizza() }

// This is clearly a distinct type
var food3: Any<Food> { Pizza() }

I think that we're going to have a lot of users confused about a misleading similarity between some and any. I think that would be a lot less likely if we used Any<T>.

Anyway, we've had discussions about this before, and overall I think either spelling of existential is better than leaving them bare like we have right now. But I did want to call out the discussion here.

3 Likes

I disagree. Most user-authored wrapper types preserve the wrapped type in the type system. The angle-bracket syntax Any<T> makes it seem like this is true here, but it is not. The concrete T is not preserved, it's erased. I am strongly opposed to an existential type syntax that appears to preserve the underlying type. I agree that the some keyword is confusing, but I don't think spelling existential types as Any<T> solves that problem.

3 Likes