[Pitch] Introduce existential `any`

Evolution Pitches
62

I half-jokingly proposed once that instead of "some" we should have "a specific" and instead of "any" we should have "an arbitrary". Here's an example that might clarify:

protocol Example {}

extension String: Example {}
extension Int: Example {}

/*
error: repl.swift:6:6: error: function declares an opaque return type, but the return statements in its body do not have matching underlying types
func illegal(_ x: Int) -> some Example {
     ^

repl.swift:7:22: note: return statement has underlying type 'Int'
  if x == 5 { return x }
                     ^

repl.swift:8:10: note: return statement has underlying type 'String'
  return "X wasn't 5"
*/
func illegal(_ x: Int) -> some Example {
  if x == 5 { return x }
  return "X wasn't 5"
}

// compiles successfully
func legal(_ x: Int) -> some Example {
  return x
}

// compiles successfully
func legal2(_ x: Int) -> any Example { // in today's Swift this would be func legal2(_ x: Int) -> Example
  if x == 5 { return x }
  return "X wasn't 5"
}
4 Likes
63

An opaque type instance has a fixed and "obscured" type, its underlying type. It cannot change dynamically, and it conforms to the protocol used in the type signature, i.e. all the protocol requirements are available.

An existential type instance has a type which is a box capable of containing any type conforming to the protocol used in the signature. It can change dynamically (e.g. if x: any Equatable, you can assign an Int or a String, and change the underlying type of x freely). An existential type, depending on the circumstances, may not give you all the protocol requirements (e.g. if x: any Equatable and y: any Equatable, you cannot do x == y, even though the == is included in the Equatable protocol, because == requires the same underlying type for both the left and the right hand sides and x and y may have different types under the hood).

6 Likes
64

This is helpful. So, I can think of an opaque type as a more constrained version of an existential type. Right?

65

The real discriminant between an existential and an opaque type is not the restriction on the former to forbid changes on its underlying type (after all, having a let existential instance would be the same as having an opaque type instance). It's the fact that opaque types are known by the compiler at compile time (and thus, they can be optimized by means of inlining code), while existentials are dynamic and known only at run time. @David_Smith's example is great to better grasp that difference:

func illegal(_ x: Int) -> some Example {
  if x == 5 { return x }
  return "X wasn't 5"
}

This function is illegal because it can return either an Int or a String dynamically. Maybe this one is even better:

func illegal() -> some Example {
  if .random() {
    return 3
  } else {
    return "three"
  }
}

There's no way for the compiler to know in advance what type will be returned. It's something that can only be known when you run the script. Opaque types lack this kind of dynamisms.

1 Like
66

For teaching purposes we may phrase them like this:

"some P means some specific type that conforms to P, while any P means an instance of any arbitrary type that conforms to P." We can further clarify that any P is a box/container with a special type (called Existential in CS lingo) that can hold any instance of any type as long as the type of the instance conforms to P.

2 Likes
67

I’ve always been unsatisfied with defining opaque types in terms of what they can’t do relative to existential types.

I like to think of it like this: for every concrete type T, there exists an existential type any T. Values of any T are wrappers around values of T. Values of type T are convertible to type any T, as are values of every type S that is a subtype of T.

However, there is no such thing as a type named some T. Rather, some T is a syntax that is allowed in function return position. The actual return type of the function is a unique alias for another type. Given a function declaration func f() -> some T, the type of f is in fact Void -> f’s opaque return type, which is either T or a subtype of T. Given another function declaration func g() -> some T, g’s return type is entirely distinct from f’s return type, even though both are known to alias T or some subtype of T, and even if f and g return a value of the exact same concrete type.

This distinction is why I have been keen on spelling existentials as Any<T> instead of any T. While @Ben_Cohen is right that you can’t write a generic type Any<T>, it is conceptually relatable to any sort of user-authored wrapper type. Whereas some T doesn’t name a type at all. Plus, there are plenty of things you can write that look like native Swift but can’t be written in native Swift, such as the “function” type(of:).

2 Likes
68

That's right. I am not proposing to change the .Protocol syntax.

I suppose it depends on how you're thinking about it, but my mental model is that the P in P.self does refer to the existential type any P. This is where metatypes are different from instances of existential types - you cannot actually create an instance of existential type that does not have an underlying concrete type. With metatypes, you can. That's why there's a difference between P.Type and P.Protocol - the former must have an underlying concrete type, and the latter does not. This is also why P.Protocol is not very useful - you don't actually have an underlying conformance, so you can't really do much with this type since there is no facility to add static methods to an existential type. So, I don't think there's a difference between "the metatype of the protocol" and "the metatype of the existential type itself" (but I'll admit that I find the terminology here very subtle and confusing). If you have the metatype of the existential type itself, i.e. (any P).self, you cannot call static protocol requirements because you don't actually have an underlying conformance with a witness. And because extension P { ... } really means "extend every type that conforms to P", you can't even call static methods that are implemented in protocol extensions. I assume that if/when we add extensions on existential types and can declare static methods there, you would be able to call those methods on an instance of (today's) P.Protocol.

Part of the reason why I feel it makes sense to have (any P).Type replace P.Type and not P.Protocol is because I think it would be confusing if this didn't work:

let x: (any P).Type = ConcreteP.self

This is totally inline with the subtype relationship of metatypes. ConcreteP is a subtype of any P, so why would this not work?

(That said, I'm definitely open to spelling P.Type as any P.Type instead if that is more clear, and I have a lingering feeling that this is the case. I haven't come to a conclusion yet, and I need to sit down and sort through my thoughts in the proposal.)

Personally, I think the subtlety of what it means to have a "metatype of the existential type itself" deserves a more obscure syntax. The "metatype of the existential type itself" is the only case where you don't have witnesses that you can call, and if/when we have extensions on existential types, I still anticipate the use case for these metatypes being much more rare than needing metatypes of concrete conforming types.

Please let me know if this made sense. I'm going to sit down later today and attempt to clarify all of this in the proposal :slightly_smiling_face:

Everywhere else in the language, the angle bracket notation signals that concrete type information is preserved in the type system by statically substituting the type parameters, and therefore those concrete types are known by the compiler. This isn't true for any P, so I think describing it as sugar for any<T> T where T: P is misleading.

2 Likes
69

Ah, yes, you're totally right. Agree that the terminology here is confusing. :sweat_smile:

So, then, under this proposal, would we replace P.self with (any P).self? Or, would we keep P.self as the spelling of the protocol metatype instance?

Without thinking too hard, I think there might be a case for maintaining the P.self spelling—otherwise we'd be perpetuating some of today's confusion in that (any P).self would not have type (any P).Type. Maybe this is another argument for spelling it as any P.Type. :thinking:

1 Like
70

I agree with this. It's not going to be obvious to many users that any creates a distinct type, while some does not.

protocol Food {}

// 'some Food' is not a distinct type.
// It is just an anonymization of some known type
var food1: some Food { Pizza() }

// 'any Food' is a distinct type, despite
// syntactic similarity with 'some Food'
var food2: any Food  { Pizza() }

// This is clearly a distinct type
var food3: Any<Food> { Pizza() }

I think that we're going to have a lot of users confused about a misleading similarity between some and any. I think that would be a lot less likely if we used Any<T>.

Anyway, we've had discussions about this before, and overall I think either spelling of existential is better than leaving them bare like we have right now. But I did want to call out the discussion here.

3 Likes
71

I disagree. Most user-authored wrapper types preserve the wrapped type in the type system. The angle-bracket syntax Any<T> makes it seem like this is true here, but it is not. The concrete T is not preserved, it's erased. I am strongly opposed to an existential type syntax that appears to preserve the underlying type. I agree that the some keyword is confusing, but I don't think spelling existential types as Any<T> solves that problem.

4 Likes
72

I don’t think I quite understand what you mean by this. A major use case for generics and wrapper types is type erasure. The type descriptor gets captured in the closures stuffed into a custom type eraser; does that count as preserving the type in the type system?

73

For a type-erased wrapper, such as AnyHashable, the types in angle brackets are still preserved in the type system statically. For example, AnySequence<Element> erases the underlying sequence type, but it preserves the Element type.

1 Like
74

Doesn’t Any<P> do the same thing? It preserves P in the type system, but not any of the associated types.

1 Like
75

My argument against Any<P> is that such type would also have to conform to P, but that's not obvious from the notation. It would feel like some kind of a special generic type. That's why I think a special syntax, like any P, is justified.

// This is clearly a distinct type, 
// but what's the interface of `Any<Food>`? 
var food3: Any<Food> { Pizza() }
3 Likes
76

IMHO, the disadvantage of Any<P> is that now you can use P as a standalone type (at least in that particular position), which is what we would like to prevent. The role of the any P spelling is to remove usages of standalone P.

Another reason any is preferable is syntactic: some has specific rules about where it can be applied, that is before a protocol, our current Any, our current AnyObject or a composition of those, eventually including a class, which happen to be the same rules that would be imposed on any.
On the other hand, Any<T> looks like a normal generic type but with the special casing that only those (mostly invalid) types could be placed in its generic argument position. It is inconsistent and hints at the possibility that P may be usable alone. If (and this is a big IF) Swift were to get higher-kinded types, Any<T> would even clash with it.

5 Likes
77

Any<P> would not conform to P for the same reasons that P does not conform to P today. In fact, the Any<P> spelling makes it quite palatable to never conform to P.

78

Are you concerned about something like this?

let e: Any<BinaryInteger> = 42 as UInt // doesn’t preserve 'UInt'

I can see why you might react negatively to this, since your whole pitch is about removing the confusion that arises from the current situation:

let numbers: [BinaryInteger] = [42 as UInt] // actually declares an array of BinaryInteger *existentials* and wraps UInt(42) in one

If I understand correctly, your pitch would require rewriting the above as:

let numbers: [any BinaryInteger] = [42 as UInt]

The Any<T> alternative would still require rewriting the above, as:

let numbers: [Any<BinaryInteger>] = [42 as UInt]

This does give Any<> a power that no other generic type has, which is initialization with a type parameter that is never spelled out. I think that’s acceptable given the pre-existence of Any, which we are not proposing to remove and can be silently converted to from, well, any type.

79

I might have expressed myself inadequately. Any<P> would still have the interface of a type conforming to P and that's not something I would expect when I see Any<P>. It would feel magical, and I don't think that's good.

Edit: Actually, I think what I originally said is correct, the type would have to be conforming to P, as that’s what an existential type is.

1 Like
80

Again, I don’t think this is necessarily true. I can easily see a requirement to cast a value of Any<P> to P before working with the value.

81

Would P even be usable as a type after this? I don't think it would be possible to cast something to P after this, if I understood the proposal correctly.