SE-0335: Introduce existential `any`

[Stan_Smida]

I understand that an implementation detail helps where an abstract construct is unclear. But IMO the notation you are proposing brings far more confusion than explanation what existential type is.

[Moximillian (Mox)]

+1 on the proposal as-is. I have followed the discussions on this topic on swift forum for many years and very happy see this mature into proposal phase.

Personally I’d prefer transition to be done in a clear way, i.e. whenever the major 6.0 version comes around that the missing ”any” in code becomes error, not warning. It would not be good for this kind of transition to drag for too long and over too many versions.

I think the alternative with angle brackets would be more confusing (generics etc) and very much prefer that any keyword is used as proposed.

As I mentioned in the pitch thread, using any will be a nice benefit when googling for existentials, e.g. in stackoverflow.

[ksluder (Kyle Sluder)]

@Stan_Smida

the notation you are proposing brings far more confusion than explanation what existential type is.

I thoroughly disagree with this. The very term “existential” is the kind of thing that contributes to Swift’s (undeserved) reputation as a language that prioritizes type theory over pragmatism.

Using a nominal type like Any is, to my mind, a fantastic educational coup. It accomplishes two goals at once: it replaces the academic term “existential” with the accessible term “Any”, and it extends an existing language concept in a way that helps the programmer understand the nature of existentials.

Whereas any P actively misleads clients into likening existentials to opaque types, Any<P> looks like a wrapper type. Its very spelling illustrates by analogy the most important aspect of existential containers.

[gregtitus]

@ksluder

Whereas any P actively misleads clients into likening existentials to opaque types, Any<P> looks like a wrapper type. Its very spelling illustrates by analogy the most important aspect of existential containers.

Spelling it Any<P> gives you insight into the wrapper-like-ness of the implementation, but I disagree that that's the most important aspect. The most common educational problem with existential P is "why doesn't existential P conform to P?" and if you spell it Any<P> then the first thing any new Swift user is going to try is extension Any<P> : P to add conformance, because not only are you analogizing to "wrapperness" with that spelling, you are analogizing to generics, and so you are giving just as many bad associations and ideas as good ones, IMO.

[hborla (Holly Borla)]

@bjhomer

I view this symmetry between some P and any P as a source of confusion , not clarity. While it's true that both some and any are keywords that can be applied to protocol constraints, that similarity hides an underlying dissimilarity

I personally don't think it's an issue that these two things behave differently -- they are, in fact, different keywords. The symmetry is most important for helping people understand the syntax, and the syntax for some and any is nearly identical. That's a really important benefit of the design in this proposal. It's similar to how class and struct have syntax symmetry while having different semantics.

Making the syntax too different feels to me like it's optimizing for folks who are seeing these keywords for the first time. There will, of course, still be a learning curve when starting to use a new feature or keyword, and the fact that this proposal introduces a keyword for existential types will greatly help us improve the documentation, error messages, and other explanations surfaced through tools to help programmers grasp these concepts.

[hborla (Holly Borla)]

@Srdan_Rasic

One question. Currently one can’t have nested types within protocols. My uneducated guess is that that has something to do with protocols not being types themselves, but constraints

Yes, my understanding is that nesting types within protocols is tricky because the nested type could "capture" Self or associated types, which would make that nested type distinct types across concrete conformances.

@Srdan_Rasic

Would this feature allow or open a future direction to allow nesting types within existential extensions?

I believe this would be possible.

[bjhomer]

@gregtitus

the first thing any new Swift user is going to try is extension Any<P> : P to add conformance

Why is this a problem? Is it any less of a problem if a user tries to do this?

extension any P: P {
  // ...
}

[gregtitus]

It is a problem because you have to explain why Any<P> is not generic like any other Foo<P>. You are breaking people's mental models of generics because this is only a partial analogy. any P doesn't look generic, so people do not bring their pre-existing ideas of generics to the table.

[ksluder (Kyle Sluder)]

@gregtitus

Spelling it Any<P> gives you insight into the wrapper-like-ness of the implementation, but I disagree that that's the most important aspect.

This is in fact a major motivation cited by the proposal:

Existential types are also significantly more expensive than using concrete types. Because they can store any value whose type conforms to the protocol, and the type of value stored can change dynamically, existential types require dynamic memory unless the value is small enough to fit within an inline 3-word buffer. In addition to heap allocation and reference counting, code using existential types incurs pointer indirection and dynamic method dispatch that cannot be optimized away.

@gregtitus

The most common educational problem with existential P is "why doesn't existential P conform to P?" and if you spell it Any<P> then the first thing any new Swift user is going to try is extension Any<P> : P to add conformance, because not only are you analogizing to "wrapperness" with that spelling, you are analogizing to generics, and so you are giving just as many bad associations and ideas as good ones, IMO.

Self-conformance is the first thing listed under Future Directions. We want people to do this! If we had this functionality, Error wouldn’t need to have been granted special self-conforming existential capabilities.

[ksluder (Kyle Sluder)]

@hborla

I personally don't think it's an issue that these two things behave differently -- they are, in fact, different keywords. The symmetry is most important for helping people understand the syntax, and the syntax for some and any is nearly identical.

This is a circular argument. The syntax is chosen by the proposal author. You have chosen a familiar syntax because you want the syntax to be familiar. That doesn’t help elucidate the meaning of any or its distinction from some.

@hborla

That's a really important benefit of the design in this proposal. It's similar to how class and struct have syntax symmetry while having different semantics.

class and struct both declare new nominal types, whereas any P is a type but some P is not. some P is a placeholder syntax for a particular dependent type. Any two places you see any P, they refer to the same type. But no two places you see some P refer to the same type.

[hborla (Holly Borla)]

@ksluder

Using a nominal type like Any

Any is not a nominal type -- it's an empty protocol composition constraint -- nor does it behave like one. There are a number of fundamental restrictions on Any that don't apply to nominal types, and I agree with @gregtitus that the Any<P> syntax will further confuse the mental model for this reason. Another example is that you cannot create an instance of Any<P>().

@ksluder

Self-conformance is the first thing listed under Future Directions. We want people to do this! If we had this functionality, Error wouldn’t need to have been granted special self-conforming existential capabilities.

Even if we pursue that future direction, it's likely that there will be fundamental differences between how you implement a protocol conformance for an existential type versus a nominal type. It's not a straightforward feature, even though I do believe it's an important tool in some use cases.

[Stan_Smida]

I cannot help myself, I just see a certain difference between existential type and Any and want the syntax to reflect this. any P and Any<P> is same (great) sound but they intuitively lead towards different associations. any P leads to a type definition while Any<P> leads to generics and other misleading associations to special types like Any, or even Optional<Wrapped>.

[xwu (Xiaodi Wu)]

@ksluder

class and struct both declare new nominal types, whereas any P is a type but some P is not .

You’ve made this claim repeatedly, but both are types, and neither are nominal types. That the runtime storage of the underlying value is different is part and parcel of the distinction between an opaque type and an existential type, no more or less.

[gregtitus]

@ksluder

some P is a placeholder syntax for a particular dependent type. Any two places you see any P , they refer to the same type. But no two places you see some P refer to the same type.

In the implementation of the language this is currently true. And I certainly get how understanding the underlying implementation helps with using something. At the same time, if you break the similarity between some P and any P you also break the analogy with predicate logic quantification, which is a valid and useful analogy for understanding what is going on as well, and IMHO, is a better and more complete analogy than Any<P> analogizing to the wrapper implementation.

func f1() -> some P
func f2() -> any P

If I want to work with the return result of f1, there exists some P that I will be dealing with. ∃x: P(x). If I want to work with the return result of f2 then given any P, I must deal with that P. ∀x: P(x). These keywords come directly from the English descriptions of ∃ and ∀, and accurately describe my situation in dealing with these function results.

You completely lose that parallel and analogy if you spell any P differently.

[ksluder (Kyle Sluder)]

@hborla

Any is not a nominal type -- it's an empty protocol composition constraint -- nor does it behave like one.

You can command-click Any and go to its definition in the standard library. Currently it is a typealias for protocol<>. Like type(of:), I believe the best design for Any is a language identifier with special treatment. However, making Any a true nominal type is not a blocker for implementing the syntax.

Using an identifier like Any has several distinct advantages:

• It is much more understandable why Any<P> doesn’t conform to P. You can command-click Any in Xcode and see that its declaration does not contain a : P. A value of any P still looks like it should conform to P, and you still have to go back to TSPL, Stack Overflow, or someone’s blog to understand why it doesn’t.

• any P does not achieve the stated goal of discouraging its own use. func f(_: any P) looks like an attractive way to accept any subtype of P. But the client really should be writing func f<T: P>(_: T). Whereas func f(_: Any<P>) looks like there needs to be an explicit conversion to Any<P> at the call site. (And in fact, the compiler might do well to insist upon it.)

• Using any P in any sort of nested context requires parentheses. This can get awfully complex, especially when dealing with metatypes. Dozens of posts in the pitch thread dealt with this concept alone. Whereas the nesting behavior of Any<P> is extremely straightforward. Any<P.Type> can be explicitly disallowed.

• The extension syntax for nominal types is already well-known. extension any P : P is a new concept, akin to extension (Int, Int). Not a deal breaker by any means, but the feature already exists for nominal types.

[ksluder (Kyle Sluder)]

@gregtitus

predicate logic quantification

See my point above about Swift’s reputation for elevating type theory over pragmatism. The duality of some P and any P is of no consequence to programmers writing code in Swift. It is a mathematical construct that helps the compiler implementor define and implement the feature. But for the same reason Optional doesn’t require understanding monads, any P should not be defended on the basis of type quantification. It puts up an educational barrier where none need exist.

[ksluder (Kyle Sluder)]

@xwu

You’ve made this claim repeatedly, but both are types

If you want to bring higher-kinded types into the discussion, then you can consider some P a type. But in the world of Swift, some P is not a type. Just look at the SIL for any code involving opaque types. The actual type is opaque return type of «function».

[gregtitus]

@ksluder

See my point above about Swift’s reputation for elevating type theory over pragmatism. The duality of some P and any P is of no consequence to programmers writing code in Swift.

Honestly first time I can remember being on the side of beauty over pragmatism.

I think understanding the idea is more important for Swift programmers than understanding the implementation. Especially when the implementation changes some day and we get what has been called "reverse generics" and thus more useful ways to take advantage of some P and the ideas will be more important than ever. The right abstractions are even more important and educational than the right implementations. But clearly, we'll have to agree to disagree.

[Moximillian (Mox)]

@ksluder

@gregtitus

predicate logic quantification

See my point above about Swift’s reputation for elevating type theory over pragmatism. The duality of some P and any P is of no consequence to programmers writing code in Swift. It is a mathematical construct that helps the compiler implementor define and implement the feature. But for the same reason Optional doesn’t require understanding monads, any P should not be defended on the basis of type quantification. It puts up an educational barrier where none need exist.

The place where any and some are similar is that you can/will be able to use where clauses with them and in those situataions it’s very nice that also the keywords are similar to each other.

[Paul_Cantrell]

@patrickgoley

For these reasons I would ask for alternative spellings to be considered, specifically something like dynamic P.

-1 on this vs the syntax as proposed, for two reasons:

1. This is “implementer viewpoint” terminology: it emphasizes the dynamic dispatch, but not the type system reason behind it. It does not elucidate some motivating theoretical principle, nor does it creates a casual heuristic with which language users can reason usefully about their own code without having to understand either language theory or language implementation. Such terminology would be at home in C++, but not in Swift.
2. It creates potential confusion with both dynamic member lookup and the existing dynamic keyword, whose meanings are related to each other (member lookup via runtime resolution of an arbitrary key) but fundamentally different from existentials (dispatch via vtable). Even if the syntax collision with dynamic is resolvable, the conceptual collision is not.

@patrickgoley

existential P could also be considered but may not be preferred because A. “existential” is a rather esoteric word, not very user friendly and still doesn’t communicate the related costs and B. dynamic dispatch is a well understood concept from many other languages, and anyone optimizing for performance will see dynamic P as an immediate red flag.

Agreed that existential is a bit esoteric to surface in the syntax.

I’d add that surfacing performance costs is clearly a non-goal of Swift syntax, not just in this proposal but across the language. That may be to the language’s detriment — it is full of surprising performance cliffs! — but it is a language that clearly, consistently favors a feeling of pleasant high-level abstraction over constant awareness of the performance implications of abstraction choices. Swift is not Rust.

Having read the proposed alternatives above, I’ll reiterate my support for the any Doodad syntax. “Does this proposal fit well with the feel and direction of Swift?” Yes it does, and it is Swiftier than any of the alternatives proposed here.