[Pitch] Introduce existential `any`

I'm wary of proposals that rely on both tooling and the future to be most useful, but what you describe makes a lot of sense. I'm still not sure this proposal should be motivated solely by changing user behavior, as it's unlikely to do so on its own.

I don't see that as the sole motivation. It also improves the clarity of the language by clearly distinguishing existentials.

3 Likes

Note that in this case not making a syntax change means making a semantic change:

Given typealias A = P, before the proposal A.Type means P.Protocol, but after means P.Type. If this compiles successfully migration to a new Swift version will silently introduce a bug. I don’t think we should rely on every Swift user having a robust test suite to catch this.

Additionally, I’m concerned that keeping typealias for both may cause a new wave of confusion, but that is less of a problem compared to the previous point:


typealias A = any P
print(A.self) // Ok
typealias B = P
print(B.self) // Compilation error. Will users understand why?

struct Scope {
    typealias C = any P // Ok to be nested
    typealias D = P // Not ok. Will users understand why?
}

typealias X = Y

// Is X.Type a concrete meta type here?
// Before checking the declaration of X was enough.
// Now I also need to check what is Y.
print(X.Type.self)

Hm… Actually I’m not sure if nested protocol aliases should be disallowed the same way as protocol declarations. I don’t remember the reasoning for disallowing protocol declarations inside non-generic types.

1 Like

This is not correct. To express a protocol metatype, you need to use the .Protocol syntax. Using .Type on a protocol (even via typealias) means "the existential metatype", i.e. the metatype that can store any concrete metatype conforming to the protocol.

protocol Q {}
typealias A = Q
let a: A.Type = Q.self // error: Cannot convert value of type 'Q.Protocol' to specified type 'A.Type' (aka 'Q.Type')

This isn't correct (at least not for how I think it ought to work; there may be disagreement here).

The future language mode would distinguish between names of constraints (e.g. P) and existential types (e.g. any P). Given typealias A = P, writing A would behave like writing P, and therefore writing A.Type would behave like writing P.Type: it would be invalid in most contexts without the any keyword.

In the "soft deprecation" that would take place in current language modes, we would still distinguish these cases, but we would merely warn about A.Type without any, and it would continue to be interpreted as the existential metatype.

If you instead had typealias B = any P, then yes, B.Type would become the singleton metatype rather than the existential metatype. However, this is no longer just a matter of compiling code under a new language mode, it's actually an explicit adoption of the feature, and it's okay for that to change behavior.

The main trouble for migration that I can see is that some people would try introducing an alias like B just to avoid having to write any P all over the place, and then they'd get themselves in trouble with B.Type. But it's extremely likely that something in their code will stop compiling if they do this, because suddenly you've have a value/parameter of the singleton metatype type, which almost certainly cannot be satisfied or used in the ways you were currently using the existential metatype.

The other possible trouble is that someone today might have a member typealias that they use to satisfy an associated type requirement (and which therefore must be changed to include the any keyword) but also use to write a generic signature (and which would therefore become ill-formed after adding any). This doesn't seem very likely, though.

7 Likes

Theoretical question regarding existentials which circles on my mind. It would be cool if someone with compiler knowledge could answer it as I‘m thinking into splitting this conversation into its own discussion thread.

If Never would ever become the bottom type (a sub-type of possibly every other type), would we then need an existential to accept both Never and an instance of some struct value (e.g. func foo(_: any S) where S is some struct)?

I‘m trying to think ahead and understand why existentials are limited to protocols. In my mind I can see other interesting use cases, but all that involves existentials to support sub-typing relationships.

Ah, indeed. I misunderstood @John_McCall’s comment. The term “existential metatype” is confusing.
My interpretation was:
P.Protocol - metatype of existential aka existential metatype
P.Type - existential of metatype aka metatype existential aka any<T: P> T.Type

What is the correct naming then?

Let's convert the protocol P into the proposed protocol-as-a-type (aka. protocol existential) any P first.

protocol existential 
   v
(any P).Protocol - 'singleton metatype' for the 'protocol existential (any P)'
   ^
   v
(any P).Type - 'existential metatype' for the 'protocol existential (any P)'

I referred to the singleton metatype as static metatype in some of my previous messages.


For non-protocol types this becomes a little bit different, which is where all the confusion originates from:

struct S
  v
  S.Type - 'singleton metatype' for 'S'
  ^
  v
  S.Type - 'existential metatype' for 'S'

The 'singleton metatype' is the result from T.self.

Furthermore I think that both 'metatype existential' and 'existential metatype' refers to the same thing!

This will only solve the problem for functions, but if we have “bare protocols” as properties and they are passed in initializers, whole type will change.
In most cases - current behaviour ( any protocol ) is probably better and intended.

If we force generic fix-its it will cause even more harm and breaking changes.

For example:


protocol Runnable { }
protocol Loadable { }

struct A {
    let runnable: Runnable
}

func otherFunctionThatUsesA(a: A) {
    // Do something with A
}

If we wanted to follow new style, we would make A generic over Runnable, then we would also need to adjust otherFunctionThatUsesA.

struct A<R: Runnable> {
    let runnable: R
}

func otherFunctionThatUsesA(a: A<???>) {
    // Do something with A
}

If in future we also want to add new property Readable to A -- we would need to now make it generic over Runnable and Readable -- and by doing so, we would need to adjust all places where the type is used ( like otherFunctionThatUsesA ).

struct A<R: Runnable, L: Loadable> {
    let runnable: R
    let loadable: L
}

func otherFunctionThatUsesA(a: A<???>) {
    // Do something with A
}

Not only this makes refactoring and changing even worse, now the internals bleed from A and all users of A might need to be adjusted.

I don't see how this change ( to break existing code to make something harder and more confusing ) is justified by "some people accidentally use bare protocol.".

If we were in early Swift days ( Swift 1/2 ) -- maybe this change would be ok, after Swift 3 promise was not to make such big code-breaks, many companies/developers have invested into Swift code and this change will waste their resources with no real benefit.

In my current company, we have up to a hundred modules, it would be a shame and waste of our time if we have to "pay" for this breaking change.

Most apps/modules don't really need the benefits of what generics provide over bare-protocols, and if some do need it - owners of those code should be the ones that need to adjust the code instead of every Swift programmer.

This is going in the opposite direction of the Swift's goal "Our goals for Swift are ambitious: we want to make programming simple things easy, and difficult things possible.".

Maybe instead of breaking the code for everyone and making things harder, we can make benefits and disadvantages of bare-protocol/generics more clear and understandable.

Currently there are no such information/guide provided by Apple/Swift documentations, why not start from there?

2 Likes

I don't think anyone is suggesting forcing generics—especially in situations where the resulting generic solution is not equivalent to the existential solution. In your Runnable example, the original A's type identity is independent of the underlying Runnable type, but the generic A is actually a family of types whose identity depends on the specific runnable type that it is instantiated with.

So, that's to say that I would expect that the fix-it in the case of a stored property of type Runnable would continue to be "add 'any'" rather than "make this struct generic."

2 Likes

Not precisely, as I showed above. It is problematic.

typealias Type<T> = T.Type
Q.self as Type // Q.Protocol

func ƒ<T>(_: T.Type) -> T.Type { T.self }
ƒ(Q.self) // Q.Protocol
1 Like

I don't think fix-it will be smart enough to distinguish the cases and decide whether to replace function with generic or just add any in front of it. Or will it be that smart?

If fix-it will make things generic, the thing that I posted above will happen.
If fix-it will add any prefix, then nothing changes, we just gain extra things to type, people will probably just add it and continue misusing it.

I don't think this whole change is enough, even if we adopt any prefix we would still need some other ways to help developers understand their differences and reduce cases where they are misused.

Maybe if we introduced any keyword, we could still leave the current code working and provide a warning to ask developer to clarify their intention with any or change it to generic ( similar way how developers are asked to clarify whether they use SomeType.None or Optional<SomeType>.None.

This way we can avoid code-break but still have the ability to communicate the change.

Both fix-its would be given where they’d be plausible, and it goes almost without saying that any breaking change will come with a language version and/or transition period where it’s only a warning and developers have time to migrate!

3 Likes

I’d like to point out that adding any before a protocol name does not make the runtime cost of existentials any more obvious than they currently are (at least to me). One still has to do out of band learning to become aware of the costs. I don’t think this syntax is possible or even desirable but something like WitnessTable<P> makes it very clear that there’s an additional data structure here and will require additional allocation and dynamic dispatch. any P at a glance just tells me that "any type conforming to P may be used here".

Also I think the proposed syntax’s similarity to opaque result types is misleading because some does not carry any of these additional costs because it’s actually a generic type, not an existential. some P and any P look very similar but are quite different in terms of their costs.

4 Likes

There’s also the benefit of making existentials more googleable, so that tips in stackoverflow and elsewhere are easier to find. Both when users are asking the questions and when answers are provided.

3 Likes

Opaque types, generics and existentials have their own quirks (e.g. performance vs code size), not just existentials. Performance optimization is not necessary for all code, especially when prototyping or with novice users. Thus we shouldn’t make existential syntax horrible just because in some situations you are better off with generics. So aim should be fix the current imbalance and to reach a more balanced situation, not to tip the scales to the opposite direction.

It’s misleading to think of some P as related to a generic function thunk. Working with a value of opaque type incurs the same costs as working with a value of a resilient type. The witness table is looked up at runtime.

Thanks for clarifying, in that case the costs of some and any should be similar. However I don’t think that either keyword necessarily communicate additional runtime overhead to the developer. If you care about tuning performance I think it’s widely understood that using a protocol introduces overhead and generic specialization can be used to remove it. This is not a problem unique to Swift. I don’t think that adding the word any in front of a protocol name is going to make the cost of existentials suddenly obvious to the uninitiated.

This is why I suggested something like WitnessTable<P> or maybe preferably witness P . It’s more clear that you’re not interacting with a P directly but using something called a witness which I can then search for and learn about the associated costs. Searching for “Swift any” is going to yield a ton of unrelated results. any P to me communicates that this is a polymorphic value, it doesn’t say anything more about the associated costs than just P .

3 Likes

I stated the opposite, in fact. any requires unwrapping an existential. some does not. some can require looking up the witness table in cases where the compiler can’t see into the definition of the some-returning function to identify the actual concrete type.

Your WitnessTable<P> suggestion is a re-spelling of Any<P>, with the same motivation, so unsurprisingly I support it. :smile:

2 Likes

Yeah, sorry. Updated the text slightly, I didn’t mean to say generics and opaque types are the same thing.