[Pitch] Introduce existential `any`

Joe_Groff · November 30, 2021, 9:11pm

Thanks for starting the conversation here, Holly! I just have a couple of comments:

hborla:

Semantics of explicit existential types

The semantics of any types are the same as existential types today. Explicit any can only be applied to protocols and protocol compositions; any cannot be applied to nominal types, structural types, and type parameters:
struct S {}

let s: any S = S() // error: 'any' has no effect on concrete type 'S'

func generic<T>(t: T) {
  let x: any T = t // error: 'any' has no effect on type parameter 'T'
}
any also cannot be applied to Any and AnyObject (unless part of a protocol composition).

This feels unnecessarily fussy to me. I think we should allow any generic constraint after any or some, including Any or AnyObject. any Any being redundant seems like enough deterrent to keep people from writing things that way. However, would any by itself potentially work as syntax for a type on its own?

With explicit any, we have an opportunity to eliminate the weirdness of .Protocol completely. The existential metatype should be spelled any (P.Type)—because it's an existential first, that happens to be a generalization over metatypes. (any P).Type should then be the metatype of the existential itself, P.Protocol today. This would hopefully make it clearer why, when you substitute T == any P into a generic T.Type, you get the latter rather than the former.

CFUsername · December 2, 2021, 6:03pm

Transition period sounds good, but I'd still argue that we should not treat current code as error even if its decided that any keyword will be accepted.

Swift already has not so good reputation in terms of code stability, there were bunch of tutorials and print books that got outdated between 2/3 and 4/5.
Making current behaviour an error would make this problem even bigger, it becomes much harder for beginners to follow tutorials or learn Swift when the sample code does not compile.

This was the whole idea and a promise of Source compat suite, but now we are making a breaking change for no real benefit ( IMHO ).

This is why I think that by no means should current syntax become an error, there is just too much harm by doing so.
Maybe a warning that would explain difference ( link to a documentation ) between existential/generics AND would still continue working as currently is would be ok, but it should not become an error even after transition period.

Zollerboy1 · December 2, 2021, 10:36pm

Swift 6 Language mode will probably bring along several other source breaking changes (e.g. Sendable enforcing iirc). That's definitely not perfect, but Swift is still quite young and IMHO it's better to make some source-breaking changes now rather than carrying on design mistakes into the unforeseeable future. But obviously the transition period, where it's still a warning and not an error has to be quite long, as we don't want to become Python (Swift 6 Language mode and Swift 5 Language mode forever...).

hborla · December 7, 2021, 12:01am

Hey folks, I've updated the proposal based on all of the feedback here. The latest proposal contains the following revisions that we've discussed in this thread (along with justification in the detailed design section):

Spell the existential metatype as any P.Type , and the protocol metatype as (any P).Type.
Preserve any through type aliases.
Allow any on Any and AnyObject.

I've also addressed the common Any<P> suggestion in the alternatives considered section.

Thanks all for the insightful discussion so far!

Les_Pruszynski · December 7, 2021, 2:01am

I think it's a great pity that we don't have the courage to change Any and AnyObject to any Value and any Object now with this migration to any. Allow it in 5.5 mode and require to change to a new spelling in mode 6.0 as per the any migration. This clean-up is absolutely worth the trouble now, in my humble opinion. This may be our only chance to do it.

hborla · December 7, 2021, 2:46am

I'd be less resistant to the change if there was a more compelling reason to make it. I truly don't believe that Any and AnyObject cause confusion, and we'd probably want to keep type aliases in the standard library so people can continue writing Any and AnyObject, partially because any project that defines its own Value or Object protocol would need some kind of disambiguation anyway. I also don't think that Value clearly describes its semantics, but I do think it would be nice to have a better way than some Any to spell an unconstrained opaque type.

BigSur · December 7, 2021, 5:18am

I have the same feeling that any Value is not good enough to replace any Any, but any AnyObject still look weird and not a suitable name for any class object.

Can we redesign the root protocol name to match the four fundamental kinds of basic types class/struct/enum/actor. Use Object as root of all values, and Class, Struct, Enum, Actor for its kind of type protocol. Looks like below

any Object //everything all object
any Class //any class object
any Struct //any struct object
any Enum //any enum object
any Actor //any actor object

and we already have Actor as root protocol of all actors. So it's perfect match to this conception. I'm not sure Struct/Enum root protocol will have future meaningful usage pattern, if proposal author feels it's unnecessary to disambiguate Struct and Enum we cam combine them as Value protocol and use any Value to represent all value objects.

DevAndArtist · December 7, 2021, 10:41am

I would like to further discuss one of the remaining issues I still view as not covered. It does not mean that the proposal itself has to provide a solution for it, but it's at least something I would like to see in the future direction section.

To begin with, I would point out that the typealias examples showed that we would like to keep two forms which both have their specific use-cases.

In that sense, I'm a bit torn by the fact that P.self will be banned to prefer (any P).self instead, or at least I got such an impression from the latest proposal revision.

It remains possible to get an existential metatype via an explicitly constrained generic type parameter (e.g. func foo<T: P>(_: any T.Type)). However it will not be possible to decompose P from any P when it's passed as a parameter, which implies that you still cannot create the existential metatype for types that conform to a specific protocol from some general generic context:

func bar<T>(_: T.Type, _: any T.Type) {}

protocol P {}
struct S: P {}

// The general issue becomes more clear now!!! 🎉
// 
// Error: S does not conform to (any P).
// Rhs becomes `any (any P).Type` which isn't much useful.
bar((any P).self, S.self) 

// 🤨 banned
// What does `P.self` even return?
// Is `P.Type` a NEW additional singleton metatype?
// Rhs would mean `any P.Type` which is what we're after!
bar(P.self, S.self)

How could we fix this problem in the future (or maybe with the current proposal)?

If we're making a split, can we apply that to P.self and (any P).self as well?

Would it make sense to alternatively revisit the removal or replacement of .self syntax (in a different proposal)?

Nickolas_Pohilets · December 7, 2021, 10:53am

AFAIU, any T.Type is not a valid syntax, because bar<T> implies that T is a type. We would need a completely new feature to indicate that T is not a type, but a protocol - something like bar<T: Protocol> or bar<protocol T>.

Note that currently this is also not possible:

func bar<T>(x: T.Protocol, y: T.Type) {}
//               ^- error: cannot use 'Protocol' with non-protocol type 'T'

So the proposal does not change the status quo.

DevAndArtist · December 7, 2021, 10:57am

I don't view the necessity of a clear distinction between a super-type and a protocol conformance constraint on a syntactical level especially because of the idea of Generalized supertype constraints in the generics manifesto.

Therefore I would strongly argue that every R in a T : R relationship can be replaced with either a valid super-type of T or a protocol it should conform to.

Nickolas_Pohilets · December 7, 2021, 11:01am

Indeed, I think something like this would be able to express what you want:

func bar<T, U>(_ : T.Type, _: U.Type) where U: T {}
protocol P {}
struct S: P {}
bar((any P).self, S.self)

But that's a different feature, completely orthogonal to the proposal.

DevAndArtist · December 7, 2021, 11:04am

Your example remains illegal because S: any P is not a valid relationship as that's what your example decomposes to.

Therefore one of my previous questions, would it make sense and or technically possible to having P.self return a new singleton metatype P.Type? This would solve this whole issue.

I think every time someone reached out to P.self, they actually expected to get something like P.Type (not to be confused with any P.Type), but instead they got confused because the resulting value was actually (any P).Type.

The proposal could make another breaking change to the language and introduce P.Type as a resulting value from P.self while it would also introduce the new, separate (any P).self from that returns (any P).Type (the old P.Protocol).

		old	new
T	T.self	T.Type	T.Type
P	P.self	P.Protocol	(new but a breaking change) P.Type
(any P)	(any P).self	P.Protocol	(any P).Type

If we would keep P.self alongside (any P).self we will have a breaking change, but it would actually introduce a much broader fix to the language expressiveness.

Nickolas_Pohilets · December 7, 2021, 11:06am

One more thing, which seems to be not discussed yet. Does proposal change behaviour of String(describing: (any P.Type).self) and String(describing: (any P).Type.self) or keep them producing "P.Type" and "P.Protocol"?

Nickolas_Pohilets · December 7, 2021, 11:42am

Ok. Two new features then: one to support generalised super type constraint, one to create relation between S and any P. But still, IMO, it is orthogonal to the proposal.

Off-topic

I was starting a discussion about the second one, and will be happy to write a pitch after this proposal is approved. I think it provides a better alternative to self-conformance, and can help to eliminate self-conformance from the language. And also addresses use cases not covered by self-conformance. If you are interested in this topic, please comment in that thread.

If would not be a metatype, maybe we can call it a metaprotocol. Of type P.Protocol. Not to be confused with old P.Protocol aka (any P).Type! Ok, let's call it P.Constraint to avoid confusion.

Then I guess you want to write something like this:

func bar<T>(_: T.Constraint) {}
//               ^- error: cannot use 'Constraint' with non-protocol 'T'

So first, you need to express that T is a protocol:

func bar<protocol T>(_: T.Constraint) {}

But once you have it, you can write:

func bar<protocol T>(_: (any T).Type, _: T.Type) {}

which addresses your initial intention without T.Constraint. So my conclusion is that so far we don't need meta-protocols.

You'd need them to operate with protocols in the runtime. But currently I cannot think of any use cases.

DevAndArtist · December 7, 2021, 11:47am

I think this makes the situation much more complex that it really should be. As I mentioned previously in this thread, the "true" P.Type kinda already exists, but it's not obtainable or properly expressible. This metatype fix via the any keyword allows us to also fill this hole.

protocol P {}

func whatAmI<T>(_ t: T.Type) {
  print(type(of: t))
}

whatAmI(P.Type.self) // P.Type.Protocol
                        ^~~~~~

Make a breaking change, and let P.self actually return that P.Type instead while shift the previous P.Protocol to be the result of (any P).self and re-brand it as (any P).Type. Done.

People are constantly reaching out for P.Type but get (any P).Type instead because of all this metatype fusion madness. This is the right time to fix this, and I disagree that it's orthogonal to this proposal. Having the ability to actually get the singleton metatype P.Type would fix sooo many issues and confusion in the swift community.

This one extra thing unlocks an entire new set of generic algorithms. We can either ignore that or push this forward while we're already changing the status quo here. I've earned a lot of disagreement in this thread, just to see the core team join the conversation and make things clear to actually include the changes I've been fighting for the whole time.

P.S.: If you have Swift compiler know-hows, implement a quick and dirty proof-of-concept of this and see how awesome the expressiveness of the language becomes. It's a win win situation.

Nickolas_Pohilets · December 7, 2021, 12:15pm

Could you please provide as an illustration a couple of algorithms generic over protocols? This would help discussion a lot.

Jumhyn · December 7, 2021, 1:20pm

The discussion here has convinced me this is the proper interpretation (thanks @DevAndArtist!), though I remain concerned about having these two spellings mean different things. I expect this will make it more confusing to elucidate the difference between the metatype existential and the singleton metatype, and certainly to tell them apart at a glance. To the extent that P.Type under this regime is a special syntax, I wonder if we could change the spelling to put a bit more lexical distance between these two forms. It would also avoid us having to explain why this .Type isn’t a real .Type (since it isn’t applied to a type). Not an actual suggestion, but something like any P.MetatypeExistential would make it clear that this is a different existential type related to P.

Overall, though, this is a pretty small issue and I won’t grumble about it any further, the pitch looks to be in quite good shape to me.

xwu · December 7, 2021, 1:35pm

Good point! Getting P from any P would be a useful feature for the reason you describe and hopefully there is room for this in future Swift. Since it's not possible to do today I agree that it could be left for a follow-on proposal, but it would be nice to have commentary on that point.

Could we not maintain the existing spelling P.Protocol and give it this new meaning (after a period of transition)? I for one would find that to be a more intuitive meaning for that spelling anyway.

DevAndArtist · December 7, 2021, 1:56pm

I'm not a compiler engineer, but if I'm not totally mistaken then there exists an "existential metatype" for every type, not just for protocols (e.g. any Int.Type). While it's not that much useful as it can only hold Int.Type, I still think this is/was a thing. Feel free to correct me. In my previous attempts explaining this I was referring to the proposed (any T).Type as meta T and any T.Type as any meta T while also introducing another pseudo keyword meta. A more pleasant to parse form added typealias Meta<T> = meta T (or introduced a standalone Meta<T> form instead).

Here's a list of all the forms:

	proposed	alternative 1	alternative 2	old proposal alternative
singleton metatype	`T.Type`	`meta T`	`Meta<T>`	`Type<T>`
existential metatype	`any T.Type`	`any meta T`	`any Meta<T>`	`AnyType<T>`

One of the goals here, was to free up the .Type name space for custom types. I'm personally tired of calling everything Kind.

this.

While I think I understand what you meant here, I'm not quite sure, because it seems to mix everything up again. P.Protocol is already taken as it will become (any P).Type under the current proposed revision. What I'm interested to hear from the compiler engineers or the core team, can we introduce the "true" P.Type (mentioned here) now?

IIRC I had a short conversation regarding this type with @Joe_Groff on twitter, but it was long long time ago. It was the point where I learned that this type is currently unavailable nor properly expressible in Swift.

P.Type - would be the singleton metatype of the protocol P
(any P).Type - would be the singleton metatype of the protocol existential any P

Nickolas_Pohilets · December 7, 2021, 2:55pm

In general I like the idea to give P.Type in any P.Type a different spelling. Keyword any is already an indication of the existential, so IMO, word "existential" in the name is redundant. Instead, I would try to formulate the name so that it clearly describes what are the values of this type. Which leads me to a suggestion of any P.ConformingType. WDYT?

But regardless of the syntax, it got me thinking about what is P.Type in any P.Type. My understanding is that "existential type = any + constraint", so "existential type - any = constraint". Which leads to a question for @hborla, will P.Type be usable as a constraint under the proposal? Can I write something like this:

protocol P {}
struct S: P {}

struct Z<T: P.Type> {
    var value: T
}
let z = Z<S.Type>(value: S.self)

typealias PType = P.Type
func dummy<T: PType>(_ x: T) -> any PType { x }