SE-0309: Unlock existential types for all protocols

Good proposal. +1

I've only read through and haven't thought about the issues very deeply but my instinct is go with the proposal as is rather than introduce new syntax (any P) as I don't really see any ambiguity or confusion that would avoid so it would seem like semantic clutter to me (although I may be missing something).

The only part I'd wish some improvement to is the error messages could potentially become more beginner friendly (although I don't have specific better wording in mind) and I certainly wouldn't want the search for that to block or delay such a major improvement.

Because some P would be an unknown concrete type conforming to P. You can pass only an instance of that concrete type in order to call the function, but you don't know what that concrete type actually is, therefore you can't pass anything. Under the hypothesis of non mutability of anyQ, its type cannot change, thus we can refer to it as of type some Q. The type signature of anyQ.takesB would then be

((type of 'anyQ').B) -> Void

If you happen to have a variable of type (type of 'anyQ').B (e.g. anyQ.b), then you can pass it as a parameter.

No, some Type implies an unknown concrete type, not an existential. Existentials would be spelled with the bare protocol name alone or eventually with any in front of it. There are already rules in place for opaque types to prevent type changes and assignments with types different than the one the variable already has.

Diagnostics also refer explicitly to the name of the variable in some simple cases

The reason why I used (type of 's') is because diagnostics already refer to path-dependent types that way:

var a: some RangeReplaceableCollection = [1]
var b = a
var c: some RangeReplaceableCollection = [1]
let ab = a + b  // valid
let ac = a + c  // error

The type of a and b is considered to be the same, while the type of c is considered as an unknown concrete type conforming to RangeReplaceableCollection unrelated to a's one.


I proposed Type since it's somewhat forbidden in some places, so we can officially "demote" it to a reserved keyword as much as Any is. The less reserved keywords there are, the better it is and Swift has a ton of reserved keywords already. I also think it reads better: p: any Type means that the type of p can be any type, but of course suggestions would be very welcome (anything reads better than any Any). Value, on the other hand, may recall the dichotomy between value- and reference-types.

1 Like

But which side is aware of the underlying type?
In case of a parameter, this is usually the (nth) caller, but not the callee opposed to opaque types in return type position.

The former is true unless inferring the type from error messages.

Unfortunately, "some" Types aren't really unknown

I don't even like some types being compared to each other, changing the underlying type in one may break linking against this library just because opaque types can be compared to their identity, but that being said, it was already decided.

The latter seems not to be true, some P and any P are both existentials, the former is a compile time existential, the latter a runtime existential.
The former is less powerful than the latter but enables for better inference leading to better performance.

Another reason against to use opaque types here is that they can't be used in argument position yet.
They have to be introduced and semantically defined in another proposal.

+1, been waiting for this for a long time

5 Likes

I'm very happy to see this moving forward finally. Improving this is definitely pushing Swift in the right direction.

One thing I'm still making my mind about is about the introduction of any P. When I read about it on the "improving the UI of generics" it made everything clear, so I like it very much. But sometimes I feel like it could make things much harder to explain, specially since I think other languages don't make this distinction and "just work". (am I right that Java/Kotlin do it?)

1 Like

Excuse me if this sounds silly, however, could I ask why any P is considered to be needed (in the future or as an alternative)? Read proposal and Improving the UI of generics but still confusing to me.

To make type declaration clear that "A protocol cannot be conform to itself"?
In that case, why some P is not sufficient?

(I'm not against the introducing the new keyword at all, but would like to understand them)

2 Likes

Two things come to my mind:

  • ABI constraints for the old syntax (without any) exist
  • any P seems to be a nice dual to some, simply writing the protocol name make it hard to know which kind of existential we mean here (some or any)
  • a protocol is theoretically not a type, just a constraint/predicate, but with this proposal we would also refer to the existential type (the protocol as type), instead this should be disambiguated with additional syntax to make the distinction more clear

Simply because of that:

  • We can use any P in parameter position but not some P yet
  • some P in argument position would be less powerful regarding non-determinism
2 Likes

Note that any P is not part of the proposal being reviewed in this thread.

6 Likes

IMO, the benefit of having new syntax would be if it could clearly distinguish type erasure from generic code. I'm not entirely sure that any P does that.

Basically, today a new developer would read TSPL, learn that "protocols are types", and that they can use a protocol anywhere they would use a type. Then they'd write a function, say...

func frobinate(item: MyType)

And then some time later, when they want to reuse this function, they'll think "hey, this code isn't actually specific to MyType; I only use stuff from the protocol P", and, given what they've learned from TSPL, go away and write:

func frobinate(item: P)

Which will work, and very much looks right - they can call this function with a MyType as before, but also using any other object whose type conforms to P -- but this isn't actually what they wanted to express. What they wanted was to make their function generic, but instead, they've got a non-generic function which accepts a box holding some unknown object conforming to P. That distinction becomes very important once you introduce any sort of complexity to the protocol or the function - associated types or Self won't work, you'll need extra indirection to use inout parameters, and you won't be able to interface with real generic code (the so-called self-conformance problem).

Consider how much clearer this would be if we made things a bit more explicit - rather than just P, or even any P, consider if we wrote BoxContainingSome<P>:

func frobinate(item: BoxContainingSome<P>)

Now it becomes a lot harder to confuse that with a generic function - yes, you can use it with any object whose type conforms to P (if you box it first), but it makes the indirection responsible for all of the above problems more obvious.

People shouldn't be reaching for existentials when they look for generics. There are uses for variables whose type might change at runtime, but they are quite limited, and giving them a more explicit, verbose spelling would probably be for the best.

7 Likes

Thanks for taking your time.

Those totally make sense!

And yes, this.
I was wondering why some P aka Opaque Types isn't sufficient enough as any keyword.

So re-read The Swift Programming Language: Redirect, then noticed that its purpose is different first of all, and understood introducing something distinct is necessary.

So... some is to cover something, any to absorb something.

@Joe_Groff and I discussed off-forum briefly. My team has a use case in favour of this proposal.

We've got a protocol for a type whose instances may cross process boundaries as XPC messages; for the sake of this discussion, let's call it Action. Action has two associated types, Input: Codable and Output: Codable, and an instance member, func invoke(passing input: Input) throws -> Output. Conforming types implement invoke(passing:) in order to transform an Input into an Output. Nothing too Earth-shattering there. :slight_smile:

Now, we want to refer to individual instances of Action in a generic context because:

  • we need to enforce type safety and security across the process boundary by defining an array of Action.Types that can be safely decoded after transmission; and
  • we want to be able to look up which Action.Type needs to be instantiated and invoked in said array.

But because of Action's associated type requirements, we can't create an Array<Action.Type> directly. We instead need to create a type-erased AnyAction protocol with no associated type requirements. In our case, we imbue this protocol with a single member function static func invoke(usingContentsOf xpcDictionary: xpc_object_t) -> Void, and we make Action conform to it. Action provides a default implementation that does very little: it decodes an instance of Self, decodes an instance of Input, invokes invoke(passing:), and yields either an Output or an Error as an XPC reply. This function has no associated type or Self requirements in its interface—they're all part of its implementation. If we could refer to Action as an existential type, then AnyAction would be unnecessary. We could make invoke(usingContentsOf:) a member of Action and could remove AnyAction from our project.

tl;dr: We have a real-world use case for referring to a protocol with associated types as an existential type. If this proposal is accepted, we'd be able to drop AnyAction and use Action directly in its place, which would mean the elimination of a fair bit of boilerplate and glue.

22 Likes

+1 to the proposal overall. Lifting this restriction to individual members is a nice compromise, and I especially like the upcasting trick for Self/associatedtypes in covariant positions. I'm also in support of @xAlien95's suggestion that implicit upcasting to Any require an explicit as coercion, like we require in other places already.

4 Likes

Amazing! Huge +1 from me! I've been wanting this for a long time!

I'd like to suggest mentioning in the proposal the unprecedented ability to dynamically check for conformance to a PAT:

func foo(_ value: Any) -> Bool {
    return value is Hashable // not possible before
}

Also, I'd like to suggest mentioning how static members and initializers will behave:

func foo(_ value: RangeReplaceableCollection) -> RangeReplaceableCollection {
    return type(of: value).init() // Okay, even though the type information is missing at compile-time and it might feel like it wouldn't work, accessing the dynamic type recovers it at run-time allowing the initializer to be used.

Also also, I'd like to suggest mentioning in the proposal the inability to refer to static members and initializers when referencing them from the protocol itself:

RangeReplaceableCollection.init() // Error: missing existential type (either static or dynamic)
8 Likes

I know programmer should be nudged to try generics first, and right now existential first wins simply because of too light syntax
func foo<T: MyType>(value: T) // generics
vs
func foo(value: MyType) // existentials

But i think it’s wrong to start punishing existential syntax simply because typically generics are used with named label (such as T above). If generics was used anonymously, we could just have
func foo(value: some MyType) // generics
vs
func foo(value: any MyType) // existentials

I think any P already makes existentials googleable and nudge enough the programmer to understand existentials are not the obvious or even preferred choice.

5 Likes

That’s a good question. I think that others have already pointed out the major advantages; I’m just going to sum them up:

  1. Existential types of protocols and those protocols are often conflated. This not only hinders searchability, but it is also unintuitive in certain cases, like Self-conformance — e.g. the existential of Hashable does not conform to Hashable. As a result, language users are met with quite confusing diagnostics: Protocol 'P' as a type cannot conform to the protocol itself

  2. The any syntax would help clarify the distinction between generics and existentials, by demoting existentials from being their "default"-abstraction position. Namely, these two abstraction mechanisms differ in terms of performance and usability characteristics. Generics are compile-time type-level abstractions; therefore, they are more performant and they enable type constraints. Existentials, on the other hand, are run-time value-level abstractions and are useful in dynamic environments and in cases where the additional complexity of generics is unnecessary.

2 Likes

Nobody is suggesting to punish existential syntax - only to make clear exactly what it does. The current spelling is so brief that it fails to communicate important information.

There seems to be some kind of assumption that any P is the heir apparent to our current spelling, given the similarity with some P as used for opaque types. I think there is room to challenge that assumption (hopefully others, who are better at coming up with names for things than I am, will have ideas). That's why I think it's good that this proposal doesn't include alternative spellings - it's an open question.

I also feel that there is some kind of implicit assumption that developers shouldn't even need to care - that they should just write whatever (waves hand) and have the optimiser figure it all out. It may be able to do that in some limited cases today, but my experience profiling Swift code tells me that there is really no good alternative to clearly saying what you mean. I have serious doubts about whether jumping back and forth between type-level and value-level abstractions willy-nilly will really have no impact on the optimiser's ability to do its job.

Thanks for the feedback!

The same way one can currently write let a: Codable = 1, they would be able to write let b: Hashable = 2. Not to mention, that a client using an existential-accepting, library-defined function would lead to the user's implicitly wrapping a value in an existential type.

Wouldn't any Sequence also need to be Self-conforming? (The printEnumerated(s:) functions needs to support passing values of existential type any Sequence and some Sequence would be transformed into a generic parameter with a conformance-constraint to Sequence.)

any P and very many other alternatives were bikeshedded a lot in the pitch threads in the past. Personally I had hoped we already had a syntax which doesn’t have any issues with the all the possible ways it could be used in the future, at least several other syntaxes had those problems.
For example, this future syntax would work great with any P:

let collection: any Collection<Self.Element == Int> = [1, 2, 3]

4 Likes

Unless we would refactor metatypes into meta T and any meta T (there are two kinds of metatypes which are currently merged, which is also why type(of:) cannot be properly implemented in Swift) I don‘t think there is a chance to unlock the T.Type syntax for custom types called Type.

1 Like

My two posts are unfortunately not relevant in some places because, while I was certain to have read the opaque type proposal at the time, what I instead read were the Improving the UI of Generics thread and the Generics Manifesto. I realize that my understanding of opaque types was partially incorrect, shame on me :slight_smile:


I was referring to the Covariant Erasure for Associated Types section specifically.

protocol P {
  associatedtype A: Hashable
  var a: A { get }
}

let p: P = pConformingInstance

let member = p.a
// error@-1: 'member' is not concrete; add explicit type annotation if this is intentional
// [fix-it]  Insert ' as Hashable'

It would almost be the same error you receive when using an heterogeneous array literal, in which case ' as [Any]' is suggested.

I suggested it due to my misunderstanding of opaque types, (off-topic)

But without the wrong assumptions I had (they are in the (off-topic) collapsible section above if you're interested), I fear there would be too much verbosity since any associated type in covariant position would need to be annotated with ' as Protocol' and it may become particularly tedious if generalized existential will come into play...

Insert ' as RandomAccessCollection<.Index == Int, .Element == (key: String, value: Double)>'
1 Like