Introduce Any<P> as a better way of writing existentials

I've tried to say this before, but hopefully this post is clearer.

I definitely support reconsidering the syntax of existential types, but I would not like to make any incremental changes to existentials without holistically considering where we'll end up. In particular, there are at least three possibilities we should consider that will affect our syntax choices:

  1. “Partial” protocol types, i.e. existentials whose protocol declares requirements not available on an instance of the existential type. It seems likely that the current “associated type or self parameter” restriction on existentials will be lifted at some point, and when that happens, we'll be faced with situations like this:

    protocol Equatable { 
      static func == (Self, Self) -> Bool
    }
    
    func test(x: Equatable, y: Equatable) -> Bool {
      x == y // ERROR!
    }
    

    where some part of the API of the declared API of the protocol in question (in this case, the whole API!) is completely unavailable on instances of that protocol type. I don't think it's possible to overstate how weird it is going to be for ordinary users that they can declare a type like Equatable and then not be able to use the declared API at all. And in fact I think it will be more damaging when only a small fraction of the API is missing on the existential because users who don't understand generics yet will happily code their way down the “easy” path of using existentials until they find themselves blocked by partiality, when it would have been more appropriate to use the protocol as a generic constraint.

  2. Constrained existentials, e.g. a Collection whose Elements have type Int. It seems obvious to me that we're going to get this feature someday, and that it will involve a where clause, e.g.

    func first(x: Collection where Element == Int) -> Int? { x.first }
    
  3. Existentials that conform to their corresponding protocol. Today, no protocol type (existential) is self-conforming, but it would be very useful (and avoid a tedious forwarding layer in many cases) if a protocol could be declared to be self-conforming, e.g.,

    public protocol Drawable: Self { func draw() }
    

    The explicit statement “: Self” is important, because self-conformance is a guarantee to clients—it determines whether they can use the existential as a generic parameter constrained to that protocol—and adding certain kinds of requirements (init and static members, and anything that makes the protocol “partial”—see 1. above) necessarily make self-conformance impossible. We wouldn't want maintainers of a self-conforming protocol to inadvertently break that guarantee.

I have the following goals for a new syntax:

  • When naming an existential type, possible partiality should be evident. For me, “Any<Equatable>” doesn't meet that bar.
  • The syntax should accommodate constraints without being overly verbose. Adding Any<…> doesn't automatically lead us to an obvious place to add constraints, and constraints are already going to add where.

Therefore, I propose we chart a path to this future state:

  • Self-conforming existentials are explicitly declared so, per Drawable above. Since they are never partial, you can name them without a “where” clause.
  • Partial existentials are spelled with a where clause. An unconstrained partial existential uses the empty where condition, “_”. For me, “Equatable where _” clearly indicates that some part of the declared API may be missing because some constraints may be missing.

If we made it an error to use a partial existential without a where clause, we'd need syntax for explicitly declaring non-self-conforming-but-non-partial protocols, so protocol authors could avoid inadvertently breaking client source by adding an init or static requirement. Therefore, IMO using a partial existential type without a “where” clause should generate a warning and a fixit: ”partial protocol type should be spelled with an empty where clause; do you want to add it?” That's a nice conclusion because it's probably the same warning we'd want as part of transitioning to the new scheme. I also like that it introduces the searchable term "partial protocol type" that we can clearly define.

Given all this, I think the near-term steps are:

  1. Add the syntax P where _ as a synonym for the existential type P.
  2. At some point appropriate to the release process, add the warning/fixit described above.
8 Likes

Do you have any thoughts about how this would apply to @objc protocols? Would they always be considered self-conforming? Otherwise, it’s going to warn on tons of “delegate” and “data-source” protocol declarations in existing code.

I think they would at least be considered non-partial, and so not trigger the warning. I assume that @objc protocols are allowed to have init and static requirements…? I know they can't have associatedtype or Self-parameter requirements.

As things currently stand in Swift, I agree with this thought. However, if we had the ability to open existentials (which is also something that seems like it's coming at some point), then it may be the case that you want to accept a parameter that you're not going to use at all; instead, you're just passing it along to someone else who will open the existential. In that case, it's not at all weird that I can't use the declared API at all, because I have no intention of doing so.

I've seen people try to design something where a generic type is needed N layers deep into a structure, and this unfortunately requires the entire structure to require generic parameters. For example, it would be convenient to be able to do something like this:

protocol DataSource {
  associatedtype Item: Hashable
  func makeItem() -> Item
}

class A {
  var dataSource: DataSource
  func doSomething() {
    let b = B(dataSource: dataSource)
    b.doSomething()
  }
}

class B {
  var dataSource: DataSource
  func doSomething() {
    // strawman syntax here for opening an existential
    let actualSource: <T: DataSource> = dataSource

    let c = C(dataSource: actualSource)
    c.doSomething()
  }
}

class C<Source: DataSource> {
  var dataSource: Source
  func doSomething() {
    let item = dataSource.makeItem()
    // Do something with item
  }	
}


let a = A(dataSource: someSpecificSource)
a.doSomething()

Currently, in order to do this, you have to make A and B take a generic parameter, just so they can pass it along to C. It's the source of complaints that "generics are viral" in Swift. But if we had the ability to open existentials to get at the wrapped type, then it becomes perfectly reasonable to accept an existential parameter that you can't actually do anything with until you open it. If we anticipate going down the road of being able to open existentials, then I'm not sure it makes sense to syntactically punish the "partial existential" case, because opening the existential regains the full expressivity of the original type.

2 Likes

I barely know where to begin. That's like saying a car with no engine isn't weird as long as I never try to drive it. The fact that you can do some things with the car without noticing the missing engine doesn't change the fact that by having the form of a car it very explicitly declares itself to be a self-powered vehicle and it's going to be a surprise when you can't get it to move.

I've seen people try to design something where a generic type is needed N layers deep into a structure, and this unfortunately requires the entire structure to require generic parameters.

Your example can easily be rewritten more simply using partial existentials without needing to add generic parameters or open existentials.

// Add this extension:
extension DataSource {
  fileprivate func c_doSomething() {
    let c = C(dataSource: self) // Body of B.doSomething in your example.
    c.doSomething()
  }
}
// Rewrite the body of B thus:
class B {
  var dataSource: DataSource where _ // my syntax for a “partial” existential
  func doSomething() {
    dataSource.c_doSomething()
  }
}

Regardless, opening existentials adds yet another level of complexity and weirdness to what will look, to the uninitiated programmer, mostly like an ordinary interface type with capabilities like those of an Objective-C protocol that is missing part of its declared API.

Currently, in order to do this, you have to make A and B take a generic parameter, just so they can pass it along to C. It's the source of complaints that "generics are viral" in Swift.

Until I see a real counterexample, I'm going with the assumption that what you call “unfortunate” is exactly the right design for that component in Swift today. I am not compelled by the fact that people complain about being “forced” to encode type relationships. After all, if we had a large contingent of adopters from the Python community, they'd complain that “types are viral in Swift,” but Swift is opinionated about static typing being a good thing. I am not necessarily arguing against the ability to open existentials, but it certainly will add complexity to the language and will do nothing to decrease the overall weirdness of partial existentials.

If we anticipate going down the road of being able to open existentials, then I'm not sure it makes sense to syntactically punish the "partial existential" case, because opening the existential regains the full expressivity of the original type.

What I'm proposing is 2 characters longer than wrapping the protocol in “Any<…>”, and those 2 characters are lightweight (“ _”). Furthermore, unlike “Any<…>,” it does not introduce any nesting. To frame this as “syntactic punishment” is hyperbolic at best. More importantly it fails to acknowledge the motivation, which is not to punish, but to clarify the fact that you'd have to do something special to access the whole declared API. Clarity is the main motivation for the proposal that started this thread.

Note that constraining the existential sufficiently is another way to get back its full declared API, and usually a better one.

1 Like

Is this a situation where opaque types could work instead and the compiler can figure out what some DataSource is three levels deeper? I do not fully understand opaque types, so I don’t know the answer.

I've been very scattered these last few weeks and haven't been able to think about this in a substantial way, so apologies for the belated reply.

The conversation has moved on and I don't want to hijack it back, but I figured I should address your points addressed to me as best I can, and chime in about one recent comment you made--

I can't speak to the "since then" part, but Any self-conforms, and that is not the case for existentials (and cannot ever be in the general case, however they are spelt). It is, I suppose, for that reason that @tbkka has referred to Any (and Error) as types that aren't really existentials for the purpose of his document on dynamic casting.

This is not the only difference in behavior that we might want for Any versus existential types. I don't pretend to have thought through a complete list, but I can give at least one example:

You've raised the point about supporting extension any T { ... } down the line. The core team has said elsewhere (IIRC) that there is no intention to support extending Any. Even if we were to want to discuss a spelling to extend all types, that should be a separate, orthogonal discussion from that regarding extending existential types.

Both some P and any P are distinct non-nominal types related to P in some way. That is the model that is exposed to end users; whether the underlying implementation requires this or that runtime support is immaterial from that standpoint. One is not more "wrapper"-like than the other in that sense.

Yes, I do disagree; I think I've commented to that effect elsewhere.

Two points about this:

(1) Since extension Any has long been used in discussions as the hypothetical spelling for a feature that adds methods to any type, I would expect by analogy that a spelling extension Any<T> would in fact extend any T; so I'm not sure that your proposed spelling solves that problem.

(2) Whatever the syntax, now that you bring it up, why not make speak() available in your example to both the existential and every conforming type?

2 Likes

Reading an older post on a related topic I had another idea for a possible spelling in lieu of using the any P spelling. As Dave illustrates in the linked post, an existential is only as much of its protocol’s API as possible to be “the most specific possible common supertype of all types conforming to the protocol.”

With this in mind, I think a more precise spelling would be as partial P to communicate that the existential type only encompasses the part of the protocol that can be common to all conforming types (I think for some protocols, that could be all of them, making the partial P name seem awkward in those instances). Additionally, any extension extending functionality to a partial P for which the compiler can determine has no accessible API, like the P in Dave’s example, should generate a warning.

1 Like

The current opaque return type feature only applies to return types, so it wouldn't work for the example as written, which wants to store a DataSource of unspecified type. That's clearly a role for an existential…
though I'll reiterate my skepticism about whether the real-world problem the example represents ought to be approached that way.

This is an interesting idea, but again when you consider future language features like constrained existentials it makes less sense, because when you add enough constraints, the existential is no longer partial. So the endpoint of the progression

  • partial P
  • partial P where A0 == Int,
  • partial P where A0 == T0, A1 == T1
  • partial P where A0 == T0, A1 == T1, A2 == T2
  • … etc.

is either incorrectly labeled partial, or suddenly no longer needs the partial prefix. Also, IMO, it's just more needless verbosity in all but the first case since all the other cases need where.

to communicate that the existential type only encompasses the part of the protocol that can be common to all conforming types

Eh, I'm not sure what you mean by that; the entire protocol is always common to all conforming types.

Perhaps there’s value here then for the compiler to require partial while the where clause doesn’t constrain it enough to expose the whole API, and then to conversely not allow it when the constraints are such that the existential is “whole.” This requires at the use site for the user to explicitly acknowledge the partiality of the existential in the unconstrained or underconstrained case which, unless I am mistaken, is the crux of the problem this pitch was meant to address.

I apologize for my imprecise wording. I meant that it only has the accessible API of, as you described, “the most specific possible common supertype of all types conforming to the protocol.”

1 Like

I like this idea a lot, where partial is only needed when the protocol type does not conforms to itself. I think the compiler should emit a warning and a fixit to remove partial whenever you have enough constrains to have the protocol type starts conforming to itself, and another to add partial (or specify constraints) when not conforming to itself.


And this now has a meaningful error message:

func test(a: partial Equatable, b: partial Equatable) -> Bool {
    a == b // error: partial Equatable has no ==
}

Someone's reaction could be to remove partial from the parameter types, which should get the compiler to propose adding constraints:

func test(a: Equatable, b: Equatable) -> Bool {
    // warning: Equatable needs constraints or should be labeled as partial
    // fix: add `partial`
    // fix: add `where Self == X` constraint
    a == b
}

Not that there's much that can be done in a situation like this, but at least it lets the user know that Self needs to be bound for the protocol to be usable.


And perhaps in the future we'll be able to write:

extension partial Equatable: Equatable {
    static func ==(a: Equatable, b: Equatable) { ... }
}

and it'd make partial Equatable conform to Equatable, which means you no longer need to write partial. This is why the == function here uses Equatable.

2 Likes

This is correct; @objc protocols can have inits and class methods, and Self in return type position only. None of these stop a protocol from being used as an existential, or an @objc protocol from self-conforming.

(The limitation on Self corresponds to instancetype being return-type-only in Objective-C; using Self as a parameter or var type in an @objc protocol declared in C causes a because its type cannot be represented in Objective-C error.)

So far, all the discussions for the syntax were revolving around the protocol. I'd like to put on the table another idea - what if instead we try to make the syntax starting from the type of the value boxed by the existential.

Any could be spelled as any<T> T
P could be spelled as any<T> T where T: P
P & Q could be spelled as any<T> T where T: P, T: Q
any Array could be spelled as any<T> Array<T>

In the most general case, this gives feature parity between existentials and generics:

any<T, U> Array<T> where T: Collection, T.Element == Optional<U>

Actually, I've tried not to refer to Any or Error as protocols. The distinction I'm working with there is between existentials (which includes protocol witness types, Any, AnyObject, and Error) and protocol witness types (which are one particular kind of existential).

1 Like

I love this idea. I think it complements very well Dave Abraham's goal of clarifying at the point of use. (Thanks for such a thorough post by the way, Dave.)

Personally I strongly prefer the spelling partial P to any P — I find the former much more intuitive. As someone who has followed this thread sporadically, I always had to "talk myself through" the difference between the any and some keywords. partial makes it much clearer to me.

I also liked Dave's suggested spelling of P where _ because of the natural "extension point" it provides. That underscore begs to be filled in. The reason I would sway toward having a partial keyword is because it's presence reinforces that the existential is not yet "whole" and therefore I should expect the API to be incomplete. Without that one may forget that Collection where Element == Int for example is still not whole because it lacks an Index constraint.

But perhaps that is too much information and clutters the point of use too much?

I’m trying to parse this to understand your meaning. Is the => some kind of new operator you are suggesting or just a shorthand for “could be spelled as” to demonstrate, for example, Any could be spelled as any<T> T. I’m assuming the latter. The crux of your suggestion is to have any act as a sort of generic keyword that has its own angle-bracketed list of generic constraints?

I don’t think it reads very clearly.

Cross posting this as I think it related to this thread as well:

The later, I’ve updated the post to avoid confusion for future readers.

Yes, you are right.

At one point in history rust language was in the same place as swift now - they found out that using just the name of the trait (protocol) to denote the trait object (existential) is a bad idea.

Their solution was to use the word dyn which is similar to impl (their equivalent of some)

I think we should use any (or maybe another short word) to preserve the symmetry, just like rust did

dyn Foo - impl Foo
any Foo - some Foo

12 Likes

I like this use of partial @michelf - the error messages become very descriptive, and the self-conformance syntax is very clear.

2 Likes
Terms of Service

Privacy Policy

Cookie Policy