Lifting the "Self or associated type" constraint on existentials

(Dave Abrahams) #102

This is the one argument I think really holds water against what I've suggested, and I've been waiting for someone to make it; sorry I missed your previous mention. My best answer is that I think the unconstrained existential is special. I think it's a serious problem that you can have a type called, simply, SomeProtocol and go look at the declaration of that type, and not be able to access some part of the API declared there. If it were spelled “SomeProtocol where _”, then I'd probably be more comfortable with some of the protocol's API being implicitly unavailable, because there's this attached “where I don't know anything other than that the type conforms” right there in the code.

Generalized existentials are an extremely powerful feature that includes a nontrivial amount of essential complexity. Learning how to work with them will necessarily involve some effort. I don't see any way around this.

I don't think this means we should not support the feature or be afraid that users won't be able to use it effectively (one thing I love about Swift is that it has mostly avoided that kind of paternalism). I think it means we need put more effort than usual into educating people about this feature, how to use it well, and what pitfalls or traps they to be aware of. This is especially necessary as generalized existentials is a relatively sophisticated feature for which I don't think there is a clear point of reference in the previous experience of most programmers.

Agreed 100%. Precisely because the feature “includes a nontrivial amount of essential complexity,” we need to be careful about how we introduce it, how it will interact in users' minds with the rest of the language, whether we're putting that complexity in too close at hand when features with fewer pitfalls are farther away, and what new problems arise when we introduce part, but not all, of the feature.

(Joe Groff) #103

This is not a fixed property of public protocols, since they can resiliently add new requirements. We wouldn't want it to be source-breaking to do so.

(David Sweeris) #104

Have we closed the door on having different behavior for compiling executables vs libraries? I can't recall if anyone has already asked... If not, this seems like a good candidate for it: the compiler can assume all types are present & accounted for when compiling "My Great App" (but not when compiling "libMyGreatLibrary" or however we're libraryifying names these days), and if there's only one type that satisfies the requirements, great, use that. Otherwise throw an "ambiguous type" error or tell the developer they'll need to make the function/type generic (depending on which is more applicable).

(Xiaodi Wu) #105

...but it is the status quo. And if we evolve in such a way that a new requirement with Self or associated type requirements merely requires a protocol opt in to the existing spelling via annotation, then even that would not be source-breaking.

(Joe Groff) #106

If we were going to introduce a new syntax for existentials, I think it'd be much less confusing if we did so uniformly, deprecating the existing syntax after a late-enough language version, than to have two syntaxes where one only works some of the time. That seems to me like it'd only make things more confusing, personally.

(David Sweeris) #107

Is there a reason (besides inertia) to not "declare these protocols differently"?

(Joe Groff) #108

It isn't clear that they are fundamentally different, and one "kind" can morph into the other over time.

(David Sweeris) #109

Does "over time" mean "during execution", "as code evolves", or something else?

(Joe Groff) #110

As code evolves, sorry.

(Jordan Rose) #111

Apps aren't the end of the line either. Unit tests often add dummy types; the Swift runtime supports dynamically-loaded in-process plug-ins; and you can even declare new types or add new conformances in the debugger if you want.

(David Sweeris) #112

Cool! I did not know that.

(Xiaodi Wu) #113

Here is a good example of a genuine use case where our current restriction would have saved some confusion if the protocol in question had Self or associated type requirements:

(Joe Groff) #114

That's a bug (I misunderstood, disregard) The non-self-conformance issue is also a good indication of why the existing restriction already doesn't really hide the complexity.

(Xiaodi Wu) #115

(See other thread.)

(Dave Abrahams) #116

I "loved" this comment when I saw it originally, but now I'm having second thoughts, because an idea I buried in an aside earlier is growing on me. Suppose:

  1. Self-conforming-protocol existentials can be spelled with just the protocol name
  2. Other existentials always need a where clause, which may be “empty:” Collection where _.

I think that might be enough to solve my problem with implicitly-unavailable APIs, and it wouldn't punish many uses of generalized existentials, which will have a where clause anyway. Finally, if we decide this was all a mistake, we can always lift restriction #2.

I realize this doesn't give us an immediate way to fix the technical problems @Joe_Groff is trying to address with this pitch, but it is an eventual future I think I could support. If we can agree that it's a good idea, maybe we can find a reasonable evolution path to get there that includes an immediate fix for the technical problems.


(Joe Groff) #117

If We Could Do It All Over Again, it might've been nice to make the type system a Lisp-2 and put protocols and types in separate namespaces. That would give you the ability to express the "simple existential" idea as a typealias, e.g.:

protocol Runcible {
  mutating func runce()

typealias Runcible = Any<Runcible>

or spell the canonical type eraser for a less trivial protocol:

protocol Collection { ... }

struct Collection<T>: Collection {
  var _value: Any<Collection where .Element == T>

  /* Collection conformance here */

As for where we are today, since "self-conforming-protocol existentials" don't exist now except in limited circumstances, and declaring that self-conformance when desired will necessarily be explicit, making a rule that non-conforming existentials must be spelled differently doesn't really help the immediate source compatibility problems, since non-self-conforming existentials already exist and are spelled without decoration.

(Jordan Rose) #118

I very much hate separate namespaces but you can have the required Any without that.

(Dave Abrahams) #119
  • I agree with @jrose so don't see the point of what you're saying. Is it about the namespaces or the required Any?
  • I recognize my proposal doesn't solve any immediate source-compatibility problems; in fact it creates an eventual source-compatibility problem. The questions I'm trying to ask are:
    1. How does my proposal sound as a place to end up?

      Note that I think Foo where _ does more to call out the unknown types and therefore the possible unavailability of API than Any<Foo> does, and it doesn't burden constrained existentials with the extra Any<…> syntax which AFAICT only benefits the unconstrained existential case.

      Note also that I include solving the technical problems your proposal addresses as part of the goal.

    2. If “good,” then what would the source evolution story be?

    3. What does that story imply about immediate next steps in the language?

(Russ Bishop) #120

Thinking through this over the holiday and reading through the arguments, ultimately I think I come down in favor of @Joe_Groff's pitch. tl;dr: I don't think we should let the perfect be the enemy of the good.

@dabrahams raises a lot of good points but IMHO relaxing the artificial restriction on existentials doesn't change the status quo (we already run into these issues and relaxing the rules solves some real problems). From what I can tell, source compatibility means future versions of the language must solve this problem in a way that isn't seriously constrained by this pitch and the restriction imposes its own set of artificial design constraints that are no worse than the theoretical ones Dave raises (though reasonable people might disagree on that).

This discussion has been enlightening; it certainly raises the desire to tackle the existential problem in Swift 6.

(Life's existential problems are left as an exercise for the reader)