Lifting the "Self or associated type" constraint on existentials

Evolution Pitches
42

I understand this to mean: currently we may already have to prevent some parts of a protocol's interface (the part provided by extension methods that use Self arguments) from being used on existentials:

$ swift
Welcome to Apple Swift version 4.2 (swiftlang-1000.11.113 clang-1000.11.45.5). Type :help for assistance.
  1> protocol P {}
  2> extension P { func f(_: Self) {} }
  3> func g<T:P>(_ x: T) { x.f(x) } // OK
  4> func h(_ x: P) { x.f(x) }
error: repl.swift:4:18: error: member 'f' cannot be used on value of protocol type 'P'; use a generic constraint instead
func h(_ x: P) { x.f(x) }
                 ^ ~

There are other ways to address this issue (see below).

Regardless, I have some general concerns about going this way and a special concern about doing it right now.

My general concerns are:

  • Our users may not sufficiently understand the difference between using type erasure and using type constraints. Right now the restrictions on existentials tend to prevent unintentional type erasure early on. Because using an existential is syntactically lightweight when compared to using type constraints, and similar to using a base class (which is likely familiar to more people), I fear that with less restrictions on existentials, users will code themselves into inappropriate type erasure by following the path of initial least resistance, and then find themselves “forced” to add complexity in the form of dynamic checks at the points where they need to recover the type information. Also, this premature type erasure will inevitably hide optimization opportunities from the compiler.
  • No matter how general we make our support for existentials, if we only lift restrictions there will always be the fundamental “weirdness” illustrated by the interactive session above: some parts of a protocol's declared interface can be unavailable on instances of the protocol type, with the inevitable consequence that an instance of an arbitrary protocol P will never conform to P. As far as I can tell, that's just really hard to explain to people in a way that makes sense. Right now the issue is something of an edge case but if we make it easier to write code that uses existentials I fear it will become a much bigger problem as people develop a significant investment in large bodies of code that now satisfy the compiler. If we push our support for generalized existentials to the limit, almost everything will seem to work: most things you could do with type constraints could also be done smoothly with type erasure, but there will be these very odd and even-harder-to-explain cases where the compiler stops you. Also, there will be a pervasive performance cost when type erasure is used.

My concern with trying to generalize existentials now is that I expect the addition of opaque result types to satisfy the majority of needs for which people want to use existentials today, and if we do both things at once:

  • people will be confused about which feature to use.
  • they will be drawn toward the syntactically lightweight use, which I think is more often inappropriate.
  • we will not get good data on which of the new features is best able to satisfy our users' needs.

I have a couple of ideas that may help address some of these problems:

  • Spelling all existentials as Any<SomeProtocol> would go some way toward addressing the problem of “least resistance”—though I'm not at all sure it's far enough.
  • I have always thought a big part of our problem is that protocols that are meant to be used for type erasure are fundamentally different from those meant to be used as constraints, yet we declare them the same way. Therefore, the compiler simply has to let us create situations like the one shown at the top, where we extend them with Self arguments or have static requirements that prevent self-conformance. Even when we develop better tools for library evolution, the compiler will have to let us evolve a protocol by adding a defaulted associated type even if that protocol is being used as an existential in some other module—because it can't see the other module. If, in order to be used as an existential, a protocol had to be so annotated:
    existential // strawman syntax
protocol P {}
    The compiler could prevent the use of Self arguments in extensions, the declaration of static requirements, and the addition of associated types.

Even if both these ideas were implemented, though, I'd still be concerned about doing this within 12 months of adding opaque result types, for reasons stated above.

16 Likes