Lifting the "Self or associated type" constraint on existentials

I do agree that we should allow implicit opening of existentials when they're passed as unique generic arguments to a generic function; I agree that will likely cover the majority of places you might want "self-conformance". I don't think banning existentials as function arguments is necessary, or really possible to begin with, since existential types still can be bound to generic type arguments. To work with an Array<Protocol>, Array's generic methods need to be able to work in terms of the Protocol type without boxing and unboxing in the unspecialized case. If existentials have to be syntactically decorated, and we had some Protocol syntax for generic arguments in addition to opaque generic returns, I don't think you need to go out of your way to ban existential arguments, since someone writing foo(x: Protocol) would at that point need to decide whether they mean foo(x: some Protocol) or foo(x: any Protocol).

15 Likes

I very much enjoyed reading what @Karl has to say here, but there's a larger issue with this sort of discussion I'd like to raise.

In this forum, when we talk about things like "box", "witness" and even "existential", we're actually discussing implementation details of the Swift compiler. These are not actual Swift language concepts.

If we can't explain parts of the Swift language (e.g. how protocols work) without appealing to implementation details, then there's something very wrong with our design — or at least our common ability to talk about our design. It leads us down the wrong path of coming up with explanations that we-who-dwell-in-the-forum accept, but which leave less-clued-in or newer users of Swift scratching their heads trying to understand how to get a grip on the language.

We either need to promote some of the implementation concepts to design concepts (which probably means giving them some recognizable syntactical form in the language itself), or adjust the design or documentation to be understandable without implementation details.

For example, "existential" might be something that needs to be promoted to a first class language concept. Other things like "box" and "witness" should probably stay as implementation details, and we should stop trying to explain things to people using them.

Yet other things, like the fact that Self-or-associated-type protocols are effectively different from unconstrained protocols, are probably controversial about whether they're design or implementation, but sorting this out would be a really good thing to do, IMO.

28 Likes

Existential types are described exactly as such in The Swift Programming Language, and are not an implementation detail:

Protocols as Types

Protocols don’t actually implement any functionality themselves. Nonetheless, you can use protocols as a fully fledged types in your code. Using a protocol as a type is sometimes called an existential type , which comes from the phrase “there exists a type T such that T conforms to the protocol”.

The term is also defined in Swift standard library documentation.

7 Likes

FWIW, this is (apparently) the entire discussion of "existential", and its implications are not entirely clear.

The first sentence (of the 3 you quoted) is false: protocols with default implementations do implement functionality.

The second sentence is inadequate: protocols (especially Self/associated-type) are hardly "fully fledged" compared to other types.

The third sentence introduces the term "existential" informally, then does nothing with it.

But the thrust of your point I agree with: existentials are probably something that we regard as part of the language, but I think we could be clearer about what that means. (If we have thread after thread here about existentials, one vaguely worded sentence in the documentation might not be enough for the real world.)

8 Likes

No protocol has default implementations in its definition. Some have default implementations extended to them. (Technically correct is the best kind of correct?)

This may be a distinction without a difference. :slight_smile:

Default implementations are defined in extensions for (basically) historical reasons. There have been multiple threads about allowing default implementations in the original protocol declaration, but no one's come up with a good syntax for that yet.

Conversely, if we extend a struct with new methods or properties, we don't say that the extensions are not really part of the struct.

Anyway, my point was a bit more hand-wavy: this aspect of the documentation is pretty light.

The first sentence is very much true. Protocols do not implement any functionality themselves. This is easy to see and a crucial point which distinguishes them from base classes: if a protocol has no conforming types, then whatever code you write to implement its requirements can never be invoked. On their own, protocols cannot offer any functionality.

The second sentence is very much adequate. Existential types are fully-fledged types just as much as any other non-nominal type (such as tuples). Like any other non-nominal type, they cannot be created using init() syntax or extended; none of this is unique to existential types.

The third sentence is not the end of the section; the remainder of the section demonstrates how to use an existential type in your code.

1 Like

One could very reasonably argue that these limitations are what make existentials, tuples, and functions not “fully-fledged”, and that the documentation is wrong in this regard.

You won't catch me kicking my baby bird out of the nest before its (left: Wing, right: Wing) can conform to the Flyable protocol.

8 Likes

Protocol types are idiosyncratic in that the body of the declaration of the type does not describe the operations that can be performed on variables of that static type. This is pretty weird, and I'm not sure it's what most people expect when they read that protocols are fully fledged types, since no other nominal types work this way.

This idiosyncracy is why I think it would be wise to only allow those protocols to be used as types bare whose bodies' only contain definitions that could be used with all variables of that protocol type, i.e. those without associatedtype, contravariant Self, static or init requirements.

1 Like

What I'm trying to achieve with that idea is to stop existentials being used to write pseudo-generic code, not to hinder real generic code which just happens to be passed a box. I'd be okay with softening that position and simply stripping such functions of their distinctive meaning :innocent: In other words, we would lower read-only existential arguments in to unique generic parameters. inout parameters and return types would remain unchanged and continue to be passed around in boxes.

So the function:

func myFunction(_ x: MyProto, _ y: MyProto)

Would be exactly the same as writing:

func myFunction<T0, T1>(_ x: T0, _ y: T1) where T0: MyProto, T1: MyProto

For these functions, the use of any Self or associated-type requirements becomes a non-issue. The user has chosen not to define the names T0 or T1 in the first style, but they "exist", the compiler sees them, and they could be surfaced in the language as opaque types in lieu of a user-specified name.

This doesn't "ban" existentials being used as function inputs, but it entirely drops their old meaning and replaces it with something else. Instead of describing a function which accepts a box, it becomes a shorthand for a certain signature pattern of a real generic function. This doesn't remove our broader need to revisit the generics syntax; I see it as patching the "expected" meaning in to the existing syntax.


inout parameters are kind of interesting. We can't promote function arguments of type inout MyProto because that would change their meaning in a way that breaks code -- they must stay as mutable boxes.

But they could be unboxed within the function's scope, meaning they could still access Self and associated types:

// What I write:

func myFunction(_ arg: inout RangeReplaceableCollection where Element == Int) {

  if arg.startIndex != arg.endIndex {
    arg.append(42)
  }
  if Bool.random() {
    arg = Array(0..<10)
  }
}

// What it gets lowered as:

// Note: not a generic type. This is still a box.
func myFunction(_ arg: inout RangeReplaceableCollection where Element == Int) {
  // Type of 'arg' fixed for any code here, allowing 'Self', assoc types.
  unboxMutable(&arg) { <T: RangeReplaceableCollection...>(unboxedArg: inout T) in
    if unboxedArg.startIndex != unboxedArg.endIndex {
      unboxedArg.append(42)
    }
  }
  if Bool.random() {
    arg = Array(0..<10) // Type may get reassigned later, though.
  }
  // Type of 'arg' similarly fixed for any code here, allowing 'Self', assoc types.
  // 'T' from the first scope is not meaningful here.
}

This still leaves us in a somewhat-awkward state. Using the "existential spelling" to write your generic functions is sometimes a convenient shorthand, but has this performance-degrading edge case for inout arguments (the inout existential is a little more capable than a generic parameter, but it is no less capable). It's easy to get out of it by introducing an angle-bracketed generic type, but it's yet another thing developers need to know about.

That's why I say we should:

  • Introduce a new spelling for the times you really want boxing (I'd even go for var x: boxed Collection where...).
  • Migrate both old generics and existential-spelling generics to a new, more concise syntax
  • Deprecate the old existential syntax

Which I don't think should introduce any major problems (beyond annoying people that the syntax has changed).

1 Like

It would not be a very useful definition of “fully fledged” in this context to make it mean that it has every feature we could reasonably expect.

Structs and enums can’t have custom subtyping relationships like classes can, even though they’re essential to the usability of types like Optional. Perhaps structs and enums aren’t fully fledged types either?

Classes can’t inherit from multiple superclasses or declare abstract requirements, unlike what’s possible when working with protocol-oriented programming. They can’t even override singly inherited methods from superclasses that rely on default implementations from a protocol. Perhaps classes aren’t fully fledged types either?

Yikes, all of a sudden, Swift has no fully fledged types. This is not a useful description for the didactic purposes of TSPL.

The meaning of the statement can only be this in context: Take all the types you’ve learned about up to this point in the textbook. Existential types can be used in all scenarios where you could use any of those other types.

2 Likes

Right - and that's one of the reasons I don't feel comfortable with the idea of existential self-conformance. It introduces a new conformance for something that's really a language construct - not anything that exists in your model.

Personally, I'm leaning towards favouring the word "box" or "boxed" for existentials. It both describes what the thing does, and makes it clear when you should reach for it (boxing is a familiar concept in most programming languages and environments).

“Existential” does not imply “boxed”, though. @objc and, I believe, AnyObject-constrained existentials are implemented without a box. The behaviour of existentials doesn’t necessarily implement the things I would expect from a “box” either – for instance, putting a struct in an existential doesn’t give it reference semantics (which is probably the most common use of explicit Box types in Swift today). It’s not clear that a box is a helpful conceptual model.

Personally, I find implicit unboxing of existentials very confusing. As a language user, I would prefer to to do this explicitly:

func myFunction(_ arg: inout RangeReplaceableCollection where Element == Int) {
  var <T: RangeReplaceableCollection> unboxedArg = arg
  if unboxedArg.startIndex != unboxedArg.endIndex {
      unboxedArg.append(42)
  }
  arg = unboxedArg
  
  if Bool.random() {
    arg = Array(0..<10) // Type may get reassigned later, though.
  }
}

Actually, I think being able to unbox existentials is more important than being able to access protocol members on the existential itself. With unboxing in the language you can be sure that any possible operation on existential can be implemented in code. While direct access to protocol members is just a syntax sugar.

I (at least) prefer not to add it. It would be noise that adds bulk to the primary definition. And that code would be a lie whenever a conforming type overrides that customization point.

2 Likes

So function declarations in class, struct, and enum definitions are considered noise too correct? Defining a default implementation for a class would be a lie if any type sub classing it overrides it. I'm just trying to play devil's advocate because I don't agree with this sentiment because we already have these "problems".

3 Likes

An implementation in a class method is never a lie; it's always used for instances of that base class itself! (Swift doesn't have abstract classes nor abstract methods for classes.). It's the combination of in-type definition and that said definitions may never be used that's the problem.

1 Like

This was a missing piece, fixed in #34005

3 Likes

Can anybody tell me why we need the syntactic augmentation value: any Protocol over value: Protocol for existentials?

You can find detailed explanation in this pitch: Introduce Any<P> as a better way of writing existentials

3 Likes