# \[Pitch #2\] Implicitly opening existentials

**URL:** <https://forums.swift.org/t/pitch-2-implicitly-opening-existentials/56360>\
**Category:** Pitches\
**Created:** [March 31, 2022, 12:19am UTC](https://forums.swift.org/t/pitch-2-implicitly-opening-existentials/56360 "2022-03-31T00:19:32Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Douglas\_Gregor](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/douglas_gregor/32/1362_2.png) [@Douglas\_Gregor](https://forums.swift.org/u/Douglas_Gregor)\
**Post date:** [March 31, 2022, 12:19am UTC](https://forums.swift.org/t/pitch-2-implicitly-opening-existentials/56360/1 "2022-03-31T00:19:32Z")

</div>

Hey all,

Thanks for the great discussion in the [first pitch](https://forums.swift.org/t/pitch-implicitly-opening-existentials/55412) on implicitly opening existentials. I've come up with a [revised proposal](https://github.com/DougGregor/swift-evolution/blob/implicit-open-existentials/proposals/nnnn-implicit-open-existentials.md) and [implementation to match](https://github.com/apple/swift/pull/41996) based on that discussion.

The proposal itself has grown a lot of detail as I've worked through all of the issues, and narrowed so that the proposed change neither adds new syntax nor affects source compatibility. Rather, it implicitly opens in cases that would have failed to type-check previously, getting us out of the "existential trap" without increasing the effective surface area of the language or changing the semantics of well-formed code.

And after a lot of thinking about it, I don't want to propose an explicit opening syntax. There's more detail in Alternatives Considered now about what an explicit opening syntax would mean for the language, and my conclusion here is that adding such a feature introduces a significant amount of surface-language complexity without a correspondingly significant improvement in expressiveness. The proposal as written is effectively invisible---and that's a benefit we want to keep.

A short change list for this revision:

- Describe contravariant erasure for parameters
- Describe the limitation on implicit existential opening to maintain order of evaluation
- Avoid opening an existential argument when the existential type already satisfies the conformance requirements of the corresponding generic parameter, to better maintain source compatibility
- Introduce `as any P` and `as? any P` as syntaxes to suppress the implicit opening of an existential value.
- Added discussion on the relationship with `some` parameters ([SE-0341](https://github.com/apple/swift-evolution/blob/main/proposals/0341-opaque-parameters.md)).
- Expand discussion of an explicit opening syntax.

Please give it a read-through or try out the toolchain ([macOS](https://ci.swift.org/job/swift-PR-toolchain-macos/120/artifact/branch-main/swift-PR-41996-120-osx.tar.gz)). I'd love to hear your thoughts.

Doug

---

<div class="post-metadata">

**Author:** ![xwu](https://avatars.discourse-cdn.com/v4/letter/x/b487fb/32.png) [@xwu](https://forums.swift.org/u/xwu)\
**Post date:** [March 31, 2022, 1:26am UTC](https://forums.swift.org/t/pitch-2-implicitly-opening-existentials/56360/2 "2022-03-31T01:26:02Z")

</div>

Overall, it seems reasonable to me.

I get the preference for a proposal that adds (almost) no new syntax. However, in introducing `as any P`, the lack of support for its dual `as some P` is only accentuated, so if the latter is not to be supported I'd think there'd need to be some good diagnostics tailored to that.

As you note, there is an expressivity gain within a single function when one has an explicit existential opening syntax, even as it would have to restricted not to 'escape' the dynamic type; to my mind, this would still be a nice-to-have, and I do wonder about the stated goal of not exposing "the notion of opened types" to the end user at all:

I am not sure how one might explain what's going on to the end user without invoking this concept while not muddying the distinction between `some P` and `any P`. If naming this operation (and, even, having an explicit syntax for it) can help users understand the behavior they observe and the rationale for its restrictions, then taking the approach of not exposing it might paint us into a corner much as was the case when the language formerly did not expose the distinction between the protocol `P` and the existential type `P`, etc.

---

<div class="post-metadata">

**Author:** ![filip-sakel](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/filip-sakel/32/8697_2.png) [@filip-sakel](https://forums.swift.org/u/filip-sakel)\
**Post date:** [March 31, 2022, 4:13am UTC](https://forums.swift.org/t/pitch-2-implicitly-opening-existentials/56360/3 "2022-03-31T04:13:37Z")

</div>

Thank you for the revised and comprehensive revision of the initial pitch!

One thing I still have trouble understanding is how ‘type(of:)’ would stop relying on special type-checking rules. This function currently takes a parameter of type , which has no constraints. So according to my comprehension of _Avoid opening when the existential type satisfies requirements_ that should result in the existential itself being passed to the function.

---

<div class="post-metadata">

**Author:** ![Douglas\_Gregor](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/douglas_gregor/32/1362_2.png) [@Douglas\_Gregor](https://forums.swift.org/u/Douglas_Gregor)\
**Post date:** [March 31, 2022, 4:41am UTC](https://forums.swift.org/t/pitch-2-implicitly-opening-existentials/56360/4 "2022-03-31T04:41:11Z")

</div>

> [@xwu](#):
>
> I get the preference for a proposal that adds (almost) no new syntax. However, in introducing `as any P` , the lack of support for its dual `as some P` is only accentuated, so if the latter is not to be supported I'd think there'd need to be some good diagnostics tailored to that.

That's a good point. You've also made we think that we don't need the `as any P` syntax at all. I added it because we wanted a way to suppress the implicit opening for cases where we want to pass the existential box. In the course of my implementation work on this, I needed to add `as any P` in just a handful of places to keep operating on the box. But now that we have the check to only implicitly open when the existential type doesn't fulfill the generic requirements, all of those `as any P`'s I added are unnecessary. I think with the latest rules `as any P` can only either break code (`any P` does not conform to `P`) or push around overload resolution in very rare circumstances. Those don't seem useful enough to specify this capability, especially given the potential for confusion you cite.

[EDIT: I've updated the proposal to remove this special syntax]

> [@xwu](#):
>
> I am not sure how one might explain what's going on to the end user without invoking this concept while not muddying the distinction between `some P` and `any P` . If naming this operation (and, even, having an explicit syntax for it) can help users understand the behavior they observe and the rationale for its restrictions, then taking the approach of not exposing it might paint us into a corner much as was the case when the language formerly did not expose the distinction between the protocol `P` and the existential type `P` , etc.

The best short explanation I have is "if the box doesn't meet the requirements, we reach into the box to pull out the underlying concrete type that does meet the requirements".

If we wanted to add the explicit opening syntax later, we could: then this feature becomes an inference rule that takes something like:

```swift
protocol P {
  associatedtype A: Q
}

func getA<T: P>(_: T) -> T.A { ... }

```

and interprets a call `getA(anyP)` as `getA(anyP as some P) as any Q`.

Even if we don't have the syntax in the language, we could use it for exposition the way we talk about "reverse generics" as being an explanation for opaque result types, despite us not necessarily ever wanting to add such a feature to the language.

> [@filip-sakel](#):
>
> One thing I still have trouble understanding is how ‘type(of:)’ would stop relying on special type-checking rules. This function currently takes a parameter of type , which has no constraints. So according to my comprehension of _Avoid opening when the existential type satisfies requirements_ that should result in the existential itself being passed to the function.

Ah, you're right, I think we lost `type(of:)` along the way. I think you were trying to tell me that on the pull request and I misunderstood, sorry. I'm okay with that trade-off: this new source-compatible formulation is a much better way to address this gap, and `type(of:)` remaining special is reasonable in my opinion.

[EDIT: I've updated the proposal to eliminate the comment about subsuming the special `type(of:)` behavior]

Doug

---

<div class="post-metadata">

**Author:** ![cocoaphony](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/cocoaphony/32/3872_2.png) [@cocoaphony](https://forums.swift.org/u/cocoaphony)\
**Post date:** [March 31, 2022, 1:46pm UTC](https://forums.swift.org/t/pitch-2-implicitly-opening-existentials/56360/5 "2022-03-31T13:46:51Z")

</div>

Dropped word in the first paragraph (that leaves me unclear about what you mean):

> that once you have a value of existential type it is _very_ hard to then use any generic operations a value of that type

Is this supposed to be " **on** a value of that type?"

---

<div class="post-metadata">

**Author:** ![Jumhyn](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/jumhyn/32/3891_2.png) [@Jumhyn](https://forums.swift.org/u/Jumhyn)\
**Post date:** [March 31, 2022, 2:35pm UTC](https://forums.swift.org/t/pitch-2-implicitly-opening-existentials/56360/6 "2022-03-31T14:35:41Z")

</div>

> [@Douglas\_Gregor](#):
>
> I think with the latest rules `as any P` can only either break code ( `any P` does not conform to `P` ) or push around overload resolution in very rare circumstances. Those don't seem useful enough to specify this capability, especially given the potential for confusion you cite.

I think I agree with this analysis. The "only open when existential doesn't satisfy constraints" rule solves the issues I was thinking of that had me advocate for `as any P` initially. It would still be _nice_ to have `as some P` (and it feels to me like it would a fairly natural part of this proposal).

I agree @xwu's intuition here regarding explicit syntax for this operation:

> [@xwu](#):
>
> If naming this operation (and, even, having an explicit syntax for it) can help users understand the behavior they observe and the rationale for its restrictions

---

<div class="post-metadata">

**Author:** ![filip-sakel](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/filip-sakel/32/8697_2.png) [@filip-sakel](https://forums.swift.org/u/filip-sakel)\
**Post date:** [March 31, 2022, 4:23pm UTC](https://forums.swift.org/t/pitch-2-implicitly-opening-existentials/56360/7 "2022-03-31T16:23:20Z")

</div>

> [@Douglas\_Gregor](#):
>
> Ah, you're right, I think we lost `type(of:)` along the way. I think you were trying to tell me that on the pull request and I misunderstood, sorry. I'm okay with that trade-off: this new source-compatible formulation is a much better way to address this gap, and `type(of:)` remaining special is reasonable in my opinion.

I guess this also means that `_openExistential` can't be subsumed yet, since a lot of use cases don't place constraints on the opened type.

---

<div class="post-metadata">

**Author:** ![Douglas\_Gregor](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/douglas_gregor/32/1362_2.png) [@Douglas\_Gregor](https://forums.swift.org/u/Douglas_Gregor)\
**Post date:** [March 31, 2022, 5:13pm UTC](https://forums.swift.org/t/pitch-2-implicitly-opening-existentials/56360/8 "2022-03-31T17:13:35Z")

</div>

> [@cocoaphony](#):
>
> Dropped word in the first paragraph (that leaves me unclear about what you mean):
> 
> > that once you have a value of existential type it is _very_ hard to then use any generic operations a value of that type
> 
> Is this supposed to be " **on** a value of that type?"

How's this?

> However, a fundamental issue with existential types remains, that once you have a value of existential type it is _very_ hard to use generics with it.

Doug

---

<div class="post-metadata">

**Author:** ![ksluder](https://avatars.discourse-cdn.com/v4/letter/k/eada6e/32.png) [@ksluder](https://forums.swift.org/u/ksluder)\
**Post date:** [March 31, 2022, 5:20pm UTC](https://forums.swift.org/t/pitch-2-implicitly-opening-existentials/56360/9 "2022-03-31T17:20:25Z")

</div>

It seems strange that obviating `type(of:)` and `_openExistential()` are not considered requirements for a feature about opening existentials.

---

<div class="post-metadata">

**Author:** ![cocoaphony](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/cocoaphony/32/3872_2.png) [@cocoaphony](https://forums.swift.org/u/cocoaphony)\
**Post date:** [March 31, 2022, 5:22pm UTC](https://forums.swift.org/t/pitch-2-implicitly-opening-existentials/56360/10 "2022-03-31T17:22:38Z")

</div>

Perfect.

I think the pitch is good. If I have a concern, it's more broadly that I'm not clear how "The Swift Programming Language" will be amended to make all of these things clear. If you already know Swift well, these changes follow along quite naturally. But if you're coming fresh, I struggle with how to teach folks why it's sometimes "any" and sometimes "some" and what the "correct" way to write Costume is. In particular, the pitch feels like it's often in terms of "you did it one way, and don't want all the trouble to refactor it." But does that suggest that "any Costume" was a mistake and this is a hack around that? I don't think that's true; as the Array example shows. But I'm left wondering what to tell students. (I'm on record teaching devs to pass existentials rather than generics when in doubt because I thought it was simpler to understand and had fewer weird edge cases, but I think I was likely wrong about that.)

I'm mostly muttering here. I think this pitch is good and definitely an improvement. Thanks for all the work.

---

<div class="post-metadata">

**Author:** ![Joe\_Groff](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/joe_groff/32/1929_2.png) [@Joe\_Groff](https://forums.swift.org/u/Joe_Groff)\
**Post date:** [March 31, 2022, 5:53pm UTC](https://forums.swift.org/t/pitch-2-implicitly-opening-existentials/56360/11 "2022-03-31T17:53:15Z")

</div>

> [@ksluder](#):
>
> It seems strange that obviating `type(of:)` and `_openExistential()` are not considered requirements for a feature about opening existentials.

Yeah, that's concerning to me as well. Perfect compatibility with existing source is a worthy goal; however, if compatibility weren't a concern, my sense is that it would usually be preferable to open an existential and pass the underlying value whenever possible, even if the existential could be passed as-is. It will usually be more efficient, because a Swift existential that conforms to its own protocol would do so by trampolining through another layer of witness table, and passing a self-conforming existential opens up the possibility of creating multi-wrapped existentials, which are mostly invisible but create multiple levels of boxing. (Or at least, they're supposed to be mostly invisible, but currently you can only practically create them using `Any`; it will be exciting to find out how well the runtime deals with other multi-wrapped existential boxes once we expose it to them in practice.) Not opening existentials when possible would also turn adding "self conformance" to a protocol into an evolution hazard, because adding self conformance would either break or add invisible overhead to existing code that passes the existential as a generic argument expecting it to be opened. Overall, I think that favoring opening is the better option. If doing so creates unacceptable practical source breaks in Swift 5, then I think we should still change the behavior in Swift 6.

---

<div class="post-metadata">

**Author:** ![filip-sakel](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/filip-sakel/32/8697_2.png) [@filip-sakel](https://forums.swift.org/u/filip-sakel)\
**Post date:** [March 31, 2022, 7:56pm UTC](https://forums.swift.org/t/pitch-2-implicitly-opening-existentials/56360/12 "2022-03-31T19:56:17Z")

</div>

What kind of performance improvement are we talking about? If these improvements are insubstantial, even in large codebases, I think we should favor source compatibility. Although this source-breaking change would be subtle, and thus rarely relied upon, its subtlety could also incur difficult debugging experiences and nasty bugs. I'm just worried that in our effort to improve Swift by correcting the mistakes of the past, such as changing `P` to `any P`, we may risk agitating users without a tangible benefit. There's already been backlash about the warning for marking existentials with `any`. I bet that if implicit opening changes in Swift 6 and libraries become slightly buggier, as a result, users will not think, "at least Swift is slightly faster"; Swift's already fast. Of course, I'm exaggerating a bit and I understand that in server applications, for example, small gains could add up to big savings, but I think we should maintain a high bar for source-breaking changes.

---

<div class="post-metadata">

**Author:** ![Joe\_Groff](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/joe_groff/32/1929_2.png) [@Joe\_Groff](https://forums.swift.org/u/Joe_Groff)\
**Post date:** [March 31, 2022, 8:45pm UTC](https://forums.swift.org/t/pitch-2-implicitly-opening-existentials/56360/13 "2022-03-31T20:45:36Z")

</div>

The primary overhead using the value will be double-indirection—the generic function will interact with the existential box as a generic type, whose implementation of the value witness operations and protocol methods all have to re-dispatch through the corresponding operations on the dynamic type inside the existential. If the existential-as-generic value gets used to create a new existential value somewhere, there will be an additional allocation to copy the existential behind the generic into an existential box. You're right that neither of these overheads are probably a big deal for typical application code that already accepts these costs. Of the concerns I raised, I'm more concerned about the library evolution angle; whether a combination of existential constraints is "self-conforming" or not isn't necessarily a fixed property of the constraints involved, and the subtlety issue you raised cuts both ways—a seemingly innocuous change in one place has unpredictable effects elsewhere in the code.

Although this is a subtle situation for humans to discern, it should be straightforward for the compiler to know where the behavior would change from this proposal. If we were going to gate the behavior change on a language version, then it seems like a migration tool could flag places where the behavior would change, allowing the developer to adapt their code.

---

<div class="post-metadata">

**Author:** ![Douglas\_Gregor](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/douglas_gregor/32/1362_2.png) [@Douglas\_Gregor](https://forums.swift.org/u/Douglas_Gregor)\
**Post date:** [March 31, 2022, 8:59pm UTC](https://forums.swift.org/t/pitch-2-implicitly-opening-existentials/56360/14 "2022-03-31T20:59:49Z")

</div>

> [@Joe\_Groff](#):
>
> Overall, I think that favoring opening is the better option. If doing so creates unacceptable practical source breaks in Swift 5, then I think we should still change the behavior in Swift 6.

Personally, I think that changing the runtime behavior implicitly in Swift 5.x is unacceptable in practice. I agree with you that I would favor opening if source compatibility wasn't a concern. Swift 6 would be an opportunity for such a change, and it feels reasonable to do at that point.

I'll integrate this discussion into the proposal. Thank you!

Doug

---

<div class="post-metadata">

**Author:** ![Joe\_Groff](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/joe_groff/32/1929_2.png) [@Joe\_Groff](https://forums.swift.org/u/Joe_Groff)\
**Post date:** [March 31, 2022, 9:04pm UTC](https://forums.swift.org/t/pitch-2-implicitly-opening-existentials/56360/15 "2022-03-31T21:04:39Z")

</div>

> [@Douglas\_Gregor](#):
>
> Personally, I think that changing the runtime behavior implicitly in Swift 5.x is unacceptable in practice. I agree with you that I would favor opening if source compatibility wasn't a concern. Swift 6 would be an opportunity for such a change, and it feels reasonable to do at that point.
> 
> I'll integrate this discussion into the proposal. Thank you!

That sounds like a reasonable compromise to me too, thanks Doug! To maintain Swift 5 compatibility, while also avoiding the problem of API evolution potentially making protocols newly self-conforming in the future, maybe we could limit the non-opening behavior to strictly the cases that are self-conforming today (`Any`, `Error`, and `@objc` protocols) and say that future native-Swift protocol types that gain self conformance in the future still implicitly open as generic arguments in both language modes. It's a weird rule in isolation admittedly, but as a compatibility concession, it strikes a balance that shouldn't break any code today, and will prevent future evolution of the language and standard library from breaking code again in the future.

---

<div class="post-metadata">

**Author:** ![Nobody1707](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/nobody1707/32/2169_2.png) [@Nobody1707](https://forums.swift.org/u/Nobody1707)\
**Post date:** [March 31, 2022, 9:29pm UTC](https://forums.swift.org/t/pitch-2-implicitly-opening-existentials/56360/16 "2022-03-31T21:29:43Z")

</div>

This looks good to me. I probably also support Joe\_Groff's special casing of non-opening behavior to only affect types that already self-conform.

---

<div class="post-metadata">

**Author:** ![Douglas\_Gregor](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/douglas_gregor/32/1362_2.png) [@Douglas\_Gregor](https://forums.swift.org/u/Douglas_Gregor)\
**Post date:** [April 4, 2022, 6:02am UTC](https://forums.swift.org/t/pitch-2-implicitly-opening-existentials/56360/17 "2022-04-04T06:02:48Z")

</div>

> [@Joe\_Groff](#):
>
> That sounds like a reasonable compromise to me too, thanks Doug! To maintain Swift 5 compatibility, while also avoiding the problem of API evolution potentially making protocols newly self-conforming in the future, maybe we could limit the non-opening behavior to strictly the cases that are self-conforming today ( `Any` , `Error` , and `@objc` protocols) and say that future native-Swift protocol types that gain self conformance in the future still implicitly open as generic arguments in both language modes. It's a weird rule in isolation admittedly, but as a compatibility concession, it strikes a balance that shouldn't break any code today, and will prevent future evolution of the language and standard library from breaking code again in the future.

I went ahead and [revised the proposal](https://github.com/DougGregor/swift-evolution/blob/implicit-open-existentials/proposals/nnnn-implicit-open-existentials.md) to incorporate all of this feedback. A brief summary of the changes:

- Only apply the source-compatibility rule, which avoids opening an existential argument when the existential box would have sufficed, in Swift 5. In Swift 6, we will open the existential argument whenever we can, providing a consistent and desirable semantics.
- Re-introduce `as any P` and `as! any P` , now that they will be useful in Swift 6.
- Clarify more about the relationship to the explicit opening syntax, which could also be a future direction.

At this point, I think it's a good time to bring this up for a wider review.

Doug

---

<div class="post-metadata">

**Author:** ![David\_Catmull](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/david_catmull/32/1007_2.png) [@David\_Catmull](https://forums.swift.org/u/David_Catmull)\
**Post date:** [April 4, 2022, 3:38pm UTC](https://forums.swift.org/t/pitch-2-implicitly-opening-existentials/56360/18 "2022-04-04T15:38:49Z")

</div>

A question - in the example `Costume.hasBellsMember`, what is the type of `C` in the call to `hasBells()`? Does that getter function get instantiated with a different type for every type that conforms to `C`?

---

<div class="post-metadata">

**Author:** ![Douglas\_Gregor](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/douglas_gregor/32/1362_2.png) [@Douglas\_Gregor](https://forums.swift.org/u/Douglas_Gregor)\
**Post date:** [April 6, 2022, 5:31am UTC](https://forums.swift.org/t/pitch-2-implicitly-opening-existentials/56360/19 "2022-04-06T05:31:16Z")

</div>

> [@David\_Catmull](#):
>
> A question - in the example `Costume.hasBellsMember` , what is the type of `C` in the call to `hasBells()` ? Does that getter function get instantiated with a different type for every type that conforms to `C` ?

It's the `Self` type of the protocol, which will end up pointing at whatever the type is---be it `Int` or `String` or whatever---that conforms to the `Costume` protocol. Swift doesn't use instantiation as the language model; there's one implementation of `hasBells()` that dynamically gets provided with type information. That type will always refer to a concrete type that conforms to `Costume`.

(FYI, the [review](https://forums.swift.org/t/se-0352-implicitly-opened-existentials/56557) of this proposal just kicked off, so I'll answer questions over there as well)

Doug

---

<div class="post-metadata">

**Author:** ![tourultimate](https://avatars.discourse-cdn.com/v4/letter/t/85f322/32.png) [@tourultimate](https://forums.swift.org/u/tourultimate)\
**Post date:** [April 6, 2022, 3:58pm UTC](https://forums.swift.org/t/pitch-2-implicitly-opening-existentials/56360/20 "2022-04-06T15:58:09Z")

</div>

Noticed a few instances of what seem to me to be typos:

> where want to "open code" the check for bells

needs a "we" after "where"?

> putting the logic into a separatae generic function

separatae -\> separate

> The After the call, any values described in terms of that dynamic type opened...

"Then after"?

> ... described in terms of that dynamic type opened existential type has to be type-erased back to an existential

Not quite sure how to fix this sentence, but something in the grammar seems off

> This affects parameters of function type that reference...

Missing a "the" or maybe type needs to be plural, not sure.
