SE-0352: Implicitly Opened Existentials

My belated review:

I’m a +1, and delighted to see this longstanding language gap filled at last. Still, it seems to me this proposal is necessary but not sufficient, so I hope the work does not end here.

Several things about the proposal originally gave me pause:

  1. It’s a bit…magic-y. Is it too magic? Will people be surprised when they accidentally encounter the feature? Does the magic introduce undesired behavior?
  2. Forcing a new function to open an existential limits refactoring options / expressivity, and doesn’t compose nicely with other language features. Couldn’t we do some lexically scoped version of this?
  3. Is the feature discoverable? When people need to unwrap an existential, how do they realize they need to pull the code that works with the concrete type into its own function? That is a curious leap.

Reading the proposal and the discussion, and doing my own fumbling experiments, I’m convinced on point 1: yes, it is a bit magical to have a type invisibly transform like this from caller to callee, but the behavior of that magic (1) is almost always going to be what people actually wanted, to the extent they won’t notice it’s magic, and (2) the magic almost never shadows other behavior people would want instead. To the extent it’s magical, it’s also desirable and intuitive.


Points 2 and 3 still stick in my craw. It is a basic building block of Alogol’s many descendants that it is generally possible to inline a function in source, as it were:

a
f(…)
d                             a
                —becomes→     b
func f(…) {                   c
    b                         d
    c
}

Under this proposal, a function that opens an existential is no longer thus inlinable. By forcing the creation of new functions that may interrupt the logic flow of code and may not have a meaningful name, this proposal will thus create a scattering of fragmentary functions. The situation is akin to how Objective-C’s target/selector approach to callbacks could leave classes littered with tiny methods that represented the tail ends of thoughts started elsewhere, much like reading a Choose Your Own Adventure book straight through out of order.

Swift’s nested functions mitigate this problem, though the resulting dance is hardly ideal:

    let foo: any P
    ...
    openFoo(foo)
    func openFoo(_ foo: some P) {
        ...
    }

Worried about all that, this sentence from the proposal put me at peace with its approach as a good starting point:

Opening an existential means looking into the existential box to find the dynamic type stored within the box, then giving a "name" to that dynamic type. That dynamic type name needs to be captured in a generic parameter somewhere, so it can be reasoned about statically, and the value with that type can be passed along to the generic function being called.

Yes, right, we need a way to name that type that resulted from the opening, or the opening isn’t worth much — and right now, Swift can only introduce generic types at function or type boundaries. Any sort of solution that offers the inlining I’m looking for necessitates a large increase in language surface area, whereas the proposal’s answer makes an urgently needed feature available with minimal impact. I’m convinced. +1, good proposal, yes, do it.


I do hope we don’t stop there, however. This proposal leaves a gap in the shape of the undocumented _openExistential. (Aside: I think it is not possible to fully recreate the behavior of _openExistential for a generic protocol P using this proposal, even in Swift 6 mode, because there is no way to specify type parameters <P, T:P> in Swift. Is that correct?)

Edit: I misremembered _openExistential as accepting a closure. I was picturing this nonexistent beast:

let foo: any Widget
...
openExistential(foo) { fooOpened in
  ...
}

…Or better yet, I like the alternative/future direction the proposal mentions of allowing opening via coercion from any P to some P. The introduction of opaque result types introduced the problem of types that are unnameable, which IIRC greatly vexed both @Chris_Lattner3 and me during the review of that proposal. Being able to refer to a specific some type would open up this future direction:

let foo: any Widget  // where P has associated type Doodad
...
let fooOpened: some Widget = foo
let bar: TypeOf(fooOpened).Doodad = fooOpened.doodad  // bad hypothetical syntax

…and this would grant my “descendants of Algol” wish from above. I do hope we head in that direction eventually.

6 Likes

That’s what @Joe_Groff said:

Thanks for the clarifications @Joe_Groff. I’m really excited for how this sets up Swift 6 to have less magic.

There does appear to be a version of _openExistential that accepts function types. It is used in swift-corelibs-foundation to implement AttributedString.

func openLHS<LHS>(_ lhs: LHS) -> Bool {
    if let rhs = rhs as? LHS {
        return CheckEqualityIfEquatable(lhs, rhs).attemptAction() ?? false
    } else {
        return false
    }
}
return _openExistential(lhs, do: openLHS)

Right, but the second argument has to actually be a function; it crashes if you pass it a closure.

Works:

func f(foo: P) {
    _openExistential(foo, do: g)
}

func g<T: P>(openedFoo: T) {
    print(openedFoo)
}

Crashes the compiler:

func f(foo: P) {
    _openExistential(foo) { openedFoo in
        print(foo)
    }
}

Without having investigated, I assume this is because closures in Swift can’t be generic, only named functions can.

(And the closure is necessary to grant my wish of having openedFoo inside the lexical scope of foo, the capability around which I’m arguing future work is necessary. Nested functions work, as per my post, but are awfully awkward.)

3 Likes

The key difference between accepting functions and accepting a closure is that a function can have type parameters which the implementation of _openExistential can bind the underlying type to.

Effectively, the Swift 6 behavior described in this pitch is like if all calls to generic functions* were made via _openExistential.

* all calls to generic functions which accept only one type parameter and exactly one formal argument, where the type of the formal argument is equal to the type parameter

2 Likes

Ah, got it.

Ah. Thank you!

The policy of thwarting calls when covariant erasure would drop constraints that could be expressed in the future makes a lot more sense now that I understand the cause of a potential source break. I like the idea of requiring an explicit coercion rather than rejecting the call altogether. Making use of contextual type information (which is what @ensan-hcl initially suggested) is not always possible, as in a reference to an overloaded function that is applied to a parameter of erased function type:

func foo(_: any Collection)
func foo(_: any Collection<Int>)

protocol P {
  associatedtype C: Collection<Int>

  func doSomething(fn: (C) -> Void)
}

let p: P

// With today's erasure, p.doSomething
// has type '((any Collection) -> Void) -> Void'.
p.doSomething(fn: foo)

Though I didn't have any confusion about as any P as suppression mechanism, as I thought about it more and more, I became more and more confused.

let x = PX() as any P

This code introduces a constant x whose type is any P. It's fine.

func foo<T: P>(_ value: T) {}
foo(x)

When user call foo with x, as said in proposal, x is implicitly opened and a value of PX is passed into foo. It's of course fine too. Also, x as any P is totally equal to x in terms of both value and type. Therefore, the following call should be equal to foo(x)

foo(x as any P)

However, this code, works unexpectedly. It would cause a compile error because implicit opening doesn't happen.
What on earth is going on? I cannot come up convincing explanation for this behavior.

1 Like

I’ve verified that yes, in the implementation provided above, this does happen. Causing further confusion, any intervening expression between the as any P and the function call makes the error disappear again:

protocol P { }
struct S: P { }
let x = S() as any P
func foo<T: P>(_ value: T) {}

foo(x)                          // ✅
foo(x as any P)                 // ❌ compile error
foo((x as any P))               // ❌ parens aren’t enough, but…
foo((x as any P, "ignored").0)  // ✅ …this works

// Or even…
extension P {
    func itself() -> Self { self }
}
foo((x as any P).itself())      // ✅

I’m not delighted with this special behavior of as when it appears in one specific position. However, reading through the discussion and playing with examples, I’m hard pressed to think of a realistic case where somebody would type as any P for a value whose type is already any P and have some intent other than “please don’t unwrap.”

3 Likes

We have somewhat similar behavior today with regards to e.g., coercion from Optional to Any:

func f(_ x: Any) {}
let x: Int? = 0

f(x) // warning: expression implicitly coerced from 'Int?' to 'Any'

where the fix is to write:

f(x as Any)

Under the simplest model for as that @ensan-hcl discusses, as Any for an argument already passed as Any should be meaningless. But as is imbued with an additional meaning which is something akin to "I want the result of this expression to really be type T." This meaning is only available in specific contexts (I'm not sure if it appears anywhere else besides the Optional-to-Any warning fix?) but I don't think it's an unreasonable meaning for as. I'd prefer this use of as to coming up with a whole new syntax for expressing the idea of "use this type exactly without implicit coercions."

7 Likes

as is useful for influencing type deduction. The simplest case is let x = 12 as Float, but you can also use it to e.g. control which override is taken.

as and is are both keywords… how about x as is? :rofl:

9 Likes

This is a good point, and there are deeper parallels here between unwrapping optionals and unwrapping…er…opening existentials: unwrapping an optional requires a new variable for the unwrapped value (with some sugar like ?? and ?. that can hide it); unwrapping/opening an existential requires a new type variable for the unwrapped type (with some sugar that can hide it). So the precedent does carry some weight.

1 Like

Sure, and as is also used for things like disambiguating overloads and bridging conversions with Objective-C types. But I agree with @ensan-hcl that there's something a little odd about using and as coercion on an expression which would (ostensibly) already be coerced to the type in question implicitly. And I also agree with @Paul_Cantrell that I can't think of a reasonable meaning for such a use of as other than "I really do mean to use this type, it's not a mistake and please don't change it on me."

Also worth noting: my comparison of as any P with the existing as Any isn't perfect, because in the as Any case it's just requiring the user to be explicit in source about a coercion that will happen the same way with or without the as, whereas with as any P we're explicitly suppressing a coercion that the compiler would perform without our guidance to the contrary. But the two still feel closely linked in my mind, enough so that I'm not troubled by the reuse of as for this purpose.

3 Likes

The oddness seems desirable, since it pushes the programmer towards the default implicit opening behavior.

Isn’t this what’s being pitched? f(x as any P) will stop the implicit conversion from any P to <T: P>.

1 Like

Yes, I'm arguing in favor of the proposed behavior despite (or as you note, perhaps because of) the oddities noted by @ensan-hcl. :slightly_smiling_face:

Thanks for reviewing this proposal, everyone! The core team has decided to return it for revision.

1 Like

3 posts were merged into an existing topic: SE-0352 (second review): Implicitly Opened Existentials

Here is the second review thread:

@ensan-hcl, if you like, I can ask the moderators to move your post there. (Or anyone else, for that matter.)

Oh, please :pray:

1 Like