SE-0352: Implicitly Opened Existentials

Hi everyone. The review of SE-0352: Implicitly Opened Existentials begins now and runs through April 18th, 2022.

Reviews are an important part of the Swift evolution process. All review feedback should be either on this forum thread or, if you would like to keep your feedback private, directly to me as the review manager by email or by direct message. When emailing directly, please keep the proposal link at the top of the message.

What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

  • What is your evaluation of the proposal?
  • Is the problem being addressed significant enough to warrant a change to Swift?
  • Does this proposal fit well with the feel and direction of Swift?
  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

More information about the Swift evolution process is available at

https://github.com/apple/swift-evolution/blob/master/process.md

Thank you,

Joe Groff
Review Manager

17 Likes

I have two questions. I'm sorry if these questions were raised in pitch threads.

First, is cannotOpen(p1, p1) possible? I think it should not be possible, but I'd like to check my understanding.

func cannotOpen<T: P>(_ a: T, _ b: T) { ... }

func testCannotOpenMultiple(p1: any P) {
  cannotOpen(p1, p1)
}

Second, how does the feature work in this case? Because of the lack of expression, there are some constraints that cannot be explicitly written. How such constraints are treated?

protocol P {
  associatedtype A: Collection where Index == Int
  func getA() -> A
}

func getA<T: P>(_ value: T) -> T.A {
  return value.getA()
}

func foo(value: any P) {
    // here, what is the type of `a`?
    // if the result is type-erased to its upper bounds, it should be any Collection<.Index == Int>, but how the constraint can be expressed?
    let a = getA(value)
}

No, opening is not possible here, because you wouldn't get a consistent T: each place where we opened an existential effectively creates a type that is unique to that opening operation.

The "upper bound" here is currently defined to be any Collection, because the language can't currently express anything tighter than that. If existentials grow the ability to handle constraints (e.g., the constrained existentials from this pitch covers most of the cases, although not your specific one), the upper bound could be tightened.

Doug

Would tightening the upper bound become a potential source break if we attempt it in the future?

2 Likes

Yes, it would. We have options here:

  1. We could disable opening if the resulting type erasure would lose information, so code that might break at some later time due to the upper bound changing would remain ill-formed now.
  2. We could accept that generalization of existentials later on might not imply that we can change the upper bound. At worst, it would mean delaying the change to the upper bound until a major language version bump.

For me, the answer depends on the constrained existentials pitch: if that pitch succeeds, so we can express any Collection<String> and use it as the upper bound, I'm fine with (2). If we cannot get constrained existentials, I would want to go with (1) until we get at least that far.

Let's make this concrete. Assuming SE-0346, we can write this:

func map<From, To>(
  _ collection: some Collection<From>, transform: (From) -> To
) -> some Collection<To> {
  return self
}

func mapIt(integers: any Collection<Int>) {
  let result = map(integers) { String($0) }
}

If we cannot express the type of result as any Collection<String>, I think we should reject this call.

I'll add a discussion of this to the proposal.

Doug

11 Likes

Is there third option?

  1. We could force the caller explicitly write the return type when there is no way to express upper limit, so that when we get way to express the constraint, callers can manually add them.
func foo(value: any P) {
    let a = getA(value)  // error
    let a: any Collection = getA(value)  // ok
    let a: any Collection<.Index == Int> = getA(value) // when we get expression
}
2 Likes

I really like this proposal and how everything is explained. I think it makes sense to me.

I think I've understood the scenarios where the existential can't be opened, all but one:

cannotOpen4(xp)            // cannot open the existential in 'X<any P>' there isn't a specific value there.

The explanation here is not enough for my little brain ^^' What does "there isn't a specific value there" mean? Couldn't the any P be opened as part of the generic type of X? Apologoies in advance because is probably obvious but I"m missing something :smiley:

In the end of the pitch there was some discussion about not erasing result type at all - and instead keeping implicitly opened type in scope. I wonder why was that idea abandoned?

I’m highly in favor of this change. I think it will make the language more approachable. And it will allow generics to be used more often without excluding use cases.

I also love that it elongated the trick where you define a method on a type erased protocol to sneak in to get the type. Those language tricks are fun, but always feel like they shouldn’t be needed.

I can definitely see the alternative of explicitly opening being useful in the future, but this approach is far better for most use cases.

In the section “Contravariant erasure for parameters of function type”:

While covariant erasure applies to the result type of a generic function, the opposite applies to other parameters of the generic function. This affects parameters of function type that reference the generic parameter binding to the opened existential, which will be type-erased to their upper bounds . For example:
func acceptValueAndFunction<T: P>(_ value: T, body: (T) -> Void) { ... }

func testContravariantErasure(p: any P) {
  acceptValueAndFunction(p) { innerValue in        // innerValue has type 'any P'
    // ... 
  }
}

Like the covariant type erasure applied to result types, this type erasure ensures that the "name" assigned to the dynamic type doesn't escape into the user-visible type system through the inferred closure parameter.

Can you elaborate on why it’s beneficial to prevent the propagation of the implicitly opened type to the closure in Swift 6 mode? Is it motivated by preserving source compatibility for the body of the trailing closure?

In “Order of evaluation restrictions”:

an existential argument cannot be opened if the generic type parameter bound to its underlying type is used in any function parameter preceding the one corresponding to the existential argument.

This seems to put API design in potential conflict with performance. It would be less of an issue if this weren’t true, because the caller could hoist the expression into a local variable:

After the call, any values described in terms of that dynamic type opened existential type has to be type-erased back to an existential so that the opened type name doesn't escape into the user-visible type system.

Is preserving the order of argument evaluation really more valuable than enabling API designers to order their arguments in the way that reads most naturally for their clients without thwarting optimization opportunities?

According to the proposal, type<T>(of: T) -> T.Type will return any Error in Swift 5 and Error in Swift 6. Is there currently even a way to spell a value of non-existential type Error today? In Swift 6, will type(of: Error() as any Error) return some meta type that cannot be spelled today?

There's no distinction. any Error conforms to Error. The type of the value is always any Error.

Is this a special case retained for Error, or am I misunderstanding the proposal? When any Error is implicitly opened to bind to T, I would expect T.Type == Error.Protocol, and therefore let e: any Error = MyError(); type(of: e) == Error.Protocol.

In the case of type(of:) specifically, I think we'd want to preserve the existing behavior, where it effectively opens all existentials already, even in Swift 5 mode, since the intent of the "don't open self-conforming existentials" rule is to maintain source compatibility with Swift 5 in the first place.

1 Like

Very happy with this proposal. This sands down another sharp edge in the language and fits well with other recent proposals that deal with existentials. I can't speak to other languages with a similar feature. I participated in earlier stages of this proposal and reviewed the revisions here carefully.


I'm very happy to see where things have ended up in terms of Swift 5 compatibility but going with the most ideal behavior for Swift 6. I think this strikes the right balance and leaves the language in the best possible shape going forward.

I'm also very pleased to learn that this means the feature still can subsume the current behavior of type(of:) (and agree with @Joe_Groff's feedback that the existing behavior should be preserved for Swift 5, which I believe is the intention anyway).

I'm also very much in agreement with the proposal's limits that preserve the left-to-right order of evaluation. It may be non-obvious initially, but much more non-obvious would be out-of-order side effects that can't be easily to troubleshoot. Such out-of-order behavior wouldn't happen anywhere else except (in a proposal stripped of these restrictions) some but not all implicit existential opening operations—try explaining that to a user when existential opening doesn't even (currently) have an explicit spelling! Therefore, I agree with the proposal and consider that any restrictions on API design that arise as a result of the proposed restrictions are well justified here.

5 Likes

Very excited about this proposal, especially in combination with SE-0309.

Playing with the implementation here, I’m surprised to find that this does not compile:

func things<T: Collection>(in collection: T) { }

things(in: [1, 2, 3] as any Collection)  // ❌ type 'any Collection' cannot conform to 'Collection'

…but this does:

func things<T: Collection>(in collection: T) { }

func existentialThings(in collection: any Collection) {
    things(in: collection)  // ✅⁉️ If the above doesn’t work, why does this?
}

existentialThings(in: [1, 2, 3] as any Collection)

Am I missing something crucial in the proposal? Or is this just a bug?

I think it’s the correct behavior.

As the proposal name suggests, it is about implicitly open a type-erased value when concrete type is needed. The implicit open should not override explicit cast specified by the user.


In this example, existentialThings body behaves like the following with implicitly opened existentials:

func existentialThings(in collection: any Collection) {
    things(in: collection as <type(of: collection)>)
}

TL;DR: If you cast the type explicitly, implicit open won’t work.

3 Likes

Looks like you’re right: it’s the presence of the as any … at the call site that makes the difference in the case above. This works:

func things<T: Collection>(in collection: T) { }
let x = [1, 2, 3] as any Collection
things(in: x)

Though curiously:

func identity<T>(_ value: T) -> T { value }

func things<T: Collection>(in collection: T) { }
let x: any Collection = [1, 2, 3]
things(in: x)            // ✅ works 
things(in: identity(x))  // ❌ nope 

…so it is not merely the presence of an as in the argument expression that makes the difference. What is the principle here?

4 Likes

Hmm, mulling it over more, I wonder if this is because of the behavior described under Avoid opening when the existential type satisfies requirements (in Swift 5)?

Is there a compiler flag to make the toolchain for this feature run in Swift 6 mode?