Lifting the "Self or associated type" constraint on existentials

As code evolves, sorry.

Apps aren't the end of the line either. Unit tests often add dummy types; the Swift runtime supports dynamically-loaded in-process plug-ins; and you can even declare new types or add new conformances in the debugger if you want.

11 Likes

Cool! I did not know that.

Here is a good example of a genuine use case where our current restriction would have saved some confusion if the protocol in question had Self or associated type requirements:

That's a bug (I misunderstood, disregard) The non-self-conformance issue is also a good indication of why the existing restriction already doesn't really hide the complexity.

1 Like

(See other thread.)

I "loved" this comment when I saw it originally, but now I'm having second thoughts, because an idea I buried in an aside earlier is growing on me. Suppose:

  1. Self-conforming-protocol existentials can be spelled with just the protocol name
  2. Other existentials always need a where clause, which may be “empty:” Collection where _.

I think that might be enough to solve my problem with implicitly-unavailable APIs, and it wouldn't punish many uses of generalized existentials, which will have a where clause anyway. Finally, if we decide this was all a mistake, we can always lift restriction #2.

I realize this doesn't give us an immediate way to fix the technical problems @Joe_Groff is trying to address with this pitch, but it is an eventual future I think I could support. If we can agree that it's a good idea, maybe we can find a reasonable evolution path to get there that includes an immediate fix for the technical problems.

Thoughts?

1 Like

If We Could Do It All Over Again, it might've been nice to make the type system a Lisp-2 and put protocols and types in separate namespaces. That would give you the ability to express the "simple existential" idea as a typealias, e.g.:

protocol Runcible {
  mutating func runce()
}

typealias Runcible = Any<Runcible>

or spell the canonical type eraser for a less trivial protocol:

protocol Collection { ... }

struct Collection<T>: Collection {
  var _value: Any<Collection where .Element == T>

  /* Collection conformance here */
}

As for where we are today, since "self-conforming-protocol existentials" don't exist now except in limited circumstances, and declaring that self-conformance when desired will necessarily be explicit, making a rule that non-conforming existentials must be spelled differently doesn't really help the immediate source compatibility problems, since non-self-conforming existentials already exist and are spelled without decoration.

2 Likes

I very much hate separate namespaces but you can have the required Any without that.

  • I agree with @jrose so don't see the point of what you're saying. Is it about the namespaces or the required Any?
  • I recognize my proposal doesn't solve any immediate source-compatibility problems; in fact it creates an eventual source-compatibility problem. The questions I'm trying to ask are:
    1. How does my proposal sound as a place to end up?

      Note that I think Foo where _ does more to call out the unknown types and therefore the possible unavailability of API than Any<Foo> does, and it doesn't burden constrained existentials with the extra Any<…> syntax which AFAICT only benefits the unconstrained existential case.

      Note also that I include solving the technical problems your proposal addresses as part of the goal.

    2. If “good,” then what would the source evolution story be?

    3. What does that story imply about immediate next steps in the language?

1 Like

Thinking through this over the holiday and reading through the arguments, ultimately I think I come down in favor of @Joe_Groff's pitch. tl;dr: I don't think we should let the perfect be the enemy of the good.

@dabrahams raises a lot of good points but IMHO relaxing the artificial restriction on existentials doesn't change the status quo (we already run into these issues and relaxing the rules solves some real problems). From what I can tell, source compatibility means future versions of the language must solve this problem in a way that isn't seriously constrained by this pitch and the restriction imposes its own set of artificial design constraints that are no worse than the theoretical ones Dave raises (though reasonable people might disagree on that).

This discussion has been enlightening; it certainly raises the desire to tackle the existential problem in Swift 6.

(Life's existential problems are left as an exercise for the reader)

7 Likes

A type system that needs these kind of hack is at best unfinished.. ... ABI stability serves Apple’s needs, but does not make the language usable for people coming from more complete type systems (no .. that does not define python).
Btw... this is JUST an opinion

1 Like

I'd love to see this restriction lifted as well. Has there been any movement on including this in Swift 6?

6 Likes

agree, type erasure can't help us forever

4 Likes

I'd love to see this pitch get more traction.

7 Likes

I went ahead and tried to remove the diagnostic and there was no fallout. Of course, I only tried a few simple test cases, so I am probably missing some edge cases.

protocol P {
    associatedtype Q
    func a() -> Q
    func b()
}

struct S: P, Equatable {
    typealias Q = Int
    func a() -> Q { 0 }
    func b() { print("b") }
}

let p: P = S()
_ = p.a() // error: member 'a' cannot be used on value of protocol type 'P'; use a generic constraint instead
p.b() // ok, prints 'b'

let arr: [P] = [S()]
arr.forEach { $0.a() } // error: member 'a' cannot be used on value of protocol type 'P'; use a generic constraint instead
arr.forEach { $0.b() } // ok, prints 'b'

func takesP(arg: P) {
    _ = arg.a() // error: member 'a' cannot be used on value of protocol type 'P'; use a generic constraint instead
    arg.b() // ok, prints 'b'
}

takesP(arg: S())

If someone is willing to write a proposal, I think we can push this forward.

15 Likes

It's great if that's already working. You should also test:

  • functions which return Self, not just associated types
  • functions that take inputs of Self / associated type
  • functions that use Self / associated types in nested positions, e.g. func returnSelfInto(callback: (Self) -> ())
  • properties and subscripts
9 Likes

Seems to work for the most part, I think the nested Self case should throw an error instead of working.

protocol P1 {
    associatedtype Q
    func returnSelf() -> Self
    func returnAssoc() -> Q
}

struct S1: P1 {
    typealias Q = Int
    func returnSelf() -> Self { self }
    func returnAssoc() -> Q { 0 }
}

let p1: P1 = S1()
_ = p1.returnSelf() // ok
_ = p1.returnAssoc() // error: member 'returnAssoc' cannot be used on value of protocol type 'P1'; use a generic constraint instead

protocol P2 {
    associatedtype Q
    func takesSelf(_: Self)
    func takesAssoc(_: Q)
    func takesNestedSelf(closure: (Self) -> ())
    func takesNestedAssoc(closure: (Q) -> ())
}

struct S2: P2 {
    typealias Q = Int
    func takesSelf(_: Self) {}
    func takesAssoc(_: Q) {}
    func takesNestedSelf(closure: (Self) -> ()) { print(closure(S2())) }
    func takesNestedAssoc(closure: (Q) -> ()) { print(closure(0)) }
}

let p2: P2 = S2()
p2.takesSelf(S2()) // error: member 'takesSelf' cannot be used on value of protocol type 'P2'; use a generic constraint instead
p2.takesAssoc(0) // error: member 'takesAssoc' cannot be used on value of protocol type 'P2'; use a generic constraint instead
p2.takesNestedSelf { _ in } // works, but it should throw an error?
p2.takesNestedAssoc { _ in } // error: cannot convert value of type '(_) -> ()' to expected argument type '(P2.Q) -> ()'

protocol P3 {
    associatedtype Q
    var assocProp: Q { get }
    subscript(q: Q) -> Q
    var selfProp: Self { get }
}

struct S3: P3 {
    typealias Q = Int
    var assocProp: Q { 0 }
    subscript(q: Q) -> Q { 0 }
    var selfProp: Self { self }
}

let p3: P3 = S3()
print(p3.assocProp) // error: member 'assocProp' cannot be used on value of protocol type 'P3'; use a generic constraint instead
print(p3[q: 0]) // error: member 'subscript' cannot be used on value of protocol type 'P3'; use a generic constraint instead
print(p3.selfProp) // error: member 'selfProp' cannot be used on value of protocol type 'P3'; use a generic constraint instead

Oh, if we allow returnSelf, we should allow takesNestedSelf. But we should reject func foo(_: () -> Self) and func bar(_: (inout Self) -> ()).

1 Like

Yeah, those get rejected:

protocol P4 {
    func foo(_: () -> Self)
    func bar(_: (inout Self) -> ())
}

struct S4: P4 {
    func foo(_: () -> Self) {}
    func bar(_: (inout Self) -> ()) {}
}

let p4: P4 = S4()
p4.foo { return S4() } // error: member 'foo' cannot be used on value of protocol type 'P4'; use a generic constraint instead
p4.bar { _ in } // error: member 'bar' cannot be used on value of protocol type 'P4'; use a generic constraint instead

Opened a PR to run some tests and see what needs to be updated or if anything regresses.

4 Likes