Scoped Conformances

Aha! This makes me think I've completely mis-interpreted what the pitch is about then. I'll go re-read and see what I got wrong.

public struct X<T> { ... }
internal extension X: Wrapper where T: Wrapper { ... }
extension X: Comparable, CustomStringConvertible where T: Wrapper { ... }
// or
extension X: Comparable, CustomStringConvertible where Self: Wrapper { ... }

So the above example would not work, correct? I know this is mixing threads a bit, but what are the implications of that for Structural? I was thinking that the compiler would provide a memberwise conformance to Structural which would sometimes have to be conditional on type parameters, but maybe I misunderstood.

That sounds sensible as well. If nobody comes up with any clear downsides maybe it is the best choice.

This conditional extension X: Comparable is only usable in scopes where the T: Wrapper conformance is visible. I'm not sure if that answers your question?

With the pitch as written, that's correct. It's possible the pitch could be extended to allow that, but we wanted to bite off a self-contained and easily-analyzed chunk of the space, and...

I know this is mixing threads a bit, but what are the implications of that for Structural ? I was thinking that the compiler would provide a memberwise conformance to Structural which would sometimes have to be conditional on type parameters, but maybe I misunderstood.

…as far as I know, we don't need this capability. A conformance to Structural merely means that you can destructure the conforming type, not necessarily each of its members.

extension Bar: internal Foo {
    func doFoo() { print(baz) }
}

Can we only allow internal and explicitly not support private or fileprivate scopes? I don’t see a reason why we should go into the business of supporting private and fileprivate. Perhaps I am missing something but I think marking the conformance as internal should be sufficient to address the core of the proposal with out getting into the issues of private vs fileprivate.

1 Like

Would it make it possible to have two variables that hold the same instance, with the same dynamic type, with the same static type, but behaving differently? Very scary.

// a.swift
class Foo { }

// b.swift
extension Foo: fileprivate CustomStringConvertible {
    fileprivate var description: String { "short description" }
    var short: CustomStringConvertible { self }
}

// c.swift
extension Foo: fileprivate CustomStringConvertible {
    fileprivate var description: String { "long description" }
    var long: CustomStringConvertible { self }
}


// main.swift

func staticType<T>(of value: T) -> T.Type { T.self }

let foo = Foo()
let a = foo.short
let b = foo.long

assert((a as? Foo) === (b as? Foo)) // they are the same object
assert(type(of: a) == type(of: b)) // they have the same dynamic type
assert(staticType(of: a) == staticType(of: b)) // they have the same static type
assert(a.description != b.description) // but they behave differently?
3 Likes

The proposal says, and I agree, that the most "natural" solution looks something like this

The proposal calls out an initial solution that might be reached for, which is also something I've attempted when faced with this problem:

extension Equatable where Self: Wrapper, Value: Comparable {
  public static func <(a: Self, b: Self) -> Bool { // error!
    a.value < b.value
  }
}

Is there a way that we could make this example meaningful, rather than add new syntax? My (uninformed) understanding of why this doesn't work today is that the declaration/implementation for < "lives" inside Equatable, and at lookup time we consult the constraints to determine whether the declaration is available. From outside the module containing Wrapper, we can't determine whether Self: Wrapper, so we can't soundly determine the availability of <.

But at the time of compilation of the Wrapper module, we know that X: Wrapper, and X: Equatable. Is there a way we can "inject" the declaration of < into the appropriate types? I.e., external to Wrapper's module, the declaration of < would just appear to be a member on all concrete types which conform to Equatable and satisfy the appropriate constraints.

In this example, at least, we wouldn't have to worry about retroactive conformances (since nothing outside the module would be able to add an extension X: Wrapper), but we could have something like:

// Module A

public protocol P {}
public protocol Q {}
protocol R {}

extension P where Self: Q, Self: R {
    func foo()
}

public struct X: R {}

// Module B
import A

protocol Foo {
    func foo()
}

extension X: Q, Foo {} // error?

IMO, it's okay if this doesn't work—the extension declared in module A would not be visible externally (since it depends on internal constraints), so the author of module B has no reason to expect that X would gain an implementation of foo simply by conforming to Q.

Would something like this be reasonable, or would it stray too far from the way extension methods are implemented today?

ETA: Ah, I see this is discussed some in the Future directions section. To me, this seems like the "simpler" version of the feature, with full-fledged scoped conformances being the extension. Is there a reason to approach them in the order this proposal suggests, rather than the other way around?

1 Like

That would be sufficient, yes, but it is anticipated that fileprivate (which is the same as private when it comes to conformances today) would be easier to implement, and would also be sufficient. So in some sense the actual minimal proposal is to support fileprivate only.

Yes.

Very scary.

Then you should already be scared. The language allows conflicting conformances in multiple modules, and there's basically no way to outlaw it.

The current behavior is insane. The only sane behavior (at least in my opinion) for any system that allows such conformances to exist, is to function according to the principles described in this pitch, which does result in the same dynamic type behaving differently depending on the scope in which it was converted to an existential or bound to a generic parameter.

If you can come up with different, workable semantics for the language that resolves conflicting conformances from multiple modules, we can talk about that. Otherwise, the camel's nose is already inside the tent, so to speak.

2 Likes

Please be careful about attributing statements, especially when you add words in quotation marks. The proposal does not, in fact, say that. The proposal implies it's the most natural solution to the problem as originally stated, but as noted later in the proposal, we have a slightly different problem to solve.

Is there a way that we could make this example meaningful, rather than add new syntax?

Perhaps we could, but it would not cover the use case of the proposal authors, in which the equivalent of X and Wrapper are both public, but the conformance of X to Wrapper should not be exposed publicly.

ETA: Ah, I see this is discussed some in the Future directions section. To me, this seems like the "simpler" version of the feature, with full-fledged scoped conformances being the extension. Is there a reason to approach them in the order this proposal suggests, rather than the other way around?

Two reasons:

  1. As noted in the pitch, to enable what you're proposing as more basic, someone has to prove that it's okay for public API to be implemented by extensions that are conditional on non-public names. I don't know whether that's a problem or not, but our pitch as stated avoids that issue. If it's not a problem, so much the better; let's have that feature too!
  2. More importantly, we'd have no way of eliminating binary size bloat for users of Structural.
1 Like

Apologies, edited to clarify!

1 Like

1:

// Top Level 
private struct Bar {..}
private protocol Foo {..}

// Some other file
extension Bar: private Foo { // Would this be allowed though redundant?
    func doFoo() { ... }
}

2:

/*internal*/ struct Bar {..}
/*internal*/ protocol Foo {..}

// Some other file
extension Bar: private Foo { //Why not just internal?
    func doFoo() { ... }
}

I guess my issue is: What is the use case to support file encapsulation when extension conformances themselves can only be applied in top level code?

What happens we we start allowing extensions in arbitrary nested scopes?
What happens when we introduce submodules?

1 Like

No, you wouldn't be able to mention Bar in “some other file” because it was declared private at the top level of a different file, which means the same thing as fileprivate. Those are just the rules we have today.

I can think of two reasons offhand that you might want a fileprivate conformance:

  1. To hide the APIs implied Bar's conformance to Foo from the rest of the module, where they are presumably irrelevant and distracting, and may even expose the ability to break Bar's invariants (though generally in that case you would tend to put that conformance in the same file as Bar's declaration).
  2. You actually have other contexts in the same module where you want Bar to be able to conform to Foo differently. The scenario is not as outlandish as it might sound at first, although the simplest example I can think of offhand is a bit mathematical: given a Monoid protocol there are two equally valid ways for UInt to conform: one is with the operation being &+ and the identity 0. The other is with the operation being &* and the identity 1. Neither of these conformances is “the right one” and both would be useful.

What happens we we start allowing extensions in arbitrary nested scopes?

If we ever decide to allow them, they work according to the same rules described here: the conformance that applies is the one visible in the nearest enclosing scope.

What happens when we introduce submodules?

I guess it depends on what the scoping rules are for this hypothetical “submodules” feature, but I think once we know what those are in general, applying the rule proposed here is straightforward.

3 Likes

Just copying a relevant insight I had here:

Semantically, all the issues of fileprivate conformances are shared by this example we can produce today:

// Module A
extension UInt: Collection { ... } // the numbers from 0 to `self`

// Module B
extension UInt: Collection { ... } // powers of 2 in self's representation

// Module C, File1.swift
import A
...

// Module C, File2.swift
import B
...

As long as nobody else imports A or B, the conformance from A is effectively private to File1.swift, and the one from B is effectively private to File2.swift, so the above would be equivalent to:

// Module C, File1.swift
extension UInt: fileprivate Collection { ... } // the numbers from 0 to `self`
...

// Module C, File2.swift
extension UInt: fileprivate Collection { ... } // powers of 2 in self's representation
...

Similarly, the semantic issues of internal conformances are shared by this example:

// Module A
extension UInt: Collection { ... } // the numbers from 0 to `self`
// Module A1, all files
import A

// Module B
extension UInt: Collection { ... } // powers of 2 in self's representation
// Module B1, all files
import B

which (as long as nobody else imports A or B), would be equivalent to:

// Module A1
extension UInt: internal Collection { ... } // the numbers from 0 to `self`

// Module B1
extension UInt: internal Collection { ... } // powers of 2 in self's representation

Scoped conformances provide the ability to eliminate:

  • module A
  • module B
  • all the metadata associated with exposing those conformances publicly

The latter goal was the original motivation for this pitch.

[Of course all of this depends on the idea that I understand the intended semantics of distinct imports in different files of the same module, which is currently probably buggy]

1 Like

These would be fileprivate conformances, no? Also, why are we able to eliminate the import A from File1.swift but not import B from File2.swift?

After following along in the linked thread I'm pretty convinced about the utility of locking down these conflicting conformances somehow. If the rule you proposed over there (as stated by Doug) of "the presence of any conflicting protocol conformance for X in a source file ... makes effectively all of the APIs that traffic in X unusable in that source file" were adopted, would that affect your thoughts on the question @anandabits asked about dynamic casts up-thread?

Your initial response called out these conflicting conformances as one reason (among several) for the proposed model for dynamic casts, but it seems like under the stated rule a dynamic cast to a protocol with conflicting conformances could be a compile error?

1 Like

We are. Thanks for pointing both of those out; I fixed and expanded the posting!

:heart:

Whether conflicting conformances for a given protocol exist at the point of a dynamic cast to that protocol can't be the only determiner: those conflicts might not have anything to do with the source type. If they do have some relationship to the source type, then IMO it depends on the static type of the source of the cast. If it's a concrete type, and the conformances are ambiguous, handling it should already be disallowed per what I said in the other thread. If it's an existential, it seems to me that whether there's a local conflict should be irrelevant, because what existentials do is to capture the properties of the concrete type in a type-erased package at the point where the existential was formed. It doesn't make sense to me to pop out of existential-land to do a check for conflicts in the local scope just because a dynamic cast is being done.

1 Like

That makes sense to me!

Diverging from the conflicting conformances tangent and looking back to your original example regarding dynamic casts,

it's not entirely obvious to me that it's desirable foo(value:) to ever print "P" under the "separate modules" circumstance. It seems like that leads to potentially surprising behavior for the authors of both File1.swift and File2.swift. For the author of File1.swift, the supposedly internal conformance X: P may be unexpectedly vended, and AFAICT the author of File1.swift cannot prevent this.

As for the author of File2.swift, they may end up with an existential whose concrete type they can verify, but with conformances they're unable to access directly. Here's an example illustrating both situations:

// Module A
public protocol P { }
public struct X { }
extension X : internal P { ... }

public func perform(_ foo: (Any) -> Void) {
  foo(X()) // X: P may escape the module!
}

// Module B
func bar() {
  perform { value in
    print(value is X) // true
    print(value is P) // true

    let x = X()
    print(value is X) // true
    print(value is P) // false (!)
  }
}

First, just to confirm: under the "conformance(s) are captured in the scope where a value is first converted to an existential type" rule that the original post proposes, are the annotated outputs of the print calls correct? If not, what have I missed?

If so, I can think of a few alternative rules (the first two of which I think I would prefer to the pitched rule):

  1. "X’s conformance (or not) to P is captured in the scope where the X value is first converted to the existential type P."
    That means we would have the following:
// Module A
public protocol P { }
public struct X { }
extension X : internal P { ... }

public func perform(_ foo: (Any) -> Void) {
  foo(X()) // X: P stays put
}

public func performP(_ foo: (Any) -> Void) {
  foo(X() as P) // X: P escapes
}

// Module B
func bar() {
  perform { print($0 is P) } // false
  performP { print($0 is P) } // true
}

This puts the control of the X: P conformance back in control of the author of Module A, but still potentially results in surprising behavior for Module B (since there's still an "inaccessible" conformance that can be observed).

  1. "X’s conformance (or not) to P is captured in the scope where the X value is first converted to the existential type P, but only until the P existential is casted away"
    This would give us:
// Module A
public protocol P { }
public struct X { }
extension X : internal P { ... }

public func perform(_ foo: (Any) -> Void) {
  foo(X()) // X: P stays put
}

public func performAsP(_ foo: (Any) -> Void) {
  foo(X() as P) // X: P stays put
}

public func performP(_ foo: (P) -> Void) {
  foo(X()) // X: P escapes
}

// Module B
func bar() {
  perform { print($0 is P) } // false
  performAsP { print($0 is P) } // false
  performP {
    print($0 is P) // true (trivially, since $0 has static type P)
    // but...
    if let x = $0 as? X {
        print(x is P) // false
    }
    let a: Any = $0
    print(a is P) // false
  }
}

This also results in surprising behavior for Module B, since the P can be "lost", but maybe this is less common (or less surprising?) than alternate rule 1?

Lastly,

  1. "X’s conformance (or not) to P is not discoverable where the conformance is not visible. The author of the scoped conformance cannot form an existential of P that would escape."
    This, I think, is unimplementable as stated. We'd have to come up with some sort of "escaping" notion for existentials, and track everywhere in A that X gets converted to P... Unless we were willing to additionally say, "you cannot form an existential with a conformance that has lower visibility than the protocol", I think this rule is unworkable.
    ETA: As I sit with it, I don't actually mind the stronger "no existentials" version of this rule—it seems like it would at least support the use-case for this proposal, and if the alternative is that the conformance is able to "escape" the scope where the conformance is declared, then I think it's a not-unreasonable response to say "okay, then you have to fully expose the conformance".
1 Like

I'll try to come back to the rest of this post later, but for now I just wanted to address this part. If you're saying, that exact code should not print "P", then I agree, because the X: P is not visible where X() was turned into an existential.

That said, if you put a public someX: Any = X() in File1.swift and then `foo(someX), IMO it should print "P". The situation is exactly analogous to this code, and it should behave the same:

// File1.swift
public class P { }
public struct X { }
internal class ConformanceXToP: P {
  init(_ value: X) { self.value = value }
  let value: X
}
public let someX: Any = ConformanceXToP(X())

// File2.swift
func foo(_ value: Any) {
  if let x = value as? P { print("P") }
}

foo(X())    // X is not a conformance to P, locally.  No output.
foo(someX)  // someX is a conformance to P.  Prints "P"

The example you gave doesn't expose the internal conformance X: P any more than my variant exposes the internal class ConformanceXToP.

1 Like

Ah, no, that was unclear. More accurately, I should have said "it's not entirely obvious to me that it's desirable foo(value:) to ever print "P" [whenever it's called with an argument of concrete type X] under the 'separate modules' circumstance". That said, I think the only option I presented which actually prevents that is #3, which has significant limitations.

Okay, that's what I thought. Thanks for clarifying!

I view these cases as subtly different, though it was somewhat difficult to phrase why. I think it comes down to the fact that for me, dynamic casts/existentials feel like "defining features" of a protocol conformance, by which I mean that when I declare a conformance to a protocol, those are (some of) the meaningful things I wish to do with that conformance.

OTOH, the "defining features" of a class are its properties, methods, initializers etc. When I declare an internal class, I intend for these features to be hidden from other modules. I don't want clients poking my properties, calling my methods, or creating instances of me (unless through a declared inheritance which I know will expose public API).

As the author of File1.swift, if I'm using a feature that describes a declared conformance X: internal P as an "internal conformance," I similarly expect the "defining features" of that conformance to be hidden outside the module. At the very least, I would expect that I, as the author of an internal conformance, am able to control when those features are exposed to clients. I.e., I would want something like what you're doing with ConformanceXToP—without explicitly "exporting" the conformance, it doesn't leave the module.

Help me, here, please. When an X escapes from A, can't we simply treat it like it does not conform to P? Shouldn't the cast always fail outside of A? I must be misunderstanding the issue.