Scoped Conformances

That's the model I was hoping to have with rule #3, but it ends up placing some pretty severe restrictions on how the conformance may be used within A. The issue is that A may form an existential P with X, and that existential value may escape A:

// Module A
public protocol P { }
public struct X { }
extension X : internal P { ... }

public var someP: P = X()

// Module B
print(someP is P) // this better be true!

After thinking about it further, I don't think rule #3 works at all. As long as the conformance is allowed to be used at all in A, you could write something like:

// Module A
public protocol P { }
public struct X { }
extension X : internal P { ... }
extension P {
  func escapeAsP() {
    someP = self
  }
}

public func escapeXAsP() {
    X().escapeAsP()
}
public var someP: P? = nil

Thank you. Please bear with me while I expand on your example and my question.

// Module B
import A
print(someP is P) 
    // This better be true!  Yes.

let x: X? = someP as? X
    // I hesitantly propose, once an instance of X is resolved 
    // in B as an X, it ceases to conform to P.
    // Is this feasible?

someP.someMethodDeclaredByP()
    // This is valid because `P`'s methods are public.
    // BUT, in this situation, if X has customized a 
    // requirement of P, that conformance won't be called.
    // Is that ok?
    // Is that ok if P doesn't have a default implementation of the requirement?

print(x is P) // false
    // That looks weird, but is there any harm? 

In other words, when an X is treated as an X outside of Module A, it loses its conformance to P. In the meantime, an X wrapped inside of an existential P can be passed around outside Module A. Is it feasible for an X wrapped inside of an existential P to be passed around in such a manner?

Is any harm caused by doing something like this?

I'm not sure.

1 Like

This is basically what I was getting at with rule #2 in my post. B could vend X in an existential P, but as soon as you “lost” the conformance, you’d be unable to recover it. IMO it should also be perfectly fine to invoke P’s requirements on that existential and have the appropriate witnesses on X get called. Particularly if we’re adopting a model that gives the author of the internal conformance control over when that conformance gets vended, I don’t see an issue. And as you note, if P doesn’t have a default implementation for a particular requirement, we don’t really have another option.

1 Like

Ah, thank you. Now, I much better understand the discussion in the preceding several posts.

So, to sum up a possible model for dynamic cast behavior outside the scope of a scoped conformance:

If a scope employs a scoped conformance to conform a type to a protocol, and vends an instance of that type beyond the scope, then, in other scopes, the conformance of the type to the protocol will be visible if and only if the instance is vended as an instance of existential type and then only for so long as the existential instance is not cast to a concrete instance. The existential instance may be dynamically cast from one form of existential to another (e.g., from Any to the protocol type). Once cast to a concrete instance, the protocol conformance ceases to be visible outside of the vendee scope, and the instance cannot be dynamically cast back to an existential instance of the protocol type. While the instance is of existential type, its conformance to the protocol's functionality will be available.

@gribozavr, is it feasible to conditionally access the protocol's functionality in this fashion?

What happens if the vendee scope provides its own conformance for the subject type to the subject protocol? I assume that the vendee scope's conformance will be used instead of the vendor's conformance.

1 Like

I think that's the model that @dabrahams ntended in the original pitch, but I'm not 100% certain about the intended model for when the conformance stops being available. Rule #1 that I proposed above is the same as what you've written here, except that the conformance is "captured" in the existential only if the initial "concrete to existential" conversion is from the concrete subject of the internal conformance to the protocol (that is, from X to P in the examples).

Rule #2 is the same as rule #1 except that the conformance to P doesn't survive a cast to any other type, existential or no. So as soon as the static type P is lost, it can't be recovered from the instance.

Good question. I think using the vendee's conformance makes sense. The one instance where it maybe (?) makes sense to use the vendor's conformance is when it has vended an existential of static type P, but off the top of my head I can't really think of compelling arguments either way.

1 Like

There's been so much discussion here it's very difficult to catch up, so I'll admit I haven't read everything. But, from what I have read, I'm trying to develop a sense of what some people are uncomfortable with about my proposed semantics for existentials.

First I think maybe it comes from the idea that the semantics of a given conformance shouldn't be observable from outside its scope, but that can't be quite right, because you can always call into the conformance's scope and observe the semantics that way. So then I think, maybe there's an idea that those semantics shouldn't be observable through directly invoking protocol requirements, but then nobody is surprised that the standard library observes the semantics of the conformance of my private type X to Comparable when I sort an Array<X>. So if somebody could describe the principles my proposed semantics are violating, that would be really helpful.

I have a strong mental model of an existential type as a base class (plus value semantics preserving COW), and a conformance as a derived class. The semantics of the derived class travel outside its scope when handled as a base class instance. It's very powerful for users when different parts of the language map onto one another creating these kinds of equivalences, because they make everything easier to reason about. Absent a good reason for breaking this equivalence, I think we should preserve it. Is there such a reason?

2 Likes

Hmm. I don't see that we've really expressed discomfort. From about post 37, forward, my perception is that we've been trying to put some finer points on the mechanics of what we think you are proposing regarding how as? functions in code that is dependent upon the module with the scoped conformance.

I agree. Module B can ask Module A to do something with an X, and behavior provided by P may influence the result returned by Module A. That is fine.

I agree. If I import a dependency like the Standard Library or Foundation into my scope, then create a scoped conformance of X: P, and then pass an instance of X to some of dependency's functionality, I think of the code from the dependency as existing within my scope. My scope is using the dependency. So, I would expect that the dependency's code will be able to see and use X's conformances, and Array<X>.sorted() would be able to see and use the scoped conformance of X to Comparable.

I don't see anything that I would call a violation.

We are thinking in the context of Module A, with its public X and P and its scoped conformance of X: P, being a framework that a program will import. When the program imports and uses A, A might create and give to the program some Xs or some Xs wrapped up as existential Ps. The recent discussions have been about whether and to what extent the program can see and use the conformance of those Xs to P.

I believe the essence of our conclusion is that, while the Xs remain packaged as existential Ps , the protocol conformances are available to be called by the rest of the program. But, once those existential P's are cast back to Xs, their connection to P evaporates while they remain outside the scope of Module A. We think that conclusion is consistent with what you are pitching. Is it?

Again, I don't envision that impacting how an X's conformance to P is handled when the Module A calls into the Standard Library or any other dependency to ask that it do something with an X. I see that as being entirely different than code that depends upon Module A that gets Xs from Module A.

Glad to explore it further.

My initial post that kicked off this line of discussion was expressing a bit of discomfort. I summarized here one reason why I view this differently from the internal class example, but let me try to directly respond to the points you raise in your most recent post, @dabrahams.

After considering the various examples that you and @mattrips have raised, I think my discomfort basically comes down to the ability of the author of a scoped conformance to control when that conformance can "escape" the scope where it's defined. With that high level idea in mind, let's see how it applies to the points you raise:

This is only true insofar as the scope of the conformance chooses to expose any interface which allows the observation of those semantics. There might be no such interface at all! And even if there, say, the following function in module A (which declares an extension Y: internal Comparable to a public type Y):

public func sortYs(_ arr: [Y]) -> [Y] {
    arr.sorted()
}

then the fact that we happen to be using the internal conformance to Comparable is essentially an implementation detail. The client can't determine, and doesn't need to know, whether we have a scoped conformance doing the heavy lifting, provided a custom predicate to sorted(by:), or wrote our own sorting algorithm from scratch.

Two things here. The first is that in today's Swift, with no notion of scoped conformances, a conformance is inherently a "public" thing (since, at the very least, they might be discovered dynamically), so of course the conformance to any public protocol might be used if an instance of the private type escapes the module.

Second, as the author of the private type X, at least in the case of sorting an array, I again have control over whether that conformance is used. Calling sort() with no arguments is implicitly giving my blessing for the compiler to utilize conformance to Comparable, since the method wouldn't even be available without that conformance!

In a world where I've declared an internal conformance of X to fail, that element of control still remains. Furthermore, if we added the is Comparable test that you've discussed in another thread, I would expect that to fail if Array<X> checked self is Comparable from within the Standard Library.

In your original example illustrating this point, the setup in module A/File1.swift was:

public class P { }
public struct X { }
internal class ConformanceXToP: P {
  init(_ value: X) { self.value = value }
  let value: X
}
public let someX: Any = ConformanceXToP(X())

The final line here is what I'm trying to get at with my notion of conformance-author control. In this model, the control of when to vend this class-as-conformance is still within the power of the module author. The main "break" in this analogy is that declaring a conformance X: P results in implicit conversions from X to P, and direct usage of X in a generic context requiring conformance to P. In the class construction, X receives neither of these benefits—the conformance must always be explicitly attached via construction of a ConformanceXToP.

To me, the analogous line in scoped-conformance-land is actually:

let someX: Any = X() as P

This maps to Rule #1 from my older post—the conformance must be "captured" explicitly by the scope that wrote the conformance in order for the conformance to "escape." With your model for the interaction between scoped conformances and existentials, my question would be: how would the author of a scoped conformance prevent that conformance from being discovered via dynamic casts if they wanted to keep it completely internal to their own scope (assuming that they're forced by some API to vend an instance of the conforming type as an existential)?

Let me know if that adequately addresses your questions about my discomfort!

If we can have a type conforming to the same protocol in more than one way, could we also imagine overriding a protocol conformance and replace it by a local one in a certain scope? I'm just spitballing, but on a regular basis I am frustated that I can't choose a conformance to Hashable when I make a Dictionary, for example to have String keys with case insensitive comparison, or to have Object keys with === comparison.

It could look like:

func buildCaseInsensitiveDictionary<T>() -> [String where override Equatable: T] {
    private extension String: override Equatable, CaseInsensitiveString {}
    return [String: T]()
}

That's not how I read post 37 in particular, but thanks for catching me up.

Of all the semantic elements involved here, I think I'm the least concerned with how as? casts behave. Yes, we need an answer, but a down-cast is a very explicit request that can't occur by accident, so in some sense it might be OK to pick any easily-described semantics as long as they don't break soundness. After that, IMO, it comes down to a question of which semantics are most broadly useful.

It seems obvious to me that q as? Set<X> can't succeed if the X: Hashable conformance used to build q conflicts with the local meaning of Set<X>. Are there other examples of potential soundness issues? Are there use cases that argue for particular variations of the semantics?

I believe the essence of our conclusion is that, while the X s remain packaged as existential P s , the protocol conformances are available to be called by the rest of the program. But, once those existential P 's are cast back to X s, their connection to P evaporates while they remain outside the scope of Module A. We think that conclusion is consistent with what you are pitching. Is it?

Yes, I think that's the only possible semantics if X is to have meaning as a static type outside of A. Otherwise, outside of A, you can always write (someP as? X) ?? localX and now you have an instance of X that may-or-may-not have a connection to P.

Sure, but that all hinges on what you mean by “escape.” For me, it means “influence the locally-discernable meaning of code outside the scope.” It's important to remember that access control is not about security, but about modularity.

I absolutely disagree. As a public method, sortYs has to have some meaning that's useful to its clients; you have to be able to document it and clients need to be able to verify that it does what it says. If Y: Comparable determines the behavior of sortYs and is invisible to clients, there's almost nothing you can say about it. I suppose you could change the name to maybeShuffleAccordingToSomeOpaqueCriteria, but then I don't think that function has a useful meaning for clients.

Sorry, I have to disagree with your premises:

  1. A conformance to an internal protocol or of an internal type to a protocol is clearly not public, as it can't affect the local meaning of code outside the module scope.
  2. An internal subclass of a public base class can be discovered dynamically; that doesn't make it public.

I'm sorry, I don't know what you mean by “declare… a conformance to fail.”

More generally, I'm not so interested in satisfying an abstract desire for “control.” These scenarios seem awfully… disconnected from anything anybody wants to actually do with a program or any real problems that need to be prevented. I might have an easier time with it if you could describe how these other semantic ideas would serve/harm actual code.

With your model for the interaction between scoped conformances and existentials, my question would be: how would the author of a scoped conformance prevent that conformance from being discovered via dynamic casts if they wanted to keep it completely internal to their own scope (assuming that they're forced by some API to vend an instance of the conforming type as an existential)?

Yeah, I hope you won't be offended, but I just don't think my proposal needs to rise to that challenge. If you can explain why that kind of control is important, it might change things for me, but since we don't have that sort of control in similar areas of the language (you can still discover the type(of:) an internal subclasses outside its module), I think it would take quite a lot to motivate me to try to address it in this one area.

1 Like

Thanks, I think that's a reasonable way to think about things that just doesn't quite line up with mine. My mental model would be closer to "can be relied on by clients outside the scope in a way that influences program behavior."

When applied to e.g. internal classes inheriting from some public base, I agree there's a bit of handwaving that has to happen, but for the most part I think the same logic applies—the internal class should obey all the same contracts of the base class, so in usage that is not obviously fragile (as in String(describing: type(of: instance)) == "Base" or similar) it should be basically indiscernible whether a client is dealing with an instance of the base class or some internal subclass. Perhaps I'm missing ways in which this model is already broken in Swift today, which would be useful to know!

Sure, but whether that meaning comes from an internal conformance, custom predicate, or custom sorting algorithm should, IMO, be an implementation detail of the module defining the sortYs function. In order to make sortYs meaningful, yes, it would have to be documented, but if I wanted to switch from one implementation strategy to another that shouldn't be a client's concern.

I think I mostly addressed these points in the first segment of this post.

Sorry, that looks an editing mistake. The words "to fail" should be elided. Thanks!

Sure. The main concern that I would have as a library author is, "what is the contract that I'm providing to my clients?". There's the solid, explicit contract, specified in the type system, method names, documentation, etc., and it should be obvious to me as the author that changing these aspects of my library may break clients.

There's also a weaker, implicit contract that is made up of the observable behavior of my library. E.g., for a sorting API, it's perfectly fine for me to switch from sort(by:) to a custom algorithm that results in the same ordering, but if I'm interested in never breaking my clients, it may not be okay for me to switch from, say, a stable sort to an unstable sort. (There's some hand-waving here for things like bug fixes, but at the very least it's reasonable to be making decisions along an axis of "how much do I need to change the semantics here" vs. "how likely is it that clients are relying on these semantics".)

Regardless of the fact that clients should not rely on behaviors that are not part of the documented API, some may, and I don't see it as an unreasonable goal for library authors to promise compatible semantics from one version to the next. My concern with the as-proposed rule for dynamic casts is that, if I have written an internal conformance, it may be impossible for me to prevent my clients from relying on that conformance, despite the fact that they may be doing the Wrong Thing.

A client which discovers the conformance X: P by accident might think, "Hey, cool, this library has an internal conformance I can use!" and write something like:

let p = (someX as? P)! // Note: always succeeds because A provides an internal conformance X: P

Now, the author of module A removing a conformance which was ostensibly "internal" might crash clients.

Yeah, that's a perfectly reasonable position to take as the proposal author. I just wanted to air my grievances with this aspect—overall I really like the idea of scoped conformances, and would even be in favor of it as-proposed if the alternative was not to have it at all.

1 Like

Oh, that's a great point! I definitely did not address the “can be relied upon by clients” part.

But I think we're describing different aspects of public-ness. I am describing it in terms of language mechanics, and you're describing it in terms of the consequences of those mechanics for the programming model (of API-stable code).

When applied to e.g. internal classes inheriting from some public base, I agree there's a bit of handwaving that has to happen, but for the most part I think the same logic applies—the internal class should obey all the same contracts of the base class, so in usage that is not obviously fragile (as in String(describing: type(of: instance)) == "Base" or similar) it should be basically indiscernible whether a client is dealing with an instance of the base class or some internal subclass.

In the same way, a type conforming to a protocol must obey the contracts of the protocol. What handwaving?

In order to make sortYs meaningful, yes, it would have to be documented, but if I wanted to switch from one implementation strategy to another that shouldn't be a client's concern.

Granted. If the documentation says you sort Ys in order of increasing .age, it should be indistinguishable to clients whether you have done that with a fileprivate conformance to Comparable or with a closure.

There's an infinite amount of incidental behavior that is unspecified by any real API, and that may even depend on details of the language implementation that could change with the next release. Any client could decide to rely on any of it without justification, so I'm not sure it's worth worrying about. That said, I'm happy to discuss how it plays out for conformances.

Oh, that's interesting. When we talk about as? casts in this contexts, I'm always thinking about

someP as? X

which IMO should succeed or fail based on how/where someP was formed.

I take it for granted that casting from a concrete type to a protocol type should be based entirely on the conformances statically visible at the point of the cast, just like the ability to use a P API on an X instance. In fact, IMO, this should always produce a warning:

someX as? P
  • if X: P isn't visible, “warning: cast always fails.”
  • if X: P is visible, “warning: cast always succeeds; don't you want to use as instead?”

Now, the author of module A removing a conformance which was ostensibly "internal" might crash clients.

Well yeah, but the same could go for someP as! X or someBase as! Derived, even if the relationship stays completely public. If my API only promised you Ps and you started relying on those Ps being Xs, because that seemed to be the case, it's the same as if my API only promised you an Int and you started relying on the Ints being non-negative, right?

2 Likes

Yeah, I think that captures the differences in our mindsets nicely.

The handwaving I'm referring to is basically just papering over the ways in which type(of:) can be used to, for instance, inspect an instance of a base class to discover the internal subclass that I'm actually using. I see a distinct difference between a client relying on something like:

String(describing: type(of: instance)) == "InternalSubclass"

and something like:

someX is P

In other words, it matters that the names X and P are visible to the client module and can be used in a first-class language construct to discover the "internal" conformance. As a library author, I'm much more comfortable breaking clients who were relying on the name of my internal class, than I am clients who were (perhaps inadvertently, even!) relying on an internal conformance which "escaped" my module.

Yeah, I concede that it's of course infeasible to hide all possible details of the implementation of a library from its clients, but as I mentioned previously, it matters to me how difficult/unlikely it is for clients to rely on those implementation details, especially inadvertently.

Ah! Yeah, seems like we have been talking past each other a bit. I'll have to think about this case a bit further to organize my thoughts.

Great. We're on the same page for the "concrete to protocol" case.

My specific objection arose from this post, where I understood you to be saying that this example:

// File1.swift
public protocol P { }
public struct X { }
extension X : internal P { ... }
public someX: Any = X()

// File2.swift
func foo(value: Any) {
  if let x = value as? P { print("P") }
}
foo(someX)

should print "P". That's where I view the issue with the conformance "leaking" to clients being problematic. (I think this should not print P.)

Yeah all of these considerations are along a continuum of tradeoffs that a library author should be thinking about. My worry is that it is not at all obvious to a library author that clients could even possibly be relying on their internal conformance—it's internal! Specifically, it seems like the fact that this conformance can leak to clients is at odds (though not directly contradictory) with this item that we agree on:

If I've implemented an internal conformance to Comparable for Y (specifically to implement sortY), and in another (unrelated) part of the library I've vended an instance of Y as an existential (having nothing to do with Comparable), I would—without having had this extensive conversation :slightly_smiling_face—be very surprised that changing my implementation strategy to use a closure actually breaks my clients! I don't even have the chance to make the choice in a reasoned way if I don't realize that certain implementation details may actually be visible to clients.

1 Like

First and foremost, my apologies for my post of late last night with the loose "us" and "our" and the generalized statements of my view of the matter. It misrepresented the discussion. Sorry for that.

As I've read the back and forth on this particular aspect of the point, I'm not troubled by the client who knowingly relies upon internal implementation details.

However, it occurs to me that a client easily could stumble into the visibility of the conformance, and then start relying upon it without realizing that it is a scoped conformance. I'm sure that sort of thing will happen.

A partial work around is to do what you suggest--make the dynamic cast fail when trying to go from Any to P. While I do see value in that approach, is there a downside to it?

Okay, I understand what you're saying. But as I mentioned below—and you agreed “we're on the same page” about— someX is P isn't ever going to reveal a conformance that isn't otherwise visible. The only interesting cases are someP is X and someP is Q (where Q is a protocol).

Personally, I'd be perfectly happy if someP is X == false unless X: P is visible in the local scope and is the same conformance that was visible when the existential was created. I think that's the least-confusing possible semantics, and anyway, when given a P there's no guarantee you know the type. When combined with @Douglas_Gregor's idea that all Xs are the same type (but not all Set<X>s), it means we can't have a simple rule like “the dynamic cast to a concrete type succeeds if the target type identity is the same as the type identity of the instance that formed the existential.” But, you know, I can live with that.

I don't see how this happens inadvertently, or that you can do anything interesting or useful with the knowledge that someP actually was an X when you don't know how X was satisfying P.

Also for the record, type(of: someBase) reveals a lot more than the name. Depending on how Base is defined, they may be able to create new instances or call class methods.

Yeah, I concede that it's of course infeasible to hide all possible details of the implementation of a library from its clients, but as I mentioned previously, it matters to me how difficult/unlikely it is for clients to rely on those implementation details, especially inadvertently.

Hey, I'm a library guy myself, so I agree with you about what matters. I just don't see how this one actually gets into trouble with this. I mean, if it's going to be a problem, you can tell me a plausible story of how that actually happens and what the consequences are, right?

Well, this is not the same as either of the things we've been discussing; this is the someP as? Q case in the form of someAny as? P.

I think I understand you to want that to behave like

(someAny as! T) as? P

where type(of: someAny) == T.self. Correct?

I could accept that if I had to, though I'm not sure if it makes sense. Have you considered how this would play out for someP as? P?

Doing that would mean making, at runtime, all of the same decisions used to compile someT as P; you may have to argue with the implementors over whether that's possible.

Or not, if they don't want to be paralyzed by infinite considerations :wink:. It's pretty basic to me that clients that rely on undocumented details may be broken at any time.

My worry is that it is not at all obvious to a library author that clients could even possibly be relying on their internal conformance—it's internal!

Your worry is still too vague for me to address. Without a description of what it could possibly mean for a client to “rely on” and internal conformance or plausible story that describes a bug caused by that reliance, I just don't feel like there's anything to go on here.

I don't see how it could, even leaving aside that Comparable can't be cast to or tested for. Tell us the story of how this happens in all its detail.

My question to @Jumhyn about this, in fewer words, is “what does this reliance look like and how does the client get broken?”

Thanks for your patience @dabrahams, and sorry it took me a few days to get back to you. I finally had a chance to write up a response!

Yeah, having had some time to think about it I think I agree. In some ways it is exactly analogous to the "not all Set<X>s are the same idea"—if I understand correctly Set<X>, as a distinct type consists of Set, a generic argument (in this case X), and an invisible bundling of X with a particular conformance to Hashable. It makes sense to me that inserting an X into a P existential also creates that "type plus conformance" bundle, such that X in module B (with no conformance or a different conformance) doesn't refer to the same X contained in someP.

Right, but unless I'm missing something, for a subclass which respects the contract of its superclass, those operations will behave according to the contract of Base, so it's difficult for the client to do something that would break if you, say, swapped out one internal subclass for another. (For subclasses which don't behave according to the promises of the superclass, you've of course already made the choice to leak implementation details).

Heh, this is why I said I think we've been talking past each other. I thought this was what we have been discussing? I perhaps didn't clarify enough when we moved from conflicting conformances to the current line of discussion. My apologies for not being more clear!

I believe that mostly captures the semantics I expect, with one exception:

I'm assuming here that someP has static type P, in which case I would actually want this cast to succeed.

Yes, I agree that this is a perfectly reasonable stance for a library author to take. But I also think it's reasonable for a library vendor to make a promise that they will avoid significant semantic changes whenever possible. I think the recent discussion that Doug has raised around the changes to #file illustrates that—AFAIK the exact format of #file was never documented, but there's a higher bar the change is being held to than "clients who relied on the format were buggy, so it's okay to break them".

Okay, let's do this.

As noted previously, I'm specifically interested in the someQ as? P case (where someQ has static type unrelated to P, e.g., Any).

Suppose we have a utility library Module U which provides a protocol that is like Identifiable, but restricted to Strings, along with some utility methods.

// Module U

protocol StringIdentifiable {
    var identifier: String { get }
}

extension Array where Element: StringIdentifiable {
    func sortedByIdentifier() -> [Self] { ... }
}

Now, I'm writing a module, Module A. I provide a public Gadget type, with an internal identifier that I use to implement some algorithms. I also utilize an internal conformance to StringIdentifiable to enable some of these algorithms. A Gadget is useful for many things, and one of the things that I use it for is to track some internal state that clients can trigger through a StateTracker protocol:

// Module A

import U

public class Gadget {
    /*internal*/ var state: Int = 0

    public init() {}
    public func triggerGadget() { state += 1 }
}

extension Gadget: internal StringIdentifier {
    var identifier: String {
        return "\(state)"
    }
}

public extension Array where Element == Gadget {
    /// Sort the array by the number of times the gadget has been triggered.
    func sortedByTriggerCount() -> [Self] {
        return sortedByIdentifier()
    }
}

public protocol StateTracker {
    func trackStateEvent()
}

extension Gadget: StateTracker {
    func trackStateEvent() {
        triggerGadget()
    }
}

var globalStateTracker: StateTracker = Gadget()

A client, writing their program in Module B, doesn't like global state floating around everywhere, so they decide to create a dependency container. Most of their dependencies conform to StringIdentifiable. In fact, globalStateTracker is the only one that doesn't! Grr. On a lark, they think, "Hmm, StateTracker is an existential, so of course it can't conform to a protocol, but maybe the underlying type does!". So they write:

// Module B

import U
import A

class DependencyContainer {
    private var dependencies: [String: StringIdentifiable] = [:]

    var globalStateTrackerIdentifier: String?
    var globalStateTracker: StateTracker? { 
        return globalStateTrackerIdentifier.flatMap { dependencies[$0]) 
    }

    // ...
}

func initializeDependencyContainer() -> DependencyContainer {
    let dependencyContainer = DependencyContainer()
    // ...
    if let globalStateTrackerIdentifiable as? StringIdentifiable {
        dependencyContainer.add(globalStateTrackerIdentifiable)
        dependencyContainer. globalStateTrackerIdentifier = globalStateTrackerIdentifiable.identifier
    }
    // ...
}

To their surprise, it works! This code is promptly forgotten about and never touched again.

Now, as the author of Module A, I realize I've done something terribly silly. My internal conformance to StringIdentifiable is totally broken! Not only does the "identity" of a gadget change in a way that doesn't really make sense over the course of its lifetime, but my sortedByTriggerCount method doesn't even sort properly. I have to fix it.

I've exposed this method publicly, and I don't want to break my clients who may have discovered the bug and implemented workarounds or changed their systems to rely on the broken ordering. So instead, I deprecate the old method and introduce a new one:

// Module A

public extension Array where Element == Gadget {
    /// Sort the array by the lexicographic ordering of the string (in base 10) representing the number of times the gadget has been triggered.
    @available(*, deprecated, message: "Use 'sortedByTriggerCountActually' instead")
    func sortedByTriggerCount() -> [Self] {
        return sorted { "\($0.state)" < "\($1.state)" }
    }

    func sortedByTriggerCountActually() -> [Self] {
        return sorted { $0.state < $1.state }
    }
}

My initial use of StringIdentifiable was totally misguided, so I decide to remove it. This appears to be a safe change to make—the conformance was internal and I've removed the only place in my program where I was depending on a conformance of Gadget to StringIdentifiable. The new version of the library ships, and Module A breaks.

Had I known that clients were relying on this conformance, I never would have removed it. Indeed, if it were a public conformance I would have never even considered it!

Let me know if anything here is unclear, happy to expand on any part that needs it.

1 Like

But the same sort of contract applies between a protocol and its conforming types as applies between a base class and its subclasses.

In the current compiler, you get a warning when it's known that you're conditionally casting from a type to itself, so you're unlikely to ever do in such a simple case. But how about this case when T == P?

func f<T>(someP: P, T.Type) -> T { someP as? T }

If you want that to succeed unconditionally when T == P, I'm a little disturbed by the nonuniformity of the semantics of f in that case.

I'll admit that when it comes to dynamic casts I have little sense of what principles they should uphold, other than general design rules like it “avoid non-uniformity” or “keep the language simple.”

The semantics you've proposed don't necessarily make any more sense to me than other possible choices. So in favor, I have “that's what @Jumhyn wants,” but what I'd really like is to be able evaluate any semantics in terms of their utility, what they allow programmers to express, or whether code using those semantics is misleading. So far, I've got no read on any of those aspects (for any possible semantics), so where I'm at right now is leaning toward the simplest, most uniform semantics we can come up with.

[schnipp a long story—thanks for writing it!]

OK, I understand your story. I think if a library vends existentials and wants to protect against such probing by clients, it already has to go pretty far out of its way. For example, you can't change the vended type from a class to a struct because that's discoverable by dynamic casting to AnyObject. You can't allow it to (publicly) conform to any public protocols. I understand the risks you're pointing at, but they seem relatively minor to me. If there's a good design that helps minimize those risks, I'm not opposed, but I would prefer to drive the design on the basis of more compelling criteria if possible.

That said, as I noted above, I don't have much of a feeling for what choices would be good in these scenarios.

1 Like

Agreed. But this line of discussion was based around your analogy between an internal subclass and an internal conformance. I'm trying to draw a distinction between the types of breaks that can occur if that internal subclass was removed and the types of breaks that can occur if the internal conformance was removed. The internal class is not easily discoverable with first-class language features, whereas the internal conformance is easily revealed via dynamic casts.

That's a good example and I see why that gives you pause. It's not as concerning to me that there would be some amount of special-casing in the situation where you're performing a dynamic cast to the static type of a variable—IMO it's reasonable that that is treated specially in such a way that it always succeeds.

Good enough for me! :wink:

Very reasonable.

Yeah, I agree with this sentiment. The generalized principle I'm reaching for here is something to the effect of,

Language features described as a "scoped" item should not have any API impact outside the specified scope (i.e., removing/modifying that item should not break clients external to the scope).

You've noted some ways in which this model is already imperfect, but IMO this is relatively learnable as, "protocol conformances inherently have the visibility of the specified protocol because they can be discovered via dynamic casts." To introduce a new feature which purports to enable an internal conformance (but still allows discovery of that conformance external to the module), rises to the level where I feel that "code using those semantics is misleading."

Now that I've fully explicated my discomfort with this aspect of the design, I'll sit back for a bit and let others chime in on this discussion. Thank you for indulging me through all of that!