Sealed protocols

(Brent Royal-Gordon) #101

How is a non-internal type supposed to fulfill a protocol requirement that isn’t visible to it?

(Matthew Johnson) #102

Non-public requirements definitely wouldn't obviate the benefit of being able to switch in all cases. I really should keep a list of use cases when I run into them but I haven't. I'll try to remember some interesting use cases for switching over conformances I've run into and post the ones I can recall.

It's worth noting that the use cases for switching over protocols is largely orthogonal to the issue of whether conformances are allowed outside the module. The interaction is when a library has a use case for switching and the protocol needs to be public (whether or not the library wants to expose the full set of conformances and ability to switch to users).

It is clear from this thread that there are a lot of people who feel that the current pitch doesn't introduce enough new capabilities to warrant inclusion in the language. Part of the reason for that is that the current design does not lend itself well to some of the things we might want to do with sealed protocols in the future because it does not provide the compiler an easy way to know about all of the conformances that might exist (despite them all being in the same module.

We can address those concerns and position ourselves for enhancing sealed protocols with the ability to switch by shifting our perspective. Instead of viewing sealed as a feature that only applies at the module boundary, I think it's worth viewing it as a feature that allows the compiler to easily have comprehensive knowledge of conformances. The simplest way to do that might be to say sealed means conformances must be declared in the same file as the protocol (although they may be implemented elsewhere). If a conformance must be declared within the same file it obviously cannot be declared outside the module!

This would position us to eventually support exhaustive switch over sealed protocols. If that feature were introduced, then private sealed protocol would make sense, and would allow us to switch over conformances of a non-public protocol without requiring the compiler to do a ton of heavy lifting to understand what the full set of conformances are.

If we accept your proposal as-is, that kind of design becomes impossible because it would be a break in source compatibility to require conformances to sealed protocols to be declared in the same file as the protocol declaration. So I think it's important that we have a broader picture of where we think we might want the design to end up. IMO, this will make the proposal a lot stronger and would hopefully help to overcome some of the resistance you have run into.

(Joe Groff) #103

Non-public protocol requirements are already possible thanks to ABI resilience. Hiding a protocol requirement via visibility is effectively the same as a public protocol requirement getting added later by a future library version from the point of view of a client of the library. The less-than-public requirements would need to have default implementations for external conformers.

(Karl) #104

Perhaps not in all cases, but generally when you're downcasting from a protocol (existential or generic parameter) to a concrete type, it's because you need to access something which isn't available on the protocol interface. And when you need to get too deep in to the type-specific internals, you can flip it and make it a dynamically-dispatched requirement.

For example, suppose you have:

sealed public protocol MyProto {}
public struct ThingOne: MyProto { 
  func a() { /* ... */ }
public struct ThingTwo: MyProto {
  func b() { /* ... */ }

public func someGenericFunction<T: MyProto>(_ value: T) {
  // Some type-specific considerations which require downcasting.
  switch value {
    case let one as ThingOne: one.a()
    case let two as ThingTwo: two.b()

You could also write it like this, without requiring exhaustive switching. It moves the type-specific considerations to the types themselves and keeps your generic code... well, more generic:

sealed public protocol MyProto {
  internal func _doSomeGenericFunction() // non-public req.
public struct ThingOne: MyProto { 
  func a() { /* ... */ }
  func _doSomeGenericFunction() { a() }
public struct ThingTwo: MyProto {
  func b() { /* ... */ }
  func _doSomeGenericFunction() { b() }

public func someGenericFunction<T: MyProto>(_ value: T) {

Still, the main objection I have to designing sealed protocols around exhaustive switching is that it would be limited to this one special flavour of protocol, and wouldn't apply to other subtyping relationships because classes don't follow the same rules. Sealed protocols are most useful in complex value-semantics (or mixed-semantics) models, and having to declare all conformances in a single file is probably more annoying in those situations.

We have whole-module optimisation, but not whole-module typechecking. If such a thing existed, there would be all kinds of cool things we could do with exhaustiveness. For example, you could exhaustively catch in-module without needing typed throws (and going down to the specific cases, not just the type).

Anyway, I can think of some members of the core team who have given presentations about avoiding downcasting in generic code. I suspect they would be very much against adding it ;)

Yes it doesn't affect ABI, but you still need some kind of marker to tell the other module that it shouldn't try to conform. For example in the cases of StringProtocol and _SyntaxBase, you really need those non-public requirements. It's not just an ABI detail - they are actually key parts of the protocol semantics.

Also, I happened to spot another protocol that should be sealed, this time in Foundation: ReferenceConvertible.

It's used for Obj-C bridging and there's a case that it should be underscored (so not the most compelling example, I admit). But people do try and conform anyway and although not supported, it does happen to work.

I'll note that neither the documentation for ReferenceConvertible or Syntax really make it explicit enough that external conformances aren't supported.

(Michel Fortin) #105

Wouldn't it be possible to implement some kind of exhaustive switching today following the same rules as for non-frozen enums? In other words: make the compiler complain about not handling any case that is visible in the current scope and force the addition of an unknown case to handle anything else. No need for the protocol to be sealed.

(Matthew Johnson) #106

I understand this very well. However this is not always what you want. This design has a number of consequences. It requires a standalone function for each pieces of logic that would otherwise be contained in a switch statement, including a common result. It also requires that this function be visible on all concrete conforming types. This approach does not work well for single-use logic that should not have such a prominent position and visibility.

Sealed protocols with exhaustive switching can be used in a way that is more analogous to enums than to classes. This enables use cases where enums just don't work because cases are not types and an enum cannot be used as a constraint on types. In these use cases the protocol is quite similar to an enum where each case has an associated value. Switching over a protocol designed in this way would not be something "dirty" and to be avoided, but rather a central aspect of its design. Part of the semantics of the protocol is the specific set of types that conform, just as part of the semantics of an enum are defined by its cases.

These enum-like protocols would be a Swifty protocol-oriented, (sometimes) value-semantic version of the object-oriented "case classes" we see in other languages like Kotlin and Scala. They are not redundant with enums but rather complementary. The discussion of protocols vs enums earlier in the thread demonstrates how they have much different properties and are therefore applicable in different scenarios. There is a hole in the design space available to us right now and exhaustive switch over sealed protocols would fill it very nicely.

If we could have exhaustive switching without any special linguistic construct that would be fine with me. However, many comments in this thread make it sound like it is unlikely to be feasible and perhaps even unlikely to be attempted to implement exhaustive switch if the compiler needs to scan an entire module to find the full set of conformances. So the reason it would be limited to this one flavor of protocol is pretty good: without this flavor of protocol we wouldn't be able to have the feature at all.

If this is true then sealed protocols are likely to be the one chance we have to realize exhaustive switch over protocols. This is an important type system feature that would open up a new range of design choices that are simply not available without it.

FWIW, if we design sealed protocols in this way there is no reason we couldn't also have sealed classes which would also support exhaustive switch. I think the motivation for this in the OO design space is weaker, especially if we have sealed protocols but there would be an argument for consistency and precedent in the Scala and Kotlin "case classes".

I admit that it would be slightly annoying when the use case is something other than the enum-like protocols I described above. But I also think opening up the design space of enum-like protocols is much more important. Keep in mind that it is only the conformance declaration that would need to be in the same file as the protocol. The implementation of all requirements could live in other files. This would be slightly unfortunate as we usually like to declare the conformance together with the implementation of the requirements, but we don't have to. And again, I think opening up the design space of enum-like protocols is more important.

Sure, but the earlier parts of the thread somewhat strongly imply that such a thing is not likely to ever exist.

This advice makes sense given the current capabilities of protocols. It wouldn't always make sense if the language supported switching over sealed protocols, because in that language we would be able to design enum-like protocols.

The earlier parts of this thread make it sound like this kind of exhaustive checking of visible conformances is problematic to implement and therefore unlikely to be implemented.