Sealed protocols

Evolution Pitches
101

How is a non-internal type supposed to fulfill a protocol requirement that isn’t visible to it?

102

Non-public requirements definitely wouldn't obviate the benefit of being able to switch in all cases. I really should keep a list of use cases when I run into them but I haven't. I'll try to remember some interesting use cases for switching over conformances I've run into and post the ones I can recall.

It's worth noting that the use cases for switching over protocols is largely orthogonal to the issue of whether conformances are allowed outside the module. The interaction is when a library has a use case for switching and the protocol needs to be public (whether or not the library wants to expose the full set of conformances and ability to switch to users).

It is clear from this thread that there are a lot of people who feel that the current pitch doesn't introduce enough new capabilities to warrant inclusion in the language. Part of the reason for that is that the current design does not lend itself well to some of the things we might want to do with sealed protocols in the future because it does not provide the compiler an easy way to know about all of the conformances that might exist (despite them all being in the same module.

We can address those concerns and position ourselves for enhancing sealed protocols with the ability to switch by shifting our perspective. Instead of viewing sealed as a feature that only applies at the module boundary, I think it's worth viewing it as a feature that allows the compiler to easily have comprehensive knowledge of conformances. The simplest way to do that might be to say sealed means conformances must be declared in the same file as the protocol (although they may be implemented elsewhere). If a conformance must be declared within the same file it obviously cannot be declared outside the module!

This would position us to eventually support exhaustive switch over sealed protocols. If that feature were introduced, then private sealed protocol would make sense, and would allow us to switch over conformances of a non-public protocol without requiring the compiler to do a ton of heavy lifting to understand what the full set of conformances are.

If we accept your proposal as-is, that kind of design becomes impossible because it would be a break in source compatibility to require conformances to sealed protocols to be declared in the same file as the protocol declaration. So I think it's important that we have a broader picture of where we think we might want the design to end up. IMO, this will make the proposal a lot stronger and would hopefully help to overcome some of the resistance you have run into.

1 Like
103

Non-public protocol requirements are already possible thanks to ABI resilience. Hiding a protocol requirement via visibility is effectively the same as a public protocol requirement getting added later by a future library version from the point of view of a client of the library. The less-than-public requirements would need to have default implementations for external conformers.

2 Likes
104

Perhaps not in all cases, but generally when you're downcasting from a protocol (existential or generic parameter) to a concrete type, it's because you need to access something which isn't available on the protocol interface. And when you need to get too deep in to the type-specific internals, you can flip it and make it a dynamically-dispatched requirement.

For example, suppose you have:

sealed public protocol MyProto {}
public struct ThingOne: MyProto { 
  func a() { /* ... */ }
}
public struct ThingTwo: MyProto {
  func b() { /* ... */ }
}

public func someGenericFunction<T: MyProto>(_ value: T) {
  // Some type-specific considerations which require downcasting.
  switch value {
    case let one as ThingOne: one.a()
    case let two as ThingTwo: two.b()
  }
}

You could also write it like this, without requiring exhaustive switching. It moves the type-specific considerations to the types themselves and keeps your generic code... well, more generic:

sealed public protocol MyProto {
  internal func _doSomeGenericFunction() // non-public req.
}
public struct ThingOne: MyProto { 
  func a() { /* ... */ }
  func _doSomeGenericFunction() { a() }
}
public struct ThingTwo: MyProto {
  func b() { /* ... */ }
  func _doSomeGenericFunction() { b() }
}

public func someGenericFunction<T: MyProto>(_ value: T) {
  value._doSomeGenericFunction()
}

Still, the main objection I have to designing sealed protocols around exhaustive switching is that it would be limited to this one special flavour of protocol, and wouldn't apply to other subtyping relationships because classes don't follow the same rules. Sealed protocols are most useful in complex value-semantics (or mixed-semantics) models, and having to declare all conformances in a single file is probably more annoying in those situations.

We have whole-module optimisation, but not whole-module typechecking. If such a thing existed, there would be all kinds of cool things we could do with exhaustiveness. For example, you could exhaustively catch in-module without needing typed throws (and going down to the specific cases, not just the type).

Anyway, I can think of some members of the core team who have given presentations about avoiding downcasting in generic code. I suspect they would be very much against adding it ;)

Yes it doesn't affect ABI, but you still need some kind of marker to tell the other module that it shouldn't try to conform. For example in the cases of StringProtocol and _SyntaxBase, you really need those non-public requirements. It's not just an ABI detail - they are actually key parts of the protocol semantics.

Also, I happened to spot another protocol that should be sealed, this time in Foundation: ReferenceConvertible.

It's used for Obj-C bridging and there's a case that it should be underscored (so not the most compelling example, I admit). But people do try and conform anyway and although not supported, it does happen to work.

I'll note that neither the documentation for ReferenceConvertible or Syntax really make it explicit enough that external conformances aren't supported.

2 Likes
105

Wouldn't it be possible to implement some kind of exhaustive switching today following the same rules as for non-frozen enums? In other words: make the compiler complain about not handling any case that is visible in the current scope and force the addition of an unknown case to handle anything else. No need for the protocol to be sealed.

106

I understand this very well. However this is not always what you want. This design has a number of consequences. It requires a standalone function for each pieces of logic that would otherwise be contained in a switch statement, including a common result. It also requires that this function be visible on all concrete conforming types. This approach does not work well for single-use logic that should not have such a prominent position and visibility.

Sealed protocols with exhaustive switching can be used in a way that is more analogous to enums than to classes. This enables use cases where enums just don't work because cases are not types and an enum cannot be used as a constraint on types. In these use cases the protocol is quite similar to an enum where each case has an associated value. Switching over a protocol designed in this way would not be something "dirty" and to be avoided, but rather a central aspect of its design. Part of the semantics of the protocol is the specific set of types that conform, just as part of the semantics of an enum are defined by its cases.

These enum-like protocols would be a Swifty protocol-oriented, (sometimes) value-semantic version of the object-oriented "case classes" we see in other languages like Kotlin and Scala. They are not redundant with enums but rather complementary. The discussion of protocols vs enums earlier in the thread demonstrates how they have much different properties and are therefore applicable in different scenarios. There is a hole in the design space available to us right now and exhaustive switch over sealed protocols would fill it very nicely.

If we could have exhaustive switching without any special linguistic construct that would be fine with me. However, many comments in this thread make it sound like it is unlikely to be feasible and perhaps even unlikely to be attempted to implement exhaustive switch if the compiler needs to scan an entire module to find the full set of conformances. So the reason it would be limited to this one flavor of protocol is pretty good: without this flavor of protocol we wouldn't be able to have the feature at all.

If this is true then sealed protocols are likely to be the one chance we have to realize exhaustive switch over protocols. This is an important type system feature that would open up a new range of design choices that are simply not available without it.

FWIW, if we design sealed protocols in this way there is no reason we couldn't also have sealed classes which would also support exhaustive switch. I think the motivation for this in the OO design space is weaker, especially if we have sealed protocols but there would be an argument for consistency and precedent in the Scala and Kotlin "case classes".

I admit that it would be slightly annoying when the use case is something other than the enum-like protocols I described above. But I also think opening up the design space of enum-like protocols is much more important. Keep in mind that it is only the conformance declaration that would need to be in the same file as the protocol. The implementation of all requirements could live in other files. This would be slightly unfortunate as we usually like to declare the conformance together with the implementation of the requirements, but we don't have to. And again, I think opening up the design space of enum-like protocols is more important.

Sure, but the earlier parts of the thread somewhat strongly imply that such a thing is not likely to ever exist.

This advice makes sense given the current capabilities of protocols. It wouldn't always make sense if the language supported switching over sealed protocols, because in that language we would be able to design enum-like protocols.

The earlier parts of this thread make it sound like this kind of exhaustive checking of visible conformances is problematic to implement and therefore unlikely to be implemented.

107

Under Future Directions, please consider adding the ability to explicitly restrict what may conform to a protocol:

//Something along these lines
sealed protocol Suit as Hearts, Diamonds, Clubs, Spades {}
struct Hearts : Suit {}
struct Diamonds : Suit {}
struct Clubs : Suit {}
struct Spades : Suit {}
108

That is not how sealed protocols should work. They will behave as normal protocols, the compiler can just provide more staticly observed information to the developer and user and potentially enabled exaustive switching over conforming types, but even that is not the core feature of sealed protocols.

1 Like
109

As discussed further upthread, to practically get any of that benefit, the compiler needs the list of conformances up front. Otherwise, it's impossible to know locally whether there are private or local types that conform to a protocol in other files without scanning the entire module, which would be a prohibitive compile-time cost.

110

Up front or at least declared in the same file, right? One benefit of the same-file approach is that it doesn’t require any syntactic additions to the language.

111

Perhaps, but then sealed has its own bespoke access control rules, which is also weird.

To bring this back to the previous discussion, if we're going to have a feature like this, we need to know what problems we're trying to solve with it, since that impacts what the design should be.

1 Like
112

I don't think sealed protocols have anything to do with exhaustive down-casting. As I've said several times before:

  • That's much more of an edge-case feature than sealed protocols (I couldn't find any other languages which have it).
  • If it is a problem, it extends beyond protocols to other kinds of subtyping relationships, like class hierarchies.
  • It has nothing to do with the motivation behind 'sealed', which is to future-proof your protocols by limiting conformances outside the defining module, not within the module.
  • It either makes the compiler far more complex, or requires tradeoffs like declaring all conformances within the same file (which I frankly find unacceptable).

I really, really do not want to litigate this point any more. It can be proposed as an orthogonal feature in the future.

Some more data points:

  • Rust has sealed traits, because it allows public traits to inherit from arbitrary non-public traits. Public protocols in Swift cannot inherit from non-public protocols (although sealed protocols likely could).
  • You can check out some of the things they do with it here. The Rust Syn library is a popular user (something like SyntaxKit, I suppose), but also the Serde JSON library, some itoa implementations, the Rocket web framework. Again, since Rust allows public traits to inherit from arbitrary non-public traits, this query may not be exhaustive

We could copy Rust's approach for Swift (so: add inheritance from non-public protocols and have everybody define their own Sealed protocol), but that would be a significantly worse solution than what is proposed here. In particular, removing the parent and 'un-sealing' the protocol would become ABI-breaking.

I'm not sure if we could hide non-public parents from the protocol's ABI. Maybe? probably? ¯\_(ツ)_/¯

As for use-cases, here are a bunch of Swift protocols which definitely/probably should be sealed:

Stdlib:

  • StringProtocol
  • CVarArg
  • UnicodeCodec (?)
  • I could imagine a design where SIMDStorage was sealed and SIMDScalar was not (to allow CGFloat and other types to be used as scalars, but only if they used one of the built-in SIMD types).

It's also worth noting that the standard library hacks around this by having a lot of public protocols which are hidden with underscores (e.g. _Pointer, _UTFParser, _UTFEncoding, _ObjectiveCBridgeable, etc). If that wasn't possible (like it isn't for 3rd party libraries), we surely would have added this a long time ago.

Take a closer look at UnicodeCodec, for instance - the parent protocol Unicode.Encoding is a typealias for the underscored protocol _UTFEncoding.

SyntaxKit:

  • Syntax (the library's main protocol)

SwiftPM (it's a library too, y'know):

  • BuildSettingsCondition (only used for erasure in BuildSettings.Assignment. Custom conditions will not work)
  • Traceable (only used for erasure in GeneralTraceStep type)

... and a bunch of stuff in Foundation like ReferenceConvertible, but I think I've made the point.

4 Likes
114

Here’s a rough sketch of the kind of problem that can only be solved with a sealed protocol.

First, we have a basic ID type that is used to model identifiers in a way that scopes them appropriately to an entity.

public protocol IdentifierScope {
    associatedtype RawIdentifier: Hashable
}
public struct ID<Scope: IdentifierScope> {
    let rawIdentifier: RawIdentifier
}

Then we have a refining protocol which is used to associate a fixed set of scopes / entities that are all related to the same domain. For the sake of discussion, assume this domain has a handful of entities but not an enormous number of them.

public sealed protocol MyDomain: IdentifierScope {}

Now I have some behavior exposed by my library which requires a different implementation for different scopes / entities.

public func doSomethingInMyDomain<Scope: MyDomain>(with id: ID<Scope>) {
    switch Scope.self {
    ...
    }
}

Of course in theory this behavior could be dispatched via the MyDomain protocol if we had non-public requirements. However, that is undesirable as it would significantly bloat the protocol with requirements that are really unrelated to the semantics it is intended to express. Instead, we have a small-ish domain and would prefer to implement the behaviors locally using a switch statement. Enums are clearly not an option in this case - we need a protocol and we need the ability to switch over it without having to provide a default clause with a fatalError().

3 Likes
115

We probably only get one chance to design sealed protocols. If exhaustive switch over conformances was truly an orthogonal feature I would leave the discussion for the future, but it is not an orthogonal feature. That means it is relevant to this discussion even if you would prefer it wasn’t.

To be clear: I’m not asking you to impelement exhaustive switch or to even include it in your proposal. What I am asking for is a design that leaves space for exhaustive switch to be added in the future.

It sounds like exhaustive switch will not be possible if you allow conformances to be declared anywhere in the module and do not require the types to be listed alongside the protocol declaration. In order to implement exhaustive switch the compiler needs to be able to determine the set of conformances when it reads the file containing the protocol declaration.

I will continue to argue for a sealed protocol design that is compatible with exhaustive switch. If you can demonstrate a viable future direction for exhaustive switch that is compatible with your proposal as written perhaps you can change my mind. But I am unable to think of one that would be acceptable if it required additional syntax beyond the design for sealed. Maybe it’s just my lack of imagination - if that is the case, please illuminate me. :slight_smile:

1 Like
116

You will need additional syntax (if only to list the conforming types). And you're probably going to want that syntax to work for classes as well. It has nothing to do with sealed which - again - is all about out-of-module conformances. It is not about organising your own code within your own module.

I completely reject that a plain sealed modifier (with no qualifications) should limit conformances to the same file. That's taking things in a different direction entirely.

A poster suggested an possible syntax for restricted in-module subtyping literally a few posts above:

It could also be done with some kind of attribute. All of that can be done as a later extension if there is demand for it.

Please start a new thread if you want to continue talking about exhaustive downcasting. Sealed protocols might help you do that in the case of one kind of decl because it provides a protocol equivalent to open/public, but it is not the motivation behind the feature. I feel the discussion is being hijacked by this.

1 Like
117

I have to agree with @anandabits that exhaustive downcasting is not an entirely orthogonal feature, so we have to discuss whether any design for sealed protocols will make it possible.

118

The onus is on the other side to show how their future plans may be blocked by this proposal. Am I supposed to go through every conceivable spelling of every feature that anybody would like?

Comments like this have it entirely backwards:

Show me how this would preclude exhaustive downcasting and I’ll be happy to adjust the proposal to accommodate. I can’t accommodate a wish-list with no concrete design.

119

Adding a new codec outside the standard library is cromulent. I’ve actually done it myself.

RangeExpression is designed as an extension point.

120

There are no sides here. We're all working together to discuss the design space for a proposed new feature. The constraint we have here is that exhaustive downcasting will likely require all conforming types to be enumerated in the same file as the protocol declaration. It doesn't serve to advance the discussion to "completely reject" this constraint and the use case it supports.

2 Likes
121

Unless I have a completely incorrect interpretation of the word... wrong, of course there are different sides here. Imho it's also ok if the starter of a pitch has limited interest in certain paths of discussion, and wants to advance it in another direction:
I think there are already some cases where people got really frustrated by endless threads, and that's rather unfortunate.

"Endless" discussions may increase the chances of turning a pitch into something that ends up in a change of Swift, and especially in the early phase, I would try to see each negative reaction as a possibility to improve your own arguments — but it should be the authors freedom to decide when to give up and accept that he won't convince everyone.

3 Likes