Sealed protocols

Evolution Pitches
1

Hope you've all been enjoying your holidays.

I've used a bit of the free time to start implementing a feature that has been bugging me for a while as something sorely missing from Swift. It's come up a few times in various discussions over the years, so I'd like to gather some feedback and put it up for review.

Sealed Protocols

Motivation

Protocols in Swift are a powerful tool to enable type-erasure and generic programming. Libraries often provide public protocols for clients to use for these purposes. As an example, the standard library provides StringProtocol for generic algorithms over both String and Substring, with the following caveat:

Do not declare new conformances to StringProtocol . Only the String and Substring types in the standard library are valid conforming types.

Similarly, protocols can be useful when designing some more complex types, such as Foundation's recently-redesigned Data type and accompanying DataProtocol type. It is not clear whether external conformances to DataProtocol should be allowed or not.

The ability for clients to add arbitrary conformances to any public protocol limits the ability of library authors to evolve those protocols in certain ways (such as adding new requirements/customisation points), unless those new members come with default implementations, which might not be possible in all cases.

Moreover, enabling the compiler to reason about the complete set of types which conform to a protocol can enable certain optimisations when using existentials involving those protocols. Current advice is to mark protocols which are only conformed-to by classes as refining AnyObject, so the compiler can omit logic related to handling non-trivial structures.

Unfortunately, this AnyObject constraint becomes part of the protocol's ABI. If a later version of the library wants to add conformance to a non-class type, removing the AnyObject refinement would be a breaking change. There are also a number of other patterns (such as trivial types, or enums with single, class-type payloads) which do not have AnyObject-style marker protocols and so miss out on potential optimisations. If we were instead to mark the protocol as 'sealed', the compiler could perform these optimisations as an implementation detail within the declaring module.

Detailed Design

I propose to add a new attribute called sealed. A sealed protocol may be made public, but may only be conformed-to from the module it was declared in.

Attempting to conform to a sealed protocol from outside of its declaring module will produce a compiler error.

// Module A.

sealed public protocol ASealedProtocol { /* ... */ }

// Okay.
extension String: ASealedProtocol { /* ... */ }

// --------------
// Module B.

// Error: cannot conform to sealed protocol 'ASealedProtocol' outside of its declaring module.
struct MyType: ASealedProtocol { /* ... */ }

// Error: cannot conform to sealed protocol 'ASealedProtocol' outside of its declaring module.
extension Int: ASealedProtocol { /* ... */ }

Refinements of sealed protocols must also be sealed, which means they too are only possible within the parent's declaring module.

// Module B.

// Error: cannot inherit from sealed protocol 'ASealedProtocol' outside of its declaring module.
protocol SpecialProtocol: ASealedProtocol {}

Other modules may still add extensions on sealed protocols and use them in protocol-compositions.

// Module B.

typealias SpecialThing = ASealedProtocol & SomeOtherProtocol // Okay.

extension ASealedProtocol { /* ... */ } // Okay.

open classes may conform to sealed protocols, which gives libraries a powerful way to provide selective customisation of a particular conformance.

Source and ABI Impact

Adding the sealed attribute to an non-sealed protocol is potentially source- and binary-breaking, as clients may have written conformances which will no longer compile, and the declaring module may be optimised in a way which is not compatible with those external conformances.

It is a source- and binary-compatible change to 'unseal' a sealed protocol and open it up to external conformances. This mirrors how the open attribute works for classes.

As previously discussed, there are a number of protocols in the standard library and Foundation which may choose to take advantage of this attribute to enforce expectations which are currently only documented in comments. There is the potential to break some projects which previously ignored the documentation, but it is probably an acceptable risk. They were warned.

Future Directions

Restricting external conformances opens the door for protocols to include non-public requirements, involving non-public types. Taking StringProtocol as an example, a requirement could be added to access the "internal" _StringGuts without ugly hacks. As for other protocol requirements, the compiler would check that all conforming types implement it, and it would be usable from any publically-visible function constrained to StringProtocol, without downcasting to a concrete type or internal refined protocol.

Alternatives Considered

  • Making sealed the default, and adding an open attribute (as we do for classes).

    In Swift, we want to make library evolution easy by making as few commitments as possible by default, and requiring library authors to opt-in to anything which might restrict them in the future. That would indicate that making public protocols be sealed by default is the right thing to do. Unfortunately, such a change would be massively source-breaking at this point, so it's pretty much a non-starter.

  • Ban open members from witnessing sealed protocol requirements.

    On the one hand, allowing conformances to be partially/totally overridden might be seen as breaking the seal. On the other hand, allowing it does not limit the ability of library authors to evolve the protocol in question (which is the primary motivation behind this feature), and expands the classes which may participate to include classes from 3rd-party/system libraries.

  • Bikeshedding.

    closed would also be okay, and more closely mirror the open keyword for classes. Previous discussions seemed to settle around the name sealed, but it would be interesting to see which name the community prefers.

  • Do nothing.

    Always an option :slight_smile: , but undesirable because it means library authors cannot publish a protocol to be used for type-erasure or generic programming without committing to only ever add requirements that can be given default implementations.

28 Likes
2

I think this restriction is irrelevant to the proposed core feature because of retroactive conformances. I think we should allow refinement of sealed protocols outside the parent module, and restrict the conformance to the new protocols only to the module where the parent sealed protocol comes from.

// Module B

// this should be okay
protocol SpecialProtocol: ASealedProtocol {}

// Error: cannot conform to protocol 'SpecialProtocol' refining a sealed protocol 'ASealedProtocol' outside of its declaring module.
struct MyTypeFromB: SpecialProtocol {} 

// Retroactive conformance should be okay
extension MyType: SpecialProtocol {}

That particular case was discussed in previous threads, I'll link those if I find them.

I always wanted this feature and I wanted it the right way from the start. In my eyes it should be as simple as open / public protocol, but this is such a pity story in Swift's history during the late pre Swift 3 timeframe where open class was introduced. Even during the review of that feature people raised concerns of the misalignment it would generate for protocols because public on protocols would mean that you can conform to the protocol outside the parent module while on classes the analog behavior of subclassing was 'sealed' under the public access modifier and can only be opened by using the open access modifier. Now the defaults are flipped for protocols and open is an exclusive access modifier for classes which in my opinion is loosey goosey.

Every time I brought up this topic, everyone shouted at me that the ship is sailed to introduce the convenient way of expressing this functionality as open / public protocol, so I guess we cannot find a way to achieve it in a source compatible and non mind bending way such as introducing open protocol and migrating every public protocol to it, then forbid somehow public protocol usage until it can be re-invented in the sealed way.

If that still holds true, and the current behavior is flipped I think the only convenient route we can go now would be by introducing another exclusive access modifier just for protocols.

classes protocols
open class public protocol
public class sealed protocol

That way sealed would be another access modifier, which like open would used for publishing API outside the parent module. Only this time it will restrict something rather than permit something (compared to open).

Here are a few related topics I could find for the readers of this thread:

4 Likes
3

Quite interested to see the links. Sorry if I sound dense right now, but why would it be important for other modules to refine a protocol they cannot conform to?

What would the purpose of that extension be? Default implementation added by an outside module?

4

Quickly brainstorming 'one' scenario:

// Module A
sealed protocol P { func foo() }

public struct S : P { 
  public func foo() { ... }
}
public struct T : P {
  public func foo() { ... }
  public func bar() { ... }
}

// Module B
import A
protocol Q : P { func bar() }

// We can only extend `T` but not `S`
extension T : Q {}

func useOnQ(_ q: Q) {
  q.foo()
  q.bar()
}

let s = S()
let t = T()

useOnQ(s) // error 'S' does not conform to `Q`
useOnQ(t) // okay

I could imagine something like this to be very handy and convenient. Since sealed protocols are still visible from the child protocols you should be able to work with them. Retroactive conformances do not break any of the sealed semantics so it should be fine to not restrict those away. The other most common use of sealed protocols outside the parent module would be an 'interface-like' usage as existentials or in generics.

1 Like
5

Wouldn’t sealed block the conformance specified in module B for sealed protocol P?

6

I'm not sure I understand the question correctly. For me sealed should only block the conformance outside the parent module to outside types, but not prevent from further refinement or retroactive conformances.

That said if from the above example module A had another type without any conformances but with the same API surface.

// module A
struct R {
   func foo() { ... }
   func bar() { ... }
}

It should be theoretically be possible to conform it to the sealed protocol declared in the same module but done from a different module.

// module B

// Conform to the sealed protocol declared in module A
extension R : P {}
// Conform to a refined sealed protocol, since both 
// `R` and `P` are from module A, the conformance is allowed.
extension R : Q {}

Even though this is considered an anti pattern, this is the result of allowing retroactive conformances in general. That said, I think introducing sealed protocols with banned retroactive conformances and no abilities of further refinement would be only a drawback of the core functionality.

On the other hand it's debatable if the retroactive conformance of sealed protocols should only be allowed on types that already conform to the sealed protocol. That would mean that both extensions I just mentioned wouldn't be valid, but those in my previous post will still be accepted by the compiler.

In other words, you can only retrofit a refined protocol on a type that already conforms to the refinement sealed protocol.

Does this answer your question?

7

No, this should not be allowed. It defeats the entire feature.

  • The author of module A is no longer able to add requirements to P without a default implementation
  • It is no longer possible to add non-public requirements to P
  • Within module A, the compiler is no longer able to make assumptions about which types conform to P

It does not matter where R is defined - it might be defined in the standard library, or a system library such as UIKit. What matters is that the conformance must be in the same module as P.

Refining sealed protocols outside of their declaring module would add a fair amount of complexity which I'm not sure is worth it. You could always add a second protocol and write extensions on P where Self: Q.

I think it's best to go for the most-restrictive version to start with, because we can loosen that rule in future if there is significant demand. I think that non-public requirements are more interesting, and would rather work on that before bothering with refinements in other modules.

5 Likes
8

Yeah because of that I wrote that we can also consider allowing retroactive conformances only on types that already conform to the sealed type.

Isn't this the other way around then?

// module B
protocol Q { ... }
extension Q where Self : P { ... }

extension T : Q {}

I cannot tell how much complexity it really adds for the implementation, but ideally I would like to have support for refinement from the start, however I can accept the mentioned workaround knowing we can / will implement the rest later. This feels too similar to how people invented protocol A where Self : SomeClass for the missing protocol A : SomeClass.

9

I don't think there's any meaningful difference between extension P where Self: Q and extension Q where Self: P.

I'll investigate the refinement issue then - maybe it's not so difficult. It's just that experience has taught me that evolution proposals are best kept to the minimal-viable product, and expanded on in follow-up proposals as needed. As you showed in your previous post - this feature has been pitched plenty of times in the past, but always seems to get forgotten/abandoned.

2 Likes
10

I’m really happy to see that you have picked up this topic and are working on an implementation!

I agree with Adrian that restricting refinement outside the module is an unnecessary restriction. Since you’re doing the implementation, if it would take too much effort I would be happy to see progress without that capability. But if it’s feasible I wouldn’t worry about it being a problem for the SE proposal. The reason this topic hasn’t gone anywhere is that nobody has decided to work on implementation, not because of any specific design details.

One topic you don’t mention is including support for exhaustive switch over closed protocols. This is something that would be very nice to have. Have you given it any thought? Are you willing to explore including this in the initial proposal?

2 Likes
11

Does it have any performance optimization effect? or it does solely restrict developers?

1 Like
12

This is a small minor thought; a bit of a bikeshed. I'm not a fan of the proliferation of keywords. So...

I'd personally prefer to see these referred to as final protocol instead of sealed protocol. We already have the keyword final to refer to a class that can be used outside of a module (assuming it's public), but it cannot be subclasses. I think it makes sense to refer to protocols as final if they're visible outside of a module, but cannot be adopted.

However, like final class, I would expect a final protocol to allow me to create extensions.

7 Likes
13

final doesn’t make sense here. A final class has no subclasses. On the other hand, a protocol is useless without conforming types. There simply is not a clean analogy between allowing no subclasses with limiting conformances to the declaring module.

Further, as @DevAndArtist already explained, it is perfectly reasonable to refine a closed protocol outside its declaring module. Conformances to the refined protocol would necessarily be limited to public types provided by the declaring module for which a conformance to the closed protocol is already provided. Despite that limitation they could be a useful tool to have in the toolbox.

So a closed protocol is not “final” in the sense that it has no conforming types and it is not “final” in the sense that it cannot be refined (even outside the module). Using final here would introduce a related but significantly different meaning for that keyword. That is potentially quite confusing and is far worse than introducing a new keyword.

1 Like
14

I'd love to have sealed protocols in Swift. When you'd want to use a sealed protocol, there's usually no other solution that works well.

But there's one more piece that I'd love to see: the ability to switch over an instance of the protocol and exhaustively check the concrete implementations:

sealed protocol P { func foo() }

public struct S : P { 
  public func foo() { ... }
  public func baz() { … }
}
public struct T : P {
  public func foo() { ... }
  public func bar() { ... }
}

func f(_ p: P) {
  switch p {
  case is S: // Not sure what this should look like
    p.baz()  // Too bad Swift doesn't restrict `p` here like TypeScript
  case is T:
    p.bar()
  // No default needed because P is sealed.
  // The compiler knows it's either S or T.
  }
}
15 Likes
15

+1. This is the exhaustive switch I mentioned upthread.

2 Likes
16

What if we were just to allow this without introducing the sealed keyword. Wouldn't this prevent types which can't see those requirements from conforming? (Basically you gain the effect of sealed without having to add a new (inconsistent) axis level)

17

The restriction against refinement seems arbitrary to me. It would cause only one of these to become illegal, though both do the same thing:

protocol Refined : ASealedProtocol {}
protocol Refined where Self : ASealedProtocol {}

The following is what I would expect when trying to use either definition:

// Okay.
extension String: Refined { /* ... */ }
// (Conformance to ASealedProtocol was already there.)

// Error: cannot conform to sealed protocol 'ASealedProtocol' outside of its declaring module.
extension Int: Refined { /* ... */ }
// (The implied new conformance to ASealedProtocol is illegal.)

And for the same reason:

// Error: cannot conform to sealed protocol 'ASealedProtocol' outside of its declaring module.
struct NewType: Refined { /* ... */ }
// (The implied conformance to ASealedProtocol is illegal.)
1 Like
18

+1 to this feature!

I've ended up with the ugly design in the past of having a public protocol with methods that a type in an external module couldn't possibly conform to if it actually tried - something like this:

public protocol SomeProtocol {
    // implementing this requires SomeType.bar(x:)
    public func foo(input: SomeType) 
}

public struct SomeType {
    internal func bar(x: SomeInternalType) -> SomeOtherInternalType { /* .... */ }
}

Calling SomeType.bar is necessary to actually implement the protocol, but the method uses types that are not meant for public consumption. Yet SomeProtocol needs to be public because various library methods take in and return conforming values.

This design leaves it to the user to figure out they can't actually fulfill the protocol requirements, and could give them the false impression that they are correctly implementing it because their code compiles. It would be great if the compiler could tell them right off the bat that their type can't conform.

Also +1 to @anandabits' exhaustive switch idea.

2 Likes
19

As mentioned in the pitch, it frees the compiler to make more optimisations if it wants to. Right now, the compiler can't assume anything about the types which conform to a public protocol, unless those restrictions are a formal part of the protocol's contract (by inheriting from AnyObject). sealed gives the compiler a way to collect informal knowledge about the types which conform to a public protocol, and make optimisations within the declaring module.

This feature does not "solely restrict developers" - it restricts some developers, but also removes restrictions from others. Most importantly - it allows library authors to add requirements to their public protocols in minor updates to their libraries.

Once the protocol has been in use for a while, the interface is settled and the library author does not expect any more changes, they may decide to open it to external conformances without breaking source or binary compatibility.

That would effectively be the same as 'sealed', but sealed protocols are useful even if they don't have non-public requirements. The most important being library evolution, as described above.

This is all great feedback so far - thanks, everybody! It seems like there is definitely some demand to be able to refine sealed protocols.

I'll also investigate exhaustive switching - but I don't expect that to be very easy due to the way the compiler is designed. I can't think of any other language features which use derived information from the entire program like that (enum cases are all neatly contained in their declarations, not scattered across the module). I'd certainly welcome anybody who wants to help/mentor.

1 Like
20

I'm not actually opposed to "sealed" protocols, but I don't think they are important enough to justify yet another keyword:
If we would adopt public/open, I wouldn't mind a new variant of protocols, as it would be the same concept that already exists for classes; imho anything else is just an inflexible and verbose workaround for the lack of union types (typealias SomeNumber = Int | Double).

I know this feature is on the list of commonly rejected proposals (although there's no reason given at all), but I'd rather not have that capability at all than adding it in a way that favors backwards compatibility over simplicity and consistency.
When open was introduced, it was decided that this change shouldn't affect protocols, and the status quo proves that we can get along fine without sealed protocols.