[Pitch] Opaque Conformance for Protocols

Problem:

In Swift, we currently have no native way to expose a protocol-conforming types without allowing external conformances. This limits our ability to expose existentials like any MyCodeCompatible without risking unwanted conformances (see "uninvited guest" problem recap at the end of the pitch). While there’s a working workaround using @_spi (also discussed at the end of the pitch), Swift may limit SPI in the future, potentially causing this workaround to produce errors.

Solution:

Introducing opaque conformance, where:

  • Protocols remain at a stricter access level (f.i. internal), hidden from users.
  • An existential type (e.g., any MyCodeCompatible ) can be exposed with a higher access level using a typealias, like public typealias AnyMyCodeCompatible = any MyCodeCompatible .
  • Users can work with the existential type in collections or other APIs, without being able to conform their own types to the internal protocol.

Key Benefits:

1. API Design Flexibility:

  • Framework authors can expose collections or APIs that work with existential types without exposing the protocol itself.
  • For example, handling a collection of AnyMyCodeCompatible:
public var items: [AnyMyCodeCompatible] = []

// Users can interact with this collection, but they can't conform new types to `MyType`.

2. Strict Conformance Control:

  • Only predefined types inside the module can conform to the protocol, preventing unwanted external conformances while still allowing use of the existential.

3. Cleaner API

  • Expose only the types you want users to work with, while keeping internal implementation details hidden. This prevents accidental misuse or inappropriate conformances and avoids exposing internal types unnecessarily.

Example Use Case:

// Internal protocol, only visible within the framework.
internal protocol MyCodeCompatible {
    // Will never be public
    static var associatedSecret: MySecret { get }
}

// Internal type needed for protocol conformance.
internal struct MySecret {
    let secret: String
    init(_ secret: String) {
        self.secret = secret
    }
}

// Only these types can conform to MyCodeCompatible.
extension Int: MyCodeCompatible {
    static var associatedSecret: MySecret { .init("This is an Integer") }
}

extension String: MyCodeCompatible {
    static var associatedSecret: MySecret { .init("This is a String") }
}

public struct MyNoSecret: MyCodeCompatible {
    static var associatedSecret: MySecret { .init("No secret here") }
}

// Public typealias, allowing users to work with the existential.
public typealias AnyMyCodeCompatible = any MyCodeCompatible

// Public API that uses AnyMyCodeCompatible.
public struct MyFramework {
    public var items: [AnyMyCodeCompatible] = []
    
    public init() {
        items = [1, "Hello", MyNoSecret()]
    }
}

Retrospective on the Problem:

The idea of closing protocol conformance has been discussed in Swift for years to prevent external conformances to public protocols while ensuring stability and control. Contributors like @Karl pitched the sealed protocols proposal, suggesting a "sealed" attribute to address the issue of unwanted conformances, often referred to as the “uninvited guest” problem. Solutions like type erasure were used as workarounds, but they added complexity. Overall, many developers contributed to the discussion, and you can find the detailed recap here.

Workaround:

There is a working workaround to this problem, which is why I suspect it wouldn't be hard to implement. The idea is that visibility of the protocol is restricted by using "@_spi", instead of internal or package, and then existential typealias is surfaced as a public API:

@_spi(Internal)
public protocol MyCodeCompatible {
}

public typealias AnyMyCodeCompatible = any MyCodeCompatible

However, Swift 6+ may introduce errors for SPI exposure.

1 Like

Sealed protocols are probably more elegant, but I'd also suggest simply adapting the ability to scope getters and setters to conformers.

public internal(conformer) protocol MyCodeCompatible {}

That way we hav more control than simply internal and users can still use the normal any MyCodeCompatible syntax.

6 Likes

This sounds similar to what I talked about here, and I hope I didn't unduly tie it to type disjunctions because it's really a distinct feature that has merit on its own even if type disjunctions are never supported.

What makes a closed protocol (that seems like that most natural syntax to me as the corollary to an open class) really useful in my opinion is not just preventing outside conformances (that is helpful though) but that closing it means the compiler knows the full list of conforming types when it compiles the protocol's module, which means it can support exhaustive switching:

closed protocol P {}

struct P1: P {}
struct P2: P {}
enum P3: P {}

let aP: any P = getAP()

switch aP {
  case let p1 as P1: ...
  case let p2 as P2: ...
  case let p3 as P3: ...
  // No open-ended default
}

I think this is a really important feature for Swift to gain because without it developers reach for enums too frequently, which is a problem because they aren't strongly typed (I can't write a function that requires a specific case of an enum) and don't support arbitrary overlapping hierarchies (hence the Russian doll nesting of subcodes in errors which would be more elegantly expressible as refined protocols). But the alternative, strongly typed option of protocols with concrete conformances can't be exhaustively switched on without introducing an enum as a manual type eraser, which reintroduces the problem that it can't model overlapping hierarchies properly.

2 Likes

This is a bug unfortunately (imagine what would happen if you generated a public swiftinterface for this module and tried to parse it back in).

2 Likes

What if we extended open and/or final to apply to protocols? (It may require some finagling to preserve source compatibility.)

I think sealed protocols are a reasonable idea and I wish we did it in Swift 3 when the open keyword was first introduced. However I’d be hesitant to promise anything on top where the compiler must have full knowledge of all types declared in a module.

It’s true today that in a -wmo build, we perform certain optimizations around class hierarchies and conformances to internal protocols, but that’s a best-effort thing that is not visible in the language.

In an incremental build, we skip parsing bodies of types and function bodies in files unless we need them. For example, imagine we rebuild a.swift because it changed:

// a.swift
protocol P {}

let p: any P = …

switch p {}

// b.swift
struct S1 : P {
  func f() {
    struct S2 : P {}
  }
}

We can only “see” S1, and we don’t know that S2 even exists because we skip parsing the body of S1 unless you perform a name lookup into S1. (And regardless we always skip bodies of functions in files that are not being rebuilt, because there is no way it can affect code generation).

6 Likes

Hmm, that's a good point. It's not just a problem of discovery by the compiler. How would you indicate S2 in an exhaustive switch? It's local to the function so it isn't in scope anywhere else. As soon as you declare a non-globally scoped conformance like that, exhaustive switching would have to be disabled. The compiler would have to look everywhere to tell if exhaustive switching is allowed.

That made me curious how Kotlin deals with that, since it supports exhaustive switching on sealed classes/interfaces, and the problem of "hiding" types inside of local scopes is even worse there because you can declare inner classes that close over their containing instance, and you can declare anonymous classes.

The documentation deals with that very clearly:

Subclasses of sealed classes must have a properly qualified name. They can't be local or anonymous objects.

So then you'd just have to ban conformance to a protocol that supports exhaustive switching by non-global types. That fixes the discovery problem.

But it causes a divergence from the desired behavior of simply forbidding outside conformances. Kotlin merges the two but that isn't necessary.

On the other hand I can see why the two concepts are more conflated than they may first seem. A library that simply cannot work with an outside conformance has to do some sort of unsafe downcast, but exhaustive switching fixes this too, you can safely downcast back to your internal type without an open-ended fatalError case. Avoiding the downcast makes the protocol not impossible but just very difficult to conform to externally (you have to fulfill all the requirements the library's implementation needs), although that might expose implementation details you'd rather keep private. If the protocol doesn't have complex requirements and you never force-downcast on it, it begs the question: are you sure your users must not supply their own conformances?

That situation may be better served by non-public requirements on a public protocol. That also seals it off from outside conformances.

So maybe this can mean an "open-ended" protocol that can be freely conformed to by non-global types but only internally:

public protocol Facade {
  ...

  internal var _internalState: _InternalState { get } // Not visible in the public module interface, compiler gives error like "protocol has internal requirements and cannot be conformed to outside its module"
}

While this signals a type you want to restrict to a closed hierarchy that you can visit:

closed protocol Department {}

struct Engineering: Department {}
struct Marketing: Department {}
struct Financing: Department {}

The former allows conformance by local (and maybe one day inner or anonymous) types and the latter does not. This may be poor syntax choice because it is no longer an analog to open class.

1 Like

You can still have nested types in other types, and extensions, but ultimately you could restrict conformances to the same file as the protocol or something drastic like that.

Another thing to keep in mind is generic conforming types. We don’t have a way to spell foo is G<_> where the generic parameter is unspecified, and if we did, conditional conformances would complicate exhaustivity checking. For example, here, only G<Int> conforms to P:

struct G<T> {}
extension G: P where T == Int {}

Rust actually has a Prolog-like solver which can enumerate conformances this way, either as a finite set or with type variables that encode infinite collections of types: GitHub - rust-lang/chalk: An implementation and definition of the Rust trait system using a PROLOG-like logic solver

1 Like

Thank you all for the responses. It’s clear that my initial idea was quite naive, while "sealed protocols" resonated with many people here.

Let me refine my understanding of the problem based on your feedback in this thread.

1. Sealed vs Closed vs Final vs Opaque

  • Opaque: I appreciate the broader discussion. My initial pitch aimed to avoid exposing a protocol’s interfaces, which can inadvertently expose module-internal types. Let's explore how Sealed|Closed|Final could address this.
  • Final: This annotation usually implies an implementation rather than an interface. It could be confusing in the context of protocols, as it’s unclear what aspect is “final.”
  • Closed: Given the relationship between “open” and classes, this could be a long association chain. “Closed” might not immediately convey its meaning when applied to protocols.
  • Sealed: I prefer this term, as it aligns well with other languages.

2. Exhaustive Knowledge of Conformances

I understand the value of exhaustive switching, but I see it more as a side benefit of sealed protocols, not the main goal. The primary focus should remain on controlling exposure, in my opinion. I’d prefer to avoid being sidetracked here.

3. Feature scope

There is an urgent need to address issues between framework developers and their clients. I see minimal benefit in applying sealed protocols at other levels or within internal code. Therefore, I suggest we limit our focus to sealing protocols at the module level, with the potential to extend this to packages in the future. This said, I wouldn't bother at all if we cannot manage to know all conformances while module itself is being compiled. We can simply require that any type conforming to a sealed protocol must be either external or have an accessibility level that is equal to or higher than that of the protocol itself.

4. Accessibility scope

Sealed protocols introduce a clear distinction: 1) the accessibility level where protocols are declared and where conformances are created, and 2) the accessibility level where they can only be used as sealed. If sealing is scoped to the module level (or potentially package level in the future), we should require mandatory external access annotations (public or package ):

public sealed protocol MyCodeCompatible {}
package sealed protocol MyCodeCompatible2 {}

5. Interface Opaqueness

This is a must-have for framework developers to control the surfaced interface. Sealed protocols could allow module-relevant access annotations: internal , public , and package to their interfaces. By default, an interface’s access level matches that of the protocol. If the interface has a lower access level than the protocol, it becomes sealed -- invisible beyond that level.

internal struct MySecret {
}

public sealed protocol MyCodeCompatible {
    // "internal" annotation allowed here for sealed protocols only.
    // Neither `foo` nor `MySecret` is visible to Module users
    internal func foo(_ secret: MySecret)

    // By default, same visibility as of the protocol itself
    func boo()

    // "package" annotation allowed here for sealed protocols only.
    package func goo()
}

// Only public conformance is supported
public struct MyType: MyCodeCompatible {
...
}

// External types are supported by sealed protocols
struct Int: MyCodeCompatible {
...
}

The above basically inspired by @aetherealtech's thoughts. With some additions.

6. Summary

  • Internal Module Use: Sealed protocols are treated as usual protocols inside the Module where they are declared. This said, I suggest that exhaustive knowledge of conformances inside a Module where sealed protocols are declared is no goal.
  • Conformance Requirements: Once the Module is linked, each sealed protocol is associated with a fixed set of known conformances. This means sealed protocols can only be conformed to by types with the same or higher access level. In other words, internal types cannot conform to sealed protocols.
  • Restricted Conformance: Module or package users are prohibited to add new conformances, but in turn they have exhaustive knowledge of all types conformed to the protocol.
  • Controlled Interface Exposure: Users can’t see sealed protocol interfaces beyond specified annotations.
2 Likes

Would you be able to provide a small concrete example of using a sealed protocol? I think that can help clarify what your goal is. And it might help motivate (or rather disprove) my suspicion that the ability to recover a concrete conformance without a fatalError path is more relevant to the goal than it may first seem.

In particular I think it would be helpful to see what you would do with this sealed protocol inside the module after it is sent in from external code. That can illustrate why it is important to prevent outside conformances, since presumably an outside conformance would break that call into the library.

Even if recovering an internal conformance exhaustively is intrinsically tied up with preventing outside conformances, it sounds like that's more of a rabbit hole, while simply preventing outside conformances would be, I imagine, pretty trivial for the compiler, so that could still be a useful first iteration.

@aetherealtech Thank you for response! I wanted to avoid potential suggestions, such as using enum containers, boxed types or Any instead of protocol instances. This is why I’ve framed this example using two sealed protocols instead of one.

Example: Library with Sealed Protocols

Objectives:

  • Library should support a fixed set of foundational types (e.g. Elements and Operations).
  • Design should enable agile and safe development of new features: adding new Operations and Types
  • Library should not expose implementation details through the API

Type Declarations:

  • Defines the Element protocol and restricts conformances to specific types within the Library.
  • Implements (lets say, large) set of processor types that conform to Operation protocol, with internal-level access for methods.
// MyLibrary

public sealed protocol Element {
    // Accessible only within the Module
    internal func processElement() -> String
}

extension Int: Element {
    internal func processElement() -> String {
        return "Processing Int: \(self)"
    }
}

extension String: Element {
    internal func processElement() -> String {
        return "Processing String: \(self)"
    }
}

extension Data: Element {
    internal func processElement() -> String {
        return "Processing Data of length: \(self.count)"
    }
}

public struct Message: Element {
    internal func processElement() -> String {
        return "Processing Message"
    }
}

public sealed protocol Operation {
    // Accessible only within the Module
    internal func internalOperationA(_ element: Element)
    internal func internalOperationB(_ element: Element) -> Element
    internal func internalOperationC(_ element: Element, secret: MySecret) -> Element
}

public struct ProcessorA: Operation {
    internal func internalOperationA(_ element: Element) {
        print("ProcessorA - Internal A: \(element.processElement())")
    }
    
    internal func internalOperationB(_ element: Element) -> Element {
        print("ProcessorA - Internal B: \(element.processElement())")
        return element
    }

    internal func internalOperationC(_ element: Element, secret: MySecret) -> Element {
        print("ProcessorA - Internal C: \(element.processElement())")
        return element
    }
}

public struct ProcessorB: Operation {
    internal func internalOperationA(_ element: Element) {
        print("ProcessorB - Internal A: \(element.processElement())")
    }
    
    internal func internalOperationB(_ element: Element) -> Element {
        print("ProcessorB - Internal B: \(element.processElement())")
        return element
    }

    internal func internalOperationC(_ element: Element, secret: MySecret) -> Element {
        print("ProcessorB - Internal C: \(element.processElement())")
        return element
    }
}

Note : Allowing users to implement Operation can leak internal types and instances, as it happens with internalOperationC method declared above. Regular protocols require internal types like MySecret to be public, and then its instance can be exposed to the Uninvited Visitor. Situation becomes more complicated, if such types conform to some protocols, like Codable -- conformance is also publicly exposed.

User-Facing API:

public struct UserDataHandlerA {
    private let processor: Operation

    public init(processor: Operation) {
        self.processor = processor
    }

    public func handleData(with elements: [any Element]) -> [any Element] {
        return elements.map { processor.internalOperationB($0) }
    }
}

public struct UserDataHandlerB {
    private let processor: Operation

    public init(processor: Operation) {
        self.processor = processor
    }

    public func handleData(with elements: [any Element]) -> [any Element] {
        return elements.map { processor.internalOperationB($0) }
    }
}

Usage form Clients Application:

  • Uses UserDataHandlerA with ProcessorA for initial processing.
  • Passes the result to UserDataHandlerB with ProcessorB for further processing.
// Hypothetical Clients Application

import MyLibrary

let handlerA = UserDataHandlerA(processor: ProcessorA())
let handlerB = UserDataHandlerB(processor: ProcessorB())
let initialElements: [any Element] = [1, "string", Data(), Message()]

// First processing pass with handlerA
let processedElements = handlerA.handleData(with: initialElements)

// Second processing pass with handlerB using the results from handlerA
let finalElements = handlerB.handleData(with: processedElements)

Alternatives to Sealed Protocols:

There are several alternatives of how to hide implementation details and internal types from the Public API.

  • Enum Containers: Instead of protocols, we can use enum containers for types supported by MyLibrary.
  • Boxed Types: We can declare single-element shells for types supported by MyLibrary and conform them to a common public protocol.

Both approaches:

  • Complicate implementation and in my opinion, do not meet requirement of agile development, as adding new types and operations becomes quite complex.
  • Require constant back-and-forth transformation between public representation and backed internal types, which is especially unfortunate if types are wrapped into complex Collections.

Instead of summary

I’m quite sure there could be more suggestions on how to achieve my objectives. However, the approach in this example works perfectly with regular protocols when these objectives aren’t required. It feels counterintuitive that I need to use workarounds to achieve these goals, rather than simply annotating my protocols.

1 Like

Thank you, this is very helpful!

I agree on the desire to avoid boxing (I think the "enum containers" fall under that category too), in general any kind of "boxing" done by hand to me indicates a missing language feature. I can succinctly summarize almost all of my desires for new Swift features as an aim to eliminate type erasers (Any... boxes) entirely.

If I parsed the example correctly, the protocols that are sealed all have internal requirements. This is what avoids the need to downcast values that come in from the outside. The capabilities you need are defined on the protocol so the module internally can use those values through the protocol without downcasting.

For that, I think you actually don't need an extra keyword like sealed. That is implied by the fact they have internal requirements. Being internal, no one in other modules could possibly fulfill those requirements. The restriction is inherent to the fact conformances must supply capabilities whose visibility is restricted to the module.

I was thinking about this more and concluded I would like to see all currently available access levels be usable on protocol requirements, with the following meanings that already line up with what they mean in general (with one exception):

  • public: available everywhere the protocol is
  • internal: available within the same module that defines the protocol
  • private: available in same-file extensions to the protocol and inside conformances
  • fileprivate: available within the same file that defines the protocol

These meanings line up with what they mean in all other cases, except for private, which works a little different. The default visibility without an explicit declaration is public (which is currently forbidden because it's implied and unmodifiable, but would be supported to add clarity).

In all cases, this means a conformance must mark its implementations of the requirements with at least the same visibility as either the requirement or the conforming type itself (whichever one is lower). For example:

public protocol P { 
  public var x: Int { get }
  internal var y: Int { get }
  private var z: Int { get }
  fileprivate var w: Int { get }
}

public struct T: P {
  public let x: Int
  internal let y: Int
  private let z: Int
  fileprivate let w: Int
}

A conformance can choose a higher visibility if it wants, just not a lower visibility one.

I don't think a new visibility like package is necessary either. The key point is that internal and fileprivate don't "inherit" new meanings where they appear. That is, internal on a protocol requirement doesn't mean internal to the module of whatever type is conforming to it (I can't imagine that being a useful feature because you're telling other modules they must define things only those modules can use, so how would you ever know they fulfilled this?). It always means internal to the module defining the protocol. Similarly with fileprivate, it doesn't mean private to whatever file a conforming type is in, for the same reason.

Each of these specifiers is significant and adds a lot of expressive power to protocols:

internal -> exactly what you're showing in your example, it restricts conformances to the module so they can be handed out to but not extended by other modules, allowing them to represent both the publicly facing facade and an abstraction of the private implementation details.
private -> effectively the protected specifier but the one we really want instead of the one shackled to OOP. A same-file extension can define a public member that accesses private requirements on the protocol. Now we have abstract classes, but without the reference semantics or inherited storage that needs initialization.
fileprivate -> similar to internal, restricts conformances to the file, enabling the same type of boundary to be drawn within a module.

Both internal and private with these meanings would be highly useful to me.

Since this is all purely access level enforcement, it is hopefully an easy feature to implement... the challenge is probably in making sure it's well-formed. If it is, it is hopefully a simple matter of enabling the same access level enforcements that already exist. And then just adding access specifier data for requirements to module interfaces.

But now... can you imagine a case where you want to restrict conformances but don't have internal requirements? That's where, in my mind, exhaustive switching inevitably gets involved. That's why I think the feature request here is really not even about explicitly marking protocols as locked down, it's just about supporting more access levels on their requirements.

1 Like

Correct, this is exactly what I meant to illustrate.

I believe an explicit directive adds clarity. My idea is that “sealing” the protocol enables access levels on its interface, which unsealed protocols wouldn’t have.

Regarding this part and the rest of your message, let me clarify my idea. Access levels on protocol interface methods/properties “seal” the protocol at their narrowest level. Conformances can only be declared at that level only; broader levels wouldn’t have access to the full interface. Thus, private access level for protocol methods/properties doesn’t make sense here, as there’d be nowhere to declare conformances.

Additionally, I want to address @Slava_Pestov’s concern regarding challenges of parsing and handling conformances across different levels. “Sealed” protocols could also limit the scope for conformances, ensuring full knowledge of conforming types at the end of module compilation.

I am thinking more from the practical standpoint. Providing protocol "sealing" at the module/package levels would immediately offer flexibility for API design and improve its cleanness, which is my main goal here. Other features can fit in later.