Pitch: Protocols with private fields

In a protocol, all fields (properties and methods) will get the access visibility of the conforming type. For instance, conforming to a protocol with a public type will prompt all of its requirements to be public.

public protocol TreeVisitor {
  mutating func visit(node: Node)
  mutating func visit(leaf: Leaf)
}

public struct TreePrinter: TreeVisitor {
  private var parentKeys: [Int] = []

  // 'visit(node:)' must be 'public'
  public mutating func visit(node: Node) {
    parentKeys.append(node.key)
    node.lhs.accept(&self)
    node.rhs.accept(&self)
    parentKeys.removeLast()
  }

  // 'visit(leaf:)' must be 'public'
  public mutating func visit(leaf: Leaf) {
    print(parentKeys.map(String.init(describing:)).joined(separator: " "), leaf.key)
  }
}

Private fields in protocols have already been discussed and I completely agree with the conclusions that have been drawn in the past:

  1. A protocol describes an API, guaranteeing to its clients that a particular set of fields will be present in the conforming type. Thus, it makes no sense to hide parts of that API.
  2. Conforming types must be able to "see" what requirements they are compelled to implement. If a protocol had fileprivate fields, for instance, then they would be technically "invisible" to a type declared in another file.

One problem, though, is that we can't define default implementations that rely on an encapsulated state (a.k.a. stateful mixins in other circles).

Imagine, for instance, that I would like to create a reusable implementation of a tree walker that simply traverses a tree, calls methods before and after visiting each node, and also records the keys it sees during the traversal. Clearly, that is simple to implement with a class:

open class TreeWalker {
  public private(set) final var parentKeys: [Int] = []

  open func willVisit(_ tree: Tree) -> Bool { true }
  open func didVisit(_ tree: Tree) -> Bool { true }

  public final func walk(_ tree: Tree) -> Bool {
    guard willVisit(tree) else { return true }
    return traverse(tree: tree) && didVisit(tree)
  }

  internal func traverse(tree: Tree) -> Bool {
    // The implementation is irrelevant to this discussion.
  }
}

Unfortunately, using a class prevents me from defining tree walkers with value semantics (well, to be fair, we could write classes that behave like values, but that's generally not straightforward).

If I wanted to turn this class into a protocol with a default implementation of the traversal logic, I wouldn't be able to make parentKeys a read-only property. That breaks the encapsulation principle, because clients of a TreeWalker need only to know about willVisit(_:), didVisit(_:) and walk(_:).

We can achieve some level of encapsulation by defining implementations in extensions, without matching requirements.

public protocol TreeWalker { ... }
extension TreeWalker {
  // 'traverse(tree:)' is not visible beyond the module boundary
  internal func traverse(tree: Tree) -> Bool { ... }
}

Unfortunately, that does not solve the issue for parentKeys, because of two problems:

  1. I'd like that property to be publicly visible, just not writable. However, I can't distinguish between these two capabilities: I must either define an implementation that is completely hidden, in an extension, or expose both read and write access as part of the protocol's API.
  2. I'd like to provide a default implementation of storage.

Note that I can't simply define parentKeys as a read-only requirement and define a default, writable alternative in extension, because of the second issue. There is no way to avoid the conforming type to implement public (up to its own visibility), writable storage, thus breaking encapsulation.

I can imagine a solution for each of these problems. Both are orthogonal, but would work together, I think. The first idea is probably much more realistically implementable. If anything, I'd like that thread to focus on that one.

Interpret access modifiers as lower bounds

A simple way to address the visibility issue would be to state that access modifiers in protocols define a lower bound on the visibility of a particular requirement in the conforming type. The absence of any modifier would denote the same semantics as now.

protocol P {
  var foo: Int { get }
  var bar: Int { fileprivate(get) }
  internal func ham() -> Int
}

public struct S: P {
  // 'foo' must at least 'public'
  public var foo: Int
  // 'bar' must be at least 'fileprivate'
  internal var: Int
  // 'ham()' must be a least 'internal'
  func ham() -> Int { 42 }
}

With that approach, I could write a default implementation that relies on fields that are not necessarily part of conforming type's public API, and that can be encapsulated. Nonetheless, the client would still be allowed to decide the access level of any requirement, should they decide to expose them anyway. That last bit is important, because we wouldn't want conformance to a specific protocol to exclude conformance to another.

Here's how I could keep parentKeys encapsulated:

public protocol TreeWalker {
  var parentKeys: [Int] { get private(set) }
  /// ...
}

// In another module

public struct ClientWalker: TreeWalker {
  // read access must be 'public', because 'ClientWalker' is 'public'
  public private(set) var parentKeys: [Int] = []
  // ...
}

One minor problem with this approach is that it changes the meaning of an access modifier in the context of a protocol declaration, which might be counterintuitive. For instance, fileprivate would mean "only visible in the files defining the types that conform to this protocol", not "only visible to this file".

Encode state in the witness table

The previous feature would still require conforming types to provide an implementation of every stateful properties. In the walker example, that means that we need to add a writable property parentKeys in all conforming types (as in the snippet above). While that is a minor inconvenience, it implies that we cannot provide default implementations that also take care of storage.

It should be stressed that the restriction makes complete sense w.r.t. the way conformance is implemented (to the extent of what I think I know and understand). If S conforms to P, then the corresponding entry in S's protocol witness table is just a collection of pointers to functions wrapping the actual implementation of each requirement defined by P. No storage required.

We could imagine that states provided by default implementations be referred in the witness table as well. We would add another entry: a pointer to a block of memory backing non-computed properties for which the conforming type provides no implementation.

More concretely, consider the following example. The keyword synthesizable indicates that conforming types need not to provide a default implementation of that property.

protocol Counter {
  synthesizable var value: Int { get set }
  mutating func inc()
}
extension Counter {
  func inc() { value += 1 }
}

struct S: Counter {}

The entry for Counter in S's protocol witness table would look like that (written in Swift for clarity):

struct SCounterWitnessTable {
  let getValue: FunctionRef
  let setValue: FunctionRef
  let modifyValue: FunctionRef
  let inc: FunctionRef

  // That would be new
  let defaultStorge: UnsafeMutablePointer<SCounterWitnessStorage>
}

struct SCounterWitnessStorage {
  var value: Int
}

Of course, the value witness table of S would also need to care about that default storage to properly copy and destroy an existential container, should the protocol contain synthesizable properties. Otherwise, there would be no impact on the current behavior, as all conforming types would still be compelled to provide an implementation.

With that feature, combined with the solution to the visibility problem above, I could rewrite TreeWalker as a protocol:

public protocol TreeWalker {
  synthesizable var parentKeys: [Int] { get private(set) }

  mutating func willVisit(_ tree: Tree) -> Bool { true }
  mutating func didVisit(_ tree: Tree) -> Bool { true }
}

extension TreeWalker {
  public mutating func walk(_ tree: Tree) -> Bool {
    guard willVisit(tree) else { return true }
    return traverse(tree: tree) && didVisit(tree)
  }

  internal mutating func traverse(tree: Tree) -> Bool {
    // The implementation is irrelevant to this discussion.
  }
}
6 Likes

Hi, Dimitri. I’ve also wanted something that’s quite a bit simpler than your many ideas here, but I think related. It consists of two closely related features:

  1. Allow a protocol to specify a requirement that will be available to extension methods but not to clients of the protocol, and
  2. allow types to expose specific members to specific protocol conformances.

In short, I’d like to be able to decouple “exposed to protocol implementation” from “exposed to protocol clients.”

Using your example of tree traversal, here is (1) in action:

protocol Tree {
  implementation var children: [Self] { get }  // ignore strawman keyword; just note the semantics

  func traverse(_ visit: (Self) -> Void)
}

extension Tree {
  func traverse(_ visitor: (Self) -> Void) {
    for child in children {  // ✅ `children` visible to protocol implementation
      child.traverse(visitor)
    }
    visitor(self)
  }
}

var foo: Tree = ...
foo.children  // ❌ `children` not visible to protocol clients

…and (2) in action:

struct Doodad: Tree {
  var children: [Doodad]  // not private, so nothing special needed here
}

struct Widget: Tree {
  @exposed(to: Tree, as: children)  // again, ignore strawman syntax and note the semantics
  private var subwidgets: [Widget]  // private, but explicitly allowed to participate in protocol conformance
}

…or maybe you expose members in the conformance, I don’t know, syntax is to work out if the bigger idea seems compelling:

struct Widget {
  private var subwidgets: [Widget]
}

extension Widget: Tree(exposing: children) {
  private var children: [Widget] { subwidgets }
}

I’ve run across a handful of situations where the ability to do this would have made code much less awkward.

It seems consistent to me with the philosophy of protocols, and of witness tables in particular, which say both (1) “T is a P”, and also (2) “here is how T is a P.” It makes sense that something could be hidden to 1, but visible to 2.

4 Likes
Terms of Service

Privacy Policy

Cookie Policy