Authors: Dave Abrahams (@dabrahams) and Dmitri Gribenko (@gribozavr), with support and feedback from the Swift for TensorFlow Team.
This pitch is related to and in support of the recently posted pitch, Automatic Requirement Satisfaction in plain Swift.
Introduction
In Swift, protocol extensions provide a beautiful way to share API implementations across value types. For example, an extension on the Equatable protocol provides a public implementation of the != operator to all public conforming types. The power of this feature is limited, however, because the conformance can never be less visible than APIs provided by extensions depending on that conformance.
For example, suppose we often find ourselves creating types of this form:
public struct X: Comparable, CustomStringConvertible {
internal var value: SomeType
public static func <(a: Self, b: Self) -> Bool {
a.value < b.value
}
public var description: String { String(describing: value) }
...
}
It’s tempting to use protocol extensions to provide the implementations of < and description; in theory we could simply write:
public struct X: Wrapper, Comparable, CustomStringConvertible {
internal var value: SomeType
}
and let extensions that depend on Wrapper do all the hard work. However, when we try to define these extensions, we’ll find that Swift won’t let us keep Wrapper internal.
internal protocol Wrapper {
associatedtype Value
var value: Value { get }
}
extension Equatable where Self: Wrapper, Value: Comparable {
// error: cannot declare a public operator function in an
// extension with internal requirements
public static func <(a: Self, b: Self) -> Bool {
a.value < b.value
}
}
// similar extension of CustomStringConvertible...
The only way to achieve the code reuse we’re after is to make Wrapper a public protocol, which in turn makes X’s conformance to Wrapper and everything needed to satisfy it (value and Value) part of Xs public interface.
Thus, we are forced to choose between boilerplate and exposing more API than intended. Even if we use underscore-prefixed APIs to hide unwanted public names, code reuse comes at a hidden cost: the compiler is forced to generate public type metadata and witness tables for all the requirements of Wrapper, bloating binary size with information that will never be used.
Because this limitation compromises a large class of otherwise-useful API building blocks, we propose to lift it.
Scoped Conformances
To begin with the minimal pitch serving the purposes of the authors, we need to modify the problem slightly; the exact motivating example is covered in “Future Directions.” In the meantime, imagine we wanted to expose Wrapper and X publicly, but not X’s conformance to Wrapper or the API that X depends on to satisfy Wrapper’s requirements (value and Value).
To keep the Wrapper conformance out of X’s public API, we propose an idea discussed briefly in the Generics Manifesto: scoped conformances. With a scoped conformance, the declaration of X could be written this way:
public struct X: fileprivate Wrapper,
Comparable, CustomStringConvertible
{
internal var value: SomeType
}
The conformance to Wrapper, being visible only in the file where X is defined, is available to allow the requirements of Comparable and CustomStringConvertible to be satisfied, but does not leak into X’s public API or force its value or Value to become public. The explicit fileprivate label cues the compiler to try to eliminate associated metadata, which—because there can be no existential Wrapper—is trivially possible.
Note: the original title of this pitch was "Non-public conformances implementing public API," to emphasize that while X: Wrapper is not visible outside the file, the func < that depends on X: Wrapper is visible publicly.
Dynamic Casts
The manifesto identifies the handling of dynamic casts as the major problem with scoped conformances, using this example:
// File1.swift
public protocol P { }
public struct X { }
extension X : internal P { ... }
// File2.swift
func foo(value: Any) {
if let x = value as? P { print("P") }
}
foo(X())
The question posed is, “Under what circumstances should [the program] print ‘P’?” Our answer is that X’s conformance (or not) to P is captured in the scope where the X value is first converted to an existential type. If the cast occurred in the scope where a conformance of X to P is visible (in the same module as File1.swift, in this case), foo will print “P”. So if File1.swift and File2.swift are in the same module, the internal conformance to P is visible at the call to foo, where the X instance is converted to Any, and “P” is printed. If File1.swift and File2.swift are in separate modules, nothing is printed.
Future directions
Thinking of the problem in terms of a public protocol and a fileprivate conformance satisfies the authors’ immediate needs, and is enough to support the vending of useful API building blocks, like Wrapper, from libraries. It also simplifies the discussion by only using a public protocol to create public API.
But what about the original example, where we want to keep Wrapper internal? We believe this is a mostly-straightforward extension of the same idea that, because the conformance does not need to escape the module scope, boils down to the same arrangement, except with Wrapper entirely hidden from X’s clients. There are, however, a few details that need to be worked out (see “Open Questions,” below).
We think it should also be possible to extend the system to support conformances scoped more narrowly—to a type or to a function body, for example—but haven’t considered the question deeply.
Beyond these examples, we believe this design provides a framework for thinking about the answers to unsolved or unspecified problems in Swift’s generics system, such as how conformances in multiple modules should work, and how algorithm specialization can work with conditional conformances of generic types, because it enshrines the idea that the visibility of conformances in the scope where a generic parameter is bound to a concrete type, or an existential is formed, determines how implementations are fulfilled.
Open Questions
We believe these issues are not difficult to address, but as the work still needs to be done, they bear mentioning:
- If we allow public requirements to be satisfied by extensions depending on non-public names (as in the opening example), must the satisfying definitions (
<anddescription) be explicitly declared public? - Should we allow the new public API that doesn’t satisfy any existing requirements to be generated by extensions that depend on non-public names? If so, presumably only when the new API is explicitly declared public?
- In a scoped conformance to a refined protocol, what scope is implied for the conformances to the protocol’s less-refined ancestors?
- How is scope resolved when two scoped conformances imply different scopes for a common ancestor protocol?
Earlier Related Discussion
In 2018, Jordan Rose raised the possibility of allowing non-public methods to satisfy public protocol requirements, a similar feature that achieves some of the same goals.