Pitch: Reparenting Resilient Protocols

Hi all,

I’d like to start a discussion about this idea for an ABI-resilient way to rebase a protocol. Keep in mind that I only have only a rudimentary proof-of-concept implementation for what is pitched. So there may be dragons!


Introduction

It's already possible to add a new parent (i.e., a refined protocol) to a protocol without causing a source break, as long as it's possible to define defaults for all of the new requirements introduced by the parent. For example, suppose I wanted to introduce a more relaxed version of some existing protocol Diver that I'll call Swimmer:

// Existing protocol
protocol Diver {
  func gentlySwim()
  func dive()
}

// New protocol
protocol Swimmer {
  func swim()
}

Since all divers conceptually are also swimmers, I want to reparent the Diver protocol so that all types conforming to Diver are now also seen as conforming to Swimmer. To do so without causing a source break, I can write an extension that provides defaults (or witnesses) for any requirements in Swimmer that the Diver doesn't satisfy:

extension Diver {
  func swim() { gentlySwim() }
}

Unfortunately, this way of reparenting an existing public protocol still causes a break in terms of ABI for any libraries that have Library Evolution enabled. The technical reasons for this break are subtle and complex, but I believe there might be a way to avoid the ABI break, given the right set of language
restrictions.

Proposed Solution

Protocols can resiliently introduce a new parent protocol in its inheritance clause if annotated with @reparented,

public protocol Diver: @reparented Swimmer {
  func gentlySwim()
  func dive()
}

@reparentable
public protocol Swimmer {
  func swim()
}

extension Diver {
  func swim() { gentlySwim() }
}

Only protocols born with @reparentable can appear in a @reparented relationship. The reason for this has to do with how Swift forms a canonical ordering of protocols in certain parts of the runtime system; reparentable protocols must be ordered differently.

Default conformances

When declaring that a protocol P is @reparented by protocol Q, a default conformance is created and validated, using the implementations of methods, properties, and typealiases within an unconstrained extension of P. This default conformance represents the universal way for all existing conformers of P to be adapted and treated as Q, even if the library is newer than the client linking against it. For example,

protocol Pedestrian {
  associatedtype Speed: Comparable
  var speed: Speed { get }

  func isWalking() -> Bool
  func isRunning() -> Bool
}

protocol Runner: @reparented Pedestrian {
  var mph: Float { get }
  func isRunning() -> Bool
}

The above alone will result in an error, unlike a regular refinement, because Runner is now checked statically to see if it provides everything required by Pedestrian. Thus, to satisfy this reparenting relationship, an unconstrained extension of Runner needs to fulfill all the requirements of Pedestrian,

extension Runner {
  typealias Speed = Float
  var speed: Float { return mph }
  func isWalking() -> Bool { return mph > 0.0 }
}

Note that an implementation of isRunning() can be omitted in the extension, because Runner already has exactly the same method requirement of its own conformers.

The default conformance created for some Runner : Pedestrian is used anytime a client built prior to the reparenting links against the library. If the client simply recompiles against the new library, the default conformance will no longer be used. Instead, each nominal type conforming to Runner will
automatically have their own conformance to Pedestrian created, as usual. So, if a client's conforming type C defines its own method C.isWalking(), then it is used instead of Runner.isWalking() to satisfy the requirement of Pedestrian.

Availability

Typical protocol refinement relationships require the base protocol to have the same or greater availability than the derived one:

@available(jetpackOS 9, *)
protocol Base {}

@available(jetpackOS 7, *)
protocol Derived: Base {}
               // `- error: 'Base' is only available in jetpackOS 9 or newer

A reparented relationship signals the opposite: the derived protocol existed before the base protocol. Thus, the availability rule is also the opposite. Given a derived protocol D that is reparented by some base protocol B, then D must have the same or greater availability than B. For example,

@reparentable
@available(jetpackOS 9, *)
protocol Base {}

@available(jetpackOS 7, *)
protocol ExistingBase {}

@available(jetpackOS 7, *)
protocol Derived: ExistingBase, @reparented Base {}
                                         // `- OK, despite 7 < 9

For clients compiling against this library whose deployment target is less than jetpackOS 9, then Derived will not be considered a subtype of Base within that client's module.

8 Likes

I assume adding a suppressed conformance to an existing protocol is a non-issue given it's been done already.

I'd like to bikeshed the attribute names, and in particular see if there's some way to eliminate the need for @reparentable, but the perfect is the enemy of the good as they say.

The Swift ABI defines a linear ordering on type parameters, based on a linear ordering of protocols. Ordinary protocols are ordered by their fully qualified name, but that’s insufficient for reparentable protocols. Associated types in reparentable protocols must be greater than (further away from the most reduced) than associated types in ordinary protocols, to avoid changing mangling of existing symbols when a reparenting relationship is introduced.

So all reparentable protocols must follow all “reparentees” in this linear order. For this reason, a protocol must be declared reparentable (and also, an @available annotation is probably required) at the protocol declaration — it wouldn’t work to allow any protocol to reparent any other.

2 Likes

Well, it’s still a subtype, but you can’t directly talk about Base or any declaration that mentions Base, because of the availability annotation on Base.

Also it’s worth mentioning when protocol requirements, including associated types, can be moved from Derived up to Base. (Sometimes Base will introduce a completely new set of operations, other times, you might want to move some stuff from Derived up to Base.)

Today, these members would have to be annotated with nonoverride to ensure that they still get a witness table entry in Derived, for example. Perhaps this could be automated somehow, by considering the case where the base protocol requirement that is being overridden has newer availability. This can’t happen today, so maybe in this case nonoverride can be inferred automatically.

1 Like

This sounds amazing.

1 Like

What happens if you reparent in version 2, and then again in version 3? (In terms of your example, say version 3 adds a “Human” protocol.) How does the compiler know in which order to linearize the two reparented protocols?

Can we solve this problem and eliminate the need for @reparented just by inferring the reparenting rules when a protocol refines another protocol with newer availability? And can the need for @reparentable be eliminated by assuming that all protocols with an availability declaration are reparentable?

Availability.

Yeah, you technically don’t need a special annotation at the point of protocol inheritance in practice, but it might still be nice to make it explicit, and also to avoid making the availability annotation mandatory.

Unfortunately also no, because we don’t want the behavior of existing protocols with availability annotations to change.

3 Likes