Dynamically-dispatched protocol extension members

This is just a pitch; I don't have an implementation. I'd like to gather feedback from the community about how desirable this feature would be, and from the compiler team about how it might be implemented:


Motivation

A protocol's members may be either dynamically or statically dispatched, depending on how they are declared. Protocol requirements are dynamically dispatched, meaning that they resolve to conformance-specific implementations in generic code, whereas members defined in extensions are statically dispatched, meaning that they may not be overridden, and generic code will not use any such overrides even if they are available.

Dynamic dispatch is very important when optimising certain generic operations. For example, Array makes use of multiple hidden protocol requirements in Sequence to optimise copying from types with compatible backing storage and from other contiguous sequences. Such optimisations are not available to types outside of the standard library, as they cannot add requirements to Sequence, Collection, or any other standard library protocols.

This proposal seeks to allow members defined in protocol extensions to be dynamically dispatched, even if defined outside of the module which owns the protocol definition. This means that if you have an algorithm which can work on a Collection, but has an optimised implementation for, say, RandomAccessCollection, and that function is marked as using dynamic dispatch, generic code with the constraint <C: Collection> will resolve calls to the optimised version when given a RandomAccessCollection.

Proposed solution

Swift already has the dynamic keyword, which allows members of reference-types to opt-in to dynamic dispatch. This proposal would enable the same keyword to be used for members in protocol extensions.

Take the following example:

protocol MyProto {}
extension MyProto { 
  func myMethod() { print("Protocol method") } 
}

struct NoOverride: MyProto {}

struct HasOverride: MyProto {
  func myMethod() { print("Struct method") }
}

func call_generic<T: MyProto>(_ object: T) { 
  object.myMethod() 
}

let defaultObject = NoOverride()
defaultObject.myMethod() // "Protocol method"
call_generic(defaultObject) // "Protocol method"

let object = HasOverride()
object.myMethod() // "Struct method"
call_generic(object) // "Protocol method"

This snippet illustrates a common source of confusion for developers who are relatively new to Swift. By marking the method myMethod as dynamic, the developer opts-in to the same effective behaviour as a protocol requirement with a default implementation. This means that the specialised implementation in HasOverride will be called in generic code, rather than the default implementation which is used for other conformances to MyProto:

extension MyProto { 
  dynamic func myMethod() { print("Protocol method") } 
}

struct NoOverride: MyProto {}

struct HasOverride: MyProto {
  func myMethod() { print("Struct method") }
}

func call_generic<T: MyProto>(_ object: T) { 
  object.myMethod() 
}

let defaultObject = NoOverride()
defaultObject.myMethod() // "Protocol method"
call_generic(defaultObject) // "Protocol method"

let object = HasOverride()
object.myMethod() // "Struct method"
call_generic(object) // "Struct method"     <-- This.

Detailed design

As it does for class members, dynamic will support both properties and functions defined in protocol extensions. Using a dynamic protocol member from generic code should resolve to the same implementation as calling the protocol member directly on the value's dynamic type.

Specifically, it should be possible to override a dynamic protocol member in a refined protocol, as protocol requirements currently can:

extension Collection {
  dynamic func myAlgorithm() -> [Element] {
    /* default implementation */
  }
}
extension RandomAccessCollection {
  func myAlgorithm() -> [Element] {
    /* optimised implementation */
  }  
}

func doIt<T: Collection>(_ items: T) -> [T.Element] {
  return items.myAlgorithm()
}

let array = [1, 2, 3, 4]
array.myAlgorithm() // should call RandomAccessCollection version, as the best match for Array.
doIt(array) // should call RandomAccessCollection version, as the best match for Array.

The override keyword is not required, mirroring the behaviour of protocol requirements with specialised default implementations. It is also not necessary for specialised implementations to repeat the dynamic keyword (although it is allowed), mirroring the behaviour of overrides in reference-types (although they do require the override keyword).

Source compatibility

This is an additive change and will not change any behaviour in existing programs.

Effect on ABI stability

This feature will almost certainly require an updated runtime.

Effect on API resilience

Adding or removing dynamic is an ABI-breaking change, as it is for classes.

Alternatives considered

  • Make dynamic dispatch the default

    This would be a source-compatible but potentially behaviour-altering change and needs to be considered carefully as its own proposal. The goal of this proposal is only to allow dynamic dispatch for protocol extension members, not to change which kind of dispatching is used implicitly.

  • Require override or repeating dynamic for specialised implementations.

    There is an argument to be made for this, but protocol requirements don't ask you to do it today. The difference between protocol requirements and extension members is confusing enough for developers today - adding more syntax differences would likely do more harm than good.

  • Do nothing

    One workaround being used today is to define a secondary protocol which duplicates the members which should be dynamically-dispatched:

    protocol MyProto_Extensions {
      func myMethod()
    }
    extension HasOverride: MyProto_Extensions {}
    
    func call_generic<T: MyProto>(_ object: T) { 
      guard let instance = object as? MyProto_Extensions else { object.myMethod(); return }
      instance.myMethod() 
    }
    

    This basically re-implements dynamic dispatching using retroactive conformances and a dynamic cast. It isn't a very obvious solution to something that should be simple, doesn't work for protocols with associated types (like Collection & friends), and only allows overrides on a per-concrete-conformance basis (you couldn't make all RandomAccessCollections conform to an "extensions" protocol, because protocols can't conform to other protocols).

8 Likes

+1 on this one. I really like the idea

I would like dynamic dispatch being the default and the source code converter to add a static keyword or nonvirtual, but I could deal with having dynamic in the extensions instead.

2 Likes

I would opt for this idea as I would give more freedom. Each implementation of the protocol would have a choice of how the method will be dispatched.

Either of these two will be chosen +1.

Yes please, let's finally fix this! Fine with the proposal or alternative of default behavior, anything to get this into the language sooner than later.

+1 this is how it should have been all along. I also agree with @Panajev that dynamic should be the default

I'd love to have something like this; my main concern would be implementability. In the absence of an implementation, do you at least have a sketch of how it could be made to work at the ABI/codegen level? One way to express such a sketch would be in terms of equivalent generated code, e.g. something like the workaround you illustrate in the "Do nothing" bullet.

3 Likes

I'm glad there's support for this feature! We see people confused about protocol dispatching from time-to-time, but AFAIK there hasn't really been a serious discussion about adding opt-in dynamic dispatch to protocol extensions.

FWIW, this is actually in the generics manifesto, under "maybe":

There are a number of features that get discussed from time-to-time, while they could fit into Swift's generics system, it's not clear that they belong in Swift at all. The important question for any feature in this category is not "can it be done" or "are there cool things we can express", but "how can everyday Swift developers benefit from the addition of such a feature?". Without strong motivating examples, none of these "maybes" will move further along.

IMO, there are plenty of strong motivating examples for this feature, so it would be very useful to everyday Swift developers. I think that section is talking about making dynamic dispatch the default, rather than opt-in.


I guess we would construct a lookup table like we do for protocol conformances. I still need to look more closely at the corner cases surrounding how exactly protocol witness tables are constructed (e.g. if you use a protocol's default implementation, but then somebody in another module implements the requirement directly on your type, which version does Swift choose to call? Are protocol requirements entirely resolved when the first module is compiled?)

The secondary protocol example shows a basic way we could implement this, if we could work-around its problems. If we collected all dynamically-dispatched members in to a protocol behind-the-scenes, it would look something like this:

protocol Collection_Extensions {
  func myAlgorithm()
}
extension Collection_Extensions where Self: Collection {
  func myAlgorithm() {
    print("Collection default")
  }
}
extension Collection_Extensions where Self: RandomAccessCollection {
  func myAlgorithm() {
    print("RandomAccessCollection default")
  }
}

func doIt<C: Collection_Extensions>(_ val: C) {
  val.myAlgorithm()
}

extension Array: Collection_Extensions {}
doIt([1,2,3]) // RandomAccessCollection default

So then the only question is how to make all Collections conform to that protocol (or - how to we tell the compiler & runtime that everything which conforms to Collection also has a conformance somewhere for this extension protocol?). Also, we would need to create separate protocols to collect fileprivate and private members in (although those would hopefully get devirtualised and optimised away).

There are still open questions about the implementation, no doubt, but it looks do-able.

1 Like
Terms of Service

Privacy Policy

Cookie Policy