Pitch: Protocol Metatype Extensions

Evolution Pitches
1

Hello All!

Swift protocols serve two roles: they define requirements that conforming types implement, and they act as types in their own right (any P). But there's a gap in what you can express about the protocol itself as opposed to the types that conform to it.

Consider a protocol for application plugins:

protocol Plugin {
  var name: String { get }
  func activate()
}

You might want to track which plugins are available or provide a default search path. Today, the only option is a protocol extension:

extension Plugin {
  static var searchPaths: [String] { ["/usr/lib/plugins", "~/.plugins"] }
}

But this has two problems:

  1. searchPaths is inherited by every conforming type: MyPlugin.searchPaths compiles and returns the same paths, even though search paths are a property of the plugin system, not of any individual plugin.
  2. Plugin.searchPaths — the natural way to access it — doesn't compile at all. Static extension members can't be accessed on protocol metatypes.

extension P.Protocol solves both problems at once:

extension Plugin.Protocol {
  static var searchPaths: [String] { ["/usr/lib/plugins", "~/.plugins"] }
}

  Plugin.searchPaths    // works
  MyPlugin.searchPaths  // error — not inherited

Scope

This proposal is limited to static members. Protocol metatype extension members are statically dispatched as they're provided directly by the extension, with no witness table involvement. They cannot reference Self, since there's no conforming type in scope. These members are effectively namespaced free functions with a natural home on the protocol.

Instance members on protocol metatypes are not addressed by this proposal. They are not inherently nonsensical — P.Protocol is itself a type, and values of that type could in principle have instance members — but P.Protocol is a singleton (there is exactly one value: P.self), so instance members would be functionally equivalent to static members with a different calling convention. The feature becomes significantly more interesting if it generalises to existential metatype extensions (P.Type), where different conforming types' metatypes are distinct values and instance members dispatch meaningfully.

Why .Protocol?

In Swift's type system, P.Protocol is already the type of the protocol value itself (the type of P.self), while P.Type is the existential metatype (the type of some unknown conforming type's metatype). Since we're extending the protocol itself and not conforming types, .Protocol is the semantically correct choice.

What this enables today

  • Protocol-level configuration: search paths, default values, environment settings, etc that belong to the abstraction itself, not to any one implementation.
  • Protocol-level utilities: registries, discovery mechanisms, factory methods, etc that are namespaced on the protocol they serve, rather than floating as free functions or hidden in enum namespaces.
  • Cleaner API design: library authors can attach metadata to their protocols without polluting the member namespace of every conforming type. Users see exactly what belongs to the protocol and what belongs to their type.

The full proposal text is available here which also links an initial implementation.

Saleem

25 Likes
2

Love it.

I think the proposal undersells another great way this can be used, since it only shows uses of static properties. The protocol metatype is also an ideal place to hang factory methods when you want the conforming type to remain completely hidden from users:

// Library
public protocol P {}
internal struct S1: P {}
internal struct S2: P {}

extension P.Protocol {
  public static func makeP(args...) -> any P {
    if something { S1() } else { S2() }
  }
}

// Client
let p = P.make(args...)

I've wanted to write that a few times and it's been frustrating having to design around it.

Some questions about the little details:

  1. What happens if with protocol refinement?

    protocol P {}
protocol Q: P {}
extension P.Protocol { static func f() {} }

// Is this valid?
Q.f()

  2. From #1, if extension Q.Protocol also defined static func f(), presumably it would be called if I wrote Q.f() instead of P.Protocol.f?

  3. If refining protocols inherit the metatype members of those they refine, how are collisions dealt with? Is this ambiguous?

    protocol P {}
protocol Q {}
extension P.Protocol { static func f() {} }
extension Q.Protocol { static func f() {} }
protocol R: P, Q {}
R.f()  // is this ok?

  4. Does this support protocol compositions? Can I define extension (P & Q).Protocol that would define metatype members on (P & Q), and depending on the answers above, possibly also on protocols that refine both?

  5. Does this work if I assign the protocol metatype to a variable?

    protocol P {}
extension P.Protocol { static func f() {} }

let p = P.self
p.f()  // is this ok?

Weird inconsistencies with protocol vs. existential metatypes in the language

Swift has accrued a lot of warts around metatypes involving protocols vs. existentials that have never been fixed. I wonder if we might need to deal with those before making progress here.

The pitch says

In Swift's type system, P.Protocol is already the type of the protocol value itself (the type of P.self), while P.Type is the existential metatype (the type of some unknown conforming type's metatype).

While this is true, it feels like the compiler's behavior doesn't consistently reflect this.

For example, it's not clear to me whether the metatype of a protocol inherits from the metatype of the protocol(s) it refines (which is why I asked the questions above). The compiler certainly doesn't help here:

protocol P1 {}
protocol P2: P1 {}
let y: P1.Protocol = P2.self
                     ˄˜˜˜˜˜˜
                     ╰─ error: cannot convert value of type '(any P2).Type' to specified type '(any P1).Type'

If SomeProtocol.self is meant to return the protocol metatype here, I'm not sure why the compiler is telling me about existential metatypes in the diagnostics. Enabling upcoming feature ExistentialAny also doesn't help here.

Here's another oddity (though it isn't strictly related to your proposal):

protocol P {}
let x = P.self
let y = (any P).self

If I dump the AST of this snippet, both x and y have the same type: (any P).Type. Is that actually right? If we're saying that the protocol metatype P.self is distinct from the existential metatype (which I would expect to write as (any P).self, shouldn't those also be different?

4 Likes
3

Note that in our modern syntax, these are spelled (any P).Type and any P.Type (or just P.Type), respectively.

2 Likes
4

Oof, I had forgotten that (any P).Type explicitly became equivalent to P.Protocol as part of the previous existential work. Personally, I find that this:

extension (any P).Type {
  static func f() {}
}

is far more confusing than the proposed

extension P.Protocol {
  static func f() {}
}

for this specific use case. If I'm accessing one of these members, it's on a bare P. any doesn't come into the picture at all.

3 Likes
5

I definitely find extension (any P).Type to be awkward.

Interestingly, I had originally started with extension P.Type, and thanks to @Jumhyn went with the extension P.Protocol spelling. One thing that I find extension P.Type allows, is as a generalisation of this pitch, a better reading in:

protocol P { }
struct S: P { }

extension P.Type {
  func instantiate() -> any P { self.init() }
}

would allow you to then do:

S.self.instantiate()
6

Yeah of the options available I find P.Protocol the least confusing, though it probably does mean we need to admit (any P).Type as the 'modern' version of that spelling. Since we have both any P.Type and (any P).Type in the language, I think that extension P.Type is underspecified.

Hm perhaps I'm misunderstanding but isn't that just static members in protocol extensions? The following works today:

protocol P { init() }
struct S: P { }

extension P {
  static func instantiate() -> any P { self.init() }
}

S.instantiate() // this is the same as `S.self.instantiate()`

I thought the whole intention here is that we're not extending conforming types by adding static members on P itself.

1 Like
7

Here's one for the real sickos. Speaking of factory functions, what if I wanted to do this:

protocol P {}

extension P.Protocol {
  static func callAsFunction(/* args */) -> any P { ... }
}

let x = P(/* args */)

Today in Swift, you can't really define a natural static callAsFunction on a concrete type T, because the syntax T(...) always attempts to resolve to T.init(...) and never to T.callAsFunction(...). To call it, you'd have to explicitly write T.callAsFunction(...), taking away the nice syntactic sugar.

With a protocol metatype, there is no such ambiguity, because P.init(...) is meaningless. So we could make this work. I don't know if we should, but we could!

9 Likes
8

Q.f() is not valid.

  • P.Protocol is (any P).Type — the metatype of the existential type any P
  • P.Type is any P.Type — the existential metatype (metatype of some unknown conforming type)

any P and any Q are distinct existential types even when Q: P. So (any Q).Type and (any P).Type are unrelated metatypes.

An extension P.Protocol adds members to (any P).Type, and Q.Protocol is (any Q).Type, which is a different type entirely.

It does not, but, if it did, the spelling should've been Q.f().

Again, this boils back down to 1: R.Protocol is distinct and thus this is not accepted.

No, this is not supported, and is properly diagnosed. We require a nominal type for the extension, and (P & Q) is non-nominal.

Yes, I believe that should work.

1 Like
9

Yes, this would not be something that is possible to do in the current proposal. I do not have a good motivating example to show for non-static members which is why the proposal is limiting to just static members.

10

Stepping back, I'm confused about the choice in this pitch to extend the metatype and mark the member as static.

Given a type T with static func x(), to invoke I write T.x(). When T is a prototype metatype P.Protocol, that means I'd write P.Protocol.x(). Yet this pitch says that I can write P.x(): that doesn't compose from the general rule?

If we want P.x(), then x() should either be an instance member of the existential metatype or a static member of the existential, no?

It seems to me the spelling for the latter would be:

extension any P {
  static func x() { ... }
}

[I checked and we do currently reject such a declaration, so it's free for the taking.]
[The confusing (but unavoidably so) thing about it is that a naive reading says this "extends any [conforming] P," when in fact it really "extends no conforming P but only specifically the one type any P that isn't a P at all"—but we bought this trouble long ago.]

9 Likes
11

I think I agree that if the spelling is extension P.Protocol (or extension (any P).Type) then the members ought to be non-static.

I worry that making extension any P meaningful-but-different from extension P puts us in quite a confusing position. I think the use for this feature is fairly narrow and extension any P would be quite an attractive nuisance especially as we encourage any P to be written in more and more places—it would be fairly natural to see any P in a type signature and write extension any P to extend it. Not wrong and might even work for your purpose but subtly different than what you probably should be doing.

So I think we'd be better off 'hiding' this feature behind the (any P).Type or P.Protocol extension.

(I think this is also the better option if/when we ever get around to allowing the declaration of self-conformance for protocol existentials. I don't think we'd ever want to allow conforming (any P) to P in a way which permitted instance-level requirements to be satisfied by anything other than forwarding to the underlying value, but IMO it feels fairly natural to say that in many cases the "missing" bit to allow self-conformance is that we don't generate an implicit conformance (any P).Type: P.Type, which you can declare and implement yourself!)

2 Likes
12

I too am a bit confused about the use of static here. And I’m not too thrilled in having an extension on the Protocol meta type either because it sounds a bit complicated to explain.

My preference would be to have it expressed the same was as regular static method but replacing static with a different modifier, perhaps like this:

extension P {
   protocol func x() { ... }
}

The precedent it builds on is how you can use class instead of static inside a class when you need the “static” method to be overridable. In this case the special meaning is it would make the “static” method only available on the protocol itself—with P.x()—instead of all the conforming types.

This to me seems more approachable than having to plunge into explanations about “meta types” and what’s a P.Type versus a P.Protocol.

7 Likes
13

I have come around to the idea that static should be dropped. What persuades me is not just the compositional argument, but the future direction toward extension P.Type. For P.Type, instance members are not optional — different conforming metatypes are distinct values, so members must dispatch on self. If P.Protocol extensions use static now, we'd end up with two metatype extension forms that work differently for no reason a user could easily explain. Dropping static on P.Protocol establishes a uniform model: metatype extensions add instance members to metatype values, whether there's one such value (P.Protocol) or many (P.Type).

That said, I do have a concern about approachability. The lack of static is technically correct, but the reader must understand what instance the member is being bound to — "the singleton metatype value P.self" is not a concept most Swift developers have thought about. Throughout the rest of Swift, static signals that a member belongs to the type itself rather than to an instance. The question is whether we should take @xwu's point to heart and begin making the distinction visible — that static members are not truly "on the type" but sugar for instance members on the metatype. Should this proposal be the place where that becomes explicit?

The original design trades off mechanics for the mental model. I can appreciate how static is inaccurate in this position, and the future-evolution story is what tips me toward dropping it.

1 Like
14

I like the idea behind the pitch, but I think the spelling of extension XXX.Protocol is breaking too many conventions.

First, as xwu have said:

Second, it has been taught well in the community that Swift nowadays only allows extensions on nominal types. I believe the pitched spelling is the first one to break this rule.

This rule may be lifted one day, of course, but extension XXX.Protocol is still a little bit different: it is more restrictive about what can be put into the body:

extension P.Protocol {
  // should we allow these non-static members?
  // if we wan't to ban them, for what reason?
  var a: Int { /*...*/ }
  func foo() {}
}

As a consequence, I prefer michelf's way that we are still writing extensions for P:

It's just reusing keyword protocol in protocol func seems awkward for me. How about

extension P {
  static(self) func x() { ... } 
}

I choose self to indicate the method is for P.self, aligning with the fact that both T.x() and T.self.x() should mean the same thing.

15

Yes, indeed, very confusing: but it does fall out of composing our existing rules. I agree that the status quo of just not allowing it is probably the most usable take though: we could have special diagnostics to redirect users to write whatever we decide to spell this as instead.

I think @michelf's alternative to riff on class func—it's a static func with special rules relevant for classes!—with protocol func here, writing them on extension P, is probably the least bad option here.

4 Likes
16

I think this is going to break this feature: protocols that conform to themselves and Objective-C protocols already effectively have metatype extensions.

private struct Instance: Error { }
extension Error where Self == any Error {
  static var instance: Self { Instance() }
}

.property as any Error
1 Like
17

The way that compiles already (the general case just requires any conforming type—Never is a fine choice given how it already conforms to so much else):

public protocol P {}
extension Never: P { }
private struct S1: P {}
public extension P where Self == Never {
  static func makeP() -> any P { S1() }
}

let p: any P = .makeP()

I think every use case of this pattern, where a protocol metatype vends itself or an opaque instance of itself, is better served by having Never adopt the protocol in question. I'd like to see this be automatic, but it's no big deal to Claude it.

This does not handle the general use case, where metatype members can return whatever they please. I'd like to see that, but it needs to not break the subset of functionality that already exists—unless it replaces that!

18

Thanks for this example, because it proves my point—the code above is highly unintuitive. It also permits nonsense constructions like let p = Never.makeP(), which is the only alternative way to call it without relying on an explicit type annotation on the binding it's assigned to or the function that accepts the existential as an argument.

It's fine if you squint at it just right once you know the trick, but it's not a pattern I'd enjoy teaching to others and I wouldn't care to adopt it in my own code.

2 Likes
19

Not all protocols can be adopted by Never, and the places where this inference kicks in are limited to cases that return an instance of the protocol (cf SE-0299, though this case isn't generic). So it's not a general-purpose thing.

Abstractly, I don't love protocol func foo() because extension MyProto behaves in practice like extension some MyProto, in that up till now everything in it has applied to some concrete type (which is why you're allowed to use Self in that context to mean "the conforming type"), and protocol func will diverge from that while still being declared alongside the other members. But in practice those members might still be related even if they don't match up in their type information, so maybe it's not a big deal. I do agree that involving any MyProto in this would be confusing, though, even if (any MyProto).Type is the compiler-correct description of where these methods live.

That said, I don't think protocol as a modifier works well for other things you might want to nest on a protocol but not its conforming types. protocol typealias? protocol struct? protocol protocol? That has me wanting a solution at the extension level again, not the member level.

More bikeshed colors, not necessarily better than `protocol func`
  • extension protocol MyProto { func foo() {} }
  • protocol extension MyProto { func foo() {} }
  • extension MyProto.self { static func foo() {} } (why static? because it goes on "the thing being extended" instead of "instances of the thing being extended", but maybe that's too subtle / unnecessary complexity)
2 Likes
20

If protocol func is used in the same vein as class func (i.e, as a 'flavor' of static), you wouldn't have protocol typealias, protocol struct, protocol protocol, right?

Just as we don't have class typealias, class class, class protocol (class protocol: class—eek).

1 Like