Pitch: Protocol Metatype Extensions

[compnerd (Saleem Abdulrasool)]

@jrose I think the bikeshedding is going back to where I started from:

metatype extension P {
 ...
}

which originated from ... @jrose (and why you are already listed in the proposal text).

[jrose (Jordan Rose)]

Putting types in extension MyProto already has a meaning (they appear on all implementing types).

[xwu (Xiaodi Wu)]

@jrose

Putting types in extension MyProto already has a meaning (they appear on all implementing types).

Sure, as does putting instance members in extensions. But this pitch is arguing that there's something special or especially compelling about static members and, to that end, bikesheds that explicitly don't generalize/permit these other uses are a plus and not a minus, right?

[compnerd (Saleem Abdulrasool)]

@xwu

But this pitch is arguing that there's something special or especially compelling about static members and, to that end, bikesheds that explicitly don't generalize/permit these other uses are a plus and not a minus, right?

I did come around to the removal of the static (Pitch: Protocol Metatype Extensions - #13 by compnerd). In fact, I've already updated the implementation and proposal text with that - the members are now instance members on extension P.Protocol.

[ksluder (Kyle Sluder)]

If I understand correctly, the useful distinction is between customizable and non-customizable type-level declarations. Swift already has ways to specify non-customizable, non-instance declarations: free functions and global properties. The only functionality unique to protocol func would be the ability to call private static methods declared in extensions on the same protocol. The discoverability advantage of keeping protocol funcs closer than free functions seems to be the biggest advantage.

But is that juice worth the squeeze?

[xwu (Xiaodi Wu)]

But is that juice worth the squeeze?

Yeah, I think this is the crux of the question.

Some of the more straightforward spellings discussed above have raised objections that they'd be "attractive nuisances" even if semantically sound, hence the talk about equivalence of metatype instance methods and type static methods, roundabout spellings like extension P where Self == Never, etc.

If we worry off the bat that the feature is worth having only if we deliberately pessimize its spelling, and one alternative is a free function...

[compnerd (Saleem Abdulrasool)]

Okay, now that https://forums.swift.org/t/pitch-a-vision-for-com-interoperability-in-swift has been posted, I can share some additional context here. This pitch is motivated by the work for COM Interop.

Reducing the COM interop design: the general idea is that COM is an ABI system for programming with bounded existential discovery. You work almost entirely with existentials and opaque types, with the root type being IUnknown. IUnknown provides 3 critical features:

1. AddRef (swift_retain)
2. Release (swift_release)
3. QueryInterface (as?)

With that in mind, each interface has type metadata associated with it: the IID (Interface ID). The pitch is meant to enable each interface (protocol) to provide metadata — its identifier. That is, IUnknown.IID would spell the protocol metatype identifier. A concrete type does not inherit this interface ID, but rather has its own unique type identifier (CLSID — the class ID).

[grynspan (Jonathan Grynspan)]

So IID is like P.Protocol while CLSID is like T.Type where T: P?

[xwu (Xiaodi Wu)]

Would it suffice (and would it be sound) to make COM interfaces "blessed" like Obj-C protocols to self-conform, since on a cursory reading it seems the features that would make it broadly unsound to self-conform are not supported for COM interfaces anyway, and thereby automagically enable extension IDrawable where Self == any IDrawable?

[compnerd (Saleem Abdulrasool)]

@xwu

Would it suffice (and would it be sound) to make COM interfaces "blessed" like Obj-C protocols to self-conform, and thereby automagically enable extension IDrawable where Self == any IDrawable?

This isn't really about self-conforming types, but rather giving the ability for a protocol to have a requirement.

protocol COMIdentifiable {
  var IID: IID { get }
}

All protocols of type IUnknown and IUnknown should be COMIdentifiable. But, instances of that protocol should not conform to that protocol.

e.g.

var object: IUnknown = ...
_ = object.Type.IID // valid
_ = object.IID // invalid

[michelf (Michel Fortin)]

But if this is compiler-synthesized, why do we need a syntax to express it? Just like P.Protocol is synthesized and magically not available on types implementing P, the IID member could be magic and not available on types conforming to a protocol.

[Joe_Groff]

Folks have asked for this functionality before not only for protocols, but also generic types, where there is sometimes desire to use the generic type's name only as a namespace for some nongeneric declarations, rather than add members to every instance of the generic type. It might be interesting to consider a syntax that could work for adding members to the namespace of not only a protocol name, but also potentially a type name in the future.

[xwu (Xiaodi Wu)]

@compnerd

This isn't really about self-conforming types, but rather giving the ability for a protocol to have a requirement.

protocol COMIdentifiable {
  var IID: IID { get }
}

All protocols of type IUnknown and IUnknown should be COMIdentifiable. But, instances of that protocol should not conform to that protocol.

Ah, I see: for the purposes of COM interoperability, this feature would be used specifically for IID as a requirement to be enforced for COM interfaces (which are themselves protocols)?

Does this have to be enforced as a protocol requirement, or can this be synthesized ad-hoc on every @COM(IID:)-decorated protocol by the macro itself (and thereby guaranteed to exist)?

[compnerd (Saleem Abdulrasool)]

@xwu

Does this have to be enforced as a protocol requirement, or can this be synthesized ad-hoc on every @COM(IID:)-decorated protocol by the macro itself (and thereby guaranteed to exist)?

The attribute is not a macro, it is a built-in compiler attribute that fundamentally alters the shape of the reference-counted type adopting the protocol. The IID, as @grynspan so concisely stated, is in effect a moniker for P.Protocol, and the CLSID is the moniker for T.Type where T: P. Recursive expansion of the protocol hierarchy forms a header prefix, an itable, permitting the object to be interpreted through any of its supported interfaces. Each entry in this table is synthesized from the interface shape, tying together the moniker (IID) and its requirements.

The construction of the type depends on inspection of the interface: we query the protocol to retrieve the IID to emit into the side table consulted during type casting of the object to its various interfaces.

While ad-hoc emission is plausible, it feels significantly more fragile than a general language enhancement, which has been requested independently of COM interop.

@xwu

Ah, I see: for the purposes of COM interoperability, this feature would be used specifically for IID as a requirement to be enforced for COM interfaces (which are themselves protocols)?

Yes, the goal is to describe the requirement that all @COM protocols provide IID. Ideally, we would then be able to extend this (as @Joe_Groff suggested) so that types that conform to any @COM protocol(s) provide a CLSID (if one is provided - it is possible to have a COM type that does not allow instantiation externally).

[compnerd (Saleem Abdulrasool)]

At the risk of derailing this conversation, I can share more of the desired eventual state:

@COM(IID: "d3dd2823-f1a9-42e3-9756-1325ee54e863")
public protocol ITerminal {
  var prompt: String { get set }
  func read() throws(COMError) -> String
  func write(_ text: borrowing String) throws(COMError)
}

@COM(IID: "d2619c0e-c5f4-484f-886b-ab572d4bcc73")
public protocol IResizable {
  func resize(height: Int, width: Int) throws(COMError)
}

@COM(CLSID: "3e72022e-fb51-4fab-b6e3-285da0531be0")
final public class Terminal: ITerminal {
  ...
}

let terminal: some ITerminal = Terminal.create()
try terminal.prompt = "$ "
try terminal.write("> ")
let input = try terminal.read()

if let resizable = terminal as? any IResizable {
  resizable.resize(height: 25, width: 80)
}

Behind the scenes, it is possible to synthesize:

extension Terminal.Type {
  public var CLSID: CLSID { CLSID("3e72022e-fb51-4fab-b6e3-285da0531be0") }

  public func create() throws(COMError) -> some ITerminal {
    try return CoCreateInstance(CLSID, ITerminal.IID)
  }
}

where CoCreateInstance is a wrapper from the COM module. But the point is, that the protocol metatype extensions are what gives us the natural spelling for __uuidof(T) in Swift rather than building that into the compiler as a special case (like clang and cl do).

[Danny]

@allevato

Thanks for this example, because it proves my point—the code above is highly unintuitive. It also permits nonsense constructions like let p = Never.makeP(), which is the only alternative way to call it without relying on an explicit type annotation on the binding it's assigned to or the function that accepts the existential as an argument.

It's fine if you squint at it just right once you know the trick, but it's not a pattern I'd enjoy teaching to others and I wouldn't care to adopt it in my own code.

You can choose hate, if you like, but it will not stop a subset of this proposed feature from already existing, and causing conflict.

Note that declaring members this way pollutes the namespace of each concrete type by creating members like DefaultToggleStyle.default , but we believe this is an acceptable trade-off to improve call site ergonomics.

[Danny]

@Joe_Groff

Folks have asked for this functionality before not only for protocols, but also generic types, where there is sometimes desire to use the generic type's name only as a namespace for some nongeneric declarations, rather than add members to every instance of the generic type. It might be interesting to consider a syntax that could work for adding members to the namespace of not only a protocol name, but also potentially a type name in the future.

Even more so than protocol metatype extensions, this feature effectively exists already, by only providing the extension for one particular type:

enum E<T> { }
extension E<Never> {
  static var i: Int { 0 }
}

E.i

What we have now allows for sub-namespacing:

extension E<Never> {
  enum E {
    static var i: Int { 0 }
  }
}

E.E.i

Are the proposed metatype extensions going to provide us a better way to do the equivalent namespacing, for protocols?

protocol P { }
extension Never: P { }
extension Int: P { }
extension P where Self == Never {
  typealias X = Never
  static var p: some P { 0 }
}

.X.p as any P

[Jumhyn (Frederick Kellison-Linn)]

@Danny

it will not stop a subset of this proposed feature from already existing, and causing conflict.

Could you elaborate on the conflict you are picturing? I’m not sure I see the issue. To the extent this would potentially allow two different ways to look up member in _: any P = .member I don’t see how that’s different in kind than any other issues we have today with members being introduced in protocol extensions, of types with conditional conformances etc. Might require some priority rules to avoid breaking source but I’m not sure I see it as a hard barrier like you’re describing?

[tera]

@allevato

extension P.Protocol {
  public static func makeP(args...) -> any P {
    if something { S1() } else { S2() }
  }
}

// Client
let p = P.make(args...)

Logically in that case it should be then called as:

P.Protocol.make(args...)

which is not ideal...

I like @mishal_shah 's idea of protocol func... Protocols (and their extensions) are special in that contrary to the other (normal) types, the bare func foo() or func foo() {} require or extend the conforming types, and to "opt out" of that default behaviour we could use something like "protocol func foo() {}" indeed. Note that it would probably not make sense to have it in the main protocol body (without brackets / implementation) – only in protocol extensions.

@jrose

because extension MyProto behaves in practice like extension some MyProto, in that up till now everything in it has applied to some concrete type (which is why you're allowed to use Self in that context to mean "the conforming type"), and protocol func will diverge from that while still being declared alongside the other members.

Up until now we didn't have protocol func... and that notion could well be the way to convey the meaning that it is NOT about extending conforming types (or making a requirement on them).

[Danny]

Along with what I've questioned above, I've got a few ideas about the conflicts/problems that might arise. I can put effort into bringing them up after the proposal is amended to do the necessary addressing:

One concrete concern is whether the kind of static member lookup proposed here would be ambiguous with static member lookup on a hypothetical future protocol metatype property. We do not believe it would be, since lookup could be prioritized on the metatype over conforming types. Further, these kinds of namespace and lookup conflicts would likely need to be addressed in a future metatype extension proposal regardless of whether the lookup extension proposed here is accepted or not.