Introducing role keywords to reduce hard-to-find bugs

[anandabits (Matthew Johnson)]

I like your approach to default implementations Doug.

@Douglas_Gregor

public protocol ExampleProtocol {
   func thud(with value: Float) { /* default implementation */... }
}

I think this is desirable regardless of how we handle default implementations in extensions.

Regarding this example:

@Douglas_Gregor

extension ExampleProtocol where Self: MyProtocol {
    func ExampleProtocol.thud(with value: Float) {
        thunk(with: value)
     }
}

Would you also suggest allowing this syntax in concrete conforming types as @DevAndArtist suggested ? This would occasionally be quite useful in that context.

You mentioned that the above syntax is a bit more verbose. It is, especially if the name of the protocol is lengthy. Would it make sense to allow Self to be used as well in protocol extensions?

extension ExampleProtocol where Self: MyProtocol {
    func Self.thud(with value: Float) {
        thunk(with: value)
     }
}

Did you have any thoughts on how to address the problem of unintentional default implementations (Error 3 in the new draft)?

@Erica_Sadun do you have concrete motivation for Error 3 being a real problem? It is clearly a potential issue but I have not run into it. It seems much less likely to cause trouble than the others and should be independently motivated.

[Paul_Cantrell]

@Douglas_Gregor

The suggestion I made about qualifying member names covers this case in a manner that is more verbose but (IMO!) more clear:

extension ExampleProtocol where Self: MyProtocol {
  func ExampleProtocol.thud(with value: Float) {
    thunk(with: value)
  }
}

I like approaches that surface & make explicit the “as if they had different names” behavior of methods that come from semantically distinct extensions.

I’m unsure whether this, and many of the other variants at hand, will do a good job of catching likely programmer errors. “Unsure” here is not a politeness; I really don’t know!

Thinking through one example out loud here, what if I intended to provide a default implementation of thud, but miss? Hmm. Here's a misspelling:

extension ExampleProtocol where Self: MyProtocol {
  func ExampleProtocol.thub(with value: Float) { … }
}

Here presumably I would get an error to the effect of “ExampleProtocol has no member thub.”

What if I misspell and forget to specify ExampleProtocol?

extension ExampleProtocol where Self: MyProtocol {
  func thub(with value: Float) { … }
}

• Near miss matching in the compiler might give me a warning.
• If the compiler doesn’t detect near misses, then I might find out about it when I get errors about my conforming types missing a thud implementation.
• If I meant to override a default thud implementation from an unqualified ExampleProtocol extension, then … we just don’t detect the error. Is that too esoteric to worry about?

Multiply that times, what, a dozen scenarios or so? I wish I'd been keeping a running catalog during this discussion of all the different extension method scenarios and all their potential developer errors.

[griotspeak (TJ Usiyan)]

What is the simplest way to satisfy two different protocol's requirements using the same implementation, in this model?

[Tino]

Imho: Leaving out the protocol names -- and I think that's absolutely ok, and we shouldn't optimize for such a corner case.

[Erica_Sadun]

@Douglas_Gregor What do you think about me starting a new thread with no solutions and a goal list. I'd list of the errors we've categorized, along with their examples, to allow discussion to continue without a design standing in the way? I am personally not married to any particular design.

[Douglas_Gregor]

@anandabits

Regarding this example:

Douglas_Gregor:

extension ExampleProtocol where Self: MyProtocol {
    func ExampleProtocol.thud(with value: Float) {
        thunk(with: value)
     }
}

Would you also suggest allowing this syntax in concrete conforming types as @DevAndArtist suggested ? This would occasionally be quite useful in that context.

Yes, absolutely.

@anandabits

You mentioned that the above syntax is a bit more verbose. It is, especially if the name of the protocol is lengthy. Would it make sense to allow Self to be used as well in protocol extensions?

That feels ambiguous again... in my example, Self conforms to both ExampleProtocol and MyProtocol.

@anandabits

Did you have any thoughts on how to address the problem of unintentional default implementations (Error 3 in the new draft)?

No, I don't have any specific thoughts on that. My not-fully-considered reaction is that I don't think that particular issue is common enough to worry about.

Doug

[Douglas_Gregor]

@Paul_Cantrell

Thinking through one example out loud here, what if I intended to provide a default implementation of thud, but miss? Hmm. Here’s a misspelling:

extension ExampleProtocol where Self: MyProtocol {
  func ExampleProtocol.thub(with value: Float) { … }
}

Here presumably I would get an error to the effect of “ExampleProtocol has no member thub.”

Yes, it would be an error.

@Paul_Cantrell

What if I misspell and forget to specify ExampleProtocol?

We don't have to guard against every mistake, especially compound mistakes. Consider the override checking we have for classes: if you both forget override and misspell the overriding method, you won't get an error. I'm okay with that, and I haven't seen particularly many user complaints about this.

Doug

[Douglas_Gregor]

Either leave off the protocol qualifier or have two functions (they're distinct entities):

extension MyType {
  func ProtocolA.foo() { }
  func ProtocolB.foo() { }
}

I suspect that this situation is quite rare.

Doug

[Douglas_Gregor]

I don't feel that the design is standing in the way; we have a proposal with one design and there are some other ideas also under discussion. I guess I'd prefer to keep the discussion going in this thread. The proposal could evolve to include more of the interesting examples, which would be great regardless of whether any of the other ideas discussed in this thread end up making it into the proposal.

Doug

[griotspeak (TJ Usiyan)]

Fair enough. My only objection to the first answer is that it makes it more attractive (to some) to leave the protocol qualifier off most of the time.

[Paul_Cantrell]

@Douglas_Gregor

Consider the override checking we have for classes: if you both forget override and misspell the overriding method, you won’t get an error.

Yes, that’s convincing.

[Erica_Sadun]

@Douglas_Gregor

Either leave off the protocol qualifier or have two functions (they’re distinct entities):

extension MyType {
func ProtocolA.foo() { }
func ProtocolB.foo() { }
}

I suspect that this situation is quite rare.

Doug

This kind of thing worries me deeply. I dislike the idea that you can satisfy overlapping protocols with different implementations.

If there were no default implementations in extensions (or in future protocol declarations), there would never be a problem. The protocols would strictly describe API surfaces, providing a bag of related semantics. You declare conformance and implement them. This was the case before adopting extensions.

Protocol extensions (and future protocol implementations) seem to be something distinct, a kind of pseudo superclass for conforming types. Unlike class, you can inherit behaviors that are in conflict because of multiple inheritance.

Leaving aside the other error types already enumerated, dealing with the issue of conforming to two protocols that you do not own that both supply default implementations is yet another source of problems.

If you separate protocols (the API surfaces) from a pseudotype of protocol extensions, you could conform to a protocol without inheriting default implementations. This would solve the "I don't own that" scenario and allow you to opt in to which protocol-supplied implementation is most appropriate to your needs.

This isn't a naming issue. ProtocolA.foo() and ProtocolB.foo() go completely against the notion that conforming to either protocol require foo(). Instead:

extension MyType: ProtocolA & ProtocolB {
   adopt ProtocolB.foo()
}

This, of course, will break down if something else in ProtocolA has some expectations of behaviors or side effects from foo() (which, bad, shouldn't), but it provides exactly one implementation of foo.

You can also pick which defaulted foo you want to use in code:

// These are actual calls, not declarations
foo() // whatever foo is defined as, yours or theirs
ProtocolA.foo() // call the default implementation of ProtocolA's foo

Using adopt here and explicit naming for member access also solves the compiler problem that you import two CocoaPods (forgive me) with conflicting default implementations under separate protocols. (It does not solve the problem of important two CocoaPods (again, forgive me) with conflicting public default implementations for the same protocol.)

I'm not even going to try to pitch any idea of renaming or restructuring protocol extensions as a kind of generic pseudo-supertype because reasons.

[Panajev (Goffredo Marocchi)]

Unpopular as it may be as a statement, beyond sharing code between value types, default implementations in a protocol are a bad idea that is almost always going to create more problems than it causes (especially when it mixes up with static dispatching, dynamic dispatching, and message passing).

A Java interface/Objective-C and Swift protocols are meant to decouple API from implementation and the moment you are depending on default implementations you are again blurring the lines between the two. I would be totally fine if there was a system only designed to allow default implementations only for value types...

[Douglas_Gregor]

@Erica_Sadun

This kind of thing worries me deeply. I dislike the idea that you can satisfy overlapping protocols with different implementations.

This seems unavoidable, albeit rare. Two separately-developed protocols might have the same requirement in them---and there's no guarantee that the semantics will be the same. If it happens, my proposal gives us a way to address the ambiguity.

@Erica_Sadun

If you separate protocols (the API surfaces) from a pseudotype of protocol extensions, you could conform to a protocol without inheriting default implementations. This would solve the “I don’t own that” scenario and allow you to opt in to which protocol-supplied implementation is most appropriate to your needs.

This isn’t a naming issue. ProtocolA.foo() and ProtocolB.foo() go completely against the notion that conforming to either protocol require foo(). Instead:

extension MyType: ProtocolA & ProtocolB {
   adopt ProtocolB.foo()
}

This, of course, will break down if something else in ProtocolA has some expectations of behaviors or side effects from foo() (which, bad, shouldn’t), but it provides exactly one implementation of foo.

While I would certainly like for there to be one foo if this case occurs in the real world, it is possible that there is no single foo implementation that satisfy the semantic requirements of ProtocolA and ProtocolB. We can say "Swift doesn't support that" (and the user will have to write a wrapper for one of the conformances, for example), or we can allow the distinction to be expressed in the language. The distinction already exists at the implementation model: the "witness table" for MyType : ProtocolA has a different entry for foo than the "witness table" for MyType: ProtocolB. In a sense, I'm arguing that this distinction should be surfaced in the language level for the (rare!) cases where it matters.

FWIW, I'm pulling this feature directly from C#.

Doug

[Erica_Sadun]

@Douglas_Gregor

This seems unavoidable, albeit rare. Two separately-developed protocols might have the same requirement in them—and there’s no guarantee that the semantics will be the same. If it happens, my proposal gives us a way to address the ambiguity.

This is when I curl up in a ball and summon the spirit of @dabrahams for guidance.

[Tino]

Afair I was the first to argue against this plan, and I still think the problem in question is better solved with Default implementations in protocol declarations (combined with the func Protocol.method() idea) instead of adding a keyword.

But I think with a small yet significant change default func could become much more useful:

extension ExampleProtocol {
    // Error 1
    // Error: Does not satisfy any known protocol requirement
    // Fixit: replace type with Float
    public default func thud(with value: Double) { ... }
}

Instead of throwing an error, we could "simply" add thud to the official methods declared in ExampleProtocol.
Basically, this would solve the disadvantage of to much repetition from the other direction, because obviously, you wouldn't include that method in the declaration anymore.

That alone imho still doesn't justify a change -- but there's more:

protocol Foo {
	var foo: String { get }
}

protocol Bar {
	var bar: String { get }
}

extension Foo where Self: Bar {
	default func foobar() {
		print(foo, bar)
	}
}

class Foobar: Foo, Bar {
	var bar: String = "bar"
	var foo: String = "foo"
}

Foobar().foobar()

If we could do this (it might be rather complicated, and nothing for the near future -- we might even have extension Foo & Bar by then ;-), we could define all kinds of functionality that is possible when some protocols are combined (maybe there's even a specific example where for that ;-)

[Paul_Cantrell]

Without having a well-formed opinion about the details, I do like the spirit of Douglas’s C#-derived proposal above.

@Douglas_Gregor

This seems unavoidable, albeit rare. Two separately-developed protocols might have the same requirement in them—and there’s no guarantee that the semantics will be the same. If it happens, my proposal gives us a way to address the ambiguity.

This at the heart of this whole thread, isn’t it? Thinking out loud here, there are two whole families of concerns we want to bring into alignment:

1. There’s the semantic implications of all these different ways of providing method implementations, and the mental model the language provides for reasoning about them. We have the twin problems of “these are semantically distinct but their names collide” and “these are supposed to be semantically coupled but the language forces them to be distinct.” Those problems will always be with us, but at least we can aspire to model them in the language so that developer understanding matches actual behavior.
2. Then there’s the implementation constraints of how each method materializes in a witness table / vtable. I barely understand all of that, and I think that's as it should be: a developer should be able to form a comfortable mental model of the language’s semantics without knowing all its implementation details. We want to facilitate that.

The general approach here, if I understand it correctly, is to let #2 drive the design as much as it needs to, but fold any behavior those implementation constraints induce into the mental model of #1.

In other words, if the way witness tables work forces a particular method not to use dynamic dispatch, we want the compiler to help the developer reason about that in terms of which methods are semantically distinct, and not in terms of how the actual dispatch is implemented.

---

In this whole thread, there’s been a lot of discussion about syntax and syntactic weight, but underneath all that I’m looking for the programmer-level mental model, the answer to #1 above, that makes me look at each use case we’ve talked about and think, “Yes, of course it should behave that way,” “Of course that would induce a warning,” etc. — all without having to reason about or even know about any implementation details.

A good example of the kind of mental model I'm searching for is what we have with structs. They’re value-typed. That's it! Things like the meaning of let vs. var just fall out nicely when you wrap your head around “the whole struct is a single value, and changing any part changes the whole value.” Implementation details like COW and optimizations that modify in place are important for performance tuning, but aren’t something we have to reason about to understand the semantics of our code.

I don’t yet see a tidy mental model of that kind for extension methods. I can’t articulate what it would be, but I can say that none of these answers will sit quite right until we find it.

What I like about Douglas’s idea is that it seems to lead in the direction of that sort of mental model. Is there a way of using it to give method implementations a uniform meaning to the developer, regardless of whether they are nested in a class or struct, an extension, or (pending inline default impls) in a protocol?

[dabrahams (Dave Abrahams)]

@Erica_Sadun

This is when I curl up in a ball and summon the spirit of @dabrahams for guidance.

Unball thyself forthwith, Ms. Sadun! Now, how may I be of assistance?

[Erica_Sadun]

@dabrahams

Unball thyself forthwith, Ms. Sadun! Now, how may I be of assistance?

Really difficult to come up with a tl;dr version here. Word of Dave™ originally stated that a protocol is a bag of related semantics. A well designed protocol should move beyond the notion of compliant API surface that allows generic functionality to be written.

That sounds like a protocol that is just a bag of syntax without
semantic requirements, which—forgive my pretentious tone—goes against
the principles of generic programming on which the standard library was
founded. We won't have a “plusable” for things you can apply the “+”
operator to, for similar reasons.

-- Dave A

So far, simple.

Then protocol extensions landed and hell rained from the sky: Two protocols might implement the same name, so which one should automatically be adopted? Two protocols might implement the same name with different implementation semantics, making it necessary to call the nearest protocol declared by the scope. Or the scope might adopt both protocols, making it necessary (like here) to distinguish which implementation should be run within the same context. To make it worse, you might not own the protocols or the extensions.

This goes well beyond my list of seven "role-hintable" error types. My errors centered on accidental name confluence, accidental misspellings, and changes in parent protocols that weren't propagated to their point of implementation.

With extensions, the notion of Dave A™ Semantics has hit the wall and has become "a group of related syntax" if we have to implement both ProtocolA.foo() and ProtocolB.foo(). How can Swift adapt to resolve these protocol naming conflicts, protocol default implementation conflicts, protocol adoption conflicts, and so forth?

I summoned you hoping you'd have some insight into something so simple that has spawned so much potential for unsafe use.

The rest is upthread.

[dabrahams (Dave Abrahams)]

@Erica_Sadun

With extensions, the notion of Dave A™ Semantics has hit the wall and has become “a group of related syntax” if we have to implement both ProtocolA.foo() and ProtocolB.foo().

I'm not sure you'll find this comforting, but all of these problems existed before protocol extensions. You could always conform to two protocols that independently define same-named requirements, or even syntactically identical requirements with different meanings. These problems even existed long before Swift. Any language with "interfaces" (c.f. Java, ObjC protocols) or multiple inheritance has exactly the same issue with different abstractions. I'm pretty sure (although I've forgotten more of the language than I ever knew) you can even find this problem in very strict languages like Haskell. Any language in which free functions can satisfy customization points (e.g. C++) has a similar problem.

Even if you discard all the language mechanisms that can get the program into trouble, the basic issue still exists: two independently-developed components can use the same or similar names with very different meanings. The computer can handle it, but it's still a problem for humans. So at its root, this is an artifact of having a library ecosystem.

@Erica_Sadun

How can Swift adapt to resolve these protocol naming conflicts, protocol default implementation conflicts, protocol adoption conflicts, and so forth?

I think Doug's suggested approach is an excellent one. He and I have been thinking about approaches to this problem since at least as far back as 2002. I always knew it would force its way to the surface in Swift eventually. Swift 1 didn't have tools for addressing it only because there were higher priorities at the time, and IMO because a feature like this one might be something you want to design after more of the language has evolved, to avoid shaping the core of the language around something that's rarely a problem in practice.

Hope this Helps,
Dave

P.S. regarding things that are “rarely a problem in practice:” I don't take that phrase as reassurance, since the problems that rarely happen are the ones nobody protects themselves against and really hurt when they bite. I'm just saying it's hard to justify adding language weight to common code in order to avoid those problems. The weight should appear only in the rare cases where some help is needed.