[Proposal] clarification around protocol implementation and protocol extensions

Evolution Discussion
1

I wrote up the following draft proposal around attempting to clarify protocol implementation and protocol extensions, via annotating methods and properties meant to satisfy protocol requirements with additional keywords:

"Types implementing protocols with optional features (either through default implementations on extensions, or optional methods on objective-C protocols) do not have a way to detect either local typos or later upstream changes to the protocol method signatures will result in behavioral changes. This proposal attempts to rectify that in a manner similar to override on subclassed types."

I’d appreciate feedback from the list! This is very much a first draft, likely filled with tyops.

-DW

2

This draft proposes something that's been discussed on this list several
times over the past few years, but now with a different spelling. However,
previous objections never centered around the spelling.

As discussed in previous threads on this topic, any such required keyword
breaks retroactive conformance. That is, if I want to conform someone
else's type to my protocol, I must edit that person's code if your idea is
accepted.

In so doing, this idea rejects an important (essential, IMO) mental model
of what protocol conformance entails. Specifically, I conform a type T to a
protocol P when I discover, having already implemented T, that it fulfills
the semantic and syntactic requirements of P. Unlike subclassing, I'm not
setting out with the intention of making a type T that conforms to P;
rather, I discover this fact to my great delight after implementation of T.
The analogy to subclassing is therefore inaccurate. Notionally, one always
inherits from a superclass first, then overrides.

This is no mere theoretical notion. It is a Swift idiom to state
conformance to protocols in an extension. Sometimes, the type will already
implement some but not all of the protocol's requirements outside that
extension because it is essential to the raison d'être of the type, and
only the balance of the requirements would then be implemented in the
extension that states conformance. It would be exceedingly unfortunate to
forbid this pattern, which is both elegant and expressive, but OTOH it
would be strange to have a method declare that it fulfills a requirement of
a protocol to which, lexically, it had not yet been conformed.

Lastly, it makes a very common task, that of conforming to a protocol,
syntactically heavy. This is exacerbated when you consider that the scheme
outlined here has the greatest benefit only when conformance to one
protocol is concerned. When conforming to more than one protocol, a similar
level of compiler help can only be obtained if you allow annotations as to
which protocol's requirement is being fulfilled. To implement this
additional syntax would be ludicrously heavy, but to ignore the issue would
be failing to consider one major aspect of protocols: composability (as
opposed to single inheritance). Again here, this is a critical difference
from subclassing that makes adapting "override" (regardless of whether you
choose a novel spelling) ineffectual.

Previous discussions on this topic have agreed that the developer
experience of conforming a type to a protocol can be greatly improved, but
the consensus has been that requiring a keyword to indicate protocol
requirements is not the way to go about it. I too would love to see
progress made on this issue, but having been previously an advocate for a
solution similar to yours, the views of some very wise people have changed
my thinking.

···

On Fri, Feb 17, 2017 at 21:29 David Waite via swift-evolution < swift-evolution@swift.org> wrote:

I wrote up the following draft proposal around attempting to clarify
protocol implementation and protocol extensions, via annotating methods and
properties meant to satisfy protocol requirements with additional keywords:

"Types implementing protocols with optional features (either through
default implementations on extensions, or optional methods on objective-C
protocols) do not have a way to detect either local typos or later upstream
changes to the protocol method signatures will result in behavioral
changes. This proposal attempts to rectify that in a manner similar
to override on subclassed types."

I’d appreciate feedback from the list! This is very much a first draft,
likely filled with tyops.

[SE-NNNN] Fullfill protocol requirements explicitly with extensions and types · GitHub

-DW
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

3

This draft proposes something that's been discussed on this list several times over the past few years, but now with a different spelling. However, previous objections never centered around the spelling.

As discussed in previous threads on this topic, any such required keyword breaks retroactive conformance. That is, if I want to conform someone else's type to my protocol, I must edit that person's code if your idea is accepted.

I believe this proposal is different in that it only requires specification for clarity not for behavioral changes, and only requires the keyword to be specified for protocols known at the site of method/property declaration.

In so doing, this idea rejects an important (essential, IMO) mental model of what protocol conformance entails. Specifically, I conform a type T to a protocol P when I discover, having already implemented T, that it fulfills the semantic and syntactic requirements of P. Unlike subclassing, I'm not setting out with the intention of making a type T that conforms to P; rather, I discover this fact to my great delight after implementation of T. The analogy to subclassing is therefore inaccurate. Notionally, one always inherits from a superclass first, then overrides.

This is no mere theoretical notion. It is a Swift idiom to state conformance to protocols in an extension. Sometimes, the type will already implement some but not all of the protocol's requirements outside that extension because it is essential to the raison d'être of the type, and only the balance of the requirements would then be implemented in the extension that states conformance. It would be exceedingly unfortunate to forbid this pattern, which is both elegant and expressive, but OTOH it would be strange to have a method declare that it fulfills a requirement of a protocol to which, lexically, it had not yet been conformed.

The declaration is only required on types for protocols which the type is known to implement at the file level. If some other code (even within the same module) extends a type to support a protocol, there is no requirement that previous methods had been documented as fulfilling that new protocol.

In this sense, not every method which is mapped into a protocol may be marked as fulfilling a protocol, but the purpose isn’t to solve every corner case. If that were indeed the purpose, I could propose something much wordier where the extension has to declare that intent to expose existing methods as part of protocol conformance.

Lastly, it makes a very common task, that of conforming to a protocol, syntactically heavy. This is exacerbated when you consider that the scheme outlined here has the greatest benefit only when conformance to one protocol is concerned. When conforming to more than one protocol, a similar level of compiler help can only be obtained if you allow annotations as to which protocol's requirement is being fulfilled. To implement this additional syntax would be ludicrously heavy, but to ignore the issue would be failing to consider one major aspect of protocols: composability (as opposed to single inheritance). Again here, this is a critical difference from subclassing that makes adapting "override" (regardless of whether you choose a novel spelling) ineffectual.

The purpose isn’t to map methods to protocols, but to clarify that a method exists to meet the requirements of a protocol. Without this, a simple typo (or redefinition of methods such as with the changes in Swift 3’s importer) causes what you thought was conforming to a protocol to either fail, or worse silently change behavior.

Previous discussions on this topic have agreed that the developer experience of conforming a type to a protocol can be greatly improved, but the consensus has been that requiring a keyword to indicate protocol requirements is not the way to go about it. I too would love to see progress made on this issue, but having been previously an advocate for a solution similar to yours, the views of some very wise people have changed my thinking.

There are often cases for the compiler to guess a developer’s intent to simplify syntax, but in this case I believe people are being bit by the compiler guessing incorrectly. If protocols didn’t make implementation of its requirements optional (via optional methods in objective-c, and protocol extensions in Swift) then there wouldn’t be a need to annotate methods to clarify intent. Given those features, I have trouble imagining a system where a compiler would be able to understand the developer’s intent and prevent issues without tagging protocol implementations in some manner.

Unless I’m incorrect in my assumption that there compiler could only infer developer intent via some sort of method tagging, then it really is a decision whether clarity or terseness is more important in this scenario.

-DW

···

On Feb 17, 2017, at 10:53 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

4

>
> This draft proposes something that's been discussed on this list several
times over the past few years, but now with a different spelling. However,
previous objections never centered around the spelling.
>
> As discussed in previous threads on this topic, any such required
keyword breaks retroactive conformance. That is, if I want to conform
someone else's type to my protocol, I must edit that person's code if your
idea is accepted.

I believe this proposal is different in that it only requires
specification for clarity not for behavioral changes, and only requires the
keyword to be specified for protocols known at the site of method/property
declaration.

If indeed you frame your proposal in this way, a best effort to add clarity
of intention, my feedback is that there are ways to gain the same (or at
least, very similar) benefits without being source-breaking, and indeed
without introducing new keywords at all.

For example:
- Doc comments are parsed. We already have `- Parameter` in doc comments,
which improved tooling could double-check to ensure that the stuff we put
there matches the stuff in the function signature (hmm, that'd be a nice
additive proposal, right?).
- The same tool could be augmented to recognize a spelling such as `-
RequiredBy:`, and you could list the specific protocols that require the
method. Then, you would be warned at compiled time if your method is in
fact not required by a protocol. This design has the side benefit of
telling the reader of your code exactly what protocols require that method,
not just that it's required by some protocol.

> In so doing, this idea rejects an important (essential, IMO) mental
model of what protocol conformance entails. Specifically, I conform a type
T to a protocol P when I discover, having already implemented T, that it
fulfills the semantic and syntactic requirements of P. Unlike subclassing,
I'm not setting out with the intention of making a type T that conforms to
P; rather, I discover this fact to my great delight after implementation of
T. The analogy to subclassing is therefore inaccurate. Notionally, one
always inherits from a superclass first, then overrides.
>
> This is no mere theoretical notion. It is a Swift idiom to state
conformance to protocols in an extension. Sometimes, the type will already
implement some but not all of the protocol's requirements outside that
extension because it is essential to the raison d'être of the type, and
only the balance of the requirements would then be implemented in the
extension that states conformance. It would be exceedingly unfortunate to
forbid this pattern, which is both elegant and expressive, but OTOH it
would be strange to have a method declare that it fulfills a requirement of
a protocol to which, lexically, it had not yet been conformed.

The declaration is only required on types for protocols which the type is
known to implement at the file level. If some other code (even within the
same module) extends a type to support a protocol, there is no requirement
that previous methods had been documented as fulfilling that new protocol.

In this sense, not every method which is mapped into a protocol may be
marked as fulfilling a protocol, but the purpose isn’t to solve every
corner case. If that were indeed the purpose, I could propose something
much wordier where the extension has to declare that intent to expose
existing methods as part of protocol conformance.

> Lastly, it makes a very common task, that of conforming to a protocol,
syntactically heavy. This is exacerbated when you consider that the scheme
outlined here has the greatest benefit only when conformance to one
protocol is concerned. When conforming to more than one protocol, a similar
level of compiler help can only be obtained if you allow annotations as to
which protocol's requirement is being fulfilled. To implement this
additional syntax would be ludicrously heavy, but to ignore the issue would
be failing to consider one major aspect of protocols: composability (as
opposed to single inheritance). Again here, this is a critical difference
from subclassing that makes adapting "override" (regardless of whether you
choose a novel spelling) ineffectual.

The purpose isn’t to map methods to protocols, but to clarify that a
method exists to meet the requirements of a protocol. Without this, a
simple typo (or redefinition of methods such as with the changes in Swift
3’s importer) causes what you thought was conforming to a protocol to
either fail, or worse silently change behavior.

As I demonstrate above, there are other designs possible that do not
require source breakage, or even any new keywords, which recover
essentially all of the benefits you list here. Such alternative designs
would be opt-in, so those who care about this additional compiler help can
use it, and those who do not won't need to, consistent with Swift's new
emphasis on compatibility and its established emphasis on progressive
disclosure.

···

On Sat, Feb 18, 2017 at 1:07 AM, David Waite <david@alkaline-solutions.com> wrote:

> On Feb 17, 2017, at 10:53 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

Previous discussions on this topic have agreed that the developer
experience of conforming a type to a protocol can be greatly improved, but
the consensus has been that requiring a keyword to indicate protocol
requirements is not the way to go about it. I too would love to see
progress made on this issue, but having been previously an advocate for a
solution similar to yours, the views of some very wise people have changed
my thinking.

There are often cases for the compiler to guess a developer’s intent to
simplify syntax, but in this case I believe people are being bit by the
compiler guessing incorrectly. If protocols didn’t make implementation of
its requirements optional (via optional methods in objective-c, and
protocol extensions in Swift) then there wouldn’t be a need to annotate
methods to clarify intent. Given those features, I have trouble imagining a
system where a compiler would be able to understand the developer’s intent
and prevent issues without tagging protocol implementations in some manner.

Unless I’m incorrect in my assumption that there compiler could only infer
developer intent via some sort of method tagging, then it really is a
decision whether clarity or terseness is more important in this scenario.

-DW

5

I believe this proposal is different in that it only requires specification for clarity not for behavioral changes, and only requires the keyword to be specified for protocols known at the site of method/property declaration.

If indeed you frame your proposal in this way, a best effort to add clarity of intention, my feedback is that there are ways to gain the same (or at least, very similar) benefits without being source-breaking, and indeed without introducing new keywords at all.

For example:
- Doc comments are parsed. We already have `- Parameter` in doc comments, which improved tooling could double-check to ensure that the stuff we put there matches the stuff in the function signature (hmm, that'd be a nice additive proposal, right?).
- The same tool could be augmented to recognize a spelling such as `- RequiredBy:`, and you could list the specific protocols that require the method. Then, you would be warned at compiled time if your method is in fact not required by a protocol. This design has the side benefit of telling the reader of your code exactly what protocols require that method, not just that it's required by some protocol.

IMHO doc comment is no different than a keyword or attribute flagging the capability other than the color paint used on the bike shed. I’d disagree with an attribute or doc comment unless ‘override’ was also moved to be similar.

Documentation styles differ greatly between developers and projects, but I personally subscribe to the idea of self-documenting internal code, resorting to documentation when that is not possible. Published documentation (doc comments) are reserved for when code is meant to be accessed externally.

Even then, I wouldn’t document an *implementation* of a protocol unless the protocol was being implemented by a public type *and* left flexibility in the behavior of the implementation, Otherwise, I would rely on the documentation of the protocol.

For the vast majority of code, I simply don’t see people using doc comments by default. So this feature would have them creating doc comments for the purpose of indicating "RequiredBy"

Second, most of the reasons for marking a method or property as being required by a protocol go away if such a feature is optional per-method. Developers scanning code looking for the cause of odd behavior aren’t going to notice that there is a typo in the method signature, still, nor are they going to see a missing “RequiredBy” doc comment as a clue unless their coding style requires such (most likely enforced by a third party linter)

All that said, I’ve seen plenty of code attempt to indicate such a “RequiredBy” feature via extensions, e.g.

  extension ClassName : ProtocolName { … }

I could see a compromise of an opt-in feature where such an extension is expected to contain the body of the protocol implementation (or of multiple protocols, e.g. Equatable and Hashable), and that any non-private method in the extension would result in a warning if it was not applied toward implementing the specified protocols.

Is that something people would be interested in? I don’t know if it is just the weekend, but this thread has been very quiet.

The purpose isn’t to map methods to protocols, but to clarify that a method exists to meet the requirements of a protocol. Without this, a simple typo (or redefinition of methods such as with the changes in Swift 3’s importer) causes what you thought was conforming to a protocol to either fail, or worse silently change behavior.

As I demonstrate above, there are other designs possible that do not require source breakage, or even any new keywords, which recover essentially all of the benefits you list here. Such alternative designs would be opt-in, so those who care about this additional compiler help can use it, and those who do not won't need to, consistent with Swift's new emphasis on compatibility and its established emphasis on progressive disclosure.

The design I specified does not result in source breakage. There would be warnings for Swift 3 code compiled under Swift 4 saying that the keywords were missing, and that the behavior as if the keywords were specified is assumed.

Swift 4 code using the keywords would not be backward compatible with Swift 3, but I know of no similar backward compatibility requirements that Swift 4 code compile on Swift 3.

-DW

···

On Feb 18, 2017, at 12:40 AM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Sat, Feb 18, 2017 at 1:07 AM, David Waite <david@alkaline-solutions.com <mailto:david@alkaline-solutions.com>> wrote: