Pre-Pitch: Explicit protocol fulfilment with the 'conformance' keyword

Evolution Pitches
12

Pitch was edited with a new "The conformance keyword could be made able to "auto-fix" some near misses" paragraph in the future directions.

13

Maybe the pitch would be better if I made explicit an underlying idea regarding documentation.

The conformance keyword is pitched as optional. One does not have to use it. One has benefits using it, because it reveals some present or future mismatches between the programmer intent, and what the compiler sees. By "present mismatch", I mean typos and near-misses. By "future mismatch", I mean code that must change because a dependency has changed the definition of a protocol.

It is a tool that I initially intended for three groups of people:

  1. Programmers who are aware of the benefits of the keyword, for their own code base.
  2. Documentation writers, who would start recommending the use of the conformance keyword in the documentation of some protocols, when relevant.
  3. Clueless documentation readers, who use the conformance keyword because that's what the sample code says in the documentation. And that's as good for them as it is good for the programmers who are aware of what they are doing.

The benefits of conformance for the first group have been explained above.

The benefits of conformance for the documentation writers is two-fold:

  1. Some documentation can be simplified. The example below comes from GRDB (original), and was considered necessary due to the high risk of miss due to type-inference:

    Note: make sure the databaseSelection property is explicitly declared as [any SQLSelectable]. If it is not, the Swift compiler may silently miss the protocol requirement, resulting in sticky SELECT * requests.

    This paragraph aims at avoiding this kind of bug:

    struct Player: TableRecord {
    // MISS: '[Column]' is not '[any SQLSelectable]'
    static let databaseSelection = [Column("id"), Column("name")]
}

    The conformance keyword would enter the sample code provided in the documentation. When the reader follows the advice, it is for their own benefit.

  2. Migration guides can be simplified. Again, consider this GRDB example (original):

    It is no longer possible to override persistence methods such as insert or update. Customizing the persistence methods is now possible with callbacks such as willSave, willInsert, or didDelete [...].

    [When upgrading] you have to remove the methods below from your own code base: [followed by a list of now-retired customization points].

    Of course the GRDB author is sorry to introduce breaking changes. That's why there is a migration guide. This guide would have been much easier to write if users could have used conformance before. The compiler would have shown the obsolete requirements without any ambiguity, guiding the user in the upgrade. Without it, an exhaustive list of functions that need to be updated is difficult to build (even if one groups conformances in dedicated extensions).

1 Like
14

What about this reunion of our ideas:

The initial pitch was: the conformance keyword looks for statically-known protocol requirements at the location it is used (and errors out if the declaration has no match).

When used inside an extension that declares a protocol conformance, we could have the conformance keyword perform requirement lookup in those protocols only.

To illustrate, consider the following program. It compiles without error with the pitch as written. But it would not with the suggested amendment:

protocol Foo { func foo() }
protocol Bar { func bar() }

struct S { }

extension S: Bar { }
extension S: Foo {
    conformance func foo() { }
    // Initial pitch: no error
    // Amended pitch: function bar() does not fulfill any 'Foo' requirement.
    conformance func bar() { }
}

I'm still not 100% convinced, because I do not like the idea that moving a declaration can make it valid or invalid. Yet this change would help solving some ambiguities that are discussed in the "Known Limits" paragraph of the pitch. The balance is difficult.

Following the same idea, we could have:

extension Foo where Self: Bar {
    // Conformance lookup performed in Foo and Bar only
    conformance func bar() { ... }
}

extension Foo {
    // Conformance lookup performed in Foo and Bar only
    conformance func bar() where Self: Bar { ... }
}

This amendment, if implemented later, would be a breaking change (since some code would stop compiling). This means it must come with the initial proposal, or not at all (making the implementation more difficult). I humbly let the community decide if it's worth it.

3 Likes
15

Can conformance be added in protocol extensions for default implementations? Also, I think it’d be nice if we could have conformance extensions, which I think would handle the design pattern outlined by @xwu. Overall, this feature has a lot of promise. I’m interested to see if there are benefits to compilation times using this approach. Additionally, adoption is another important question. I think conformance should become the default as it aids readability, diagnosing near misses and potentially the compiler. If we decide that this keyword should be required, then features such as autocomplete and migration strategies should be discussed to facilitate adoption.

2 Likes
16

Maybe

conformance extension MyType: SomeProtocol {
  func ...
  var ...
}

…could be sugar for

extension MyType: SomeProtocol {
  conformance func ...
  conformance var ...
}

… and that such a "conformance extension" would produce diagnostics if it included (non-private) members that did not conform to the protocol(s) listed?

7 Likes
17

This looks nice at first sight (maybe as a future direction, since this does not look required for the pitch to provide value).

Yet beware: such a conformance extension would ONLY accept declarations that fulfill requirements. This is the hard version of the "group protocol conformances in extensions" styling, because even related methods would become forbidden in such an extension:

conformance extension SortedArray: Collection {
    var first: ...  // OK
    var last: ...   // OK
    var median: ... // error: property 'median' does not fulfill any protocol requirement. 
}

To me, this is the exact point were something nice that always help turns into an ill-designed tool that frequently annoys. I let the community consider if they prefer something good or something too perfect.

As you say, @sveinhal, we'd ignore methods that do not match the visibility of the protocol:

conformance extension SortedArray: Collection {
    var first: ...  // OK
    var last: ...   // OK
    private func someHelper() { ... } // OK
}
5 Likes
18

This is a great pitch, and given the precedent of override, seems like maybe it was an oversight in the original Swift implementation. I like that it is opt-in for compatibility, but would also like to see a way to opt-in to require this.

I do think we should move directly to including the conforming protocol too, as it solves a lot of remaining problems and can leverage the implementation for @_implements.

19

Maybe besides the point for the pitch at hand, but for whatever it's worth: I actually prefer this hard line. Allow for private implementation details such as helper functions, but everything that is not part of the implementation, or privately aids the implementation, should be kept outside the extension, imho.

4 Likes
20

This is true. It would be tricky to distinguish between helper methods and methods that are actually intended to conform to another protocol.

I have had cases where I have two protocols that are very closely related, and I ended up moving methods from one protocol to the other (kinda like table view data source vs delegate - the line can be a bit blurry). With such a change, any protocol-specific extensions are now incorrectly organized, but the compiler doesn't care. So I'm looking for a way to express my intent to the compiler.

21

+1

I hit this again the other day. We have really strange witness matching that often feels inconsistent.

For example, if you have a protocol requirement which accepts a concrete type, a generic method can witness it:

protocol MyProtocol {
  func doSomething(_: String)
}

struct MyStruct: MyProtocol {
  // Works, because String conforms to StringProtocol
  func doSomething<S: StringProtocol>(_: S) { fatalError() }
}

So you might expect, if there is a protocol requirement which returns an existential, that a method which returns a concrete conforming type would also be able to witness it:

protocol MyProtocol {
  func doSomething() -> any StringProtocol
}

struct MyStruct: MyProtocol {
  // Computer says no.
  func doSomething() -> String { fatalError() }
}

If the requirement has a default implementation, the type still conforms, but resolves to an unexpected implementation :frowning:

The idea of only warning about these mismatches in extensions does not seem like an adequate solution, to me. Some types are designed to perform one task and be written in a single, compact declaration - so it is unusual that anybody would split the implementation up across extensions, and hence they would not receive any diagnostics. Types declared inside functions don't support extensions at all:

func someFunction() {
  struct MyStruct {}
  extension MyStruct {}
  ^^^^^^^^^
  error: declaration is only valid at file scope
}
4 Likes
22

Maybe this is what you are looking for: Pre-Pitch: Explicit protocol fulfilment with the 'conformance' keyword - #14 by gwendal.roue

extension T {
     // Requires any conformance
     conformance func foo() { ... }
}

extension T: MyProtocol {
     // Requires a MyProtocol conformance
     conformance func foo() { ... }
}

The pitch has not been amended with this feature yet - I'm still not sure it should belong to the pitch at this point.

23

As another data point, there is mutating for mutating func other property accessors so conforming could play a decent role here too

24

If a type T continues to fulfill the protocol requirements of protocol P, but contains extra APIs that are no longer necessary for conformance, this is not a correctness issue. Presumably, those APIs were implemented in a way that is semantically correct for type T, and they will continue to function correctly when called by users whether or not they are required by P because they make sense for the concrete type T.*

If those APIs never made sense for T, then T should not have conformed to an earlier version of P in the first place. If there's some narrow circumstance where it was desired that T should not expose some API required by P but for its requirement to conform, but nonetheless T could properly implement those APIs in a semantically sound way, then that's an argument for standardizing the @_implements feature because it would have done better by allowing T to conform to all versions of P without directly exposing the APIs in question under the required name.

Users are free to use any style they want, but it is self-evidently true that expressing the same idea one way in code can be clearer about intentionality than another way.

Near-miss diagnostics for protocol conformances are available for extensions because it is in that context that the user's intentionality is clear enough for those diagnostics not to be nuisances, not because the people implementing the feature decided that they wanted to "favor one particular programming style."

It is fair enough to consider if the design of the language can be changed to improve expressivity so that the user's intention can be made clear more consistently, but I'm skeptical of additions to Swift's syntax that are made expressly because a user doesn't want to write code in a certain style which already solves the problem. In a sense, such accommodation—and not the status quo—is what's favoring a particular programming style.

You are describing today's Swift: a member in an extension declaring a protocol conformance which is a near-miss match for a requirement currently produces a warning that can be silenced by moving the declaration to another extension. Are you making the argument that this warning is actively harmful, and do you have evidence for such harm? Independent of what you are pitching, are you arguing that the current warning should be removed from the compiler?

4 Likes
25

I am a hard-liner too. I'd say:

extension T {
     // no conformance allowed here
     conformance func foo() { ... } // error
}

extension T: Prot1 {
     conformance func foo() { ... } // from Prot1, ok
     conformance func bar() { ... } // Error, no bar in Prot1
}

extension T: Prot1, Prot2 {
     conformance func foo() { ... } // from Prot1, ok
     conformance func bar() { ... } // from Prot2, ok
}

obviously this as well:

extension T: Prot1 {
     func foo() { ... } // Error, "conformance" keyword required
     conformance func baz() { ... } // not in Prot1
}
1 Like
26

This is an interesting and very useful case you bring up, which should have a good solution.

I haven't tried it out, but I would have thought that protocol requirements can have availability annotations; certainly, that is what I would have reached for if confronted with such a scenario. Does Swift currently support—and if not, would it have solved your problem here if it did support—the ability to label customization points as deprecated or obsolete? Conforming types which implement such an API would then get the specified warning/error.

27

Can you elaborate on this a bit? Is static var databaseSelection: [any SQLSelectable] { get } a requirement of your protocol? If so, and a conforming type doesn't implement it because of a near-miss, how is Swift allowing the code to compile despite an invalid conformance?

28

No, it is not.

For example, GRDB exposes several "record protocols" that user types can adopt, for converting themselves to or from database row, as well as feeding an SQL query builder from the name of a table. The PersistableRecord protocol has deeply modified its customization points in GRDB6. Previously, users were able to provide a custom implementation of methods such as insert. They must no longer do that, due to the breaking changes. Instead, the customization points are now named willInsert, didInsert, etc.

The conformance has always been intended by the programmer, and correctly implemented.

But the protocol has changed, and programmer's code has to be updated. The consequences of a breaking change. Breaking changes happen in the wild.

If the conformance keyword would exist, the compiler would perform, for the programmer, the exhaustive listing of methods that have to be reconsidered:

struct Player: PersistableRecord {
    // After GRDB6 upgrade:
    // error: function insert(...) does not fulfill any protocol requirement
    conformance func insert(...) { ... }
}

Now the programmer has a choice:

  • Keep the method, but remove the conformance keyword, since the programmer must not express the intent to fulfil a requirement, when no protocol requires such a method.
  • Delete the method.
  • Delete the method and reimplement the feature with the new apis.

What's important is that the program won't compile until the programmer has made this choice. That's the security brought by conformance.

29

I think Karl has well explained that some types can't provide conformances with extensions:

I understand that you like extensions, but this pitch really wants to provide useful features to users who don't, or can't, or did not learn to group conformances in extensions. Call this my personal touch :-)

Now, in several answers above I have extended the original pitch with more features dedicated to such extensions. Example. The idea was further developed in other responses. I think it can be fruitful.

30

Yes, it is.

If so, and a conforming type doesn't implement it because of a near-miss, how is Swift allowing the code to compile despite an invalid conformance?

Because the requirement has a default implementation. If users don't specify which database columns they want to fetch, the default is to fetch all of them (SELECT *).

31

This is not what is pitched, as explained above:

  • In this response, you'll see why only relying on extension-that-declare-a-conformance was not chosen.
  • In this response, Karl reminds that some types can not be extended.

This is why the pitch brings value even without such extensions.

However, in several answers above, the idea to give more features to such extensions is explored. The pitch can surely be improved :-)

1 Like