[Proposal] Explicit Synthetic Behaviour

Yes, I better understand now. Please forgive the pollution.

Gwendal

···

Le 7 sept. 2017 à 19:53, Tony Allevato <tony.allevato@gmail.com> a écrit :

Again, this is not the issue that Haravikk is describing in this thread.

Right, let's make sure we're talking about the right thing here. Gwendal, your issue isn't with synthesis in the form of Codable or the new additions to Equatable/Hashable which are opt-in-by-conformance, it's with the specific case of raw value enums or enums without associated values where the synthesis is implicit with no way to opt-out. That's a big difference.

Yes.

I can definitely see the latter being an issue if it were more widespread, and I'd be supportive of those enums being required to declare their conformance for consistency (though it would be source breaking).

Yes, unfortunately.

However, I still haven't seen a real issue that has come up because of the distinction being drawn here between default implementations vs. implementations that can access other parts of the concrete type. It sounds like this discussion is trying to protect against a hypothetical problem that hasn't happened yet and may not happen; it would be helpful to show some motivating real-world cases where this is indeed a severe problem.

Yes. I'm not talking about implementation itself. I know this has been the main topic until I have tried to bring in the topic of the consequences of non-avoidable synthesis (extra methods that may conflict with userland methods).

If you ask me for a real-world case, then I think I gave one. Let me rephrase it:

it's impossible to define a value-backed enum without getting free Equatable conformance. This free conformance is sometimes unwanted, and I gave the example of DSLs. Now this problem is not *severe*. It's more a blind spot in the language, and finally just an unwanted side-effect of a compiler convenience,

Again, this is not the issue that Haravikk is describing in this thread.

I'll clarify—your issue is specifically with the fact that enums with raw values and enums without associated values receive Equatable even without explicitly conforming to it, and therefore users have no way of opting out of it. This predates SE-0185, and I didn't propose making any changes to the conformance of those enums for source compatibility reasons, though I wouldn't be opposed to it because it makes them consistent across the board.

Haravikk's argument is about synthesized conformances like Codable and Equatable/Hashable in SE-0185, where the user must explicitly conform the type to those protocols. His claim is that that act of opting in is not sufficient and that it is still dangerous if those synthesized conformances can access members that are not also declared in the protocol. That's a completely separate issue to yours, and one that I hope he'll present more evidence of. Right now, requiring that you not only explicitly conform to the protocol but also explicitly request the synthesis feels like a solution without an actual problem, and is a situation we already have today with default method implementations.

The simplest real-world case is easy:

  struct Foo { var data:String }
  extension Foo : Equatable {} // This currently produces an error, in future it will not

Why is this a problem? It's no different than if someone extended Foo to conform to a protocol with a default implementation that was written in code.

I'm sorry but I have now explained why it isn't multiple times; a non-reflective default conformance can ONLY act upon methods and properties that the protocol itself has defined, meaning that it knows everything it needs to know in order to do whatever it wants to do with those methods and properties because it defined them.

Reflective/synthesised default implementations must by their very nature make assumptions about a concrete type that are not cannot be guaranteed to be correct. The properties and methods they may end up interacting with may have nothing at all to do with the protocol. Equatable remains by far the simplest example; just because a developer has used equatable properties does not guarantee that all of them should be compared during a check for equality.

These things are two very different beasts.

While a developer may wish to override a default implementation, it should only be to provide optimisations that a protocol cannot; e.g- providing a logarithmic time search to replace to a linear time search. The end result however should be the same; i.e- there is no change in behaviour, only detail.

A synthesised/reflective implementation however may return a result that is simply incorrect, because it is based on assumptions made by the protocol developer, with no input from the developer of the concrete type. In this case the developer must override it in to provide correct behaviour.

That's not a fair characterization. Just because your concerns were disagreed with does not mean they were ignored; my understanding is that the core team views these synthesized conformances as a different kind of default method (and one which could be hoisted out of the compiler once sufficient metaprogramming facilities are available).

The way to handle synthesized conformances was discussed during the review period for Codable, during the earlier pitch a few months ago for what became SE-0185, and again during its formal review. It's not accurate to reduce the argument you disagree with to "but Codable does it" when what you're referring to is established precedent based on those prior discussions.

If that is the case then surely someone, including members of the core team, could have easily summarised what the rationale behind that was and why they think it applies to Equatable/Hashable. They did not, hence, ignored. I even framed my argument purely on the basis of why the Equatable/Hashable case differs from Codable and still nothing. It is absolutely fair for me to characterise that as being ignored.

I feel like we keep going back to this, but this statement applies equally to non-synthesized default implementations. Are you suggesting that users should have to opt-in specifically to all default implementations provided by a protocol in some way beyond merely conforming to that protocol? If not, what specifically makes synthesized conformances a special case?

As mentioned, this is a special case because the behaviour being provided is not constrained to the protocol that provides it; it is invasive of a concrete type, and by its very nature can potentially produce incorrect results.

···

On 7 Sep 2017, at 19:36, Tony Allevato <tony.allevato@gmail.com> wrote:
On Thu, Sep 7, 2017 at 11:18 AM Haravikk via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 7 Sep 2017, at 18:53, Tony Allevato via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
On Thu, Sep 7, 2017 at 10:39 AM Gwendal Roué <gwendal.roue@gmail.com <mailto:gwendal.roue@gmail.com>> wrote:

Le 7 sept. 2017 à 14:45, Tony Allevato <tony.allevato@gmail.com <mailto:tony.allevato@gmail.com>> a écrit :

Right, let's make sure we're talking about the right thing here. Gwendal, your issue isn't with synthesis in the form of Codable or the new additions to Equatable/Hashable which are opt-in-by-conformance, it's with the specific case of raw value enums or enums without associated values where the synthesis is implicit with no way to opt-out. That's a big difference.

Yes.

I can definitely see the latter being an issue if it were more widespread, and I'd be supportive of those enums being required to declare their conformance for consistency (though it would be source breaking).

Yes, unfortunately.

However, I still haven't seen a real issue that has come up because of the distinction being drawn here between default implementations vs. implementations that can access other parts of the concrete type. It sounds like this discussion is trying to protect against a hypothetical problem that hasn't happened yet and may not happen; it would be helpful to show some motivating real-world cases where this is indeed a severe problem.

Yes. I'm not talking about implementation itself. I know this has been the main topic until I have tried to bring in the topic of the consequences of non-avoidable synthesis (extra methods that may conflict with userland methods).

If you ask me for a real-world case, then I think I gave one. Let me rephrase it:

it's impossible to define a value-backed enum without getting free Equatable conformance. This free conformance is sometimes unwanted, and I gave the example of DSLs. Now this problem is not *severe*. It's more a blind spot in the language, and finally just an unwanted side-effect of a compiler convenience,

Again, this is not the issue that Haravikk is describing in this thread.

I'll clarify—your issue is specifically with the fact that enums with raw values and enums without associated values receive Equatable even without explicitly conforming to it, and therefore users have no way of opting out of it. This predates SE-0185, and I didn't propose making any changes to the conformance of those enums for source compatibility reasons, though I wouldn't be opposed to it because it makes them consistent across the board.

Haravikk's argument is about synthesized conformances like Codable and Equatable/Hashable in SE-0185, where the user must explicitly conform the type to those protocols. His claim is that that act of opting in is not sufficient and that it is still dangerous if those synthesized conformances can access members that are not also declared in the protocol. That's a completely separate issue to yours, and one that I hope he'll present more evidence of. Right now, requiring that you not only explicitly conform to the protocol but also explicitly request the synthesis feels like a solution without an actual problem, and is a situation we already have today with default method implementations.

The simplest real-world case is easy:

  struct Foo { var data:String }
  extension Foo : Equatable {} // This currently produces an error, in future it will not

Why is this a problem? It's no different than if someone extended Foo to conform to a protocol with a default implementation that was written in code.

I'm sorry but I have now explained why it isn't multiple times; a non-reflective default conformance can ONLY act upon methods and properties that the protocol itself has defined, meaning that it knows everything it needs to know in order to do whatever it wants to do with those methods and properties because it defined them.

Just because it might have defined the properties does not necessarily mean that those properties are sufficient context for providing a default implementation:
protocol Fooable : Equatable { // Equatable is just a simple example
    var myFoo: Int { get }
}

extension Fooable {
    static func ==(_ lhs: Self, _ rhs: Self) -> Bool {
        return lhs.myFoo == rhs.myFoo
    }
}

struct X : Fooable {
    let myFoo: Int
    let myName: String
    // Whoops, forgot to give an implementation of ==
}

print(X(myFoo: 42, myName: "Alice") == X(myFoo: 42, myName: "Bob")) // true
This property is necessary, but not sufficient to provide a correct implementation. A default implementation might be able to assume something about the types that it defines, but it does not necessarily know enough.

Reflective/synthesised default implementations must by their very nature make assumptions about a concrete type that are not cannot be guaranteed to be correct. The properties and methods they may end up interacting with may have nothing at all to do with the protocol. Equatable remains by far the simplest example; just because a developer has used equatable properties does not guarantee that all of them should be compared during a check for equality.

In the same way that you might consider synthesized conformances to overreach into a type and touch things which are not related to a protocol, default implementations can be considered underreach in that they don’t know anything about properties which are necessary for providing a correct implementation.

These things are two very different beasts.

I think they’re very much two sides of the same coin.

···

On Sep 7, 2017, at 1:43 PM, Haravikk via swift-evolution <swift-evolution@swift.org> wrote:

On 7 Sep 2017, at 19:36, Tony Allevato <tony.allevato@gmail.com <mailto:tony.allevato@gmail.com>> wrote:
On Thu, Sep 7, 2017 at 11:18 AM Haravikk via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 7 Sep 2017, at 18:53, Tony Allevato via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
On Thu, Sep 7, 2017 at 10:39 AM Gwendal Roué <gwendal.roue@gmail.com <mailto:gwendal.roue@gmail.com>> wrote:

Le 7 sept. 2017 à 14:45, Tony Allevato <tony.allevato@gmail.com <mailto:tony.allevato@gmail.com>> a écrit :

While a developer may wish to override a default implementation, it should only be to provide optimisations that a protocol cannot; e.g- providing a logarithmic time search to replace to a linear time search. The end result however should be the same; i.e- there is no change in behaviour, only detail.

A synthesised/reflective implementation however may return a result that is simply incorrect, because it is based on assumptions made by the protocol developer, with no input from the developer of the concrete type. In this case the developer must override it in to provide correct behaviour.

That's not a fair characterization. Just because your concerns were disagreed with does not mean they were ignored; my understanding is that the core team views these synthesized conformances as a different kind of default method (and one which could be hoisted out of the compiler once sufficient metaprogramming facilities are available).

The way to handle synthesized conformances was discussed during the review period for Codable, during the earlier pitch a few months ago for what became SE-0185, and again during its formal review. It's not accurate to reduce the argument you disagree with to "but Codable does it" when what you're referring to is established precedent based on those prior discussions.

If that is the case then surely someone, including members of the core team, could have easily summarised what the rationale behind that was and why they think it applies to Equatable/Hashable. They did not, hence, ignored. I even framed my argument purely on the basis of why the Equatable/Hashable case differs from Codable and still nothing. It is absolutely fair for me to characterise that as being ignored.

I feel like we keep going back to this, but this statement applies equally to non-synthesized default implementations. Are you suggesting that users should have to opt-in specifically to all default implementations provided by a protocol in some way beyond merely conforming to that protocol? If not, what specifically makes synthesized conformances a special case?

As mentioned, this is a special case because the behaviour being provided is not constrained to the protocol that provides it; it is invasive of a concrete type, and by its very nature can potentially produce incorrect results.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

This is a particularly interesting use case that demonstrates a weakness
without a good workaround. I, for one, am glad you brought it up, and agree
with comments above that auto-generated mandatory enum conformance is a
topic perhaps worth revisiting, given this compelling example.

···

On Thu, Sep 7, 2017 at 14:32 Gwendal Roué via swift-evolution < swift-evolution@swift.org> wrote:

Le 7 sept. 2017 à 19:53, Tony Allevato <tony.allevato@gmail.com> a écrit :

Again, this is not the issue that Haravikk is describing in this thread.

Yes, I better understand now. Please forgive the pollution.

Gwendal

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

protocol Fooable : Equatable { // Equatable is just a simple example
    var myFoo: Int { get }
}

extension Fooable {
    static func ==(_ lhs: Self, _ rhs: Self) -> Bool {
        return lhs.myFoo == rhs.myFoo
    }
}

struct X : Fooable {
    let myFoo: Int
    let myName: String
    // Whoops, forgot to give an implementation of ==
}

print(X(myFoo: 42, myName: "Alice") == X(myFoo: 42, myName: "Bob")) // true
This property is necessary, but not sufficient to provide a correct implementation. A default implementation might be able to assume something about the types that it defines, but it does not necessarily know enough.

Sorry but that's a bit of a contrived example; in this case the protocol should not implement the equality operator if more information may be required to define equality. It should only be implemented if the protocol is absolutely clear that .myFoo is the only part of a Fooable that can or should be compared as equatable, e.g- if a Fooable is a database record and .myFoo is a primary key, the data could differ but it would still be a reference to the same record.

To be clear, I'm not arguing that someone can't create a regular default implementation that also makes flawed assumptions, but that synthesised/reflective implementations by their very nature have to, as they cannot under every circumstance guarantee correctness when using parts of a concrete type that they know nothing about.

Reflective/synthesised default implementations must by their very nature make assumptions about a concrete type that are not cannot be guaranteed to be correct. The properties and methods they may end up interacting withmay have nothing at all to do with the protocol. Equatable remains by far the simplest example; just because a developer has used equatable properties does not guarantee that all of them should be compared during a check for equality.

In the same way that you might consider synthesized conformances to overreach into a type and touch things which are not related to a protocol, default implementations can be considered underreach in that they don’t know anything about properties which are necessary for providing a correct implementation.

If more information is necessary to provide a correct implementation, then a default implementation shouldn't be provided. This is what unimplemented properties and methods are for; either getting the developer to provide the missing information, or getting them to implement the correct behaviour.

···

On 7 Sep 2017, at 22:02, Itai Ferber <iferber@apple.com> wrote:

What if we tried a completely different type of solution (which might solve the issue for protocol default conformances too)?

Right now there are errors and warnings. What if we add a third type of compiler feedback (let’s call it a note). A note is a little bit like a warning, but it can be dismissed/acknowledged. It is only generated once for a particular issue, and then it is remembered that you dismissed it so it won’t come back every time you compile.

So in this case, when you add an Equatable conformance post-hoc, you get a note (let’s say it looks like a warning, but is grey instead of yellow and has a different icon) saying something like “Equatable conformance was automatically synthesized”. If you want to remember to add your own conformance, you can just leave the note there as a reminder to yourself until you do it. If you want the automatic conformance then you just dismiss the note and it never bothers you again.

Thanks,
Jon

···

On Sep 7, 2017, at 12:52 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:

This is a particularly interesting use case that demonstrates a weakness without a good workaround. I, for one, am glad you brought it up, and agree with comments above that auto-generated mandatory enum conformance is a topic perhaps worth revisiting, given this compelling example.
On Thu, Sep 7, 2017 at 14:32 Gwendal Roué via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Le 7 sept. 2017 à 19:53, Tony Allevato <tony.allevato@gmail.com <mailto:tony.allevato@gmail.com>> a écrit :

Again, this is not the issue that Haravikk is describing in this thread.

Yes, I better understand now. Please forgive the pollution.

Gwendal

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

protocol Fooable : Equatable { // Equatable is just a simple example
    var myFoo: Int { get }
}

extension Fooable {
    static func ==(_ lhs: Self, _ rhs: Self) -> Bool {
        return lhs.myFoo == rhs.myFoo
    }
}

struct X : Fooable {
    let myFoo: Int
    let myName: String
    // Whoops, forgot to give an implementation of ==
}

print(X(myFoo: 42, myName: "Alice") == X(myFoo: 42, myName: "Bob")) // true
This property is necessary, but not sufficient to provide a correct implementation. A default implementation might be able to assume something about the types that it defines, but it does not necessarily know enough.

Sorry but that's a bit of a contrived example; in this case the protocol should not implement the equality operator if more information may be required to define equality. It should only be implemented if the protocol is absolutely clear that .myFoo is the only part of a Fooable that can or should be compared as equatable, e.g- if a Fooable is a database record and .myFoo is a primary key, the data could differ but it would still be a reference to the same record.

To be clear, I'm not arguing that someone can't create a regular default implementation that also makes flawed assumptions, but that synthesised/reflective implementations by their very nature have to, as they cannot under every circumstance guarantee correctness when using parts of a concrete type that they know nothing about.

You can’t argue this both ways:
If you’re arguing this on principle, that in order for synthesized implementations to be correct, they must be able to — under every circumstance — guarantee correctness, then you have to apply the same reasoning to default protocol implementations. Given a default protocol implementation, it is possible to come up with a (no matter how contrived) case where the default implementation is wrong. Since you’re arguing this on principle, you cannot reject contrived examples.
If you are arguing this in practice, then you’re going to have to back up your argument with evidence that synthesized examples are more often wrong than default implementations. You can’t declare that synthesized implementations are by nature incorrect but allow default implementations to slide because in practice, many implementations are allowable. There’s a reason why synthesis passed code review and was accepted: in the majority of cases, synthesis was deemed to be beneficial, and would provide correct behavior. If you are willing to say that yes, sometimes default implementations are wrong but overall they’re correct, you’re going to have to provide hard evidence to back up the opposite case for synthesized implementations. You stated in a previous email that "A synthesised/reflective implementation however may return a result that is simply incorrect, because it is based on assumptions made by the protocol developer, with no input from the developer of the concrete type. In this case the developer must override it in to provide correct behaviour." — if you can back this up with evidence (say, taking a survey of a large number of model types and see if in the majority of cases synthesized implementation would be incorrect) to provide a compelling argument, then this is something that we should in that case reconsider.

Reflective/synthesised default implementations must by their very nature make assumptions about a concrete type that are not cannot be guaranteed to be correct. The properties and methods they may end up interacting withmay have nothing at all to do with the protocol. Equatable remains by far the simplest example; just because a developer has used equatable properties does not guarantee that all of them should be compared during a check for equality.

In the same way that you might consider synthesized conformances to overreach into a type and touch things which are not related to a protocol, default implementations can be considered underreach in that they don’t know anything about properties which are necessary for providing a correct implementation.

If more information is necessary to provide a correct implementation, then a default implementation shouldn't be provided. This is what unimplemented properties and methods are for; either getting the developer to provide the missing information, or getting them to implement the correct behaviour.

I agree, but you can’t selectively argue this.

···

On Sep 8, 2017, at 12:46 AM, Haravikk via swift-evolution <swift-evolution@swift.org> wrote:

On 7 Sep 2017, at 22:02, Itai Ferber <iferber@apple.com <mailto:iferber@apple.com>> wrote:

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

protocol Fooable : Equatable { // Equatable is just a simple example
    var myFoo: Int { get }}
extension Fooable {
    static func ==(_ lhs: Self, _ rhs: Self) -> Bool {
        return lhs.myFoo == rhs.myFoo
    }}
struct X : Fooable {
    let myFoo: Int
    let myName: String
    // Whoops, forgot to give an implementation of ==}
print(X(myFoo: 42, myName: "Alice") == X(myFoo: 42, myName: "Bob")) // true

This property is *necessary*, but not *sufficient* to provide a correct
implementation. A default implementation might be able to *assume* something
about the types that it defines, but it does not necessarily know enough.

Sorry but that's a bit of a contrived example; in this case the protocol
should *not* implement the equality operator if more information may be
required to define equality. It should only be implemented if the protocol
is absolutely clear that .myFoo is the only part of a Fooable that can or
should be compared as equatable, e.g- if a Fooable is a database record and
.myFoo is a primary key, the data could differ but it would still be a
reference to the same record.

To be clear, I'm not arguing that someone can't create a regular default
implementation that also makes flawed assumptions, but that
synthesised/reflective implementations *by their very nature have to*, as
they cannot under every circumstance guarantee correctness when using parts
of a concrete type that they know nothing about.

You can’t argue this both ways:

   - If you’re arguing this on principle, that in order for synthesized
   implementations to be correct, they *must* be able to — *under every
   circumstance* — guarantee correctness, then you have to apply the same
   reasoning to default protocol implementations. Given a default protocol
   implementation, it is possible to come up with a (no matter how contrived)
   case where the default implementation is wrong. Since you’re arguing this *on
   principle*, you cannot reject contrived examples.
   - If you are arguing this *in practice*, then you’re going to have to
   back up your argument with evidence that synthesized examples are more
   often wrong than default implementations. You can’t declare that
   synthesized implementations are *by nature* incorrect but allow
   default implementations to slide because *in practice*, many
   implementations are allowable. There’s a reason why synthesis passed code
   review and was accepted: in the majority of cases, synthesis was deemed to
   be beneficial, and would provide correct behavior. If you are willing to
   say that yes, sometimes default implementations are wrong but overall
   they’re correct, you’re going to have to provide hard evidence to back up
   the opposite case for synthesized implementations. You stated in a previous
   email that "A synthesised/reflective implementation however may return
   a result that is simply incorrect, because it is based on assumptions made
   by the protocol developer, with no input from the developer of the concrete
   type. In this case the developer must override it in to provide
   *correct* behaviour." — if you can back this up with evidence (say,
   taking a survey of a large number of model types and see if in the majority
   of cases synthesized implementation would be incorrect) to provide a
   compelling argument, then this is something that we should in that case
   reconsider.

Well put, and I agree with this position 100%. However, to play devil's
advocate here, let me summarize what I think Haravikk is saying:

I think the "synthesized" part of this is a red herring, if I understand
Haravikk's argument correctly. Instead, it is this:

(1) In principle, it is possible to have a default implementation for a
protocol requirement that produces the correct result--though not
necessarily in the most performant way--for all possible conforming types,
where by conforming we mean that the type respects both the syntactic
requirements (enforced by the compiler) and the semantic requirements
(which may not necessarily be enforceable by the compiler) of the protocol
in question.

(2) However, there exist *some* requirements that, by their very nature,
cannot have default implementations which are guaranteed to produce the
correct result for all conforming types. In Haravikk's view, no default
implementations should be provided in these cases. (I don't necessarily
subscribe to this view in absolute terms, but for the sake of argument
let's grant this premise.)

(3) Equatable, Hashable, and Codable requirements are, by their very
nature, such requirements that cannot have default implementations
guaranteed to be correct for all conforming types. Therefore, they should
not have a default implementation. It just so happens that a default
implementation cannot currently be written in Swift itself and must be
synthesized, but Haravikk's point is that even if they could be written in
native Swift through a hypothetical reflection facility, they should not
be, just as many other protocol requirements currently could have default
implementations written in Swift but should not have them because they
cannot be guaranteed to produce the correct result.

My response to this line of argumentation is as follows:

For any open protocol (i.e., a protocol for which the universe of possible
conforming types cannot be enumerated a priori by the protocol designer)
worthy of being a protocol by the Swift standard ("what useful thing can
you do with such a protocol that you could not without?"), any sufficiently
interesting requirement (i.e., one for which user ergonomics would
measurably benefit from a default implementation) either cannot have a
universally guaranteed correct implementation or has an implementation
which is also going to be the most performant one (which can therefore be a
non-overridable protocol extension method rather than an overridable
protocol requirement with a default implementation).

···

On Fri, Sep 8, 2017 at 4:00 PM, Itai Ferber via swift-evolution < swift-evolution@swift.org> wrote:

On Sep 8, 2017, at 12:46 AM, Haravikk via swift-evolution < > swift-evolution@swift.org> wrote:
On 7 Sep 2017, at 22:02, Itai Ferber <iferber@apple.com> wrote:

Reflective/synthesised default implementations *must by their very nature* make
assumptions about a concrete type that are *not* cannot be guaranteed to
be correct. The properties and methods they may end up interacting with*may
have nothing at all to do with the protocol*. Equatable remains by far
the simplest example; just because a developer has used equatable
properties does *not* guarantee that all of them should be compared
during a check for equality.

In the same way that you might consider synthesized conformances to
overreach into a type and touch things which are not related to a protocol,
default implementations can be considered *underreach* in that they don’t
know anything about properties which are necessary for providing a correct
implementation.

If more information is necessary to provide a correct implementation, then
a default implementation shouldn't be provided. This is what unimplemented
properties and methods are for; either getting the developer to provide the
missing information, or getting them to implement the correct behaviour.

I agree, but you can’t selectively argue this.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Counter-example: `index(of:)`, or rather, the underscored requirement underlying `index(of:)`. The "loop over all indices and return the first whose element matches" default implementation is universally guaranteed to be correct, but a collection like `Set` or `SortedArray` can provide an implementation which is more performant than the default.

···

On Sep 8, 2017, at 6:03 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:

For any open protocol (i.e., a protocol for which the universe of possible conforming types cannot be enumerated a priori by the protocol designer) worthy of being a protocol by the Swift standard ("what useful thing can you do with such a protocol that you could not without?"), any sufficiently interesting requirement (i.e., one for which user ergonomics would measurably benefit from a default implementation) either cannot have a universally guaranteed correct implementation or has an implementation which is also going to be the most performant one (which can therefore be a non-overridable protocol extension method rather than an overridable protocol requirement with a default implementation).

--
Brent Royal-Gordon
Architechies

You're close, but still missing key points:

I am not arguing that features like these should not be provided, but that they should not be provided implicitly, and that the developer should actually be allowed to request them. That is exactly what this proposal is about, yet no matter what I say everyone seems to be treating me like I'm against these features entirely; I am not.
A non-reflective default implementation can only operate on the basis of what the protocol itself defines; this means that it can always be correct within the context of only what the protocol declares. At worst, a "pure" default implementation can only be overly cautious, but strictly speaking this does not make its behaviour incorrect, though, if it is a likely case it suggests that a default implementation is not a good idea (as it is better to ensure the developer provides any further information).
Synthetic/reflective implementations by their very nature must always make assumptions that are tenuous at best, especially in cases of examining unknown properties defined for an unknown purpose by an unknown type.

Another way of thinking about the difference is that a non-reflective default implementation can only omit information that a developer must provide, but this is an easily solved case as that's precisely what unimplemented protocol requirements are for (to ask for more information/specific implementation details).

With reflective behaviour however the default implementation can (and will) go too far in what information it uses; while it might work fine some of the time, this is a fundamental risk, and there is no mechanism through which a protocol may force the developer to alter that behaviour. This makes the behaviour insidious in nature, which is why it is better that a developer specifically opt into it as it keeps the end developer in control, especially since these behaviours are predicated on the idea that they are supposed to be a convenience, but when the protocol is fucking around with properties it knows nothing about then only the end developer can determine if that is the case.

And all of this continues to be a side-issue to the fact that in the specific case of Equatable/Hashable, which thus far has gone ignored, is that bolting this on retroactively to an existing protocol hides bugs. The issue of reflective default implementations is less of a concern on very clearly and well defined new protocols, though I still prefer more, rather than less, control, but in the specific case of existing protocols this fucking about with behaviours is reckless and foolish in the extreme, yet no-one on the core teams seems willing or able to justify it, which only opens much wider concerns (how am I to have any faith in Swift's development if the core team can't or won't justify the creation of new bugs?).

···

On 9 Sep 2017, at 02:02, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Fri, Sep 8, 2017 at 4:00 PM, Itai Ferber via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Sep 8, 2017, at 12:46 AM, Haravikk via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 7 Sep 2017, at 22:02, Itai Ferber <iferber@apple.com <mailto:iferber@apple.com>> wrote:

protocol Fooable : Equatable { // Equatable is just a simple example
    var myFoo: Int { get }
}

extension Fooable {
    static func ==(_ lhs: Self, _ rhs: Self) -> Bool {
        return lhs.myFoo == rhs.myFoo
    }
}

struct X : Fooable {
    let myFoo: Int
    let myName: String
    // Whoops, forgot to give an implementation of ==
}

print(X(myFoo: 42, myName: "Alice") == X(myFoo: 42, myName: "Bob")) // true
This property is necessary, but not sufficient to provide a correct implementation. A default implementation might be able to assume something about the types that it defines, but it does not necessarily know enough.

Sorry but that's a bit of a contrived example; in this case the protocol should not implement the equality operator if more information may be required to define equality. It should only be implemented if the protocol is absolutely clear that .myFoo is the only part of a Fooable that can or should be compared as equatable, e.g- if a Fooable is a database record and .myFoo is a primary key, the data could differ but it would still be a reference to the same record.

To be clear, I'm not arguing that someone can't create a regular default implementation that also makes flawed assumptions, but that synthesised/reflective implementations by their very nature have to, as they cannot under every circumstance guarantee correctness when using parts of a concrete type that they know nothing about.

You can’t argue this both ways:
If you’re arguing this on principle, that in order for synthesized implementations to be correct, they must be able to — under every circumstance — guarantee correctness, then you have to apply the same reasoning to default protocol implementations. Given a default protocol implementation, it is possible to come up with a (no matter how contrived) case where the default implementation is wrong. Since you’re arguing this on principle, you cannot reject contrived examples.
If you are arguing this in practice, then you’re going to have to back up your argument with evidence that synthesized examples are more often wrong than default implementations. You can’t declare that synthesized implementations are by nature incorrect but allow default implementations to slide because in practice, many implementations are allowable. There’s a reason why synthesis passed code review and was accepted: in the majority of cases, synthesis was deemed to be beneficial, and would provide correct behavior. If you are willing to say that yes, sometimes default implementations are wrong but overall they’re correct, you’re going to have to provide hard evidence to back up the opposite case for synthesized implementations. You stated in a previous email that "A synthesised/reflective implementation however may return a result that is simply incorrect, because it is based on assumptions made by the protocol developer, with no input from the developer of the concrete type. In this case the developer must override it in to provide correct behaviour." — if you can back this up with evidence (say, taking a survey of a large number of model types and see if in the majority of cases synthesized implementation would be incorrect) to provide a compelling argument, then this is something that we should in that case reconsider.

Well put, and I agree with this position 100%. However, to play devil's advocate here, let me summarize what I think Haravikk is saying:

I think the "synthesized" part of this is a red herring, if I understand Haravikk's argument correctly. Instead, it is this:

(1) In principle, it is possible to have a default implementation for a protocol requirement that produces the correct result--though not necessarily in the most performant way--for all possible conforming types, where by conforming we mean that the type respects both the syntactic requirements (enforced by the compiler) and the semantic requirements (which may not necessarily be enforceable by the compiler) of the protocol in question.

(2) However, there exist *some* requirements that, by their very nature, cannot have default implementations which are guaranteed to produce the correct result for all conforming types. In Haravikk's view, no default implementations should be provided in these cases. (I don't necessarily subscribe to this view in absolute terms, but for the sake of argument let's grant this premise.)

(3) Equatable, Hashable, and Codable requirements are, by their very nature, such requirements that cannot have default implementations guaranteed to be correct for all conforming types. Therefore, they should not have a default implementation. It just so happens that a default implementation cannot currently be written in Swift itself and must be synthesized, but Haravikk's point is that even if they could be written in native Swift through a hypothetical reflection facility, they should not be, just as many other protocol requirements currently could have default implementations written in Swift but should not have them because they cannot be guaranteed to produce the correct result.

My response to this line of argumentation is as follows:

For any open protocol (i.e., a protocol for which the universe of possible conforming types cannot be enumerated a priori by the protocol designer) worthy of being a protocol by the Swift standard ("what useful thing can you do with such a protocol that you could not without?"), any sufficiently interesting requirement (i.e., one for which user ergonomics would measurably benefit from a default implementation) either cannot have a universally guaranteed correct implementation or has an implementation which is also going to be the most performant one (which can therefore be a non-overridable protocol extension method rather than an overridable protocol requirement with a default implementation).

protocol Fooable : Equatable { // Equatable is just a simple example
    var myFoo: Int { get }}
extension Fooable {
    static func ==(_ lhs: Self, _ rhs: Self) -> Bool {
        return lhs.myFoo == rhs.myFoo
    }}
struct X : Fooable {
    let myFoo: Int
    let myName: String
    // Whoops, forgot to give an implementation of ==}
print(X(myFoo: 42, myName: "Alice") == X(myFoo: 42, myName: "Bob")) // true

This property is *necessary*, but not *sufficient* to provide a correct
implementation. A default implementation might be able to *assume* something
about the types that it defines, but it does not necessarily know enough.

Sorry but that's a bit of a contrived example; in this case the protocol
should *not* implement the equality operator if more information may be
required to define equality. It should only be implemented if the protocol
is absolutely clear that .myFoo is the only part of a Fooable that can or
should be compared as equatable, e.g- if a Fooable is a database record and
.myFoo is a primary key, the data could differ but it would still be a
reference to the same record.

To be clear, I'm not arguing that someone can't create a regular default
implementation that also makes flawed assumptions, but that
synthesised/reflective implementations *by their very nature have to*,
as they cannot under every circumstance guarantee correctness when using
parts of a concrete type that they know nothing about.

You can’t argue this both ways:

   - If you’re arguing this on principle, that in order for synthesized
   implementations to be correct, they *must* be able to — *under every
   circumstance* — guarantee correctness, then you have to apply the
   same reasoning to default protocol implementations. Given a default
   protocol implementation, it is possible to come up with a (no matter how
   contrived) case where the default implementation is wrong. Since you’re
   arguing this *on principle*, you cannot reject contrived examples.
   - If you are arguing this *in practice*, then you’re going to have to
   back up your argument with evidence that synthesized examples are more
   often wrong than default implementations. You can’t declare that
   synthesized implementations are *by nature* incorrect but allow
   default implementations to slide because *in practice*, many
   implementations are allowable. There’s a reason why synthesis passed code
   review and was accepted: in the majority of cases, synthesis was deemed to
   be beneficial, and would provide correct behavior. If you are willing to
   say that yes, sometimes default implementations are wrong but overall
   they’re correct, you’re going to have to provide hard evidence to back up
   the opposite case for synthesized implementations. You stated in a previous
   email that "A synthesised/reflective implementation however may
   return a result that is simply incorrect, because it is based on
   assumptions made by the protocol developer, with no input from the
   developer of the concrete type. In this case the developer must override it
   in to provide *correct* behaviour." — if you can back this up with
   evidence (say, taking a survey of a large number of model types and see if
   in the majority of cases synthesized implementation would be incorrect) to
   provide a compelling argument, then this is something that we should in
   that case reconsider.

Well put, and I agree with this position 100%. However, to play devil's
advocate here, let me summarize what I think Haravikk is saying:

I think the "synthesized" part of this is a red herring, if I understand
Haravikk's argument correctly. Instead, it is this:

(1) In principle, it is possible to have a default implementation for a
protocol requirement that produces the correct result--though not
necessarily in the most performant way--for all possible conforming types,
where by conforming we mean that the type respects both the syntactic
requirements (enforced by the compiler) and the semantic requirements
(which may not necessarily be enforceable by the compiler) of the protocol
in question.

(2) However, there exist *some* requirements that, by their very nature,
cannot have default implementations which are guaranteed to produce the
correct result for all conforming types. In Haravikk's view, no default
implementations should be provided in these cases. (I don't necessarily
subscribe to this view in absolute terms, but for the sake of argument
let's grant this premise.)

(3) Equatable, Hashable, and Codable requirements are, by their very
nature, such requirements that cannot have default implementations
guaranteed to be correct for all conforming types. Therefore, they should
not have a default implementation. It just so happens that a default
implementation cannot currently be written in Swift itself and must be
synthesized, but Haravikk's point is that even if they could be written in
native Swift through a hypothetical reflection facility, they should not
be, just as many other protocol requirements currently could have default
implementations written in Swift but should not have them because they
cannot be guaranteed to produce the correct result.

My response to this line of argumentation is as follows:

For any open protocol (i.e., a protocol for which the universe of possible
conforming types cannot be enumerated a priori by the protocol designer)
worthy of being a protocol by the Swift standard ("what useful thing can
you do with such a protocol that you could not without?"), any sufficiently
interesting requirement (i.e., one for which user ergonomics would
measurably benefit from a default implementation) either cannot have a
universally guaranteed correct implementation or has an implementation
which is also going to be the most performant one (which can therefore be a
non-overridable protocol extension method rather than an overridable
protocol requirement with a default implementation).

You're close, but still missing key points:

   1. I am not arguing that features like these should *not* be provided,
   but that they should *not* be provided implicitly, and that the
   developer should actually be allowed to request them. That is exactly what
   this proposal is about, yet no matter what I say everyone seems to be
   treating me like I'm against these features entirely; *I am not*.

You are entirely against Equatable having a default implementation for ==.
This is unequivocally stated. Others favor such a default implementation
and feel that in the absence of a way to spell this in Swift itself, it
should be magic for the time being. For the purposes of this argument it
really is not pertinent that you are not also against something else;
you're asking us to discuss why you are against a particular thing that
others are for.

   1.
   2. A non-reflective default implementation can *only* operate on the
   basis of what the protocol itself defines; this means that it can always be
   correct within the context of only what the protocol declares. At worst, a
   "pure" default implementation can only be overly cautious, but strictly
   speaking this does not make its behaviour incorrect, though, if it is a
   likely case it suggests that a default implementation is not a good idea
   (as it is better to ensure the developer provides any further information).
   3. Synthetic/reflective implementations *by their very nature* must
   *always* make assumptions that are tenuous at best, especially in
   cases of examining unknown properties defined for an unknown purpose by an
   unknown type.

Another way of thinking about the difference is that a non-reflective
default implementation can only *omit* information that a developer must
provide, but this is an easily solved case as that's precisely what
unimplemented protocol requirements are for (to ask for more
information/specific implementation details).

With reflective behaviour however the default implementation can (and
will) go too far in what information it uses; while it might work fine some
of the time, this is a fundamental risk, and there is no mechanism through
which a protocol may force the developer to alter that behaviour. This
makes the behaviour insidious in nature, which is why it is better that a
developer specifically opt into it as it keeps the end developer in
control, especially since these behaviours are predicated on the idea that
they are supposed to be a convenience, but when the protocol is fucking
around with properties it knows *nothing* about then *only* the end
developer can determine if that is the case.

As repeatedly answered by others, nothing here is specific to synthesized
default implementations, as more powerful reflection will gradually allow
them to be non-synthesized.

As pointed out very cogently by Itai, you assert but offer no evidence,
either in principle or empirically, that going too far by reflection is
worse than going not far enough without reflection in terms of likelihood
of a default implementation being inappropriate for conforming types.

Therefore, your argument reduces to one about which default implementations
generally ought or ought not to be provided--that is, that they ought to be
provided only when their correctness can be guaranteed for all (rather than
almost all) possible conforming types. To which point I sketched a rebuttal
above.

And all of this continues to be a side-issue to the fact that in the

specific case of Equatable/Hashable, which thus far has gone ignored, is
that bolting this on retroactively to an existing protocol *hides bugs*.
The issue of reflective default implementations is less of a concern on
very clearly and well defined *new* protocols, though I still prefer
more, rather than less, control, but in the specific case of *existing* protocols
this fucking about with behaviours is reckless and foolish in the extreme,
yet no-one on the core teams seems willing or able to justify it, which
only opens much wider concerns (how am I to have any faith in Swift's
development if the core team can't or won't justify the creation of new
bugs?).

This has emphatically not gone ignored, as I have myself responded to this
point in an earlier thread in which you commented, as well as many others.
Crucially, no existing conforming type changes its behavior, as they have
all had to implement these requirements themselves. And as I said to you
already, the addition of a synthesized default implementation no more
"hides bugs" going forward than the addition of a non-synthesized default
implementation to an existing protocol, and we do that with some frequency
without even Swift Evolution review.

Put another way, what the proposal about synthesizing implementations for
Equatable and Hashable was about can be thought of in two parts: (a) should
there be default implementations; and (b) given that it is impossible to
write these in Swift, should we use magic? Now, as I said above, adding
default implementations isn't (afaik) even considered an API change that
requires review on this list. Really, what people were debating was (b),
whether it is worth it to implement compiler-supported magic to make these
possible. Your disagreement has to do with (a) and not (b).

···

On Sat, Sep 9, 2017 at 02:47 Haravikk via swift-evolution < swift-evolution@swift.org> wrote:

On 9 Sep 2017, at 02:02, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Fri, Sep 8, 2017 at 4:00 PM, Itai Ferber via swift-evolution < > swift-evolution@swift.org> wrote:

On Sep 8, 2017, at 12:46 AM, Haravikk via swift-evolution < >> swift-evolution@swift.org> wrote:
On 7 Sep 2017, at 22:02, Itai Ferber <iferber@apple.com> wrote:

_______________________________________________

swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

protocol Fooable : Equatable { // Equatable is just a simple example
    var myFoo: Int { get }
}

extension Fooable {
    static func ==(_ lhs: Self, _ rhs: Self) -> Bool {
        return lhs.myFoo == rhs.myFoo
    }
}

struct X : Fooable {
    let myFoo: Int
    let myName: String
    // Whoops, forgot to give an implementation of ==
}

print(X(myFoo: 42, myName: "Alice") == X(myFoo: 42, myName: "Bob")) // true
This property is necessary, but not sufficient to provide a correct implementation. A default implementation might be able to assume something about the types that it defines, but it does not necessarily know enough.

Sorry but that's a bit of a contrived example; in this case the protocol should not implement the equality operator if more information may be required to define equality. It should only be implemented if the protocol is absolutely clear that .myFoo is the only part of a Fooable that can or should be compared as equatable, e.g- if a Fooable is a database record and .myFoo is a primary key, the data could differ but it would still be a reference to the same record.

To be clear, I'm not arguing that someone can't create a regular default implementation that also makes flawed assumptions, but that synthesised/reflective implementations by their very nature have to, as they cannot under every circumstance guarantee correctness when using parts of a concrete type that they know nothing about.

You can’t argue this both ways:
If you’re arguing this on principle, that in order for synthesized implementations to be correct, they must be able to — under every circumstance — guarantee correctness, then you have to apply the same reasoning to default protocol implementations. Given a default protocol implementation, it is possible to come up with a (no matter how contrived) case where the default implementation is wrong. Since you’re arguing this on principle, you cannot reject contrived examples.
If you are arguing this in practice, then you’re going to have to back up your argument with evidence that synthesized examples are more often wrong than default implementations. You can’t declare that synthesized implementations are by nature incorrect but allow default implementations to slide because in practice, many implementations are allowable. There’s a reason why synthesis passed code review and was accepted: in the majority of cases, synthesis was deemed to be beneficial, and would provide correct behavior. If you are willing to say that yes, sometimes default implementations are wrong but overall they’re correct, you’re going to have to provide hard evidence to back up the opposite case for synthesized implementations. You stated in a previous email that "A synthesised/reflective implementation however may return a result that is simply incorrect, because it is based on assumptions made by the protocol developer, with no input from the developer of the concrete type. In this case the developer must override it in to provide correct behaviour." — if you can back this up with evidence (say, taking a survey of a large number of model types and see if in the majority of cases synthesized implementation would be incorrect) to provide a compelling argument, then this is something that we should in that case reconsider.

Well put, and I agree with this position 100%. However, to play devil's advocate here, let me summarize what I think Haravikk is saying:

I think the "synthesized" part of this is a red herring, if I understand Haravikk's argument correctly. Instead, it is this:

(1) In principle, it is possible to have a default implementation for a protocol requirement that produces the correct result--though not necessarily in the most performant way--for all possible conforming types, where by conforming we mean that the type respects both the syntactic requirements (enforced by the compiler) and the semantic requirements (which may not necessarily be enforceable by the compiler) of the protocol in question.

(2) However, there exist *some* requirements that, by their very nature, cannot have default implementations which are guaranteed to produce the correct result for all conforming types. In Haravikk's view, no default implementations should be provided in these cases. (I don't necessarily subscribe to this view in absolute terms, but for the sake of argument let's grant this premise.)

(3) Equatable, Hashable, and Codable requirements are, by their very nature, such requirements that cannot have default implementations guaranteed to be correct for all conforming types. Therefore, they should not have a default implementation. It just so happens that a default implementation cannot currently be written in Swift itself and must be synthesized, but Haravikk's point is that even if they could be written in native Swift through a hypothetical reflection facility, they should not be, just as many other protocol requirements currently could have default implementations written in Swift but should not have them because they cannot be guaranteed to produce the correct result.

My response to this line of argumentation is as follows:

For any open protocol (i.e., a protocol for which the universe of possible conforming types cannot be enumerated a priori by the protocol designer) worthy of being a protocol by the Swift standard ("what useful thing can you do with such a protocol that you could not without?"), any sufficiently interesting requirement (i.e., one for which user ergonomics would measurably benefit from a default implementation) either cannot have a universally guaranteed correct implementation or has an implementation which is also going to be the most performant one (which can therefore be a non-overridable protocol extension method rather than an overridable protocol requirement with a default implementation).

You're close, but still missing key points:

I am not arguing that features like these should not be provided, but that they should not be provided implicitly, and that the developer should actually be allowed to request them. That is exactly what this proposal is about, yet no matter what I say everyone seems to be treating me like I'm against these features entirely; I am not.

You are entirely against Equatable having a default implementation for ==. This is unequivocally stated. Others favor such a default implementation and feel that in the absence of a way to spell this in Swift itself, it should be magic for the time being. For the purposes of this argument it really is not pertinent that you are not also against something else; you're asking us to discuss why you are against a particular thing that others are for.

FFS, how much clearer can I make this? I AM NOT AGAINST THE FEATURE.

What I am against is the way in which it is being provided implicitly rather than explicitly, in particular as a retroactive change to existing protocols in a way that introduces potential for bugs that are currently impossible, but also in general.

As repeatedly answered by others, nothing here is specific to synthesized default implementations, as more powerful reflection will gradually allow them to be non-synthesised.

And as repeatedly stated by me; I am not treating synthesised vs. run-time reflection any differently, I specifically included both in the original proposal.

As pointed out very cogently by Itai, you assert but offer no evidence, either in principle or empirically, that going too far by reflection is worse than going not far enough without reflection in terms of likelihood of a default implementation being inappropriate for conforming types.

As I have also repeatedly pointed out it is not an issue of "not going far enough" vs. "going too far"; if a default implementation lacks information then it should not be provided, doing so regardless is a flaw in the protocol design and not something that this proposal attempts to address (as such a thing is likely impossible).

Reflective implementations necessarily go too far, because they literally know nothing about the concrete type with any certainty, except for the properties that are defined in the protocol (which do not require reflection or synthesis in the first place).

And precisely what kind of "evidence" am I expected to give? This is a set of features that do not exist yet, I am trying to argue in favour of an explicit end-developer centric opt-in rather than an implicit protocol designer centric one. Yet no-one seems interested in the merits of allowing developers to choose what they want, rather than having implicit behaviours appear potentially unexpectedly.

Therefore, your argument reduces to one about which default implementations generally ought or ought not to be provided--that is, that they ought to be provided only when their correctness can be guaranteed for all (rather than almost all) possible conforming types. To which point I sketched a rebuttal above.

If a protocol defines something, and creates a default implementation based only upon those definitions then it must by its very nature be correct. A concrete type may later decided to go further, but that is a feature of the concrete type, not a failure of the protocol itself which can function correctly within the context it created. You want to talk evidence, yet there has been no example given that proves otherwise; thus far only Itai has attempted to do so, but I have already pointed out the flaws with that example.

The simple fact is that a default implementation may either be flawed or not within the context of the protocol itself; but a reflective or synthetic implementation by its very nature goes beyond what the protocol defines and so is automatically flawed because as it does not rely on the end-developer to confirm correctness, not when provided implicitly at least.

And all of this continues to be a side-issue to the fact that in the specific case of Equatable/Hashable, which thus far has gone ignored, is that bolting this on retroactively to an existing protocol hides bugs. The issue of reflective default implementations is less of a concern on very clearly and well defined new protocols, though I still prefer more, rather than less, control, but in the specific case of existing protocols this fucking about with behaviours is reckless and foolish in the extreme, yet no-one on the core teams seems willing or able to justify it, which only opens much wider concerns (how am I to have any faith in Swift's development if the core team can't or won't justify the creation of new bugs?).

This has emphatically not gone ignored, as I have myself responded to this point in an earlier thread in which you commented, as well as many others. Crucially, no existing conforming type changes its behavior, as they have all had to implement these requirements themselves. And as I said to you already, the addition of a synthesized default implementation no more "hides bugs" going forward than the addition of a non-synthesized default implementation to an existing protocol, and we do that with some frequency without even Swift Evolution review.

Feel free to a supply a non-synthesised default implementation for Equatable without the use of reflection. Go-on, I'll wait.
You insist on suggesting these are the same thing, yet if you can't provide one then clearly they are not.

Put another way, what the proposal about synthesizing implementations for Equatable and Hashable was about can be thought of in two parts: (a) should there be default implementations; and (b) given that it is impossible to write these in Swift, should we use magic? Now, as I said above, adding default implementations isn't (afaik) even considered an API change that requires review on this list. Really, what people were debating was (b), whether it is worth it to implement compiler-supported magic to make these possible. Your disagreement has to do with (a) and not (b).

Wrong. The use of magic in this case produces something else entirely; that's the whole point. It is not the same, otherwise it wouldn't be needed at all. It doesn't matter if it's compiler magic, some external script or a native macro, ultimately they are all doing something with a concrete type that is currently not possible.

And once again; I am not arguing against a default implementation that cuts boilerplate, I am arguing against it being implicit. What I want is to be the one asking for it, because it is not reasonable to assume that just throwing it in there is always going to be fine, because it quite simply is not.

···

On 9 Sep 2017, at 09:33, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Sat, Sep 9, 2017 at 02:47 Haravikk via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 9 Sep 2017, at 02:02, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:
On Fri, Sep 8, 2017 at 4:00 PM, Itai Ferber via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Sep 8, 2017, at 12:46 AM, Haravikk via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 7 Sep 2017, at 22:02, Itai Ferber <iferber@apple.com <mailto:iferber@apple.com>> wrote:

Don't get me started on Swift's handling of equality and arrays with NaN.
_customIndexOfEquatable, if I'm not mistaken, is a part of that whole
tangle of performance optimizations which gleefully refuse to acknowledge
Equatable's semantic peephole for some values of a type being unordered
with respect to everything else. In a world where this trade-off between
performance and correctness had not been taken, I don't imagine that it
would be possible to make the protocol extension method 'index(of:)' any
more performant than 'index(where: { $0 == $1 })'.

···

On Sat, Sep 9, 2017 at 07:51 Brent Royal-Gordon <brent@architechies.com> wrote:

On Sep 8, 2017, at 6:03 PM, Xiaodi Wu via swift-evolution < > swift-evolution@swift.org> wrote:

For any open protocol (i.e., a protocol for which the universe of possible
conforming types cannot be enumerated a priori by the protocol designer)
worthy of being a protocol by the Swift standard ("what useful thing can
you do with such a protocol that you could not without?"), any sufficiently
interesting requirement (i.e., one for which user ergonomics would
measurably benefit from a default implementation) either cannot have a
universally guaranteed correct implementation or has an implementation
which is also going to be the most performant one (which can therefore be a
non-overridable protocol extension method rather than an overridable
protocol requirement with a default implementation).

Counter-example: `index(of:)`, or rather, the underscored requirement
underlying `index(of:)`. The "loop over all indices and return the first
whose element matches" default implementation is universally guaranteed to
be correct, but a collection like `Set` or `SortedArray` can provide an
implementation which is more performant than the default.

Hello Haravikk,

I'lm worried that you fail at preventing a real problem. May I suggest a change in your strategy?

Sometimes, sample code greatly helps turning subtle ideas into blatant evidence. After all, subtleties are all about corner cases, and corner cases are the blind spots of imagination. What about giving that little something that would help your readers grasp your arguments?

I don't quite know what example you will provide, but I could suggest the exhibition of a practical problem with Equatable synthesis. We'll know better if the problem can arise in the Standard lib, in third-party libraries, at application level, or at several scales at the same time. It would also be nice to see your solution to the problem, that is to say an alternative that still provides code synthesis for developers that want to opt in the feature, but avoids the caveat of the initial example. I hope this would greatly help the discussion move forward.

Last general comment about the topic: if Haravikk is right, and that code synthesis should indeed be explicit, then that wouldn't be such a shame.

My two cents,
Gwendal Roué

···

Le 9 sept. 2017 à 13:41, Haravikk via swift-evolution <swift-evolution@swift.org> a écrit :

On 9 Sep 2017, at 09:33, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:

On Sat, Sep 9, 2017 at 02:47 Haravikk via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 9 Sep 2017, at 02:02, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:

On Fri, Sep 8, 2017 at 4:00 PM, Itai Ferber via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Sep 8, 2017, at 12:46 AM, Haravikk via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 7 Sep 2017, at 22:02, Itai Ferber <iferber@apple.com <mailto:iferber@apple.com>> wrote:

protocol Fooable : Equatable { // Equatable is just a simple example
    var myFoo: Int { get }
}

extension Fooable {
    static func ==(_ lhs: Self, _ rhs: Self) -> Bool {
        return lhs.myFoo == rhs.myFoo
    }
}

struct X : Fooable {
    let myFoo: Int
    let myName: String
    // Whoops, forgot to give an implementation of ==
}

print(X(myFoo: 42, myName: "Alice") == X(myFoo: 42, myName: "Bob")) // true
This property is necessary, but not sufficient to provide a correct implementation. A default implementation might be able to assume something about the types that it defines, but it does not necessarily know enough.

Sorry but that's a bit of a contrived example; in this case the protocol should not implement the equality operator if more information may be required to define equality. It should only be implemented if the protocol is absolutely clear that .myFoo is the only part of a Fooable that can or should be compared as equatable, e.g- if a Fooable is a database record and .myFoo is a primary key, the data could differ but it would still be a reference to the same record.

To be clear, I'm not arguing that someone can't create a regular default implementation that also makes flawed assumptions, but that synthesised/reflective implementations by their very nature have to, as they cannot under every circumstance guarantee correctness when using parts of a concrete type that they know nothing about.

You can’t argue this both ways:
If you’re arguing this on principle, that in order for synthesized implementations to be correct, they must be able to — under every circumstance — guarantee correctness, then you have to apply the same reasoning to default protocol implementations. Given a default protocol implementation, it is possible to come up with a (no matter how contrived) case where the default implementation is wrong. Since you’re arguing this on principle, you cannot reject contrived examples.
If you are arguing this in practice, then you’re going to have to back up your argument with evidence that synthesized examples are more often wrong than default implementations. You can’t declare that synthesized implementations are by nature incorrect but allow default implementations to slide because in practice, many implementations are allowable. There’s a reason why synthesis passed code review and was accepted: in the majority of cases, synthesis was deemed to be beneficial, and would provide correct behavior. If you are willing to say that yes, sometimes default implementations are wrong but overall they’re correct, you’re going to have to provide hard evidence to back up the opposite case for synthesized implementations. You stated in a previous email that "A synthesised/reflective implementation however may return a result that is simply incorrect, because it is based on assumptions made by the protocol developer, with no input from the developer of the concrete type. In this case the developer must override it in to provide correct behaviour." — if you can back this up with evidence (say, taking a survey of a large number of model types and see if in the majority of cases synthesized implementation would be incorrect) to provide a compelling argument, then this is something that we should in that case reconsider.

Well put, and I agree with this position 100%. However, to play devil's advocate here, let me summarize what I think Haravikk is saying:

I think the "synthesized" part of this is a red herring, if I understand Haravikk's argument correctly. Instead, it is this:

(1) In principle, it is possible to have a default implementation for a protocol requirement that produces the correct result--though not necessarily in the most performant way--for all possible conforming types, where by conforming we mean that the type respects both the syntactic requirements (enforced by the compiler) and the semantic requirements (which may not necessarily be enforceable by the compiler) of the protocol in question.

(2) However, there exist *some* requirements that, by their very nature, cannot have default implementations which are guaranteed to produce the correct result for all conforming types. In Haravikk's view, no default implementations should be provided in these cases. (I don't necessarily subscribe to this view in absolute terms, but for the sake of argument let's grant this premise.)

(3) Equatable, Hashable, and Codable requirements are, by their very nature, such requirements that cannot have default implementations guaranteed to be correct for all conforming types. Therefore, they should not have a default implementation. It just so happens that a default implementation cannot currently be written in Swift itself and must be synthesized, but Haravikk's point is that even if they could be written in native Swift through a hypothetical reflection facility, they should not be, just as many other protocol requirements currently could have default implementations written in Swift but should not have them because they cannot be guaranteed to produce the correct result.

My response to this line of argumentation is as follows:

For any open protocol (i.e., a protocol for which the universe of possible conforming types cannot be enumerated a priori by the protocol designer) worthy of being a protocol by the Swift standard ("what useful thing can you do with such a protocol that you could not without?"), any sufficiently interesting requirement (i.e., one for which user ergonomics would measurably benefit from a default implementation) either cannot have a universally guaranteed correct implementation or has an implementation which is also going to be the most performant one (which can therefore be a non-overridable protocol extension method rather than an overridable protocol requirement with a default implementation).

You're close, but still missing key points:

I am not arguing that features like these should not be provided, but that they should not be provided implicitly, and that the developer should actually be allowed to request them. That is exactly what this proposal is about, yet no matter what I say everyone seems to be treating me like I'm against these features entirely; I am not.

You are entirely against Equatable having a default implementation for ==. This is unequivocally stated. Others favor such a default implementation and feel that in the absence of a way to spell this in Swift itself, it should be magic for the time being. For the purposes of this argument it really is not pertinent that you are not also against something else; you're asking us to discuss why you are against a particular thing that others are for.

FFS, how much clearer can I make this? I AM NOT AGAINST THE FEATURE.

What I am against is the way in which it is being provided implicitly rather than explicitly, in particular as a retroactive change to existing protocols in a way that introduces potential for bugs that are currently impossible, but also in general.

As repeatedly answered by others, nothing here is specific to synthesized default implementations, as more powerful reflection will gradually allow them to be non-synthesised.

And as repeatedly stated by me; I am not treating synthesised vs. run-time reflection any differently, I specifically included both in the original proposal.

As pointed out very cogently by Itai, you assert but offer no evidence, either in principle or empirically, that going too far by reflection is worse than going not far enough without reflection in terms of likelihood of a default implementation being inappropriate for conforming types.

As I have also repeatedly pointed out it is not an issue of "not going far enough" vs. "going too far"; if a default implementation lacks information then it should not be provided, doing so regardless is a flaw in the protocol design and not something that this proposal attempts to address (as such a thing is likely impossible).

Reflective implementations necessarily go too far, because they literally know nothing about the concrete type with any certainty, except for the properties that are defined in the protocol (which do not require reflection or synthesis in the first place).

And precisely what kind of "evidence" am I expected to give? This is a set of features that do not exist yet, I am trying to argue in favour of an explicit end-developer centric opt-in rather than an implicit protocol designer centric one. Yet no-one seems interested in the merits of allowing developers to choose what they want, rather than having implicit behaviours appear potentially unexpectedly.

Therefore, your argument reduces to one about which default implementations generally ought or ought not to be provided--that is, that they ought to be provided only when their correctness can be guaranteed for all (rather than almost all) possible conforming types. To which point I sketched a rebuttal above.

If a protocol defines something, and creates a default implementation based only upon those definitions then it must by its very nature be correct. A concrete type may later decided to go further, but that is a feature of the concrete type, not a failure of the protocol itself which can function correctly within the context it created. You want to talk evidence, yet there has been no example given that proves otherwise; thus far only Itai has attempted to do so, but I have already pointed out the flaws with that example.

The simple fact is that a default implementation may either be flawed or not within the context of the protocol itself; but a reflective or synthetic implementation by its very nature goes beyond what the protocol defines and so is automatically flawed because as it does not rely on the end-developer to confirm correctness, not when provided implicitly at least.

And all of this continues to be a side-issue to the fact that in the specific case of Equatable/Hashable, which thus far has gone ignored, is that bolting this on retroactively to an existing protocol hides bugs. The issue of reflective default implementations is less of a concern on very clearly and well defined new protocols, though I still prefer more, rather than less, control, but in the specific case of existing protocols this fucking about with behaviours is reckless and foolish in the extreme, yet no-one on the core teams seems willing or able to justify it, which only opens much wider concerns (how am I to have any faith in Swift's development if the core team can't or won't justify the creation of new bugs?).

This has emphatically not gone ignored, as I have myself responded to this point in an earlier thread in which you commented, as well as many others. Crucially, no existing conforming type changes its behavior, as they have all had to implement these requirements themselves. And as I said to you already, the addition of a synthesized default implementation no more "hides bugs" going forward than the addition of a non-synthesized default implementation to an existing protocol, and we do that with some frequency without even Swift Evolution review.

Feel free to a supply a non-synthesised default implementation for Equatable without the use of reflection. Go-on, I'll wait.
You insist on suggesting these are the same thing, yet if you can't provide one then clearly they are not.

Put another way, what the proposal about synthesizing implementations for Equatable and Hashable was about can be thought of in two parts: (a) should there be default implementations; and (b) given that it is impossible to write these in Swift, should we use magic? Now, as I said above, adding default implementations isn't (afaik) even considered an API change that requires review on this list. Really, what people were debating was (b), whether it is worth it to implement compiler-supported magic to make these possible. Your disagreement has to do with (a) and not (b).

Wrong. The use of magic in this case produces something else entirely; that's the whole point. It is not the same, otherwise it wouldn't be needed at all. It doesn't matter if it's compiler magic, some external script or a native macro, ultimately they are all doing something with a concrete type that is currently not possible.

And once again; I am not arguing against a default implementation that cuts boilerplate, I am arguing against it being implicit. What I want is to be the one asking for it, because it is not reasonable to assume that just throwing it in there is always going to be fine, because it quite simply is not.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

protocol Fooable : Equatable { // Equatable is just a simple example
    var myFoo: Int { get }}
extension Fooable {
    static func ==(_ lhs: Self, _ rhs: Self) -> Bool {
        return lhs.myFoo == rhs.myFoo
    }}
struct X : Fooable {
    let myFoo: Int
    let myName: String
    // Whoops, forgot to give an implementation of ==}
print(X(myFoo: 42, myName: "Alice") == X(myFoo: 42, myName: "Bob")) // true

This property is *necessary*, but not *sufficient* to provide a correct
implementation. A default implementation might be able to *assume* something
about the types that it defines, but it does not necessarily know enough.

Sorry but that's a bit of a contrived example; in this case the protocol
should *not* implement the equality operator if more information may be
required to define equality. It should only be implemented if the protocol
is absolutely clear that .myFoo is the only part of a Fooable that can or
should be compared as equatable, e.g- if a Fooable is a database record and
.myFoo is a primary key, the data could differ but it would still be a
reference to the same record.

To be clear, I'm not arguing that someone can't create a regular default
implementation that also makes flawed assumptions, but that
synthesised/reflective implementations *by their very nature have to*,
as they cannot under every circumstance guarantee correctness when using
parts of a concrete type that they know nothing about.

You can’t argue this both ways:

   - If you’re arguing this on principle, that in order for synthesized
   implementations to be correct, they *must* be able to — *under every
   circumstance* — guarantee correctness, then you have to apply the
   same reasoning to default protocol implementations. Given a default
   protocol implementation, it is possible to come up with a (no matter how
   contrived) case where the default implementation is wrong. Since you’re
   arguing this *on principle*, you cannot reject contrived examples.
   - If you are arguing this *in practice*, then you’re going to have
   to back up your argument with evidence that synthesized examples are more
   often wrong than default implementations. You can’t declare that
   synthesized implementations are *by nature* incorrect but allow
   default implementations to slide because *in practice*, many
   implementations are allowable. There’s a reason why synthesis passed code
   review and was accepted: in the majority of cases, synthesis was deemed to
   be beneficial, and would provide correct behavior. If you are willing to
   say that yes, sometimes default implementations are wrong but overall
   they’re correct, you’re going to have to provide hard evidence to back up
   the opposite case for synthesized implementations. You stated in a previous
   email that "A synthesised/reflective implementation however may
   return a result that is simply incorrect, because it is based on
   assumptions made by the protocol developer, with no input from the
   developer of the concrete type. In this case the developer must override it
   in to provide *correct* behaviour." — if you can back this up with
   evidence (say, taking a survey of a large number of model types and see if
   in the majority of cases synthesized implementation would be incorrect) to
   provide a compelling argument, then this is something that we should in
   that case reconsider.

Well put, and I agree with this position 100%. However, to play devil's
advocate here, let me summarize what I think Haravikk is saying:

I think the "synthesized" part of this is a red herring, if I understand
Haravikk's argument correctly. Instead, it is this:

(1) In principle, it is possible to have a default implementation for a
protocol requirement that produces the correct result--though not
necessarily in the most performant way--for all possible conforming types,
where by conforming we mean that the type respects both the syntactic
requirements (enforced by the compiler) and the semantic requirements
(which may not necessarily be enforceable by the compiler) of the protocol
in question.

(2) However, there exist *some* requirements that, by their very nature,
cannot have default implementations which are guaranteed to produce the
correct result for all conforming types. In Haravikk's view, no default
implementations should be provided in these cases. (I don't necessarily
subscribe to this view in absolute terms, but for the sake of argument
let's grant this premise.)

(3) Equatable, Hashable, and Codable requirements are, by their very
nature, such requirements that cannot have default implementations
guaranteed to be correct for all conforming types. Therefore, they should
not have a default implementation. It just so happens that a default
implementation cannot currently be written in Swift itself and must be
synthesized, but Haravikk's point is that even if they could be written in
native Swift through a hypothetical reflection facility, they should not
be, just as many other protocol requirements currently could have default
implementations written in Swift but should not have them because they
cannot be guaranteed to produce the correct result.

My response to this line of argumentation is as follows:

For any open protocol (i.e., a protocol for which the universe of
possible conforming types cannot be enumerated a priori by the protocol
designer) worthy of being a protocol by the Swift standard ("what useful
thing can you do with such a protocol that you could not without?"), any
sufficiently interesting requirement (i.e., one for which user ergonomics
would measurably benefit from a default implementation) either cannot have
a universally guaranteed correct implementation or has an implementation
which is also going to be the most performant one (which can therefore be a
non-overridable protocol extension method rather than an overridable
protocol requirement with a default implementation).

You're close, but still missing key points:

   1. I am not arguing that features like these should *not* be
   provided, but that they should *not* be provided implicitly, and that
   the developer should actually be allowed to request them. That is exactly
   what this proposal is about, yet no matter what I say everyone seems to be
   treating me like I'm against these features entirely; *I am not*.

You are entirely against Equatable having a default implementation for ==.
This is unequivocally stated. Others favor such a default implementation
and feel that in the absence of a way to spell this in Swift itself, it
should be magic for the time being. For the purposes of this argument it
really is not pertinent that you are not also against something else;
you're asking us to discuss why you are against a particular thing that
others are for.

FFS, how much clearer can I make this? *I AM NOT AGAINST THE FEATURE.*

What I am against is the way in which it is being provided implicitly
rather than explicitly, in particular as a retroactive change to existing
protocols in a way that introduces potential for bugs that are currently
impossible, but also in general.

You are against a default implementation for ==, i.e. an implementation
that is provided for you if you conform a type to the protocol and do
nothing else ("implicitly rather than explicitly"), and you are against the
default implementation being on the existing protocol Equatable
("retroactive change"). So, to summarize, what you are against is precisely
a default implementation for the == requirement on Equatable.

This is the topic of discussion here; I am attempting to convince you that
you should be for rather than against these things.

As repeatedly answered by others, nothing here is specific to synthesized
default implementations, as more powerful reflection will gradually allow
them to be non-synthesised.

And as repeatedly stated by me; I am not treating synthesised vs. run-time
reflection any differently, I specifically included both in the original
proposal.

As pointed out very cogently by Itai, you assert but offer no evidence,
either in principle or empirically, that going too far by reflection is
worse than going not far enough without reflection in terms of likelihood
of a default implementation being inappropriate for conforming types.

As I have also repeatedly pointed out it is not an issue of "not going far
enough" vs. "going too far"; if a default implementation lacks information
then it should not be provided, doing so regardless is a flaw in the
protocol design and not something that this proposal attempts to address
(as such a thing is likely impossible).

Right, one must consider the semantics of the specific protocol requirement
and ask whether a reasonable default can be provided for it.

Reflective implementations *necessarily* go too far, because they literally

know *nothing* about the concrete type with any certainty, except for the
properties that are defined in the protocol (which do not require
reflection or synthesis in the first place).

I am confused why you are trying to argue in general terms about the
universe of all possible default implementations that use reflection. This
is necessarily a more difficult argument to make, and if it is to be
convincing for all default implementations it must also be convincing for
the two specific protocol requirements we are talking about here. Start
small:

We have agreed, as a community, that there is a reasonable default
implementation for Equatable.== when certain conditions are met (for value
types only at the moment, I believe). Namely, given two values of a type
that has only Equatable stored properties, those values are equal if their
stored properties are all equal. The author of a new value type who wishes
to make her type Equatable but chooses not to implement a custom == then
benefits from this default when all stored properties are Equatable.

And precisely what kind of "evidence" am I expected to give? This is a set

of features that *do not exist yet*, I am trying to argue in favour of an
explicit end-developer centric opt-in rather than an implicit protocol
designer centric one. Yet no-one seems interested in the merits of allowing
developers to choose what they want, rather than having implicit behaviours
appear potentially unexpectedly.

Both options were examined for Codable and for Equatable/Hashable. The
community and core team decided to prefer the current design. At this
point, new insights that arise which could not be anticipated at the time
of review could prompt revision. However, so far, you have presented
arguments already considered during review.

Therefore, your argument reduces to one about which default implementations

generally ought or ought not to be provided--that is, that they ought to be
provided only when their correctness can be guaranteed for all (rather than
almost all) possible conforming types. To which point I sketched a rebuttal
above.

If a protocol defines something, and creates a default implementation
based only upon those definitions then it must by its very nature be
correct. A concrete type may later decided to go further, but that is a
feature of the concrete type, not a failure of the protocol itself which
can function correctly within the context it created. You want to talk
evidence, yet there has been no example given that proves otherwise; thus
far only Itai has attempted to do so, but I have already pointed out the
flaws with that example.

The simple fact is that a default implementation may either be flawed or
not within the context of the protocol itself; but a reflective or
synthetic implementation by its very nature goes beyond what the protocol
defines and so is automatically flawed because as it does not rely on the
end-developer to confirm correctness, not when provided implicitly at least.

Again, if it applies generally, it must apply specifically. What is
"automatically flawed" about the very reasonable synthesized default
implementation of ==?

And all of this continues to be a side-issue to the fact that in the

specific case of Equatable/Hashable, which thus far has gone ignored, is
that bolting this on retroactively to an existing protocol *hides bugs*.
The issue of reflective default implementations is less of a concern on
very clearly and well defined *new* protocols, though I still prefer
more, rather than less, control, but in the specific case of *existing* protocols
this fucking about with behaviours is reckless and foolish in the extreme,
yet no-one on the core teams seems willing or able to justify it, which
only opens much wider concerns (how am I to have any faith in Swift's
development if the core team can't or won't justify the creation of new
bugs?).

This has emphatically not gone ignored, as I have myself responded to this
point in an earlier thread in which you commented, as well as many others.
Crucially, no existing conforming type changes its behavior, as they have
all had to implement these requirements themselves. And as I said to you
already, the addition of a synthesized default implementation no more
"hides bugs" going forward than the addition of a non-synthesized default
implementation to an existing protocol, and we do that with some frequency
without even Swift Evolution review.

Feel free to a supply a non-synthesised default implementation for
Equatable without the use of reflection. Go-on, I'll wait.
You insist on suggesting these are the same thing, yet if you can't
provide one then clearly they are not.

That is not the argument. The argument is that they are indistinguishable
in the sense that the author of a type who intends to supply a custom
implementation but neglects to do so will have a default implementation
supplied for them. It is plainly true that this is no more or less likely
to happen simply because the default implementation is synthesized.

Put another way, what the proposal about synthesizing implementations for

Equatable and Hashable was about can be thought of in two parts: (a) should
there be default implementations; and (b) given that it is impossible to
write these in Swift, should we use magic? Now, as I said above, adding
default implementations isn't (afaik) even considered an API change that
requires review on this list. Really, what people were debating was (b),
whether it is worth it to implement compiler-supported magic to make these
possible. Your disagreement has to do with (a) and not (b).

Wrong. The use of magic in this case produces something else entirely;
that's the whole point. It is *not the same*, otherwise it wouldn't be
needed at all. It doesn't matter if it's compiler magic, some external
script or a native macro, ultimately they are all doing something with a
concrete type that is currently not possible.

And once again; *I am not arguing against a default implementation that
cuts boilerplate*, I am arguing against it being implicit. What I want is
to be the one asking for it, because it is not reasonable to assume that
just throwing it in there is always going to be fine, because it quite
simply is not.

If you have to ask for it, then it's not a default. You *are* against a
default implementation.

···

On Sat, Sep 9, 2017 at 06:41 Haravikk via swift-evolution < swift-evolution@swift.org> wrote:

On 9 Sep 2017, at 09:33, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Sat, Sep 9, 2017 at 02:47 Haravikk via swift-evolution < > swift-evolution@swift.org> wrote:

On 9 Sep 2017, at 02:02, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Fri, Sep 8, 2017 at 4:00 PM, Itai Ferber via swift-evolution < >> swift-evolution@swift.org> wrote:

On Sep 8, 2017, at 12:46 AM, Haravikk via swift-evolution < >>> swift-evolution@swift.org> wrote:
On 7 Sep 2017, at 22:02, Itai Ferber <iferber@apple.com> wrote:

_______________________________________________

swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

All right, I'll be more positive: our science, IT, is a *constructive* science, by *essence*. If there is a problem, there must be a way to show it.

It you can't, then there is no problem.

Gwendal

···

Le 9 sept. 2017 à 15:26, Gwendal Roué <gwendal.roue@gmail.com> a écrit :

Hello Haravikk,

I'lm worried that you fail at preventing a real problem. May I suggest a change in your strategy?

Sometimes, sample code greatly helps turning subtle ideas into blatant evidence. After all, subtleties are all about corner cases, and corner cases are the blind spots of imagination. What about giving that little something that would help your readers grasp your arguments?

I don't quite know what example you will provide, but I could suggest the exhibition of a practical problem with Equatable synthesis. We'll know better if the problem can arise in the Standard lib, in third-party libraries, at application level, or at several scales at the same time. It would also be nice to see your solution to the problem, that is to say an alternative that still provides code synthesis for developers that want to opt in the feature, but avoids the caveat of the initial example. I hope this would greatly help the discussion move forward.

Last general comment about the topic: if Haravikk is right, and that code synthesis should indeed be explicit, then that wouldn't be such a shame.

My two cents,
Gwendal Roué

Le 9 sept. 2017 à 13:41, Haravikk via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> a écrit :

On 9 Sep 2017, at 09:33, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:

On Sat, Sep 9, 2017 at 02:47 Haravikk via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 9 Sep 2017, at 02:02, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:

On Fri, Sep 8, 2017 at 4:00 PM, Itai Ferber via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Sep 8, 2017, at 12:46 AM, Haravikk via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 7 Sep 2017, at 22:02, Itai Ferber <iferber@apple.com <mailto:iferber@apple.com>> wrote:

protocol Fooable : Equatable { // Equatable is just a simple example
    var myFoo: Int { get }
}

extension Fooable {
    static func ==(_ lhs: Self, _ rhs: Self) -> Bool {
        return lhs.myFoo == rhs.myFoo
    }
}

struct X : Fooable {
    let myFoo: Int
    let myName: String
    // Whoops, forgot to give an implementation of ==
}

print(X(myFoo: 42, myName: "Alice") == X(myFoo: 42, myName: "Bob")) // true
This property is necessary, but not sufficient to provide a correct implementation. A default implementation might be able to assume something about the types that it defines, but it does not necessarily know enough.

Sorry but that's a bit of a contrived example; in this case the protocol should not implement the equality operator if more information may be required to define equality. It should only be implemented if the protocol is absolutely clear that .myFoo is the only part of a Fooable that can or should be compared as equatable, e.g- if a Fooable is a database record and .myFoo is a primary key, the data could differ but it would still be a reference to the same record.

To be clear, I'm not arguing that someone can't create a regular default implementation that also makes flawed assumptions, but that synthesised/reflective implementations by their very nature have to, as they cannot under every circumstance guarantee correctness when using parts of a concrete type that they know nothing about.

You can’t argue this both ways:
If you’re arguing this on principle, that in order for synthesized implementations to be correct, they must be able to — under every circumstance — guarantee correctness, then you have to apply the same reasoning to default protocol implementations. Given a default protocol implementation, it is possible to come up with a (no matter how contrived) case where the default implementation is wrong. Since you’re arguing this on principle, you cannot reject contrived examples.
If you are arguing this in practice, then you’re going to have to back up your argument with evidence that synthesized examples are more often wrong than default implementations. You can’t declare that synthesized implementations are by nature incorrect but allow default implementations to slide because in practice, many implementations are allowable. There’s a reason why synthesis passed code review and was accepted: in the majority of cases, synthesis was deemed to be beneficial, and would provide correct behavior. If you are willing to say that yes, sometimes default implementations are wrong but overall they’re correct, you’re going to have to provide hard evidence to back up the opposite case for synthesized implementations. You stated in a previous email that "A synthesised/reflective implementation however may return a result that is simply incorrect, because it is based on assumptions made by the protocol developer, with no input from the developer of the concrete type. In this case the developer must override it in to provide correct behaviour." — if you can back this up with evidence (say, taking a survey of a large number of model types and see if in the majority of cases synthesized implementation would be incorrect) to provide a compelling argument, then this is something that we should in that case reconsider.

Well put, and I agree with this position 100%. However, to play devil's advocate here, let me summarize what I think Haravikk is saying:

I think the "synthesized" part of this is a red herring, if I understand Haravikk's argument correctly. Instead, it is this:

(1) In principle, it is possible to have a default implementation for a protocol requirement that produces the correct result--though not necessarily in the most performant way--for all possible conforming types, where by conforming we mean that the type respects both the syntactic requirements (enforced by the compiler) and the semantic requirements (which may not necessarily be enforceable by the compiler) of the protocol in question.

(2) However, there exist *some* requirements that, by their very nature, cannot have default implementations which are guaranteed to produce the correct result for all conforming types. In Haravikk's view, no default implementations should be provided in these cases. (I don't necessarily subscribe to this view in absolute terms, but for the sake of argument let's grant this premise.)

(3) Equatable, Hashable, and Codable requirements are, by their very nature, such requirements that cannot have default implementations guaranteed to be correct for all conforming types. Therefore, they should not have a default implementation. It just so happens that a default implementation cannot currently be written in Swift itself and must be synthesized, but Haravikk's point is that even if they could be written in native Swift through a hypothetical reflection facility, they should not be, just as many other protocol requirements currently could have default implementations written in Swift but should not have them because they cannot be guaranteed to produce the correct result.

My response to this line of argumentation is as follows:

For any open protocol (i.e., a protocol for which the universe of possible conforming types cannot be enumerated a priori by the protocol designer) worthy of being a protocol by the Swift standard ("what useful thing can you do with such a protocol that you could not without?"), any sufficiently interesting requirement (i.e., one for which user ergonomics would measurably benefit from a default implementation) either cannot have a universally guaranteed correct implementation or has an implementation which is also going to be the most performant one (which can therefore be a non-overridable protocol extension method rather than an overridable protocol requirement with a default implementation).

You're close, but still missing key points:

I am not arguing that features like these should not be provided, but that they should not be provided implicitly, and that the developer should actually be allowed to request them. That is exactly what this proposal is about, yet no matter what I say everyone seems to be treating me like I'm against these features entirely; I am not.

You are entirely against Equatable having a default implementation for ==. This is unequivocally stated. Others favor such a default implementation and feel that in the absence of a way to spell this in Swift itself, it should be magic for the time being. For the purposes of this argument it really is not pertinent that you are not also against something else; you're asking us to discuss why you are against a particular thing that others are for.

FFS, how much clearer can I make this? I AM NOT AGAINST THE FEATURE.

What I am against is the way in which it is being provided implicitly rather than explicitly, in particular as a retroactive change to existing protocols in a way that introduces potential for bugs that are currently impossible, but also in general.

As repeatedly answered by others, nothing here is specific to synthesized default implementations, as more powerful reflection will gradually allow them to be non-synthesised.

And as repeatedly stated by me; I am not treating synthesised vs. run-time reflection any differently, I specifically included both in the original proposal.

As pointed out very cogently by Itai, you assert but offer no evidence, either in principle or empirically, that going too far by reflection is worse than going not far enough without reflection in terms of likelihood of a default implementation being inappropriate for conforming types.

As I have also repeatedly pointed out it is not an issue of "not going far enough" vs. "going too far"; if a default implementation lacks information then it should not be provided, doing so regardless is a flaw in the protocol design and not something that this proposal attempts to address (as such a thing is likely impossible).

Reflective implementations necessarily go too far, because they literally know nothing about the concrete type with any certainty, except for the properties that are defined in the protocol (which do not require reflection or synthesis in the first place).

And precisely what kind of "evidence" am I expected to give? This is a set of features that do not exist yet, I am trying to argue in favour of an explicit end-developer centric opt-in rather than an implicit protocol designer centric one. Yet no-one seems interested in the merits of allowing developers to choose what they want, rather than having implicit behaviours appear potentially unexpectedly.

Therefore, your argument reduces to one about which default implementations generally ought or ought not to be provided--that is, that they ought to be provided only when their correctness can be guaranteed for all (rather than almost all) possible conforming types. To which point I sketched a rebuttal above.

If a protocol defines something, and creates a default implementation based only upon those definitions then it must by its very nature be correct. A concrete type may later decided to go further, but that is a feature of the concrete type, not a failure of the protocol itself which can function correctly within the context it created. You want to talk evidence, yet there has been no example given that proves otherwise; thus far only Itai has attempted to do so, but I have already pointed out the flaws with that example.

The simple fact is that a default implementation may either be flawed or not within the context of the protocol itself; but a reflective or synthetic implementation by its very nature goes beyond what the protocol defines and so is automatically flawed because as it does not rely on the end-developer to confirm correctness, not when provided implicitly at least.

And all of this continues to be a side-issue to the fact that in the specific case of Equatable/Hashable, which thus far has gone ignored, is that bolting this on retroactively to an existing protocol hides bugs. The issue of reflective default implementations is less of a concern on very clearly and well defined new protocols, though I still prefer more, rather than less, control, but in the specific case of existing protocols this fucking about with behaviours is reckless and foolish in the extreme, yet no-one on the core teams seems willing or able to justify it, which only opens much wider concerns (how am I to have any faith in Swift's development if the core team can't or won't justify the creation of new bugs?).

This has emphatically not gone ignored, as I have myself responded to this point in an earlier thread in which you commented, as well as many others. Crucially, no existing conforming type changes its behavior, as they have all had to implement these requirements themselves. And as I said to you already, the addition of a synthesized default implementation no more "hides bugs" going forward than the addition of a non-synthesized default implementation to an existing protocol, and we do that with some frequency without even Swift Evolution review.

Feel free to a supply a non-synthesised default implementation for Equatable without the use of reflection. Go-on, I'll wait.
You insist on suggesting these are the same thing, yet if you can't provide one then clearly they are not.

Put another way, what the proposal about synthesizing implementations for Equatable and Hashable was about can be thought of in two parts: (a) should there be default implementations; and (b) given that it is impossible to write these in Swift, should we use magic? Now, as I said above, adding default implementations isn't (afaik) even considered an API change that requires review on this list. Really, what people were debating was (b), whether it is worth it to implement compiler-supported magic to make these possible. Your disagreement has to do with (a) and not (b).

Wrong. The use of magic in this case produces something else entirely; that's the whole point. It is not the same, otherwise it wouldn't be needed at all. It doesn't matter if it's compiler magic, some external script or a native macro, ultimately they are all doing something with a concrete type that is currently not possible.

And once again; I am not arguing against a default implementation that cuts boilerplate, I am arguing against it being implicit. What I want is to be the one asking for it, because it is not reasonable to assume that just throwing it in there is always going to be fine, because it quite simply is not.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

protocol Fooable : Equatable { // Equatable is just a simple example
    var myFoo: Int { get }
}

extension Fooable {
    static func ==(_ lhs: Self, _ rhs: Self) -> Bool {
        return lhs.myFoo == rhs.myFoo
    }
}

struct X : Fooable {
    let myFoo: Int
    let myName: String
    // Whoops, forgot to give an implementation of ==
}

print(X(myFoo: 42, myName: "Alice") == X(myFoo: 42, myName: "Bob")) // true
This property is necessary, but not sufficient to provide a correct implementation. A default implementation might be able to assume something about the types that it defines, but it does not necessarily know enough.

Sorry but that's a bit of a contrived example; in this case the protocol should not implement the equality operator if more information may be required to define equality. It should only be implemented if the protocol is absolutely clear that .myFoo is the only part of a Fooable that can or should be compared as equatable, e.g- if a Fooable is a database record and .myFoo is a primary key, the data could differ but it would still be a reference to the same record.

To be clear, I'm not arguing that someone can't create a regular default implementation that also makes flawed assumptions, but that synthesised/reflective implementations by their very nature have to, as they cannot under every circumstance guarantee correctness when using parts of a concrete type that they know nothing about.

You can’t argue this both ways:
If you’re arguing this on principle, that in order for synthesized implementations to be correct, they must be able to — under every circumstance — guarantee correctness, then you have to apply the same reasoning to default protocol implementations. Given a default protocol implementation, it is possible to come up with a (no matter how contrived) case where the default implementation is wrong. Since you’re arguing this on principle, you cannot reject contrived examples.
If you are arguing this in practice, then you’re going to have to back up your argument with evidence that synthesized examples are more often wrong than default implementations. You can’t declare that synthesized implementations are by nature incorrect but allow default implementations to slide because in practice, many implementations are allowable. There’s a reason why synthesis passed code review and was accepted: in the majority of cases, synthesis was deemed to be beneficial, and would provide correct behavior. If you are willing to say that yes, sometimes default implementations are wrong but overall they’re correct, you’re going to have to provide hard evidence to back up the opposite case for synthesized implementations. You stated in a previous email that "A synthesised/reflective implementation however may return a result that is simply incorrect, because it is based on assumptions made by the protocol developer, with no input from the developer of the concrete type. In this case the developer must override it in to provide correct behaviour." — if you can back this up with evidence (say, taking a survey of a large number of model types and see if in the majority of cases synthesized implementation would be incorrect) to provide a compelling argument, then this is something that we should in that case reconsider.

Well put, and I agree with this position 100%. However, to play devil's advocate here, let me summarize what I think Haravikk is saying:

I think the "synthesized" part of this is a red herring, if I understand Haravikk's argument correctly. Instead, it is this:

(1) In principle, it is possible to have a default implementation for a protocol requirement that produces the correct result--though not necessarily in the most performant way--for all possible conforming types, where by conforming we mean that the type respects both the syntactic requirements (enforced by the compiler) and the semantic requirements (which may not necessarily be enforceable by the compiler) of the protocol in question.

(2) However, there exist *some* requirements that, by their very nature, cannot have default implementations which are guaranteed to produce the correct result for all conforming types. In Haravikk's view, no default implementations should be provided in these cases. (I don't necessarily subscribe to this view in absolute terms, but for the sake of argument let's grant this premise.)

(3) Equatable, Hashable, and Codable requirements are, by their very nature, such requirements that cannot have default implementations guaranteed to be correct for all conforming types. Therefore, they should not have a default implementation. It just so happens that a default implementation cannot currently be written in Swift itself and must be synthesized, but Haravikk's point is that even if they could be written in native Swift through a hypothetical reflection facility, they should not be, just as many other protocol requirements currently could have default implementations written in Swift but should not have them because they cannot be guaranteed to produce the correct result.

My response to this line of argumentation is as follows:

For any open protocol (i.e., a protocol for which the universe of possible conforming types cannot be enumerated a priori by the protocol designer) worthy of being a protocol by the Swift standard ("what useful thing can you do with such a protocol that you could not without?"), any sufficiently interesting requirement (i.e., one for which user ergonomics would measurably benefit from a default implementation) either cannot have a universally guaranteed correct implementation or has an implementation which is also going to be the most performant one (which can therefore be a non-overridable protocol extension method rather than an overridable protocol requirement with a default implementation).

You're close, but still missing key points:

I am not arguing that features like these should not be provided, but that they should not be provided implicitly, and that the developer should actually be allowed to request them. That is exactly what this proposal is about, yet no matter what I say everyone seems to be treating me like I'm against these features entirely; I am not.

You are entirely against Equatable having a default implementation for ==. This is unequivocally stated. Others favor such a default implementation and feel that in the absence of a way to spell this in Swift itself, it should be magic for the time being. For the purposes of this argument it really is not pertinent that you are not also against something else; you're asking us to discuss why you are against a particular thing that others are for.

FFS, how much clearer can I make this? I AM NOT AGAINST THE FEATURE.

What I am against is the way in which it is being provided implicitly rather than explicitly, in particular as a retroactive change to existing protocols in a way that introduces potential for bugs that are currently impossible, but also in general.

You are against a default implementation for ==, i.e. an implementation that is provided for you if you conform a type to the protocol and do nothing else ("implicitly rather than explicitly"), and you are against the default implementation being on the existing protocol Equatable ("retroactive change"). So, to summarize, what you are against is precisely a default implementation for the == requirement on Equatable.

This is the topic of discussion here; I am attempting to convince you that you should be for rather than against these things.

I am for it, the difference is that I want to ask for it, not have it thrust upon me.

Reflective implementations necessarily go too far, because they literally know nothing about the concrete type with any certainty, except for the properties that are defined in the protocol (which do not require reflection or synthesis in the first place).

I am confused why you are trying to argue in general terms about the universe of all possible default implementations that use reflection. This is necessarily a more difficult argument to make, and if it is to be convincing for all default implementations it must also be convincing for the two specific protocol requirements we are talking about here. Start small:

We have agreed, as a community, that there is a reasonable default implementation for Equatable.== when certain conditions are met (for value types only at the moment, I believe). Namely, given two values of a type that has only Equatable stored properties, those values are equal if their stored properties are all equal. The author of a new value type who wishes to make her type Equatable but chooses not to implement a custom == then benefits from this default when all stored properties are Equatable.

See, this is another flawed assumption; you are assuming that omitting a custom implementation of == is always intentional rather than an oversight, which is not guaranteed. This is one of my gripes with the retroactive change to Equatable, as it is currently impossible to omit an implementation.

And precisely what kind of "evidence" am I expected to give? This is a set of features that do not exist yet, I am trying to argue in favour of an explicit end-developer centric opt-in rather than an implicit protocol designer centric one. Yet no-one seems interested in the merits of allowing developers to choose what they want, rather than having implicit behaviours appear potentially unexpectedly.

Both options were examined for Codable and for Equatable/Hashable. The community and core team decided to prefer the current design. At this point, new insights that arise which could not be anticipated at the time of review could prompt revision. However, so far, you have presented arguments already considered during review.

And so far all I have heard about this is how it was "decided"; no-one seems interested in showing how any of these concerns were addressed (if at all), so as far as I can tell they were not, or they were wilfully ignored.

Therefore, your argument reduces to one about which default implementations generally ought or ought not to be provided--that is, that they ought to be provided only when their correctness can be guaranteed for all (rather than almost all) possible conforming types. To which point I sketched a rebuttal above.

If a protocol defines something, and creates a default implementation based only upon those definitions then it must by its very nature be correct. A concrete type may later decided to go further, but that is a feature of the concrete type, not a failure of the protocol itself which can function correctly within the context it created. You want to talk evidence, yet there has been no example given that proves otherwise; thus far only Itai has attempted to do so, but I have already pointed out the flaws with that example.

The simple fact is that a default implementation may either be flawed or not within the context of the protocol itself; but a reflective or synthetic implementation by its very nature goes beyond what the protocol defines and so is automatically flawed because as it does not rely on the end-developer to confirm correctness, not when provided implicitly at least.

Again, if it applies generally, it must apply specifically. What is "automatically flawed" about the very reasonable synthesized default implementation of ==?

It makes the assumption that every equatable property of a type is necessarily relevant to its equality. Consider for example if a type stores a collection index for performance reasons; this isn't an intrinsic part of the type, nor relevant to testing equality, yet this default implementation will treat it as such because it knows nothing about the concrete type's properties. If a protocol does not define a property then any action taken upon such a property is necessarily based upon an assumption; just because it might be fine some of the time, does not make it any less flawed.

The big difference here between explicit and implicit synthetic implementations is where this assumption originates; if a method is synthesised implicitly then the assumption is made by the protocol designer alone, with no real involvement by the end developer. If I explicitly opt-in to that default however I am signalling to the protocol that it is okay to proceed. In the former case the assumption is unreasonable, in the latter it is explicitly authorised. It is a difference between "I want to make the decision on what's correct" and "I am happy for you (the protocol designer) to decide".

Right now, when I conform to Equatable, it is a declaration of "I will implement this", but with this retroactive implicit change it is now a declaration of "implement this for me", these are two entirely different things. Consider; what if I'm working on a piece of code that requires types to be Equatable, but one of the types I'm using currently isn't, so I quickly throw Equatable conformance onto it and go back to what I was doing, with the intention of completing conformance later. With this change that type may now receive a default implementation that is wrong, and I've lost the safety net that currently exists.

And all of this continues to be a side-issue to the fact that in the specific case of Equatable/Hashable, which thus far has gone ignored, is that bolting this on retroactively to an existing protocol hides bugs. The issue of reflective default implementations is less of a concern on very clearly and well defined new protocols, though I still prefer more, rather than less, control, but in the specific case of existing protocols this fucking about with behaviours is reckless and foolish in the extreme, yet no-one on the core teams seems willing or able to justify it, which only opens much wider concerns (how am I to have any faith in Swift's development if the core team can't or won't justify the creation of new bugs?).

This has emphatically not gone ignored, as I have myself responded to this point in an earlier thread in which you commented, as well as many others. Crucially, no existing conforming type changes its behavior, as they have all had to implement these requirements themselves. And as I said to you already, the addition of a synthesized default implementation no more "hides bugs" going forward than the addition of a non-synthesized default implementation to an existing protocol, and we do that with some frequency without even Swift Evolution review.

Feel free to a supply a non-synthesised default implementation for Equatable without the use of reflection. Go-on, I'll wait.
You insist on suggesting these are the same thing, yet if you can't provide one then clearly they are not.

That is not the argument. The argument is that they are indistinguishable in the sense that the author of a type who intends to supply a custom implementation but neglects to do so will have a default implementation supplied for them. It is plainly true that this is no more or less likely to happen simply because the default implementation is synthesised.

A non-synthesised/reflective implementation cannot strictly be incorrect, because as long as it is implemented properly it will always be correct within the context of the protocol itself. It may not go quite as far as an end developer might want, but that is because they want to add something onto the protocol, not because the protocol is wrong.

A synthesised/reflective implementation differs because if it goes too far it is wrong not only within the context of the concrete type, but also the protocol itself, it is simply incorrect.

Put another way, what the proposal about synthesizing implementations for Equatable and Hashable was about can be thought of in two parts: (a) should there be default implementations; and (b) given that it is impossible to write these in Swift, should we use magic? Now, as I said above, adding default implementations isn't (afaik) even considered an API change that requires review on this list. Really, what people were debating was (b), whether it is worth it to implement compiler-supported magic to make these possible. Your disagreement has to do with (a) and not (b).

Wrong. The use of magic in this case produces something else entirely; that's the whole point. It is not the same, otherwise it wouldn't be needed at all. It doesn't matter if it's compiler magic, some external script or a native macro, ultimately they are all doing something with a concrete type that is currently not possible.

And once again; I am not arguing against a default implementation that cuts boilerplate, I am arguing against it being implicit. What I want is to be the one asking for it, because it is not reasonable to assume that just throwing it in there is always going to be fine, because it quite simply is not.

If you have to ask for it, then it's not a default. You *are* against a default implementation.

A default implementation is an implementation that I, as the concrete type developer, do not have to provide myself. If you want default to mean only "automatic" then your attempt to pigeon-hole what I am arguing is incorrect, because what I am arguing is then neither about default implementations nor the means of actually implementing it, but something else entirely.

But as far as I'm concerned it still absolutely still a default implementation whether it is requested or not; the difference is I, as the end developer, am able to refine what type of defaults that I want.

All right, I'll be more positive: our science, IT, is a *constructive* science, by *essence*. If there is a problem, there must be a way to show it.
It you can't, then there is no problem.

You mean just as I have asked for examples that prove non-synthetic/reflective default implementations are as dangerous as synthetic/reflective ones? Plenty have suggested this is the case yet no reasonable examples of that have been given either.

However, examples highlighting problems with the synthesised behaviour are simple:

struct Foo : Equatable { var data:String } // Currently an error, won't be in future

Or something a bit more substantial:

struct KeyPair : Equatable {
  static var count:Int = 0

  var count:Int
  let key:String // This is the only property that should be equatable
  var value:String

  init(key:String, value:String) {
    let count = KeyPair.count &+ 1
    KeyPair.count = count; self.count = count
    self.key = key; self.value = value
  }
}

Here the only important property in the key pair is the key, the value isn't important (only the keys are to be considered unique) and the count is just a throwaway value. The synthesised default implementation for this concrete type will therefore be completely wrong, likewise for Hashable, which will likely produce radically different results for instances that should be the same.

···

On 9 Sep 2017, at 18:41, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Sat, Sep 9, 2017 at 06:41 Haravikk via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 9 Sep 2017, at 09:33, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:
On Sat, Sep 9, 2017 at 02:47 Haravikk via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 9 Sep 2017, at 02:02, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:
On Fri, Sep 8, 2017 at 4:00 PM, Itai Ferber via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Sep 8, 2017, at 12:46 AM, Haravikk via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 7 Sep 2017, at 22:02, Itai Ferber <iferber@apple.com <mailto:iferber@apple.com>> wrote:

On 9 Sep 2017, at 23:17, Gwendal Roué <gwendal.roue@gmail.com> wrote:

I think I do understand Haravikk's argument (actually it seems quite straightforward to me).

An example should be:

struct Foo : Equatable {
    var x: Int
    var cachedLabel: String? = nil

    init(x: Int) {
        self.x = x
    }

    mutating func label() {
        if let label = cachedLabel {
            return label
        }
        let label = calculateLabel()
        cachedLabel = label
        return cachedLabel
    }
}

var foo1 = Foo(x: 1)
var foo2 = Foo(x: 1)
foo1 == foo2 // true
var label = foo1.label()
foo1 == foo2 // now false, due to cachedString being falsely included in the comparison

The problem is that the developer was not required to implement the protocol and so might forget it.
The difference to other default implementations is that those use the protocol itself as building blocks and so are correct with regards to the protocol's semantics, whereas the synthesized equality reaches deeply into the private innards of a struct and therefore is much more likely to be wrong as in the example above.

Why not just write

     struct Foo : deriving Equatable {...}

to request the synthesized implementation?

-Thorsten

···

Am 09.09.2017 um 19:42 schrieb Xiaodi Wu via swift-evolution <swift-evolution@swift.org>:

On Sat, Sep 9, 2017 at 06:41 Haravikk via swift-evolution <swift-evolution@swift.org> wrote:

On 9 Sep 2017, at 09:33, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Sat, Sep 9, 2017 at 02:47 Haravikk via swift-evolution <swift-evolution@swift.org> wrote:

On 9 Sep 2017, at 02:02, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Fri, Sep 8, 2017 at 4:00 PM, Itai Ferber via swift-evolution <swift-evolution@swift.org> wrote:

On Sep 8, 2017, at 12:46 AM, Haravikk via swift-evolution <swift-evolution@swift.org> wrote:

On 7 Sep 2017, at 22:02, Itai Ferber <iferber@apple.com> wrote:

protocol Fooable : Equatable { // Equatable is just a simple example
    var myFoo: Int { get }
}

extension Fooable {
    static func ==(_ lhs: Self, _ rhs: Self) -> Bool {
        return lhs.myFoo == rhs.myFoo
    }
}

struct X : Fooable {
    let myFoo: Int
    let myName: String
    // Whoops, forgot to give an implementation of ==
}

print(X(myFoo: 42, myName: "Alice") == X(myFoo: 42, myName: "Bob")) // true
This property is necessary, but not sufficient to provide a correct implementation. A default implementation might be able to assume something about the types that it defines, but it does not necessarily know enough.

Sorry but that's a bit of a contrived example; in this case the protocol should not implement the equality operator if more information may be required to define equality. It should only be implemented if the protocol is absolutely clear that .myFoo is the only part of a Fooable that can or should be compared as equatable, e.g- if a Fooable is a database record and .myFoo is a primary key, the data could differ but it would still be a reference to the same record.

To be clear, I'm not arguing that someone can't create a regular default implementation that also makes flawed assumptions, but that synthesised/reflective implementations by their very nature have to, as they cannot under every circumstance guarantee correctness when using parts of a concrete type that they know nothing about.

You can’t argue this both ways:
If you’re arguing this on principle, that in order for synthesized implementations to be correct, they must be able to — under every circumstance — guarantee correctness, then you have to apply the same reasoning to default protocol implementations. Given a default protocol implementation, it is possible to come up with a (no matter how contrived) case where the default implementation is wrong. Since you’re arguing this on principle, you cannot reject contrived examples.
If you are arguing this in practice, then you’re going to have to back up your argument with evidence that synthesized examples are more often wrong than default implementations. You can’t declare that synthesized implementations are by nature incorrect but allow default implementations to slide because in practice, many implementations are allowable. There’s a reason why synthesis passed code review and was accepted: in the majority of cases, synthesis was deemed to be beneficial, and would provide correct behavior. If you are willing to say that yes, sometimes default implementations are wrong but overall they’re correct, you’re going to have to provide hard evidence to back up the opposite case for synthesized implementations. You stated in a previous email that "A synthesised/reflective implementation however may return a result that is simply incorrect, because it is based on assumptions made by the protocol developer, with no input from the developer of the concrete type. In this case the developer must override it in to provide correct behaviour." — if you can back this up with evidence (say, taking a survey of a large number of model types and see if in the majority of cases synthesized implementation would be incorrect) to provide a compelling argument, then this is something that we should in that case reconsider.

Well put, and I agree with this position 100%. However, to play devil's advocate here, let me summarize what I think Haravikk is saying:

I think the "synthesized" part of this is a red herring, if I understand Haravikk's argument correctly. Instead, it is this:

(1) In principle, it is possible to have a default implementation for a protocol requirement that produces the correct result--though not necessarily in the most performant way--for all possible conforming types, where by conforming we mean that the type respects both the syntactic requirements (enforced by the compiler) and the semantic requirements (which may not necessarily be enforceable by the compiler) of the protocol in question.

(2) However, there exist *some* requirements that, by their very nature, cannot have default implementations which are guaranteed to produce the correct result for all conforming types. In Haravikk's view, no default implementations should be provided in these cases. (I don't necessarily subscribe to this view in absolute terms, but for the sake of argument let's grant this premise.)

(3) Equatable, Hashable, and Codable requirements are, by their very nature, such requirements that cannot have default implementations guaranteed to be correct for all conforming types. Therefore, they should not have a default implementation. It just so happens that a default implementation cannot currently be written in Swift itself and must be synthesized, but Haravikk's point is that even if they could be written in native Swift through a hypothetical reflection facility, they should not be, just as many other protocol requirements currently could have default implementations written in Swift but should not have them because they cannot be guaranteed to produce the correct result.

My response to this line of argumentation is as follows:

For any open protocol (i.e., a protocol for which the universe of possible conforming types cannot be enumerated a priori by the protocol designer) worthy of being a protocol by the Swift standard ("what useful thing can you do with such a protocol that you could not without?"), any sufficiently interesting requirement (i.e., one for which user ergonomics would measurably benefit from a default implementation) either cannot have a universally guaranteed correct implementation or has an implementation which is also going to be the most performant one (which can therefore be a non-overridable protocol extension method rather than an overridable protocol requirement with a default implementation).

You're close, but still missing key points:

I am not arguing that features like these should not be provided, but that they should not be provided implicitly, and that the developer should actually be allowed to request them. That is exactly what this proposal is about, yet no matter what I say everyone seems to be treating me like I'm against these features entirely; I am not.

You are entirely against Equatable having a default implementation for ==. This is unequivocally stated. Others favor such a default implementation and feel that in the absence of a way to spell this in Swift itself, it should be magic for the time being. For the purposes of this argument it really is not pertinent that you are not also against something else; you're asking us to discuss why you are against a particular thing that others are for.

FFS, how much clearer can I make this? I AM NOT AGAINST THE FEATURE.

What I am against is the way in which it is being provided implicitly rather than explicitly, in particular as a retroactive change to existing protocols in a way that introduces potential for bugs that are currently impossible, but also in general.

You are against a default implementation for ==, i.e. an implementation that is provided for you if you conform a type to the protocol and do nothing else ("implicitly rather than explicitly"), and you are against the default implementation being on the existing protocol Equatable ("retroactive change"). So, to summarize, what you are against is precisely a default implementation for the == requirement on Equatable.

This is the topic of discussion here; I am attempting to convince you that you should be for rather than against these things.

As repeatedly answered by others, nothing here is specific to synthesized default implementations, as more powerful reflection will gradually allow them to be non-synthesised.

And as repeatedly stated by me; I am not treating synthesised vs. run-time reflection any differently, I specifically included both in the original proposal.

As pointed out very cogently by Itai, you assert but offer no evidence, either in principle or empirically, that going too far by reflection is worse than going not far enough without reflection in terms of likelihood of a default implementation being inappropriate for conforming types.

As I have also repeatedly pointed out it is not an issue of "not going far enough" vs. "going too far"; if a default implementation lacks information then it should not be provided, doing so regardless is a flaw in the protocol design and not something that this proposal attempts to address (as such a thing is likely impossible).

Right, one must consider the semantics of the specific protocol requirement and ask whether a reasonable default can be provided for it.

Reflective implementations necessarily go too far, because they literally know nothing about the concrete type with any certainty, except for the properties that are defined in the protocol (which do not require reflection or synthesis in the first place).

I am confused why you are trying to argue in general terms about the universe of all possible default implementations that use reflection. This is necessarily a more difficult argument to make, and if it is to be convincing for all default implementations it must also be convincing for the two specific protocol requirements we are talking about here. Start small:

We have agreed, as a community, that there is a reasonable default implementation for Equatable.== when certain conditions are met (for value types only at the moment, I believe). Namely, given two values of a type that has only Equatable stored properties, those values are equal if their stored properties are all equal. The author of a new value type who wishes to make her type Equatable but chooses not to implement a custom == then benefits from this default when all stored properties are Equatable.

And precisely what kind of "evidence" am I expected to give? This is a set of features that do not exist yet, I am trying to argue in favour of an explicit end-developer centric opt-in rather than an implicit protocol designer centric one. Yet no-one seems interested in the merits of allowing developers to choose what they want, rather than having implicit behaviours appear potentially unexpectedly.

Both options were examined for Codable and for Equatable/Hashable. The community and core team decided to prefer the current design. At this point, new insights that arise which could not be anticipated at the time of review could prompt revision. However, so far, you have presented arguments already considered during review.

Therefore, your argument reduces to one about which default implementations generally ought or ought not to be provided--that is, that they ought to be provided only when their correctness can be guaranteed for all (rather than almost all) possible conforming types. To which point I sketched a rebuttal above.

If a protocol defines something, and creates a default implementation based only upon those definitions then it must by its very nature be correct. A concrete type may later decided to go further, but that is a feature of the concrete type, not a failure of the protocol itself which can function correctly within the context it created. You want to talk evidence, yet there has been no example given that proves otherwise; thus far only Itai has attempted to do so, but I have already pointed out the flaws with that example.

The simple fact is that a default implementation may either be flawed or not within the context of the protocol itself; but a reflective or synthetic implementation by its very nature goes beyond what the protocol defines and so is automatically flawed because as it does not rely on the end-developer to confirm correctness, not when provided implicitly at least.

Again, if it applies generally, it must apply specifically. What is "automatically flawed" about the very reasonable synthesized default implementation of ==?

And all of this continues to be a side-issue to the fact that in the specific case of Equatable/Hashable, which thus far has gone ignored, is that bolting this on retroactively to an existing protocol hides bugs. The issue of reflective default implementations is less of a concern on very clearly and well defined new protocols, though I still prefer more, rather than less, control, but in the specific case of existing protocols this fucking about with behaviours is reckless and foolish in the extreme, yet no-one on the core teams seems willing or able to justify it, which only opens much wider concerns (how am I to have any faith in Swift's development if the core team can't or won't justify the creation of new bugs?).

This has emphatically not gone ignored, as I have myself responded to this point in an earlier thread in which you commented, as well as many others. Crucially, no existing conforming type changes its behavior, as they have all had to implement these requirements themselves. And as I said to you already, the addition of a synthesized default implementation no more "hides bugs" going forward than the addition of a non-synthesized default implementation to an existing protocol, and we do that with some frequency without even Swift Evolution review.

Feel free to a supply a non-synthesised default implementation for Equatable without the use of reflection. Go-on, I'll wait.
You insist on suggesting these are the same thing, yet if you can't provide one then clearly they are not.

That is not the argument. The argument is that they are indistinguishable in the sense that the author of a type who intends to supply a custom implementation but neglects to do so will have a default implementation supplied for them. It is plainly true that this is no more or less likely to happen simply because the default implementation is synthesized.

Put another way, what the proposal about synthesizing implementations for Equatable and Hashable was about can be thought of in two parts: (a) should there be default implementations; and (b) given that it is impossible to write these in Swift, should we use magic? Now, as I said above, adding default implementations isn't (afaik) even considered an API change that requires review on this list. Really, what people were debating was (b), whether it is worth it to implement compiler-supported magic to make these possible. Your disagreement has to do with (a) and not (b).

Wrong. The use of magic in this case produces something else entirely; that's the whole point. It is not the same, otherwise it wouldn't be needed at all. It doesn't matter if it's compiler magic, some external script or a native macro, ultimately they are all doing something with a concrete type that is currently not possible.

And once again; I am not arguing against a default implementation that cuts boilerplate, I am arguing against it being implicit. What I want is to be the one asking for it, because it is not reasonable to assume that just throwing it in there is always going to be fine, because it quite simply is not.

If you have to ask for it, then it's not a default. You *are* against a default implementation.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Mainly because we want synthesis without having to debate keyword naming and killing the proposal, which is what happened every other time before the latest proposal that was actually implemented. I’ll take the tiny danger for the feature in the 99% of other cases which I want it.

Jon

···

On Sep 11, 2017, at 2:54 PM, Thorsten Seitz via swift-evolution <swift-evolution@swift.org> wrote:

I think I do understand Haravikk's argument (actually it seems quite straightforward to me).

An example should be:

struct Foo : Equatable {
    var x: Int
    var cachedLabel: String? = nil

    init(x: Int) {
        self.x = x
    }

    mutating func label() {
        if let label = cachedLabel {
            return label
        }
        let label = calculateLabel()
        cachedLabel = label
        return cachedLabel
    }
}

var foo1 = Foo(x: 1)
var foo2 = Foo(x: 1)
foo1 == foo2 // true
var label = foo1.label()
foo1 == foo2 // now false, due to cachedString being falsely included in the comparison

The problem is that the developer was not required to implement the protocol and so might forget it.
The difference to other default implementations is that those use the protocol itself as building blocks and so are correct with regards to the protocol's semantics, whereas the synthesized equality reaches deeply into the private innards of a struct and therefore is much more likely to be wrong as in the example above.

Why not just write

     struct Foo : deriving Equatable {...}

to request the synthesized implementation?

-Thorsten

Am 09.09.2017 um 19:42 schrieb Xiaodi Wu via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>>:

On Sat, Sep 9, 2017 at 06:41 Haravikk via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 9 Sep 2017, at 09:33, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:

On Sat, Sep 9, 2017 at 02:47 Haravikk via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 9 Sep 2017, at 02:02, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:

On Fri, Sep 8, 2017 at 4:00 PM, Itai Ferber via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Sep 8, 2017, at 12:46 AM, Haravikk via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 7 Sep 2017, at 22:02, Itai Ferber <iferber@apple.com <mailto:iferber@apple.com>> wrote:

protocol Fooable : Equatable { // Equatable is just a simple example
    var myFoo: Int { get }
}

extension Fooable {
    static func ==(_ lhs: Self, _ rhs: Self) -> Bool {
        return lhs.myFoo == rhs.myFoo
    }
}

struct X : Fooable {
    let myFoo: Int
    let myName: String
    // Whoops, forgot to give an implementation of ==
}

print(X(myFoo: 42, myName: "Alice") == X(myFoo: 42, myName: "Bob")) // true
This property is necessary, but not sufficient to provide a correct implementation. A default implementation might be able to assume something about the types that it defines, but it does not necessarily know enough.

Sorry but that's a bit of a contrived example; in this case the protocol should not implement the equality operator if more information may be required to define equality. It should only be implemented if the protocol is absolutely clear that .myFoo is the only part of a Fooable that can or should be compared as equatable, e.g- if a Fooable is a database record and .myFoo is a primary key, the data could differ but it would still be a reference to the same record.

To be clear, I'm not arguing that someone can't create a regular default implementation that also makes flawed assumptions, but that synthesised/reflective implementations by their very nature have to, as they cannot under every circumstance guarantee correctness when using parts of a concrete type that they know nothing about.

You can’t argue this both ways:
If you’re arguing this on principle, that in order for synthesized implementations to be correct, they must be able to — under every circumstance — guarantee correctness, then you have to apply the same reasoning to default protocol implementations. Given a default protocol implementation, it is possible to come up with a (no matter how contrived) case where the default implementation is wrong. Since you’re arguing this on principle, you cannot reject contrived examples.
If you are arguing this in practice, then you’re going to have to back up your argument with evidence that synthesized examples are more often wrong than default implementations. You can’t declare that synthesized implementations are by nature incorrect but allow default implementations to slide because in practice, many implementations are allowable. There’s a reason why synthesis passed code review and was accepted: in the majority of cases, synthesis was deemed to be beneficial, and would provide correct behavior. If you are willing to say that yes, sometimes default implementations are wrong but overall they’re correct, you’re going to have to provide hard evidence to back up the opposite case for synthesized implementations. You stated in a previous email that "A synthesised/reflective implementation however may return a result that is simply incorrect, because it is based on assumptions made by the protocol developer, with no input from the developer of the concrete type. In this case the developer must override it in to provide correct behaviour." — if you can back this up with evidence (say, taking a survey of a large number of model types and see if in the majority of cases synthesized implementation would be incorrect) to provide a compelling argument, then this is something that we should in that case reconsider.

Well put, and I agree with this position 100%. However, to play devil's advocate here, let me summarize what I think Haravikk is saying:

I think the "synthesized" part of this is a red herring, if I understand Haravikk's argument correctly. Instead, it is this:

(1) In principle, it is possible to have a default implementation for a protocol requirement that produces the correct result--though not necessarily in the most performant way--for all possible conforming types, where by conforming we mean that the type respects both the syntactic requirements (enforced by the compiler) and the semantic requirements (which may not necessarily be enforceable by the compiler) of the protocol in question.

(2) However, there exist *some* requirements that, by their very nature, cannot have default implementations which are guaranteed to produce the correct result for all conforming types. In Haravikk's view, no default implementations should be provided in these cases. (I don't necessarily subscribe to this view in absolute terms, but for the sake of argument let's grant this premise.)

(3) Equatable, Hashable, and Codable requirements are, by their very nature, such requirements that cannot have default implementations guaranteed to be correct for all conforming types. Therefore, they should not have a default implementation. It just so happens that a default implementation cannot currently be written in Swift itself and must be synthesized, but Haravikk's point is that even if they could be written in native Swift through a hypothetical reflection facility, they should not be, just as many other protocol requirements currently could have default implementations written in Swift but should not have them because they cannot be guaranteed to produce the correct result.

My response to this line of argumentation is as follows:

For any open protocol (i.e., a protocol for which the universe of possible conforming types cannot be enumerated a priori by the protocol designer) worthy of being a protocol by the Swift standard ("what useful thing can you do with such a protocol that you could not without?"), any sufficiently interesting requirement (i.e., one for which user ergonomics would measurably benefit from a default implementation) either cannot have a universally guaranteed correct implementation or has an implementation which is also going to be the most performant one (which can therefore be a non-overridable protocol extension method rather than an overridable protocol requirement with a default implementation).

You're close, but still missing key points:

I am not arguing that features like these should not be provided, but that they should not be provided implicitly, and that the developer should actually be allowed to request them. That is exactly what this proposal is about, yet no matter what I say everyone seems to be treating me like I'm against these features entirely; I am not.

You are entirely against Equatable having a default implementation for ==. This is unequivocally stated. Others favor such a default implementation and feel that in the absence of a way to spell this in Swift itself, it should be magic for the time being. For the purposes of this argument it really is not pertinent that you are not also against something else; you're asking us to discuss why you are against a particular thing that others are for.

FFS, how much clearer can I make this? I AM NOT AGAINST THE FEATURE.

What I am against is the way in which it is being provided implicitly rather than explicitly, in particular as a retroactive change to existing protocols in a way that introduces potential for bugs that are currently impossible, but also in general.

You are against a default implementation for ==, i.e. an implementation that is provided for you if you conform a type to the protocol and do nothing else ("implicitly rather than explicitly"), and you are against the default implementation being on the existing protocol Equatable ("retroactive change"). So, to summarize, what you are against is precisely a default implementation for the == requirement on Equatable.

This is the topic of discussion here; I am attempting to convince you that you should be for rather than against these things.

As repeatedly answered by others, nothing here is specific to synthesized default implementations, as more powerful reflection will gradually allow them to be non-synthesised.

And as repeatedly stated by me; I am not treating synthesised vs. run-time reflection any differently, I specifically included both in the original proposal.

As pointed out very cogently by Itai, you assert but offer no evidence, either in principle or empirically, that going too far by reflection is worse than going not far enough without reflection in terms of likelihood of a default implementation being inappropriate for conforming types.

As I have also repeatedly pointed out it is not an issue of "not going far enough" vs. "going too far"; if a default implementation lacks information then it should not be provided, doing so regardless is a flaw in the protocol design and not something that this proposal attempts to address (as such a thing is likely impossible).

Right, one must consider the semantics of the specific protocol requirement and ask whether a reasonable default can be provided for it.

Reflective implementations necessarily go too far, because they literally know nothing about the concrete type with any certainty, except for the properties that are defined in the protocol (which do not require reflection or synthesis in the first place).

I am confused why you are trying to argue in general terms about the universe of all possible default implementations that use reflection. This is necessarily a more difficult argument to make, and if it is to be convincing for all default implementations it must also be convincing for the two specific protocol requirements we are talking about here. Start small:

We have agreed, as a community, that there is a reasonable default implementation for Equatable.== when certain conditions are met (for value types only at the moment, I believe). Namely, given two values of a type that has only Equatable stored properties, those values are equal if their stored properties are all equal. The author of a new value type who wishes to make her type Equatable but chooses not to implement a custom == then benefits from this default when all stored properties are Equatable.

And precisely what kind of "evidence" am I expected to give? This is a set of features that do not exist yet, I am trying to argue in favour of an explicit end-developer centric opt-in rather than an implicit protocol designer centric one. Yet no-one seems interested in the merits of allowing developers to choose what they want, rather than having implicit behaviours appear potentially unexpectedly.

Both options were examined for Codable and for Equatable/Hashable. The community and core team decided to prefer the current design. At this point, new insights that arise which could not be anticipated at the time of review could prompt revision. However, so far, you have presented arguments already considered during review.

Therefore, your argument reduces to one about which default implementations generally ought or ought not to be provided--that is, that they ought to be provided only when their correctness can be guaranteed for all (rather than almost all) possible conforming types. To which point I sketched a rebuttal above.

If a protocol defines something, and creates a default implementation based only upon those definitions then it must by its very nature be correct. A concrete type may later decided to go further, but that is a feature of the concrete type, not a failure of the protocol itself which can function correctly within the context it created. You want to talk evidence, yet there has been no example given that proves otherwise; thus far only Itai has attempted to do so, but I have already pointed out the flaws with that example.

The simple fact is that a default implementation may either be flawed or not within the context of the protocol itself; but a reflective or synthetic implementation by its very nature goes beyond what the protocol defines and so is automatically flawed because as it does not rely on the end-developer to confirm correctness, not when provided implicitly at least.

Again, if it applies generally, it must apply specifically. What is "automatically flawed" about the very reasonable synthesized default implementation of ==?

And all of this continues to be a side-issue to the fact that in the specific case of Equatable/Hashable, which thus far has gone ignored, is that bolting this on retroactively to an existing protocol hides bugs. The issue of reflective default implementations is less of a concern on very clearly and well defined new protocols, though I still prefer more, rather than less, control, but in the specific case of existing protocols this fucking about with behaviours is reckless and foolish in the extreme, yet no-one on the core teams seems willing or able to justify it, which only opens much wider concerns (how am I to have any faith in Swift's development if the core team can't or won't justify the creation of new bugs?).

This has emphatically not gone ignored, as I have myself responded to this point in an earlier thread in which you commented, as well as many others. Crucially, no existing conforming type changes its behavior, as they have all had to implement these requirements themselves. And as I said to you already, the addition of a synthesized default implementation no more "hides bugs" going forward than the addition of a non-synthesized default implementation to an existing protocol, and we do that with some frequency without even Swift Evolution review.

Feel free to a supply a non-synthesised default implementation for Equatable without the use of reflection. Go-on, I'll wait.
You insist on suggesting these are the same thing, yet if you can't provide one then clearly they are not.

That is not the argument. The argument is that they are indistinguishable in the sense that the author of a type who intends to supply a custom implementation but neglects to do so will have a default implementation supplied for them. It is plainly true that this is no more or less likely to happen simply because the default implementation is synthesized.

Put another way, what the proposal about synthesizing implementations for Equatable and Hashable was about can be thought of in two parts: (a) should there be default implementations; and (b) given that it is impossible to write these in Swift, should we use magic? Now, as I said above, adding default implementations isn't (afaik) even considered an API change that requires review on this list. Really, what people were debating was (b), whether it is worth it to implement compiler-supported magic to make these possible. Your disagreement has to do with (a) and not (b).

Wrong. The use of magic in this case produces something else entirely; that's the whole point. It is not the same, otherwise it wouldn't be needed at all. It doesn't matter if it's compiler magic, some external script or a native macro, ultimately they are all doing something with a concrete type that is currently not possible.

And once again; I am not arguing against a default implementation that cuts boilerplate, I am arguing against it being implicit. What I want is to be the one asking for it, because it is not reasonable to assume that just throwing it in there is always going to be fine, because it quite simply is not.

If you have to ask for it, then it's not a default. You *are* against a default implementation.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I think I do understand Haravikk's argument (actually it seems quite
straightforward to me).

An example should be:

struct Foo : Equatable {
    var x: Int
    var cachedLabel: String? = nil

    init(x: Int) {
        self.x = x
    }

    mutating func label() {
        if let label = cachedLabel {
            return label
        }
        let label = calculateLabel()
        cachedLabel = label
        return cachedLabel
    }
}

var foo1 = Foo(x: 1)
var foo2 = Foo(x: 1)
foo1 == foo2 // true
var label = foo1.label()
foo1 == foo2 // now false, due to cachedString being falsely included in
the comparison

The problem is that the developer was not required to implement the
protocol and so might forget it.
The difference to other default implementations is that those use the
protocol itself as building blocks and so are correct with regards to the
protocol's semantics, whereas the synthesized equality reaches deeply into
the private innards of a struct and therefore is much more likely to be
wrong as in the example above.

The example above wouldn't actually get a synthesized ==, because
Optional<T> doesn't conform to Equatable. We can't get that until we get
conditional conformances.

But if we ignore that, we see that the user has opted-in to the synthesized
== by explicitly claiming that their type conforms to a protocol that
contains a default implementation. Their decision—either by intention or
forgetful omission—to not provide a custom implementation means that the
default behavior will be used, as would any default method implementation.

I don't find the argument that "someone might say that their type conforms
to Equatable to shut the compiler up elsewhere and then forget to fill in
the implementation" very compelling—that's clearly programmer error. The
same could be true for any default method. I could say that my type
conforms to protocol X which comes with a default implementation of foo()
that is inappropriate for my type for some reason and then forget to fill
it in later. The argument that Equatable accesses properties that are not
among its requirements doesn't really make a difference—someone could just
as easily design a protocol X with some non-synthesized default method
implementations that are incorrect for some class of types (for example,
maybe there are preconditions on other requirements that can only be
verified at runtime).

If people want to argue that we need to distinguish between synthesized
default implementations and non-synthesized default implementations, it
would be helpful to provide evidence that there is a significantly large
class of bugs that will be difficult to diagnose and resolve because of
synthesized default implementations that warrants the creation of a
*completely new axis of protocol conformance* that will increase the
complexity of the language.

···

On Mon, Sep 11, 2017 at 11:56 AM Thorsten Seitz via swift-evolution < swift-evolution@swift.org> wrote:

Why not just write

     *struct* Foo : *deriving* Equatable {...}

to request the synthesized implementation?

-Thorsten

Am 09.09.2017 um 19:42 schrieb Xiaodi Wu via swift-evolution < > swift-evolution@swift.org>:

On Sat, Sep 9, 2017 at 06:41 Haravikk via swift-evolution < > swift-evolution@swift.org> wrote:

On 9 Sep 2017, at 09:33, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Sat, Sep 9, 2017 at 02:47 Haravikk via swift-evolution < >> swift-evolution@swift.org> wrote:

On 9 Sep 2017, at 02:02, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Fri, Sep 8, 2017 at 4:00 PM, Itai Ferber via swift-evolution < >>> swift-evolution@swift.org> wrote:

On Sep 8, 2017, at 12:46 AM, Haravikk via swift-evolution < >>>> swift-evolution@swift.org> wrote:

On 7 Sep 2017, at 22:02, Itai Ferber <iferber@apple.com> wrote:

protocol Fooable : Equatable { // Equatable is just a simple example
    var myFoo: Int { get }}
extension Fooable {
    static func ==(_ lhs: Self, _ rhs: Self) -> Bool {
        return lhs.myFoo == rhs.myFoo
    }}
struct X : Fooable {
    let myFoo: Int
    let myName: String
    // Whoops, forgot to give an implementation of ==}
print(X(myFoo: 42, myName: "Alice") == X(myFoo: 42, myName: "Bob")) // true

This property is *necessary*, but not *sufficient* to provide a
correct implementation. A default implementation might be able to
*assume* something about the types that it defines, but it does not
necessarily know enough.

Sorry but that's a bit of a contrived example; in this case the
protocol should *not* implement the equality operator if more
information may be required to define equality. It should only be
implemented if the protocol is absolutely clear that .myFoo is the only
part of a Fooable that can or should be compared as equatable, e.g- if a
Fooable is a database record and .myFoo is a primary key, the data could
differ but it would still be a reference to the same record.

To be clear, I'm not arguing that someone can't create a regular
default implementation that also makes flawed assumptions, but that
synthesised/reflective implementations *by their very nature have to*,
as they cannot under every circumstance guarantee correctness when using
parts of a concrete type that they know nothing about.

You can’t argue this both ways:

   - If you’re arguing this on principle, that in order for
   synthesized implementations to be correct, they *must* be able to — *under
   every circumstance* — guarantee correctness, then you have to apply
   the same reasoning to default protocol implementations. Given a default
   protocol implementation, it is possible to come up with a (no matter how
   contrived) case where the default implementation is wrong. Since you’re
   arguing this *on principle*, you cannot reject contrived examples.
   - If you are arguing this *in practice*, then you’re going to have
   to back up your argument with evidence that synthesized examples are more
   often wrong than default implementations. You can’t declare that
   synthesized implementations are *by nature* incorrect but allow
   default implementations to slide because *in practice*, many
   implementations are allowable. There’s a reason why synthesis passed code
   review and was accepted: in the majority of cases, synthesis was deemed to
   be beneficial, and would provide correct behavior. If you are willing to
   say that yes, sometimes default implementations are wrong but overall
   they’re correct, you’re going to have to provide hard evidence to back up
   the opposite case for synthesized implementations. You stated in a previous
   email that "A synthesised/reflective implementation however may
   return a result that is simply incorrect, because it is based on
   assumptions made by the protocol developer, with no input from the
   developer of the concrete type. In this case the developer must override it
   in to provide *correct* behaviour." — if you can back this up with
   evidence (say, taking a survey of a large number of model types and see if
   in the majority of cases synthesized implementation would be incorrect) to
   provide a compelling argument, then this is something that we should in
   that case reconsider.

Well put, and I agree with this position 100%. However, to play devil's
advocate here, let me summarize what I think Haravikk is saying:

I think the "synthesized" part of this is a red herring, if I understand
Haravikk's argument correctly. Instead, it is this:

(1) In principle, it is possible to have a default implementation for a
protocol requirement that produces the correct result--though not
necessarily in the most performant way--for all possible conforming types,
where by conforming we mean that the type respects both the syntactic
requirements (enforced by the compiler) and the semantic requirements
(which may not necessarily be enforceable by the compiler) of the protocol
in question.

(2) However, there exist *some* requirements that, by their very nature,
cannot have default implementations which are guaranteed to produce the
correct result for all conforming types. In Haravikk's view, no default
implementations should be provided in these cases. (I don't necessarily
subscribe to this view in absolute terms, but for the sake of argument
let's grant this premise.)

(3) Equatable, Hashable, and Codable requirements are, by their very
nature, such requirements that cannot have default implementations
guaranteed to be correct for all conforming types. Therefore, they should
not have a default implementation. It just so happens that a default
implementation cannot currently be written in Swift itself and must be
synthesized, but Haravikk's point is that even if they could be written in
native Swift through a hypothetical reflection facility, they should not
be, just as many other protocol requirements currently could have default
implementations written in Swift but should not have them because they
cannot be guaranteed to produce the correct result.

My response to this line of argumentation is as follows:

For any open protocol (i.e., a protocol for which the universe of
possible conforming types cannot be enumerated a priori by the protocol
designer) worthy of being a protocol by the Swift standard ("what useful
thing can you do with such a protocol that you could not without?"), any
sufficiently interesting requirement (i.e., one for which user ergonomics
would measurably benefit from a default implementation) either cannot have
a universally guaranteed correct implementation or has an implementation
which is also going to be the most performant one (which can therefore be a
non-overridable protocol extension method rather than an overridable
protocol requirement with a default implementation).

You're close, but still missing key points:

   1. I am not arguing that features like these should *not* be
   provided, but that they should *not* be provided implicitly, and
   that the developer should actually be allowed to request them. That is
   exactly what this proposal is about, yet no matter what I say everyone
   seems to be treating me like I'm against these features entirely; *I
   am not*.

You are entirely against Equatable having a default implementation for
==. This is unequivocally stated. Others favor such a default
implementation and feel that in the absence of a way to spell this in Swift
itself, it should be magic for the time being. For the purposes of this
argument it really is not pertinent that you are not also against something
else; you're asking us to discuss why you are against a particular thing
that others are for.

FFS, how much clearer can I make this? *I AM NOT AGAINST THE FEATURE.*

What I am against is the way in which it is being provided implicitly
rather than explicitly, in particular as a retroactive change to existing
protocols in a way that introduces potential for bugs that are currently
impossible, but also in general.

You are against a default implementation for ==, i.e. an implementation
that is provided for you if you conform a type to the protocol and do
nothing else ("implicitly rather than explicitly"), and you are against the
default implementation being on the existing protocol Equatable
("retroactive change"). So, to summarize, what you are against is precisely
a default implementation for the == requirement on Equatable.

This is the topic of discussion here; I am attempting to convince you that
you should be for rather than against these things.

As repeatedly answered by others, nothing here is specific to synthesized
default implementations, as more powerful reflection will gradually allow
them to be non-synthesised.

And as repeatedly stated by me; I am not treating synthesised vs.
run-time reflection any differently, I specifically included both in the
original proposal.

As pointed out very cogently by Itai, you assert but offer no evidence,
either in principle or empirically, that going too far by reflection is
worse than going not far enough without reflection in terms of likelihood
of a default implementation being inappropriate for conforming types.

As I have also repeatedly pointed out it is not an issue of "not going
far enough" vs. "going too far"; if a default implementation lacks
information then it should not be provided, doing so regardless is a flaw
in the protocol design and not something that this proposal attempts to
address (as such a thing is likely impossible).

Right, one must consider the semantics of the specific protocol
requirement and ask whether a reasonable default can be provided for it.

Reflective implementations *necessarily* go too far, because they

literally know *nothing* about the concrete type with any certainty,
except for the properties that are defined in the protocol (which do not
require reflection or synthesis in the first place).

I am confused why you are trying to argue in general terms about the
universe of all possible default implementations that use reflection. This
is necessarily a more difficult argument to make, and if it is to be
convincing for all default implementations it must also be convincing for
the two specific protocol requirements we are talking about here. Start
small:

We have agreed, as a community, that there is a reasonable default
implementation for Equatable.== when certain conditions are met (for value
types only at the moment, I believe). Namely, given two values of a type
that has only Equatable stored properties, those values are equal if their
stored properties are all equal. The author of a new value type who wishes
to make her type Equatable but chooses not to implement a custom == then
benefits from this default when all stored properties are Equatable.

And precisely what kind of "evidence" am I expected to give? This is a set

of features that *do not exist yet*, I am trying to argue in favour of
an explicit end-developer centric opt-in rather than an implicit protocol
designer centric one. Yet no-one seems interested in the merits of allowing
developers to choose what they want, rather than having implicit behaviours
appear potentially unexpectedly.

Both options were examined for Codable and for Equatable/Hashable. The
community and core team decided to prefer the current design. At this
point, new insights that arise which could not be anticipated at the time
of review could prompt revision. However, so far, you have presented
arguments already considered during review.

Therefore, your argument reduces to one about which default

implementations generally ought or ought not to be provided--that is, that
they ought to be provided only when their correctness can be guaranteed for
all (rather than almost all) possible conforming types. To which point I
sketched a rebuttal above.

If a protocol defines something, and creates a default implementation
based only upon those definitions then it must by its very nature be
correct. A concrete type may later decided to go further, but that is a
feature of the concrete type, not a failure of the protocol itself which
can function correctly within the context it created. You want to talk
evidence, yet there has been no example given that proves otherwise; thus
far only Itai has attempted to do so, but I have already pointed out the
flaws with that example.

The simple fact is that a default implementation may either be flawed or
not within the context of the protocol itself; but a reflective or
synthetic implementation by its very nature goes beyond what the protocol
defines and so is automatically flawed because as it does not rely on the
end-developer to confirm correctness, not when provided implicitly at least.

Again, if it applies generally, it must apply specifically. What is
"automatically flawed" about the very reasonable synthesized default
implementation of ==?

And all of this continues to be a side-issue to the fact that in the

specific case of Equatable/Hashable, which thus far has gone ignored, is
that bolting this on retroactively to an existing protocol *hides bugs*.
The issue of reflective default implementations is less of a concern on
very clearly and well defined *new* protocols, though I still prefer
more, rather than less, control, but in the specific case of *existing* protocols
this fucking about with behaviours is reckless and foolish in the extreme,
yet no-one on the core teams seems willing or able to justify it, which
only opens much wider concerns (how am I to have any faith in Swift's
development if the core team can't or won't justify the creation of new
bugs?).

This has emphatically not gone ignored, as I have myself responded to
this point in an earlier thread in which you commented, as well as many
others. Crucially, no existing conforming type changes its behavior, as
they have all had to implement these requirements themselves. And as I said
to you already, the addition of a synthesized default implementation no
more "hides bugs" going forward than the addition of a non-synthesized
default implementation to an existing protocol, and we do that with some
frequency without even Swift Evolution review.

Feel free to a supply a non-synthesised default implementation for
Equatable without the use of reflection. Go-on, I'll wait.
You insist on suggesting these are the same thing, yet if you can't
provide one then clearly they are not.

That is not the argument. The argument is that they are indistinguishable
in the sense that the author of a type who intends to supply a custom
implementation but neglects to do so will have a default implementation
supplied for them. It is plainly true that this is no more or less likely
to happen simply because the default implementation is synthesized.

Put another way, what the proposal about synthesizing implementations for

Equatable and Hashable was about can be thought of in two parts: (a) should
there be default implementations; and (b) given that it is impossible to
write these in Swift, should we use magic? Now, as I said above, adding
default implementations isn't (afaik) even considered an API change that
requires review on this list. Really, what people were debating was (b),
whether it is worth it to implement compiler-supported magic to make these
possible. Your disagreement has to do with (a) and not (b).

Wrong. The use of magic in this case produces something else entirely;
that's the whole point. It is *not the same*, otherwise it wouldn't be
needed at all. It doesn't matter if it's compiler magic, some external
script or a native macro, ultimately they are all doing something with a
concrete type that is currently not possible.

And once again; *I am not arguing against a default implementation that
cuts boilerplate*, I am arguing against it being implicit. What I want
is to be the one asking for it, because it is not reasonable to assume that
just throwing it in there is always going to be fine, because it quite
simply is not.

If you have to ask for it, then it's not a default. You *are* against a
default implementation.

_______________________________________________

swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution