[Proposal] Explicit Synthetic Behaviour

[Haravikk3 (Haravikk)]

Some of you will have seen my impassioned pleas on the synthesised Equatable/Hashable thread against implementing implicit synthesised behaviour on protocols that must, by necessity, make assumptions about a concrete type that could be incorrect.

For those that haven't, the concept of synthetic behaviour in discussion here is essentially any kind of default behaviour for a protocol that is automatically generated based upon the concrete type itself (rather than just what the protocol defines). Currently this refers to compiler magic as proposed for Codable, Equatable and Hashable, but also includes the reflection API and any future native macro support for Swift. Using any of these to implement default methods for protocols should IMO be made explicit to developers using any such protocol so that they can specifically opt-in to the behaviour only if they want to and understand what it does.

This proposal idea is essentially for a new attribute @synthetic (name is up for debate). This attribute is required for any default implementation that includes reflective type compiler magic, use of the reflection API against `self` or, in future, any native Swift macros within the method (possibly limited to specific features, will depend on the macro language and its capabilities). If a default method does not have this attribute, then the compiler will produce an error with the appropriate fix-it. For convenience this attribute can be applied to any extension block or even a protocol definition in order to mark all methods in that block/type as synthetic, though it's worth noting that doing so will prevent these default implementations from being provided if they don't actually need this attribute.

Basically the intention is that any protocol default implementation that requires more knowledge about the concrete type than merely what the protocol (and its parents) provide direct access to, must be marked as synthetic.

To use the synthetic behaviour of a protocol, developers must then use the @synthetic keyword when conforming to it, explicitly indicating that they want the extra behaviours rather than implementing the method(s) for themselves. To ignore the synthetic behaviour (and thus require some kind of manual implementation of methods as normal), simply omit the keyword:

  struct Foo : @synthetic Equatable { var someData:String }
    // Type fully conforms to Equatable using synthetic behaviour (equatable properties must be equal)
  struct Foo : Equatable { var someData:String }
    // Error due to unimplemented methods, but offers @synthetic as a fix-it if all unimplemented methods are @synthetic

It is possible that the attribute could be expanded to have parameters, allowing for synthetic conformance only on specific methods, but I'm unsure if that'd be the best way to do it, or how likely that is to be needed.

With this kind of explicit declaration it becomes obvious within code when a developer is specifically choosing to benefit from synthetic behaviour; this hopefully makes it more likely that a developer will fully consider what the implications of this may be, rather than doing it accidentally. The idea in part is to distinguish such types as having separate protocol and synthesised behaviour, where conformance without the @synthetic attribute specifically requires that all protocol requirements be met in full, and that any default behaviour is implemented only on the basis of the protocol itself, while adding @synthetic identifies that more invasive automated behaviour is permitted/requested.

At this stage I don't think there should be much of an impact for existing code; as far as I can tell it should only affect any protocols that happen to be using Mirror(reflecting: self) for some reason within a default implementation, which I can't imagine represents a huge subsection of existing code, and the fix is the simple addition of an attribute.

Anyway, this is basically just a rough dump of the ideas for how the synthesised Codable, Equatable and Hashable behaviours (and anything else anyone can think of) should be changed before Swift 4 is released. I'm hoping for feedback before I make a formal proposal as I don't have a lot of time at the moment, so am looking to do the more structured document once stuff has been hammered out a bit.

- Haravikk

[bob]

I take issue with the fact that this problem is no different from accidentally gaining the default inheritance of *any* member required by a protocol and implemented in an extension of that protocol. The fact that in this case conformance is synthesized by the compiler instead of written in source code somewhere is immaterial; in principle, nothing is (was?) stopping the same default implementation from being implemented with a Mirror instead of the current approach.

I still think that the role keyword proposal is the best solution to this problem proposed so far. [swift-evolution] [Pitch] Introducing role keywords to reduce hard-to-find bugs

···

On Sep 5, 2017, at 4:02 PM, Haravikk via swift-evolution <swift-evolution@swift.org> wrote:

Some of you will have seen my impassioned pleas on the synthesised Equatable/Hashable thread against implementing implicit synthesised behaviour on protocols that must, by necessity, make assumptions about a concrete type that could be incorrect.

For those that haven't, the concept of synthetic behaviour in discussion here is essentially any kind of default behaviour for a protocol that is automatically generated based upon the concrete type itself (rather than just what the protocol defines). Currently this refers to compiler magic as proposed for Codable, Equatable and Hashable, but also includes the reflection API and any future native macro support for Swift. Using any of these to implement default methods for protocols should IMO be made explicit to developers using any such protocol so that they can specifically opt-in to the behaviour only if they want to and understand what it does.

This proposal idea is essentially for a new attribute @synthetic (name is up for debate). This attribute is required for any default implementation that includes reflective type compiler magic, use of the reflection API against `self` or, in future, any native Swift macros within the method (possibly limited to specific features, will depend on the macro language and its capabilities). If a default method does not have this attribute, then the compiler will produce an error with the appropriate fix-it. For convenience this attribute can be applied to any extension block or even a protocol definition in order to mark all methods in that block/type as synthetic, though it's worth noting that doing so will prevent these default implementations from being provided if they don't actually need this attribute.

Basically the intention is that any protocol default implementation that requires more knowledge about the concrete type than merely what the protocol (and its parents) provide direct access to, must be marked as synthetic.

To use the synthetic behaviour of a protocol, developers must then use the @synthetic keyword when conforming to it, explicitly indicating that they want the extra behaviours rather than implementing the method(s) for themselves. To ignore the synthetic behaviour (and thus require some kind of manual implementation of methods as normal), simply omit the keyword:

  struct Foo : @synthetic Equatable { var someData:String }
    // Type fully conforms to Equatable using synthetic behaviour (equatable properties must be equal)
  struct Foo : Equatable { var someData:String }
    // Error due to unimplemented methods, but offers @synthetic as a fix-it if all unimplemented methods are @synthetic

It is possible that the attribute could be expanded to have parameters, allowing for synthetic conformance only on specific methods, but I'm unsure if that'd be the best way to do it, or how likely that is to be needed.

With this kind of explicit declaration it becomes obvious within code when a developer is specifically choosing to benefit from synthetic behaviour; this hopefully makes it more likely that a developer will fully consider what the implications of this may be, rather than doing it accidentally. The idea in part is to distinguish such types as having separate protocol and synthesised behaviour, where conformance without the @synthetic attribute specifically requires that all protocol requirements be met in full, and that any default behaviour is implemented only on the basis of the protocol itself, while adding @synthetic identifies that more invasive automated behaviour is permitted/requested.

At this stage I don't think there should be much of an impact for existing code; as far as I can tell it should only affect any protocols that happen to be using Mirror(reflecting: self) for some reason within a default implementation, which I can't imagine represents a huge subsection of existing code, and the fix is the simple addition of an attribute.

Anyway, this is basically just a rough dump of the ideas for how the synthesised Codable, Equatable and Hashable behaviours (and anything else anyone can think of) should be changed before Swift 4 is released. I'm hoping for feedback before I make a formal proposal as I don't have a lot of time at the moment, so am looking to do the more structured document once stuff has been hammered out a bit.

- Haravikk
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[benrimmington]

Swift 4, Xcode 9 and iOS 11 will **probably** be released on September 12:

<https://www.apple.com/apple-events/september-2017/&gt;

All previous major versions have also been released in mid-September:

<https://github.com/apple/swift/blob/master/CHANGELOG.md&gt;

There isn't enough time to propose, review and implement @synthetic (AFAIK).

-- Ben

···

On 5 Sep 2017, at 21:02, Haravikk wrote:

Anyway, this is basically just a rough dump of the ideas for how the synthesised Codable, Equatable and Hashable behaviours (and anything else anyone can think of) should be changed before Swift 4 is released.

[Haravikk3 (Haravikk)]

I take issue with the fact that this problem is no different from accidentally gaining the default inheritance of *any* member required by a protocol and implemented in an extension of that protocol. The fact that in this case conformance is synthesized by the compiler instead of written in source code somewhere is immaterial; in principle, nothing is (was?) stopping the same default implementation from being implemented with a Mirror instead of the current approach.

This is why I'm proposing that Mirrors of `self` should likewise require the new attribute; while I realise it may be possible to sneak self-reflection in somehow regardless by tricking the compiler, this should at least pick up the most obvious cases. A more complete solution could possibly just consider all use of reflection to be synthetic, requiring methods callable from protocol extensions (global functions, static methods etc.) to use the attribute as well, so that the compiler can detect any function call with `self` that could potentially result in reflective behaviour.

The issue here isn't that there might be other ways to do it (these can be addressed), it's that all methods of doing this should require developers to explicitly opt-in, otherwise it leads to potential bugs and/or unwanted behaviour. It's also IMO a gross overreach for protocols to begin with, and sets a dangerous precedent for a language that's supposed to be about safety and prevention of bugs, especially when in the case of Equatable and Hashable it will actually hide bugs that are currently caught.

As a general rule I would argue that Mirrors should almost never be used for any purpose, except perhaps debugging; in production code they can lead to subtle and misleading problems, not to mention any performance impacts. Even for things like serialising types, it is not a desirable way to do it, and should only be used as a last resort because of missing features in Swift.

I still think that the role keyword proposal is the best solution to this problem proposed so far. [swift-evolution] [Pitch] Introducing role keywords to reduce hard-to-find bugs

While I support the idea of these role keywords, they don't solve all of the same problems. Consider for example a missing protocol requirement; currently it is caught if a protocol cannot offer a default implementation for it, however, in the presence of synthetic behaviour it is possible that the requirement is met, but by a method that will not work as desired.

The main issue here is that we're talking about methods that appear like default implementations, but go well beyond what the protocol itself defines; when you start delving into concrete types from within a protocol you are making assumptions about that concrete type that simply cannot be guaranteed. Any mistake or omission by a developer could lead to behaviour they do not want, and that may not be easy to debug, and in the case of Equatable and Hashable is a potential bug that is currently impossible.

···

On 6 Sep 2017, at 01:36, Robert Bennett <rltbennett@icloud.com> wrote:

On Sep 5, 2017, at 4:02 PM, Haravikk via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
Some of you will have seen my impassioned pleas on the synthesised Equatable/Hashable thread against implementing implicit synthesised behaviour on protocols that must, by necessity, make assumptions about a concrete type that could be incorrect.

For those that haven't, the concept of synthetic behaviour in discussion here is essentially any kind of default behaviour for a protocol that is automatically generated based upon the concrete type itself (rather than just what the protocol defines). Currently this refers to compiler magic as proposed for Codable, Equatable and Hashable, but also includes the reflection API and any future native macro support for Swift. Using any of these to implement default methods for protocols should IMO be made explicit to developers using any such protocol so that they can specifically opt-in to the behaviour only if they want to and understand what it does.

This proposal idea is essentially for a new attribute @synthetic (name is up for debate). This attribute is required for any default implementation that includes reflective type compiler magic, use of the reflection API against `self` or, in future, any native Swift macros within the method (possibly limited to specific features, will depend on the macro language and its capabilities). If a default method does not have this attribute, then the compiler will produce an error with the appropriate fix-it. For convenience this attribute can be applied to any extension block or even a protocol definition in order to mark all methods in that block/type as synthetic, though it's worth noting that doing so will prevent these default implementations from being provided if they don't actually need this attribute.

Basically the intention is that any protocol default implementation that requires more knowledge about the concrete type than merely what the protocol (and its parents) provide direct access to, must be marked as synthetic.

To use the synthetic behaviour of a protocol, developers must then use the @synthetic keyword when conforming to it, explicitly indicating that they want the extra behaviours rather than implementing the method(s) for themselves. To ignore the synthetic behaviour (and thus require some kind of manual implementation of methods as normal), simply omit the keyword:

  struct Foo : @synthetic Equatable { var someData:String }
    // Type fully conforms to Equatable using synthetic behaviour (equatable properties must be equal)
  struct Foo : Equatable { var someData:String }
    // Error due to unimplemented methods, but offers @synthetic as a fix-it if all unimplemented methods are @synthetic

It is possible that the attribute could be expanded to have parameters, allowing for synthetic conformance only on specific methods, but I'm unsure if that'd be the best way to do it, or how likely that is to be needed.

With this kind of explicit declaration it becomes obvious within code when a developer is specifically choosing to benefit from synthetic behaviour; this hopefully makes it more likely that a developer will fully consider what the implications of this may be, rather than doing it accidentally. The idea in part is to distinguish such types as having separate protocol and synthesised behaviour, where conformance without the @synthetic attribute specifically requires that all protocol requirements be met in full, and that any default behaviour is implemented only on the basis of the protocol itself, while adding @synthetic identifies that more invasive automated behaviour is permitted/requested.

At this stage I don't think there should be much of an impact for existing code; as far as I can tell it should only affect any protocols that happen to be using Mirror(reflecting: self) for some reason within a default implementation, which I can't imagine represents a huge subsection of existing code, and the fix is the simple addition of an attribute.

Anyway, this is basically just a rough dump of the ideas for how the synthesised Codable, Equatable and Hashable behaviours (and anything else anyone can think of) should be changed before Swift 4 is released. I'm hoping for feedback before I make a formal proposal as I don't have a lot of time at the moment, so am looking to do the more structured document once stuff has been hammered out a bit.

[Haravikk3 (Haravikk)]

I take issue with the fact that this problem is no different from accidentally gaining the default inheritance of *any* member required by a protocol and implemented in an extension of that protocol. The fact that in this case conformance is synthesized by the compiler instead of written in source code somewhere is immaterial; in principle, nothing is (was?) stopping the same default implementation from being implemented with a Mirror instead of the current approach.

This is why I'm proposing that Mirrors of `self` should likewise require the new attribute; while I realise it may be possible to sneak self-reflection in somehow regardless by tricking the compiler, this should at least pick up the most obvious cases. A more complete solution could possibly just consider all use of reflection to be synthetic, requiring methods callable from protocol extensions (global functions, static methods etc.) to use the attribute as well, so that the compiler can detect any function call with `self` that could potentially result in reflective behaviour.

The issue here isn't that there might be other ways to do it (these can be addressed), it's that all methods of doing this should require developers to explicitly opt-in, otherwise it leads to potential bugs and/or unwanted behaviour. It's also IMO a gross overreach for protocols to begin with, and sets a dangerous precedent for a language that's supposed to be about safety and prevention of bugs, especially when in the case of Equatable and Hashable it will actually hide bugs that are currently caught.

As a general rule I would argue that Mirrors should almost never be used for any purpose, except perhaps debugging; in production code they can lead to subtle and misleading problems, not to mention any performance impacts. Even for things like serialising types, it is not a desirable way to do it, and should only be used as a last resort because of missing features in Swift.

I still think that the role keyword proposal is the best solution to this problem proposed so far. [swift-evolution] [Pitch] Introducing role keywords to reduce hard-to-find bugs

While I support the idea of these role keywords, they don't solve all of the same problems. Consider for example a missing protocol requirement; currently it is caught if a protocol cannot offer a default implementation for it, however, in the presence of synthetic behaviour it is possible that the requirement is met, but by a method that will not work as desired.

The main issue here is that we're talking about methods that appear like default implementations, but go well beyond what the protocol itself defines; when you start delving into concrete types from within a protocol you are making assumptions about that concrete type that simply cannot be guaranteed. Any mistake or omission by a developer could lead to behaviour they do not want, and that may not be easy to debug, and in the case of Equatable and Hashable is a potential bug that is currently impossible.

···

On 6 Sep 2017, at 01:36, Robert Bennett <rltbennett@icloud.com <mailto:rltbennett@icloud.com>> wrote:

On Sep 5, 2017, at 4:02 PM, Haravikk via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
Some of you will have seen my impassioned pleas on the synthesised Equatable/Hashable thread against implementing implicit synthesised behaviour on protocols that must, by necessity, make assumptions about a concrete type that could be incorrect.

For those that haven't, the concept of synthetic behaviour in discussion here is essentially any kind of default behaviour for a protocol that is automatically generated based upon the concrete type itself (rather than just what the protocol defines). Currently this refers to compiler magic as proposed for Codable, Equatable and Hashable, but also includes the reflection API and any future native macro support for Swift. Using any of these to implement default methods for protocols should IMO be made explicit to developers using any such protocol so that they can specifically opt-in to the behaviour only if they want to and understand what it does.

This proposal idea is essentially for a new attribute @synthetic (name is up for debate). This attribute is required for any default implementation that includes reflective type compiler magic, use of the reflection API against `self` or, in future, any native Swift macros within the method (possibly limited to specific features, will depend on the macro language and its capabilities). If a default method does not have this attribute, then the compiler will produce an error with the appropriate fix-it. For convenience this attribute can be applied to any extension block or even a protocol definition in order to mark all methods in that block/type as synthetic, though it's worth noting that doing so will prevent these default implementations from being provided if they don't actually need this attribute.

Basically the intention is that any protocol default implementation that requires more knowledge about the concrete type than merely what the protocol (and its parents) provide direct access to, must be marked as synthetic.

To use the synthetic behaviour of a protocol, developers must then use the @synthetic keyword when conforming to it, explicitly indicating that they want the extra behaviours rather than implementing the method(s) for themselves. To ignore the synthetic behaviour (and thus require some kind of manual implementation of methods as normal), simply omit the keyword:

  struct Foo : @synthetic Equatable { var someData:String }
    // Type fully conforms to Equatable using synthetic behaviour (equatable properties must be equal)
  struct Foo : Equatable { var someData:String }
    // Error due to unimplemented methods, but offers @synthetic as a fix-it if all unimplemented methods are @synthetic

It is possible that the attribute could be expanded to have parameters, allowing for synthetic conformance only on specific methods, but I'm unsure if that'd be the best way to do it, or how likely that is to be needed.

With this kind of explicit declaration it becomes obvious within code when a developer is specifically choosing to benefit from synthetic behaviour; this hopefully makes it more likely that a developer will fully consider what the implications of this may be, rather than doing it accidentally. The idea in part is to distinguish such types as having separate protocol and synthesised behaviour, where conformance without the @synthetic attribute specifically requires that all protocol requirements be met in full, and that any default behaviour is implemented only on the basis of the protocol itself, while adding @synthetic identifies that more invasive automated behaviour is permitted/requested.

At this stage I don't think there should be much of an impact for existing code; as far as I can tell it should only affect any protocols that happen to be using Mirror(reflecting: self) for some reason within a default implementation, which I can't imagine represents a huge subsection of existing code, and the fix is the simple addition of an attribute.

Anyway, this is basically just a rough dump of the ideas for how the synthesised Codable, Equatable and Hashable behaviours (and anything else anyone can think of) should be changed before Swift 4 is released. I'm hoping for feedback before I make a formal proposal as I don't have a lot of time at the moment, so am looking to do the more structured document once stuff has been hammered out a bit.

[Brent_Royal-Gordon]

"Use of the reflection API against `self`"? `String(describing:)` and `String(reflecting:)` sometimes do that.

I see zero justification for having @synthetic cover all of these random things, but not ordinary default implementations—they have the same amount of dangerous implicitness. And I think we agree that using default methods only when @synthetic is specified is a non-starter, if only because default methods are a backwards compatibility tool for library evolution.

We should show these pseudo-default methods in the generated interface for the module and be done with it. They only seem scary right now because people haven't had time to grow comfortable with them.

···

On Sep 5, 2017, at 1:02 PM, Haravikk via swift-evolution <swift-evolution@swift.org> wrote:

This proposal idea is essentially for a new attribute @synthetic (name is up for debate). This attribute is required for any default implementation that includes reflective type compiler magic, use of the reflection API against `self` or, in future, any native Swift macros within the method (possibly limited to specific features, will depend on the macro language and its capabilities).

--
Brent Royal-Gordon
Architechies

[Eagle_Offshore]

I'm just going to toss in that you and I apparently have diametrically opposed needs and belief systems about what makes for a good programming language. I believe the exact opposite.

After 20+ years of Cocoa development, I rely on these features heavily and consider any language that lacks them to be more or less "dead" where I consider languages that have them to be "self aware" and "lively".

For instance, I think relying on the compiler to generate special code to implement Codable rather than just exposing the meta facilities required to do introspection is taking the long way around rather than the short cut.

So add my vote for powerful reflection capabilities.

···

On Sep 6, 2017, at 1:32 AM, Haravikk via swift-evolution <swift-evolution@swift.org> wrote:

As a general rule I would argue that Mirrors should almost never be used for any purpose, except perhaps debugging; in production code they can lead to subtle and misleading problems, not to mention any performance impacts. Even for things like serialising types, it is not a desirable way to do it, and should only be used as a last resort because of missing features in Swift.

[Haravikk3 (Haravikk)]

Sorry I think my remark was unclear; my comment there was specifically about the use of run-time reflection, which I believe usually indicates a failure of a language to provide other, more appropriate features (like Codable in fact). I'm not against the features where they make sense and are implemented in well, I was just pointing out that the use of run-time reflection in code is usually a sign of a last resort.

That's not really important though; the issue I'm trying to raise is that when those, and similar features, are used in synthesised behaviour (default implementations based upon the concrete type), that these behaviours should be opted into explicitly, otherwise they open up potential for all kinds of bugs, even when the assumptions being made about the concrete type are simple such as in the case for Equatable/Hashable. There's just too much potential for this kind of reflective protocol implementation to overreach; to me it feels very much like going into a restaurant and the waiter coming across and force-feeding me something I don't want instead of taking my order.

···

On 6 Sep 2017, at 16:26, Eagle Offshore <eagleoffshore@mac.com> wrote:

On Sep 6, 2017, at 1:32 AM, Haravikk via swift-evolution <swift-evolution@swift.org> wrote:
As a general rule I would argue that Mirrors should almost never be used for any purpose, except perhaps debugging; in production code they can lead to subtle and misleading problems, not to mention any performance impacts. Even for things like serialising types, it is not a desirable way to do it, and should only be used as a last resort because of missing features in Swift.

I'm just going to toss in that you and I apparently have diametrically opposed needs and belief systems about what makes for a good programming language. I believe the exact opposite.

After 20+ years of Cocoa development, I rely on these features heavily and consider any language that lacks them to be more or less "dead" where I consider languages that have them to be "self aware" and "lively".

For instance, I think relying on the compiler to generate special code to implement Codable rather than just exposing the meta facilities required to do introspection is taking the long way around rather than the short cut.

So add my vote for powerful reflection capabilities.

[Haravikk3 (Haravikk)]

This proposal idea is essentially for a new attribute @synthetic (name is up for debate). This attribute is required for any default implementation that includes reflective type compiler magic, use of the reflection API against `self` or, in future, any native Swift macros within the method (possibly limited to specific features, will depend on the macro language and its capabilities).

"Use of the reflection API against `self`"? `String(describing:)` and `String(reflecting:)` sometimes do that.

I see zero justification for having @synthetic cover all of these random things, but not ordinary default implementations—they have the same amount of dangerous implicitness.

Actually they don't; the problem here is that through reflection you are accessing and manipulating concrete types. A non-reflective default implementation only has access to what the protocol itself has defined. The synthetic alternatives are instead diving into parts of a concrete type that may have nothing to do with the protocol at all, and must therefore make assumptions that cannot be guaranteed to be correct, this is what makes them dangerous.

the issue I'm trying to raise is that when those, and similar features, are used in synthesised behaviour (default implementations based upon the concrete type), that these behaviours should be opted into explicitly, otherwise they open up potential for all kinds of bugs, even when the assumptions being made about the concrete type are simple such as in the case for Equatable/Hashable. There's just too much potential for this kind of reflective protocol implementation to overreach; to me it feels very much like going into a restaurant and the waiter coming across and force-feeding me something I don't want instead of taking my order.

I might suggest that instead it is like you have gone into a pizza shop and said, “I’d like a large veggie pizza please.” And they made you a pizza with their standard dough and their standard sauce and their standard cheese and their standard selection of vegetables.

Actually I don't think that's quite it either; to strain the analogy even further, I'd say it's more like going into a pizza shop and saying "I'd like a pizza" and the staff looking at you and deciding you look like a vegetarian and giving you a vegetarian pizza.

The crux of the issue here are the assumptions that are being made; for a standard default implementation there are no assumptions, because you're operating on the basis of methods and properties that you yourself have defined as the protocol creator that, where properly implemented, have precisely defined requirements, behaviours etc. When you're doing it with some form of compile-time or run-time reflection however you're messing around with parts of a concrete type that the protocol itself doesn't actually know anything about with any certainty.

It's the same reason that run-time reflection isn't something you should ever want to do, because while it might work with all of the types you test directly, if it supports arbitrary types (which it'd need to, or there'd be no point to reflecting) then there's always the risk of encountering a type where some part of it doesn't match the assumptions that you've made. Of course there are use-cases where this may not matter, e.g- if you're just dumping data from a type and don't really care if you're storing stuff that isn't important, but in other cases such as Equatable and Hashable it can make a difference, as it can drastically affect behaviour when those assumptions fail.

···

On 7 Sep 2017, at 00:11, Brent Royal-Gordon <brent@architechies.com> wrote:

On Sep 5, 2017, at 1:02 PM, Haravikk via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 6 Sep 2017, at 23:43, Nevin Brackett-Rozinsky <nevin.brackettrozinsky@gmail.com> wrote:

On Wed, Sep 6, 2017 at 5:42 PM, Haravikk via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On 7 Sep 2017, at 05:00, Andrew Thompson <mrwerdo331@me.com> wrote:
Perhaps we could find a solution by meeting in the middle. Introduce a compiler flag that will disable the automatic synthesis and revert to the old behaviour (i.e. swiftc main.swift —disable-automatic-synthesis )

Thanks for the suggestion, but that's a bit too all-or-nothing; I realise I might seem very negative but to be clear, I do want these synthesised features, I just don't want them in their current form. What I want is for them to be explicitly opted into where I, as the developer, need them to be, so that I'm in absolute, unambiguous control of when and where they are used.

I do absolutely support features that eliminate tedious boiler-plate, I just think that doing so implicitly in invasive ways is not the right way to do it.

[Nevin]

I might suggest that instead it is like you have gone into a pizza shop and
said, “I’d like a large veggie pizza please.” And they made you a pizza
with their standard dough and their standard sauce and their standard
cheese and their standard selection of vegetables.

If you wanted some other dough or sauce or cheese or toppings you would
have ordered it. And the fact is you *did* order a veggie pizza (ie.
conformed to a certain protocol). It’s not like they brought you something
you didn’t order. You ordered what you wanted, and that’s what they brought
you.

And for those times when you *do* want to customize your order, that is
perfectly fine too.

Nevin

···

On Wed, Sep 6, 2017 at 5:42 PM, Haravikk via swift-evolution < swift-evolution@swift.org> wrote:

the issue I'm trying to raise is that when those, and similar features,
are used in synthesised behaviour (default implementations based upon the
concrete type), that these behaviours should be opted into explicitly,
otherwise they open up potential for all kinds of bugs, even when the
assumptions being made about the concrete type are simple such as in the
case for Equatable/Hashable. There's just too much potential for this kind
of reflective protocol implementation to overreach; to me it feels very
much like going into a restaurant and the waiter coming across and
force-feeding me something I don't want instead of taking my order.

[gwendal.roue (Gwendal Roué)]

Hello,

I'm interested in this debate because I've been bitten by automatic synthesis recently.

I'm reading your discussion, but I don't have a strong opinion. I only have an anecdote: implicit, automatic, and unavoidable code synthesis code can make it difficult to write some DSLs (aka Domain-Specific Languages).

I did stumble against undesired Equatable synthesis while developping a library[1] that generates SQL snippets. In this library, the `==` operator does not return a Bool: it returns an SQL expression:

  // SELECT * FROM players WHERE bestScore = 1000
  Player.filter(bestScore == 1000)

Since the library is free to define == as an operator that returns an SQL expression, this works quite well. Even when both operands have the same type:

  // SELECT * FROM players WHERE lastScore = bestScore
  Player.filter(lastScore == bestScore)

However, as soon as the type is (also) Equatable, an ambiguity occurs, and the DSL is basically broken:

  Player.filter(lastScore == bestScore) // which == please?

In this case, the == from synthesized Equatable conformance is not welcome at all. It prevents the library from controlling the return type of the == operator. Equatable conformance is unavoidable for enums based on String or generally any raw value type that adopts Equatable. The consequence is that my library can't allow its users to define an enum of table columns.

This is not a deal breaker. Everybody can live with this little caveat, and I guess I'm the only one who wishes things had been more *consistent*. But still: this story helps realizing that code synthesis can bite in plenty of unexpected ways.

Thanks for reading,
Gwendal Roué
[1] GitHub - groue/GRDB.swift: A toolkit for SQLite databases, with a focus on application development

···

Le 7 sept. 2017 à 12:20, Haravikk via swift-evolution <swift-evolution@swift.org> a écrit :

On 7 Sep 2017, at 00:11, Brent Royal-Gordon <brent@architechies.com <mailto:brent@architechies.com>> wrote:

On Sep 5, 2017, at 1:02 PM, Haravikk via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

This proposal idea is essentially for a new attribute @synthetic (name is up for debate). This attribute is required for any default implementation that includes reflective type compiler magic, use of the reflection API against `self` or, in future, any native Swift macros within the method (possibly limited to specific features, will depend on the macro language and its capabilities).

"Use of the reflection API against `self`"? `String(describing:)` and `String(reflecting:)` sometimes do that.

I see zero justification for having @synthetic cover all of these random things, but not ordinary default implementations—they have the same amount of dangerous implicitness.

Actually they don't; the problem here is that through reflection you are accessing and manipulating concrete types. A non-reflective default implementation only has access to what the protocol itself has defined. The synthetic alternatives are instead diving into parts of a concrete type that may have nothing to do with the protocol at all, and must therefore make assumptions that cannot be guaranteed to be correct, this is what makes them dangerous.

On 6 Sep 2017, at 23:43, Nevin Brackett-Rozinsky <nevin.brackettrozinsky@gmail.com <mailto:nevin.brackettrozinsky@gmail.com>> wrote:

On Wed, Sep 6, 2017 at 5:42 PM, Haravikk via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
the issue I'm trying to raise is that when those, and similar features, are used in synthesised behaviour (default implementations based upon the concrete type), that these behaviours should be opted into explicitly, otherwise they open up potential for all kinds of bugs, even when the assumptions being made about the concrete type are simple such as in the case for Equatable/Hashable. There's just too much potential for this kind of reflective protocol implementation to overreach; to me it feels very much like going into a restaurant and the waiter coming across and force-feeding me something I don't want instead of taking my order.

I might suggest that instead it is like you have gone into a pizza shop and said, “I’d like a large veggie pizza please.” And they made you a pizza with their standard dough and their standard sauce and their standard cheese and their standard selection of vegetables.

Actually I don't think that's quite it either; to strain the analogy even further, I'd say it's more like going into a pizza shop and saying "I'd like a pizza" and the staff looking at you and deciding you look like a vegetarian and giving you a vegetarian pizza.

The crux of the issue here are the assumptions that are being made; for a standard default implementation there are no assumptions, because you're operating on the basis of methods and properties that you yourself have defined as the protocol creator that, where properly implemented, have precisely defined requirements, behaviours etc. When you're doing it with some form of compile-time or run-time reflection however you're messing around with parts of a concrete type that the protocol itself doesn't actually know anything about with any certainty.

It's the same reason that run-time reflection isn't something you should ever want to do, because while it might work with all of the types you test directly, if it supports arbitrary types (which it'd need to, or there'd be no point to reflecting) then there's always the risk of encountering a type where some part of it doesn't match the assumptions that you've made. Of course there are use-cases where this may not matter, e.g- if you're just dumping data from a type and don't really care if you're storing stuff that isn't important, but in other cases such as Equatable and Hashable it can make a difference, as it can drastically affect behaviour when those assumptions fail.

On 7 Sep 2017, at 05:00, Andrew Thompson <mrwerdo331@me.com <mailto:mrwerdo331@me.com>> wrote:
Perhaps we could find a solution by meeting in the middle. Introduce a compiler flag that will disable the automatic synthesis and revert to the old behaviour (i.e. swiftc main.swift —disable-automatic-synthesis )

Thanks for the suggestion, but that's a bit too all-or-nothing; I realise I might seem very negative but to be clear, I do want these synthesised features, I just don't want them in their current form. What I want is for them to be explicitly opted into where I, as the developer, need them to be, so that I'm in absolute, unambiguous control of when and where they are used.

I do absolutely support features that eliminate tedious boiler-plate, I just think that doing so implicitly in invasive ways is not the right way to do it.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

[anandabits (Matthew Johnson)]

Hello,

I'm interested in this debate because I've been bitten by automatic synthesis recently.

I'm reading your discussion, but I don't have a strong opinion. I only have an anecdote: implicit, automatic, and unavoidable code synthesis code can make it difficult to write some DSLs (aka Domain-Specific Languages).

I did stumble against undesired Equatable synthesis while developping a library[1] that generates SQL snippets. In this library, the `==` operator does not return a Bool: it returns an SQL expression:

  // SELECT * FROM players WHERE bestScore = 1000
  Player.filter(bestScore == 1000)

Since the library is free to define == as an operator that returns an SQL expression, this works quite well. Even when both operands have the same type:

  // SELECT * FROM players WHERE lastScore = bestScore
  Player.filter(lastScore == bestScore)

However, as soon as the type is (also) Equatable, an ambiguity occurs, and the DSL is basically broken:

  Player.filter(lastScore == bestScore) // which == please?

In this case, the == from synthesized Equatable conformance is not welcome at all. It prevents the library from controlling the return type of the == operator. Equatable conformance is unavoidable for enums based on String or generally any raw value type that adopts Equatable. The consequence is that my library can't allow its users to define an enum of table columns.

This is not a deal breaker. Everybody can live with this little caveat, and I guess I'm the only one who wishes things had been more *consistent*. But still: this story helps realizing that code synthesis can bite in plenty of unexpected ways.

I don't understand what this has to do with synthesized Equatable. Wouldn't manually implemented Equatable have the same impact? The design of a DSL should be able to accommodate conformance to basic protocols without ambiguity.

We generally want as many types to be Equatable and Hashable as possible. Synthesized conformance means more types will have these conformance and that's a good thing in all cases (so long as the implementation is correct).

···

Sent from my iPad

On Sep 7, 2017, at 7:07 AM, Gwendal Roué via swift-evolution <swift-evolution@swift.org> wrote:

Thanks for reading,
Gwendal Roué
[1] GitHub - groue/GRDB.swift: A toolkit for SQLite databases, with a focus on application development

Le 7 sept. 2017 à 12:20, Haravikk via swift-evolution <swift-evolution@swift.org> a écrit :

On 7 Sep 2017, at 00:11, Brent Royal-Gordon <brent@architechies.com> wrote:

On Sep 5, 2017, at 1:02 PM, Haravikk via swift-evolution <swift-evolution@swift.org> wrote:

This proposal idea is essentially for a new attribute @synthetic (name is up for debate). This attribute is required for any default implementation that includes reflective type compiler magic, use of the reflection API against `self` or, in future, any native Swift macros within the method (possibly limited to specific features, will depend on the macro language and its capabilities).

"Use of the reflection API against `self`"? `String(describing:)` and `String(reflecting:)` sometimes do that.

I see zero justification for having @synthetic cover all of these random things, but not ordinary default implementations—they have the same amount of dangerous implicitness.

Actually they don't; the problem here is that through reflection you are accessing and manipulating concrete types. A non-reflective default implementation only has access to what the protocol itself has defined. The synthetic alternatives are instead diving into parts of a concrete type that may have nothing to do with the protocol at all, and must therefore make assumptions that cannot be guaranteed to be correct, this is what makes them dangerous.

On 6 Sep 2017, at 23:43, Nevin Brackett-Rozinsky <nevin.brackettrozinsky@gmail.com> wrote:

On Wed, Sep 6, 2017 at 5:42 PM, Haravikk via swift-evolution <swift-evolution@swift.org> wrote:
the issue I'm trying to raise is that when those, and similar features, are used in synthesised behaviour (default implementations based upon the concrete type), that these behaviours should be opted into explicitly, otherwise they open up potential for all kinds of bugs, even when the assumptions being made about the concrete type are simple such as in the case for Equatable/Hashable. There's just too much potential for this kind of reflective protocol implementation to overreach; to me it feels very much like going into a restaurant and the waiter coming across and force-feeding me something I don't want instead of taking my order.

I might suggest that instead it is like you have gone into a pizza shop and said, “I’d like a large veggie pizza please.” And they made you a pizza with their standard dough and their standard sauce and their standard cheese and their standard selection of vegetables.

Actually I don't think that's quite it either; to strain the analogy even further, I'd say it's more like going into a pizza shop and saying "I'd like a pizza" and the staff looking at you and deciding you look like a vegetarian and giving you a vegetarian pizza.

The crux of the issue here are the assumptions that are being made; for a standard default implementation there are no assumptions, because you're operating on the basis of methods and properties that you yourself have defined as the protocol creator that, where properly implemented, have precisely defined requirements, behaviours etc. When you're doing it with some form of compile-time or run-time reflection however you're messing around with parts of a concrete type that the protocol itself doesn't actually know anything about with any certainty.

It's the same reason that run-time reflection isn't something you should ever want to do, because while it might work with all of the types you test directly, if it supports arbitrary types (which it'd need to, or there'd be no point to reflecting) then there's always the risk of encountering a type where some part of it doesn't match the assumptions that you've made. Of course there are use-cases where this may not matter, e.g- if you're just dumping data from a type and don't really care if you're storing stuff that isn't important, but in other cases such as Equatable and Hashable it can make a difference, as it can drastically affect behaviour when those assumptions fail.

On 7 Sep 2017, at 05:00, Andrew Thompson <mrwerdo331@me.com> wrote:
Perhaps we could find a solution by meeting in the middle. Introduce a compiler flag that will disable the automatic synthesis and revert to the old behaviour (i.e. swiftc main.swift —disable-automatic-synthesis )

Thanks for the suggestion, but that's a bit too all-or-nothing; I realise I might seem very negative but to be clear, I do want these synthesised features, I just don't want them in their current form. What I want is for them to be explicitly opted into where I, as the developer, need them to be, so that I'm in absolute, unambiguous control of when and where they are used.

I do absolutely support features that eliminate tedious boiler-plate, I just think that doing so implicitly in invasive ways is not the right way to do it.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[anandabits (Matthew Johnson)]

Right, let's make sure we're talking about the right thing here. Gwendal, your issue isn't with synthesis in the form of Codable or the new additions to Equatable/Hashable which are opt-in-by-conformance, it's with the specific case of raw value enums or enums without associated values where the synthesis is implicit with no way to opt-out. That's a big difference.

I can definitely see the latter being an issue if it were more widespread, and I'd be supportive of those enums being required to declare their conformance for consistency (though it would be source breaking).

Ahh, thanks for clearing that up Tony. I missed this. I agree that this could be problematic in some cases (as Gwendal found) and should be fixed. Now that we have a well established model for synthesis we should follow it consistently.

···

Sent from my iPad

On Sep 7, 2017, at 7:45 AM, Tony Allevato <tony.allevato@gmail.com> wrote:

However, I still haven't seen a real issue that has come up because of the distinction being drawn here between default implementations vs. implementations that can access other parts of the concrete type. It sounds like this discussion is trying to protect against a hypothetical problem that hasn't happened yet and may not happen; it would be helpful to show some motivating real-world cases where this is indeed a severe problem.

On Thu, Sep 7, 2017 at 5:37 AM Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

Sent from my iPad

On Sep 7, 2017, at 7:07 AM, Gwendal Roué via swift-evolution <swift-evolution@swift.org> wrote:

Hello,

I'm interested in this debate because I've been bitten by automatic synthesis recently.

I'm reading your discussion, but I don't have a strong opinion. I only have an anecdote: implicit, automatic, and unavoidable code synthesis code can make it difficult to write some DSLs (aka Domain-Specific Languages).

I did stumble against undesired Equatable synthesis while developping a library[1] that generates SQL snippets. In this library, the `==` operator does not return a Bool: it returns an SQL expression:

  // SELECT * FROM players WHERE bestScore = 1000
  Player.filter(bestScore == 1000)

Since the library is free to define == as an operator that returns an SQL expression, this works quite well. Even when both operands have the same type:

  // SELECT * FROM players WHERE lastScore = bestScore
  Player.filter(lastScore == bestScore)

However, as soon as the type is (also) Equatable, an ambiguity occurs, and the DSL is basically broken:

  Player.filter(lastScore == bestScore) // which == please?

In this case, the == from synthesized Equatable conformance is not welcome at all. It prevents the library from controlling the return type of the == operator. Equatable conformance is unavoidable for enums based on String or generally any raw value type that adopts Equatable. The consequence is that my library can't allow its users to define an enum of table columns.

This is not a deal breaker. Everybody can live with this little caveat, and I guess I'm the only one who wishes things had been more *consistent*. But still: this story helps realizing that code synthesis can bite in plenty of unexpected ways.

I don't understand what this has to do with synthesized Equatable. Wouldn't manually implemented Equatable have the same impact? The design of a DSL should be able to accommodate conformance to basic protocols without ambiguity.

We generally want as many types to be Equatable and Hashable as possible. Synthesized conformance means more types will have these conformance and that's a good thing in all cases (so long as the implementation is correct).

Thanks for reading,
Gwendal Roué
[1] GitHub - groue/GRDB.swift: A toolkit for SQLite databases, with a focus on application development

Le 7 sept. 2017 à 12:20, Haravikk via swift-evolution <swift-evolution@swift.org> a écrit :

On 7 Sep 2017, at 00:11, Brent Royal-Gordon <brent@architechies.com> wrote:

On Sep 5, 2017, at 1:02 PM, Haravikk via swift-evolution <swift-evolution@swift.org> wrote:

This proposal idea is essentially for a new attribute @synthetic (name is up for debate). This attribute is required for any default implementation that includes reflective type compiler magic, use of the reflection API against `self` or, in future, any native Swift macros within the method (possibly limited to specific features, will depend on the macro language and its capabilities).

"Use of the reflection API against `self`"? `String(describing:)` and `String(reflecting:)` sometimes do that.

I see zero justification for having @synthetic cover all of these random things, but not ordinary default implementations—they have the same amount of dangerous implicitness.

Actually they don't; the problem here is that through reflection you are accessing and manipulating concrete types. A non-reflective default implementation only has access to what the protocol itself has defined. The synthetic alternatives are instead diving into parts of a concrete type that may have nothing to do with the protocol at all, and must therefore make assumptions that cannot be guaranteed to be correct, this is what makes them dangerous.

On 6 Sep 2017, at 23:43, Nevin Brackett-Rozinsky <nevin.brackettrozinsky@gmail.com> wrote:

On Wed, Sep 6, 2017 at 5:42 PM, Haravikk via swift-evolution <swift-evolution@swift.org> wrote:
the issue I'm trying to raise is that when those, and similar features, are used in synthesised behaviour (default implementations based upon the concrete type), that these behaviours should be opted into explicitly, otherwise they open up potential for all kinds of bugs, even when the assumptions being made about the concrete type are simple such as in the case for Equatable/Hashable. There's just too much potential for this kind of reflective protocol implementation to overreach; to me it feels very much like going into a restaurant and the waiter coming across and force-feeding me something I don't want instead of taking my order.

I might suggest that instead it is like you have gone into a pizza shop and said, “I’d like a large veggie pizza please.” And they made you a pizza with their standard dough and their standard sauce and their standard cheese and their standard selection of vegetables.

Actually I don't think that's quite it either; to strain the analogy even further, I'd say it's more like going into a pizza shop and saying "I'd like a pizza" and the staff looking at you and deciding you look like a vegetarian and giving you a vegetarian pizza.

The crux of the issue here are the assumptions that are being made; for a standard default implementation there are no assumptions, because you're operating on the basis of methods and properties that you yourself have defined as the protocol creator that, where properly implemented, have precisely defined requirements, behaviours etc. When you're doing it with some form of compile-time or run-time reflection however you're messing around with parts of a concrete type that the protocol itself doesn't actually know anything about with any certainty.

It's the same reason that run-time reflection isn't something you should ever want to do, because while it might work with all of the types you test directly, if it supports arbitrary types (which it'd need to, or there'd be no point to reflecting) then there's always the risk of encountering a type where some part of it doesn't match the assumptions that you've made. Of course there are use-cases where this may not matter, e.g- if you're just dumping data from a type and don't really care if you're storing stuff that isn't important, but in other cases such as Equatable and Hashable it can make a difference, as it can drastically affect behaviour when those assumptions fail.

On 7 Sep 2017, at 05:00, Andrew Thompson <mrwerdo331@me.com> wrote:
Perhaps we could find a solution by meeting in the middle. Introduce a compiler flag that will disable the automatic synthesis and revert to the old behaviour (i.e. swiftc main.swift —disable-automatic-synthesis )

Thanks for the suggestion, but that's a bit too all-or-nothing; I realise I might seem very negative but to be clear, I do want these synthesised features, I just don't want them in their current form. What I want is for them to be explicitly opted into where I, as the developer, need them to be, so that I'm in absolute, unambiguous control of when and where they are used.

I do absolutely support features that eliminate tedious boiler-plate, I just think that doing so implicitly in invasive ways is not the right way to do it.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[allevato (Tony Allevato)]

Right, let's make sure we're talking about the right thing here. Gwendal,
your issue isn't with synthesis in the form of Codable or the new additions
to Equatable/Hashable which are opt-in-by-conformance, it's with the
specific case of raw value enums or enums without associated values where
the synthesis is implicit with no way to opt-out. That's a big difference.

I can definitely see the latter being an issue if it were more widespread,
and I'd be supportive of those enums being required to declare their
conformance for consistency (though it would be source breaking).

However, I still haven't seen a real issue that has come up because of the
distinction being drawn here between default implementations vs.
implementations that can access other parts of the concrete type. It sounds
like this discussion is trying to protect against a hypothetical problem
that hasn't happened yet and may not happen; it would be helpful to show
some motivating real-world cases where this is indeed a severe problem.

···

On Thu, Sep 7, 2017 at 5:37 AM Matthew Johnson via swift-evolution < swift-evolution@swift.org> wrote:

Sent from my iPad

On Sep 7, 2017, at 7:07 AM, Gwendal Roué via swift-evolution < > swift-evolution@swift.org> wrote:

Hello,

I'm interested in this debate because I've been bitten by automatic
synthesis recently.

I'm reading your discussion, but I don't have a strong opinion. I only
have an anecdote: implicit, automatic, and unavoidable code synthesis code
can make it difficult to write some DSLs (aka Domain-Specific Languages).

I did stumble against undesired Equatable synthesis while developping a
library[1] that generates SQL snippets. In this library, the `==` operator
does not return a Bool: it returns an SQL expression:

// SELECT * FROM players WHERE bestScore = 1000
Player.filter(bestScore == 1000)

Since the library is free to define == as an operator that returns an SQL
expression, this works quite well. Even when both operands have the same
type:

// SELECT * FROM players WHERE lastScore = bestScore
Player.filter(lastScore == bestScore)

However, as soon as the type is (also) Equatable, an ambiguity occurs, and
the DSL is basically broken:

Player.filter(lastScore == bestScore) // which == please?

In this case, the == from synthesized Equatable conformance is not welcome
at all. It prevents the library from controlling the return type of the ==
operator. Equatable conformance is unavoidable for enums based on String or
generally any raw value type that adopts Equatable. The consequence is that
my library can't allow its users to define an enum of table columns.

This is not a deal breaker. Everybody can live with this little caveat,
and I guess I'm the only one who wishes things had been more *consistent*.
But still: this story helps realizing that code synthesis can bite in
plenty of unexpected ways.

I don't understand what this has to do with synthesized Equatable.
Wouldn't manually implemented Equatable have the same impact? The design
of a DSL should be able to accommodate conformance to basic protocols
without ambiguity.

We generally want as many types to be Equatable and Hashable as possible.
Synthesized conformance means more types will have these conformance and
that's a good thing in all cases (so long as the implementation is
correct).

Thanks for reading,
Gwendal Roué
[1] GitHub - groue/GRDB.swift: A toolkit for SQLite databases, with a focus on application development

Le 7 sept. 2017 à 12:20, Haravikk via swift-evolution < > swift-evolution@swift.org> a écrit :

On 7 Sep 2017, at 00:11, Brent Royal-Gordon <brent@architechies.com> > wrote:

On Sep 5, 2017, at 1:02 PM, Haravikk via swift-evolution < > swift-evolution@swift.org> wrote:

This proposal idea is essentially for a new attribute @synthetic (name is
up for debate). This attribute is required for any default implementation
that includes reflective type compiler magic, use of the reflection API
against `self` or, in future, any native Swift macros within the method
(possibly limited to specific features, will depend on the macro language
and its capabilities).

"Use of the reflection API against `self`"? `String(describing:)` and
`String(reflecting:)` sometimes do that.

I see zero justification for having @synthetic cover all of these random
things, but not ordinary default implementations—they have the same amount
of dangerous implicitness.

Actually they don't; the problem here is that through reflection you are
accessing and manipulating concrete types. A non-reflective default
implementation *only* has access to what the protocol itself has defined.
The synthetic alternatives are instead diving into parts of a concrete type
that may have nothing to do with the protocol at all, and must therefore
make assumptions that cannot be guaranteed to be correct, this is what
makes them dangerous.

On 6 Sep 2017, at 23:43, Nevin Brackett-Rozinsky < > nevin.brackettrozinsky@gmail.com> wrote:

On Wed, Sep 6, 2017 at 5:42 PM, Haravikk via swift-evolution < > swift-evolution@swift.org> wrote:

the issue I'm trying to raise is that when those, and similar features,
are used in synthesised behaviour (default implementations based upon the
concrete type), that these behaviours should be opted into explicitly,
otherwise they open up potential for all kinds of bugs, even when the
assumptions being made about the concrete type are simple such as in the
case for Equatable/Hashable. There's just too much potential for this kind
of reflective protocol implementation to overreach; to me it feels very
much like going into a restaurant and the waiter coming across and
force-feeding me something I don't want instead of taking my order.

I might suggest that instead it is like you have gone into a pizza shop
and said, “I’d like a large veggie pizza please.” And they made you a pizza
with their standard dough and their standard sauce and their standard
cheese and their standard selection of vegetables.

Actually I don't think that's quite it either; to strain the analogy even
further, I'd say it's more like going into a pizza shop and saying "I'd
like a pizza" and the staff looking at you and deciding you look like a
vegetarian and giving you a vegetarian pizza.

The crux of the issue here are the assumptions that are being made; for a
standard default implementation there are no assumptions, because you're
operating on the basis of methods and properties that you yourself have
defined as the protocol creator that, where properly implemented, have
precisely defined requirements, behaviours etc. When you're doing it with
some form of compile-time or run-time reflection however you're messing
around with parts of a concrete type that the protocol itself doesn't
actually know anything about with any certainty.

It's the same reason that run-time reflection isn't something you should
ever want to do, because while it might work with all of the types you test
directly, if it supports arbitrary types (which it'd need to, or there'd be
no point to reflecting) then there's always the risk of encountering a type
where some part of it doesn't match the assumptions that you've made. Of
course there are use-cases where this may not matter, e.g- if you're just
dumping data from a type and don't really care if you're storing stuff that
isn't important, but in other cases such as Equatable and Hashable it
*can* make a difference, as it can drastically affect behaviour when
those assumptions fail.

On 7 Sep 2017, at 05:00, Andrew Thompson <mrwerdo331@me.com> wrote:

Perhaps we could find a solution by meeting in the middle. Introduce a
compiler flag that will disable the automatic synthesis and revert to the
old behaviour (i.e. swiftc main.swift —disable-automatic-synthesis )

Thanks for the suggestion, but that's a bit too all-or-nothing; I realise
I might seem very negative but to be clear, I *do* want these synthesised
features, I just don't want them in their current form. What I want is for
them to be explicitly opted into where I, as the developer, need them to
be, so that I'm in absolute, unambiguous control of when and where they are
used.

I do absolutely support features that eliminate tedious boiler-plate, I
just think that doing so implicitly in invasive ways is not the right way
to do it.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[gwendal.roue (Gwendal Roué)]

I don't understand what this has to do with synthesized Equatable. Wouldn't manually implemented Equatable have the same impact? The design of a DSL should be able to accommodate conformance to basic protocols without ambiguity.

I'll explain you:

The problem with synthesized Equatable is that it adds an unwanted == operator that returns a Bool.

This operator is unwanted because it conflicts with the == operator defined by the DSL which does not return a Bool.

  // Without synthesised Equatable
  let r = (a == b) // the type defined by the DSL

  // With synthesised Equatable
  let r = (a == b) // ambiguous

This is the same kind of conflict that happen when a function is overloaded with two return types:

  func f() -> Int { ... }
  func f() -> String { ... }
  f() // ambiguous

Without the synthesized Equatable, the type would not have any == operator that returns a Bool, and thus no conflict with the == operator defined by the DSL (the one that returns an SQL expression, in our particular context).

I hope that I have explained how synthesized conformance may impact code by the mere fact that they define methods. I'm not talking about the correctness of the synthesized code. I'm talking about its mere existence.

We generally want as many types to be Equatable and Hashable as possible. Synthesized conformance means more types will have these conformance and that's a good thing in all cases (so long as the implementation is correct).

Sure, of course. I'm with you. I'm not talking against code synthesis. Again, I'm not talking about the correctness either.

I'm talking about the consequences of implicit and non-avoidable synthesis. Exactly the theme of this thread, unless I'm totally mistaken.

Gwendal Roué

···

Le 7 sept. 2017 à 14:37, Matthew Johnson <matthew@anandabits.com> a écrit :

[gwendal.roue (Gwendal Roué)]

Right, let's make sure we're talking about the right thing here. Gwendal, your issue isn't with synthesis in the form of Codable or the new additions to Equatable/Hashable which are opt-in-by-conformance, it's with the specific case of raw value enums or enums without associated values where the synthesis is implicit with no way to opt-out. That's a big difference.

Yes.

I can definitely see the latter being an issue if it were more widespread, and I'd be supportive of those enums being required to declare their conformance for consistency (though it would be source breaking).

Yes, unfortunately.

However, I still haven't seen a real issue that has come up because of the distinction being drawn here between default implementations vs. implementations that can access other parts of the concrete type. It sounds like this discussion is trying to protect against a hypothetical problem that hasn't happened yet and may not happen; it would be helpful to show some motivating real-world cases where this is indeed a severe problem.

Yes. I'm not talking about implementation itself. I know this has been the main topic until I have tried to bring in the topic of the consequences of non-avoidable synthesis (extra methods that may conflict with userland methods).

If you ask me for a real-world case, then I think I gave one. Let me rephrase it:

it's impossible to define a value-backed enum without getting free Equatable conformance. This free conformance is sometimes unwanted, and I gave the example of DSLs. Now this problem is not *severe*. It's more a blind spot in the language, and finally just an unwanted side-effect of a compiler convenience,

This example gives a little argument, but still an argument, for "explicit synthetic behavior", the topic of this thread.

Gwendal Roué

···

Le 7 sept. 2017 à 14:45, Tony Allevato <tony.allevato@gmail.com> a écrit :

[itaiferber]

I think there is a bit of confusion here as to what code synthesis does — synthesized conformances (whether `Equatable`, `Hashable`, or `Codable`) merely provide default implementations for something which _already_ conforms to one of these protocols; they do not _add_ conformance to types on your behalf.

struct X {
     let val: Int
}

under synthesized `Equatable` does _not_ get an `==` defined for it in the same way that it does not get `encode(to:)` or `init(from:)`. Since it does not conform to the `Equatable` (or `Codable`) protocol, no synthesis happens for it.

As opposed to

struct Y : Equatable {
     let val: Int
}

which would get a default implementation for `static func ==(…)`, which it would otherwise already have to implement, by definition.

Synthesis does not add methods on your behalf; it only gives implementations for methods you’d have to implement, no matter what. I don’t know what’s going on in your case, but it’s not caused by synthesis — if your type conforms to `Equatable`, either you would have to define `==` yourself, or you’d get a free one. You’d see ambiguity regardless, since you asked for the type to be `Equatable` (or inherited that requirement).

···

On 7 Sep 2017, at 10:32, Gwendal Roué via swift-evolution wrote:

> Le 7 sept. 2017 à 14:37, Matthew Johnson <matthew@anandabits.com> a écrit :

I don't understand what this has to do with synthesized Equatable. Wouldn't manually implemented Equatable have the same impact? The design of a DSL should be able to accommodate conformance to basic protocols without ambiguity.

I'll explain you:

The problem with synthesized Equatable is that it adds an unwanted == operator that returns a Bool.

This operator is unwanted because it conflicts with the == operator defined by the DSL which does not return a Bool.

  // Without synthesised Equatable
  let r = (a == b) // the type defined by the DSL

  // With synthesised Equatable
  let r = (a == b) // ambiguous

This is the same kind of conflict that happen when a function is overloaded with two return types:

  func f() -> Int { ... }
  func f() -> String { ... }
  f() // ambiguous

Without the synthesized Equatable, the type would not have any == operator that returns a Bool, and thus no conflict with the == operator defined by the DSL (the one that returns an SQL expression, in our particular context).

I hope that I have explained how synthesized conformance may impact code by the mere fact that they define methods. I'm not talking about the correctness of the synthesized code. I'm talking about its mere existence.

We generally want as many types to be Equatable and Hashable as possible. Synthesized conformance means more types will have these conformance and that's a good thing in all cases (so long as the implementation is correct).

Sure, of course. I'm with you. I'm not talking against code synthesis. Again, I'm not talking about the correctness either.

I'm talking about the consequences of implicit and non-avoidable synthesis. Exactly the theme of this thread, unless I'm totally mistaken.

Gwendal Roué

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[allevato (Tony Allevato)]

Right, let's make sure we're talking about the right thing here. Gwendal,
your issue isn't with synthesis in the form of Codable or the new additions
to Equatable/Hashable which are opt-in-by-conformance, it's with the
specific case of raw value enums or enums without associated values where
the synthesis is implicit with no way to opt-out. That's a big difference.

Yes.

I can definitely see the latter being an issue if it were more widespread,
and I'd be supportive of those enums being required to declare their
conformance for consistency (though it would be source breaking).

Yes, unfortunately.

However, I still haven't seen a real issue that has come up because of the
distinction being drawn here between default implementations vs.
implementations that can access other parts of the concrete type. It sounds
like this discussion is trying to protect against a hypothetical problem
that hasn't happened yet and may not happen; it would be helpful to show
some motivating real-world cases where this is indeed a severe problem.

Yes. I'm not talking about implementation itself. I know this has been the
main topic until I have tried to bring in the topic of the consequences of
non-avoidable synthesis (extra methods that may conflict with userland
methods).

If you ask me for a real-world case, then I think I gave one. Let me
rephrase it:

it's impossible to define a value-backed enum without getting free
Equatable conformance. This free conformance is sometimes unwanted, and I
gave the example of DSLs. Now this problem is not *severe*. It's more a
blind spot in the language, and finally just an unwanted side-effect of a
compiler convenience,

Again, this is not the issue that Haravikk is describing in this thread.

I'll clarify—your issue is specifically with the fact that enums with raw
values and enums without associated values receive Equatable even without
explicitly conforming to it, and therefore users have no way of opting out
of it. This predates SE-0185, and I didn't propose making any changes to
the conformance of those enums for source compatibility reasons, though I
wouldn't be opposed to it because it makes them consistent across the board.

Haravikk's argument is about synthesized conformances like Codable and
Equatable/Hashable in SE-0185, where the user must explicitly conform the
type to those protocols. His claim is that that act of opting in is not
sufficient and that it is still dangerous if those synthesized conformances
can access members that are not also declared in the protocol. That's a
completely separate issue to yours, and one that I hope he'll present more
evidence of. Right now, requiring that you not only explicitly conform to
the protocol but also explicitly request the synthesis feels like a
solution without an actual problem, and is a situation we already have
today with default method implementations.

···

On Thu, Sep 7, 2017 at 10:39 AM Gwendal Roué <gwendal.roue@gmail.com> wrote:

Le 7 sept. 2017 à 14:45, Tony Allevato <tony.allevato@gmail.com> a écrit :

This example gives a little argument, but still an argument, for "explicit
synthetic behavior", the topic of this thread.

Gwendal Roué

[allevato (Tony Allevato)]

:

Right, let's make sure we're talking about the right thing here. Gwendal,
your issue isn't with synthesis in the form of Codable or the new additions
to Equatable/Hashable which are opt-in-by-conformance, it's with the
specific case of raw value enums or enums without associated values where
the synthesis is implicit with no way to opt-out. That's a big difference.

Yes.

I can definitely see the latter being an issue if it were more
widespread, and I'd be supportive of those enums being required to declare
their conformance for consistency (though it would be source breaking).

Yes, unfortunately.

However, I still haven't seen a real issue that has come up because of
the distinction being drawn here between default implementations vs.
implementations that can access other parts of the concrete type. It sounds
like this discussion is trying to protect against a hypothetical problem
that hasn't happened yet and may not happen; it would be helpful to show
some motivating real-world cases where this is indeed a severe problem.

Yes. I'm not talking about implementation itself. I know this has been
the main topic until I have tried to bring in the topic of the consequences
of non-avoidable synthesis (extra methods that may conflict with userland
methods).

If you ask me for a real-world case, then I think I gave one. Let me
rephrase it:

it's impossible to define a value-backed enum without getting free
Equatable conformance. This free conformance is sometimes unwanted, and I
gave the example of DSLs. Now this problem is not *severe*. It's more a
blind spot in the language, and finally just an unwanted side-effect of a
compiler convenience,

Again, this is not the issue that Haravikk is describing in this thread.

I'll clarify—your issue is specifically with the fact that enums with raw
values and enums without associated values receive Equatable even without
explicitly conforming to it, and therefore users have no way of opting out
of it. This predates SE-0185, and I didn't propose making any changes to
the conformance of those enums for source compatibility reasons, though I
wouldn't be opposed to it because it makes them consistent across the board.

Haravikk's argument is about synthesized conformances like Codable and
Equatable/Hashable in SE-0185, where the user must explicitly conform the
type to those protocols. His claim is that that act of opting in is not
sufficient and that it is still dangerous if those synthesized conformances
can access members that are not also declared in the protocol. That's a
completely separate issue to yours, and one that I hope he'll present more
evidence of. Right now, requiring that you not only explicitly conform to
the protocol but also explicitly request the synthesis feels like a
solution without an actual problem, and is a situation we already have
today with default method implementations.

The simplest real-world case is easy:

struct Foo { var data:String }
extension Foo : Equatable {} // This currently produces an error, in
future it will not

Why is this a problem? It's no different than if someone extended Foo to
conform to a protocol with a default implementation that was written in
code.

I argued this point on the specific topic for Equatable/Hashable, but it
was, both during and after review, essentially ignored and the decision to
synthesise implicitly never sufficiently justified. The closest that I got
was "but Codable does it" which is about as weak a justification as you
could possibly get as I don't really agree with it in the case of Codable
either. In the case of Equatable/Hashable specifically this is arguably a
breaking change that has IMO been totally ignored by the core team who
haven't given any reasonable response.

That's not a fair characterization. Just because your concerns were
disagreed with does not mean they were ignored; my understanding is that
the core team views these synthesized conformances as a different kind of
default method (and one which could be hoisted out of the compiler once
sufficient metaprogramming facilities are available).

The way to handle synthesized conformances was discussed during the review
period for Codable, during the earlier pitch a few months ago for what
became SE-0185, and again during its formal review. It's not accurate to
reduce the argument you disagree with to "but Codable does it" when what
you're referring to is established precedent based on those prior
discussions.

In the broader case however I still feel that synthesised behaviour should
require explicit rather than implicit opt-in, as it allows us to
distinguish between a developer who wants to implement requirements
themselves, versus one who is happy to have one derived from their concrete
type automatically. The current setup does not allow this at all.

I feel like we keep going back to this, but this statement applies equally
to non-synthesized default implementations. Are you suggesting that users
should have to opt-in specifically to all default implementations provided
by a protocol in some way beyond merely conforming to that protocol? If
not, what specifically makes synthesized conformances a special case?

···

On Thu, Sep 7, 2017 at 11:18 AM Haravikk via swift-evolution < swift-evolution@swift.org> wrote:

On 7 Sep 2017, at 18:53, Tony Allevato via swift-evolution < > swift-evolution@swift.org> wrote:
On Thu, Sep 7, 2017 at 10:39 AM Gwendal Roué <gwendal.roue@gmail.com> > wrote:

Le 7 sept. 2017 à 14:45, Tony Allevato <tony.allevato@gmail.com> a écrit

In future if the role keywords are adopted we will always be able to
distinguish one who wants to explicitly implicit more than just the minimum
requirements.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[Haravikk3 (Haravikk)]

The simplest real-world case is easy:

  struct Foo { var data:String }
  extension Foo : Equatable {} // This currently produces an error, in future it will not

I argued this point on the specific topic for Equatable/Hashable, but it was, both during and after review, essentially ignored and the decision to synthesise implicitly never sufficiently justified. The closest that I got was "but Codable does it" which is about as weak a justification as you could possibly get as I don't really agree with it in the case of Codable either. In the case of Equatable/Hashable specifically this is arguably a breaking change that has IMO been totally ignored by the core team who haven't given any reasonable response.

In the broader case however I still feel that synthesised behaviour should require explicit rather than implicit opt-in, as it allows us to distinguish between a developer who wants to implement requirements themselves, versus one who is happy to have one derived from their concrete type automatically. The current setup does not allow this at all. In future if the role keywords are adopted we will always be able to distinguish one who wants to explicitly implicit more than just the minimum requirements.

···

On 7 Sep 2017, at 18:53, Tony Allevato via swift-evolution <swift-evolution@swift.org> wrote:

On Thu, Sep 7, 2017 at 10:39 AM Gwendal Roué <gwendal.roue@gmail.com <mailto:gwendal.roue@gmail.com>> wrote:

Le 7 sept. 2017 à 14:45, Tony Allevato <tony.allevato@gmail.com <mailto:tony.allevato@gmail.com>> a écrit :

Right, let's make sure we're talking about the right thing here. Gwendal, your issue isn't with synthesis in the form of Codable or the new additions to Equatable/Hashable which are opt-in-by-conformance, it's with the specific case of raw value enums or enums without associated values where the synthesis is implicit with no way to opt-out. That's a big difference.

Yes.

I can definitely see the latter being an issue if it were more widespread, and I'd be supportive of those enums being required to declare their conformance for consistency (though it would be source breaking).

Yes, unfortunately.

However, I still haven't seen a real issue that has come up because of the distinction being drawn here between default implementations vs. implementations that can access other parts of the concrete type. It sounds like this discussion is trying to protect against a hypothetical problem that hasn't happened yet and may not happen; it would be helpful to show some motivating real-world cases where this is indeed a severe problem.

Yes. I'm not talking about implementation itself. I know this has been the main topic until I have tried to bring in the topic of the consequences of non-avoidable synthesis (extra methods that may conflict with userland methods).

If you ask me for a real-world case, then I think I gave one. Let me rephrase it:

it's impossible to define a value-backed enum without getting free Equatable conformance. This free conformance is sometimes unwanted, and I gave the example of DSLs. Now this problem is not *severe*. It's more a blind spot in the language, and finally just an unwanted side-effect of a compiler convenience,

Again, this is not the issue that Haravikk is describing in this thread.

I'll clarify—your issue is specifically with the fact that enums with raw values and enums without associated values receive Equatable even without explicitly conforming to it, and therefore users have no way of opting out of it. This predates SE-0185, and I didn't propose making any changes to the conformance of those enums for source compatibility reasons, though I wouldn't be opposed to it because it makes them consistent across the board.

Haravikk's argument is about synthesized conformances like Codable and Equatable/Hashable in SE-0185, where the user must explicitly conform the type to those protocols. His claim is that that act of opting in is not sufficient and that it is still dangerous if those synthesized conformances can access members that are not also declared in the protocol. That's a completely separate issue to yours, and one that I hope he'll present more evidence of. Right now, requiring that you not only explicitly conform to the protocol but also explicitly request the synthesis feels like a solution without an actual problem, and is a situation we already have today with default method implementations.