[Pitch] Overridable Members in Extensions

[jrose (Jordan Rose)]

Hey, everyone. Here's a small feature with ABI implications, ready for feedback. In addition to comments on the proposal itself, I'm also interested in hearing how often this comes up for people:

- extending a class you don't own
- to add an overridable method
- where some of the overriders might be outside the current module

Jordan

jrose-apple/swift-evolution/blob/overridable-members-in-extensions/proposals/nnnn-overridable-members-in-extensions.md

···

---

Overridable Members in Extensions

Proposal: SE-NNNN
Author: Jordan Rose <https://github.com/jrose-apple&gt;
Status: Awaiting review
Review manager: TBD
<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions

Today, methods introduced in an extension of a class cannot override or be overridden unless the method is (implicitly or explicitly) marked @objc. This proposal lifts the blanket restriction while still enforcing safety.

Note: it's already plan-of-record that if the extension is in the same module as the class, the methods will be treated as if they were declared in the class itself. This proposal only applies to extensions declared in a different module.
<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions

This is used to add operations to system or library classes that you can customize in your own classes, as seen in the Apple AdaptivePhotos <https://developer.apple.com/library/ios/samplecode/AdaptivePhotos/Listings/AdaptiveCode_AdaptiveCode_UIViewController_PhotoContents_swift.html&gt; sample code.

extension UIViewController {
  func containsPhoto(photo: Photo) -> Bool {
    return false
  }
}
class ConversationViewController : UIViewController {
  // …
  override func containsPhoto(photo: Photo) -> Bool {
    return self.conversation.photos.contains(photo)
  }
}
Additional motivation: parity with Objective-C. If Objective-C didn't allow this, we might not have done it, but right now the answer is "if your method is ObjC-compatible, just slap an attribute on it; otherwise you're out of luck", which isn't really a sound design choice.

<https://github.com/jrose-apple/swift-evolution/tree/overridable-members-in-extensions#todays-workaround&gt;Today&#39;s Workaround

If you know every class that needs a custom implementation of a method, you can use dynamic casts to get the same effect:

extension UIViewController {
  final func containsPhoto(photo: Photo) -> Bool {
    switch self {
    case is ListTableViewController:
      return true
    case let cvc as ConversationViewController:
      return cvc.conversation.photos.contains(photo)
    default:
      return false
    }
  }
}
But this is not possible if there may be subclasses outside of the module, and it either forces all of the implementations into a single method body or requires adding dummy methods to each class.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions solution

This proposal lifts the restriction on non-@objc extension methods (and properties, and subscripts) by requiring an alternate dispatch mechanism that can be arbitrarily extended. To preserve safety and correctness, a new, narrower restriction will be put in place:

If an extension in module B is extending a class in module A, it may only override members added in module B.

Any other rule can result in two modules trying to add an override for the same method on the same class.

Note: This rule applies to @objc members as well as non-@objc members.
There is no restriction on extensions adding new overridable members. These members can be overridden by any extension in the same module (by the above rule) and by a subclass in any module, whether in the class declaration itself or in an extension in the same module.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions design

Besides safety, the other reason we didn't add this feature is because the Swift method dispatch mechanism uses a single virtual dispatch table for a class, which cannot be arbitrarily extended after the fact. The implementation would require an alternate dispatch mechanism that can be arbitrarily extended.

On Apple platforms this is implemented by the Objective-C method table; we would provide a simplified implementation of the same on Linux. For a selector we would use the mangled name of the original overridden method. These methods would still use Swift calling conventions; they're just being stored in the same lookup table as Objective-C methods.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions Evolution

As with any other method <swift/LibraryEvolution.rst at main · apple/swift · GitHub, it is legal to "move" an extension method up to an extension on the base class, as long as the original declaration is not removed entirely. The new entry point will forward over to the original entry point in order to preserve binary compatibility.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions on existing code

No existing semantics will be affected. Dispatch for methods in extensions may get slower, since it's no longer using a direct call.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions considered

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions extension methods are final

This is sound, and doesn't rule out the "closed class hierarchy" partial workaround described above. However, it does prevent some reasonable patterns that were possible in Objective-C, and it is something we've seen developers try to do (with or without @objc).

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions extension methods are overridable, non-@objc methods are not

This is a practical answer, since it requires no further implementation work. We could require members in extensions to be explicitly annotated dynamic and final, respectively, so that the semantics are at least clear. However, it's not a very principled design choice: either overridable extension members are useful, or they aren't.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions extensions

The restriction that an extension cannot override a method from another module is intended for safety purposes, preventing two modules from each adding their own override. It's possible to make this a link-time failure rather than a compile-time failure by emitting a dummy symbol representing the (class, member) pair. Because of this, it may be useful to have an "I know what I'm doing" annotation that promises that no one else will add the same member; if it does happen then the program will fail to link.

(Indeed, we probably should do this anyway for @objc overrides, which run the risk of run-time collision because of Objective-C language semantics.)

If we ever have an "SPI" feature that allows public API to be restricted to certain clients, it would be reasonable to consider relaxing the safety restrictions for those clients specifically on the grounds that the library author trusts them to know what they're doing.

[Philippe_Hausler]

Looks really great; this will definitely solve some problems we had with the Foundation implementation.

···

On Feb 10, 2016, at 5:45 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org> wrote:

Hey, everyone. Here's a small feature with ABI implications, ready for feedback. In addition to comments on the proposal itself, I'm also interested in hearing how often this comes up for people:

- extending a class you don't own
- to add an overridable method
- where some of the overriders might be outside the current module

Jordan

https://github.com/jrose-apple/swift-evolution/blob/overridable-members-in-extensions/proposals/nnnn-overridable-members-in-extensions.md

---

Overridable Members in Extensions

Proposal: SE-NNNN
Author: Jordan Rose <https://github.com/jrose-apple&gt;
Status: Awaiting review
Review manager: TBD
<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions

Today, methods introduced in an extension of a class cannot override or be overridden unless the method is (implicitly or explicitly) marked @objc. This proposal lifts the blanket restriction while still enforcing safety.

Note: it's already plan-of-record that if the extension is in the same module as the class, the methods will be treated as if they were declared in the class itself. This proposal only applies to extensions declared in a different module.
<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions

This is used to add operations to system or library classes that you can customize in your own classes, as seen in the Apple AdaptivePhotos <https://developer.apple.com/library/ios/samplecode/AdaptivePhotos/Listings/AdaptiveCode_AdaptiveCode_UIViewController_PhotoContents_swift.html&gt; sample code.

extension UIViewController {
  func containsPhoto(photo: Photo) -> Bool {
    return false
  }
}
class ConversationViewController : UIViewController {
  // …
  override func containsPhoto(photo: Photo) -> Bool {
    return self.conversation.photos.contains(photo)
  }
}
Additional motivation: parity with Objective-C. If Objective-C didn't allow this, we might not have done it, but right now the answer is "if your method is ObjC-compatible, just slap an attribute on it; otherwise you're out of luck", which isn't really a sound design choice.

<https://github.com/jrose-apple/swift-evolution/tree/overridable-members-in-extensions#todays-workaround&gt;Today&#39;s Workaround

If you know every class that needs a custom implementation of a method, you can use dynamic casts to get the same effect:

extension UIViewController {
  final func containsPhoto(photo: Photo) -> Bool {
    switch self {
    case is ListTableViewController:
      return true
    case let cvc as ConversationViewController:
      return cvc.conversation.photos.contains(photo)
    default:
      return false
    }
  }
}
But this is not possible if there may be subclasses outside of the module, and it either forces all of the implementations into a single method body or requires adding dummy methods to each class.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions solution

This proposal lifts the restriction on non-@objc extension methods (and properties, and subscripts) by requiring an alternate dispatch mechanism that can be arbitrarily extended. To preserve safety and correctness, a new, narrower restriction will be put in place:

If an extension in module B is extending a class in module A, it may only override members added in module B.

Any other rule can result in two modules trying to add an override for the same method on the same class.

Note: This rule applies to @objc members as well as non-@objc members.
There is no restriction on extensions adding new overridable members. These members can be overridden by any extension in the same module (by the above rule) and by a subclass in any module, whether in the class declaration itself or in an extension in the same module.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions design

Besides safety, the other reason we didn't add this feature is because the Swift method dispatch mechanism uses a single virtual dispatch table for a class, which cannot be arbitrarily extended after the fact. The implementation would require an alternate dispatch mechanism that can be arbitrarily extended.

On Apple platforms this is implemented by the Objective-C method table; we would provide a simplified implementation of the same on Linux. For a selector we would use the mangled name of the original overridden method. These methods would still use Swift calling conventions; they're just being stored in the same lookup table as Objective-C methods.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions Evolution

As with any other method <https://github.com/apple/swift/blob/master/docs/LibraryEvolution.rst#classes&gt;, it is legal to "move" an extension method up to an extension on the base class, as long as the original declaration is not removed entirely. The new entry point will forward over to the original entry point in order to preserve binary compatibility.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions on existing code

No existing semantics will be affected. Dispatch for methods in extensions may get slower, since it's no longer using a direct call.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions considered

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions extension methods are final

This is sound, and doesn't rule out the "closed class hierarchy" partial workaround described above. However, it does prevent some reasonable patterns that were possible in Objective-C, and it is something we've seen developers try to do (with or without @objc).

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions extension methods are overridable, non-@objc methods are not

This is a practical answer, since it requires no further implementation work. We could require members in extensions to be explicitly annotated dynamic and final, respectively, so that the semantics are at least clear. However, it's not a very principled design choice: either overridable extension members are useful, or they aren't.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions extensions

The restriction that an extension cannot override a method from another module is intended for safety purposes, preventing two modules from each adding their own override. It's possible to make this a link-time failure rather than a compile-time failure by emitting a dummy symbol representing the (class, member) pair. Because of this, it may be useful to have an "I know what I'm doing" annotation that promises that no one else will add the same member; if it does happen then the program will fail to link.

(Indeed, we probably should do this anyway for @objc overrides, which run the risk of run-time collision because of Objective-C language semantics.)

If we ever have an "SPI" feature that allows public API to be restricted to certain clients, it would be reasonable to consider relaxing the safety restrictions for those clients specifically on the grounds that the library author trusts them to know what they're doing.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[gparker42 (Greg Parker)]

Hey, everyone. Here's a small feature with ABI implications, ready for feedback. In addition to comments on the proposal itself, I'm also interested in hearing how often this comes up for people:

- extending a class you don't own
- to add an overridable method
- where some of the overriders might be outside the current module

Jordan

https://github.com/jrose-apple/swift-evolution/blob/overridable-members-in-extensions/proposals/nnnn-overridable-members-in-extensions.md

---

Overridable Members in Extensions

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions Evolution
As with any other method <https://github.com/apple/swift/blob/master/docs/LibraryEvolution.rst#classes&gt;, it is legal to "move" an extension method up to an extension on the base class, as long as the original declaration is not removed entirely. The new entry point will forward over to the original entry point in order to preserve binary compatibility.

What happens if the base class adds an independent implementation of the same method?

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions extensions

The restriction that an extension cannot override a method from another module is intended for safety purposes, preventing two modules from each adding their own override. It's possible to make this a link-time failure rather than a compile-time failure by emitting a dummy symbol representing the (class, member) pair.

Dummy symbol enforcement only helps when two conflicting modules have a linkage relationship (i.e. one links to the other). It won't catch the case where two independent modules collide.

The old ObjC "qualified selectors" proposal handled a similar situation by mangling the original provider of the overridable method into the method name. This makes evolution trickier (you need to preserve the same mangled name even if the method moves into the base class or into a different module). On the plus side it allows independent extensions and subclasses to do independent things. It also preserves binary compatibility of those extensions and subclasses if the base class adds its own unrelated method with the same name. (Source compatibility in that case is not preserved. The extensions and subclasses would need to add further annotations to preserve their independence when recompiled.)

···

On Feb 10, 2016, at 5:45 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org> wrote:

--
Greg Parker gparker@apple.com Runtime Wrangler

[Tino]

+1

I've used this technique in Objective-C for dependency injection, and already had to realize that I can't do so with Swift.
As long as the feature has no negative impact, I would be glad if it would be available — even if there is some penalty associated with it (I'd expect that this could be avoided with final).

[marco.masser]

+1 because I think this is a good thing in terms of consistency and the principle of least astonishment

The introduction states in the note that in the same module, this is planned to work anyways, so if I understand correctly, there are two groups of people who are impacted by the same thing:
- Library/framework authors can freely decide to split up their classes into separate extensions without worrying about impacting clients of that library/framework.
- Library/framework clients aren’t affected by the author’s choice of splitting up their classes into separate extensions.

It would really be unfortunate if I couldn’t override a specific method of NSButton just because the App Kit team implemented that method in an extension instead of the class proper.

I tend to split up my classes in a couple of extensions that each group members that somehow belong together, e.g. a separate extension for all @IBActions, one extension implementing conformance to a protocol, one private extension with methods dealing with X, etc.
Right now, I always tend to write all methods that override something into the class itself, not an extension because I know there are weird issues you can run into (e.g. https://bugs.swift.org/browse/SR-584\). I’m looking forward to when these will be fixed for classes in the same target (as indicated by the note in the introduction), but when the proposal in question gets implemented, this would also work as expected for classes in different modules.
Therefore: +1 for consistency and least astonishment.

Marco

···

On 2016-02-11, at 02:45, Jordan Rose via swift-evolution <swift-evolution@swift.org> wrote:

Hey, everyone. Here's a small feature with ABI implications, ready for feedback. In addition to comments on the proposal itself, I'm also interested in hearing how often this comes up for people:

- extending a class you don't own
- to add an overridable method
- where some of the overriders might be outside the current module

Jordan

https://github.com/jrose-apple/swift-evolution/blob/overridable-members-in-extensions/proposals/nnnn-overridable-members-in-extensions.md

---

Overridable Members in Extensions

Proposal: SE-NNNN
Author: Jordan Rose <https://github.com/jrose-apple&gt;
Status: Awaiting review
Review manager: TBD
<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions

Today, methods introduced in an extension of a class cannot override or be overridden unless the method is (implicitly or explicitly) marked @objc. This proposal lifts the blanket restriction while still enforcing safety.

Note: it's already plan-of-record that if the extension is in the same module as the class, the methods will be treated as if they were declared in the class itself. This proposal only applies to extensions declared in a different module.
<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions

This is used to add operations to system or library classes that you can customize in your own classes, as seen in the Apple AdaptivePhotos <https://developer.apple.com/library/ios/samplecode/AdaptivePhotos/Listings/AdaptiveCode_AdaptiveCode_UIViewController_PhotoContents_swift.html&gt; sample code.

extension UIViewController {
  func containsPhoto(photo: Photo) -> Bool {
    return false
  }
}
class ConversationViewController : UIViewController {
  // …
  override func containsPhoto(photo: Photo) -> Bool {
    return self.conversation.photos.contains(photo)
  }
}
Additional motivation: parity with Objective-C. If Objective-C didn't allow this, we might not have done it, but right now the answer is "if your method is ObjC-compatible, just slap an attribute on it; otherwise you're out of luck", which isn't really a sound design choice.

<https://github.com/jrose-apple/swift-evolution/tree/overridable-members-in-extensions#todays-workaround&gt;Today&#39;s Workaround

If you know every class that needs a custom implementation of a method, you can use dynamic casts to get the same effect:

extension UIViewController {
  final func containsPhoto(photo: Photo) -> Bool {
    switch self {
    case is ListTableViewController:
      return true
    case let cvc as ConversationViewController:
      return cvc.conversation.photos.contains(photo)
    default:
      return false
    }
  }
}
But this is not possible if there may be subclasses outside of the module, and it either forces all of the implementations into a single method body or requires adding dummy methods to each class.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions solution

This proposal lifts the restriction on non-@objc extension methods (and properties, and subscripts) by requiring an alternate dispatch mechanism that can be arbitrarily extended. To preserve safety and correctness, a new, narrower restriction will be put in place:

If an extension in module B is extending a class in module A, it may only override members added in module B.

Any other rule can result in two modules trying to add an override for the same method on the same class.

Note: This rule applies to @objc members as well as non-@objc members.
There is no restriction on extensions adding new overridable members. These members can be overridden by any extension in the same module (by the above rule) and by a subclass in any module, whether in the class declaration itself or in an extension in the same module.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions design

Besides safety, the other reason we didn't add this feature is because the Swift method dispatch mechanism uses a single virtual dispatch table for a class, which cannot be arbitrarily extended after the fact. The implementation would require an alternate dispatch mechanism that can be arbitrarily extended.

On Apple platforms this is implemented by the Objective-C method table; we would provide a simplified implementation of the same on Linux. For a selector we would use the mangled name of the original overridden method. These methods would still use Swift calling conventions; they're just being stored in the same lookup table as Objective-C methods.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions Evolution

As with any other method <https://github.com/apple/swift/blob/master/docs/LibraryEvolution.rst#classes&gt;, it is legal to "move" an extension method up to an extension on the base class, as long as the original declaration is not removed entirely. The new entry point will forward over to the original entry point in order to preserve binary compatibility.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions on existing code

No existing semantics will be affected. Dispatch for methods in extensions may get slower, since it's no longer using a direct call.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions considered

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions extension methods are final

This is sound, and doesn't rule out the "closed class hierarchy" partial workaround described above. However, it does prevent some reasonable patterns that were possible in Objective-C, and it is something we've seen developers try to do (with or without @objc).

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions extension methods are overridable, non-@objc methods are not

This is a practical answer, since it requires no further implementation work. We could require members in extensions to be explicitly annotated dynamic and final, respectively, so that the semantics are at least clear. However, it's not a very principled design choice: either overridable extension members are useful, or they aren't.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions extensions

The restriction that an extension cannot override a method from another module is intended for safety purposes, preventing two modules from each adding their own override. It's possible to make this a link-time failure rather than a compile-time failure by emitting a dummy symbol representing the (class, member) pair. Because of this, it may be useful to have an "I know what I'm doing" annotation that promises that no one else will add the same member; if it does happen then the program will fail to link.

(Indeed, we probably should do this anyway for @objc overrides, which run the risk of run-time collision because of Objective-C language semantics.)

If we ever have an "SPI" feature that allows public API to be restricted to certain clients, it would be reasonable to consider relaxing the safety restrictions for those clients specifically on the grounds that the library author trusts them to know what they're doing.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[jrose (Jordan Rose)]

Hey, everyone. Here's a small feature with ABI implications, ready for feedback. In addition to comments on the proposal itself, I'm also interested in hearing how often this comes up for people:

- extending a class you don't own
- to add an overridable method
- where some of the overriders might be outside the current module

Jordan

https://github.com/jrose-apple/swift-evolution/blob/overridable-members-in-extensions/proposals/nnnn-overridable-members-in-extensions.md

---

Overridable Members in Extensions

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions Evolution
As with any other method <https://github.com/apple/swift/blob/master/docs/LibraryEvolution.rst#classes&gt;, it is legal to "move" an extension method up to an extension on the base class, as long as the original declaration is not removed entirely. The new entry point will forward over to the original entry point in order to preserve binary compatibility.

What happens if the base class adds an independent implementation of the same method?

If it's an @objc method, you're hosed, just like you always were, but at least you'll (probably) find out when you recompile. I don't think there's anything we can do about this other than banning @objc methods in extensions, and I don't think that'll fly.

If it's a Swift method, it's not the "same" method; it's just another method with the same full-name. Existing binaries won't conflict. When you recompile, though, you'll get a conflict, and we don't currently have any annotations that allow you to fix that conflict, or for subclasses in a client module to deal with the intermediate module not yet having fixed the conflict. This isn't specific to extensions, though; you have the same problem with methods declared in the class body.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions extensions

The restriction that an extension cannot override a method from another module is intended for safety purposes, preventing two modules from each adding their own override. It's possible to make this a link-time failure rather than a compile-time failure by emitting a dummy symbol representing the (class, member) pair.

Dummy symbol enforcement only helps when two conflicting modules have a linkage relationship (i.e. one links to the other). It won't catch the case where two independent modules collide.

If two linker modules provide the same public symbol, that won't cause an issue?

(On the other hand, maybe we don't want to be producing extra public symbols just to check if someone made a mistake.)

The old ObjC "qualified selectors" proposal handled a similar situation by mangling the original provider of the overridable method into the method name. This makes evolution trickier (you need to preserve the same mangled name even if the method moves into the base class or into a different module). On the plus side it allows independent extensions and subclasses to do independent things. It also preserves binary compatibility of those extensions and subclasses if the base class adds its own unrelated method with the same name. (Source compatibility in that case is not preserved. The extensions and subclasses would need to add further annotations to preserve their independence when recompiled.)

Yep, that's pretty much what the Swift implementation strategy is. I think my paragraph above is basically a rewrite of this.

Jordan

···

On Feb 10, 2016, at 18:29 , Greg Parker <gparker@apple.com> wrote:

On Feb 10, 2016, at 5:45 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

[jrose (Jordan Rose)]

Can I ask what problems those were? I would have figured that any extensions you had were for classes in the same module. (I know methods in extensions within the same module are also not overridable, but that's just a bug at this point.)

Jordan

···

On Feb 10, 2016, at 18:11 , Philippe Hausler <phausler@apple.com> wrote:

Looks really great; this will definitely solve some problems we had with the Foundation implementation.

On Feb 10, 2016, at 5:45 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Hey, everyone. Here's a small feature with ABI implications, ready for feedback. In addition to comments on the proposal itself, I'm also interested in hearing how often this comes up for people:

- extending a class you don't own
- to add an overridable method
- where some of the overriders might be outside the current module

Jordan

https://github.com/jrose-apple/swift-evolution/blob/overridable-members-in-extensions/proposals/nnnn-overridable-members-in-extensions.md

---

Overridable Members in Extensions

Proposal: SE-NNNN
Author: Jordan Rose <https://github.com/jrose-apple&gt;
Status: Awaiting review
Review manager: TBD
<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions

Today, methods introduced in an extension of a class cannot override or be overridden unless the method is (implicitly or explicitly) marked @objc. This proposal lifts the blanket restriction while still enforcing safety.

Note: it's already plan-of-record that if the extension is in the same module as the class, the methods will be treated as if they were declared in the class itself. This proposal only applies to extensions declared in a different module.
<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions

This is used to add operations to system or library classes that you can customize in your own classes, as seen in the Apple AdaptivePhotos <https://developer.apple.com/library/ios/samplecode/AdaptivePhotos/Listings/AdaptiveCode_AdaptiveCode_UIViewController_PhotoContents_swift.html&gt; sample code.

extension UIViewController {
  func containsPhoto(photo: Photo) -> Bool {
    return false
  }
}
class ConversationViewController : UIViewController {
  // …
  override func containsPhoto(photo: Photo) -> Bool {
    return self.conversation.photos.contains(photo)
  }
}
Additional motivation: parity with Objective-C. If Objective-C didn't allow this, we might not have done it, but right now the answer is "if your method is ObjC-compatible, just slap an attribute on it; otherwise you're out of luck", which isn't really a sound design choice.

<https://github.com/jrose-apple/swift-evolution/tree/overridable-members-in-extensions#todays-workaround&gt;Today&#39;s Workaround

If you know every class that needs a custom implementation of a method, you can use dynamic casts to get the same effect:

extension UIViewController {
  final func containsPhoto(photo: Photo) -> Bool {
    switch self {
    case is ListTableViewController:
      return true
    case let cvc as ConversationViewController:
      return cvc.conversation.photos.contains(photo)
    default:
      return false
    }
  }
}
But this is not possible if there may be subclasses outside of the module, and it either forces all of the implementations into a single method body or requires adding dummy methods to each class.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions solution

This proposal lifts the restriction on non-@objc extension methods (and properties, and subscripts) by requiring an alternate dispatch mechanism that can be arbitrarily extended. To preserve safety and correctness, a new, narrower restriction will be put in place:

If an extension in module B is extending a class in module A, it may only override members added in module B.

Any other rule can result in two modules trying to add an override for the same method on the same class.

Note: This rule applies to @objc members as well as non-@objc members.
There is no restriction on extensions adding new overridable members. These members can be overridden by any extension in the same module (by the above rule) and by a subclass in any module, whether in the class declaration itself or in an extension in the same module.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions design

Besides safety, the other reason we didn't add this feature is because the Swift method dispatch mechanism uses a single virtual dispatch table for a class, which cannot be arbitrarily extended after the fact. The implementation would require an alternate dispatch mechanism that can be arbitrarily extended.

On Apple platforms this is implemented by the Objective-C method table; we would provide a simplified implementation of the same on Linux. For a selector we would use the mangled name of the original overridden method. These methods would still use Swift calling conventions; they're just being stored in the same lookup table as Objective-C methods.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions Evolution

As with any other method <https://github.com/apple/swift/blob/master/docs/LibraryEvolution.rst#classes&gt;, it is legal to "move" an extension method up to an extension on the base class, as long as the original declaration is not removed entirely. The new entry point will forward over to the original entry point in order to preserve binary compatibility.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions on existing code

No existing semantics will be affected. Dispatch for methods in extensions may get slower, since it's no longer using a direct call.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions considered

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions extension methods are final

This is sound, and doesn't rule out the "closed class hierarchy" partial workaround described above. However, it does prevent some reasonable patterns that were possible in Objective-C, and it is something we've seen developers try to do (with or without @objc).

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions extension methods are overridable, non-@objc methods are not

This is a practical answer, since it requires no further implementation work. We could require members in extensions to be explicitly annotated dynamic and final, respectively, so that the semantics are at least clear. However, it's not a very principled design choice: either overridable extension members are useful, or they aren't.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions extensions

The restriction that an extension cannot override a method from another module is intended for safety purposes, preventing two modules from each adding their own override. It's possible to make this a link-time failure rather than a compile-time failure by emitting a dummy symbol representing the (class, member) pair. Because of this, it may be useful to have an "I know what I'm doing" annotation that promises that no one else will add the same member; if it does happen then the program will fail to link.

(Indeed, we probably should do this anyway for @objc overrides, which run the risk of run-time collision because of Objective-C language semantics.)

If we ever have an "SPI" feature that allows public API to be restricted to certain clients, it would be reasonable to consider relaxing the safety restrictions for those clients specifically on the grounds that the library author trusts them to know what they're doing.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

[hlovatt (Howard Lovatt)]

+1 from me. I have wanted this feature to extend a class I don't own.
Equally importantly I find the present semantics confusing.

···

On Thursday, 11 February 2016, Philippe Hausler via swift-evolution < swift-evolution@swift.org> wrote:

Looks really great; this will definitely solve some problems we had with
the Foundation implementation.

On Feb 10, 2016, at 5:45 PM, Jordan Rose via swift-evolution < > swift-evolution@swift.org > <javascript:_e(%7B%7D,'cvml','swift-evolution@swift.org');>> wrote:

Hey, everyone. Here's a small feature with ABI implications, ready for
feedback. In addition to comments on the proposal itself, I'm also
interested in hearing how often this comes up for people:

- extending a class you don't own
- to add an overridable method
- where some of the overriders might be outside the current module

Jordan

https://github.com/jrose-apple/swift-evolution/blob/overridable-members-in-extensions/proposals/nnnn-overridable-members-in-extensions.md

---

Overridable Members in Extensions

   - Proposal: SE-NNNN
   - Author: Jordan Rose <https://github.com/jrose-apple&gt;
   - Status: *Awaiting review*
   - Review manager: TBD

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions;
Introduction

Today, methods introduced in an extension of a class cannot override or be
overridden unless the method is (implicitly or explicitly) marked @objc.
This proposal lifts the blanket restriction while still enforcing safety.

Note: it's already plan-of-record that if the extension is in the same
module as the class, the methods will be treated as if they were declared
in the class itself. This proposal only applies to extensions declared in a
different module.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions;
Motivation

This is used to add operations to system or library classes that you can
customize in your own classes, as seen in the Apple AdaptivePhotos
<https://developer.apple.com/library/ios/samplecode/AdaptivePhotos/Listings/AdaptiveCode_AdaptiveCode_UIViewController_PhotoContents_swift.html&gt; sample
code.

extension UIViewController {
  func containsPhoto(photo: Photo) -> Bool {
    return false
  }
}

class ConversationViewController : UIViewController {
  // …
  override func containsPhoto(photo: Photo) -> Bool {
    return self.conversation.photos.contains(photo)
  }
}

Additional motivation: parity with Objective-C. If Objective-C didn't
allow this, we might not have done it, but right now the answer is "if your
method is ObjC-compatible, just slap an attribute on it; otherwise you're
out of luck", which isn't really a sound design choice.

<https://github.com/jrose-apple/swift-evolution/tree/overridable-members-in-extensions#todays-workaround&gt;Today&#39;s
Workaround

If you know every class that needs a custom implementation of a method,
you can use dynamic casts to get the same effect:

extension UIViewController {
  final func containsPhoto(photo: Photo) -> Bool {
    switch self {
    case is ListTableViewController:
      return true
    case let cvc as ConversationViewController:
      return cvc.conversation.photos.contains(photo)
    default:
      return false
    }
  }
}

But this is not possible if there may be subclasses outside of the module,
and it either forces all of the implementations into a single method body
or requires adding dummy methods to each class.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions
solution

This proposal lifts the restriction on non-@objc extension methods (and
properties, and subscripts) by requiring an alternate dispatch mechanism
that can be arbitrarily extended. To preserve safety and correctness, a
new, narrower restriction will be put in place:

*If an extension in module B is extending a class in module A, it may only
override members added in module B.*

Any other rule can result in two modules trying to add an override for the
same method on the same class.

Note: This rule applies to @objc members as well as non-@objc members.

There is no restriction on extensions adding new *overridable* members.
These members can be overridden by any extension in the same module (by the
above rule) and by a subclass in any module, whether in the class
declaration itself or in an extension in the same module.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions
design

Besides safety, the other reason we didn't add this feature is because the
Swift method dispatch mechanism uses a single virtual dispatch table for a
class, which cannot be arbitrarily extended after the fact. The
implementation would require an alternate dispatch mechanism that *can* be
arbitrarily extended.

On Apple platforms this is implemented by the Objective-C method table; we
would provide a simplified implementation of the same on Linux. For a
selector we would use the mangled name of the original overridden method.
These methods would still use Swift calling conventions; they're just being
stored in the same lookup table as Objective-C methods.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions
Evolution

As with any other method
<https://github.com/apple/swift/blob/master/docs/LibraryEvolution.rst#classes&gt;,
it is legal to "move" an extension method up to an extension on the base
class, as long as the original declaration is not removed entirely. The new
entry point will forward over to the original entry point in order to
preserve binary compatibility.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions
on existing code

No existing semantics will be affected. Dispatch for methods in extensions
may get slower, since it's no longer using a direct call.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions
considered
<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions
extension methods are final

This is sound, and doesn't rule out the "closed class hierarchy" partial
workaround described above. However, it does prevent some reasonable
patterns that were possible in Objective-C, and it is something we've seen
developers try to do (with or without @objc).

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions;
@objc extension methods are overridable, non-@objc methods are not

This is a practical answer, since it requires no further implementation
work. We could require members in extensions to be explicitly annotated
dynamic and final, respectively, so that the semantics are at least
clear. However, it's not a very principled design choice: either
overridable extension members are useful, or they aren't.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions
extensions

The restriction that an extension cannot override a method from another
module is intended for safety purposes, preventing two modules from each
adding their own override. It's possible to make this a link-time failure
rather than a compile-time failure by emitting a dummy symbol representing
the (class, member) pair. Because of this, it may be useful to have an "I
know what I'm doing" annotation that promises that no one else will add the
same member; if it does happen then the program will fail to link.

(Indeed, we probably should do this anyway for @objc overrides, which run
the risk of run-time collision because of Objective-C language semantics.)
If we ever have an "SPI" feature that allows public API to be restricted
to certain clients, it would be reasonable to consider relaxing the safety
restrictions for those clients specifically on the grounds that the library
author trusts them to know what they're doing.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
<javascript:_e(%7B%7D,'cvml','swift-evolution@swift.org');>
https://lists.swift.org/mailman/listinfo/swift-evolution

--
-- Howard.

[jrose (Jordan Rose)]

[Proposal: https://github.com/jrose-apple/swift-evolution/blob/overridable-members-in-extensions/proposals/nnnn-overridable-members-in-extensions.md\]

Sorry, I think you're misinterpreting the proposal. The note in the beginning says that for someone else's library, you don't need to care whether something came from a class or an extension. That doesn't work today, but it's already a direction we're planning to go.

This proposal covers the case where the extension is for a class in another module. In this case, things are trickier, because any number of people could be extending classes they don't own. That's why the proposal has the new (bolded) safety rule.

Jordan

···

On Feb 18, 2016, at 2:07 , Marco Masser <lists@duckcode.com> wrote:

+1 because I think this is a good thing in terms of consistency and the principle of least astonishment

The introduction states in the note that in the same module, this is planned to work anyways, so if I understand correctly, there are two groups of people who are impacted by the same thing:
- Library/framework authors can freely decide to split up their classes into separate extensions without worrying about impacting clients of that library/framework.
- Library/framework clients aren’t affected by the author’s choice of splitting up their classes into separate extensions.

It would really be unfortunate if I couldn’t override a specific method of NSButton just because the App Kit team implemented that method in an extension instead of the class proper.

I tend to split up my classes in a couple of extensions that each group members that somehow belong together, e.g. a separate extension for all @IBActions, one extension implementing conformance to a protocol, one private extension with methods dealing with X, etc.
Right now, I always tend to write all methods that override something into the class itself, not an extension because I know there are weird issues you can run into (e.g. https://bugs.swift.org/browse/SR-584\). I’m looking forward to when these will be fixed for classes in the same target (as indicated by the note in the introduction), but when the proposal in question gets implemented, this would also work as expected for classes in different modules.
Therefore: +1 for consistency and least astonishment.

Marco

On 2016-02-11, at 02:45, Jordan Rose via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Hey, everyone. Here's a small feature with ABI implications, ready for feedback. In addition to comments on the proposal itself, I'm also interested in hearing how often this comes up for people:

- extending a class you don't own
- to add an overridable method
- where some of the overriders might be outside the current module

Jordan

https://github.com/jrose-apple/swift-evolution/blob/overridable-members-in-extensions/proposals/nnnn-overridable-members-in-extensions.md

---

Overridable Members in Extensions

Proposal: SE-NNNN
Author: Jordan Rose <https://github.com/jrose-apple&gt;
Status: Awaiting review
Review manager: TBD
<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions

Today, methods introduced in an extension of a class cannot override or be overridden unless the method is (implicitly or explicitly) marked @objc. This proposal lifts the blanket restriction while still enforcing safety.

Note: it's already plan-of-record that if the extension is in the same module as the class, the methods will be treated as if they were declared in the class itself. This proposal only applies to extensions declared in a different module.
<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions

This is used to add operations to system or library classes that you can customize in your own classes, as seen in the Apple AdaptivePhotos <https://developer.apple.com/library/ios/samplecode/AdaptivePhotos/Listings/AdaptiveCode_AdaptiveCode_UIViewController_PhotoContents_swift.html&gt; sample code.

extension UIViewController {
  func containsPhoto(photo: Photo) -> Bool {
    return false
  }
}
class ConversationViewController : UIViewController {
  // …
  override func containsPhoto(photo: Photo) -> Bool {
    return self.conversation.photos.contains(photo)
  }
}
Additional motivation: parity with Objective-C. If Objective-C didn't allow this, we might not have done it, but right now the answer is "if your method is ObjC-compatible, just slap an attribute on it; otherwise you're out of luck", which isn't really a sound design choice.

<https://github.com/jrose-apple/swift-evolution/tree/overridable-members-in-extensions#todays-workaround&gt;Today&#39;s Workaround

If you know every class that needs a custom implementation of a method, you can use dynamic casts to get the same effect:

extension UIViewController {
  final func containsPhoto(photo: Photo) -> Bool {
    switch self {
    case is ListTableViewController:
      return true
    case let cvc as ConversationViewController:
      return cvc.conversation.photos.contains(photo)
    default:
      return false
    }
  }
}
But this is not possible if there may be subclasses outside of the module, and it either forces all of the implementations into a single method body or requires adding dummy methods to each class.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions solution

This proposal lifts the restriction on non-@objc extension methods (and properties, and subscripts) by requiring an alternate dispatch mechanism that can be arbitrarily extended. To preserve safety and correctness, a new, narrower restriction will be put in place:

If an extension in module B is extending a class in module A, it may only override members added in module B.

Any other rule can result in two modules trying to add an override for the same method on the same class.

Note: This rule applies to @objc members as well as non-@objc members.
There is no restriction on extensions adding new overridable members. These members can be overridden by any extension in the same module (by the above rule) and by a subclass in any module, whether in the class declaration itself or in an extension in the same module.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions design

Besides safety, the other reason we didn't add this feature is because the Swift method dispatch mechanism uses a single virtual dispatch table for a class, which cannot be arbitrarily extended after the fact. The implementation would require an alternate dispatch mechanism that can be arbitrarily extended.

On Apple platforms this is implemented by the Objective-C method table; we would provide a simplified implementation of the same on Linux. For a selector we would use the mangled name of the original overridden method. These methods would still use Swift calling conventions; they're just being stored in the same lookup table as Objective-C methods.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions Evolution

As with any other method <https://github.com/apple/swift/blob/master/docs/LibraryEvolution.rst#classes&gt;, it is legal to "move" an extension method up to an extension on the base class, as long as the original declaration is not removed entirely. The new entry point will forward over to the original entry point in order to preserve binary compatibility.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions on existing code

No existing semantics will be affected. Dispatch for methods in extensions may get slower, since it's no longer using a direct call.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions considered

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions extension methods are final

This is sound, and doesn't rule out the "closed class hierarchy" partial workaround described above. However, it does prevent some reasonable patterns that were possible in Objective-C, and it is something we've seen developers try to do (with or without @objc).

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions extension methods are overridable, non-@objc methods are not

This is a practical answer, since it requires no further implementation work. We could require members in extensions to be explicitly annotated dynamic and final, respectively, so that the semantics are at least clear. However, it's not a very principled design choice: either overridable extension members are useful, or they aren't.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions extensions

The restriction that an extension cannot override a method from another module is intended for safety purposes, preventing two modules from each adding their own override. It's possible to make this a link-time failure rather than a compile-time failure by emitting a dummy symbol representing the (class, member) pair. Because of this, it may be useful to have an "I know what I'm doing" annotation that promises that no one else will add the same member; if it does happen then the program will fail to link.

(Indeed, we probably should do this anyway for @objc overrides, which run the risk of run-time collision because of Objective-C language semantics.)

If we ever have an "SPI" feature that allows public API to be restricted to certain clients, it would be reasonable to consider relaxing the safety restrictions for those clients specifically on the grounds that the library author trusts them to know what they're doing.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

[Andrew (Andrew Bennett)]

Sounds interesting, I know I've wanted to do similar, and had some weird
bugs.

Many of my concerns have been discussed already, I've added a few more
below:

I'm also a little concerned about the potential for unexpected side-effects
from methods called by the original class, *this* seems to break the
principle of "least astonishment", you're less likely to know the base
classes implementation than that of your extension.

Other points worth considering:

   - Can methods be marked as final, or something else if it's unsafe to
   override?
   - What about methods marked as inline, called by the super-class, is
   this unexpected?
   - *If* most people only want this for testing should it only be allowed
   on classes imported with @testable (only restrict this with (unsafe)
   ObjC classes?).
   - You probably want to swap/replace the implementation rather than just
   override it for testing.
   - I'm presuming that this will not allow you to swap out one
   implementation for another? If not then it may be necessary to be able to
   call or refer to the original version of the method.

···

On Thu, Feb 18, 2016 at 7:07 PM, Marco Masser via swift-evolution < swift-evolution@swift.org> wrote:

+1 because I think this is a good thing in terms of consistency and the
principle of least astonishment

The introduction states in the note that in the same module, this is
planned to work anyways, so if I understand correctly, there are two groups
of people who are impacted by the same thing:
- Library/framework authors can freely decide to split up their classes
into separate extensions without worrying about impacting clients of that
library/framework.
- Library/framework clients aren’t affected by the author’s choice of
splitting up their classes into separate extensions.

It would really be unfortunate if I couldn’t override a specific method of
NSButton just because the App Kit team implemented that method in an
extension instead of the class proper.

I tend to split up my classes in a couple of extensions that each group
members that somehow belong together, e.g. a separate extension for all
@IBActions, one extension implementing conformance to a protocol, one
private extension with methods dealing with X, etc.
Right now, I always tend to write all methods that override something into
the class itself, not an extension because I know there are weird issues
you can run into (e.g. https://bugs.swift.org/browse/SR-584\). I’m looking
forward to when these will be fixed for classes in the same target (as
indicated by the note in the introduction), but when the proposal in
question gets implemented, this would also work as expected for classes in
different modules.
Therefore: +1 for consistency and least astonishment.

Marco

On 2016-02-11, at 02:45, Jordan Rose via swift-evolution < > swift-evolution@swift.org> wrote:

Hey, everyone. Here's a small feature with ABI implications, ready for
feedback. In addition to comments on the proposal itself, I'm also
interested in hearing how often this comes up for people:

- extending a class you don't own
- to add an overridable method
- where some of the overriders might be outside the current module

Jordan

https://github.com/jrose-apple/swift-evolution/blob/overridable-members-in-extensions/proposals/nnnn-overridable-members-in-extensions.md

---

Overridable Members in Extensions

   - Proposal: SE-NNNN
   - Author: Jordan Rose <https://github.com/jrose-apple&gt;
   - Status: *Awaiting review*
   - Review manager: TBD

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions;
Introduction

Today, methods introduced in an extension of a class cannot override or be
overridden unless the method is (implicitly or explicitly) marked @objc.
This proposal lifts the blanket restriction while still enforcing safety.

Note: it's already plan-of-record that if the extension is in the same
module as the class, the methods will be treated as if they were declared
in the class itself. This proposal only applies to extensions declared in a
different module.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions;
Motivation

This is used to add operations to system or library classes that you can
customize in your own classes, as seen in the Apple AdaptivePhotos
<https://developer.apple.com/library/ios/samplecode/AdaptivePhotos/Listings/AdaptiveCode_AdaptiveCode_UIViewController_PhotoContents_swift.html&gt; sample
code.

extension UIViewController {
  func containsPhoto(photo: Photo) -> Bool {
    return false
  }
}

class ConversationViewController : UIViewController {
  // …
  override func containsPhoto(photo: Photo) -> Bool {
    return self.conversation.photos.contains(photo)
  }
}

Additional motivation: parity with Objective-C. If Objective-C didn't
allow this, we might not have done it, but right now the answer is "if your
method is ObjC-compatible, just slap an attribute on it; otherwise you're
out of luck", which isn't really a sound design choice.

<https://github.com/jrose-apple/swift-evolution/tree/overridable-members-in-extensions#todays-workaround&gt;Today&#39;s
Workaround

If you know every class that needs a custom implementation of a method,
you can use dynamic casts to get the same effect:

extension UIViewController {
  final func containsPhoto(photo: Photo) -> Bool {
    switch self {
    case is ListTableViewController:
      return true
    case let cvc as ConversationViewController:
      return cvc.conversation.photos.contains(photo)
    default:
      return false
    }
  }
}

But this is not possible if there may be subclasses outside of the module,
and it either forces all of the implementations into a single method body
or requires adding dummy methods to each class.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions
solution

This proposal lifts the restriction on non-@objc extension methods (and
properties, and subscripts) by requiring an alternate dispatch mechanism
that can be arbitrarily extended. To preserve safety and correctness, a
new, narrower restriction will be put in place:

*If an extension in module B is extending a class in module A, it may only
override members added in module B.*

Any other rule can result in two modules trying to add an override for the
same method on the same class.

Note: This rule applies to @objc members as well as non-@objc members.

There is no restriction on extensions adding new *overridable* members.
These members can be overridden by any extension in the same module (by the
above rule) and by a subclass in any module, whether in the class
declaration itself or in an extension in the same module.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions
design

Besides safety, the other reason we didn't add this feature is because the
Swift method dispatch mechanism uses a single virtual dispatch table for a
class, which cannot be arbitrarily extended after the fact. The
implementation would require an alternate dispatch mechanism that *can* be
arbitrarily extended.

On Apple platforms this is implemented by the Objective-C method table; we
would provide a simplified implementation of the same on Linux. For a
selector we would use the mangled name of the original overridden method.
These methods would still use Swift calling conventions; they're just being
stored in the same lookup table as Objective-C methods.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions
Evolution

As with any other method
<https://github.com/apple/swift/blob/master/docs/LibraryEvolution.rst#classes&gt;,
it is legal to "move" an extension method up to an extension on the base
class, as long as the original declaration is not removed entirely. The new
entry point will forward over to the original entry point in order to
preserve binary compatibility.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions
on existing code

No existing semantics will be affected. Dispatch for methods in extensions
may get slower, since it's no longer using a direct call.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions
considered
<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions
extension methods are final

This is sound, and doesn't rule out the "closed class hierarchy" partial
workaround described above. However, it does prevent some reasonable
patterns that were possible in Objective-C, and it is something we've seen
developers try to do (with or without @objc).

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions;
@objc extension methods are overridable, non-@objc methods are not

This is a practical answer, since it requires no further implementation
work. We could require members in extensions to be explicitly annotated
dynamic and final, respectively, so that the semantics are at least
clear. However, it's not a very principled design choice: either
overridable extension members are useful, or they aren't.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions
extensions

The restriction that an extension cannot override a method from another
module is intended for safety purposes, preventing two modules from each
adding their own override. It's possible to make this a link-time failure
rather than a compile-time failure by emitting a dummy symbol representing
the (class, member) pair. Because of this, it may be useful to have an "I
know what I'm doing" annotation that promises that no one else will add the
same member; if it does happen then the program will fail to link.

(Indeed, we probably should do this anyway for @objc overrides, which run
the risk of run-time collision because of Objective-C language semantics.)
If we ever have an "SPI" feature that allows public API to be restricted
to certain clients, it would be reasonable to consider relaxing the safety
restrictions for those clients specifically on the grounds that the library
author trusts them to know what they're doing.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[marco.masser]

[Proposal: https://github.com/jrose-apple/swift-evolution/blob/overridable-members-in-extensions/proposals/nnnn-overridable-members-in-extensions.md\]

[Proposal: https://github.com/jrose-apple/swift-evolution/blob/overridable-members-in-extensions/proposals/nnnn-overridable-members-in-extensions.md\]

Sorry, I think you're misinterpreting the proposal. The note in the beginning says that for someone else's library, you don't need to care whether something came from a class or an extension. That doesn't work today, but it's already a direction we're planning to go.

This proposal covers the case where the extension is for a class in another module. In this case, things are trickier, because any number of people could be extending classes they don't own. That's why the proposal has the new (bolded) safety rule.

Sorry, when re-reading, I think my original answer wasn’t clear – either that, or I really misinterpreted the proposal!

I tried to explain why I think the proposal is a good idea in three stages:

1. By stating how it works – or will work – when everything is in the same module:

The introduction states in the note that in the same module, this is planned to work anyways, so if I understand correctly, there are two groups of people who are impacted by the same thing:
- Library/framework authors can freely decide to split up their classes into separate extensions without worrying about impacting clients of that library/framework.
- Library/framework clients aren’t affected by the author’s choice of splitting up their classes into separate extensions.

It would really be unfortunate if I couldn’t override a specific method of NSButton just because the App Kit team implemented that method in an extension instead of the class proper.

(Note: the last sentence is not a complaint but an example of why the way it works – or will work – is a good idea).

2. By explaining how I tend to do things:

I tend to split up my classes in a couple of extensions that each group members that somehow belong together, e.g. a separate extension for all @IBActions, one extension implementing conformance to a protocol, one private extension with methods dealing with X, etc.
Right now, I always tend to write all methods that override something into the class itself, not an extension because I know there are weird issues you can run into (e.g. https://bugs.swift.org/browse/SR-584\). I’m looking forward to when these will be fixed for classes in the same target (as indicated by the note in the introduction), […]

3. By concluding why I think the proposal is a good idea:

[…] but when the proposal in question gets implemented, this would also work as expected for classes in different modules.
Therefore: +1 for consistency and least astonishment.

What I mean by that last point is that by implementing this proposal, my habit of doing things (splitting up a class into a couple of extensions) would not only work when everything is in the same module, but also when the class and its subclasses are split across different modules. If I understand correctly, the proposal takes overriding members out of the decision making process for splitting classes into extensions, i.e. whether those extensions can override something or not depending on whether the class or superclass is defined in the same module or not. I would therefore say that it improves consistency in that it doesn’t matter in which module I write an overriding member in an extension: it works the same everywhere.

I hope this clears up my original answer. Either by getting the point across better, or by proving that I misinterpreted the proposal

Marco

···

On 2016-02-18, at 18:52, Jordan Rose <jordan_rose@apple.com> wrote:

[gparker42 (Greg Parker)]

Hey, everyone. Here's a small feature with ABI implications, ready for feedback. In addition to comments on the proposal itself, I'm also interested in hearing how often this comes up for people:

- extending a class you don't own
- to add an overridable method
- where some of the overriders might be outside the current module

Jordan

https://github.com/jrose-apple/swift-evolution/blob/overridable-members-in-extensions/proposals/nnnn-overridable-members-in-extensions.md

---

Overridable Members in Extensions

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions Evolution
As with any other method <https://github.com/apple/swift/blob/master/docs/LibraryEvolution.rst#classes&gt;, it is legal to "move" an extension method up to an extension on the base class, as long as the original declaration is not removed entirely. The new entry point will forward over to the original entry point in order to preserve binary compatibility.

What happens if the base class adds an independent implementation of the same method?

If it's an @objc method, you're hosed, just like you always were, but at least you'll (probably) find out when you recompile. I don't think there's anything we can do about this other than banning @objc methods in extensions, and I don't think that'll fly.

If it's a Swift method, it's not the "same" method; it's just another method with the same full-name. Existing binaries won't conflict. When you recompile, though, you'll get a conflict, and we don't currently have any annotations that allow you to fix that conflict, or for subclasses in a client module to deal with the intermediate module not yet having fixed the conflict. This isn't specific to extensions, though; you have the same problem with methods declared in the class body.

Good. I think that is the best result we can hope for here.

<GitHub - jrose-apple/swift-evolution at overridable-members-in-extensions extensions

The restriction that an extension cannot override a method from another module is intended for safety purposes, preventing two modules from each adding their own override. It's possible to make this a link-time failure rather than a compile-time failure by emitting a dummy symbol representing the (class, member) pair.

Dummy symbol enforcement only helps when two conflicting modules have a linkage relationship (i.e. one links to the other). It won't catch the case where two independent modules collide.

If two linker modules provide the same public symbol, that won't cause an issue?

No, at least not with OS X two-level symbol lookup. If some linkage unit tried to use that symbol and linked to two different providers of the symbol then you would get a link-time error. Otherwise there wouldn't be any link-time or load-time problems.

···

On Feb 10, 2016, at 7:57 PM, Jordan Rose <jordan_rose@apple.com> wrote:

On Feb 10, 2016, at 18:29 , Greg Parker <gparker@apple.com <mailto:gparker@apple.com>> wrote:

On Feb 10, 2016, at 5:45 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

--
Greg Parker gparker@apple.com Runtime Wrangler

[jrose (Jordan Rose)]

[Proposal: https://github.com/jrose-apple/swift-evolution/blob/overridable-members-in-extensions/proposals/nnnn-overridable-members-in-extensions.md\]

Sounds interesting, I know I've wanted to do similar, and had some weird bugs.

Many of my concerns have been discussed already, I've added a few more below:

I'm also a little concerned about the potential for unexpected side-effects from methods called by the original class, *this* seems to break the principle of "least astonishment", you're less likely to know the base classes implementation than that of your extension.

Other points worth considering:

Good questions! I'll answer them here but also update the document later today.

Can methods be marked as final, or something else if it's unsafe to override?

Yep, `final` should work just like it does today.

What about methods marked as inline, called by the super-class, is this unexpected?

Inlining is discussed in great depth in the Library Evolution doc <http://jrose-apple.github.io/swift-library-evolution/&gt;, but the relevant points are that marking something inlineable doesn't guarantee it'll be inlined, and that you can't inline a call unless you know the dynamic type of the receiver (and therefore know which override will be called).

If most people only want this for testing should it only be allowed on classes imported with @testable (only restrict this with (unsafe) ObjC classes?).

I know of at least one non-testing use case that I probably can't share, so let's just say I don't think we need to artificially limit the feature. (It's at least no worse than Objective-C.)

What we could do is say that methods in extensions of classes you don't own are @nonobjc by default, and that you have to opt into making them @objc. What do you think?

You probably want to swap/replace the implementation rather than just override it for testing.

That's out of scope for this proposal. If you're interested in this, I'd suggest coming up with a version of 'dynamic' that works for all Swift methods, not just @objc ones, and then further annotating that as "but only for testing".

I'm presuming that this will not allow you to swap out one implementation for another? If not then it may be necessary to be able to call or refer to the original version of the method.

This is only for subclass-overriding, not ObjC-category-style method replacement.

Jordan

···

On Feb 18, 2016, at 5:13 , Andrew Bennett <cacoyi@gmail.com> wrote:

[jrose (Jordan Rose)]

[Proposal: https://github.com/jrose-apple/swift-evolution/blob/overridable-members-in-extensions/proposals/nnnn-overridable-members-in-extensions.md\] <https://github.com/jrose-apple/swift-evolution/blob/overridable-members-in-extensions/proposals/nnnn-overridable-members-in-extensions.md]&gt;

···

On Feb 18, 2016, at 10:29, Erica Sadun <erica@ericasadun.com> wrote:

I'm presuming that this will not allow you to swap out one implementation for another? If not then it may be necessary to be able to call or refer to the original version of the method.

This is only for subclass-overriding, not ObjC-category-style method replacement.

Jordan

I hate to go all Abrahams on you, but do you have any use-cases where you're delivered a subclass and a superclass in a module, where there's no subclass implementation, and you want to add one that's an override via an extension rather than just sub-subclassing?

That violates the "what if two modules did this? <The Old New Thing; rule. Either you own the class and you can do whatever you want to it (accepted work), or you own the method and you can override it all you want (this proposal). Anything else is unsafe (and not future-proof, if the library is part of the OS), and therefore at the very least outside the scope of this proposal.

(It's also harder to implement with a vtable instead of an ObjC-style method table, because the original class has all of its overrides and non-overrides listed in the constant section of its binary.)

Jordan

[Erica_Sadun]

I'm presuming that this will not allow you to swap out one implementation for another? If not then it may be necessary to be able to call or refer to the original version of the method.

This is only for subclass-overriding, not ObjC-category-style method replacement.

Jordan

I hate to go all Abrahams on you, but do you have any use-cases where you're delivered a subclass and a superclass in a module, where there's no subclass implementation, and you want to add one that's an override via an extension rather than just sub-subclassing?

-- E"brahams"

[Erica_Sadun]

Going completely off into left field: Is there an ongoing discussion of module conflicts? Where there are two modules named, for example, SwiftString, and module implementation overlaps, and strategies for partial imports? Sort of "What if importing two modules does this?"

-- E

···

On Feb 18, 2016, at 11:38 AM, Jordan Rose <jordan_rose@apple.com> wrote:

[Proposal: https://github.com/jrose-apple/swift-evolution/blob/overridable-members-in-extensions/proposals/nnnn-overridable-members-in-extensions.md\] <https://github.com/jrose-apple/swift-evolution/blob/overridable-members-in-extensions/proposals/nnnn-overridable-members-in-extensions.md]&gt;

On Feb 18, 2016, at 10:29, Erica Sadun <erica@ericasadun.com <mailto:erica@ericasadun.com>> wrote:

I'm presuming that this will not allow you to swap out one implementation for another? If not then it may be necessary to be able to call or refer to the original version of the method.

This is only for subclass-overriding, not ObjC-category-style method replacement.

Jordan

I hate to go all Abrahams on you, but do you have any use-cases where you're delivered a subclass and a superclass in a module, where there's no subclass implementation, and you want to add one that's an override via an extension rather than just sub-subclassing?

That violates the "what if two modules did this? <The Old New Thing; rule. Either you own the class and you can do whatever you want to it (accepted work), or you own the method and you can override it all you want (this proposal). Anything else is unsafe (and not future-proof, if the library is part of the OS), and therefore at the very least outside the scope of this proposal.

(It's also harder to implement with a vtable instead of an ObjC-style method table, because the original class has all of its overrides and non-overrides listed in the constant section of its binary.)

Jordan

[jrose (Jordan Rose)]

Below the module level (i.e. "assuming that module names are unique"), we've been going by the idea that nothing will conflict at the binary/symbol/runtime level as long as you don't touch Objective-C. That doesn't always reflect up happily at the source level (it's not too difficult to contrive unresolvable ambiguities with extension members), but the attitude there has been that we'll be able to dig ourselves out with language extensions that let you specify modules in various places explicitly.

When ObjC enters the picture, members have to be careful of selector conflicts. That's not wonderful, but it's no worse than things were in Objective-C itself, and making 'private' things @nonobjc by default may help with some of the inadvertant conflicts people have hit in the past. Types will only conflict if explicitly renamed (classes) or marked @objc (enums).

But when modules themselves conflict, we're in trouble. Module name uniqueness is a pretty strong part of guaranteeing the uniqueness of everything else. You can't just rename them if there's a conflict, because they show up in, say, NSCoding archives. We've mostly just pushed the ObjC prefix problem up a level.

Max from the Package Manager team may have more thoughts here; over on swift-build-dev this is a very practical problem.

Jordan

···

On Feb 18, 2016, at 11:10, Erica Sadun <erica@ericasadun.com> wrote:

On Feb 18, 2016, at 11:38 AM, Jordan Rose <jordan_rose@apple.com <mailto:jordan_rose@apple.com>> wrote:

[Proposal: https://github.com/jrose-apple/swift-evolution/blob/overridable-members-in-extensions/proposals/nnnn-overridable-members-in-extensions.md\] <https://github.com/jrose-apple/swift-evolution/blob/overridable-members-in-extensions/proposals/nnnn-overridable-members-in-extensions.md]&gt;

On Feb 18, 2016, at 10:29, Erica Sadun <erica@ericasadun.com <mailto:erica@ericasadun.com>> wrote:

I'm presuming that this will not allow you to swap out one implementation for another? If not then it may be necessary to be able to call or refer to the original version of the method.

This is only for subclass-overriding, not ObjC-category-style method replacement.

Jordan

I hate to go all Abrahams on you, but do you have any use-cases where you're delivered a subclass and a superclass in a module, where there's no subclass implementation, and you want to add one that's an override via an extension rather than just sub-subclassing?

That violates the "what if two modules did this? <The Old New Thing; rule. Either you own the class and you can do whatever you want to it (accepted work), or you own the method and you can override it all you want (this proposal). Anything else is unsafe (and not future-proof, if the library is part of the OS), and therefore at the very least outside the scope of this proposal.

(It's also harder to implement with a vtable instead of an ObjC-style method table, because the original class has all of its overrides and non-overrides listed in the constant section of its binary.)

Jordan

Going completely off into left field: Is there an ongoing discussion of module conflicts? Where there are two modules named, for example, SwiftString, and module implementation overlaps, and strategies for partial imports? Sort of "What if importing two modules does this?"

[jrose (Jordan Rose)]

[Proposal: https://github.com/jrose-apple/swift-evolution/blob/overridable-members-in-extensions/proposals/nnnn-overridable-members-in-extensions.md\]

···

On Feb 19, 2016, at 2:23 , Marco Masser <lists@duckcode.com> wrote:

On 2016-02-18, at 18:52, Jordan Rose <jordan_rose@apple.com <mailto:jordan_rose@apple.com>> wrote:

Sorry, I think you're misinterpreting the proposal. The note in the beginning says that for someone else's library, you don't need to care whether something came from a class or an extension. That doesn't work today, but it's already a direction we're planning to go.

This proposal covers the case where the extension is for a class in another module. In this case, things are trickier, because any number of people could be extending classes they don't own. That's why the proposal has the new (bolded) safety rule.

Sorry, when re-reading, I think my original answer wasn’t clear – either that, or I really misinterpreted the proposal!

I tried to explain why I think the proposal is a good idea in three stages:

1. By stating how it works – or will work – when everything is in the same module:

The introduction states in the note that in the same module, this is planned to work anyways, so if I understand correctly, there are two groups of people who are impacted by the same thing:
- Library/framework authors can freely decide to split up their classes into separate extensions without worrying about impacting clients of that library/framework.
- Library/framework clients aren’t affected by the author’s choice of splitting up their classes into separate extensions.

It would really be unfortunate if I couldn’t override a specific method of NSButton just because the App Kit team implemented that method in an extension instead of the class proper.

(Note: the last sentence is not a complaint but an example of why the way it works – or will work – is a good idea).

2. By explaining how I tend to do things:

I tend to split up my classes in a couple of extensions that each group members that somehow belong together, e.g. a separate extension for all @IBActions, one extension implementing conformance to a protocol, one private extension with methods dealing with X, etc.
Right now, I always tend to write all methods that override something into the class itself, not an extension because I know there are weird issues you can run into (e.g. https://bugs.swift.org/browse/SR-584\). I’m looking forward to when these will be fixed for classes in the same target (as indicated by the note in the introduction), […]

3. By concluding why I think the proposal is a good idea:

[…] but when the proposal in question gets implemented, this would also work as expected for classes in different modules.
Therefore: +1 for consistency and least astonishment.

What I mean by that last point is that by implementing this proposal, my habit of doing things (splitting up a class into a couple of extensions) would not only work when everything is in the same module, but also when the class and its subclasses are split across different modules. If I understand correctly, the proposal takes overriding members out of the decision making process for splitting classes into extensions, i.e. whether those extensions can override something or not depending on whether the class or superclass is defined in the same module or not. I would therefore say that it improves consistency in that it doesn’t matter in which module I write an overriding member in an extension: it works the same everywhere.

I hope this clears up my original answer. Either by getting the point across better, or by proving that I misinterpreted the proposal

Yes, sorry, the proposal doesn't quite do that, because of the new safety rule:

"If an extension in module B is extending a class in module A, the extension may only override members added in module B."

…by the same reasoning in my reply to Erica: allowing this would violate the "what if two modules did this? <The Old New Thing; rule. You know that you've broken up your plain old app into multiple modules, and that no one else is going to extend the classes you defined there, but the compiler doesn't. (And moreover, it's not as easy to add overrides after the fact with Swift as it is in Objective-C.)

This proposal is more about adding new overridable members in extensions than it is about overriding existing methods.

Jordan

[Max_Howell]

But when modules themselves conflict, we're in trouble. Module name uniqueness is a pretty strong part of guaranteeing the uniqueness of everything else. You can't just rename them if there's a conflict, because they show up in, say, NSCoding archives. We've mostly just pushed the ObjC prefix problem up a level.

Max from the Package Manager team may have more thoughts here; over on swift-build-dev this is a very practical problem.

There is no conversation on this topic yet. For me there’s a few solutions:

* Namespacing that the PM can enforce but can be transparent (mostly) to users. Specifically, I compile modules with an additional namespace in front (for example, with a github project this could be the username) (this would presumably be a flag to the compiler). Then in general use this is hidden, you import modules the same. It’s only when both modules are to be imported the user has to be specific. This option obviously would require buy-in from Swift-core as well as lots of evolution discussion.
* SwiftPM mangles module names transparently. This would require a pre-processor step that transforms all sources into a second form with the mangled import names.

2) is obviously gross, but the option exists.

If you want us to get started discussing this then by all means copy this to a new evolution thread. But otherwise I’ll start the conversation in a few weeks.

The problem with 1) is it may not be doable in a Swift 3 timeframe, but bringing the conversation up will at least confirm or deny that.

[marco.masser]

[Proposal: https://github.com/jrose-apple/swift-evolution/blob/overridable-members-in-extensions/proposals/nnnn-overridable-members-in-extensions.md\]

···

On 2016-02-19, at 18:16, Jordan Rose <jordan_rose@apple.com> wrote:

On Feb 19, 2016, at 2:23 , Marco Masser <lists@duckcode.com <mailto:lists@duckcode.com>> wrote:

[…]

I hope this clears up my original answer. Either by getting the point across better, or by proving that I misinterpreted the proposal

Yes, sorry, the proposal doesn't quite do that, because of the new safety rule:

"If an extension in module B is extending a class in module A, the extension may only override members added in module B."

…by the same reasoning in my reply to Erica: allowing this would violate the "what if two modules did this? <The Old New Thing; rule. You know that you've broken up your plain old app into multiple modules, and that no one else is going to extend the classes you defined there, but the compiler doesn't. (And moreover, it's not as easy to add overrides after the fact with Swift as it is in Objective-C.)

This proposal is more about adding new overridable members in extensions than it is about overriding existing methods.

Ah, I think I got it now. Sorry for going off into the weeds here. I’ll give the proposal and this thread a thorough read again and think about it some more.

Thank you for taking the time to explain this!

Marco