Allow `override` of `open` methods in & from extensions in same file as main class

One-line Summary

Treat extensions of a struct / class that are in the same file as the main declaration as if they are in the same block

Current Use

I use extension a lot to clarify my code, separating the main declaration of a class/struct and its properties from the methods I define on it (a pattern I’ve seen in Swift a lot in Swift’s standard library, as well).

// MARK: -SimpleExample
class SimpleExample {
    var foo: Int = 0
}

// MARK: methods
extension SimpleExample {
    func increment() { foo += 1 }
}

The Problems

This is great, but there are some limitations on extension that seem like missing compiler features and that make it harder to code cleanly:

⑴ You can’t override a method that was defined in an extension of a superclass in an subclass, so:

class SimpleChild : SimpleExample {
    override open func increment() { foo += 2 }
}

gives the error Overriding non-@objc declarations from extensions is not supported

⑵ Related but slightly different, you also can’t override methods declared in a class in an extension to a subclass:

// MARK: -SimpleExample
class SimpleExample {
    var foo: Int = 0

    open func decrement() { foo -= 1 }
}

// MARK: -SimpleChild
class SimpleChild : SimpleExample {
}

// MARK: methods
extension SimpleChild {
    override open func decrement() { foo -= 2 }
}

also gives the error Overriding non-@objc declarations from extensions is not supported

⑶ In a stretch goal, I’d like to see stored properties allowed in extensions (from the same file) as well:

class SimpleExample {
}

// MARK: methods
private extension SimpleExample {
    var foo: Int
}

this currently gives the error Extensions must not contain stored properties


Simple, Hand-wavy Solution

The thought occurred to me that a quick way to achieve these goals for extensions in the same file as the main class/struct definition might be to have the compiler smoosh all extensions in the main file into a single big block. Obviously I’m no fancy compilerist so this could maybe be more difficult than I think, or maybe it has consequences I haven’t thought of, but I have run the idea past an Xcode person who thought it sounded reasonable.

Obviously it’d require a little more work than just, like, removing the words “extension” from the main file, because extensions can have different modifiers on them (public, @objc, etc), but my thought it is the compiler could just remember what modifiers were requested on the extension block and apply them to each individual item inside the block when forming the MEGA-BLOCK.


Effects on Existing Source / Binaries / ABI / API

It’s possible I haven’t thought through all the cases, but assuming I didn’t miss anything (or the things I missed can be worked around) this change is intended to be totally additive, and should have no effect on existing code / binaries.

14 Likes

Way back in 2016 I was looking at allowing overridable methods in cross-module extensions (with an implementation similar to objc_msgSend), as [Pitch] Overridable Members in Extensions. I never got the chance to work on it, but at the time I included this:

Note: it's already plan-of-record that if the extension is in the same module as the class, the methods will be treated as if they were declared in the class itself. This proposal only applies to extensions declared in a different module.

Whoops. That didn't happen. But I think it still could happen, for open methods and for non-open methods alike. The implementation would be something like this:

When emitting the class, scan all extensions for non-final members and add them to the dispatch table (vtable). This is a little expensive but not very expensive; these methods already need to be parsed to detect naming conflicts, and computing declaration types is a lot cheaper than type-checking function bodies. The implementations of these members are still emitted in their own file.

(I don't see any reason to limit this to one file except maybe compilation time, and in practice I don't think it'll be a significant difference anyway. Did you have a reason why "just the main file"?)

So given this implementation strategy, should we make this change? The main advantage seems to be "my code would be clearer if I could put these methods in an extension, but they also need to be overridable, so I don't have a choice", with a follow-on advantage being "this is one less limitation on extensions to teach". In my mind, there aren't really any major disadvantages.

There is one interesting point here, which is that "all Xs must be in the original declaration rather than extensions" can be useful for people reading the code, especially when they need to know the complete set of Xs to write the code properly. There are a few of these in particular:

  • Stored properties for a struct must be in the main declaration

    • anyone writing a non-delegating initializer (within the module) needs to know all of these
    • unless they have initial values
  • Stored properties for a class must be in the main declaration

    • anyone writing a designated initializer (must be in the main declaration also) needs to know all of these
    • unless they have initial values
  • Designated initializers for a class must be in the main declaration

    • anyone subclassing the class who wants to inherit convenience initializers needs to know these
    • but not interesting if the class is final
  • EDIT: a deinit for a class must be in the main declaration

    • this is mostly because there's only one; I don't think it's the same category as the rest of these, but I think it's worth mentioning here for completeness
  • Cases for an enum must be in the enum

    • pretty much necessary for anyone switching over the enum
  • Protocol requirements must be in the protocol

    • anyone adopting the protocol needs to know all of these
    • unless the requirement has a default (which is actually rather hard to find out without something like Apple's docs…)
  • Overridable members need to be in the class

    • you pretty much never have to know all of these

It's true that you'd no longer be able to just look at the class declaration to see what methods are overridable, but I don't think that's a huge loss. It seems more valuable to me to allow overridable methods to live in different files and therefore be grouped with other code that might be related. So overall I'm in favor of this proposal. (It seems minorly telling that no one's spoken up against it either.)


You did also mention allowing stored properties, which I think is a proposal best done separately even if the implementation would be rather similar. I feel a bit iffy about doing this for structs, but I'm not sure there's any real reason to be concerned; someone outside the module should mostly be able to treat a struct's stored and computed properties uniformly. With classes in particular it's a more favorable tradeoff: the only people who have to know all of the stored properties are the ones implementing the designated initializers, which are always at least in the same module. I do suspect some people to want the language to enforce that all the stored properties are in one place, though.

9 Likes

You suspect right! I strongly prefer a style with small type declarations only containing stored properties, typealiases and init (as well as overrides and deinit for classes). This style makes it easy to see at a glance the state that a type holds. I find this very much increases readability by reducing cognitive friction. Our ability to understand the state of a program is really important. Having to scroll through a potentially large declaration to do that is always a bummer.

Notice that the benefits have to do with readability, not writeability of the code. Code is read far more often than it is written. An analysis such as the one you provide is incomplete without also considering this side of the equation.

4 Likes

A bit of a tangent here, but it would be nice if there were a standard tool that would analyze a module (or library or project or...) and output, for each protocol, a complete list of:

  • Its requirements.
  • The conditions in which each requirement has a default implementation.
  • Its extension methods.
  • The conditions in which each extension method exists.
2 Likes

I hadn’t thought about the potential disruption for moving different things out of the main declaration, but it is very worth thinking about.

Adding stored properties to extensions could be an ugly mess but I also think it could help out a lot — it’d be wonderful if everything I needed to add to conform to a protocol were in one extension together. In particular it makes it a lot easier for me to remember what a protocol was — like, right now I’ll see a group of properties in my main declaration and think “ah yes, the <Blah> protocol” and then later I’ll see I have a whole extension just for the <Blah> methods. It’s really a lot easier to understand if they’re all together in one extension.

Similarly with, say, <Codable> right now the init(for:) has to be in the main block, but I’d much rather declare <Codable> on an extension so it’s very clear to any reader what these methods are and when they’ll be called. (I certainly don’t want to move just the <Decodable> part to its own block, although that’s possible right now.)

I also think it’d be nice if I could move all my open stuff into its own extension, because those tend to be pretty special (like, often I mark them with // subclass me! because they’re really just my poor-man’s attempts at using something like a protocol when I’m constrained to a class).

My philosophy is generally that we programmers should be given a set of paints and some brushes, and left to decide for ourselves how to best present our creations to future readers. There obviously could be some very non-readable code created by adding stored properties willy-nilly, but I don’t see a way to have the flexibility to write really good code without having the flexibility to write really bad code.

Like, if I have an extension that centers on handling the mouse (which I do), I think it’d be great to have some stored private state properties (previous mouse position? snapped mouse position?) directly in that extension, nice and close to the methods that use them. To me this is just like C99 adding the ability to declare variables in the middle of a function call — yes, you could go insane with it, but in general it led to much easier-to-read code because it gave us locality.

And, yes, it’d be annoying as heck if someone hid a bunch of public stored properties throughout their code, but right now you can do this with computed properties — we trust the programmers to move public things to a position of prominence. We also trust the programmer right now with the ability to scatter public methods throughout the file, but we hope and encourage them to place them prominently instead.

4 Likes

if that were possible, it would also make refactoring much easier, cause i could just cut/paste the extension somewhere else without needing to hunt around for the disembodied instance vars

+1 from me on that point

2 Likes

I encountered instance method 'foo' declared in 'Bar' cannot be overridden from extension for the first time today.

I'm unclear on the reason for the limitation which disallows method overrides in extensions. What should be in my mental model of Swift classes to understand that? Is it likely to change in a future Swift?

The workaround I've been using for several years now is to subclass NSObject and then use @objc func ... so Swift will let me override the method.
Apart from the need to subclass, this is also rather limiting as it prevents usage of any pure Swift type (enum and structs) as argument or result.

Would really love to see this allowed in the language as both Wil and Jordan proposed.