For a bit of background this stems from a bug I (and a number of other people) have reported on bugs.swift.org, the original being here: [SR-103] Protocol Extension: function's implementation cannot be overridden by a subclass · Issue #42725 · apple/swift · GitHub.
The Current Situation
For context let's start by looking at basic class inheritance, something we all know and can reason about:
class A {
func whatAmI() {
print("superclass")
}
}
class B: A {
override func whatAmI() {
print("subclass")
}
}
A().whatAmI() // prints 'superclass'
B().whatAmI() // prints 'subclass'
(B() as A).whatAmI() // prints 'subclass'
If we create an instance of A then we know that the method on A will be called. If we create a subclass of A called B, override the whatAmI method and then call it, we expect the method on B to be called. This is even the case when we cast an instance of B to A because we are still operating on an instance of B.
Now let's throw a protocol into the mix. Let's call this protocol P, give it a default implementation through an extension and have A conform to it.
protocol P {
func whatAmI()
}
extension P {
func whatAmI() {
print("protocol")
}
}
class A: P {
func whatAmI() {
print("superclass")
}
}
class B: A {
override func whatAmI() {
print("subclass")
}
}
A().whatAmI() // prints 'superclass'
B().whatAmI() // prints 'subclass'
(B() as A).whatAmI() // prints 'subclass'
As you can see the general behaviour hasn't changed. We still call through to the expected implementations just like in our first example.
But what happens if we remove the function implementation in A to rely on the default implementation in the protocol? To do that we also have to remove the override keyword for the function in B as we are no longer overriding any implementation in A. Once we do that we get the following:
protocol P {
func whatAmI()
}
extension P {
func whatAmI() {
print("protocol")
}
}
class A: P {
}
class B: A {
func whatAmI() {
print("subclass")
}
}
A().whatAmI() // prints 'protocol'
B().whatAmI() // prints 'subclass'
(B() as A).whatAmI() // prints 'protocol'
Calling the function on an instance of A does what we would expect, it doesn't have an implementation of the function any more so it relies on the default implementation in the protocol. With an instance of B we also get what we would expect, it just uses its implementation. The strange things start to happen when we start casting an instance of B to A, for some reason now we ignore the implementation in B and instead call through to the default protocol implementation.
This seems inconsistent to me, but it kind of makes sense if we look at it from the perspective of the compiler.
Just to note that I'm not compiler engineer and I don't pretend to know the inner workings of how the Swift compiler works, I'm simply making an assumption based on the results I'm observing.
What looks like is happening is that there are several tables at work that keep track of various method implementations in different places. Based on whichever method signatures match in these tables one implementation is being used over another. In the case of our examples above, there appears to be a table thats keeping track of methods that override methods further up the inheritance tree and the methods in this table always take president over whichever method they override in the superclass. If that assumption is correct, it makes sense that when we stop using the override keyword in B, the implementation in B never gets added to this overrides table and when we cast to A, A has nowhere to look to see if B has overridden any of its methods... which it technically hasn't as the implementation is coming from the protocol.
This is an interesting hypothesis so let's dig a bit further. Let's go back to a known state where we have a method implementation in A again, and override it in B. What happens if A calls whatAmI from within its init method?
protocol P {
func whatAmI()
}
extension P {
func whatAmI() {
print("protocol")
}
}
class A: P {
init() {
self.whatAmI()
}
func whatAmI() {
print("superclass")
}
}
class B: A {
override func whatAmI() {
print("subclass")
}
}
let a = A() // prints 'superclass'
a.whatAmI() // prints 'superclass'
let b = B() // prints 'subclass'
b.whatAmI() // prints 'subclass'
(b as A).whatAmI() // prints 'subclass'
So as you would expect, by using self.whatAmI() from within the init of A and creating an instance of B which inherits from A, it calls to the override of that method, not its own implementation.
So now what happens if we remove the implementation of whatAmI in A again? What method is called from the init in A? If the hypothesis is correct then A won't know where to look in any subclasses to see if the method has been overridden, so it will fall back to the implementation in the protocol extension.
protocol P {
func whatAmI()
}
extension P {
func whatAmI() {
print("protocol")
}
}
class A: P {
init() {
self.whatAmI()
}
}
class B: A {
func whatAmI() {
print("subclass")
}
}
let a = A() // prints 'protocol'
a.whatAmI() // prints 'protocol'
let b = B() // prints 'protocol'
b.whatAmI() // prints 'subclass'
(b as A).whatAmI() // prints 'protocol'
Looking at the instance of A it's doing what we would expect. It has no implementation of the method whatAmI, so the call from within the init and the call from outside both go to the default protocol implementation.
Now we get to the instance of B where things appear to get very weird from a user point of view, but actually just fit nicely along with the hypothesis of whats going on in the compiler. So when we create an instance of B it goes to the init defined in A, typically the call in A would look into the overrides table to see if B had overridden any of it's methods, but since it hasn't it then reverts back to the default implementation in the protocol extension. This is in stark contrast to the behaviour you would typically experience if A did in fact have an implementation of the whatAmI method where the implementation on B would be used.
Moving on to calling the method from outside, we can see that the implementation in B is now called as you would expect, but then as soon as we cast B to A we revert back to the protocol implementation.
The Issue
We currently have an inconsistent behaviour between subtile changes in scenario. Simply removing the base implementation in a superclass that inherits from a protocol with a default implementation can change the entire behaviour of how a subclass works. This is pretty jarring to someone who doesn't know whats going on and is the source of a number of bugs (see the link at the top to see how many duplicates have been made about this).
Hopefully the examples above have highlighted what the problem is and how it can be reproduced.
The Future
The interesting part here is to discuss is what we expect to happen, what behaviour is predictable to the user and what implications any proposed changes have elsewhere.
From a totally simplistic point of view, a solution would be to have Swift understand that a superclass somewhere up the stack conforms to a protocol which has a default implementation. That default implementation needs to be overridden by any subclass no matter if the implementation is in the superclass or in a protocol extension. This would force the use of the override keyword in places that it currently isn't required, but it will make it more obvious to the user that there is some functionality that they are overriding.
The other thing to consider is how super is treated. In the above change it would mean that we could now call through to the protocol extension implementation by calling super in a subclass where the superclass didn't have an implementation. This is definitely an interesting change but I'm not sure if it's a desirable one or not. Unfortunately the implications of such a change as this are quite far reaching as it means broken source compatibility.
Another solution would be so have Swift just make the right choices without requiring adding the override keyword. As much as this is desirable from a source compatibility point of view, I would say that it's highly undesirable from a code understandability and 'sudden silent change from one version to another version' point of view.
I'm interested to know others thoughts on this and if there are other solutions that might be desirable.
