Possible Compiler Bug with Static Resolution of Overloaded Generic Functions

Using Swift
1

Issue

Could someone please have a look at the following code and tell me whether this is a bug or expected behavior?

Based on my knowledge of Swift it doesn't seem like this should be happening, but perhaps I'm misunderstanding something.

Protocols

Given the following protocols:

public protocol LogWriter {
    func write<T: LogEvent>(_ logEvent: @autoclosure () -> T)
}

public protocol LogEvent {
    var write: ((LogWriter) -> Void)? { get }
}

public protocol DefaultEvent: LogEvent {}

Implementations

And given the following basic implementations of the protocols:

public struct Writer: LogWriter {
    public init() {}
}

public struct DefaultLogEvent: DefaultEvent { 
    public init() {
        self.write = { [self] (logWriter: LogWriter) in
            logWriter.write(self)
        }
    }
    
    public var write: ((LogWriter) -> Void)? = nil
}

Extensions

And given the following extension of LogWriter protocol:

extension LogWriter {
    public func write<T: LogEvent>(_ logEvent: @autoclosure () -> T) {
        print("logEvent: \(T.self)")
    }

    public func write<T: DefaultEvent>(_ logEvent: @autoclosure () -> T) {
        print("defaultEvent: \(T.self)")
    }
}

The Problem

Given all the above, when I run the following code:

let writer = Writer()
let defaultEvent = DefaultLogEvent()
defaultEvent.write?(writer)

... then I would expect that write<T: DefaultEvent>(...) above would run and print "defaultEvent: defaultLogEvent" to the console.

However, instead, write<T: LogEvent>(...) runs and prints "logEvent: DefaultLogEvent".

Why? This is a bug, right?

I mean, DefaultLogEvent clearly conforms to DefaultEvent, which is a sub-protocol of LogEvent, so why is the compiler ignoring write<T: DefaultEvent>(...)?

It Gets Weirder...

Note that if we add the following extension:

extension LogWriter {
    public func write(_ logEvent: @autoclosure () -> DefaultLogEvent) {
         print("DefaultLogEvent! WTAF")
    }
}

Then it gets called as expected (instead of write<T: LogEvent>(...)).

So why does overloading the generic function with a more specialized protocol fail, while overloading the generic function with a non-generic function succeeds?

Current workaround

UPDATE: I found a workaround, which is to change the declaration of LogWriter to just be an empty protocol:

public protocol LogWriter {}

Now, the correct function func write<T: DefaultEvent>(...) gets called in the extension on LogWriter.

But why does this workaround work?

I do not understand.

1 Like
2

So, the logic that explains this behavior can be found here, and the comments should make it pretty readable even for anyone unfamiliar with the compiler code. When comparing two overloads (such as write in the call to logWriter.write(self)) the first check we do is to see whether one of the overloads is a generic declaration and the other is not:

  // A non-generic declaration is more specialized than a generic declaration.

This is why your func write(_ logEvent: @autoClosure () -> DefaultLogEvent) gets called instead of func write<T: LogEvent>(_ logEvent: @autoclosure () -> T).

The very next check we do is whether one or both of the overloads come from a protocol extension. If exactly one of the overloads is from a protocol extension:

    // One member is in a protocol extension, the other is in a concrete type.
    // Prefer the member in the concrete type.

So this is why func write<T: LogEvent>(_ logEvent: @autoclosure () -> T) is preferred over public func write<T: DefaultEvent>(_ logEvent: @autoclosure () -> T) (since the former is declared in the protocol body), even though the latter has more strict constraints on the generic parameter.

When you remove the declaration from the protocol body, then both declarations come from a protocol extension, and so the check no longer applies.

6 Likes
3

Does that mean that if the protocol itself contained two overloads, this wouldn't happen?

Also, why should the less specialized of two implementations "win" just because it's mentioned in the protocol itself?

We want to make types that are "open for extension but closed for modification." However it's hard to do that if the code in the extension cannot successfully override the code in the thing it's extending.

Any suggestions welcome. Thanks

4

Yes, that's correct. If both declarations were declared as protocol requirements, then this check wouldn't apply.

I'm far (far) from an expert in this area, so someone else can likely answer the "why" question much better than I could, but I will note that there's a subtle semantic distinction here. The method that performs these checks is called isDeclAsSpecializedAs, so for all intents and purposes, the ranking returned by this method determines which declaration is most specialized. Thus, it doesn't make sense to say that the "less specialized" declaration wins just because it's declared in the protocol body—the declaration that wins is the one that is more specialized.

Here's my personal justification of this behavior: making a particular declaration into a protocol requirement is an indication to the compiler that you want to achieve implementation selection based on dynamic dispatch. If you want to achieve implementation based on static dispatch, according to the static type of the arguments, then you should avoid using a tool (protocol requirements) intended for dynamic dispatch.

5

I don't see anywhere that says protocols are only for dynamic dispatch. Using protocols as generic constraints was supposed to be about static dispatch.

How can I declare something in my library such that other people can add protocol extensions with more specialized implementations than the base protocol method, and have those get used instead? Can I add @specializable or something to the protocol so that calls will always get routed through the most specialized implementation?

Given some protocols, P and Q: P, I want to have a protocol A that defines @specializable func f<T: P>(_ x: T) such that an imlementer can make a protocol extension with f<T: Q>(_ x: T) that intercepts any calls to f whenever x conforms to Q.

This is easy to do in Kotlin but Swift does not seem to want to let us perform routing based on type without resorting to explicit casts. In order to catch Swift up to Kotlin we're going to need this. Take a look at Kotlin's "when" clauses and sealed classes to see how powerful this is, and how much better it is than using enums for everything.

7

Sure, protocols are not only for dynamic dispatch (indeed, methods defined only in a protocol extension receive static dispatch), but making something into a protocol requirement is very much a tool to achieve dynamic dispatch.

I don't think this agrees with the formal semantics of protocol-constrained generics. Consider that a generic function f<T: P>(_ t: T) may be called with a type that is entirely unknown at compile time (e.g. by using _openExistential on a value of static type P). The compiler may specialize f in a way that allows specific implementations for requirements from P to be selected statically, but only if those selections are the same as whatever selection would be made dynamically.

Unless I'm misunderstanding something, I think you could achieve this by defining f<T: P> only in an extension of the protocol in question, rather than defining it as a requirement. E.g.:

protocol P {}
protocol Q: P {}

protocol R {}

extension R {
    func f<T: P>(_ x: T) {
        print("T: P")
    }
}

extension R {
    func f<T: Q>(_ x: T) {
        print("T: Q")
    }
}


struct X: P {}
struct Y: Q {}
struct Z: R {}

Z().f(X()) // T: P
Z().f(Y()) // T: Q

Because isDeclAsSpecializedAs prefers protocol requirements over extension members, there's really no such thing as "protocol extensions with more specialized implementations than the base protocol method" (if the base protocol method is a protocol requirement). Protocol requirements are, by definition, more specialized than extension members.

3 Likes