Overridden method not called on class where expected

Using Swift
1

I had a bit of an issue and partly got to the bottom of it. I thought it might be an interesting discussion anyway.

I have the code below which has two versions of a method on a class, one is intended to be called in production code, the other version is intended to be called in test/mock code. The classes are instantiated with the type LsoSession<URLSession> in production and something like LsoSession<MockNetworkSession> in unit tests.

So there are two implementations of download(for:) in the class, I was expecting the more specific overload to be called in production and the general overload to be called in other cases (e.g. unit tests).

What happened in reality was the more general version was getting called in all cases.

So the if clause at the start of the more general version of the download method (the version without a generic constraint) should never be run. But it was. That's why I had to add that and use that rather inelegant if clause...

public class LsoSession<BaseNetworkSession: DownloadNetworkSession>: NSObject, DownloadNetworkSession, LongLifespanDownloadsNetworkSession {
    public func download(for urlRequest: URLRequest) -> BaseNetworkSession.DownloadTask
    where BaseNetworkSession == URLSession {
        baseSession.downloadTask(with: urlRequest)
    }

    // this method should only be used in unit tests at the moment
    public func download(for urlRequest: URLRequest) -> BaseNetworkSession.DownloadTask {
        if let baseSession = baseSession as? URLSession {
            guard let downloadTask = baseSession.downloadTask(with: urlRequest) as? BaseNetworkSession.DownloadTask else {
                fatalError("impossible?")
            }

            return downloadTask
        }

        return DownloadTask()
    }

    private var baseSession: BaseNetworkSession

    public init(baseSession: BaseNetworkSession) {
        self.baseSession = baseSession
        super.init()
    }

I then looked at the call site...

extension FileDownloadService where BaseNetworkSession: LongLifespanDownloadsNetworkSession {
    public func downloadFileInBackground() async throws -> BaseNetworkSession.DownloadTask {
        let downloadTask = network.download(for: try request())
        downloadTask.resume()
        return downloadTask
    }
}

which is an extension on...

public final class FileDownloadService<BaseNetworkSession: DownloadNetworkSession>: Service, GetMethod, LsoService
where BaseNetworkSession.DownloadTask: NSObject, BaseNetworkSession: NSObject {
    public typealias RequestId = String
    typealias Dependencies = RequestId
    
    let network: BaseNetworkSession

...so I think in some way the compiler must be calling download via a witness table or something. I had thought the compiler would be able to work out the correct version of the method to call but it seems it's not.

Interested in people's thoughts on this. I think this is the sort of thing that will improve in Swift 6 because it will always have to be explicit when you're creating any kind of indirection through a witness table like this as I understand it because when polymorphism/indirection for protocol types goes via a witness table it will have to be annotated with any, which should hopefully make all this more clear. Possibly I'm misunderstanding it.

2

Overload resolution always happens with the information available at the call site. Withitn downloadFileInBackground() we must pick a declaration to call, and at the call site of download(for:) we don't know for certain whether BaseNetworkSession == URLSession, so that overload isn't viable. That we may have BaseNetworkSession == URLSession is immaterial, because the overload is resolved at compile time and not whatever generic arguments happen to be supplied when the method is actually called.

If you want to call different methods dynamically based on what session type happens to be present at runtime then perhaps you want the sessions themselves to implement some sort of NetworkSession protocol with a download(for:) requirement. Then the correct method would be called depending on the actual type of BaseNetworkSession at runtime.

7 Likes
3

I see your point but...

FileDownloadService is also a generic class, with BaseNetworkSession as its generic type parameter.

So when the compiler is building the SIL/IL for downloadFileInBackground it should "know" what class we are dealing with. This should show up in two ways, there should be a generic version of FileDownloadService that takes the generic type parameter. And there may or may not be some generic specialisations created at various points in compilation as optimisations, but the two should behave the same.

So when the compiler is lowering downloadFileInBackground it can implement a check to say something like "if generic type parameter 1 == URLSession then call this overload".

It seems to me like the compiler definitely can emit that code and it would create behaviour that better matches what you'd expect?

In the case where a generic specialisation is emitted by the optimiser for FileDownloadService<URLSession>.downloadFileInBackground then it would not need to perform that check and could always call the right version of download.

It feels like this is an area of Swift where it's perhaps not entirely clear what people would expect because I don't know of any documentation. It certainly feels like "buggy" behaviour by the compiler if I'm honest!

Carl

p.s. All the above is with the HUGE caveat that I'm not a compiler developer and I could be absolutely wrong about the compiler internals. I'm just going on what I know. Also I'm not very familiar with the optimisations in question.

p.p.s. I guess though there's a slight danger to that if this is sort of a "bug" or a "missed opportunity to have the compiler behave in a more reasonable way" then changing it could break existing code though?

4

Well, the compiler is already matching what Iā€™d expect. :slight_smile:

I should have perhaps emphasized more in my first post that this isnā€™t really a matter of ā€˜canā€™tā€™; itā€™s simply how the generics model in Swift works. Folks coming to Swift from C++ may find this surprising compared to a fully-monomorphized model where names arenā€™t bound until we know the actual generic substitutions applied, but itā€™s not a ā€˜bugā€™.

In Swift, specialization/monomorphization is applied only as an optimizationā€”it is not guaranteed and importantly is not permitted to change the semantics of the program (by, e.g., picking a different overload that what would be selected with purely local information). This has some beneficial properties compared to a more C++-like model: authors of generic code can definitively reason about what declaration will be called by an overloaded reference on a generic value, and code that uses generics is able to be fully compiled separately from clients without having to ā€˜ship the ASTā€™ to allow substitutions to be applied later.

7 Likes
5

There is a bit of ā€œcanā€™tā€: if things behaved the C++ way, Apple wouldn't be able to update any generic functions in iOS 20 without everyone recompiling their apps, because the generic would have to have been copied into each app when it was built in order to work with the appā€™s specific types. In Swift terms, C++ ā€œsolvesā€ this by requiring all generics (templates) to be frozen and inlinable, but that was never on the table for Swift as a language for Apple system APIs. (The same applies to binary frameworks with library distribution, of course.)

4 Likes
6

The only mechanisms for dynamic dispatch in Swift are protocol and class method calls, which dynamically dispatch on the type of self. Overloads are essentially distinct declarations - as if they had different names or parameter labels ā€” and we select exactly one possible overload based on the static types known at the call site.

For example, overloads can have different types, or new overloads can be added by a third library in an extension, etc. We would need something like objc_msgSend but with multiple dispatch to implement the general case of this.

3 Likes
7

Interesting stuff. Thanks all!

I'll try to do some more experiments tomorrow to help my understanding (at least) and I'll post back anything interesting I find.

1 Like
8

You could have BaseNetworkSession conform to some protocol, then instead of overloading download(for:) you can have one implementation that dispatches on the BaseNetworkSession.

3 Likes
9

Yeah that's probably a better design here.

p.s. I think I had a bit of a cognitive dissonance because I'm thinking "the compiler definitely knows the type at compile time, at the call site, so it should be able to use the correct overload". It didn't feel like true polymorphism, where the compiler cannot know the type at compile time, so it has to put in some kind of indirection so it can be decided at runtime. But I can see how it could rapidly get impossibly complicated. Like you said, if I had added the function overload with a generic type constraint in another module/library then it would start to become quite a hard problem to solve at compile time. Combined with the fact that it would become tricky to keep the pure generic function implementation and the specialisations in line, it becomes impossible or impractical I think.

I've run out of time to play with it and have to move onto another project but I hope this thread is useful for somebody some day!

Thank you very much for all your help and advice everyone.

C

1 Like
10

(post deleted by author)