KeyPath with generics and subclassing

[deze333 (Deze)]

I am trying to use KeyPath to set properties of a child class that inherits from a superclass. It works when a child class instance is passed explicitly but fails if superclass tries to use same KeyPath. Somehow Swift can't match type information from KeyPath against class instance, yet it's not obvious what's happening since all debug shows correct types being used. Below is a simple playground that illustrates the problem.

I can successfully set Child properties when I pass it as an object via child.setValues(obj: child, from: newValues), not beautiful obviously.

But it fails if I use its Base superclass method child.setValues(from: newValues) — the switch fails to match provided KeyPath. My debug print shows that class/object remains the same Child (not Base), the keyPath looks correct too... yet the switch logic seems to think it's something else:

Function: setValues(obj:from:) ...
__lldb_expr_27.Child
__lldb_expr_27.Child
__lldb_expr_27.Child
Mirror for Child
Function: setValues(obj:from:) [OK]
Function: setValues(from:) ...
__lldb_expr_27.Child
__lldb_expr_27.Child
__lldb_expr_27.Child
Mirror for Child
Function: setValues(obj:from:) ...
__lldb_expr_27.Child
__lldb_expr_27.Child
__lldb_expr_27.Child
Mirror for Child
Fatal error: Failed KeyPath: Swift.ReferenceWritableKeyPath<__lldb_expr_27.Child, Swift.Int> for type: __lldb_expr_27.Child: file KeyPath Generic.playground, line 56

I am puzzled as it seems that Swift erases the object's or KeyPath's type info somewhere yet I can't figure out what's happening. Any ideas?

Here's the full playground code:

import Foundation

class Base {
}

class Child: Base {
    var name: String = ""
    var number: Int = 0
    
    enum CodingKeys: String, CodingKey, Hashable, CaseIterable, KeyPathMappable {
        case name, number
        
        var keyPath: PartialKeyPath<Child> {
            switch self {
            case .name: return \.name
            case .number: return \.number
            }
        }
    }
}

protocol KeyPathMappable {
    associatedtype T
    var keyPath: PartialKeyPath<T> { get }
}

extension Base {
    
    func setValues<K>(from dict: [K : Any?]) where K: Hashable & KeyPathMappable {
        print("Function: \(#function) ...")
        debugPrint(Self.self)
        debugPrint(self)
        debugPrint(self.self)
        print(Mirror(reflecting: self))
        
        setValues(obj: self, from: dict)
    }
    
    func setValues<T, K>(obj: T, from dict: [K : Any?]) where K: Hashable & KeyPathMappable {
        print("Function: \(#function) ...")
        debugPrint(Self.self)
        debugPrint(obj)
        debugPrint(obj.self)
        print(Mirror(reflecting: obj))

        assert(obj.self is Self, "Object must be \(Self.self)")
        
        for (ck, v) in dict {
            let kp = ck.keyPath
            switch kp {
                case let p as ReferenceWritableKeyPath<T, String>:
                    obj[keyPath: p] = v as? String ?? String(describing: v)
                case let p as ReferenceWritableKeyPath<T, Int>:
                    obj[keyPath: p] = v as? Int ?? 0
                default:
                    assertionFailure("Failed KeyPath: \(kp) for type: \(self)")
            }
        }
        
        print("Function: \(#function) [OK]")
    }
}

let newValues: [Child.CodingKeys: Any] = [
    .name: "Adam",
    .number: 137,
]

let child = Child()
child.setValues(obj: child, from: newValues) // works
child.setValues(from: newValues) // case assertion fails

[Lantua]

No one wants complex question. Let's downsize it a bit:

class Base { 
  var baseName: String = ""
}
class Child: Base {
    var name: String = ""
}

extension Base {
    func foo<K>(keyPath: PartialKeyPath<K>) -> String {
        foo(obj: self, keyPath: keyPath)
    }
    func foo<T, K>(obj: T, keyPath: PartialKeyPath<K>) -> String {
        switch keyPath {
        case let p as ReferenceWritableKeyPath<T, String>: return "String key path"
        default: fatalError("We want String key path")
        }
    }
}

let child = Child()
child.foo(obj: child, keyPath: \Child.name) // String key path
child.foo(keyPath: \Child.name) // failure: We want String key path 

[Lantua]

• During the first call T is of type Child, so you can cast the keys to ReferenceWritableKeyPath<Child, ...>.
• During the second call, you call from Base method where self is Base, so T is set to Base. But your keys can't be converted to ReferenceWritableKeyPath<Base, ...> given that the key paths are not Base key path so it fell through the switch.

It is more apparent like this:

class Base { var baseName: String = "" }
class Child: Base { var name: String = "" }

let a: PartialKeyPath<Base> = \Child.baseName // ok
let b: PartialKeyPath<Base> = \Child.name // fail

---

I'm not sure how to fix the api as is, but I do find it easier to do from the Dictionary:

extension Dictionary where Key: KeyPathMappable, Key.T: AnyObject {
  func assign(to target: Key.T) {
    for (key, value) in self {
      switch key.keyPath {
      case let p as ReferenceWritableKeyPath<Key.T, String>:
        target[keyPath: p] = value as? String ?? String(describing: value)
      case let p as ReferenceWritableKeyPath<Key.T, Int>:
        target[keyPath: p] = value as? Int ?? 0
      default:
        assertionFailure("Failed KeyPath: \(key.keyPath) for type: \(Key.T.self)")
      }
    }
  }
}

let newValues: [Child.CodingKeys: Any] = [
  .name: "Adam",
  .number: 137,
]

let child = Child()
newValues.assign(to: child)

Seems to be more type-safe to boot.

[deze333 (Deze)]

Thanks for your concise example!
Funny enough somehow I found that simply adding self as! Self to your code fixes the problem — both your code and the original playground now work well. Wonders of generics.

class Base {
    var baseName: String = ""
}
class Child: Base {
    var name: String = ""
}

extension Base {
    func foo<K>(keyPath: PartialKeyPath<K>) -> String {
        foo(obj: self as! Self, keyPath: keyPath) // added "as! Self"
    }
    func foo<T, K>(obj: T, keyPath: PartialKeyPath<K>) -> String {
        switch keyPath {
        case let p as ReferenceWritableKeyPath<T, String>: return "String key path"
        default: fatalError("We want String key path")
        }
    }
}

let child = Child()
child.foo(obj: child, keyPath: \Child.name) // String key path
child.foo(keyPath: \Child.name) // String key path

[BigSur ({ @MainActor  in M1.Ultra }(Swift))]

@deze333

simply adding self as! Self to your code fixes the problem

Well done! One more question, why it must be self as! Self, does self as Self work as the same way?

[deze333 (Deze)]

That's what the compiler asked for. If I use self as Self then the following error pops:

'Base' is not convertible to 'Self'; did you mean to use 'as!' to force downcast?

So adding ! as the compiler requests does the trick. Some voodoo magic in action.

What truly puzzles me is the fact that once the base class has been subclassed, there seems to be no debuggable confirmation of its Base nature, all instances of the class are seen as Child so are all KeyPaths and yet compiler seems to be confused.

A simple dump of class hierarchy:

class Base {
    var baseName: String = ""
    
    func whoAmI(prefix: String) {
        debugPrint(prefix, self)
        debugPrint(prefix, Self.self)
        let mirror = Mirror(reflecting: self)
        debugPrint(prefix, "Mirror sees:", mirror.description)
    }
}

class Child: Base {
    var name: String = ""
}

let obj: Any = Child()
if let obj = obj as? Child {
    obj.whoAmI(prefix: "As Child")
}
if let obj = obj as? Base {
    obj.whoAmI(prefix: "As Base ")
}

Produces an output like this — no matter how the object is accessed it's still the same class instance:

"As Child" __lldb_expr_23.Child
"As Child" __lldb_expr_23.Child
"As Child" "Mirror sees:" "Mirror for Child"
"As Base " __lldb_expr_23.Child
"As Base " __lldb_expr_23.Child
"As Base " "Mirror sees:" "Mirror for Child"

I wonder if somebody can cast some light on this superclass/subclass mystery and why KeyPath can't refer to its owner class when all class information points in the right direction.

[Lantua]

@deze333

What truly puzzles me is the fact that once the base class has been subclassed, there seems to be no debuggable confirmation of its Base nature.

You're inside Base extension, nothing screams the static type of self better than debugPrint(Base.self)

@deze333

So adding ! as the compiler requests does the trick. Some voodoo magic in action.

You're using self inside Base extension, so self must be of type Base. You can't cast Base to Self at compile time because, well, Self is a subclass of Base. self is a special variable that we know is always convertible to Self, but I doubt we will add that annotation anytime soon. That's why you need to force unwrap.

@deze333

Funny enough somehow I found that simply adding self as! Self to your code fixes the problem

It works with your code too. If you do

setValues(obj: self as! Self, from: dict)

---

If you're confused about how T is decided, you can try to look up static dispatch and dynamic dispatch in this forum.

---

PS

Why do you need the second method (with T)? It doesn't use self at all. It seems to fit static function, or global function better.

[BigSur ({ @MainActor  in M1.Ultra }(Swift))]

@Lantua

self is a special variable that we know is always convertible to Self , but I doubt we will add that annotation anytime soon. That's why you need to force unwrap.

Could we treat self as! Self as c/c++ reinterpret_cast<Self>(self) semantic or something else.. forced type casting operation? @Lantua

[Lantua]

We could treat self as! Self as normal as!/as? operation.

You got a variable of type Base, and you want to cast it to Self (which is Child in this case). If succeed, you get Self object, otherwise, you get nil. Since as! also force unwrap for you, it raise exception on the otherwise case, which we know in this scenario won't happen.

I believe it's more akin to dynamic_cast if my rusty C++ is still to be trusted. You shouldn't encounter reinterpret_cast equivalents unless you go into Unsafe territory.

[BigSur ({ @MainActor  in M1.Ultra }(Swift))]

Thanks for clarification, as you said we can conclude that as the following rules, right?

v as T
*  succeed: T
*  failed: nil

v as? T
*  succeed: T? //will need v! to get the real value
*  failed: nil

v as! T // forced unwrap casting - `dynamic_cast` in C
*  succeed: T // T is a subtype of v
*  failed: lead to runtime error // T is not a subtype of v

[Lantua]

Close, you're correct about as? and as!. as? results in T? and is nil on failure. as! results in T and crashes on failure. Note also that both castings happen at runtime, so you can succeed if user provides the right type even if the code doesn't look like it.

as is a tricky one since it doesn't really do much work. You always succeed with as because, well, the compiler can prove it. It's more of an annotation to help the compiler with type inference more than a casting tool.

func foo<T>(block: () -> T) {}

foo {
    // It would have been `Int` without `as`.
    3 as Double
}

[BigSur ({ @MainActor  in M1.Ultra }(Swift))]

Got it, very interesting example, appreciate your help.

[deze333 (Deze)]

@Lantua

I believe it's more akin to dynamic_cast if my rusty C++ is still to be trusted. You shouldn't encounter reinterpret_cast equivalents unless you go into Unsafe territory.

Found this great "self vs. Self" explanation here, quoting:

In a class/static method, Self can be used as the return type, to indicate that the return type is the type of the class to which the method was sent, rather than the class in which the method is declared. It's similar to instancetype in Obj-C.

So perhaps my understanding should be along the lines that a class type as seen by the compiler at compile time obviously MAY NOT translate into the same class type at run time — but the compiler surely tells you that it "knows" what self is.

Hence self as! Self is a kind of adapter plug between two realities: the compiler's view of the world and the runtime.

Seeing this as instancetype simplifies things a lot. An interesting example of Swift somewhat cryptic nature — there are certain things one can't intuitively grasp (due to the lack of intuitive names) unless a full picture of the language is held in the head.

Also, there's (seemingly) no way to debug class behaviour as illustrated by the KeyPath dilemma:

• the runtime self tells you it's a Child instance
• but when it applies KeyPath it actually treats Child as Base unless the instance is explicitly cast by self as! Self

It's a kind of a logical fallacy, isn't it?

[Lantua]

@deze333

• but when it applies KeyPath it actually treats Child as Base unless the instance is explicitly cast by self as! Self

That's not quite the right place. In your code, it fails as early as the conversion at switch case, before applying it.

T is a generic placeholder, and so it'll stick to whatever type it can infer at compile time, even if the runtime type can be slightly different (but of course compatible). As I said, in the scenario where it fails, T IS Base. You're trying to create a Base key path from \Child.name and failed. The reason self as! Self fixes it is because you shift the type of T from Base to Self, which is Child, and so you can create Self key path from \Child.name.

[deze333 (Deze)]

@Lantua

The reason self as! Self fixes it is because you shift the type of T from Base to Self

Totally understand what you're saying. Yet there's a lingering feeling that something isn't right still.

Run this playground amended to match our discussion. I know, it's not a two-liner but it illustrates the mechanics well.

import Foundation

class Base {
}

class Child: Base {
    var name: String = ""
    var number: Int = 0
    
    enum CodingKeys: String, CodingKey, Hashable, CaseIterable, KeyPathMappable {
        case name, number
        
        var keyPath: PartialKeyPath<Child> {
            switch self {
            case .name: return \.name
            case .number: return \.number
            }
        }
    }
}

protocol KeyPathMappable {
    associatedtype T
    var keyPath: PartialKeyPath<T> { get }
}

extension Base {

    // Works well thanks to 'self as! Self'
    func setValues<K>(from dict: [K : Any?]) where K: Hashable & KeyPathMappable {
        let obj = self as! Self
        for (ck, v) in dict {
            setValue(obj: obj, keyPath: ck.keyPath, value: v)
        }
    }
    
    // Doesn't work even though 'self is Self' evaluates as true
    func setValuesFailing<K>(from dict: [K : Any?]) where K: Hashable & KeyPathMappable {
        if self is Self {  // Hmm... can it be true?
            print ("I think that am looking good to go since I am \(self) and \(Self.self)")
        }
        for (ck, v) in dict {
            setValue(obj: self, keyPath: ck.keyPath, value: v)
        }
    }
    
    func setValue<T: Base, K>(obj: T, keyPath: K, value v: Any?) {
        switch keyPath {
            case let p as ReferenceWritableKeyPath<T, String>:
                obj[keyPath: p] = v as? String ?? String(describing: v)
            case let p as ReferenceWritableKeyPath<T, Int>:
                obj[keyPath: p] = v as? Int ?? 0
            default:
                assertionFailure("Couldn't match KeyPath: \(keyPath) for type: \(obj.self)")
        }
    }
}

let newValues: [Child.CodingKeys: Any] = [
    .name: "Adam",
    .number: 137,
]

let child = Child()
child.setValues(from: newValues) // this works thanks to 'self as! Self'
child.setValuesFailing(from: newValues) // switch logic will fail, even though 'self is Self' passes

Method setValues(from:) works well because it uses self as! Self trick.

Method setValuesFailing(from:) fails even though it successfully verifies that self is Self. Two points:

• it makes sense at compile time since self is clearly Base from the compiler standpoint
• it doesn't make sense at runtime when self matches Self which is Child if you print it ... yet runtime handles it as Base

Perhaps all that is due to compile/runtime dissociation. It would be super helpful if self is Self failed because Base is not Child. Am I going crazy here?

[Lantua]

You're not the only one confused about the dynamic dispatch vs static dispatch. If you search this forum for dynamic dispatch, or function overload, you'll find a lot of the same post, described in countless different ways.

@deze333

even though it successfully verifies that self is Self

@deze333

It would be super helpful if self is Self failed because Base is not Child .

Well, is is a dynamic checking, so... yeah, doesn't really help us here. Actually, most of them are dynamic: as?, as!, is, type(of:). You'll find that there aren't a lot of static counterparts. Perhaps because, well, you can just use type:

let a: T = ...

if a static_is T {
  // But of course, it goes here
} else {
  // Ummm, what?
}

if let a = a static_as? T {
  // Why are we even checking...
}

Debugger could of course get smarter by providing the static type of a variable, but I don't know if we have anything like that.

[deze333 (Deze)]

What is actually jarring is the construct of self as! Self being a logical fallacy in its own right because it doesn't necessarily matches self is Self, it's a kind of transitivity(?) failure. Makes the mathematical universe frown.

Wouldn't it be better if the language designers ditched self as! Self in favour of something explicit like let obj = instance(of: self) that very clearly explains what is going to happen.

[xwu (Xiaodi Wu)]

Generics work with static types; is and as! work with dynamic types. As a rule of thumb, it is a code smell to mix the two.

When you write self is Self, you are testing whether the dynamic type of self is Self. If this evaluates to true, then you know that casting self as! Self will succeed at runtime.

But this has nothing to do with the static type of self, which is what is involved when you call a function that is generic over the type of self.

[BigSur ({ @MainActor  in M1.Ultra }(Swift))]

A little bit confuse right now again...
Let's design a crazy function to check whether self is Self?

@xwu

When you write self is Self , you are testing whether the dynamic type of self is Self . If this evaluates to true , then you know that casting self as! Self will succeed at runtime.

func self_as_Self () -> Self
{
   let self = self is Self ? self as Self : self as! Self
   return self
}

In this specific case, self is Self evaluate to false, so self as Self compile failed, but self as! Self compile succeed and fixed the problem.
Because at runtime, self.self != Self.self, so we have to use as! Self to force type casting.

Anybody'd better correct me, if I'm out of mind!

[deze333 (Deze)]

What I am pointing at is of purely mathematics and logic domain. You know, we've heard so much about "algebraic data types" in Swift and how Swift model is close to mathematical abstractions... until this one.

The fundamental discrepancy between self as! Self and self is Self can be seen as a corruption of logic in computer programming. Speaking from purely philosophical standpoint here.