Pitch: Key-Path Member Lookup

Key Path Member Lookup

Dynamic member lookup allows a type to opt in to extending member lookup ("dot" syntax) for arbitrary member names, turning them into a string that can then be resolved at runtime. Dynamic member lookup allows interoperability with dynamic languages where the members of a particular instance can only be determined at runtime... but no earlier. Dynamic member lookups therefore tend to work with type-erased wrappers around foreign language objects (e.g., PyVal for an arbitrary Python object), which don't provide much static type information.

On the other hand, key paths provide a dynamic representation of a property that can be used to read or write the referenced property. Key paths maintain static type information about the type of the property being accessed, making them a good candidate for abstractly describing a reference to data that is modeled via Swift types. However, key paths can be cumbersome to create and apply. Consider a type Lens<T> that abstractly refers to some value of type T, through which one can read (and possibly write) the value of that T:

struct Lens<T> {
  let getter: () -> T
  let setter: (T) -> Void
  
  var value: T {
    get {
      return getter()
    }
    set {
      setter(newValue)
    }
  }
}

Given some Lens, we would like to produce a new Lens referring to a property of the value produced by the lens. Key paths allow us to write such a projection function directly:

extension Lens {
  func project<U>(_ keyPath: WritableKeyPath<T, U>) -> Lens<U> {
    return Lens<U>(
        getter: { self.value[keyPath: keyPath] },
        setter: { self.value[keyPath: keyPath] = $0 })
  }
}

As an example, consider a Lens<Rectangle>:

struct Point {
  var x, y: Double
}

struct Rectangle {
  var topLeft, bottomRight: Point
}

func projections(lens: Lens<Rectangle>) {
  let topLeft = lens.project(\.topLeft)   // inferred type is Lens<Point>
  let top = lens.project(\.topLeft.y)     // inferred type is Lens<Double>
}

Forming the projection is a bit unwieldy: it's a call to project in which we need to use \. to then describe the key path. Why not support the most direct syntax to form a lens referring to some part of the stored value, e.g., lens.topLeft or lens.topLeft.y, respectively?

Proposed solution

Introduce a new attribute @keyPathMemberLookup that can be placed on the definition of a type. For such types, "dot" syntax to access a member will be rewritten as a use of a special subscript whose argument is a key path describing the member. Here, we reimplement Lens in terms of @keyPathMemberLookup:

@keyPathMemberLookup
struct Lens<T> {
  let getter: () -> T
  let setter: (T) -> Void
  
  var value: T {
    get {
      return getter()
    }
    set {
      setter(newValue)
    }
  }
  
  subscript<U>(keyPathMember keyPath: WritableKeyPath<T, U>) -> Lens<U> {
    return Lens<U>(
        getter: { self.value[keyPath: keyPath] },
        setter: { self.value[keyPath: keyPath] = $0 })
  }
}

Given a Lens<Rectangle> named lens, the expression lens.topLeft will be evaluated as lens[keyPathMember: \.topLeft], allowing normal member accesses on a Lens to produce a new Lens. Such accesses can be chained: the expression lens.topLeft.y will be evaluated as lens[keyPathMember: \.topLeft][keyPathMember: \.y].

Detailed design

The design of @keyPathMemberLookup follows that of @dynamicMemberLookup fairly closely, because both involve alternate interpretations of basic member access ("dot" syntax). Specifically, @keyPathMemberLookup adopts similar restrictions to @dynamicMemberLookup:

  • Key path member lookup only applies when the @keyPathMemberLookup type does not contain a member with the given name. This privileges the members of the @keyPathMemberLookup type (e.g., Lens<Rectangle>), hiding those of whatever type is at the root of the key path (e.g., Rectangle).
  • @keyPathMemberLookup can only be written directly on the definition of a type, not an extension of that type.
  • A @keyPathMemberLookup type must define a subscript with a single, non-variadic parameter whose argument label is keyPathMember and that accepts one of the key path types (e.g., KeyPath, WritableKeyPath).

Source compatibility

This is an additive proposal, which makes cuurently ill-formed syntax well-formed but otherwise does not affect existing code. First, only types that opt in to @keyPathMemberLookup will be affected. Second, even for types that adopt @keyPathMemberLookup, the change is source-compatible because the transformation to use subscript(keyPathMember:) is only applied when there is no member of the given name.

Effect on ABI stability

This feature is implementable entirely in the type checker, as (effectively) a syntactic transformation on member access expressions. It therefore has no impact on the ABI.

Effect on API resilience

Adding @keyPathMemberLookup is a resilient change to a type, as is the addition of the subscript

30 Likes

Why do you need a new attribute? Would it make sense to extend the implementation of the existing attribute to look for both stringly-typed and keypath-typed subscript overloads?

6 Likes

That's a good point! We could re-use the existing attribute, and look for both kinds of subscripts. I should really clarify what happens if you have both on the same type---I assume we'd prefer the keypath-typed subscript (when it works) to the stringly-typed subscript, because we want to maintain as much static type information as we can.

Doug

10 Likes

If possible, I would not only use @dynamicMemberLookup as the attribute, but also have it translate to a subscript(dynamicMember:) that takes KeyPaths instead of Strings. That makes the mental model of this feature into an extension of the existing feature rather than a parallel one.

(It's still "dynamic" because there's still a runtime lookup—it's just a lookup by keypath instead of by string.)

8 Likes

Yes, it should be possible, and I do like the idea of keeping this as one conceptual feature that has a both more-strongly-typed and a less-strongly-typed variants. I'll have to get over that this proposal doesn't feel like it fits with the term "dynamic", given that it kicks in given entirely static information (and makes it slightly more dynamic by going through a KeyPath).

Doug

Yes please! Having written lenses recently, I’ve wanted exactly this feature.

My initial thought when running into this need was actually ”maybe I can use @dynamicMemberLookup”, but I quickly realized I would lose all static typing. Adding this to @dynamicMemberLookup would feel completely natural to me – the lookup is still dynamic, it just preserves more static type info.

Can we have it in 5.1? :wink:

Could this feature be used to allow conformances to a protocol P through dynamic lookup if the key paths available are of the from subscript(KeyPath<T, U>) -> U where T: P? I'm seeing this as a useful way to implement a kind of decorator pattern, without inheritance:

protocol PersonType {
    var firstName: String { get }
    var lastName: String { get }
}

final class Person: PersonType {
    let firstName: String
    let lastName: String
}

@dynamicMemberLookup
class Decorator<T> {
    private var decoratedObject: T
  
    subscript<U>(keyPath keyPath: KeyPath<T, U>) -> U {
        return decoratedObject[keyPath: keyPath]
    }

    subscript<U>(keyPath keyPath: WritableKeyPath<T, U>) -> U {
        get { return decoratedObject[keyPath: keyPath] }
        set { decoratedObject[keyPath: keyPath] = value }
    }
}

final class PersonDecorator: Decorator<Person>, PersonType {
    var fullName: String { return "\(self.firstName) \(self.lastName)" }
}

@tkremenek can we add a new requirement for formal proposals that they explicitly mention backwards compatibility? For example this feature has no impact on ABI and it's a language feature, but can I use it on older OS's if I update the project to whatever Swift version this potentially will introduced with, and if so on what range of OS's and why?! I think this is very important information that must be required now to avoid peoples need to ask about that, like myself in this particular case.

5 Likes

I think you need to make your decorator types into classes.

Other than that, it would be interesting to see what we can do with it when combined with RawRepresentable protocol. The only missing gaps of this feature are static members (as we don't have static subscript yet because it's reserved for potentially different language features) and no function key-paths.

Maybe we should call it 'impact on runtime' instead of backward compatibility. Choosing to make it backward compatible (either by updating old OS runtime or by providing stubs) should be an OS vendor decision.

Of course, a feature with no impact on ABI and runtime will be implicitly backward compatible, but for the other one, this is IMHO beyond a proposal scope to define if it will be backward compatible and with witch OS version (especially as it may be very difficult to define it if the number of supported OSes grows).

1 Like

Correct. Fixed!

It’s a great suggestion.

1 Like

For this specific proposal, the change is limited to the type checker: it does not affect ABI nor does it require additions or changes to the Swift runtime, so one could use this feature and still backward-deploy to an older runtime.

Doug

7 Likes

Obligatory "a lens is a key path, what you're describing is a reference" terminology correction. Other than that, SGTM.

3 Likes

Maybe too off topic, but curious what advantage this has over

extension PersonType {
    var fullName: String { return "\(self.firstName) \(self.lastName)" }
}

?

It looks like the compiler will be able to type check the key paths from T to U since Lens is generic over T. Will this usable for non-generic types? Could I write something like this? (I don't want to, just trying to get a sense of how this works.)

@keyPathMemberLookup
struct NotAnInt {
    var value: Int

    subscript<T>(keyPathMember path: WriteableKeyPath<Int, T> -> T {
        return value[keyPath: path]
    }
}

let x = NotAnInt(value: 4)
let y = x.nonzeroBitCount    // 1

This sounds pretty great. The big distinctions from @dynamicMemberLookup is that you have type checking the whole way through, and can return different types from the subscript, but I think reusing the attribute is still fine.

1 Like

This won’t work the way the feature is defined today, because dynamic member lookup operates on expressions (eg, a.b), whereas conformances are checked by matching up declarations. Many of the kinds of conversions that work in expressions don’t kick in for conformances, either, such as subtyping.

Doug

Yes, as long as the “root” of the KeyPath in the subscript parameter is something the compiler can reason about (Int in your example, T from the enclosing Lens type in mine), it’ll work.

What about reusing the subscript argument label, too?

Doug

This is a bit less dynamic than the current thing we’ve named dynamic, so I think the subscript label in your pitch is better. We have precedent for overloads with different spellings with @dynamicCallable, so I think we’re better off just giving it the most apt name, rather than aiming for re-use.

1 Like

It was a dummy example. I have more appropriate but more complex examples in my own projects. This becomes useful when you want to add properties to the decorator. Decorator pattern - Wikipedia

1 Like