Smart KeyPaths

[Michael_J_LeHew_Jr]

Hi friendly swift-evolution folks,

The Foundation and Swift team would like for you to consider the following proposal:

Many thanks,
-Michael

Smart KeyPaths: Better Key-Value Coding for Swift
Proposal: SE-NNNN
Authors: David Smith <https://github.com/Catfish-Man&gt;, Michael LeHew <https://github.com/mlehew&gt;, Joe Groff <https://github.com/jckarter&gt;
Review Manager: TBD
Status: Awaiting Review
Associated PRs:
#644 <https://github.com/apple/swift-evolution/pull/644&gt;
Introduction
We propose a family of concrete Key Path types that represent uninvoked references to properties that can be composed to form paths through many values and directly get/set their underlying values.

Motivation
We Can Do Better than String

On Darwin platforms Swift's existing #keyPath() syntax provides a convenient way to safely refer to properties. Unfortunately, once validated, the expression becomes a String which has a number of important limitations:

Loss of type information (requiring awkward Any APIs)
Unnecessarily slow to parse
Only applicable to NSObjects
Limited to Darwin platforms
Use/Mention Distinctions

While methods can be referred to without invoking them (let x = foo.bar instead of let x = foo.bar()), this is not currently possible for properties and subscripts.

Making indirect references to a properties' concrete types also lets us expose metadata about the property, and in the future additional behaviors.

More Expressive KeyPaths

We would also like to support being able to use Key Paths to access into collections, which is not currently possible.

Proposed solution
We propose introducing a new expression akin to Type.method, but for properties and subscripts. These property reference expressions produce KeyPath objects, rather than Strings. KeyPaths are a family of generic classes (structs and protocols here would be ideal, but requires generalized existentials) which encapsulate a property reference or chain of property references, including the type, mutability, property name(s), and ability to set/get values.

Here's a sample of it in use:

Swift
struct Person {
    var name: String
    var friends: [Person]
    var bestFriend: Person?
}

var han = Person(name: "Han Solo", friends: )
var luke = Person(name: "Luke Skywalker", friends: [han])

let firstFriendsNameKeyPath = Person.friends[0].name

let firstFriend = luke[path] // han

// or equivalently, with type inferred from context
let firstFriendName = luke[.friends[0].name]

// rename Luke's first friend
luke[firstFriendsNameKeyPath] = "A Disreputable Smuggler"

let bestFriendsName = luke[.bestFriend]?.name // nil, if he is the last jedi
Detailed design
Core KeyPath Types

KeyPaths are a hierarchy of progressively more specific classes, based on whether we have prior knowledge of the path through the object graph we wish to traverse.

Unknown Path / Unknown Root Type

AnyKeyPath is fully type-erased, referring to 'any route' through an object/value graph for 'any root'. Because of type-erasure many operations can fail at runtime and are thus nillable.

Swift
class AnyKeyPath: CustomDebugStringConvertible, Hashable {
    // MARK - Composition
    // Returns nil if path.rootType != self.valueType
    func appending(path: AnyKeyPath) -> AnyKeyPath?
    
    // MARK - Runtime Information
    class var rootType: Any.Type
    class var valueType: Any.Type
    
    static func == (lhs: AnyKeyPath, rhs: AnyKeyPath) -> Bool
    var hashValue: Int
}
Unknown Path / Known Root Type

If we know a little more type information (what kind of thing the key path is relative to), then we can use PartialKeyPath <Root>, which refers to an 'any route' from a known root:

Swift
class PartialKeyPath<Root>: AnyKeyPath {
    // MARK - Composition
    // Returns nil if Value != self.valueType
    func appending(path: AnyKeyPath) -> PartialKeyPath<Root>?
    func appending<Value, AppendedValue>(path: KeyPath<Value, AppendedValue>) -> KeyPath<Root, AppendedValue>?
    func appending<Value, AppendedValue>(path: ReferenceKeyPath<Value, AppendedValue>) -> ReferenceKeyPath<Root, AppendedValue>?
}
Known Path / Known Root Type

When we know both what the path is relative to and what it refers to, we can use KeyPath. Thanks to the knowledge of the Root and Value types, all of the failable operations lose their Optional.

Swift
public class KeyPath<Root, Value>: PartialKeyPath<Root> {
    // MARK - Composition
    func appending<AppendedValue>(path: KeyPath<Value, AppendedValue>) -> KeyPath<Root, AppendedValue>
    func appending<AppendedValue>(path: WritableKeyPath<Value, AppendedValue>) -> Self
    func appending<AppendedValue>(path: ReferenceWritableKeyPath<Value, AppendedValue>) -> ReferenceWritableKeyPath<Root, AppendedValue>
}
Value/Reference Mutation Semantics Mutation

Finally, we have a pair of subclasses encapsulating value/reference mutation semantics. These have to be distinct because mutating a copy of a value is not very useful, so we need to mutate an inout value.

Swift
class WritableKeyPath<Root, Value>: KeyPath<Root, Value> {
    // MARK - Composition
    func appending<AppendedPathValue>(path: WritableKeyPath<Value, AppendedPathValue>) -> WritableKeyPath<Root, AppendedPathValue>
}

class ReferenceWritableKeyPath<Root, Value>: WritableKeyPath<Root, Value> {
    override func appending<AppendedPathValue>(path: WritableKeyPath<Value, AppendedPathValue>) -> ReferenceWritableKeyPath<Root, AppendedPathValue>
}
Access and Mutation Through KeyPaths

To get or set values for a given root and key path we effectively add the following subscripts to all Swift types.

Swift
extension Any {
    subscript(path: AnyKeyPath) -> Any? { get }
    subscript<Root: Self>(path: PartialKeyPath<Root>) -> Any { get }
    subscript<Root: Self, Value>(path: KeyPath<Root, Value>) -> Value { get }
    subscript<Root: Self, Value>(path: WritableKeyPath<Root, Value>) -> Value { set, get }
}
This allows for code like

Swift
person[.name] // Self.type is inferred
which is both appealingly readable, and doesn't require read-modify-write copies (subscripts access self inout). Conflicts with existing subscripts are avoided by using generic subscripts to specifically only accept key paths with a Root of the type in question.

Referencing Key Paths

Forming a KeyPath borrows from the same syntax used to reference methods and initializers,Type.instanceMethod only now working for properties and collections. Optionals are handled via optional-chaining. Multiply dotted expressions are allowed as well, and work just as if they were composed via the appending methods on KeyPath.

There is no change or interaction with the #keyPath() syntax introduced in Swift 3.

Performance

The performance of interacting with a property via KeyPaths should be close to the cost of calling the property directly.

Source compatibility
This change is additive and there should no affect on existing source.

Effect on ABI stability
This feature adds the following requirements to ABI stability:

mechanism to access key paths of public properties
We think a protocol-based design would be preferable once the language has sufficient support for generalized existentials to make that ergonomic. By keeping the class hierarchy closed and the concrete implementations private to the implementation it should be tractable to provide compatibility with an open protocol-based design in the future.

Effect on API resilience
This should not significantly impact API resilience, as it merely provides a new mechanism for operating on existing APIs.

Alternatives considered
More Features

Various drafts of this proposal have included additional features (decomposable key paths, prefix comparisons, support for custom KeyPath subclasses, creating a KeyPath from a String at runtime, KeyPaths conforming to Codable, bound key paths as a concrete type, etc.). We anticipate approaching these enhancements additively once the core KeyPath functionality is in place.

Spelling

We also explored many different spellings, each with different strengths. We have chosen the current syntax due to the balance with existing function type references.

Current #keyPath Lisp-style
Person.friends[0].name #keyPath(Person, .friends[0].name) `Person.friend.name
luke[.friends[0].name] #keyPath(luke, .friends[0].name) luke`.friends[0].name
luke.friends[0][.name] #keyPath(luke.friends[0], .name) luke.friends[0]`.name
While the crispness is very appealing, the spelling of the 'escape' character was hard to agree upon (along with the fact that it requires parentheses to reduce ambiguity). #keyPath was very specific, but verbose especially when composing multiple key paths together.

[Brent_Royal-Gordon]

I don't know how to express the level of "Yes" I feel in response to this proposal without using language that's inappropriate on a public mailing list.

A few questions:

How do key paths interact with Optionals? Can you form a key path to `Person.bestFriend.name`, and is that the syntax, or is it `Person.bestFriend?.name`?

Foundation key paths have a sometimes-useful property where, if they traverse a collection, the result becomes a collection of the ending type. Is a similar feature planned for smart key paths—perhaps something like `Person.friends.name`?

Given that Swift has a syntax called keypath which is unrelated to these "key paths", have you considered using a different name to avoid confusion? Maybe "property paths" or "accessor paths"?

···

On Mar 17, 2017, at 10:04 AM, Michael LeHew via swift-evolution <swift-evolution@swift.org> wrote:
Introduction
We propose a family of concrete Key Path types that represent uninvoked references to properties that can be composed to form paths through many values and directly get/set their underlying values.

--
Brent Royal-Gordon
Architechies

[bjhomer]

This looks great!

What happens in the case when there is a static property by the same name as an instance property? e.g.

struct Foo {
    static var bar: Bar
    var bar: Bar
}

Foo.bar // What is this?

Is it still possible to reference both the static property and a KeyPath to the instance method?

-BJ

···

On Mar 17, 2017, at 11:04 AM, Michael LeHew via swift-evolution <swift-evolution@swift.org> wrote:

Hi friendly swift-evolution folks,

The Foundation and Swift team would like for you to consider the following proposal:

Many thanks,
-Michael

Smart KeyPaths: Better Key-Value Coding for Swift
Proposal: SE-NNNN
Authors: David Smith <https://github.com/Catfish-Man&gt;, Michael LeHew <https://github.com/mlehew&gt;, Joe Groff <https://github.com/jckarter&gt;
Review Manager: TBD
Status: Awaiting Review
Associated PRs:
#644 <https://github.com/apple/swift-evolution/pull/644&gt;
Introduction
We propose a family of concrete Key Path types that represent uninvoked references to properties that can be composed to form paths through many values and directly get/set their underlying values.

Motivation
We Can Do Better than String

On Darwin platforms Swift's existing keypath() syntax provides a convenient way to safely refer to properties. Unfortunately, once validated, the expression becomes a String which has a number of important limitations:

Loss of type information (requiring awkward Any APIs)
Unnecessarily slow to parse
Only applicable to NSObjects
Limited to Darwin platforms
Use/Mention Distinctions

While methods can be referred to without invoking them (let x = foo.bar instead of let x = foo.bar()), this is not currently possible for properties and subscripts.

Making indirect references to a properties' concrete types also lets us expose metadata about the property, and in the future additional behaviors.

More Expressive KeyPaths

We would also like to support being able to use Key Paths to access into collections, which is not currently possible.

Proposed solution
We propose introducing a new expression akin to Type.method, but for properties and subscripts. These property reference expressions produce KeyPath objects, rather than Strings. KeyPaths are a family of generic classes (structs and protocols here would be ideal, but requires generalized existentials) which encapsulate a property reference or chain of property references, including the type, mutability, property name(s), and ability to set/get values.

Here's a sample of it in use:

Swift
struct Person {
    var name: String
    var friends: [Person]
    var bestFriend: Person?
}

var han = Person(name: "Han Solo", friends: )
var luke = Person(name: "Luke Skywalker", friends: [han])

let firstFriendsNameKeyPath = Person.friends[0].name

let firstFriend = luke[path] // han

// or equivalently, with type inferred from context
let firstFriendName = luke[.friends[0].name]

// rename Luke's first friend
luke[firstFriendsNameKeyPath] = "A Disreputable Smuggler"

let bestFriendsName = luke[.bestFriend]?.name // nil, if he is the last jedi
Detailed design
Core KeyPath Types

KeyPaths are a hierarchy of progressively more specific classes, based on whether we have prior knowledge of the path through the object graph we wish to traverse.

Unknown Path / Unknown Root Type

AnyKeyPath is fully type-erased, referring to 'any route' through an object/value graph for 'any root'. Because of type-erasure many operations can fail at runtime and are thus nillable.

Swift
class AnyKeyPath: CustomDebugStringConvertible, Hashable {
    // MARK - Composition
    // Returns nil if path.rootType != self.valueType
    func appending(path: AnyKeyPath) -> AnyKeyPath?
    
    // MARK - Runtime Information
    class var rootType: Any.Type
    class var valueType: Any.Type
    
    static func == (lhs: AnyKeyPath, rhs: AnyKeyPath) -> Bool
    var hashValue: Int
}
Unknown Path / Known Root Type

If we know a little more type information (what kind of thing the key path is relative to), then we can use PartialKeyPath <Root>, which refers to an 'any route' from a known root:

Swift
class PartialKeyPath<Root>: AnyKeyPath {
    // MARK - Composition
    // Returns nil if Value != self.valueType
    func appending(path: AnyKeyPath) -> PartialKeyPath<Root>?
    func appending<Value, AppendedValue>(path: KeyPath<Value, AppendedValue>) -> KeyPath<Root, AppendedValue>?
    func appending<Value, AppendedValue>(path: ReferenceKeyPath<Value, AppendedValue>) -> ReferenceKeyPath<Root, AppendedValue>?
}
Known Path / Known Root Type

When we know both what the path is relative to and what it refers to, we can use KeyPath. Thanks to the knowledge of the Root and Value types, all of the failable operations lose their Optional.

Swift
public class KeyPath<Root, Value>: PartialKeyPath<Root> {
    // MARK - Composition
    func appending<AppendedValue>(path: KeyPath<Value, AppendedValue>) -> KeyPath<Root, AppendedValue>
    func appending<AppendedValue>(path: WritableKeyPath<Value, AppendedValue>) -> Self
    func appending<AppendedValue>(path: ReferenceWritableKeyPath<Value, AppendedValue>) -> ReferenceWritableKeyPath<Root, AppendedValue>
}
Value/Reference Mutation Semantics Mutation

Finally, we have a pair of subclasses encapsulating value/reference mutation semantics. These have to be distinct because mutating a copy of a value is not very useful, so we need to mutate an inout value.

Swift
class WritableKeyPath<Root, Value>: KeyPath<Root, Value> {
    // MARK - Composition
    func appending<AppendedPathValue>(path: WritableKeyPath<Value, AppendedPathValue>) -> WritableKeyPath<Root, AppendedPathValue>
}

class ReferenceWritableKeyPath<Root, Value>: WritableKeyPath<Root, Value> {
    override func appending<AppendedPathValue>(path: WritableKeyPath<Value, AppendedPathValue>) -> ReferenceWritableKeyPath<Root, AppendedPathValue>
}
Access and Mutation Through KeyPaths

To get or set values for a given root and key path we effectively add the following subscripts to all Swift types.

Swift
extension Any {
    subscript(path: AnyKeyPath) -> Any? { get }
    subscript<Root: Self>(path: PartialKeyPath<Root>) -> Any { get }
    subscript<Root: Self, Value>(path: KeyPath<Root, Value>) -> Value { get }
    subscript<Root: Self, Value>(path: WritableKeyPath<Root, Value>) -> Value { set, get }
}
This allows for code like

Swift
person[.name] // Self.type is inferred
which is both appealingly readable, and doesn't require read-modify-write copies (subscripts access self inout). Conflicts with existing subscripts are avoided by using generic subscripts to specifically only accept key paths with a Root of the type in question.

Referencing Key Paths

Forming a KeyPath borrows from the same syntax used to reference methods and initializers,Type.instanceMethod only now working for properties and collections. Optionals are handled via optional-chaining. Multiply dotted expressions are allowed as well, and work just as if they were composed via the appending methods on KeyPath.

There is no change or interaction with the keypath() syntax introduced in Swift 3.

Performance

The performance of interacting with a property via KeyPaths should be close to the cost of calling the property directly.

Source compatibility
This change is additive and there should no affect on existing source.

Effect on ABI stability
This feature adds the following requirements to ABI stability:

mechanism to access key paths of public properties
We think a protocol-based design would be preferable once the language has sufficient support for generalized existentials to make that ergonomic. By keeping the class hierarchy closed and the concrete implementations private to the implementation it should be tractable to provide compatibility with an open protocol-based design in the future.

Effect on API resilience
This should not significantly impact API resilience, as it merely provides a new mechanism for operating on existing APIs.

Alternatives considered
More Features

Various drafts of this proposal have included additional features (decomposable key paths, prefix comparisons, support for custom KeyPath subclasses, creating a KeyPath from a String at runtime, KeyPaths conforming to Codable, bound key paths as a concrete type, etc.). We anticipate approaching these enhancements additively once the core KeyPath functionality is in place.

Spelling

We also explored many different spellings, each with different strengths. We have chosen the current syntax due to the balance with existing function type references.

Current keypath Lisp-style
Person.friends[0].name keypath(Person, .friends[0].name) `Person.friend.name
luke[.friends[0].name] keypath(luke, .friends[0].name) luke`.friends[0].name
luke.friends[0][.name] keypath(luke.friends[0], .name) luke.friends[0]`.name
While the crispness is very appealing, the spelling of the 'escape' character was hard to agree upon (along with the fact that it requires parentheses to reduce ambiguity). keypath was very specific, but verbose especially when composing multiple key paths together.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[anandabits (Matthew Johnson)]

Hi friendly swift-evolution folks,

The Foundation and Swift team would like for you to consider the following proposal:

This proposal is really incredible! It is an invaluable addition to the language - far better than simple first-class properties. I really can’t wait to see it implemented! The design looks very solid. I’m especially happy to see that a path to eventually get away from using classes has already been identified and planned for.

Thank you so much for bringing this forward in Swift 4. It is a wonderful (and rather unexpected) surprise!

Seeing this makes me *really* wish we had a way to get at a collection of `PartialKeyPath<Self>` for all the (visible) properties of a type. I guess the visible part of that makes it tricky. We can always work around it in the meantime.

···

On Mar 17, 2017, at 12:04 PM, Michael LeHew via swift-evolution <swift-evolution@swift.org> wrote:

Many thanks,
-Michael

Smart KeyPaths: Better Key-Value Coding for Swift
Proposal: SE-NNNN
Authors: David Smith <https://github.com/Catfish-Man&gt;, Michael LeHew <https://github.com/mlehew&gt;, Joe Groff <https://github.com/jckarter&gt;
Review Manager: TBD
Status: Awaiting Review
Associated PRs:
#644 <https://github.com/apple/swift-evolution/pull/644&gt;
Introduction
We propose a family of concrete Key Path types that represent uninvoked references to properties that can be composed to form paths through many values and directly get/set their underlying values.

Motivation
We Can Do Better than String

On Darwin platforms Swift's existing keypath() syntax provides a convenient way to safely refer to properties. Unfortunately, once validated, the expression becomes a String which has a number of important limitations:

Loss of type information (requiring awkward Any APIs)
Unnecessarily slow to parse
Only applicable to NSObjects
Limited to Darwin platforms
Use/Mention Distinctions

While methods can be referred to without invoking them (let x = foo.bar instead of let x = foo.bar()), this is not currently possible for properties and subscripts.

Making indirect references to a properties' concrete types also lets us expose metadata about the property, and in the future additional behaviors.

More Expressive KeyPaths

We would also like to support being able to use Key Paths to access into collections, which is not currently possible.

Proposed solution
We propose introducing a new expression akin to Type.method, but for properties and subscripts. These property reference expressions produce KeyPath objects, rather than Strings. KeyPaths are a family of generic classes (structs and protocols here would be ideal, but requires generalized existentials) which encapsulate a property reference or chain of property references, including the type, mutability, property name(s), and ability to set/get values.

Here's a sample of it in use:

Swift
struct Person {
    var name: String
    var friends: [Person]
    var bestFriend: Person?
}

var han = Person(name: "Han Solo", friends: )
var luke = Person(name: "Luke Skywalker", friends: [han])

let firstFriendsNameKeyPath = Person.friends[0].name

let firstFriend = luke[path] // han

// or equivalently, with type inferred from context
let firstFriendName = luke[.friends[0].name]

// rename Luke's first friend
luke[firstFriendsNameKeyPath] = "A Disreputable Smuggler"

let bestFriendsName = luke[.bestFriend]?.name // nil, if he is the last jedi
Detailed design
Core KeyPath Types

KeyPaths are a hierarchy of progressively more specific classes, based on whether we have prior knowledge of the path through the object graph we wish to traverse.

Unknown Path / Unknown Root Type

AnyKeyPath is fully type-erased, referring to 'any route' through an object/value graph for 'any root'. Because of type-erasure many operations can fail at runtime and are thus nillable.

Swift
class AnyKeyPath: CustomDebugStringConvertible, Hashable {
    // MARK - Composition
    // Returns nil if path.rootType != self.valueType
    func appending(path: AnyKeyPath) -> AnyKeyPath?
    
    // MARK - Runtime Information
    class var rootType: Any.Type
    class var valueType: Any.Type
    
    static func == (lhs: AnyKeyPath, rhs: AnyKeyPath) -> Bool
    var hashValue: Int
}
Unknown Path / Known Root Type

If we know a little more type information (what kind of thing the key path is relative to), then we can use PartialKeyPath <Root>, which refers to an 'any route' from a known root:

Swift
class PartialKeyPath<Root>: AnyKeyPath {
    // MARK - Composition
    // Returns nil if Value != self.valueType
    func appending(path: AnyKeyPath) -> PartialKeyPath<Root>?
    func appending<Value, AppendedValue>(path: KeyPath<Value, AppendedValue>) -> KeyPath<Root, AppendedValue>?
    func appending<Value, AppendedValue>(path: ReferenceKeyPath<Value, AppendedValue>) -> ReferenceKeyPath<Root, AppendedValue>?
}
Known Path / Known Root Type

When we know both what the path is relative to and what it refers to, we can use KeyPath. Thanks to the knowledge of the Root and Value types, all of the failable operations lose their Optional.

Swift
public class KeyPath<Root, Value>: PartialKeyPath<Root> {
    // MARK - Composition
    func appending<AppendedValue>(path: KeyPath<Value, AppendedValue>) -> KeyPath<Root, AppendedValue>
    func appending<AppendedValue>(path: WritableKeyPath<Value, AppendedValue>) -> Self
    func appending<AppendedValue>(path: ReferenceWritableKeyPath<Value, AppendedValue>) -> ReferenceWritableKeyPath<Root, AppendedValue>
}
Value/Reference Mutation Semantics Mutation

Finally, we have a pair of subclasses encapsulating value/reference mutation semantics. These have to be distinct because mutating a copy of a value is not very useful, so we need to mutate an inout value.

Swift
class WritableKeyPath<Root, Value>: KeyPath<Root, Value> {
    // MARK - Composition
    func appending<AppendedPathValue>(path: WritableKeyPath<Value, AppendedPathValue>) -> WritableKeyPath<Root, AppendedPathValue>
}

class ReferenceWritableKeyPath<Root, Value>: WritableKeyPath<Root, Value> {
    override func appending<AppendedPathValue>(path: WritableKeyPath<Value, AppendedPathValue>) -> ReferenceWritableKeyPath<Root, AppendedPathValue>
}
Access and Mutation Through KeyPaths

To get or set values for a given root and key path we effectively add the following subscripts to all Swift types.

Swift
extension Any {
    subscript(path: AnyKeyPath) -> Any? { get }
    subscript<Root: Self>(path: PartialKeyPath<Root>) -> Any { get }
    subscript<Root: Self, Value>(path: KeyPath<Root, Value>) -> Value { get }
    subscript<Root: Self, Value>(path: WritableKeyPath<Root, Value>) -> Value { set, get }
}
This allows for code like

Swift
person[.name] // Self.type is inferred
which is both appealingly readable, and doesn't require read-modify-write copies (subscripts access self inout). Conflicts with existing subscripts are avoided by using generic subscripts to specifically only accept key paths with a Root of the type in question.

Referencing Key Paths

Forming a KeyPath borrows from the same syntax used to reference methods and initializers,Type.instanceMethod only now working for properties and collections. Optionals are handled via optional-chaining. Multiply dotted expressions are allowed as well, and work just as if they were composed via the appending methods on KeyPath.

There is no change or interaction with the keypath() syntax introduced in Swift 3.

Performance

The performance of interacting with a property via KeyPaths should be close to the cost of calling the property directly.

Source compatibility
This change is additive and there should no affect on existing source.

Effect on ABI stability
This feature adds the following requirements to ABI stability:

mechanism to access key paths of public properties
We think a protocol-based design would be preferable once the language has sufficient support for generalized existentials to make that ergonomic. By keeping the class hierarchy closed and the concrete implementations private to the implementation it should be tractable to provide compatibility with an open protocol-based design in the future.

Effect on API resilience
This should not significantly impact API resilience, as it merely provides a new mechanism for operating on existing APIs.

Alternatives considered
More Features

Various drafts of this proposal have included additional features (decomposable key paths, prefix comparisons, support for custom KeyPath subclasses, creating a KeyPath from a String at runtime, KeyPaths conforming to Codable, bound key paths as a concrete type, etc.). We anticipate approaching these enhancements additively once the core KeyPath functionality is in place.

Spelling

We also explored many different spellings, each with different strengths. We have chosen the current syntax due to the balance with existing function type references.

Current keypath Lisp-style
Person.friends[0].name keypath(Person, .friends[0].name) `Person.friend.name
luke[.friends[0].name] keypath(luke, .friends[0].name) luke`.friends[0].name
luke.friends[0][.name] keypath(luke.friends[0], .name) luke.friends[0]`.name
While the crispness is very appealing, the spelling of the 'escape' character was hard to agree upon (along with the fact that it requires parentheses to reduce ambiguity). keypath was very specific, but verbose especially when composing multiple key paths together.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[hartbit (David Hart)]

Sent off-list by mistake:

Nice proposal. I have a few comments inline:

Hi friendly swift-evolution folks,

The Foundation and Swift team would like for you to consider the following proposal:

Many thanks,
-Michael

Smart KeyPaths: Better Key-Value Coding for Swift
Proposal: SE-NNNN
Authors: David Smith <https://github.com/Catfish-Man&gt;, Michael LeHew <https://github.com/mlehew&gt;, Joe Groff <https://github.com/jckarter&gt;
Review Manager: TBD
Status: Awaiting Review
Associated PRs:
#644 <https://github.com/apple/swift-evolution/pull/644&gt;
Introduction
We propose a family of concrete Key Path types that represent uninvoked references to properties that can be composed to form paths through many values and directly get/set their underlying values.

Motivation
We Can Do Better than String

On Darwin platforms Swift's existing keypath() syntax provides a convenient way to safely refer to properties. Unfortunately, once validated, the expression becomes a String which has a number of important limitations:

Loss of type information (requiring awkward Any APIs)
Unnecessarily slow to parse
Only applicable to NSObjects
Limited to Darwin platforms
Use/Mention Distinctions

While methods can be referred to without invoking them (let x = foo.bar instead of let x = foo.bar()), this is not currently possible for properties and subscripts.

Making indirect references to a properties' concrete types also lets us expose metadata about the property, and in the future additional behaviors.

What metadata is attached? How is it accessed? What future features are you thinking about?

More Expressive KeyPaths

We would also like to support being able to use Key Paths to access into collections, which is not currently possible.

Proposed solution
We propose introducing a new expression akin to Type.method, but for properties and subscripts. These property reference expressions produce KeyPath objects, rather than Strings. KeyPaths are a family of generic classes (structs and protocols here would be ideal, but requires generalized existentials)

How different would the design be with generalized existentials? Are they plans to migrate to that design once we do get generalized existentials?

which encapsulate a property reference or chain of property references, including the type, mutability, property name(s), and ability to set/get values.

Here's a sample of it in use:

Swift
struct Person {
    var name: String
    var friends: [Person]
    var bestFriend: Person?
}

var han = Person(name: "Han Solo", friends: )
var luke = Person(name: "Luke Skywalker", friends: [han])

let firstFriendsNameKeyPath = Person.friends[0].name

let firstFriend = luke[path] // han

// or equivalently, with type inferred from context
let firstFriendName = luke[.friends[0].name]

// rename Luke's first friend
luke[firstFriendsNameKeyPath] = "A Disreputable Smuggler"

let bestFriendsName = luke[.bestFriend]?.name // nil, if he is the last jedi
Detailed design
Core KeyPath Types

KeyPaths are a hierarchy of progressively more specific classes, based on whether we have prior knowledge of the path through the object graph we wish to traverse.

Unknown Path / Unknown Root Type

AnyKeyPath is fully type-erased, referring to 'any route' through an object/value graph for 'any root'. Because of type-erasure many operations can fail at runtime and are thus nillable.

Swift
class AnyKeyPath: CustomDebugStringConvertible, Hashable {
    // MARK - Composition
    // Returns nil if path.rootType != self.valueType
    func appending(path: AnyKeyPath) -> AnyKeyPath?
    
    // MARK - Runtime Information
    class var rootType: Any.Type
    class var valueType: Any.Type
    
    static func == (lhs: AnyKeyPath, rhs: AnyKeyPath) -> Bool
    var hashValue: Int
}
Unknown Path / Known Root Type

If we know a little more type information (what kind of thing the key path is relative to), then we can use PartialKeyPath <Root>, which refers to an 'any route' from a known root:

Swift
class PartialKeyPath<Root>: AnyKeyPath {
    // MARK - Composition
    // Returns nil if Value != self.valueType
    func appending(path: AnyKeyPath) -> PartialKeyPath<Root>?
    func appending<Value, AppendedValue>(path: KeyPath<Value, AppendedValue>) -> KeyPath<Root, AppendedValue>?
    func appending<Value, AppendedValue>(path: ReferenceKeyPath<Value, AppendedValue>) -> ReferenceKeyPath<Root, AppendedValue>?
}
Known Path / Known Root Type

When we know both what the path is relative to and what it refers to, we can use KeyPath. Thanks to the knowledge of the Root and Value types, all of the failable operations lose their Optional.

Swift
public class KeyPath<Root, Value>: PartialKeyPath<Root> {
    // MARK - Composition
    func appending<AppendedValue>(path: KeyPath<Value, AppendedValue>) -> KeyPath<Root, AppendedValue>
    func appending<AppendedValue>(path: WritableKeyPath<Value, AppendedValue>) -> Self
    func appending<AppendedValue>(path: ReferenceWritableKeyPath<Value, AppendedValue>) -> ReferenceWritableKeyPath<Root, AppendedValue>
}
Value/Reference Mutation Semantics Mutation

Finally, we have a pair of subclasses encapsulating value/reference mutation semantics. These have to be distinct because mutating a copy of a value is not very useful, so we need to mutate an inout value.

Swift
class WritableKeyPath<Root, Value>: KeyPath<Root, Value> {
    // MARK - Composition
    func appending<AppendedPathValue>(path: WritableKeyPath<Value, AppendedPathValue>) -> WritableKeyPath<Root, AppendedPathValue>
}

class ReferenceWritableKeyPath<Root, Value>: WritableKeyPath<Root, Value> {
    override func appending<AppendedPathValue>(path: WritableKeyPath<Value, AppendedPathValue>) -> ReferenceWritableKeyPath<Root, AppendedPathValue>
}
Access and Mutation Through KeyPaths

To get or set values for a given root and key path we effectively add the following subscripts to all Swift types.

Swift
extension Any {
    subscript(path: AnyKeyPath) -> Any? { get }
    subscript<Root: Self>(path: PartialKeyPath<Root>) -> Any { get }
    subscript<Root: Self, Value>(path: KeyPath<Root, Value>) -> Value { get }
    subscript<Root: Self, Value>(path: WritableKeyPath<Root, Value>) -> Value { set, get }
}
This allows for code like

Swift
person[.name] // Self.type is inferred

Perhaps I'm missing something, but what does that syntax bring compared to person.name? I see quite a few examples of the key paths being used literally in the subscript syntax but fail to see the usefulness of doing that. Can you give use cases?

which is both appealingly readable, and doesn't require read-modify-write copies (subscripts access self inout). Conflicts with existing subscripts are avoided by using generic subscripts to specifically only accept key paths with a Root of the type in question.

Referencing Key Paths

Forming a KeyPath borrows from the same syntax used to reference methods and initializers,Type.instanceMethod only now working for properties and collections. Optionals are handled via optional-chaining. Multiply dotted expressions are allowed as well, and work just as if they were composed via the appending methods on KeyPath.

There is no change or interaction with the keypath() syntax introduced in Swift 3.

Performance

The performance of interacting with a property via KeyPaths should be close to the cost of calling the property directly.

Source compatibility
This change is additive and there should no affect on existing source.

Effect on ABI stability
This feature adds the following requirements to ABI stability:

mechanism to access key paths of public properties
We think a protocol-based design would be preferable once the language has sufficient support for generalized existentials to make that ergonomic. By keeping the class hierarchy closed and the concrete implementations private to the implementation it should be tractable to provide compatibility with an open protocol-based design in the future.

Effect on API resilience
This should not significantly impact API resilience, as it merely provides a new mechanism for operating on existing APIs.

Alternatives considered
More Features

Various drafts of this proposal have included additional features (decomposable key paths, prefix comparisons, support for custom KeyPath subclasses, creating a KeyPath from a String at runtime, KeyPaths conforming to Codable, bound key paths as a concrete type, etc.). We anticipate approaching these enhancements additively once the core KeyPath functionality is in place.

Spelling

We also explored many different spellings, each with different strengths. We have chosen the current syntax due to the balance with existing function type references.

Current keypath Lisp-style
Person.friends[0].name keypath(Person, .friends[0].name) `Person.friend.name
luke[.friends[0].name] keypath(luke, .friends[0].name) luke`.friends[0].name
luke.friends[0][.name] keypath(luke.friends[0], .name) luke.friends[0]`.name
While the crispness is very appealing, the spelling of the 'escape' character was hard to agree upon (along with the fact that it requires parentheses to reduce ambiguity). keypath was very specific, but verbose especially when composing multiple key paths together.

The proposal is elegant but I don't yet see the use cases this feature would allow. Could you give us examples of how the Standard Library or a user's code could benefit from it? I mean, KVC is really powerful in Objective-C it comes hand in hand with KVO. Do you envision similar observation libraries be built on top of key paths?

···

On 17 Mar 2017, at 18:04, Michael LeHew via swift-evolution <swift-evolution@swift.org> wrote:

_______________________________________________

swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[VladimirS (Vladimir)]

Just my 2 cents. FWIW, I feel like the proposed syntax is VERY confusing.

Person.friends[0].name - like access to static property. T.method can't have subscript for it and then some field. So, I believe such key paths deserve its own syntax.

luke[path] - very close to access the subscript of instance. In compare to T.method, we can call result of T.method(i.e. let m = T.method; m()), just like the T.method itself(foo.method()). But here we are calling some kind of subscript instead of access property by "path".

There is no some 'special' marker for me that saying "hey, there is references to properties used" from the first look.

Probably we can consider another separators, like

Person:friends[0].name
luke:path

Person->friends[0].name
luke->path

Person[friends[0].name] // single brackets for type
luke[[path]] // double brackets for instance

Person@friends[0].name
luke@path

or other with help of community.

···

On 17.03.2017 20:04, Michael LeHew via swift-evolution wrote:

Hi friendly swift-evolution folks,

The Foundation and Swift team would like for you to consider the following
proposal:

Many thanks,
-Michael

  Smart KeyPaths: Better Key-Value Coding for Swift
...

[Martijn_Walraven]

Smart KeyPaths: Better Key-Value Coding for Swift
Proposal: SE-NNNN
Authors: David Smith <https://github.com/Catfish-Man&gt;, Michael LeHew <https://github.com/mlehew&gt;, Joe Groff <https://github.com/jckarter&gt;
Review Manager: TBD
Status: Awaiting Review
Associated PRs:
#644 <https://github.com/apple/swift-evolution/pull/644&gt;
Introduction
We propose a family of concrete Key Path types that represent uninvoked references to properties that can be composed to form paths through many values and directly get/set their underlying values

Really happy to see this proposal! What isn’t clear to me is if/how this design could also be used for dynamic paths through collections. A concrete use case of this would be getting/setting values in a JSON response consisting of nested dictionaries and arrays.

I’ve recently started using a design similar to that described by Ole Begemann in this blog post: Accessing Dictionaries with Key Paths – Ole Begemann

That allows me to use strings as key paths to navigate a nested structure:
data[keyPath: “person.bestFriend.name"] = “Han Solo”

It uses labeled subscripts to get back values of a known type, which is useful if you want to modify an array for example:
data[arrayAt: “person.friends"]?.append(["name": “Luke Skywalker”])

Generic subscripts might make this more elegant, although it seems you’d still need a way to provide enough context to let the type be inferred, and I’m not sure where that information would come from with these dynamic structures.

Curious to hear if this proposal would help with use cases like this!

Martijn

[Jon_Hull]

This looks fantastic!

The one worry I would have, as others have brought up, is confusion when there are static properties with the same name. Maybe we could have a special static property called ‘keyPath’ or ‘path’ that would be reserved, and anything that followed would be a key path.

So instead of:

  let myPath = Person.friends[0].name

you would have:

  let myPath = Person.keyPath.friends[0].name

Or if we want to choose a sentinel, I would nominate ‘$’:

  let myPath = $Person.friends[0].name

Thanks,
Jon

···

On Mar 17, 2017, at 10:04 AM, Michael LeHew via swift-evolution <swift-evolution@swift.org> wrote:

Hi friendly swift-evolution folks,

The Foundation and Swift team would like for you to consider the following proposal:

Many thanks,
-Michael

Smart KeyPaths: Better Key-Value Coding for Swift
Proposal: SE-NNNN
Authors: David Smith <https://github.com/Catfish-Man&gt;, Michael LeHew <https://github.com/mlehew&gt;, Joe Groff <https://github.com/jckarter&gt;
Review Manager: TBD
Status: Awaiting Review
Associated PRs:
#644 <https://github.com/apple/swift-evolution/pull/644&gt;
Introduction
We propose a family of concrete Key Path types that represent uninvoked references to properties that can be composed to form paths through many values and directly get/set their underlying values.

Motivation
We Can Do Better than String

On Darwin platforms Swift's existing keypath() syntax provides a convenient way to safely refer to properties. Unfortunately, once validated, the expression becomes a String which has a number of important limitations:

Loss of type information (requiring awkward Any APIs)
Unnecessarily slow to parse
Only applicable to NSObjects
Limited to Darwin platforms
Use/Mention Distinctions

While methods can be referred to without invoking them (let x = foo.bar instead of let x = foo.bar()), this is not currently possible for properties and subscripts.

Making indirect references to a properties' concrete types also lets us expose metadata about the property, and in the future additional behaviors.

More Expressive KeyPaths

We would also like to support being able to use Key Paths to access into collections, which is not currently possible.

Proposed solution
We propose introducing a new expression akin to Type.method, but for properties and subscripts. These property reference expressions produce KeyPath objects, rather than Strings. KeyPaths are a family of generic classes (structs and protocols here would be ideal, but requires generalized existentials) which encapsulate a property reference or chain of property references, including the type, mutability, property name(s), and ability to set/get values.

Here's a sample of it in use:

Swift
struct Person {
    var name: String
    var friends: [Person]
    var bestFriend: Person?
}

var han = Person(name: "Han Solo", friends: )
var luke = Person(name: "Luke Skywalker", friends: [han])

let firstFriendsNameKeyPath = Person.friends[0].name

let firstFriend = luke[path] // han

// or equivalently, with type inferred from context
let firstFriendName = luke[.friends[0].name]

// rename Luke's first friend
luke[firstFriendsNameKeyPath] = "A Disreputable Smuggler"

let bestFriendsName = luke[.bestFriend]?.name // nil, if he is the last jedi
Detailed design
Core KeyPath Types

KeyPaths are a hierarchy of progressively more specific classes, based on whether we have prior knowledge of the path through the object graph we wish to traverse.

Unknown Path / Unknown Root Type

AnyKeyPath is fully type-erased, referring to 'any route' through an object/value graph for 'any root'. Because of type-erasure many operations can fail at runtime and are thus nillable.

Swift
class AnyKeyPath: CustomDebugStringConvertible, Hashable {
    // MARK - Composition
    // Returns nil if path.rootType != self.valueType
    func appending(path: AnyKeyPath) -> AnyKeyPath?
    
    // MARK - Runtime Information
    class var rootType: Any.Type
    class var valueType: Any.Type
    
    static func == (lhs: AnyKeyPath, rhs: AnyKeyPath) -> Bool
    var hashValue: Int
}
Unknown Path / Known Root Type

If we know a little more type information (what kind of thing the key path is relative to), then we can use PartialKeyPath <Root>, which refers to an 'any route' from a known root:

Swift
class PartialKeyPath<Root>: AnyKeyPath {
    // MARK - Composition
    // Returns nil if Value != self.valueType
    func appending(path: AnyKeyPath) -> PartialKeyPath<Root>?
    func appending<Value, AppendedValue>(path: KeyPath<Value, AppendedValue>) -> KeyPath<Root, AppendedValue>?
    func appending<Value, AppendedValue>(path: ReferenceKeyPath<Value, AppendedValue>) -> ReferenceKeyPath<Root, AppendedValue>?
}
Known Path / Known Root Type

When we know both what the path is relative to and what it refers to, we can use KeyPath. Thanks to the knowledge of the Root and Value types, all of the failable operations lose their Optional.

Swift
public class KeyPath<Root, Value>: PartialKeyPath<Root> {
    // MARK - Composition
    func appending<AppendedValue>(path: KeyPath<Value, AppendedValue>) -> KeyPath<Root, AppendedValue>
    func appending<AppendedValue>(path: WritableKeyPath<Value, AppendedValue>) -> Self
    func appending<AppendedValue>(path: ReferenceWritableKeyPath<Value, AppendedValue>) -> ReferenceWritableKeyPath<Root, AppendedValue>
}
Value/Reference Mutation Semantics Mutation

Finally, we have a pair of subclasses encapsulating value/reference mutation semantics. These have to be distinct because mutating a copy of a value is not very useful, so we need to mutate an inout value.

Swift
class WritableKeyPath<Root, Value>: KeyPath<Root, Value> {
    // MARK - Composition
    func appending<AppendedPathValue>(path: WritableKeyPath<Value, AppendedPathValue>) -> WritableKeyPath<Root, AppendedPathValue>
}

class ReferenceWritableKeyPath<Root, Value>: WritableKeyPath<Root, Value> {
    override func appending<AppendedPathValue>(path: WritableKeyPath<Value, AppendedPathValue>) -> ReferenceWritableKeyPath<Root, AppendedPathValue>
}
Access and Mutation Through KeyPaths

To get or set values for a given root and key path we effectively add the following subscripts to all Swift types.

Swift
extension Any {
    subscript(path: AnyKeyPath) -> Any? { get }
    subscript<Root: Self>(path: PartialKeyPath<Root>) -> Any { get }
    subscript<Root: Self, Value>(path: KeyPath<Root, Value>) -> Value { get }
    subscript<Root: Self, Value>(path: WritableKeyPath<Root, Value>) -> Value { set, get }
}
This allows for code like

Swift
person[.name] // Self.type is inferred
which is both appealingly readable, and doesn't require read-modify-write copies (subscripts access self inout). Conflicts with existing subscripts are avoided by using generic subscripts to specifically only accept key paths with a Root of the type in question.

Referencing Key Paths

Forming a KeyPath borrows from the same syntax used to reference methods and initializers,Type.instanceMethod only now working for properties and collections. Optionals are handled via optional-chaining. Multiply dotted expressions are allowed as well, and work just as if they were composed via the appending methods on KeyPath.

There is no change or interaction with the keypath() syntax introduced in Swift 3.

Performance

The performance of interacting with a property via KeyPaths should be close to the cost of calling the property directly.

Source compatibility
This change is additive and there should no affect on existing source.

Effect on ABI stability
This feature adds the following requirements to ABI stability:

mechanism to access key paths of public properties
We think a protocol-based design would be preferable once the language has sufficient support for generalized existentials to make that ergonomic. By keeping the class hierarchy closed and the concrete implementations private to the implementation it should be tractable to provide compatibility with an open protocol-based design in the future.

Effect on API resilience
This should not significantly impact API resilience, as it merely provides a new mechanism for operating on existing APIs.

Alternatives considered
More Features

Various drafts of this proposal have included additional features (decomposable key paths, prefix comparisons, support for custom KeyPath subclasses, creating a KeyPath from a String at runtime, KeyPaths conforming to Codable, bound key paths as a concrete type, etc.). We anticipate approaching these enhancements additively once the core KeyPath functionality is in place.

Spelling

We also explored many different spellings, each with different strengths. We have chosen the current syntax due to the balance with existing function type references.

Current keypath Lisp-style
Person.friends[0].name keypath(Person, .friends[0].name) `Person.friend.name
luke[.friends[0].name] keypath(luke, .friends[0].name) luke`.friends[0].name
luke.friends[0][.name] keypath(luke.friends[0], .name) luke.friends[0]`.name
While the crispness is very appealing, the spelling of the 'escape' character was hard to agree upon (along with the fact that it requires parentheses to reduce ambiguity). keypath was very specific, but verbose especially when composing multiple key paths together.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[Colin_Barrett]

It's so nice that this is finally seeing the light of day :) Great work
everyone!

Re: subscripts, it's definitely a great solution for "the Swift we have
now", but I'm not sure in "the Swift we'll have in a few years." If, for
instance, someday we're able to return inouts (or really just lvalues in
general), we'd be able to do a lot of this with regular functions and some
combinators.

Count me as a +1 for the Lisp-style syntax. If ` isn't to people's liking,
I wonder about @. I think both "luke @ .friends[0].name" and "
.friends[0].name @ luke" read quiet nicely, and gives the idea of a key
path as a kind of addressing scheme.

-Colin

P.S. Has anyone thought about writing an HTTP + JSON DSL with these?

···

On Fri, Mar 17, 2017 at 1:04 PM Michael LeHew via swift-evolution < swift-evolution@swift.org> wrote:

Hi friendly swift-evolution folks,

The Foundation and Swift team would like for you to consider the
following proposal:

Many thanks,
-Michael

Smart KeyPaths: Better Key-Value Coding for Swift

   - Proposal: SE-NNNN
   - Authors: David Smith <https://github.com/Catfish-Man&gt;, Michael LeHew
   <https://github.com/mlehew&gt;, Joe Groff <https://github.com/jckarter&gt;
   - Review Manager: TBD
   - Status: *Awaiting Review*
   - Associated PRs:
      - #644 <https://github.com/apple/swift-evolution/pull/644&gt;

Introduction

We propose a family of concrete *Key Path* types that represent uninvoked
references to properties that can be composed to form paths through many
values and directly get/set their underlying values.
MotivationWe Can Do Better than String

On Darwin platforms Swift's existing keypath() syntax provides a
convenient way to safely *refer* to properties. Unfortunately, once
validated, the expression becomes a String which has a number of
important limitations:

   - Loss of type information (requiring awkward Any APIs)
   - Unnecessarily slow to parse
   - Only applicable to NSObjects
   - Limited to Darwin platforms

Use/Mention Distinctions

While methods can be referred to without invoking them (let x = foo.bar instead
of let x = foo.bar()), this is not currently possible for properties and
subscripts.

Making indirect references to a properties' concrete types also lets us
expose metadata about the property, and in the future additional behaviors.
More Expressive KeyPaths

We would also like to support being able to use *Key Paths* to access
into collections, which is not currently possible.
Proposed solution

We propose introducing a new expression akin to Type.method, but for
properties and subscripts. These property reference expressions produce
KeyPath objects, rather than Strings. KeyPaths are a family of generic
classes *(structs and protocols here would be ideal, but requires
generalized existentials)* which encapsulate a property reference or
chain of property references, including the type, mutability, property
name(s), and ability to set/get values.

Here's a sample of it in use:
Swift

struct Person {
    var name: String
    var friends: [Person]
    var bestFriend: Person?}
var han = Person(name: "Han Solo", friends: )var luke = Person(name: "Luke Skywalker", friends: [han])
let firstFriendsNameKeyPath = Person.friends[0].name
let firstFriend = luke[path] // han
// or equivalently, with type inferred from contextlet firstFriendName = luke[.friends[0].name]
// rename Luke's first friend
luke[firstFriendsNameKeyPath] = "A Disreputable Smuggler"
let bestFriendsName = luke[.bestFriend]?.name // nil, if he is the last jedi

Detailed designCore KeyPath Types

KeyPaths are a hierarchy of progressively more specific classes, based on
whether we have prior knowledge of the path through the object graph we
wish to traverse.
Unknown Path / Unknown Root Type

AnyKeyPath is fully type-erased, referring to 'any route' through an
object/value graph for 'any root'. Because of type-erasure many operations
can fail at runtime and are thus nillable.
Swift

class AnyKeyPath: CustomDebugStringConvertible, Hashable {
    // MARK - Composition
    // Returns nil if path.rootType != self.valueType
    func appending(path: AnyKeyPath) -> AnyKeyPath?

    // MARK - Runtime Information
    class var rootType: Any.Type
    class var valueType: Any.Type

    static func == (lhs: AnyKeyPath, rhs: AnyKeyPath) -> Bool
    var hashValue: Int}

Unknown Path / Known Root Type

If we know a little more type information (what kind of thing the key path
is relative to), then we can use PartialKeyPath <Root>, which refers to
an 'any route' from a known root:
Swift

class PartialKeyPath<Root>: AnyKeyPath {
    // MARK - Composition
    // Returns nil if Value != self.valueType
    func appending(path: AnyKeyPath) -> PartialKeyPath<Root>?
    func appending<Value, AppendedValue>(path: KeyPath<Value, AppendedValue>) -> KeyPath<Root, AppendedValue>?
    func appending<Value, AppendedValue>(path: ReferenceKeyPath<Value, AppendedValue>) -> ReferenceKeyPath<Root, AppendedValue>?}

Known Path / Known Root Type

When we know both what the path is relative to and what it refers to, we
can use KeyPath. Thanks to the knowledge of the Root and Value types, all
of the failable operations lose their Optional.
Swift

public class KeyPath<Root, Value>: PartialKeyPath<Root> {
    // MARK - Composition
    func appending<AppendedValue>(path: KeyPath<Value, AppendedValue>) -> KeyPath<Root, AppendedValue>
    func appending<AppendedValue>(path: WritableKeyPath<Value, AppendedValue>) -> Self
    func appending<AppendedValue>(path: ReferenceWritableKeyPath<Value, AppendedValue>) -> ReferenceWritableKeyPath<Root, AppendedValue>}

Value/Reference Mutation Semantics Mutation

Finally, we have a pair of subclasses encapsulating value/reference
mutation semantics. These have to be distinct because mutating a copy of a
value is not very useful, so we need to mutate an inout value.
Swift

class WritableKeyPath<Root, Value>: KeyPath<Root, Value> {
    // MARK - Composition
    func appending<AppendedPathValue>(path: WritableKeyPath<Value, AppendedPathValue>) -> WritableKeyPath<Root, AppendedPathValue>}
class ReferenceWritableKeyPath<Root, Value>: WritableKeyPath<Root, Value> {
    override func appending<AppendedPathValue>(path: WritableKeyPath<Value, AppendedPathValue>) -> ReferenceWritableKeyPath<Root, AppendedPathValue>}

Access and Mutation Through KeyPaths

To get or set values for a given root and key path we effectively add the
following subscripts to all Swift types.
Swift

extension Any {
    subscript(path: AnyKeyPath) -> Any? { get }
    subscript<Root: Self>(path: PartialKeyPath<Root>) -> Any { get }
    subscript<Root: Self, Value>(path: KeyPath<Root, Value>) -> Value { get }
    subscript<Root: Self, Value>(path: WritableKeyPath<Root, Value>) -> Value { set, get }}

This allows for code like
Swift

person[.name] // Self.type is inferred

which is both appealingly readable, and doesn't require read-modify-write
copies (subscripts access self inout). Conflicts with existing subscripts
are avoided by using generic subscripts to specifically only accept key
paths with a Root of the type in question.
Referencing Key Paths

Forming a KeyPath borrows from the same syntax used to reference methods
and initializers,Type.instanceMethod only now working for properties and
collections. Optionals are handled via optional-chaining. Multiply dotted
expressions are allowed as well, and work just as if they were composed via
the appending methods on KeyPath.

There is no change or interaction with the keypath() syntax introduced in
Swift 3.
Performance

The performance of interacting with a property via KeyPaths should be
close to the cost of calling the property directly.
Source compatibility

This change is additive and there should no affect on existing source.
Effect on ABI stability

This feature adds the following requirements to ABI stability:

   - mechanism to access key paths of public properties

We think a protocol-based design would be preferable once the language has
sufficient support for generalized existentials to make that ergonomic. By
keeping the class hierarchy closed and the concrete implementations private
to the implementation it should be tractable to provide compatibility with
an open protocol-based design in the future.
Effect on API resilience

This should not significantly impact API resilience, as it merely provides
a new mechanism for operating on existing APIs.
Alternatives consideredMore Features

Various drafts of this proposal have included additional features
(decomposable key paths, prefix comparisons, support for custom KeyPath subclasses,
creating a KeyPath from a String at runtime, KeyPaths conforming to
Codable, bound key paths as a concrete type, etc.). We anticipate
approaching these enhancements additively once the core

[rparada (Ricardo Parada)]

It looks awesome.

I don’t understand the details yet but I always felt it would be super cool if swift allowed you to express key paths elegantly which seems what this is.

I would love to be able to create what used to be called EOQualifier in WebObjects/Enterprise Objects Framework (i.e. NSPredicate in CoreData I think) using a better syntax. Same for EOSortOrdering, equivalent to NSSortDescriptor. For example:

let predicate = Person.lastName.like(“Para*").and(Person.birthDate.greaterThan(aDate))

Fetch objects from the database into a managed object context like this:

let people = context.fetch(Person.self, predicate: predicate) // people is inferred as Array<Person>

Or perhaps filter elements from an array like this:

let matches = people.filtered(predicate) // matches is inferred as Array<Person>

Or create sort orderings like this:

let orderings = Person.age.desc().then(Person.lastName.asc().then(Person.lastName.asc())) // orderings is Array<SortOrdering>

And sort like this:

let sortedPeople = people.sorted(orderings) // sortedPeople is Array<Person>

And predicates for to-many relationships, for example, if we had a Family class and a Pet class and a pets to-many relationship Family <—>> Pet, then building a predicate like this would be cool:

// Families with at least one pet
let predicate = Family.pets.isNotEmpty()

// Families with no pets
let predicate = Family.pets.isEmpty()

// Families with a cat or dog
let predicate = Family.pets.hasAtLeastOneObjectSatisfying(Pet.type.equals(.dog))

// Families with a puppy
let predicate = Family.pets.hasAtLeastOneObjectSatisfying(Pet.type.equals(.dog).and(Pet.ageInMonths.lessThan(12))

In SQL these translate to an EXISTS qualifier. For example, the last predicate would translate to something like this in SQL:

SELECT t0.family_id, …
FROM family t0
WHERE
    EXISTS (
        select p.pet_id
        from pet p
        where p.family_id = t0.family_id
            and p.pet_type = ‘dog’
            and p.age_in_months < 12
    )

Would these Star KeyPaths enable this sort of expressiveness?

Or creating ad hoc queries like this:

// records is Array<Dictionary<String,Object>>
let records = Query()
    .select (Claim.provider, Claim.sumExpectedAmount) // Claim.sumExpectedAmount is a derived non-class property defined as "SUM(expectedAmount)"
    .from (Claim.entityName) // Claim.entityName is a static property for string “Claim”
    .where (Claim.userGroup.equals(aUserGroup))
    .groupBy (Claim.provider)
    .having (Claim.sumExpectedAmount.greaterThan(1000.0))
    .orderBy (Claim.sumExpectedAmount.asc())
    .fetch (editingContext)
    
// Or to fetch into a custom class

// objects is Array<Foo>
let objects = Query()
    .select (Claim.provider, Claim.sumExpectedAmount) // Claim.sumExpectedAmount is a derived non-class property defined as "SUM(expectedAmount)"
    .from (Claim.entityName) // Claim.entityName is a static property for string “Claim”
    .where (Claim.userGroup.equals(aUserGroup))
    .groupBy (Claim.provider)
    .having (Claim.sumExpectedAmount.greaterThan(1000.0))
    .orderBy (Claim.sumExpectedAmount.asc())
    .fetch (context) {
        Foo(context: ec, data: row)
    }

Then print the results like this:

for obj in objects {
    Print(“Health provider: \(obj.provider?.fullName), Total Expected: \(obj.sumExpectedAmount)”)
}

Where Foo would be a custom class to hold the results and provide some type checking on the getters for the data fetched:

class Foo {
    var data : [String: Any]

    init(context: EditingContext, data row: [String: Any]) {
       data = row
    }

    var provider : Provider? {
        get {
           return data[“provider”] as? Provider
        }
    }

    var provider : Double? {
        get {
           return data[“sumExpectedAmount”] as? Double
        }
    }
}

By the way I do this kind of stuff with WebObjects and Project Wonder. It’s just that the key paths are static variables defined in the class in ALL CAPS, i.e. Person.FIRST_NAME.asc() to get an EOSortOrdering for ascending firstName. They also look not as elegant for key paths, i.e. Claim.PROVIDER.dot(Provider.LAST_NAME).like(“Para*”) for an EOQualifier. This is all provided by a class named ERXKey.

And for ad hoc queries I use ERXQuery which I recently created a pull request to add to project Wonder. It’s in my repository rparada/wonder on GitHub in the erxquery branch.

···

On Mar 17, 2017, at 1:04 PM, Michael LeHew via swift-evolution <swift-evolution@swift.org> wrote:

Hi friendly swift-evolution folks,

The Foundation and Swift team would like for you to consider the following proposal:

Many thanks,
-Michael

Smart KeyPaths: Better Key-Value Coding for Swift
Proposal: SE-NNNN
Authors: David Smith <https://github.com/Catfish-Man&gt;, Michael LeHew <https://github.com/mlehew&gt;, Joe Groff <https://github.com/jckarter&gt;
Review Manager: TBD
Status: Awaiting Review
Associated PRs:
#644 <https://github.com/apple/swift-evolution/pull/644&gt;
Introduction
We propose a family of concrete Key Path types that represent uninvoked references to properties that can be composed to form paths through many values and directly get/set their underlying values.

Motivation
We Can Do Better than String

On Darwin platforms Swift's existing keypath() syntax provides a convenient way to safely refer to properties. Unfortunately, once validated, the expression becomes a String which has a number of important limitations:

Loss of type information (requiring awkward Any APIs)
Unnecessarily slow to parse
Only applicable to NSObjects
Limited to Darwin platforms
Use/Mention Distinctions

While methods can be referred to without invoking them (let x = foo.bar instead of let x = foo.bar()), this is not currently possible for properties and subscripts.

Making indirect references to a properties' concrete types also lets us expose metadata about the property, and in the future additional behaviors.

More Expressive KeyPaths

We would also like to support being able to use Key Paths to access into collections, which is not currently possible.

Proposed solution
We propose introducing a new expression akin to Type.method, but for properties and subscripts. These property reference expressions produce KeyPath objects, rather than Strings. KeyPaths are a family of generic classes (structs and protocols here would be ideal, but requires generalized existentials) which encapsulate a property reference or chain of property references, including the type, mutability, property name(s), and ability to set/get values.

Here's a sample of it in use:

Swift
struct Person {
    var name: String
    var friends: [Person]
    var bestFriend: Person?
}

var han = Person(name: "Han Solo", friends: )
var luke = Person(name: "Luke Skywalker", friends: [han])

let firstFriendsNameKeyPath = Person.friends[0].name

let firstFriend = luke[path] // han

// or equivalently, with type inferred from context
let firstFriendName = luke[.friends[0].name]

// rename Luke's first friend
luke[firstFriendsNameKeyPath] = "A Disreputable Smuggler"

let bestFriendsName = luke[.bestFriend]?.name // nil, if he is the last jedi
Detailed design
Core KeyPath Types

KeyPaths are a hierarchy of progressively more specific classes, based on whether we have prior knowledge of the path through the object graph we wish to traverse.

Unknown Path / Unknown Root Type

AnyKeyPath is fully type-erased, referring to 'any route' through an object/value graph for 'any root'. Because of type-erasure many operations can fail at runtime and are thus nillable.

Swift
class AnyKeyPath: CustomDebugStringConvertible, Hashable {
    // MARK - Composition
    // Returns nil if path.rootType != self.valueType
    func appending(path: AnyKeyPath) -> AnyKeyPath?
    
    // MARK - Runtime Information
    class var rootType: Any.Type
    class var valueType: Any.Type
    
    static func == (lhs: AnyKeyPath, rhs: AnyKeyPath) -> Bool
    var hashValue: Int
}
Unknown Path / Known Root Type

If we know a little more type information (what kind of thing the key path is relative to), then we can use PartialKeyPath <Root>, which refers to an 'any route' from a known root:

Swift
class PartialKeyPath<Root>: AnyKeyPath {
    // MARK - Composition
    // Returns nil if Value != self.valueType
    func appending(path: AnyKeyPath) -> PartialKeyPath<Root>?
    func appending<Value, AppendedValue>(path: KeyPath<Value, AppendedValue>) -> KeyPath<Root, AppendedValue>?
    func appending<Value, AppendedValue>(path: ReferenceKeyPath<Value, AppendedValue>) -> ReferenceKeyPath<Root, AppendedValue>?
}
Known Path / Known Root Type

When we know both what the path is relative to and what it refers to, we can use KeyPath. Thanks to the knowledge of the Root and Value types, all of the failable operations lose their Optional.

Swift
public class KeyPath<Root, Value>: PartialKeyPath<Root> {
    // MARK - Composition
    func appending<AppendedValue>(path: KeyPath<Value, AppendedValue>) -> KeyPath<Root, AppendedValue>
    func appending<AppendedValue>(path: WritableKeyPath<Value, AppendedValue>) -> Self
    func appending<AppendedValue>(path: ReferenceWritableKeyPath<Value, AppendedValue>) -> ReferenceWritableKeyPath<Root, AppendedValue>
}
Value/Reference Mutation Semantics Mutation

Finally, we have a pair of subclasses encapsulating value/reference mutation semantics. These have to be distinct because mutating a copy of a value is not very useful, so we need to mutate an inout value.

Swift
class WritableKeyPath<Root, Value>: KeyPath<Root, Value> {
    // MARK - Composition
    func appending<AppendedPathValue>(path: WritableKeyPath<Value, AppendedPathValue>) -> WritableKeyPath<Root, AppendedPathValue>
}

class ReferenceWritableKeyPath<Root, Value>: WritableKeyPath<Root, Value> {
    override func appending<AppendedPathValue>(path: WritableKeyPath<Value, AppendedPathValue>) -> ReferenceWritableKeyPath<Root, AppendedPathValue>
}
Access and Mutation Through KeyPaths

To get or set values for a given root and key path we effectively add the following subscripts to all Swift types.

Swift
extension Any {
    subscript(path: AnyKeyPath) -> Any? { get }
    subscript<Root: Self>(path: PartialKeyPath<Root>) -> Any { get }
    subscript<Root: Self, Value>(path: KeyPath<Root, Value>) -> Value { get }
    subscript<Root: Self, Value>(path: WritableKeyPath<Root, Value>) -> Value { set, get }
}
This allows for code like

Swift
person[.name] // Self.type is inferred
which is both appealingly readable, and doesn't require read-modify-write copies (subscripts access self inout). Conflicts with existing subscripts are avoided by using generic subscripts to specifically only accept key paths with a Root of the type in question.

Referencing Key Paths

Forming a KeyPath borrows from the same syntax used to reference methods and initializers,Type.instanceMethod only now working for properties and collections. Optionals are handled via optional-chaining. Multiply dotted expressions are allowed as well, and work just as if they were composed via the appending methods on KeyPath.

There is no change or interaction with the keypath() syntax introduced in Swift 3.

Performance

The performance of interacting with a property via KeyPaths should be close to the cost of calling the property directly.

Source compatibility
This change is additive and there should no affect on existing source.

Effect on ABI stability
This feature adds the following requirements to ABI stability:

mechanism to access key paths of public properties
We think a protocol-based design would be preferable once the language has sufficient support for generalized existentials to make that ergonomic. By keeping the class hierarchy closed and the concrete implementations private to the implementation it should be tractable to provide compatibility with an open protocol-based design in the future.

Effect on API resilience
This should not significantly impact API resilience, as it merely provides a new mechanism for operating on existing APIs.

Alternatives considered
More Features

Various drafts of this proposal have included additional features (decomposable key paths, prefix comparisons, support for custom KeyPath subclasses, creating a KeyPath from a String at runtime, KeyPaths conforming to Codable, bound key paths as a concrete type, etc.). We anticipate approaching these enhancements additively once the core KeyPath functionality is in place.

Spelling

We also explored many different spellings, each with different strengths. We have chosen the current syntax due to the balance with existing function type references.

Current keypath Lisp-style
Person.friends[0].name keypath(Person, .friends[0].name) `Person.friend.name
luke[.friends[0].name] keypath(luke, .friends[0].name) luke`.friends[0].name
luke.friends[0][.name] keypath(luke.friends[0], .name) luke.friends[0]`.name
While the crispness is very appealing, the spelling of the 'escape' character was hard to agree upon (along with the fact that it requires parentheses to reduce ambiguity). keypath was very specific, but verbose especially when composing multiple key paths together.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[hexdreamer (Kenny Leung)]

Hi All.

I’m not sure I’m understanding this proposal properly. In (old) Cocoa, two places where key paths were used extensively was EOF/CoreData, and WebObjects. I’m wondering how Smart KeyPaths will solve these two problems:

1. fetching data from a database and stuff it into objects that are not known at compile time (since you’ve written the framework ahead of time)

2. Token replacing text in a template, like ${person.firstName}

Will there be some conversion of key paths to/from strings?

Thanks!

-Kenny

[Douglas_Gregor]

Hi friendly swift-evolution folks,

The Foundation and Swift team would like for you to consider the following proposal:

The Swift core team discussed this proposal draft and had a little bit of pre-review feedback.

Access and Mutation Through KeyPaths
To get or set values for a given root and key path we effectively add the following subscripts to all Swift types.

Swift
extension Any {
    subscript(path: AnyKeyPath) -> Any? { get }
    subscript<Root: Self>(path: PartialKeyPath<Root>) -> Any { get }
    subscript<Root: Self, Value>(path: KeyPath<Root, Value>) -> Value { get }
    subscript<Root: Self, Value>(path: WritableKeyPath<Root, Value>) -> Value { set, get }
}

Swift doesn’t currently have the ability to extend Any, so this is (currently) pseudocode for compiler magic that one day we might be able to place. Additionally, the “Root: Self” constraint isn’t something we support in the generics system. A small note indicating that this is pseudo-code meant to get the point across (rather than real code to drop into the standard library/Foundation) would be appreciated.

More importantly, this adds an unlabeled subscript to every type, which raises concerns about introducing ambiguities—even if not hard ambiguities that prevent code from compiling (e.g., from a Dictionary<AnyKeyPath, …>)---they can still show up in code completion, diagnostics, etc.

The core team would prefer that this subscript distinguish itself more, e.g., by labeling the first parameter “keyPath” (or some better name, if there is one). Syntactically, that would look like:

  person[keyPath: theKeyPathIHave]

Referencing Key Paths

Forming a KeyPath borrows from the same syntax used to reference methods and initializers,Type.instanceMethod only now working for properties and collections. Optionals are handled via optional-chaining. Multiply dotted expressions are allowed as well, and work just as if they were composed via the appending methods on KeyPath.

The core team was concerned about the use of the Type.instanceProperty syntax for a few reasons:

  * It doesn’t work for forming keypaths to class/static properties (or is ambiguous with the existing meaning(, so we would need another syntax to deal with that case
  * It’s quite subtle, even more so that the existing Type.instanceMethod syntax for currying instance methods

There is no change or interaction with the keypath() syntax introduced in Swift 3.

The core team felt that extending the keypath syntax was a better syntactic direction to produce key-paths.

  - Doug

···

On Mar 17, 2017, at 10:04 AM, Michael LeHew via swift-evolution <swift-evolution@swift.org> wrote:

[David_Smith]

Sent off-list by mistake:

Nice proposal. I have a few comments inline:

Hi friendly swift-evolution folks,

The Foundation and Swift team would like for you to consider the following proposal:

Many thanks,
-Michael

Smart KeyPaths: Better Key-Value Coding for Swift
Proposal: SE-NNNN
Authors: David Smith <https://github.com/Catfish-Man&gt;, Michael LeHew <https://github.com/mlehew&gt;, Joe Groff <https://github.com/jckarter&gt;
Review Manager: TBD
Status: Awaiting Review
Associated PRs:
#644 <https://github.com/apple/swift-evolution/pull/644&gt;
Introduction
We propose a family of concrete Key Path types that represent uninvoked references to properties that can be composed to form paths through many values and directly get/set their underlying values.

Motivation
We Can Do Better than String

On Darwin platforms Swift's existing keypath() syntax provides a convenient way to safely refer to properties. Unfortunately, once validated, the expression becomes a String which has a number of important limitations:

Loss of type information (requiring awkward Any APIs)
Unnecessarily slow to parse
Only applicable to NSObjects
Limited to Darwin platforms
Use/Mention Distinctions

While methods can be referred to without invoking them (let x = foo.bar instead of let x = foo.bar()), this is not currently possible for properties and subscripts.

Making indirect references to a properties' concrete types also lets us expose metadata about the property, and in the future additional behaviors.

What metadata is attached? How is it accessed? What future features are you thinking about?

The most obvious bit of metadata is the CustomDebugStringConvertible conformance.

A shortish-term future direction would be adding a Codable conformance (which would be useful for Interface Builder style tools, as well as for IPC).

Longer term, both future directions for KVO and the "Property Behaviors" proposal from earlier could make use of it; for example if a memoizing property behavior required a "reset()" method, then extending key paths to memoizing properties to have the reset method would be a reasonable way of avoiding namespace conflicts.

More Expressive KeyPaths

We would also like to support being able to use Key Paths to access into collections, which is not currently possible.

Proposed solution
We propose introducing a new expression akin to Type.method, but for properties and subscripts. These property reference expressions produce KeyPath objects, rather than Strings. KeyPaths are a family of generic classes (structs and protocols here would be ideal, but requires generalized existentials)

How different would the design be with generalized existentials? Are they plans to migrate to that design once we do get generalized existentials?

Not all that different; mostly just that it would be structs + protocols, which would let us eliminate the type erasers and avoid some heap allocations. We've sketched out enough of a migration plan that we're pretty confident about being able to switch to this when the time comes.

which encapsulate a property reference or chain of property references, including the type, mutability, property name(s), and ability to set/get values.

Here's a sample of it in use:

Swift
struct Person {
    var name: String
    var friends: [Person]
    var bestFriend: Person?
}

var han = Person(name: "Han Solo", friends: )
var luke = Person(name: "Luke Skywalker", friends: [han])

let firstFriendsNameKeyPath = Person.friends[0].name

let firstFriend = luke[path] // han

// or equivalently, with type inferred from context
let firstFriendName = luke[.friends[0].name]

// rename Luke's first friend
luke[firstFriendsNameKeyPath] = "A Disreputable Smuggler"

let bestFriendsName = luke[.bestFriend]?.name // nil, if he is the last jedi
Detailed design
Core KeyPath Types

KeyPaths are a hierarchy of progressively more specific classes, based on whether we have prior knowledge of the path through the object graph we wish to traverse.

Unknown Path / Unknown Root Type

AnyKeyPath is fully type-erased, referring to 'any route' through an object/value graph for 'any root'. Because of type-erasure many operations can fail at runtime and are thus nillable.

Swift
class AnyKeyPath: CustomDebugStringConvertible, Hashable {
    // MARK - Composition
    // Returns nil if path.rootType != self.valueType
    func appending(path: AnyKeyPath) -> AnyKeyPath?
    
    // MARK - Runtime Information
    class var rootType: Any.Type
    class var valueType: Any.Type
    
    static func == (lhs: AnyKeyPath, rhs: AnyKeyPath) -> Bool
    var hashValue: Int
}
Unknown Path / Known Root Type

If we know a little more type information (what kind of thing the key path is relative to), then we can use PartialKeyPath <Root>, which refers to an 'any route' from a known root:

Swift
class PartialKeyPath<Root>: AnyKeyPath {
    // MARK - Composition
    // Returns nil if Value != self.valueType
    func appending(path: AnyKeyPath) -> PartialKeyPath<Root>?
    func appending<Value, AppendedValue>(path: KeyPath<Value, AppendedValue>) -> KeyPath<Root, AppendedValue>?
    func appending<Value, AppendedValue>(path: ReferenceKeyPath<Value, AppendedValue>) -> ReferenceKeyPath<Root, AppendedValue>?
}
Known Path / Known Root Type

When we know both what the path is relative to and what it refers to, we can use KeyPath. Thanks to the knowledge of the Root and Value types, all of the failable operations lose their Optional.

Swift
public class KeyPath<Root, Value>: PartialKeyPath<Root> {
    // MARK - Composition
    func appending<AppendedValue>(path: KeyPath<Value, AppendedValue>) -> KeyPath<Root, AppendedValue>
    func appending<AppendedValue>(path: WritableKeyPath<Value, AppendedValue>) -> Self
    func appending<AppendedValue>(path: ReferenceWritableKeyPath<Value, AppendedValue>) -> ReferenceWritableKeyPath<Root, AppendedValue>
}
Value/Reference Mutation Semantics Mutation

Finally, we have a pair of subclasses encapsulating value/reference mutation semantics. These have to be distinct because mutating a copy of a value is not very useful, so we need to mutate an inout value.

Swift
class WritableKeyPath<Root, Value>: KeyPath<Root, Value> {
    // MARK - Composition
    func appending<AppendedPathValue>(path: WritableKeyPath<Value, AppendedPathValue>) -> WritableKeyPath<Root, AppendedPathValue>
}

class ReferenceWritableKeyPath<Root, Value>: WritableKeyPath<Root, Value> {
    override func appending<AppendedPathValue>(path: WritableKeyPath<Value, AppendedPathValue>) -> ReferenceWritableKeyPath<Root, AppendedPathValue>
}
Access and Mutation Through KeyPaths

To get or set values for a given root and key path we effectively add the following subscripts to all Swift types.

Swift
extension Any {
    subscript(path: AnyKeyPath) -> Any? { get }
    subscript<Root: Self>(path: PartialKeyPath<Root>) -> Any { get }
    subscript<Root: Self, Value>(path: KeyPath<Root, Value>) -> Value { get }
    subscript<Root: Self, Value>(path: WritableKeyPath<Root, Value>) -> Value { set, get }
}
This allows for code like

Swift
person[.name] // Self.type is inferred

Perhaps I'm missing something, but what does that syntax bring compared to person.name? I see quite a few examples of the key paths being used literally in the subscript syntax but fail to see the usefulness of doing that. Can you give use cases?

Yeah, that example is contrived, since it's just trying to show that the base can be omitted contextually. A better example might be self.willChangeValueForKey(.someProperty).

which is both appealingly readable, and doesn't require read-modify-write copies (subscripts access self inout). Conflicts with existing subscripts are avoided by using generic subscripts to specifically only accept key paths with a Root of the type in question.

Referencing Key Paths

Forming a KeyPath borrows from the same syntax used to reference methods and initializers,Type.instanceMethod only now working for properties and collections. Optionals are handled via optional-chaining. Multiply dotted expressions are allowed as well, and work just as if they were composed via the appending methods on KeyPath.

There is no change or interaction with the keypath() syntax introduced in Swift 3.

Performance

The performance of interacting with a property via KeyPaths should be close to the cost of calling the property directly.

Source compatibility
This change is additive and there should no affect on existing source.

Effect on ABI stability
This feature adds the following requirements to ABI stability:

mechanism to access key paths of public properties
We think a protocol-based design would be preferable once the language has sufficient support for generalized existentials to make that ergonomic. By keeping the class hierarchy closed and the concrete implementations private to the implementation it should be tractable to provide compatibility with an open protocol-based design in the future.

Effect on API resilience
This should not significantly impact API resilience, as it merely provides a new mechanism for operating on existing APIs.

Alternatives considered
More Features

Various drafts of this proposal have included additional features (decomposable key paths, prefix comparisons, support for custom KeyPath subclasses, creating a KeyPath from a String at runtime, KeyPaths conforming to Codable, bound key paths as a concrete type, etc.). We anticipate approaching these enhancements additively once the core KeyPath functionality is in place.

Spelling

We also explored many different spellings, each with different strengths. We have chosen the current syntax due to the balance with existing function type references.

Current keypath Lisp-style
Person.friends[0].name keypath(Person, .friends[0].name) `Person.friend.name
luke[.friends[0].name] keypath(luke, .friends[0].name) luke`.friends[0].name
luke.friends[0][.name] keypath(luke.friends[0], .name) luke.friends[0]`.name
While the crispness is very appealing, the spelling of the 'escape' character was hard to agree upon (along with the fact that it requires parentheses to reduce ambiguity). keypath was very specific, but verbose especially when composing multiple key paths together.

The proposal is elegant but I don't yet see the use cases this feature would allow. Could you give us examples of how the Standard Library or a user's code could benefit from it? I mean, KVC is really powerful in Objective-C it comes hand in hand with KVO. Do you envision similar observation libraries be built on top of key paths?

KVC's use isn't limited to KVO, though KVO is likely the most frequent consumer of key paths. We're aware KVO is an important technology and have ideas building on this proposal in mind, but don't have anything to propose for Swift 4.

  David

···

On Mar 17, 2017, at 12:34 PM, David Hart via swift-evolution <swift-evolution@swift.org> wrote:

On 17 Mar 2017, at 18:04, Michael LeHew via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

_______________________________________________

swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

[David_Smith]

Introduction
We propose a family of concrete Key Path types that represent uninvoked references to properties that can be composed to form paths through many values and directly get/set their underlying values.

I don't know how to express the level of "Yes" I feel in response to this proposal without using language that's inappropriate on a public mailing list.

A few questions:

How do key paths interact with Optionals? Can you form a key path to `Person.bestFriend.name`, and is that the syntax, or is it `Person.bestFriend?.name`?

Yes and yes. You'll just end up with Value being an optional type.

Foundation key paths have a sometimes-useful property where, if they traverse a collection, the result becomes a collection of the ending type. Is a similar feature planned for smart key paths—perhaps something like `Person.friends.name`?

We've talked about stuff like this, but a number of the obvious approaches to it end up impacting our ability to make key paths Codable in the future. We can get many of the KVC collection operators with conditional extensions, e.g.

extension Collection where Element : Numeric /*or whatever it ended up being named, I'm behind on email */ {
    static let sum : Int
}

And then you could do Person.friends.sum where in ObjC you would have done @"friends.@sum", and so on.

Given that Swift has a syntax called keypath which is unrelated to these "key paths", have you considered using a different name to avoid confusion? Maybe "property paths" or "accessor paths"?

We're hoping we can make that syntax basically vestigial by providing overlays that take the new types. It never worked outside of Darwin platforms anyway.

  David

···

On Mar 17, 2017, at 1:58 PM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

On Mar 17, 2017, at 10:04 AM, Michael LeHew via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[Michael_J_LeHew_Jr]

Hi Vladimir,

Responses inline, thank you for feedback! :)

Hi friendly swift-evolution folks,

The Foundation and Swift team would like for you to consider the following
proposal:

Many thanks,
-Michael

Smart KeyPaths: Better Key-Value Coding for Swift
...

Just my 2 cents. FWIW, I feel like the proposed syntax is VERY confusing.

Person.friends[0].name - like access to static property. T.method can't have subscript for it and then some field. So, I believe such key paths deserve its own syntax.

For most cases that wouldn't be a typical collision (between instance property / static/class property). Joe, how were you thinking we'd differentiate between unexecuted static property and wanting the value of the key path?

luke[path] - very close to access the subscript of instance. In compare to T.method, we can call result of T.method(i.e. let m = T.method; m()), just like the T.method itself(foo.method()). But here we are calling some kind of subscript instead of access property by "path".

The subscript is serving us in a couple of ways. For one, self is always inout for subscripts, which has important performance impact for value types (materializeForSet, e.g. not having to read the value type through the key path only to write it back). It also allows us to have mutations share the same spelling for value and/reference properties. We had explored having get/set methods here:

  get(from: Root) -> Value
  set(inout into: Root, _ newValue: Value)
  set(into: Root, _ newValue: Value)

but then using a key path involved a reverse yoda-speak:

let lukesName = Person.name.get(luke)

There is no some 'special' marker for me that saying "hey, there is references to properties used" from the first look.

Probably we can consider another separators, like

Person:friends[0].name
luke:path

Person->friends[0].name
luke->path

Person[friends[0].name] // single brackets for type
luke[[path]] // double brackets for instance

Person@friends[0].name
luke@path

or other with help of community.

The one we worked with throughout discussions was the beloved ` from Lisp. `Person.friends[0].name Of course, ` has its own problems, and any other character would probably work. Upon talking with members of the Swift compiler team, they offered that we don't need an escape, and it makes for a smaller change to the language if we don't add such a thing, so we agreed.

I do agree there is some human benefit to noting you are 'escaping' the execution;

···

On Mar 17, 2017, at 11:27 AM, Vladimir.S <svabox@gmail.com> wrote:
On 17.03.2017 20:04, Michael LeHew via swift-evolution wrote:

[anandabits (Matthew Johnson)]

Sent off-list by mistake:

Nice proposal. I have a few comments inline:

Hi friendly swift-evolution folks,

The Foundation and Swift team would like for you to consider the following proposal:

Many thanks,
-Michael

Smart KeyPaths: Better Key-Value Coding for Swift
Proposal: SE-NNNN
Authors: David Smith <https://github.com/Catfish-Man&gt;, Michael LeHew <https://github.com/mlehew&gt;, Joe Groff <https://github.com/jckarter&gt;
Review Manager: TBD
Status: Awaiting Review
Associated PRs:
#644 <https://github.com/apple/swift-evolution/pull/644&gt;
Introduction
We propose a family of concrete Key Path types that represent uninvoked references to properties that can be composed to form paths through many values and directly get/set their underlying values.

Motivation
We Can Do Better than String

On Darwin platforms Swift's existing keypath() syntax provides a convenient way to safely refer to properties. Unfortunately, once validated, the expression becomes a String which has a number of important limitations:

Loss of type information (requiring awkward Any APIs)
Unnecessarily slow to parse
Only applicable to NSObjects
Limited to Darwin platforms
Use/Mention Distinctions

While methods can be referred to without invoking them (let x = foo.bar instead of let x = foo.bar()), this is not currently possible for properties and subscripts.

Making indirect references to a properties' concrete types also lets us expose metadata about the property, and in the future additional behaviors.

What metadata is attached? How is it accessed? What future features are you thinking about?

More Expressive KeyPaths

We would also like to support being able to use Key Paths to access into collections, which is not currently possible.

Proposed solution
We propose introducing a new expression akin to Type.method, but for properties and subscripts. These property reference expressions produce KeyPath objects, rather than Strings. KeyPaths are a family of generic classes (structs and protocols here would be ideal, but requires generalized existentials)

How different would the design be with generalized existentials? Are they plans to migrate to that design once we do get generalized existentials?

which encapsulate a property reference or chain of property references, including the type, mutability, property name(s), and ability to set/get values.

Here's a sample of it in use:

Swift
struct Person {
    var name: String
    var friends: [Person]
    var bestFriend: Person?
}

var han = Person(name: "Han Solo", friends: )
var luke = Person(name: "Luke Skywalker", friends: [han])

let firstFriendsNameKeyPath = Person.friends[0].name

let firstFriend = luke[path] // han

// or equivalently, with type inferred from context
let firstFriendName = luke[.friends[0].name]

// rename Luke's first friend
luke[firstFriendsNameKeyPath] = "A Disreputable Smuggler"

let bestFriendsName = luke[.bestFriend]?.name // nil, if he is the last jedi
Detailed design
Core KeyPath Types

KeyPaths are a hierarchy of progressively more specific classes, based on whether we have prior knowledge of the path through the object graph we wish to traverse.

Unknown Path / Unknown Root Type

AnyKeyPath is fully type-erased, referring to 'any route' through an object/value graph for 'any root'. Because of type-erasure many operations can fail at runtime and are thus nillable.

Swift
class AnyKeyPath: CustomDebugStringConvertible, Hashable {
    // MARK - Composition
    // Returns nil if path.rootType != self.valueType
    func appending(path: AnyKeyPath) -> AnyKeyPath?
    
    // MARK - Runtime Information
    class var rootType: Any.Type
    class var valueType: Any.Type
    
    static func == (lhs: AnyKeyPath, rhs: AnyKeyPath) -> Bool
    var hashValue: Int
}
Unknown Path / Known Root Type

If we know a little more type information (what kind of thing the key path is relative to), then we can use PartialKeyPath <Root>, which refers to an 'any route' from a known root:

Swift
class PartialKeyPath<Root>: AnyKeyPath {
    // MARK - Composition
    // Returns nil if Value != self.valueType
    func appending(path: AnyKeyPath) -> PartialKeyPath<Root>?
    func appending<Value, AppendedValue>(path: KeyPath<Value, AppendedValue>) -> KeyPath<Root, AppendedValue>?
    func appending<Value, AppendedValue>(path: ReferenceKeyPath<Value, AppendedValue>) -> ReferenceKeyPath<Root, AppendedValue>?
}
Known Path / Known Root Type

When we know both what the path is relative to and what it refers to, we can use KeyPath. Thanks to the knowledge of the Root and Value types, all of the failable operations lose their Optional.

Swift
public class KeyPath<Root, Value>: PartialKeyPath<Root> {
    // MARK - Composition
    func appending<AppendedValue>(path: KeyPath<Value, AppendedValue>) -> KeyPath<Root, AppendedValue>
    func appending<AppendedValue>(path: WritableKeyPath<Value, AppendedValue>) -> Self
    func appending<AppendedValue>(path: ReferenceWritableKeyPath<Value, AppendedValue>) -> ReferenceWritableKeyPath<Root, AppendedValue>
}
Value/Reference Mutation Semantics Mutation

Finally, we have a pair of subclasses encapsulating value/reference mutation semantics. These have to be distinct because mutating a copy of a value is not very useful, so we need to mutate an inout value.

Swift
class WritableKeyPath<Root, Value>: KeyPath<Root, Value> {
    // MARK - Composition
    func appending<AppendedPathValue>(path: WritableKeyPath<Value, AppendedPathValue>) -> WritableKeyPath<Root, AppendedPathValue>
}

class ReferenceWritableKeyPath<Root, Value>: WritableKeyPath<Root, Value> {
    override func appending<AppendedPathValue>(path: WritableKeyPath<Value, AppendedPathValue>) -> ReferenceWritableKeyPath<Root, AppendedPathValue>
}
Access and Mutation Through KeyPaths

To get or set values for a given root and key path we effectively add the following subscripts to all Swift types.

Swift
extension Any {
    subscript(path: AnyKeyPath) -> Any? { get }
    subscript<Root: Self>(path: PartialKeyPath<Root>) -> Any { get }
    subscript<Root: Self, Value>(path: KeyPath<Root, Value>) -> Value { get }
    subscript<Root: Self, Value>(path: WritableKeyPath<Root, Value>) -> Value { set, get }
}
This allows for code like

Swift
person[.name] // Self.type is inferred

Perhaps I'm missing something, but what does that syntax bring compared to person.name? I see quite a few examples of the key paths being used literally in the subscript syntax but fail to see the usefulness of doing that. Can you give use cases?

which is both appealingly readable, and doesn't require read-modify-write copies (subscripts access self inout). Conflicts with existing subscripts are avoided by using generic subscripts to specifically only accept key paths with a Root of the type in question.

Referencing Key Paths

Forming a KeyPath borrows from the same syntax used to reference methods and initializers,Type.instanceMethod only now working for properties and collections. Optionals are handled via optional-chaining. Multiply dotted expressions are allowed as well, and work just as if they were composed via the appending methods on KeyPath.

There is no change or interaction with the keypath() syntax introduced in Swift 3.

Performance

The performance of interacting with a property via KeyPaths should be close to the cost of calling the property directly.

Source compatibility
This change is additive and there should no affect on existing source.

Effect on ABI stability
This feature adds the following requirements to ABI stability:

mechanism to access key paths of public properties
We think a protocol-based design would be preferable once the language has sufficient support for generalized existentials to make that ergonomic. By keeping the class hierarchy closed and the concrete implementations private to the implementation it should be tractable to provide compatibility with an open protocol-based design in the future.

Effect on API resilience
This should not significantly impact API resilience, as it merely provides a new mechanism for operating on existing APIs.

Alternatives considered
More Features

Various drafts of this proposal have included additional features (decomposable key paths, prefix comparisons, support for custom KeyPath subclasses, creating a KeyPath from a String at runtime, KeyPaths conforming to Codable, bound key paths as a concrete type, etc.). We anticipate approaching these enhancements additively once the core KeyPath functionality is in place.

Spelling

We also explored many different spellings, each with different strengths. We have chosen the current syntax due to the balance with existing function type references.

Current keypath Lisp-style
Person.friends[0].name keypath(Person, .friends[0].name) `Person.friend.name
luke[.friends[0].name] keypath(luke, .friends[0].name) luke`.friends[0].name
luke.friends[0][.name] keypath(luke.friends[0], .name) luke.friends[0]`.name
While the crispness is very appealing, the spelling of the 'escape' character was hard to agree upon (along with the fact that it requires parentheses to reduce ambiguity). keypath was very specific, but verbose especially when composing multiple key paths together.

The proposal is elegant but I don't yet see the use cases this feature would allow. Could you give us examples of how the Standard Library or a user's code could benefit from it? I mean, KVC is really powerful in Objective-C it comes hand in hand with KVO. Do you envision similar observation libraries be built on top of key paths?

Key paths are basically lenses. There are many cool things you can do with lenses - I’m sure a few minutes on Google would find some interesting examples. Here’s a talk discussing using lenses in Swift to get you started: https://www.youtube.com/watch?v=ofjehH9f-CU\.

One example of a use case is to propagate values from an arbitrary property on one type to an arbitrary property on another type (both properties must be of the same type of course). Lenses allow you to write a library that does this. One example of a kind of library that needs to do this is a React / Elm style UI framework.

···

On Mar 17, 2017, at 2:34 PM, David Hart via swift-evolution <swift-evolution@swift.org> wrote:

On 17 Mar 2017, at 18:04, Michael LeHew via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

_______________________________________________

swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[Michael_J_LeHew_Jr]

This looks great!

What happens in the case when there is a static property by the same name as an instance property? e.g.

struct Foo {
    static var bar: Bar
    var bar: Bar
}

Foo.bar // What is this?

Is it still possible to reference both the static property and a KeyPath to the instance method?

This is essentially the same question that I arrived at in my reply to Vladimir. I think Joe might be best able to answer here.

-Michael

···

On Mar 17, 2017, at 2:21 PM, BJ Homer <bjhomer@gmail.com> wrote:

-BJ

On Mar 17, 2017, at 11:04 AM, Michael LeHew via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Hi friendly swift-evolution folks,

The Foundation and Swift team would like for you to consider the following proposal:

Many thanks,
-Michael

Smart KeyPaths: Better Key-Value Coding for Swift
Proposal: SE-NNNN
Authors: David Smith <https://github.com/Catfish-Man&gt;, Michael LeHew <https://github.com/mlehew&gt;, Joe Groff <https://github.com/jckarter&gt;
Review Manager: TBD
Status: Awaiting Review
Associated PRs:
#644 <https://github.com/apple/swift-evolution/pull/644&gt;
Introduction
We propose a family of concrete Key Path types that represent uninvoked references to properties that can be composed to form paths through many values and directly get/set their underlying values.

Motivation
We Can Do Better than String

On Darwin platforms Swift's existing keypath() syntax provides a convenient way to safely refer to properties. Unfortunately, once validated, the expression becomes a String which has a number of important limitations:

Loss of type information (requiring awkward Any APIs)
Unnecessarily slow to parse
Only applicable to NSObjects
Limited to Darwin platforms
Use/Mention Distinctions

While methods can be referred to without invoking them (let x = foo.bar instead of let x = foo.bar()), this is not currently possible for properties and subscripts.

Making indirect references to a properties' concrete types also lets us expose metadata about the property, and in the future additional behaviors.

More Expressive KeyPaths

We would also like to support being able to use Key Paths to access into collections, which is not currently possible.

Proposed solution
We propose introducing a new expression akin to Type.method, but for properties and subscripts. These property reference expressions produce KeyPath objects, rather than Strings. KeyPaths are a family of generic classes (structs and protocols here would be ideal, but requires generalized existentials) which encapsulate a property reference or chain of property references, including the type, mutability, property name(s), and ability to set/get values.

Here's a sample of it in use:

Swift
struct Person {
    var name: String
    var friends: [Person]
    var bestFriend: Person?
}

var han = Person(name: "Han Solo", friends: )
var luke = Person(name: "Luke Skywalker", friends: [han])

let firstFriendsNameKeyPath = Person.friends[0].name

let firstFriend = luke[path] // han

// or equivalently, with type inferred from context
let firstFriendName = luke[.friends[0].name]

// rename Luke's first friend
luke[firstFriendsNameKeyPath] = "A Disreputable Smuggler"

let bestFriendsName = luke[.bestFriend]?.name // nil, if he is the last jedi
Detailed design
Core KeyPath Types

KeyPaths are a hierarchy of progressively more specific classes, based on whether we have prior knowledge of the path through the object graph we wish to traverse.

Unknown Path / Unknown Root Type

AnyKeyPath is fully type-erased, referring to 'any route' through an object/value graph for 'any root'. Because of type-erasure many operations can fail at runtime and are thus nillable.

Swift
class AnyKeyPath: CustomDebugStringConvertible, Hashable {
    // MARK - Composition
    // Returns nil if path.rootType != self.valueType
    func appending(path: AnyKeyPath) -> AnyKeyPath?
    
    // MARK - Runtime Information
    class var rootType: Any.Type
    class var valueType: Any.Type
    
    static func == (lhs: AnyKeyPath, rhs: AnyKeyPath) -> Bool
    var hashValue: Int
}
Unknown Path / Known Root Type

If we know a little more type information (what kind of thing the key path is relative to), then we can use PartialKeyPath <Root>, which refers to an 'any route' from a known root:

Swift
class PartialKeyPath<Root>: AnyKeyPath {
    // MARK - Composition
    // Returns nil if Value != self.valueType
    func appending(path: AnyKeyPath) -> PartialKeyPath<Root>?
    func appending<Value, AppendedValue>(path: KeyPath<Value, AppendedValue>) -> KeyPath<Root, AppendedValue>?
    func appending<Value, AppendedValue>(path: ReferenceKeyPath<Value, AppendedValue>) -> ReferenceKeyPath<Root, AppendedValue>?
}
Known Path / Known Root Type

When we know both what the path is relative to and what it refers to, we can use KeyPath. Thanks to the knowledge of the Root and Value types, all of the failable operations lose their Optional.

Swift
public class KeyPath<Root, Value>: PartialKeyPath<Root> {
    // MARK - Composition
    func appending<AppendedValue>(path: KeyPath<Value, AppendedValue>) -> KeyPath<Root, AppendedValue>
    func appending<AppendedValue>(path: WritableKeyPath<Value, AppendedValue>) -> Self
    func appending<AppendedValue>(path: ReferenceWritableKeyPath<Value, AppendedValue>) -> ReferenceWritableKeyPath<Root, AppendedValue>
}
Value/Reference Mutation Semantics Mutation

Finally, we have a pair of subclasses encapsulating value/reference mutation semantics. These have to be distinct because mutating a copy of a value is not very useful, so we need to mutate an inout value.

Swift
class WritableKeyPath<Root, Value>: KeyPath<Root, Value> {
    // MARK - Composition
    func appending<AppendedPathValue>(path: WritableKeyPath<Value, AppendedPathValue>) -> WritableKeyPath<Root, AppendedPathValue>
}

class ReferenceWritableKeyPath<Root, Value>: WritableKeyPath<Root, Value> {
    override func appending<AppendedPathValue>(path: WritableKeyPath<Value, AppendedPathValue>) -> ReferenceWritableKeyPath<Root, AppendedPathValue>
}
Access and Mutation Through KeyPaths

To get or set values for a given root and key path we effectively add the following subscripts to all Swift types.

Swift
extension Any {
    subscript(path: AnyKeyPath) -> Any? { get }
    subscript<Root: Self>(path: PartialKeyPath<Root>) -> Any { get }
    subscript<Root: Self, Value>(path: KeyPath<Root, Value>) -> Value { get }
    subscript<Root: Self, Value>(path: WritableKeyPath<Root, Value>) -> Value { set, get }
}
This allows for code like

Swift
person[.name] // Self.type is inferred
which is both appealingly readable, and doesn't require read-modify-write copies (subscripts access self inout). Conflicts with existing subscripts are avoided by using generic subscripts to specifically only accept key paths with a Root of the type in question.

Referencing Key Paths

Forming a KeyPath borrows from the same syntax used to reference methods and initializers,Type.instanceMethod only now working for properties and collections. Optionals are handled via optional-chaining. Multiply dotted expressions are allowed as well, and work just as if they were composed via the appending methods on KeyPath.

There is no change or interaction with the keypath() syntax introduced in Swift 3.

Performance

The performance of interacting with a property via KeyPaths should be close to the cost of calling the property directly.

Source compatibility
This change is additive and there should no affect on existing source.

Effect on ABI stability
This feature adds the following requirements to ABI stability:

mechanism to access key paths of public properties
We think a protocol-based design would be preferable once the language has sufficient support for generalized existentials to make that ergonomic. By keeping the class hierarchy closed and the concrete implementations private to the implementation it should be tractable to provide compatibility with an open protocol-based design in the future.

Effect on API resilience
This should not significantly impact API resilience, as it merely provides a new mechanism for operating on existing APIs.

Alternatives considered
More Features

Various drafts of this proposal have included additional features (decomposable key paths, prefix comparisons, support for custom KeyPath subclasses, creating a KeyPath from a String at runtime, KeyPaths conforming to Codable, bound key paths as a concrete type, etc.). We anticipate approaching these enhancements additively once the core KeyPath functionality is in place.

Spelling

We also explored many different spellings, each with different strengths. We have chosen the current syntax due to the balance with existing function type references.

Current keypath Lisp-style
Person.friends[0].name keypath(Person, .friends[0].name) `Person.friend.name
luke[.friends[0].name] keypath(luke, .friends[0].name) luke`.friends[0].name
luke.friends[0][.name] keypath(luke.friends[0], .name) luke.friends[0]`.name
While the crispness is very appealing, the spelling of the 'escape' character was hard to agree upon (along with the fact that it requires parentheses to reduce ambiguity). keypath was very specific, but verbose especially when composing multiple key paths together.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

[Joe_Groff]

We already encounter this situation with static vs instance methods, since `Foo.bar` can refer to either a static method `bar` or an unbound instance method `bar`. We use type context to disambiguate, favoring the static member if context doesn't help:

struct X {
  static func foo() {}
  func foo() {}
}

let foo1 = X.foo // Defaults to static member
let foo2: () -> () = X.foo // Picks static member by type context
let foo3: (X) -> () -> () = X.foo // Picks instance member by type context

-Joe

···

On Mar 17, 2017, at 2:53 PM, Michael LeHew <lehewjr@apple.com> wrote:

On Mar 17, 2017, at 2:21 PM, BJ Homer <bjhomer@gmail.com <mailto:bjhomer@gmail.com>> wrote:

This looks great!

What happens in the case when there is a static property by the same name as an instance property? e.g.

struct Foo {
    static var bar: Bar
    var bar: Bar
}

Foo.bar // What is this?

Is it still possible to reference both the static property and a KeyPath to the instance method?

This is essentially the same question that I arrived at in my reply to Vladimir. I think Joe might be best able to answer here.

[David_Smith]

Hi friendly swift-evolution folks,

The Foundation and Swift team would like for you to consider the following proposal:

This proposal is really incredible! It is an invaluable addition to the language - far better than simple first-class properties. I really can’t wait to see it implemented! The design looks very solid. I’m especially happy to see that a path to eventually get away from using classes has already been identified and planned for.

Thank you so much for bringing this forward in Swift 4. It is a wonderful (and rather unexpected) surprise!

Seeing this makes me *really* wish we had a way to get at a collection of `PartialKeyPath<Self>` for all the (visible) properties of a type. I guess the visible part of that makes it tricky. We can always work around it in the meantime.

Joe mentioned some ideas along these lines in the "Swift's reflection" thread today. Definitely seems like a solid direction to investigate.

  David

···

On Mar 17, 2017, at 3:08 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

On Mar 17, 2017, at 12:04 PM, Michael LeHew via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Many thanks,
-Michael

Smart KeyPaths: Better Key-Value Coding for Swift
Proposal: SE-NNNN
Authors: David Smith <https://github.com/Catfish-Man&gt;, Michael LeHew <https://github.com/mlehew&gt;, Joe Groff <https://github.com/jckarter&gt;
Review Manager: TBD
Status: Awaiting Review
Associated PRs:
#644 <https://github.com/apple/swift-evolution/pull/644&gt;
Introduction
We propose a family of concrete Key Path types that represent uninvoked references to properties that can be composed to form paths through many values and directly get/set their underlying values.

Motivation
We Can Do Better than String

On Darwin platforms Swift's existing keypath() syntax provides a convenient way to safely refer to properties. Unfortunately, once validated, the expression becomes a String which has a number of important limitations:

Loss of type information (requiring awkward Any APIs)
Unnecessarily slow to parse
Only applicable to NSObjects
Limited to Darwin platforms
Use/Mention Distinctions

While methods can be referred to without invoking them (let x = foo.bar instead of let x = foo.bar()), this is not currently possible for properties and subscripts.

Making indirect references to a properties' concrete types also lets us expose metadata about the property, and in the future additional behaviors.

More Expressive KeyPaths

We would also like to support being able to use Key Paths to access into collections, which is not currently possible.

Proposed solution
We propose introducing a new expression akin to Type.method, but for properties and subscripts. These property reference expressions produce KeyPath objects, rather than Strings. KeyPaths are a family of generic classes (structs and protocols here would be ideal, but requires generalized existentials) which encapsulate a property reference or chain of property references, including the type, mutability, property name(s), and ability to set/get values.

Here's a sample of it in use:

Swift
struct Person {
    var name: String
    var friends: [Person]
    var bestFriend: Person?
}

var han = Person(name: "Han Solo", friends: )
var luke = Person(name: "Luke Skywalker", friends: [han])

let firstFriendsNameKeyPath = Person.friends[0].name

let firstFriend = luke[path] // han

// or equivalently, with type inferred from context
let firstFriendName = luke[.friends[0].name]

// rename Luke's first friend
luke[firstFriendsNameKeyPath] = "A Disreputable Smuggler"

let bestFriendsName = luke[.bestFriend]?.name // nil, if he is the last jedi
Detailed design
Core KeyPath Types

KeyPaths are a hierarchy of progressively more specific classes, based on whether we have prior knowledge of the path through the object graph we wish to traverse.

Unknown Path / Unknown Root Type

AnyKeyPath is fully type-erased, referring to 'any route' through an object/value graph for 'any root'. Because of type-erasure many operations can fail at runtime and are thus nillable.

Swift
class AnyKeyPath: CustomDebugStringConvertible, Hashable {
    // MARK - Composition
    // Returns nil if path.rootType != self.valueType
    func appending(path: AnyKeyPath) -> AnyKeyPath?
    
    // MARK - Runtime Information
    class var rootType: Any.Type
    class var valueType: Any.Type
    
    static func == (lhs: AnyKeyPath, rhs: AnyKeyPath) -> Bool
    var hashValue: Int
}
Unknown Path / Known Root Type

If we know a little more type information (what kind of thing the key path is relative to), then we can use PartialKeyPath <Root>, which refers to an 'any route' from a known root:

Swift
class PartialKeyPath<Root>: AnyKeyPath {
    // MARK - Composition
    // Returns nil if Value != self.valueType
    func appending(path: AnyKeyPath) -> PartialKeyPath<Root>?
    func appending<Value, AppendedValue>(path: KeyPath<Value, AppendedValue>) -> KeyPath<Root, AppendedValue>?
    func appending<Value, AppendedValue>(path: ReferenceKeyPath<Value, AppendedValue>) -> ReferenceKeyPath<Root, AppendedValue>?
}
Known Path / Known Root Type

When we know both what the path is relative to and what it refers to, we can use KeyPath. Thanks to the knowledge of the Root and Value types, all of the failable operations lose their Optional.

Swift
public class KeyPath<Root, Value>: PartialKeyPath<Root> {
    // MARK - Composition
    func appending<AppendedValue>(path: KeyPath<Value, AppendedValue>) -> KeyPath<Root, AppendedValue>
    func appending<AppendedValue>(path: WritableKeyPath<Value, AppendedValue>) -> Self
    func appending<AppendedValue>(path: ReferenceWritableKeyPath<Value, AppendedValue>) -> ReferenceWritableKeyPath<Root, AppendedValue>
}
Value/Reference Mutation Semantics Mutation

Finally, we have a pair of subclasses encapsulating value/reference mutation semantics. These have to be distinct because mutating a copy of a value is not very useful, so we need to mutate an inout value.

Swift
class WritableKeyPath<Root, Value>: KeyPath<Root, Value> {
    // MARK - Composition
    func appending<AppendedPathValue>(path: WritableKeyPath<Value, AppendedPathValue>) -> WritableKeyPath<Root, AppendedPathValue>
}

class ReferenceWritableKeyPath<Root, Value>: WritableKeyPath<Root, Value> {
    override func appending<AppendedPathValue>(path: WritableKeyPath<Value, AppendedPathValue>) -> ReferenceWritableKeyPath<Root, AppendedPathValue>
}
Access and Mutation Through KeyPaths

To get or set values for a given root and key path we effectively add the following subscripts to all Swift types.

Swift
extension Any {
    subscript(path: AnyKeyPath) -> Any? { get }
    subscript<Root: Self>(path: PartialKeyPath<Root>) -> Any { get }
    subscript<Root: Self, Value>(path: KeyPath<Root, Value>) -> Value { get }
    subscript<Root: Self, Value>(path: WritableKeyPath<Root, Value>) -> Value { set, get }
}
This allows for code like

Swift
person[.name] // Self.type is inferred
which is both appealingly readable, and doesn't require read-modify-write copies (subscripts access self inout). Conflicts with existing subscripts are avoided by using generic subscripts to specifically only accept key paths with a Root of the type in question.

Referencing Key Paths

Forming a KeyPath borrows from the same syntax used to reference methods and initializers,Type.instanceMethod only now working for properties and collections. Optionals are handled via optional-chaining. Multiply dotted expressions are allowed as well, and work just as if they were composed via the appending methods on KeyPath.

There is no change or interaction with the keypath() syntax introduced in Swift 3.

Performance

The performance of interacting with a property via KeyPaths should be close to the cost of calling the property directly.

Source compatibility
This change is additive and there should no affect on existing source.

Effect on ABI stability
This feature adds the following requirements to ABI stability:

mechanism to access key paths of public properties
We think a protocol-based design would be preferable once the language has sufficient support for generalized existentials to make that ergonomic. By keeping the class hierarchy closed and the concrete implementations private to the implementation it should be tractable to provide compatibility with an open protocol-based design in the future.

Effect on API resilience
This should not significantly impact API resilience, as it merely provides a new mechanism for operating on existing APIs.

Alternatives considered
More Features

Various drafts of this proposal have included additional features (decomposable key paths, prefix comparisons, support for custom KeyPath subclasses, creating a KeyPath from a String at runtime, KeyPaths conforming to Codable, bound key paths as a concrete type, etc.). We anticipate approaching these enhancements additively once the core KeyPath functionality is in place.

Spelling

We also explored many different spellings, each with different strengths. We have chosen the current syntax due to the balance with existing function type references.

Current keypath Lisp-style
Person.friends[0].name keypath(Person, .friends[0].name) `Person.friend.name
luke[.friends[0].name] keypath(luke, .friends[0].name) luke`.friends[0].name
luke.friends[0][.name] keypath(luke.friends[0], .name) luke.friends[0]`.name
While the crispness is very appealing, the spelling of the 'escape' character was hard to agree upon (along with the fact that it requires parentheses to reduce ambiguity). keypath was very specific, but verbose especially when composing multiple key paths together.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

[Michael_J_LeHew_Jr]

Hi friendly swift-evolution folks,

The Foundation and Swift team would like for you to consider the following proposal:

This proposal is really incredible! It is an invaluable addition to the language - far better than simple first-class properties. I really can’t wait to see it implemented! The design looks very solid. I’m especially happy to see that a path to eventually get away from using classes has already been identified and planned for.

Thank you so much for bringing this forward in Swift 4. It is a wonderful (and rather unexpected) surprise!

Seeing this makes me *really* wish we had a way to get at a collection of `PartialKeyPath<Self>` for all the (visible) properties of a type. I guess the visible part of that makes it tricky. We can always work around it in the meantime.

We had discussed that a future application where KeyPath's would make a lot of sense is with the Mirror API. Of course in the interest of the finiteness of time, we aren't pursuing that right now.

One thing that gets interesting with the scope-restricted visibility of KeyPaths, is what happens if an fileprivate KeyPath gets leaked out of the file? That's a scary/maybe useful thing? But a complication that emerges pretty quick and thus another reason not to pursue that just now.

···

On Mar 17, 2017, at 3:08 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On Mar 17, 2017, at 12:04 PM, Michael LeHew via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Many thanks,
-Michael

Smart KeyPaths: Better Key-Value Coding for Swift
Proposal: SE-NNNN
Authors: David Smith <https://github.com/Catfish-Man&gt;, Michael LeHew <https://github.com/mlehew&gt;, Joe Groff <https://github.com/jckarter&gt;
Review Manager: TBD
Status: Awaiting Review
Associated PRs:
#644 <https://github.com/apple/swift-evolution/pull/644&gt;
Introduction
We propose a family of concrete Key Path types that represent uninvoked references to properties that can be composed to form paths through many values and directly get/set their underlying values.

Motivation
We Can Do Better than String

On Darwin platforms Swift's existing keypath() syntax provides a convenient way to safely refer to properties. Unfortunately, once validated, the expression becomes a String which has a number of important limitations:

Loss of type information (requiring awkward Any APIs)
Unnecessarily slow to parse
Only applicable to NSObjects
Limited to Darwin platforms
Use/Mention Distinctions

While methods can be referred to without invoking them (let x = foo.bar instead of let x = foo.bar()), this is not currently possible for properties and subscripts.

Making indirect references to a properties' concrete types also lets us expose metadata about the property, and in the future additional behaviors.

More Expressive KeyPaths

We would also like to support being able to use Key Paths to access into collections, which is not currently possible.

Proposed solution
We propose introducing a new expression akin to Type.method, but for properties and subscripts. These property reference expressions produce KeyPath objects, rather than Strings. KeyPaths are a family of generic classes (structs and protocols here would be ideal, but requires generalized existentials) which encapsulate a property reference or chain of property references, including the type, mutability, property name(s), and ability to set/get values.

Here's a sample of it in use:

Swift
struct Person {
    var name: String
    var friends: [Person]
    var bestFriend: Person?
}

var han = Person(name: "Han Solo", friends: )
var luke = Person(name: "Luke Skywalker", friends: [han])

let firstFriendsNameKeyPath = Person.friends[0].name

let firstFriend = luke[path] // han

// or equivalently, with type inferred from context
let firstFriendName = luke[.friends[0].name]

// rename Luke's first friend
luke[firstFriendsNameKeyPath] = "A Disreputable Smuggler"

let bestFriendsName = luke[.bestFriend]?.name // nil, if he is the last jedi
Detailed design
Core KeyPath Types

KeyPaths are a hierarchy of progressively more specific classes, based on whether we have prior knowledge of the path through the object graph we wish to traverse.

Unknown Path / Unknown Root Type

AnyKeyPath is fully type-erased, referring to 'any route' through an object/value graph for 'any root'. Because of type-erasure many operations can fail at runtime and are thus nillable.

Swift
class AnyKeyPath: CustomDebugStringConvertible, Hashable {
    // MARK - Composition
    // Returns nil if path.rootType != self.valueType
    func appending(path: AnyKeyPath) -> AnyKeyPath?
    
    // MARK - Runtime Information
    class var rootType: Any.Type
    class var valueType: Any.Type
    
    static func == (lhs: AnyKeyPath, rhs: AnyKeyPath) -> Bool
    var hashValue: Int
}
Unknown Path / Known Root Type

If we know a little more type information (what kind of thing the key path is relative to), then we can use PartialKeyPath <Root>, which refers to an 'any route' from a known root:

Swift
class PartialKeyPath<Root>: AnyKeyPath {
    // MARK - Composition
    // Returns nil if Value != self.valueType
    func appending(path: AnyKeyPath) -> PartialKeyPath<Root>?
    func appending<Value, AppendedValue>(path: KeyPath<Value, AppendedValue>) -> KeyPath<Root, AppendedValue>?
    func appending<Value, AppendedValue>(path: ReferenceKeyPath<Value, AppendedValue>) -> ReferenceKeyPath<Root, AppendedValue>?
}
Known Path / Known Root Type

When we know both what the path is relative to and what it refers to, we can use KeyPath. Thanks to the knowledge of the Root and Value types, all of the failable operations lose their Optional.

Swift
public class KeyPath<Root, Value>: PartialKeyPath<Root> {
    // MARK - Composition
    func appending<AppendedValue>(path: KeyPath<Value, AppendedValue>) -> KeyPath<Root, AppendedValue>
    func appending<AppendedValue>(path: WritableKeyPath<Value, AppendedValue>) -> Self
    func appending<AppendedValue>(path: ReferenceWritableKeyPath<Value, AppendedValue>) -> ReferenceWritableKeyPath<Root, AppendedValue>
}
Value/Reference Mutation Semantics Mutation

Finally, we have a pair of subclasses encapsulating value/reference mutation semantics. These have to be distinct because mutating a copy of a value is not very useful, so we need to mutate an inout value.

Swift
class WritableKeyPath<Root, Value>: KeyPath<Root, Value> {
    // MARK - Composition
    func appending<AppendedPathValue>(path: WritableKeyPath<Value, AppendedPathValue>) -> WritableKeyPath<Root, AppendedPathValue>
}

class ReferenceWritableKeyPath<Root, Value>: WritableKeyPath<Root, Value> {
    override func appending<AppendedPathValue>(path: WritableKeyPath<Value, AppendedPathValue>) -> ReferenceWritableKeyPath<Root, AppendedPathValue>
}
Access and Mutation Through KeyPaths

To get or set values for a given root and key path we effectively add the following subscripts to all Swift types.

Swift
extension Any {
    subscript(path: AnyKeyPath) -> Any? { get }
    subscript<Root: Self>(path: PartialKeyPath<Root>) -> Any { get }
    subscript<Root: Self, Value>(path: KeyPath<Root, Value>) -> Value { get }
    subscript<Root: Self, Value>(path: WritableKeyPath<Root, Value>) -> Value { set, get }
}
This allows for code like

Swift
person[.name] // Self.type is inferred
which is both appealingly readable, and doesn't require read-modify-write copies (subscripts access self inout). Conflicts with existing subscripts are avoided by using generic subscripts to specifically only accept key paths with a Root of the type in question.

Referencing Key Paths

Forming a KeyPath borrows from the same syntax used to reference methods and initializers,Type.instanceMethod only now working for properties and collections. Optionals are handled via optional-chaining. Multiply dotted expressions are allowed as well, and work just as if they were composed via the appending methods on KeyPath.

There is no change or interaction with the keypath() syntax introduced in Swift 3.

Performance

The performance of interacting with a property via KeyPaths should be close to the cost of calling the property directly.

Source compatibility
This change is additive and there should no affect on existing source.

Effect on ABI stability
This feature adds the following requirements to ABI stability:

mechanism to access key paths of public properties
We think a protocol-based design would be preferable once the language has sufficient support for generalized existentials to make that ergonomic. By keeping the class hierarchy closed and the concrete implementations private to the implementation it should be tractable to provide compatibility with an open protocol-based design in the future.

Effect on API resilience
This should not significantly impact API resilience, as it merely provides a new mechanism for operating on existing APIs.

Alternatives considered
More Features

Various drafts of this proposal have included additional features (decomposable key paths, prefix comparisons, support for custom KeyPath subclasses, creating a KeyPath from a String at runtime, KeyPaths conforming to Codable, bound key paths as a concrete type, etc.). We anticipate approaching these enhancements additively once the core KeyPath functionality is in place.

Spelling

We also explored many different spellings, each with different strengths. We have chosen the current syntax due to the balance with existing function type references.

Current keypath Lisp-style
Person.friends[0].name keypath(Person, .friends[0].name) `Person.friend.name
luke[.friends[0].name] keypath(luke, .friends[0].name) luke`.friends[0].name
luke.friends[0][.name] keypath(luke.friends[0], .name) luke.friends[0]`.name
While the crispness is very appealing, the spelling of the 'escape' character was hard to agree upon (along with the fact that it requires parentheses to reduce ambiguity). keypath was very specific, but verbose especially when composing multiple key paths together.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution