KeyPaths and Codable

Swift 4's key paths provide a great way for referencing a type's properties in a Swifty way. Since Swift 4's release, many ORMs have adopted key paths in lieu of strings for doing query building. Below are a couple examples from real ORMs built on Swift 4.

Fluent (SQL / NoSQL)

Planet.query(on: conn).filter(\.type == .smallRocky).all()

Perfect CRUD (SQL)

personTable.order(by: \.lastName, \.firstName)

Kuery (Core Data)

Query(Person.self).filter(\Person.age > 20)

Advantages

KeyPaths offer a couple key advantages over Strings:

  • Type-safe: The compiler helps guarantee you've spelled the key path correctly.
  • Auto-completable: If the force is strong with you, Xcode's autocomplete can help you fill in key paths automatically.
  • Strong type info: Key paths know the type of their value allowing you to use leading dot syntax.

Comparing Fluent version 2 (pre Swift 4) to 3 helps highlight this:

// Fluent 2
Planet.makeQuery().filter("type" == PlanetType.smallRocky).all()
// Fluent 3
Planet.query(on: conn).filter(\.type == .smallRocky)

The Problem

The problem with using key paths in this manner is that the relation between key paths and codable is not entirely clear. Let's take the Fluent query for example:

Planet.query(on: conn).filter(\.type == .smallRocky)

When used with a SQL database, this query should result in something along the lines of:

SELECT * FROM "planets" WHERE "type" = ? ["smallRocky"]

While this seems pretty straight forward, the conversion from \.type to "type" is actually quite complex.

Take the Planet model for example:

struct Planet: Codable {
    var name: String
    var type: PlanetType
}

Here it's pretty clear that \.type should yield the string "type". But what happens if we were to define a custom CodingKeys enum for our type?

struct Planet: Codable {
    enum CodingKeys: String, CodingKey {
        case name = "name"
        case type = "planet_type"
    }
    var name: String
    var type: PlanetType
}

Given this Planet struct, the aforementioned SQL query should now be:

SELECT * FROM "planets" WHERE "planet_type" = ? ["smallRocky"]

This is an important distinction because it means that: we need a way to relate a key path into a given Codable type to its coding keys.

Potential API

Ideally, we could use a key path to query a codable model for the associated coding key. Spelling aside, the API could look something like this.

struct Planet: Codable {
    enum CodingKeys: String, CodingKey {
        case name = "name"
        case type = "planet_type"
    }
    var name: String
    var type: PlanetType
}

let key: CodingKey? = Planet.codingKey(for: \.type)
print(key?.stringValue) // Optional("planet_type")

Current Solution

The Swift ORMs I mentioned earlier have some unique (some might call hacky) ways for handling this. Fluent and Perfect CRUD use specialized decoders + some fancy tricks to determine which [CodingKey] path is related to a given KeyPath. Learn more here.

The CoreData based Kuery ORM is able to use the Objective-C bridged _kvcKeyPathString value to get the string value for a key path. As far as I know, this is not affected by codable.

This issue has been touched on a bit before (see SR-5220) but I have yet to see anyone discuss key paths in relation to codable specifically.

Discussion

I would love to get the Core Team's opinion on whether this usage of key paths is something that Swift might support more formally in the future or if it is somewhat of an unintended side effect.

16 Likes

This has been brought up a few times in the past, and I’d be happy to explore the relationships between KeyPaths and CodingKeys further. I don’t myself have any good answers at the moment, but the main difficulties that I see in this area are primarily related to the fact that CodingKeys do not need to map 1-to-1 to properties, unlike KeyPaths.

If you look primarily at synthesized Codable implementations one might assume that CodingKeys map well to properties, but keep in mind that:

  1. A given property need not have an associated CodingKey — for Encodable types, any property may be left out of the CodingKeys enum if one does not wish it to be encoded; similarly, for Decodable types, properties with default values may be left out of the CodingKeys enum if one does not wish for them to be decoded
  2. The further one diverges from the synthesized implementation, the more this mapping falls apart. If you implement encode(to:) and init(from:) directly, there is no requirement that CodingKeys map in any way to properties — CodingKeys can be completely made up, have different names, or simply not exist. Types may choose to encode data in nested containers under multiple different CodingKey types, which totally breaks any sort of mapping
  3. A type need not even define its own CodingKeys enum or encode with any keys — types can encode into unkeyed and single-value containers just fine

Given this, how would we reconcile these two different concepts? Would we only support types which do offer a 1-to-1 mapping? (Note that there isn’t a way to tell dynamically whether this mapping exists, and if a type overrides encode(to:) or init(from:), there’s little we can statically tell about how a CodingKey type is used, too.)

[FWIW, I wouldn’t necessarily call this an unintended side-effect — I think these two features have a lot of overlap which may cause them to appear superficially related in many ways, but in fact, their goals can be quite distinct, in a potentially irreconcilable way.]

1 Like

I suggested the ability to create "prescribed" coding keys for developer contexts where we need auto-encoding/decoding mechanisms, not just for the use case you mention but for creating JSON codable objects with encrypted coding keys (done at client site) evaluated by the keypath hash system on the host (server). This form of encryption would be lightweight and harder to break because the confusion Eve would have trying to know what the keys are , let alone the value would add decision complexity. If you have a base codable protocol of properties that you use across various contexts, you'll make your JSON encryption harder to break by diffusing the plaintext across multiple horizontal contexts referable by your black box cipher mechanism. You can replace everywhere I said "encryption" with "hashable" to get your big data organization needs.

This was to create relationships between codable values and key paths that I on my own started to use in my enums and graph/trees structures to capture, process and safeguard the semantic information of clients in my application . But my thread was closed because I talked down on people using the same words they used to down talk me rather than engage my discussion. Hope you can work something.

1 Like

You bring up a lot of good points.

If a property does not have a match in the CodingKeys, then attempting to resolve the key path to a coding key should fail (return nil or otherwise). That's expected since the key will not appear in the encoded data.

In the case of custom encode / decode methods, the model could potentially be required to define manually how key paths relate to coding keys.


Another approach to the problem of using key paths in ORMs would be allowing developers to get the string representation of the key path (what _kvcKeyPathString does).

Given the complexity around relating properties to encoded data, I think it would be a fair tradeoff to say that types with non-standard coding key to key path naming conventions would need to describe these relations manually. (i.e., planet.type encoding to key "planet_type" instead of "type" would need to declare this to the ORM). In that case, ORMs could utilize the ability to "reflect" the key path's string name as a reasonable default as long as a method is provided for the type to define its own conversions.

1 Like

I recently discovered https://github.com/IBM-Swift/TypeDecoder which also attempts to solve this problem.

/cc @Kyle_Jessup @IanPartridge

In the long term, I think this would be interesting to explore. In the fullness of time, I'd like to see a more "protocol-oriented" and extensible approach to key paths, which could allow for CodingKeyPath to be a refinement of KeyPath with the added requirements of coding, such as being able to map to a key string/integer value, and be usable as part of initialization from a coder.

8 Likes

Do you think something similar to _kvcKeyPathString could be considered in the shorter term? i.e., Swift 5.0 or 5.1?

Basically something like this:

let key = \Planet.type
print(key.pathString) // Optional("type")

Or perhaps the path would be an array, to account for nesting:

let key = \Planet.type.description
print(key.pathStrings) // Optional(["type", "description"])

I'd be happy to submit a proposal for this if so.

That would be really great! I'm also happy to help work on proposals for this system as a longer-term solution.

2 Likes

Extending reflection support beyond what we have now is probably out of scope for Swift 5.

Since a better solution to this problem is out of scope until at least Swift 6, I've worked a bit to improve Vapor's Codable reflection algorithm and I will be moving it to a separate package. Anyone needing the ability to reflect types or convert key paths to strings should take a look at this and consider contributing!

5 Likes
Terms of Service

Privacy Policy

Cookie Policy