How does automatic Codable compliance work? For the sake of extending it

This is an extension of my "Yet another tuple/protocol thread, but restricted" post. This time I'm asking on how En/Decoder use a CodingKey type to assist in (de)serializing a En/Decodable type. Does the automatic synthesis for En/Decodable include synthesizing a CodingKey type? If it does, then my suggestion to extend Equatable and Hashable for tuple-bearing nominal types (by recursively piercing the tuple members) to the En/Decodable needs use to come up with names for the inner tuple members. We can use period-separated member names as the string values, but what would the enum cases be called? Does it matter; can a user access the cases of a synthesized CodingKey type? If not, then randomly-generated identifiers would be fine.

If you want to understand how the compiler synthesises it for you:

  1. swift/DerivedConformanceCodingKey.cpp at main · apple/swift · GitHub
  2. swift/DerivedConformanceCodable.cpp at main · apple/swift · GitHub

For understanding how JSONEncoder/JSONDecoder works, you can read the source code for it here.

2 Likes

You can indeed read the code to get the specifics, but to summarize the process more generally:

  1. Codable synthesis (there are actually two passes — an Encodable synthesis pass, and a Decodable synthesis pass, but they largely share work) primarily uses the CodingKeys enum nested type on a type to influence how to implement init(from:) and encode(to:)
    • Specifically, if a CodingKeys type is not found, one will be synthesized
    • The process for synthesizing a CodingKeys enum involves iterating all stored properties on the owning type — each stored property is validated for Encodable/Decodable conformance, and if everything checks out, the CodingKeys enum gets a case with the same name as the property
  2. Once the type is validated to have a CodingKeys enum (either via generation, or a user-supplied one), every case is checked against what the owning type has: every stored property needs to map 1-to-1 with an enum case (for Encodable, stored properties which don't have a matching enum case are explicitly not encoded in encode(to:), but for Decodable, every stored property needs to be assigned an enum case)
  3. Once this validation has passed, encode(to:) and init(from:) are generated by enumerating all of the cases in the CodingKeys enum, and generating an encode/encodeIfPresent/decode/decodeIfPresent call based on the type

Largely, the CodingKeys enum is the "source of truth" for what to encode and decode, and the enum cases (and their names) matter. To answer your more specific questions:

This would matter primarily if you're looking to make the tuple itself Codable and rely on the existing synthesis machinery to generate a conformance for it. Since tuples are not nominal, this wouldn't work anyway; instead, inside of encode(to:) and init(from:) generation, you could do something totally custom, making up custom names on the spot to encode into a nested container, for instance. There's flexibility here, assuming the tuple is checked for consistency up-front (e.g., all of its members are also Codable themselves)

Yes, a type can access its own synthesized CodingKeys enum — by default the enum is synthesized as private, but if you implement your own init(from:)/encode(to:) [but not both], you will be able to access it and its cases by name.


One solution for this is to generate a CodingKeys type for the tuple nested somewhere/given a private name which contains auto-generated case names, but explicit String values which contain the actual key names you're looking for, e.g.

struct Foo : Codable {
    let bar: (String, Int)

    // Synthesized by Codable conformance:
    private enum CodingKeys: String, CodingKey {
        case bar
    }

    // Synthesized by additional tuple synthesis:
    private enum _bar_CodingKeys: String, CodingKey {
        case member0 = "0"
        case member1 = "1"
    }

    // Synthesized by Codable conformance with tuple additions:
    public func encode(to encoder: Encoder) throws {
        var container = try encoder.container(keyedBy: CodingKeys.self)
        var barContainer = try encoder.nestedContainer(keyedBy: _bar_CodingKeys.self, forKey: .bar)
        try barContainer.encode(bar.0, forKey: .member0)
        try barContainer.encode(bar.1, forKey: .member1)
    }
}

This would produce a result that looks like

{ "bar": { "0": "Foo", "1": 42 } }

You can imagine that if the tuple members have labels, those labels are used:

  • (name: String, Int){ "name": "Foo", "1": 42 }
  • (name: String, identifier: Int) { "name": "Foo", "identifier": 42 }

There's also something to be said for ditching the member keys and just encoding an unkeyed container instead:

struct Foo : Codable {
    let bar: (String, Int)

    // Synthesized by Codable conformance:
    private enum CodingKeys: String, CodingKey {
        case bar
    }

    // Synthesized by Codable conformance with tuple additions:
    public func encode(to encoder: Encoder) throws {
        var container = try encoder.container(keyedBy: CodingKeys.self)
        var barContainer = try encoder.nestedUnkeyedContainer(forKey: .bar)
        try barContainer.encode(bar.0)
        try barContainer.encode(bar.1)
    }
}

This produces

{ "bar": ["Foo", "42"] }
4 Likes

I was wondering, whether there will be something such as DerivedConformace attribute, so programmers would be able to provide their own auto-synthesized conformances.

I am aware, that some "meta code" would be needed, but considering the thread about variadic types and generics, there is a need for such an approach.

I am asking, because I dislike DerivedConformace in which "every single type needs it's own cpp file". I see how it makes my code significantly shorter, however, I don't like the magic.

1 Like

This is something we'd ideally like to work toward over time, esp. with something like a hygienic macros system. Ideally, with enough language features, Codable synthesis could eventually be pulled out of the compiler and into the standard library. There's no timeline for this happening, though, and there'd need to be a lot of infrastructure laid down before this would be possible.

2 Likes

I like to hear that. I understand this is no simple task. Especially done in manner, that would fit Swift nicely.

1 Like