Persisting unknown fields in JSON with Codable

Using Swift
, ,
1

I'm writing an editor that stores projects as JSON. I've defined Codable models for all the known fields but sometimes the project contains unknown fields that are being lost at the decode step. I was thinking of adding a dictionary of String: Any to all my models to store unknown fields but I'm unsure how to handle arbitrary keys and types.
For example, if I have a model for foo, bar, and baz I need to get the "unknown" fields back in at encoding:

{
"foo": {
    "bar": {
          "known": "known",
          "unknown": 1
    }
    "baz": [
                   {
                         "known": "known",
                         "unknown": "unknown"
                   },
                   {
                         "known": "known",
                         "unknown": "unknown"
                   }
              ]
},
"unknown": true
}

I'd appreciate any advice about how to do this with Codable or if there's a better way of persisting those values across JSON's, Thanks

2

Consider JSONSerialization, it might work better in this case.

3

Something like GitHub - Flight-School/AnyCodable: Type-erased wrappers for Encodable, Decodable, and Codable values can help too

4

There's unfortunately no great way to do this in the general case through Codable at the moment — because for certain formats, the data format can be ambiguous unless you know the type and structure of the specific data you're looking to decode, it can be difficult (or impossible) to decode or otherwise hold on to unknown data. There's an old Radar I used to have floating around for supporting this, but it's a pretty big lift.

For JSON specifically, though, the AnyCodable wrappers that @AlexisQapa links to can be useful: they basically attempt to decode all known base Codable types (Int, Double, Bool, String, etc.) until something succeeds, and for JSON, this largely works because types are indeed somewhat encoded in the JSON format itself.

If you do try to use them, the general method might look something like this:

/// A CodingKey that can take on any String or Int value.
/// Used to inspect keys in an arbitrary keyed container.
private struct AnyCodingKey: CodingKey {
    let intValue: Int?
    let stringValue: String
    
    init(intValue: Int) {
        self.intValue = intValue
        self.stringValue = "\(intValue)"
    }
    
    init(stringValue: String) {
        self.intValue = Int(stringValue)
        self.stringValue = stringValue
    }
}

struct S: Codable {
    // ... known properties ...

    let extraData: [String: AnyCodable]
    
    private enum CodingKeys: String, CodingKey {
        // ... known properties ...
    }
    
    init(from decoder: Decoder) throws {
        let knownValueContainer = try decoder.container(keyedBy: CodingKeys.self)
        // ... decode known properties via `knownValueContainer` ...
        
        let unknownValueContainer = try decoder.container(keyedBy: AnyCodingKey.self)
        var extraData = [String: AnyCodable]()
        for key in unknownValueContainer.allKeys {
            guard CodingKeys(stringValue: key.stringValue) == nil else {
                // This key is a `CodingKeys` key we should have already decoded above.
                continue
            }
            
            extraData[key.stringValue] = try unknownValueContainer.decode(AnyCodable.self, forKey: key)
        }
        
        self.extraData = extraData
    }
    
    func encode(to encoder: Encoder) throws {
        var knownValueContainer = encoder.container(keyedBy: CodingKeys.self)
        // ... encode known properties via `knownValueContainer`
        
        var unknownValueContainer = encoder.container(keyedBy: AnyCodingKey.self)
        for (key, value) in extraData {
            try unknownValueContainer.encode(value, forKey: AnyCodingKey(stringValue: key))
        }
    }
}

Unfortunately, this is pretty boilerplate-heavy and tedious to apply to many types. Depending on the size and complexity of your data, it may indeed just be easier to reach in and grab only values you need via JSONSerialization, as @tera suggests.

1 Like
5

Thanks mate, that works great.

1 Like