Codable on partially-unstructured JSON

Using Swift
1

I'm attempting to integrate with an API which returns Microsoft's Adaptive Cards JSON for some properties (http://adaptivecards.io). The problem I'm running in to is that the Adaptive Card JSON structure is extremely complex and essentially a black box. Moreover, the framework expects the JSON to be sent in as a string, at which point it does its own internal JSON parsing using a C++ library.

Example response:
{
"type": "card",
"payload": { // Here the card JSON }
}

For that reason, I was hoping to just treat the property that holds the card JSON as data, but the JSONDecoder has already completely parsed the object graph into native objects, so I'm told I the property is an object, not data.

The only solution I've been able to come up with is using @mattt's AnyCodable library on the card JSON, then running it through JSONSerialization.data(withJSONObject:options:) to create a string out of it. That seems gross and inefficient, so I was hoping someone knew a trick to tell Codable, "just ignore this substructure and treat it as a string".

Both the web and Android clients have a way of doing this, so I'm going to have trouble convincing our API guys to force everyone into doing things in a way that's convenient for iOS (like base-64 encoding the card JSON).

3 Likes
2

I came across this problem a while back and wrote a small library called "ValueCodable" that allows me to leave parts of a JSON tree in an "opaque" type Value: GitHub - finestructure/ValueCodable: A Value struct to decode arbitrary data

Perhaps this might cover your use case? Or maybe this is also what AnyCodable is doing?

Note there's not a lot of documentation but the unit tests should demonstrate usage.

1 Like
3

Thank you for your response! It looks like your library is doing the same thing as AnyCodable: recursively attempting to cast every element of an unknown JSON structure to a Codable type with the end result of creating a dictionary.

4

Have you tried superDecoder? You can probably cache the payload part using superDecoder(key: ) and then use a JSONDecoder.DataDecodingStrategy.custom() to decode the payload?

1 Like
5

I tried that but it complained that the object was a dictionary, not Data. I think the DataDecodingStrategy only applies to pre-encoded data. By the time you reach init(with:), it's already been fully parsed into native objects.

6

The issue is that all these solutions work in the sense that the model indeed keeps some unknown data intact, but it doesn't change the fact that the decoder and the encoder do work with the unknown data, causing unnecessary computation, which I think was a desired effect in the request...
The only way to make this work is to write a separate decoder (maybe on top of an existing one, even generic maybe!) that has a way to know paths to ignore :(

1 Like
7

The problem here is the first thing JSONEncoder does in encode is

topLevel = try JSONSerialization.jsonObject(with: data)

So the Data is turned into a JSON Dictionary right away. Once you get to Init(with decoder: Decoder) there's no reference to the decoding entry as a String or Data.

So I think the only simple solution is customizing the decoding to convert the JSONObject back into a JSON String.

Or you could write a custom Decoder that handles this the way you want it to. Though at that point you might as well just decode the JSON manually.

If you have a say in the API, they could also escape the JSON String on the server so it doesn't have to be converted into Base64. That would be my personal preference. e.g.:

struct Thing: Codable {
    let index: Int
    let other: String
}

let json = """
{
    "index": 4,
    "other": "{\\"index\\": 4,\\"other\\": \\"hi\\"}"
}
"""
if let decoded = try? JSONDecoder().decode(Thing.self, from: json.data(using: .utf8)!) {
    print(decoded) //prints Thing(index: 4, other: "{\"index\": 4,\"other\": \"hi\"}")
}
1 Like
8

I just ended up going with AnyCodable and then converting the resulting dictionary back into JSON data using JSONSerialization. It's inelegant, but it works and I'm not too concerned with performance given the relatively small and infrequent occurrence of the Adaptive Cards.

1 Like
9

I assume you ended up doing something like:

let any = try container.decode(AnyDecodable.self, forKey: .payload)
self.payload = try JSONSerialization.data(withJSONObject: any.value, options: [])

Since this container should have a reference to the Any object, it doesn't make much sense if we'd need AnyDecodable just to access it via decode().

I think it'd make much more sense if container had a method like below.

self.payload = try container.data(forKey: .payload, options: [])

I'm not sure if there's enough compelling use cases, though.

10

The problem with extracting the payload is that the original JSON string still needs to be parsed all the way down into the card JSON, in order to match the outer braces properly.

So, there's no really cost-free way to extract part of the JSON.

11

I got frustrated that there was no simple solution to this problem, so I developed my own package to address it. It’s based on OpenAI’s Node.js library.

For now it parses to any dictionary / array and doesn’t support Codable.

Check it out:
https://github.com/itruf/PartialJSON