Testing JSONEncoder Output

I am trying to test the output of my conformance to Encodable. Specifically what my type looks like when encoded with JSONEncodable. Unfortunately, the solutions I have devised are, in my opinion, bad.

I was wondering if the community had any best practices or lessons learned that they have used to come up with a testing strategy they are happy with.

What have you considered thus far, and what qualifies as a "bad" solution for you?

1 Like

What aspects of the conformance are you looking to test? Are you looking to ensure a general structure is matched, or do you want to check exact byte-wise output?

You might be able to take some inspiration from the JSONEncoder tests for Foundation:

How do these strategies compare to your existing solutions?

(There's also the possibility of encoding with JSONEncoder, then decoding with JSONSerialization to look through the structure programmatically without caring about the byte-wise output. You can do the same by then re-encoding with JSONSerialization and decoding with JSONDecoder.)

1 Like

I intentionally left it vague because I wanted to see what other people are doing. I cannot rule out that my subjective metrics for bad or good are not misguided. All that having been said...

The goal that I have is to verify the structure/semantics of the encoded JSON in isolation. The secondary issue is being able to succinctly identify what is actually wrong.

For instance, in a overly simplified case, if I had these JSON strings:

a

{
  "dateAdded": "2020-09-23T17:58:14Z",
  "id": 1,
  "value": "super duper value"
}

b

{ "value": "super duper value", "dateAdded": "2020-09-23T17:58:14Z", "id": 1 }

c

{
  "id": 2,  
  "value": "super duper value",
  "dateAdded": "2020-09-23T17:58:14Z"
}

a and b are equal because property order and white-space is irrelevant to me. Likewise, a and c are unequal as are b and c. In the case of the unequal case I'd like some way to succinctly report, in the event of failure, that the id property was 2 instead of 1 (or vice versa).

Writing the code I wish I had would look something like this:

// The result of `JSONEncoder.encode(a)` is the JSON illustrated as `a` above
XCTAssertHypotheticalWish(try JSONEncoder.encode(a), matches: """
{
  "value": "super duper value",
  "dateAdded": "2020-09-23T17:58:14Z",
  "id": 1
}
""")

Most of what I have tried to get to XCTAssertHypotheticalWish usually ends up making me decode encodedA, then decode the matches argument (into the same type). Then assert their equality. I dislike that approach because:

  1. It relies on a correct implementation of Decodable.
    • Am I testing my implementation of Decodable or Encodable?
    • Also, what if I have no need for an implementation of Decodable? I had to write one just for the tests? Where does that implementation live?
    • What if my decoding is different than what I encode to?
  2. Now I also have to implement Equatable.

Then there is the naive string comparison solution. But it is easily broken if the keys are out of order...

1 Like

Sounds like what you want is more along the lines of my previous suggestion of using JSONSerialization to decode the encoded JSON into a structure without hitting Decodable. Assuming you're targeting Darwin (this is still possible on Linux, but with a bit more invasive conversion), you can do something like

struct Record: Encodable {
    let id: Int
    let dateAdded: Date
    let value: String
}

func hypotheticalWish<T: Encodable>(_ value: T, matches structure: Any, configuration: ((inout JSONEncoder) -> Void)? = nil) throws {
    var encoder = JSONEncoder()
    configuration?(&encoder)
    
    let data = try encoder.encode(value)
    let decodedStructure = try JSONSerialization.jsonObject(with: data, options: .allowFragments)    

    // The `AnyObject` cast and call through Obj-C's `isEqual` is what's easier on Darwin than Linux.
    // I'm using this for simplicity but you can also very effectively introspect the objects yourself for equality.
    assert((decodedStructure as AnyObject).isEqual(to: structure as AnyObject), "Encoding of \(value) does not match expected structure <\(structure)>: \(decodedStructure)")
}

Given your example, the following works as expected:

let formatter = DateFormatter()
formatter.timeZone = TimeZone(identifier: "gmt")
formatter.locale = Locale(identifier: "en_US_POSIX")
formatter.dateFormat = "yyyy-MM-dd'T'HH:mm:ss'Z'"

let record = Record(id: 1, dateAdded: Date(timeIntervalSinceReferenceDate: 622591094.0), value: "super duper value")
try! hypotheticalWish(record, matches: ["id": 1, "dateAdded": "2020-09-23T17:58:14Z", "value": "super duper value"]) { encoder in
    encoder.dateEncodingStrategy = .formatted(formatter)
}

If you'd like to match against JSON strings, that's a relatively simple addition:

func hypotheticalWish<T: Encodable>(_ value: T, matchesJSON json: String, configuration: ((inout JSONEncoder) -> Void)? = nil) throws {
    let structure = try JSONSerialization.jsonObject(with: json.data(using: .utf8)!, options: .allowFragments)    
    try hypotheticalWish(value, matches: structure, configuration: configuration)
}

try! hypotheticalWish(record, matchesJSON: """
{
    "value": "super duper value",
    "dateAdded": "2020-09-23T17:58:14Z",
    "id": 1
}
""") { encoder in encoder.dateEncodingStrategy = .formatted(formatter) }

Though I'd suggest against including both forms of hypotheticalWish because it's really easy to fall into the trap of accidentally calling hypotheticalWish(_:matches:configuration:) instead of matchesJSON: because it takes an Any (happened to me during testing and it was definitely a headscratcher for a moment!)