Decode a JSON object of unknown format into a Dictionary with Decodable in Swift 4

[itaiferber]

@cherrywoods

I do not like the idea of such an abstract Unevaluated type, although I think it is in general the right way to solve the problem discussed in this thread.

I think it would be better to add something like this specifically for JSON. I furthermore think this could be a great place to provide some more functionality for accessing "raw" JSON by making Unevaluated more than just a marker type.

Let's pull this apart a little bit, if you don't mind, because I think there's been a bit of correlation here between proposals that I'd like to separate. I'm going to partially reorder your comment to answer sequentially.

Im sticking a bit too much to the implementation example for above. Beyond this, an abstract implementation would leave us with those problems:

• There is no indication that there is just a limited set of options, but there is actually just a limited set of possible types for this Any value, because it is JSON, as we found out.
• What is the decoders internal representation if it isn't JSON?

The original request for Unevaluated, both partially here, and in SR-5311 is an answer to the currently intractable case of "there is a part of my serialized object graph that I know nothing about, and need to be able to decode without knowing anything about it, for the sake of preserving the values as-is." Usually this comes up when you are decoding parts of a payload whose structure is variable — you need some of the information from it, but not all, and the parts you don't need (and know nothing about) need to be preserved for re-encoding back later on.

The idea here behind Unevaluated is to give you that data, as-is, in a format that may or may not be opaque to you. You can later re-encode the Unevaluated instance, and ideally, get back the same representation that you had before.

This is separate from a request for returning a parsed response in a reasonably consumable format.

The problem I see, is this: You somehow find out, that the format you are currently decoding from is JSON. Then you decode such an Unevaluated value from the decoder. Now you have to do something with either the NSDictionary, or NSArray, or whatever you get as .value. Those types are as I see it an implementation detail of JSONDecoder (O and what you get with Unevalutuated is actually a "raw" view into the decoders storage. This has in my opinion several disadvantages:

• It would be just annoying to refactor the custom decoding code, if JSONDecoder started to use Dictionary instead of NSDictionary, Array instead of NSArray, and so on.

This shouldn't be problematic as those types bridge.

• The resulting code won't be easily understandable I guess, because you won't be able to figure out right away, that this code is working with "raw" JSON / I would not immediately connect NSDictionary with JSON's "{ }". At least it won't be self documenting.

This is also no different from manually working with a response from JSONSerialization. You would be getting back the same response, wrapped up in an Unevaluated.

Also, I do dislike the concept of an abstract type for all formats, because I fell like it does not match the design of Decoder. I would consider it better to have some method like abstractRepresentation alongside with keyedContainer, unkeyedContainer and singleValueContainer. The problem with this is, that the return type is still open...

As you say, what would the result of .abstractRepresentation be? How would it be different from the abstract representation contained in an Unevaluated?

If we had a type such as RawJSON (I'm really not good at naming), we could add general working-with-JSON capabilities and not show the NSDictionarys under our cloths to the world. One could ask this type whether it has a keyed container for us, or a number, and so on. We could also tell it that we want to have a number, also if there is a string (only if it is convertible of course). These examples are certainly not well designed right now, but I think it shows what I mean. This new type would could take a way lager role in the JSON decoding game. We could start with such a RawJSON, do some lookup and extraction (like resolving the upper container levels that maybe contain information we don't really want to have as a type on it's own, I don't know if this is happens often, but I have seen something similar with another format) and then at some other point, tell JSONDecoder to decode the remaining JSON for us. If we need to handle something like the meta data from this issue, we could go on manually working with the JSON and if it contained other data, that would rather like to have as an instance of one of our custom types, we could just go on as we started: We would tell JSONDecoder to decode the remaining RawJSON extracted before.

I don't think wo should miss this chance with adding an abstract type. Other implementations for other formats could follow the same way, but would not be required to do so.

This, to me, sounds like the real request here — working with a concrete result type closer to the JSON type I gave an example for in another thread or that @anandabits just posted as well. [This enum is possible today, and relatively easy to put together.]

However, again, I don't think we want to offer types specifically for JSON (especially when they're relatively easy to put together without special internal knowledge). See part of my response upthread to @hooman:

@itaiferber

When designing Codable, one of the big decisions we made early on was that we wanted Codable to be as format-agnostic as we could make it (without making it so abstract as to no longer be helpful); our goal was to help abstract away just enough details to make a given Codable implementation work with many different formats. As such, many existing Codable implementations would work as-is whether encoded through JSONEncoder, PropertyListEncoder, or any other Encoder you might write.

With this, I recognize that the #1 use-case by far for Codable at the moment is for serializing to and from JSON; it's a really popular format at the moment, and when most folks write their Codable implementations, they've got JSON in mind.

<snip>

In any case, our philosophy on this is generally that if JSON can benefit from such a feature, other formats should be able to benefit as well. We'd rather not gate this on JSON adoption specifically, especially as we're looking to add new encoders and decoders ourselves to support further formats.

If you need a JSON enum like above, it's possible to write it yourself (and please feel free to use my gist as reference); however, you cannot today write Unevaluated on your own as it requires explicit adoption by JSONEncoder/JSONDecoder.