New "Unevaluated" type for Decoder to allow later re-encoding of data with unknown structure

[cherrywoods]

@itaiferber

apologies on the delay

No need to apologise, I'm not in a hurry .
Thank you for giving my thoughts API relevant context!

@itaiferber

What use-case does making Unevaluated consumable solve? Is there something you can do by getting an Unevaluated that you can’t by using existing APIs directly?

I actually can't see a specific use case. This is also part of what I wanted to point out: I think we can do all this with a decoder or a container.

Right now, I would want to ask another question for Unevaluated in general:

What I have in mind here is motivated by this thread: Using Unsafe pointers to manipulate a JSON Encoder

The question is: Using Unevaluated, would there be a way to get only the parts of the underlying data as Unevaluated, that I haven't evaluated? Less abstract:

init(from decoder: Decoder) throws {
    let container = try decoder.container(keyedBy: /*some CodingKey enum*/)
    let name = try container.decode(String.self, forKey: /*some key*/)
    let remaining = try container.remainingUnevaluatedData() // would this be this possible?
}

This would not be an unsolvable issue if not, the way to solve it could be this:

• ask the keyed container for all coding keys, via .allKeys
• decode Unevaluated for all the keys
• store the results in a dictionary and use it for re-encoding

I see that this not a big thing, since it is possible to do. However, to me it seems like this is part of the use case for which Unevaluated is actually meant (various examples like the one in the thread above).

To ask an even more general question: All cases I have seen until now for Unevaluated were specific to a concrete format (JSON, actually). Is there a such generic use case, that it can not be solved by a JSONValue (or MessagePackValue, or what ever)?
The one I can see is transferring to another format, which could be very easy with a thing like Unevaluated, but the current Unevaluated isn't usable for it.

@itaiferber

The original request for Unevaluated, both partially here, and in SR-5311 is an answer to the currently intractable case of “there is a part of my serialized object graph that I know nothing about, and need to be able to decode without knowing anything about it, for the sake of preserving the values as-is.” Usually this comes up when you are decoding parts of a payload whose structure is variable — you need some of the information from it, but not all, and the parts you don’t need (and know nothing about) need to be preserved for re-encoding back later on.

The question is if this kind of need-to-re-encode-later issue only comes up in format specific decoding code, or if there are truly format independent structures (like Dictionary, Array, Set, Float, LinkedLists, other data structures) that need this. It is of course nice to keep custom decoding code as generic as possible, but to me, it seems as if you could make the necessary assumptions to use something like JSONValue in all the cases I have seen for Unevaluated. The assumptions I mean are: What are the possible structurings (this is actually set by decoder: keyed and unkeyed containers) and what are the "Primitives“ of the data, in case of a JSON specific API, this would be Strings, Bools, Numbers and Null. In msgpack you would additionally have binary data and extension values. Maybe the API sets some restrictions on the possible primitives too. In my current view, you would only really need Unevaluated if you have an API that is not bound to any format or to a format you can put data into in ways not even the format knows. I don‘t know if something like this exists. Maybe I‘m mistaken here.

If this just isn‘t clear for me and we are talking about adding Unevaluated because it makes it easier to handle such format specific cases, I would be fine with it.