New "Unevaluated" type for Decoder to allow later re-encoding of data with unknown structure

[itaiferber]

Thanks for putting together some thoughts on the issue, and apologies on the delay — I had most of this typed out yesterday but didn't get a chance to finish. There's a lot to unpack here, so let's take a step back for a moment.

@itaiferber

We can certainly add some dynamic lookup features to Unevaluated to make it more accessible, and I think that would be a great extension to the feature (and a good use-case for dynamic lookup)!

I think I should have gated this statement a bit better, and with some thought, I've changed my mind a little. I think adding dynamic lookup to Unevaluated would be a great addition if we

1. Identify a clear use-case for adding the feature
2. Decide on beneficial semantics of the implementation that would integrate well with the existing API

Before tackling #2 here, I think we need to figure out #1 — is there indeed a use-case here that merits the potential complexity of the feature?

To reiterate what I said in the other thread,

@itaiferber

The original request for Unevaluated, both partially here, and in SR-5311 is an answer to the currently intractable case of "there is a part of my serialized object graph that I know nothing about, and need to be able to decode without knowing anything about it, for the sake of preserving the values as-is." Usually this comes up when you are decoding parts of a payload whose structure is variable — you need some of the information from it, but not all, and the parts you don't need (and know nothing about) need to be preserved for re-encoding back later on.

The idea here behind Unevaluated is to give you that data, as-is, in a format that may or may not be opaque to you. You can later re-encode the Unevaluated instance, and ideally, get back the same representation that you had before.

This is separate from a request for returning a parsed response in a reasonably consumable format.

The primary goal for introducing Unevaluated is to solve the issue of not being able to decode data you know nothing about for the purpose of preserving it. At its core, the representation of this data will be opaque to you, since the purpose is not to consume the data but to preserve it for future re-encodes.

So before we decide on dynamic lookup or anything similar, we need to decide on whether making Unevaluated consumable in a reasonable way is something we want to do or not; all API needs motivation for its introduction, not motivation against its introduction. Questions:

1. What use-case does making Unevaluated consumable solve? Is there something you can do by getting an Unevaluated that you can't by using existing APIs directly?
2. What sort of patterns might we enable by making Unevaluated consumable?

I think the answer to #1 is "no" at the moment. Right now, I can't think of anything you can't decode by writing your own AnyCodable/JSON/what-have-you enum to decode arbitrary contents of a payload in a way that lets you inspect them in a type-safe way.

As for #2, I think your example shows how we might expect some folks to use the feature:

@cherrywoods

init(from decoder: Decoder) throws {
    let unevaluated = decoder.unevaluatedData()
    self.value = unevaluated.value
    self.one = unevaluated.otherStuff.0
    self.two = unevaluated.otherStuff[1]
}

Error-handling and casting aside, I don't think it would be unreasonable for many developers to flock to a potentially "easier" and less verbose way of getting at their values. Given two ways of doing the same thing with one of them being easier at the cost of some safety, I think many would understandably go with the easier option. In general, we try to avoid offering two ways of doing the same thing, especially when a major goal of the Codable design is to offer strong type safety for working with data that's generally not typed; undermining that goal won't be much help.

So, if we're looking to add dynamic lookup to something like Unevaluated, let's motivate it — is there a problem that we would be solving (in a format-agnostic way)?

---

Keep in mind, again, that this is in contrast to offering your own, say AnyCodable type (possible today) which would allow you to do something like this:

init(from decoder: Decoder) throws {
    let container = try decoder.singleValueContainer()
    let stuff = try container.decode(AnyCodable.self)
    guard case .dictionary(let dictionary) = stuff,
          case .array(let array) = dictionary["otherStuff"] else {
        // throw
    }

    self.one = array[0]
    self.two = array[1]
}

If you do offer your own type, you can also definitely add dynamic lookup to make it more dynamic too.