1

I don't believe it is currently possible to use the synthesized implementation of Codable and have consistent encoding and decoding of the Date properties of a type when it is used with JSONEncoder and JSONDecoder. Instead, Date's implementation of Codable delegates to the encoder / decoder's dateEncodingStrategy / dateDecodingStrategy.

This behavior is not always acceptable. When it is not acceptable a manual implementation of Codable is necessary. If a team decides consistent encoding / decoding to JSON of all model types is important Codable synthesis is no longer possible for any model types that store Date values.

More importantly, the reality is that some APIs return dates in more than one format in different parts of a response. A single encoding / decoding strategy for an entire payload is not always sufficient.

This steep cliff of language support is very unfortunate. It would be nice if the encoding / decoding strategy could be specified at the property level (and perhaps the type level if a type has several dates that should all be encoded / decoded using the same strategy).

A related use case I have encountered is that a type in a framework which should not be initializable outside the framework cannot conform to Decodable. Types like this defeat Decodable synthesis for all types that need to store them because init(from:) would be public. Unfortunately, synthesis is defeated even when an internal an initializer which takes a Decodable value is available. It would be nice if it were possible to specify a custom decoding strategy (i.e. expression) for any property, although I'm not sure what the best way to express that would be.

1 Like
2

Closely related: Excluding properties from automatic Codable

I believe the best way to address this and similar problems would be, like @anon31136981 writes in the linked thread, a general annotation/decoration mechanism where variables (maybe also functions/types) can be annotated with arbitrary information, e.g. a coding strategy, a marker to ignore a property completely for coding, etc, that Codable synthesis (or maybe something at runtime?) could then use.

1 Like
3

Hi Matthew, a few comments here:

You're in luck! This is exactly what the JSONDecoder.DateDecodingStrategy.custom strategy was written for. You get a Decoder (which has a codingPath for you to inspect) and can make a decision on a per-property basis on how to decode that date.

Even with the .formatted strategy, you can always pass in a DateFormatter subclass which tries multiple date formats that you can decide on supporting ahead of time.

[This is, of course, one approach to solving this per-property behavior, which could also be served by the discussion over at Excluding properties from automatic Codable - #7 by anon31136981 that @ahti links to.]

This isn't unique to Decodable, really — if you have a protocol which requires an init and a type which cannot vend such inits, well, there isn't much you can do.

What is entirely possible is for you to write a wrapper type for the external type you don't own and conform that to Decodable. This is the recommended way to conform others' types to Decodable, too, without conflicting with any Decodable definition they might want to provide later on.

4

@itaiferber unless I misunderstand something, this does not address the use case I am bringing up is not addressed by JSONDecoder.DateDecodingStrategy.custom. I am looking to ensure that a property is consistently encoded and decoded using a specific strategy for all instances of JSONEncoder / JSONDecoder regardless of what date encoding / decoding strategy they are configured with. Without this capability it is necessary to provide a manual implementation and explicitly lower the value to String or Int which is sub-optimal when the target is plist instead of json.

I'm not asking for the synthesized implementation of Decodable to use an init that does not exist. I'm even willing to provide an init with the signature of init(from:)! I just don't want the initializer to be public and hence don't want the type to actually conform to Decodable. What I am asking for is a slightly more general synthesis mechanism for public types that is able to make use of an internal initializer. Perhaps this will have to wait until we have more general synthesis mechanisms available in Swift and use Sourcery or something similar in the meantime...

5

I saw the linked thread but didn't want to go off topic there. I agree with this as a long-term direction but I think the limitations of Date encoding and decoding are worth addressing sooner. I don't think consistent encoding and decoding of types (i.e. independent of the encoder / decoder configuration) is too much to ask from language support.

This is not possible with Date when encoding / decoding to and from json (which is what most of us are using these days). To my knowledge no other types suffer from this limitation. Given the importance of Date and json it seems like to me like this is an important gap to prioritize filling.

6

Ah, got it. You're looking for the opposite of decoding strategies, which is to become exempt from them. Indeed, this is not possible without manually encoding Dates in terms of the underlying representation that you want.

The point of encoding/decoding strategies is to ensure consistency in payloads, which is why we encourage them only if you own all of the types being encoded and are sure you want a consistent format. They do affect types you don't own, which is why their usage should be sparing at best.

Yes, potentially. It's not possible for a type to conform to a protocol without meeting its requirements; if you add a public Decodable conformance, the init(from:) must also be public (which is a general rule in Swift) — but yes, having synthesis be dependent on a protocol conformance introduces this limitation.

On the other hand, though — what are you looking to do with a type that has Decodable synthesis but does not conform to the protocol? It wouldn't be usable within the rest of the Codable infrastructure.

7

To be more clear about your use case — are you vending a type to others which contains a Date that you want encoded/decoded in a specific manner? Or can you give an example?

If it's the former, well, it's up to the person consuming your type to understand that if they apply a date strategy, it will override your Dates — that's part of the API contract, and shouldn't be something you need to enforce.

8

I'd say it's the other way around: Other types don't get the flexibility that Date currently gets. They always get encoded with the equivalent of DateEncodingStrategy.deferredToDate.

I do wish we had more flexibility on a per-containing-type level, but I don't think patching over the specific case of dates with a quick fix is really worth it, since manually implementing encode(to:) is a rather simple workaround (that could even be automated with something like Sourcery if need be).

1 Like
9

Exempt from the strategy configured on the JSONEncoder / JSONDecoder yes, but not only that. A simple exemption would probably imply the behavior of deferredToDate (what else would it do?). Instead, I want to be able to actually specify an encoding / decoding strategy for synthesis to use with in the implementation for a specific property.

This is part of the reason why I believe the current language support is insufficient. Opting out of the "dynamic" encoding strategy for a type and into a "static" encoding strategy for each property is much more work. Language support is absent and requires a manual implementation. Since dates are an extremely primitive type this can push a team towards a custom synthesis solution. This really feels like an area where the language support is letting us down.

Yes. Think of a model designed in accordance with a json API spec.

This is an unnecessarily fragile API contract. I would like to avoid that fragility. Further, as I mentioned earlier, a global date encoding / decoding strategy is not always sufficient. Yes, we can write this boilerplate manually or generate code using Sourcery (or something else). It still feels like the language support is letting us down here. Unfortunately, dates are a source of much essential complexity and encoding / decoding is not an exception to that.

One man's flexibility is another man's limitation. It's a matter of perspective. In this case, there is no way to use synthesis if you want the consistent behavior other types get. There is no way to say "ignore the dynamic decoding strategy". You have to write a manual Decodable conformance for all types that store a Date to do this. Further, the encoding strategies exist for a reason. Simply suppressing the dynamic strategy isn't enough. We need to be able to specify a static strategy to get have a consistent and correct implementation (correct according to a specific API spec).

I assume you mean init(from:) rather than encode(to:). If this were more of an edge case I would agree. However, Date is a very fundamental type which is stored pervasively by model types. In practice it means that if storing a date requires abandoning synthesis then an alternate synthesis solution is necessary. I think it's unfortunate to be pushed away from language support so easily by a type as primitive as Date.

I'm not looking to use synthesis with a type that doesn't conform. It's precisely the opposite. I'm looking to use synthesis with a type that does conform, but has a stored property that does not conform publicly but provides the same interface internally. The motivation for this is to avoid vending any initializers for opaque token types outside of the framework.

As I think about this further, I realize that we may not need to vend Codable conformances for the model types that store them either. Using private types for encoding / decoding might be a better solution. It would require additional boilerplate but we can generate that with a tool like Sourcery.

I wonder how common the requirement to encode / decode a type within the declaring module while not exposing that behavior publicly is...

10

Based on this information, let's see if we can reframe things a bit. A few points:

  • The purpose of encoding/decoding strategies is to allow consumer A of types B, C, and D to express that A knows better than B, C, and D how they should encode specific fields of data. The point here is that B, C, and D can have notions of how they want to encode, but A might actually know better based on their needs — say, B, C, and D are general types that encode Dates and Data and don't particularly care in what format, but A knows that the API they are working with requires a different format
    • This gives consumers of framework/library types a way to say "for my use case, I know that what I really need is a different format, in a way that you couldn't anticipate"
  • These encoding/decoding strategies do not affect how synthesized implementations work — they simply override behavior for all fields of that type, whether or not they come from a synthesized implementation: a Date is affected by the dateEncodingStrategy regardless of whether or not it was part of a synthesized implementation
    • I take your comments about abandoning synthesis not as a misunderstanding of this point, but rather that if you want to enforce a specific format, you have to abandon synthesis in order to encode a Date as a String/Int/Double/what-have-you manually, correct?
    • I further take your language support comments to mean that we should have a way to express "don't really encode this as a Date (which would follow the dateDecodingStrategy) but as some other underlying format that I specify, in a way that follows my spec", correct?
    • If so, this is exactly the type of problem that adaptors would seek to solve — a per-property way of tapping in to encode/decode calls without having to give up full synthesis. I've discussed this in the past (and briefly in the linked thread above), but this isn't something that's been fully fleshed out because it would depend on other language features that aren't there yet
  • The same complaint that you have about Dates being such common, fundamental types is actually the reason we have these strategies in the first place — because JSON doesn't have a native format for dates, someone has to decide how to encode them; sometimes the best person to know how to do that is the person who wrote the type (you), and sometimes it is the person using the type (say, me). This will always be in conflict because there are situations in which either side can be "right" (and in the majority of cases, the person would be me, since I'm actually the one working with the API)

What I see being most in tension here is that this flexibility for consumers in the general case conflicts with your type's assertion that it knows how to encode itself. Some questions about that:

  1. Is your type domain-specific to working with only a single API spec?
  2. Is it conceivable that someone could want to use your type to encode in a format beside JSON, or use that type with their own API?

If the answer to both of these is "yes", then you can clearly see the conflict here: you can't both win out.

  • If this type is domain-specific and intended to work with only one API, then I would say that it is incorrect for a consumer of your type to apply a dateEncodingStrategy/dateDecodingStrategy willy-nilly, not just in the context of your type: their own types sent to that API will simply be wrong. That's not on you — that's on them. ¯\(ツ)/¯ There are many ways of incorrectly encoding, but if the API specifies a required date format, you as the author of types B, C, and D are not the one who needs to control that: it's consumer A who needs to ensure that, not just for your types, but for theirs as well. If they need to apply different formats in different places, they can use the .custom strategy to do that work; that's not something special about your type, but their responsibility regardless
  • If this type is not domain-specific and can be encoded elsewhere, then I would not recommend you hard-code a format, since that defeats the flexibility that your consumers need to write the types out elsewhere

This is something that can be helped with language support in the explicit overriding case, but I recommend taking a look at how your types are intended to be used. Only you can tell us — what is the case here? Can you share the specifics for more context?

Completely separately, about your other issue:

Sorry, I was on the same page here, but perhaps poorly phrased my comment. The intent is to have the following:

public struct Foo : Codable {
    public var name: String
    public var something: MyThingWhichCantPubliclyBeCodable
}

yes? If something could conform to Codable, you'd have no problem, but it's because something can't that you don't get synthesis.

If I've understood you correctly, something can't conform to Codable because you don't want to publicly expose the init(from:), right? If so, this is also not Codable-specific; Swift doesn't allow for private/internal conformances of protocols on public types for good reasons.

This isn't specific to synthesis, either: you couldn't encode something manually regardless, since it doesn't conform to the protocol. All of the generic methods are written in terms of Encodable/Decodable conformance for a reason, and if you can't conform to the protocol, there isn't much you can do with the type.

Looks like you have your solution below, though:

11

I can imagine this being useful in some cases. But in other cases, a type is designed to work with a specific specification and in those use cases, allowing clients this flexibility introduces fragility.

Correct. I think this use case of Codable types designed to precisely implement an API spec is extremely common and Date properties are extremely common. I am arguing that synthesis should be able to handle this use case.

I am suggesting that there should be a way to specify how a specific Date property is lowered. This is what I have called a "static encoding / decoding strategy".

It's awesome to know you're thinking about this. I agree that a more general adapter solution would be awesome! By language features that aren't there, are you referring to property behaviors? I think it would be unfortunate to have to wait for those to be implemented, but it's good to know a solution will come eventually.

This makes sense. The point I am making is that sometimes they type needs to decide and the current design does not support that use case well.

Yes, this is the use case I am discussing.

No, I am talking about model types that mirror a specific API spec.

Understood, but there are two reasons that this is not good enough. First, it's fragile and opens the door to unnecessary mistakes. Second, there are some APIs which use different date formats in different parts of a response. This is not great but it does happen in the wild.

I hope I have answered the questions adequately above. The use case is very general - think any framework designed as a layer over a specific REST API. When a model is tightly coupled to a specific API there is only one correct way to encode a date value for a specific property.

Understood.

Right. Conformance is not necessary for an implementation to call a method on a type if it is visible directly on the concrete type in the scope in which it is called. I wasn't thinking this all the way though - in this case invoking init(from:) directly is not what you need to do!