Codable Improvements and Refinements

[Chris_Lattner3 (Chris Lattner)]

@hartbit

We could start with a dynamic solution (ease of use for developers), and wait until we have the macro system to use it statically and only then rewrite Hashable, Equatable, and Codable using them.

Yes, that approach would make sense.

-Chris

[Karl]

So, I think we all want user-defined attributes. There's lots of precedent in other languages and they can be used for some really cool things. On the other hand, if we really want arbitrary, user-defined attributes, we'd need some kind of reflection API to access them.

Personally, I'm not sure that coding-keys belong as part of the model type, particularly when you have some kind of external API and multiple serialisation formats to contend with. I think this is really an Encoder/Decoder-level detail.

I think the best thing would be to create some kind of Schema type which represents how to encode/decode a particular type in a particular format. For example, if you're encoding in to some format which has a native representation for Range, you could choose that schema in place of the JSON-compatible two-unlabelled-values approach.

That's the "Enhanced Contextual Overriding of Encoded Representations" option, basically.

[rovertsnikle (Trevor Elkins)]

1. Can only speak for the annotation-heavy approach here, none of the suggestions I've seen would sacrifice type safety. I'm assuming the implementation would require importing a static annotation type similar to how other languages handle it. See GSON's @SerializedName annotation.

2. as the two options is extremely clear and easy to teach

   Easy according to you, but that has not been my experience browsing StackOverflow and /r/Swift compared to other approaches. Part of the problem is the locality of the init(from:) and CodingKey abstractions. Users are working at the property-level and suddenly have to shift to implement an initializer or entirely new CodingKey type. A lot of new developers have a hard time making the mental connection.

   From a mile-high view, annotations can be thought of like the decorator pattern, which IMO is a much cleaner and easier abstraction to understand. It's also closely coupled to the actual property that you're working with which reduces the mental load.

Similarly, we don't need bespoke syntax for defining an error domain just to write 20 characters less.

I think that's oversimplifying things a bit. And even so, I'd say it's in the best interest of every language/library to monitor/eliminate boiler-plate code for the sake of self-preservation.

[soroush (Soroush Khanlou)]

While all the changes mentioned here would be great, I want to throw out something that hasn't been brought up yet.

I'd like for enums to get free Codable synthesis when they're not RawRepresentable. Right now, if you have an enum that isn't RawRepresentable (usually achieved with a String or Int), you have to write the full encoding and decoding functions yourself, which can be very painful. This is even worse when you have associated values on your enum. If you associated values, you can't get the Codable method synthesis at all.

Not allowing non-RawRepresentable enums to get Codable synthesis is one of those technically-correct-yet-practically-frustrating things in Swift that I'd like to see go away. I think we could choose some decent defaults that, even if they wouldn't work great for JSON decoding from web services, would be excellent for simple stuff like writing and object to disk and reading it back out again.

[rbishop (Russ Bishop)]

I wonder... could this be done in the form of compile-time evaluation? I haven’t given it enough thought but it strikes me as an interesting avenue to explore. If we passed the AST in and out it gives us “macros” but without token pasting.

[ahti (Lukas Stabe 🙃)]

There has been some talk about that (and why it's kind of difficult) here:

Automatic Codable conformance for enums with associated values that themselves conform to Codable

At least as of Swift 4.1, an enum with one or more cases with associated values cannot participate in synthesized Codable conformance. For example, the following will yield compiler errors: enum MyOtherEnum: Codable { // errors: does not conform to Encodable/Decodable case foo case bar(label: Int) } This is true even if all the associated values in said enum themselves conform to Codable. In a similar fashion as the synthesized Hashable / Equatable implementations for enums with associ…

Imo this is one of the limitations for which a solution would really benefit from custom attributes to allow the user to express how exactly their enum cases and associated values should be en/decoded.

[anandabits (Matthew Johnson)]

@ahti

Imo this is one of the limitations for which a solution would really benefit from custom attributes to allow the user to express how exactly their enum cases and associated values should be en/decoded.

In general this is true. My team uses a Sourcery template which supports a large number of sum type encodings. This is unfortunately necessary as long as the industry uses serialization formats that do not specify a canonical way of encoding sum types and often cloud languages that also do not support sum types. API developers find many different ways of expressing data that is really best modeled as a sum type.

While that is the case, @soroush specifically says:

I think we could choose some decent defaults that, even if they wouldn't work great for JSON decoding from web services, would be excellent for simple stuff like writing and object to disk and reading it back out again.

The experience my team has gained with a plethora of sum type encodings has lead us to prefer one encoding over the rest. It's hard to say whether the community could agree on a canonical / default, but if we can maybe it's ok to support synthesizing Codable for enums with associated values in the canonical format. Annotations could be added to support more formats in the future.

[soroush (Soroush Khanlou)]

@anandabits

It's hard to say whether the community could agree on a canonical / default, but if we can maybe it's ok to support synthesizing Codable for enums with associated values in the canonical format.

Even if we could get the Swift community to agree, we'd have to convince all of our api engineers, which would be even more impossible, haha.

I also want to mention Swift on the server projects, where using Codable on both sides means that the exact interchange format doesn't even really matter, as long as we do pick a default.

[mergesort (Joe Fabisevich)]

I wanted to chime in with an option that I'm not sure has been mentioned, maybe rightfully so. I think the compiler synthesizing the encoder and decoder currently is fantastic, but there are times when you need to deviate only one or two keys/properties that operate in a funky way, and that's when you have to add a lot of boilerplate. I'd like to propose an alternative option that I hope is viable, though I haven't dove in enough to understand how it would be accomplished.

---

Creating a closure-based API that allows me to customize some values. I would like the concept from KeyDecodingStrategy and apply it to the actual type, assuming the default implementation if no overwrite is specified.

[Ponyboy47 (Jacob Williams)]

@mergesort

there are times when you need to deviate only one or two keys/properties that operate in a funky way, and that's when you have to add a lot of boilerplate.

This.

There have been so many times when I have a struct where all the properties can be synthesized except one (something like a tuple or enum without a RawValue). It's super annoying to have to write the entire Codable conformance for everything just because of a single variable.

[xwu (Xiaodi Wu)]

@Ponyboy47

This.

There have been so many times when I have a struct where all the properties can be synthesized except one (something like a tuple or enum without a RawValue). It's super annoying to have to write the entire Codable conformance for everything just because of a single variable.

This thread is about soliciting and comparing ideas to address precisely this issue. How do you feel about the various solutions presented?

[Ponyboy47 (Jacob Williams)]

@xwu Thank you for keeping this thread focused and I apologize for my unproductive reply in the thread

---

My personal views reflect those of several others stated upthread that a compiler-centric solution would be optimal over the few library-centric solutions proposed. I'm a lazy person and I'd rather not have to do more work to get my types conforming to Codable. I never really liked having to build an entire CodingKeys enum to get Codable conformance so I agree that using annotations would be a good solution that would feel like less boilerplate, plus with the pitch for user defined attributes I feel that this is tipping towards attribute annotations anyways.

Some have posited that the CodingKeys enum should be replaced and I am inclined to agree, though I also see the benefits that the enum has over simple annotations. Although I do feel that for most use cases the attribute annotations would be adequate, perhaps it would be best to allow one or the other instead of wholly removing the enum with something else. Then the developer can pick the right tool for the job and not all devs are punished with a high amount of boilerplate for simple customizations.

[anandabits (Matthew Johnson)]

@Ponyboy47

Some have posited that the CodingKeys enum should be replaced and I am inclined to agree, though I also see the benefits that the enum has over simple annotations. Although I do feel that for most use cases the attribute annotations would be adequate, perhaps it would be best to allow one or the other instead of wholly removing the enum with something else. Then the developer can pick the right tool for the job and not all devs are punished with a high amount of boilerplate for simple customizations.

I think the CodingKeys enum is useful (I use it in test JSON for example). It definitely isn't actively harmful, so I think it's here to stay. But I agree that it is highly desirable to have an annotation-based solution used by the compiler's synthesized CodingKeys enum.