# Serialization in Swift **URL:** \ **Category:** Discussion\ **Created:** [March 18, 2021, 12:46am UTC](https://forums.swift.org/t/serialization-in-swift/46641 "2021-03-18T00:46:13Z")\ **Posts on this page:** 20\ **Page:** 2
**Author:** ![cal](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/cal/32/35557_2.png) [@cal](https://forums.swift.org/u/cal)\ **Post date:** [March 19, 2021, 7:41pm UTC](https://forums.swift.org/t/serialization-in-swift/46641/22 "2021-03-19T19:41:31Z")
Is the core team interested in reviewing those sorts of proposals that build on top of existing Codable infrastructure? I've had a [proposal PR](https://github.com/apple/swift-evolution/pull/1133) (with a [compiler implementation](https://github.com/apple/swift/pull/30570)) up since March 2020 and never heard from the core team about it. ---
**Author:** ![phoneyDev](https://avatars.discourse-cdn.com/v4/letter/p/49beb7/32.png) [@phoneyDev](https://forums.swift.org/u/phoneyDev)\ **Post date:** [March 19, 2021, 11:01pm UTC](https://forums.swift.org/t/serialization-in-swift/46641/23 "2021-03-19T23:01:47Z")
The great thing about `Codable` for simple types is that no code is required. But the slightest need for something not simple requires a lot of boilerplate code. Being able to accomplish less simple things with no code should be a goal. Here are a few pain points for me: Dates! The default of `Date` decoding to seconds isn't sensible for JSON. The lack of support out of the box for fractional seconds has been a problem for me. It is possible to have an internet date parser that accepts all forms of international dates. I use SwiftDate for this but it requires custom code. Default values. I guess there are property wrappers for this but I haven't investigated them. It should be possible to supply default values directly to the decode() methods. It should also be possible to provide the default values in a more declarative way, so no init method. Future proofing enums. I commonly have simple enums, either `String` or `Int` based, where the server API adds new cases from time to time. Any given version of my app only knows about the current cases. The only way I deal with this is try/catch with decoding in the try block and assigning a default value in the catch block. If I forget to add this then parsing fails if there's a new case. ---
**Author:** ![Zhu\_Shengqi](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/zhu_shengqi/32/1199_2.png) [@Zhu\_Shengqi](https://forums.swift.org/u/Zhu_Shengqi)\ **Post date:** [March 20, 2021, 12:44am UTC](https://forums.swift.org/t/serialization-in-swift/46641/24 "2021-03-20T00:44:13Z")
> [@tomerd](#): > > I'v seen this one (and similar) solved with property wrappers. what do folks think about that solution? The syntax that property wrappers provide is really nice. One minor issue is that with property wrappers around the underlying data types, it's difficult to do more abstraction over a model type (or I have to put the property wrapper into common model protocols). Thanks for the direction! > [@tomerd](#): > > something like [https://forums.swift.org/t/add-support-for-encoding-and-decoding-nested-json-keys/](https://forums.swift.org/t/add-support-for-encoding-and-decoding-nested-json-keys/)? One question about the existing (and the proposal you mentioned) encoding/decoding architecture is that why we need the `CodingKeys` stuff (put the implementation limitation aside)? I think it's quite natural that when I define a property like `let date: Date?` inside a model type, I just want to extract the `date` field from a JSON object; and if I define the property as (ignore how we implement it) `@field("outer.inner.date") let date: Date?`, I want we go into two nested structure of a JSON object to find the `date` field. With all the encoding/decoding logic defined alongside a single property itself, we could have several benefits: 1. It's easier for other engineers to read/understand the model type definition. Eyes won't need to be targeted at different areas of a source file interchangeably. 2. It's also easier to add/remove/modify the fields, since we don't need to maintain the extra `CodingKeys` or `NestedKeyDecodingStrategy` stuff. 3. If we want a same property in another model type, we just need to copy the single definition of the property into the new model type. It's that simple! To some point I agree with @ktoso . Maybe if there is a powerful meta programming interface (like the macro in C or the `gyb` file in the Swift repo, but I prefer a native macro system in Swift), I can create a JSON encoding/decoding library myself (with the features I mentioned earlier). Because the scenarios and cases may seem different a lot (in this thread some people mentioned the performance issue, while I focused more on logic coherence around single properties), a general macro system may be a good direction. ---
**Author:** ![Optional](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/optional/32/12543_2.png) [@Optional](https://forums.swift.org/u/Optional)\ **Post date:** [March 20, 2021, 1:08am UTC](https://forums.swift.org/t/serialization-in-swift/46641/25 "2021-03-20T01:08:40Z")
> [@rnantes](#): > > - 如果在序列化源中找不到属性，则可以设置属性的默认值 👍👍 ---
**Author:** ![rauhul](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/rauhul/32/35207_2.png) [@rauhul](https://forums.swift.org/u/rauhul)\ **Post date:** [March 20, 2021, 2:25am UTC](https://forums.swift.org/t/serialization-in-swift/46641/26 "2021-03-20T02:25:03Z")
+1 to the idea of a general meta programming system. I would love the idea to walk an AST-like structure and synthesize custom type initializers. Essentially lifting the work the compiler does for Codable into libraries. This combined with a compile time asserts would allow libraries to design high performance serializers + deserializers with compile time error checking. For example, it could allow swift-argument-parser to move it's runtime checks for detecting duplicate commands into compile time errors. ---
**Author:** ![anandabits](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/anandabits/32/355_2.png) [@anandabits](https://forums.swift.org/u/anandabits)\ **Post date:** [March 20, 2021, 2:48am UTC](https://forums.swift.org/t/serialization-in-swift/46641/27 "2021-03-20T02:48:58Z")
> [@rauhul](#): > > +1 to the idea of a general meta programming system. +1 from me as well assuming there is good tooling support. I have found it invaluable in working with Sourcery to be able to see the code that gets generated. Beyond this, we commit generated code to our repo which has the benefit of providing a free regression test when working on the template. I don’t know exactly how to best provide those benefits with a built-in language feature but it’s something worth considering. ---
**Author:** ![Doug](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/doug/32/7679_2.png) [@Doug](https://forums.swift.org/u/Doug)\ **Post date:** [March 21, 2021, 11:05am UTC](https://forums.swift.org/t/serialization-in-swift/46641/28 "2021-03-21T11:05:14Z")
From the perspective of making Codable more flexible, I'd love a way to work with data that contains typed objects. For example, given this: > **JSON** > > ```swift > [ > { > "name": "Doug", > "type": "person", > "date": 1616323199446 > }, > { > "url": "https://swift.org/assets/images/swift.svg", > "type": "resource", > "date": 1616323199000 > } > ] > > ``` I'd like to be able to define something like this: > **Swift Types** > > ```swift > protocol Response { > var date: Date { get } > } > > struct Person: Response, Codable { > let name: String > let date: Date > } > > struct Resource: Response, Codable { > let url: URL > let date: Date > } > > ``` And somehow decode it as an array of `Response`. ---
**Author:** ![nidegen](https://avatars.discourse-cdn.com/v4/letter/n/7feea3/32.png) [@nidegen](https://forums.swift.org/u/nidegen)\ **Post date:** [March 21, 2021, 2:50pm UTC](https://forums.swift.org/t/serialization-in-swift/46641/29 "2021-03-21T14:50:04Z")
As already stated in the thread [KeyPaths and Codable](https://forums.swift.org/t/keypaths-and-codable/13945), accessing the coding key from a keypath is the biggest need I have currently:) ---
**Author:** ![DevAndArtist](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/devandartist/32/287_2.png) [@DevAndArtist](https://forums.swift.org/u/DevAndArtist)\ **Post date:** [March 21, 2021, 3:00pm UTC](https://forums.swift.org/t/serialization-in-swift/46641/30 "2021-03-21T15:00:48Z")
I'll keep the feedback short, one of our needs is the ability to decode and encode a binary layout from and into custom swift types. It would be a huge benefit if this was possible via `Codable` or some kind of an extension of that feature as otherwise this requires a lot of potentially error prone boilerplate code. ---
**Author:** ![hassila](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/hassila/32/22647_2.png) [@hassila](https://forums.swift.org/u/hassila)\ **Post date:** [March 21, 2021, 7:41pm UTC](https://forums.swift.org/t/serialization-in-swift/46641/31 "2021-03-21T19:41:50Z")
I don’t know enough about codable to comment on that specifically, but I do have a decent amount of experience with systems with high performance requirements for serialization / deserialization (automatic trading systems). FWIW and just in the off chance that not everyone have seen them and without fully groking the full requirements for this effort, I just wanted to point out [https://capnproto.org/](https://capnproto.org/) and [GitHub - google/flatbuffers: FlatBuffers: Memory Efficient Serialization Library](https://github.com/google/flatbuffers). I realize these are not anywhere near flexible or feature complete enough, but I wanted to point them out as if performance is a real concern aiming for zero serialization/deserialization should at least be considered. ---
**Author:** ![hassila](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/hassila/32/22647_2.png) [@hassila](https://forums.swift.org/u/hassila)\ **Post date:** [March 21, 2021, 8:20pm UTC](https://forums.swift.org/t/serialization-in-swift/46641/32 "2021-03-21T20:20:34Z")
In my experience that is a quite common issue and tooling / support for doing that more easily and efficient would be great. ---
**Author:** ![drexin](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/drexin/32/7573_2.png) [@drexin](https://forums.swift.org/u/drexin)\ **Post date:** [March 22, 2021, 6:09am UTC](https://forums.swift.org/t/serialization-in-swift/46641/33 "2021-03-22T06:09:45Z")
I just wanted to mention that this discussion is not about specific implementations of serialization formats, but about the general support for serialization in Swift and how we can improve it to make it easier for developers of serialization libraries to develop flexible and efficient implementations and also for users of those libraries to more efficiently use them. ---
**Author:** ![ktoso](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/ktoso/32/44526_2.png) [@ktoso](https://forums.swift.org/u/ktoso)\ **Post date:** [March 22, 2021, 8:01am UTC](https://forums.swift.org/t/serialization-in-swift/46641/34 "2021-03-22T08:01:40Z")
Might as well drop some talks for inspiration. A mechanism to Scala 3's inline macros / derivation could be a very scalable mechanism, though I'm not sure how well it would work in Swift. There was a fantastic walkthrough through the feature recently here: [Generic Derivation is the New Reflection by Alexander Ioffe](https://www.youtube.com/watch?v=E9L1-rkYPng) Most notably, note that with such mechanism it is possible to achieve 1:1 as efficient code as manually hand-written code, as well as it is much nicer than what we currently do in the compiler with manually weaving ASTs and SIL together, which one cannot print easily and there's no real type system to help getting it right on that level -- just that things fail later on in the pipeline. It is a bit nasty to write those but not too bad, and definitely most powerful. For simple derivation one might want to have quasiquote mechanisms, perhaps we could have both...? ---
**Author:** ![hassila](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/hassila/32/22647_2.png) [@hassila](https://forums.swift.org/u/hassila)\ **Post date:** [March 22, 2021, 9:02am UTC](https://forums.swift.org/t/serialization-in-swift/46641/35 "2021-03-22T09:02:08Z")
Ok, understood - assuming this was partly directed to my post as I referenced two specific implementations, I just wanted to clarify that my point was more that a good general support framework for serialisation (and something that allows for efficient implementation of such a serialisation library) should consider that being able to build zero-ser/des implementations would be a good thing and wanted to provide those pointers as some food-for-though input as the original opening question was fairly open "The core team would like to initiate this conversation with the community to gather requirements and discuss future designs and their trade-offs". But if that isn't interesting or out of scope I'll drop it and apologies for the noise in that case - it is just that in my experience the pendulum of serialisation solutions often seems to swing far into the flexibility and expressiveness spectrum and become unusable for most high performance needs as it is never really seriously considered from the beginning. ---
**Author:** ![tevelee](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/tevelee/32/9775_2.png) [@tevelee](https://forums.swift.org/u/tevelee)\ **Post date:** [March 22, 2021, 9:55am UTC](https://forums.swift.org/t/serialization-in-swift/46641/36 "2021-03-22T09:55:33Z")
The concept of generic derivation is absolutely genius! Based on this walkthrough, it requires a few missing links we don't have in Swift: meta-programming capabilities, compile time evaluated macros, tuple composition/decomposition operators, etc. But the result is as efficient as manually written code, and it's flexible enough to achieve any type of serialization needs. Furthermore, these concepts can all be reused for many other use-cases where we previously required dedicated compiler features (e.g. automatic protocol conformances). I would love to take some of those responsibilities out from the compiler, for the benefit of both the compiler and language pro-users. ---
**Author:** ![Anachron](https://avatars.discourse-cdn.com/v4/letter/a/bc8723/32.png) [@Anachron](https://forums.swift.org/u/Anachron)\ **Post date:** [March 22, 2021, 10:37am UTC](https://forums.swift.org/t/serialization-in-swift/46641/37 "2021-03-22T10:37:30Z")
would it be possible to go from ```swift init(from decoder: Decoder) ``` and ```swift encode(to encoder: Encoder) ``` to ```swift init(from decoder: D) ``` and ```swift encode(to encoder) ``` ? ---
**Author:** ![Morten\_Bek\_Ditlevsen](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/morten_bek_ditlevsen/32/28133_2.png) [@Morten\_Bek\_Ditlevsen](https://forums.swift.org/u/Morten_Bek_Ditlevsen)\ **Post date:** [March 22, 2021, 1:52pm UTC](https://forums.swift.org/t/serialization-in-swift/46641/38 "2021-03-22T13:52:08Z")
My two cents: I use `Codable` in a json context, and I have always loved that feature of Swift. 🙂 For most of the issues that I am facing, I think there are evolutionary steps to make the current system better. 1. Enum coding: with [SE-295](https://github.com/apple/swift-evolution/blob/main/proposals/0295-codable-synthesis-for-enums-with-associated-values.md) I will get to reduce a fair bit of boilerplate code. And if any subsequent proposals add more configurability that will allow me to discard even more boilerplate. 2. My own pet peeve: using non-String keys for `Codable` Dictionaries. This is fixable as well and I hope that the Core Team has bandwidth to put my proposal [https://github.com/apple/swift-evolution/pull/1288](https://github.com/apple/swift-evolution/pull/1288) through review in the near future. 🙂 3. Key conversion between camel case and snake case is currently two conversions which results in hard to explain requirements for the keys. I hope that the general direction of [https://github.com/apple/swift/pull/14039](https://github.com/apple/swift/pull/14039) could become a reality some day. It appears that the pr above was first abandoned and later closed due to maintenance, but the issue is still there - and worth solving. These are the main issues I have, and they all have solutions. That said, both performance and meta programming features would of course both be awesome to have. 🙂 ---
**Author:** ![woolsweater](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/woolsweater/32/2402_2.png) [@woolsweater](https://forums.swift.org/u/woolsweater)\ **Post date:** [March 22, 2021, 6:55pm UTC](https://forums.swift.org/t/serialization-in-swift/46641/39 "2021-03-22T18:55:38Z")
> [@drexin](#): > > I just wanted to mention that this discussion is not about specific implementations of serialization formats, but about the general support for serialization in Swift and how we can improve it to make it easier for developers of serialization libraries to develop flexible and efficient implementations and also for users of those libraries to more efficiently use them. Specific case studies seem like valuable, if not crucial, input, for deciding directions to take to achieve that goal. How better to "gather requirements and discuss future designs and their trade-offs" (from the opening post) than to look at the things people are _actually_ having trouble doing with the current system? ---
**Author:** ![David\_Sweeris](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/david_sweeris/32/31302_2.png) [@David\_Sweeris](https://forums.swift.org/u/David_Sweeris)\ **Post date:** [March 22, 2021, 7:32pm UTC](https://forums.swift.org/t/serialization-in-swift/46641/40 "2021-03-22T19:32:18Z")
> [@tomerd](#): > > The core team would like to initiate this conversation with the community to gather requirements and discuss future designs and their trade-offs, ranging from improvements to Codable to additional tools and APIs. Our goal is to use the information gathered in this thread to inform future proposals that will bring a more complete serialization solution to Swift. I’m kinda curious if the core team is thinking more towards something similar to the existing `Codable` protocols or if they’re looking for a more general code synthesis solution? ---
**Author:** ![George](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/george/32/37757_2.png) [@George](https://forums.swift.org/u/George)\ **Post date:** [March 22, 2021, 7:58pm UTC](https://forums.swift.org/t/serialization-in-swift/46641/41 "2021-03-22T19:58:19Z")
While we are studying use cases, I don't think anyone has highlighted [GitHub - apple/swift-argument-parser: Straightforward, type-safe argument parsing for Swift](https://github.com/apple/swift-argument-parser). Argument Parser currently utilizes `Decodable` to initialize a parseable command, but the current state of Codable forces this to be pretty hand-wavey and requires quite a few strategic [fatalErrors](https://github.com/apple/swift-argument-parser/blob/380eb7237897b2b56728630460fd0d9b1efd4eca/Sources/ArgumentParser/Parsing/ArgumentDecoder.swift#L94). I think this sort of use case should sit front-and-center in a debate over the future of Codable (and a static structural reflection approach will address this use case nicely). [Previous page](https://forums.swift.org/t/serialization-in-swift/46641.md?page=1) [Next page](https://forums.swift.org/t/serialization-in-swift/46641.md?page=3)