Codable implementation which omits default values?

I'm looking for a Codable implementation that "skips" values who have their default values. Ideally it'd be JSON, but I'm flexible as to the actual output format. I'm just looking for guidance/examples on these points:

  • How can I check if a value is equal to its default value? For example,
struct MyThing {
    var foo: Int = 42
    var bar: Int = 99
}

let thing = MyThing(bar: 101)

MyEncoder().encode(thing) // -> {"bar": 101} // (note that 'foo' is omitted)

In this example, how would MyEncoder know that the 'foo' property is equal to its default value? For my purposes, I can assume that all structs in the hierarchy I'm serializing implement Codable and Equatable.

  • Also, is there an "extension" point in the existing JSONEncoder/Decoder where I can a) omit default values, and b) instantiate default values when keys are missing? It's unclear to me from reading the source if that's possible. If it's not, does anyone know of an existing Codable Encoder/Decoder implementation which does this kind of thing?

Default values are actually used at initialization time. It's not like foo is an empty property that returns the default until someone sets it. So the encoder doesn't care about defaults because the line above initializes MyThing with foo == 42.

It sounds like you are looking for something similar to what UserDefaults does where you can "register" a default that is used if the key is not already set, but it's not actually present in the storage underneath the defaults itself (it's just present at runtime after the register method call).

Codable doesn't provide any automatic synthesis of this. But you could almost get it by defining a private _foo: Int? that is optional. Then define a public foo: Int getter that is merely _foo ?? 42 and a setter that sets _foo. Then the 42 wouldn't be encoded directly if it was missing.

That double-property definition is a nice solution! But yes, I'm interested to see if there's a way to have extremely simple/readable structs for the data layer, with "plain data."

If I'm understanding you correctly, there is not a way at compile-time to get the initialisation value.

Just writing this now, I'm realizing I might be able to accomplish my goal with something closer to this:

func encode<T>(thing: T) {
    let defaultThing = T()
    if thing == defaultThing {
        // actually encode `thing`
    }
}

Assuming my structs are Equatable and "init"-able that might work.

...but actually it would have to be something more like

func encode<T>(thing: T, keyPath: KeyPath) {
    let defaultThing = T()
    if thing[keyPath] != defaultThing[keyPath] {
        writeToSerializationStream(thing[keyPath])
    }
}

I'll see what I can come up with...

Maybe property wrappers could help? I'm not sure the "simple/readable structs" goal is compatible with the way Codable works today. Except maybe you can make the simple/readable structs be the type everyone uses in the process but then it gets wrapped by another type that is actually Codable. Then you can hide all the ugliness.

After some more investigation, I see now that this goal might be even more difficult than I realized, since you cannot write values with Mirror.

something like below? although:

  • lots of boilerplate :(
  • requires N custom types for N types for which you want to specify default values. 5 lines per type but still it feels too heavy.
  • encodes an absent value with "{}" in json (ideally the key should be absent)
  • be careful with floating points EQ, some compare with tolerance would be needed.

i won't use this personally, Int? is so much simpler. btw, the struct with optional is still POD.

Summary
import Foundation

protocol HasDefaultValue: Codable {
    associatedtype T: Codable & Equatable
    static var defaultValue: T { get }
    var value: T { get }
    init(value: T)
}

extension HasDefaultValue {
    func encode(to encoder: Encoder) throws {
        var container = encoder.singleValueContainer()
        if value != Self.defaultValue {
            try container.encode(value)
        }
    }
    
    init(from decoder: Decoder) throws {
        let container = try decoder.singleValueContainer()
        guard let v = try? container.decode(T.self) else {
            self.init(value: Self.defaultValue)
            return
        }
        self.init(value: v)
    }
}

extension Encodable {
    var jsonData: Data? {
        try? JSONEncoder().encode(self)
    }
}

extension Decodable {
    init?(jsonData: Data) {
        guard let v = try? JSONDecoder().decode(Self.self, from: jsonData) else {
            return nil
        }
        self = v
    }
}

extension Data {
    var string: String? {
        String(data: self, encoding: .utf8)
    }
}

// MARK: test area

struct Int_1234567890: HasDefaultValue, Equatable {
    static var defaultValue: Int { 1234567890 }
    var value: Int
}

struct Int_2718281828: HasDefaultValue, Equatable {
    static var defaultValue: Int { 2718281828 }
    var value: Int
}

struct Foo: Codable, Equatable {
    var x: Int
    var y: Int_1234567890
    var z: Int_2718281828
}

func test(_ a: Foo) {
    let data = a.jsonData!
    let str = data.string!
    print(str)
    let b = Foo(jsonData: data)
    assert(a == b)
}

func main() {
    test(Foo(x: 1, y: .init(value: 123), z: .init(value: 456)))
        // {"x":1,"y":123,"z":456}
    test(Foo(x: 1, y: .init(value: 1234567890), z: .init(value: 555555555)))
        // {"x":1,"y":{},"z":555555555}
    test(Foo(x: 1, y: .init(value: 1234567890), z: .init(value: 2718281828)))
        // {"x":1,"y":{},"z":{}}
    print()
}

main()

prints:

{"x":1,"y":123,"z":456}
{"x":1,"y":{},"z":555555555}
{"x":1,"y":{},"z":{}}

Yeah for me @tera this implementation is unusable because the type can only have one default value. I'm looking to "look up" the normal default value ala var thing = 42. I think the only way to accomplish this is to enforce that the parent struct has a "default" and argument-less init().

Why do you want to omit default values when encoding objects?

I want to be able to use Codable but add fields to my data objects later.

I think the easiest would be to have the struct with optional fields for the purposes of decoding. And if you don't to want to pollute the main code base to deal with optionality you can have two parallel structs: the "main" struct with non optional fields and a "decoding" struct with the same fields marked optional; once you've decoded the data convert the decoding struct into the main struct applying default values for missing fields.

Yes, those are the alternatives I have tried. I was looking for a way to do this without optionals everywhere and without having to have a parallel set of structs.

(the best i can think of right now is this:)

struct MyThing {
var foo: Int! = 42
var bar: Int! = 99
}

nope, this doesn't work, the missing keys are getting reset to nil :(

we can ask apple to have a new opt-in strategy for JSONDecoder to not override missing fields that have default values. (see the other related thread)

Why does this necesitate omitting default values during encoding? I don't understand the connection here at all. Give an example.

I assume you're referring to the Swift types that conform to Codable. Correct me if you're wrong.

Furthermore, if you omit default values during encoding, you can no longer use the default implementation of Decodable to decode your data: Even if a property has an initialization expression (e.g., var foo: Int = 42) this property must still be present in the JSON in order for it to be decoded successfully (unless it's an Optional type). Why would you want to give up this behavior?

Here's an implementation of encode(to:) that omits default values. It's far from idiomatic, but set that aside for now. How is this useful?

struct MyThing: Codable {
    var foo: Int = 42
    var bar: Int = 99
    
    func encode(to encoder: Encoder) throws {
        let defaultInstance = MyThing()
        
        var container = encoder.container(keyedBy: CodingKeys.self)
        
        if self.foo != defaultInstance.foo {
            try container.encode(self.foo, forKey: .foo)
        }
        if self.bar != defaultInstance.bar {
            try container.encode(self.bar, forKey: .bar)
        }
    }

}

Notice that, if you encode an instance of MyThing with default value(s), the synthesized version of init(from:) will fail to decode the data:

let thing = MyThing()
let jsonData = try JSONEncoder().encode(thing)
let dataString = String(data: jsonData, encoding: .utf8)!
print("encoded data:", dataString, terminator: "\n\n")

let decodedThing = try JSONDecoder().decode(MyThing.self, from: jsonData)

Output:

encoded data: {}

Playground execution terminated: An error was thrown and was not caught:
▿ DecodingError
  ▿ keyNotFound : 2 elements
    - .0 : CodingKeys(stringValue: "foo", intValue: nil)
    ▿ .1 : Context
      - codingPath : 0 elements
      - debugDescription : "No value associated with key CodingKeys(stringValue: \"foo\", intValue: nil) (\"foo\")."
      - underlyingError : nil

Omitting default values during encoding is secondary on my priority list here, but it was a clean way to keep the serialized form small and to make the serialized form a "delta" on the original template defined by your default values.

I understand that the behavior of Codable does not work in the way I'm asking.

But to reiterate my original question: I was hoping to do boiler-plate-free serialization in a future-proof way, without having all the code that uses my structs have to check if things are optional. I want adding a new field to my structs to be as easy as adding a line

var newField = 42

I don't necessarily need to use Codable, but I was drawn to it because a) the compiler can synthesize the encode and decode methods for you, as long as your fields are also Codable and b) I don't have to write a JSON decoder/encoder if I use it, and c) you cannot write values with Mirror, so some kind of compiler synthesis is the only way to accomplish what I'm after (instantiating my structs from data without boilerplate serialization code).

I suspect you could make a custom template for Sourcery to generate Codable implementations that behave how you want, but I haven't tried it myself.

This is my first attempt at writing a Sourcery template. There are probably a lot of uncovered edge cases, but here's a starting point. With this template:

{% for type in types.implementing.DifferenceCodable %}
extension {{ type.name }}: Codable {
  private enum CodingKeys: String, CodingKey {
    {% for var in type.variables %}
    case {{ var.name }}
    {% endfor %}
  }

  {{ type.accessLevel }} init(from decoder: Decoder) throws {
    let container = try decoder.container(keyedBy: CodingKeys.self)

    {% for var in type.variables %}
    {% if var.defaultValue != nil %}
    self.{{ var.name }} = try container.decodeIfPresent({{ var.typeName }}.self, forKey: .{{ var.name }}) ?? {{ var.defaultValue }}
    {% else %}
    self.{{ var.name }} = try container.decode({{ var.typeName }}.self, forKey: .{{ var.name }})
    {% endif %}
    {% endfor %}
  }

  {{ type.accessLevel }} func encode(to encoder: Encoder) throws {
    var container = encoder.container(keyedBy: CodingKeys.self)

    {% for var in type.variables %}
    {% if var.defaultValue != nil %}
    if {{ var.name }} != {{ var.defaultValue }} {
      try container.encode({{ var.name }}, forKey: .{{ var.name }})
    }
    {% else %}
    try container.encode({{ var.name }}, forKey: .{{ var.name }})
    {% endif %}
    {% endfor %}
  }
}
{% endfor %}

and this Swift code as input:

protocol DifferenceCodable {}

struct MyThing: DifferenceCodable {
    var foo: Int = 42
    var bar: Int = 99
    var baz: Int
}

Sourcery will generate this Codable conformance which should behave how you're requesting:

// Generated using Sourcery 1.6.0 — https://github.com/krzysztofzablocki/Sourcery
// DO NOT EDIT

extension MyThing: Codable {
  private enum CodingKeys: String, CodingKey {
    case foo
    case bar
    case baz
  }

  internal init(from decoder: Decoder) throws {
    let container = try decoder.container(keyedBy: CodingKeys.self)

    self.foo = try container.decodeIfPresent(Int.self, forKey: .foo) ?? 42
    self.bar = try container.decodeIfPresent(Int.self, forKey: .bar) ?? 99
    self.baz = try container.decode(Int.self, forKey: .baz)
  }

  internal func encode(to encoder: Encoder) throws {
    var container = encoder.container(keyedBy: CodingKeys.self)

    if foo != 42 {
      try container.encode(foo, forKey: .foo)
    }
    if bar != 99 {
      try container.encode(bar, forKey: .bar)
    }
    try container.encode(baz, forKey: .baz)
  }
}

This is exactly what I was hoping to find, thank you!

1 Like

Interesting to note that SwiftProtobuf behaves this way by default for both protobuf binary encoding and for JSON encoding. That design was motivated largely by the desire to minimize the size of the data on the network (default values are "known" by the recipient and therefore don't need to be transferred).

1 Like
Terms of Service

Privacy Policy

Cookie Policy