[Request for Feedback] Providing defaults for <Codable> reading and writing.


(William Shipley) #1

Automatic substitution / removal of default values is very useful when reading or writing a file, respectively, and should be supported by the <Codable> family of protocols and objects:

• When reading, swapping in a default value for missing or corrupted values makes it so hand-created or third-party-created files don’t have to write every single value to make a valid file, and allows slightly corrupted files to auto-repair (or get close, and let the user fix up any data that needs it after) rather than completely fail to load. (Repairing on read creates a virtuous cycle with user-created files, as the user will get _some_ feedback on her input even if she’s messed up, for example, the type of one of the properties.)

• When writing, providing a default value allows the container to skip keys that don’t contain useful information. This can dramatically reduce file sizes, but I think its other advantages are bigger wins: just like having less source code makes a program easier to debug, having less “data code” makes files easier to work with in every way — they’re easier to see differences in, easier to determine corruption in, easier to edit by hand, and easier to learn from.

My first pass attempt at adding defaults to Codable looks like this:

public class ReferencePieceFromModel : Codable {

    // MARK: properties
    public let name: String = ""
    public let styles: [String] = []

    // MARK: <Codable>
    public required init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)

        self.name = container.decode(String.self, forKey: .name, defaults: type(of: self).defaultsByCodingKey)
        self.styles = container.decode([String].self, forKey: .styles, defaults: type(of: self).defaultsByCodingKey)
    }
    public func encode(to encoder: Encoder) throws {
        var container = encoder.container(keyedBy: CodingKeys.self)

        try container.encode(name, forKey: .name, defaults: type(of: self).defaultsByCodingKey)
        try container.encode(styles, forKey: .styles, defaults: type(of: self).defaultsByCodingKey)
    }
    private static let defaultsByCodingKey: [CodingKeys : Any] = [
        .name : "",
        .styles : [String]()
    ]

    // MARK: private
    private enum CodingKeys : String, CodingKey {
        case name
        case styles
    }
}

With just a couple additions to the Swift libraries:

extension KeyedDecodingContainer where Key : Hashable {
    func decode<T>(_ type: T.Type, forKey key: Key, defaults: [Key : Any]) -> T where T : Decodable {
        if let typedValueOptional = try? decodeIfPresent(T.self, forKey: key), let typedValue = typedValueOptional {
            return typedValue
        } else {
            return defaults[key] as! T
        }
    }
}

extension KeyedEncodingContainer where Key : Hashable {
    mutating func encode<T>(_ value: T, forKey key: Key, defaults: [Key : Any]) throws where T : Encodable & Equatable {
        if value != (defaults[key] as! T) {
            try encode(value, forKey: key)
        }
    }

    mutating func encode<T>(_ value: [T], forKey key: Key, defaults: [Key : Any]) throws where T : Encodable & Equatable { // I AM SO SORRY THIS IS ALL I COULD FIGURE OUT TO MAKE [String] WORK!
        if value != (defaults[key] as! [T]) {
            try encode(value, forKey: key)
        }
    }
}

(Note the horrible hack on KeyedEncodingContainer where I had to special-case arrays of <Equatable>s, I guess because the compiler doesn’t know an array of <Equatable>s is Equatable itself?)

Problems with this technique I’ve identified are:

⑴ It doesn’t allow one to add defaults without manually writing the init(from:) and encode(to:), ugh.
⑵ The programmer has to add 'type(of: self).defaultsByCodingKey’ to every call, ugh.

Both of these could possibly be worked around if we could add an optional method to the <Codable> protocol, that would look something like:

    public static func default<Key>(keyedBy type: Key.Type, key: Key) -> Any? where Key : CodingKey

(the above line isn’t tested and doubtlessly won’t work as typed and has tons of think-os.)

This would get called by KeyedEncodingContainers and KeyedDecodingContainers only for keys that are Hashable (which I think is all keys, but you can stick un-keyed sub-things in Keyed containers and obviously those can’t have defaults just for them) and the container would be asked to do the comparison itself, with ‘==‘.

Something I haven’t tried to address here is what to do if values are NOT <Equatable> — then of course ‘==‘ won’t work. One approach to this would be to provide a way for the static func above to return ‘Hey, I don’t have anything meaningful for you for this particular property, because it’s not Equatable.’ This could be as simple as returning ‘nil’, which would also be a decent way to say, “This property has no meaningful default” which is also needed.

Alternatively, one could imagine adding TWO callbacks in the <Codable> for this kind of case, which are essentially *WAVES HANDS*:

     public static func isThisValueTheDefault(_ value: Any, forKey key: Self.Key) throws -> Any?
     public static func defaultValue<Key>(keyedBy type: Key.Type, key: Key) -> Any? where Key : CodingKey

These might also need a 'keyedBy type: Key.Type’ parameter — to be honest I haven’t messed with different key spaces so I’m not sure how they work. Also I’m not the best at generics yet. (At this point I’m not even sure if protocols can contain ‘class’ functions, so maybe none of this would work.)

Another advantage to the two-method approach (besides not requiring the values to be < Equatable >) is that it allows one to provide defaults for floating values, which can often be changed just by floating-point error by like 0.00000000001 and then end up registering false changes. In the isValueDefault(…) the programmer could implement a comparison with a ‘slop’ so if the encoder were about to write 0.000000000001 and the default were 0 nothing would be written.

-Wil


(Greg Parker) #2

Correct. Swift does not yet have the necessary language machinery to express "Array<T> is Equatable whenever T is Equatable".

SE-0143 "Conditional conformances" is approved but not yet implemented.
https://github.com/apple/swift-evolution/blob/master/proposals/0143-conditional-conformances.md

···

On Jul 10, 2017, at 5:16 PM, William Shipley via swift-evolution <swift-evolution@swift.org> wrote:

(Note the horrible hack on KeyedEncodingContainer where I had to special-case arrays of <Equatable>s, I guess because the compiler doesn’t know an array of <Equatable>s is Equatable itself?)

--
Greg Parker gparker@apple.com <mailto:gparker@apple.com> Runtime Wrangler


(Randy Eckenrode) #3

It seems like it would be cleaner to extend CodingKey. There might be a more general way of doing this than just requiring a Dictionary, but it seems to work.

protocol DefaultingCodingKey: CodingKey, Hashable {
    static var defaults: [Self: Any] { get }
}

// Implementing the other overrides left as an exercise to the reader
extension KeyedDecodingContainer where Key: DefaultingCodingKey {

    func decode(_ type: String.Type, forKey key: Key) throws -> String {
        if let t = try self.decodeIfPresent(type, forKey: key) {
            return t
        } else {
            return Swift.type(of: key).defaults[key] as! String
        }
    }

    func decode<T: Codable>(_ type: T.Type, forKey key: Key) throws -> T {
        if let t = try self.decodeIfPresent(type, forKey: key) {
            return t
        } else {
            return Swift.type(of: key).defaults[key] as! T
        }
    }

}

extension KeyedEncodingContainer where Key: DefaultingCodingKey {

    mutating func encode(_ value: String, forKey key: Key) throws {
        guard value != type(of: key).defaults[key] as! String else { return }
        try self.encodeIfPresent(value, forKey: key)
    }

    mutating func encode<T: Encodable & Equatable>(_ value: [T], forKey key: Key) throws {
        guard value != type(of: key).defaults[key] as! [T] else { return }
        try self.encodeIfPresent(value, forKey: key)
    }

    mutating func encode<T: Encodable & Equatable>(_ value: T, forKey key: Key) throws {
        guard value != type(of: key).defaults[key] as! T else { return }
        try self.encodeIfPresent(value, forKey: key)
    }

}

class ReferencePieceFromModel: Codable {

    public var name: String = ""
    public var styles: [String] = []

    private enum CodingKeys: String, DefaultingCodingKey {

        case name, styles

        static let defaults: [CodingKeys: Any] = [
            .name: "",
            .styles: [String]()
        ]
    }
}

Putting all of this into a playground….

let x = ReferencePieceFromModel()

let encoder = JSONEncoder()

let json = try! encoder.encode(x)
print(String(data: json, encoding: .utf8)!)

let decoder = JSONDecoder()

let a = try! decoder.decode(ReferencePieceFromModel.self, from: json)
print(a.name)
print(a.styles)

let refWithName = "{\"name\": \"Randy\"}"
let b = try! decoder.decode(ReferencePieceFromModel.self, from: refWithName.data(using: .utf8)!)
print(b.name)
print(b.styles)

let ref = "{\"name\": \"Randy\", \"styles\": [\"Swifty\"]}"
let c = try! decoder.decode(ReferencePieceFromModel.self, from: ref.data(using: .utf8)!)
print(c.name)
print(c.styles)

Prints out…

{}

[]
Randy
[]
Randy
["Swifty"]

···

--
Randy

On Jul 10, 2017, at 8:16 PM, William Shipley via swift-evolution <swift-evolution@swift.org> wrote:

Automatic substitution / removal of default values is very useful when reading or writing a file, respectively, and should be supported by the <Codable> family of protocols and objects:

• When reading, swapping in a default value for missing or corrupted values makes it so hand-created or third-party-created files don’t have to write every single value to make a valid file, and allows slightly corrupted files to auto-repair (or get close, and let the user fix up any data that needs it after) rather than completely fail to load. (Repairing on read creates a virtuous cycle with user-created files, as the user will get _some_ feedback on her input even if she’s messed up, for example, the type of one of the properties.)

• When writing, providing a default value allows the container to skip keys that don’t contain useful information. This can dramatically reduce file sizes, but I think its other advantages are bigger wins: just like having less source code makes a program easier to debug, having less “data code” makes files easier to work with in every way — they’re easier to see differences in, easier to determine corruption in, easier to edit by hand, and easier to learn from.

My first pass attempt at adding defaults to Codable looks like this:

public class ReferencePieceFromModel : Codable {

    // MARK: properties
    public let name: String = ""
    public let styles: [String] = []

    // MARK: <Codable>
    public required init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)

        self.name = container.decode(String.self, forKey: .name, defaults: type(of: self).defaultsByCodingKey)
        self.styles = container.decode([String].self, forKey: .styles, defaults: type(of: self).defaultsByCodingKey)
    }
    public func encode(to encoder: Encoder) throws {
        var container = encoder.container(keyedBy: CodingKeys.self)

        try container.encode(name, forKey: .name, defaults: type(of: self).defaultsByCodingKey)
        try container.encode(styles, forKey: .styles, defaults: type(of: self).defaultsByCodingKey)
    }
    private static let defaultsByCodingKey: [CodingKeys : Any] = [
        .name : "",
        .styles : [String]()
    ]

    // MARK: private
    private enum CodingKeys : String, CodingKey {
        case name
        case styles
    }
}

With just a couple additions to the Swift libraries:

extension KeyedDecodingContainer where Key : Hashable {
    func decode<T>(_ type: T.Type, forKey key: Key, defaults: [Key : Any]) -> T where T : Decodable {
        if let typedValueOptional = try? decodeIfPresent(T.self, forKey: key), let typedValue = typedValueOptional {
            return typedValue
        } else {
            return defaults[key] as! T
        }
    }
}

extension KeyedEncodingContainer where Key : Hashable {
    mutating func encode<T>(_ value: T, forKey key: Key, defaults: [Key : Any]) throws where T : Encodable & Equatable {
        if value != (defaults[key] as! T) {
            try encode(value, forKey: key)
        }
    }

    mutating func encode<T>(_ value: [T], forKey key: Key, defaults: [Key : Any]) throws where T : Encodable & Equatable { // I AM SO SORRY THIS IS ALL I COULD FIGURE OUT TO MAKE [String] WORK!
        if value != (defaults[key] as! [T]) {
            try encode(value, forKey: key)
        }
    }
}

(Note the horrible hack on KeyedEncodingContainer where I had to special-case arrays of <Equatable>s, I guess because the compiler doesn’t know an array of <Equatable>s is Equatable itself?)

Problems with this technique I’ve identified are:

⑴ It doesn’t allow one to add defaults without manually writing the init(from:) and encode(to:), ugh.
⑵ The programmer has to add 'type(of: self).defaultsByCodingKey’ to every call, ugh.

Both of these could possibly be worked around if we could add an optional method to the <Codable> protocol, that would look something like:

    public static func default<Key>(keyedBy type: Key.Type, key: Key) -> Any? where Key : CodingKey

(the above line isn’t tested and doubtlessly won’t work as typed and has tons of think-os.)

This would get called by KeyedEncodingContainers and KeyedDecodingContainers only for keys that are Hashable (which I think is all keys, but you can stick un-keyed sub-things in Keyed containers and obviously those can’t have defaults just for them) and the container would be asked to do the comparison itself, with ‘==‘.

Something I haven’t tried to address here is what to do if values are NOT <Equatable> — then of course ‘==‘ won’t work. One approach to this would be to provide a way for the static func above to return ‘Hey, I don’t have anything meaningful for you for this particular property, because it’s not Equatable.’ This could be as simple as returning ‘nil’, which would also be a decent way to say, “This property has no meaningful default” which is also needed.

Alternatively, one could imagine adding TWO callbacks in the <Codable> for this kind of case, which are essentially *WAVES HANDS*:

     public static func isThisValueTheDefault(_ value: Any, forKey key: Self.Key) throws -> Any?
     public static func defaultValue<Key>(keyedBy type: Key.Type, key: Key) -> Any? where Key : CodingKey

These might also need a 'keyedBy type: Key.Type’ parameter — to be honest I haven’t messed with different key spaces so I’m not sure how they work. Also I’m not the best at generics yet. (At this point I’m not even sure if protocols can contain ‘class’ functions, so maybe none of this would work.)

Another advantage to the two-method approach (besides not requiring the values to be < Equatable >) is that it allows one to provide defaults for floating values, which can often be changed just by floating-point error by like 0.00000000001 and then end up registering false changes. In the isValueDefault(…) the programmer could implement a comparison with a ‘slop’ so if the encoder were about to write 0.000000000001 and the default were 0 nothing would be written.

-Wil

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Itai Ferber) #4

Hi Wil,

Thanks for putting this together! My biggest thought on this is — what does this provide that you can’t already do yourself today?
Since you have to go through the work to put together default values and override `init(from:)` and `encode(to:)` to use them, I’m wondering whether this saves you any work over doing something like the following:

struct Theme {
     private static let _defaultName = ""
     private static let _defaultStyles: [String] = []

     public let name: String
     public let styles: [String]

     private enum CodingKeys : String, CodingKey {
         case name
         case styles
     }

     public init(from decoder: Decoder) throws {
         let container = try decoder.container(keyedBy: CodingKeys.self)
         name = try? decoder.decode(String.self, forKey: .name) ?? Theme._defaultName
         styles = try? decoder.decode([String.self], forKey: .styles) ?? Theme._defaultStyles
     }

     public func encode(to encoder: Encoder) throws {
         var container = encoder.container(keyedBy: CodingKeys.self)
         if (name != Theme._defaultName) try container.encode(name, forKey: .name)
         if (styles != Theme._defaultStyles) try container.encode(styles, forKey: .styles)
     }
}

This reads just as clearly to me as the `defaults:` variation while having the added benefit of low complexity and stronger type safety (as there’s no `as!`-casting down from `Any`, which could fail).

Thoughts?

— Itai

···

On 10 Jul 2017, at 17:16, William Shipley via swift-evolution wrote:

Automatic substitution / removal of default values is very useful when reading or writing a file, respectively, and should be supported by the <Codable> family of protocols and objects:

• When reading, swapping in a default value for missing or corrupted values makes it so hand-created or third-party-created files don’t have to write every single value to make a valid file, and allows slightly corrupted files to auto-repair (or get close, and let the user fix up any data that needs it after) rather than completely fail to load. (Repairing on read creates a virtuous cycle with user-created files, as the user will get _some_ feedback on her input even if she’s messed up, for example, the type of one of the properties.)

• When writing, providing a default value allows the container to skip keys that don’t contain useful information. This can dramatically reduce file sizes, but I think its other advantages are bigger wins: just like having less source code makes a program easier to debug, having less “data code” makes files easier to work with in every way — they’re easier to see differences in, easier to determine corruption in, easier to edit by hand, and easier to learn from.

My first pass attempt at adding defaults to Codable looks like this:

public class ReferencePieceFromModel : Codable {

    // MARK: properties
    public let name: String = ""
    public let styles: [String] = []

    // MARK: <Codable>
    public required init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)

        self.name = container.decode(String.self, forKey: .name, defaults: type(of: self).defaultsByCodingKey)
        self.styles = container.decode([String].self, forKey: .styles, defaults: type(of: self).defaultsByCodingKey)
    }
    public func encode(to encoder: Encoder) throws {
        var container = encoder.container(keyedBy: CodingKeys.self)

        try container.encode(name, forKey: .name, defaults: type(of: self).defaultsByCodingKey)
        try container.encode(styles, forKey: .styles, defaults: type(of: self).defaultsByCodingKey)
    }
    private static let defaultsByCodingKey: [CodingKeys : Any] = [
        .name : "",
        .styles : [String]()
    ]

    // MARK: private
    private enum CodingKeys : String, CodingKey {
        case name
        case styles
    }
}

With just a couple additions to the Swift libraries:

extension KeyedDecodingContainer where Key : Hashable {
    func decode<T>(_ type: T.Type, forKey key: Key, defaults: [Key : Any]) -> T where T : Decodable {
        if let typedValueOptional = try? decodeIfPresent(T.self, forKey: key), let typedValue = typedValueOptional {
            return typedValue
        } else {
            return defaults[key] as! T
        }
    }
}

extension KeyedEncodingContainer where Key : Hashable {
    mutating func encode<T>(_ value: T, forKey key: Key, defaults: [Key : Any]) throws where T : Encodable & Equatable {
        if value != (defaults[key] as! T) {
            try encode(value, forKey: key)
        }
    }

    mutating func encode<T>(_ value: [T], forKey key: Key, defaults: [Key : Any]) throws where T : Encodable & Equatable { // I AM SO SORRY THIS IS ALL I COULD FIGURE OUT TO MAKE [String] WORK!
        if value != (defaults[key] as! [T]) {
            try encode(value, forKey: key)
        }
    }
}

(Note the horrible hack on KeyedEncodingContainer where I had to special-case arrays of <Equatable>s, I guess because the compiler doesn’t know an array of <Equatable>s is Equatable itself?)

Problems with this technique I’ve identified are:

⑴ It doesn’t allow one to add defaults without manually writing the init(from:) and encode(to:), ugh.
⑵ The programmer has to add 'type(of: self).defaultsByCodingKey’ to every call, ugh.

Both of these could possibly be worked around if we could add an optional method to the <Codable> protocol, that would look something like:

    public static func default<Key>(keyedBy type: Key.Type, key: Key) -> Any? where Key : CodingKey

(the above line isn’t tested and doubtlessly won’t work as typed and has tons of think-os.)

This would get called by KeyedEncodingContainers and KeyedDecodingContainers only for keys that are Hashable (which I think is all keys, but you can stick un-keyed sub-things in Keyed containers and obviously those can’t have defaults just for them) and the container would be asked to do the comparison itself, with ‘==‘.

Something I haven’t tried to address here is what to do if values are NOT <Equatable> — then of course ‘==‘ won’t work. One approach to this would be to provide a way for the static func above to return ‘Hey, I don’t have anything meaningful for you for this particular property, because it’s not Equatable.’ This could be as simple as returning ‘nil’, which would also be a decent way to say, “This property has no meaningful default” which is also needed.

Alternatively, one could imagine adding TWO callbacks in the <Codable> for this kind of case, which are essentially *WAVES HANDS*:

     public static func isThisValueTheDefault(_ value: Any, forKey key: Self.Key) throws -> Any?
     public static func defaultValue<Key>(keyedBy type: Key.Type, key: Key) -> Any? where Key : CodingKey

These might also need a 'keyedBy type: Key.Type’ parameter — to be honest I haven’t messed with different key spaces so I’m not sure how they work. Also I’m not the best at generics yet. (At this point I’m not even sure if protocols can contain ‘class’ functions, so maybe none of this would work.)

Another advantage to the two-method approach (besides not requiring the values to be < Equatable >) is that it allows one to provide defaults for floating values, which can often be changed just by floating-point error by like 0.00000000001 and then end up registering false changes. In the isValueDefault(…) the programmer could implement a comparison with a ‘slop’ so if the encoder were about to write 0.000000000001 and the default were 0 nothing would be written.

-Wil

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(William Shipley) #5

You’re right, my current implementation doesn’t win anything over what you’re written - in fact your technique is basically what I wrote at first.

I was trying to work towards encapsulating the behavior in the encoder/decoder so that the automatic init/encode methods could work, so I wanted to introduce my first (more manual) attempt and then say, here’s where I’d like to get with this.

-Wil

···

On Jul 11, 2017, at 10:16 AM, Itai Ferber <iferber@apple.com> wrote:

Hi Wil,

Thanks for putting this together! My biggest thought on this is — what does this provide that you can’t already do yourself today?
Since you have to go through the work to put together default values and override init(from:) and encode(to:) to use them, I’m wondering whether this saves you any work over doing something like the following:

struct Theme {
    private static let _defaultName = ""
    private static let _defaultStyles: [String] = []

    public let name: String
    public let styles: [String]

    private enum CodingKeys : String, CodingKey {
        case name
        case styles
    }

    public init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        name = try? decoder.decode(String.self, forKey: .name) ?? Theme._defaultName
        styles = try? decoder.decode([String.self], forKey: .styles) ?? Theme._defaultStyles
    }

    public func encode(to encoder: Encoder) throws {
        var container = encoder.container(keyedBy: CodingKeys.self)
        if (name != Theme._defaultName) try container.encode(name, forKey: .name)
        if (styles != Theme._defaultStyles) try container.encode(styles, forKey: .styles)
    }
}
This reads just as clearly to me as the defaults: variation while having the added benefit of low complexity and stronger type safety (as there’s no as!-casting down from Any, which could fail).

Thoughts?

— Itai

On 10 Jul 2017, at 17:16, William Shipley via swift-evolution wrote:

Automatic substitution / removal of default values is very useful when reading or writing a file, respectively, and should be supported by the <Codable> family of protocols and objects:

• When reading, swapping in a default value for missing or corrupted values makes it so hand-created or third-party-created files don’t have to write every single value to make a valid file, and allows slightly corrupted files to auto-repair (or get close, and let the user fix up any data that needs it after) rather than completely fail to load. (Repairing on read creates a virtuous cycle with user-created files, as the user will get _some_ feedback on her input even if she’s messed up, for example, the type of one of the properties.)

• When writing, providing a default value allows the container to skip keys that don’t contain useful information. This can dramatically reduce file sizes, but I think its other advantages are bigger wins: just like having less source code makes a program easier to debug, having less “data code” makes files easier to work with in every way — they’re easier to see differences in, easier to determine corruption in, easier to edit by hand, and easier to learn from.

My first pass attempt at adding defaults to Codable looks like this:

public class ReferencePieceFromModel : Codable {

// MARK: properties
public let name: String = ""
public let styles: [String] = []

// MARK: <Codable>
public required init(from decoder: Decoder) throws {
let container = try decoder.container(keyedBy: CodingKeys.self)

self.name = container.decode(String.self, forKey: .name, defaults: type(of: self).defaultsByCodingKey)
self.styles = container.decode([String].self, forKey: .styles, defaults: type(of: self).defaultsByCodingKey)
}
public func encode(to encoder: Encoder) throws {
var container = encoder.container(keyedBy: CodingKeys.self)

try container.encode(name, forKey: .name, defaults: type(of: self).defaultsByCodingKey)
try container.encode(styles, forKey: .styles, defaults: type(of: self).defaultsByCodingKey)
}
private static let defaultsByCodingKey: [CodingKeys : Any] = [
.name : "",
.styles : [String]()
]

// MARK: private
private enum CodingKeys : String, CodingKey {
case name
case styles
}
}

With just a couple additions to the Swift libraries:

extension KeyedDecodingContainer where Key : Hashable {
func decode<T>(_ type: T.Type, forKey key: Key, defaults: [Key : Any]) -> T where T : Decodable {
if let typedValueOptional = try? decodeIfPresent(T.self, forKey: key), let typedValue = typedValueOptional {
return typedValue
} else {
return defaults[key] as! T
}
}
}

extension KeyedEncodingContainer where Key : Hashable {
mutating func encode<T>(_ value: T, forKey key: Key, defaults: [Key : Any]) throws where T : Encodable & Equatable {
if value != (defaults[key] as! T) {
try encode(value, forKey: key)
}
}

mutating func encode<T>(_ value: [T], forKey key: Key, defaults: [Key : Any]) throws where T : Encodable & Equatable { // I AM SO SORRY THIS IS ALL I COULD FIGURE OUT TO MAKE [String] WORK!
if value != (defaults[key] as! [T]) {
try encode(value, forKey: key)
}
}
}

(Note the horrible hack on KeyedEncodingContainer where I had to special-case arrays of <Equatable>s, I guess because the compiler doesn’t know an array of <Equatable>s is Equatable itself?)

Problems with this technique I’ve identified are:

⑴ It doesn’t allow one to add defaults without manually writing the init(from:) and encode(to:), ugh.
⑵ The programmer has to add 'type(of: self).defaultsByCodingKey’ to every call, ugh.

Both of these could possibly be worked around if we could add an optional method to the <Codable> protocol, that would look something like:

public static func default<Key>(keyedBy type: Key.Type, key: Key) -> Any? where Key : CodingKey

(the above line isn’t tested and doubtlessly won’t work as typed and has tons of think-os.)

This would get called by KeyedEncodingContainers and KeyedDecodingContainers only for keys that are Hashable (which I think is all keys, but you can stick un-keyed sub-things in Keyed containers and obviously those can’t have defaults just for them) and the container would be asked to do the comparison itself, with ‘==‘.

Something I haven’t tried to address here is what to do if values are NOT <Equatable> — then of course ‘==‘ won’t work. One approach to this would be to provide a way for the static func above to return ‘Hey, I don’t have anything meaningful for you for this particular property, because it’s not Equatable.’ This could be as simple as returning ‘nil’, which would also be a decent way to say, “This property has no meaningful default” which is also needed.

Alternatively, one could imagine adding TWO callbacks in the <Codable> for this kind of case, which are essentially *WAVES HANDS*:

public static func isThisValueTheDefault(_ value: Any, forKey key: Self.Key) throws -> Any?
public static func defaultValue<Key>(keyedBy type: Key.Type, key: Key) -> Any? where Key : CodingKey

These might also need a 'keyedBy type: Key.Type’ parameter — to be honest I haven’t messed with different key spaces so I’m not sure how they work. Also I’m not the best at generics yet. (At this point I’m not even sure if protocols can contain ‘class’ functions, so maybe none of this would work.)

Another advantage to the two-method approach (besides not requiring the values to be < Equatable >) is that it allows one to provide defaults for floating values, which can often be changed just by floating-point error by like 0.00000000001 and then end up registering false changes. In the isValueDefault(…) the programmer could implement a comparison with a ‘slop’ so if the encoder were about to write 0.000000000001 and the default were 0 nothing would be written.

-Wil

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Itai Ferber) #6

That’s fair. :slight_smile:
I think in the time frame of Swift 4, this would be too big of an addition and would require more thought, but:

1. When the conditional conformance feature arrives in a future Swift release, a lot of the hacks surrounding `Equatable` can go away here, because we’ll get things like `Array<Element> : Equatable where Element : Equatable` and `Array<Element> : Codable where Element : Codable`
2. This seems like an easily additive feature — overloads taking defaults can be added after the fact (given a default implementation which does something similar to what you and Randy suggested):

// Just an example:
extension KeyedEncodingContainerProtocol {
     func encode<T : Codable>(_ value: T, forKey key: Key, defaultValues defaults: [Key : Any]) throws where T : Equatable {
         guard let defaultValue = defaults[key],
               value != defaultValue else {
             return try encode(value, forKey: key)
         }
     }
}

extension KeyedDecodingContainerProtocol {
	func decode<T : Decodable>(_ type: T.Type, forKey key: Key, defaultValues defaults: [Key : Any]) throws -> T {
	    guard let defaultValue = defaults[key] else {
	        return try decode(type, forKey: key)
	    }

	    if let value = try decodeIfPresent(type, forKey: key) {
	        return value
	    } else {
	        return defaultValue
	    }
	}
}
···

On 11 Jul 2017, at 13:16, William Shipley wrote:

You’re right, my current implementation doesn’t win anything over what you’re written - in fact your technique is basically what I wrote at first.

I was trying to work towards encapsulating the behavior in the encoder/decoder so that the automatic init/encode methods could work, so I wanted to introduce my first (more manual) attempt and then say, here’s where I’d like to get with this.

-Wil

On Jul 11, 2017, at 10:16 AM, Itai Ferber <iferber@apple.com> wrote:

Hi Wil,

Thanks for putting this together! My biggest thought on this is — what does this provide that you can’t already do yourself today?
Since you have to go through the work to put together default values and override init(from:) and encode(to:) to use them, I’m wondering whether this saves you any work over doing something like the following:

struct Theme {
    private static let _defaultName = ""
    private static let _defaultStyles: [String] = []

    public let name: String
    public let styles: [String]

    private enum CodingKeys : String, CodingKey {
        case name
        case styles
    }

    public init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        name = try? decoder.decode(String.self, forKey: .name) ?? Theme._defaultName
        styles = try? decoder.decode([String.self], forKey: .styles) ?? Theme._defaultStyles
    }

    public func encode(to encoder: Encoder) throws {
        var container = encoder.container(keyedBy: CodingKeys.self)
        if (name != Theme._defaultName) try container.encode(name, forKey: .name)
        if (styles != Theme._defaultStyles) try container.encode(styles, forKey: .styles)
    }
}
This reads just as clearly to me as the defaults: variation while having the added benefit of low complexity and stronger type safety (as there’s no as!-casting down from Any, which could fail).

Thoughts?

— Itai

On 10 Jul 2017, at 17:16, William Shipley via swift-evolution wrote:

Automatic substitution / removal of default values is very useful when reading or writing a file, respectively, and should be supported by the <Codable> family of protocols and objects:

• When reading, swapping in a default value for missing or corrupted values makes it so hand-created or third-party-created files don’t have to write every single value to make a valid file, and allows slightly corrupted files to auto-repair (or get close, and let the user fix up any data that needs it after) rather than completely fail to load. (Repairing on read creates a virtuous cycle with user-created files, as the user will get _some_ feedback on her input even if she’s messed up, for example, the type of one of the properties.)

• When writing, providing a default value allows the container to skip keys that don’t contain useful information. This can dramatically reduce file sizes, but I think its other advantages are bigger wins: just like having less source code makes a program easier to debug, having less “data code” makes files easier to work with in every way — they’re easier to see differences in, easier to determine corruption in, easier to edit by hand, and easier to learn from.

My first pass attempt at adding defaults to Codable looks like this:

public class ReferencePieceFromModel : Codable {

// MARK: properties
public let name: String = ""
public let styles: [String] = []

// MARK: <Codable>
public required init(from decoder: Decoder) throws {
let container = try decoder.container(keyedBy: CodingKeys.self)

self.name = container.decode(String.self, forKey: .name, defaults: type(of: self).defaultsByCodingKey)
self.styles = container.decode([String].self, forKey: .styles, defaults: type(of: self).defaultsByCodingKey)
}
public func encode(to encoder: Encoder) throws {
var container = encoder.container(keyedBy: CodingKeys.self)

try container.encode(name, forKey: .name, defaults: type(of: self).defaultsByCodingKey)
try container.encode(styles, forKey: .styles, defaults: type(of: self).defaultsByCodingKey)
}
private static let defaultsByCodingKey: [CodingKeys : Any] = [
.name : "",
.styles : [String]()
]

// MARK: private
private enum CodingKeys : String, CodingKey {
case name
case styles
}

With just a couple additions to the Swift libraries:

extension KeyedDecodingContainer where Key : Hashable {
func decode<T>(_ type: T.Type, forKey key: Key, defaults: [Key : Any]) -> T where T : Decodable {
if let typedValueOptional = try? decodeIfPresent(T.self, forKey: key), let typedValue = typedValueOptional {
return typedValue
} else {
return defaults[key] as! T
}

extension KeyedEncodingContainer where Key : Hashable {
mutating func encode<T>(_ value: T, forKey key: Key, defaults: [Key : Any]) throws where T : Encodable & Equatable {
if value != (defaults[key] as! T) {
try encode(value, forKey: key)
}

mutating func encode<T>(_ value: [T], forKey key: Key, defaults: [Key : Any]) throws where T : Encodable & Equatable { // I AM SO SORRY THIS IS ALL I COULD FIGURE OUT TO MAKE [String] WORK!
if value != (defaults[key] as! [T]) {
try encode(value, forKey: key)
}

(Note the horrible hack on KeyedEncodingContainer where I had to special-case arrays of <Equatable>s, I guess because the compiler doesn’t know an array of <Equatable>s is Equatable itself?)

Problems with this technique I’ve identified are:

⑴ It doesn’t allow one to add defaults without manually writing the init(from:) and encode(to:), ugh.
⑵ The programmer has to add 'type(of: self).defaultsByCodingKey’ to every call, ugh.

Both of these could possibly be worked around if we could add an optional method to the <Codable> protocol, that would look something like:

public static func default<Key>(keyedBy type: Key.Type, key: Key) -> Any? where Key : CodingKey

(the above line isn’t tested and doubtlessly won’t work as typed and has tons of think-os.)

This would get called by KeyedEncodingContainers and KeyedDecodingContainers only for keys that are Hashable (which I think is all keys, but you can stick un-keyed sub-things in Keyed containers and obviously those can’t have defaults just for them) and the container would be asked to do the comparison itself, with ‘==‘.

Something I haven’t tried to address here is what to do if values are NOT <Equatable> — then of course ‘==‘ won’t work. One approach to this would be to provide a way for the static func above to return ‘Hey, I don’t have anything meaningful for you for this particular property, because it’s not Equatable.’ This could be as simple as returning ‘nil’, which would also be a decent way to say, “This property has no meaningful default” which is also needed.

Alternatively, one could imagine adding TWO callbacks in the <Codable> for this kind of case, which are essentially *WAVES HANDS*:

public static func isThisValueTheDefault(_ value: Any, forKey key: Self.Key) throws -> Any?
public static func defaultValue<Key>(keyedBy type: Key.Type, key: Key) -> Any? where Key : CodingKey

These might also need a 'keyedBy type: Key.Type’ parameter — to be honest I haven’t messed with different key spaces so I’m not sure how they work. Also I’m not the best at generics yet. (At this point I’m not even sure if protocols can contain ‘class’ functions, so maybe none of this would work.)

Another advantage to the two-method approach (besides not requiring the values to be < Equatable >) is that it allows one to provide defaults for floating values, which can often be changed just by floating-point error by like 0.00000000001 and then end up registering false changes. In the isValueDefault(…) the programmer could implement a comparison with a ‘slop’ so if the encoder were about to write 0.000000000001 and the default were 0 nothing would be written.

-Wil

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Benjamin Spratling) #7

Speaking of this, what concerns me is the really long list of approved swift features that haven’t been implemented. We had a bunch of features approved for Swift 3 that never made it in. Seems like even more this time. Do we just have scores more people proposing and approving features than implementing? I’d help out, but I can’t stand writing in C++.

Are we likely to get all the approved features by the Xcode 9 GM? Or is there going to be a 4.1 later in the winter? Not having 68, 75, 143 and 157 implemented is a nuisance multiple times per day.

-Ben Spratling

···

On Jul 12, 2017, at 7:33 PM, Itai Ferber via swift-evolution <swift-evolution@swift.org> wrote:
When the conditional conformance feature arrives in a future Swift release,