[Proposal] Foundation Swift Archival & Serialization


(Itai Ferber) #1

Hi everyone,

The following introduces a new Swift-focused archival and serialization API as part of the Foundation framework. We’re interested in improving the experience and safety of performing archival and serialization, and are happy to receive community feedback on this work.
Because of the length of this proposal, the Appendix and Alternatives Considered sections have been omitted here, but are available in the full proposal <https://github.com/apple/swift-evolution/pull/639> on the swift-evolution repo. The full proposal also includes an Unabridged API for further consideration.

Without further ado, inlined below.

— Itai

Swift Archival & Serialization
Proposal: SE-NNNN <https://github.com/apple/swift-evolution/pull/639>
Author(s): Itai Ferber <https://github.com/itaiferber>, Michael LeHew <https://github.com/mlehew>, Tony Parker <https://github.com/parkera>
Review Manager: TBD
Status: Awaiting review
Associated PRs:
#8124 <https://github.com/apple/swift/pull/8124>
#8125 <https://github.com/apple/swift/pull/8125>
Introduction
Foundation's current archival and serialization APIs (NSCoding, NSJSONSerialization, NSPropertyListSerialization, etc.), while fitting for the dynamism of Objective-C, do not always map optimally into Swift. This document lays out the design of an updated API that improves the developer experience of performing archival and serialization in Swift.

Specifically:

It aims to provide a solution for the archival of Swift struct and enum types
It aims to provide a more type-safe solution for serializing to external formats, such as JSON and plist
Motivation
The primary motivation for this proposal is the inclusion of native Swift enum and struct types in archival and serialization. Currently, developers targeting Swift cannot participate in NSCoding without being willing to abandon enum and structtypes — NSCoding is an @objc protocol, conformance to which excludes non-class types. This is can be limiting in Swift because small enums and structs can be an idiomatic approach to model representation; developers who wish to perform archival have to either forgo the Swift niceties that constructs like enumsprovide, or provide an additional compatibility layer between their "real" types and their archivable types.

Secondarily, we would like to refine Foundation's existing serialization APIs (NSJSONSerialization and NSPropertyListSerialization) to better match Swift's strong type safety. From experience, we find that the conversion from the unstructured, untyped data of these formats into strongly-typed data structures is a good fit for archival mechanisms, rather than taking the less safe approach that 3rd-party JSON conversion approaches have taken (described further in an appendix below).

We would like to offer a solution to these problems without sacrificing ease of use or type safety.

Agenda
This proposal is the first stage of three that introduce different facets of a whole Swift archival and serialization API:

This proposal describes the basis for this API, focusing on the protocols that users adopt and interface with
The next stage will propose specific API for new encoders
The final stage will discuss how this new API will interop with NSCoding as it is today
SE-NNNN provides stages 2 and 3.

Proposed solution
We will be introducing the following new types:

protocol Codable: Adopted by types to opt into archival. Conformance may be automatically derived in cases where all properties are also Codable.
protocol CodingKey: Adopted by types used as keys for keyed containers, replacing String keys with semantic types. Conformance may be automatically derived in most cases.
protocol Encoder: Adopted by types which can take Codable values and encode them into a native format.
class KeyedEncodingContainer<Key : CodingKey>: Subclasses of this type provide a concrete way to store encoded values by CodingKey. Types adopting Encoder should provide subclasses of KeyedEncodingContainer to vend.
protocol SingleValueEncodingContainer: Adopted by types which provide a concrete way to store a single encoded value. Types adopting Encoder should provide types conforming to SingleValueEncodingContainer to vend (but in many cases will be able to conform to it themselves).
protocol Decoder: Adopted by types which can take payloads in a native format and decode Codable values out of them.
class KeyedDecodingContainer<Key : CodingKey>: Subclasses of this type provide a concrete way to retrieve encoded values from storage by CodingKey. Types adopting Decoder should provide subclasses of KeyedDecodingContainer to vend.
protocol SingleValueDecodingContainer: Adopted by types which provide a concrete way to retrieve a single encoded value from storage. Types adopting Decoder should provide types conforming to SingleValueDecodingContainer to vend (but in many cases will be able to conform to it themselves).
For end users of this API, adoption will primarily involve the Codable and CodingKey protocols. In order to participate in this new archival system, developers must add Codable conformance to their types:

// If all properties are Codable, implementation is automatically derived:
public struct Location : Codable {
    public let latitude: Double
    public let longitude: Double
}

public enum Animal : Int, Codable {
    case chicken = 1
    case dog
    case turkey
    case cow
}

public struct Farm : Codable {
    public let name: String
    public let location: Location
    public let animals: [Animal]
}
With developer participation, we will offer encoders and decoders (described in SE-NNNN, not here) that take advantage of this conformance to offer type-safe serialization of user models:

let farm = Farm(name: "Old MacDonald's Farm",
                location: Location(latitude: 51.621648, longitude: 0.269273),
                animals: [.chicken, .dog, .cow, .turkey, .dog, .chicken, .cow, .turkey, .dog])
let payload: Data = try JSONEncoder().encode(farm)

do {
    let farm = try JSONDecoder().decode(Farm.self, from: payload)

    // Extracted as user types:
    let coordinates = "\(farm.location.latitude, farm.location.longitude)"
} catch {
    // Encountered error during deserialization
}
This gives developers access to their data in a type-safe manner and a recognizable interface.

Detailed design
To support user types, we expose the Codable protocol:

/// Conformance to `Codable` indicates that a type can marshal itself into and out of an external representation.
public protocol Codable {
    /// Initializes `self` by decoding from `decoder`.
    ///
    /// - parameter decoder: The decoder to read data from.
    /// - throws: An error if reading from the decoder fails, or if read data is corrupted or otherwise invalid.
    init(from decoder: Decoder) throws

    /// Encodes `self` into the given encoder.
    ///
    /// If `self` fails to encode anything, `encoder` will encode an empty `.default` container in its place.
    ///
    /// - parameter encoder: The encoder to write data to.
    /// - throws: An error if any values are invalid for `encoder`'s format.
    func encode(to encoder: Encoder) throws
}
By adopting Codable, user types opt in to this archival system.

Structured types (i.e. types which encode as a collection of properties) encode and decode their properties in a keyed manner. Keys may be String-convertible or Int-convertible (or both), and user types which have properties should declare semantic key enums which map keys to their properties. Keys must conform to the CodingKey protocol:

/// Conformance to `CodingKey` indicates that a type can be used as a key for encoding and decoding.
public protocol CodingKey {
    /// The string to use in a named collection (e.g. a string-keyed dictionary).
    var stringValue: String? { get }

    /// Initializes `self` from a string.
    ///
    /// - parameter stringValue: The string value of the desired key.
    /// - returns: An instance of `Self` from the given string, or `nil` if the given string does not correspond to any instance of `Self`.
    init?(stringValue: String)

    /// The int to use in an indexed collection (e.g. an int-keyed dictionary).
    var intValue: Int? { get }

    /// Initializes `self` from an integer.
    ///
    /// - parameter intValue: The integer value of the desired key.
    /// - returns: An instance of `Self` from the given integer, or `nil` if the given integer does not correspond to any instance of `Self`.
    init?(intValue: Int)
}
For most types, String-convertible keys are a reasonable default; for performance, however, Int-convertible keys are preferred, and Encoders may choose to make use of Ints over Strings. Framework types should provide keys which have both for flexibility and performance across different types of Encoders. It is generally an error to provide a key which has neither a stringValue nor an intValue.

By default, CodingKey conformance can be derived for enums which have either String or Int backing:

enum Keys1 : CodingKey {
    case a // (stringValue: "a", intValue: nil)
    case b // (stringValue: "b", intValue: nil)
}

enum Keys2 : String, CodingKey {
    case c = "foo" // (stringValue: "foo", intValue: nil)
    case d // (stringValue: "d", intValue: nil)
}

enum Keys3 : Int, CodingKey {
    case e = 4 // (stringValue: "e", intValue: 4)
    case f // (stringValue: "f", intValue: 5)
    case g = 9 // (stringValue: "g", intValue: 9)
}
Coding keys which are not enums, have associated values, or have other raw representations must implement these methods manually.

In addition to automatic CodingKey conformance derivation for enums, Codableconformance can be automatically derived for certain types as well:

Types whose properties are all either Codable or primitive get an automatically derived String-backed CodingKeys enum mapping properties to case names
Types falling into (1) and types which provide a CodingKeys enum (directly or via a typealias) whose case names map to properties which are all Codableget automatic derivation of init(from:) and encode(to:) using those properties and keys. Types may choose to provide a custom init(from:) or encode(to:) (or both); whichever they do not provide will be automatically derived
Types which fall into neither (1) nor (2) will have to provide a custom key type and provide their own init(from:) and encode(to:)
Many types will either allow for automatic derivation of all codability (1), or provide a custom key subset and take advantage of automatic method derivation (2).

Encoding and Decoding

Types which are encodable encode their data into a container provided by their Encoder:

/// An `Encoder` is a type which can encode values into a native format for external representation.
public protocol Encoder {
    /// Populates `self` with an encoding container (of `.default` type) and returns it, keyed by the given key type.
    ///
    /// - parameter type: The key type to use for the container.
    /// - returns: A new keyed encoding container.
    /// - precondition: May not be called after a previous `self.container(keyedBy:)` call of a different `EncodingContainerType`.
    /// - precondition: May not be called after a value has been encoded through a prior `self.singleValueContainer()` call.
    func container<Key : CodingKey>(keyedBy type: Key.Type) -> KeyedEncodingContainer<Key>

    /// Returns an encoding container appropriate for holding a single primitive value.
    ///
    /// - returns: A new empty single value container.
    /// - precondition: May not be called after a prior `self.container(keyedBy:)` call.
    /// - precondition: May not be called after a value has been encoded through a previous `self.singleValueContainer()` call.
    func singleValueContainer() -> SingleValueEncodingContainer

    /// The path of coding keys taken to get to this point in encoding.
    var codingKeyContext: [CodingKey] { get }
}

// Continuing examples from before; below is automatically generated by the compiler if no customization is needed.
public struct Location : Codable {
    private enum CodingKeys : CodingKey {
        case latitutude
        case longitude
    }

    public func encode(to encoder: Encoder) throws {
        // Generic keyed encoder gives type-safe key access: cannot encode with keys of the wrong type.
        let container = encoder.container(keyedBy: CodingKeys.self)

        // The encoder is generic on the key -- free key autocompletion here.
        try container.encode(latitude, forKey: .latitude)
        try container.encode(longitude, forKey: .longitude)
    }
}

public struct Farm : Codable {
    private enum CodingKeys : CodingKey {
        case name
        case location
        case animals
    }

    public func encode(to encoder: Encoder) throws {
        let container = encoder.container(keyedBy: CodingKeys.self)
        try container.encode(name, forKey: .name)
        try container.encode(location, forKey: .location)
        try container.encode(animals, forKey: .animals)
    }
}
Similarly, decodable types initialize from data read from their Decoder's container:

/// A `Decoder` is a type which can decode values from a native format into in-memory representations.
public protocol Decoder {
    /// Returns the data stored in `self` as represented in a container keyed by the given key type.
    ///
    /// - parameter type: The key type to use for the container.
    /// - returns: A keyed decoding container view into `self`.
    /// - throws: `CocoaError.coderTypeMismatch` if the encountered stored value is not a keyed container.
    func container<Key : CodingKey>(keyedBy type: Key.Type) throws -> KeyedDecodingContainer<Key>

    /// Returns the data stored in `self` as represented in a container appropriate for holding a single primitive value.
    ///
    /// - returns: A single value container view into `self`.
    /// - throws: `CocoaError.coderTypeMismatch` if the encountered stored value is not a single value container.
    func singleValueContainer() throws -> SingleValueDecodingContainer

    /// The path of coding keys taken to get to this point in decoding.
    var codingKeyContext: [CodingKey] { get }
}

// Continuing examples from before; below is automatically generated by the compiler if no customization is needed.
public struct Location : Codable {
    public init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        latitude = try container.decode(Double.self, forKey: .latitude)
        longitude = try container.decode(Double.self, forKey: .longitude)
    }
}

public struct Farm : Codable {
    public init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        name = try container.decode(String.self, forKey: .name)
        location = try container.decode(Location.self, forKey: .location)
        animals = try container.decode([Animal].self, forKey: .animals)
    }
}
Keyed Encoding Containers

Keyed encoding containers are the primary interface that most Codable types interact with for encoding and decoding. Through these, Codable types have strongly-keyed access to encoded data by using keys that are semantically correct for the operations they want to express.

Since semantically incompatible keys will rarely (if ever) share the same key type, it is impossible to mix up key types within the same container (as is possible with Stringkeys), and since the type is known statically, keys get autocompletion by the compiler.

/// `KeyedEncodingContainer` is a generic abstract base class that provides a view into an `Encoders` storage and is used to hold the encoded properties of a `Codable` type.
///
/// Encoders should provide subclasses of `KeyedEncodingContainer` for their format.
open class KeyedEncodingContainer<Key : CodingKey> {
    /// Encodes the given value for the given key.
    ///
    /// - parameter value: The value to encode.
    /// - parameter key: The key to associate the value with.
    /// - throws: `CocoaError.coderInvalidValue` if the given value is invalid in the current context for this format.
    /// - precondition: The key must have a `stringValue` or `intValue` appropriate for the encoding container type.
    open func encode<Value : Codable>(_ value: Value?, forKey key: Key) throws

    /// Encodes the given value for the given key.
    ///
    /// - parameter value: The value to encode.
    /// - parameter key: The key to associate the value with.
    /// - throws: `CocoaError.coderInvalidValue` if the given value is invalid in the current context for this format.
    /// - precondition: The key must have a `stringValue` or `intValue` appropriate for the encoding container type.
    open func encode(_ value: Bool?, forKey key: Key) throws
    open func encode(_ value: Int?, forKey key: Key) throws
    open func encode(_ value: Int8?, forKey key: Key) throws
    open func encode(_ value: Int16?, forKey key: Key) throws
    open func encode(_ value: Int32?, forKey key: Key) throws
    open func encode(_ value: Int64?, forKey key: Key) throws
    open func encode(_ value: UInt?, forKey key: Key) throws
    open func encode(_ value: UInt8?, forKey key: Key) throws
    open func encode(_ value: UInt16?, forKey key: Key) throws
    open func encode(_ value: UInt32?, forKey key: Key) throws
    open func encode(_ value: UInt64?, forKey key: Key) throws
    open func encode(_ value: Float?, forKey key: Key) throws
    open func encode(_ value: Double?, forKey key: Key) throws
    open func encode(_ value: String?, forKey key: Key) throws
    open func encode(_ value: Data?, forKey key: Key) throws

    /// Encodes the given object weakly for the given key.
    ///
    /// For `Encoder`s that implement this functionality, this will only encode the given object and associate it with the given key if it encoded unconditionally elsewhere in the archive (either previously or in the future).
    ///
    /// For formats which don't support this feature, the default implementation encodes the given object unconditionally.
    ///
    /// - parameter object: The object to encode.
    /// - parameter key: The key to associate the object with.
    /// - throws: `CocoaError.coderInvalidValue` if the given value is invalid in the current context for this format.
    /// - precondition: The key must have a `stringValue` or `intValue` appropriate for the encoding container type.
    open func encodeWeak<Object : AnyObject & Codable>(_ object: Object?, forKey key: Key) throws

    /// The path of coding keys taken to get to this point in encoding.
    open var codingKeyContext: [CodingKey]
}

/// `KeyedDecodingContainer` is a generic abstract base class that provides a view into an `Decoders` storage and is used to hold the encoded properties of a `Codable` type.
///
/// Decoders should provide subclasses of `KeyedDecodingContainer` for their format.
open class KeyedDecodingContainer<Key : CodingKey> {
    /// All the keys the `Decoder` has for this container.
    ///
    /// Different keyed containers from the same `Decoder` may return different keys here; it is possible to encode with multiple key types which are not convertible to one another. This should report all keys present which are convertible to the requested type.
    open var allKeys: [Key]

    /// Returns whether the `Decoder` contains a value associated with the given key.
    ///
    /// The value associated with the given key may be a null value as appropriate for the data format.
    ///
    /// - parameter key: The key to search for.
    /// - returns: Whether the `Decoder` has an entry for the given key.
    open func contains(_ key: Key) -> Bool

    /// Decodes a value of the given type for the given key.
    ///
    /// A default implementation is given for these types which calls into the abstract `decodeIfPresent` implementations below.
    ///
    /// - parameter type: The type of value to decode.
    /// - parameter key: The key that the decoded value is associated with.
    /// - returns: A value of the requested type, if present for the given key and convertible to the requested type.
    /// - throws: `CocoaError.coderTypeMismatch` if the encountered encoded value is not convertible to the requested type.
    /// - throws: `CocoaError.coderValueNotFound` if `self` does not have an entry for the given key or if the value is null.
    open func decode(_ type: Bool.Type, forKey key: Key) throws -> Bool
    open func decode(_ type: Int.Type, forKey key: Key) throws -> Int
    open func decode(_ type: Int8.Type, forKey key: Key) throws -> Int8
    open func decode(_ type: Int16.Type, forKey key: Key) throws -> Int16
    open func decode(_ type: Int32.Type, forKey key: Key) throws -> Int32
    open func decode(_ type: Int64.Type, forKey key: Key) throws -> Int64
    open func decode(_ type: UInt.Type, forKey key: Key) throws -> UInt
    open func decode(_ type: UInt8.Type, forKey key: Key) throws -> UInt8
    open func decode(_ type: UInt16.Type, forKey key: Key) throws -> UInt16
    open func decode(_ type: UInt32.Type, forKey key: Key) throws -> UInt32
    open func decode(_ type: UInt64.Type, forKey key: Key) throws -> UInt64
    open func decode(_ type: Float.Type, forKey key: Key) throws -> Float
    open func decode(_ type: Double.Type, forKey key: Key) throws -> Double
    open func decode(_ type: String.Type, forKey key: Key) throws -> String
    open func decode(_ type: Data.Type, forKey key: Key) throws -> Data
    open func decode<Value : Codable>(_ type: Value.Type, forKey key: Key) throws -> Value

    /// Decodes a value of the given type for the given key, if present.
    ///
    /// This method returns `nil` if the container does not have a value associated with `key`, or if the value is null. The difference between these states can be disambiguated with a `contains(_:)` call.
    ///
    /// - parameter type: The type of value to decode.
    /// - parameter key: The key that the decoded value is associated with.
    /// - returns: A decoded value of the requested type, or `nil` if the `Decoder` does not have an entry associated with the given key, or if the value is a null value.
    /// - throws: `CocoaError.coderTypeMismatch` if the encountered encoded value is not convertible to the requested type.
    open func decodeIfPresent(_ type: Bool.Type, forKey key: Key) throws -> Bool?
    open func decodeIfPresent(_ type: Int.Type, forKey key: Key) throws -> Int?
    open func decodeIfPresent(_ type: Int8.Type, forKey key: Key) throws -> Int8?
    open func decodeIfPresent(_ type: Int16.Type, forKey key: Key) throws -> Int16?
    open func decodeIfPresent(_ type: Int32.Type, forKey key: Key) throws -> Int32?
    open func decodeIfPresent(_ type: Int64.Type, forKey key: Key) throws -> Int64?
    open func decodeIfPresent(_ type: UInt.Type, forKey key: Key) throws -> UInt?
    open func decodeIfPresent(_ type: UInt8.Type, forKey key: Key) throws -> UInt8?
    open func decodeIfPresent(_ type: UInt16.Type, forKey key: Key) throws -> UInt16?
    open func decodeIfPresent(_ type: UInt32.Type, forKey key: Key) throws -> UInt32?
    open func decodeIfPresent(_ type: UInt64.Type, forKey key: Key) throws -> UInt64?
    open func decodeIfPresent(_ type: Float.Type, forKey key: Key) throws -> Float?
    open func decodeIfPresent(_ type: Double.Type, forKey key: Key) throws -> Double?
    open func decodeIfPresent(_ type: String.Type, forKey key: Key) throws -> String?
    open func decodeIfPresent(_ type: Data.Type, forKey key: Key) throws -> Data?
    open func decodeIfPresent<Value : Codable>(_ type: Value.Type, forKey key: Key) throws -> Value?

    /// The path of coding keys taken to get to this point in decoding.
    open var codingKeyContext: [CodingKey]
}
These encode(_:forKey:) and decode(_:forKey:) overloads give strong, static type guarantees about what is encodable (preventing accidental attempts to encode an invalid type), and provide a list of primitive types which are common to all encoders and decoders that users can rely on.

Coming in Swift 4 is the ability to express that "a collection of things which are Codable is Codable" (conditional conformance), allowing collections which we extend (Array, Dictionary, etc.) to fall into these overloads as well.

Encoding Container Types

For some types, the container into which they encode has meaning. Especially when coding for a specific output format (e.g. when communicating with a JSON API), a type may wish to explicitly encode as an array or a dictionary:

// Continuing from before
public protocol Encoder {
    /// Populates `self` with an encoding container of the given type and returns it, keyed by the given key type.
    ///
    /// A default implementation of `Encoder.container(keyedBy:)` calls this method with a container type of `.default`.
    ///
    /// - parameter keyType: The key type to use for the container.
    /// - parameter containerType: The container type to create.
    /// - returns: A new keyed encoding container.
    /// - precondition: May not be called after a previous `self.container(keyedBy:)` call of a different `EncodingContainerType`.
    /// - precondition: May not be called after a value has been encoded through a prior `self.singleValueContainer()` call.
    func container<Key : CodingKey>(keyedBy keyType: Key.Type, type containerType: EncodingContainerType) -> KeyedEncodingContainer<Key>
}

/// An `EncodingContainerType` specifies the type of container an `Encoder` should use to store values.
public enum EncodingContainerType {
    /// The `Encoder`'s preferred container type; equivalent to either `.array` or `.dictionary` as appropriate for the encoder.
    case `default`

    /// Explicitly requests the use of an array to store encoded values.
    case array

    /// Explicitly requests the use of a dictionary to store encoded values.
    case dictionary
}
Single Value Containers

For other types, an array or dictionary container may not even make sense (e.g. values which are RawRepresentable as a single primitive value). Those types may encode and decode directly as a single value, instead of requesting a keyed container:

/// A `SingleValueEncodingContainer` is a container which can support the storage and direct encoding of a single non-keyed value.
public protocol SingleValueEncodingContainer {
    /// Encodes a single value of the given type.
    ///
    /// - parameter value: The value to encode.
    /// - throws: `CocoaError.coderInvalidValue` if the given value is invalid in the current context for this format.
    /// - precondition: May not be called after a previous `self.encode(_:)` call.
    func encode(_ value: Bool) throws
    func encode(_ value: Int) throws
    func encode(_ value: Int8) throws
    func encode(_ value: Int16) throws
    func encode(_ value: Int32) throws
    func encode(_ value: Int64) throws
    func encode(_ value: UInt) throws
    func encode(_ value: UInt8) throws
    func encode(_ value: UInt16) throws
    func encode(_ value: UInt32) throws
    func encode(_ value: UInt64) throws
    func encode(_ value: Float) throws
    func encode(_ value: Double) throws
    func encode(_ value: String) throws
    func encode(_ value: Data) throws
}

/// A `SingleValueDecodingContainer` is a container which can support the storage and direct decoding of a single non-keyed value.
public protocol SingleValueDecodingContainer {
    /// Decodes a single value of the given type.
    ///
    /// - parameter type: The type to decode as.
    /// - returns: A value of the requested type.
    /// - throws: `CocoaError.coderTypeMismatch` if the encountered encoded value cannot be converted to the requested type.
    func decode(_ type: Bool.Type) throws -> Bool
    func decode(_ type: Int.Type) throws -> Int
    func decode(_ type: Int8.Type) throws -> Int8
    func decode(_ type: Int16.Type) throws -> Int16
    func decode(_ type: Int32.Type) throws -> Int32
    func decode(_ type: Int64.Type) throws -> Int64
    func decode(_ type: UInt.Type) throws -> UInt
    func decode(_ type: UInt8.Type) throws -> UInt8
    func decode(_ type: UInt16.Type) throws -> UInt16
    func decode(_ type: UInt32.Type) throws -> UInt32
    func decode(_ type: UInt64.Type) throws -> UInt64
    func decode(_ type: Float.Type) throws -> Float
    func decode(_ type: Double.Type) throws -> Double
    func decode(_ type: String.Type) throws -> String
    func decode(_ type: Data.Type) throws -> Data
}

// Continuing example from before; below is automatically generated by the compiler if no customization is needed.
public enum Animal : Int, Codable {
    public func encode(to encoder: Encoder) throws {
        // Encode as a single value; no keys.
        try encoder.singleValueContainer.encode(self.rawValue)
    }

    public init(from decoder: Decoder) throws {
        // Decodes as a single value; no keys.
        let intValue = try decoder.singleValueContainer().decode(Int.self)
        if let value = Self(rawValue: intValue) {
            self = value
        } else {
            throw CocoaError.error(.coderReadCorrupt)
        }
    }
}
In the example given above, since Animal uses a single value container, [.chicken, .dog, .cow, .turkey, .dog, .chicken, .cow, .turkey, .dog]would encode directly as [1, 2, 4, 3, 2, 1, 4, 3, 2].

Nesting

In practice, some types may also need to control how data is nested within their container, or potentially nest other containers within their container. Keyed containers allow this by returning nested containers of differing key types:

// Continuing from before
open class KeyedEncodingContainer<Key : CodingKey> {
    /// Stores an encoding container for the given key and returns it.
    ///
    /// - parameter keyType: The key type to use for the container.
    /// - parameter containerType: The container type to create.
    /// - parameter key: The key to encode the container for.
    /// - returns: A new keyed encoding container.
    open func nestedContainer<NestedKey : CodingKey>(keyedBy keyType: NestedKey.Type, type containerType: EncodingContainerType, forKey key: Key) -> KeyedEncodingContainer<NestedKey>
}

open class KeyedDecodingContainer<Key : CodingKey> {
    /// Returns the data stored for the given key as represented in a container keyed by the given key type.
    ///
    /// - parameter type: The key type to use for the container.
    /// - parameter key: The key that the nested container is associated with.
    /// - returns: A keyed decoding container view into `self`.
    /// - throws: `CocoaError.coderTypeMismatch` if the encountered stored value is not a container.
    open func nestedContainer<NestedKey : CodingKey>(keyedBy type: NestedKey.Type, forKey key: Key) throws -> KeyedDecodingContainer<NestedKey>
}
This can be common when coding against specific external data representations:

// User type for interfacing with a specific JSON API. JSON API expects encoding as {"id": ..., "properties": {"name": ..., "timestamp": ...}}. Swift type differs from encoded type, and encoding needs to match a spec:
struct Record : Codable {
    // We care only about these values from the JSON payload
    let id: Int
    let name: String
    let timestamp: Double

    // ...

    private enum Keys : CodingKey {
        case id
        case properties
    }

    private enum PropertiesKeys : CodingKey {
        case name
        case timestamp
    }

    public func encode(to encoder: Encoder) throws {
        let container = encoder.container(keyedBy: Keys.self, type: .dictionary)
        try container.encode(id, forKey: .id)

        // Set a dictionary for the "properties" key
        let nested = container.nestedContainer(keyedBy: PropertiesKeys.self, type: .dictionary, forKey: .properties)
        try nested.encode(name, forKey: .name)
        try nested.encode(timestamp, forKey: .timestamp)
    }

    public init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: Keys.self)
        id = try container.decode(Int.self, forKey: .id)

        let nested = try container.nestedContainer(keyedBy: PropertiesKeys.self, forKey: .properties)
        name = try nested.decode(String.self, forKey: .name)
        timestamp = try nested.decode(Double.self, forKey: .timestamp)
    }
}
Inheritance

Inheritance in this system is supported much like it is with NSCoding — on encoding, objects which inherit from a type that is Codable encode super using their encoder, and pass a decoder to super.init(from:) on decode. With the existing NSCoding API, this is most often done like so, by convention:

- (void)encodeWithCoder:(NSCoder *)encoder {
    [super encodeWithCoder:encoder];
    // ... encode properties
}

- (instancetype)initWithCoder:(NSCoder *)decoder {
    if ((self = [super initWithCoder:decoder])) {
        // ... decode properties
    }

    return self;
}
In practice, this approach means that the properties of self and the properties of super get encoded into the same container: if self encodes values for keys "a", "b", and "c", and super encodes "d", "e", and "f", the resulting object is archived as {"a": ..., "b": ..., "c": ..., "d": ..., "e": ..., "f": ...}. This approach has two drawbacks:

Things which self encodes may overwrite super's (or vice versa, depending on when -[super encodeWithCoder:] is called
self and super may not encode into different container types (e.g. self in a sequential fashion, and super in a keyed fashion)
The second point is not an issue for NSKeyedArchiver, since all values encode with keys (sequentially coded elements get autogenerated keys). This proposed API, however, allows for self and super to explicitly request conflicting containers (.arrayand .dictionary, which may not be mixed, depending on the data format).

To remedy both of these points, we adopt a new convention for inheritance-based coding — encoding super as a sub-object of self:

public class MyCodable : SomethingCodable {
    public func encode(to encoder: Encoder) throws {
        let container = encoder.container(keyedBy: CodingKeys.self)
        // ... encode some properties

        // superEncoder() gives `super` a nested container to encode into (for
        // a predefined key).
        try super.encode(to: container.superEncoder())
    }

    public init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        // ... decode some properties

        // Allow `super` to decode from the nested container.
        try super.init(from: container.superDecoder())
    }
}
If a shared container is desired, it is still possible to call super.encode(to: encoder) and super.init(from: decoder), but we recommend the safer containerized option.

superEncoder() and superDecoder() are provided on KeyedEncodingContainer and KeyedDecodingContainer to provide handles to containers for super to use. While users may specify a custom key to encode super with, the default behavior is to use a key with a stringValue of "super" and an intValue of 0:

// Continuing from before
open class KeyedEncodingContainer<Key : CodingKey> {
    /// Stores a new nested container for the default `super` key and returns a new `Encoder` instance for encoding `super` into that container.
    ///
    /// Equivalent to calling `superEncoder(forKey:)` with `Key(stringValue: "super", intValue: 0)`.
    ///
    /// - returns: A new `Encoder` to pass to `super.encode(to:)`.
    open func superEncoder() -> Encoder

    /// Stores a new nested container for the given key and returns a new `Encoder` instance for encoding `super` into that container.
    ///
    /// - parameter key: The key to encode `super` for.
    /// - returns: A new `Encoder` to pass to `super.encode(to:)`.
    /// - precondition: The key must have a `stringValue` or `intValue` appropriate for the encoding container type.
    open func superEncoder(forKey key: Key) -> Encoder
}

open class KeyedDecodingContainer<Key : CodingKey> {
    /// Returns a `Decoder` instance for decoding `super` from the container associated with the default `super` key.
    ///
    /// Equivalent to calling `superDecoder(forKey:)` with `Key(stringValue: "super", intValue: 0)`.
    ///
    /// - returns: A new `Decoder` to pass to `super.init(from:)`.
    /// - throws: `CocoaError.coderValueNotFound` if `self` does not have an entry for the default `super` key, or if the stored value is null.
    open func superDecoder() throws -> Decoder

    /// Returns a `Decoder` instance for decoding `super` from the container associated with the given key.
    ///
    /// - parameter key: The key to decode `super` for.
    /// - returns: A new `Decoder` to pass to `super.init(from:)`.
    /// - throws: `CocoaError.coderValueNotFound` if `self` does not have an entry for the given key, or if the stored value is null.
    open func superDecoder(forKey key: Key) throws -> Decoder
}
Primitive Codable Conformance

The encoding container types offer overloads for working with and processing the API's primitive types (String, Int, Double, etc.). However, for ease of implementation (both in this API and others), it can be helpful for these types to conform to Codable themselves. Thus, along with these overloads, we will offer Codableconformance on these types:

extension Bool : Codable {
    public init(from decoder: Decoder) throws {
        self = try decoder.singleValueContainer().decode(Bool.self)
    }

    public func encode(to encoder: Encoder) throws {
        try encoder.singleValueContainer().encode( self)
    }
}

// Repeat for others...
This conformance allows one to write functions which accept Codable types without needing specific overloads for the fifteen primitive types as well. This also simplifies conditional conformance (e.g. expressing "extension Array : Codable where Element : Codable") by removing the need for additional explicit conformances for these types.

Since Swift's function overload rules prefer more specific functions over generic functions, the specific overloads are chosen where possible (e.g. encode("Hello, world!", forKey: .greeting) will choose encode(_: String, forKey: Key) over encode<T : Codable>(_: T, forKey: Key)). This maintains performance over dispatching through the Codable existential, while allowing for the flexibility of fewer overloads where applicable.

Additional Extensions

Along with the primitive Codable conformance above, extensions on CodableRawRepresentable types whose RawValue is a primitive types will provide default implementations for encoding and decoding:

public extension RawRepresentable where RawValue == Bool, Self : Codable {
    public init(from decoder: Decoder) throws {
        let decoded = try decoder.singleValueContainer().decode(RawValue.self)
        guard let value = Self(rawValue: decoded) else {
            throw CocoaError.error(.coderReadCorrupt)
        }

        self = value
    }

    public func encode(to encoder: Encoder) throws {
        try encoder.singleValueContainer().encode(self.rawValue)
    }
}

// Repeat for others...
This allows for trivial Codable conformance of enum types (and manual RawRepresentable implementations) with primitive backing.

Source compatibility
This proposal is additive — existing code will not have to change due to this API addition. This implementation can be made available in both Swift 4 and the Swift 3 compatibility mode.

Effect on ABI stability
The addition of this API will not be an ABI-breaking change. However, this will add limitations for changes in future versions of Swift, as parts of the API will have to remain unchanged between versions of Swift (barring some additions, discussed below).

Effect on API resilience
Much like new API added to the standard library, once added, many changes to this API will be ABI- and source-breaking changes. In particular, changes which change the types or names of methods or arguments, add required methods on protocols or classes, or remove supplied default implementations will break client behavior.

The following protocols and classes may not have methods added to them without providing default implementations:

Codable
CodingKey
Encoder
SingleValueEncodingContainer
KeyedEncodingContainer
Decoder
SingleValueDecodingContainer
KeyedDecodingContainer
The following classes may not remove existing default implementations:

KeyedEncodingContainer
KeyedDecodingContainer
Various extensions to Swift primitive types (Bool, Int, Double, etc.) and to RawRepresentable types (where RawValue == Bool, == Int, == Double, etc.) may also not be removed.

In general, changes to the proposed types will be restricted as described in the library evolution document <https://github.com/apple/swift/blob/master/docs/LibraryEvolution.rst> in the Swift repository.


(Joe Groff) #2

Congrats on getting this out! A question from the field:

https://twitter.com/mdiep/status/842178457115230210 Why does the Swift Serialization API proposal use abstract base classes?

-Joe

···

On Mar 15, 2017, at 3:40 PM, Itai Ferber via swift-evolution <swift-evolution@swift.org> wrote:

Hi everyone,

The following introduces a new Swift-focused archival and serialization API as part of the Foundation framework. We’re interested in improving the experience and safety of performing archival and serialization, and are happy to receive community feedback on this work.
Because of the length of this proposal, the Appendix and Alternatives Considered sections have been omitted here, but are available in the full proposal on the swift-evolution repo. The full proposal also includes an Unabridged API for further consideration.

Without further ado, inlined below.

— Itai

Swift Archival & Serialization
  • Proposal: SE-NNNN
  • Author(s): Itai Ferber, Michael LeHew, Tony Parker
  • Review Manager: TBD
  • Status: Awaiting review
  • Associated PRs:
    • #8124
    • #8125
Introduction
Foundation's current archival and serialization APIs (NSCoding, NSJSONSerialization, NSPropertyListSerialization, etc.), while fitting for the dynamism of Objective-C, do not always map optimally into Swift. This document lays out the design of an updated API that improves the developer experience of performing archival and serialization in Swift.

Specifically:

  • It aims to provide a solution for the archival of Swift struct and enum types
  • It aims to provide a more type-safe solution for serializing to external formats, such as JSON and plist
Motivation
The primary motivation for this proposal is the inclusion of native Swift enum and struct types in archival and serialization. Currently, developers targeting Swift cannot participate in NSCoding without being willing to abandon enum and structtypes — NSCoding is an @objc protocol, conformance to which excludes non-class types. This is can be limiting in Swift because small enums and structs can be an idiomatic approach to model representation; developers who wish to perform archival have to either forgo the Swift niceties that constructs like enumsprovide, or provide an additional compatibility layer between their "real" types and their archivable types.

Secondarily, we would like to refine Foundation's existing serialization APIs (NSJSONSerialization and NSPropertyListSerialization) to better match Swift's strong type safety. From experience, we find that the conversion from the unstructured, untyped data of these formats into strongly-typed data structures is a good fit for archival mechanisms, rather than taking the less safe approach that 3rd-party JSON conversion approaches have taken (described further in an appendix below).

We would like to offer a solution to these problems without sacrificing ease of use or type safety.

Agenda
This proposal is the first stage of three that introduce different facets of a whole Swift archival and serialization API:

  • This proposal describes the basis for this API, focusing on the protocols that users adopt and interface with
  • The next stage will propose specific API for new encoders
  • The final stage will discuss how this new API will interop with NSCoding as it is today
SE-NNNN provides stages 2 and 3.

Proposed solution
We will be introducing the following new types:

  • protocol Codable: Adopted by types to opt into archival. Conformance may be automatically derived in cases where all properties are also Codable.
  • protocol CodingKey: Adopted by types used as keys for keyed containers, replacing String keys with semantic types. Conformance may be automatically derived in most cases.
  • protocol Encoder: Adopted by types which can take Codable values and encode them into a native format.
    • class KeyedEncodingContainer<Key : CodingKey>: Subclasses of this type provide a concrete way to store encoded values by CodingKey. Types adopting Encoder should provide subclasses of KeyedEncodingContainer to vend.
    • protocol SingleValueEncodingContainer: Adopted by types which provide a concrete way to store a single encoded value. Types adopting Encoder should provide types conforming to SingleValueEncodingContainer to vend (but in many cases will be able to conform to it themselves).
  • protocol Decoder: Adopted by types which can take payloads in a native format and decode Codable values out of them.
    • class KeyedDecodingContainer<Key : CodingKey>: Subclasses of this type provide a concrete way to retrieve encoded values from storage by CodingKey. Types adopting Decoder should provide subclasses of KeyedDecodingContainer to vend.
    • protocol SingleValueDecodingContainer: Adopted by types which provide a concrete way to retrieve a single encoded value from storage. Types adopting Decoder should provide types conforming to SingleValueDecodingContainer to vend (but in many cases will be able to conform to it themselves).
For end users of this API, adoption will primarily involve the Codable and CodingKey protocols. In order to participate in this new archival system, developers must add Codable conformance to their types:

// If all properties are Codable, implementation is automatically derived:
public struct Location : Codable {

public let latitude: Double

public let longitude: Double
}

public enum Animal : Int, Codable {

case chicken = 1

case
dog
    
case
turkey
    
case
cow

}

public struct Farm : Codable {

public let name: String

public let location: Location

public let animals: [Animal]
}
With developer participation, we will offer encoders and decoders (described in SE-NNNN, not here) that take advantage of this conformance to offer type-safe serialization of user models:

let farm = Farm(name: "Old MacDonald's Farm",

                location
: Location(latitude: 51.621648, longitude: 0.269273),

                animals
: [.chicken, .dog, .cow, .turkey, .dog, .chicken, .cow, .turkey, .dog])
let payload: Data = try JSONEncoder().encode(farm)

do {

let farm = try JSONDecoder().decode(Farm.self, from: payload)

// Extracted as user types:

let coordinates = "\(farm.location.latitude, farm.location.longitude)"
} catch {

// Encountered error during deserialization
}
This gives developers access to their data in a type-safe manner and a recognizable interface.

Detailed design
To support user types, we expose the Codable protocol:

/// Conformance to `Codable` indicates that a type can marshal itself into and out of an external representation.
public protocol Codable {

/// Initializes `self` by decoding from `decoder`.

///

/// - parameter decoder: The decoder to read data from.

/// - throws: An error if reading from the decoder fails, or if read data is corrupted or otherwise invalid.

init(from decoder: Decoder) throws

/// Encodes `self` into the given encoder.

///

/// If `self` fails to encode anything, `encoder` will encode an empty `.default` container in its place.

///

/// - parameter encoder: The encoder to write data to.

/// - throws: An error if any values are invalid for `encoder`'s format.

func encode(to encoder: Encoder) throws
}
By adopting Codable, user types opt in to this archival system.

Structured types (i.e. types which encode as a collection of properties) encode and decode their properties in a keyed manner. Keys may be String-convertible or Int-convertible (or both), and user types which have properties should declare semantic key enums which map keys to their properties. Keys must conform to the CodingKey protocol:

/// Conformance to `CodingKey` indicates that a type can be used as a key for encoding and decoding.
public protocol CodingKey {

/// The string to use in a named collection (e.g. a string-keyed dictionary).

var stringValue: String? { get }

/// Initializes `self` from a string.

///

/// - parameter stringValue: The string value of the desired key.

/// - returns: An instance of `Self` from the given string, or `nil` if the given string does not correspond to any instance of `Self`.

init?(stringValue: String)

/// The int to use in an indexed collection (e.g. an int-keyed dictionary).

var intValue: Int? { get }

/// Initializes `self` from an integer.

///

/// - parameter intValue: The integer value of the desired key.

/// - returns: An instance of `Self` from the given integer, or `nil` if the given integer does not correspond to any instance of `Self`.

init?(intValue: Int)
}
For most types, String-convertible keys are a reasonable default; for performance, however, Int-convertible keys are preferred, and Encoders may choose to make use of Ints over Strings. Framework types should provide keys which have both for flexibility and performance across different types of Encoders. It is generally an error to provide a key which has neither a stringValue nor an intValue.

By default, CodingKey conformance can be derived for enums which have either String or Int backing:

enum Keys1 : CodingKey {

case a // (stringValue: "a", intValue: nil)

case b // (stringValue: "b", intValue: nil)
}

enum Keys2 : String, CodingKey {

case c = "foo" // (stringValue: "foo", intValue: nil)

case d // (stringValue: "d", intValue: nil)
}

enum Keys3 : Int, CodingKey {

case e = 4 // (stringValue: "e", intValue: 4)

case f // (stringValue: "f", intValue: 5)

case g = 9 // (stringValue: "g", intValue: 9)
}
Coding keys which are not enums, have associated values, or have other raw representations must implement these methods manually.

In addition to automatic CodingKey conformance derivation for enums, Codableconformance can be automatically derived for certain types as well:

  • Types whose properties are all either Codable or primitive get an automatically derived String-backed CodingKeys enum mapping properties to case names
  • Types falling into (1) and types which provide a CodingKeys enum (directly or via a typealias) whose case names map to properties which are all Codableget automatic derivation of init(from:) and encode(to:) using those properties and keys. Types may choose to provide a custom init(from:) or encode(to:) (or both); whichever they do not provide will be automatically derived
  • Types which fall into neither (1) nor (2) will have to provide a custom key type and provide their own init(from:) and encode(to:)
Many types will either allow for automatic derivation of all codability (1), or provide a custom key subset and take advantage of automatic method derivation (2).

Encoding and Decoding

Types which are encodable encode their data into a container provided by their Encoder:

/// An `Encoder` is a type which can encode values into a native format for external representation.
public protocol Encoder {

/// Populates `self` with an encoding container (of `.default` type) and returns it, keyed by the given key type.

///

/// - parameter type: The key type to use for the container.

/// - returns: A new keyed encoding container.

/// - precondition: May not be called after a previous `self.container(keyedBy:)` call of a different `EncodingContainerType`.

/// - precondition: May not be called after a value has been encoded through a prior `self.singleValueContainer()` call.

func container<Key : CodingKey>(keyedBy type: Key.Type) -> KeyedEncodingContainer<Key>

/// Returns an encoding container appropriate for holding a single primitive value.

///

/// - returns: A new empty single value container.

/// - precondition: May not be called after a prior `self.container(keyedBy:)` call.

/// - precondition: May not be called after a value has been encoded through a previous `self.singleValueContainer()` call.

func singleValueContainer() -> SingleValueEncodingContainer

/// The path of coding keys taken to get to this point in encoding.

var codingKeyContext: [CodingKey] { get }
}

// Continuing examples from before; below is automatically generated by the compiler if no customization is needed.
public struct Location : Codable {

private enum CodingKeys : CodingKey {

case
latitutude
        
case
longitude
    
}

public func encode(to encoder: Encoder) throws {

// Generic keyed encoder gives type-safe key access: cannot encode with keys of the wrong type.

let container = encoder.container(keyedBy: CodingKeys.self)

// The encoder is generic on the key -- free key autocompletion here.

try container.encode(latitude, forKey: .latitude)

try container.encode(longitude, forKey: .longitude)

}
}

public struct Farm : Codable {

private enum CodingKeys : CodingKey {

case
name
        
case
location
        
case
animals
    
}

public func encode(to encoder: Encoder) throws {

let container = encoder.container(keyedBy: CodingKeys.self)

try container.encode(name, forKey: .name)

try container.encode(location, forKey: .location)

try container.encode(animals, forKey: .animals)

}
}
Similarly, decodable types initialize from data read from their Decoder's container:

/// A `Decoder` is a type which can decode values from a native format into in-memory representations.
public protocol Decoder {

/// Returns the data stored in `self` as represented in a container keyed by the given key type.

///

/// - parameter type: The key type to use for the container.

/// - returns: A keyed decoding container view into `self`.

/// - throws: `CocoaError.coderTypeMismatch` if the encountered stored value is not a keyed container.

func container<Key : CodingKey>(keyedBy type: Key.Type) throws -> KeyedDecodingContainer<Key>

/// Returns the data stored in `self` as represented in a container appropriate for holding a single primitive value.

///

/// - returns: A single value container view into `self`.

/// - throws: `CocoaError.coderTypeMismatch` if the encountered stored value is not a single value container.

func singleValueContainer() throws -> SingleValueDecodingContainer

/// The path of coding keys taken to get to this point in decoding.

var codingKeyContext: [CodingKey] { get }
}

// Continuing examples from before; below is automatically generated by the compiler if no customization is needed.
public struct Location : Codable {

public init(from decoder: Decoder) throws {

let container = try decoder.container(keyedBy: CodingKeys.self)

        latitude
= try container.decode(Double.self, forKey: .latitude)

        longitude
= try container.decode(Double.self, forKey: .longitude)

}
}

public struct Farm : Codable {

public init(from decoder: Decoder) throws {

let container = try decoder.container(keyedBy: CodingKeys.self)

        name
= try container.decode(String.self, forKey: .name)

        location
= try container.decode(Location.self, forKey: .location)

        animals
= try container.decode([Animal].self, forKey: .animals)

}
}
Keyed Encoding Containers

Keyed encoding containers are the primary interface that most Codable types interact with for encoding and decoding. Through these, Codable types have strongly-keyed access to encoded data by using keys that are semantically correct for the operations they want to express.

Since semantically incompatible keys will rarely (if ever) share the same key type, it is impossible to mix up key types within the same container (as is possible with Stringkeys), and since the type is known statically, keys get autocompletion by the compiler.

/// `KeyedEncodingContainer` is a generic abstract base class that provides a view into an `Encoders` storage and is used to hold the encoded properties of a `Codable` type.
///
/// Encoders should provide subclasses of `KeyedEncodingContainer` for their format.

open
class KeyedEncodingContainer<Key : CodingKey> {

/// Encodes the given value for the given key.

///

/// - parameter value: The value to encode.

/// - parameter key: The key to associate the value with.

/// - throws: `CocoaError.coderInvalidValue` if the given value is invalid in the current context for this format.

/// - precondition: The key must have a `stringValue` or `intValue` appropriate for the encoding container type.

    open
func encode<Value : Codable>(_ value: Value?, forKey key: Key) throws

/// Encodes the given value for the given key.

///

/// - parameter value: The value to encode.

/// - parameter key: The key to associate the value with.

/// - throws: `CocoaError.coderInvalidValue` if the given value is invalid in the current context for this format.

/// - precondition: The key must have a `stringValue` or `intValue` appropriate for the encoding container type.

    open
func encode(_ value: Bool?, forKey key: Key) throws

    open
func encode(_ value: Int?, forKey key: Key) throws

    open
func encode(_ value: Int8?, forKey key: Key) throws

    open
func encode(_ value: Int16?, forKey key: Key) throws

    open
func encode(_ value: Int32?, forKey key: Key) throws

    open
func encode(_ value: Int64?, forKey key: Key) throws

    open
func encode(_ value: UInt?, forKey key: Key) throws

    open
func encode(_ value: UInt8?, forKey key: Key) throws

    open
func encode(_ value: UInt16?, forKey key: Key) throws

    open
func encode(_ value: UInt32?, forKey key: Key) throws

    open
func encode(_ value: UInt64?, forKey key: Key) throws

    open
func encode(_ value: Float?, forKey key: Key) throws

    open
func encode(_ value: Double?, forKey key: Key) throws

    open
func encode(_ value: String?, forKey key: Key) throws

    open
func encode(_ value: Data?, forKey key: Key) throws

/// Encodes the given object weakly for the given key.

///

/// For `Encoder`s that implement this functionality, this will only encode the given object and associate it with the given key if it encoded unconditionally elsewhere in the archive (either previously or in the future).

///

/// For formats which don't support this feature, the default implementation encodes the given object unconditionally.

///

/// - parameter object: The object to encode.

/// - parameter key: The key to associate the object with.

/// - throws: `CocoaError.coderInvalidValue` if the given value is invalid in the current context for this format.

/// - precondition: The key must have a `stringValue` or `intValue` appropriate for the encoding container type.

    open
func encodeWeak<Object : AnyObject & Codable>(_ object: Object?, forKey key: Key) throws

/// The path of coding keys taken to get to this point in encoding.

    open
var codingKeyContext: [CodingKey]
}

/// `KeyedDecodingContainer` is a generic abstract base class that provides a view into an `Decoders` storage and is used to hold the encoded properties of a `Codable` type.
///
/// Decoders should provide subclasses of `KeyedDecodingContainer` for their format.

open
class KeyedDecodingContainer<Key : CodingKey> {

/// All the keys the `Decoder` has for this container.

///

/// Different keyed containers from the same `Decoder` may return different keys here; it is possible to encode with multiple key types which are not convertible to one another. This should report all keys present which are convertible to the requested type.

    open
var allKeys: [Key]

/// Returns whether the `Decoder` contains a value associated with the given key.

///

/// The value associated with the given key may be a null value as appropriate for the data format.

///

/// - parameter key: The key to search for.

/// - returns: Whether the `Decoder` has an entry for the given key.

    open
func contains(_ key: Key) -> Bool

/// Decodes a value of the given type for the given key.

///

/// A default implementation is given for these types which calls into the abstract `decodeIfPresent` implementations below.

///

/// - parameter type: The type of value to decode.

/// - parameter key: The key that the decoded value is associated with.

/// - returns: A value of the requested type, if present for the given key and convertible to the requested type.

/// - throws: `CocoaError.coderTypeMismatch` if the encountered encoded value is not convertible to the requested type.

/// - throws: `CocoaError.coderValueNotFound` if `self` does not have an entry for the given key or if the value is null.

    open
func decode(_ type: Bool.Type, forKey key: Key) throws -> Bool

    open
func decode(_ type: Int.Type, forKey key: Key) throws -> Int

    open
func decode(_ type: Int8.Type, forKey key: Key) throws -> Int8

    open
func decode(_ type: Int16.Type, forKey key: Key) throws -> Int16

    open
func decode(_ type: Int32.Type, forKey key: Key) throws -> Int32

    open
func decode(_ type: Int64.Type, forKey key: Key) throws -> Int64

    open
func decode(_ type: UInt.Type, forKey key: Key) throws -> UInt

    open
func decode(_ type: UInt8.Type, forKey key: Key) throws -> UInt8

    open
func decode(_ type: UInt16.Type, forKey key: Key) throws -> UInt16

    open
func decode(_ type: UInt32.Type, forKey key: Key) throws -> UInt32

    open
func decode(_ type: UInt64.Type, forKey key: Key) throws -> UInt64

    open
func decode(_ type: Float.Type, forKey key: Key) throws -> Float

    open
func decode(_ type: Double.Type, forKey key: Key) throws -> Double

    open
func decode(_ type: String.Type, forKey key: Key) throws -> String

    open
func decode(_ type: Data.Type, forKey key: Key) throws -> Data

    open
func decode<Value : Codable>(_ type: Value.Type, forKey key: Key) throws -> Value

/// Decodes a value of the given type for the given key, if present.

///

/// This method returns `nil` if the container does not have a value associated with `key`, or if the value is null. The difference between these states can be disambiguated with a `contains(_:)` call.

///

/// - parameter type: The type of value to decode.

/// - parameter key: The key that the decoded value is associated with.

/// - returns: A decoded value of the requested type, or `nil` if the `Decoder` does not have an entry associated with the given key, or if the value is a null value.

/// - throws: `CocoaError.coderTypeMismatch` if the encountered encoded value is not convertible to the requested type.

    open
func decodeIfPresent(_ type: Bool.Type, forKey key: Key) throws -> Bool?

    open
func decodeIfPresent(_ type: Int.Type, forKey key: Key) throws -> Int?

    open
func decodeIfPresent(_ type: Int8.Type, forKey key: Key) throws -> Int8?

    open
func decodeIfPresent(_ type: Int16.Type, forKey key: Key) throws -> Int16?

    open
func decodeIfPresent(_ type: Int32.Type, forKey key: Key) throws -> Int32?

    open
func decodeIfPresent(_ type: Int64.Type, forKey key: Key) throws -> Int64?

    open
func decodeIfPresent(_ type: UInt.Type, forKey key: Key) throws -> UInt?

    open
func decodeIfPresent(_ type: UInt8.Type, forKey key: Key) throws -> UInt8?

    open
func decodeIfPresent(_ type: UInt16.Type, forKey key: Key) throws -> UInt16?

    open
func decodeIfPresent(_ type: UInt32.Type, forKey key: Key) throws -> UInt32?

    open
func decodeIfPresent(_ type: UInt64.Type, forKey key: Key) throws -> UInt64?

    open
func decodeIfPresent(_ type: Float.Type, forKey key: Key) throws -> Float?

    open
func decodeIfPresent(_ type: Double.Type, forKey key: Key) throws -> Double?

    open
func decodeIfPresent(_ type: String.Type, forKey key: Key) throws -> String?

    open
func decodeIfPresent(_ type: Data.Type, forKey key: Key) throws -> Data?

    open
func decodeIfPresent<Value : Codable>(_ type: Value.Type, forKey key: Key) throws -> Value?

/// The path of coding keys taken to get to this point in decoding.

    open
var codingKeyContext: [CodingKey]
}
These encode(_:forKey:) and decode(_:forKey:) overloads give strong, static type guarantees about what is encodable (preventing accidental attempts to encode an invalid type), and provide a list of primitive types which are common to all encoders and decoders that users can rely on.

Coming in Swift 4 is the ability to express that "a collection of things which are Codable is Codable" (conditional conformance), allowing collections which we extend (Array, Dictionary, etc.) to fall into these overloads as well.

Encoding Container Types

For some types, the container into which they encode has meaning. Especially when coding for a specific output format (e.g. when communicating with a JSON API), a type may wish to explicitly encode as an array or a dictionary:

// Continuing from before
public protocol Encoder {

/// Populates `self` with an encoding container of the given type and returns it, keyed by the given key type.

///

/// A default implementation of `Encoder.container(keyedBy:)` calls this method with a container type of `.default`.

///

/// - parameter keyType: The key type to use for the container.

/// - parameter containerType: The container type to create.

/// - returns: A new keyed encoding container.

/// - precondition: May not be called after a previous `self.container(keyedBy:)` call of a different `EncodingContainerType`.

/// - precondition: May not be called after a value has been encoded through a prior `self.singleValueContainer()` call.

func container<Key : CodingKey>(keyedBy keyType: Key.Type, type containerType: EncodingContainerType) -> KeyedEncodingContainer<Key>
}

/// An `EncodingContainerType` specifies the type of container an `Encoder` should use to store values.
public enum EncodingContainerType {

/// The `Encoder`'s preferred container type; equivalent to either `.array` or `.dictionary` as appropriate for the encoder.

case `default
`

/// Explicitly requests the use of an array to store encoded values.

case
array

/// Explicitly requests the use of a dictionary to store encoded values.

case
dictionary

}
Single Value Containers

For other types, an array or dictionary container may not even make sense (e.g. values which are RawRepresentable as a single primitive value). Those types may encode and decode directly as a single value, instead of requesting a keyed container:

/// A `SingleValueEncodingContainer` is a container which can support the storage and direct encoding of a single non-keyed value.
public protocol SingleValueEncodingContainer {

/// Encodes a single value of the given type.

///

/// - parameter value: The value to encode.

/// - throws: `CocoaError.coderInvalidValue` if the given value is invalid in the current context for this format.

/// - precondition: May not be called after a previous `self.encode(_:)` call.

func encode(_ value: Bool) throws

func encode(_ value: Int) throws

func encode(_ value: Int8) throws

func encode(_ value: Int16) throws

func encode(_ value: Int32) throws

func encode(_ value: Int64) throws

func encode(_ value: UInt) throws

func encode(_ value: UInt8) throws

func encode(_ value: UInt16) throws

func encode(_ value: UInt32) throws

func encode(_ value: UInt64) throws

func encode(_ value: Float) throws

func encode(_ value: Double) throws

func encode(_ value: String) throws

func encode(_ value: Data) throws
}

/// A `SingleValueDecodingContainer` is a container which can support the storage and direct decoding of a single non-keyed value.
public protocol SingleValueDecodingContainer {

/// Decodes a single value of the given type.

///

/// - parameter type: The type to decode as.

/// - returns: A value of the requested type.

/// - throws: `CocoaError.coderTypeMismatch` if the encountered encoded value cannot be converted to the requested type.

func decode(_ type: Bool.Type) throws -> Bool

func decode(_ type: Int.Type) throws -> Int

func decode(_ type: Int8.Type) throws -> Int8

func decode(_ type: Int16.Type) throws -> Int16

func decode(_ type: Int32.Type) throws -> Int32

func decode(_ type: Int64.Type) throws -> Int64

func decode(_ type: UInt.Type) throws -> UInt

func decode(_ type: UInt8.Type) throws -> UInt8

func decode(_ type: UInt16.Type) throws -> UInt16

func decode(_ type: UInt32.Type) throws -> UInt32

func decode(_ type: UInt64.Type) throws -> UInt64

func decode(_ type: Float.Type) throws -> Float

func decode(_ type: Double.Type) throws -> Double

func decode(_ type: String.Type) throws -> String

func decode(_ type: Data.Type) throws -> Data
}

// Continuing example from before; below is automatically generated by the compiler if no customization is needed.
public enum Animal : Int, Codable {

public func encode(to encoder: Encoder) throws {

// Encode as a single value; no keys.

try encoder.singleValueContainer.encode(self.rawValue)

}

public init(from decoder: Decoder) throws {

// Decodes as a single value; no keys.

let intValue = try decoder.singleValueContainer().decode(Int.self)

if let value = Self(rawValue: intValue) {

self =
value
        
} else {

throw CocoaError.error(.coderReadCorrupt)

}

}
}
In the example given above, since Animal uses a single value container, [.chicken, .dog, .cow, .turkey, .dog, .chicken, .cow, .turkey, .dog]would encode directly as [1, 2, 4, 3, 2, 1, 4, 3, 2].

Nesting

In practice, some types may also need to control how data is nested within their container, or potentially nest other containers within their container. Keyed containers allow this by returning nested containers of differing key types:

// Continuing from before

open
class KeyedEncodingContainer<Key : CodingKey> {

/// Stores an encoding container for the given key and returns it.

///

/// - parameter keyType: The key type to use for the container.

/// - parameter containerType: The container type to create.

/// - parameter key: The key to encode the container for.

/// - returns: A new keyed encoding container.

    open
func nestedContainer<NestedKey : CodingKey>(keyedBy keyType: NestedKey.Type, type containerType: EncodingContainerType, forKey key: Key) -> KeyedEncodingContainer<NestedKey>
}

open
class KeyedDecodingContainer<Key : CodingKey> {

/// Returns the data stored for the given key as represented in a container keyed by the given key type.

///

/// - parameter type: The key type to use for the container.

/// - parameter key: The key that the nested container is associated with.

/// - returns: A keyed decoding container view into `self`.

/// - throws: `CocoaError.coderTypeMismatch` if the encountered stored value is not a container.

    open
func nestedContainer<NestedKey : CodingKey>(keyedBy type: NestedKey.Type, forKey key: Key) throws -> KeyedDecodingContainer<NestedKey>
}
This can be common when coding against specific external data representations:

// User type for interfacing with a specific JSON API. JSON API expects encoding as {"id": ..., "properties": {"name": ..., "timestamp": ...}}. Swift type differs from encoded type, and encoding needs to match a spec:
struct Record : Codable {

// We care only about these values from the JSON payload

let id: Int

let name: String

let timestamp: Double

// ...

private enum Keys : CodingKey {

case
id
        
case
properties
    
}

private enum PropertiesKeys : CodingKey {

case
name
        
case
timestamp
    
}

public func encode(to encoder: Encoder) throws {

let container = encoder.container(keyedBy: Keys.self, type: .dictionary)

try container.encode(id, forKey: .id)

// Set a dictionary for the "properties" key

let nested = container.nestedContainer(keyedBy: PropertiesKeys.self, type: .dictionary, forKey: .properties)

try nested.encode(name, forKey: .name)

try nested.encode(timestamp, forKey: .timestamp)

}

public init(from decoder: Decoder) throws {

let container = try decoder.container(keyedBy: Keys.self)

        id
= try container.decode(Int.self, forKey: .id)

let nested = try container.nestedContainer(keyedBy: PropertiesKeys.self, forKey: .properties)

        name
= try nested.decode(String.self, forKey: .name)

        timestamp
= try nested.decode(Double.self, forKey: .timestamp)

}
}
Inheritance

Inheritance in this system is supported much like it is with NSCoding — on encoding, objects which inherit from a type that is Codable encode super using their encoder, and pass a decoder to super.init(from:) on decode. With the existing NSCoding API, this is most often done like so, by convention:

- (void)encodeWithCoder:(NSCoder *)encoder {

[super encodeWithCoder:encoder];

// ... encode properties
}

- (instancetype)initWithCoder:(NSCoder *)decoder {

if ((self = [super initWithCoder:decoder])) {

// ... decode properties

}

return self;
}
In practice, this approach means that the properties of self and the properties of super get encoded into the same container: if self encodes values for keys "a", "b", and "c", and super encodes "d", "e", and "f", the resulting object is archived as {"a": ..., "b": ..., "c": ..., "d": ..., "e": ..., "f": ...}. This approach has two drawbacks:

  • Things which self encodes may overwrite super's (or vice versa, depending on when -[super encodeWithCoder:] is called
  • self and super may not encode into different container types (e.g. self in a sequential fashion, and super in a keyed fashion)
The second point is not an issue for NSKeyedArchiver, since all values encode with keys (sequentially coded elements get autogenerated keys). This proposed API, however, allows for self and super to explicitly request conflicting containers (.arrayand .dictionary, which may not be mixed, depending on the data format).

To remedy both of these points, we adopt a new convention for inheritance-based coding — encoding super as a sub-object of self:

public class MyCodable : SomethingCodable {

public func encode(to encoder: Encoder) throws {

let container = encoder.container(keyedBy: CodingKeys.self)

// ... encode some properties

// superEncoder() gives `super` a nested container to encode into (for

// a predefined key).

try super.encode(to: container.superEncoder())

}

public init(from decoder: Decoder) throws {

let container = try decoder.container(keyedBy: CodingKeys.self)

// ... decode some properties

// Allow `super` to decode from the nested container.

try super.init(from: container.superDecoder())

}
}
If a shared container is desired, it is still possible to call super.encode(to: encoder) and super.init(from: decoder), but we recommend the safer containerized option.

superEncoder() and superDecoder() are provided on KeyedEncodingContainer and KeyedDecodingContainer to provide handles to containers for super to use. While users may specify a custom key to encode super with, the default behavior is to use a key with a stringValue of "super" and an intValue of 0:

// Continuing from before

open
class KeyedEncodingContainer<Key : CodingKey> {

/// Stores a new nested container for the default `super` key and returns a new `Encoder` instance for encoding `super` into that container.

///

/// Equivalent to calling `superEncoder(forKey:)` with `Key(stringValue: "super", intValue: 0)`.

///

/// - returns: A new `Encoder` to pass to `super.encode(to:)`.

    open
func superEncoder() -> Encoder

/// Stores a new nested container for the given key and returns a new `Encoder` instance for encoding `super` into that container.

///

/// - parameter key: The key to encode `super` for.

/// - returns: A new `Encoder` to pass to `super.encode(to:)`.

/// - precondition: The key must have a `stringValue` or `intValue` appropriate for the encoding container type.

    open
func superEncoder(forKey key: Key) -> Encoder
}

open
class KeyedDecodingContainer<Key : CodingKey> {

/// Returns a `Decoder` instance for decoding `super` from the container associated with the default `super` key.

///

/// Equivalent to calling `superDecoder(forKey:)` with `Key(stringValue: "super", intValue: 0)`.

///

/// - returns: A new `Decoder` to pass to `super.init(from:)`.

/// - throws: `CocoaError.coderValueNotFound` if `self` does not have an entry for the default `super` key, or if the stored value is null.

    open
func superDecoder() throws -> Decoder

/// Returns a `Decoder` instance for decoding `super` from the container associated with the given key.

///

/// - parameter key: The key to decode `super` for.

/// - returns: A new `Decoder` to pass to `super.init(from:)`.

/// - throws: `CocoaError.coderValueNotFound` if `self` does not have an entry for the given key, or if the stored value is null.

    open
func superDecoder(forKey key: Key) throws -> Decoder
}
Primitive Codable Conformance

The encoding container types offer overloads for working with and processing the API's primitive types (String, Int, Double, etc.). However, for ease of implementation (both in this API and others), it can be helpful for these types to conform to Codable themselves. Thus, along with these overloads, we will offer Codableconformance on these types:

extension Bool : Codable {

public init(from decoder: Decoder) throws {

self = try decoder.singleValueContainer().decode(Bool.self)

}

public func encode(to encoder: Encoder) throws {

try encoder.singleValueContainer().encode( self)

}
}

// Repeat for others...
This conformance allows one to write functions which accept Codable types without needing specific overloads for the fifteen primitive types as well. This also simplifies conditional conformance (e.g. expressing "extension Array : Codable where Element : Codable") by removing the need for additional explicit conformances for these types.

Since Swift's function overload rules prefer more specific functions over generic functions, the specific overloads are chosen where possible (e.g. encode("Hello, world!", forKey: .greeting) will choose encode(_: String, forKey: Key) over encode<T : Codable>(_: T, forKey: Key)). This maintains performance over dispatching through the Codable existential, while allowing for the flexibility of fewer overloads where applicable.

Additional Extensions

Along with the primitive Codable conformance above, extensions on CodableRawRepresentable types whose RawValue is a primitive types will provide default implementations for encoding and decoding:

public extension RawRepresentable where RawValue == Bool, Self : Codable {

public init(from decoder: Decoder) throws {

let decoded = try decoder.singleValueContainer().decode(RawValue.self)

guard let value = Self(rawValue: decoded) else {

throw CocoaError.error(.coderReadCorrupt)

}

self =
value
    
}

public func encode(to encoder: Encoder) throws {

try encoder.singleValueContainer().encode(self.rawValue)

}
}

// Repeat for others...
This allows for trivial Codable conformance of enum types (and manual RawRepresentable implementations) with primitive backing.

Source compatibility
This proposal is additive — existing code will not have to change due to this API addition. This implementation can be made available in both Swift 4 and the Swift 3 compatibility mode.

Effect on ABI stability
The addition of this API will not be an ABI-breaking change. However, this will add limitations for changes in future versions of Swift, as parts of the API will have to remain unchanged between versions of Swift (barring some additions, discussed below).

Effect on API resilience
Much like new API added to the standard library, once added, many changes to this API will be ABI- and source-breaking changes. In particular, changes which change the types or names of methods or arguments, add required methods on protocols or classes, or remove supplied default implementations will break client behavior.

The following protocols and classes may not have methods added to them without providing default implementations:

  • Codable
  • CodingKey
  • Encoder
  • SingleValueEncodingContainer
  • KeyedEncodingContainer
  • Decoder
  • SingleValueDecodingContainer
  • KeyedDecodingContainer
The following classes may not remove existing default implementations:

  • KeyedEncodingContainer
  • KeyedDecodingContainer
Various extensions to Swift primitive types (Bool, Int, Double, etc.) and to RawRepresentable types (where RawValue == Bool, == Int, == Double, etc.) may also not be removed.

In general, changes to the proposed types will be restricted as described in the library evolution document in the Swift repository.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Zachary Waldowski) #3

Holy cow. There's much to digest here (so much so that my initial
response, which quoted its content, was denied by the mailing list).
After an initial reading, I don't just want this now, I want it
yesterday. I'm already imaging the encoder/decoders I want to build.
Very exciting.

Best,

  Zachary Waldowski

  zach@waldowski.me


(Brent Royal-Gordon) #4

Hi everyone,

The following introduces a new Swift-focused archival and serialization API as part of the Foundation framework. We’re interested in improving the experience and safety of performing archival and serialization, and are happy to receive community feedback on this work.

Thanks to all of the people who've worked on this. It's a great proposal.

Specifically:

  • It aims to provide a solution for the archival of Swift struct and enum types

I see a lot of discussion here of structs and classes, and an example of an enum without associated values, but I don't see any discussion of enums with associated values. Can you sketch how you see people encoding such types?

For example, I assume that `Optional` is going to get some special treatment, but if it doesn't, how would you write its `encode(to:)` method?

What about a more complex enum, like the standard library's `UnicodeDecodingResult`:

  enum UnicodeDecodingResult {
    case emptyInput
    case error
    case scalarValue(UnicodeScalar)
  }

Or, say, an `Error`-conforming type from one of my projects:

  public enum SQLError: Error {
      case connectionFailed(underlying: Error)
      case executionFailed(underlying: Error, statement: SQLStatement)
      case noRecordsFound(statement: SQLStatement)
      case extraRecordsFound(statement: SQLStatement)
      case columnInvalid(underlying: Error, key: ColumnSpecifier, statement: SQLStatement)
      case valueInvalid(underlying: Error, key: AnySQLColumnKey, statement: SQLStatement)
  }

(You can assume that all the types in the associated values are `Codable`.)

I don't necessarily assume that the compiler should write conformances to these sorts of complicated enums for me (though that would be nice!); I'm just wondering what the designers of this feature envision people doing in cases like these.

  • protocol Codable: Adopted by types to opt into archival. Conformance may be automatically derived in cases where all properties are also Codable.

Have you given any consideration to supporting types which only need to decode? That seems likely to be common when interacting with web services.

  • protocol CodingKey: Adopted by types used as keys for keyed containers, replacing String keys with semantic types. Conformance may be automatically derived in most cases.
  • protocol Encoder: Adopted by types which can take Codable values and encode them into a native format.
    • class KeyedEncodingContainer<Key : CodingKey>: Subclasses of this type provide a concrete way to store encoded values by CodingKey. Types adopting Encoder should provide subclasses of KeyedEncodingContainer to vend.
    • protocol SingleValueEncodingContainer: Adopted by types which provide a concrete way to store a single encoded value. Types adopting Encoder should provide types conforming to SingleValueEncodingContainer to vend (but in many cases will be able to conform to it themselves).
  • protocol Decoder: Adopted by types which can take payloads in a native format and decode Codable values out of them.
    • class KeyedDecodingContainer<Key : CodingKey>: Subclasses of this type provide a concrete way to retrieve encoded values from storage by CodingKey. Types adopting Decoder should provide subclasses of KeyedDecodingContainer to vend.
    • protocol SingleValueDecodingContainer: Adopted by types which provide a concrete way to retrieve a single encoded value from storage. Types adopting Decoder should provide types conforming to SingleValueDecodingContainer to vend (but in many cases will be able to conform to it themselves).

I do want to note that, at this point in the proposal, I was sort of thinking you'd gone off the deep end modeling this. Having read the whole thing, I now understand what all of these things do, but this really is a very large subsystem. I think it's worth asking if some of these types can be eliminated or combined.

Structured types (i.e. types which encode as a collection of properties) encode and decode their properties in a keyed manner. Keys may be String-convertible or Int-convertible (or both),

What does "may" mean here? That, at runtime, the encoder will test for the preferred key type and fall back to the other one? That seems a little bit problematic.

I'm also quite worried about how `Int`-convertible keys will interact with code synthesis. The obvious way to assign integers—declaration order—would mean that reordering declarations would invisibly break archiving, potentially (if the types were compatible) without breaking anything in an error-causing way even at runtime. You could sort the names, but then adding a new property would shift the integers of the properties "below" it. You could hash the names, but then there's no obvious relationship between the integers and key cases.

At the same time, I also think that using arbitrary integers is a poor match for ordering. If you're making an ordered container, you don't want arbitrary integers wrapped up in an abstract type. You want adjacent integers forming indices of an eventual array. (Actually, you may not want indices at all—you may just want to feed elements in one at a time!)

So I would suggest the following changes:

* The coding key always converts to a string. That means we can eliminate the `CodingKey` protocol and instead use `RawRepresentable where RawValue == String`, leveraging existing infrastructure. That also means we can call the `CodingKeys` associated type `CodingKey` instead, which is the correct name for it—we're not talking about an `OptionSet` here.

* If, to save space on disk, you want to also people to use integers as the serialized representation of a key, we might introduce a parallel `IntegerCodingKey` protocol for that, but every `CodingKey` type should map to `String` first and foremost. Using a protocol here ensures that it can be statically determined at compile time whether a type can be encoded with integer keys, so the compiler can select an overload of `container(keyedBy:)`.

* Intrinsically ordered data is encoded as a single value containers of type `Array<Codable>`. (I considered having an `orderedContainer()` method and type, but as I thought about it, I couldn't think of an advantage it would have over `Array`.)

    /// Returns an encoding container appropriate for holding a single primitive value.
    ///
    /// - returns: A new empty single value container.
    /// - precondition: May not be called after a prior `self.container(keyedBy:)` call.
    /// - precondition: May not be called after a value has been encoded through a previous `self.singleValueContainer()` call.
    func singleValueContainer() -> SingleValueEncodingContainer

Speaking of which, I'm not sure about single value containers. My first instinct is to say that methods should be moved from them to the `Encoder` directly, but that would probably cause code duplication. But...isn't there already duplication between the `SingleValue*Container` and the `Keyed*Container`? Why, yes, yes there is. So let's talk about that.

    open func encode<Value : Codable>(_ value: Value?, forKey key: Key) throws
    open func encode(_ value: Bool?, forKey key: Key) throws
    open func encode(_ value: Int?, forKey key: Key) throws
    open func encode(_ value: Int8?, forKey key: Key) throws
    open func encode(_ value: Int16?, forKey key: Key) throws
    open func encode(_ value: Int32?, forKey key: Key) throws
    open func encode(_ value: Int64?, forKey key: Key) throws
    open func encode(_ value: UInt?, forKey key: Key) throws
    open func encode(_ value: UInt8?, forKey key: Key) throws
    open func encode(_ value: UInt16?, forKey key: Key) throws
    open func encode(_ value: UInt32?, forKey key: Key) throws
    open func encode(_ value: UInt64?, forKey key: Key) throws
    open func encode(_ value: Float?, forKey key: Key) throws
    open func encode(_ value: Double?, forKey key: Key) throws
    open func encode(_ value: String?, forKey key: Key) throws
    open func encode(_ value: Data?, forKey key: Key) throws

Wait, first, a digression for another issue: I'm concerned that, if you look at the `decode` calls, there are plain `decode(…)` calls which throw if a `nil` was originally encoded and `decodeIfPresent` calls which return optional. The result is, essentially, that the encoding system eats a level of optionality for its own purposes—seemingly good, straightforward-looking code like this:

  struct MyRecord: Codable {
    var id: Int?
    …
    
    func encode(to encoder: Encoder) throws {
      let container = encoder.container(keyedBy: CodingKey.self)
      try container.encode(id, forKey: .id)
      …
    }
    
    init(from decoder: Decoder) throws {
      let container = decoder.container(keyedBy: CodingKey.self)
      id = try container.decode(Int.self, forKey: .id)
      …
    }
  }

Will crash. (At least, I assume that's what will happen.)

I think we'd be better off having `encode(_:forKey:)` not take an optional; instead, we should have `Optional` conform to `Codable` and behave in some appropriate way. Exactly how to implement it might be a little tricky because of nested optionals; I suppose a `none` would have to measure how many levels of optionality there are between it and a concrete value, and then encode that information into the data. I think our `NSNull` bridging is doing something broadly similar right now.

I know that this is not the design you would use in Objective-C, but Swift uses `Optional` differently from how Objective-C uses `nil`. Swift APIs consider `nil` and absent to be different things; where they can both occur, good Swift APIs use doubled-up Optionals to be precise about the situation. I think the design needs to be a little different to accommodate that.

Now, back to the `SingleValue*Container`/`Keyed*Container` issue. The list above is, frankly, gigantic. You specify a *lot* of primitives in `Keyed*Container`; there's a lot to implement here. And then you have to implement it all *again* in `SingleValue*Container`:

    func encode(_ value: Bool) throws
    func encode(_ value: Int) throws
    func encode(_ value: Int8) throws
    func encode(_ value: Int16) throws
    func encode(_ value: Int32) throws
    func encode(_ value: Int64) throws
    func encode(_ value: UInt) throws
    func encode(_ value: UInt8) throws
    func encode(_ value: UInt16) throws
    func encode(_ value: UInt32) throws
    func encode(_ value: UInt64) throws
    func encode(_ value: Float) throws
    func encode(_ value: Double) throws
    func encode(_ value: String) throws
    func encode(_ value: Data) throws

This is madness.

Look, here's what we do. You have two types: `Keyed*Container` and `Value*Container`. `Keyed*Container` looks something like this:

  final public class KeyedEncodingContainer<EncoderType: Encoder, Key: RawRepresentable> where Key.RawValue == String {
      public let encoder: EncoderType
      
      public let codingKeyContext: [RawRepresentable where RawValue == String]
      // Hmm, we might need a CodingKey protocol after all.
      // Still, it could just be `protocol CodingKey: RawRepresentable where RawValue == String {}`
      
      subscript (key: Key) -> ValueEncodingContainer {
          return encoder.makeValueEncodingContainer(forKey: key)
      }
  }

It's so simple, it doesn't even need to be specialized. You might even be able to get away with combining the encoding and decoding variants if the subscript comes from a conditional extension. `Value*Container` *does* need to be specialized; it looks like this (modulo the `Optional` issue I mentioned above):

  public protocol ValueEncodingContainer {
      func encode<Value : Codable>(_ value: Value?, forKey key: Key) throws
      func encode(_ value: Bool?) throws
      func encode(_ value: Int?) throws
      func encode(_ value: Int8?) throws
      func encode(_ value: Int16?) throws
      func encode(_ value: Int32?) throws
      func encode(_ value: Int64?) throws
      func encode(_ value: UInt?) throws
      func encode(_ value: UInt8?) throws
      func encode(_ value: UInt16?) throws
      func encode(_ value: UInt32?) throws
      func encode(_ value: UInt64?) throws
      func encode(_ value: Float?) throws
      func encode(_ value: Double?) throws
      func encode(_ value: String?) throws
      func encode(_ value: Data?) throws
      
      func encodeWeak<Object : AnyObject & Codable>(_ object: Object?) throws
      
      var codingKeyContext: [CodingKey]
  }

And use sites would look like:

  func encode(to encoder: Encoder) throws {
    let container = encoder.container(keyedBy: CodingKey.self)
    try container[.id].encode(id)
    try container[.name].encode(name)
    try container[.birthDate].encode(birthDate)
  }

Decoding is slightly tricker. You could either make the subscript `Optional`, which would be more like `Dictionary` but would be inconsistent with `Encoder` and would give the "never force-unwrap anything" crowd conniptions, or you could add a `contains()` method to `ValueDecodingContainer` and make `decode(_:)` throw. Either one works.

Also, another issue with the many primitives: swiftc doesn't really like large overload sets very much. Could this set be reduced? I'm not sure what the logic was in choosing these particular types, but many of them share protocols in Swift—you might get away with just this:

  public protocol ValueEncodingContainer {
      func encode<Value : Codable>(_ value: Value?, forKey key: Key) throws
      func encode(_ value: Bool?, forKey key: Key) throws
      func encode<Integer: SignedInteger>(_ value: Integer?, forKey key: Key) throws
      func encode<UInteger: UnsignedInteger>(_ value: UInteger?, forKey key: Key) throws
      func encode<Floating: FloatingPoint>(_ value: Floating?, forKey key: Key) throws
      func encode(_ value: String?, forKey key: Key) throws
      func encode(_ value: Data?, forKey key: Key) throws
      
      func encodeWeak<Object : AnyObject & Codable>(_ object: Object?, forKey key: Key) throws
      
      var codingKeyContext: [CodingKey]
  }

To accommodate my previous suggestion of using arrays to represent ordered encoded data, I would add one more primitive:

      func encode(_ values: [Codable]) throws

(Also, is there any sense in adding `Date` to this set, since it needs special treatment in many of our formats?)

Encoding Container Types

For some types, the container into which they encode has meaning. Especially when coding for a specific output format (e.g. when communicating with a JSON API), a type may wish to explicitly encode as an array or a dictionary:

// Continuing from before
public protocol Encoder {
    func container<Key : CodingKey>(keyedBy keyType: Key.Type, type containerType: EncodingContainerType) -> KeyedEncodingContainer<Key>
}

/// An `EncodingContainerType` specifies the type of container an `Encoder` should use to store values.
public enum EncodingContainerType {
    /// The `Encoder`'s preferred container type; equivalent to either `.array` or `.dictionary` as appropriate for the encoder.
    case `default`
    
    /// Explicitly requests the use of an array to store encoded values.
    case array

    /// Explicitly requests the use of a dictionary to store encoded values.
    case dictionary
}

I see what you're getting at here, but I don't think this is fit for purpose, because arrays are not simply dictionaries with integer keys—their elements are adjacent and ordered. See my discussion earlier about treating inherently ordered containers as simply single-value `Array`s.

Nesting

In practice, some types may also need to control how data is nested within their container, or potentially nest other containers within their container. Keyed containers allow this by returning nested containers of differing key types:

[snip]

This can be common when coding against specific external data representations:

// User type for interfacing with a specific JSON API. JSON API expects encoding as {"id": ..., "properties": {"name": ..., "timestamp": ...}}. Swift type differs from encoded type, and encoding needs to match a spec:

This comes very close to—but doesn't quite—address something else I'm concerned about. What's the preferred way to handle differences in serialization to different formats?

Here's what I mean: Suppose I have a BlogPost model, and I can both fetch and post BlogPosts to a cross-platform web service, and store them locally. But when I fetch and post remotely, I ned to conform to the web service's formats; when I store an instance locally, I have a freer hand in designing my storage, and perhaps need to store some extra metadata. How do you imagine handling that sort of situation? Is the answer simply that I should use two different types?

To remedy both of these points, we adopt a new convention for inheritance-based coding — encoding super as a sub-object of self:

[snip]

        try super.encode(to: container.superEncoder())

This seems like a good idea to me. However, it brings up another point: What happens if you specify a superclass of the originally encoded class? In other words:

  let joe = Employee(…)
  let payload = try SomeEncoder().encode(joe)
  …
  let someone = try SomeDecoder().decode(Person.self, from: payload)
  print(type(of: someone)) // Person, Employee, or does `decode(_:from:)` fail?

The encoding container types offer overloads for working with and processing the API's primitive types (String, Int, Double, etc.). However, for ease of implementation (both in this API and others), it can be helpful for these types to conform to Codable themselves. Thus, along with these overloads, we will offer Codable conformance on these types:

[snip]

Since Swift's function overload rules prefer more specific functions over generic functions, the specific overloads are chosen where possible (e.g. encode("Hello, world!", forKey: .greeting) will choose encode(_: String, forKey: Key) over encode<T : Codable>(_: T, forKey: Key)). This maintains performance over dispatching through the Codable existential, while allowing for the flexibility of fewer overloads where applicable.

How important is this performance? If the answer is "eh, not really that much", I could imagine a setup where every "primitive" type eventually represents itself as `String` or `Data`, and each `Encoder`/`Decoder` can use dynamic type checks in `encode(_:)`/`decode(_:)` to define whatever "primitives" it wants for its own format.

* * *

One more thing. In Alternatives Considered, you present two designs—#2 and #3—where you generate a separate instance which represents the type in a fairly standardized way for the encoder to examine.

This design struck me as remarkably similar to the reflection system and its `Mirror` type, which is also a separate type describing an original instance. My question was: Did you look at the reflection system when you were building this design? Do you think there might be anything that can be usefully shared between them?

Thank you for your attention. I hope this was helpful!

···

On Mar 15, 2017, at 3:40 PM, Itai Ferber via swift-evolution <swift-evolution@swift.org> wrote:

--
Brent Royal-Gordon
Architechies


(Zachary Waldowski) #5

Another issue of scale - I had to switch to a native mail client as replying inline severely broke my webmail client. :wink:

Again, lots of love here. Responses inline.

Proposed solution
We will be introducing the following new types:

protocol Codable: Adopted by types to opt into archival. Conformance may be automatically derived in cases where all properties are also Codable.

FWIW I think this is acceptable compromise. If the happy path is derived conformances, only-decodable or only-encodable types feel like a lazy way out on the part of a user of the API, and builds a barrier to proper testing.

[snip]

Structured types (i.e. types which encode as a collection of properties) encode and decode their properties in a keyed manner. Keys may be String-convertible or Int-convertible (or both), and user types which have properties should declare semantic key enums which map keys to their properties. Keys must conform to the CodingKey protocol:
public protocol CodingKey { <##snip##> }

A few things here:

The protocol leaves open the possibility of having both a String or Int representation, or neither. What should a coder do in either case? Are the representations intended to be mutually exclusive, or not? The protocol design doesn’t seem particularly matching with the flavor of Swift; I’d expect something along the lines of a CodingKey enum and the protocol CodingKeyRepresentable. It’s also possible that the concerns of the two are orthogonal enough that they deserve separate container(keyedBy:) requirements.

Speaking of the mutually exclusive representations - what above serializations that doesn’t code as one of those two things? YAML can have anything be a “key”, and despite that being not particularly sane, it is a use case.

For most types, String-convertible keys are a reasonable default; for performance, however, Int-convertible keys are preferred, and Encoders may choose to make use of Ints over Strings. Framework types should provide keys which have both for flexibility and performance across different types of Encoders. It is generally an error to provide a key which has neither a stringValue nor an intValue.

Could you speak a little more to using Int-convertible keys for performance? I get the feeling int-based keys parallel the legacy of NSCoder’s older design, and I don’t really see anyone these days supporting non-keyed archivers. They strike me as fragile. What other use cases are envisioned for ordered archiving than that?

[snip]

Keyed Encoding Containers

Keyed encoding containers are the primary interface that most Codable types interact with for encoding and decoding. Through these, Codable types have strongly-keyed access to encoded data by using keys that are semantically correct for the operations they want to express.

Since semantically incompatible keys will rarely (if ever) share the same key type, it is impossible to mix up key types within the same container (as is possible with Stringkeys), and since the type is known statically, keys get autocompletion by the compiler.

open class KeyedEncodingContainer<Key : CodingKey> {

Like others, I’m a little bummed about this part of the design. Your reasoning up-thread is sound, but I chafe a bit on having to reabstract and a little more on having to be a reference type. Particularly knowing that it’s got a bit more overhead involved… I /like/ that NSKeyedArchiver can simply push some state and pass itself as the next encoding container down the stack.

    open func encode<Value : Codable>(_ value: Value?, forKey key: Key) throws

Does this win anything over taking a Codable?

    open func encode(_ value: Bool?, forKey key: Key) throws
    open func encode(_ value: Int?, forKey key: Key) throws
    open func encode(_ value: Int8?, forKey key: Key) throws
    open func encode(_ value: Int16?, forKey key: Key) throws
    open func encode(_ value: Int32?, forKey key: Key) throws
    open func encode(_ value: Int64?, forKey key: Key) throws
    open func encode(_ value: UInt?, forKey key: Key) throws
    open func encode(_ value: UInt8?, forKey key: Key) throws
    open func encode(_ value: UInt16?, forKey key: Key) throws
    open func encode(_ value: UInt32?, forKey key: Key) throws
    open func encode(_ value: UInt64?, forKey key: Key) throws
    open func encode(_ value: Float?, forKey key: Key) throws
    open func encode(_ value: Double?, forKey key: Key) throws
    open func encode(_ value: String?, forKey key: Key) throws
    open func encode(_ value: Data?, forKey key: Key) throws

What is the motivation behind abandoning the idea of “primitives” from the Alternatives Considered? Performance? Being unable to close the protocol?

What ways is encoding a value envisioned to fail? I understand wanting to allow maximum flexibility, and being symmetric to `decode` throwing, but there are plenty of “conversion” patterns the are asymmetric in the ways they can fail (Date formatters, RawRepresentable, LosslessStringConvertible, etc.).

    /// For `Encoder`s that implement this functionality, this will only encode the given object and associate it with the given key if it encoded unconditionally elsewhere in the archive (either previously or in the future).
    open func encodeWeak<Object : AnyObject & Codable>(_ object: Object?, forKey key: Key) throws

Is this correct that if I send a Cocoa-style object graph (with weak backrefs), an encoder could infinitely recurse? Or is a coder supposed to detect that?

    open var codingKeyContext: [CodingKey]
}
[snippity snip]

Alright, those are just my first thoughts. I want to spend a little time marinating in the code from PR #8124 before I comment further. Cheers! I owe you, Michael, and Tony a few drinks for sure.

Zach Waldowski
zach@waldowski.me

···

On Mar 15, 2017, at 6:40 PM, Itai Ferber via swift-evolution <swift-evolution@swift.org> wrote:


(David Hart) #6

First of all, great proposal :smiley:

Brent, earlier in the thread makes a lot of good points. But I’d still like to discuss two subjects:

1) What makes the proposal really stand on its feet compared to third-party libraries is the compiler generation magic. I feel divided about it. On one hand, this is the only solution today to have this level of type and key safety. But on another hand, I have the impression that future versions of Swift (with more reflection, property behaviours, lenses, etc…) would dramatically affect how this subject is treated and implemented. Are you worried that we are asking the compiler to do work which might be un-necessary in the future? That this topic would be better expressed with more powerful language features? Any plans to migrate for this API to smoothly migrate to those features in the future?

2) Libraries like Marshal (https://github.com/utahiosmac/Marshal) and Unbox (https://github.com/JohnSundell/Unbox) don’t require the decoding functions to provide the type: those functions are generic on the return turn and it’s automatically inferred:

func decode<T>(key: Key) -> T

self.stringProperty = decode(key: .stringProperty) // correct specialisation of the generic function chosen by the compiler

Is there a reason the proposal did not choose this solution? Its quite sweet.

···

Swift Archival & Serialization
Proposal: SE-NNNN <https://github.com/apple/swift-evolution/pull/639>
Author(s): Itai Ferber <https://github.com/itaiferber>, Michael LeHew <https://github.com/mlehew>, Tony Parker <https://github.com/parkera>
Review Manager: TBD
Status: Awaiting review
Associated PRs:
#8124 <https://github.com/apple/swift/pull/8124>
#8125 <https://github.com/apple/swift/pull/8125>


(Slava Pestov) #7

Hi Itai,

I’m wondering what the motivation is for keeping this as part of Foundation and not the standard library. It seems like you’re landing an implementation of this in the Foundation overlay on master, and another copy of all the code will have to go into swift-corelibs-foundation. This seems suboptimal. Or are there future plans to unify the Foundation overlay with corelibs-foundation somehow?

Also the implementation uses some Foundation-isms (NSMutableArray, NSNumber) and it would be nice to stick with idiomatic Swift as much as possible instead.

Finally you should take a look at the integer protocol work (https://github.com/apple/swift-evolution/blob/master/proposals/0104-improved-integers.md) to replace the repetitive code surrounding primitive types, however I don’t know if this has landed in master yet.

Slava

···

On Mar 15, 2017, at 3:40 PM, Itai Ferber via swift-evolution <swift-evolution@swift.org> wrote:

Hi everyone,

The following introduces a new Swift-focused archival and serialization API as part of the Foundation framework. We’re interested in improving the experience and safety of performing archival and serialization, and are happy to receive community feedback on this work.
Because of the length of this proposal, the Appendix and Alternatives Considered sections have been omitted here, but are available in the full proposal <https://github.com/apple/swift-evolution/pull/639> on the swift-evolution repo. The full proposal also includes an Unabridged API for further consideration.

Without further ado, inlined below.

— Itai

Swift Archival & Serialization
Proposal: SE-NNNN <https://github.com/apple/swift-evolution/pull/639>
Author(s): Itai Ferber <https://github.com/itaiferber>, Michael LeHew <https://github.com/mlehew>, Tony Parker <https://github.com/parkera>
Review Manager: TBD
Status: Awaiting review
Associated PRs:
#8124 <https://github.com/apple/swift/pull/8124>
#8125 <https://github.com/apple/swift/pull/8125>
Introduction
Foundation's current archival and serialization APIs (NSCoding, NSJSONSerialization, NSPropertyListSerialization, etc.), while fitting for the dynamism of Objective-C, do not always map optimally into Swift. This document lays out the design of an updated API that improves the developer experience of performing archival and serialization in Swift.

Specifically:

It aims to provide a solution for the archival of Swift struct and enum types
It aims to provide a more type-safe solution for serializing to external formats, such as JSON and plist
Motivation
The primary motivation for this proposal is the inclusion of native Swift enum and struct types in archival and serialization. Currently, developers targeting Swift cannot participate in NSCoding without being willing to abandon enum and structtypes — NSCoding is an @objc protocol, conformance to which excludes non-class types. This is can be limiting in Swift because small enums and structs can be an idiomatic approach to model representation; developers who wish to perform archival have to either forgo the Swift niceties that constructs like enumsprovide, or provide an additional compatibility layer between their "real" types and their archivable types.

Secondarily, we would like to refine Foundation's existing serialization APIs (NSJSONSerialization and NSPropertyListSerialization) to better match Swift's strong type safety. From experience, we find that the conversion from the unstructured, untyped data of these formats into strongly-typed data structures is a good fit for archival mechanisms, rather than taking the less safe approach that 3rd-party JSON conversion approaches have taken (described further in an appendix below).

We would like to offer a solution to these problems without sacrificing ease of use or type safety.

Agenda
This proposal is the first stage of three that introduce different facets of a whole Swift archival and serialization API:

This proposal describes the basis for this API, focusing on the protocols that users adopt and interface with
The next stage will propose specific API for new encoders
The final stage will discuss how this new API will interop with NSCoding as it is today
SE-NNNN provides stages 2 and 3.

Proposed solution
We will be introducing the following new types:

protocol Codable: Adopted by types to opt into archival. Conformance may be automatically derived in cases where all properties are also Codable.
protocol CodingKey: Adopted by types used as keys for keyed containers, replacing String keys with semantic types. Conformance may be automatically derived in most cases.
protocol Encoder: Adopted by types which can take Codable values and encode them into a native format.
class KeyedEncodingContainer<Key : CodingKey>: Subclasses of this type provide a concrete way to store encoded values by CodingKey. Types adopting Encoder should provide subclasses of KeyedEncodingContainer to vend.
protocol SingleValueEncodingContainer: Adopted by types which provide a concrete way to store a single encoded value. Types adopting Encoder should provide types conforming to SingleValueEncodingContainer to vend (but in many cases will be able to conform to it themselves).
protocol Decoder: Adopted by types which can take payloads in a native format and decode Codable values out of them.
class KeyedDecodingContainer<Key : CodingKey>: Subclasses of this type provide a concrete way to retrieve encoded values from storage by CodingKey. Types adopting Decoder should provide subclasses of KeyedDecodingContainer to vend.
protocol SingleValueDecodingContainer: Adopted by types which provide a concrete way to retrieve a single encoded value from storage. Types adopting Decoder should provide types conforming to SingleValueDecodingContainer to vend (but in many cases will be able to conform to it themselves).
For end users of this API, adoption will primarily involve the Codable and CodingKey protocols. In order to participate in this new archival system, developers must add Codable conformance to their types:

// If all properties are Codable, implementation is automatically derived:
public struct Location : Codable {
    public let latitude: Double
    public let longitude: Double
}

public enum Animal : Int, Codable {
    case chicken = 1
    case dog
    case turkey
    case cow
}

public struct Farm : Codable {
    public let name: String
    public let location: Location
    public let animals: [Animal]
}
With developer participation, we will offer encoders and decoders (described in SE-NNNN, not here) that take advantage of this conformance to offer type-safe serialization of user models:

let farm = Farm(name: "Old MacDonald's Farm",
                location: Location(latitude: 51.621648, longitude: 0.269273),
                animals: [.chicken, .dog, .cow, .turkey, .dog, .chicken, .cow, .turkey, .dog])
let payload: Data = try JSONEncoder().encode(farm)

do {
    let farm = try JSONDecoder().decode(Farm.self, from: payload)

    // Extracted as user types:
    let coordinates = "\(farm.location.latitude, farm.location.longitude)"
} catch {
    // Encountered error during deserialization
}
This gives developers access to their data in a type-safe manner and a recognizable interface.

Detailed design
To support user types, we expose the Codable protocol:

/// Conformance to `Codable` indicates that a type can marshal itself into and out of an external representation.
public protocol Codable {
    /// Initializes `self` by decoding from `decoder`.
    ///
    /// - parameter decoder: The decoder to read data from.
    /// - throws: An error if reading from the decoder fails, or if read data is corrupted or otherwise invalid.
    init(from decoder: Decoder) throws

    /// Encodes `self` into the given encoder.
    ///
    /// If `self` fails to encode anything, `encoder` will encode an empty `.default` container in its place.
    ///
    /// - parameter encoder: The encoder to write data to.
    /// - throws: An error if any values are invalid for `encoder`'s format.
    func encode(to encoder: Encoder) throws
}
By adopting Codable, user types opt in to this archival system.

Structured types (i.e. types which encode as a collection of properties) encode and decode their properties in a keyed manner. Keys may be String-convertible or Int-convertible (or both), and user types which have properties should declare semantic key enums which map keys to their properties. Keys must conform to the CodingKey protocol:

/// Conformance to `CodingKey` indicates that a type can be used as a key for encoding and decoding.
public protocol CodingKey {
    /// The string to use in a named collection (e.g. a string-keyed dictionary).
    var stringValue: String? { get }

    /// Initializes `self` from a string.
    ///
    /// - parameter stringValue: The string value of the desired key.
    /// - returns: An instance of `Self` from the given string, or `nil` if the given string does not correspond to any instance of `Self`.
    init?(stringValue: String)

    /// The int to use in an indexed collection (e.g. an int-keyed dictionary).
    var intValue: Int? { get }

    /// Initializes `self` from an integer.
    ///
    /// - parameter intValue: The integer value of the desired key.
    /// - returns: An instance of `Self` from the given integer, or `nil` if the given integer does not correspond to any instance of `Self`.
    init?(intValue: Int)
}
For most types, String-convertible keys are a reasonable default; for performance, however, Int-convertible keys are preferred, and Encoders may choose to make use of Ints over Strings. Framework types should provide keys which have both for flexibility and performance across different types of Encoders. It is generally an error to provide a key which has neither a stringValue nor an intValue.

By default, CodingKey conformance can be derived for enums which have either String or Int backing:

enum Keys1 : CodingKey {
    case a // (stringValue: "a", intValue: nil)
    case b // (stringValue: "b", intValue: nil)
}

enum Keys2 : String, CodingKey {
    case c = "foo" // (stringValue: "foo", intValue: nil)
    case d // (stringValue: "d", intValue: nil)
}

enum Keys3 : Int, CodingKey {
    case e = 4 // (stringValue: "e", intValue: 4)
    case f // (stringValue: "f", intValue: 5)
    case g = 9 // (stringValue: "g", intValue: 9)
}
Coding keys which are not enums, have associated values, or have other raw representations must implement these methods manually.

In addition to automatic CodingKey conformance derivation for enums, Codableconformance can be automatically derived for certain types as well:

Types whose properties are all either Codable or primitive get an automatically derived String-backed CodingKeys enum mapping properties to case names
Types falling into (1) and types which provide a CodingKeys enum (directly or via a typealias) whose case names map to properties which are all Codableget automatic derivation of init(from:) and encode(to:) using those properties and keys. Types may choose to provide a custom init(from:) or encode(to:) (or both); whichever they do not provide will be automatically derived
Types which fall into neither (1) nor (2) will have to provide a custom key type and provide their own init(from:) and encode(to:)
Many types will either allow for automatic derivation of all codability (1), or provide a custom key subset and take advantage of automatic method derivation (2).

Encoding and Decoding

Types which are encodable encode their data into a container provided by their Encoder:

/// An `Encoder` is a type which can encode values into a native format for external representation.
public protocol Encoder {
    /// Populates `self` with an encoding container (of `.default` type) and returns it, keyed by the given key type.
    ///
    /// - parameter type: The key type to use for the container.
    /// - returns: A new keyed encoding container.
    /// - precondition: May not be called after a previous `self.container(keyedBy:)` call of a different `EncodingContainerType`.
    /// - precondition: May not be called after a value has been encoded through a prior `self.singleValueContainer()` call.
    func container<Key : CodingKey>(keyedBy type: Key.Type) -> KeyedEncodingContainer<Key>

    /// Returns an encoding container appropriate for holding a single primitive value.
    ///
    /// - returns: A new empty single value container.
    /// - precondition: May not be called after a prior `self.container(keyedBy:)` call.
    /// - precondition: May not be called after a value has been encoded through a previous `self.singleValueContainer()` call.
    func singleValueContainer() -> SingleValueEncodingContainer

    /// The path of coding keys taken to get to this point in encoding.
    var codingKeyContext: [CodingKey] { get }
}

// Continuing examples from before; below is automatically generated by the compiler if no customization is needed.
public struct Location : Codable {
    private enum CodingKeys : CodingKey {
        case latitutude
        case longitude
    }

    public func encode(to encoder: Encoder) throws {
        // Generic keyed encoder gives type-safe key access: cannot encode with keys of the wrong type.
        let container = encoder.container(keyedBy: CodingKeys.self)

        // The encoder is generic on the key -- free key autocompletion here.
        try container.encode(latitude, forKey: .latitude)
        try container.encode(longitude, forKey: .longitude)
    }
}

public struct Farm : Codable {
    private enum CodingKeys : CodingKey {
        case name
        case location
        case animals
    }

    public func encode(to encoder: Encoder) throws {
        let container = encoder.container(keyedBy: CodingKeys.self)
        try container.encode(name, forKey: .name)
        try container.encode(location, forKey: .location)
        try container.encode(animals, forKey: .animals)
    }
}
Similarly, decodable types initialize from data read from their Decoder's container:

/// A `Decoder` is a type which can decode values from a native format into in-memory representations.
public protocol Decoder {
    /// Returns the data stored in `self` as represented in a container keyed by the given key type.
    ///
    /// - parameter type: The key type to use for the container.
    /// - returns: A keyed decoding container view into `self`.
    /// - throws: `CocoaError.coderTypeMismatch` if the encountered stored value is not a keyed container.
    func container<Key : CodingKey>(keyedBy type: Key.Type) throws -> KeyedDecodingContainer<Key>

    /// Returns the data stored in `self` as represented in a container appropriate for holding a single primitive value.
    ///
    /// - returns: A single value container view into `self`.
    /// - throws: `CocoaError.coderTypeMismatch` if the encountered stored value is not a single value container.
    func singleValueContainer() throws -> SingleValueDecodingContainer

    /// The path of coding keys taken to get to this point in decoding.
    var codingKeyContext: [CodingKey] { get }
}

// Continuing examples from before; below is automatically generated by the compiler if no customization is needed.
public struct Location : Codable {
    public init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        latitude = try container.decode(Double.self, forKey: .latitude)
        longitude = try container.decode(Double.self, forKey: .longitude)
    }
}

public struct Farm : Codable {
    public init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        name = try container.decode(String.self, forKey: .name)
        location = try container.decode(Location.self, forKey: .location)
        animals = try container.decode([Animal].self, forKey: .animals)
    }
}
Keyed Encoding Containers

Keyed encoding containers are the primary interface that most Codable types interact with for encoding and decoding. Through these, Codable types have strongly-keyed access to encoded data by using keys that are semantically correct for the operations they want to express.

Since semantically incompatible keys will rarely (if ever) share the same key type, it is impossible to mix up key types within the same container (as is possible with Stringkeys), and since the type is known statically, keys get autocompletion by the compiler.

/// `KeyedEncodingContainer` is a generic abstract base class that provides a view into an `Encoders` storage and is used to hold the encoded properties of a `Codable` type.
///
/// Encoders should provide subclasses of `KeyedEncodingContainer` for their format.
open class KeyedEncodingContainer<Key : CodingKey> {
    /// Encodes the given value for the given key.
    ///
    /// - parameter value: The value to encode.
    /// - parameter key: The key to associate the value with.
    /// - throws: `CocoaError.coderInvalidValue` if the given value is invalid in the current context for this format.
    /// - precondition: The key must have a `stringValue` or `intValue` appropriate for the encoding container type.
    open func encode<Value : Codable>(_ value: Value?, forKey key: Key) throws

    /// Encodes the given value for the given key.
    ///
    /// - parameter value: The value to encode.
    /// - parameter key: The key to associate the value with.
    /// - throws: `CocoaError.coderInvalidValue` if the given value is invalid in the current context for this format.
    /// - precondition: The key must have a `stringValue` or `intValue` appropriate for the encoding container type.
    open func encode(_ value: Bool?, forKey key: Key) throws
    open func encode(_ value: Int?, forKey key: Key) throws
    open func encode(_ value: Int8?, forKey key: Key) throws
    open func encode(_ value: Int16?, forKey key: Key) throws
    open func encode(_ value: Int32?, forKey key: Key) throws
    open func encode(_ value: Int64?, forKey key: Key) throws
    open func encode(_ value: UInt?, forKey key: Key) throws
    open func encode(_ value: UInt8?, forKey key: Key) throws
    open func encode(_ value: UInt16?, forKey key: Key) throws
    open func encode(_ value: UInt32?, forKey key: Key) throws
    open func encode(_ value: UInt64?, forKey key: Key) throws
    open func encode(_ value: Float?, forKey key: Key) throws
    open func encode(_ value: Double?, forKey key: Key) throws
    open func encode(_ value: String?, forKey key: Key) throws
    open func encode(_ value: Data?, forKey key: Key) throws

    /// Encodes the given object weakly for the given key.
    ///
    /// For `Encoder`s that implement this functionality, this will only encode the given object and associate it with the given key if it encoded unconditionally elsewhere in the archive (either previously or in the future).
    ///
    /// For formats which don't support this feature, the default implementation encodes the given object unconditionally.
    ///
    /// - parameter object: The object to encode.
    /// - parameter key: The key to associate the object with.
    /// - throws: `CocoaError.coderInvalidValue` if the given value is invalid in the current context for this format.
    /// - precondition: The key must have a `stringValue` or `intValue` appropriate for the encoding container type.
    open func encodeWeak<Object : AnyObject & Codable>(_ object: Object?, forKey key: Key) throws

    /// The path of coding keys taken to get to this point in encoding.
    open var codingKeyContext: [CodingKey]
}

/// `KeyedDecodingContainer` is a generic abstract base class that provides a view into an `Decoders` storage and is used to hold the encoded properties of a `Codable` type.
///
/// Decoders should provide subclasses of `KeyedDecodingContainer` for their format.
open class KeyedDecodingContainer<Key : CodingKey> {
    /// All the keys the `Decoder` has for this container.
    ///
    /// Different keyed containers from the same `Decoder` may return different keys here; it is possible to encode with multiple key types which are not convertible to one another. This should report all keys present which are convertible to the requested type.
    open var allKeys: [Key]

    /// Returns whether the `Decoder` contains a value associated with the given key.
    ///
    /// The value associated with the given key may be a null value as appropriate for the data format.
    ///
    /// - parameter key: The key to search for.
    /// - returns: Whether the `Decoder` has an entry for the given key.
    open func contains(_ key: Key) -> Bool

    /// Decodes a value of the given type for the given key.
    ///
    /// A default implementation is given for these types which calls into the abstract `decodeIfPresent` implementations below.
    ///
    /// - parameter type: The type of value to decode.
    /// - parameter key: The key that the decoded value is associated with.
    /// - returns: A value of the requested type, if present for the given key and convertible to the requested type.
    /// - throws: `CocoaError.coderTypeMismatch` if the encountered encoded value is not convertible to the requested type.
    /// - throws: `CocoaError.coderValueNotFound` if `self` does not have an entry for the given key or if the value is null.
    open func decode(_ type: Bool.Type, forKey key: Key) throws -> Bool
    open func decode(_ type: Int.Type, forKey key: Key) throws -> Int
    open func decode(_ type: Int8.Type, forKey key: Key) throws -> Int8
    open func decode(_ type: Int16.Type, forKey key: Key) throws -> Int16
    open func decode(_ type: Int32.Type, forKey key: Key) throws -> Int32
    open func decode(_ type: Int64.Type, forKey key: Key) throws -> Int64
    open func decode(_ type: UInt.Type, forKey key: Key) throws -> UInt
    open func decode(_ type: UInt8.Type, forKey key: Key) throws -> UInt8
    open func decode(_ type: UInt16.Type, forKey key: Key) throws -> UInt16
    open func decode(_ type: UInt32.Type, forKey key: Key) throws -> UInt32
    open func decode(_ type: UInt64.Type, forKey key: Key) throws -> UInt64
    open func decode(_ type: Float.Type, forKey key: Key) throws -> Float
    open func decode(_ type: Double.Type, forKey key: Key) throws -> Double
    open func decode(_ type: String.Type, forKey key: Key) throws -> String
    open func decode(_ type: Data.Type, forKey key: Key) throws -> Data
    open func decode<Value : Codable>(_ type: Value.Type, forKey key: Key) throws -> Value

    /// Decodes a value of the given type for the given key, if present.
    ///
    /// This method returns `nil` if the container does not have a value associated with `key`, or if the value is null. The difference between these states can be disambiguated with a `contains(_:)` call.
    ///
    /// - parameter type: The type of value to decode.
    /// - parameter key: The key that the decoded value is associated with.
    /// - returns: A decoded value of the requested type, or `nil` if the `Decoder` does not have an entry associated with the given key, or if the value is a null value.
    /// - throws: `CocoaError.coderTypeMismatch` if the encountered encoded value is not convertible to the requested type.
    open func decodeIfPresent(_ type: Bool.Type, forKey key: Key) throws -> Bool?
    open func decodeIfPresent(_ type: Int.Type, forKey key: Key) throws -> Int?
    open func decodeIfPresent(_ type: Int8.Type, forKey key: Key) throws -> Int8?
    open func decodeIfPresent(_ type: Int16.Type, forKey key: Key) throws -> Int16?
    open func decodeIfPresent(_ type: Int32.Type, forKey key: Key) throws -> Int32?
    open func decodeIfPresent(_ type: Int64.Type, forKey key: Key) throws -> Int64?
    open func decodeIfPresent(_ type: UInt.Type, forKey key: Key) throws -> UInt?
    open func decodeIfPresent(_ type: UInt8.Type, forKey key: Key) throws -> UInt8?
    open func decodeIfPresent(_ type: UInt16.Type, forKey key: Key) throws -> UInt16?
    open func decodeIfPresent(_ type: UInt32.Type, forKey key: Key) throws -> UInt32?
    open func decodeIfPresent(_ type: UInt64.Type, forKey key: Key) throws -> UInt64?
    open func decodeIfPresent(_ type: Float.Type, forKey key: Key) throws -> Float?
    open func decodeIfPresent(_ type: Double.Type, forKey key: Key) throws -> Double?
    open func decodeIfPresent(_ type: String.Type, forKey key: Key) throws -> String?
    open func decodeIfPresent(_ type: Data.Type, forKey key: Key) throws -> Data?
    open func decodeIfPresent<Value : Codable>(_ type: Value.Type, forKey key: Key) throws -> Value?

    /// The path of coding keys taken to get to this point in decoding.
    open var codingKeyContext: [CodingKey]
}
These encode(_:forKey:) and decode(_:forKey:) overloads give strong, static type guarantees about what is encodable (preventing accidental attempts to encode an invalid type), and provide a list of primitive types which are common to all encoders and decoders that users can rely on.

Coming in Swift 4 is the ability to express that "a collection of things which are Codable is Codable" (conditional conformance), allowing collections which we extend (Array, Dictionary, etc.) to fall into these overloads as well.

Encoding Container Types

For some types, the container into which they encode has meaning. Especially when coding for a specific output format (e.g. when communicating with a JSON API), a type may wish to explicitly encode as an array or a dictionary:

// Continuing from before
public protocol Encoder {
    /// Populates `self` with an encoding container of the given type and returns it, keyed by the given key type.
    ///
    /// A default implementation of `Encoder.container(keyedBy:)` calls this method with a container type of `.default`.
    ///
    /// - parameter keyType: The key type to use for the container.
    /// - parameter containerType: The container type to create.
    /// - returns: A new keyed encoding container.
    /// - precondition: May not be called after a previous `self.container(keyedBy:)` call of a different `EncodingContainerType`.
    /// - precondition: May not be called after a value has been encoded through a prior `self.singleValueContainer()` call.
    func container<Key : CodingKey>(keyedBy keyType: Key.Type, type containerType: EncodingContainerType) -> KeyedEncodingContainer<Key>
}

/// An `EncodingContainerType` specifies the type of container an `Encoder` should use to store values.
public enum EncodingContainerType {
    /// The `Encoder`'s preferred container type; equivalent to either `.array` or `.dictionary` as appropriate for the encoder.
    case `default`

    /// Explicitly requests the use of an array to store encoded values.
    case array

    /// Explicitly requests the use of a dictionary to store encoded values.
    case dictionary
}
Single Value Containers

For other types, an array or dictionary container may not even make sense (e.g. values which are RawRepresentable as a single primitive value). Those types may encode and decode directly as a single value, instead of requesting a keyed container:

/// A `SingleValueEncodingContainer` is a container which can support the storage and direct encoding of a single non-keyed value.
public protocol SingleValueEncodingContainer {
    /// Encodes a single value of the given type.
    ///
    /// - parameter value: The value to encode.
    /// - throws: `CocoaError.coderInvalidValue` if the given value is invalid in the current context for this format.
    /// - precondition: May not be called after a previous `self.encode(_:)` call.
    func encode(_ value: Bool) throws
    func encode(_ value: Int) throws
    func encode(_ value: Int8) throws
    func encode(_ value: Int16) throws
    func encode(_ value: Int32) throws
    func encode(_ value: Int64) throws
    func encode(_ value: UInt) throws
    func encode(_ value: UInt8) throws
    func encode(_ value: UInt16) throws
    func encode(_ value: UInt32) throws
    func encode(_ value: UInt64) throws
    func encode(_ value: Float) throws
    func encode(_ value: Double) throws
    func encode(_ value: String) throws
    func encode(_ value: Data) throws
}

/// A `SingleValueDecodingContainer` is a container which can support the storage and direct decoding of a single non-keyed value.
public protocol SingleValueDecodingContainer {
    /// Decodes a single value of the given type.
    ///
    /// - parameter type: The type to decode as.
    /// - returns: A value of the requested type.
    /// - throws: `CocoaError.coderTypeMismatch` if the encountered encoded value cannot be converted to the requested type.
    func decode(_ type: Bool.Type) throws -> Bool
    func decode(_ type: Int.Type) throws -> Int
    func decode(_ type: Int8.Type) throws -> Int8
    func decode(_ type: Int16.Type) throws -> Int16
    func decode(_ type: Int32.Type) throws -> Int32
    func decode(_ type: Int64.Type) throws -> Int64
    func decode(_ type: UInt.Type) throws -> UInt
    func decode(_ type: UInt8.Type) throws -> UInt8
    func decode(_ type: UInt16.Type) throws -> UInt16
    func decode(_ type: UInt32.Type) throws -> UInt32
    func decode(_ type: UInt64.Type) throws -> UInt64
    func decode(_ type: Float.Type) throws -> Float
    func decode(_ type: Double.Type) throws -> Double
    func decode(_ type: String.Type) throws -> String
    func decode(_ type: Data.Type) throws -> Data
}

// Continuing example from before; below is automatically generated by the compiler if no customization is needed.
public enum Animal : Int, Codable {
    public func encode(to encoder: Encoder) throws {
        // Encode as a single value; no keys.
        try encoder.singleValueContainer.encode(self.rawValue)
    }

    public init(from decoder: Decoder) throws {
        // Decodes as a single value; no keys.
        let intValue = try decoder.singleValueContainer().decode(Int.self)
        if let value = Self(rawValue: intValue) {
            self = value
        } else {
            throw CocoaError.error(.coderReadCorrupt)
        }
    }
}
In the example given above, since Animal uses a single value container, [.chicken, .dog, .cow, .turkey, .dog, .chicken, .cow, .turkey, .dog]would encode directly as [1, 2, 4, 3, 2, 1, 4, 3, 2].

Nesting

In practice, some types may also need to control how data is nested within their container, or potentially nest other containers within their container. Keyed containers allow this by returning nested containers of differing key types:

// Continuing from before
open class KeyedEncodingContainer<Key : CodingKey> {
    /// Stores an encoding container for the given key and returns it.
    ///
    /// - parameter keyType: The key type to use for the container.
    /// - parameter containerType: The container type to create.
    /// - parameter key: The key to encode the container for.
    /// - returns: A new keyed encoding container.
    open func nestedContainer<NestedKey : CodingKey>(keyedBy keyType: NestedKey.Type, type containerType: EncodingContainerType, forKey key: Key) -> KeyedEncodingContainer<NestedKey>
}

open class KeyedDecodingContainer<Key : CodingKey> {
    /// Returns the data stored for the given key as represented in a container keyed by the given key type.
    ///
    /// - parameter type: The key type to use for the container.
    /// - parameter key: The key that the nested container is associated with.
    /// - returns: A keyed decoding container view into `self`.
    /// - throws: `CocoaError.coderTypeMismatch` if the encountered stored value is not a container.
    open func nestedContainer<NestedKey : CodingKey>(keyedBy type: NestedKey.Type, forKey key: Key) throws -> KeyedDecodingContainer<NestedKey>
}
This can be common when coding against specific external data representations:

// User type for interfacing with a specific JSON API. JSON API expects encoding as {"id": ..., "properties": {"name": ..., "timestamp": ...}}. Swift type differs from encoded type, and encoding needs to match a spec:
struct Record : Codable {
    // We care only about these values from the JSON payload
    let id: Int
    let name: String
    let timestamp: Double

    // ...

    private enum Keys : CodingKey {
        case id
        case properties
    }

    private enum PropertiesKeys : CodingKey {
        case name
        case timestamp
    }

    public func encode(to encoder: Encoder) throws {
        let container = encoder.container(keyedBy: Keys.self, type: .dictionary)
        try container.encode(id, forKey: .id)

        // Set a dictionary for the "properties" key
        let nested = container.nestedContainer(keyedBy: PropertiesKeys.self, type: .dictionary, forKey: .properties)
        try nested.encode(name, forKey: .name)
        try nested.encode(timestamp, forKey: .timestamp)
    }

    public init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: Keys.self)
        id = try container.decode(Int.self, forKey: .id)

        let nested = try container.nestedContainer(keyedBy: PropertiesKeys.self, forKey: .properties)
        name = try nested.decode(String.self, forKey: .name)
        timestamp = try nested.decode(Double.self, forKey: .timestamp)
    }
}
Inheritance

Inheritance in this system is supported much like it is with NSCoding — on encoding, objects which inherit from a type that is Codable encode super using their encoder, and pass a decoder to super.init(from:) on decode. With the existing NSCoding API, this is most often done like so, by convention:

- (void)encodeWithCoder:(NSCoder *)encoder {
    [super encodeWithCoder:encoder];
    // ... encode properties
}

- (instancetype)initWithCoder:(NSCoder *)decoder {
    if ((self = [super initWithCoder:decoder])) {
        // ... decode properties
    }

    return self;
}
In practice, this approach means that the properties of self and the properties of super get encoded into the same container: if self encodes values for keys "a", "b", and "c", and super encodes "d", "e", and "f", the resulting object is archived as {"a": ..., "b": ..., "c": ..., "d": ..., "e": ..., "f": ...}. This approach has two drawbacks:

Things which self encodes may overwrite super's (or vice versa, depending on when -[super encodeWithCoder:] is called
self and super may not encode into different container types (e.g. self in a sequential fashion, and super in a keyed fashion)
The second point is not an issue for NSKeyedArchiver, since all values encode with keys (sequentially coded elements get autogenerated keys). This proposed API, however, allows for self and super to explicitly request conflicting containers (.arrayand .dictionary, which may not be mixed, depending on the data format).

To remedy both of these points, we adopt a new convention for inheritance-based coding — encoding super as a sub-object of self:

public class MyCodable : SomethingCodable {
    public func encode(to encoder: Encoder) throws {
        let container = encoder.container(keyedBy: CodingKeys.self)
        // ... encode some properties

        // superEncoder() gives `super` a nested container to encode into (for
        // a predefined key).
        try super.encode(to: container.superEncoder())
    }

    public init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        // ... decode some properties

        // Allow `super` to decode from the nested container.
        try super.init(from: container.superDecoder())
    }
}
If a shared container is desired, it is still possible to call super.encode(to: encoder) and super.init(from: decoder), but we recommend the safer containerized option.

superEncoder() and superDecoder() are provided on KeyedEncodingContainer and KeyedDecodingContainer to provide handles to containers for super to use. While users may specify a custom key to encode super with, the default behavior is to use a key with a stringValue of "super" and an intValue of 0:

// Continuing from before
open class KeyedEncodingContainer<Key : CodingKey> {
    /// Stores a new nested container for the default `super` key and returns a new `Encoder` instance for encoding `super` into that container.
    ///
    /// Equivalent to calling `superEncoder(forKey:)` with `Key(stringValue: "super", intValue: 0)`.
    ///
    /// - returns: A new `Encoder` to pass to `super.encode(to:)`.
    open func superEncoder() -> Encoder

    /// Stores a new nested container for the given key and returns a new `Encoder` instance for encoding `super` into that container.
    ///
    /// - parameter key: The key to encode `super` for.
    /// - returns: A new `Encoder` to pass to `super.encode(to:)`.
    /// - precondition: The key must have a `stringValue` or `intValue` appropriate for the encoding container type.
    open func superEncoder(forKey key: Key) -> Encoder
}

open class KeyedDecodingContainer<Key : CodingKey> {
    /// Returns a `Decoder` instance for decoding `super` from the container associated with the default `super` key.
    ///
    /// Equivalent to calling `superDecoder(forKey:)` with `Key(stringValue: "super", intValue: 0)`.
    ///
    /// - returns: A new `Decoder` to pass to `super.init(from:)`.
    /// - throws: `CocoaError.coderValueNotFound` if `self` does not have an entry for the default `super` key, or if the stored value is null.
    open func superDecoder() throws -> Decoder

    /// Returns a `Decoder` instance for decoding `super` from the container associated with the given key.
    ///
    /// - parameter key: The key to decode `super` for.
    /// - returns: A new `Decoder` to pass to `super.init(from:)`.
    /// - throws: `CocoaError.coderValueNotFound` if `self` does not have an entry for the given key, or if the stored value is null.
    open func superDecoder(forKey key: Key) throws -> Decoder
}
Primitive Codable Conformance

The encoding container types offer overloads for working with and processing the API's primitive types (String, Int, Double, etc.). However, for ease of implementation (both in this API and others), it can be helpful for these types to conform to Codable themselves. Thus, along with these overloads, we will offer Codableconformance on these types:

extension Bool : Codable {
    public init(from decoder: Decoder) throws {
        self = try decoder.singleValueContainer().decode(Bool.self)
    }

    public func encode(to encoder: Encoder) throws {
        try encoder.singleValueContainer().encode( self)
    }
}

// Repeat for others...
This conformance allows one to write functions which accept Codable types without needing specific overloads for the fifteen primitive types as well. This also simplifies conditional conformance (e.g. expressing "extension Array : Codable where Element : Codable") by removing the need for additional explicit conformances for these types.

Since Swift's function overload rules prefer more specific functions over generic functions, the specific overloads are chosen where possible (e.g. encode("Hello, world!", forKey: .greeting) will choose encode(_: String, forKey: Key) over encode<T : Codable>(_: T, forKey: Key)). This maintains performance over dispatching through the Codable existential, while allowing for the flexibility of fewer overloads where applicable.

Additional Extensions

Along with the primitive Codable conformance above, extensions on CodableRawRepresentable types whose RawValue is a primitive types will provide default implementations for encoding and decoding:

public extension RawRepresentable where RawValue == Bool, Self : Codable {
    public init(from decoder: Decoder) throws {
        let decoded = try decoder.singleValueContainer().decode(RawValue.self)
        guard let value = Self(rawValue: decoded) else {
            throw CocoaError.error(.coderReadCorrupt)
        }

        self = value
    }

    public func encode(to encoder: Encoder) throws {
        try encoder.singleValueContainer().encode(self.rawValue)
    }
}

// Repeat for others...
This allows for trivial Codable conformance of enum types (and manual RawRepresentable implementations) with primitive backing.

Source compatibility
This proposal is additive — existing code will not have to change due to this API addition. This implementation can be made available in both Swift 4 and the Swift 3 compatibility mode.

Effect on ABI stability
The addition of this API will not be an ABI-breaking change. However, this will add limitations for changes in future versions of Swift, as parts of the API will have to remain unchanged between versions of Swift (barring some additions, discussed below).

Effect on API resilience
Much like new API added to the standard library, once added, many changes to this API will be ABI- and source-breaking changes. In particular, changes which change the types or names of methods or arguments, add required methods on protocols or classes, or remove supplied default implementations will break client behavior.

The following protocols and classes may not have methods added to them without providing default implementations:

Codable
CodingKey
Encoder
SingleValueEncodingContainer
KeyedEncodingContainer
Decoder
SingleValueDecodingContainer
KeyedDecodingContainer
The following classes may not remove existing default implementations:

KeyedEncodingContainer
KeyedDecodingContainer
Various extensions to Swift primitive types (Bool, Int, Double, etc.) and to RawRepresentable types (where RawValue == Bool, == Int, == Double, etc.) may also not be removed.

In general, changes to the proposed types will be restricted as described in the library evolution document <https://github.com/apple/swift/blob/master/docs/LibraryEvolution.rst> in the Swift repository.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Slava Pestov) #8

Hi Itai,

I’m wondering what the motivation is for keeping this as part of Foundation and not the standard library. It seems like you’re landing an implementation of this in the Foundation overlay on master, and another copy of all the code will have to go into swift-corelibs-foundation. This seems suboptimal. Or are there future plans to unify the Foundation overlay with corelibs-foundation somehow?

Also the implementation uses some Foundation-isms (NSMutableArray, NSNumber) and it would be nice to stick with idiomatic Swift as much as possible instead.

Finally you should take a look at the integer protocol work (https://github.com/apple/swift-evolution/blob/master/proposals/0104-improved-integers.md) to replace the repetitive code surrounding primitive types, however I don’t know if this has landed in master yet.

Slava


(Matthew Johnson) #9

This is a fantastic proposal! I am very much looking forward to robust Swift-native encoding and decoding in Foundation. The compiler synthesized conformances is especially great! I want to thank everyone who worked on it. It is clear that a lot of work went into the proposal.

The proposal covers a lot of ground so I’m breaking my comments up by topic in the order the occur in the proposal.

Encode / Decode only types:

Brent raised the question of decode only types. Encode only types are also not uncommon when an API accepts an argument payload that gets serialized into the body of a request. The compiler synthesis feature in the proposal makes providing both encoding and decoding easy in common cases but this won’t always work as needed.

The obvious alternative is to have Decodable and Encodable protocols which Codable refines. This would allow us to omit a conformance we don’t need when it can’t be synthesized.

Your reply to Brent mentions using `fatalError` to avoid implementing the direction that isn't needed. I think it would be better if the conformance can reflect what is actually supported by the type. Requiring us to write `fatalError` as a stub for functionality we don’t need is a design problem IMO. I don’t think the extra protocols are really that big a burden. They don’t add any new functionality and are very easy to understand, especially considering the symmetry they would have with the other types you are introducing.

Coding Keys:

As others have mentioned, the design of this protocol does not require a value of a conforming type to actually be a valid key (it can return nil for both `intValue` and `stringValue`). This seems problematic to me.

In the reply to Brent again you mention throwing and `preconditionFailure` as a way to handle incompatible keys. This also seems problematic to me and feels like a design problem. If we really need to support more than one underlying key type and some encoders will reject some key types this information should be captured in the type system. An encoder would only vend a keyed container for keys it actually supports. Ideally the conformance of a type’s CodingKeys could be leveraged to produce a compiler error if an attempt was made to encode this type into an encoder that can’t support its keys. In general, the idea is to produce static errors as close to the origin of the programming mistake as possible.

I would very much prefer that we don’t defer to runtime assertions or thrown errors, etc for conditions that could be caught statically at compile time given the right design. Other comments have indicated that static guarantees are important to the design (encoders *must* guarantee support of primitives specified by the protocols, etc). Why is a static guarantee of compatible coding keys considered less important?

Keyed Containers:

Joe posted raised the topic of the alternative of using manual type erasure for the keyed containers rather than abstract classes. Did you explore this direction at all? It feels like a more natural approach for Swift and as Joe noted, it can be designed in such a way that eases migration to existentials in the future if that is the “ideal” design (which you mentioned in your response).

Decoding Containers:

returns: A value of the requested type, if present for the given key and convertible to the requested type.

Can you elaborate on the details of “convertible to the requested type” means? It think this is an important detail for the proposal.

For example, I would expect numeric values to attempt conversion using the SE-0080 failable numeric conversion initializers (decoding JSON was a primary motivating use case for that proposal). If the requested type conforms to RawRepresentable and the encoded value can be converted to RawValue (perhaps using a failable numeric initializer) I would expect the raw value initializer to be used to attempt conversion. If Swift ever gained a standard mechanism for generalized value conversion I would also expect that to be used if a conversion is available from the encoded type to the requested type.

If either of those conversions fail I would expect something like an “invalid value” error or a “conversion failure” error rather than a “type mismatch” error. The types don’t exactly mismatch, we just have a failable conversion process that did not succeed.

Context:

I’m glad Brent raised the topic of supporting multiple formats. In his example the difference was remote and local formats. I’ve also seen cases where the same API requires the same model to be encoded differently in different endpoints. This is awful, but it also happens sometimes in the wild. Supporting an application specified encoding context would be very useful for handling these situations (the `codingKeyContex` would not always be sufficient and would usually not be a good way to determine the required encoding or decoding format).

A `[UserInfoKey: Any]` context was mentioned as a possibility. This would be better than nothing, but why not use the type system to preserve information about the type of the context? I have a slightly different approach in mind. Why not just have a protocol that refines Codable with context awareness?

public protocol ContextAwareCodable: Codable {
    associatedtype Context
    init(from decoder: Decoder, with context: Context?) throws
    func encode(to encoder: Encoder, with context: Context?) throws
}
extension ContextAwareCodable {
    init(from decoder: Decoder) throws {
        try self.init(from: decoder, with: nil
    }
    func encode(to encoder: Encoder) throws {
        try self.encode(to: encoder, with: nil)
    }
}

Encoders and Decoders would be encouraged to support a top level encode / decode method which is generic and takes an application supplied context. When the context is provided it would be given to all `ContextAwareCodable` types that are aware of this context type during encoding or decoding. The coding container protocols would include an overload for `ContextAwareCodable` allowing the container to know whether the Value understands the context given to the top level encoder / decoder:

open func encode<Value : ContextAwareCodable>(_ value: Value?, forKey key: Key) throws

A common top level signature for coders and decoders would look something like this:

open func encode<Value : Codable, Context>(_ value: Value, with context: Context) throws -> Data

This approach would preserve type information about the encoding / decoding context. It falls back to the basic Codable implementation when a Value doesn’t know about the current context. The default implementation simply translates this to a nil context allowing ContextAwareCodable types to have a single implementation of the initializer and the encoding method that is used whether they are able to understand the current context or not.

- Matthew


(Kenny Leung) #10

Hi All.

Forgive me if I missed it - I haven’t read the proposal in full detail - but it seems to make no mention of archiving graphs with circular references. Is this implicitly supported, or explicitly unsupported?

While we’re at it, my only real exposure to archiving is through Foundation, so I’d like to know how everybody else understands these terms:

serialization - the process of “flattening” out an object graph into a serial stream of objects

encoding - the process of converting internal object data into an external format

archiving - the whole enchilada of serialization + encoding

Thanks!

-Kenny


(Colin Barrett) #11

Hi Itai,

Glad to see these proposal! I'm curious, have you or the other Swift folks
thought about how *users* of these new Codable protocols will interact with
resilience domains?

What I mean is that what appear to be private or internal identifiers, and
thus changeable at will, may actually be fragile in that changing them will
break the ability to decode archives encoded by previous versions.

Making this safer could mean:
- Encoding only public properties
- Adding some form of indirection (a la ObjC non-fragile ivars?)
- Compiler warning (or disallowing) changes to properties in certain
situations.

I imagine the specifics would need to follow the rest of the plans for
resilience.

It's likely that this could be addressed by a future proposal, as for the
time being developers can simply "not hold it wrong" :wink:

Thanks,
-Colin

···

On Wed, Mar 15, 2017 at 6:52 PM Itai Ferber via swift-evolution < swift-evolution@swift.org> wrote:

Hi everyone,

The following introduces a new Swift-focused archival and serialization
API as part of the Foundation framework. We’re interested in improving the
experience and safety of performing archival and serialization, and are
happy to receive community feedback on this work.
Because of the length of this proposal, the *Appendix* and *Alternatives
Considered* sections have been omitted here, but are available in the full
proposal <https://github.com/apple/swift-evolution/pull/639> on the
swift-evolution repo. The full proposal also includes an *Unabridged API* for
further consideration.

Without further ado, inlined below.

— Itai

Swift Archival & Serialization

   - Proposal: SE-NNNN <https://github.com/apple/swift-evolution/pull/639>
   - Author(s): Itai Ferber <https://github.com/itaiferber>, Michael LeHew
   <https://github.com/mlehew>, Tony Parker <https://github.com/parkera>
   - Review Manager: TBD
   - Status: *Awaiting review*
   - Associated PRs:
      - #8124 <https://github.com/apple/swift/pull/8124>
      - #8125 <https://github.com/apple/swift/pull/8125>

Introduction

Foundation's current archival and serialization APIs (NSCoding,
NSJSONSerialization, NSPropertyListSerialization, etc.), while fitting
for the dynamism of Objective-C, do not always map optimally into Swift.
This document lays out the design of an updated API that improves the
developer experience of performing archival and serialization in Swift.

Specifically:

   - It aims to provide a solution for the archival of Swift struct and
   enum types
   - It aims to provide a more type-safe solution for serializing to
   external formats, such as JSON and plist

Motivation

The primary motivation for this proposal is the inclusion of native Swift
enum and struct types in archival and serialization. Currently,
developers targeting Swift cannot participate in NSCoding without being
willing to abandon enum and structtypes — NSCoding is an @objc protocol,
conformance to which excludes non-class types. This is can be limiting in
Swift because small enums and structs can be an idiomatic approach to
model representation; developers who wish to perform archival have to
either forgo the Swift niceties that constructs like enumsprovide, or
provide an additional compatibility layer between their "real" types and
their archivable types.

Secondarily, we would like to refine Foundation's existing serialization
APIs (NSJSONSerialization and NSPropertyListSerialization) to better
match Swift's strong type safety. From experience, we find that the
conversion from the unstructured, untyped data of these formats into
strongly-typed data structures is a good fit for archival mechanisms,
rather than taking the less safe approach that 3rd-party JSON conversion
approaches have taken (described further in an appendix below).

We would like to offer a solution to these problems without sacrificing
ease of use or type safety.
Agenda

This proposal is the first stage of three that introduce different facets
of a whole Swift archival and serialization API:

   1. This proposal describes the basis for this API, focusing on the
   protocols that users adopt and interface with
   2. The next stage will propose specific API for new encoders
   3. The final stage will discuss how this new API will interop with
   NSCoding as it is today

SE-NNNN provides stages 2 and 3.
Proposed solution

We will be introducing the following new types:

   - protocol Codable: Adopted by types to opt into archival. Conformance
   may be automatically derived in cases where all properties are also
   Codable.
   - protocol CodingKey: Adopted by types used as keys for keyed
   containers, replacing String keys with semantic types. Conformance may
   be automatically derived in most cases.
   - protocol Encoder: Adopted by types which can take Codable values and
   encode them into a native format.
      - class KeyedEncodingContainer<Key : CodingKey>: Subclasses of this
      type provide a concrete way to store encoded values by CodingKey.
      Types adopting Encoder should provide subclasses of
      KeyedEncodingContainer to vend.
      - protocol SingleValueEncodingContainer: Adopted by types which
      provide a concrete way to store a single encoded value. Types adopting
      Encoder should provide types conforming to
      SingleValueEncodingContainer to vend (but in many cases will be
      able to conform to it themselves).
   - protocol Decoder: Adopted by types which can take payloads in a
   native format and decode Codable values out of them.
      - class KeyedDecodingContainer<Key : CodingKey>: Subclasses of this
      type provide a concrete way to retrieve encoded values from storage by
      CodingKey. Types adopting Decoder should provide subclasses of
      KeyedDecodingContainer to vend.
      - protocol SingleValueDecodingContainer: Adopted by types which
      provide a concrete way to retrieve a single encoded value from storage.
      Types adopting Decoder should provide types conforming to
      SingleValueDecodingContainer to vend (but in many cases will be
      able to conform to it themselves).

For end users of this API, adoption will primarily involve the Codable
and CodingKey protocols. In order to participate in this new archival
system, developers must add Codable conformance to their types:

// If all properties are Codable, implementation is automatically derived:public struct Location : Codable {
    public let latitude: Double
    public let longitude: Double}
public enum Animal : Int, Codable {
    case chicken = 1
    case dog
    case turkey
    case cow}
public struct Farm : Codable {
    public let name: String
    public let location: Location
    public let animals: [Animal]}

With developer participation, we will offer encoders and decoders
(described in SE-NNNN, not here) that take advantage of this conformance to
offer type-safe serialization of user models:

let farm = Farm(name: "Old MacDonald's Farm",
                location: Location(latitude: 51.621648, longitude: 0.269273),
                animals: [.chicken, .dog, .cow, .turkey, .dog, .chicken, .cow, .turkey, .dog])let payload: Data = try JSONEncoder().encode(farm)
do {
    let farm = try JSONDecoder().decode(Farm.self, from: payload)

    // Extracted as user types:
    let coordinates = "\(farm.location.latitude, farm.location.longitude)"} catch {
    // Encountered error during deserialization}

This gives developers access to their data in a type-safe manner and a
recognizable interface.
Detailed design

To support user types, we expose the Codable protocol:

/// Conformance to `Codable` indicates that a type can marshal itself into and out of an external representation.public protocol Codable {
    /// Initializes `self` by decoding from `decoder`.
    ///
    /// - parameter decoder: The decoder to read data from.
    /// - throws: An error if reading from the decoder fails, or if read data is corrupted or otherwise invalid.
    init(from decoder: Decoder) throws

    /// Encodes `self` into the given encoder.
    ///
    /// If `self` fails to encode anything, `encoder` will encode an empty `.default` container in its place.
    ///
    /// - parameter encoder: The encoder to write data to.
    /// - throws: An error if any values are invalid for `encoder`'s format.
    func encode(to encoder: Encoder) throws}

By adopting Codable, user types opt in to this archival system.

Structured types (i.e. types which encode as a collection of properties)
encode and decode their properties in a keyed manner. Keys may be String-convertible
or Int-convertible (or both), and user types which have properties should
declare semantic key enums which map keys to their properties. Keys must
conform to the CodingKey protocol:

/// Conformance to `CodingKey` indicates that a type can be used as a key for encoding and decoding.public protocol CodingKey {
    /// The string to use in a named collection (e.g. a string-keyed dictionary).
    var stringValue: String? { get }

    /// Initializes `self` from a string.
    ///
    /// - parameter stringValue: The string value of the desired key.
    /// - returns: An instance of `Self` from the given string, or `nil` if the given string does not correspond to any instance of `Self`.
    init?(stringValue: String)

    /// The int to use in an indexed collection (e.g. an int-keyed dictionary).
    var intValue: Int? { get }

    /// Initializes `self` from an integer.
    ///
    /// - parameter intValue: The integer value of the desired key.
    /// - returns: An instance of `Self` from the given integer, or `nil` if the given integer does not correspond to any instance of `Self`.
    init?(intValue: Int)}

For most types, String-convertible keys are a reasonable default; for
performance, however, Int-convertible keys are preferred, and Encoders may
choose to make use of Ints over Strings. Framework types should provide
keys which have both for flexibility and performance across different types
of Encoders. It is generally an error to provide a key which has neither
a stringValue nor an intValue.

By default, CodingKey conformance can be derived for enums which have
either String or Int backing:

enum Keys1 : CodingKey {
    case a // (stringValue: "a", intValue: nil)
    case b // (stringValue: "b", intValue: nil)}
enum Keys2 : String, CodingKey {
    case c = "foo" // (stringValue: "foo", intValue: nil)
    case d // (stringValue: "d", intValue: nil)}
enum Keys3 : Int, CodingKey {
    case e = 4 // (stringValue: "e", intValue: 4)
    case f // (stringValue: "f", intValue: 5)
    case g = 9 // (stringValue: "g", intValue: 9)}

Coding keys which are not enums, have associated values, or have other
raw representations must implement these methods manually.

In addition to automatic CodingKey conformance derivation for enums,
Codableconformance can be automatically derived for certain types as well:

   1. Types whose properties are all either Codable or primitive get an
   automatically derived String-backed CodingKeys enum mapping properties
   to case names
   2. Types falling into (1) and types which provide a CodingKeys enum (directly
   or via a typealias) whose case names map to properties which are all
   Codableget automatic derivation of init(from:) and encode(to:) using
   those properties and keys. Types may choose to provide a custom
   init(from:) or encode(to:) (or both); whichever they do not provide
   will be automatically derived
   3. Types which fall into neither (1) nor (2) will have to provide a
   custom key type and provide their own init(from:) and encode(to:)

Many types will either allow for automatic derivation of all codability
(1), or provide a custom key subset and take advantage of automatic method
derivation (2).
Encoding and Decoding

Types which are encodable encode their data into a container provided by
their Encoder:

/// An `Encoder` is a type which can encode values into a native format for external representation.public protocol Encoder {
    /// Populates `self` with an encoding container (of `.default` type) and returns it, keyed by the given key type.
    ///
    /// - parameter type: The key type to use for the container.
    /// - returns: A new keyed encoding container.
    /// - precondition: May not be called after a previous `self.container(keyedBy:)` call of a different `EncodingContainerType`.
    /// - precondition: May not be called after a value has been encoded through a prior `self.singleValueContainer()` call.
    func container<Key : CodingKey>(keyedBy type: Key.Type) -> KeyedEncodingContainer<Key>

    /// Returns an encoding container appropriate for holding a single primitive value.
    ///
    /// - returns: A new empty single value container.
    /// - precondition: May not be called after a prior `self.container(keyedBy:)` call.
    /// - precondition: May not be called after a value has been encoded through a previous `self.singleValueContainer()` call.
    func singleValueContainer() -> SingleValueEncodingContainer

    /// The path of coding keys taken to get to this point in encoding.
    var codingKeyContext: [CodingKey] { get }}
// Continuing examples from before; below is automatically generated by the compiler if no customization is needed.public struct Location : Codable {
    private enum CodingKeys : CodingKey {
        case latitutude
        case longitude
    }

    public func encode(to encoder: Encoder) throws {
        // Generic keyed encoder gives type-safe key access: cannot encode with keys of the wrong type.
        let container = encoder.container(keyedBy: CodingKeys.self)

        // The encoder is generic on the key -- free key autocompletion here.
        try container.encode(latitude, forKey: .latitude)
        try container.encode(longitude, forKey: .longitude)
    }}
public struct Farm : Codable {
    private enum CodingKeys : CodingKey {
        case name
        case location
        case animals
    }

    public func encode(to encoder: Encoder) throws {
        let container = encoder.container(keyedBy: CodingKeys.self)
        try container.encode(name


(Ben Rimmington) #12

Instead of a CodingKeys enum, could the KeyPath proposal <https://github.com/apple/swift-evolution/pull/644> be utilized somehow?

-- Ben

···

On 15 Mar 2017, at 22:40, Itai Ferber wrote:

The following introduces a new Swift-focused archival and serialization API as part of the Foundation framework. We’re interested in improving the experience and safety of performing archival and serialization, and are happy to receive community feedback on this work.


(Itai Ferber) #13

Hi Oliver,

Thanks for your comments! We thought about this and we agree overall — we will incorporate this suggestion along with others in the next batch update as long as nothing prohibitive comes up.

— Itai

···

On 23 Mar 2017, at 7:49, Oliver Jones wrote:

Like everyone I’m excited by this new proposal. But…

protocol Codable: Adopted by types to opt into archival. Conformance may be automatically derived in cases where all properties are also Codable.

… can I make one suggestion. Please do not repeat the mistakes of NSCoding in combining the encoding and decoding into a single protocol. Just as there are Encoder and Decoder classes their should be Encodable and Decodable protocols (maybe have an aggregate Codable protocol for convenience but do not force it).

My reasoning:

Sometimes you only want to decode or encode and object and not vice versa. This is often the case with Web APIs and JSON serialisation.

Eg:

Often an app only consumes (decodes) JSON encoded objects and never writes them out (a read only app for example). So the encode(to:) methods are completely redundant and someone adopting Codable should not be forced to write them.

If only I had a dollar for all the times I’ve seen this sort of code in projects:

class MyClass : NSCoding {
init?(coder: NSCoder) {
  // ... some decoding code
}

func encode(with aCoder: NSCoder) {
   preconditionFailure(“Not implemented”)
}

Another example:

Web APIs often take data in a different structure as input (i.e. “Request” objects) than they output. These request objects are only ever encoded and never decoded by an application so implementing init(from:) is completely redundant.

Personally I think the approach taken by libraries like Wrap (https://github.com/johnsundell/wrap) and Unbox (https://github.com/JohnSundell/Unbox) is a much better design. Encoding and decoding should not be the same protocol.

Yes I understand that Codable could provide no-op (or preconditionFailure) protocol extension based default implementations of init(from:) and encode(to:) (or try to magic up implementations based on the Codable nature of public properties as suggested in the proposal) but to me that seems like a hack that is papering over bad design. I think this joint Codable design probably fails the Liskov substitution principle too.

So I again implore you to consider splitting Codable into two protocols, one for encoding and another for decoding.

Sorry if I’m repeating what other people have already said. I’ve not read every response to this proposal on the list.

Regards


(Oliver Jones) #14

Like everyone I’m excited by this new proposal. But…

protocol Codable: Adopted by types to opt into archival. Conformance may be automatically derived in cases where all properties are also Codable.

… can I make one suggestion. Please do not repeat the mistakes of NSCoding in combining the encoding and decoding into a single protocol. Just as there are Encoder and Decoder classes their should be Encodable and Decodable protocols (maybe have an aggregate Codable protocol for convenience but do not force it).

My reasoning:

Sometimes you only want to decode or encode and object and not vice versa. This is often the case with Web APIs and JSON serialisation.

Eg:

Often an app only consumes (decodes) JSON encoded objects and never writes them out (a read only app for example). So the encode(to:) methods are completely redundant and someone adopting Codable should not be forced to write them.

If only I had a dollar for all the times I’ve seen this sort of code in projects:

class MyClass : NSCoding {
init?(coder: NSCoder) {
  // ... some decoding code
}

func encode(with aCoder: NSCoder) {
   preconditionFailure(“Not implemented”)
}
}

Another example:

Web APIs often take data in a different structure as input (i.e. “Request” objects) than they output. These request objects are only ever encoded and never decoded by an application so implementing init(from:) is completely redundant.

Personally I think the approach taken by libraries like Wrap (https://github.com/johnsundell/wrap) and Unbox (https://github.com/JohnSundell/Unbox) is a much better design. Encoding and decoding should not be the same protocol.

Yes I understand that Codable could provide no-op (or preconditionFailure) protocol extension based default implementations of init(from:) and encode(to:) (or try to magic up implementations based on the Codable nature of public properties as suggested in the proposal) but to me that seems like a hack that is papering over bad design. I think this joint Codable design probably fails the Liskov substitution principle too.

So I again implore you to consider splitting Codable into two protocols, one for encoding and another for decoding.

Sorry if I’m repeating what other people have already said. I’ve not read every response to this proposal on the list.

Regards


(Itai Ferber) #15

Hi everyone,

With feedback from swift-evolution and additional internal review, we've pushed updates to this proposal, and to the Swift Encoders <https://github.com/apple/swift-evolution/pull/640> proposal.
In the interest of not blowing up mail clients with the full HTML again, I'll simply be linking to the swift-evolution PR here <https://github.com/apple/swift-evolution/pull/639>, as well as the specific diff <https://github.com/apple/swift-evolution/pull/639/commits/d619eef9166f8b45ffac152d06376cbdab536241> of what's changed.

At a high level:

The Codable protocol has been split up into Encodable and Decodable
String keys on CodingKey are no longer optional
KeyedEncodingContainer has become KeyedEncodingContainerProtocol, with a concrete type-erased KeyedEncodingContainer struct to hold it
Array responsibilities have been removed from KeyedEncodingContainer, and have been added to a new UnkeyedEncodingContainer type
codingKeyContext has been renamed codingPath
There are some specific changes inline — I know it might be a bit of a pain, but let's keep discussion here on the mailing list instead of on GitHub.
We'll be looking to start the official review process very soon, so we're interested in any additional feedback.

Thanks!

— Itai


(Itai Ferber) #16

Thanks Zach, that's encouraging to hear!
Apologies for the original message size — to be honest I didn't realize how large it was myself until my own email was delayed in moderation.

I guess, then, **when responding to the original email, please quote only the sections that you are responding to to best ensure that your email makes it through.** Sorry for the inconvenience!

···

On 15 Mar 2017, at 17:45, Zach Waldowski via swift-evolution wrote:

Holy cow. There's much to digest here (so much so that my initial
response, which quoted its content, was denied by the mailing list).
After an initial reading, I don't just want this now, I want it
yesterday. I'm already imaging the encoder/decoders I want to build.
Very exciting.

Best,

  Zachary Waldowski

  zach@waldowski.me

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Itai Ferber) #17

Thanks Joe, and thanks for passing this along!

To those who are curious, we use abstract base classes for a cascading list of reasons:

* We need to be able to represent keyed encoding and decoding containers as abstract types which are generic on a key type
* There are two ways to support abstraction in this way: protocol & type constraints, and generic types
   * Since Swift protocols are not generic, we unfortunately cannot write `protocol KeyedEncodingContainer<Key : CodingKey> { ... }`, which is the "ideal" version of what we're trying to represent
* Let's try this with a protocol first (simplified here):

   protocol Container {
         associatedtype Key : CodingKey
   }

   func container<Key : CodingKey, Cont : Container>(_ type: Key.Type) -> Cont where Cont.Key == Key {
         // return something
   }

   This looks promising so far — let's try to make it concrete:

   struct ConcreteContainer<K : CodingKey> : Container {
         typealias Key = K
   }

   func container<Key : CodingKey, Cont : Container>(_ type: Key.Type) -> Cont where Cont.Key == Key {
         return ConcreteContainer<Key>() // error: Cannot convert return expression of type 'ConcreteContainer<Key>' to return type 'Cont'
   }

   Joe or anyone from the Swift team can describe this better, but this is my poor-man's explanation of why this happens. Swift's type constraints are "directional" in a sense. You can constrain a type going _into_ a function, but not _out of_ a function. There is no type I could return from inside of `container()` which would satisfy this constraint, because the constraint can only be satisfied by turning `Cont` into a concrete type from the _outside_.

   Okay, well let's try this:

   func container... {
         return ConcreteContainer<Key>() as! Cont
   }

   This compiles fine! Hmm, let's try to use it:

   container(Int.self) // error: Generic parameter 'Cont' could not be inferred

   The type constraint can only be fulfilled from the outside, not the inside. The function call itself has no context for the concrete type that this would return, so this is a no-go.

* If we can't do it with type constraints in this way, is it possible with generic types? Yep! Generic types satisfy this without a problem. However, since we don't have generic protocols, we have to use a generic abstract base class to represent the same concept — an abstract container generic on the type of key which dynamically dispatches to the "real" subclassed type

Hopes that gives some simplified insight into the nature of this decision.

···

On 15 Mar 2017, at 18:18, Joe Groff wrote:

Congrats on getting this out! A question from the field:

https://twitter.com/mdiep/status/842178457115230210 Why does the Swift Serialization API proposal use abstract base classes?

-Joe

On Mar 15, 2017, at 3:40 PM, Itai Ferber via swift-evolution >> <swift-evolution@swift.org> wrote:

Hi everyone,

The following introduces a new Swift-focused archival and serialization API as part of the Foundation framework. We’re interested in improving the experience and safety of performing archival and serialization, and are happy to receive community feedback on this work.
Because of the length of this proposal, the Appendix and Alternatives Considered sections have been omitted here, but are available in the full proposal on the swift-evolution repo. The full proposal also includes an Unabridged API for further consideration.

Without further ado, inlined below.

— Itai

Swift Archival & Serialization
  • Proposal: SE-NNNN
  • Author(s): Itai Ferber, Michael LeHew, Tony Parker
  • Review Manager: TBD
  • Status: Awaiting review
  • Associated PRs:
    • #8124
    • #8125
Introduction
Foundation's current archival and serialization APIs (NSCoding, NSJSONSerialization, NSPropertyListSerialization, etc.), while fitting for the dynamism of Objective-C, do not always map optimally into Swift. This document lays out the design of an updated API that improves the developer experience of performing archival and serialization in Swift.

Specifically:

  • It aims to provide a solution for the archival of Swift struct and enum types
  • It aims to provide a more type-safe solution for serializing to external formats, such as JSON and plist
Motivation
The primary motivation for this proposal is the inclusion of native Swift enum and struct types in archival and serialization. Currently, developers targeting Swift cannot participate in NSCoding without being willing to abandon enum and structtypes — NSCoding is an @objc protocol, conformance to which excludes non-class types. This is can be limiting in Swift because small enums and structs can be an idiomatic approach to model representation; developers who wish to perform archival have to either forgo the Swift niceties that constructs like enumsprovide, or provide an additional compatibility layer between their "real" types and their archivable types.

Secondarily, we would like to refine Foundation's existing serialization APIs (NSJSONSerialization and NSPropertyListSerialization) to better match Swift's strong type safety. From experience, we find that the conversion from the unstructured, untyped data of these formats into strongly-typed data structures is a good fit for archival mechanisms, rather than taking the less safe approach that 3rd-party JSON conversion approaches have taken (described further in an appendix below).

We would like to offer a solution to these problems without sacrificing ease of use or type safety.

Agenda
This proposal is the first stage of three that introduce different facets of a whole Swift archival and serialization API:

  • This proposal describes the basis for this API, focusing on the protocols that users adopt and interface with
  • The next stage will propose specific API for new encoders
  • The final stage will discuss how this new API will interop with NSCoding as it is today
SE-NNNN provides stages 2 and 3.

Proposed solution
We will be introducing the following new types:

  • protocol Codable: Adopted by types to opt into archival. Conformance may be automatically derived in cases where all properties are also Codable.
  • protocol CodingKey: Adopted by types used as keys for keyed containers, replacing String keys with semantic types. Conformance may be automatically derived in most cases.
  • protocol Encoder: Adopted by types which can take Codable values and encode them into a native format.
    • class KeyedEncodingContainer<Key : CodingKey>: Subclasses of this type provide a concrete way to store encoded values by CodingKey. Types adopting Encoder should provide subclasses of KeyedEncodingContainer to vend.
    • protocol SingleValueEncodingContainer: Adopted by types which provide a concrete way to store a single encoded value. Types adopting Encoder should provide types conforming to SingleValueEncodingContainer to vend (but in many cases will be able to conform to it themselves).
  • protocol Decoder: Adopted by types which can take payloads in a native format and decode Codable values out of them.
    • class KeyedDecodingContainer<Key : CodingKey>: Subclasses of this type provide a concrete way to retrieve encoded values from storage by CodingKey. Types adopting Decoder should provide subclasses of KeyedDecodingContainer to vend.
    • protocol SingleValueDecodingContainer: Adopted by types which provide a concrete way to retrieve a single encoded value from storage. Types adopting Decoder should provide types conforming to SingleValueDecodingContainer to vend (but in many cases will be able to conform to it themselves).
For end users of this API, adoption will primarily involve the Codable and CodingKey protocols. In order to participate in this new archival system, developers must add Codable conformance to their types:

// If all properties are Codable, implementation is automatically derived:
public struct Location : Codable {

public let latitude: Double

public let longitude: Double
}

public enum Animal : Int, Codable {

case chicken = 1

case
dog

case
turkey

case
cow

}

public struct Farm : Codable {

public let name: String

public let location: Location

public let animals: [Animal]
}
With developer participation, we will offer encoders and decoders (described in SE-NNNN, not here) that take advantage of this conformance to offer type-safe serialization of user models:

let farm = Farm(name: "Old MacDonald's Farm",

                location
: Location(latitude: 51.621648, longitude: 0.269273),

                animals
: [.chicken, .dog, .cow, .turkey, .dog, .chicken, .cow, .turkey, .dog])
let payload: Data = try JSONEncoder().encode(farm)

do {

let farm = try JSONDecoder().decode(Farm.self, from: payload)

// Extracted as user types:

let coordinates = "\(farm.location.latitude, farm.location.longitude)"
} catch {

// Encountered error during deserialization
}
This gives developers access to their data in a type-safe manner and a recognizable interface.

Detailed design
To support user types, we expose the Codable protocol:

/// Conformance to `Codable` indicates that a type can marshal itself into and out of an external representation.
public protocol Codable {

/// Initializes `self` by decoding from `decoder`.

///

/// - parameter decoder: The decoder to read data from.

/// - throws: An error if reading from the decoder fails, or if read data is corrupted or otherwise invalid.

init(from decoder: Decoder) throws

/// Encodes `self` into the given encoder.

///

/// If `self` fails to encode anything, `encoder` will encode an empty `.default` container in its place.

///

/// - parameter encoder: The encoder to write data to.

/// - throws: An error if any values are invalid for `encoder`'s format.

func encode(to encoder: Encoder) throws
}
By adopting Codable, user types opt in to this archival system.

Structured types (i.e. types which encode as a collection of properties) encode and decode their properties in a keyed manner. Keys may be String-convertible or Int-convertible (or both), and user types which have properties should declare semantic key enums which map keys to their properties. Keys must conform to the CodingKey protocol:

/// Conformance to `CodingKey` indicates that a type can be used as a key for encoding and decoding.
public protocol CodingKey {

/// The string to use in a named collection (e.g. a string-keyed dictionary).

var stringValue: String? { get }

/// Initializes `self` from a string.

///

/// - parameter stringValue: The string value of the desired key.

/// - returns: An instance of `Self` from the given string, or `nil` if the given string does not correspond to any instance of `Self`.

init?(stringValue: String)

/// The int to use in an indexed collection (e.g. an int-keyed dictionary).

var intValue: Int? { get }

/// Initializes `self` from an integer.

///

/// - parameter intValue: The integer value of the desired key.

/// - returns: An instance of `Self` from the given integer, or `nil` if the given integer does not correspond to any instance of `Self`.

init?(intValue: Int)
}
For most types, String-convertible keys are a reasonable default; for performance, however, Int-convertible keys are preferred, and Encoders may choose to make use of Ints over Strings. Framework types should provide keys which have both for flexibility and performance across different types of Encoders. It is generally an error to provide a key which has neither a stringValue nor an intValue.

By default, CodingKey conformance can be derived for enums which have either String or Int backing:

enum Keys1 : CodingKey {

case a // (stringValue: "a", intValue: nil)

case b // (stringValue: "b", intValue: nil)
}

enum Keys2 : String, CodingKey {

case c = "foo" // (stringValue: "foo", intValue: nil)

case d // (stringValue: "d", intValue: nil)
}

enum Keys3 : Int, CodingKey {

case e = 4 // (stringValue: "e", intValue: 4)

case f // (stringValue: "f", intValue: 5)

case g = 9 // (stringValue: "g", intValue: 9)
}
Coding keys which are not enums, have associated values, or have other raw representations must implement these methods manually.

In addition to automatic CodingKey conformance derivation for enums, Codableconformance can be automatically derived for certain types as well:

  • Types whose properties are all either Codable or primitive get an automatically derived String-backed CodingKeys enum mapping properties to case names
  • Types falling into (1) and types which provide a CodingKeys enum (directly or via a typealias) whose case names map to properties which are all Codableget automatic derivation of init(from:) and encode(to:) using those properties and keys. Types may choose to provide a custom init(from:) or encode(to:) (or both); whichever they do not provide will be automatically derived
  • Types which fall into neither (1) nor (2) will have to provide a custom key type and provide their own init(from:) and encode(to:)
Many types will either allow for automatic derivation of all codability (1), or provide a custom key subset and take advantage of automatic method derivation (2).

Encoding and Decoding

Types which are encodable encode their data into a container provided by their Encoder:

/// An `Encoder` is a type which can encode values into a native format for external representation.
public protocol Encoder {

/// Populates `self` with an encoding container (of `.default` type) and returns it, keyed by the given key type.

///

/// - parameter type: The key type to use for the container.

/// - returns: A new keyed encoding container.

/// - precondition: May not be called after a previous `self.container(keyedBy:)` call of a different `EncodingContainerType`.

/// - precondition: May not be called after a value has been encoded through a prior `self.singleValueContainer()` call.

func container<Key : CodingKey>(keyedBy type: Key.Type) -> KeyedEncodingContainer<Key>

/// Returns an encoding container appropriate for holding a single primitive value.

///

/// - returns: A new empty single value container.

/// - precondition: May not be called after a prior `self.container(keyedBy:)` call.

/// - precondition: May not be called after a value has been encoded through a previous `self.singleValueContainer()` call.

func singleValueContainer() -> SingleValueEncodingContainer

/// The path of coding keys taken to get to this point in encoding.

var codingKeyContext: [CodingKey] { get }
}

// Continuing examples from before; below is automatically generated by the compiler if no customization is needed.
public struct Location : Codable {

private enum CodingKeys : CodingKey {

case
latitutude

case
longitude

}

public func encode(to encoder: Encoder) throws {

// Generic keyed encoder gives type-safe key access: cannot encode with keys of the wrong type.

let container = encoder.container(keyedBy: CodingKeys.self)

// The encoder is generic on the key -- free key autocompletion here.

try container.encode(latitude, forKey: .latitude)

try container.encode(longitude, forKey: .longitude)

}

public struct Farm : Codable {

private enum CodingKeys : CodingKey {

case
name

case
location

case
animals

}

public func encode(to encoder: Encoder) throws {

let container = encoder.container(keyedBy: CodingKeys.self)

try container.encode(name, forKey: .name)

try container.encode(location, forKey: .location)

try container.encode(animals, forKey: .animals)

}
Similarly, decodable types initialize from data read from their Decoder's container:

/// A `Decoder` is a type which can decode values from a native format into in-memory representations.
public protocol Decoder {

/// Returns the data stored in `self` as represented in a container keyed by the given key type.

///

/// - parameter type: The key type to use for the container.

/// - returns: A keyed decoding container view into `self`.

/// - throws: `CocoaError.coderTypeMismatch` if the encountered stored value is not a keyed container.

func container<Key : CodingKey>(keyedBy type: Key.Type) throws -> KeyedDecodingContainer<Key>

/// Returns the data stored in `self` as represented in a container appropriate for holding a single primitive value.

///

/// - returns: A single value container view into `self`.

/// - throws: `CocoaError.coderTypeMismatch` if the encountered stored value is not a single value container.

func singleValueContainer() throws -> SingleValueDecodingContainer

/// The path of coding keys taken to get to this point in decoding.

var codingKeyContext: [CodingKey] { get }
}

// Continuing examples from before; below is automatically generated by the compiler if no customization is needed.
public struct Location : Codable {

public init(from decoder: Decoder) throws {

let container = try decoder.container(keyedBy: CodingKeys.self)

        latitude
= try container.decode(Double.self, forKey: .latitude)

        longitude
= try container.decode(Double.self, forKey: .longitude)

}

public struct Farm : Codable {

public init(from decoder: Decoder) throws {

let container = try decoder.container(keyedBy: CodingKeys.self)

        name
= try container.decode(String.self, forKey: .name)

        location
= try container.decode(Location.self, forKey: .location)

        animals
= try container.decode([Animal].self, forKey: .animals)

}
Keyed Encoding Containers

Keyed encoding containers are the primary interface that most Codable types interact with for encoding and decoding. Through these, Codable types have strongly-keyed access to encoded data by using keys that are semantically correct for the operations they want to express.

Since semantically incompatible keys will rarely (if ever) share the same key type, it is impossible to mix up key types within the same container (as is possible with Stringkeys), and since the type is known statically, keys get autocompletion by the compiler.

/// `KeyedEncodingContainer` is a generic abstract base class that provides a view into an `Encoders` storage and is used to hold the encoded properties of a `Codable` type.
///
/// Encoders should provide subclasses of `KeyedEncodingContainer` for their format.

open
class KeyedEncodingContainer<Key : CodingKey> {

/// Encodes the given value for the given key.

///

/// - parameter value: The value to encode.

/// - parameter key: The key to associate the value with.

/// - throws: `CocoaError.coderInvalidValue` if the given value is invalid in the current context for this format.

/// - precondition: The key must have a `stringValue` or `intValue` appropriate for the encoding container type.

    open
func encode<Value : Codable>(_ value: Value?, forKey key: Key) throws

/// Encodes the given value for the given key.

///

/// - parameter value: The value to encode.

/// - parameter key: The key to associate the value with.

/// - throws: `CocoaError.coderInvalidValue` if the given value is invalid in the current context for this format.

/// - precondition: The key must have a `stringValue` or `intValue` appropriate for the encoding container type.

    open
func encode(_ value: Bool?, forKey key: Key) throws

    open
func encode(_ value: Int?, forKey key: Key) throws

    open
func encode(_ value: Int8?, forKey key: Key) throws

    open
func encode(_ value: Int16?, forKey key: Key) throws

    open
func encode(_ value: Int32?, forKey key: Key) throws

    open
func encode(_ value: Int64?, forKey key: Key) throws

    open
func encode(_ value: UInt?, forKey key: Key) throws

    open
func encode(_ value: UInt8?, forKey key: Key) throws

    open
func encode(_ value: UInt16?, forKey key: Key) throws

    open
func encode(_ value: UInt32?, forKey key: Key) throws

    open
func encode(_ value: UInt64?, forKey key: Key) throws

    open
func encode(_ value: Float?, forKey key: Key) throws

    open
func encode(_ value: Double?, forKey key: Key) throws

    open
func encode(_ value: String?, forKey key: Key) throws

    open
func encode(_ value: Data?, forKey key: Key) throws

/// Encodes the given object weakly for the given key.

///

/// For `Encoder`s that implement this functionality, this will only encode the given object and associate it with the given key if it encoded unconditionally elsewhere in the archive (either previously or in the future).

///

/// For formats which don't support this feature, the default implementation encodes the given object unconditionally.

///

/// - parameter object: The object to encode.

/// - parameter key: The key to associate the object with.

/// - throws: `CocoaError.coderInvalidValue` if the given value is invalid in the current context for this format.

/// - precondition: The key must have a `stringValue` or `intValue` appropriate for the encoding container type.

    open
func encodeWeak<Object : AnyObject & Codable>(_ object: Object?, forKey key: Key) throws

/// The path of coding keys taken to get to this point in encoding.

    open
var codingKeyContext: [CodingKey]
}

/// `KeyedDecodingContainer` is a generic abstract base class that provides a view into an `Decoders` storage and is used to hold the encoded properties of a `Codable` type.
///
/// Decoders should provide subclasses of `KeyedDecodingContainer` for their format.

open
class KeyedDecodingContainer<Key : CodingKey> {

/// All the keys the `Decoder` has for this container.

///

/// Different keyed containers from the same `Decoder` may return different keys here; it is possible to encode with multiple key types which are not convertible to one another. This should report all keys present which are convertible to the requested type.

    open
var allKeys: [Key]

/// Returns whether the `Decoder` contains a value associated with the given key.

///

/// The value associated with the given key may be a null value as appropriate for the data format.

///

/// - parameter key: The key to search for.

/// - returns: Whether the `Decoder` has an entry for the given key.

    open
func contains(_ key: Key) -> Bool

/// Decodes a value of the given type for the given key.

///

/// A default implementation is given for these types which calls into the abstract `decodeIfPresent` implementations below.

///

/// - parameter type: The type of value to decode.

/// - parameter key: The key that the decoded value is associated with.

/// - returns: A value of the requested type, if present for the given key and convertible to the requested type.

/// - throws: `CocoaError.coderTypeMismatch` if the encountered encoded value is not convertible to the requested type.

/// - throws: `CocoaError.coderValueNotFound` if `self` does not have an entry for the given key or if the value is null.

    open
func decode(_ type: Bool.Type, forKey key: Key) throws -> Bool

    open
func decode(_ type: Int.Type, forKey key: Key) throws -> Int

    open
func decode(_ type: Int8.Type, forKey key: Key) throws -> Int8

    open
func decode(_ type: Int16.Type, forKey key: Key) throws -> Int16

    open
func decode(_ type: Int32.Type, forKey key: Key) throws -> Int32

    open
func decode(_ type: Int64.Type, forKey key: Key) throws -> Int64

    open
func decode(_ type: UInt.Type, forKey key: Key) throws -> UInt

    open
func decode(_ type: UInt8.Type, forKey key: Key) throws -> UInt8

    open
func decode(_ type: UInt16.Type, forKey key: Key) throws -> UInt16

    open
func decode(_ type: UInt32.Type, forKey key: Key) throws -> UInt32

    open
func decode(_ type: UInt64.Type, forKey key: Key) throws -> UInt64

    open
func decode(_ type: Float.Type, forKey key: Key) throws -> Float

    open
func decode(_ type: Double.Type, forKey key: Key) throws -> Double

    open
func decode(_ type: String.Type, forKey key: Key) throws -> String

    open
func decode(_ type: Data.Type, forKey key: Key) throws -> Data

    open
func decode<Value : Codable>(_ type: Value.Type, forKey key: Key) throws -> Value

/// Decodes a value of the given type for the given key, if present.

///

/// This method returns `nil` if the container does not have a value associated with `key`, or if the value is null. The difference between these states can be disambiguated with a `contains(_:)` call.

///

/// - parameter type: The type of value to decode.

/// - parameter key: The key that the decoded value is associated with.

/// - returns: A decoded value of the requested type, or `nil` if the `Decoder` does not have an entry associated with the given key, or if the value is a null value.

/// - throws: `CocoaError.coderTypeMismatch` if the encountered encoded value is not convertible to the requested type.

    open
func decodeIfPresent(_ type: Bool.Type, forKey key: Key) throws -> Bool?

    open
func decodeIfPresent(_ type: Int.Type, forKey key: Key) throws -> Int?

    open
func decodeIfPresent(_ type: Int8.Type, forKey key: Key) throws -> Int8?

    open
func decodeIfPresent(_ type: Int16.Type, forKey key: Key) throws -> Int16?

    open
func decodeIfPresent(_ type: Int32.Type, forKey key: Key) throws -> Int32?

    open
func decodeIfPresent(_ type: Int64.Type, forKey key: Key) throws -> Int64?

    open
func decodeIfPresent(_ type: UInt.Type, forKey key: Key) throws -> UInt?

    open
func decodeIfPresent(_ type: UInt8.Type, forKey key: Key) throws -> UInt8?

    open
func decodeIfPresent(_ type: UInt16.Type, forKey key: Key) throws -> UInt16?

    open
func decodeIfPresent(_ type: UInt32.Type, forKey key: Key) throws -> UInt32?

    open
func decodeIfPresent(_ type: UInt64.Type, forKey key: Key) throws -> UInt64?

    open
func decodeIfPresent(_ type: Float.Type, forKey key: Key) throws -> Float?

    open
func decodeIfPresent(_ type: Double.Type, forKey key: Key) throws -> Double?

    open
func decodeIfPresent(_ type: String.Type, forKey key: Key) throws -> String?

    open
func decodeIfPresent(_ type: Data.Type, forKey key: Key) throws -> Data?

    open
func decodeIfPresent<Value : Codable>(_ type: Value.Type, forKey key: Key) throws -> Value?

/// The path of coding keys taken to get to this point in decoding.

    open
var codingKeyContext: [CodingKey]
}
These encode(_:forKey:) and decode(_:forKey:) overloads give strong, static type guarantees about what is encodable (preventing accidental attempts to encode an invalid type), and provide a list of primitive types which are common to all encoders and decoders that users can rely on.

Coming in Swift 4 is the ability to express that "a collection of things which are Codable is Codable" (conditional conformance), allowing collections which we extend (Array, Dictionary, etc.) to fall into these overloads as well.

Encoding Container Types

For some types, the container into which they encode has meaning. Especially when coding for a specific output format (e.g. when communicating with a JSON API), a type may wish to explicitly encode as an array or a dictionary:

// Continuing from before
public protocol Encoder {

/// Populates `self` with an encoding container of the given type and returns it, keyed by the given key type.

///

/// A default implementation of `Encoder.container(keyedBy:)` calls this method with a container type of `.default`.

///

/// - parameter keyType: The key type to use for the container.

/// - parameter containerType: The container type to create.

/// - returns: A new keyed encoding container.

/// - precondition: May not be called after a previous `self.container(keyedBy:)` call of a different `EncodingContainerType`.

/// - precondition: May not be called after a value has been encoded through a prior `self.singleValueContainer()` call.

func container<Key : CodingKey>(keyedBy keyType: Key.Type, type containerType: EncodingContainerType) -> KeyedEncodingContainer<Key>
}

/// An `EncodingContainerType` specifies the type of container an `Encoder` should use to store values.
public enum EncodingContainerType {

/// The `Encoder`'s preferred container type; equivalent to either `.array` or `.dictionary` as appropriate for the encoder.

case `default
`

/// Explicitly requests the use of an array to store encoded values.

case
array

/// Explicitly requests the use of a dictionary to store encoded values.

case
dictionary

}
Single Value Containers

For other types, an array or dictionary container may not even make sense (e.g. values which are RawRepresentable as a single primitive value). Those types may encode and decode directly as a single value, instead of requesting a keyed container:

/// A `SingleValueEncodingContainer` is a container which can support the storage and direct encoding of a single non-keyed value.
public protocol SingleValueEncodingContainer {

/// Encodes a single value of the given type.

///

/// - parameter value: The value to encode.

/// - throws: `CocoaError.coderInvalidValue` if the given value is invalid in the current context for this format.

/// - precondition: May not be called after a previous `self.encode(_:)` call.

func encode(_ value: Bool) throws

func encode(_ value: Int) throws

func encode(_ value: Int8) throws

func encode(_ value: Int16) throws

func encode(_ value: Int32) throws

func encode(_ value: Int64) throws

func encode(_ value: UInt) throws

func encode(_ value: UInt8) throws

func encode(_ value: UInt16) throws

func encode(_ value: UInt32) throws

func encode(_ value: UInt64) throws

func encode(_ value: Float) throws

func encode(_ value: Double) throws

func encode(_ value: String) throws

func encode(_ value: Data) throws
}

/// A `SingleValueDecodingContainer` is a container which can support the storage and direct decoding of a single non-keyed value.
public protocol SingleValueDecodingContainer {

/// Decodes a single value of the given type.

///

/// - parameter type: The type to decode as.

/// - returns: A value of the requested type.

/// - throws: `CocoaError.coderTypeMismatch` if the encountered encoded value cannot be converted to the requested type.

func decode(_ type: Bool.Type) throws -> Bool

func decode(_ type: Int.Type) throws -> Int

func decode(_ type: Int8.Type) throws -> Int8

func decode(_ type: Int16.Type) throws -> Int16

func decode(_ type: Int32.Type) throws -> Int32

func decode(_ type: Int64.Type) throws -> Int64

func decode(_ type: UInt.Type) throws -> UInt

func decode(_ type: UInt8.Type) throws -> UInt8

func decode(_ type: UInt16.Type) throws -> UInt16

func decode(_ type: UInt32.Type) throws -> UInt32

func decode(_ type: UInt64.Type) throws -> UInt64

func decode(_ type: Float.Type) throws -> Float

func decode(_ type: Double.Type) throws -> Double

func decode(_ type: String.Type) throws -> String

func decode(_ type: Data.Type) throws -> Data
}

// Continuing example from before; below is automatically generated by the compiler if no customization is needed.
public enum Animal : Int, Codable {

public func encode(to encoder: Encoder) throws {

// Encode as a single value; no keys.

try encoder.singleValueContainer.encode(self.rawValue)

}

public init(from decoder: Decoder) throws {

// Decodes as a single value; no keys.

let intValue = try decoder.singleValueContainer().decode(Int.self)

if let value = Self(rawValue: intValue) {

self =
value

} else {

throw CocoaError.error(.coderReadCorrupt)

}

}
In the example given above, since Animal uses a single value container, [.chicken, .dog, .cow, .turkey, .dog, .chicken, .cow, .turkey, .dog]would encode directly as [1, 2, 4, 3, 2, 1, 4, 3, 2].

Nesting

In practice, some types may also need to control how data is nested within their container, or potentially nest other containers within their container. Keyed containers allow this by returning nested containers of differing key types:

// Continuing from before

open
class KeyedEncodingContainer<Key : CodingKey> {

/// Stores an encoding container for the given key and returns it.

///

/// - parameter keyType: The key type to use for the container.

/// - parameter containerType: The container type to create.

/// - parameter key: The key to encode the container for.

/// - returns: A new keyed encoding container.

    open
func nestedContainer<NestedKey : CodingKey>(keyedBy keyType: NestedKey.Type, type containerType: EncodingContainerType, forKey key: Key) -> KeyedEncodingContainer<NestedKey>
}

open
class KeyedDecodingContainer<Key : CodingKey> {

/// Returns the data stored for the given key as represented in a container keyed by the given key type.

///

/// - parameter type: The key type to use for the container.

/// - parameter key: The key that the nested container is associated with.

/// - returns: A keyed decoding container view into `self`.

/// - throws: `CocoaError.coderTypeMismatch` if the encountered stored value is not a container.

    open
func nestedContainer<NestedKey : CodingKey>(keyedBy type: NestedKey.Type, forKey key: Key) throws -> KeyedDecodingContainer<NestedKey>
}
This can be common when coding against specific external data representations:

// User type for interfacing with a specific JSON API. JSON API expects encoding as {"id": ..., "properties": {"name": ..., "timestamp": ...}}. Swift type differs from encoded type, and encoding needs to match a spec:
struct Record : Codable {

// We care only about these values from the JSON payload

let id: Int

let name: String

let timestamp: Double

// ...

private enum Keys : CodingKey {

case
id

case
properties

}

private enum PropertiesKeys : CodingKey {

case
name

case
timestamp

}

public func encode(to encoder: Encoder) throws {

let container = encoder.container(keyedBy: Keys.self, type: .dictionary)

try container.encode(id, forKey: .id)

// Set a dictionary for the "properties" key

let nested = container.nestedContainer(keyedBy: PropertiesKeys.self, type: .dictionary, forKey: .properties)

try nested.encode(name, forKey: .name)

try nested.encode(timestamp, forKey: .timestamp)

}

public init(from decoder: Decoder) throws {

let container = try decoder.container(keyedBy: Keys.self)

        id
= try container.decode(Int.self, forKey: .id)

let nested = try container.nestedContainer(keyedBy: PropertiesKeys.self, forKey: .properties)

        name
= try nested.decode(String.self, forKey: .name)

        timestamp
= try nested.decode(Double.self, forKey: .timestamp)

}
Inheritance

Inheritance in this system is supported much like it is with NSCoding — on encoding, objects which inherit from a type that is Codable encode super using their encoder, and pass a decoder to super.init(from:) on decode. With the existing NSCoding API, this is most often done like so, by convention:

- (void)encodeWithCoder:(NSCoder *)encoder {

[super encodeWithCoder:encoder];

// ... encode properties
}

- (instancetype)initWithCoder:(NSCoder *)decoder {

if ((self = [super initWithCoder:decoder])) {

// ... decode properties

}

return self;
}
In practice, this approach means that the properties of self and the properties of super get encoded into the same container: if self encodes values for keys "a", "b", and "c", and super encodes "d", "e", and "f", the resulting object is archived as {"a": ..., "b": ..., "c": ..., "d": ..., "e": ..., "f": ...}. This approach has two drawbacks:

  • Things which self encodes may overwrite super's (or vice versa, depending on when -[super encodeWithCoder:] is called
  • self and super may not encode into different container types (e.g. self in a sequential fashion, and super in a keyed fashion)
The second point is not an issue for NSKeyedArchiver, since all values encode with keys (sequentially coded elements get autogenerated keys). This proposed API, however, allows for self and super to explicitly request conflicting containers (.arrayand .dictionary, which may not be mixed, depending on the data format).

To remedy both of these points, we adopt a new convention for inheritance-based coding — encoding super as a sub-object of self:

public class MyCodable : SomethingCodable {

public func encode(to encoder: Encoder) throws {

let container = encoder.container(keyedBy: CodingKeys.self)

// ... encode some properties

// superEncoder() gives `super` a nested container to encode into (for

// a predefined key).

try super.encode(to: container.superEncoder())

}

public init(from decoder: Decoder) throws {

let container = try decoder.container(keyedBy: CodingKeys.self)

// ... decode some properties

// Allow `super` to decode from the nested container.

try super.init(from: container.superDecoder())

}
If a shared container is desired, it is still possible to call super.encode(to: encoder) and super.init(from: decoder), but we recommend the safer containerized option.

superEncoder() and superDecoder() are provided on KeyedEncodingContainer and KeyedDecodingContainer to provide handles to containers for super to use. While users may specify a custom key to encode super with, the default behavior is to use a key with a stringValue of "super" and an intValue of 0:

// Continuing from before

open
class KeyedEncodingContainer<Key : CodingKey> {

/// Stores a new nested container for the default `super` key and returns a new `Encoder` instance for encoding `super` into that container.

///

/// Equivalent to calling `superEncoder(forKey:)` with `Key(stringValue: "super", intValue: 0)`.

///

/// - returns: A new `Encoder` to pass to `super.encode(to:)`.

    open
func superEncoder() -> Encoder

/// Stores a new nested container for the given key and returns a new `Encoder` instance for encoding `super` into that container.

///

/// - parameter key: The key to encode `super` for.

/// - returns: A new `Encoder` to pass to `super.encode(to:)`.

/// - precondition: The key must have a `stringValue` or `intValue` appropriate for the encoding container type.

    open
func superEncoder(forKey key: Key) -> Encoder
}

open
class KeyedDecodingContainer<Key : CodingKey> {

/// Returns a `Decoder` instance for decoding `super` from the container associated with the default `super` key.

///

/// Equivalent to calling `superDecoder(forKey:)` with `Key(stringValue: "super", intValue: 0)`.

///

/// - returns: A new `Decoder` to pass to `super.init(from:)`.

/// - throws: `CocoaError.coderValueNotFound` if `self` does not have an entry for the default `super` key, or if the stored value is null.

    open
func superDecoder() throws -> Decoder

/// Returns a `Decoder` instance for decoding `super` from the container associated with the given key.

///

/// - parameter key: The key to decode `super` for.

/// - returns: A new `Decoder` to pass to `super.init(from:)`.

/// - throws: `CocoaError.coderValueNotFound` if `self` does not have an entry for the given key, or if the stored value is null.

    open
func superDecoder(forKey key: Key) throws -> Decoder
}
Primitive Codable Conformance

The encoding container types offer overloads for working with and processing the API's primitive types (String, Int, Double, etc.). However, for ease of implementation (both in this API and others), it can be helpful for these types to conform to Codable themselves. Thus, along with these overloads, we will offer Codableconformance on these types:

extension Bool : Codable {

public init(from decoder: Decoder) throws {

self = try decoder.singleValueContainer().decode(Bool.self)

}

public func encode(to encoder: Encoder) throws {

try encoder.singleValueContainer().encode( self)

}

// Repeat for others...
This conformance allows one to write functions which accept Codable types without needing specific overloads for the fifteen primitive types as well. This also simplifies conditional conformance (e.g. expressing "extension Array : Codable where Element : Codable") by removing the need for additional explicit conformances for these types.

Since Swift's function overload rules prefer more specific functions over generic functions, the specific overloads are chosen where possible (e.g. encode("Hello, world!", forKey: .greeting) will choose encode(_: String, forKey: Key) over encode<T : Codable>(_: T, forKey: Key)). This maintains performance over dispatching through the Codable existential, while allowing for the flexibility of fewer overloads where applicable.

Additional Extensions

Along with the primitive Codable conformance above, extensions on CodableRawRepresentable types whose RawValue is a primitive types will provide default implementations for encoding and decoding:

public extension RawRepresentable where RawValue == Bool, Self : Codable {

public init(from decoder: Decoder) throws {

let decoded = try decoder.singleValueContainer().decode(RawValue.self)

guard let value = Self(rawValue: decoded) else {

throw CocoaError.error(.coderReadCorrupt)

}

self =
value

}

public func encode(to encoder: Encoder) throws {

try encoder.singleValueContainer().encode(self.rawValue)

}

// Repeat for others...
This allows for trivial Codable conformance of enum types (and manual RawRepresentable implementations) with primitive backing.

Source compatibility
This proposal is additive — existing code will not have to change due to this API addition. This implementation can be made available in both Swift 4 and the Swift 3 compatibility mode.

Effect on ABI stability
The addition of this API will not be an ABI-breaking change. However, this will add limitations for changes in future versions of Swift, as parts of the API will have to remain unchanged between versions of Swift (barring some additions, discussed below).

Effect on API resilience
Much like new API added to the standard library, once added, many changes to this API will be ABI- and source-breaking changes. In particular, changes which change the types or names of methods or arguments, add required methods on protocols or classes, or remove supplied default implementations will break client behavior.

The following protocols and classes may not have methods added to them without providing default implementations:

  • Codable
  • CodingKey
  • Encoder
  • SingleValueEncodingContainer
  • KeyedEncodingContainer
  • Decoder
  • SingleValueDecodingContainer
  • KeyedDecodingContainer
The following classes may not remove existing default implementations:

  • KeyedEncodingContainer
  • KeyedDecodingContainer
Various extensions to Swift primitive types (Bool, Int, Double, etc.) and to RawRepresentable types (where RawValue == Bool, == Int, == Double, etc.) may also not be removed.

In general, changes to the proposed types will be restricted as described in the library evolution document in the Swift repository.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Brent Royal-Gordon) #18

Re-reading this after sending it, I realized I should probably be a lot more concrete about what I envision here.

Basically, what I think is:

* There should be primitives for `encode(_ value: NSNull)` and `decode(_ type: NSNull.Type)`. Yes, even though it's a singleton. In JSON, this should emit/read a `null`; in plist format, I suppose it will simply have to interpret absence of the corresponding key as `null`, because there's nothing better we can do there.

* An `Optional.some(x)` should encode as an `x` would.

* An `Optional.none`, where `Wrapped` is not itself an `Optional`, should encode as an `NSNull`.

* An `Optional.none`, where `Wrapped` *is* itself an `Optional`, should encode as one of the Swift runtime's magic optional sentinel objects.

That will do the best job we can manage of correctly representing nested optionals.

···

On Mar 15, 2017, at 9:19 PM, Brent Royal-Gordon <brent@architechies.com> wrote:

I think we'd be better off having `encode(_:forKey:)` not take an optional; instead, we should have `Optional` conform to `Codable` and behave in some appropriate way. Exactly how to implement it might be a little tricky because of nested optionals; I suppose a `none` would have to measure how many levels of optionality there are between it and a concrete value, and then encode that information into the data. I think our `NSNull` bridging is doing something broadly similar right now.

I know that this is not the design you would use in Objective-C, but Swift uses `Optional` differently from how Objective-C uses `nil`. Swift APIs consider `nil` and absent to be different things; where they can both occur, good Swift APIs use doubled-up Optionals to be precise about the situation. I think the design needs to be a little different to accommodate that.

--
Brent Royal-Gordon
Architechies


(Goffredo Marocchi) #19

Congrats on getting this out! A question from the field:

https://twitter.com/mdiep/status/842178457115230210 Why does the Swift Serialization API proposal use abstract base classes?

Hopefully because we realise we need abstract classes as a feature ;)?

···

Sent from my iPhone

On 16 Mar 2017, at 01:18, Joe Groff via swift-evolution <swift-evolution@swift.org> wrote:

-Joe

On Mar 15, 2017, at 3:40 PM, Itai Ferber via swift-evolution <swift-evolution@swift.org> wrote:

Hi everyone,

The following introduces a new Swift-focused archival and serialization API as part of the Foundation framework. We’re interested in improving the experience and safety of performing archival and serialization, and are happy to receive community feedback on this work.
Because of the length of this proposal, the Appendix and Alternatives Considered sections have been omitted here, but are available in the full proposal on the swift-evolution repo. The full proposal also includes an Unabridged API for further consideration.

Without further ado, inlined below.

— Itai

Swift Archival & Serialization
   • Proposal: SE-NNNN
   • Author(s): Itai Ferber, Michael LeHew, Tony Parker
   • Review Manager: TBD
   • Status: Awaiting review
   • Associated PRs:
       • #8124
       • #8125
Introduction
Foundation's current archival and serialization APIs (NSCoding, NSJSONSerialization, NSPropertyListSerialization, etc.), while fitting for the dynamism of Objective-C, do not always map optimally into Swift. This document lays out the design of an updated API that improves the developer experience of performing archival and serialization in Swift.

Specifically:

   • It aims to provide a solution for the archival of Swift struct and enum types
   • It aims to provide a more type-safe solution for serializing to external formats, such as JSON and plist
Motivation
The primary motivation for this proposal is the inclusion of native Swift enum and struct types in archival and serialization. Currently, developers targeting Swift cannot participate in NSCoding without being willing to abandon enum and structtypes — NSCoding is an @objc protocol, conformance to which excludes non-class types. This is can be limiting in Swift because small enums and structs can be an idiomatic approach to model representation; developers who wish to perform archival have to either forgo the Swift niceties that constructs like enumsprovide, or provide an additional compatibility layer between their "real" types and their archivable types.

Secondarily, we would like to refine Foundation's existing serialization APIs (NSJSONSerialization and NSPropertyListSerialization) to better match Swift's strong type safety. From experience, we find that the conversion from the unstructured, untyped data of these formats into strongly-typed data structures is a good fit for archival mechanisms, rather than taking the less safe approach that 3rd-party JSON conversion approaches have taken (described further in an appendix below).

We would like to offer a solution to these problems without sacrificing ease of use or type safety.

Agenda
This proposal is the first stage of three that introduce different facets of a whole Swift archival and serialization API:

   • This proposal describes the basis for this API, focusing on the protocols that users adopt and interface with
   • The next stage will propose specific API for new encoders
   • The final stage will discuss how this new API will interop with NSCoding as it is today
SE-NNNN provides stages 2 and 3.

Proposed solution
We will be introducing the following new types:

   • protocol Codable: Adopted by types to opt into archival. Conformance may be automatically derived in cases where all properties are also Codable.
   • protocol CodingKey: Adopted by types used as keys for keyed containers, replacing String keys with semantic types. Conformance may be automatically derived in most cases.
   • protocol Encoder: Adopted by types which can take Codable values and encode them into a native format.
       • class KeyedEncodingContainer<Key : CodingKey>: Subclasses of this type provide a concrete way to store encoded values by CodingKey. Types adopting Encoder should provide subclasses of KeyedEncodingContainer to vend.
       • protocol SingleValueEncodingContainer: Adopted by types which provide a concrete way to store a single encoded value. Types adopting Encoder should provide types conforming to SingleValueEncodingContainer to vend (but in many cases will be able to conform to it themselves).
   • protocol Decoder: Adopted by types which can take payloads in a native format and decode Codable values out of them.
       • class KeyedDecodingContainer<Key : CodingKey>: Subclasses of this type provide a concrete way to retrieve encoded values from storage by CodingKey. Types adopting Decoder should provide subclasses of KeyedDecodingContainer to vend.
       • protocol SingleValueDecodingContainer: Adopted by types which provide a concrete way to retrieve a single encoded value from storage. Types adopting Decoder should provide types conforming to SingleValueDecodingContainer to vend (but in many cases will be able to conform to it themselves).
For end users of this API, adoption will primarily involve the Codable and CodingKey protocols. In order to participate in this new archival system, developers must add Codable conformance to their types:

// If all properties are Codable, implementation is automatically derived:
public struct Location : Codable {

public let latitude: Double

public let longitude: Double
}

public enum Animal : Int, Codable {

case chicken = 1

case
dog

case
turkey

case
cow

}

public struct Farm : Codable {

public let name: String

public let location: Location

public let animals: [Animal]
}
With developer participation, we will offer encoders and decoders (described in SE-NNNN, not here) that take advantage of this conformance to offer type-safe serialization of user models:

let farm = Farm(name: "Old MacDonald's Farm",

               location
: Location(latitude: 51.621648, longitude: 0.269273),

               animals
: [.chicken, .dog, .cow, .turkey, .dog, .chicken, .cow, .turkey, .dog])
let payload: Data = try JSONEncoder().encode(farm)

do {

let farm = try JSONDecoder().decode(Farm.self, from: payload)

// Extracted as user types:

let coordinates = "\(farm.location.latitude, farm.location.longitude)"
} catch {

// Encountered error during deserialization
}
This gives developers access to their data in a type-safe manner and a recognizable interface.

Detailed design
To support user types, we expose the Codable protocol:

/// Conformance to `Codable` indicates that a type can marshal itself into and out of an external representation.
public protocol Codable {

/// Initializes `self` by decoding from `decoder`.

///

/// - parameter decoder: The decoder to read data from.

/// - throws: An error if reading from the decoder fails, or if read data is corrupted or otherwise invalid.

init(from decoder: Decoder) throws

/// Encodes `self` into the given encoder.

///

/// If `self` fails to encode anything, `encoder` will encode an empty `.default` container in its place.

///

/// - parameter encoder: The encoder to write data to.

/// - throws: An error if any values are invalid for `encoder`'s format.

func encode(to encoder: Encoder) throws
}
By adopting Codable, user types opt in to this archival system.

Structured types (i.e. types which encode as a collection of properties) encode and decode their properties in a keyed manner. Keys may be String-convertible or Int-convertible (or both), and user types which have properties should declare semantic key enums which map keys to their properties. Keys must conform to the CodingKey protocol:

/// Conformance to `CodingKey` indicates that a type can be used as a key for encoding and decoding.
public protocol CodingKey {

/// The string to use in a named collection (e.g. a string-keyed dictionary).

var stringValue: String? { get }

/// Initializes `self` from a string.

///

/// - parameter stringValue: The string value of the desired key.

/// - returns: An instance of `Self` from the given string, or `nil` if the given string does not correspond to any instance of `Self`.

init?(stringValue: String)

/// The int to use in an indexed collection (e.g. an int-keyed dictionary).

var intValue: Int? { get }

/// Initializes `self` from an integer.

///

/// - parameter intValue: The integer value of the desired key.

/// - returns: An instance of `Self` from the given integer, or `nil` if the given integer does not correspond to any instance of `Self`.

init?(intValue: Int)
}
For most types, String-convertible keys are a reasonable default; for performance, however, Int-convertible keys are preferred, and Encoders may choose to make use of Ints over Strings. Framework types should provide keys which have both for flexibility and performance across different types of Encoders. It is generally an error to provide a key which has neither a stringValue nor an intValue.

By default, CodingKey conformance can be derived for enums which have either String or Int backing:

enum Keys1 : CodingKey {

case a // (stringValue: "a", intValue: nil)

case b // (stringValue: "b", intValue: nil)
}

enum Keys2 : String, CodingKey {

case c = "foo" // (stringValue: "foo", intValue: nil)

case d // (stringValue: "d", intValue: nil)
}

enum Keys3 : Int, CodingKey {

case e = 4 // (stringValue: "e", intValue: 4)

case f // (stringValue: "f", intValue: 5)

case g = 9 // (stringValue: "g", intValue: 9)
}
Coding keys which are not enums, have associated values, or have other raw representations must implement these methods manually.

In addition to automatic CodingKey conformance derivation for enums, Codableconformance can be automatically derived for certain types as well:

   • Types whose properties are all either Codable or primitive get an automatically derived String-backed CodingKeys enum mapping properties to case names
   • Types falling into (1) and types which provide a CodingKeys enum (directly or via a typealias) whose case names map to properties which are all Codableget automatic derivation of init(from:) and encode(to:) using those properties and keys. Types may choose to provide a custom init(from:) or encode(to:) (or both); whichever they do not provide will be automatically derived
   • Types which fall into neither (1) nor (2) will have to provide a custom key type and provide their own init(from:) and encode(to:)
Many types will either allow for automatic derivation of all codability (1), or provide a custom key subset and take advantage of automatic method derivation (2).

Encoding and Decoding

Types which are encodable encode their data into a container provided by their Encoder:

/// An `Encoder` is a type which can encode values into a native format for external representation.
public protocol Encoder {

/// Populates `self` with an encoding container (of `.default` type) and returns it, keyed by the given key type.

///

/// - parameter type: The key type to use for the container.

/// - returns: A new keyed encoding container.

/// - precondition: May not be called after a previous `self.container(keyedBy:)` call of a different `EncodingContainerType`.

/// - precondition: May not be called after a value has been encoded through a prior `self.singleValueContainer()` call.

func container<Key : CodingKey>(keyedBy type: Key.Type) -> KeyedEncodingContainer<Key>

/// Returns an encoding container appropriate for holding a single primitive value.

///

/// - returns: A new empty single value container.

/// - precondition: May not be called after a prior `self.container(keyedBy:)` call.

/// - precondition: May not be called after a value has been encoded through a previous `self.singleValueContainer()` call.

func singleValueContainer() -> SingleValueEncodingContainer

/// The path of coding keys taken to get to this point in encoding.

var codingKeyContext: [CodingKey] { get }
}

// Continuing examples from before; below is automatically generated by the compiler if no customization is needed.
public struct Location : Codable {

private enum CodingKeys : CodingKey {

case
latitutude

case
longitude

}

public func encode(to encoder: Encoder) throws {

// Generic keyed encoder gives type-safe key access: cannot encode with keys of the wrong type.

let container = encoder.container(keyedBy: CodingKeys.self)

// The encoder is generic on the key -- free key autocompletion here.

try container.encode(latitude, forKey: .latitude)

try container.encode(longitude, forKey: .longitude)

}
}

public struct Farm : Codable {

private enum CodingKeys : CodingKey {

case
name

case
location

case
animals

}

public func encode(to encoder: Encoder) throws {

let container = encoder.container(keyedBy: CodingKeys.self)

try container.encode(name, forKey: .name)

try container.encode(location, forKey: .location)

try container.encode(animals, forKey: .animals)

}
}
Similarly, decodable types initialize from data read from their Decoder's container:

/// A `Decoder` is a type which can decode values from a native format into in-memory representations.
public protocol Decoder {

/// Returns the data stored in `self` as represented in a container keyed by the given key type.

///

/// - parameter type: The key type to use for the container.

/// - returns: A keyed decoding container view into `self`.

/// - throws: `CocoaError.coderTypeMismatch` if the encountered stored value is not a keyed container.

func container<Key : CodingKey>(keyedBy type: Key.Type) throws -> KeyedDecodingContainer<Key>

/// Returns the data stored in `self` as represented in a container appropriate for holding a single primitive value.

///

/// - returns: A single value container view into `self`.

/// - throws: `CocoaError.coderTypeMismatch` if the encountered stored value is not a single value container.

func singleValueContainer() throws -> SingleValueDecodingContainer

/// The path of coding keys taken to get to this point in decoding.

var codingKeyContext: [CodingKey] { get }
}

// Continuing examples from before; below is automatically generated by the compiler if no customization is needed.
public struct Location : Codable {

public init(from decoder: Decoder) throws {

let container = try decoder.container(keyedBy: CodingKeys.self)

       latitude
= try container.decode(Double.self, forKey: .latitude)

       longitude
= try container.decode(Double.self, forKey: .longitude)

}
}

public struct Farm : Codable {

public init(from decoder: Decoder) throws {

let container = try decoder.container(keyedBy: CodingKeys.self)

       name
= try container.decode(String.self, forKey: .name)

       location
= try container.decode(Location.self, forKey: .location)

       animals
= try container.decode([Animal].self, forKey: .animals)

}
}
Keyed Encoding Containers

Keyed encoding containers are the primary interface that most Codable types interact with for encoding and decoding. Through these, Codable types have strongly-keyed access to encoded data by using keys that are semantically correct for the operations they want to express.

Since semantically incompatible keys will rarely (if ever) share the same key type, it is impossible to mix up key types within the same container (as is possible with Stringkeys), and since the type is known statically, keys get autocompletion by the compiler.

/// `KeyedEncodingContainer` is a generic abstract base class that provides a view into an `Encoders` storage and is used to hold the encoded properties of a `Codable` type.
///
/// Encoders should provide subclasses of `KeyedEncodingContainer` for their format.

open
class KeyedEncodingContainer<Key : CodingKey> {

/// Encodes the given value for the given key.

///

/// - parameter value: The value to encode.

/// - parameter key: The key to associate the value with.

/// - throws: `CocoaError.coderInvalidValue` if the given value is invalid in the current context for this format.

/// - precondition: The key must have a `stringValue` or `intValue` appropriate for the encoding container type.

   open
func encode<Value : Codable>(_ value: Value?, forKey key: Key) throws

/// Encodes the given value for the given key.

///

/// - parameter value: The value to encode.

/// - parameter key: The key to associate the value with.

/// - throws: `CocoaError.coderInvalidValue` if the given value is invalid in the current context for this format.

/// - precondition: The key must have a `stringValue` or `intValue` appropriate for the encoding container type.

   open
func encode(_ value: Bool?, forKey key: Key) throws

   open
func encode(_ value: Int?, forKey key: Key) throws

   open
func encode(_ value: Int8?, forKey key: Key) throws

   open
func encode(_ value: Int16?, forKey key: Key) throws

   open
func encode(_ value: Int32?, forKey key: Key) throws

   open
func encode(_ value: Int64?, forKey key: Key) throws

   open
func encode(_ value: UInt?, forKey key: Key) throws

   open
func encode(_ value: UInt8?, forKey key: Key) throws

   open
func encode(_ value: UInt16?, forKey key: Key) throws

   open
func encode(_ value: UInt32?, forKey key: Key) throws

   open
func encode(_ value: UInt64?, forKey key: Key) throws

   open
func encode(_ value: Float?, forKey key: Key) throws

   open
func encode(_ value: Double?, forKey key: Key) throws

   open
func encode(_ value: String?, forKey key: Key) throws

   open
func encode(_ value: Data?, forKey key: Key) throws

/// Encodes the given object weakly for the given key.

///

/// For `Encoder`s that implement this functionality, this will only encode the given object and associate it with the given key if it encoded unconditionally elsewhere in the archive (either previously or in the future).

///

/// For formats which don't support this feature, the default implementation encodes the given object unconditionally.

///

/// - parameter object: The object to encode.

/// - parameter key: The key to associate the object with.

/// - throws: `CocoaError.coderInvalidValue` if the given value is invalid in the current context for this format.

/// - precondition: The key must have a `stringValue` or `intValue` appropriate for the encoding container type.

   open
func encodeWeak<Object : AnyObject & Codable>(_ object: Object?, forKey key: Key) throws

/// The path of coding keys taken to get to this point in encoding.

   open
var codingKeyContext: [CodingKey]
}

/// `KeyedDecodingContainer` is a generic abstract base class that provides a view into an `Decoders` storage and is used to hold the encoded properties of a `Codable` type.
///
/// Decoders should provide subclasses of `KeyedDecodingContainer` for their format.

open
class KeyedDecodingContainer<Key : CodingKey> {

/// All the keys the `Decoder` has for this container.

///

/// Different keyed containers from the same `Decoder` may return different keys here; it is possible to encode with multiple key types which are not convertible to one another. This should report all keys present which are convertible to the requested type.

   open
var allKeys: [Key]

/// Returns whether the `Decoder` contains a value associated with the given key.

///

/// The value associated with the given key may be a null value as appropriate for the data format.

///

/// - parameter key: The key to search for.

/// - returns: Whether the `Decoder` has an entry for the given key.

   open
func contains(_ key: Key) -> Bool

/// Decodes a value of the given type for the given key.

///

/// A default implementation is given for these types which calls into the abstract `decodeIfPresent` implementations below.

///

/// - parameter type: The type of value to decode.

/// - parameter key: The key that the decoded value is associated with.

/// - returns: A value of the requested type, if present for the given key and convertible to the requested type.

/// - throws: `CocoaError.coderTypeMismatch` if the encountered encoded value is not convertible to the requested type.

/// - throws: `CocoaError.coderValueNotFound` if `self` does not have an entry for the given key or if the value is null.

   open
func decode(_ type: Bool.Type, forKey key: Key) throws -> Bool

   open
func decode(_ type: Int.Type, forKey key: Key) throws -> Int

   open
func decode(_ type: Int8.Type, forKey key: Key) throws -> Int8

   open
func decode(_ type: Int16.Type, forKey key: Key) throws -> Int16

   open
func decode(_ type: Int32.Type, forKey key: Key) throws -> Int32

   open
func decode(_ type: Int64.Type, forKey key: Key) throws -> Int64

   open
func decode(_ type: UInt.Type, forKey key: Key) throws -> UInt

   open
func decode(_ type: UInt8.Type, forKey key: Key) throws -> UInt8

   open
func decode(_ type: UInt16.Type, forKey key: Key) throws -> UInt16

   open
func decode(_ type: UInt32.Type, forKey key: Key) throws -> UInt32

   open
func decode(_ type: UInt64.Type, forKey key: Key) throws -> UInt64

   open
func decode(_ type: Float.Type, forKey key: Key) throws -> Float

   open
func decode(_ type: Double.Type, forKey key: Key) throws -> Double

   open
func decode(_ type: String.Type, forKey key: Key) throws -> String

   open
func decode(_ type: Data.Type, forKey key: Key) throws -> Data

   open
func decode<Value : Codable>(_ type: Value.Type, forKey key: Key) throws -> Value

/// Decodes a value of the given type for the given key, if present.

///

/// This method returns `nil` if the container does not have a value associated with `key`, or if the value is null. The difference between these states can be disambiguated with a `contains(_:)` call.

///

/// - parameter type: The type of value to decode.

/// - parameter key: The key that the decoded value is associated with.

/// - returns: A decoded value of the requested type, or `nil` if the `Decoder` does not have an entry associated with the given key, or if the value is a null value.

/// - throws: `CocoaError.coderTypeMismatch` if the encountered encoded value is not convertible to the requested type.

   open
func decodeIfPresent(_ type: Bool.Type, forKey key: Key) throws -> Bool?

   open
func decodeIfPresent(_ type: Int.Type, forKey key: Key) throws -> Int?

   open
func decodeIfPresent(_ type: Int8.Type, forKey key: Key) throws -> Int8?

   open
func decodeIfPresent(_ type: Int16.Type, forKey key: Key) throws -> Int16?

   open
func decodeIfPresent(_ type: Int32.Type, forKey key: Key) throws -> Int32?

   open
func decodeIfPresent(_ type: Int64.Type, forKey key: Key) throws -> Int64?

   open
func decodeIfPresent(_ type: UInt.Type, forKey key: Key) throws -> UInt?

   open
func decodeIfPresent(_ type: UInt8.Type, forKey key: Key) throws -> UInt8?

   open
func decodeIfPresent(_ type: UInt16.Type, forKey key: Key) throws -> UInt16?

   open
func decodeIfPresent(_ type: UInt32.Type, forKey key: Key) throws -> UInt32?

   open
func decodeIfPresent(_ type: UInt64.Type, forKey key: Key) throws -> UInt64?

   open
func decodeIfPresent(_ type: Float.Type, forKey key: Key) throws -> Float?

   open
func decodeIfPresent(_ type: Double.Type, forKey key: Key) throws -> Double?

   open
func decodeIfPresent(_ type: String.Type, forKey key: Key) throws -> String?

   open
func decodeIfPresent(_ type: Data.Type, forKey key: Key) throws -> Data?

   open
func decodeIfPresent<Value : Codable>(_ type: Value.Type, forKey key: Key) throws -> Value?

/// The path of coding keys taken to get to this point in decoding.

   open
var codingKeyContext: [CodingKey]
}
These encode(_:forKey:) and decode(_:forKey:) overloads give strong, static type guarantees about what is encodable (preventing accidental attempts to encode an invalid type), and provide a list of primitive types which are common to all encoders and decoders that users can rely on.

Coming in Swift 4 is the ability to express that "a collection of things which are Codable is Codable" (conditional conformance), allowing collections which we extend (Array, Dictionary, etc.) to fall into these overloads as well.

Encoding Container Types

For some types, the container into which they encode has meaning. Especially when coding for a specific output format (e.g. when communicating with a JSON API), a type may wish to explicitly encode as an array or a dictionary:

// Continuing from before
public protocol Encoder {

/// Populates `self` with an encoding container of the given type and returns it, keyed by the given key type.

///

/// A default implementation of `Encoder.container(keyedBy:)` calls this method with a container type of `.default`.

///

/// - parameter keyType: The key type to use for the container.

/// - parameter containerType: The container type to create.

/// - returns: A new keyed encoding container.

/// - precondition: May not be called after a previous `self.container(keyedBy:)` call of a different `EncodingContainerType`.

/// - precondition: May not be called after a value has been encoded through a prior `self.singleValueContainer()` call.

func container<Key : CodingKey>(keyedBy keyType: Key.Type, type containerType: EncodingContainerType) -> KeyedEncodingContainer<Key>
}

/// An `EncodingContainerType` specifies the type of container an `Encoder` should use to store values.
public enum EncodingContainerType {

/// The `Encoder`'s preferred container type; equivalent to either `.array` or `.dictionary` as appropriate for the encoder.

case `default
`

/// Explicitly requests the use of an array to store encoded values.

case
array

/// Explicitly requests the use of a dictionary to store encoded values.

case
dictionary

}
Single Value Containers

For other types, an array or dictionary container may not even make sense (e.g. values which are RawRepresentable as a single primitive value). Those types may encode and decode directly as a single value, instead of requesting a keyed container:

/// A `SingleValueEncodingContainer` is a container which can support the storage and direct encoding of a single non-keyed value.
public protocol SingleValueEncodingContainer {

/// Encodes a single value of the given type.

///

/// - parameter value: The value to encode.

/// - throws: `CocoaError.coderInvalidValue` if the given value is invalid in the current context for this format.

/// - precondition: May not be called after a previous `self.encode(_:)` call.

func encode(_ value: Bool) throws

func encode(_ value: Int) throws

func encode(_ value: Int8) throws

func encode(_ value: Int16) throws

func encode(_ value: Int32) throws

func encode(_ value: Int64) throws

func encode(_ value: UInt) throws

func encode(_ value: UInt8) throws

func encode(_ value: UInt16) throws

func encode(_ value: UInt32) throws

func encode(_ value: UInt64) throws

func encode(_ value: Float) throws

func encode(_ value: Double) throws

func encode(_ value: String) throws

func encode(_ value: Data) throws
}

/// A `SingleValueDecodingContainer` is a container which can support the storage and direct decoding of a single non-keyed value.
public protocol SingleValueDecodingContainer {

/// Decodes a single value of the given type.

///

/// - parameter type: The type to decode as.

/// - returns: A value of the requested type.

/// - throws: `CocoaError.coderTypeMismatch` if the encountered encoded value cannot be converted to the requested type.

func decode(_ type: Bool.Type) throws -> Bool

func decode(_ type: Int.Type) throws -> Int

func decode(_ type: Int8.Type) throws -> Int8

func decode(_ type: Int16.Type) throws -> Int16

func decode(_ type: Int32.Type) throws -> Int32

func decode(_ type: Int64.Type) throws -> Int64

func decode(_ type: UInt.Type) throws -> UInt

func decode(_ type: UInt8.Type) throws -> UInt8

func decode(_ type: UInt16.Type) throws -> UInt16

func decode(_ type: UInt32.Type) throws -> UInt32

func decode(_ type: UInt64.Type) throws -> UInt64

func decode(_ type: Float.Type) throws -> Float

func decode(_ type: Double.Type) throws -> Double

func decode(_ type: String.Type) throws -> String

func decode(_ type: Data.Type) throws -> Data
}

// Continuing example from before; below is automatically generated by the compiler if no customization is needed.
public enum Animal : Int, Codable {

public func encode(to encoder: Encoder) throws {

// Encode as a single value; no keys.

try encoder.singleValueContainer.encode(self.rawValue)

}

public init(from decoder: Decoder) throws {

// Decodes as a single value; no keys.

let intValue = try decoder.singleValueContainer().decode(Int.self)

if let value = Self(rawValue: intValue) {

self =
value

} else {

throw CocoaError.error(.coderReadCorrupt)

}

}
}
In the example given above, since Animal uses a single value container, [.chicken, .dog, .cow, .turkey, .dog, .chicken, .cow, .turkey, .dog]would encode directly as [1, 2, 4, 3, 2, 1, 4, 3, 2].

Nesting

In practice, some types may also need to control how data is nested within their container, or potentially nest other containers within their container. Keyed containers allow this by returning nested containers of differing key types:

// Continuing from before

open
class KeyedEncodingContainer<Key : CodingKey> {

/// Stores an encoding container for the given key and returns it.

///

/// - parameter keyType: The key type to use for the container.

/// - parameter containerType: The container type to create.

/// - parameter key: The key to encode the container for.

/// - returns: A new keyed encoding container.

   open
func nestedContainer<NestedKey : CodingKey>(keyedBy keyType: NestedKey.Type, type containerType: EncodingContainerType, forKey key: Key) -> KeyedEncodingContainer<NestedKey>
}

open
class KeyedDecodingContainer<Key : CodingKey> {

/// Returns the data stored for the given key as represented in a container keyed by the given key type.

///

/// - parameter type: The key type to use for the container.

/// - parameter key: The key that the nested container is associated with.

/// - returns: A keyed decoding container view into `self`.

/// - throws: `CocoaError.coderTypeMismatch` if the encountered stored value is not a container.

   open
func nestedContainer<NestedKey : CodingKey>(keyedBy type: NestedKey.Type, forKey key: Key) throws -> KeyedDecodingContainer<NestedKey>
}
This can be common when coding against specific external data representations:

// User type for interfacing with a specific JSON API. JSON API expects encoding as {"id": ..., "properties": {"name": ..., "timestamp": ...}}. Swift type differs from encoded type, and encoding needs to match a spec:
struct Record : Codable {

// We care only about these values from the JSON payload

let id: Int

let name: String

let timestamp: Double

// ...

private enum Keys : CodingKey {

case
id

case
properties

}

private enum PropertiesKeys : CodingKey {

case
name

case
timestamp

}

public func encode(to encoder: Encoder) throws {

let container = encoder.container(keyedBy: Keys.self, type: .dictionary)

try container.encode(id, forKey: .id)

// Set a dictionary for the "properties" key

let nested = container.nestedContainer(keyedBy: PropertiesKeys.self, type: .dictionary, forKey: .properties)

try nested.encode(name, forKey: .name)

try nested.encode(timestamp, forKey: .timestamp)

}

public init(from decoder: Decoder) throws {

let container = try decoder.container(keyedBy: Keys.self)

       id
= try container.decode(Int.self, forKey: .id)

let nested = try container.nestedContainer(keyedBy: PropertiesKeys.self, forKey: .properties)

       name
= try nested.decode(String.self, forKey: .name)

       timestamp
= try nested.decode(Double.self, forKey: .timestamp)

}
}
Inheritance

Inheritance in this system is supported much like it is with NSCoding — on encoding, objects which inherit from a type that is Codable encode super using their encoder, and pass a decoder to super.init(from:) on decode. With the existing NSCoding API, this is most often done like so, by convention:

- (void)encodeWithCoder:(NSCoder *)encoder {

[super encodeWithCoder:encoder];

// ... encode properties
}

- (instancetype)initWithCoder:(NSCoder *)decoder {

if ((self = [super initWithCoder:decoder])) {

// ... decode properties

}

return self;
}
In practice, this approach means that the properties of self and the properties of super get encoded into the same container: if self encodes values for keys "a", "b", and "c", and super encodes "d", "e", and "f", the resulting object is archived as {"a": ..., "b": ..., "c": ..., "d": ..., "e": ..., "f": ...}. This approach has two drawbacks:

   • Things which self encodes may overwrite super's (or vice versa, depending on when -[super encodeWithCoder:] is called
   • self and super may not encode into different container types (e.g. self in a sequential fashion, and super in a keyed fashion)
The second point is not an issue for NSKeyedArchiver, since all values encode with keys (sequentially coded elements get autogenerated keys). This proposed API, however, allows for self and super to explicitly request conflicting containers (.arrayand .dictionary, which may not be mixed, depending on the data format).

To remedy both of these points, we adopt a new convention for inheritance-based coding — encoding super as a sub-object of self:

public class MyCodable : SomethingCodable {

public func encode(to encoder: Encoder) throws {

let container = encoder.container(keyedBy: CodingKeys.self)

// ... encode some properties

// superEncoder() gives `super` a nested container to encode into (for

// a predefined key).

try super.encode(to: container.superEncoder())

}

public init(from decoder: Decoder) throws {

let container = try decoder.container(keyedBy: CodingKeys.self)

// ... decode some properties

// Allow `super` to decode from the nested container.

try super.init(from: container.superDecoder())

}
}
If a shared container is desired, it is still possible to call super.encode(to: encoder) and super.init(from: decoder), but we recommend the safer containerized option.

superEncoder() and superDecoder() are provided on KeyedEncodingContainer and KeyedDecodingContainer to provide handles to containers for super to use. While users may specify a custom key to encode super with, the default behavior is to use a key with a stringValue of "super" and an intValue of 0:

// Continuing from before

open
class KeyedEncodingContainer<Key : CodingKey> {

/// Stores a new nested container for the default `super` key and returns a new `Encoder` instance for encoding `super` into that container.

///

/// Equivalent to calling `superEncoder(forKey:)` with `Key(stringValue: "super", intValue: 0)`.

///

/// - returns: A new `Encoder` to pass to `super.encode(to:)`.

   open
func superEncoder() -> Encoder

/// Stores a new nested container for the given key and returns a new `Encoder` instance for encoding `super` into that container.

///

/// - parameter key: The key to encode `super` for.

/// - returns: A new `Encoder` to pass to `super.encode(to:)`.

/// - precondition: The key must have a `stringValue` or `intValue` appropriate for the encoding container type.

   open
func superEncoder(forKey key: Key) -> Encoder
}

open
class KeyedDecodingContainer<Key : CodingKey> {

/// Returns a `Decoder` instance for decoding `super` from the container associated with the default `super` key.

///

/// Equivalent to calling `superDecoder(forKey:)` with `Key(stringValue: "super", intValue: 0)`.

///

/// - returns: A new `Decoder` to pass to `super.init(from:)`.

/// - throws: `CocoaError.coderValueNotFound` if `self` does not have an entry for the default `super` key, or if the stored value is null.

   open
func superDecoder() throws -> Decoder

/// Returns a `Decoder` instance for decoding `super` from the container associated with the given key.

///

/// - parameter key: The key to decode `super` for.

/// - returns: A new `Decoder` to pass to `super.init(from:)`.

/// - throws: `CocoaError.coderValueNotFound` if `self` does not have an entry for the given key, or if the stored value is null.

   open
func superDecoder(forKey key: Key) throws -> Decoder
}
Primitive Codable Conformance

The encoding container types offer overloads for working with and processing the API's primitive types (String, Int, Double, etc.). However, for ease of implementation (both in this API and others), it can be helpful for these types to conform to Codable themselves. Thus, along with these overloads, we will offer Codableconformance on these types:

extension Bool : Codable {

public init(from decoder: Decoder) throws {

self = try decoder.singleValueContainer().decode(Bool.self)

}

public func encode(to encoder: Encoder) throws {

try encoder.singleValueContainer().encode( self)

}
}

// Repeat for others...
This conformance allows one to write functions which accept Codable types without needing specific overloads for the fifteen primitive types as well. This also simplifies conditional conformance (e.g. expressing "extension Array : Codable where Element : Codable") by removing the need for additional explicit conformances for these types.

Since Swift's function overload rules prefer more specific functions over generic functions, the specific overloads are chosen where possible (e.g. encode("Hello, world!", forKey: .greeting) will choose encode(_: String, forKey: Key) over encode<T : Codable>(_: T, forKey: Key)). This maintains performance over dispatching through the Codable existential, while allowing for the flexibility of fewer overloads where applicable.

Additional Extensions

Along with the primitive Codable conformance above, extensions on CodableRawRepresentable types whose RawValue is a primitive types will provide default implementations for encoding and decoding:

public extension RawRepresentable where RawValue == Bool, Self : Codable {

public init(from decoder: Decoder) throws {

let decoded = try decoder.singleValueContainer().decode(RawValue.self)

guard let value = Self(rawValue: decoded) else {

throw CocoaError.error(.coderReadCorrupt)

}

self =
value

}

public func encode(to encoder: Encoder) throws {

try encoder.singleValueContainer().encode(self.rawValue)

}
}

// Repeat for others...
This allows for trivial Codable conformance of enum types (and manual RawRepresentable implementations) with primitive backing.

Source compatibility
This proposal is additive — existing code will not have to change due to this API addition. This implementation can be made available in both Swift 4 and the Swift 3 compatibility mode.

Effect on ABI stability
The addition of this API will not be an ABI-breaking change. However, this will add limitations for changes in future versions of Swift, as parts of the API will have to remain unchanged between versions of Swift (barring some additions, discussed below).

Effect on API resilience
Much like new API added to the standard library, once added, many changes to this API will be ABI- and source-breaking changes. In particular, changes which change the types or names of methods or arguments, add required methods on protocols or classes, or remove supplied default implementations will break client behavior.

The following protocols and classes may not have methods added to them without providing default implementations:

   • Codable
   • CodingKey
   • Encoder
   • SingleValueEncodingContainer
   • KeyedEncodingContainer
   • Decoder
   • SingleValueDecodingContainer
   • KeyedDecodingContainer
The following classes may not remove existing default implementations:

   • KeyedEncodingContainer
   • KeyedDecodingContainer
Various extensions to Swift primitive types (Bool, Int, Double, etc.) and to RawRepresentable types (where RawValue == Bool, == Int, == Double, etc.) may also not be removed.

In general, changes to the proposed types will be restricted as described in the library evolution document in the Swift repository.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Vincent Esche) #20

+1 on this (having skimmed through it briefly)!

···

On Thu, Mar 16, 2017 at 1:45 AM, Zach Waldowski via swift-evolution < swift-evolution@swift.org> wrote:

Holy cow. There's much to digest here (so much so that my initial
response, which quoted its content, was denied by the mailing list). After
an initial reading, I don't just want this now, I want it yesterday. I'm
already imaging the encoder/decoders I want to build. Very exciting.