JSONDecoder and JSONEncode appear out of sync when it comes to stored properties with type T?? . The encodings always reflect the value of the inner Optional. During decoding, the value of the inner Optional is set only if the inner value not .null. Instead, the outer Optional is set to nil.
This is a problem when using decoding to make new instances of structs using a mirror and new values. I.e., it is sometimes desirable to be able to set the value of a property of type String?? to Optional(nil) using a decoder (when you can't set up an init(from:Decoder) for the type of the struct).
Example
import Foundation
struct Bar: Codable {
var optOptString1: String?? = .some(.none)
var optOptString2: String?? = nil
}
Bar() encoded: {"optOptString1":null}
-------------------------------------
Bar() : Bar(optOptString1: Optional(nil), optOptString2: nil)
Bar decoded from "{"optOptString1":null}" : Bar(optOptString1: nil, optOptString2: nil)
Bar decoded from "{}" : Bar(optOptString1: nil, optOptString2: nil)
Code
@main
struct Main {
static func main() async throws {
let bar1 = Bar()
let data1 = try JSONEncoder().encode(bar1)
let codedBar1 = String(data: data1, encoding: .utf8)!
let bar2 = try JSONDecoder().decode(Bar.self, from: data1)
let data3 = "{}".data(using: .utf8)!
let bar3 = try JSONDecoder().decode(Bar.self, from: data3)
print("Bar() encoded: \(codedBar1)")
print("-------------------------------------")
print("Bar() : \(bar1)")
print("Bar decoded from \"\(codedBar1)\" : \(bar2)")
print("Bar decoded from \"{}\" : \(bar3)")
}
}
Discussion
In the example, the instance of JSONDecoder
decodes {"optOptString1":"foo"} by setting optOptString1 to Optional(Optional("foo"))
decodes {"optOptString1":null} by setting optString1 to nil
decodes {} by setting opOptString1 to nil
I.e., JSONDecoder treats a {"optOptString1":null} the same as {}.
Is this the intended behavior? Has it always been this way? Is there a way to opt out of this?
Assuming one cannot not opt out, is there a way to decode a String to an instance of String?? with value .some(.none) when decoding a struct - without using a custom .init(from: Decoder).
Thank you for answering my question and for the suggestions.
I agree that JSONDecoder should be able to "correctly" (IMHO) decode into a stored property with type Optional<T?>, perhaps, as you suggest, through an option.
I think that In my case the suggestions would not work (due to factors I did not mention).
My code is in a default method of a protocol that a struct, say S, can conform to if it has init().
All of S's stored properties must conform to certain types. (Checked at runtime using a mirror.)
I need an instance of S, initialized form data passed in from an external source
let instance = JSONDecoder().decode(Self.self, from: data)
I can make data from the external data using generics and a default JSONEncoder (no need for JSONSerialization or a custom parser)
I have all this working except for the property with type Optional<T?> glitch.
I have never written an init(from:)but it appears that it would be impossible to write it as a default initializer for Self in a protocol - the code would need to now too much about the conforming struct.
I think I am stuck with the odd behavior that we are discussing.
Granted this is a corner case, but I think it shows that this odd decoding behavior can be harmful.
I'm pretty sure it has always been this way. I'd say it's a result of the decision that Optional encodes itself "flat", i.e. there's generally no difference in the encoded data between .some(x) and x. This also means that information about nested Optionals gets lost during encoding.
In theory, specific encoders could override this behavior and preserve nested Optionals, but that requires extra work by the encoder. JSONEncoder doesn’t do this, but it would be a good idea at least for encoders that serialize to formats with a native Optional data type.
extension Optional: Encodable where Wrapped: Encodable {
public func encode(to encoder: any Encoder) throws {
var container = encoder.singleValueContainer()
switch self {
case .none: try container.encodeNil()
case .some(let wrapped): try container.encode(wrapped)
}
}
}
Observations:
Optional encodes itself as nil in the .none case.
In the .some case (including .some(.none) for a nested Optional) it passes its wrapped value to the encoding container’s generic encode(_:) method.
Note that Optional doesn’t create any form of wrapper/container that says "this is an Optional". From the container's perspective, encoding .some(x) and x look identical.
Control now passes to the concrete encoding container, which is being provided by the specific encoder, giving the encoder the chance to customize how it wants to encode certain types.
JSONEncoder’s single value container eventually ends up calling the __JSONEncoder.wrapGeneric method. This method includes special cases for some types, such as Data, Date, and URL, but it has no special handling for Optionals.
When no special handling branch matches, the method’s default branch calls try value.encode(to: self). For a nested Optional, this brings us back to the start: Optional.encode is now being called again, but with the outermost Optional layer removed. So now the outer Optional layer has vanished without leaving a trace in the encoded representation.
encodeIfPresent won't encode a key at all if the given value is nil ({}) whereas encode will ({ "key": null }) (encodeIfPresent unwraps one layer of optionality, encode defers to Optional itself)
decodeIfPresent will return nil if the key doesn't exist ({}) whereas decode will throw an error (decodeIfPresent similarly "unwraps" a layer of optionality)
For Optional properties in a type which gets a synthesized Encodable/Decodable implementation, the compiler uses encodeIfPresent/decodeIfPresent calls, notencode/decode calls, so the behavior you see is at least partially dependent on that (e.g., it may not encode a key you expect to be encoded, and may be more lenient about a key not being present when you expect it to be strict).
For testing and understanding, it may be best to write out the conformance explicitly so you know what you're testing.