Weak -1 in its current form. I'm glad to see that this is being worked on, but this is not the right approach, in my opinion.
Yes, absolutely.
I hesitate to say so. As was said in this thread, there are multiple ways for enums with associated values to be decoded. Without providing a way to customize it, the proposal locks us in a specific format that I'm afraid will stick with no way to evolve towards increased flexibility.
I feel that the Codable protocols in their current form are very JSON-centric. It's also supported by the fact that the proposal uses JSON exclusively in its examples. I would like it to at least provide different .plist examples, and preferrably examples for coders for other formats too (YAML, XML etc). The proposal then should evaluate how well the suggested encoding would work with such formats. An alternative proposal could tackle the issue holistically, reworking Codable to be more format-agnostic, with improvements to its flexibility and performance.
I'm a maintainer of XMLCoder, which provides its own set of customizations on top of Codable to make it work for enums with associated values. Given that the proposal is JSON-focused, it worries me that issues that arise for implementors of coders for other formats are overlooked here.
I've read the full proposal and read through the discussion thread.
I think @michelf gave a pretty good example above.
Another example is a payload where one case is valid and the other case is valid for the format but not a valid case for the enum:
{
"load": {
"key": "MyKey"
},
"store": 1789.0
}
If a server validates this data, stores it, then reads it back from storage, the server could hit the valid case during validation but the invalid case when reading back from storage. And if the server assumes that the data is valid when reading back from storage, it might even trap on decoding errors, leading to non-deterministic crashes. Besides being hard to debug, that sounds like a vector for a denial-of-service attack.
The real problem in your example is storing unsanitized data in the database. If instead you take the parsed data and store that in the database, this would not be a problem. Storing unsanitized data can also lead to problems with the existing Codable implementations. E.g. when we are looking at evolving the model. Say we start with the following:
struct Foo: Codable {
let x: String
let y: Int
}
and get the following input:
{
"x": "test",
"y": 42,
"z": {
"a": 43
}
}
This decodes fine, so we are storing the raw input in our database.
Now we are adding a new field in a backwards compatible way:
struct Foo: Codable {
let x: String
let y: Int
let z: Double?
}
When loading the data from the database, we are getting an error. If we instead re-encode and store the decoded object in our database, we would not get an error.
That said, I already mentioned above that I would be okay with rejecting multiple keys, but I don't think it is any safer or less safe to go with either option. In the end the data has to be sanitized before storing it.
Safety problems generally occur when someone writes code under wrong assumptions. It can be about parsing unicode strings, the boundaries of something, the inputs you'll receive, what's stored in a database, etc.
In this case I think most people would reasonably assume that given the same program and the same input (even a faulty input) you'll get the same result when decoding. Breaking that assumption would be really bad. Even writing a command line validator for some format you read using Codable could give a different result at each run, which would be really crappy for a validator. I think you absolutely need to validate that there is only one discriminant.
But it goes a bit deeper than that. Programs evolve: what v1 saves in a file or database (and how it gets validated) is going to be read by v2 which will implement things differently. One version of a program might use Codable while another might use something else. One might be written in Swift and the other in some foreign language for a niche platform. I'm a bit concerned by the serialization format using a dictionary to store the discriminant because other parsers are going to be written for the format, many with less scrutiny, and some will inevitably have this problem. That's a reason to prefer @hisekaldma's counterproposal: there's no room for that situation to arise since there's no "assumed to be one value" collection of discriminant in the format.
Ok, there's a whole lot to comment on here, and it feels like the proposal as a whole has been muddied a bit.
To me it seems like the configurability aspect of the synthesis is almost the greatest concern since people clearly have different needs and expectations.
So perhaps an exploration of how the synthesis could be configured to either have a special discriminant key with the enum case name as it's value versus using the enum case name as key. Are there other 'standard' encodings of the case?
Is it possible/sensible to use static properties on the type to configure this?
Is there precedent in swift for having synthesized code be affected by static properties?
Would it be possible/sensible to have a global property to set a global default (so you don't need to 'pollute' every enum type with configuration in case you always use the same strategy)?
Is it better/possible to use protocols somehow?
Is it possible to come up with a default that would satisfy everyone?
With regards to selecting a default, perhaps surveys of other languages/libraries that deal with coding of enums with associated values would be possible. Perhaps also a survey of open source swift code to see which patterns tend to be used?
Now for a few replies to various discussions points above:
A contrived example, but definitely the associated values can have different semantics based on the case name:
enum Transaction {
case payment(amount: Decimal)
case reversal(amount: Decimal)
}
This definitely convinced me that configurability of the synthesis is an absolute necessity.
I agree with your description of the difference between enums and structs, and that's exactly what I was trying to point out using the term 'duality', in the sense of the word: An instance of opposition or contrast between two concepts or two aspects of something.
I agree completely! Itās not the strongest example in that sense. But I still think that what it illustrates is true: people will write code under the assumption that the same program decoding the same data will always result in the same thing, and if we break that assumption, that can be exploited.
Thereās the āmiddle groundā too: encoding the discriminator as a value, and then encoding the associated values into its own container next to it:
{
"case": "load",
"value": {
"key": "MyKey"
}
}
This was mentioned in an old thread, but I donāt actually think itās as common in the wild.
I was mulling this over too, and since this only affects the synthesis, not the semantics of the Codable protocol, could an attribute work?
@caseAsValue enum Command: Codable {
case load(key: String)
case store(key: String, value: Int)
}
@caseAsKey enum Command: Codable {
case load(key: String)
case store(key: String, value: Int)
}
As a small exploration, I tried creating a MirrorEncodable protocol that uses the Mirror API to encode a type:
struct MirrorCodingKey: CodingKey {
var stringValue: String
var intValue: Int?
init?(stringValue: String) {
self.stringValue = stringValue
}
init?(intValue: Int) {
self.stringValue = "\(intValue)"
}
}
protocol MirrorEncodable: Encodable {}
extension MirrorEncodable {
func encode(to encoder: Encoder) throws {
var container = encoder.container(keyedBy: MirrorCodingKey.self)
let mirror = Mirror(reflecting: self)
for child in mirror.children {
guard let label = child.label, let codingKey = MirrorCodingKey(stringValue: label) else { return }
if let encodable = child.value as? Encodable {
try container.encode(AnyEncodable(encodable), forKey: codingKey)
}
}
}
}
struct AnyEncodable: Encodable {
var _encodeFunc: (Encoder) throws -> Void
init(_ encodable: Encodable) {
func _encode(to encoder: Encoder) throws {
try encodable.encode(to: encoder)
}
self._encodeFunc = _encode
}
func encode(to encoder: Encoder) throws {
try _encodeFunc(encoder)
}
}
struct Both<A: Encodable, B: Encodable>: MirrorEncodable {
var a: A
var b: B
}
let encoder = JSONEncoder()
encoder.keyEncodingStrategy = .convertToSnakeCase
let val = Both<Int, Int>(a: 1, b: 2)
if let encoded = try? encoder.encode(val), let str = String(data: encoded, encoding: .utf8) {
print(str)
// Outputs:
// {"a":1,"b":2}
}
So by conforming to this type, you basically get the same output as you get by the synthesized Encodable conformance.
An interestring property of the MirrorEncoder is that it also works on enums:
enum Either<A: Encodable, B: Encodable>: MirrorEncodable {
case a(A)
case b(B)
}
let val = Either<Int, Int>.a(1)
if let encoded = try? encoder.encode(val), let str = String(data: encoded, encoding: .utf8) {
print(str)
// Outputs:
// {"a":1}
}
I am of course trying to make the case for the case name as key is a very natural thing point of view, but I do find it nice that this encoding (when the the associated value is directly encodable) requires no 'invention' of new concepts.
This experiment also suggested another thing for me: Coding of enums cases with tuple values is no different than coding of structs with tuple values.
By that consideration, tuples are not Codable on their own, but tuples that are part of a struct (or class) or enum could be encoded/decoded as part of the Codable synthesis of the containing struct/enum (in case each of the tuple values are Codable).
In that way, if we solve the configurability concerns with Codable synthesis for enums and land this feature, we could also allow structs containing tuple properties to become Codable.
Consider the following extension to the MirrorEncodable above:
struct MirrorCodingKey: CodingKey {
var stringValue: String
var intValue: Int?
init?(stringValue: String) {
self.stringValue = stringValue
}
init?(intValue: Int) {
self.stringValue = "\(intValue)"
}
}
protocol MirrorEncodable: Encodable {}
extension MirrorEncodable {
func encode(to encoder: Encoder) throws {
var container = encoder.container(keyedBy: MirrorCodingKey.self)
let mirror = Mirror(reflecting: self)
for child in mirror.children {
guard let label = child.label, let codingKey = MirrorCodingKey(stringValue: label) else { return }
if let encodable = child.value as? Encodable {
try container.encode(AnyEncodable(encodable), forKey: codingKey)
} else {
let childMirror = Mirror(reflecting: child.value)
var childContainer = container.nestedContainer(keyedBy: MirrorCodingKey.self, forKey: codingKey)
for grandChild in childMirror.children {
if let encodable = grandChild.value as? Encodable, let label = grandChild.label, let codingKey = MirrorCodingKey(stringValue: label) {
try childContainer.encode(AnyEncodable(encodable), forKey: codingKey)
}
}
}
}
}
}
struct AnyEncodable: Encodable {
var _encodeFunc: (Encoder) throws -> Void
init(_ encodable: Encodable) {
func _encode(to encoder: Encoder) throws {
try encodable.encode(to: encoder)
}
self._encodeFunc = _encode
}
func encode(to encoder: Encoder) throws {
try _encodeFunc(encoder)
}
}
let encoder = JSONEncoder()
encoder.keyEncodingStrategy = .convertToSnakeCase
struct Nested: Encodable {
var x: Int
var y: Int
}
struct Both<A: Encodable, B: Encodable>: MirrorEncodable {
var a: (a1: A, a2: String)
var b: B
}
enum Either<A: Encodable, B: Encodable>: MirrorEncodable {
case a(a1: A, a2: String)
case b(B)
}
let val = Both<Nested, Int>(a: (a1: Nested(x: 8, y: 9), a2: "a2 value"), b: 1)
if let sko = try? encoder.encode(val), let str = String(data: sko, encoding: .utf8) {
print(str)
// Outputs:
// {"a":{"a2":"a2 value","a1":{"x":8,"y":9}},"b":1}
}
let val2 = Either<Nested, Int>.a(a1: Nested(x: 6, y: 7), a2: "a2 value")
if let sko = try? encoder.encode(val2), let str = String(data: sko, encoding: .utf8) {
print(str)
// Outputs:
// {"a":{"a2":"a2 value","a1":{"x":6,"y":7}}}
}
With this implementation, there is a very similar handling of tuples no matter whether they are part of a struct or an enum.
Furthermore, having a 'name' for the tuple in both situations (case label, variable name) allow for customizing the key coding of the tuple labels, as suggested by the proposal - and extended to cover customization of unlabelled tuple values by @anandabits here:
And this would work the same for tuples in enums, structs and classes.
I find that kind of neat!
Note that the proposal and this topic has also seen alternative suggestions for serializing tuples (as arrays). That could also be a configuration point, but even then, the tuple coding could work equally for enums and structs.
I understand the duality of enums and structs well, and the concept of duality in general (I'm a mathematician). But I disagree on the duality of representations that you suggest. They do not automatically fall out of the duality of concepts, methinks.
Ah, I see - sorry for misinterpreting your objection.
And you are right about your observation: I used the word 'duality' to compare the case label and the variable name, which is wrong. The dualities are the struct vs. enum concepts.
Perhaps I should have said that the case label and the variable name may be considered to serve a similar purpose of identifying a value.
Although I also get your point that the case is part of the value and thus is not identifying it...
Terminology aside, I think there are some differences between the two, that makes the case name sufficiently different from property names, that they can't always be used interchangeably.
Whether coding keys is such a case, I guess is up for debate. But at least to me, it feels too contended to be used as a default synthesis for all Swift users. Auto-synthesis should be used when there is a single obviously correct implementation.
Iāve been looking at common practices some more, and there does indeed seem to be a clear split between ācase as valueā and ācase as keyā.
One thing that stands out to me is that many of the ācase as keyā examples only have single unlabeled associated values, e.g. Either<A,B> or IntOrString. It makes perfect sense to me that if this is how you mostly use associated values, then itās more intuitive to encode the case as a key. Whereas if your associated values mostly have labels, and sometimes even overlap between cases, maybe itās more intuitive to encode the case as a value.
That makes me wonder if a workable compromise could be to divide synthesis into different cases (no pun intended):
A. Enums with single unlabeled associated values
These would be encoded using the case as the key, and the associated value as the value, i.e. not wrapped in a keyed or unkeyed container.
enum IntOrString: Codable {
case int(Int)
case string(String)
}
is encoded as
{
"int": 3
}
{
"string": "Hello"
}
B. Enums with labeled associated values
These would be encoded using the case as a value, and the associated values in the same container.
enum Command: Codable {
case load(key: String)
case store(key: String, value: Double)
}
C. Enums with multiple unlabeled associated values, or both labeled and unlabeled associated values
These wouldnāt get synthesis. Synthesis is only a convenience after all, and if thereās no obviously correct behavior the best thing is to write the conformance manually.
This seems to me like it might be a good compromise that gets us most of the benefit of synthesis with the least surprising behavior for both camps.
It would also avoid a lot of the complexity of both the proposal and the counter proposal: Thereās no question of what to do with mixed labeled and unlabeled associated values. And we donāt need a way to organize and name separate coding keys for each case.
Finally, thereās an easy way to switch between the two encodings, without extra protocols or attributes:
Want behavior A but with labeled associated values? Wrap them in structs.
Want behavior B but with single associated values? Give them labels.
Whenever I want a type to get some default implementation of a protocol Proto, I oftentimes declare a new protocol ProtoByProxy: Proto { ... } , with new requirements, and then provide a default implementation for Proto in an extension of ProtoByProxy using the new proxy requirements.
CaseInsensitiveEnumDecodable example
For instance, if I want my string-backed enums to be instantiable using case-insensitive raw strings, I will declare a protocol like so
protocol CaseInsensitiveRawRepresentable: CaseIterable, RawRepresentable where RawValue == String {}
extension CaseInsensitiveRawRepresentable {
init?(rawValue: Self.RawValue) {
guard let value = Self.allCases.first(where: {
$0.rawValue.caseInsensitiveCompare(rawValue) == .orderedSame
}) else { return nil }
self = value
}
}
By conforming my string-backed enums to this new protocol, it will also be RawRepresentable, but it will inherit a default implementation based on the protocol requirements for CaseIterable.
I can define another refined protocol, CaseInsensitiveEnumDecodable, like so:
Now, by simply declaring conformance to this new CaseInsensitiveEnumDecodable my enums will automatically be Decodable through RawRepresentable and my enums will accept any casing:
enum Foo: String, CaseInsensitiveEnumDecodable {
case foo, bar
}
try JSONDecoder().decode([Foo].self, from: #"["fOO", "BAr"]"#.data(using: .utf8)!)
// [.foo, .bar]
IdentifiableByKeyPath example
A lot of my models are de-facto identifiable, but their identifier aren't always called id. In fact, they almost never are. So I have this protocol together with a default implementation for Identifiable:
protocol IdentifiableByKeyPath: Identifiable {
associatedtype Identifier
static var idKey: KeyPath<Self, Identifier> { get }
}
extension IdentifiableByKeyPath {
public var id: Identifier { self[keyPath: Self.idKey] }
}
For my models, I can now choose to implement either the var id: ID { get } requirement or the static var idKey: KeyPath { get } requirement. In any case, my models will be identifiable.
I've used this pattern for many use cases where I want to provide conformance to some protocol, indirectly through some other requirements.
Case in point:
Can we use a mechanism like this, to declare a few new protocols, say DecodableByKeyedDiscriminator, DecodableByReflection, DecodableByWhatever and so on, which will inherit from Decodable and provide a default implementation with specific semantics through new requirements? That way, Swift user can choose if they want their enums to conform to one or the other protocol.
Some of these new protocols could be endorsed by the compiler with unambiguous automatic synthesis
Does this feel swifty? Too magical? Too convoluted?
I think that this is a neat technical workaround, but I would be concerned that the behavior is too surprizing for many users.
I think that this is a great way to solve the synthesis issue. If this proposal is returned for modifications, I would suggest that something like this is used to provide a way to configure the synthesis.
Another solution would be using attributes as suggested here:
That could work too, but I think that the protocol version is just as good, and might require less changes to the language.
