Custom binary encoder

Swift standard library provides two Encoder implementations: for JSON and for Property List. Both (not 100 percent sure about PL though) support keyed containers for storing structures. What if the destination format hasn't notion of keyed objects and just stores fields in order? Can Codable protocol be used in this scenario? If not, how to implement something similar? Specifically, who is actually responsible for a generation of a serialisation code for structs inheriting Codable? In Rust for comparison it would be done by a macro, but I'm very new to Swift

Yes that works, I implemented a binary coder as an experiment some time ago based on this blog post: mikeash.com: Friday Q&A 2017-07-28: A Binary Coder for Swift

The encode/decode functions are called in source order so it's possible to ignore the keys (at least in the current Codable implementation, I don't think it's officially documented).

1 Like

It's up to the author of the Encoder to decides what happens in this case. For example, you could store keyed objects in [key, object, key, object, key, object, ...] linear order, then re-hydrate an actual dictionary when decoding.

Codable code synthesis is currently performed by the compiler.

I would strongly advise against throwing away keys, as otherwise-harmless source code changes (e.g., moving a property around in the source file) could lead to significant breakages. Right now, the compiler synthesizes your CodingKeys enum based on the order of your properties in source code, and then synthesizes your init(from:)/encode(to:) based on the order of the keys in CodingKeys, but this is not guaranteed in any way. And, even if you define CodingKeys manually and are very, very careful to never reorder keys in the enum, there's nothing requiring the compiler to synthesize code which uses those in order

1 Like

Yes I agree that one has to be careful with code that relies on the source order, but it can also be very useful.

Would it be possible to codify the current behavior? Would this need a swift-evolution proposal?

Quite possibly! This could be treated similarly to SE-0372: Document Sorting as Stable.

1 Like

That's not generally workable. There are many legitimate use-cases for an Encodable type that don't have a fixed set of key-values, let-alone a fixed order. e.g. where one key is only encoded at all based on some program state, or (in decoding) where the decoder conditionalises based on the presence of a key (e.g. to support schema evolution).

If you completely control the entire serdes pipeline, now and forever, then of course it's your option to hard-code limitations and sharp edges. But even then, it might be safer to just implement your own serdes system independent of Codable (or use NSCoding, which allows but doesn't require use of keyed coding).

If you implement the encoding/decoding methods manually on your type, you can just use an „UnkeyedCodingContainer“ in codable. I think this would be very similar to what you would do with NSCoding.

I would strongly advise against throwing away keys, as otherwise-harmless source code changes (e.g., moving a property around in the source file) could lead to significant breakages

This a whole point actually. The struct must define it's own layout.

Codable code synthesis is currently performed by the compiler.

So is it just hardcoded?

Yes. The implementation lives in lib/sema/DerivedConformanceCodable.cpp if you're curious.

So long as the types you're encoding are under your control or are specifically built with this requirement in mind, then this may be something you can rely on. (Again, assuming the compiler doesn't change.)