I'm the maintainer of the MessagePack.swift library on GitHub. Since the introduction of Swift 4, there has been interest in the new Codable functionality. However, we've noticed that there is a lot of code duplication required in order to write a new Encoder / Decoder implementation. One user wrote a great implementation of MessagePackEncoder / MessagePackDecoder but it's highly based on the existing JSON implementation.
What are the best ways to write new Encoder or Decoder implementations without duplicating the code from the Swift standard library? This is not only for code licensing reasons but also developer experience.
While waiting for a real answer, you might want to check out this great blog post by Mike Ash about implementing a binary encoder and decoder, there could be some tricks or insights that would help you.
One thing we've been considering doing in Foundation is splitting out the core implementations of JSONEncoder/PropertyListEncoder and JSONDecoder/PropertyListDecoder into separate classes that provide a good amount of default implementation for folks to subclass; at the moment, a lot of implementation is shared indirectly between them, and we think this can be done better, especially for the benefit of third parties.
Questions which have come up about how to do this have been somewhat organizational, without good answers. This new encoder/decoder pair wouldn't be able to decode anything on their own, but would be a convenience for developers writing their own encoders and decoders; this makes the API a bit awkward in that it's not really useful to developers using Foundation, but rather, consumers of other frameworks which do end up using it as an intermediate.
This raises a few questions regarding naming (how to we make it clear what these classes are useful for?) and organization (is it possible to add this API into a submodule of Foundation so it doesn't get imported by default?), but we don't have great answers at the moment.
There was some amount of discussion in a previous thread (PITCH: Export _JSONEncoder / _JSONDecoder) from before the forums were introduced, but maybe we can get some more opinions on this.
In any case, this effort wouldn't be one that would arrive in Swift 5.
I understand well the concerns exposed by @itaiferber on this feature. At this point, I'm not sure what the best route would be either. I understand the lack of user facing benefits for a consumer of Foundation, but also understand the need / power the default implementation provides. I really like your 'submodule' idea.
We could end up with something like
import Foundation
import Foundation.Decoder
class MyDecoder: Foundation.Decoder.JSON { // the original _JSONDecoder
override func container<Key>(keyedBy: Key.Type) -> KeyedEncodingContainer<Key> {
return CustomContainer()
}
}
class MyOtherDecoder: Foundation.Decoder.PropertyList { // the original _JSONDecoder
override func container<Key>(keyedBy: Key.Type) -> KeyedEncodingContainer<Key> {
return CustomContainer()
}
}
struct CustomContainer: UnkeyedEncodingContainer {
internal let container: _JSONUnkeyedEncodingContainer
fileprivate init(referencing encoder: _JSONEncoder, codingPath: [CodingKey], wrapping container: NSMutableArray) {
self.container = _JSONUnkeyedEncodingContainer(referencing: self.encoder, codingPath: self.codingPath, wrapping: array)
}
// using proxy pattern invocations for UnkeyedEncodingContainer
}
Subclassing is not that important, the important part is to be able to reuse part of the EncodingContainers for common operations.
As for scoping, the only strategy I would know now is struct wrapping, so it would 'Hide'. Not sure if it's in scope to publish module inside the foundation module.
In any case, this effort wouldn’t be one that would arrive in Swift 5.
I can dedicate a certain amount of time in the writing of those, if we can agree on a form.
I already implemented a framework that abstracts from Encoder/Decoder and supplies a interface that's easier to work with (I do think). I'm going to explain my approach at the link from above (PITCH: Export _JSONEncoder / _JSONDecoder4).
I already opened a pull request on @alex.akersMessagePack.swift repository with my implementation, but maybe someone else will stumble upon this topic...
I do think that this approach is the best I considered, because it reduces code duplication, therefor makes it easier to fix, it isn't touched by any licensing concerns, as far as I can see, because it should be different enough from JSONEncoder to be not concerned a derivate work (but I'm no lawyer) and it still supplies the same functionality as JSONEncoder/JSONDecoder. On the other side it of course adds a dependency to the framework and the code that needs to be written is more compact but a but ugly (but maybe I have just not found the best way to implement it).
The overall solution I prefer concretely for MessagePack.swift, is providing another framework, that implements the code that's necessary to convert any Codable instance to a MessagePackValue. This MessagePackValue can then be encoded and the other way around. The advantages of this solution in contrast to directly serialising to Data, is that MessagePackValue can still be embedded in a map, which is, as far as I can see, a recurring use case. A further advantage is that you can "feed MessagePackValue instances into the Encoder" or obtain them from a Decoder, which is useful if you liked to manually convert certain properties to MessagePack, but do not want to convert the remaining parts.