Pack: A Swift package to serialize and deserialize various data types into an external representation

Hi Everyone,

I thought I'd share a package that I wrote a while ago that might be useful for the general swift community.

Pack is a Swift package for serialising and deserialising various data types into an external binary representation.

This can be useful for writing custom file formats, packing data efficiently for transmission...etc.

Pack is similar in functionality to the built-in Codable protocols, however, unlike Codable, Pack is not key/value based, and as such is intend for packing and unpacking binary data for efficient storage. Because there are 101 ways to pack complex data, basic swift types are supported, but adding support for more complex types is left to the end-user who is responsible to ensure the data layout matches the end-use case.

The full documentation is available here: Documentation

Serialising and deserialising data

To serialise some data, you create a BinaryPack object and pack the data:

// Initialize a new Packer
let packer = BinaryPack()

// Pack an Integer
try packer.pack(12345)

// Pack a Double
try packer.pack(6789.0)

// Pack a String with utf8 encoding
try packer.pack("Hello, world!", using: .utf8)

You can then get the data as a Swift Data object, and use a BinaryPack object to unpack the data.

// Initialize a new Unpacker, and specify the data that should be unpacked
let unpacker = BinaryPack(from: packer.data)

// Unpack an Integer
let int = try unpacker.unpack(Int.self)

// Unpack an Double
let double = try unpacker.unpack(Double.self)

// Unpack an String that was packed with utf8 encoding
let string = try unpacker.unpack(String.self, using: .utf8)

Serialising and deserialising complex types

Pack supports serialisation and deserialisation of basic swift types, but support for more complex types can be added by the client, who is responsible for the layout of data in memory.

struct Color {
    let name: String
    var red: Double
    var green: Double
    var blue: Double
    var alpha: Double = 1.0
}

To allow this struct to be serialised and deserialised, it must add conformance to Packed, which is shorthand for Packable and Unpackable.

extension Color: Packed {
    init(from unpacker: inout Unpacker) throws {
        self.name = try unpacker.unpack(String.self, using: .utf16)
        self.red = try unpacker.unpack(Double.self)
        self.green = try unpacker.unpack(Double.self)
        self.blue = try unpacker.unpack(Double.self)
        self.alpha = try unpacker.unpack(Double.self)
    }

    func pack(to packer: inout Packer) throws {
        try packer.pack(name, using: .utf16)
        try packer.pack(red)
        try packer.pack(green)
        try packer.pack(blue)
        try packer.pack(alpha)
    }
}

And that's basically it. I know there's other packages to do essentially this, but hopefully this is useful for a few people.

The full documentation is available here: Documentation

Thanks.
-Matt

5 Likes

Amazing work, Matt. I wonder how this relates to prior art (like CBOR, BSON, Protobuf, Cap'n'Proto, FlatBuffers, et. al.) in terms of performance and packing efficiency?

3 Likes