"BinaryWriting" package to match BinaryParsing?

I’m making a lot of use of the BinaryParsing package; it’s a godsend for reading binary data.

But I also need to write structured binary formats, and AFAIK there’s not a good solution for that, so I’ve started writing my own. I’ve got a BinaryWriter protocol with a variety of write() methods that take Spans, UnsafeBufferPointers, integers, Strings… One implementation appends to a Data object, the other writes into a fixed-capacity MutableSpan.

If I’m reinventing the wheel, please let me know! Otherwise, would there be interest in having me extract this into a reusable package?

/** Protocol for a binary output stream, which may or may not have a limited capacity. */
package protocol BinaryWriter: ~Copyable, ~Escapable {
    /// The number of bytes written to this stream.
    var count: Int {get}

    /// The number of bytes more that can be written to this stream.
    /// (If the stream has no fixed capacity, this will be something close to `Int.max`!)
    var available: Int {get}

    /// Writes raw bytes to the stream. May throw if there's not enough capacity.
    mutating func write(unsafeBytes bytes: UnsafeRawPointer?, length: Int)

    /// The closure is passed a MutableSpan whose size is `maxCount` if there's enough capacity,
    /// else the writer's remaining capacity.
    /// It should write to some prefix of the buffer, then return the number of bytes it wrote.
    mutating func writeInto(maxCount: Int,
                            fn: (inout MutableSpan<UInt8>) throws -> Int) rethrows

    /// Passes the closure a span of the already-written bytes at the given range.
    func readBytes<T>(range: some RangeExpression<Int>, fn: (Span<UInt8>) -> T) -> T

    /// Reads or writes an already-written byte.
    subscript(index: Int) -> UInt8 {get set}
}

/** Free bonus methods for BinaryWriter!!! */
package extension BinaryWriter where Self : ~Copyable, Self : ~Escapable {
    /// True if no more bytes can be written.
    var isFull: Bool {available == 0}

    mutating func write(unsafeBuffer: UnsafeRawBufferPointer)

    mutating func write(bytes: RawSpan)

    mutating func write(byte: UInt8)

    mutating func write(bigEndian: some FixedWidthInteger)

    /// Writes a non-negative integer in (Go-style) varint form.
    mutating func write(varint: some BinaryInteger)

    /// Writes a String in UTF-8 encoding.
    /// - note Does not write a trailing 0 byte!
    mutating func write(utf8: String)
}

5 Likes

Just in case you haven't seen it, is this pitch (Safe loading API for RawSpan) related?

That's somewhat related, but more of a building-block that we'd like to have available for BinaryParsing. @nnnnnnnn and I have discussed API for binary formatting to mirror BinaryParsing in the past; it's definitely something that we'd like to have available but we happened to have a more immediate need for parsing. That might take the form of its own package or just be rolled into the existing binary parsing package.

2 Likes

I recently wrote a BinaryCodable implementation to make save games work on a personal project, and this would've been helpful. I think adapting the various types to output was more of the work and outside of the scope of what you're suggesting, but not having to write the write functions would've been helpful enough!

Here’s an earlier discussion with a few more links:

https://forums.swift.org/t/how-can-i-encode-a-struct-to-data-binary