Shoehorning ByteBuffer/ByteBufferView into RangeReplaceableCollection

Related Projects SwiftNIO
1

i’m having a really hard time getting ByteBuffer to get along with my RangeReplaceableCollection-based serializers.

basically, i have an Output type that looks like:

// module: 'BSON', does not have 'NIOCore' dependency

extension BSON
{
    @frozen public
    struct Output<Destination> 
        where Destination:RangeReplaceableCollection<UInt8>
    {
        public
        var destination:Destination

        @inlinable public
        init(capacity:Int)
        {
            self.destination = .init()
            self.destination.reserveCapacity(capacity)
        }
    }
}

extension BSON.Output
{
    /// Appends a single byte to the output destination.
    @inlinable public mutating
    func append(_ byte:UInt8)
    {
        self.destination.append(byte)
    }
    /// Appends a sequence of bytes to the output destination.
    @inlinable public mutating
    func append(_ bytes:some Sequence<UInt8>)
    {
        self.destination.append(contentsOf: bytes)
    }
}

and everything else is built around that Output serializer. for example:

// module: 'MongoWireProtocol', does not have `NIOCore` dependency
import BSON

extension BSON.Output
{
    @inlinable public mutating
    func serialize(message:Mongo.Message<some RandomAccessCollection<UInt8>>)
    {
        ...
    }
}

this works with Array, ArraySlice, etc.

now i want to do:

import BSON
import MongoWireProtocol
import NIOCore

var output:BSON.Output<ByteBuffer> = .init(capacity: message.size)
    output.serialize(message: message)
self.wrapOutboundOut(output.destination)

but i cannot do that, because as we all know, ByteBuffer is not a RangeReplaceableCollection<UInt8>.

on the other hand, ByteBufferView is range-replaceable. but i also cannot do:

var output:BSON.Output<ByteBufferView> = .init(capacity: message.size)
    output.serialize(message: message)
self.wrapOutboundOut(output.destination._buffer)

because ByteBufferView._buffer is internal. and it also ignores the size hint, because ByteBufferView has no witness for RangeReplaceableCollection.reserveCapacity(_:).

i’m not challenging whether or not ByteBuffer ought to be range-replaceable or not, and i’m not criticizing that design choice. if we are being honest the real problem here is RangeReplaceableCollection is way too aggressive in its requirements, we really only need the ability to append to the end of the collection.

protocol AppendableCollection:Collection
{
    public mutating
    func append(_ element:Element)

    public mutating
    func append(contentsOf elements:some Sequence<Element>)
}
protocol RangeReplaceableCollection:AppendableCollection
{
}

but this isn’t practical because:

  1. AppendableCollection doesn’t belong in BSON, or any library (besides the standard library) for that matter.

  2. even is BSON started vending AppendableCollection, it can’t make RangeReplaceableCollection inherit from it. (and i assume the standard library can’t either, because ABI)

  3. code importing BSON would have to retroactively conform types like Array, ArraySlice, etc. to AppendableCollection, which runs afoul of the “never extend another module’s type to conform to another module’s protocol” principle.

at this point, i am thinking my best bet is to write a wrapper around ByteBuffer that is a RangeReplaceableCollection, and then:

  1. copy-and-paste the replaceSubrange(_:with:) implementation from ByteBufferView, and

  2. add real, not-defaulted witnesses for reserveCapacity(_:), append(_:), and append(contentsOf:) (using ByteBuffer.writeBytes(_:)) so that BSON.Output can actually write to it efficiently.

but of course, i am not eager to re-invent ByteBufferView, so i’m wondering if there’s a better way to go about all of this.

2

Yeah I think you’re jumping the gun, there's no need for a whole new type.

With regard to _buffer, it’s internal because that’s not how you turn a ByteBufferView into a ByteBuffer. Instead, do ByteBuffer(output.destination).

With regard to the size hint, implementing it is trivial so I'd recommend that either you or I produce a patch to add the witness for reserveCapacity, append(_:), and append(contentsOf:). ByteBuffer and ByteBufferView are not frozen types, so when you find deficiencies in them we should fix them.

3 Likes
3

aha thanks!

opened a PR here:

2 Likes