CountableSequence protocol?

i’ve got some generic database-insertion logic that needs to perform validation by checking the number of inserted rows:

func insert(_ elements:some Sequence<Element>) async throws
{
    let response = ...

    if  response.inserted != elements.count
    {
        throw response.error
    }
}

unfortunately Sequence doesn’t have the concept of a count, only Collection and its refinements do.

and requiring some Collection<Element> would be fine if i only needed to insert Array, ArraySlice, etc.

but i have a “chain3-shaped” type that can only be iterated as such:

extension Records.Masters:Sequence
{
    @inlinable public
    func makeIterator() -> Iterator
    {
        .a(self.a.makeIterator(),
            next: self.b.makeIterator(),
            then: self.c.makeIterator())
    }

    @inlinable public
    var underestimatedCount:Int { self.count }
}
extension Records.Masters
{
    @frozen public
    enum Iterator
    {
        case a(IndexingIterator<[Record.Master.A]>,
            next:IndexingIterator<[Record.Master.B]>,
            then:IndexingIterator<[Record.Master.C]>)
        case b(IndexingIterator<[Record.Master.B]>,
            then:IndexingIterator<[Record.Master.C]>)
        case c(IndexingIterator<[Record.Master.C]>)
        case exhausted
    }
}

it can compute its length in constant time:

extension Records.Masters
{
    @inlinable public
    var count:Int
    {
        self.a.count + self.b.count + self.c.count
    }
}

but it is not compatible with Collection-based APIs because it is not a Collection. so i need to have two parallel implementations, one that calls the Collection.count witness and another that loads Records.Masters.count without consulting any Collection conformances.

perhaps there is a missing intermediate protocol here?

(for what it’s worth, i’m totally aware that it’s possible to genuinely implement Collection for Records.Masters by defining a custom index type and index(after:). but this is difficult and more importantly, not needed!)

Can't you just use your own Countable protocol and protocol composition?

protocol Countable {
  var count: Int { get }
}

func insert<T: Sequence & Countable>(_ elements: T) async throws 
1 Like

things that already conform to Collection will not automatically inherit a conformance to such a Countable protocol.

if Countable is an internal-only protocol, it is fine to retroactively declare conformances to it en-masse, but Countable is so general that it inevitably becomes a public protocol, and mass-conforming external types to a public protocol is not hygienic.

Types that conform to the Collection protocol shouldn't inherit it anyway. Instead you could make a wrapper struct that will adapt a Collection type to Countable and make two insert functions.

protocol Countable {
  var count: Int { get }
}

struct CountableCollection<C: Collection>: Countable, Sequence {
  var underlying: C
  var count: Int { underlying.count }
  func makeIterator() -> C.Iterator { ... }
}

func insert<T: Collection>(_ elements: T) async throws {
  try await insert(CountableCollection(elements))
}

func insert<T: Sequence & Countable>(_ elements: T) async throws {
    let response = ...

    if  response.inserted != elements.count
    {
        throw response.error
    }
}
2 Likes

good idea. i actually ended up dispensing with the generic wrapper entirely and just made three insert functions:

extension DatabaseCollection<Record.Master>
{
    func insert(_ elements:Records.Masters) async throws
    {
        try await self.insert(count: elements.count, elements: elements)
    }
}
extension DatabaseCollection
{
    func insert(_ elements:some Collection<Element>) async throws
    {
        try await self.insert(count: elements.count, elements: elements)
    }

    private
    func insert(count:Int, elements:some Sequence<Element>) async throws
    {
        ...
    }
}

Nice, very similar ideas: just handle Collection in it's own thunk, but provide to actual implementation as few info as it needs.

1 Like

Forgive me if this is a silly question, but can't you just count how many items you see as you iterate over the sequence? Presumably you have to iterate it anyway in order to perform the insertion.

1 Like

the iteration is performed by the database driver, this is to avoid allocating intermediate buffers when the intent is only to serialize the sequence elements to a binary buffer. the driver does this internally when serializing queries to be sent to the database server, and it doesn’t keep a count of elements serialized because that is specific to the semantics of Insert, Update, Delete, etc.

i suppose the driver could special-case the CRUD commands, since they all have similar ideas about success vs. failure. but right now it does not distinguish between commands with ‘counts’ and more general commands like DropDatabase, etc.

perhaps there are missing features in the driver.

As another option - if you want to stick with a single implementation that's generic to all Sequences - you could make a wrapper sequence that remembers how many items it's seen, e.g.:

struct CountTrackingSequence<T: Sequence>: IteratorProtocol, Sequence where Element == T.Element {
    private var underlyingSequence: T
    private(set) var count = 0

    init(_ s: T) {
        self.underlyingSequence = s
    }

    mutating func next() -> Element? {
        guard nextElement = underlyingSequence.next() else { return nil }
        count += 1
        return nextElement
    }
}

(typed here; untested)

Then:

func insert(_ elements: some Sequence<Element>) async throws {
    var countedElements = CountTrackingSequence(elements)

    let response = driver.insert(countedElements)

    guard response.inserted == countedElements.count else {
        throw response.error
    }
}

One difference here is that it counts how many elements were read from the sequence, not necessarily how many it contains. Depending on your precise needs that could be either beneficial or a problem.

doesn’t this pass countedElements by value? if i understand the idea correctly, the driver’s insert function needs to take the sequence’s iterator inout. but i think it is quite uncommon to write APIs in terms of some IteratorProtocol

Ah yes, you're quite right. Old-fashioned reference-centric thinking on my part. It needs to be a class, which is fine. Here's an actual working version (tested in a playground, this time):

class CountTrackingSequence<Element, T: Sequence>: IteratorProtocol, Sequence where Element == T.Element {
    private var underlyingSequence: T.Iterator
    private(set) var count = 0

    init(_ s: T) {
        self.underlyingSequence = s.makeIterator()
    }

    func next() -> Element? {
        guard let nextElement = underlyingSequence.next() else { return nil }
        count += 1
        return nextElement
    }
}

Tangentially I was hoping this could be done using explicit borrowing (as of Swift 5.9), but you can't do borrowing on copyable types and being copyable is a prerequisite for conforming to Sequence and IteratorProtocol. Plus that'd require code changes to the driver, which it sounds like you might not be able / enthused to do.