Swift Async Algorithms: Buffer

The other very interesting thing that seems to be emerging here is a general pattern that we're seeing time and time again in these algorithms.

That is, something that conforms (or could conform) to AsyncIteratorProtocol:

@rethrows
public protocol AsyncIteratorProtocol {
associatedtype Element
mutating func next() async throws -> Element?
}

And something that is what you might call an AsyncSource with a shape similar to:

public protocol AsyncSource {
associatedtype Input
mutating func send(_ input: Input) async // maybe this should throw?
mutating func finish() // maybe this should take an Error?
}

So then a buffer would be:

public protocol AsyncBuffer: AsyncSource, AsyncIteratorProtocol {}

This shape appears in AsyncBuffer, it appears in the context for AsyncChannel and it also appears in AsyncRelay. I can see other potential uses for it to. So perhaps 'buffer' doesn't describe the versatility of this type.

What appears to be emerging to me is something more akin to a Rx style Subject or what's sometimes called a Processor. The key difference is that AsyncBuffer isn't an AsynchronousSequence, but an AsynchronousIteratorProtocol.

And that's a really interesting difference because when you think of an Rx Subject you know more or less that it must be multicast – at least all the stock ones are. That's a lot of baggage for an implementor of a Subject. With an AsyncBuffer there's no need to worry about that – it just spits out its next element to whoever asks. That makes it so much easier to reason about.

It also makes me wonder if an AsyncBuffer of this shape is the more natural home for the 'top' of a pipeline. I think, understandably, people get very confused why AsyncStream can't be iterated twice and I don't have a good answer for people who ask.

Creating an AsyncBuffer as the top of the pipeline, then feeding that into the init of some kind of multicasting algorithm might make more sense to people. It's clear an iterator is unicast. It also keeps the separation of concerns between source/consumption – something I've heard expressed as an important distinction that needs to be upheld.

1 Like