Swift Async Algorithms: Buffer

Related Projects Swift Async Algorithms
, ,
21

Perhaps we ought to look at from the perspective of a potential use case. Something that @twittemb and I were discussing is the value of having a back pressure supporting buffer. A buffer that allows a producer to run ahead of the consumer to a pre-determined limit. This kind of buffer would be used as a smoothing mechanism to increase throughput in a back pressure supporting pipeline.

channel.buffer { ThroughputBuffer(limit: 5) }

There's a few things that can be discussed here:

  1. If programmers choose to use a buffer that increases throughput, they're concerned about performance. Adding an actor as an intermediary here will very often necessitate an actor-hop and adversely impact that throughput. It also runs against the prevailing advice which is to batch operations which are to be run on a separate actor. By their very nature, asynchronous sequences are very 'non-batch like'. Actor-hopping is slow, especially when the actors are on different executors.
  2. Using an actor doesn't actually prevent deadlocks in and of itself. As we will be providing the 'stock' buffers, the custom implementations are likely to have a level of complexity to them in any case. Enforcing that implementors use an actor may in fact create a false sense of security as we already know how confusing people find actor re-entrancy. Quickly, people will be using Checked/UnsafeContinuation to suspend callers, to be resumed later. That requires a relatively good level of understanding from the implementor. So regardless of whether an actor is used or not, the docs will need to stress that a level of care is required to do this right. (For example, by ensuring they resume all their stashed continuations.)
  3. Related to point two, if people are stashing continuations, they'll need to be a cancel() method as otherwise the suspended Tasks won't resume. I'm not sure how needing to call cancel() from an async context may complicate things, but typically the practice has been to make calling cancel synchronous so this may need to be nonisolated.

In summary, I think that if we provide 1) a good base of stock implementations, and 2) a thorough explanation in the docs that this type requires careful implementation, there should be no need to lock people down who wish to implement their own. So I agree with @FranzBusch , this should be Sendable with a mutating pop/push – and an additional cancel.

1 Like