Need help using Combine Demand (back pressure)

Using Swift
1

Combine has two related functions that support "demand", where Subscribers inform Publishers on the desired number of elements passed to them in a "receive" function. The below ignores infinite demand.

  1. func request(_ demand: Subscribers.Demand)
    Subscriptions provide this function, and as the Apple Docs say:

"Tells a publisher that it may send more values to the subscriber."

Matt Gallagher supposes in his excellent 22 Combine Tests article that each of these demands should be additive, and when the Subscription sends elements to the Subscriber, it decrements the count it maintains.

  1. func receive(_ input: Self.Input) -> Subscribers.Demand

When a Subscriber receives data, it returns another demand, which the Apple docs state is:

"A Subscribers.Demand instance indicating how many more elements the subscriber expects to receive."

I have seen various interpretations on how these numbers relate, and I of course have my own that I'll postulate here.

A Publisher has a one element, and it gets a 'request(.max(10))' When it sends the element to the Subscriber, then the return demand should be '.max(9)', a reminder to the Subscription that it's expecting 9 more elements.

If for some reason the Subscriber decides to send in another request for .max(10), and the Subscription gets one more element, then messages the Subscriber with that one element, the return should then be .max(18), meaning, Subscriber wanted 10, then it wanted 10 more, but it has only received 2 and still wants 18 more.

Alternate interpretations seem to be that the return from receive is additive to the running demand. In this scenario any number other than 0 will increase what the Subscription can send.

Would be super if anyone in the know could help clarify!!!

2

All demands, both those given through the subscriptions request(_:) method and those returned from the receive(_:) method, are cumulative.

They have to be cumulative because, for example, by the time you are returning from receive(_:), the subscription may have irrevocably queued more calls to receive(_:) by handing them off to a Scheduler. It's too late to retract your demand.

3 Likes
3

Agree that request() demands should be additive. And your introduction of queued request(), I believe, makes an even better case for the returned demand being the total for what the Subscriber is willing to accept at that moment, and not additive.

  • Subscriber queues a request(.max(10)), notes it has a request for 10 outstanding
  • ... time goes by
  • Subscriber queues a request(.max(10)), notes it has requests totally 20 outstanding
  • Subscription gets data, sends it to the Subscriber via receive()

The Subscriber has no idea if the second queued request has been received or not. I argue that the only proper return at this point is .max(19). The Subscription could then, if it has more data to send, send up to 19 more elements. In this case, it may note that its running demand has gone negative - because it would appear its sent more items than the requests would indicate. A good reason for max to be an Int and not a Uint.

If on the other hand, the returned demand is additive, then the Subscriber has a choice:

  1. if sending multiple request()s , always return .max(0), since it never knows when subsequent requests() are actually received

  2. only send one request() - the initial one - and from then on only indicate its desire for more data by returning a demand when it gets a receive. In this case, it can't ever request more data once its cumulative demand has been met (since it won't get any more receives())

It seems to me that either approach is possible, what I would dearly love is someone on the Combine team to provide guidance. In fact, if I hadn't read that multiple requests should be additive, I would have assumed 2 above.

A good case for 1 IMHO is that if you have a time based scheduler, and a Subscriber is willing to accept 10 elements per unit time, but the Subscription is temporarily blocked, then the stream of requests signal the Subscription that the demand is increasing and it might be able to take proactive steps to get more "stuff" in its pipeline.

4

In Combine, Demand is accumulative (additive). If a subscriber has demanded two (2) elements, and then requests Subscribers.Demand(.max(3)) , the publisher’s unsatisfied demand is now five (5) elements. If the publisher then sends an element, the unsatisfied demand decreases to four (4). Publishing elements is the only way to reduce unsatisfied demand; subscribers can’t request negative demand.

For additional information about applying back pressure, please see Processing Published Elements with Subscribers.

2 Likes
5

OK - that link is definitive! Was it just published? It has a 2020 copyright in any case.

So it makes it 100% clear - all are additive. Thanks so much for the link!

1 Like
6

Seems @pxpgraphics beat me to mentioning that article but there's been a number of great Combine docs that have been published recently after some hard work.

3 Likes
7

I figure it must be pretty new as I googled like mad when I was doing my work a week ago. Thanks for your effort!

1 Like
8

Yeah it was published very recently so it wasn't your searching skills :stuck_out_tongue_closed_eyes:

1 Like