What could cause an unbounded AsyncStream to drop elements?

[taylorswift]

@FranzBusch

So the big benefit over just yielding from a channel handler to an AsyncStream is that the inbound AsyncSequence is correctly retaining backpressure. If you are slow in consuming the inboundStream then the no more reads will be issued in the channel pipeline. As Cory noted above the whole system has to retain backpressure and this is one of the places where we do it.

the inner inboundStream is retaining backpressure, but the outer inboundStream is not: it is adding a child task to the DiscardingTaskGroup and immediately continuing to the next connection. instead of allowing a pileup in an AsyncStream of requests, we are just moving the pileup to the task group. am i missing something here?

@FranzBusch

It certainly doesn't have to, it is just how you done it. If you make an outbound request to a database I don't see why an AsyncStream must be used.

the site serves requests in a linear queue to insulate mongod from concurrent requests. an actor-based synchronization model doesn’t provide this guarantee because actors are re-entrant.

@FranzBusch

The one thing that you are not handling here is task cancellation so if a request would come and you spawn a potentially expensive work in the requestStream. While the expensive work is being handled the request/connection goes away nothing is going to cancel it. This might open up an attack vector for your application.

the swift application is really just an intermediary between the outside world and a local mongod process. there is not much point in “cancelling” a database request - they are written to execute within a predictable amount of time so cancelling it would have the same amount of latency as just waiting for the query to return.

@FranzBusch

However, just from looking at code and without further metrics it is really a stab in the dark. There might be a lot of other things going on here like a specific request causing unbounded amounts of work which might have nothing to do with your server being necessarily attacked with thousands of requests.

yeah… persistent metrics are hard. data visualization is hard.

however, it seems unlikely that this is a Query of Death situation, precisely because the site serves requests in a linear queue. if the problem were a specific query, then normal queries made during an attack wouldn’t also be as slow as i’ve observed.