Hi folks, I’m using .share(bufferingPolicy: .bufferingNewest(1)) and was a bit surprised by what I got: when all existing consumers have processed that single buffered element, a new consumer will not receive it. From reading back the various discussions I understand this is a deliberate decision, but I’m wondering what folks more expert than I would suggest to get the behaviour I expected. (Namely: I’m specifying a fixed-size buffer and a drop policy, and I would like new consumers to receive that buffer-size of previous elements if they have ever been sent.)
I can’t say I fully understand the source code, but it looks as if a small adjustment to trimBuffer would give me what I want, holding back trimming to ensure that at least buffer-size elements are kept even if the consumers have all moved on. (This seems straightforward for .bufferingNewest and .bufferingOldest, I think it would be ok for .bounded, and presumably a bad idea for .unbounded as it amounts to infinite replay.) But I’m not at all enthusiastic about maintaining a fork of the package adding this odd little bit of behaviour.
Is there some sensible way to get this behaviour by composing pieces we already have?
(Expertise calibration: I know that back-pressure means the producer will not resume until the consumer has advanced; I know that .bounded does not imply dropped elements; I cannot follow the reasoning in the algorithm’s comments about isolation and why nonisolated(unsafe) is acceptable here.)
(There is no workaround for this issue, since it sends the iterator; if it sent the sequence we could at least say that it's safe if the upstream sequence is Sendable; since it sends the iterator there's no guarantee even of that, although it is still likely)
The buffering behavior you are thinking of is a different (but related) algorithm in the same family of .share. This was discussed during the pitch/review phase of share - it is a replay algorithm; which is something that could be added in the future. Currently this does not exist since the buffering of that would be rather tricky to hold an indefinite and filled buffer.
The nonisolated(unsafe) is a work-around for lacking a concept of a sending box. In this particular case it is safe because of the isolation guarantees by the careful dance around @_inheritIsolation's behavior in conjunction with the isolated attribute on the closure. It only allows creation of the iterator and then immediate passing of that iterator back to the isolation of the inherited context. So suffice it to say it is a very niche use case and very carefully manipulated. Plus there has been buy-in by the compiler and runtime folks that this is an "approved" mechanism for this specific case. I would however very much discourage anyone from using this particular pattern in other scenarios since it is rather specific for the constraints.
It's not safe, as the bug report demonstrates. Because the iterator ends up in a different isolation from the upstream sequence, anything which results in the modification of state shared between the sequence and the iterator can and will race. Since the upstream sequence isn't (required to be) Sendable, there's no particular reason that this shared state should be locked.
I think what that bug report demonstrates is actually the issue that was brought up during development which had non concrete example: why it probably should require Sendable for the base sequence. Which in your example applying Sendable (iiuc) would require the base shared state to somehow be isolated and alleviate the bug; so it is not per se the transferring box but instead the base shared state from the reference type of parent in your bug report. I think that could be addressed if we do it quickly w.r.t the Sendable requirement - i'll look into that one here in the next few days and follow up in the bug report.
It is technically possible that a Sendable upstream sequence might share non-Sendable state with a non-Sendable iterator. I think that's very difficult to write in current Swift, but I can't rule it out; I think you could achieve it by checking dispatch_get_current_queue() == dispatch_get_main_queue() inside makeAsyncIterator. Whether that counts as "safe Swift" is debatable, but there may be other options.
Thanks, yes I understand this was already discussed. It’s precisely because this is tricky to get right that I’m asking what good options exist that don’t involve rolling my own. So far the answer appears to be “none”.