Swift Async Algorithms Proposal: Broadcast (Previously Shared)

Hi everyone and happy new year.

To push the thinking a but further on that matter, I've implemented a version of broadcast based on a buffering strategy.

Let me know what you think.

2 Likes

Coming back to this after the holidays now. I agree that there are multiple different behaviours; however, I believe we should start with the "consume at the speed of the fastest consumer including a buffer". This is to me the safest option. If we realise that further down the line we want the other one we can always create that and provide an option for it.

The option you laid out without back pressure is basically just a buffer attached to the upstream and a "fastest consumer" strategy. So I wouldn't spend any time on that.

W.r.t. to "Subjects" I would like to see how the ecosystem evolves once we get broadcast, because as you said, you can compose an AsyncStream and broadcast to get the desired behaviour. From my experience "Subjects" have sometimes lead to confusion so maybe a more explicit composition is good.

My point is that perhaps this is something we should consider holistically.

Subject's may appear complex on the surface, but perhaps that's because they solve multiple problems in a single-shot:

  1. Performing the Source Sequence role
  2. Multicasting unicast source sequences/pipelines (including conversion of custom unicast AsyncSequence types to be multi-cast)
  3. Side-stepping the multi-pass AsyncSequence issue

My concern is that if we continue down the path of solving only no. 2, we will slowly move away from solving no. 1 and no. 3.

My opinion is that there seems to be an attachment to the API of AsyncStream, and particularly breaking apart the 'producer' and 'sequence' parts, similar to the discussion about enhancing AsyncStream with a new initialiser, in the name of simplicity. However, the alternative seems to be many more types with limited composability. So far, I'm not sure this is a progression of the benefits provided by a Subject like type, and to me it feels even more complex.

For example, now programmers must remember that they always need to chain a broadcast sequence to their asynchronous sequence pipelines if they wish to vend to an unknown number of consumers. Worse, if the sequence is type erased (any AsyncSequence), they'll need to do this defensively to avoid undefined behaviour.

Of course, a Subject equivalent would need to be carefully designed to leverage the unique properties of AsyncSequence (i.e. autoconnect doesn't make much sense in the context of a back pressure supporting pipeline), but on the whole, my guess is the surface complexity of Subject will result in a net reduction in complexity when viewed holistically and compared to the current direction.

Hope this is helpful – happy to discuss any of the points raised in further detail if useful.

Is there any traction on this?
After wwdc I was hoping to see async algorithms bring more feature parity to what Combine offers but it doesn't seem like it is happening any time soon and your post was the only thing going in that direction. Are there any initiatives for this to be taken into consideration or to produce solutions which tackle the problems you are trying to solve here?

Thank you for your effort btw :)

I agree, progress on asynchronous sequences does appear to have slowed down somewhat.

I don't have any particular insights into why, but my guess would be that we're dependent on decisions in other areas of the language before AsyncSequence can really get into a good shape. For example, a decision on typed throws will determine the shape of a protocol for production side of an AsyncSequence – an AsyncSource or whatever it ends up being called. It would also allow for primary associated types to be locked-in for AsyncSequence, or failing that whether a stop-gap AnyAsyncSquence type should have 1 generic parameter, or include a 2nd parameter for a typed error.

More widely, I'm not sure asynchronous sequences can ever be a complete replacement for Combine as the focus for asynchronous sequences is decidedly on the asynchronous part. Probably quite rightly, It's very hard to wrestle Swift concurrency into acting synchronously as has been discussed quite extensively on the Observable proposal.

Maybe, Observable will eventually fill much of this particular gap, but that remains to be seen.

1 Like

Observation has taken a lot of my time to get that over the line - broadcast is definitely one of the first post 1.0 things to tackle.

Things that need to be done:

  • There are a few lurking sendability/concurrency warnings in strict mode that need to be addressed.
  • The initializers for types should be audited; e.g. the extension accessors for things should be the primary access point and we should have a uniform construction story (iirc there are a few w/ public initializers on the types)
  • The inlining needs to be audited for ABI boundaries
3 Likes

Is there any chance to get a rough estimated timeline on when a 1.0 can be expected and when things like broadcast will have a decision which path to take?

1 Like

The faster we can get some of those issues I pointed out addressed the quicker we can start to work on those other parts.

I have a couple of PRs up for some of them -

But truth be told there are bits that folks could help out getting those done faster. Reviews definitely are useful, also confirmation of testing of those changes. Additional audits of inlines, sendability and warnings will go a long way to shoring up the remaining bits.

7 Likes

With Async Algorithm 1.0 released in December last year, what is the status on getting Broadcast / Shared / Multicast implemented in Async Algorithm package? Anything being worked on? :pray:

7 Likes