AsyncSequence and Publisher from the Combine framework are two different implementations of the same idea. I think it is worthwhile to discuss if it makes sense to build additional APIs for AsyncSequence to a point where it can replace Combine, or should AsyncSequence be something else on purpose?
4 Likes
Since Combine is still a private Apple framework, whether it gets replaced or not is up to Apple.
Linux/Windows/BSD/Android/WASM users: “What’s a Combine framework?”
We should build Swift’s async story in a way that makes sense for the language and gives all users of the language a complete experience, entirely unafraid to step on Combine’s toes, build a bypass outside its front-door, or drink its beer.
That’s how we treat every other library. We wouldn’t hold back on language-integrated regexes because there was a library somewhere which also did pattern-matching. That’s not to say that we ignore it or purposefully make life harder for it, but we don’t need to limit our ambitions to leave “room” for it.
20 Likes
It is not the intention of this discussion if Apple should deprecate Combine. But enhancing AsyncSequence would allow replacing Combine if wanted. There is nothing wrong with Combine - AsyncSequence just could replace it semantically.
Ok, I am going to have to be very careful here with my response: the word "should" infers two things - a moral obligation, and also it infers future direction. Neither of which are things I can tread into.
However I think it is useful and productive to discuss the functionality that we can add to AsyncSequence (especially since it is part of the Swift open source efforts). We should also outline the both functional and imperative sides of things that would be useful to flesh out the concepts of a set of algorithms that would make asynchronous things that happen more than once; a set of algorithms that are just functional are likely to not have general appeal, likewise for a set of algorithms that are purely focused on imperative code could have some detractors as well. So I think any solutions should be a balanced approach catering to the things that are common tasks that developers face.
I think there are a few examples of categories that are worth talking about:
combinators - things like zip that bring multiple async sequences together
distributors - things that do the reverse of zip, that allow multiple interested parties to deal with an AsyncSequence
composition - things that allow building stuff up (I could imagine things like @resultBuilder could be neat here to bridge that gap of functional versus imperative
value transformation - most of which this type of thing is covered by the similar APIs to Sequence, but there could be others that would be interesting to investigate
ordering transformation - Sequence does not have these since it is not asynchronous so this might be a neat area of discussion
temporal transformation - again this is another area that is missing and would be good to talk about
There are a number of VERY related efforts that are of interest:
Obviously for the temporal transformation category: [Pitch] Clock, Instant, Date, and Duration is a big part. Having a duration type will allow for the concept of throttling or debouncing etc.
For combinators such as zip: Pitching The Start of Variadic Generics is in my experience developing other similar APIs a must-have. Else you are limited to a fixed number of combinators
For general ergonomics SE-0328: Structural opaque result types is I think a very critical proposal because it makes the concept of erasure much easier to reason about (avoiding having perhaps questionable hanging statements like .eraseToAnyPublisher() or the equivalent)
28 Likes
Just a general note about my intention starting this thread. I did not mean to start a discussion about if we should make a Combine clone using the new async APIs from swift. What I want to discuss is if it makes sense to develop AsyncSequence to a point where it can be used instead of Combine due to its functionality.
1 Like
How about collecting ideas/needs for each category you listed on a semantic level? Something like:
Combinators:
1. We need something that takes the `nth`element of `m` async sequences and produces a stream of m-tuples.
1 Like
Thinking the same thing!
Just to be clear; I don't feel like my list of things is a gospel of what should be done, but more so the parts that were deferred because it was just the first step. I have tinkered with a lot of different examples in my exploration previously but I think the important part here is to ensure that it is an open and collaborative effort.
4 Likes
Starting small is the right way to go. And I think we should not aim for a list of everything that could be useful for an AsyncSequence. But we could start with a list of things we know to be useful.
Each of these are ordered in the usefulness/occurrence I have seen. This is a non-exhaustive collection of things I have gathered from either working with folks directly adopting AsyncSequence or discussions on the forums. There is definitely prior art for a number of these, so we have precedent on our side here. This isn't exactly a small list of things but it isn't really a huge list either I guess...
Combinators:
Emitting a tuple of values combined from N AsyncSequences where each emission awaits the combination to be distinct
Emitting a tuple of values combined from N AsyncSequences where each emission awaits for the most current update to be distinct
Emitting an AsyncSequence of values derived from when any of N AsyncSequences produce a value
Distributors:
Sending a value to an AsyncSequence where the send awaits the consumption.
Sharing the iteration in a safe manner across multiple consumers where each consumer gets a distinct value
Sharing the values from iteration among a known number of consumers
Composition:
Being able to take an AsyncSequence and flow that into a bit of logic that is written in imperative style. Given the analogy of structured versus unstructured concurrency - if unstructured concurrency is to UIKit constructed by hand as structured concurrency is like storyboards, then composition is the counterpart to declarative syntax like SwiftUI (specifically the body of Views).
Value Transformation:
The only one that really seems missing that I know of is .enumerated() which to be honest I am not sure if it is really used that much.
Ordering Transformation:
When dealing with emissions of emissions (read higher ordered items) it is useful to be able to switch to the latest thing that has emitted
Collecting values by limits is useful for buffering
Flattening a transformation is very useful. flatMap already exists in the non-concurrent variant, but perhaps a concurrency limited version would be useful
Temporal Transformation:
Debouncing values over a specific window of time is very useful
Throttling values over a specific window of time can be useful
Side note - from my exploration w.r.t. the Clock/Interval/Date/Duration proposal these two transformations need a clock and a duration since the reference point temporally is relative to the last value emitted.
Failure Transformation:
Concepts like catching or retrying in async sequences might be useful in the functional side of things however this would require some features not currently slated for work: namely typed throws or generic effects. These seem like reasonable cases where we could defer this type of work until it is more of an area of interest for the rest of Swift.
Performance:
Another area of interest is a batching accessor to AsyncSequence itself. AsyncBytes is quite fast, but I think we can make it even faster by building an "unsafe variant" accessor similar to the underscored APIs of Sequence.
General Async/Await:
There are a few needed parts to make those. Obviously the proposals I listed are some key players, but also there needs to be a couple of critical APIs to build these up.
Getting the first result from N running tasks
4 Likes
As to "Temporal Transformation":
Coalescing values over a time interval using a reducer, resulting in emitting at a steady (slow) pace for high frequency sources, has my vote too 
1 Like
I'm interested in this, are you speaking of throttling of some sort? or something in the actual family of reduce?
Yes, throttling by coalescing the values instead of skipping them. Using a closure like that used by reduce().
So the first value emitted from source results in waiting to see if more values are emitted, and after the waiting period the reduced result is emitted.
Edit: Useful default behaviors would be just emitting the last value received, or emitting an array of values.
good idea, so for example one could pick out the "meaningful values" and use those instead of just dealing with a leading or trailing edge of the signal.
I wonder if that could be composed somehow by parts, or is that a primitive that can compose other things (e.g. like throttle that drops)
1 Like
Just wanted to mention that as far as time is concerned, it'd be nice to know what AsyncSequence has to say about scheduling. Combine's Scheduler protocol makes it possible to control time in a fully synchronous way that makes testing nuanced, time-based operations fairly easy to do once you have a test scheduler at your disposal. While the internal plan may be to utilize "custom executors," I haven't seen a whole lot of discussion on the topic.
9 Likes
I think the closest analogy would be a custom Clock adopter, that can trivially allow the manual increments for "step based scheduling" which is VERY useful for testing. The custom executor part is something more generally for how actors behave, not per se how the temporal side of things work.
One of the carve-outs in the Clock/Instant/Date/Duration proposal is specifically to bring in that concept of custom clocks. Which was made specifically with the use case in mind of things just like you mentioned - a "test scheduler".
I would expect all temporal based transformations to have a funnel variant that is generic upon the clock (granted there may be some that have handy shortcuts of common clock types).
5 Likes
I think debouncing is another good example of a "temporal transformation".
1 Like
I know in the realms of testing this one is quite important, but I think also has some application in non testing code (especially when used in conjunction with combinators):
A way to go from a regular sequence to an async sequence. Similar to how lazy works. That way you can take an indefinite async sequence and zip it with a finite, pre-known, non-async sequence.
1 Like
I’m all for deprecating combine as long as they fill in the feature gap.
Combine has a lot of useful operators that don’t exist for async sequences (debounce, scan, etc…)
Also swiftUI relies heavily on combine.
If they got rid of combine, I think they would have to backport some changes to earlier versions of SwiftUI.
Lets try to keep this focused on what we need and not speculate on Combine or SwiftUI - those are future directions that distinctly limit the interaction we can have here.
But you mention scan. That is a totally reasonable ask (which falls into the category of the missing value transformations).
In the form that @jmjauer put:
We need something that provides a continuous production of reduced values using a closure.
Specifically this is useful for when the AsyncSequence of values may be indefinite and developers need to fold/scan a reduction down.
I am trying to gather that "etc" part to get an understanding of what other things are needed. Some of the operators from Combine just flat out don't really make total sense in the async/await world: things like receive(on:). Other things are perhaps better suited to hop out of the chain and write imperatively; like for await in or catch.