Will SwiftNIO adapt to the new Combine framework?

Related Projects SwiftNIO
21

Is URLSession integration with Combine using Publishers.Promise?

1 Like
22

Ahh, cool. Looks like maybe Deferred<Once> probably maps to what I’m thinking of. Thanks!

23

Great to hear how much consideration went into this!

Maybe a convenience initializer on Subscribers.Sink which takes a Result based closure would be helpful. Of course the finished completion would have to be represented as a kind of Error then.

Talking about Sink - maybe Source and ValueSource would be good naming alternatives to Subject and CurrentValueSubject?

Finally - what is your current thinking of async/await with Combine now in place? I still feel that reactive programming is more tailored towards data-flow use-cases whereas async/await covers control flow cases. But the lines are blurry of course with things like coroutines/generators.

1 Like
24

We've talked about some kind of ResultSink, but it's not part of the current API (obviously). It's super easy to add operators in this kind of library but hard to remove them, so we've opted to focus on the ones that are harder to implement via composition first.

Source is another overloaded word in the Cooca SDK - RunLoop has them, and so does Dispatch. Since Combine is part of the OS, we've tried to avoid confusing newcomers to the API by reusing names they may be familiar with from elsewhere. This is unavoidable in some cases, but here we felt we still had a choice.

With respect to async/await, I think that Combine will be able to integrate well with any reasonable future direction of the language. Some of the goals of async/await and Combine are similar, so perhaps we can use this as well as experience from other APIs like it to help inform the future language direction.

8 Likes
25

Ping @millenomi regarding availability on Linux

4 Likes
26

Ok, it seems you have a strong opinion on this, so this is my last try :slight_smile:

What about Emitter with emit() instead of send()?

Implementations would be:

  • StatelessEmitter instead of: PassthroughSubject
  • StatefulEmitter instead of: CurrentValueSubject

Cheers

27

In case you would like to explore and build your own Emitter, rather than having it be part of the core Combine (perhaps it could be added if it proves itself in the future):

The thing with "emitting into a stream from the outside" is the following consideration:

  • you may not be actually allowed to do so (!), since there may not be enough demand to push (downstream) the such emitted element.

So if you would:

emitter.emit(1) // may not have had any demand so this may have to "explode"
emitter.emit(2) // and here this means we assumed we had 2 demand... but we might not have had (!) -> boom

So an example of making this still in bounded buffer space yet "nicer to use" is making the emitter queue based and also signal back to whomever calls emit() if the push can be fulfilled or not, with varying strategies (e.g. drop elements or blow up if would have to drop); An example of such implementation is from Akka where we did it as Source.queue (ignore naming, as does not 1:1 map nicely to Swift land): Source.queue • Akka Documentation

So usage becomes more "informed"

queue.offer(x).map { ... check if was pushed or not etc }

Having that said, it is simple enough and possible to write such Publishers/Subscribers as extensions to the library and usually it is the best to start them out externally so once they prove themselves they could be included in the core (not saying that they will, just saying that's how we rolled with Akka Streams :slight_smile:).

And it is actually quite awesome that it is so easy to add your own publishers like that so I recommend playing around with it :)

28

Could you point me to something explaining how? I've managed to finally watch the sessions and it didn't look very promising. It seems a lot like the Node v1 streams which proactively publish everything they get, instead of waiting for read demand like v3 streams.
Do publishers have a .suspend/.pause mechanism?

In a NIO context say you have to process 10k incoming JSON records to be persisted in a DB. You'd want the reading socket suspend publishing (aka reading) until the DB actually inserted a batch of records. (NIO itself can do this w/ the auto-read thing I think).
What's the construct in Combine to accomplish that?

29

I can't answer your other questions, but Combine does have the notion of demand.

30

Publishers must wait for demand. When a Subscriber subscribes to a Publisher, they get a Subscription. This Subscription allows a Subscriber to request some data using a metric called Demand. Subscribers can also grant a Publisher more Demand when they receive more input.

It is a violation of the contract to provide inputs to the Subscriber in excess of the Demand they provide.

Combine provides pull-based flow control, so a suspend/pause mechanism is not required: it's implicit.

Don't offer demand.

3 Likes
31

Thanks for the explanation. The demand does "bubble" up through the various operators like map etc, right? (e.g. in a setup like jsonReceiver.map.filter.dbInserter)

32

In general the requirement is that each subscriber must not receive inputs for which they have not granted demand. In practice operators may issue more demand than the leaf subscriber does in some cases, e.g. removeDuplicates may want to do so if a duplicate is received.

2 Likes
33

Looks like @millenomi is not responding - @johannesweiss any other way to find out if Combine will be open source and available on Linux?

2 Likes
34

One more ping since it's been a month @johannesweiss @millenomi ^

35

You are probably not going to get an answer until Apple unveils its plans. Apple employees like Lily are not allowed to comment on any future directions Apple might take, or what they are planning. Since Combine is an Apple-private framework at this point, Apple won't respond until it does.

7 Likes
36

I think this perfectly sums up the hundreds of unanswered questions that people have asked for apple to answer here on these forums.

6 Likes
37

Makes sense... Johannes raised my expectation haha. And I wished Lily would at least leave a word about it.

38

I think it would make sense to have Combine as open source. I think they need time to prepare the code to make it available for the community, review, and lawyer stuff. Combine is both of high general value and being open source would help spread Swift more largely, to the benefit of Apple ecosystem.

Note that some people already started to reimplement it in Open Source: GitHub - OpenCombine/OpenCombine: Open source implementation of Apple's Combine framework for processing values over time.

2 Likes
39

And, making it open source also limits the ability of Apple to further modify it to meet its needs!

I think there is an assumption that Combine is "finished." iOS 13 and macOS 10.14 haven't been released yet, and I suspect that the Combine functionality implementation has been "just enough" to allow Apple to get iOS 13 and macOS 10.14 out for release, as well as start the developer feedback loop. I would not be surprised if there is a bunch of functionality in the final release that is not in the betas now (although it's probably pretty close in beta 7, at this late date). I would also suspect that it might be a year or so before Apple feels that it is stable enough for release as open source, if they plan to release it for open source. Then, they have to do the stuff you mentioned.

But, I could be wrong!

1 Like