I'm building out a Swift wrapper for SRT (a transport that is gaining some traction in the video industry) and am trying to build this wrapper to fit within the swift-nio paradigm so that I can use it in a similar way to TCP or UDP within my projects.
That said I'm finding that there is just a ton of stuff I need to copy over because of access levels. For example, protocols such as "BaseSocketProtocol", "ServerSocketProtocol" and "SocketProtocol". Similarly with Channels it looks like I need to copy over BaseSocketChannel and probably a bunch of other stuff.
I think in the end I'll probably give up on this idea of modelling it after swift-nio for the time because because so much stuff needs to be copied and it would be a nightmare to keep synchronized. It would be great if these facilities were publicly accessible though so that we could make swift-nio compatible extensions without having to copy half the project. I don't imagine SRT is a thing that would be interesting for the main project since it's pretty niche, but it would be good to have some level of interoperability.
Hi @jamesgh, thanks for your question! First of all a disclaimer: I really don't know anything about SRT but from some quick googling it seems that SRT is a protocol on top of UDP. SwiftNIO does support UDP (only in non-connected mode on Linux right now but we're interested in adding connected mode anyway) so I'd expect you to be able to implement SRT using SwiftNIO without changing swift-nio itself. Is there anything that SwiftNIO doesn't support that you'd need to implement SRT?
You're correct, I could make a new implementation of SRT using swift-nio and maybe that's worth doing down the road. It's a pretty complicated protocol and still under development so I would prefer to wrap the official reference implementation for the time being. It's also got its own quirks which makes it a bit of a pain (their non-standard epoll mechanism is one that sticks out). But if it's literally just me who would make use of a concept of "swift-nio extensions" then it's probably not worth doing.
It's not just you: this is an area we're interested in investigating. We've held out so far because we don't want to generalise an API too early: this kind of extra API surface area is hard to put back in the box once you provide it, so it's important to get it right.
However, as @Helge_Hess1 correctly points out, a "SwiftNIO extension" would not help you anyway. To extend SwiftNIO you actually have to extend a specific event loop. An event loop type is unified essentially by how it waits for I/O. There are three main event loops in the NIO ecosystem today: EmbeddedEventLoop (waits for I/O by having user space drive it, for testing only), SelectableEventLoop (waits for I/O by using kqueue or epoll, for file descriptors), and NIOTSEventLoop (waits for I/O using Network.framework, and could also use DispatchSources). To extend an event loop you'd still have to plug in to one of those three methods.
The SRT documentation makes it clear that we could not extend any of those three event loops:
SRT socket being a user level concept, the system epoll (or other select) cannot be used to handle SRT non-blocking mode events. Instead, SRT provides a user-level epoll that supports both SRT and system sockets.
For this reason, you have no choice but to make your own event loop. This event loop may conceptually be similar to SelectableEventLoop, but the APIs must change. As this is required anyway, it likely wouldn't hurt too much to do a substantial copy of those objects and replace the uses of the low-level socket API with the SRT ones.
Thanks for the feedback - yeah I've made a new EventLoop for this already along with the sockets, I just got to the point of doing the Channels implementation and was like - ok I'm gonna have to copy a bunch of stuff.
But this feedback is helpful. At least between what you're saying and how swift-nio-transport-services was implemented it looks like I haven't gotten too far off track here.
Absolutely, you're definitely in the right space. Unfortunately when you move outside of the abstractions NIO gives you it can become clear how much work NIO does to expose its specific API surface, which sadly means there's a lot for you to reimplement.
Please do ask any clarifying questions you may have though, we're happy to explain why things are done a certain way or what is/is not necessary.
You guys have done a good job keeping the code clean and easy to understand so frankly I don't have any questions about the hows or whys - I just needed a bit of clarification on the overall direction I was going.