Future of Swift-NIO in light of Concurrency Roadmap

So what does it mean for us? Rest in peace EventLoopFutures? I realise that it's just a roadmap and a series of pitches, and things might (will) change a lot, but it's certainly the future, and it's unlikely that Swift-NIO would want to ignore it :slight_smile:

Probably, it will take not one but two major versions to adopt it. Or maybe even zero, and we just add async API as additional package, like swift-nio-transport-services, though maybe I read the threads inattentively, because I didn't find a way for working with external async stuff, so can't tell right now how converting an EventLoopFuture to async call would look like (yeah I mean API is kinda straightforward — public func getResult() async throws -> Value, but how would it work under the hood?)


It depends on exactly what the particular API is trying to do.

In some cases, methods will drop EventLoopFuture as a means to the async/await portion of the API intent, in others it will return a potential Task, as cancellation is potentially needed.

In some cases it might be "possible" to not have them be marked as async or return a Task at all, and the type itself becomes an actor class that handles the synchronization.

One thing for sure is that SwiftNIO will be a huge case study for the impacts of the proposed syntax and semantics that will help shape the final version that lands into Swift. :smile:

and given that the Core Team has mentioned that these features will land across a few version of Swift, it's likely that we might see small incremental changes in major versions, unless the SwiftNIO team decides to wait to focus an entire major version for the new concurrency model (I'm a bit in favor of this TBH)

1 Like

Hey there,
realistically all this will take quite some time to pan out and fully understand all the implications on APIs.

However a few "simple" things that can be done early are:

The concurrency design offers Task.Handle APIs which effectively are "futures" but married together with the async world. When you look at it you realize it's not very special and simply has a func get() async throws -> T that can be awaited on. NIO could adopt this pattern and offer an awaitable function on EventLoopFuture #if swift(>=6), this function has to be invoked from an async context through... so that's a bit more tricky so all NIO functions would need to be async as well -- that is harder to adopt without breaking compat.

It is not entirely clear how NIO would be expressed as actors, I think we'll need to learn and experiment much more to see if, when or how it is viable to do so. Handlers are pretty similar to actors in that they isolate state and run not concurrently -- the language design offers ways to optimize away not necessary "hops" between actors -- however it does so by actor identity -- NIO needs more than that, it needs to do so based on executor identity. NIO will want to "don't hop to other executor because that executor is the same event loop as my event loop" -- today's concurrency design does this check only about actors ("that actor is not me, so perform a hop"), we would need to extend this to check executor equality.

In addition to that we will have to ensure in the concurrency design though that an actor can be put on a NIO EventLoop (i.e. an EventLoop be an Executor) which has specific execution semantics after all -- I can say that I'm definitely looking at this topic and trying to ensure that the actors are flexible enough to express other executors rather than "just" some dispatch based one. If we had this, actors "on" EventLoops could become viable... Caveat that this is just rough ideas and not something we've planned out at all so far.

Long story short: Adoption in NIO likely will require a major version bump. Other than adopting awaitability on futures it is not clear, yet, how language concurrency will be adopted in NIO.

Cory and the NIO team will likely chime in with their thoughts shortly though :slight_smile:


Thanks for bringing this up, this is an important topic and it’s worth understanding.

Let’s start at a very high level with the top-line answer to “What does this mean for us?” The answer from the NIO core team as of the 31st of October 2020 is we don’t know yet. This proposal is very young, there is little running code, we haven’t been able to prototype and investigate.

More importantly, this is not something for the NIO team to decide by fiat. We want to land in a world where users are able to write high performance, flexible code with minimal overhead when that is necessary, but also to have an easy time writing scalable concurrent code. How these things trade off is not necessarily straightforward to reason about.

With that said, let’s dive into some more detailed thinking.

The Swift core team has signalled that they are considering a breaking language change as part of stage 2 of the concurrency work. With that signaled, I would highly prioritise adopting stage 1 incrementally, without a semver major in NIO. Having a language-break accompanied by a NIO break is a nice natural fit.

This would mean that in the short term we’d be looking at incremental changes. The goal will be to enable users to adopt actors in NIO-based programs without needing to fundamentally rewrite much of what they have already written. Existing programs should continue to work just as they do now.

So where will these abstractions come into being?

I think there are three places to investigate, in roughly the following priority order:

  1. Allowing async calls to the Channel API, or something similar
  2. Allowing EventLoops to be Executors
  3. Allowing actor ChannelHandlers

All of these are built on a foundational point, which is “work out how to plumb Futures through the concurrency code sensibly”.

I think the first point is the most important. The vast majority of users don’t write ChannelHandlers, they work on top of Channels. It would be extremely valuable if users could treat a Channel like an actor. In my mind, this will also want to include having an async function you can call to read data from as well, something the Channel API doesn’t provide today.

Normally this would mean turning a Channel into an actor, but because we don’t want to have breaking changes, this will likely instead mean having a way to perform the transformation. This implies an actor class that wraps a Channel and forwards the calls as needed.

This kind of abstraction frees higher level libraries (such as @mordil’s RediStack) up to reinvent themselves as actor-based libraries. They would continue to provide regular, synchronous ChannelHandlers to do the low-level protocol work, but the high-level connection wrangling would become actor-based, at least optionally.

The second point seems to go hand-in-hand. NIO has a core executor concept: it’s an event loop. We need to do the work to enable users to have an actor that lives on an event loop. However, as @ktoso has noted, the executor concept as it exists in the current pitch isn’t very useful to us. This is because being able to enqueue on an event loop isn’t itself any more useful than being able to enqueue onto a dispatch queue. What matters is that you can take advantage of that knowledge to build a big shared mutex. While we should do this irregardless, for this to be actually useful in server side programs we need to press the core team to consider allowing us to say that actors with the same executor do not need to dispatch across each other.

Finally, ChannelHandlers. The ChannelHandler API is low-level and, for the most part, entirely synchronous. The only wrinkle is that sometimes ChannelHandlers want to do things in response to writes completing, say, or closures happening. It would be nice to be able to provide users the opportunity to write their ChannelHandler as an actor and use the source code transformation that actors provide. For the NIO 2 timeframe, we’d probably do that in a similar way to how we do ByteToMessageDecoder: users would implement a different protocol and we’d wrap it in a ChannelHandler to manage the bridging.

I think doing the ChannelHandler thing is contingent on having a way to make confident assertions about what executor is being used. I don’t know how easy it’ll be to do this and I don’t know if it’ll be worth it.

Longer term, when breaking changes come along, they may warrant a more substantial rethink of the NIO API. In particular, they will challenge us to decide whether NIO’s API needs reinvention in terms of these new abstractions, at least in part. Some obvious options include requiring ChannelHandlers to be actorlocal types, which would solidify their current thread-safety requirements, redefining Channels as actors entirely, and so-on.

How much of that we do, and how quickly, will depend on the performance story. Moving all of NIO over to an actor model requires that we do not give up too much performance. If we have to, then we’ll always want to make the actor interface an opt-in higher level abstraction that can be used in less performance-sensitive contexts.


Good morning everyone :wave:

To follow up here, the concurrency efforts now gained another proposal: Custom Executors which relates to some of the questions posed in the thread.

As in: an EventLoop could be a custom executor.


Indeed, custom executors will be really fantastic for SwiftNIO (and any other ecosystem that wants to do I/O with the OS interfaces and also benefit from async/await).

Let me quickly outline where we are:

This is now possible to play with, with the main snapshots available today. You'll need to apply this PR which adds async/await support for SwiftNIO.

This is the step that is not yet supported with today's snapshots but will (hopefully) be added with custom executors.

What this means is that in a program like this one (called NIOAsyncAwaitDemo)

let channel = try await makeHTTPChannel(host: "httpbin.org", port: 80)
print("OK, connected to \(channel)")

print("Sending request 1", terminator: "")
let response1 = try await channel.sendRequest(HTTPRequestHead(version: .http1_1,
                                                             method: .GET,
                                                             uri: "/base64/SGVsbG8gV29ybGQsIGZyb20gSFRUUEJpbiEgCg==",
                                                             headers: ["host": "httpbin.org"]))
print(", response:", String(buffer: response1.body ?? ByteBuffer()))

print("Sending request 2", terminator: "")
let response2 = try await channel.sendRequest(HTTPRequestHead(version: .http1_1,
                                                             method: .GET,
                                                             uri: "/get",
                                                             headers: ["host": "httpbin.org"]))
print(", response:", String(buffer: response2.body ?? ByteBuffer()))

try await channel.close()

We would need to do at least 6 thread switches:

  1. From the executor that runs the async code to NIO's I/O threads (EventLoop) to create the Channel (network connection)
  2. Back from NIO's I/O threads to the executor to report back the result (OK, or Error thrown)
  3. Back onto the I/O threads to send & receive the HTTP request/response
  4. Back onto the executor to report the result of the first HTTP request
  5. Once again, onto the I/O threads to send & receive the HTTP request/response
  6. And finally back once more to report the result of the second HTTP request

In reality, there will be more switches. With custom executors (which would mean that the EventLoop itself can become an executor) we could do the whole thing with 0 thread switches (like we can with the futures directly).

That would in theory be possible today although it would be really slow. Before custom executors, we wouldn't even just need to switch once per send/receive, we'd need to switch twice per "actor ChannelHandler" (in and out). With custom executors, this should become possible too. If that's sensible or not remains to be seen, it'll likely be a performance question because calling an async function is cheap but still more expensive than calling a synchronous function.

tl;dr: Custom executors will allow SwiftNIO (and any other system that needs to do I/O directly) to be fast with async/await because it doesn't force us to thread-switch all the time.


I'm wondering. Currently the only way to run async context from Channel*Handler is to Task.runDetached {...}. But is there a way of forcing it to run in given event loop context? Not the event loop itself, of course, but in event loop's thread. I presume it should optimise the runtime a bit (until we have custom executors and proper bridging between them and NIO). Will

context.channel.eventLoop.execute {
    Task.runDetached {

work as I expect it to work? Or runDetached ignores current thread?

Currently that is correct. Since we don’t have the necessary facilities for NIO to participate in the async world yet (custom executors).

Detach is “detach from whatever context you’re in” and yes, it runs the closure on the global default executor by default.

With custom executors there will be detach(someExecutor) as well. NIO’s event loops should be able to be Executors in the future, once custom executors land.

Terms of Service

Privacy Policy

Cookie Policy