Global executor hooks + SwiftNIO

Server
, ,
1

Swift Concurrency has hooks that allow you to implement a custom global executor and avoid Dispatch altogether. Check out this example:

Refer to these files for understanding:

swift/stdlib/public/Concurrency/GlobalExecutor.cpp
swift/include/swift/ABI/Executor.h

main.swift:

extension UnownedSerialExecutor {
    static var generic: Self {
        unsafeBitCast((0, 0), to: self)
    }
}

do {
    typealias OpaqueJob = UnsafeMutableRawPointer
    typealias EnqueueOriginal = @convention(c) (OpaqueJob) -> Void
    typealias EnqueueHook = @convention(c) (OpaqueJob, EnqueueOriginal) -> Void
    
    let handle = dlopen(nil, 0)
    let enqueueGlobal_hook_ptr = dlsym(handle, "swift_task_enqueueGlobal_hook")!.assumingMemoryBound(to: EnqueueHook.self)
    
    enqueueGlobal_hook_ptr.pointee = { opaque_job, original in
        print("simple example succeeded")
        original(opaque_job)

        // In real implementation, move job to your workloop
        // and eventually run:

        // let job = unsafeBitCast(opaque_job, to: UnownedJob.self)
        // job._runSynchronously(on: .generic)
    }
}

Task.detached {
    exit(0)
}

dispatchMain()

This is a proof of concept that implements only 1 of 5 hooks and also invokes swift_job_run in a hacky but convenient way.

Implementing a custom global executor in SwiftNIO would allow running concurrency jobs on the same threads that do I/O syscalls and drive epoll or io_uring instances.

Is the SwiftNIO project interested in this development?

2

That hook takes over ALL scheduling so it may not be appropriate in many situations. Generally using custom actor executors is the more appropriate way most of the time.

We’re also interested in allowing to swap the main actors executor or specifying task executors potentially.

So yeah the hook exists but I’m unsure if we should be using it in normal systems other than for testing or in runtimes which entirely take over all threading…

It’s not “wrong” per se though. Cc @lukasa @FranzBusch

2 Likes
3

We have experimented with overriding the global executor with a NIO EventLoop already but we came to the conclusion that this is not something we want people to do. It forces all async work onto the EventLoop and gives users almost no way to move their heavy compute workload off the IO EventLoops.

We also explored EventLoops as custom actor executors and have added support for this in one of our recent releases. This already allows some patterns to work nicely and avoid thread hops between the actor and the underlying NIO primitives. However, the current rule that all non-isolated methods are forced onto the global executor makes it impossible to run high performance IO that is, at least partially, using Swift Concurrency.

What we really would love to see is support for setting the executor for a given task which is then inherited down the tree. This would give users the possibility to hang a task tree on a NIO EventLoop and performantly interact with NIO primitives from Swift Concurrency. This would also play into our larger overall goal of getting most users to use our new async interfaces instead of writing NIO ChannelHandlers.

2 Likes
4

Both of you are right that implementing a custom global executor is a dramatic action. However, jobs running on this "generic" global executor are privileged; they are able to synchronously switch to "default actor" executors. This is the only way for an actor to be an efficient async lock accessible from multiple threads (I'm aware that uncontended traditional locks are still 10x faster). However, a server app built to reap the potential benefits of this design would probably look very different from SwiftNIO.

A new type of Task that is tied to a specific custom executor would be a great addition to the language and work well with SwiftNIO's current design. This task would stay on its custom executor when calling non-isolated async functions (modifying the rule set in SE-338), and its children should inherit its custom executor.

2 Likes
5

We're exploring ways to extend the amount of control over task execution in some upcoming proposals. Exact shapes TBD but we'd solve that with language/library features, rather than encouraging the big hammer that is this global hook

3 Likes
6

Hey, I consider this a great way to run server application that need high-performance I/O. We have also been playing with this and I opened a SwiftNIO PR to add similar functionality to SwiftNIO.

I haven't gotten feedback from all of the SwiftNIO team yet but I don't see why we wouldn't ship this with SwiftNIO. The full solution will hopefully land with @ktoso's great Task executor preference proposal. Once that proposal is implemented and lands in Swift I'd expect the vast vast majority of users to prefer that over completely taking over the Swift Concurrency pool.

2 Likes