Tasks are all run on the same thread

Using Swift
, ,
1

I'm using swift-nio which distributes load well over the EventLoopGroup created but as soon as i dispatch an async Task from any of the threads currently running the channel, they are all run from the same single thread.

// ChannelInboundHandler.channelRead
let promise = eventLoop.makePromise(of: ...)
print("Sync: \(Thread.current.name!)") // any of the·NIO EventLoopGroup threads.
promise.completeWithTask {
    print("Async: \(Thread.current.name!)")·//·Always the same thread.
}

This seems to tank performance, as the request handlers, which run via async/await, all run on the same thread this way. It shows on the flamegraph that one thread is having more than double the load of the others. A _dispatch_dispose call is visible there.

Using priority .background or .high did not yield any different behavior.

Using DispatchQueue.global() or DispatchQueue.main results in the same behavior.

Is there any way to use async/await with a thread pool?

Im using Swift 5.7.3 on WSL2 (Ubuntu)

2

As a general rule, async/await does use a thread pool, with the same size as the number of cores on the machine. How are you observing this pattern?

3

I observed with printing out the thread name and using linux perf_events and a flamegraph.

The following example:

public func test() async -> Response {
  print("Async: \(Thread.current.name!)")                                                                                                                                                
  return·Response(buf: nil)                                                                                                                                                              }

// ChannelInboundHandler.channelRead
print("Channel handler: \(Thread.current.name!)")
let promise = eventLoop.makePromise(of: Response.self)
promise.completeWithTask {
  return await test()
}

Prints the following:

Channel handler: NIO-ELT-0-#5
Channel handler: NIO-ELT-0-#0
Channel handler: NIO-ELT-0-#2
Async: NIO-ELT-0-#2
Async: NIO-ELT-0-#2
Async: NIO-ELT-0-#2
Async: NIO-ELT-0-#2
Async: NIO-ELT-0-#2
Channel handler: NIO-ELT-0-#2
Channel handler: NIO-ELT-0-#6
Async: NIO-ELT-0-#0
Channel handler: NIO-ELT-0-#7
Channel handler: NIO-ELT-0-#1
Channel handler: NIO-ELT-0-#2
Async: NIO-ELT-0-#2
Async: NIO-ELT-0-#2
Async: NIO-ELT-0-#2
Channel handler: NIO-ELT-0-#7
Channel handler: NIO-ELT-0-#2
Channel handler: NIO-ELT-0-#7
Async: NIO-ELT-0-#2
Async: NIO-ELT-0-#2
Async: NIO-ELT-0-#2
Async: NIO-ELT-0-#2

Edit:

And flamegraph:

image
image1099×401 32.9 KB

There are some minimal _dispatch_dispose calls on other threads, but one thread takes the main load.

Thinking about it. I have an async function which has no await.

I updated the example to have an await call in my test() function.

public func getResponse() async -> Response {
    let t = Task.detached {
        print("Async2: \(Thread.current.name!)")
        try! await Task.sleep(for: .seconds(0.1))
        let response = Response(buf: nil)
        response.status = .ok
        return response
    }

    return await t.value
}

public func test() async -> Response {
    print("Async: \(Thread.current.name!)")
    return await getResponse()
}

Channel handler: NIO-ELT-0-#1
Async2: NIO-ELT-0-#4
Async: MyApp
Async2: NIO-ELT-0-#4
Async2: MyApp
Async2: NIO-ELT-0-#4
Async: MyApp
Async: MyApp
Async2: NIO-ELT-0-#4
Async2: NIO-ELT-0-#4
Channel handler: NIO-ELT-0-#0
Channel handler: NIO-ELT-0-#7
Async2: MyApp
Async2: NIO-ELT-0-#4
Async: NIO-ELT-0-#4
Async2: NIO-ELT-0-#4
Async2: NIO-ELT-0-#4
Async2: NIO-ELT-0-#4
Async2: MyApp
Async2: NIO-ELT-0-#4
Async2: NIO-ELT-0-#4
Async2: MyApp
Async2: NIO-ELT-0-#4
Async2: NIO-ELT-0-#4

Only with Task.detach i see one additional thread (NIO #4). As before, there are some sporadic calls on other threads, but they are so few. Some threads (#4 and MyApp) take the main load again.

I use a benchmark script to test the server, so the load is high and i would expect it to distribute across threads because of the Task.sleep. But instead the MyApp thread is just getting slower.

I have the feeling i'm missing something fundamentally.

Edit: Found that async functions are called on other threads, but one or two threads take the main load. Still way slower than just using NIOs EventLoopFuture

4

I have removed the prints and just run the code above (with Task.sleep) on a release build

image
image1106×249 30.4 KB

One thread (NIO-ELT-0-#3) this time takes 40% of the load and is the only thread handling the async calls.

5

Those answers don't seem entirely possible: async functions cannot execute on NIO event loop threads. I'm inclined to assume the thread names are broken here.

1 Like
6

Note also that a regular profiler won't produce the results you want, as async functions don't use the regular C call stack.

7

Thanks for the reply.

You are right, the thread names are broken.

When I observe the process with htop, i see that the same number of processes as my core count spawn additionally to the nio ones (fixed to 8, #0 to #7). My cpu has 12 cores.

image

They are spawned having some random NIO name, sometimes all async/await threads have the name of my application.

This explains why I observe the same thread name in every async function and i have sometimes some random additional thread (like #2 as seen in htop) printed out.

This also explains why Flamegraph/perf_events maps those calls to some NIO thread.

Thanks for your help.

1 Like
8

This is sadly a bug in libdispatch on Linux: It inherits random thread names which is super unhelpful :frowning:

5 Likes