DispatchQueue behaviour under load

Using Swift
1

Hey all, I've been trying to benchmark a couple of concurrency approaches and I'm hitting a bit of a wall with dispatch queues- it seems that beyond a certain point (about 5k blocks submitted consecutively in the same loop on My Machine™), async { ... } take significantly longer to return with every invocation. It's like beyond some high water mark it's doing some O(n^2) operation on every submission. However, putting the submitting thread to sleep momentarily gets rid of the problem. A trivial example:

let queue = DispatchQueue(label: "justTesting")
var counter = 0

let max = 1000000
for _ in 0..<max {
    queue.async {
        counter += 1
        if counter % 1000 == 0 {
            print("still going @\(counter)")
        }
    }
    // comment out to stall
    Thread.sleep(forTimeInterval: 0.0000001)
}

print("done queuing")

repeat {
    Thread.sleep(forTimeInterval: 0.1)
} while counter < max

print("done")

If something under the hood is mis-sized, how do I tune it?

2

It would be helpful to know in which environment you're seeing the problem.

3

Sure, sorry for the lack of context, full code is here. Am building with Swift 5.3 on macOS 10.15.7, uname -a gives Darwin Chriss-MacBook-Air.local 19.6.0 Darwin Kernel Version 19.6.0: Mon Aug 31 22:12:52 PDT 2020; root:xnu-6153.141.2~1/RELEASE_X86_64 x86_64. I haven't tried on Linux.

Slowdown happens in both debug and release configurations. I ran it through instruments quickly, calls to dispatch_event_loop_poke seem to creep up in frequency but I don't know if that's normal or not.

Apple Swift version 5.3 (swiftlang-1200.0.29.2 clang-1200.0.30.1)
Target: x86_64-apple-darwin19.6.0
4

Interesting. I thought you might be on Linux, there: I see no problem with the same version of macOS and the same compiler on a moderately-configured iMac 19,1.

5

The plan is to get this all running on linux but I'm falling at the first hurdle :blush: I'll have a fiddle tomorrow, am running this on a MbA 9,1 / i7 but I hardly think that's the issue.

Just tested it on Linux in release config and it doesn't stall. I guess since that's what I'm primarily targeting, I can live with it just running in a ci/cd pipeline for now...