DispatchQueue behaviour under load

Hey all, I've been trying to benchmark a couple of concurrency approaches and I'm hitting a bit of a wall with dispatch queues- it seems that beyond a certain point (about 5k blocks submitted consecutively in the same loop on My Machine™), async { ... } take significantly longer to return with every invocation. It's like beyond some high water mark it's doing some O(n^2) operation on every submission. However, putting the submitting thread to sleep momentarily gets rid of the problem. A trivial example:

let queue = DispatchQueue(label: "justTesting")
var counter = 0

let max = 1000000
for _ in 0..<max {
    queue.async {
        counter += 1
        if counter % 1000 == 0 {
            print("still going @\(counter)")
        }
    }
    // comment out to stall
    Thread.sleep(forTimeInterval: 0.0000001)
}

print("done queuing")

repeat {
    Thread.sleep(forTimeInterval: 0.1)
} while counter < max

print("done")

If something under the hood is mis-sized, how do I tune it?

It would be helpful to know in which environment you're seeing the problem.

Sure, sorry for the lack of context, full code is here. Am building with Swift 5.3 on macOS 10.15.7, uname -a gives Darwin Chriss-MacBook-Air.local 19.6.0 Darwin Kernel Version 19.6.0: Mon Aug 31 22:12:52 PDT 2020; root:xnu-6153.141.2~1/RELEASE_X86_64 x86_64. I haven't tried on Linux.

Slowdown happens in both debug and release configurations. I ran it through instruments quickly, calls to dispatch_event_loop_poke seem to creep up in frequency but I don't know if that's normal or not.

Apple Swift version 5.3 (swiftlang-1200.0.29.2 clang-1200.0.30.1)
Target: x86_64-apple-darwin19.6.0

Interesting. I thought you might be on Linux, there: I see no problem with the same version of macOS and the same compiler on a moderately-configured iMac 19,1.

The plan is to get this all running on linux but I'm falling at the first hurdle :blush: I'll have a fiddle tomorrow, am running this on a MbA 9,1 / i7 but I hardly think that's the issue.

Just tested it on Linux in release config and it doesn't stall. I guess since that's what I'm primarily targeting, I can live with it just running in a ci/cd pipeline for now...

Terms of Service

Privacy Policy

Cookie Policy