Is @concurrent now the standard tool for shifting expensive synchronous work off the main actor?

[John_McCall]

DispatchQueue implements TaskExecutor.

[rayx (Huan Xiong)]

@jeremyabannister

The main thing that feels a little weird is marking zipUserData as async when in reality it has no suspension points.

It's because zipUserData itself is a suspension point. In my understanding async/await is required so that compiler can translate this:

    let result = await zipUserData(userData)
    print(result)

to something like:

    DispatchQueue.global(qos: .background).async {
        let result = zipUserData(userData)
        DispatchQueue.main.async {
            print(result)
        }
    }

[Andropov (Raúl Montón)]

@David_Smith

1. Modern computing devices are unbelievably powerful. Have you verified that there’s actually a problem? “Just do it synchronously” works remarkably often, and “async but on the main actor” covers a surprisingly large chunk of the rest
2. If there is a problem, have you used Instruments and related tools to profile and optimize the code? Fast is almost always superior to slow-but-async, for battery reasons among others
3. Having verified that you really do need to be async: do you need to be parallel? Phones have had 6 cores for quite a while, occupying one of them for a bit is no big deal
4. If you really do need to go 6-wide with long-running tasks, stick some yields in

While I agree on the point about premature optimization, you really don't want to block a Swift Concurrency thread for a long time ever. Experimentally, you might find that it doesn't immediately cause a problem, but try composing a few features like that and things will quickly break down. And if the workload is partially user defined, you may need to assume the worst and have it run on a background thread anyway.

As for parallelization: I hope in the future one can use Swift without Dispatch to run expensive, trivially parallel work and maximize its performance without leaving the cooperative thread pool unresponsive. Thinking, for example, about exporting a bunch of data: you want it done as soon as possible, and it may benefit greatly from parallelization, but you still want to be able to schedule other work in the thread pool.

Yielding could be an option, but for truly sensitive work yielding too much hurts performance and yielding too little will make the cooperative thread pool unresponsive in slower devices.

[nkbelov (Nikita Belov)]

Sorry, I was replying more generally towards the discussion of having dedicated threads for heavy compute while avoiding GCD too

I was suggesting that we effectively schedule (the otherwise-nonisolated work of) detached tasks on exactly that sort of executor

I don't think that this should be an implicit behavior, it kind of conflicts with the general idea that the concurrency thread pool is constrained to a known limit, and users creating tons of such detached tasks would undermine the idea. Echoing @David_Smith's reply, this probably should be a more controlled consideration, where it makes sense that users implement (or at least explicitly choose) their own executors to suite such specific needs — although there could well be a common library of such executor flavors so that it's not a chore.

[tera]

@Jon_Shier

No, the concurrency runtime is specifically engineered to avoid the thread explosion issue that could cause, unlike DispatchQueue.

DispatchQueue is capped to something like 512 threads to avoid thread explosion, no?

[John_McCall]

Building up to anywhere near 512 threads is effectively still thread explosion, but yes, there is also a cap on how many threads you can create in GCD.

[David_Smith]

@Andropov

Yielding could be an option, but for truly sensitive work yielding too much hurts performance and yielding too little will make the cooperative thread pool unresponsive in slower devices.

I would be very surprised if yielding was more expensive than having additional threads in general. Context switches aren’t free, kernel memory isn’t free, the scheduler decay curve takes into account the number of runnable threads (so your main thread priority will decay faster if many threads are waiting).

Like, we designed it this way for a reason. We genuinely do think not having extra threads hanging around is the right call for most situations.

It is true that picking how often to yield can be a bit tricky in some cases. We should make sure that code path in the runtime is as fast as possible to help make that choice easier.

[Andropov (Raúl Montón)]

@David_Smith

I would be very surprised if yielding was more expensive than having additional threads in general. Context switches aren’t free, kernel memory isn’t free, the scheduler decay curve takes into account the number of runnable threads (so your main thread priority will decay faster if many threads are waiting).

Like, we designed it this way for a reason. We genuinely do think not having extra threads hanging around is the right call for most situations.

Fair. I went back to the last project where I had faced this, and after a bit of profiling to ensure I was comparing apples to apples, the Swift Concurrency version of the code is actually slightly faster (by ~10%) than the GCD one if I pick a good yield frequency. Neat.

But it is possible to yield too much: a more conservative yield frequency doubles the execution time, and naively yielding on every iteration makes the code ~60x slower (of course, that's because it's yielding at the μs scale, but that's hardly apparent from the code alone).

I guess my point here is that GCD gives you a very nice behavior out of the box: you can create a concurrent queue, throw in a bunch of long running work, and have near-maximal performance without needing to worry about other work not being able to be scheduled.

I wish there was a tool in Swift Concurrency that allowed me to do just that. Task groups come close conceptually, but its simplest usage (no yielding, or yielding all the time) comes with huge caveats.

Perhaps something as simple as having a low-overhead way of yielding conditionally if a task has been running for a while would suffice. IDK.

[David_Smith]

@Andropov

Perhaps something as simple as having a low-overhead way of yielding conditionally if a task has been running for a while would suffice. IDK

Neat idea. Seems worth exploring. Another thought I had was that yielding when there’s no pending work might be optimizable; stick a flag in an inline TSD slot for “is there pending work?” or something.

Haven’t looked into it though. Also not sure how often it’s relevant (ie how often there’s a frequently yielding task and no pending work at the same time)

[hassila (Joakim Hassila)]

If it was cheaply possible to yield if there is pending work might be nice on the surface, but would have the risk of ping-ponging between tasks quite easily and might be a footgun.

We use the pattern in subtasks:

iterations += 1
if iterations.isMultiple(of: 10_000) { // some constant to amortise the overhead across multiple iterations
  try Task.checkCancellation() // and/or yield
}

But it would be nice to be able to bake this into an iterator really - an automatically yieldable/canceallable iterator that does it at some reasonable frequency (or after X time passed)?

Most of our use cases involves an iterator processing some large amount of data, so this might be a nice way to make that automatic.

[asdf_bro]

Swift or Dispatch needs built-in support for this pattern:

let p1 = DispatchQueue(width: 4)
let p2 = DispatchQueue(width: 4)
let p3 = DispatchQueue(width: 4)

Task.detached(executorPreference: p1) { ... }
Task.detached(executorPreference: p2) { ... }
Task.detached(executorPreference: p3) { ... }

I call this pattern “islands of cooperation in a sea of preemption”. It recognizes that preemption is expensive (the thread count is bounded), but also that preemption is sometimes necessary (the jobs on p1 don’t cooperate well with the jobs on p2, p3, or the default pool).

[tera]

@asdf_bro

let p1 = DispatchQueue(width: 4)

OperationQueue's maxConcurrentOperationCount has this superpower.

[robert.ryan]

@Andropov

I guess my point here is that GCD gives you a very nice behavior out of the box: you can create a concurrent queue, throw in a bunch of long running work, and have near-maximal performance without needing to worry about other work not being able to be scheduled.

The problem with using a concurrent queue “out of the box” is that it doesn’t really handle the thread-explosion scenario well. So, it comes down to your definition of “a bunch of long running work”. If just a few items, a concurrent queue is fine. But if you’ve got a lot, you might want to be careful.

The DispatchQueue.concurrentPerform is the “go to” solution for GCD thread-explosion scenarios (often married with striding, to optimize how much work is done on each thread). You can then bridge this back to Swift concurrency with a continuation.

That avoids blocking the cooperative thread pool, mitigates thread explosion, etc.

But the OP’s situation is obviously much simpler, not requiring any of this…