[Pitch 3] Custom Main and Global Executors

Evolution Pitches
1

This is a third pitch for the ability to write custom default executors (both the main executor, as used for the main actor, and the global executor, used by default for Swift Task s). You can find the first pitch and the second pitch here.

The draft Custom Main and Global Executors PR has been updated with the following changes following feedback from the second pitch:

  • Clocks no longer support Instant or Duration conversion.
  • Instead, Clock has grown run(_ job:at instant:tolerance:clock:) and enqueue(_ job:on executor:at instant:tolerance:) methods.
    • These allow an executor that finds itself asked to schedule a job using a Clock it doesn't recognise to hand the problem off to that Clock, the idea being that the Clock will know how to do the right thing.
  • Expose the built-in Unimplemented(Main|Task)Executor implementations.
  • Update the text surrounding the task allocator and executor private data to indicate that these facilities are only for use until the point at which a Job is executed. Executors much release any resources prior to that point, and in the case of the task allocator, must do so in strict reverse order of allocation.
  • Add a discussion of the "magic typealias"-based mechanism for selecting the default executors.
13 Likes
2

Thanks for the update @al45tair!

This looks really good, I think especially the way this redesigns the enqueues is very promising and I think this can work well enough -- even with clocks one would not recognize, we can then forward to the clock's enqueue etc.

The exposed executor types also sounds good to me. And we're finally giving a real swift type to the previously-known-as "the default global concurrent executor (which happens to be dispatch based)" with the DispatchGlobalTaskExecutor type :slight_smile:

AFAICS this addresses all previous discussion points quite nicely. Thanks for the docs updates as well on the allocator, they read nice now. We could maybe make it even more scary but I don't necessarily want to promise how we'll crash if you do something wrong there...

Let's see if anyone else has some points here but otherwise I think we could get this up for an official review soon enough :slight_smile:

2 Likes
3

I just gave the proposal another review and in general this looks great. I really like the changes to how clocks are handled. This makes the implementation of SchedulingExecutors a lot more straight forward. I only have a few comments left:

Executors that are both RunLoopExecutor and TaskExecutor

Some executors might want to conform to both RunLoopExecutor and TaskExecutor. What is the expectation for how this should work? In particular, should such an executor execute jobs without the run method being called or is the expectation that for RunLoopExecutors the run method must be called so that the executor processes any job?

Dispatch and CF executors

The proposal doesn't specify this but in what module are those executor implementations living? My understanding is that those should be coming from Dispatch and Foundation respectively and not the Concurrency module otherwise we are adding a hard dependency on those two modules from Concurrency. It ought to be possible to avoid linking Dispatch and Foundation in the future while still having a concurrency runtime with a custom executor factory, right?

CooperativeExecutor

Is this the defaultExecutor used by the Concurrency runtime by default? The documentation makes it seem like this is only a single thread so there must be another executor that is a pool of CooperativeExecutors right?

Final classes

Should the following classes be final?

class CooperativeExecutor
class UnimplementedMainExecutor
class UnimplementedTaskExecutor

Naming nits

public func withUnsafeExecutorPrivateData<R, E>(body: (UnsafeMutableRawBufferPointer) throws(E) -> R) throws(E) -> R

Can we give the generic types here proper names such as withUnsafeExecutorPrivateData<Return, Failure: Error>?

Does DispatchGlobalTaskExecutor need the Global in the name or would DispatchTaskExecutor be enough? Or is this name coming from the global queue that's used under the hood?

2 Likes
4

I think for executors that do that, the exact semantics should be specified by the executor implementation. Some executors of that variety may require that the run method be called before they are usable as a TaskExecutor — this is true for CooperativeExecutor, for instance, and I would expect that that was the norm here, but I can imagine that there might be executors for which that wasn't true.

They're in the Concurrency module itself; they have to be because they are the default executor implementations and we can't really move them to Dispatch or Foundation. You make a good point about the dependency issue — let's just not expose them directly (which will fix the problem, since we can then remove them without difficulty).

Normally the Dispatch-based executor (or its CF counterpart) will be the default; the CooperativeExecutor might be the default for some embedded use-cases — and it is only ever single-threaded, by design. It's essentially a replacement for some C++ code that implemented something very similar.

Yes.

Using R and E here is the style used in the standard library code; we're just matching that.

Since we're making it non-public, I think we can ignore the Global :-) But yes, it explicitly uses the global queue.

1 Like
5

FWIW, I've just pushed an update to the proposal with some of the above.

1 Like
6

Thanks for making those changes. I’m really liking the look of those proposal now. With the change to allow protocols inherited by @main types to provide default executor factories, I’m a big +1. That makes a huge difference for libraries such as SwiftCrossUI which aim to provide correct behaviour across all platforms by default (i.e. without requiring users to set custom executors on specific platforms etc).

2 Likes
7

Agreed. I think it's a good decision to make them non-public. I was also thinking similarly that the proposed CooperativeExecutor shouldn't be part of this proposal and is potentially better located in a swift-executors package where can offer more executors for different use-cases.

8

While I think we still need the implementation because of the SWIFT_CONCURRENCY_GLOBAL_EXECUTOR CMake setting (if it's set to singlethreaded, then CooperativeExecutor will be the default executor), I think you're right that it might be better to not expose this one here either, then we can add something similar to a separate package. I'll make some updates to that effect.

1 Like
9

My library for Swift-Node.js interop, node-swift, has an issue where Node's libuv runloop starves the main thread, preventing MainActor tasks from running. I have a WIP solution that uses GCD SPI but the pitched API would provide a more stable solution. Big fan.

I do have a couple questions regarding gaps in the current feature set. I imagine much of this will be out of scope of this pitch but would be good to know whether it's something that might be addressed in the language in the future.

Polling

An important function of application run loops is to provide polling functionality — after all, run loop implementations (including CoreFoundation, GLib, and libuv) spend most of their time blocking on a select/poll call or similar.

The proposed SchedulingExecutor is a good solution for time-based events, but it would be great to see a similar API that allows executors to watch a file descriptor/Win32 handle/Mach port. Perhaps something like this?

protocol PollingExecutor: Executor {
  #if os(Windows)
  typealias PollTarget = HANDLE
  #elseif canImport(Darwin)
  typealias PollTarget = mach_port_t
  #else
  typealias PollTarget = CInt // file descriptor
  #endif

  func enqueue(
    _ job: consuming ExecutorJob,
    whenReadable target: PollTarget
  )
}

(aside: it would be nicer if we could use System.FileDescriptor and System.Mach.Port here for the unix cases but I imagine that's not possible due to dependency inversion. Also, if we aren't comfortable using Mach ports in public APIs it's possible to wrap the port in a file descriptor on the runtime side with EVFILT_MACHPORT.)

Looking ahead, this would be very useful for implementing APIs for async IO without a direct GCD/CF dependency. It could also be very useful if you do want to support GCD/CF, which brings me to…

Handling GCD and CF sources

I imagine a custom main executor would result in lower level CFRunLoop/DispatchQueue ports/sources/blocks not being serviced. It would be great to see this problem addressed in the current pitch at least for common cases like DispatchQueue.main.async, which shows up in tons of Swift code today. Perhaps that particular case could be hard-coded to forward to the MainExecutor?

This is also especially relevant on Darwin where a lot of system libraries (all the way from Foundation to SwiftUI) count on the existence of a main CFRunLoop/DispatchQueue. Further complicating matters, many of those instances will probably not be covered by special-casing DispatchQueue.main.async, whether that's because they use the IO features or more esoteric things like RunLoop modes. This is another instance where PollingExecutor could provide a solution. Specifically, if CF could wrap the RunLoop's __CFPortSet in a file descriptor/[HANDLE] and pass it to the PollingExecutor, we'd be able to have our cake and eat it too: a custom executor that's also able to service CoreFoundation and libdispatch!

Side note: my suggestion is motivated by the fact that this is exactly how GCD plugs into CF, i.e. GCD gives CF 1) a handle that it can poll and 2) a function it can call to wake GCD back up. This is the GCD SPI I mentioned at the top of my post.

10

Indeed, this is something we're thinking about, but it won't form part of this particular Swift Evolution proposal.

It's also the case that the fact that run loops handle both scheduling and asynchronous I/O is something of an implementation detail, and is mainly the case for UNIX-style select()/poll()/epoll()/kqueue() based I/O. Other asynchronous I/O mechanisms like POSIX aio, Windows "overlapped" I/O and to some extent io_uring work somewhat differently and don't necessarily require a run loop to operate.

Additionally, Swift's standard library doesn't really provide any synchronous I/O of its own, which makes it a little odd plumbing asynchronous I/O into Swift Concurrency without also thinking about the synchronous case. As such, I think we want to think carefully about exactly what kind of I/O abstraction(s) we want to provide and how.

You are correct, and an earlier version of this proposal did include a general-purpose event mechanism that I had intended to use to address this problem, however it proved tricky to get the implementation exactly right and since I wasn't proposing to adopt its use immediately in Dispatch I decided to drop it for now.

Again, this is something we're thinking about, but it won't be a part of this proposal. For now, programs that use custom main executors will need to use Swift Concurrency rather than Dispatch if they expect to be able to execute code on the main executor.

3 Likes
11

Ah, that’s probably a little bit of a deal breaker for SwiftCrossUI’s use case then. I would imagine that many Swift libraries that people would like to use in consumer apps use Dispatch under the hood for multi threading, as it’s not a hard requirement to drop Dispatch to support Swift 6 mode.

I’ll probably stick with my interim solution for now, which involves servicing the main run loop occasionally from Gtk timeouts. It guarantees that all Swift threading/concurrency mechanisms that consumers could use just work (afaik).

12

@tonyschr Copied from the old pitch thread.

13

Yes, that's what I would imagine.

Typically I would expect that you would (a) send some kind of notification to the message pump when a job is enqueued by your executor, so that the message pump wakes up, and (b) register somehow with the message pump to be called back at some point during its loop (one option here is to hang off whatever idle handler exists on the message pump).

I'm not familiar with Chromium so I can't give specifics, but that's broadly what I'd expect.

14

Hi, I'm Tony working with @stevenbrix and others at The Browser Company and am interested in understanding the proposal better and figuring out some near-term ways to work towards a solution.

Our scenario is that we use Chromium's message pump on Windows for our main UI thread, which in addition to dispatching its own tasks has battle-tested logic for managing lots of edge cases involving input.

The scenario is similar to Android, as @Geordie_J has talked about, so I understand that our custom executor's run method would synchronously call our hostedChromium.run(), which is then responsible for processing everything on the main thread until it's told to stop.

Given this, I see two core requirements:

  1. A way of knowing that there's pending work that only the RunLoop knows how to process.
  2. A way to process the pending work in the RunLoop without the risk of it blocking, and without side-effects such as dispatching input messages or throwing away messages it doesn't know about.

Currently we're using heuristics for #1 and limitDate for #2.

There's an immediate need and I've been investigating all the way down into dispatch!_dispatch_runloop_queue_class_poke, to see what's available to hook off of in a clean, performant way and provide an event/callback off of Foundation that piggybacks on this or similar.

Additionally, for the idea of introducing an API similar to limitDate that doesn't generically call TranslateMessage/DispatchMessage on Windows, I noticed in CFRunLoop.c there's rlm->_msgPump(), set by _CFRunLoopSetWindowsMessageQueueHandler, though it doesn't seem to be used and for our purposes it seems fine just to leave messages in the queue.

Since I believe these two things are needed, and I'm sure people in this thread have ideas about the implementation and shape of the APIs, I'd like to better understand what we can do now to unblock work and then become an early adopter of the long-term solution. Several people on the team already actively contribute to the Swift toolchain, so we're up for making the changes and getting them vetted.

2 Likes
15

Thanks, and apologies for waking up the old thread. Reading through the newer posts by @stackotter and @kabiroberai, and your responses, sort-of answers my question.

I think I see how this will work end-to-end with Swift Concurrency after this work is done. We're still looking forward to be an early adopter.

In the meantime, and for legacy code and libraries that rely on things like DispatchQueue.main.async, we were really hoping to find a solution cleaner than the heuristics & limitDate solution we're using right now.

I was hoping that there would be a stepping stone, or Least Bad way of dealing with those scenarios, informed by the work being done here. Maybe it's best for those in the same boat to collaborate on a separate thread to avoid getting too off-topic here.

2 Likes
16

This is something we're actively thinking about — it's clear that we need a solution that lets Dispatch.main.async continue to work with a custom main executor; it just isn't going to land in this first pass. The problem here is that Dispatch and Core Foundation have a private interface that they use to make this work for CFRunLoop, but that interface is system specific and isn't suitable for use as an API.

I'd like to add a generic mechanism to allow things to integrate with the main executor, then we can have Dispatch use that mechanism instead when there's a custom main executor. I had some ideas in a previous version of the pitch, but I wasn't happy with the way the API surface turned out, and I also wasn't proposing to patch Dispatch at the same time anyway, so I'd removed it from this proposal.

Of course, ideally, people would replace use of Dispatch.main.async with Swift Concurrency, thus avoiding the whole problem :-) (I realise this is not always practical, though.)

2 Likes