Concurrency runtime main executor hook behaviour

Using Swift
1

I have been toying with the Concurrency runtime hooks to see if I can replace the default global concurrent executor with a custom implementation, which might not be a good idea but a great learning experience nonetheless.

I succeeded with this endeavour and then I decided to try to take over the MainExecutor which was still using the Dispatch main queue and not my own "main queue". I tried hooking the swift_task_enqueueMainExecutor_hook but it never seems to be called.

For example the following code will never print enqueueMainExecutor but enqueueGlobal is printed a bunch of times as expected

swift_task_enqueueGlobal_hook = { job, original in 
    print("enqueueGlobal")
    original(job)
}

swift_task_enqueueMainExecutor_hook = { job, original in 
    print("enqueueMainExecutor")
    original(job)
}

@MainActor
func someMainActorFunc() async {
    print("We are running on the main actor?")
}

Task { @MainActor in 
    print("Main actor task 1")
}

Task.detached { @MainActor in
    print("Main actor task 2")
}

Task.detached {
    print("Non main actor task")
}

await Task.detached {
    await someMainActorFunc()
    await withTaskGroup(of: Void.self) { group in 
        for _ in 0..<5 {
            group.addTask {
                try! await Task.sleep(nanoseconds: 1_000_000_000)
            }
        }
    }
    print("Done")
}.value

Is the above behaviour expected? I would expect that at some point the jobs would need to be enqueued to/through the MainExecutor but it seems that they are enqueued straight to the main dispatch queue.

1 Like
2

@MarSe32m I am experiencing exactly the same issue, on both Mac and Android.

@John_McCall FYI, since I mentioned the same thing here

The only way I can think of debugging this is to build Swift and put logs everywhere to try to understand the data flow, but
a) the last time I tried to build Swift it took days of my time and I didn't get it working in the end, and
b) surely there's a better way?

@kateinoigakukun you apparently got this working for SwiftWasm with the cooperative global executor, do you have any ideas why it wouldn't work when using the DispatchGlobalExecutor?

edit: Actually, since Wasm only has a single thread anyway, everything can be enqueued via swift_task_enqueueGlobal_hook. I have a hunch that swift_task_enqueueMainExecutor_hook is not actually called there either, currently. Which seems especially likely given that the global hook, as written, hooks all jobs and does not forward them to the original impl.

3

I filed an issue about this.

You can kinda workaround this by implementing your "own" MainActor like this:

@_silgen_name("swift_task_enqueueMainExecutor")
func _enqueueMain(_ job: UnownedJob)

@globalActor
final actor MyMainActor: GlobalActor, SerialExecutor {
    static let shared: MyMainActor = MyMainActor()
    
    nonisolated var unownedExecutor: UnownedSerialExecutor {
        asUnownedSerialExecutor()
    }

    nonisolated func enqueue(_ job: UnownedJob) {
        _enqueueMain(job)
    }
    
    nonisolated func asUnownedSerialExecutor() -> UnownedSerialExecutor {
        UnownedSerialExecutor(ordinary: self)
    }
}

and then use that instead of the MainActor defined in the standard library. This is of course not very plausible nor portable but it will get the job done. With this actor I can see the main executor enqueue hook being called.

But I did some digging and might have found something (now I'm a hobbyist and not a compiler / runtime engineer so I might be totally off the track). The current implementation of the MainActor can be found here. So when someone wants to run some task on the MainActor, they will implicitly use the executor associated with the MainActor which is obtained via a builtin

@inlinable
  public nonisolated var unownedExecutor: UnownedSerialExecutor {
    #if compiler(>=5.5) && $BuiltinBuildMainExecutor
    return UnownedSerialExecutor(Builtin.buildMainActorExecutorRef())
    #else
    fatalError("Swift compiler is incompatible with this SDK version")
    #endif
  }

This builtin (implementation) calls the swift_task_getMainExecutor (or emits a call to, I don't know the parlance regarding this), which returns an ExecutorRef with a SerialExecutorWitnessTable. The witness table is returned from _swift_task_getDispatchQueueSerialExecutorWitnessTable which references the DispatchQueueShim which is an implementation of the SerialExecutor protocol. From the implementation we can see that this executor enqueues the jobs directly to dispatch queues

func enqueue(_ job: UnownedJob) {
     _enqueueOnDispatchQueue(job, queue: self) 
}

since _enqueueOnDispatchQueue is implemented in the runtime as

void swift::swift_task_enqueueOnDispatchQueue(Job *job,
                                              HeapObject *_queue) {
  JobPriority priority = job->getPriority();
  auto queue = reinterpret_cast<dispatch_queue_t>(_queue);
  dispatchEnqueue(queue, job, (dispatch_qos_class_t)priority, queue);
}

So if I have understood the process correctly, when someone calls a @MainActor function, the executor is fetched with the builtin as described above. This executor is basically the main dispatch queue and the job is enqueued directly to it, bypassing the swift_task_enqueueMainExecutor method.

I don't know the reason why it is implemented this way, maybe for performance reasons, but this way the jobs enqueued to the MainActor will never go through the swift_task_enqueueMainExecutor which in turn will never call the swift_task_enqueueMainExecutor_hook.

At some point in time the implementation of the MainActor was like this

@globalActor public final actor MainActor: SerialExecutor, GlobalActor {
   public static let shared = MainActor()

   @inlinable
   public nonisolated var unownedExecutor: UnownedSerialExecutor {
     return asUnownedSerialExecutor()
   }

   @inlinable
   public nonisolated func asUnownedSerialExecutor() -> UnownedSerialExecutor {
     return UnownedSerialExecutor(ordinary: self)
   }

   @inlinable
   public nonisolated func enqueue(_ job: UnownedJob) {
     _enqueueOnMain(job)
   }
 }

and with this implementation the hook is called.

1 Like
4

That's really interesting, thanks @MarSe32m!

I also made a bit of progress and noticed (as I'm sure you did) that even MainActor tasks are enqueued via swift_task_enqueueGlobal "only".

What I also found with your workaround is that tasks are enqueued via swift_task_enqueueGlobal first, and then forwarded on to swift_task_enqueueMainExecutor internally. Which makes me think it shouldn't be essential for the enqueueMain hook to work if we can find out from the job whether it's destined for the MainActor or not.

edit: I played around trying to do that for a while and hit a bunch of dead ends. Basically what I was trying to do was to compare swift_task_getMainExecutor with the result of executorForEnqueuedJob to see whether the job is destined for the main thread.

Unfortunately, executorForEnqueuedJob is not emitted anywhere (it's static), so I don't think it's possible to call it from my external code as I attempted to. And swift_task_getMainExecutor returns an ExecutorRef, which is a type I couldn't easily define or import either. If both of those were working, we could continue that path of experimentation.

That said, I'm not even sure if executorForEnqueuedJob would work properly on a job that hasn't actually been enqueued yet. Indeed, it seems the relevant private flags that set the destination queue are not set by the time the job reaches swift_task_enqueueGlobal, so maybe the above approach wouldn't even work. In general, I'd rather not base a "solution" on a bunch of hacks and internals which are subject to change.

Instead, I'm just going to wait until more of these APIs become public and more refined before playing around with this any longer. I am hoping that – with this thread and others – @John_McCall, @Douglas_Gregor and the team have heard the call that we need some way of hooking into enqueuing on main that works reliably. And will leave this in their capable hands.

For now, for our particular case of needing to use @MainActor without Foundation, we are able to use the private _dispatch_main_queue_callback_4CF to drain the Dispatch main queue in a cooperative manner, i.e. without taking control over the main thread. I would love to "do it properly" on Android soon though, and also remove our use of Dispatch entirely.

1 Like