Calling `swift_task_asyncMainDrainQueue` by hand to reproduce async `main` behavior?

Using Swift
1

I'm writing a small command line tool that does two things, and the second thing is async, so I'd like to write this:

@main struct Main {
  static func main() async {
    doThing1()
    await doThing2()
  }
}

Unfortunately for me, doThing1() breaks for reasons that I don't fully understand when it's called from an async main(), but works fine when called from a non-async main(). I don't have the source code to doThing1() nor can I modify it in any way. [1]

So to make these two things work nicely together, I want to do essentially this:

static func main() {  // not async!
  doThing1()  // Now this is happy
  Task { @MainActor in
    await doThing2()
  }
  // TODO: Wait until the task finishes,
  // but I can't block the main actor!
}

I've tried the various unsafe and unrecommended tricks like using a semaphore or spinning a run loop here but—as expected—they cause deadlock if doThing2() tries to queue up work on the main actor. So looking at what the compiler codegens for async main(), I tried this instead:

static func main() {  // not async!
  doThing1()  // Now this is happy
  Task { @MainActor in
    await doThing2()
    exit(0)
  }
  _asyncMainDrainQueue()
}
@_silgen_name("swift_task_asyncMainDrainQueue")
func _asyncMainDrainQueue() -> Never

This works in my testing, but is it unsafe in any way that I'm missing? Is there a better way to do what I need here (maybe one that isn't using a low-level ABI)?

  1. The function is XCTestSuite.default.run(), which I use to manually run test suites in a standard executable rather than an .xctest bundle. If main is async and XCTest tries to run an async test method, swift::StackAllocator::dealloc() aborts with "freed pointer was not the last allocation". I suspect that XCTest internally does some shenanigans to run @MainActor async test methods and safely wait for them to complete even though it gets called from the main actor and this is interfering with async main() somehow. ↩︎

2

CC @fabianfett This is interesting for Lambda too I think.

3

Yeah, that's pretty much what the async main entry does.
The asyncMainDrainQueue is just there to start the main runloop, so if you're using Dispatch/Foundation on Apple platforms at least, you could call RunLoop.main.run(). Alternatively, dispatch_main() or CFRunLoopRun() are also available. If you've hooked the concurrency runtime and have a custom main executor, then you'll want to call the thing that kicks it off instead.

Yeah, a big part of the problem here is that the runloop isn't running yet, so you've enqueued a task on the queue, but nothing is going to dequeue it and actually run the code yet. To do that (and adhere to the rules of the main actor), you'd need the main thread to dequeue it and run the body of that task, which is one of the main jobs of the runloop.

Edit: Whether it's safe or not, it is ABI, so there are limits to how much it can change. It would probably be best to understand what's going on with XCTest though. CC @grynspan in case you have any ideas.

1 Like
4

Thanks for confirming! I wasn't really worried about the behavior changing since it's stable ABI, more than I might be leaving some other detail out before using it.

I'll try to file a feedback with a small XCTest reproducer soon (it only affects the Darwin implementation, not the open-source one), but my current priority is to get the thing working first. :slightly_smiling_face:

1 Like
5

XCTest on Darwin spins the main run loop automatically, but there are edge cases where it's spinning the main run loop but the main dispatch queue has pending work that's never getting drained. Unfortunately, because XCTest is primarily a synchronous, Objective-C API, these edge cases aren't easy to resolve (and may be impossible due to the layering between CFRunLoop and dispatch_queue_t.)

Please do file feedback with Apple if you have a reproducible case. (It may end up being marked as a duplicate of an existing issue, but that doesn't mean it's being ignored.)

2 Likes
6

I've filed FB14867171 with a small reproducer.

7

Thanks! We'll track it internally.

1 Like