Swift Concurrency in non-eventloop contexts (e.g. CLI tools)

Using Swift
1

I've been playing with concurrency on a CLI tool, and what I've found is that it's somewhat easy to go from sync contexts to async by using the async or detached functions, but it's a bit hard to go back from async to sync.

What I have so far is:

struct MainApp: ParsableCommand {
    func run() throws {
        detach {
            do {
                try await runAsync()
                Darwin.exit(0)
            } catch {
                print("Error:", error)
                Darwin.exit(1)
            }
        }

        dispatchMain()
    }

    func runAsync() async throws {
        // Do stuff in async
    }
}

I tried using a semaphore to block but that had issues when playing with @MainActor. It feels like if the generated task had a way to do try task.wait() from a sync context this might be easier. And indeed the fact that you can await a task and get the returned value makes it look really similar to a future/promise, so I'm wondering if by adding wait we could get lightweight future-like support (which is not necessarily a goal of Swift concurrency, but could easy the semantic understanding of tasks as futures-like).

I'm not super familiar with the internal details, and maybe what I'm asking is not safe nor easy, but I'd be fine if it was called unsafeWait or something, just so that I could more easily use async/await on CLI tools.

2

The language now allows for a @main type to provide a top-level main function that's async, and the runtime will take care of keeping the process alive long enough for the main task to complete. Once swift-argument-parser adopts async, it should be possible to write:

struct MainApp: ParsableCommand {
    func run() async throws {
        // do stuff in async
    }
}

without any work on your own.

4 Likes
3

What i would do if i were to create cli tools that also would use threads and async calls:

Split in two main parts: frontend and backend.
The frontend is synchronous and the backend is split in workers for each tool action available. You can have a state machine in the middle to help you with the dispatch of the actions from the frontend to the backend.

Those backend actions can run on their own threads and would signal the backend when they are finished which would in turn signal its delegate (the frontend) with the result.

The frontend would then decide if it's time to leave, when every action returned, and if so just break out the main loop.

Here is the thing: your frontend manager/scheduler would simply run a runloop and have a way to stop it when all the actions that should run executed and returned in a async way with the result.

With this you don't need to block any thread and you still can use a multi-thread program in a cli setting where the ultimate response to the user must be synchronous.

I don't know why you would need something that fancy like this, but if you need a cli tool that deals with complicated IO like the network, something like this might be needed.

There are times that we need to really block on some thread, but they should be a last resort, giving they can be avoided most of the time by rethinking the architecture of our applications.

Note: i've barely touched the Swift actors, so i dont know anything more specific, but this have the advantage to work in other places too.

4

Ah great, I'll wait for swift-argument-parser to adopt it then.

But swift-argument-parser still needs MainApp.main() to be invoked, and it should handle the awaiting of the main task through the mechanism I'm asking about in this thread, no? i.e. swift-argument-parser doesn't use @main afaict.

And also, while @main would solve this by allowing the async version of main, it still can't be used in SPM executable targets, since it's incompatible with main.swift, are there plans for that to be fixed?

5

that might be true for cli tools with larger scopes, but if I just want to download a json file and decode it to display some data, I should be able to use async/await to simplify the code I'm writing, without resorting to a massive architectural design with frontend and backend code

1 Like
6

Apparently this will already be backed in the language giving the other answer, and you can wait for it which is nice.

But if you cant wait and its more a temporary thing with just one case, using the runloop while waiting for the result still applies, once the result is there just ping it back and break the main runloop/message loop. The good thing is that you are not blocking the main thread.

But i agree with you, giving this opiniated threaded design path was taken, the language should simply provide a way to do it.

7

The Swift runtime uses its own mechanism to manage the lifetime of the process. Unfortunately, main.swift does not yet support async code, but I would generally recommend using an @main type with an async main function, instead of trying to do your own synchronization. I believe swiftpm should support executables that use @main instead of main.swift now.

1 Like
8

Ahh, perfect, wasn't aware that main.swift limitation was resolved. Updated my prototype with this new info in Add support for async/await where available · Issue #326 · apple/swift-argument-parser · GitHub

1 Like