Meet SwiftCommand, a new Swift package that makes creating child processes very easy!

Community Showcase
1

Recently, I published version 1.0.0 (and a few versions containing bug-fixes) of the new SwiftCommand package. It makes executing command line programs and handling their I/O from your Swift applications very easy. This is achieved by wrapping Foundation.Process in an API that is inspired by Rust's std::process::Command.

Here is a little example that shows how you would describe the terminal command

cat < SomeFile.txt | grep Ba

using SwiftCommand:

import SwiftCommand

let catProcess = try Command.findInPath(withName: "cat")!
                            .setStdin(.read(fromFile: "SomeFile.txt"))
                            .setStdout(.pipe)
                            .spawn()

let grepProcess = try Command.findInPath(withName: "grep")!
                             .addArgument("Ba")
                             .setStdin(.pipe(from: catProcess.stdout))
                             .setStdout(.pipe)
                             .spawn()

for try await line in grepProcess.stdout.lines {
    print(line)
}
// Assuming the file 'SomeFile.txt' contains the following three lines:
// Foo
// Bar
// Baz
// This loop prints 'Bar' and 'Baz'.

try catProcess.wait()
try grepProcess.wait()
// Ensure the processes are terminated before exiting the parent process

SwiftCommand is tested using Swift 5.6 (and Swift 5.7, which is currently in beta) on macOS, as well as on multiple Linux distros.

Further information can be found either on the package's GitHub repository or on its Swift Package Index Page, where a complete DocC-documentation can be found as well.

18 Likes
2

This looks incredibly useful.

1 Like
3

Thanks!
When I'm trying to get things done in Swift, I'm often complaining about the lack of nice Frameworks for simple tasks like that which necessitates to fall back to using the often unswifty APIs in Foundation & Co...

But of course the real solution is to stop complaining and start implementing a nice package so that I and everyone else can use that instead for the next time. And that's exactly what I did ;)

3 Likes
4

See also ShellOut and Sundell's other work. I believe Paul Hudson also had a framework for more script-like Swift to help with this.

2 Likes
5

Yeah, ShellOut is indeed a nice and simple package, however, it is a bit too simple for my use cases, as in it doesn’t support everything that I need it to do.
Additionally, it’s a bit dated by now (the last version release was 2½ years ago) and therefore doesn’t support features like e.g. async/await.

7 Likes
6

Very cool, congrats on releasing this! I felt I should do a write up of some problems I ran into while building something similar, which may cause problems in your implementation depending on how it is used.

It looks like you are using Process to actually launch the child process. You should be aware that there is currently a bug on Linux where if you are launching processes concurrently, you may unintentionally leak file descriptors into the child process. This can cause problems (including deadlock) if a process depends on its stdin to be closed to detect completion, but the stdin file descriptor is leaked into another process (such as in a context-switch immediately before this line).

The issue is that the following code in Foundation is not atomic, and file descriptors may be opened in a context switch after findMaximumOpenFD() returns but before the process executes (meaning the corresponding AddClose won't be called and the file descriptor will remain open).

In my project, GitHub - GeorgeLyon/Shwift: Shell scripting in Swift, I work around this by providing a custom implementation of spawning a child process (found here, feel free to steal). I also have a test which can somewhat reliably detect when this causes a deadlock.

A second thing to note is that you read the output data all at once, which may be problematic for large inputs. Shwift uses swift-nio to process input as it becomes available. This functionality is tested here: Shwift/Main.swift at 46ea4106ddd9b8ce557c95090ca31d5139d3641a · GeorgeLyon/Shwift · GitHub

5 Likes
7

Thanks for your detailed description of the problems I could (or probably already have) run into!

I don't know if this has anything to do with a problem I actually already had to deal with, but I encountered a strange bug (?) that only occurred in the XCTestCases on Linux: the spawned processes strangely do not respond to SIGINT or SIGTERM (not even if sent from the command line using kill -SIGINT/kill -SIGTERM). In the end I had to exclude the lines from the test on linux. The strange thing about that is that the exact same code from the test case works completely fine when copied into a normal executable target.

But, at least for now, everything else I tried to terminate a child process (including closing the stdin of a cat) seemed to work just fine on Linux.

No, I don't do that. Or at least, it's not the only way to read output data. If you spawn a child process, you can at any point call either availableData or read(upToCount:) on the stdout or stderr handle. Also, the lines and characters asynchronous sequences on ChildProcess.OutputHandle are built on top of (a custom implementation of) AsyncBytes, which doesn't read all the data at once either. So there are at least some methods to read large amounts of data in chunks, that can be chosen if just calling waitForOutput() on a Command doesn't work anymore...

1 Like
8

I really like your approach to make the async streams for files available on other platforms. It annoys me that this is only macos. Thanks a lot for porting the async lines!

However I noticed the linux build got broken in release 1.1.2. I made a PR to fix the small issue with a missing #else. @Zollerboy1 could you merge that so I could use it in our code base? Would be a grate help!

9

Thanks for pointing out this issue to me!

It should be fixed in version 1.2.0 now.

1 Like