Trouble reading new data from standard output pipe

Using Swift
,
1

Hi all,

I am writing a MacOS app that makes use of the Android Debug Bridge (ADB) tool. To achieve this I am using a Process to run an ADB executable.

My issue is that when I run a command, I don't see anything come through the standardOutput of the Process until the command finishes.

For example, if I run the ADB pull command in the terminal, it outputs the completion percentage of the file transfer as it is running. This output occurs over one line and is updated as the percentage updates. It looks something like this:

[  5%] myFile.txt

Once the transfer finishes, the line changes to something like this:

myFile.txt: 1 file pulled, 0 skipped. 34.9 MB/s (1459978240 bytes in 39.871s)

When I run this same command through a Process in my MacOS app, I do not see the percentage updates come through; only the final line confirming the transfer was successful.

I can't figure out why this is the case. I am fairly sure my process is configured correctly; if I instead make it execute a shell script that slowly outputs a fake loading bar (updating over a single line), I receive those updates through the standard output pipe as I would expect.

Here is what my testing code looks like:

    let outputPipe = Pipe()
    var progressObserver: NSObjectProtocol!
    
    func testAdb() {
        let process = Process()
        process.standardOutput = outputPipe
        process.arguments = adbArgs
        process.executableURL = Bundle.main.url(forResource: "adb", withExtension: nil)!
        
        outputPipe.fileHandleForReading.waitForDataInBackgroundAndNotify()
        
        progressObserver = NotificationCenter.default.addObserver(
            forName: NSNotification.Name.NSFileHandleDataAvailable,
            object: outputPipe.fileHandleForReading,
            queue: nil
        ) { notification -> Void in
            print("Data available")
            let data = self.outputPipe.fileHandleForReading.availableData
            
            if data.count > 0 {
                let str = String(data: data, encoding: String.Encoding.utf8)
                print(str!)
                self.outputPipe.fileHandleForReading.waitForDataInBackgroundAndNotify()
            } else {
                NotificationCenter.default.removeObserver(self.progressObserver!)
            }
        }
        
        do {
            print("Running...")
            try process.run()
            
            print("...checking available data...")
            let data = try outputPipe.fileHandleForReading.readToEnd()!
            
            print("...got available data!")
            let output = String(data: data, encoding: .utf8)!
            let sanitisedOutput = output.trimmingCharacters(in: .whitespacesAndNewlines)
            print("Result: \(sanitisedOutput)")
        } catch {
            print("ADB Error: \(error)")
        }
    }

I have also tried setting the readabilityHandler on the Pipe but that hasn't helped. My guess is that ADB might be doing something differently than a simple shell script would.

I'm not too familiar with how the standard output works, so I am hoping that someone here could help me with this. Thanks.

2

My guess is that ADB [1] might be doing something differently
than a simple shell script would.

Quite possibly. Many command-line tools use isatty to check whether their stdout is wired up to a terminal and behave differently in that case.

One trick here is to run the tool’s output through cat. For example:

% scp test victim.local.:
test                                          100%  100MB 233.5MB/s   00:00    
% scp test victim.local.: | cat
%

As you can see, the progress output from scp is gated on whether stdout is a terminal.

If the tool does behave this way, you have two options:

  • Look for command-line options to get the output you need regardless of how stdout is wired up. Many tools do support that sort of thing.

  • Wire the tool’s stdout to a pseudo terminal, so that the isatty check passes. This gets very complicated very fast )-:

ps I generally don’t recommend using FileHandle for working with child processes, despite the fact that they are the obvious path forward. I have a DevForums post, Running a Child Process with Standard Input and Output, that shows my preferred method [2].

Share and Enjoy

Quinn “The Eskimo!” @ DTS @ Apple

[1] I’m obviously showing my age here but ADB is, and forever will be, Apple Desktop Bus.

[2] Well, my preferred method until Subprocess lands, which can’t be soon enough.

2 Likes
3

Thanks for the reply @eskimo. I appreciate the help.

Yep, you were spot on with that one. Unfortunately the ADB tool doesn't have any command-line options to help me out, so satisfying the isatty check is the only solution here.

Fortunately, I was able to get it working. It took me a while to figure out why my pseudo terminal wasn't doing what I expected, and then I realised that ADB requires an environment variable to be set as well.

Here is a minimum working solution here in case anyone else is interested:

var primaryDescriptor: Int32 = 0
var replicaDescriptor: Int32 = 0
guard openpty(&primaryDescriptor, &replicaDescriptor, nil, nil, nil) != -1 else {
    print("Failed to open pty")
    return
}
let primaryHandle = FileHandle(fileDescriptor: primaryDescriptor, closeOnDealloc: true)
let replicaHandle = FileHandle(fileDescriptor: replicaDescriptor, closeOnDealloc: true)

let process = Process()
process.standardInput = replicaHandle
process.standardOutput = replicaHandle
process.standardError = replicaHandle
process.arguments = adbArgs
process.executableURL = adbExecutableUrl
process.environment = [
    "TERM": "SMART"
]

primaryHandle.readabilityHandler = { handle in
    guard let line = String(data: handle.availableData, encoding: .utf8) else { return }
    guard !line.isEmpty else { return }
    print("ptyHandle: \(line)")
}

do {
    try process.run()
    process.waitUntilExit()
    let data = try primaryHandle.readToEnd()
    let output = String(data: data ?? Data(), encoding: .utf8)!
    print("Final output: \(output)")
    try primaryHandle.close()
    try replicaHandle.close()
} catch {
    print("ADB Error: \(error)")
}
1 Like