Rework `print` by adding observable `StandardOutput` and `StandardError` streams to Standard Library

Evolution Pitches
1

Right now print is automatically writing everything to stdout. Therefore it's up for the developer to build their own StandardError if necessary. However this still does not allow one to observe the data that is written to these streams. In some cases it might ne necessary to redirect or at least mirror the data written one of these streams to a different place (UI or file). In an own module this isn't a huge deal, but if we want to observe logs from a different module that we depend on this becomes very frustrating.

Therefore I'd propose we rework print slightly and expose/add two default text output streams to the standard library, which one can use to write easily to but also be able to hook into and observe the stream.

This idea has to be fully fleshed out yet.

Thoughts?

5 Likes
2

Isn't this already in Foundation as FileHandle.standardOutput and FileHandle.standardError?

3

Well the problem with that is that FileHandle doesn't conform to TextOutputStream, as far as I can tell. So you can't write print("blah", to: FileHandle.standardError) or dump(myObject, to: FileHandle.standardError) or anything like that.
Adding that conformance, however, seems trivial. Maybe that's a better path forward?

2 Likes
4

I'd forgotten about the several earlier conversations about this issue on the list forum.

Conformance is indeed trivial; there is an ergonomics issue in that you have to pass the value as inout, which requires a little dance:

var stderr = FileHandle.standardError
print("blah", to: &stderr)

So it's not truly the most ideal scenario, but there are simple ways to address these concerns down the road, and the crucial point is that there is usable access to standard error today without rolling your own.

I do believe the last time this came up, some objected to the entire premise of Foundation and demanded that the entire library be rewritten or else any functionality in Foundation doesn't count as being available. That was, predictably, not a productive avenue of discussion.

3 Likes
5

Hmm. That's frustrating. I had forgotten that TextOutputStream.write(_:) is mutating.

If it weren't for that mutation dance, I'd say conforming FileHandle to TextOutputStream would be a no-brainer, but it seems like this requires some more thought.

[Edit: I still think it's probably a good idea, but it doesn't seem like it solves this particular problem perfectly]

6

Well I'm fully aware that it's very trivial to build some mutable wrappers to provide the ability of writing to those streams, yet this wasn't the main issue why I wanted these. To be fair, I'm asking for default streams that are observable, which is indeed not a very common task. If we could centralize these streams into stdlib then we could provide a little more API then just .write(_:) from TextOutputStream.

Once I tried to redirect file descriptors so that I was able to observe the data that was written to the whole stream of my current process. It kinda worked for a moment, but then I realized that this solution only worked while the project was running attached to Xcode and stopped emitting any new data otherwise. First I though that maybe print was preventing it in release mode or when the debugger is not attached, however I couldn't find anything in the source code for that matter.

It was necessary for my project that I was able to redirect not only my own logs, but also logs from a 3rd party library, but this turned out to be not a trivial task at all. In the end I had to fork the third party library and provide a custom hook into their log system. This might worked for me, because that library was open sourced, but there are other modules that may provide useful logs but without a way for us to provide such observable hooks.

For more reference you can have glance over this very old blog post.

I think the reason why the target stream must be passed as inout is because String and similar types conform to TextOutputStreamable.

var storage = ""
print("Swift", to: &storage)

Furthermore the standard library already provides an internal implementation for one of the streams (without observation API).

internal struct _Stdout : TextOutputStream { ... }

That's basically my reasoning why I would want to a two global centrialzed streams to the standard library with a little more API than what TextOutputStream requires. Then if possible add another parameter to this print function with a default value to the new StandardOutput stream.

I'm thinking about something like this in the stdlib:

public struct StandardError : TextOutputStream { ... }
public struct StandardOutput : TextOutputStream {
  /* API for observation */
  internal init() { ... }
}

public standardOutput = StandardOutput()
public standardError = StandardError()
7

There are C POSIX APIs, namely dup(2), that do what you want. Are you suggesting providing a Swifty abstraction layer over them?

8

In fact I tried using dup in my attempt to solve the issue on my own, which as mentioned in my previous post only worked while the project was attached to the debugger (which isn't what I wanted to achieve in the first place). With the addition of two centialized streams we wouldn't need to use something like dup anymore because we'd have some kind of observable mechanism in between.

print ---- (observable mechanism) ----> actual write to stdout/stderr

This is still something that we'd need to flesh out, but I want to gather more feedback and hopefully supporting opinions for the main idea first.

9

I've used dup and dup2 in projects (EDIT: here's one) that run outside of the debugger, so this is certainly possible.

How many observers are you planning to support?

10

Well I wasn't able to solve it. Printing something to a redirected file descriptor only worked when the project was started by Xcode, otherwise when I relaunched the app on my own nothing happend at all.

I think such API must support an arbitrary number of observers. Personally I'm used to observable APIs like RxSwift but for this task I'd go a purely Swifty solution, maybe using closures.

One advantage of these streams is that as soon as they're added (assuming for a second that they will be) then you can start observing all logs printed by any module that you depend on, of course if the logs are printed using print(_:separator:terminator:).

EDIT: @saagarjha the app that failed to redirect while the debugger wasn't attached was an iOS app. Just wanted to clarify that.

11

This would be useful, and would be a generalization of a capability that's in the stdlib already. Try this out:

var printed = ""
_playgroundPrintHook = { p in printed += p }
print(1)
print(2)
_playgroundPrintHook = { _ in }
print(3)
print(printed)
2 Likes
12

I also would like to have an easy way of hook into thise output streams. I recently had to tweak one of my libs to accept the output streams via injection so I could manually hook from the outside.

13

In case anybody's interested, I blogged about _playgroundPrintHook some time ago: _playgroundPrintHook – Ole Begemann

1 Like
14

Just read the article. To be fair, I never really payed much attention to the _playgroundPrintHook function which indeed could helped me to get the data I wanted (at least from stdout stream).

I think as @nnnnnnnn correctly sayed we can use this as an argument in favor of such streams in the stdlib, because it's indeed a generalization and also clean up of existing functionality. Furthermore we now have the chance to go a step further and provide a little more API that can satisfy most or even all possible needs for such streams.

To sum up my personal needs:

  • An generalized and easy way of writing to stdout and stderr:

    print("Swift") /* is the same as */ print("Swift", to: &standardOutput) 
print("Swift", to: &standardError)

  • Allow multiple subscriptions directly on the streams so that we don't have to worry about messing with a single global closure like mentioned in @ole 's blog post. This solution shouldn't rely on complicated extra types. An extra protocol for convenience might still be needed. This solution can then be used to remove _playgroundPrintHook and it would allow us to observe logs from other modules if they're using print or print(_:to: &standardError).

  • Such streams should be global, and I think there is no need for us to be able to instantiate new instances.

15

I could imagine an implementation along these lines:

public protocol ObservableTextOutputStream : TextOutputStream {
	associatedtype Token
	func addObserver(_ observer: @escaping (String) -> Void) -> Token
	func removeObserver(for token: Token)
}

fileprivate final class Box {
	let stream: ...
	var observers: [Something_Hashable: (String) -> Void]
	func write(_ string: String) {
		lock()
		observers.values.forEach { $0(string) }
		stream.write(string)
		unlock()
	}

	...
}

public struct StandardOutput {
	internal let box: Box
	internal init() {
    	box = Box(stream: ...)
    }
}

extension StandardOutput : TextOutputStream {
	public mutating func write(_ string: String) {
		box.write(string)
	}
}

extension StandardOutput : ObservableTextOutputStream {
	public struct Token {
		let identifier: Something_Hashable
	}

	public func addObserver(_ observer: @escaping (String) -> Void) -> Token {
		return Token(identifier: box.addObserver(observer))
	}

	public func removeObserver(for token: Token) {
		box.removeObserver(for: token.identifier)
	}
}

public var standardOutput = StandardOutput()