Thoughts on "Why does SwiftLog not have multiple log destinations / formatters / colouring / some other feature"

The rule of three struck again, and as always somebody asked me a very good question that I have given my thoughts on a few times before.

One set of questions around SwiftLog comes up a lot (for a very good reason):

  • Why does SwiftLog only have one logging destination?
  • Why does SwiftLog not have configurable formatting?
  • Why does SwiftLog only log to stdout in this super boring and unconfigurable way?
  • Why can SwiftLog not load logging configuration from a file?
  • Why does SwiftLog not support my favourite feature some other logging library has?
  • Why won't SwiftLog allow me to change the default LogHandler at runtime?
  • ...

I think the core of those questions basically boils down to

Why is SwiftLog so basic?

Turns out, it's deliberately very basic and let me try to explain why: SwiftLog tries to be minimal in a way: Everything that can be as efficiently implemented in a LogHandler, should be in a LogHandler.

In the simplest case, a LogHandler is fairly similar to a "logging destination" (+ formatting, ...). And in fact many of today's backends (LogHandlers) work exactly that way, they implement a logging destination (+ formatting etc) as a LogHandler. My hope for the future however is that we can transition to a slightly different world which looks like this:

The reason I'm hoping for new packages that are "middle pieces" is that is it quite a lot of work to implement a LogHandler that does everything (formatting, sending to destination, managing metadata, buffering/pressing back/dropping messages ...). And each of the current LogHandlers needs to repeat all of the same work. Let's say that there's a package upstream that has really nice colouring but logs to stdout. And now let's assume you want to make a package that has the same colouring but logs to stderr. The only way you can do this is to duplicate all of the code of the first package. That's not great.

Or let's say, there's a package that ships a LogHandler that fulfils all your formatting and destination needs but unfortunately it's a little bit too slow so logging shows up as taking too much time. You'd like to keep the same formats/destinations but you'd like to switch to using a super fast ring buffer where a background thread is doing the heavy lifting log processing.

However, if we had a few (maybe only one really great?) "middle pieces" (which are LogHandlers) and those middle pieces implement features like:

  • configurable destinations
  • configurable formatting
  • multiple destinations
  • maybe formatting that can be conditional on the destination
  • having an opinion on what to do if the log messages come in faster than they can be processed. There's at least two options: 1) slowing down the logging threads 2) dropping log messages. What the right answer is very much depends on your software, SwiftLog shouldn't decide that for you.
  • are we doing the log processing in the log emitting thread or in a background thread? Again, the right answer depends on your software.

Then it would be much much easier to say adjust the formatting, or add a log destination, or something else.

Of course, it's a valid question why we didn't make SwiftLog the API and the middle piece in one. The answer is really straightforward: The more you put into a library, the more necessary it will become to evolve (and maybe break) the public API. Also the more functionality you put in, the more opinion you have to put into your package (for example what to do if there are more log messages than we can process). Of course we could've made SwiftLog very opinionated and feature rich. The problems however may appear slightly further down the line when people are unhappy about the opinions, or maybe they're unhappy the way we say designed the formatting or destination API, ... If people are unhappy with an API, often it's the easiest to create a new major version. But if you're an API package and you up your major version, that can create a significant problem for the ecosystem. Now two totally unrelated libraries A & B that used to be compatible could suddenly conflict because A may depend on version 1.x of the API but B depends on version 2.x of the API.

By making SwiftLog kind of minimal as well as supporting plenty of different logging styles (by delegating a lot to the LogHandlers) we think we can avoid this situation where we need to change the public API all too much. So in a way, I hope that at some point soon, there will be a few really powerful "middle pieces" into which people plug in their destination/formatter/... packages. Those middle pieces can be opinionated about what they do and do not provide. So in a way these "middle pieces" can implement all the SwiftLog features that everybody wants. The big benefit is that if say a middle piece changes its public API, then that works without much trouble: Libraries that are merely emitting log messages should only depend on SwiftLog. They should be entirely unaware which LogHandler is currently installed so switching from SuperGreatMiddlePieceLibrary version 1 to SuperGreatMiddlePieceLibrary version 2 should be a very straightforward switch that only affects main.swift of your application where you set up the actual LogHandler.

Please feel free to comment/question/... here if you want.


I deliberately emphasised that these are my thoughts because they're definitely not an official position of say the SSWG, or my employer, Apple, literally just my thoughts/hopes :slight_smile:. I have however been involved in the design of SwiftLog.

8 Likes

I quite like this approach, I imagine SwiftNIO shares this philosophy too in some way. I wish more core libraries in the Swift ecosystem were this minimalistic and extensible (e.g. Foundation and XCTest :wink::wink::wink:, and I do realize there's a certain history with both of them, but the problem is that we don't have any alternatives). On the other hand, I think there's a certain risk of ending up with the churn that the JavaScript ecosystem has, which pushes this philosophy to the extreme. Admittedly, the Swift ecosystem is not as prolific yet (a blessing and a curse I think), so it's not something we'd worry about any time soon, but something to keep in mind.

Maybe SwiftLog could endorse some well-designed "middle pieces", at least by linking to them in its README.md? I don't have any candidates to propose unfortunately, just wondering if that's something to be considered?

A problem with having a lot of small modules (let alone packages) is that it makes everything slower in Swift. I did this mistake in Noze.io and thoroughly regretted that.
Fetching is slower, building is slower, the generated code can be significantly slower (unless you make everything at-inlinable).

All your points are completely valid, but I'd still prefer to have one log4j style package (can have multiple targets if you wish) that comes w/ common loggers you'd want, plus the stuff to easily build own loggers when required.
A little like NIO, which also comes with HTTP and WebSockets, but doesn't leave out the option to have more complex additional loggers outside (like HTTP/2 or SSL live outside of NIO).

For now, you need to do swift build -c release -Xswiftc -cross-module-optimization to fix the slowness without @inlinable. In the future, this should become the default with SwiftPM :slight_smile:.

Taking this to the extreme is absolutely not what I'm hoping for. Unfortunately, I wasn't very clear with my language regarding modules and packages. I'd be 100% happy if there's just one big "middle piece" package that has all the required log destinations, formatters, options, ... I'd also be happy with 2 or 3 middle piece packages and a lot of destination/formatter packages. Remember, you'd use SwiftLog, one middle piece, maybe 1 or 2 destinations and probably one formatter. Even in the case that they're all from separate packages, that'd make 5 packages.
What I expect is a number of "bigger" middle piece/backend packages that ship a large number of options. But only time can tell.

The main intent of my post was "Why is it not all in SwiftLog?". If somebody wants a very small number of packages, they could still create a package that contains everything they want apart from the API (SwiftLog). That would make it 2 packages for logging and I think that's cool.

3 Likes