[Feedback] Server Logging API (with revisions)

(Jacob Williams) #13

Many other languages provide the ability to support custom logging levels. I see that the Logging.Level enum uses an Int raw type, would anyone else think it worthwhile to support custom logging levels?


public enum Level {
    public var rawValue: Int {
        switch self {
            case .custom(let level): return level
            case .trace: return 0
            case .debug: return 10
            case .info: return 20
            case .warning: return 30
            case .error: return 40
    case trace, debug, info, warning, error
    case custom(Int)


public struct Level: RawRepresentable, ExpressibleByIntegerLiteral {
    public let rawValue: Int

    public static let trace: Level = 0
    public static let debug: Level = 10
    public static let info: Level = 20
    public static let warning: Level = 30
    public static let error: Level = 40
    public static func custom(_ level: Int) -> Level {
        return Level(rawValue: level)

    public init(integerLiteral value: Int) {
        self.init(rawValue: value)

    public init(rawValue: Int) {
        self.rawValue = rawValue


  • Developers can make their own logging levels for increase control over their logs
  • struct version can easily conform to Comparable also


  • Added complexity the Level code
  • The struct version is not quite as easy to switch over as the enum
  • The enum version now has to compute the rawValue every time for its Comparable conformance

(Matthew Johnson) #14

Another option is to make Logger generic over a custom Comparable log level type that has factories for the standard levels (i.e. debug, error, etc)

(Johannes Weiss) #15

Thank you! Yes, that was an oversight. I'll make a small PR later fixing small things like this.

(Johannes Weiss) #16

Thank you!

(Johannes Weiss) #17

Thanks for still sending feedback. We had two discussion threads before (here and there). Tomer and I compiled an API out of that which should allow all existing solutions to migrate to this. This is not the final version of the implementation but the rough shape of this proposal is more or less final and the outcome of previous discussion threads. There are a things I'm personally not very happy with - some of which you mention in your post - but I believe logging is so fundamental that we need to come up with an API that everybody can use and adopt without too much friction. So asking everybody to re-architect would not work I believe. Therefore, we propose the current version to the community and the SSWG will decide whether to move along with this proposal or not.

But: If accepted into the SSWG's process, the resulting package will be a regular open-source SwiftPM package and the code will evolve over time. So implementation issues and details can be addressed whilst the proposed package is going through the Sandbox/Incubation stages of the SSWG process.

I agree with your sentiment and share your opinion. However, after talking to many people and going through two discussion threads it doesn't appear like we can force the ecosystem into some logging model without risking the API package not being widely adopted which is the main important point of an API package. One of the popular packages for example only offers a global logging facility. Other people want one logger per class and configure those with different logging levels so the ability of being able to create loggers is something we need at this stage I believe.

Well, for a value-typed LogHandlers that are explicitly passed around I think this is all good. For reference-typed LogHandlers, especially global ones I do agree with your sentiment. I know we're squaring the circle here and trying to get many different ways of logging into one API but we can't ignore what's there right now and the requirements people have. We need to rely on libraries not messing with the log level etc. My hope is that over time best practices will be established and the ecosystem itself evolves and we can evolve the APIs to capture the best practices more precisely.

It's not, that's a typo in the markdown only. I'll fix later today along with other typos. In the first proposal it was indeed optional because we were toying with the idea of allowing LogHandlers to not support metadata at all. But I think that was a bad idea and therefore we removed it.

The dictionary is meant as the primary way of storing metadata in your program, ie. the LogHandler may store it however it wants (maybe a dictionary, maybe a tree, whatever). The property that exposes the whole metadata as a dictionary and it's there to support MDC (mapped metadata context) where the context can be saved/restored to/from a thread-local variable. Again, personally not a fan of that but it's a common request that you can use a thread-local to store the metadata and when crossing threads you'd take the metadata out, ship it over to another thread and install it there...

Thanks, just by accident, will fix in code.

I think that might turn out to be a good idea. If this proposal is accepted, I'd be more than interested to work with you on getting that into the codebase before we leave the Sanbox stage. At some point obviously we'll have a public API so breaking changes will be more problematic. But in the early Sandbox stage we won't immediately declare version 1.0.0, so I'd say this is a more than acceptable change. This feels inline with the spirit of the proposal, so we don't need to propose a whole new version for that.

That is also something I'd like to talk about should this proposal be accepted.

It depends on the LogHandler, squaring the circle...

(Johannes Weiss) #18

I consider this a small enough change that I would encourage a PR if this proposal gets accepted.

This is borderline but I think we could make this a PR or if it's too much of a change propose this as a follow-up change. What do you think?

(Johannes Weiss) #19

I think this is a worthwhile small addition that can be proposed as a PR should this proposal get accepted as it's within the spirit of this proposal.

(Johannes Weiss) #20

I think this can make sense but also has serious downsides, the ones you mentioned and more opportunity for abuse. We have deliberately decided not to provide this functionality.

(Damian) #21


I think it would be to have some project level logging config which allows managing logging of his application and packages that he uses.

Sometimes packages that we use produce very big amounts of log messages on trace level. It would be nice if a user could set logging level for a given package or logger, so the lowest logged level is info.

Of course, it could be configurable per environment, so I don't have to deal locally with tons of logs that I do not need.

It would be even better when connected with hierarchical loggers. MyApp.request logger could be configured using its parent MyApp and some settings might be overridden using MyApp.request logger.

Great job with the proposal, logging is what I missed most in Swift! Keep doing what you do!

(Neal Lester) #22

Should I add this as a feature request issue to swift-server-logging-api-proposal?

(Ian Partridge) #23

Thanks so much @tomerd and @johannesweiss for driving this forwards. A few thoughts...

  1. Currently there is no support for making a compile-time decision about the minimum possible log level. As processing a disabled log message is not free (even with @autoclosure being used) it would be nice if, for example, the user could decide at compile-time that all "trace" log messages will be ignored. This could perhaps be achieved through some #ifery in the Logger methods (but I haven't tried prototyping this yet).

  2. Similarly, it would be nice if the user could decide that certain low log levels are only enabled when compiling in debug mode, and are compiled out in release mode.

  3. Log4J includes a FATAL log level, which is more severe than ERROR. What do people think about adding this?

  4. Did you consider making the label on Logging.make() optional with a default of nil? After all, in the included StdoutLogger it is thrown away, and if it is required people might reasonably assume it is used for something.

  5. Also, and maybe alternatively, did you consider giving the label on Logging.make() an autogenerated default value? For example public static func make(_ label: String = "\(#file):\(#line)") -> Logger { }.

  6. The use of @autoclosure in the message parameter is good, but in the case of a simple "non-interpolated string" log message I suspect the autoclosure is more expensive than a normal string would be, because of the closure and ARC overhead? Could we add overloads that take a StaticString instead of the autoclosure? This would mean adding another function to LogHandler that takes a StaticString message instead of a String. The reason I'm thinking of this is that since it's easy to attach metadata to log messages, I think it's more likely that people will use hardcoded strings in the message field, and it would be a shame to always pay for the autoclosure if we don't need to.

  7. The included StdoutLogger merges the metadata from the log call and the current LogHandler into one dictionary. Is this a good idea? If there are duplicate keys, the loghandler's element is discarded. I think it these should be separate and it should be clear which metadata dictionary the elements came from.

Thank you again!

(Jacob Williams) #24

I thought the list of logging levels looked like it was missing something, but I couldn't quite put my finger on which one. I would be +1 to adding a fatal log level.

On a separate, but related note. Python includes an exception logging level which is basically the same as error, except that it will also log the traceback for the error. Would anyone else find this to be useful?

(Cory Benfield) #25

That's not an accurate assessment of Python's Logger.exception. In Python, Logger.exception attaches the stack trace of the active exception to the log message, and is required to be invoked from within an except block. This is simply not possible in Swift: Swift errors do not carry a traceback with them, and it is not possible to magically attach the traceback of a currently-caught error.

The best that could be done would be to provide a TracebackError protocol that requires providing a Traceback object on the Error, and then provide a way to construct the Traceback object, and the provide a log method that automatically handles this. In my opinion this is wildly out of scope for this proposal, and is functionality that can be provided by community extensions in the future if it's of interest.

(Matthew Johnson) #26

Thanks for continuing to work on this proposal and respond to new feedback!

This does not seem inline with the spirit of Swift Evolution, which is to accept feedback from newcomers through the end of the review period and often results in changes to the proposal (sometimes significant) followed by additional round(s) of review. You posted the initial proposal draft for discussion during the holiday season, a time during which many of us are extremely busy. I wanted to provide feedback sooner but just did not have time to do so.

Neither do I! I am not sure where you get the idea that I am asking anyone to re-architect. Unless you are interpreting my suggestions as requiring a re-architecture of the implementation of this proposal I don't understand how you would get that from my feedback.

I think it's important to the health of the community to follow the Swift evolution process even if that means changes to the implementation after feedback during the discussion and review process. It is not intended to be a yes or no process, but one in which designs can be improved based upon feedback from the community. We want to welcome people with all levels of involvement and time commitment in the process. This is why reviews are formally announced ahead of time and reviewers are not expected to have participated prior to review.

With the above in mind, I think following statement is not in line with how the Swift evolution community is intended to function

I didn't suggest trying to force anyone into a specific logging model. I suggested that you look for ways to support the global logger model without automatically making a global logger available in every project that uses this API. In fact, doing this is exactly the kind of thing that would incentivize some projects to look elsewhere. If your goal is pervasive adoption you need to take seriously the fact that not everyone wants a global logger available.

Sure, and I am not opposed to that. But is it really necessary to require every project using this API to include the ability to construct a logger out of thin air targeting an unknown handler? Again, this is a similar concern to the global logger issue. If you include this ability in your library users cannot remove it. However, if you adopt a design that allows users to opt-in to a global logger and / or a global logger factory by creating their own globally visible instance of these capabilities, then it is possible to serve the needs of more projects. Projects that don't wish to have such facilities available simply do not configure them.

Maybe for some projects it's ok, but I strongly disagree in general. It is often desirable to configure the log level centrally and not accidentally drop log entries because of a bug somewhere that sets the log level too low.

I am bringing forward requirements that I believe are important, especially for projects that wish to prioritize clean semantics.

If the sole intended use case for the dictionary is to be able to shuttle the metadata across threads why does it need to be a dictionary? Instead you might consider making it opaque. The way the API is currently designed I can imagine a whole lot of code accessing that dictionary instead of the intended subscript APIs.

It's confusing to me that you want to punt on issues like this. Again, I apologize for not having been able to provide feedback during December, but this proposal is not even in the review stage yet.

I don't like this answer at all. It means anyone writing code against Logger has no idea what the semantics of the type are. If you instead adopt the derived logger approach along with a different implementation strategy you would be able to provide an API with clear and consistent semantics regardless of the log handler implementation.

Please don't take my feedback the wrong way, I think the idea of a common logging API that everyone uses is a good one. My goal is to bring up semantic issues I see in the current design that could lead to people choosing not to use this API, thus defeating the stated goal of the proposal. I think it's possible to come up with a design that has more clear semantics while preserving the ability to support all the logging models necessary. This would be a stronger proposal without some of the design flaws that could motivate people to look elsewhere. (This is subjective of course, but there are certainly others who will share my opinion - it sounds like maybe you even feel this way about the current API to some degree re: the global logger, etc)

I hope it's not too late in the process to continue refining the design. And again, I apologize for not contributing to the earlier discussions, I just wasn't able to make time for it until the latest draft was posted after the holidays.

There are a few approaches supporting this. One is to use different loggers targeting the same handler, but configured with different log levels. I haven't looked closely at how well this proposal supports that approach.

Another feature that could support this use case would be to support multi-context loggers that is generic over the context type (usually a simple enum) and which require a context parameter to log calls (which could be defaulted to a default context). Multi-context loggers could also support the ability to derive a new logger that has a fixed context (for passing to a subsystem).

(Johannes Weiss) #27

Apologies if that came across the wrong way. Feedback is always accepted and appreciated. But just like on Swift Evolution a proposal is proposed and 'owned' by the author(s) who propose what in they currently believe is the way forward for the problem at hand, John captures that well here. As done before, I will certainly work in improvements that 'fit the spirit' of this proposal as we did before.

I was referring to this comment of yours:

I am personally not fond of global loggers either but I think we have to accept that certain higher-level libraries/frameworks might want to have a global logging facility and we'll need to be compatible with that.

Apologies, that was poorly worded. What I should have said is that I believe that moving forward with a logging solution is very important for the ecosystem. And the way I read your feedback is that you don't agree with some core design choices, for example you say that you "really don't think a library should be introducing global logger or the ability to create on on the fly". I don't believe this would work because we can't force people into not wanting just a global logger so we need to support higher-level libraries/frameworks creating one.

Wait, there must be a misunderstanding here. The proposed API does not expose any Logger instances, each and every one is created by the application/other libraries. In fact, I was arguing the same way you are that we should not have a global 'default' Logger.

The handler is not unknown, the only thing that will happen on Logging.make is that the closure configured with Logging.bootstrap is run and will therefore produce a LogHandler. It's totally up to the implementor of the package that ships the LogHandler to decide what to do.

Well, backend systems with very many active connections will likely want to configure the log level of a specific request to a certain level. Or if you have a rare bug that is experienced in roughly 0.1% of the requests, you might want to change the log-level of every 10,000th request to .debug or so trying to sample one of the bad requests without penalising every single request's performance by setting the log level to .debug globally.

The issue at hand is that some people want one global log level, others want one log level per sub-system, yet others want a log level per request (but the same for all sub-systems executed with that request).

Thank you for that. I am also very much in favour of clean semantics but people seem to have conflicting ideas/requirements about logging which lead to compromises.

Thanks, that is a good point, I filed the issue for now but will definitely have a think if that will work.

The very point you mentioned seemed compatible with the proposal and minor enough that we could PR that later after the inception of the server logging API project (if accepted). Please note, that there are differences between Swift Evolution and SSWG Proposals. For Swift the language, every language change needs to go through Swift Evolution. SSWG Proposals are about proposing an independent OSS project to the SSWG Incubation Process, in this case starting at Sandbox level.

If this proposal gets accepted to the Sandbox stage, the first commit will be the code that is now part of the proposal repository (minus the example code etc) but the development will continue.

I understand and share your unhappiness here, so far we just haven't found a better solution that can support the major opinions about how logging should work. When you say the 'derived logger approach', do you mean essentially immutable loggers + 'mutation factories' (ie. let myChangedLogger = myLogger.makeDerivedLogger(logLevel: .error, metadata: myLogger.metadata + [...])) to set the log level to .error and add a metadata value?

(Johannes Weiss) #28

So you're right that there's no way to make a compile-time decision but I would argue that this is not necessary. Let me explain why: The reason we have struct Logger with everything @inlinable is that the only thing we need to do if we don't log is check the log level, everything else should be behind a branch in the compiled binary.

In other words, let's imagine you wrote

logger.debug("hello", metadata: ["foo": "bar", "buz": "qux"])

what the compiler will emit into the calling function (because @inlinable) is something more akin to this

if logger.logHandler.logLevel <= .debug {
    /* do stuff to prepare the log message, like evaluate the autoclosures etc */
    logger.logHandler.log(.debug, "hello", ["foo": "bar", ...])

So the idea is that all the expensive stuff is only done iff the comparison to the log level returns true. To show you what I mean, let's have a look at the generated control flow graph (from the assembly) for this function:

func foobar(logger: Logger) {
    logger.debug("hello", metadata: ["foo": "bar", "buz": "qux"])

So as you can see, we have a prologue (the purple box), then a branch to the left which does loads of stuff and a branch to the right which goes straight into the epilogue (bottom box). So what we want is that if nothing is logged, we go straight from the purple box to the bottom one. So let's zoom into the purple box:

So what decides if we go left (expensive) or right (cheap) is the result of call _$S7LoggingAAO5LevelO1loiySbAD_ADtFZ which (when passed through swift demangle) revels itself as static Logging.Logging.Level.< infix(Logging.Logging.Level, Logging.Logging.Level) -> Swift.Bool.

Pretty good! We only get the log level from the LogHandler existential and then invoke the Logging.Level's less than function. So we don't do anything with the @autoclosures if we don't log which is pretty good.

I realise that a little work is still more than no work at all, how important do you think this would be? Because in production you'll run Swift in release mode because debug is quite very slow :).

I'm totally happy with this. Happy to either make a change to the proposal or if accepted this can be a PR because IMHO it clearly doesn't change the spirit of the proposal at all. Please let me know what you think.

I think it's good if we enforce that everybody passes some string there to identify themselves as I reckon more serious loggers than StdoutLogger might want to do something with the information and if we make it optional there's just be many nils in there I suggest. But I'm not married to not having it optional, just don't see any real benefit.

That I like a lot more and actually think we should do this. Let's see if anyone has objections here.

Nope, that can all be inlined. As you can see in the assembly below, it's just one load from a static address, just like StaticString.

That's a good question :slight_smile:. I consider StdoutLogger more a toy at this point in time that we should finish when/if this goes live . But as a general question we should document what merge strategy we think is useful. I haven't thought about it too much yet. If anyone has opinions, please let us know :slight_smile:.

Thank you very much for your great feedback.

(Matthew Johnson) #29

Absolutely! My goal is to try and convince you that this is not the best way forward by making a compelling case. All I ask is that you consider the argument I make with an open mind. :slight_smile:

I don't disagree, but I think there are other ways to be compatible with it than to just impose a pervasive global logging facility on everyone.

Can you explain why you think it is necessary to make a pervasively visible global logger in order to allow those who wish to use one in their library to be able to do so? An alternative path would be for such libraries to provide a top-level configuration API to inject a logger which they make available as an internal global within their library. This approach allows the library to use a globally visible logger in their code without polluting the global scope of every other module.

Perhaps you're right - looking again it looks like maybe it is just the handler that is global. However, this is just as problematic IMO, perhaps even more-so! The proposal even says "Libraries should never change the logging implementation as that should owned by the application".

If you use a design based on derived loggers instead of a global handler, logger factory, and Logging.bootstrap function then it simply wouldn't be possible for anyone to stomp on the handler. Code would use the logger provided to it and / or new loggers derived from it. Libraries would just need to provide a configuration point to receive the top-level logger instance from which other loggers are derived.

The handler is unknown to the code that is creating the logger. I think I missed the fact that you are only supporting a single global handler when I wrote this though. That is also unfortunate IMO. If you use derived loggers it wouldn't be that hard to support multiple back-ends and there are clear semantics for the handler used by a derived logger: it is the same handler as the logger from which it is derived. This would allow an application to use different handlers for different subsystems, for example.

There are other design approaches that could address these use cases without requiring everyone to accept the ability to set the log level everywhere in their code. If you are receptive to considering alternatives I would be happy to give this further thought and provide a concrete suggestion.

I am not convinced compromises to the semantics of the API itself are necessary in order to meet the requirements at hand. Perhaps small compromises (such as exposing a configuration point for the top-level logger a library uses instead of the library just calling Logger.make) are necessary, but I think they are reasonable and I hope people would find them acceptable.

Thanks for explaining that. I was not as familiar with the differences as I should have been. :blush:

I definitely support the goals project in general, I just don't want to see it get locked into a design that makes some significant (and IMO unnecessary) compromises. Once people start using this there will be compatibility concerns which will make significant semantic changes difficult or impossible to make.

Exactly. myLogger continues to behave exactly as it did prior to that call, while myChangedLogger exhibits the modified behavior: additional metadata, additional layer to its "name", perhaps a modified level, etc.

Ideally the ability to set the level when deriving a new logger is not something that would be available on all loggers. There are a number of different behaviors people might want here and quite a few options for how to implement an API flexible enough to support the various use cases. I would be happy to spend time making more concrete suggestions if this is a direction you are willing to explore.

(Neal Lester) #30

Making the logger label optional could encourage library authors to provide a nil which would then be problematic for the consumers of that library. There is also the issue of potential clashes between different libraries. An explicit convention for the label could help with that or perhaps having the label default to the module name.

(Johannes Weiss) #31


Okay, there it is our misunderstanding. When I say global logger, I mean a global facility that allows you to emit log messages. The proposed API does not offer that. What you meant with global logger is the Logging.bootstrap which is indeed public but that isn't a logging facility, that is a (meant to be one-time) selection of the logging backend implementation.

IMHO, one important thing for logging infrastructures is that all parts of the program (including all libraries) by default log to the same logging backend, so that all messages end up for example in the same file, in the correct order etc.

One way to achieve that (and that seems to be both your any my preferred way) is to explicitly hand loggers around to code that logs. But that is at odds with people just wanting to log from anywhere and I don't believe that we can ignore those people.

If I understand your proposed solution correctly, somewhere in the main function of the program a 'root logger' is constructed and from there on it's passed around and the logger (or derivations of it) has to reach all code that intends to log. Is that correct? And if yes: How would then a library that doesn't want to change its API log to the same destination as the rest of the code?

Btw, I really think we should provide a public API to make a Logger from a LogHandler which doesn't require any globals for testing/special cases etc. But I still expect all real-world code that wants to create a logger out of thin air (and these users really exist) to go through the global.


It's quite easy to create a multiplexing LogHandler that sends everything to multiple LogHandlers, we do in fact now ship one now because it was a somewhat common request.
You are right however that with the current API it's hard to support different logging backends for different sub-systems. That's because I didn't think that is very compelling but if we just added an API to create a Logger from a LogHandler that would be a very easy solution catering to the people that really want that.

Obviously, that doesn't take your main issue (the globally configured logging implementation) away but I really don't think we can do away with it.

Thanks, I would indeed be very interested to see more concrete thoughts on this.

Here is where it gets tricky. Convincing the library authors is one thing that's hard but even harder is convincing application developers to configure each and every library with the right Logger/LogHandler. Or would you want to also design and ship a dependency injection framework?

Okay, after reading everything you wrote, am I right in thinking that what you dislike basically boils down to two issues:

  1. the global logger creation facility that has only one (can be multiplexed though) logging backend configured globally
  2. using mutating properties to change the logger configuration instead of derivation

We've talked (and misunderstood each other) a lot about (1) and my position remains that not being able to make up loggers out of thin air is necessary. I would be very very happy if I were wrong on that case because I really dislike globals of that sort and believe those things should be explicit. However what's IMHO more important than my personal taste is that the ecosystem gains a logging solution that is very widely adopted. It's all OSS and the ecosystem is young and used to changes so I believe that over time we can establish best practices and eventually maybe even get rid of that global.

I find (2) quite interesting however. What you call derived loggers is I believe pretty much what I called value-typed LogHandlers. Assuming a value-typed LogHandler, there is no difference between

let originalLogger = ...
let changedLogger = originalLogger.addMetadata("foo", "bar")
changedLogger.debug("hello world")


let originalLogger = ...
var changedLogger = originalLogger
changedLogger[metadataKey: "foo"] = "bar"
changedLogger.debug("hello world")

in both cases, changedLogger is a derivation from originalLogger and originalLogger remains unchanged. I like the look and feel of second version better. But obviously as you have pointed out before and as the proposal says, the semantics of the mutation aren't clear here. originalLogger will only be unchanged iff the configured LogHandler has value semantics.
I'm sure this make us both feel quite uneasy but after many discussions I have accepted this as a necessary tradeoff. And I can totally understand if you don't. But there'll be people who want to change the log level globally throughout the whole application etc and with the current design you can make a LogHandler which will allow you to do that.

(Johannes Weiss) #32

Agreed. Unfortunately, the module name is afaik not available, just #function, #file, #line, #column. But I like @IanPartridge's idea to default it to "\(#file):\(#line)" I think.