# \[Discussion\] Server Logging API

**URL:** <https://forums.swift.org/t/discussion-server-logging-api/18834>\
**Category:** Proposals\
**Created:** [December 15, 2018, 12:16pm UTC](https://forums.swift.org/t/discussion-server-logging-api/18834 "2018-12-15T12:16:00Z")\
**Posts on this page:** 20\
**Page:** 3

<div class="post-metadata">

**Author:** ![PaulSolt](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/paulsolt/32/16477_2.png) [@PaulSolt](https://forums.swift.org/u/PaulSolt)\
**Post date:** [December 18, 2018, 2:29am UTC](https://forums.swift.org/t/discussion-server-logging-api/18834/41 "2018-12-18T02:29:45Z")

</div>

Is this only server logging?

Or could it be used in an iOS or macOS app?

---

<div class="post-metadata">

**Author:** ![toffaletti](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/toffaletti/32/5605_2.png) [@toffaletti](https://forums.swift.org/u/toffaletti)\
**Post date:** [December 18, 2018, 4:32am UTC](https://forums.swift.org/t/discussion-server-logging-api/18834/42 "2018-12-18T04:32:32Z")

</div>

I would like to see some exploration of structured logging in addition to the metadata. [GitHub - uber-go/zap: Blazing fast, structured, leveled logging in Go.](https://github.com/uber-go/zap) for example, does this really well. Primary use case would be JSON log output that can be easily indexed by splunk/elk. Neal provided some thoughts on this above, but I didn’t see anyone addresss those points.

---

<div class="post-metadata">

**Author:** ![tomerd](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/tomerd/32/3402_2.png) [@tomerd](https://forums.swift.org/u/tomerd)\
**Post date:** [December 18, 2018, 5:57am UTC](https://forums.swift.org/t/discussion-server-logging-api/18834/43 "2018-12-18T05:57:52Z")

</div>

> [@toffaletti](#):
>
> I would like to see some exploration of structured logging in addition to the metadata. [https://github.com/uber-go/zap](https://github.com/uber-go/zap) for example, does this really well.

the goal of metadata is to enable structured logging. there has been a bit of a discussion above about also adding "temporary metadata" per logging call, which is an addition under review. see discussion between @johannesweiss, @artemredkin and @ktoso above

> [@toffaletti](#):
>
> Primary use case would be JSON log output that can be easily indexed by splunk/elk. Neal provided some thoughts on this above, but I didn’t see anyone addresss those points.

exactly. that would be the role of a logging library implementation (aka LogHandler implementation) which will format & output the entries in different formats like json or key-value. see [background overview](https://forums.swift.org/t/server-logging-api/18834/17) above

note that this proposal is for the logging API that enables the creation of such logging library, but does not (and should not) include one. once the API is settled, the SSWG will work with the community to implement one or more

---

<div class="post-metadata">

**Author:** ![tomerd](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/tomerd/32/3402_2.png) [@tomerd](https://forums.swift.org/u/tomerd)\
**Post date:** [December 18, 2018, 6:06am UTC](https://forums.swift.org/t/discussion-server-logging-api/18834/44 "2018-12-18T06:06:42Z")

</div>

> [@PaulSolt](#):
>
> Is this only server logging?
> 
> Or could it be used in an iOS or macOS app?

os\_log has serious performance optimizations on darwin, so if you are building an iOS or macOS app you should use it and not worry about this API

this API is designed for non-darwin and/or cross-platform code, so you can plug in different backend. see [background overview](https://forums.swift.org/t/server-logging-api/18834/17) above

---

<div class="post-metadata">

**Author:** ![johannesweiss](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/johannesweiss/32/38183_2.png) [@johannesweiss](https://forums.swift.org/u/johannesweiss)\
**Post date:** [December 18, 2018, 9:08am UTC](https://forums.swift.org/t/discussion-server-logging-api/18834/45 "2018-12-18T09:08:49Z")

</div>

> [@tomerd](#):
>
> os\_log has serious performance optimizations on darwin, so if you are building an iOS or macOS app you should use it and not worry about this API

Indeed. We should however point out that when [this](https://forums.swift.org/t/custom-string-interpolation-and-compile-time-interpretation-applied-to-logging/18799) lands, we should be able to adopt a few of the osLog optimisations for the server logging effort too and you should then be able to use osLog as a backend (LogHandler) with acceptable performance. Plain osLog however will always remain faster as it decides many things at compile-time that we will need to do at runtime because the server world needs more flexibility.

---

<div class="post-metadata">

**Author:** ![ktoso](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/ktoso/32/44526_2.png) [@ktoso](https://forums.swift.org/u/ktoso)\
**Post date:** [December 18, 2018, 10:29am UTC](https://forums.swift.org/t/discussion-server-logging-api/18834/46 "2018-12-18T10:29:37Z")

</div>

Good ideas here.

I explored the "context offers logger" proposal in depth today.

* * *

**Discussing the Optional around LoggingMetadata** , there's also more discussion around this with Tom in my "exploring things" PR: [[DO NOT MERGE] Exploring slight changes to APIs by ktoso · Pull Request #6 · weissi/swift-server-logging-api-proposal · GitHub](https://github.com/weissi/swift-server-logging-api-proposal/pull/6#discussion_r242418606)

> [@johannesweiss](#):
>
> > **Metadata: signature discussion: var metadata: LoggingMetadata?**
> > 
> > [...] I would propose skipping the `Optional` wrapper on this type signature, and ask implementations which do not handle metadata to use `= LoggingMetadata()` (which is exactly `[:]` ) as the "nil object" instead. [...]
> 
> Actually, it being `nil` was more meant for `LogHandler` s that don't want to support logging meta-data at all (they would just always return `nil` from the getter and throw the value away in the setter). But maybe we just won't support that at all.  
> As you point out, removing everything is just `logger.metadata = [:]` or `logger.metadata.removeAll()`

Uff, was a bunch of back and forth on this one 🙂

I also chatted with Tom inside my "exploring" PR and I believe he has use cases in mind which make use of the optional to know if "empty was set" or nothing was set at all.

We discussed the "set or not" and "inheriting" styles in depth in: [[DO NOT MERGE] Exploring slight changes to APIs by ktoso · Pull Request #6 · weissi/swift-server-logging-api-proposal · GitHub](https://github.com/weissi/swift-server-logging-api-proposal/pull/6/files#r242418606)

And I think the conclusion is:

- in order to support people who would like to do the style of "fall back to a global one if nothing was set" as Tom proposes, the optional has to stay.
- the optional does not prevent anyone, semantically, from implementing the "more and more specific scopes that share the outer scopes context", so while it does not look pretty I can live with it and implement anything I'd like to.

**Proposed outcome:** As the proposal is about a generic API to enable all kinds of implementations, I think it is fine to let the optional stay.

* * *

> [@johannesweiss](#):
>
> > **Metadata: temporary metadata**
> > 
> > The just mentioned by @artemredkin addition for per log statement metadata may be fine, however it may be worth adding this as functionality instead as a scope: `.withMetadata(@autoclosure ...) { within this block that metadata is added }` instead.
> > 
> > This allows for nested calls within such block to work properly – in addition to just "the calls I know about". This shows up again in tracing examples where sub calls should expose the same "request id" or similar as as the encapsulating call.
> 
> This would require us to store the information in a thread-local variable and I'm not sure if we should offer this with the base API. You could build that as library functionality but I see how it might be nicer but my personal opinion is still to start without it and to maybe add it at a later stage. What do you think?

For reference: I don't think we's need thread locals for this. Examples follow.  
Though I think it would also be possible to leave this our of the MVP which this logging proposal is setting out to define, and one may add such extensions via a library or on their own if someone would like this.

Concurrent writers are assumed not to be present in these btw – as some other means of achieving synchronization would be in place anyway – so I think this is a fair assumption.

So a quick rough sketch of what it might be could be something like:

```swift
public extension Logger {

    // I think I prefer option 2, since it is "as lazy as it gets"

    // option 1) requires mutating outer logger; I think this is less preferable in general
    mutating public func withMetadata(_ additionalMetadata: @autoclosure () -> LoggingMetadata, _ block: (Logger) -> Void) {
        let old = self.metadata
        self.metadata.merge(additionalMetadata(), uniquingKeysWith: { (l, r) in r })
        defer { self.metadata = old }

        return block(self)
    }

    // option 2) creates composite logger; difference is that "per log call" can set/reset the additional metadata...
    // does not protect from concurrent writes in any case though (not that it should be a goal IMHO anyway though)
    public func withMetadataDelegate(_ additionalMetadata: @escaping @autoclosure () -> LoggingMetadata, _ block: (Logger) -> Void) {
        let proxyHandler = ProxyCompositeMetadata(underlying: self.handler, additionalMetadata: additionalMetadata)

        return block(Logger(proxyHandler))
    }
}
final class ProxyCompositeMetadata: LogHandler {
    private var underlying: LogHandler
    private let additionalMetadata: () -> LoggingMetadata

    private lazy var compositeMetadata: LoggingMetadata? = nil

    init(underlying: LogHandler, additionalMetadata: @escaping @autoclosure () -> LoggingMetadata) {
        self.underlying = underlying
        self.additionalMetadata = additionalMetadata
    }

    func log(level: LogLevel, message: String, file: String, function: String, line: UInt) {
        if level >= self.logLevel {
            let old = self.underlying.metadata

            // attempt avoiding to have to re merge the metadata maps all the time
            let composedMeta: LoggingMetadata
            switch self.compositeMetadata {
            case .some(let meta):
                // we already composed the maps once, lets reuse that
                composedMeta = meta
            case .none:
                composedMeta = old.merging(self.additionalMetadata(), uniquingKeysWith: { (l, r) in r })
                self.compositeMetadata = composedMeta
            }

            self.underlying.metadata = composedMeta
            defer { self.underlying.metadata = old } // reset on a "call by call" basis
            self.underlying.log(level: level, message: message, file: file, function: function, line: line)
        }
    }

    private var _logLevel: LogLevel = .info
    public var logLevel: LogLevel {
        get {
            return self.underlying.logLevel
        }
        set {
            self.underlying.logLevel = newValue
        }
    }

    public var metadata: LoggingMetadata {
        get {
            return self.underlying.metadata
        }
        set {
            self.underlying.self.metadata = newValue
        }
    }

    public subscript(metadataKey metadataKey: LoggingMetadata.Key) -> LoggingMetadata.Value? {
        get {
            return self.underlying.metadata[metadataKey]
        }
        set {
            self.underlying.metadata[metadataKey] = newValue
        }
    }
}

```

Usage being:

```swift

outerLoggerVar.withMetadataDelegate(["hello": "helloooooooo!"]) { scoped in
    scoped.info("EXAMPLE withMetadataDelegate var logger: with additional meta")
    scoped.info("EXAMPLE withMetadataDelegate var logger: with additional meta")
    scoped.info("EXAMPLE withMetadataDelegate var logger: with additional meta")
}
outerLoggerVar.info("EXAMPLE withMetadataDelegate var logger: No more `hello` value in meta")

outerLoggerLet.withMetadataDelegate(["hello": "helloooooooo!"]) { scoped in
    scoped.info("EXAMPLE withMetadataDelegate let logger: with additional meta")
    scoped.info("EXAMPLE withMetadataDelegate let logger: with additional meta")
    scoped.info("EXAMPLE withMetadataDelegate let logger: with additional meta")
}
outerLoggerLet.info("EXAMPLE withMetadataDelegate let logger: No more `hello` value in meta")

// requires `var`, since we mutate directly
outerLoggerVar.withMetadata(["hello": "helloooooooo!"]) { scoped in
    scoped.info("EXAMPLE requires var, mutates outer: with additional meta")
    scoped.info("EXAMPLE requires var, mutates outer: with additional meta")
    scoped.info("EXAMPLE requires var, mutates outer: with additional meta")
    scoped.info("EXAMPLE requires var, mutates outer: with additional meta")

    // direct overlapping access is protected as:
    // outerLoggerVar.info("NOPE")
    // /Users/ktoso/code/swift-server-logging-api-proposal/Sources/Examples/ExploringPerformanceExample.swift:215:13: error: overlapping accesses to 'outerLoggerVar', but modification requires exclusive access; consider copying to a local variable
    // outerLoggerVar.withMetadata(["hello": "helloooooooo!"]) { scoped in
    // ^ ~~~~~~~~~~~~~
    // /Users/ktoso/code/swift-server-logging-api-proposal/Sources/Examples/ExploringPerformanceExample.swift:221:17: note: conflicting access is here
    // outerLoggerVar.info("NOPE")
    // ^ ~~~~~~~~~~~~~
}
outerLoggerVar.info("EXAMPLE: No more `hello` value in meta")

// [2018-12-18 06:22:44 +0000][info][shared-meta=shared_YAY hello=helloooooooo!] EXAMPLE withMetadataDelegate var logger: with additional meta
// [2018-12-18 06:22:44 +0000][info][shared-meta=shared_YAY hello=helloooooooo!] EXAMPLE withMetadataDelegate var logger: with additional meta
// [2018-12-18 06:22:44 +0000][info][shared-meta=shared_YAY hello=helloooooooo!] EXAMPLE withMetadataDelegate var logger: with additional meta
// [2018-12-18 06:22:44 +0000][info][shared-meta=shared_YAY] EXAMPLE withMetadataDelegate var logger: No more `hello` value in meta
// [2018-12-18 06:22:44 +0000][info][hello=helloooooooo!] EXAMPLE withMetadataDelegate let logger: with additional meta
// [2018-12-18 06:22:44 +0000][info][hello=helloooooooo!] EXAMPLE withMetadataDelegate let logger: with additional meta
// [2018-12-18 06:22:44 +0000][info][hello=helloooooooo!] EXAMPLE withMetadataDelegate let logger: with additional meta
// [2018-12-18 06:22:44 +0000][info][] EXAMPLE withMetadataDelegate let logger: No more `hello` value in meta
// [2018-12-18 06:22:44 +0000][info][shared-meta=shared_YAY hello=helloooooooo!] EXAMPLE requires var, mutates outer: with additional meta
// [2018-12-18 06:22:44 +0000][info][shared-meta=shared_YAY hello=helloooooooo!] EXAMPLE requires var, mutates outer: with additional meta
// [2018-12-18 06:22:44 +0000][info][shared-meta=shared_YAY hello=helloooooooo!] EXAMPLE requires var, mutates outer: with additional meta
// [2018-12-18 06:22:44 +0000][info][shared-meta=shared_YAY hello=helloooooooo!] EXAMPLE requires var, mutates outer: with additional meta
// [2018-12-18 06:22:44 +0000][info][shared-meta=shared_YAY] EXAMPLE: No more `hello` value in meta

```

**Summing up "scoped metadata"** : The existing API proposal seems to be enough to roll-your-own this feature. That was another good sanity check that it's flexible enough, I suppose.

I do _not_ think it "has to" ship with the core library as keeping it as minimal as it gets would be the best course of action here I think, since it is to be shared across many many libraries, any additional method is a risk of introducing some edge cases.

If we end up deciding to ship such methods with the APIs I'm fine with that though, anything similar to the above would likely be fine. The worry about including it comes from that the pattern to do these things may change slightly if we get "context object" style things, though we would manage to deprecate and move to contexts if ever that needed to be the case I think.

* * *

And finally **context** and relation to logger metadata:

I have to say that the general pattern of "context exposes logger" is absolutely something I'd subscribe to.  
It would also allow avoiding that "forces strings" issue that I yammer about currently, since the context could be well-typed, and only rendered into strings when we actually hit a log() (that does trigger, i.e. not filtered out by log level).

I explored this quite in depth today and can see why you are worried about putting "mdc" style here, if we are to consider a future more general Context abstraction. I sadly don't yet have a good idea what the best approach here is, migrating if we ever do context or co-existing.

Traces or and similar indeed do not live "inside loggers", though you may want to log some of the ids, so there is some interplay in place here.

I don't have a great answer here yet, but will try to experiment some more. In general though I'm a big fan of not letting the "perfect be the enemy of good", and a plain simple logging API as we discuss here is something I absolutely subscribe to, regardless when or if-at-all we do context separately as you mentioned @johannesweiss.

* * *

**Errata to previous nitpicks**

Retracting naming nitpick:

> [@ktoso](#):
>
> `Logging.make("hello.world")` – Not in love with the "make" in the name... reads very weird.

I retract the complaint about the `make` method name btw. Just reminded myself that that is following the standard naming conventions: [API Design Guidelines | Swift.org](https://swift.org/documentation/api-design-guidelines/#strive-for-fluent-usage)  
That's definitely the naming to use then it seems, I had forgotten that part of the style guide 👍

---

<div class="post-metadata">

**Author:** ![johannesweiss](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/johannesweiss/32/38183_2.png) [@johannesweiss](https://forums.swift.org/u/johannesweiss)\
**Post date:** [December 18, 2018, 11:23am UTC](https://forums.swift.org/t/discussion-server-logging-api/18834/47 "2018-12-18T11:23:52Z")

</div>

Sorry for the delayed response and thank you for the detailed thoughts!

> [@gwynne](#):
>
> ## Formatting
> 
> On reflection, I find myself oddly ambivalent about how I think formatting - especially for metadata - should be handled. Consider these examples:
> 
> ```swift
> protocol CustomLoggerMetadataConvertible {
> var descriptionForLogging: String { get }
> }
> extension CustomLoggerMetadataConvertible {
> /// Default to simple description for all objects
> var descriptionForLogging: String { return String(describing: self) }
> }
> 
> struct MyComplexContextuallyImportantThing { /* ... */ }
> extension MyComplexContextuallyImportantThing: CustomLoggerMetadataConvertible {
> var descriptionForLogging: String { return "\(/* machine-parseable logging format */)" }
> }
> extension MyComplexContextuallyImportantThing: CustomStringConvertible {
> var description: String { return "\(/* human-readable format */)" }
> }
> 
> //...
> let context = MyComplexContextuallyImportantThing(/*blah*/)
> myLogger[metadata: context]
> // OR, remove the `CustomLoggerMetadataConvertible` conformance and:
> myLogger[metadata: "\(context./*logging format*/)"]
> 
> ```
> 
> The two different ways of setting the context as metadata have some interesting tradeoffs:
> 
> - The protocol conformance allows the same magic that `CustomStringConvertible` does for `String.init(describing:)` : Each type knows how best to represent itself for logging and can do so without that knowledge leaking into the logger setup.
> - On the other hand, the protocol conformance represents (a very constrained form of) spooky action at a distance - the code setting the metadata has no control over how a given object may represent itself and changes in that representation may break things.
> - Using a new protocol instead of `CustomStringConvertible` allows an object to represent itself differently inside a logging context than outside, and the default implementation avoids needing anything other than the conformance declared fro objects that don't care.
> - You still end up needing the conformance, unless the subscript takes `Any` and does `as?` checks for conformance at runtime.
> 
> This is ultimately the same problem as shows up in any use of conformances - the decision about where the control should be is complicated. I see the metadata subscript as having several possible signatures:
> 
> ```swift
> // Strings are represented directly
> public subscript(metadataKey metadataKey: String) -> String? { get set }
> 
> // `String(describing:)` is used
> public subscript(metadataKey metadataKey: Any) -> Any? { get set }
> 
> // Same as previous, but also checking for a `CustomLoggerMetadataConvertible` conformance, and using it if present
> public subscript(metadataKey metadataKey: Any) -> Any? { get set }
> 
> // Only conforming types may be used - good thing `CustomLoggerMetadataConvertible` doesn't have Self or associated type requirements!
> public subscript(metadataKey metadataKey: CustomLoggerMetadataConvertible) -> CustomLoggerMetadataConvertible? { get set }
> 
> // Same as above, but using `CustomStringConvertible` instead of a separate protocol
> public subscript(metadataKey metadataKey: CustomStringConvertible) -> CustomStringConvertible? { get set }
> 
> ```
> 
> I'm not sure where I personally land on this one. I like `CustomLoggerMetadataConvertible` (name open to debate of course 🙂), but I don't like having to conform _all_ metadata I might use to it and I also don't like the idea of the accessor having to take and return `Any` . Maybe have two overloads, where the second one takes `String` or `CustomStringConvertible` , though that obviously complicates `LoggingMetadata` 's internal representation.

I know this might be a bit annoying but I feel we should pitch these kinds of things separately in case the base + smaller changes get accepted. There will then still be time before the API will be tagged 1.0.0. In case this proposal proceeds, it'll first go to the ['sandbox'](https://github.com/swift-server/sswg/blob/master/process/incubation.md#process-diagram) level where people are invited to try it out but we can still make changes before it even reaches 'incubation'.

> [@gwynne](#):
>
> ## LogHandler API
> 
> Log handlers need `func openLog(file: String, function: String, line: UInt)` (maybe without the parameters, I'm not sure how useful they are in this context) so logs that care about/need initialization after `init()` but before the first log message aren't forced to maintain a "did I do my extra setup yet" flag.
> 
> There's also potential room for `func closeLog(/* etc */)` , with the caveat that a log handler can never rely on the close function being called since it wouldn't be in the case of sudden, abnormal, or crashing termination. ... In fact, come to think of it, that caveat is so significant and so easy to accidentally ignore (there's certainly no way for the compiler to enforce it!) that it's probably better not to provide a close method at all. I'm only leaving the thought in this post for the sake of transparency of thought process (in other words I don't want anyone to accuse me of not thinking of it 😆).

I think you talked yourself out of `closeLog` and I think I agree because we can't have logging implementation only flush in `closeLog` because otherwise we'd lose messages just before a crash...

Regarding `openLog`, I think you're touching on something that we should really think through. It's very possible that a `LogHandler` may need to perform some initialisation before being able to be used. I think we should probably have some function (maybe called `openLog`) called at the time a `LogHandler` is 'installed' through `Logging.bootstrap`. [filed here](https://github.com/weissi/swift-server-logging-api-proposal/issues/8)

> [@gwynne](#):
>
> ## Custom log levels?
> 
> An additional issue I find I run into in real-world usage is that the predefined log levels provided by a given logging library are almost never perfectly suited (often not even well-suited) to the needs of my project. For example, in Vapor-based services I tend to use `LogLevel.custom("database")` to enable separating (often very noisy) database query logging from other logging. Obviously this is difficult to make cleanly extensible in Swift. I find Vapor's `.custom(String)` API undesirable due to being stringly-typed - I always end up wrapping it in a non-trivial amount of extra logic to avoid accidental misuse, and that still doesn't address the problem possible memory and performance issues from adding a case with a payload to the level enumeration.
> 
> The problem is I don't know what an alternative to the stringly-typed case would be. Even if pure Swift supported "open" enums (which it doesn't), conflicts between custom cases would be immediately apparent between modules (which is part of why open enums aren't a thing, if I recall!). This is also a problem with the stringly-typed method. Namespacing of log levels would tend to be painful to use, and reading the results as much or more so. This might be addressable by making `LogLevel` a `struct` (in the style of `OptionSet` ) instead of an `enum` and working out a means by which conflicts would be minimized.
> 
> The real question is, at that point, is it worth the complexity? I find that it is, but I don't know that it would be for the majority of projects.

I remain unconvinced that custom logging levels are worth the complexity. Your example with the Vapor database system is a good one but I believe you can achieve the same thing in two slightly different ways depending on how you plan to use the logger:

- if you go for the explicitly passing the logger around model, I'd just add a `logger[metadataKey: "subsystem"] = "com.vapor.database"` and you'll later be able to filter out all database messages easily
- if you go for the 'one logger per sub-system' then you'd have the Vapor database layer create a logger with identifier `com.vapor.database` and again, the filtering should be rather straightforward.

Think that would work?

> [@gwynne](#):
>
> ## Logging configuration
> 
> This brings me to the other feature I find myself missing consistently in Vapor's support that this proposal doesn't yet address: Configurability of log levels. This covers:
> 
> - Do I log messages at this level at all?
> - Are log levels strongly ordered (so that a `currentLogLevel` of `warn` covers both `warn` and `error` but nothing else) or individually controlled (such that the current level would be represented by something like `Set<LogLevel>` )? If the latter, is the "level" more accurately described as a "type"?

Yes, they are strongly ordered and if the log-level is at `warn`, the `LogHandler` won't even see any messages that are below `warn`. This was a deliberate decision so that we don't need to forward the `@autoclosure` to the existential `LogHandler` type. Why? Because currently `@autoclosure` will _alloctate_ if the called function cannot be inlined. And you can never inline across an existential (because well the implementation is chosen at runtime...).

There is however quite an easy workaround: Your log handler can accept setting log levels (and store that) but when asked about the log level it just always returns `.trace`. This way the `LogHandler` can see all messages and can optionally add some fancier stuff.

> [@gwynne](#):
>
> - Is there value in an additional field such as "subsystem" and/or "category" (as found in `os_log()` ) for providing (even) more control?

I believe that convention and logging meta-data is good enough here. So if everybody uses the `"subsystem"` key, it should work. We could also (later) add the strings as proper constants to the logging API.

> [@gwynne](#):
>
> - Should this decision be left up to `LogHandler` s or be encapsulated by the `Logger` ?
> - Is "current level" global, per- `Logger` , per- `LogHandler` , or something else?

it _depends_. There are so many different approaches to logging that this needs to be decided by the `LogHandler`s themselves. Maybe in the future one way of logging will 'win' and we can adjust this but currently this is deliberately left to the `LogHandler`.

> [@gwynne](#):
>
> I'm not sure whether or not this is even in scope for this proposal, but I thought it worth mentioning. If it is, I do have some thoughts on API for it, but it's not sensible to take the time for that unless it's in scope and until some of the questions are discussed.

If it covers the basic API I think it's worth mentioning. For example I'm thinking about (in a separate thread) asking the community if they like

`logger.info("Hello World")` and `logger.error("bad")` (as proposed so far)

or

`logger.log(level: .info, "Hello World")` and `logger.log(level: .error, "bad")`

better.

> [@gwynne](#):
>
> ## And so on
> 
> I could keep going with more ideas, there's so many things a really "robust" logging API might do, but I think I'm already on the knife edge of the complexity threshold - one of the explicit design goals of having this in SSWG is to make it both simple _and_ robust, after all.

Yes, I think you're right here. I think we should start with something that we feel is good enough and over time we can evolve.

---

<div class="post-metadata">

**Author:** ![gwynne](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/gwynne/32/24337_2.png) [@gwynne](https://forums.swift.org/u/gwynne)\
**Post date:** [December 18, 2018, 12:38pm UTC](https://forums.swift.org/t/discussion-server-logging-api/18834/48 "2018-12-18T12:38:01Z")

</div>

> [@johannesweiss](#):
>
> Sorry for the delayed response and thank you for the detailed thoughts!
> 
> > [@gwynne](#):
> >
> > ## Custom log levels?
> > 
> > [snip]
> 
> I remain unconvinced that custom logging levels are worth the complexity. Your example with the Vapor database system is a good one but I believe you can achieve the same thing in two slightly different ways depending on how you plan to use the logger:
> 
> - if you go for the explicitly passing the logger around model, I'd just add a `logger[metadataKey: "subsystem"] = "com.vapor.database"` and you'll later be able to filter out all database messages easily
> - if you go for the 'one logger per sub-system' then you'd have the Vapor database layer create a logger with identifier `com.vapor.database` and again, the filtering should be rather straightforward.
> 
> Think that would work?

I do somewhat, but I have concerns about using the metadata in particularly this way - more on that below. I would almost always prefer to do "one logger per subsystem", but more than once in practice it's turned out that I have a lot of trouble separating out what logs belong to what subsystem, even with very well-isolated abstractions. In particular, really good abstraction layers are not something I tend to have during the early stages of a project's development - I find myself going back and changing how my logging is organized over and over and over.

The biggest hurdle with the "one logger per subsystem" model, in my experience, is keeping the logger objects accessible to the contexts using them. Calling `Logger("com.exmaple.foo")` everywhere that wants that subsystem (even if the API supported using it this way, which I wouldn't want it to!) presupposes either extremely cheap initialization, a global cache (with all the myriad attendant issues of _that_), or compile-time evaluation, none of which are desirable requirements. At the same time, passing `Logger`s around tends to demand certain compromises in design - vars or functions in global scope, objects that conform to a hypothetical `ContainsContextualLogger` protocol, ivars/methods with mostly repeated implementations in many different files and/or objects, and so on. Refactoring becomes increasingly messy. And I don't know if this has been brought up in this thread already, but there's also the question of preserving contextual metadata from one subsystem in a `Logger` that belongs to a "lower-level" subsystem - so now you need to be able to do `Logger("com.example.bar", inheriting: fooLogger)`, or carefully build your subsystems to have the needed context or not need to log it without losing the specificity, or otherwise have a `LogContext` (which I know I've seen mentioned somewhere in here).

As unfortunately usual, I don't have a good suggestion for solutions to the issue, I'd need to spend some time tinkering. I just don't want it to pass unmentioned 😕

> [@johannesweiss](#):
>
> > [@gwynne](#):
> >
> > ## Logging configuration
> > 
> > [snip]
> > 
> > - Are log levels strongly ordered (so that a `currentLogLevel` of `warn` covers both `warn` and `error` but nothing else) or individually controlled (such that the current level would be represented by something like `Set<LogLevel>` )? If the latter, is the "level" more accurately described as a "type"?
> 
> Yes, they are strongly ordered and if the log-level is at `warn`, the `LogHandler` won't even see any messages that are below `warn`. This was a deliberate decision so that we don't need to forward the `@autoclosure` to the existential `LogHandler` type. Why? Because currently `@autoclosure` will _alloctate_ if the called function cannot be inlined. And you can never inline across an existential (because well the implementation is chosen at runtime...).

I've read over this several times and still can't quite work out in my head why the `@autoclosure` would need to be forwarded. I think I'm missing a piece somewhere in the flow of data 😅. In fact, re-reading this again, I'm not quite clear on which thing you were saying was deliberate by design: "passing only the configured messages to the handler", or "having only one configured logging level rather than a set". Can you clarify?

> [@johannesweiss](#):
>
> There is however quite an easy workaround: Your log handler can accept setting log levels (and store that) but when asked about the log level it just always returns `.trace`. This way the `LogHandler` can see all messages and can optionally add some fancier stuff.

I like this - though in other logging systems I've seen, it's defined by API contract that a `LogHandler` _always_ receives _all_ log messages, regardless of level, and can decide what to do with them on that basis. I don't know how that would look for performance, but it has the virtue of both simplicity and consistency.

> [@johannesweiss](#):
>
> > [@gwynne](#):
> >
> > - Is there value in an additional field such as "subsystem" and/or "category" (as found in `os_log()` ) for providing (even) more control?
> 
> I believe that convention and logging meta-data is good enough here. So if everybody uses the `"subsystem"` key, it should work. We could also (later) add the strings as proper constants to the logging API.

This is one where I'm just not sure I agree. Well-known subscripts to an arbitrary metadata map are still stringly-typed data, and that level of flexibility is typically most useful to clients of an API, not the API itself. If the API goes to the trouble to define "well-known" metadata in the first place, it should make that metadata part of the API contract and provide explicit properties. This preserves the maximum amount of static type information, keeps the metadata namespace unambiguous, and makes it less likely that clients will accidentally miss the existence of the keys and end up pointlessly duplicating the functionality.

That being said, this is certainly something that would stand outside the scope of the first version/draft of the API - it deals much more in concrete implementations than protocols. I would say file an issue for this and address it later, but _do_ address it.

> [@johannesweiss](#):
>
> > [@gwynne](#):
> >
> > - Should this decision be left up to `LogHandler` s or be encapsulated by the `Logger` ?
> > - Is "current level" global, per- `Logger` , per- `LogHandler` , or something else?
> 
> it _depends_. There are so many different approaches to logging that this needs to be decided by the `LogHandler`s themselves. Maybe in the future one way of logging will 'win' and we can adjust this but currently this is deliberately left to the `LogHandler`.

On this one, I'm pretty sure I come down on the side of leaving it to the `LogHandler`, especially if we end up saying that handlers always receive all messages. I don't see a need to provide extra guidance from the API for it, all that could ever do is limit the API's capabilities.

> [@johannesweiss](#):
>
> > [@gwynne](#):
> >
> > I'm not sure whether or not this is even in scope for this proposal, but I thought it worth mentioning. If it is, I do have some thoughts on API for it, but it's not sensible to take the time for that unless it's in scope and until some of the questions are discussed.
> 
> If it covers the basic API I think it's worth mentioning. For example I'm thinking about (in a separate thread) asking the community if they like
> 
> `logger.info("Hello World")` and `logger.error("bad")` (as proposed so far)
> 
> or
> 
> `logger.log(level: .info, "Hello World")` and `logger.log(level: .error, "bad")`
> 
> better.

Well, I certainly can't speak for the community as a whole 🙂. But speaking for myself, I very much prefer the former (current) style as bother more "fluent" and more terse. Ultimately, I would say "provide both but visibly prefer the shorter form".

---

<div class="post-metadata">

**Author:** ![johannesweiss](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/johannesweiss/32/38183_2.png) [@johannesweiss](https://forums.swift.org/u/johannesweiss)\
**Post date:** [December 18, 2018, 1:18pm UTC](https://forums.swift.org/t/discussion-server-logging-api/18834/49 "2018-12-18T13:18:29Z")

</div>

> [@gwynne](#):
>
> > I remain unconvinced that custom logging levels are worth the complexity. Your example with the Vapor database system is a good one but I believe you can achieve the same thing in two slightly different ways depending on how you plan to use the logger:
> > 
> > - if you go for the explicitly passing the logger around model, I'd just add a `logger[metadataKey: "subsystem"] = "com.vapor.database"` and you'll later be able to filter out all database messages easily
> > - if you go for the 'one logger per sub-system' then you'd have the Vapor database layer create a logger with identifier `com.vapor.database` and again, the filtering should be rather straightforward.
> > 
> > Think that would work?
> 
> I do somewhat, but I have concerns about using the metadata in particularly this way - more on that below. I would almost always prefer to do "one logger per subsystem", but more than once in practice it's turned out that I have a lot of trouble separating out what logs belong to what subsystem, even with very well-isolated abstractions. In particular, really good abstraction layers are not something I tend to have during the early stages of a project's development - I find myself going back and changing how my logging is organized over and over and over.

Right, but that's not solved by custom log levels either, right?

> [@gwynne](#):
>
> The biggest hurdle with the "one logger per subsystem" model, in my experience, is keeping the logger objects accessible to the contexts using them. Calling `Logger("com.exmaple.foo")` everywhere that wants that subsystem (even if the API supported using it this way, which I wouldn't want it to!) presupposes either extremely cheap initialization, a global cache (with all the myriad attendant issues of _that_ ), or compile-time evaluation, none of which are desirable requirements. At the same time, passing `Logger` s around tends to demand certain compromises in design - vars or functions in global scope, objects that conform to a hypothetical `ContainsContextualLogger` protocol, ivars/methods with mostly repeated implementations in many different files and/or objects, and so on. Refactoring becomes increasingly messy. And I don't know if this has been brought up in this thread already, but there's also the question of preserving contextual metadata from one subsystem in a `Logger` that belongs to a "lower-level" subsystem - so now you need to be able to do `Logger("com.example.bar", inheriting: fooLogger)` , or carefully build your subsystems to have the needed context or not need to log it without losing the specificity, or otherwise have a `LogContext` (which I know I've seen mentioned somewhere in here).

Yes, these are all valid issues. I personally am the 'passing the logger around explicitly' kind of person which makes the contextual logging fairly straightforward and `Logger`s can be value types etc. But it does make it harder to have per sub-system logging messages. Sometimes you can use `#file` as a proxy for what sub-system it is but it's not great either. The thing is that I don't think there's a good answer for everybody, we just can't force everybody to pass loggers around explicitly nor can we tell people that that's not supported.

If you want to have both 'sub-system' loggers and contextual logging then I think the only way is to built the context separately and then pass it onto the logging methods manually (with added `metadata:` parameters to every log method, like @artemredkin was proposing up-thread). Or obviously MDC which propagates the contextual information through thread-locals but sub-system loggers + MDC + Dispatch == needs-manual-assistance propagating the context 🙂

> [@gwynne](#):
>
> As unfortunately usual, I don't have a good suggestion for solutions to the issue, I'd need to spend some time tinkering. I just don't want it to pass unmentioned 😕

I know, there's no perfect solutions here that work for every architecture and belief...

> [@gwynne](#):
>
> ![](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/johannesweiss/40/684_1.png) johannesweiss:
> 
> > ![](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/graskind/40/5587_1.png) graskind:
> > 
> > > ## Logging configuration
> > > 
> > > [snip]
> > > 
> > > - Are log levels strongly ordered (so that a `currentLogLevel` of `warn` covers both `warn` and `error` but nothing else) or individually controlled (such that the current level would be represented by something like `Set<LogLevel>` )? If the latter, is the "level" more accurately described as a "type"?
> > 
> > Yes, they are strongly ordered and if the log-level is at `warn` , the `LogHandler` won't even see any messages that are below `warn` . This was a deliberate decision so that we don't need to forward the `@autoclosure` to the existential `LogHandler` type. Why? Because currently `@autoclosure` will _alloctate_ if the called function cannot be inlined. And you can never inline across an existential (because well the implementation is chosen at runtime...).
> 
> I've read over this several times and still can't quite work out in my head why the `@autoclosure` would need to be forwarded. I think I'm missing a piece somewhere in the flow of data 😅. In fact, re-reading this again, I'm not quite clear on which thing you were saying was deliberate by design: "passing only the configured messages to the handler", or "having only one configured logging level rather than a set". Can you clarify?

Sure, at the moment in Swift, both `@autoclosure` and closure arguments (with non-`nil` contexts) will allocate (ie. call `malloc`). So if you have a `ModuleA` which defines `func foo(_ body: () -> String)` and you call it from `ModuleB` as

```swift
let a = "hello"
foo {
  return a
}

```

an allocation will happen when `foo` is called for the closure's context (the captured variables). `@autoclosure`s are the same, they just are a nicer syntax for this. The only way to get around that is inlining. Ie. making it `@inlinable func foo(_ body: () -> String)`. That way the compiler will (hopefully) inline foo's body into the caller and it's no longer actually building a closure.

That's why `Logger` is a `struct` and all its methods are inlineable: Within the (hopefully) inlined code we know if we need to produce the log message string (by evaluating the `@autoclosure`) or not. And only after having decided that we do need to log, we evaluate the `@autoclosure` and pass the result on as a `String` to the `LogHandler`. So with a bit of support from the inlining gods, we no longer need to allocate.

If we passed the `@autoclosure` straight through to the `LogHandler`, it can't inline that and we do actually need to allocate for the closure. And there's no way we can inline the `LogHandler` implementation into the caller because the `LogHandler` is selected at _runtime_.

Really, this should be addressed in the compiler and the closure context should be stack allocated (because the closure is non-escaping) but that [hasn't been done AFAIK](https://bugs.swift.org/browse/SR-904).

> [@gwynne](#):
>
> ![](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/johannesweiss/40/684_1.png) johannesweiss:
> 
> > There is however quite an easy workaround: Your log handler can accept setting log levels (and store that) but when asked about the log level it just always returns `.trace` . This way the `LogHandler` can see all messages and can optionally add some fancier stuff.
> 
> I like this - though in other logging systems I've seen, it's defined by API contract that a `LogHandler` _always_ receives _all_ log messages, regardless of level, and can decide what to do with them on that basis. I don't know how that would look for performance, but it has the virtue of both simplicity and consistency.

Agreed, but then every log message would be an allocation, if actually logged or not. That's unacceptable for a non-starter for now. I'd hope that this is fixed in Swift rather sooner than later but I think we need a solution before that. For more information, check [this thread](https://forums.swift.org/t/zero-cost-service-provider-interface-signature-packages/6935), especially the [newer messages](https://forums.swift.org/t/zero-cost-service-provider-interface-signature-packages/6935/8).

> [@gwynne](#):
>
> ![](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/johannesweiss/40/684_1.png) johannesweiss:
> 
> > ![](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/graskind/40/5587_1.png) graskind:
> > 
> > > - Is there value in an additional field such as "subsystem" and/or "category" (as found in `os_log()` ) for providing (even) more control?
> > 
> > I believe that convention and logging meta-data is good enough here. So if everybody uses the `"subsystem"` key, it should work. We could also (later) add the strings as proper constants to the logging API.
> 
> This is one where I'm just not sure I agree. Well-known subscripts to an arbitrary metadata map are still stringly-typed data, and that level of flexibility is typically most useful to clients of an API, not the API itself. If the API goes to the trouble to define "well-known" metadata in the first place, it should make that metadata part of the API contract and provide explicit properties. This preserves the maximum amount of static type information, keeps the metadata namespace unambiguous, and makes it less likely that clients will accidentally miss the existence of the keys and end up pointlessly duplicating the functionality.
> 
> That being said, this is certainly something that would stand outside the scope of the first version/draft of the API - it deals much more in concrete implementations than protocols. I would say file an issue for this and address it later, but _do_ address it.

I still think that it might open a kitchen sink of information that one or the other person needs and (totally contrary to my normal opinions) I think here stringly typed is okay. Why? Because everybody I've seen on the server-side analyses logs with textual tools anyway. I mean we teach Splunk how to parse the messages that we serialised costly on the other end. But I don't want to say 'no' to this but for this proposal at this stage I don't think we should do it. If this goes forward and we have gained experience, then I think we can pitch enhancements but at this point I feel it's too risky to design features for a system that nobody has used in production quite yet.

> [@gwynne](#):
>
> ![](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/johannesweiss/40/684_1.png) johannesweiss:
> 
> > ![](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/graskind/40/5587_1.png) graskind:
> > 
> > > - Should this decision be left up to `LogHandler` s or be encapsulated by the `Logger` ?
> > > - Is "current level" global, per- `Logger` , per- `LogHandler` , or something else?
> > 
> > it _depends_ . There are so many different approaches to logging that this needs to be decided by the `LogHandler` s themselves. Maybe in the future one way of logging will 'win' and we can adjust this but currently this is deliberately left to the `LogHandler` .
> 
> On this one, I'm pretty sure I come down on the side of leaving it to the `LogHandler` , especially if we end up saying that handlers always receive all messages. I don't see a need to provide extra guidance from the API for it, all that could ever do is limit the API's capabilities.

yes

> [@gwynne](#):
>
> ![](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/johannesweiss/40/684_1.png) johannesweiss:
> 
> > ![](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/graskind/40/5587_1.png) graskind:
> > 
> > > I'm not sure whether or not this is even in scope for this proposal, but I thought it worth mentioning. If it is, I do have some thoughts on API for it, but it's not sensible to take the time for that unless it's in scope and until some of the questions are discussed.
> > 
> > If it covers the basic API I think it's worth mentioning. For example I'm thinking about (in a separate thread) asking the community if they like
> > 
> > `logger.info("Hello World")` and `logger.error("bad")` (as proposed so far)
> > 
> > or
> > 
> > `logger.log(level: .info, "Hello World")` and `logger.log(level: .error, "bad")`
> > 
> > better.
> 
> Well, I certainly can't speak for the community as a whole 🙂. But speaking for myself, I very much prefer the former (current) style as bother more "fluent" and more terse. Ultimately, I would say "provide both but visibly prefer the shorter form".

I do too but the same applies to me, can't speak for the community hence I'm considering asking (on a separate thread) if we should change. But that's really a small detail so maybe not now :).

---

<div class="post-metadata">

**Author:** ![neallester](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/neallester/32/4528_2.png) [@neallester](https://forums.swift.org/u/neallester)\
**Post date:** [December 18, 2018, 5:15pm UTC](https://forums.swift.org/t/discussion-server-logging-api/18834/50 "2018-12-18T17:15:50Z")

</div>

> [@johannesweiss](#):
>
> If it covers the basic API I think it's worth mentioning. For example I'm thinking about (in a separate thread) asking the community if they like
> 
> `logger.info("Hello World")` and `logger.error("bad")` (as proposed so far)
> 
> or
> 
> `logger.log(level: .info, "Hello World")` and `logger.log(level: .error, "bad")`

Sorry to reply to your question in this thread (If you do open another thread soon I will try to reply there also but may not be able to in a timely manner depending on when you post).

Including the .log(level: style would make it possible to make the Logger generic on LogLevel. This would add a lot of flexibility to the API and the .warn style API could still be provided through conditional extensions for the default LogLevel enum.

That said, I'm ok with a hard coded LogLevel provided that a **case emergency** is added (although I would prefer the levels I outlined previously).

To me, warning indicate events which may or may not require action, while errors generally do require corrective action. However, while one user encountered an unsupported edge case and the application can't reach the database may both require action (and thus classify as errors) they are different enough to justify different log levels.

---

<div class="post-metadata">

**Author:** ![Jon\_Shier](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/jon_shier/32/921_2.png) [@Jon\_Shier](https://forums.swift.org/u/Jon_Shier)\
**Post date:** [December 19, 2018, 2:11am UTC](https://forums.swift.org/t/discussion-server-logging-api/18834/51 "2018-12-19T02:11:53Z")

</div>

`os_log` has serious issues that prevent it from being used in apps, even if the API update suggested goes through. Console-only logs being useless for anyone who isn't a dev most of all. So this effort will likely be adopted by the larger community as the basis for their logging efforts. I think it's important to keep that in mind here (though I don't think such a consideration would really change anything so far).

---

<div class="post-metadata">

**Author:** ![nick.keets](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/nick.keets/32/2023_2.png) [@nick.keets](https://forums.swift.org/u/nick.keets)\
**Post date:** [December 19, 2018, 10:37am UTC](https://forums.swift.org/t/discussion-server-logging-api/18834/52 "2018-12-19T10:37:36Z")

</div>

> [@johannesweiss](#):
>
> Let's have a look at what is needed to go from nothing to a log message:
> 
> 1. create a package: `swift package init --type=executable`
> 2. add `.package(url: "https://github.com/weissi/swift-server-logging-api-proposal", .branch("master")),` to the `Package.swift` in `dependencies: [...]`
> 3. (assuming Xcode) `swift package generate-xcodeproj`
> 4. `xed .` or `open MyApp.xcodeproj` or open the project in Xcode
> 5. `import Logging`
> 6. `let logger = Logging.make("MyApp")`
> 7. `logger.info("Hello World")`
> 
> Is the only step we could avoid is step (6), is that one line really the make or break issue here? Especially that the documentation will need to tell you the module name (so you can `import Logging` ), we can just add to the documentation that if you don't want to learn anything more you just copy & paste

That sounds very shortsighted. I'm assuming Xcode will eventually integrate SPM, turning steps 1-5 into a few clicks. Also what about UI apps? What about CLI apps? Logging is something generally usefull, why artifically constrain this to pure server apps?

> [@johannesweiss](#):
>
> Stepping back, I don't think if it's two or three lines of code is the most interesting point of discussion because we're talking about a logging API for a whole server application. Even ignoring deploying this to the cloud or something you'll use some (probably web) framework like Vapor, Kitura, Perfect, smoke or so because otherwise your barrier to entry is likely parsing/serialising HTTP. The authors of those frameworks know their target audience best and if they feel logging is too hard, then they can always offer a very straightforward logging functionality, be it a free function, a static class or whatever they feel is the best. They can offer a global logger that logs under `com.company.framework.user-code` or something.

Again why are you constraining this to only "enterprisey" projects?

> [@johannesweiss](#):
>
> The door is also open for another package which could be called `JustLog` which nothing but a global `public func log(level: LogLevel = .info, message: String, logger: Logger = defaultGlobalLogger) { ... }` . Again, it would be a compatible solution that I wouldn't recommend but also have nothing against.

So you are advocating for left-pad style minimal functionality packages to cover the shortcomings of the builtin libraries? Or maybe follow Python's "lead" and go for Logging2 and Logging3 packages?

In general, I agree with @xwu. Logging is something that any app can use, from single-file scripts, to CLI apps, to iOS apps, to simple servers, to enterprise servers. By assuming that it is only going to be used by multi-layer, multi-framework enterprise server apps, you are compromising the usability of all other scenarios.

---

<div class="post-metadata">

**Author:** ![bhawkeslewis](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/bhawkeslewis/32/5616_2.png) [@bhawkeslewis](https://forums.swift.org/u/bhawkeslewis)\
**Post date:** [December 19, 2018, 12:40pm UTC](https://forums.swift.org/t/discussion-server-logging-api/18834/53 "2018-12-19T12:40:59Z")

</div>

@johannesweiss @tomerd Thanks for your efforts pushing this forward!

> [@johannesweiss](#):
>
> I still think that it might open a kitchen sink of information that one or the other person needs and (totally contrary to my normal opinions) I think here stringly typed is okay. Why? Because everybody I've seen on the server-side analyses logs with textual tools anyway. I mean we teach Splunk how to parse the messages that we serialised costly on the other end.

Log management systems like Splunk do not always operate on flat string-to-string maps.

A value in Splunk can be a list ([“multi-value”](https://docs.splunk.com/Splexicon:Multivaluefield) in Splunk-speak) and Splunk supports locators into deeply nested list and map structures from ingested XML or JSON (see the documentation for [`spath`](https://docs.splunk.com/Documentation/Splunk/7.2.1/SearchReference/Spath)).

[LogStash supports a fairly rich explicit type system](https://www.elastic.co/guide/en/logstash/current/configuration-file-structure.html#plugin-value-types) including lists, hashes, bytes, numbers, URIs, strings.

Some example use cases for complex metadata (lists and maps) include:

- A service fulfills user requests by spawning a tree of sub-requests to other services. When logging from those services, you want to include the request ancestry as list of request IDs in a `request_lineage` field, then query for all sub-requests of a node in the tree using that field when analyzing logs.

- A service enables users to share some content with a selection of users. You want to log the creation of the share including a list of users in a `sharees` field, then query for logs with specific sharees within that field when analyzing logs.

- A service consumes and produces events with complex structures. You want to produce logs that include triggering and generated events in an `event` field, then query within the events when analyzing logs.

Forcing structured data such as a list or dictionary into a string-to-string flat map implies an extra serialization/deserialization hop somewhere. For example, take a triggering event, serialize it to JSON, stick the JSON in a string in a log entry, serialize the log entry to JSON, then after deserializing the log entry from JSON also deserialize the nested event JSON on the other end.

For comparison purposes, structured logging libraries in other ecosystems tend to support complex values like Rust’s `slog` and Python’s `structlog`.

In the Swift ecosystem:

- [Nike’s Willow types log entry metadata as `[String: Any]`](https://github.com/Nike-Inc/Willow/blob/96a51f83e53c6e62877af08e0649d0b0a5de4d07/Source/LogMessage.swift#L33).

- [SwiftyBeaver types log entry metadata as `Any?`](https://github.com/SwiftyBeaver/SwiftyBeaver/blob/562a818/Sources/SwiftyBeaver.swift#L117).

I think it would be good at least to have a plan for how this API could be evolved to support complex metadata, even if it does not support it from the start.

---

<div class="post-metadata">

**Author:** ![johannesweiss](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/johannesweiss/32/38183_2.png) [@johannesweiss](https://forums.swift.org/u/johannesweiss)\
**Post date:** [December 19, 2018, 7:24pm UTC](https://forums.swift.org/t/discussion-server-logging-api/18834/54 "2018-12-19T19:24:33Z")

</div>

> [@bhawkeslewis](#):
>
> Forcing structured data such as a list or dictionary into a string-to-string flat map implies an extra serialization/deserialization hop somewhere. For example, take a triggering event, serialize it to JSON, stick the JSON in a string in a log entry, serialize the log entry to JSON, then after deserializing the log entry from JSON also deserialize the nested event JSON on the other end.

I agree with what you say and this API is very much built with JSON/XML/... logging in mind but as you say, currently only one level deep.

I’m fine lifting that restriction but I don’t think `[String: Any]` is a solution. It would be burdensome for the LogHandlers to handle sensibly. What types should they support? Mostly String and `[String: Any]` nested? If that’s what we’re after then we should make it explicit and the meta-data dict should be `[String: MetadataValue]` where `enum MetadataValue { case string(String); indirect case dictionary([String: MetadataValue])`.

We can make `MetadataValue` `StringLiteralConvertible` so people can pass string literals.

String:Any will just make it unclear for LogHandlers what they should do with the value but I can see why one would need more than one level deep meta-data dictionaries.

Let me know if that would help you.

---

<div class="post-metadata">

**Author:** ![PaulSolt](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/paulsolt/32/16477_2.png) [@PaulSolt](https://forums.swift.org/u/PaulSolt)\
**Post date:** [December 20, 2018, 1:33am UTC](https://forums.swift.org/t/discussion-server-logging-api/18834/55 "2018-12-20T01:33:02Z")

</div>

> [@tomerd](#):
>
> os\_log has serious performance optimizations on darwin, so if you are building an iOS or macOS app you should use it and not worry about this API

Yes, but I don't care for the "extreme" performance, I want _ease of use_ as an app developer. `os_log()` doesn't fill those needs as @davedelong mentioned in the other thread.

> [@](#):
>
> this API is designed for non-darwin and/or cross-platform code, so you can plug in different backend. see [background overview](https://forums.swift.org/t/server-logging-api/18834/17) above

Sorry if this is a beginner question. Out of the box, do I have to change the backend to get it functional?

Or can I just log to a file on the iOS filesystem, and then allow the user to email the file from their Documents directory (in-app)?

---

<div class="post-metadata">

**Author:** ![ktoso](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/ktoso/32/44526_2.png) [@ktoso](https://forums.swift.org/u/ktoso)\
**Post date:** [December 20, 2018, 5:40am UTC](https://forums.swift.org/t/discussion-server-logging-api/18834/56 "2018-12-20T05:40:35Z")

</div>

> [@johannesweiss](#):
>
> I’m fine lifting that restriction but I don’t think `[String: Any]` is a solution. It would be burdensome for the LogHandlers to handle sensibly. What types should they support? Mostly String and `[String: Any]` nested? If that’s what we’re after then we should make it explicit and the meta-data dict should be `[String: MetadataValue]` where `enum MetadataValue { case string(String); indirect case dictionary([String: MetadataValue])` .
> 
> We can make `MetadataValue` `StringLiteralConvertible` so people can pass string literals.
> 
> String:Any will just make it unclear for LogHandlers what they should do with the value but I can see why one would need more than one level deep meta-data dictionaries.

As the `[String: String]` remains my probably single remaining issue about the proposal I think it would indeed be good to do something about it. And have it be future proof, for a potential "context carrying" if/when it were to happen.

From my end, as library author, I'd be happy with any container that allows delaying the `-> String` rendering until it actually is used (if used at least once, we can cache the metadata string in the logger though; discussed in depth more with Tom in the "exploring" PR I made to the proposal). The rationale is to not penalize performance and have to avoid useful metadata that adds costs "up front", rather than costing only when it is in use _in logging_ (while it would always be there in "the context" but as the raw object, so no overhead there.

`MetadataValue` sounds fine as a container. However the `case string(String)` does _not_ solve the above issue, one still would always have to render the string when populating the metadata, regardless if it will be actually printed or not. It would have to be `: () -> String` I suppose; though we could offer both then, `string(:String)` and `lazyString(:() -> String)` perhaps?

Having that said, I need to do more actual benchmarking and “homework” before committing fully here... looking at some examples I have in mind and simple checks I did it seems it should definitely help... but I’ll do more homework on this soon (now stuck in airplane for many hours tho :-))

As for what "container types" to support in it I'm less opinionated. Perhaps just dictionary so things can express nesting would be good enough™. Seems to me like it is general enough to be able to cover many cases.

WDYT?

---

<div class="post-metadata">

**Author:** ![johannesweiss](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/johannesweiss/32/38183_2.png) [@johannesweiss](https://forums.swift.org/u/johannesweiss)\
**Post date:** [December 20, 2018, 8:45am UTC](https://forums.swift.org/t/discussion-server-logging-api/18834/57 "2018-12-20T08:45:55Z")

</div>

> [@PaulSolt](#):
>
> Sorry if this is a beginner question. Out of the box, do I have to change the backend to get it functional?

No, you don’t need to change it, by default it’ll just log to stdout.

> [@PaulSolt](#):
>
> Or can I just log to a file on the iOS filesystem, and then allow the user to email the file from their Documents directory (in-app)?

Yes, you can do that. That would require you to

`Logging.bootstrap(MyFileSystemLoggingHandler.self)`

You could either implement `class MyFileSystemLoggignHandler` yourself or more advisable you would use a package that provides such a LogHandler which formats it to your liking.

Why aren’t we providing that here? This is just the API, there will hopefully be plenty of concrete logging packages we will then work with all packages that adopt the API without further changes.

---

<div class="post-metadata">

**Author:** ![johannesweiss](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/johannesweiss/32/38183_2.png) [@johannesweiss](https://forums.swift.org/u/johannesweiss)\
**Post date:** [December 20, 2018, 8:59am UTC](https://forums.swift.org/t/discussion-server-logging-api/18834/58 "2018-12-20T08:59:41Z")

</div>

> [@ktoso](#):
>
> `MetadataValue` sounds fine as a container. However the `case string(String)` does _not_ solve the above issue, one still would always have to render the string when populating the metadata, regardless if it will be actually printed or not. It would have to be `: () -> String` I suppose; though we could offer both then, `string(:String)` and `lazyString(:() -> String)` perhaps?

Yes, that would work.

As discussed earlier with @artemredkin and others I’ll add a `metadata` parameter to each of the logging functions when I’ll put together a revision (very early next year) of this proposal. That parameter will also be `@autoclosure` which will delay the string rendering.

But I reckon you will also want the ability to get `logger[metadataKey: “foo”] = .lazy({ makeExpensiveString })`?

Performance-wise this should be okay as we’ll only allocate (the closure context) for all the lazy ones and not the strict ones... I’ll think about it but I think I can be convinced.

@gwynne you also asked for more general meta-data values, does this sound okay? @bhawkeslewis does this cover your use-cases? We could even make the `MetadataValue` both String and DictionaryLiteralConvertible so unless you’d use `.lazy` you wouldn’t even know it’s not an `Any`.

---

<div class="post-metadata">

**Author:** ![Alejandro\_Martinez](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/alejandro_martinez/32/821_2.png) [@Alejandro\_Martinez](https://forums.swift.org/u/Alejandro_Martinez)\
**Post date:** [December 20, 2018, 5:40pm UTC](https://forums.swift.org/t/discussion-server-logging-api/18834/59 "2018-12-20T17:40:16Z")

</div>

> [@PaulSolt](#):
>
> Is this only server logging?
> 
> Or could it be used in an iOS or macOS app?

> [@johannesweiss](#):
>
> Indeed. We should however point out that when [this](https://forums.swift.org/t/custom-string-interpolation-and-compile-time-interpretation-applied-to-logging/18799) lands, we should be able to adopt a few of the osLog optimisations for the server logging effort too and you should then be able to use osLog as a backend (LogHandler) with acceptable performance. Plain osLog however will always remain faster as it decides many things at compile-time that we will need to do at runtime because the server world needs more flexibility.

Although I'm really excited for server logging and I agree it's really important and needs maybe a different set of requirements, I would love to see it used also for application development.

I can see this flexibility be really good for apps to be able to send logs to different services (Fabric, analytics, inhouse, etc).

---

<div class="post-metadata">

**Author:** ![PaulSolt](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/paulsolt/32/16477_2.png) [@PaulSolt](https://forums.swift.org/u/PaulSolt)\
**Post date:** [December 21, 2018, 4:57pm UTC](https://forums.swift.org/t/discussion-server-logging-api/18834/60 "2018-12-21T16:57:03Z")

</div>

Thank you for working on this @johannesweiss and @tomerd

1. Are there any considerations that I need to keep in mind if I write a file system logging handler?

2. Maybe this is a beginner question, but is it common to put "main()" functions in enum types? What's the rational on that (from your tests)?

[Previous page](https://forums.swift.org/t/discussion-server-logging-api/18834.md?page=2)

[Next page](https://forums.swift.org/t/discussion-server-logging-api/18834.md?page=4)