Road to Tracing 1.0

After the introduction of Task-local values in Swift 5.5, Tracing is about to get some new improvement which will eventually lead to a stable 1.0 release.

This thread started from the Vapor 5 discussion and I (along with @graskind) would like to invite @ktoso to share the progress and see what the community can help:)


Is there a public link to the Vapor 5 discussion?

It started from the Vapor Discord server, you can join from:

1 Like

Hi there,
I can summarize where we stand right now. We've been collaborating on the final steps for quite a while with @slashmo and @fabianfett but we have not had the necessary "full time" to get it done just yet.

Specifically the things holding off the 1.0 is:

  • figuring out the logging story
    • Swift-log adoption is crucial for swift-distributed-tracing's success, and we need to find the right way forward for that adoption. We have some design principles in mind but have not proven them entirely just yet.
    • Part of this work is figuring out where and how we configure "which keys (from baggage), should the logger log". We had a few design ideas, like the metadata providers we PoCed here: [WIP] Metadata providers by slashmo ยท Pull Request #206 ยท apple/swift-log ยท GitHub Reviews and thoughts on this would be welcome. It is important that we can throw "any" baggage at the logging system, and it'll only log values the user intended to log; we cannot just log "everything".
    • The other part of the design is how logger metadata shall live with baggage; we're planning to do a writeup about this, but have not had the time yet. Long story short: baggage is the data "within a trace", like trace ids etc, but metadata can be "set" on a logger or backend and shall be static to that logger.

We need to figure out a good way to marry logging and the "revised" tracing APIs, now that the LoggingContext is gone (and good that it's gone, it was a workaround until we gained task-locals). As we do so, we intend to adopt it for logging in swift-async-http-client, and swift grpc. Doing so would prove that we got the designs right for NIO based internals, and we can easily confirm the other side that it works in "swift concurrency". Once we have this, we'd call a 1.0 and merge the adoption PRs into those projects.

We must find a good way to do so that does not incurs too many un-necessary copies, and "just works" when Swift Concurrency is used, while still allowing non-swift concurrency use (e.g. because NIO based internals, will not have this luxury, and they will have to pass baggage containing trace IDs etc).

Specifically, we'd probably arrive at something like:

  • logging might end up depending on swift-distributed-tracing-baggage (this has zero dependencies, is just tye baggage type)
  • logging gains log()/info()/warn()/... APIs that accept baggage
  • the baggage in those functions can be
    baggage: @autoclosure () -> Baggage? = { Baggage.current }, only invoked when we actually log something
  • we need to measure the performance impact of this change, decide if and how to roll it out. If baggage "picking up" shall always be "enabled" or if we need a new set of types which do so, in addition to ones which don't etc etc. This must be informed by benchmarking and use-case analysis.

Long story short: we need to sit down, and write up the detailed proposal about these changes and their expected performance impact on the logging APIs etc.

Realistically, progress on the 1.0 of tracing is delayed (at least on my end, as co-author/maintainer of both swift-log and tracing) until we stabilize distributed actor features. Having that said, this still probably means the early 2022 but I don't have specific plans to share, and things might end up getting re-scheduled. @slashmo has been making good progress overall with this, but we're a bit offline now during the x-mas breaks. It someone wanted to help out, let's start in this thread and look into the details of the designs, and we can collaborate here perhaps.

I'm happy to spin off a working group as well if people have active time to dedicate to this, it is not a simple task but it is doable with enough prototyping and "making sure" phases around the designs.

// cc @johannesweiss @tomerd


Is there anything we can do now to push the distributed tracing work?

According to the latest NIO policy, every NIO client should be using Swift 5.5 or newer after Swift 5.7 is released, so it seems we can also experiment Tracing with task-locals within other packages now.

1 Like

Just as I said in my previous post -- this isn't about NIO holding up anything, it is about swift-log integration that we need to figure out, and then we can get to a 1.0 I believe. @slashmo has been picking up minor things here and there with swift-distributed-tracing-extras I believe, not sure about latest status though.

It still is before WWDC so let it settle a little bit and we'll pick up tracing again :slight_smile:

More than happy to welcome others PoCing or designing the logging integration if people have time. But it's also something I'd like to focus on as we stabilize distributed actors now -- so the plan is the same as I've outlined in my post above a few months ago, nothing really changed.