[GSoC 2024] Swift Distributed Tracing support to AsyncHTTPClient - Potential Proposal and Discussion

Hello everyone :wave:

I am Niccolò, I am interested in joining the Google Summer of Code this year.

I would be happy to contribute to adding Swift Distributed Tracing support to AsyncHTTPClient since it touches my interest in distributed systems and Swift on the Server in general.

I open this topic to make sure I've correctly understood the requirements of the project and discuss about them to finally create a proposal.

Overview

Distributed tracing allows correlating “spans” (start time, end time, and additional information) of traces, made across nodes in a distributed system. In HTTP, this means attaching extra trace headers to outgoing HTTP requests.

The recommended HTTP client for server applications in Swift is async-http-client, which currently doesn’t have first-class support for distributed tracing.

The following code should emit trace information to the configured backend:

import AsyncHTTPClient
import Tracing

let httpClient = HTTPClient(eventLoopGroupProvider: .singleton)

try await tracer.withSpan("Prepare dinner") { span in
  let carrotsRequest = HTTPClientRequest(url: "https://example.com/shop/vegetable/carrot?count=2")
  let carrots = try await httpClient.execute(request, timeout: .seconds(30))

  let meatRequest = HTTPClientRequest(url: "https://example.com/shop/meat?count=1")
  let meat = try await httpClient.execute(request, timeout: .seconds(30))

  print("Dinner ready: \(makeDinner(carrots, meat))")
}

The above should result in 3 spans being recorded:

  • the overall “Prepare dinner” span
  • one client span for the duration of the carrots HTTP request
  • one client span for the duration of the meat HTTP request

Implementation

In the official documentation of the swift-distributed-tracing library, there is a detailed guide on how to add support for distributed tracing in a library or framework.

  1. Inject trace information into the outgoing request:
func get(url: String) -> HTTPResponse {
    var request = HTTPRequest(url: url)
  
    // All we have to do is query for the current task-local ServiceContext value, 
    // and if one is present, call on the instrumentation system to inject it into the request.
    if let context = ServiceContext.current {
        // The library does not know anything about what tracing system or instrumentation is installed,
        // because it cannot know that ahead of time.
        InstrumentationSystem.instrument.inject(
            context,
            into: &request,
            using: HTTPRequestInjector()
        )
    }
  
    ...
}
  1. Register a span for the outgoing request:
func get(url: String) -> HTTPResponse {
    var request = HTTPRequest(url: url)
  
    ...
  
    // Start the span for the outgoing request
    return try await InstrumentationSystem.tracer.withSpan(url, context: context, ofKind: .client) { span in
		// Send the request here and maybe add informative span events
		return try await _send(request)
    }
}

Testing

The testing part will require to write comprehensive unit tests to make sure that:

  • The HTTP client correctly injects the ServiceContext metadata in every request’s header
  • A span is registered for every outgoing requests

There should be also a small “demo” docker example prepared, such that developers can quickly try out the functionality.

Open Points

  • Is there anything I am missing here? Just asking to make sure I correctly understand the objective of this project.
  • Do we want to allow users to explicitly disable the tracing support?

@ktoso looking forward to connect with you as the potential mentor for the project.

Thanks.

Best,
Niccolò

2 Likes

Hi there, this is a good writeup! You got that right.

No, there’s no need to specialize disabling tracing — disabling tracing is done by not having bootstrapped a tracer by the end user. Tracing then is a no-op.

I should share with you though that we are a bit nervous with this project that it may be too small. Instrumenting the async http client may be a little too little work for warrant the short project period even hm.

Maybe you can think about other libraries in the server ecosystem which we could adopt tracing in as well and make this a project about comprehensive adoption?

GRPC already has support, but maybe redis and other database drivers and framrworks? Hummingbird already as support as well hm…

1 Like

Awesome, thank you.

I like the idea of making it a project about comprehensive adoption. We could also add the distributed tracing support to:

Don't know if this may be enough or not for the project tho.

2 Likes