Swift OTel provides an OTLP backend for the Swift observability packages (Swift Log, Swift Metrics, and Swift Distributed Tracing). This is an important package for the server ecosystem and so we would like to converge on a stable 1.0 release.
Following discussions with the other Swift OTel contributors, we've defined the scope and plan for the release.
Thanks for your hard work on that all! It's been a long time coming and great to see come together.
The changes look good and it's great to move towards a 1.0 here. The shapes of APIs also look like we'll be easily able to add things when we need to, so that seems to strike a good balance for a 1.0 -- even if someone may uncover something missing, it's always easier to add than deprecate
I know this is explicitly NOT the intended usecase, but in my company there is growing interest in using OTel also on iOS.
This package would definitely be our first choice since it integrates super nicely into SwiftLogging/Tracing.
One of the issues i see coming soon is the introduction of GRPC2 which will raise the required iOS deployment target to iOS18 (we support iOS16)
Would it be possible to consider splitting out the GRPC Exporter into a separate package?
This would allow the "core" and the "grpc-extensions" to have different OS version requirements.
This would i think somewhat follow the approach that DistributedTracing took, where IIRC the Instrumentation package only has the "bare minimum" requirements.
As i said i can understand this is not the primary concern of this package and comes with some extra challenges to the development workflow, but i thought id bring it up
Yes, I second that. Splitting out the grpc part to not have the iOS 18 target requirement would be very welcome!
Or maybe just annotate the grpc transport as iOS 18+ is what I mean and not limit the entire package's deployment targets.
Thanks for sharing your use-cases for Swift OTel on iOS
I'm not in favor of splitting out parts of the library into separate repositories because it would make our desired "one-line" configuration API impossible. One big downside of the current public API IMHO is that you have to remember to not only import OTel but also OTLPGRPC to get access to the gRPC exporter. Having to additionally add a dependency to another package would make this worse.
Or maybe just annotate the grpc transport as iOS 18+ is what I mean and not limit the entire package's deployment targets. @tkrajacic
Totally understand.
I have seen other OTel libraries (python) which are "plug and play" (just bootstrap once with a oneliner and it uses grpc) and that is a pretty nice experience and applies to most users.
I only suggested this approach because annotating entire targets seemed a bit cumbersome but maybe its actually not that bad
Another topic in the same direction would be the swift-service-lifecycle dependency.
It just doesnt make all that much sense one iOS (or could it? ). We actively try to avoid pulling dependencies we dont use.
Maybe this could have a "opt out" using package traits?
Im happy to try and contribute a patch some of these topics if it helps.
Overall this looks great and I'm excited for this package to get its first major release. I agree with the general direction to reduce the amount of API surface and focus on providing backends for Swift's observability APIs. Some comments while reading through the API doc.
extension OTel { ... }
It looks like everything is nested under one top level enum. I'm not sure this is necessary since we already have the various models for namespacing. Additionally, the enum does have the same name as the module which is known to cause problems. I would recommend either:
Getting rid of the enum and moving the static methods to global methods
Just moving the static methods behind a Bootstrapping enum
public static func makeMetricsBackend(
configuration: OTel.Configuration = .default,
detectEnvironmentOverrides: Bool = true
) throws -> (factory: any CoreMetrics.MetricsFactory, service: some Service)
public static func makeTracingBackend(
configuration: OTel.Configuration = .default,
detectEnvironmentOverrides: Bool = true
) throws -> (factory: any Tracing.Instrument, service: some Service)
Those two can return a some MetricsFactory and some Instrument right?
public struct LogLevel
How does the OTel log level work together with the Logger.LogLevel? Is this configuring at what level the OTel library itself logs or the level which logs get exported?
public struct TracesConfiguration
public struct MetricsConfiguration
public struct LogsConfiguration
Can we nest those types inside the Configuration struct? This way we don't need the Configuration suffix.
public var certificateFilePath: String?
public var clientKeyFilePath: String?
public var clientCertificateFilePath: String?
Do we need a way to provide those as bytes or provide a custom TLS callback like swift-nio-ssl offers?
Swift OTel will handle this by having it's own Logger, which it will construct without the bootstrapped logging backend and will log such diagnostics to standard error.
Future implementations may support customizing the internal logger used by the SDK.
I think this Logger should be passed in rather than self constructed. I would propose that the various bootstrap APIs take a Logger parameter. We can default this to a stderr based logger but it allows everyone to customize where those logs go as well.
FWIW, I think that ServiceLifecycle provides value in iOS applications as well since you can put all services into a ServiceGroup and run that group off a SwiftUI.Views lifecycle like this:
struct MyView: View {
var body: some View {
Label("Hello").task {
try await ServiceGroup(services: [Service1, Service2]).run()
}
}
}
Note that Swift OTel is a backend for DistributedTracing so it's not the instrumentation package. It's not attempting be a whole OTel SDK (as the OTel spec refers to it), but rather an OTLP backend for the Swift observability instrumentation "API packages" (Swift Log, Swift Metrics, and Swift Distributed Tracing).
Put another way, it's not likely you'll depend on this package without the use of one of the exporters (OTLP/HTTP or OTLP/gRPC) since it does not offer an instrumentation API, rather only a backend for the instrumentation APIs provided by the other packages.
While we could theoretically trait-guard the dependency on Swift Service Lifecycle, I'd like to first explore the motivation behind the question.
@anreitersimon I wonder if this question relates to your other reply regarding this package offering an thin instrumentation API, which as we're exploring in the other thread, this package is not (it's instead offering an exporter backend for the Swift observability instrumenation APIs).
Given that, then to the question of whether it makes sense on iOS: yes, you'll still need to run the background work for the exporter(s).
We currently do this by returning an opaque some ServiceLifecycle.Service, but protocol ServiceLifecycle.Service is just a thin protocol over a func run() async throws requirement.
If we were to drop the package dependency on Swift Service Lifecycle, we'd likely end up offering a similar abstraction and opaque return type, with the contract remaining the same: the caller must run the background work using the run() method, somehow.
Swift Service Lifecycle offers some convenient ways to do this, along with the ability to shutdown gracefully on cancellation. It further provides for the notion of a ServiceGroup, which allows for a set of dependent services to be started and shutdown gracefully, in order.
I think this dependency is worth it. Are there specific concerns about having this dependency be present on iOS.
//cc @FranzBusch who might have opinions here as an expert of Swift Service Lifecycle.
Seems that service-lifecycle isnt a "heavy" dependency and it might even make sense for other areas in a iOS app.
I guess i was thinking of the otel package as being layered a bit
the "otel-instrumentation" layer (i am mostly concerned with the exporter protocol)
The "otel-backend" layer (this has the implementations of the protocols)
The main library would create a nicely preconfigured "backend“ for you.
But you can still drop down to the "low-level" api and construct your own exporter.
Hope that make somewhat sense
What i would ideally be able to achieve is to write my own HTTP based Exporter thats backed by URLSession. (Ideally i get handed the the protobuf models that i can convert to json)
Preferably i could avoid pulling in the GRPC and HTTPExporter that are provided out of the box.
This proposed API is mostly static functions, so I opted to group them behind the enum OTel namespace because:
The alternative would be polluting the global namespace with a bunch of static functions, which I typically object to.
It provides a single point of discoverability for the APIs.
It's a place to anchor top-level docs (almost module-level docs) that are accessible in code, which I think will be very helpful for this package specifically.
IIUC these are historical problems that have now been fully addressed.
The concrete problem we're discussing here is if an adopter takes a new dependency on this package but already has a type called OTel in their codebase. If they cannot rename this type to avoid the clash (e.g. because it forms part of the adopter's API surface) then this can be mitigated using a module alias when adding the dependency.
As it happens the answer to this is yes to some MetricsFactory and no to some Instrument. That's a quirk of the current backing implementations of protocol MetricsFactory and protocol Instrument are pretty asymmetric. The former, the PeriodicMetricsReader holds an any MetricsExporter as an existential, so, regardless of the choice of exporter (OTLP/HTTP or OTLP/gRPC), we could return the same concrete type for the some MetricsFactory. However, the implementation of Tracer is a nested generic type over the exporter, so, depending on the choice of exporter we would be returning a different concrete type.
Now, all these types are becoming internal so the asymmetry in their API isn't something that adopters will need to concern themselves with. But it did mean that we'd either have to have that asymmetry exposed in this API (with one returning an existential and the other an opaque type), or regain some symmetry here as in the proposed API, with them both returning existentials.
While we could wrap a closed set of concrete generic types behind an enum internally, this could make it harder to offer a more extensible API in the future.
Finally, the logging factory, for which there is no nominal type in Swift Log, must be an existential. So, I'd prefer these to all look the same.
Since these are both passed to the corresponding bootstrap method each take existentials, I don't see any value in avoiding the existential here:
static InstrumentationSystem.bootstrap( _ instrument: any Instrument)
static MetricsSystem.bootstrap( _ instrument: any MetricsFactory)
I apologise for the lack of documentation here, but all of these are mapping to the documented configurable properties in the OTel spec. In this instance this is the log level of the internal SDK logger, nothing to do with the logs that are being collected and exported by this library:
The proposal already has these nested, but they have this suffix for both clarity and to avoid clashing with the property of this type, which would otherwise have the same name. E.g. OTel.Configuration.traces: TracesConfiguration, vs. OTel.Configuration.traces: Traces. These are not traces, ofc.
The current proposal does not make affordance for this. The above values are what you can provide in the OTel spec. For a v1, I think this is OK, because:
This isn't expected from an OTel SDK according to the spec.
We've never supported this in the pre-1.0 Swift OTel.
We can leave room for this to be added in the future (*).
(*) I anticipate that the OTel config be defined to at least follow the OTel spec, but can accommodate additional configuration properties for features we want to add on. Concretely, if the spec has a spelling for a configuration, we'll use that in preference to us redefining our own spelling. For things that do not have a spec spelling, we can have additive config.
So, yes, I think we could add a custom callback, but I'd prefer to do this after 1.0 because we'd likely end up having a wider discussion about the currency types we'd like to use, e.g. Swift Certificates?
The design of this API is trying to avoid a long list of parameters. Do you think it would be acceptable for this feedback to be able to provide the logger as a property in the configuration? Something like this?
extension OTel.Configuration {
public struct InternalLoggerConfiguration {
internal enum Backing { /* ... */ }
public static let stderr: Self
public static func custom(_: Logger) -> Self
}
public var internalLogger: InternalLoggerConfiguration // <- new
}
The proposal here is to stop providing all the low-level building blocks and expecting people to implement things, but rather provide a simple-to-use, opinionated, spec-client API that exports telemetry collected through the Swift observability APIs over OTLP, with some configuration as to where/how.
I'd like to understand this desire better. What would you like to do differently in this custom HTTP exporter that isn't what the spec dictates? Or is it purely that you want an HTTP exporter but you'd prefer one that only depends on URLSession, and not AsyncHTTPClient?
Understanding that, might help us work out a path forward, if there is one.
For example, one option is to make room in the traits for a URLSession-based HTTP exporter, by naming the AHC one appropriately, i.e. renaming the OTLPHTTP trait to OTLPAHC, and leaving room for an OTLPURLSession trait. To be clear, we'd need some more discussion on whether an URLSession exporter is something we'd want to provide or not, but just a hypothetical solution.
Yes, my primary concern is to avoid the AsyncHTTPClient (and swift-nio) dependency.
So i would be very happy with a prebuilt URLSession backend+trait.
One minor concern is that we sometimes have, let's say "unique" requirements.
One example is authenticating requests.
We roughly do this flow:
get an auth-token
send authenticated request
if it fails refresh the token and retry the original request.
So getting some "low-level" control does seem appealing.
But i do get the goal of really keeping the public API surface to a minimum.
Just to put the idea out there in case this wasn't already considered.
Maybe instead of providing its own URLSessionExporter it would be better to build this around swift-http-types
Since its a platform-agnostic abstraction.
I was imagining a very simple protocol that basically that does not expose any of the internals (just abstracts sending a http-request)
protocol OTLPHTTPExporter {
/// OTel would pass a request thats already got body,header,... set
/// but can be further modified before sending it off.
func send(request: HTTPRequest) async throws
}
This would allow for a AsyncHTTPClient based implementation which i think (just guessing TBH) already supports it, but also one based on URLSession.
This could then be used by default unless disabled by traits.
This would reduce a bit the "low-level" internals that are exposed.
So you have a use case that uses token-based auth for an OTLP/HTTP endpoint?
Putting aside the dislike of the underlying HTTP client library, this is partially catered for since the OTel spec states that users can configure the header fields for the OTLP exporter, regardless of whether it's gRPC or HTTP.
What will be missing here will be the ability to refresh that token on failure.
This is looking very similar to what we've done in Swift OpenAPI, which defines a protocol ClientTransport which has a single method requirement that supports executing an HTTP request, defined in terms of Swift HTTP Types. We then provide separate packages, swift-openapi-urlsession, and swift-openapi-async-http-client, respectively. In practice this only partly solves your use case and we would likely also need to provide a middleware abstraction (which we also provide in Swift OpenAPI).
gRPC Swift (v2) has also gone this route, with the grpc-swift now only providing GRPCCore and an in-process transport. The network transports are now in separate packages, e.g. grpc-swift-nio-transport.
In the absence of any ecosystem-wide abstarction for a HTTP client transport, one option for Swift OTel could be to define its own HTTPClientTransport and support this in the configuration:
extension OTel.Configuration.OTLPExporterConfiguration {
public var httpTransport: OTel.HTTPClientTransportConfiguration
public struct HTTPClientTransportConfiguration {
internal enum Backing {
case ahc, urlSession, custom(any OTel.HTTPClientTransport)
}
func custom(_ transport: any OTel.HTTPClientTransport) -> Self {
Self(backing: .custom(transport)
}
public static var default: Self {
#if os(Linux) && OTLPHTTPAHC
self.init(backing: .ahc)
#elseif os(Linux)
#error("Use of OTLP/HTTP exporter on Linux requires the OTLPHTTPAHC trait")
#elseif canImport(Darwin)
self.init(backing: .urlsession)
#endif
}
}
}
This could allow users to get something sensible out of the box for either platform but still allow for something custom passed in if the defaults don't work for them.
A similar pattern could be added for customising the gRPC transport used.
But I'm very on the fence about exposing this level of extensibility in the v1 API.
I'd like some more input on this, but we could try and leave room for this by moving the core OTLP/gRPC and OTLP/HTTP exporting logic to either (1) not be trait-guarded and having only the transports be trait-guarded, or (2) having two levels of trait-guards.
Have OTLPHTTPAHC guard the dependency on AHC, and OTLPGRPCNIO guard the dependency on the NIO gRPC transport, but package unconditionally depends on HTTP Types, Swift Protobuf, and gRPC Swift (core).
Have OTLPHTTP guard the dependency on HTTP Types and Swift Protobuf, and have OTLPGRPC guard the dependency on Swift Protobuf and gRPC Swift; AND also introduce the OTLPHTTPAHC and OTLPGRPCNIO traits as in (1).
I'd like some input from some of the other maintainers on whether we think this is something we should (a) commit to do in v1; (b) leave room for in the API and trait structure; or (c) explicitly land on this level of extensibility being a non-goal.
This is very much intended to work with the official OTel Collector (capital 'C') and I completely agree we will need to be able to export to Collectors with authentication enabled.
FWICT, the relevant parts of the OTel spec that cover what should be configurable in a Language SDK for the communication between the SDK and the Collector are:
Yeah, personally I'm with Si on this one... the "from mobile device" use case may exist, but is riddled with concerns. The proposed API shape allows for adding transport customization points in the future and I think we can decouple this from the 1.0 APIs. Adding more parameters to the make backends or bootstraps is simple, as is adding more protocols.
To do this I think we should have more time to really make sure this is necessary and get the shapes right, and I don't think it's required for the 1.0 "baseline" feature set.
The OTLPHTTP name I think is fine since it's the "default", and I'm not sure we need to rename it to imply AHC specifically. If we ever do others, they can be more specific than the default ...HTTP one, specifying the transport exactly.
Overall I'm very supportive of the proposed API; it looks considerably easier to use than the existing API.
I like that the configuration closely follows the OTel spec; this has some great advantages: it's easy to translate commentary around the spec or OTel docs (not Swift OTel docs) to Swift OTel. It should (presumably) also lower the barrier to entry for folks who've used OTel in other languages.
For 1.0.0 I'm pretty strongly against expanding the API surface to include a transport abstraction; the focus should be to serve the 99% use case in the best possible way. In my mind that should be achieved by having a very small and well defined API surface. API is much easier to add than it is to remove.
If there are legitimate issues which can't be addressed with the proposed API then the maintainers can always address these in future releases by expanding the API if necessary.
gRPC Swift 2 requires you iOS 18 and friends to use but no longer requires iOS 18 for it to be built in the same package graph. Swift OTel won't (and shouldn't) have to set its deployment targets to iOS 18.
). We actively try to avoid pulling dependencies we dont use.