Hi all,
I'm interested in joining Google Summer of Code this year. The distributed tracing project proposal resonated the most with me as I implemented multiple Server-Side Swift projects over the last couple of semesters. Being able to see the Swift projects traced alongside e.g. Golang projects would've been very helpful.
@ktoso In the proposal you mentioned that the implementation should adhere to the OpenTracing standard. If I understand it correctly OpenTracing got now merged with OpenCensus and is being redefined as OpenTelemetry so I'm not sure whether an OpenTracing client would still be needed. OpenTelemetry already started a Swift client yesterday so I'm curious to hear your thoughts on whether it'd still be possible to be involved in realizing distributed tracing for Swift on the server.
Ah yes the specific spec we’d adhere to is open for discussion, you’re right that open census et al have kind of subsumed the OT project. The timing here is quite funny — that the project was just kicked off there yesterday...
I’m sure we’ll be able to figure out either if we can collaborate or refocus the task a little bit given this recent development by OpenTelemetry.
Let me reach out a bit internally and externally given that repository has just been kicked off— and we’ll figure out how to make best use of everyone’s time and efforts here :-)
Hi Moritz and Konrad,
I am the current maintainer of the OpenTelemetry-Swift implementation.
The project is totally open to new contributions and maintainers. Is still in the early stages, some areas of the project are more advanced (like Tracing and CorrelationContext) while others are less advanced (jaeger exporter or stdout exporter), or even not started at all (Metrics integration, Opentracing shims or custom instrumentation).
As it landed recently in the OpenTelemetry domain ( it comes from a contribution started outside of the OpenTelemetry community: https://github.com/open-telemetry/community/issues/261), the development plan is still waiting for the team to complete but there are many areas to contribute
Hi Ignacio,
great to hear from you! It's really great to see movement on the OT front for Swift
Overall we're definitely open to contribute to Open Telemetry, however for this project we'd like to address the wider need of the Swift ecosystem itself (as a whole), and while we're at it also addressing the Open Telemetry story by integrating with it.
We did the same with metrics with SwiftMetrics [1] as well as other projects/frameworks like ServiceTalk [2], and I took a similar approach in a past open source life as well.
I worked out a "strategic plan" so we can get discussing more specifically what we'd like to showcase and/or achieve with this project. The plan draft is available (open, editable) here: GSoC 2020, Swift Tracing: Strategic Plan
In spirit of Managing the Plan, I think it'd be useful if we discussed it a little bit @slashmo, to see if it aligns with what you're interested in, and/or what we could change or clarify more to be reasonable and fun to pull off during this project. Feel free to comment or edit in-line there, and we can also jump on a facetime if you'd like to some time?
Excited to flesh out the details and get working on this :-)
If you want to catch up or discuss some more let me know, otherwise you can come up with a detailed plan (the rough idea is outlined above) and some of your background and we can take it from there. Would be good if we could get a draft some days before the deadline so we can help you polish the proposal.
Don't hestitate to reach out if you'd need anything.
I'm super interested to see how this develops, having used earlier tracing libraries in other languages (Zipkin originally, and more recently Jaeger with python and javascript server-based projects). I'm planning on keeping my eye on the GSoC stuff as it develops, but in the meantime I'm fiddling around with some of the models pulled from OpenTelemetry directly - as I want to do something a bit odd - layer in tracing between clients communicating with MultiPeer. That's super specific to Apple platform pieces, I get that - and so it's not likely going to be directly interesting to this thread.
There may be some pieces that are easily shared - so I've recently made a swift package (not yet versioned) that takes the proto files that represent the definitive models from OpenTelemetry, generates the swift for them, and makes the resulting "thing" available in a swift package.
The downside of this particular package is that raw protobuf models are kind of awkward to deal with, so as I've been using the resulting package I'm making notes and helpers, and will be dragging/refactoring some of that back into that package to make it easier to work with the spans, events, traces, etc. I'm planning on getting it to where I feel like it's more minimally useful before calling it version "0.3.0" to match with the opentelemetry spec as it evolves.
If it helps someone else, then perfect - that's why I have it public in my repo.
Hi there Joseph Great to see more interest in the topic from the community.
So AFAICS these are simply the protos that the OT projects has but generated for swift along with a few type aliases? I definitely think using such protos will be the case for binary transports (I have a few in mind) so these will be useful. I'm not sure sure if we'd depend on the project itself or if rather each project does the proto dance itself though, we'll see and remain in touch
Yep, you're correct - currently it's just the raw data models and some aliases.
That's where it's starting, but it's annoying to use at that level, and of minimal value (even to me), so while I'm working out easier ways of using it in my experiments with peer to peer tracing, I'll be adding onto it as I go.
I don't know if it would ultimately overlap with what the GSoC efforts would want to do or not, but wanted to share in case it was useful. If these had existed somewhere else, I would have happily used them there.
thanks for sharing you work @Joseph_Heck, I think this use case is very interesting and we should collaborate to make sure it is covered in the effort for a general purpose tracing. what are you using for networking? allot of the complexity here would come down to integration of [async] context at that level
The specific use case is a personal project that beats on/uses the Apple specific peer to peer advertising and data transfer framework. It really all started as some questions I wanted to answer about what to expect in terms of connection times and efficiency of use (bandwidth) over the various possible channels. Then it evolved to "how could I measure that", and in the end I thought it was a perfect excuse to see if I could use spans and events as a reasonably compressed data structure to do some of this measuring, capture it, and transfer it back and forth. The underlying mechanics of creating spans, annotating events onto them, and wrapping them up I suspect may be pretty universal and re-usable.
I added some conveniences as extensions over this past weekend, but there are still some corner cases that are more-awkward to use than I'd like, and my implementation has begged some questions I need to go research to understand what the current spec actually says and how the current theory is that it should work.
I've been adding tests that illustrate the conveniences as I'm adding them.
Very cool, I like the direction / idea. Agree very much that instrumenting a system like that is exactly the way to gain understanding about the characteristics of a system in the real world – same reason why I'm so interested and excited about metrics/events/traces everywhere to begin with: being able to gather such holistic view of distributed systems.
Right, I think on that front there'll be plenty to collaborate on -- and this is pretty much what the GSoC project would end up focusing on. Once we know if we've been accepted to kick the project off via GSoC let's definitely stay in touch to ensure your use cases would also be covered by the "context propagation" APIs or style that we'll be working on here (Design requirements / ideas and help very much welcome - the work would be done in the open ).
Great to see more interest in the topic from the community.