[Feedback] Swift Statsd Client Implementation

Proposal Review: SSWG-0007 (Statsd Client)

After the discussion thread, we are proposing this as a final revision of this proposal and enter the proposal review phase which will run until the 29th July 2019.

The feedback model will be very similar to the one known from Swift Evolution. The community is asked to provide feedback in the way outlined below and after the review period finishes, the SSWG will -- based on the community feedback -- decide whether to promote the proposal to the Sandbox maturity level or not.

What goes into a review of a proposal?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the evolution of the server-side Swift ecosystem.

When reviewing a proposal, here are some questions to consider:

  • What is your evaluation of the proposal?

  • Is the problem being addressed significant enough?

  • Does this proposal fit well with the feel and direction of Swift on Server?

  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

Thank you for contributing to the Swift Server Work Group!

What happens if the proposal gets accepted?

If this proposal gets accepted, the official repository will be created and the code (minus examples, the proposal text, etc) will be submitted. The repository will then become usable as a SwiftPM package and a version (likely 0.1.0) will be tagged. The development (in form of pull requests) will continue as a regular open-source project.


Statsd Client

Package Description

metrics backend for swift-metrics that uses the statsd protocol

Package Name swift-statsd-client
Module Name StatsdClient
Proposed Maturity Level Sandbox
License Apache 2.0
Dependencies swift-nio > 1.0.0, swift-metrics > 1.0.0

Introduction

for a background on metrics in swift applications, see the metrics proposal

the statsd protocol is simpler than most custom metrics solutions (eg prometheus), yet extremely popular since it is language agnostic and easy to implement both client and server side. as such, it can be used to integrate applications with many observability solutions such as:

Motivation

a statsd client will allow server applications written in swift to easily integrate with many popular observability solutions. the client, like the protocol, is designed to be lightweight and delegate most computation to the observability server

Proposed solution

the proposed solution is a client library that implements the swift metric api and uses swift nio to establish a UDP connection to the statsd server.

metrics types are mapped as following:

Metrics API Statsd
Counter Counter
Gauge Gauge
Timer Timer
Recorder Histogram

Detailed design

api

the client is primarily designed to be used within the context of the swift metrics api. as such, the application needs to create an instance of the StatsdClient and bootstrap the MertricsSystem as part of the application's main:

let statsdClient = try StatsdClient(host: host, port: port)
MetricsSystem.bootstrap(statsdClient)

to cleanly release the networking resources occupied by the client, the application should also shutdown the client before it terminates:

statsdClient.shutdown()

as mentioned above, usage of the client is designed to be done primarily via the swift metrics api and the client does not provide additional functionality beyond the metrics api surface area. in other words, users are not expected to interact with the client directly beyond bootstrap and shutdown. emitting metrics information is done via the standard metrics api, as described in https://github.com/apple/swift-metrics#metric-types

protocol

the client implements the statsd protocol as following:

counter <name>:<value>|c
timer <name>:<value>|ms
gauge <name>:<value>|g
histogram <name>:<value>|h

name is constructed from the metric object label and dimensions concatenated with a dot between them

value is computed from the data provided to the metrics API. no in-memory aggregation is done at the client level, and data is mostly pass-through. note that some statsd metrics types accept positive values only in the UInt64 range, while the metrics API generally accepts Int64. in cases of such mismatch, the client truncates negative values to zero

networking

thew client uses swift-nio to establish a UDP connection to the statsd server. in later phases, the client will also support TCP connections.

Maturity Justification

this is a fairly new library, not enough production usage to justify anything beyond sandbox

Alternatives considered

no alternatives exist to my knowledge

1 Like

+1, in favour of adopting to the sandbox maturity level.

Yes. Statsd is widely deployed and this will provide useful ability to integrate with the statsd interfaces that exist today.

Absolutely: it's a sensible and necessarily evolutionary step following the adoption of swift-metrics.

In-depth study of both the implementation and proposal.

I have one minor grammatical note below as well:

This should probably be "statsd" protocol: looks like a simple autocorrect issue.

1 Like

good catch, fixed!

+1, very much in favor, sandbox level seems right.

Absolutely, and it'll hopefully be a door opener for swift on the server as a whole.

statsd is super widely accepted by a variety of backends even if they have their own custom ingestion protocols. This will help us get metrics into these metrics backends, even before they built swift specific "their special protocol"-specific connectors.

Yes, it adopts the existing Metrics API, and does not expose any new types;

Built libraries in the past which emitted metrics via statsd and some Graphite and Graphana dashboards creation experience.

Read through implementation once but thoroughly; Used the backend in my real projects and confirmed the metrics were emitted and consumed by a statsd server properly.

PS: Very glad about the [...].EventLoopGroupProvider pattern; glad to see it in all/most libs in SSWG projects.

@ktoso @tomerd

Hi, I just found out this topic and would like to mention that we published (together with @vi4m) our own StatsD package couple of weeks ago.

It has TCP and UDP clients implemented and I've added support for swift-metrics standard API couple of days ago.

Maybe it's worth to take a look at it: https://github.com/allegro/swiftbox-metrics-statsd

It still has Vapor dependency due to Logging package name conflict between Vapor and swift-logging, but it will be removed as soon as the problem is gone.

Let me know what do you think :slight_smile:

1 Like

@Blejwi thanks for reaching out. there are two options:

  1. we keep both clients and let users pick the one that works for them, sounds like yours is more geared towards vapor ecosystem and this one is generic
  2. we collaborate to consolidate our efforts into one solution, maybe offer vapor extensions separately

happy either way, and looking for collaborators so could be great!

Terms of Service

Privacy Policy

Cookie Policy