One of the benefits of swift-metrics being an API-only package is that it doesn't contain anything you don't need. The drawback being that it may not contain some of the things you do want!
As such I'd like to pitch a small package containing a set of tools related to swift-metrics.
Proposed tools:
Logging factory: a MetricsFactory which logs any changes to the underlying instruments. Logging would be limited to swift-log or print. This is helpful for general debugging and sanity-checking during development.
Delegating factory: MetricsFactory which delegates to a "current" factory. Since it is only possible to bootstrap the metrics system once per process (testable imports aside) it makes testing more difficult as noted in Re-initializing swift-metrics in tests · Issue #60 · apple/swift-metrics · GitHub. The factory would be a singleton which would delegate to a swappable, user-provided factory. It would also have to keep track of where each instrument originates in order to correctly destroy them. This would allow the factory to be swapped out depending on test requirements. It could also handle the awkwardness of installing itself since the user would otherwise have to do keep track of this in tests.
Capture factory: a MetricsFactory which records structured events in a way that is suitable for testing. The rough idea being "I executed this code path and I want to check that this counter was incremented by this value and has this label and these dimensions". This requires some more exploration as to what shape this would take to be maximally useful here.
I'm curious as to whether the community thinks there's enough value in having a such a package, whether the proposed tools would be useful, and if there are any other suggestions for factories or otherwise which could be included here.
The 2nd (a way to change backend "per test") and 3rd one (a way to capture and assert on metrics/logs) I had to (re-)invent myself in a project I work on basically immediately as we shipped swift-metrics, so yes, I agree they're valuable in certain use-cases. It is an unusual form of testing, however can be pretty useful if one wants to ensure some effects that otherwise would be hard to observe (without exposing tons of internals), so I'm +1 on that.
For what it's worth I'm also using the same approach for logs and find it tremendously valuable. Specifically, each test gets it's "LogCapture" and executes, if the tests passed -> no output, however if they failed all logs (including trace, debug, ... level logs) are dumped which makes for a good experience when a flaky test happens -- no need to ever adjust "increase logging" and be worried it'll pollute the successful executions.
Similarly, if a library expects that the main way others will diagnose problems when things go wrong is metrics and logs, it makes sense to sometimes for the lib to add tests that "yup, the log we make here reads well and always happens in this weird situation" etc.
So... swift-metrics-extras and swift-log-extras?
Or should we rather phrase and focus those more specifically around testing -- swift-metrics-testkit and swift-log-testkit?
These are intended for testing (of sorts) so I'm happy to set expectations upfront and include "test" in the name. I'd be okay with the -testkit names unless better alternatives are suggested.
Tagging @tomerd@weissi for ideas where this could end up living (eventually might be nice for those to be close to where the API packages live or swift-server hm...) and how to get there
Maybe we PoC them out somewhere, do a pitch and they could graduate them eventually?
It's a difficult one; having a separate product obviously makes it much easier to adopt. It also makes it much harder to say no to other factories being included in each project ("why is this one allowed in the package but this one isn't?").
Oh I think I misread @Mordil's response here. To clarify (seems I can't "un-like" ) my position on this is in line with what George is saying:
I do not think it's a good idea to add any implementations to the API projects.
As @georgebarnett says, the API projects then become harder to defend from feature creep -- they really should remain API projects, and nothing more to keep them as light and flexible as possible, without any creep. (That the logging one has a default impl is still a bit annoying IMHO... it's stopped people from implementing a really-great stdout one since the one shipping by default is good enough™ for simple stuff...).
Sounds like we can try to PoC something quickly soon out and come up with a pitch soon
As far as pitching and where to let these live goes. I think we can start with hosting it on the GH of whoever starts working on it, and discuss in the pitch/proposal stage where to let it land. Could be apple/ or swift-server/ for both, but I'm personally also not opposed to having it at a single person's GH (like SwiftPrometheus, APNSwift and RediStack)
However I think getting a pitch & PoC up are more important than figuring out where exactly to host the repo
+1 on the idea and +1 to start something as a personal repo for the purpose of a pitch / proposal and as it matures we can decide on a good home for it. this is what we did when we originally developer swift-log and swift-metrics proposals.
It is an unusual form of testing, however can be pretty useful if one wants to ensure some effects that otherwise would be hard to observe (without exposing tons of internals), so I'm +1 on that.
) my position on this is in line with what George is saying: