Is Benchmark appropriate for us to measure things within a module? Specifically, if there are certain areas I want to track and measure performance of in production, is it safe/a good idea to use it for that? Anything to keep in mind? I often see libraries where people write custom benchmarking closures that run like this and wrap certain areas of code (Periphery is one that is coming to mind)
For what it's worth, I've found it exceptionally useful to use package-benchmark within a module. I do it explicitly to pin down performance within library code that I'm writing, or to just get a baseline before I attempt an optimization effort.
I've often set it up as a subproject (a folder within the main project that uses a local reference to the library/module in question). It's a bit more work, but I prefer keeping the dependency setup of it all a bit more separate. That's just my tastes though, no real need to do that. If you want to see a project doing that, take a look at swift-noise's ExternalBenchmarks directory.
5 Likes
I think the approach of using a separate project with a local dependency has become best practice really - to be able to have different dependencies and toolchain requirements on the benchmarks vs. the real project is quite useful in practice and doesn’t pollute the top project with an extra dependency on Benchmark either, so +1 for that approach.
2 Likes
Btw, if you want to measure operations similar to the one you linked to, I’d consider using GitHub - HdrHistogram/hdrhistogram-swift: Swift port of HdrHistogram for capturing your samples and not log each operation separately - and instead output the histogram for analysis either on demand or on shutdown - much cheaper than logging in general. If all depends on why you want to benchmark though.
1 Like
Thank you so much, this is a very helpful example
Appreciate the response and very intrigued (but also a bit overwhelmed) by the benchmark library, will take all of this away and try to figure out what makes sense