I've been dropping in benchmarks in a single, top-level global for the package-benchmark system to pick them up and run them, basically something akin to:
let benchmarks = {
Benchmark("SeedHeightmap") { benchmark in
for _ in benchmark.scaledIterations {
blackHole(_ = Heightmap(width: 1025, height: 1025, seed: 437_347_632))
}
}
...
}
As I've been iterating, I've added new benchmarks - to the point where I've got quite a few and it's getting a little weird to have them all in a single, long file.
Is there a better pattern here for breaking these declarations up and adding them through multiple files? I didn't see an alternative example illustrated in the Writing Benchmarks article, and I'm feeling like I'm missing something obvious.
hassila
(Joakim Hassila)
January 10, 2025, 9:34am
2
Hi @Joseph_Heck ,
Sorry for late reply, bandwidth is a bit low at the moment.
It is simple to do, the obvious approach is added to this sample:
Basically your benchmark file is:
let benchmarks = {
addSomeBenchmarks()
addSomeMoreBenchmarks()
}
And then two (or more) separate files with whatever structure you want:
import Benchmark
import Foundation
func addSomeBenchmarks() {
Benchmark("SomeBenchmark") { benchmark in
for _ in benchmark.scaledIterations {
blackHole(Date())
}
}
}
import Benchmark
import Foundation
func addSomeMoreBenchmarks() {
Benchmark("SomeMoreBenchmark") { benchmark in
for _ in benchmark.scaledIterations {
blackHole(Date())
}
}
}
> swift package benchmark --target MultiFileExample
...
Building MultiFileExample
==================
Running Benchmarks
==================
100% [------------------------------------------------------------] ETA: 00:00:00 | MultiFileExample:SomeBenchmark
100% [------------------------------------------------------------] ETA: 00:00:00 | MultiFileExample:SomeMoreBenchmark
=====================================================================================================
Baseline 'Current_run'
=====================================================================================================
Host 'ice.local' with 20 'arm64' processors with 128 GB memory, running:
Darwin Kernel Version 24.2.0: Fri Dec 6 19:01:59 PST 2024; root:xnu-11215.61.5~2/RELEASE_ARM64_T6000
================
MultiFileExample
================
SomeBenchmark
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโคโโโโโโโโโโคโโโโโโโโโโคโโโโโโโโโโคโโโโโโโโโโคโโโโโโโโโโคโโโโโโโโโโคโโโโโโโโโโคโโโโโโโโโโ
โ Metric โ p0 โ p25 โ p50 โ p75 โ p90 โ p99 โ p100 โ Samples โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโชโโโโโโโโโโชโโโโโโโโโโชโโโโโโโโโโชโโโโโโโโโโชโโโโโโโโโโชโโโโโโโโโโชโโโโโโโโโโชโโโโโโโโโโก
โ Instructions * โ 1235 โ 1244 โ 1244 โ 1244 โ 1244 โ 1244 โ 6404 โ 10000 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโค
โ Malloc (total) * โ 0 โ 0 โ 0 โ 0 โ 0 โ 0 โ 0 โ 10000 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโค
โ Memory (resident peak) (K) โ 9355 โ 9724 โ 9724 โ 9732 โ 9732 โ 9732 โ 9732 โ 10000 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโค
โ Throughput (# / s) (K) โ 1848 โ 1716 โ 1713 โ 1601 โ 1601 โ 1334 โ 79 โ 10000 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโค
โ Time (total CPU) (ns) * โ 2458 โ 2543 โ 2585 โ 2667 โ 2875 โ 3541 โ 12333 โ 10000 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโค
โ Time (wall clock) (ns) * โ 541 โ 583 โ 584 โ 625 โ 625 โ 750 โ 12667 โ 10000 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโงโโโโโโโโโโงโโโโโโโโโโงโโโโโโโโโโงโโโโโโโโโโงโโโโโโโโโโงโโโโโโโโโโงโโโโโโโโโโงโโโโโโโโโโ
SomeMoreBenchmark
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโคโโโโโโโโโโคโโโโโโโโโโคโโโโโโโโโโคโโโโโโโโโโคโโโโโโโโโโคโโโโโโโโโโคโโโโโโโโโโคโโโโโโโโโโ
โ Metric โ p0 โ p25 โ p50 โ p75 โ p90 โ p99 โ p100 โ Samples โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโชโโโโโโโโโโชโโโโโโโโโโชโโโโโโโโโโชโโโโโโโโโโชโโโโโโโโโโชโโโโโโโโโโชโโโโโโโโโโชโโโโโโโโโโก
โ Instructions * โ 1204 โ 1244 โ 1244 โ 1244 โ 1244 โ 1244 โ 10058 โ 10000 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโค
โ Malloc (total) * โ 0 โ 0 โ 0 โ 0 โ 0 โ 0 โ 0 โ 10000 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโค
โ Memory (resident peak) (K) โ 9404 โ 9773 โ 9773 โ 9781 โ 9781 โ 9781 โ 9781 โ 10000 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโค
โ Throughput (# / s) (K) โ 1848 โ 1716 โ 1713 โ 1601 โ 1500 โ 800 โ 78 โ 10000 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโค
โ Time (total CPU) (ns) * โ 2374 โ 2459 โ 2543 โ 2751 โ 3291 โ 5043 โ 32417 โ 10000 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโค
โ Time (wall clock) (ns) * โ 541 โ 583 โ 584 โ 625 โ 667 โ 1250 โ 12791 โ 10000 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโงโโโโโโโโโโงโโโโโโโโโโงโโโโโโโโโโงโโโโโโโโโโงโโโโโโโโโโงโโโโโโโโโโงโโโโโโโโโโงโโโโโโโโโโ
1 Like
Thank you, quite clear and likely should have been obvious, but I was looking right past it. Appreciate you spelling it out for me.
hassila
(Joakim Hassila)
January 10, 2025, 3:19pm
4
No worries, sorry for late response!
1 Like