Hi, I've been extensively use this swift-collections-benchmark package lately, and I just noticed this API Benchmark.add. This API provides a Timer parameter to my custom closure.
However, I have trouble understanding how to use this parameter. I thought I could use it to achieve partial measuring, like this
benchmark.add(title: "xxx", input: Int.self) { count in
return { timer in
someSetup()
timer.measure {
coreLogic()
}
}
}
However, the above code will trigger a runtime assertion.
It's hard to tell what goes wrong without looking at the sources. But I'd first check for one of the following issues:
timer.measure is only expected to be called at most once. Double check there is no way for the outer closure to end up calling it multiple times. For example, this will not work correctly:
The tasks are assumed to always execute the same way when given the same input -- if the task executes timer.measure, then it must do so every time. For example, this will not work correctly:
benchmark.add(title: "Foo", input: Int.self) { count in
return { timer in
if (0 ..< 10).randomElement() < 2 { return }
timer.measure { doSomething(count) }
}
}
The add method allows the task to refuse to run on some inputs (by returning nil), and it lets you measure only part of the task's execution. addSimple is for the simple case, when you always want to run the task, and you want to measure the entire duration.
