Is there possibility for adding performance tests before general release? (something like measure/measureMetrics from XCTest)
5 Likes
Thanks for bringing this up @machineko. Performance testing is definitely something we’re interested in supporting in swift-testing. Right now, the project is still getting established and we are focused on what we consider its most essential functionality – things like defining tests and validating expected behaviors. Performance testing likely won't be a top priority for us until those fundamentals are more mature.
That said, we’re interested in hearing what sort of considerations you’d like to see in an eventual perf testing API, to make sure it meets everyone’s needs. Are there specific aspects of performance you are interested in validating? Do you think existing libraries handle it well or are there opportunities for improvement you think we should consider when adding this?
One note I’ll make is that swift-testing is committed to matching Swift’s supported platforms, so any additions in this space should support all platforms to the greatest extent possible.
5 Likes
In the short to medium term, using package-benchmark is probably your best option for easy benchmark tests.
Just take a look at the aforementioned library, it basically includes everything people would want. Essentially, various (and customizable) metrics to measure for particular tests.
9 Likes
Big +1 on that
I just want to add to what @Jon_Shier said here. We have been adopting package-benchmark in various Swift on Server repositories such as swift-nio and swift-certificates. Overall, we are incredibly happy with what package-benchmark provides and the user experience of wiring benchmark tests is really good. (Especially compared to our own crafted shell scripts from before).
6 Likes