Is there a way to run a test using XCTest multiple times just with different values?

Using Swift
1

I'm currently working on a project trying to determine the how long different sorting algorithms take to sort different sized arrays. To measure the time, I've decided to use XCTest in Swift Playgrounds since it can automate the process of running the algorithm multiple times and averaging it out. But I have an issue with this method because I have to test a large variety array sizes from 15 elements up to 1500 or so, at 5 element intervals (ie. 15 elements, 20 elements, 25 elements...).

The only way I've been able to do this with one test is multiple functions with the different size and measuring the performance. Here is a sample of what that looks like:

class insertionSortPerformanceTest: XCTestCase {
    func testMeasure10() {
        measure {
            _ = insertionSort(arrLen: 10)
        }
    }
    func testMeasure15() {
        measure {
            _ = insertionSort(arrLen: 15)
        }
    }
    func testMeasure20() {
        measure {
            _ = insertionSort(arrLen: 20)
        }
    }
}

insertionSort() works by generating an array of length arrLen and populating it with random numbers, and then sorting it using (as the name suggests) insertion sort.

Is there a way to automate this process somehow?

Also, is there a way to take the output in the console and save it as a string so I can parse it for the relevant information later?

2 Likes
2

I don’t know for sure but I would imagine playgrounds run in debug mode. That’s probably not great for benchmarking.

It might be better to pull that out into a command line tool where you feed it parameters and can compile it in release mode.

1 Like
3

I did something like this recently just for a quick demo for students to see the differences in putting data to lists vs arrays and accessing lists vs arrays. Check out how this console app uses command line parameters to specify sizes of containers and times to repeat the tests with larger containers. That code saves timing data to a tsv file for easy import into spreadsheet app for creating charts.

And yes, you should absolutely build for Release when testing performance, using (from command line):

xcodebuild -configuration Release

1 Like