Running benchmarks?


(Lars Kotthoff) #1

Hi list,

I'm trying to run the benchmarks included in the swift repository and I can't
get it to work. I'm following the instructions at https://github.com/apple/swift/tree/master/benchmark

build-script doesn't seem to have a --benchmark flag (unless it's different from
the one used to build swift?). When I try to build them manually and run cmake
as described, I'm getting
CMake Error at cmake/modules/AddSwiftBenchmarkSuite.cmake:18 (message):
  Unable to find Swift driver
Call Stack (most recent call first):
  CMakeLists.txt:118 (runcmd)

After setting SWIFT_EXEC (and CLANG_EXEC, which it doesn't find automatically
either -- this requires changing CMakeLists.txt):
cmake -DSWIFT_EXEC=../../../build/Ninja-DebugAssert/swift-linux-x86_64/bin/swiftc -DCLANG_EXEC=../../../build/Ninja-DebugAssert/llvm-linux-x86_64/bin/clang

I'm getting:

CMake Error at CMakeLists.txt:137 (set):
  Syntax error in cmake code at

    /mnt/swift/swift/benchmark/CMakeLists.txt:138

  when parsing string

    A semicolon separated list of benchmark configurations. \

  Available configurations: <Optlevel>_SINGLEFILE, <Optlevel>_MULTITHREADED

  syntax error, unexpected cal_SYMBOL, expecting $end (131)

At this point I don't know what else to try.

How do I compile and run the benchmarks?

Cheers,

Lars


(Michael Gottesman) #2

Hi list,

I'm trying to run the benchmarks included in the swift repository and I can't
get it to work. I'm following the instructions at https://github.com/apple/swift/tree/master/benchmark

build-script doesn't seem to have a --benchmark flag (unless it's different from
the one used to build swift?).

build-script should build the benchmarks by default without needing to pass in any --benchmark flag (see the Readme).

When I try to build them manually and run cmake
as described, I'm getting
CMake Error at cmake/modules/AddSwiftBenchmarkSuite.cmake:18 (message):
Unable to find Swift driver
Call Stack (most recent call first):
CMakeLists.txt:118 (runcmd)

After setting SWIFT_EXEC (and CLANG_EXEC, which it doesn't find automatically
either -- this requires changing CMakeLists.txt):

This is an inadequacy in the cmake benchmark script.

cmake -DSWIFT_EXEC=../../../build/Ninja-DebugAssert/swift-linux-x86_64/bin/swiftc -DCLANG_EXEC=../../../build/Ninja-DebugAssert/llvm-linux-x86_64/bin/clang

I imagine this is related to the benchmarks not being supported currently on linux. Patches/bug reports for both of these are welcome.

···

On Mar 3, 2016, at 2:32 PM, Lars Kotthoff via swift-users <swift-users@swift.org> wrote:

I'm getting:

CMake Error at CMakeLists.txt:137 (set):
Syntax error in cmake code at

   /mnt/swift/swift/benchmark/CMakeLists.txt:138

when parsing string

   A semicolon separated list of benchmark configurations. \

Available configurations: <Optlevel>_SINGLEFILE, <Optlevel>_MULTITHREADED

syntax error, unexpected cal_SYMBOL, expecting $end (131)

At this point I don't know what else to try.

How do I compile and run the benchmarks?

Cheers,

Lars
_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users


(Lars Kotthoff) #3

Thanks for your reply.

build-script should build the benchmarks by default without needing to pass
in any --benchmark flag (see the Readme).

The readme says: "To run Swift benchmarks, pass the --benchmark flag to
build-script." This is what I'm trying to do and what doesn't work.

I imagine this is related to the benchmarks not being supported currently on
linux.

What systems are supported? Only Mac OS X?

Cheers,

Lars


(Michael Gottesman) #4

Thanks for your reply.

build-script should build the benchmarks by default without needing to pass
in any --benchmark flag (see the Readme).

The readme says: "To run Swift benchmarks, pass the --benchmark flag to
build-script." This is what I'm trying to do and what doesn't work.

Ok. We miscommunicated. To build you don't need any flags, but it seems you are correct that to run it you are supposed to pass in the "--benchmark" flag. So we were both correct ; ).

I imagine this is related to the benchmarks not being supported currently on
linux.

What systems are supported? Only Mac OS X?

Currently. But that solely because no one has spent the time to bring it up on Linux.

A patch adding such support would be most welcome = ).

Michael

···

On Mar 5, 2016, at 12:06 PM, Lars Kotthoff <larsko@cs.ubc.ca> wrote:

Cheers,

Lars