Need clarification on Swift CI benchmark


(Proyb P) #1

What those PR in Swift CI benchmark shown "Regression", "Old_min",
"new_min", "delta" and "speedup" really mean? Speedup can be confusing when
Delta is either a positive or negative value.

https://github.com/apple/swift/pull/3796


(Saagar Jha) #2

Delta is (OLD_MIN - NEW_MIN) / OLD_MIN expressed as a percent; speedup is (OLD_MIN - NEW_MIN) / NEW_MIN as a decimal. A negative delta corresponds to a speedup less than 1 (i.e the newer one is slower) and a positive delta to a speedup greater than one (i.e. the newer one is faster).

Saagar Jha

···

On Apr 20, 2017, at 07:48, Proyb P via swift-users <swift-users@swift.org> wrote:

What those PR in Swift CI benchmark shown "Regression", "Old_min", "new_min", "delta" and "speedup" really mean? Speedup can be confusing when Delta is either a positive or negative value.

https://github.com/apple/swift/pull/3796

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users


(Proyb P) #3

Not sure if your explaination is correct, I think the Apple team explained
Speedup <1.0x is slower under Regression and >1.0x is faster under
improvement.

The table seriously need improvement or everyone will be confused.

···

On Fri, Apr 21, 2017 at 12:27 AM, Saagar Jha <saagar@saagarjha.com> wrote:

Delta is (OLD_MIN - NEW_MIN) / OLD_MIN expressed as a percent; speedup is
(OLD_MIN - NEW_MIN) / NEW_MIN as a decimal. A negative delta corresponds to
a speedup less than 1 (i.e the newer one is slower) and a positive delta to
a speedup greater than one (i.e. the newer one is faster).

Saagar Jha

On Apr 20, 2017, at 07:48, Proyb P via swift-users <swift-users@swift.org> > wrote:

What those PR in Swift CI benchmark shown "Regression", "Old_min",
"new_min", "delta" and "speedup" really mean? Speedup can be confusing when
Delta is either a positive or negative value.

https://github.com/apple/swift/pull/3796

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users