# Need clarification on Swift CI benchmark

**URL:** <https://forums.swift.org/t/need-clarification-on-swift-ci-benchmark/5738>\
**Category:** Using Swift\
**Created:** [April 20, 2017, 2:48pm UTC](https://forums.swift.org/t/need-clarification-on-swift-ci-benchmark/5738 "2017-04-20T14:48:58Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Proyb\_P](https://avatars.discourse-cdn.com/v4/letter/p/839c29/32.png) [@Proyb\_P](https://forums.swift.org/u/Proyb_P)\
**Post date:** [April 20, 2017, 2:48pm UTC](https://forums.swift.org/t/need-clarification-on-swift-ci-benchmark/5738/1 "2017-04-20T14:48:58Z")

</div>

What those PR in Swift CI benchmark shown "Regression", "Old\_min",  
"new\_min", "delta" and "speedup" really mean? Speedup can be confusing when  
Delta is either a positive or negative value.

[https://github.com/apple/swift/pull/3796](https://github.com/apple/swift/pull/3796)

---

<div class="post-metadata">

**Author:** ![saagarjha](https://avatars.discourse-cdn.com/v4/letter/s/85f322/32.png) [@saagarjha](https://forums.swift.org/u/saagarjha)\
**Post date:** [April 20, 2017, 4:27pm UTC](https://forums.swift.org/t/need-clarification-on-swift-ci-benchmark/5738/2 "2017-04-20T16:27:39Z")

</div>

Delta is (OLD\_MIN - NEW\_MIN) / OLD\_MIN expressed as a percent; speedup is (OLD\_MIN - NEW\_MIN) / NEW\_MIN as a decimal. A negative delta corresponds to a speedup less than 1 (i.e the newer one is slower) and a positive delta to a speedup greater than one (i.e. the newer one is faster).

Saagar Jha

> **···**
>
> > On Apr 20, 2017, at 07:48, Proyb P via swift-users \<swift-users@swift.org\> wrote:
> > 
> > What those PR in Swift CI benchmark shown "Regression", "Old\_min", "new\_min", "delta" and "speedup" really mean? Speedup can be confusing when Delta is either a positive or negative value.
> > 
> > [https://github.com/apple/swift/pull/3796](https://github.com/apple/swift/pull/3796)
> > 
> > \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_  
> > swift-users mailing list  
> > swift-users@swift.org  
> > [https://lists.swift.org/mailman/listinfo/swift-users](https://lists.swift.org/mailman/listinfo/swift-users)

---

<div class="post-metadata">

**Author:** ![Proyb\_P](https://avatars.discourse-cdn.com/v4/letter/p/839c29/32.png) [@Proyb\_P](https://forums.swift.org/u/Proyb_P)\
**Post date:** [April 21, 2017, 2:32pm UTC](https://forums.swift.org/t/need-clarification-on-swift-ci-benchmark/5738/3 "2017-04-21T14:32:15Z")

</div>

Not sure if your explaination is correct, I think the Apple team explained  
Speedup \<1.0x is slower under Regression and \>1.0x is faster under  
improvement.

The table seriously need improvement or everyone will be confused.

> **···**
>
> On Fri, Apr 21, 2017 at 12:27 AM, Saagar Jha \<saagar@saagarjha.com\> wrote:
> 
> > Delta is (OLD\_MIN - NEW\_MIN) / OLD\_MIN expressed as a percent; speedup is  
> > (OLD\_MIN - NEW\_MIN) / NEW\_MIN as a decimal. A negative delta corresponds to  
> > a speedup less than 1 (i.e the newer one is slower) and a positive delta to  
> > a speedup greater than one (i.e. the newer one is faster).
> > 
> > Saagar Jha
> > 
> > On Apr 20, 2017, at 07:48, Proyb P via swift-users \<swift-users@swift.org\> \> wrote:
> > 
> > What those PR in Swift CI benchmark shown "Regression", "Old\_min",  
> > "new\_min", "delta" and "speedup" really mean? Speedup can be confusing when  
> > Delta is either a positive or negative value.
> > 
> > [https://github.com/apple/swift/pull/3796](https://github.com/apple/swift/pull/3796)
> > 
> > \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_  
> > swift-users mailing list  
> > swift-users@swift.org  
> > [https://lists.swift.org/mailman/listinfo/swift-users](https://lists.swift.org/mailman/listinfo/swift-users)
