1

Unrelated and harmless: this test is probabilistic.

···

On Mar 16, 2016, at 10:00 , no-reply@swift.org wrote:

[FAILURE] oss-swift-incremental-RA-osx [#2761]

Build URL: https://ci.swift.org/job/oss-swift-incremental-RA-osx/2761/
Project: oss-swift-incremental-RA-osx
Date of build: Wed, 16 Mar 2016 09:31:51 -0700
Build duration: 29 min
Identified problems:

Regression test failed: This build failed because a regression test in the test suite FAILed. Below is a list of all errors:
Indication 1 <https://ci.swift.org//job/oss-swift-incremental-RA-osx/2761/consoleFull#-1004795036fca400bf-2f4a-462e-b517-e058d770b2d7&gt;
Tests:

Name: Swift
Failed: 1 test(s), Passed: 7772 test(s), Total: 7773 test(s)
Failed: Swift.stdlib.Hashing.swift <https://ci.swift.org/job/oss-swift-incremental-RA-osx/2761/testReport/junit/Swift/stdlib/Hashing_swift&gt;
Name: Swift-Unit
Failed: 0 test(s), Passed: 278 test(s), Total: 278 test(s)

Changes

Commit c43943d7a884f7b43a6ac06877c29656d5214653 by jordan_rose:
[build-script] Remove -X option for "edit-only" Xcode projects.

edit: utils/build-script-impl
edit: README.md
edit: CMakeLists.txt
edit: cmake/modules/SwiftAddCustomCommandTarget.cmake
edit: utils/build-script

2

I was tired of seeing and writing these emails:

commit 24f685c1d29cd926b1674e099bfcca6ed8673491
Author: Ben Langmuir <blangmuir@apple.com>

    Decrease the probability that stdlib/Hashing.swift will fail
    
    Bump the number of trials from 10 to 14, which makes the test orders of
    magnitude less likely to fail. For a range of size 10, doing 10 trials
    meant that a single call to checkRange would fail ~0.03% of the time;
    with 14 trials the spurious failure rate is ~0.0003%. We have 10 calls
    to checkRange in this test with ranges of size <= 10.
    
    While this test didn't fail that often before, we have a very large
    number of automated builds and it has been a constant low-level source
    of friction that this test fails and we have to look at and then ignore
    the results.

···

Date: Wed Mar 16 10:36:02 2016

On Mar 16, 2016, at 10:01 AM, Jordan Rose via swift-dev <swift-dev@swift.org> wrote:

Unrelated and harmless: this test is probabilistic.

On Mar 16, 2016, at 10:00 , no-reply@swift.org <mailto:no-reply@swift.org> wrote:

[FAILURE] oss-swift-incremental-RA-osx [#2761]

Build URL: https://ci.swift.org/job/oss-swift-incremental-RA-osx/2761/
Project: oss-swift-incremental-RA-osx
Date of build: Wed, 16 Mar 2016 09:31:51 -0700
Build duration: 29 min
Identified problems:

Regression test failed: This build failed because a regression test in the test suite FAILed. Below is a list of all errors:
Indication 1 <https://ci.swift.org//job/oss-swift-incremental-RA-osx/2761/consoleFull#-1004795036fca400bf-2f4a-462e-b517-e058d770b2d7&gt;
Tests:

Name: Swift
Failed: 1 test(s), Passed: 7772 test(s), Total: 7773 test(s)
Failed: Swift.stdlib.Hashing.swift <https://ci.swift.org/job/oss-swift-incremental-RA-osx/2761/testReport/junit/Swift/stdlib/Hashing_swift&gt;
Name: Swift-Unit
Failed: 0 test(s), Passed: 278 test(s), Total: 278 test(s)

Changes

Commit c43943d7a884f7b43a6ac06877c29656d5214653 by jordan_rose:
[build-script] Remove -X option for "edit-only" Xcode projects.

edit: utils/build-script-impl
edit: README.md
edit: CMakeLists.txt
edit: cmake/modules/SwiftAddCustomCommandTarget.cmake
edit: utils/build-script

_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

3

IMO, we should remove/disable this test unless it can be “really” fixed. It is unacceptable to have a test that we know injects noise into our CI systems. Having to second guess whether failures are “real” or not undermines their value, and we should continue to stomp out any nondeterminism from the build and test cycle.

-Chris

···

On Mar 16, 2016, at 11:34 AM, Ben Langmuir via swift-dev <swift-dev@swift.org> wrote:

I was tired of seeing and writing these emails:

commit 24f685c1d29cd926b1674e099bfcca6ed8673491
Author: Ben Langmuir <blangmuir@apple.com <mailto:blangmuir@apple.com>>
Date: Wed Mar 16 10:36:02 2016

    Decrease the probability that stdlib/Hashing.swift will fail
    
    Bump the number of trials from 10 to 14, which makes the test orders of
    magnitude less likely to fail. For a range of size 10, doing 10 trials
    meant that a single call to checkRange would fail ~0.03% of the time;
    with 14 trials the spurious failure rate is ~0.0003%. We have 10 calls
    to checkRange in this test with ranges of size <= 10.
    
    While this test didn't fail that often before, we have a very large
    number of automated builds and it has been a constant low-level source
    of friction that this test fails and we have to look at and then ignore
    the results.

On Mar 16, 2016, at 10:01 AM, Jordan Rose via swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:

Unrelated and harmless: this test is probabilistic.

On Mar 16, 2016, at 10:00 , no-reply@swift.org <mailto:no-reply@swift.org> wrote:

[FAILURE] oss-swift-incremental-RA-osx [#2761]

Build URL: https://ci.swift.org/job/oss-swift-incremental-RA-osx/2761/
Project: oss-swift-incremental-RA-osx
Date of build: Wed, 16 Mar 2016 09:31:51 -0700
Build duration: 29 min
Identified problems:

Regression test failed: This build failed because a regression test in the test suite FAILed. Below is a list of all errors:
Indication 1 <https://ci.swift.org//job/oss-swift-incremental-RA-osx/2761/consoleFull#-1004795036fca400bf-2f4a-462e-b517-e058d770b2d7&gt;
Tests:

Name: Swift
Failed: 1 test(s), Passed: 7772 test(s), Total: 7773 test(s)
Failed: Swift.stdlib.Hashing.swift <https://ci.swift.org/job/oss-swift-incremental-RA-osx/2761/testReport/junit/Swift/stdlib/Hashing_swift&gt;
Name: Swift-Unit
Failed: 0 test(s), Passed: 278 test(s), Total: 278 test(s)

Changes

Commit c43943d7a884f7b43a6ac06877c29656d5214653 by jordan_rose:
[build-script] Remove -X option for "edit-only" Xcode projects.

edit: utils/build-script-impl
edit: README.md
edit: CMakeLists.txt
edit: cmake/modules/SwiftAddCustomCommandTarget.cmake
edit: utils/build-script

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

4

I completely agree.

···

On Mar 16, 2016, at 10:33 PM, Chris Lattner via swift-dev <swift-dev@swift.org> wrote:

IMO, we should remove/disable this test unless it can be “really” fixed. It is unacceptable to have a test that we know injects noise into our CI systems. Having to second guess whether failures are “real” or not undermines their value, and we should continue to stomp out any nondeterminism from the build and test cycle.

-Chris

On Mar 16, 2016, at 11:34 AM, Ben Langmuir via swift-dev <swift-dev@swift.org> wrote:

I was tired of seeing and writing these emails:

commit 24f685c1d29cd926b1674e099bfcca6ed8673491
Author: Ben Langmuir <blangmuir@apple.com>
Date: Wed Mar 16 10:36:02 2016

    Decrease the probability that stdlib/Hashing.swift will fail
    
    Bump the number of trials from 10 to 14, which makes the test orders of
    magnitude less likely to fail. For a range of size 10, doing 10 trials
    meant that a single call to checkRange would fail ~0.03% of the time;
    with 14 trials the spurious failure rate is ~0.0003%. We have 10 calls
    to checkRange in this test with ranges of size <= 10.
    
    While this test didn't fail that often before, we have a very large
    number of automated builds and it has been a constant low-level source
    of friction that this test fails and we have to look at and then ignore
    the results.

On Mar 16, 2016, at 10:01 AM, Jordan Rose via swift-dev <swift-dev@swift.org> wrote:

Unrelated and harmless: this test is probabilistic.

On Mar 16, 2016, at 10:00 , no-reply@swift.org wrote:

[FAILURE] oss-swift-incremental-RA-osx [#2761]

Build URL: https://ci.swift.org/job/oss-swift-incremental-RA-osx/2761/
Project: oss-swift-incremental-RA-osx
Date of build: Wed, 16 Mar 2016 09:31:51 -0700
Build duration: 29 min
Identified problems:

Regression test failed: This build failed because a regression test in the test suite FAILed. Below is a list of all errors:
Indication 1
Tests:

Name: Swift
Failed: 1 test(s), Passed: 7772 test(s), Total: 7773 test(s)
Failed: Swift.stdlib.Hashing.swift
Name: Swift-Unit
Failed: 0 test(s), Passed: 278 test(s), Total: 278 test(s)

Changes

Commit c43943d7a884f7b43a6ac06877c29656d5214653 by jordan_rose:
[build-script] Remove -X option for "edit-only" Xcode projects.

edit: utils/build-script-impl
edit: README.md
edit: CMakeLists.txt
edit: cmake/modules/SwiftAddCustomCommandTarget.cmake
edit: utils/build-script

_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev