[Swift CI] Build Failure: OSS - Swift Package - Ubuntu 15.10 (master) #1930


(Daniel Dunbar) #1

Failing in lldb:
Command invoked: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py --executable /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb --rerun-all-issues -C /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang -s 2016-08-18-15_15_07 --results-port 41187 --inferior -p TestStubReverseConnect.py /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test --event-add-entries worker_index=14:int

Configuration: arch=x86_64 compiler=/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang

···

----------------------------------------------------------------------
Collected 2 tests

======================================================================
ERROR: test_reverse_connect_works_llgs (TestStubReverseConnect.TestStubReverseConnect)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py", line 121, in wrapper
    func(*args, **kwargs)
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py", line 121, in wrapper
    func(*args, **kwargs)
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py", line 89, in test_reverse_connect_works_llgs
    self.reverse_connect_works()
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py", line 67, in reverse_connect_works
    (stub_socket, address) = self.listener_socket.accept()
  File "/usr/lib/python2.7/socket.py", line 206, in accept
    sock, addr = self._sock.accept()
timeout: timed out
Config=x86_64-/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang-3.9
----------------------------------------------------------------------
Ran 2 tests in 22.006s

RESULT: FAILED (0 passes, 0 failures, 1 errors, 1 skipped, 0 expected failures, 0 unexpected successes)
Session logs for test failures/errors/unexpected successes can be found in directory '2016-08-18-15_15_07'

<>[TestStubReverseConnect.py FAILED]
Command invoked: /usr/bin/python /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py --executable /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb --rerun-all-issues -C /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang -s 2016-08-18-15_15_07 --results-port 41187 --inferior -p TestStubReverseConnect.py /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test --event-add-entries worker_index=14:int

- Daniel

On Aug 18, 2016, at 9:18 PM, no-reply@swift.org wrote:

[FAILURE] oss-swift-package-linux-ubuntu-15_10 [#1930]

Build URL: https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/
Project: oss-swift-package-linux-ubuntu-15_10
Date of build: Thu, 18 Aug 2016 20:26:52 -0700
Build duration: 51 min
Identified problems:

Regression test failed: This build failed because a regression test in the test suite FAILed. Below is a list of all errors:
Indication 1 <https://ci.swift.org//job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7>
Changes

Commit 2ebab526bfd7a5ce6ef460bebee0399fe669612d by xi_ge:
[FixCode] Apply coercion fixits on return statement and initialization

edit: test/FixCode/fixits-apply-objc.swift
edit: test/FixCode/fixits-apply-objc.swift.result
edit: lib/Sema/CSDiag.cpp
edit: test/FixCode/fixits-apply.swift.result

Commit 6447a2d16edcfbceab940d58a321b3ef70525f1a by daniel_dunbar:
[Basic] Update Thread shim to match changes on Linux side.

edit: Sources/Basic/Thread.swift


(Mishal Shah) #2

+ swift-lldb-dev group.

···

On Aug 18, 2016, at 9:47 PM, Daniel Dunbar <daniel_dunbar@apple.com> wrote:

Failing in lldb:
Command invoked: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py --executable /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb --rerun-all-issues -C /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang -s 2016-08-18-15_15_07 --results-port 41187 --inferior -p TestStubReverseConnect.py /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test --event-add-entries worker_index=14:int

Configuration: arch=x86_64 compiler=/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
----------------------------------------------------------------------
Collected 2 tests

======================================================================
ERROR: test_reverse_connect_works_llgs (TestStubReverseConnect.TestStubReverseConnect)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py", line 121, in wrapper
    func(*args, **kwargs)
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py", line 121, in wrapper
    func(*args, **kwargs)
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py", line 89, in test_reverse_connect_works_llgs
    self.reverse_connect_works()
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py", line 67, in reverse_connect_works
    (stub_socket, address) = self.listener_socket.accept()
  File "/usr/lib/python2.7/socket.py", line 206, in accept
    sock, addr = self._sock.accept()
timeout: timed out
Config=x86_64-/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang-3.9
----------------------------------------------------------------------
Ran 2 tests in 22.006s

RESULT: FAILED (0 passes, 0 failures, 1 errors, 1 skipped, 0 expected failures, 0 unexpected successes)
Session logs for test failures/errors/unexpected successes can be found in directory '2016-08-18-15_15_07'

<>[TestStubReverseConnect.py FAILED]
Command invoked: /usr/bin/python /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py --executable /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb --rerun-all-issues -C /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang -s 2016-08-18-15_15_07 --results-port 41187 --inferior -p TestStubReverseConnect.py /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test --event-add-entries worker_index=14:int

- Daniel

On Aug 18, 2016, at 9:18 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:

[FAILURE] oss-swift-package-linux-ubuntu-15_10 [#1930]

Build URL: https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/
Project: oss-swift-package-linux-ubuntu-15_10
Date of build: Thu, 18 Aug 2016 20:26:52 -0700
Build duration: 51 min
Identified problems:

Regression test failed: This build failed because a regression test in the test suite FAILed. Below is a list of all errors:
Indication 1 <https://ci.swift.org//job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7>
Changes

Commit 2ebab526bfd7a5ce6ef460bebee0399fe669612d by xi_ge:
[FixCode] Apply coercion fixits on return statement and initialization

edit: test/FixCode/fixits-apply-objc.swift
edit: test/FixCode/fixits-apply-objc.swift.result
edit: lib/Sema/CSDiag.cpp
edit: test/FixCode/fixits-apply.swift.result

Commit 6447a2d16edcfbceab940d58a321b3ef70525f1a by daniel_dunbar:
[Basic] Update Thread shim to match changes on Linux side.

edit: Sources/Basic/Thread.swift


(Kate Stone) #3

Where is this failure being reported? I’m not seeing anything on swift-3.0-branch or master on ci.swift.org <http://ci.swift.org/>.

Kate Stone k8stone@apple.com <mailto:k8stone@apple.com>
 Xcode Low Level Tools

···

On Aug 19, 2016, at 12:15 AM, mishal_shah via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:

+ swift-lldb-dev group.

On Aug 18, 2016, at 9:47 PM, Daniel Dunbar <daniel_dunbar@apple.com <mailto:daniel_dunbar@apple.com>> wrote:

Failing in lldb:
Command invoked: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py --executable /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb --rerun-all-issues -C /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang -s 2016-08-18-15_15_07 --results-port 41187 --inferior -p TestStubReverseConnect.py /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test --event-add-entries worker_index=14:int

Configuration: arch=x86_64 compiler=/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
----------------------------------------------------------------------
Collected 2 tests

======================================================================
ERROR: test_reverse_connect_works_llgs (TestStubReverseConnect.TestStubReverseConnect)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py", line 121, in wrapper
    func(*args, **kwargs)
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py", line 121, in wrapper
    func(*args, **kwargs)
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py", line 89, in test_reverse_connect_works_llgs
    self.reverse_connect_works()
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py", line 67, in reverse_connect_works
    (stub_socket, address) = self.listener_socket.accept()
  File "/usr/lib/python2.7/socket.py", line 206, in accept
    sock, addr = self._sock.accept()
timeout: timed out
Config=x86_64-/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang-3.9
----------------------------------------------------------------------
Ran 2 tests in 22.006s

RESULT: FAILED (0 passes, 0 failures, 1 errors, 1 skipped, 0 expected failures, 0 unexpected successes)
Session logs for test failures/errors/unexpected successes can be found in directory '2016-08-18-15_15_07'

<>[TestStubReverseConnect.py FAILED]
Command invoked: /usr/bin/python /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py --executable /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb --rerun-all-issues -C /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang -s 2016-08-18-15_15_07 --results-port 41187 --inferior -p TestStubReverseConnect.py /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test --event-add-entries worker_index=14:int

- Daniel

On Aug 18, 2016, at 9:18 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:

[FAILURE] oss-swift-package-linux-ubuntu-15_10 [#1930]

Build URL: https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/
Project: oss-swift-package-linux-ubuntu-15_10
Date of build: Thu, 18 Aug 2016 20:26:52 -0700
Build duration: 51 min
Identified problems:

Regression test failed: This build failed because a regression test in the test suite FAILed. Below is a list of all errors:
Indication 1 <https://ci.swift.org//job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7>
Changes

Commit 2ebab526bfd7a5ce6ef460bebee0399fe669612d by xi_ge:
[FixCode] Apply coercion fixits on return statement and initialization

edit: test/FixCode/fixits-apply-objc.swift
edit: test/FixCode/fixits-apply-objc.swift.result
edit: lib/Sema/CSDiag.cpp
edit: test/FixCode/fixits-apply.swift.result

Commit 6447a2d16edcfbceab940d58a321b3ef70525f1a by daniel_dunbar:
[Basic] Update Thread shim to match changes on Linux side.

edit: Sources/Basic/Thread.swift

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev


(xRain Dong) #4

RF在iiFc4 7

发送方式 Android 版 BlackBerry Hub

**发件人:**swift-dev@swift.org

**发送时间:**2016年8月19日 15:15

**收件人:**swift-lldb-dev@swift.org

**回复:**mishal_shah@apple.com

**抄送:**swift-dev@swift.org

**主题:**Re: [swift-dev] [Swift CI] Build Failure: OSS - Swift Package - Ubuntu 15.10 (master) #1930


(Todd Fiala) #5

It looks like this may be the packaging build:

oss-swift-package-linux-ubuntu-15_10

It was this build:

https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7

It has since gone blue.

I don’t think I recall ever seeing that test fail before in an intermittent way, but that might be what happened here.

-Todd

···

On Aug 19, 2016, at 11:15 AM, Kate Stone via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:

Where is this failure being reported? I’m not seeing anything on swift-3.0-branch or master on ci.swift.org <http://ci.swift.org/>.

Kate Stone k8stone@apple.com <mailto:k8stone@apple.com>
 Xcode Low Level Tools

On Aug 19, 2016, at 12:15 AM, mishal_shah via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

+ swift-lldb-dev group.

On Aug 18, 2016, at 9:47 PM, Daniel Dunbar <daniel_dunbar@apple.com <mailto:daniel_dunbar@apple.com>> wrote:

Failing in lldb:
Command invoked: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py --executable /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb --rerun-all-issues -C /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang -s 2016-08-18-15_15_07 --results-port 41187 --inferior -p TestStubReverseConnect.py /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test --event-add-entries worker_index=14:int

Configuration: arch=x86_64 compiler=/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
----------------------------------------------------------------------
Collected 2 tests

======================================================================
ERROR: test_reverse_connect_works_llgs (TestStubReverseConnect.TestStubReverseConnect)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py", line 121, in wrapper
    func(*args, **kwargs)
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py", line 121, in wrapper
    func(*args, **kwargs)
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py", line 89, in test_reverse_connect_works_llgs
    self.reverse_connect_works()
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py", line 67, in reverse_connect_works
    (stub_socket, address) = self.listener_socket.accept()
  File "/usr/lib/python2.7/socket.py", line 206, in accept
    sock, addr = self._sock.accept()
timeout: timed out
Config=x86_64-/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang-3.9
----------------------------------------------------------------------
Ran 2 tests in 22.006s

RESULT: FAILED (0 passes, 0 failures, 1 errors, 1 skipped, 0 expected failures, 0 unexpected successes)
Session logs for test failures/errors/unexpected successes can be found in directory '2016-08-18-15_15_07'

<>[TestStubReverseConnect.py FAILED]
Command invoked: /usr/bin/python /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py --executable /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb --rerun-all-issues -C /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang -s 2016-08-18-15_15_07 --results-port 41187 --inferior -p TestStubReverseConnect.py /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test --event-add-entries worker_index=14:int

- Daniel

On Aug 18, 2016, at 9:18 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:

[FAILURE] oss-swift-package-linux-ubuntu-15_10 [#1930]

Build URL: https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/
Project: oss-swift-package-linux-ubuntu-15_10
Date of build: Thu, 18 Aug 2016 20:26:52 -0700
Build duration: 51 min
Identified problems:

Regression test failed: This build failed because a regression test in the test suite FAILed. Below is a list of all errors:
Indication 1 <https://ci.swift.org//job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7>
Changes

Commit 2ebab526bfd7a5ce6ef460bebee0399fe669612d by xi_ge:
[FixCode] Apply coercion fixits on return statement and initialization

edit: test/FixCode/fixits-apply-objc.swift
edit: test/FixCode/fixits-apply-objc.swift.result
edit: lib/Sema/CSDiag.cpp
edit: test/FixCode/fixits-apply.swift.result

Commit 6447a2d16edcfbceab940d58a321b3ef70525f1a by daniel_dunbar:
[Basic] Update Thread shim to match changes on Linux side.

edit: Sources/Basic/Thread.swift

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev


(Enrico Granata) #6

It looks like this may be the packaging build:

oss-swift-package-linux-ubuntu-15_10

It was this build:

https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7

It has since gone blue.

I don’t think I recall ever seeing that test fail before in an intermittent way, but that might be what happened here.

Given the failure mode, it does sound likely:

  File "/usr/lib/python2.7/socket.py", line 206, in accept
    sock, addr = self._sock.accept()
timeout: timed out

-Todd

Where is this failure being reported? I’m not seeing anything on swift-3.0-branch or master on ci.swift.org <http://ci.swift.org/>.

Kate Stone k8stone@apple.com <mailto:k8stone@apple.com>
 Xcode Low Level Tools

+ swift-lldb-dev group.

Failing in lldb:
Command invoked: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py --executable /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb --rerun-all-issues -C /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang -s 2016-08-18-15_15_07 --results-port 41187 --inferior -p TestStubReverseConnect.py /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test --event-add-entries worker_index=14:int

Configuration: arch=x86_64 compiler=/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
----------------------------------------------------------------------
Collected 2 tests

======================================================================
ERROR: test_reverse_connect_works_llgs (TestStubReverseConnect.TestStubReverseConnect)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py", line 121, in wrapper
    func(*args, **kwargs)
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py", line 121, in wrapper
    func(*args, **kwargs)
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py", line 89, in test_reverse_connect_works_llgs
    self.reverse_connect_works()
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py", line 67, in reverse_connect_works
    (stub_socket, address) = self.listener_socket.accept()
  File "/usr/lib/python2.7/socket.py", line 206, in accept
    sock, addr = self._sock.accept()
timeout: timed out
Config=x86_64-/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang-3.9
----------------------------------------------------------------------
Ran 2 tests in 22.006s

RESULT: FAILED (0 passes, 0 failures, 1 errors, 1 skipped, 0 expected failures, 0 unexpected successes)
Session logs for test failures/errors/unexpected successes can be found in directory '2016-08-18-15_15_07'

<>[TestStubReverseConnect.py FAILED]
Command invoked: /usr/bin/python /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py --executable /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb --rerun-all-issues -C /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang -s 2016-08-18-15_15_07 --results-port 41187 --inferior -p TestStubReverseConnect.py /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test --event-add-entries worker_index=14:int

- Daniel

[FAILURE] oss-swift-package-linux-ubuntu-15_10 [#1930]

Build URL: https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/
Project: oss-swift-package-linux-ubuntu-15_10
Date of build: Thu, 18 Aug 2016 20:26:52 -0700
Build duration: 51 min
Identified problems:

Regression test failed: This build failed because a regression test in the test suite FAILed. Below is a list of all errors:
Indication 1 <https://ci.swift.org//job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7>
Changes

Commit 2ebab526bfd7a5ce6ef460bebee0399fe669612d by xi_ge:
[FixCode] Apply coercion fixits on return statement and initialization

edit: test/FixCode/fixits-apply-objc.swift
edit: test/FixCode/fixits-apply-objc.swift.result
edit: lib/Sema/CSDiag.cpp
edit: test/FixCode/fixits-apply.swift.result

Commit 6447a2d16edcfbceab940d58a321b3ef70525f1a by daniel_dunbar:
[Basic] Update Thread shim to match changes on Linux side.

edit: Sources/Basic/Thread.swift

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

Thanks,
- Enrico
:envelope_with_arrow: egranata@.com :phone:️ 27683

···

On Aug 19, 2016, at 11:20 AM, Todd Fiala via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:

On Aug 19, 2016, at 11:15 AM, Kate Stone via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

On Aug 19, 2016, at 12:15 AM, mishal_shah via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

On Aug 18, 2016, at 9:47 PM, Daniel Dunbar <daniel_dunbar@apple.com <mailto:daniel_dunbar@apple.com>> wrote:

On Aug 18, 2016, at 9:18 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:


(Todd Fiala) #7

It looks like this may be the packaging build:

oss-swift-package-linux-ubuntu-15_10

It was this build:

https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7

It has since gone blue.

I don’t think I recall ever seeing that test fail before in an intermittent way, but that might be what happened here.

Given the failure mode, it does sound likely:

  File "/usr/lib/python2.7/socket.py", line 206, in accept
    sock, addr = self._sock.accept()
timeout: timed out

If we see that again, we may be able to tweak the socket timeout to allow for heavily loaded test CI.

···

On Aug 19, 2016, at 1:42 PM, Enrico Granata <egranata@apple.com> wrote:

On Aug 19, 2016, at 11:20 AM, Todd Fiala via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

-Todd

On Aug 19, 2016, at 11:15 AM, Kate Stone via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

Where is this failure being reported? I’m not seeing anything on swift-3.0-branch or master on ci.swift.org <http://ci.swift.org/>.

Kate Stone k8stone@apple.com <mailto:k8stone@apple.com>
 Xcode Low Level Tools

On Aug 19, 2016, at 12:15 AM, mishal_shah via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

+ swift-lldb-dev group.

On Aug 18, 2016, at 9:47 PM, Daniel Dunbar <daniel_dunbar@apple.com <mailto:daniel_dunbar@apple.com>> wrote:

Failing in lldb:
Command invoked: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py --executable /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb --rerun-all-issues -C /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang -s 2016-08-18-15_15_07 --results-port 41187 --inferior -p TestStubReverseConnect.py /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test --event-add-entries worker_index=14:int

Configuration: arch=x86_64 compiler=/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
----------------------------------------------------------------------
Collected 2 tests

======================================================================
ERROR: test_reverse_connect_works_llgs (TestStubReverseConnect.TestStubReverseConnect)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py", line 121, in wrapper
    func(*args, **kwargs)
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py", line 121, in wrapper
    func(*args, **kwargs)
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py", line 89, in test_reverse_connect_works_llgs
    self.reverse_connect_works()
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py", line 67, in reverse_connect_works
    (stub_socket, address) = self.listener_socket.accept()
  File "/usr/lib/python2.7/socket.py", line 206, in accept
    sock, addr = self._sock.accept()
timeout: timed out
Config=x86_64-/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang-3.9
----------------------------------------------------------------------
Ran 2 tests in 22.006s

RESULT: FAILED (0 passes, 0 failures, 1 errors, 1 skipped, 0 expected failures, 0 unexpected successes)
Session logs for test failures/errors/unexpected successes can be found in directory '2016-08-18-15_15_07'

<>[TestStubReverseConnect.py FAILED]
Command invoked: /usr/bin/python /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py --executable /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb --rerun-all-issues -C /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang -s 2016-08-18-15_15_07 --results-port 41187 --inferior -p TestStubReverseConnect.py /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test --event-add-entries worker_index=14:int

- Daniel

On Aug 18, 2016, at 9:18 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:

[FAILURE] oss-swift-package-linux-ubuntu-15_10 [#1930]

Build URL: https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/
Project: oss-swift-package-linux-ubuntu-15_10
Date of build: Thu, 18 Aug 2016 20:26:52 -0700
Build duration: 51 min
Identified problems:

Regression test failed: This build failed because a regression test in the test suite FAILed. Below is a list of all errors:
Indication 1 <https://ci.swift.org//job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7>
Changes

Commit 2ebab526bfd7a5ce6ef460bebee0399fe669612d by xi_ge:
[FixCode] Apply coercion fixits on return statement and initialization

edit: test/FixCode/fixits-apply-objc.swift
edit: test/FixCode/fixits-apply-objc.swift.result
edit: lib/Sema/CSDiag.cpp
edit: test/FixCode/fixits-apply.swift.result

Commit 6447a2d16edcfbceab940d58a321b3ef70525f1a by daniel_dunbar:
[Basic] Update Thread shim to match changes on Linux side.

edit: Sources/Basic/Thread.swift

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

Thanks,
- Enrico
:envelope_with_arrow: egranata@ <about:blank> <about:blank>.com <about:blank> :phone:️ 27683


(Daniel Dunbar) #8

The amount of flakiness in the LLDB tests is disturbing... is there a high-level bug tracking improving this? It seems like it might be worth running a stress test of the tests on a loaded machine to try and shake out such problems.

- Daniel

···

On Aug 19, 2016, at 1:58 PM, Todd Fiala via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:

On Aug 19, 2016, at 1:42 PM, Enrico Granata <egranata@apple.com <mailto:egranata@apple.com>> wrote:

On Aug 19, 2016, at 11:20 AM, Todd Fiala via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

It looks like this may be the packaging build:

oss-swift-package-linux-ubuntu-15_10

It was this build:

https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7

It has since gone blue.

I don’t think I recall ever seeing that test fail before in an intermittent way, but that might be what happened here.

Given the failure mode, it does sound likely:

  File "/usr/lib/python2.7/socket.py", line 206, in accept
    sock, addr = self._sock.accept()
timeout: timed out

If we see that again, we may be able to tweak the socket timeout to allow for heavily loaded test CI.

-Todd

On Aug 19, 2016, at 11:15 AM, Kate Stone via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

Where is this failure being reported? I’m not seeing anything on swift-3.0-branch or master on ci.swift.org <http://ci.swift.org/>.

Kate Stone k8stone@apple.com <mailto:k8stone@apple.com>
 Xcode Low Level Tools

On Aug 19, 2016, at 12:15 AM, mishal_shah via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

+ swift-lldb-dev group.

On Aug 18, 2016, at 9:47 PM, Daniel Dunbar <daniel_dunbar@apple.com <mailto:daniel_dunbar@apple.com>> wrote:

Failing in lldb:
Command invoked: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py --executable /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb --rerun-all-issues -C /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang -s 2016-08-18-15_15_07 --results-port 41187 --inferior -p TestStubReverseConnect.py /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test --event-add-entries worker_index=14:int

Configuration: arch=x86_64 compiler=/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
----------------------------------------------------------------------
Collected 2 tests

======================================================================
ERROR: test_reverse_connect_works_llgs (TestStubReverseConnect.TestStubReverseConnect)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py", line 121, in wrapper
    func(*args, **kwargs)
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py", line 121, in wrapper
    func(*args, **kwargs)
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py", line 89, in test_reverse_connect_works_llgs
    self.reverse_connect_works()
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py", line 67, in reverse_connect_works
    (stub_socket, address) = self.listener_socket.accept()
  File "/usr/lib/python2.7/socket.py", line 206, in accept
    sock, addr = self._sock.accept()
timeout: timed out
Config=x86_64-/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang-3.9
----------------------------------------------------------------------
Ran 2 tests in 22.006s

RESULT: FAILED (0 passes, 0 failures, 1 errors, 1 skipped, 0 expected failures, 0 unexpected successes)
Session logs for test failures/errors/unexpected successes can be found in directory '2016-08-18-15_15_07'

<>[TestStubReverseConnect.py FAILED]
Command invoked: /usr/bin/python /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py --executable /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb --rerun-all-issues -C /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang -s 2016-08-18-15_15_07 --results-port 41187 --inferior -p TestStubReverseConnect.py /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test --event-add-entries worker_index=14:int

- Daniel

On Aug 18, 2016, at 9:18 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:

[FAILURE] oss-swift-package-linux-ubuntu-15_10 [#1930]

Build URL: https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/
Project: oss-swift-package-linux-ubuntu-15_10
Date of build: Thu, 18 Aug 2016 20:26:52 -0700
Build duration: 51 min
Identified problems:

Regression test failed: This build failed because a regression test in the test suite FAILed. Below is a list of all errors:
Indication 1 <https://ci.swift.org//job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7>
Changes

Commit 2ebab526bfd7a5ce6ef460bebee0399fe669612d by xi_ge:
[FixCode] Apply coercion fixits on return statement and initialization

edit: test/FixCode/fixits-apply-objc.swift
edit: test/FixCode/fixits-apply-objc.swift.result
edit: lib/Sema/CSDiag.cpp
edit: test/FixCode/fixits-apply.swift.result

Commit 6447a2d16edcfbceab940d58a321b3ef70525f1a by daniel_dunbar:
[Basic] Update Thread shim to match changes on Linux side.

edit: Sources/Basic/Thread.swift

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

Thanks,
- Enrico
:envelope_with_arrow: egranata@ <about:blank> <about:blank>.com <about:blank> :phone:️ 27683

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev


(Kate Stone) #9

I don’t disagree. There’s no clear signal to be had from a flaky test, so we’ve already made strides with some of the worst offenders. The existing timeout strategy

We’re actively working on strategies to make the test suite more robust (moving pexcept tests to a different strategy, for one, as we have for one of the most troublesome examples.) I expect this to be part of our quality push for the next year. We have existing radars tracking some of the expected work on the suite, but just in case this isn’t being tracked I’ve filed:

<rdar://problem/27930082> Replace the timeout strategy used in our test suite with something more robust

Kate Stone k8stone@apple.com <mailto:k8stone@apple.com>
 Xcode Low Level Tools

···

On Aug 19, 2016, at 2:10 PM, Daniel Dunbar via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:

The amount of flakiness in the LLDB tests is disturbing... is there a high-level bug tracking improving this? It seems like it might be worth running a stress test of the tests on a loaded machine to try and shake out such problems.

- Daniel

On Aug 19, 2016, at 1:58 PM, Todd Fiala via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

On Aug 19, 2016, at 1:42 PM, Enrico Granata <egranata@apple.com <mailto:egranata@apple.com>> wrote:

On Aug 19, 2016, at 11:20 AM, Todd Fiala via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

It looks like this may be the packaging build:

oss-swift-package-linux-ubuntu-15_10

It was this build:

https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7

It has since gone blue.

I don’t think I recall ever seeing that test fail before in an intermittent way, but that might be what happened here.

Given the failure mode, it does sound likely:

  File "/usr/lib/python2.7/socket.py", line 206, in accept
    sock, addr = self._sock.accept()
timeout: timed out

If we see that again, we may be able to tweak the socket timeout to allow for heavily loaded test CI.

-Todd

On Aug 19, 2016, at 11:15 AM, Kate Stone via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

Where is this failure being reported? I’m not seeing anything on swift-3.0-branch or master on ci.swift.org <http://ci.swift.org/>.

Kate Stone k8stone@apple.com <mailto:k8stone@apple.com>
 Xcode Low Level Tools

On Aug 19, 2016, at 12:15 AM, mishal_shah via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

+ swift-lldb-dev group.

On Aug 18, 2016, at 9:47 PM, Daniel Dunbar <daniel_dunbar@apple.com <mailto:daniel_dunbar@apple.com>> wrote:

Failing in lldb:
Command invoked: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py --executable /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb --rerun-all-issues -C /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang -s 2016-08-18-15_15_07 --results-port 41187 --inferior -p TestStubReverseConnect.py /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test --event-add-entries worker_index=14:int

Configuration: arch=x86_64 compiler=/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
----------------------------------------------------------------------
Collected 2 tests

======================================================================
ERROR: test_reverse_connect_works_llgs (TestStubReverseConnect.TestStubReverseConnect)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py", line 121, in wrapper
    func(*args, **kwargs)
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py", line 121, in wrapper
    func(*args, **kwargs)
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py", line 89, in test_reverse_connect_works_llgs
    self.reverse_connect_works()
  File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py", line 67, in reverse_connect_works
    (stub_socket, address) = self.listener_socket.accept()
  File "/usr/lib/python2.7/socket.py", line 206, in accept
    sock, addr = self._sock.accept()
timeout: timed out
Config=x86_64-/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang-3.9
----------------------------------------------------------------------
Ran 2 tests in 22.006s

RESULT: FAILED (0 passes, 0 failures, 1 errors, 1 skipped, 0 expected failures, 0 unexpected successes)
Session logs for test failures/errors/unexpected successes can be found in directory '2016-08-18-15_15_07'

<>[TestStubReverseConnect.py FAILED]
Command invoked: /usr/bin/python /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py --executable /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb --rerun-all-issues -C /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang -s 2016-08-18-15_15_07 --results-port 41187 --inferior -p TestStubReverseConnect.py /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test --event-add-entries worker_index=14:int

- Daniel

On Aug 18, 2016, at 9:18 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:

[FAILURE] oss-swift-package-linux-ubuntu-15_10 [#1930]

Build URL: https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/
Project: oss-swift-package-linux-ubuntu-15_10
Date of build: Thu, 18 Aug 2016 20:26:52 -0700
Build duration: 51 min
Identified problems:

Regression test failed: This build failed because a regression test in the test suite FAILed. Below is a list of all errors:
Indication 1 <https://ci.swift.org//job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7>
Changes

Commit 2ebab526bfd7a5ce6ef460bebee0399fe669612d by xi_ge:
[FixCode] Apply coercion fixits on return statement and initialization

edit: test/FixCode/fixits-apply-objc.swift
edit: test/FixCode/fixits-apply-objc.swift.result
edit: lib/Sema/CSDiag.cpp
edit: test/FixCode/fixits-apply.swift.result

Commit 6447a2d16edcfbceab940d58a321b3ef70525f1a by daniel_dunbar:
[Basic] Update Thread shim to match changes on Linux side.

edit: Sources/Basic/Thread.swift

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

Thanks,
- Enrico
:envelope_with_arrow: egranata@ <about:blank> <about:blank>.com <about:blank> :phone:️ 27683

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev


(Todd Fiala) #10

First off, I think the whole LLDB team can agree on the simple principle
that tests that sometimes fail is not something we’re shooting for, so
we’re on the same page with the general sentiment.

But, rather than talking in sweeping generalizations, let’s take this exact
failure as an example. It was a socket attempting to connect. It uses
general socket timeouts. If it’s using a default, it will typically be
something like 30 seconds. Per my previous comment, this is not something
we see fail with any kind of frequency. We just had a failure on it now.
Is the solution to say we want 35 seconds? 1 minute? 2 minutes? We’re
using end-to-end testing here, and therefore we are subject to how a real
system behaves. If we crank that up, there could be a time where we
overload the server further, and the socket facility just fails. What do
you do then, claim the test as unworthy?

The challenge we run into on LLDB is that, while we do have a growing
number of unit tests that are strict input/output verification, far too
much of a functioning debugger’s operations are not covered adequately by
that. So we have a large body of end to end tests. Sometimes those fail
under heavy load. Sometimes those issues are ours - we have hidden races
that are only exposed when the machine is grinding to a halt. Sometimes it
is that the OS fails to handle requests for resources that the debugger
needs.

So it isn’t as simple as saying “file a bug to make the LLDB test suite
more reliable.” (I wish it were!)

On the positive side, we have more resources going into improving our
quality in the upcoming months, and we now have a dedicated quality
engineer working with us. I expect this will help things as we move
forward. And we’re looking to sink some of the system testing that
currently exists end-to-end in LLDB to a more input-output style testing
that we can do without needing the whole of LLDB (e.g. DWARF verification,
DWARF type round-tripping through the compiler, etc. - Greg and Sean can
speak more to what we’re doing there).

The net effect is that we should be hitting less of this noise.

-Todd

···

On August 19, 2016 at 2:10:14 PM, Daniel Dunbar via swift-lldb-dev ( swift-lldb-dev@swift.org) wrote:

The amount of flakiness in the LLDB tests is disturbing... is there a
high-level bug tracking improving this? It seems like it might be worth
running a stress test of the tests on a loaded machine to try and shake out
such problems.

- Daniel

On Aug 19, 2016, at 1:58 PM, Todd Fiala via swift-lldb-dev < swift-lldb-dev@swift.org> wrote:

On Aug 19, 2016, at 1:42 PM, Enrico Granata <egranata@apple.com> wrote:

On Aug 19, 2016, at 11:20 AM, Todd Fiala via swift-lldb-dev < swift-lldb-dev@swift.org> wrote:

It looks like this may be the packaging build:

oss-swift-package-linux-ubuntu-15_10

It was this build:

https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7

It has since gone blue.

I don’t think I recall ever seeing that test fail before in an intermittent
way, but that might be what happened here.

Given the failure mode, it does sound likely:

File "/usr/lib/python2.7/socket.py", line 206, in accept sock, addr =
self._sock.accept() timeout: timed out

If we see that again, we may be able to tweak the socket timeout to allow
for heavily loaded test CI.

-Todd

On Aug 19, 2016, at 11:15 AM, Kate Stone via swift-lldb-dev < swift-lldb-dev@swift.org> wrote:

Where is this failure being reported? I’m not seeing anything on
swift-3.0-branch or master on ci.swift.org.

Kate Stone k8stone@apple.com
 Xcode Low Level Tools

On Aug 19, 2016, at 12:15 AM, mishal_shah via swift-lldb-dev < swift-lldb-dev@swift.org> wrote:

+ swift-lldb-dev group.

On Aug 18, 2016, at 9:47 PM, Daniel Dunbar <daniel_dunbar@apple.com> wrote:

Failing in lldb:
Command invoked:
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py
--executable
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb
--rerun-all-issues -C
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
-s 2016-08-18-15_15_07 --results-port 41187 --inferior -p
TestStubReverseConnect.py
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test
--event-add-entries worker_index=14:int Configuration: arch=x86_64
compiler=/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
----------------------------------------------------------------------
Collected 2 tests

ERROR: test_reverse_connect_works_llgs
(TestStubReverseConnect.TestStubReverseConnect)
----------------------------------------------------------------------
Traceback (most recent call last): File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py",
line 121, in wrapper func(*args, **kwargs) File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py",
line 121, in wrapper func(*args, **kwargs) File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py",
line 89, in test_reverse_connect_works_llgs self.reverse_connect_works()
File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py",
line 67, in reverse_connect_works (stub_socket, address) =
self.listener_socket.accept() File "/usr/lib/python2.7/socket.py", line
206, in accept sock, addr = self._sock.accept() timeout: timed out
Config=x86_64-/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang-3.9
---------------------------------------------------------------------- Ran
2 tests in 22.006s RESULT: FAILED (0 passes, 0 failures, 1 errors, 1
skipped, 0 expected failures, 0 unexpected successes) Session logs for test
failures/errors/unexpected successes can be found in directory
'2016-08-18-15_15_07' [TestStubReverseConnect.py FAILED] Command invoked:
/usr/bin/python
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py
--executable
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb
--rerun-all-issues -C
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
-s 2016-08-18-15_15_07 --results-port 41187 --inferior -p
TestStubReverseConnect.py
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test
--event-add-entries worker_index=14:int

- Daniel

On Aug 18, 2016, at 9:18 PM, no-reply@swift.org wrote:

[FAILURE] oss-swift-package-linux-ubuntu-15_10 [#1930]
Build URL:
https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/
Project: oss-swift-package-linux-ubuntu-15_10
Date of build: Thu, 18 Aug 2016 20:26:52 -0700
Build duration: 51 min Identified problems:

   - Regression test failed: This build failed because a regression test in
   the test suite FAILed. Below is a list of all errors:
      - Indication 1
      <https://ci.swift.org//job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7>

Changes

   - Commit *2ebab526bfd7a5ce6ef460bebee0399fe669612d* by *xi_ge:*

   [FixCode] Apply coercion fixits on return statement and initialization
   - *edit*: test/FixCode/fixits-apply-objc.swift
      - *edit*: test/FixCode/fixits-apply-objc.swift.result
      - *edit*: lib/Sema/CSDiag.cpp
      - *edit*: test/FixCode/fixits-apply.swift.result
   -
   - Commit *6447a2d16edcfbceab940d58a321b3ef70525f1a* by *daniel_dunbar:*

   [Basic] Update Thread shim to match changes on Linux side.
   - *edit*: Sources/Basic/Thread.swift

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

Thanks,
*- Enrico*
:envelope_with_arrow: egranata@ <about:blank> <about:blank>.com <about:blank> :phone:️ 27683

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev


(Daniel Dunbar) #11

First off, I think the whole LLDB team can agree on the simple principle that tests that sometimes fail is not something we’re shooting for, so we’re on the same page with the general sentiment.

But, rather than talking in sweeping generalizations, let’s take this exact failure as an example. It was a socket attempting to connect. It uses general socket timeouts. If it’s using a default, it will typically be something like 30 seconds. Per my previous comment, this is not something we see fail with any kind of frequency. We just had a failure on it now. Is the solution to say we want 35 seconds? 1 minute? 2 minutes? We’re using end-to-end testing here, and therefore we are subject to how a real system behaves. If we crank that up, there could be a time where we overload the server further, and the socket facility just fails. What do you do then, claim the test as unworthy?

The challenge we run into on LLDB is that, while we do have a growing number of unit tests that are strict input/output verification, far too much of a functioning debugger’s operations are not covered adequately by that. So we have a large body of end to end tests. Sometimes those fail under heavy load. Sometimes those issues are ours - we have hidden races that are only exposed when the machine is grinding to a halt. Sometimes it is that the OS fails to handle requests for resources that the debugger needs.

So it isn’t as simple as saying “file a bug to make the LLDB test suite more reliable.” (I wish it were!)

This is specifically why I proposed a concrete strategy like running a stress test of the test suite. That is actionable, and should shake some bugs I would hope?

Another actionable strategies (although one I'm not a big fan of) is enhance the test suite to automatically re-attempt flaky tests.

On the positive side, we have more resources going into improving our quality in the upcoming months, and we now have a dedicated quality engineer working with us. I expect this will help things as we move forward. And we’re looking to sink some of the system testing that currently exists end-to-end in LLDB to a more input-output style testing that we can do without needing the whole of LLDB (e.g. DWARF verification, DWARF type round-tripping through the compiler, etc. - Greg and Sean can speak more to what we’re doing there).

Great!

- Daniel

···

On Aug 19, 2016, at 2:22 PM, Todd Fiala <todd.fiala@gmail.com> wrote:

The net effect is that we should be hitting less of this noise.

-Todd
On August 19, 2016 at 2:10:14 PM, Daniel Dunbar via swift-lldb-dev (swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>) wrote:

The amount of flakiness in the LLDB tests is disturbing... is there a high-level bug tracking improving this? It seems like it might be worth running a stress test of the tests on a loaded machine to try and shake out such problems.

- Daniel

On Aug 19, 2016, at 1:58 PM, Todd Fiala via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

On Aug 19, 2016, at 1:42 PM, Enrico Granata <egranata@apple.com <mailto:egranata@apple.com>> wrote:

On Aug 19, 2016, at 11:20 AM, Todd Fiala via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

It looks like this may be the packaging build:

oss-swift-package-linux-ubuntu-15_10

It was this build:

https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7

It has since gone blue.

I don’t think I recall ever seeing that test fail before in an intermittent way, but that might be what happened here.

Given the failure mode, it does sound likely:

File "/usr/lib/python2.7/socket.py", line 206, in accept sock, addr
= self._sock.accept() timeout: timed out

If we see that again, we may be able to tweak the socket timeout to allow for heavily loaded test CI.

-Todd

On Aug 19, 2016, at 11:15 AM, Kate Stone via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

Where is this failure being reported? I’m not seeing anything on swift-3.0-branch or master on ci.swift.org <http://ci.swift.org/>.

Kate Stone k8stone@apple.com <mailto:k8stone@apple.com>
 Xcode Low Level Tools

On Aug 19, 2016, at 12:15 AM, mishal_shah via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

+ swift-lldb-dev group.

On Aug 18, 2016, at 9:47 PM, Daniel Dunbar <daniel_dunbar@apple.com <mailto:daniel_dunbar@apple.com>> wrote:

Failing in lldb:

Command invoked:
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py
--executable
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb
--rerun-all-issues -C
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
-s 2016-08-18-15_15_07 --results-port 41187 --inferior -p
TestStubReverseConnect.py
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test
--event-add-entries worker_index=14:int Configuration: arch=x86_64
compiler=/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
----------------------------------------------------------------------
Collected 2 tests

ERROR: test_reverse_connect_works_llgs
(TestStubReverseConnect.TestStubReverseConnect)
----------------------------------------------------------------------
Traceback (most recent call last): File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py",
line 121, in wrapper func(*args, **kwargs) File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py",
line 121, in wrapper func(*args, **kwargs) File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py",
line 89, in test_reverse_connect_works_llgs
self.reverse_connect_works() File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py",
line 67, in reverse_connect_works (stub_socket, address) =
self.listener_socket.accept() File "/usr/lib/python2.7/socket.py",
line 206, in accept sock, addr = self._sock.accept() timeout: timed
out
Config=x86_64-/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang-3.9
----------------------------------------------------------------------
Ran 2 tests in 22.006s RESULT: FAILED (0 passes, 0 failures, 1
errors, 1 skipped, 0 expected failures, 0 unexpected successes)
Session logs for test failures/errors/unexpected successes can be
found in directory '2016-08-18-15_15_07'

<>[TestStubReverseConnect.py
FAILED]
Command invoked: /usr/bin/python
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py
--executable
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb
--rerun-all-issues -C
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
-s 2016-08-18-15_15_07 --results-port 41187 --inferior -p
TestStubReverseConnect.py
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test
--event-add-entries worker_index=14:int

- Daniel

On Aug 18, 2016, at 9:18 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:

[FAILURE] oss-swift-package-linux-ubuntu-15_10 [#1930]

Build URL: https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/
Project: oss-swift-package-linux-ubuntu-15_10
Date of build: Thu, 18 Aug 2016 20:26:52 -0700
Build duration: 51 min
Identified problems:

Regression test failed: This build failed because a regression test in the test suite FAILed. Below is a list of all errors:
Indication 1 <https://ci.swift.org//job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7>
Changes

Commit 2ebab526bfd7a5ce6ef460bebee0399fe669612d by xi_ge:
[FixCode] Apply
coercion fixits on return statement and initialization

edit: test/FixCode/fixits-apply-objc.swift
edit: test/FixCode/fixits-apply-objc.swift.result
edit: lib/Sema/CSDiag.cpp
edit: test/FixCode/fixits-apply.swift.result

Commit 6447a2d16edcfbceab940d58a321b3ef70525f1a by daniel_dunbar:
[Basic] Update
Thread shim to match changes on Linux side.

edit: Sources/Basic/Thread.swift

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

Thanks,
- Enrico
:envelope_with_arrow: egranata@ <about:blank> <about:blank>.com <about:blank> :phone:️ 27683

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev


(Jim Ingham) #12

First off, I think the whole LLDB team can agree on the simple principle that tests that sometimes fail is not something we’re shooting for, so we’re on the same page with the general sentiment.

But, rather than talking in sweeping generalizations, let’s take this exact failure as an example. It was a socket attempting to connect. It uses general socket timeouts. If it’s using a default, it will typically be something like 30 seconds. Per my previous comment, this is not something we see fail with any kind of frequency. We just had a failure on it now. Is the solution to say we want 35 seconds? 1 minute? 2 minutes? We’re using end-to-end testing here, and therefore we are subject to how a real system behaves. If we crank that up, there could be a time where we overload the server further, and the socket facility just fails. What do you do then, claim the test as unworthy?

The challenge we run into on LLDB is that, while we do have a growing number of unit tests that are strict input/output verification, far too much of a functioning debugger’s operations are not covered adequately by that. So we have a large body of end to end tests. Sometimes those fail under heavy load. Sometimes those issues are ours - we have hidden races that are only exposed when the machine is grinding to a halt. Sometimes it is that the OS fails to handle requests for resources that the debugger needs.

So it isn’t as simple as saying “file a bug to make the LLDB test suite more reliable.” (I wish it were!)

This is specifically why I proposed a concrete strategy like running a stress test of the test suite. That is actionable, and should shake some bugs I would hope?

Another actionable strategies (although one I'm not a big fan of) is enhance the test suite to automatically re-attempt flaky tests.

The test suite already does this.

Jim

···

On Aug 19, 2016, at 2:26 PM, Daniel Dunbar via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:

On Aug 19, 2016, at 2:22 PM, Todd Fiala <todd.fiala@gmail.com> wrote:

On the positive side, we have more resources going into improving our quality in the upcoming months, and we now have a dedicated quality engineer working with us. I expect this will help things as we move forward. And we’re looking to sink some of the system testing that currently exists end-to-end in LLDB to a more input-output style testing that we can do without needing the whole of LLDB (e.g. DWARF verification, DWARF type round-tripping through the compiler, etc. - Greg and Sean can speak more to what we’re doing there).

Great!

- Daniel

The net effect is that we should be hitting less of this noise.

-Todd
On August 19, 2016 at 2:10:14 PM, Daniel Dunbar via swift-lldb-dev (swift-lldb-dev@swift.org) wrote:

The amount of flakiness in the LLDB tests is disturbing... is there a high-level bug tracking improving this? It seems like it might be worth running a stress test of the tests on a loaded machine to try and shake out such problems.

- Daniel

On Aug 19, 2016, at 1:58 PM, Todd Fiala via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:

On Aug 19, 2016, at 1:42 PM, Enrico Granata <egranata@apple.com> wrote:

On Aug 19, 2016, at 11:20 AM, Todd Fiala via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:

It looks like this may be the packaging build:

oss-swift-package-linux-ubuntu-15_10

It was this build:

https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7

It has since gone blue.

I don’t think I recall ever seeing that test fail before in an intermittent way, but that might be what happened here.

Given the failure mode, it does sound likely:

File "/usr/lib/python2.7/socket.py", line 206, in accept sock, addr
= self._sock.accept() timeout: timed out

If we see that again, we may be able to tweak the socket timeout to allow for heavily loaded test CI.

-Todd

On Aug 19, 2016, at 11:15 AM, Kate Stone via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:

Where is this failure being reported? I’m not seeing anything on swift-3.0-branch or master on ci.swift.org.

Kate Stone k8stone@apple.com
 Xcode Low Level Tools

On Aug 19, 2016, at 12:15 AM, mishal_shah via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:

+ swift-lldb-dev group.

On Aug 18, 2016, at 9:47 PM, Daniel Dunbar <daniel_dunbar@apple.com> wrote:

Failing in lldb:

Command invoked:
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py
--executable
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb
--rerun-all-issues -C
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
-s 2016-08-18-15_15_07 --results-port 41187 --inferior -p
TestStubReverseConnect.py
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test
--event-add-entries worker_index=14:int Configuration: arch=x86_64
compiler=/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
----------------------------------------------------------------------
Collected 2 tests

ERROR: test_reverse_connect_works_llgs
(TestStubReverseConnect.TestStubReverseConnect)
----------------------------------------------------------------------
Traceback (most recent call last): File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py",
line 121, in wrapper func(*args, **kwargs) File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py",
line 121, in wrapper func(*args, **kwargs) File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py",
line 89, in test_reverse_connect_works_llgs
self.reverse_connect_works() File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py",
line 67, in reverse_connect_works (stub_socket, address) =
self.listener_socket.accept() File "/usr/lib/python2.7/socket.py",
line 206, in accept sock, addr = self._sock.accept() timeout: timed
out
Config=x86_64-/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang-3.9
----------------------------------------------------------------------
Ran 2 tests in 22.006s RESULT: FAILED (0 passes, 0 failures, 1
errors, 1 skipped, 0 expected failures, 0 unexpected successes)
Session logs for test failures/errors/unexpected successes can be
found in directory '2016-08-18-15_15_07'

[TestStubReverseConnect.py
FAILED]

Command invoked: /usr/bin/python
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py
--executable
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb
--rerun-all-issues -C
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
-s 2016-08-18-15_15_07 --results-port 41187 --inferior -p
TestStubReverseConnect.py
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test
--event-add-entries worker_index=14:int

- Daniel

On Aug 18, 2016, at 9:18 PM, no-reply@swift.org wrote:

[FAILURE] oss-swift-package-linux-ubuntu-15_10 [#1930]

Build URL: https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/
Project: oss-swift-package-linux-ubuntu-15_10
Date of build: Thu, 18 Aug 2016 20:26:52 -0700
Build duration: 51 min
Identified problems:

  • Regression test failed: This build failed because a regression test in the test suite FAILed. Below is a list of all errors:
    • Indication 1

Changes

  • Commit 2ebab526bfd7a5ce6ef460bebee0399fe669612d by xi_ge:
[FixCode] Apply
coercion fixits on return statement and initialization

    • edit: test/FixCode/fixits-apply-objc.swift
    • edit: test/FixCode/fixits-apply-objc.swift.result
    • edit: lib/Sema/CSDiag.cpp
    • edit: test/FixCode/fixits-apply.swift.result

  • Commit 6447a2d16edcfbceab940d58a321b3ef70525f1a by daniel_dunbar:
[Basic] Update
Thread shim to match changes on Linux side.

    • edit: Sources/Basic/Thread.swift

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

Thanks,
- Enrico
:envelope_with_arrow: egranata@.com :phone:️ 27683

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev


(Todd Fiala) #13

First off, I think the whole LLDB team can agree on the simple principle
that tests that sometimes fail is not something we’re shooting for, so
we’re on the same page with the general sentiment.

But, rather than talking in sweeping generalizations, let’s take this exact
failure as an example. It was a socket attempting to connect. It uses
general socket timeouts. If it’s using a default, it will typically be
something like 30 seconds. Per my previous comment, this is not something
we see fail with any kind of frequency. We just had a failure on it now.
Is the solution to say we want 35 seconds? 1 minute? 2 minutes? We’re
using end-to-end testing here, and therefore we are subject to how a real
system behaves. If we crank that up, there could be a time where we
overload the server further, and the socket facility just fails. What do
you do then, claim the test as unworthy?

The challenge we run into on LLDB is that, while we do have a growing
number of unit tests that are strict input/output verification, far too
much of a functioning debugger’s operations are not covered adequately by
that. So we have a large body of end to end tests. Sometimes those fail
under heavy load. Sometimes those issues are ours - we have hidden races
that are only exposed when the machine is grinding to a halt. Sometimes it
is that the OS fails to handle requests for resources that the debugger
needs.

So it isn’t as simple as saying “file a bug to make the LLDB test suite
more reliable.” (I wish it were!)

This is specifically why I proposed a concrete strategy like running a
stress test of the test suite. That is actionable, and should shake some
bugs I would hope?

The stress test idea seems like a good one to me. I think that would be a
great opportunity for our quality engineer, who is also looking at other
aspects like code coverage.

-Todd

Another actionable strategies (although one I'm not a big fan of) is
enhance the test suite to automatically re-attempt flaky tests.

On the positive side, we have more resources going into improving our
quality in the upcoming months, and we now have a dedicated quality
engineer working with us. I expect this will help things as we move
forward. And we’re looking to sink some of the system testing that
currently exists end-to-end in LLDB to a more input-output style testing
that we can do without needing the whole of LLDB (e.g. DWARF verification,
DWARF type round-tripping through the compiler, etc. - Greg and Sean can
speak more to what we’re doing there).

Great!

- Daniel

The net effect is that we should be hitting less of this noise.

-Todd

···

On August 19, 2016 at 2:26:18 PM, Daniel Dunbar (daniel_dunbar@apple.com) wrote:
On Aug 19, 2016, at 2:22 PM, Todd Fiala <todd.fiala@gmail.com> wrote:

On August 19, 2016 at 2:10:14 PM, Daniel Dunbar via swift-lldb-dev ( swift-lldb-dev@swift.org) wrote:

The amount of flakiness in the LLDB tests is disturbing... is there a
high-level bug tracking improving this? It seems like it might be worth
running a stress test of the tests on a loaded machine to try and shake out
such problems.

- Daniel

On Aug 19, 2016, at 1:58 PM, Todd Fiala via swift-lldb-dev < swift-lldb-dev@swift.org> wrote:

On Aug 19, 2016, at 1:42 PM, Enrico Granata <egranata@apple.com> wrote:

On Aug 19, 2016, at 11:20 AM, Todd Fiala via swift-lldb-dev < swift-lldb-dev@swift.org> wrote:

It looks like this may be the packaging build:

oss-swift-package-linux-ubuntu-15_10

It was this build:

https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7

It has since gone blue.

I don’t think I recall ever seeing that test fail before in an intermittent
way, but that might be what happened here.

Given the failure mode, it does sound likely:

File "/usr/lib/python2.7/socket.py", line 206, in accept sock, addr =
self._sock.accept() timeout: timed out

If we see that again, we may be able to tweak the socket timeout to allow
for heavily loaded test CI.

-Todd

On Aug 19, 2016, at 11:15 AM, Kate Stone via swift-lldb-dev < swift-lldb-dev@swift.org> wrote:

Where is this failure being reported? I’m not seeing anything on
swift-3.0-branch or master on ci.swift.org.

Kate Stone k8stone@apple.com
 Xcode Low Level Tools

On Aug 19, 2016, at 12:15 AM, mishal_shah via swift-lldb-dev < swift-lldb-dev@swift.org> wrote:

+ swift-lldb-dev group.

On Aug 18, 2016, at 9:47 PM, Daniel Dunbar <daniel_dunbar@apple.com> wrote:

Failing in lldb:
Command invoked:
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py
--executable
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb
--rerun-all-issues -C
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
-s 2016-08-18-15_15_07 --results-port 41187 --inferior -p
TestStubReverseConnect.py
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test
--event-add-entries worker_index=14:int Configuration: arch=x86_64
compiler=/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
----------------------------------------------------------------------
Collected 2 tests

ERROR: test_reverse_connect_works_llgs
(TestStubReverseConnect.TestStubReverseConnect)
----------------------------------------------------------------------
Traceback (most recent call last): File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py",
line 121, in wrapper func(*args, **kwargs) File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py",
line 121, in wrapper func(*args, **kwargs) File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py",
line 89, in test_reverse_connect_works_llgs self.reverse_connect_works()
File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py",
line 67, in reverse_connect_works (stub_socket, address) =
self.listener_socket.accept() File "/usr/lib/python2.7/socket.py", line
206, in accept sock, addr = self._sock.accept() timeout: timed out
Config=x86_64-/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang-3.9
---------------------------------------------------------------------- Ran
2 tests in 22.006s RESULT: FAILED (0 passes, 0 failures, 1 errors, 1
skipped, 0 expected failures, 0 unexpected successes) Session logs for test
failures/errors/unexpected successes can be found in directory
'2016-08-18-15_15_07'[TestStubReverseConnect.py FAILED] Command invoked:
/usr/bin/python
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py
--executable
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb
--rerun-all-issues -C
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
-s 2016-08-18-15_15_07 --results-port 41187 --inferior -p
TestStubReverseConnect.py
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test
--event-add-entries worker_index=14:int

- Daniel

On Aug 18, 2016, at 9:18 PM, no-reply@swift.org wrote:

[FAILURE] oss-swift-package-linux-ubuntu-15_10 [#1930]
Build URL:
https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/
Project: oss-swift-package-linux-ubuntu-15_10
Date of build: Thu, 18 Aug 2016 20:26:52 -0700
Build duration: 51 min Identified problems:

   - Regression test failed: This build failed because a regression test in
   the test suite FAILed. Below is a list of all errors:
      - Indication 1
      <https://ci.swift.org//job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7>

Changes

   - Commit *2ebab526bfd7a5ce6ef460bebee0399fe669612d* by *xi_ge:*

   [FixCode] Apply coercion fixits on return statement and initialization
   - *edit*: test/FixCode/fixits-apply-objc.swift
      - *edit*: test/FixCode/fixits-apply-objc.swift.result
      - *edit*: lib/Sema/CSDiag.cpp
      - *edit*: test/FixCode/fixits-apply.swift.result
   -
   - Commit *6447a2d16edcfbceab940d58a321b3ef70525f1a* by *daniel_dunbar:*

   [Basic] Update Thread shim to match changes on Linux side.
   - *edit*: Sources/Basic/Thread.swift

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

Thanks,
*- Enrico*
:envelope_with_arrow: egranata@ <about:blank> <about:blank>.com <about:blank> :phone:️ 27683

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev


(Enrico Granata) #14

First off, I think the whole LLDB team can agree on the simple principle that tests that sometimes fail is not something we’re shooting for, so we’re on the same page with the general sentiment.

But, rather than talking in sweeping generalizations, let’s take this exact failure as an example. It was a socket attempting to connect. It uses general socket timeouts. If it’s using a default, it will typically be something like 30 seconds. Per my previous comment, this is not something we see fail with any kind of frequency. We just had a failure on it now. Is the solution to say we want 35 seconds? 1 minute? 2 minutes? We’re using end-to-end testing here, and therefore we are subject to how a real system behaves. If we crank that up, there could be a time where we overload the server further, and the socket facility just fails. What do you do then, claim the test as unworthy?

The challenge we run into on LLDB is that, while we do have a growing number of unit tests that are strict input/output verification, far too much of a functioning debugger’s operations are not covered adequately by that. So we have a large body of end to end tests. Sometimes those fail under heavy load. Sometimes those issues are ours - we have hidden races that are only exposed when the machine is grinding to a halt. Sometimes it is that the OS fails to handle requests for resources that the debugger needs.

So it isn’t as simple as saying “file a bug to make the LLDB test suite more reliable.” (I wish it were!)

This is specifically why I proposed a concrete strategy like running a stress test of the test suite. That is actionable, and should shake some bugs I would hope?

Another actionable strategies (although one I'm not a big fan of) is enhance the test suite to automatically re-attempt flaky tests.

The test suite already does this.

But only for tests that are explicitly marked as flakey - which this test wasn't because - as Todd points out - it is not one of the usual known culprits

Jim

On the positive side, we have more resources going into improving our quality in the upcoming months, and we now have a dedicated quality engineer working with us. I expect this will help things as we move forward. And we’re looking to sink some of the system testing that currently exists end-to-end in LLDB to a more input-output style testing that we can do without needing the whole of LLDB (e.g. DWARF verification, DWARF type round-tripping through the compiler, etc. - Greg and Sean can speak more to what we’re doing there).

Great!

- Daniel

The net effect is that we should be hitting less of this noise.

-Todd

The amount of flakiness in the LLDB tests is disturbing... is there a high-level bug tracking improving this? It seems like it might be worth running a stress test of the tests on a loaded machine to try and shake out such problems.

- Daniel

It looks like this may be the packaging build:

oss-swift-package-linux-ubuntu-15_10

It was this build:

https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7

It has since gone blue.

I don’t think I recall ever seeing that test fail before in an intermittent way, but that might be what happened here.

Given the failure mode, it does sound likely:

File "/usr/lib/python2.7/socket.py", line 206, in accept sock, addr
= self._sock.accept() timeout: timed out

If we see that again, we may be able to tweak the socket timeout to allow for heavily loaded test CI.

-Todd

Where is this failure being reported? I’m not seeing anything on swift-3.0-branch or master on ci.swift.org.

Kate Stone k8stone@apple.com
 Xcode Low Level Tools

+ swift-lldb-dev group.

Failing in lldb:

Command invoked:
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py
--executable
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb
--rerun-all-issues -C
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
-s 2016-08-18-15_15_07 --results-port 41187 --inferior -p
TestStubReverseConnect.py
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test
--event-add-entries worker_index=14:int Configuration: arch=x86_64
compiler=/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
----------------------------------------------------------------------
Collected 2 tests

ERROR: test_reverse_connect_works_llgs
(TestStubReverseConnect.TestStubReverseConnect)
----------------------------------------------------------------------
Traceback (most recent call last): File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py",
line 121, in wrapper func(*args, **kwargs) File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py",
line 121, in wrapper func(*args, **kwargs) File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py",
line 89, in test_reverse_connect_works_llgs
self.reverse_connect_works() File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py",
line 67, in reverse_connect_works (stub_socket, address) =
self.listener_socket.accept() File "/usr/lib/python2.7/socket.py",
line 206, in accept sock, addr = self._sock.accept() timeout: timed
out
Config=x86_64-/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang-3.9
----------------------------------------------------------------------
Ran 2 tests in 22.006s RESULT: FAILED (0 passes, 0 failures, 1
errors, 1 skipped, 0 expected failures, 0 unexpected successes)
Session logs for test failures/errors/unexpected successes can be
found in directory '2016-08-18-15_15_07'

[TestStubReverseConnect.py
FAILED]

Command invoked: /usr/bin/python
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py
--executable
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb
--rerun-all-issues -C
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
-s 2016-08-18-15_15_07 --results-port 41187 --inferior -p
TestStubReverseConnect.py
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test
--event-add-entries worker_index=14:int

- Daniel

[FAILURE] oss-swift-package-linux-ubuntu-15_10 [#1930]

Build URL: https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/
Project: oss-swift-package-linux-ubuntu-15_10
Date of build: Thu, 18 Aug 2016 20:26:52 -0700
Build duration: 51 min
Identified problems:

  • Regression test failed: This build failed because a regression test in the test suite FAILed. Below is a list of all errors:
    • Indication 1

Changes

  • Commit 2ebab526bfd7a5ce6ef460bebee0399fe669612d by xi_ge:
[FixCode] Apply
coercion fixits on return statement and initialization

    • edit: test/FixCode/fixits-apply-objc.swift
    • edit: test/FixCode/fixits-apply-objc.swift.result
    • edit: lib/Sema/CSDiag.cpp
    • edit: test/FixCode/fixits-apply.swift.result

  • Commit 6447a2d16edcfbceab940d58a321b3ef70525f1a by daniel_dunbar:
[Basic] Update
Thread shim to match changes on Linux side.

    • edit: Sources/Basic/Thread.swift

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

Thanks,
- Enrico
:envelope_with_arrow: egranata@.com :phone:️ 27683

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

Thanks,
- Enrico
:envelope_with_arrow: egranata@.com :phone:️ 27683

···

On Aug 19, 2016, at 2:44 PM, Jim Ingham via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:

On Aug 19, 2016, at 2:26 PM, Daniel Dunbar via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:

On Aug 19, 2016, at 2:22 PM, Todd Fiala <todd.fiala@gmail.com> wrote:
On August 19, 2016 at 2:10:14 PM, Daniel Dunbar via swift-lldb-dev (swift-lldb-dev@swift.org) wrote:

On Aug 19, 2016, at 1:58 PM, Todd Fiala via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:

On Aug 19, 2016, at 1:42 PM, Enrico Granata <egranata@apple.com> wrote:

On Aug 19, 2016, at 11:20 AM, Todd Fiala via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:

On Aug 19, 2016, at 11:15 AM, Kate Stone via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:

On Aug 19, 2016, at 12:15 AM, mishal_shah via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:

On Aug 18, 2016, at 9:47 PM, Daniel Dunbar <daniel_dunbar@apple.com> wrote:

On Aug 18, 2016, at 9:18 PM, no-reply@swift.org wrote:


(Todd Fiala) #15

-Todd

First off, I think the whole LLDB team can agree on the simple principle that tests that sometimes fail is not something we’re shooting for, so we’re on the same page with the general sentiment.

But, rather than talking in sweeping generalizations, let’s take this exact failure as an example. It was a socket attempting to connect. It uses general socket timeouts. If it’s using a default, it will typically be something like 30 seconds. Per my previous comment, this is not something we see fail with any kind of frequency. We just had a failure on it now. Is the solution to say we want 35 seconds? 1 minute? 2 minutes? We’re using end-to-end testing here, and therefore we are subject to how a real system behaves. If we crank that up, there could be a time where we overload the server further, and the socket facility just fails. What do you do then, claim the test as unworthy?

The challenge we run into on LLDB is that, while we do have a growing number of unit tests that are strict input/output verification, far too much of a functioning debugger’s operations are not covered adequately by that. So we have a large body of end to end tests. Sometimes those fail under heavy load. Sometimes those issues are ours - we have hidden races that are only exposed when the machine is grinding to a halt. Sometimes it is that the OS fails to handle requests for resources that the debugger needs.

So it isn’t as simple as saying “file a bug to make the LLDB test suite more reliable.” (I wish it were!)

This is specifically why I proposed a concrete strategy like running a stress test of the test suite. That is actionable, and should shake some bugs I would hope?

The stress test idea seems like a good one to me. I think that would be a great opportunity for our quality engineer, who is also looking at other aspects like code coverage.

The other thing we're going to do that should help here (stress test or not) is, on the macOS side, running sample on the process that times out, and archiving that data with the build. That way we can home in on issues that are originating in LLDB, and potentially giving insight into issues locking up elsewhere.

···

On Aug 19, 2016, at 3:09 PM, Todd Fiala <todd.fiala@gmail.com> wrote:

On August 19, 2016 at 2:26:18 PM, Daniel Dunbar (daniel_dunbar@apple.com) wrote:

On Aug 19, 2016, at 2:22 PM, Todd Fiala <todd.fiala@gmail.com> wrote:

-Todd

Another actionable strategies (although one I'm not a big fan of) is enhance the test suite to automatically re-attempt flaky tests.

On the positive side, we have more resources going into improving our quality in the upcoming months, and we now have a dedicated quality engineer working with us. I expect this will help things as we move forward. And we’re looking to sink some of the system testing that currently exists end-to-end in LLDB to a more input-output style testing that we can do without needing the whole of LLDB (e.g. DWARF verification, DWARF type round-tripping through the compiler, etc. - Greg and Sean can speak more to what we’re doing there).

Great!

- Daniel

The net effect is that we should be hitting less of this noise.

-Todd

On August 19, 2016 at 2:10:14 PM, Daniel Dunbar via swift-lldb-dev (swift-lldb-dev@swift.org) wrote:

The amount of flakiness in the LLDB tests is disturbing... is there a high-level bug tracking improving this? It seems like it might be worth running a stress test of the tests on a loaded machine to try and shake out such problems.

- Daniel

On Aug 19, 2016, at 1:58 PM, Todd Fiala via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:

On Aug 19, 2016, at 1:42 PM, Enrico Granata <egranata@apple.com> wrote:

On Aug 19, 2016, at 11:20 AM, Todd Fiala via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:

It looks like this may be the packaging build:

oss-swift-package-linux-ubuntu-15_10

It was this build:

https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7

It has since gone blue.

I don’t think I recall ever seeing that test fail before in an intermittent way, but that might be what happened here.

Given the failure mode, it does sound likely:

File "/usr/lib/python2.7/socket.py", line 206, in accept sock, addr
= self._sock.accept() timeout: timed out

If we see that again, we may be able to tweak the socket timeout to allow for heavily loaded test CI.

-Todd

On Aug 19, 2016, at 11:15 AM, Kate Stone via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:

Where is this failure being reported? I’m not seeing anything on swift-3.0-branch or master on ci.swift.org.

Kate Stone k8stone@apple.com
 Xcode Low Level Tools

On Aug 19, 2016, at 12:15 AM, mishal_shah via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:

+ swift-lldb-dev group.

On Aug 18, 2016, at 9:47 PM, Daniel Dunbar <daniel_dunbar@apple.com> wrote:

Failing in lldb:

Command invoked:
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py
--executable
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb
--rerun-all-issues -C
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
-s 2016-08-18-15_15_07 --results-port 41187 --inferior -p
TestStubReverseConnect.py
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test
--event-add-entries worker_index=14:int Configuration: arch=x86_64
compiler=/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
----------------------------------------------------------------------
Collected 2 tests

ERROR: test_reverse_connect_works_llgs
(TestStubReverseConnect.TestStubReverseConnect)
----------------------------------------------------------------------
Traceback (most recent call last): File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py",
line 121, in wrapper func(*args, **kwargs) File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py",
line 121, in wrapper func(*args, **kwargs) File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py",
line 89, in test_reverse_connect_works_llgs
self.reverse_connect_works() File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py",
line 67, in reverse_connect_works (stub_socket, address) =
self.listener_socket.accept() File "/usr/lib/python2.7/socket.py",
line 206, in accept sock, addr = self._sock.accept() timeout: timed
out
Config=x86_64-/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang-3.9
----------------------------------------------------------------------
Ran 2 tests in 22.006s RESULT: FAILED (0 passes, 0 failures, 1
errors, 1 skipped, 0 expected failures, 0 unexpected successes)
Session logs for test failures/errors/unexpected successes can be
found in directory '2016-08-18-15_15_07'
[TestStubReverseConnect.py
FAILED] Command
invoked: /usr/bin/python
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py
--executable
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb
--rerun-all-issues -C
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
-s 2016-08-18-15_15_07 --results-port 41187 --inferior -p
TestStubReverseConnect.py
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test
--event-add-entries worker_index=14:int

- Daniel

On Aug 18, 2016, at 9:18 PM, no-reply@swift.org wrote:

[FAILURE] oss-swift-package-linux-ubuntu-15_10 [#1930]

Build URL: https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/
Project: oss-swift-package-linux-ubuntu-15_10
Date of build: Thu, 18 Aug 2016 20:26:52 -0700
Build duration: 51 min
Identified problems:

Regression test failed: This build failed because a regression test in the test suite FAILed. Below is a list of all errors:
Indication 1

Changes

Commit 2ebab526bfd7a5ce6ef460bebee0399fe669612d by xi_ge:
[FixCode] Apply
coercion fixits on return statement and initialization

edit: test/FixCode/fixits-apply-objc.swift
edit: test/FixCode/fixits-apply-objc.swift.result
edit: lib/Sema/CSDiag.cpp
edit: test/FixCode/fixits-apply.swift.result

Commit 6447a2d16edcfbceab940d58a321b3ef70525f1a by daniel_dunbar:
[Basic] Update
Thread shim to match changes on Linux side.

edit: Sources/Basic/Thread.swift

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

Thanks,
- Enrico
:envelope_with_arrow: egranata@.com :phone:️ 27683

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev


(Todd Fiala) #16

First off, I think the whole LLDB team can agree on the simple principle that tests that sometimes fail is not something we’re shooting for, so we’re on the same page with the general sentiment.

But, rather than talking in sweeping generalizations, let’s take this exact failure as an example. It was a socket attempting to connect. It uses general socket timeouts. If it’s using a default, it will typically be something like 30 seconds. Per my previous comment, this is not something we see fail with any kind of frequency. We just had a failure on it now. Is the solution to say we want 35 seconds? 1 minute? 2 minutes? We’re using end-to-end testing here, and therefore we are subject to how a real system behaves. If we crank that up, there could be a time where we overload the server further, and the socket facility just fails. What do you do then, claim the test as unworthy?

The challenge we run into on LLDB is that, while we do have a growing number of unit tests that are strict input/output verification, far too much of a functioning debugger’s operations are not covered adequately by that. So we have a large body of end to end tests. Sometimes those fail under heavy load. Sometimes those issues are ours - we have hidden races that are only exposed when the machine is grinding to a halt. Sometimes it is that the OS fails to handle requests for resources that the debugger needs.

So it isn’t as simple as saying “file a bug to make the LLDB test suite more reliable.” (I wish it were!)

This is specifically why I proposed a concrete strategy like running a stress test of the test suite. That is actionable, and should shake some bugs I would hope?

Another actionable strategies (although one I'm not a big fan of) is enhance the test suite to automatically re-attempt flaky tests.

The test suite already does this.

But only for tests that are explicitly marked as flakey - which this test wasn't because - as Todd points out - it is not one of the usual known culprits

I’m running the tests with —rerun-all-issues (per the swift build script). That actually will rerun all issues, not just ones marked flaky. Most of the upstream LLVM.org <http://llvm.org/> buidbots run in the mode where only flaky tests are rerun, but we’re not there yet across all our platforms on the GitHub side.

···

On Aug 19, 2016, at 3:02 PM, Enrico Granata via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:

On Aug 19, 2016, at 2:44 PM, Jim Ingham via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

On Aug 19, 2016, at 2:26 PM, Daniel Dunbar via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

On Aug 19, 2016, at 2:22 PM, Todd Fiala <todd.fiala@gmail.com <mailto:todd.fiala@gmail.com>> wrote:

Jim

On the positive side, we have more resources going into improving our quality in the upcoming months, and we now have a dedicated quality engineer working with us. I expect this will help things as we move forward. And we’re looking to sink some of the system testing that currently exists end-to-end in LLDB to a more input-output style testing that we can do without needing the whole of LLDB (e.g. DWARF verification, DWARF type round-tripping through the compiler, etc. - Greg and Sean can speak more to what we’re doing there).

Great!

- Daniel

The net effect is that we should be hitting less of this noise.

-Todd
On August 19, 2016 at 2:10:14 PM, Daniel Dunbar via swift-lldb-dev (swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>) wrote:

The amount of flakiness in the LLDB tests is disturbing... is there a high-level bug tracking improving this? It seems like it might be worth running a stress test of the tests on a loaded machine to try and shake out such problems.

- Daniel

On Aug 19, 2016, at 1:58 PM, Todd Fiala via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

On Aug 19, 2016, at 1:42 PM, Enrico Granata <egranata@apple.com <mailto:egranata@apple.com>> wrote:

On Aug 19, 2016, at 11:20 AM, Todd Fiala via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

It looks like this may be the packaging build:

oss-swift-package-linux-ubuntu-15_10

It was this build:

https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7

It has since gone blue.

I don’t think I recall ever seeing that test fail before in an intermittent way, but that might be what happened here.

Given the failure mode, it does sound likely:

File "/usr/lib/python2.7/socket.py", line 206, in accept sock, addr
= self._sock.accept() timeout: timed out

If we see that again, we may be able to tweak the socket timeout to allow for heavily loaded test CI.

-Todd

On Aug 19, 2016, at 11:15 AM, Kate Stone via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

Where is this failure being reported? I’m not seeing anything on swift-3.0-branch or master on ci.swift.org <http://ci.swift.org/>.

Kate Stone k8stone@apple.com <mailto:k8stone@apple.com>
 Xcode Low Level Tools

On Aug 19, 2016, at 12:15 AM, mishal_shah via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

+ swift-lldb-dev group.

On Aug 18, 2016, at 9:47 PM, Daniel Dunbar <daniel_dunbar@apple.com <mailto:daniel_dunbar@apple.com>> wrote:

Failing in lldb:

Command invoked:
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py
--executable
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb
--rerun-all-issues -C
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
-s 2016-08-18-15_15_07 --results-port 41187 --inferior -p
TestStubReverseConnect.py
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test
--event-add-entries worker_index=14:int Configuration: arch=x86_64
compiler=/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
----------------------------------------------------------------------
Collected 2 tests

ERROR: test_reverse_connect_works_llgs
(TestStubReverseConnect.TestStubReverseConnect)
----------------------------------------------------------------------
Traceback (most recent call last): File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py",
line 121, in wrapper func(*args, **kwargs) File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py",
line 121, in wrapper func(*args, **kwargs) File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py",
line 89, in test_reverse_connect_works_llgs
self.reverse_connect_works() File
"/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py",
line 67, in reverse_connect_works (stub_socket, address) =
self.listener_socket.accept() File "/usr/lib/python2.7/socket.py",
line 206, in accept sock, addr = self._sock.accept() timeout: timed
out
Config=x86_64-/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang-3.9
----------------------------------------------------------------------
Ran 2 tests in 22.006s RESULT: FAILED (0 passes, 0 failures, 1
errors, 1 skipped, 0 expected failures, 0 unexpected successes)
Session logs for test failures/errors/unexpected successes can be
found in directory '2016-08-18-15_15_07'

[TestStubReverseConnect.py
FAILED]

Command invoked: /usr/bin/python
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py
--executable
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb
--rerun-all-issues -C
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
-s 2016-08-18-15_15_07 --results-port 41187 --inferior -p
TestStubReverseConnect.py
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test
--event-add-entries worker_index=14:int

- Daniel

On Aug 18, 2016, at 9:18 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:

[FAILURE] oss-swift-package-linux-ubuntu-15_10 [#1930]

Build URL: https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/
Project: oss-swift-package-linux-ubuntu-15_10
Date of build: Thu, 18 Aug 2016 20:26:52 -0700
Build duration: 51 min
Identified problems:

  • Regression test failed: This build failed because a regression test in the test suite FAILed. Below is a list of all errors:
    • Indication 1

Changes

  • Commit 2ebab526bfd7a5ce6ef460bebee0399fe669612d by xi_ge:
[FixCode] Apply
coercion fixits on return statement and initialization

    • edit: test/FixCode/fixits-apply-objc.swift
    • edit: test/FixCode/fixits-apply-objc.swift.result
    • edit: lib/Sema/CSDiag.cpp
    • edit: test/FixCode/fixits-apply.swift.result

  • Commit 6447a2d16edcfbceab940d58a321b3ef70525f1a by daniel_dunbar:
[Basic] Update
Thread shim to match changes on Linux side.

    • edit: Sources/Basic/Thread.swift

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

Thanks,
- Enrico
:envelope_with_arrow: egranata@.com <about:blank> :phone:️ 27683

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

Thanks,
- Enrico
:envelope_with_arrow: egranata@.com :phone:️ 27683

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev