[Swift CI] Build Still Failing: 0. OSS - LLDB Incremental - Ubuntu 14.04 (master) #1915


(Ben Langmuir) #1

This is the same old failure we’ve been seeing for a while. I filed a radar to improve the “new issue” detection so it won’t send all these false positives for lldb tests:
<rdar://problem/24961012> Don't report "new issue" when test failures happen in different orders

Ben

···

On Mar 3, 2016, at 1:20 PM, no-reply@swift.org wrote:

https://ci.swift.org/job/oss-lldb-incremental-linux-ubuntu-14_04/1915/


(Todd Fiala) #2

These are real failures. They are due to a Swift change that is emitting invalid symbols. Lang/Roman are working on a fix. I didn’t mark these as failures because they are working on the fix.

If we wanted to limit noise, the right thing to do here would have been to revert that change until they could work out the bad symbol names reported by the JIT. (I was okay with not insisting on that because they were going to quickly track this down).

-Todd

···

On Mar 3, 2016, at 1:31 PM, Ben Langmuir via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:

This is the same old failure we’ve been seeing for a while. I filed a radar to improve the “new issue” detection so it won’t send all these false positives for lldb tests:
<rdar://problem/24961012 <rdar://problem/24961012>> Don't report "new issue" when test failures happen in different orders

Ben

On Mar 3, 2016, at 1:20 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:

https://ci.swift.org/job/oss-lldb-incremental-linux-ubuntu-14_04/1915/

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev


(Ben Langmuir) #3

My point is they aren’t “new issues” in each commit, so we shouldn’t get emailed. We have a script that checks if the issue is new, but that script doesn’t take into account that the test order may be inconsistent.

Ben

···

On Mar 3, 2016, at 2:32 PM, Todd Fiala <tfiala@apple.com> wrote:

These are real failures. They are due to a Swift change that is emitting invalid symbols. Lang/Roman are working on a fix. I didn’t mark these as failures because they are working on the fix.

If we wanted to limit noise, the right thing to do here would have been to revert that change until they could work out the bad symbol names reported by the JIT. (I was okay with not insisting on that because they were going to quickly track this down).

-Todd

On Mar 3, 2016, at 1:31 PM, Ben Langmuir via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

This is the same old failure we’ve been seeing for a while. I filed a radar to improve the “new issue” detection so it won’t send all these false positives for lldb tests:
<rdar://problem/24961012 <rdar://problem/24961012>> Don't report "new issue" when test failures happen in different orders

Ben

On Mar 3, 2016, at 1:20 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:

https://ci.swift.org/job/oss-lldb-incremental-linux-ubuntu-14_04/1915/

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev


(Todd Fiala) #4

These are real failures. They are due to a Swift change that is emitting invalid symbols. Lang/Roman are working on a fix. I didn’t mark these as failures because they are working on the fix.

<rdar://problem/24934249> Swift JIT mode strips leading _ in symbols on Linux

···

On Mar 3, 2016, at 2:32 PM, Todd Fiala via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:

If we wanted to limit noise, the right thing to do here would have been to revert that change until they could work out the bad symbol names reported by the JIT. (I was okay with not insisting on that because they were going to quickly track this down).

-Todd

On Mar 3, 2016, at 1:31 PM, Ben Langmuir via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

This is the same old failure we’ve been seeing for a while. I filed a radar to improve the “new issue” detection so it won’t send all these false positives for lldb tests:
<rdar://problem/24961012 <rdar://problem/24961012>> Don't report "new issue" when test failures happen in different orders

Ben

On Mar 3, 2016, at 1:20 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:

https://ci.swift.org/job/oss-lldb-incremental-linux-ubuntu-14_04/1915/

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev


(Todd Fiala) #5

Ah I see the distinction you are making.

Thanks for clarifying. (Mishal, that would be great to distinguish here).

-Todd

···

On Mar 3, 2016, at 2:33 PM, Ben Langmuir <blangmuir@apple.com> wrote:

My point is they aren’t “new issues” in each commit, so we shouldn’t get emailed. We have a script that checks if the issue is new, but that script doesn’t take into account that the test order may be inconsistent.

Ben

On Mar 3, 2016, at 2:32 PM, Todd Fiala <tfiala@apple.com <mailto:tfiala@apple.com>> wrote:

These are real failures. They are due to a Swift change that is emitting invalid symbols. Lang/Roman are working on a fix. I didn’t mark these as failures because they are working on the fix.

If we wanted to limit noise, the right thing to do here would have been to revert that change until they could work out the bad symbol names reported by the JIT. (I was okay with not insisting on that because they were going to quickly track this down).

-Todd

On Mar 3, 2016, at 1:31 PM, Ben Langmuir via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

This is the same old failure we’ve been seeing for a while. I filed a radar to improve the “new issue” detection so it won’t send all these false positives for lldb tests:
<rdar://problem/24961012 <rdar://problem/24961012>> Don't report "new issue" when test failures happen in different orders

Ben

On Mar 3, 2016, at 1:20 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:

https://ci.swift.org/job/oss-lldb-incremental-linux-ubuntu-14_04/1915/

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev


(Mishal Shah) #6

Ben and I had a chat about this, its due to how lldb test suite prints to console. I will need to update the “New issue” script to handle this better.

Thanks,
Mishal Shah

···

On Mar 3, 2016, at 2:34 PM, Todd Fiala <tfiala@apple.com> wrote:

Ah I see the distinction you are making.

Thanks for clarifying. (Mishal, that would be great to distinguish here).

-Todd

On Mar 3, 2016, at 2:33 PM, Ben Langmuir <blangmuir@apple.com <mailto:blangmuir@apple.com>> wrote:

My point is they aren’t “new issues” in each commit, so we shouldn’t get emailed. We have a script that checks if the issue is new, but that script doesn’t take into account that the test order may be inconsistent.

Ben

On Mar 3, 2016, at 2:32 PM, Todd Fiala <tfiala@apple.com <mailto:tfiala@apple.com>> wrote:

These are real failures. They are due to a Swift change that is emitting invalid symbols. Lang/Roman are working on a fix. I didn’t mark these as failures because they are working on the fix.

If we wanted to limit noise, the right thing to do here would have been to revert that change until they could work out the bad symbol names reported by the JIT. (I was okay with not insisting on that because they were going to quickly track this down).

-Todd

On Mar 3, 2016, at 1:31 PM, Ben Langmuir via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

This is the same old failure we’ve been seeing for a while. I filed a radar to improve the “new issue” detection so it won’t send all these false positives for lldb tests:
<rdar://problem/24961012 <rdar://problem/24961012>> Don't report "new issue" when test failures happen in different orders

Ben

On Mar 3, 2016, at 1:20 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:

https://ci.swift.org/job/oss-lldb-incremental-linux-ubuntu-14_04/1915/

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev


(Todd Fiala) #7

Ben and I had a chat about this, its due to how lldb test suite prints to console. I will need to update the “New issue” script to handle this better.

Ideally it is just pulling info from the xUnit results and working off of that?

···

On Mar 3, 2016, at 2:37 PM, Mishal Shah <mishal_shah@apple.com> wrote:

Thanks,
Mishal Shah

On Mar 3, 2016, at 2:34 PM, Todd Fiala <tfiala@apple.com <mailto:tfiala@apple.com>> wrote:

Ah I see the distinction you are making.

Thanks for clarifying. (Mishal, that would be great to distinguish here).

-Todd

On Mar 3, 2016, at 2:33 PM, Ben Langmuir <blangmuir@apple.com <mailto:blangmuir@apple.com>> wrote:

My point is they aren’t “new issues” in each commit, so we shouldn’t get emailed. We have a script that checks if the issue is new, but that script doesn’t take into account that the test order may be inconsistent.

Ben

On Mar 3, 2016, at 2:32 PM, Todd Fiala <tfiala@apple.com <mailto:tfiala@apple.com>> wrote:

These are real failures. They are due to a Swift change that is emitting invalid symbols. Lang/Roman are working on a fix. I didn’t mark these as failures because they are working on the fix.

If we wanted to limit noise, the right thing to do here would have been to revert that change until they could work out the bad symbol names reported by the JIT. (I was okay with not insisting on that because they were going to quickly track this down).

-Todd

On Mar 3, 2016, at 1:31 PM, Ben Langmuir via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

This is the same old failure we’ve been seeing for a while. I filed a radar to improve the “new issue” detection so it won’t send all these false positives for lldb tests:
<rdar://problem/24961012 <rdar://problem/24961012>> Don't report "new issue" when test failures happen in different orders

Ben

On Mar 3, 2016, at 1:20 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:

https://ci.swift.org/job/oss-lldb-incremental-linux-ubuntu-14_04/1915/

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev


(Todd Fiala) #8

Ben and I had a chat about this, its due to how lldb test suite prints to console. I will need to update the “New issue” script to handle this better.

Ideally it is just pulling info from the xUnit results and working off of that?

(Like, can’t the Jenkins xUnit plug-in already do this?)

···

On Mar 3, 2016, at 2:41 PM, Todd Fiala via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:

On Mar 3, 2016, at 2:37 PM, Mishal Shah <mishal_shah@apple.com <mailto:mishal_shah@apple.com>> wrote:

Thanks,
Mishal Shah

On Mar 3, 2016, at 2:34 PM, Todd Fiala <tfiala@apple.com <mailto:tfiala@apple.com>> wrote:

Ah I see the distinction you are making.

Thanks for clarifying. (Mishal, that would be great to distinguish here).

-Todd

On Mar 3, 2016, at 2:33 PM, Ben Langmuir <blangmuir@apple.com <mailto:blangmuir@apple.com>> wrote:

My point is they aren’t “new issues” in each commit, so we shouldn’t get emailed. We have a script that checks if the issue is new, but that script doesn’t take into account that the test order may be inconsistent.

Ben

On Mar 3, 2016, at 2:32 PM, Todd Fiala <tfiala@apple.com <mailto:tfiala@apple.com>> wrote:

These are real failures. They are due to a Swift change that is emitting invalid symbols. Lang/Roman are working on a fix. I didn’t mark these as failures because they are working on the fix.

If we wanted to limit noise, the right thing to do here would have been to revert that change until they could work out the bad symbol names reported by the JIT. (I was okay with not insisting on that because they were going to quickly track this down).

-Todd

On Mar 3, 2016, at 1:31 PM, Ben Langmuir via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

This is the same old failure we’ve been seeing for a while. I filed a radar to improve the “new issue” detection so it won’t send all these false positives for lldb tests:
<rdar://problem/24961012 <rdar://problem/24961012>> Don't report "new issue" when test failures happen in different orders

Ben

On Mar 3, 2016, at 1:20 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:

https://ci.swift.org/job/oss-lldb-incremental-linux-ubuntu-14_04/1915/

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev


(Mishal Shah) #9

“New issue” script looks at Identified problems and compares them with last build, xUnit plug-in can only handle test failures.

···

On Mar 3, 2016, at 2:42 PM, Todd Fiala <tfiala@apple.com> wrote:

On Mar 3, 2016, at 2:41 PM, Todd Fiala via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

On Mar 3, 2016, at 2:37 PM, Mishal Shah <mishal_shah@apple.com <mailto:mishal_shah@apple.com>> wrote:

Ben and I had a chat about this, its due to how lldb test suite prints to console. I will need to update the “New issue” script to handle this better.

Ideally it is just pulling info from the xUnit results and working off of that?

(Like, can’t the Jenkins xUnit plug-in already do this?)

Thanks,
Mishal Shah

On Mar 3, 2016, at 2:34 PM, Todd Fiala <tfiala@apple.com <mailto:tfiala@apple.com>> wrote:

Ah I see the distinction you are making.

Thanks for clarifying. (Mishal, that would be great to distinguish here).

-Todd

On Mar 3, 2016, at 2:33 PM, Ben Langmuir <blangmuir@apple.com <mailto:blangmuir@apple.com>> wrote:

My point is they aren’t “new issues” in each commit, so we shouldn’t get emailed. We have a script that checks if the issue is new, but that script doesn’t take into account that the test order may be inconsistent.

Ben

On Mar 3, 2016, at 2:32 PM, Todd Fiala <tfiala@apple.com <mailto:tfiala@apple.com>> wrote:

These are real failures. They are due to a Swift change that is emitting invalid symbols. Lang/Roman are working on a fix. I didn’t mark these as failures because they are working on the fix.

If we wanted to limit noise, the right thing to do here would have been to revert that change until they could work out the bad symbol names reported by the JIT. (I was okay with not insisting on that because they were going to quickly track this down).

-Todd

On Mar 3, 2016, at 1:31 PM, Ben Langmuir via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

This is the same old failure we’ve been seeing for a while. I filed a radar to improve the “new issue” detection so it won’t send all these false positives for lldb tests:
<rdar://problem/24961012 <rdar://problem/24961012>> Don't report "new issue" when test failures happen in different orders

Ben

On Mar 3, 2016, at 1:20 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:

https://ci.swift.org/job/oss-lldb-incremental-linux-ubuntu-14_04/1915/

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev


(Todd Fiala) #10

“New issue” script looks at Identified problems and compares them with last build, xUnit plug-in can only handle test failures.

Ah I see. We need to cover both build issues and test issues. Makes sense. Drats, was hoping for an “already implemented” solution…

-Todd

···

On Mar 3, 2016, at 2:48 PM, Mishal Shah via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:

On Mar 3, 2016, at 2:42 PM, Todd Fiala <tfiala@apple.com <mailto:tfiala@apple.com>> wrote:

On Mar 3, 2016, at 2:41 PM, Todd Fiala via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

On Mar 3, 2016, at 2:37 PM, Mishal Shah <mishal_shah@apple.com <mailto:mishal_shah@apple.com>> wrote:

Ben and I had a chat about this, its due to how lldb test suite prints to console. I will need to update the “New issue” script to handle this better.

Ideally it is just pulling info from the xUnit results and working off of that?

(Like, can’t the Jenkins xUnit plug-in already do this?)

Thanks,
Mishal Shah

On Mar 3, 2016, at 2:34 PM, Todd Fiala <tfiala@apple.com <mailto:tfiala@apple.com>> wrote:

Ah I see the distinction you are making.

Thanks for clarifying. (Mishal, that would be great to distinguish here).

-Todd

On Mar 3, 2016, at 2:33 PM, Ben Langmuir <blangmuir@apple.com <mailto:blangmuir@apple.com>> wrote:

My point is they aren’t “new issues” in each commit, so we shouldn’t get emailed. We have a script that checks if the issue is new, but that script doesn’t take into account that the test order may be inconsistent.

Ben

On Mar 3, 2016, at 2:32 PM, Todd Fiala <tfiala@apple.com <mailto:tfiala@apple.com>> wrote:

These are real failures. They are due to a Swift change that is emitting invalid symbols. Lang/Roman are working on a fix. I didn’t mark these as failures because they are working on the fix.

If we wanted to limit noise, the right thing to do here would have been to revert that change until they could work out the bad symbol names reported by the JIT. (I was okay with not insisting on that because they were going to quickly track this down).

-Todd

On Mar 3, 2016, at 1:31 PM, Ben Langmuir via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:

This is the same old failure we’ve been seeing for a while. I filed a radar to improve the “new issue” detection so it won’t send all these false positives for lldb tests:
<rdar://problem/24961012 <rdar://problem/24961012>> Don't report "new issue" when test failures happen in different orders

Ben

On Mar 3, 2016, at 1:20 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:

https://ci.swift.org/job/oss-lldb-incremental-linux-ubuntu-14_04/1915/

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev

_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev