This is the same old failure we’ve been seeing for a while. I filed a radar to improve the “new issue” detection so it won’t send all these false positives for lldb tests:
<rdar://problem/24961012> Don't report "new issue" when test failures happen in different orders
Ben
On Mar 3, 2016, at 1:20 PM, no-reply@swift.org wrote:
https://ci.swift.org/job/oss-lldb-incremental-linux-ubuntu-14_04/1915/
These are real failures. They are due to a Swift change that is emitting invalid symbols. Lang/Roman are working on a fix. I didn’t mark these as failures because they are working on the fix.
If we wanted to limit noise, the right thing to do here would have been to revert that change until they could work out the bad symbol names reported by the JIT. (I was okay with not insisting on that because they were going to quickly track this down).
-Todd
On Mar 3, 2016, at 1:31 PM, Ben Langmuir via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:
This is the same old failure we’ve been seeing for a while. I filed a radar to improve the “new issue” detection so it won’t send all these false positives for lldb tests:
<rdar://problem/24961012 <rdar://problem/24961012>> Don't report "new issue" when test failures happen in different ordersBen
On Mar 3, 2016, at 1:20 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:
https://ci.swift.org/job/oss-lldb-incremental-linux-ubuntu-14_04/1915/
_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev
My point is they aren’t “new issues” in each commit, so we shouldn’t get emailed. We have a script that checks if the issue is new, but that script doesn’t take into account that the test order may be inconsistent.
Ben
On Mar 3, 2016, at 2:32 PM, Todd Fiala <tfiala@apple.com> wrote:
These are real failures. They are due to a Swift change that is emitting invalid symbols. Lang/Roman are working on a fix. I didn’t mark these as failures because they are working on the fix.
If we wanted to limit noise, the right thing to do here would have been to revert that change until they could work out the bad symbol names reported by the JIT. (I was okay with not insisting on that because they were going to quickly track this down).
-Todd
On Mar 3, 2016, at 1:31 PM, Ben Langmuir via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:
This is the same old failure we’ve been seeing for a while. I filed a radar to improve the “new issue” detection so it won’t send all these false positives for lldb tests:
<rdar://problem/24961012 <rdar://problem/24961012>> Don't report "new issue" when test failures happen in different ordersBen
On Mar 3, 2016, at 1:20 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:
https://ci.swift.org/job/oss-lldb-incremental-linux-ubuntu-14_04/1915/
_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev
These are real failures. They are due to a Swift change that is emitting invalid symbols. Lang/Roman are working on a fix. I didn’t mark these as failures because they are working on the fix.
<rdar://problem/24934249> Swift JIT mode strips leading _ in symbols on Linux
On Mar 3, 2016, at 2:32 PM, Todd Fiala via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:
If we wanted to limit noise, the right thing to do here would have been to revert that change until they could work out the bad symbol names reported by the JIT. (I was okay with not insisting on that because they were going to quickly track this down).
-Todd
On Mar 3, 2016, at 1:31 PM, Ben Langmuir via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:
This is the same old failure we’ve been seeing for a while. I filed a radar to improve the “new issue” detection so it won’t send all these false positives for lldb tests:
<rdar://problem/24961012 <rdar://problem/24961012>> Don't report "new issue" when test failures happen in different ordersBen
On Mar 3, 2016, at 1:20 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:
https://ci.swift.org/job/oss-lldb-incremental-linux-ubuntu-14_04/1915/
_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev
Ah I see the distinction you are making.
Thanks for clarifying. (Mishal, that would be great to distinguish here).
-Todd
On Mar 3, 2016, at 2:33 PM, Ben Langmuir <blangmuir@apple.com> wrote:
My point is they aren’t “new issues” in each commit, so we shouldn’t get emailed. We have a script that checks if the issue is new, but that script doesn’t take into account that the test order may be inconsistent.
Ben
On Mar 3, 2016, at 2:32 PM, Todd Fiala <tfiala@apple.com <mailto:tfiala@apple.com>> wrote:
These are real failures. They are due to a Swift change that is emitting invalid symbols. Lang/Roman are working on a fix. I didn’t mark these as failures because they are working on the fix.
If we wanted to limit noise, the right thing to do here would have been to revert that change until they could work out the bad symbol names reported by the JIT. (I was okay with not insisting on that because they were going to quickly track this down).
-Todd
On Mar 3, 2016, at 1:31 PM, Ben Langmuir via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:
This is the same old failure we’ve been seeing for a while. I filed a radar to improve the “new issue” detection so it won’t send all these false positives for lldb tests:
<rdar://problem/24961012 <rdar://problem/24961012>> Don't report "new issue" when test failures happen in different ordersBen
On Mar 3, 2016, at 1:20 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:
https://ci.swift.org/job/oss-lldb-incremental-linux-ubuntu-14_04/1915/
_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev
Ben and I had a chat about this, its due to how lldb test suite prints to console. I will need to update the “New issue” script to handle this better.
Thanks,
Mishal Shah
On Mar 3, 2016, at 2:34 PM, Todd Fiala <tfiala@apple.com> wrote:
Ah I see the distinction you are making.
Thanks for clarifying. (Mishal, that would be great to distinguish here).
-Todd
On Mar 3, 2016, at 2:33 PM, Ben Langmuir <blangmuir@apple.com <mailto:blangmuir@apple.com>> wrote:
My point is they aren’t “new issues” in each commit, so we shouldn’t get emailed. We have a script that checks if the issue is new, but that script doesn’t take into account that the test order may be inconsistent.
Ben
On Mar 3, 2016, at 2:32 PM, Todd Fiala <tfiala@apple.com <mailto:tfiala@apple.com>> wrote:
These are real failures. They are due to a Swift change that is emitting invalid symbols. Lang/Roman are working on a fix. I didn’t mark these as failures because they are working on the fix.
If we wanted to limit noise, the right thing to do here would have been to revert that change until they could work out the bad symbol names reported by the JIT. (I was okay with not insisting on that because they were going to quickly track this down).
-Todd
On Mar 3, 2016, at 1:31 PM, Ben Langmuir via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:
This is the same old failure we’ve been seeing for a while. I filed a radar to improve the “new issue” detection so it won’t send all these false positives for lldb tests:
<rdar://problem/24961012 <rdar://problem/24961012>> Don't report "new issue" when test failures happen in different ordersBen
On Mar 3, 2016, at 1:20 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:
https://ci.swift.org/job/oss-lldb-incremental-linux-ubuntu-14_04/1915/
_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev
Ben and I had a chat about this, its due to how lldb test suite prints to console. I will need to update the “New issue” script to handle this better.
Ideally it is just pulling info from the xUnit results and working off of that?
On Mar 3, 2016, at 2:37 PM, Mishal Shah <mishal_shah@apple.com> wrote:
Thanks,
Mishal ShahOn Mar 3, 2016, at 2:34 PM, Todd Fiala <tfiala@apple.com <mailto:tfiala@apple.com>> wrote:
Ah I see the distinction you are making.
Thanks for clarifying. (Mishal, that would be great to distinguish here).
-Todd
On Mar 3, 2016, at 2:33 PM, Ben Langmuir <blangmuir@apple.com <mailto:blangmuir@apple.com>> wrote:
My point is they aren’t “new issues” in each commit, so we shouldn’t get emailed. We have a script that checks if the issue is new, but that script doesn’t take into account that the test order may be inconsistent.
Ben
On Mar 3, 2016, at 2:32 PM, Todd Fiala <tfiala@apple.com <mailto:tfiala@apple.com>> wrote:
These are real failures. They are due to a Swift change that is emitting invalid symbols. Lang/Roman are working on a fix. I didn’t mark these as failures because they are working on the fix.
If we wanted to limit noise, the right thing to do here would have been to revert that change until they could work out the bad symbol names reported by the JIT. (I was okay with not insisting on that because they were going to quickly track this down).
-Todd
On Mar 3, 2016, at 1:31 PM, Ben Langmuir via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:
This is the same old failure we’ve been seeing for a while. I filed a radar to improve the “new issue” detection so it won’t send all these false positives for lldb tests:
<rdar://problem/24961012 <rdar://problem/24961012>> Don't report "new issue" when test failures happen in different ordersBen
On Mar 3, 2016, at 1:20 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:
https://ci.swift.org/job/oss-lldb-incremental-linux-ubuntu-14_04/1915/
_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev
Ben and I had a chat about this, its due to how lldb test suite prints to console. I will need to update the “New issue” script to handle this better.
Ideally it is just pulling info from the xUnit results and working off of that?
(Like, can’t the Jenkins xUnit plug-in already do this?)
On Mar 3, 2016, at 2:41 PM, Todd Fiala via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:
On Mar 3, 2016, at 2:37 PM, Mishal Shah <mishal_shah@apple.com <mailto:mishal_shah@apple.com>> wrote:
Thanks,
Mishal ShahOn Mar 3, 2016, at 2:34 PM, Todd Fiala <tfiala@apple.com <mailto:tfiala@apple.com>> wrote:
Ah I see the distinction you are making.
Thanks for clarifying. (Mishal, that would be great to distinguish here).
-Todd
On Mar 3, 2016, at 2:33 PM, Ben Langmuir <blangmuir@apple.com <mailto:blangmuir@apple.com>> wrote:
My point is they aren’t “new issues” in each commit, so we shouldn’t get emailed. We have a script that checks if the issue is new, but that script doesn’t take into account that the test order may be inconsistent.
Ben
On Mar 3, 2016, at 2:32 PM, Todd Fiala <tfiala@apple.com <mailto:tfiala@apple.com>> wrote:
These are real failures. They are due to a Swift change that is emitting invalid symbols. Lang/Roman are working on a fix. I didn’t mark these as failures because they are working on the fix.
If we wanted to limit noise, the right thing to do here would have been to revert that change until they could work out the bad symbol names reported by the JIT. (I was okay with not insisting on that because they were going to quickly track this down).
-Todd
On Mar 3, 2016, at 1:31 PM, Ben Langmuir via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:
This is the same old failure we’ve been seeing for a while. I filed a radar to improve the “new issue” detection so it won’t send all these false positives for lldb tests:
<rdar://problem/24961012 <rdar://problem/24961012>> Don't report "new issue" when test failures happen in different ordersBen
On Mar 3, 2016, at 1:20 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:
https://ci.swift.org/job/oss-lldb-incremental-linux-ubuntu-14_04/1915/
_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev
“New issue” script looks at Identified problems and compares them with last build, xUnit plug-in can only handle test failures.
On Mar 3, 2016, at 2:42 PM, Todd Fiala <tfiala@apple.com> wrote:
On Mar 3, 2016, at 2:41 PM, Todd Fiala via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:
On Mar 3, 2016, at 2:37 PM, Mishal Shah <mishal_shah@apple.com <mailto:mishal_shah@apple.com>> wrote:
Ben and I had a chat about this, its due to how lldb test suite prints to console. I will need to update the “New issue” script to handle this better.
Ideally it is just pulling info from the xUnit results and working off of that?
(Like, can’t the Jenkins xUnit plug-in already do this?)
Thanks,
Mishal ShahOn Mar 3, 2016, at 2:34 PM, Todd Fiala <tfiala@apple.com <mailto:tfiala@apple.com>> wrote:
Ah I see the distinction you are making.
Thanks for clarifying. (Mishal, that would be great to distinguish here).
-Todd
On Mar 3, 2016, at 2:33 PM, Ben Langmuir <blangmuir@apple.com <mailto:blangmuir@apple.com>> wrote:
My point is they aren’t “new issues” in each commit, so we shouldn’t get emailed. We have a script that checks if the issue is new, but that script doesn’t take into account that the test order may be inconsistent.
Ben
On Mar 3, 2016, at 2:32 PM, Todd Fiala <tfiala@apple.com <mailto:tfiala@apple.com>> wrote:
These are real failures. They are due to a Swift change that is emitting invalid symbols. Lang/Roman are working on a fix. I didn’t mark these as failures because they are working on the fix.
If we wanted to limit noise, the right thing to do here would have been to revert that change until they could work out the bad symbol names reported by the JIT. (I was okay with not insisting on that because they were going to quickly track this down).
-Todd
On Mar 3, 2016, at 1:31 PM, Ben Langmuir via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:
This is the same old failure we’ve been seeing for a while. I filed a radar to improve the “new issue” detection so it won’t send all these false positives for lldb tests:
<rdar://problem/24961012 <rdar://problem/24961012>> Don't report "new issue" when test failures happen in different ordersBen
On Mar 3, 2016, at 1:20 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:
https://ci.swift.org/job/oss-lldb-incremental-linux-ubuntu-14_04/1915/
_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev
“New issue” script looks at Identified problems and compares them with last build, xUnit plug-in can only handle test failures.
Ah I see. We need to cover both build issues and test issues. Makes sense. Drats, was hoping for an “already implemented” solution…
-Todd
On Mar 3, 2016, at 2:48 PM, Mishal Shah via swift-lldb-dev <swift-lldb-dev@swift.org> wrote:
On Mar 3, 2016, at 2:42 PM, Todd Fiala <tfiala@apple.com <mailto:tfiala@apple.com>> wrote:
On Mar 3, 2016, at 2:41 PM, Todd Fiala via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:
On Mar 3, 2016, at 2:37 PM, Mishal Shah <mishal_shah@apple.com <mailto:mishal_shah@apple.com>> wrote:
Ben and I had a chat about this, its due to how lldb test suite prints to console. I will need to update the “New issue” script to handle this better.
Ideally it is just pulling info from the xUnit results and working off of that?
(Like, can’t the Jenkins xUnit plug-in already do this?)
Thanks,
Mishal ShahOn Mar 3, 2016, at 2:34 PM, Todd Fiala <tfiala@apple.com <mailto:tfiala@apple.com>> wrote:
Ah I see the distinction you are making.
Thanks for clarifying. (Mishal, that would be great to distinguish here).
-Todd
On Mar 3, 2016, at 2:33 PM, Ben Langmuir <blangmuir@apple.com <mailto:blangmuir@apple.com>> wrote:
My point is they aren’t “new issues” in each commit, so we shouldn’t get emailed. We have a script that checks if the issue is new, but that script doesn’t take into account that the test order may be inconsistent.
Ben
On Mar 3, 2016, at 2:32 PM, Todd Fiala <tfiala@apple.com <mailto:tfiala@apple.com>> wrote:
These are real failures. They are due to a Swift change that is emitting invalid symbols. Lang/Roman are working on a fix. I didn’t mark these as failures because they are working on the fix.
If we wanted to limit noise, the right thing to do here would have been to revert that change until they could work out the bad symbol names reported by the JIT. (I was okay with not insisting on that because they were going to quickly track this down).
-Todd
On Mar 3, 2016, at 1:31 PM, Ben Langmuir via swift-lldb-dev <swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>> wrote:
This is the same old failure we’ve been seeing for a while. I filed a radar to improve the “new issue” detection so it won’t send all these false positives for lldb tests:
<rdar://problem/24961012 <rdar://problem/24961012>> Don't report "new issue" when test failures happen in different ordersBen
On Mar 3, 2016, at 1:20 PM, no-reply@swift.org <mailto:no-reply@swift.org> wrote:
https://ci.swift.org/job/oss-lldb-incremental-linux-ubuntu-14_04/1915/
_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org <mailto:swift-lldb-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-lldb-dev_______________________________________________
swift-lldb-dev mailing list
swift-lldb-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-lldb-dev