XCTest Assertion failure in +[XCTRunnerDaemonSession sharedSession]

Hello!

I'm running a pretty straightforward integration test target with swift test --enable-code-coverage, but the test runner seems to crash and exit sporadically during the test cases. It mentions the codecov folder not being found. Is there something I should be looking out for here?

2019-09-16 15:52:48.571 xctest[24286:5117115] *** Assertion failure in +[XCTRunnerDaemonSession sharedSession], /Library/Caches/com.apple.xbs/Sources/XCTest/XCTest-14806/Sources/XCTestFramework/Harness/XCTRunnerDaemonSession.m:103
2019-09-16 15:52:48.576 xctest[24286:5117115] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Failed to determine capabilities of daemon connection: Interrupted by waiter.'
*** First throw call stack:
(
        0   CoreFoundation                      0x00007fff31f75cf9 __exceptionPreprocess + 256
        1   libobjc.A.dylib                     0x00007fff5cb0aa17 objc_exception_throw + 48
        2   CoreFoundation                      0x00007fff31f90a16 +[NSException raise:format:arguments:] + 98
        3   Foundation                          0x00007fff34222e11 -[NSAssertionHandler handleFailureInMethod:object:file:lineNumber:description:] + 194
        4   XCTest                              0x000000010f5dd774 __39+[XCTRunnerDaemonSession sharedSession]_block_invoke + 400
        5   libdispatch.dylib                   0x00007fff5e2eb63d _dispatch_client_callout + 8
        6   libdispatch.dylib                   0x00007fff5e2ecd4c _dispatch_once_callout + 20
        7   XCTest                              0x000000010f5dd5e2 +[XCTRunnerDaemonSession sharedSession] + 101
        8   XCTest                              0x000000010f5a224d -[XCUIDevice initLocalDeviceWithPlatform:] + 587
        9   XCTest                              0x000000010f5a1fed __25+[XCUIDevice localDevice]_block_invoke + 37
        10  libdispatch.dylib                   0x00007fff5e2eb63d _dispatch_client_callout + 8
        11  libdispatch.dylib                   0x00007fff5e2ecd4c _dispatch_once_callout + 20
        12  XCTest                              0x000000010f5a1fc5 +[XCUIDevice localDevice] + 45
        13  XCTest                              0x000000010f5d485d __51-[XCTestCase(Failures) _recordActivityWithFailure:]_block_invoke + 166
        14  XCTest                              0x000000010f5e50e4 -[XCTContext _runActivityNamed:type:block:] + 304
        15  XCTest                              0x000000010f56b0ee -[XCTestCase startActivityWithTitle:type:block:] + 218
        16  XCTest                              0x000000010f5d46dc -[XCTestCase(Failures) _recordActivityWithFailure:] + 594
        17  XCTest                              0x000000010f5d3eed -[XCTestCase(Failures) _recordFailureWithDescription:inFile:atLine:expected:] + 371
        18  XCTest                              0x000000010f55fced -[XCTestCase recordFailureWithDescription:inFile:atLine:expected:] + 66
        19  XCTest                              0x000000010f5d3547 -[XCTestCase(Failures) _dequeueFailures] + 1128
        20  XCTest                              0x000000010f561a8c __26-[XCTestCase performTest:]_block_invoke.327 + 363
        21  CoreFoundation                      0x00007fff31f1ce88 __CFRUNLOOP_IS_CALLING_OUT_TO_AN_OBSERVER_CALLBACK_FUNCTION__ + 23
        22  CoreFoundation                      0x00007fff31f1cdbd __CFRunLoopDoObservers + 451
        23  CoreFoundation                      0x00007fff31ebee49 CFRunLoopRunSpecific + 514
        24  XCTest                              0x000000010f5cf185 -[XCTWaiter waitForExpectations:timeout:enforceOrder:] + 989
        25  XCTest                              0x000000010f54bd63 -[XCTestCase(AsynchronousTesting) waitForExpectations:timeout:enforceOrder:] + 103
        26  AWRESTKitPackageTests               0x000000011263d829 $s25AWRESTKitIntegrationTests012AWEntryPointC0C14testBuildVisityyF + 553
        27  AWRESTKitPackageTests               0x000000011263daeb $s25AWRESTKitIntegrationTests012AWEntryPointC0C14testBuildVisityyFTo + 43
        28  CoreFoundation                      0x00007fff31ecf02c __invoking___ + 140
        29  CoreFoundation                      0x00007fff31eceeff -[NSInvocation invoke] + 311
        30  XCTest                              0x000000010f560976 __24-[XCTestCase invokeTest]_block_invoke_2.214 + 65
        31  XCTest                              0x000000010f5ee09b -[XCTMemoryChecker _assertInvalidObjectsDeallocatedAfterScope:] + 51
        32  XCTest                              0x000000010f56b85e -[XCTestCase assertInvalidObjectsDeallocatedAfterScope:] + 116
        33  XCTest                              0x000000010f560906 __24-[XCTestCase invokeTest]_block_invoke.208 + 204
        34  XCTest                              0x000000010f5d49d8 -[XCTestCase(Failures) performFailableBlock:testCaseRun:shouldInterruptTest:] + 57
        35  XCTest                              0x000000010f5d48f5 -[XCTestCase(Failures) _performTurningExceptionsIntoFailuresInterruptAfterHandling:block:] + 96
        36  XCTest                              0x000000010f5603b7 __24-[XCTestCase invokeTest]_block_invoke + 1120
        37  XCTest                              0x000000010f55feb0 -[XCTestCase testContextPerformInScope:] + 184
        38  XCTest                              0x000000010f55ff4a -[XCTestCase invokeTest] + 137
        39  XCTest                              0x000000010f561d29 __26-[XCTestCase performTest:]_block_invoke_2 + 43
        40  XCTest                              0x000000010f5d49d8 -[XCTestCase(Failures) performFailableBlock:testCaseRun:shouldInterruptTest:] + 57
        41  XCTest                              0x000000010f5d48f5 -[XCTestCase(Failures) _performTurningExceptionsIntoFailuresInterruptAfterHandling:block:] + 96
        42  XCTest                              0x000000010f561c40 __26-[XCTestCase performTest:]_block_invoke.342 + 88
        43  XCTest                              0x000000010f5e576e +[XCTContext runInContextForTestCase:block:] + 219
        44  XCTest                              0x000000010f5613b3 -[XCTestCase performTest:] + 668
        45  XCTest                              0x000000010f5b3573 -[XCTest runTest] + 57
        46  XCTest                              0x000000010f55ba53 __27-[XCTestSuite performTest:]_block_invoke + 365
        47  XCTest                              0x000000010f55b17d -[XCTestSuite _performProtectedSectionForTest:testSection:] + 54
        48  XCTest                              0x000000010f55b47a -[XCTestSuite performTest:] + 355
        49  XCTest                              0x000000010f5b3573 -[XCTest runTest] + 57
        50  XCTest                              0x000000010f55ba53 __27-[XCTestSuite performTest:]_block_invoke + 365
        51  XCTest                              0x000000010f55b17d -[XCTestSuite _performProtectedSectionForTest:testSection:] + 54
        52  XCTest                              0x000000010f55b47a -[XCTestSuite performTest:] + 355
        53  XCTest                              0x000000010f5b3573 -[XCTest runTest] + 57
        54  XCTest                              0x000000010f55ba53 __27-[XCTestSuite performTest:]_block_invoke + 365
        55  XCTest                              0x000000010f55b17d -[XCTestSuite _performProtectedSectionForTest:testSection:] + 54
        56  XCTest                              0x000000010f55b47a -[XCTestSuite performTest:] + 355
        57  XCTest                              0x000000010f5b3573 -[XCTest runTest] + 57
        58  XCTest                              0x000000010f5ffc6a __44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke + 171
        59  XCTest                              0x000000010f5ffd66 __44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke.84 + 118
        60  XCTest                              0x000000010f582349 -[XCTestObservationCenter _observeTestExecutionForBlock:] + 594
        61  XCTest                              0x000000010f5ffa28 -[XCTTestRunSession runTestsAndReturnError:] + 623
        62  XCTest                              0x000000010f53e6f0 -[XCTestDriver runTestsAndReturnError:] + 458
        63  XCTest                              0x000000010f5e1ab6 _XCTestMain + 2473
        64  xctest                              0x000000010f4880ca main + 245
        65  libdyld.dylib                       0x00007fff5e3383d5 start + 1
)
libc++abi.dylib: terminating with uncaught exception of type NSException
error: Error Domain=NSCocoaErrorDomain Code=260 "The folder “codecov” doesn’t exist." UserInfo={NSFilePath=/Users/dev.user/bamboo-agent-home/xml-data/build-dir/CB-IR-JOB1/ios_restkit/.build/x86_64-apple-macosx/debug/codecov, NSUserStringVariant=(
    Folder
), NSUnderlyingError=0x7fa64dcc0030 {Error Domain=NSOSStatusErrorDomain Code=-43 "fnfErr: File not found"}}

Huh, this might a bug in SwiftPM. Do you mind filing a JIRA with a sample package that reproduces the issue?

I'd love to!

I'll see if I can repro this in a sample package. If not, can this be tracked in a Radar?

Huh... I also see this without the --enable-code-coverage flag:

Ana:ios_restkit stephenciauri$ swift test --filter AWRESTKitIntegrationTests.AWEntryPointTests/testBuildVisit
[3/3] Linking AWRESTKitPackageTests
Test Suite 'Selected tests' started at 2019-09-16 17:48:34.438
Test Suite 'AWRESTKitPackageTests.xctest' started at 2019-09-16 17:48:34.438
Test Suite 'AWEntryPointTests' started at 2019-09-16 17:48:34.438
Test Case '-[AWRESTKitIntegrationTests.AWEntryPointTests testBuildVisit]' started.
/Users/stephenciauri/Developer/ios_restkit/Tests/AWRESTKitIntegrationTests/AWEntryPointEntityTests.swift:215: error: -[AWRESTKitIntegrationTests.AWEntryPointTests testBuildVisit] : failed - The provider has been marked as offline
2019-09-16 17:48:40.929 xctest[2050:125456] *** Assertion failure in +[XCTRunnerDaemonSession sharedSession], /Library/Caches/com.apple.xbs/Sources/XCTest/XCTest-14794.1/Sources/XCTestFramework/Harness/XCTRunnerDaemonSession.m:103
2019-09-16 17:48:40.932 xctest[2050:125456] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Failed to determine capabilities of daemon connection: Interrupted by waiter.'
*** First throw call stack:
(
	0   CoreFoundation                      0x00007fff326792fd __exceptionPreprocess + 256
	1   libobjc.A.dylib                     0x00007fff5cd5ba17 objc_exception_throw + 48
	2   CoreFoundation                      0x00007fff32694016 +[NSException raise:format:arguments:] + 98
	3   Foundation                          0x00007fff34933791 -[NSAssertionHandler handleFailureInMethod:object:file:lineNumber:description:] + 194
	4   XCTest                              0x0000000100db9765 __39+[XCTRunnerDaemonSession sharedSession]_block_invoke + 400
	5   libdispatch.dylib                   0x00007fff5e4dc63d _dispatch_client_callout + 8
	6   libdispatch.dylib                   0x00007fff5e4ddd4b _dispatch_once_callout + 20
	7   XCTest                              0x0000000100db95d3 +[XCTRunnerDaemonSession sharedSession] + 101
	8   XCTest                              0x0000000100d7ef15 -[XCUIDevice initLocalDeviceWithPlatform:] + 587
	9   XCTest                              0x0000000100d7ecb5 __25+[XCUIDevice localDevice]_block_invoke + 37
	10  libdispatch.dylib                   0x00007fff5e4dc63d _dispatch_client_callout + 8
	11  libdispatch.dylib                   0x00007fff5e4ddd4b _dispatch_once_callout + 20
	12  XCTest                              0x0000000100d7ec8d +[XCUIDevice localDevice] + 45
	13  XCTest                              0x0000000100db11c1 __51-[XCTestCase(Failures) _recordActivityWithFailure:]_block_invoke + 166
	14  XCTest                              0x0000000100dc1008 -[XCTContext _runActivityNamed:type:block:] + 304
	15  XCTest                              0x0000000100d47f96 -[XCTestCase startActivityWithTitle:type:block:] + 218
	16  XCTest                              0x0000000100db1040 -[XCTestCase(Failures) _recordActivityWithFailure:] + 594
	17  XCTest                              0x0000000100db0851 -[XCTestCase(Failures) _recordFailureWithDescription:inFile:atLine:expected:] + 371
	18  XCTest                              0x0000000100d3cb95 -[XCTestCase recordFailureWithDescription:inFile:atLine:expected:] + 66
	19  XCTest                              0x0000000100dafeab -[XCTestCase(Failures) _dequeueFailures] + 1128
	20  XCTest                              0x0000000100d3e934 __26-[XCTestCase performTest:]_block_invoke.327 + 363
	21  CoreFoundation                      0x00007fff3261ff28 __CFRUNLOOP_IS_CALLING_OUT_TO_AN_OBSERVER_CALLBACK_FUNCTION__ + 23
	22  CoreFoundation                      0x00007fff3261fe5d __CFRunLoopDoObservers + 451
	23  CoreFoundation                      0x00007fff325c1ef9 CFRunLoopRunSpecific + 514
	24  XCTest                              0x0000000100dabae9 -[XCTWaiter waitForExpectations:timeout:enforceOrder:] + 989
	25  XCTest                              0x0000000100d28c33 -[XCTestCase(AsynchronousTesting) waitForExpectations:timeout:enforceOrder:] + 103
	26  AWRESTKitPackageTests               0x000000010264f188 $s25AWRESTKitIntegrationTests012AWEntryPointC0C14testBuildVisityyF + 536
	27  AWRESTKitPackageTests               0x000000010264f40b $s25AWRESTKitIntegrationTests012AWEntryPointC0C14testBuildVisityyFTo + 43
	28  CoreFoundation                      0x00007fff325d20cc __invoking___ + 140
	29  CoreFoundation                      0x00007fff325d1f97 -[NSInvocation invoke] + 311
	30  XCTest                              0x0000000100d3d81e __24-[XCTestCase invokeTest]_block_invoke_2.214 + 65
	31  XCTest                              0x0000000100dc9fbf -[XCTMemoryChecker _assertInvalidObjectsDeallocatedAfterScope:] + 51
	32  XCTest                              0x0000000100d48706 -[XCTestCase assertInvalidObjectsDeallocatedAfterScope:] + 116
	33  XCTest                              0x0000000100d3d7ae __24-[XCTestCase invokeTest]_block_invoke.208 + 204
	34  XCTest                              0x0000000100db133c -[XCTestCase(Failures) performFailableBlock:testCaseRun:shouldInterruptTest:] + 57
	35  XCTest                              0x0000000100db1259 -[XCTestCase(Failures) _performTurningExceptionsIntoFailuresInterruptAfterHandling:block:] + 96
	36  XCTest                              0x0000000100d3d25f __24-[XCTestCase invokeTest]_block_invoke + 1120
	37  XCTest                              0x0000000100d3cd58 -[XCTestCase testContextPerformInScope:] + 184
	38  XCTest                              0x0000000100d3cdf2 -[XCTestCase invokeTest] + 137
	39  XCTest                              0x0000000100d3ebd1 __26-[XCTestCase performTest:]_block_invoke_2 + 43
	40  XCTest                              0x0000000100db133c -[XCTestCase(Failures) performFailableBlock:testCaseRun:shouldInterruptTest:] + 57
	41  XCTest                              0x0000000100db1259 -[XCTestCase(Failures) _performTurningExceptionsIntoFailuresInterruptAfterHandling:block:] + 96
	42  XCTest                              0x0000000100d3eae8 __26-[XCTestCase performTest:]_block_invoke.342 + 88
	43  XCTest                              0x0000000100dc1692 +[XCTContext runInContextForTestCase:block:] + 219
	44  XCTest                              0x0000000100d3e25b -[XCTestCase performTest:] + 668
	45  XCTest                              0x0000000100d90041 -[XCTest runTest] + 57
	46  XCTest                              0x0000000100d388fb __27-[XCTestSuite performTest:]_block_invoke + 365
	47  XCTest                              0x0000000100d38025 -[XCTestSuite _performProtectedSectionForTest:testSection:] + 54
	48  XCTest                              0x0000000100d38322 -[XCTestSuite performTest:] + 355
	49  XCTest                              0x0000000100d90041 -[XCTest runTest] + 57
	50  XCTest                              0x0000000100d388fb __27-[XCTestSuite performTest:]_block_invoke + 365
	51  XCTest                              0x0000000100d38025 -[XCTestSuite _performProtectedSectionForTest:testSection:] + 54
	52  XCTest                              0x0000000100d38322 -[XCTestSuite performTest:] + 355
	53  XCTest                              0x0000000100d90041 -[XCTest runTest] + 57
	54  XCTest                              0x0000000100d388fb __27-[XCTestSuite performTest:]_block_invoke + 365
	55  XCTest                              0x0000000100d38025 -[XCTestSuite _performProtectedSectionForTest:testSection:] + 54
	56  XCTest                              0x0000000100d38322 -[XCTestSuite performTest:] + 355
	57  XCTest                              0x0000000100d90041 -[XCTest runTest] + 57
	58  XCTest                              0x0000000100ddb8fd __44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke + 171
	59  XCTest                              0x0000000100ddb9f9 __44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke.84 + 118
	60  XCTest                              0x0000000100d5f115 -[XCTestObservationCenter _observeTestExecutionForBlock:] + 594
	61  XCTest                              0x0000000100ddb6bb -[XCTTestRunSession runTestsAndReturnError:] + 623
	62  XCTest                              0x0000000100d1b650 -[XCTestDriver runTestsAndReturnError:] + 458
	63  XCTest                              0x0000000100dbd9da _XCTestMain + 2473
	64  xctest                              0x0000000100c670fa main + 245
	65  libdyld.dylib                       0x00007fff5e5293d5 start + 1
	66  ???                                 0x0000000000000004 0x0 + 4
)
libc++abi.dylib: terminating with uncaught exception of type NSException
Exited with signal code 6

Yes, sure!

I think there's a bug with incremental building with code coverage so you might want to try running swift package clean when you change the code coverage flag.

Radar submitted! FB7286373. Hopefully we can sort this out. I had a hard time reproducing it in a sterile project, so I must be doing something wacky in the sample project attached to the radar.