Dynamically call XCTFail in SPM module without importing XCTest?

Using Swift
, ,
1

TL;DR: Is it possible to invoke XCTFail dynamically without importing XCTest?

We're developing an SPM package that ships with a test support module that has a helper that invokes XCTFail. Unfortunately, it turns out that SPM does not support the use case where a MyLibraryTestSupport library product depends on a MyLibrary library product, because downstream apps that import the library into their app and the test support library in tests will break due to duplicate symbols.

We've tried mitigating this issue by marking both packages .dynamic, but this prevents downstream packages from reliably depending on our package (Bug with explicitly defining LibraryType as 'dynamic' ¡ Issue #60 ¡ pointfreeco/swift-composable-architecture ¡ GitHub).

A few other bugs we've discovered:

We could move our test support library to its own package/repository, but this seems like a non-starter, as it makes package maintenance and consumption more burdensome.

What we'd like to do instead is merge the libraries, but the test helper's call to import XCTest immediately causes problems. So we're wondering if it's somehow possible to get a dynamic handle on XCTFail so that our test helper can be invoked on test runs but not cause any problems for applications that link to our library.

Does anyone know if this is possible or have another solution given the above?

We're also tracking this issue here: Merge the core library with the test support library ¡ Issue #70 ¡ pointfreeco/swift-composable-architecture ¡ GitHub

3 Likes
2

For those interested, this magic appears to work:

typealias XCTCurrentTestCase = @convention(c) () -> AnyObject
typealias XCTFailureHandler
  = @convention(c) (AnyObject, Bool, UnsafePointer<CChar>, UInt, String, String?) -> Void

func _XCTFail(_ message: String = "", file: StaticString = #file, line: UInt = #line) {
  guard
    let _XCTest = NSClassFromString("XCTest")
      .flatMap(Bundle.init(for:))
      .flatMap({ $0.executablePath })
      .flatMap({ dlopen($0, RTLD_NOW) })
    else { return }

  guard
    let _XCTFailureHandler = dlsym(_XCTest, "_XCTFailureHandler")
      .map({ unsafeBitCast($0, to: XCTFailureHandler.self) })
    else { return }

  guard
    let _XCTCurrentTestCase = dlsym(_XCTest, "_XCTCurrentTestCase")
      .map({ unsafeBitCast($0, to: XCTCurrentTestCase.self) })
    else { return }

  _XCTFailureHandler(_XCTCurrentTestCase(), true, "\(file)", line, message, nil)
}
3 Likes
3

I don’t think the problem would have occurred in the first place if none of the products were dynamic. I’ve certainly never encountered issues importing this XCTest‐dependent module from all over the place.

If the main library needs to be dynamic, then having a second library that’s also dynamic and shares any modules would cause duplicate symbols. The current inability to split out the shared symbols into their own dynamic library without having a separate package seems to be the real root of the issue. To do that, we would need the ability to depend on products from the same package.

It happens to be what I asked about on my very first post on these forums a long time ago, but nothing has ever come of it.

It also reappeared here:

4

This is indeed some black magic on display ^^ Nicely done but can't this be a reason for apps rejection by Apple ? :fearful: In theory they just reject apps that give random strings to these dlopen/dlsym methods but it seems they have done it even for apps that used fixed strings (https://github.com/nicklockwood/GZIP/issues/24)

1 Like
5

I'm not sure if I'm missing something, but I've updated the libraries to be static and still need the workaround. If I try to import XCTest in the library then any app that depends on it fails to compile with the following log:

ld: warning: Could not find or use auto-linked library 'XCTestSwiftSupport'
ld: warning: Could not find or use auto-linked framework 'XCTest'
Undefined symbols for architecture arm64:
"XCTest.XCTFail(_: Swift.String, file: Swift.StaticString, line: Swift.UInt) -> ()", referenced from: …

6

Great point! We'll hide this logic away in a #if DEBUG check.

2 Likes
7

The problem would still exist for app-hosted tests. The app and the test bundle would contain the same symbols and the test bundle will be loaded in the app's address space when using TEST_HOST.

8

Ah, yes, I see.

9

Hi @stephencelis,

I have managed this issue in a ugly way but it’s working for xcode preview and testing.

I have forked the project and split it into two libraries with different names one dynamic linking for test(that is the one available on point free but renamed) and one static linking for the main project(without the test utilities).

It’s working pretty well but it’s ugly, cumbersome to maintain and might definitively not be what you want to support.

Have you investigated using Cocoapods 1.9 and Configuration-based dependencies(CocoaPods 1.9 Beta has arrived! - CocoaPods Blog) ?

10

Have you tried latest master? We merged in the workaround above and it seems to be working nicely.

We really want to support SPM out of the box, but I believe both CocoaPods and Carthage by default avoid the particular issue we were seeing.

11

Thanks for supporting SPM out of the box because that's a requirement for us to use it in our current project :)

1 Like