Build test bundle w/o running, or run it w/ custom test runners

Development Package Manager
,
1

I have a SwiftPM test target that I want to build, but to run it in a special way (with a WebAssembly runtime). Unfortunately, swift test always attempts to run the tests after building.
To elaborate, I have .testTarget(name: "Tests") in my Package.swift . What swift test does is that it creates a test bundle for the whole package. It then attempts to run it (with either xctest runner on macOS or directly on Linux) and fails because the WebAssembly bundle can only be run with a WebAssembly runtime. I've tried to build tests separately without running, but swift build --target Tests doesn't build the test bundle for me.

I don't think there is a clean workaround to get the test bundle built without a failing swift test invocation. I currently see multiple potential solutions:

  1. Adding a new --skip-test-run flag to swift test , which will produce a test bundle, but won't run it, so a user later can do whatever they want with the bundle (feed it to a WASI runtime in our case).
  2. Adding a new --test-runner flag to swift test , which takes a runtime command as an argument. This one is more intrusive and complicated, and the test runner itself could also take arguments (e.g. if it's a browser runtime, we need to pass some additional arguments to launch a browser properly).
  3. A separate flag for swift build that builds a package test bundle without running it. Just another variation of option 1, but shifts the end-user interface from swift test to swift build.

Overall I'm inclined towards option 1, and hoping to prepare a PR if there are no objections, but maybe I'm missing anything? Any thoughts or suggestions? Thanks!

2

You can build tests without running them using swift build --build-tests.

1 Like
3

This is great, thanks! Sorry I missed that, but now trying it out, it looks like it doesn't work for cross-compilation. swift build --destination destination.json works great, swift build —-build-tests separately works fine too, but using both of them at the same time gives me this, is it expected?

% swift build —-build-tests --destination destination.json
error: unexpected argument —-build-tests; use --help to list available arguments

EDIT: ok, the long dash copied directly from the snippet is the culprit, it should be --build-tests, not —build-tests.

1 Like
4

Oops, corrected my post.

5

So it did end up working?

If not, it isn’t cross‐compilation that is the problem, since I use --build-tests with Android all the time.

6

Yes, it did work. I'm still only able to build test targets targeting WASI by explicitly adding a forked XCTest to Package.swift as a dependency of a test target. That's because we don't build XCTest as a part of the toolchain build process, caused by the use Foundation stubs instead of the real Foundation. Also can't copy forked XCTest build artifacts out from SwiftPM's .build directory and ship it in the toolchain as if it was a proper XCTest build, because SwiftPM doesn't produce static library archives (libXCTest.a) for WASI (yet), but that's a whole different story and I'm diggressing :slightly_smiling_face: