`swift test` tries to build all targets instead of just those needed for testing

Using Swift
,
1

We are trying to go all in on Swift Package Manager, reorganising our dependency trees etc. around it. It’s going well, big thank you to everyone who has contributed to SwiftPM’s success!

We have a small, but fairly important (for us) hiccup with this: swift test tries to build all targets, instead of just those needed to build and run the tests.

The reason this is important to us: we have a Swift Package containing a core library (that we want to test), and then numerous other dynamic library and executable targets that use the core library, but often on esoteric platforms like postgresql, as a python plugin, etc.

To build the esoteric Products we need to provide specific build invocations with multiple flags, header search paths, etc. We can’t put those flags into Package.swift specific to those targets, because they require unsafe flags, which from my understanding would make our package unconsumable from other packages, including our own.

Basically we’re stuck with a workaround, and even it doesn’t really work reliably:

swift build --product OurCoreLibrary
swift test --skip-build

The workaround also doesn’t play well with Xcode, VSCode, or other “IDE-like” environments that want to be able to just run swift test.

At the end of the day, what I think would solve the entire issue (and significantly improve build speeds in some cases) would be for SwiftPM to treat our testTarget as a separate product, and only build it and its dependencies.

That said, I think all of the above actually also applies to swift build --product XYZ – instead of just building the targets required for XYZ, it tries to build everything. At least, this was my experience many times in the past; maybe this particular issue has been fixed in the meantime.

Is this a known issue? Can we expect that this is on someone’s radar already, or is this something I should try to tackle myself and try to bring a PR to SwiftPM?

2

Please file an issue on the SwiftPM repository first, if one doesn't exist yet.

1 Like
3

swift test does a build by default to ensure the test target is built. Providing --skip-build should skip building the test target.

4

Funny you say that, I was just playing around with that option. It’s actually worse than I thought:

% swift test --build-system=swiftbuild
Building for debugging...
[97/100] Math 139 / 142
[100/100]
Build complete! (9,22 sec.)
error: no tests found; create a target in the 'Tests' directory

It turns out I ran swift test in the wrong directory, but SwiftPM still goes through and builds everything before realising that there aren’t even any tests to run in this Package..

I don’t really know how to describe this bug concisely for the sake of a GitHub Issue but I will try now.

5

I created a GitHub Issue here and a minimal reproducer here.

1 Like
6

(replying to your edited message) That sounds like a fairly blunt instrument. swift build as a command is a great starting point for simple packages, but in this particular package that’s actually never what we want. We always want to build a specific product (by that I also mean the tests as a separate product).

Logically I can’t see any reason to build everything at the time of swift test – is not doing so (and instead building only what’s required) effectively an optimization that no-one has gotten around to? Or am I missing some case where it’s actually necessary to build non-dependencies?

As I mentioned above, swift test --skip-build is something that we’ve tried as a workaround, but it doesn’t work well in our case unless the build directory is clean (this is a separate SwiftPM bug that occurs when switching between building for Embedded and not), and it doesn’t play well with IDEs and other tooling.

7

You can use swift build --target to just build a test target and its dependencies, right? At least that seems to work in the simple package I just tried it with. I agree that swift test should also support the target/product options that build supports.

It’s probably more debatable what the default behavior should be without arguments and it’s probably not necessarily a good idea to just change it at this point.

8

I haven’t checked, but wasn’t this fixed in Swift 6.2?

1 Like
9

This might change the situation somewhat but it’s more of a workaround than a fix. I‘d much prefer that SwiftPM just didn’t build what we don’t need.

For example, if devs don’t have PostgreSQL server installed then our build would still fail because there’s no „winning“ unsafe flags we could reasonably provide. (see also: build performance). We‘re talking about a „very optional“ product that I’d even argue should not be built, most of the time.

If certain targets are never referenced from testTargets, be it directly or indirectly, I don’t think it‘d be controversial to make them simply not build on swift test. What could that reasonably break?

I’m honestly surprised that it’s even up for debate that SwiftPM shouldn’t be building random things that are not part of the dependency graph. How is that helpful?

If this point is indeed open for debate, can we try to get to the bottom of it? I think finding clarity on this will clarify what to do about the issue I reported.

Unsurprisingly, I think targets should only be built if they’re part of the dependency graph of what was actually requested. In my understanding of what is practically useful from SwiftPM, swift build should imply building all products (and – only – their dependencies), and swift test should imply building all testTargets (and – only – their dependencies) into a test runner and then running them. In the absence of one or the other, the respective command should be a no-op.

2 Likes
10

Maybe. But where do you go from there?

To run the tests we need a built product, not just a target – right?

11

True, I guess you need to build the PackageTests product for this to actually work. Something like swift build --product libPackageTests where lib is the name of your package.

12

That’s pretty much exactly what I’m after. Personally though I don’t need that level of granularity: I’d be fine with “all actual test targets and their actual dependencies”.

But: swift build --product XYZ suffers from exactly the same issue as swift test in that it tries to build all targets instead of just what’s needed (see the reproduction repo).

13

There isn’t actually much granularity here, libPackageTests is just a single product that contains all tests of the package lib. It’s created by SwiftPM under the hood (in the native build system, it’s different when using Xcode’s build system and the upcoming Swift Build implementation).

I missed that there’s a bug with --product, so yah this won’t help much in practice right now.

1 Like
14

I’ve commented on the GH issue. The latest main toolchain does a better job with swiftbuild. It’s still a work in progress. But definitely an issue with swift test.