Robustly find the path to an executable target from within swift-testing

Development Foundation
1

I’m writing some functional tests that work an executable target that I create with SwiftPM, and want to check in on the “most robust way” to get a path to the executable target from within the scope of a test/swift-testing.

In doing a bit of searching, there’s a mechanism to search using the FileManager and Bundles, especially if there’s a class where you can use the Bundle(for:)method and then compute/derive the result from there. (per a semi-related suggestion in Access file in tests with swift-testing)

I also saw some references (from LLM generated answers) to reference and build on getting a location from BUILT_PRODUCTS_DIR, and appending the executable name from there to derive a path. But I wasn’t sure if that was a guarantee, or an option that may or may not be passed in, and which - if either - is a more robust means across platforms (I’m aiming for macOS and Linux, but I’d ideally like to support Windows, Android, etc. if I can do so up front).

Is there a suggested path or provided API from swift-testing to help or support this kind of scenario?

2

When running tests, the executable target isn't built as a separate product. Rather, its object files are linked into the test target which is run, depending on platform, as either a hosted bundle or as an executable.

The new #bundle macro in Foundation is the approximate replacement for Bundle.init(for:) but, if you have a class handy, Bundle.init(for:) will do what you need.

2 Likes
3

I am on Swift 6.2.3. I wonder why this #bundle is not available on Linux.

4

I believe it's available when you import Foundation, but beyond that I'd have to direct you to the Foundation team for further assistance. Sorry!

1 Like
5

You always have to import Foundation when using Bundle. However, the issue still exists.

6

Then I'd have to direct you to the Foundation team for further assistance! I can move this thread to their category if you like.

1 Like
7

Thanks. Please.

8

A few points of clarification:

#bundle is indeed missing from Foundation on non-Darwin platforms. I’ve opened this Github Issue to track this: GitHub · Where software is built. The macro implementation exists in the Foundation Macros library on non-Darwin, we need to add the macro declaration to the Foundation module (where Bundle exists) and implement the Bundle(_dsoHandle:) initializer that the macro uses in swift-corelibs-foundation.

However, #bundle does not actually help with this problem as #bundle is your current module’s resource bundle and not executable bundle. In a standard app these bundles are the same (the same app bundle contains both executables and resources). However, SwiftPM’s build products produce two separate bundles: a resource bundle (referenced by #bundle/Bundle.module with all of your resources) and an executable (whose bundle would be found by Bundle(for:) / Bundle.main).

It’s entirely possible that Bundle is the right place to provide some form of API to find this executable, but before we can do that:

  • SwiftPM needs a way to mark unit tests as dependent upon an executable target (if you do so today, as @grynspan mentioned it links your executable object files into your test executable and doesn’t guarantee the executable itself is built)
  • SwiftPM needs to expose some information about where these executables are from the build system to runtime (and the information can be then exposed to clients via APIs on Bundle in Foundation, or from Testing, etc. - but first the information needs to be somewhere before we can expose it)
2 Likes
9

Thanks. I just saw your PR. Does indicate that this will be available in Swift 5.4?

10

My PR implements #bundle on the main branch which will mean that it will be available in Swift 6.4+

2 Likes
11

This is probably obvious to most of you, but I am curious. Once you have the executable, what are you going to use it for, and how? Do you run it with arguments using process and compare expected output to piped output from process?

Then perhaps you could easily run test cases from the terminal and copy the input and output for pasting into, say, and array of (input, expected-output) and run tests on the pairs. etc.

12

That's pretty much exactly the thing I was doing - invoking the executable with various arguments and verifying that the it operated the way I expected. I wasn't actually checking the positive path of it working, but verifying how it failed in a couple of scenarios I had.

13

Hi,

You might already be doing something like this, or something better.

But I thought it might be of interest anyway.

If you wanted to use execute your executable inside a test function, something like this might work.

Assume your package is in /Users/xx/Demo:

Demo> tree -L 3
.
Demo> tree -L 3
.
├── Package.resolved
├── Package.swift
├── Sources
│   ├── Demo
│   │   └── Main.swift
│   └── Support
│       └── Support.swift
└── Tests
    └── DemoTests
        └── Tests.swift

and that the name of the executable is "demo".

You can run swift test with the --scratch-path option

> mkdir mybuild
Demo> swift test --scratch-path /Users/xx/Demo/mybuild

Presumably, the debug build executable will always be in the "debug" subfolder of the scratch path, in this case, at /Users/xx/Demo/mybuild/debug. It works today, at least:

Demo> /Users/xx/Demo/mybuild/debug/demo --version
version 0.1.0

Again, assuming you are using swift test --scratch-path, you get the absolute path of the package like this:

 let packagePath = FileManager.default.currentDirectoryPath

Which makes it easy to set up the excutable URL.

Unless there a way set the scratch path when running Xcode, this will not work if you run the test from within Xcode. If there is, good. Otherwise this approach is pretty fragile. But in some cases it could be helpful.

14

Variations on that exact theme are how I've worked around this so far - setting up conventions for where to expect a scratch path and searching it, or using absolutely paths where I can take advantage of that. The only real downside is that puts some values outside the knowledge scope of the test itself, which doesn't always know up front where that scratch path is, so you're having to manage somewhat fragile test setups if you don't adhere to the conventions.

When I step into driving functional and integration tests using one of these frameworks (swift-testing typically), the other gotcha that strikes is the relative path of where it's being executed from - it's different between macOS and Linux with SwiftPM - different executables that end up driving the testing work.