Should SPM allow colocating source code and unit test?

Development Package Manager
1

Currently Swift Package Manager has the very explicit requirement that source code must be placed within the 'Sources' directory and any test code must be placed within the 'Tests' directory. Attempting to move any test cases into the Sources directory will no longer make the tests executable.

This prevents colocating source code with their tests, in order to be better discoverable and provide a grouping. Especially in large codebases, it can be hard to find their related tests - especially if there are test cases that are not named like the source code but still related. Colocating source code with their tests cases does not only improve discoverability but also requires less context switching and avoids having to manually find the test case somewhere hidden in the tests directory.

For regular Xcode projects this has always worked, as the file's target can be assigned separately within e.g. the Inspector and does not depend on the location of the file on disk. The file's target is stored within the .xcproject file (which, luckily, doesn't exist in SPM).

I imagine three (likely very naive) solutions:

  1. any source code file that imports the XCTest module
  2. any source code files that contain Tests or Test as suffix in their names (this could be potentially dynamic and be specified within the Package manifest, although I'm not sure about the value here and consider the naming recommended)
  3. utilize the same mechanism that is being used for automatic test discoverability on Linux and Apple Platforms (available since Swift 5.1) for discovering any test cases

For the first and second option, I'd always consider the entire file to belong to the tests.

To be fair, I didn't dig into any of these details, as I would like to first validate my assumption that this is something useful not only for myself. I've been using test colocation for a very long time now and had very good experiences with doing so.

I read @codafi's proposal in 2018 about introducing in-line test annotations and its thread but it seems to be way more complex and ambitious than what I'm asking for. Thanks @briancroom for guiding me here and suggesting to start a thread!

I would love to hear what y'all think and if there's actually any need for this. Please be patient with me, as this is my first contribution on the forums. :smile:

I apologize if this is a duplicate, please let me know if it is. I tried my best finding a different topic that pitches the same idea but ultimately failed.

2 Likes
2

You can already put any target wherever you want by not leaving the path argument to be inferred. The only difficulty is in interleaving the sources of two targets in the same directory, which currently requires shenanigans with extensive exclude or sources arguments.

Either of these would break existing packages that use those in legitimate non‐test targets, including the SwiftPM package itself.

I am pretty sure this would be of no help to you on this issue.

You are not the only one who has wanted tests to be right alongside the code they deal with. Some of us have wished not just for side‐by‐side files, but for tests embedded at the declaration. Here is one way it could be done:

/// Computes something.
///
/// Examples:
///
/// ```@test
/// XCTAssertEqual(computeSomething(), 1)
/// ```
func computeSomething() {
  // ...
}

As demonstrated above, it might already be possible today now that we have plugins. The plugin would scan the source, extract the tests and spit them out as generated sources in the conventional format during the build.

But for now, doing so would lose syntax highlighting, code completion and related features. Refactoring the syntax to enable them would require support in the compiler at the language level, which would have little to do with SwiftPM.

3

Actually, the following would probably work just fine with a plugin too. The otherwise unsued function would be stripped in optimized builds anyway.

func computeSomething() {
  // ...
}
/* @test */ func testComputeSomething() {
  assert(computeSomething(), 1)
}
4

This approach assumes that you are writing very narrow tests that cover individual files, instead of wider scoped tests that test larger behaviours. Without getting into the weeds of which is better, we shouldn't be making it difficult for one approach vs the other, so I'd be against this change