Hello everyone! I know this topic has been discussed before, but most arguments ended with the claim that unit tests in source code would slow down compilation speed. Please feel free to link any relevant discussion if you feel more nuanced arguments were made, but here are some of the benefits of unit tests in source.
Testing is usually treated as an afterthought due to the boilerplate required and the context switching between the test and source-code files. By having tests and the tested functionality in the same file, users will have an easier time figuring out what they need to test and doing the actual debugging.
With the current approach of
@testable imports, users often mark otherwise
fileprivate declarations as
internal. This not only pollutes the namespace but also reduces local reasoning over code, since it's easy to forget that a property shouldn't be touched outside of its file, accidentally introducing a bug in your code. In other words, current testing practices undermine access control.
Easily accessible tests act like a kind of documentation, that can help maintainers understand what a piece of code is supposed to do and how it should be used. Even if other documentation perfectly describes a given declaration, seeing how it should be used in one place is always helpful.
Of course, the reason why unit tests are separated from source code isn't just because some engineer didn't of the benefits; tests within source code present significant challenges.
Swift is already criticized for its slow compilation times, but adding tests in the source could slow down the compiler even more. At the end of the day, unit tests are more code that needs to be parsed and type-checked. While there may be some improvements from stricter access control as discussed above, many large projects (such as the compiler itself) separate building the source and tests to accelerate compilation times.
When prototyping, most users quickly implement a feature, debug it, make adjustments, and repeat until this feature meets some criteria. Testing usually comes after that to properly enforce said criteria and ensure that there are no regressions. If the program takes longer to compile, the feature will take longer to implement. Pototyping is further inhibited in the case where a tested feature is extended (e.g. changing from a struct from a
CustomStringConvertible conformance). In this case, the user would have to implement the new conformance, but also change the tests that reference
description. These changes happen before actually running and debugging the new code, thus adding an extra, arguably unproductive step between the implementation and debugging steps.
While the limitations are significant, they can be circumvented. The benefits mostly arise from unit tests being in close vicinity to their corresponding source code, while limitations are a result of being forced to build source and test code together.
After some quick reading, it seems that Rust uses
#[cfg(test)] in unit tests, which appear near production code, to tell the compiler that the unit tests should only be compiled only when
cargo test is run. In Swift, that would be similar to enclosing unit tests in
#if(Test) when they are in the Sources directory. This way, all the benefits are preserved but the limitations are mostly alleviated since code in
#if directives will only parse. Arguably, a compiler directive to also bypass parsing of the enclosed code would ensure minimal effect on compilation times.
SwiftUI previews are not exactly tests, which have their own directory within a SwiftUI app, but are a way of verifying code within the source file. They only featured
#if directives in the preliminary betas, but that was dropped because they usually delegate to a view and are a direct way of debugging the given view. Nevertheless, they prove the need of verifying code within the source file.
All in all,
XCTests have carried a lot of technical debt from their Objective C days and have started to show their age. Tests are still defined in classes and the assertion methods are prefixed with
XCT. I think we have an opportunity to design a testing library that is consistent across platforms and better adheres to Swift's design principles. Whether unit tests should be written alongside source code is just one design consideration, of what should IMO be an overhaul of testing in Swift.