Packge + import or Internal + @testable import?

Now that we are one year in to the availability of package access SE-0386… are engineers all starting to see this as the future direction of writing testable code (when the test target is defined as a module in the same package)?

It looks like the default swift package init command still produces a test file template with the @testable import pattern. Was there any discussion about migrating that template directly to import (and then using this as an opportunity to introduce engineers to the package access control… or would losing the "default" internal access control lead to too much friction (with engineers no longer understanding why their production code is not available in the test target)?

1 Like

I think the package access level can help a lot in making code more testable. I'm personally not certain that it should completely replace @testable in all cases though—at least, not in its current form, without some additional improvements.

AFAIK, applying package access level to an API causes the symbols for that API to always be exported. For library maintainers, this might be undesirable if it exposes symbols which should never be exposed. It also might prevent useful optimizations, even in release builds. In contrast, @testable exposes the symbols for declarations with internal access level, but only in debug builds or when -enable-testability is enabled. In release builds, those APIs' symbols are never exposed and are still eligible for optimization.

For this reason, I personally would be hesitant to recommend unconditionally replacing @testable with package access level in SwiftPM templates — at least, for library targets. Perhaps there are some templates for which this doesn't matter as much, such as executable targets. Maybe SwiftPM could use @testable only in test targets for library and other targets where these concerns are relevant, but move forward with adopting package access level in other templates where this isn't a concern.

I would be very interested to hear someone more familiar with the Swift compiler talk about what, if any, challenges that maintaining support for @testable style builds has, and whether there have been any ideas for replacements for it that don't have the downsides I mentioned above.

3 Likes

IIRC, the potential relationship between @testable and package came up back when we were discussing it, but I unfortunately don't recall the details.

Maybe @elsh has some thoughts?

1 Like

I think both the code size impact of testing-only package declarations is relevant, as is the annotation burden in the source code. Whether package in its current form is a good replacement for @testable imports is a decision that needs to be made on a case-by-case basis. There is one notable advantage for package authors to using package instead of @testable, which is that it makes it possible to test the optimized build of the code (-enable-testing is only compatible with debug builds).

It would be nice to have some kind of "whole package" optimization that eliminates code associated with unused package declarations at link time (assuming the end product is a statically linked binary). That would probably address the code size concern for packages that would need to mark a lot of declarations package just to facilitate testing.

2 Likes

We don’t recommend adding package access level solely for testing purposes. However, if package is already extensively used in your production code, it would be a better alternative to @testable, as we've been discouraging the use of that annotation due to unmaintained bugs, with plans to eventually deprecate it. That said, if your project contains few package symbols or needs access to internal symbols for testing, @testable can still be an option.

For the default SPM test template, though, it might be worth moving away from @testable since package access control already grants visibility to symbols within the same package across modules, thus the need for @testable becomes redundant. cc @Max_Desiatov

2 Likes