This post describes the set of SPI groups used in swift-testing. In general, two groups of SPI exist in the testing library:
Interfaces that are not needed by test authors, but which may be needed by tools that use the testing library such as Swift Package Manager; and
Interfaces that are available for test authors to use, but which are experimental or under active development and which may be modified or removed in the future.
For interfaces used to integrate with external tools, the SPI group @_spi(ForToolsIntegrationOnly) is used. The name is a hint to adopters that they should not be using such SPI if they aren't building tooling around the testing library.
For interfaces that are experimental or under active development, the SPI group @_spi(Experimental) is used. Such interfaces are intended to eventually become public, stable API, so test authors are encouraged to hold off adopting them until that happens.
Other SPI groups
The testing library currently makes use of various other SPI groups such as @_spi(ExperimentalTestRunning). Interfaces in these groups will be migrated to one (or both!) of the two above as time permits.
SPI stability
The testing library does not guarantee SPI stability for either group of SPI.
For SPI marked @_spi(ForToolsIntegrationOnly), breaking changes will be preceded by deprecation (where possible) to allow tool authors time to migrate to newer interfaces.
SPI marked @_spi(Experimental) should be assumed to be unstable. It may be modified or removed at any time.
API and ABI stability
Once swift-testing reaches its 1.0 release, API changes will follow the same general rules as those in the Swift standard library: removal will be a last resort and will always be preceded by deprecation to allow tool and test authors time to migrate to newer interfaces.
As a general rule, ABI stability is not guaranteed by the testing library.
Edit: We have archived this post in the swift-testing repository here for posterity.
it will be challenging to document these until lib/SymbolGraphGen gains the ability to understand @_spi names, as today they are just pooled together with all the non-SPI symbols. i know you’ve mentioned this offline, and i’m wondering if there are any plans to include information about SPIs in symbol graphs? cc @QuietMisdreavus
It almost seems like we want a separate "swift-testing for tool authors" DocC bundle that only contains symbols marked @_spi(ForToolsIntegrationOnly) and articles specific to it. I don't think that's something we can build today though.
correct, that would require lib/SymbolGraphGen to be aware of SPI names, which it currently is not.
if lib/SymbolGraphGen even just printed the attribute in its subclass of DeclFragmentPrinter, we could extract the names within the documentation compiler. but it currently strips the attributes entirely for some reason.
Something else you may need to look at filtering are symbols whose names start with underscores. Such symbols are, as a Swift convention, exposed for technical reasons but not intended to be used by clients.
swift-testing has a number of these symbols (many with double underscores, which is the extra-serious "don't even look at me!" zone.) Which isn't to say these symbols are uninteresting to somebody digging into the repo (and hey, maybe there should even be a "show me the secret symbols" checkbox?) but they're not going to be useful to the average test author.
yes, we are 100% aware of this issue and filtering is going to be a key focus of the next Unidoc release after 0.8.
we can and should be filtering underscored symbols from members lists, this effort has simply been a victim of frontend bikeshedding at our org (“how are we going to make this look different from the deprecated symbol styling?”) and i’m personally going to see that this is prioritized moving forward.
removing them from the sidebar is a little bit more nuanced, since we probably still want to preserve a way to access them without combing through individual pages. the sidebar is not dynamic enough to support “checkbox filters” right now. that’s a limitation of the sidebar. a checkbox filter would be a great feature.
@_spi is problematic for reasons we’ve discussed offline, but to summarize for others:
lib/SymbolGraphGen has no awareness of @_spi other than knowing if at least one attribute is present. so we can only treat these symbols as belonging to an imaginary __combined__ SPI.
imo, this is an issue in SymbolGraphGen, and it needs to be fixed eventually in SymbolGraphGen.
Unidoc/Swiftinit has an ABI, and we did not want to bake the concept of a “__combined__” SPI into the symbol graph ABI since that would conflict with the real SPIs if/when we ever gain access to that information.
those were a plausible enough excuse to kick the @_spi can down the road in the past, but i am slowly coming to accept that lib/SymbolGraphGen is unmaintained today, and i think it is reasonable for us to address the issue at the documentation compiler level.