[Preview] Embedded Swift support in Swift Testing

Hi Swift friends (and… Swift enemies?)! I'm here with an exciting new update about Swift Testing.

For those of you who keep a close eye on Swift Testing's GitHub repository, you may have noticed a lot of changes landing recently with the label embedded-swift on them. Well, those changes weren't just for fun, you know: Swift Testing now has fully-functional experimental support for Embedded Swift. :partying_face:

Swift 6.4 introduced several major changes to Embedded Swift that have let us make real progress toward supporting it in Swift Testing. I'd like to thank everybody who worked on those features: they make Embedded Swift better and better with each release.

We've tried it out…

… on a Playdate[1] … … and even on a 23-year-old GameBoy Advance SP[2]!

What works and what doesn't

Almost the entire Swift Testing API surface is supported in Embedded Swift, including the basics like @Test and #expect() and more complex features like custom test scoping traits and structured JSON Lines output. It's much easier to just list what doesn't work:

  • Tests cannot be nested in extensions to types. When you nest a function with the @Test attribute in a type, that type automatically becomes a suite, which you can then use to organize related tests and other suite types. This functionality is not supported for type extensions in Embedded Swift because it relies on the Swift runtime's ability to report a type's fully-qualified name (for example, "Swift.String.UTF8View") which is not implemented in Embedded Swift. You can nest tests in type declarations with or without the @Suite attribute:

    struct S1 { @Test func f() { /*...*/ } } // ✅ OK
    @Suite struct S2 { /*...*/ } // ✅ OK
    extension S3 { @Test func f() { /*...*/ } } // 🛑 ERROR
    
  • Parallelized testing is not supported. Embedded Swift targets may or may not support parallelism to various degrees, and whether or not they do is not visible to Swift Testing at compile time. For the moment, we run all tests serially. async on test functions is still supported, as is the use of any Swift Concurrency interfaces that are otherwise available in Embedded Swift and supported on a given target.

    As a consequence of this constraint, the .serialized and .timeLimit(_:) traits are not available in Embedded Swift.

  • Runtime value capture has limits. Because String.init(describing:) is not available for arbitrary values in Embedded Swift, we can only provide string descriptions of values that conform to Swift Testing's CustomTestStringConvertible protocol. As a convenience, conformance to this protocol is automatic for a variety of types in the Swift Standard Library.

    Separately, if you throw an error from a test function or within #expect(throws:), the testing library cannot cast it to a form that can be stringified even if it conforms to CustomTestStringConvertible. As such, the descriptions of those errors are limited too. All is not lost, however: the Error protocol has two underscored protocol requirements, _domain and _code, that you can implement to provide basic descriptions of errors you throw.

  • Tags cannot be declared in extensions to types nested in Tag. As with suite types, the @Tag attribute relies on the fully-qualified name of the type on which it is declared. Since that information is not available in Embedded Swift, tag declarations are only allowed in extensions to Tag (or in types declared within the same extension):

    extension Tag { @Tag static var performance: Tag } // ✅ OK
    extension Tag { struct Priority { @Tag static var high: Tag } } // ✅ OK
    extension Tag.Priority { @Tag static var low: Tag } // 🛑 ERROR
    

What you need to do before you can use it

  • If you want to try out Swift Testing on your Embedded Swift target, you'll need to make sure it supports Swift's Platform Abstraction Layer. In addition to the functions used by the Swift runtime, you'll also need to define a few more functions declared by Swift Testing in an annex to the Platform Abstraction Layer.

    We've written reference implementations of the Swift Testing Platform Abstraction Layer annex functions for POSIX-shaped targets, for WASI, and for Pico SDK-compatible devices. If you need to write your own implementation, note that most of the annex's functions can be no-ops if you don't need them. Review the header linked above for details.

  • For the moment, unlike with non-Embedded Swift, you'll also need to define a main function that calls into Swift Testing via its swift_testing_embeddedMain() function:

    int main(int argc, char *argv[], char *envp[]) {
      swift_testing_embeddedMain(argc, argv, envp);
    }
    

    The main() function is easiest to write in C, but you can write it in Swift if you don't need to pass in any arguments or environment variables:

    @main struct Main {
      static func main() {
        swift_testing_embeddedMain(0, nil, nil)
      }
    }
    

Trying it out

The easiest way to build tests for Embedded Swift is to create a package that lists Swift Testing's main branch as a dependency, as well as the Platform Abstraction Layer annex implementation appropriate to your target:

// swift-tools-version: 6.4
import PackageDescription

let package = Package(
  name: "MyPackage",
  products: [
    .executable(
      name: "MyTests",
      targets: ["MyTests"]
    ),
  ],
  dependencies: [
    .package(url: "https://github.com/swiftlang/swift-testing.git", branch: "main"),
  ],
  targets: [
    .executableTarget(
      name: "MyTests",
      dependencies: [
        .product(name: "Testing", package: "swift-testing"),
        .product(name: "EmbeddedPlatformPOSIX+Testing", package: "swift-testing"),
      ]
    ),
  ]
)

Note that, for the moment, test targets in Swift Package Manager are not supported for Embedded Swift.

When you build your test product, set the SWT_EMBEDDED environment variable to 1 to ensure Swift Testing builds with the correct configuration:

SWT_EMBEDDED=1 swift build --product MyTests --toolset ... --swift-sdk ...

If and when everything builds and you install the build product on your target, you'll see output similar to what you see when you run Swift Testing on macOS, Linux, etc.

For more information about building Swift Testing as a package dependency, see our documentation on the subject here.

Future directions

Building Swift Testing from source can take some time as it requires building Swift Testing's macros and, indirectly, the swift-syntax package. We plan to start including Swift Testing in the Embedded Swift toolchain in an upcoming Swift release, ideally Swift 6.5. For a complete developer experience, this will require some work in other components such as Swift Package Manager; we'll be reaching out to code owners and workgroups in the near future to plan and schedule the necessary work.


  1. Courtesy of @SlaunchaMan (Jeff Kelley) ↩︎

  2. Courtesy of @harlanhaskins (Harlan Haskins) ↩︎

31 Likes

This is amazing and I'm very much looking forward to making use of this for some upcoming projects. Excellent job from all involved!

1 Like

This is awesome work! Love to see Swift Testing work for embedded environments.

Doug

3 Likes

"Can it run Swift Testing?" will be the new "Can it run Doom?"

I'm gonna try it out on Pi Nano and Micro:Bit.

1 Like