Hi everyone,
I’m excited to announce a new open source project exploring improvements to the testing experience for Swift. My colleagues @briancroom, @grynspan, @chefski, @Dennis and I have been working on this in recent months and have some early progress we're excited to share.
Inspired by what’s possible with Swift macros, we’ve built a testing library API that can:
- Provide granular details about individual tests using an attached macro named
@Test. This enables many new features like expressing requirements, passing arguments, or adding custom tags, all directly in code instead of separate configuration files. - Validate expected conditions in tests with detailed and actionable failure information using an expression macro spelled
#expect(...). This works by capturing the values of passed-in expressions and their source code automatically to inform failure messages, and is also easier to learn than specialized assertion functions since it accepts built-in operator expressions like#expect(a == b). - Easily repeat a test multiple times with different inputs by adding a parameter to the function and specifying its arguments in the
@Testattribute.
Here's an example: It shows one test function, denoted using @Test , which includes two traits: a custom display name and a condition which decides whether the test should run. The test creates a food truck, stocks it with food, then uses #expect to check whether the quantity of food is equal to the value we expect:
@Test("The Food Truck has enough burritos",
.enabled(if: FoodTruck.isAvailable))
func foodAvailable() async throws {
let foodTruck = FoodTruck()
try await foodTruck.stock(.burrito, quantity: 15)
#expect(foodTruck.quantity(of: .burrito) == 20)
}
If the above test were to fail, #expect would capture the values of sub-expressions like quantity(of: .burrito) as well as the source code text. This allows rich diagnostic information to be included in the output:
✘ Test "The Food Truck has enough burritos" recorded an issue at FoodTruckTests.swift:8:6:
Expectation failed: (foodTruck.quantity(of: .burrito) → 15) == 20
Repeating a test multiple times with different inputs—known as parameterized testing—is also simple using this approach. A @Test attribute may contain arguments, and the function will be called repeatedly and passed each argument:
@Test(arguments: [Food.burrito, .taco, .iceCream])
func foodAvailable(food: Food) {
let foodTruck = FoodTruck()
#expect(foodTruck.quantity(of: food) == 0)
}
A New API Direction for Testing in Swift provides an in-depth look at our vision, describes the project's goals, and shows more examples of our proposed approach.
These ideas have been prototyped in a new package named swift-testing, which is currently considered experimental and not yet recommended for general production use. If you’re interested, we encourage you to clone it, explore its implementation, and try using it to write tests for your project. See Getting Started for instructions.
We would love to hear your feedback about these ideas or your experience using the experimental swift-testing package. Feel free to reply here, or create topics in the newly-created swift-testing forum category with your thoughts about this new approach. Some questions to consider when providing feedback:
- What do you find difficult about testing in Swift today?
- Does this address those challenges?
- Are there additional features or improvements you'd like to see?
With this new experimental swift-testing package now open source, we plan to begin working on it in the open and are exploring work in closely-related components, such as the compiler, swift-syntax, and the package manager. Over time, we expect that these efforts will help this project mature with the hopes of providing a superior solution to XCTest.
If you're passionate about software quality, we invite you to join this exciting journey with us and help shape the future of testing in Swift!
Stuart
