[FR] Test with arguments and closures: DX suggestions to clarify all arguments were tested

Development Swift Testing
1

I have this setup for Swift Testing where I pass closures that each need testing:

@Suite(.serialized) @MainActor struct magnetLogic {
	typealias MoveFn = @Sendable (_ m: Magister, _ eid: EID) async -> Void
	static func getArgs() -> [MoveFn] {
		[
			{ @MainActor m, eid in /** ... */ },
			{ @MainActor m, eid in /** ... */ },
			// ...
		]
	}
	@Test(arguments: await getArgs()) func aPileBringsHuggers(
		moveFn: MoveFn
	) async throws {
		// ...
	}
}

However, running the test I only see this in Xcode:

image
image928×194 30.2 KB

I just see the one closure there called "(Function)"

and it logs out this:

​􁁛 Test aPileBringsHuggers(moveFn:) passed after 4.859 seconds.
􁁛 Suite magnetLogic passed after 4.859 seconds.
􁁛 Suite E2ETests passed after 4.860 seconds.
􁁛 Test run with 1 test passed after 4.860 seconds.
Program ended with exit code: 0

Searching for "passed" just gives this part, so I'm not sure Swift Testing is really testing all my closures.

How to know for sure each closure is properly being tested?

2

Function references do not have identity in Swift, which means that Xcode cannot disambiguate them. If you run the function from Xcode, it should run all cases, which you can verify from the console output or test report.

3

Thanks for the info!

which you can verify from the console output or test report.

That's the thing. In the console it only says
​􁁛 Test aPileBringsHuggers(moveFn:) passed after 4.859 seconds.

It doesn't say how many arguments or parameters it tested.

I believe if the default message in the console would say the following:

􁁛 Test aPileBringsHuggers(moveFn:) with 2 arguments passed after 4.859 seconds.

it would be a really great step forward in understanding!

4

Even better of course is if this could just include the index of the function:

Like:

  • :white_check_mark: (Function) 0
  • :white_check_mark: (Function) 1
  • :white_check_mark: (Function) 2

Instead of just one:

image
image596×374 17.9 KB

That way you could run a closure argument at a specific index, which would greatly improve the DX in my opinion!

5

Individual test cases are recorded separately before the overall test is reported as passing or failing.

6

Ah yes, I found that it said ​􀟈 Passing 1 argument moveFn → (Function) to aPileBringsHuggers(moveFn:) twice.

So the way it went:

􀟈 Test aPileBringsHuggers(moveFn:) started.
􀟈 Passing 1 argument moveFn → (Function) to aPileBringsHuggers(moveFn:)
// ...
​􀟈 Passing 1 argument moveFn → (Function) to aPileBringsHuggers(moveFn:)
// ...
​􁁛 Test aPileBringsHuggers(moveFn:) passed after 4.677 seconds.

Given that these logs easily get buried, I think something like this would be way better:

􀟈 Test aPileBringsHuggers(moveFn:) started.
􀟈 Passing 1 argument moveFn → (Function) to aPileBringsHuggers(moveFn:)
// ...
​􁁛 Argument in aPileBringsHuggers(moveFn:) passed after 2 seconds.
​􀟈 Passing 1 argument moveFn → (Function) to aPileBringsHuggers(moveFn:)
// ...
​􁁛 Argument in aPileBringsHuggers(moveFn:) passed after 2 seconds.
​􁁛 Test aPileBringsHuggers(moveFn:) with 2 arguments passed after 4.677 seconds.

Here above, I suggested

  1. an extra log after each argument passes or fails, and
  2. a total amount of arguments tested to the final log.
7

Would you please file a GitHub issue linking to this thread? I see no reason we can't log the total test case count.

1 Like
8

Thank you very much!