I'm converting some xctest tests over to the swift testing framework and I have something that looks like
@Test func test1() throws {
let a = try getAnObject()
let b = try a.getADifferentObject()
...
}
I'm not expecting either getAnObject() or getaDifferentObject() to throw an error, but if they do, the reported failure from the testing framework is weak - the line number reported is always the start of the function so you can't tell which one of the functions threw the error. To try to get better localization of the problem, I tried a couple of different ways
let a: MyObject
#expect(throws: Never.self) {
a = try GetAnObject()
}
But the compiler doesn't like that due to the assignment being inside a closure: error:cannot assign to value: 'a' is a 'let' constant
guard let a = try? getAnObject() else {
Issue.record("get failed");
return
}
This works, but you don't have access to the error value.
var a: MyObject? = nil
try #require(throws: Never.self) {
a = try getAnObject()
}
This also works and you can see both the proper line number and error value from the framework, but the compiler issues a warning: warning:Passing 'Never.self' to '#require(throws:_:)' is redundant; invoke non-throwing test code directly instead (from macro 'require')
Does anyone have any better way of getting the proper line number and error value for this situation?
The testing library attempts to determine the source location and backtrace when an error is thrown, and the intent is for your original code to "just work" and report the correct line if an error is thrown. It sounds like that isn't working for you, though, so we can try to troubleshoot that.
Are you using Xcode and running these tests on an Apple platform? If so, one potential reason it might not be working is if your test code, or a library involved in the thrown error call stack, does not have debugging symbol information available. You could check the value of the DEBUG_INFORMATION_FORMAT Xcode build setting for these targets.
If that's not the problem, it would help to have a reproducible scenario to troubleshoot further. If the problem is seen when running Xcode, I'd suggest filing a Feedback with Apple and attaching an example if possible. If it's on another platform, you could start by filing a GitHub issue in the swift-testing repo.
It's expected that within the trailing closure of a macro invocation the source location refers to the start of that macro. That's a constraint of Swift macros. We do have an issue tracking it, although I am not able to look it up right this instant.
Yes, please. I know there is an issue with collecting an accurate source location for macros with trailing closures, but I am more focused on the problem you originally mentioned about plain try expressions, without any #expect(...), not reporting the correct location.
The issue got closed as no intent to fix. Can I put a bug in somebody's ear about having a cleaner way of getting better localization when you need the return value. E.g.
let foo = try #require(expression)
I'm not in any way proposing this syntax, just saying it'd be nice to have a straightforward mechanism to report the line number where the throw occurs when you also need the return value of the expression for later on.
Oh yeah, in my case, the object being returned is ~Copyable which probably makes things tougher.
That's ok if you don't need the result of the getAnObject() afterwards, but if you need the result for a later expression in the test, it becomes uglier.
Sorry, I should probably have left it open. I have reopened it.
In practical terms, the information you want here simply does not exist at runtime and there would need to be major compiler-side changes to add it. That doesn't mean we shouldn't track the issue, but it does mean that a solution is not likely in the near term.
They're both equally important for the default failure report output. And of course, you'd like to optionally be able to specify a personalized string to output extra information to be able to determine the cause of the failure without having to add debugging output and re-run the test.
Understood. I'd be happy enough with an enhancement to #require or #expect - or a completely new macro - that produced a failure report if an exception was thrown and returned the value of an expression if not.