Issue.record message prefixes

Using Swift
1

In migrating to swift-testing, I have extensive test code using API's that handle validation and rendering, so I generally won't use #expect(...) and only really need a way to signal the failure, which I can do with Issue.record(...) or #expect(Bool(false), ...)

However, those end up prefixing my messages with their own kind. E.g.,

  • Issue.record(Error, ...): Caught error: ...
  • Issue.record(Comment, ...): Issue recorded: ...
  • #expect(Bool(false), ...: Expectation failed: Bool(false) ...

This is needless boilerplate. Worse, since Xcode positions issue text at the end of the line, even minimal messages end up hidden.

Solutions? Best way to engineer/contribute a solution?

2

We have a GitHub issue tracking, broadly, custom issue types here: New Issue.Kind case for third party expectations. · Issue #490 · swiftlang/swift-testing · GitHub

In general though, I would probably recommend not fighting the library here.

3

Is your concern primarily that these prefix strings are shown in Xcode’s failure messages which appear on/above the failing line of code? I understand they’re shown in other places too but I’m trying to determine which spot you find the most bothersome.

Either way, please do file a Feedback with Apple about the way this is shown in Xcode (and post the FB number here if you don’t mind), since that aspect is not directly controlled by the open source Swift Testing project.

4

Thanks for the reminder/suggestion...

Good point. I'll ping Xcode on point.

From swift-testing (or SPM?) the output reasonably puts the message on the next line for Comment and #expect

✘ Test testTestingSourceLocation() recorded an issue at SrcLoc.swift:12:36: Expectation failed: Bool(false)
↳ a-ok Unequal  -  exp: sa , act: a

✘ Test testTestingSourceLocation() recorded an issue at SrcLoc.swift:12:36: Issue recorded
↳ a-ok Unequal  -  exp: sa , act: a

But the message seems to be on the same line for Error:

✘ Test testTestingSourceLocation() recorded an issue at SrcLoc.swift:12:36: Caught error: e("a-ok Unequal  -  exp: sa , act: a")

(I can check if that depends on how the message is created.)

I personally would prefer output closer to type file:line:col function: message:

FAIL SrcLoc.swift:12:36 TestClass.testFunction(): error message

But I imagine swift-testing needn't get tangled in arguments about output format from clients.

1 Like
5

In the long term, I want to get Swift Testing out of the business of formatting text/output as much as is possible, and to leave that stuff up to the tools that call into Swift Testing (e.g. Xcode, SwiftPM, and VSCode.)

In a perfect world, a developer could specify their own format string or something like that for messages from the testing library, although that level of customization is probably overkill.

3 Likes