Error traces from `swift test`?

Development Source Tooling
1

When you run Swift tests in Xcode, and a test gets stopped by a thrown error that bubbles all the way back up to the test, Xcode lets you expand the error in the sidebar to see where the error was thrown, and all the other stack frames in between the top-level test code and the error. This is really helpful. However, when running tests manually via swift test in the CLI, all you get from the output is the fact that the error was thrown, but no information on where. Does the CLI tooling have a way to emit this information, or is this some magic that only Xcode does?

2 Likes
4

Spam posts aside, it would be nice to know if there's a good answer to this question...

5

@testing-workgroup, is this something we can enable?

6

I experimented with exactly this problem a week or so ago:

Screenshot of macOS's Terminal.app showing more information about an error thrown from within a test.
Screenshot of macOS's Terminal.app showing more information about an error thrown from within a test.1848×1080 229 KB

It required some serious unsanctioned hijinks, it was way too slow to use in production, and it won't work if you're building on one system and running tests on another, but… it was a fun experiment. :slightly_smiling_face:

There are also some platform-specific problems we'd need to overcome:

  • On Darwin, we can symbolicate backtraces if a .dSYM is available (which it usually will be in this context), but there's a private Apple framework called Core Symbolication that does most of the heavy lifting and gets us extra info like line numbers. Core Symbolication is also able to symbolicate backtraces captured from other processes, which is key to the feature working well in Xcode when running tests on another device like an iPhone. We can't use Core Symbolication directly from Swift Testing.
  • On ELF-based platforms, we do not have the ability to symbolicate backtraces. The stdlib's Backtrace type gets us a significant portion of what we need, but is missing API to let us construct an instance of Backtrace from stack frames we've previously gathered or gathered out-of-process.
  • On Windows, the API for symbolication is… fun. I haven't dug in as much on Windows but the API for symbolication is (bafflingly) not thread-safe. It does let you symbolicate backtraces from other processes though, at least some of the time?
  • On Wasm, we just plain don't have backtraces, so I imagine we'd just omit the feature there.
3 Likes