First impressions adopting swift-testing

[grynspan (Jonathan Grynspan)]

Thanks for the feedback! A few notes:

@taylorswift

one thing i think could be improved is the documentation should really be targeted towards Swift 5.10 users instead of nightly toolchain users, who probably make up a tiny fraction of the library’s potential audience. i often felt like the critical information i needed in order to use the library on Swift 5.10 was buried in asides and footnotes, and the articles were primarily addressed towards nightly users.

As we've previously discussed, support for Swift 5.10 is temporary to aid developers who are not prepared to install a nightly main-branch toolchain.

@taylorswift

i should also note that if you already are using macros in your project, you will have a different probem, which is that swift-testing depends on version 509 and not 510, which will cause a dependency conflict. this conflict will be propogated to anyone using your project as well.

We've actually updated to swift-syntax 5.10 on our main branch and our next tag will use it. That tag should become available soon.

@taylorswift

i don’t really have crazy expectations for tooling integrations in Swift, i’m generally happy as long as sourcekit-lsp works.

My colleague @bnbarham knows a lot more about SourceKit-LSP than I do; I've asked him to stop by and reply. Regarding VSCode integration in general: we're only just starting to look at it, so it's not surprising the experience isn't stellar yet. For now, we recommend using Swift Package Manager from the command line.

@taylorswift

running tests did not work for me at all; it crashes and you have to wait for backtracing to complete to get the terminal back.

This crash is new to us as we regularly test on Ubuntu 22. It does not reproduce on the main swift-testing branch as we've refactored the code that's causing a problem, so I expect our next tag (as mentioned above) will resolve the issue. If you're able to reliably reproduce the crash with the next tagged version of swift-testing, please file an issue. Thanks!

@taylorswift

i considered that this might be due to the XCTest behavior described in that section’s documentation, but the --disable-xctest option mentioned there does not appear to exist at all, at least on Swift 5.10:

As discussed previously, Swift Package Manager integration is not available in Swift 5.10 or earlier. You will need to use a nightly main-branch Swift toolchain in order to use --enable-experimental-swift-testing or --disable-xctest (or any of the other SwiftPM changes we've made to add experimental support for swift-testing.)