Test discovery on Linux

I've been slowly working on test discovery for Linux and I think the implementation is in a reasonable state now. I've tested it on some OSS Swift packages but it would be great if others can try it out and report any issues they run into. It can be enabled by passing --enable-test-discovery flag to the swift test invocation.

The toolchain is available here.

Known issues:

  • Running tests in release mode requires passing --enable-index-store flag.
18 Likes

Would would be really useful is being able to do a diff between user defined linuxmain files and what is auto generated and in this way I have more confidence that I can just delete all the manual files.

You can diff the output of swift test --list-tests and swift test --enable-test-discovery --list-tests.

You can also see the generated test list in .build/x86_64-unknown-linux/debug/testlist.derived if you want to do a manual inspection.

6 Likes

This is really amazing Ankit, thanks a lot for publishing the toolchain too.

I just tried on some projects I have laying around and found no issues in discovering tests in any of them, it correctly discovered all of our tests :heart:


I have discovered one difference though; which we actually do not rely on, but perhaps someone could / does, so might as well mention the case right away, even if we'd e.g. ship a first version without supporting that (which would already be tremendously useful, as I don't know if people actually do this base class "trick"):

Structure:

  • class CommonTests: XCTestCase { func test_common() {} }
  • final class SpecificOneTests: CommonTests {}

On mac this discovers the SpecificOneTests.test_common as well as the one in the base class, the current --enable-test-discovery does not discover the one in the SpecificOneTests.

Having that said, I'm not sure how critical this is, since I've not seen it used, but I could be totally mistaken and people could expect it to work. Even the existing PRed discovery would already be amazing and I'd love to be able to use it in normal swift builds :slight_smile:

I created a docker image on Docker Hub as norionomura/swift:pr-25685 that supports --enable-test-discovery.

5 Likes

Ah, yeah. This is a little complex to implement in the current approach but it should be technically possible. I don't think this is a common enough scenario though and it is easy to workaround (by just explicitly writing a new test method in the subclass and then calling the method in the super class).

1 Like

Yeah, agreed that it's not a show stopper; wanted to make sure we're aware of it (might want to document it).

All in all though this is awesome, can't wait :blush:

thank you for putting this together @Aciid, this is fantastic for server developers

2 Likes
Terms of Service

Privacy Policy

Cookie Policy