Make test discovery on by default

Swift 5.1 introduced test discovery to allow tests to be discovered on Linux. This works great but also means that if you use swift build having deleted the LinuxMain.swift it will fail to build without the --enable-test-discovery flag.

Can we make --enable-test-discovery on by default going forward?

2 Likes

swift build shouldn’t fail if LinuxMain.swift is absent. That sounds like a bug. But yeah I think we can make test discovery on by default. Maybe we should emit a warning if there’s a LinuxMain.swift file present unless the flag is explicitly passed.

3 Likes

Doesn't swift build also build the test executable as well, so it needs either the test discovery flag or LinuxMain.swift?

+1 to making --enable-test-discovery on by default.

No, swift build only builds non test targets by default.

It has been finding everything flawlessly for me so far.

I can confirm this. It is the build plan assembly that fails, not compilation. The exact message reads:

error: missing LinuxMain.swift file in the Tests directory

In particular, this is where the error fires.

A completely empty file named LinuxMain.swift is sufficient to suppress the error. None of the XCTestManifests.swift files are necessary.

1 Like

I might propose that, for the moment, rather than having an empty LinuxMain.swift it might be better to have one that contains an explanatory fatalError. That will ensure both that users who forget to use —enable-test-discovery will get a helpful message and that accidentally configuring a CI tool that forgets to pass this flag won’t lead to a quiet success.

5 Likes

Yes, please. +1.

@Aciid bumping this - what's the best way to get this progressed?

1 Like

There are some subtle behaviour differences w/ automatic test discovery but IMO it’s worth changing the default. Maybe someone should come up with a small proposal that describes what would be the impact of making this change and how you can use the current LinuxMain.swift method if needed? We can also use this opportunity to rename LinuxMain.swift to something more platform neutral..

I'll write something up!

3 Likes

Awesome!

@Aciid here you go!