Filter Code Coverage

Using Swift
1

In the thread, How is –enable-code-coverage intended to be used? and subsequent PR, a flag --show-codecov-path is introduced.

I have been using it recently and I noticed that the coverage data includes dependencies in my Package.swift file.

Is there a way to filter dependencies out?

Alternatively, I have been playing with going directly to llvm-cov but cannot find what to send as a covered executable or object file that represents my entire module.

For instance, I came up with:

xcrun llvm-cov report -instr-profile $(swift test --show-codecov-path | xargs dirname)/default.profdata .build/debug/ProjectName.build/**/*.o

But that only covers the first .o file in the directory. There are dozens of files that should be included. I think that is because llvm-cov expects a single .o file that contains the whole module; honestly I am not sure.

2

What you already know from those links is all that is built into SwiftPM. Any filtering or interpretation of the results must be done by a separate tool.

This function is how I read in the results back into Swift for processing, ignoring dependencies. If using my package directly doesn’t suit your needs, you may reverse engineer whatever you want instead.

Another module in the package also contains an equivalent method for working with Xcode (because SwiftPM cannot check coverage for iOS, etc).

If you have any further questions, feel free to open an issue over at the package’s GitHub repository.

3

My cursory glance says that you are parsing the JSON and just filtering yourself, is that right?

4

Yes. (Mostly anyway. Some effort has also been made to clean up some quirks in the results.)

5

Someone moved this thread into the "Using Swift" category. I put it where I did because I was gathering information on my way to a proposal/question. Hopefully, the developers of SwiftPM will still see it here. I digress.

Obviously as @SDGGiesbrecht has shown for the specific use case I have, filtering code coverage, a simple JSON parser might be the shortest distance between here and there.

However, I did want to point out that llvm-cov does have a filter feature (among others) that might also be useful to expose. I think I would like to propose making some of the internals of swift package manager available to the swift test tool.

Expose testBinary: AbsolutePath as a flag on swift test

Introduce a new flag, e.g., --show-test-binary-path (can bike shed later), that prints the path to the test binary. This would be the same path that is currently provided to exportCodeCovAsJSON.

Note: It is entirely possible that this flag, or something similar to it already exists and if so, oops.

With the path to the test binary easily accessible developers can build their own interactions with llvm-cov directly. Thus preventing the package manager from having to provide all the different methods/interfaces/formats developers might want to manipulate/visualize coverage data.

For instance, to use llvm-cov to filter dependencies and tests:

xcrun llvm-cov export \
  --ignore-filename-regex='(.build|Tests)[/\\].*' \
  -instr-profile $(swift test --show-codecov-path | xargs dirname)/default.profdata \
  $(swift test --show-test-binary-path)

Or to generate lcov data instead of JSON:

xcrun llvm-cov export \
  --format=lcov \
  -instr-profile $(swift test --show-codecov-path | xargs dirname)/default.profdata \
  $(swift test --show-test-binary-path)

Or just get a tabular ASCII report:

xcrun llvm-cov report \
  -instr-profile $(swift test --show-codecov-path | xargs dirname)/default.profdata \
  $(swift test --show-test-binary-path)
1 Like