Code coverage?


(Jacob Bandes-Storch) #1

Has there been any thought put into generating code coverage reports for
the compiler? Or, are there some known steps community members could take
to help out (such as adding a flag to build-script which enables
-fprofile-arcs -ftest-coverage)?

I think it'd be really interesting to see what's covered by the
(validation) tests and what isn't.

Jacob


(Harlan Haskins) #2

Luke Larson and I added a few things to support testing code coverage of the compiler.

You can build with

build-script -t --swift-analyze-code-coverage merged

which will rebuild the compiler with coverage instrumentation and run all the tests you've selected with coverage enabled.

Once all the tests are done, you'll get a set of .profraw files in the swift-test-results directory. Just merge them together with

llvm-profdata merge *.profraw -o swiftc.profdata

and you can use llvm-cov to inspect that data.

Note: you'll need to build swift with an upstream version of llvm and clang, because there's a very recent 'pooled' coverage system that we use.

You'll probably see that coverage is really low, somewhere around ~50%. It's that low because the coverage technically considers _all files_, including those from llvm and clang. Those files are, naturally, not tested by the Swift test suite, so they contribute a significant amount of 'un-covered' areas.

When we isolate the Swift compiler's files, I found that coverage was somewhere around 79.45%

If you need any help, let me know!

-- Harlan

ยทยทยท

On Aug 25, 2016, at 2:46 AM, Jacob Bandes-Storch via swift-dev <swift-dev@swift.org> wrote:

Has there been any thought put into generating code coverage reports for the compiler? Or, are there some known steps community members could take to help out (such as adding a flag to build-script which enables -fprofile-arcs -ftest-coverage)?

I think it'd be really interesting to see what's covered by the (validation) tests and what isn't.

Jacob
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev