Code coverage with WMO

Hey everyone,

A few years ago I filed a few radars related to code coverage + WMO (or at least it seems like these in combination is the issue):

http://www.openradar.me/42562585
http://www.openradar.me/42564768

And while both of these still appear to be issues, my bigger question at this point is should code coverage be accurate in general with WMO? Having a cursory (and uniformed) look at the compiler code around this area (SILProfiler.cpp / SILCoverageMap.cpp) I don't see anything obvious that makes it look like these are incompatible, and since they don't error when you pass -whole-module-optimization alongside -profile-generate -profile-coverage-mapping I assume they should work together, but I'm wondering what the expectation is here.

Thanks!

4 Likes

I've spent a bit more time looking into this and for the case of Malformed instrumentation profile data I believe this is caused by functions that are stripped out of the binary because they are unused (which I guess only becomes apparent with -whole-module-optimization since these are -Onone builds?) not being found in the coverage data, which results in an empty function name being returned here https://github.com/apple/llvm-project/blob/4d3a9e1f354b2f67c52239048afc8e7bc29f4420/llvm/include/llvm/ProfileData/InstrProf.h#L569 which turns into this failure mode https://github.com/apple/llvm-project/blob/4d3a9e1f354b2f67c52239048afc8e7bc29f4420/llvm/lib/ProfileData/Coverage/CoverageMappingReader.cpp#L435-L436

The interesting thing is that if you alter llvm-cov to return some bogus values here, coverage data appears to be correct for files where it correctly marks those functions as not being covered.

My (again uniformed) assumption is that the functions that are stripped out shouldn't end up in the coverage data at all but they are because of the order of operations?

Any advice on how to debug more would definitely be appreciated!

2 Likes

What we have observed is, WMO+ Code Coverage is working with XCode11.4; the coverage value is increased though (compared to single file mode). Probably because as you explained above, unused methods are stripped.

1 Like

Brief update here, the Malformed instrumentation profile data case is fixed as of Xcode 11.4 / Swift 5.2, but if you're using -num-threads with -wmo you still end up missing files in coverage data. I'm currently trying to reduce this a bit.

I believe this change fixes the final known issue here https://github.com/apple/swift/pull/32216

2 Likes

I've seen something similar when trying to run llvm-cov on a SwiftPM project using xcodebuild. When running llvm-cov I am seeing Failed to load coverage: Malformed coverage data. I'm using Xcode 11.5 and Apple Swift version 5.2.4 (swiftlang-1103.0.32.9 clang-1103.0.32.53).

Is this just something that will be fixed #32216 is deployed?

As far as I knew that issue was fixed in Swift 5.2 like I said above, so it would be useful if you could try to isolate it. Also if it's easy enough trying a master snapshot which will have #32216 might be worth it.

Terms of Service

Privacy Policy

Cookie Policy