Code coverage with WMO

Development Compiler
1

Hey everyone,

A few years ago I filed a few radars related to code coverage + WMO (or at least it seems like these in combination is the issue):

http://www.openradar.me/42562585
http://www.openradar.me/42564768

And while both of these still appear to be issues, my bigger question at this point is should code coverage be accurate in general with WMO? Having a cursory (and uniformed) look at the compiler code around this area (SILProfiler.cpp / SILCoverageMap.cpp) I don't see anything obvious that makes it look like these are incompatible, and since they don't error when you pass -whole-module-optimization alongside -profile-generate -profile-coverage-mapping I assume they should work together, but I'm wondering what the expectation is here.

Thanks!

3 Likes
2

I've spent a bit more time looking into this and for the case of Malformed instrumentation profile data I believe this is caused by functions that are stripped out of the binary because they are unused (which I guess only becomes apparent with -whole-module-optimization since these are -Onone builds?) not being found in the coverage data, which results in an empty function name being returned here llvm-project/InstrProf.h at 4d3a9e1f354b2f67c52239048afc8e7bc29f4420 · apple/llvm-project · GitHub which turns into this failure mode llvm-project/CoverageMappingReader.cpp at 4d3a9e1f354b2f67c52239048afc8e7bc29f4420 · apple/llvm-project · GitHub

The interesting thing is that if you alter llvm-cov to return some bogus values here, coverage data appears to be correct for files where it correctly marks those functions as not being covered.

My (again uniformed) assumption is that the functions that are stripped out shouldn't end up in the coverage data at all but they are because of the order of operations?

Any advice on how to debug more would definitely be appreciated!

2 Likes
3

What we have observed is, WMO+ Code Coverage is working with XCode11.4; the coverage value is increased though (compared to single file mode). Probably because as you explained above, unused methods are stripped.

1 Like
4

Brief update here, the Malformed instrumentation profile data case is fixed as of Xcode 11.4 / Swift 5.2, but if you're using -num-threads with -wmo you still end up missing files in coverage data. I'm currently trying to reduce this a bit.

5

I believe this change fixes the final known issue here Emit coverage mappings for all modules by keith · Pull Request #32216 · apple/swift · GitHub

2 Likes
6

I've seen something similar when trying to run llvm-cov on a SwiftPM project using xcodebuild. When running llvm-cov I am seeing Failed to load coverage: Malformed coverage data. I'm using Xcode 11.5 and Apple Swift version 5.2.4 (swiftlang-1103.0.32.9 clang-1103.0.32.53).

Is this just something that will be fixed #32216 is deployed?

7

As far as I knew that issue was fixed in Swift 5.2 like I said above, so it would be useful if you could try to isolate it. Also if it's easy enough trying a master snapshot which will have #32216 might be worth it.

8

Unfortunately while trying to reduce it. I "fixed" it.

The project with the issue relies on BigInt. The BigInt type was just cut-n-paste into the module. After removing that code the coverage is produced without incident.

Unfortunately it's beyond my skillset to be able to reduce it and still exemplify the crash. Thanks for the feedback though.