Generating Code Coverage for a section of the compiler

Development Compiler
1

Hi, I recently wanted to analyze the code coverage for AutoDiff section of the compiler codebase.
These are the steps I took:

  1. Generate a regular build with build-script. No tests.
  2. Edit build/folder/swift-macosx-arm64/CMakeCache.txt by adding these flags:
    CMAKE_CXX_FLAGS:STRING = ... -fprofile-instr-generate -fcoverage-mapping
    CMAKE_Swift_FLAGS:STRING= -profile-generate -profile-coverage-mapping
  3. ReRun the same build command as in step 1. which will build profiled version of compiler, generating *.profraw files
  4. Run tests with a filter, eg. lit.py tests --filter AutoDiff to generate coverage report for only certain sections of the Swift compiler (and not exhaust storage capacity too)
  5. Merge together the generated profiled data with llvm-cov merge

Steps 4 and 5 result in errors:

Step 4: 40 of 169 AutoDiff tests failed. Errors have been consistent:

Command Output (stderr):
--
Undefined symbols for architecture arm64:
  "___llvm_profile_runtime", referenced from:
      ___llvm_profile_runtime_user in libswiftCompatibilityConcurrency.a(CompatibilityConcurrency.cpp.o)
      ___llvm_profile_runtime_user in libswiftCompatibility56.a(Overrides.cpp.o)
      ___llvm_profile_runtime_user in libswiftCompatibility56.a(Task.cpp.o)
      ___llvm_profile_runtime_user in libswiftCompatibility56.a(TaskLocal.cpp.o)
      ___llvm_profile_runtime_user in libswiftCompatibility56.a(TaskStatus.cpp.o)
      ___llvm_profile_runtime_user in libswiftCompatibility56.a(Error.cpp.o)
      ___llvm_profile_runtime_user in libswiftCompatibility56.a(Actor.cpp.o)
      ...
     (maybe you meant: ___llvm_profile_runtime_user)

Which is similar to the error I get while using Code Coverage Swift flag from this 7 year old feature Code coverage?

And a similar error when running it on an x86 macOS.
Running the full test suite on the instrumented compiler results in ~2-4k test failures.

Step 5: Merging these profiled profraw results in zlib error for 5 files.
viz.

warning: build/inst/swift-macosx-arm64/test-macosx-arm64/AutoDiff/Parse/default.profraw: failed to uncompress data (zlib)
warning: build/inst/swift-macosx-arm64/test-macosx-arm64/AutoDiff/validation-test/default.profraw: failed to uncompress data (zlib)
warning: build/inst/swift-macosx-arm64/test-macosx-arm64/AutoDiff/Sema/default.profraw: failed to uncompress data (zlib)
warning: build/inst/swift-macosx-arm64/test-macosx-arm64/AutoDiff/Serialization/default.profraw: malformed instrumentation profile data
warning: build/inst/swift-macosx-arm64/test-macosx-arm64/AutoDiff/SILGen/default.profraw: failed to uncompress data (zlib)
error: no profile can be merged

So while merging with llvm-cov I ignore these files and merge the other profraw generated with running tests on the profiled compiler. The profraw generated files are always less than 500 MBs.

What can I do to better generate code coverage for certain sections of the compiler?