1

Has there been any thought put into generating code coverage reports for
the compiler? Or, are there some known steps community members could take
to help out (such as adding a flag to build-script which enables
-fprofile-arcs -ftest-coverage)?

I think it'd be really interesting to see what's covered by the
(validation) tests and what isn't.

Jacob

2

Luke Larson and I added a few things to support testing code coverage of the compiler.

You can build with

build-script -t --swift-analyze-code-coverage merged

which will rebuild the compiler with coverage instrumentation and run all the tests you've selected with coverage enabled.

Once all the tests are done, you'll get a set of .profraw files in the swift-test-results directory. Just merge them together with

llvm-profdata merge *.profraw -o swiftc.profdata

and you can use llvm-cov to inspect that data.

Note: you'll need to build swift with an upstream version of llvm and clang, because there's a very recent 'pooled' coverage system that we use.

You'll probably see that coverage is really low, somewhere around ~50%. It's that low because the coverage technically considers _all files_, including those from llvm and clang. Those files are, naturally, not tested by the Swift test suite, so they contribute a significant amount of 'un-covered' areas.

When we isolate the Swift compiler's files, I found that coverage was somewhere around 79.45%

If you need any help, let me know!

-- Harlan

···

On Aug 25, 2016, at 2:46 AM, Jacob Bandes-Storch via swift-dev <swift-dev@swift.org> wrote:

Has there been any thought put into generating code coverage reports for the compiler? Or, are there some known steps community members could take to help out (such as adding a flag to build-script which enables -fprofile-arcs -ftest-coverage)?

I think it'd be really interesting to see what's covered by the (validation) tests and what isn't.

Jacob
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

1 Like
3

I wanted to try running the code coverage for the compiler but have been hitting a wall
Any clues would be helpful

/Applications/CMake.app/Contents/bin/cmake --build /Users/viranchee/appleSwift/build/Ninja+cmark-DebugAssert+llvm-DebugAssert+swift-DebugAssertCoverage+stdlib-DebugAssert/swift-macosx-arm64 -- -j10 all swift-test-stdlib-macosx-arm64 swift-libexec-macosx-arm64 swift-benchmark-macosx-arm64

[1/759] Linking CXX shared library lib/libswiftDemangle.dylib
FAILED: lib/libswiftDemangle.dylib 
: && /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -Wno-unknown-warning-option -Werror=unguarded-availability-new -fPIC -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wimplicit-fallthrough -Wcovered-switch-default -Wno-class-memaccess -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported -index-store-path "/Users/viranchee/appleSwift/build/Ninja+cmark-DebugAssert+llvm-DebugAssert+swift-DebugAssertCoverage+stdlib-DebugAssert/swift-macosx-arm64/IndexStore" -fdiagnostics-color -Werror=switch -Wdocumentation -Wimplicit-fallthrough -Wunreachable-code -Woverloaded-virtual -DOBJC_OLD_DISPATCH_PROTOTYPES=0 -g -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk -dynamiclib -Wl,-headerpad_max_install_names -Wl -lobjc   -target arm64-apple-macosx10.13 -Xlinker -compatibility_version -Xlinker 1 -o lib/libswiftDemangle.dylib -install_name @rpath/libswiftDemangle.dylib lib/SwiftDemangle/CMakeFiles/swiftDemangle.dir/SwiftDemangle.cpp.o lib/SwiftDemangle/CMakeFiles/swiftDemangle.dir/MangleHack.cpp.o -L/Users/viranchee/appleSwift/build/Ninja+cmark-DebugAssert+llvm-DebugAssert+swift-DebugAssertCoverage+stdlib-DebugAssert/llvm-macosx-arm64/./lib   -L/Users/viranchee/appleSwift/build/Ninja+cmark-DebugAssert+llvm-DebugAssert+swift-DebugAssertCoverage+stdlib-DebugAssert/swift-macosx-arm64/lib   -L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/lib/swift   -L/Users/viranchee/appleSwift/build/Ninja+cmark-DebugAssert+llvm-DebugAssert+swift-DebugAssertCoverage+stdlib-DebugAssert/swift-macosx-arm64/./lib/swift/macosx   -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx -Wl,-rpath,/usr/lib/swift  lib/libswiftDemangling.a && :
Undefined symbols for architecture arm64:
  "___llvm_profile_runtime", referenced from:
      ___llvm_profile_runtime_user in SwiftDemangle.cpp.o
      ___llvm_profile_runtime_user in MangleHack.cpp.o
      ___llvm_profile_runtime_user in libswiftDemangling.a(Demangler.cpp.o)
      ___llvm_profile_runtime_user in libswiftDemangling.a(NodePrinter.cpp.o)
      ___llvm_profile_runtime_user in libswiftDemangling.a(Context.cpp.o)
      ___llvm_profile_runtime_user in libswiftDemangling.a(Remangler.cpp.o)
      ___llvm_profile_runtime_user in libswiftDemangling.a(NodeDumper.cpp.o)
      ...
     (maybe you meant: ___llvm_profile_runtime_user)
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
[10/759] Building CXX object lib/SIL/CMakeFiles/swiftSIL.dir/IR/SILDeclRef.cpp.o
ninja: build stopped: subcommand failed.
4

Hi @Viranchee,

This is a thread from the last decade about incorporating features into the compiler. Please post any questions about using Swift in a new, separate thread under "Using Swift," and any issues with compiling the compiler itself in a new thread under the "Development" category. Thanks!

6

Hi @xwu I want to generate code coverage on the Swift Compiler itself