Line coverage report using gcov/lcov

When using unit-tests to add robustness to development projects, it is nice (even important) to be able to see which parts of your code is actually being tested.

If you are using gcc, this is exceedingly easy to accomplish, using the built in tool gcov. All you need to do is use the compile flags -fprofile-arcs -ftest-coverage and link flags -lgcov -fprofile-arcs when compiling with gcc.

This creates a .gcno file for each compilation unit gcc processes. These files don't contain any actual coverage information (which makes sense, since no code has yet been executed). They only keep some basic data on code blocks and source code line numbers.

The key is that when a program built with gcov is executed, it creates .gcda files, one for each compilation unit. The .gcda-files contain the interesting coverage data -- which lines were touched, how many times, and which branches it went through. With the help of lcov, these files are processed to generate an html-page.

This scribble is divided into three parts:
1. A general outline of the steps to create a clean coverage report, which should apply to any gcc project.
2. How I arrived at these steps (cheating a bit for brevity)
3. Finally I'll summarize the working script, an implementation of the general outline.

1. General outline

  1. Clean project completely (you want a complete rebuild)
  2. Recompile everything with gcov compile and link flags.
  3. Use lcov to process initial .gcno files.
  4. Run all tests.
  5. Use lcov to process .gcna coverage files.
  6. Merge the initial and final lcov files (step 3 and step 5).
  7. Filter lcov output to only include project files.
  8. Filter lcov output to exclude unit test files, and main.cpp.
  9. Generate html coverage report from final filtered lcov output..

The point of step 3 and step 6 is to include files that have no coverage at all, and don't get a corresponding .gcna file. Without this, these untouched files wouldn't show up in the final coverage report.

2. Trial and error

First, a quick note on my SCons setup: All source files are contained in the ./src/ directory. During the build process, all used source code files are copied to a ./build/ directory, and all generated files (.o, .gcno, etc) also end up here.

This completely separates the build and src directories, keeping a clean shop, just how I like it.

I should also mention that my SCons script looks for environment variables CPPFLAGS and CCFLAGS, and uses these when present. So that step 1 and step 2 from the general outline become:

2.1 First attempt

Let's skip step 3 for now, and run the tests (step 4), process the coverage output (step 5), and generate the html report (step 9):

The result is quite messy. For some reason, it includes coverage on boost, cxxtest and gcc-stl source files.

2.2 Second attempt - Excluding external files
 Let's clean this up with some filtering that only includes files in the project (step 7).

Much better. It still incorrectly lists directories that don't exist. Also, when browsing to the individual source files, it only shows the error message(missing file).

2.3 Third attempt - Specifying source directory

Turns out that lcov needs to be told what the base directory is, using the -b flag.

Almost there. Unit test files are still included in the coverage report. Since these will always have 100% coverage, it's better to not include them.

2.4 Fourth attempt - Excluding unit test files

Above we extracted our project files, now let's further remove all Test*.* files, which belong to the unit tests (step 8 ).

Only relevant data now. However, I know that there are several files that are not tested at all, and don't show up. Even a whole directory (audio) is missing.

2.5 Fifth attempt - Including static .gcno files

This is where step 3 and step 6 save the day. By using the-i flag it looks for the initial static .gcno files. If you remember, these were created when compiling, and there is one for each compilation unit. We merge this output with the one from running the tests, and filter the result like before.

Finally, it shows my hitherto sloppy test coverage in all earnestness.

2.6 Sixth (final) attempt - Merging folders

The /build/ folder is just a by-product of compilation, a temporary mirror of files in the /src/ directory. It would be nice to pretend that the coverage output for the files in/build/ were located in /src/, merging together with the ones already there. How can that be done? I'm glad you asked!

That's right, a simple search-and-replace does the trick!

3. Summary - final script

Note: As of lcov version 1.10, you can use --no-external in step 5and thus skip step 7.