Test Coverage

DRAFT

The Gnu Compiler Collection (GCC) comes with the gcov test coverage program, which among others makes it possible to locate untested code. For the Web Service ARC residing in the arc1 trunk, unit tests are carried out when doing a make check. With the help of the gcov tool untested lines of code can easily be located, when running make check. When creating unit tests developers should try to achieve the higest possible test coverage. However, due to the limitations of unit tests not every unit can be tested with this method. Every class in HED should have a unit test associated with it. If a class, method or unit is incapable of being unit tested the reason should be stated in the unit test.

During the creation of nightly packages, a test coverage web page is also generated. View [link test coverage of the latest nightlies].

Howto use the test coverage tool

First the arc1 trunk should be configured to generate test coverage. Before running configure the environment variables CFLAGS, CXXFLAGS and LIBS should be defined as follows:

CFLAGS="-Wall -W -pedantic -O0 -fprofile-arcs -ftest-coverage"
CXXFLAGS="-Wall -W -pedantic -O0 -fprofile-arcs -ftest-coverage"
LIBS=-lgcov

This can be done by running the command below:

CFLAGS="-Wall -W -pedantic -O0 -fprofile-arcs -ftest-coverage"  \
 CXXFLAGS="-Wall -W -pedantic -O0 -fprofile-arcs -ftest-coverage" \
 LIBS=-lgcov ./configure

Now build and run the code you want to generate a test coverage of, e.g. make and make check. When building the code a .gcno file will be generated for every object file. And additionally when running the code a .gcda file is generated of each object file.

After executing the code, the gcov command should be used to generate .gcov files which shows the test coverage. Also when running the gcov command a summary is printed for every file used.

An example might be helpful. Lets generate the test coverage of the XMLNodeTest unit test, i.e. which parts of the XMLNode class code is not executed by the XMLNodeTest unit test, if any.

cd <arc1-trunk>
CFLAGS="-Wall -W -pedantic -O0 -fprofile-arcs -ftest-coverage" \
 CXXFLAGS="-Wall -W -pedantic -O0 -fprofile-arcs -ftest-coverage" \
 LIBS=-lgcov ./configure
cd src/hed/libs/common
make && make check
gcov -o .libs/libarccommon_la-XMLNode.gcno XMLNode.cpp

Which will output something similiar to:

<snip>

File 'XMLNode.cpp'
Lines executed:15.75% of 838
XMLNode.cpp:creating 'XMLNode.cpp.gcov'

<snip>

File 'XMLNode.h'
Lines executed:14.29% of 14
XMLNode.h:creating 'XMLNode.h.gcov'

<snip>

This will generate two files .libs/XMLNode.{cpp,h}.gcov which shows which lines in the associated files which have not been executed, cannot be executed or the number of times the line was executed.

In the .gcov file every line is prepended with one of following set of characters:

• -: line cannot be executed,
• #####: line was not executed,
• <number>: line was executed <number> of times.