Skip to content

Latest commit

 

History

History
171 lines (135 loc) · 5.73 KB

README.md

File metadata and controls

171 lines (135 loc) · 5.73 KB
Raw

Maven plug-in

The GZoltar Maven plug-in provides a GZoltar runtime agent to collect the coverage of the project's test cases and allows the creation of a text-based and a html-based fault localization report. The project's test cases are run as usually, i.e., via Maven Surefire, which allows GZoltar Maven plug-in to support JUnit3, JUnit4, and TestNG test cases out-of-the-box.

In order to be able to collect any coverage, the target classes must be compiled with debug information.

When using the maven-surefire-plugin to run the project's test cases, the option forkCount should not be set to 0 and the option forkMode should not be set to never. Otherwise, the project's test cases will not be executed with javaagent and therefore no coverage would be collected.

To receive a full list of goals and available parameters you can invoke maven-help-plugin as:

mvn help:describe -Dplugin=com.gzoltar:com.gzoltar.maven -Ddetail

A simple working Maven project can be found in here.

Configuring the coverage collection

To use GZoltar's Maven plug-in, it first needs to be configured in the pom.xml of the target project. For example:

<plugin>
  <groupId>com.gzoltar</groupId>
  <artifactId>com.gzoltar.maven</artifactId>
  <version>1.7.2</version>
  <executions>
    <execution>
      <id>pre-unit-test</id>
      <goals>
        <goal>prepare-agent</goal>
      </goals>
    </execution>
  </executions>
</plugin>

Besides including the plug-in in the pom.xml file, you also need to configure the maven-surefire-plugin plug-in to instantiate a GZoltar's listener to collect the coverage of each unit test case.

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.20</version>
  <configuration>
    <properties>
      <property>
        <name>listener</name>
        <value>${GZoltarListener}</value>
      </property>
    </properties>
  </configuration>
</plugin>

Where ${GZoltarListener} can either be com.gzoltar.internal.core.listeners.JUnitListener for JUnit test cases, or com.gzoltar.internal.core.listeners.TestNGListener for TestNG test cases.

By default, GZoltar (1) instruments all classes loaded by any test case and located inside ${project.build.directory}, (2) collects coverage at line level during the execution of all existing unit test cases, and (3) serializes all data to ${project.build.directory}/gzoltar.ser when the JVM terminates.

Configuring the report generation

Once the coverage has been collected, different types of fault localization reports can be generated by GZoltar. In order to do it, GZoltar's Maven plug-in has to be configured as follows:

<plugin>
  <groupId>com.gzoltar</groupId>
  <artifactId>com.gzoltar.maven</artifactId>
  <version>1.7.2</version>
  <executions>
    <execution>
      <id>post-unit-test</id>
      <phase>test</phase>
      <goals>
        <goal>fl-report</goal>
      </goals>
    </execution>
  </executions>
</plugin>

By default, GZoltar generates two Spectrum-based Fault Localization reports (a text-based report and a html-based report) for the following fault localization formulas:

  • Barinel
  • DStar
  • Ochiai
  • Tarantula

to a default directory ${project.build.directory}/site/gzoltar. For instance the text-based report for the Ochiai formula can be found in ${project.build.directory}/site/gzoltar/sfl/txt/ochiai.ranking.csv and the html-based report for the same formula in ${project.build.directory}/site/gzoltar/sfl/html/ochiai/sunburst.html.

In ${project.build.directory}/site/gzoltar/sfl/txt/ four additional files can also be found:

  • matrix.txt - a binary coverage matrix produced by GZoltar where each row represents the coverage of each individual test case and its outcome ("-" if the test case failed, "+" otherwise), and each column a component (e.g., line of code). 1 means that a test case covered a component, 0 otherwise.
  • spectra.csv - a list of all components identified by GZoltar (one per row) of all instrumented classes.
  • tests.csv - a list of all test cases listened by GZoltar's listener (one per row).
  • statistics.csv - a set of metrics (default: matrix density, component ambiguity score, and entropy) for each fault localization formula. Each row represents a metric of a formula (e.g., the entropy of the ranking returned by Ochiai formula). Thus, the total number of rows in this file is number of fault localization formulas * number of metrics.

Usage

To run the project's test cases, collect their coverage, and generate a fault localization report, you just need to execute the following command:

mvn test

Caveats

JUnit3 and JUnit4 Compatibility

Although GZoltar's Maven plug-in works for both JUnit3 and JUnit4 test cases, please make sure that the JUnit dependency included in the project's pom.xml is at least version 4.6.

This requirement is due to the fact that JUnit only provides their org.junit.runner.notification.RunListener API after version 4.6. The listener API is used so that per unit test coverage can be gathered. For JUnit3 test cases, the appropriate test runner will still be used.

argLine

If there is an argLine parameter set in the declaration of maven-surefire-plugin, that will override GZoltar's Maven plug-in request to add an agent to the test JVM. To circumvent this, you can add your JVM options in GZoltar's Maven plug-in argLine parameter.

restrictOutputDirectory

To know the location of each class at runtime, GZoltar uses the java.security.ProtectionDomain API. Depending on the security policy being enforced, access to that specific API may be blocked.