JQF Maven Plugin

The JQF maven plugin allows you to fuzz your programs simply using the command mvn jqf:fuzz. This means that you do not have to clone the JQF repo and run shell scripts. Once you have written a JQF test using a dependency on jqf-fuzz for the API, you are ready to use the plugin.

Note: The JQF Maven Plugin launches the Zest engine by default.

Setup

Add this in your pom.xml file:

<build>
    <plugins>
        <plugin>
            <groupId>edu.berkeley.cs.jqf</groupId>
            <artifactId>jqf-maven-plugin</artifactId>
            <version>1.8</version>
        </plugin>
    </plugins>
</build>

Note: You may want to double-check what is the latest released version, since this wiki page may go out of date. Use the version string after the prefix "jqf-" (e.g. if the release is "jqf-1.X" then use "1.X" as version).

Fuzzing

Run the command:

mvn jqf:fuzz -Dclass=<fully-qualified-class-name> -Dmethod=<method-name>

Where the class and method names are for the target you are trying to fuzz. See Writing a JQF test.

For example, in the JQF examples project directory,

mvn jqf:fuzz -Dclass=edu.berkeley.cs.jqf.examples.closure.CompilerTest -Dmethod=testWithGenerator

The results of fuzzing are usually saved in a directory such as target/fuzz-results/<class-name>/<method-name>, but this can be changed (along with many other things such timeouts and which classes to include/exclude from instrumentation). To get a full list of configuration commands, ask Maven to describe the plugin's options:

mvn help:describe -Dplugin=jqf -Ddetail

Reproducing tests

Once the fuzzing stops due to a timeout or a CTRL+C, you can reproduce a generated input (such as a failure). Run the command:

mvn jqf:repro -Dclass=<fully-qualified-class-name> -Dmethod=<method-name> -Dinput=<file-or-directory-name>

For example,

mvn jqf:repro -Dclass=edu.berkeley.cs.jqf.examples.closure.CompilerTest -Dmethod=testWithGenerator -Dinput=target/fuzz-results/edu.berkeley.cs.jqf.examples.closure.CompilerTest/testWithGenerator/corpus/id_000028

or

mvn jqf:repro -Dclass=edu.berkeley.cs.jqf.examples.closure.CompilerTest -Dmethod=testWithGenerator -Dinput=target/fuzz-results/edu.berkeley.cs.jqf.examples.closure.CompilerTest/testWithGenerator/corpus

Configuration Options

To list all the flags supported by the JQF Maven Plugin, run:

mvn help:describe -Dplugin=jqf -Ddetail

Flags common to jqf:fuzz and jqf:repro

• -Dexcludes=<list>: Comma-separated list of fully-qualified class-name prefixes to exclude from coverage instrumentation (e.g. org.apache.commons). Default is empty.
• -Dincludes=<list>: Comma-separated list of fully-qualified class-name prefixes to forcibly include, even if they match an exclude (e.g. org.apache.commons.io). Default is empty.

Flags only supported by jqf:fuzz

• -Dtime=<timeout>: Stop fuzzing after a timeout. Examples of timeouts include 30s, 60m or 24h. Default is empty, implying no timeout.
• -DrunTimeout=<timeout>: Sets a timeout on a single trial. This is useful for catching tests that may hang in infinite or very long loops.
• -DexitOnCrash: Stop fuzzing if a failure is encountered. Useful if you only care about finding the existence of failure and not necessarily generating many failing test cases. Also useful for debugging fuzzing runs where the first input appears to fail.
• -Dblind: Do not save any inputs during fuzzing. Every input is generated randomly from scratch instead of by mutation. This is equivalent to running only junit-quickcheck, as it does not rely on a feedback-directed search.
• -DnoCov: Do not instrument classes for code coverage. This flag only makes sense with when -Dblind is also enabled, since coverage-guided fuzzing cannot work without instrumentation. When this flag is set, the speed at which inputs are generated is significantly higher for blind fuzzing, but no coverage numbers are reported on the status screen.
• -DsaveAll: Save ALL inputs ever generated during fuzzing in a subdirectory called all in the output directory. This flag will lead to a very large number of files being created, and could potentially slow down the fuzzing throughput.
• -Dquiet: Do not show status screen while fuzzing.
• -Dtarget=<dir>: Change the root directory for results (default: target)
• -Dout=<dir>: Name of subdirectory (within target) for storing results (default: fuzz-results/$TARGET_CLASS/$TARGET_METHOD).
• -Din=<dir>: Name of directory containing inputs to initialize the fuzzing queue with. Can be used for seed inputs or for resuming a fuzzing session. Default is empty, meaning fuzzing starts with randomly generated seed inputs.

• -DlibFuzzerCompatOutput: Make the output conform to libFuzzer, for use in feeding into other engines that process fuzzer output (e.g. visualization tools).
• -DfixedSize: Make the fuzzer treat inputs as fixed-sized array of bytes, instead of dynamically resizable input streams. This is typically only useful when fuzzing programs that process arbitrary length InputStream objects.

Flags only supported by jqf:repro

• -DlogCoverage=<file>: Output file to dump coverage info (e.g. coverage.out). Default is empty, implying no dump file.
• -DprintArgs: Print arguments to test method, after running the input file through junit-quickcheck generators (if any), before running the test method. Arguments are serialized by calling the toString() method on each argument object.
• -DdumpArgs=<dir>: Like -DprintArgs, but writes out generated arguments (converted to strings) in files in the given directory. A suffix of .<idx> is added to the basename of the repro'd input file, where <idx> is the index of the argument in the test method (e.g. foo.0 is the first generated argument when repro'ing an execution with saved input foo).