MoFuzz: A Fuzzer Suite for Testing Model-Driven Software Engineering Tools

This repository provides the implementation and evaluation subjects for the paper MoFuzz: A Fuzzer Suite for Testing Model-Driven Software Engineering Tools accepted for the research track of ASE'2020.

MoFuzz utilizes coverage-guided fuzzing and automated model generation to test Model-Driven Software Engineering (MDSE) tools.

Authors: Hoang Lam Nguyen, Nebras Nassar, Timo Kehrer, and Lars Grunske

Installation

MoFuzz is built on top of JQF: a feedback-directed fuzz testing platform for Java. We provide instructions to install and run MoFuzz locally or inside a Docker container:

Setup locally

Requirements

• Git, Maven
• Java JDK = 1.8

1. Clone repository:

git clone https://github.com/hub-se/MoFuzz.git

2. Build MoFuzz

cd MoFuzz/mofuzz
mvn package
cd ..

Setup as Docker container

Requirements

• Docker

1. Clone repository:

git clone https://github.com/hub-se/MoFuzz.git

2. Build container:

cd MoFuzz
docker build -t mofuzz .

3. Run container:

docker run -dt --name=mofuzz-container mofuzz
docker exec -it mofuzz-container /bin/bash

Running MoFuzz

After finishing the setup as described above, MoFuzz can be executed using one of the scripts inside the scripts/ folder. The following input generation strategies are available (for detailed descriptions please check out the paper):

1. MoFuzz-emf-modelgen (scripts/mofuzz-emf-modelgen.sh): black-box, graph-grammar based
2. MoFuzz-cgf-emfedit (scripts/mofuzz-cgf-emfedit.sh): coverage-guided, mutation-based
3. MoFuzz-cgf-cpeo (scripts/mofuzz-cgf-cpeo.sh): coverage-guided, rule-based
4. Random (scripts/random_instantiator.sh): random, containment-tree based
5. Zest (scripts/zest.sh): coverage-guided, containment-tree based

The scripts are used as follows:

./scripts/selected_strategy.sh TEST_CLASS TEST_METHOD

We provide the following subjects from the original evaluation of MoFuzz:

Name	TEST_CLASS	TEST_METHOD
UML2Java	AcceleoUML2JavaHeliosTest	simpleGeneratorTest
EcoreUtil	EcoreUtilsTest	completeTest
EMFCompare	EMFCompareTest	diffTest
UMLValidator	UML2ValidatorTest	test
UML2OWL	UML2OWLTest	test
EMF2GraphViz	EMF2GraphvizTest	test

For example, running MoFuzz using the random strategy on the EcoreUtil subject results in the following command:

./scripts/random_instantiator.sh EcoreUtilsTest completeTest

After some initialization time, the output should look like this:

Coverage-guided Modelfuzzing
--------------------------

Test name:            fr.inria.atlanmod.instantiator.benchmarks.EcoreUtilsTest#completeTest
Results directory:    /workspace/MoFuzz/evaluation/results/random_instantiator/EcoreUtilsTest_completeTest
Elapsed time:         20s (no time limit)
Number of executions: 82
Valid inputs:         82 (100.00%)
Cycles completed:     0
Unique failures:      0
Queue size:           0
Current parent input: <seed>
Execution speed:      6/sec now | 3/sec overall
Total coverage:       1,211 branches (1.85% of map)

Running MoFuzz this way results in an infinite fuzzing loop and must be manually aborted (CTRL+C).

To prevent this, a timeout can be specified using the timeout command:

timeout 3600s ./scripts/mofuzz-emf-modelgen.sh UML2ValidatorTest test

Detailed evaluation results (log data and coverage stats over time) can be found in the following subdirectory: evaluation/results.