Testing with JMockit

Testing with JMockit

Drill is built as a highly-coupled network of classes. That is, each part of Drill directly depends on many other parts. This structure, in part, reflects the complexity of the problem that Drill solves, but the resulting tight coupling works against us when testing components in isolation. (The very definition of tight coupling is that components cannot be easily isolated.)

A solution to this problem is "mocking": the idea that we substitute an "artificial" dependency for the real one. Drill uses the Mockit framework for this purpose.

Consider a test such as org.apache.drill.exec.expr.ExpressionTest which requires a RecordBatch to test each expression. (In Drill, we use the data itself, rather than the schema, to get the definition of columns.) Building a RecordBatch is complex and requires building up a set of value vectors and other machinery. Yet, all the test needs is to return the type of a single field. This is a perfect case for mocking. Some tests need the RecordBatch} but never use it. Those tests look like this:

  @Test
  public void testBasicExpression(@Injectable RecordBatch batch) throws Exception {
    checkExpressionCode("if(true) then 1 else 0 end", batch, "/code/expr/basicExpr.txt");
  }

Others need a single method to return the correct (or "expected") value:

  @Test
  public void testExprParseUpperExponent(@Injectable RecordBatch batch) throws Exception {
    final TypeProtos.MajorType type = Types.optional(MinorType.FLOAT8);
    final TypedFieldId tfid = new TypedFieldId(type, false, 0);

    new Expectations( ) {{
      batch.getValueVectorId(new SchemaPath("$f0", ExpressionPosition.UNKNOWN)); result=tfid;
    }};

    checkExpressionCode("multiply(`$f0`, 1.0E-4)", batch, "/code/expr/upperExponent.txt");
  }

Mocking is a quick and easy way to focus on one bit of code (the expression parser) without doing a bunch of work to set up other parts that are not actually needed. The mechanism is, however, not entirely obvious, so you'll want to spend time reading up on JMockit and looking at the existing examples in Drill.

JMockit Versions

JMockit is currently on version 1.29, but Drill still uses version 1.3. There are quite a few differences in the versions. The largest is how the Expectations mechanism works.

The JUnit @Before Annotation and JMockit Expectations

The latest JMockit defines two kinds of expectations: strict and non-strict. The default is non-strict with at-least-once semantics for each mocked method. That is, JMockit will verify that each of the mocked methods you define is called at least once. There are many tests, however, that set up a "generic" DrillbitContext with methods that may be called 0, 1 or more times. For this, we use the JMockit minTimes field set to 0. However, JMockit allows the use of 0 only in code called from a @Before block. (See TestSimpleProjection for an example.) The solution is to use the following idiom:

public class TestSimpleProjection extends ExecTest {
  private @Injectable DrillbitContext bitContext;
  
  @Before
  public void init( ) throws Exception {
    mockDrillbitContext(bitContext);
  }

  @Test
  public void myTest() {
    // do something here with bitContext...

That is, use the mocked object as a test class field rather than as an injected method parameter.

Dummy Objects

Many tests require a UserClientConnection object, but don't actually use it for anything. Again, consider TestSimpleProjection for an example. In this case, we can mock the object, but not set up any expectations (since no methods will be called):

  @Test
  public void project(@Injectable UserClientConnection connection) throws Throwable {
    ...
    final FragmentContext context = new FragmentContext(..., connection, ...);

References

• JMockit Tutorial - A comprehensive guide to the purpose and operation of JMockit.
• Getting Started - More detail.
• Mocking - Detail on the various forms of mocking.
• Development History - JMockit changes quickly. Drill has been on a very old version.
• Advanced JMockit - Shows how to set up mocks in the JUnit @Before block.
• JMockit Code Coverage - Not something Drill currency uses, but work a look.

Using JMockit with Eclipse

Using JMockit is a bit fussy. See the Getting Started page for info. See also the Development Tips page for properly setting up Eclipse.

One problem encountered is that Eclipse itself may be set up to run using a different Java version than the application. When this occurs, you will get very unhelpful error messages that the required test runner class can't be found, and/or Java version errors.

java.lang.NoClassDefFoundError: org.junit.runner.Runner
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:61)
        ...

This is more and more likely: Java 8 is has been the current Java version for quite some time, but Drill still builds with Java 7. If you use Java 8, you will get conflicts. To work around the issue, standardize on a single version. For Java 8, change the drill-root/pom.xml file as follows:

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <configuration>
          <source>1.8</source>
          <target>1.8</target>
          ...

That is, change "1.7" to "1.8". This won't be necessary once Drill itself upgrades.