Add support to prevent execution

.travis.yml

@@ -3,10 +3,19 @@ language: python
 matrix:
   include:
     - python: "2.7"
+      before_install:
+        - pip install coveralls pylint
+      env: COVERAGE="--with-coverage --cover-package=rebench"
+      after_success:
+        coveralls
     - python: "3.6"
+      before_install:
+        - pip install coveralls pylint
     # PyPy versions
     - python: pypy
+      env: DO_LINT="echo On PyPy, we won't "
     - python: pypy3
+      env: DO_LINT="echo On PyPy, we won't "
 
 dist: trusty
 sudo: false
@@ -17,14 +26,10 @@ addons:
       - time
 
 install:
-  - pip install coveralls pylint
   - pip install .
 
 # command to run tests
 script:
- - nosetests --with-coverage --cover-package=rebench
+ - nosetests ${COVERAGE}
  - (cd rebench && rebench -N ../rebench.conf e:TestRunner2)
- - pylint rebench
-
-after_success:
-  coveralls
+ - ${DO_LINT} pylint rebench

docs/usage.md

@@ -143,6 +143,17 @@ For this purpose we use *schedulers* to determine the execution order.
                         random [default: batch]
 ```
 
+#### Prevent Execution to Verify Configuration
+
+To check whether a configuration is correct, it can be useful to avoid
+execution altogether. For such *testing* or *dry run*, we have the following
+option:
+
+```text
+-E, --no-execution    Disables execution. It allows to verify the
+                      configuration file and other parameters.
+```  
+
 #### Continuous Performance Tracking
 
 ReBench supports [Codespeed][1] as platform for continuous performance

rebench/rebench.py

@@ -116,6 +116,11 @@ def shell_options(self):
             default='batch',
             help='execution order of benchmarks: '
                  'batch, round-robin, random [default: %(default)s]')
+        execution.add_argument(
+            '-E', '--no-execution', action='store_true', dest='no_execution',
+            default=False,
+            help='Disables execution.'
+                 ' It allows to verify the configuration file and other parameters.')
 
         data = parser.add_argument_group(
             'Data and Reporting',
@@ -220,7 +225,11 @@ def execute_experiment(self, runs):
                             self._config.options.debug,
                             scheduler_class,
                             self._config.build_log)
-        return executor.execute()
+
+        if self._config.options.no_execution:
+            return True