feat: simplify the cucumber parser

This change simplifies the cucumber parsers and brings them down to 1 parser.
The docs are also updated with usage examples.

Closes #71

BREAKING CHANGE: cucumberMulti parser can't be used anymore

To migrate change `cucumberMulti` to `cucumber`

docs/cucumber.md

@@ -1,32 +1,105 @@
 # Cucumberjs parser
-There are 2 parsers that can be used, the `cucumber`- and the `cucumberMulti`-parser. Just select one of them based on the below description and use them like this
 
-```
+## Intro
+There are multiple version of CucumberJS that all differ a bit in the way they log failures. This parser will parse the log based on 2 log lines. That combination will determine if a `*.feature`-file needs to be run again.
+The 2 log lines are
+
+- `Failures:`
+- `Specs: /path/to/your/featurefile/flaky.feature`.
+
+The `Failures` log line is always printed on failed scenario's. The `Specs: /path/to/your/featurefile/flaky.feature` is not always printed, due to the capability settings in the Protractor configuration. See [here](#capability-configuration)
+
+
+## Usage
+The following 2 things need to be done to get the parser working
+
+### Configure protractor-flake
+Via the CLI:
+```shell
 // how to use the `cucumber`-parser
 protractor-flake --protractor-path=/path/to/protractor --parser cucumber --node-bin node --max-attempts=3 -- path/to/protractor.conf.js
+```
+
+Or programmatically:
+
+```js
+var protractorFlake = require('protractor-flake');
+
+// Default Options
+protractorFlake({
+  parser: 'cucumber'
+}, function (status, output) {
+  proces.exit(status)
+})
+```
+
+### Add a `Specs: ..` log statement
+Depending on the version of CucumberJS that is used 1 of the hooks defined below can be used. This can be saved to a file like for example `afterHooks.js`. Make sure that this file is required in the `protractor.conf.js` file like this
 
-// how to use the `cucumberMulti`-parser
-protractor-flake --protractor-path=/path/to/protractor --parser cucumberMulti --node-bin node --max-attempts=3 -- path/to/protractor.conf.js
+```js
+cucumberOpts: {
+    require: [
+        'path/to/your/afterHooks.js'
+    ]
+}
+```
+
+This will **always** print the `Specs:` in the testoutput when a scenario fails. The parser will filter double specs.
+
+#### Version 1
+```js
+'use strict';
+
+module.exports = function () {
+    this.After(function (scenarioResult) {
+        if (scenarioResult.isFailed()) {
+            // Log the spec to the console for protractor-flake to be able to rerun the failed specs
+            console.log('Specs:', scenarioResult.getUri());
+        }
+
+        return Promise.resolve();
+    });
+};
 ```
 
+#### Version 2
+```js
+'use strict';
 
+const {defineSupportCode} = require('cucumber');
 
-## cucumber
-> This parser only works with `cucumberjs` version < v0.9.0
+defineSupportCode(({After}) => {
+    After(function (scenarioResult) {
+        if (scenarioResult.status === 'failed') {
+            // Log the spec to the console for protractor-flake to be able to rerun the failed specs
+            console.log('Specs:', scenarioResult.scenario.uri);
+        }
+
+        return Promise.resolve();
+    });
+});
+```
 
-The `cucumber` parser will search the testoutput for this piece of text `/path/to/your/featurefile/flakey.feature:4 # Scenario: Flakey scenario`. This is the line that says that a scenario in a featurefile failed.
-The `/path/to/your/featurefile/flakey.feature` piece will be stripped out and passed to `protractor-flake` to rerun the file(s) for the amount of retries that have been given.
+#### Version 3
+```js
+'use strict';
 
-## cucumberMulti
-> This parser is based on `cucumberjs` > v0.9.0. These versions have a different logging and also a different way in returning the path of the runned specs. (See example [logging](../test/unit/support/fixtures/cucumberjs/)).
+const {defineSupportCode, Status} = require('cucumber');
 
-> **This parser can parse testoutput from `capabilities` and `multiCapabilities`**
+defineSupportCode(({After}) => {
+    After(function (testCase) {
+        if (testCase.status === 'failed') {
+            // Log the spec to the console for protractor-flake to be able to rerun the failed specs
+            console.log('Specs:', testCase.scenario.uri);
+        }
 
-The `cucumberMulti` parser will search the testoutput for the text combination `Failures:`and `Specs: /path/to/your/featurefile/flakey.feature`. 
-The parser needs to search for both pieces of text because succeeded specs are also being logged.
-The `/path/to/your/featurefile/flakey.feature` piece will be stripped out and passed to `protractor-flake` to rerun the file(s) for the amount of retries that have been given.
+        return Promise.resolve();
+    });
+});
+```
 
-Based on the way the capability in Protractor has been configured the `Specs:` are printed. See the table below for the results.	
+## Capability configuration
+Based on the way the capability in Protractor has been configured the `Specs:` are printed. See the table below for the results.
 
 ```
 // Single
@@ -65,46 +138,3 @@ multiCapabilities = [
 |  multi     |     false      |   >1    |   true   |   **FALSE**   |
 |  multi     |      true      |   >1    |  false   |    **TRUE**   |
 |  multi     |     false      |   >1    |  false   |     false     |
-
-When no `Specs:` are printed in the logging Protractor will run the Protractor command again which means that all the specs are run on **ALL** the instances (also the succeeded specs).
-
-### Keep in mind
-Keep in mind that if the testsuite holds multiple featurefiles and only 1 featurefile is flakey the following will occur. For example:
-* Run 1:
-  * 2 featurefiles are run 
-  * 1 is flakey => `protractor-flake` will parse the output, finds 1 failed specs output (see table) and reruns protractor
-* Run 2:
-  * 1 featurefile is run
-  * featurefile is flakey => `protractor-flake` will parse the output and finds NO specs in the output (see table) and reruns protractor
-* Run 3:
-  * protractor is rerun by `protractor-flake` but gets no `--specs`, this means that ALL the featurefiles are run again 
-
-To avoid this behaviour a simple "workaround" can be implemented, see "Always print specs"
-
-### Always print specs
-There is a way to always print `Specs:`. This can be done with the following hook, see [cucumberjs](https://github.com/cucumber/cucumber-js) for more hook info:
-
-```
-var afterHook = function () {
-  this.After(function (scenario, callback) {
-        if (scenario.isFailed()) {
-            console.log('Specs:', scenario.getUri());
-        }
-        callback();
-  });
-};
-
-module.exports = afterHook;
-```
-
-Make sure that the file is required in the `protractor.conf.js` file like this
-
-```
-cucumberOpts: {
-    require: [
-        'path/to/your/afterHooks.js'
-    ]
-}
-```
-
-This will always print the `Specs:` in the testoutput. The parser will filter double specs. 

src/parsers/cucumber.js

@@ -1,14 +1,21 @@
 export default {
-  name: 'cucumber',
-
+  name: 'cucumberMulti',
   parse (output) {
     let match = null
     let failedSpecs = []
-    let FAILED_LINES = /(.*?):\d+ # Scenario:.*/g
-    while (match = FAILED_LINES.exec(output)) { // eslint-disable-line no-cond-assign
-      failedSpecs.push(match[1])
-    }
-
-    return failedSpecs
+    let testsOutput = output.split('------------------------------------')
+    let RESULT_FAIL = 'Failures:'
+    let SPECFILE_REG = /Specs:\s(.*\.feature)/g
+    testsOutput.forEach(function (test) {
+      // only check specs when RESULT_FAIL, ` Specs: ` is always printed when at least multiple features on 1 instance
+      // are run with `shardTestFiles: true`
+      if (test.indexOf(RESULT_FAIL) > -1) { // eslint-disable-line no-cond-assign
+        while (match = SPECFILE_REG.exec(test)) { // eslint-disable-line no-cond-assign
+          failedSpecs.push(match[1])
+        }
+      }
+    })
+    // Remove double values
+    return [...new Set(failedSpecs)]
   }
 }

src/parsers/index.js

@@ -1,10 +1,9 @@
 import cucumber from './cucumber'
-import cucumberMulti from './cucumber.multi'
 import { extname } from 'path'
 import multi from './multi'
 import standard from './standard'
 
-let all = { cucumber, cucumberMulti, multi, standard }
+let all = { cucumber, multi, standard }
 
 function handleObject (parserObject) {
   if (typeof parserObject.parse !== 'function') {

test/unit/parsers/cucumber.test.js

@@ -2,19 +2,60 @@ import readFixture from '../support/read-fixture'
 import cucumberParser from '../../../src/parsers/cucumber'
 
 describe('cucumberParser', () => {
-  const failedOutput = readFixture('failed-cucumberjs-output.txt')
-
   describe('#parse', () => {
-    it('properly identifies failed cucumberjs feature files', () => {
-      expect(cucumberParser.parse(failedOutput)).to.eql([
-        '/Users/jrust/code/features/automated/fail.feature'
+    it('properly handles error output in multicapabilities tests with a feature', function () {
+      let output = readFixture('cucumberjs/cucumberjs-multi-output-feature-failure.txt')
+
+      expect(cucumberParser.parse(output)).to.eql([
+        '/Users/wswebcreation/test/e2e/features/functional/flakey.feature'
+      ])
+    })
+
+    it('properly handles error output in sharded multicapabilities tests with a feature', function () {
+      let output = readFixture('cucumberjs/cucumberjs-multi-output-shared-feature-failure.txt')
+
+      expect(cucumberParser.parse(output)).to.eql([
+        '/Users/wswebcreation/test/e2e/features/functional/flakey.feature'
+      ])
+    })
+
+    it('returns an empty array if cucumberjs output has no matches for a multicapabilities tests with a feature', function () {
+      let output = readFixture('cucumberjs/cucumberjs-multi-output-feature-success.txt')
+
+      expect(cucumberParser.parse(output)).to.eql([])
+    })
+
+    it('returns an empty array if cucumberjs output has no matches for a sharded multicapabilities tests with a feature', function () {
+      let output = readFixture('cucumberjs/cucumberjs-multi-output-shared-feature-success.txt')
+
+      expect(cucumberParser.parse(output)).to.eql([])
+    })
+
+    it('can\'t handles error output in multicapabilities tests with features', function () {
+      let output = readFixture('cucumberjs/cucumberjs-multi-output-features-failures.txt')
+
+      expect(cucumberParser.parse(output)).to.eql([])
+    })
+
+    it('properly handles error output in sharded multicapabilities tests with features and double failures', function () {
+      let output = readFixture('cucumberjs/cucumberjs-multi-output-shared-features-failures.txt')
+
+      expect(cucumberParser.parse(output)).to.eql([
+        '/Users/wswebcreation/protractor-flake-tests/features/flakey.feature',
+        '/Users/wswebcreation/protractor-flake-tests/features/another.flakey.feature'
       ])
     })
 
-    it('returns an empty array if cucumberjs output has no matches', () => {
-      let successOutput = readFixture('success-cucumberjs-output.txt')
+    it('returns an empty array if cucumberjs output has no matches for a multicapabilities tests with features', function () {
+      let output = readFixture('cucumberjs/cucumberjs-multi-output-features-success.txt')
+
+      expect(cucumberParser.parse(output)).to.eql([])
+    })
+
+    it('returns an empty array if cucumberjs output has no matches for a sharded multicapabilities tests with features', function () {
+      let output = readFixture('cucumberjs/cucumberjs-multi-output-shared-features-success.txt')
 
-      expect(cucumberParser.parse(successOutput)).to.eql([])
+      expect(cucumberParser.parse(output)).to.eql([])
     })
   })
 })