Work with jasmine-reporters output

[NickTomlin]

We've had a lot of questions about usage with reporters like jasmine Junit and others. We should verify and document some use cases in the README or an examples folder.

[NickTomlin]

Looks like this is due to a difference in output from jasmine-reporters a failed spec looks like this:

Stacktrace:
  Error: Expected 'SriHarsha' to be 'Sri Harsha'.
 at /Users/ntomlin/Downloads/sampletests/sample tests/test_singledesc_3its.js:24:26

Which does not pass our simplistic regex. We should try to include this format as well.

[NickTomlin]

I haven't had much time to dig into this, but a very gross workaround would be to a console reporter in addition to the junit reporter in protractor's onPrepare:

{
  framework: 'jasmine2',
  onPrepare: function () {
    var jasmineReporters = require('jasmine-reporters');
    jasmine.getEnv().addReporter(new jasmineReporters.JUnitXmlReporter({
      consolidateAll: true,
      savePath: 'testresults',
      filePrefix: 'xmloutput'
    }));
    jasmine.getEnv().addReporter(new jasmine.ConsoleReporter({print: console.log}));
  }
}

Which would spit the proper output out to the console as well as write the junit file.

[harsha509]

Hi Nick,

I have tried all possible test scenarios by placing the above code in onPrepare: method. Still having the same issue, Along with failed test passed tests are also being executed.

[NickTomlin]

@harsha509

After doing some digging with the sample you provided (via email) it turns out that there's a different stack trace for errors within a promise:

E.g.

 expect(element(By.binding("person.name")).getText()).toEqual('My Name')

Would output a stack formatted the way protractor-flake wants ([object Object].<anonymous>)

1) Sample tests Should type a name and verify
 Message:
   Expected 'name' to equal 'My Name'.
 Stacktrace:
   Error: Failed expectation
  at [object Object].<anonymous> (/Users/ntomlin/workspace/sri-protractor-flake-example/test_singledesc_3its.js:18:62)

but putting the expectation inside the promise does not

element(By.binding("person.name")).getText().then(function(text){
    expect(text).toBe("My Name");
    element(By.buttonText("CONTINUE")).click();
});

Would output:

1) Sample tests Should type a name and verify
 Message:
   Expected 'Name' to be 'My Name'.
 Stacktrace:
   Error: Expected 'name' to be 'My Name'.
  at /Users/ntomlin/workspace/sri-protractor-flake-example/test_singledesc_3its.js:19:26
  at Array.forEach (native)

We could try to get around this by creating a custom reporter that looks for a filename, but jasmine 1.x doesn't seem to provide that on the spec object :\

[kogai]

I have same problem, and resolved it like below.

First, I flagged shardTestFiles to true.

exports.config = {
  capabilities: {
    shardTestFiles: true,
...

Then protoractor's stdout like below.

[chrome #1-0] PID: 17662
[chrome #1-0] Specs: /path/to/mySpecFile.js
[chrome #1-0]
[chrome #1-0] Starting selenium standalone server...
[chrome #1-0] Selenium standalone server started at http://10.0.4.1:42064/wd/hub
[chrome #1-0] bargainDetailE2ETest
[chrome #1-0]   some-test-result-1 - pass
[chrome #1-0]   some-test-result-2 - fail
[chrome #1-0]   some-test-result-3 - pass
[chrome #1-0]
[chrome #1-0] Failures:
[chrome #1-0]
[chrome #1-0]   1) mySpecFile some-test-result-2
[chrome #1-0]    Message:
[chrome #1-0]      Error: [Error message here]
[chrome #1-0]    Stacktrace:
[chrome #1-0]      Error: [Error message here]
[chrome #1-0]     at Array.forEach (native)
[chrome #1-0]     at process._tickCallback (node.js:382:9)
[chrome #1-0] Error
[chrome #1-0]     at [object Object].<anonymous> (/path/to/mySpecFile.js:26:44)
[chrome #1-0] From: Task: Asynchronous test function: it()
[chrome #1-0] Error
[chrome #1-0]     at [object Object].<anonymous> (/path/to/mySpecFile.js:18:5)
...

And then, the result of all the tests will be as follows.

[launcher] chrome #1-0 failed 1 test(s)
[launcher] chrome #1-1 passed
[launcher] chrome #1-2 passed
[launcher] chrome #1-3 passed
...

So, parse test IDs from [launcher] chrome #1-0 failed 1 test(s) > ['#1-0'] .
Then, get path to spec file from line of [chrome #1-0] Specs: /path/to/mySpecFile.js.

It works well, either inside or outside Promise.

*my parser is here.

[NickTomlin]

I'm going to go ahead and close this since we now have custom parsing support. This should allow users to tailor a parser to their specific output. Commit: 9791183

[surendraJ]

@kogai I tried your custom parser for shardParser. That is fine if we execute as maxAttempts: 2. Whenever protractor flake start to re run on failure specs in 2nd attempts chrome ids are disappearing from log( for example [chrome #1-0] PID: 17662)...so that it is taking all specs in 3rd attempt...