Rerun failed tests cause more tests executed.

[windyliu1118]

I executed the whole auto test suites in jennkins, and when I refer to the execution output, I found there are only 2 tests and 1 spec failed, but 4 tests and 3 specs are executed when using the gauge run --failed.
This is really weird, and for these months audit, this situation will always happen.
My gauge version:

Gauge version: 1.4.1
Commit Hash: a82478a

Plugins
-------
html-report (4.0.10)
js (2.3.17)
screenshot (0.0.1)
xml-report (0.2.3)

taiko version is Version: 1.0.18 (Chromium: 86.0.4214.0) RELEASE

And my shell script is as below:

#!/bin/bash


source ~/.nvm/nvm.sh

nvm use 10.16.3

export NODE_OPTIONS="--max-old-space-size=8192"


echo "remove previous reports and screenshots"

rm .gauge/screenshots/*
rm -r reports/*

npm install

gauge -v

taiko -v

echo "run auto tests"

gauge run --max-retries-count=2 --env staging_jenkins specs

if [ $? -eq 0 ]; then
  echo "All test pass at the running"
  exit 0
else
  echo "Rerun the failed tests"
  gauge run --failed
fi

1st executed result

result of rerun the failed tests:

Could you pls take a look and confirm why this issue happens? Thanks a lot.

[windyliu1118]

Can anyone take a look at this issue?

[sriv]

what does failure.json contain post the run?

if you have a sample project to replicate this issue, I can take a deeper look

[zabil]

Looks like this is related to #2158

[windyliu1118]

yeah, maybe that's the issue. @zabil is there any plan to fix this?

[zabil]

There are no plans for fix this at the moment.

[windyliu1118]

@zabil for such issue #2158, do we have any plan to fix it?

[zabil]

No plans to fix this at the moment. The issue is labelled as "help wanted" in case anyone wants to pick it up for contribution.