Question: Generating cucumber reports with hooks

[tcydik]

I am trying to utilize cucumberReporter feature and I noticed that it is struggling to generate a report in case a test fails and contains "After" hooks.

My playwright config file:
reporter: [
['list'],
['html', { open: 'never' }],
['junit', { outputFile: 'results.xml' }],
cucumberReporter('html', {
outputFile: 'cucumber-report/bdd_report.html',
}),
],
and the hook it has problem with:
After({ tags: '@delete' }, async function ({ page, $testInfo }) {
// Account cleanup
const accountSettingsPage = createAccountSettingsPage(page, $testInfo)
await accountSettingsPage.deleteAccount()
})

The message I get:
"Error in reporter Error: BDD data attachment is not found for test: As an user, I would like to create a basic account @smoke @regression @Validation @test
Did you set enrichReporterData: true in the Playwright config?

at TestCaseRun.getBddData (...\node_modules\playwright-bdd\src\reporter\cucumber\messagesBuilder\TestCaseRun.ts:71:13)
at new TestCaseRun (...\node_modules\playwright-bdd\src\reporter\cucumber\messagesBuilder\TestCaseRun.ts:51:25)
at ...\node_modules\playwright-bdd\src\reporter\cucumber\messagesBuilder\Builder.ts:122:27
at Array.forEach (<anonymous>)
at MessagesBuilder.createTestCaseRuns (...\node_modules\playwright-bdd\src\reporter\cucumber\messagesBuilder\Builder.ts:117:21)
at MessagesBuilder.doBuildMessages (...\node_modules\playwright-bdd\src\reporter\cucumber\messagesBuilder\Builder.ts:73:10)"

Please note that this is not causing any issues if test case is passing although same hooks are applied.

Wanted to hear if this is a bug or I am settings up something wrong?

Thank you!

[vitalets]

Sound like a bug. Could you create a reproducable example in https://github.com/vitalets/playwright-bdd-example

[tcydik]

Sorry for spamming if you get edits as well, but I think now I found the actual issue. So basically what happens is that cucumber report stops in case one of the hooks fail. In my setup I have a hook running, which does not take into account specific conditions, so in some cases it is failing, which was not causing any issues for playwright report as they seem to ignore hook failures, but in playwright-bdd hook failures are blocking report generation. In summary it is more of a optimization and not actual bug:)

[vitalets]

Hmm.. In playwright-bdd error in hook should not block the report generation, it should show failing hook in the report. Something like this:

Reproducible demo would be helpful. Or could you just show the code of hook that causes this BDD data attachment is not found... error (if possible).

[tcydik]

Scenario with tags from .feature file:

StepDefinition file:

Error output:

Playwright report output:

As you can see in this case I am simulating that the user could not enter the product and the code in the hook is based on deleting account within the product, which could not run and that is why hook failed. Hope this helps.

[vitalets]

I've created the similar setup in https://github.com/vitalets/playwright-bdd-example/tree/failing-hook-107
But can't reproduce the issue:
There is error in step and then error in after hook. Cucumber html report correctly shows both errors:

Playwright html report looks similar to yours:

Could you compare this example code with your setup and try to reproduce on it?

[tcydik]

Hello, I tried this solution and it works for me as well. However when I add a sleep (i.e. 60000 msec) in the After hook, which is longer than Playwright configured timeout, which by default is 30000 msec, then I can reproduce it. So it something to do with timeout error.

[vitalets]

Reproduced now, good catch! Checking for the fix.

[vitalets]

Fixed in 6.0.3.
Please re-check on your side and reopen if there are any issues.

[tcydik]

Works fine now. Thanks!