Bug: scenarios in the same feature create tests using different fixtures

[wootra]

short description
I am using classes extending one from others to share functionalities.
But as you guys know class can hold state as member variables.
even though I use the method that is inherited from the parent class, the state can be different.
in below test, even though I used the "Given" decorator to choose the child class, the result is just chosen based on where the specific decorator is located (parent class).
When the feature file is marked with specific fixture, it should consistently use the same fixture, but in below example, you will see the fixture for each tests are dynamically chosen based on the location of the each decorator.
To fix this temporarily, I had to use a specific decorator that only belong to the fixture and it makes my BDD more verbose.

Given

playright config

import { devices, defineConfig } from "@playwright/test";

/**
 * Read environment variables from file.
 * https://github.com/motdotla/dotenv
 */
import dotenv from "dotenv";
import { defineBddConfig } from "playwright-bdd";


dotenv.config({
	path: "./.env",
});
console.log("setting on env:"+ process.env.NODE_ENV);

dotenv.config({
	path: `./.env.${process.env.NODE_ENV}`,
});
dotenv.config({
	path: `./.env.${process.env.NODE_ENV}.local`,
});

console.log("public url is:", process.env.PUBLIC_URL);
const isUi = process.env.UI === 'true';
const testDir = defineBddConfig({
	paths: ["./features"],
	importTestFrom: './steps/fixtures.ts',
	require: ['./steps/*.ts'],
	quotes: 'backtick' // when fixture filles are converted to spec.ts files, quotes will be replaced to the backticks
});

// 
export default defineConfig({
	testDir,
	timeout: 60 * 1000,
	expect: {
		/**
		 * Maximum time expect() should wait for the condition to be met.
		 * For example in `await expect(locator).toHaveText();`
		 */
		timeout: 10000,
	},
	/* Run tests in files in parallel */
	fullyParallel: true,
	/* Fail the build on CI if you accidentally left test.only in the source code. */
	forbidOnly: !!process.env.CI,
	/* Retry on CI only */
	retries: process.env.CI ? 2 : 0,
	/* Opt out of parallel tests on CI. */
	workers: process.env.CI ? 1 : undefined,
	reporter: [["html", { outputFolder: "./reports" }]],
	use: {
	  	actionTimeout: 0,
		/* Base URL to use in actions like `await page.goto('/')`. */
		baseURL: process.env.PUBLIC_URL ?? 'http://my-dev.dimensional.com',
		screenshot: 'only-on-failure',
		/* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
		trace: "on-first-retry",
		testIdAttribute: "data-qa",
	},
	projects: [
		{
			name: "setup",
			testMatch: /\/__[a-zA-Z0-9].[a-zA-Z0-9_-]+\.feature\.spec\.js$/,
		},
		{
			name: "chromium",
			use: {
				...devices["Desktop Chrome"],
				// Use prepared auth state.
				storageState: ".auth/user.json",
			},
			testMatch: /\/[a-zA-Z0-9].[a-zA-Z0-9_-]+\.feature\.spec\.js$/,
			...(isUi? {}: {
				dependencies: ["setup"], // when UI mode, don't make dependency to save time.
			}),
		},
});

feature

@fixture:fundCenter
Feature: Some Feature
  Given: 
    Given I am on fund center  <=== choosing fund center. this whole feature should use fund center

  Scenario: Scenario 1
    Then I see "Some button" button
  Scenario: Scenario 2
    Then I see "Some tab" tab   <==== this decorator is in model center (parent class)

fixture

export
@Fixture<typeof test>("fundCenter")
class FundCenter extends ModelCenter{ // <==== extended from ModelCenter
        constructor(){
           super('fund');        
        }

        @Given("I am in fund center")
	async onFundCenter()
	{
		await this.openList('fund');
		this.modelStates.currentPage = "fund list";
	}	
       
        @Then("I see {string} button")
        async shouldSeeButton(button: string){
           ...
        }
}


export
@Fixture<typeof test>("modelCenter")
class ModelCenter {
     listType:string;
    constructor(listType: string = "model"){
       this.listType = listType; // by default, it is model, but it can be changed by child class
     }

    @Given("I am in model center")
	async onModelCenter()
	{
		await this.openList('fund');
		this.modelStates.currentPage = "fund list";
	}
        @Then("I see {string} tab")
        async shouldSeeTab(tab: string){
           ...
        }
}

fixtures.ts

type Pages = {
	modelCenter: ModelCenter;
	fundCenter: FundCenter;
};

export const test = base.extend<Pages>({
	modelCenter:async ({ page }, use) => use(new ModelCenter(page)),
	fundCenter:async ({ page }, use) => use(new FundCenter(page)),
});

When

npx bddgen

Then

It creates test using different fixtures

test.describe(`Some Feature`, () => {
  test.beforeEach(async ({ Given, fundCenter }) => {
    await Given(`I am on fundCenter`, null, { fundCenter });
  });
  test(`Scenario 1`, async ({ Then, fundCenter }) => {
    await Then(`I  see "some button" button`, null, { fundCenter });
  });

  test(`Scenario 2`, async ({ Then, modelCenter }) => {
    await Then(`I  see "some tab" tab`, null, { modelCenter }); // <== using modelCenter instead of fundCenter
  });

});

Expected behavior

for the same fixture, (especially when I explicitly defined @fixture: fundCenter), it should use the same fixture instead of using the class that holds original decorator.

Isolated demo

Environment

Playwright-bdd environment info:

platform: darwin
node: v16.14.2
playwright-bdd: v5.2.0
@playwright/test: v1.37.1
@cucumber/cucumber: v9.4.0
Playwright config file: playwright.config.ts

[vitalets]

This is definitely bug, related to Background's own scope. I will investigate.
thanks @wootra

[vitalets]

Fixed in 5.3.0. Now file level @fixture:xxx should be considered in all scenarios of the file.
@wootra could you please re-check on your side?

[wootra]

Sorry for late checking. I'll re-check tomorrow. Thank you for addressing it.

[wootra]

I have tested, and got a worse situation.
It is hard to describe but basically, the right class name is not showing.

as you see modelCenter, it looks like the description belongs to modelCenter class, but it is not.

it is supposed to comes from modelCompare. not because I added @fixture:modelCompare but because it belongs to ModelCompare class that is marked as modelCompare.

as you see, my ModelCompare is extended from ModelCenter (marked as modelCenter).
this is my features file. I don't see any problem in here as well.

previously, at least the step description belongs to the class(feature), it could choose the right class. but not anymore.

[wootra]

About the bug that is relative to this issue, it is showing the same issue.

this is one of the result file. as you see, beforeEach and other tests are using different fixture.

FYI, modelBuildModal is the parent class of modelCenter, and I am in model center belongs to modelCenter fixture.

[vitalets]

Hi @wootra ! Looking on your screenshots I've noticed that ModelCompare class is marked as modelCenter fixture:

I think this is the root of the issue. Could you re-check?
On my side I'm going to add checking of fixture name duplicates and throw error in such cases.

[wootra]

@vitalets I fixed what you mentioned.
One behavior that I mentioned before (using different fixture for the step description) is fixed:

(now modelCompare is used for I see compare models page)

But still the feature.spec.js file is not using only modelCompare even though I set background for the modelCompare and set @fixture: modelCompare.

here is the extend states of my classes:

[vitalets]

@wootra so the issue is that modelCompare is not used in Open Build Modal in Model Center (Blank) scenario. It uses modelBuildModal instead?

[vitalets]

Tried to reproduce the behavior in https://github.com/vitalets/playwright-bdd-example/pull/4/files but works correctly.
Using similar class structure I've got the following generated test:

test.describe(`feature`, () => {

  test.beforeEach(async ({ Given, modelCompare }) => {
    await Given(`step from modelCenter`, null, { modelCompare });
  });

  test(`Open Build Modal in Model Center (Blank)`, async ({ And, modelCompare }) => {
    await And(`step from modelBuildModal`, null, { modelCompare });
  });

});

Could you have a look on this to find what extra code in your case leads to incorrect structure?

[wootra]

@vitalets I did not recognize the version is increased to 5.4.0 version. Let me try with the new version tomorrow. thanks!

[wootra]

with the version 5.4.0, confirmed the issue is solved. @vitalets . Thanks for taking care of it!

[vitalets]

Thanks for raising this! 😊