ERROR: When using several calls of defineBddConfig() after installing v4.0.0

[jzaratei]

Hello,
After installing v4.0.0, I get this error when doing the npx bddgen:

ERROR: When using several calls of defineBddConfig() please manually provide different "outputDir" option.

I'm running my features in different projects and I have differents outputDir for each project:

projects: [
    { 
      name: 'setup',
      testDir: defineBddConfig({
        outputDir: '.features-gen/login',
        importTestFrom: 'page-objects/basePage.ts',
        paths: ['features/login.feature'],
        require: ['steps/loginSteps.ts'],
      }),
    },
    {
      name: 'Regresion tests using chromium',
      testDir: defineBddConfig({
        outputDir: '.features-gen/regression',
        importTestFrom: 'page-objects/basePage.ts',
        paths: ['features/project.feature'],
        require: ['steps/projectSteps.ts'],
      }),
      dependencies: ['setup'],
      use: { 
        ...devices['Desktop Chrome'],
        storageState: STORAGE_STATE, 
      },
     {
      name: 'Regresion tests using edge',
      testDir: defineBddConfig({
        outputDir: '.features-gen/regression',
        importTestFrom: 'page-objects/basePage.ts',
        paths: ['features/project.feature'],
        require: ['steps/projectSteps.ts'],
      }),
      dependencies: ['setup'],
      use: { 
        ...devices['Desktop Edge'],
        storageState: STORAGE_STATE, 
      },
    },

Note: when rolling back to 3.3.0, it works just fine

[vitalets]

This is expected - there are equal defineBddConfig() calls in the config:

You can move it to single call and everything should work:

const testDir = defineBddConfig({
        outputDir: '.features-gen/regression',
        importTestFrom: 'page-objects/basePage.ts',
        paths: ['features/project.feature'],
        require: ['steps/projectSteps.ts'],
});

// ...

projects: [
    { 
      name: 'setup',
      testDir: defineBddConfig({
        outputDir: '.features-gen/login',
        importTestFrom: 'page-objects/basePage.ts',
        paths: ['features/login.feature'],
        require: ['steps/loginSteps.ts'],
      }),
    },
    {
      name: 'Regresion tests using chromium',
      testDir,
      dependencies: ['setup'],
      use: { 
        ...devices['Desktop Chrome'],
        storageState: STORAGE_STATE, 
      },
     {
      name: 'Regresion tests using edge',
      testDir,
      dependencies: ['setup'],
      use: { 
        ...devices['Desktop Edge'],
        storageState: STORAGE_STATE, 
      },
    },

Also it's better from performance reason - test files are generated once and used for both projects.

[jzaratei]

@vitalets Sry I did a mistake when describing the issue. This is how I really have my playwright.config:

All the outputDir are differents and still I have that error.
Using the testDir constant, I got the same results.

[vitalets]

@jzaratei could you post in text your actual playwright.config.ts and the error message?

[jzaratei]

Never mind, it seems a version issue. I was having this issue yesterday when I installed the v4.0.0, but doing a fresh install today it seems that the latest version is 4.4.0 and all seems to work pretty good.

I'm closing this issue.

[vitalets]

Excellent!
Thank you!

[jzaratei]

Sorry @vitalets I had to reopen this because it still happening in the latest version 4.0.0

Error:

ERROR: When using several calls of defineBddConfig() please manually provide different "outputDir" option.
ERROR: When using several calls of defineBddConfig() please manually provide different "outputDir" option.

This is my playwright.config.ts:

export default defineConfig({
  reporter: [
    ["html"],
    ["junit", { outputFile: "test-results/results.xml" }],
  ],

  /* Configure projects for major browsers */
  projects: [
    {
      name: "Login setup",
      testDir: defineBddConfig({
        outputDir: ".features-gen/login",
        importTestFrom: "page-objects/basePage.ts",
        paths: ["features/login.feature"],
        require: ["steps/loginSteps.ts"],
      }),
      use: {
        ...devices["Desktop Chrome"],
        channel: "chrome",
      },
    },
    {
      name: "Regresion tests using chromium",
      testDir: defineBddConfig({
        outputDir: ".features-gen/regression-chrome",
        importTestFrom: "page-objects/basePage.ts",
        paths: ["features/regression/*.feature"],
        require: ["steps/*.ts"],
        verbose: true,
      }),
      dependencies: ["Login setup"],
      use: {
        ...devices["Desktop Chrome"],
        channel: "chrome",
        storageState: STORAGE_STATE,
      },
    },

    /* Test against branded browsers. */
    {
      name: "Regresion tests using Microsoft Edge",
      testDir: defineBddConfig({
        outputDir: ".features-gen/regression-edge",
        importTestFrom: "page-objects/basePage.ts",
        paths: ["features/regression/*.feature"],
        require: ["steps/*.ts"],
      }),
      use: {
        ...devices["Desktop Edge"],
        channel: "msedge",
        storageState: STORAGE_STATE,
      },
      dependencies: ["Login setup"],
    },
  ],
});

And I have organized my features this way

[vitalets]

No problem, will try to reproduce.
Thanks for your feedback 😊

[vitalets]

@jzaratei I've created a very similar config in playwright-bdd-example: https://github.com/vitalets/playwright-bdd-example/blob/several-projects/playwright.config.ts
But still don't see this error. Could you clone it on several-projects branch and compare with your setup/steps?

[jzaratei]

After some hours of tests, I manage to reproduce the issue. It seems that adding a storage_state in a login step is causing the error:

Given('I am on home page', async ({ page }) => {
  await page.goto('https://playwright.dev');
  await page.context().storageState({ path: STORAGE_STATE });
});

In the playwright.config.ts:

import { defineBddConfig } from 'playwright-bdd';
import path from "path";
export const STORAGE_STATE = path.join(__dirname, ".auth/user.json");

export default defineConfig({
...

I did a pull request to your branch so you can replicate it as well.
vitalets/playwright-bdd-example#1

[vitalets]

Thank you for your effort @jzaratei ! Checking..
This is my fault - when reproducing something we should try to use all the same code. I thought that at least storageState should not have any impact and didn't add it.

[vitalets]

The problem is in this line - importing STORAGE_STATE in steps/index.ts re-evaluates playwright config in worker:

// steps/index.ts

import { STORAGE_STATE } from "../playwright.config";

The solution is to move STORAGE_STATE definition to separate const.ts file or to Login setup project steps. For example:

// const.ts
export const STORAGE_STATE = path.join(__dirname, ".auth/user.json");

and then:

import { STORAGE_STATE } from "../const";

I know that there is example in Playwright docs that uses exporting STORAGE_STATE from playwright.config.ts. For that case I will add a check on playwright-bdd side in the nearest release.
But my personal opinion is that re-evaluating config file in worker is redundant and does not add performance to the tests run.

@jzaratei could you try moving STORAGE_STATE to separate file and check on your side that error disappears?

[jzaratei]

yeah that worked! I moved STORAGE_STATE to a const file and the generation process is working again.
So I guess we could close this issue, when you add that check.

[vitalets]

Ok, agree.

Btw, what is the reason why you keep two separate calls of defineBddConfig with the same data except outputDir?
In my mind this should be covered by single call on top, but maybe I don't know some cases..

[jzaratei]

@vitalets I do it that way because I have to execute all my regression test twice, one for each browser. If I group them in a single call like:

const testDir = defineBddConfig({
        outputDir: '.features-gen/regression',
        importTestFrom: 'page-objects/basePage.ts',
        paths: ['features/regression/*.feature'],
        require: ['steps/*.ts'],
});

the outputDir would be the same for both of them, and that is generating only testcases for the first browser.

[vitalets]

I'm testing on main branch of playwright-bdd-example config - single call of defineBddConfig for two projects: chromium / firefox. Tests run for both projects:

  ✓  1 [firefox] › features/todopage.feature.spec.js:6:7 › Todo Page › Adding todos (1.0s)
  ✓  2 [firefox] › features/homepage.feature.spec.js:6:7 › Home Page › Check title (1.1s)
  ✓  3 [chromium] › features/homepage.feature.spec.js:6:7 › Home Page › Check title (649ms)
  ✓  4 [chromium] › features/todopage.feature.spec.js:6:7 › Todo Page › Adding todos (688ms)
  ✓  5 [chromium] › features/homepage.feature.spec.js:11:7 › Home Page › Check get started (1.5s)
  ✓  6 [firefox] › features/homepage.feature.spec.js:11:7 › Home Page › Check get started (711ms)

Isn't it your case?

[vitalets]

Released in v5.0.0. Changelog.

[jzaratei]

I'm testing on main branch of playwright-bdd-example config - single call of defineBddConfig for two projects: chromium / firefox. Tests run for both projects:

  ✓  1 [firefox] › features/todopage.feature.spec.js:6:7 › Todo Page › Adding todos (1.0s)
  ✓  2 [firefox] › features/homepage.feature.spec.js:6:7 › Home Page › Check title (1.1s)
  ✓  3 [chromium] › features/homepage.feature.spec.js:6:7 › Home Page › Check title (649ms)
  ✓  4 [chromium] › features/todopage.feature.spec.js:6:7 › Todo Page › Adding todos (688ms)
  ✓  5 [chromium] › features/homepage.feature.spec.js:11:7 › Home Page › Check get started (1.5s)
  ✓  6 [firefox] › features/homepage.feature.spec.js:11:7 › Home Page › Check get started (711ms)

Isn't it your case?

Yeah, that is the case. I updated my playwright.config.ts to use the testDir constant and everything looks cleaner now. Thanks for your support!

[vitalets]

You are welcome :)