[Feature] Implement test.step for long test scenarios

[emakhachek]

Edit from maintainers: see this comment for feature description

---

Hi All,
I am currently using: jest, playwright, jest-playwright-preset.
Thanks to testRunner: “jest-circus/runner” I was able to override testEnvironment with a custom “fail fast” testEnvironment like it was mentioned here:
jestjs/jest#6527 (comment)
The goal is to fail the whole test suite (only one) if at least one test is failed (custom config for specific suite where all test are related to each other).
I found maxFailures: 1 option but it's shared between all projects.
I want this option to be applied only for one suite (test file) via setting maxFailures: 1 for one of the Projects combining with testIgnore or testDir options.
Thanks)

[aslushnikov]

@emakhachek (i assume we're talking about @playwright/test testrunner)

I want this option to be applied only for one suite (test file) via setting maxFailures: 1 for one of the Projects combining with testIgnore or testDir options.

It actually sounds like you use tests as "steps". So instead of having something like this that you try to configure with the custom "fail-fast" logic:

describe('test renting a car', () => {
  // SOMEHOW configure this to fail the whole test suite (only one)
  // if at least one test is failed
  test('step 1 - login', async () => { /* ... login workflow ... */ });
  test('step 2 - rent a car', async () => { /* ... rent a car sequence ... */ });
  test('step 3 - checkout', async () => { /* ... checkout cart ... */ });
  test('step 4 - check status', async () => { /* ... validate success ...*/ });
});

You actually will get the needed behavior with an explicit test.step() API:

test(`test renting a car`, async () => {
  await test.step('step 1 - login');
  /* ... login workflow ... */ 
  await test.step('step 2 - rent a car');
  /* ... rent a car sequence ... */ 
  await test.step('step 3 - checkout');
  /* ... checkout cart ... */
  await test.step('step 4 - check status');
  /* ... validate success ...*/ 
});

We don't have test.step implemented just yet - but we are collecting feedback if it is something worthy to add.
See also a suggestion for "optional" steps that continue the test even after the assertion failure: #7482 (comment).

Let us know if this is something you're looking for and you'd love to have implemented!

[emakhachek]

I like this solution more than to override test environment with a custom one. I hope anyone vote for it. Thanks)

[bgotink]

This would be very interesting to have! Especially if the reporters know what steps succeeded / failed, so they can update the status of these steps in testware.

[mxschmitt]

cc @ortoniKC might be interesting for you to subscribe to this issue. Since it makes it possible to split long tests into smaller parts while keeping the same page object.

[dgozman]

#7956 implemented the following basic syntax:

test('my test', async ({ page }) => {
  await test.step('login', async () => {
    await page.goto('myurl');
    await page.fill('#username', 'username');
    await page.fill('#password', 'password');
    await page.click('text=Log in');
  });

  await test.step('another step', async () => {
    // ...
  });
});

See here: https://playwright.dev/docs/api/class-test#test-step