[Bug]: Failing tests have a trace with no images and with steps missing

[cstlh]

Version

1.45.1

Steps to reproduce

• Clone https://github.com/cstlh/playwright-trace-issue

• Run npm -i

• Run npx playwright test

• Open report

• Check test that is failing and is using baseTest.ts has a mostly empty trace and most steps are missing
  

• Check that failed test that is not using baseTest.ts, but instead using @playwright/test directly has all trace information
  

• Check that passed test using baseTest.ts has full trace and steps

Additional steps:

• Revert back to playwright 1.43.1 npm i -D @playwright/test@1.43.1
• Run tests again and check that all reports have full trace

Expected behavior

I expected that the whole trace is present in all failing tests

Actual behavior

Traces are mostly empty in failed tests

Additional context

I have tried the same setup in 1.43 and 1.44 and everything was working fine, this started failing for me when I tried to update the version to 1.45 and 1.45.1

Environment

System:
    OS: macOS 14.4.1
    CPU: (8) arm64 Apple M2
    Memory: 1004.56 MB / 24.00 GB
  Binaries:
    Node: 20.11.1 - /usr/local/bin/node
    npm: 10.8.0 - /usr/local/bin/npm
    pnpm: 9.4.0 - /usr/local/bin/pnpm
  IDEs:
    VSCode: 1.90.2 - /usr/local/bin/code
  Languages:
    Bash: 3.2.57 - /bin/bash
  npmPackages:
    @playwright/test: ^1.45.1 => 1.45.1

[mxschmitt]

I can reproduce that failTestNoTrace.spec reproduces no trace in 1.45 but produces one in 1.44.

[rberovskis]

I was about to create a bug for the same issue myself. I am facing the same issue just without the use of base class.

I've done some investigation and I think its even more straightforward - the issue might lie in the fact that it creates context and doesn't save it. It used to work just fine, but its no longer working.

I have an example repository here based on default playwright tutorial examples

For me, workaround is pretty straightforward - just save Context outside of beforeAll and then close it. This works with "@playwright/test": "^1.45.1",

Basically this used to work

let page: Page;

//This used to work just fine
test.beforeAll(async ({ browser }) => {
  // Same thing - no longer works as well
  // const context = await browser.newContext();
  // page = await context.newPage();

  page = await (await browser.newContext()).newPage();
});  

test("get started link", async () => {
  await page.goto("https://playwright.dev/");

  // Click the get started link.
  await page.getByRole("link", { name: "Get started" }).click();

  // Expects page to have a heading with the name of Installation.
  await expect(
    page.getByRole("heading", { name: "Installationnn" })
  ).toBeVisible();
});

Now I need to do this

  let page: Page;
  let context: Context;

  test.beforeAll(async ({ browser }) => {
    context = await browser.newContext();
    page = await context.newPage();
  });

  test("get started link", async () => {
    await page.goto("https://playwright.dev/");

    // Click the get started link.
    await page.getByRole("link", { name: "Get started" }).click();

    // Expects page to have a heading with the name of Installation.
    await expect(
      page.getByRole("heading", { name: "Installationnn" })
    ).toBeVisible();
  });
  
  
  test.afterAll("Cleanup", async () => {
    await page.close();
    await context.close();
  });
  

[mxschmitt]

I can verify that this bug (by the original reporter) is fixed in @beta and should be available in v1.45.2 soon.