Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: focus failed and flaky tests, no links in title #24269

Merged
merged 1 commit into from
Jul 18, 2023
Merged

chore: focus failed and flaky tests, no links in title #24269

merged 1 commit into from
Jul 18, 2023

Conversation

yury-s
Copy link
Member

@yury-s yury-s commented Jul 18, 2023 •
edited
Loading

Links don't work in the check titles:

image

@yury-s yury-s merged commit 91346c7 into microsoft:main Jul 18, 2023
24 of 26 checks passed
@yury-s yury-s deleted the failed-faky-query branch July 18, 2023 00:48
@github-actions
Copy link
Contributor

Test results for "tests 1"

3 failed
❌ [playwright-test-windows-latest-node16] › reporter-html.spec.ts:1452:7 › created › labels › filter should update stats
❌ [playwright-test-windows-latest-node16] › ui-mode-trace.spec.ts:53:5 › should merge web assertion events
❌ [playwright-test-windows-latest-node16] › ui-mode-trace.spec.ts:126:5 › should show snapshots for sync assertions

12 flaky
⚠️ [chromium-ubuntu-22.04-node18] › library/tracing.spec.ts:429:5 › should work with multiple chunks
⚠️ [playwright-test-macos-latest-node16] › ui-mode-trace.spec.ts:22:5 › should merge trace events
⚠️ [playwright-test-windows-latest-node16] › ui-mode-test-output.spec.ts:60:5 › should print buffers
⚠️ [playwright-test-windows-latest-node16] › ui-mode-test-progress.spec.ts:218:5 › should show trace w/ multiple contexts
⚠️ [chromium-ubuntu-22.04-node16] › page/page-event-request.spec.ts💯3 › should report navigation requests and responses handled by service worker
⚠️ [firefox-ubuntu-22.04-node16] › library/tracing.spec.ts:706:5 › should not emit after w/o before
⚠️ [webkit-ubuntu-22.04-node16] › library/browsercontext-reuse.spec.ts:50:1 › should reset serviceworker
⚠️ [webkit-ubuntu-22.04-node16] › library/inspector/cli-codegen-pytest.spec.ts:56:5 › should save the codegen output to a file if specified
⚠️ [chromium-ubuntu-22.04-node20] › library/inspector/cli-codegen-javascript.spec.ts:87:5 › should save the codegen output to a file if specified
⚠️ [chromium-ubuntu-22.04-node20] › library/tracing.spec.ts:239:5 › should not include trace resources from the provious chunks
⚠️ [chromium-ubuntu-22.04-node20] › library/tracing.spec.ts:706:5 › should not emit after w/o before
⚠️ [chromium-ubuntu-22.04-node20] › page/page-event-request.spec.ts💯3 › should report navigation requests and responses handled by service worker

21674 passed, 509 skipped
✔️✔️✔️

❌ [playwright-test-windows-latest-node16] › reporter-html.spec.ts:1452:7 › created › labels › filter should update stats 

Error: expect(received).toBe(expected) // Object.is equality

Expected: "Total time: 1032ms"
Received: "Total time: 1.0s"

  1774 |         await expect(page.locator('.chip', { hasText: 'c.test.js' })).toHaveCount(1);
  1775 |
> 1776 |         await flakyButton.click();
       |                                 ^
  1777 |
  1778 |         await expect(searchInput).toHaveValue('@regression @flaky');
  1779 |         await expect(page).toHaveURL(/@regression%20@flaky/);

    at checkTotalDuration (D:\a\playwright\playwright\tests\playwright-test\reporter-html.spec.ts:1776:33)
    at D:\a\playwright\playwright\tests\playwright-test\reporter-html.spec.ts:1792:9

❌ [playwright-test-windows-latest-node16] › ui-mode-trace.spec.ts:53:5 › should merge web assertion events 

Error: action list

- Expected  -  6
+ Received  + 10

  Array [
-   /Before Hooks[\d.]+m?s/,
-   /page.setContent[\d.]+m?s/,
-   /expect.toBeVisiblelocator\('button'\)[\d.]+m?s/,
-   /After Hooks[\d.]+m?s/,
-   /fixture: page[\d.]+m?s/,
-   /fixture: context[\d.]+m?s/,
+   "Before Hooks1.7s",
+   "fixture: browser1.3s",
+   "fixture: context92ms",
+   "fixture: page279ms",
+   "browserContext.newPage270ms",
+   "page.setContent46ms",
+   "expect.toBeVisiblelocator('button')45ms",
+   "After Hooks2ms",
+   "fixture: page0ms",
+   "fixture: context0ms",
  ]
Call log:
  - expect.toHaveText with timeout 75000ms
  - waiting for getByTestId('action-list').getByRole('listitem')
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements


  68 |       listItem,
  69 |       'action list'
> 70 |   ).toHaveText([
     |     ^
  71 |     /Before Hooks[\d.]+m?s/,
  72 |     /page.setContent[\d.]+m?s/,
  73 |     /expect.toBeVisiblelocator\('button'\)[\d.]+m?s/,

    at D:\a\playwright\playwright\tests\playwright-test\ui-mode-trace.spec.ts:70:5

Retry 1:

Error: action list

- Expected  -  6
+ Received  + 10

  Array [
-   /Before Hooks[\d.]+m?s/,
-   /page.setContent[\d.]+m?s/,
-   /expect.toBeVisiblelocator\('button'\)[\d.]+m?s/,
-   /After Hooks[\d.]+m?s/,
-   /fixture: page[\d.]+m?s/,
-   /fixture: context[\d.]+m?s/,
+   "Before Hooks1.0s",
+   "fixture: browser709ms",
+   "fixture: context77ms",
+   "fixture: page206ms",
+   "browserContext.newPage199ms",
+   "page.setContent34ms",
+   "expect.toBeVisiblelocator('button')37ms",
+   "After Hooks2ms",
+   "fixture: page0ms",
+   "fixture: context0ms",
  ]
Call log:
  - expect.toHaveText with timeout 75000ms
  - waiting for getByTestId('action-list').getByRole('listitem')
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements


  68 |       listItem,
  69 |       'action list'
> 70 |   ).toHaveText([
     |     ^
  71 |     /Before Hooks[\d.]+m?s/,
  72 |     /page.setContent[\d.]+m?s/,
  73 |     /expect.toBeVisiblelocator\('button'\)[\d.]+m?s/,

    at D:\a\playwright\playwright\tests\playwright-test\ui-mode-trace.spec.ts:70:5

Retry 2:

Error: action list

- Expected  -  6
+ Received  + 10

  Array [
-   /Before Hooks[\d.]+m?s/,
-   /page.setContent[\d.]+m?s/,
-   /expect.toBeVisiblelocator\('button'\)[\d.]+m?s/,
-   /After Hooks[\d.]+m?s/,
-   /fixture: page[\d.]+m?s/,
-   /fixture: context[\d.]+m?s/,
+   "Before Hooks1.0s",
+   "fixture: browser628ms",
+   "fixture: context27ms",
+   "fixture: page320ms",
+   "browserContext.newPage314ms",
+   "page.setContent46ms",
+   "expect.toBeVisiblelocator('button')35ms",
+   "After Hooks2ms",
+   "fixture: page0ms",
+   "fixture: context0ms",
  ]
Call log:
  - expect.toHaveText with timeout 75000ms
  - waiting for getByTestId('action-list').getByRole('listitem')
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements


  68 |       listItem,
  69 |       'action list'
> 70 |   ).toHaveText([
     |     ^
  71 |     /Before Hooks[\d.]+m?s/,
  72 |     /page.setContent[\d.]+m?s/,
  73 |     /expect.toBeVisiblelocator\('button'\)[\d.]+m?s/,

    at D:\a\playwright\playwright\tests\playwright-test\ui-mode-trace.spec.ts:70:5

Retry 3:

Error: action list

- Expected  -  6
+ Received  + 10

  Array [
-   /Before Hooks[\d.]+m?s/,
-   /page.setContent[\d.]+m?s/,
-   /expect.toBeVisiblelocator\('button'\)[\d.]+m?s/,
-   /After Hooks[\d.]+m?s/,
-   /fixture: page[\d.]+m?s/,
-   /fixture: context[\d.]+m?s/,
+   "Before Hooks968ms",
+   "fixture: browser595ms",
+   "fixture: context27ms",
+   "fixture: page310ms",
+   "browserContext.newPage303ms",
+   "page.setContent42ms",
+   "expect.toBeVisiblelocator('button')39ms",
+   "After Hooks2ms",
+   "fixture: page0ms",
+   "fixture: context0ms",
  ]
Call log:
  - expect.toHaveText with timeout 75000ms
  - waiting for getByTestId('action-list').getByRole('listitem')
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements
  -   locator resolved to 10 elements


  68 |       listItem,
  69 |       'action list'
> 70 |   ).toHaveText([
     |     ^
  71 |     /Before Hooks[\d.]+m?s/,
  72 |     /page.setContent[\d.]+m?s/,
  73 |     /expect.toBeVisiblelocator\('button'\)[\d.]+m?s/,

    at D:\a\playwright\playwright\tests\playwright-test\ui-mode-trace.spec.ts:70:5

❌ [playwright-test-windows-latest-node16] › ui-mode-trace.spec.ts:126:5 › should show snapshots for sync assertions 

Error: action list

- Expected  -  7
+ Received  + 11

  Array [
-   /Before Hooks[\d.]+m?s/,
-   /page\.setContent[\d.]+m?s/,
-   /locator\.clickgetByRole\('button'\)[\d.]+m?s/,
-   /expect\.toBe[\d.]+m?s/,
-   /After Hooks[\d.]+m?s/,
-   /fixture: page[\d.]+m?s/,
-   /fixture: context[\d.]+m?s/,
+   "Before Hooks964ms",
+   "fixture: browser614ms",
+   "fixture: context29ms",
+   "fixture: page286ms",
+   "browserContext.newPage279ms",
+   "page.setContent51ms",
+   "locator.clickgetByRole('button')76ms",
+   "expect.toBe1ms",
+   "After Hooks2ms",
+   "fixture: page0ms",
+   "fixture: context0ms",
  ]
Call log:
  - expect.toHaveText with timeout 75000ms
  - waiting for getByTestId('action-list').getByRole('listitem')
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements


  142 |       listItem,
  143 |       'action list'
> 144 |   ).toHaveText([
      |     ^
  145 |     /Before Hooks[\d.]+m?s/,
  146 |     /page\.setContent[\d.]+m?s/,
  147 |     /locator\.clickgetByRole\('button'\)[\d.]+m?s/,

    at D:\a\playwright\playwright\tests\playwright-test\ui-mode-trace.spec.ts:144:5

Retry 1:

Error: action list

- Expected  -  7
+ Received  + 11

  Array [
-   /Before Hooks[\d.]+m?s/,
-   /page\.setContent[\d.]+m?s/,
-   /locator\.clickgetByRole\('button'\)[\d.]+m?s/,
-   /expect\.toBe[\d.]+m?s/,
-   /After Hooks[\d.]+m?s/,
-   /fixture: page[\d.]+m?s/,
-   /fixture: context[\d.]+m?s/,
+   "Before Hooks990ms",
+   "fixture: browser630ms",
+   "fixture: context34ms",
+   "fixture: page288ms",
+   "browserContext.newPage281ms",
+   "page.setContent111ms",
+   "locator.clickgetByRole('button')88ms",
+   "expect.toBe1ms",
+   "After Hooks2ms",
+   "fixture: page0ms",
+   "fixture: context0ms",
  ]
Call log:
  - expect.toHaveText with timeout 75000ms
  - waiting for getByTestId('action-list').getByRole('listitem')
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 1 element
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements


  142 |       listItem,
  143 |       'action list'
> 144 |   ).toHaveText([
      |     ^
  145 |     /Before Hooks[\d.]+m?s/,
  146 |     /page\.setContent[\d.]+m?s/,
  147 |     /locator\.clickgetByRole\('button'\)[\d.]+m?s/,

    at D:\a\playwright\playwright\tests\playwright-test\ui-mode-trace.spec.ts:144:5

Retry 2:

Error: action list

- Expected  -  7
+ Received  + 11

  Array [
-   /Before Hooks[\d.]+m?s/,
-   /page\.setContent[\d.]+m?s/,
-   /locator\.clickgetByRole\('button'\)[\d.]+m?s/,
-   /expect\.toBe[\d.]+m?s/,
-   /After Hooks[\d.]+m?s/,
-   /fixture: page[\d.]+m?s/,
-   /fixture: context[\d.]+m?s/,
+   "Before Hooks1.0s",
+   "fixture: browser620ms",
+   "fixture: context32ms",
+   "fixture: page343ms",
+   "browserContext.newPage336ms",
+   "page.setContent47ms",
+   "locator.clickgetByRole('button')74ms",
+   "expect.toBe1ms",
+   "After Hooks2ms",
+   "fixture: page0ms",
+   "fixture: context0ms",
  ]
Call log:
  - expect.toHaveText with timeout 75000ms
  - waiting for getByTestId('action-list').getByRole('listitem')
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements


  142 |       listItem,
  143 |       'action list'
> 144 |   ).toHaveText([
      |     ^
  145 |     /Before Hooks[\d.]+m?s/,
  146 |     /page\.setContent[\d.]+m?s/,
  147 |     /locator\.clickgetByRole\('button'\)[\d.]+m?s/,

    at D:\a\playwright\playwright\tests\playwright-test\ui-mode-trace.spec.ts:144:5

Retry 3:

Error: action list

- Expected  -  7
+ Received  + 11

  Array [
-   /Before Hooks[\d.]+m?s/,
-   /page\.setContent[\d.]+m?s/,
-   /locator\.clickgetByRole\('button'\)[\d.]+m?s/,
-   /expect\.toBe[\d.]+m?s/,
-   /After Hooks[\d.]+m?s/,
-   /fixture: page[\d.]+m?s/,
-   /fixture: context[\d.]+m?s/,
+   "Before Hooks1.0s",
+   "fixture: browser621ms",
+   "fixture: context118ms",
+   "fixture: page241ms",
+   "browserContext.newPage235ms",
+   "page.setContent39ms",
+   "locator.clickgetByRole('button')70ms",
+   "expect.toBe1ms",
+   "After Hooks2ms",
+   "fixture: page0ms",
+   "fixture: context0ms",
  ]
Call log:
  - expect.toHaveText with timeout 75000ms
  - waiting for getByTestId('action-list').getByRole('listitem')
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 0 elements
  -   locator resolved to 9 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements
  -   locator resolved to 11 elements


  142 |       listItem,
  143 |       'action list'
> 144 |   ).toHaveText([
      |     ^
  145 |     /Before Hooks[\d.]+m?s/,
  146 |     /page\.setContent[\d.]+m?s/,
  147 |     /locator\.clickgetByRole\('button'\)[\d.]+m?s/,

    at D:\a\playwright\playwright\tests\playwright-test\ui-mode-trace.spec.ts:144:5

Merge workflow run.

@github-actions
Copy link
Contributor

Test results for "tests 1"

1 failed
❌ [playwright-test-windows-latest-node16] › reporter-html.spec.ts:1452:7 › merged › labels › filter should update stats

10 flaky
⚠️ [playwright-test-macos-latest-node16] › ui-mode-trace.spec.ts:53:5 › should merge web assertion events
⚠️ [playwright-test-ubuntu-latest-node20] › ui-mode-test-progress.spec.ts:167:5 › should update tracing network live
⚠️ [firefox-ubuntu-22.04-node16] › library/tracing.spec.ts:706:5 › should not emit after w/o before
⚠️ [chromium-ubuntu-22.04-node20] › library/tracing.spec.ts:239:5 › should not include trace resources from the provious chunks
⚠️ [chromium-ubuntu-22.04-node20] › page/page-event-request.spec.ts:129:3 › should report navigation requests and responses handled by service worker with routing
⚠️ [chromium-ubuntu-22.04-node18] › library/inspector/cli-codegen-pytest.spec.ts:56:5 › should save the codegen output to a file if specified
⚠️ [chromium-ubuntu-22.04-node18] › page/page-event-request.spec.ts:129:3 › should report navigation requests and responses handled by service worker with routing
⚠️ [webkit-ubuntu-22.04-node16] › library/browsercontext-fetch.spec.ts:1002:3 › should abort requests when browser context closes
⚠️ [webkit-ubuntu-22.04-node16] › library/browsercontext-reuse.spec.ts:50:1 › should reset serviceworker
⚠️ [webkit-ubuntu-22.04-node16] › library/inspector/cli-codegen-pytest.spec.ts:56:5 › should save the codegen output to a file if specified

21678 passed, 509 skipped
✔️✔️✔️

❌ [playwright-test-windows-latest-node16] › reporter-html.spec.ts:1452:7 › merged › labels › filter should update stats 

Error: expect(received).toBe(expected) // Object.is equality

Expected: "Total time: 1288ms"
Received: "Total time: 1.3s"

  1774 |         await expect(page.locator('.chip', { hasText: 'c.test.js' })).toHaveCount(1);
  1775 |
> 1776 |         await flakyButton.click();
       |                                 ^
  1777 |
  1778 |         await expect(searchInput).toHaveValue('@regression @flaky');
  1779 |         await expect(page).toHaveURL(/@regression%20@flaky/);

    at checkTotalDuration (D:\a\playwright\playwright\tests\playwright-test\reporter-html.spec.ts:1776:33)
    at D:\a\playwright\playwright\tests\playwright-test\reporter-html.spec.ts:1782:9

Merge workflow run.

@github-actions
Copy link
Contributor

Test results for "tests 2"

4 failed
❌ [firefox] › library/debug-controller.spec.ts:233:1 › should reset routes before reuse
❌ [webkit] › library/debug-controller.spec.ts:233:1 › should reset routes before reuse
❌ [chromium] › library/debug-controller.spec.ts:233:1 › should reset routes before reuse
❌ [chromium-chromium-tip-of-tree] › library/debug-controller.spec.ts:233:1 › should reset routes before reuse

1 flaky
⚠️ [webkit] › library/inspector/cli-codegen-3.spec.ts:450:7 › cli codegen › should generate getByLabel

12139 passed, 741 skipped
✔️✔️✔️

❌ [firefox] › library/debug-controller.spec.ts:233:1 › should reset routes before reuse 

Error: Must stop trace file before stopping tracing

Retry 1:

Error: Must stop trace file before stopping tracing

Retry 2:

Error: Must stop trace file before stopping tracing

Retry 3:

Error: Must stop trace file before stopping tracing

❌ [webkit] › library/debug-controller.spec.ts:233:1 › should reset routes before reuse 

Error: Must stop trace file before stopping tracing

Retry 1:

Error: Must stop trace file before stopping tracing

Retry 2:

Error: Must stop trace file before stopping tracing

Retry 3:

Error: Must stop trace file before stopping tracing

❌ [chromium] › library/debug-controller.spec.ts:233:1 › should reset routes before reuse 

Error: Must stop trace file before stopping tracing

Retry 1:

Error: Must stop trace file before stopping tracing

Retry 2:

Error: Must stop trace file before stopping tracing

Retry 3:

Error: Must stop trace file before stopping tracing

❌ [chromium-chromium-tip-of-tree] › library/debug-controller.spec.ts:233:1 › should reset routes before reuse 

Error: Must stop trace file before stopping tracing

Retry 1:

Error: Must stop trace file before stopping tracing

Retry 2:

Error: Must stop trace file before stopping tracing

Retry 3:

Error: Must stop trace file before stopping tracing

Merge workflow run.

Germandrummer92 pushed a commit to OctoMind-dev/playwright that referenced this pull request Oct 27, 2023
@yury-s @Germandrummer92
chore: focus failed and flaky tests, no links in title (microsoft#24269)
cec62ea 
Links don't work in the check titles:

<img width="503" alt="image"
src="https://github.com/microsoft/playwright/assets/9798949/6ad8f3d3-b96f-4f62-9d4e-cc3d473fd97a">
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pavelfeldman pavelfeldman pavelfeldman approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@yury-s @pavelfeldman