Add Model Evaluation Panel #5978
Annotations
10 errors, 7 warnings, and 1 notice
Run Playwright tests:
e2e-pw/src/oss/specs/grid-tagging.spec.ts#L1
1) [chromium] › oss/specs/grid-tagging.spec.ts:42:1 › grid tagging ───────────────────────────────
Test timeout of 60000ms exceeded.
|
Run Playwright tests:
e2e-pw/src/oss/poms/grid/index.ts#L70
1) [chromium] › oss/specs/grid-tagging.spec.ts:42:1 › grid tagging ───────────────────────────────
Error: locator.scrollIntoViewIfNeeded: Test timeout of 60000ms exceeded.
Call log:
- attempting scroll into view action
- waiting for element to be stable
- element is not visible
- retrying scroll into view action, attempt #1
- waiting for element to be stable
- element is not visible
- retrying scroll into view action, attempt #2
- waiting 20ms
- waiting for element to be stable
- element is not visible
- retrying scroll into view action, attempt #3
- waiting 100ms
- waiting for element to be stable
- element is not visible
- retrying scroll into view action, attempt #4
- waiting 100ms
- waiting for element to be stable
- element is not visible
- retrying scroll into view action, attempt #5
- waiting 500ms
- waiting for element to be stable
- element is not visible
- retrying scroll into view action, attempt #6
- waiting 500ms
- waiting for element to be stable
- element is not visible
- retrying scroll into view action, attempt #7
- waiting 500ms
- waiting for element to be stable
- element is not visible
- retrying scroll into view action, attempt #8
- waiting 500ms
- waiting for element to be stable
- element is not visible
- retrying scroll into view action, attempt #9
- waiting 500ms
- waiting for element to be stable
- element is not visible
- retrying scroll into view action, attempt #10
- waiting 500ms
- waiting for element to be stable
- element is not visible
- retrying scroll into view action, attempt #11
- waiting 500ms
- waiting for element to be stable
- element is not visible
- retrying scroll into view action, attempt #12
- waiting 500ms
- waiting for element to be stable
- element is not visible
- retrying scroll into view action, attempt #13
- waiting 500ms
- waiting for element to be stable
- element is not visible
- retrying scroll into view action, attempt #14
- waiting 500ms
- waiting for element to be stable
- element is not visible
- retrying scroll into view action, attempt #15
- waiting 500ms
- waiting for element to be stable
- element is not visible
- retrying scroll into view action, attempt #16
- waiting 500ms
- waiting for element to be stable
- element is not visible
- retrying scroll into view action, attempt #17
- waiting 500ms
- waiting for element to be stable
- element is not visible
- retrying scroll into view action, attempt #18
- waiting 500ms
- waiting for element to be stable
- element is not visible
- retrying scroll into view action, attempt #19
- waiting 500ms
- waiting for element to be stable
- element is not visible
- retrying scroll into view action, attempt #20
- waiting 500ms
- waiting for element to be stable
- element is not visible
- retrying scroll into view action, attempt #21
- waiting 500ms
- waiting for element to be stable
- element is not visible
- retrying scroll into view action, attempt #22
- waiting 500ms
- waiting for element to be stable
- element is not visible
- retrying scroll into view action, attempt #23
- waiting 500ms
- waiting for element to be stable
- element is not visible
- retrying scroll into view action, attempt #24
- waiting 500ms
- waiting for element to be stable
- element is not visible
- retrying scroll into view action, attempt #25
- waiting 500ms
- waiting for element to be stable
|
Run Playwright tests:
e2e-pw/src/oss/specs/grid-tagging.spec.ts#L47
1) [chromium] › oss/specs/grid-tagging.spec.ts:42:1 › grid tagging ───────────────────────────────
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: Screenshot comparison failed:
6416 pixels (ratio 0.02 of all image pixels) are different.
Expected: /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/grid-tagging.spec.ts-snapshots/grid-untagged-chromium-linux.png
Received: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-grid-tagging-grid-tagging-chromium-retry1/grid-untagged-actual.png
Diff: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-grid-tagging-grid-tagging-chromium-retry1/grid-untagged-diff.png
Call log:
- expect.toHaveScreenshot(grid-untagged.png) with timeout 5000ms
- verifying given screenshot expectation
- waiting for getByTestId('fo-grid')
- locator resolved to <div data-cy="fo-grid" class="_spotlightGrid_1sk68_42" id="3cc64847-fc90-4de1-9eaf-958a7db5cf7b">…</div>
- taking element screenshot
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- 6416 pixels (ratio 0.02 of all image pixels) are different.
- waiting 100ms before taking screenshot
- waiting for getByTestId('fo-grid')
- locator resolved to <div data-cy="fo-grid" class="_spotlightGrid_1sk68_42" id="3cc64847-fc90-4de1-9eaf-958a7db5cf7b">…</div>
- taking element screenshot
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- captured a stable screenshot
- 6416 pixels (ratio 0.02 of all image pixels) are different.
45 | await sidebar.clickFieldCheckbox("tags");
46 | await grid.scrollBottom();
> 47 | await expect(await grid.locator).toHaveScreenshot("grid-untagged.png", {
| ^
48 | animations: "allow",
49 | });
50 |
at /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/grid-tagging.spec.ts:47:36
|
Run Playwright tests:
e2e-pw/src/oss/specs/grid-tagging.spec.ts#L47
1) [chromium] › oss/specs/grid-tagging.spec.ts:42:1 › grid tagging ───────────────────────────────
Retry #2 ───────────────────────────────────────────────────────────────────────────────────────
Error: Screenshot comparison failed:
6416 pixels (ratio 0.02 of all image pixels) are different.
Expected: /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/grid-tagging.spec.ts-snapshots/grid-untagged-chromium-linux.png
Received: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-grid-tagging-grid-tagging-chromium-retry2/grid-untagged-actual.png
Diff: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-grid-tagging-grid-tagging-chromium-retry2/grid-untagged-diff.png
Call log:
- expect.toHaveScreenshot(grid-untagged.png) with timeout 5000ms
- verifying given screenshot expectation
- waiting for getByTestId('fo-grid')
- locator resolved to <div data-cy="fo-grid" class="_spotlightGrid_1sk68_42" id="336f5431-d5d2-471f-9b45-f833a7daed04">…</div>
- taking element screenshot
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- 6416 pixels (ratio 0.02 of all image pixels) are different.
- waiting 100ms before taking screenshot
- waiting for getByTestId('fo-grid')
- locator resolved to <div data-cy="fo-grid" class="_spotlightGrid_1sk68_42" id="336f5431-d5d2-471f-9b45-f833a7daed04">…</div>
- taking element screenshot
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- captured a stable screenshot
- 6416 pixels (ratio 0.02 of all image pixels) are different.
45 | await sidebar.clickFieldCheckbox("tags");
46 | await grid.scrollBottom();
> 47 | await expect(await grid.locator).toHaveScreenshot("grid-untagged.png", {
| ^
48 | animations: "allow",
49 | });
50 |
at /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/grid-tagging.spec.ts:47:36
|
Run Playwright tests:
e2e-pw/src/oss/specs/grid-tagging.spec.ts#L47
1) [chromium] › oss/specs/grid-tagging.spec.ts:42:1 › grid tagging ───────────────────────────────
Retry #3 ───────────────────────────────────────────────────────────────────────────────────────
Error: Screenshot comparison failed:
6416 pixels (ratio 0.02 of all image pixels) are different.
Expected: /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/grid-tagging.spec.ts-snapshots/grid-untagged-chromium-linux.png
Received: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-grid-tagging-grid-tagging-chromium-retry3/grid-untagged-actual.png
Diff: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-grid-tagging-grid-tagging-chromium-retry3/grid-untagged-diff.png
Call log:
- expect.toHaveScreenshot(grid-untagged.png) with timeout 5000ms
- verifying given screenshot expectation
- waiting for getByTestId('fo-grid')
- locator resolved to <div data-cy="fo-grid" class="_spotlightGrid_1sk68_42" id="b8a6fa23-fe71-4142-99ce-d8bc9fd35d03">…</div>
- taking element screenshot
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- 6416 pixels (ratio 0.02 of all image pixels) are different.
- waiting 100ms before taking screenshot
- waiting for getByTestId('fo-grid')
- locator resolved to <div data-cy="fo-grid" class="_spotlightGrid_1sk68_42" id="b8a6fa23-fe71-4142-99ce-d8bc9fd35d03">…</div>
- taking element screenshot
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- captured a stable screenshot
- 6416 pixels (ratio 0.02 of all image pixels) are different.
45 | await sidebar.clickFieldCheckbox("tags");
46 | await grid.scrollBottom();
> 47 | await expect(await grid.locator).toHaveScreenshot("grid-untagged.png", {
| ^
48 | animations: "allow",
49 | });
50 |
at /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/grid-tagging.spec.ts:47:36
|
Run Playwright tests:
e2e-pw/src/oss/specs/plugins/histograms.spec.ts#L78
2) [chromium] › oss/specs/plugins/histograms.spec.ts:46:1 › histograms panel ─────────────────────
Error: Screenshot comparison failed:
6688 pixels (ratio 0.02 of all image pixels) are different.
Expected: /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/plugins/histograms.spec.ts-snapshots/bool-histogram-chromium-linux.png
Received: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-plugins-histograms-histograms-panel-chromium/bool-histogram-actual.png
Diff: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-plugins-histograms-histograms-panel-chromium/bool-histogram-diff.png
Call log:
- expect.toHaveScreenshot(bool-histogram.png) with timeout 5000ms
- verifying given screenshot expectation
- waiting for getByTestId('histograms-container')
- locator resolved to <div class="sc-cgMlnP iMhHXS" data-cy="histograms-container">…</div>
- taking element screenshot
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- 6688 pixels (ratio 0.02 of all image pixels) are different.
- waiting 100ms before taking screenshot
- waiting for getByTestId('histograms-container')
- locator resolved to <div class="sc-cgMlnP iMhHXS" data-cy="histograms-container">…</div>
- taking element screenshot
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- captured a stable screenshot
- 6688 pixels (ratio 0.02 of all image pixels) are different.
76 | "tags",
77 | ]);
> 78 | await expect(await histogram.locator).toHaveScreenshot("bool-histogram.png", {
| ^
79 | animations: "allow",
80 | });
81 |
at /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/plugins/histograms.spec.ts:78:41
|
Run Playwright tests:
e2e-pw/src/oss/specs/plugins/histograms.spec.ts#L78
2) [chromium] › oss/specs/plugins/histograms.spec.ts:46:1 › histograms panel ─────────────────────
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: Screenshot comparison failed:
6688 pixels (ratio 0.02 of all image pixels) are different.
Expected: /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/plugins/histograms.spec.ts-snapshots/bool-histogram-chromium-linux.png
Received: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-plugins-histograms-histograms-panel-chromium-retry1/bool-histogram-actual.png
Diff: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-plugins-histograms-histograms-panel-chromium-retry1/bool-histogram-diff.png
Call log:
- expect.toHaveScreenshot(bool-histogram.png) with timeout 5000ms
- verifying given screenshot expectation
- waiting for getByTestId('histograms-container')
- locator resolved to <div class="sc-cgMlnP iMhHXS" data-cy="histograms-container">…</div>
- taking element screenshot
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- 6688 pixels (ratio 0.02 of all image pixels) are different.
- waiting 100ms before taking screenshot
- waiting for getByTestId('histograms-container')
- locator resolved to <div class="sc-cgMlnP iMhHXS" data-cy="histograms-container">…</div>
- taking element screenshot
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- captured a stable screenshot
- 6688 pixels (ratio 0.02 of all image pixels) are different.
76 | "tags",
77 | ]);
> 78 | await expect(await histogram.locator).toHaveScreenshot("bool-histogram.png", {
| ^
79 | animations: "allow",
80 | });
81 |
at /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/plugins/histograms.spec.ts:78:41
|
Run Playwright tests:
e2e-pw/src/oss/specs/plugins/histograms.spec.ts#L78
2) [chromium] › oss/specs/plugins/histograms.spec.ts:46:1 › histograms panel ─────────────────────
Retry #2 ───────────────────────────────────────────────────────────────────────────────────────
Error: Screenshot comparison failed:
6688 pixels (ratio 0.02 of all image pixels) are different.
Expected: /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/plugins/histograms.spec.ts-snapshots/bool-histogram-chromium-linux.png
Received: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-plugins-histograms-histograms-panel-chromium-retry2/bool-histogram-actual.png
Diff: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-plugins-histograms-histograms-panel-chromium-retry2/bool-histogram-diff.png
Call log:
- expect.toHaveScreenshot(bool-histogram.png) with timeout 5000ms
- verifying given screenshot expectation
- waiting for getByTestId('histograms-container')
- locator resolved to <div class="sc-cgMlnP iMhHXS" data-cy="histograms-container">…</div>
- taking element screenshot
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- 6688 pixels (ratio 0.02 of all image pixels) are different.
- waiting 100ms before taking screenshot
- waiting for getByTestId('histograms-container')
- locator resolved to <div class="sc-cgMlnP iMhHXS" data-cy="histograms-container">…</div>
- taking element screenshot
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- captured a stable screenshot
- 6688 pixels (ratio 0.02 of all image pixels) are different.
76 | "tags",
77 | ]);
> 78 | await expect(await histogram.locator).toHaveScreenshot("bool-histogram.png", {
| ^
79 | animations: "allow",
80 | });
81 |
at /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/plugins/histograms.spec.ts:78:41
|
Run Playwright tests:
e2e-pw/src/oss/specs/plugins/histograms.spec.ts#L78
2) [chromium] › oss/specs/plugins/histograms.spec.ts:46:1 › histograms panel ─────────────────────
Retry #3 ───────────────────────────────────────────────────────────────────────────────────────
Error: Screenshot comparison failed:
6688 pixels (ratio 0.02 of all image pixels) are different.
Expected: /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/plugins/histograms.spec.ts-snapshots/bool-histogram-chromium-linux.png
Received: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-plugins-histograms-histograms-panel-chromium-retry3/bool-histogram-actual.png
Diff: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-plugins-histograms-histograms-panel-chromium-retry3/bool-histogram-diff.png
Call log:
- expect.toHaveScreenshot(bool-histogram.png) with timeout 5000ms
- verifying given screenshot expectation
- waiting for getByTestId('histograms-container')
- locator resolved to <div class="sc-cgMlnP iMhHXS" data-cy="histograms-container">…</div>
- taking element screenshot
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- 6688 pixels (ratio 0.02 of all image pixels) are different.
- waiting 100ms before taking screenshot
- waiting for getByTestId('histograms-container')
- locator resolved to <div class="sc-cgMlnP iMhHXS" data-cy="histograms-container">…</div>
- taking element screenshot
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- captured a stable screenshot
- 6688 pixels (ratio 0.02 of all image pixels) are different.
76 | "tags",
77 | ]);
> 78 | await expect(await histogram.locator).toHaveScreenshot("bool-histogram.png", {
| ^
79 | animations: "allow",
80 | });
81 |
at /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/plugins/histograms.spec.ts:78:41
|
Run Playwright tests:
e2e-pw/src/oss/specs/groups/dynamic-groups.spec.ts#L1
3) [chromium] › oss/specs/groups/dynamic-groups.spec.ts:70:3 › pcd dynamic group pagination bar ──
Test timeout of 60000ms exceeded.
|
Run Playwright tests:
e2e-pw/[chromium] › oss/specs/smoke-tests/field-visibility.spec.ts#L1
e2e-pw/[chromium] › oss/specs/smoke-tests/field-visibility.spec.ts took 56.0s
|
Run Playwright tests:
e2e-pw/[chromium] › oss/specs/smoke-tests/saved-views.spec.ts#L1
e2e-pw/[chromium] › oss/specs/smoke-tests/saved-views.spec.ts took 50.2s
|
Run Playwright tests:
e2e-pw/[chromium] › oss/specs/sidebar/sidebar.spec.ts#L1
e2e-pw/[chromium] › oss/specs/sidebar/sidebar.spec.ts took 17.8s
|
Run Playwright tests:
e2e-pw/[chromium] › oss/specs/selection.spec.ts#L1
e2e-pw/[chromium] › oss/specs/selection.spec.ts took 17.7s
|
Run Playwright tests:
e2e-pw/[chromium] › oss/specs/smoke-tests/tagger.spec.ts#L1
e2e-pw/[chromium] › oss/specs/smoke-tests/tagger.spec.ts took 16.7s
|
Cache playwright browser
Unexpected input(s) 'working-directory', valid inputs are ['path', 'key', 'restore-keys', 'upload-chunk-size', 'enableCrossOsArchive', 'fail-on-cache-miss', 'lookup-only', 'save-always']
|
Configure
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
|
Run Playwright tests
2 failed
[chromium] › oss/specs/grid-tagging.spec.ts:42:1 › grid tagging ────────────────────────────────
[chromium] › oss/specs/plugins/histograms.spec.ts:46:1 › histograms panel ──────────────────────
13 flaky
[chromium] › oss/specs/groups/dynamic-groups.spec.ts:70:3 › pcd dynamic group pagination bar ───
[chromium] › oss/specs/groups/dynamic-groups.spec.ts:70:3 › png dynamic group pagination bar ───
[chromium] › oss/specs/groups/nested-dynamic-groups.spec.ts:116:1 › dynamic groups of groups works
[chromium] › oss/specs/groups/sparse-groups.spec.ts:96:3 › mp4 second slice ────────────────────
[chromium] › oss/specs/groups/sparse-groups.spec.ts:96:3 › png second slice ────────────────────
[chromium] › oss/specs/regression-tests/group-video/group-video-label.spec.ts:87:3 › groups video labels › video plays with correct label for each slice
[chromium] › oss/specs/regression-tests/index-page.spec.ts:24:1 › index page ───────────────────
[chromium] › oss/specs/smoke-tests/embeddings.spec.ts:62:3 › embeddings on quickstart dataset › lasso samples work
[chromium] › oss/specs/smoke-tests/quickstart-groups.spec.ts:79:5 › quickstart-groups › modal › navigation works
[chromium] › oss/specs/smoke-tests/quickstart-groups.spec.ts:104:5 › quickstart-groups › modal › group media visibility toggle works
[chromium] › oss/specs/smoke-tests/quickstart-groups.spec.ts:139:3 › quickstart-groups › modal with grid filter
[chromium] › oss/specs/smoke-tests/quickstart.spec.ts:63:3 › quickstart › entry counts text when toPatches then groupedBy
[chromium] › oss/specs/smoke-tests/quickstart.spec.ts:80:3 › quickstart › selection bookmark ───
2 skipped
88 passed (21.5m)
|
Loading