Skip to content

Add Model Evaluation Panel #5975

Add Model Evaluation Panel

Add Model Evaluation Panel #5975

Triggered via pull request October 30, 2024 14:35
Status Failure
Total duration 29m 41s
Artifacts 2

pr.yml

on: pull_request
Matrix: test / test-python
all-tests
0s
all-tests
Fit to window
Zoom out
Zoom in

Annotations

10 errors, 21 warnings, and 1 notice
[chromium] › oss/specs/grid-tagging.spec.ts:42:1 › grid tagging: e2e-pw/src/oss/specs/grid-tagging.spec.ts#L1
1) [chromium] › oss/specs/grid-tagging.spec.ts:42:1 › grid tagging ─────────────────────────────── Test timeout of 60000ms exceeded.
[chromium] › oss/specs/grid-tagging.spec.ts:42:1 › grid tagging: e2e-pw/src/oss/poms/grid/index.ts#L70
1) [chromium] › oss/specs/grid-tagging.spec.ts:42:1 › grid tagging ─────────────────────────────── Error: locator.scrollIntoViewIfNeeded: Test timeout of 60000ms exceeded. Call log: - attempting scroll into view action - waiting for element to be stable - element is not visible - retrying scroll into view action, attempt #1 - waiting for element to be stable - element is not visible - retrying scroll into view action, attempt #2 - waiting 20ms - waiting for element to be stable - element is not visible - retrying scroll into view action, attempt #3 - waiting 100ms - waiting for element to be stable - element is not visible - retrying scroll into view action, attempt #4 - waiting 100ms - waiting for element to be stable - element is not visible - retrying scroll into view action, attempt #5 - waiting 500ms - waiting for element to be stable - element is not visible - retrying scroll into view action, attempt #6 - waiting 500ms - waiting for element to be stable - element is not visible - retrying scroll into view action, attempt #7 - waiting 500ms - waiting for element to be stable - element is not visible - retrying scroll into view action, attempt #8 - waiting 500ms - waiting for element to be stable - element is not visible - retrying scroll into view action, attempt #9 - waiting 500ms - waiting for element to be stable - element is not visible - retrying scroll into view action, attempt #10 - waiting 500ms - waiting for element to be stable - element is not visible - retrying scroll into view action, attempt #11 - waiting 500ms - waiting for element to be stable - element is not visible - retrying scroll into view action, attempt #12 - waiting 500ms - waiting for element to be stable - element is not visible - retrying scroll into view action, attempt #13 - waiting 500ms - waiting for element to be stable - element is not visible - retrying scroll into view action, attempt #14 - waiting 500ms - waiting for element to be stable - element is not visible - retrying scroll into view action, attempt #15 - waiting 500ms - waiting for element to be stable - element is not visible - retrying scroll into view action, attempt #16 - waiting 500ms - waiting for element to be stable - element is not visible - retrying scroll into view action, attempt #17 - waiting 500ms - waiting for element to be stable - element is not visible - retrying scroll into view action, attempt #18 - waiting 500ms - waiting for element to be stable - element is not visible - retrying scroll into view action, attempt #19 - waiting 500ms - waiting for element to be stable - element is not visible - retrying scroll into view action, attempt #20 - waiting 500ms - waiting for element to be stable - element is not visible - retrying scroll into view action, attempt #21 - waiting 500ms - waiting for element to be stable - element is not visible - retrying scroll into view action, attempt #22 - waiting 500ms - waiting for element to be stable - element is not visible - retrying scroll into view action, attempt #23 - waiting 500ms - waiting for element to be stable - element is not visible - retrying scroll into view action, attempt #24 - waiting 500ms - waiting for element to be stable - element is not visible - retrying scroll into view action, attempt #25 - waiting 500ms - waiting for element to be stable
[chromium] › oss/specs/grid-tagging.spec.ts:42:1 › grid tagging: e2e-pw/src/oss/specs/grid-tagging.spec.ts#L47
1) [chromium] › oss/specs/grid-tagging.spec.ts:42:1 › grid tagging ─────────────────────────────── Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: Screenshot comparison failed: 6416 pixels (ratio 0.02 of all image pixels) are different. Expected: /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/grid-tagging.spec.ts-snapshots/grid-untagged-chromium-linux.png Received: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-grid-tagging-grid-tagging-chromium-retry1/grid-untagged-actual.png Diff: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-grid-tagging-grid-tagging-chromium-retry1/grid-untagged-diff.png Call log: - expect.toHaveScreenshot(grid-untagged.png) with timeout 5000ms - verifying given screenshot expectation - waiting for getByTestId('fo-grid') - locator resolved to <div data-cy="fo-grid" class="_spotlightGrid_1sk68_42" id="2e237129-4837-45c7-b033-ffb18fd4eefb">…</div> - taking element screenshot - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - 6416 pixels (ratio 0.02 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for getByTestId('fo-grid') - locator resolved to <div data-cy="fo-grid" class="_spotlightGrid_1sk68_42" id="2e237129-4837-45c7-b033-ffb18fd4eefb">…</div> - taking element screenshot - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - captured a stable screenshot - 6416 pixels (ratio 0.02 of all image pixels) are different. 45 | await sidebar.clickFieldCheckbox("tags"); 46 | await grid.scrollBottom(); > 47 | await expect(await grid.locator).toHaveScreenshot("grid-untagged.png", { | ^ 48 | animations: "allow", 49 | }); 50 | at /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/grid-tagging.spec.ts:47:36
[chromium] › oss/specs/grid-tagging.spec.ts:42:1 › grid tagging: e2e-pw/src/oss/specs/grid-tagging.spec.ts#L47
1) [chromium] › oss/specs/grid-tagging.spec.ts:42:1 › grid tagging ─────────────────────────────── Retry #2 ─────────────────────────────────────────────────────────────────────────────────────── Error: Screenshot comparison failed: 6416 pixels (ratio 0.02 of all image pixels) are different. Expected: /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/grid-tagging.spec.ts-snapshots/grid-untagged-chromium-linux.png Received: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-grid-tagging-grid-tagging-chromium-retry2/grid-untagged-actual.png Diff: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-grid-tagging-grid-tagging-chromium-retry2/grid-untagged-diff.png Call log: - expect.toHaveScreenshot(grid-untagged.png) with timeout 5000ms - verifying given screenshot expectation - waiting for getByTestId('fo-grid') - locator resolved to <div data-cy="fo-grid" class="_spotlightGrid_1sk68_42" id="fe83b5fd-0965-4d6a-ac84-5d58932819d9">…</div> - taking element screenshot - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - 6416 pixels (ratio 0.02 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for getByTestId('fo-grid') - locator resolved to <div data-cy="fo-grid" class="_spotlightGrid_1sk68_42" id="fe83b5fd-0965-4d6a-ac84-5d58932819d9">…</div> - taking element screenshot - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - captured a stable screenshot - 6416 pixels (ratio 0.02 of all image pixels) are different. 45 | await sidebar.clickFieldCheckbox("tags"); 46 | await grid.scrollBottom(); > 47 | await expect(await grid.locator).toHaveScreenshot("grid-untagged.png", { | ^ 48 | animations: "allow", 49 | }); 50 | at /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/grid-tagging.spec.ts:47:36
[chromium] › oss/specs/grid-tagging.spec.ts:42:1 › grid tagging: e2e-pw/src/oss/specs/grid-tagging.spec.ts#L47
1) [chromium] › oss/specs/grid-tagging.spec.ts:42:1 › grid tagging ─────────────────────────────── Retry #3 ─────────────────────────────────────────────────────────────────────────────────────── Error: Screenshot comparison failed: 6416 pixels (ratio 0.02 of all image pixels) are different. Expected: /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/grid-tagging.spec.ts-snapshots/grid-untagged-chromium-linux.png Received: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-grid-tagging-grid-tagging-chromium-retry3/grid-untagged-actual.png Diff: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-grid-tagging-grid-tagging-chromium-retry3/grid-untagged-diff.png Call log: - expect.toHaveScreenshot(grid-untagged.png) with timeout 5000ms - verifying given screenshot expectation - waiting for getByTestId('fo-grid') - locator resolved to <div data-cy="fo-grid" class="_spotlightGrid_1sk68_42" id="6e70389b-6343-4997-a78c-c8d8cb22db76">…</div> - taking element screenshot - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - 6416 pixels (ratio 0.02 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for getByTestId('fo-grid') - locator resolved to <div data-cy="fo-grid" class="_spotlightGrid_1sk68_42" id="6e70389b-6343-4997-a78c-c8d8cb22db76">…</div> - taking element screenshot - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - captured a stable screenshot - 6416 pixels (ratio 0.02 of all image pixels) are different. 45 | await sidebar.clickFieldCheckbox("tags"); 46 | await grid.scrollBottom(); > 47 | await expect(await grid.locator).toHaveScreenshot("grid-untagged.png", { | ^ 48 | animations: "allow", 49 | }); 50 | at /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/grid-tagging.spec.ts:47:36
[chromium] › oss/specs/plugins/histograms.spec.ts:46:1 › histograms panel: e2e-pw/src/oss/specs/plugins/histograms.spec.ts#L78
2) [chromium] › oss/specs/plugins/histograms.spec.ts:46:1 › histograms panel ───────────────────── Error: Screenshot comparison failed: 6688 pixels (ratio 0.02 of all image pixels) are different. Expected: /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/plugins/histograms.spec.ts-snapshots/bool-histogram-chromium-linux.png Received: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-plugins-histograms-histograms-panel-chromium/bool-histogram-actual.png Diff: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-plugins-histograms-histograms-panel-chromium/bool-histogram-diff.png Call log: - expect.toHaveScreenshot(bool-histogram.png) with timeout 5000ms - verifying given screenshot expectation - waiting for getByTestId('histograms-container') - locator resolved to <div class="sc-cgMlnP iMhHXS" data-cy="histograms-container">…</div> - taking element screenshot - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - 6688 pixels (ratio 0.02 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for getByTestId('histograms-container') - locator resolved to <div class="sc-cgMlnP iMhHXS" data-cy="histograms-container">…</div> - taking element screenshot - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - captured a stable screenshot - 6688 pixels (ratio 0.02 of all image pixels) are different. 76 | "tags", 77 | ]); > 78 | await expect(await histogram.locator).toHaveScreenshot("bool-histogram.png", { | ^ 79 | animations: "allow", 80 | }); 81 | at /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/plugins/histograms.spec.ts:78:41
[chromium] › oss/specs/plugins/histograms.spec.ts:46:1 › histograms panel: e2e-pw/src/oss/specs/plugins/histograms.spec.ts#L78
2) [chromium] › oss/specs/plugins/histograms.spec.ts:46:1 › histograms panel ───────────────────── Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: Screenshot comparison failed: 6688 pixels (ratio 0.02 of all image pixels) are different. Expected: /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/plugins/histograms.spec.ts-snapshots/bool-histogram-chromium-linux.png Received: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-plugins-histograms-histograms-panel-chromium-retry1/bool-histogram-actual.png Diff: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-plugins-histograms-histograms-panel-chromium-retry1/bool-histogram-diff.png Call log: - expect.toHaveScreenshot(bool-histogram.png) with timeout 5000ms - verifying given screenshot expectation - waiting for getByTestId('histograms-container') - locator resolved to <div class="sc-cgMlnP iMhHXS" data-cy="histograms-container">…</div> - taking element screenshot - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - 6688 pixels (ratio 0.02 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for getByTestId('histograms-container') - locator resolved to <div class="sc-cgMlnP iMhHXS" data-cy="histograms-container">…</div> - taking element screenshot - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - captured a stable screenshot - 6688 pixels (ratio 0.02 of all image pixels) are different. 76 | "tags", 77 | ]); > 78 | await expect(await histogram.locator).toHaveScreenshot("bool-histogram.png", { | ^ 79 | animations: "allow", 80 | }); 81 | at /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/plugins/histograms.spec.ts:78:41
[chromium] › oss/specs/plugins/histograms.spec.ts:46:1 › histograms panel: e2e-pw/src/oss/specs/plugins/histograms.spec.ts#L78
2) [chromium] › oss/specs/plugins/histograms.spec.ts:46:1 › histograms panel ───────────────────── Retry #2 ─────────────────────────────────────────────────────────────────────────────────────── Error: Screenshot comparison failed: 6688 pixels (ratio 0.02 of all image pixels) are different. Expected: /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/plugins/histograms.spec.ts-snapshots/bool-histogram-chromium-linux.png Received: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-plugins-histograms-histograms-panel-chromium-retry2/bool-histogram-actual.png Diff: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-plugins-histograms-histograms-panel-chromium-retry2/bool-histogram-diff.png Call log: - expect.toHaveScreenshot(bool-histogram.png) with timeout 5000ms - verifying given screenshot expectation - waiting for getByTestId('histograms-container') - locator resolved to <div class="sc-cgMlnP iMhHXS" data-cy="histograms-container">…</div> - taking element screenshot - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - 6688 pixels (ratio 0.02 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for getByTestId('histograms-container') - locator resolved to <div class="sc-cgMlnP iMhHXS" data-cy="histograms-container">…</div> - taking element screenshot - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - captured a stable screenshot - 6688 pixels (ratio 0.02 of all image pixels) are different. 76 | "tags", 77 | ]); > 78 | await expect(await histogram.locator).toHaveScreenshot("bool-histogram.png", { | ^ 79 | animations: "allow", 80 | }); 81 | at /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/plugins/histograms.spec.ts:78:41
[chromium] › oss/specs/plugins/histograms.spec.ts:46:1 › histograms panel: e2e-pw/src/oss/specs/plugins/histograms.spec.ts#L78
2) [chromium] › oss/specs/plugins/histograms.spec.ts:46:1 › histograms panel ───────────────────── Retry #3 ─────────────────────────────────────────────────────────────────────────────────────── Error: Screenshot comparison failed: 6688 pixels (ratio 0.02 of all image pixels) are different. Expected: /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/plugins/histograms.spec.ts-snapshots/bool-histogram-chromium-linux.png Received: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-plugins-histograms-histograms-panel-chromium-retry3/bool-histogram-actual.png Diff: /home/runner/work/fiftyone/fiftyone/e2e-pw/test-results/oss-specs-plugins-histograms-histograms-panel-chromium-retry3/bool-histogram-diff.png Call log: - expect.toHaveScreenshot(bool-histogram.png) with timeout 5000ms - verifying given screenshot expectation - waiting for getByTestId('histograms-container') - locator resolved to <div class="sc-cgMlnP iMhHXS" data-cy="histograms-container">…</div> - taking element screenshot - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - 6688 pixels (ratio 0.02 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for getByTestId('histograms-container') - locator resolved to <div class="sc-cgMlnP iMhHXS" data-cy="histograms-container">…</div> - taking element screenshot - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - captured a stable screenshot - 6688 pixels (ratio 0.02 of all image pixels) are different. 76 | "tags", 77 | ]); > 78 | await expect(await histogram.locator).toHaveScreenshot("bool-histogram.png", { | ^ 79 | animations: "allow", 80 | }); 81 | at /home/runner/work/fiftyone/fiftyone/e2e-pw/src/oss/specs/plugins/histograms.spec.ts:78:41
[chromium] › oss/specs/groups/dynamic-groups.spec.ts:70:3 › pcd dynamic group pagination bar: e2e-pw/src/oss/specs/groups/dynamic-groups.spec.ts#L1
3) [chromium] › oss/specs/groups/dynamic-groups.spec.ts:70:3 › pcd dynamic group pagination bar ── Test timeout of 60000ms exceeded.
lint / eslint: app/packages/operators/src/CustomPanel.tsx#L40
React Hook useEffect has missing dependencies: 'panelId', 'panelName', 'setPanelCloseEffect', and 'trackEvent'. Either include them or remove the dependency array
lint / eslint: app/packages/operators/src/CustomPanel.tsx#L104
React Hook useEffect has a missing dependency: 'dimensions'. Either include it or remove the dependency array
lint / eslint: app/packages/operators/src/built-in-operators.ts#L815
'ctx' is defined but never used. Allowed unused args must match /^_/u
lint / eslint
'ctx' is defined but never used. Allowed unused args must match /^_/u
lint / eslint
'ctx' is defined but never used. Allowed unused args must match /^_/u
lint / eslint
'ctx' is defined but never used. Allowed unused args must match /^_/u
lint / eslint
'ctx' is defined but never used. Allowed unused args must match /^_/u
lint / eslint
'path' is never reassigned. Use 'const' instead
lint / eslint
'value' is never reassigned. Use 'const' instead
lint / eslint
'ctx' is defined but never used. Allowed unused args must match /^_/u
test / test-python (ubuntu-latest-m, 3.9)
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
test / test-python (ubuntu-latest-m, 3.10)
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
test / test-python (ubuntu-latest-m, 3.11)
Codecov: Failed to properly create commit: The process '/home/runner/work/_actions/codecov/codecov-action/v4/dist/codecov' failed with exit code 1
test / test-python (ubuntu-latest-m, 3.11)
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
Slow Test: e2e-pw/[chromium] › oss/specs/smoke-tests/field-visibility.spec.ts#L1
e2e-pw/[chromium] › oss/specs/smoke-tests/field-visibility.spec.ts took 1.1m
Slow Test: e2e-pw/[chromium] › oss/specs/smoke-tests/saved-views.spec.ts#L1
e2e-pw/[chromium] › oss/specs/smoke-tests/saved-views.spec.ts took 51.6s
Slow Test: e2e-pw/[chromium] › oss/specs/selection.spec.ts#L1
e2e-pw/[chromium] › oss/specs/selection.spec.ts took 21.0s
Slow Test: e2e-pw/[chromium] › oss/specs/smoke-tests/tagger.spec.ts#L1
e2e-pw/[chromium] › oss/specs/smoke-tests/tagger.spec.ts took 18.7s
Slow Test: e2e-pw/[chromium] › oss/specs/sidebar/sidebar.spec.ts#L1
e2e-pw/[chromium] › oss/specs/sidebar/sidebar.spec.ts took 18.5s
e2e / test-e2e
Unexpected input(s) 'working-directory', valid inputs are ['path', 'key', 'restore-keys', 'upload-chunk-size', 'enableCrossOsArchive', 'fail-on-cache-miss', 'lookup-only', 'save-always']
e2e / test-e2e
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
🎭 Playwright Run Summary
2 failed [chromium] › oss/specs/grid-tagging.spec.ts:42:1 › grid tagging ──────────────────────────────── [chromium] › oss/specs/plugins/histograms.spec.ts:46:1 › histograms panel ────────────────────── 12 flaky [chromium] › oss/specs/groups/dynamic-groups.spec.ts:70:3 › pcd dynamic group pagination bar ─── [chromium] › oss/specs/groups/dynamic-groups.spec.ts:70:3 › png dynamic group pagination bar ─── [chromium] › oss/specs/groups/sparse-groups.spec.ts:96:3 › mp4 second slice ──────────────────── [chromium] › oss/specs/groups/sparse-groups.spec.ts:96:3 › png second slice ──────────────────── [chromium] › oss/specs/regression-tests/group-video/group-video-label.spec.ts:87:3 › groups video labels › video plays with correct label for each slice [chromium] › oss/specs/regression-tests/index-page.spec.ts:24:1 › index page ─────────────────── [chromium] › oss/specs/smoke-tests/embeddings.spec.ts:62:3 › embeddings on quickstart dataset › lasso samples work [chromium] › oss/specs/smoke-tests/quickstart-groups.spec.ts:79:5 › quickstart-groups › modal › navigation works [chromium] › oss/specs/smoke-tests/quickstart-groups.spec.ts:104:5 › quickstart-groups › modal › group media visibility toggle works [chromium] › oss/specs/smoke-tests/quickstart-groups.spec.ts:139:3 › quickstart-groups › modal with grid filter [chromium] › oss/specs/smoke-tests/quickstart.spec.ts:63:3 › quickstart › entry counts text when toPatches then groupedBy [chromium] › oss/specs/smoke-tests/quickstart.spec.ts:80:3 › quickstart › selection bookmark ─── 2 skipped 89 passed (22.3m)

Artifacts

Produced during runtime
Name Size
dist
19.8 MB
playwright-report
82.4 MB