Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build: Improve waiting detection on E2E to fix flake #30057

Merged
merged 3 commits into from
Dec 15, 2024

Conversation

yannbf
Copy link
Member

@yannbf yannbf commented Dec 13, 2024

Closes #

What I did

Ongoing effort to fix E2E tests flake

Checklist for Contributors

Testing

The changes in this PR are covered in the following automated tests:

  • stories
  • unit tests
  • integration tests
  • end-to-end tests

Manual testing

This section is mandatory for all contributions. If you believe no manual test is necessary, please state so explicitly. Thanks!

Documentation

  • Add or update documentation reflecting your changes
  • If you are deprecating/removing a feature, make sure to update
    MIGRATION.MD

Checklist for Maintainers

  • When this PR is ready for testing, make sure to add ci:normal, ci:merged or ci:daily GH label to it to run a specific set of sandboxes. The particular set of sandboxes can be found in code/lib/cli-storybook/src/sandbox-templates.ts

  • Make sure this PR contains one of the labels below:

    Available labels
    • bug: Internal changes that fixes incorrect behavior.
    • maintenance: User-facing maintenance tasks.
    • dependencies: Upgrading (sometimes downgrading) dependencies.
    • build: Internal-facing build tooling & test updates. Will not show up in release changelog.
    • cleanup: Minor cleanup style change. Will not show up in release changelog.
    • documentation: Documentation only changes. Will not show up in release changelog.
    • feature request: Introducing a new feature.
    • BREAKING CHANGE: Changes that break compatibility in some way with current major version.
    • other: Changes that don't fit in the above categories.

🦋 Canary release

This PR does not have a canary release associated. You can request a canary release of this pull request by mentioning the @storybookjs/core team here.

core team members can create a canary release here or locally with gh workflow run --repo storybookjs/storybook canary-release-pr.yml --field pr=<PR_NUMBER>

name before after diff z %
createSize 0 B 0 B 0 B - -
generateSize 77.7 MB 77.7 MB -15 B 0.32 0%
initSize 133 MB 133 MB -5.99 kB 1.04 0%
diffSize 55.3 MB 55.3 MB -5.98 kB 1.08 0%
buildSize 7.24 MB 6.87 MB -364 kB -0.33 -5.3%
buildSbAddonsSize 1.88 MB 1.51 MB -363 kB -0.33 -24%
buildSbCommonSize 195 kB 195 kB 0 B - 0%
buildSbManagerSize 1.86 MB 1.86 MB -960 B -0.16 -0.1%
buildSbPreviewSize 0 B 0 B 0 B - -
buildStaticSize 0 B 0 B 0 B - -
buildPrebuildSize 3.93 MB 3.57 MB -364 kB -0.33 -10.2%
buildPreviewSize 3.3 MB 3.3 MB -168 B -0.97 0%
testBuildSize 0 B 0 B 0 B - -
testBuildSbAddonsSize 0 B 0 B 0 B - -
testBuildSbCommonSize 0 B 0 B 0 B - -
testBuildSbManagerSize 0 B 0 B 0 B - -
testBuildSbPreviewSize 0 B 0 B 0 B - -
testBuildStaticSize 0 B 0 B 0 B - -
testBuildPrebuildSize 0 B 0 B 0 B - -
testBuildPreviewSize 0 B 0 B 0 B - -
name before after diff z %
createTime 7s 27s 19.9s 1.81 🔺73.8%
generateTime 19.5s 20.1s 574ms -0.65 2.9%
initTime 12.4s 15.2s 2.8s 0.3 18.8%
buildTime 8.7s 10.6s 1.9s 2.56 🔺18.1%
testBuildTime 0ms 0ms 0ms - -
devPreviewResponsive 4.6s 4.8s 127ms -0.65 2.6%
devManagerResponsive 3.6s 3.6s 12ms -0.66 0.3%
devManagerHeaderVisible 605ms 551ms -54ms -0.39 -9.8%
devManagerIndexVisible 631ms 858ms 227ms 2.28 🔺26.5%
devStoryVisibleUncached 1.7s 1.7s 8ms -0.06 0.4%
devStoryVisible 629ms 580ms -49ms -0.7 -8.4%
devAutodocsVisible 604ms 494ms -110ms -0.37 -22.3%
devMDXVisible 491ms 488ms -3ms -0.42 -0.6%
buildManagerHeaderVisible 573ms 508ms -65ms -0.89 -12.8%
buildManagerIndexVisible 665ms 601ms -64ms -0.88 -10.6%
buildStoryVisible 535ms 473ms -62ms -0.78 -13.1%
buildAutodocsVisible 443ms 431ms -12ms -0.35 -2.8%
buildMDXVisible 431ms 401ms -30ms -0.64 -7.5%

Greptile Summary

Enhanced the waiting detection in Storybook's E2E test utility to ensure UI elements are fully rendered before test interactions begin, reducing test flakiness particularly in Vite projects.

  • Modified waitUntilLoaded method in code/e2e-tests/util.ts to verify #storybook-root has child elements
  • Added additional wait condition to ensure complete UI rendering before test execution
  • Targets specifically flaky behavior in Vite project tests
  • Maintains minimal, focused change to core E2E test infrastructure

@yannbf yannbf added the build Internal-facing build tooling & test updates label Dec 13, 2024
@yannbf yannbf self-assigned this Dec 13, 2024
@yannbf yannbf added the ci:merged Run the CI jobs that normally run when merged. label Dec 13, 2024
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

1 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings | Greptile

Copy link

nx-cloud bot commented Dec 13, 2024

☁️ Nx Cloud Report

CI is running/has finished running commands for commit a30a5f3. As they complete they will appear below. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this CI Pipeline Execution


✅ Successfully ran 1 target

Sent with 💌 from NxCloud.

@storybook-pr-benchmarking
Copy link

storybook-pr-benchmarking bot commented Dec 13, 2024

Package Benchmarks

Commit: a30a5f3, ran on 13 December 2024 at 18:53:15 UTC

No significant changes detected, all good. 👏

@yannbf yannbf force-pushed the yann/fix-e2e-flake-vite-projects branch from 8feffc1 to 691f002 Compare December 13, 2024 15:12
@yannbf yannbf force-pushed the yann/fix-e2e-flake-vite-projects branch from 691f002 to a30a5f3 Compare December 13, 2024 18:43
@valentinpalkovic valentinpalkovic merged commit 22bcecd into next Dec 15, 2024
68 of 70 checks passed
@valentinpalkovic valentinpalkovic deleted the yann/fix-e2e-flake-vite-projects branch December 15, 2024 09:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Internal-facing build tooling & test updates ci:merged Run the CI jobs that normally run when merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants