Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Switch performance tests to Playwright #52022

Merged
merged 46 commits into from
Aug 16, 2023
Merged

Switch performance tests to Playwright #52022

merged 46 commits into from
Aug 16, 2023

Commits on Jun 28, 2023

  1. Run with Playwright in CI

    @WunderBart
    WunderBart committed Jun 28, 2023
    Configuration menu
    Copy the full SHA
    a88918c View commit details
    Browse the repository at this point in the history

  2. Print stdout on error

    @WunderBart
    WunderBart committed Jun 28, 2023
    Configuration menu
    Copy the full SHA
    4ce6571 View commit details
    Browse the repository at this point in the history

  3. Run build:packages to use e2e utils

    @WunderBart
    WunderBart committed Jun 28, 2023
    Configuration menu
    Copy the full SHA
    f6d9851 View commit details
    Browse the repository at this point in the history

  4. Extend timeout for the canvas spinner

    @WunderBart
    WunderBart committed Jun 28, 2023
    Configuration menu
    Copy the full SHA
    6fde09d View commit details
    Browse the repository at this point in the history

  5. Add missing front end perf report

    @WunderBart
    WunderBart committed Jun 28, 2023
    Configuration menu
    Copy the full SHA
    6f28c60 View commit details
    Browse the repository at this point in the history

Commits on Jun 29, 2023

  1. Remove obsolete step 

    The first character is excluded from the results later on, so let's not double-exclude.
    @WunderBart
    WunderBart committed Jun 29, 2023
    Configuration menu
    Copy the full SHA
    b9b0440 View commit details
    Browse the repository at this point in the history

  2. Make the typing testing consistent

    @WunderBart
    WunderBart committed Jun 29, 2023
    Configuration menu
    Copy the full SHA
    ea6891f View commit details
    Browse the repository at this point in the history

Commits on Jul 4, 2023

  1. Improve post editor perf testing consistency 

    - define fixed time to wait until browser becomes idle instead random values
- make each test record 10 samples + 1 throwaway to address initial metric spikes
- improve overall code consistency
    @WunderBart
    WunderBart committed Jul 4, 2023
    Configuration menu
    Copy the full SHA
    aeae2ec View commit details
    Browse the repository at this point in the history

Commits on Jul 5, 2023

  1. Do not report slow tests

    @WunderBart
    WunderBart committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    309b9a2 View commit details
    Browse the repository at this point in the history

  2. Use the same methodology for measuring post and site editors metrics

    @WunderBart
    WunderBart committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    b47d2b7 View commit details
    Browse the repository at this point in the history

  3. Make the results table transformer more readable

    @WunderBart
    WunderBart committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    0427a37 View commit details
    Browse the repository at this point in the history

  4. Streamline the results processing p.1

    @WunderBart
    WunderBart committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    542456e View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2023

  1. Improve visual step separation a tad :)

    @WunderBart
    WunderBart committed Jul 6, 2023
    Configuration menu
    Copy the full SHA
    c06ab19 View commit details
    Browse the repository at this point in the history

  2. First step to separate metrics from the runner

    @WunderBart
    WunderBart committed Jul 6, 2023
    Configuration menu
    Copy the full SHA
    df1216e View commit details
    Browse the repository at this point in the history

  3. Define the results file suffix

    @WunderBart
    WunderBart committed Jul 6, 2023
    Configuration menu
    Copy the full SHA
    2e13186 View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'origin' into refactor/playwright-perfor… 

    …mance-tests-p2
    @WunderBart
    WunderBart committed Jul 6, 2023
    Configuration menu
    Copy the full SHA
    ed1e0cc View commit details
    Browse the repository at this point in the history

  5. cosmetic

    @WunderBart
    WunderBart committed Jul 6, 2023
    Configuration menu
    Copy the full SHA
    fecc1a0 View commit details
    Browse the repository at this point in the history

  6. Fix matching pattern typo

    @WunderBart
    WunderBart committed Jul 6, 2023
    Configuration menu
    Copy the full SHA
    4a394d8 View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2023

  1. Make performance.js comparison runner results-agnostic

    @WunderBart
    WunderBart committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    647ea57 View commit details
    Browse the repository at this point in the history

  2. Fix results handling

    @WunderBart
    WunderBart committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    1a18006 View commit details
    Browse the repository at this point in the history

  3. Use Playwright's attachments API to handle results

    @WunderBart
    WunderBart committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    f03e671 View commit details
    Browse the repository at this point in the history

  4. Upload raw results as well

    @WunderBart
    WunderBart committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    5db75c3 View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2023

  1. Be consistent with calc utils

    @WunderBart
    WunderBart committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    da43808 View commit details
    Browse the repository at this point in the history

  2. Streamline results handling

    @WunderBart
    WunderBart committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    00f641d View commit details
    Browse the repository at this point in the history

  3. Use the same sampling convention for front-end metrics

    @WunderBart
    WunderBart committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    2b9a5ac View commit details
    Browse the repository at this point in the history

  4. Fix the metrics output in perf runner

    @WunderBart
    WunderBart committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    1f8ecd1 View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2023

  1. tmp: compare performance with base branch

    @WunderBart
    WunderBart committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    d649aac View commit details
    Browse the repository at this point in the history

  2. tmp: compare performance with base branch (run 2)

    @WunderBart
    WunderBart committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    ae40542 View commit details
    Browse the repository at this point in the history

  3. tmp: compare performance with base branch (run 3)

    @WunderBart
    WunderBart committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    18fe132 View commit details
    Browse the repository at this point in the history

  4. Revert "tmp: compare performance with base branch" 

    This reverts commit d649aac.
    @WunderBart
    WunderBart committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    8f15066 View commit details
    Browse the repository at this point in the history

  5. Remove unused code and do the switch

    @WunderBart
    WunderBart committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    b1d7f83 View commit details
    Browse the repository at this point in the history

  6. Merge remote-tracking branch 'origin' into refactor/playwright-perfor… 

    …mance-tests-p2
    @WunderBart
    WunderBart committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    e37cbc2 View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2023

  1. Fix site editor typing spec

    @WunderBart
    WunderBart committed Jul 26, 2023
    Configuration menu
    Copy the full SHA
    2c305da View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin' into refactor/playwright-perfor… 

    …mance-tests-p2
    @WunderBart
    WunderBart committed Jul 26, 2023
    Configuration menu
    Copy the full SHA
    c9c625e View commit details
    Browse the repository at this point in the history

Commits on Jul 27, 2023

  1. Add inline comment

    @WunderBart
    WunderBart committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    bbae249 View commit details
    Browse the repository at this point in the history

  2. Expain why we need larger timeout for site editor visit util

    @WunderBart
    WunderBart committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    894896b View commit details
    Browse the repository at this point in the history

  3. Use dirent for getting files from dir

    @WunderBart
    WunderBart committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    7ac254e View commit details
    Browse the repository at this point in the history

  4. Do not use the browser.newPage shortcut 

    https://playwright.dev/docs/api/class-browser#browser-new-page
    @WunderBart
    WunderBart committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    84bb262 View commit details
    Browse the repository at this point in the history

  5. Merge remote-tracking branch 'origin' into refactor/playwright-perfor… 

    …mance-tests-p2
    @WunderBart
    WunderBart committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    182fbc1 View commit details
    Browse the repository at this point in the history

  6. Use trace buffer instead of tmp file

    @WunderBart
    WunderBart committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    1cf298a View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2023

  1. Merge remote-tracking branch 'origin' into refactor/playwright-perfor… 

    …mance-tests-p2
    @WunderBart
    WunderBart committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    f787f1e View commit details
    Browse the repository at this point in the history

  2. Fix 'unterminated string' error

    @WunderBart
    WunderBart committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    5a68bbc View commit details
    Browse the repository at this point in the history

  3. Fix 'unterminated string' error p.2

    @WunderBart
    WunderBart committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    d0d81fb View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2023

  1. Make sure sanitized branch name is used for reading results json

    @WunderBart
    WunderBart committed Aug 16, 2023
    Configuration menu
    Copy the full SHA
    37ef946 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin' into refactor/playwright-perfor… 

    …mance-tests-p2
    @WunderBart
    WunderBart committed Aug 16, 2023
    Configuration menu
    Copy the full SHA
    127b21e View commit details
    Browse the repository at this point in the history

  3. Prevent logging in via test.use() instead of creating new context 

    Each test is run with a fresh context by default, so we can just set the storageState via test.use instead.
    @WunderBart
    WunderBart committed Aug 16, 2023
    Configuration menu
    Copy the full SHA
    568d9d7 View commit details
    Browse the repository at this point in the history