Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(ci): shard e2e components tests #7581

Merged
merged 1 commit into from
Oct 4, 2024

Conversation

binoy14
Copy link
Contributor

@binoy14 binoy14 commented Oct 3, 2024

Description

This PR shards e2e components tests so they run on 2 shards per browser. This should significantly speed up the test run.

What to review

Changes makes sense and the tests pass

Testing

N/A

Notes for release

N/A

Copy link

vercel bot commented Oct 3, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
page-building-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Oct 3, 2024 8:23pm
performance-studio ✅ Ready (Inspect) Visit Preview Oct 3, 2024 8:23pm
test-compiled-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Oct 3, 2024 8:23pm
test-next-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Oct 3, 2024 8:23pm
test-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Oct 3, 2024 8:23pm
1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
studio-workshop ⬜️ Ignored (Inspect) Visit Preview Oct 3, 2024 8:23pm

Copy link
Contributor Author

binoy14 commented Oct 3, 2024

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @binoy14 and the rest of your teammates on Graphite Graphite

Copy link
Contributor

github-actions bot commented Oct 3, 2024

No changes to documentation

Copy link
Contributor

github-actions bot commented Oct 3, 2024

⚡️ Editor Performance Report

Updated Thu, 03 Oct 2024 20:40:22 GMT

Benchmark reference
latency of sanity@latest
experiment
latency of this branch
Δ (%)
latency difference
article (title) 18.0 efps (56ms) 18.5 efps (54ms) -2ms (-2.7%)
article (body) 60.1 efps (17ms) 63.3 efps (16ms) -1ms (-5.1%)
article (string inside object) 20.0 efps (50ms) 19.8 efps (51ms) +1ms (+1.0%)
article (string inside array) 15.3 efps (66ms) 15.4 efps (65ms) -1ms (-0.8%)
recipe (name) 31.3 efps (32ms) 31.3 efps (32ms) +0ms (-/-%)
recipe (description) 34.5 efps (29ms) 35.7 efps (28ms) -1ms (-3.4%)
recipe (instructions) 99.9+ efps (6ms) 99.9+ efps (6ms) +0ms (-/-%)
synthetic (title) 15.4 efps (65ms) 15.2 efps (66ms) +1ms (+1.5%)
synthetic (string inside object) 16.1 efps (62ms) 16.0 efps (63ms) +1ms (+0.8%)

efps — editor "frames per second". The number of updates assumed to be possible within a second.

Derived from input latency. efps = 1000 / input_latency

Detailed information

🏠 Reference result

The performance result of sanity@latest

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 56ms 59ms 67ms 235ms 1095ms 13.8s
article (body) 17ms 18ms 21ms 112ms 164ms 5.3s
article (string inside object) 50ms 53ms 59ms 148ms 685ms 8.3s
article (string inside array) 66ms 70ms 79ms 185ms 1631ms 9.9s
recipe (name) 32ms 33ms 36ms 76ms 27ms 9.1s
recipe (description) 29ms 30ms 36ms 86ms 5ms 6.3s
recipe (instructions) 6ms 8ms 8ms 10ms 0ms 3.2s
synthetic (title) 65ms 70ms 74ms 288ms 2091ms 16.5s
synthetic (string inside object) 62ms 70ms 81ms 311ms 1770ms 9.7s

🧪 Experiment result

The performance result of this branch

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 54ms 57ms 65ms 124ms 905ms 13.4s
article (body) 16ms 18ms 21ms 100ms 185ms 5.5s
article (string inside object) 51ms 53ms 60ms 127ms 684ms 8.4s
article (string inside array) 65ms 70ms 74ms 93ms 1494ms 9.5s
recipe (name) 32ms 34ms 37ms 64ms 40ms 9.1s
recipe (description) 28ms 30ms 33ms 56ms 18ms 6.0s
recipe (instructions) 6ms 8ms 9ms 11ms 0ms 3.3s
synthetic (title) 66ms 71ms 86ms 219ms 1963ms 16.7s
synthetic (string inside object) 63ms 67ms 75ms 336ms 1708ms 9.9s

📚 Glossary

column definitions

  • benchmark — the name of the test, e.g. "article", followed by the label of the field being measured, e.g. "(title)".
  • latency — the time between when a key was pressed and when it was rendered. derived from a set of samples. the median (p50) is shown to show the most common latency.
  • p75 — the 75th percentile of the input latency in the test run. 75% of the sampled inputs in this benchmark were processed faster than this value. this provides insight into the upper range of typical performance.
  • p90 — the 90th percentile of the input latency in the test run. 90% of the sampled inputs were faster than this. this metric helps identify slower interactions that occurred less frequently during the benchmark.
  • p99 — the 99th percentile of the input latency in the test run. only 1% of sampled inputs were slower than this. this represents the worst-case scenarios encountered during the benchmark, useful for identifying potential performance outliers.
  • blocking time — the total time during which the main thread was blocked, preventing user input and UI updates. this metric helps identify performance bottlenecks that may cause the interface to feel unresponsive.
  • test duration — how long the test run took to complete.

Copy link
Contributor

github-actions bot commented Oct 3, 2024

Component Testing Report Updated Oct 3, 2024 8:31 PM (UTC)

✅ All Tests Passed -- expand for details
File Status Duration Passed Skipped Failed
comments/CommentInput.spec.tsx ✅ Passed (Inspect) 1m 1s 15 0 0
formBuilder/ArrayInput.spec.tsx ✅ Passed (Inspect) 8s 3 0 0
formBuilder/inputs/PortableText/Annotations.spec.tsx ✅ Passed (Inspect) 32s 6 0 0
formBuilder/inputs/PortableText/copyPaste/CopyPaste.spec.tsx ✅ Passed (Inspect) 37s 11 7 0
formBuilder/inputs/PortableText/copyPaste/CopyPasteFields.spec.tsx ✅ Passed (Inspect) 0s 0 12 0
formBuilder/inputs/PortableText/Decorators.spec.tsx ✅ Passed (Inspect) 17s 6 0 0
formBuilder/inputs/PortableText/DisableFocusAndUnset.spec.tsx ✅ Passed (Inspect) 10s 3 0 0
formBuilder/inputs/PortableText/DragAndDrop.spec.tsx ✅ Passed (Inspect) 2m 33s 1 0 0
formBuilder/inputs/PortableText/FocusTracking.spec.tsx ✅ Passed (Inspect) 44s 15 0 0
formBuilder/inputs/PortableText/Input.spec.tsx ✅ Passed (Inspect) 1m 54s 21 0 0
formBuilder/inputs/PortableText/ObjectBlock.spec.tsx ✅ Passed (Inspect) 1m 18s 18 0 0
formBuilder/inputs/PortableText/PresenceCursors.spec.tsx ✅ Passed (Inspect) 9s 3 9 0
formBuilder/inputs/PortableText/RangeDecoration.spec.tsx ✅ Passed (Inspect) 27s 9 0 0
formBuilder/inputs/PortableText/Styles.spec.tsx ✅ Passed (Inspect) 18s 6 0 0
formBuilder/inputs/PortableText/Toolbar.spec.tsx ✅ Passed (Inspect) 36s 12 0 0
formBuilder/tree-editing/TreeEditing.spec.tsx ✅ Passed (Inspect) 0s 0 3 0
formBuilder/tree-editing/TreeEditingNestedObjects.spec.tsx ✅ Passed (Inspect) 0s 0 3 0

@binoy14 binoy14 marked this pull request as ready for review October 3, 2024 20:33
@binoy14 binoy14 requested a review from a team as a code owner October 3, 2024 20:33
@binoy14 binoy14 requested review from cngonzalez and removed request for a team October 3, 2024 20:33
@binoy14 binoy14 added this pull request to the merge queue Oct 4, 2024
Merged via the queue into next with commit 6036b3e Oct 4, 2024
64 checks passed
@binoy14 binoy14 deleted the 10-03-chore_ci_shard_e2e_components_tests branch October 4, 2024 14:12
ricokahler pushed a commit that referenced this pull request Oct 4, 2024
### Description

<!--
What changes are introduced?
Why are these changes introduced?
What issue(s) does this solve? (with link, if possible)
-->

This PR shards e2e components tests so they run on 2 shards per browser.
This should significantly speed up the test run.

### What to review

<!--
What steps should the reviewer take in order to review?
What parts/flows of the application/packages/tooling is affected?
-->

Changes makes sense and the tests pass

### Testing

<!--
Did you add sufficient testing for this change?
If not, please explain how you tested this change and why it was not
possible/practical for writing an automated test
-->

N/A

### Notes for release

<!--
Engineers do not need to worry about the final copy,
but they must provide the docs team with enough context on:

* What changed
* How does one use it (code snippets, etc)
* Are there limitations we should be aware of

If this is PR is a partial implementation of a feature and is not
enabled by default or if
this PR does not contain changes that needs mention in the release notes
(tooling chores etc),
please call this out explicitly by writing "Part of feature X" or "Not
required" in this section.
-->

N/A
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants