Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve reporting and usability of UI benchmarking #6647

Open
3 tasks
chikeichan opened this issue May 23, 2019 · 2 comments
Open
3 tasks

Improve reporting and usability of UI benchmarking #6647

chikeichan opened this issue May 23, 2019 · 2 comments

Comments

@chikeichan
Copy link
Contributor

chikeichan commented May 23, 2019

We record various page load metrics on each Pull Request, added in #7871

However, as pointed out in the comments by @Gudahtt, these measurements are not stable and often vary wildly from build to build. We also do not record this data over time to get an eye for the impact of changes.

  • Investigate and stabilize large variance in load times (make them more reliable)
  • Track metrics overtime so that large changes in load time can be flagged
  • Automate benchmarking & integrate with build
@brad-decker
Copy link
Contributor

@danjm @Gudahtt do we think this is more of a tooling/testing (like using lighthouse or something in the build process) or an analytics/metrics issue?

@Gudahtt
Copy link
Member

Gudahtt commented Jan 21, 2022

The former. And actually we have already implemented this: #7871

Though in practice it's not very useful yet because we don't keep track of this over time, which we'd need to start doing in order to spot regressions. There are also a lot of improvements we could make to make the measurement more stable (it can vary based on various factors at the moment that are outside of our control, so we might get "false positive" performance regressions).

@brad-decker brad-decker changed the title Add new analystics to log load time Improve reporting and usability of UI benchmarking Jan 21, 2022
@brad-decker brad-decker added area-buildSystem related to our build system area-testSuite and removed area-metrics area-buildSystem related to our build system labels Jan 21, 2022
@hilvmason hilvmason added the MV3 label May 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants