High memory consumption from trace engine #16111
Comments
|
I think your assessment is correct. Invalidation events are extremely noisy and have led to problems in the past like #15869. The |
|
Hey @mattzeunert thanks for the report. Is it possible for you to share the website you were using that caused this high memory? Just to aid our debugging. I absolutely understand if it's private and you are unable to though. Thanks! |
copybara-service bot
pushed a commit
to ChromeDevTools/devtools-frontend
that referenced
this issue
Jul 12, 2024
I was investigating this bug in LH [GoogleChrome/lighthouse#16111] where the trace engine OOMs, and the reporter notes that the InvalidationsHandler map is 328MB in size. During my initial investigation I realised we were creating synthetic invalidations for no real purpose; all the information exists in the events as is, and instead we can just create a type to group them and everything gets much more simple. I don't expect this to resolve the problem in that bug report, but it is firstly cleaner to not create synthetic events if we don't need to, and it does also mean we are now not going to allocate a new object per every invalidation we want to store, so I expect this to be a positive change. Bug: 352680159 Change-Id: I4817774ce692c0dd73eb2156389148d7b756208c Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/5702081 Auto-Submit: Jack Franklin <jacktfranklin@chromium.org> Commit-Queue: Andres Olivares <andoli@chromium.org> Reviewed-by: Andres Olivares <andoli@chromium.org>
copybara-service bot
pushed a commit
to ChromeDevTools/devtools-frontend
that referenced
this issue
Aug 8, 2024
We have seen instances where traces with a large amount of invalidations can cause the trace engine to take a long time processing and hold onto a lot of memory - even causing LH to OOM in Node (GoogleChrome/lighthouse#16111) In these instances we've seen traces where one Layout event (as an example) can be linked to thousands of invalidations. Once you get to this point we aren't helping the user to debug by showing them a list of every single invalidation. Therefore, to fix the performance of this case, and to make the UI not a huge list, we now cap the number of invalidations we will track. We also count the total, and in the UI we now show this total. By making this a configurable value, we can also allow a user to disable Invalidations by setting it to 0. In this case we shortcut the handler and will not process any events. Bug: 352680159 Change-Id: I63c8dc41577e4a350788885a7bc23967ace0c10c Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/5766691 Commit-Queue: Jack Franklin <jacktfranklin@chromium.org> Auto-Submit: Jack Franklin <jacktfranklin@chromium.org> Reviewed-by: Adam Raine <asraine@chromium.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
FAQ
URL
No shareable repro
What happened?
Node ran out of memory when running Lighthouse.
The memory profiler shows that
invalidationsForEventuses 328 MB.The
invalidationsForEventmap has around 900 entries, but some of them have 10,000 or soSyntheticInvalidationevents.What did you expect?
Lighthouse should run without crashing the Node process.
What have you tried?
No response
How were you running Lighthouse?
node
Lighthouse Version
12.1.0
Chrome Version
No response
Node Version
No response
OS
No response
Relevant log output
No response
The text was updated successfully, but these errors were encountered: