Cypress Open Crashing Chrome with "Aw Snap" after upgrade to Chrome 100

[dwaynegreen]

Current behavior

When running Cypress Open with multiple tests, Chrome 100 is more likely to crash with an "Aw, Snap" error message than previous versions of Chrome:

I was able to determine that Chrome 100 is more likely to crash by using a Chromium version from https://chromium.cypress.io/. I was then able to run the standard Cypress Tests examples with Chrome 99.0.4844.0 by specifying "npx cypress open --browser "pathToChromiumVersion\chrome.exe"

Here is three screenshots from running cypress open showing memory footprint with Chrome 99 using the Cypress Examples:

Here is three screenshots from running cypress open showing memory footprint with Chrome 100 using the Cypress Examples:

Desired behavior

I believe that Cypress should be able to keep a similar memory footprint between Chrome 99 and Chrome 100 or provide a better way to garbage collect between tests.
I have looked at using numTestsKeptInMemory (https://docs.cypress.io/guides/references/configuration#Options) but we only have 14 tests in our build verification suite.

I would also like to have the numTestsKeptInMemory configurable to only keep tests that have failed in memory instead of the last X tests.

Test code to reproduce

Using Cypress Version 9.5.4 (as well as other versions), Cypress Examples that are installed with cypress on npm install -D cypress

Cypress Version

9.5.4

Other

No response

[rockhold]

@davidmunechika, did you have a chance to reproduce this against the 10.0-release branch? Just curious if it'll still be an issue before long.

[BlueWinds]

@rockhold - I've confirmed this for both 9.x and 10.x. Chrome/Chromium 100/101 with Cypress seems to use more memory than previous versions.

I did a bit of searching around Chromium bugs, and the best I've found is https://bugs.chromium.org/p/chromium/issues/detail?id=1309201#c9, which seems to indicate that the Chrome team is aware of some ongoing memory issues introduced around Chrome 100 and not yet resolved in 101; I don't know enough of the internals to make complete sense of this conversation.

With that said, I suspect that this in a Chromium regression, which Cypress will be powerless to resolve. :( Perhaps 102 or 103 will contain more solutions.

[BlueWinds]

The issue they mention related to extensions::Event::DeepCopy in particular seems like a likely culprit - Cypress is deeply bound up with browser events, and we create a ton of event for our internal use, far more than most "real sites" ever would. If there's a memory leak there, Cypress is perfectly positioned to pound it repeatedly.

[brian-mann]

If that's the case then maybe we shouldn't update the embedded Chromium in Electron to v100 yet...

@BlueWinds can you check our perf charts and see if there's anything egregious in there

[mjhenkes]

@brian-mann, we have already updated chromium in electron to v100 #21418

[extracker-ppetrov]

I can still reproduce this on Cypress 10.0.3 using Chrome.
My "tests kept in memory" setting is set to 2.
It crashes if I run more than 2 tests in a single run.

If the test kept in memory is set to 0, it is all good, no crashes with any browser.
If the "tests kept in memory" setting is set to 2 and the browser is set to Electron, it is also all good, no crashes.

[Livven]

Seeing this very reproducibly on certain tests with Cypress 12.9.0, both Chrome 112 and Electron 106. Firefox works flawlessly though 🦊

[EmreCihanbeyoglu]

Any update, improvement?

[DobQA]

This happens to me on latest version of Cypress - its not a machine resource issue, running latest Chrome 64GB Ram and latest Intel chip. I notice its todo perhaps with the site being tested, site using Angular Framework, happens at certain points.

[nagash77]

Have folks tried using the config flag experimentalMemoryManagement link to docs which was added in 12.4.0? I would try setting that to true and seeing if that helps with your specific use case. One note - if you are running a large single test this change will likely not improve performance for you and breaking your large test into smaller tests may be beneficial.

[nagash77]

It's been a couple of weeks and I have not heard back from folks. I am going to close this issue. If you are still encountering this issue after turning on that config flag, please open a new issue and we will be happy to investigate. Please note this issue in your comment.

[brandonLyonMM]

@nagash77 you said that if we are running a large single test this change will likely not improve performance for us and breaking a large test into smaller tests may be beneficial. Since that was true for me, I did not reply. I was trying to run a large single test. Unfortunately breaking the test into smaller tests is not an option for me in this case.

[nagash77]

@brandonLyonMM please open a new issue with your specific use case and we will be happy to investigate. Please mention this ticket in your issue.

[nathan5x]

I'm running into a similar issue where a single test does many things, and it is hard to break the flow.
@brandonLyonMM - Did you open a new issue?

[brandonLyonMM]

I'm running into a similar issue where a single test does many things, and it is hard to break the flow. @brandonLyonMM - Did you open a new issue?

No I did not

[extracker-ppetrov]

I am still facing the same issue after 12.4.0.
The way I started writing my tests is by grouping steps from one test in separate It()s.
Each spec file is a single test for me and the steps inside are broken in separate It() This way I can control the crashes.

[andreynav]

I am still facing the same issue after 12.4.0.
The way I started writing my tests is by grouping steps from one test in separate It()s.
Each spec file is a single test for me and the steps inside are broken in separate It() This way I can control the crashes.

Agree!
I faced the same crashes if I run API test in cycle

it('GET request to each region should have status 200', () => {
    cy.get<RegionsT>('@regionsData').then((regionsData) => {
      regionsData.regions.forEach((item) => {
        cy.api(`https://restcountries.com/v3.1/region/${item.region}`).then((response) => {
          expect(response.status).eq(200)
        })
      })
    })
  })

There are only 5 regions and almost in 50% of runs browser is crashes!

If I separate this test to 6 tests

it('GET request to a region should have status 200', () => {
  cy.api(`https://restcountries.com/v3.1/region/Africa`).then((response) => {
    expect(response.status).eq(200)
  })
})

tests pass OK

@nagash77 please take a look at this crashes! People still have the problems!

[mike-plummer]

@andreynav Unfortunately your reproduction is incomplete so I am unable to review it to identify what your issue is - the shape of the @regionsData alias could have major impacts on how many service calls actually get made here. In addition, in your example you don't actually need to call a GET on that endpoint since you're just asserting that the endpoint returns a 200 - you could instead use a HEAD request which has massive memory and performance improvements.

If you have a complete reproduction that you believe demonstrates a legitimate memory issue in Cypress I invite you to open a new issue so we can investigate

[extracker-ppetrov]

@mike-plummer
We all have valid tests that fail with the chrome browser blowing up with 'out of memory' If you create a long enough E2E test that deals with UI elements only, and monitor the chrome browser in DevTools, you will see that the memory consumption grows with insane rates.

https://www.loom.com/share/0f4c387b11964f57a4c4ddf90ec5547e

Once the memory consumption reaches 6-7GB, the browser crashes. Same with Electron, same with Edge.

I do not see a valid reason to close this ticket, when all these people are still facing the same issue.

[mike-plummer]

@extracker-ppetrov I've reopened as requested - can you provide a reproduction case that demonstrates the issue while following Cypress' existing recommendations around reducing memory use? These recommendations primarily revolve around the fact that Chromium renderer processes are restricted to ~4GB of JS memory heap.

There are many things you can do to reduce the risk of exhausting memory:

1. Split very long tests into smaller tests
2. Avoid asserting against complex objects or very long strings - can you instead assert against a subset or more targeted item?
3. Use experimentalMemoryManagement - this cleans up memory between tests, so it helps after splitting large tests up. If running in open mode the numTestsKeptInMemory flag and disabling the command log also help
4. Reduce unnecessary aliases
5. Avoid intercepting/requesting very large datasets (can you use a fixture instead? Can you do a HEAD instead of GET request?)
6. Ensure you aren't introducing memory leaks by persisting cross-test state in your specs (for example, global or shared variables)
7. Enable testIsolation to reduce risk of application-side memory leaks
8. Disable video recording
   ...and many more

[nagash77]

@extracker-ppetrov are you able to provide a reproducible example using Cypress Test Tiny?

[nagash77]

Unfortunately we have to close this issue due to inactivity. Please comment if there is new information to provide concerning the original issue and we can reopen.

[LouisGinger]

Happens all the time even with tests kept in memory set to 0. This is actually making cypress unusable at present and needs fixing.

[juliancastroorozco]

Cypress crashing so much that makes it unusable, sadly my company is looking to move away from cypress because of this, wish you the best

[jennifer-shehane]

@juliancastroorozco @LouisGinger Please comment or open a new issue detailing what is happening and providing a reproducible example of that issue. We can't assist without having a way to reproduce this ourselves. We run 1000s of tests every minute in Cypress, so it must be something specific to your application, test suite, environment, browser, etc - there's lots of factors for us to track down.

[silvio-romero-intelex]

Getting the same issue after upgrading to latest version of chrome