Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cypress is running really slow after upgrading to v10 #22868

Closed
SadiqRahim opened this issue Jul 20, 2022 · 70 comments
Closed

Cypress is running really slow after upgrading to v10 #22868

SadiqRahim opened this issue Jul 20, 2022 · 70 comments
Labels
E2E Issue related to end-to-end testing Reproducible Can be reproduced type: performance 🏃‍♀️ Performance related

Comments

@SadiqRahim
Copy link

SadiqRahim commented Jul 20, 2022

Current behavior

We were on 9.5.4. Then did the upgrade to the latest version of cypress 10.3.1.
Now the test execution time takes too long. This appears to occur when loading a page.
Just the cypress package and cypress.config.js file changed. Screenshot attached with the differences on the same test code. Hopefully the information provided helps

Screenshot 2022-07-20 at 15 52 30
Screenshot 2022-07-20 at 15 52 54

Desired behavior

We expect to run faster or at the same speed

Test code to reproduce

it('Create new forecast', () => {
cy.intercept('/transformation/').as('wait')
cy.visit(${global.url}/company/adient/forecasting/annual-iu62wfojz8iourjk74wn)
cy.wait('@wait').its('response.statusCode').should('be.oneOf', [200, 307])
cy.wait('@wait').its('response.statusCode').should('be.oneOf', [200, 307])
cy.wait(1500)

cy.window().then((appWindow) => {
  let spreadHostElement = appWindow.document.querySelector(pageObj.forecastObj.hostElement)
  let spread = appWindow.GC.Spread.Sheets.findControl(spreadHostElement)
  let activeSheet = spread.getActiveSheet()
  activeSheet.setActiveCell(4, 5)
  activeSheet.startEdit(true, '10')
  cy.get('.cc-forecasting.cc-ui-block').type('{enter}')
  cy.wait(300)
})

cy.get(':nth-child(3) > .cc-ui-button').click({force: true})
cy.get('.cc-company-forecasting-save > .cc-ui-field > input').type('New forecast')

cy.intercept('POST', '/graphql', (req) => {
  aliasMutation(req, 'SaveForecast')
})
cy.get('.cc-company-forecasting-save__actions > .cc-ui-button').click({force: true})
cy.wait('@SaveForecast')
cy.get('.is-active > .cc-ui-icon > .iconPlus').click({ force: true })
cy.wait(1500)

})

Cypress Version

10.3.1

Other

No response

@mjhenkes mjhenkes self-assigned this Jul 20, 2022
@cypress-bot cypress-bot bot added the stage: investigating Someone from Cypress is looking into this label Jul 20, 2022
@mjhenkes
Copy link
Member

On first glance this appears to be a duplicate of #22353

@lmiller1990
Copy link
Contributor

lmiller1990 commented Jul 21, 2022

I do not think this is duplicate of #22353, because that is only related to Component Testing, where we changed from 1 tab per run to 1 tab per spec. E2E changed from one browser launch per spec to a single browser launch, opening a new per spec, which is substantially faster.

Also, #22353 points out the entire run time is slower, but the individual spec run time is not - the time is getting lost on rendering the runner UI, parsing JS, etc. This screenshot seems to imply the actual spec execution is slower, which would indicate a different issue.

@SadiqRahim I'm quite curious here... your test takes 20 seconds to execute:

cy.window().then((appWindow) => {
  let spreadHostElement = appWindow.document.querySelector(pageObj.forecastObj.hostElement)
  let spread = appWindow.GC.Spread.Sheets.findControl(spreadHostElement)
  let activeSheet = spread.getActiveSheet()
  activeSheet.setActiveCell(4, 5)
  activeSheet.startEdit(true, '10')
  cy.get('.cc-forecasting.cc-ui-block').type('{enter}')
  cy.wait(300)
})

cy.get(':nth-child(3) > .cc-ui-button').click({force: true})
cy.get('.cc-company-forecasting-save > .cc-ui-field > input').type('New forecast')

cy.intercept('POST', '/graphql', (req) => {
  aliasMutation(req, 'SaveForecast')
})
cy.get('.cc-company-forecasting-save__actions > .cc-ui-button').click({force: true})
cy.wait('@SaveForecast')
cy.get('.is-active > .cc-ui-icon > .iconPlus').click({ force: true })
cy.wait(1500)

Do you have any feeling on which part of this is slow? cy.get and click are basically instant. The only thing that looks like it would be taking time here is waiting for API requests. Are you able to clarify which part(s) of your test seem to be taking longer/account for the additional 5 seconds?

@mjhenkes
Copy link
Member

@SadiqRahim, Is it just that test in question that is substantially slower or are other tests also slower? Could you comment out lines in that test to try to narrow down which step is taking the most time?

@SadiqRahim
Copy link
Author

SadiqRahim commented Jul 21, 2022

@lmiller1990 @mjhenkes thanks for looking at it. It slows down ridiculously at the cy.visit() command. I use the cy.visit() command in two different places on the code

  • One on the before hook, which is to load the login page where I also use cy.session to cache the session
  • Then the other place is on the code above to go to a particular page

I run the same code

  • On 9.5.4 where it finishes the test in 25 seconds
  • Then on any of the v10 including the latest version 10.3.1, it took about 3 mins & 30 seconds to finish the test

Note that I have tried running them on electron and chrome and I see the same slowness on both the browsers

The migration instruction to v10 does not have any comments/notes about any lack of performance nor does it have any related fixes in the late releases. Thus I'm not quite sure what's going on.

@lmiller1990
Copy link
Contributor

lmiller1990 commented Jul 22, 2022

I completely misread your screenshots - I didn't fully grasp how much slower this was for you.

I think it's something on your end, but happy to help you debug - if cy.visit was really 6x slower, we'd see it in our test suite, or the entire product would be on fire.

The best way to start debugging is using our debug logs. Run your test with DEBUG=cypress:* yarn cypress ... and it will print lots of logs in the terminal. You could compare 9.x and 10.x and see if there's anything different. Especially pay attention to what is printed during cy.visit.

If you could do this and copy paste the logs here (they'll be huge, no problem) that will greatly help. Without a minimal reproduction or some logs, we can only really speculate.

@cypress-bot cypress-bot bot added stage: awaiting response Potential fix was proposed; awaiting response and removed stage: investigating Someone from Cypress is looking into this labels Jul 25, 2022
@mjhenkes
Copy link
Member

@SadiqRahim Have you been able to grab the debug logs? I'm fairly curious about what is going on with the slowness.

@mjhenkes
Copy link
Member

Possibly related? #22968 (comment)

@SadiqRahim
Copy link
Author

@mjhenkes Thanks for looking at it. Sorry that I just got back from my holidays. Let me look at the issue #22528 and get back to you with more info.

@mjhenkes mjhenkes assigned AtofStryker and unassigned mjhenkes Aug 2, 2022
@SadiqRahim
Copy link
Author

SadiqRahim commented Aug 4, 2022

@mjhenkes @AtofStryker @lmiller1990 My apologies for the late in reply. I did a quick debug only by visiting to see the difference just with the following code,

cy.visit('https://qa.cognitivecredit.com')

I ran the same line of code with 10v & < and 9v & >

  • With 10v and above, takes about 15 sec to run
  • With 9.7.0v and below, takes about 4 sec to run

The obvious diff I see between them are the time taken on cypress:server:request and cypress:network:cors and 10v & < running additional requests, which I do not see on 9.7.0v & below. Also, I'm able to see this issue only on our lower environments and one of the url is called on the code above.

This is replicable if you try with the pertinent versions. Something has evidently changed on the 10v and above. Appreciate any help on this pls as we're blocked to upgrade this to the latest cypress version,

@cypress-bot cypress-bot bot added stage: investigating Someone from Cypress is looking into this and removed stage: awaiting response Potential fix was proposed; awaiting response labels Aug 5, 2022
@AtofStryker
Copy link
Contributor

Hey @SadiqRahim. What operating system are you on? Is the issue reproducible just from running cy.visit('https://qa.cognitivecredit.com/') between v9 and v10?

@SadiqRahim
Copy link
Author

SadiqRahim commented Aug 12, 2022

@AtofStryker I'm trying to run it on my MacOS Monterey version 12.1. Yes its reproducible just from running that line of code between v9 and v10. Thanks for looking at this. Let me know should you need any more details pls.

@AtofStryker
Copy link
Contributor

@SadiqRahim I was able to reproduce the issue and made a repository here

@lmiller1990
Copy link
Contributor

lmiller1990 commented Aug 16, 2022

I think the problem is our network proxy has a bottleneck when there is many XHR requests. This website's login page makes over 800 XHR requests, and ~500 CSS resources. Just to clarify, @SadiqRahim, is that intentional? It takes 30 seconds to finish even outside of Cypress - I think what's happening is Cypress is waiting for all network activity to stop, which takes a very long time.

Prod in browser:

image

Cypress:

image

@SadiqRahim
Copy link
Author

SadiqRahim commented Aug 16, 2022

@lmiller1990 Those are icons, images downloads. It appears to be because some of the popular browsers does show everything within the xhr panel nowadays particularly chrome I think. However, this area was not changed lately from our development point of view. Also no indication of such issue occurred on cypress 9v or below. The problem appears to be only on any of the 10v. So i'm unclear as to what has changed on 10v that this was loading faster on all other versions below 10v :(

@cypress-bot cypress-bot bot added stage: routed to e2e-core and removed stage: investigating Someone from Cypress is looking into this labels Aug 16, 2022
@lmiller1990
Copy link
Contributor

More info... it looks like @flotwig fixed this bug in #23227. The original issue the OP reported about his website loading slowly is nice and quick on the latest version of Cypress. @SadiqRahim can you try Cypress 11 (or Cypress 10.6, which has the fix, but always good to be on the latest).


I am still having some issues with the reproduction I put together, though, where I load 400 JS modules. Checking out the commit before #22462 fixes it. I also found I can remove this code and it fixes it:

this.sweepInterval = setInterval(() => {
const now = Date.now()
this.pendingPreRequests.removeMatching(({ timestamp, browserPreRequest }) => {
if (timestamp + this.requestTimeout * 2 < now) {
debugVerbose('timed out unmatched pre-request: %o', browserPreRequest)
metrics.unmatchedPreRequests++
return false
}
return true
})
}, this.requestTimeout * 2)

I wonder if the logic around sweepInterval is incorrect. cc @BlueWinds - maybe you have some context. The original PR says:

It also sweeps unmatched prerequests more than 10s old

But looking at the code:

}, this.requestTimeout * 2)

, this.requestTimeout * 2)

requestTimeout is 500 so we sweep every 1 second. Is the code incorrect? I think it should be https://github.com/cypress-io/cypress/compare/fix/correct-sweep-interval?expand=1?. I tried adding that code and the regression I encountered goes away.

@lmiller1990
Copy link
Contributor

Tentative PR #25209

@lmiller1990
Copy link
Contributor

lmiller1990 commented Dec 19, 2022

@VictorGosse @kyleburnett @jueinin can any of you try the pre-release binaries with my attempted fix here? 6b78454#comments. Thanks!

I still think there is some other weirdness going on somewhere, for apps with many JS module requests, Cy 9 appears slightly quicker - I'm going to keep digging, but if you can at least verify if this does or doesn't help with the slowness you've encountered, that'd be great.

@jueinin
Copy link

jueinin commented Dec 22, 2022

sorry, due to time constraints, I have removed cypress and construct our e2e testing with plain mocha and vite. it's very fast!

so may be i can not reproduce this bug again.

@lmiller1990
Copy link
Contributor

I think #25209 should address some issues. It'll be in the next release.

I'll leave this open a little while more.

@dcherevatenko
Copy link

Seems like updating to Cypress 12.3 from 11.2 (with wiping "node_modules" folder) helped me to restore test perfomance on Windows distro (I compared it with WSL distro on the same machine). Thanks!

@lmiller1990
Copy link
Contributor

It's been a month since we shipped the fix. Happy to reopen if there are other perf issues (definitely possible) but most likely we should make a new issue. Thanks everyone for the patience and help debugging!

@inorganik
Copy link

Thought I would just share a finding - when running my tests with cypress set to Chrome 113, they ran extremely slow. When I switched to Electron 100, it ran as fast as it used to when Chrome was selected. It's possible something changed in chrome that is slowing cypress down.

@lmiller1990
Copy link
Contributor

@inorganik I had a similar issue, the problem I was having was chrome was running in emulation mode (I had M1 but was a darwin binary, I think - or something along these lines). Not sure if that's an issue you might be running into, but sharing in case it helps anyone else.

@jon-robinson
Copy link

jon-robinson commented May 15, 2023

Thought I would just share a finding - when running my tests with cypress set to Chrome 113, they ran extremely slow. When I switched to Electron 100, it ran as fast as it used to when Chrome was selected. It's possible something changed in chrome that is slowing cypress down.

I'm having the same problem actually. I'm using v9.7.0 (can't upgrade easily right now, hence the older version). Chrome 113 and Edge 113 are running horribly. Very laggy, ever since upgrading to 113 (112 was fine). Same issue either in open or run mode. macOS 12.6.5 M1

As you can see from this quick video, the duration clock is lagging quite a lot...
https://github.com/cypress-io/cypress/assets/16207039/eba30235-6155-4ebf-9743-f2ee53555465

@jon-robinson
Copy link

jon-robinson commented May 15, 2023

@inorganik I had a similar issue, the problem I was having was chrome was running in emulation mode (I had M1 but was a darwin binary, I think - or something along these lines). Not sure if that's an issue you might be running into, but sharing in case it helps anyone else.

Hey @inorganik @lmiller1990
Out of interest, what did you do to resolve the issue? Assuming you did fix it somehow?

@inorganik
Copy link

inorganik commented May 15, 2023

My fix is to use Electron 100 instead of Chrome. But I suspect when Electron gets to 113 it may become slow as well. I would suggest the Cypress team try to find the underlying issue here.

@Daniel-Shiosaky
Copy link

Thought I would just share a finding - when running my tests with cypress set to Chrome 113, they ran extremely slow. When I switched to Electron 100, it ran as fast as it used to when Chrome was selected. It's possible something changed in chrome that is slowing cypress down.

I'm having the same problem actually. I'm using v9.7.0 (can't upgrade easily right now, hence the older version). Chrome 113 and Edge 113 are running horribly. Very laggy, ever since upgrading to 113 (112 was fine). Same issue either in open or run mode. macOS 12.6.5 M1

As you can see from this quick video, the duration clock is lagging quite a lot... https://github.com/cypress-io/cypress/assets/16207039/eba30235-6155-4ebf-9743-f2ee53555465

we are having the same issue, we noticed this especially with people using Macs with M1 chip was horribly slow we are still using cypress 9.7.0
is there an open ticket for this issue?

@srotak5
Copy link

srotak5 commented May 23, 2023

Same issue on 12.12.0

@nagash77
Copy link
Contributor

@srotak5 please open a new issue with a reproducible example and the Cypress team will be happy to investigate.

@jon-robinson
Copy link

jon-robinson commented May 30, 2023

Anyone, do we have a linked, open ticket?

@scorpyto
Copy link

scorpyto commented Jun 8, 2023

I've just tried to run some tests with "run --" command and it was ultra slow compared to "open". I've disabled the video and it seems to work faster now. You can try it too.
Cypress version 12.12.0

@jamiek-acl
Copy link

jamiek-acl commented Jul 19, 2023

Using Cypress 10.4.0 (Mac M1 Pro)

When run test with Chrome v114, Cypress is almost un-usably slow (like 1s per keystroke, mocking API calls sometimes times out after 10s, pressing the 'Re-run' button is super laggy and doesnt always work without restarting). The odd time I'm able to get the test to run, it takes 77 seconds.

Changed it to use Firefox v113, test takes 7 seconds.

Changed it to use Electron v102, test takes 6 seconds.

@AtofStryker
Copy link
Contributor

@jamiek-acl are you able to open a new issue with a reproducible example?

@alpo22
Copy link

alpo22 commented Jul 19, 2023

Here you go: #27344

@kylemh
Copy link

kylemh commented Aug 11, 2023

@inorganik I had a similar issue, the problem I was having was chrome was running in emulation mode (I had M1 but was a darwin binary, I think - or something along these lines). Not sure if that's an issue you might be running into, but sharing in case it helps anyone else.

@lmiller1990 how do I check if this is an issue and how do I resolve it? Feels very much like tha problem for me.

Basically seeing the same as others where Firefox and Electron take seconds, but Cypress crawls

@nickls
Copy link

nickls commented Sep 28, 2023

@kylemh -- I'm having the same issue on M1. You can confirm in the Chrome About page.

image

@mikejackowski
Copy link

why was this closed? it's still happening.

@lmiller1990
Copy link
Contributor

@inorganik I had a similar issue, the problem I was having was chrome was running in emulation mode (I had M1 but was a darwin binary, I think - or something along these lines). Not sure if that's an issue you might be running into, but sharing in case it helps anyone else.

@lmiller1990 how do I check if this is an issue and how do I resolve it? Feels very much like tha problem for me.

Basically seeing the same as others where Firefox and Electron take seconds, but Cypress crawls

I don't quite remember how I fixed this, sorry! The issue was related to the browser running in emulation mode (eg M1 but running using an x86 binary + emulation).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
E2E Issue related to end-to-end testing Reproducible Can be reproduced type: performance 🏃‍♀️ Performance related
Projects
None yet
Development

No branches or pull requests