v10.4.0 Component test React 18 warnings with mid sized apps

[muratkeremozcan]

Current behavior

In a simple repo, like a fresh CRA, or small a-la-carte components as in 330+ cypress-react-component-test-examples v10.4.0 works as intented; there are no React 18 warnings in the console.

In mid sized or larger apps the React 18 console warnings may still persist.

Here are 2 apps where the issue is observed. Install and run any component test and view the console.

Repo 1: Kent Dodds' Epic React final app, with Cypress e2e and component tests:

git clone https://github.com/muratkeremozcan/bookshelf.git
cd bookshelf
npm i
npm run cy:open-ct

Repo 2: React Hooks in Action book with Cypress e2e and component tests

git clone https://github.com/muratkeremozcan/react-hooks-in-action-with-cypress.git
cd react-hooks-in-action-with-cypress
yarn 
yarn cy:open-ct

clear Cypress cache, remove node_modules, clear npm / yarn cache to ensure clean installs

Desired behavior

Regardless of the app size, there are no React 18 console warnings and we can use new React hooks in the tests.

Test code to reproduce

Linked above.

Cypress Version

10.4.0

Other

No response

[AtofStryker]

possibly fixed by #23182?

[rockindahizzy]

Yes, changing the import in cypress/support/component.js

// From
import {mount} from 'cypress/react'

// To
import {mount} from 'cypress/react18'

Will clear the warnings, but there is currently an open issue for React 18 support.

[lmiller1990]

It will be fixed in the next release.

[muratkeremozcan]

Yes, changing the import in cypress/support/component.js

// From
import {mount} from 'cypress/react'

// To
import {mount} from 'cypress/react18'

Will clear the warnings, but there is currently an open issue for React 18 support.

This works. I upgraded React to v18 and Cypress to 10.4.0, and added the new React 18 hooks.

But, headless local or CI execution is so slow that CI times out, and local is dreadful. There has to be something not right.

Please make sure to try this in a large repo.

You can reproduce through this PR muratkeremozcan/cypress-react-component-test-examples#280. Just toggle 'cypress/react18' vs 'cypress/react'.

git clone https://github.com/muratkeremozcan/cypress-react-component-test-examples.git
yarn 
yarn cy:run-ct

The React 18 unmount issue is reproducible too, but that can be worked around with separate tests & files. The headless execution speed issue is the real blocker.

[lmiller1990]

Headless (well "run mode" where you run all specs) execution for Component Testing will improve significantly (up to ~50% on large code bases, improvement linear based on # of specs) with an experimental flag that is landing soon.

FYI: You can also pass --headed to do a full execution with a headed browser (does not change performance).

As for the experimental feature:

• Pull Request: feat: experimental single tab run mode for component testing #23104
• Issue with context: CT Performance Improvements: v10 slower than v9 #22353

I will try the experimental flag on your code base now. You can even try it by installing one of these pre-release binaries: 293a0f4#comments. You need to enable the experiment. Example.

Edit: wow this code base is taking forever to run! I'll get you some before/after numbers with the experimental flag. Give me a moment.

export default defineConfig({
  component: {
    experimentalSingleTabRunMode: true // <= this is the flag to enable the single tab run mode
  }
})

React 18 should be 💯 once the bug fix it out.

[lmiller1990]

Finishing running your project before/after:

205 specs
Before: 42m
After: 4m

Final numbers (run on my Macbook arm64)

# experimentalSingleTabRunMode: false 
yarn cy:run-ct --config video=false --browser chrome  
17.88s user 9.95s system 11% cpu 4:00.11 total

# experimentalSingleTabRunMode: true 
yarn cy:run-ct --config video=false --browser chrome  
2585.68s user 125.62s system 105% cpu 42:42.53 total

Wow, unreal. I've tested other webpack repos which saw improvements around 2x. I am not sure what's going on with your project but this is definitely not the expected performance. Other projects I've tested and used running with CRA + webpack don't exhibit anything near the slowness you are experiencing.

Either way, the expeirmentalSingleTabRunMode will help, but maybe we need to dig deeper on your specific issue, I am fairly sure it should not be this slow.

[muratkeremozcan]

Thanks a ton for giving it a try.

On main, with React 17, the local run time is about 12 mins. In CI it is all right, under 4 mins with parallelization https://dashboard.cypress.io/projects/62pyqm/runs/933/specs.

If the experimental feature fixes the issue occurence with React 18, and increases the speed 2-3 times, works for me.

[lmiller1990]

Great, I hope we can ship the experimental flag with 10.5. If it goes smoothly, we can make this the default in the future. 🤞 Keep your eyes on #23104 - just waiting for a few internal reviews.

[muratkeremozcan]

#23104

I tried experimentalSingleTabRunMode , React 18, and the new hooks with v10.6.0. Ran out of parallelization while wrestling the changes, but got one last run https://dashboard.cypress.io/projects/62pyqm/runs/991/specs. The parallel duration of 340+ tests came down from ~5 mins to ~3 mins.

If it works in this repo, it should work in most production apps.

muratkeremozcan/cypress-react-component-test-examples#297

Edit: When I ran out of parallelization and had to run all the tests in one machine, tests that had to do with hovering using Kovalenko's plugin realHover() started failing often. That might be the primary thing to watch out for. Example: cypress/component/hooks/usehooks.com/useHover.cy.tsx

[lmiller1990]

Hmm that's odd, I wonder why the CDP real-events are failing 🤔

Glad the experiment is helping - if it goes well over the next few weeks and no-one reports any bugs, it may become the default. Just to clarify - are you saying the real-events library is more flaky when using the single tab experiment?