RFC0684: Introducing E2E testing in React Native core #684

kelset · 2023-07-06T15:14:38Z

Summary

Testing is how we validate that the code is good, and ensure that what we build our products on top of is solid and our customers are happy.

For the longest time, in React Native core this validation has been done in a few ways:

via Meta using the source code in their own products first
via internal testing with custom tooling in the Meta monorepo
via (mostly) unit testing on the CircleCI GitHub react-native repo
via local testing by the release crew ahead of generating new releases
via "pre-stable" releases (AKA Release Candidates and nightlies) that members from the community could use to test and report back

There is a great opportunity for improvement here, by introducing End-to-End (E2E) testing in the CircleCI GitHub react-native repository to introduce one more level of validation that aims to replace the need for (at least) the local testing from the release crew. By adding tooling that spins up the RNTester app in the codebase, and "black box" (the RNTester app is not aware that the agent using it is not a human) testing it, we can trust that the code is more solid.

This will make the codebase more stable, surface issues rapidly to the commit authors, and reduce the workload for the release crew.

Click here to view the rendered RFC

proposals/0684-adding-e2e-in-core.md

szymonrybczak

🙌

cortinico

LGTM mostly nits

proposals/0684-adding-e2e-in-core.md

cortinico · 2023-07-25T12:44:12Z

proposals/0684-adding-e2e-in-core.md

+
+Because this proposal doesn't directly affect consumers of React Native, the need for "teaching content" is scoped to people who wants to learn more about this infrastructure is set up and they want to contribute with more tests.
+
+The first place where this type of documentation will be added is the readme of the dedicated new folder, `packages/rn-tester-e2e/README.md` *(you can already check out a draft of it [in the first PR](https://github.com/mateuszm22/react-native/blob/k+m/new-rn-tester-E2E/packages/rn-tester-e2e/README.md) already in the works)*. In this file, there are going to be information on how the setup works, how to test it locally and how to add new tests.


TODO Let's update the link once the PR is merged

proposals/0684-adding-e2e-in-core.md

…#36267) Summary: The motivation is to create an E2E testing solution for the RNTester app that can be used to flag when things break and the release crew can use to validate more of the codebase ahead of a release. This first PR wants to just showcase the main flow (how tests are made, where the E2E folder lives, how it's used by CircleCI) and then we can build on top of it with subsequent PRs, with the help of an umbrella issue to get the community involved in making more tests! This work is being done cooperation with Callstack developers (mateuszm22 , adzironman, and others) - reason why the branch [lives in a fork](https://github.com/mateuszm22/react-native/tree/k%2Bm/new-rn-tester-E2E). ### [Read the RFC for more details ](react-native-community/discussions-and-proposals#684) ### Extra notes We are still on WebDriverIO 7 because during the exploratory phase, WDIO was a bit problematic and the workarounds needed to make it work don't seem very nice - see this PR: kelset/react-native-e2e-jest-appium-webdriverio#4 --- this will be revisited in the future, once there's a better path for upgrading. ### Things that will be handled as follow up work * upgrade to WDIO 8 * [an automated yarn run-e2e-test script](#36267 (comment)) * [a small refactoring into a js script](#36267 (comment)) ## Changelog: [INTERNAL] [ADDED] - Added first working configuration for e2e testing Pull Request resolved: #36267 Test Plan: You can play with this locally by [follow the readme](https://github.com/mateuszm22/react-native/blob/k+m/new-rn-tester-E2E/packages/rn-tester-e2e/README.md). CircleCI jobs will be added. Reviewed By: cipolleschi Differential Revision: D46225747 Pulled By: cortinico fbshipit-source-id: e5428c439afd84ed3dd219d6ada538263120d7c6

Co-authored-by: Nicola Corti <corti.nico@gmail.com>

…#36267) Summary: The motivation is to create an E2E testing solution for the RNTester app that can be used to flag when things break and the release crew can use to validate more of the codebase ahead of a release. This first PR wants to just showcase the main flow (how tests are made, where the E2E folder lives, how it's used by CircleCI) and then we can build on top of it with subsequent PRs, with the help of an umbrella issue to get the community involved in making more tests! This work is being done cooperation with Callstack developers (mateuszm22 , adzironman, and others) - reason why the branch [lives in a fork](https://github.com/mateuszm22/react-native/tree/k%2Bm/new-rn-tester-E2E). ### [Read the RFC for more details ](react-native-community/discussions-and-proposals#684) ### Extra notes We are still on WebDriverIO 7 because during the exploratory phase, WDIO was a bit problematic and the workarounds needed to make it work don't seem very nice - see this PR: kelset/react-native-e2e-jest-appium-webdriverio#4 --- this will be revisited in the future, once there's a better path for upgrading. ### Things that will be handled as follow up work * upgrade to WDIO 8 * [an automated yarn run-e2e-test script](#36267 (comment)) * [a small refactoring into a js script](#36267 (comment)) ## Changelog: [INTERNAL] [ADDED] - Added first working configuration for e2e testing Pull Request resolved: #36267 Test Plan: You can play with this locally by [follow the readme](https://github.com/mateuszm22/react-native/blob/k+m/new-rn-tester-E2E/packages/rn-tester-e2e/README.md). CircleCI jobs will be added. Reviewed By: cipolleschi Differential Revision: D47763012 Pulled By: cortinico fbshipit-source-id: 6eb9674182b8ee97aea4784158c69bf017f379e5

…facebook#36267) Summary: The motivation is to create an E2E testing solution for the RNTester app that can be used to flag when things break and the release crew can use to validate more of the codebase ahead of a release. This first PR wants to just showcase the main flow (how tests are made, where the E2E folder lives, how it's used by CircleCI) and then we can build on top of it with subsequent PRs, with the help of an umbrella issue to get the community involved in making more tests! This work is being done cooperation with Callstack developers (mateuszm22 , adzironman, and others) - reason why the branch [lives in a fork](https://github.com/mateuszm22/react-native/tree/k%2Bm/new-rn-tester-E2E). ### [Read the RFC for more details ](react-native-community/discussions-and-proposals#684) ### Extra notes We are still on WebDriverIO 7 because during the exploratory phase, WDIO was a bit problematic and the workarounds needed to make it work don't seem very nice - see this PR: kelset/react-native-e2e-jest-appium-webdriverio#4 --- this will be revisited in the future, once there's a better path for upgrading. ### Things that will be handled as follow up work * upgrade to WDIO 8 * [an automated yarn run-e2e-test script](facebook#36267 (comment)) * [a small refactoring into a js script](facebook#36267 (comment)) ## Changelog: [INTERNAL] [ADDED] - Added first working configuration for e2e testing Pull Request resolved: facebook#36267 Test Plan: You can play with this locally by [follow the readme](https://github.com/mateuszm22/react-native/blob/k+m/new-rn-tester-E2E/packages/rn-tester-e2e/README.md). CircleCI jobs will be added. Reviewed By: cipolleschi Differential Revision: D46225747 Pulled By: cortinico fbshipit-source-id: e5428c439afd84ed3dd219d6ada538263120d7c6

…facebook#36267) Summary: The motivation is to create an E2E testing solution for the RNTester app that can be used to flag when things break and the release crew can use to validate more of the codebase ahead of a release. This first PR wants to just showcase the main flow (how tests are made, where the E2E folder lives, how it's used by CircleCI) and then we can build on top of it with subsequent PRs, with the help of an umbrella issue to get the community involved in making more tests! This work is being done cooperation with Callstack developers (mateuszm22 , adzironman, and others) - reason why the branch [lives in a fork](https://github.com/mateuszm22/react-native/tree/k%2Bm/new-rn-tester-E2E). ### [Read the RFC for more details ](react-native-community/discussions-and-proposals#684) ### Extra notes We are still on WebDriverIO 7 because during the exploratory phase, WDIO was a bit problematic and the workarounds needed to make it work don't seem very nice - see this PR: kelset/react-native-e2e-jest-appium-webdriverio#4 --- this will be revisited in the future, once there's a better path for upgrading. ### Things that will be handled as follow up work * upgrade to WDIO 8 * [an automated yarn run-e2e-test script](facebook#36267 (comment)) * [a small refactoring into a js script](facebook#36267 (comment)) ## Changelog: [INTERNAL] [ADDED] - Added first working configuration for e2e testing Pull Request resolved: facebook#36267 Test Plan: You can play with this locally by [follow the readme](https://github.com/mateuszm22/react-native/blob/k+m/new-rn-tester-E2E/packages/rn-tester-e2e/README.md). CircleCI jobs will be added. Reviewed By: cipolleschi Differential Revision: D47763012 Pulled By: cortinico fbshipit-source-id: 6eb9674182b8ee97aea4784158c69bf017f379e5

…facebook#36267) Summary: The motivation is to create an E2E testing solution for the RNTester app that can be used to flag when things break and the release crew can use to validate more of the codebase ahead of a release. This first PR wants to just showcase the main flow (how tests are made, where the E2E folder lives, how it's used by CircleCI) and then we can build on top of it with subsequent PRs, with the help of an umbrella issue to get the community involved in making more tests! This work is being done cooperation with Callstack developers (mateuszm22 , adzironman, and others) - reason why the branch [lives in a fork](https://github.com/mateuszm22/react-native/tree/k%2Bm/new-rn-tester-E2E). ### [Read the RFC for more details ](react-native-community/discussions-and-proposals#684) ### Extra notes We are still on WebDriverIO 7 because during the exploratory phase, WDIO was a bit problematic and the workarounds needed to make it work don't seem very nice - see this PR: kelset/react-native-e2e-jest-appium-webdriverio#4 --- this will be revisited in the future, once there's a better path for upgrading. ### Things that will be handled as follow up work * upgrade to WDIO 8 * [an automated yarn run-e2e-test script](facebook#36267 (comment)) * [a small refactoring into a js script](facebook#36267 (comment)) ## Changelog: [INTERNAL] [ADDED] - Added first working configuration for e2e testing Pull Request resolved: facebook#36267 Test Plan: You can play with this locally by [follow the readme](https://github.com/mateuszm22/react-native/blob/k+m/new-rn-tester-E2E/packages/rn-tester-e2e/README.md). CircleCI jobs will be added. Reviewed By: cipolleschi Differential Revision: D46225747 Pulled By: cortinico fbshipit-source-id: e5428c439afd84ed3dd219d6ada538263120d7c6

…facebook#36267) Summary: The motivation is to create an E2E testing solution for the RNTester app that can be used to flag when things break and the release crew can use to validate more of the codebase ahead of a release. This first PR wants to just showcase the main flow (how tests are made, where the E2E folder lives, how it's used by CircleCI) and then we can build on top of it with subsequent PRs, with the help of an umbrella issue to get the community involved in making more tests! This work is being done cooperation with Callstack developers (mateuszm22 , adzironman, and others) - reason why the branch [lives in a fork](https://github.com/mateuszm22/react-native/tree/k%2Bm/new-rn-tester-E2E). ### [Read the RFC for more details ](react-native-community/discussions-and-proposals#684) ### Extra notes We are still on WebDriverIO 7 because during the exploratory phase, WDIO was a bit problematic and the workarounds needed to make it work don't seem very nice - see this PR: kelset/react-native-e2e-jest-appium-webdriverio#4 --- this will be revisited in the future, once there's a better path for upgrading. ### Things that will be handled as follow up work * upgrade to WDIO 8 * [an automated yarn run-e2e-test script](facebook#36267 (comment)) * [a small refactoring into a js script](facebook#36267 (comment)) ## Changelog: [INTERNAL] [ADDED] - Added first working configuration for e2e testing Pull Request resolved: facebook#36267 Test Plan: You can play with this locally by [follow the readme](https://github.com/mateuszm22/react-native/blob/k+m/new-rn-tester-E2E/packages/rn-tester-e2e/README.md). CircleCI jobs will be added. Reviewed By: cipolleschi Differential Revision: D47763012 Pulled By: cortinico fbshipit-source-id: 6eb9674182b8ee97aea4784158c69bf017f379e5

…facebook#36267) Summary: The motivation is to create an E2E testing solution for the RNTester app that can be used to flag when things break and the release crew can use to validate more of the codebase ahead of a release. This first PR wants to just showcase the main flow (how tests are made, where the E2E folder lives, how it's used by CircleCI) and then we can build on top of it with subsequent PRs, with the help of an umbrella issue to get the community involved in making more tests! This work is being done cooperation with Callstack developers (mateuszm22 , adzironman, and others) - reason why the branch [lives in a fork](https://github.com/mateuszm22/react-native/tree/k%2Bm/new-rn-tester-E2E). ### [Read the RFC for more details ](react-native-community/discussions-and-proposals#684) ### Extra notes We are still on WebDriverIO 7 because during the exploratory phase, WDIO was a bit problematic and the workarounds needed to make it work don't seem very nice - see this PR: kelset/react-native-e2e-jest-appium-webdriverio#4 --- this will be revisited in the future, once there's a better path for upgrading. ### Things that will be handled as follow up work * upgrade to WDIO 8 * [an automated yarn run-e2e-test script](facebook#36267 (comment)) * [a small refactoring into a js script](facebook#36267 (comment)) ## Changelog: [INTERNAL] [ADDED] - Added first working configuration for e2e testing Pull Request resolved: facebook#36267 Test Plan: You can play with this locally by [follow the readme](https://github.com/mateuszm22/react-native/blob/k+m/new-rn-tester-E2E/packages/rn-tester-e2e/README.md). CircleCI jobs will be added. Reviewed By: cipolleschi Differential Revision: D46225747 Pulled By: cortinico fbshipit-source-id: e5428c439afd84ed3dd219d6ada538263120d7c6

…facebook#36267) Summary: The motivation is to create an E2E testing solution for the RNTester app that can be used to flag when things break and the release crew can use to validate more of the codebase ahead of a release. This first PR wants to just showcase the main flow (how tests are made, where the E2E folder lives, how it's used by CircleCI) and then we can build on top of it with subsequent PRs, with the help of an umbrella issue to get the community involved in making more tests! This work is being done cooperation with Callstack developers (mateuszm22 , adzironman, and others) - reason why the branch [lives in a fork](https://github.com/mateuszm22/react-native/tree/k%2Bm/new-rn-tester-E2E). ### [Read the RFC for more details ](react-native-community/discussions-and-proposals#684) ### Extra notes We are still on WebDriverIO 7 because during the exploratory phase, WDIO was a bit problematic and the workarounds needed to make it work don't seem very nice - see this PR: kelset/react-native-e2e-jest-appium-webdriverio#4 --- this will be revisited in the future, once there's a better path for upgrading. ### Things that will be handled as follow up work * upgrade to WDIO 8 * [an automated yarn run-e2e-test script](facebook#36267 (comment)) * [a small refactoring into a js script](facebook#36267 (comment)) ## Changelog: [INTERNAL] [ADDED] - Added first working configuration for e2e testing Pull Request resolved: facebook#36267 Test Plan: You can play with this locally by [follow the readme](https://github.com/mateuszm22/react-native/blob/k+m/new-rn-tester-E2E/packages/rn-tester-e2e/README.md). CircleCI jobs will be added. Reviewed By: cipolleschi Differential Revision: D47763012 Pulled By: cortinico fbshipit-source-id: 6eb9674182b8ee97aea4784158c69bf017f379e5

proposals/0684-adding-e2e-in-core.md

Co-authored-by: Szymon Rybczak <szymon.rybczak@gmail.com>

add rfc - starting point

b68e060

kelset added the 💡 Proposal This label identifies a proposal label Jul 6, 2023

kelset changed the title ~~add rfc - starting point~~ RFC0684: Introducing E2E testing in React Native core Jul 6, 2023

kelset changed the title ~~RFC0684: Introducing E2E testing in React Native core~~ RFC0684: Introducing E2E testing in React Native core (WIP) Jul 6, 2023

kelset added 4 commits July 10, 2023 11:25

tweaks

a8901e1

checkpoint

c063172

fill teaching section

daa45b3

add detailed proposal

82b33d7

kelset changed the title ~~RFC0684: Introducing E2E testing in React Native core (WIP)~~ RFC0684: Introducing E2E testing in React Native core Jul 11, 2023

kelset marked this pull request as ready for review July 19, 2023 10:05

kelset mentioned this pull request Jul 19, 2023

add RNTester-E2E: tests for iOS and Android via Appium, WDIO and Jest facebook/react-native#36267

Closed

cipolleschi reviewed Jul 19, 2023

View reviewed changes

proposals/0684-adding-e2e-in-core.md Show resolved Hide resolved

cipolleschi reviewed Jul 19, 2023

View reviewed changes

proposals/0684-adding-e2e-in-core.md Show resolved Hide resolved

cipolleschi reviewed Jul 19, 2023

View reviewed changes

proposals/0684-adding-e2e-in-core.md Show resolved Hide resolved

kelset added 2 commits July 21, 2023 15:50

address feedback

a80ec24

graph tweak

8998abf

cipolleschi approved these changes Jul 21, 2023

View reviewed changes

szymonrybczak approved these changes Jul 24, 2023

View reviewed changes

cortinico approved these changes Jul 25, 2023

View reviewed changes

kelset and others added 2 commits July 26, 2023 10:48

Update proposals/0684-adding-e2e-in-core.md

d9e6db0

Co-authored-by: Nicola Corti <corti.nico@gmail.com>

Update proposals/0684-adding-e2e-in-core.md

490b384

cipolleschi approved these changes Jul 26, 2023

View reviewed changes

szymonrybczak reviewed Aug 2, 2023

View reviewed changes

proposals/0684-adding-e2e-in-core.md Outdated Show resolved Hide resolved

Update proposals/0684-adding-e2e-in-core.md

760c4e0

Co-authored-by: Szymon Rybczak <szymon.rybczak@gmail.com>

cortinico approved these changes Aug 2, 2023

View reviewed changes

cortinico merged commit d907dfa into main Aug 2, 2023

cortinico deleted the kelset/rfc-e2e-testing branch August 2, 2023 14:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RFC0684: Introducing E2E testing in React Native core #684

RFC0684: Introducing E2E testing in React Native core #684

kelset commented Jul 6, 2023 •

edited

Loading

szymonrybczak left a comment

cortinico left a comment

cortinico Jul 25, 2023


		Because this proposal doesn't directly affect consumers of React Native, the need for "teaching content" is scoped to people who wants to learn more about this infrastructure is set up and they want to contribute with more tests.

		The first place where this type of documentation will be added is the readme of the dedicated new folder, `packages/rn-tester-e2e/README.md` (you can already check out a draft of it [in the first PR](https://github.com/mateuszm22/react-native/blob/k+m/new-rn-tester-E2E/packages/rn-tester-e2e/README.md) already in the works). In this file, there are going to be information on how the setup works, how to test it locally and how to add new tests.

RFC0684: Introducing E2E testing in React Native core #684

RFC0684: Introducing E2E testing in React Native core #684

Conversation

kelset commented Jul 6, 2023 • edited Loading

Summary

Click here to view the rendered RFC

szymonrybczak left a comment

Choose a reason for hiding this comment

cortinico left a comment

Choose a reason for hiding this comment

cortinico Jul 25, 2023

Choose a reason for hiding this comment

kelset commented Jul 6, 2023 •

edited

Loading