From e4d09821bdfcf1264e25afb3d3407f4a100f3228 Mon Sep 17 00:00:00 2001 From: Lorenzo Sciandra Date: Tue, 18 Oct 2022 16:50:27 +0100 Subject: [PATCH] chore(release): Update release testing section for new E2E testing script (#3368) * remove E2E section until we'll re-add it post E2E setup * remove mention of the E2E script * rework the release testing page * add mention of where to find the info to turn on new arch * typo * re-add section --- .../contributing/how-to-contribute-code.md | 14 +-- .../how-to-run-and-write-tests.md | 11 +- website/contributing/release-testing.md | 112 ++++++++++-------- 3 files changed, 69 insertions(+), 68 deletions(-) diff --git a/website/contributing/how-to-contribute-code.md b/website/contributing/how-to-contribute-code.md index d8336d2e9a5..3a312052b4b 100644 --- a/website/contributing/how-to-contribute-code.md +++ b/website/contributing/how-to-contribute-code.md @@ -17,19 +17,19 @@ Please refer to the [Environment Setup](/docs/environment-setup) guide to setup After cloning React Native, open the directory and run `yarn` to install its dependencies. -Then, you can run several commands: +Now you are set up to run several commands: - `yarn start` starts the Metro packager server. - `yarn lint` checks the code style. - `yarn format` automatically formats your code. -- `yarn test` runs the JavaScript test suite. -- `yarn test --watch` runs an interactive JavaScript test watcher. -- `yarn test ` runs JavaScript tests with matching filenames. +- `yarn test` runs the Jest-based JavaScript test suite. + - `yarn test --watch` runs an interactive JavaScript test watcher. + - `yarn test ` runs JavaScript tests with matching filenames. - `yarn flow` runs the [Flow](https://flowtype.org/) typechecks. -- `yarn flow-check-android` does a full Flow check over `*.android.js` files. -- `yarn flow-check-ios` does a full Flow check over `*.ios.js` files. + - `yarn flow-check-android` does a full Flow check over `*.android.js` files. + - `yarn flow-check-ios` does a full Flow check over `*.ios.js` files. +- `yarn test-typescript` runs the [TypeScript](https://www.typescriptlang.org/) typechecks. - `yarn test-ios` runs the iOS test suite (macOS required). -- `node ./scripts/run-ci-e2e-tests.js --js --android --ios` runs the end-to-tend JavaScript, Android, and iOS tests. ## Testing your Changes diff --git a/website/contributing/how-to-run-and-write-tests.md b/website/contributing/how-to-run-and-write-tests.md index 94a0a4c4532..8b98e08ad4f 100644 --- a/website/contributing/how-to-run-and-write-tests.md +++ b/website/contributing/how-to-run-and-write-tests.md @@ -48,6 +48,7 @@ Xcode also allows running individual tests through its Test Navigator. You can a If you are making modifications to `Podfile` configurations then there are Ruby tests that can verify these. To run the ruby tests: + ```bash cd scripts sh run_ruby_tests.sh @@ -79,16 +80,6 @@ Then, run the Android integration tests: ./scripts/run-android-local-integration-tests.sh ``` -### End-to-end Tests - -Finally, make sure end-to-end tests run successfully by executing the following script: - -```bash -./scripts/test-manual-e2e.sh -``` - -This script will make you select to run the RNTester app (that lives within `packages/rn-tester`) or it will generate a fresh new project in `/tmp/RNTestProject`. Follow the step-by-step instructions from the script to test successfully if your changes. - ## Writing Tests Whenever you are fixing a bug or adding new functionality to React Native, it is a good idea to add a test that covers it. Depending on the change you're making, there are different types of tests that may be appropriate. diff --git a/website/contributing/release-testing.md b/website/contributing/release-testing.md index aea3c951e96..fb01b4dc637 100644 --- a/website/contributing/release-testing.md +++ b/website/contributing/release-testing.md @@ -3,11 +3,20 @@ id: release-testing title: How to Test a Release --- -## Test source in the release branch - These steps needs to be followed by the release crew as part of the release process, to ensure that new versions published have a good level of stability. -### Pre-requisites +:::info + +An important element of testing source in the release branch is that this process needs to be preferably be done twice, on two separate machines - there are multiple scenarios in which something might fail on a machine but not on another. By running tests on two computers, we want to reach a step of double confirmation that everything works fine. +::: + +## Pre-requisites + +:::note + +Currently, this flow can only be done on macOS machines. + +::: - Have a clone of `react-native` repo and be on the release branch (`0.XX-stable`). @@ -19,14 +28,18 @@ These steps needs to be followed by the release crew as part of the release proc echo '{}' > .watchmanconfig ``` -- Have Android and iOS development environment set-up. Follow instructions for macOS/iOS and macOS/Android from the [Environment Setup](/docs/environment-setup) guide. +- Have Android and iOS development environment set-up. Follow instructions for `React Native CLI quickstart` for macOS/iOS and macOS/Android from the [Environment Setup](/docs/environment-setup) guide. #### Additional steps for Android - Gradle should now install [the appropriate NDK](https://github.com/facebook/react-native/blob/main/template/android/build.gradle). Verify that you have in your path the `ANDROID_NDK` variable, pointing to it. - In case you are on macOS Catalina (or higher), you might also need to run `sudo xattr -r -d com.apple.quarantine /path/to/ndk` to avoid the e2e script to fail. (_That said, this should not happen anymore since from NDK 21 and higher the Android team started signing the NDK._) -### Steps +## Steps + +### Clean up the local state + +When testing locally, we want to ensure that we start from a clean slate to avoid caches polluting our testing. 1. Delete `RNTester` and `RNTestProject` from your Android emulator and iOS simulator if leftover from previous test. 2. Remove any temporary files from the `react-native` repo: @@ -35,76 +48,73 @@ These steps needs to be followed by the release crew as part of the release proc git clean -fdx ``` + For `main` branch, and versions of RN >=0.71, you can instead use `yarn test-e2e-local-clean`. + 3. Install dependencies: ```bash yarn install - pushd packages/rn-tester - pod install --repo-update - popd ``` -4. Use the `test-manual-e2e` script to test `RNTester` and the template app (`RNTestProject`): +### Generating the projects - ```bash - # This will run you through the different variants in Test Dimensions table - ./scripts/test-manual-e2e.sh - ``` +The local testing for a release consist of running the [test project](https://github.com/facebook/react-native/tree/main/packages/rn-tester) of the react-native repository, `RNTester`, which contains an in-depth list of components implementations, and generating a fresh new project based on the local code, `RNTestProject`, that will simulate accurately how a `react-native init` project will behave. -5. Turn on Hermes in the `RNTestProject` and ensures it builds successfully. +To generate the the right project with the specific configuration desired, you can use the command - - Enable Hermes for Android: +```bash +yarn test-e2e-local [options] +``` - ```bash - # Update `/tmp/RNTestProject/android/app/build.gradle` to `enableHermes` - project.ext.react = [ - enableHermes: true, // clean and rebuild if changing - ] +Followed by the wanted options: - # Clean and rebuild - /tmp/RNTestProject/android$ ./gradlew clean - /tmp/RNTestProject$ npx react-native run-android - ``` +```bash + --help Show help [boolean] + --version Show version number [boolean] + -t, --target [choices: "RNTester", "RNTestProject"] [default: "RNTester"] + -p, --platform [choices: "iOS", "Android"] [default: "iOS"] + -h, --hermes [boolean] [default: true] +``` - - Enable Hermes for iOS: +#### Versions older than 71 - ```bash - # Update `/tmp/RNTestProject/ios/Podfile` and then run `pod install` - use_react_native!( - :path => config[:reactNativePath], - # to enable hermes on iOS, change `false` to `true` and then install pods - :hermes_enabled => true - ) +You need to use the interactive script run you through the different variants in [Test Dimensions](#test-dimensions): - # Install pods and run - /tmp/RNTestProject/ios$ pod install - /tmp/RNTestProject$ npx react-native run-ios - ``` +```bash +./scripts/test-manual-e2e.sh +``` -### Test Dimensions +This script will ask you to select which platform and which project you want to test, and then to execute a series of extra steps during the process. Bear in mind, when testing RNTester on Android, you need to start the Android emulator ahead of time or it will fail. -Covered by running `test-manual-e2e.sh`, see [issue](https://github.com/facebook/react-native/issues/33015) about supporting those "manual" cases. +## What to test? -| Variant | RNTester | Template App | -| ---------------- | ------------------------ | ------------------------ | -| Android - JSC | via `test-manual-e2e.sh` | via `test-manual-e2e.sh` | -| Android - Hermes | via `test-manual-e2e.sh` | manual | -| iOS - JSC | via `test-manual-e2e.sh` | via `test-manual-e2e.sh` | -| iOS - Hermes | via `test-manual-e2e.sh` | manual | +Aside from verifying that the building process is successful, once the app spawn by the script is up and running, we want to run a series of manual tests to ensure that some core functionalities work, like Fast Refresh and the Flipper debugger. -**Note well:** Starting from RN 0.70, Hermes is turned on by default so for the template app JSC will need to be manually tested by switching off Hermes. +In the `RNTester` you want to also play around with the app and try different components: some important onces are `Flatlist`, `Image` and the "New Architecture Component" (should be the very last one in the list). -### What to test? +### Test Dimensions -Aside from verifying that the building process is successful, once the app spawn by `test-manual-e2e.sh` is up and running, we want to run a series of manual tests to ensure that some core functionalities work, like Fast Refresh and the Flipper debugger. +To ensure that we cover the most use cases, we need to ensure we test all these different combination of configurations: -In the `RNTester` you want to also play around with the app and try different components: some important onces are `Flatlist`, `Image` and the "New Architecture Component" (should be the very last one in the list). +- RNTester + iOS + Hermes +- RNTester + iOS + JSC +- RNTester + Android + Hermes +- RNTester + Android + JSC +- RNTestProject + iOS + Hermes +- RNTestProject + iOS + JSC +- RNTestProject + Android + Hermes +- RNTestProject + Android + JSC + +:::note + +Bear in mind that RNTester project is already onboarded in the new architecture. `RNTestProject` is not - new architecture mode needs to be [enabled](/docs/the-new-architecture/use-app-template#enable-the-new-architecture) and tested separately. +::: -An important element of testing source in the release branch is that this process needs to be preferably be done twice, on two separate machines - there are multiple scenarios in which something might fail on a machine but not on another. By running tests on two computers, we want to reach a step of "double confirmation" that everything works fine. +## Testing pre-releases (RC) on production apps -## Testing an RC on a project - checklist +During the Release Candidate (RC) phase of a release cycle, we ask for the community to set as dependency in their apps the latest RC available and report in the related "Road to 0.XX" how it performs ([example](https://github.com/reactwg/react-native-releases/discussions/26)). -If you are a [release tester](./release-roles-responsibilities#release-tester-responsibilities), the ask for you is to set as dependency in your app the latest RC available and report in the related "Road to 0.XX" how it went ([example](https://github.com/reactwg/react-native-releases/discussions/26)). To help provide the relevant information, we have prepared this template: you can copy/pasted it in a comment and fill it accordingly. +To help provide the relevant information, we have prepared this template they can use as blueprint for what is important to test - they can copy/pasted it in a comment and fill it accordingly. ```markdown | Link to branch: | |