Skip to content

Commit

Permalink
Move E2E tests to larger machines to see if flakyness decreases (#46960)
Browse files Browse the repository at this point in the history
Summary:
This change move E2E tests to larger mac machines to see if we can reduce the flakyness.
Most of the time the flakyness is due to slowness in connecting to the simulator or slowness in Metro to provide the bundle to RNTester when we run the tests in Debug.
More power should speed-up those processes.

E2E tests took < 30 min in this diff
on Main, they took > 120 min

## Changelog:
[Internal] - Move the E2E tests to bigger machines

Pull Request resolved: #46960

Test Plan: GHA

Reviewed By: blakef

Differential Revision: D64237659

Pulled By: cipolleschi

fbshipit-source-id: c2458413c8de70c07fae8f2b5f202371c6293815
  • Loading branch information
cipolleschi authored and facebook-github-bot committed Oct 11, 2024
1 parent 2c45675 commit d4edbcf
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 5 deletions.
10 changes: 9 additions & 1 deletion .github/actions/maestro-ios/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ runs:
run: |
cd ${{ inputs.working-directory }}
yarn start &
sleep 5 # to give metro time to load
- name: Run tests
id: run-tests
shell: bash
Expand All @@ -68,8 +69,13 @@ runs:
echo "Launch the app"
xcrun simctl launch $UDID ${{ inputs.app-id }}
if [[ ${{ inputs.flavor }} == 'Debug' ]]; then
# To give the app time to warm the metro's cache
sleep 5
fi
echo "Running tests with Maestro"
export MAESTRO_DRIVER_STARTUP_TIMEOUT=1800000 # 30 min. CI is extremely slow
export MAESTRO_DRIVER_STARTUP_TIMEOUT=1200000 # 20 min. CI is extremely slow
# Add retries for flakyness
MAX_ATTEMPTS=5
Expand Down Expand Up @@ -112,6 +118,8 @@ runs:
video_record_1.mov
video_record_2.mov
video_record_3.mov
video_record_4.mov
video_record_5.mov
report.xml
- name: Store Logs
if: failure() && steps.run-tests.outcome == 'failure'
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/test-all.yml
Original file line number Diff line number Diff line change
Expand Up @@ -183,8 +183,8 @@ jobs:
react-native-version: ${{ needs.prepare_hermes_workspace.outputs.react-native-version }}

test_e2e_ios_rntester:
if: ${{ github.ref == 'refs/heads/main' || contains(github.ref, 'stable') || inputs.run-e2e-tests }}
runs-on: macos-13
# if: ${{ github.ref == 'refs/heads/main' || contains(github.ref, 'stable') || inputs.run-e2e-tests }}
runs-on: macos-13-large
needs:
[build_apple_slices_hermes, prepare_hermes_workspace, build_hermes_macos]
env:
Expand Down Expand Up @@ -221,8 +221,8 @@ jobs:
flavor: ${{ matrix.flavor }}

test_e2e_ios_templateapp:
if: ${{ github.ref == 'refs/heads/main' || contains(github.ref, 'stable') || inputs.run-e2e-tests }}
runs-on: macos-13
# if: ${{ github.ref == 'refs/heads/main' || contains(github.ref, 'stable') || inputs.run-e2e-tests }}
runs-on: macos-13-large
needs: build_npm_package
env:
HERMES_WS_DIR: /tmp/hermes
Expand Down

0 comments on commit d4edbcf

Please sign in to comment.