-
-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test: add blacklist URLs to automated tests #8558
Conversation
CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes. |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #8558 +/- ##
==========================================
+ Coverage 40.90% 41.08% +0.18%
==========================================
Files 1245 1245
Lines 30150 30330 +180
Branches 2895 2962 +67
==========================================
+ Hits 12333 12462 +129
- Misses 17114 17125 +11
- Partials 703 743 +40 ☔ View full report in Codecov by Sentry. |
E2E test started on Bitrise: https://app.bitrise.io/app/be69d4368ee7e86d/pipelines/8d629dc6-7dc7-446f-aa7c-3e327d6ce10c |
E2E test started on Bitrise: https://app.bitrise.io/app/be69d4368ee7e86d/pipelines/f008d8e9-648a-4be2-ad34-8efb9862b323 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
E2E test started on Bitrise: https://app.bitrise.io/app/be69d4368ee7e86d/pipelines/f2e0bd98-1f5c-45dc-a610-b54099b97c46 |
Quality Gate passedIssues Measures |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
✅ ✅
Description
This PR decreases delays in mobile automated testruns by blacklisting URLs that do not have any impact on tests. Request made from the app that take awhile to resolve delays the tests running. Blacklisted URLs that don't impact tests will allow the test to continue to execute without waiting for request to resolve for blacklisted URLs.
Running automated tests locally shows a more noticeable improvement.
Information related to how blacklisting URLs using detox: https://wix.github.io/Detox/docs/api/device/#deviceseturlblacklisturls
https://wix.github.io/Detox/docs/api/device/#11-detoxurlblacklistregexinitialize-the-url-blacklist-at-app-launch
https://wix.github.io/Detox/docs/api/device/#deviceapplaunchargs
Challenges encountered can be found here.
Related issues
Fixes: #https://github.com/MetaMask/mobile-planning/issues/1532
Manual testing steps
Scenario: Running tests on bitrise
Given I am on bitrise
And blacklisted URLs have been implemented
When I kick-off automation tests pipeline
Then all tests pass when pipelines are complete
And testrun duration is decreased
And bitrise credit usage is decreased
Scenario: Running tests locally
Given blacklist URLs have been implemented
When I run automation tests on my local system
Then blacklisted URLs will not show as waiting for response in terminal output
And all tests pass
Screenshots/Recordings
Before
Screen.Recording.2024-02-12.at.12.41.06.AM.mov
https://app.bitrise.io/app/be69d4368ee7e86d/pipelines/a8db46fc-f8ba-4e9b-9180-3e0b0656d1fc
https://app.bitrise.io/app/be69d4368ee7e86d/pipelines/2adb4469-2e8d-4541-b608-8f8974eb097f
After
Screen.Recording.2024-02-12.at.11.08.18.AM.mov
https://app.bitrise.io/app/be69d4368ee7e86d/pipelines/e6127952-b019-41e5-9e7b-b0a2bd449f01
https://app.bitrise.io/app/be69d4368ee7e86d/pipelines/772698e3-594f-4f68-bf76-63b01b87398a
Pre-merge author checklist
Pre-merge reviewer checklist