Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stabilize e2e tests for phishing detection #21152

Merged
merged 3 commits into from
Oct 3, 2023
Merged

Conversation

chloeYue
Copy link
Contributor

@chloeYue chloeYue commented Oct 2, 2023

Description

Several e2e testcases about phishing detection are flaky.

Why they are flaky?

After investigation, i think because when we open a malicious website, selenium fetch very quickly all the DOM elements and consider they are complete. When the page redirects to phishing detection page, it dose not re-fetch elements so selenium cannot find buttons to click.

How i resolve it?

I explicitly force selenium to switch to the phishing detection page each time after redirection so it will re-fetch all elements on this page.

Why do i think the fix works

For each testcase, I ran locally 20 times and they all passed.

Fixes #21150

@github-actions
Copy link
Contributor

github-actions bot commented Oct 2, 2023

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

@chloeYue chloeYue changed the title [DRAFT] Stabilize e2e tests for phishing detection Stabilize e2e tests for phishing detection Oct 2, 2023
@gauthierpetetin gauthierpetetin added needs-dev-review PR needs reviews from other engineers (in order to receive required approvals) in-progress and removed needs-dev-review PR needs reviews from other engineers (in order to receive required approvals) labels Oct 2, 2023
@codecov
Copy link

codecov bot commented Oct 3, 2023

Codecov Report

All modified lines are covered by tests ✅

Comparison is base (0482ab6) 68.57% compared to head (cdb5a82) 68.57%.
Report is 2 commits behind head on develop.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop   #21152   +/-   ##
========================================
  Coverage    68.57%   68.57%           
========================================
  Files         1013     1013           
  Lines        40615    40615           
  Branches     10854    10854           
========================================
  Hits         27848    27848           
  Misses       12767    12767           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@metamaskbot
Copy link
Collaborator

Builds ready [cdb5a82]
Page Load Metrics (866 ± 399 ms)
PlatformPageMetricMin (ms)Max (ms)Average (ms)StandardDeviation (ms)MarginOfError (ms)
ChromeHomefirstPaint911781262713
domContentLoaded751651102512
load872252866832399
domInteractive751651102512
Bundle size diffs
  • background: 0 Bytes (0.00%)
  • ui: 0 Bytes (0.00%)
  • common: 0 Bytes (0.00%)

@chloeYue chloeYue merged commit 0139b99 into develop Oct 3, 2023
64 checks passed
@chloeYue chloeYue deleted the stabilize-test-chloe branch October 3, 2023 16:32
@github-actions github-actions bot locked and limited conversation to collaborators Oct 3, 2023
@metamaskbot metamaskbot added the release-11.4.0 Issue or pull request that will be included in release 11.4.0 label Oct 3, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
release-11.4.0 Issue or pull request that will be included in release 11.4.0 team-extension-platform
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Stabilize e2e tests: phishing detection
5 participants