Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix flakey tests #3227

Merged
merged 16 commits into from
Jan 29, 2019
Merged

fix flakey tests #3227

merged 16 commits into from
Jan 29, 2019

Conversation

kuceb
Copy link
Contributor

@kuceb kuceb commented Jan 25, 2019

culprit 1

driver/src/cy/timers does not fire queued timers if page transitions while paused

estimated failure rate: 8%

example

https://circleci.com/gh/cypress-io/cypress/54032#tests/containers/1

image

fix

upping the timeout here seems to fix it

culprit 2

src/cy/commands/xhr abort does not set aborted on XHR's that have completed by have had .abort() called

estimated failure rate: 5%

example

https://circleci.com/gh/cypress-io/cypress/54474#tests/containers/1
image

fix

apparently a .should chained off a .then will not retry.

so I used cy.wrap(null).should

@kuceb
Copy link
Contributor Author

kuceb commented Jan 25, 2019

ironically another flaky test just failed

@brian-mann brian-mann self-requested a review January 25, 2019 20:57
Copy link
Member

@brian-mann brian-mann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the timers aren't fired while the page is transition that's a bigger problem and one that we need to investigate and try to fix for real...

i can pair with you on it

@kuceb kuceb force-pushed the flaky-timers_spec branch 2 times, most recently from 1add788 to 623c4f0 Compare January 25, 2019 22:04
@kuceb kuceb force-pushed the flaky-timers_spec branch from 623c4f0 to f98bad0 Compare January 25, 2019 22:09
@kuceb kuceb changed the title fix flakey timers_spec fix flakey tests Jan 25, 2019
@kuceb
Copy link
Contributor Author

kuceb commented Jan 25, 2019

if the timers aren't fired while the page is transition that's a bigger problem and one that we need to investigate and try to fix for real...

i can pair with you on it

since this moves the chains, can it go ahead and go in?

@brian-mann
Copy link
Member

absolutely

@kuceb kuceb requested a review from brian-mann January 28, 2019 22:37
@jennifer-shehane jennifer-shehane added the process: tests Related to our internal tests label Jan 29, 2019
// there were times where the driver's webserver
// was sending bytes after 200ms (TTFB) that caused
// this test to be flaky.
win.setTimeout(stub1, 500)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just thought of a better approach here...

Instead of hoping the webserver sends us back the response in 500ms, we could instead use cypress events to wait for the window:unload event, gauranteeing this has happened.

At that point we could then set the setTimeout and then assert its not called after waiting.

@brian-mann brian-mann merged commit acceb79 into develop Jan 29, 2019
@cypress-bot
Copy link
Contributor

cypress-bot bot commented Jan 30, 2019

Released in 3.1.5.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
process: flaky test Related to test(s) that have flake in our internal tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants