HTTP requests time out when Cypress is executed within Docker and Jenkins #1039

ckrohn · 2017-12-07T21:11:58Z

Is this a Feature or Bug?

Bug

Current behavior:

Within our company we're planning to introduce Cypress as our main tool for testing end-to-end processes within our web applications.

First of all: Cypress looks amazing and we really wan't to give it a shot!

Unfortunately, we're having some trouble while automating the tests within our Docker- and Jenkins-based build environment.

Our build infrastructure looks like this:

For our test case, we've got a VM with RHEL 7.4 with kernel 3.10.0 that is basically only running an installation of Docker (version 17.06.2-ce).
On this machine we're currently running Jenkins 2.93 within a Docker container and one single Jenkins slave container based on the official Cypress Docker image on the same VM.

Before I continue: we've tried every single Jenkins version from 2.60 until 2.93 and all of the Cypress base images (including the Chrome integration which works smoothly since 1.1.4).

Every combination shows the same behaviour:

When we're trying to execute our Cypress tests on the before mentioned Jenkins Slave Container, everything works fine until Cypress actually calls the website under test. What happens is, that all the requests run in a timeout.

When I look into the video recorded during the test run within the CI setup, I can see that Cypress calls the correct website but loads indefinitely. I've tried to increase all the configurable timeouts to ridiculous values with no effect.

HTTP requests sent manually from within the Docker container work flawlessly in no time and a similar setup on different local machines works perfectly too.

What might be important: We're working within a secure environment where we have to open every single communication way within the firewall. I suspect, that Cypress might possibly do something when requesting a website that a simple wget or curl might not and that we simply have to configure our firewall appropriately.

The website we're trying to test runs on a different machine in a different network but the communication via 80/TCP and 443/TCP has been opened (as our manual tests from within the container prove).

Is there anything special we have to consider within our firewall or network configuration?

Desired behavior:

Requests are executed correctly and don't run in a timeout.

How to reproduce:

Actually I'm not sure how to reproduce this behaviour because everything works on our local machines.

Test code:

CYPRESS_BINARY_VERSION=/home/person/cypress.zip npm install --no-progress --verbose
DEBUG=cypress:* $(npm bin)/cypress run --env HOST=dev --record --key our-key

Additional Info (images, stack traces, etc)

CypressError: Timed out after waiting '60000ms' for your remote page to load.

Your page did not fire its 'load' event within '60000ms'.

You can try increasing the 'pageLoadTimeout' value in 'cypress.json' to wait longer.

Browsers will not fire the 'load' event until all stylesheets and scripts are done downloading.

When this 'load' event occurs, Cypress will continue running commands.

Because this error occurred during a 'before all' hook we are skipping the remaining tests in the current suite: 'Elastic Search'
    at Object.cypressErr (http://-G4/__cypress/runner/cypress_runner.js:65873:11)
    at Object.throwErr (http://our.test.website/__cypress/runner/cypress_runner.js:65838:18)
    at Object.throwErrByPath (http://our.test.website/__cypress/runner/cypress_runner.js:65865:17)
    at timedOutWaitingForPageLoad (http://our.test.website/__cypress/runner/cypress_runner.js:56915:17)
    at http://our.test.website/__cypress/runner/cypress_runner.js:57516:18
    at tryCatcher (http://our.test.website/__cypress/runner/cypress_runner.js:6268:23)
    at http://our.test.website/__cypress/runner/cypress_runner.js:1585:41
    at tryCatcher (http://our.test.website/__cypress/runner/cypress_runner.js:6268:23)
    at Promise._settlePromiseFromHandler (http://our.test.website/__cypress/runner/cypress_runner.js:4290:31)
    at Promise._settlePromise (http://our.test.website/__cypress/runner/cypress_runner.js:4347:18)
    at Promise._settlePromise0 (http://our.test.website/__cypress/runner/cypress_runner.js:4392:10)
    at Promise._settlePromises (http://our.test.website/__cypress/runner/cypress_runner.js:4467:18)
    at Async._drainQueue (http://our.test.website/__cypress/runner/cypress_runner.js:1200:16)
    at Async._drainQueues (http://our.test.website/__cypress/runner/cypress_runner.js:1210:10)
    at Async.drainQueues (http://our.test.website/__cypress/runner/cypress_runner.js:1084:14)]]></failure>
    </testcase>
    <testcase name="Elastic Search &quot;after all&quot; hook for &quot;with map&quot;" time="60.079" classname="&quot;after all&quot; hook for &quot;with map&quot;">
      <failure><![CDATA[Error: Cypress command timeout of '4000ms' exceeded.

Because this error occurred during a 'after all' hook we are skipping the remaining tests in the current suite: 'Elastic Search'
    at http://our.real.domain/__cypress/runner/cypress_runner.js:64279:25]]></failure>

Operating System: RHEL 7.4 (3.10.0)
Cypress Version: 1.1.3, 1.1.4
Browser Version: Electron, Chrome 62

The text was updated successfully, but these errors were encountered:

brian-mann · 2017-12-07T22:02:19Z

Cypress is proxying all of the requests that the browser makes.

By chance does your environment also require a proxy? You mentioned firewall, but is it possible that a proxy is also part of this setup?

Can you see anything via cy.visit? In other words is the page partially loading, but never completing? Or does it just sit at a blank white screen?

Also can you run a single test using cy.request instead of a cy.visit to see if those go out correctly?

They actually use different implementations under the hood and that would be valuable to know.

There was an issue opened a couple days ago specifically here: #1013 (comment)

They refer to requests indefinitely timing out (but I haven't actually looked at this yet) Perhaps this is a related problem.

Whatever it is I'm confident its in the proxy layer and we'll be able to sort it out. Reproducing your environment would be nice, but I'm not sure if its feasible. Perhaps there is some docker configuration you can share that reproduces it.

The only other thing I can offer is direct support from our team itself to work with you 1:1 to get it resolved.

ckrohn · 2017-12-07T22:29:49Z

Thanks for the blazingly fast reply!

I'm pretty sure our environment doesn't require a proxy but I'll talk to our IT guys tomorrow to clarify this.

When I'm using cy.visit, the page just sits at a blank white screen and basically loads as long as the configured pageLoadTimeout.

Funnily enough calling the application via cy.request results in an HTTP 200 within the expected response time, what makes me pretty confident that we'll be able fix this somehow.

If you already have some suggestions based on my previous statement, let me know.

If you need further information about our docker and network configuration anyways, I'll try to put it together tomorrow and attach it to this issue.

PS: The issue has been labeled with CI: travis but I think it should be CI: jenkins :)

ggauravr · 2018-02-13T23:17:01Z

@brian-mann @christopher-krohn any update on this guys? I'm seeing a similar issue, but with Docker and TeamCity though. Sounds like if it's something with the Docker or Cypress config, might fix my issue as well.

brian-mann · 2018-02-13T23:35:19Z

We actually may have discovered something causing network request delays in Chrome. It's the argument --disable-background-networking. This would only affect Chrome, not Electron.

In the newest version of Cypress 2.0.0 (pending release) we've removed this flag - but you can utilize the plugin API to slice it out of the browser args today if you'd like to see what it's like to run without it.

https://docs.cypress.io/api/plugins/browser-launch-api.html

americos · 2018-03-15T18:57:01Z

I'm having this same issue, almost exact same environment... but I'm sure I'm behind a corporate proxy.

Would be great if we could pass a proxy setting on cy.visit() or cy.request()

andreyselitsky · 2018-07-20T08:57:57Z

We are experiencing the same issue and we are on Cypress 3.0.2. Also it occurs sporadically, making our tests flaky. Any ideas what we could do here?

To give a little bit of the context - it is Angular 5 app, there are multiple .js chunks that are being loaded dynamically during the start of anapplication (default ng/webpack setup).

We managed to reproduce this issue locally and we saw very strange picture in Chrome's network tab - it was showing that request is loading for minutes (2 minutes unless ng timeout aborted it). At the same time we checked nginx/node logs and they are showing that file was served in milliseconds.

jennifer-shehane · 2018-07-20T21:23:08Z

@andreyselitsky May be related to this other request timeout at 2 mins issue: #2055 (comment)

andreyselitsky · 2018-07-23T09:25:59Z

@jennifer-shehane it might be somehow related. We will try increasing the pageLoadTimeout to 3 minutes and see if we see anything after 2 minutes. Because right now our tests are just failing with pageLoadTimeout and it is hard to say which request was problematic.

UPDATE: So we've got the following error in cypress console when debug was enabled:

Mon, 23 Jul 2018 10:16:50 GMT cypress:server:proxy request failed { url: '*****.***/main.fefede1e40e5b31fe2f5.js', status: 500, error: 'Error: ESOCKETTIMEDOUT\n    

at ClientRequest.<anonymous>(/Users/*/Library/Caches/Cypress/3.0.2/Cypress.app/Contents/Resources/app/packages/server/node_modules/request/request.js:812:19)\n    
at Object.onceWrapper (events.js:314:30)\n    
at emitNone (events.js:105:13)\n    
at ClientRequest.emit (events.js:207:7)\n    
at TLSSocket.emitTimeout (_http_client.js:722:34)\n    
at Object.onceWrapper (events.js:314:30)\n    
at emitNone (events.js:105:13)\n    
at TLSSocket.emit (events.js:207:7)\n    
at TLSSocket.Socket._onTimeout (net.js:402:8)\n    
at ontimeout (timers.js:469:11)\n    
at tryOnTimeout (timers.js:304:5)\n    
at Timer.listOnTimeout (timers.js:264:5)\n' }

Any ideas?

mattvb91 · 2018-08-27T09:07:56Z

👍 for visibility. Same issue here. No proxy required in our env. v3.1.0

darenyong · 2018-09-05T15:58:39Z

I'm also having a similar issue. Same error displayed, the docker image works fine on my mac, but when I run the container in the corporate lab (where Jenkins will ssh into), the page load timeout error occurs when running cypress after the node app has been started.

Note: If I map the container port 80 to host 8080, and hit the http://lab.machine:8080, I can load and view the node app just fine.

darenyong · 2018-09-05T17:33:26Z

I have tracked down my docker cypress proxy issue to my CSS.

To add some detail to my situation, I have defined the HTTP_PROXY, HTTPS_PROXY and NO_PROXY, by passing -e parameter to docker run.

When I shell into the container, and execute the following command, the proxy settings appear to work and the request succeeds.

curl "https://fonts.googleapis.com/css?family=Source%20Sans%20Pro:400,700,400italic,700italic&subset=latin"

When I run Cypress test to cy.visit() my node app, it loads the index.html, but appears to hang inside Electron waiting for a response for loading the google fonts above.

If I remove loading the CSS in my node app, then Cypress works fine, the test runs to completion, AND a video of the test is successfully uploaded to the Cypress Dashboard in the cloud (which indicates Cypress IS using the proxy for the video upload)

However, when running the actual test, Cypress appears to not use the corporate proxy and the request to fonts.googleapis.com will hang.

JamesRamm · 2018-09-19T08:08:04Z

I have the same problem, but simply running docker on localhost.

In fact, I can launch all my services with docker and then run cypress on the host (not in docker) and all my tests will past.

But if I launch cypress in a docker container, I get random timeout errors. This is running docker on localhost.

andreyselitsky · 2018-10-10T09:29:53Z

Folks, we managed to resolve this by replacing fetch with XHR and this problem disappeared #95 (comment)

Though, It might be also fixed by some other changes in application code, we are not sure 100%.

mattvb91 · 2018-10-11T09:27:07Z

@andreyselitsky could you expand on that? Using that exact snippet fixed the timeout issue in docker/jenkins? Cheers

andreyselitsky · 2018-10-11T09:43:54Z

@mattvb91 this is my guess, I can't prove it.

But when we switched from the native fetch to XHR we didn't get timeouts inside docker on ci anymore.

We basically used this recipe https://github.com/cypress-io/cypress-example-recipes/blob/master/examples/stubbing-spying__window-fetch/cypress/integration/polyfill-fetch-from-tests-spec.js

mattvb91 · 2018-10-11T11:46:32Z

@andreyselitsky thanks will try it out and report back.

JohnnyQQQQ · 2018-12-13T19:10:38Z

We are having the same issue with GitLab-CI running docker and the latest cypress version.

cookuricoo · 2018-12-25T15:01:41Z

Same issue with Jenkins

jkwohlfahrt · 2019-01-09T20:39:32Z

I also am seeing this in my company's Jenkins pipelines instance.

jkwohlfahrt · 2019-01-09T22:18:54Z

Maybe I can provide a little more context as well. I am running our cypress test suite across two environments, qa and production. Our qa test suite runs fine and snappy in Docker and Jenkins pipelines. Our production test suite has been having the timeout issues in the Docker/Jenkins CI setup, but runs consistently without issue running locally in Docker. I bumped our config timeout from the default of 60 seconds to a much longer three minute timeout, and the requests finally did resolve on production, but sometimes took a little over two minutes to resolve. I guess extending the timeout is somewhat of a workaround, but certainly not a great one. I'm working with our in-house devops engineer to see if we can discover anything further. Will report back if I do.

jennifer-shehane · 2019-01-29T04:57:20Z

I know this is difficult to provide a reproducible example of, but there seem to be several issues described within here - some with proxy, some without - some with fetch, some without.

We'll need a reproducible example to narrow this down exactly - something we can run locally and see these exact timeouts. Can anyone provide one?

mattvb91 · 2019-01-30T10:12:44Z

Just to add some more info this issue is not restricted to docker/jenkins. Just attempted to install a new gitlab-runner as "shell" executor and have cypress running on the host system without docker and still getting timeouts (albeit less)

jkwohlfahrt · 2019-02-01T20:39:44Z

I was finally able to fix this issue for my test suite. After doing some network tracing on the url I was trying to visit, i noticed the page trying to load a different domain. Because cypress does not allow this, the request times out after two minutes, I'm guessing due to #2055 (comment). I was able to work around this by blacklisting the problematic domain. I'm wondering if this might be the issue for others on this thread.

mattvb91 · 2019-02-08T10:38:37Z

@jkwohlfahrt been tracing the network requests too including blacklisting and noticed that even with a blacklist on certain domains and the network tab showing status code 503 for the blacklisted domain it still hangs on "content download". I do think this maybe the right direction. Does it still download the content on your side even tho it has been blacklisted?

Edit: The request above with "content download time" and status of 503 has the "x-cypress-matched-blacklisted-host" response header in it matching the blacklisted domain so im not sure why it hangs there.

Related to #2714?

Edit 2: After more network tracing I've actually managed to get the timeout on my local machine (without docker) and the timeline for the request that fails looks like this (Failed attempting to load a css file, css is hosted on the same domain the tests are ran on this is not an external domain.):

mattvb91 · 2019-02-15T14:46:56Z

@jennifer-shehane any idea from your side if the comment above could be related to this otherwise I might open up a new issue for it.

cypress-bot · 2019-04-01T03:39:31Z

The code for this is done in cypress-io/cypress#3531, but has yet to be released.
We'll update this issue and reference the changelog when it's released.

cypress-bot · 2019-05-17T18:52:55Z

Released in 3.3.0.

jennifer-shehane added CI: travis labels Dec 7, 2017

jennifer-shehane added CI: jenkins and removed CI: travis labels Dec 7, 2017

jennifer-shehane mentioned this issue Mar 16, 2018

Support proxies and improve proxy performance #1469

Closed

10 tasks

slawiko mentioned this issue Jul 23, 2018

Unexpected child sending timer id #2197

Closed

mabedan mentioned this issue Jan 14, 2019

getting random http response code 500 #2218

Open

jennifer-shehane added the CI: gitlab label Jan 29, 2019

jennifer-shehane added the stage: needs information Not enough info to reproduce the issue label Jan 29, 2019

jennifer-shehane mentioned this issue Jan 31, 2019

Cypress behind corporate proxy with cross-domain static content times out #2295

Closed

mattvb91 mentioned this issue Feb 8, 2019

Page occasionally does not load from cy.visit() #2938

Closed

flotwig mentioned this issue Feb 20, 2019

Improved proxy support #3531

Merged

38 tasks

cypress-bot bot added stage: work in progress and removed stage: needs information Not enough info to reproduce the issue labels Mar 1, 2019

brian-mann assigned flotwig Apr 1, 2019

brian-mann closed this as completed in #3531 Apr 1, 2019

cypress-bot bot added stage: pending release and removed stage: work in progress labels Apr 1, 2019

cypress-bot bot removed the stage: pending release label May 17, 2019

dialex mentioned this issue Jul 5, 2019

Blacklisting only works sometimes (flaky tests) #4656

Closed

cypress-io locked as resolved and limited conversation to collaborators Dec 5, 2019

jennifer-shehane added CI General issues involving running in a CI provider and removed CI: jenkins labels Oct 5, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HTTP requests time out when Cypress is executed within Docker and Jenkins #1039

HTTP requests time out when Cypress is executed within Docker and Jenkins #1039

ckrohn commented Dec 7, 2017 •

edited by jennifer-shehane

Loading

brian-mann commented Dec 7, 2017 •

edited

Loading

ckrohn commented Dec 7, 2017 •

edited

Loading

ggauravr commented Feb 13, 2018

brian-mann commented Feb 13, 2018

americos commented Mar 15, 2018

andreyselitsky commented Jul 20, 2018 •

edited

Loading

jennifer-shehane commented Jul 20, 2018

andreyselitsky commented Jul 23, 2018 •

edited

Loading

mattvb91 commented Aug 27, 2018

darenyong commented Sep 5, 2018

darenyong commented Sep 5, 2018 •

edited by jennifer-shehane

Loading

JamesRamm commented Sep 19, 2018 •

edited by jennifer-shehane

Loading

andreyselitsky commented Oct 10, 2018

mattvb91 commented Oct 11, 2018

andreyselitsky commented Oct 11, 2018

mattvb91 commented Oct 11, 2018

JohnnyQQQQ commented Dec 13, 2018

cookuricoo commented Dec 25, 2018

jkwohlfahrt commented Jan 9, 2019 •

edited

Loading

jkwohlfahrt commented Jan 9, 2019 •

edited by jennifer-shehane

Loading

jennifer-shehane commented Jan 29, 2019

mattvb91 commented Jan 30, 2019

jkwohlfahrt commented Feb 1, 2019

mattvb91 commented Feb 8, 2019 •

edited

Loading

mattvb91 commented Feb 15, 2019

cypress-bot bot commented Apr 1, 2019

cypress-bot bot commented May 17, 2019

HTTP requests time out when Cypress is executed within Docker and Jenkins #1039

HTTP requests time out when Cypress is executed within Docker and Jenkins #1039

Comments

ckrohn commented Dec 7, 2017 • edited by jennifer-shehane Loading

Is this a Feature or Bug?

Current behavior:

Desired behavior:

How to reproduce:

Test code:

Additional Info (images, stack traces, etc)

brian-mann commented Dec 7, 2017 • edited Loading

ckrohn commented Dec 7, 2017 • edited Loading

ggauravr commented Feb 13, 2018

brian-mann commented Feb 13, 2018

americos commented Mar 15, 2018

andreyselitsky commented Jul 20, 2018 • edited Loading

jennifer-shehane commented Jul 20, 2018

andreyselitsky commented Jul 23, 2018 • edited Loading

mattvb91 commented Aug 27, 2018

darenyong commented Sep 5, 2018

darenyong commented Sep 5, 2018 • edited by jennifer-shehane Loading

JamesRamm commented Sep 19, 2018 • edited by jennifer-shehane Loading

andreyselitsky commented Oct 10, 2018

mattvb91 commented Oct 11, 2018

andreyselitsky commented Oct 11, 2018

mattvb91 commented Oct 11, 2018

JohnnyQQQQ commented Dec 13, 2018

cookuricoo commented Dec 25, 2018

jkwohlfahrt commented Jan 9, 2019 • edited Loading

jkwohlfahrt commented Jan 9, 2019 • edited by jennifer-shehane Loading

jennifer-shehane commented Jan 29, 2019

mattvb91 commented Jan 30, 2019

jkwohlfahrt commented Feb 1, 2019

mattvb91 commented Feb 8, 2019 • edited Loading

mattvb91 commented Feb 15, 2019

cypress-bot bot commented Apr 1, 2019

cypress-bot bot commented May 17, 2019

ckrohn commented Dec 7, 2017 •

edited by jennifer-shehane

Loading

brian-mann commented Dec 7, 2017 •

edited

Loading

ckrohn commented Dec 7, 2017 •

edited

Loading

andreyselitsky commented Jul 20, 2018 •

edited

Loading

andreyselitsky commented Jul 23, 2018 •

edited

Loading

darenyong commented Sep 5, 2018 •

edited by jennifer-shehane

Loading

JamesRamm commented Sep 19, 2018 •

edited by jennifer-shehane

Loading

jkwohlfahrt commented Jan 9, 2019 •

edited

Loading

jkwohlfahrt commented Jan 9, 2019 •

edited by jennifer-shehane

Loading

mattvb91 commented Feb 8, 2019 •

edited

Loading