-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Large XHR response objects can exceed the maximum header size. #76
Comments
Okay this is not a quick or easy fix. Node sets a constant maximum total header size at Probably the only way to do this is transfer the data over websockets while the server begins handling the request. That means managing all kinds of state, but this would work. There are some other features of the XHR refactor that would also introduce websockets, so I'll hold off on implementing this until it can go in with some new architecture. In the mean time you'll have to reduce the response sizes to less than 80kb, or just use a fixture and not an alias. So just move cy.route("POST", /route/, "fixture:reallyLargeObject") |
Trying to find a workaround for this, e.g. manipulating the request, or the response, or the server object. The available hooks fire after the request is sent, and after the response is delivered to the app, so there's no opportunity to influence it. Also haven't found any way to access the server object, e.g. to override a method. Any suggestions? Using a fixture doesn't help, because it's fixed. ;) |
If your fixture exceeds the maximum headers you'll need to modify your app code, such as using a conditional if Then ahead of time in your test code, you can use a fixture and set that property on the Cypress object so your app code can pick it up. |
The key takeaway here is that Cypress is just javascript and it has native access to everything. You can share properties and data. You can collaborate between Cypress and your application code. If your application code is "hardwired" to take a specific path, you can just modify it to open a seam to which you can use Cypress to modify that behavior. |
Just realized I could wrap onreadstatechange during onRequest, to flow the data into the app. cy.route({
url: '/api/bookmarks/bookmark*',
method: 'GET',
onRequest: xhr => {
var orsc = xhr.xhr.onreadystatechange;
xhr.xhr.onreadystatechange = function() {
if (this.readyState === 4) {
Object.defineProperty(this, 'response', {
writable: true
});
this.response = content;
}
orsc.apply(this, arguments);
};
},
response: 'placeholder',
}); The Cypress console shows response 'placeholder', but otherwise this functions as a workaround. |
Yup that would work. You could also bind to |
Thank you for your suggestions. For guys coming from google. Here is our workaround: put this in e.g. export default function loadLargeFixture(url, response, method = 'GET') {
return cy.route({
url,
method,
onRequest: xhr => {
const originalOnLoad = xhr.xhr.onload;
xhr.xhr.onload = function() {
Object.defineProperty(this, 'response', {
writable: true
});
this.response = response;
originalOnLoad.apply(this, xhr);
};
}
});
} Import the json and the import largeFixture from '../fixtures/veryLargeJson.json';
import loadLargeFixture from '../utils/loadLargeFixture';
describe('demo', () => {
it('should work with large fixture', () => {
cy.server();
loadLargeFixture('/api/large', largeFixture).as('data');
// do stuff
cy.wait('@data')
// do stuff
});
}); |
I spent some time stuck on this today. 😢 It wasn't very obvious because I made a change to my fixtures that must have JUST put it over size limit for responses. So working yesterday -> not working today. The
I was able to inspect the network panel and see that the response had a size of 0 bytes, which clued me in on it being this issue. This only breaks when I generate a bunch of data for an array response (like to check pagination), so would be nice to have fixed. |
hah just found a super simple solution: cy.fixture(`${endpoint}.json`).as(`fixture${endpoint}`);
cy.route(path, `@fixture${endpoint}.json`)
.as(endpoint); now: cy.route(path, `fixture:${endpoint}.json`)
.as(endpoint); Cleaner and it works :) |
This error mixed with the no support for fetch() ruined my afternoon, and made me think I was crazy. Please add note in docs where it is suggested to use cy.fixture aliases when it seems to not be great for large jsons. Just not using aliases like @tolicodes suggested fixed everything. |
I just ran into this today! @tolicodes method worked perfectly for me with a 1Mb fixture! |
@jennifer-shehane i'm getting the same Looking at the solution from @brian-mann at the top; I'm confused about the solution and why what I'm doing won't work... I'm not using an alias. options = {
method: 'POST',
status: 200,
url: '/playlist/search'
}
cy.fixture('playlist/search/playlist-search.dto.mock.json').then((data) => {
options['response'] = data;
cy.route(options);
}); Cheers |
@jennifer-shehane @brian-mann I've found something interesting, as the response sent through the server is meant to be in the body, why the response is sitting in the headers? Is there any reason to do that ? Because of this, the header gets much, much bigger than it needs Is this a behavior or something else ? [EDIT] I've managed to make the header less than 80kb and make it work again, but, still, can we make it work not by sending responses through the header ? |
So I unfortunately and traumatically ran into this bug as well. Unfortunately, I thought it was an issue on my side so I started to debug it and after rigorous testing found that the issue relates to big response size passed to As I mentioned before - Our fixtures are programmatic, which means that we don't have static json files holding fixtures and rather we have functions that we call and return as customizable fixtures. Therefore, @tolicodes solution doesn't seem to be relevant for us (unless I missed something about how to use cypress
|
Similarly to @nitzansan none of the workarounds with xhr events worked for my programmatic data. What did work is a nasty hack of pre-creating a fixture programmatically:
Can work as long as unique file names used and the fixtures saved this way are |
* fix specs * use debugger protocol for cookie handling in electron * use latest gulp * use rimraf instead of gulp-clean * use electron 3.1.8 and node 10.2.1 * use gulp 4 in packages/static * fix sendCommandAsync, log Schema.getDomains on CDP connect * autofill e2e test name [skip ci] * electron@5.0.7, see what new failures exist * --no-sandbox for launching Electron * update cookies logic for electron * node 12 * update snapshot for new node * update error message for new node * stub sendCommandAsync * only connect to socket if path has been replaced, fixes #4776 * update node-sass to support node 12 * skip wacky socket tests for now * snapshot * fix run_plugins_spec snapshot, don't include stack trace * use --no-sandbox on linux to run as root * allow sendCommandAsync to resolve * use euid for root check * log domains even if undefined * don't worry about ending 1xx responses immediately anymore * use --max-http-header-size, change max size from 8kb to 1mb, fix #76 * do not send 502 on failed websocket, just send back ECONNRESET * update websocket spec port to not collide with other test * update outdated expect * Revert "only connect to socket if path has been replaced, fixes #4776" This reverts commit f179eda. * update gulp in root * update https-proxy unit tests * update network spec to properly close server * update reporter spec * update https-proxy-agent to fix node 10.10.0 change discussion: https://github.com/nodejs/node/issues/24474\#issuecomment-511963799 * only pass --max-http-header-size on node >=12 * use own server-destroy implementation that supports secureConnect events * oops * update socket_spec * electron 6.0.0 * console.table introduced in node 10 * change browserify entry to init.js * handle edge case when no response body * console.table added in node 10 * do not exit app when all BrowserWindows are closed * update e2e snapshots * value may not be null * update plugins spec * correct cookie expiry, use browser.getversion for CDP version check * fix snapshotting for require stacks * reorder cookies in spec * warn when depreated electron callback apis are used * only report 1 plugin error per process * cleanup * node 12.4.0, cypress/browsers:node12.4.0-chrome76 docker image * update shell.openExternal to promisified * update dialog.showOpenDialog to promisified * update webContents.session.setProxy to promisified * updating native dependencies since we don't need ancient node ABI support anymore * WIP: switch cookies to simpler, jar-less approach * WIP: switch cookies to simpler, jar-less approach * making tests pass * improve cookie filtering logic * Remove unneeded Promise.try * filter what makes it to the extension * properly re-set superdomain cookies on cross-origin cy.visit * allow comma-separated list of e2e tests * sort cookies in order of expiration date, ascending * updating tests, cleanup * update tests * version electron as a devDependency, electron@6.0.1 * cleanup, remove old automation * cleanup, remove old automation * bump chokidar to fix win10 + node12 issue was seeing this on windows: nuxt/nuxt#6035 fixed with version bump * enable now-supported quit role, re-enable old tests * don't need that arg there * remove last deprecated callback electron invocations * Delete cypress.json * responding to PR feedback * cleanup * invoke * use 'quit' role * Use new appMenu role for Cypress menu on mac * electron@6.0.2 * electron@6.0.3 * remove domain: cookie.domain and see what happens * remove setErrorHandler * Revert "remove domain: cookie.domain and see what happens" This reverts commit 49e9168. * add unit tests for cookies * ci * fix project-content css * electron@6.0.4 * fix specs_list test * electron@6.0.7 * some cleanup * electron@6.0.9 * Update 8_reporters_spec.coffee.js * electron@5.0.10 - Chromium 73, Node 12 * cli: fix the STDIN pipe on Windows (#5045) * cli: pipe stdin * uggh, here is the actual change * update cli unit tests * add unit test * more permissive check for json to include application/vnd.api+j… (#5166) * more permissive check for json to include * add json test for content-type application/vnd.api+json * cruder solution passes e2e tests locally, so let's go with that * Remove 'charset' from content-type before checking if JSON * fix eslint for fixture specs (#5176) * update eslint to lint files within 'fixtures' in support files - ignore some edge cases like jquery, jsx and obvious js files we wrote with broken code * Fixes from eslint to 'fixtures' files * Catch env variable with reserved name CYPRESS_ENV 1621 (#1626) * server: check CYPRESS_ENV variable when merging configs * catch invalid CYPRESS_ENV value in CLI, close #1621 * linting * sanitize platform in test snapshot * linting * update error message text * add missing comma * fix finally merge in JS code * pass CLI linter * fix log reference, should be debug * use correct sinon reference * update message, show first part in red * update error message text * Addresses #2953 (#5174) * Addresses #2953 * Added proper test for new error message * Didn't realize it ran this test as well, whoops * Implementing changes as suggested by @jennifer-shehane * Fixing tests and error output. Moved the checks to the start of the get command to ensure we always catch improper options * Removing issue test since the querying spec covers it * Using coffescript isArray check * depromisify things that were promisified b/t electron 5 <=> 6 Revert "update shell.openExternal to promisified" This reverts commit 8b6460d. Revert "update dialog.showOpenDialog to promisified" This reverts commit 5f178b0. Revert "update webContents.session.setProxy to promisified" This reverts commit 727df3a. * node12.4.0-chrome76 => node12.0.0-chrome75 * fix tests for electron downgrade * node12.0.0-chrome75 => node12.0.0-chrome73 Co-authored-by: Zach Bloomquist <github@chary.us> Co-authored-by: Brian Mann <brian.mann86@gmail.com>
The code for this is done in cypress-io/cypress#4720, but has yet to be released. |
* fix specs * use debugger protocol for cookie handling in electron * use latest gulp * use rimraf instead of gulp-clean * use electron 3.1.8 and node 10.2.1 * use gulp 4 in packages/static * fix sendCommandAsync, log Schema.getDomains on CDP connect * autofill e2e test name [skip ci] * electron@5.0.7, see what new failures exist * --no-sandbox for launching Electron * update cookies logic for electron * node 12 * update snapshot for new node * update error message for new node * stub sendCommandAsync * only connect to socket if path has been replaced, fixes cypress-io#4776 * update node-sass to support node 12 * skip wacky socket tests for now * snapshot * fix run_plugins_spec snapshot, don't include stack trace * use --no-sandbox on linux to run as root * allow sendCommandAsync to resolve * use euid for root check * log domains even if undefined * don't worry about ending 1xx responses immediately anymore * use --max-http-header-size, change max size from 8kb to 1mb, fix cypress-io#76 * do not send 502 on failed websocket, just send back ECONNRESET * update websocket spec port to not collide with other test * update outdated expect * Revert "only connect to socket if path has been replaced, fixes cypress-io#4776" This reverts commit f179eda. * update gulp in root * update https-proxy unit tests * update network spec to properly close server * update reporter spec * update https-proxy-agent to fix node 10.10.0 change discussion: https://github.com/nodejs/node/issues/24474\#issuecomment-511963799 * only pass --max-http-header-size on node >=12 * use own server-destroy implementation that supports secureConnect events * oops * update socket_spec * electron 6.0.0 * console.table introduced in node 10 * change browserify entry to init.js * handle edge case when no response body * console.table added in node 10 * do not exit app when all BrowserWindows are closed * update e2e snapshots * value may not be null * update plugins spec * correct cookie expiry, use browser.getversion for CDP version check * fix snapshotting for require stacks * reorder cookies in spec * warn when depreated electron callback apis are used * only report 1 plugin error per process * cleanup * node 12.4.0, cypress/browsers:node12.4.0-chrome76 docker image * update shell.openExternal to promisified * update dialog.showOpenDialog to promisified * update webContents.session.setProxy to promisified * updating native dependencies since we don't need ancient node ABI support anymore * WIP: switch cookies to simpler, jar-less approach * WIP: switch cookies to simpler, jar-less approach * making tests pass * improve cookie filtering logic * Remove unneeded Promise.try * filter what makes it to the extension * properly re-set superdomain cookies on cross-origin cy.visit * allow comma-separated list of e2e tests * sort cookies in order of expiration date, ascending * updating tests, cleanup * update tests * version electron as a devDependency, electron@6.0.1 * cleanup, remove old automation * cleanup, remove old automation * bump chokidar to fix win10 + node12 issue was seeing this on windows: nuxt/nuxt#6035 fixed with version bump * enable now-supported quit role, re-enable old tests * don't need that arg there * remove last deprecated callback electron invocations * Delete cypress.json * responding to PR feedback * cleanup * invoke * use 'quit' role * Use new appMenu role for Cypress menu on mac * electron@6.0.2 * electron@6.0.3 * remove domain: cookie.domain and see what happens * remove setErrorHandler * Revert "remove domain: cookie.domain and see what happens" This reverts commit 49e9168. * add unit tests for cookies * ci * fix project-content css * electron@6.0.4 * fix specs_list test * electron@6.0.7 * some cleanup * electron@6.0.9 * Update 8_reporters_spec.coffee.js * electron@5.0.10 - Chromium 73, Node 12 * cli: fix the STDIN pipe on Windows (cypress-io#5045) * cli: pipe stdin * uggh, here is the actual change * update cli unit tests * add unit test * more permissive check for json to include application/vnd.api+j… (cypress-io#5166) * more permissive check for json to include * add json test for content-type application/vnd.api+json * cruder solution passes e2e tests locally, so let's go with that * Remove 'charset' from content-type before checking if JSON * fix eslint for fixture specs (cypress-io#5176) * update eslint to lint files within 'fixtures' in support files - ignore some edge cases like jquery, jsx and obvious js files we wrote with broken code * Fixes from eslint to 'fixtures' files * Catch env variable with reserved name CYPRESS_ENV 1621 (cypress-io#1626) * server: check CYPRESS_ENV variable when merging configs * catch invalid CYPRESS_ENV value in CLI, close cypress-io#1621 * linting * sanitize platform in test snapshot * linting * update error message text * add missing comma * fix finally merge in JS code * pass CLI linter * fix log reference, should be debug * use correct sinon reference * update message, show first part in red * update error message text * Addresses cypress-io#2953 (cypress-io#5174) * Addresses cypress-io#2953 * Added proper test for new error message * Didn't realize it ran this test as well, whoops * Implementing changes as suggested by @jennifer-shehane * Fixing tests and error output. Moved the checks to the start of the get command to ensure we always catch improper options * Removing issue test since the querying spec covers it * Using coffescript isArray check * depromisify things that were promisified b/t electron 5 <=> 6 Revert "update shell.openExternal to promisified" This reverts commit 8b6460d. Revert "update dialog.showOpenDialog to promisified" This reverts commit 5f178b0. Revert "update webContents.session.setProxy to promisified" This reverts commit 727df3a. * node12.4.0-chrome76 => node12.0.0-chrome75 * fix tests for electron downgrade * node12.0.0-chrome75 => node12.0.0-chrome73 Co-authored-by: Zach Bloomquist <github@chary.us> Co-authored-by: Brian Mann <brian.mann86@gmail.com>
The |
Released in |
If I respond to
route
with a very large object, tests waiting for that response will fail withERR_EMPTY_RESPONSE
The text was updated successfully, but these errors were encountered: