Unhandled Rejection - 96,000 Page Build #12021
Labels
stale?
Issue that may be closed soon due to the original author not responding any more.
status: awaiting author response
Additional information has been requested from the author
status: confirmed
Issue with steps to reproduce the bug that’s been verified by at least one reviewer.
type: bug
An issue or pull request relating to a bug in Gatsby
Comments
youmustfight
changed the title
|
Thank you for the report! We can't do the build with the linked repro, cause it uses some private packages. I wonder if it's possible to remove them, that would really help us. Thanks a lot! |
freiksenet
added
type: bug
|
Hey @freiksenet! I just removed the private dependencies and environment requirements so you should be able to build it now. Big update though... found something interesting. For times sake I decide to prune out StaticQuery components and page level queries for things I was grabbing from Contentful. This reduced the graphql query count from around ~96,000 to 83,000. But most interesting off all, the build was successful!!! 🤔 ??? This is either because a) I eliminated some other content types that can lead to a few thousand pages and we fell to an acceptable page limit or b) some thing I deleted in the city templates meant some other limit did not get tripped. 1Removed 2Removed 3Removed the StaticQuery(via Contentful) for a "Read User Stories" footer component that sits on every city page and their various children pages. 4Don't think this matters but just mentioning incase. There is a "Read Bankruptcy Articles" component with another StaticQuery for Contentful but this component is hidden because a property of the footer is false. 5For the sake of getting things working quickly, I pruned some loops and lookups. Wondering if some of those were expensive operations that maybe were causing timeouts or just realllly long processing times? I'm going to try adding things back in and see if I can get it to break again. Will keep you posted! UPDATE: I tried removing that component's static query in point 3 and the production build we have still broke :( so that might not be as promising a path as I hoped |
|
Thanks a lot for the repro! I managed to build it on my thinkpad (though I do have 16GB of RAM). Seems it might be dependent on that. The big issue atm is that we can't see the actual error because it's swallowed by the console reporter. Could you try running it again with console.log in the "error" function in reporter/index.js? (You can for example edit it directly in node_modules). I'll ask my colleagues if they can get the build to fail too. |
|
Not sure if this is helpful but some input/outputs for reporter and create-cli Create-CLI Reporter's Reporter's error => Create CLI handleP's Promise error handler => Oh this just got spit out from reporter's panic function |
|
Hiya! This issue has gone quiet. Spooky quiet. 👻 We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here. If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open! Thanks for being a part of the Gatsby community! 💪💜 |
gatsbot
bot
added
the
stale?
|
Hey again! It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it. Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m Thanks again for being part of the Gatsby community! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description
Our nonprofit is creating programmatic content for a bunch of different localities and legal needs in the U.S. We've been enjoying Gatsby up to the 12,000 page range, but we recently wanted to expand our legal templates and go zipcode based. This means we've gone from 12,000 pages to 96,000 pages. This has forced me to do builds on a higher performance EC2 instance rather than my Macbook Air.
The build breaks however, right after bootstrap finishes and before I see a consoled message about building JS/CSS bundles. Here's the error message:
NPM debug-log...
One key note is that I can successfully build the site if I only do 4,000 cities at a time (3 pages per city). Only when I try building the full set does it break (27,000 cities with 3 pages per city). Two of the three templates are quite large in word count and elements.
Steps to reproduce
You can except a problem to arise when you just type:
npm run build->node --max_old_space_size=16000 --stack-size=1000000 ./node_modules/.bin/gatsby build --verboseExpected result
Expected a successful build of all 96,000 pages 🎉
Actual result
Seems to break when I'm around the ~20,000 pages mark
Environment
Related?
Maybe this is related to #4746 ?
The text was updated successfully, but these errors were encountered: