-
Notifications
You must be signed in to change notification settings - Fork 10.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
perf(gatsby-source-contentful): dont re-create nodes #28642
Conversation
This makes code harder to reason with. The real issue is inside gatsby-node where we merge existing nodes together with new nodes. That part needs to refactor so I'm inclined to close that PR for that reason |
I was missing a lot of context on the why here. I'm fine with this patch as it does improve performance. If we could run some tests to make sure references and all are still being updated 👍 |
Heja there 👋 got hinted to this PR by @axe312ger, I'm one of Contentful's backend engineers and want to increase your confidence into the solution. 🙂 For a single given Contentful entity, a write towards it will update its That means using the Phrased differently:
Hope this helps! |
Rerunning CI |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
A warm build will currently still attempt to create the node even if it already exists. This PR will do an early check, compare the digest (
entry.sys.updatedAt
), and return early if the node already exists for that digest.There might be a simpler solution but for now, this cuts the warm "no changes" build for a 2 million node site down from to about 894s (120s for node createion step) down to 512s (35s for the node creation step), saving about 5 minutes.
Need verification from Contentful that this approach is sound. But I think it is. (Tests will fail too, will fix them before we merge)