This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
Build is not incremental #30202
Comments
gatsbot
bot
added
the
status: triage needed
|
I also have this issue. Although there was no page-change, all pages get rebuild. This was working in Gatsby v2 behind the experimental conditional build flag. A log output: |
|
@feedm3 thanks I'm somewhat happy to see it's just me :) |
|
Unfortunately it's on a private company repositry which I'm not allowed to share :-/ |
This is expected, changing jsx result in changed webpack compilation. Some assets generetad by webpack WILL have changed hashes in filenames so we do have to regenerate pages so
Right but still no .html files were built:
So the feature work. The biggest time consuming step is webpack compilation which we will look to be more cachable and take less time |
|
I'll see if I can bake in some logging on WHY pages are marked to rebuild - specifically for cases when all pages are marked to rebuild |
pieh
added
topic: build
|
Just a note: If I run the build with |
|
@pieh Did this work differently with the |
|
same issue exist at Hello world gatsby starter example , simply add a new page under Pages folder then change text in 1 page only and build the project u will see that 2 pages are updated |
There were some changes and handling of cases that weren't handled before (for example changes to static queries). Here's rough outline on what was changed since v2 - #29140 |
|
For folks reporting that all pages always get rebuilt - can you try installing Example output I'm looking for will look something like this: |
this is the result for my project with 10500 page , |
|
@ahmadkhalaf1 is this from first build or subsequent build? the |
Sadly this is from subsequent builds :( and its a private repo for my company. But I tried incremental build with gatsby fresh hello world project, and the result is the same , maybe u can test the nre version there . |
vladar
added
the
status: needs reproduction
|
I did a bunch of incremental builds with
Edit 1: I could resolve Edit 2: I finally fixed it! The fix is quite special for our project, but maybe it helps others to spot it in their project. It was a "trial and error" approach: deleting all code in the
Like in the static query, we also used the date in a shared Gatsby context object we put into every I just removed it, as it was not necessary anymore in this project. The information is also redundant as we already have it with the build date plugin. The second problem was even more unique: Due to some infrastructure constrains we need to copy all assets into a separate subfolder. We are using gatsby-plugin-asset-path for this. The The problem there was the Conclusion Gatsby was working correctly all the time. The problems we had were valid reasons to rebuild all pages. The issue is that it's very complicated to track down the problem. The log messages was super helpful to spot the issues, I would propose that the flag-log-message makes it into the normal Gatsby package! It would also be helpful to make the error messages more detailed - not sure if this is technically working (like "there was a change in your |
|
It looks like there is one small bug: After Gatsby went through all It rebuilds only 1 changed page (the one that was changed, so this is correct) but it says that 1823 pages were changed. So the log message is not correct here. |
|
@feedm3 thanks for super detailed description of what was causing to not hit "happy path" for inc builds. I do wonder now what kind of diagnostic information we could actually display to users to help with tracking root causes down because
Agree, it's confusing (especially because that part of logs actually doesn't relate at all which html files get rebuild :( ) |
|
Ok, maybe in absence of better diagnostic tools so far I could propose following flow to make it easier to pin point what is actually changing and where to start looking for things in site (or used plugins): What this should do is to actually print which files DID change in what way - this will be a looooong output for larger sites so maybe consider piping it to a file The |
so i have tried your steps , and now i have the build-diff.txt file for my 10500 page :) and i understand nothing from it , its mixed up and super hard to know anything inside , any hint what should i look for or how can i read it ? |
So generating diff is first step - just to talk concretely - here's example diff I created replicating some of what feedm3 described in their comments - https://gist.github.com/pieh/f0e10a2b8a4eb33177c85cf0a80fb0e3 - I used DISCLAIMER: this is just example - it will be unlikely that it can apply to every situation - this is more showing general steps of tracking down root cause The few things you can get from it are: Genareted JS bundle did changeYou can tell by seeing that both exports.onCreateWebpackConfig = ({ actions, plugins }) => {
const dateString = new Date().toString()
actions.setWebpackConfig({
plugins: [
plugins.define({
"process.env.DEFINED_DATE": JSON.stringify(dateString),
}),
],
})
}Used data has changedOther things in the diff is we see changes in: Page query is quite straight forward to figure out because it points to actual page and you can see what data changed. Static query is a bit more messy - you can see what content changed but not quite easily to tell which static query it is - we don't expose much information about it, so probably fastest way is just to check static queries you have in your site and just match the result from .json file to static query in your code by comparing what data is there and what you query in those. So result in public is something like: {"data":{"site":{"buildTime":"2021-03-18 10:47 am UTC"}}}And I track it down to static query in one of my components that is used in all pages: const data = useStaticQuery(graphql`
{
site {
buildTime(formatString: "YYYY-MM-DD hh:mm a z")
}
}
`)Because it's used on all pages, when result of it changes, it will "invalidate" all html files (of course there can be situation when static query is used on subset of all pages too, but point is that changing it can invalidate 1 or more pages) Note - you can see additional changes in the diff like |
|
Since we added a doc page about these pitfalls and there's nothing to "fix" here I'll move this to discussions :) |
This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
Description
My Gatsby 3.0 site is being built on every push with a Github action. I'm seeing that many pages are being re-built even though I only made a change in a single page.
Steps to reproduce
This is a bit tricky for now because I can reproduce in a private repo.
But the steps are simple, my site has a variety of markdown based pages as well as plain old JSX pages.
verbose Found 102 changed pages. Still slow.Here are the logs to show it happening:
Expected result
Fast build to update only /faq
Actual result
Slow build and everything is updated.
Environment
The text was updated successfully, but these errors were encountered: