Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(conditional-page-builds): make it default #29548

Merged
merged 3 commits into from
Feb 18, 2021
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 0 additions & 11 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -258,15 +258,6 @@ jobs:
- e2e-test:
test_path: integration-tests/artifacts

# temporary
integration_tests_artifacts_conditional_page_builds:
executor: node
steps:
- e2e-test:
test_path: integration-tests/artifacts
environment:
GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES: 1

integration_tests_ssr:
executor: node
steps:
Expand Down Expand Up @@ -606,8 +597,6 @@ workflows:
<<: *e2e-test-workflow
- integration_tests_artifacts:
<<: *e2e-test-workflow
- integration_tests_artifacts_conditional_page_builds:
<<: *e2e-test-workflow
- integration_tests_ssr:
<<: *e2e-test-workflow
- integration_tests_images:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -304,7 +304,7 @@ Page queries that were queued up earlier from query extraction are run so the da

With everything ready for the HTML pages in place, HTML is compiled and written out to files so it can be served up statically. Since HTML is being produced in a Node.js server context, [references to browser APIs like `window` can break the build](/docs/debugging-html-builds/) and must be conditionally applied.

By default, Gatsby rebuilds static HTML for all pages on each build. There is an experimental feature flag `GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES` which enables [conditional page builds](/docs/conditional-page-builds/).
Gatsby will smartly rebuild only needed HTML files. This can result in no HTML files being generated if nothing used for HTML files changed. The opposite can also be true and lead to a full site rebuild when data is used on all pages or when a code change happens.

## What do you get from a successful build?

Expand Down
55 changes: 0 additions & 55 deletions docs/docs/conditional-page-builds.md

This file was deleted.

12 changes: 5 additions & 7 deletions integration-tests/artifacts/__tests__/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -463,13 +463,11 @@ describe(`Second run (different pages created, data changed)`, () => {
})
})

if (process.env.GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES) {
it(`should recreate only some html files`, () => {
expect(manifest[runNumber].generated.sort()).toEqual(
expectedPagesToBeGenerated.sort()
)
})
}
it(`should recreate only some html files`, () => {
expect(manifest[runNumber].generated.sort()).toEqual(
expectedPagesToBeGenerated.sort()
)
})
})

describe(`page-data files`, () => {
Expand Down
10 changes: 6 additions & 4 deletions packages/gatsby-cli/src/create-cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -243,18 +243,20 @@ function buildLocalCommands(cli: yargs.Argv, isLocalSite: boolean): void {
type: `string`,
describe: `Tracer configuration file (OpenTracing compatible). See https://gatsby.dev/tracing`,
})
// log-pages and write-to-file are specific to experimental GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES feature
// because of that they are hidden from `--help` but still defined so `yargs` know about them
// log-pages and write-to-file were added specifically to experimental GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES feature
// in gatsby@2. They are useful, but not very applicable (specifically `--write-to-file`) as generic approach, as it only
// list pages without other artifacts, so it's useful in very narrow scope. Because we don't have alternative right now
// those toggles are kept for users that rely on them, but we won't promote them and will keep them "hidden".
.option(`log-pages`, {
type: `boolean`,
default: false,
describe: `Log the pages that changes since last build (only available when using GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES).`,
describe: `Log the pages that changes since last build.`,
hidden: true,
})
.option(`write-to-file`, {
type: `boolean`,
default: false,
describe: `Save the log of changed pages for future comparison (only available when using GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES).`,
describe: `Save the log of changed pages for future comparison.`,
hidden: true,
}),
handler: handlerP(
Expand Down
10 changes: 3 additions & 7 deletions packages/gatsby/src/commands/build-html.ts
Original file line number Diff line number Diff line change
Expand Up @@ -271,13 +271,9 @@ export async function buildHTMLPagesAndDeleteStaleArtifacts({
}> {
buildUtils.markHtmlDirtyIfResultOfUsedStaticQueryChanged()

const { toRegenerate, toDelete } = process.env
.GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES
? buildUtils.calcDirtyHtmlFiles(store.getState())
: {
toRegenerate: [...store.getState().pages.keys()],
toDelete: [],
}
const { toRegenerate, toDelete } = buildUtils.calcDirtyHtmlFiles(
store.getState()
)

if (toRegenerate.length > 0) {
const buildHTMLActivityProgress = reporter.createProgress(
Expand Down
10 changes: 2 additions & 8 deletions packages/gatsby/src/commands/build.ts
Original file line number Diff line number Diff line change
Expand Up @@ -233,10 +233,7 @@ module.exports = async function build(program: IBuildArgs): Promise<void> {
workerPool.end()
buildActivity.end()

if (
process.env.GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES &&
process.argv.includes(`--log-pages`)
) {
if (process.argv.includes(`--log-pages`)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't we move this to the cli? 😬
Can you create a follow up ticket?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, we probably already get things on program for it because we do let yargs know about them (just hide them from --help) -

// log-pages and write-to-file were added specifically to experimental GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES feature
// in gatsby@2. They are useful, but not very applicable (specifically `--write-to-file`) as generic approach, as it only
// list pages without other artifacts, so it's useful in very narrow scope. Because we don't have alternative right now
// those toggles are kept for users that rely on them, but we won't promote them and will keep them "hidden".
.option(`log-pages`, {
type: `boolean`,
default: false,
describe: `Log the pages that changes since last build.`,
hidden: true,
})
.option(`write-to-file`, {
type: `boolean`,
default: false,
describe: `Save the log of changed pages for future comparison.`,
hidden: true,
}),

I'll create follow up ticket on that

if (toRegenerate.length) {
report.info(
`Built pages:\n${toRegenerate
Expand All @@ -254,10 +251,7 @@ module.exports = async function build(program: IBuildArgs): Promise<void> {
}
}

if (
process.env.GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES &&
process.argv.includes(`--write-to-file`)
) {
if (process.argv.includes(`--write-to-file`)) {
const createdFilesPath = path.resolve(
`${program.directory}/.cache`,
`newPages.txt`
Expand Down
10 changes: 4 additions & 6 deletions packages/gatsby/src/services/initialize.ts
Original file line number Diff line number Diff line change
Expand Up @@ -297,14 +297,12 @@ export async function initialize({
const cacheJsonDirExists = fs.existsSync(`${cacheDirectory}/json`)
const publicDirExists = fs.existsSync(publicDirectory)

// During builds, delete html and css files from the public directory as we don't want
// deleted pages and styles from previous builds to stick around.
// For Conditional Page Builds, we do want to remove those when there is `public` dir
// but cache doesn't exist
// For builds in case public dir exists, but cache doesn't, we need to clean up potentially stale
// artifacts from previous builds (due to cache not being available, we can't rely on tracking of artifacts)
if (
process.env.NODE_ENV === `production` &&
(!process.env.GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES ||
(publicDirExists && !cacheJsonDirExists))
publicDirExists &&
!cacheJsonDirExists
) {
activity = reporter.activityTimer(
`delete html and css files from previous builds`,
Expand Down