Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(conditional-page-build): tracking PR #29140

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 0 additions & 11 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -258,15 +258,6 @@ jobs:
- e2e-test:
test_path: integration-tests/artifacts

# temporary
integration_tests_artifacts_conditional_page_builds:
executor: node
steps:
- e2e-test:
test_path: integration-tests/artifacts
environment:
GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES: 1

integration_tests_ssr:
executor: node
steps:
Expand Down Expand Up @@ -606,8 +597,6 @@ workflows:
<<: *e2e-test-workflow
- integration_tests_artifacts:
<<: *e2e-test-workflow
- integration_tests_artifacts_conditional_page_builds:
<<: *e2e-test-workflow
- integration_tests_ssr:
<<: *e2e-test-workflow
- integration_tests_images:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -304,7 +304,7 @@ Page queries that were queued up earlier from query extraction are run so the da

With everything ready for the HTML pages in place, HTML is compiled and written out to files so it can be served up statically. Since HTML is being produced in a Node.js server context, [references to browser APIs like `window` can break the build](/docs/debugging-html-builds/) and must be conditionally applied.

By default, Gatsby rebuilds static HTML for all pages on each build. There is an experimental feature flag `GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES` which enables [conditional page builds](/docs/conditional-page-builds/).
Gatsby will smartly rebuild only needed HTML files. This might mean 0 html files being generated if nothing that was used for html files changed, some part of pages in case if data that is used changed or all files in case of code change.

## What do you get from a successful build?

Expand Down
55 changes: 0 additions & 55 deletions docs/docs/conditional-page-builds.md

This file was deleted.

12 changes: 5 additions & 7 deletions integration-tests/artifacts/__tests__/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -463,13 +463,11 @@ describe(`Second run (different pages created, data changed)`, () => {
})
})

if (process.env.GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES) {
it(`should recreate only some html files`, () => {
expect(manifest[runNumber].generated.sort()).toEqual(
expectedPagesToBeGenerated.sort()
)
})
}
it(`should recreate only some html files`, () => {
expect(manifest[runNumber].generated.sort()).toEqual(
expectedPagesToBeGenerated.sort()
)
})
})

describe(`page-data files`, () => {
Expand Down
10 changes: 6 additions & 4 deletions packages/gatsby-cli/src/create-cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -243,18 +243,20 @@ function buildLocalCommands(cli: yargs.Argv, isLocalSite: boolean): void {
type: `string`,
describe: `Tracer configuration file (OpenTracing compatible). See https://gatsby.dev/tracing`,
})
// log-pages and write-to-file are specific to experimental GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES feature
// because of that they are hidden from `--help` but still defined so `yargs` know about them
// log-pages and write-to-file were added specifically to experimental GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES feature
// in gatsby@2. They are useful, but not very applicable (specifically `--write-to-file`) as generic approach, as it only
// list pages without other artifacts, so it's useful in very narrow scope. Because we don't have alternative right now
// those toggles are kept for users that rely on them, but we won't promote them and will keep them "hidden".
.option(`log-pages`, {
type: `boolean`,
default: false,
describe: `Log the pages that changes since last build (only available when using GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES).`,
describe: `Log the pages that changes since last build.`,
hidden: true,
})
.option(`write-to-file`, {
type: `boolean`,
default: false,
describe: `Save the log of changed pages for future comparison (only available when using GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES).`,
describe: `Save the log of changed pages for future comparison.`,
hidden: true,
}),
handler: handlerP(
Expand Down
10 changes: 3 additions & 7 deletions packages/gatsby/src/commands/build-html.ts
Original file line number Diff line number Diff line change
Expand Up @@ -271,13 +271,9 @@ export async function buildHTMLPagesAndDeleteStaleArtifacts({
}> {
buildUtils.markHtmlDirtyIfResultOfUsedStaticQueryChanged()

const { toRegenerate, toDelete } = process.env
.GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES
? buildUtils.calcDirtyHtmlFiles(store.getState())
: {
toRegenerate: [...store.getState().pages.keys()],
toDelete: [],
}
const { toRegenerate, toDelete } = buildUtils.calcDirtyHtmlFiles(
store.getState()
)

if (toRegenerate.length > 0) {
const buildHTMLActivityProgress = reporter.createProgress(
Expand Down
10 changes: 2 additions & 8 deletions packages/gatsby/src/commands/build.ts
Original file line number Diff line number Diff line change
Expand Up @@ -233,10 +233,7 @@ module.exports = async function build(program: IBuildArgs): Promise<void> {
workerPool.end()
buildActivity.end()

if (
process.env.GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES &&
process.argv.includes(`--log-pages`)
) {
if (process.argv.includes(`--log-pages`)) {
if (toRegenerate.length) {
report.info(
`Built pages:\n${toRegenerate
Expand All @@ -254,10 +251,7 @@ module.exports = async function build(program: IBuildArgs): Promise<void> {
}
}

if (
process.env.GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES &&
process.argv.includes(`--write-to-file`)
) {
if (process.argv.includes(`--write-to-file`)) {
const createdFilesPath = path.resolve(
`${program.directory}/.cache`,
`newPages.txt`
Expand Down
14 changes: 0 additions & 14 deletions packages/gatsby/src/redux/actions/public.js
Original file line number Diff line number Diff line change
Expand Up @@ -1310,20 +1310,6 @@ actions.createPageDependency = (
}
}

/**
* Set page data in the store, saving the pages content data and context.
*
* @param {Object} $0
* @param {string} $0.id the path to the page.
* @param {string} $0.resultHash pages content hash.
*/
actions.setPageData = (pageData: PageData) => {
return {
type: `SET_PAGE_DATA`,
payload: pageData,
}
}

/**
* Record that a page was visited on the server..
*
Expand Down
9 changes: 0 additions & 9 deletions packages/gatsby/src/redux/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -344,7 +344,6 @@ export type ActionsUnion =
| ISetWebpackConfigAction
| ITouchNodeAction
| IUpdatePluginsHashAction
| ISetPageDataAction
| ICreateJobV2Action
| IEndJobV2Action
| IRemoveStaleJobV2Action
Expand Down Expand Up @@ -640,14 +639,6 @@ export interface IDeleteCacheAction {
cacheIsCorrupt?: boolean
}

export interface ISetPageDataAction {
type: `SET_PAGE_DATA`
payload: {
id: Identifier
resultHash: string
}
}

export interface IRemoveTemplateComponentAction {
type: `REMOVE_STATIC_QUERIES_BY_TEMPLATE`
payload: {
Expand Down
10 changes: 4 additions & 6 deletions packages/gatsby/src/services/initialize.ts
Original file line number Diff line number Diff line change
Expand Up @@ -297,14 +297,12 @@ export async function initialize({
const cacheJsonDirExists = fs.existsSync(`${cacheDirectory}/json`)
const publicDirExists = fs.existsSync(publicDirectory)

// During builds, delete html and css files from the public directory as we don't want
// deleted pages and styles from previous builds to stick around.
// For Conditional Page Builds, we do want to remove those when there is `public` dir
// but cache doesn't exist
// For builds in case public dir exists, but cache doesn't, we need to clean up potentially stale
// artifacts from previous builds (due to cache not being available, we can't rely on tracking of artifacts)
if (
process.env.NODE_ENV === `production` &&
(!process.env.GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES ||
(publicDirExists && !cacheJsonDirExists))
publicDirExists &&
!cacheJsonDirExists
) {
activity = reporter.activityTimer(
`delete html and css files from previous builds`,
Expand Down