feat(core): Docusaurus Faster - SSG worker threads #10826
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
slorber
added
pr: new feature
✅ [V2]
To edit notification comments on pull requests, go to your Netlify site configuration. |
|
Size Change: +1.13 kB (+0.01%) Total Size: 11.8 MB
ℹ️ View Unchanged
|
slorber
added
the
Argos
⚡️ Lighthouse report for the deploy preview of this PR
|
|
Size Change: +1.14 kB (+0.01%) Total Size: 12.3 MB
ℹ️ View Unchanged
|
|
The latest updates on your projects. Learn more about Argos notifications ↗︎
|
|
👍 Dependency issues cleared. Learn more about Socket for GitHub ↗︎ This PR previously contained dependency changes with security issues that have been resolved, removed, or ignored. |
Co-authored-by: Sébastien Lorber <slorber@users.noreply.github.com>
…10847) Fix `<DocCardList>` usage on docs at root of a sidebar
…ializable-data' into slorber/poc-remove-helmet-nonserializable-data
slorber
changed the title
|
Report too large to display inline |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
This PR introduces a new Docusaurus Faster option:
siteConfig.future.experimental_faster.ssgWorkerThreads.It permits you to optimize the static site generation phase of your site, by executing SSG tasks in a worker thread pool.
This enables you to distribute load across your computer CPUs more evenly, and can greatly decrease SSG time for larger sites. It is particularly impactful on large sites, and sites using features that produce many pages (tags, blog pagination, blog authors pages).
To use this option, the v4 future flag
siteConfig.future.v4.removeLegacyPostBuildHeadAttributemust be turned on.Note: we recommend you try to turn all v4 future flags on globally:
future.v4: true. This opt-in for upcoming v4 breaking changes and prepare your site for it.If possible, use the following config:
Benchmark
Using a thread pool to distribute the SSG load over multiple workers can significantly improve the SSG execution time. According to my local tests (Mac M3 Pro), SSG can be divided by ~2 and probably more if there are many pathnames to render.
On our site, SSG goes from 12 seconds to 6 seconds (executing
yarn build:website --locale en), which reduces the total build time by ~6 seconds using the default thread pool settings:This build benchmark only reports a 15% improvement because SSG is only a part of the full Docusaurus build. The SSG time went from 12s to 6s locally so it remains significant for that particular phase (50% improvement). For sites with thousands of pages, the impact can be even more significant. Once we activate Rspack persistent caching (faster rebuilds), saving 50% of SSG time will be much more important to have super fast site rebuilds.
Fine-tuning
Providing sensible defaults for the worker thread pool config is difficult. We'll improve the current defaults according to early adopter's feedback.
There are env variables that you can use to adjust settings:
Note: these variables are implementation details. We may refactor/rename them in minor versions. Eventually, we'll expose a public API for fine-tuning thread pool configuration.
Using
DOCUSAURUS_SSG_WORKER_THREAD_COUNT=1permits to not use worker threads: SSG will be executed in the current main thread (historical SSG behavior of Docusaurus).We recommend you benchmark with various settings, find out what works best for you, and report your results to the community here. Note that settings that work better for you locally might not be the best when running on your CI / CDN build.
Example benchmark you can try locally using hyperfine:
Test Plan
CI + dogfood + Argos
Test links
https://deploy-preview-10826--docusaurus-2.netlify.app/
Related issues/PRs
Follow up of:
postBuild({routesBuildMetadata})API, deprecateheadattribute + v4 future flag #10850Also the expected next step for Docusaurus Faster: #10556