perf: disable streaming in SSG #9603

ematipico · 2024-01-04T09:02:04Z

Changes

Disable streaming in case we do an SSG build.

Internally, we noticed disabling streaming in certain projects yields to better build times.

Testing

Manual testing and launch the benchmark job

Docs

N/A

changeset-bot · 2024-01-04T09:02:09Z

⚠️ No Changeset found

Latest commit: a79a834

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

ematipico · 2024-01-04T09:02:21Z

!preview ssg-no-streaming

ematipico · 2024-01-04T09:03:02Z

!bench

ematipico · 2024-01-04T09:03:44Z

!bench memory

github-actions · 2024-01-04T09:04:14Z

Snapshots have been released for the following packages:

astro@experimental--ssg-no-streaming
create-astro@experimental--ssg-no-streaming

Publish Log

🦋  warn ===============================IMPORTANT!===============================
🦋  warn Packages will be released under the experimental--ssg-no-streaming tag
🦋  warn ----------------------------------------------------------------------
🦋  info npm info astro
🦋  info npm info @astrojs/prism
🦋  info npm info @astrojs/rss
🦋  info npm info create-astro
🦋  info npm info @astrojs/alpinejs
🦋  info npm info @astrojs/lit
🦋  info npm info @astrojs/markdoc
🦋  info npm info @astrojs/mdx
🦋  info npm info @astrojs/node
🦋  info npm info @astrojs/partytown
🦋  info npm info @astrojs/preact
🦋  info npm info @astrojs/react
🦋  info npm info @astrojs/sitemap
🦋  info npm info @astrojs/solid-js
🦋  info npm info @astrojs/svelte
🦋  info npm info @astrojs/tailwind
🦋  info npm info @astrojs/vercel
🦋  info npm info @astrojs/vue
🦋  info npm info @astrojs/internal-helpers
🦋  info npm info @astrojs/markdown-remark
🦋  info npm info @astrojs/telemetry
🦋  info npm info @astrojs/underscore-redirects
🦋  info npm info @astrojs/upgrade
🦋  info astro is being published because our local version (0.0.0-ssg-no-streaming-20240104090316) has not been published on npm
🦋  warn @astrojs/prism is not being published because version 3.0.0 is already published on npm
🦋  warn @astrojs/rss is not being published because version 4.0.1 is already published on npm
🦋  info create-astro is being published because our local version (0.0.0-ssg-no-streaming-20240104090316) has not been published on npm
🦋  warn @astrojs/alpinejs is not being published because version 0.3.2 is already published on npm
🦋  warn @astrojs/lit is not being published because version 4.0.1 is already published on npm
🦋  warn @astrojs/markdoc is not being published because version 0.8.2 is already published on npm
🦋  warn @astrojs/mdx is not being published because version 2.0.3 is already published on npm
🦋  warn @astrojs/node is not being published because version 7.0.4 is already published on npm
🦋  warn @astrojs/partytown is not being published because version 2.0.3 is already published on npm
🦋  warn @astrojs/preact is not being published because version 3.0.2 is already published on npm
🦋  warn @astrojs/react is not being published because version 3.0.9 is already published on npm
🦋  warn @astrojs/sitemap is not being published because version 3.0.4 is already published on npm
🦋  warn @astrojs/solid-js is not being published because version 3.0.3 is already published on npm
🦋  warn @astrojs/svelte is not being published because version 5.0.3 is already published on npm
🦋  warn @astrojs/tailwind is not being published because version 5.1.0 is already published on npm
🦋  warn @astrojs/vercel is not being published because version 6.1.2 is already published on npm
🦋  warn @astrojs/vue is not being published because version 4.0.7 is already published on npm
🦋  warn @astrojs/internal-helpers is not being published because version 0.2.1 is already published on npm
🦋  warn @astrojs/markdown-remark is not being published because version 4.0.1 is already published on npm
🦋  warn @astrojs/telemetry is not being published because version 3.0.4 is already published on npm
🦋  warn @astrojs/underscore-redirects is not being published because version 0.3.3 is already published on npm
🦋  warn @astrojs/upgrade is not being published because version 0.2.2 is already published on npm
🦋  info Publishing "astro" at "0.0.0-ssg-no-streaming-20240104090316"
🦋  info Publishing "create-astro" at "0.0.0-ssg-no-streaming-20240104090316"
🦋  success packages published successfully:
🦋  astro@0.0.0-ssg-no-streaming-20240104090316
🦋  create-astro@0.0.0-ssg-no-streaming-20240104090316
🦋  Creating git tags...
🦋  New tag:  astro@0.0.0-ssg-no-streaming-20240104090316
🦋  New tag:  create-astro@0.0.0-ssg-no-streaming-20240104090316

Build Log


> root@0.0.0 build /home/runner/work/astro/astro
> turbo run build --filter=astro --filter=create-astro --filter="@astrojs/*" --filter="@benchmark/*"

• Packages in scope: @astrojs/alpinejs, @astrojs/cloudflare, @astrojs/internal-helpers, @astrojs/lit, @astrojs/markdoc, @astrojs/markdown-remark, @astrojs/mdx, @astrojs/netlify, @astrojs/node, @astrojs/partytown, @astrojs/preact, @astrojs/prism, @astrojs/react, @astrojs/rss, @astrojs/sitemap, @astrojs/solid-js, @astrojs/svelte, @astrojs/tailwind, @astrojs/telemetry, @astrojs/underscore-redirects, @astrojs/upgrade, @astrojs/vercel, @astrojs/vue, @benchmark/timer, astro, create-astro
• Running build in 26 packages
• Remote caching enabled
::group::@astrojs/prism:build
cache hit, suppressing logs 3aa7f82954d2c6a4
::endgroup::
::group::@astrojs/internal-helpers:build
cache hit, suppressing logs dcd2e47ddeb4319c
::endgroup::
::group::@astrojs/upgrade:build
cache hit, suppressing logs 87f52b7a0803dd23
::endgroup::
::group::@astrojs/telemetry:build
cache hit, suppressing logs c98d31a4b10de42a
::endgroup::
::group::@astrojs/markdown-remark:build
cache hit, suppressing logs c08381de7599ea31
::endgroup::
::group::create-astro:build
cache miss, executing b5c458e0e426ddbe

> create-astro@0.0.0-ssg-no-streaming-20240104090316 build /home/runner/work/astro/astro/packages/create-astro
> astro-scripts build "src/index.ts" --bundle && tsc

::endgroup::
::group::astro:build
cache miss, executing c55649dc728ed755

> astro@0.0.0-ssg-no-streaming-20240104090316 build /home/runner/work/astro/astro/packages/astro
> pnpm run prebuild && astro-scripts build "src/**/*.{ts,js}" && tsc && pnpm run postbuild


> astro@0.0.0-ssg-no-streaming-20240104090316 prebuild /home/runner/work/astro/astro/packages/astro
> astro-scripts prebuild --to-string "src/runtime/server/astro-island.ts" "src/runtime/client/{idle,load,media,only,visible}.ts"


> astro@0.0.0-ssg-no-streaming-20240104090316 postbuild /home/runner/work/astro/astro/packages/astro
> astro-scripts copy "src/**/*.astro" && astro-scripts copy "src/**/*.wasm"

::endgroup::
::group::@astrojs/alpinejs:build
cache miss, executing ad350977132d4f5c

> @astrojs/alpinejs@0.3.2 build /home/runner/work/astro/astro/packages/integrations/alpinejs
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@benchmark/timer:build
cache miss, executing 091f99dae361932d

> @benchmark/timer@0.0.0 build /home/runner/work/astro/astro/benchmark/packages/timer
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/partytown:build
cache miss, executing c5ecf8edca791a0f

> @astrojs/partytown@2.0.3 build /home/runner/work/astro/astro/packages/integrations/partytown
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/rss:build
cache miss, executing cbfa1a9fdfdf04ef

> @astrojs/rss@4.0.1 build /home/runner/work/astro/astro/packages/astro-rss
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/solid-js:build
cache miss, executing 1564ef93bd3f7556

> @astrojs/solid-js@3.0.3 build /home/runner/work/astro/astro/packages/integrations/solid
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/node:build
cache miss, executing f147aa76e656a5a2

> @astrojs/node@7.0.4 build /home/runner/work/astro/astro/packages/integrations/node
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/vercel:build
cache miss, executing 9113d8bd9b69ca9a

> @astrojs/vercel@6.1.2 build /home/runner/work/astro/astro/packages/integrations/vercel
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/markdoc:build
cache miss, executing 68a1f339b58e94ad

> @astrojs/markdoc@0.8.2 build /home/runner/work/astro/astro/packages/integrations/markdoc
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/preact:build
cache miss, executing ed122e215a98f84e

> @astrojs/preact@3.0.2 build /home/runner/work/astro/astro/packages/integrations/preact
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/mdx:build
cache miss, executing 9e64b96bb7c88084

> @astrojs/mdx@2.0.3 build /home/runner/work/astro/astro/packages/integrations/mdx
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/tailwind:build
cache miss, executing b5cbe8ae13d2fd3f

> @astrojs/tailwind@5.1.0 build /home/runner/work/astro/astro/packages/integrations/tailwind
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/underscore-redirects:build
cache miss, executing 209098bb5be4f82b

> @astrojs/underscore-redirects@0.3.3 build /home/runner/work/astro/astro/packages/underscore-redirects
> astro-scripts build "src/**/*.ts" && tsc -p tsconfig.json

::endgroup::
::group::@astrojs/react:build
cache miss, executing 9f93300bb1d64af2

> @astrojs/react@3.0.9 build /home/runner/work/astro/astro/packages/integrations/react
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/lit:build
cache miss, executing e71c6477b7fd6a9b

> @astrojs/lit@4.0.1 build /home/runner/work/astro/astro/packages/integrations/lit
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/sitemap:build
cache miss, executing 82126d7f295d8d98

> @astrojs/sitemap@3.0.4 build /home/runner/work/astro/astro/packages/integrations/sitemap
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/vue:build
cache miss, executing 0b4686c972a0a609

> @astrojs/vue@4.0.7 build /home/runner/work/astro/astro/packages/integrations/vue
> astro-scripts build "src/index.ts" && astro-scripts build "src/editor.cts" --force-cjs --no-clean-dist && tsc

::endgroup::
::group::@astrojs/svelte:build
cache miss, executing 76c28bb66571dea3

> @astrojs/svelte@5.0.3 build /home/runner/work/astro/astro/packages/integrations/svelte
> astro-scripts build "src/index.ts" && astro-scripts build "src/editor.cts" --force-cjs --no-clean-dist && tsc

::endgroup::

 Tasks:    24 successful, 24 total
Cached:    5 cached, 24 total
  Time:    38.036s

github-actions · 2024-01-04T09:04:40Z

PR Benchmark

Memory

	Elapsed time (s)	Memory used (MB)	Final memory (MB)
SSR build	2.83	86.50	125.45
Client build	0.08	0.28	125.57
Static generate	0.46	5.46	130.54

Main Benchmark

Memory

	Elapsed time (s)	Memory used (MB)	Final memory (MB)
SSR build	2.97	86.58	125.51
Client build	0.11	0.21	125.63
Static generate	0.46	5.37	130.60

github-actions · 2024-01-04T09:06:27Z

PR Benchmark

Memory

	Elapsed time (s)	Memory used (MB)	Final memory (MB)
SSR build	2.79	86.71	125.47
Client build	0.08	0.29	125.60
Static generate	0.46	5.37	130.57

Render

Page	Avg (ms)	Stdev (ms)	Max (ms)
/astro	4.34	4.10	34.18
/md	0.63	2.27	22.77
/mdx	122.89	19.38	219.34

Server stress

	Avg	Stdev	Max
Latency	3615.23 ms	784.39 ms	6031 ms

	Avg	Stdev	Min	Total in 30s
Req/Sec	260	316.51	46	7.8k requests
Bytes/Sec	46.4 MB	55.7 MB	9.38 MB	1.39 GB read

CLI Startup

Command	Avg (ms)	Stdev (ms)	Max (ms)
astro --help	194.29	2.05	198.27
astro info	623.65	2.71	626.85

Main Benchmark

Memory

	Elapsed time (s)	Memory used (MB)	Final memory (MB)
SSR build	2.88	86.37	125.45
Client build	0.08	0.29	125.57
Static generate	0.46	5.23	130.54

Render

Page	Avg (ms)	Stdev (ms)	Max (ms)
/astro	4.36	4.89	39.87
/md	0.65	2.53	25.63
/mdx	123.75	19.47	212.86

Server stress

	Avg	Stdev	Max
Latency	3598.05 ms	781.67 ms	5991 ms

	Avg	Stdev	Min	Total in 30s
Req/Sec	260	301.44	106	7.8k requests
Bytes/Sec	46.4 MB	53.5 MB	13.3 MB	1.39 GB read

CLI Startup

Command	Avg (ms)	Stdev (ms)	Max (ms)
astro --help	195.16	2.07	197.69
astro info	626.49	4.18	634.86

packages/astro/src/core/build/buildPipeline.ts

Co-authored-by: Chris Swithinbank <swithinbank@gmail.com>

ematipico · 2024-01-04T12:37:49Z

!bench

ematipico · 2024-01-04T12:37:59Z

!preview ssg-no-streaming

ematipico · 2024-01-04T12:38:11Z

!bench memory

github-actions · 2024-01-04T12:39:37Z

PR Benchmark

Memory

	Elapsed time (s)	Memory used (MB)	Final memory (MB)
SSR build	2.83	86.73	125.42
Client build	0.08	0.27	125.51
Static generate	0.46	5.37	130.49

Main Benchmark

Memory

	Elapsed time (s)	Memory used (MB)	Final memory (MB)
SSR build	2.86	86.93	125.52
Client build	0.08	0.24	125.63
Static generate	0.46	5.24	130.60

github-actions · 2024-01-04T12:40:00Z

Snapshots have been released for the following packages:

astro@experimental--ssg-no-streaming
create-astro@experimental--ssg-no-streaming

Publish Log

🦋  warn ===============================IMPORTANT!===============================
🦋  warn Packages will be released under the experimental--ssg-no-streaming tag
🦋  warn ----------------------------------------------------------------------
🦋  info npm info astro
🦋  info npm info @astrojs/prism
🦋  info npm info @astrojs/rss
🦋  info npm info create-astro
🦋  info npm info @astrojs/alpinejs
🦋  info npm info @astrojs/lit
🦋  info npm info @astrojs/markdoc
🦋  info npm info @astrojs/mdx
🦋  info npm info @astrojs/node
🦋  info npm info @astrojs/partytown
🦋  info npm info @astrojs/preact
🦋  info npm info @astrojs/react
🦋  info npm info @astrojs/sitemap
🦋  info npm info @astrojs/solid-js
🦋  info npm info @astrojs/svelte
🦋  info npm info @astrojs/tailwind
🦋  info npm info @astrojs/vercel
🦋  info npm info @astrojs/vue
🦋  info npm info @astrojs/internal-helpers
🦋  info npm info @astrojs/markdown-remark
🦋  info npm info @astrojs/telemetry
🦋  info npm info @astrojs/underscore-redirects
🦋  info npm info @astrojs/upgrade
🦋  info astro is being published because our local version (0.0.0-ssg-no-streaming-20240104123901) has not been published on npm
🦋  warn @astrojs/prism is not being published because version 3.0.0 is already published on npm
🦋  warn @astrojs/rss is not being published because version 4.0.1 is already published on npm
🦋  info create-astro is being published because our local version (0.0.0-ssg-no-streaming-20240104123901) has not been published on npm
🦋  warn @astrojs/alpinejs is not being published because version 0.3.2 is already published on npm
🦋  warn @astrojs/lit is not being published because version 4.0.1 is already published on npm
🦋  warn @astrojs/markdoc is not being published because version 0.8.2 is already published on npm
🦋  warn @astrojs/mdx is not being published because version 2.0.3 is already published on npm
🦋  warn @astrojs/node is not being published because version 7.0.4 is already published on npm
🦋  warn @astrojs/partytown is not being published because version 2.0.3 is already published on npm
🦋  warn @astrojs/preact is not being published because version 3.0.2 is already published on npm
🦋  warn @astrojs/react is not being published because version 3.0.9 is already published on npm
🦋  warn @astrojs/sitemap is not being published because version 3.0.4 is already published on npm
🦋  warn @astrojs/solid-js is not being published because version 3.0.3 is already published on npm
🦋  warn @astrojs/svelte is not being published because version 5.0.3 is already published on npm
🦋  warn @astrojs/tailwind is not being published because version 5.1.0 is already published on npm
🦋  warn @astrojs/vercel is not being published because version 6.1.2 is already published on npm
🦋  warn @astrojs/vue is not being published because version 4.0.7 is already published on npm
🦋  warn @astrojs/internal-helpers is not being published because version 0.2.1 is already published on npm
🦋  warn @astrojs/markdown-remark is not being published because version 4.0.1 is already published on npm
🦋  warn @astrojs/telemetry is not being published because version 3.0.4 is already published on npm
🦋  warn @astrojs/underscore-redirects is not being published because version 0.3.3 is already published on npm
🦋  warn @astrojs/upgrade is not being published because version 0.2.2 is already published on npm
🦋  info Publishing "astro" at "0.0.0-ssg-no-streaming-20240104123901"
🦋  info Publishing "create-astro" at "0.0.0-ssg-no-streaming-20240104123901"
🦋  success packages published successfully:
🦋  astro@0.0.0-ssg-no-streaming-20240104123901
🦋  create-astro@0.0.0-ssg-no-streaming-20240104123901
🦋  Creating git tags...
🦋  New tag:  astro@0.0.0-ssg-no-streaming-20240104123901
🦋  New tag:  create-astro@0.0.0-ssg-no-streaming-20240104123901

Build Log


> root@0.0.0 build /home/runner/work/astro/astro
> turbo run build --filter=astro --filter=create-astro --filter="@astrojs/*" --filter="@benchmark/*"

• Packages in scope: @astrojs/alpinejs, @astrojs/cloudflare, @astrojs/internal-helpers, @astrojs/lit, @astrojs/markdoc, @astrojs/markdown-remark, @astrojs/mdx, @astrojs/netlify, @astrojs/node, @astrojs/partytown, @astrojs/preact, @astrojs/prism, @astrojs/react, @astrojs/rss, @astrojs/sitemap, @astrojs/solid-js, @astrojs/svelte, @astrojs/tailwind, @astrojs/telemetry, @astrojs/underscore-redirects, @astrojs/upgrade, @astrojs/vercel, @astrojs/vue, @benchmark/timer, astro, create-astro
• Running build in 26 packages
• Remote caching enabled
::group::@astrojs/upgrade:build
cache hit, suppressing logs 87f52b7a0803dd23
::endgroup::
::group::@astrojs/telemetry:build
cache hit, suppressing logs c98d31a4b10de42a
::endgroup::
::group::@astrojs/internal-helpers:build
cache hit, suppressing logs dcd2e47ddeb4319c
::endgroup::
::group::@astrojs/prism:build
cache hit, suppressing logs 3aa7f82954d2c6a4
::endgroup::
::group::@astrojs/markdown-remark:build
cache hit, suppressing logs c08381de7599ea31
::endgroup::
::group::create-astro:build
cache miss, executing f5328df0a6bf7058

> create-astro@0.0.0-ssg-no-streaming-20240104123901 build /home/runner/work/astro/astro/packages/create-astro
> astro-scripts build "src/index.ts" --bundle && tsc

::endgroup::
::group::astro:build
cache miss, executing 56f1fde50464be88

> astro@0.0.0-ssg-no-streaming-20240104123901 build /home/runner/work/astro/astro/packages/astro
> pnpm run prebuild && astro-scripts build "src/**/*.{ts,js}" && tsc && pnpm run postbuild


> astro@0.0.0-ssg-no-streaming-20240104123901 prebuild /home/runner/work/astro/astro/packages/astro
> astro-scripts prebuild --to-string "src/runtime/server/astro-island.ts" "src/runtime/client/{idle,load,media,only,visible}.ts"


> astro@0.0.0-ssg-no-streaming-20240104123901 postbuild /home/runner/work/a

perf: disable streaming in SSG

b47af21

github-actions bot added the pkg: astro Related to the core `astro` package (scope) label Jan 4, 2024

delucis mentioned this pull request Jan 4, 2024

Slow sidebar generation when there are a lot of items withastro/starlight#1318

Closed

1 task

delucis reviewed Jan 4, 2024

View reviewed changes

packages/astro/src/core/build/buildPipeline.ts Outdated Show resolved Hide resolved

Update packages/astro/src/core/build/buildPipeline.ts

a79a834

Co-authored-by: Chris Swithinbank <swithinbank@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

perf: disable streaming in SSG #9603

perf: disable streaming in SSG #9603

ematipico commented Jan 4, 2024

changeset-bot bot commented Jan 4, 2024 •

edited

Loading

ematipico commented Jan 4, 2024

ematipico commented Jan 4, 2024

ematipico commented Jan 4, 2024

github-actions bot commented Jan 4, 2024

github-actions bot commented Jan 4, 2024

github-actions bot commented Jan 4, 2024

ematipico commented Jan 4, 2024

ematipico commented Jan 4, 2024

ematipico commented Jan 4, 2024

github-actions bot commented Jan 4, 2024

github-actions bot commented Jan 4, 2024

perf: disable streaming in SSG #9603

perf: disable streaming in SSG #9603

Conversation

ematipico commented Jan 4, 2024

Changes

Testing

Docs

changeset-bot bot commented Jan 4, 2024 • edited Loading

⚠️ No Changeset found

ematipico commented Jan 4, 2024

ematipico commented Jan 4, 2024

ematipico commented Jan 4, 2024

github-actions bot commented Jan 4, 2024

github-actions bot commented Jan 4, 2024

PR Benchmark

Memory

Main Benchmark

Memory

github-actions bot commented Jan 4, 2024

PR Benchmark

Memory

Render

Server stress

CLI Startup

Main Benchmark

Memory

Render

Server stress

CLI Startup

ematipico commented Jan 4, 2024

ematipico commented Jan 4, 2024

ematipico commented Jan 4, 2024

github-actions bot commented Jan 4, 2024

PR Benchmark

Memory

Main Benchmark

Memory

github-actions bot commented Jan 4, 2024

changeset-bot bot commented Jan 4, 2024 •

edited

Loading