[Memory Leak]: Vitepress builds incorporating @shikijs/vitepress-twoslash crash with JavaScript heap out of memory errors

[IMax153]

Validations

• Follow our Code of Conduct
• Read the Contributing Guide.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.

Describe the bug

Hello 👋 from the Effect-TS community :)

We have recently been experimenting with using different frameworks for building our documentation website. I decided to try building our documentation with Vitepress because of the native integration with Shiki Twoslash (via @shikijs/vitepress-twoslash), which we make extensive use of in our docs.

However, when I attempt to build our documentation with Vitepress and Shiki Twoslash, I consistently receive a JavaScript heap out of memory error. I have attempted to increase the --max-old-space-size to ~8GB and still encounter this issue.

With Twoslash Integration

In my .vitepress/config.ts, if I remove the transformerTwoslash from the markdown.codeTransformers array, the build succeeds without issue in ~20 seconds.

Without Twoslash Integration

Tracking issue with Vitepress: vuejs/vitepress#4242

Expected behavior

My expected behavior would be for Vitepress to be able to build our docs with Shiki Twoslash integrated without issue.

System Info

System:
    OS: macOS 14.6.1
    CPU: (10) arm64 Apple M1 Pro
    Memory: 1.99 GB / 16.00 GB
    Shell: 5.9 - /nix/store/w53pgqwh5dm38skdd7n7wn0k6cw72jzj-zsh-5.9/bin/zsh
Binaries:
    Node: 20.17.0 - /nix/store/hxl1k8qgmrm1vfq5f419iv4wybz9szqq-nodejs-20.17.0/bin/node
    npm: 10.8.2 - /nix/store/hxl1k8qgmrm1vfq5f419iv4wybz9szqq-nodejs-20.17.0/bin/npm
    pnpm: 9.10.0 - /nix/store/ddjsxwcpw09llisgvw7drvr92dbkg4mi-corepack-nodejs-20.17.0/bin/pnpm
Browsers:
    Chrome: 129.0.6668.70
    Safari: 17.6
npmPackages:
    vitepress: ^1.3.4 => 1.3.4

Note: I use Nix for provisioning my system, hence the strange looking binary paths.

Additional context

I've included a script in the package.json that reduces the available NodeJS heap memory (to force an OOM earlier) and takes a heap snapshot right before an OOM error occurs in case that is useful for you folks.

pnpm build:snapshot

Reproduction

https://github.com/IMax153/vitepress-memory-leak-repro.git

1. Clone our docs experiment

   git clone https://github.com/IMax153/vitepress-memory-leak-repro.git

2. Install dependencies

   pnpm install

3. Attempt to build

   pnpm build

Contributes

• I am willing to submit a PR to fix this issue
• I am willing to submit a PR with failing tests

[ap0nia]

Just adding my experience, I replicated the vitepress-twoslash plugin for rspress and was able to handle more twoslash code blocks before running out of memory. Like vitepress-twoslash, I used the core @shikijs/twoslash library. I'll see if I can figure out anything else...