Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vite build error: out of memory #2433

Open
vijay19920608 opened this issue Mar 8, 2021 · 168 comments
Open

vite build error: out of memory #2433

vijay19920608 opened this issue Mar 8, 2021 · 168 comments
Labels
has workaround help wanted Extra attention is needed p4-important Violate documented behavior or significantly improves performance (priority) performance Performance related enhancement

Comments

@vijay19920608
Copy link

vijay19920608 commented Mar 8, 2021

NOTE - PLEASE READ

This issue has affected a number of people. Please do not comment with comments along the lines of "I'm also hitting this", but add a thumbs up to the issue instead. We know this is an important issue and "me too" comments will only drown out the comments from people attempting to resolve this issue and will be hidden. Please note however that Vite is a community driven project and a fix may need to come from the community

For tips on working around the issue, please see https://rollupjs.org/guide/en/#error-javascript-heap-out-of-memory

Describe the bug

vite build error: out of memory.

Reproduction

https://github.com/Axeldeblen/realworld-big-build

Possible improvements

Reduce magic string memory usage when building source maps: Rich-Harris/magic-string#161 (comment)

Logs (Optional if provided reproduction)

<--- JS stacktrace --->

==== JS stack trace =========================================

0: ExitFrame [pc: 00007FF69790ABBD]
Security context: 0x01e6a86408d1
1: decode(aka decode) [000002AD02F874D1] [E:\vite-template\node_modules_rollup@2.40.0@rollup\dist\shared\rollup.js:~133] [pc=0000039464A55451](this=0x037824a004b1 ,0x017863480119 <Very long string[1502653]>)
2: decodedSourcemap(aka decodedSourcemap) [000002AD02F8A979] [E:\vite-template\node_modules_rollup@2.40.0@rollup\dist\shared\roll...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
1: 00007FF696CD180F napi_wrap+119407
2: 00007FF696C787E6 v8::internal::OrderedHashTablev8::internal::OrderedHashSet,1::NextTableOffset+38102
3: 00007FF696C795E6 node::OnFatalError+438
4: 00007FF6974B5A6E v8::Isolate::ReportExternalAllocationLimitReached+94
5: 00007FF69749DC21 v8::SharedArrayBuffer::Externalize+833
6: 00007FF69734F3FC v8::internal::Heap::EphemeronKeyWriteBarrierFromCode+1436
7: 00007FF69735A640 v8::internal::Heap::ProtectUnprotectedMemoryChunks+1312
8: 00007FF697357154 v8::internal::Heap::PageFlagsAreConsistent+3204
9: 00007FF69734C953 v8::internal::Heap::CollectGarbage+1283
10: 00007FF69734AFC4 v8::internal::Heap::AddRetainedMap+2500
11: 00007FF69736C30D v8::internal::Factory::NewFillerObject+61
12: 00007FF6970CF76F v8::internal::interpreter::JumpTableTargetOffsets::iterator::operator=+1295
13: 00007FF69790ABBD v8::internal::SetupIsolateDelegate::SetupHeap+546925
14: 0000039464A55451

@alexgrozav
Copy link

alexgrozav commented Mar 8, 2021

Update It worked when I allocated a whopping 16GB of RAM for Vite's initialization cycle.

As a temporary workaround, you can use the following commands. If it still fails, just increase the size.

Temporary workaround

a. Using command line arguments

node --max_old_space_size=16384 ./node_modules/vite/bin/vite.js

b. Using environment variables

export NODE_OPTIONS=--max-old-space-size=32768

I'm getting the same error on https://github.com/inkline/inkline/tree/inkline3 after updating from 2.0.0-beta.60 to 2.0.5, but during dev, not build.

It looks like a memory leak to me. I've tried starting from a clean project and gradually added my code to it in order to find the issue. I discovered that it only happens when I rm -r node_modules, npm install and then npm run dev. If I work on the project with Vite already initialised and cached, it won't happen. I know that because I've started from a clean project 5 times. Every time after I move the files over to the old location and go through the installation, it will crash.

I tried increasing the memory allocation size to 9GB, but it didn't do the trick.

> vite


<--- Last few GCs --->

[17331:0x108008000]    46671 ms: Mark-sweep (reduce) 9192.9 (9207.7) -> 9192.9 (9207.7) MB, 8.7 / 0.0 ms  (average mu = 0.232, current mu = 0.001) allocation failure scavenge might not succeed
[17331:0x108008000]    46681 ms: Mark-sweep (reduce) 9192.9 (9204.7) -> 9192.9 (9205.7) MB, 9.4 / 0.0 ms  (average mu = 0.149, current mu = 0.029) last resort GC in old space requested


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x1012e4da5 node::Abort() (.cold.1) [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
 2: 0x1000a6239 node::Abort() [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
 3: 0x1000a639f node::OnFatalError(char const*, char const*) [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
 4: 0x1001e9007 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
 5: 0x1001e8fa3 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
 6: 0x100397e95 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
 7: 0x10039995a v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
 8: 0x100395029 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
 9: 0x1003928c1 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
10: 0x1003932d2 v8::internal::Heap::CollectAllAvailableGarbage(v8::internal::GarbageCollectionReason) [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
11: 0x1003a121e v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
12: 0x10036eb87 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
13: 0x1006ed8d8 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
14: 0x100a7a239 Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_NoBuiltinExit [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
15: 0x100accdde Builtins_RegExpSplit [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
zsh: abort      npm run dev

Randomly, I also get this error:

<--- Last few GCs --->

[18228:0x110008000]    43509 ms: Mark-sweep (reduce) 9192.9 (9204.7) -> 9192.9 (9205.7) MB, 8.9 / 0.0 ms  (average mu = 0.154, current mu = 0.039) last resort GC in old space requested
[18228:0x110008000]    43518 ms: Mark-sweep (reduce) 9192.9 (9204.7) -> 9192.9 (9205.7) MB, 9.0 / 0.0 ms  (average mu = 0.087, current mu = 0.001) last resort GC in old space requested


<--- JS stacktrace --->

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: 0x1012e4da5 node::Abort() (.cold.1) [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
 2: 0x1000a6239 node::Abort() [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
 3: 0x1000a639f node::OnFatalError(char const*, char const*) [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
 4: 0x1001e9007 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
 5: 0x1001e8fa3 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
 6: 0x100397e95 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
 7: 0x1003a128c v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
 8: 0x10036eb87 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
 9: 0x1006ed8d8 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
10: 0x100a7a239 Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_NoBuiltinExit [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
11: 0x100accdde Builtins_RegExpSplit [/Users/alexgrozav/.nvm/versions/node/v14.16.0/bin/node]
zsh: abort      node --max_old_space_size=9192 ./node_modules/vite/bin/vite.js

@danranVm

This comment was marked as off-topic.

@fadi-george
Copy link

For me at least, Vite was compiling my tests coverage folder which led to this heap error so I had to specify root: src in the config. Maybe optimizeDeps could also be set to exclude coverage but I haven't tried it.

@cpati-kochava
Copy link

I could only get it to work with node 15 on an Apple M1 without having to increase the max_old_space_size value. Either node 12 or 14 would run out of memory.

@Shinigami92 Shinigami92 added the p2-edge-case Bug, but has workaround or limited in scope (priority) label Apr 15, 2021
@TheDutchCoder
Copy link

TheDutchCoder commented Apr 15, 2021

It turns out I also had a coverage directory that was the cause of this issue.
I can't seem to set the root to 'src', no matter what I try, unless (I guess) I move the index.html file into src?

Either way, it would be good to have some sort of exclude option in the Vite config for this.

Edit: I wonder if it's due to there being an index.html file in one of those directories and Vite picking that up as if it were an entry point. Might be worth investigating from that angle.

@Shinigami92

This comment was marked as off-topic.

@samuveth

This comment was marked as off-topic.

@Shinigami92
Copy link
Member

@samuveth You should workaround this by using --max-old-space-size=<your-mem -512mb or so> for now

@nihalgonsalves
Copy link
Member

Also, just set it as an env variable and you don't have to call Vite's js script directly:

# where xxxx = (your memory in GiB) * 1024 - 512
export NODE_OPTIONS='--max-old-space-size=xxxx'

@agileago

This comment was marked as spam.

@eyalw

This comment was marked as off-topic.

@Niputi
Copy link
Contributor

Niputi commented Jan 31, 2022

contributions are welcome

@Niputi

This comment was marked as resolved.

@fadi-george

This comment was marked as off-topic.

@Darasimi-Ajewole
Copy link

For me, I was experiencing memory errors because of some heavy libraries used in my React application. I observed Vite was transforming over 7000 files every time I added these heavy libraries to the production build process.

To fix it, I had to find alternative ways to import those libraries or use their minified versions or import just the needed sub-packages for libraries with sub-packages like _lodash.

Lastly, disabling the source map in the Vite config reduced the memory needed to build my application by more than 500MB.

@jzs11

This comment was marked as off-topic.

@aleclarson
Copy link
Member

aleclarson commented Apr 3, 2022

I see some people are encountering this with vite dev. In that case, you should (1) connect the Chrome inspector to your Vite process and (2) pause the step debugger (do this shortly after starting the dev server and without loading anything in your browser). It's possible that you have a relatively complex circular directory symlink chain, and readdirp is being tricked into infinitely crawling those directories (but asynchronously, thereby avoiding a stack overflow). If the step debugger opens a readdirp file, you can be sure this is your issue. This is common if you're developing a chain of packages locally and linking one of them into your Vite project.

Using the server.watch.ignored option in Vite config, you can prevent chokidar (the file watcher that Vite uses) from watching the circular directory chain.

@samelie
Copy link
Contributor

samelie commented Feb 28, 2023

Perhaps counting the cpus helps?

import { cpus } from "os";
maxParallelFileOps: Math.max(1, cpus().length - 1),

@am2222
Copy link

am2222 commented Mar 1, 2023

@samelie I tried that and it didn't work.

P.s: for some reason the problem I had automatically got fixed. I feel gh actions were catching something and it got fixed automatically. I am still confused tho!

@OlielNoy
Copy link

OlielNoy commented Mar 5, 2023

I had this issue when I was building vite through a github action. As a temporary fix, you can add to NODE_OPTIONS the flag:
max-old-space-size

with a larger specification than the one it failed on (for me 'max-old-space-size=4096' did the trick).
For general information: the github lowest cost machines (ubunto-latest) reach 7GB RAM so the flag should be able to fix the issue for the node process / Vite that is the actual cause.

This can be done by creating a .npmrc file at the root of your project (where the package.json usually resides) and adding the following line in it:
max-old-space-size=4096

If you are not sure that the memory changed, you can check it by creating a script that runs the following:

`
import v8 from "v8";

console.table(v8.getHeapStatistics());
`

and running it as part of your github action steps after the installation phase

Hope that helps!

@pinchez254
Copy link

pinchez254 commented Mar 20, 2023

I finally found a fix on my react project which was legacy code base that I migrated from react rewired to vite. I tried all the hacks explained above and none seamed to work. After comparing with other projects I realised compile options in tsconfig.json was the one causing my all the headaches with memory allocation issues. I am using
vite v4.2.0 building for production and Node 18.13

Here is my working tsconfig.json config

  "compilerOptions": {
    "target": "ESNext",
    "useDefineForClassFields": true,
    "lib": ["DOM", "DOM.Iterable", "ESNext"],
    "allowJs": false,
    "skipLibCheck": true,
    "esModuleInterop": false,
    "allowSyntheticDefaultImports": true,
    "strict": true,
    "forceConsistentCasingInFileNames": true,
    "module": "ESNext",
    "moduleResolution": "Node",
    "resolveJsonModule": true,
    "isolatedModules": true,
    "noEmit": true,
    "jsx": "react-jsx"
  },
  "include": ["src /**/*"],
  "references": [{ "path": "./tsconfig.node.json" }],
  "lib": ["WebWorker"]
} 

and this is my tsconfig.node.json

  "compilerOptions": {
    "composite": true,
    "module": "ESNext",
    "moduleResolution": "Node",
    "allowSyntheticDefaultImports": true
  },
  "include": ["vite.config.ts"  , "src/**/*"]
}

and finally vite.config.ts

import { defineConfig, splitVendorChunkPlugin } from "vite";
import react from "@vitejs/plugin-react-swc";
import svgr from "vite-plugin-svgr";
import sass from "sass";
import path from "path";
const resolvePath = (str: string) => path.resolve(__dirname, str);
// https://vitejs.dev/config/
export default defineConfig({
  plugins: [react(), svgr()],
  // Configure the necessary loaders
  css: {
    preprocessorOptions: {
      scss: {
        implementation: sass,
        sassOptions: {
          includePaths: ["./node_modules"],
        },
      },
    },
  },
  resolve: {
    alias: {
      "@": resolvePath("src"),
    },
  },

  server: {
    port: 3000,
  },
  define: {
    "process.env": {},
  },
  build: {
    rollupOptions: {
      input: {
        main: resolvePath("index.html"),
        legacy: resolvePath("index.html"),
      },
    },
  },
});

@bluwy
Copy link
Member

bluwy commented Mar 20, 2023

After comparing with other projects I realised compile options in tsconfig.json was the one causing my all the headaches with memory allocation issues

@pinchez254 Can you elaborate on what compilerOptions did you changed to help with the memory allocation issue?

@abieleck
Copy link

After comparing with other projects I realised compile options in tsconfig.json was the one causing my all the headaches with memory allocation issues.

@pinchez254 I compared your tsconfig.json with mine. What worked for me was simply changing "skipLibCheck": true to false.

@EfstathiadisD
Copy link

An option that worked for me, even though it not perfect is the following. For me the packages that broke Vite + Rollup were one of @mui/material, @emotion/react, @emotion/style as all were imported at the same time as the out of memory error on Gitlab started to appear..

So, after some optimization on rollup made some improvements but only without sourcemap generation, I decide to split my app into smaller chunks with various techniques and also avoid generating sourcemaps for these libraries. With a small custom plugin..

function differMuiSourcemapsPlugins() {
  const muiPackages = ["@mui/material", "@emotion/styled", "@emotion/react"];

  return {
    name: "differ-mui-sourcemap",
    transform(code: string, id: string) {
      if (muiPackages.some((pkg) => id.includes(pkg))) {
        return {
          code: code,
          map: null,
        };
      }
    },
  };
}

And here is my partial ViteConfig, the parts that matter:

export default defineConfig(({ mode }) => {
  const env = loadEnv(mode, process.cwd());

  return {
    plugins: [
      react(),
      eslint(),
      createHtmlPlugin({
        inject: {
          data: env,
        },
      }),
      differMuiSourcemapsPlugins(),
    ],
    ...,
        build: {
      outDir: "build",
      sourcemap: mode === "development" || "hidden",
      rollupOptions: {
        maxParallelFileOps: Math.max(1, cpus().length - 1),
        output: {
          manualChunks: (id) => {
            if (id.includes("node_modules")) {
              return "vendor";
            }
          },
          sourcemapIgnoreList: (relativeSourcePath) => {
            const normalizedPath = path.normalize(relativeSourcePath);
            return normalizedPath.includes("node_modules");
          },
        },
        cache: false,
      },
    },
  };
});

Hope that helps some people. Tbh, I would like the underlying issue to be fixed in general as I tried a lot of different vite versions and none actually worked. Have a nice day!

@Leshe4ka
Copy link

Any updates about this bug? I've got this issue too (in GH actions), but none of the methods above worked. This bug unreproducible on local machine, and GH logs don't say something about the reason of the problem.

@andokai
Copy link

andokai commented Mar 28, 2023

Any updates about this bug? I've got this issue too (in GH actions), but none of the methods above worked. This bug unreproducible on local machine, and GH logs don't say something about the reason of the problem.

I got GH actions working by installing cross-env and altering the NPM build script to the following

"build": "cross-env NODE_OPTIONS=--max-old-space-size=8192 vite build",

@fc
Copy link
Contributor

fc commented Mar 28, 2023

We are testing out a plugin inspired by @EfstathiadisD's comment that disables source maps for all node modules instead of just a subset.

@samelie
Copy link
Contributor

samelie commented Mar 28, 2023

Thanks @EfstathiadisD for charting the course. Confirming this resolved all the problems.

import type { Plugin } from "vite";

interface SourcemapExclude {
    excludeNodeModules?: boolean;
}
export function sourcemapExclude(opts?: SourcemapExclude): Plugin {
    return {
        name: "sourcemap-exclude",
        transform(code: string, id: string) {
            if (opts?.excludeNodeModules && id.includes("node_modules")) {
                return {
                    code,
                    // https://github.com/rollup/rollup/blob/master/docs/plugin-development/index.md#source-code-transformations
                    map: { mappings: "" },
                };
            }
        },
    };
}

// USAGE

sourcemapExclude({ excludeNodeModules: true }),

@Leshe4ka
Copy link

I find a reason of problem in my case. It's import of elkjs library by the following way presented in their docs for typescript usage import ELK from 'elkjs/lib/elk.bundled' . But anyway no one solution doesn't work, also submitted by @EfstathiadisD and @samelie

@Kodnot
Copy link

Kodnot commented Apr 6, 2023

In case it helps someone else, what finally worked for us:

  • Raising the memory limit to 3GB with NODE_OPTIONS=--max-old-space-size=3072
  • Raising the build agent memory limit to 4GB, since it didn't have memory swap enabled it would crash after exceeding 3GB limit with no clear error.

All the other various workarounds listed in the comments didn't resolve the issue.

Something that helped debug the issue was building the project in WSL, I couldn't reproduce the issue when running build locally on windows but could do so when building the project through wsl.

@VHQDevFS
Copy link

VHQDevFS commented Apr 7, 2023

Any updates about this bug? I've got this issue too (in GH actions), but none of the methods above worked. This bug unreproducible on local machine, and GH logs don't say something about the reason of the problem.

I got GH actions working by installing cross-env and altering the NPM build script to the following

"build": "cross-env NODE_OPTIONS=--max-old-space-size=8192 vite build",

Thank your solution.

@EfstathiadisD

This comment was marked as duplicate.

@andokai

This comment was marked as spam.

@EfstathiadisD

This comment was marked as off-topic.

@IanVS
Copy link
Contributor

IanVS commented Apr 11, 2023

@patak-dev I wonder if perhaps this issue should be considered an upstream problem with rollup, and closed here, since there is a lot of noise and I think everything that can be said, has been said. In the end, there's nothing I think Vite can do to solve this issue.

@patak-dev
Copy link
Member

Even if it is upstream, it is an important issue for a lot of folks so it makes sense to keep tracking it here. I'm afraid people will create new issues if we close this one. Let's keep it open but read-only to avoid generating so many notifications to everybody until it is solved in rollup.

@vitejs vitejs locked and limited conversation to collaborators Apr 11, 2023
oneforalone added a commit to PhantaBlock/revite that referenced this issue Jul 12, 2023
@patak-dev patak-dev removed the bug label Feb 10, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
has workaround help wanted Extra attention is needed p4-important Violate documented behavior or significantly improves performance (priority) performance Performance related enhancement
Projects
None yet
Development

No branches or pull requests