Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Node process for "Build" command is taking much more RAM (6GB) than configured "max_old_space_size" option (4GB) #26553

Closed
1 task
sangalao opened this issue Dec 1, 2023 · 2 comments
Closed
1 task

Node process for "Build" command is taking much more RAM (6GB) than configured "max_old_space_size" option (4GB) #26553

sangalao opened this issue Dec 1, 2023 · 2 comments

Comments

@sangalao
Copy link

sangalao commented Dec 1, 2023 •
edited
Loading

Command

build

Is this a regression?

  • Yes, this behavior used to work in the previous version

The previous version in which this bug was not present was

No response

Description

Hello,

The build command I run for my project is :

node --max_old_space_size=4096 node_modules/@angular/cli/bin/ng build --configuration production

This commands runs successfully, but however the NodeJS process for building app takes up to 6GB.

To be more precise, during beginning-middle of building phase it takes less than 4GB,
and always at the middle-end of building phase, it goes up to 6GB just before end of successful build.

To start investigation, I launched build command with "-trace-gc" option :

node --trace-gc --max_old_space_size=4096 node_modules/@angular/cli/bin/ng build --configuration production

I saw, with this option, that there are several Node "V8 isolates" that are created in the same NodeJS process ("V8 isolate" = JS Heap instance, based on Node GC documentation) :

Among those "V8 isolates", with a proper heap for each, 2 of them have particular high heap amount at the same time :

  • The main one (Instance : 0000017EBED1F120) goes up to 3.6 GB of heap
  • A another one (Instance : 0000017FCBC9E030), created at middle-end of build phase, goes up to 2GB

What also have to be noticed is that the second one consumes such a memory ONLY when option "production > optimization > scripts" in "angular.json" is set to true.
The "production > optimization > scripts" option in "angular.json" is required for my production performance requirements (minification, tree-shaking, dead-code elimination).

So my questions/remarks are :

  • The "max_old_space_size" parameter seems to mainly apply to the MAIN V8 isolate, so I'm not able to configure "global" memory consumption during build phase. Is there a parameter to configure the GLOBAL heap used by NodeJS process for building phase ? If not, I think it could be useful to have one, and/or be able to fine-tune memory for all V8 isolates with high memory consumption

  • I cannot find any documentation explaning the fact that "production > optimization > scripts" option can result in high memory consumption, why it takes such memory, how I could tune it. Is is possible to have such a documentation / explanation ?

  • I also found this issue about memory limits for several "V8 isolates" in NodeJS, and was wondering whether it could be linked to my issue : worker: resourceLimits overridden by --max-old-space-size

Thanks

Minimal Reproduction

Command run :

node --trace-gc node_modules/@angular/cli/bin/ng build --configuration production

Packages :

  "dependencies": {
    "@angular/animations": "~14.2.0",
    "@angular/cdk": "~14.2.0",
    "@angular/cdk-experimental": "~14.2.0",
    "@angular/common": "~14.2.0",
    "@angular/compiler": "~14.2.0",
    "@angular/core": "~14.2.0",
    "@angular/flex-layout": "^14.0.0-beta.40",
    "@angular/forms": "~14.2.0",
    "@angular/material": "~14.2.0",
    "@angular/material-moment-adapter": "~14.2.0",
    "@angular/platform-browser": "~14.2.0",
    "@angular/platform-browser-dynamic": "~14.2.0",
    "@angular/router": "~14.2.0",
    "@danmarshall/deckgl-typings": "4.9.12",
    "@dashjoin/json-schema-form": "^0.9.0",
    "@deck.gl/aggregation-layers": "8.7.7",
    "@deck.gl/core": "8.7.7",
    "@deck.gl/extensions": "8.7.7",
    "@deck.gl/geo-layers": "8.7.7",
    "@deck.gl/json": "8.7.7",
    "@deck.gl/layers": "8.7.7",
    "@deck.gl/mapbox": "8.7.7",
    "@deck.gl/mesh-layers": "8.7.7",
    "@loaders.gl/core": "3.1.8",
    "@loaders.gl/draco": "3.1.8",
    "@loaders.gl/gltf": "3.1.8",
    "@loaders.gl/worker-utils": "3.1.8",
    "@luma.gl/constants": "8.5.10",
    "@luma.gl/core": "8.5.10",
    "@mapbox/mapbox-gl-draw": "^1.2.0",
    "@nebula.gl/layers": "^1.0.3-rev-2",
    "@nebula.gl/overlays": "1.0.4",
    "@ngx-translate/core": "^14.0.0",
    "@ngx-translate/http-loader": "^7.0.0",
    "@turf/along": "6.3.0",
    "@turf/destination": "6.3.0",
    "@turf/distance": "6.3.0",
    "@turf/helpers": "6.3.0",
    "@turf/length": "6.3.0",
    "angular-oauth2-oidc": "^13.0.1",
    "angular-oauth2-oidc-jwks": "^13.0.1",
    "angular-resize-event": "^3.2.0",
    "angular-tag-cloud-module": "^13.0.0",
    "aws-sdk": "^2.1104.0",
    "bpmn-js": "^11.5.0",
    "chroma-js": "2.1.0",
    "core-js": "^3.21.1",
    "css-loader": "^6.7.1",
    "csscolorparser": "1.0.3",
    "diagram-js": "^11.11.0",
    "echarts": "^5.4.0",
    "file-saver": "^2.0.2",
    "golden-layout": "^1.5.9",
    "html2canvas": "1.3.2",
    "ip-address": "5.9.0",
    "jquery": "^3.4.1",
    "jsep": "0.3.4",
    "jsonata": "^1.8.5",
    "jsoneditor": "^9.1.4",
    "jspdf": "^2.1.1",
    "jspdf-autotable": "^3.5.14",
    "jszip": "^3.10.1",
    "lodash": "^4.17.21",
    "mapbox-gl": "1.13.2",
    "moment": "^2.29.1",
    "mqtt": "4.3.7",
    "ng5-slider": "^1.2.4",
    "ngx-color": "^5.1.4",
    "ngx-echarts": "^14.0.0",
    "ngx-material-timepicker": "^5.5.3",
    "ngx-mqtt": "9.0.5",
    "ngx-papaparse": "^4.0.4",
    "ngx-quill-upload": "^2.0.0",
    "ngx-scrollbar": "^7.4.1",
    "ngx-ui-loader": "^13.0.0",
    "object-hash": "3.0.0",
    "primeicons": "^6.0.0",
    "primeng": "^14.2.1",
    "quill": "^2.0.0-dev.4-rev-1",
    "quill-better-table": "^1.2.10-rev-1",
    "quill-blot-formatter": "v1.0.5-rev-1",
    "quill-delta": "^5.0.0-rev-1",
    "roboto-fontface": "^0.10.0",
    "rxjs": "~7.5.0",
    "rxjs-compat": "^6.6.7",
    "stream": "0.0.2",
    "supercluster": "7.1.3",
    "three": "^0.131.3",
    "timers": "^0.1.1",
    "tslib": "^2.3.0",
    "util": "^0.12.4",
    "uuid": "8.3.2",
    "vis-timeline": "^7.7.0",
    "xml2js": "^0.4.23",
    "zone.js": "~0.11.4"

Exception or Error

No response

Your Environment

_                      _                 ____ _     ___
    / \   _ __   __ _ _   _| | __ _ _ __     / ___| |   |_ _|
   / △ \ | '_ \ / _` | | | | |/ _` | '__|   | |   | |    | |
  / ___ \| | | | (_| | |_| | | (_| | |      | |___| |___ | |
 /_/   \_\_| |_|\__, |\__,_|_|\__,_|_|       \____|_____|___|
                |___/


Angular CLI: 14.2.11
Node: 16.19.1
Package Manager: npm 8.19.3
OS: win32 x64

Angular: 14.2.12
... animations, common, compiler, compiler-cli, core, forms
... language-service, platform-browser, platform-browser-dynamic
... router

Package                            Version
------------------------------------------------------------
@angular-devkit/architect          0.1402.11
@angular-devkit/build-angular      14.2.11
@angular-devkit/core               14.2.11
@angular-devkit/schematics         14.2.11
@angular/cdk                       14.2.7
@angular/cdk-experimental          14.2.7
@angular/cli                       14.2.11
@angular/flex-layout               14.0.0-beta.41
@angular/material                  14.2.7
@angular/material-moment-adapter   14.2.7
@schematics/angular                14.2.11
rxjs                               7.5.7
typescript                         4.6.4

Anything else relevant?

Extract of "angular.json" used for production build :

"production": {
              "fileReplacements": [
                {
                  "replace": "src/environments/environment.ts",
                  "with": "src/environments/environment.prod.ts"
                }
              ],
              "optimization": {
                "scripts": true,
                "styles": true,
                "fonts": true
              },
              "outputHashing": "bundles",
              "sourceMap": false,
              "namedChunks": false,
              "aot": true,
              "extractLicenses": true,
              "vendorChunk": false,
              "buildOptimizer": false,
              "budgets": [
                {
                  "type": "initial",
                  "maximumWarning": "8mb",
                  "maximumError": "30mb"
                }
              ]
            }

Extract of build logs, with GC events (middle-end of build phase, with 2 V8 isolates having high amount of heap at the same time) :

[864:0000017FCBC9E030]    76944 ms: Scavenge 1794.3 (2071.8) -> 1790.7 (2071.8) MB, 2.2 / 0.0 ms  (average mu = 0.983, current mu = 0.982) allocation failure 
[864:0000017FCBC9E030]    76950 ms: Scavenge 1794.4 (2071.8) -> 1790.8 (2071.8) MB, 2.2 / 0.0 ms  (average mu = 0.983, current mu = 0.982) allocation failure 
[864:0000017FCBC9E030]    76958 ms: Scavenge 1794.6 (2071.8) -> 1791.0 (2071.8) MB, 2.5 / 0.0 ms  (average mu = 0.983, current mu = 0.982) allocation failure 
[864:0000017FCBC9E030]    76966 ms: Scavenge 1794.8 (2071.8) -> 1791.1 (2071.8) MB, 2.4 / 0.0 ms  (average mu = 0.983, current mu = 0.982) allocation failure 
[864:0000017FCBC9E030]    76972 ms: Scavenge 1794.9 (2071.8) -> 1791.3 (2071.8) MB, 2.3 / 0.0 ms  (average mu = 0.983, current mu = 0.982) allocation failure 
[864:0000017FCBC9E030]    76980 ms: Scavenge 1795.2 (2072.5) -> 1791.8 (2071.8) MB, 2.5 / 0.0 ms  (average mu = 0.983, current mu = 0.982) allocation failure 
[864:0000017EBED1F120]   380146 ms: Scavenge 3005.1 (3664.5) -> 3004.9 (3660.5) MB, 3.9 / 0.0 ms  (average mu = 0.932, current mu = 0.883) allocation failure 
[864:0000017EC91A0350]    77134 ms: Scavenge 308.7 (524.0) -> 305.4 (520.0) MB, 0.8 / 0.0 ms  (average mu = 0.988, current mu = 0.997) task 
[864:0000017FCCAC1060]    77142 ms: Mark-sweep 37.7 (46.9) -> 11.4 (36.8) MB, 4.0 / 0.0 ms  (+ 0.2 ms in 4 steps since start of marking, biggest step 0.2 ms, walltime since start of marking 9 ms) (average mu = 1.000, current mu = 1.000) finalize incremental marking via task GC in old space requested
[864:0000017EBED1F120]   381101 ms: Mark-sweep 3041.4 (3697.1) -> 2541.2 (3408.1) MB, 932.7 / 0.0 ms  (average mu = 0.984, current mu = 0.989) allocation failure scavenge might not succeed
[864:0000017EBED1F120]   381447 ms: Scavenge 2544.0 (3410.0) -> 2543.5 (3410.0) MB, 12.5 / 0.0 ms  (average mu = 0.984, current mu = 0.989) external memory pressure 
[864:0000017EBED1F120]   381451 ms: Scavenge 2543.5 (3410.0) -> 2543.6 (3410.0) MB, 3.5 / 0.0 ms  (average mu = 0.984, current mu = 0.989) external memory pressure 
[864:0000017EBED1F120]   382769 ms: Mark-sweep (reduce) 2543.7 (3410.0) -> 2538.6 (3096.7) MB, 72.6 / 0.1 ms  (+ 903.8 ms in 923 steps since start of marking, biggest step 2.8 ms, walltime since start of marking 999 ms) (average mu = 0.963, current mu = 0.415) finalize incremental marking via task GC in old space requested
[864:0000017EBED1F120]   395967 ms: Mark-sweep 2538.9 (3096.7) -> 2538.6 (3069.2) MB, 41.2 / 0.1 ms  (+ 789.9 ms in 1294 steps since start of marking, biggest step 19.7 ms, walltime since start of marking 850 ms) (average mu = 0.954, current mu = 0.937) finalize incremental marking via task GC in old space requested
[864:0000017EBED1F120]   414545 ms: Scavenge 2542.9 (3069.4) -> 2540.1 (3069.2) MB, 15.3 / 0.0 ms  (average mu = 0.954, current mu = 0.937) allocation failure 
[864:0000017EBED1F120]   414559 ms: Scavenge 2542.7 (3069.2) -> 2539.9 (3069.2) MB, 3.9 / 0.0 ms  (average mu = 0.954, current mu = 0.937) allocation failure 
[864:0000017EBED1F120]   414580 ms: Scavenge 2543.6 (3069.2) -> 2540.4 (3069.2) MB, 3.8 / 0.0 ms  (average mu = 0.954, current mu = 0.937) allocation failure
@clydin
Copy link
Member

clydin commented Dec 1, 2023

Angular v14 is no longer supported. Please consider updating to a more recent and supported version. There have been significant improvements in build performance since the v14 timeframe. For more information regarding currently supported Angular versions, please see: https://angular.dev/reference/releases#actively-supported-versions

If you are still encountering issues after updating, please open an issue with updated project information.

@clydin clydin closed this as completed Dec 1, 2023
@angular-automatic-lock-bot
Copy link

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

This action has been performed automatically by a bot.

@angular-automatic-lock-bot angular-automatic-lock-bot bot locked and limited conversation to collaborators Jan 4, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@sangalao @clydin