v0.12.0: upstream fix to --max_old_space_size=2048 integer overflow

[bsnote]

v0.12.0 crashes when I specify --max_old_space_size=2048 or greater:

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory

https://code.google.com/p/v8/issues/detail?id=3857
https://codereview.chromium.org/897543002

[trevnorris]

Thanks for reporting. Unfortunately I can't reproduce the issue on my x64 Linux box. What are you running?

[bsnote]

repro.js:

var a = { };

for (var i = 0; i < 1000000; i++)
  a[i] = i;

$ node --max_old_space_size=2048 repro
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
Aborted (core dumped)
$ node --max_old_space_size=2047 repro
$ 

[trevnorris]

Hm. I get a different error message:

$ /usr/bin/time ./node --max_old_space_size=2048 /tmp/break.js                                              
FATAL ERROR: invalid table size Allocation failed - process out of memory
Command terminated by signal 6
2.12user 1.04system 0:03.99elapsed 79%CPU (0avgtext+0avgdata 1438704maxresident)k

[bsnote]

@trevnorris, did you try it with v0.12.0? It looks like a different issue preventing the reported one from happening.

[yunong]

We're also OOM errors on 0.12 64bit Linux. Running node with --max_old_space_size=6144.
This used to work on v0.10 and v0.11

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory

[bsnote]

I guess #9185 should be merged first.

[trevnorris]

@bsnote Sorry. I can't reproduce the issue you're having. Here's the script:

$ /usr/bin/time node --max_old_space_size=6144 -e 'var a={}; for(var i=0;i<7e7;i++) a[i]={i:i};'
3.31user 0.59system 0:03.76elapsed 103%CPU (0avgtext+0avgdata 3690436maxresident)k

Please share your system specs and anything else you think might be helpful for me to reproduce this issue.

[yunong]

@trevnorris I can repro on my AWS Instance. We're running r3.large instance types.

shaktitest@shakti-develop-i-3342c7dc:~$ node --max_old_space_size=6144 -e 'var a={}; for(var i=0;i<7e7;i++) a[i]={i:i};'
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
Aborted (core dumped)
shaktitest@shakti-develop-i-3342c7dc:~$ uname -a
Linux shakti-develop-i-3342c7dc 3.2.0-74-virtual #109-Ubuntu SMP Tue Dec 9 17:04:48 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
shaktitest@shakti-develop-i-3342c7dc:~$ node -v
v0.12.0
shaktitest@shakti-develop-i-3342c7dc:~$ file `which node`
/apps/node/bin//node: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.9, not stripped

[trevnorris]

@yunong are you using the exec from the website, or building your own?

[trevnorris]

@yunong Okay, just took the download from the website and was able to reproduce.

@tjfontaine @misterdjules There's something different between the builds that I'm using and the ones from the website that's causing this issue.

[yunong]

@trevnorris I'm using the 64bit Linux binary downloaded from nodejs.org

[trevnorris]

The amount of memory used isn't close to even the default maximum allowed:

$ /usr/bin/time ./bin/node --max_old_space_size=6144 -e 'var a={}; for(var i=0;i<7e7;i++) a[i]={i:i};'
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
Command terminated by signal 6
0.33user 0.04system 0:00.39elapsed 95%CPU (0avgtext+0avgdata 60736maxresident)k

Only 61MB?

[misterdjules]

@trevnorris The Linux binaries available on the website are built with make binary, which notably passes --with-intl=small-icu --without-snapshot to the configure script. Do you have some time to try if one of these flags (or both) causes this issue?

[tjfontaine]

@trevnorris at the very least it will be your compiler version, the older compiler here is probably tickling this overflow resulting in the failure

[trevnorris]

@tjfontaine That must be it. I ran make binary, but still can't reproduce on my machine. I'm using the latest clang possible:

Ubuntu clang version 3.6.0-svn225359-1~exp1 (trunk) (based on LLVM 3.6.0)
Target: x86_64-pc-linux-gnu
Thread model: posix
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9

I also just confirmed this is happening with io.js. Using https://iojs.org/dist/v1.2.0/iojs-v1.2.0-linux-x64.tar.gz I was able to reproduce the issue:

$ /usr/bin/time ./bin/iojs --max_old_space_size=6144 -e 'var a={}; for(var i=0;i<7e7;i++) a[i]={i:i};'
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
Command terminated by signal 6
0.30user 0.03system 0:00.36elapsed 94%CPU (0avgtext+0avgdata 58644maxresident)k

This may be a V8 bug.

/cc @bnoordhuis @indutny

[bsnote]

Anything wrong with the V8 upstream fix by @bnoordhuis that I attached to this issue?

[misterdjules]

@bsnote I totally missed the mention of the V8 issue and fix in your original comment, sorry for the confusion. I think @trevnorris might have missed that too, hence his question.

[trevnorris]

Ah yes. I did miss the linked V8 issue.

[trevnorris]

LGTM.

/cc @tjfontaine @misterdjules

[misterdjules]

@trevnorris LGTM, thanks!

[trevnorris]

Because the V8 version is still being sorted out in #9185, want this to land sooner rather than later. Will cherry-pick if/when V8 is updated again.

Thanks. Landed in 2fc5eeb.

[edgework]

@trevnorris Sorry if I'm missing this but will the fix be in an actual node.js release? 0.12.1 ?

[misterdjules]

@edgework Yes, this fix will be in node v0.12.1.