memory leak when WASM used in conjunction with Jest

[bcoe]

There's a memory leak outlined here when the wasm version of this module is used in conjunction with Jest (which cleans up state between each test run).

It seems like perhaps new WebAssembly.Module is not being appropriately garbage collected; this might ultimately be an issue with the WASM implementation in Node.js, if so we can keep kicking this issue around 😝

[dcodeIO]

Very strange. Seems this could be one of three things:

• The long bundle creating a reference somewhere that the tooling doesn't see or isn't able to release
• Tooling not releasing all references to the long module properly
• Browsers/node not collecting WebAssembly instances

[kirillgroshkov]

• The long bundle creating a reference somewhere

Just a small comment that "somewhere" links to dist/long.js, but the actually used file is src/long.js as per package.json/main pointing to it. Don't know if that's intentional or not. Looked not obvious when I was debugging it.

[dcodeIO]

Yeah, I didn't know which entry point is used, so I pointed to the webpack'ed bundle in case it does something that just require("long") doesn't do. If the bundle isn't used, this makes the first option more unlikely.

[dcodeIO]

Fwiw, I haven't yet managed to trigger this reliably, except when I also enable the inspector:

// wat.js
var val = 1;

function makeGarbage() {
  var wasm = new WebAssembly.Instance(new WebAssembly.Module(new Uint8Array([0, 97, 115, 109, 1, 0, 0, 0, 1, 13, 2, 96, 0, 1, 127, 96, 4, 127, 127, 127, 127, 1, 127, 3, 7, 6, 0, 1, 1, 1, 1, 1, 6, 6, 1, 127, 1, 65, 0, 11, 7, 50, 6, 3, 109, 117, 108, 0, 1, 5, 100, 105, 118, 95, 115, 0, 2, 5, 100, 105, 118, 95, 117, 0, 3, 5, 114, 101, 109, 95, 115, 0, 4, 5, 114, 101, 109, 95, 117, 0, 5, 8, 103, 101, 116, 95, 104, 105, 103, 104, 0, 0, 10, 191, 1, 6, 4, 0, 35, 0, 11, 36, 1, 1, 126, 32, 0, 173, 32, 1, 173, 66, 32, 134, 132, 32, 2, 173, 32, 3, 173, 66, 32, 134, 132, 126, 34, 4, 66, 32, 135, 167, 36, 0, 32, 4, 167, 11, 36, 1, 1, 126, 32, 0, 173, 32, 1, 173, 66, 32, 134, 132, 32, 2, 173, 32, 3, 173, 66, 32, 134, 132, 127, 34, 4, 66, 32, 135, 167, 36, 0, 32, 4, 167, 11, 36, 1, 1, 126, 32, 0, 173, 32, 1, 173, 66, 32, 134, 132, 32, 2, 173, 32, 3, 173, 66, 32, 134, 132, 128, 34, 4, 66, 32, 135, 167, 36, 0, 32, 4, 167, 11, 36, 1, 1, 126, 32, 0, 173, 32, 1, 173, 66, 32, 134, 132, 32, 2, 173, 32, 3, 173, 66, 32, 134, 132, 129, 34, 4, 66, 32, 135, 167, 36, 0, 32, 4, 167, 11, 36, 1, 1, 126, 32, 0, 173, 32, 1, 173, 66, 32, 134, 132, 32, 2, 173, 32, 3, 173, 66, 32, 134, 132, 130, 34, 4, 66, 32, 135, 167, 36, 0, 32, 4, 167, 11])), {}).exports;
  wasm.mul(val, 0, 1, 0);
  return wasm.get_high();
}

for (let i = 0; ; ++i) {
  val += makeGarbage();
  if (!(i % 1000)) console.log(val);
}

$> node --inspect wat.js

which after a couple console.logs explodes with

<--- Last few GCs --->

[28332:0000022CB3D4EB80]     9225 ms: Mark-sweep 4.0 (8.2) -> 4.0 (8.2) MB, 3.4 / 0.1 ms  (average mu = 0.970, current mu = 0.004) memory pressure GC in old space requested
[28332:0000022CB3D4EB80]     9228 ms: Mark-sweep 4.0 (8.2) -> 4.0 (8.2) MB, 3.2 / 0.1 ms  (average mu = 0.944, current mu = 0.004) memory pressure GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 000000D78B250481]
    1: ConstructFrame [pc: 000000D78B20A0E6]
    2: StubFrame [pc: 000000D78B28C2C0]
Security context: 0x01afa7a1d9b1 <JSObject>
    3: makeGarbage [0000026A0ACD1641] [long\wat.js:4] [bytecode=0000026A0ACAFFE9 offset=45](this=0x03b7ff003609 <JSGlobal Object>)
    4: /* anonymous */ [000003E023DB0ED9] [long\wat.js:10] [bytecode=0000026A0ACE02E9 offset=24](this=0x03e023db0111 <Object map = 0000038...

FATAL ERROR: WasmCodeManager::NewNativeModule Allocation failed - process out of memory
 1: 00007FF638BE896A v8::internal::GCIdleTimeHandler::GCIdleTimeHandler+4554
 2: 00007FF638B98956 uv_loop_fork+85542
 3: 00007FF638B9941D uv_loop_fork+88301
 4: 00007FF638FBE72E v8::internal::FatalProcessOutOfMemory+798
 5: 00007FF638FBE667 v8::internal::FatalProcessOutOfMemory+599
 6: 00007FF638F65E3E v8::internal::wasm::WasmCodeManager::NewNativeModule+670
 7: 00007FF638F9966A v8::internal::AccountingAllocator::ClearPool+2538
 8: 00007FF639302563 unibrow::Utf8::ValidateEncoding+17299
 9: 00007FF638F6026A v8::internal::wasm::WasmEngine::SyncCompile+458
10: 00007FF63966FC17 v8::WasmStreaming::Unpack+5399
11: 00007FF6391FDBFF v8::internal::PassesFilter+847
12: 00007FF6391FEE37 v8::internal::PassesFilter+5511
13: 00007FF6391FE0AC v8::internal::PassesFilter+2044
14: 00007FF6391FDFCB v8::internal::PassesFilter+1819
15: 000000D78B250481

node v11.12.0, Win10

[bcoe]

@dcodeIO fascinating, what's interesting is there's an inspector session running in Node.js by default; feels like a bug that might be worth surfacing on Node.js itself.

[dcodeIO]

Maybe related:

• Pages using WebAssembly eventually crash with enough reloads brave/brave-browser#3366
• WebAssembly "Out of memory: wasm memory" bug polkadot-js/wasm#17

[dcodeIO]

Also, the crash goes away when moving compilation out of the loop, leaving instantiation within:

// wat.js
var val = 1;
var mod = new WebAssembly.Module(new Uint8Array([0, 97, 115, 109, 1, 0, 0, 0, 1, 13, 2, 96, 0, 1, 127, 96, 4, 127, 127, 127, 127, 1, 127, 3, 7, 6, 0, 1, 1, 1, 1, 1, 6, 6, 1, 127, 1, 65, 0, 11, 7, 50, 6, 3, 109, 117, 108, 0, 1, 5, 100, 105, 118, 95, 115, 0, 2, 5, 100, 105, 118, 95, 117, 0, 3, 5, 114, 101, 109, 95, 115, 0, 4, 5, 114, 101, 109, 95, 117, 0, 5, 8, 103, 101, 116, 95, 104, 105, 103, 104, 0, 0, 10, 191, 1, 6, 4, 0, 35, 0, 11, 36, 1, 1, 126, 32, 0, 173, 32, 1, 173, 66, 32, 134, 132, 32, 2, 173, 32, 3, 173, 66, 32, 134, 132, 126, 34, 4, 66, 32, 135, 167, 36, 0, 32, 4, 167, 11, 36, 1, 1, 126, 32, 0, 173, 32, 1, 173, 66, 32, 134, 132, 32, 2, 173, 32, 3, 173, 66, 32, 134, 132, 127, 34, 4, 66, 32, 135, 167, 36, 0, 32, 4, 167, 11, 36, 1, 1, 126, 32, 0, 173, 32, 1, 173, 66, 32, 134, 132, 32, 2, 173, 32, 3, 173, 66, 32, 134, 132, 128, 34, 4, 66, 32, 135, 167, 36, 0, 32, 4, 167, 11, 36, 1, 1, 126, 32, 0, 173, 32, 1, 173, 66, 32, 134, 132, 32, 2, 173, 32, 3, 173, 66, 32, 134, 132, 129, 34, 4, 66, 32, 135, 167, 36, 0, 32, 4, 167, 11, 36, 1, 1, 126, 32, 0, 173, 32, 1, 173, 66, 32, 134, 132, 32, 2, 173, 32, 3, 173, 66, 32, 134, 132, 130, 34, 4, 66, 32, 135, 167, 36, 0, 32, 4, 167, 11]));

function makeGarbage() {
  var wasm = new WebAssembly.Instance(mod, {}).exports;
  wasm.mul(val, 0, 1, 0);
  return wasm.get_high();
}

for (let i = 0; ; ++i) {
  val += makeGarbage();
  if (!(i % 1000)) console.log(val);
}

[bcoe]

@dcodeIO I wonder if this is a worthwhile refactor? perhaps memoize new WebAssembly.Module?

[dcodeIO]

The node module already does that, but not only the WASM module but also the WASM instance. In a normal node environment, this works just fine because a node module is created exactly once (i.e. every new Long uses the module-global memoized WASM instance already). The issue arises where a library like jest tries hard to unload the entire long module and require it again (or anything else that would make top-level statements within the module execute multiple times), which node wouldn't do by itself. My assumption is that this is indeed a node or v8 bug.

[tiagonapoli]

@dcodeIO @bcoe Webpack has some dependencies that uses this lib, so just requiring webpack increases virtual memory by 1GB, is there a way to not use WASM? (I don't know anything about it, just did some digging at webpack code and deps and got here)

webpack/webpack#9781

[sokra]

I've saw a similar memory leak before when using eval or other ways to create new scripts in v8 (new Function, new Script). The Source Code seem to be kept alive by the inspector forever.

e. g. here is a v8 issue about that: https://bugs.chromium.org/p/v8/issues/detail?id=7527

Seems like the same is true for WASM. Makes sense as it's another kind of script which is visible in the inspector.

You can't blame long.js for this problem, because it only allocates a single WASM module.
This is only a problem because jest is reloads node modules multiple times, which is unusual for node.js.
Anyway this seem to be to be v8 bug.

[mvasilkov]

@tiagonapoli

is there a way to not use WASM

As a workaround, you could replace the global WebAssembly object with null or something before loading the library — long.js will run the pure-JS branch then. Globals can be injected using e.g. webpack's ProvidePlugin

[dcodeIO]

There's also this hackish inline workaround:

var Instance = WebAssembly.Instance;
delete WebAssembly.Instance;
var Long = require("long");
WebAssembly.Instance = Instance;

[tiagonapoli]

This is indeed a v8 issue:
nodejs/node#29533
Apparently it was just fixed in V8 v7.9:
https://bugs.chromium.org/p/v8/issues/detail?id=9477
nodejs/node#29533 (comment)

[bcoe]

@tiagonapoli awesome 👍 I think we can close this then?