FinalizationGroup in combination with buffers seems to allocate memory without releasing it

[puzpuzpuz]

• Version: v12.13.1
• Platform: Linux apechkurov-laptop 4.15.0-72-generic fs: fix fd leak on early readstream destroy #81-Ubuntu SMP Tue Nov 26 12:20:02 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
• Subsystem: Buffer, FinalizationGroup API

Recently I was working on an experimental Buffer pool implementation based on FinalizationGroup API (see #30683) and encountered a weirdness that I'd like clarify. It seems that node doesn't free memory when FinalizationGroup is used to track Buffers (as "holdings") under certain conditions. This issue may be related with off-heap memory allocator or OS memory behavior, but I'd like to confirm that.

Here is the most simple reproducer that I could find:

'use strict';

const fg = new FinalizationGroup(finalizer);

// 8 ticks, 1GB per each => 8GB total
const ticks = 8;
const bufsPerTick = 1024;
const size = 1024 * 1024;

let slices = [];
let tick = 0;
setInterval(() => {
  tick += 1;
  if (tick === ticks) console.log('Registered all slices');
  if (tick > ticks) {
    slices = [];
    return;
  }

  slices = [];
  for (let i = 0; i < bufsPerTick; i++) {
    const buf = Buffer.alloc(size);
    const slice = buf.slice();
    slices.push(slice);
    fg.register(slice, buf);
  }
}, 500);

let finalized = 0;
function finalizer(iter) {
  for (const _ of iter) {
    finalized += 1;
    if (finalized === ticks * bufsPerTick)
      console.log('All finalizer callbacks are triggered');
  }
}

When this script is run under --harmony-weak-refs flag, node process consumes about 2GB of physical memory on my machine (and about 2.5GB of virtual memory) and that value doesn't decrease even after 10 minutes.

On the other hand, if you comment the fg.register(slice, buf); line, you'll see that resident memory consumption eventually goes down to ~32KB (virtual is ~600MB).

Once again, I'm not sure if that's a bug, but I'd like to understand the reason of such behavior.

[Jamesernator]

There's a decent chance it's on the v8 side, check this thread for implementation status: https://bugs.chromium.org/p/v8/issues/detail?id=8179

Also try on Node 13, it's possible the issue has already been fixed.

[puzpuzpuz]

Thanks @Jamesernator. I've tried running the script on v13.3.1-nightly20191204355b48bd06 and still see the same behavior with FinalizationGroup.register being used.

What's even more interesting, with commented fg.register(slice, buf); line, I can see about 900MB of resident memory consumption and it doesn't go down. It's totally different from memory consumption of v12.13.1 in the same scenario.

So, maybe it's not related with FG API itself and it's something different?

[puzpuzpuz]

This issue may be related with suboptimal behavior of glibc, which may have large RSS values in such benchmarks. See #21973

[puzpuzpuz]

Closing this one, as it doesn't seem to be related with node or v8. Feel free to comment if you also experience the same behavior.