Possible memory leaks #1669

danez · 2016-08-17T09:20:44Z

We are running constantly into out of memory errors with node-sass. Our scss setup is rather big and consists of 1076 scss files whereas 383 of them start with _.

We recently changed one of the scss underscore files to contain a map with all checksums for our images and this file is around 500kB big. What we did not expect is that node-sass/libsass memory usage will increase nearly by 100% which in our case means it jumped from 1.3GB to 2.1GB RAM usage.

All this memory is not freed after node-sass/libsass finished its work, and the watch tasks that run in the same node process after the full sass build still consumes > 2GB.

Even without our recent change 1.3GB is a massive amount of memory already and looks very much like memory leaks.

I profiled node-sass, but the profiler tells me that node only uses 30MB of memory, so the rest is probably coming from the native extension.
I'm not a c++ developer and do not know how to profile and debug the native extension. I could provide more details if anyone tells me how to get them. Unfortunately I cannot share our scss codebase.

version info:

$ node-sass --version
node-sass       3.8.0   (Wrapper)       [JavaScript]
libsass         3.3.6   (Sass Compiler) [C/C++]

saper · 2016-08-17T13:30:38Z

@danez would that be possible to drop some code to demonstrate the problem? libsass has various issues with regards to the memory management and it would be great to track this one down.

danez · 2016-08-18T21:14:27Z

Thanks for the response. I'm going to talk to our Head of Engineering next week and see how we can proceed here. Do you have an email address where we would be able to reach out to you? You can also shoot me an email, mine is in my profile.

saper · 2016-08-18T21:39:15Z

Sure, you can reach me at saper@saper.info

danez · 2016-08-31T09:06:20Z

Okay I'm going to summarize what we found out while investigating:
My colleague mailed saper directly (sorry if you were annoyed about that), but in case you don't have time and to not further annoy you with mails we thought it might be better to do it here and share our findings publicly.

the memory leak (I'm not sure if it is one, but I'm going to call it that) is not in node-sass but probably in libsass, as we were able to reproduce the behaviour also with sassc.
~~It only happens on linux, on mac it seems that the memory is freed after each file is processed.~~ Actually we are not sure about that, but if you are not able to reproduce it might be worth trying on linux.
Everything works better if we do not work with big maps. (500 map-gets in all 3 examples)
- Empty map: Peak Memory 20%, End memory: 4.6%
- Small map (247 entires): Peak Memory 19%, End memory: 5%
- Big map (6104 entires): Peak Memory 33,9%, End memory: 28,8%

(The memory is in % to easily see the difference 30% is around 1.3GB)

Here is a test sample that reproduces the big map problem:

@import './random-map';

@for $i from 1 through 500 {
  @debug map-get($map, "0f99ae3b-147f-4bc4-93aa-4e62b31b380c");
  @debug $i;
}

The big map can be found here: https://gist.github.com/danez/a979b74702e3057c04a9086a994e5962

And attached is the output of valgrind —tool=massif
log2.txt

thanks also to my college @detonator413 who did all the investigation.

mgreter · 2016-09-07T09:10:12Z

This should be addressed by sass/libsass#2171

danez · 2016-09-28T12:23:46Z

Thank you.
I compiled node-sass from source as it is written in the Readme and replaced the binding in our node_modules folder.

The example I posted above now does free up the memory after finish, but the peak memory usage is still around 1.3GB for this simple example.

Our initial problem stays exactly the same and still does OOM errors and does not free the memory. So we are going to do more investigation on this why.

danez · 2016-09-28T14:00:06Z

Okay here is another stripped down example:

@import './random-map';

@each $entry in $map {
  $value : map-get($map, $entry);
}

This is even worse, as with more than 3k items in the map it always runs into Unable to allocate memory: std::bad_alloc (On a machine with 4GB RAM)

The _random-map.scss is the same, but to avoid the allocation error I uploaded a shorter map with only ~2600 entries which uses around 2.7GB of memory. If this is to much remove entires from the map then the memory also goes down.
https://gist.github.com/danez/3c9f63046ed2e4dda1634ce1c4d6a9e8

Should I create a new Issue for this? In libsass/node-sass? Or is it okay here?

xzyfer · 2016-09-28T15:16:35Z

As @mgreter mentioned we have made improved to memory in sass/libsass#2171. You can compile node-sass with the latest LibSass using the following diff.

diff --git a/package.json b/package.json
index df179e1..bb97391 100644
--- a/package.json
+++ b/package.json
@@ -1,7 +1,7 @@
 {
   "name": "node-sass",
-  "version": "3.10.0",
-  "libsass": "3.3.6",
+  "version": "3.11.0",
+  "libsass": "3.4.0",
   "description": "Wrapper around libsass",
   "license": "MIT",
   "bugs": "https://github.com/sass/node-sass/issues",
diff --git a/src/libsass b/src/libsass
index 3ae9a20..2fcd639 160000
--- a/src/libsass
+++ b/src/libsass
@@ -1 +1 @@
-Subproject commit 3ae9a2066152f9438aebaaacd12f39deaceaebc2
+Subproject commit 2fcd6390a9d94949695ee99aa823781d53ea1760
diff --git a/src/libsass.gyp b/src/libsass.gyp
index fe22917..ce5729e 100644
--- a/src/libsass.gyp
+++ b/src/libsass.gyp
@@ -15,6 +15,7 @@
         'libsass/src/base64vlq.cpp',
         'libsass/src/bind.cpp',
         'libsass/src/cencode.c',
+        'libsass/src/check_nesting.cpp',
         'libsass/src/color_maps.cpp',
         'libsass/src/constants.cpp',
         'libsass/src/context.cpp',

You build new binary with

npm install

danez · 2016-09-28T15:19:14Z

All my tests today were done with a selfcompiled node-sass from master and I thought this includes the latest libsass. Will try with your diff.

danez · 2016-09-30T10:16:53Z

Okay I finally compiled the correct version, but now our codebase immediately throws

SystemStackError: stack level too deep
                    on line 12 of _placeholders.scss
 >>     @include background-color($color-alias, 1, false);
    -------------^

I tried increasing the max-stacklevel to 1000 but same problem.
What does stack level mean? stack in the sense of @imports?

danez · 2016-09-30T10:29:47Z

This error appears in a different file with different locations on each run with grunt-sass, so I think that parallel sass.render() calls either do not reset the recursion counter correctly or somehow share the counter if they run in parallel.

If I change grunt-sass to use renderSync everything works fine.

zephraph · 2016-10-13T18:03:37Z

Just to pitch in, my company is seeing a very similar issue. Our theming system is heavily based on sass maps. We can see upwards of 6 gigs of usage when building any given brand css file out.

As a little insight, we store pretty much all of our styles inside sass maps which are then pulled in via a system of mixins. We're actively looking into solutions or migration strategies to combat this.

nschonni · 2016-12-23T20:41:38Z

@danez since 4 has been released, it might be worth testing again because I believe there where a few performance related bits that got added in libsass

danez · 2016-12-23T20:47:29Z

@nschonni We tried upgrading last week, but as written above node-sass does not work anymore for our codebase at all and throws SystemStackError: stack level too deep. So we are stuck on v3 for now until someone figures out why this error happens randomly, because sometimes it also worked. Seems it has todo with concurrency of multiple parallel node-sass/libsass processes.

I wanted to try to create a repro case and open a new issue, but haven't had time to do so and might not have time for the next weeks.

xzyfer · 2016-12-28T05:46:46Z

We are aware of the memory issue. A patch will landing the next LibSass 3.5.beta. The LibSass 3.5.beta will land in Node Sass 5 betas in the new year.

stepmr · 2017-01-02T11:03:24Z

Thanks @xzyfer. Just ran into this too.

benallfree · 2017-01-02T21:13:02Z

In case it helps anyone isolate or understand the bug, I made a code demo that reproduces the issue on my machine: https://github.com/benallfree/node-sass-1669

DrEnter · 2017-01-05T04:04:39Z

We are running into this as well. Many thanks for the heads-up @xzyfer. @danez, I can confirm the problem occurs in Mac OS as well as Linux.

adampetrie · 2017-01-05T19:26:48Z

Is there a version of node-sass that we can downgrade to in the interim? I am also seeing this issue with:

node-sass	3.14.0-0	(Wrapper)	[JavaScript]
libsass  	3.3.6	(Sass Compiler)	[C/C++]

dbpolito · 2017-01-05T19:45:28Z

Is there a version of node-sass that we can downgrade to in the interim? I am also seeing this issue with:

Would love to know this too =)

benallfree · 2017-01-05T19:46:10Z

@adampetrie @dbpolito 3.x works.

edit: 3.14 as shown in https://github.com/benallfree/node-sass-1669

mgreter · 2017-01-05T22:07:26Z

Those are def. two different issues here! See sass/libsass#2286 for an explanation and fix for the "stack too deep" error. Thanks @benallfree for the test case, finding this would have been much harder without 👍

Anybody knows with what "trickery" they acheive the concurrency? Not sure if you should expect other hard to detect and/or random errors when running this way. LibSass is not guaranteed to be thread safe!

benallfree · 2017-01-05T22:50:42Z

@mgreter If libsass isn't thread-safe that may help explain why a bash concurrency (separate processes) works while webpack concurrency (separate threads?) doesn't. But it doesn't explain why node-sass 3.14 works while node-sass 4.x doesn't.

I'm coming to this issue from https://github.com/jtangelder/sass-loader#333 where I discovered it.

@adampetrie See https://github.com/benallfree/node-sass-1669 for an example that works with node-sass 3.14.

mgreter · 2017-01-05T23:33:47Z

@benallfree it's pretty clear what is happening, but not why. And yes, separating them in different processes will always work safely! Older node-sass work because this check was AFAIR added quite recently. sass/libsass#2286 should fix this issue, but I'm still a bit worried that there are other unknown side effects hidden in LibSass. I guess time will tell.

Here's a simplified node-sass only reproduction of the second issue:

var sass = require('node-sass');
function render() {
  sass.render({
    data: [
      "@mixin test2() {",
      "  foo: bar;",
      "}",
      "@mixin test() {",
      "  @for $i from 1 to 100000 {",
      "  }",
      "}",
      "foo {",
      "  @include test2;",
      "  @include test;",
      "  @include test2;",
      "}",
    ].join("\n"),
  }, function(error, result) {
    if (!result) console.log('error ', error);
  });
}

for (var i = 0; i < 100; i ++) {
  setTimeout(render, Math.random() * 1000);
}

FWIW: The original issue reported here is just how LibSass handled memory. Not really a memory leak. This will improve drastically with my latest memory patches landing in LibSass.

adampetrie · 2017-01-06T15:05:18Z

For others who come along, I can confirm that 3.14 works correctly too 😄

xzyfer · 2017-01-07T03:12:52Z

So it turns out I messed up the LibSass 3.4.2 release, and accidentally included the memory fixes scheduled for 3.5. Which means the memory fixes will should landing in the next node-sass minor this week.

xzyfer · 2017-01-09T06:31:12Z

This should be fixed in 4.2.0. Please let me know how y'all go

danez · 2017-01-17T13:26:43Z

The recursion bug is fixed, but now we have random segmentation faults in random files. Not sure how I should report that.

xzyfer · 2017-01-17T13:38:27Z

@danez do these segfault occur when using node-sass directly or via a build tool like grunt or gulp? I have seen this happen with grunt, but have been unable to reproduce whit node-sass directly.

danez · 2017-01-17T13:54:31Z

Yes they happen with grunt-sass. The problem might be that grunt-sass is calling sass.render() multiple times in an async loop and as @mgreter said before libsass is not thread safe, but that's just a guess.

But even if I change grunt-sass to do sass.renderSync in a normal loop it seems to fail.

xzyfer · 2017-01-17T13:59:09Z

I thought that might be case but was unable to reproduce the issue using the same async loop extracted out from grunt. Could easily be timing sensitive though.

danez · 2017-01-17T14:34:53Z

I created #1861

DrEnter · 2017-02-21T16:08:16Z

@xzyfer - We've successfully worked around the problem by adding a "useRenderSync" option to grunt-sass which calls "renderSync" instead of "render". Haven't seen a single issue since we did that. Note that the fix isn't as simple as changing "render" to "renderSync", as the loop itself needs to be modified. This wasn't intended as a clean and nice fix, more of a quick and dirty hack to get it to work, but you can try it here.

aelkoussy · 2019-08-05T11:58:14Z

Did anyone find a solution for this? we have a memory leak issue & we suspect that it is related to this bug, since rubocop-thread_safety only shows the libsass within the node-sass to be the only offender for thread safety ...

Is there a workaround at least to make the project thread safe?

saper · 2019-08-05T12:25:19Z

This issue was related to the memory problem @aelkoussy How are you testing node-sass with rubocop?

aelkoussy · 2019-08-05T16:53:19Z

@saper Actually I am not testing node-sass especially, I am testing my project using this cmd:

rubocop -r rubocop-thread_safety --only ThreadSafety
and then it showed some issues in my code & some issues in node-sass (especially libsass that is inside node-sass)

aelkoussy · 2019-08-05T16:55:36Z

Giving output like this (I truncated the output)

node_modules/node-sass/src/libsass/contrib/libsass.spec:47:1: E: Lint/Syntax: %post: unknown type of percent-literal
(Using Ruby 2.6 parser; configure using TargetRubyVersion parameter, under AllCops)
%post -p /sbin/ldconfig
^^
node_modules/node-sass/src/libsass/contrib/libsass.spec:49:1: E: Lint/Syntax: %postun: unknown type of percent-literal
(Using Ruby 2.6 parser; configure using TargetRubyVersion parameter, under AllCops)
%postun -p /sbin/ldconfig
^^
node_modules/node-sass/src/libsass/contrib/libsass.spec:49:1: E: Lint/Syntax: unexpected token error
(Using Ruby 2.6 parser; configure using TargetRubyVersion parameter, under AllCops)
%postun -p /sbin/ldconfig
^^^^^^^^
node_modules/node-sass/src/libsass/contrib/libsass.spec:52:1: E: Lint/Syntax: %files: unknown type of percent-literal
(Using Ruby 2.6 parser; configure using TargetRubyVersion parameter, under AllCops)
%files
^^
node_modules/node-sass/src/libsass/contrib/libsass.spec:52:1: E: Lint/Syntax: unexpected token error
(Using Ruby 2.6 parser; configure using TargetRubyVersion parameter, under AllCops)
%files ...
^^^^^^
node_modules/node-sass/src/libsass/contrib/libsass.spec:56:1: E: Lint/Syntax: %files: unknown type of percent-literal
(Using Ruby 2.6 parser; configure using TargetRubyVersion parameter, under AllCops)
%files devel
^^
node_modules/node-sass/src/libsass/contrib/libsass.spec:56:1: E: Lint/Syntax: unexpected token error
(Using Ruby 2.6 parser; configure using TargetRubyVersion parameter, under AllCops)
%files devel
^^^^^^^
node_modules/node-sass/src/libsass/contrib/libsass.spec:64:14: E: Lint/Syntax: unexpected token tINTEGER
(Using Ruby 2.6 parser; configure using TargetRubyVersion parameter, under AllCops)
* Tue Feb 10 2015 Gawain Lynch <gawain.lynch@gmail.com> - 3.1.0-1
             ^^^^
node_modules/node-sass/src/libsass/contrib/libsass.spec:64:45: E: Lint/Syntax: unexpected token tIVAR
(Using Ruby 2.6 parser; configure using TargetRubyVersion parameter, under AllCops)
* Tue Feb 10 2015 Gawain Lynch <gawain.lynch@gmail.com> - 3.1.0-1
                                            ^^^^^^
node_modules/node-sass/src/libsass/contrib/libsass.spec:64:62: E: Lint/Syntax: no .<digit> floating literal anymore; put 0 before dot
(Using Ruby 2.6 parser; configure using TargetRubyVersion parameter, under AllCops)
* Tue Feb 10 2015 Gawain Lynch <gawain.lynch@gmail.com> - 3.1.0-1
                                                             ^^
node_modules/node-sass/src/libsass/contrib/libsass.spec:65:11: E: Lint/Syntax: unexpected token tCONSTANT
(Using Ruby 2.6 parser; configure using TargetRubyVersion parameter, under AllCops)
- Initial SPEC file
          ^^^^

272 files inspected, 45 offenses detected

saper · 2019-08-05T23:42:01Z

Those errors are reported because https://github.com/sass/libsass/blob/8057222005fbb9fc173f70b240befd25e15673db/contrib/libsass.spec file is an https://github.com/rpm-software-management/rpm RPM Package Manager file, not a Ruby file. It seems like you are running into an issue rubocop/rubocop#4553 which is not related to node-sass or libsass at all.

aelkoussy · 2019-08-07T09:52:26Z

I felt a bit weird about the files but due to that issue I thought it might be a cause for my memory problem, thanks for explaining @saper

saper added the Awaiting Response label Aug 17, 2016

xzyfer added this to the next.libsass milestone Sep 8, 2016

xzyfer added Bug - Confirmed External - LibSass and removed Awaiting Response labels Sep 8, 2016

nschonni modified the milestones: 4.0, next.libsass Sep 15, 2016

danez mentioned this issue Oct 7, 2016

Implementation in another language? babel/babylon#159

Closed

samuliheljo mentioned this issue Jan 2, 2017

DT-1730 Downgrade node-sass HSLdevcom/digitransit-ui#1191

Merged

gcarothers mentioned this issue Jan 2, 2017

Remove synchronous compilation webpack-contrib/sass-loader#333

Closed

shouze mentioned this issue Jan 5, 2017

Bump node-sass@v4.0.0 stevenschobert/metalsmith-sass#40

Merged

mgreter mentioned this issue Jan 5, 2017

Fix thread-safety for mixin recursions check sass/libsass#2286

Merged

xzyfer mentioned this issue Jan 7, 2017

LibSass 3.4.3 #1848

Merged

xzyfer closed this as completed in #1848 Jan 7, 2017

Possible memory leaks #1669

Possible memory leaks #1669

Comments

danez commented Aug 17, 2016

saper commented Aug 17, 2016

danez commented Aug 18, 2016 • edited Loading

saper commented Aug 18, 2016

danez commented Aug 31, 2016 • edited Loading

mgreter commented Sep 7, 2016

danez commented Sep 28, 2016 • edited Loading

danez commented Sep 28, 2016 • edited Loading

xzyfer commented Sep 28, 2016

danez commented Sep 28, 2016

danez commented Sep 30, 2016

danez commented Sep 30, 2016 • edited Loading

zephraph commented Oct 13, 2016

nschonni commented Dec 23, 2016

danez commented Dec 23, 2016 • edited Loading

xzyfer commented Dec 28, 2016

stepmr commented Jan 2, 2017

benallfree commented Jan 2, 2017 • edited Loading

DrEnter commented Jan 5, 2017

adampetrie commented Jan 5, 2017

dbpolito commented Jan 5, 2017

benallfree commented Jan 5, 2017 • edited Loading

mgreter commented Jan 5, 2017

benallfree commented Jan 5, 2017 • edited Loading

mgreter commented Jan 5, 2017

adampetrie commented Jan 6, 2017

xzyfer commented Jan 7, 2017

xzyfer commented Jan 9, 2017

danez commented Jan 17, 2017

xzyfer commented Jan 17, 2017

danez commented Jan 17, 2017 • edited Loading

xzyfer commented Jan 17, 2017

danez commented Jan 17, 2017

DrEnter commented Feb 21, 2017

aelkoussy commented Aug 5, 2019

saper commented Aug 5, 2019

aelkoussy commented Aug 5, 2019

aelkoussy commented Aug 5, 2019

saper commented Aug 5, 2019

aelkoussy commented Aug 7, 2019

danez commented Aug 18, 2016 •

edited

Loading

danez commented Aug 31, 2016 •

edited

Loading

danez commented Sep 28, 2016 •

edited

Loading

danez commented Sep 28, 2016 •

edited

Loading

danez commented Sep 30, 2016 •

edited

Loading

danez commented Dec 23, 2016 •

edited

Loading

benallfree commented Jan 2, 2017 •

edited

Loading

benallfree commented Jan 5, 2017 •

edited

Loading

benallfree commented Jan 5, 2017 •

edited

Loading

danez commented Jan 17, 2017 •

edited

Loading