-
Notifications
You must be signed in to change notification settings - Fork 29.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
src: add check against non-weak BaseObjects at process exit #35490
Conversation
When a process exits cleanly, i.e. because the event loop ends up without things to wait for, the Node.js objects that are left on the heap should be: 1. weak, i.e. ready for garbage collection once no longer referenced, or 2. detached, i.e. scheduled for destruction once no longer referenced, or 3. an unrefed libuv handle, i.e. does not keep the event loop alive, or 4. an inactive libuv handle (essentially the same here) There are a few exceptions to this rule, but generally, if there are C++-backed Node.js objects on the heap that do not fall into the above categories, we may be looking at a potential memory leak. Most likely, the cause is a missing `MakeWeak()` call on the corresponding object. In order to avoid this kind of problem, we check the list of BaseObjects for these criteria. In this commit, we only do so when explicitly instructed to or when in debug mode (where --verify-base-objects is always-on). In particular, this avoids the kinds of memory leak issues that were fixed in the PRs referenced below. Refs: nodejs#35488 Refs: nodejs#35487 Refs: nodejs#35481
Review requested:
|
// XXX: The garbage collection rules for ModuleWrap are *super* unclear. | ||
// Do these objects ever get GC'd? Are we just okay with leaking them? | ||
return true; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I know the bot already pinged the team, but I’d be curious to know if @nodejs/modules knows more about this … especially for ModuleWrap
s created by the vm
module, creating non-GC-able objects can’t be ideal, right…?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought they were all weak at one point?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
Landed in 40364b1 |
When a process exits cleanly, i.e. because the event loop ends up without things to wait for, the Node.js objects that are left on the heap should be: 1. weak, i.e. ready for garbage collection once no longer referenced, or 2. detached, i.e. scheduled for destruction once no longer referenced, or 3. an unrefed libuv handle, i.e. does not keep the event loop alive, or 4. an inactive libuv handle (essentially the same here) There are a few exceptions to this rule, but generally, if there are C++-backed Node.js objects on the heap that do not fall into the above categories, we may be looking at a potential memory leak. Most likely, the cause is a missing `MakeWeak()` call on the corresponding object. In order to avoid this kind of problem, we check the list of BaseObjects for these criteria. In this commit, we only do so when explicitly instructed to or when in debug mode (where --verify-base-objects is always-on). In particular, this avoids the kinds of memory leak issues that were fixed in the PRs referenced below. Refs: #35488 Refs: #35487 Refs: #35481 PR-URL: #35490 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
@addaleax This looks caused macOS build failed in github action ../src/env.cc:1227:22: error: lambda capture 'this' is not used [-Werror,-Wunused-lambda-capture]
ForEachBaseObject([this](BaseObject* obj) {
^~~~
1 error generated. |
@gengjiawen See #35547 :) |
This didn't land cleanly on v14.x, please raise a backport PR if this should land |
When a process exits cleanly, i.e. because the event loop ends up without things to wait for, the Node.js objects that are left on the heap should be: 1. weak, i.e. ready for garbage collection once no longer referenced, or 2. detached, i.e. scheduled for destruction once no longer referenced, or 3. an unrefed libuv handle, i.e. does not keep the event loop alive, or 4. an inactive libuv handle (essentially the same here) There are a few exceptions to this rule, but generally, if there are C++-backed Node.js objects on the heap that do not fall into the above categories, we may be looking at a potential memory leak. Most likely, the cause is a missing `MakeWeak()` call on the corresponding object. In order to avoid this kind of problem, we check the list of BaseObjects for these criteria. In this commit, we only do so when explicitly instructed to or when in debug mode (where --verify-base-objects is always-on). In particular, this avoids the kinds of memory leak issues that were fixed in the PRs referenced below. Refs: nodejs#35488 Refs: nodejs#35487 Refs: nodejs#35481 PR-URL: nodejs#35490 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
When a process exits cleanly, i.e. because the event loop ends up without things to wait for, the Node.js objects that are left on the heap should be: 1. weak, i.e. ready for garbage collection once no longer referenced, or 2. detached, i.e. scheduled for destruction once no longer referenced, or 3. an unrefed libuv handle, i.e. does not keep the event loop alive, or 4. an inactive libuv handle (essentially the same here) There are a few exceptions to this rule, but generally, if there are C++-backed Node.js objects on the heap that do not fall into the above categories, we may be looking at a potential memory leak. Most likely, the cause is a missing `MakeWeak()` call on the corresponding object. In order to avoid this kind of problem, we check the list of BaseObjects for these criteria. In this commit, we only do so when explicitly instructed to or when in debug mode (where --verify-base-objects is always-on). In particular, this avoids the kinds of memory leak issues that were fixed in the PRs referenced below. Refs: nodejs#35488 Refs: nodejs#35487 Refs: nodejs#35481 PR-URL: nodejs#35490 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
When a process exits cleanly, i.e. because the event loop ends up without things to wait for, the Node.js objects that are left on the heap should be: 1. weak, i.e. ready for garbage collection once no longer referenced, or 2. detached, i.e. scheduled for destruction once no longer referenced, or 3. an unrefed libuv handle, i.e. does not keep the event loop alive, or 4. an inactive libuv handle (essentially the same here) There are a few exceptions to this rule, but generally, if there are C++-backed Node.js objects on the heap that do not fall into the above categories, we may be looking at a potential memory leak. Most likely, the cause is a missing `MakeWeak()` call on the corresponding object. In order to avoid this kind of problem, we check the list of BaseObjects for these criteria. In this commit, we only do so when explicitly instructed to or when in debug mode (where --verify-base-objects is always-on). In particular, this avoids the kinds of memory leak issues that were fixed in the PRs referenced below. Refs: #35488 Refs: #35487 Refs: #35481 PR-URL: #35490 Backport-PR-URL: #38786 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
When a process exits cleanly, i.e. because the event loop ends up without things to wait for, the Node.js objects that are left on the heap should be: 1. weak, i.e. ready for garbage collection once no longer referenced, or 2. detached, i.e. scheduled for destruction once no longer referenced, or 3. an unrefed libuv handle, i.e. does not keep the event loop alive, or 4. an inactive libuv handle (essentially the same here) There are a few exceptions to this rule, but generally, if there are C++-backed Node.js objects on the heap that do not fall into the above categories, we may be looking at a potential memory leak. Most likely, the cause is a missing `MakeWeak()` call on the corresponding object. In order to avoid this kind of problem, we check the list of BaseObjects for these criteria. In this commit, we only do so when explicitly instructed to or when in debug mode (where --verify-base-objects is always-on). In particular, this avoids the kinds of memory leak issues that were fixed in the PRs referenced below. Refs: #35488 Refs: #35487 Refs: #35481 PR-URL: #35490 Backport-PR-URL: #38786 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
When a process exits cleanly, i.e. because the event loop ends up without things to wait for, the Node.js objects that are left on the heap should be:
There are a few exceptions to this rule, but generally, if there are C++-backed Node.js objects on the heap that do not fall into the above categories, we may be looking at a potential memory leak. Most likely, the cause is a missing
MakeWeak()
call on the corresponding object.In order to avoid this kind of problem, we check the list of BaseObjects for these criteria. In this commit, we only do so when explicitly instructed to or when in debug mode (where
--verify-base-objects
is always-on).In particular, this avoids the kinds of memory leak issues that were fixed in the PRs referenced below (hence the
blocked
label).Refs: #35488
Refs: #35487
Refs: #35481
Checklist
make -j4 test
(UNIX), orvcbuild test
(Windows) passes