Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory and CPU issue with Nilaway on Kubebuilder projects #148

Open
hoptical opened this issue Dec 3, 2023 · 5 comments
Open

Memory and CPU issue with Nilaway on Kubebuilder projects #148

hoptical opened this issue Dec 3, 2023 · 5 comments
Labels
documentation Improvements or additions to documentation enhancement New feature or request

Comments

@hoptical
Copy link

hoptical commented Dec 3, 2023

I'm having trouble using Nilaway to detect nil pointers in the S3 Operator project. My OS hangs due to high memory and CPU usage. They are totally utilized by Nilaway. What is wrong?

System specification running Nilaway:
RAM: 16GB
CPU: Intel® Core™ i7-10510U CPU @ 1.80GHz × 8
OS: Ubuntu 22.04 LTS

Run commands leading to the crash:

go install go.uber.org/nilaway/cmd/nilaway@latest
nilaway ./...
@painhardcore
Copy link

I'm getting a huge swap file in the end. But yeah, cpu and ram consumption is a problem
SCR-20231206-pkwt

@yuxincs
Copy link
Contributor

yuxincs commented Dec 6, 2023

Hi! NilAway by default analyzes everything (meaning all 3rd party dependencies and standard libraries), and the linter driver from go/analysis stores everything in memory.

If you specify only the first party packages for NilAway analysis (meaning we will apply optimistic defaults for packages not analyzed) by giving -include-pkgs flags (see https://github.com/uber-go/nilaway/wiki/Configuration#include-pkgs for details), the memory consumption should be significantly lower (assuming your Go project isn't 10M LOC of course).

(We are thinking of making it default to only analyze first party packages, or at least make it more explicit in our README, so I'm keeping this issue open for such efforts)

@yuxincs yuxincs added documentation Improvements or additions to documentation enhancement New feature or request labels Dec 6, 2023
@hoptical
Copy link
Author

hoptical commented Dec 7, 2023

@painhardcore @yuxincs Thanks for the following up.
@yuxincs I just used -include-pkgs to include only the first-party code and it got solved.

We had faced a nil pointer error in this line of the project in the runtime. I checked it out with Nilaway with this command since I knew it's caused by the subreconciler package:

nilaway -include-pkgs github.com/snapp-incubator/s3-operator,github.com/opdev/subreconciler ./...

However, Nilaway didn't recognize this part as a nil pointer and dangerous part of the code. I just wanted to mention it and you might find it helpful for further contributions to the package.

Thanks in advance.

@yuxincs
Copy link
Contributor

yuxincs commented Dec 7, 2023

@hoptical This is an interesting case, would you mind opening a new issue to elaborate it further?

I briefly took a look at the package but lost track of the call chain to see where the root cause is. We would be interested to investigate why NilAway fails to detect such false negatives.

@hoptical
Copy link
Author

@yuxincs Thanks. Sure. I submitted the issue in #159.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants