Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

en memory increase #75

Closed
yoomee1313 opened this issue Aug 23, 2024 · 3 comments
Closed

en memory increase #75

yoomee1313 opened this issue Aug 23, 2024 · 3 comments
Labels
bug Something isn't working

Comments

@yoomee1313
Copy link
Contributor

yoomee1313 commented Aug 23, 2024

Describe the bug
There's an issue reported that EN memory is keep increasing.
image

How to reproduce
Call debug_traceBlockByNumberRange for the latest 10 blocks once every 10 seconds.
But it seems that debug_traceBlockByNumber also can reproduce the problem.

Expected memory usage
The memory usage should be stable. The binary version of the next screenshot is kaia-v1.0.2 + #73, #74.
image

@yoomee1313 yoomee1313 added the bug Something isn't working label Aug 23, 2024
@calebcall
Copy link

We are also seeing this same memory growth issue. It'll consume up to the system limit and then OOM. We originally had our node on a 64Gb system and saw OOM errors, moved to a 128Gb system and the errors continued (just longer between)
Screenshot 2024-08-24 at 10 51 29 AM

@yoomee1313
Copy link
Contributor Author

Yes, we confirmed that there was a memory leak, and by applying the #73, #74, we can see that the memory is stabilized(expected memory usage). �Debug trace API is called periodically, isn't it? It might be good to try applying the above changes.

@yoomee1313
Copy link
Contributor Author

You can use https://github.com/kaiachain/kaia/tree/oom-fix branch. Or if you want a binary, please let us know.

@ian0371 ian0371 mentioned this issue Sep 25, 2024
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants