fix: safeRequest memory leak #443
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
miki725
force-pushed
the
memoryleak
branch
2 times, most recently
from
9b5a974 to
df11dc8
Compare
con4m populates external commands via its callback however this means that chalk indefinitely records external commands, even when reporting state is otherwise flushed (e.g. for each heartbeat report) as this can use unneeded memory, its better to clean it when wiping reporting state
Some plugins maintain their own state which is currently persisted even if its never needed anymore For example lambda/ecs plugins cache AWS responses during exec report. After exec report is sent, there is no more need to keep that state as cloud metadata is not reported during heartbeats however its memory is still used. With the clear state callback each plugin can manage its own cache as necessary not to waste memory.
safeRequest was creating new SSL context for each request however it was not property getting cleaned up via destroyContext(). As such each request was computing fresh SSL context which included all root certs which would accumulate ~1Mb of memory
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
CHANGELOG.mdif necessaryIssue
chalk running in as daemon with heartbeats had ever-increasing memory footprint
Description
Each heartbeat report, it would send HTTP request to chalk api. That was done via
safeRequestfrom nimutils which would initializeSslContext. That context included all root CA certs/etc and would use ~1Mb on average. At the end of the request, it wasnt correctly discarded as nim would GC its reference however it would call into openssl functions to clear all the inner data-structures. Now we explicitly destroy the context after each request to ensure its memory is correctly cleared.for more:
crashappsec/nimutils#81
Tests
Run long running process with heartbeats enabled and check its memory utilization.