Test to showcase high memory consumption of batch_script_get_history
#89
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When running `batch_script_get_history` on a Linux machine with processing many scripts the memory consumption is enormous and does not go down. This test showcases this behavior. When triggering the test with 7000 scripts we see a memory consumption of about 4GB. This problem could be reproduced on multiple Linux machine, but not on MacOS. Note that you can add a loop to re-trigger the batching to see that the memory does not get freed.
|
Thanks for test to repro this issue, any ideas why it might be happening? And do you think this would be a common usecase? In most cases I'd expect this to be used with smaller single users wallets and for very larger server wallets something like the bitcond RPC client (or a future BIP 157/158 client). |
|
Memory never being freed is an issue no matter what size though. I feel like there is a lot to improve in the way we are using electrum at every level. I know @evanlinjin has ideas. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When running
batch_script_get_historyon a Linux machine with processing many scripts the memory consumption is enormous and does not go down. This test showcases this behavior.When triggering the test with
7000scripts we see a memory consumption of about4GB.Note that you can add a loop to re-trigger the batching to see that the memory does not get freed.
This problem could be reproduced on multiple Linux machine, but not on MacOS.