Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

63 enhancement reduce overall memory consumption and improve read speed #64

Conversation

koculu
Copy link
Owner

@koculu koculu commented Jul 19, 2024

To reduce the memory consumption of ZoneTree, the following optimizations have been implemented:

  • Convert byte arrays (byte[]) into Memory<byte> to eliminate unnecessary buffer copies.
  • Utilize ArrayPool for efficient memory management.
  • Enhance read block caches by replacing the unstable CircularBlockCache.
  • Pin iterators to single block cache to prevent multiple block caches from consuming memory during a table scan.
  • Create and persist default sparse arrays upon disk segment creation.
  • Implement key and value caches for each disk segment to minimize IO operations.
  • Optimize the default configuration options.

koculu added 11 commits July 20, 2024 00:35
(cherry picked from commit d05b21d)
(cherry picked from commit 26888da)
(cherry picked from commit 28f19a1)
(cherry picked from commit 03a7c060996ca4485d948f95a7b42c47e6afa991)
(cherry picked from commit f8b42a712f9e6e90309966391fde911351b2a55f)
(cherry picked from commit f75bc0a231dcf67751d2d88fed9d55f46c5a40e6)
(cherry picked from commit a0f1862744ba092ad3e1b0d8fb52ac60d5e175d4)
(cherry picked from commit c8150aed1782b4b6110b5254f5da02d5af11df2d)
… reads.

(cherry picked from commit 47287b25591313b0c1fba0e6e801c66f74dbf56a)
@koculu koculu linked an issue Jul 19, 2024 that may be closed by this pull request
@koculu koculu merged commit 85a5a81 into main Jul 19, 2024
1 check passed
@koculu koculu deleted the 63-enhancement-reduce-overall-memory-consumption-and-improve-read-speed branch July 19, 2024 22:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Enhancement] Reduce overall memory consumption and improve read speed.
1 participant