Commits on Apr 25, 2024

1. sstable: reduce block cache memory fragmentation …

   Currently, the sstable writer contains heuristics to flush sstable
   blocks once the size reaches a specified threshold. In CRDB this is
   defined as 32KiB. However, when these blocks are loaded into memory
   additional metadata is allocated sometimes exceeding the 32KiB threshold.
   Since CRDB uses jemalloc, these allocations use a 40KiB size class which
   leads to significant internal fragmentation. In addition, since the
   system is unaware of these size classes we cannot design heuristics that
   prioritize reducing memory fragmentation. Reducing internal
   fragmentation can help reduce CRDB's memory footprint. This commit
   decrements the target block size to prevent internal fragmentation for
   small key-value pairs and adds support for optionally specifying size
   classes to enable a new set of heuristics that will reduce internal
   fragmentation for workloads with larger key-value pairs.
   
   Fixes: cockroachdb#999.

   

   CheranMahalingam committed Apr 25, 2024
   Configuration menu

   • View commit details
   • Copy full SHA for 96d6f61
   • Browse repository at this point

   Copy the full SHA

   96d6f61 View commit details

   Browse the repository at this point in the history