Allow limiting the number of threads spawned for housekeeping and invalidation.

[metasim]

Not sure how to express this (through the builder)? But for small resource use cases it would nice to limit the number of threads used.

[tatsuya6502]

Thanks for raising the request. Exposing these configurations should be easy, so I will just do it.

These thread pools are shared across all cache instances, so these limits will be cache-global configurations. Also, the thread pool library we use now does not support growing and shrinking the number of worker threads. So for now, they will be fixed numbers of worker threads, instead of limits (the maximum numbers of worker threads).

One limitation is that you cannot change those numbers while cache instances exist. So you will set them before creating any cache.

Just to clarify, this is how they work now (as of Moka v0.8.5):
(I will put them to somewhere in the doc)

• There are two global thread pools:
  1. Housekeeping
     • Used by all concurrent caches (sync, future, experimental dash caches).
     • Used for:
       • Updating internal cache data structures such as LRU queue and LFU filter.
       • Removing expired entries.
       • etc.
  2. Invalidation
     • Used by sync and future caches with invalidation_closures configuration explicitly enabled.
     • Used for executing user-provided invalidation closures on cache entries.
• A pool will be shared across all cache instance:
  • There should be zero or one housekeeping pool, and zero or one invalidation pool, no matter how many cache instances have been created.
• A pool will be created when a cache using it is created.
  • I think many users do not set invalidation closures, so only the housekeeping thread pool will be created in many cases.
• A pool will be dropped when all cache using it have been dropped.
• A pool has a fixed number of worker threads:
  • The number is the same to available logical cores of the processor(s) of the machine.
  • I will make the number configurable.

[tatsuya6502]

Closing as complete.

• In v0.9.3 (released in last summer, August 4th, 2022), we added an option to the cache builder of the sync caches not to start and use the global thread pools for housekeeping tasks.
  • Optionally disable thread pools in sync::Cache #165
• In v0.12.0 (currently beta), we removed the thread pool completely from all cache types.
  • Remove the thread pool from sync caches #316
  • Remove the thread pool from future::Cache #294

v0.12.0 has major breaking changes. Please read the MIGRATION-GUIDE.md for details.