Is there a method to return the max capacity size?

[shivam-880]

I was testing cache size based eviction.

let cache = Cache::builder()
            .max_capacity(1)
            .time_to_idle(Duration::from_secs(3))
            .build();

assert!(!cache.contains_key(&1));
assert!(!cache.contains_key(&2));

cache.insert(1, "one".to_string());
cache.insert(2, "two".to_string());

assert!(cache.contains_key(&1));
assert!(cache.contains_key(&2));

Despite the max_capacity being set to 1, when I make more than one entry in the cache, it retains both the value!
I wanted to see what is the max_capacity set for the cache but couldn't find any function available!

Also, what is wrong with my understanding here? Doesn't max_capacity represents the maximum number of entries. Isn't the cache supposed to evict older entries with the newer ones?

[tatsuya6502]

Hi. Thank you for using moka.

I wanted to see what is the max_capacity set for the cache but couldn't find any function available!

You can get it by calling cache.policy().max_capacity(). It will return Option<u64> where None means that there is no limit set.

Doesn't max_capacity represents the maximum number of entries. Isn't the cache supposed to evict older entries with the newer ones?

Yes, the cache will evict the least recently used entry (or temporary admitted entry) when the max_capacity is reached. But it will not happen immediately. The cache delays evictions to improve performance under concurrent read/write operations. It will evict a batch of entries when there are (1) enough read/write operations have been performed, or (2) enough time has passed. The same thing happens when a cached entry is expired.

We call these delayed tasks as pending maintenance tasks, and you can manually clean them up by calling cache.run_pending_tasks(). The following section explains about the maintenance tasks in detail:

• https://github.com/moka-rs/moka/blob/main/MIGRATION-GUIDE.md#the-maintenance-tasks

Another thing to mention is the lifetime of a cached entry.

All entries will be "temporary admitted" to the cache even when the max capacity is exceeded. When the pending tasks are processed, these temporary admitted entries may be officially admitted to the cache or evicted, depending on its popularity. The cache compares the popularity (how often it has been accessed) of the temporary admitted entry against the least recently used entry in the cache (the LRU in the above diagram). If the temporary admitted entry is more popular, it will be admitted to the cache, and the least recently used entry will be evicted. Otherwise, the temporary admitted entry will be evicted.

The cache is doing this to improve the hit ratio. The following section explains about the lifecycle and the cache policy:

• https://github.com/moka-rs/moka/wiki#admission-and-eviction-policies

Finally this program demonstrates the behavior of the cache with max_capacity and time_to_idle set:

use std::time::Duration;
use moka::sync::Cache;

let cache = Cache::builder()
    .max_capacity(1)
    .time_to_idle(Duration::from_secs(3))
    .build();

println!(" 1. max capacity: {:?}", cache.policy().max_capacity());

println!(" 2. cached entries: {:?}", cache);

cache.insert(1, "one".to_string());
cache.insert(2, "two".to_string());
println!(" 3. inserted 1 and 2");

// The both of the entries are temporary admitted even though
// the cache has overcapacity.
println!(" 4. cached entries: {:?}", cache);

// The entry count should be wrong here, because it will be updated
// when pending tasks are processed.
println!(" 5. entry count: {}", cache.entry_count());

// Clean up the pending tasks. This should do:
// - Admit 1 because there are enough room in the cache.
// - Evict 2 because the cache has overcapacity, and 2 is not
//   more popular than 1.
cache.run_pending_tasks();
println!(" 6. called run_pending_tasks");

println!(" 7. cached entries: {:?}", cache);
println!(" 8. entry count: {}", cache.entry_count());

// Calling get on 2 will make 2 more popular than 1 even if
// 2 is not in the cache.
assert!(cache.get(&2).is_none());
cache.insert(2, "two".to_string());
println!(" 9. called get on 2 and inserted it again");

// This should evict 1 because 1 is the least recently used
// admitted entry and 2 is more popular than 1.
cache.run_pending_tasks();
println!("10. called run_pending_tasks");

println!("11. cached entries: {:?}", cache);
println!("12. entry count: {}", cache.entry_count());

println!("13. sleeping for 4 seconds");
std::thread::sleep(Duration::from_secs(4));

// This should not return 2 because it has been expired.
println!("14. cached entries: {:?}", cache);
// But the entry count is still 1 because the cache has not
// yet ran the maintenance tasks to evict 2. (It is still
// in the cache)
println!("15. entry count: {}", cache.entry_count());

cache.run_pending_tasks();
println!("16. called run_pending_tasks");

println!("17. cached entries: {:?}", cache);
println!("18. entry count: {}", cache.entry_count());

$ cargo run
 1. max capacity: Some(1)
 2. cached entries: {}
 3. inserted 1 and 2
 4. cached entries: {1: "one", 2: "two"}
 5. entry count: 0
 6. called run_pending_tasks
 7. cached entries: {1: "one"}
 8. entry count: 1
 9. called get on 2 and inserted it again
10. called run_pending_tasks
11. cached entries: {2: "two"}
12. entry count: 1
13. sleeping for 4 seconds
14. cached entries: {}
15. entry count: 1
16. called run_pending_tasks
17. cached entries: {}
18. entry count: 0