Skip to content

Commit

Permalink
Merge pull request #390 from moka-rs/lru-policy
Browse files Browse the repository at this point in the history
Support the plain LRU policy
  • Loading branch information
tatsuya6502 authored Jan 29, 2024
2 parents 2e904db + 27dd941 commit b5560df
Show file tree
Hide file tree
Showing 11 changed files with 430 additions and 48 deletions.
24 changes: 23 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,26 @@
# Moka Cache — Change Log

## Version 0.12.5

### Added

- Added support for a plain LRU (Least Recently Used) eviction policy
([#390][gh-pull-0390]):
- The LRU policy is enabled by calling the `eviction_policy` method of the cache
builder with a policy obtained by `EvictionPolicy::lru` function.
- The default eviction policy remains the TinyLFU (Tiny, Least Frequently Used)
as it maintains better hit rate than LRU for most use cases. TinyLFU combines
LRU eviction policy and popularity-based admission policy. A probabilistic data
structure is used to estimate historical popularity of both hit and missed
keys. (not only the keys currently in the cache.)
- However, some use cases may prefer LRU policy over TinyLFU. An example is
recency biased workload such as streaming data processing. LRU policy can be
used for them to achieve better hit rate.
- Note that we are planning to add an adaptive eviction/admission policy called
Window-TinyLFU in the future. It will adjust the balance between recency and
frequency based on the current workload.


## Version 0.12.4

### Fixed
Expand All @@ -8,7 +29,7 @@
- `crossbeam-epoch` crate provides an epoch-based memory reclamation scheme for
concurrent data structures. It is used by Moka cache to safely drop cached
entries while they are still being accessed by other threads.
- `crossbeam-epoch` does the best to reclaim memory (drop the entries evicted
- `crossbeam-epoch` does its best to reclaim memory (drop the entries evicted
from the cache) when the epoch is advanced. However, it does not guarantee that
memory will be reclaimed immediately after the epoch is advanced. This means
that entries can remain in the memory for a while after the cache is dropped.
Expand Down Expand Up @@ -829,6 +850,7 @@ The minimum supported Rust version (MSRV) is now 1.51.0 (Mar 25, 2021).
[gh-issue-0034]: https://github.com/moka-rs/moka/issues/34/
[gh-issue-0031]: https://github.com/moka-rs/moka/issues/31/

[gh-pull-0390]: https://github.com/moka-rs/moka/pull/390/
[gh-pull-0384]: https://github.com/moka-rs/moka/pull/384/
[gh-pull-0382]: https://github.com/moka-rs/moka/pull/382/
[gh-pull-0376]: https://github.com/moka-rs/moka/pull/376/
Expand Down
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "moka"
version = "0.12.4"
version = "0.12.5"
edition = "2021"
# Rust 1.65 was released on Nov 3, 2022.
rust-version = "1.65"
Expand Down
37 changes: 26 additions & 11 deletions src/future/base_cache.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ use crate::{
},
future::CancelGuard,
notification::{AsyncEvictionListener, RemovalCause},
policy::ExpirationPolicy,
policy::{EvictionPolicy, EvictionPolicyConfig, ExpirationPolicy},
sync_base::iter::ScanningGet,
Entry, Expiry, Policy, PredicateError,
};
Expand Down Expand Up @@ -167,6 +167,7 @@ where
initial_capacity: Option<usize>,
build_hasher: S,
weigher: Option<Weigher<K, V>>,
eviction_policy: EvictionPolicy,
eviction_listener: Option<AsyncEvictionListener<K, V>>,
expiration_policy: ExpirationPolicy<K, V>,
invalidator_enabled: bool,
Expand All @@ -187,6 +188,7 @@ where
initial_capacity,
build_hasher,
weigher,
eviction_policy,
eviction_listener,
r_rcv,
w_rcv,
Expand Down Expand Up @@ -1041,6 +1043,7 @@ pub(crate) struct Inner<K, V, S> {
read_op_ch: Receiver<ReadOp<K, V>>,
write_op_ch: Receiver<WriteOp<K, V>>,
maintenance_task_lock: RwLock<()>,
eviction_policy: EvictionPolicyConfig,
expiration_policy: ExpirationPolicy<K, V>,
valid_after: AtomicInstant,
weigher: Option<Weigher<K, V>>,
Expand Down Expand Up @@ -1191,6 +1194,7 @@ where
initial_capacity: Option<usize>,
build_hasher: S,
weigher: Option<Weigher<K, V>>,
eviction_policy: EvictionPolicy,
eviction_listener: Option<AsyncEvictionListener<K, V>>,
read_op_ch: Receiver<ReadOp<K, V>>,
write_op_ch: Receiver<WriteOp<K, V>>,
Expand Down Expand Up @@ -1247,6 +1251,7 @@ where
read_op_ch,
write_op_ch,
maintenance_task_lock: RwLock::default(),
eviction_policy: eviction_policy.config,
expiration_policy,
valid_after: AtomicInstant::default(),
weigher,
Expand Down Expand Up @@ -1451,7 +1456,9 @@ where
.await;
}

if self.should_enable_frequency_sketch(&eviction_state.counters) {
if self.eviction_policy == EvictionPolicyConfig::TinyLfu
&& self.should_enable_frequency_sketch(&eviction_state.counters)
{
self.enable_frequency_sketch(&eviction_state.counters).await;
}

Expand Down Expand Up @@ -1742,15 +1749,21 @@ where
}
}

let mut candidate = EntrySizeAndFrequency::new(new_weight);
candidate.add_frequency(freq, kh.hash);
// TODO: Refactoring the policy implementations.
// https://github.com/moka-rs/moka/issues/389

// Try to admit the candidate.
//
// NOTE: We need to call `admit` here, instead of a part of the `match`
// expression. Otherwise the future returned from this `handle_upsert` method
// will not be `Send`.
let admission_result = Self::admit(&candidate, &self.cache, deqs, freq);
let admission_result = match &self.eviction_policy {
EvictionPolicyConfig::TinyLfu => {
let mut candidate = EntrySizeAndFrequency::new(new_weight);
candidate.add_frequency(freq, kh.hash);
Self::admit(&candidate, &self.cache, deqs, freq)
}
EvictionPolicyConfig::Lru => AdmissionResult::Admitted {
victim_keys: SmallVec::default(),
},
};

match admission_result {
AdmissionResult::Admitted { victim_keys } => {
// Try to remove the victims from the hash map.
Expand Down Expand Up @@ -2760,7 +2773,7 @@ fn is_expired_by_ttl(

#[cfg(test)]
mod tests {
use crate::policy::ExpirationPolicy;
use crate::policy::{EvictionPolicy, ExpirationPolicy};

use super::BaseCache;

Expand All @@ -2779,8 +2792,9 @@ mod tests {
None,
RandomState::default(),
None,
EvictionPolicy::default(),
None,
Default::default(),
ExpirationPolicy::default(),
false,
);
cache.inner.enable_frequency_sketch_for_testing().await;
Expand Down Expand Up @@ -3127,6 +3141,7 @@ mod tests {
None,
RandomState::default(),
None,
EvictionPolicy::default(),
None,
ExpirationPolicy::new(
Some(Duration::from_secs(TTL)),
Expand Down
27 changes: 22 additions & 5 deletions src/future/builder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ use super::{Cache, FutureExt};
use crate::{
common::{builder_utils, concurrent::Weigher},
notification::{AsyncEvictionListener, ListenerFuture, RemovalCause},
policy::ExpirationPolicy,
policy::{EvictionPolicy, ExpirationPolicy},
Expiry,
};

Expand Down Expand Up @@ -60,6 +60,7 @@ pub struct CacheBuilder<K, V, C> {
max_capacity: Option<u64>,
initial_capacity: Option<usize>,
weigher: Option<Weigher<K, V>>,
eviction_policy: EvictionPolicy,
eviction_listener: Option<AsyncEvictionListener<K, V>>,
expiration_policy: ExpirationPolicy<K, V>,
invalidator_enabled: bool,
Expand All @@ -77,6 +78,7 @@ where
max_capacity: None,
initial_capacity: None,
weigher: None,
eviction_policy: EvictionPolicy::default(),
eviction_listener: None,
expiration_policy: ExpirationPolicy::default(),
invalidator_enabled: false,
Expand Down Expand Up @@ -116,6 +118,7 @@ where
self.initial_capacity,
build_hasher,
self.weigher,
self.eviction_policy,
self.eviction_listener,
self.expiration_policy,
self.invalidator_enabled,
Expand Down Expand Up @@ -212,6 +215,7 @@ where
self.initial_capacity,
hasher,
self.weigher,
self.eviction_policy,
self.eviction_listener,
self.expiration_policy,
self.invalidator_enabled,
Expand Down Expand Up @@ -245,6 +249,19 @@ impl<K, V, C> CacheBuilder<K, V, C> {
}
}

/// Sets the eviction (and admission) policy of the cache.
///
/// The default policy is TinyLFU. See [`EvictionPolicy`][eviction-policy] for
/// more details.
///
/// [eviction-policy]: ../policy/struct.EvictionPolicy.html
pub fn eviction_policy(self, policy: EvictionPolicy) -> Self {
Self {
eviction_policy: policy,
..self
}
}

/// Sets the weigher closure to the cache.
///
/// The closure should take `&K` and `&V` as the arguments and returns a `u32`
Expand Down Expand Up @@ -276,8 +293,8 @@ impl<K, V, C> CacheBuilder<K, V, C> {
///
/// It is very important to make the listener closure not to panic. Otherwise,
/// the cache will stop calling the listener after a panic. This is an intended
/// behavior because the cache cannot know whether is is memory safe or not to
/// call the panicked lister again.
/// behavior because the cache cannot know whether it is memory safe or not to
/// call the panicked listener again.
///
/// [removal-cause]: ../notification/enum.RemovalCause.html
/// [example]: ./struct.Cache.html#per-entry-expiration-policy
Expand Down Expand Up @@ -316,8 +333,8 @@ impl<K, V, C> CacheBuilder<K, V, C> {
///
/// It is very important to make the listener closure not to panic. Otherwise,
/// the cache will stop calling the listener after a panic. This is an intended
/// behavior because the cache cannot know whether is is memory safe or not to
/// call the panicked lister again.
/// behavior because the cache cannot know whether it is memory safe or not to
/// call the panicked listener again.
///
/// [removal-cause]: ../notification/enum.RemovalCause.html
/// [listener-future]: ../notification/type.ListenerFuture.html
Expand Down
Loading

0 comments on commit b5560df

Please sign in to comment.