Remove the thread pool from sync caches

.cirrus.yml → .cirrus.yml-DISABLED

@@ -66,12 +66,6 @@ linux_arm64_task:
     # Run tests (release, sync feature)
     - cargo test -j 1 --release --features sync -- --test-threads=$NUM_CPUS
 
-    # Run tests (release, sync feature, thread-pool test for sync::Cache)
-    - cargo test --release --lib --features sync sync::cache::tests::enabling_and_disabling_thread_pools -- --exact --ignored
-
-    # Run tests (release, sync feature, thread-pool test for sync::SegmentedCache)
-    - cargo test --release --lib --features sync sync::segment::tests::enabling_and_disabling_thread_pools -- --exact --ignored
-
     # Run tests (sync feature, key lock test for notification)
     - cargo test --release --lib --features sync sync::cache::tests::test_key_lock_used_by_immediate_removal_notifications -- --exact --ignored
 

.github/workflows/CI.yml

@@ -74,7 +74,7 @@ jobs:
         uses: actions-rs/cargo@v1
         with:
           command: tree
-          args: --features future
+          args: --features 'sync, future'
 
       - name: Run tests (debug, sync feature)
         uses: actions-rs/cargo@v1
@@ -92,18 +92,6 @@ jobs:
         env:
           RUSTFLAGS: '--cfg rustver'
 
-      - name: Run tests (sync feature, thread-pool test for sync::Cache)
-        uses: actions-rs/cargo@v1
-        with:
-          command: test
-          args: --release --lib --features sync sync::cache::tests::enabling_and_disabling_thread_pools -- --exact --ignored
-
-      - name: Run tests (sync feature, thread-pool test for sync::SegmentedCache)
-        uses: actions-rs/cargo@v1
-        with:
-          command: test
-          args: --release --lib --features sync sync::segment::tests::enabling_and_disabling_thread_pools -- --exact --ignored
-
       - name: Run tests (sync feature, key lock test for notification)
         uses: actions-rs/cargo@v1
         with:

.github/workflows/Kani.yml

@@ -31,3 +31,5 @@ jobs:
 
       - name: Run Kani
         uses: model-checking/kani-github-action@v0.28
+        with:
+          args: --features 'sync, future'

.github/workflows/Miri.yml

@@ -44,10 +44,10 @@ jobs:
         uses: actions-rs/cargo@v1
         with:
           command: miri
-          args: test deque
+          args: test deque --features 'sync, future'
 
       - name: Run Miri test (timer_wheel)
         uses: actions-rs/cargo@v1
         with:
           command: miri
-          args: test timer_wheel
+          args: test deque --features 'sync, future'

.github/workflows/Skeptic.yml

@@ -38,11 +38,11 @@ jobs:
         with:
           command: clean
 
-      - name: Run tests (no features)
+      - name: Run tests (sync feature)
         uses: actions-rs/cargo@v1
         with:
           command: test
-          args: --release
+          args: --release --features sync
         env:
           RUSTFLAGS: '--cfg skeptic'
 

CHANGELOG.md

@@ -2,28 +2,33 @@
 
 ## Version 0.12.0 (Currently Beta)
 
-**IMPORTANT**: This release has major breaking changes.
-
-- `future::Cache`
-    - The thread pool was removed from `future::Cache`. It no longer spawns
-      background threads.
-    - The `notification::DeliveryMode` for eviction listener was changed from
-      `Queued` to `Immediate`.
-    - To support these changes, some of the APIs were changed. Please see the
-      [MIGRATION-GUIDE.md](./MIGRATION-GUIDE.md#migrating-to-v0120-from-a-prior-version)
-      for more details.
-- `sync::Cache` and `sync::SegmentedCache`
-    - As of 0.12.0-beta.1, no breaking changes have been made to these caches.
-    - However, the future beta releases will have the following changes:
-        - (Not in 0.12.0-beta.1) `sync` caches will be no longer enabled by default.
-          Use a crate feature `sync` to enable it.
-        - (Not in 0.12.0-beta.1) The thread pool will be disabled by default.
+**ATTENTION**: `v0.12.0` has major breaking changes on the API and internal behavior.
+Please read the [MIGRATION-GUIDE.md][migration-guide-v012] for the details.
+
+**`sync` caches are no longer enabled by default**: Please use a crate feature `sync`
+to enable it.
+
+**No more background threads**: All cache types `future::Cache`, `sync::Cache`, and
+`sync::SegmentedCache` no longer spawn background threads.
+
+- The `scheduled-thread-pool` crate was removed from the dependency.
+- Because of this change, many external and internal methods of `future::Cache`
+  were converted to `async` methods. You may need to add `.await` to your code for
+  those methods.
+
+**Immediate notification delivery**: The `notification::DeliveryMode` enum for the
+eviction listener was removed. Now all cache types work as if the `Immediate`
+delivery mode is specified.
+
+[migration-guide-v012]: ./MIGRATION-GUIDE.md#migrating-to-v0120-from-a-prior-version
 
 ### Changed
 
-- Remove the thread pool from `future::Cache`. ([#294][gh-pull-0294])
+- Remove the thread pool from `sync` caches. ([#316][gh-pull-0316])
+- Remove the thread pool from `future` cache`. ([#294][gh-pull-0294])
 - Add support for `Immediate` notification delivery mode to future cache.
  ([#228][gh-issue-0228])
+- Improve async cancellation safety of `future::Cache`. ([#309][gh-pull-0309])
 
 
 ## Version 0.11.3
@@ -712,6 +717,8 @@ The minimum supported Rust version (MSRV) is now 1.51.0 (Mar 25, 2021).
 [gh-issue-0034]: https://github.com/moka-rs/moka/issues/34/
 [gh-issue-0031]: https://github.com/moka-rs/moka/issues/31/
 
+[gh-pull-0316]: https://github.com/moka-rs/moka/pull/316/
+[gh-pull-0309]: https://github.com/moka-rs/moka/pull/309/
 [gh-pull-0295]: https://github.com/moka-rs/moka/pull/295/
 [gh-pull-0294]: https://github.com/moka-rs/moka/pull/294/
 [gh-pull-0277]: https://github.com/moka-rs/moka/pull/277/

Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "moka"
-version = "0.12.0-beta.1"
+version = "0.12.0-beta.2"
 edition = "2018"
 # Rust 1.65 was released on Nov 3, 2022.
 rust-version = "1.65"
@@ -16,11 +16,10 @@ exclude = [".circleci", ".devcontainer", ".github", ".gitpod.yml", ".vscode"]
 build = "build.rs"
 
 [features]
-default = ["sync", "atomic64", "quanta"]
+default = ["atomic64", "quanta"]
 
-# This feature is enabled by default. Disable it when you do not need
-# `moka::sync::{Cache, SegmentedCache}`
-sync = ["scheduled-thread-pool"]
+# Enable this feature to use `moka::sync::{Cache, SegmentedCache}`
+sync = []
 
 # Enable this feature to use `moka::future::Cache`.
 future = ["async-lock", "async-trait", "futures-util"]
@@ -64,9 +63,6 @@ triomphe = { version = "0.1.3", default-features = false }
 # Optional dependencies (enabled by default)
 quanta = { version = "0.11.0", optional = true }
 
-# Optional dependencies (sync)
-scheduled-thread-pool = { version = "0.2.7", optional = true }
-
 # Optional dependencies (future)
 async-lock = { version = "2.4", optional = true }
 async-trait = { version = "0.1.58", optional = true }

MIGRATION-GUIDE.md

@@ -5,43 +5,95 @@
 v0.12.0 has major breaking changes on the API and internal behavior. This section
 describes the code changes required to migrate to v0.12.0.
 
-### `future::Cache`
-
-- The thread pool was removed from `future::Cache`. The background threads are no
-  longer spawned.
-- The `notification::DeliveryMode` for eviction listener was changed from `Queued` to
-  `Immediate`.
-
-To support these changes, the following API changes were made:
-
-1. `future::Cache::get` method is now `async fn`, so you must `await` for the result.
-2. `future::Cache::blocking` method was removed.
+### Highlights v0.12
+
+**`sync` caches are no longer enabled by default**: Please use a crate feature `sync`
+to enable it.
+
+**No more background threads**: All cache types `future::Cache`, `sync::Cache`, and
+`sync::SegmentedCache` no longer spawn background threads.
+
+- The `scheduled-thread-pool` crate was removed from the dependency.
+- Because of this change, many external and internal methods of `future::Cache`
+  were converted to `async` methods. You may need to add `.await` to your code for
+  those methods.
+
+**Immediate notification delivery**: The `notification::DeliveryMode` enum for the
+eviction listener was removed. Now all cache types work as if the `Immediate`
+delivery mode is specified.
+
+- It had two variants `Immediate` and `Queued`.
+    - The former should be easier to use than other as it guarantees to preserve the
+      order of events on a given cache key.
+    - The latter would provide higher performance under heavy cache writes.
+- Now all cache types work as if the `Immediate` mode is specified.
+    - **`future::Cache`**: In earlier versions of `future::Cache`, the queued mode
+      was used. Now it works as if the immediate mode is specified.
+    - **`sync` caches**: From earlier versions of `sync::Cache` and
+      `sync::SegmentedCache`, the immediate mode is the default mode. So this change
+      should only affects those of you who are explicitly using the queued mode.
+- The queued mode was implemented by using a background thread. The queue mode was
+  removed because there is no thread pool available anymore.
+- If you need the queue mode back, please file a GitHub issue. We could provide
+  a way to use a user supplied thread pool.
+
+The following sections will describe about the changes you might need to make to your
+code.
+
+- [`sync::Cache` and `sync::SegmentedCache`](#synccache-and-syncsegmentedcache-v012)
+- [`future::Cache`](#futurecache-v012)
+- [The maintenance tasks](#the-maintenance-tasks)
+
+### `sync::Cache` and `sync::SegmentedCache` v0.12
+
+1. Please use a crate feature `sync` to enable `sync` caches.
+2. Since the background threads are removed, the maintenance tasks such as removing
+   expired entries are not executed periodically anymore.
+   - The `thread_pool_enabled` in `sync::CacheBuilder` was removed. The thread pool
+     is always disabled.
+   - See the [maintenance tasks](#the-maintenance-tasks) section for more details.
+3. The `sync` method of the `sync::ConcurrentCacheExt` trait was moved to
+   `sync::Cache` and `sync::SegmentedCache` types and renamed to `run_pending_tasks`.
+4. Now `sync` caches always work as if the immediate delivery mode is specified
+   for the eviction listener.
+   - In older versions, the immediate mode was the default mode. And the queued
+     mode can be optionally selected.
+
+### `future::Cache` v0.12
+
+#### API changes
+
+1. `get` method is now `async fn`, so you must `await` for the result.
+2. `blocking` method was removed.
    - Please use async runtime's blocking API instead.
-   - See [Replacing the blocking API](#replacing-the-blocking-api) for more details.
+   - See the [replacing the blocking API](#replacing-the-blocking-api) section for
+     more details.
 3. Now `or_insert_with_if` method of the entry API requires `Send` bound for the
    `replace_if` closure.
 4. `eviction_listener_with_queued_delivery_mode` method of `future::CacheBuilder` was
    removed.
-   - Please use one of the new methods `eviction_listener` or
-     `async_eviction_listener` instead.
-   - See [Updating the eviction listener](#updating-the-eviction-listener) for more
-     details.
-5. `future::ConcurrentCacheExt::sync` method was renamed to
-   `future::Cache::run_pending_tasks`. It was also changed to `async fn`.
-
-The following internal behavior changes were made:
-
-1. Maintenance tasks such as removing expired entries are not executed periodically
-   anymore.
-   - See [Maintenance tasks](#maintenance-tasks) for more details.
-2. Now `future::Cache` only supports `Immediate` delivery mode for eviction listener.
-   - In older versions, only `Queued` delivery mode was supported.
-   - If you need `Queued` delivery mode back, please file an issue.
+   - Please use one of the new methods instead:
+     - `eviction_listener`
+     - `async_eviction_listener`
+   - See the [updating the eviction listener](#updating-the-eviction-listener)
+     section for more details.
+5. The `sync` method of the `future::ConcurrentCacheExt` trait was moved to
+   `future::Cache` type and renamed to `run_pending_tasks`. It was also changed to
+   `async fn`.
+
+#### Behavior changes
+
+1. Since the background threads are removed, the maintenance tasks such as removing
+   expired entries are not executed periodically anymore.
+   - See the [maintenance tasks](#the-maintenance-tasks) section for more details.
+2. Now `future::Cache` always works as if the immediate delivery mode is specified
+   for eviction listener.
+   - In older versions, the queued delivery mode was used.
 
 #### Replacing the blocking API
 
-`future::Cache::blocking` method was removed. Please use async runtime's blocking API
-instead.
+The `blocking` method of `future::Cache` was removed. Please use async runtime's
+blocking API instead.
 
 **Tokio**
 
@@ -145,7 +197,7 @@ let cache = Cache::builder()
 ##### `async_eviction_listener` method
 
 `async_eviction_listener` takes a closure that returns a `Future`. If you need to
-`.await` something in the eviction listener, use this method. The actual return type
+`await` something in the eviction listener, use this method. The actual return type
 of the closure is `notification::ListenerFuture`, which is a type alias of
 `Pin<Box<dyn Future<Output = ()> + Send>>`. You can use the `boxed` method of
 `future::FutureExt` trait to convert a regular `Future` into this type.
@@ -191,11 +243,11 @@ let cache = Cache::builder()
 
 [listener-ex2]: https://docs.rs/moka/latest/moka/future/struct.Cache.html#example-eviction-listener
 
-#### Maintenance tasks
+### The maintenance tasks
 
 In older versions, the maintenance tasks needed by the cache were periodically
-executed in background by a global thread pool managed by `moka`. Now `future::Cache`
-does not use the thread pool anymore, so those maintenance tasks are executed
+executed in background by a global thread pool managed by `moka`. Now all cache types
+do not use the thread pool anymore, so those maintenance tasks are executed
 _sometimes_ in foreground when certain cache methods (`get`, `get_with`, `insert`,
 etc.) are called by user code.
 
@@ -206,46 +258,41 @@ Figure 1. The lifecycle of cached entries
 These maintenance tasks include:
 
 1. Determine whether to admit a "temporary admitted" entry or not.
-2. Apply the recording of cache reads and writes to the internal data structures,
-   such as LFU filter, LRU queues, and timer wheels.
-3. When cache's max capacity is exceeded, select existing entries to evict and remove
-   them from cache.
+2. Apply the recording of cache reads and writes to the internal data structures for
+   the cache policies, such as the LFU filter, LRU queues, and hierarchical timer
+   wheels.
+3. When cache's max capacity is exceeded, remove least recently used (LRU) entries.
 4. Remove expired entries.
-5. Remove entries that have been invalidated by `invalidate_all` or
+5. Find and remove the entries that have been invalidated by `invalidate_all` or
    `invalidate_entries_if` methods.
 6. Deliver removal notifications to the eviction listener. (Call the eviction
-   listener closure with the information about evicted entry)
+   listener closure with the information about the evicted entry)
+
+They will be executed in the following cache methods when one of the following
+conditions is met:
 
-They will be executed in the following cache methods when necessary:
+Cache Methods:
 
 - All cache write methods: `insert`, `get_with`, `invalidate`, etc.
 - Some of the cache read methods: `get`
 - `run_pending_tasks` method, which executes the pending maintenance tasks
   explicitly.
 
-Although expired entries will not be removed until the pending maintenance tasks are
-executed, they will not be returned by cache read methods such as `get`, `get_with`
-and `contains_key`. So unless you need to remove expired entries immediately (e.g. to
-free some memory), you do not need to call `run_pending_tasks` method.
+Conditions:
 
-### `sync::Cache` and `sync::SegmentedCache`
+- When one of the numbers of pending read and write recordings exceeds the threshold.
+    - It is currently hard-coded to 64 items.
+- When the time since the last execution of the maintenance tasks exceeds the
+  threshold.
+    - It is currently hard-coded to 300 milliseconds.
 
-1. (Not in v0.12.0-beta.1) `sync` caches will be no longer enabled by default. Use a
-   crate feature `sync` to enable it.
-2. (Not in v0.12.0-beta.1) The thread pool will be disabled by default.
-   - In older versions, the thread pool was used to execute maintenance tasks in
-     background.
-   - When disabled:
-      - those maintenance tasks are executed _sometimes_ in foreground when certain
-        cache methods (`get`, `get_with`, `insert`, etc.) are called by user code
-      -  See [Maintenance tasks](#maintenance-tasks) for more details.
-   - To enable it, see [Enabling the thread pool](#enabling-the-thread-pool) for more
-     details.
 
-#### Enabling the thread pool
+#### `run_pending_tasks` method
 
-To enable the thread pool, do the followings:
+You can execute the pending maintenance tasks explicitly by calling the
+`run_pending_tasks` method. This method is available for all cache types.
 
-- Specify a crate feature `thread-pool`.
-- At the cache creation time, call the `thread_pool_enabled` method of
-  `CacheBuilder`.
+Note that cache read methods such as `get`, `get_with` and `contains_key` never
+return expired entries although they are not removed immediately from the cache when
+they expire. You will not need to call `run_pending_tasks` method to remove expired
+entries unless you want to remove them immediately (e.g. to free some memory).