Add support for Immediate notification delivery mode to future cache

[tatsuya6502]

Split from #145 (comment).

I implemented the blocking and non-blocking modes to the sync caches, and implemented only non-blocking mode to the future cache. To configure an eviction listener with a mode, I created an enum DeliveryMode with two variants Immediate, meaning blocking, and Queued, meaning non-blocking.

I could not implement non-worker threads version of the immediate/queued modes just because lack of time. So currently both rely on the worker threads.

• This issue will cover the Immediate delivery mode in the first paragraph above.
• I will open separate issues to implement the "non-worker threads version" in the second paragraph.
  • (I am thinking to remove the worker threads completely from moka instead of having both worker-threads/non-worker-threads versions")

I started some experiments. It seems implementing Immediate mode to future cache will require the following API changes:

• The get method will become an async fn.
• The blocking().insert(..) and blocking().invalidate(..) methods will be removed.
  • We cannot provide it without relying on a specific async runtime (Tokio, async-std, or actix-rt, etc).
• The return type of the event listener closure will be changed from () to ListenerFuture. which is Pin<Box<dyn Future<Output = ()> + Send>>:
  • So, the new signature of the event listener closure will be:
    • impl Fn(Arc<K>, V, RemovalCause) -> ListenerFuture + Send + Sync + 'static

[tatsuya6502]

I started some experiments in a separate repository. Here is the diff from v0.10.0 tag:

• https://gitlab.com/moka-labs/moka-async-runtimes/moka-experiments/-/compare/v0.10.0...async-runtimes

Note that some features are not working now for the future cache:

• Queued notification delivery mode is temporary(?) removed.
• invalidate_entries_if is temporary removed.
• The get method is still a regular fn (not async fn), and does not perform the housekeeping tasks.

Currently, the diff contains +3,680 additions and 499 deletions! Most of the additions are duplicate codes to sync cache. This change requires many internal functions to become async fn, so that it can await on future-aware lock etc.

I will do some refactoring to remove some duplicate codes between future and sync caches.

[tatsuya6502]

I started to work on this task again for v0.12.0 release:

• Remove the thread pool from future::Cache #294

[tatsuya6502]

On Feb 13, 2023, tatsuya6502 wrote:

I started some experiments in a separate repository. Here is the diff from v0.10.0 tag:

• https://gitlab.com/moka-labs/moka-async-runtimes/moka-experiments/-/compare/v0.10.0...async-runtimes

Note that some features are not working now for the future cache:

• Queued notification delivery mode is temporary(?) removed.
• invalidate_entries_if is temporary removed.
• The get method is still a regular fn (not async fn), and does not perform the housekeeping tasks.

• v0.12.0 will not have Queued notification delivery mode for future::Cache.
  • If anybody asks the Queued mode back, we will add it in a future release.
• invalidate_entries_if is added back.
  • It was already done in February 2023 in the above branch.
• The get method is now async fn, so it does perform the housekeeping tasks.
  • It was already done in February 2023 in the above branch.

[tatsuya6502]

Closing as completed.

For v0.12.0, we added changed the notification delivery mode for the future cache to the immediate mode:

• Remove the thread pool from future::Cache #294

We removed the queued delivery mode from all cache types (both future and sync) as we removed the thread pool completely from moka crate.