Memory cache eviction is inefficient and error-prone

[warriordog]

💡 Summary

Cache eviction from MemoryKVCache<T> is currently based on a hard-coded 3 minute timer. When this timer ticks, the gc() method loops through all entries and removes any that have been evicted. This has a few issues:

1. The 3-minute interval adds considerable jitter to eviction times. Some caches have lifetimes as short as 10 minutes, which translates to a 30% margin of error.
2. The full loop wastes CPU resources on larger caches, like user & public keys.
3. If the cache is not properly disposed, then it creates a memory leak and timer leak.

This can be solved by modifying the implementation to create a new timer per entry. When one of these timers ticks, it removes the associated row without any loops or checks. The timer ID can be stored within the cache entry and automatically cancelled when the entry is deleted. This has three benefits:

1. Eviction timing is much more accurate.
2. The full loop is removed.
3. If the cache is not disposed, then all entries will slowly evict until the cache is empty. Then it will be garbage collected and not produce a leak.

---

This bug was first discovered in the Sharkey fork, but exists in Misskey as well. For additional details, see Sharkey issue #601 and Sharkey MR #580.

Related to #14310.

🥰 Expected Behavior

Cache eviction should be predictable and not consume significant system resources.

🤬 Actual Behavior

Cache eviction is slow and inconsistent.

📝 Steps to Reproduce

Run a federated instance for an extended period of time and wait for caches to fill. The "garbage collection" process will consume a notable amount of CPU time every 3 minutes or so.

💻 Frontend Environment

N/A - this is a backend issue. The frontend environment does not matter.

🛰 Backend Environment (for server admin)

N/A - this effects all instances running a Misskey version newer than March 2021. The backend environment is irrelevant.

Do you want to address this bug yourself?

• Yes, I will patch the bug myself and send a pull request