[Tiered Caching] Indices Request cache stalekey management (opensearc…

…h-project#12625) * Introduce IndicesRequestCacheCleanupManager Signed-off-by: Kiran Prakash <awskiran@amazon.com> * using cleanup mgr to enqueue cleanups Signed-off-by: Kiran Prakash <awskiran@amazon.com> * readability improvements Signed-off-by: Kiran Prakash <awskiran@amazon.com> * update updateCleanupKeyToCountMap on new cache entry Signed-off-by: Kiran Prakash <awskiran@amazon.com> * create IndicesRequestCacheCleanupManager & settings and validators Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Add IRC specific cache cleaner and remove from IndicesService Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Unit Tests Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Update CHANGELOG.md Signed-off-by: Kiran Prakash <awskiran@amazon.com> * move cachecleaner inside mgr Signed-off-by: Kiran Prakash <awskiran@amazon.com> * remove processCleanupKeys Signed-off-by: Kiran Prakash <awskiran@amazon.com> * minor cleanups Signed-off-by: Kiran Prakash <awskiran@amazon.com> * add updateCleanupKeyToCountMapOnCacheEviction Signed-off-by: Kiran Prakash <awskiran@amazon.com> * remove locks and make all methods synchronized Signed-off-by: Kiran Prakash <awskiran@amazon.com> * spotless Signed-off-by: Kiran Prakash <awskiran@amazon.com> * updateCleanupKeyToCountMapOnCacheEviction Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Testing Signed-off-by: Kiran Prakash <awskiran@amazon.com> * add Reschedule back to indices service Signed-off-by: Kiran Prakash <awskiran@amazon.com> * rename updateStaleKeysCount to incrementStaleKeysCount Signed-off-by: Kiran Prakash <awskiran@amazon.com> * rename getStaleKeysCountForTesting to getStaleKeysCount Signed-off-by: Kiran Prakash <awskiran@amazon.com> * rename threshold to stalenessThreshold Signed-off-by: Kiran Prakash <awskiran@amazon.com> * check for cleanupKey.entity == null Signed-off-by: Kiran Prakash <awskiran@amazon.com> * use computeIfPresent with keycountmap Signed-off-by: Kiran Prakash <awskiran@amazon.com> * log both staleKeysInCache & Staleness in debug logs Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Use HashMap instead of ConcurrentMap Signed-off-by: Kiran Prakash <awskiran@amazon.com> * Address b/w compatibility Signed-off-by: Kiran Prakash <awskiran@amazon.com> * remove synchronized for updateCleanupKeyToCountMapOnCacheEviction Signed-off-by: Kiran Prakash <awskiran@amazon.com> * make cleanCache synchronized Signed-off-by: Kiran Prakash <awskiran@amazon.com> * remove shouldRemoveKey Signed-off-by: Kiran Prakash <awskiran@amazon.com> * spotlessApply Signed-off-by: Kiran Prakash <awskiran@amazon.com> --------- Signed-off-by: Kiran Prakash <awskiran@amazon.com> (cherry picked from commit b4da802)
kiranprakash154 · Mar 17, 2024 · 6697fec · 6697fec
1 parent 5c5be46
commit 6697fec
Show file tree

Hide file tree

Showing 4 changed files with 800 additions and 66 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -21,6 +21,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 - Remote reindex: Add support for configurable retry mechanism ([#12561](https://github.com/opensearch-project/OpenSearch/pull/12561))
 - Tracing for deep search path ([#12103](https://github.com/opensearch-project/OpenSearch/pull/12103))
 - [Metrics Framework] Adds support for asynchronous gauge metric type. ([#12642](https://github.com/opensearch-project/OpenSearch/issues/12642))
+- [Tiered caching] Add Stale keys Management and CacheCleaner to IndicesRequestCache ([#12625](https://github.com/opensearch-project/OpenSearch/pull/12625))
 
 ### Dependencies
 - Bump `com.squareup.okio:okio` from 3.7.0 to 3.8.0 ([#12290](https://github.com/opensearch-project/OpenSearch/pull/12290))
@@ -65,7 +66,6 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 - Fix get task API does not refresh resource stats ([#11531](https://github.com/opensearch-project/OpenSearch/pull/11531))
 - Fix for deserilization bug in weighted round-robin metadata ([#11679](https://github.com/opensearch-project/OpenSearch/pull/11679))
 - onShardResult and onShardFailure are executed on one shard causes opensearch jvm crashed ([#12158](https://github.com/opensearch-project/OpenSearch/pull/12158))
-- Avoid overflow when sorting missing last on `epoch_millis` datetime field ([#12676](https://github.com/opensearch-project/OpenSearch/pull/12676))
 
 ### Security