fix(backend): memory leak in memory caches #14363
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
5 tasks
|
このPRによるapi.jsonの差分 差分はこちら |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #14363 +/- ##
===========================================
- Coverage 39.90% 39.87% -0.03%
===========================================
Files 1547 1547
Lines 190875 190901 +26
Branches 3569 2655 -914
===========================================
- Hits 76166 76123 -43
- Misses 114116 114216 +100
+ Partials 593 562 -31 ☔ View full report in Codecov by Sentry. |
kakkokari-gtyih
changed the title
packages/backend/src/queue/processors/DeliverProcessorService.ts
Outdated
Show resolved
Hide resolved
Co-authored-by: かっこかり <67428053+kakkokari-gtyih@users.noreply.github.com>
|
See also: MisskeyIO#174 MisskeyIO#289 |
syuilo
reviewed
Aug 9, 2024
|
🙏🏻 |
LemonDouble
pushed a commit
to LemonDouble/misskey
that referenced
this pull request
Aug 19, 2024
* encapsulate `MemoryKVCache<T>` * remove infinity caches * encapsulate other caches * add missing awaits to internally synchronize caches * implement pull-through caching * tune cache lifetimes * optimize cache GC by stopping early * summarize changes in CHANGELOG.md * Fix timeout comments Co-authored-by: かっこかり <67428053+kakkokari-gtyih@users.noreply.github.com> * add comments about awaiting the redis write --------- Co-authored-by: かっこかり <67428053+kakkokari-gtyih@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What
This PR resolves #14310 and #14311 by replacing all "infinite" cache lifetimes and cleaning up
MemoryKVCache<T>. Expiration lifetimes of all memory / redis caches have been reset to a reasonable number and normalized between similar caches.Redis caches have a synchronization fix, and an optimization is applied to
MemoryKVCache.gc().Why
The linked issues pose a serious reliability issue that is affecting many instances, including my own. Resolving the memory leaks will improve stability, uptime, and performance.
Fix #14310
Fix #14311
Additional info (optional)
This change is upstreamed from the Sharkey fork. Please see Sharkey MR #580 for additional details and context.
This replaces #14314, which was an earlier attempt at a similar fix.
Checklist