Fix runtime stats when cache hit #18503
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
beinan
force-pushed
the
fix_runtime_stats_when_cache_hit
branch
from
6fe43b3 to
c69d301
Compare
JiamingMai
reviewed
Feb 4, 2024
| @@ -927,10 +929,20 @@ private int getPage(PageInfo pageInfo, int pageOffset, int bytesToRead, | |||
| // data read from page store is inconsistent from the metastore | |||
| LOG.error("Failed to read page {}: supposed to read {} bytes, {} bytes actually read", | |||
| pageInfo.getPageId(), bytesToRead, ret); | |||
| target.offset(originOffset); //reset the offset | |||
| //best efforts to delete the corrupted file without acquire the write lock | |||
| deletePage(pageInfo, false); | |||
There was a problem hiding this comment.
Do we need to delete the metadata as well?
There was a problem hiding this comment.
Deleting the metadata here will acquire more locks, which might cause dead lock. So I just delete the page file, and page store will re-cache this page in the next visit.
There was a problem hiding this comment.
I think it's a good call, probably we can have a further improvement in the near future. But just for now, ,et's merge this pr and stop the bleeding of the data corruption issue
JiamingMai
reviewed
Feb 4, 2024
| LOG.error("Data corrupted page {} from pageStore", pageInfo.getPageId(), e); | ||
| target.offset(originOffset); //reset the offset | ||
| //best efforts to delete the corrupted file without acquire the write lock | ||
| deletePage(pageInfo, false); |
There was a problem hiding this comment.
Do we need to delete the metadata as well?
JiamingMai
approved these changes
Feb 4, 2024
|
Hi @beinan , this fix works from our testing. Would it be possible to make it into version 308? |
beinan
force-pushed
the
fix_runtime_stats_when_cache_hit
branch
from
c69d301 to
637364c
Compare
|
alluxio-bot, merge this please. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes are proposed in this pull request?
Fix runtime stats when cache hit
Why are the changes needed?
We already count the runtime stats in local cache manager, but we didn't pass in a proper cache context
Does this PR introduce any user facing changes?
Please list the user-facing changes introduced by your change, including