-
Notifications
You must be signed in to change notification settings - Fork 332
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: Adds FileHandle to ChunkStream #1255
fix: Adds FileHandle to ChunkStream #1255
Conversation
d84b253
to
721867e
Compare
721867e
to
d7d5910
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do not merge: this is still a work in progress
50a3a13
to
7856eea
Compare
cdcd660
to
1bac539
Compare
Codecov Report
@@ Coverage Diff @@
## develop #1255 +/- ##
===========================================
- Coverage 85.64% 85.61% -0.03%
===========================================
Files 495 495
Lines 73428 73573 +145
===========================================
+ Hits 62887 62991 +104
- Misses 10541 10582 +41 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Skip src/storage/src/region/tests/compact.rs
, other parts look good to me
* test: Add compaction test * test: Test read during compaction * test: Add s3 object store to test * test: only run compact test * feat: Hold file handle in chunk stream * test: check files still exist after compact * feat: Revert changes to develop.yaml * test: Simplify MockPurgeHandler
I hereby agree to the terms of the GreptimeDB CLA
What's changed and what's your intention?
This PR adds the FileHandle to the ChunkStream to avoid the purger purging the file during scan.
The parquet reader actually returns a ChunkStream to scan the file, so we must keep the file handle in the stream
greptimedb/src/storage/src/sst/parquet.rs
Lines 260 to 262 in 0ffa628
To test this patch, this PR also adds a method to trigger compaction.
Checklist
Refer to a related PR or issue link (optional)