Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: read parquet metadata #3199

Merged
merged 7 commits into from
Jan 21, 2024

Conversation

QuenKar
Copy link
Contributor

@QuenKar QuenKar commented Jan 19, 2024

I hereby agree to the terms of the GreptimeDB CLA

What's changed and what's your intention?

  1. introduce MetadataLoader
  2. using opendal read_with and range to read parquet metadata instead of using reader.

Checklist

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.
  • This PR does not require documentation updates.

Refer to a related PR or issue link (optional)

Closes #3191

@github-actions github-actions bot added docs-not-required This change does not impact docs. Size: M labels Jan 19, 2024
Copy link

codecov bot commented Jan 19, 2024

Codecov Report

Attention: 37 lines in your changes are missing coverage. Please review.

Comparison is base (2e4c48a) 85.78% compared to head (0284ca4) 85.33%.
Report is 3 commits behind head on main.

❗ Current head 0284ca4 differs from pull request most recent head a8c347b. Consider uploading reports for the commit a8c347b to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3199      +/-   ##
==========================================
- Coverage   85.78%   85.33%   -0.46%     
==========================================
  Files         833      834       +1     
  Lines      136802   137192     +390     
==========================================
- Hits       117362   117076     -286     
- Misses      19440    20116     +676     

@QuenKar QuenKar requested review from evenyag and zhongzc January 19, 2024 06:27
src/mito2/src/sst/parquet/metadata.rs Outdated Show resolved Hide resolved
src/mito2/src/sst/parquet/metadata.rs Outdated Show resolved Hide resolved
src/mito2/src/sst/parquet/metadata.rs Outdated Show resolved Hide resolved
src/mito2/src/sst/parquet/metadata.rs Outdated Show resolved Hide resolved
src/mito2/src/sst/parquet/metadata.rs Outdated Show resolved Hide resolved
QuenKar and others added 2 commits January 19, 2024 18:38
Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>
@evenyag evenyag added this pull request to the merge queue Jan 21, 2024
Merged via the queue into GreptimeTeam:main with commit e5a8831 Jan 21, 2024
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs-not-required This change does not impact docs.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Read parquet metadata with opendal range reader
3 participants