-
Notifications
You must be signed in to change notification settings - Fork 79
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Modify layout of grouping files #4525
Modify layout of grouping files #4525
Conversation
3e67090
to
401114a
Compare
fda131e
to
9535401
Compare
9535401
to
3ce0a5b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Only one actual comment, others are cosmetic.
extensions/parquet/table/src/main/java/io/deephaven/parquet/table/ParquetTools.java
Outdated
Show resolved
Hide resolved
extensions/parquet/table/src/main/java/io/deephaven/parquet/table/ParquetTools.java
Outdated
Show resolved
Hide resolved
extensions/parquet/table/src/main/java/io/deephaven/parquet/table/ParquetTools.java
Outdated
Show resolved
Hide resolved
...nsions/parquet/table/src/test/java/io/deephaven/parquet/table/ParquetTableReadWriteTest.java
Outdated
Show resolved
Hide resolved
extensions/parquet/table/src/main/java/io/deephaven/parquet/table/layout/ParquetFileHelper.java
Outdated
Show resolved
Hide resolved
...nsions/parquet/table/src/test/java/io/deephaven/parquet/table/ParquetTableReadWriteTest.java
Outdated
Show resolved
Hide resolved
75aa4b8
to
66717a8
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks pretty good to me. A few minor comments, and a suggestion for further fallback.
extensions/parquet/table/src/main/java/io/deephaven/parquet/table/layout/ParquetFileHelper.java
Outdated
Show resolved
Hide resolved
final String prefix = minusParquetSuffix(path); | ||
return columnName -> prefix + "_" + columnName + "_grouping.parquet"; | ||
/** | ||
* Generates a grouping file path relative to the table destination file path. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought we were re-branding "grouping" to "indexing" in this PR for new/changed code, to avoid creating a subsequent task?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lbooker42 mentioned that he will be doing renaming in his changes, so I didn't do it here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My complaint was more that we're adding more work with some of the new code, and creating a "moving target" for him.
extensions/parquet/table/src/main/java/io/deephaven/parquet/table/ParquetTools.java
Outdated
Show resolved
Hide resolved
extensions/parquet/table/src/main/java/io/deephaven/parquet/table/ParquetTools.java
Show resolved
Hide resolved
...nsions/parquet/table/src/test/java/io/deephaven/parquet/table/ParquetTableReadWriteTest.java
Show resolved
Hide resolved
...s/parquet/table/src/main/java/io/deephaven/parquet/table/location/ParquetColumnLocation.java
Outdated
Show resolved
Hide resolved
...s/parquet/table/src/main/java/io/deephaven/parquet/table/location/ParquetColumnLocation.java
Outdated
Show resolved
Hide resolved
extensions/parquet/table/src/main/java/io/deephaven/parquet/table/ParquetTools.java
Show resolved
Hide resolved
2582672
to
91502ab
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The renaming of group to index looks good. Considering the code base still supports grouping (and not multi-column indexes) I think this is as far as we should go in this PR.
extensions/parquet/table/src/main/java/io/deephaven/parquet/table/ParquetTools.java
Show resolved
Hide resolved
e50c4b4
to
ec762d3
Compare
ec762d3
to
bbeef06
Compare
Closes #4283
Proposal document for new layout.