Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parquet: Add union method to RowSelection #6307

Closed
sdd opened this issue Aug 25, 2024 · 1 comment · Fixed by #6308
Closed

Parquet: Add union method to RowSelection #6307

sdd opened this issue Aug 25, 2024 · 1 comment · Fixed by #6308
Labels
enhancement Any new improvement worthy of a entry in the changelog parquet Changes to the parquet crate

Comments

@sdd
Copy link
Contributor

sdd commented Aug 25, 2024

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

RowSelection::intersection already exists for computing the intersection of two RowSelections. This is useful when you have a filter predicate that filters on two columns that have been AND'ed together and need the intersection of the RowSelection of each side of the AND.

Correspondingly, if you have a filter that is A OR B rather than A AND B, it is desirable to derive the union of the row selections of A and B.

Therefore I propose that RowSelection::union be added as a feature, to complement RowSelection::intersection.

Describe the solution you'd like

#3003 introduced RowSelection::intersection, for which #3047 was submitted and merged.

I will shortly be submitting a similar PR for RowSelection::union.

Describe alternatives you've considered

None

Additional context

I have a PR open on iceberg-rust, where I am using RowSelection::intersection but needed to implement a RowSelection union function myself. I think it makes more sense for it to live in here alongside intersection where others can make use of it too.

@sdd sdd added the enhancement Any new improvement worthy of a entry in the changelog label Aug 25, 2024
@sdd sdd changed the title Add union method to RowSelection Parquet: Add union method to RowSelection Aug 25, 2024
@alamb alamb added the parquet Changes to the parquet crate label Aug 31, 2024
@alamb
Copy link
Contributor

alamb commented Aug 31, 2024

label_issue.py automatically added labels {'parquet'} from #6308

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Any new improvement worthy of a entry in the changelog parquet Changes to the parquet crate
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants