Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(flow): impl for MapFilterProject #3359

Merged
merged 14 commits into from
Feb 27, 2024

Conversation

discord9
Copy link
Contributor

@discord9 discord9 commented Feb 22, 2024

I hereby agree to the terms of the GreptimeDB CLA

What's changed and what's your intention?

Added several utility functions for MapFilterProject:

  • Added functions like map, filter and project to map, filter and project to final output from input columns.
  • Added compose to compose two mfp. permute to rename column indices.
  • Added evalutate_into, evaluate_iter & evaluate_inner for SafeMfpPlan to suit different cases for eval mfp.
  • Added create_from and evaluate for MfpPlan which support temproal filter(basically just added some record and remove them after given time)

limitations of the current code (optional):
Still far from a complete proof of concept code, lacking reduce operator's code and render dataflow's graph's code

Checklist

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.
  • This PR does not require documentation updates.

Refer to a related PR or issue link (optional)

@github-actions github-actions bot added the docs-not-required This change does not impact docs. label Feb 22, 2024
@discord9 discord9 marked this pull request as ready for review February 23, 2024 03:43
@zhongzc zhongzc self-requested a review February 23, 2024 03:46
@discord9 discord9 requested a review from waynexia February 23, 2024 06:52
src/flow/src/expr/linear.rs Outdated Show resolved Hide resolved
src/flow/src/expr/linear.rs Outdated Show resolved Hide resolved
Copy link

codecov bot commented Feb 26, 2024

Codecov Report

Attention: Patch coverage is 84.16801% with 98 lines in your changes are missing coverage. Please review.

Project coverage is 84.57%. Comparing base (564fe3b) to head (f014837).
Report is 31 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3359      +/-   ##
==========================================
- Coverage   85.38%   84.57%   -0.81%     
==========================================
  Files         889      893       +4     
  Lines      145476   147899    +2423     
==========================================
+ Hits       124215   125090     +875     
- Misses      21261    22809    +1548     

src/flow/src/expr/linear.rs Outdated Show resolved Hide resolved
src/flow/src/expr/linear.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@killme2008 killme2008 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

src/flow/src/expr/linear.rs Outdated Show resolved Hide resolved
src/flow/src/expr/linear.rs Outdated Show resolved Hide resolved
src/flow/src/expr/scalar.rs Outdated Show resolved Hide resolved
src/flow/src/expr/scalar.rs Outdated Show resolved Hide resolved
@killme2008 killme2008 added this pull request to the merge queue Feb 27, 2024
Merged via the queue into GreptimeTeam:main with commit dbb1ce1 Feb 27, 2024
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs-not-required This change does not impact docs.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants