-
Notifications
You must be signed in to change notification settings - Fork 28.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-48924][PS] Add a pandas-like make_interval
helper function
#47385
[SPARK-48924][PS] Add a pandas-like make_interval
helper function
#47385
Conversation
@@ -130,19 +130,6 @@ def _resamplekey_type(self) -> DataType: | |||
def _agg_columns_scols(self) -> List[Column]: | |||
return [s.spark.column for s in self._agg_columns] | |||
|
|||
def get_make_interval( # type: ignore[return] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just wondering if we skip some units intentionally here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
some units was ignored due to not invoked at that time, I am adding them back to make the helper function cover all pandas units :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great thanks!
LGTM thank you! |
Merged to master. |
### What changes were proposed in this pull request? Add a pandas-like `make_interval` helper function ### Why are the changes needed? factor it out as a helper function to be reusable ### Does this PR introduce _any_ user-facing change? No, internal change only ### How was this patch tested? CI ### Was this patch authored or co-authored using generative AI tooling? No Closes apache#47385 from zhengruifeng/ps_simplify_make_interval. Authored-by: Ruifeng Zheng <ruifengz@apache.org> Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
### What changes were proposed in this pull request? Add a pandas-like `make_interval` helper function ### Why are the changes needed? factor it out as a helper function to be reusable ### Does this PR introduce _any_ user-facing change? No, internal change only ### How was this patch tested? CI ### Was this patch authored or co-authored using generative AI tooling? No Closes apache#47385 from zhengruifeng/ps_simplify_make_interval. Authored-by: Ruifeng Zheng <ruifengz@apache.org> Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
What changes were proposed in this pull request?
Add a pandas-like
make_interval
helper functionWhy are the changes needed?
factor it out as a helper function to be reusable
Does this PR introduce any user-facing change?
No, internal change only
How was this patch tested?
CI
Was this patch authored or co-authored using generative AI tooling?
No