Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Use dispatch_to_series for combine_const #22751

Closed
wants to merge 6 commits into from

Conversation

jbrockmendel
Copy link
Member

This and #22019 get rid of the only two usages of BlockManager.eval and Block.eval. Getting rid of those will be a good day.

This is pretty ugly at the moment. There are some design issues (namely, what broadcasting rules are intended to be supported) that need to be cleared up before this can be made pretty.

@pep8speaks
Copy link

pep8speaks commented Sep 18, 2018

Hello @jbrockmendel! Thanks for updating the PR.

Line 4927:15: W504 line break after binary operator
Line 4936:15: W504 line break after binary operator
Line 4937:15: W504 line break after binary operator
Line 4954:15: W504 line break after binary operator

Line 1727:11: W504 line break after binary operator
Line 1728:11: W504 line break after binary operator

Comment last updated on October 03, 2018 at 21:58 Hours UTC

return ops.dispatch_to_series(self, right, func)

elif (np.ndim(other) == 1 and
isinstance(other, (tuple, np.ndarray)) and
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a test for that goes through this path with a tuple, but none with a list. IIRC, no tests fail if list is included here.

Similarly, in the case above, only np.ndarray cases are tested. Should tuple and/or list be allowed?

return ops.dispatch_to_series(self, other, func)

elif (np.ndim(other) == 2 and isinstance(other, np.ndarray) and
other.shape[0] == 1 and other.shape[1] == len(self.columns)):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the transposed case be supported?

@jbrockmendel
Copy link
Member Author

Likely the easiest way to address the inconsistencies (and ugliness that goes with it) would be to call _align_method_FRAME in ops._comp_method_FRAME. It is called in the other two FRAME functions but not that one.

@codecov
Copy link

codecov bot commented Sep 20, 2018

Codecov Report

Merging #22751 into master will decrease coverage by 0.04%.
The diff coverage is 75%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #22751      +/-   ##
==========================================
- Coverage   92.18%   92.14%   -0.05%     
==========================================
  Files         169      169              
  Lines       50823    50805      -18     
==========================================
- Hits        46853    46816      -37     
- Misses       3970     3989      +19
Flag Coverage Δ
#multiple 90.56% <75%> (-0.05%) ⬇️
#single 42.3% <0%> (-0.06%) ⬇️
Impacted Files Coverage Δ
pandas/core/ops.py 96.68% <50%> (-0.75%) ⬇️
pandas/core/frame.py 97.02% <81.81%> (-0.19%) ⬇️
pandas/core/arrays/timedeltas.py 91.54% <0%> (-1%) ⬇️
pandas/core/groupby/base.py 91.11% <0%> (-0.73%) ⬇️
pandas/io/parquet.py 73.04% <0%> (-0.69%) ⬇️
pandas/plotting/_compat.py 90.9% <0%> (-0.4%) ⬇️
pandas/core/internals/blocks.py 93.17% <0%> (-0.31%) ⬇️
pandas/core/arrays/interval.py 92.55% <0%> (-0.26%) ⬇️
pandas/core/resample.py 96.78% <0%> (-0.21%) ⬇️
... and 33 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ee80803...914fd80. Read the comment docs.

@gfyoung gfyoung added Internals Related to non-user accessible pandas implementation Clean labels Sep 23, 2018
@jorisvandenbossche
Copy link
Member

I am personally a bit hesitant to do much refactoring on something rather fragile as broadcasting behaviour without some better testing/documenting of the actual/desired broadcasting behaviour (but it might be a good opportunity to actually do that!).

@jbrockmendel
Copy link
Member Author

I am personally a bit hesitant to do much refactoring on something rather fragile as broadcasting

I definitely agree. In case it is not clear, this PR is not intended to be merged in its present form. The idea behind the ugly, verbose implementation here is to show precisely what behavior is currently tested. The discussion I'm hoping for is to clarify what the "spec" is supposed to be so that can be documented/tested and then implemented cleanly.

@jorisvandenbossche
Copy link
Member

OK, good to hear.

The idea behind the ugly, verbose implementation here is to show precisely what behavior is currently tested. The discussion I'm hoping for is to clarify what the "spec" is supposed to be so that can be documented/tested and then implemented cleanly.

I think it would be a good idea to try to translate the 'ugly code' you wrote now to a written explanation of the broadcasting rules.

@jbrockmendel jbrockmendel deleted the mops4 branch October 13, 2018 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Clean Internals Related to non-user accessible pandas implementation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants