Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need to test DataFrame.product with small DataFrames #1464

Closed
gshimansky opened this issue May 19, 2020 · 1 comment · Fixed by #1827
Closed

Need to test DataFrame.product with small DataFrames #1464

gshimansky opened this issue May 19, 2020 · 1 comment · Fixed by #1827
Assignees
Labels
bug 🦗 Something isn't working pandas 🤔 Weird Behaviors of Pandas
Milestone

Comments

@gshimansky
Copy link
Collaborator

System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04):

Ubuntu 19.10

  • Modin version (modin.__version__):

0.7.3+42.gc78828e.dirty

  • Python version:

Python 3.7.5

  • Code we can use to reproduce:
diff --git a/modin/pandas/test/utils.py b/modin/pandas/test/utils.py
index 1faaf38..d4d257c 100644
--- a/modin/pandas/test/utils.py
+++ b/modin/pandas/test/utils.py
@@ -63,6 +63,11 @@ test_data = {
         ]
         for i in range(NCOLS)
     },
+    "small": {
+        "col0": [1, 2, 3, 4],
+        "col1": [8.0, 9.4, 10.1, 11.3],
+        "col2": [4, 5, 6, 7],
+    },
     # "int_float_object_data": {
     #     "col3": [1, 2, 3, 4],
     #     "col4": [4, 5, 6, 7],

Describe the problem

When dataframe size is small and is smaller than min_count parameter value for product it causes a lot of tests to fail. Pandas behaves quite strangely and Modin works differently.

Source code / logs

@dchigarev
Copy link
Collaborator

I've found out that pandas have a strange output for product and sum only if numeric_only parameter specified or skipna=False, so I've changed modin behavior to it mirrors pandas in #1827 and also added test cases that were suggested by @gshimansky above

dchigarev added a commit to dchigarev/modin that referenced this issue Jul 28, 2020
… fixed

Signed-off-by: Dmitry Chigarev <dmitry.chigarev@intel.com>
anmyachev pushed a commit that referenced this issue Jul 29, 2020
Signed-off-by: Dmitry Chigarev <dmitry.chigarev@intel.com>
aregm pushed a commit to aregm/modin that referenced this issue Sep 16, 2020
… fixed (modin-project#1827)

Signed-off-by: Dmitry Chigarev <dmitry.chigarev@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug 🦗 Something isn't working pandas 🤔 Weird Behaviors of Pandas
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants