Simple groupby causes TypeError

[ienkovich]

Here is a failing test:

import pandas as pd
import modin.pandas as mpd

data = {
    "a": [1, 1, 2],
    "b": [11, 11, 22],
    "c": [111, 111, 222]
}

df = pd.DataFrame(data)
df = pd.concat([df])
ref = df.groupby(["a", "b", df["c"]]).size()
print(ref)

df = mpd.DataFrame(data)
df = mpd.concat([df])
exp = df.groupby(["a", "b", df["c"]]).size()
print(exp)

The reason of fail is in DataFrame.groupby code. When we see that 'by' is a list, but it doesn't hold column labels only, we compare its length to index length. If lengths match, we assume, that 'by' holds key data, not a list of key columns/Series. In the test above this assumption is wrong.

Also, this length check triggers execution for lazy frame, which we want to avoid. So, we need to first check if 'by' is a list of key columns/Series and check index length only when we have to assume, that 'by' holds key data.

[devin-petersohn]

Is this an issue for OmniSci only? Please choose an appropriate label so we can classify this issue. Thanks!

[ienkovich]

This is a back-end agnostic issue. The problem is in front-end.

[devin-petersohn]

I see, thanks for clarifying. I have added to next release since there is not time to fix for this release.

[ienkovich]

I want to mention in advance, that while #1842 fixes error for this particular test case, it doesn't fix the original problem in the code. We can slightly change groupby statement to reproduce the original problem after #1842 is applied:

df.groupby(["a", "b", df["c"] + 1]).size()

[devin-petersohn]

@ienkovich Good catch, that example is significantly more complicated because the grouping values come from a transformation on the data rather than the data itself. We should track that particular case separately because it will require special care.

[itamarst]

OK, I will open a separate issue for the additional case.