Groupby excludes groups with nan entry in group-defining columns

[soerenwolfers]

Code Sample, a copy-pastable example if possible

import pandas as pd
import numpy as np
df = pd.DataFrame({'ind':[np.nan,1],'col':[0,1]}).groupby('ind').sum()
df

results in

     col
ind     
1.0    1

Problem description

Groupby ignores groups that contain a np.nan in any group-defining column.

I'd understand (well, not really, but it would be slightly more sane) if groups were ignored that had np.nans in grouped columns, but it doesn't make any sense to me to ignore groups that have np.nans in group-defining columns.

Until this is fixed properly, does anyone have a quick and dirty fix for me?

Expected Output

     col
ind     
NaN    0
1.0    1

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.6.6.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor:
byteorder: little
LC_ALL: None
LANG: en_GB.UTF-8
LOCALE: None.None

pandas: 0.24.2
pytest: None
pip: 19.0.3
setuptools: 41.0.1
Cython: 0.28.4
numpy: 1.15.4
scipy: 1.2.1
pyarrow: 0.14.0
xarray: None
IPython: 7.5.0
sphinx: 1.7.5
patsy: 0.5.0
dateutil: 2.8.0
pytz: 2018.5
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 2.2.2
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml.etree: None
bs4: None
html5lib: 1.0.1
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: 0.3.2
pandas_gbq: None
pandas_datareader: None
gcsfs: None

[TomAugspurger]

Thanks. Duplicate of #3729 I believe.

[itamarst]

This still happens in 1.2.4 and master as of May 2021. It should probably be reopened. I will try to implement a fix.

[itamarst]

Ah, no, it is fixed, just need to do dropna=False in the groupby().