.loc Multiindex DateTime slicing failures

[attack68]

related to #16637 and #14946

Code Sample, a copy-pastable example if possible

import pandas as pd

dt_idx = pd.to_datetime(['2017-05-04','2017-05-05'])
m_idx = pd.MultiIndex.from_product([dt_idx,dt_idx], names=['Idx1','Idx2'])
df = pd.DataFrame(data=[[1,2],[3,4],[5,6],[7,6]], index=m_idx, columns=['C1','C2'])

level1 = df.index.get_level_values(1)
_mask = (level1 > '2017-05-04')

print(df.loc[(dt_idx[0]), 'C1'])              # WORKS OK
print(df.loc[(slice(None), _mask), 'C1'])     # WORKS OK
print(df.loc[('2017-05-04', _mask), 'C1'])    # WORKS OK
print(df.loc[(dt_idx[0], _mask), 'C1'])       # FAILS TypeError, invalid key

Problem description

Inconsistent behaviour of .loc slicer when combining with mask (or possibly other data types) leads to fail.
I believe this is either tangentially or directly related to other issues with similar headers but this may be a new variant.

Expected Output

Failed line expected to be equivalent to the working line above.

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.5.2.final.0 python-bits: 64 OS: Darwin OS-release: 16.4.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: None LOCALE: en_GB.UTF-8 pandas: 0.20.2 pytest: 3.0.5 pip: 9.0.1 setuptools: 32.1.0.post20161217 Cython: 0.25.2 numpy: 1.11.2 scipy: 0.18.1 xarray: None IPython: 5.1.0 sphinx: 1.5 patsy: 0.4.1 dateutil: 2.6.0 pytz: 2016.10 blosc: None bottleneck: 1.2.0 tables: 3.3.0 numexpr: 2.6.1 feather: None matplotlib: 1.5.3 openpyxl: 2.4.0 xlrd: 1.0.0 xlwt: 1.1.2 xlsxwriter: 0.9.4 lxml: None bs4: 4.5.1 html5lib: None sqlalchemy: 1.1.4 pymysql: None psycopg2: None jinja2: 2.8 s3fs: 0.0.7 pandas_gbq: None pandas_datareader: None

[jreback]

this should work. welcome to have you look into and submit a PR for this. also, can you add xref to other issues that look relevant. (you can do at the very top of the issue).

[jongmmm]

Related problem:

import pandas as pd

dt_idx = pd.to_datetime(['2017-05-04','2017-05-05'])
m_idx = pd.MultiIndex.from_product([dt_idx,dt_idx], names=['Idx1','Idx2'])
df = pd.DataFrame(data=[[1,2],[3,4],[5,6],[7,6]], index=m_idx, columns=['C1','C2'])

print(df.loc[[(pd.to_datetime('2017-05-04'),pd.to_datetime('2017-05-04'))]]) # works as expected
print(df.loc[[('2017-05-04','2017-05-04')]]) # NaN returned
print(df.loc[[pd.to_datetime('2017-05-04')]]) # works
print(df.loc[['2017-05-04']]) # KeyError: "['2017-05-04'] not in index"

[mroeschke]

This appears to work on master. Could use a test.

In [275]: import pandas as pd
     ...:
     ...: dt_idx = pd.to_datetime(['2017-05-04','2017-05-05'])
     ...: m_idx = pd.MultiIndex.from_product([dt_idx,dt_idx], names=['Idx1','Idx2'])
     ...: df = pd.DataFrame(data=[[1,2],[3,4],[5,6],[7,6]], index=m_idx, columns=['C1','C2'])
     ...:
     ...: level1 = df.index.get_level_values(1)
     ...: _mask = (level1 > '2017-05-04')
     ...:
     ...: print(df.loc[(dt_idx[0]), 'C1'])              # WORKS OK
     ...: print(df.loc[(slice(None), _mask), 'C1'])     # WORKS OK
     ...: print(df.loc[('2017-05-04', _mask), 'C1'])    # WORKS OK
     ...: print(df.loc[(dt_idx[0], _mask), 'C1'])
Idx2
2017-05-04    1
2017-05-05    3
Name: C1, dtype: int64
Idx1        Idx2
2017-05-04  2017-05-05    3
2017-05-05  2017-05-05    7
Name: C1, dtype: int64
Idx1        Idx2
2017-05-04  2017-05-05    3
Name: C1, dtype: int64
Idx1        Idx2
2017-05-04  2017-05-05    3
Name: C1, dtype: int64

In [276]: pd.__version__
Out[276]: '0.26.0.dev0+593.g9d45934af'