Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DEPR: string indexing along index for datetimes #36179

Merged
merged 15 commits into from
Sep 24, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions doc/source/user_guide/timeseries.rst
Original file line number Diff line number Diff line change
Expand Up @@ -579,7 +579,12 @@ This type of slicing will work on a ``DataFrame`` with a ``DatetimeIndex`` as we
partial string selection is a form of label slicing, the endpoints **will be** included. This
would include matching times on an included date:

.. warning::

Indexing ``DataFrame`` rows with strings is deprecated in pandas 1.2.0 and will be removed in a future version. Use ``frame.loc[dtstring]`` instead.

.. ipython:: python
:okwarning:

dft = pd.DataFrame(np.random.randn(100000, 1), columns=['A'],
index=pd.date_range('20130101', periods=100000, freq='T'))
Expand All @@ -590,24 +595,28 @@ This starts on the very first time in the month, and includes the last date and
time for the month:

.. ipython:: python
:okwarning:

dft['2013-1':'2013-2']
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not actually deprecated, right? (and thus shouldn't raise a warning?)

Copy link
Member Author

@jbrockmendel jbrockmendel Sep 27, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. I think this should be deprecated too.

Edit: On further thought, not sure.


This specifies a stop time **that includes all of the times on the last day**:

.. ipython:: python
:okwarning:

dft['2013-1':'2013-2-28']

This specifies an **exact** stop time (and is not the same as the above):

.. ipython:: python
:okwarning:

dft['2013-1':'2013-2-28 00:00:00']

We are stopping on the included end-point as it is part of the index:

.. ipython:: python
:okwarning:

dft['2013-1-15':'2013-1-15 12:30:00']

Expand All @@ -631,6 +640,7 @@ We are stopping on the included end-point as it is part of the index:
Slicing with string indexing also honors UTC offset.

.. ipython:: python
:okwarning:

df = pd.DataFrame([0], index=pd.DatetimeIndex(['2019-01-01'], tz='US/Pacific'))
df
Expand Down Expand Up @@ -681,6 +691,7 @@ If index resolution is second, then the minute-accurate timestamp gives a
If the timestamp string is treated as a slice, it can be used to index ``DataFrame`` with ``[]`` as well.

.. ipython:: python
:okwarning:

dft_minute = pd.DataFrame({'a': [1, 2, 3], 'b': [4, 5, 6]},
index=series_minute.index)
Expand Down Expand Up @@ -2027,6 +2038,7 @@ You can pass in dates and strings to ``Series`` and ``DataFrame`` with ``PeriodI
Passing a string representing a lower frequency than ``PeriodIndex`` returns partial sliced data.

.. ipython:: python
:okwarning:

ps['2011']

Expand Down
1 change: 1 addition & 0 deletions doc/source/whatsnew/v0.11.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -367,6 +367,7 @@ Enhancements
- You can now select with a string from a DataFrame with a datelike index, in a similar way to a Series (:issue:`3070`)

.. ipython:: python
:okwarning:

idx = pd.date_range("2001-10-1", periods=5, freq='M')
ts = pd.Series(np.random.rand(len(idx)), index=idx)
Expand Down
1 change: 1 addition & 0 deletions doc/source/whatsnew/v1.2.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,7 @@ Deprecations
- Date parser functions :func:`~pandas.io.date_converters.parse_date_time`, :func:`~pandas.io.date_converters.parse_date_fields`, :func:`~pandas.io.date_converters.parse_all_fields` and :func:`~pandas.io.date_converters.generic_parser` from ``pandas.io.date_converters`` are deprecated and will be removed in a future version; use :func:`to_datetime` instead (:issue:`35741`)
- :meth:`DataFrame.lookup` is deprecated and will be removed in a future version, use :meth:`DataFrame.melt` and :meth:`DataFrame.loc` instead (:issue:`18682`)
- The :meth:`Index.to_native_types` is deprecated. Use ``.astype(str)`` instead (:issue:`28867`)
- Deprecated indexing :class:`DataFrame` rows with datetime-like strings ``df[string]``, use ``df.loc[string]`` instead (:issue:`36179`)

.. ---------------------------------------------------------------------------

Expand Down
11 changes: 10 additions & 1 deletion pandas/core/indexing.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
from typing import TYPE_CHECKING, Hashable, List, Tuple, Union
import warnings

import numpy as np

Expand Down Expand Up @@ -2191,7 +2192,15 @@ def convert_to_index_sliceable(obj: "DataFrame", key):
# slice here via partial string indexing
if idx._supports_partial_string_indexing:
try:
return idx._get_string_slice(key)
res = idx._get_string_slice(key)
warnings.warn(
"Indexing on datetimelike rows with `frame[string]` is "
"deprecated and will be removed in a future version. "
"Use `frame.loc[string]` instead.",
FutureWarning,
stacklevel=3,
)
return res
except (KeyError, ValueError, NotImplementedError):
return None

Expand Down
4 changes: 3 additions & 1 deletion pandas/tests/indexes/datetimes/test_partial_slicing.py
Original file line number Diff line number Diff line change
Expand Up @@ -228,7 +228,9 @@ def test_partial_slicing_dataframe(self):
tm.assert_series_equal(result, expected)

# Frame should return slice as well
result = df[ts_string]
with tm.assert_produces_warning(FutureWarning):
# GH#36179 deprecated this indexing
result = df[ts_string]
expected = df[theslice]
tm.assert_frame_equal(result, expected)

Expand Down
15 changes: 9 additions & 6 deletions pandas/tests/series/indexing/test_datetime.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
"""
Also test support for datetime64[ns] in Series / DataFrame
"""
from datetime import datetime, timedelta
import re

Expand All @@ -11,10 +14,6 @@
from pandas import DataFrame, DatetimeIndex, NaT, Series, Timestamp, date_range
import pandas._testing as tm

"""
Also test support for datetime64[ns] in Series / DataFrame
"""


def test_fancy_getitem():
dti = date_range(
Expand Down Expand Up @@ -605,7 +604,9 @@ def test_indexing():
expected.name = "A"

df = DataFrame(dict(A=ts))
result = df["2001"]["A"]
with tm.assert_produces_warning(FutureWarning):
# GH#36179 string indexing on rows for DataFrame deprecated
result = df["2001"]["A"]
tm.assert_series_equal(expected, result)

# setting
Expand All @@ -615,7 +616,9 @@ def test_indexing():

df.loc["2001", "A"] = 1

result = df["2001"]["A"]
with tm.assert_produces_warning(FutureWarning):
# GH#36179 string indexing on rows for DataFrame deprecated
result = df["2001"]["A"]
tm.assert_series_equal(expected, result)

# GH3546 (not including times on the last day)
Expand Down