Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed apply_index #35165

Merged
merged 12 commits into from
Jul 15, 2020
18 changes: 18 additions & 0 deletions doc/source/reference/offset_frequency.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ Methods
:toctree: api/

DateOffset.apply
DateOffset.apply_index
DateOffset.copy
DateOffset.isAnchored
DateOffset.onOffset
Expand Down Expand Up @@ -117,6 +118,7 @@ Methods
:toctree: api/

BusinessHour.apply
BusinessHour.apply_index
BusinessHour.copy
BusinessHour.isAnchored
BusinessHour.onOffset
Expand Down Expand Up @@ -201,6 +203,7 @@ Methods
:toctree: api/

CustomBusinessHour.apply
CustomBusinessHour.apply_index
CustomBusinessHour.copy
CustomBusinessHour.isAnchored
CustomBusinessHour.onOffset
Expand Down Expand Up @@ -401,6 +404,7 @@ Methods
:toctree: api/

CustomBusinessMonthEnd.apply
CustomBusinessMonthEnd.apply_index
CustomBusinessMonthEnd.copy
CustomBusinessMonthEnd.isAnchored
CustomBusinessMonthEnd.onOffset
Expand Down Expand Up @@ -447,6 +451,7 @@ Methods
:toctree: api/

CustomBusinessMonthBegin.apply
CustomBusinessMonthBegin.apply_index
CustomBusinessMonthBegin.copy
CustomBusinessMonthBegin.isAnchored
CustomBusinessMonthBegin.onOffset
Expand Down Expand Up @@ -586,6 +591,7 @@ Methods
:toctree: api/

WeekOfMonth.apply
WeekOfMonth.apply_index
WeekOfMonth.copy
WeekOfMonth.isAnchored
WeekOfMonth.onOffset
Expand Down Expand Up @@ -622,6 +628,7 @@ Methods
:toctree: api/

LastWeekOfMonth.apply
LastWeekOfMonth.apply_index
LastWeekOfMonth.copy
LastWeekOfMonth.isAnchored
LastWeekOfMonth.onOffset
Expand Down Expand Up @@ -938,6 +945,7 @@ Methods
:toctree: api/

FY5253.apply
FY5253.apply_index
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to add these back if we are deprecating?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know why, but sphinx started complaining about these not being present. Which is technically correct, since it's a method on those classes. It just raises.

So I think keep for now, and just remove them all when we enforce the deprecation in 2.0

FY5253.copy
FY5253.get_rule_code_suffix
FY5253.get_year_end
Expand Down Expand Up @@ -977,6 +985,7 @@ Methods
:toctree: api/

FY5253Quarter.apply
FY5253Quarter.apply_index
FY5253Quarter.copy
FY5253Quarter.get_rule_code_suffix
FY5253Quarter.get_weeks
Expand Down Expand Up @@ -1013,6 +1022,7 @@ Methods
:toctree: api/

Easter.apply
Easter.apply_index
Easter.copy
Easter.isAnchored
Easter.onOffset
Expand Down Expand Up @@ -1053,6 +1063,7 @@ Methods
Tick.is_on_offset
Tick.__call__
Tick.apply
Tick.apply_index

Day
---
Expand Down Expand Up @@ -1087,6 +1098,7 @@ Methods
Day.is_on_offset
Day.__call__
Day.apply
Day.apply_index

Hour
----
Expand Down Expand Up @@ -1121,6 +1133,7 @@ Methods
Hour.is_on_offset
Hour.__call__
Hour.apply
Hour.apply_index

Minute
------
Expand Down Expand Up @@ -1155,6 +1168,7 @@ Methods
Minute.is_on_offset
Minute.__call__
Minute.apply
Minute.apply_index

Second
------
Expand Down Expand Up @@ -1189,6 +1203,7 @@ Methods
Second.is_on_offset
Second.__call__
Second.apply
Second.apply_index

Milli
-----
Expand Down Expand Up @@ -1223,6 +1238,7 @@ Methods
Milli.is_on_offset
Milli.__call__
Milli.apply
Milli.apply_index

Micro
-----
Expand Down Expand Up @@ -1257,6 +1273,7 @@ Methods
Micro.is_on_offset
Micro.__call__
Micro.apply
Micro.apply_index

Nano
----
Expand Down Expand Up @@ -1291,6 +1308,7 @@ Methods
Nano.is_on_offset
Nano.__call__
Nano.apply
Nano.apply_index

.. _api.frequencies:

Expand Down
1 change: 1 addition & 0 deletions doc/source/whatsnew/v1.1.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -795,6 +795,7 @@ Deprecations
- :meth:`DatetimeIndex.week` and `DatetimeIndex.weekofyear` are deprecated and will be removed in a future version, use :meth:`DatetimeIndex.isocalendar().week` instead (:issue:`33595`)
- :meth:`DatetimeArray.week` and `DatetimeArray.weekofyear` are deprecated and will be removed in a future version, use :meth:`DatetimeArray.isocalendar().week` instead (:issue:`33595`)
- :meth:`DateOffset.__call__` is deprecated and will be removed in a future version, use ``offset + other`` instead (:issue:`34171`)
- :meth:`~BusinessDay.apply_index` is deprecated and will be removed in a future version. Use ``offset + other`` instead (:issue:`34580`)
TomAugspurger marked this conversation as resolved.
Show resolved Hide resolved
- :meth:`DataFrame.tshift` and :meth:`Series.tshift` are deprecated and will be removed in a future version, use :meth:`DataFrame.shift` and :meth:`Series.shift` instead (:issue:`11631`)
- Indexing an :class:`Index` object with a float key is deprecated, and will
raise an ``IndexError`` in the future. You can manually convert to an integer key
Expand Down
99 changes: 84 additions & 15 deletions pandas/_libs/tslibs/offsets.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -86,19 +86,38 @@ cdef bint _is_normalized(datetime dt):
return True


def apply_wrapper_core(func, self, other) -> ndarray:
result = func(self, other)
result = np.asarray(result)

if self.normalize:
# TODO: Avoid circular/runtime import
from .vectorized import normalize_i8_timestamps
result = normalize_i8_timestamps(result.view("i8"), None)

return result


def apply_index_wraps(func):
# Note: normally we would use `@functools.wraps(func)`, but this does
# not play nicely with cython class methods
def wrapper(self, other) -> np.ndarray:
def wrapper(self, other):
# other is a DatetimeArray
result = apply_wrapper_core(func, self, other)
result = type(other)(result)
warnings.warn("'Offset.apply_index(other)' is deprecated. "
"Use 'offset + other' instead.", FutureWarning)
return result

result = func(self, other)
result = np.asarray(result)
return wrapper

if self.normalize:
# TODO: Avoid circular/runtime import
from .vectorized import normalize_i8_timestamps
result = normalize_i8_timestamps(result.view("i8"), None)

def apply_array_wraps(func):
# Note: normally we would use `@functools.wraps(func)`, but this does
# not play nicely with cython class methods
def wrapper(self, other) -> np.ndarray:
# other is a DatetimeArray
result = apply_wrapper_core(func, self, other)
return result

# do @functools.wraps(func) manually since it doesn't work on cdef funcs
Expand Down Expand Up @@ -515,19 +534,36 @@ cdef class BaseOffset:
raises NotImplementedError for offsets without a
vectorized implementation.

.. deprecated:: 1.1.0

Use ``offset + dtindex`` instead.

Parameters
----------
index : DatetimeIndex

Returns
-------
DatetimeIndex

Raises
------
NotImplementedError
When the specific offset subclass does not have a vectorized
implementation.
"""
raise NotImplementedError(
f"DateOffset subclass {type(self).__name__} "
"does not have a vectorized implementation"
)

@apply_array_wraps
def _apply_array(self, dtarr):
raise NotImplementedError(
f"DateOffset subclass {type(self).__name__} "
"does not have a vectorized implementation"
)

def rollback(self, dt) -> datetime:
"""
Roll provided date backward to next offset only if not on offset.
Expand Down Expand Up @@ -992,7 +1028,11 @@ cdef class RelativeDeltaOffset(BaseOffset):
-------
ndarray[datetime64[ns]]
"""
dt64other = np.asarray(dtindex)
return self._apply_array(dtindex)

@apply_array_wraps
def _apply_array(self, dtarr):
dt64other = np.asarray(dtarr)
kwds = self.kwds
relativedelta_fast = {
"years",
Expand Down Expand Up @@ -1321,7 +1361,11 @@ cdef class BusinessDay(BusinessMixin):

@apply_index_wraps
def apply_index(self, dtindex):
i8other = dtindex.view("i8")
return self._apply_array(dtindex)

@apply_array_wraps
def _apply_array(self, dtarr):
i8other = dtarr.view("i8")
return shift_bdays(i8other, self.n)

def is_on_offset(self, dt: datetime) -> bool:
Expand Down Expand Up @@ -1804,8 +1848,12 @@ cdef class YearOffset(SingleConstructorOffset):

@apply_index_wraps
def apply_index(self, dtindex):
return self._apply_array(dtindex)

@apply_array_wraps
def _apply_array(self, dtarr):
shifted = shift_quarters(
dtindex.view("i8"), self.n, self.month, self._day_opt, modby=12
dtarr.view("i8"), self.n, self.month, self._day_opt, modby=12
)
return shifted

Expand Down Expand Up @@ -1957,8 +2005,12 @@ cdef class QuarterOffset(SingleConstructorOffset):

@apply_index_wraps
def apply_index(self, dtindex):
return self._apply_array(dtindex)

@apply_array_wraps
def _apply_array(self, dtarr):
shifted = shift_quarters(
dtindex.view("i8"), self.n, self.startingMonth, self._day_opt
dtarr.view("i8"), self.n, self.startingMonth, self._day_opt
)
return shifted

Expand Down Expand Up @@ -2072,7 +2124,11 @@ cdef class MonthOffset(SingleConstructorOffset):

@apply_index_wraps
def apply_index(self, dtindex):
shifted = shift_months(dtindex.view("i8"), self.n, self._day_opt)
return self._apply_array(dtindex)

@apply_array_wraps
def _apply_array(self, dtarr):
shifted = shift_months(dtarr.view("i8"), self.n, self._day_opt)
return shifted

cpdef __setstate__(self, state):
Expand Down Expand Up @@ -2209,8 +2265,14 @@ cdef class SemiMonthOffset(SingleConstructorOffset):
@cython.wraparound(False)
@cython.boundscheck(False)
def apply_index(self, dtindex):
return self._apply_array(dtindex)

@apply_array_wraps
@cython.wraparound(False)
@cython.boundscheck(False)
def _apply_array(self, dtarr):
cdef:
int64_t[:] i8other = dtindex.view("i8")
int64_t[:] i8other = dtarr.view("i8")
Py_ssize_t i, count = len(i8other)
int64_t val
int64_t[:] out = np.empty(count, dtype="i8")
Expand Down Expand Up @@ -2368,12 +2430,16 @@ cdef class Week(SingleConstructorOffset):

@apply_index_wraps
def apply_index(self, dtindex):
return self._apply_array(dtindex)

@apply_array_wraps
def _apply_array(self, dtarr):
if self.weekday is None:
td = timedelta(days=7 * self.n)
td64 = np.timedelta64(td, "ns")
return dtindex + td64
return dtarr + td64
else:
i8other = dtindex.view("i8")
i8other = dtarr.view("i8")
return self._end_apply_index(i8other)

@cython.wraparound(False)
Expand Down Expand Up @@ -3146,6 +3212,9 @@ cdef class CustomBusinessDay(BusinessDay):
def apply_index(self, dtindex):
raise NotImplementedError

def _apply_array(self, dtarr):
raise NotImplementedError

def is_on_offset(self, dt: datetime) -> bool:
if self.normalize and not _is_normalized(dt):
return False
Expand Down
2 changes: 1 addition & 1 deletion pandas/core/arrays/datetimes.py
Original file line number Diff line number Diff line change
Expand Up @@ -683,7 +683,7 @@ def _add_offset(self, offset):
values = self.tz_localize(None)
else:
values = self
result = offset.apply_index(values)
result = offset._apply_array(values)
result = DatetimeArray._simple_new(result)
result = result.tz_localize(self.tz)

Expand Down
7 changes: 6 additions & 1 deletion pandas/tests/tseries/offsets/test_offsets.py
Original file line number Diff line number Diff line change
Expand Up @@ -3663,14 +3663,19 @@ def test_offset(self, case):

@pytest.mark.parametrize("case", offset_cases)
def test_apply_index(self, case):
# https://github.com/pandas-dev/pandas/issues/34580
offset, cases = case
s = DatetimeIndex(cases.keys())
exp = DatetimeIndex(cases.values())

with tm.assert_produces_warning(None):
# GH#22535 check that we don't get a FutureWarning from adding
# an integer array to PeriodIndex
result = offset + s
tm.assert_index_equal(result, exp)

exp = DatetimeIndex(cases.values())
with tm.assert_produces_warning(FutureWarning):
result = offset.apply_index(s)
tm.assert_index_equal(result, exp)

on_offset_cases = [
Expand Down