DEPR: Deprecate week, weekofyear in Series.dt,DatetimeIndex #33595

mgmarino · 2020-04-16T20:02:13Z

This PR is a followup to #33220 and implements what was discussed there.

A few comments:

DatetimeIndex.isocalendar does not set the index on the returned dataframe,
leading to some cumbersome syntax. This may be worth changing, please let me
know what you think.
Should we also deprecate Timestamp.week/weekofyear?
closes #xxxx
tests added / passed
passes black pandas
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

mgmarino · 2020-04-16T20:07:42Z

pandas/tests/groupby/transform/test_transform.py

-    stocks["week_id"] = pd.to_datetime(stocks.index).week
+    stocks["week_id"] = (
+        pd.to_datetime(stocks.index).isocalendar().set_index(stocks.index).week
+    )


This is the cumbersome syntax I referenced in the PR message.

pd.to_datetime(stocks.index) is just dates, ditto stocks.index, so this can become dates.isocalendar().set_index(dates)["week"]

jreback

this looks fine to me

jreback · 2020-04-16T20:47:38Z

pandas/core/indexes/accessors.py

+        Series.dt.weekofyear and Series.dt.week have been deprecated.
+        Please use Series.dt.isocalendar().week instead.
+        """
+        import warnings


you can put this import at the top of the file (same everywhere)

jreback · 2020-04-16T20:48:10Z

pandas/core/indexes/accessors.py

+        )
+        week_series = self.isocalendar().week
+        week_series.name = self.name
+        if week_series.hasnans:


we do it like this now?

Yes, in the current week accessor the array is converted to a float and masked if there are nans. This reproduces that behavior instead of passing back a UInt32 series.

jreback · 2020-04-16T20:48:26Z

pandas/tests/arrays/test_datetimes.py

+
+
+def test_week_and_week_of_year_are_deprecated():
+    arr = DatetimeArray._from_sequence(["2000-01-03"])


at the issue number as a comment

jreback · 2020-04-16T20:49:24Z

pandas/tests/indexes/datetimes/test_misc.py

@@ -375,6 +375,14 @@ def test_iter_readonly():
    list(dti)


+def test_week_and_weekofyear_are_deprecated():
+    idx = pd.date_range(start="2019-12-29", freq="D", periods=4)


jreback · 2020-04-16T20:49:43Z

pandas/tests/series/test_datetime_values.py

+
+
+def test_week_and_weekofyear_are_deprecated():
+    series = pd.to_datetime(pd.Series(["2020-01-01"]))


jreback · 2020-04-16T20:50:44Z

DatetimeIndex.isocalendar does not set the index on the returned dataframe,
leading to some cumbersome syntax. This may be worth changing, please let me
know what you think.

hmm you mean the index of the original dataframe? I think we do this on the accessors, so we should do this.

jbrockmendel · 2020-04-16T21:04:41Z

pandas/core/indexes/datetimes.py

@@ -809,6 +809,32 @@ def indexer_between_time(

        return mask.nonzero()[0]

+    @property
+    def weekofyear(self):


we should be inheriting this from the underlying DatetimeArray using inherit_names

Do you mean the deprecation should be handled by the underlying DatetimeArray? I started trying to do it that way. The reason for doing it this way is to get the stacklevel right for the warning. If we are ok with having it "broken" for a direct call to DatetimeArray.week, then I can go that route.

it will almost always be seen by users via the DatetimeIndex, so i think thats the case to prioritize getting the stacklevel right for

Ok, I moved the deprecation In DatetimeIndex to only be in DatetimeArray and optimized the stacklevel appropriately.

jbrockmendel · 2020-04-16T21:08:48Z

DatetimeIndex.isocalendar does not set the index on the returned dataframe,
leading to some cumbersome syntax

agreed

mgmarino · 2020-04-17T07:08:27Z

hmm you mean the index of the original dataframe? I think we do this on the accessors, so we should do this.

Yes, I mean that the DataFrame returned by DatetimeIndex.isocalendar() should probably have its index set to the DatetimeIndex. This is done e.g. in Series.dt.isocalendar(): the returned DataFrame has it's index set to that of the Series.

For Index classes, the current behavior of inherit_names when wrapping is to convert the result of the delegated call to the same type if possible (i.e. DatetimeIndex => DatetimeIndex) and to fall back to a generic Index if not.

DatetimeIndex.isocalendar does not set the index on the returned dataframe,
leading to some cumbersome syntax

agreed

Ok, the question would then be where to do this:

It can be done in inherit_names, but the code there would need to be adapted to handle a DataFrame (or something more generic?) and this is currently the only case for that.
Or the delegation can be done "manually" in an explicitly definedDatetimeIndex.isocalendar method.

jorisvandenbossche

I am personally not sure a deprecation is needed, see #33503

mgmarino · 2020-04-20T21:35:40Z

Build failing due to numba version. Will wait for #33686 or patch from @mroeschke

mgmarino · 2020-04-21T08:42:39Z

@jbrockmendel @jreback
Sorry for the delay, I believe all open comments have been addressed. Tests green.

Regarding the setting of the index, I would like to do that in a separate PR if there are no objections.

jreback · 2020-04-21T12:47:34Z

pandas/tests/scalar/test_nat.py

@@ -66,6 +66,9 @@ def test_nat_vector_field_access():
        # on NaT/Timestamp for compat with datetime
        if field == "weekday":
            continue
+        if field in ["week", "weekofyear"]:


we should properly parameterize these tests (if you would like to make an issue would be great), PR even better :->

Im happy to make another PR, will wait until this is merged.

jbrockmendel · 2020-04-21T15:04:32Z

pandas/core/arrays/datetimes.py

+            FutureWarning,
+            stacklevel=3,
+        )
+        return pd.Int64Index(self.isocalendar().week)


this should return ndarray here and be wrapped on DatetimeIndex to return Int64Index

Of course, thanks.

- This is then wrapped by DatetimeIndex.week

mgmarino · 2020-04-27T13:14:29Z

@mroeschke @jreback @jbrockmendel Just checking in here to see if anything else needs to be done re discussion/PR? Thanks!

…-year

mgmarino · 2020-05-12T18:25:30Z

@mroeschke @jreback I would like to ping one more time, to try to keep this from getting too stale. If more discussion needs to be add about this, I'm totally fine with that, but I would like to also address the two issues that came up in the PR here. If this needs more time to merge, then I would go ahead and address those already.

mroeschke · 2020-05-12T18:53:18Z

Sorry for the delay @mgmarino. I think everything looked good to me a while back. Looks like there's a merge conflict, but once that's resolved I think this should be good

mroeschke

LGTM @jreback

jreback · 2020-05-13T00:11:10Z

yeah this is ok, ping on green. @jorisvandenbossche unless any objections here.

…-year

mgmarino · 2020-05-13T06:16:04Z

Build failure due to a version bump of flake in the CI (now 3.8.1, last succeeded with 3.7.9), see #34150.

…-year

mgmarino · 2020-05-13T13:49:31Z

@jreback back to green

jreback · 2020-05-25T22:17:19Z

this lgtm. we can merge on green.

mroeschke · 2020-05-26T03:22:26Z

Thanks for sticking with this @mgmarino. Great PR as always!

mgmarino marked this pull request as ready for review April 16, 2020 20:05

mgmarino marked this pull request as draft April 16, 2020 20:05

mgmarino force-pushed the deprecate-week-of-year branch from 898a358 to 5942241 Compare April 16, 2020 20:06

mgmarino commented Apr 16, 2020

View reviewed changes

mgmarino marked this pull request as ready for review April 16, 2020 20:34

jreback requested changes Apr 16, 2020

View reviewed changes

jreback added Deprecate Functionality to remove in pandas Datetime Datetime data dtype labels Apr 16, 2020

jorisvandenbossche mentioned this pull request Apr 16, 2020

DEPR: log of deprecations in 1.x (to be removed in 2.0) #30228

Closed

jbrockmendel reviewed Apr 16, 2020

View reviewed changes

jorisvandenbossche requested changes Apr 17, 2020

View reviewed changes

mgmarino force-pushed the deprecate-week-of-year branch 2 times, most recently from 943e8f9 to 29ce5ac Compare April 20, 2020 20:49

mgmarino force-pushed the deprecate-week-of-year branch 2 times, most recently from f651e61 to 6e90ae7 Compare April 21, 2020 05:35

jreback approved these changes Apr 21, 2020

View reviewed changes

jbrockmendel reviewed Apr 21, 2020

View reviewed changes

mgmarino added 8 commits April 21, 2020 18:11

Deprecate week and weekofyear in Series.dt/DatetimeIndex

4007381

Add What's New for deprecations

38efb17

Ignore week/weekofyear attributes in tests

f5b42b8

Remove week/weekofyear attributes from tests

22c96f5

Add isocalendar test to replace removed week/weekofyear tests

9688dd0

Add PR issue number

cd702eb

Import warnings at top of module

b4eb746

Tag deprecation tests with PR

56874ec

Refactor test to improve readability

0bda2e8

mgmarino force-pushed the deprecate-week-of-year branch 2 times, most recently from 4539708 to 47c3d32 Compare April 21, 2020 16:50

Return ndarray from DatetimeArray.week

658506d

- This is then wrapped by DatetimeIndex.week

mgmarino force-pushed the deprecate-week-of-year branch from 47c3d32 to 658506d Compare April 21, 2020 16:57

mroeschke approved these changes Apr 23, 2020

View reviewed changes

mgmarino added 2 commits April 30, 2020 10:02

Merge remote-tracking branch 'upstream/master' into deprecate-week-of…

734ae8f

…-year

Merge remote-tracking branch 'upstream/master' into deprecate-week-of…

762dc81

…-year

mroeschke approved these changes May 12, 2020

View reviewed changes

jreback added this to the 1.1 milestone May 13, 2020

Merge remote-tracking branch 'upstream/master' into deprecate-week-of…

b81536b

…-year

mgmarino force-pushed the deprecate-week-of-year branch from 1a12fd8 to b81536b Compare May 13, 2020 05:22

Merge remote-tracking branch 'upstream/master' into deprecate-week-of…

e87c26c

…-year

mgmarino force-pushed the deprecate-week-of-year branch from 3872e96 to e87c26c Compare May 13, 2020 12:58

Merge branch 'master' into deprecate-week-of-year

acf2bbb

mroeschke merged commit 3ae33c6 into pandas-dev:master May 26, 2020

mgmarino deleted the deprecate-week-of-year branch May 26, 2020 06:36

mgmarino mentioned this pull request May 26, 2020

ENH: Set index of returned dataframe in DatetimeIndex.isocalendar #34392

Merged

mroeschke mentioned this pull request Oct 28, 2022

DEPR: Remove week & weekofyear for datetimes #49380

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DEPR: Deprecate week, weekofyear in Series.dt,DatetimeIndex #33595

DEPR: Deprecate week, weekofyear in Series.dt,DatetimeIndex #33595

mgmarino commented Apr 16, 2020 •

edited by jorisvandenbossche

Loading

mgmarino Apr 16, 2020

jbrockmendel Apr 16, 2020

jreback left a comment

jreback Apr 16, 2020

jreback Apr 16, 2020

mgmarino Apr 17, 2020

jreback Apr 16, 2020

jreback Apr 16, 2020

jreback Apr 16, 2020

jreback commented Apr 16, 2020

jbrockmendel Apr 16, 2020

mgmarino Apr 17, 2020

jbrockmendel Apr 17, 2020

mgmarino Apr 21, 2020

jbrockmendel commented Apr 16, 2020

mgmarino commented Apr 17, 2020

jorisvandenbossche left a comment

mgmarino commented Apr 20, 2020

mgmarino commented Apr 21, 2020

jreback Apr 21, 2020

mgmarino Apr 21, 2020

jbrockmendel Apr 21, 2020

mgmarino Apr 21, 2020

mgmarino commented Apr 27, 2020

mgmarino commented May 12, 2020

mroeschke commented May 12, 2020

mroeschke left a comment

jreback commented May 13, 2020

mgmarino commented May 13, 2020 •

edited

Loading

mgmarino commented May 13, 2020

jreback commented May 25, 2020

mroeschke commented May 26, 2020



		def test_week_and_week_of_year_are_deprecated():
		arr = DatetimeArray._from_sequence(["2000-01-03"])



		def test_week_and_weekofyear_are_deprecated():
		series = pd.to_datetime(pd.Series(["2020-01-01"]))

DEPR: Deprecate week, weekofyear in Series.dt,DatetimeIndex #33595

DEPR: Deprecate week, weekofyear in Series.dt,DatetimeIndex #33595

Conversation

mgmarino commented Apr 16, 2020 • edited by jorisvandenbossche Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Apr 16, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jbrockmendel commented Apr 16, 2020

mgmarino commented Apr 17, 2020

jorisvandenbossche left a comment

Choose a reason for hiding this comment

mgmarino commented Apr 20, 2020

mgmarino commented Apr 21, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mgmarino commented Apr 27, 2020

mgmarino commented May 12, 2020

mroeschke commented May 12, 2020

mroeschke left a comment

Choose a reason for hiding this comment

jreback commented May 13, 2020

mgmarino commented May 13, 2020 • edited Loading

mgmarino commented May 13, 2020

jreback commented May 25, 2020

mroeschke commented May 26, 2020

mgmarino commented Apr 16, 2020 •

edited by jorisvandenbossche

Loading

mgmarino commented May 13, 2020 •

edited

Loading