DEPR: Deprecate passing range-like arguments to DatetimeIndex, TimedeltaIndex #23919

jbrockmendel · 2018-11-26T03:31:46Z

Also verify_integrity since allowing that to be False complicates things, and internally if its False we should be using simple_new anyway

closes API: deprecate range-generating keywords in DatetimeIndex #20535
tests added / passed
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

pep8speaks · 2018-11-26T03:32:16Z

Hello @jbrockmendel! Thanks for submitting the PR.

There are no PEP8 issues in the file pandas/core/indexes/datetimes.py !
There are no PEP8 issues in the file pandas/core/indexes/timedeltas.py !
There are no PEP8 issues in the file pandas/core/resample.py !
There are no PEP8 issues in the file pandas/tests/frame/test_indexing.py !
There are no PEP8 issues in the file pandas/tests/groupby/test_groupby.py !
There are no PEP8 issues in the file pandas/tests/groupby/test_timegrouper.py !
There are no PEP8 issues in the file pandas/tests/indexes/datetimes/test_construction.py !
There are no PEP8 issues in the file pandas/tests/indexes/datetimes/test_date_range.py !
There are no PEP8 issues in the file pandas/tests/indexes/datetimes/test_datetime.py !
There are no PEP8 issues in the file pandas/tests/indexes/datetimes/test_formats.py !
There are no PEP8 issues in the file pandas/tests/indexes/datetimes/test_indexing.py !
There are no PEP8 issues in the file pandas/tests/indexes/datetimes/test_misc.py !
There are no PEP8 issues in the file pandas/tests/indexes/datetimes/test_ops.py !
There are no PEP8 issues in the file pandas/tests/indexes/datetimes/test_partial_slicing.py !
There are no PEP8 issues in the file pandas/tests/indexes/datetimes/test_scalar_compat.py !
There are no PEP8 issues in the file pandas/tests/indexes/datetimes/test_setops.py !
There are no PEP8 issues in the file pandas/tests/indexes/datetimes/test_timezones.py !
There are no PEP8 issues in the file pandas/tests/indexes/period/test_indexing.py !
There are no PEP8 issues in the file pandas/tests/indexes/period/test_partial_slicing.py !
There are no PEP8 issues in the file pandas/tests/indexes/period/test_tools.py !
There are no PEP8 issues in the file pandas/tests/indexes/timedeltas/test_construction.py !
There are no PEP8 issues in the file pandas/tests/indexes/timedeltas/test_indexing.py !
There are no PEP8 issues in the file pandas/tests/indexes/timedeltas/test_setops.py !
There are no PEP8 issues in the file pandas/tests/io/formats/test_to_html.py !
There are no PEP8 issues in the file pandas/tests/io/parser/test_parse_dates.py !
There are no PEP8 issues in the file pandas/tests/plotting/test_datetimelike.py !
There are no PEP8 issues in the file pandas/tests/reshape/merge/test_merge_asof.py !
There are no PEP8 issues in the file pandas/tests/reshape/test_concat.py !
There are no PEP8 issues in the file pandas/tests/series/indexing/test_datetime.py !
There are no PEP8 issues in the file pandas/tests/series/test_datetime_values.py !
There are no PEP8 issues in the file pandas/tests/test_resample.py !
There are no PEP8 issues in the file pandas/tests/tseries/offsets/test_offsets.py !
There are no PEP8 issues in the file pandas/tseries/holiday.py !
There are no PEP8 issues in the file pandas/util/testing.py !

doc/source/whatsnew/v0.24.0.rst

pandas/tests/indexes/timedeltas/test_construction.py

…ntle

jorisvandenbossche

Thanks! Looks good to me, a minor comment.

Further, can you add an assertion for the warning on verify_integrity ?

Also, small suggestion: you now left one DatetimeIndex in there with an assert_produces_warning. I would maybe do such an assertion in a separate test (now that assertion is a bit buried in a bigger test that is for the rest using date_range)

doc/source/whatsnew/v0.24.0.rst

jorisvandenbossche · 2018-11-26T14:09:29Z

pandas/tests/groupby/test_timegrouper.py

@@ -385,7 +385,7 @@ def test_groupby_groups_datetimeindex(self):
        groups = grouped.groups
        assert isinstance(list(groups.keys())[0], datetime)

-        # GH 11442
+        # GH#11442


not important, but was just wondering: why are you adding the '#' everywhere?

Mostly for internal consistency (not a big deal, but for grepping purposes). A little bit because I was curious how long it would take before someone asked about it.

we have have the gh-xxxx style as well, slight prefernce for that

…ntle

jorisvandenbossche · 2018-11-26T23:28:13Z

pandas/core/indexes/datetimes.py

+    with warnings.catch_warnings():
+        # we ignore warnings from passing verify_integrity=False
+        # TODO: If we knew what was going in to **d, we might be able to
+        #  go through _simple_new instead


you can check __reduce__, d should be _data and the attributes dict

good idea, that seems to work; just pushed

Looks like this broke the legacy pickle tests; reverted.

codecov · 2018-11-26T23:43:37Z

Codecov Report

❗ No coverage uploaded for pull request base (master@30c1290). Click here to learn what that means.
The diff coverage is 100%.

@@            Coverage Diff            @@
##             master   #23919   +/-   ##
=========================================
  Coverage          ?   92.31%           
=========================================
  Files             ?      161           
  Lines             ?    51489           
  Branches          ?        0           
=========================================
  Hits              ?    47533           
  Misses            ?     3956           
  Partials          ?        0

Flag	Coverage Δ
#multiple	`90.71% <100%> (?)`
#single	`42.43% <53.33%> (?)`

Impacted Files	Coverage Δ
pandas/util/testing.py	`86.09% <100%> (ø)`
pandas/core/resample.py	`96.99% <100%> (ø)`
pandas/io/packers.py	`88.08% <100%> (ø)`
pandas/core/indexes/timedeltas.py	`89.44% <100%> (ø)`
pandas/tseries/holiday.py	`93.17% <100%> (ø)`
pandas/core/indexes/datetimes.py	`96.51% <100%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 30c1290...cc40717. Read the comment docs.

jreback · 2018-11-27T01:11:12Z

pandas/core/indexes/timedeltas.py

@@ -131,10 +132,18 @@ def __new__(cls, data=None, unit=None, freq=None, start=None, end=None,
                periods=None, closed=None, dtype=None, copy=False,
                name=None, verify_integrity=True):

+        if verify_integrity is not True:


why isn't the default of verify_integrity None?

Because by default we do verify the integrity of a passed frequency. Best guess as to initial motivation is that verify_integrity=False is kind of like fastpath=True and was never really intended to be user-facing.

right but aren't you deprecating it?

Yes. In the future it will just be set to verify_integrity=True at the top of __new__. (and when the time comes, in the TDA/DTA constructors it will always be True.

We still need the variable to exist because there are cases in which we can determine it is not necessary, in which case we can skip a potentially-expensive check.

I don't recall that (and dont see how it would work), but yah, not passing freq would make verify_integrity unnecessary.

We still need the variable to exist because there are cases in which we can determine it is not necessary, in which case we can skip a potentially-expensive check.

@jbrockmendel Isn't the end goal to actually remove the verify_integrity keyword from the Index constructors? (not just keep it with a fixed True value). Because otherwise, why are you deprecating verify_integrity=False if we actually want to use it ourselves?
In cases that we determine the check is not necessary, we can use _simple_new ?

Eventually verify-integrity will not be in the signature. The first line of the method will just define it to be True.

OK, but then as @jreback said, we should put it at None so we can also deprecate the case for somebody setting it to True explicitly.

jreback · 2018-11-27T01:12:07Z

pandas/tests/groupby/test_timegrouper.py

@@ -385,7 +385,7 @@ def test_groupby_groups_datetimeindex(self):
        groups = grouped.groups
        assert isinstance(list(groups.keys())[0], datetime)

-        # GH 11442
+        # GH#11442


we have have the gh-xxxx style as well, slight prefernce for that

…ntle

jreback · 2018-11-27T17:19:33Z

lgtm

…ntle

jbrockmendel · 2018-11-28T00:02:50Z

circle fail is Hypothesis; de-facto green

…ntle

mroeschke · 2018-11-28T05:19:38Z

Shouldn't we do the same for PeriodIndex as well?

jbrockmendel · 2018-11-28T14:38:48Z

Shouldn't we do the same for PeriodIndex as well?

Possibly. There are also fields kwargs there we might want to deprecate. Prefer to do this separately.

…ntle

jreback · 2018-11-28T16:19:42Z

@jorisvandenbossche

…ntle

jorisvandenbossche · 2018-11-28T23:35:15Z

thanks!

…ltaIndex (pandas-dev#23919)

jbrockmendel added 2 commits November 25, 2018 19:29

Deprecate passing range-like arguments to DatetimeIndex, TimedeltaIndex

f8bed85

GH reference

5d8a107

gfyoung added Datetime Datetime data dtype API Design Deprecate Functionality to remove in pandas labels Nov 26, 2018

gfyoung reviewed Nov 26, 2018

View reviewed changes

doc/source/whatsnew/v0.24.0.rst Outdated Show resolved Hide resolved

gfyoung reviewed Nov 26, 2018

View reviewed changes

pandas/tests/indexes/timedeltas/test_construction.py Show resolved Hide resolved

jbrockmendel added 3 commits November 26, 2018 05:43

Merge branch 'master' of https://github.com/pandas-dev/pandas into ge…

28e2184

…ntle

Flake8 fixups, whatsnew typo

0f75b9d

update usage in resample

2c40c3a

jorisvandenbossche reviewed Nov 26, 2018

View reviewed changes

jbrockmendel added 7 commits November 26, 2018 08:58

Merge branch 'master' of https://github.com/pandas-dev/pandas into ge…

d67f87c

…ntle

targeted tests

afdab5b

backticks

8f435ed

test the right thing

4d7c9e2

avoid verify_integrity warnings

2e587e3

avoid verify_integrity

0469b74

isort

43a52fc

jorisvandenbossche requested changes Nov 26, 2018

View reviewed changes

jbrockmendel force-pushed the gentle branch from 3486e69 to 43a52fc Compare November 27, 2018 01:12

jreback requested changes Nov 27, 2018

View reviewed changes

jbrockmendel added 4 commits November 27, 2018 06:44

Merge branch 'master' of https://github.com/pandas-dev/pandas into ge…

f4e281e

…ntle

change default to None

15e6c30

set verify_integrity conditionally

5dc66f3

Merge branch 'master' of https://github.com/pandas-dev/pandas into ge…

b931878

…ntle

jreback approved these changes Nov 27, 2018

View reviewed changes

Merge branch 'master' of https://github.com/pandas-dev/pandas into ge…

07bfc45

…ntle

Merge branch 'master' of https://github.com/pandas-dev/pandas into ge…

e209a81

…ntle

Merge branch 'master' of https://github.com/pandas-dev/pandas into ge…

d6df7a3

…ntle

jreback added this to the 0.24.0 milestone Nov 28, 2018

jbrockmendel added 2 commits November 28, 2018 10:57

Merge branch 'master' of https://github.com/pandas-dev/pandas into ge…

eb5d9c5

…ntle

restore edit that got lost in rebase

cc40717

jorisvandenbossche approved these changes Nov 28, 2018

View reviewed changes

jorisvandenbossche merged commit 6b3490f into pandas-dev:master Nov 28, 2018

jbrockmendel deleted the gentle branch November 28, 2018 23:38

Pingviinituutti pushed a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019

DEPR: Deprecate passing range-like arguments to DatetimeIndex, Timede…

122fa0c

…ltaIndex (pandas-dev#23919)

Pingviinituutti pushed a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019

DEPR: Deprecate passing range-like arguments to DatetimeIndex, Timede…

933b82b

…ltaIndex (pandas-dev#23919)

jsexauer mentioned this pull request Nov 28, 2019

DEPR: Clean up list of deprecations from prior versions #6581

Closed

1 task

jreback mentioned this pull request Dec 2, 2019

DEPR: deprecations log for removed issues #13777

Closed

johnclary mentioned this pull request Feb 26, 2020

TCP Business Days Elapsed Script is Failing cityofaustin/atd-data-tech#1421

Closed

jreback mentioned this pull request Jul 23, 2020

QST: DatetimeIndex changes #35390

Closed

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DEPR: Deprecate passing range-like arguments to DatetimeIndex, TimedeltaIndex #23919

DEPR: Deprecate passing range-like arguments to DatetimeIndex, TimedeltaIndex #23919

jbrockmendel commented Nov 26, 2018

pep8speaks commented Nov 26, 2018

jorisvandenbossche left a comment

jorisvandenbossche Nov 26, 2018

jbrockmendel Nov 26, 2018

jreback Nov 27, 2018

jorisvandenbossche Nov 26, 2018

jbrockmendel Nov 26, 2018

jbrockmendel Nov 27, 2018

codecov bot commented Nov 26, 2018 •

edited

Loading

jreback Nov 27, 2018

jbrockmendel Nov 27, 2018

jreback Nov 27, 2018

jbrockmendel Nov 27, 2018

jbrockmendel Nov 27, 2018

jbrockmendel Nov 27, 2018

jorisvandenbossche Nov 27, 2018

jbrockmendel Nov 27, 2018

jorisvandenbossche Nov 27, 2018

jbrockmendel Nov 27, 2018

jreback Nov 27, 2018

jreback commented Nov 27, 2018

jbrockmendel commented Nov 28, 2018

mroeschke commented Nov 28, 2018

jbrockmendel commented Nov 28, 2018

jreback commented Nov 28, 2018

jorisvandenbossche commented Nov 28, 2018

DEPR: Deprecate passing range-like arguments to DatetimeIndex, TimedeltaIndex #23919

DEPR: Deprecate passing range-like arguments to DatetimeIndex, TimedeltaIndex #23919

Conversation

jbrockmendel commented Nov 26, 2018

pep8speaks commented Nov 26, 2018

jorisvandenbossche left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Nov 26, 2018 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Nov 27, 2018

jbrockmendel commented Nov 28, 2018

mroeschke commented Nov 28, 2018

jbrockmendel commented Nov 28, 2018

jreback commented Nov 28, 2018

jorisvandenbossche commented Nov 28, 2018

codecov bot commented Nov 26, 2018 •

edited

Loading