Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Datetimelike index / arrays ignore integer sign and size in astype #24381

Closed
TomAugspurger opened this issue Dec 21, 2018 · 2 comments · Fixed by #38544
Closed

Datetimelike index / arrays ignore integer sign and size in astype #24381

TomAugspurger opened this issue Dec 21, 2018 · 2 comments · Fixed by #38544
Labels
Bug Datetime Datetime data dtype Dtype Conversions Unexpected or buggy dtype conversions
Milestone

Comments

@TomAugspurger
Copy link
Contributor

TomAugspurger commented Dec 21, 2018

For datetimelike indexes, .astype ignore the sign and size of an integer dtype, and always uses int64

In [1]: import pandas as pd

In [2]: idx = pd.date_range('2000', periods=4)

In [3]: idx.astype('uint16')
Out[3]:
Int64Index([946684800000000000, 946771200000000000, 946857600000000000,
            946944000000000000],
           dtype='int64')

We don't have Index subclasses for each of these types (though we do have uint64 now).

In #24024 we agreed(?) that following the Index was the best behavior for the array class.

In [7]: idx._data.astype('uint16')
Out[7]:
array([946684800000000000, 946771200000000000, 946857600000000000,
       946944000000000000])

(that's also int64).


Some questions:

  1. Do we want to change this? Note that for "normal" usage of datetimes, this is going to truncate / overflow. This is particularly unfriendly to 32-bit systems, where "int" is "int32".
In [9]: idx._data.astype('int32').astype("int32")
Out[9]: array([1380122624, -476971008, 1960902656,  103809024], dtype=int32)

So maybe we would want safe casting?

In [10]: idx._data.astype('int32').astype("int32", casting="safe")
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-10-2c2a4a677a5c> in <module>
----> 1 idx._data.astype('int32').astype("int32", casting="safe")

TypeError: Cannot cast array from dtype('int64') to dtype('int32') according to the rule 'safe'
  1. ExtensionIndex classes are in the not too distant future, so we may be able to relatively easily get Index classes that work with these different integer sizes / signs. So we might be able to properly support this in the future (if we want to).
@TomAugspurger TomAugspurger added Datetime Datetime data dtype Dtype Conversions Unexpected or buggy dtype conversions labels Dec 21, 2018
@TomAugspurger TomAugspurger added this to the Someday milestone Dec 21, 2018
@jreback
Copy link
Contributor

jreback commented Dec 21, 2018

this is not likely easy to change as we don’t have the Index subclasses to support this

nor is it a big deal ; if/when we have subclasses then can supoort

@mroeschke mroeschke added the Bug label Apr 1, 2020
@jbrockmendel
Copy link
Member

This should raise TypeError, no?

@jreback jreback modified the milestones: Someday, 1.3 Dec 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Datetime Datetime data dtype Dtype Conversions Unexpected or buggy dtype conversions
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants