-
-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix indexing with datetime64[ns] with pandas=1.1 #4292
Conversation
Fixes pydata#4283 The underlying issue is that calling `.item()` on a NumPy array with `dtype=datetime64[ns]` returns an _integer_, rather than an `np.datetime64 scalar. This is somewhat baffling but works this way because `.item()` returns native Python types, but `datetime.datetime` doesn't support nanosecond precision. `pandas.Index.get_loc` used to support these integers, but now is more strict. Hence we get errors. We can fix this by using `array[()]` to convert 0d arrays into NumPy scalars instead of calling `array.item()`. I've added a crude regression test. There may well be a better way to test this but I haven't figured it out yet.
you should be able to isolate this to just def test_convert_label_indexer_datetime(self):
index = pd.to_datetime(["2000-01-01", "2001-01-01", "2002-01-01"])
actual = indexing.convert_label_indexer(index, "2001-01-01")
expected = (1, None)
assert actual == expected
actual = indexing.convert_label_indexer(index, index.to_numpy()[1])
assert actual == expected The failing tests are due to if label.dtype.kind in "mM":
label_value = label[()]
else:
label_value = label.item() Edit: |
there are lots of people that stumble into this, so I think it might be good to get this to work as soon as possible and issue a bugfix release. |
gentle ping, @shoyer. Any updates on this? |
65f6b9a
to
ef98c32
Compare
it seems we have to cast because |
it seems
this was introduced in |
I am +1 on merging this quickly and issuing a bugfix release. We can always make cleanups later... |
Co-authored-by: keewis <keewis@users.noreply.github.com>
OK, submitting this! Thanks @keewis for making this fix actually work :) |
Does this fix #4363? |
No, that seems to be unrelated |
I can do a patch release this weekend |
Fixes #4283
The underlying issue is that calling
.item()
on a NumPy array withdtype=datetime64[ns]
returns an integer, rather than annp.datetime64
scalar. This is somewhat baffling but works this way because
.item()
returns native Python types, but
datetime.datetime
doesn't supportnanosecond precision.
pandas.Index.get_loc
used to support these integers, but now is more strict.Hence we get errors.
We can fix this by using
array[()]
to convert 0d arrays into NumPy scalarsinstead of calling
array.item()
.I've added a crude regression test. There may well be a better way to test this
but I haven't figured it out yet.
isort . && black . && mypy . && flake8