DatetimeProperties.to_pydatetime is deprecated

[VelizarVESSELINOV]

/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/_plotly_utils/basevalidators.py:105: FutureWarning:

The behavior of DatetimeProperties.to_pydatetime is deprecated, in a future version this will return a Series containing python datetime objects instead of an ndarray. To retain the old behavior, call np.array on the result

[sp-pablo-losada]

Same happens here!

[fsadock]

It's a pandas issue. To remove the warning try downgrading to 2.0.3 instead of using the new 2.1.0

[ned2]

After upgrading to the latest Pandas, I'm now seeing this FutureWarning in a bunch of my plots made with Plotly Express that has datetimes. You can suppress the warning in the meantime, but if the behaviour is going to change, Plotly will need to make an appropriate change before this happens.

See the deprecation note at the top here: https://pandas.pydata.org/docs/dev/reference/api/pandas.Series.dt.to_pydatetime.html

It's unfortunate that the recommended approach for restoring the functionality of returning an numpy array is just to wrap the result in np.array, because the FutureWarning will still persist until whenever the behaviour change is implemented. Perhaps an alternative way of generating an array of datetimes could be used instead of using Series.dt.to_pydatetime?

[ned2]

hey @alexcjohnson how thorny do you think this one is? is there a simple change to a different way of producing numpy arrays that could be swapped over? if so I could be up for submitting a PR.

[alexcjohnson]

Hmph yeah frustrating that their recommendation does not silence the warning. We could do the same thing Pandas does in their own code and filter the warning out while wrapping in np.asarray. Feels a little risky to me, what if we end up filtering out something else that still matters? I don't see another good option though. Or we could go for the lower-level thing Pandas added to their code ie use the private ser._values.to_pydatetime() which apparently is not deprecated, but that feels even more dangerous.

[ned2]

The first of those options feels like the best one to me. By using the warnings.catch_warnings() context manager to do the suppressing, it would be localising the risk of a false positive being suppressed to a very small surface area.

Here's a proof of concept for _plotly_utils/basevalidators.py. There's also two other modules I see using Series.dt.to_pydatetime(), but they also look fairly low risk to me.

[alexcjohnson]

Thanks @ned2 - let's go with that. Your POC looks great (but nd -> np?), and let's include the other modules you found as well!

[ned2]

great thanks @alexcjohnson. have raised a PR over at #4379. there were a few failing tests but they seemed unrelated, so we can see what happens in CI.

(ah yes oops, copying error from when I moved the change from the monkey-patched installed plotly package I validated it on.)