DateTime object not rendered correctly in plot tooltip

[samjett247]

I noticed that DateTime objects in Python do not render correctly in the plot tooltips when they are included as hover_data arguments in plotly express functions. See example below.

Desired behavior would be the way that IPython renders the DateTime objects using the display function. See below:

The exact datatype of the Timestamp Series is dtype('<M8[ns]'), which is a numpy specific dtype.

[dimucciojonathan]

Where you input your date to the tooltip, I'm assuming you're doing something like:

'Timestamp=' + df['Timestamp']

Try converting the series to a normal array with '.values' like:

'Timestamp=' + df['Timestamp'].values

[samjett247]

I was plotting it via plotly express. However, I tried passing in the Timestamp.values (instead of hover_data =['Timestamp'] I used hover_data=[df['Timestamp'].values]) but got the same result

[ennosigaeus]

Plotly allows you passing in a formatting string when using hover_data (see: https://plotly.com/python/hover-text-and-formatting/):

data = px.scatter(df, x='utc_time', y='name', color='name', hover_data={'utc_time': '|%Y-%m-%d %H:%M:%S.%LZ', ...})

However, I noticed this does not work if df['utc_time'] is an actual DateTime object, which leads to a tooltip with the following output:

I needed to "cast" the DateTime object to string, and only then it worked:

data = px.scatter(df, x='utc_time', y='name', color='name', hover_data={'utc_time': '|%Y-%m-%d %H:%M:%S.%LZ', ...})

It took me a while to figure this out as, intuitively, I would have assumed this transformation to be done implicitly. Maybe this could be added as a feature?

[dpasqualin]

I have a similar problem, definitely a bug.

This is my dataframe:

dtypes for "Start" and "Finish are datetime64[ns, UTC], and this is the tooltip (look at the Year in the Finish column):

I thought it could be related to #2658, but I have plotly 4.11.1 installed and I tried to increase the difference between Start/Stop to more than 10 seconds. I also tried @ennosigaeus suggestion of converting to string, and I tried to convert to python datetime objects as well, nothing worked.

This is the code I'm using to plot the chart:

fig = px.timeline(
    df,
    x_start="Start",
    x_end="Finish",
    y='Task',
    color="Resource",
    hover_data={
        'Start': '|%Y/%m/%d %H:%M:%S',
        'Finish': '|%Y/%m/%d %H:%M:%S'
    }
)

[nicolaskruchten]

Related to/duplicate of #2934 ... it seems like at the moment, using text="End Date" or having "End Date" in hover_data (with px.timeline()) causes type issues.

[nicolaskruchten]

Could someone seeing this issue please post a self-contained, reproducible example of this issue happening with some other function than px.timeline(), so that I can isolate the differences between this issue and #2934?

[nicolaskruchten]

OK, I've got a replicable test case which #3018 resolves:

import plotly.express as px
import pandas as pd

df = px.data.stocks()
df["date"] = pd.to_datetime(df["date"])

fig = px.scatter(df, x="date", y="GOOG", hover_data=dict(date="|%Y"))
fig.show()

Unfortunately, if the column being formatted is not one of x/y/z/base, #3018 doesn't resolve it:

import plotly.express as px
import pandas as pd

df = px.data.stocks()
df["date"] = pd.to_datetime(df["date"])
df["other_date"] = df["date"]

fig = px.scatter(df, x="date", y="GOOG", hover_data=dict(other_date="|%Y"))
fig.show()

This seems to be because customdata is getting serialized differently from x. @jonmmease do you have a sense of why that might be? In the second case here, fig.data[0].customdata (which is what the hovertemplate tries to use) contains very large integers rather than stringified datetimes.

[jonmmease]

@nicolaskruchten, I don't remember the specifics off the top of my head, but I did run into a few limitation with datetime/Timestamps serialization while working on #2955. I think there's a good chance the new serialization logic in that PR would fix this

[nicolaskruchten]

That would be an unexpected bonus :)

[nicolaskruchten]

No such luck. I think it's something special about how customdata gets serialized upstream of JSON serialization.

[jonmmease]

Ok, I don't have any other ideas off the top of my head. Are things already messed up in fig.to_dict()?

[nicolaskruchten]

Are things already messed up in fig.to_dict()?

Yep. I think that whatever logic is used to manage Pandas date columns just isn't applying to customdata.

[nicolaskruchten]

Maybe here https://github.com/plotly/plotly.py/blob/master/packages/python/plotly/_plotly_utils/basevalidators.py#L383 we're not handling the shape of customdata quite right? Inside PX customdata gets set to a pandas DataFrame and right now that ends up in the figure as row-oriented list of lists, which is good, but we're not applying the date-handling logic that something like x gets.

[nicolaskruchten]

Simple example to show the problem:

import plotly.graph_objects as go
import pandas as pd
df = pd.DataFrame(dict(t=["2010-01-01", "2010-01-02"]))
df["t"] = pd.to_datetime(df["t"])
print(go.Figure(go.Scatter(x=df["t"], customdata=df[["t"]]), layout=dict(template="none")).to_json())
print(go.Figure(go.Scatter(x=df["t"], customdata=df["t"]), layout=dict(template="none")).to_json())

the first line is what PX does, the second line shows that customdata can do the same thing as x but only for series-like things, not dataframe-like things.

[samjett247]

Thanks for the fix @nicolaskruchten and @jonmmease . Your help is appreciated!