dataframe client writes nothing when using json protocol

[d3banjan]

my environment

$ conda list | grep "pandas\|influx"                      
influxdb                  5.2.2                    pypi_0    pypi
pandas                    0.24.1           py36hf484d3e_0    conda-forge

from influxdb import DataFrameClient
import pandas as pd

df = pd.DataFrame(data=dict(
			a=[0,1,2,3],
			b=[50,51,52,53]),
		index=pd.date_range(
			start=pd.to_datetime('2018-01-01'),
			end=pd.to_datetime('2019-01-01'),
			periods=4))

dfi = DataFrameClient(database='testdb')

json_output = dfi._convert_dataframe_to_json(df,'test-measurement')
line_output = dfi._convert_dataframe_to_lines(df,'test-measurement')
print("dataframe payload:\n", df)
print("json output=", json_output)
print("line output=","\n".join(line_output), sep="\n")

• returns

In [28]: %run ~/Desktop/influx-json-bug.py                                                                                                               
dataframe payload:
                            a   b
2018-01-01 00:00:00+00:00  0  50
2018-05-02 16:00:00+00:00  1  51
2018-09-01 08:00:00+00:00  2  52
2019-01-01 00:00:00+00:00  3  53
json output= []
line output=
test-measurement a=0i,b=50i 1514764800000000000
test-measurement a=1i,b=51i 1525276800000000000
test-measurement a=2i,b=52i 1535788800000000000
test-measurement a=3i,b=53i 1546300800000000000

[richardhanson]

Hi @d3banjan, I had the same issue. It looks like it's a change in the Pandas API that returns an empty list if you call to_dict('record') on an empty DataFrame. Looking into possible solutions, but one that should definitely solve your issue is downgrading Pandas to 0.23.4

[PRemmen]

@richardhanson @d3banjan thanks for the discussion!

Downgrading pandas to 0.23.4 solved problem also for me, however, looking forward to a solution with using latest version of pandas :)

[macd2]

same issue here, after updating pandas influx did not receive any data from DataframeClient.write_points(format='jason'), python program still run without any errors, just no data went trough 0.23.4 solved issue. A error message would be great !

[d3banjan]

Thanks @richardhanson and others for the hint.

I found it interesting that the pandas changelog https://pandas.pydata.org/pandas-docs/stable/whatsnew/v0.24.1.html noted a regression fix in to_dict --

Fixed regression in DataFrame.to_dict() with records orient raising an AttributeError when the DataFrame contained more than 255 columns, or wrongly converting column names that were not valid python identifiers (GH24939, GH24940).

There is a "new regression" (oxymoron!!) probably introduced by the following PR --
pandas-dev/pandas#24965 ! So I will close this.

[nathan-sixnines]

This bug breaks the tutorial example here: https://github.com/influxdata/influxdb-python/blob/master/examples/tutorial_pandas.py

If you try it with the current version of pandas

[d3banjan]

The workarounds that work for me are downgrading pandas to 0.23 or use protocol='line' when writing the dataframe to influx.

[nathan-sixnines]

#738