Pandas 0.24.0 TypeError TzInfo

[theophilechevalier]

I just upgraded from pandas 0.23.4 to pandas 0.24.0.

With pandas 0.24.0 and influxdb 5.2.1:

from influxdb import DataFrameClient

client = DataFrameClient(host='localhost', database='any_db')
client.query('select * from table_1 limit 2;')

leads to TypeError: Already tz-aware, use tz_convert to convert.

The same query directly on the database returns:

name: table_1
time                value              sensor
----                -----              ------
1493103600000000000 168.4412670512293  CTA201___DP04____MesF_PV
1493103600000000000 8.964873657408459  CTA201___TE05____MesF_PV

Note that it works great with pandas 0.23.4 for me despite #671

[artemius-x]

I got the same in the following environment:

Python 3.5.2 
influxdb==5.2.1
pandas==0.24.0

Here is what influx client returns:

> select * from filter_cpu
name: filter_cpu
time                host    value
----                ----    -----
1521143918501304146 sensor1 0.6
1521143923581011996 sensor1 0.7
1521143925260967735 sensor1 0.8
1521144515152000716 sensor1 0.9
1521145876055695298 sensor1 0.6

And when I try to do the same query using influxdb python module, my attempt ends up with following exception:

from influxdb import DataFrameClient
client = DataFrameClient(host, port, user, password, dbname)
response = client.query("SELECT * from filter_cpu")

TypeError: Already tz-aware, use tz_convert to convert.

I have fixed the problem locally by replacement invocation of df.index = df.index.tz_localize('UTC') with df.index = 'df.index.tz_convert('UTC') in influxdb/_dataframe_client.py::_to_dataframe() module. May be this could help someone.

[atthom]

I've upgraded my package and fell on the same problem.
I've done the fix given by @artemius-x. Not pretty but it's working!

[clslgrnc]

I also encountered this issue while developing #678, I solved it in a similar way: 659a338

[nickos556]

Hi guys, hitting the same issue. Any ideas when this fix might make it into a release? thanks :)

[Afrojam]

Got it too. When also I have a problem when using InfluxDBClient and casting the points from the result set to a pandas DataFrame where some columns get duplicated (it happens only if a lot of data is fetched).

[carbolymer]

Here's patch for lazy ones, who need this urgently. It can be easily applied using git apply. Credits to @clslgrnc .
influxdb-5.2.1.patch.txt

[tech-inventory]

I checked tz_localize() in pandas,
I found out that the behavior of tz_localize () in pandas has changed.

following is sample codes and works on pandas version 0.23.4.

 a = pd.Series(['2019-02-20T15:00:09.999Z'])
 a = pd.to_datetime(a)
 a.dt.tz_localize('UTC')

# output
0   2019-02-20 15:00:09.999000+00:00
dtype: datetime64[ns, UTC]

I was worried about this.
DataFrameClient was very useful, but it will be unusable on pandas version 0.24.0 and later.

[clslgrnc]

@tech-inventory I have never used panda nor DataFrameClient, do you think #679 solves the problem?

influxdb-python/influxdb/_dataframe_client.py

Lines 204 to 207 in 5da62e9

	df.set_index('time', inplace=True)
	if df.index.tzinfo is None:
	df.index = df.index.tz_localize('UTC')
	df.index.name = None

Would it be better to add tz_convert in an else clause after the tz_localize?

[tech-inventory]

thanks for reply,
I tried #679 , and confirmed it soved probelm.
(because just i did'nt use latest version of influxdb client)

@tech-inventory I have never used panda nor DataFrameClient, do you think #679 solves the problem?

influxdb-python/influxdb/_dataframe_client.py

Lines 204 to 207 in 5da62e9

df.set_index('time', inplace=True)
if df.index.tzinfo is None:
df.index = df.index.tz_localize('UTC')
df.index.name = None
Would it be better to add tz_convert in an else clause after the tz_localize?

[stephanpcpeters]

Just to confirm, also running into this problem on a new box with a fresh install

[etoricky]

df.index = df.index.tz_localize('UTC') with df.index = df.index.tz_convert('UTC') does fix it

[clslgrnc]

df.index = df.index.tz_localize('UTC') with df.index = df.index.tz_convert('UTC') does fix it

Only if your data is localized, and if it is you can just remove the line. Hopefully #679 or #684 will eventually get merged... @xginn8 any progress?

[emrekuecuek]

Hello, I've been dealing with this issue for some time and I didn't want to change an external dist-package. Since I was trying to figure a way out to "convert tz", I assisgned an epoch value as parameter rather than leaving it "None" by default.

rs = client.query(iquery, epoch="ns")

Maybe you can try something like this.
After doing this, I managed to get a return value from query function but didn't have the opportunity to test the differences between difference epoch options. Some of them gives me a date from 1970, even though I am expecting to get an updated one like today.

[clslgrnc]

Should be solved in 5.2.2