convert time to UTC in pandas

Question

I have multiple csv files, I’ve set DateTime as the index.

df6.set_index("gmtime", inplace=True)
#correct the underscores in old datetime format
df6.index = [" ".join( str(val).split("_")) for val in df6.index]
df6.index = pd.to_datetime(df6.index)

The time was put in GMT, but I think it’s been saved as BST (British summertime) when I set the clock for raspberry pi.
I want to shift the time one hour backwards. When I use

df6.tz_convert(pytz.timezone('utc'))

it gives me below error as it assumes that the time is correct.

Cannot convert tz-naive timestamps, use tz_localize to localize

How can I shift the time to one hour?

Asked By: Sedi

||

Source

Answer 1

Given a column that contains date/time info as string, you would convert to datetime, localize to a time zone (here: Europe/London), then convert to UTC. You can do that before you set as index.

Ex:

import pandas as pd

dti = pd.to_datetime(["2021-09-01"]).tz_localize("Europe/London").tz_convert("UTC")

print(dti) # notice 1 hour shift:
# DatetimeIndex(['2021-08-31 23:00:00+00:00'], dtype='datetime64[ns, UTC]', freq=None)

Note: setting a time zone means that DST is accounted for, i.e. here, during winter you’d have UTC+0 and during summer UTC+1.

Answered By: FObersteiner

Answer 2

To add to FObersteiner’s response (sorry,new user, can’t comment on answers yet):

I’ve noticed that in all the real world situations I’ve run across it (with full dataframes or pandas series instead of just a single date), .tz_localize() and .tz_convert() need to be called slightly differently.

What’s worked for me is

df['column'] = pd.to_datetime(df['column']).dt.tz_localize('Europe/London').dt.tz_convert('UTC')

Without the .dt, I get "index is not a valid DatetimeIndex or PeriodIndex."

Answered By: snakeeyes021

convert time to UTC in pandas

Question:

Answers: