How to convert UTC to EST with Python and take care of daylight saving automatically?
Question:
If I have a bunch of data with date & time in UTC format
, how can I convert them to EST
.
It can determine when they will be -4(in summer)
and –5(in winter)
automatically every year?
Thanks
Answers:
You’ll need to use the pytz
module (available from PyPI):
import pytz
from datetime import datetime
est = pytz.timezone('US/Eastern')
utc = pytz.utc
fmt = '%Y-%m-%d %H:%M:%S %Z%z'
winter = datetime(2016, 1, 24, 18, 0, 0, tzinfo=utc)
summer = datetime(2016, 7, 24, 18, 0, 0, tzinfo=utc)
print(winter.strftime(fmt))
print(summer.strftime(fmt))
print(winter.astimezone(est).strftime(fmt))
print(summer.astimezone(est).strftime(fmt))
which will print:
2016-01-24 18:00:00 UTC+0000
2016-07-24 18:00:00 UTC+0000
2016-01-24 13:00:00 EST-0500
2016-07-24 14:00:00 EDT-0400
The reason why you’ll need to use 'US/Eastern'
and not 'EST'
is exemplified in the last two lines of output.
If you have a pandas series with object datatype, you can first convert it into a DateTime series using pd.to_datetime()
df[col] = pd.to_datetime(your_series, format = '%Y-%m-%d %H:%M:%S', errors ='coerce')
Check if it is timezone aware or not by using series.dt.tz
df[col].dt.tz
If it’s not timezone aware, we should make it timezone aware by using series.dt.tz_localize(). Also, do read about the ambiguous and nonexistent parameters of this function
df[col] = your_series[col].dt.tz_localize('UTC')
Now convert this series into the required timezone by series.dt.tz_convert()
df[col] = your_series[col].dt.tz_convert('US/Eastern')
The above method will take care of daylight savings time. If you want to check more timezones you can pip install pytz and
import pytz
pytz.common_timezones
As mentioned above, you can use pandas.DataFrame.tz_convert()
like this:
import pandas as pd
from datetime import datetime
df = pd.read_csv("your_data_file_path.csv", index_col=False, engine='python')
df['Date'] = pd.to_datetime(df['Date'])
df['Date'] = df['Date'].dt.tz_localize('US/Eastern').dt.tz_convert('UTC')
df['Date'] = df['Date'].apply(lambda x: datetime.replace(x, tzinfo=None))
What the last row does is removing the timezone info from the datetime object, so you can operate with the date and time only (don’t worry, that doesn’t change the timezone again, it just strips it from the timestamp string).
In case you just want the normalized hour offset for your existing timedelta shifting:
from datetime import datetime
import pytz
def curr_est_offset():
tz_est = pytz.timezone('US/Eastern')
offset = tz_est.utcoffset(datetime.utcnow())
offset_seconds = (offset.days * 86400) + offset.seconds
offset_hours = offset_seconds // 3600
return offset_hours # -4 or -5
This is thebjorn’s answer converted from Python 2 to Python 3 with a few additional comments. Thanks to thebjorn.
For convention, I use these terms:
- EST: Eastern Standard Time (winter)
- EDT: Eastern Daylight Time
(summer)
- EPT: Eastern Prevailing Time (standard or daylight saving
time as appropriate)
Code:
# Convert EPT / UTC
import pytz
import datetime from datetime as dtdt
# Timezones
ept = pytz.timezone('US/Eastern')
utc = pytz.utc
# str format
fmt = '%Y-%m-%d %H:%M:%S %Z%z'
print("nEPT/UTC examples:")
print("nWinter (EST) example:")
# Create a UTC time in the winter
winter_utc = dtdt(2016, 1, 24, 18, 0, 0, tzinfo=utc)
print(" UTC: ", winter_utc.strftime(fmt))
# Convert from UTC to eastern prevailing time. Since, the timestamp is in the
# winter, prevailing time is standard time.
winter_ept = winter_utc.astimezone(ept)
print(" EPT: ", winter_ept.strftime(fmt))
# Let's convert back to UTC to show we get back to the original value.
winter_utc2 = winter_ept.astimezone(utc)
print(" UTC: ", winter_utc2.strftime(fmt))
# Let's do that again for a summer datetime.
print("nSummer (EDT) example:")
summer_utc = dtdt(2016, 7, 24, 18, 0, 0, tzinfo=utc)
print(" UTC: ", summer_utc.strftime(fmt))
# Convert from UTC to eastern prevailing time. Since, the timestamp is in the
# winter, prevailing time is daylight saving time.
summer_ept = summer_utc.astimezone(ept)
print(" EPT: ", summer_ept.strftime(fmt))
# Let's convert back to UTC to show we get back to the original value.
summer_utc2 = summer_ept.astimezone(utc)
print(" UTC: ", summer_utc2.strftime(fmt))
Console:
EPT/UTC examples:
Winter (EST) example:
UTC: 2016-01-24 18:00:00 UTC+0000
EPT: 2016-01-24 13:00:00 EST-0500
UTC: 2016-01-24 18:00:00 UTC+0000
Summer (EDT) example:
UTC: 2016-07-24 18:00:00 UTC+0000
EPT: 2016-07-24 14:00:00 EDT-0400
UTC: 2016-07-24 18:00:00 UTC+0000
If I have a bunch of data with date & time in UTC format
, how can I convert them to EST
.
It can determine when they will be -4(in summer)
and –5(in winter)
automatically every year?
Thanks
You’ll need to use the pytz
module (available from PyPI):
import pytz
from datetime import datetime
est = pytz.timezone('US/Eastern')
utc = pytz.utc
fmt = '%Y-%m-%d %H:%M:%S %Z%z'
winter = datetime(2016, 1, 24, 18, 0, 0, tzinfo=utc)
summer = datetime(2016, 7, 24, 18, 0, 0, tzinfo=utc)
print(winter.strftime(fmt))
print(summer.strftime(fmt))
print(winter.astimezone(est).strftime(fmt))
print(summer.astimezone(est).strftime(fmt))
which will print:
2016-01-24 18:00:00 UTC+0000
2016-07-24 18:00:00 UTC+0000
2016-01-24 13:00:00 EST-0500
2016-07-24 14:00:00 EDT-0400
The reason why you’ll need to use 'US/Eastern'
and not 'EST'
is exemplified in the last two lines of output.
If you have a pandas series with object datatype, you can first convert it into a DateTime series using pd.to_datetime()
df[col] = pd.to_datetime(your_series, format = '%Y-%m-%d %H:%M:%S', errors ='coerce')
Check if it is timezone aware or not by using series.dt.tz
df[col].dt.tz
If it’s not timezone aware, we should make it timezone aware by using series.dt.tz_localize(). Also, do read about the ambiguous and nonexistent parameters of this function
df[col] = your_series[col].dt.tz_localize('UTC')
Now convert this series into the required timezone by series.dt.tz_convert()
df[col] = your_series[col].dt.tz_convert('US/Eastern')
The above method will take care of daylight savings time. If you want to check more timezones you can pip install pytz and
import pytz
pytz.common_timezones
As mentioned above, you can use pandas.DataFrame.tz_convert()
like this:
import pandas as pd
from datetime import datetime
df = pd.read_csv("your_data_file_path.csv", index_col=False, engine='python')
df['Date'] = pd.to_datetime(df['Date'])
df['Date'] = df['Date'].dt.tz_localize('US/Eastern').dt.tz_convert('UTC')
df['Date'] = df['Date'].apply(lambda x: datetime.replace(x, tzinfo=None))
What the last row does is removing the timezone info from the datetime object, so you can operate with the date and time only (don’t worry, that doesn’t change the timezone again, it just strips it from the timestamp string).
In case you just want the normalized hour offset for your existing timedelta shifting:
from datetime import datetime
import pytz
def curr_est_offset():
tz_est = pytz.timezone('US/Eastern')
offset = tz_est.utcoffset(datetime.utcnow())
offset_seconds = (offset.days * 86400) + offset.seconds
offset_hours = offset_seconds // 3600
return offset_hours # -4 or -5
This is thebjorn’s answer converted from Python 2 to Python 3 with a few additional comments. Thanks to thebjorn.
For convention, I use these terms:
- EST: Eastern Standard Time (winter)
- EDT: Eastern Daylight Time
(summer) - EPT: Eastern Prevailing Time (standard or daylight saving
time as appropriate)
Code:
# Convert EPT / UTC
import pytz
import datetime from datetime as dtdt
# Timezones
ept = pytz.timezone('US/Eastern')
utc = pytz.utc
# str format
fmt = '%Y-%m-%d %H:%M:%S %Z%z'
print("nEPT/UTC examples:")
print("nWinter (EST) example:")
# Create a UTC time in the winter
winter_utc = dtdt(2016, 1, 24, 18, 0, 0, tzinfo=utc)
print(" UTC: ", winter_utc.strftime(fmt))
# Convert from UTC to eastern prevailing time. Since, the timestamp is in the
# winter, prevailing time is standard time.
winter_ept = winter_utc.astimezone(ept)
print(" EPT: ", winter_ept.strftime(fmt))
# Let's convert back to UTC to show we get back to the original value.
winter_utc2 = winter_ept.astimezone(utc)
print(" UTC: ", winter_utc2.strftime(fmt))
# Let's do that again for a summer datetime.
print("nSummer (EDT) example:")
summer_utc = dtdt(2016, 7, 24, 18, 0, 0, tzinfo=utc)
print(" UTC: ", summer_utc.strftime(fmt))
# Convert from UTC to eastern prevailing time. Since, the timestamp is in the
# winter, prevailing time is daylight saving time.
summer_ept = summer_utc.astimezone(ept)
print(" EPT: ", summer_ept.strftime(fmt))
# Let's convert back to UTC to show we get back to the original value.
summer_utc2 = summer_ept.astimezone(utc)
print(" UTC: ", summer_utc2.strftime(fmt))
Console:
EPT/UTC examples: Winter (EST) example: UTC: 2016-01-24 18:00:00 UTC+0000 EPT: 2016-01-24 13:00:00 EST-0500 UTC: 2016-01-24 18:00:00 UTC+0000 Summer (EDT) example: UTC: 2016-07-24 18:00:00 UTC+0000 EPT: 2016-07-24 14:00:00 EDT-0400 UTC: 2016-07-24 18:00:00 UTC+0000