How do I translate an ISO 8601 datetime string into a Python datetime object?
Question:
I’m getting a datetime string in a format like “2009-05-28T16:15:00” (this is ISO 8601, I believe). One hackish option seems to be to parse the string using time.strptime
and passing the first six elements of the tuple into the datetime constructor, like:
datetime.datetime(*time.strptime("2007-03-04T21:08:12", "%Y-%m-%dT%H:%M:%S")[:6])
I haven’t been able to find a “cleaner” way of doing this. Is there one?
Answers:
Since Python 3.7 and no external libraries, you can use the fromisoformat function from the datetime module:
datetime.datetime.fromisoformat('2019-01-04T16:41:24+02:00')
Python 2 doesn’t support the %z
format specifier, so it’s best to explicitly use Zulu time everywhere if possible:
datetime.datetime.strptime("2007-03-04T21:08:12Z", "%Y-%m-%dT%H:%M:%SZ")
I haven’t tried it yet, but pyiso8601 promises to support this.
I prefer using the dateutil library for timezone handling and generally solid date parsing. If you were to get an ISO 8601
string like: 2010-05-08T23:41:54.000Z
you’d have a fun time parsing that with strptime, especially if you didn’t know up front whether or not the timezone was included. pyiso8601
has a couple of issues (check their tracker) that I ran into during my usage and it hasn’t been updated in a few years. dateutil, by contrast, has been active and worked for me:
from dateutil import parser
yourdate = parser.parse(datestring)
Isodate seems to have the most complete support.
import datetime, time
def convert_enddate_to_seconds(self, ts):
"""Takes ISO 8601 format(string) and converts into epoch time."""
dt = datetime.datetime.strptime(ts[:-7],'%Y-%m-%dT%H:%M:%S.%f')+
datetime.timedelta(hours=int(ts[-5:-3]),
minutes=int(ts[-2:]))*int(ts[-6:-5]+'1')
seconds = time.mktime(dt.timetuple()) + dt.microsecond/1000000.0
return seconds
This also includes the milliseconds and time zone.
If the time is ‘2012-09-30T15:31:50.262-08:00’, this will convert into epoch time.
>>> import datetime, time
>>> ts = '2012-09-30T15:31:50.262-08:00'
>>> dt = datetime.datetime.strptime(ts[:-7],'%Y-%m-%dT%H:%M:%S.%f')+ datetime.timedelta(hours=int(ts[-5:-3]), minutes=int(ts[-2:]))*int(ts[-6:-5]+'1')
>>> seconds = time.mktime(dt.timetuple()) + dt.microsecond/1000000.0
>>> seconds
1348990310.26
Arrow looks promising for this:
>>> import arrow
>>> arrow.get('2014-11-13T14:53:18.694072+00:00').datetime
datetime.datetime(2014, 11, 13, 14, 53, 18, 694072, tzinfo=tzoffset(None, 0))
Arrow is a Python library that provides a sensible, intelligent way of creating, manipulating, formatting and converting dates and times. Arrow is simple, lightweight and heavily inspired by moment.js and requests.
You should keep an eye on the timezone information, as you might get into trouble when comparing non-tz-aware datetimes with tz-aware ones.
It’s probably the best to always make them tz-aware (even if only as UTC), unless you really know why it wouldn’t be of any use to do so.
#-----------------------------------------------
import datetime
import pytz
import dateutil.parser
#-----------------------------------------------
utc = pytz.utc
BERLIN = pytz.timezone('Europe/Berlin')
#-----------------------------------------------
def to_iso8601(when=None, tz=BERLIN):
if not when:
when = datetime.datetime.now(tz)
if not when.tzinfo:
when = tz.localize(when)
_when = when.strftime("%Y-%m-%dT%H:%M:%S.%f%z")
return _when[:-8] + _when[-5:] # Remove microseconds
#-----------------------------------------------
def from_iso8601(when=None, tz=BERLIN):
_when = dateutil.parser.parse(when)
if not _when.tzinfo:
_when = tz.localize(_when)
return _when
#-----------------------------------------------
Because ISO 8601 allows many variations of optional colons and dashes being present, basically CCYY-MM-DDThh:mm:ss[Z|(+|-)hh:mm]
. If you want to use strptime, you need to strip out those variations first.
The goal is to generate a UTC datetime object.
If you just want a basic case that work for UTC with the Z suffix like 2016-06-29T19:36:29.3453Z
:
datetime.datetime.strptime(timestamp.translate(None, ':-'), "%Y%m%dT%H%M%S.%fZ")
If you want to handle timezone offsets like 2016-06-29T19:36:29.3453-0400
or 2008-09-03T20:56:35.450686+05:00
use the following. These will convert all variations into something without variable delimiters like 20080903T205635.450686+0500
making it more consistent/easier to parse.
import re
# This regex removes all colons and all
# dashes EXCEPT for the dash indicating + or - utc offset for the timezone
conformed_timestamp = re.sub(r"[:]|([-](?!((d{2}[:]d{2})|(d{4}))$))", '', timestamp)
datetime.datetime.strptime(conformed_timestamp, "%Y%m%dT%H%M%S.%f%z" )
If your system does not support the %z
strptime directive (you see something like ValueError: 'z' is a bad directive in format '%Y%m%dT%H%M%S.%f%z'
) then you need to manually offset the time from Z
(UTC). Note %z
may not work on your system in Python versions < 3 as it depended on the C library support which varies across system/Python build type (i.e., Jython, Cython, etc.).
import re
import datetime
# This regex removes all colons and all
# dashes EXCEPT for the dash indicating + or - utc offset for the timezone
conformed_timestamp = re.sub(r"[:]|([-](?!((d{2}[:]d{2})|(d{4}))$))", '', timestamp)
# Split on the offset to remove it. Use a capture group to keep the delimiter
split_timestamp = re.split(r"([+|-])",conformed_timestamp)
main_timestamp = split_timestamp[0]
if len(split_timestamp) == 3:
sign = split_timestamp[1]
offset = split_timestamp[2]
else:
sign = None
offset = None
# Generate the datetime object without the offset at UTC time
output_datetime = datetime.datetime.strptime(main_timestamp +"Z", "%Y%m%dT%H%M%S.%fZ" )
if offset:
# Create timedelta based on offset
offset_delta = datetime.timedelta(hours=int(sign+offset[:-2]), minutes=int(sign+offset[-2:]))
# Offset datetime with timedelta
output_datetime = output_datetime + offset_delta
Here is a super simple way to do these kind of conversions.
No parsing, or extra libraries required.
It is clean, simple, and fast.
import datetime
import time
################################################
#
# Takes the time (in seconds),
# and returns a string of the time in ISO8601 format.
# Note: Timezone is UTC
#
################################################
def TimeToISO8601(seconds):
strKv = datetime.datetime.fromtimestamp(seconds).strftime('%Y-%m-%d')
strKv = strKv + "T"
strKv = strKv + datetime.datetime.fromtimestamp(seconds).strftime('%H:%M:%S')
strKv = strKv +"Z"
return strKv
################################################
#
# Takes a string of the time in ISO8601 format,
# and returns the time (in seconds).
# Note: Timezone is UTC
#
################################################
def ISO8601ToTime(strISOTime):
K1 = 0
K2 = 9999999999
K3 = 0
counter = 0
while counter < 95:
K3 = (K1 + K2) / 2
strK4 = TimeToISO8601(K3)
if strK4 < strISOTime:
K1 = K3
if strK4 > strISOTime:
K2 = K3
counter = counter + 1
return K3
################################################
#
# Takes a string of the time in ISO8601 (UTC) format,
# and returns a python DateTime object.
# Note: returned value is your local time zone.
#
################################################
def ISO8601ToDateTime(strISOTime):
return time.gmtime(ISO8601ToTime(strISOTime))
#To test:
Test = "2014-09-27T12:05:06.9876"
print ("The test value is: " + Test)
Ans = ISO8601ToTime(Test)
print ("The answer in seconds is: " + str(Ans))
print ("And a Python datetime object is: " + str(ISO8601ToDateTime(Test)))
Both ways:
Epoch to ISO time:
isoTime = time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime(epochTime))
ISO time to Epoch:
epochTime = time.mktime(time.strptime(isoTime, '%Y-%m-%dT%H:%M:%SZ'))
I’m getting a datetime string in a format like “2009-05-28T16:15:00” (this is ISO 8601, I believe). One hackish option seems to be to parse the string using time.strptime
and passing the first six elements of the tuple into the datetime constructor, like:
datetime.datetime(*time.strptime("2007-03-04T21:08:12", "%Y-%m-%dT%H:%M:%S")[:6])
I haven’t been able to find a “cleaner” way of doing this. Is there one?
Since Python 3.7 and no external libraries, you can use the fromisoformat function from the datetime module:
datetime.datetime.fromisoformat('2019-01-04T16:41:24+02:00')
Python 2 doesn’t support the %z
format specifier, so it’s best to explicitly use Zulu time everywhere if possible:
datetime.datetime.strptime("2007-03-04T21:08:12Z", "%Y-%m-%dT%H:%M:%SZ")
I haven’t tried it yet, but pyiso8601 promises to support this.
I prefer using the dateutil library for timezone handling and generally solid date parsing. If you were to get an ISO 8601
string like: 2010-05-08T23:41:54.000Z
you’d have a fun time parsing that with strptime, especially if you didn’t know up front whether or not the timezone was included. pyiso8601
has a couple of issues (check their tracker) that I ran into during my usage and it hasn’t been updated in a few years. dateutil, by contrast, has been active and worked for me:
from dateutil import parser
yourdate = parser.parse(datestring)
Isodate seems to have the most complete support.
import datetime, time
def convert_enddate_to_seconds(self, ts):
"""Takes ISO 8601 format(string) and converts into epoch time."""
dt = datetime.datetime.strptime(ts[:-7],'%Y-%m-%dT%H:%M:%S.%f')+
datetime.timedelta(hours=int(ts[-5:-3]),
minutes=int(ts[-2:]))*int(ts[-6:-5]+'1')
seconds = time.mktime(dt.timetuple()) + dt.microsecond/1000000.0
return seconds
This also includes the milliseconds and time zone.
If the time is ‘2012-09-30T15:31:50.262-08:00’, this will convert into epoch time.
>>> import datetime, time
>>> ts = '2012-09-30T15:31:50.262-08:00'
>>> dt = datetime.datetime.strptime(ts[:-7],'%Y-%m-%dT%H:%M:%S.%f')+ datetime.timedelta(hours=int(ts[-5:-3]), minutes=int(ts[-2:]))*int(ts[-6:-5]+'1')
>>> seconds = time.mktime(dt.timetuple()) + dt.microsecond/1000000.0
>>> seconds
1348990310.26
Arrow looks promising for this:
>>> import arrow
>>> arrow.get('2014-11-13T14:53:18.694072+00:00').datetime
datetime.datetime(2014, 11, 13, 14, 53, 18, 694072, tzinfo=tzoffset(None, 0))
Arrow is a Python library that provides a sensible, intelligent way of creating, manipulating, formatting and converting dates and times. Arrow is simple, lightweight and heavily inspired by moment.js and requests.
You should keep an eye on the timezone information, as you might get into trouble when comparing non-tz-aware datetimes with tz-aware ones.
It’s probably the best to always make them tz-aware (even if only as UTC), unless you really know why it wouldn’t be of any use to do so.
#-----------------------------------------------
import datetime
import pytz
import dateutil.parser
#-----------------------------------------------
utc = pytz.utc
BERLIN = pytz.timezone('Europe/Berlin')
#-----------------------------------------------
def to_iso8601(when=None, tz=BERLIN):
if not when:
when = datetime.datetime.now(tz)
if not when.tzinfo:
when = tz.localize(when)
_when = when.strftime("%Y-%m-%dT%H:%M:%S.%f%z")
return _when[:-8] + _when[-5:] # Remove microseconds
#-----------------------------------------------
def from_iso8601(when=None, tz=BERLIN):
_when = dateutil.parser.parse(when)
if not _when.tzinfo:
_when = tz.localize(_when)
return _when
#-----------------------------------------------
Because ISO 8601 allows many variations of optional colons and dashes being present, basically CCYY-MM-DDThh:mm:ss[Z|(+|-)hh:mm]
. If you want to use strptime, you need to strip out those variations first.
The goal is to generate a UTC datetime object.
If you just want a basic case that work for UTC with the Z suffix like 2016-06-29T19:36:29.3453Z
:
datetime.datetime.strptime(timestamp.translate(None, ':-'), "%Y%m%dT%H%M%S.%fZ")
If you want to handle timezone offsets like 2016-06-29T19:36:29.3453-0400
or 2008-09-03T20:56:35.450686+05:00
use the following. These will convert all variations into something without variable delimiters like 20080903T205635.450686+0500
making it more consistent/easier to parse.
import re
# This regex removes all colons and all
# dashes EXCEPT for the dash indicating + or - utc offset for the timezone
conformed_timestamp = re.sub(r"[:]|([-](?!((d{2}[:]d{2})|(d{4}))$))", '', timestamp)
datetime.datetime.strptime(conformed_timestamp, "%Y%m%dT%H%M%S.%f%z" )
If your system does not support the %z
strptime directive (you see something like ValueError: 'z' is a bad directive in format '%Y%m%dT%H%M%S.%f%z'
) then you need to manually offset the time from Z
(UTC). Note %z
may not work on your system in Python versions < 3 as it depended on the C library support which varies across system/Python build type (i.e., Jython, Cython, etc.).
import re
import datetime
# This regex removes all colons and all
# dashes EXCEPT for the dash indicating + or - utc offset for the timezone
conformed_timestamp = re.sub(r"[:]|([-](?!((d{2}[:]d{2})|(d{4}))$))", '', timestamp)
# Split on the offset to remove it. Use a capture group to keep the delimiter
split_timestamp = re.split(r"([+|-])",conformed_timestamp)
main_timestamp = split_timestamp[0]
if len(split_timestamp) == 3:
sign = split_timestamp[1]
offset = split_timestamp[2]
else:
sign = None
offset = None
# Generate the datetime object without the offset at UTC time
output_datetime = datetime.datetime.strptime(main_timestamp +"Z", "%Y%m%dT%H%M%S.%fZ" )
if offset:
# Create timedelta based on offset
offset_delta = datetime.timedelta(hours=int(sign+offset[:-2]), minutes=int(sign+offset[-2:]))
# Offset datetime with timedelta
output_datetime = output_datetime + offset_delta
Here is a super simple way to do these kind of conversions.
No parsing, or extra libraries required.
It is clean, simple, and fast.
import datetime
import time
################################################
#
# Takes the time (in seconds),
# and returns a string of the time in ISO8601 format.
# Note: Timezone is UTC
#
################################################
def TimeToISO8601(seconds):
strKv = datetime.datetime.fromtimestamp(seconds).strftime('%Y-%m-%d')
strKv = strKv + "T"
strKv = strKv + datetime.datetime.fromtimestamp(seconds).strftime('%H:%M:%S')
strKv = strKv +"Z"
return strKv
################################################
#
# Takes a string of the time in ISO8601 format,
# and returns the time (in seconds).
# Note: Timezone is UTC
#
################################################
def ISO8601ToTime(strISOTime):
K1 = 0
K2 = 9999999999
K3 = 0
counter = 0
while counter < 95:
K3 = (K1 + K2) / 2
strK4 = TimeToISO8601(K3)
if strK4 < strISOTime:
K1 = K3
if strK4 > strISOTime:
K2 = K3
counter = counter + 1
return K3
################################################
#
# Takes a string of the time in ISO8601 (UTC) format,
# and returns a python DateTime object.
# Note: returned value is your local time zone.
#
################################################
def ISO8601ToDateTime(strISOTime):
return time.gmtime(ISO8601ToTime(strISOTime))
#To test:
Test = "2014-09-27T12:05:06.9876"
print ("The test value is: " + Test)
Ans = ISO8601ToTime(Test)
print ("The answer in seconds is: " + str(Ans))
print ("And a Python datetime object is: " + str(ISO8601ToDateTime(Test)))
Both ways:
Epoch to ISO time:
isoTime = time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime(epochTime))
ISO time to Epoch:
epochTime = time.mktime(time.strptime(isoTime, '%Y-%m-%dT%H:%M:%SZ'))