pandas save date in ISO format?

Question

I’m trying to generate a Pandas DataFrame where date_range is an index. Then save it to a CSV file so that the dates are written in ISO-8601 format.

import pandas as pd
import numpy as np
from pandas import DataFrame, Series

NumberOfSamples = 10
dates = pd.date_range('20130101',periods=NumberOfSamples,freq='90S')
df3 = DataFrame(index=dates)
df3.to_csv('dates.txt', header=False)

The current output to dates.txt is:

2013-01-01 00:00:00
2013-01-01 00:01:30
2013-01-01 00:03:00
2013-01-01 00:04:30
...................

I’m trying to get it to look like:

2013-01-01T00:00:00Z
2013-01-01T00:01:30Z
2013-01-01T00:03:00Z
2013-01-01T00:04:30Z
....................

Asked By: Borisw37

||

Source

Answer 1

Use datetime.strftime and call map on the index:

In [72]:

NumberOfSamples = 10
import datetime as dt
dates = pd.date_range('20130101',periods=NumberOfSamples,freq='90S')
df3 = pd.DataFrame(index=dates)
df3.index = df3.index.map(lambda x: dt.datetime.strftime(x, '%Y-%m-%dT%H:%M:%SZ'))
df3
Out[72]:
Empty DataFrame
Columns: []
Index: [2013-01-01T00:00:00Z, 2013-01-01T00:01:30Z, 2013-01-01T00:03:00Z, 2013-01-01T00:04:30Z, 2013-01-01T00:06:00Z, 2013-01-01T00:07:30Z, 2013-01-01T00:09:00Z, 2013-01-01T00:10:30Z, 2013-01-01T00:12:00Z, 2013-01-01T00:13:30Z]

Alternatively and better in my view (thanks to @unutbu) you can pass a format specifier to to_csv:

df3.to_csv('dates.txt', header=False, date_format='%Y-%m-%dT%H:%M:%SZ')

Answered By: EdChum

Answer 2

With `pd.Index.strftime`:

If you’re sure that all your dates are UTC, you can hardcode the format:

df3.index = df3.index.strftime('%Y-%m-%dT%H:%M:%SZ')

which gives you 2013-01-01T00:00:00Z and so on. Note that the Z denotes UTC. You can use %Z instead to dynamically get the actual timezone but that won’t work in your toy example because the timestamps are timezone-naive.

With `pd.Timestamp.isoformat` and `pd.Index.map`:

df3.index = df3.index.map(lambda timestamp: timestamp.isoformat())

This gives you 2013-01-01T00:00:00. If you attach a timezone to your dates first (e.g. by passing tz="UTC" to date_range), you’ll get: 2013-01-01T00:00:00+00:00 which also conforms to ISO-8601 but is a different notation. This should work for any dateutil or pytz timezone, leaving no room for ambiguity when clocks switch from daylight saving to standard time.

Answered By: Joooeey

pandas save date in ISO format?

Question:

Answers:

With `pd.Index.strftime`:

With `pd.Timestamp.isoformat` and `pd.Index.map`:

pandas save date in ISO format?

Question:

Answers:

With pd.Index.strftime:

With pd.Timestamp.isoformat and pd.Index.map:

With `pd.Index.strftime`:

With `pd.Timestamp.isoformat` and `pd.Index.map`: