Dropping time from datetime <[M8] in Pandas
Question:
So I have a ‘Date’ column in my data frame where the dates have the format like this
0 1998-08-26 04:00:00
If I only want the Year month and day how do I drop the trivial hour?
Answers:
The quickest way is to use DatetimeIndex’s normalize (you first need to make the column a DatetimeIndex):
In [11]: df = pd.DataFrame({"t": pd.date_range('2014-01-01', periods=5, freq='H')})
In [12]: df
Out[12]:
t
0 2014-01-01 00:00:00
1 2014-01-01 01:00:00
2 2014-01-01 02:00:00
3 2014-01-01 03:00:00
4 2014-01-01 04:00:00
In [13]: pd.DatetimeIndex(df.t).normalize()
Out[13]:
<class 'pandas.tseries.index.DatetimeIndex'>
[2014-01-01, ..., 2014-01-01]
Length: 5, Freq: None, Timezone: None
In [14]: df['date'] = pd.DatetimeIndex(df.t).normalize()
In [15]: df
Out[15]:
t date
0 2014-01-01 00:00:00 2014-01-01
1 2014-01-01 01:00:00 2014-01-01
2 2014-01-01 02:00:00 2014-01-01
3 2014-01-01 03:00:00 2014-01-01
4 2014-01-01 04:00:00 2014-01-01
DatetimeIndex also has some other useful attributes, e.g. .year, .month, .day.
From 0.15 they’ll be a dt attribute, so you can access this (and other methods) with:
df.t.dt.normalize()
# equivalent to
pd.DatetimeIndex(df.t).normalize()
Another Possibility is using str.split
df['Date'] = df['Date'].str.split(' ',expand=True)[0]
This should split the ‘Date’ column into two columns marked 0 and 1. Using the whitespace in between the date and time as the split indicator.
Column 0 of the returned dataframe then includes the date, and column 1 includes the time.
Then it sets the ‘Date’ column of your original dataframe to column [0] which should be just the date.
Another option
df['my_date_column'].dt.date
Would give
0 2019-06-15
1 2019-06-15
2 2019-06-15
3 2019-06-15
4 2019-06-15
At read_csv with date_parser
to_date = lambda times : [t[0:10] for t in times]
df = pd.read_csv('input.csv',
parse_dates={date: ['time']},
date_parser=to_date,
index_col='date')
So I have a ‘Date’ column in my data frame where the dates have the format like this
0 1998-08-26 04:00:00
If I only want the Year month and day how do I drop the trivial hour?
The quickest way is to use DatetimeIndex’s normalize (you first need to make the column a DatetimeIndex):
In [11]: df = pd.DataFrame({"t": pd.date_range('2014-01-01', periods=5, freq='H')})
In [12]: df
Out[12]:
t
0 2014-01-01 00:00:00
1 2014-01-01 01:00:00
2 2014-01-01 02:00:00
3 2014-01-01 03:00:00
4 2014-01-01 04:00:00
In [13]: pd.DatetimeIndex(df.t).normalize()
Out[13]:
<class 'pandas.tseries.index.DatetimeIndex'>
[2014-01-01, ..., 2014-01-01]
Length: 5, Freq: None, Timezone: None
In [14]: df['date'] = pd.DatetimeIndex(df.t).normalize()
In [15]: df
Out[15]:
t date
0 2014-01-01 00:00:00 2014-01-01
1 2014-01-01 01:00:00 2014-01-01
2 2014-01-01 02:00:00 2014-01-01
3 2014-01-01 03:00:00 2014-01-01
4 2014-01-01 04:00:00 2014-01-01
DatetimeIndex also has some other useful attributes, e.g. .year, .month, .day.
From 0.15 they’ll be a dt attribute, so you can access this (and other methods) with:
df.t.dt.normalize()
# equivalent to
pd.DatetimeIndex(df.t).normalize()
Another Possibility is using str.split
df['Date'] = df['Date'].str.split(' ',expand=True)[0]
This should split the ‘Date’ column into two columns marked 0 and 1. Using the whitespace in between the date and time as the split indicator.
Column 0 of the returned dataframe then includes the date, and column 1 includes the time.
Then it sets the ‘Date’ column of your original dataframe to column [0] which should be just the date.
Another option
df['my_date_column'].dt.date
Would give
0 2019-06-15
1 2019-06-15
2 2019-06-15
3 2019-06-15
4 2019-06-15
At read_csv with date_parser
to_date = lambda times : [t[0:10] for t in times]
df = pd.read_csv('input.csv',
parse_dates={date: ['time']},
date_parser=to_date,
index_col='date')