Pandas add one day to column
Question:
I need to add 1 day to each date I want to get the begining date of the following month eg 2014-01-2014 for the 1st item in the dataframe.
Tried:
montdist['date'] + pd.DateOffset(1)
Which gives me:
TypeError: cannot use a non-absolute DateOffset in datetime/timedelta operations [<DateOffset>]
Have a Dataframe:
Units mondist date
1 6491 0.057785 2013-12-31 00:00:00
2 7377 0.065672 2014-01-31 00:00:00
3 9990 0.088934 2014-02-28 00:00:00
4 10362 0.092245 2014-03-31 00:00:00
5 11271 0.100337 2014-04-30 00:00:00
6 11637 0.103596 2014-05-31 00:00:00
7 10199 0.090794 2014-06-30 00:00:00
8 10486 0.093349 2014-07-31 00:00:00
9 9282 0.082631 2014-08-31 00:00:00
10 8632 0.076844 2014-09-30 00:00:00
11 8204 0.073034 2013-10-31 00:00:00
12 8400 0.074779 2013-11-30 00:00:00
Answers:
Make it a DatetimeIndex first:
pd.DatetimeIndex(montdist['date']) + pd.DateOffset(1)
Note: I think there is a feature request that this could work with date columns…
In action:
In [11]: df = pd.DataFrame([[1, 2], [3, 4]], columns=['A', 'B'])
In [12]: df['date'] = pd.to_datetime(['21-11-2013', '22-11-2013'])
In [13]: pd.DatetimeIndex(df.date) + pd.DateOffset(1)
Out[13]:
<class 'pandas.tseries.index.DatetimeIndex'>
[2013-11-22 00:00:00, 2013-11-23 00:00:00]
Length: 2, Freq: None, Timezone: None
In [14]: pd.DatetimeIndex(df.date) + pd.offsets.Hour(1)
Out[14]:
<class 'pandas.tseries.index.DatetimeIndex'>
[2013-11-21 01:00:00, 2013-11-22 01:00:00]
Length: 2, Freq: None, Timezone: Non
Try to use timedelta():
mondist['shifted_date']=mondist.date + datetime.timedelta(days=1)
As far as I can tell tshift is a bit faster than doing math such as + pd.DateOffset etc. Of course it only applies to Series or Dataframe indices, not columns.. but you could do:
df['newdate'] = pd.Series(index=df.index).tshift(periods=1, freq='D').index
If your df is large, this may shave off half the time – at least it did for me, which is why I’m using it.
No need to turn into an index. Just using .apply() works:
df['newdate'] = pd.to_datetime(df['date']).apply(pd.DateOffset(1))
I think that the cleanest way to do this is a variant of szu’s answer. Pandas has nearly full support datetime built into its functionality, so there is no need to load datetime; instead, if you are already using pandas, create the new column like this:
mondist['shifted_date'] = mondist.date + pd.Timedelta(days=1)
One quick mention. if you are using data-frames and your datatype is datetime64[ns] non indexed, Then I would go as below:
Assuming the date column name is ‘Date to Change by 1’ and you want to change all dates by 1 day.
import time
from datetime import datetime, timedelta, date, time
before
['Date to Change by 1'] = 1/31/2020
df['Date to Change by 1'] = (pd.to_datetime(df['Date to Change by 1']) +
timedelta(1)
After
['Date to Change by 1'] = 2/01/2020
I need to add 1 day to each date I want to get the begining date of the following month eg 2014-01-2014 for the 1st item in the dataframe.
Tried:
montdist['date'] + pd.DateOffset(1)
Which gives me:
TypeError: cannot use a non-absolute DateOffset in datetime/timedelta operations [<DateOffset>]
Have a Dataframe:
Units mondist date
1 6491 0.057785 2013-12-31 00:00:00
2 7377 0.065672 2014-01-31 00:00:00
3 9990 0.088934 2014-02-28 00:00:00
4 10362 0.092245 2014-03-31 00:00:00
5 11271 0.100337 2014-04-30 00:00:00
6 11637 0.103596 2014-05-31 00:00:00
7 10199 0.090794 2014-06-30 00:00:00
8 10486 0.093349 2014-07-31 00:00:00
9 9282 0.082631 2014-08-31 00:00:00
10 8632 0.076844 2014-09-30 00:00:00
11 8204 0.073034 2013-10-31 00:00:00
12 8400 0.074779 2013-11-30 00:00:00
Make it a DatetimeIndex first:
pd.DatetimeIndex(montdist['date']) + pd.DateOffset(1)
Note: I think there is a feature request that this could work with date columns…
In action:
In [11]: df = pd.DataFrame([[1, 2], [3, 4]], columns=['A', 'B'])
In [12]: df['date'] = pd.to_datetime(['21-11-2013', '22-11-2013'])
In [13]: pd.DatetimeIndex(df.date) + pd.DateOffset(1)
Out[13]:
<class 'pandas.tseries.index.DatetimeIndex'>
[2013-11-22 00:00:00, 2013-11-23 00:00:00]
Length: 2, Freq: None, Timezone: None
In [14]: pd.DatetimeIndex(df.date) + pd.offsets.Hour(1)
Out[14]:
<class 'pandas.tseries.index.DatetimeIndex'>
[2013-11-21 01:00:00, 2013-11-22 01:00:00]
Length: 2, Freq: None, Timezone: Non
Try to use timedelta():
mondist['shifted_date']=mondist.date + datetime.timedelta(days=1)
As far as I can tell tshift is a bit faster than doing math such as + pd.DateOffset etc. Of course it only applies to Series or Dataframe indices, not columns.. but you could do:
df['newdate'] = pd.Series(index=df.index).tshift(periods=1, freq='D').index
If your df is large, this may shave off half the time – at least it did for me, which is why I’m using it.
No need to turn into an index. Just using .apply() works:
df['newdate'] = pd.to_datetime(df['date']).apply(pd.DateOffset(1))
I think that the cleanest way to do this is a variant of szu’s answer. Pandas has nearly full support datetime built into its functionality, so there is no need to load datetime; instead, if you are already using pandas, create the new column like this:
mondist['shifted_date'] = mondist.date + pd.Timedelta(days=1)
One quick mention. if you are using data-frames and your datatype is datetime64[ns] non indexed, Then I would go as below:
Assuming the date column name is ‘Date to Change by 1’ and you want to change all dates by 1 day.
import time
from datetime import datetime, timedelta, date, time
before
['Date to Change by 1'] = 1/31/2020
df['Date to Change by 1'] = (pd.to_datetime(df['Date to Change by 1']) +
timedelta(1)
After
['Date to Change by 1'] = 2/01/2020