How to merge two dataframes using index location and backfill or forward fill for missing rows
Question:
df = 2018-01-03 0.034
2019-05-01 0.095
2020-05-28 0.110
......
2020-05-29 0.697
df.iloc[:,0] = pd.to_datetime(df.iloc[:,0])
df.shape()
(605, 2)
s = pd.Series(pd.date_range(str(df.iloc[:,0].min()), str(df.iloc[:,0].max()), freq='D'),
name= None)
s = s[s.dt.dayofweek.lt(5)]
df2 = s.to_frame()
df2 = 2018-01-03
2018-01-04
2018-01-05
.......
2020-05-28
2020-05-29
df2.shape()
(628, 1)
I am trying to merge the above two dataframes by using index location. Df columns are Date and rates. Shape of df is (605,2). Df2 has only one column, which is the date and has a shape of 628,1
. I want to merge df with df2 and forward or backfill the missing rates. My expected output shape should be 628,2. Can you please suggest how it can be possible by using above code.
Answers:
You can do reindex
df=df.set_index(df.columns[0]).reindex(s, method = 'ffill').reset_index()
df = 2018-01-03 0.034
2019-05-01 0.095
2020-05-28 0.110
......
2020-05-29 0.697
df.iloc[:,0] = pd.to_datetime(df.iloc[:,0])
df.shape()
(605, 2)
s = pd.Series(pd.date_range(str(df.iloc[:,0].min()), str(df.iloc[:,0].max()), freq='D'),
name= None)
s = s[s.dt.dayofweek.lt(5)]
df2 = s.to_frame()
df2 = 2018-01-03
2018-01-04
2018-01-05
.......
2020-05-28
2020-05-29
df2.shape()
(628, 1)
I am trying to merge the above two dataframes by using index location. Df columns are Date and rates. Shape of df is (605,2). Df2 has only one column, which is the date and has a shape of 628,1
. I want to merge df with df2 and forward or backfill the missing rates. My expected output shape should be 628,2. Can you please suggest how it can be possible by using above code.
You can do reindex
df=df.set_index(df.columns[0]).reindex(s, method = 'ffill').reset_index()