Pandas filter dataframe by time

Question

This is not a duplicate of: filter pandas dataframe by time because the solution offered there doesn’t address the same column type that needs to be filtered.

I have the following dataframe:

i = pd.date_range('2018-04-09', periods=4, freq='1D20min')
ts = pd.DataFrame({'A': [1, 2, 3, 4],
               'B':i})
ts['date'] = pd.to_datetime(ts['B']).dt.date
ts['time'] = pd.to_datetime(ts['B']).dt.time
ts = ts.drop('B', axis = 1)

I want to filter on just the time columns and i tried this:

ts['time'].between_time('0:45', '0:15')

But it doesn’t work. I get the error: TypeError: Index must be DatetimeIndex

Do you have any idea how to do this? thanks

Asked By: steve zissou

||

Source

Answer 1

EDIT: Solution without B column:

If need filter by time column use Series.between:

from datetime import  time

df = ts[ts['time'].between(time(0,15,0), time(0,45,0))]
print (df)
   A                   B        date      time
1  2 2018-04-10 00:20:00  2018-04-10  00:20:00
2  3 2018-04-11 00:40:00  2018-04-11  00:40:00

Original solution with B column:

Create DatetimeIndex if need filter by DataFrame.between_time:

df = ts.set_index('B').between_time('0:15', '0:45')
print (df)
                     A        date      time
B                                           
2018-04-10 00:20:00  2  2018-04-10  00:20:00
2018-04-11 00:40:00  3  2018-04-11  00:40:00

Solution again with DatetimeIndex with DatetimeIndex.indexer_between_time for positions of matched rows and selecting by DataFrame.iloc:

df = ts.iloc[ts.set_index('B').index.indexer_between_time('0:15', '0:45')]
print (df)
   A                   B        date      time
1  2 2018-04-10 00:20:00  2018-04-10  00:20:00
2  3 2018-04-11 00:40:00  2018-04-11  00:40:00

Answered By: jezrael

Pandas filter dataframe by time

Question:

Answers: