pandas dataframe how to compare datetime while ignoring the date

Question

The built-in functionality of datetime supports comparing two datetime.datetime objects directly using ‘< > =…’. However, I would like to compare the two datetime with respect to only the hour and minute.

For example, if we have ‘2016-07-01 11:00:00’ and ‘2017-07-01 09:00:00’, I want to say that ‘2016-07-01 11:00:00’ is greater since 11:00 > 9:00. I did not see any built-in functionality that we can use.

Instead, I did things like comparing each row whether

data = data[time_start.hour * 60 + time_start.minute 
            < (data['time'].hour * 60 + data['time'].minute) 
            < time_end.hour * 60 + time_end.minute ]

But there is error:

AttributeError: 'Series' object has no attribute 'hour'

I am using python 2.7, is this also a problem in python 3?

What would be a good way of doing such comparison? Thanks!

Asked By: jack

||

Source

Answer 1

I am using python 2.7, is this also a problem in python 3?

This has nothing to do with the Python version.

If you are using pandas > 0.16 you will need to use the dt accessor:

data['time'].dt.hour and data['time'].dt.minute

For example:

import pandas as pd
import datetime as dt

df = pd.DataFrame({'a': [dt.datetime(2016, 7, 1, 11), dt.datetime(2017, 7, 7, 9)]})
df['b'] = df['a'].dt.hour

print(df)

#                      a   b
#  0 2016-07-01 11:00:00  11
#  1 2017-07-07 09:00:00   9

df = df[df['a'].dt.hour > 10]

print(df)

#                      a   b
#  0 2016-07-01 11:00:00  11

Answered By: DeepSpace

Answer 2

If the input dates is a str and are arranged from yyyy-mm-dd hh:mm:ss, why don’t just compare this as a string.

import pandas as pd
dates = [ '2016-07-01 11:00:00','2016-07-01 13:00:00','2016-07-01 15:00:00']
df = pd.DataFrame(dates,columns=['dates'])

a = (df['dates'].str[-8:] > '09:00:00') & (df['dates'].str[-8:] <= '11:00:00')

print(df [a])

Answered By: Michael H.

Answer 3

I suspect the trap for beginners here is that if you get a string from time with the time only, then you need to make sure that the time for the condition has a zero in front of single hour digits, or it won’t do the comparison. I just lost 2 hours on that…I’m green, very green.
This worked for me

df["just_time"] = df["datecolumn"].dt.strftime('%H:%M:%S')  
df["timepass"] = [1 if i >= opentime and i < closetime else 0 for i in data["just_time"]]

where opentime looks like "09:30:00"

Answered By: Andrew Martin

pandas dataframe how to compare datetime while ignoring the date

Question:

Answers: