Remove row comparing two datetime

Question

I have a dataset of New York taxi rides. There are some wrong values in the pickup_datetime and dropoff_datetime, because the dropoff is before than the pickup. How can i compare this two values and drop the row?

Asked By: Sergio

||

Source

Answer 1

You can do with:

import pandas as pd
import numpy as np

df=pd.read_excel("Taxi.xlsx")

#convert to datetime
df["Pickup"]=pd.to_datetime(df["Pickup"])
df["Drop"]=pd.to_datetime(df["Drop"])

#Identify line to remove
df["ToRemove"]=np.where(df["Pickup"]>df["Drop"],1,0)
print(df)

#filter the dataframe if pickup time is higher than drop time
dfClean=df[df["ToRemove"]==0]
print(dfClean)

Result:
df:

                Pickup                Drop  Price
0  2022-01-01 10:00:00 2022-01-01 10:05:00      5
1  2022-01-01 10:20:00 2022-01-01 10:25:00      8
2  2022-01-01 10:40:00 2022-01-01 10:45:00      3
3  2022-01-01 11:00:00 2022-01-01 10:05:00     10
4  2022-01-01 11:20:00 2022-01-01 11:25:00      5
5  2022-01-01 11:40:00 2022-01-01 08:45:00      8
6  2022-01-01 12:00:00 2022-01-01 12:05:00      3
7  2022-01-01 12:20:00 2022-01-01 12:25:00     10
8  2022-01-01 12:40:00 2022-01-01 12:45:00      5
9  2022-01-01 13:00:00 2022-01-01 13:05:00      8
10 2022-01-01 13:20:00 2022-01-01 13:25:00      3

Df Clean

                Pickup                Drop  Price  ToRemove
0  2022-01-01 10:00:00 2022-01-01 10:05:00      5         0
1  2022-01-01 10:20:00 2022-01-01 10:25:00      8         0
2  2022-01-01 10:40:00 2022-01-01 10:45:00      3         0
4  2022-01-01 11:20:00 2022-01-01 11:25:00      5         0
6  2022-01-01 12:00:00 2022-01-01 12:05:00      3         0
7  2022-01-01 12:20:00 2022-01-01 12:25:00     10         0
8  2022-01-01 12:40:00 2022-01-01 12:45:00      5         0
9  2022-01-01 13:00:00 2022-01-01 13:05:00      8         0
10 2022-01-01 13:20:00 2022-01-01 13:25:00      3         0

Answered By: Renaud

Remove row comparing two datetime

Question:

Answers: