Efficient pandas operation to get a new column from a shifted value of another column (with additional rules)

Question

I have a dataframe like the following:

>>> df = pd.DataFrame({'days':['Fri', 'Sat', 'Sun', 'Mon', 'Tues', 'Wed', 'Thurs'], 'values':[1, 2, 3, 4, 5, 6, 7]})
>>> df
    days  values
0    Fri       1
1    Sat       2
2    Sun       3
3    Mon       4
4   Tues       5
5    Wed       6
6  Thurs       7

and I want to get a new column, last_rel_value, which is the previous value in this dataframe for a similar day (where similar means weekday vs weekend). So, for example, the last similar day to Friday would be the prior Thursday, the previous day for Wednesday would be Tuesday, the previous day for Sunday would be the prior Saturday, the previous day for Monday would be last Friday, and the previous day for Saturday would be the prior Sunday. The outcome of this process should be a dataframe like so:

>>> df
    days  values  last_rel_value
0    Fri       1             NaN
1    Sat       2             NaN
2    Sun       3             2.0
3    Mon       4             1.0
4   Tues       5             4.0
5    Wed       6             5.0
6  Thurs       7             6.0

I can do this in a fairly sloppy way by creating a bunch of different columns with varying .shift() calls, and then choosing which column to use in an .apply, but I was wondering if there was a more elegant way to do this (maybe using some generalization of .rolling for example)

Asked By: gammapoint

||

Source

Answer 1

You could try grouping the weekend days and weekdays separately then shifting.

import pandas as pd
df = pd.DataFrame({'days':['Fri', 'Sat', 'Sun', 'Mon', 'Tues', 'Wed', 'Thurs'], 'values':[1, 2, 3, 4, 5, 6, 7]})
df['last_rel_value'] = df.groupby(df.days.isin(['Sat','Sun']))['values'].shift()

Output

    days  values  last_rel_value
0    Fri       1             NaN
1    Sat       2             NaN
2    Sun       3             2.0
3    Mon       4             1.0
4   Tues       5             4.0
5    Wed       6             5.0
6  Thurs       7             6.0

Answered By: Chris

Efficient pandas operation to get a new column from a shifted value of another column (with additional rules)

Question:

Answers: