How to filter Pandas dataframe based on grouped multiple column conditions?
Question:
Given the following dataframe:
currency index product price
AUD A STOCK $10.00
AUD A BOND $10.00
AUD B OPTION $11.00
AUD B STOCK $12.00
USD A STOCK $14.00
USD A BOND $11.00
USD A OPTION $19.00
USD B BOND $12.00
For a given currency and given index, if that index & currency contains options, filter out stock and bond rows.
Therefore the expected output will be:
currency index product price
AUD A STOCK $10.00
AUD A BOND $10.00
AUD B OPTION $11.00
USD A OPTION $19.00
USD B BOND $12.00
Answers:
You can make the index & currency group with OPTION
to True then filter the OPTION
out with exclusive OR
m = df['product'].eq('OPTION')
m1 = m.groupby([df['currency'], df['index']]).transform('any')
out = df[~(m != m1)]
# or
out = df[(df.groupby(['currency', 'index'])['product']
.transform(lambda col: col.eq('OPTION').any() == col.eq('OPTION')))]
print(out)
currency index product price
0 AUD A STOCK $10.00
1 AUD A BOND $10.00
2 AUD B OPTION $11.00
6 USD A OPTION $19.00
7 USD B BOND $12.00
Given the following dataframe:
currency index product price
AUD A STOCK $10.00
AUD A BOND $10.00
AUD B OPTION $11.00
AUD B STOCK $12.00
USD A STOCK $14.00
USD A BOND $11.00
USD A OPTION $19.00
USD B BOND $12.00
For a given currency and given index, if that index & currency contains options, filter out stock and bond rows.
Therefore the expected output will be:
currency index product price
AUD A STOCK $10.00
AUD A BOND $10.00
AUD B OPTION $11.00
USD A OPTION $19.00
USD B BOND $12.00
You can make the index & currency group with OPTION
to True then filter the OPTION
out with exclusive OR
m = df['product'].eq('OPTION')
m1 = m.groupby([df['currency'], df['index']]).transform('any')
out = df[~(m != m1)]
# or
out = df[(df.groupby(['currency', 'index'])['product']
.transform(lambda col: col.eq('OPTION').any() == col.eq('OPTION')))]
print(out)
currency index product price
0 AUD A STOCK $10.00
1 AUD A BOND $10.00
2 AUD B OPTION $11.00
6 USD A OPTION $19.00
7 USD B BOND $12.00