pandas df explode and implode to remove specific dict from the list

Question

I have pandas dataframe with multiple columns. On the the column called request_headers is in a format of list of dictionaries, example:

[{"name": "name1", "value": "value1"}, {"name": "name2", "value": "value2"}]

I would like to remove only those elements from that list which do contains specific name. For example with:

blacklist = "name2"

I should get the same dataframe, with all the columns including request_headers, but it’s value (based on the example above) should be:

[{"name": "name1", "value": "value1"}]

How to achieve it ? I’ve tried first to explode, then filter, but was not able to "implode" correctly.

Thanks,

Asked By: user2913139

||

Source

Answer 1

Exploding is expensive, rather us a list comprehension:

blacklist = "name2"

df['request_headers'] = [[d for d in l if 'name' in d and d['name'] != blacklist]
                         for l in df['request_headers']]

Output:

                          request_headers
0  [{'name': 'name1', 'value': 'value1'}]

Answered By: mozway

Answer 2

can use a .apply function:

blacklist = 'name2'
df['request_headers'] = df['request_headers'].apply(lambda x: [d for d in x if blacklist not in d.values()])

Answered By: Jacob Kearney

Answer 3

df1=pd.DataFrame([{"name": "name1", "value": "value1"}, {"name": "name2", "value": "value2"}])
blacklist = "name2"
col1=df1.name.eq(blacklist)
df1.loc[col1]

out：

    name   value
1  name2  value2

Answered By: G.G

pandas df explode and implode to remove specific dict from the list

Question:

Answers: