Pandas filter a list of items present in a column Question: I have a dataframe like as below df = pd.DataFrame({‘text’: ["Hi how","I am fine","Ila say Hi"], ‘tokens’:[[‘Hi’,’how’],[‘I’,’am’,’fine’],[‘Ila’,’say’,’Hi’]], ‘labels’:[[‘A’,’B’],[‘C’,’B’,’A’],[‘D’,’B’,’A’]]}) I would like to do the below a) Filter the df using tokens AND labels column b) Filter based on the values Hi, Ila for tokens …
pythonic equivalent of R double pivot and filter Question: A task I come up against reasonable often is something like the following transformation: from: home_team_id away_team_id home_team away_team 1 1 2 Arsenal Tottenham 2 2 3 Tottenham Chelsea to team value 1 Arsenal 1 2 Tottenham 2 3 Tottenham 2 4 Chelsea 3 in my …
Python filter larger text by quantile Question: Assume I am process a very large text file, I have the following pseudocode xx_valueList = [] lines=[] with line in file: xx_value = calc_xxValue(line) xx_valueList.append(xx_value) lines.append(lines) # get_quantile_value is a function return the cutoff value with a specific quantile precent cut_offvalue = get_quantile_value(xx_valueList, precent=0.05) for line in …
Filter a Python list using Dictionary keys and values Question: GOAL: Filter a list of lists using dictionary as reference in Python 3.8+ CASE USE: When reviewing a nested list — a series of survey responses — filtering out responses based on control questions. In the dictionary, the responses to questions 3 (index 2 in …
Filter rows from a grouped data frame based on string columns Question: I have a data frame grouped by multiple columns but in this example it would be grouped only by Year. Year Animal1 Animal2 0 2002 Dog Mouse,Lion 1 2002 Mouse 2 2002 Lion 3 2002 Duck 4 2010 Dog Cat 5 2010 Cat …
Pandas filter using one column and replace on another column Question: I have a dataframe like as below df = pd.DataFrame( {‘stud_id’ : [101, 101, 101, 101, 101, 101, 101, 101], ‘sub_code’ : [‘CSE01’, ‘CSE02’, ‘CSE03’, ‘CSE06’, ‘CSE05’, ‘CSE04’, ‘CSE07’, ‘CSE08’], ‘marks’ : [‘A’,’B’,’C’,’D’, ‘E’,’F’,’G’,’H’]} ) I would like to do the below a) Filter …
Filtering a numpy.dstack Question: I have a dstack like this: import numpy as np a = np.array((1,2,6)) b = np.array((2,3,4)) c = np.array((8,3,0)) stack = np.dstack((a,b,c)) print(stack) #[[[1 2 8] #[2 3 3] #[6 4 0]]] and I want to filter out the lists where the 2 element is less then 1. Something like this: …
What is the fastest way to filter a pandas time series? Question: What is the fastest way to filter a pandas time series? For now I use boolean masking to filter the time series ts: import time from datetime import datetime import pandas as pd import statistics # create time series idx = pd.date_range(start=’2022-01-01′, end=’2023-01-01′, …
Python pandas drop columns if their partial name is in a list or column in pandas Question: I have the following dataframe called dropthese. | partname | x1 | x2 | x3…. 0 text1_mid1 1 another1_mid2 2 yet_another And another dataframe called df that looks like this. text1_mid1_suffix1 | text1_mid1_suffix2 | … | something_else | …
Why is Python list.remove() not working properly? Question: I have some lists of strings and I’m trying to find the most repeated value for each one of them. For doing so, I recreated a function whose input is the desired list; however, many of those lists contain the following element ‘\N’. (two backlash and a …
