How to increment duplicate time axis Question: I have a pd.DataFrame which has duplicate time in its index. Example: from datetime import datetime import pandas as pd time_index = [ datetime(2017, 1, 1, 0, 4, 1, 80000), datetime(2017, 1, 1, 0, 4, 1, 80000), datetime(2017, 1, 1, 0, 4, 1, 80000), datetime(2017, 1, 1, 0, …
Pandas Cumcount() over multiple columns Question: I have a dataframe that looks like this: data = {‘exercise’: [‘squat’, ‘squat’, ‘squat’, ‘squat’, ‘bench’, ‘bench’, ‘bench’, ‘bench’, ‘squat’, ‘squat’, ‘squat’, ‘squat’, ‘bench’, ‘bench’, ‘bench’, ‘bench’], ‘session’: [0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1], ‘weight’: [100, 100, 120, 120, …
How can I add a counter column that counts specific values in another column in a pandas dataframe? Question: I have a large dataframe (>16M rows) which has a column named ‘user’. Every user have more than one occurrences. I want to add a new column ‘counter’ that increases every time a specific user has …
Manipulation of a dataframe index on the basis of values from another column Question: Suppose I have a dataframe which currently has data like this: T week 0 T-1 1 T-1 2 T-1 3 T-1 4 T-2 5 T-2 6 T-2 7 T-3 8 T-3 9 T-3 10 T-3 I want to group the index …
Adding Counter on Duplicate Keys for Merging Question: I created a key for merging. Unfortunately, there are some duplicate keys. But I need to keep these rows. I am thinking that for each set of duplicate keys, I can just add counting number 1, 2, 3 and so on to each of the duplicate keys …
How to create rank column in Python based on other columns Question: I have a python dataframe that looks like the following: This dataframe has been sorted in descending order by ‘transaction_count’. I want to create another column in that dataframe called ‘rank’ that contains the count of occurrences of cust_ID. My desired output would …
find equal time and incrementally add a constant Question: I have a dataframe df containing some timestamps df[‘Date’].values Out[16]: array([‘2015-03-25T14:36:39.199994000’, ‘2015-03-25T14:36:39.199994000’, ‘2015-03-26T10:05:03.699999000’, ‘2015-04-19T16:01:49.680009000’, ‘2015-04-19T16:36:10.040007000’, ‘2015-04-19T16:36:10.040007000’, ‘2015-04-19T16:36:10.040007000′], dtype=’datetime64[ns]’) As you can see the first and the second timestamps are equal, but also the last 3. I would like to scan the dataframe and if there are …
Sort by column and append counter using pandas Question: I have a large data frame (>1m rows, 10+ cols) that I need to do the following to: Group by two of the columns (A & B in example) Sort within the grouping by another column (C in example) Append an incremental counter to another column, …
How to add one column according to the order after `groupby` in pandas? Question: My data is like and I want to calculate the NDCG metric pid query title label predict prob 0 1 A a 0 2 0.9 1 1 A b 2 0 0.8 2 1 A c 1 2 0.7 3 2 …
Count duplicate rows and fill in column Question: I have created a DataFrame, and now need to count each duplicate row (by for example df[‘Gender’]. Suppose Gender ‘Male’ occurs twice and Female three times, I need this column to be made: Gender Occurrence Male 1 Male 2 Female 1 Female 2 Female 3 Is there …
