How can I pivot a dataframe? Question: What is pivot? How do I pivot? Long format to wide format? I’ve seen a lot of questions that ask about pivot tables, even if they don’t know it. It is virtually impossible to write a canonical question and answer that encompasses all aspects of pivoting… But I’m …
Pandas keep the most complete rows Question: Lets say I have a dataframe that has a lot of missing data: df = pd.DataFrame({‘id’: [‘a’,’a’,’b’,’b’,’b’,’c’,’d’,’e’,’e’,’e’], ‘q1’: [1,1,np.NaN,np.NaN,0,np.NaN,1,np.NaN,1,0], ‘q2’: [‘low’,np.NaN,np.NaN,’high’,’low’,’high’,’high’,np.NaN,np.NaN,’low’], ‘q3’: [np.NaN,1,np.NaN,1,0,0,1,0,np.NaN,np.NaN] }) Which looks like this: id q1 q2 q3 0 a 1.0 low NaN 1 a 1.0 NaN 1.0 2 b NaN NaN NaN 3 …
groupby DataFrame by N columns or N rows Question: I’d like to find a general solution to groupby a DataFrame by a specified amount of rows or columns. Example DataFrame: df = pd.DataFrame(0, index=[‘a’, ‘b’, ‘c’, ‘d’, ‘e’, ‘f’], columns=[‘c1’, ‘c2’, ‘c3’, ‘c4’, ‘c5’, ‘c6’, ‘c7’]) c1 c2 c3 c4 c5 c6 c7 a 0 …
How to get groupby sum of multiple columns Question: I have a pandas dataframe which looks like this: index col1 col2 col3 col4 col5 0 a c 1 2 f 1 a c 1 2 f 2 a d 1 2 f 3 b d 1 2 g 4 b e 1 2 g 5 …
How to do a conditional count after groupby on a Pandas Dataframe? Question: I have the following dataframe: key1 key2 0 a one 1 a two 2 b one 3 b two 4 a one 5 c two Now, I want to group the dataframe by the key1 and count the column key2 with the …
How can I group by month from a date field using Python and Pandas? Question: I have a dataframe, df, which is as follows: | date | Revenue | |———–|———| | 6/2/2017 | 100 | | 5/23/2017 | 200 | | 5/20/2017 | 300 | | 6/22/2017 | 400 | | 6/21/2017 | 500 | …
pandas groupby mean with nan Question: I have the following dataframe: date id cars 2012 1 4 2013 1 6 2014 1 NaN 2012 2 10 2013 2 20 2014 2 NaN Now, I want to get the mean of cars over the years for each id ignoring the NaN’s. The result should be like …
Use Pandas groupby() + apply() with arguments Question: I would like to use df.groupby() in combination with apply() to apply a function to each row per group. I normally use the following code, which usually works (note, that this is without groupby()): df.apply(myFunction, args=(arg1,)) With the groupby() I tried the following: df.groupby(‘columnName’).apply(myFunction, args=(arg1,)) However, I …
group by pandas dataframe and select latest in each group Question: How to group values of pandas dataframe and select the latest(by date) from each group? For example, given a dataframe sorted by date: id product date 0 220 6647 2014-09-01 1 220 6647 2014-09-03 2 220 6647 2014-10-16 3 826 3380 2014-11-11 4 826 …
Bar graph from dataframe groupby Question: import pandas as pd import numpy as np import matplotlib.pyplot as plt df = pd.read_csv("arrests.csv") df = df.replace(np.nan,0) df = df.groupby([‘home_team’])[‘arrests’].mean() I’m trying to create a bar graph for dataframe. Under home_team are a bunch of team names. Under arrests are a number of arrests at each date. I’ve …
