Pandas apply casts None dtype to object or float depending on other outputs Question: I would like to control the output dtypes for apply on a row. foo and bar below have multiple outputs. import pandas as pd def foo(x): return x[‘a’] * x[‘b’], None, x[‘a’] > x[‘b’] def bar(x): return x[‘a’] * x[‘b’], None …
Efficiently add a value to a new column in a large DataFrame Question: I have two dataframes, adv_text with about 9,000 rows and events with over 900,000 rows. events is essentially an expanded version of adv_text with about 100 rows per row in adv_text. I want to add three columns from adv_text to events. The …
Pandas :How to improve performance, comparing rows inside groups Question: I have done a python program to compare rows inside groups.But the performances are poor. The data are coming from a Change Data Capture system. For every change, there is a Sequence id , and an Operation number. For an Update operation, there is two …
I am stuck in writing one code. I wanted to sort one column first by integer then combination of integer and string then string Question: I wanted to sort one column first by integer then combination of integer and string then string. Below is my data frame. import pandas as pd df = pd.DataFrame({‘name’: [‘aaa’, …
Replace column values using another pandas dataframe mapping Question: Imagine I have the following dirty data of employee information of their contracts across countries (df1): ID Country Name Job Date Grade 1 CZ John Office 2021-01-01 Senior 1 SK John . 2021-01-01 Assistant 2 AE Peter Carpinter 2000-05-03 3 PE Marcia Cleaner 1989-11-11 ERROR! 3 …
Most efficient way to run matching table (500 x 3) on (1m x 10) pandas dataset Question: The problem in 1 sentence: I am currently using a pandas apply function to replicate an Excel Vlookup, however I feel like this is not the most efficient way, as it takes a long time to process (10 …
Apply varying function for pandas dataframe depending on column arguments being passed Question: I would like to apply a function to each row of a pandas dataframe. Instead of the argument being variable across rows, it’s the function itself that is different for each row depending on the values in its columns. Let’s be more …
using pandas apply lambda with conditions Question: Product Price Timestamp AdjPrice A1 1142 1/3/2022 A1 1148 1/4/2022 A1 1177 1/5/2022 A1 1163 1/6/2022 A1 1160 1/7/2022 I have the above dataframe. The last column (AdjPrice) must be populated by multiplying the Price with a static factor (0.15 for instance), if the Timestamp is less than …
Pandas extract phrases in string that occur in a list Question: I have a data frame with a column text which has strings as shown below text my name is abc xyz is a fruit abc likes per I also have a list of phrases as shown below [‘abc’, ‘fruit’, ‘likes per’] I want to …
Apply a function to dataframe which includes previous row data Question: I have an input dataframe for daily fruit spend which looks like this: spend_df Date Apples Pears Grapes 01/01/22 10 47 0 02/01/22 0 22 3 03/01/22 11 0 3 … For each fruit, I need to apply a function using their respective parameters …
