Calling round(), ceiling(), floor(), min(), max() in pandas eval

Question

As title says, Is there a way to support round, ceiling, min, max, floor functions in pandas eval.

DataFrame:

import pandas as pd
import numexpr as ne
op_d = {'ID': [1, 2,3],'V':['F','G','H'],'AAA':[0,1,1],'E':[102014,112019,122017] ,'D':['2019/02/04','2019/02/01','2019/01/01'],'DD':['2019-12-01','2016-05-31','2015-02-15'],'CurrentRate':[7.5,2,2],'NoteRate':[2,3,3],'BBB':[0,00,4],'Q1':[2,8,00],'Q2':[3,5,7],'Q3':[5,6,8]}
df = pd.DataFrame(data=op_d)

abs() and sqrt() function works with pandas eval. i.e.

df.eval('TT = abs(sqrt(Q1+Q2)-Q2)',inplace=True)
df

can anyone suggest how to access rest of the functions in eval? I also tried ‘local_dict’ in eval to see if I can define custom functions and call them but it didn’t work.

Note:

Arithmetic operations inside these functions are necessary (i.e. sum, multiplication, div of two columns).
I am aware of issues around usage of ‘eval’ functions and taking necessary measurements.

Asked By: Prish

||

Source

Answer 1

You can’t

DataFrame.eval only supports a limited set of math operations

Arithmetic operations except for the left shift (<<) and right shift
(>>) operators, e.g., df + 2 * pi / s ** 4 % 42 - the_golden_ratio

Math functions: sin, cos, exp, log, expm1, log1p, sqrt, sinh, cosh, tanh, arcsin, arccos, arctan, arccosh, arcsinh, arctanh, abs, arctan2 and log10.

If it’s not on that list you can’t call it beacause “Function calls other than math functions [are not allowed syntax]”

That being said, it might be possible to implement some of those functions in terms of more basic operations. Here I’ve implemented an eval equivalent of np.sign. But IMO that obfuscates the operation far too much and isn’t very useful, so really you need to move away from eval

Answered By: ALollz

Answer 2

With the help of ‘py_expression_eval’ libary, I was able to work out arithmatic operations inside user defined functions.

from py_expression_eval import Parser
parser = Parser()

dct = {'Q1':df['Q1'],'Q2':df['Q2'],'max':max1,'round':getround}
df['check']=parser.parse('round(Q1/Q2)').evaluate(dct)

library source: https://github.com/Axiacore/py-expression-eval

Hope this helps others.

Answered By: Prish

Answer 3

Late answer, but you can do round,min, and max the way you want. If you set the backend engine in eval to Python (although I think it is rather slow). Seems to have been there since Pandas v1.0 at least.

df.eval('TTR = TT.round(2)', engine='python', inplace=True)
df.eval('TT_min = TT.min()', engine='python', inplace=True)
df.eval('TT_max = TT.max()', engine='python', inplace=True)
df

For floor and ceil operations you have to figure something else out (e.g. using apply).

Answered By: Magnus Persson

Calling round(), ceiling(), floor(), min(), max() in pandas eval

Question:

Answers:

You can’t