pandas rounding when converting float to integer
Question:
I’ve got a pandas DataFrame with a float (on decimal) index which I use to look up values (similar to a dictionary). As floats are not exactly the value they are supposed to be multiplied everything by 10 and converted it to integers .astype(int) before setting it as index. However this seems to do a floor instead of rounding. Thus 1.999999999999999992 is converted to 1 instead of 2. Rounding with the pandas.DataFrame.round() method before does not avoid this problem as the values are still stored as floats.
The original idea (which obviously rises a key error) was this:
idx = np.arange(1,3,0.001)
s = pd.Series(range(2000))
s.index=idx
print(s[2.022])
trying with converting to integers:
idx_int = idx*1000
idx_int = idx_int.astype(int)
s.index = idx_int
for i in range(1000,3000):
print(s[i])
the output is always a bit random as the ‘real’ value of an integer can be slightly above or below the wanted value. In this case the index contains two times the value 1000 and does not contain the value 2999.
Answers:
You are right, astype(int) does a conversion toward zero:
‘integer’ or ‘signed’: smallest signed int dtype
from pandas.to_numeric documentation (which is linked from astype() for numeric conversions).
If you want to round, you need to do a float round, and then convert to int:
df.round(0).astype(int)
Use other rounding functions, according your needs.
If I understand right you could just perform the rounding operation followed by converting it to an integer?
s1 = pd.Series([1.2,2.9])
s1 = s1.round().astype(int)
Which gives the output:
0 1
1 3
dtype: int32
In case the data frame contains both, numeric and non-numeric values and you only want to touch numeric fields:
df = df.applymap(lambda x: int(round(x, 0)) if isinstance(x, (int, float)) else x)
There is a potential that NA as a float type exists in the dataframe. so an alternative solution is: df.fillna(0).astype('int')
I’ve got a pandas DataFrame with a float (on decimal) index which I use to look up values (similar to a dictionary). As floats are not exactly the value they are supposed to be multiplied everything by 10 and converted it to integers .astype(int) before setting it as index. However this seems to do a floor instead of rounding. Thus 1.999999999999999992 is converted to 1 instead of 2. Rounding with the pandas.DataFrame.round() method before does not avoid this problem as the values are still stored as floats.
The original idea (which obviously rises a key error) was this:
idx = np.arange(1,3,0.001)
s = pd.Series(range(2000))
s.index=idx
print(s[2.022])
trying with converting to integers:
idx_int = idx*1000
idx_int = idx_int.astype(int)
s.index = idx_int
for i in range(1000,3000):
print(s[i])
the output is always a bit random as the ‘real’ value of an integer can be slightly above or below the wanted value. In this case the index contains two times the value 1000 and does not contain the value 2999.
You are right, astype(int) does a conversion toward zero:
‘integer’ or ‘signed’: smallest signed int dtype
from pandas.to_numeric documentation (which is linked from astype() for numeric conversions).
If you want to round, you need to do a float round, and then convert to int:
df.round(0).astype(int)
Use other rounding functions, according your needs.
If I understand right you could just perform the rounding operation followed by converting it to an integer?
s1 = pd.Series([1.2,2.9])
s1 = s1.round().astype(int)
Which gives the output:
0 1
1 3
dtype: int32
In case the data frame contains both, numeric and non-numeric values and you only want to touch numeric fields:
df = df.applymap(lambda x: int(round(x, 0)) if isinstance(x, (int, float)) else x)
There is a potential that NA as a float type exists in the dataframe. so an alternative solution is: df.fillna(0).astype('int')