How to get rid of the in place FutureWarning when setting an entire column from an array?

Question

In pandas v.1.5.0 a new warning has been added, which is shown, when a column is set from an array of different dtype. The FutureWarning informs about a planned semantic change, when using iloc: the change will be done in-place in future versions. The changelog instructs what to do to get the old behavior, but there is no hint how to handle the situation, when in-place operation is in fact the right choice.

The example from the changelog:

df = pd.DataFrame({'price': [11.1, 12.2]}, index=['book1', 'book2'])
original_prices = df['price']
new_prices = np.array([98, 99])
df.iloc[:, 0] = new_prices
df.iloc[:, 0]

This is the warning, which is printed in pandas 1.5.0:

FutureWarning: In a future version, df.iloc[:, i] = newvals will
attempt to set the values inplace instead of always setting a new
array. To retain the old behavior, use either df[df.columns[i]] = newvals or, if columns are non-unique, df.isetitem(i, newvals)

How to get rid of the warning, if I don’t care about in-place or not, but want to get rid of the warning? Am I supposed to change dtype explicitly? Do I really need to catch the warning every single time I need to use this feature? Isn’t there a better way?

Asked By: lumbric

||

Source

Answer 1

I haven’t found any better way than suppressing the warning using the warnings module:

import numpy as np
import pandas as pd
import warnings

df = pd.DataFrame({"price": [11.1, 12.2]}, index=["book1", "book2"])
original_prices = df["price"]
new_prices = np.array([98, 99])
with warnings.catch_warnings():
    # Setting values in-place is fine, ignore the warning in Pandas >= 1.5.0
    # This can be removed, if Pandas 1.5.0 does not need to be supported any longer.
    # See also: https://stackoverflow.com/q/74057367/859591
    warnings.filterwarnings(
        "ignore",
        category=FutureWarning,
        message=(
            ".*will attempt to set the values inplace instead of always setting a new array. "
            "To retain the old behavior, use either.*"
        ),
    )

    df.iloc[:, 0] = new_prices

df.iloc[:, 0]

Answered By: lumbric

Answer 2

I am just filtering all future warnings for now:

import warnings
warnings.simplefilter("ignore", category=FutureWarning)

Answered By: Rafael de Rezende

Answer 3

Post here because I can’t comment yet.

For now I think I will also suppress the warnings, because I don’t want the old behavior, never expected to use it that way. And the suggested syntax has the danger to trigger the SettingWithCopyWarning warning.

Answered By: Dong Wang

Answer 4

As the changelog states, the warning is printed when setting an entire column from an array with different dtype, so adjusting the dtype is one way to silence it:

df = pd.DataFrame({'price': [11.1, 12.2]}, index=['book1', 'book2'])
original_prices = df['price']
new_prices = np.array([98, 99]).astype(float)
df.iloc[:, 0] = new_prices
df.iloc[:, 0]

Note the additional .astype(float). Not an ideal solution, but a solution.

Answered By: Alperino

Answer 5

I ran into this and not sure why people are suggesting suppressing warnings as a solution. This worked in my case, using my example:

dfin.loc[:, 'Value'] = pd.to_numeric(dfin['Value'], errors='coerce')

changed to

dfin[dfin.columns[9]] = pd.to_numeric(dfin[dfin.columns[9]], errors='coerce')

It requires using index number of the column in question.

Answered By: mdkb

How to get rid of the in place FutureWarning when setting an entire column from an array?

Question:

Answers: