pandas cut with infinite upper/lower bounds

Question

The pandas cut() documentation states that: "Out of bounds values will be NA in the resulting Categorical object." This makes it difficult when the upper bound is not necessarily clear or important. For example:

cut (weight, bins=[10,50,100,200])

Will produce the bins:

[(10, 50] < (50, 100] < (100, 200]]

So cut (250, bins=[10,50,100,200]) will produce a NaN, as will cut (5, bins=[10,50,100,200]). What I’m trying to do is produce something like > 200 for the first example and < 10 for the second.

I realize I could do cut (weight, bins=[float("inf"),10,50,100,200,float("inf")]) or the equivalent, but the report style I am following doesn’t allow things like (200, inf]. I realize too I could actually specify custom labels via the labels parameter on cut(), but that means remembering to adjust them every time I adjust bins, which could be often.

Have I exhausted all the possibilities, or is there something in cut() or elsewhere in pandas that would help me do this? I’m thinking about writing a wrapper function for cut() that would automatically generate the labels in desired format from the bins, but I wanted to check here first.

Asked By: sparc_spread

||

Source

Answer 1

After waiting a few days, still no answers posted – I think that’s probably because there really is no way around this other than writing the cut() wrapper function. I am posting my version of it here and marking the question as answered. I will change that if new answers come along.

def my_cut (x, bins,
            lower_infinite=True, upper_infinite=True,
            **kwargs):
    r"""Wrapper around pandas cut() to create infinite lower/upper bounds with proper labeling.

    Takes all the same arguments as pandas cut(), plus two more.

    Args :
        lower_infinite (bool, optional) : set whether the lower bound is infinite
            Default is True. If true, and your first bin element is something like 20, the
            first bin label will be '<= 20' (depending on other cut() parameters)
        upper_infinite (bool, optional) : set whether the upper bound is infinite
            Default is True. If true, and your last bin element is something like 20, the
            first bin label will be '> 20' (depending on other cut() parameters)
        **kwargs : any standard pandas cut() labeled parameters

    Returns :
        out : same as pandas cut() return value
        bins : same as pandas cut() return value
    """

    # Quick passthru if no infinite bounds
    if not lower_infinite and not upper_infinite:
        return pd.cut(x, bins, **kwargs)

    # Setup
    num_labels      = len(bins) - 1
    include_lowest  = kwargs.get("include_lowest", False)
    right           = kwargs.get("right", True)

    # Prepend/Append infinities where indiciated
    bins_final = bins.copy()
    if upper_infinite:
        bins_final.insert(len(bins),float("inf"))
        num_labels += 1
    if lower_infinite:
        bins_final.insert(0,float("-inf"))
        num_labels += 1

    # Decide all boundary symbols based on traditional cut() parameters
    symbol_lower  = "<=" if include_lowest and right else "<"
    left_bracket  = "(" if right else "["
    right_bracket = "]" if right else ")"
    symbol_upper  = ">" if right else ">="

    # Inner function reused in multiple clauses for labeling
    def make_label(i, lb=left_bracket, rb=right_bracket):
        return "{0}{1}, {2}{3}".format(lb, bins_final[i], bins_final[i+1], rb)

    # Create custom labels
    labels=[]
    for i in range(0,num_labels):
        new_label = None

        if i == 0:
            if lower_infinite:
                new_label = "{0} {1}".format(symbol_lower, bins_final[i+1])
            elif include_lowest:
                new_label = make_label(i, lb="[")
            else:
                new_label = make_label(i)
        elif upper_infinite and i == (num_labels - 1):
            new_label = "{0} {1}".format(symbol_upper, bins_final[i])
        else:
            new_label = make_label(i)

        labels.append(new_label)

    # Pass thru to pandas cut()
    return pd.cut(x, bins_final, labels=labels, **kwargs)

Answered By: sparc_spread

Answer 2

You can use float("inf") as the upper bound and -float("inf") as the lower bound in the bins list. It will remove NaN values.

Answered By: Daniel MM. Kamani

Answer 3

just add np.inf, e.g.:

pd.cut(df['weight'], [0, 50, 100, np.inf], labels=['0-50', '50-100', '100-'])

Answered By: AlizaminJ

pandas cut with infinite upper/lower bounds

Question:

Answers: