Map each list value to its corresponding percentile

If I understand you correctly, all you want to do, is to define the percentile this element represents in the array, how much of the array is before that element. as in [1, 2, 3, 4, 5]
should be [0.0, 0.25, 0.5, 0.75, 1.0]

I believe such code will be enough:

def percentileListEdited(List):
    uniqueList = list(set(List))
    increase = 1.0/(len(uniqueList)-1)
    newList = {}
    for index, value in enumerate(uniqueList):
        newList[index] = 0.0 + increase * index
    return [newList[val] for val in List]

I think you want scipy.stats.percentileofscore

Example:

percentileofscore([1, 2, 3, 4], 3)
75.0
percentiles = [percentileofscore(data, i) for i in data]

this might look oversimplyfied but what about this:

def percentile(x):
    pc = float(1)/(len(x)-1)
    return ["%.2f"%(n*pc) for n, i in enumerate(x)]

EDIT:

def percentile(x):
    unique = set(x)
    mapping = {}
    pc = float(1)/(len(unique)-1)
    for n, i in enumerate(unique):
        mapping[i] = "%.2f"%(n*pc)
    return [mapping.get(el) for el in x]

In terms of complexity, I think reptilicus’s answer is not optimal. It takes O(n^2) time.

Here is a solution that takes O(n log n) time.

def list_to_percentiles(numbers):
    pairs = zip(numbers, range(len(numbers)))
    pairs.sort(key=lambda p: p[0])
    result = [0 for i in range(len(numbers))]
    for rank in xrange(len(numbers)):
        original_index = pairs[rank][1]
        result[original_index] = rank * 100.0 / (len(numbers)-1)
    return result

I’m not sure, but I think this is the optimal time complexity you can get. The rough reason I think it’s optimal is because the information of all of the percentiles is essentially equivalent to the information of the sorted list, and you can’t get better than O(n log n) for sorting.

EDIT: Depending on your definition of “percentile” this may not always give the correct result. See BrenBarn’s answer for more explanation and for a better solution which makes use of scipy/numpy.

Pure numpy version of Kevin’s solution

As Kevin said, optimal solution works in O(n log(n)) time. Here is fast version of his code in numpy, which works almost the same time as stats.rankdata:

percentiles = numpy.argsort(numpy.argsort(array)) * 100. / (len(array) - 1)

PS. This is one if my favourite tricks in numpy.

For me the best solution is to use QuantileTransformer in sklearn.preprocessing.

from sklearn.preprocessing import QuantileTransformer
fn = lambda input_list : QuantileTransformer(100).fit_transform(np.array(input_list).reshape([-1,1])).ravel().tolist()
input_raw = [1, 2, 3, 4, 17]
output_perc = fn( input_raw )

print "Input=", input_raw
print "Output=", np.round(output_perc,2)

Here is the output

Input= [1, 2, 3, 4, 17]
Output= [ 0.    0.25  0.5   0.75  1.  ]

Note: this function has two salient features:

1. input raw data is NOT necessarily sorted.
2. input raw data is NOT necessarily single column.

This version allows also to pass exact percentiles values used to ranking:

def what_pctl_number_of(x, a, pctls=np.arange(1, 101)):
    return np.argmax(np.sign(np.append(np.percentile(x, pctls), np.inf) - a))

So it’s possible to find out what’s percentile number value falls for provided percentiles:

_x = np.random.randn(100, 1)
what_pctl_number_of(_x, 1.6, [25, 50, 75, 100])

Output:

3

so it hits to 75 ~ 100 range

for a pure python function to calculate a percentile score for a given item, compared to the population distribution (a list of scores), I pulled this from the scipy source code and removed all references to numpy:

def percentileofscore(a, score, kind='rank'):    
    n = len(a)
    if n == 0:
        return 100.0
    left = len([item for item in a if item < score])
    right = len([item for item in a if item <= score])
    if kind == 'rank':
        pct = (right + left + (1 if right > left else 0)) * 50.0/n
        return pct
    elif kind == 'strict':
        return left / n * 100
    elif kind == 'weak':
        return right / n * 100
    elif kind == 'mean':
        pct = (left + right) / n * 50
        return pct
    else:
        raise ValueError("kind can only be 'rank', 'strict', 'weak' or 'mean'")

source: https://github.com/scipy/scipy/blob/v1.2.1/scipy/stats/stats.py#L1744-L1835

Given that calculating percentiles is trickier than one would think, but way less complicated than the full scipy/numpy/scikit package, this is the best for light-weight deployment. The original code filters for only nonzero-values better, but otherwise, the math is the same. The optional parameter controls how it handles values that are in between two other values.

For this use case, one can call this function for each item in a list using the map() function.

I tried Scipy’s percentile score but it turned out to be very slow for one of my tasks. So, simply implemented it this way. Can be modified if a weak ranking is needed.

def assign_pct(X):
    mp = {}
    X_tmp = np.sort(X)
    pct = []
    cnt = 0
    for v in X_tmp:
        if v in mp:
            continue
        else:
            mp[v] = cnt
            cnt+=1
    for v in X:
        pct.append(mp[v]/cnt)
    return pct        

Calling the function

assign_pct([23,4,1,43,1,6])

Output of function

[0.75, 0.25, 0.0, 1.0, 0.0, 0.5]