Getting indices of ascending order of list
Question:
I know that this question has been asked a hundred times, but the answer always seems to be “use numpy’s argsort”. But either I am misinterpreting what most people are asking, or the answers are not correct for the question. Whatever be the case, I wish to get indices of a list’s ascending order. The phrasing is confusing, so as an example, given a list [4, 2, 1, 3] I expect to get a list back [3, 1, 0, 2]. The smallest item is 1, so it gets index 0, the largest one is 4 so it gets index 3. It seems to me that argsort is often suggested, but it just doesn’t seem to do that.
from numpy import argsort
l = [4, 2, 1, 3]
print(argsort(l))
# [2, 1, 3, 0]
# Expected [3, 1, 0, 2]
Clearly argsort is doing something else, so what is it actually doing and how is it similar to the expected behaviour so that it is so often (wrongly) suggested? And, more importantly, how can I get the desired output?
Answers:
The argsort() is basically converting your list to a sorted list of indices.
l = [4, 2, 1, 3]
First it gets index of each element in the list so new list becomes:
indexed=[0, 1, 2, 3]
Then it sorts the indexed list according to the items in the original list. As 4:0 , 2:1 , 1:2 and 3:3 where : means "corresponds to".
Sorting the original list we get
l=[1, 2, 3, 4]
And placing values of each corresponding index of old list
new=[2,1,3,0]
So basically it sorts the indices of a list according to the original list.
The reason why you are not getting the ‘right,’ or expected, answer is because you are asking the wrong question!
What you are after is the element rank after sort while Numpy’s argsort() returns the sorted index list, as documented!. These are not the same thing (as you found out 😉 )!
@hpaulj answered me correctly, but in a comment. And you can’t see him.
His answer helped me a lot, it allows me to get what I want.
import numpy as np
l = [4, 2, 1, 3]
print(np.argsort(np.argsort(l)))
Return:
[3, 1, 0, 2]
This is what you expect. This method returns the indices for the array if it were sorted.
⚠️ But note that if the input array contains repetitions, then there is an interesting effect:
import numpy as np
l = [4, 2, 1, 3, 4]
print(np.argsort(np.argsort(l)))
Return:
[3 1 0 2 4]
He may not harm you, but he does harm to me. I solve this problem like this:
import numpy as np
l = [4, 2, 1, 3, 4]
ret2 = np.vectorize(lambda val: np.searchsorted(np.unique(l), val))(l)
print('Returned', ret2)
print('Expected', [3, 1, 0, 2, 3])
Return:
Returned [3 1 0 2 3]
Expected [3, 1, 0, 2, 3]
True, my solution will be slow due to the vectorize function.
But nothing prevents you from using numba. I haven’t tested it though .
I know that this question has been asked a hundred times, but the answer always seems to be “use numpy’s argsort”. But either I am misinterpreting what most people are asking, or the answers are not correct for the question. Whatever be the case, I wish to get indices of a list’s ascending order. The phrasing is confusing, so as an example, given a list [4, 2, 1, 3] I expect to get a list back [3, 1, 0, 2]. The smallest item is 1, so it gets index 0, the largest one is 4 so it gets index 3. It seems to me that argsort is often suggested, but it just doesn’t seem to do that.
from numpy import argsort
l = [4, 2, 1, 3]
print(argsort(l))
# [2, 1, 3, 0]
# Expected [3, 1, 0, 2]
Clearly argsort is doing something else, so what is it actually doing and how is it similar to the expected behaviour so that it is so often (wrongly) suggested? And, more importantly, how can I get the desired output?
The argsort() is basically converting your list to a sorted list of indices.
l = [4, 2, 1, 3]
First it gets index of each element in the list so new list becomes:
indexed=[0, 1, 2, 3]
Then it sorts the indexed list according to the items in the original list. As 4:0 , 2:1 , 1:2 and 3:3 where : means "corresponds to".
Sorting the original list we get
l=[1, 2, 3, 4]
And placing values of each corresponding index of old list
new=[2,1,3,0]
So basically it sorts the indices of a list according to the original list.
The reason why you are not getting the ‘right,’ or expected, answer is because you are asking the wrong question!
What you are after is the element rank after sort while Numpy’s argsort() returns the sorted index list, as documented!. These are not the same thing (as you found out 😉 )!
@hpaulj answered me correctly, but in a comment. And you can’t see him.
His answer helped me a lot, it allows me to get what I want.
import numpy as np
l = [4, 2, 1, 3]
print(np.argsort(np.argsort(l)))
Return:
[3, 1, 0, 2]
This is what you expect. This method returns the indices for the array if it were sorted.
⚠️ But note that if the input array contains repetitions, then there is an interesting effect:
import numpy as np
l = [4, 2, 1, 3, 4]
print(np.argsort(np.argsort(l)))
Return:
[3 1 0 2 4]
He may not harm you, but he does harm to me. I solve this problem like this:
import numpy as np
l = [4, 2, 1, 3, 4]
ret2 = np.vectorize(lambda val: np.searchsorted(np.unique(l), val))(l)
print('Returned', ret2)
print('Expected', [3, 1, 0, 2, 3])
Return:
Returned [3 1 0 2 3]
Expected [3, 1, 0, 2, 3]
True, my solution will be slow due to the vectorize function.
But nothing prevents you from using numba. I haven’t tested it though .