Find indices of numpy array based on values in another numpy array

Question

I want to find the indices in a larger array if they match the values of a different, smaller array. Something like new_array below:

import numpy as np
summed_rows = np.random.randint(low=1, high=14, size=9999)
common_sums = np.array([7,10,13])
new_array = np.where(summed_rows == common_sums)

However, this returns:

__main__:1: DeprecationWarning: elementwise comparison failed; this will raise an error in the future. 
>>>new_array 
(array([], dtype=int64),)

The closest I’ve gotten is:

new_array = [np.array(np.where(summed_rows==important_sum)) for important_sum in common_sums[0]]

This gives me a list with three numpy arrays (one for each ‘important sum’), but each is a different length which produces further downstream problems with concatenation and vstacking. To be clear, I do not want to use the line above. I want to use numpy to index into summed_rows. I’ve looked at various answers using numpy.where, numpy.argwhere, and numpy.intersect1d, but am having trouble putting the ideas together. I figured I’m missing something simple and it would be faster to ask.

Thanks in advance for your recommendations!

Asked By: T Walker

||

Source

Answer 1

Taking into account the proposed options on the comments, and adding an extra option with numpy’s in1d option:

>>> import numpy as np
>>> summed_rows = np.random.randint(low=1, high=14, size=9999)
>>> common_sums = np.array([7,10,13])
>>> ind_1 = (summed_rows==common_sums[:,None]).any(0).nonzero()[0]   # Option of @Brenlla
>>> ind_2 = np.where(summed_rows == common_sums[:, None])[1]   # Option of @Ravi Sharma
>>> ind_3 = np.arange(summed_rows.shape[0])[np.in1d(summed_rows, common_sums)]
>>> ind_4 = np.where(np.in1d(summed_rows, common_sums))[0]
>>> ind_5 = np.where(np.isin(summed_rows, common_sums))[0]   # Option of @jdehesa

>>> np.array_equal(np.sort(ind_1), np.sort(ind_2))
True
>>> np.array_equal(np.sort(ind_1), np.sort(ind_3))
True
>>> np.array_equal(np.sort(ind_1), np.sort(ind_4))
True
>>> np.array_equal(np.sort(ind_1), np.sort(ind_5))
True

If you time it, you can see that all of them are quite similar, but @Brenlla’s option is the fastest one

python -m timeit -s 'import numpy as np; np.random.seed(0); a = np.random.randint(low=1, high=14, size=9999); b = np.array([7,10,13])' 'ind_1 = (a==b[:,None]).any(0).nonzero()[0]'
10000 loops, best of 3: 52.7 usec per loop

python -m timeit -s 'import numpy as np; np.random.seed(0); a = np.random.randint(low=1, high=14, size=9999); b = np.array([7,10,13])' 'ind_2 = np.where(a == b[:, None])[1]'
10000 loops, best of 3: 191 usec per loop

python -m timeit -s 'import numpy as np; np.random.seed(0); a = np.random.randint(low=1, high=14, size=9999); b = np.array([7,10,13])' 'ind_3 = np.arange(a.shape[0])[np.in1d(a, b)]'
10000 loops, best of 3: 103 usec per loop

python -m timeit -s 'import numpy as np; np.random.seed(0); a = np.random.randint(low=1, high=14, size=9999); b = np.array([7,10,13])' 'ind_4 = np.where(np.in1d(a, b))[0]'
10000 loops, best of 3: 63 usec per loo

python -m timeit -s 'import numpy as np; np.random.seed(0); a = np.random.randint(low=1, high=14, size=9999); b = np.array([7,10,13])' 'ind_5 = np.where(np.isin(a, b))[0]'
10000 loops, best of 3: 67.1 usec per loop

Answered By: iblasi

Answer 2

Use np.isin:

import numpy as np
summed_rows = np.random.randint(low=1, high=14, size=9999)
common_sums = np.array([7, 10, 13])
new_array = np.where(np.isin(summed_rows, common_sums))

Answered By: jdehesa

Answer 3

For anyone loking for this for not equal numbers in the array but nearest equal value, this is a straight forward way to do the same for not exactly equal values. for huge summed_rows, might be memory intensive.

    import numpy  
    summed_rows = np.random.randint(low=1, high=14, size=9999) 
    common_sums = np.array([7,10,13])
    
    repeat_array = np.repeat(summed_rows, len(common_sums)).reshape(len(summed_rows), len(common_sums)) 
    search_index = np.argmin(np.abs(repeat_array - common_sums), axis=0)

Answered By: MBV

Find indices of numpy array based on values in another numpy array

Question:

Answers: