Efficient vector selection in numpy

Question

Is there an efficient numpy mechanism to generate an array of values from a 2D array given a list of indexes into that array?

Specifically, I have a list of 2D coordinates that represent interesting values in a 2D numpy array. I calculate those coordinates as follows:

nonzeroValidIndices = numpy.where((array2d != noDataValue) & (array2d != 0))
nonzeroValidCoordinates = zip(nonzeroValidIndices[0],nonzeroValidIndices[1])

From there, I’m building a map by looping over the coordinates and indexing into the numpy array one at a time similarly to this simplified example:

for coord in nonzeroValidCoordinates:       
    map[coord] = array2d[coord]

I have several massive datasets I’m iterating this algorithm over so I’m interested in an efficient solution. Through profiling, I suspect that array2d[coord] line is causing some pain. Is there a better vector form to generate an entire vector of values from array2d or am I stuck with indexing one at a time?

Asked By: Rich

||

Source

Answer 1

How about something like this:

a = np.arange(100).reshape((10,10))
ii = np.where(a > 27) # your nonzeroValidIndices
b = np.zeros_like(a) # your map
b[ii] = a[ii]

You can use the result of np.where to index an array as I show above. This should accomplish something similar to what you’re doing without looping, but I’m not entirely clear what your target 2D array is actually suppose to be from your question. Not knowing what map is, it seems just like you’re copying data over into the same sized array.

Answered By: JoshAdel

Answer 2

I think you could try something like:

array2d[ix_(nonzeroValidIndices[0],nonzeroValidIndices[1])]

Or if you really want to use nonzeroValidCoordinates:

unzip = lambda l: [list(li) for li in zip(*l)]
array2d[ix_(unzip(nonzeroValidCoordinates))]

Source: Link

Answered By: Simon

Answer 3

Yes, of course, you can get the values as

nonZeroData = array2d[nonzeroValidIndices]

if map is a new dict, you could do

map = dict(zip(nonzeroValidCoordinates,nonZeroData))

If it is an existing dict,

map.update(zip(nonzeroValidCoordinates,nonZeroData))

If it is an array, then

map[nonzeroValidIndices] = nonZeroData

Answered By: highBandWidth

Efficient vector selection in numpy

Question:

Answers: