Numpy way to temporarily remove NaN values from array, with ability to place them back later

Question

I am trying to get a version of an array without the NaN values, with the ability to place them back later. Example:

Array = [1,2,nan,4,5,2,5,6,nan,1,nan,nan,nan,nan,8,7,5,2]
Array_non_nan = [1,2,4,5,2,5,6,1,8,7,5,2]

This can be achieved with array[~np.isnan(array)]. However now I want to be able to remember the locations of the NaN values, and place them back later.

I tried:

array = np.array([nan,  1.,  2.,  3.,  4.,  5., nan,  6.,  7.,  8.,  9.])  
no_nan= array[~np.isnan(array)]      
locs = (np.argwhere(np.isnan(array))).flatten()      
original_array = np.insert(no_nan, obj = locs, values = np.NaN)

However in that case the insertion is unsuccesfull because of a problem with the indexing:

array([nan,  1.,  2.,  3.,  4.,  5.,  6., nan,  7.,  8.,  9.])

I need a vectorized solution, simply writing a loop would be to slow.

Asked By: FeddeHasQuestion

||

Source

Answer 1

Hey have you considered using deepcopy to create two versions of your array ? (one with NaN values, one without)

Answered By: J.Chapuy

Answer 2

Well here is the solution. You store the ids and you can insert it back when needed. However I would higly reccomend the other options the other users are suggesting. Store it somewhere or deepcopy

Array = [1,2,np.nan,4,5,2,5,6,np.nan,1,np.nan,np.nan,np.nan,np.nan,8,7,5,2]
Array_non_nan = [x for x in Array if not np.isnan(x)]
list_id = [id_ for id_,elem_ in enumerate(Array) if np.isnan(elem_) ]
for id_ in list_id:
    Array_non_nan.insert(id_, np.nan)

Answered By: minattosama

Answer 3

You can save the indices where the nan values existed via as a boolean mask. Then use that mask to fill in the the value you want.

a = np.array([nan,  1.,  2.,  3.,  4.,  5., nan,  6.,  7.,  8.,  9.])  
nan_mask = np.isnan(a)
a_no_nan = a[~nan_mask]

So far it is basically what you have. If you want to push the values of a_no_nan to a new array with same shape as a, but keep the nan locations, you can create an empty array, fill it with nan values, then push values using the mask.

c = np.empty_like(a)
c.fill(np.nan)
c[~nan_mask] = a_no_nan
c
# returns:
array([nan,  1.,  2.,  3.,  4.,  5., nan,  6.,  7.,  8.,  9.])

Answered By: James

Numpy way to temporarily remove NaN values from array, with ability to place them back later

Question:

Answers: