how to replace np.nan with blank in numpy array
Question:
How to replace nan in numpy array into blank or empty string. I googled it and it also related nan inside a pandas dataframe instead of numpy array.
Answers:
You can use built-in functions to replace particular values, for example:
import numpy as np
arr = np.array((np.nan, 1, 0, np.nan, -42))
arr[np.isnan(arr)] = -100
print(arr)
The output would be:
array([-100., 1., 0., -100., -42.])
Note: you should be careful about what value you replace np.nan with, as it should be the same type as the array (i.e. if your array is of type str you can replace with an empty string).
Using fillna np methods :
Ex:
df2 = df.fillna("")
You can also convert berween numpy array to dataFram as following:
df = pd.DataFrame(numpy_array)
For more please check following:
https://sparkbyexamples.com/pandas/pandas-replace-nan-with-blank-empty-string/#:~:text=Convert%20Nan%20to%20Empty%20String,in%20the%20Pandas%20DataFrame%20column.
You can use np.where() method to do that in this way:
a = np.array([[nan, 2], [3, nan]])
a = np.where(np.isnan(a), '', a)
print(a)
Output:
[['' '2.0']
['3.0' '']]
Process finished with exit code 0
Also if you want to replace it with a number value you could use np.nan_to_num() method:
a = np.array([[nan, 2], [3, nan]])
a = np.nan_to_num(a, nan=0)
print(a)
Output:
[[0. 2.]
[3. 0.]]
Process finished with exit code 0
An array with np.nan will be float dtype (let’s not talk about object dtypes here :))
In [274]: arr = np.array([1,2,np.nan, 4,np.nan])
In [275]: arr
Out[275]: array([ 1., 2., nan, 4., nan])
In [277]: arr[[2,4]]
Out[277]: array([nan, nan])
We can’t replace any value in such array with a string!
In [278]: arr[[2,4]] = ' '
Traceback (most recent call last):
Input In [278] in <cell line: 1>
arr[[2,4]] = ' '
ValueError: could not convert string to float: ''
But if we first convert the float dtype to string:
In [279]: sarr = arr.astype(str)
In [280]: sarr
Out[280]: array(['1.0', '2.0', 'nan', '4.0', 'nan'], dtype='<U32')
In [281]: sarr[[2,4]] = ' '
In [282]: sarr
Out[282]: array(['1.0', '2.0', ' ', '4.0', ' '], dtype='<U32')
In a string dtype array, 'nan' isn’t special, not like it is in a float.
We have to use isnan to identify float nan:
In [283]: np.isnan(arr)
Out[283]: array([False, False, True, False, True])
In [284]: np.nonzero(np.isnan(arr))
Out[284]: (array([2, 4]),)
but use ordinary == to test for string 'nan':
In [285]: sarr = arr.astype(str)
In [286]: sarr == 'nan'
Out[286]: array([False, False, True, False, True])
Several answers suggest pandas – as in:
In [287]: S = pd.Series(arr)
In [288]: S
Out[288]:
0 1.0
1 2.0
2 NaN
3 4.0
4 NaN
dtype: float64
In [289]: S.replace?
In [290]: S.replace(np.nan, ' ')
Out[290]:
0 1.0
1 2.0
2
3 4.0
4
dtype: object
Note though the change dtype – from float to object. In this case, the series contains floats and strings.
In [292]: _.to_numpy()
Out[292]: array([1.0, 2.0, ' ', 4.0, ' '], dtype=object)
While the question only asks how to replace nan’s with blank strings, I suspect the reason for wanting to do this is only regarding the appearance of the printed array. Not because the nan’s in the floating point array are actually an issue.
In that case, you can leave the nan’s in the floating point array and use numpy.set_printoptions(nanstr='') to simply have a blank space printed at any nan location, when you print the array.
See full documentation here (including many other options when printing numpy arrays):
https://numpy.org/doc/stable/reference/generated/numpy.set_printoptions.html
How to replace nan in numpy array into blank or empty string. I googled it and it also related nan inside a pandas dataframe instead of numpy array.
You can use built-in functions to replace particular values, for example:
import numpy as np
arr = np.array((np.nan, 1, 0, np.nan, -42))
arr[np.isnan(arr)] = -100
print(arr)
The output would be:
array([-100., 1., 0., -100., -42.])
Note: you should be careful about what value you replace np.nan with, as it should be the same type as the array (i.e. if your array is of type str you can replace with an empty string).
Using fillna np methods :
Ex:
df2 = df.fillna("")
You can also convert berween numpy array to dataFram as following:
df = pd.DataFrame(numpy_array)
For more please check following:
https://sparkbyexamples.com/pandas/pandas-replace-nan-with-blank-empty-string/#:~:text=Convert%20Nan%20to%20Empty%20String,in%20the%20Pandas%20DataFrame%20column.
You can use np.where() method to do that in this way:
a = np.array([[nan, 2], [3, nan]])
a = np.where(np.isnan(a), '', a)
print(a)
Output:
[['' '2.0']
['3.0' '']]
Process finished with exit code 0
Also if you want to replace it with a number value you could use np.nan_to_num() method:
a = np.array([[nan, 2], [3, nan]])
a = np.nan_to_num(a, nan=0)
print(a)
Output:
[[0. 2.]
[3. 0.]]
Process finished with exit code 0
An array with np.nan will be float dtype (let’s not talk about object dtypes here :))
In [274]: arr = np.array([1,2,np.nan, 4,np.nan])
In [275]: arr
Out[275]: array([ 1., 2., nan, 4., nan])
In [277]: arr[[2,4]]
Out[277]: array([nan, nan])
We can’t replace any value in such array with a string!
In [278]: arr[[2,4]] = ' '
Traceback (most recent call last):
Input In [278] in <cell line: 1>
arr[[2,4]] = ' '
ValueError: could not convert string to float: ''
But if we first convert the float dtype to string:
In [279]: sarr = arr.astype(str)
In [280]: sarr
Out[280]: array(['1.0', '2.0', 'nan', '4.0', 'nan'], dtype='<U32')
In [281]: sarr[[2,4]] = ' '
In [282]: sarr
Out[282]: array(['1.0', '2.0', ' ', '4.0', ' '], dtype='<U32')
In a string dtype array, 'nan' isn’t special, not like it is in a float.
We have to use isnan to identify float nan:
In [283]: np.isnan(arr)
Out[283]: array([False, False, True, False, True])
In [284]: np.nonzero(np.isnan(arr))
Out[284]: (array([2, 4]),)
but use ordinary == to test for string 'nan':
In [285]: sarr = arr.astype(str)
In [286]: sarr == 'nan'
Out[286]: array([False, False, True, False, True])
Several answers suggest pandas – as in:
In [287]: S = pd.Series(arr)
In [288]: S
Out[288]:
0 1.0
1 2.0
2 NaN
3 4.0
4 NaN
dtype: float64
In [289]: S.replace?
In [290]: S.replace(np.nan, ' ')
Out[290]:
0 1.0
1 2.0
2
3 4.0
4
dtype: object
Note though the change dtype – from float to object. In this case, the series contains floats and strings.
In [292]: _.to_numpy()
Out[292]: array([1.0, 2.0, ' ', 4.0, ' '], dtype=object)
While the question only asks how to replace nan’s with blank strings, I suspect the reason for wanting to do this is only regarding the appearance of the printed array. Not because the nan’s in the floating point array are actually an issue.
In that case, you can leave the nan’s in the floating point array and use numpy.set_printoptions(nanstr='') to simply have a blank space printed at any nan location, when you print the array.
See full documentation here (including many other options when printing numpy arrays):
https://numpy.org/doc/stable/reference/generated/numpy.set_printoptions.html