ValueError: all the input arrays must have same number of dimensions
Question:
I’m having a problem with np.append.
I’m trying to duplicate the last column of 20×361 matrix n_list_converted by using the code below:
n_last = []
n_last = n_list_converted[:, -1]
n_lists = np.append(n_list_converted, n_last, axis=1)
But I get error:
ValueError: all the input arrays must have same number of dimensions
However, I’ve checked the matrix dimensions by doing
print(n_last.shape, type(n_last), n_list_converted.shape, type(n_list_converted))
and I get
(20L,) (20L, 361L)
so the dimensions match? Where is the mistake?
Answers:
(n,) and (n,1) are not the same shape. Try casting the vector to an array by using the [:, None] notation:
n_lists = np.append(n_list_converted, n_last[:, None], axis=1)
Alternatively, when extracting n_last you can use
n_last = n_list_converted[:, -1:]
to get a (20, 1) array.
The reason why you get your error is because a “1 by n” matrix is different from an array of length n.
I recommend using hstack() and vstack() instead.
Like this:
import numpy as np
a = np.arange(32).reshape(4,8) # 4 rows 8 columns matrix.
b = a[:,-1:] # last column of that matrix.
result = np.hstack((a,b)) # stack them horizontally like this:
#array([[ 0, 1, 2, 3, 4, 5, 6, 7, 7],
# [ 8, 9, 10, 11, 12, 13, 14, 15, 15],
# [16, 17, 18, 19, 20, 21, 22, 23, 23],
# [24, 25, 26, 27, 28, 29, 30, 31, 31]])
Notice the repeated “7, 15, 23, 31” column.
Also, notice that I used a[:,-1:] instead of a[:,-1]. My version generates a column:
array([[7],
[15],
[23],
[31]])
Instead of a row array([7,15,23,31])
Edit: append() is much slower. Read this answer.
If I start with a 3×4 array, and concatenate a 3×1 array, with axis 1, I get a 3×5 array:
In [911]: x = np.arange(12).reshape(3,4)
In [912]: np.concatenate([x,x[:,-1:]], axis=1)
Out[912]:
array([[ 0, 1, 2, 3, 3],
[ 4, 5, 6, 7, 7],
[ 8, 9, 10, 11, 11]])
In [913]: x.shape,x[:,-1:].shape
Out[913]: ((3, 4), (3, 1))
Note that both inputs to concatenate have 2 dimensions.
Omit the :, and x[:,-1] is (3,) shape – it is 1d, and hence the error:
In [914]: np.concatenate([x,x[:,-1]], axis=1)
...
ValueError: all the input arrays must have same number of dimensions
The code for np.append is (in this case where axis is specified)
return concatenate((arr, values), axis=axis)
So with a slight change of syntax append works. Instead of a list it takes 2 arguments. It imitates the list append is syntax, but should not be confused with that list method.
In [916]: np.append(x, x[:,-1:], axis=1)
Out[916]:
array([[ 0, 1, 2, 3, 3],
[ 4, 5, 6, 7, 7],
[ 8, 9, 10, 11, 11]])
np.hstack first makes sure all inputs are atleast_1d, and then does concatenate:
return np.concatenate([np.atleast_1d(a) for a in arrs], 1)
So it requires the same x[:,-1:] input. Essentially the same action.
np.column_stack also does a concatenate on axis 1. But first it passes 1d inputs through
array(arr, copy=False, subok=True, ndmin=2).T
This is a general way of turning that (3,) array into a (3,1) array.
In [922]: np.array(x[:,-1], copy=False, subok=True, ndmin=2).T
Out[922]:
array([[ 3],
[ 7],
[11]])
In [923]: np.column_stack([x,x[:,-1]])
Out[923]:
array([[ 0, 1, 2, 3, 3],
[ 4, 5, 6, 7, 7],
[ 8, 9, 10, 11, 11]])
All these ‘stacks’ can be convenient, but in the long run, it’s important to understand dimensions and the base np.concatenate. Also know how to look up the code for functions like this. I use the ipython ?? magic a lot.
And in time tests, the np.concatenate is noticeably faster – with a small array like this the extra layers of function calls makes a big time difference.
You can also cast (n,) to (n,1) by enclosing within brackets [ ].
e.g. Instead of np.append(b,a,axis=0) use np.append(b,[a],axis=0)
a=[1,2]
b=[[5,6],[7,8]]
np.append(b,[a],axis=0)
returns
array([[5, 6],
[7, 8],
[1, 2]])
I’m having a problem with np.append.
I’m trying to duplicate the last column of 20×361 matrix n_list_converted by using the code below:
n_last = []
n_last = n_list_converted[:, -1]
n_lists = np.append(n_list_converted, n_last, axis=1)
But I get error:
ValueError: all the input arrays must have same number of dimensions
However, I’ve checked the matrix dimensions by doing
print(n_last.shape, type(n_last), n_list_converted.shape, type(n_list_converted))
and I get
(20L,) (20L, 361L)
so the dimensions match? Where is the mistake?
(n,) and (n,1) are not the same shape. Try casting the vector to an array by using the [:, None] notation:
n_lists = np.append(n_list_converted, n_last[:, None], axis=1)
Alternatively, when extracting n_last you can use
n_last = n_list_converted[:, -1:]
to get a (20, 1) array.
The reason why you get your error is because a “1 by n” matrix is different from an array of length n.
I recommend using hstack() and vstack() instead.
Like this:
import numpy as np
a = np.arange(32).reshape(4,8) # 4 rows 8 columns matrix.
b = a[:,-1:] # last column of that matrix.
result = np.hstack((a,b)) # stack them horizontally like this:
#array([[ 0, 1, 2, 3, 4, 5, 6, 7, 7],
# [ 8, 9, 10, 11, 12, 13, 14, 15, 15],
# [16, 17, 18, 19, 20, 21, 22, 23, 23],
# [24, 25, 26, 27, 28, 29, 30, 31, 31]])
Notice the repeated “7, 15, 23, 31” column.
Also, notice that I used a[:,-1:] instead of a[:,-1]. My version generates a column:
array([[7],
[15],
[23],
[31]])
Instead of a row array([7,15,23,31])
Edit: append() is much slower. Read this answer.
If I start with a 3×4 array, and concatenate a 3×1 array, with axis 1, I get a 3×5 array:
In [911]: x = np.arange(12).reshape(3,4)
In [912]: np.concatenate([x,x[:,-1:]], axis=1)
Out[912]:
array([[ 0, 1, 2, 3, 3],
[ 4, 5, 6, 7, 7],
[ 8, 9, 10, 11, 11]])
In [913]: x.shape,x[:,-1:].shape
Out[913]: ((3, 4), (3, 1))
Note that both inputs to concatenate have 2 dimensions.
Omit the :, and x[:,-1] is (3,) shape – it is 1d, and hence the error:
In [914]: np.concatenate([x,x[:,-1]], axis=1)
...
ValueError: all the input arrays must have same number of dimensions
The code for np.append is (in this case where axis is specified)
return concatenate((arr, values), axis=axis)
So with a slight change of syntax append works. Instead of a list it takes 2 arguments. It imitates the list append is syntax, but should not be confused with that list method.
In [916]: np.append(x, x[:,-1:], axis=1)
Out[916]:
array([[ 0, 1, 2, 3, 3],
[ 4, 5, 6, 7, 7],
[ 8, 9, 10, 11, 11]])
np.hstack first makes sure all inputs are atleast_1d, and then does concatenate:
return np.concatenate([np.atleast_1d(a) for a in arrs], 1)
So it requires the same x[:,-1:] input. Essentially the same action.
np.column_stack also does a concatenate on axis 1. But first it passes 1d inputs through
array(arr, copy=False, subok=True, ndmin=2).T
This is a general way of turning that (3,) array into a (3,1) array.
In [922]: np.array(x[:,-1], copy=False, subok=True, ndmin=2).T
Out[922]:
array([[ 3],
[ 7],
[11]])
In [923]: np.column_stack([x,x[:,-1]])
Out[923]:
array([[ 0, 1, 2, 3, 3],
[ 4, 5, 6, 7, 7],
[ 8, 9, 10, 11, 11]])
All these ‘stacks’ can be convenient, but in the long run, it’s important to understand dimensions and the base np.concatenate. Also know how to look up the code for functions like this. I use the ipython ?? magic a lot.
And in time tests, the np.concatenate is noticeably faster – with a small array like this the extra layers of function calls makes a big time difference.
You can also cast (n,) to (n,1) by enclosing within brackets [ ].
e.g. Instead of np.append(b,a,axis=0) use np.append(b,[a],axis=0)
a=[1,2]
b=[[5,6],[7,8]]
np.append(b,[a],axis=0)
returns
array([[5, 6],
[7, 8],
[1, 2]])