non-broadcastable output operand with shape (5377,1) doesn't match the broadcast shape (5377,15)

Question

When I want to transform back to original form by using inverse_transform , I get the following error:

   X_train = []
   y_train = []
for i in range(120, data_training.shape[0]):
 X_train.append(data_training[i-120:i])
 y_train.append(data_training[i,0])
X_train , y_train = np.array(X_train) , np.array(y_train)
X_train.shape , y_train.shape
((5377, 120, 15), (5377,))
train_pred=scaler.inverse_transform(train_pred)               #error

---------------------------------------------------------------------------

ValueError                                Traceback (most recent call last)
<ipython-input-44-2819933dc538> in <module>()
      1 #Transformback to original form
----> 2 train_pred=scaler.inverse_transform(train_pred)

/usr/local/lib/python3.7/dist-packages/sklearn/preprocessing/_data.py in inverse_transform(self, X)
    434                         force_all_finite="allow-nan")
    435 
--> 436         X -= self.min_
    437         X /= self.scale_
    438         return X

ValueError: non-broadcastable output operand with shape (5377,1) doesn't match the broadcast shape (5377,15)

I just started out with the artificial intelligence, and don’t understand what this error means.

Could someone please explain what it means and how to fix it?

Asked By: Oussama El khadir

||

Source

Answer 1

This error tells you that NumPy can’t perform element-wise operation on these two arrays.

This happens because, as described here, NumPy uses a set of strict rules:

If the two arrays differ in their number of dimensions, the shape of the one with fewer dimensions is padded with ones on its leading (left) side.
If the shape of the two arrays does not match in any dimension, the array with shape equal to 1 in that dimension is stretched to match the other shape.
If in any dimension the sizes disagree and neither is equal to 1, an error is raised.

As the shapes of your arrays are (5377,1) and (5377,15) – they fall under the second category, so the first array, with the dimension of (5377,1), is stretched to fit the one with shape (5377,15).

Then, as indicated by the -=, you try to assign it back to the one with the shape of (5377,1), which throws an error.

What you can do is:

Use X = X - self.min_ instead of X -= self.min_:

import numpy as np

X = np.random.randn(5377).reshape((5377,1))
min_ = np.random.randn(5377*15).reshape((5377,15))

X = X - min_
X

Alternatively, if X should stay of the initial shape (5377, 1), you can sum over the columns:

import numpy as np

X = np.random.randn(5377).reshape((5377,1))
min_ = np.random.randn(5377*15).reshape((5377,15))

X = (X - min_).sum(axis=1)
X

But you should think if this outcome fits your needs in the perspective of the algorithm you are using.

Cheers

Answered By: Michael

Answer 2

This train_pred must have the same shape as the dataset on which you fitted the scaler. To do the inverse_transform you can extract the needed attributes from your scaler and apply them to your prediction.

mm_scaler = MinMaxScaler()
df_scaled = mm_scaler.fit_transform(df)
#you do you data split and modeling ...

# inverse transform your prediction
mm_scaler_pred = MinMaxScaler()
mm_scaler_pred.min_, mm_scaler_pred.scale_ = mm_scaler.min_[1], mm_scaler.scale_[1]

prediction = mm_scaler_pred.inverse_transform(y_pred)

Here you have a nicely explained solution.

Answered By: Ksyu

non-broadcastable output operand with shape (5377,1) doesn't match the broadcast shape (5377,15)

Question:

Answers: