ValueError: Shape of passed values is (2270, 2), indices imply (2270, 4)

Following a youtube video (https://www.youtube.com/watch?v=VtRLrQ3Ev-U) about training a model with dataset (https://colab.research.google.com/drive/1UxmeNX_MaIO0ni26cg9H6mtJcRFafWiR?usp=sharing#scrollTo=sOqdGfTza-Gl).

In this code:

X = df[df.columns[3:4]].values
y = df[df.columns[-1]].values

I used the positions "3:4" because is the only column with numeric value (in my own dataset).
After this, i execute the following code:

over = RandomOverSampler()
X, y = over.fit_resample(X, y)
data = np.hstack((X, np.reshape(y, (-1,1))))
transformed_df = pd.DataFrame(data, columns=df.columns)

But when I execute it in Colab, i receive this:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-222-eaff3b92a197> in <module>()
      1 data = np.hstack((X, np.reshape(y, (-1,1))))
----> 2 transformed_df = pd.DataFrame(data, columns=df.columns)

2 frames
/usr/local/lib/python3.7/dist-packages/pandas/core/internals/construction.py in _check_values_indices_shape_match(values, index, columns)
    391         passed = values.shape
    392         implied = (len(index), len(columns))
--> 393         raise ValueError(f"Shape of passed values is {passed}, indices imply {implied}")
    394 
    395 

ValueError: Shape of passed values is (2270, 2), indices imply (2270, 4)

Somebody can help me to fix this?

THanks in advance!

Adding all the information!

Purpose of the IA, I give it a text and based on 1000 lines of my dataset, autodetect if I want to review it or not and classify with a category if its possible.

Dataset example:
Category|Sector|News|review

Malware|Other sector|Joker malware infects over 500 000 Huawei Android devices – More than 500,000 Huawei users have downloaded from the company’s official Android store applications infected with Joker malware that subscribes to premium mobile services.Researchers found ten seemingly harmless apps in AppGallery that contained code for connecting to malicious command and control server to receive configurations and additional components.Masked by functional appsA report from antivirus maker Doctor Web notes that the malicious apps retained their advertised functionality but downloaded components that subscribed users to premium mobile services.To keep users in the dark the infected apps requested access to notifications, which allowed them to intercept confirmation codes delivered over SMS by the subscription service.According to the researchers, the malware could subscribe a user to a maximum of five services, although the threat actor could modify this limitation at any time.The list of malicious applications included virtual keyboards, a camera app, a launcher, an online messenger, a sticker collection, coloring programs, and a game.Most of them came from one developer (Shanxi Kuailaipai Network Technology Co., Ltd.) and two from a different one. These ten apps were downloaded by more than 538,000 Huawei users, Doctor Web says.Doctor Web informed Huawei of these apps and the company removed them from AppGallery. While new users can no longer download them, those that already have the apps running on their devices need to run a manual cleanup. The table below lists the name name of the application and its package:Application name Package name Super Keyboard com.nova.superkeyboard Happy Colour com.colour.syuhgbvcff Fun Color com.funcolor.toucheffects New 2021 Keyboard com.newyear.onekeyboard Camera MX – Photo Video Camera com.sdkfj.uhbnji.dsfeff BeautyPlus Camera com.beautyplus.excetwa.camera Color RollingIcon com.hwcolor.jinbao.rollingicon Funney Meme Emoji com.meme.rouijhhkl Happy Tapping com.tap.tap.duedd All-in-One Messenger com.messenger.sjdoifoThe researchers say that the same modules downloaded by the infected apps in AppGallery were also present in other apps on Google Play, used by other versions of Joker malware. The full list of indicators of compromise is available here.Once active, the malware communicates to its remote server to get the configuration file, which contains a list of tasks, websites for premium services, JavaScript that mimics user interaction.Joker malware’s history goes as far back as 2017 and constantly found its way in apps distributed through Google Play store. In October 2019, Tatyana Shishkova, Android malware analyst at Kaspersky, tweeted about more than 70 compromised apps that had made it into the official store.And the reports about the malware in Google Play kept coming. In early 2020, Google announced that since 2017, it had removed about 1,700 apps infected with Joker.Last February, Joker was still present in the store and it continued to slip past Google’s defenses even in July last year.|0

Full code:

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
import tensorflow as tf
from imblearn.over_sampling import RandomOverSampler

df = pd.read_csv('news_parsed_end.csv', delimiter='|', error_bad_lines=False)

X = df[df.columns[3:4]].values
y = df[df.columns[-1]].values

over = RandomOverSampler()
X, y = over.fit_resample(X, y)
data = np.hstack((X, np.reshape(y, (-1,1))))
transformed_df = pd.DataFrame(data, columns=df.columns)

X_train, X_temp, y_train, y_temp = train_test_split(X, y, test_size=0.6, random_state=1000)
X_valid, X_test, y_valid, y_test = train_test_split(X_temp, y_temp, test_size=0.2, random_state=1000)

model = tf.keras.Sequential([
                             tf.keras.layers.Dense(16, activation='relu'), # if x <= 0 --> 0, x > 0 --> x
                             tf.keras.layers.Dense(16, activation='relu'),
                             tf.keras.layers.Dense(1, activation="sigmoid")
])

model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.005),
              loss=tf.keras.losses.BinaryCrossentropy(),
              metrics=['accuracy'])

model.evaluate(X_train, y_train)
model.evaluate(X_valid, y_valid)

model.fit(X_train, y_train, batch_size=16, epochs=200, validation_data=(X_valid, y_valid))

model.evaluate(X_test, y_test)

---

Solution:

It worked thanks to @hpaulj !

I added the following:

data = np.hstack((X, np.reshape(y, (-1,1))))
data = data.reshape(1135,4)
data.shape

And now I can do this:

transformed_df = pd.DataFrame(data, columns=df.columns)