CNN model accuracy is not improving

Question:

I have many time series data (time & voltage) from different devices. I was trying to convert this time series data to an image array so that it can be used as an image and I can apply CNN on the image and can classify from which device the data came from. It is based on assumption that different device produces different voltage signature which should be differentiable. I applied a function that split the time series data to an image array and then used CNN to classify. The image size is 14×100 pixel. The problem is the accuracy of the model is not changing or it is barely changing. Please find the complete coding below

import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
import os
import tensorflow as tf
from sklearn.metrics import plot_confusion_matrix, ConfusionMatrixDisplay, confusion_matrix

#Reading data from directory
import pathlib
data_dir = ('Test')
data_dir = pathlib.Path(data_dir)

#File name dictionary
file_name_dict = {
    'Device_1' : list(data_dir.glob('Device_1/*.csv')),
    'Device_2' : list(data_dir.glob('Device_2/*.csv')),
    'Device_3' : list(data_dir.glob('Device_3/*.csv')),
    'Device_4' : list(data_dir.glob('Device_4/*.csv')),
    'Device_5' : list(data_dir.glob('Device_5/*.csv')),
    'Device_6' : list(data_dir.glob('Device_6/*.csv'))
}

#File label dictionary
file_label_dict = {
    'Device_1' : 0,
    'Device_2' : 1,
    'Device_3' : 2,
    'Device_4' : 3,
    'Device_5' : 4,
    'Device_6' : 5,
}


#Processing and labelling the data

device_list, device_label = [],[]   #compressed

#Reading each file, replacing original value with moving average value
for device_name, folder in file_name_dict.items():
    for file in folder:
        file_df = pd.read_csv(str(file))
        file_df.columns = ['time', 'voltage']
        file_df['MA'] = file_df['voltage'].rolling(10,min_periods=0).mean()# moving average
        file_df= file_df.drop('voltage',axis=1)
        file_df.rename(columns={'MA':'voltage'},inplace=True)
        
        #Applying a function
        threshold = file_df['voltage'].diff().gt(1)
        group = (threshold&~threshold.shift(fill_value=False)).cumsum().add(1)
        time= lambda i: i['time'].groupby(group).apply(lambda j: j- j.iloc[0])
        df_2 = (file_df.assign(bit=group,time=time).pivot(index='bit', columns='time', values='voltage'))

        df_3 = df_2.copy()
        df_3.reset_index(drop=True, inplace=True)
        df_3 = df_3.rename_axis(None, axis=1)

        #Compressing to  14 rows 100 columns
        df_4=df_3.iloc[:10, :100]
        
        #Filling out null values
        df_4= df_4.fillna(method='pad')
        
        #Saving each dataframe to a list
        device_list.append(df_4)
        device_label.append(file_label_dict[device_name])
        
#Converting to numpy array
X = np.array(device_list)
y = np.array(device_label)


#Train test split
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size= 0.30,random_state=42)

#Adding 1 dimension to make it, two dimension, to use Conv2D
X_train = X_train.reshape(X_train.shape[0],X_train.shape[1],X_train.shape[2],1)
X_test = X_test.reshape(X_test.shape[0],X_test.shape[1],X_test.shape[2],1)

#scaling data from 0 to 1
X_train_scaled = X_train/36 #Max voltage value 36
X_test_scaled = X_test/36

#Building a CNN a model
#Set random seed
tf.random.set_seed(42)
model_2 = tf.keras.Sequential([
    
    tf.keras.layers.Conv2D(filters=3, kernel_size=3, strides=1, padding="same", activation='relu',kernel_initializer='normal',
                           input_shape=(X_train_scaled[0].shape)),
    tf.keras.layers.MaxPool2D(pool_size=2),
    
    
    tf.keras.layers.Conv2D(6,3, padding="same", activation='relu',kernel_initializer='normal',),
    tf.keras.layers.MaxPool2D(pool_size=2),
    
    tf.keras.layers.Conv2D(12,3, padding="same",activation='relu',kernel_initializer='normal',),
    tf.keras.layers.MaxPool2D(pool_size=2),
    
    
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(72,activation='relu',kernel_initializer='normal',),
    tf.keras.layers.Dense(6, activation= 'softmax') #Output layer

])
model_2.summary()

#Training
from tensorflow import keras
opt = keras.optimizers.Adam(lr=0.01)
model_2.compile(optimizer=opt,
              loss='sparse_categorical_crossentropy',
                metrics= ['accuracy'])

             
history= model_2.fit(X_train_scaled, y_train,batch_size= 128 , epochs=200,validation_split=0.20, verbose=1) 
history

#Test
loss, accuracy= model_2.evaluate(X_test_scaled, y_test)
print(f'Loss: {loss}, Accuracy: {accuracy}')
print('Test accuracy : ',accuracy*100,'%')

enter image description here
enter image description here
enter image description here

The accuracy starts from 0.16 and goes up to 0.18, not going above 0.18. I tried to change different parameters of the CNN model like adding more convolutional layer, dense layer, changing the learning rate of the adam optimizer, using different optimizer, tried with different batch_size but the accuracy of the model is not improving at all.

I am just confused if I have done something wrong during processing of the data. Could someone please look at the coding and tell me the way I labeled the data is OK, or there is something wrong with my coding. If the code is fine, any suggestion on how can I improve the accuracy is much appreciated. I am just not sure if it a coding problem or there is any problem with the model or the data is not good enough for classification task.

Asked By: Nazmul1001

||

Answers:

CNNs are usually used for spatial relations between the data (as you said, e.g. images) but it seems, that your data doesn’t have this kind of spatial relation. Instead, you describe them as time series and for time series, it’s common to use so-called recurrent neural networks (RNN). They take as input one step of a time series and transport a hidden state to the next step additionally to the next time series step. Maybe this kind of models fit better to your needs.

Answered By: Fatorice

Reducing the learning rate to 0.001 solved the issue.

Answered By: Nazmul1001