<p>How many times do you come across large volumes of text data and pause to think of an easier way to decipher it? This is where this project comes in handy as it employs sophisticated RNNs and LSTM techniques in its implementation. It aims at intelligently classifying real-life customer complaints. Be it identity theft or credit card issues, this project takes the text and applies it to real-life problems. </p>

Multi-class text classification using RNN and LSTM for analyzing customer complaints, and providing real-world business insights and solutions.

Build Multi-Class Text Classification Models with RNN and LSTM

<h2>Project Overview</h2><p>The purpose of this undertaking is to investigate the classification of text as comprehensively as possible using Python, PyTorch, and Natural Language Processing. To begin with the proper data set; cleansing, formatting, and vectorization of corporeal text into GloVe embeddings is explained. Following that the RNN and the LSTM neural networks where the models have already been trained on sample data to predict the category of the complaint are constructed using the tokenized data. The models developed are checked for performance using accuracy and confusion matrices among other metrics.</p><p>Step by step and with good code, the course demonstrates how to normalize text, create embeddings, and build classification models. It’s a practical approach to understand the concepts of Natural language processing, deep learning, and the use of AI for problem-solving. Ideal for programmers, data analysts, and enthusiasts of machine learning!</p><h2>Prerequisites</h2><p>Before commencing this project, ensure that you have the following skills and tools: </p><ul><li><strong>Python</strong>: You need to know how to program in Python and use libraries such as NumPy and Pandas.  </li><li><strong>Natural Language Processing Basics:</strong> You should be familiar with how tokenization works, what embeddings are, and basic text preprocessing techniques.  </li><li><strong>Knowledge of PyTorch</strong>: Knowing how to use PyTorch in the creation and training of neural networks is a must.  </li><li><strong>Machine Learning</strong>: Knowledge in classification, classification loss, and accuracy.  </li><li><strong>GloVe Embeddings</strong>: Be able to explain how and why word embeddings are useful in representing and manipulating text data.  </li><li><strong>Tools Installed</strong>: Confirm that NLTK, Scikit-learn, Matplotlib, and Seaborn libraries are present in your Python environment.</li></ul><p>Provided all these prerequisites are in place, you are good to go! Let’s begin with the art of text classification!</p><h2>Approach</h2><p>This project carries out text classification using deep learning models such as RNN and LSTM in an organized manner. The project commences with data preprocessing. In this case, the raw text is cleaned of unwanted characters, such as digits and other special characters, as well as excess whitespace. Each text fragment is processed into tensors of a certain size using tokenization. Then turns of these tokens are converted from words to relevant quantized vectors using GloVe embedding techniques that help in maintaining the semantic relationships among the words.</p><p>The prepared data comprises three datasets, for training, validation, and model testing. To optimally fill in the data inside the model, the custom data sets and data loaders based on the Pytorch framework are created. Recurrent Neural Networks (RNN) and Long Short Term Memory (LSTM) are established. The models are trained using cross-entropy loss which has been subjected to Adam optimization and also validated on the training for overfitting.</p><p>Once training is complete, the models are now validated against new datasets that were not presented to them. The performance evaluation takes place using metrics such as accuracy, confusion matrices, and classification reports. The methodology is straightforward and clear and comprises simple but powerful text processing techniques combined with deep learning approaches to promote an effective text classification strategy.</p><h2>Workflow and Methodology</h2><h3>Workflow</h3><ul><li>First load and collect the dataset.  </li><li>Preprocess it by eliminating unnecessary factors, preparing and even correcting the missing data, and synthesizing, or reorganizing existing information.  </li><li>Perform tokenization of the textual information and encode it into a vectorized form by employing GloVe.  </li><li>Organize the data into training, validation, and test sets to facilitate the evaluation of the model.  </li><li>Implement the training process of RNN and LSTM networks with PyTorch framework and with cross-entropy loss and Adam optimizer.  </li><li>Measure how well the trained model assesses and classifies the data using the following performance measurement tools: accuracy, confusion matrix, and classification report.</li></ul><h3>Methodology</h3><ul><li>Make use of the GloVe embeddings to represent text data in an appropriate form.  </li><li>Utilize RNN and LSTM networks to model temporal patterns of complaint Narratives.  </li><li>Improve the performance of the networks by reducing the given loss function and also monitoring the validation phase.  </li><li>Keep the network that has the best performance on the validation data set so that the performance on the test set is as good as possible.  </li><li>Evaluate the test predictions with the help of confusion matrices and extensive classification reports.</li></ul><h2>Data Collection and Preparation</h2><h4>Data Collection:</h4><p>In this project, we collected the dataset from a public repository. If you are looking to work on a real-world problem, you can get these kinds of datasets from publicly available repositories such as Kaggle, UCI Machine Learning Repository, or company-specific data. We will provide the dataset in this project so that you can work on the same dataset.</p><h4>Data Preparation Workflow:</h4><ul><li>The dataset should be imported and preprocessed including noise removal and treatment of empty or missing data.  </li><li>Perform text tokenization, padding or truncating, and token mapping into Glove embedding vectors.  </li><li>For model assessment, the data should be split into training, validation, and test sets.</li></ul>

<h2>Code explanation</h2><p>Here&rsquo;s what is happening under the hood. Let&rsquo;s go through it step by step:</p><h3><span style="color: inherit; font-family: inherit; font-size: 1.5rem;">Mounting Google Drive</span></h3><p>First, mount Google Drive to access the dataset that is stored in the cloud.</p>


<h4>Installing Required Libraries</h4><p>This code installs a few important libraries: NLTK which allows natural language processing, NumPy for working with arrays, Pandas when dealing with data, Pytorch for deep learning tasks, TQDM provides easy-to-use progress bars, and Scikit-learn is a package used for machine learning applications.</p>

<h4>Importing Libraries and Handling Warnings</h4><p>This piece of code imports fundamental libraries required for natural language processing, building and implementing machine learning models, and visualizing the information generated with the help of NLTK, PyTorch, and Scikit-learn. Moreover, certain warnings are turned off to keep the workspace clean and organized.</p>

<h4>Defining Configuration and File Paths</h4><p>This code snippet initializes important configurations such as learning rate, input dimensions, and model files used for training and testing. It also defines directories for data, models, tokenizers, embeddings, and product mapping for uniform label encoding.</p>

<h4>Functions to Save and Load a File</h4><p>This code introduces two functions: saving an object in a pickle format using the function save_file and bringing back the object into existence using the function load_file.</p>

<h3><span style="color: inherit; font-family: inherit; font-size: 1.5rem;">Reading GloVe Embeddings</span></h3><p>This code reads the GloVe embeddings file and calculates the total number of unique words.</p>


<h4>Retrieval of the First Entry in the GloVe Embedding Dataset</h4><p>This piece of code fetches the first entry from the GloVe embeddings file and prints out the first word along with a vector assigned to it.</p>

<h4>Extracting the Initial Word from GloVe Embeddings</h4><p>The following code separates the initial word present in the first entry of the GloVe embeddings from its respective vector values.</p>

<h4>Extraction of Embedding Values of the First Word</h4><p>The given code snippet extracts the details or vector values (embeddings) corresponding to the first word in the first line of the GloVe embedding file.</p>

<h4>Forming a Vocabulary and Creating Embeddings Array</h4><p>This code creates a vocabulary of words and associated embeddings matrix using GloVe embeddings. The embedding matrix is transformed into a float32 numpy array whose shape denotes the number of words and their respective vector sizes.</p>

<h4>Displaying the First 10 Words in the Vocabulary</h4><p>This section of code fetches the first ten entries appearing on the lexicon constructed along with the GloVe embedding and outputs them for visualization purposes.</p>

<h4>Updating Vocabulary and Embeddings</h4><p>The algorithm incorporates special tokens \&lt;pad&gt; and \&lt;unk&gt; to the lexical content and attaches the respective vectors: all ones for the \&lt;pad&gt; vector and the mean of the existing embeddings for \&lt;unk&gt;. It modifies the embeddings matrix correspondingly and displays the new vocabulary size and shape of embeddings.</p>

<h4>Saving Embeddings and Vocabulary</h4><p>The code saves the revised embedding matrix and the vocabulary list to pickle files using the assigned directory paths for potential use in the future.</p>

<h3><span style="color: inherit; font-family: inherit; font-size: 1.5rem;">Importing and Processing Data</span></h3><p>The code loads the dataset provided in CSV format, deletes the corresponding entries of the rows where they have null values in the text column, and also normalizes the target column by clearing the existing duplicates using the pre-developed product map.</p>


<h4>Label Encoding</h4><p>The provided code implements a LabelEncoder, applies it to the dataset’s target labels, and encodes them into integers. It then returns the numeric representation of the first label.</p>

<h4>Retrieving Label Classes</h4><p>This code displays all the unique label classes that the LabelEncoder has learned from the dataset.</p>

<h4>Observing the Target Column</h4><p>The following code obtains the values in the target column (label_col) from the dataset and presents it, as it appears before the label is converted into an encoded form.</p>

<h4>Saving Labels and Label Encoder</h4><p>The code saves the numeric labels and the trained LabelEncoder as pickle files for later use in the project.</p>

<h4>Processing Text Input</h4><p>This code transforms the entire text in the given column into lowercase so that all the entries are the same during processing. It has a progress bar to show the estimate of how much the conversion has gone.</p>

<h4>Eliminating Special Characters in Text</h4><p>The script eliminates special characters, punctuation, and symbols from the corresponding text, which only allows words, numbers, and spaces. This guarantees cleaner input for the subsequent steps.</p>

<h4>Removing Numbers From Text</h4><p>The purpose of this code is to remove any numeric digits present in the text to concentrate only on the words which in turn makes the data more clean for analysis</p>

<h4>Eliminating Consecutive 'x' Characters</h4><p>The purpose of this particular code is to erase all sequences in input_text that consist of two or more consecutive 'x' characters using regular expression thereby enhancing the cleanup of the text.</p>

<h4>Removing Extra Spaces</h4><p>This code removes multiple consecutive spaces in the input_text and replaces them with a single space, ensuring cleaner text formatting.</p>

<h4>Tokenizing Texts to Words</h4><p>Here, for the first 100 data in the input_text, we apply NLTK's word_tokenize. Each entry turns into a list of tokens, which are further used for processing.</p>

<h4>Converting Text into Tokens of Equal Size</h4><p>This code makes sure each text is tokenized into exactly 20 tokens. In the event the text exceeds 20 tokens, the remaining tokens are omitted and only the first 20 are retained. For the shorter texts, \&lt;pad&gt; tokens are added to fill the gap.</p>

<h4>Converting Tokens to Indices</h4><p>This specific function is utilized to assign a numerical index to the word tokens available in the vocabulary. Any token that does not exist is replaced with the index value of \&lt;unk&gt;, thus all the words have a numerical representation.</p>

<h4>Mapping Tokens to Indices</h4><p>This code converts a list of tokens into their corresponding numerical indices using the vocabulary for further steps in machine learning models.</p>

<h4>Checking the Number of Tokenized Texts</h4><p>This code returns the overall count of tokenized texts which is equivalent to the number of rows in the data.</p>

<h4>Observing the Indices of the Initial Tokenized Text</h4><p>The code below presents the numeric indices of the first tokenized text and illustrates the relationship of each word with the vocabulary.</p>

<h4>Displaying the First Few Rows of the Dataset</h4><p>This code shows the first five rows of the dataset, providing a quick overview of the data structure and content.</p>

<h4>Extracting the Initial Lexeme from a List of Numeric Codes</h4><p>The code extracts the first word from the vocabulary that matches the first token index of the first tokenized document.</p>

<h4>Saving Tokenized Data</h4><p>The code saves the tokenized text data as a pickle file for reuse in future processing or model training.</p>

<h3><span style="color: inherit; font-family: inherit; font-size: 1.5rem;">Creating a Custom Dataset Class</span></h3><p>The TextDataset class is designed for preparing text-related data for use with PyTorch models. It takes tokenized data, embeddings, and labels and makes access to samples by indices efficient. The __getitem__ method gets the label and the word embeddings belonging to that label for a particular data point.</p>


<h4>Creating an RNN Model Class</h4><p>Here, the RNNNetwork class creates an RNN model for the task of text classification. It has an RNN Unit, which is meant for handling intermittent sequences of data, and a linear layer that channels the hidden state to the various output classes. In short, the forward method accepts input data, runs it through an RNN, and generates output predictions.</p>

<h4>Create an LSTMNetwork Class</h4><p>The LSTM Network class creates a Long Short-Term Memory (LSTM) network with the core application of classifying textual data. Long Short-Term Memory (LSTM) layer is included in the design, to deal with the sequential input data, as well as a linear layer that returns the class predictions. The forward method handles the input data and the hidden state is used for classification.</p>

<h4>The Train Function of Neural Networks</h4><p>This train function is used to train and test the provided Pytorch model for several epochs. It measures the training and validation losses, performs backpropagation to adjust model weights, and stores the model if the validation loss increases. </p>

<h3><span style="color: inherit; font-family: inherit; font-size: 1.5rem;">Model Assessment through Test Function</span></h3><p>The test function assesses the performance of a trained model using the testing phase data. It calculates the test loss, performs label predictions, and computes the accuracy of the model while maintaining constant weights of the model.</p>


<h4>Loading Preprocessed Data and Model Parameters</h4><p>This code loads previously saved tokens, labels, embeddings, and the label encoder and also calculates the total number of output classes for the model.</p>

<h4>Dividing into Train, Validation, and Test Sets</h4><p>This code is used to divide tokenized data and labels into the train, validation, and test sets; 20% is reserved for the test set and 25% of the training set is reserved for validation.</p>

<h4>Creating Datasets for Training, Validation, and Testing</h4><p>In this code, we create TextDataset objects for training, validation, and test datasets, and integrate tokenized inputs, their embeddings, and the relevant labels for each of the datasets.</p>

<h4>Constructing Data Loaders for Model Training and Evaluation Tasks</h4><p>This piece of code creates DataLoader objects in the PyTorch framework for training, validation, and test purposes. This ensures that there is efficient processing of data in batches with a batch size of sixteen, shuffling during training, and stable loading during validation and testing.</p>

<h4>Setting up the RNN Model</h4><p>This code creates an object of the RNNNetwork class using the provided input, hidden layer dimensions, and the count of output classes.</p>

<h4>Configuring Your Device for Training the Model</h4><p>This piece of code evaluates if a GPU (cuda) or a CPU will be used for model training and transfers the RNN model to the appropriate device.</p>

<h4>Specifying Loss Function, Optimizer, and Device</h4><p>The following code defines the configuration of a cross-entropy loss function intended for classification tasks, an Adam optimization algorithm responsible for updating the parameters of the model, and results to a GPU (GPU:0) where applicable, and a CPU otherwise.</p>

<h4>Executing the RNN Model Training Process</h4><p>The following code block trains the RNN model using the function train, making use of the training and validation data loaders, the loss function, the optimizer, and the number of epochs defined. The model with the optimal performance is saved at the location specified.</p>

<h4>Assessing the Model with Comprehensive Reports</h4><p>This section of the code tests the trained model on the test data and produces a classification report in addition to the confusion matrix and implementation accuracy. It begins by loading the best model and proceeds to make predictions which are then compared with actual observations to provide a performance overview of the model.</p>

<h4>Generating the Classification Report</h4><p>The code prints a detailed classification report, which includes precision, recall, F1-score, and support values for all classes.</p>

<h4>Confusion Matrix Visualization</h4><p>The following code plots the confusion matrix and emphasizes how well the model was able to classify the classes. It adds title and axis labels for better understanding.</p>

<h4>Initializing the LSTM Model</h4><p>An instance of the LSTMNetwork model is created here by specifying the input size and hidden size along with the number of classes in the case of modeling for classification.</p>

Build Multi-Class Text Classification Models with RNN and LSTM

Project Outcomes

Requirements:

Project Description