Build Multi-Class Text Classification Models with RNN and LSTM

Project Overview

The purpose of this undertaking is to investigate the classification of text as comprehensively as possible using Python, PyTorch, and Natural Language Processing. To begin with the proper data set; cleansing, formatting, and vectorization of corporeal text into GloVe embeddings is explained. Following that the RNN and the LSTM neural networks where the models have already been trained on sample data to predict the category of the complaint are constructed using the tokenized data. The models developed are checked for performance using accuracy and confusion matrices among other metrics.

Step by step and with good code, the course demonstrates how to normalize text, create embeddings, and build classification models. It’s a practical approach to understand the concepts of Natural language processing, deep learning, and the use of AI for problem-solving. Ideal for programmers, data analysts, and enthusiasts of machine learning!

Prerequisites

Before commencing this project, ensure that you have the following skills and tools:

• Python: You need to know how to program in Python and use libraries such as NumPy and Pandas.
• Natural Language Processing Basics: You should be familiar with how tokenization works, what embeddings are, and basic text preprocessing techniques.
• Knowledge of PyTorch: Knowing how to use PyTorch in the creation and training of neural networks is a must.
• Machine Learning: Knowledge in classification, classification loss, and accuracy.
• GloVe Embeddings: Be able to explain how and why word embeddings are useful in representing and manipulating text data.
• Tools Installed: Confirm that NLTK, Scikit-learn, Matplotlib, and Seaborn libraries are present in your Python environment.

Provided all these prerequisites are in place, you are good to go! Let’s begin with the art of text classification!

Approach

This project carries out text classification using deep learning models such as RNN and LSTM in an organized manner. The project commences with data preprocessing. In this case, the raw text is cleaned of unwanted characters, such as digits and other special characters, as well as excess whitespace. Each text fragment is processed into tensors of a certain size using tokenization. Then turns of these tokens are converted from words to relevant quantized vectors using GloVe embedding techniques that help in maintaining the semantic relationships among the words.

The prepared data comprises three datasets, for training, validation, and model testing. To optimally fill in the data inside the model, the custom data sets and data loaders based on the Pytorch framework are created. Recurrent Neural Networks (RNN) and Long Short Term Memory (LSTM) are established. The models are trained using cross-entropy loss which has been subjected to Adam optimization and also validated on the training for overfitting.

Once training is complete, the models are now validated against new datasets that were not presented to them. The performance evaluation takes place using metrics such as accuracy, confusion matrices, and classification reports. The methodology is straightforward and clear and comprises simple but powerful text processing techniques combined with deep learning approaches to promote an effective text classification strategy.

Workflow and Methodology

Workflow

• First load and collect the dataset.
• Preprocess it by eliminating unnecessary factors, preparing and even correcting the missing data, and synthesizing, or reorganizing existing information.
• Perform tokenization of the textual information and encode it into a vectorized form by employing GloVe.
• Organize the data into training, validation, and test sets to facilitate the evaluation of the model.
• Implement the training process of RNN and LSTM networks with PyTorch framework and with cross-entropy loss and Adam optimizer.
• Measure how well the trained model assesses and classifies the data using the following performance measurement tools: accuracy, confusion matrix, and classification report.

Methodology

• Make use of the GloVe embeddings to represent text data in an appropriate form.
• Utilize RNN and LSTM networks to model temporal patterns of complaint Narratives.
• Improve the performance of the networks by reducing the given loss function and also monitoring the validation phase.
• Keep the network that has the best performance on the validation data set so that the performance on the test set is as good as possible.
• Evaluate the test predictions with the help of confusion matrices and extensive classification reports.

Data Collection and Preparation

Data Collection:

In this project, we collected the dataset from a public repository. If you are looking to work on a real-world problem, you can get these kinds of datasets from publicly available repositories such as Kaggle, UCI Machine Learning Repository, or company-specific data. We will provide the dataset in this project so that you can work on the same dataset.

Data Preparation Workflow:

• The dataset should be imported and preprocessed including noise removal and treatment of empty or missing data.
• Perform text tokenization, padding or truncating, and token mapping into Glove embedding vectors.
• For model assessment, the data should be split into training, validation, and test sets.

Code explanation

Here’s what is happening under the hood. Let’s go through it step by step:

STEP 1:

Mounting Google Drive

First, mount Google Drive to access the dataset that is stored in the cloud.

Installing Required Libraries

This code installs a few important libraries: NLTK which allows natural language processing, NumPy for working with arrays, Pandas when dealing with data, Pytorch for deep learning tasks, TQDM provides easy-to-use progress bars, and Scikit-learn is a package used for machine learning applications.

Importing Libraries and Handling Warnings

This piece of code imports fundamental libraries required for natural language processing, building and implementing machine learning models, and visualizing the information generated with the help of NLTK, PyTorch, and Scikit-learn. Moreover, certain warnings are turned off to keep the workspace clean and organized.

Defining Configuration and File Paths

This code snippet initializes important configurations such as learning rate, input dimensions, and model files used for training and testing. It also defines directories for data, models, tokenizers, embeddings, and product mapping for uniform label encoding.

Functions to Save and Load a File

This code introduces two functions: saving an object in a pickle format using the function save_file and bringing back the object into existence using the function load_file.

STEP 2:

Reading GloVe Embeddings

This code reads the GloVe embeddings file and calculates the total number of unique words.

Retrieval of the First Entry in the GloVe Embedding Dataset

This piece of code fetches the first entry from the GloVe embeddings file and prints out the first word along with a vector assigned to it.

Extracting the Initial Word from GloVe Embeddings

The following code separates the initial word present in the first entry of the GloVe embeddings from its respective vector values.

Extraction of Embedding Values of the First Word

The given code snippet extracts the details or vector values (embeddings) corresponding to the first word in the first line of the GloVe embedding file.

Forming a Vocabulary and Creating Embeddings Array

This code creates a vocabulary of words and associated embeddings matrix using GloVe embeddings. The embedding matrix is transformed into a float32 numpy array whose shape denotes the number of words and their respective vector sizes.

Displaying the First 10 Words in the Vocabulary

This section of code fetches the first ten entries appearing on the lexicon constructed along with the GloVe embedding and outputs them for visualization purposes.

Updating Vocabulary and Embeddings

The algorithm incorporates special tokens \<pad> and \<unk> to the lexical content and attaches the respective vectors: all ones for the \<pad> vector and the mean of the existing embeddings for \<unk>. It modifies the embeddings matrix correspondingly and displays the new vocabulary size and shape of embeddings.

Saving Embeddings and Vocabulary

The code saves the revised embedding matrix and the vocabulary list to pickle files using the assigned directory paths for potential use in the future.

STEP 3:

Importing and Processing Data

The code loads the dataset provided in CSV format, deletes the corresponding entries of the rows where they have null values in the text column, and also normalizes the target column by clearing the existing duplicates using the pre-developed product map.

Label Encoding

The provided code implements a LabelEncoder, applies it to the dataset’s target labels, and encodes them into integers. It then returns the numeric representation of the first label.

Retrieving Label Classes

This code displays all the unique label classes that the LabelEncoder has learned from the dataset.

Observing the Target Column

The following code obtains the values in the target column (label_col) from the dataset and presents it, as it appears before the label is converted into an encoded form.

Saving Labels and Label Encoder

The code saves the numeric labels and the trained LabelEncoder as pickle files for later use in the project.

Processing Text Input

This code transforms the entire text in the given column into lowercase so that all the entries are the same during processing. It has a progress bar to show the estimate of how much the conversion has gone.

Eliminating Special Characters in Text

The script eliminates special characters, punctuation, and symbols from the corresponding text, which only allows words, numbers, and spaces. This guarantees cleaner input for the subsequent steps.

Removing Numbers From Text

The purpose of this code is to remove any numeric digits present in the text to concentrate only on the words which in turn makes the data more clean for analysis