This project compares Vanilla GAN and WGAN regarding generating real images of MNIST. It evaluates these methods using qualitative metrics like FID and Inception Score. The study also attempts to understand the ability of each of these GAN architectures to generate high-quality images.

Analyze Vanilla GAN vs. WGAN for MNIST image generation, using FID and Inception Score to evaluate and compare the quality of generated images.

PyTorch Project to Build a GAN Model on MNIST Dataset

<h2>Project Overview</h2>With this project, we will enter the world of GANs through comparative experimentation between two of its models - Vanilla GAN and WGAN - to produce real-life images using the MNIST dataset and assess which model can achieve this better!We begin by feeding all real images into both GANs, through which some fake images will be generated. Then, we shall do some fun visualization by combining the real and <a href="https://www.aionlinecourse.com/ai-projects/playground/image-generation-model-fine-tuning-with-diffusers-models" target="_blank">fake images</a> to assess their closeness.After that, it's on to computing the FID, or Fr&eacute;chet Inception Distance, to compare how similar the generated images are to the real deal. It calculates an Inception Score to measure image diversity and quality. Hopefully, by this, one will have understood comparison-wise how the Vanilla GAN and WGAN perform to generate realistic and diverse images. Quite exciting, right?<h2>Prerequisite</h2><ul><li>Good knowledge about GANs. </li><li>Python experience and expertise in other libraries, especially PyTorch. </li><li>Understanding neural networks, specifically the generators and discriminators utilized in GAN. </li><li>Some level of prior understanding of various transforms, which is simply resizing an image and normalization. </li><li>Familiarity with performance metrics like FID (Fr&eacute;chet Inception Distance) and Inception Score. </li><li>Working with data visualization libraries, for example, matplotlib and torchvision.</li></ul><h2>Approach</h2>In this project, the approach begins with loading the MNIST dataset and preprocessing the images for use in GANs. We will keep initializing and training two models: Vanilla GAN and WGAN, each with a generator and discriminator. Both generators learn to create fake images from random noise vectors, while the discriminator learns to recognize real vs. fake images. After training, we get fake images from both and compare them visually. Then, we measure the quality of generated images using the FID score, determining how similar the generated images are to the real ones, and the Inception score, which indicates how diverse and qualified the produced images are. This approach allows us to compare performance in terms of the generation of realistic images after comparison of Vanilla GAN with WGAN and draw better results, which model could give<h2>Workflow</h2>The workflow comprises the following steps:<ul><li>Load and Preprocess Images for Training of the MNIST Dataset. </li><li>Initialize and Train Vanilla GAN and WGAN Models. </li><li>Generate Fake Images with Vanilla GAN and WGAN Generators. </li><li>Display Real vs. Fake Images Side by Side for Comparison. </li><li>Compute the FID Score between Real and Fake Images. </li><li>Calculate the Inception Score for Quality and Diversity Measurement of the Generated Images. </li><li>Compare the Results of Both Models to Determine which One would be Best.</li></ul><h2>Methodology</h2><ul><li>Use random noise vectors as input into the generators to create fake images. </li><li>Train the discriminators to tell the difference between genuine and impostor images. </li><li>Input preprocessing for images will involve resizing, cropping, and normalization. </li><li>Model performance assessment using FID scores and Inception Scores to obtain a quantified measure of image quality. </li><li>Evaluate the performance of both GANs visually by generating images in grids to observe and compare their performance.</li></ul><h2>Data Collection and Preparation</h2><h4>Data Collection</h4><ul><li>This project utilizes the MNIST dataset, which contains handwritten digits ranging from 0 to 9. </li><li>The dataset is available for download via the torchvision.datasets library. </li><li>The dataset comprises 60,000 training and 10,000 testing images in 28x28 pixel sizes. </li><li>All images are grayscale, and it is a dataset with good digit images for training image generation models. </li><li>The dataset is preprocessed by resizing, normalizing, and converting into tensors compatible with GAN models.</li></ul><h4>Data Preparation Workflow</h4><ul><li>Load the MNIST dataset with torchvision.datasets.MNIST under image generation tasks. </li><li>Perform preprocessing transformations such as resizing, normalization, and grayscale-to-RGB conversion. </li><li>Convert images to pytorch tensors to use with <a href="https://www.aionlinecourse.com/ai-basics/convolutional-neural-network" target="_blank">neural networks.</a> </li><li>Use a DataLoader to load images in batch for efficient training of models. </li><li>Put images to the appropriate device (CPU or GPU) for processing by model.</li></ul>

<h2>Code Explanation</h2>This code installs Python libraries: torch for <a href="https://www.aionlinecourse.com/ai-projects/categories/deep-learning-projects" target="_blank">deep learning</a>, torchvision for computer vision tasks, and matplotlib for plotting and graphing. Thus, you are preparing your environment for machine learning and <a href="https://www.aionlinecourse.com/ai-basics/data-visualization" target="_blank">data visualization.</a>

This code imports all the needed libraries for deep learning and computer vision tasks. Libraries include PyTorch (torch), torchvision to build datasets and transform images, <a href="https://www.aionlinecourse.com/ai-basics/hyperparameter-optimization" target="_blank">optimization</a> and neural network libraries, and libraries for plotting. This code also imports all the tools for working with images, like PIL and matplotlib, to visualize results.

This code sets up the MNIST dataset for training. It converts input images to tensors and normalizes them. The dataset is loaded in batches of size 64 and shuffled for training using a DataLoader.

This function visualizes a set of training MNIST images. It selects a batch of training images and arranges them in a grid for display using matplotlib. The number of images to show is controlled by the num_images argument, whose default value is 10.

<h3>This code describes general structures for the generator </h3> and discriminator for a Vanilla GAN. The Generator receives a random noise vector (z) and generates a 28x28 image using fully connected layers, batch normalization, and LeakyReLU activations.The Discriminator class tries to determine whether an input image is legitimate or counterfeit by flattening the image and conducting it through several fully connected layers with dropout and LeakyReLU for regularization. It finally outputs a probability signifying whether the image is real or fake.

This code defines generator-discriminator models for the Wasserstein GAN (WGAN). The WGANGenerator constructs images from random noise using a combination of fully connected layers, LeakyReLU activations, and batch normalization. The WGANDiscriminator evaluates the authenticity of images through fully connected layers and LeakyReLU activations, returning a final probability, whether real or fake.

This function trains a GAN, alternately operating first on the weight updates for the discriminator and then at other times on the weight updates of the generator. Here, the discriminator acts as a classifier that identifies between the original (real) and fake images, while the generator tries to convert the data so that it would fool the discriminator and transmute the fake image into that of the real one. Eventually, the results regarding both models are to be computed and recorded for plotting during later stages, whereas progress will also be printed out after the completion of one epoch. Finally, the return statement contains loss values for both models through all epochs.

<h3>The function extracts general features from a given model, which defaults to the InceptionV3 model, pre-processes by resizing and cropping, converting grayscale into RGB format, normalizes and converts them to tensor arrays, and feeds these images through the model without weight updating to obtain the feature representation of the image.</h3>

This function computes the FID score for real and fake images. It first extracts feature representations of both sets of images with the InceptionV3 model. Thereafter, it computes the mean and covariance for each and finally computes the FID which reflects the distance between feature distributions of real versus fake images.

In this function, the Inception score (IS) is calculated for the produced images. Images are preprocessed matching the input size of InceptionV3, and then it computes softmax probabilities for every image using the model. Further, it calculates the KL divergence of the conditional distribution (given the image) for the marginal distribution (across the images) and finally takes an exponentiated average of this formality, such as an Inception score, which refers to the quality and variety in the images generated as output.

This function visualizes the generated images from the GAN's generator. It creates a batch of random noise vectors (z), generates fake images with the generator, and arranges them in a grid. The grid is displayed with matplotlib, showing the generated images in a clear, 8x8 grid format.

This function plots the loss curves for both the discriminator as well as the generator during training. It takes in the discriminator (d_losses) and generator (g_losses) loss values over epochs, and it visualizes them. Additionally, there are some labels on the plot, a legend, and axis titles to keep track of the model's progress.

This line selects the device for PyTorch, in particular assigning cuda if it's available, or else the system uses any CPU if it doesn't detect a proper GPU for use. It makes sure that the model runs on the most efficient hardware available for computation.

<h3>This initializes the Vanilla GAN generator and discriminator and trains them using train_gan. The training sample run is for 20 epochs with a random noise dimension of 100 and stores losses for both discriminator and generator during training.</h3>

This initializes the WGAN generator and discriminator and trains them using train_gan. The training sample run is for 20 epochs with a random noise dimension of 100 and stores losses for both discriminator and generator during training.

<h3>This line visualizes images generated by the generator_vanilla within the framework of the Vanilla GAN. It creates random noise vectors, generates fake images, and displays them in a grid using the matplotlib library. The pictures generated will be displayed in an 8x8 grid.</h3>

This plots the loss curves for the Vanilla GAN's discriminator and generator. The figure shows how the losses evolve across the epochs of training, which will help track the performance and stability of the model. The solid line will show the loss of the discriminator and the dashed line will show the loss of the generator.

This line visualizes images generated by the generator_wgan within the framework of the WGAN. It creates random noise vectors, generates fake images, and displays them in a grid using the matplotlib library. The pictures generated will be displayed in an 8x8 grid.

This plots the loss curves for the WGAN's discriminator and generator. The figure shows how the losses evolve across the epochs of training, which will help track the performance and stability of the model. The solid line will show the loss of the discriminator and the dashed line will show the loss of the generator.

This code extracts a bunch of real images from the train loader. Then images are retrieved using an iterator and moved to the correct device (GPU or CPU) for further processing. The images are now ready for model training or evaluation.

This code produces a small number of fake images using model generator_vanilla. It creates noise vectors (z) equal to the size of real images in the batch and sends them through the generator to create fake images. These images can then be used in evaluations or comparisons against real images.

The code calculates FID between the real images of the Vanilla GAN-generated fake images. The calculate_fid function compares the feature distributions between real and fake images, while the final score will be printed. The lower the FID score, the closer the fake images are to real ones.

This code takes a batch of images and generates fake images from a pre-defined generator_wgan under the WGAN model, and then computes for FID of the real and fake images. The calculate_fid function measures the difference of one feature distribution of the real images to the one of the fake images. The value of the FID is computed for the WGAN and the score is printed to the console with the understanding that the smaller the score the better image quality.

This function visualizes and compares real images with fake images produced using both the Vanilla GAN and WGAN models, creates a grid of images for each of them (real, Vanilla GAN fake, WGAN fake), and shows them scissors-side. The arrangement of images is in 8x8 grids, so all the images from each set can be studied in a separate subplot for easy comparison.

The following code generates artificial images using the generator_vanilla of Vanilla GAN. It generates random noise vectors (z_vanilla) that have the same batch size as real images, and these vectors are fed to the fictitious images in turn. The images are then ready for processing or evaluation.

This code generates fake images using the generator_wgan from the WGAN model. It creates random noise vectors (z_wgan) with the same batch size as the real images and passes them through the WGAN generator to produce fake images. These images can now be used for evaluation or comparison with real images.

This code visualizes and compares true images with false images generated using both Vanilla GAN and WGAN models. For easy and side-by-side comparison, it uses the visualize_comparison function that displays these images in 8x8 grids. There will be three separate subplots for real images, Vanilla GAN generated false images, and WGAN generated false images.

This code computes the Inception Score (IS) of the synthetic images produced by the Vanilla GAN. It accepts the mean and standard deviation of the IS value computed by the inception_score function, which indicates the quality and diversity of the generated images. The score indicates the higher the value score, the better the image quality and diversity.

This code computes the Inception Score (IS) for the fake images generated using WGAN. The mean and standard deviation of IS calculated through inception_score indicate the quality as well as the diversity of generated images. It prints scores and a higher score indicates better performance in terms of image quality and diversity.

<h2>Conclusion</h2>The project successfully compares the Vanilla GAN and WGAN based on image generation from the MNIST dataset. The generated images are evaluated for their quality, diversity, and realism through metrics such as FID scores and Inception Scores. The visual comparison and evaluation results will help to judge which architecture of GAN performs better in producing high-quality images. This study has touched on all aspects of the strengths and weaknesses of different GAN models for image generation.<h2>Challenges New Coders Might Face</h2>Challenge: Difficulty in Training GANs (Convergence Problems - Mode Collapse, Vanishing Gradients). Solution: Utilizing techniques such as gradient clipping and the use of a learning rate schedule will also help mitigate these difficulties in both Vanilla GAN and WGAN.Challenge: Bad Quality Images from Vanilla GAN Solution: One can attain better sharp, more realistic images by switching to WGAN with its Wasserstein loss, besides improving discriminator architecture.Challenge: High Memory Usage Solution: Some techniques, such as model-checkpointing to save intermediate results or changes in batch size, can manage memory consumption effectively.Challenge: Less Diversity of the Generated Images Solution: Tamper with the model architecture to improve diversity and use principles such as feature matching and different activation functions or <a href="https://www.aionlinecourse.com/ai-basics/batch-normalization" target="_blank">batch normalization</a> layers.<h2>Frequently Asked Questions (FAQs)</h2>Question 1: How do I use the MNIST dataset for GAN image generation? Answer: With PyTorch's torchvision.datasets.MNIST, the MNIST dataset is loaded featuring handwritten digits within 28x28 pixel grayscale images with sufficient data for training a GAN to produce similar images.Question 2: What can I do when there are high memory consumption problems during GAN training? Answer: To manage memory usage, reduce batch size, use model checkpointing, and ensure efficient data loading. Also, accelerate the training process with GPU.Question 3: How can I visualize GAN-generated images? Answer: To visualize images generated by GAN, they can be represented in a grid using torchvision.utils.make_grid and viewed with matplotlib; this makes it quicker to compare real with generated images.Question 4: Why is my GAN training unstable or slow? Answer: Various causes of instability in GAN training include mode collapse and vanishing gradients. Model usage, like WGAN, and gradient clipping, as well as trying different learning rates, may also help improve stability and speed. Question 5: What is Inception Score and how does it measure image quality? Answer: Inception Score measures the quality and diversity of generated images with the help of InceptionV3 and computes the KL-divergence between the predicted class probabilities and their marginal distribution.Interested in GANs or other AI experiments and want to know more? Visit our <a href="https://www.aionlinecourse.com/ai-projects" target="_blank">AI Projects</a> Page for additional guided tutorials, materials, and innovative approaches to upgrade your skills!Ready to test your AI knowledge? Head over to our <a href="https://www.aionlinecourse.com/ai-quiz-questions" target="_blank">AI Quiz Page</a> and challenge yourself with expert-level AI quiz questions to boost your learning!

PyTorch Project to Build a GAN Model on MNIST Dataset

Project Outcomes

Requirements:

Project Description