<p>Think about it if diagnosing skin cancer could be done by uploading a picture of the skin. In this project, deep learning is used to make that a possibility. Here we have incorporated state-of-the-art approaches of <a href="https://www.aionlinecourse.com/ai-basics/convolutional-neural-network" target="_blank">convolutional neural networks</a> to design a system that would be able to classify skin cancer. It helps healthcare professionals detect skin cancer faster and more accurately. With <a href="https://www.aionlinecourse.com/ai-basics/pre-trained-models" target="_blank">pre-trained models</a> such as EfficientNetB4 and DenseNet121 and basic CNN, we can infuse medical imaging with the ability to diagnose skin cancer and detect it during the early stages.</p>
<p>The potential impact is early detection of skin cancer may save lives. This project is not simply developing lines of code. It's about enhancing healthcare by using advanced technology as the key tool.</p>

Learn how deep learning models can identify skin cancer early. This AI-driven method helps doctors diagnose skin cancer with over 80% accuracy.

Skin Cancer Detection project leverages advanced deep learning models, including CNN, DenseNet121, and EfficientNetB4, to accurately classify skin cancer images. This initiative aims to improve early diagnosis and patient outcomes.

Skin Cancer Detection Using Deep Learning

<h2>Project Overview</h2><p>In this project, we have tried to make the challenge of creating a skin cancer detection system using deep learning easier. Based on the dataset of skin cancer images, we fine-tune two advanced pre-trained models of EfficientNetB4 and DenseNet121, as well as a basic CNN to classify skin lesions into different types such as melanoma, pigmented benign keratosis, etc. With EfficientNetB4, we obtained an accuracy of over 80%.</p><p>The outcome is forecasts that help doctors in the actual process of diagnosing and may help eliminate deadly diseases at an early stage. Whether you&rsquo;re a young professional interested in healthcare or an everyday internet user wondering how AI can improve human life, this project demonstrates how machine learning can help.</p><h2>Prerequisites</h2><p><strong>Before we jump into the code, here&rsquo;s what you&rsquo;ll need:</strong></p><ul><li>An understanding of Python programming and usage of Google Colab  </li><li>Basic knowledge about deep learning and medical images.  </li><li>Comfortable using frameworks like Tensorflow, Keras, Numpy, OpenCV, and Seaborn to handle data and build models and visualize data and performance of models  </li><li>Skin cancer dataset.</li></ul><p>Once you organize these tools, you will notice how almost all of them can be used in the following step. Also, do not stress if you are not a Python master&mdash;through the tutorial, you will understand every line of the code!</p><h2>Approach</h2><p>The approach involves building, training, and evaluating deep learning models on a skin cancer dataset. We use image-processing techniques and deep-learning architectures to classify skin lesions into different types of cancer. By using pre-trained models like DenseNet and EfficientNet, we enhance the performance of the classification system while also improving accuracy.</p><p><strong>The major steps involve:</strong></p><ul><li>Obtaining and preparing data (augmentation, resizing, normalizing)  </li><li>Training and measuring the performance of several architectures  </li><li>Visualizing performance with confusion matrices and accuracy plots</li></ul><h2>Workflow and Methodology</h2><p><strong>This project can be divided into the following basic steps:</strong></p><ul><li><strong>Data Collection</strong>: We collected the skin cancer dataset labeled with different cancer names from Kaggle   </li><li><strong>Data preprocess</strong>: To improve the model performance and achieve higher accuracy, we applied different preprocessing techniques. First, we augmented the dataset to create a balanced dataset. Then we resized and normalized the images in 0 to 1 pixel values.  </li><li><strong>Model Selection</strong>: In this project, there are three models used (Custom CNN, EfficientNetB4, and DenseNet21).  </li><li><strong>Training and Testing</strong>: Each of the Models has been trained on the preprocessed dataset and later, tested on the dataset that was not used during training.  </li><li><strong>Model Evaluation</strong>: The evaluation of the model's performance is done by evaluating accuracy, precision, recall, confusion matrix, etc.</li></ul><h3>The methodology includes</h3><ul><li><p><strong>Data Preprocessing</strong>: The images are resized, normalized, and augmented to improve the performance of models based on them.</p></li><li><p><strong>Model Training</strong>: Each model is trained with 100 epochs to enhance the level of performance.</p></li><li><p><strong>Evaluation:</strong> Standard metrics (accuracy, precision, recall, f1-score, and confusion matrix) are applied to assess the efficiency of the models.</p></li></ul><h2>Dataset Collection</h2><p>The dataset we used had 2,500 images which were scaled through augmentation to 4,500 images. The dataset was divided in the following manner 80/20 which means 80% of the data was used for training the model and 20% each for validation of the model.</p><h2>Data Preparation</h2><p>The dataset was pre-processed by resizing the images to a size of 128 * 128 pixels and scaling the pixels to the range 0 to 255. To increase the variability of the dataset, primarily data augmentation techniques were applied.</p><h3>Data Preparation Workflow</h3><ul><li>Load Dataset from Google Drive  </li><li>Rotation, flipping, and changes in contrast, among others, are employed to increase the diversity of the datasets.  </li><li>Process and Resize as per Standards used in the model. This helps to standardize the input of the models.  </li><li>Further, the collected dataset has to be split into training and validation sets.</li></ul>


<h2>Code Explanation</h2><h3><span style="color: inherit; font-family: inherit; font-size: 1.5rem;">Mounting Google Drive</span></h3><p>This command mounts your Google Drive to the indicated folder path (/content/drive). After this step has been performed, you will need to allow access to your Google Drive account. After the access has been granted, reading and writing files will become straightforward as you can do this straight from your Drive, which is very helpful in loading datasets and saving the results of the models during the project.<br></p>


<h4>Import the necessary libraries.</h4><p>This code block imports all the required libraries for this project for creating, training, and evaluating models. It also imports image processing libraries like PIL and OpenCV for handling images, and matplotlib and seaborn for data visualization. Scikit-learn utilities facilitate model evaluation using metrics such as confusion matrices.<br></p>


<h3><span style="color: inherit; font-family: inherit; font-size: 1.5rem;">Data collection and preparation</span></h3><h5>Load Dataset</h5><p>This section of code is mainly focused on arranging the paths of the dataset. It starts by guiding the program to the main folder containing the skin cancer Datasets located on Google Drive. After that, it defines two different paths. One for the training set and another for the validation set.<br></p>


<h4>Listing categories</h4><p>The code sets the size of the images, creates a list to hold the names of different classes, checks which classes are available in the training folder, and then prints those class names. This makes it easier to keep track of and understand the different types of images that will be used for training the model.<br></p>


<p>This function iterates over different folders containing categories of images. Where it also performs reading and resizing images. Also keeps count of images across all categories and stores the processed images alongside their corresponding class numbers in a list. This takes care of the image preparation needed for training a model afterward.</p>


<p>This code is called the process_data function. This function processes all the images in the training folder, resizes them, labels them by category, and then prints the total number of training images processed.</p>


<p class="">This code creates a visual bar chart that displays the count of images in each class for the training data. It helps in visualizing the distribution of data across different categories. This highlights whether it is balanced or skewed.</p>


<p>This code calls the process_data function. This function processes all the images in the validation folder, resizes them, labels them by category, and then prints the total number of validation images processed.</p>


<p class="">This code creates a visual bar chart that displays the count of images in each class for the validation data.</p>


<p class="">This code&rsquo;s task is to arrange and ready images (X_train) and labels (Y_train) for conducting the training process. It reshapes the images in the required form of 128 x 128 pixels and 3 color channels and creates NumPy arrays ready to be fed to a neural network.</p>


<p class="">This code&rsquo;s task is to arrange and ready images (X_test) and labels (Y_test) for conducting the validation process. It reshapes the images in the required form of 128 x 128 pixels and 3 color channels and creates NumPy arrays ready to be fed to a neural network.</p>


<h3><span style="color: inherit; font-family: inherit; font-size: 1.5rem;">Visualization</span></h3><p>This code randomly selects three images from every category in the training dataset and arranges them in a grid. Each image features its category name as the title. It makes an easy visualization of a sample from each class.<br></p>


<h3><span style="color: inherit; font-family: inherit; font-size: 1.5rem;">Model Building</span></h3><h5>Building a Basic CNN Model</h5><p>This code builds a simple architecture of a Convolutional Neural Network (CNN) to classify images into 9 different classes. </p><p>The first two layers are the convolutional layers, comprising 32 filters and 64 filters receptively. We use ReLU activation for these two levels and use MaxPooling to reduce the size of the shape afterward. After flattening the output it passes through a fully connected layer of 128 neurons with ReLU activation. The final layer uses <strong>softmax activation</strong> with 9 output neurons for multi-class classification. </p><p>The model is built to use the Adam optimizer and sparse categorical cross-entropy loss while monitoring the accuracy of the model.<br></p>


<p class="">This code creates a checkpoint and saves the best version of the model while training depending on the validation accuracy. The model is trained with 100 epochs, with a batch size of 32 using the training data. The checkpoint is set only to basic_cnn.h5 if the loss of the validation decreases and in this way, the best model is saved for further use.</p>


<p class="">This code produces two plots side by side. One plot displays the CNN model's accuracy improvement over time. And the other shows the changes in loss. This effectively visualizes the model's performance throughout the training and validation phases.</p>


<h4>Evaluating Model performance</h4><p>This code evaluates the CNN model&rsquo;s performance on the training, validation, and test datasets by assessing accuracy and loss for each. It offers a clear insight into the model's performance.<br></p>


<p class="">This code loads a pre-trained model named <code>basic_cnn.h5</code> using TensorFlow's <code>load_model</code> function and assigns it to the variable <code>model</code>. It then evaluates the model's performance on the test data (<code>X_test</code> and <code>Y_test</code>), returning the loss and accuracy metrics. The <code>loss2</code> variable stores the computed loss, while <code>accuracy</code> holds the accuracy value. Finally, it prints the accuracy as a percentage, formatted to two decimal places. This output gives a clear indication of the model's effectiveness on unseen test data.</p>


<h4>Plotting Confusion Matrix</h4><p>The code predicts the test data and creates a confusion matrix. It provides a classification report. The confusion matrix is used to visually analyze the number of correct and incorrect predictions. The classification report presents class-wise metrics for the performance of the model. That is useful in assessing the model&rsquo;s ability.<br></p>


<h4>Building a Densenet121</h4><p>Here is the implementation of the DenseNet121 architecture. In the architecture, ImageNet weights are included for all sections except for the top layers. Additionally, all the layers in the model are frozen. A New Sequential model is developed based on DenseNet21. Then comes the applying flattening, a dense layer with 512 units and ReLU followed by a softmax layer for multi-class classification. </p><p>The model employs the Adam optimization algorithm with a learning rate of 0.0001, while the loss function utilized is sparse categorical cross-entropy, which is necessary to improve the performance on a classification task.<br></p>


<p class="">The model is trained with 100 epochs, with a batch size of 64 using the training data.</p>


<p class="">This code produces two plots side by side. One plot displays the DenseNet21 model's accuracy improvement over time. And the other shows the changes in loss. This effectively visualizes the model's performance throughout the training and validation phases.</p>


<p class="">This code evaluates the DesnseNet21 model&rsquo;s performance on the training, validation, and test datasets by assessing accuracy and loss for each. It offers a clear insight into the model performance.</p>


<p class="">This code evaluates the DenseNet121_custom_model on the test data (X_test and Y_test), calculating the loss and accuracy. The loss variable holds the loss value, and accuracy stores the accuracy metric. It then prints the accuracy as a percentage, formatted to two decimal places, providing a quick summary of the model's performance on the test dataset.</p>


<p class="">The code predicts the test data and creates a confusion matrix. It provides a classification report. The confusion matrix is used to visually analyze the number of correct and incorrect predictions. The classification report presents class-wise metrics for the performance of the model. Which is useful in assessing the model&rsquo;s ability.</p>


<h4>Building an Efficientnet_b4</h4><h5>Installing EfficientNet Library</h5><p>The command <code>!pip install -q efficientnet</code> installs the <code>efficientnet</code> library quietly (without verbose output). This library provides pre-trained EfficientNet models, which are commonly used for tasks in image classification due to their efficiency and high performance.<br></p>


<h4>Building a Custom Classifier with Pre-trained EfficientNetB4</h4><p>In this code block, we load a pre-trained EfficientNetB4 model using weights that have been trained on the ImageNet dataset. By setting include_top=False, we exclude the fully connected layers. To preserve the pre-trained layers, set trainable=False. </p><p>A new sequential model is built using EfficientNetB4 as the base. It begins with a Global Max Pooling layer. This is followed by a dense layer, with 256 units. Batch normalization and 50% dropout are applied to prevent overfitting. The final layer is a dense softmax layer for classification. The Adam optimizer with a sparse categorical cross-entropy loss is used to compile the model.<br></p>


<p class="">This code trains the EfficientNetB4 model on the dataset for 100 epochs using a batch size of 64.</p>


<p class="">This code evaluates the EfficientNetB4 model&rsquo;s performance on the training, validation, and test datasets by assessing accuracy and loss for each. It offers a clear insight into the model performance.</p>


<p class="">This code evaluates the e_model_b4 model on the test data (X_test and Y_test), calculating the loss and accuracy. It then prints the accuracy as a percentage, formatted to two decimal places, providing a summary of the model's performance on the test dataset.</p>


<h3><span style="color: inherit; font-family: inherit; font-size: 1.25rem;">Image Classification and Display with Pre-trained EfficientNetB4 Model</span></h3><p><strong>{start_text}</strong><br>This code loads a pre-trained model (efficientnet_b4_model.h5) and an image from a specified path, resizing the image to fit the model's input requirements. The image&rsquo;s pixel values are normalized to a range of 0 to 1, and the array's dimensions are expanded to include a batch size, making it ready for the model&rsquo;s input format. The model then predicts the class of the image, retrieves the predicted class label, and prints it. Finally, it displays the image with a title showing the predicted class, providing a visual alongside the classification result.<br></p>


<h2>Project Conclusion</h2><p>The current work aimed to develop a robust solution for skin cancer detection based on <a href="https://www.aionlinecourse.com/ai-basics/deep-learning" target="_blank">deep learning</a> principles. Using such <a href="https://www.aionlinecourse.com/ai-basics/convolutional-neural-network" target="_blank">advanced CNN</a> architectures as EfficientNetB4 and DenseNet121, we developed a classifier that identified different skin cancers with more than 80% accuracy. Unlike most academic projects, this is not just a project work, it demonstrates how deep learning in health care can be used to change early detection and diagnosis and give doctors a tool that makes a difference.</p><p>Throughout the journey, we tackled real-world issues: from working with a limited and skewed data sample to how our model couldn’t overlearn this data to prevent it from being ineffective in practice. Thus, applying such smart practices as data augmentation, regularization (dropout), and transfer learning on previously trained models, we aimed to achieve high results with minimal computational overhead.</p><p>Regardless of whether you operate in the field of biomedical research and development, healthcare-oriented start-ups, or AI technology, the approaches and instruments represented here contain strong guidelines for using <strong>AI in diagnostics</strong>. With additional refinement, this system may be expanded far beyond mammography toward other varieties of medical imaging, and be of exceptional use to the numerous members of the healthcare community who embrace it.</p><h2>Challenges and Troubleshoot</h2><p>Implementing such a large project, of course, has its own challenges. Let’s break them down and look at how you can overcome them:</p><ul><li><p><strong>Data Imbalance</strong>: A few categories of skin cancer may not have enough data which may lead to biased predictions and poor performance on the minority class.   </p><ul><li><strong>Solution</strong>: Implement data augmentation strategies such as rotation, flipping, or zooming enhancing the class size of the datasets where the class instances are underrepresented for better generalization.  </li></ul></li><li><p><strong>Small Dataset</strong>: Limitation of data may result in inadequate performance of the model.   </p><ul><li><strong>Solution</strong>: Utilize Transfer Learning by applying the EfficientNet or DenseNet pre-trained models which are less data-greedy and designed for more adaptation.  </li></ul></li><li><p><strong>Overfitting</strong>: When the data sample is limited or the data has an imbalanced nature, the model easily overfits and performs badly when generalizing.   </p><ul><li><strong>Solution</strong>: Incorporate Dropout layers to control overfitting and also implement early stopping when validation does not show any improvements.</li></ul></li><li><h2>Computational Resources: Training deep artificial networks requires a large amount of resources</h2><ul><li><h2>Solution: Train on the free GPU offered by Google Colab or use a lighter pre-trained model.</h2></li></ul></li></ul><h2>FAQ</h2><p><strong>Question 1: Which type of models were applied in skin cancer detection?</strong><br /><strong>Answer:</strong> For classification purposes, we employed three different models. Basic CNN model, pre-trained CNN architectures which are DenseNet121 and EfficientNetB4.</p><p><strong>Question 2: How accurate is the model?</strong><br /><strong>Answer:</strong> EfficientNetB4 gave the highest accuracy above 80% on the test set.</p><p><strong>Question 3: What type of dataset was employed?</strong><br /><strong>Answer:</strong> The dataset we used had 2,500 images and the image categories included, melanoma, benign keratosis, and basal cell carcinoma. Which were scaled through augmentation to 4,500 images. </p><p><strong>Question 4: Why use pre-trained models like EfficientNetB4 and DenseNet121 in medical image classification?</strong><br /><strong>Answer:</strong> Most of these models have been trained on large-scale datasets so are capable of learning a series of patterns in images. It is uncommon to make wrong predictions using it especially when working with small-sized data sets.</p><p><strong>Question 5: How is overfitting handled in medical image classification?</strong><br /><strong>Answer:</strong> To avoid signs of overfitting we employed other strategies such as implementing the Dropout layer and other forms of stopping once the model starts to over-fit.</p>

Skin Cancer Detection Using Deep Learning

Project Outcomes

Requirements:

Project Description