<p>Ready to explore polynomial regression? Imagine yourself driving around data - uncovering some hidden structure, some pattern that isn't a simple line. Polynomial regression is like enhancing the power of the model that helps to capture complex curves and trends. Increasing the accuracy for predicting the result and accomplishing actual tasks is such a great idea.</p>

Learn how to implement polynomial regression to capture complex patterns in data. Discover applications in finance, healthcare, and forecasting with detailed insights and methods.

Learn to Build a Polynomial Regression Model from Scratch

<h2>Project Overview</h2><p>In this project, you will be learning how to create your polynomial regression model from scratch. You will realize how to mold a basic linear model to your advantage. First, let’s collect data and clean it. After that, we will discuss how polynomial regression is different from simple linear regression. Last, we will proceed with coding step by step while explaining it in detail. By the end of this project, you will have a working model that can handle non-linear trends perfectly. Ready to start? Let’s dive in!</p><h2>Prerequisites</h2><ul><li><strong>Basic Python Skills</strong>: Be able to write loops, functions, and variables.  </li><li><strong>Understanding of Linear Regression</strong>: Familiar with drawing a straight line to fit data.  </li><li><strong>Basic Math Knowledge:</strong> Knowledge in simple algebra, power, and exponentiation.  </li><li><strong>Libraries: NumPy and Matplotlib</strong>: Familiarity with calculations, data manipulations, and data visualization.</li></ul><p>With these basics, you’re ready to start the project!</p><h2>Approach</h2><p>Our approach starts by setting up the dataset. After that, we'll use the dataset to train and test our model. We start with a quick review of linear regression to see where polynomial regression can add value. Next, we’ll transform our data by adding polynomial features. This allows our model to capture curves instead of just straight lines. Once the data is prepared, we’ll code the polynomial regression model step-by-step. We will employ the Python programming language and its NumPy library for numerical calculations and transformations. After training, we'll present the results using Matplotlib to assess the data-fitting ability of the model. </p><p>In the end, we will do a comparison of these two models and perform a more advanced analysis to show the clear benefit of using this model in practical applications dealing with databases.</p><h2>Workflow</h2><ol><li><strong>Data Collection</strong>: First, we gathered the datasets made of the images of healthy and infected leaf samples.  </li><li><strong>Data preprocessing</strong>: Once we gathered samples, we handled missing values. Then the images were normalized. After that, it was divided into data for training and others for validation.  </li><li><strong>Feature Engineering</strong><br />Add polynomial features to the dataset, which will allow our model to fit curves.  </li><li><strong>Model Building</strong>: Implementing polynomial regression from scratch using Python and NumPy to handle the math.  </li><li><strong>Training the Model</strong>: The model is then trained on the training dataset to learn from the patterns.  </li><li><strong>Evaluation</strong>: Carrying out model testing and its performance assessment.  </li><li><strong>Comparison with Linear Regression</strong><br />Compare results with a basic linear regression model to highlight the power of polynomial regression.</li></ol><h2>Methodology</h2><p>To construct a polynomial regression model, we start by increasing the dimension of our features by adding polynomial terms of the original features like squares and cubes of the original respectively. This enables the model to fit higher-order trends. We then use least squares estimation to train the model and this is just a concept of getting closer to the real values. By mapping each step, from feature engineering up to training and evaluation, we shall not only know how it works but also find out why the model suits non-linear data. We can visualize and compare the effects of polynomial regression, thus making a practical and useful methodology!</p><h2>Data Collection</h2><p>First, we collected a public dataset that shows a non-linear pattern. This will allow us to see the benefits of polynomial regression. You can also use real-world data like housing prices or stock trends. In this dataset, values fluctuate in complex ways.</p><p>If you're a beginner, try to create artificial data with a curved pattern. This way at least you will be in control of the data. This makes it easier to know whether the model we are using is functioning as required.</p><h2>Data Preparation</h2><p>Once the dataset has been collected, it is then time to prepare the data set. We will start with the missing values so that it doesn’t mess up our data set. Following this, we will normalize our features to ensure all of them are in the same range. This is especially important in polynomial regression. Also, the last step to perform is transforming (squaring, cubing, or taking to higher powers) the original built features to make new features. This transformation enables our model to fit curves rather than mere straight lines. So that more relationships can be captured.</p><h3>Data Preparation Workflow</h3><ul><li><strong>Handle Missing Values</strong>: Impute missing values so that it does not reduce the total number of features in a dataset on which the training is going to be done.  </li><li><strong>Feature Scaling:</strong> Scale features to the same scale for training an accurate model.  </li><li><strong>Generate Polynomial Feature</strong>s: To incorporate non-linear relationships, add polynomial features.  </li><li><strong>Split the Data:</strong> Divide the available data into training and validation data.<br /><strong>Final Check:</strong> Check all features for modeling with the conditions that no outliers are influencing the results and no need for scale transformations.</li></ul>

<h2><span style="color: inherit; font-family: inherit; font-size: 1.75rem; font-weight: 500;">Code explanation</span></h2><p>Here&rsquo;s what is happening under the hood. Let&rsquo;s go through it step by step:</p><h4>Mount Google Drive</h4><p>Mount your Google Drive to access and save datasets, models, and other resources.</p>


<h3>Suppress Warnings</h3><p>It excludes non-critical warnings from the output, producing a cleaner view of the results.</p>


<h3>Install Required Libraries</h3><p>This installs the LightGBM and Scikit-Learn libraries. LightGBM is often used for gradient boosting in machine learning, while Scikit-Learn provides essential tools for building and evaluating models.</p>


<h3>Import Libraries</h3>
<p>This section imports the necessary library for data manipulation (pandas, numpy), graph plotting (seaborn, plotly, matplotlib), statistical operations (scipy), and machine learning models (sklearn).</p>


<h2><span style="color: inherit; font-family: inherit; font-size: 1.75rem; font-weight: 500;">Load the Dataset</span></h2><p>The code employs the pandas library to retrieve the dataset from Google Drive. The head() method shows the first few records within the dataset to have an overview of the dataset.</p>


<h3>Check the Dataset Dimensions</h3><p>This statement displays the dimension of the dataset in terms of how many rows and columns it has. For example, if it says (317, 7), this means that the dataset has 317 rows and 7 columns, thereby helping to verify that the correct amount of data was indeed loaded.</p>


<h3>Display Column Names</h3><p>The code is designed to print the names of the dataset's columns. This helps to quickly understand the features present in the data to be analyzed. </p>


<h3>Dataset Overview</h3><p>Before starting the data cleaning or the analysis process, it is wise to understand the structure of the data and look for missing values. This command is very useful in undertaking this task, as it lets the user easily perceive the characteristics of the data.</p>


<h3>Renamed Columns for Easy Analysis</h3><p>The code alters the column names of the data DataFrame to make simpler names and produces a new DataFrame df with such column names. It then uses df.head(10) to present the first 10 rows to check the modifications made. </p>


<h2><span style="color: inherit; font-family: inherit; font-size: 1.75rem; font-weight: 500;">Visualizing Data Distribution and Outliers for Deeper Insights into Points</span></h2><p>This code creates three visualizations for the Points variable: These include a density plot, a transformed density plot (using the square root of Points), and a boxplot. For detail, each of the plots assists us in identifying different aspects of data distribution such as spread, skewness, and outliers. The layout is changed to make it easier to view the plots, and the plots are shown beside each other for contrast.</p>


<h3>Viewing the Last 100 Rows of the Dataset</h3><p>The code displays the last 100 rows of the df DataFrame. </p>


<h3>Creating Violin and Box Plots for Each Variable by Team</h3><p>The function plotting_box_violin_plots() is constructed to create violin and box plots for any given variable against the &lsquo;Team&rsquo; column to better visualize the spread and the central tendency of the data. This function is subsequently called in a loop to draw these plots for each variable (&lsquo;WL&rsquo;, &lsquo;Yoga&rsquo;, &lsquo;Laps&rsquo;, &lsquo;WI&rsquo;, &lsquo;PAFS&rsquo;) and allow for comparison for each of them against the &lsquo;Team&rsquo; label. </p>


<h2><span style="color: inherit; font-family: inherit; font-size: 1.75rem; font-weight: 500;">Identifying Outliers in Selected Columns</span></h2><p>The function find_outliers() is employed to determine outliers in a given column by computing the Interquartile Range (IQR) of that column. In a for-loop manner, this function is executed column-wise on the columns (&lsquo;WL&rsquo;, &lsquo;Yoga&rsquo;, &lsquo;Laps&rsquo;, &lsquo;WI&rsquo;, &lsquo;PAFS&rsquo;) and any values that are beyond the composed range are displayed, making it possible to identify data points that need additional attention.</p>


<h3>Eliminating Certain Rows for Data Cleaning Purposes</h3><p>The code constructs a revised DataFrame df_clean such that the rows with index numbers 142, 143, and 144 in df are excluded. In addition, the command df_clean.shape is employed to provide information regarding the size of the cleaned DataFrame.</p>


<h3>Replacing Invalid Values with NaN in the Cleaned Data</h3><p>The specified code eliminates all the instances of the value 1111111.0 present in the 'WL' column of df_clean by substituting it with NaN denoting it as missing data. On displaying df_clean['WL&rsquo;] the modified column is presented which helps in identifying the records that will require additional treatment during the analysis. </p>


<h2><span style="color: inherit; font-family: inherit; font-size: 1.75rem; font-weight: 500;">Calculating and Displaying Missing Data Proportion</span></h2><p>The dataframe ncounts display the amount of missing data per column of the dataframe df_clean. It resorts to df_clean.isna().mean() to get the count of the NaN values as a percentage and the column is then transposed and given the name data_missing for ease of reference.</p>


<h3>Visualizing Missing Data Proportion</h3><p>The purpose of this code is to generate a horizontal bar graph showing the proportion of missing values against each of the columns in ncounts. Here, ncounts.plot(kind='barh', title='% of missing values across each column&rsquo;) gives an intuitive understanding of the missing data in the columns and assists in determining which sections may need data winnowing. Lastly, the output has been generated using plt.show().</p>


<h3>Comparing Data Shapes Before and After Dropping Missing Values</h3><p>This comparison helps you understand the impact of dropping rows or columns with missing data on the dataset&rsquo;s size.</p>


<h3>Getting an Overview of the Cleaned Data</h3><p>This summary provides insight into the arrangement of the prepared dataset before carrying out any evaluation.</p>


<h3>Filling Missing Values in the 'WL' Column</h3><p>The code replaces all NaN values in the 'WL' column with -1.</p>


<h2><span style="color: inherit; font-family: inherit; font-size: 1.75rem; font-weight: 500;">Visualizing the Effect of Filling Missing Values with Mean and Median</span></h2><p>The presented code illustrates the effect on the distribution of the WL column when its missing values are replaced with the mean or the median. The placing of these two plots next to each other facilitates the comparison between the two imputation approaches and gives an idea of which method is better at retaining the original characteristics of the data.</p>


<h3>Calculating Mean 'WL' per Team</h3><p>This piece of code computes the average of the WL column for every distinct Team in df_clean, utilizing groupby(), and outputs the result in the form of a dictionary.</p>


<h3>Replacing NaN Values in the 'WL' Column on a Team-Wise Basis</h3><p>This code goes through every row of df_clean. In cases where the WL value is missing within a specific column, the void is filled with the mean WL value, which is associated with the team from the mean_WL dictionary.</p>


<h3>Depicting the Distribution of 'WL', Once the NA's are Filled</h3><p>Here&rsquo;s a code that is aimed at creating a distribution plot for the column WL in a cleaned data frame where missing data has been filled with mean scores corresponding to each of the teams. </p>


<h3>Using Simple Imputer to Handle Missing Values</h3><p>This snippet fills up the missing values in the defined columns of the data frame by employing a SimpleImputer using the mean strategy. A new data frame, named si_impt_df is created, which consists of all the values present in the original data frame without any missing values, as each column has been filled with the mean of that particular column.</p>


<h3>Applying Iterative Imputer in Handling Missing Data</h3><p>This script executes an IterativeImputer with 10 max_iter to handle missing values in the specified columns. It predictively improves any missing values using other features and generates thus imputed data in a new DataFrame called ITI_impt_df. This imputation is complex compared to filling values using a mean.</p>


<h3>Using KNN Imputer for Missing Value Imputation</h3><p>This code employs a KNNImputation with n_neighbors=3 that fills in missing values by averaging the values of the three closest neighbors. The imputed data is returned in a new DataFrame, KNN_impt_df that provides instance-based filling of missing values in the other entries of the dataset.</p>


<h2><span style="color: inherit; font-family: inherit; font-size: 1.75rem; font-weight: 500;">Handle Missing Value Imputation through Iterative Imputer with LightGBM</span></h2><p>This code defines an IterativeImputer object, the estimator of which is a LightGBMRegressor used to perform the missing value imputation in the given columns. The imputed data set is stored in a new DataFrame called lgbm_impt_df, making highly accurate imputations with the help of LightGBM capability for prediction.</p>


<h3>Inspecting and Renaming Columns in the Imputed Data</h3><p>This section of the code assigns the lgbm_impt_df DataFrame filled with LightGBM-imputed data and creates a new DataFrame called df_new. </p>


<h3>Visualizing Distributions and Boxplots of Transformed Features</h3><p>The function creates the grid of plots per number of features in df_new that examines their distributions and boxplots. It also transformed the distributions (square root and cube root) and created boxplots for WL, Yoga, and PAFS to enable a clear comparison of how the data is spaced out as well as the possible outliers. </p>


<h3>Adding 'Points' and 'Team' Columns to the Imputed Data</h3><p>This code copies the 'Points' and 'Team' columns from df_clean into the df_new FrameData.</p>


<h3>Calculating the Correlation Matrix for Numerical Columns</h3><p>This piece of code filters out numerical columns from the df. It also computes the correlation matrix with the help of numerical_df.corr(). The correlation matrix correlation_matrix outlines how numerical attributes are related helping in spotting out strong relationships that may be positive or negative for further investigation.</p>


<h3>Using a Heatmap to Illustrate the Correlation Matrix</h3><p>The code provided here uses sns.heatmap() to construct a correlation matrix as a heatmap and adds the values of correlation on the map (annot=True). This is utilized to show the existence of positive and negative correlations, making strong relationships between variables apparent.</p>


<h3>Creating a Pair Plot to Visualize Relationships Between Features</h3><p>This code creates scatter plots of all feature pairs in df_new, with points colored by the Team column. This helps to visually explore relationships and patterns between variables across different teams.</p>


<h3>Creating a Pair Plot to Visualize Relationships Between Features</h3><p>This code creates scatter plots of all feature pairs in df_new. This helps to visually explore relationships and patterns between features.</p>


<p><strong>Calculating Custom Correlation Using Ranked Data</strong></p><p>The function aionlinecourse_corr() calculates a custom correlation coefficient between two variables, x, and y, in the DataFrame df.</p>

<h3>Calculating Custom Correlation Between Variables</h3><p>This piece of code implements a function called aionlinecourse_corr(), which computes a certain correlation coefficient which is based on rankings. It is then used to find out how well Points correlate with the rest of the variables (WI, Laps, PAFS, Yoga) and helps see how strong the associations are towards those variables.</p>


<h2><span style="color: inherit; font-family: inherit; font-size: 1.75rem; font-weight: 500;">Creating a Sample DataFrame and Grouping by Team</span></h2><p>This snippet of code draws a random sample containing 50 rows of the df_new data frame with emphasis on the 'Team' and 'Points' columns. The sample is then divided into groups regarding 'Teams' to assess how many times each one occurs effectively summarizing how the teams are spread out in the sample.</p>


<h3>Creating Probability Plots per Each Distinct Team</h3><p>This script generates probability plots for the distribution of Points for each distinct team contained in sample_df and arranges them in a grid.</p>


<h3>Calculating the Ratio of Maximum to Minimum Standard Deviation Across Teams</h3><p>This piece of code computes the standard deviation of Points for each of the teams in sample_df and afterward calculates and returns the value of the maximum standard deviation for all the teams to the smallest standard deviation. This helps to understand the differences in variability in the Points among the teams and how far apart or clustered the team's scores are from each other.</p>


<h3>Creating an ANOVA Table for an Analysis of Variance for Groups</h3><p>The code creates an empty ANOVA table called anova_table and has specific sections for different sources of variation along with their corresponding Sum of Squares (SS), degrees of freedom (df), Mean Square (MS), F value, P value and F critical values for between-groups, within groups and total. </p><p>The Sum of Squares Between Groups (SSTR) is obtained as the difference between the mean Points of each team to the overall mean (x_bar) to capture the extent of variation among team means. The SSTR calculated is then populated into the ANOVA table under the &ldquo;Between Groups&rdquo; section in the &ldquo;SS&rdquo; column. This is important in the analysis of the research question that seeks to determine whether there are significant differences within teams.</p>


<h3>Computation of the Between-Groups Sum of Squares (SSE)</h3><p>SSE is computed in the code by determining the variance within each of the teams (clusters) and multiplying that by the sample size of each team subtracting one. This measure captures the variability in Points to the Scores within the Team Showing how much each score varies from the average score of the team.</p>


<h3>Updating ANOVA Table with Sum of Squares Within Groups (SSE)</h3><p>The code adds the total SSE for Points to the "SS" column, which is listed as "<strong>Within Groups</strong>" in the anova_table. This update helps to finalize the accurate sum of squares and prepares the ANOVA table for subsequent operations.</p>


<h3>Calculating and Adding the Total Sum of Squares to ANOVA Table</h3><p>This section of code calculated the Total Sum of Squares (SST) by aggregating the two components, SSE and SSTR for Points, and it considered the overall variations present in the data set. The total is then inserted into the &ldquo;SS&rdquo; row of the anova_table in the column labeled &lsquo;Total&rsquo;, thereby finalizing the sum of squares section in the ANOVA table.</p>


<h3>Calculating and Adding Degrees of Freedom to the ANOVA Table</h3><p>This code calculates and adds the degrees of freedom for "Between Groups," "Within Groups," and "Total" to the anova_table.</p>


<h3>Computation of Mean Squares, F Value, and P Value for ANOVA Table</h3><p>This code calculates the Mean Square (MS) for each variation category by dividing the Sum of Squares (SS) against the degrees of freedom (df). For the &ldquo;Between Groups&rdquo; effect, the F value is obtained from the ratio of MS of &ldquo;Between Groups&rdquo; to the MS of &ldquo;Within Groups.&rdquo; The P value for &ldquo;Between Groups&rdquo; is further computed based on the cumulative distribution function of the F-distributed random variable. These values are appended to anova_table facilitating analysis of the statistical significance of differences in groups.</p>


<h3>Calculating F Critical Value for Hypothesis Testing</h3><p>The present code calculates the F critical value for &ldquo;Between Groups&rdquo; in a two-tailed test by modifying the significant level alpha and incorporating the F-distribution. This value is also incorporated within anova_table to conduct the final hypothesis testing.</p>


<h2><span style="color: inherit; font-family: inherit; font-size: 1.75rem; font-weight: 500;">Interpreting the P-Value and Drawing a Conclusion</span></h2><p>This snippet of code explains the results obtained from the ANOVA output by calculating the P value for the &ldquo;Between Groups&rdquo; and comparing it with alpha. In the case where the P value is less than alpha, this leads to the rejection of the null hypothesis, more so indicating that there is a difference between the groups. If not, as a result, it states that the null hypothesis is not tenable. It then proceeds to display the F value, the P value, and the summary for further understanding.</p>


Learn to Build a Polynomial Regression Model from Scratch

Project Outcomes

Requirements:

Project Description