dropout_regularization_ann.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7mAOEIOw6yHt"
      },
      "source": [
        "<h2 align='center' style='color:blue'>Dropout Regularization In Deep Neural Network</h2>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kuJcNpfH6yHw"
      },
      "source": [
        "This is a dataset that describes sonar chirp returns bouncing off different services. The 60 input variables are the strength of the returns at different angles. It is a **binary classification problem** that requires a model to differentiate rocks from metal cylinders.\n",
        "\n",
        "Dataset information: https://archive.ics.uci.edu/ml/datasets/Connectionist+Bench+(Sonar,+Mines+vs.+Rocks)\n",
        "Download it from here: https://archive.ics.uci.edu/ml/machine-learning-databases/undocumented/connectionist-bench/sonar/sonar.all-data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "tKrSwdt-6yHx"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8N1s4OKJ6yHy"
      },
      "outputs": [],
      "source": [
        "import warnings\n",
        "warnings.filterwarnings('ignore')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7SprEGuB6yHz",
        "outputId": "b6edba13-bfc1-4a3f-ca1d-f782240d1149"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>0</th>\n",
              "      <th>1</th>\n",
              "      <th>2</th>\n",
              "      <th>3</th>\n",
              "      <th>4</th>\n",
              "      <th>5</th>\n",
              "      <th>6</th>\n",
              "      <th>7</th>\n",
              "      <th>8</th>\n",
              "      <th>9</th>\n",
              "      <th>...</th>\n",
              "      <th>51</th>\n",
              "      <th>52</th>\n",
              "      <th>53</th>\n",
              "      <th>54</th>\n",
              "      <th>55</th>\n",
              "      <th>56</th>\n",
              "      <th>57</th>\n",
              "      <th>58</th>\n",
              "      <th>59</th>\n",
              "      <th>60</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>199</th>\n",
              "      <td>0.0116</td>\n",
              "      <td>0.0744</td>\n",
              "      <td>0.0367</td>\n",
              "      <td>0.0225</td>\n",
              "      <td>0.0076</td>\n",
              "      <td>0.0545</td>\n",
              "      <td>0.1110</td>\n",
              "      <td>0.1069</td>\n",
              "      <td>0.1708</td>\n",
              "      <td>0.2271</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0141</td>\n",
              "      <td>0.0103</td>\n",
              "      <td>0.0100</td>\n",
              "      <td>0.0034</td>\n",
              "      <td>0.0026</td>\n",
              "      <td>0.0037</td>\n",
              "      <td>0.0044</td>\n",
              "      <td>0.0057</td>\n",
              "      <td>0.0035</td>\n",
              "      <td>M</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>174</th>\n",
              "      <td>0.0191</td>\n",
              "      <td>0.0173</td>\n",
              "      <td>0.0291</td>\n",
              "      <td>0.0301</td>\n",
              "      <td>0.0463</td>\n",
              "      <td>0.0690</td>\n",
              "      <td>0.0576</td>\n",
              "      <td>0.1103</td>\n",
              "      <td>0.2423</td>\n",
              "      <td>0.3134</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0040</td>\n",
              "      <td>0.0136</td>\n",
              "      <td>0.0137</td>\n",
              "      <td>0.0172</td>\n",
              "      <td>0.0132</td>\n",
              "      <td>0.0110</td>\n",
              "      <td>0.0122</td>\n",
              "      <td>0.0114</td>\n",
              "      <td>0.0068</td>\n",
              "      <td>M</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>31</th>\n",
              "      <td>0.0084</td>\n",
              "      <td>0.0153</td>\n",
              "      <td>0.0291</td>\n",
              "      <td>0.0432</td>\n",
              "      <td>0.0951</td>\n",
              "      <td>0.0752</td>\n",
              "      <td>0.0414</td>\n",
              "      <td>0.0259</td>\n",
              "      <td>0.0692</td>\n",
              "      <td>0.1753</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0236</td>\n",
              "      <td>0.0114</td>\n",
              "      <td>0.0136</td>\n",
              "      <td>0.0117</td>\n",
              "      <td>0.0060</td>\n",
              "      <td>0.0058</td>\n",
              "      <td>0.0031</td>\n",
              "      <td>0.0072</td>\n",
              "      <td>0.0045</td>\n",
              "      <td>R</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>129</th>\n",
              "      <td>0.1371</td>\n",
              "      <td>0.1226</td>\n",
              "      <td>0.1385</td>\n",
              "      <td>0.1484</td>\n",
              "      <td>0.1776</td>\n",
              "      <td>0.1428</td>\n",
              "      <td>0.1773</td>\n",
              "      <td>0.2161</td>\n",
              "      <td>0.1630</td>\n",
              "      <td>0.2067</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0118</td>\n",
              "      <td>0.0129</td>\n",
              "      <td>0.0344</td>\n",
              "      <td>0.0065</td>\n",
              "      <td>0.0067</td>\n",
              "      <td>0.0022</td>\n",
              "      <td>0.0079</td>\n",
              "      <td>0.0146</td>\n",
              "      <td>0.0051</td>\n",
              "      <td>M</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>126</th>\n",
              "      <td>0.0715</td>\n",
              "      <td>0.0849</td>\n",
              "      <td>0.0587</td>\n",
              "      <td>0.0218</td>\n",
              "      <td>0.0862</td>\n",
              "      <td>0.1801</td>\n",
              "      <td>0.1916</td>\n",
              "      <td>0.1896</td>\n",
              "      <td>0.2960</td>\n",
              "      <td>0.4186</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0153</td>\n",
              "      <td>0.0121</td>\n",
              "      <td>0.0096</td>\n",
              "      <td>0.0196</td>\n",
              "      <td>0.0042</td>\n",
              "      <td>0.0066</td>\n",
              "      <td>0.0099</td>\n",
              "      <td>0.0083</td>\n",
              "      <td>0.0124</td>\n",
              "      <td>M</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 61 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "         0       1       2       3       4       5       6       7       8   \\\n",
              "199  0.0116  0.0744  0.0367  0.0225  0.0076  0.0545  0.1110  0.1069  0.1708   \n",
              "174  0.0191  0.0173  0.0291  0.0301  0.0463  0.0690  0.0576  0.1103  0.2423   \n",
              "31   0.0084  0.0153  0.0291  0.0432  0.0951  0.0752  0.0414  0.0259  0.0692   \n",
              "129  0.1371  0.1226  0.1385  0.1484  0.1776  0.1428  0.1773  0.2161  0.1630   \n",
              "126  0.0715  0.0849  0.0587  0.0218  0.0862  0.1801  0.1916  0.1896  0.2960   \n",
              "\n",
              "         9   ...      51      52      53      54      55      56      57  \\\n",
              "199  0.2271  ...  0.0141  0.0103  0.0100  0.0034  0.0026  0.0037  0.0044   \n",
              "174  0.3134  ...  0.0040  0.0136  0.0137  0.0172  0.0132  0.0110  0.0122   \n",
              "31   0.1753  ...  0.0236  0.0114  0.0136  0.0117  0.0060  0.0058  0.0031   \n",
              "129  0.2067  ...  0.0118  0.0129  0.0344  0.0065  0.0067  0.0022  0.0079   \n",
              "126  0.4186  ...  0.0153  0.0121  0.0096  0.0196  0.0042  0.0066  0.0099   \n",
              "\n",
              "         58      59  60  \n",
              "199  0.0057  0.0035   M  \n",
              "174  0.0114  0.0068   M  \n",
              "31   0.0072  0.0045   R  \n",
              "129  0.0146  0.0051   M  \n",
              "126  0.0083  0.0124   M  \n",
              "\n",
              "[5 rows x 61 columns]"
            ]
          },
          "execution_count": 73,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# header = none -- the dataset has no header values\n",
        "df = pd.read_csv(\"./sonar_dataset.csv\", header=None)\n",
        "df.sample(5)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JUPOTjhe6yHz",
        "outputId": "20ec114a-8f6a-441f-e5bd-e50a5a9b6da8"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "(208, 61)"
            ]
          },
          "execution_count": 74,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "df.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "x5oEnxn_6yH0",
        "outputId": "2ff6084e-43cd-4c68-c194-4f31d74a9d4b"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "0     0\n",
              "1     0\n",
              "2     0\n",
              "3     0\n",
              "4     0\n",
              "     ..\n",
              "56    0\n",
              "57    0\n",
              "58    0\n",
              "59    0\n",
              "60    0\n",
              "Length: 61, dtype: int64"
            ]
          },
          "execution_count": 75,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# check for nan values\n",
        "df.isna().sum()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "677IiPD_6yH0",
        "outputId": "61d2b52e-be9e-4b7e-ac38-55004b33f188"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "Int64Index([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16,\n",
              "            17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33,\n",
              "            34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50,\n",
              "            51, 52, 53, 54, 55, 56, 57, 58, 59, 60],\n",
              "           dtype='int64')"
            ]
          },
          "execution_count": 76,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "df.columns"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "0VHPS5Us6yH0",
        "outputId": "d95d507f-5c87-44c4-d097-edb1374ce8f7"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "M    111\n",
              "R     97\n",
              "Name: 60, dtype: int64"
            ]
          },
          "execution_count": 77,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "df[60].value_counts() # label is not skewed"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jym4lvD56yH0",
        "outputId": "478d62bb-3ee5-40be-df0b-a27379c33303"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "0    R\n",
              "1    R\n",
              "2    R\n",
              "3    R\n",
              "4    R\n",
              "Name: 60, dtype: object"
            ]
          },
          "execution_count": 78,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "X = df.drop(60, axis=1)\n",
        "y = df[60]\n",
        "y.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vvxilFOd6yH1",
        "outputId": "be5ebc46-69a4-45ff-f554-b2233b195b57"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>R</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>59</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>101</th>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>184</th>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>141</th>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>191</th>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "     R\n",
              "59   1\n",
              "101  0\n",
              "184  0\n",
              "141  0\n",
              "191  0"
            ]
          },
          "execution_count": 79,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "y = pd.get_dummies(y, drop_first=True)\n",
        "y.sample(5) # R --> 1 and M --> 0"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FLRTpaNj6yH1",
        "outputId": "48bfd83c-44d4-4465-ca65-c822a869b443"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "R\n",
              "0    111\n",
              "1     97\n",
              "dtype: int64"
            ]
          },
          "execution_count": 80,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "y.value_counts()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "f6YceiNo6yH1",
        "outputId": "b588af82-ddd6-4b33-a4b8-16a514137634"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>0</th>\n",
              "      <th>1</th>\n",
              "      <th>2</th>\n",
              "      <th>3</th>\n",
              "      <th>4</th>\n",
              "      <th>5</th>\n",
              "      <th>6</th>\n",
              "      <th>7</th>\n",
              "      <th>8</th>\n",
              "      <th>9</th>\n",
              "      <th>...</th>\n",
              "      <th>50</th>\n",
              "      <th>51</th>\n",
              "      <th>52</th>\n",
              "      <th>53</th>\n",
              "      <th>54</th>\n",
              "      <th>55</th>\n",
              "      <th>56</th>\n",
              "      <th>57</th>\n",
              "      <th>58</th>\n",
              "      <th>59</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0.0200</td>\n",
              "      <td>0.0371</td>\n",
              "      <td>0.0428</td>\n",
              "      <td>0.0207</td>\n",
              "      <td>0.0954</td>\n",
              "      <td>0.0986</td>\n",
              "      <td>0.1539</td>\n",
              "      <td>0.1601</td>\n",
              "      <td>0.3109</td>\n",
              "      <td>0.2111</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0232</td>\n",
              "      <td>0.0027</td>\n",
              "      <td>0.0065</td>\n",
              "      <td>0.0159</td>\n",
              "      <td>0.0072</td>\n",
              "      <td>0.0167</td>\n",
              "      <td>0.0180</td>\n",
              "      <td>0.0084</td>\n",
              "      <td>0.0090</td>\n",
              "      <td>0.0032</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.0453</td>\n",
              "      <td>0.0523</td>\n",
              "      <td>0.0843</td>\n",
              "      <td>0.0689</td>\n",
              "      <td>0.1183</td>\n",
              "      <td>0.2583</td>\n",
              "      <td>0.2156</td>\n",
              "      <td>0.3481</td>\n",
              "      <td>0.3337</td>\n",
              "      <td>0.2872</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0125</td>\n",
              "      <td>0.0084</td>\n",
              "      <td>0.0089</td>\n",
              "      <td>0.0048</td>\n",
              "      <td>0.0094</td>\n",
              "      <td>0.0191</td>\n",
              "      <td>0.0140</td>\n",
              "      <td>0.0049</td>\n",
              "      <td>0.0052</td>\n",
              "      <td>0.0044</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0.0262</td>\n",
              "      <td>0.0582</td>\n",
              "      <td>0.1099</td>\n",
              "      <td>0.1083</td>\n",
              "      <td>0.0974</td>\n",
              "      <td>0.2280</td>\n",
              "      <td>0.2431</td>\n",
              "      <td>0.3771</td>\n",
              "      <td>0.5598</td>\n",
              "      <td>0.6194</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0033</td>\n",
              "      <td>0.0232</td>\n",
              "      <td>0.0166</td>\n",
              "      <td>0.0095</td>\n",
              "      <td>0.0180</td>\n",
              "      <td>0.0244</td>\n",
              "      <td>0.0316</td>\n",
              "      <td>0.0164</td>\n",
              "      <td>0.0095</td>\n",
              "      <td>0.0078</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0.0100</td>\n",
              "      <td>0.0171</td>\n",
              "      <td>0.0623</td>\n",
              "      <td>0.0205</td>\n",
              "      <td>0.0205</td>\n",
              "      <td>0.0368</td>\n",
              "      <td>0.1098</td>\n",
              "      <td>0.1276</td>\n",
              "      <td>0.0598</td>\n",
              "      <td>0.1264</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0241</td>\n",
              "      <td>0.0121</td>\n",
              "      <td>0.0036</td>\n",
              "      <td>0.0150</td>\n",
              "      <td>0.0085</td>\n",
              "      <td>0.0073</td>\n",
              "      <td>0.0050</td>\n",
              "      <td>0.0044</td>\n",
              "      <td>0.0040</td>\n",
              "      <td>0.0117</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0.0762</td>\n",
              "      <td>0.0666</td>\n",
              "      <td>0.0481</td>\n",
              "      <td>0.0394</td>\n",
              "      <td>0.0590</td>\n",
              "      <td>0.0649</td>\n",
              "      <td>0.1209</td>\n",
              "      <td>0.2467</td>\n",
              "      <td>0.3564</td>\n",
              "      <td>0.4459</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0156</td>\n",
              "      <td>0.0031</td>\n",
              "      <td>0.0054</td>\n",
              "      <td>0.0105</td>\n",
              "      <td>0.0110</td>\n",
              "      <td>0.0015</td>\n",
              "      <td>0.0072</td>\n",
              "      <td>0.0048</td>\n",
              "      <td>0.0107</td>\n",
              "      <td>0.0094</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 60 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "       0       1       2       3       4       5       6       7       8   \\\n",
              "0  0.0200  0.0371  0.0428  0.0207  0.0954  0.0986  0.1539  0.1601  0.3109   \n",
              "1  0.0453  0.0523  0.0843  0.0689  0.1183  0.2583  0.2156  0.3481  0.3337   \n",
              "2  0.0262  0.0582  0.1099  0.1083  0.0974  0.2280  0.2431  0.3771  0.5598   \n",
              "3  0.0100  0.0171  0.0623  0.0205  0.0205  0.0368  0.1098  0.1276  0.0598   \n",
              "4  0.0762  0.0666  0.0481  0.0394  0.0590  0.0649  0.1209  0.2467  0.3564   \n",
              "\n",
              "       9   ...      50      51      52      53      54      55      56  \\\n",
              "0  0.2111  ...  0.0232  0.0027  0.0065  0.0159  0.0072  0.0167  0.0180   \n",
              "1  0.2872  ...  0.0125  0.0084  0.0089  0.0048  0.0094  0.0191  0.0140   \n",
              "2  0.6194  ...  0.0033  0.0232  0.0166  0.0095  0.0180  0.0244  0.0316   \n",
              "3  0.1264  ...  0.0241  0.0121  0.0036  0.0150  0.0085  0.0073  0.0050   \n",
              "4  0.4459  ...  0.0156  0.0031  0.0054  0.0105  0.0110  0.0015  0.0072   \n",
              "\n",
              "       57      58      59  \n",
              "0  0.0084  0.0090  0.0032  \n",
              "1  0.0049  0.0052  0.0044  \n",
              "2  0.0164  0.0095  0.0078  \n",
              "3  0.0044  0.0040  0.0117  \n",
              "4  0.0048  0.0107  0.0094  \n",
              "\n",
              "[5 rows x 60 columns]"
            ]
          },
          "execution_count": 81,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "X.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "OxuBEyP16yH1"
      },
      "outputs": [],
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=1)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "aEo9vePg6yH2",
        "outputId": "e24fb06b-90a2-42c8-e2f9-7caf19f635b6"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>0</th>\n",
              "      <th>1</th>\n",
              "      <th>2</th>\n",
              "      <th>3</th>\n",
              "      <th>4</th>\n",
              "      <th>5</th>\n",
              "      <th>6</th>\n",
              "      <th>7</th>\n",
              "      <th>8</th>\n",
              "      <th>9</th>\n",
              "      <th>...</th>\n",
              "      <th>50</th>\n",
              "      <th>51</th>\n",
              "      <th>52</th>\n",
              "      <th>53</th>\n",
              "      <th>54</th>\n",
              "      <th>55</th>\n",
              "      <th>56</th>\n",
              "      <th>57</th>\n",
              "      <th>58</th>\n",
              "      <th>59</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>67</th>\n",
              "      <td>0.0368</td>\n",
              "      <td>0.0403</td>\n",
              "      <td>0.0317</td>\n",
              "      <td>0.0293</td>\n",
              "      <td>0.0820</td>\n",
              "      <td>0.1342</td>\n",
              "      <td>0.1161</td>\n",
              "      <td>0.0663</td>\n",
              "      <td>0.0155</td>\n",
              "      <td>0.0506</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0058</td>\n",
              "      <td>0.0091</td>\n",
              "      <td>0.0160</td>\n",
              "      <td>0.0160</td>\n",
              "      <td>0.0081</td>\n",
              "      <td>0.0070</td>\n",
              "      <td>0.0135</td>\n",
              "      <td>0.0067</td>\n",
              "      <td>0.0078</td>\n",
              "      <td>0.0068</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>0.0124</td>\n",
              "      <td>0.0433</td>\n",
              "      <td>0.0604</td>\n",
              "      <td>0.0449</td>\n",
              "      <td>0.0597</td>\n",
              "      <td>0.0355</td>\n",
              "      <td>0.0531</td>\n",
              "      <td>0.0343</td>\n",
              "      <td>0.1052</td>\n",
              "      <td>0.2120</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0078</td>\n",
              "      <td>0.0083</td>\n",
              "      <td>0.0057</td>\n",
              "      <td>0.0174</td>\n",
              "      <td>0.0188</td>\n",
              "      <td>0.0054</td>\n",
              "      <td>0.0114</td>\n",
              "      <td>0.0196</td>\n",
              "      <td>0.0147</td>\n",
              "      <td>0.0062</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>164</th>\n",
              "      <td>0.0163</td>\n",
              "      <td>0.0198</td>\n",
              "      <td>0.0202</td>\n",
              "      <td>0.0386</td>\n",
              "      <td>0.0752</td>\n",
              "      <td>0.1444</td>\n",
              "      <td>0.1487</td>\n",
              "      <td>0.1484</td>\n",
              "      <td>0.2442</td>\n",
              "      <td>0.2822</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0027</td>\n",
              "      <td>0.0077</td>\n",
              "      <td>0.0026</td>\n",
              "      <td>0.0031</td>\n",
              "      <td>0.0083</td>\n",
              "      <td>0.0020</td>\n",
              "      <td>0.0084</td>\n",
              "      <td>0.0108</td>\n",
              "      <td>0.0083</td>\n",
              "      <td>0.0033</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>179</th>\n",
              "      <td>0.0394</td>\n",
              "      <td>0.0420</td>\n",
              "      <td>0.0446</td>\n",
              "      <td>0.0551</td>\n",
              "      <td>0.0597</td>\n",
              "      <td>0.1416</td>\n",
              "      <td>0.0956</td>\n",
              "      <td>0.0802</td>\n",
              "      <td>0.1618</td>\n",
              "      <td>0.2558</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0118</td>\n",
              "      <td>0.0146</td>\n",
              "      <td>0.0040</td>\n",
              "      <td>0.0114</td>\n",
              "      <td>0.0032</td>\n",
              "      <td>0.0062</td>\n",
              "      <td>0.0101</td>\n",
              "      <td>0.0068</td>\n",
              "      <td>0.0053</td>\n",
              "      <td>0.0087</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19</th>\n",
              "      <td>0.0126</td>\n",
              "      <td>0.0149</td>\n",
              "      <td>0.0641</td>\n",
              "      <td>0.1732</td>\n",
              "      <td>0.2565</td>\n",
              "      <td>0.2559</td>\n",
              "      <td>0.2947</td>\n",
              "      <td>0.4110</td>\n",
              "      <td>0.4983</td>\n",
              "      <td>0.5920</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0153</td>\n",
              "      <td>0.0092</td>\n",
              "      <td>0.0035</td>\n",
              "      <td>0.0098</td>\n",
              "      <td>0.0121</td>\n",
              "      <td>0.0006</td>\n",
              "      <td>0.0181</td>\n",
              "      <td>0.0094</td>\n",
              "      <td>0.0116</td>\n",
              "      <td>0.0063</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 60 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "         0       1       2       3       4       5       6       7       8   \\\n",
              "67   0.0368  0.0403  0.0317  0.0293  0.0820  0.1342  0.1161  0.0663  0.0155   \n",
              "14   0.0124  0.0433  0.0604  0.0449  0.0597  0.0355  0.0531  0.0343  0.1052   \n",
              "164  0.0163  0.0198  0.0202  0.0386  0.0752  0.1444  0.1487  0.1484  0.2442   \n",
              "179  0.0394  0.0420  0.0446  0.0551  0.0597  0.1416  0.0956  0.0802  0.1618   \n",
              "19   0.0126  0.0149  0.0641  0.1732  0.2565  0.2559  0.2947  0.4110  0.4983   \n",
              "\n",
              "         9   ...      50      51      52      53      54      55      56  \\\n",
              "67   0.0506  ...  0.0058  0.0091  0.0160  0.0160  0.0081  0.0070  0.0135   \n",
              "14   0.2120  ...  0.0078  0.0083  0.0057  0.0174  0.0188  0.0054  0.0114   \n",
              "164  0.2822  ...  0.0027  0.0077  0.0026  0.0031  0.0083  0.0020  0.0084   \n",
              "179  0.2558  ...  0.0118  0.0146  0.0040  0.0114  0.0032  0.0062  0.0101   \n",
              "19   0.5920  ...  0.0153  0.0092  0.0035  0.0098  0.0121  0.0006  0.0181   \n",
              "\n",
              "         57      58      59  \n",
              "67   0.0067  0.0078  0.0068  \n",
              "14   0.0196  0.0147  0.0062  \n",
              "164  0.0108  0.0083  0.0033  \n",
              "179  0.0068  0.0053  0.0087  \n",
              "19   0.0094  0.0116  0.0063  \n",
              "\n",
              "[5 rows x 60 columns]"
            ]
          },
          "execution_count": 83,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "X_train.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o0MQYTsd6yH2"
      },
      "source": [
        "# Using Deep Learning Model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Zi0d8kTq6yH2"
      },
      "source": [
        "## Model without Dropout Layer"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "EGm3JVk-6yH2"
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf\n",
        "from tensorflow import keras"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "scrolled": true,
        "id": "sz8aG56p6yH2",
        "outputId": "1bbd0e70-5531-49cf-a6fb-c049d29834c4"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Epoch 1/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6895 - accuracy: 0.5321\n",
            "Epoch 2/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6669 - accuracy: 0.5833\n",
            "Epoch 3/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.6512 - accuracy: 0.6474\n",
            "Epoch 4/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6189 - accuracy: 0.6923\n",
            "Epoch 5/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.5780 - accuracy: 0.7564\n",
            "Epoch 6/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.5420 - accuracy: 0.8141\n",
            "Epoch 7/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.5194 - accuracy: 0.7756\n",
            "Epoch 8/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.4909 - accuracy: 0.7885\n",
            "Epoch 9/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.4450 - accuracy: 0.8013\n",
            "Epoch 10/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.4145 - accuracy: 0.8462\n",
            "Epoch 11/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.3802 - accuracy: 0.8333\n",
            "Epoch 12/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.3669 - accuracy: 0.8590\n",
            "Epoch 13/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.3465 - accuracy: 0.8654\n",
            "Epoch 14/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.3427 - accuracy: 0.8397\n",
            "Epoch 15/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.3336 - accuracy: 0.8654\n",
            "Epoch 16/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.3140 - accuracy: 0.8846\n",
            "Epoch 17/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.3288 - accuracy: 0.8397\n",
            "Epoch 18/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.2863 - accuracy: 0.9038\n",
            "Epoch 19/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.2882 - accuracy: 0.9038\n",
            "Epoch 20/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.2654 - accuracy: 0.9231\n",
            "Epoch 21/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.2573 - accuracy: 0.8974\n",
            "Epoch 22/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.2967 - accuracy: 0.8846\n",
            "Epoch 23/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.2742 - accuracy: 0.9038\n",
            "Epoch 24/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.2375 - accuracy: 0.9038\n",
            "Epoch 25/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.2559 - accuracy: 0.9103\n",
            "Epoch 26/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.2348 - accuracy: 0.9103\n",
            "Epoch 27/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.2922 - accuracy: 0.8718\n",
            "Epoch 28/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.2213 - accuracy: 0.9167\n",
            "Epoch 29/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.2142 - accuracy: 0.9103\n",
            "Epoch 30/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.1910 - accuracy: 0.9167\n",
            "Epoch 31/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.1929 - accuracy: 0.9103\n",
            "Epoch 32/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.2011 - accuracy: 0.9359\n",
            "Epoch 33/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.1888 - accuracy: 0.9103\n",
            "Epoch 34/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.1721 - accuracy: 0.9423\n",
            "Epoch 35/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.1699 - accuracy: 0.9423\n",
            "Epoch 36/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.1504 - accuracy: 0.9487\n",
            "Epoch 37/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.1447 - accuracy: 0.9551\n",
            "Epoch 38/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.1388 - accuracy: 0.9615\n",
            "Epoch 39/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.1512 - accuracy: 0.9423\n",
            "Epoch 40/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.1380 - accuracy: 0.9551\n",
            "Epoch 41/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.1287 - accuracy: 0.9615\n",
            "Epoch 42/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.1336 - accuracy: 0.9551\n",
            "Epoch 43/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.1299 - accuracy: 0.9551\n",
            "Epoch 44/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.1019 - accuracy: 0.9679\n",
            "Epoch 45/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.1005 - accuracy: 0.9872\n",
            "Epoch 46/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0891 - accuracy: 0.9744\n",
            "Epoch 47/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0912 - accuracy: 0.9744\n",
            "Epoch 48/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0860 - accuracy: 0.9744\n",
            "Epoch 49/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0740 - accuracy: 0.9872\n",
            "Epoch 50/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0779 - accuracy: 0.9808\n",
            "Epoch 51/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0629 - accuracy: 0.9936\n",
            "Epoch 52/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0605 - accuracy: 0.9872\n",
            "Epoch 53/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0540 - accuracy: 0.9872\n",
            "Epoch 54/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0493 - accuracy: 0.9936\n",
            "Epoch 55/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0537 - accuracy: 0.9936\n",
            "Epoch 56/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0506 - accuracy: 0.9872\n",
            "Epoch 57/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0512 - accuracy: 0.9936\n",
            "Epoch 58/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0435 - accuracy: 0.9936\n",
            "Epoch 59/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0396 - accuracy: 0.9936\n",
            "Epoch 60/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0395 - accuracy: 0.9936\n",
            "Epoch 61/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0355 - accuracy: 1.0000\n",
            "Epoch 62/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0268 - accuracy: 1.0000\n",
            "Epoch 63/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0310 - accuracy: 1.0000\n",
            "Epoch 64/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0255 - accuracy: 1.0000\n",
            "Epoch 65/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0297 - accuracy: 1.0000\n",
            "Epoch 66/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0263 - accuracy: 1.0000\n",
            "Epoch 67/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0224 - accuracy: 1.0000\n",
            "Epoch 68/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0198 - accuracy: 1.0000\n",
            "Epoch 69/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0169 - accuracy: 1.0000\n",
            "Epoch 70/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0149 - accuracy: 1.0000\n",
            "Epoch 71/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0146 - accuracy: 1.0000\n",
            "Epoch 72/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0164 - accuracy: 1.0000\n",
            "Epoch 73/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0128 - accuracy: 1.0000\n",
            "Epoch 74/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0130 - accuracy: 1.0000\n",
            "Epoch 75/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0205 - accuracy: 1.0000\n",
            "Epoch 76/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0210 - accuracy: 1.0000\n",
            "Epoch 77/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0109 - accuracy: 1.0000\n",
            "Epoch 78/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.0095 - accuracy: 1.0000\n",
            "Epoch 79/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0126 - accuracy: 1.0000\n",
            "Epoch 80/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0111 - accuracy: 1.0000\n",
            "Epoch 81/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0089 - accuracy: 1.0000\n",
            "Epoch 82/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0076 - accuracy: 1.0000\n",
            "Epoch 83/100\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0071 - accuracy: 1.0000\n",
            "Epoch 84/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0067 - accuracy: 1.0000\n",
            "Epoch 85/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0064 - accuracy: 1.0000\n",
            "Epoch 86/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0061 - accuracy: 1.0000\n",
            "Epoch 87/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0063 - accuracy: 1.0000\n",
            "Epoch 88/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0057 - accuracy: 1.0000\n",
            "Epoch 89/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0057 - accuracy: 1.0000\n",
            "Epoch 90/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0053 - accuracy: 1.0000\n",
            "Epoch 91/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0055 - accuracy: 1.0000\n",
            "Epoch 92/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0049 - accuracy: 1.0000\n",
            "Epoch 93/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0043 - accuracy: 1.0000\n",
            "Epoch 94/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0043 - accuracy: 1.0000\n",
            "Epoch 95/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0040 - accuracy: 1.0000\n",
            "Epoch 96/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0041 - accuracy: 1.0000\n",
            "Epoch 97/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0040 - accuracy: 1.0000\n",
            "Epoch 98/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0040 - accuracy: 1.0000\n",
            "Epoch 99/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0039 - accuracy: 1.0000\n",
            "Epoch 100/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.0039 - accuracy: 1.0000\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "<tensorflow.python.keras.callbacks.History at 0x21797a18070>"
            ]
          },
          "execution_count": 85,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "model = keras.Sequential([\n",
        "    keras.layers.Dense(60, input_dim=60, activation='relu'),\n",
        "    keras.layers.Dense(30, activation='relu'),\n",
        "    keras.layers.Dense(15, activation='relu'),\n",
        "    keras.layers.Dense(1, activation='sigmoid')\n",
        "])\n",
        "\n",
        "model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])\n",
        "\n",
        "model.fit(X_train, y_train, epochs=100, batch_size=8)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nXeIR09o6yH2",
        "outputId": "f48ee8b7-640f-438a-90b7-95070810b90a"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2/2 [==============================] - 0s 1ms/step - loss: 0.8629 - accuracy: 0.7692\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "[0.8629363775253296, 0.7692307829856873]"
            ]
          },
          "execution_count": 86,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "model.evaluate(X_test, y_test)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bedHdUoN6yH3"
      },
      "source": [
        "Training Accuracy >>> Test Accuracy"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cmKC-EZl6yH3",
        "outputId": "e01ce86d-d15c-41cf-d770-6ad7260a2105"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[7.1083555e-08 3.7778303e-01 9.9051148e-01 2.6755422e-05 9.9999976e-01\n",
            " 9.9999321e-01 6.4399661e-03 9.9999988e-01 9.1076899e-06 9.9999988e-01]\n",
            "[0. 0. 1. 0. 1. 1. 0. 1. 0. 1.]\n"
          ]
        }
      ],
      "source": [
        "y_pred = model.predict(X_test).reshape(-1)\n",
        "print(y_pred[:10])\n",
        "\n",
        "# round the values to nearest integer ie 0 or 1\n",
        "y_pred = np.round(y_pred)\n",
        "print(y_pred[:10])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "e0dnjnNO6yH3",
        "outputId": "2f971fe8-2905-4011-8cdf-d0e0b53505f2"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>R</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>186</th>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>155</th>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>165</th>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>200</th>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>58</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>34</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>151</th>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>18</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>202</th>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>62</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "     R\n",
              "186  0\n",
              "155  0\n",
              "165  0\n",
              "200  0\n",
              "58   1\n",
              "34   1\n",
              "151  0\n",
              "18   1\n",
              "202  0\n",
              "62   1"
            ]
          },
          "execution_count": 88,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "y_test[:10]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "P1m93MDF6yH3",
        "outputId": "63e7e301-87c5-45f8-c630-cceb61fab4b8"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.71      0.93      0.81        27\n",
            "           1       0.88      0.60      0.71        25\n",
            "\n",
            "    accuracy                           0.77        52\n",
            "   macro avg       0.80      0.76      0.76        52\n",
            "weighted avg       0.80      0.77      0.76        52\n",
            "\n"
          ]
        }
      ],
      "source": [
        "from sklearn.metrics import confusion_matrix , classification_report\n",
        "\n",
        "print(classification_report(y_test, y_pred))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_A6QMDLn6yH3"
      },
      "source": [
        "### Model with Dropout Layer for Overfitting issues"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "scrolled": true,
        "id": "76rOLD1H6yH3",
        "outputId": "3245caa1-c041-4536-8d62-2a42f84724c5"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Epoch 1/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.7653 - accuracy: 0.4744\n",
            "Epoch 2/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6923 - accuracy: 0.5256\n",
            "Epoch 3/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.7149 - accuracy: 0.4872\n",
            "Epoch 4/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.7439 - accuracy: 0.4423\n",
            "Epoch 5/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6974 - accuracy: 0.5064\n",
            "Epoch 6/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.7053 - accuracy: 0.5000\n",
            "Epoch 7/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.7035 - accuracy: 0.4808\n",
            "Epoch 8/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6994 - accuracy: 0.4679\n",
            "Epoch 9/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6984 - accuracy: 0.5385\n",
            "Epoch 10/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6833 - accuracy: 0.5000\n",
            "Epoch 11/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6910 - accuracy: 0.4679\n",
            "Epoch 12/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6803 - accuracy: 0.5641\n",
            "Epoch 13/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6887 - accuracy: 0.5385\n",
            "Epoch 14/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.6772 - accuracy: 0.5769\n",
            "Epoch 15/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6914 - accuracy: 0.5064\n",
            "Epoch 16/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.6757 - accuracy: 0.6026\n",
            "Epoch 17/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6685 - accuracy: 0.5962\n",
            "Epoch 18/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6687 - accuracy: 0.5897\n",
            "Epoch 19/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6872 - accuracy: 0.5192\n",
            "Epoch 20/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6616 - accuracy: 0.6346\n",
            "Epoch 21/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6522 - accuracy: 0.6346\n",
            "Epoch 22/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6605 - accuracy: 0.5833\n",
            "Epoch 23/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.6714 - accuracy: 0.5897\n",
            "Epoch 24/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6519 - accuracy: 0.6538\n",
            "Epoch 25/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6577 - accuracy: 0.6282\n",
            "Epoch 26/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6252 - accuracy: 0.6987\n",
            "Epoch 27/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6241 - accuracy: 0.6603\n",
            "Epoch 28/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6093 - accuracy: 0.6731\n",
            "Epoch 29/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6306 - accuracy: 0.6538\n",
            "Epoch 30/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.6008 - accuracy: 0.6923\n",
            "Epoch 31/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.5760 - accuracy: 0.7500\n",
            "Epoch 32/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6171 - accuracy: 0.6603\n",
            "Epoch 33/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.6315 - accuracy: 0.6603\n",
            "Epoch 34/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.5704 - accuracy: 0.6859\n",
            "Epoch 35/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.5848 - accuracy: 0.6859\n",
            "Epoch 36/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.5785 - accuracy: 0.6987\n",
            "Epoch 37/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.5493 - accuracy: 0.7372\n",
            "Epoch 38/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.5312 - accuracy: 0.7500\n",
            "Epoch 39/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.5531 - accuracy: 0.7308\n",
            "Epoch 40/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.5497 - accuracy: 0.7500\n",
            "Epoch 41/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.5843 - accuracy: 0.7372\n",
            "Epoch 42/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.5336 - accuracy: 0.7372\n",
            "Epoch 43/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.4994 - accuracy: 0.7628\n",
            "Epoch 44/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.5225 - accuracy: 0.7756\n",
            "Epoch 45/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.5242 - accuracy: 0.7436\n",
            "Epoch 46/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.4783 - accuracy: 0.7692\n",
            "Epoch 47/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.5040 - accuracy: 0.7564\n",
            "Epoch 48/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.5070 - accuracy: 0.7692\n",
            "Epoch 49/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.4904 - accuracy: 0.8013\n",
            "Epoch 50/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.4318 - accuracy: 0.8333\n",
            "Epoch 51/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.4261 - accuracy: 0.8013\n",
            "Epoch 52/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.5395 - accuracy: 0.7436\n",
            "Epoch 53/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.4932 - accuracy: 0.7564\n",
            "Epoch 54/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.4554 - accuracy: 0.7949\n",
            "Epoch 55/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.5198 - accuracy: 0.7500\n",
            "Epoch 56/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.4238 - accuracy: 0.8205\n",
            "Epoch 57/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.4558 - accuracy: 0.7821\n",
            "Epoch 58/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.4618 - accuracy: 0.7756\n",
            "Epoch 59/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.4335 - accuracy: 0.7949\n",
            "Epoch 60/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.4415 - accuracy: 0.8141\n",
            "Epoch 61/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.3873 - accuracy: 0.8333\n",
            "Epoch 62/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.4032 - accuracy: 0.8269\n",
            "Epoch 63/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.3989 - accuracy: 0.8141\n",
            "Epoch 64/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.4432 - accuracy: 0.8077\n",
            "Epoch 65/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.4252 - accuracy: 0.8205\n",
            "Epoch 66/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.4451 - accuracy: 0.8013\n",
            "Epoch 67/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.3673 - accuracy: 0.8590\n",
            "Epoch 68/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.3783 - accuracy: 0.8526\n",
            "Epoch 69/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.4337 - accuracy: 0.8269\n",
            "Epoch 70/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.3985 - accuracy: 0.8269\n",
            "Epoch 71/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.3747 - accuracy: 0.8526\n",
            "Epoch 72/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.3114 - accuracy: 0.8846\n",
            "Epoch 73/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.3631 - accuracy: 0.8718\n",
            "Epoch 74/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.3989 - accuracy: 0.8205\n",
            "Epoch 75/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.3730 - accuracy: 0.8269\n",
            "Epoch 76/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.3628 - accuracy: 0.8590\n",
            "Epoch 77/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.3613 - accuracy: 0.8397\n",
            "Epoch 78/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.4003 - accuracy: 0.8269\n",
            "Epoch 79/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.4395 - accuracy: 0.8141\n",
            "Epoch 80/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.3944 - accuracy: 0.8269\n",
            "Epoch 81/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.4066 - accuracy: 0.8205\n",
            "Epoch 82/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.3896 - accuracy: 0.8654\n",
            "Epoch 83/100\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "20/20 [==============================] - 0s 2ms/step - loss: 0.3426 - accuracy: 0.8590\n",
            "Epoch 84/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.3245 - accuracy: 0.9038\n",
            "Epoch 85/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.3396 - accuracy: 0.8526\n",
            "Epoch 86/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.3587 - accuracy: 0.8590\n",
            "Epoch 87/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.3705 - accuracy: 0.8205\n",
            "Epoch 88/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.2835 - accuracy: 0.8654\n",
            "Epoch 89/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.3712 - accuracy: 0.8013\n",
            "Epoch 90/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.3640 - accuracy: 0.8333\n",
            "Epoch 91/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.3402 - accuracy: 0.8974\n",
            "Epoch 92/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.3445 - accuracy: 0.8654\n",
            "Epoch 93/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.2922 - accuracy: 0.8910\n",
            "Epoch 94/100\n",
            "20/20 [==============================] - 0s 1ms/step - loss: 0.3192 - accuracy: 0.8782\n",
            "Epoch 95/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.2977 - accuracy: 0.8846\n",
            "Epoch 96/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.3482 - accuracy: 0.8269\n",
            "Epoch 97/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.3393 - accuracy: 0.8269\n",
            "Epoch 98/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.3268 - accuracy: 0.8526\n",
            "Epoch 99/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.2882 - accuracy: 0.8974\n",
            "Epoch 100/100\n",
            "20/20 [==============================] - 0s 2ms/step - loss: 0.2939 - accuracy: 0.9167\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "<tensorflow.python.keras.callbacks.History at 0x21799056c40>"
            ]
          },
          "execution_count": 90,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "modeld = keras.Sequential([\n",
        "    keras.layers.Dense(60, input_dim=60, activation='relu'),\n",
        "    keras.layers.Dropout(0.5), # Drop 50% of the neurons at Random\n",
        "    keras.layers.Dense(30, activation='relu'),\n",
        "    keras.layers.Dropout(0.5),\n",
        "    keras.layers.Dense(15, activation='relu'),\n",
        "    keras.layers.Dropout(0.5),\n",
        "    keras.layers.Dense(1, activation='sigmoid')\n",
        "])\n",
        "\n",
        "modeld.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])\n",
        "\n",
        "modeld.fit(X_train, y_train, epochs=100, batch_size=8)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "MvwMV9cu6yH3",
        "outputId": "c752e5ed-9a9d-4804-c28f-7b9e13d877be"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2/2 [==============================] - 0s 1ms/step - loss: 0.3962 - accuracy: 0.8077\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "[0.39620405435562134, 0.807692289352417]"
            ]
          },
          "execution_count": 91,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "modeld.evaluate(X_test, y_test)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v2PsDyuM6yH4"
      },
      "source": [
        "Training Accuracy is still good but Test Accuracy Improved"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "D6e9PeT06yH4",
        "outputId": "c72d92f9-3096-45ce-c376-b3a000ad7808"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[0.00204679 0.855612   0.87832576 0.02817929 0.99947876 0.92398506\n",
            " 0.24040593 0.99971884 0.04001041 0.99982685]\n",
            "[0. 1. 1. 0. 1. 1. 0. 1. 0. 1.]\n"
          ]
        }
      ],
      "source": [
        "y_pred = modeld.predict(X_test).reshape(-1)\n",
        "print(y_pred[:10])\n",
        "\n",
        "# round the values to nearest integer ie 0 or 1\n",
        "y_pred = np.round(y_pred)\n",
        "print(y_pred[:10])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "scrolled": true,
        "id": "BcQAGi8l6yH4",
        "outputId": "e5e16ff0-2c71-4767-ecd0-e22c2add0cdd"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.76      0.93      0.83        27\n",
            "           1       0.89      0.68      0.77        25\n",
            "\n",
            "    accuracy                           0.81        52\n",
            "   macro avg       0.83      0.80      0.80        52\n",
            "weighted avg       0.82      0.81      0.80        52\n",
            "\n"
          ]
        }
      ],
      "source": [
        "from sklearn.metrics import confusion_matrix , classification_report\n",
        "\n",
        "print(classification_report(y_test, y_pred))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "smZx4t5P6yH4"
      },
      "source": [
        "**You can see that by using dropout layer test accuracy increased from 0.77 to 0.81**"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.5"
    },
    "colab": {
      "provenance": []
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}