machine-learning/classification/naïve-bayes-classification-implementation-using-scikit-learn.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Naïve Bayes Classification Implementation Using Scikit-Learn",
      "provenance": [],
      "authorship_tag": "ABX9TyOeBUoE5Sogq+CMA7thTymy"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dZe5Fp3fzscj"
      },
      "source": [
        "# **Naïve Bayes Classification Implementation Using Scikit-Learn**\n",
        "\n",
        "*By Carlos Santiago Bañón*\n",
        "\n",
        "* **Year:** 2020\n",
        "* **Technologies:** Python, Scikit-Learn, Pandas, NumPy\n",
        "* **Areas**: Machine Learning, Classification, Bayesian Learning\n",
        "* **Keywords:** `bayesian-learning`, `classification`, `machine-learning`, `naïve-bayes`, `naïve-bayes-classification`\n",
        "* **Description:** This notebook presents an implementation of naïve Bayes classification using the Scikit-Learn library. The data used is a preprocessed version of the Kaggle Titanic dataset hosted in the GitHub repository for this notebook."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UX94UXm8z7Nv"
      },
      "source": [
        "## 1. Import Statements\n",
        "---"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dnrBRGX3z9mQ"
      },
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "from sklearn.metrics import f1_score\n",
        "from sklearn.metrics import recall_score\n",
        "from sklearn.metrics import precision_score\n",
        "from sklearn.metrics import accuracy_score\n",
        "from sklearn.model_selection import cross_val_score\n",
        "from sklearn.naive_bayes import GaussianNB"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8wyC0ZRn0LHm"
      },
      "source": [
        "## 2. Load the Data\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LWxu2qDP2fsE"
      },
      "source": [
        "First, we import the preprocessed Kaggle Titanic dataset hosted in the GitHub repository for this notebook."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mM6bKcvH0Nva"
      },
      "source": [
        "# Import the data into Pandas DataFrames.\n",
        "train_df = pd.read_csv('https://bit.ly/39AQRJj')\n",
        "test_df = pd.read_csv('https://bit.ly/3aoJzHG')\n",
        "y_test_df = pd.read_csv('https://bit.ly/2YxfKzi')"
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 419
        },
        "id": "jZhgO8OJ0a_E",
        "outputId": "3d3867d6-b292-4e64-9b80-714471dea8bf"
      },
      "source": [
        "# Show the training set.\n",
        "train_df"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Age</th>\n",
              "      <th>Survived</th>\n",
              "      <th>Pclass</th>\n",
              "      <th>SibSp</th>\n",
              "      <th>Fare</th>\n",
              "      <th>Gender</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>886</th>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>887</th>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>888</th>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>889</th>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>890</th>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>891 rows × 6 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "     Age  Survived  Pclass  SibSp  Fare  Gender\n",
              "0      2         0       3      1     0       0\n",
              "1      3         1       1      1     3       1\n",
              "2      2         1       3      0     1       1\n",
              "3      3         1       1      1     3       1\n",
              "4      3         0       3      0     1       0\n",
              "..   ...       ...     ...    ...   ...     ...\n",
              "886    2         0       2      0     1       0\n",
              "887    2         1       1      0     2       1\n",
              "888    2         0       3      1     2       1\n",
              "889    2         1       1      0     2       0\n",
              "890    3         0       3      0     0       0\n",
              "\n",
              "[891 rows x 6 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 419
        },
        "id": "Hg4hhA3F0c4b",
        "outputId": "8f1b4491-0bb5-4620-f2c4-02fa3bccc968"
      },
      "source": [
        "# Show the test set.\n",
        "test_df"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Age</th>\n",
              "      <th>Pclass</th>\n",
              "      <th>SibSp</th>\n",
              "      <th>Fare</th>\n",
              "      <th>Gender</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>3</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>4</td>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>5</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2</td>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>413</th>\n",
              "      <td>3</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>414</th>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>415</th>\n",
              "      <td>3</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>416</th>\n",
              "      <td>3</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>417</th>\n",
              "      <td>3</td>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>418 rows × 5 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "     Age  Pclass  SibSp  Fare  Gender\n",
              "0      3       3      0     0       0\n",
              "1      4       3      1     0       1\n",
              "2      5       2      0     1       0\n",
              "3      2       3      0     1       0\n",
              "4      2       3      1     1       1\n",
              "..   ...     ...    ...   ...     ...\n",
              "413    3       3      0     1       0\n",
              "414    3       1      0     3       1\n",
              "415    3       3      0     0       0\n",
              "416    3       3      0     1       0\n",
              "417    3       3      1     2       0\n",
              "\n",
              "[418 rows x 5 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ekK9B6Yd0khJ"
      },
      "source": [
        "# Set up the learning matrices.\n",
        "X_train = train_df.drop('Survived', axis=1, inplace=False).to_numpy()\n",
        "y_train = train_df[['Survived']].to_numpy()\n",
        "X_test = test_df.to_numpy()\n",
        "y_test = y_test_df.drop('PassengerId', axis=1, inplace=False).to_numpy()"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "E7P0ZAo60rXW"
      },
      "source": [
        "## 3. Naïve Bayes Classification\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "18K0JkTx0upe"
      },
      "source": [
        "### 3.1. Define and Fit the Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2xAX9ucx0_Cr"
      },
      "source": [
        "# Define the Gaussian naïve Bayes classifier.\n",
        "gnb = GaussianNB()"
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "B0DSl9dU1CCl"
      },
      "source": [
        "# Fit the Gaussian naïve Bayes classifier.\n",
        "y_pred_gnb = gnb.fit(X_train, y_train.ravel()).predict(test_df.to_numpy())"
      ],
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eJi74y7y04i_"
      },
      "source": [
        "### 3.2. Get the Evaluation Metrics Using Cross-Validation"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a4BLRMhU1IES"
      },
      "source": [
        "# Calculate the evaluation metrics using cross-validation.\n",
        "f1_score = cross_val_score(gnb, X_train, y_train.ravel(), cv=5, scoring='f1').mean()\n",
        "accuracy_score = cross_val_score(gnb, X_train, y_train.ravel(), cv=5, scoring='accuracy').mean()\n",
        "precision_score = cross_val_score(gnb, X_train, y_train.ravel(), cv=5, scoring='precision').mean()\n",
        "recall_score = cross_val_score(gnb, X_train, y_train.ravel(), cv=5, scoring='recall').mean()"
      ],
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QWLZ44op6Vph",
        "outputId": "7c6ed542-1139-46c8-b18b-b60f39ed7da7"
      },
      "source": [
        "# Show the evaluation metrics.\n",
        "print(\"F1 Score:\", f1_score)\n",
        "print(\"Accuracy Score:\", accuracy_score)\n",
        "print(\"Precision Score:\", precision_score)\n",
        "print(\"Recall Score:\", recall_score)"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "F1 Score: 0.7282419532419533\n",
            "Accuracy Score: 0.7710752620676669\n",
            "Precision Score: 0.6705436720142602\n",
            "Recall Score: 0.7982949701619779\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1RcBpok-6WUE"
      },
      "source": [
        "### 3.3. Show the Prediction Results"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "z4l2sHPQ6Yu4",
        "outputId": "9a46de35-6bee-429e-e66e-955c90e517fe"
      },
      "source": [
        "# Show the prediction results.\n",
        "print(y_pred_gnb)"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[0 1 0 0 1 0 1 0 1 0 0 1 1 0 1 1 0 0 1 1 1 0 1 1 1 0 1 0 1 0 0 0 1 1 1 0 1\n",
            " 1 0 0 0 1 0 1 1 0 1 0 1 1 1 0 1 1 0 0 0 0 0 1 0 0 0 1 0 1 1 1 1 1 1 0 1 1\n",
            " 1 1 0 1 0 1 0 1 1 0 0 0 1 1 1 0 1 0 1 0 1 0 1 0 1 0 1 0 0 0 1 0 0 0 0 0 0\n",
            " 1 1 1 1 0 0 1 1 1 1 0 1 0 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 0 0 1 1 0 1 0 1 0\n",
            " 1 0 1 0 0 1 0 0 1 1 1 1 1 0 1 0 0 1 1 0 1 1 0 0 0 0 0 1 1 1 1 1 0 1 1 0 1\n",
            " 0 1 0 0 0 0 1 0 0 0 0 1 1 0 1 1 0 1 1 0 1 1 0 1 0 0 0 0 1 0 1 1 1 1 0 1 0\n",
            " 1 0 1 1 0 1 0 0 0 1 0 0 1 0 1 0 1 1 1 1 1 0 0 1 1 0 1 1 1 0 1 0 0 0 0 0 1\n",
            " 0 0 0 1 1 0 0 0 0 1 0 1 0 1 1 0 1 0 0 0 0 1 0 1 1 1 0 0 1 0 0 1 1 0 1 0 0\n",
            " 1 0 1 0 0 0 0 0 1 1 1 0 1 1 0 0 0 1 1 1 1 0 0 0 0 0 0 1 1 0 1 1 0 0 1 1 0\n",
            " 1 0 1 0 0 0 0 0 0 0 1 0 1 0 1 0 1 1 0 0 0 1 1 1 0 0 1 0 1 1 0 1 1 0 1 1 0\n",
            " 0 1 0 0 1 1 0 0 1 0 0 0 1 1 0 1 0 0 0 0 1 1 0 0 0 1 0 1 0 0 1 0 1 1 1 0 0\n",
            " 1 1 1 1 1 1 0 1 0 0 0]\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}