CSI4506-AMSTitanic_Automne20.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Notebook 5 - Prédiction sur le Titanic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "CSI4506 Intelligence Artificielle  \n",
    "Automne 2020  \n",
    "Preparé par Joel Muteba, Julian Templeton et Caroline Barrière"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***INTRODUCTION***:\n",
    "\n",
    "La tâche de classification supervisée abordée dans ce notebook est de déterminer si un passager a survécu ou non sur le Titanic. Il s'agit d'un problème d'introduction courant à l'apprentissage machine supervisé et constitue un défi sur un site populaire appelé Kaggle. Kaggle contient des compétitions auxquelles les utilisateurs peuvent participer. Ces compétitions fournissent généralement un ensemble de données avec lequel travailler et une description du problème à résoudre. Cela dit, tous les ensembles de données n'incluront pas la valeur à prédire pour l'ensemble de test. Ou bien, un concours peut uniquement fournir un ensemble d'entraînement et de validation, pas l'ensemble de test. Cela signifie que les utilisateurs ne sauront pas comment leurs modèles fonctionnent sur les données réelles invisibles jusqu'à ce que le ou les hôtes testent la soumission de l'utilisateur sur l'ensemble de test privé et publient les résultats.\n",
    "\n",
    "Ce notebook enrichira vos connaissances du notebook précédent et présentera le concept de gestion du déséquilibre de classe grâce à l'utilisation d'une technique appelée suréchantillonnage. Vous verrez également comment un ensemble de données peut avoir des problèmes avec ses données et que tous les attributs ne doivent pas être utilisées pour entraîner et tester un modèle. Vous effectuerez également l'entraînement, les tests et l'évaluation de manière légèrement nouvelle puisque nous travaillerons avec un type différent d'ensemble de données.\n",
    "\n",
    "Encore une fois, ce notebook utilise **scikit-learn** (http://scikit-learn.org/stable/), **Matplot**, **Numpy** et **Pandas**. Cependant, nous utiliserons également **imblearn** qui fournit des techniques robustes pour équilibrer le nombre d'instances de différentes classes. Pour installer ce package, utilisez la commande ***pip install imbalanced-learn***. Si cela pose des problèmes, vous pouvez également essayer *pip install imblearn*.\n",
    "\n",
    "Dans ce notebook, nous utiliserons l'algorithme Naive Bayes et l'algorithme de régression logistique pour effectuer la classification de ce problème. Après avoir évalué les deux modèles, vous déterminerez celui qui a le mieux fonctionné et pourquoi. Enfin, vous recommencerez le processus pour déterminer si une technique de suréchantillonnage commune améliorera ou non les résultats obtenus."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***Devoir***:\n",
    "\n",
    "Parcourez le notebook en exécutant chaque cellule, une à la fois.\n",
    "Recherchez **(TO DO)** pour les tâches que vous devez effectuer. Ne modifiez pas le code en dehors des questions auxquelles vous êtes invité à répondre à moins que cela ne vous soit spécifiquement demandé. Une fois que vous avez terminé, signez le notebook (à la fin du notebook) et soumettez-le.\n",
    "\n",
    "*Le notebook sera noté le 30.\n",
    "Chaque **(TO DO)** a un certain nombre de points qui lui sont associés.*\n",
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**1. Exploration de l'ensemble de données**   \n",
    "\n",
    "Tout d'abord, nous allons configurer les données avec lesquelles nous travaillerons. Ces données sont incluses avec le notebook et peuvent être trouvées auprès de Kaggle, qui contient également une description des données avec lesquelles nous travaillerons (https://www.kaggle.com/c/titanic/data). Il est fortement recommandé de jeter un coup d'œil rapide au dictionnaire de données dans la section Overview qui est disponible via le lien ci-dessus. Cela vous aidera à comprendre les données avec lesquelles nous travaillerons.\n",
    "\n",
    "Comme évoqué dans l'introduction, ce concours Kaggle ne fournit que des données d'entraînement annotées, tandis que les données de test ne sont pas annotées. Les valeurs de sorties (prédites) de l'ensemble de données de test peuvent-être trouvées en ligne, mais nous nous concentrerons uniquement sur l'utilisation de l'ensemble de données d'entraînement pour créer et tester notre modèle. Cela vous fournira un type de processus d'expérimentation d'apprentissage machine différent.\n",
    "\n",
    "Il existe de nombreux cas où une compétition ne donnera qu'un ensemble d'entraînement, nous essaierons donc de savoir si un passager a survécu ou non sur le Titanic à partir de ces données d'entraînement."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import the required packages for data analysis and machine learning\n",
    "from imblearn.over_sampling import SMOTE\n",
    "from sklearn.naive_bayes import MultinomialNB\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.model_selection import train_test_split\n",
    "import datetime\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Tout d'abord, nous allons charger l'ensemble de données d'entraînement (*train.csv*) dans un dataframe avec Pandas et explorer les dix premiers échantillons. Notez que cet ensemble de données contient une variété de colonnes (features) potentielles, contrairement aux critiques de films avec lesquelles nous avons travaillé dans le dernier notebook qui contenait des critiques de films que nous devions transformer. Cet ensemble de données contient à la fois des valeurs continues et discrètes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Moran, Mr. James</td>\n",
       "      <td>male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>330877</td>\n",
       "      <td>8.4583</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>McCarthy, Mr. Timothy J</td>\n",
       "      <td>male</td>\n",
       "      <td>54.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>17463</td>\n",
       "      <td>51.8625</td>\n",
       "      <td>E46</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Palsson, Master. Gosta Leonard</td>\n",
       "      <td>male</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>349909</td>\n",
       "      <td>21.0750</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>9</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Johnson, Mrs. Oscar W (Elisabeth Vilhelmina Berg)</td>\n",
       "      <td>female</td>\n",
       "      <td>27.0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>347742</td>\n",
       "      <td>11.1333</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Nasser, Mrs. Nicholas (Adele Achem)</td>\n",
       "      <td>female</td>\n",
       "      <td>14.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>237736</td>\n",
       "      <td>30.0708</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "2            3         1       3   \n",
       "3            4         1       1   \n",
       "4            5         0       3   \n",
       "5            6         0       3   \n",
       "6            7         0       1   \n",
       "7            8         0       3   \n",
       "8            9         1       3   \n",
       "9           10         1       2   \n",
       "\n",
       "                                                Name     Sex   Age  SibSp  \\\n",
       "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
       "2                             Heikkinen, Miss. Laina  female  26.0      0   \n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0      1   \n",
       "4                           Allen, Mr. William Henry    male  35.0      0   \n",
       "5                                   Moran, Mr. James    male   NaN      0   \n",
       "6                            McCarthy, Mr. Timothy J    male  54.0      0   \n",
       "7                     Palsson, Master. Gosta Leonard    male   2.0      3   \n",
       "8  Johnson, Mrs. Oscar W (Elisabeth Vilhelmina Berg)  female  27.0      0   \n",
       "9                Nasser, Mrs. Nicholas (Adele Achem)  female  14.0      1   \n",
       "\n",
       "   Parch            Ticket     Fare Cabin Embarked  \n",
       "0      0         A/5 21171   7.2500   NaN        S  \n",
       "1      0          PC 17599  71.2833   C85        C  \n",
       "2      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3      0            113803  53.1000  C123        S  \n",
       "4      0            373450   8.0500   NaN        S  \n",
       "5      0            330877   8.4583   NaN        Q  \n",
       "6      0             17463  51.8625   E46        S  \n",
       "7      1            349909  21.0750   NaN        S  \n",
       "8      2            347742  11.1333   NaN        S  \n",
       "9      0            237736  30.0708   NaN        C  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Read the dataset, show top ten rows\n",
    "X = pd.read_csv(\"train.csv\")\n",
    "X.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "De ce qui précède, nous pouvons voir une variété de colonnes. Sur ces colonnes, la colonne ***Survived*** représente les valeurs de classe que nous souhaitons prédire. Un passager a survécu du Titanic si *Survived* vaut 1 et n'a pas survécu du Titanic si *Survived* est 0. Ce sont nos deux classes que nous prédirons (1 et 0). Les autres colonnes représentent des attributs potentiels qui peuvent être utilisées par les algorithmes d'apprentissage machine pour apprendre à prédire avec précision la classe cible.\n",
    "\n",
    "Une remarque importante des données ci-dessus est que tous les échantillons ne contiennent pas de données pour chaque colonne. Par exemple, le passager 6 (Moran, M. James) ne contient pas de valeur pour son *âge* et ne contient pas la *cabine* dans laquelle il résidait. Étant donné que les algorithmes d'apprentissage machine apprennent à partir de données, un Data Scientist devra apprendre à remplir avec précision les données manquantes. Les méthodes robustes pour ce faire dépassent le cadre de ce notebook, mais nous remplirons plus tard la plupart des données manquantes par des moyens simples.\n",
    "\n",
    "Une autre remarque est qu'il est important de déterminer si une classe est vue plus fréquemment qu'une autre classe dans les données. Une classe avec le plus d'instances est appelée la classe *majoritaire* et une classe avec le moins d'instances est appelée la classe *minoritaire*. Si une classe contient plus d'instances qu'une autre classe, l'algorithme peut se concentrer sur l'apprentissage de cette classe majoritaire plus que de la classe minoritaire. Dans des problèmes tels que le diagnostic du cancer, il s'agit d'un problème majeur (car beaucoup plus de personnes n'ont pas de cancer). Ainsi, un autre problème que nous explorerons dans ce notebook est le concept d'utilisation du suréchantillonnage pour équilibrer la distribution des classes. Ce concept sera expliqué en détail plus loin dans le notebook.  \n",
    "\n",
    "Vous trouverez ci-dessous un graphique du nombre d'instances pour chaque classe. Dans ce scénario, sur base des données disponibles de l'ensemble d'entraînement, plus de personnes n'ont pas survécu sur le Titanic que celles qui ont survécu (comme on pourrait s'y attendre dans ce scénario). Puisque nous ne pouvons pas simplement collecter plus de données sur les passagers qui ont survécu au Titanic, nous devrons réfléchir à des moyens d'équilibrer la répartition des classes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEFCAYAAAAYKqc0AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAARY0lEQVR4nO3de5CkVX3G8e8TEI3XBRlX2ItLijUWJlGpVTFaKRWjgsqSilpegivBbExplcYrWjFqaRSTVDQmKS0ihgXlJl5YkRg34CXeWRRRxMuEAtkFYZWLIGoEfvmjz0ozzNA9OzM7u4fvp6prznve0+f9vb2zT799pnsmVYUkqS+/tdgFSJLmn+EuSR0y3CWpQ4a7JHXIcJekDhnuktQhw127nCTvT/KmBZj3LUk+NMc5bkryO/NUzxuTfKC1VyWpJHvO09wrW617zMd82v0Y7hpLkick+XKSG5Jcm+RLSR69EMeqqpdW1dsWYu6ZJHlikttaIN6UZEuSM6aeY1Xdt6ouHWOuLaOOWVXvqKqXzLX2dszLkjxlaO4ftVpvnY/5tfsx3DVSkvsDZwP/AuwDLAPeCvxqB+ZKkl31++7KqrovcD/gEOB7wP8kOXS+DzRfV+jSTHbV/2TatTwUoKpOrapbq+oXVfWZqroI7rzcMXWJIcnnkvxdki8BNwOvTbJ5+ABJ/jrJxtY+McnbW/uSJM8cGrdnkm1JDm7bh7RXFNcn+VaSJw6NPSDJ55PcmGQTsO84J1sDW6rqb4EPAO8amrOSHNjahyf5bpt/a5LXJLkP8J/A/kOvAvZvj9GZST6U5GfAi2dYJvrzJFcmuSrJa4aO+5vHpG3/5tVBkpOBlcAn2/FeN82/wf5JNrZXXZNJ/mJorre0VykntXO5OMmacR4r7boMd43jB8CtSTYkOSzJ3jswx1HAegZXxe8HfjfJ6qH9LwBOmeZ+pwLPH9p+GvCTqvpGkmXAp4C3M3hF8Rrgo0km2thTgAsYhPrbgHU7UPfHgINbaE91AvCXVXU/4PeA86rq58BhtFcB7XZlG78WOBNYAnx4huM9CVgNPBV4/fBSy0yq6ijgR8Cz2vH+fpphpwFbgP2BZwPvSPLkof1HtDFLgI3Av446rnZthrtGqqqfAU8ACvh3YFu7Clw6i2lOrKqLq+qWqroBOIsW2i3kH8YgVKY6BTgiyb3b9gsYBD7AnwHnVNU5VXVbVW0CNgOHJ1kJPBp4U1X9qqq+AHxyNufdXAmEQehN9WvgoCT3r6rrquobI+b6SlV9otX6ixnGvLWqfl5V3wb+gzs+se2QJCuAxwOvr6pfVtWFDF6RvGho2Bfb43grcDLwiLkeV4vLcNdYquqSqnpxVS1ncJW6P/CeWUxxxZTtU7g9uF4AfKKqbp7muJPAJcCzWsAfwe1X+A8BntOWZK5Pcj2DJ6H9Wn3XtSvp7S6fRb3bLWPwpHb9NPv+FDgcuLwt/zxuxFxTH4NRYy5ncB5ztT9wbVXdOGXuZUPbPx5q3wzcy58L7N4Md81aVX0POJFByAP8HLj30JAHT3e3KdubgIkkj2QQ8tMtyWy3fWlmLfDdFvgwCMKTq2rJ0O0+VXUccBWw95TllJUjT+7O/gT4xpQnicEJVZ1fVWuBBwGfAM7YvmuGucb5FawrhtorGbxygNGP8V3NfSWwT5L7TZl76xj1aDdluGukJA9L8uoky9v2CgZh+9U25ELgjzJ4b/UDgDeMmrOqfg18BPgHBuvlm+5i+GkM1qD/ijs+CXyIwRX905LskeRe7QeNy6vqcgZLNG9NsleSJwDPGvN8k2RZkjcDLwHeOM2YvZK8MMkD2rn8DLit7b4aeGB7LGbrTUnuneThwNHA6a3/QgbLTfskeTDwyin3uxqY9v33VXUF8GXgne0x+gPgGAaPnzpluGscNwKPBb6W5OcMQv07wKsB2lr36cBFDH6AefaY854CPAX4SFXdMtOgqroK+Arwh9wedttDay2D8N3G4Er+tdz+ff2CVve1wJuBk0bUs3+Sm4CbgPOB3weeWFWfmWH8UcBl7d0vLwVe2Or6HoNXG5e25aLZLK18HpgEzgX+cejYJwPfAi4DPsPQ49C8E/ibdrzXcGfPB1YxuIr/OPDmqvrvWdSl3Uz8Yx2S1B+v3CWpQ4a7JHXIcJekDhnuktQhw12SOrRLfAJt3333rVWrVi12GZK0W7ngggt+UlUT0+3bJcJ91apVbN68efRASdJvJJnxV2q4LCNJHTLcJalDhrskdchwl6QOGe6S1CHDXZI6ZLhLUocMd0nq0C7xIabdxapjP7XYJXTlsuOesdglSN3yyl2SOmS4S1KHDHdJ6pDhLkkdMtwlqUOGuyR1yHCXpA4Z7pLUIcNdkjpkuEtShwx3SeqQ4S5JHTLcJalDhrskdWiscE9yWZJvJ7kwyebWt0+STUl+2L7u3fqT5L1JJpNclOTghTwBSdKdzebK/UlV9ciqWtO2jwXOrarVwLltG+AwYHW7rQfeN1/FSpLGM5dlmbXAhtbeABw51H9SDXwVWJJkvzkcR5I0S+OGewGfSXJBkvWtb2lVXdXaPwaWtvYy4Iqh+25pfZKknWTcP7P3hKramuRBwKYk3xveWVWVpGZz4PYksR5g5cqVs7mrJGmEsa7cq2pr+3oN8HHgMcDV25db2tdr2vCtwIqhuy9vfVPnPL6q1lTVmomJiR0/A0nSnYwM9yT3SXK/7W3gqcB3gI3AujZsHXBWa28EXtTeNXMIcMPQ8o0kaScYZ1lmKfDxJNvHn1JVn05yPnBGkmOAy4HntvHnAIcDk8DNwNHzXrUk6S6NDPequhR4xDT9PwUOnaa/gJfNS3WSpB3iJ1QlqUOGuyR1yHCXpA4Z7pLUIcNdkjpkuEtShwx3SeqQ4S5JHTLcJalDhrskdchwl6QOGe6S1CHDXZI6ZLhLUocMd0nqkOEuSR0y3CWpQ4a7JHXIcJekDhnuktQhw12SOmS4S1KHDHdJ6pDhLkkdMtwlqUOGuyR1yHCXpA4Z7pLUIcNdkjo0drgn2SPJN5Oc3bYPSPK1JJNJTk+yV+u/Z9uebPtXLVDtkqQZzObK/RXAJUPb7wLeXVUHAtcBx7T+Y4DrWv+72zhJ0k40VrgnWQ48A/hA2w7wZODMNmQDcGRrr23btP2HtvGSpJ1k3Cv39wCvA25r2w8Erq+qW9r2FmBZay8DrgBo+29o4yVJO8nIcE/yTOCaqrpgPg+cZH2SzUk2b9u2bT6nlqS7vXGu3B8PHJHkMuA0Bssx/wwsSbJnG7Mc2NraW4EVAG3/A4CfTp20qo6vqjVVtWZiYmJOJyFJuqOR4V5Vb6iq5VW1CngecF5VvRD4LPDsNmwdcFZrb2zbtP3nVVXNa9WSpLs0l/e5vx54VZJJBmvqJ7T+E4AHtv5XAcfOrURJ0mztOXrI7arqc8DnWvtS4DHTjPkl8Jx5qE2StIP8hKokdchwl6QOGe6S1CHDXZI6ZLhLUocMd0nqkOEuSR0y3CWpQ4a7JHXIcJekDhnuktQhw12SOmS4S1KHDHdJ6pDhLkkdMtwlqUOz+mMdknZNq4791GKX0JXLjnvGYpcwZ165S1KHDHdJ6pDhLkkdMtwlqUOGuyR1yHCXpA4Z7pLUIcNdkjpkuEtShwx3SeqQ4S5JHRoZ7knuleTrSb6V5OIkb239ByT5WpLJJKcn2av137NtT7b9qxb4HCRJU4xz5f4r4MlV9QjgkcDTkxwCvAt4d1UdCFwHHNPGHwNc1/rf3cZJknaikeFeAze1zXu0WwFPBs5s/RuAI1t7bdum7T80SearYEnSaGOtuSfZI8mFwDXAJuB/geur6pY2ZAuwrLWXAVcAtP03AA+cx5olSSOMFe5VdWtVPRJYDjwGeNhcD5xkfZLNSTZv27ZtrtNJkobM6t0yVXU98FngccCSJNv/2MdyYGtrbwVWALT9DwB+Os1cx1fVmqpaMzExsWPVS5KmNc67ZSaSLGnt3wb+GLiEQcg/uw1bB5zV2hvbNm3/eVVV81izJGmEcf7M3n7AhiR7MHgyOKOqzk7yXeC0JG8Hvgmc0MafAJycZBK4FnjeAtQtSboLI8O9qi4CHjVN/6UM1t+n9v8SeM68VCdJ2iF+QlWSOmS4S1KHDHdJ6pDhLkkdMtwlqUOGuyR1yHCXpA4Z7pLUIcNdkjpkuEtShwx3SeqQ4S5JHTLcJalDhrskdchwl6QOGe6S1CHDXZI6ZLhLUocMd0nqkOEuSR0y3CWpQ4a7JHXIcJekDhnuktQhw12SOmS4S1KHDHdJ6pDhLkkdMtwlqUMjwz3JiiSfTfLdJBcneUXr3yfJpiQ/bF/3bv1J8t4kk0kuSnLwQp+EJOmOxrlyvwV4dVUdBBwCvCzJQcCxwLlVtRo4t20DHAasbrf1wPvmvWpJ0l0aGe5VdVVVfaO1bwQuAZYBa4ENbdgG4MjWXgucVANfBZYk2W++C5ckzWxWa+5JVgGPAr4GLK2qq9quHwNLW3sZcMXQ3ba0PknSTjJ2uCe5L/BR4JVV9bPhfVVVQM3mwEnWJ9mcZPO2bdtmc1dJ0ghjhXuSezAI9g9X1cda99Xbl1va12ta/1ZgxdDdl7e+O6iq46tqTVWtmZiY2NH6JUnTGOfdMgFOAC6pqn8a2rURWNfa64Czhvpf1N41cwhww9DyjSRpJ9hzjDGPB44Cvp3kwtb3RuA44IwkxwCXA89t+84BDgcmgZuBo+ezYEnSaCPDvaq+CGSG3YdOM76Al82xLknSHPgJVUnqkOEuSR0y3CWpQ4a7JHXIcJekDhnuktQhw12SOmS4S1KHDHdJ6pDhLkkdMtwlqUOGuyR1yHCXpA4Z7pLUIcNdkjpkuEtShwx3SeqQ4S5JHTLcJalDhrskdchwl6QOGe6S1CHDXZI6ZLhLUocMd0nqkOEuSR0y3CWpQ4a7JHXIcJekDo0M9yQfTHJNku8M9e2TZFOSH7ave7f+JHlvkskkFyU5eCGLlyRNb5wr9xOBp0/pOxY4t6pWA+e2bYDDgNXtth543/yUKUmajZHhXlVfAK6d0r0W2NDaG4Ajh/pPqoGvAkuS7DdPtUqSxrSja+5Lq+qq1v4xsLS1lwFXDI3b0vokSTvRnH+gWlUF1Gzvl2R9ks1JNm/btm2uZUiShuxouF+9fbmlfb2m9W8FVgyNW9767qSqjq+qNVW1ZmJiYgfLkCRNZ0fDfSOwrrXXAWcN9b+ovWvmEOCGoeUbSdJOsueoAUlOBZ4I7JtkC/Bm4DjgjCTHAJcDz23DzwEOByaBm4GjF6BmSdIII8O9qp4/w65DpxlbwMvmWpQkaW78hKokdchwl6QOGe6S1CHDXZI6ZLhLUocMd0nqkOEuSR0y3CWpQ4a7JHXIcJekDhnuktQhw12SOmS4S1KHDHdJ6pDhLkkdMtwlqUOGuyR1yHCXpA4Z7pLUIcNdkjpkuEtShwx3SeqQ4S5JHTLcJalDhrskdchwl6QOGe6S1CHDXZI6ZLhLUocWJNyTPD3J95NMJjl2IY4hSZrZvId7kj2AfwMOAw4Cnp/koPk+jiRpZgtx5f4YYLKqLq2q/wNOA9YuwHEkSTPYcwHmXAZcMbS9BXjs1EFJ1gPr2+ZNSb6/ALXcXe0L/GSxixgl71rsCrQI/N6cXw+ZacdChPtYqup44PjFOn7PkmyuqjWLXYc0ld+bO89CLMtsBVYMbS9vfZKknWQhwv18YHWSA5LsBTwP2LgAx5EkzWDel2Wq6pYkLwf+C9gD+GBVXTzfx9FdcrlLuyq/N3eSVNVi1yBJmmd+QlWSOmS4S1KHDHdJ6tCivc9d8yPJwxh8AnhZ69oKbKyqSxavKkmLzSv33ViS1zP49Q4Bvt5uAU71F7ZpV5bk6MWuoXe+W2Y3luQHwMOr6tdT+vcCLq6q1YtTmXTXkvyoqlYudh09c1lm93YbsD9w+ZT+/do+adEkuWimXcDSnVnL3ZHhvnt7JXBukh9y+y9rWwkcCLx8sYqSmqXA04DrpvQH+PLOL+fuxXDfjVXVp5M8lMGvWR7+ger5VXXr4lUmAXA2cN+qunDqjiSf2+nV3M245i5JHfLdMpLUIcNdkjpkuEtShwx3SeqQ4S5JHfp/5LuNim69yR8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure() # Creates a new figure\n",
    "X[\"Survived\"].value_counts().plot(kind=\"bar\", title=\"Survived Distribution\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dans cet esprit, nous allons maintenant déplacer les valeurs de classe (de sortie) du dataframe X de pandas dans un tableau numpy appelé y. X est généralement utilisé pour désigner les attributs d'entrées et y est généralement utilisé pour représenter les valeurs de classe (ou sortie)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This can ONLY BE DONE ONCE, as we pop the values into a new variable to be used as predicted class\n",
    "y = X.pop(\"Survived\").values\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Puisque nous avons examiné la distribution des classes, nous allons maintenant examiner les autres attributs disponibles dans l'ensemble de données. Ci-dessous, nous imprimons une liste de tous les attributs disponibles et explorons les propriétés de la colonne *Embarked*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['PassengerId', 'Pclass', 'Name', 'Sex', 'Age', 'SibSp', 'Parch', 'Ticket', 'Fare', 'Cabin', 'Embarked']\n",
      "(891, 11)\n",
      "S    644\n",
      "C    168\n",
      "Q     77\n",
      "Name: Embarked, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "# Show all attributes\n",
    "print(list(X))\n",
    "# Examples of data exploration\n",
    "print(X.shape)\n",
    "print(X['Embarked'].value_counts())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "D'après l'avis ci-dessus, on peut voir que l'attribut *Embarked* contient 644 valeurs *S*, 168 *C* valeurs et 77 *Q* valeurs. Mais c'est en fait deux de moins que le total de 891 instances. Ainsi, il y a deux valeurs manquantes dans cet attribut. Ceci est indiqué via le code ci-dessous:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>61</th>\n",
       "      <td>62</td>\n",
       "      <td>1</td>\n",
       "      <td>Icard, Miss. Amelie</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>113572</td>\n",
       "      <td>80.0</td>\n",
       "      <td>B28</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>829</th>\n",
       "      <td>830</td>\n",
       "      <td>1</td>\n",
       "      <td>Stone, Mrs. George Nelson (Martha Evelyn)</td>\n",
       "      <td>female</td>\n",
       "      <td>62.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>113572</td>\n",
       "      <td>80.0</td>\n",
       "      <td>B28</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     PassengerId  Pclass                                       Name     Sex  \\\n",
       "61            62       1                        Icard, Miss. Amelie  female   \n",
       "829          830       1  Stone, Mrs. George Nelson (Martha Evelyn)  female   \n",
       "\n",
       "      Age  SibSp  Parch  Ticket  Fare Cabin Embarked  \n",
       "61   38.0      0      0  113572  80.0   B28      NaN  \n",
       "829  62.0      0      0  113572  80.0   B28      NaN  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find the rows with a null Embarked value\n",
    "X[X['Embarked'].isnull()]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(TO DO) Q1 - 2 points**   \n",
    "En vous basant sur l'exemple ci-dessus, explorez les attributs *Age* et *Cabin*. Imprimez les valeurs qu'ils contiennent et les lignes où il leur manque des valeurs. Notez que certaines valeurs peuvent ne pas être explicitement affichées lorsqu'elles sont appelées (représentées par ...), donc tant que les premières et les dernières lignes apparaissent, c'est bien."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24.00    30\n",
      "22.00    27\n",
      "18.00    26\n",
      "19.00    25\n",
      "30.00    25\n",
      "         ..\n",
      "55.50     1\n",
      "70.50     1\n",
      "66.00     1\n",
      "23.50     1\n",
      "0.42      1\n",
      "Name: Age, Length: 88, dtype: int64\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>3</td>\n",
       "      <td>Moran, Mr. James</td>\n",
       "      <td>male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>330877</td>\n",
       "      <td>8.4583</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>18</td>\n",
       "      <td>2</td>\n",
       "      <td>Williams, Mr. Charles Eugene</td>\n",
       "      <td>male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>244373</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>20</td>\n",
       "      <td>3</td>\n",
       "      <td>Masselmani, Mrs. Fatima</td>\n",
       "      <td>female</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2649</td>\n",
       "      <td>7.2250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>27</td>\n",
       "      <td>3</td>\n",
       "      <td>Emir, Mr. Farred Chehab</td>\n",
       "      <td>male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2631</td>\n",
       "      <td>7.2250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>29</td>\n",
       "      <td>3</td>\n",
       "      <td>O'Dwyer, Miss. Ellen \"Nellie\"</td>\n",
       "      <td>female</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>330959</td>\n",
       "      <td>7.8792</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>859</th>\n",
       "      <td>860</td>\n",
       "      <td>3</td>\n",
       "      <td>Razi, Mr. Raihed</td>\n",
       "      <td>male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2629</td>\n",
       "      <td>7.2292</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>863</th>\n",
       "      <td>864</td>\n",
       "      <td>3</td>\n",
       "      <td>Sage, Miss. Dorothy Edith \"Dolly\"</td>\n",
       "      <td>female</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8</td>\n",
       "      <td>2</td>\n",
       "      <td>CA. 2343</td>\n",
       "      <td>69.5500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>868</th>\n",
       "      <td>869</td>\n",
       "      <td>3</td>\n",
       "      <td>van Melkebeke, Mr. Philemon</td>\n",
       "      <td>male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>345777</td>\n",
       "      <td>9.5000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>878</th>\n",
       "      <td>879</td>\n",
       "      <td>3</td>\n",
       "      <td>Laleff, Mr. Kristo</td>\n",
       "      <td>male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>349217</td>\n",
       "      <td>7.8958</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>888</th>\n",
       "      <td>889</td>\n",
       "      <td>3</td>\n",
       "      <td>Johnston, Miss. Catherine Helen \"Carrie\"</td>\n",
       "      <td>female</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>W./C. 6607</td>\n",
       "      <td>23.4500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>177 rows × 11 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     PassengerId  Pclass                                      Name     Sex  \\\n",
       "5              6       3                          Moran, Mr. James    male   \n",
       "17            18       2              Williams, Mr. Charles Eugene    male   \n",
       "19            20       3                   Masselmani, Mrs. Fatima  female   \n",
       "26            27       3                   Emir, Mr. Farred Chehab    male   \n",
       "28            29       3             O'Dwyer, Miss. Ellen \"Nellie\"  female   \n",
       "..           ...     ...                                       ...     ...   \n",
       "859          860       3                          Razi, Mr. Raihed    male   \n",
       "863          864       3         Sage, Miss. Dorothy Edith \"Dolly\"  female   \n",
       "868          869       3               van Melkebeke, Mr. Philemon    male   \n",
       "878          879       3                        Laleff, Mr. Kristo    male   \n",
       "888          889       3  Johnston, Miss. Catherine Helen \"Carrie\"  female   \n",
       "\n",
       "     Age  SibSp  Parch      Ticket     Fare Cabin Embarked  \n",
       "5    NaN      0      0      330877   8.4583   NaN        Q  \n",
       "17   NaN      0      0      244373  13.0000   NaN        S  \n",
       "19   NaN      0      0        2649   7.2250   NaN        C  \n",
       "26   NaN      0      0        2631   7.2250   NaN        C  \n",
       "28   NaN      0      0      330959   7.8792   NaN        Q  \n",
       "..   ...    ...    ...         ...      ...   ...      ...  \n",
       "859  NaN      0      0        2629   7.2292   NaN        C  \n",
       "863  NaN      8      2    CA. 2343  69.5500   NaN        S  \n",
       "868  NaN      0      0      345777   9.5000   NaN        S  \n",
       "878  NaN      0      0      349217   7.8958   NaN        S  \n",
       "888  NaN      1      2  W./C. 6607  23.4500   NaN        S  \n",
       "\n",
       "[177 rows x 11 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# TODO Q1: For the Age attribute\n",
    "print(X['Age'].value_counts())\n",
    "X[X['Age'].isnull()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "G6             4\n",
      "C23 C25 C27    4\n",
      "B96 B98        4\n",
      "C22 C26        3\n",
      "E101           3\n",
      "              ..\n",
      "C47            1\n",
      "B42            1\n",
      "E31            1\n",
      "C62 C64        1\n",
      "B86            1\n",
      "Name: Cabin, Length: 147, dtype: int64\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>3</td>\n",
       "      <td>Moran, Mr. James</td>\n",
       "      <td>male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>330877</td>\n",
       "      <td>8.4583</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>3</td>\n",
       "      <td>Palsson, Master. Gosta Leonard</td>\n",
       "      <td>male</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>349909</td>\n",
       "      <td>21.0750</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>884</th>\n",
       "      <td>885</td>\n",
       "      <td>3</td>\n",
       "      <td>Sutehall, Mr. Henry Jr</td>\n",
       "      <td>male</td>\n",
       "      <td>25.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>SOTON/OQ 392076</td>\n",
       "      <td>7.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>885</th>\n",
       "      <td>886</td>\n",
       "      <td>3</td>\n",
       "      <td>Rice, Mrs. William (Margaret Norton)</td>\n",
       "      <td>female</td>\n",
       "      <td>39.0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>382652</td>\n",
       "      <td>29.1250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>886</th>\n",
       "      <td>887</td>\n",
       "      <td>2</td>\n",
       "      <td>Montvila, Rev. Juozas</td>\n",
       "      <td>male</td>\n",
       "      <td>27.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>211536</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>888</th>\n",
       "      <td>889</td>\n",
       "      <td>3</td>\n",
       "      <td>Johnston, Miss. Catherine Helen \"Carrie\"</td>\n",
       "      <td>female</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>W./C. 6607</td>\n",
       "      <td>23.4500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>890</th>\n",
       "      <td>891</td>\n",
       "      <td>3</td>\n",
       "      <td>Dooley, Mr. Patrick</td>\n",
       "      <td>male</td>\n",
       "      <td>32.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>370376</td>\n",
       "      <td>7.7500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>687 rows × 11 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     PassengerId  Pclass                                      Name     Sex  \\\n",
       "0              1       3                   Braund, Mr. Owen Harris    male   \n",
       "2              3       3                    Heikkinen, Miss. Laina  female   \n",
       "4              5       3                  Allen, Mr. William Henry    male   \n",
       "5              6       3                          Moran, Mr. James    male   \n",
       "7              8       3            Palsson, Master. Gosta Leonard    male   \n",
       "..           ...     ...                                       ...     ...   \n",
       "884          885       3                    Sutehall, Mr. Henry Jr    male   \n",
       "885          886       3      Rice, Mrs. William (Margaret Norton)  female   \n",
       "886          887       2                     Montvila, Rev. Juozas    male   \n",
       "888          889       3  Johnston, Miss. Catherine Helen \"Carrie\"  female   \n",
       "890          891       3                       Dooley, Mr. Patrick    male   \n",
       "\n",
       "      Age  SibSp  Parch            Ticket     Fare Cabin Embarked  \n",
       "0    22.0      1      0         A/5 21171   7.2500   NaN        S  \n",
       "2    26.0      0      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "4    35.0      0      0            373450   8.0500   NaN        S  \n",
       "5     NaN      0      0            330877   8.4583   NaN        Q  \n",
       "7     2.0      3      1            349909  21.0750   NaN        S  \n",
       "..    ...    ...    ...               ...      ...   ...      ...  \n",
       "884  25.0      0      0   SOTON/OQ 392076   7.0500   NaN        S  \n",
       "885  39.0      0      5            382652  29.1250   NaN        Q  \n",
       "886  27.0      0      0            211536  13.0000   NaN        S  \n",
       "888   NaN      1      2        W./C. 6607  23.4500   NaN        S  \n",
       "890  32.0      0      0            370376   7.7500   NaN        Q  \n",
       "\n",
       "[687 rows x 11 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# TODO Q1: For the Cabin attribute\n",
    "print(X['Cabin'].value_counts())\n",
    "X[X['Cabin'].isnull()]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**2. Nettoyage des données**    \n",
    "\n",
    "Après avoir exploré l'ensemble de données, nous allons maintenant travailler sur le nettoyage de certaines des valeurs manquant dans les attributs que nous utiliserons pour les algorithmes d'apprentissage machine.\n",
    "\n",
    "Plus précisément, nous remplirons les valeurs manquantes pour les attributs *Age* et *Embarked*. Bien qu'il soit également possible de le faire pour l'attribut *Cabin*, cet attribut nécessite des précautions supplémentaires lors du remplissage de ses valeurs manquantes. Ainsi, ce notebook n'explorera pas l'attribut *Cabin* malgré son importance (puisque la localisation du passager est un identifiant important pour savoir s'il a survécu ou non).\n",
    "\n",
    "Pour déterminer comment remplir les valeurs manquantes, nous considérerons la valeur moyenne de l'attribut lui-même ou d'autres attributs pour remplir les valeurs manquantes. Cependant, des méthodes plus robustes peuvent être utilisées. Pour votre propre intérêt, l'article suivant fait un bon travail en discutant de certaines des méthodes disponibles: https://towardsdatascience.com/how-to-handle-missing-data-8646b18db0d4. Notez que le terme *imputation* fait référence au processus de remplissage des valeurs manquantes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pour l'attribut *Age*, nous remplirons les valeurs manquantes avec une méthode triviale. Les valeurs manquantes seront remplies avec l'âge moyen des passagers qui contiennent une valeur d'âge non nulle. Bien que cela causera probablement des problèmes, comme faire apprendre à l'algorithme des modèles inexacts à partir des valeurs d'âge moyennes, il s'agit de l'une des nombreuses approches trival pour remplir les données manquantes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Update the dataframe by filling in all missing age values as the mean of existing age values.\n",
    "X[\"Age\"].fillna(X[\"Age\"].mean(), inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pour l'attribut *Embarked*, nous ferons un peu plus d'investigation sur ce que devraient être les valeurs. Puisqu'il n'y a que deux valeurs manquantes et qu'il y a trois options disponibles pour ces valeurs, nous sélectionnerons manuellement la valeur à affecter aux lignes manquantes. Ci-dessous, nous voyons les moyennes de tous les attributs numériques de l'ensemble de données, pour chacune des trois valeurs *Embarked* possibles."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "When Embarked equals S:\n",
      "      PassengerId    Pclass        Age     SibSp     Parch       Fare\n",
      "mean    449.52795  2.350932  29.480855  0.571429  0.413043  27.079812\n",
      "\n",
      "When Embarked equals C:\n",
      "      PassengerId    Pclass        Age     SibSp     Parch       Fare\n",
      "mean   445.357143  1.886905  30.562419  0.386905  0.363095  59.954144\n",
      "\n",
      "When Embarked equals Q:\n",
      "      PassengerId    Pclass        Age     SibSp     Parch      Fare\n",
      "mean   417.896104  2.909091  29.113724  0.428571  0.168831  13.27603\n"
     ]
    }
   ],
   "source": [
    "# Isolate the rows for each possible value of Embarked (S, C, Q)\n",
    "# Get the mean values when equal to S\n",
    "df_embarked_S = X.loc[X[\"Embarked\"] == 'S']\n",
    "print(\"When Embarked equals S:\")\n",
    "print(df_embarked_S.describe().loc[['mean']])\n",
    "# Get the mean values when equal to C\n",
    "df_embarked_C = X.loc[X[\"Embarked\"] == 'C']\n",
    "print(\"\\nWhen Embarked equals C:\")\n",
    "print(df_embarked_C.describe().loc[['mean']])\n",
    "# Get the mean values when equal to Q\n",
    "df_embarked_Q = X.loc[X[\"Embarked\"] == 'Q']\n",
    "print(\"\\nWhen Embarked equals Q:\")\n",
    "print(df_embarked_Q.describe().loc[['mean']])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Maintenant, jetons un dernier coup d'œil aux lignes avec des valeurs *Embarked* manquantes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>61</th>\n",
       "      <td>62</td>\n",
       "      <td>1</td>\n",
       "      <td>Icard, Miss. Amelie</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>113572</td>\n",
       "      <td>80.0</td>\n",
       "      <td>B28</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>829</th>\n",
       "      <td>830</td>\n",
       "      <td>1</td>\n",
       "      <td>Stone, Mrs. George Nelson (Martha Evelyn)</td>\n",
       "      <td>female</td>\n",
       "      <td>62.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>113572</td>\n",
       "      <td>80.0</td>\n",
       "      <td>B28</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     PassengerId  Pclass                                       Name     Sex  \\\n",
       "61            62       1                        Icard, Miss. Amelie  female   \n",
       "829          830       1  Stone, Mrs. George Nelson (Martha Evelyn)  female   \n",
       "\n",
       "      Age  SibSp  Parch  Ticket  Fare Cabin Embarked  \n",
       "61   38.0      0      0  113572  80.0   B28      NaN  \n",
       "829  62.0      0      0  113572  80.0   B28      NaN  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Look at the rows with missing Embarked values again\n",
    "X[X['Embarked'].isnull()]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Étant donné que les deux lignes ci-dessus contiennent principalement des données similaires, il est prudent de dire qu'elles se verront attribuer les mêmes valeurs *Embarked*. En comparant les valeurs de ces lignes aux valeurs moyennes vues ci-dessus, on peut voir que lorsque *Embarked* est égal à C, les moyennes de tous les attributs (à l'exception de PassengerId, qui n'a pas de sens) sont les plus proches des valeurs d'attribut trouvées dans les deux ces lignes. Par conséquent, nous attribuerons la valeur C à l'attribut * Embarqué * des deux lignes ci-dessus."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Replace all NaN values for the Embarked column with the value 'C'\n",
    "X[\"Embarked\"].fillna('C', inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(TO DO) Q2 - 2 points**   \n",
    "Imaginons que nous voulions remplir les valeurs manquantes pour les attributs *Age* et *Embarked* via une autre méthode. Proposez une méthode unique pour remplir les valeurs manquantes (peuvent être triviales ou complexes) pour l'attribut *Age* et une autre méthode unique pour remplir les valeurs manquantes pour l'attribut *Embarked*. Répondez dans la cellule ci-dessous."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TODO: Q2       \n",
    "Pour l'attribut Age, on peut trouver les valeurs manquantes en utilisant la régression.\n",
    "Puisque pour l'attribut Embarked, il y en a que 2 rangées pour lesquelles la valeur de Embarked est manquante, on pourrait supprimer ces deux rangées de notre ensemble d'entrainement pour rendre le modèle plus robuste."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Avant de sélectionner les attributs à utiliser, nous convertirons les données catégorielles dans les colonnes *Sex* et *Embarked* en numériques. Cela se fera via un le one-hot-encoding, où chaque valeur catégorielle possible pour un attribut est conçue pour être son propre attribut. Ensuite, un seul de ces trois attributs contiendra la valeur 1 pour une ligne, où les autres contiennent 0. Dans le prochain notebook, nous explorerons le OneHotEncoder fourni par sklearn, mais nous traiterons cela manuellement avec les pandas pour ce notebook (via la fonction *get_dummies*)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Pclass                                               Name  \\\n",
       "0            1       3                            Braund, Mr. Owen Harris   \n",
       "1            2       1  Cumings, Mrs. John Bradley (Florence Briggs Th...   \n",
       "2            3       3                             Heikkinen, Miss. Laina   \n",
       "3            4       1       Futrelle, Mrs. Jacques Heath (Lily May Peel)   \n",
       "4            5       3                           Allen, Mr. William Henry   \n",
       "\n",
       "      Sex   Age  SibSp  Parch            Ticket     Fare Cabin Embarked  \n",
       "0    male  22.0      1      0         A/5 21171   7.2500   NaN        S  \n",
       "1  female  38.0      1      0          PC 17599  71.2833   C85        C  \n",
       "2  female  26.0      0      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3  female  35.0      1      0            113803  53.1000  C123        S  \n",
       "4    male  35.0      0      0            373450   8.0500   NaN        S  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Convert the categorical values stored within the Sex column to be numerical via One-Hot-Encoding\n",
    "X = pd.concat([X, pd.get_dummies(X['Sex'], prefix='Sex')], axis=1)\n",
    "# Drop the original column\n",
    "X.drop(['Sex'], axis=1, inplace=True)\n",
    "# Convert the categorical values stored within the Embarked column to be numerical via One-Hot-Encoding\n",
    "X = pd.concat([X, pd.get_dummies(X['Embarked'], prefix='Embarked')], axis=1)\n",
    "# Drop the original column\n",
    "X.drop(['Embarked'], axis=1, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Jetons un dernier coup d'œil aux 5 premières entrées de notre ensemble de données. Regardez en quoi les colonnes diffèrent de la sortie ci-dessus."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Sex_female</th>\n",
       "      <th>Sex_male</th>\n",
       "      <th>Embarked_C</th>\n",
       "      <th>Embarked_Q</th>\n",
       "      <th>Embarked_S</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Pclass                                               Name  \\\n",
       "0            1       3                            Braund, Mr. Owen Harris   \n",
       "1            2       1  Cumings, Mrs. John Bradley (Florence Briggs Th...   \n",
       "2            3       3                             Heikkinen, Miss. Laina   \n",
       "3            4       1       Futrelle, Mrs. Jacques Heath (Lily May Peel)   \n",
       "4            5       3                           Allen, Mr. William Henry   \n",
       "\n",
       "    Age  SibSp  Parch            Ticket     Fare Cabin  Sex_female  Sex_male  \\\n",
       "0  22.0      1      0         A/5 21171   7.2500   NaN           0         1   \n",
       "1  38.0      1      0          PC 17599  71.2833   C85           1         0   \n",
       "2  26.0      0      0  STON/O2. 3101282   7.9250   NaN           1         0   \n",
       "3  35.0      1      0            113803  53.1000  C123           1         0   \n",
       "4  35.0      0      0            373450   8.0500   NaN           0         1   \n",
       "\n",
       "   Embarked_C  Embarked_Q  Embarked_S  \n",
       "0           0           0           1  \n",
       "1           1           0           0  \n",
       "2           0           0           1  \n",
       "3           0           0           1  \n",
       "4           0           0           1  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**3. Sélection des attributs et définition des ensembles de test et validation**    \n",
    "\n",
    "Maintenant que les données sont prêtes à être utilisées, nous allons maintenant sélectionner les attributs avec lesquels nous allons travailler et définir les ensembles d'entraînement et de validation à partir des données disponibles. Pour notre sélection d'attributs, nous sélectionnerons simplement tous les attributs non nuls à l'exception de *PassengerId*, *Ticket* et *Name*.\n",
    "\n",
    "PassengerId est un numéro unique pour chaque passager et ne peut donc pas fournir de modèles utiles concernant les valeurs de classe cible. Il est également peu probable que les noms permettent de savoir si un passager a survécu ou non.\n",
    "\n",
    "Bien que nous puissions modifier l'attribut *Nom* pour qu'il devienne une valeur numérique basée sur le nombre de lettres dans le nom (peut-être que les noms avec plus de lettres de l'alphabet avaient un taux de survie plus élevé), nous ne le ferons pas car c'est une ligne de pensée illogique pour ce problème.\n",
    "\n",
    "Ticket est une valeur qui ne semble pas non plus fournir d'informations utiles. Par conséquent, nous ne l'utiliserons pas.\n",
    "\n",
    "Examinons les attributs restants avec des valeurs nulles après notre nettoyage des données pour voir ce qui ne sera pas sélectionné en tant qu'attribut."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PassengerId      0\n",
       "Pclass           0\n",
       "Name             0\n",
       "Age              0\n",
       "SibSp            0\n",
       "Parch            0\n",
       "Ticket           0\n",
       "Fare             0\n",
       "Cabin          687\n",
       "Sex_female       0\n",
       "Sex_male         0\n",
       "Embarked_C       0\n",
       "Embarked_Q       0\n",
       "Embarked_S       0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Look at what is null\n",
    "X.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Comme nous pouvons le voir, tous les attributs à l'exception de *Cabin* ne contiennent aucune valeur manquante. Bien que *Cabin* soit important, nous l'exclurons car nous ne l'avons pas correctement nettoyé et il manque une valeur à la majorité des données. Bien que nous puissions ignorer toutes les lignes qui ne contiennent pas de valeur *Cabin* ou que nous puissions attribuer une valeur «inconnue» aux valeurs manquantes, il y a trop de valeurs manquantes pour que cela soit raisonnable.\n",
    "\n",
    "Pour le reste des attributs, bien que nous puissions bénéficier de la combinaison de certains attributs similaires (tels que Parch et SibSp), nous les utiliserons tels quels car ce sont des attributs qui peuvent donner un aperçu du fait qu'un passager a survécu ou non.\n",
    "       \n",
    "Par exemple, peut-être plus de femmes ont survécu que d'hommes ou peut-être de jeunes passagers ont survécu plus fréquemment que de passagers plus âgés."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the list of attributes to use as features for the algorithms.\n",
    "featureSet = ['Pclass', 'Sex_female', 'Sex_male', 'Age', 'SibSp', \n",
    "              'Parch', 'Fare', 'Embarked_C', 'Embarked_Q', 'Embarked_S']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the dataframe to only contain the desired variables\n",
    "X = X[featureSet].copy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Une fois les attributs sélectionnées, nous allons maintenant diviser les données en un ensemble d'entraînemet et de validation à partir des données disponibles. Puisque nous n'avons pas les valeurs de classe de l'ensemble de test explicite (mais que nous avons un ensemble de test), l'ensemble de test produit par scikit-learn sera utilisé pour valider si l'apprentissage semble réussi. Ceci est considéré comme l'ensemble de validation malgré le nom de l'appel de fonction de sklearn."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(712, 10)\n",
      "(179, 10)\n",
      "(712,)\n",
      "(179,)\n"
     ]
    }
   ],
   "source": [
    "# split the large dataset into train and test\n",
    "X_train, X_val, y_train, y_val = train_test_split(X, y, test_size = 0.2, random_state=2)\n",
    "# Look at the shape of the outputs\n",
    "print(X_train.shape)\n",
    "print(y_val.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex_female</th>\n",
       "      <th>Sex_male</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked_C</th>\n",
       "      <th>Embarked_Q</th>\n",
       "      <th>Embarked_S</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>40.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>27.7208</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>16.7000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>873</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>47.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>182</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>9.0</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>31.3875</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>876</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>20.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9.8458</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Pclass  Sex_female  Sex_male   Age  SibSp  Parch     Fare  Embarked_C  \\\n",
       "30        1           0         1  40.0      0      0  27.7208           1   \n",
       "10        3           1         0   4.0      1      1  16.7000           0   \n",
       "873       3           0         1  47.0      0      0   9.0000           0   \n",
       "182       3           0         1   9.0      4      2  31.3875           0   \n",
       "876       3           0         1  20.0      0      0   9.8458           0   \n",
       "\n",
       "     Embarked_Q  Embarked_S  \n",
       "30            0           0  \n",
       "10            0           1  \n",
       "873           0           1  \n",
       "182           0           1  \n",
       "876           0           1  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Look at the training set\n",
    "X_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex_female</th>\n",
       "      <th>Sex_male</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked_C</th>\n",
       "      <th>Embarked_Q</th>\n",
       "      <th>Embarked_S</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>707</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>42.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>26.2875</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>21.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>615</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>24.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>65.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>169</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>28.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>56.4958</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>68</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Pclass  Sex_female  Sex_male   Age  SibSp  Parch     Fare  Embarked_C  \\\n",
       "707       1           0         1  42.0      0      0  26.2875           0   \n",
       "37        3           0         1  21.0      0      0   8.0500           0   \n",
       "615       2           1         0  24.0      1      2  65.0000           0   \n",
       "169       3           0         1  28.0      0      0  56.4958           0   \n",
       "68        3           1         0  17.0      4      2   7.9250           0   \n",
       "\n",
       "     Embarked_Q  Embarked_S  \n",
       "707           0           1  \n",
       "37            0           1  \n",
       "615           0           1  \n",
       "169           0           1  \n",
       "68            0           1  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Look at the validation set\n",
    "X_val.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**4. Techniques d'évaluation**    \n",
    "\n",
    "Plus tard, nous utiliserons davantage de techniques d'évaluation pour mieux évaluer les performances des modèles. Pour l'instant, nous allons définir des fonctions pour calculer les métriques *Précision*, *Rappel* et *Accuracy*. Tous ces éléments ont été abordés dans le notebook précédent.\n",
    "\n",
    "L'accuracy représente le pourcentage de classifications correctement effectuées par le modèle."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "def precision(actualTags, predictions, classOfInterest):\n",
    "    '''\n",
    "    Calculates the precision for a specific class, given the ground truth and predicted values.\n",
    "    '''\n",
    "    totalFound = 0\n",
    "    for i in range(len(actualTags)):\n",
    "        if (actualTags[i] == classOfInterest and actualTags[i] == predictions[i]):\n",
    "            totalFound += 1\n",
    "    return totalFound / np.count_nonzero(predictions == classOfInterest)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "def recall(actualTags, predictions, classOfInterest):\n",
    "    '''\n",
    "    Calculates the recall for a specific class, given the ground truth and predicted values.\n",
    "    '''\n",
    "    totalFound = 0\n",
    "    for i in range(len(actualTags)):\n",
    "        if (actualTags[i] == classOfInterest and actualTags[i] == predictions[i]):\n",
    "            totalFound += 1\n",
    "    return totalFound / np.count_nonzero(actualTags == classOfInterest)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "def accuracy(actualTags, predictions):\n",
    "    '''\n",
    "    Calculates the average number of correct predictions.\n",
    "        - actualTags: The ground truth\n",
    "        - predictions: What the model predicts\n",
    "    '''\n",
    "    totalFound = 0\n",
    "    for i in range(len(actualTags)):\n",
    "        if (actualTags[i] == predictions[i]):\n",
    "            totalFound += 1\n",
    "    return totalFound / len(predictions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.4"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Example of calculating accuracy\n",
    "accuracy([0, 1, 1, 1, 0], [1, 1, 1, 0, 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**5. Naive Bayes**    \n",
    "\n",
    "Avec les données prétraitées et plusieurs fonctions d'évaluation définies, nous sommes maintenant prêts à utiliser certains algorithmes d'apprentissage machine pour effectuer les prédictions de savoir si un passager a survécu ou non sur le Titanic.\n",
    "\n",
    "La première approche que nous utiliserons est l'algorithme Naive Bayes fourni par scikit-learn. Ce sera un processus très similaire à ce que vous avez fait dans le notebook 4. Vous entraînerez le modèle en appelant la fonction *fit* et récupérerez les prédictions du modèle en appelant la fonction *predict*. Contrairement au dernier notebook, nous avons déjà toutes nos données prétraitées en tant qu'attributs numériques dans un dataframe pandas, avec les étiquettes de classe sous forme de tableaux numpy (y_train et y_val). Ainsi, nous n'aurons pas besoin d'effectuer des transformations, telles que celles qui ont été effectuées dans le notebook 3.\n",
    "\n",
    "La première tâche consiste à entraîner le modèle avec la fonction .fit()."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Train the model with the training set by calling the .fit() function\n",
    "# X_train contains the features used to train the model\n",
    "# y_train contains the class labels for the samples from X_train\n",
    "clf_nb = MultinomialNB().fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nous pouvons ensuite voir comment l'algorithme fonctionne lors de la prédiction des échantillons dans l'ensemble d'apprentissage via le code suivant."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Comparing the first ten actual values to the predicted values:\n",
      "[0 1 0 0 0 0 0 1 1 0]\n",
      "[0 1 0 1 0 0 0 0 0 0]\n",
      "Calculating the total accuracy for the training set:\n",
      "0.6980337078651685\n"
     ]
    }
   ],
   "source": [
    "print(\"Comparing the first ten actual values to the predicted values:\")\n",
    "# Print the first ten class labels for the training set\n",
    "print(y_train[0:10])\n",
    "# Predict whether the passengers did or did not survive the Titanic on the training set\n",
    "nb_train_predictions = clf_nb.predict(X_train)\n",
    "# Print the first ten predictions\n",
    "print(nb_train_predictions[0:10])\n",
    "print(\"Calculating the total accuracy for the training set:\")\n",
    "print(accuracy(y_train, nb_train_predictions))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "D'après ce qui précède, nous pouvons voir qu'après avoir été entraîné, le modèle peut prédire correctement un peu moins de 70% de tous les échantillons de l'ensemble d'apprentissage. Mais comment fonctionne-t-il sur l'ensemble de validation?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(TO DO) Q3 - 3 points**   \n",
    "En suivant l'exemple fourni ci-dessus, obtenez les prédictions du modèle Naive Bayes sur l'ensemble de validation (X_val, y_val) et imprimez la précision, le rappel et l'accuracy de l'ensemble de validation (pour les deux classes, 1 et 0)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Precision when class = 0: 0.6535433070866141\n",
      "Precision when class = 1: 0.6730769230769231\n",
      "Recall when class = 0: 0.83\n",
      "Recall when class = 1: 0.4430379746835443\n",
      "Accuracy: 0.659217877094972\n"
     ]
    }
   ],
   "source": [
    "# TODO\n",
    "\n",
    "# Get the predictions for the validation set\n",
    "nb_val_predictions = clf_nb.predict(X_val)\n",
    "# Retrieve and print the precision values for class 1 and class 0\n",
    "print(\"Precision when class = 0: \" + str(precision(y_val, nb_val_predictions,0)))\n",
    "print(\"Precision when class = 1: \" + str(precision(y_val, nb_val_predictions,1)))\n",
    "# Retrieve and print the recall values for class 1 and class 0\n",
    "print(\"Recall when class = 0: \" + str(recall(y_val, nb_val_predictions,0)))\n",
    "print(\"Recall when class = 1: \" + str(recall(y_val, nb_val_predictions,1)))\n",
    "# Retrieve and print the accuracy for the model\n",
    "print(\"Accuracy: \" + str(accuracy(y_val,nb_val_predictions)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**6. Régression Logistique**    \n",
    "\n",
    "Avec les prédictions et les données obtenues pour l'algorithme d'apprentissage machine Naive Bayes, nous allons essayer d'utiliser un autre algorithme d'apprentissage machine qui peut ou non fonctionner mieux que l'approche Naive Bayes. Plus précisément, nous utiliserons l'algorithme d'apprentissage machine de régression logistique pour prédire qui a survécu et n'a pas survécu sur le Titanic.\n",
    "\n",
    "Puisque l'algorithme de régression logistique que nous allons utiliser provient également de scikit-learn, le processus général sera presque identique à ce que nous avons fait pour le classificateur Naive Bayes. La principale différence est que nous allons maintenant utiliser un algorithme différent contenant certaines options que nous devrons définir. Tous les détails des façons possibles d'ajuster le modèle de régression logistique sont disponibles [ici] (https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html).\n",
    "\n",
    "Nous allons d'abord initialiser et entraîner le modèle avec les données d'apprentissage."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Train the model with the training set by calling the .fit() function\n",
    "# X_train contains the features used to train the model\n",
    "# y_train contains the class labels for the samples from X_train\n",
    "clf_lr = LogisticRegression(solver='lbfgs', max_iter=1000, random_state=1).fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ensuite, nous verrons comment le modèle fonctionne sur les données d'entraînement pour avoir une idée de la façon dont l'entraînement s'est déroulé. Puisque nous nous sommes entraînés sur ces données, nous espérons obtenir des résultats solides."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Comparing the first ten actual values to the predicted values:\n",
      "[0 1 0 0 0 0 0 1 1 0]\n",
      "[0 1 0 0 0 0 0 1 0 0]\n",
      "Calculating the total accuracy for the training set:\n",
      "0.8103932584269663\n"
     ]
    }
   ],
   "source": [
    "print(\"Comparing the first ten actual values to the predicted values:\")\n",
    "# Print the first ten class labels for the training set\n",
    "print(y_train[0:10])\n",
    "# Predict whether the passengers did or did not survive the Titanic on the training set\n",
    "lr_train_predictions = clf_lr.predict(X_train)\n",
    "# Print the first ten predictions\n",
    "print(lr_train_predictions[0:10])\n",
    "print(\"Calculating the total accuracy for the training set:\")\n",
    "print(accuracy(y_train, lr_train_predictions))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(TO DO) Q4 - 3 points**   \n",
    "En suivant l'exemple fourni ci-dessus, obtenez les prédictions du modèle de régression logistique sur l'ensemble de validation (X_val, y_val) et imprimez la précision, le rappel et l'accuracy de l'ensemble de validation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Precision when class = 0: 0.7583333333333333\n",
      "Precision when class = 1: 0.847457627118644\n",
      "Recall when class = 0: 0.91\n",
      "Recall when class = 1: 0.6329113924050633\n",
      "Accuracy: 0.7877094972067039\n"
     ]
    }
   ],
   "source": [
    "# TODO\n",
    "\n",
    "# Get the predictions for the validation set\n",
    "lr_val_predictions = clf_lr.predict(X_val)\n",
    "# Retrieve and print the precision values for class 1 and class 0\n",
    "print(\"Precision when class = 0: \" + str(precision(y_val, lr_val_predictions,0)))\n",
    "print(\"Precision when class = 1: \" + str(precision(y_val, lr_val_predictions,1)))\n",
    "# Retrieve and print the recall values for class 1 and class 0\n",
    "print(\"Recall when class = 0: \" + str(recall(y_val, lr_val_predictions,0)))\n",
    "print(\"Recall when class = 1: \" + str(recall(y_val, lr_val_predictions,1)))\n",
    "# Retrieve and print the accuracy for the model\n",
    "print(\"Accuracy: \" + str(accuracy(y_val, lr_val_predictions)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**7. Plus de techniques d'évaluation**    \n",
    "\n",
    "Maintenant que vous avez entraîné et testé les modèles ci-dessus, vous allez définir deux méthodes d'évaluation supplémentaires. La première sera la micro-moyenne sur les précisions et la seconde sera la macro-moyenne sur les précisions. Les définitions de ceux-ci se trouvent dans votre matériel de cours.\n",
    "\n",
    "Bien que nous puissions également définir la micro-moyenne et la macro-moyenne pour les rappels, nous ne le ferons pas pour ce notebook afin d'éviter d'avoir trop de mesures d'évaluation à prendre en compte. En réalité, chaque ensemble de données aura des critères d'évaluation qui sont de plus en moins importants pour évaluer le problème."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(TO DO) Q5 - 2 points**   \n",
    "\n",
    "Dans la cellule ci-dessous, complétez la définition de la fonction micro_precision_average. Vous devez utiliser uniquement les paramètres fournis et ne pouvez utiliser aucune fonctionnalité de la bibliothèque de scikit-learn."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO\n",
    "def micro_precision_average(actualTags, predictions, class1, class2):\n",
    "    '''\n",
    "    Calculates the Micro-average on precisions.\n",
    "        - actualTags: The ground truth\n",
    "        - predictions: The predicted class values\n",
    "        - class1: The value of the first class\n",
    "        - class2: The value of the second class\n",
    "    '''\n",
    "    totalClass1 = 0\n",
    "    totalClass2 = 0\n",
    "    for i in range(len(actualTags)):\n",
    "        if (actualTags[i] == class1 and actualTags[i] == predictions[i]):\n",
    "            totalClass1 += 1\n",
    "        if (actualTags[i] == class2 and actualTags[i] == predictions[i]):\n",
    "            totalClass2 += 1\n",
    "    lenClass1 = np.count_nonzero(predictions == class1)\n",
    "    lenClass2 = np.count_nonzero(predictions == class2)\n",
    "    micro = (totalClass1 + totalClass2) / (lenClass1 + lenClass2)\n",
    "    return micro\n",
    "        \n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(TO DO) Q6 - 2 points**   \n",
    "Dans la cellule ci-dessous, complétez la définition de la fonction macro_precision_average. Vous devez utiliser uniquement les paramètres fournis et ne pouvez utiliser aucune fonctionnalité de la bibliothèque de scikit-learn."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO\n",
    "def macro_precision_average(actualTags, predictions, class1, class2):\n",
    "    '''\n",
    "    Calculates the Macro-average on precisions.\n",
    "        - actualTags: The ground truth\n",
    "        - predictions: The predicted class values\n",
    "        - class1: The value of the first class\n",
    "        - class2: The value of the second class\n",
    "    '''\n",
    "\n",
    "    precisionClass1 = precision(actualTags, predictions, class1)\n",
    "    precisionClass2 = precision(actualTags, predictions, class2)\n",
    "    return ((precisionClass1 + precisionClass2)/2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(TO DO) Q7 - 2 points**   \n",
    "Pour tester ces fonctions d'évaluation, évaluez la micro-moyenne et la macro-moyenne sur les précisions de vos tests sur l'ensemble de validation avec les modèles Naive Bayes et de régression logistique. Imprimez les résultats."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For Naive Bayes:\n",
      "micro-precision: \n",
      "0.659217877094972\n",
      "macro-precision: \n",
      "0.6633101150817686\n",
      "For Logistic Regression:\n",
      "micro-precision: \n",
      "0.7877094972067039\n",
      "macro-precision: \n",
      "0.8028954802259887\n"
     ]
    }
   ],
   "source": [
    "# TODO\n",
    "\n",
    "print(\"For Naive Bayes:\")\n",
    "print(\"micro-precision: \")\n",
    "print(str(micro_precision_average(y_val,nb_val_predictions,0,1)))\n",
    "print(\"macro-precision: \")\n",
    "print(str(macro_precision_average(y_val,nb_val_predictions,0,1)))\n",
    "\n",
    "print(\"For Logistic Regression:\")\n",
    "print(\"micro-precision: \")\n",
    "print(str(micro_precision_average(y_val,lr_val_predictions,0,1)))\n",
    "print(\"macro-precision: \")\n",
    "print(str(macro_precision_average(y_val,lr_val_predictions,0,1)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**8. Discussion**    \n",
    "\n",
    "Comme pour toutes les expériences d'apprentissage machine, nous devons utiliser les données obtenues pour comprendre quel modèle est le meilleur et pourquoi. Dans ce scénario, nous avons les valeurs de précision, de rappel, de moyenne, de micro-moyenne et de macro-moyenne. Vous trouverez ci-dessous une question de discussion séparée en sections auxquelles vous devez répondre."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(TO DO) Q8 (a) - 2 points**   \n",
    "Copiez les sorties pour la précision, le rappel, l'accuracy, la micro-moyenne et la macro-moyenne obtenues pour les deux modèles dans le tableau ci-dessous (uniquement à partir des tests de validation). Entrez chaque valeur dans la balise <td\\> </td\\> correspondante. Pour chaque approche d'évaluation, indiquez quel modèle d'apprentissage machine fournit les meilleurs résultats."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TODO:     \n",
    "<table>\n",
    "    <tr>\n",
    "        <td></td>\n",
    "        <td><strong>Naive Bayes</strong></td>\n",
    "        <td><strong>Régression Logistique</strong></td>\n",
    "        <td><strong>Meilleur modèle</strong></td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td><strong>Précision (classe=0)</strong></td>\n",
    "        <td>0.6535433070866141</td>\n",
    "        <td>0.7583333333333333</td>\n",
    "        <td>Régression Logistique</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td><strong>Précision (classe=1)</strong></td>\n",
    "        <td>0.6730769230769231</td>\n",
    "        <td>0.847457627118644</td>\n",
    "        <td>Régression Logistique</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td><strong>Rappel (classe=0)</strong></td>\n",
    "        <td>0.83</td>\n",
    "        <td>0.91</td>\n",
    "        <td>Régression Logistique</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td><strong>Rappel (classe=1)</strong></td>\n",
    "        <td>0.4430379746835443</td>\n",
    "        <td>0.6329113924050633</td>\n",
    "        <td>Régression Logistique</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td><strong>Accuracy</strong></td>\n",
    "        <td>0.7877094972067039</td>\n",
    "        <td>0.7877094972067039</td>\n",
    "        <td>Égalité</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td><strong>Micro-moyenne</strong></td>\n",
    "        <td>0.659217877094972</td>\n",
    "        <td>0.7877094972067039</td>\n",
    "        <td>Régression Logistique</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td><strong>Macro-moyenne</strong></td>\n",
    "        <td>0.6633101150817686</td>\n",
    "        <td>0.8028954802259887</td>\n",
    "        <td>Régression Logistique</td>\n",
    "    </tr>    \n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(TO DO) Q8 (b) - 2 points**   \n",
    "D'après vos résultats, quel modèle est le plus performant pour prédire si un passager survit ou non sur le Titanic? Justifiez votre réponse."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TODO:     \n",
    "La régression logistique est le plus performant.\n",
    "On peut voir dans la table ci-dessus que la Régression Logistique est le meilleur modèle pour toutes les évaluations(sauf l'accuracy pour laquelle il y a une égalité avec le Naive Bayes)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**9. Équilibrer la distribution des classes**    \n",
    "\n",
    "Comme mentionné au début du notebook, cet ensemble de données contient plus de passagers qui n'ont pas survécu sur le Titanic que ceux qui ont survécu. Il en résulte un déséquilibre de classe où la classe majoritaire (0) contient plus d'instances que la classe minoritaire (1). Il existe de nombreuses méthodes pour gérer ce problème. Une méthode consiste simplement à prendre la classe minoritaire et, grâce à une méthode, à produire plus d'instances de cette classe. Par exemple, une méthode triviale pourrait consister à sélectionner au hasard un nombre d'instances et à les dupliquer pour correspondre au nombre d'instances de la classe majoritaire. Ce concept est appelé * suréchantillonnage *. Vous pouvez également supprimer des instances de la classe majoritaire pour équilibrer la distribution de classe (appelée * sous-échantillonnage *).\n",
    "\n",
    "Pour ce notebook, le suréchantillonnage sera effectué via l'implémentation par imblearn d'une technique appelée SMOTE. Comprendre SMOTE dépasse le cadre de ce notebook, mais plus d'informations peuvent être trouvées [ici] (https://machinelearningmastery.com/smote-oversampling-for-imbalanced-classification/). La chose importante à comprendre est que le nombre d'instances pour chaque valeur de classe sera le même en créant artificiellement de nouvelles instances de passagers ayant survécu au Titanic."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the SMOTE instance\n",
    "smote = SMOTE(random_state=0, sampling_strategy=\"minority\")\n",
    "# Retrieve the oevrsampled data\n",
    "X_os, y_os = smote.fit_sample(X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Notez que l'ensemble de données suréchantillonné est maintenant défini dans X_os et y_os, et non dans X et y. Nous utiliserons ces valeurs pour le reste du notebook. Ci-dessous, nous examinons maintenant combien d'instances de notre nouvel ensemble de données contiennent la classe 0 et combien contiennent la classe 1. Ces valeurs sont maintenant équilibrées par rapport au graphique présenté dans la section 1 de ce notebook. Notez que X_os est maintenant un tableau numpy plutôt qu'une dataframe en raison de cette conversion. Cela ne change rien à la façon dont nous l'utilisons, mais change la méthode de traçage (graphique) des données."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 contains 549 instances.\n",
      "1 contains 549 instances.\n"
     ]
    }
   ],
   "source": [
    "# Print the number of instances for each class\n",
    "class_names, totals = np.unique(y_os, return_counts=True)\n",
    "print(str(class_names[0]) + \" contains \" + str(totals[0]) + \" instances.\")\n",
    "print(str(class_names[1]) + \" contains \" + str(totals[1]) + \" instances.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Avec l'ensemble de données suréchantillonné, nous allons maintenant définir nos nouveaux ensembles de train et de validation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(878, 10)\n",
      "(220,)\n"
     ]
    }
   ],
   "source": [
    "# split the large dataset into train and test\n",
    "X_train_os, X_val_os, y_train_os, y_val_os = train_test_split(X_os, y_os, test_size = 0.2, random_state=2)\n",
    "# Look at the shape of the outputs\n",
    "print(X_train_os.shape)\n",
    "print(y_val_os.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Avec tout cela défini, nous allons maintenant entraîner et tester un nouveau modèle Naive Bayes et de régression logistique, évaluer les modèles et déterminer si le suréchantillonnage a donné de meilleurs résultats que sans le suréchantillonnage."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(TO DO) Q9 - 10 points au total**   \n",
    "Répétez l'expérience d'apprentissage machine effectuée dans les sections précédentes sur les nouvelles données suréchantillonnées. Assurez-vous d'utiliser les noms appropriés (***ne pas oublier les _os dans les noms*** pour éviter les solutions incorrectes)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(TO DO) Q9 (a) - 3 points**   \n",
    "Entraîner, tester et évaluer l'ensemble de validation (avec toutes les métriques d'évaluation) avec un classificateur Naive Bayes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Precision when class = 0: 0.5874125874125874\n",
      "Precision when class = 1: 0.7662337662337663\n",
      "Recall when class = 0: 0.8235294117647058\n",
      "Recall when class = 1: 0.5\n",
      "Accuracy: 0.65\n",
      "For Naive Bayes:\n",
      "micro-precision: \n",
      "0.65\n",
      "macro-precision: \n",
      "0.6768231768231768\n"
     ]
    }
   ],
   "source": [
    "# TODO: Q9 (a)\n",
    "\n",
    "clf_nb_os = MultinomialNB().fit(X_train_os, y_train_os)\n",
    "# Get the predictions for the validation set\n",
    "nb_val_os_predictions = clf_nb_os.predict(X_val_os)\n",
    "# Retrieve and print the precision values for class 1 and class 0\n",
    "print(\"Precision when class = 0: \" + str(precision(y_val_os, nb_val_os_predictions,0)))\n",
    "print(\"Precision when class = 1: \" + str(precision(y_val_os, nb_val_os_predictions,1)))\n",
    "# Retrieve and print the recall values for class 1 and class 0\n",
    "print(\"Recall when class = 0: \" + str(recall(y_val_os, nb_val_os_predictions,0)))\n",
    "print(\"Recall when class = 1: \" + str(recall(y_val_os, nb_val_os_predictions,1)))\n",
    "# Retrieve and print the accuracy for the model\n",
    "print(\"Accuracy: \" + str(accuracy(y_val_os,nb_val_os_predictions)))\n",
    "\n",
    "print(\"For Naive Bayes:\")\n",
    "print(\"micro-precision: \")\n",
    "print(str(micro_precision_average(y_val_os,nb_val_os_predictions,0,1)))\n",
    "print(\"macro-precision: \")\n",
    "print(str(macro_precision_average(y_val_os,nb_val_os_predictions,0,1)))\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(TO DO) Q9 (b) - 3 points**   \n",
    "Entraîner, tester et évaluer l'ensemble de validation (avec toutes les métriques d'évaluation) avec un classificateur régression logistique."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Precision when class = 0: 0.8035714285714286\n",
      "Precision when class = 1: 0.8888888888888888\n",
      "Recall when class = 0: 0.8823529411764706\n",
      "Recall when class = 1: 0.8135593220338984\n",
      "Accuracy: 0.8454545454545455\n",
      "For Logistic Regression:\n",
      "micro-precision: \n",
      "0.8454545454545455\n",
      "macro-precision: \n",
      "0.8462301587301587\n"
     ]
    }
   ],
   "source": [
    "# TODO: Q9 (b)\n",
    "clf_lr_os = LogisticRegression(solver='lbfgs', max_iter=1000, random_state=1).fit(X_train_os, y_train_os)\n",
    "# Get the predictions for the validation set\n",
    "lr_val_os_predictions = clf_lr_os.predict(X_val_os)\n",
    "# Retrieve and print the precision values for class 1 and class 0\n",
    "print(\"Precision when class = 0: \" + str(precision(y_val_os, lr_val_os_predictions,0)))\n",
    "print(\"Precision when class = 1: \" + str(precision(y_val_os, lr_val_os_predictions,1)))\n",
    "# Retrieve and print the recall values for class 1 and class 0\n",
    "print(\"Recall when class = 0: \" + str(recall(y_val_os, lr_val_os_predictions,0)))\n",
    "print(\"Recall when class = 1: \" + str(recall(y_val_os, lr_val_os_predictions,1)))\n",
    "# Retrieve and print the accuracy for the model\n",
    "print(\"Accuracy: \" + str(accuracy(y_val_os,lr_val_os_predictions)))\n",
    "\n",
    "print(\"For Logistic Regression:\")\n",
    "print(\"micro-precision: \")\n",
    "print(str(micro_precision_average(y_val_os,lr_val_os_predictions,0,1)))\n",
    "print(\"macro-precision: \")\n",
    "print(str(macro_precision_average(y_val_os,lr_val_os_predictions,0,1)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(TO DO) Q9 (c) - 2 points**   \n",
    "Comparez les résultats entre les deux expériences ci-dessus (de Q9 (a) et Q9 (b)), quel modèle est le meilleur et pourquoi?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TODO:     \n",
    "<table>\n",
    "    <tr>\n",
    "        <td></td>\n",
    "        <td><strong>Naive Bayes (avec suréchantillonnage) </strong></td>\n",
    "        <td><strong>Régression Logistique (avec suréchantillonnage) </strong></td>\n",
    "        <td><strong>Meilleur modèle</strong></td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td><strong>Précision (classe=0)</strong></td>\n",
    "        <td>0.5874125874125874</td>\n",
    "        <td>0.8035714285714286</td>\n",
    "        <td>Régression Logistique</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td><strong>Précision (classe=1)</strong></td>\n",
    "        <td> 0.7662337662337663</td>\n",
    "        <td>0.8888888888888888</td>\n",
    "        <td>Régression Logistique</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td><strong>Rappel (classe=0)</strong></td>\n",
    "        <td>0.8235294117647058</td>\n",
    "        <td>0.8823529411764706</td>\n",
    "        <td>Régression Logistique</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td><strong>Rappel (classe=1)</strong></td>\n",
    "        <td>0.5</td>\n",
    "        <td>0.8135593220338984</td>\n",
    "        <td>Régression Logistique</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td><strong>Accuracy</strong></td>\n",
    "        <td> 0.65</td>\n",
    "        <td>0.8454545454545455</td>\n",
    "        <td>Régression Logistique</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td><strong>Micro-moyenne</strong></td>\n",
    "        <td>0.65</td>\n",
    "        <td>0.8454545454545455</td>\n",
    "        <td>Régression Logistique</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td><strong>Macro-moyenne</strong></td>\n",
    "        <td>0.6768231768231768</td>\n",
    "        <td>0.8462301587301587</td>\n",
    "        <td>Régression Logistique</td>\n",
    "    </tr>    \n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TODO: ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(TO DO) Q9 (d) - 2 points**   \n",
    "Le modèle sélectionné à partir de Q9 (c) fonctionne-t-il mieux ou moins bien que le modèle sélectionné à Q8 (b)? Cela signifie-t-il que le suréchantillonnage aide l'apprentissage machine à mieux ou moins bien apprendre dans ce scénario? Justifiez votre réponse.\n",
    "\n",
    "Remarque: il n'est pas nécessaire de mettre toutes les données dans un autre tableau (vous pouvez certainement le faire si vous souhaitez mieux l'organiser). Fournissez simplement les réponses et les justifications expliquant pourquoi."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TODO: Le modèle de 9 c) donne presque le même résultat que le modèle de 8 b). Avec le suréchantillonage, les valeurs des différentes évaluations ont changé mais le pattern est toujours le même (la régression logistique ayant des valeurs supérieures ou égales au Naives Bayesien dans toutes les évaluations).\n",
    "Dans ce scénario, le suréchantillonage nous a permis de conclure avec plus de certitude que le modèle de régression logistique est meilleur que le modèle Naives Bayesien car les écarts entre les différentes évaluations sont plus larges. De ce point de vue, on pourrait affirmer que le modèle de 9 c) fonctionne mieux que le modèle de 8 c)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(TO DO) OPTIONNEL - Aucune note**   \n",
    "Si vous êtes intéressé, vous pouvez essayer d'autres approches pour gérer le problème de déséquilibre de classe via les méthodes disponibles avec [imblearn] (https://imbalanced-learn.readthedocs.io/en/stable/api.html). Ensuite, vous pouvez comparer ces résultats pour voir quelle méthode fonctionne le mieux pour ce problème en fonction de la façon dont nous avons nettoyé les données et sélectionné les attributs.\n",
    "\n",
    "Vous pouvez également essayer de travailler avec le fichier de données de test pour voir quelles sont vos sorties."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***SIGNATURE:***\n",
    "My name is ---Bhavika Sewpal-----------------------.\n",
    "My student number is --300089940---------------.\n",
    "I certify being the author of this assignment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}