Fake Pictures Detector.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c19246cf",
   "metadata": {},
   "source": [
    "# Fake pictures detector "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "101c6f4e",
   "metadata": {},
   "source": [
    "## Topic\n",
    "\n",
    "In this project, I will be going through many real people pictures and fake photoshop made pictures, using dimensionality reduction technique, more specifically PCA, reduce the dimendion of the pictures and then apply classification algorithms on them. The row pictures are downloaded and transformed into arrays, only one color channel is kept with the values at each pixel stored into an array, PCA is then apply twice, first time to reduce the photo matrices into their 3 principal components and second to reduce those 3 components in to one principal components which contains 50% of the information in the picture.\n",
    "Different classification algorithms are then applied on the decomposed matrices to try and predict whether a certain picture is real or fake."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ceecf01",
   "metadata": {},
   "source": [
    "## Objectives\n",
    "\n",
    "- Reduce the dimension of the photos\n",
    "- Apply classification algorithms to determine whether a picture is real or fake"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e18a40d",
   "metadata": {},
   "source": [
    "## Summary\n",
    "\n",
    "- Importing Libraries\n",
    "- Quick look at the dataset\n",
    "- Data pre-processing\n",
    "- Logistic Regression\n",
    "- Naive Bayes Classifier\n",
    "- Support Vector Machines\n",
    "- XGBoost Classifier\n",
    "- K Nearest Neighbours\n",
    "- Decision Tree Classifier"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0ab9248c",
   "metadata": {},
   "source": [
    "## Importing libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "3569cea2",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\xgboost\\compat.py:36: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
      "  from pandas import MultiIndex, Int64Index\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from matplotlib.pyplot import imread\n",
    "import matplotlib.pyplot as plt\n",
    "import os\n",
    "import matplotlib.image as mplib \n",
    "import pandas as pd\n",
    "from sklearn.decomposition import PCA\n",
    "from PIL import Image\n",
    "import cv2\n",
    "from sklearn.model_selection import train_test_split, GridSearchCV, RandomizedSearchCV\n",
    "from sklearn.metrics import  classification_report, confusion_matrix,f1_score\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "import xgboost \n",
    "from xgboost import XGBClassifier\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.naive_bayes import GaussianNB\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.tree import DecisionTreeClassifier"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8bff2dc0",
   "metadata": {},
   "source": [
    "## The Dataset\n",
    "The dataset is two folders containing real images of different people and photoshop made images.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "98c15b7d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# get two lists containing the names of the files\n",
    "list_real = os.listdir(r\"C:\\Users\\imane\\OneDrive\\Desktop\\eigenfaces\\training_real\")\n",
    "list_fake = os.listdir(r\"C:\\Users\\imane\\OneDrive\\Desktop\\eigenfaces\\training_fake\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "9f58d199",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1081\n"
     ]
    }
   ],
   "source": [
    "print(len(list_real))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "bae5a790",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "960\n"
     ]
    }
   ],
   "source": [
    "print(len(list_fake))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "afe1677d",
   "metadata": {},
   "source": [
    "I have 1081 images of real people and 960 images of photoshoped fake images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "cd36c433",
   "metadata": {},
   "outputs": [],
   "source": [
    "# get paths to the image folders\n",
    "path_real = r\"C:\\Users\\imane\\OneDrive\\Desktop\\eigenfaces\\training_real\"\n",
    "path_fake = r\"C:\\Users\\imane\\OneDrive\\Desktop\\eigenfaces\\training_fake\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b4395b34",
   "metadata": {},
   "outputs": [],
   "source": [
    "# get a list of the paths to real images\n",
    "pics_real = [os.path.join(path_real, i) for i in list_real]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "aaa3b06d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# get a list of the paths to fake images\n",
    "pics_fake = [os.path.join(path_fake, i) for i in list_fake]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "59a47c90",
   "metadata": {},
   "outputs": [],
   "source": [
    "# get a list of the paths to the real photos\n",
    "paths_real = [os.path.join(path_real,i) for i in list_real]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "f6b526a2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# get a list of the paths to the fake photos\n",
    "paths_fake = [os.path.join(path_fake,i) for i in list_fake]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "a5430505",
   "metadata": {},
   "outputs": [],
   "source": [
    "arrays_real = []\n",
    "for p in paths_real:\n",
    "    img = Image.open(p)\n",
    "    array = np.asarray(img)\n",
    "    arrays_real.append(array)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71ebafba",
   "metadata": {},
   "source": [
    "In this part, I opened every pictures in the real picture folder, transformed it into an array and stored all the arrays in a list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "7abf6832",
   "metadata": {},
   "outputs": [],
   "source": [
    "arrays_fake = []\n",
    "for p in paths_fake:\n",
    "    img = Image.open(p)\n",
    "    array = np.asarray(img)\n",
    "    arrays_fake.append(array)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23b8c6b2",
   "metadata": {},
   "source": [
    "I did the same for fake images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "adac69b9",
   "metadata": {},
   "outputs": [],
   "source": [
    "img = arrays_real[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "4b98054c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(600, 600, 3)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# get the shape of images\n",
    "img.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42253f31",
   "metadata": {},
   "source": [
    "The images are in shape 600*600*3 , meaning length and width of 600 times the three color channels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "7e92d603",
   "metadata": {},
   "outputs": [],
   "source": [
    "# split the picture into its 3 channels\n",
    "blue,green,red = cv2.split(img)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6883bf20",
   "metadata": {},
   "source": [
    "Here I plotted the picture in its 3 channels red blue and green."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "85937fbf",
   "metadata": {},
   "source": [
    "## Data Pre-processing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "9092b9bd",
   "metadata": {},
   "outputs": [],
   "source": [
    "b_real = []\n",
    "g_real = []\n",
    "r_real = []\n",
    "for a in arrays_real:\n",
    "    blue, green, red = cv2.split(a)\n",
    "    b_real.append(blue)\n",
    "    g_real.append(green)\n",
    "    r_real.append(red)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "8ecbd3b9",
   "metadata": {},
   "outputs": [],
   "source": [
    "b_fake = []\n",
    "g_fake = []\n",
    "r_fake = []\n",
    "for a in arrays_fake:\n",
    "    blue, green, red = cv2.split(a)\n",
    "    b_fake.append(blue)\n",
    "    g_fake.append(green)\n",
    "    r_fake.append(red)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69d71ed9",
   "metadata": {},
   "source": [
    "In the above, I iterated through every picture array, split it into the three channels and stored each channels information into a list."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23b93c00",
   "metadata": {},
   "source": [
    "I then printed channel red for the real pictures and channel green for the fake pictures, the lists contains pixel data in the subsequant channel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "443d763b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.36470588, 0.34901961, 0.32156863, ..., 0.54509804, 0.53333333,\n",
       "        0.54509804],\n",
       "       [0.30980392, 0.30588235, 0.30980392, ..., 0.54117647, 0.5254902 ,\n",
       "        0.5372549 ],\n",
       "       [0.24313725, 0.2627451 , 0.29019608, ..., 0.51764706, 0.49803922,\n",
       "        0.51372549],\n",
       "       ...,\n",
       "       [0.41960784, 0.43529412, 0.45882353, ..., 0.1254902 , 0.12156863,\n",
       "        0.1254902 ],\n",
       "       [0.41176471, 0.42745098, 0.44705882, ..., 0.12156863, 0.12156863,\n",
       "        0.12941176],\n",
       "       [0.42745098, 0.44705882, 0.46666667, ..., 0.11764706, 0.12941176,\n",
       "        0.12941176]])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r_real[0] = r_real[0]/255\n",
    "r_real[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "905a040e",
   "metadata": {},
   "source": [
    "While experimenting with one picture, I first regularised the pixel data by deviding the array by 255."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "d09a2e85",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 3.16414896,  3.48667185,  2.68854441],\n",
       "       [ 3.18294632,  3.47764011,  2.65265939],\n",
       "       [ 3.18803467,  3.48657433,  2.60465319],\n",
       "       ...,\n",
       "       [-1.53686586,  2.2624265 , -1.47116219],\n",
       "       [-1.45086557,  2.31215672, -1.46733577],\n",
       "       [-1.37750035,  2.3594457 , -1.45624457]])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pca_3 = PCA(n_components=3)\n",
    "pca_3.fit_transform(r_real[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69121cb8",
   "metadata": {},
   "source": [
    "I then applied PCA with 3 components on the regularised array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "693e67c6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.24249098 0.19232581 0.14318063]\n"
     ]
    }
   ],
   "source": [
    "#  ration of variance -  identify how significant is each principal component \n",
    "print(pca_3.explained_variance_ratio_)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a43c939",
   "metadata": {},
   "source": [
    "Here I printed the eigenvalues of the first 3 components, we see that the first principal components only holds 0,24 of the information in the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "3e7d6929",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[39.44850311 35.13189002 30.31271671]\n"
     ]
    }
   ],
   "source": [
    "print(pca_3.singular_values_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "bd36a5e5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 3.16414896,  3.48667185,  2.68854441],\n",
       "       [ 3.18294632,  3.47764011,  2.65265939],\n",
       "       [ 3.18803467,  3.48657433,  2.60465319],\n",
       "       ...,\n",
       "       [-1.53686586,  2.2624265 , -1.47116219],\n",
       "       [-1.45086557,  2.31215672, -1.46733577],\n",
       "       [-1.37750035,  2.3594457 , -1.45624457]])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = pca_3.transform(r_real[0])\n",
    "t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "2a2def0f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.0047902 , -0.0059443 , -0.00633799, ...,  0.02503156,\n",
       "         0.02481148,  0.02476085],\n",
       "       [ 0.06456074,  0.06529843,  0.06761094, ...,  0.05572289,\n",
       "         0.05612572,  0.05663146],\n",
       "       [-0.05000346, -0.04721452, -0.04359481, ...,  0.04547275,\n",
       "         0.04415632,  0.0429455 ]])"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Get the principal components (eigenvectors)\n",
    "pca_3.components_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "1e762633",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.41953645])"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pca_1 = PCA(n_components = 1)\n",
    "pca_1.fit_transform(t)\n",
    "pca_1.explained_variance_ratio_"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14dd26d1",
   "metadata": {},
   "source": [
    "Then I applied a second PCA on my 3 principal components to try and reduce the picture into only one array. We see that the only principal components holds 50% of the information in the data, which is not bad considering how much dementiality reduction has been done."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "4fc0e8db",
   "metadata": {},
   "outputs": [],
   "source": [
    "pca_3 = PCA(n_components = 3)\n",
    "pca_1 = PCA(n_components = 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c52123a9",
   "metadata": {},
   "source": [
    "Here I created two PCAs one with 3 princiapl components and the second with 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "0a67b923",
   "metadata": {},
   "outputs": [],
   "source": [
    "features_real = []\n",
    "for a in r_real:\n",
    "    a = a/255\n",
    "    t = pca_3.fit_transform(a)    \n",
    "    tt = pca_1.fit_transform(t)    \n",
    "    features_real.append(tt)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba33f528",
   "metadata": {},
   "source": [
    "Choosing only the red channel, I regularised every picture array, transformed it with PCA and kept its three components, reapplied PCA and got only the one principal component."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "b4dc5a2e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Transform the features list into an array\n",
    "features_real = np.array(features_real)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "225aa9b1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1081, 600, 1)"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# get the shape of the array\n",
    "features_real.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "2919ed03",
   "metadata": {},
   "outputs": [],
   "source": [
    "# keep only two dimensions in the array\n",
    "features_real = np.squeeze(features_real)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "901cb6fb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.24084273e-02,  1.24821424e-02,  1.25020968e-02, ...,\n",
       "        -6.02692496e-03, -5.68966891e-03, -5.40196216e-03],\n",
       "       [ 7.30636659e+00,  7.21360264e+00,  7.02718064e+00, ...,\n",
       "         4.41739845e+00,  4.44166430e+00,  4.50502853e+00],\n",
       "       [ 5.31470470e+00,  5.30138942e+00,  5.31749393e+00, ...,\n",
       "         3.40083155e+00,  3.38134085e+00,  3.37459692e+00],\n",
       "       ...,\n",
       "       [-4.50300413e+00, -4.50300413e+00, -4.50300413e+00, ...,\n",
       "        -3.89060792e+00, -4.01996381e+00, -4.02535108e+00],\n",
       "       [ 4.48359948e+00,  4.46362139e+00,  4.46088475e+00, ...,\n",
       "        -2.15519021e+00, -2.14926006e+00, -2.14464395e+00],\n",
       "       [-4.26687830e+00, -4.26609212e+00, -4.25940654e+00, ...,\n",
       "        -1.46389829e+00, -1.35613416e+00, -1.35867978e+00]])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "features_real"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "139dacca",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Transform the array into a pandas dataframe\n",
    "df_real = pd.DataFrame(features_real)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "7f4546ae",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Adding the target column, 1 for real\n",
    "df_real[\"target\"] = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1487bff",
   "metadata": {},
   "source": [
    "In the above, I transformed every picture into one array line and stored it in a pandas dataframe, then added the target feature which is 1 for real pictures and 0 for fake pictures. Below I did the same for fake pictures."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "16353aa0",
   "metadata": {},
   "outputs": [],
   "source": [
    "features_fake = []\n",
    "for a in r_fake:\n",
    "    a = a/255\n",
    "    t = pca_3.fit_transform(a)    \n",
    "    tt = pca_1.fit_transform(t)    \n",
    "    features_fake.append(tt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "id": "81436349",
   "metadata": {},
   "outputs": [],
   "source": [
    "features_fake = np.array(features_fake)\n",
    "features_fake = np.squeeze(features_fake)\n",
    "df_fake = pd.DataFrame(features_fake)\n",
    "df_fake[\"target\"] = 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "2f80e2f1",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\imane\\AppData\\Local\\Temp/ipykernel_1504/1117223117.py:1: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.\n",
      "  df = df_real.append(df_fake)\n"
     ]
    }
   ],
   "source": [
    "df = df_real.append(df_fake)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d30d61a",
   "metadata": {},
   "source": [
    "I added the real pictures dataframe to the fake pictures dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "5fea0acc",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.sample(frac = 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdb53f3d",
   "metadata": {},
   "source": [
    "Then shuffled the rows with target 1 with eows with target 0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "7f4675bd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>...</th>\n",
       "      <th>591</th>\n",
       "      <th>592</th>\n",
       "      <th>593</th>\n",
       "      <th>594</th>\n",
       "      <th>595</th>\n",
       "      <th>596</th>\n",
       "      <th>597</th>\n",
       "      <th>598</th>\n",
       "      <th>599</th>\n",
       "      <th>target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>3.287063</td>\n",
       "      <td>3.342324</td>\n",
       "      <td>3.462696</td>\n",
       "      <td>3.608864</td>\n",
       "      <td>3.683111</td>\n",
       "      <td>3.764133</td>\n",
       "      <td>3.824438</td>\n",
       "      <td>3.902464</td>\n",
       "      <td>3.929437</td>\n",
       "      <td>4.014838</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.747367</td>\n",
       "      <td>-1.925979</td>\n",
       "      <td>-1.949774</td>\n",
       "      <td>-1.889547</td>\n",
       "      <td>-1.888047</td>\n",
       "      <td>-1.908365</td>\n",
       "      <td>-1.980823</td>\n",
       "      <td>-2.026748</td>\n",
       "      <td>-2.095880</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>309</th>\n",
       "      <td>1.237257</td>\n",
       "      <td>2.167152</td>\n",
       "      <td>2.047360</td>\n",
       "      <td>1.759997</td>\n",
       "      <td>1.946670</td>\n",
       "      <td>2.011273</td>\n",
       "      <td>1.801660</td>\n",
       "      <td>1.773356</td>\n",
       "      <td>1.499285</td>\n",
       "      <td>1.616764</td>\n",
       "      <td>...</td>\n",
       "      <td>6.913927</td>\n",
       "      <td>6.864207</td>\n",
       "      <td>6.892620</td>\n",
       "      <td>6.892296</td>\n",
       "      <td>6.969018</td>\n",
       "      <td>6.990944</td>\n",
       "      <td>6.954823</td>\n",
       "      <td>6.998709</td>\n",
       "      <td>7.072531</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>665</th>\n",
       "      <td>1.926135</td>\n",
       "      <td>1.997347</td>\n",
       "      <td>2.144765</td>\n",
       "      <td>2.255001</td>\n",
       "      <td>2.329303</td>\n",
       "      <td>2.377606</td>\n",
       "      <td>2.375842</td>\n",
       "      <td>2.345305</td>\n",
       "      <td>2.293009</td>\n",
       "      <td>2.263539</td>\n",
       "      <td>...</td>\n",
       "      <td>5.339002</td>\n",
       "      <td>5.384694</td>\n",
       "      <td>5.354743</td>\n",
       "      <td>5.339224</td>\n",
       "      <td>5.270135</td>\n",
       "      <td>5.219827</td>\n",
       "      <td>5.204131</td>\n",
       "      <td>5.161036</td>\n",
       "      <td>4.692395</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>711</th>\n",
       "      <td>-4.979350</td>\n",
       "      <td>-5.082941</td>\n",
       "      <td>-5.122791</td>\n",
       "      <td>-5.082695</td>\n",
       "      <td>-5.012335</td>\n",
       "      <td>-5.008449</td>\n",
       "      <td>-4.907021</td>\n",
       "      <td>-4.786197</td>\n",
       "      <td>-4.735909</td>\n",
       "      <td>-4.767556</td>\n",
       "      <td>...</td>\n",
       "      <td>2.610620</td>\n",
       "      <td>2.610986</td>\n",
       "      <td>2.603435</td>\n",
       "      <td>2.527036</td>\n",
       "      <td>2.500861</td>\n",
       "      <td>2.457423</td>\n",
       "      <td>2.458748</td>\n",
       "      <td>2.412879</td>\n",
       "      <td>2.325164</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>419</th>\n",
       "      <td>-2.551828</td>\n",
       "      <td>-2.557496</td>\n",
       "      <td>-2.560609</td>\n",
       "      <td>-2.568806</td>\n",
       "      <td>-2.569560</td>\n",
       "      <td>-2.567642</td>\n",
       "      <td>-2.569066</td>\n",
       "      <td>-2.556792</td>\n",
       "      <td>-2.562415</td>\n",
       "      <td>-2.544865</td>\n",
       "      <td>...</td>\n",
       "      <td>3.873017</td>\n",
       "      <td>3.866005</td>\n",
       "      <td>3.859453</td>\n",
       "      <td>3.853207</td>\n",
       "      <td>3.876146</td>\n",
       "      <td>3.923818</td>\n",
       "      <td>3.976807</td>\n",
       "      <td>4.068387</td>\n",
       "      <td>4.167744</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>212</th>\n",
       "      <td>5.366236</td>\n",
       "      <td>5.266136</td>\n",
       "      <td>5.150376</td>\n",
       "      <td>5.089555</td>\n",
       "      <td>5.065846</td>\n",
       "      <td>5.044894</td>\n",
       "      <td>5.025801</td>\n",
       "      <td>5.013602</td>\n",
       "      <td>4.936835</td>\n",
       "      <td>4.830000</td>\n",
       "      <td>...</td>\n",
       "      <td>5.404031</td>\n",
       "      <td>5.355861</td>\n",
       "      <td>5.346964</td>\n",
       "      <td>5.305434</td>\n",
       "      <td>5.273613</td>\n",
       "      <td>5.247769</td>\n",
       "      <td>5.221568</td>\n",
       "      <td>5.172945</td>\n",
       "      <td>5.146442</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>219</th>\n",
       "      <td>6.662252</td>\n",
       "      <td>6.675446</td>\n",
       "      <td>6.744643</td>\n",
       "      <td>6.847190</td>\n",
       "      <td>6.969999</td>\n",
       "      <td>7.052656</td>\n",
       "      <td>7.191046</td>\n",
       "      <td>7.309616</td>\n",
       "      <td>7.439942</td>\n",
       "      <td>7.561720</td>\n",
       "      <td>...</td>\n",
       "      <td>-2.124505</td>\n",
       "      <td>-2.125954</td>\n",
       "      <td>-2.116591</td>\n",
       "      <td>-2.107513</td>\n",
       "      <td>-2.109763</td>\n",
       "      <td>-2.109634</td>\n",
       "      <td>-2.124127</td>\n",
       "      <td>-2.146639</td>\n",
       "      <td>-2.155785</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>3.274436</td>\n",
       "      <td>3.290791</td>\n",
       "      <td>3.328279</td>\n",
       "      <td>3.375026</td>\n",
       "      <td>3.389203</td>\n",
       "      <td>3.391874</td>\n",
       "      <td>3.452699</td>\n",
       "      <td>3.542644</td>\n",
       "      <td>3.619321</td>\n",
       "      <td>3.685639</td>\n",
       "      <td>...</td>\n",
       "      <td>0.798853</td>\n",
       "      <td>0.799844</td>\n",
       "      <td>0.806401</td>\n",
       "      <td>0.817191</td>\n",
       "      <td>0.804972</td>\n",
       "      <td>0.778306</td>\n",
       "      <td>0.768206</td>\n",
       "      <td>0.772035</td>\n",
       "      <td>0.773125</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>105</th>\n",
       "      <td>4.694958</td>\n",
       "      <td>4.659804</td>\n",
       "      <td>4.658375</td>\n",
       "      <td>4.641747</td>\n",
       "      <td>4.642281</td>\n",
       "      <td>4.655392</td>\n",
       "      <td>4.682671</td>\n",
       "      <td>4.686828</td>\n",
       "      <td>4.716162</td>\n",
       "      <td>4.688738</td>\n",
       "      <td>...</td>\n",
       "      <td>1.524015</td>\n",
       "      <td>1.536951</td>\n",
       "      <td>1.523328</td>\n",
       "      <td>1.547623</td>\n",
       "      <td>1.544236</td>\n",
       "      <td>1.512267</td>\n",
       "      <td>1.496291</td>\n",
       "      <td>1.526792</td>\n",
       "      <td>1.533531</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>5.187797</td>\n",
       "      <td>5.087882</td>\n",
       "      <td>4.938285</td>\n",
       "      <td>5.031524</td>\n",
       "      <td>5.102392</td>\n",
       "      <td>5.088203</td>\n",
       "      <td>5.058500</td>\n",
       "      <td>5.214746</td>\n",
       "      <td>5.266651</td>\n",
       "      <td>5.381662</td>\n",
       "      <td>...</td>\n",
       "      <td>0.954750</td>\n",
       "      <td>0.913811</td>\n",
       "      <td>0.885019</td>\n",
       "      <td>0.736545</td>\n",
       "      <td>0.569546</td>\n",
       "      <td>0.523264</td>\n",
       "      <td>0.497791</td>\n",
       "      <td>0.469944</td>\n",
       "      <td>0.467775</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2041 rows × 601 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            0         1         2         3         4         5         6  \\\n",
       "28   3.287063  3.342324  3.462696  3.608864  3.683111  3.764133  3.824438   \n",
       "309  1.237257  2.167152  2.047360  1.759997  1.946670  2.011273  1.801660   \n",
       "665  1.926135  1.997347  2.144765  2.255001  2.329303  2.377606  2.375842   \n",
       "711 -4.979350 -5.082941 -5.122791 -5.082695 -5.012335 -5.008449 -4.907021   \n",
       "419 -2.551828 -2.557496 -2.560609 -2.568806 -2.569560 -2.567642 -2.569066   \n",
       "..        ...       ...       ...       ...       ...       ...       ...   \n",
       "212  5.366236  5.266136  5.150376  5.089555  5.065846  5.044894  5.025801   \n",
       "219  6.662252  6.675446  6.744643  6.847190  6.969999  7.052656  7.191046   \n",
       "98   3.274436  3.290791  3.328279  3.375026  3.389203  3.391874  3.452699   \n",
       "105  4.694958  4.659804  4.658375  4.641747  4.642281  4.655392  4.682671   \n",
       "43   5.187797  5.087882  4.938285  5.031524  5.102392  5.088203  5.058500   \n",
       "\n",
       "            7         8         9  ...       591       592       593  \\\n",
       "28   3.902464  3.929437  4.014838  ... -1.747367 -1.925979 -1.949774   \n",
       "309  1.773356  1.499285  1.616764  ...  6.913927  6.864207  6.892620   \n",
       "665  2.345305  2.293009  2.263539  ...  5.339002  5.384694  5.354743   \n",
       "711 -4.786197 -4.735909 -4.767556  ...  2.610620  2.610986  2.603435   \n",
       "419 -2.556792 -2.562415 -2.544865  ...  3.873017  3.866005  3.859453   \n",
       "..        ...       ...       ...  ...       ...       ...       ...   \n",
       "212  5.013602  4.936835  4.830000  ...  5.404031  5.355861  5.346964   \n",
       "219  7.309616  7.439942  7.561720  ... -2.124505 -2.125954 -2.116591   \n",
       "98   3.542644  3.619321  3.685639  ...  0.798853  0.799844  0.806401   \n",
       "105  4.686828  4.716162  4.688738  ...  1.524015  1.536951  1.523328   \n",
       "43   5.214746  5.266651  5.381662  ...  0.954750  0.913811  0.885019   \n",
       "\n",
       "          594       595       596       597       598       599  target  \n",
       "28  -1.889547 -1.888047 -1.908365 -1.980823 -2.026748 -2.095880       1  \n",
       "309  6.892296  6.969018  6.990944  6.954823  6.998709  7.072531       1  \n",
       "665  5.339224  5.270135  5.219827  5.204131  5.161036  4.692395       1  \n",
       "711  2.527036  2.500861  2.457423  2.458748  2.412879  2.325164       0  \n",
       "419  3.853207  3.876146  3.923818  3.976807  4.068387  4.167744       1  \n",
       "..        ...       ...       ...       ...       ...       ...     ...  \n",
       "212  5.305434  5.273613  5.247769  5.221568  5.172945  5.146442       0  \n",
       "219 -2.107513 -2.109763 -2.109634 -2.124127 -2.146639 -2.155785       0  \n",
       "98   0.817191  0.804972  0.778306  0.768206  0.772035  0.773125       1  \n",
       "105  1.547623  1.544236  1.512267  1.496291  1.526792  1.533531       0  \n",
       "43   0.736545  0.569546  0.523264  0.497791  0.469944  0.467775       0  \n",
       "\n",
       "[2041 rows x 601 columns]"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "c2163bfa",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "28     1\n",
       "309    1\n",
       "665    1\n",
       "711    0\n",
       "419    1\n",
       "      ..\n",
       "212    0\n",
       "219    0\n",
       "98     1\n",
       "105    0\n",
       "43     0\n",
       "Name: target, Length: 2041, dtype: int64"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Separate the target from the dataset\n",
    "target = df['target'].copy()\n",
    "target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "7c122ddd",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.drop(columns = [\"target\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "76c065ad",
   "metadata": {},
   "outputs": [],
   "source": [
    "# split the dataset for training and testing samples\n",
    "X_train, X_test, y_train, y_test = train_test_split( df, target, test_size=0.2, random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "601cf710",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<BarContainer object of 2 artists>"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAPWUlEQVR4nO3de4zlZ13H8ffHrtsKSLttx1q2ld2GKilEaN3UCgQDS0Ivhq0RsARkwTUrWhGsiSz2DwyJsUVjgWggmxZYlJRixXQVvJS2xBjs4hRKr5ROL9BdexlKW0XCpfD1j/MsPTvM7JztzJmz+/T9Sk7O83ue53fO9/zm7Gd+85zLpqqQJPXlxyZdgCRp+RnuktQhw12SOmS4S1KHDHdJ6tCqSRcAcOyxx9a6desmXYYkHVJuuOGGr1fV1HxjB0W4r1u3junp6UmXIUmHlCRfXWjMZRlJ6pDhLkkdMtwlqUOGuyR1yHCXpA4Z7pLUIcNdkjpkuEtShwx3SerQQfEJValn67Z9atIl6CB270XnjOV2PXOXpA4Z7pLUIcNdkjpkuEtShwx3SeqQ4S5JHTLcJalDhrskdchwl6QOGe6S1CHDXZI6NFK4J/mDJLcmuSXJ5UmOSLI+ya4kM0muSLK6zT28bc+08XVjfQSSpB+xaLgnWQv8PrChqp4PHAacB1wMXFJVzwEeAba0XbYAj7T+S9o8SdIKGnVZZhXwE0lWAU8D7gdeDlzZxncA57b2prZNG9+YJMtSrSRpJIuGe1XtAf4C+BqDUH8MuAF4tKoeb9N2A2tbey1wX9v38Tb/mLm3m2Rrkukk07Ozs0t9HJKkIaMsy6xhcDa+HngW8HTgzKXecVVtr6oNVbVhampqqTcnSRoyyrLMK4B7qmq2qr4HfBJ4MXBUW6YBOAHY09p7gBMB2viRwMPLWrUkab9GCfevAWckeVpbO98I3AZcB7y6zdkMXNXaO9s2bfzaqqrlK1mStJhR1tx3MXhh9AvAzW2f7cA7gAuSzDBYU7+s7XIZcEzrvwDYNoa6JUn7MdL/oVpV7wLeNaf7buD0eeZ+G3jN0kuTJD1ZfkJVkjpkuEtSh0ZaljmYrdv2qUmXoIPYvRedM+kSpInwzF2SOmS4S1KHDHdJ6pDhLkkdMtwlqUOGuyR1yHCXpA4Z7pLUIcNdkjpkuEtShwx3SeqQ4S5JHTLcJalDhrskdchwl6QOGe6S1CHDXZI6ZLhLUocMd0nqkOEuSR0y3CWpQ4a7JHXIcJekDhnuktQhw12SOmS4S1KHDHdJ6pDhLkkdMtwlqUOGuyR1yHCXpA4Z7pLUIcNdkjpkuEtShwx3SeqQ4S5JHRop3JMcleTKJF9OcnuSX0pydJKrk9zZrte0uUny/iQzSW5Kctp4H4Ikaa5Rz9zfB/xLVT0XeAFwO7ANuKaqTgauadsAZwEnt8tW4APLWrEkaVGLhnuSI4GXApcBVNV3q+pRYBOwo03bAZzb2puAj9bA9cBRSY5f5rolSfsxypn7emAW+HCSLya5NMnTgeOq6v425wHguNZeC9w3tP/u1rePJFuTTCeZnp2dffKPQJL0I0YJ91XAacAHqupU4P94YgkGgKoqoA7kjqtqe1VtqKoNU1NTB7KrJGkRo4T7bmB3Ve1q21cyCPsH9y63tOuH2vge4MSh/U9ofZKkFbJouFfVA8B9SX6udW0EbgN2Aptb32bgqtbeCbyxvWvmDOCxoeUbSdIKWDXivLcCH0uyGrgbeDODXwyfSLIF+Crw2jb308DZwAzwrTZXkrSCRgr3qroR2DDP0MZ55hZw/tLKkiQthZ9QlaQOGe6S1CHDXZI6ZLhLUocMd0nqkOEuSR0y3CWpQ4a7JHXIcJekDhnuktQhw12SOmS4S1KHDHdJ6pDhLkkdMtwlqUOGuyR1yHCXpA4Z7pLUIcNdkjpkuEtShwx3SeqQ4S5JHTLcJalDhrskdchwl6QOGe6S1CHDXZI6ZLhLUocMd0nqkOEuSR0y3CWpQ4a7JHXIcJekDhnuktQhw12SOmS4S1KHDHdJ6pDhLkkdMtwlqUMjh3uSw5J8Mck/te31SXYlmUlyRZLVrf/wtj3TxteNqXZJ0gIO5Mz9bcDtQ9sXA5dU1XOAR4AtrX8L8Ejrv6TNkyStoJHCPckJwDnApW07wMuBK9uUHcC5rb2pbdPGN7b5kqQVMuqZ+3uBPwJ+0LaPAR6tqsfb9m5gbWuvBe4DaOOPtfn7SLI1yXSS6dnZ2SdXvSRpXouGe5JfAR6qqhuW846rantVbaiqDVNTU8t505L0lLdqhDkvBl6V5GzgCOCZwPuAo5KsamfnJwB72vw9wInA7iSrgCOBh5e9cknSghY9c6+qd1bVCVW1DjgPuLaqXg9cB7y6TdsMXNXaO9s2bfzaqqplrVqStF9LeZ/7O4ALkswwWFO/rPVfBhzT+i8Ati2tREnSgRplWeaHquqzwGdb+27g9HnmfBt4zTLUJkl6kvyEqiR1yHCXpA4Z7pLUIcNdkjpkuEtShwx3SeqQ4S5JHTLcJalDhrskdchwl6QOGe6S1CHDXZI6ZLhLUocMd0nqkOEuSR0y3CWpQ4a7JHXIcJekDhnuktQhw12SOmS4S1KHDHdJ6pDhLkkdMtwlqUOGuyR1yHCXpA4Z7pLUIcNdkjpkuEtShwx3SeqQ4S5JHTLcJalDhrskdchwl6QOGe6S1CHDXZI6ZLhLUocMd0nqkOEuSR1aNNyTnJjkuiS3Jbk1ydta/9FJrk5yZ7te0/qT5P1JZpLclOS0cT8ISdK+Rjlzfxz4w6o6BTgDOD/JKcA24JqqOhm4pm0DnAWc3C5bgQ8se9WSpP1aNNyr6v6q+kJr/y9wO7AW2ATsaNN2AOe29ibgozVwPXBUkuOXu3BJ0sIOaM09yTrgVGAXcFxV3d+GHgCOa+21wH1Du+1ufXNva2uS6STTs7OzB1q3JGk/Rg73JM8A/h54e1X9z/BYVRVQB3LHVbW9qjZU1YapqakD2VWStIiRwj3JjzMI9o9V1Sdb94N7l1va9UOtfw9w4tDuJ7Q+SdIKGeXdMgEuA26vqr8cGtoJbG7tzcBVQ/1vbO+aOQN4bGj5RpK0AlaNMOfFwG8ANye5sfX9MXAR8IkkW4CvAq9tY58GzgZmgG8Bb17OgiVJi1s03KvqP4AsMLxxnvkFnL/EuiRJS+AnVCWpQ4a7JHXIcJekDhnuktQhw12SOmS4S1KHDHdJ6pDhLkkdMtwlqUOGuyR1yHCXpA4Z7pLUIcNdkjpkuEtShwx3SeqQ4S5JHTLcJalDhrskdchwl6QOGe6S1CHDXZI6ZLhLUocMd0nqkOEuSR0y3CWpQ4a7JHXIcJekDhnuktQhw12SOmS4S1KHDHdJ6pDhLkkdMtwlqUOGuyR1yHCXpA4Z7pLUIcNdkjpkuEtShwx3SerQWMI9yZlJ7kgyk2TbOO5DkrSwZQ/3JIcBfw2cBZwCvC7JKct9P5KkhY3jzP10YKaq7q6q7wIfBzaN4X4kSQtYNYbbXAvcN7S9G/jFuZOSbAW2ts1vJrljDLUsp2OBr0+6iBFY55BcvOSbOFSOJxw6tVrnkCU+R5+90MA4wn0kVbUd2D6p+z9QSaarasOk61iMdS6vQ6VOOHRqtc6VMY5lmT3AiUPbJ7Q+SdIKGUe4/xdwcpL1SVYD5wE7x3A/kqQFLPuyTFU9nuT3gH8FDgM+VFW3Lvf9TMChsoRkncvrUKkTDp1arXMFpKomXYMkaZn5CVVJ6pDhLkkdMtybJEcnuTrJne16zTxzXpjkP5PcmuSmJL8+NPaRJPckubFdXjiGGvf7tQ5JDk9yRRvflWTd0Ng7W/8dSV653LUdYJ0XJLmtHcNrkjx7aOz7Q8dwrC/Ej1Dnm5LMDtXzW0Njm9tz5c4kmydc5yVDNX4lyaNDYyt5PD+U5KEktywwniTvb4/jpiSnDY2t5PFcrM7Xt/puTvK5JC8YGru39d+YZHqcdS5ZVXkZvO7wHmBba28DLp5nzs8CJ7f2s4D7gaPa9keAV4+xvsOAu4CTgNXAl4BT5sz5XeCDrX0ecEVrn9LmHw6sb7dz2ATrfBnwtNb+nb11tu1vrtDPe5Q63wT81Tz7Hg3c3a7XtPaaSdU5Z/5bGbyJYUWPZ7uvlwKnAbcsMH428M9AgDOAXSt9PEes80V775/B16jsGhq7Fzh2pY7pUi6euT9hE7CjtXcA586dUFVfqao7W/u/gYeAqRWqb5SvdRh+DFcCG5Ok9X+8qr5TVfcAM+32JlJnVV1XVd9qm9cz+CzESlvK12S8Eri6qr5RVY8AVwNnHiR1vg64fEy17FdV/Tvwjf1M2QR8tAauB45KcjwrezwXrbOqPtfqgMk9P5fMcH/CcVV1f2s/ABy3v8lJTmdwJnXXUPeftj/nLkly+DLXN9/XOqxdaE5VPQ48Bhwz4r4rWeewLQzO5vY6Isl0kuuTnDuG+vYatc5faz/TK5Ps/XDeQXk82/LWeuDaoe6VOp6jWOixrOTxPFBzn58F/FuSG9pXqBy0Jvb1A5OQ5DPAT88zdOHwRlVVkgXfI9rONv4G2FxVP2jd72TwS2E1g/fHvgN493LU3askbwA2AL881P3sqtqT5CTg2iQ3V9Vd89/C2P0jcHlVfSfJbzP4q+jlE6plFOcBV1bV94f6DqbjeUhJ8jIG4f6Soe6XtOP5U8DVSb7c/hI46Dylztyr6hVV9fx5LlcBD7bQ3hveD813G0meCXwKuLD9abn3tu9vf25+B/gwy7/sMcrXOvxwTpJVwJHAwyPuu5J1kuQVDH6pvqodMwCqak+7vhv4LHDqpOqsqoeHarsU+IVR913JOoecx5wlmRU8nqNY6LEcdF9ZkuTnGfzMN1XVw3v7h47nQ8A/ML7lzaWb9KL/wXIB/px9X1B9zzxzVgPXAG+fZ+z4dh3gvcBFy1zfKgYvNK3niRfWnjdnzvns+4LqJ1r7eez7gurdjO8F1VHqPJXBctbJc/rXAIe39rHAneznxcMVqPP4ofavAte39tHAPa3eNa199KTqbPOey+DFvkzieA7d5zoWfqHyHPZ9QfXzK308R6zzZxi8LvWiOf1PB35yqP054Mxx1rmkxzjpAg6WC4O16WvaP4DP7H1yMVg2uLS13wB8D7hx6PLCNnYtcDNwC/C3wDPGUOPZwFdaMF7Y+t7N4OwX4Ajg79oT8/PASUP7Xtj2uwM4a8zHcrE6PwM8OHQMd7b+F7Vj+KV2vWXCdf4ZcGur5zrguUP7/mY7zjPAmydZZ9v+E+acUEzgeF7O4B1k32Owbr4FeAvwljYeBv+Rz12tng0TOp6L1Xkp8MjQ83O69Z/UjuWX2vPiwnHWudSLXz8gSR16Sq25S9JTheEuSR0y3CWpQ4a7JHXIcJekDhnuktQhw12SOvT/jnL5PIoRhf0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "cat_train = pd.DataFrame(y_train)\n",
    "cat_train[\"count\"] = 1\n",
    "cat_train = cat_train.groupby(\"target\").sum().reset_index()\n",
    "x = cat_train[\"target\"]\n",
    "y = cat_train[\"count\"]\n",
    "plt.bar(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b8aa218",
   "metadata": {},
   "source": [
    "Here I just wanted to make sure that the amount of real photos in the training set is not too far from the number of fake photos."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80017339",
   "metadata": {},
   "source": [
    "## Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "ee4746d9",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\model_selection\\_validation.py:372: FitFailedWarning: \n",
      "70 fits failed out of a total of 140.\n",
      "The score on these train-test partitions for these parameters will be set to nan.\n",
      "If these failures are not expected, you can try to debug them by setting error_score='raise'.\n",
      "\n",
      "Below are more details about the failures:\n",
      "--------------------------------------------------------------------------------\n",
      "70 fits failed with the following error:\n",
      "Traceback (most recent call last):\n",
      "  File \"c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\model_selection\\_validation.py\", line 681, in _fit_and_score\n",
      "    estimator.fit(X_train, y_train, **fit_params)\n",
      "  File \"c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py\", line 1461, in fit\n",
      "    solver = _check_solver(self.solver, self.penalty, self.dual)\n",
      "  File \"c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py\", line 447, in _check_solver\n",
      "    raise ValueError(\n",
      "ValueError: Solver lbfgs supports only 'l2' or 'none' penalties, got l1 penalty.\n",
      "\n",
      "  warnings.warn(some_fits_failed_message, FitFailedWarning)\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\model_selection\\_search.py:969: UserWarning: One or more of the test scores are non-finite: [       nan 0.50797546        nan 0.49142227        nan 0.48591202\n",
      "        nan 0.48346177        nan 0.4902252         nan 0.48162502\n",
      "        nan 0.47857624]\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=10, estimator=LogisticRegression(),\n",
       "             param_grid={'C': array([1.e-03, 1.e-02, 1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03]),\n",
       "                         'penalty': ['l1', 'l2']})"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid={\"C\":np.logspace(-3,3,7), \"penalty\":[\"l1\",\"l2\"]}# l1 lasso l2 ridge\n",
    "logreg = LogisticRegression()\n",
    "logreg_cv = GridSearchCV(logreg,grid,cv=10)\n",
    "logreg_cv.fit(X_train,y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7161642",
   "metadata": {},
   "source": [
    "running the grid search to find best parameters for the base line model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "8cfe43b4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'C': 0.001, 'penalty': 'l2'}"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Get best parameters\n",
    "logreg_cv.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "07a7c59c",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=0.001)"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Fit the data\n",
    "lr = LogisticRegression(C= 0.001, penalty='l2')\n",
    "lr.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "78ff7ca4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5594362745098039"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# accuracy on training\n",
    "lr.score(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "291a568d",
   "metadata": {},
   "outputs": [],
   "source": [
    "lr_pred = lr.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "b0aefc8f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5061124694376528"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Accuracy on test set\n",
    "acc_lr = lr.score(X_test, y_test)\n",
    "acc_lr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "e02a9563",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.61003861003861"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "f1_lr = f1_score(y_test, lr_pred)\n",
    "f1_lr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "aad09976",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.46      0.25      0.33       193\n",
      "           1       0.52      0.73      0.61       216\n",
      "\n",
      "    accuracy                           0.51       409\n",
      "   macro avg       0.49      0.49      0.47       409\n",
      "weighted avg       0.49      0.51      0.48       409\n",
      "\n"
     ]
    }
   ],
   "source": [
    "cr_lr = classification_report(y_test,lr_pred )\n",
    "print(cr_lr)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "67d9c02b",
   "metadata": {},
   "source": [
    "The classification report shows a higher than average job done on real pictures, although the precision and recall are better for real picrures, the classifier can make some correct predictions of fake pictures.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "1348e007",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfYAAAHSCAYAAAAe1umcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAWJUlEQVR4nO3cebCldZ3f8c+3u2mgWRqafVExI4qOEKIoOCMMuI1LLHVEFMrSmcKAJgODC45JKsZJhlSh1kzCkDgCUhQucWK5xB0hooALgyA0yJTICCLNTrMIDTTd/csf96BN0/TGcvt+eb2qujj3d57znO+91HPe93nO6a4xRgCAHmZN9wAAwONH2AGgEWEHgEaEHQAaEXYAaETYAaCROdM9wJNtbm06NssW0z0GtLXD8x+Y7hGgvV9eseS2McYOq7vvKRf2zbJF9q+XT/cY0NZRX/7ldI8A7R225yW/erT7XIoHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdjY6s8bIJ8Y5+a/jgiTJvuOW/K9xTk4Z38nx46LMGiumeUKYWfb70K/y+v0X5lWvvfIR9z37UzfnLXtekrmLlz1sfduF9+bNe12S3b51x5M1Jo+TdQp7Vb2xqkZV7bUO2x5XVfM2dKCq+tOqOnk161VVJ1XV1VW1sKpesKHPwcbtTflFrstWSZIaI8fnopyQ/XNUvSq3ZF5elV9N84Qws1z7Jwty/unPesT65jcuzU4X3J17d5378DuWj+zzsUW5+aVbP0kT8nha1zP2w5NcMPnv2hyXZIPDvgavSbLn5M9RST7xBDwH02z7sST758Z8K89MkmydpVmWWVlUU6G/ODvmwCyazhFhxrntxVtl6fzZj1jf94Trs/CDuyX18PU9z7w11//xtnlgwZwnaUIeT2sNe1VtmeSlSY5M8raV1mdX1cer6orJGfQxVXVskl2TnFtV5062u2elxxxaVWdMbr++qi6sqp9W1TlVtdNaRnlDkjPHlB8n2aaqdpn8Oa+qLp3McuB6/gzYiLwnl+XU7JOHLrbflbmZnZFnj8VJkoOyKDtkyfQNCE3ses6duW+nTXLXcx9+HrbZTUuz29l35p+P2H6aJuOxWpdfx96Q5NtjjKuq6vaqeuEY4+JMnTXvkWTfMcayqlowxlhcVe9LcsgY47a17PeCJAeMMUZVvSvJB5O8fw3b75bk1yt9ff1k7Y+SnDXGOKGqZueJuVrAk2D/cUPuzKb5RW2bfcYtU4tVOWHsn3fnsmwyVuTi7JQVq55eAOtl9n0rstcnbsp5Z+z5iPv2PeH6LDx+t2SW42ymWpewH57kf0xuf37y9cVJXpHk78cYy5JkjMkp1brbPck/VNUuSeYmuWY9H/+Qi5KcXlWbJPnKGOPSVTeoqqMy9YtINtP9jdbv5/a8JDfmxeObmZvlmZdl+cvxjzmxXpz35ZAkyQvHTdk996xlT8CabHHdA9ni+qV51ev/KUmy+U1L88o3/lPO+eJeWXDFkhzw3qmX403vWJadv393xpzKDa/cZhonZn2sMexVtSDJy5LsXVUjyewko6qOX4/nGCvd3myl23+X5G/GGF+tqoOTfGQt+1mU5Gkrfb17kkVjjBur6qAkr0tyRlX9zRjjzIcNMMYpSU5Jkq1rwcrzsBE5vfbO6dk7SbLPuCVvyVU5sV6cbcb9ubM2yyZjed6an+dzee40Twoz293P2Txfu3Cf33792oOvyDlf2itLF8zJN899/m/XX/TBa3PDIfNFfYZZ23vshyb59BjjGWOMPcYYT8vUmfWBSc5OcnRVzUl++0tAkvwmmXykecrNVfXcqpqV5E0rrc9PfvspqHeuw6xfTfKOyafjD0hy1yTqz0hy8xjj1CSnJfFp+WbekqvyqXFWPpmz86Psmktrx+keCWaU/Y+7Ji877OfZ6pr787qXXp49vrC2d0qZydZ2Kf7wJCeusvbFyfoxSZ6dZGFVPZjk1CQnZ+rM+NtVdcMY45AkH0ry9SS3JvlJki0n+/lIki9U1R1JvptMPgb96L6Z5LVJrk6yJMmfTdYPTnL8ZIZ7krxjLfthBlhYO2ZhpgJ+au2TU7PPWh4BPJoL//uaX16/+b3nr3b9oo/u8QRMwxOtxnhqXZneuhaM/evl0z0GtHXUVb+c7hGgvcP2vOTiMcZ+q7vPvzwHAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQyJzpHuDJVnM3yZydd5/uMaCtN2956XSPAE9pztgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAamTPdA8Cqzlj0qSyZNTcrUlles/IXOx+Rf7H0lhyz+LvZZCzL8pqV/7nty3LVpjtP96gwI9R7b07OXpJsPzvje0+fWvv47cln7062m50kGf9+u+TlWyQPjtT7b0kufyBZNjLeslVy7ILpHJ/1tE5n7FX1xqoaVbXXOmx7XFXN29CBqupPq+rk1azvVVU/qqoHquoDG7p/ZoYP7Xho/nyXt+cvdj4iSXLknRfks/P3z5/v8vZ8Zv5LcuSd50/zhDBzjMO2zvjcLo9cP2qbjHOennHO06einiRfuydZOjLOfXrGWU9Lffru5NcPPskT81is66X4w5NcMPnv2hyXZIPDvgaLkxyb5ONPwL7ZyI0k81YsTZLMW/FAbp+95fQOBDPJSzZPtp29bttWkiUrkmUjuX8kcyvZ0ru2M8la/29V1ZZJXprkyCRvW2l9dlV9vKquqKqFVXVMVR2bZNck51bVuZPt7lnpMYdW1RmT26+vqgur6qdVdU5V7bSmOcYYt4wxLkrysF8dq2qLqvpGVV02meWt6/zds1EaqZxwy5dy0o2fy2vuuTxJ8sltD86Rd16QMxedlnfdeX7O2OYPp3dIaKBOvyv1suumLtXfuXxq8V9vmcyblfqX16T2uzbj3dus+y8FbBTW5T32NyT59hjjqqq6vapeOMa4OMlRSfZIsu8YY1lVLRhjLK6q9yU5ZIxx21r2e0GSA8YYo6releSDSd6/Ad/Dq5PcMMZ4XZJU1fwN2AcbkQ/sdFhun7Nl5i9fkv92y5fy6znb5qVLrs4p2x6UH8zbMwfee1WOu/3s/Ied3jzdo8KMNd45P3nvgqSSOnFx6q9uy/jbnZKf3p/MSsalz0zuWp5646KMg+Ylz9hkukdmHa3L9ZXDk3x+cvvz+d3l+Fck+eQYY1mSjDEWr+dz757krKq6PMnxSX5/PR//kMuTvLKqTqyqA8cYd626QVUdVVU/qaqfLF1+3wY+DU+W2+dMXWa/a/a8/HDz38tzlt6cV9x7ZX6w+bOSJOfP2zPPWXrzdI4IM98Oc5LZlcyqjLdvnfz0gSRJffmejEPmJZtUsv2c5EWbJZfdP83Dsj7WGPaqWpDkZUlOq6prMxXgw6qq1uM5xkq3N1vp9t8lOXmMsXeSo1e5b913PsZVSV6QqcD/dVV9eDXbnDLG2G+Msd/c2ZtvyNPwJNl0xYPZfPJe+qYrHswL7r8u126yXW6fvUX2fuD6JMm+D/w6i+ZsM41TQgM3L/vd7W/em+w1N0kydpuT+sHkBGjJiuTi+5NnzZ2GAdlQa7sUf2iST48xjn5ooaq+n+TAJGcnObqqzl35UnyS3yTZKslDl+JvrqrnJvl5kjdN7k+S+UkWTW6/c0O/garaNcniMcZnqurOJO/a0H0x/bZdsST/6davJUlmZ0W+N2+vXLz5HjmpNsnRd3w/s7MiS2t2Ttru5dM8Kcwc9Z6bkh/elyxennrBNRkf2C71w/uSnz0w9WG5p83J+OiOUxv/2fzkuJtTf3RdMkbG27ZOnrfptM7P+llb2A9PcuIqa1+crB+T5NlJFlbVg0lOTXJyklOSfLuqbhhjHJLkQ0m+nuTWJD9J8tDHmT+S5AtVdUeS7yZ55poGqaqdJ4/fOsmKqjouyfOS7J3kY1W1IlMfrHvPWr4nNmI3zZmff7fL2x+x/rPNdsuxuxwxDRPBzDc+8ch/82EcsfXqN95iVsapj/yrccwcNcZY+1aNzN90p/EHOwsEPFG+ceHXp3sEaG/2LldfPMbYb3X3+cuJANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADRSY4zpnuFJVVW3JvnVdM/Betk+yW3TPQQ05zibWZ4xxthhdXc85cLOzFNVPxlj7Dfdc0BnjrM+XIoHgEaEHQAaEXZmglOmewB4CnCcNeE9dgBoxBk7ADQi7KyzqlpeVZdW1RVV9YWqmvcY9nVGVR06uX1aVT1vDdseXFV/sAHPcW1Vbb+a9RdW1eVVdXVVnVRVtb77hidKo+PshKr6dVXds7775LERdtbHfWOMfccYz0+yNMm7V76zquZsyE7HGO8aY1y5hk0OTrLeLzhr8Ikk/ybJnpM/r34c9w2PVZfj7GtJXvw47o91JOxsqPOTPGvyW/75VfXVJFdW1eyq+lhVXVRVC6vq6CSpKSdX1c+r6pwkOz60o6r6XlXtN7n96qq6pKouq6r/V1V7ZOqF7b2Ts5gDq2qHqvri5Dkuqqo/nDx2u6r6TlX9rKpOS/KIM/Gq2iXJ1mOMH4+pD5icmeSNk/uOraorJ3N//gn82cG6mpHHWZJMjrEbV12vqrdMrkZcVlXnPc4/L5Js0G9+PLVNzhhek+Tbk6UXJHn+GOOaqjoqyV1jjBdV1aZJflBV30nyr5I8J8nzkuyU5Mokp6+y3x2SnJrkoMm+FowxFlfV3ye5Z4zx8cl2n0vyt2OMC6rq6UnOSvLcJP85yQVjjP9SVa9LcuRqxt8tyfUrfX39ZC1JPpTkmWOMB6pqmw3/CcFjN8OPszX5cJI/HmMscpw9MYSd9bF5VV06uX1+kk9l6tLdP44xrpmsvyrJPg+9r5dkfqYudx+U5H+PMZYnuaGqvrua/R+Q5LyH9jXGWPwoc7wiyfNWemt866racvIcfzJ57Deq6o71/P4WJvlsVX0lyVfW87HweOl+nP0gyRlV9X+SfGk9H8s6EHbWx31jjH1XXpgc9PeuvJTkmDHGWats99rHcY5ZSQ4YY9y/mlnWZlGS3Vf6evfJWpK8LlMvWq9P8h+rau8xxrLHPi6slw7H2aMaY7y7qvbP1PF2cVW9cIxx+2PaKQ/jPXYeb2cleU9VbZIkVfXsqtoiyXlJ3jp5b3CXJIes5rE/TnJQVT1z8tgFk/XfJNlqpe2+k+SYh76oqn0nN89LcsRk7TVJtl31CSbv+d1dVQfU1CvUO5L836qaleRpY4xzk/xlps6AttyA7x+eDBv1cbYmVfV7Y4wLxxgfTnJrkqetz+NZO2Hn8XZapt7Xu6SqrkjyyUxdGfpykl9M7jszyY9WfeAY49YkRyX5UlVdluQfJnd9LcmbHvpQT5Jjk+w3+dDQlfndp4b/KlMvWD/L1KXC6x5lxn87mfPqJP+c5FtJZif5TFVdnuSnSU4aY9y5wT8FeGJt9MdZVX20qq5PMq+qrq+qj0zu+lhN/XXTK5L8MMllj+UHwSP5l+cAoBFn7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA08v8BskK+XEOKgcQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 576x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "cm_lr = confusion_matrix(y_test,lr_pred)\n",
    "fig, ax = plt.subplots(figsize=(8, 8))\n",
    "ax.imshow(cm_lr)\n",
    "ax.grid(False)# defining parameter range\n",
    "ax.xaxis.set(ticks=(0, 1), ticklabels=('Predicted 0s', 'Predicted 1s'))\n",
    "ax.yaxis.set(ticks=(0, 1), ticklabels=('Actual 0s', 'Actual 1s'))\n",
    "ax.set_ylim(1.5, -0.5)\n",
    "for i in range(2):\n",
    "    for j in range(2):\n",
    "        ax.text(j, i, cm_lr[i, j], ha='center', va='center', color='red')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42273659",
   "metadata": {},
   "source": [
    "The model has done a fairly better job at detecting real pictures than detecting fake ones.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2325a2d",
   "metadata": {},
   "source": [
    "## Naive Bayes Classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "408ed27e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 100 candidates, totalling 500 fits\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'var_smoothing': 0.15199110829529336}"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nb_classifier = GaussianNB()\n",
    "\n",
    "params_NB = {'var_smoothing': np.logspace(0,-9, num=100)}\n",
    "gs_NB = GridSearchCV(estimator=nb_classifier, \n",
    "                 param_grid=params_NB, \n",
    "                 verbose=1, \n",
    "                 scoring='accuracy') \n",
    "gs_NB.fit(X_train, y_train)\n",
    "\n",
    "gs_NB.best_params_"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2cc46a64",
   "metadata": {},
   "source": [
    "Performing a quick grid search to determine best params."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "id": "1ac74ecc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5502450980392157"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nb = GaussianNB(var_smoothing= 0.15)\n",
    "nb.fit(X_train, y_train)\n",
    "nb.score(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "id": "16c51a4e",
   "metadata": {},
   "outputs": [],
   "source": [
    "nb_pred = nb.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "id": "d98e4f80",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5281173594132029"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "acc_nb = nb.score(X_test, y_test)\n",
    "acc_nb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "id": "ba233973",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5720620842572062"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "f1_nb = f1_score(y_test, nb_pred)\n",
    "f1_nb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "id": "93de36e4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.50      0.45      0.47       193\n",
      "           1       0.55      0.60      0.57       216\n",
      "\n",
      "    accuracy                           0.53       409\n",
      "   macro avg       0.52      0.52      0.52       409\n",
      "weighted avg       0.53      0.53      0.53       409\n",
      "\n"
     ]
    }
   ],
   "source": [
    "cr_nb = classification_report(y_test,nb_pred )\n",
    "print(cr_nb)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9c44e25",
   "metadata": {},
   "source": [
    "The results of Naive Bayes are more balances then those of Logistic regression, although the accuracy droped a bit with NB, its F1 score, ie average of precision and recall, have balances out."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "id": "8d60a12b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfYAAAHSCAYAAAAe1umcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAWM0lEQVR4nO3cebBmdZ3f8c+3u6Gh6WZHRZCBERxFUcBmZIwyIq44RCTEcZlRGBXUCHFDZypVxlgzlTBazohU3EhiUIwzE4lxiaAsiogLICBLBhRFBGRrkKWhpZdf/rhPx6ZteoPmdn99vaq67nPPec55vs+te573Pee5t2uMEQCghxnTPQAA8MgRdgBoRNgBoBFhB4BGhB0AGhF2AGhk1nQP8GjbfOaWY8tZ20z3GNDWosdsNt0jQHsP3HDD7WOMnVa17ncu7FvO2ibP3uW10z0GtHXNW3eZ7hGgvZ+e8K6fP9Q6l+IBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEZmTfcAsKLD77o4L7nniowk122+Yz6844uzuGbm9Xd+J89ZeE2W1Yx8dd7T86Vt9p/uUWGT8Z/+8fN5/lX/Nwvmzs1L331CkmSb++7LSZ89NbveeWdu2G67HPdnr8vdc+bkTd88N//yhz9MksxatixPvPWWHPD+D+SuOXOm8ymwDtbqjL2qDq+qUVVPXov7vr2q1vs7oKqOqqqTV7G8quqkqvpJVf2oqryyN7PDknvy8rsvyfGPf03esuvrMyMjf7zw6rzw3iuz49J7csyuR+fYXY/Kt+au8dsQWMEX5h+Qo9/4pgcte/M5Z+eCPffKIe/9q1yw515587nnJEk+9byDc9g735XD3vmufPDQQ/OD33+iqG9i1vZS/KuTnD/5uCZvT7IhvgtemmSvyb9jknxsAzwG02zmWJbNx5LMGMsye9ni3DFzq7zs7svyuW0PzKhKktw104sMrIsLf/+J+dVKcX7BVVfm9PkHJElOn39AXnjlFb+13WGXXJIv77ffozIjj5w1hr2q5iZ5TpI3JHnVCstnVtWHquqKyRn0cVV1fJLHJzm3qs6d3O/eFbY5sqo+Pbl9WFV9v6ouqaqzquqxaxjl5UlOHVO+l2Tbqtp58u+8qrp0Mstz1/FrwEZiwax5+cI283PqL07J567/RO6bMTs/nLN7dl5yV/544TX5yI2n5QM3n57HL75zukeFTd6O99yT27beOkly27x52fGeex60fosHHshBV/9zztjn6dMxHg/D2pyxvzzJGWOMa5IsqKpnTpYfk2T3JPuOMZ6e5LQxxklJbkpy8Bjj4DXs9/wkB44x9kvy+STvWcP9d0nyixU+v2Gy7DVJzhxj7JvkGUkuXYvnxEZo7tJFOfC+a3P0E96Q1+52TGaPxTn43quy2ViaB2pm/u0ur80Z8/bJO277+nSPCr1U/f8rYssdctWVuXj3PVyG3wStTdhfnanwZvJx+eX4FyT5xBhjSZKMMe5Yx8feNcmZVXV5khOSPHUdt1/uwiRHV9X7k+wzxrhn5TtU1TFVdVFVXfTA0vvW82HY0PZddH1umbV17po5J0trZi6Ys1f2XvTL3D5rbr4zZ68kyQVz9sweD9w2zZPCpu/2efOy0913J0l2uvvuLJg790Hr/+TSS12G30StNuxVtX2S5yc5paquy1SAX1m10o92qzdWuL3FCrc/muTkMcY+SY5dad2q3JjkCSt8vmuSG8cY5yU5aLL+01X1ut8aYIxPjjHmjzHmb+792Y3WbTPn5cm/vjmzly1Oxsi+i67PLzbbPt+ds2eesWjqYs0+i27IjZttN82Twqbv7L2fmiMuujBJcsRFF+asvX9zbjX3/vvzhz+9Nmc9dX3Pt5hOa/pztyOTfGaMcezyBVX1rSTPTfKNJMdW1bljjCVVtf3krP2eJPOS3D7Z5JaqekqSq5O8YrI+SbbJVIyT5PVrMeuXkrytqj6f5FlJ7hpj/LKqfi/JDWOMT1XV7CT7Jzl1LfbHRubqLXbO+VvtlY/e9NkszYxcu/lj8rWt98nmy5bkPbd9LYffdXEWzdg8f7/ji6Z7VNik/P1pn8mzrr022y1cmPP/+gP5yItenI8f/Px89LOn5pUX/iA3brtdjvvz35wTvfiKy3P+k/4g928+exqnZn3VGOOhV079AtyJY4wzVlh2fJKnJDkuyd8meUmSxUk+NcY4uaqOS/K2JDeNMQ6uqiOTnJjktiQXJZk7xjiqql6e5O+S3JnknCQHjDGeV1VHJZk/xnjbSrNUkpMnj3dfkqPHGBdV1eszdSVhcZJ7k7xujPGzh3pO28x+3Hj2Lq9d+68QsE6ueesu0z0CtPfTE9518Rhj/qrWrTbsHQk7bFjCDhve6sLuv5QFgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoZNZ0D/BoGw88kCXXXT/dY0BbP/6zL033CNDezBMeep0zdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEZmTfcAsKIjxjV5aa7LSHJdtskHMz+H5md5RX6cXbIw/yqH5e6aPd1jwial3nFL8o37kh1nZnxzt6llJy5Izlw4dXq3w8yMjzw2edys5IL7UkfdnOw2lYdx6NzkndtP4/Ssq7U6Y6+qw6tqVNWT1+K+b6+qOes7UFUdVVUnr2L5k6vqu1X166p69/run43XDuP+HJ6f5N/kkBxTL8qMjBycX+SK7JD35qDcnPX+toLfaeOVW2d8bucHL3vrdhnn7JZx1m4ZL9wq9eE7frPyWVtMLT9rN1HfBK3tpfhXJzl/8nFN3p5skFfgO5Icn+RDG2DfbCRmZmR2lmbGWJbZWZIF2SLX1na5pbaa7tFg0/VHWybbzXzwsnkrvPzftyypR3ckNpw1hr2q5iZ5TpI3JHnVCstnVtWHquqKqvpRVR1XVccneXySc6vq3Mn97l1hmyOr6tOT24dV1fer6pKqOquqHru6OcYYt44xLkyyeKX5tqqqr1bVZZNZ/nStnz0blQW1Zf5nnpTT8tX8Q76ShdksF9fjpnssaKv+44LUM69LnX5vxgk7/GbFxYtSh1yfes1NydW/nr4BWS9rc8b+8iRnjDGuSbKgqp45WX5Mkt2T7DvGeHqS08YYJyW5KcnBY4yD17Df85McOMbYL8nnk7xnfZ5AkpckuWmM8YwxxtOSnLGe+2GazR0P5I9yU/48h+ZV+ZNskaU5ZPx8useCtsZf7ZBx8e4ZR8xN/bdfTS3cZ4uMC3fPOHu3jDdskzr65mmdkXW3NmF/dabCm8nH5ZfjX5DkE2OMJUkyxrhjFduuzq5Jzqyqy5OckOSp67j9cpcneWFVnVhVzx1j3LXyHarqmKq6qKouWhw/fW6s9s+tuTlb5a6anaU1I+dnl+ydBdM9FvR3xLzkqwunbs+bkWw1ScMhWyWLR7Jg6fTNxjpbbdiravskz09ySlVdl6kAv7Kq1uXdmLHC7S1WuP3RJCePMfZJcuxK69Z+51NXEvbPVOD/uqret4r7fHKMMX+MMX+z+I3qjdWt2TJPyR2ZPZYkY2S/3Jrrs/V0jwU9/fSB39w+c2Gy52ZTt2+dOv6SJJcsSpYl2d5fRm9K1vTnbkcm+cwY49jlC6rqW0mem+QbSY6tqnPHGEuqavvJWfs9SeYluX2yyS1V9ZQkVyd5xWR9kmyT5MbJ7dev7xOoqscnuWOM8dmq+lWSN67vvphe/1w75Ntjl/znnJ2lqVybbfN/skcOHz/OK3NNts+ifDLfyA/G4/Lhmj/d48Imo95yc3LB/ckdS1P7/yzj3Tukzl6YXLt46vRu11kZJz5m6s5fuTf13++eqsMWlfHxxybrdC7HdFtT2F+d5MSVln1hsvy4JE9K8qOqWpzkU0lOTvLJJGdU1U2T99n/MslXktyW5KIkcyf7eX+Sf6qqO5Ock2SP1Q1SVY+bbL91kmVV9fYkeyfZJ8kHq2pZpn6x7i1reE5sxE6tp+bUld6V+WL2yhez1zRNBJu+8bHf/iXU8ZqHuBr2F9tm/MW2G3YgNqgaY6z5Xo1sXduPZ9Uh0z0GtHXmTZdO9wjQ3sydf3LxGGOVly69cQIAjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAIzXGmO4ZHlVVdVuSn0/3HKyTHZPcPt1DQHOOs03L740xdlrVit+5sLPpqaqLxhjzp3sO6Mxx1odL8QDQiLADQCPCzqbgk9M9APwOcJw14T12AGjEGTsANCLsrLWqWlpVl1bVFVX1T1U152Hs69NVdeTk9ilVtfdq7vu8qnr2ejzGdVW14yqWP7OqLq+qn1TVSVVV67pv2FAaHWd/U1W/qKp713WfPDzCzrq4f4yx7xjjaUkeSPLmFVdW1az12ekY441jjKtWc5fnJVnnF5zV+FiSNyXZa/LvJY/gvuHh6nKcfTnJHz6C+2MtCTvr69tJ9pz8lP/tqvpSkquqamZVfbCqLqyqH1XVsUlSU06uqqur6qwkj1m+o6r6ZlXNn9x+SVX9sKouq6qzq2r3TL2wvWNyFvPcqtqpqr4weYwLq+pfTLbdoaq+XlVXVtUpSX7rTLyqdk6y9Rjje2PqF0xOTXL4ZN3xVXXVZO7Pb8CvHaytTfI4S5LJMfbLlZdX1b+eXI24rKrOe4S/XiRZr5/8+N02OWN4aZIzJov2T/K0McbPquqYJHeNMQ6oqtlJvlNVX0+yX5I/SLJ3kscmuSrJf11pvzsl+VSSgyb72n6McUdVfTzJvWOMD03u97kkfzfGOL+qdktyZpKnJPn3Sc4fY3ygql6W5A2rGH+XJDes8PkNk2VJ8pdJ9hhj/Lqqtl3/rxA8fJv4cbY670vy4jHGjY6zDUPYWRdbVtWlk9vfTvJfMnXp7gdjjJ9Nlr8oydOXv6+XZJtMXe4+KMn/GGMsTXJTVZ2ziv0fmOS85fsaY9zxEHO8IMneK7w1vnVVzZ08xhGTbb9aVXeu4/P7UZLTquqLSb64jtvCI6X7cfadJJ+uqn9Mcvo6bstaEHbWxf1jjH1XXDA56BeuuCjJcWOMM1e636GP4Bwzkhw4xli0ilnW5MYku67w+a6TZUnysky9aB2W5N9V1T5jjCUPf1xYJx2Os4c0xnhzVT0rU8fbxVX1zDHGgoe1Ux7Ee+w80s5M8paq2ixJqupJVbVVkvOS/OnkvcGdkxy8im2/l+Sgqtpjsu32k+X3JJm3wv2+nuS45Z9U1b6Tm+clec1k2UuTbLfyA0ze87u7qg6sqVeo1yX531U1I8kTxhjnJnlvps6A5q7H84dHw0Z9nK1OVT1xjPH9Mcb7ktyW5Anrsj1rJuw80k7J1Pt6P6yqK5J8IlNXhv5Xkh9P1p2a5LsrbzjGuC3JMUlOr6rLkvzDZNWXk7xi+S/1JDk+yfzJLw1dld/81vB/yNQL1pWZulR4/UPM+NbJnD9Jcm2SryWZmeSzVXV5kkuSnDTG+NV6fxVgw9roj7Oq+tuquiHJnKq6oareP1n1wZr6c9MrklyQ5LKH84Xgt/mf5wCgEWfsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADTy/wCOruaJmaciuAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "cm_nb = confusion_matrix(y_test,nb_pred)\n",
    "fig, ax = plt.subplots(figsize=(8, 8))\n",
    "ax.imshow(cm_nb)\n",
    "ax.grid(False)\n",
    "ax.xaxis.set(ticks=(0, 1), ticklabels=('Predicted 0s', 'Predicted 1s'))\n",
    "ax.yaxis.set(ticks=(0, 1), ticklabels=('Actual 0s', 'Actual 1s'))\n",
    "ax.set_ylim(1.5, -0.5)\n",
    "for i in range(2):\n",
    "    for j in range(2):\n",
    "        ax.text(j, i, cm_nb[i, j], ha='center', va='center', color='red')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f24d0506",
   "metadata": {},
   "source": [
    "The confusion matrix shows slightly better performance on detecting real photos then fake one."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6ec43f2",
   "metadata": {},
   "source": [
    "## Support Vector Machines"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "51c665bf",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 25 candidates, totalling 125 fits\n",
      "[CV 1/5] END ........C=0.1, gamma=1, kernel=rbf;, score=0.529 total time=   0.7s\n",
      "[CV 2/5] END ........C=0.1, gamma=1, kernel=rbf;, score=0.529 total time=   0.6s\n",
      "[CV 3/5] END ........C=0.1, gamma=1, kernel=rbf;, score=0.531 total time=   0.6s\n",
      "[CV 4/5] END ........C=0.1, gamma=1, kernel=rbf;, score=0.531 total time=   0.6s\n",
      "[CV 5/5] END ........C=0.1, gamma=1, kernel=rbf;, score=0.531 total time=   0.6s\n",
      "[CV 1/5] END ......C=0.1, gamma=0.1, kernel=rbf;, score=0.529 total time=   0.6s\n",
      "[CV 2/5] END ......C=0.1, gamma=0.1, kernel=rbf;, score=0.529 total time=   0.6s\n",
      "[CV 3/5] END ......C=0.1, gamma=0.1, kernel=rbf;, score=0.531 total time=   0.6s\n",
      "[CV 4/5] END ......C=0.1, gamma=0.1, kernel=rbf;, score=0.531 total time=   0.6s\n",
      "[CV 5/5] END ......C=0.1, gamma=0.1, kernel=rbf;, score=0.531 total time=   0.5s\n",
      "[CV 1/5] END .....C=0.1, gamma=0.01, kernel=rbf;, score=0.529 total time=   0.4s\n",
      "[CV 2/5] END .....C=0.1, gamma=0.01, kernel=rbf;, score=0.529 total time=   0.6s\n",
      "[CV 3/5] END .....C=0.1, gamma=0.01, kernel=rbf;, score=0.531 total time=   0.9s\n",
      "[CV 4/5] END .....C=0.1, gamma=0.01, kernel=rbf;, score=0.531 total time=   0.8s\n",
      "[CV 5/5] END .....C=0.1, gamma=0.01, kernel=rbf;, score=0.531 total time=   0.7s\n",
      "[CV 1/5] END ....C=0.1, gamma=0.001, kernel=rbf;, score=0.529 total time=   0.8s\n",
      "[CV 2/5] END ....C=0.1, gamma=0.001, kernel=rbf;, score=0.529 total time=   0.7s\n",
      "[CV 3/5] END ....C=0.1, gamma=0.001, kernel=rbf;, score=0.531 total time=   0.7s\n",
      "[CV 4/5] END ....C=0.1, gamma=0.001, kernel=rbf;, score=0.531 total time=   0.5s\n",
      "[CV 5/5] END ....C=0.1, gamma=0.001, kernel=rbf;, score=0.531 total time=   0.5s\n",
      "[CV 1/5] END ...C=0.1, gamma=0.0001, kernel=rbf;, score=0.529 total time=   0.6s\n",
      "[CV 2/5] END ...C=0.1, gamma=0.0001, kernel=rbf;, score=0.529 total time=   0.5s\n",
      "[CV 3/5] END ...C=0.1, gamma=0.0001, kernel=rbf;, score=0.531 total time=   0.7s\n",
      "[CV 4/5] END ...C=0.1, gamma=0.0001, kernel=rbf;, score=0.531 total time=   0.5s\n",
      "[CV 5/5] END ...C=0.1, gamma=0.0001, kernel=rbf;, score=0.531 total time=   0.5s\n",
      "[CV 1/5] END ..........C=1, gamma=1, kernel=rbf;, score=0.529 total time=   0.6s\n",
      "[CV 2/5] END ..........C=1, gamma=1, kernel=rbf;, score=0.529 total time=   0.5s\n",
      "[CV 3/5] END ..........C=1, gamma=1, kernel=rbf;, score=0.531 total time=   0.5s\n",
      "[CV 4/5] END ..........C=1, gamma=1, kernel=rbf;, score=0.531 total time=   0.6s\n",
      "[CV 5/5] END ..........C=1, gamma=1, kernel=rbf;, score=0.531 total time=   0.5s\n",
      "[CV 1/5] END ........C=1, gamma=0.1, kernel=rbf;, score=0.529 total time=   0.6s\n",
      "[CV 2/5] END ........C=1, gamma=0.1, kernel=rbf;, score=0.529 total time=   0.5s\n",
      "[CV 3/5] END ........C=1, gamma=0.1, kernel=rbf;, score=0.531 total time=   0.6s\n",
      "[CV 4/5] END ........C=1, gamma=0.1, kernel=rbf;, score=0.531 total time=   0.6s\n",
      "[CV 5/5] END ........C=1, gamma=0.1, kernel=rbf;, score=0.528 total time=   0.6s\n",
      "[CV 1/5] END .......C=1, gamma=0.01, kernel=rbf;, score=0.529 total time=   0.5s\n",
      "[CV 2/5] END .......C=1, gamma=0.01, kernel=rbf;, score=0.529 total time=   0.4s\n",
      "[CV 3/5] END .......C=1, gamma=0.01, kernel=rbf;, score=0.512 total time=   0.5s\n",
      "[CV 4/5] END .......C=1, gamma=0.01, kernel=rbf;, score=0.518 total time=   0.5s\n",
      "[CV 5/5] END .......C=1, gamma=0.01, kernel=rbf;, score=0.531 total time=   0.4s\n",
      "[CV 1/5] END ......C=1, gamma=0.001, kernel=rbf;, score=0.520 total time=   0.4s\n",
      "[CV 2/5] END ......C=1, gamma=0.001, kernel=rbf;, score=0.508 total time=   0.4s\n",
      "[CV 3/5] END ......C=1, gamma=0.001, kernel=rbf;, score=0.555 total time=   0.4s\n",
      "[CV 4/5] END ......C=1, gamma=0.001, kernel=rbf;, score=0.537 total time=   0.4s\n",
      "[CV 5/5] END ......C=1, gamma=0.001, kernel=rbf;, score=0.488 total time=   0.4s\n",
      "[CV 1/5] END .....C=1, gamma=0.0001, kernel=rbf;, score=0.514 total time=   0.4s\n",
      "[CV 2/5] END .....C=1, gamma=0.0001, kernel=rbf;, score=0.523 total time=   0.3s\n",
      "[CV 3/5] END .....C=1, gamma=0.0001, kernel=rbf;, score=0.561 total time=   0.4s\n",
      "[CV 4/5] END .....C=1, gamma=0.0001, kernel=rbf;, score=0.555 total time=   0.4s\n",
      "[CV 5/5] END .....C=1, gamma=0.0001, kernel=rbf;, score=0.561 total time=   0.4s\n",
      "[CV 1/5] END .........C=10, gamma=1, kernel=rbf;, score=0.529 total time=   0.6s\n",
      "[CV 2/5] END .........C=10, gamma=1, kernel=rbf;, score=0.529 total time=   0.6s\n",
      "[CV 3/5] END .........C=10, gamma=1, kernel=rbf;, score=0.531 total time=   0.6s\n",
      "[CV 4/5] END .........C=10, gamma=1, kernel=rbf;, score=0.531 total time=   0.5s\n",
      "[CV 5/5] END .........C=10, gamma=1, kernel=rbf;, score=0.531 total time=   0.5s\n",
      "[CV 1/5] END .......C=10, gamma=0.1, kernel=rbf;, score=0.529 total time=   0.5s\n",
      "[CV 2/5] END .......C=10, gamma=0.1, kernel=rbf;, score=0.529 total time=   0.5s\n",
      "[CV 3/5] END .......C=10, gamma=0.1, kernel=rbf;, score=0.531 total time=   0.6s\n",
      "[CV 4/5] END .......C=10, gamma=0.1, kernel=rbf;, score=0.531 total time=   0.6s\n",
      "[CV 5/5] END .......C=10, gamma=0.1, kernel=rbf;, score=0.528 total time=   0.5s\n",
      "[CV 1/5] END ......C=10, gamma=0.01, kernel=rbf;, score=0.532 total time=   0.5s\n",
      "[CV 2/5] END ......C=10, gamma=0.01, kernel=rbf;, score=0.526 total time=   0.6s\n",
      "[CV 3/5] END ......C=10, gamma=0.01, kernel=rbf;, score=0.525 total time=   0.7s\n",
      "[CV 4/5] END ......C=10, gamma=0.01, kernel=rbf;, score=0.506 total time=   0.6s\n",
      "[CV 5/5] END ......C=10, gamma=0.01, kernel=rbf;, score=0.528 total time=   0.6s\n",
      "[CV 1/5] END .....C=10, gamma=0.001, kernel=rbf;, score=0.502 total time=   0.7s\n",
      "[CV 2/5] END .....C=10, gamma=0.001, kernel=rbf;, score=0.508 total time=   0.7s\n",
      "[CV 3/5] END .....C=10, gamma=0.001, kernel=rbf;, score=0.488 total time=   0.7s\n",
      "[CV 4/5] END .....C=10, gamma=0.001, kernel=rbf;, score=0.518 total time=   0.5s\n",
      "[CV 5/5] END .....C=10, gamma=0.001, kernel=rbf;, score=0.497 total time=   0.5s\n",
      "[CV 1/5] END ....C=10, gamma=0.0001, kernel=rbf;, score=0.529 total time=   0.4s\n",
      "[CV 2/5] END ....C=10, gamma=0.0001, kernel=rbf;, score=0.505 total time=   0.5s\n",
      "[CV 3/5] END ....C=10, gamma=0.0001, kernel=rbf;, score=0.546 total time=   0.4s\n",
      "[CV 4/5] END ....C=10, gamma=0.0001, kernel=rbf;, score=0.549 total time=   0.5s\n",
      "[CV 5/5] END ....C=10, gamma=0.0001, kernel=rbf;, score=0.515 total time=   0.4s\n",
      "[CV 1/5] END ........C=100, gamma=1, kernel=rbf;, score=0.529 total time=   0.6s\n",
      "[CV 2/5] END ........C=100, gamma=1, kernel=rbf;, score=0.529 total time=   0.5s\n",
      "[CV 3/5] END ........C=100, gamma=1, kernel=rbf;, score=0.531 total time=   0.5s\n",
      "[CV 4/5] END ........C=100, gamma=1, kernel=rbf;, score=0.531 total time=   0.6s\n",
      "[CV 5/5] END ........C=100, gamma=1, kernel=rbf;, score=0.531 total time=   0.5s\n",
      "[CV 1/5] END ......C=100, gamma=0.1, kernel=rbf;, score=0.529 total time=   0.7s\n",
      "[CV 2/5] END ......C=100, gamma=0.1, kernel=rbf;, score=0.529 total time=   0.7s\n",
      "[CV 3/5] END ......C=100, gamma=0.1, kernel=rbf;, score=0.531 total time=   0.6s\n",
      "[CV 4/5] END ......C=100, gamma=0.1, kernel=rbf;, score=0.531 total time=   0.5s\n",
      "[CV 5/5] END ......C=100, gamma=0.1, kernel=rbf;, score=0.528 total time=   0.8s\n",
      "[CV 1/5] END .....C=100, gamma=0.01, kernel=rbf;, score=0.532 total time=   0.6s\n",
      "[CV 2/5] END .....C=100, gamma=0.01, kernel=rbf;, score=0.526 total time=   0.6s\n",
      "[CV 3/5] END .....C=100, gamma=0.01, kernel=rbf;, score=0.525 total time=   0.5s\n",
      "[CV 4/5] END .....C=100, gamma=0.01, kernel=rbf;, score=0.506 total time=   0.4s\n",
      "[CV 5/5] END .....C=100, gamma=0.01, kernel=rbf;, score=0.528 total time=   0.5s\n",
      "[CV 1/5] END ....C=100, gamma=0.001, kernel=rbf;, score=0.505 total time=   0.5s\n",
      "[CV 2/5] END ....C=100, gamma=0.001, kernel=rbf;, score=0.486 total time=   0.4s\n",
      "[CV 3/5] END ....C=100, gamma=0.001, kernel=rbf;, score=0.488 total time=   0.4s\n",
      "[CV 4/5] END ....C=100, gamma=0.001, kernel=rbf;, score=0.497 total time=   0.4s\n",
      "[CV 5/5] END ....C=100, gamma=0.001, kernel=rbf;, score=0.488 total time=   0.4s\n",
      "[CV 1/5] END ...C=100, gamma=0.0001, kernel=rbf;, score=0.468 total time=   0.5s\n",
      "[CV 2/5] END ...C=100, gamma=0.0001, kernel=rbf;, score=0.483 total time=   0.8s\n",
      "[CV 3/5] END ...C=100, gamma=0.0001, kernel=rbf;, score=0.528 total time=   0.8s\n",
      "[CV 4/5] END ...C=100, gamma=0.0001, kernel=rbf;, score=0.567 total time=   0.7s\n",
      "[CV 5/5] END ...C=100, gamma=0.0001, kernel=rbf;, score=0.500 total time=   0.5s\n",
      "[CV 1/5] END .......C=1000, gamma=1, kernel=rbf;, score=0.529 total time=   0.5s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 2/5] END .......C=1000, gamma=1, kernel=rbf;, score=0.529 total time=   0.5s\n",
      "[CV 3/5] END .......C=1000, gamma=1, kernel=rbf;, score=0.531 total time=   0.5s\n",
      "[CV 4/5] END .......C=1000, gamma=1, kernel=rbf;, score=0.531 total time=   0.5s\n",
      "[CV 5/5] END .......C=1000, gamma=1, kernel=rbf;, score=0.531 total time=   0.5s\n",
      "[CV 1/5] END .....C=1000, gamma=0.1, kernel=rbf;, score=0.529 total time=   0.5s\n",
      "[CV 2/5] END .....C=1000, gamma=0.1, kernel=rbf;, score=0.529 total time=   0.5s\n",
      "[CV 3/5] END .....C=1000, gamma=0.1, kernel=rbf;, score=0.531 total time=   0.5s\n",
      "[CV 4/5] END .....C=1000, gamma=0.1, kernel=rbf;, score=0.531 total time=   0.5s\n",
      "[CV 5/5] END .....C=1000, gamma=0.1, kernel=rbf;, score=0.528 total time=   0.6s\n",
      "[CV 1/5] END ....C=1000, gamma=0.01, kernel=rbf;, score=0.532 total time=   0.5s\n",
      "[CV 2/5] END ....C=1000, gamma=0.01, kernel=rbf;, score=0.526 total time=   0.6s\n",
      "[CV 3/5] END ....C=1000, gamma=0.01, kernel=rbf;, score=0.525 total time=   0.5s\n",
      "[CV 4/5] END ....C=1000, gamma=0.01, kernel=rbf;, score=0.506 total time=   0.4s\n",
      "[CV 5/5] END ....C=1000, gamma=0.01, kernel=rbf;, score=0.528 total time=   0.4s\n",
      "[CV 1/5] END ...C=1000, gamma=0.001, kernel=rbf;, score=0.505 total time=   0.4s\n",
      "[CV 2/5] END ...C=1000, gamma=0.001, kernel=rbf;, score=0.489 total time=   0.4s\n",
      "[CV 3/5] END ...C=1000, gamma=0.001, kernel=rbf;, score=0.491 total time=   0.4s\n",
      "[CV 4/5] END ...C=1000, gamma=0.001, kernel=rbf;, score=0.497 total time=   0.5s\n",
      "[CV 5/5] END ...C=1000, gamma=0.001, kernel=rbf;, score=0.488 total time=   0.4s\n",
      "[CV 1/5] END ..C=1000, gamma=0.0001, kernel=rbf;, score=0.462 total time=   0.8s\n",
      "[CV 2/5] END ..C=1000, gamma=0.0001, kernel=rbf;, score=0.483 total time=   1.2s\n",
      "[CV 3/5] END ..C=1000, gamma=0.0001, kernel=rbf;, score=0.512 total time=   0.8s\n",
      "[CV 4/5] END ..C=1000, gamma=0.0001, kernel=rbf;, score=0.549 total time=   0.9s\n",
      "[CV 5/5] END ..C=1000, gamma=0.0001, kernel=rbf;, score=0.488 total time=   0.8s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "GridSearchCV(estimator=SVC(),\n",
       "             param_grid={'C': [0.1, 1, 10, 100, 1000],\n",
       "                         'gamma': [1, 0.1, 0.01, 0.001, 0.0001],\n",
       "                         'kernel': ['rbf']},\n",
       "             verbose=3)"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# defining parameter range\n",
    "param_grid = {'C': [0.1, 1, 10, 100, 1000],\n",
    "              'gamma': [1, 0.1, 0.01, 0.001, 0.0001],\n",
    "              'kernel': ['rbf']}\n",
    " \n",
    "grid = GridSearchCV(SVC(), param_grid, refit = True, verbose = 3)\n",
    " \n",
    "# fitting the model for grid search\n",
    "grid.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2249578a",
   "metadata": {},
   "source": [
    "Here I ran a grid serch for the parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "id": "632dc285",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'C': 1, 'gamma': 0.0001, 'kernel': 'rbf'}"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "a3ef85cf",
   "metadata": {},
   "outputs": [],
   "source": [
    "svc = SVC(C= 10, gamma = 0.0001, kernel= 'rbf')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "id": "a51f198e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVC(C=10, gamma=0.0001)"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "svc.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "id": "ed326c66",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7083333333333334"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "svc.score(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "id": "f140d090",
   "metadata": {},
   "outputs": [],
   "source": [
    "svc_pred = svc.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "id": "51f03c8d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.58679706601467"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "acc_svc = svc.score(X_test, y_test)\n",
    "acc_svc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "id": "cfca4ad7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6442105263157895"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "f1_svc = f1_score(y_test, svc_pred)\n",
    "f1_svc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "id": "818edd82",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.58      0.45      0.51       193\n",
      "           1       0.59      0.71      0.64       216\n",
      "\n",
      "    accuracy                           0.59       409\n",
      "   macro avg       0.59      0.58      0.58       409\n",
      "weighted avg       0.59      0.59      0.58       409\n",
      "\n"
     ]
    }
   ],
   "source": [
    "cr_svc = classification_report(y_test,svc_pred )\n",
    "print(cr_svc)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd43914b",
   "metadata": {},
   "source": [
    "The performance of SVC is higher than the baseline model for both fake and real pictures, the precision is almost the same for both classes but recall is better for real photos"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "id": "59860ce4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfYAAAHSCAYAAAAe1umcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAWuklEQVR4nO3cebCddZ3n8c+XBIEkBBNEjSyirSIINkIQ2sEFhHFrxmVQ26VcRgWxxba1dbSmx1ar7RmVKVtlxgWnR6WxtS211XEERVFEWyEoKuDSIMoqhn2HEH7zxz1AhJDkXgg3+fp6VaXuuc/znOd8z616zvs+zzk3NcYIANDDJrM9AABwzxF2AGhE2AGgEWEHgEaEHQAaEXYAaGTubA9wb9t0s/ljs/mLZ3sMaGvFvNmeAPq76YLzLxljbLO6dX9wYd9s/uLsduBfzPYY0NbvlroQCOvbr978xt/c1TpHIAA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQyNzZHgBW9We/ODEHnXNyRpKzt1qSdz32eXn/t4/KvJtvSJIsuuHanLl4+7xl35fN6pywMfnv//zp7P+zn+XSBQvytDe+KUmy1XXX5QPHfDLbXXZ5zl+8KIe/6CW5at68JMneZ5+Vv/7SFzP3lpW5fN78vPCwP5/N8ZmmdQp7VT0ryReS7DzG+Platn19ko+OMa6byUBV9bIkS8cYr73D8kry/iRPT3JdkpeNMX44k8dgw7TNdVfmuWedlBc+5U25ce6m+dvvHZ0Dzj0th+3/mtu2+bvvfiLf2fZRszglbHw+t3SvHP24fXPEZ/7ptmWvPuEb+d7DHp6P7PfkHHrCN/Lqb30z73n6n2bL66/PO77w+bz8Fa/KRYsWZetrrp7FyZmJdb0U/4IkJ02+rs3rk8yb6UBr8LQkD5/8OyTJh9bDYzDL5txySzZbuSJzblmZzVeuyCVbLLxt3bwVN2TP352db2+76yxOCBufUx76R7li3u+/LB9wxhn5/J57JUk+v+deOfD005Mk/+FHP8zXdt0tFy1alCS5dMGW9+6w3G1rPWOvqgVJ9k2yX5IvJ/mbyfI5Sd6d5KlJbklyVJJK8qAkJ1TVJWOM/arqmjHGgsl9Dk7yp2OMl1XVQUn+Osl9klya5EVjjIvXMMozk3xyjDGSfL+q7ltVSybrPpNk4eT5HDbG+M60fgpsEJbP2yqf2umJ+cJX3pUb52yakx/wiJz8wJ1uW//EC07Psgc8LNdtuvksTgk93O+aq7N84dQvzsu33DL3m5yZP+SS5Zm7cmWO+fD/yoIbb8zH9318vrDn0tkclWlalzP2ZyY5dozxyySXVtWek+WHJNkxye5jjEcnOWaM8YEkFybZb4yx31r2e1KSfcYYj0ny6SRvXsv22yY5b5Xvz58se2GS48YYuyf54ySnrcNzYgO05U3X5fEXnpH/+PS35qCD/ms2v/mmPOU3p962/sBzT8vXd9h99gaErqoyqpJMXTXb9YLz88r/9Iq87JWvymuP/3p2XL58lgdkOtYl7C/IVHgz+Xrr5fgDknxkjHFzkowxLpvmY2+X5Liq+mmSNyWZ6RunpyR5eVW9PcluY4w7vSFUVYdU1bKqWrbixmtm+DCsb3td/G+5aP7iXLH5gqzcZE6+vd2u2e2S3yRJtrrx2uxy2Xn53pKdZ3lK6OGSBVtmm6uuSpJsc9VVuXT+giTJb7faKt95xE65/j6b5fL5C3LyQx+anS+6cDZHZZrWGPaqWpxk/yQfq6pfZyrAz5t8kG1djVVur3oN9YNJjhxj7Jbk0DusW50Lkmy/yvfbJblgjHFikidM1n+8ql5ypwHG+OgYY+kYY+mmmy2Yxujcm347b1Eedem52ezmm5IxsvTis/LrhfdPkux/3k/y3SU756Y5m87ylNDDN3Z5VJ5z6ilJkuecekqOf9TUudXxu+yapb8+J3NWrszmN92U3c89N2ff//6zOSrTtLb32A9OcvQY49BbF1TVt5M8PsnXkxxaVSeMMW6uqsWTs/ark2yZ5JLJXS6uqp2T/CLJsyfrk2SrTMU4SV66DrN+Kclrq+rTSfZOcuUY46KqenCS88cYR1XVZkn2SPLJddgfG5gzt94hJ2y3Wz7x9b/PzbVJfrlo23zxofskSQ4477Qc/ci1vbsDrM7fH3N09v7V2Vl07bU56V3vzPsPfEo+vN/++eAxn8zzTj45FyxalMNfPHVOdPYDHpATH/HIfOV9/yOjKp957N755QOXrOUR2JDU1GfR7mJl1QlJ3j3GOHaVZa9LsnOSw5O8J1MfnluR5KgxxpFVdXiS1ya5cPLhuYMz9SG75UmWJVkw+fDcM5O8L8nlSb6ZZK8xxpPW8uduR04e77okLx9jLKuql2bqSsKKJNckeckY45y7ek4LFm8/djvwL9b9JwRMy++W+n+vYH371ZvfeOoYY7Wfalxj2DsSdli/hB3WvzWF3REIAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQSI0xZnuGe9XCWjz2rifP9hjQ1nEXnjbbI0B7c5acdeoYY+nq1jljB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaGTubA8AdzR/3JQ35NTsmKuSJEdkaR6bi/K4XJSR5Ipslvdmr1xaW8zuoLCRqL+8OPn6dcn95mR8a4epZUdcmhxzVbL1nCTJeOvWyZPnJz+6IfWm303dcSTjjYuTpy+YrdGZgXU6Y6+qZ1XVqKpHrsO2r6+qeTMdqKpeVlVHrmb5I6vqX6vqxqr6q5nunw3fa/LjLMsD84p6Sg7NgTk3W+az2SmH1oF5dR2Y72dJXpyfzfaYsNEYz1uY8akld15+yH0zjt8h4/gdpqKeJDvdJ+PY7aeWf+pBqTcvT24e9/LE3B3rein+BUlOmnxdm9cnmXHY1+CyJK9LcsR62DcbiHljRXbL8nw1OyZJbq5Ncm3dJ9fVprdts3lWxssMTMOfbJEsmrNu287bJJlbU7dvHEmtv7FYP9Ya9qpakGTfJK9I8merLJ9TVUdU1elV9ZOqOryqXpfkQUlOqKoTJttds8p9Dq6qj09uH1RVP6iqH1XV8VX1gDXNMcb43RjjlCQr7jDf/Kr6SlX9eDLL89f52bPBWZJrc2U2y5uyLB8ax+cNY1k2HzcnSV4+Ts8x4yvZP+fmE3nULE8KG7/6hytT+587dan+ipW3r/jhDaknnpva79yMd29ze+jZKKzLGfszkxw7xvhlkkuras/J8kOS7Jhk9zHGo5McM8b4QJILk+w3xthvLfs9Kck+Y4zHJPl0kjfP5AkkeWqSC8cYfzzG2DXJsTPcDxuAObklD88V+XIemsPqgNyQuXl+fp4k+T+1a15Uz8g3s0OembNmeVLYuI2XbpXx/QdnHL99cv+5qXdccvvKPTbP+PYOGV/dPvXBy5Mbbpm9QZm2dQn7CzIV3ky+3no5/oAkHxlj6nRqjHHZNB97uyTHVdVPk7wpmfEp2E+THFhV766qx48xrrzjBlV1SFUtq6plK3LjDB+Ge8PyzMvybJGf19ZJkhOzbR6eK35vm29kh+ybC2ZhOmhkm7nJnEo2qYwXL0x+tJrXxkfcJ5m/SfLzm+79+ZixNYa9qhYn2T/Jx6rq15kK8POqajrXZVZ9O3TzVW5/MMmRY4zdkhx6h3XrvvOpKwl7ZCrwf1tVb1vNNh8dYywdYyzdNJvN5GG4l1xem2d5tsh24+okyWPyu/wmC7Pt5PskeVwuzHnZcrZGhB4uvvn22//v2uSR95m6fe6K2z8sd96K5Kybku03vfP92WCt7c/dDk5y9Bjj0FsXVNW3kzw+ydeTHFpVJ4wxbq6qxZOz9quTbJnk1us6F1fVzkl+keTZk/VJslVy22nXS2f6BKrqQUkuG2P8Y1VdkeSVM90XG4b/mcfkrTk5c8ctuSjzc0SW5g05NduNqzNSuTjz8v7sMdtjwkajDvtt8r3rk8tWpvY4J+Ovtk597/rkjBunPhy3/dyM99x/auMfXJ868opk0yRVGf9tm9v+JI6Nw9rC/oIk777Dss9Nlh+e5BFJflJVK5IcleTIJB9NcmxVXTh5n/0tSf5vkuVJliW59Q8i357ks1V1eZJvJnnImgapqgdO7r8wyS1V9fokuyTZLcl7q+qWTH2w7rC1PCc2cGfXffPnefLvLXtn/mSWpoGN3/jQA++87IULV7/xcxdmPPcu1rFRqDH+sP5waGEtHnvXk9e+ITAjx1142myPAO3NWXLWqWOMpatb57+UBYBGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaCRGmPM9gz3qqpanuQ3sz0H03K/JJfM9hDQnONs4/LgMcY2q1vxBxd2Nj5VtWyMsXS254DOHGd9uBQPAI0IOwA0IuxsDD462wPAHwDHWRPeYweARpyxA0Ajws46q6qVVXVaVZ1eVZ+tqnl3Y18fr6qDJ7c/VlW7rGHbJ1XV42bwGL+uqvutZvmeVfXTqjqrqj5QVTXdfcP60ug4e1dVnVdV10x3n9w9ws50XD/G2H2MsWuSm5K8etWVVTV3JjsdY7xyjHHmGjZ5UpJpv+CswYeSvCrJwyf/nnoP7hvuri7H2ZeTPPYe3B/rSNiZqe8kedjkt/zvVNWXkpxZVXOq6r1VdUpV/aSqDk2SmnJkVf2iqo5Pcv9bd1RV36qqpZPbT62qH1bVj6vqG1W1Y6Ze2P5ychbz+Krapqo+N3mMU6rq303uu3VVfa2qzqiqjyW505l4VS1JsnCM8f0x9QGTTyZ51mTd66rqzMncn16PPztYVxvlcZYkk2Psojsur6rnTq5G/LiqTryHf14kmdFvfvxhm5wxPC3JsZNFeyTZdYxxTlUdkuTKMcZeVbVZku9W1deSPCbJTkl2SfKAJGcm+Yc77HebJEclecJkX4vHGJdV1YeTXDPGOGKy3aeSvG+McVJV7ZDkuCQ7J/mbJCeNMd5ZVc9I8orVjL9tkvNX+f78ybIkeUuSh4wxbqyq+878JwR330Z+nK3J25I8ZYxxgeNs/RB2pmOLqjptcvs7Sf53pi7dnTzGOGey/N8nefSt7+sl2SpTl7ufkOSfxhgrk1xYVd9czf73SXLirfsaY1x2F3MckGSXVd4aX1hVCyaP8ZzJfb9SVZdP8/n9JMkxVfUvSf5lmveFe0r34+y7ST5eVf+c5PPTvC/rQNiZjuvHGLuvumBy0F+76qIkh48xjrvDdk+/B+fYJMk+Y4wbVjPL2lyQZLtVvt9usixJnpGpF62DkvyXqtptjHHz3R8XpqXDcXaXxhivrqq9M3W8nVpVe44xLr1bO+X3eI+de9pxSQ6rqk2TpKoeUVXzk5yY5PmT9waXJNlvNff9fpInVNVDJvddPFl+dZItV9nua0kOv/Wbqtp9cvPEJC+cLHtakkV3fIDJe35XVdU+NfUK9ZIkX6yqTZJsP8Y4Icl/ztQZ0IIZPH+4N2zQx9maVNUfjTF+MMZ4W5LlSbafzv1ZO2HnnvaxTL2v98OqOj3JRzJ1ZegLSf5tsu6TSf71jnccYyxPckiSz1fVj5N8ZrLqy0mefeuHepK8LsnSyYeGzsztnxp+R6ZesM7I1KXCc+9ixtdM5jwrydlJvppkTpJ/rKqfJvlRkg+MMa6Y8U8B1q8N/jirqvdU1flJ5lXV+VX19smq99bUn5uenuR7SX58d34Q3Jn/eQ4AGnHGDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Aj/x+8BRvI9xXW+AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "cm_svc = confusion_matrix(y_test,svc_pred)\n",
    "fig, ax = plt.subplots(figsize=(8, 8))\n",
    "ax.imshow(cm_svc)\n",
    "ax.grid(False)\n",
    "ax.xaxis.set(ticks=(0, 1), ticklabels=('Predicted 0s', 'Predicted 1s'))\n",
    "ax.yaxis.set(ticks=(0, 1), ticklabels=('Actual 0s', 'Actual 1s'))\n",
    "ax.set_ylim(1.5, -0.5)\n",
    "for i in range(2):\n",
    "    for j in range(2):\n",
    "        ax.text(j, i, cm_svc[i, j], ha='center', va='center', color='red')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6fa8e34d",
   "metadata": {},
   "source": [
    "The Classification report shows more correct guesses than wrong one for both classes, which is a step up from only being able to detect real photos like the models before did."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d787f2ba",
   "metadata": {},
   "source": [
    "## XGBoost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "59376565",
   "metadata": {},
   "outputs": [],
   "source": [
    "xgb = XGBClassifier(learning_rate=0.02, n_estimators=600, objective='binary:logistic',\n",
    "                    silent=True, nthread=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "id": "57febaf0",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\xgboost\\sklearn.py:1224: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
      "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n",
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\xgboost\\data.py:262: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
      "  elif isinstance(data.columns, (pd.Int64Index, pd.RangeIndex)):\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[21:53:09] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.5.1/src/learner.cc:576: \n",
      "Parameters: { \"silent\" } might not be used.\n",
      "\n",
      "  This could be a false alarm, with some parameters getting used by language bindings but\n",
      "  then being mistakenly passed down to XGBoost core, or some parameter actually being used\n",
      "  but getting flagged wrongly here. Please open an issue if you find any such cases.\n",
      "\n",
      "\n",
      "[21:53:09] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.5.1/src/learner.cc:1115: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "XGBClassifier(base_score=0.5, booster='gbtree', colsample_bylevel=1,\n",
       "              colsample_bynode=1, colsample_bytree=1, enable_categorical=False,\n",
       "              gamma=0, gpu_id=-1, importance_type=None,\n",
       "              interaction_constraints='', learning_rate=0.02, max_delta_step=0,\n",
       "              max_depth=6, min_child_weight=1, missing=nan,\n",
       "              monotone_constraints='()', n_estimators=600, n_jobs=1, nthread=1,\n",
       "              num_parallel_tree=1, predictor='auto', random_state=0,\n",
       "              reg_alpha=0, reg_lambda=1, scale_pos_weight=1, silent=True,\n",
       "              subsample=1, tree_method='exact', validate_parameters=1,\n",
       "              verbosity=None)"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xgb.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "id": "a0ef4b6e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9993872549019608"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xgb.score(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "id": "a23f85e8",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\imane\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\xgboost\\data.py:262: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
      "  elif isinstance(data.columns, (pd.Int64Index, pd.RangeIndex)):\n"
     ]
    }
   ],
   "source": [
    "xgb_pred = xgb.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "id": "394d1d99",
   "metadata": {},
   "outputs": [],
   "source": [
    "acc_xgb = xgb.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "id": "0292c8bb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5887445887445888"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "f1_xgb = f1_score(y_test, xgb_pred)\n",
    "f1_xgb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "id": "cc7b047b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.51      0.43      0.47       193\n",
      "           1       0.55      0.63      0.59       216\n",
      "\n",
      "    accuracy                           0.54       409\n",
      "   macro avg       0.53      0.53      0.53       409\n",
      "weighted avg       0.53      0.54      0.53       409\n",
      "\n"
     ]
    }
   ],
   "source": [
    "cr_xgb = classification_report(y_test,xgb_pred )\n",
    "print(cr_xgb)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cdb453c7",
   "metadata": {},
   "source": [
    "The accuracy of XGB is almost 1 on the training data, which means that the algo has most probably overfit the training data, its accuracy on the test set is much more reasonable. In terms of F1 score, it's average and slightly better for detecting real pictures."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "id": "1411ba52",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfYAAAHSCAYAAAAe1umcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAWeklEQVR4nO3cefTddX3n8df7lxCWJJAFFRAQpkLdZYkVpuqAW5EWFw7VolOlRwaUEUbHap3pjNKeemYYbTtDc8ZRmdHBpS5V26oV3EW0LrgAioOC7JQtATQIhCSf+eN3sSGEbCy/5O3jcU5O7u/zvd/vff9+59z7/H2/9yY1xggA0MPUTA8AADxwhB0AGhF2AGhE2AGgEWEHgEaEHQAamT3TAzzU5kztMHacNX+mx4C27thj+5keAdpbeeU1N40xHra+bb9yYd9x1vwcuuDomR4D2vrRHz96pkeA9q488Y1X3Nc2l+IBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEZmz/QAsK4X3X5Bjrjj4owkl89alD+f/6/ymhVfy36rbkwluXrWLvnz+YfljtpupkeFbcLb/u9H8swLf5Rl8+fluW95fZLkyO9ckNd98nN59HU35Plvek0u3GevX97/pM98MS/52rezeqpy6ktekHMe/+szNTpbYJPO2KvqhVU1quoxm3Df11bVTls6UFUdV1VL17NeVXV6VV1SVRdU1UFb+hhsvRavvi0vuP2HOXnBi/Kqhb+bqYwcdueleefcQ3PSwmPy6oXH5MapeXn+7T+c6VFhm/HRQ5fkFae88h5rP97jETnxVb+fb+637z3W97v2+hx13vl5zlten1eccnz+7IOfyNSaNQ/luNxPm3op/tgk507+3pjXJtnisG/A85LsN/lzQpJ3PAiPwVZgVtZkzliVqbEm249VWTY1N7+YmjO9cYzMyaqMmR0Rtinf2v9f5Jad7vmyfMnuj8hPd3v4ve77nPN/mE8ueXJWbjc7V+26KJc/fNcccNlVD9WoPAA2eim+quYleVqSw5N8MslbJuuzkpyW5Igka5K8O0kl2SPJl6rqpjHG4VW1Yowxb7LPMUl+Z4xxXFUdleQ/JZmTZFmSl40xrt/AKC9IcuYYYyT5RlUtqKrdJ9s+nGTnyffz6jHGVzfrp8BWY9msufmbHZ+U9y3/YO6s2fnunD3z3Tl7Jkn+/c+/nKesvCpXzl6Qd889dGYHhaZ2u+Vn+d6+e//y6+sW7pLdbrl1Bidic23KGfsLkpw1xvhxkmVVdfBk/YQk+yQ5YIzxpCQfGGOcnuTaJIePMQ7fyHHPTXLIGOPAJB9K8saN3P+RSdb+tfHqydpLk5w9xjggyZOTfH8Tvie2UvPW3JlDV16R4xYdm5ct+tfZYdyVZ97xkyTJX8w/LC9b9LJcOWthnnHnpTM8KcDWaVPCfmymw5vJ33dfjn92kneOMVYlyRhj+WY+9p5Jzq6qC5O8IcnjN3P/u307yR9U1alJnjjG+Pm6d6iqE6rqvKo6b+WaO7bwYXgoHHjXNbl+an5undoxq2sqX5uzbx676p8v5KypqXxl+1/L0+68bAanhL6uW7Bzdr/5ll9+vdvNt+a6BbvM3EBstg2GvaoWJXlmkjOq6vJMB/jFVVWb8Rhrvx26w1q3/yrJ0jHGE5OcuM629bkmyV5rfb1nkmvGGOckecZk+3ur6uX3GmCMd40xlowxlsyZ2tjDMJNumJqXx6y6IduPVckYOeCua3LVrAXZffXkUuAYOWTlFblq9oIZnRO6+tyTH5ejzjs/c+5alb1uWp59b7gp3993r43vyFZjY++xH5PkfWOME+9eqKqvJHl6ks8lObGqvjTGWFVViyZn7T9PMj/JTZNdrq+qxya5OMmLJtuTZJdMxzhJXrEJs/59ktdU1YeSPDXJrWOMf6qqRyW5eozx7qraPslBSc7chOOxFbp4u4fnq3P2zdJbPpbVmcqlsxfnMzs8Nv/11k9lp7EyleSnsxdn6dynzfSosM04/YwP5NCLf5qFK27LN/7orfnLo56TW+bulD/50N9l0YoVec/S9+SivfbIy//d8fnJHrvl0wc/KZ8/9e1ZNWsq//nYF2bNlP/yZFuysbAfm+kPyK3tY5P1k5Psn+SCqror0x+eW5rkXUnOqqprJ++zvynJp5LcmOS8JPMmxzk1yUer6uYkX0xyz39zcW//kOTIJJck+UWSP5isH5bkDZMZViS51xk725b3z12S989dco+11y94wQxNA9u+U45/2XrXzz7wCetdX3rks7L0yGc9mCPxIKrpD5n/6thlu4eNQxccPdNjQFs/euujZ3oEaO/KE9/4nTHGkvVtc30FABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgkdkzPcBDbaxandXLls/0GNDWZc9/10yPAO3NOvG+tzljB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaGT2TA8A6zp6/DjPy+UZSS7PLnlblmRx7sh/zDeyc1bmJ1mY0/IbWVV+L4VNUa+7PvncL5JdZ2V8ee/ptdOWJWffNn16t3hWxv94RLLbJAlf/0XqzTcldyVZNJXxiT1nbHY23ya9MlbVC6tqVNVjNuG+r62qnbZ0oKo6rqqWrmf9MVX1j1V1Z1X94ZYen63b4nF7XphL8m/zrJxQz81URg7PVTk+F+bj2T/H1fOyInNyRC6b6VFhmzFevHPGB3e/59pJCzO+uHfG5/fOeM7c1F8sn95w6+rUm27MeO/uGV/ZO+Pdu83AxNwfm3rKc2yScyd/b8xrk2xx2DdgeZJTkrz9QTg2W5FZGdk+qzM11mT7rMry7JADckPOySOTJJ/No/KbuXaGp4RtyKE7Jgtn3XNt/lov/79Yk9Tk9idWJEfOS/bcbvrrXV3Y3dZsNOxVNS/J05K8MsnvrbU+q6reXlU/qKoLqurkqjolyR5JvlRVX5rcb8Va+xxTVe+d3D6qqr5ZVd+rqs9X1SM2NMcY44YxxrczfXFo7fnmVtWnq+r8ySwv2eTvnq3Ostoxf5P984F8Oh/Op3JbtsuPszArsl3WTC6935Qdszi3z/CksO2r/7IsdfDlqY+vyHjD4um1S1cmt65JHX116rlXJR/52QxPyebalDP2FyQ5a4zx4yTLqurgyfoJSfZJcsAY40lJPjDGOD3JtUkOH2McvpHjnpvkkDHGgUk+lOSNW/INJDkiybVjjCePMZ6Q5KwtPA5bgXljZQ7Ntfn9HJnfy+9kh6zOU3LdTI8FLY3/sDjjO/tkHD0v9Z5bphdXJ7ngjoz375Hx13uk/vvNyaUrZ3JMNtOmhP3YTIc3k7/vvhz/7CTvHGOsSpIxxvLNfOw9k5xdVRcmeUOSx2/m/ne7MMlzquq0qnr6GOPWde9QVSdU1XlVdd5duXMLH4aHwkG5Iddlbm6t7bO6pnJuHpnHZ1nm5a5MjTVJkl1ze5ZlxxmeFBo5en7y6duSJGP32clhOyU7TSWLZyWH7JBcJOzbkg2GvaoWJXlmkjOq6vJMB/jFVVUb2m8dY63bO6x1+6+SLB1jPDHJiets2/SDT19JOCjTgf+zqnrzeu7zrjHGkjHGku2y/ZY8DA+RG7JjHpvl2X6sSsbIgbkhV2TnnJ+H5Rm5Jkny3FyRr2ePGZ4UtnE/XSvWZ9+WPHrynvpvzU2+dUeyaky/9/7dO5P9tpuZGdkiG/tUxDFJ3jfGOPHuhar6SpKnJ/lckhOr6ktjjFVVtWhy1v7zJPOT3DTZ5fqqemySi5O8aLI9SXZJJq/UySu29Buoqj2SLB9jvL+qbkly/JYei5n3/2pxvjoemf+ZL2R1KpdmQf4h++ab2S1/nG/muPGDXJoFOSv7zPSosM2oV1+XfP32ZPnq1EGXZfzh4tQXbksuvWv69G7P2RmnPXz6zvvPyTh8p9Qzr0ymKuOlOyePcUK0LdlY2I9Ncto6ax+brJ+cZP8kF1TVXUnenWRpknclOauqrp28z/6mJJ9KcmOS85LMmxzn1CQfraqbk3wxyb4bGqSqdpvsv3OSNVX12iSPS/LEJG+rqjWZ/mDdqzfyPbGVO7MenzPXeWfmuszLyXnWDE0E27bxjnv/k7Xx0p3ve4eTFmactPBBnIgHU40xNn6vRnauReOpJRDwYDn72u/P9AjQ3qzdL/nOGGPJ+rb5r7sAoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaKTGGDM9w0Oqqm5McsVMz8Fm2TXJTTM9BDTnebZtedQY42Hr2/ArF3a2PVV13hhjyUzPAZ15nvXhUjwANCLsANCIsLMteNdMDwC/AjzPmvAeOwA04owdABoRdjZZVa2uqu9X1Q+q6qNVtdP9ONZ7q+qYye0zqupxG7jvYVX1L7fgMS6vql3Xs35wVV1YVZdU1elVVZt7bHiwNHqevbWqrqqqFZt7TO4fYWdz3D7GOGCM8YQkK5O8au2NVTV7Sw46xjh+jHHRBu5yWJLNfsHZgHck+TdJ9pv8OeIBPDbcX12eZ59M8hsP4PHYRMLOlvpqkkdPfsv/alX9fZKLqmpWVb2tqr5dVRdU1YlJUtOWVtXFVfX5JA+/+0BV9eWqWjK5fURVfbeqzq+qL1TVPpl+YXvd5Czm6VX1sKr62OQxvl1VvznZd3FVfbaqflhVZyS515l4Ve2eZOcxxjfG9AdMzkzywsm2U6rqosncH3oQf3awqbbJ51mSTJ5j/7TuelX97uRqxPlVdc4D/PMiyRb95sevtskZw/OSnDVZOijJE8YYl1XVCUluHWM8paq2T/K1qvpskgOT/HqSxyV5RJKLkvyfdY77sCTvTvKMybEWjTGWV9X/SrJijPH2yf0+mOQvxxjnVtXeSc5O8tgkb0ly7hjjT6vqt5O8cj3jPzLJ1Wt9ffVkLUnelGTfMcadVbVgy39CcP9t48+zDXlzkt8aY1zjefbgEHY2x45V9f3J7a8m+d+ZvnT3rTHGZZP15yZ50t3v6yXZJdOXu5+R5K/HGKuTXFtVX1zP8Q9Jcs7dxxpjLL+POZ6d5HFrvTW+c1XNmzzG0ZN9P11VN2/m93dBkg9U1d8m+dvN3BceKN2fZ19L8t6q+kiSj2/mvmwCYWdz3D7GOGDthcmT/ra1l5KcPMY4e537HfkAzjGV5JAxxh3rmWVjrkmy51pf7zlZS5LfzvSL1lFJ/riqnjjGWHX/x4XN0uF5dp/GGK+qqqdm+vn2nao6eIyx7H4dlHvwHjsPtLOTvLqqtkuSqtq/quYmOSfJSybvDe6e5PD17PuNJM+oqn0n+y6arP88yfy17vfZJCff/UVVHTC5eU6Sl07Wnpdk4boPMHnP72dVdUhNv0K9PMnfVdVUkr3GGF9K8keZPgOatwXfPzwUturn2YZU1a+NMb45xnhzkhuT7LU5+7Nxws4D7YxMv6/33ar6QZJ3ZvrK0CeS/GSy7cwk/7jujmOMG5OckOTjVXV+kg9PNn0yyYvu/lBPklOSLJl8aOii/POnhv8k0y9YP8z0pcIr72PGkyZzXpLk0iSfSTIryfur6sIk30ty+hjjli3+KcCDa6t/nlXVf6uqq5PsVFVXV9Wpk01vq+l/bvqDJF9Pcv79+UFwb/7nOQBoxBk7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI38f5b+2A1/YHwgAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "cm_xgb = confusion_matrix(y_test,xgb_pred)\n",
    "fig, ax = plt.subplots(figsize=(8, 8))\n",
    "ax.imshow(cm_xgb)\n",
    "ax.grid(False)\n",
    "ax.xaxis.set(ticks=(0, 1), ticklabels=('Predicted 0s', 'Predicted 1s'))\n",
    "ax.yaxis.set(ticks=(0, 1), ticklabels=('Actual 0s', 'Actual 1s'))\n",
    "ax.set_ylim(1.5, -0.5)\n",
    "for i in range(2):\n",
    "    for j in range(2):\n",
    "        ax.text(j, i, cm_xgb[i, j], ha='center', va='center', color='red')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5b45342",
   "metadata": {},
   "source": [
    "## K Nearest Neighbour"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "id": "23035120",
   "metadata": {},
   "outputs": [],
   "source": [
    "knn = KNeighborsClassifier()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "id": "183070c1",
   "metadata": {},
   "outputs": [],
   "source": [
    "k_range = list(range(1, 31))\n",
    "param_grid = dict(n_neighbors=k_range)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "id": "7fe601f0",
   "metadata": {},
   "outputs": [],
   "source": [
    "grid = GridSearchCV(knn, param_grid, cv=10, scoring='accuracy', return_train_score=False,verbose=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "id": "c7bea3c7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 10 folds for each of 30 candidates, totalling 300 fits\n"
     ]
    }
   ],
   "source": [
    "grid_search = grid.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "id": "35833b6c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'n_neighbors': 15}"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid_search.best_params_"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1cfc1159",
   "metadata": {},
   "source": [
    "The grid search has determined 20 neighbours to be the best parameter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "id": "e80163ae",
   "metadata": {},
   "outputs": [],
   "source": [
    "knn = KNeighborsClassifier(n_neighbors = 15)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "id": "c8d9a337",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier(n_neighbors=15)"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "knn.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "id": "06bcb99e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.616421568627451"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "knn.score(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "id": "e66d15f5",
   "metadata": {},
   "outputs": [],
   "source": [
    "knn_pred = knn.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "id": "a5cd8b9b",
   "metadata": {},
   "outputs": [],
   "source": [
    "acc_knn = knn.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "id": "71fd8d1f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5899581589958159"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "f1_knn = f1_score(y_test, knn_pred)\n",
    "f1_knn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "id": "b4f1c5ea",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.49      0.37      0.42       193\n",
      "           1       0.54      0.65      0.59       216\n",
      "\n",
      "    accuracy                           0.52       409\n",
      "   macro avg       0.51      0.51      0.51       409\n",
      "weighted avg       0.52      0.52      0.51       409\n",
      "\n"
     ]
    }
   ],
   "source": [
    "cr_knn = classification_report(y_test,knn_pred )\n",
    "print(cr_knn)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b710ac4f",
   "metadata": {},
   "source": [
    "The accuracy of KNN is average as well as its F1 score, although it has done slightly better at detecting real pictures."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "id": "f71afe82",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfYAAAHSCAYAAAAe1umcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAVZklEQVR4nO3ce/DddX3n8dc7v4R7wl0BQWS7VfGKEAvWy4KKRV2L7FIV26m6KhdbHJxWy05dtU7bWSrjbpFZLVhL8bK4jta1loK3KOKtXEREqhYX1BAUFEFuhoR89o/fAUMISX4hcJK3j8dMJud8vt/zPe/fb/L9PfM95yQ1xggA0MO8aQ8AAGw6wg4AjQg7ADQi7ADQiLADQCPCDgCNzJ/2AA+1rWrrsU22n/YY0FY9esG0R4D2fv7d638yxth9bdt+5cK+TbbPwfWcaY8Bbc2csde0R4D2zj/0tO/f3zYvxQNAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4Ajcyf9gCwur3HLXlzvnrP/T1yW/4+j89uuSOH5LqszLwsy/Y5NYtzW201xUlhy/GGUz6Tg79ydW7aadscf9bvJUle8+4Lc/CXr87KBfOybK8d884/OTy3Ldw6C2++I29+67l59Levz6eP2D//66RDpzs8c7ZBV+xV9eKqGlX12A3Y96Sq2m5jB6qqV1bV6WtZr6o6raquqqrLq+rAjX0ONl9La2GOr8NzfB2e1+W5WZ6ZfCl75dI8LK/N4TmuDs+12SHH5NvTHhW2GJ8+Yv+8+a+OvNfapYv3yXF/97s54X2/m2v32Tkv/dDFSZI7t5qfs//L03LmCc+YxqhsAhv6UvwxSS6c/L4+JyXZ6LCvw/OT/Prk17FJ3v0gPAebkafkx7kuO+T62j6X1B5ZVbN/XP81u2a33DHl6WDLccWTH5FbFm5zr7VLn7pvVs2fPae+/bg9stsNtyZJlm+7IN960l5ZsdXMQz4nm8Z6w15VOyR5RpJXJ3nZauszVXVqVV0xuYI+sapen2SvJEuqaslkv1tXe8zRVXXW5PaLquprVfX1qvpMVT18PaMcmeTsMeurSXaqqj0nvy6oqssmszxzjt8DNlOHZmmWZJ/7rP9WrslF2WMKE0FPzzv3W7n4N/ad9hhsIhtyxX5kkvPGGN9N8tOqOmiyfmySRyU5YIzxpCQfHGOclmRZksPGGIet57gXJjlkjPGUJOckedN69n9Ekh+udn/pZO3lSc4fYxyQ5MlJLtuAr4nN3PyxKk/Lsnwhe99r/eXjX3NXKp/NI6c0GfTysvdflLtm5uVzhz9m2qOwiWzIh+eOSfLXk9vnTO5fkuS5Sd4zxliZJGOMG+f43Hsn+XBV7ZlkqyRXz/Hxd7soyfuqakGSj48xLltzh6o6NrN/Eck2D8q7BGxqT82PclV2yk31y5cPnzeuycG5Lm/Ks5KqKU4HPRz+z1fm4K9cnZPfeZRzqpF1XrFX1S5Jnp3kvVV1TZI3JnlJ1Zz+BIzVbq/+Js+7kpw+xnhikuPW2LY21yb3el127yTXjjEuSPKsyfazqur37zPAGGeMMRaPMRYvyNZzGJ1pOSw/yJLVrsoXjx/lJflO3pKnZ3n5xxzwQB30tWty9DmX5G1/+R+zfJsF0x6HTWh9PyGPTvL+McZxdy9U1ReSPDPJp5McV1VLxhgrq2qXyVX7LUkWJvnJ5CE/rqr9k3wnyVGT7UmyY2ZjnCSv2IBZP5HkD6vqnCQHJ7l5jHFdVe2bZOkY48yq2jrJgUnO3oDjsZnaZqzMQbk+/zMH3bP2h/l6FmRVTskFyZj9AN1f+4cRsEFOfvt5edJlS7Po5l/k/Uf/bT7wqkPy0g9enAUr7spf/tHHk8x+gO5df/TsJMnfv/Tvst3td2b+ilV52oXfy5+e+uL84FG7TvErYC7WF/ZjkpyyxtpHJ+snJnl0ksurakWSM5OcnuSMJOdV1bLJ++wnJ/lkkhuSXJxkh8lx3pbkI1X1sySfS7LfemY5N8kLklyV5PYkr5qsH5rkjZMZbk1ynyt2tiy/qPn5z/nte629sp4/pWlgy/ff33LEfdbOf+Hj73f/V3z4Vfe7jc1fjTHWv1cji2qXcXA9Z9pjQFszS/aa9gjQ3vmHnnbJGGPx2rb5L2UBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaARYQeARoQdABoRdgBoRNgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAamT/tAR5qNTOTmUU7TnsMaOvcx5w77RGgvZl1bHPFDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0Mj8aQ8Aq9v7rpvyX29bcs/9Pe66Je/f9sDsMO7MEcu/k5vnbZMkOWvbxblowT7TGhO2KPWGHyefvj3ZbSbj84+898b3/Czz/uynWXXFfsmuM8m/3Tm7/zeXZ5y8a3LCztMZmo22QVfsVfXiqhpV9dgN2PekqtpuYweqqldW1elrWX9sVX2lqpZX1R9v7PHZvC2d2Sl/sOio/MGio3LiwiOzvObnywv2TZL8wzZPuGebqMOGGy9ZlPGhPe+74doVqc/fnvGI1a7xdp6X8ee7J8cL+pZqQ1+KPybJhZPf1+ekJBsd9nW4Mcnrk5z6IBybzdABK5flunkLc/3MwmmPAlu2p22b7Dxzn+V6608y/ttuSa22uNv85IBtkgUP3XhsWusNe1XtkOQZSV6d5GWrrc9U1alVdUVVXV5VJ1bV65PslWRJVS2Z7Hfrao85uqrOmtx+UVV9raq+XlWfqaqHr2uOMcb1Y4yLkqxYY77tq+qfquobk1leusFfPZu1/3Dn/8vnt/q1e+7/9vIr8+6ffyxvuO2C7LBq+RQngwbOuzXZY37y+K2nPQmb2IZcsR+Z5LwxxneT/LSqDpqsH5vkUUkOGGM8KckHxxinJVmW5LAxxmHrOe6FSQ4ZYzwlyTlJ3rQxX0CSI5IsG2M8eYzxhCTnbeRx2IzMH3flkBU/yBe32i9J8smt98+rFv1OXrfwqNw4b7u89o6vTXlC2ILdvip12s8y3rTLtCfhQbAhYT8ms+HN5Pe7X45/bpK/GWOsTJIxxo1zfO69k5xfVd9M8sYkj5/j4+/2zSSHV9UpVfXMMcbNa+5QVcdW1cVVdfGd446NfBoeSotXLM1VM7vmpnnbJklumrdtVtW8jKqct9Vj8piVN0x5QtiCfX9F8oOVqef8MPXUa5LrVqae98Pk+pXTnoxNYJ2fiq+qXZI8O8kTq2okmUkyquqNc3iOsdrtbVa7/a4k7xxjfKKqDk3ytjkc85cHH+O7VXVgkhck+fOq+uwY4+1r7HNGkjOSZMf5u4+1HIbNzKF3fu9eL8Pvsur23Dhv9qMbv7ni+7lmxgd7YKPtv3XGFfvdc7eeek3GefvMfiqeLd76/rnb0UneP8Y47u6FqvpCkmcm+XSS46pqyRhjZVXtMrlqvyXJwiQ/mTzkx1W1f5LvJDlqsj1Jdkxy7eT2Kzb2C6iqvZLcOMb4QFXdlOQ1G3ssNg9bjxU5cOWynLb9M+5Ze/Ud/5J/t/LGpJIfz1uY07Z7+hQnhC1LnfCj5Mt3JDfelTrw6ow/3jV5+aK173z9ytQRP0xuWZWaV8mZN2V8Yd9kof/2ZEuxvrAfk+SUNdY+Olk/Mcmjk1xeVSuSnJnk9MxeGZ9XVcsm77OfnOSTSW5IcnGSHSbHeVuSj1TVz5J8Lsl+WYeq2mPy+EVJVlXVSUkel+SJSd5RVasy+8G6E9bzNbGZW14L8pKdfu9ea+/Y/tDpDAMNjHfvse7tFz3ql3ceNj/j0nX+OGYzV2P8ar0yveP83cfTFh057TGgrXOv/MK0R4D2Zva86pIxxuK1bfPaCgA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCN1Bhj2jM8pKrqhiTfn/YczMluSX4y7SGgOefZlmXfMcbua9vwKxd2tjxVdfEYY/G054DOnGd9eCkeABoRdgBoRNjZEpwx7QHgV4DzrAnvsQNAI67YAaARYWeDVdVdVXVZVV1RVR+pqu0ewLHOqqqjJ7ffW1WPW8e+h1bVb27Ec1xTVbutZf2gqvpmVV1VVadVVc312PBgaXSe/UVV/bCqbp3rMXlghJ25uGOMccAY4wlJ7kxy/Oobq2r+xhx0jPGaMcaV69jl0CRz/oGzDu9O8tokvz75dcQmPDY8UF3Os39M8hub8HhsIGFnY30xyb+f/C3/i1X1iSRXVtVMVb2jqi6qqsur6rgkqVmnV9V3quozSR5294Gq6vNVtXhy+4iqurSqvlFVn62qR2X2B9sbJlcxz6yq3avqo5PnuKiqnj557K5V9amq+lZVvTfJfa7Eq2rPJIvGGF8dsx8wOTvJiyfbXl9VV07mPudB/N7Bhtoiz7MkmZxj1625XlW/M3k14htVdcEm/n6RZKP+5sevtskVw/OTnDdZOjDJE8YYV1fVsUluHmM8taq2TvKlqvpUkqckeUySxyV5eJIrk7xvjePunuTMJM+aHGuXMcaNVfWeJLeOMU6d7PehJP9jjHFhVT0yyflJ9k/y1iQXjjHeXlUvTPLqtYz/iCRLV7u/dLKWJCcn2W+Msbyqdtr47xA8cFv4ebYub0nyW2OMa51nDw5hZy62rarLJre/mORvM/vS3b+MMa6erD8vyZPufl8vyY6Zfbn7WUn+9xjjriTLqupzazn+IUkuuPtYY4wb72eO5yZ53GpvjS+qqh0mz/GfJo/9p6r62Ry/vsuTfLCqPp7k43N8LGwq3c+zLyU5q6r+T5KPzfGxbABhZy7uGGMcsPrC5KS/bfWlJCeOMc5fY78XbMI55iU5ZIzxi7XMsj7XJtl7tft7T9aS5IWZ/aH1oiR/WlVPHGOsfODjwpx0OM/u1xjj+Ko6OLPn2yVVddAY46cP6KDci/fY2dTOT3JCVS1Ikqp6dFVtn+SCJC+dvDe4Z5LD1vLYryZ5VlXtN3nsLpP1W5IsXG2/TyU58e47VXXA5OYFSV4+WXt+kp3XfILJe34/r6pDavYn1O8n+b9VNS/JPmOMJUn+JLNXQDtsxNcPD4XN+jxbl6r6tTHG18YYb0lyQ5J95vJ41k/Y2dTem9n39S6tqiuS/E1mXxn6hyT/Ntl2dpKvrPnAMcYNSY5N8rGq+kaSD082/WOSo+7+UE+S1ydZPPnQ0JX55aeG/yyzP7C+ldmXCn9wPzO+bjLnVUm+l+Sfk8wk+UBVfTPJ15OcNsa4aaO/C/Dg2uzPs6r6q6pammS7qlpaVW+bbHpHzf5z0yuSfDnJNx7IN4L78j/PAUAjrtgBoBFhB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaOT/A5QjXvoDyZarAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "cm_knn = confusion_matrix(y_test,knn_pred)\n",
    "fig, ax = plt.subplots(figsize=(8, 8))\n",
    "ax.imshow(cm_knn)\n",
    "ax.grid(False)\n",
    "ax.xaxis.set(ticks=(0, 1), ticklabels=('Predicted 0s', 'Predicted 1s'))\n",
    "ax.yaxis.set(ticks=(0, 1), ticklabels=('Actual 0s', 'Actual 1s'))\n",
    "ax.set_ylim(1.5, -0.5)\n",
    "for i in range(2):\n",
    "    for j in range(2):\n",
    "        ax.text(j, i, cm_knn[i, j], ha='center', va='center', color='red')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "006f31ae",
   "metadata": {},
   "source": [
    "## Decision Tree Classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "id": "5e4a6d12",
   "metadata": {},
   "outputs": [],
   "source": [
    "dec_tree = DecisionTreeClassifier()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "id": "88383068",
   "metadata": {},
   "outputs": [],
   "source": [
    "params = {\"criterion\": [\"gini\"],\n",
    "       \"max_depth\": range(4,6),\n",
    "       \"min_samples_split\": range(30, 35),\n",
    "       \"min_samples_leaf\": range(75,80)}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "id": "840c2507",
   "metadata": {},
   "outputs": [],
   "source": [
    "grid = GridSearchCV(dec_tree, param_grid = params)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "id": "dd22d2ce",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(estimator=DecisionTreeClassifier(),\n",
       "             param_grid={'criterion': ['gini'], 'max_depth': range(4, 6),\n",
       "                         'min_samples_leaf': range(75, 80),\n",
       "                         'min_samples_split': range(30, 35)})"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "id": "9f570632",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'criterion': 'gini',\n",
       " 'max_depth': 4,\n",
       " 'min_samples_leaf': 76,\n",
       " 'min_samples_split': 31}"
      ]
     },
     "execution_count": 103,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "id": "aa57e1c5",
   "metadata": {},
   "outputs": [],
   "source": [
    "tree = DecisionTreeClassifier(criterion = \"gini\", max_depth = 4,min_samples_leaf = 76, min_samples_split = 31)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "id": "0a6cae57",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeClassifier(max_depth=4, min_samples_leaf=76, min_samples_split=31)"
      ]
     },
     "execution_count": 105,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tree.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "id": "5ac116a6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5906862745098039"
      ]
     },
     "execution_count": 106,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tree.score(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "id": "59958bd3",
   "metadata": {},
   "outputs": [],
   "source": [
    "tree_pred = tree.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "id": "508eeaf1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5427872860635696"
      ]
     },
     "execution_count": 108,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "acc_tree = tree.score(X_test, y_test)\n",
    "acc_tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "id": "b3ea38a2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6504672897196262"
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "f1_tree = f1_score(y_test, tree_pred)\n",
    "f1_tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "id": "95e57f52",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.53      0.25      0.34       193\n",
      "           1       0.55      0.81      0.65       216\n",
      "\n",
      "    accuracy                           0.54       409\n",
      "   macro avg       0.54      0.53      0.49       409\n",
      "weighted avg       0.54      0.54      0.50       409\n",
      "\n"
     ]
    }
   ],
   "source": [
    "cr_tree = classification_report(y_test,tree_pred )\n",
    "print(cr_tree)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5cfaec8a",
   "metadata": {},
   "source": [
    "The combined F1 score for both classes is higher than average although the Decision Tree also did better on real pictures than detecting fake one, the model didn't overfit and its accuracy on the test set is average."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "id": "e4eb747a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfYAAAHSCAYAAAAe1umcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAV/UlEQVR4nO3cefTddX3n8dc7gQAJJBCBCiKCC65QiojYQQXHBaSMYHUU26lOa0FmqoPWraeLaNvjoLghPVq11uJSHMeOuIKKWEAFFWWTqYqHKYRVVklACMln/vhdIISYDeIvvzePxzk53Pv53vu97/s7uff5+977JTXGCADQw6zpHgAAeOAIOwA0IuwA0IiwA0Ajwg4AjQg7ADSyyXQP8Os2Z9bmY4tZW033GNDW5rstm+4RoL1r/++N140xtlvVtgdd2LeYtVWeNv8F0z0GtLXbJ5dM9wjQ3vv3Ounff9U2H8UDQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI1sMt0DwMpmjeU5/paTc/2seXnLls/NnkuvzCtv+24qI7/Mpjlu3jNy1ez50z0mzBgvO+acPPHMK3PLws3zPz9z0L22HfDxf8th7zkvf3baYVmyzWZ59PevyR+/7qxcv+O8JMkFz9oppxzxpOkYm/W0VkfsVXVoVY2qetxa3Pboqpq7vgNV1Suq6oRVrFdVHV9Vl1TVBVW11/o+Bhu3Q2//US6ftfXd1//k1m/l2Hn757/PPyynz3lkXvbL86ZtNpiJzjlk13zghGfeZ33rq5fkcd+5Ojc89N5v2T/bc7u846QD846TDhT1GWhtP4o/PMlZk/+uydFJ1jvsq3FQksdM/hyR5AMb4DGYZtsuX5KnLL08p2z22BVWK3PHHUmSeWNprp+1If56QV8/e/L2uXXBnPusv/BdP8zJR/9mRk3DUGwwa/wovqq2TLJfkgOSfCHJWybrs5Mcm+TAJMuTfDhJJdkxyelVdd0Y44CqWjzG2HJynxcl+Z0xxiuq6pAkf5FkTpLrk/zeGOOa1YzygiQnjjFGkrOrauuq2mGy7dNJ5k+ez1FjjDPX6afARuPIW8/OP2yxT+aOpXevvWfufvnrxV/N7ZmdW2tOXjv/kGmcEHrY/ZuLctP2c3PlbtvcZ9uuF16XN73klNy83Rb53Gv3zNWPWjANE7K+1uaI/QVJThlj/CTJ9VX15Mn6EUl2SbLnGGOPJJ8cYxyf5MokB4wxDljDfs9Ksu8Y47eSnJTkjWu4/cOSXL7C9UWTtZclOXWMsWeS30xy3lo8JzZC+9xxWW6atXku2WTbe62/8PaL8pdbPjf/ZevD87XNHpMjbj1nmiaEHja97c4856MX58uvuu/H7IsetzBv+dIhOfbTB+aMlz4mr3yd46SZZm1Onjs8yfsml0+aXD83ybOTfHCMcWeSjDFuWMfH3inJpydH3XOSXLqO97/L95J8tKo2TfK5McZ5K9+gqo7I1C8i2XzWvPV8GDa0Jy67JvvecVn2Wboom45lmTvuyNtuOTU7Lb85P95k+yTJv855ZP7mllOneVKY2bZdtDgPuWJJ3vTSU5IkW197W97we6fmXSc+J7dsu8Xdt7t4vx3z4rd/P/NuvD1LttlsusZlHa027FW1MMmzkuxeVSPJ7CSjqt6wDo8xVri8+QqX35/k3WOMz1fV/kmOWcN+rkjy8BWu75TkijHGVVX1jCQHJ/lYVb17jHHivQYY40NJPpQkCzbZbsV52Ij84xZPyT9u8ZQkyR5Lr8rv3n5h3jrv2fnnmz+Vhy27OVfMXpC9ll6Ry2dvPb2Dwgx31WO2zp+fdtjd199y8Odz3CeelyXbbJatrrsttzxk86QqO190fWokS7a+7/fzbLzWdMT+oiQfH2MceddCVf1rkqcn+VqSI6vq9DHGnVW1cHLUfkuSrZJcN7nLNVX1+CQ/TnLYZHuSLMhUrJPk5Wsx6+eT/ElVnZTkqUlunkT9EUkWjTE+XFWbJdkryYmr2xEzx/KalffN3S9/sfi0jKosrjl599ynT/dYMKO8/M++nUefe222vOn2vO3Ak/PlVz0pZx/6qFXeds+vX579/vclWT57VpZuNjv/9PbfTsrZdTPJmsJ+eKZOkFvRZyfrr06yW5ILqmpppk6eOyFTR8anVNWVk+/Z35zki0l+nuT7Sbac7OeYJJ+pqhuTfCPJrmuY5ctJnp/kkiS3Jvmvk/X9k7xhMsPiJH+whv0wA1yw6Q65YNOpcyO/PWeXfHvOLtM7EMxg//T2317t9rd+6T/dffnMl+6WM1+624YeiQ2opk4yf/BYsMl242nzXzDdY0Bbu522ZLpHgPbev9dJ544x9l7VNv+kLAA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCObTPcAv25j2bIsu+nm6R4D2nrvDudN9wjQ3vtXs80ROwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCObTPcAsLJZY+Tvclquy+b5y9ovbx7nZLfcmDszKz/Owrw3e2VZ+Z0U1la99prka7cm287O+ObOU2tHXp387I6pG9y8PFkwK+PrO99zp0VLU8+8LOP1C5OjtpmGqVlfa/XuWFWHVtWoqsetxW2Prqq56ztQVb2iqk5Yxfrjquo7VXV7Vb1+fffPxu+w/DSXZau7r38jO+cP87wckedksyzLQbl0GqeDmWf85/kZn9rh3mt//9CMr+88FfODt8x4/pb32l7HXJc8a73fyplGa3vYc3iSsyb/XZOjk2yIvw03JHlNkuM2wL7ZSGw7bs1Tc1W+kl3vXvtu7ZBUJVX5t2yT7XLbNE4IM9DTtki2mb3qbWMkX1icHLpC2L+yONl50+Sxc3498/GAWmPYq2rLJPsl+aMkL11hfXZVHVdVF1XVBVX16qp6TZIdk5xeVadPbrd4hfu8qKo+Nrl8SFWdU1U/rKqvV9VvrG6OMca1Y4zvJVm60nzzqupLVXX+ZJaXrPWzZ6NzVM7Ph7NHlq9i2+yxPM/OZfleVvtXBVgXZ/8y2XZ28shJxJcsT/3djRl/unB652K9rc137C9IcsoY4ydVdX1VPXmMcW6SI5LskmTPMcadVbVwjHFDVb0uyQFjjOvWsN+zkuw7xhhV9cokb0zyp+vxHA5McuUY4+AkqaoF67EPNgJPHVfmpmyWn9Y22WNce5/tr8kPc2G2zUW13TRMBz3V527JOOyeo/U67oaMI7ZO5jmPZaZam7AfnuR9k8snTa6fm+TZST44xrgzScYYN6zjY++U5NNVtUOSOcl6f3F6YZJ3VdWxSb44xjhz5RtU1RGZ+kUkm2+Qbwl4IDwx1+dpuSr7jC9nTpZlbu7Mm8Z3c2ztk98fF2dBbs9787TpHhP6uHMkX16SnPrwe9Z+8MvUFxcnf3198ovlqVnJ2KySP9x62sZk3aw27FW1MMmzkuxeVSPJ7CSjqt6wDo8xVri8+QqX35/k3WOMz1fV/kmOWYd93rPzqU8S9kry/CR/U1WnjTHettJtPpTkQ0kyvxaOVeyGjcBHa/d8NLsnSfYY1+bF+UmOrX1y0Lg0e+fqvDHPzKia5imhkTNuTR69abLjPSkYJ+909+U67vqMebNEfYZZ02ctL0ry8THGI8YYu4wxHp6pI+unJ/lakiOrapPk7l8CkuSWZIVTmpNrqurxVTUryWErrC9IcsXk8svX9wlU1Y5Jbh1jfCLJO5Pstb77YuP0P/KDbJPbc3y+kQ+Or+X3x8XTPRLMKHXU1anfWZT87I7UXpcmn/rF1PrJizMO3WoN92amWdNH8YcnOXaltc9O1l+dZLckF1TV0iQfTnJCpo6MT6mqK8cYByR5c5IvJvl5ku8nuevLnGOSfKaqbkzyjWSF06BXoaoeOrn//CTLq+roJE9IsnuSd1bV8kydWHfUGp4TM8AFtX0uyPZJkgPrd6d5GpjZxgceuur1963+RNTx+odsiHHYwGqMB9cn0/Nr4Xhq/cfpHgPaOvXK86Z7BGhv9g6XnDvG2HtV25z2CACNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0Ajwg4AjQg7ADQi7ADQiLADQCPCDgCNCDsANCLsANCIsANAI8IOAI0IOwA0IuwA0IiwA0AjNcaY7hl+rarq50n+fbrnYJ1sm+S66R4CmvM6m1keMcbYblUbHnRhZ+apqu+PMfae7jmgM6+zPnwUDwCNCDsANCLszAQfmu4B4EHA66wJ37EDQCOO2AGgEWFnrVXVsqo6r6ouqqrPVNXc+7Gvj1XViyaXP1JVT1jNbfevqt9ej8f4f1W17SrWn1xVF1bVJVV1fFXVuu4bNpRGr7O/rarLq2rxuu6T+0fYWRe3jTH2HGM8KckdSV614saq2mR9djrGeOUY4+LV3GT/JOv8hrMaH0jyx0keM/lz4AO4b7i/urzOvpBknwdwf6wlYWd9nZnk0ZPf8s+sqs8nubiqZlfVO6vqe1V1QVUdmSQ15YSq+nFVfT3J9nftqKq+WVV7Ty4fWFU/qKrzq+q0qtolU29sr50cxTy9qrarqs9OHuN7VfUfJvd9SFV9tap+VFUfSXKfI/Gq2iHJ/DHG2WPqBJMTkxw62faaqrp4MvdJG/BnB2trRr7OkmTyGrtq5fWqevHk04jzq+qMB/jnRZL1+s2PB7fJEcNBSU6ZLO2V5EljjEur6ogkN48xnlJVmyX5VlV9NclvJXlskick+Y0kFyf56Er73S7Jh5M8Y7KvhWOMG6rqg0kWjzGOm9zuU0neM8Y4q6p2TnJqkscneUuSs8YYb6uqg5P80SrGf1iSRStcXzRZS5I3J9l1jHF7VW29/j8huP9m+Otsdf4qyfPGGFd4nW0Yws662KKqzptcPjPJP2Tqo7vvjjEunaw/N8ked32vl2RBpj7ufkaSfx5jLEtyZVV9YxX73zfJGXfta4xxw6+Y49lJnrDCV+Pzq2rLyWO8cHLfL1XVjev4/C5I8smq+lySz63jfeGB0v119q0kH6uq/5XkX9bxvqwFYWdd3DbG2HPFhcmLfsmKS0lePcY4daXbPf8BnGNWkn3HGL9cxSxrckWSnVa4vtNkLUkOztSb1iFJ/ryqdh9j3Hn/x4V10uF19iuNMV5VVU/N1Ovt3Kp68hjj+vu1U+7Fd+w80E5NclRVbZokVbVbVc1LckaSl0y+G9whyQGruO/ZSZ5RVbtO7rtwsn5Lkq1WuN1Xk7z6ritVtefk4hlJXjZZOyjJNis/wOQ7v19U1b419Q71B0lOrqpZSR4+xjg9yZsydQS05Xo8f/h12KhfZ6tTVY8aY5wzxvirJD9P8vB1uT9rJuw80D6Sqe/1flBVFyX5+0x9MvR/kvx0su3EJN9Z+Y5jjJ8nOSLJv1TV+Uk+Pdn0hSSH3XVST5LXJNl7ctLQxbnnrOG3ZuoN60eZ+qjwsl8x43+bzHlJkp8l+UqS2Uk+UVUXJvlhkuPHGDet908BNqyN/nVWVe+oqkVJ5lbVoqo6ZrLpnTX1v5telOTbSc6/Pz8I7su/PAcAjThiB4BGhB0AGhF2AGhE2AGgEWEHgEaEHQAaEXYAaETYAaCR/w9g7KRxcMPpnAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "cm_tree = confusion_matrix(y_test,tree_pred)\n",
    "fig, ax = plt.subplots(figsize=(8, 8))\n",
    "ax.imshow(cm_tree)\n",
    "ax.grid(False)\n",
    "ax.xaxis.set(ticks=(0, 1), ticklabels=('Predicted 0s', 'Predicted 1s'))\n",
    "ax.yaxis.set(ticks=(0, 1), ticklabels=('Actual 0s', 'Actual 1s'))\n",
    "ax.set_ylim(1.5, -0.5)\n",
    "for i in range(2):\n",
    "    for j in range(2):\n",
    "        ax.text(j, i, cm_tree[i, j], ha='center', va='center', color='red')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cdae6b7c",
   "metadata": {},
   "source": [
    "The Decision tree did also better at detecting real images"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb421d14",
   "metadata": {},
   "source": [
    "## Models comparision"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "id": "3d0f5689",
   "metadata": {},
   "outputs": [],
   "source": [
    "acc = [acc_lr, acc_nb, acc_svc, acc_xgb, acc_knn, acc_tree]\n",
    "f1 = [f1_lr, f1_nb, f1_svc, f1_xgb, f1_knn, f1_tree]\n",
    "models = [\"LR\", \"NB\", \"SVC\", \"XGB\", \"KNN\", \"DT\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "id": "50865274",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x25180e7b7f0>"
      ]
     },
     "execution_count": 113,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeMAAAHwCAYAAACVNQcNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAkPUlEQVR4nO3de7gV9X3v8fdX0GDEGC+YGkGhqfGRIKBBvCamjcZLFKMkFWs1eozYk6Jpo22xzfGWtE1q1SSNsXri3QoaW5BEeqQGPcaaKJgQFS8BDQrERMQrIkHg2z/WbLNc2bgXuvb+7b14v55nP6yZ+c3Md4a99mf9ZmbNRGYiSZLK2aR0AZIkbewMY0mSCjOMJUkqzDCWJKkww1iSpMIMY0mSCjOMpQ0QEe+LiLsj4pWIuKh0PaVFxOYR8b2IeCkivtuD670rIj7XZNuMiD/o7pqkd8IwVtuLiEUR8VpErIiIX0fENREx8G0ubiLwHPCezDyzhWX2VZ8G3gdsm5mfaZwYEedVYfiFhvFfqMaf10N1Sr2aYayNxZGZORDYExgDfGlDZo6aTYCdgUfybdwtJyL6b+g8fcDOwM8zc81btPk5cGLDuM9W4yVhGGsjk5lLgf8ERgBExD4RcW9EvBgRP4uIj3W0rQ6F/n1E/DewEriOWoj8ddXLPigi3hURX4+IX1Y/X4+Id1XzfywilkTE30TEr4Crq57idyPihupQ90MR8cGIODsino2IxRHxiboaTo6IR6u2T0bEaXXTOpZ/ZjXvMxFxct30zSPiooh4qjqMfE9EbN7VdjeKiN2qffFiRMyPiHHV+POBc4Bjq/1xynoWMQd4d0R8qJrvQ8CAanz9ek6NiIUR8XxEzIiI99dNOzgiHqu241tANMz7v6r99EJE3B4RO69nWw6PiEeq/bk0Is5a33ZLPckw1kYlIoYAhwM/jYgdgduArwDbAGcB/x4Rg+pmOYHaoektgZOBfwP+KTMHZuYdwN8B+wCjgVHAWN7c6/69atk7V8sBOBK4Htga+ClwO7X34o7ABcDldfM/CxwBvKda/yURsWfD8req5j0FuDQitq6m/TPwYWC/qoa/BtY1ud0d+2tT4HvALGB74HTg3yJi18w8F/gH4KZqf1zZOH+d6/lt7/iz1XD9ev4I+Efgj4EdgKeAqdW07YD/oLZftwOeAPavm/co4G+BY4BBwA+BKeup40rgtMzcktoHstlvUbPUczLTH3/a+gdYBKwAXqT2R/7bwObA3wDXN7S9Hfhs9fou4IKG6dcAX6kbfgI4vG74EGBR9fpjwGpgQN3084D/qhs+sqqtXzW8JZDAe9ezLdOBL9Qt/zWgf930Z6l9ONikmjaqk2W85XY3jP8I8Ctgk7pxU4Dz6rbnhrfY9+cBNwA7AU8Dm1b/DqnGdyznSmofcjrmGwi8DgylFuI/rpsWwBLgc9XwfwKn1E3fhNqRjJ2r4QT+oHr9NHAatXP+xX83/fGn48eesTYWn8rM92bmzpn5+cx8jVpv9TPV4dcXI+JF4ABqPbMOi7tY7vupBXyHp6pxHZZl5qqGeX5d9/o14LnMXFs3DLUwIiIOi4gfV4duX6TWq9+ubv7l+ebztSurebejdij4iU5qbma767dvcWaua9jGHTtpu16Z+TSwkFpPekFmNu7XN+3HzFwBLK/W837q/h8yM3nz/8vOwDfqtuV5aoHdWY3jqe3DpyLi/0fEvhuyHVJ3accLSqRmLabWQzz1Ldp0daHWL6mFwfxqeKdqXLPzr1d17vnfqfUMb83M1yNiOg3nS9fjOWAV8AHgZw3TmtnuDr8EhkTEJnWBvBNv7+Kr64CrqB1u72w9b5znjYgtgG2BpcAz1HrSHdOifpja9vx9Zv5bVwVk5hzgqOrw+yTg5oZlSUXYM9bG7AbgyIg4JCL6RcSA6qKowRuwjCnAlyJiUHVu85xqua2wGfAuYBmwJiIOAz7x1rPUVMF5FXBxRLy/2r59q4DfkO2+j1pv+68jYtPqQq8jqc7nbqCbqvpv7mTaFODkiBhd1fgPwH2ZuYja+e0PRcQxUbsi/Qxq58o7/Ctwdt0FYltFRGdfs9osIo6PiK0y83XgZWBdYzupBMNYG63qUGnHxT/LqPWw/ooNe198BZgLPAg8BPykGteK+l6hFjw3Ay8AfwLM2IBFnFXVNIfaoduvUTv32/R2Z+ZqauF7GLXe9reBEzPzsbexPa9l5h3VKYLGaXcA/4fakYBnqPXoJ1TTngM+A3yV2qHrXYD/rpt3WrVtUyPiZeDhqt7OnAAsqtr9GXD8hm6H1B2idvpFkiSVYs9YkqTCDGNJkgozjCVJKswwliSpMMNYkqTCit30Y7vttsuhQ4eWWr0kST3qgQceeC4zf+ce8FAwjIcOHcrcuXNLrV6SpB4VEU+tb5qHqSVJKswwliSpMMNYkqTCetVTm15//XWWLFnCqlWNT5xTswYMGMDgwYPZdNNNS5ciSWpSrwrjJUuWsOWWWzJ06FBqT0nThshMli9fzpIlSxg2bFjpciRJTepVh6lXrVrFtttuaxC/TRHBtttu65EFSepjelUYAwbxO+T+k6S+p9eFcW8wffp0IoLHHtvgR7ZKkrTBetU540ZDJ9/W0uUt+uonm2o3ZcoUDjjgAKZMmcL555/f0ho6rF27ln79+nXLsiVJfYs94wYrVqzgnnvu4corr2Tq1KlALTjPOussRowYwciRI/mXf/kXAObMmcN+++3HqFGjGDt2LK+88grXXHMNkyZNemN5RxxxBHfddRcAAwcO5Mwzz2TUqFH86Ec/4oILLmCvvfZixIgRTJw4kcwEYOHChRx00EGMGjWKPffckyeeeIITTzyR6dOnv7Hc448/nltvvbVndookqVv16p5xCbfeeiuHHnooH/zgB9l222154IEHuP/++1m0aBHz5s2jf//+PP/886xevZpjjz2Wm266ib322ouXX36ZzTff/C2X/eqrr7L33ntz0UUXATB8+HDOOeccAE444QS+//3vc+SRR3L88cczefJkjj76aFatWsW6des45ZRTuOSSS/jUpz7FSy+9xL333su1117b7ftDktT97Bk3mDJlChMmTABgwoQJTJkyhTvuuIPTTjuN/v1rn1222WYbHn/8cXbYYQf22msvAN7znve8MX19+vXrx/jx498YvvPOO9l7773ZfffdmT17NvPnz+eVV15h6dKlHH300UDte8Pvfve7OfDAA1mwYAHLli1jypQpjB8/vsv1SZL6Bv+a13n++eeZPXs2Dz30EBHB2rVriYg3ArcZ/fv3Z926dW8M13/NaMCAAW+cJ161ahWf//znmTt3LkOGDOG8887r8itJJ554IjfccANTp07l6quv3sCtkyT1VvaM69xyyy2ccMIJPPXUUyxatIjFixczbNgwRo0axeWXX86aNWuAWmjvuuuuPPPMM8yZMweAV155hTVr1jB06FDmzZvHunXrWLx4Mffff3+n6+oI3u22244VK1Zwyy23ALDlllsyePDgN84P/+Y3v2HlypUAnHTSSXz9618Haoe4JUntwTCuM2XKlDcOD3cYP348zzzzDDvttBMjR45k1KhR3HjjjWy22WbcdNNNnH766YwaNYqDDz6YVatWsf/++zNs2DCGDx/OGWecwZ577tnput773vdy6qmnMmLECA455JA39b6vv/56vvnNbzJy5Ej2228/fvWrXwHwvve9j912242TTz65+3aCJKnHRccVvD1tzJgx2fg840cffZTddtutSD19wcqVK9l99935yU9+wlZbbbXedu5HSep9IuKBzBzT2TR7xn3EHXfcwW677cbpp5/+lkEsSep7vICrjzjooIN46qmnSpchSeoG9owlSSrMnrEkqW84r4dP0Z33Uo+typ6xJEmFGcaSJBVmGDfo168fo0ePfuNn0aJFLF++nD/8wz9k4MCBb3oIhCRJrdC7zxm3+vxAE8f/N998c+bNm/emca+++ipf/vKXefjhh3n44YdbW9N6ZCaZySab+HmpLbTxuS5J75x/6ZuwxRZbcMABBzBgwIC3bDd58mSGDx/OyJEjOeusswD49a9/zdFHH82oUaMYNWoU9957LwAXX3wxI0aMYMSIEW/c4nLRokXsuuuunHjiiYwYMYLFixdz4YUXstdeezFy5EjOPffcbt1OSVIZvbtnXMBrr73G6NGjARg2bBjTpk1rar7ly5czbdo0HnvsMSKCF198EYAzzjiDAw88kGnTprF27VpWrFjBAw88wNVXX819991HZrL33ntz4IEHsvXWW7NgwQKuvfZa9tlnH2bNmsWCBQu4//77yUzGjRvH3XffzUc/+tFu2npJUgmGcYPODlM3Y6uttmLAgAGccsopHHHEERxxxBEAzJ49m+uuuw6onY/eaqutuOeeezj66KPZYostADjmmGP44Q9/yLhx49h5553ZZ599AJg1axazZs1ijz32AGDFihUsWLDAMJakNmMYt0j//v25//77+cEPfsAtt9zCt771LWbPnr3By+kIaKidNz777LM57bTTWlmqJKmX8Zxxi6xYsYKXXnqJww8/nEsuuYSf/exnAHz84x/nsssuA2Dt2rW89NJLfOQjH2H69OmsXLmSV199lWnTpvGRj3zkd5Z5yCGHcNVVV7FixQoAli5dyrPPPttzGyVJ6hH2jJs0dOhQXn75ZVavXs306dOZNWvWm54p/Morr3DUUUexatUqMpOLL74YgG984xtMnDiRK6+8kn79+nHZZZex7777ctJJJzF27FgAPve5z7HHHnuwaNGiN63zE5/4BI8++ij77rsvAAMHDuSGG25g++2375mNliT1CB+h2Ibcj72QX22S3rk+/j7yEYqSJPVihrEkSYUZxpIkFdbrwrjUOex24f6TpL6nV4XxgAEDWL58uYHyNmUmy5cv7/K2nZKk3qVXfbVp8ODBLFmyhGXLlpUupc8aMGAAgwcPLl2GJGkD9Kow3nTTTRk2bFjpMiRJ6lG96jC1JEkbI8NYkqTCDGNJkgozjCVJKswwliSpMMNYkqTCetVXm/qUPv70EElS72HPWJKkwgxjSZIKayqMI+LQiHg8IhZGxOT1tPnjiHgkIuZHxI2tLVOSpPbV5TnjiOgHXAocDCwB5kTEjMx8pK7NLsDZwP6Z+UJEbN9dBUuS1G6a6RmPBRZm5pOZuRqYChzV0OZU4NLMfAEgM59tbZmSJLWvZsJ4R2Bx3fCSaly9DwIfjIj/jogfR8ShrSpQkqR216qvNvUHdgE+BgwG7o6I3TPzxfpGETERmAiw0047tWjVkiT1bc30jJcCQ+qGB1fj6i0BZmTm65n5C+Dn1ML5TTLziswck5ljBg0a9HZrliSprTQTxnOAXSJiWERsBkwAZjS0mU6tV0xEbEftsPWTrStTkqT21WUYZ+YaYBJwO/AocHNmzo+ICyJiXNXsdmB5RDwC3An8VWYu766iJUlqJ02dM87MmcDMhnHn1L1O4IvVjyRJ2gDegUuSpMJ8UIQk9TY+iGajYxhLKsfQkQDDWBupoZNv69H1LRrQo6uT1Md4zliSpMIMY0mSCjOMJUkqzDCWJKkww1iSpMIMY0mSCjOMJUkqzDCWJKkww1iSpMK8A5day9sbStIGs2csSVJhhrEkSYUZxpIkFWYYS5JUmGEsSVJhhrEkSYUZxpIkFWYYS5JUmGEsSVJhhrEkSYUZxpIkFWYYS5JUmGEsSVJhhrEkSYUZxpIkFWYYS5JUmGEsSVJh/UsXIEnqm4ZOvq1H17doQI+urkfZM5YkqTDDWJKkwgxjSZIKM4wlSSrMC7gkvcELcqQy7BlLklSYYSxJUmGGsSRJhRnGkiQV5gVcbc4LciSp97NnLElSYYaxJEmFGcaSJBXmOWNJ6oLXXqi72TOWJKkww1iSpMIMY0mSCjOMJUkqzDCWJKkww1iSpMIMY0mSCjOMJUkqzDCWJKmwpu7AFRGHAt8A+gHfycyvNkw/CbgQWFqN+lZmfqeFdXbJO+RIkvqqLsM4IvoBlwIHA0uAORExIzMfaWh6U2ZO6oYaJUlqa80cph4LLMzMJzNzNTAVOKp7y5IkaePRTBjvCCyuG15SjWs0PiIejIhbImJIZwuKiIkRMTci5i5btuxtlCtJUvtp1QVc3wOGZuZI4L+AaztrlJlXZOaYzBwzaNCgFq1akqS+rZkwXgrU93QH89sLtQDIzOWZ+Ztq8DvAh1tTniRJ7a+ZMJ4D7BIRwyJiM2ACMKO+QUTsUDc4Dni0dSVKktTeuryaOjPXRMQk4HZqX226KjPnR8QFwNzMnAGcERHjgDXA88BJ3VizJEltpanvGWfmTGBmw7hz6l6fDZzd2tIkSdo4eAcuSZIKM4wlSSrMMJYkqTDDWJKkwgxjSZIKM4wlSSrMMJYkqTDDWJKkwgxjSZIKM4wlSSrMMJYkqTDDWJKkwgxjSZIKM4wlSSrMMJYkqTDDWJKkwgxjSZIKM4wlSSrMMJYkqTDDWJKkwgxjSZIKM4wlSSrMMJYkqTDDWJKkwgxjSZIKM4wlSSrMMJYkqTDDWJKkwgxjSZIKM4wlSSrMMJYkqTDDWJKkwgxjSZIKM4wlSSrMMJYkqTDDWJKkwgxjSZIKM4wlSSrMMJYkqTDDWJKkwgxjSZIKM4wlSSrMMJYkqTDDWJKkwgxjSZIKM4wlSSrMMJYkqTDDWJKkwgxjSZIKM4wlSSrMMJYkqTDDWJKkwgxjSZIKM4wlSSqsqTCOiEMj4vGIWBgRk9+i3fiIyIgY07oSJUlqb12GcUT0Ay4FDgOGA8dFxPBO2m0JfAG4r9VFSpLUzprpGY8FFmbmk5m5GpgKHNVJuy8DXwNWtbA+SZLaXjNhvCOwuG54STXuDRGxJzAkM29rYW2SJG0U3vEFXBGxCXAxcGYTbSdGxNyImLts2bJ3umpJktpCM2G8FBhSNzy4GtdhS2AEcFdELAL2AWZ0dhFXZl6RmWMyc8ygQYPeftWSJLWRZsJ4DrBLRAyLiM2ACcCMjomZ+VJmbpeZQzNzKPBjYFxmzu2WiiVJajNdhnFmrgEmAbcDjwI3Z+b8iLggIsZ1d4GSJLW7/s00ysyZwMyGceesp+3H3nlZkiRtPLwDlyRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhRnGkiQVZhhLklRYU2EcEYdGxOMRsTAiJncy/c8i4qGImBcR90TE8NaXKklSe+oyjCOiH3ApcBgwHDiuk7C9MTN3z8zRwD8BF7e6UEmS2lUzPeOxwMLMfDIzVwNTgaPqG2Tmy3WDWwDZuhIlSWpv/ZtosyOwuG54CbB3Y6OI+HPgi8BmwB+1pDpJkjYCLbuAKzMvzcwPAH8DfKmzNhExMSLmRsTcZcuWtWrVkiT1ac2E8VJgSN3w4Grc+kwFPtXZhMy8IjPHZOaYQYMGNV2kJEntrJkwngPsEhHDImIzYAIwo75BROxSN/hJYEHrSpQkqb11ec44M9dExCTgdqAfcFVmzo+IC4C5mTkDmBQRBwGvAy8An+3OoiVJaifNXMBFZs4EZjaMO6fu9RdaXJckSRsN78AlSVJhhrEkSYUZxpIkFWYYS5JUmGEsSVJhhrEkSYUZxpIkFWYYS5JUmGEsSVJhhrEkSYUZxpIkFWYYS5JUmGEsSVJhhrEkSYUZxpIkFWYYS5JUmGEsSVJhhrEkSYUZxpIkFWYYS5JUmGEsSVJhhrEkSYUZxpIkFWYYS5JUmGEsSVJhhrEkSYUZxpIkFWYYS5JUmGEsSVJhhrEkSYUZxpIkFWYYS5JUmGEsSVJhhrEkSYUZxpIkFWYYS5JUmGEsSVJhhrEkSYUZxpIkFWYYS5JUmGEsSVJhhrEkSYUZxpIkFWYYS5JUmGEsSVJhhrEkSYUZxpIkFWYYS5JUmGEsSVJhhrEkSYUZxpIkFWYYS5JUmGEsSVJhhrEkSYUZxpIkFdZUGEfEoRHxeEQsjIjJnUz/YkQ8EhEPRsQPImLn1pcqSVJ76jKMI6IfcClwGDAcOC4ihjc0+ykwJjNHArcA/9TqQiVJalfN9IzHAgsz88nMXA1MBY6qb5CZd2bmymrwx8Dg1pYpSVL7aiaMdwQW1w0vqcatzynAf3Y2ISImRsTciJi7bNmy5quUJKmNtfQCroj4U2AMcGFn0zPziswck5ljBg0a1MpVS5LUZ/Vvos1SYEjd8OBq3JtExEHA3wEHZuZvWlOeJEntr5me8Rxgl4gYFhGbAROAGfUNImIP4HJgXGY+2/oyJUlqX12GcWauASYBtwOPAjdn5vyIuCAixlXNLgQGAt+NiHkRMWM9i5MkSQ2aOUxNZs4EZjaMO6fu9UEtrkuSpI2Gd+CSJKkww1iSpMIMY0mSCjOMJUkqzDCWJKkww1iSpMIMY0mSCjOMJUkqzDCWJKkww1iSpMIMY0mSCjOMJUkqzDCWJKkww1iSpMIMY0mSCjOMJUkqzDCWJKkww1iSpMIMY0mSCjOMJUkqzDCWJKkww1iSpMIMY0mSCjOMJUkqzDCWJKkww1iSpMIMY0mSCjOMJUkqzDCWJKkww1iSpMIMY0mSCjOMJUkqzDCWJKkww1iSpMIMY0mSCjOMJUkqzDCWJKkww1iSpMIMY0mSCjOMJUkqzDCWJKkww1iSpMIMY0mSCjOMJUkqzDCWJKkww1iSpMIMY0mSCjOMJUkqzDCWJKkww1iSpMIMY0mSCjOMJUkqzDCWJKkww1iSpMKaCuOIODQiHo+IhRExuZPpH42In0TEmoj4dOvLlCSpfXUZxhHRD7gUOAwYDhwXEcMbmj0NnATc2OoCJUlqd/2baDMWWJiZTwJExFTgKOCRjgaZuaiatq4bapQkqa01c5h6R2Bx3fCSapwkSWqBHr2AKyImRsTciJi7bNmynly1JEm9VjNhvBQYUjc8uBq3wTLziswck5ljBg0a9HYWIUlS22kmjOcAu0TEsIjYDJgAzOjesiRJ2nh0GcaZuQaYBNwOPArcnJnzI+KCiBgHEBF7RcQS4DPA5RExvzuLliSpnTRzNTWZOROY2TDunLrXc6gdvpYkSRvIO3BJklSYYSxJUmGGsSRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhRnGkiQVZhhLklSYYSxJUmGGsSRJhTUVxhFxaEQ8HhELI2JyJ9PfFRE3VdPvi4ihLa9UkqQ21WUYR0Q/4FLgMGA4cFxEDG9odgrwQmb+AXAJ8LVWFypJUrtqpmc8FliYmU9m5mpgKnBUQ5ujgGur17cAH4+IaF2ZkiS1r2bCeEdgcd3wkmpcp20ycw3wErBtKwqUJKnd9e/JlUXERGBiNbgiIh7vyfW3UsB2wHM9tsLz+8aBBvdL59wvnXO/dM790rk22C87r29CM2G8FBhSNzy4GtdZmyUR0R/YCljeuKDMvAK4ool19noRMTczx5Suo7dxv3TO/dI590vn3C+da+f90sxh6jnALhExLCI2AyYAMxrazAA+W73+NDA7M7N1ZUqS1L667Bln5pqImATcDvQDrsrM+RFxATA3M2cAVwLXR8RC4HlqgS1JkprQ1DnjzJwJzGwYd07d61XAZ1pbWq/XFofbu4H7pXPul865Xzrnfulc2+6X8GiyJElleTtMSZIKM4ybEBErOhl3XkQsjYh5EfFIRBxXorZSIiIj4qK64bMi4rzqdf2+eSwiLouItv5di4i/i4j5EfFgtd3nRsQ/NrQZHRGPVq8HRsTlEfFERDwQEXdFxN5lqm+9iBgSEb+IiG2q4a2r4aERsUtEfL9u2++MiI9W7U6KiGXVPpwfEbdExLvLbk3r1P8tiYjDI+LnEbFz9Z5ZGRHbr6ftet9v7Sgi1tb9DvwsIs6MiE0i4pBq/LyIWFHdpnleRFxXuuZ3qq3/QPaASzJzNLU7kF0eEZsWrqcn/QY4JiK2W8/0jn0zHNgdOLCnCutpEbEvcASwZ2aOBA4C7gSObWg6AZhSvf4OtYsdd8nMDwMnU/sOZVvIzMXAZcBXq1FfpXa+71fAbcAVmfmBattPB36/bvabMnN0Zn4IWM3v7sc+LyI+DnwTOCwzn6pGPwecuZ5Zunq/tZvX6n4HDqZ2O+ZzM/P2avxoYC5wfDV8YsliW8EwboHMXACsBLYuXUsPWkPtj+tfdtFuM2AA8EK3V1TODsBzmfkbgMx8LjPvBl5o6O3+MTAlIj4A7A18KTPXVfP8IjNv6+nCu9klwD4R8RfAAcA/A8cDP6q+hQFAZj6cmdc0zlzds2AL2ux3pzoK8H+BIzLzibpJVwHHdhxNaNDs+63tZOaz1G4WNamdb7NsGLdAROwJLKh+aTYmlwLHR8RWnUz7y4iYBzwD/Dwz5/VkYT1sFjCkOuT47YjoOAowheprfhGxD/B89cHtQ8C8zFxbptyekZmvA39FLZT/ohr+EPCTLmY9tvrdWQpsA3yvO+vsYe8CpgOfyszHGqatoBbIX1jPvG/1fmtrmfkkta/Wbt9V277KMH5n/jIi5gP3AX9fupielpkvA9cBZ3QyueMw9fbAFhHRtt89z8wVwIepfXpfBtwUEScBNwGfrs6X1x+i3pgcRu0D2YjOJkbEtIh4OCL+o270TdXvzu8BD1EL9HbxOnAvtSfddeabwGcjYsvGCV2839THGcbvzCXVOY3xwJURMaB0QQV8ndofli06m1j1hv4f8NEerKnHZebazLwrM88FJgHjq/Omv6B2vnw8tXAGmA+Mqh5P2rYiYjS18337UPvgugO1bd+zo01mHg2cRK0H/CbVXfy+R3v97qyjdrpibET8bePEzHwRuBH48/XM/3Xe4v3WriLi94G1QNsefTSMW6A6/zWX394SdKORmc8DN7OeT/rVOZ79gSc6m94OImLXiNilbtRooOOinCnUDtM+mZlLAKrzhHOB8zvOgVVXGX+y56ruXtV2XUbt8PTTwIXUzhnfCOwfEePqmr/V1dIH0Ga/O5m5EvgktUPOnb1vLgZOo5ObMnX1fmtHETEI+FfgW+18m2XDuDnvjogldT9f7KTNBcAX2/0rPOtxEb97JXDHOeOHqZ3r+XZPF9WDBgLXRu0rbg9Su4L8vGrad6mdJ208RP054H3Awoh4GLiG9vrUfyrwdGb+VzX8bWA3as9HPwL4s4h4MiJ+BHwJ+ErdvMdWX1d5ENgD+HIP1t0jqlA9FPhSwwcTMvM5YBq188ud6ez91m427/hqE3AHtesyzi9cU7fyDlySJBW2MfbiJEnqVQxjSZIKM4wlSSrMMJYkqTDDWJKkwgxjqY+pnuBzQ91w/+pJR9/fwOUs6urBA820kfTOGcZS3/MqMCIiNq+GD6Z2H2dJfZRhLPVNM6ndxQngOOpuKhIR20TE9Kg9W/nHETGyGr9tRMyqnhH7HSDq5vnTiLi/utHC5Y236oyILSLiturZsg9HRNs91lAqyTCW+qapwITqfugjqT2spMP5wE+rZyv/LbWHCwCcC9xT3U99GrATQETsRu2ZwftXD2hYS+1Rh/UOBX6ZmaMycwS1+41LapHfufeppN4vMx+MiKHUesUzGyYfQO3BFGTm7KpH/B5qD1w4php/W0R0PCf449SeOjWnulX25vzurTkfAi6KiK8B38/MH7Z+q6SNl2Es9V0zqD184WPAtu9gOQFcm5lnr69BZv68em734cBXIuIHmXnBO1inpDoeppb6rquA8zPzoYbxP6Q6zBwRHwOeq56FezfwJ9X4w4Ctq/Y/oPbc5e2radtExM71C4yI9wMrM/MGak9g2hNJLWPPWOqjqkcyfrOTSecBV1VPPVrJbx/teT4wpXoSzr3A09VyHomILwGzqqeOvU7tebpP1S1zd+DCiFhXTf/frd8iaePlU5skSSrMw9SSJBVmGEuSVJhhLElSYYaxJEmFGcaSJBVmGEuSVJhhLElSYYaxJEmF/Q+PKdSZ3H9UIAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "X_axis = np.arange(len(models))\n",
    "plt.figure(figsize=(8, 8))  \n",
    "plt.bar(X_axis - 0.2, acc , 0.4, label = 'Accuracy')\n",
    "plt.bar(X_axis + 0.2, f1, 0.4, label = 'F1 score')\n",
    "  \n",
    "plt.xticks(X_axis, models)\n",
    "plt.xlabel(\"Models\")\n",
    "plt.title(\"Performance of Models\")\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8815851",
   "metadata": {},
   "source": [
    "The above plot shows the performance of the different algorithms used. Now from first glance, it seems that SVC is doing the best job because eventhough its accuracy isn't as high as that of Decision tree, but its F1 score compensates for that. If I'm to choose I would pick SVM."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "331b97c3",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "In this notebook I tried to build models capable of detecting fake pictures. Firstly, I tried to reduce the dimensionality of the images using PCA, then I attempted with different classification algorithms. The results of my models are average if not too bad (like XGB and SVC), and I can argue its because of :\n",
    "- The method of pre-processing I used: maybe the PCA that kept only half of the information  in the picture has let go of so much information necessary for the models to differentiate between the two classes.\n",
    "- Machine learning algorithms are not able to handle well this particular problem and that neural networks should be given a shot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a51f904e",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e6fb0e5a",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4169d2c8",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "004ce126",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}