Classifiers.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Loading all required libraries\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "import itertools\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib.ticker import NullFormatter\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.ticker as ticker\n",
    "from sklearn import preprocessing\n",
    "%matplotlib inline\n",
    "from sklearn import metrics\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.tree import DecisionTreeClassifier \n",
    "from sklearn.metrics import confusion_matrix\n",
    "from sklearn import svm\n",
    "from sklearn.metrics import jaccard_similarity_score\n",
    "from sklearn.metrics import log_loss\n",
    "from sklearn.metrics import f1_score\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.metrics import classification_report"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### About dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "This dataset is about past loans. The __Loan_train.csv__ data set includes details of 346 customers whose loan are already paid off or defaulted. It includes following fields:\n",
    "\n",
    "| Field          | Description                                                                           |\n",
    "|----------------|---------------------------------------------------------------------------------------|\n",
    "| Loan_status    | Whether a loan is paid off on in collection                                           |\n",
    "| Principal      | Basic principal loan amount at the                                                    |\n",
    "| Terms          | Origination terms which can be weekly (7 days), biweekly, and monthly payoff schedule |\n",
    "| Effective_date | When the loan got originated and took effects                                         |\n",
    "| Due_date       | Since it’s one-time payoff schedule, each loan has one single due date                |\n",
    "| Age            | Age of applicant                                                                      |\n",
    "| Education      | Education of applicant                                                                |\n",
    "| Gender         | The gender of applicant                                                               |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets download the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2020-04-07 13:35:16--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv\n",
      "Resolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196\n",
      "Connecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 23101 (23K) [text/csv]\n",
      "Saving to: ‘loan_train.csv’\n",
      "\n",
      "100%[======================================>] 23,101      --.-K/s   in 0.001s  \n",
      "\n",
      "2020-04-07 13:35:16 (19.9 MB/s) - ‘loan_train.csv’ saved [23101/23101]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget -O loan_train.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Load Data From CSV File  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>10/7/2016</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>10/7/2016</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>9/22/2016</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/9/2016</td>\n",
       "      <td>10/8/2016</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/9/2016</td>\n",
       "      <td>10/8/2016</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30       9/8/2016   \n",
       "1           2             2     PAIDOFF       1000     30       9/8/2016   \n",
       "2           3             3     PAIDOFF       1000     15       9/8/2016   \n",
       "3           4             4     PAIDOFF       1000     30       9/9/2016   \n",
       "4           6             6     PAIDOFF       1000     30       9/9/2016   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0  10/7/2016   45  High School or Below    male  \n",
       "1  10/7/2016   33              Bechalor  female  \n",
       "2  9/22/2016   27               college    male  \n",
       "3  10/8/2016   28               college  female  \n",
       "4  10/8/2016   29               college    male  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('loan_train.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(346, 10)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Convert to date time object "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30     2016-09-08   \n",
       "1           2             2     PAIDOFF       1000     30     2016-09-08   \n",
       "2           3             3     PAIDOFF       1000     15     2016-09-08   \n",
       "3           4             4     PAIDOFF       1000     30     2016-09-09   \n",
       "4           6             6     PAIDOFF       1000     30     2016-09-09   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0 2016-10-07   45  High School or Below    male  \n",
       "1 2016-10-07   33              Bechalor  female  \n",
       "2 2016-09-22   27               college    male  \n",
       "3 2016-10-08   28               college  female  \n",
       "4 2016-10-08   29               college    male  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['due_date'] = pd.to_datetime(df['due_date'])\n",
    "df['effective_date'] = pd.to_datetime(df['effective_date'])\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Data visualization and pre-processing\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Let’s see how many of each class is in our data set "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PAIDOFF       260\n",
       "COLLECTION     86\n",
       "Name: loan_status, dtype: int64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['loan_status'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "260 people have paid off the loan on time while 86 have gone into collection \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets plot some columns to underestand data better:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Solving environment: done\n",
      "\n",
      "# All requested packages already installed.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# notice: installing seaborn might takes a few minutes\n",
    "!conda install -c anaconda seaborn -y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAG4xJREFUeJzt3XucFOWd7/HPV5wVFaIioyKIMyKKqGTAWY3XJbCyqPF2jAbjUdx4DtFoXDbxeMt5aTa+1nghMclRibhyyCaKGrKgSxINUTmKiRfAEcELITrqKCAQN8YgBPB3/qiaSYM9zKV7pmu6v+/Xq15T9VTVU7+umWd+XU9XP6WIwMzMLGt2KHUAZmZm+ThBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJjlBdRFJe0u6T9LrkhZJ+q2kM4tU92hJc4tRV3eQNF9SfanjsNIop7YgqVrSs5JekHR8Fx7nw66quydxguoCkgTMAZ6MiAMi4ghgAjCoRPHsWIrjmpVhWxgLvBoRIyPiqWLEZK1zguoaY4C/RMQPmwsi4s2I+D8AknpJulXS85KWSPpyWj46vdqYJelVSfemDRxJ49OyBcB/a65X0q6Spqd1vSDp9LT8Qkk/lfSfwK8KeTGSZkiaKumJ9F3w36XHfEXSjJztpkpaKGmZpH9ppa5x6TvoxWl8fQqJzTKvbNqCpDrgFuBkSQ2Sdm7t71lSo6Qb03ULJY2S9Kik30u6ON2mj6TH0n1fao43z3H/V875yduuylZEeCryBFwO3Lad9ZOA/53O7wQsBGqB0cAfSd5d7gD8FjgO6A28DQwFBDwIzE33vxH47+n87sByYFfgQqAJ6NdKDE8BDXmmv8+z7Qzg/vTYpwMfAIenMS4C6tLt+qU/ewHzgRHp8nygHugPPAnsmpZfBVxX6t+Xp66byrAtXAjcns63+vcMNAKXpPO3AUuAvkA18F5aviPwqZy6VgBKlz9Mf44DpqWvdQdgLnBCqX+v3TW566cbSLqDpHH9JSL+luSPboSkz6eb7EbS4P4CPBcRTel+DUAN8CHwRkT8Li3/CUnDJq3rNElXpMu9gcHp/LyI+EO+mCKio/3n/xkRIeklYHVEvJTGsiyNsQE4R9IkkoY3ABhO0jCbfSYtezp9M/w3JP94rEKUSVto1tbf88Ppz5eAPhHxJ+BPkjZI2h34M3CjpBOAj4GBwN7Aqpw6xqXTC+lyH5Lz82QnY+5RnKC6xjLgrOaFiLhUUn+Sd4eQvBv6akQ8mruTpNHAxpyiLfz1d9TaoIkCzoqI17ap6yiSBpB/J+kpknd027oiIn6dp7w5ro+3ifFjYEdJtcAVwN9GxPtp11/vPLHOi4hzW4vLyk45toXc423v73m7bQY4j+SK6oiI2CSpkfxt5tsRcdd24ihb/gyqazwO9JZ0SU7ZLjnzjwKXSKoCkHSQpF23U9+rQK2kIelyboN4FPhqTv/8yPYEGBHHR0Rdnml7DXJ7PkXyT+CPkvYGTsqzzTPAsZIOTGPdRdJBnTye9Qzl3BYK/XvejaS7b5OkzwL759nmUeBLOZ9tDZS0VweO0aM5QXWBSDqPzwD+TtIbkp4DfkTSRw3wb8DLwGJJS4G72M7VbERsIOnG+Hn6wfCbOatvAKqAJWldNxT79bRHRLxI0g2xDJgOPJ1nmzUkffgzJS0haeDDujFM62bl3BaK8Pd8L1AvaSHJ1dSreY7xK+A+4Ldp9/os8l/tlaXmD+TMzMwyxVdQZmaWSU5QZmaWSU5QZmaWSU5QZmaWSZlIUOPHjw+S7zZ48lQuU9G4fXgqs6ndMpGg1q5dW+oQzDLL7cMqVSYSlJmZ2bacoMzMLJOcoMzMLJM8WKyZlZVNmzbR1NTEhg0bSh1KRevduzeDBg2iqqqq03U4QZlZWWlqaqJv377U1NSQjhtr3SwiWLduHU1NTdTW1na6HnfxmVlZ2bBhA3vuuaeTUwlJYs899yz4KtYJyirG/gMGIKko0/4DBpT65dh2ODmVXjF+B+7is4rx1qpVNO07qCh1DXq3qSj1mFnrfAVlZmWtmFfO7b167tWrF3V1dRx22GGcffbZrF+/vmXd7NmzkcSrr/718U+NjY0cdthhAMyfP5/ddtuNkSNHcvDBB3PCCScwd+7creqfNm0aw4YNY9iwYRx55JEsWLCgZd3o0aM5+OCDqauro66ujlmzZm0VU/PU2NhYyGntFr6CMrOyVswrZ2jf1fPOO+9MQ0MDAOeddx4//OEP+drXvgbAzJkzOe6447j//vv55je/mXf/448/viUpNTQ0cMYZZ7DzzjszduxY5s6dy1133cWCBQvo378/ixcv5owzzuC5555jn332AeDee++lvr6+1Zh6ijavoCRNl/Re+oTK5rJvSnpHUkM6nZyz7hpJKyS9JukfuipwM7Oe4Pjjj2fFihUAfPjhhzz99NPcc8893H///e3av66ujuuuu47bb78dgJtvvplbb72V/v37AzBq1CgmTpzIHXfc0TUvoITa08U3Axifp/y2iKhLp18ASBoOTAAOTfe5U1KvYgVrZtaTbN68mV/+8pccfvjhAMyZM4fx48dz0EEH0a9fPxYvXtyuekaNGtXSJbhs2TKOOOKIrdbX19ezbNmyluXzzjuvpStv3bp1AHz00UctZWeeeWYxXl6Xa7OLLyKelFTTzvpOB+6PiI3AG5JWAEcCv+10hGZmPUxzMoDkCuqiiy4Cku69yZMnAzBhwgRmzpzJqFGj2qwvYvuDgEfEVnfNlUsXXyGfQV0m6QJgIfD1iHgfGAg8k7NNU1r2CZImAZMABg8eXEAYZuXH7aNny5cM1q1bx+OPP87SpUuRxJYtW5DELbfc0mZ9L7zwAocccggAw4cPZ9GiRYwZM6Zl/eLFixk+fHhxX0QGdPYuvqnAEKAOWAl8Jy3Pd+N73tQfEdMioj4i6qurqzsZhll5cvsoP7NmzeKCCy7gzTffpLGxkbfffpva2tqt7sDLZ8mSJdxwww1ceumlAFx55ZVcddVVLV13DQ0NzJgxg6985Std/hq6W6euoCJidfO8pLuB5nsgm4D9cjYdBLzb6ejMzAo0eJ99ivq9tcHpnXIdNXPmTK6++uqtys466yzuu+8+rrrqqq3Kn3rqKUaOHMn69evZa6+9+MEPfsDYsWMBOO2003jnnXc45phjkETfvn35yU9+woAy/PK42urbBEg/g5obEYelywMiYmU6/8/AURExQdKhwH0knzvtCzwGDI2ILdurv76+PhYuXFjI6zBrk6SiflG3jbZTtKEM3D465pVXXmnpDrPSauV30e620eYVlKSZwGigv6Qm4HpgtKQ6ku67RuDLABGxTNKDwMvAZuDStpKTmZlZPu25i+/cPMX3bGf7fwX+tZCgzMzMPNSRmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZW1fQcNLurjNvYd1L6RPVatWsWECRMYMmQIw4cP5+STT2b58uUsW7aMMWPGcNBBBzF06FBuuOGGlq8szJgxg8suu+wTddXU1LB27dqtymbMmEF1dfVWj9B4+eWXAVi+fDknn3wyBx54IIcccgjnnHMODzzwQMt2ffr0aXkkxwUXXMD8+fP53Oc+11L3nDlzGDFiBMOGDePwww9nzpw5LesuvPBCBg4cyMaNGwFYu3YtNTU1HfqdtJcft2FmZW3lO29z1HWPFK2+Z7+Vb+zsrUUEZ555JhMnTmwZtbyhoYHVq1dz4YUXMnXqVMaNG8f69es566yzuPPOO1tGiuiIL3zhCy2jnDfbsGEDp5xyCt/97nc59dRTAXjiiSeorq5uGX5p9OjRTJkypWW8vvnz57fs/+KLL3LFFVcwb948amtreeONNzjxxBM54IADGDFiBJA8W2r69OlccsklHY65I3wFZWZWZE888QRVVVVcfPHFLWV1dXUsX76cY489lnHjxgGwyy67cPvtt3PTTTcV7dj33XcfRx99dEtyAvjsZz/b8kDEtkyZMoVrr72W2tpaAGpra7nmmmu49dZbW7aZPHkyt912G5s3by5a3Pk4QZmZFdnSpUs/8UgMyP+ojCFDhvDhhx/ywQcfdPg4ud12dXV1fPTRR60eu73a8ziPwYMHc9xxx/HjH/+408dpD3fxmZl1k20fi5GrtfLtydfFV6h8MeYru/baaznttNM45ZRTinr8XL6CMjMrskMPPZRFixblLd92XMXXX3+dPn360Ldv3y49dkf23zbGfI/zOPDAA6mrq+PBBx/s9LHa4gRlZlZkY8aMYePGjdx9990tZc8//zxDhw5lwYIF/PrXvwaSBxtefvnlXHnllUU79he/+EV+85vf8POf/7yl7JFHHuGll15q1/5XXHEF3/72t2lsbASgsbGRG2+8ka9//euf2PYb3/gGU6ZMKUrc+biLz8zK2oCB+7XrzruO1NcWScyePZvJkydz00030bt3b2pqavje977HQw89xFe/+lUuvfRStmzZwvnnn7/VreUzZszY6rbuZ55JngE7YsQIdtghuaY455xzGDFiBA888MBWz5O68847OeaYY5g7dy6TJ09m8uTJVFVVMWLECL7//e+36/XV1dVx8803c+qpp7Jp0yaqqqq45ZZbWp4QnOvQQw9l1KhR7X50fUe163EbXc2PE7Du4MdtVAY/biM7Cn3cRptdfJKmS3pP0tKcslslvSppiaTZknZPy2skfSSpIZ1+2N5AzMzMcrXnM6gZwLbXx/OAwyJiBLAcuCZn3e8joi6dLsbMzKwT2kxQEfEk8Idtyn4VEc3f0HqG5NHuZmaZkIWPLipdMX4HxbiL70vAL3OWayW9IOn/STq+tZ0kTZK0UNLCNWvWFCEMs/Lh9tF5vXv3Zt26dU5SJRQRrFu3jt69exdUT0F38Un6Bsmj3e9Ni1YCgyNinaQjgDmSDo2IT3xFOiKmAdMg+RC4kDjMyo3bR+cNGjSIpqYmnNhLq3fv3gwaVFjnWqcTlKSJwOeAsZG+VYmIjcDGdH6RpN8DBwG+BcnMukVVVVXLOHLWs3Wqi0/SeOAq4LSIWJ9TXi2pVzp/ADAUeL0YgZqZWWVp8wpK0kxgNNBfUhNwPcldezsB89LxmZ5J79g7AfiWpM3AFuDiiPhD3orNzMy2o80EFRHn5im+p5Vtfwb8rNCgzMzMPBafmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllUrsSlKTpkt6TtDSnrJ+keZJ+l/7cIy2XpB9IWiFpiaRRXRW8mZmVr/ZeQc0Axm9TdjXwWEQMBR5LlwFOInmS7lBgEjC18DDNzKzStCtBRcSTwLZPxj0d+FE6/yPgjJzyf4/EM8DukgYUI1gzM6schXwGtXdErARIf+6Vlg8E3s7Zrikt24qkSZIWSlq4Zs2aAsIwKz9uH2Zdc5OE8pTFJwoipkVEfUTUV1dXd0EYZj2X24dZYQlqdXPXXfrzvbS8CdgvZ7tBwLsFHMfMzCpQIQnqYWBiOj8ReCin/IL0br7PAH9s7go0MzNrrx3bs5GkmcBooL+kJuB64CbgQUkXAW8BZ6eb/wI4GVgBrAf+scgxm5lZBWhXgoqIc1tZNTbPtgFcWkhQZmZmHknCzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyqV2jmecj6WDggZyiA4DrgN2B/wk0P6f62oj4RacjNDOzitTpBBURrwF1AJJ6Ae8As0me/3RbREwpSoRmZlaRitXFNxb4fUS8WaT6zMyswhUrQU0AZuYsXyZpiaTpkvbIt4OkSZIWSlq4Zs2afJuYVSy3D7MiJChJfwOcBvw0LZoKDCHp/lsJfCfffhExLSLqI6K+urq60DDMyorbh1lxrqBOAhZHxGqAiFgdEVsi4mPgbuDIIhzDzMwqTDES1LnkdO9JGpCz7kxgaRGOYWZmFabTd/EBSNoFOBH4ck7xLZLqgAAat1lnZmbWLgUlqIhYD+y5Tdn5BUVkZmaGR5IwM7OMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMKug2c7OeRL2qGPRuU9HqMrOu5QRlFSO2bOKo6x4pSl3Pfmt8Ueoxs9a5i8/MzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDKp4NvMJTUCfwK2AJsjol5SP+ABoIbkmVDnRMT7hR7LzMwqR7GuoD4bEXURUZ8uXw08FhFDgcfSZasw+w8YgKSCp/0HDGj7YGZWdrrqi7qnA6PT+R8B84GruuhYllFvrVpF076DCq6nWKM/mFnPUowrqAB+JWmRpElp2d4RsRIg/bnXtjtJmiRpoaSFa9asKUIYZuXD7cOsOAnq2IgYBZwEXCrphPbsFBHTIqI+Iuqrq6uLEIZZ+XD7MCtCgoqId9Of7wGzgSOB1ZIGAKQ/3yv0OGZmVlkKSlCSdpXUt3keGAcsBR4GJqabTQQeKuQ4ZmZWeQq9SWJvYLak5rrui4hHJD0PPCjpIuAt4OwCj2NmZhWmoAQVEa8Dn85Tvg4YW0jdZmZW2TyShJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZfIJ2F31RF0zM+tBsvgEbF9BmZlZJnU6QUnaT9ITkl6RtEzSP6Xl35T0jqSGdDq5eOGamVmlKKSLbzPw9YhYnD60cJGkeem62yJiSuHhmZlZpep0goqIlcDKdP5Pkl4BBhYrMDMzq2xF+QxKUg0wEng2LbpM0hJJ0yXt0co+kyQtlLRwzZo1xQjDrGy4fZgVIUFJ6gP8DJgcER8AU4EhQB3JFdZ38u0XEdMioj4i6qurqwsNw6ysuH2YFZigJFWRJKd7I+I/ACJidURsiYiPgbuBIwsP08zMKk0hd/EJuAd4JSK+m1Oe+y2tM4GlnQ/PzMwqVSF38R0LnA+8JKkhLbsWOFdSHRBAI/DlgiI0M7OKVMhdfAsA5Vn1i86HY2ZmlvBIEmZmlkkei8+6jHpVFWVcLvWqKkI0ZtbTOEFZl4ktmzjqukcKrufZb40vQjRm1tO4i8/MzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMrJtl8fHqWeS7+MzMulkWH6+eRb6CMjOzTHKCMjOzTHIXn5mZZXLkFycoMzPL5Mgv7uIzM7NM6rIEJWm8pNckrZB0daH1+bZMM7PK0iVdfJJ6AXcAJwJNwPOSHo6Ilztbp2/LNDOrLF31GdSRwIqIeB1A0v3A6UCnE1TW7D9gAG+tWlVwPYP32Yc3V64sQkTlTcr3bEzLIreNthXrhoQdelWVddtQRBS/UunzwPiI+B/p8vnAURFxWc42k4BJ6eLBwGtFD6T9+gNrS3j8Qjj20mgr9rUR0elPizPUPsr5d5Rl5Rx7u9tGV11B5UvpW2XCiJgGTOui43eIpIURUV/qODrDsZdGV8eelfbh31FpOPZEV90k0QTsl7M8CHi3i45lZmZlqKsS1PPAUEm1kv4GmAA83EXHMjOzMtQlXXwRsVnSZcCjQC9gekQs64pjFUnJu1IK4NhLoyfH3hE9+XU69tIoWuxdcpOEmZlZoTyShJmZZZITlJmZZVLFJChJvSS9IGluulwr6VlJv5P0QHozB5J2SpdXpOtrShz37pJmSXpV0iuSjpbUT9K8NPZ5kvZIt5WkH6SxL5E0qsSx/7OkZZKWSpopqXdWz7uk6ZLek7Q0p6zD51nSxHT730ma2J2vobPcNkoSu9tGO1RMggL+CXglZ/lm4LaIGAq8D1yUll8EvB8RBwK3pduV0veBRyJiGPBpktdwNfBYGvtj6TLAScDQdJoETO3+cBOSBgKXA/URcRjJzTITyO55nwFs++XBDp1nSf2A64GjSEZTub654Wac20Y3ctvoQNuIiLKfSL6H9RgwBphL8kXitcCO6fqjgUfT+UeBo9P5HdPtVKK4PwW8se3xSUYVGJDODwBeS+fvAs7Nt10JYh8IvA30S8/jXOAfsnzegRpgaWfPM3AucFdO+VbbZXFy23DbaGfMJWkblXIF9T3gSuDjdHlP4L8iYnO63ETyRwN//eMhXf/HdPtSOABYA/zftAvm3yTtCuwdESvTGFcCe6Xbt8Seyn1d3Soi3gGmAG8BK0nO4yJ6xnlv1tHznJnz3wFuG93MbWOr8u0q+wQl6XPAexGxKLc4z6bRjnXdbUdgFDA1IkYCf+avl9L5ZCb29PL9dKAW2BfYleTyf1tZPO9taS3WnvQa3DbcNrpCUdtG2Sco4FjgNEmNwP0kXRnfA3aX1PxF5dyhmFqGaUrX7wb8oTsDztEENEXEs+nyLJJGuVrSAID053s522dliKm/B96IiDURsQn4D+AYesZ5b9bR85yl898ebhul4bbRzvNf9gkqIq6JiEERUUPyQeTjEXEe8ATw+XSzicBD6fzD6TLp+scj7TTtbhGxCnhb0sFp0ViSR5bkxrht7Bekd9J8Bvhj82V4CbwFfEbSLpLEX2PP/HnP0dHz/CgwTtIe6bvkcWlZJrltuG0UoHvaRik+JCzVBIwG5qbzBwDPASuAnwI7peW90+UV6foDShxzHbAQWALMAfYg6X9+DPhd+rNfuq1IHhT5e+AlkruEShn7vwCvAkuBHwM7ZfW8AzNJPg/YRPJu76LOnGfgS+lrWAH8Y6n/5jvw+t02ujd2t412HNtDHZmZWSaVfRefmZn1TE5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QGSZpi6SGdMTjn0rapZXtfiFp907Uv6+kWQXE1yipf2f3N+sst43K4NvMM0zShxHRJ52/F1gUEd/NWS+S3+HHrdXRxfE1knzPYW0pjm+Vy22jMvgKqud4CjhQUo2SZ9/cCSwG9mt+t5az7m4lz5r5laSdASQdKOnXkl6UtFjSkHT7pen6CyU9JOkRSa9Jur75wJLmSFqU1jmpJK/erHVuG2XKCaoHSMffOonkm9kABwP/HhEjI+LNbTYfCtwREYcC/wWclZbfm5Z/mmTcr3zDvBwJnEfyDf2zJdWn5V+KiCOAeuBySaUeSdkMcNsod05Q2bazpAaS4VzeAu5Jy9+MiGda2eeNiGhI5xcBNZL6AgMjYjZARGyIiPV59p0XEesi4iOSASyPS8svl/Qi8AzJgI9DC35lZoVx26gAO7a9iZXQRxFRl1uQdK3z5+3sszFnfguwM/mHus9n2w8kQ9JoktGXj46I9ZLmk4wNZlZKbhsVwFdQFSAiPgCaJJ0BIGmnVu56OlFSv7Rv/gzgaZKh/d9PG+Aw4DPdFrhZF3PbyDYnqMpxPkl3xBLgN8A+ebZZQDKycgPws4hYCDwC7JjudwNJV4ZZOXHbyCjfZm5AcqcSyW2xl5U6FrMscdsoHV9BmZlZJvkKyszMMslXUGZmlklOUGZmlklOUGZmlklOUGZmlklOUGZmlkn/H+LDZoiBEQ8dAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x216 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import seaborn as sns\n",
    "\n",
    "bins = np.linspace(df.Principal.min(), df.Principal.max(), 10)\n",
    "g = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\n",
    "g.map(plt.hist, 'Principal', bins=bins, ec=\"k\")\n",
    "\n",
    "g.axes[-1].legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAGfZJREFUeJzt3XuQVOW57/HvTxgdFbygo4yMwKgoopIBZ3tDDYJy2N49XuKOR7GOJx4Naqjo8ZZTVrLdZbyVmhwvkUQLK1HUmA26SUWDCidi4gVwRBBv0UFHQS7RKAchgs/5o9fMHqBhembWTK/u+X2qVnWvt1e/61lMvzy93vX2uxQRmJmZZc02xQ7AzMwsHycoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCeolEjaU9Ijkt6XNE/SXySdkVLdoyXNSKOu7iBptqT6YsdhxVdO7UJSlaSXJb0m6Zgu3M/qrqq71DhBpUCSgOnAnyJin4g4FDgXqClSPL2LsV+z1sqwXYwF3oqIERHxQhox2dY5QaVjDPCPiPhFc0FELImI/wMgqZek2yS9KmmBpP+ZlI9OzjaekPSWpIeTRo2k8UnZHOC/NtcraUdJDyZ1vSbptKT8Qkm/lfQfwB87czCSpki6T9Ks5Jvvt5N9LpY0pdV290maK2mRpJ9soa5xybfm+Ul8fToTm5WUsmkXkuqAW4ETJTVI2n5Ln21JjZJuSl6bK2mkpGck/VXSJck2fSQ9l7z3jeZ48+z3f7X698nbxspaRHjp5AJcAdy5ldcvBv538nw7YC5QC4wG/k7uG+U2wF+Ao4FK4CNgCCDgcWBG8v6bgP+WPN8FeAfYEbgQaAL6bSGGF4CGPMvxebadAjya7Ps04AvgkCTGeUBdsl2/5LEXMBsYnqzPBuqB3YE/ATsm5dcANxT77+Wle5YybBcXAncnz7f42QYagUuT53cCC4C+QBWwPCnvDezUqq73ACXrq5PHccDk5Fi3AWYAxxb779qdi7uCuoCke8g1qH9ExD+R+6ANl3RWssnO5BrZP4BXIqIpeV8DMBhYDXwQEe8m5b8h15hJ6jpV0lXJeiUwMHk+MyL+li+miGhvn/l/RERIegP4NCLeSGJZlMTYAJwj6WJyja0aGEauMTY7Iil7MfkCvC25/2ysByqTdtGsrc/2U8njG0CfiPgS+FLSWkm7AP8PuEnSscA3wABgT2BZqzrGJctryXofcv8+f+pgzCXHCSodi4Azm1ciYqKk3cl9I4TcN6DLI+KZ1m+SNBpY16poA//5N9nSJIkCzoyItzep63ByH/r8b5JeIPctblNXRcSzecqb4/pmkxi/AXpLqgWuAv4pIj5Luv4q88Q6MyL+ZUtxWVkrx3bRen9b+2xvtf0A55E7ozo0Ir6W1Ej+9vPTiLh/K3GUNV+DSsfzQKWkS1uV7dDq+TPApZIqACTtL2nHrdT3FlArad9kvXUjeAa4vFWf/IhCAoyIYyKiLs+ytUa4NTuRa/h/l7Qn8M95tnkJGCVpvyTWHSTt38H9Wekp53bR2c/2zuS6+76WdBwwKM82zwD/vdW1rQGS9mjHPkqeE1QKItdhfDrwbUkfSHoFeIhcvzTAr4A3gfmSFgL3s5Wz14hYS67r4vfJxeAlrV6+EagAFiR13Zj28RQiIl4n1/WwCHgQeDHPNivI9dtPlbSAXKMe2o1hWhGVc7tI4bP9MFAvaS65s6m38uzjj8AjwF+SrvYnyH+2V7aaL8qZmZllis+gzMwsk5ygzMwsk5ygzMwsk5ygzMwsk7o1QY0fPz7I/Y7Bi5dyXTrN7cRLD1gK0q0JauXKld25O7OS5HZiluMuPjMzyyQnKDMzyyQnKDMzyyRPFmtmZefrr7+mqamJtWvXFjuUHq2yspKamhoqKio69H4nKDMrO01NTfTt25fBgweTzB9r3SwiWLVqFU1NTdTW1naoDnfxmVnZWbt2LbvttpuTUxFJYrfdduvUWawTVDcaVF2NpFSWQdXVxT4cs0xzciq+zv4N3MXXjT5ctoymvWpSqavmk6ZU6jEzyyqfQZlZ2Uuz96LQHoxevXpRV1fHwQcfzNlnn82aNWtaXps2bRqSeOut/7wNVGNjIwcffDAAs2fPZuedd2bEiBEccMABHHvsscyYMWOj+idPnszQoUMZOnQohx12GHPmzGl5bfTo0RxwwAHU1dVRV1fHE088sVFMzUtjY2Nn/lm7nM+gzKzspdl7AYX1YGy//fY0NDQAcN555/GLX/yCH/7whwBMnTqVo48+mkcffZQf//jHed9/zDHHtCSlhoYGTj/9dLbffnvGjh3LjBkzuP/++5kzZw6777478+fP5/TTT+eVV16hf//+ADz88MPU19dvMaZS4DMoM7Mudswxx/Dee+8BsHr1al588UUeeOABHn300YLeX1dXxw033MDdd98NwC233MJtt93G7rvvDsDIkSOZMGEC99xzT9ccQJE4QZmZdaH169fzhz/8gUMOOQSA6dOnM378ePbff3/69evH/PnzC6pn5MiRLV2CixYt4tBDD93o9fr6ehYtWtSyft5557V05a1atQqAr776qqXsjDPOSOPwupS7+MzMukBzMoDcGdRFF10E5Lr3Jk2aBMC5557L1KlTGTlyZJv1RWx9EvCI2GjUXDl08RWUoCQ1Al8CG4D1EVEvqR/wGDAYaATOiYjPuiZMM7PSki8ZrFq1iueff56FCxciiQ0bNiCJW2+9tc36XnvtNQ488EAAhg0bxrx58xgzZkzL6/Pnz2fYsGHpHkSRtaeL77iIqIuI5pR8LfBcRAwBnkvWzcxsC5544gkuuOAClixZQmNjIx999BG1tbUbjcDLZ8GCBdx4441MnDgRgKuvvpprrrmmpeuuoaGBKVOm8P3vf7/Lj6E7daaL7zRgdPL8IWA2cE0n4zEzS93A/v1T/e3gwGSkXHtNnTqVa6/d+Lv8mWeeySOPPMI112z83+cLL7zAiBEjWLNmDXvssQc///nPGTt2LACnnnoqH3/8MUcddRSS6Nu3L7/5zW+oLrMf8Kutfk0ASR8An5G7E+L9ETFZ0ucRsUurbT6LiF3zvPdi4GKAgQMHHrpkyZLUgi81klL9oW4hfzvrdh366bzbSboWL17c0h1mxbWFv0VB7aTQLr5RETES+GdgoqRjCw0uIiZHRH1E1FdVVRX6NrMexe3EbHMFJaiI+CR5XA5MAw4DPpVUDZA8Lu+qIM3MrOdpM0FJ2lFS3+bnwDhgIfAUMCHZbALwZFcFaWZmPU8hgyT2BKYl4+t7A49ExNOSXgUel3QR8CFwdteFaWZmPU2bCSoi3ge+lad8FTC2K4IyMzPzVEdmZpZJTlBmVvb2qhmY6u029qoZWNB+ly1bxrnnnsu+++7LsGHDOPHEE3nnnXdYtGgRY8aMYf/992fIkCHceOONLT8bmTJlCpdddtlmdQ0ePJiVK1duVDZlyhSqqqo2uoXGm2++CcA777zDiSeeyH777ceBBx7IOeecw2OPPdayXZ8+fVpuyXHBBRcwe/ZsTj755Ja6p0+fzvDhwxk6dCiHHHII06dPb3ntwgsvZMCAAaxbtw6AlStXMnjw4Hb9TQrhufgKMKi6mg+XLSt2GGbWQUs//ojDb3g6tfpe/tfxbW4TEZxxxhlMmDChZdbyhoYGPv30Uy688ELuu+8+xo0bx5o1azjzzDO59957W2aKaI/vfOc7LbOcN1u7di0nnXQSd9xxB6eccgoAs2bNoqqqqmX6pdGjR3P77be3zNc3e/bslve//vrrXHXVVcycOZPa2lo++OADTjjhBPbZZx+GDx8O5O4t9eCDD3LppZe2O+ZCOUEVIK17yfguuGY9x6xZs6ioqOCSSy5pKaurq+OBBx5g1KhRjBs3DoAddtiBu+++m9GjR3coQeXzyCOPcOSRR7YkJ4Djjjuu4PfffvvtXH/99dTW1gJQW1vLddddx2233cavf/1rACZNmsSdd97J9773vVRizsddfGZmXWDhwoWb3RID8t8qY99992X16tV88cUX7d5P6267uro6vvrqqy3uu1CF3M5j4MCBHH300S0Jqyv4DMrMrBtteluM1rZUvjX5uvg6K1+M+cquv/56Tj31VE466aRU99/MZ1BmZl3goIMOYt68eXnL586du1HZ+++/T58+fejbt2+X7rs97980xny389hvv/2oq6vj8ccf7/C+tsYJysysC4wZM4Z169bxy1/+sqXs1VdfZciQIcyZM4dnn30WyN3Y8IorruDqq69Obd/f/e53+fOf/8zvf//7lrKnn36aN954o6D3X3XVVfz0pz+lsbERgMbGRm666SauvPLKzbb90Y9+xO23355K3JtyF5+Zlb3qAXsXNPKuPfW1RRLTpk1j0qRJ3HzzzVRWVjJ48GDuuusunnzySS6//HImTpzIhg0bOP/88zcaWj5lypSNhnW/9NJLAAwfPpxttsmdV5xzzjkMHz6cxx57bKP7Sd17770cddRRzJgxg0mTJjFp0iQqKioYPnw4P/vZzwo6vrq6Om655RZOOeUUvv76ayoqKrj11ltb7hDc2kEHHcTIkSMLvnV9exR0u4201NfXx6anjaUgrdtk1HzS5NttlL8O3W6jtVJtJ1ni221kR3fcbsPMzKxbOUGZmVkmOUGZWVlyF3jxdfZv4ARlZmWnsrKSVatWOUkVUUSwatUqKisrO1yHR/GZWdmpqamhqamJFStWFDuUHq2yspKamo4PDHOCKlHb0bFfneczsH9/lixdmkpdZllQUVHRMo+clS4nqBK1DlIdsm5mljUFX4OS1EvSa5JmJOu1kl6W9K6kxyRt23VhmplZT9OeQRI/ABa3Wr8FuDMihgCfARelGZiZmfVsBSUoSTXAScCvknUBY4Ankk0eAk7vigDNzKxnKvQM6i7gauCbZH034POIWJ+sNwED8r1R0sWS5kqa6xE1Zvm5nZhtrs0EJelkYHlEtJ67Pd/wsbw/OIiIyRFRHxH1VVVVHQzTrLy5nZhtrpBRfKOAUyWdCFQCO5E7o9pFUu/kLKoG+KTrwjQzs56mzTOoiLguImoiYjBwLvB8RJwHzALOSjabADzZZVGamVmP05mpjq4BfijpPXLXpB5IJyQzM7N2/lA3ImYDs5Pn7wOHpR+SmZmZJ4s1M7OMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMajNBSaqU9Iqk1yUtkvSTpLxW0suS3pX0mKRtuz5cMzPrKQo5g1oHjImIbwF1wHhJRwC3AHdGxBDgM+CirgvTzMx6mjYTVOSsTlYrkiWAMcATSflDwOldEqGZmfVIBV2DktRLUgOwHJgJ/BX4PCLWJ5s0AQO28N6LJc2VNHfFihVpxGxWdtxOzDZXUIKKiA0RUQfUAIcBB+bbbAvvnRwR9RFRX1VV1fFIzcqY24nZ5to1ii8iPgdmA0cAu0jqnbxUA3ySbmhmZtaTFTKKr0rSLsnz7YHjgcXALOCsZLMJwJNdFaSZmfU8vdvehGrgIUm9yCW0xyNihqQ3gUcl/RvwGvBAF8ZpZmY9TJsJKiIWACPylL9P7nqUmZlZ6jyThJmZZZITlJmZZZITlJmZZZITlJmZZVLZJqhB1dVISmUxM7PuV8gw85L04bJlNO1Vk0pdNZ80pVKPmZkVrmzPoMzMrLQ5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSa1maAk7S1plqTFkhZJ+kFS3k/STEnvJo+7dn24ZmbWUxRyBrUeuDIiDgSOACZKGgZcCzwXEUOA55J1MzOzVLSZoCJiaUTMT55/CSwGBgCnAQ8lmz0EnN5VQZqZWc/TrmtQkgYDI4CXgT0jYinkkhiwxxbec7GkuZLmrlixonPRmpUptxOzzRWcoCT1AX4HTIqILwp9X0RMjoj6iKivqqrqSIxmZc/txGxzBSUoSRXkktPDEfHvSfGnkqqT16uB5V0TopmZ9USFjOIT8ACwOCLuaPXSU8CE5PkE4Mn0w7PusB20edv7QpZB1dXFPhQzKyOF3PJ9FHA+8IakhqTseuBm4HFJFwEfAmd3TYjW1dYBTXvVdLqemk+aOh+MmVmizQQVEXMAbeHlsemGk03qVZHKf77qvW1q/4mrV0Uq9ZiZZVUhZ1A9Xmz4msNveLrT9bz8r+NTqae5LjOzcuapjszMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJPKdiaJtKYnMjOz4ijbBJXW9ETgaYXMzIrBXXxmZpZJTlBmZpZJTlBmZpZJZXsNqtylOQjE95ayrBlUXc2Hy5Z1up7tt+nFV99sSCEiGNi/P0uWLk2lLiuME1SJ8iAQK2cfLluW2l2e06inuS7rXm128Ul6UNJySQtblfWTNFPSu8njrl0bppmZ9TSFXIOaAmz6Ffta4LmIGAI8l6xbD7cdICmVZVB1dbEPx8yKrM0uvoj4k6TBmxSfBoxOnj8EzAauSTEuK0HrwN0pZpaajo7i2zMilgIkj3tsaUNJF0uaK2nuihUrOrg7s/JWDu1kUHV1amfQZtANgyQiYjIwGaC+vj66en9mpagc2klaAxvAZ9CW09EzqE8lVQMkj8vTC8nMzKzjCeopYELyfALwZDrhmJmZ5RQyzHwq8BfgAElNki4CbgZOkPQucEKybmZmlppCRvH9yxZeGptyLGZmZi0yNRefRwGZmVmzTE115FFAZmbWLFMJyoojrYlnPemsmaXJCcpSm3jWk86aWZoydQ3KzMysmROUmZllkhOUmZllkhOUmZllkhOUZZLvLdU9/NtDyzKP4rNM8r2luod/e2hZ5gRlqUnr91TNdZlZz+YEZalJ6/dU4N9UmZmvQZmZWUb5DMoyKc3uwm16VaRyEX9g//4sWbo0hYjKU6pdvL239fRbBRhUXc2Hy5alUlcWP99OUJZJaXcXpjEQwIMAti7tv5mn32pbuQ9ycRefmZllUqbOoNLsIjAzs9KWqQTlUWBmZtasUwlK0njgZ0Av4FcRcXMqUZmlqBzvd5XmxXErTFqDbQC26V3BN+u/TqWuctbhBCWpF3APcALQBLwq6amIeDOt4MzSUI73u0rr4ri71Av3jQfudLvODJI4DHgvIt6PiH8AjwKnpROWmZn1dIqIjr1ROgsYHxH/I1k/Hzg8Ii7bZLuLgYuT1QOAtzsebovdgZUp1JMFPpZs6uixrIyIdp9qdVE7Af9NsqqnH0tB7aQz16DydcZulu0iYjIwuRP72XzH0tyIqE+zzmLxsWRTdx9LV7QT8N8kq3wshelMF18TsHer9Rrgk86FY2ZmltOZBPUqMERSraRtgXOBp9IJy8zMeroOd/FFxHpJlwHPkBtm/mBELEotsq1LvSukiHws2VQux1IuxwE+lqzqsmPp8CAJMzOzruS5+MzMLJOcoMzMLJMyn6Ak7S1plqTFkhZJ+kFS3k/STEnvJo+7FjvWtkiqlPSKpNeTY/lJUl4r6eXkWB5LBp1knqRekl6TNCNZL8njAJDUKOkNSQ2S5iZlJfMZczvJtnJpK93dTjKfoID1wJURcSBwBDBR0jDgWuC5iBgCPJesZ906YExEfAuoA8ZLOgK4BbgzOZbPgIuKGGN7/ABY3Gq9VI+j2XERUdfqNx2l9BlzO8m2cmor3ddOIqKkFuBJcvP/vQ1UJ2XVwNvFjq2dx7EDMB84nNyvsHsn5UcCzxQ7vgLir0k+jGOAGeR+uF1yx9HqeBqB3TcpK9nPmNtJdpZyaivd3U5K4QyqhaTBwAjgZWDPiFgKkDzuUbzICpec6jcAy4GZwF+BzyNifbJJEzCgWPG1w13A1cA3yfpulOZxNAvgj5LmJdMOQel+xgbjdpIl5dRWurWdZOp+UFsjqQ/wO2BSRHyR1rT33S0iNgB1knYBpgEH5tuse6NqH0knA8sjYp6k0c3FeTbN9HFsYlREfCJpD2CmpLeKHVBHuJ1kSxm2lW5tJyWRoCRVkGt0D0fEvyfFn0qqjoilkqrJfdMqGRHxuaTZ5K4X7CKpd/KNqhSmjBoFnCrpRKAS2Inct8RSO44WEfFJ8rhc0jRys/WX1GfM7SSTyqqtdHc7yXwXn3JfAR8AFkfEHa1eegqYkDyfQK7PPdMkVSXfCJG0PXA8uQuns4Czks0yfywRcV1E1ETEYHJTXD0fEedRYsfRTNKOkvo2PwfGAQspoc+Y20k2lVNbKUo7KfZFtwIuyh1N7vR3AdCQLCeS68d9Dng3eexX7FgLOJbhwGvJsSwEbkjK9wFeAd4DfgtsV+xY23FMo4EZpXwcSdyvJ8si4EdJecl8xtxOsr+UelspRjvxVEdmZpZJme/iMzOznskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJqsRJmp5M3LioefJGSRdJekfSbEm/lHR3Ul4l6XeSXk2WUcWN3qx7uJ2UJv9Qt8RJ6hcRf0umhHkV+C/Ai8BI4EvgeeD1iLhM0iPAvRExR9JAclP855uE06ysuJ2UppKYLNa26gpJZyTP9wbOB/5vRPwNQNJvgf2T148HhrWa4XonSX0j4svuDNisCNxOSpATVAlLpu8/HjgyItYksz6/Tf5bE0CuS/fIiPiqeyI0Kz63k9Lla1ClbWfgs6TRDSV3S4IdgG9L2lVSb+DMVtv/EbiseUVSXbdGa1YcbiclygmqtD0N9Ja0ALgReAn4GLiJ3N1UnwXeBP6ebH8FUC9pgaQ3gUu6P2Szbud2UqI8SKIMSeoTEauTb4bTgAcjYlqx4zLLEreT7PMZVHn6saQGcvfS+QCYXuR4zLLI7STjfAZlZmaZ5DMoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLpP8PlTlGZbaTvVAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x216 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "bins = np.linspace(df.age.min(), df.age.max(), 10)\n",
    "g = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\n",
    "g.map(plt.hist, 'age', bins=bins, ec=\"k\")\n",
    "\n",
    "g.axes[-1].legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Pre-processing:  Feature selection/extraction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Lets look at the day of the week people get the loan "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAGepJREFUeJzt3XmcVPW55/HPV2gvIriC2tIBWkQQldtgR+OCQUh4EdzwuoTEKGTMdTQuYQyDSzImN84YF8YlcSVq8EbEhUTMJTcaVIjgztKCiCFebbEVFJgYYxQFfeaPOt1poKGr6VPU6erv+/WqV1edOud3ntNdTz91fnXq91NEYGZmljU7FDsAMzOzprhAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlApUTS3pLuk/S6pAWSnpV0ckptD5U0M422tgdJcyRVFzsOK75SygtJ3SU9L2mRpCEF3M+HhWq7rXGBSoEkATOApyJiv4g4FBgDVBQpno7F2K9ZYyWYF8OBVyNiUETMTSMm2zoXqHQMAz6NiNvrF0TEmxHxcwBJHSRdJ+lFSYsl/fdk+dDkbGO6pFclTU2SGkkjk2XzgH+pb1fSzpLuTtpaJOmkZPk4SQ9J+g/gD605GElTJN0maXbyzvfLyT6XSZrSaL3bJM2XtFTSv22hrRHJu+aFSXxdWhObtSklkxeSqoBrgVGSaiTttKXXtqRaSVclz82XNFjSY5L+S9K5yTpdJD2RbLukPt4m9vs/G/1+msyxkhYRvrXyBlwE3LCV588Bfpjc/ydgPlAJDAX+Su4d5Q7As8DRQCfgLaAvIOBBYGay/VXAt5L7uwHLgZ2BcUAdsMcWYpgL1DRx+0oT604B7k/2fRLwAXBIEuMCoCpZb4/kZwdgDjAweTwHqAa6AU8BOyfLLwGuKPbfy7ftcyvBvBgH3Jzc3+JrG6gFzkvu3wAsBroC3YH3kuUdgV0atfUaoOTxh8nPEcDk5Fh3AGYCxxT777o9b+4KKgBJt5BLqE8j4ovkXmgDJZ2arLIruST7FHghIuqS7WqA3sCHwBsR8edk+b3kkpmkrRMlTUgedwJ6JvdnRcT/ayqmiGhpn/l/RERIWgK8GxFLkliWJjHWAKdLOodcspUDA8glY70vJcueTt4A70jun421QyWSF/Wae23/Nvm5BOgSEX8D/iZpnaTdgL8DV0k6Bvgc6AHsDaxq1MaI5LYoedyF3O/nqW2Muc1xgUrHUuCU+gcRcb6kbuTeEULuHdCFEfFY440kDQU+abToM/7xN9nSIIkCTomIP23S1uHkXvRNbyTNJfcublMTIuLxJpbXx/X5JjF+DnSUVAlMAL4YEX9Juv46NRHrrIj4xpbispJWinnReH9be21vNX+AM8idUR0aEesl1dJ0/vw0Iu7YShwlzZ9BpeNJoJOk8xot69zo/mPAeZLKACQdIGnnrbT3KlApqU/yuHESPAZc2KhPflA+AUbEkIioauK2tSTcml3IJf5fJe0NfK2JdZ4DjpK0fxJrZ0kHbOP+rO0p5bxo7Wt7V3LdfeslHQv0amKdx4D/1uizrR6S9mrBPto8F6gURK7DeDTwZUlvSHoBuIdcvzTAncArwEJJLwN3sJWz14hYR67r4nfJh8FvNnr6SqAMWJy0dWXax5OPiHiJXNfDUuBu4Okm1llNrt9+mqTF5JK6/3YM04qolPMihdf2VKBa0nxyZ1OvNrGPPwD3Ac8mXe3Tafpsr2TVfyhnZmaWKT6DMjOzTHKBMjOzTHKBMjOzTHKBMjOzTNquBWrkyJFB7nsMvvlWqrdWc5741g5uedmuBWrNmjXbc3dmbZLzxCzHXXxmZpZJLlBmZpZJLlBmZpZJHizWzErO+vXrqaurY926dcUOpV3r1KkTFRUVlJWVbdP2LlBmVnLq6uro2rUrvXv3Jhk/1raziGDt2rXU1dVRWVm5TW24i8/MSs66devYc889XZyKSBJ77rlnq85iXaCs5PUqL0dSq2+9ysuLfSjWAi5Oxdfav4G7+KzkrVi1irp9K1rdTsU7dSlEY2b58hmUmZW8tM6iW3I23aFDB6qqqjj44IM57bTT+Oijjxqee/jhh5HEq6/+Yxqo2tpaDj74YADmzJnDrrvuyqBBg+jXrx/HHHMMM2fO3Kj9yZMn079/f/r3789hhx3GvHnzGp4bOnQo/fr1o6qqiqqqKqZPn75RTPW32tra1vxaCy6vMyhJ/wP4DrkhKpYA3wbKgfuBPYCFwJkR8WmB4jQz22ZpnUXXy+dseqeddqKmpgaAM844g9tvv52LL74YgGnTpnH00Udz//338+Mf/7jJ7YcMGdJQlGpqahg9ejQ77bQTw4cPZ+bMmdxxxx3MmzePbt26sXDhQkaPHs0LL7zAPvvsA8DUqVOprq7eYkxtQbNnUJJ6ABcB1RFxMNABGANcA9wQEX2BvwBnFzJQM7O2asiQIbz22msAfPjhhzz99NPcdddd3H///XltX1VVxRVXXMHNN98MwDXXXMN1111Ht27dABg8eDBjx47llltuKcwBFEm+XXwdgZ0kdQQ6AyuBYeSmIIbcNM6j0w/PzKxt27BhA7///e855JBDAJgxYwYjR47kgAMOYI899mDhwoV5tTN48OCGLsGlS5dy6KGHbvR8dXU1S5cubXh8xhlnNHTlrV27FoCPP/64YdnJJ5+cxuEVVLNdfBHxtqRJwArgY+APwALg/YjYkKxWB/RoantJ5wDnAPTs2TONmM1KjvOk9NQXA8idQZ19dq6Tadq0aYwfPx6AMWPGMG3aNAYPHtxsexFbHwQ8Ija6aq4UuviaLVCSdgdOAiqB94GHgK81sWqTv72ImAxMBqiurs57mHWz9sR5UnqaKgZr167lySef5OWXX0YSn332GZK49tprm21v0aJFHHjggQAMGDCABQsWMGzYsIbnFy5cyIABA9I9iCLLp4vvK8AbEbE6ItYDvwGOBHZLuvwAKoB3ChSjmVlJmD59OmeddRZvvvkmtbW1vPXWW1RWVm50BV5TFi9ezJVXXsn5558PwMSJE7nkkksauu5qamqYMmUK3/3udwt+DNtTPlfxrQC+JKkzuS6+4cB8YDZwKrkr+cYCjxQqSDOz1ui5zz6pfo+tZ3KlXEtNmzaNSy+9dKNlp5xyCvfddx+XXHLJRsvnzp3LoEGD+Oijj9hrr7342c9+xvDhwwE48cQTefvttznyyCORRNeuXbn33nspL7Evk6u5fk0ASf8GfB3YACwid8l5D/5xmfki4FsR8cnW2qmuro758+e3NmazFpGU2hd188iXVg9f4DxpvWXLljV0h1lxbeFvkVee5PU9qIj4EfCjTRa/DhyWz/ZmZmYt5ZEkzMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzKzk7VvRM9XpNvatyG84qlWrVjFmzBj69OnDgAEDGDVqFMuXL2fp0qUMGzaMAw44gL59+3LllVc2fIVhypQpXHDBBZu11bt3b9asWbPRsilTptC9e/eNptB45ZVXAFi+fDmjRo1i//3358ADD+T000/ngQceaFivS5cuDVNynHXWWcyZM4fjjz++oe0ZM2YwcOBA+vfvzyGHHMKMGTManhs3bhw9evTgk09y3yxas2YNvXv3btHfJB+esNDMSt7Kt9/i8CseTa29538ystl1IoKTTz6ZsWPHNoxaXlNTw7vvvsu4ceO47bbbGDFiBB999BGnnHIKt956a8NIES3x9a9/vWGU83rr1q3juOOO4/rrr+eEE04AYPbs2XTv3r1h+KWhQ4cyadKkhvH65syZ07D9Sy+9xIQJE5g1axaVlZW88cYbfPWrX2W//fZj4MCBQG5uqbvvvpvzzjuvxTHny2dQZmYFMHv2bMrKyjj33HMbllVVVbF8+XKOOuooRowYAUDnzp25+eabufrqq1Pb93333ccRRxzRUJwAjj322IYJEZszadIkLr/8ciorKwGorKzksssu47rrrmtYZ/z48dxwww1s2LBhS820mguUmVkBvPzyy5tNiQFNT5XRp08fPvzwQz744IMW76dxt11VVRUff/zxFvedr3ym8+jZsydHH300v/rVr7Z5P81xF5+Z2Xa06bQYjW1p+dY01cXXWk3F2NSyyy+/nBNPPJHjjjsu1f3X8xmUmVkBHHTQQSxYsKDJ5ZuOtfj666/TpUsXunbtWtB9t2T7TWNsajqP/fffn6qqKh588MFt3tfWuECZmRXAsGHD+OSTT/jFL37RsOzFF1+kb9++zJs3j8cffxzITWx40UUXMXHixNT2/c1vfpNnnnmG3/3udw3LHn30UZYsWZLX9hMmTOCnP/0ptbW1ANTW1nLVVVfx/e9/f7N1f/CDHzBp0qRU4t6Uu/jMrOSV9/hCXlfetaS95kji4YcfZvz48Vx99dV06tSJ3r17c+ONN/LII49w4YUXcv755/PZZ59x5plnbnRp+ZQpUza6rPu5554DYODAgeywQ+684vTTT2fgwIE88MADG80ndeutt3LkkUcyc+ZMxo8fz/jx4ykrK2PgwIHcdNNNeR1fVVUV11xzDSeccALr16+nrKyMa6+9tmGG4MYOOuggBg8enPfU9S2R13QbafE0AlYMnm6j/fF0G9nRmuk23MVnZmaZlKkC1au8PLVvevcqsZklzczam0x9BrVi1apUumKAVKd3NrO2Z2uXc9v20dqPkDJ1BmVmloZOnTqxdu3aVv+DtG0XEaxdu5ZOnTptcxuZOoMyM0tDRUUFdXV1rF69utihtGudOnWiomLbe8VcoMys5JSVlTWMI2dtl7v4zMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzMwsk/IqUJJ2kzRd0quSlkk6QtIekmZJ+nPyc/dCB2tmZu1HvmdQNwGPRkR/4J+BZcClwBMR0Rd4InlsZmaWimYLlKRdgGOAuwAi4tOIeB84CbgnWe0eYHShgjQzs/YnnzOo/YDVwC8lLZJ0p6Sdgb0jYiVA8nOvpjaWdI6k+ZLm+1vdZk1znphtLp8C1REYDNwWEYOAv9OC7ryImBwR1RFR3b17920M06y0OU/MNpdPgaoD6iLi+eTxdHIF611J5QDJz/cKE6KZmbVHzRaoiFgFvCWpX7JoOPAK8FtgbLJsLPBIQSI0M7N2Kd/BYi8EpkraEXgd+Da54vagpLOBFcBphQnRrHXUoSyV+cHUoSyFaMwsX3kVqIioAaqbeGp4uuGYpS8+W8/hVzza6nae/8nIFKIxs3x5JAkzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8ukvAuUpA6SFkmamTyulPS8pD9LekDSjoUL08zM2puWnEF9D1jW6PE1wA0R0Rf4C3B2moGZmVn7lleBklQBHAfcmTwWMAyYnqxyDzC6EAGamVn7lO8Z1I3ARODz5PGewPsRsSF5XAf0aGpDSedImi9p/urVq1sVrFmpcp6Yba7ZAiXpeOC9iFjQeHETq0ZT20fE5Iiojojq7t27b2OYZqXNeWK2uY55rHMUcKKkUUAnYBdyZ1S7SeqYnEVVAO8ULkwzM2tvmj2DiojLIqIiInoDY4AnI+IMYDZwarLaWOCRgkVpZmbtTmu+B3UJcLGk18h9JnVXOiGZmZnl18XXICLmAHOS+68Dh6UfkpmZmUeSMDOzjHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKB2o56lZcjKZVbr/LyYh+OmVlBtWg+KGudFatWUbdvRSptVbxTl0o7ZmZZ5TMoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLpGYLlKQvSJotaZmkpZK+lyzfQ9IsSX9Ofu5e+HDNzKy9yOcMagPw/Yg4EPgScL6kAcClwBMR0Rd4InlsZmaWimYLVESsjIiFyf2/AcuAHsBJwD3JavcAowsVpJmZtT8t+gxKUm9gEPA8sHdErIRcEQP22sI250iaL2n+6tWrWxetWYlynphtLu8CJakL8GtgfER8kO92ETE5Iqojorp79+7bEqNZyXOemG0urwIlqYxccZoaEb9JFr8rqTx5vhx4rzAhmplZe5TPVXwC7gKWRcT1jZ76LTA2uT8WeCT98MzMrL3KZ8LCo4AzgSWSapJllwNXAw9KOhtYAZxWmBDNzKw9arZARcQ8QFt4eni64ZiZWTH0Ki9nxapVqbTVc599eHPlyla34ynfzcyMFatWUbdvRSptVbxTl0o7HurIMqlXeTmSUrmVorR+P73Ky4t9KGZb5DMoy6QsvpvLkrR+P6X4u7HS4TMoMzPLpJI9g/onSK17J60P/Cx/6lDmd/dm7VzJFqhPwF1EbVh8tp7Dr3g0lbae/8nIVNoxs+3LXXxmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJJTvUkZmZ5S/N8S/VoSyVdlygzMwsk+NfuovPrB2rH/Xfkx9aFvkMyqwd86j/lmU+gzIzs0xygbLU7FvRM7XuIjMzd/FZala+/VbmPmQ1s7YrUwUqi5c5mtn216u8nBWrVrW6nZ777MObK1emEJEVQ6YKVBYvc8yq+quv0uAktqxZsWpVKhdv+MKNtq1VBUrSSOAmoANwZ0RcnUpU1ixffWVmpW6bL5KQ1AG4BfgaMAD4hqQBaQVmZtZaWf2eV6/y8lRi6tyhY0lfmNSaM6jDgNci4nUASfcDJwGvpBGYmVlrZbWnIc0uzCweX1oUEdu2oXQqMDIivpM8PhM4PCIu2GS9c4Bzkof9gD9tpdluwJptCqht8PG1bfkc35qIaPEHoC3Mk3xjact8fG1bc8eXV5605gyqqXPCzapdREwGJufVoDQ/IqpbEVOm+fjatkIeX0vypNCxZIGPr21L6/ha80XdOuALjR5XAO+0LhwzM7Oc1hSoF4G+kiol7QiMAX6bTlhmZtbebXMXX0RskHQB8Bi5y8zvjoilrYwn7y6ONsrH17Zl6fiyFEsh+PjatlSOb5svkjAzMyskDxZrZmaZ5AJlZmaZlJkCJWmkpD9Jek3SpcWOJ02SviBptqRlkpZK+l6xY0qbpA6SFkmaWexYCkHSbpKmS3o1+TseUaQ4nCdtXCnnStp5konPoJJhk5YDXyV3+fqLwDcioiRGpZBUDpRHxEJJXYEFwOhSOT4ASRcD1cAuEXF8seNJm6R7gLkRcWdy1WrniHh/O8fgPCkBpZwraedJVs6gGoZNiohPgfphk0pCRKyMiIXJ/b8By4AexY0qPZIqgOOAO4sdSyFI2gU4BrgLICI+3d7FKeE8aeNKOVcKkSdZKVA9gLcaPa6jxF6Y9ST1BgYBzxc3klTdCEwEPi92IAWyH7Aa+GXSNXOnpJ2LEIfzpO0r5VxJPU+yUqDyGjaprZPUBfg1MD4iPih2PGmQdDzwXkQsKHYsBdQRGAzcFhGDgL8Dxfj8x3nShrWDXEk9T7JSoEp+2CRJZeSSbmpE/KbY8aToKOBESbXkupyGSbq3uCGlrg6oi4j6d/PTySViMeJwnrRdpZ4rqedJVgpUSQ+bpNxkK3cByyLi+mLHk6aIuCwiKiKiN7m/25MR8a0ih5WqiFgFvCWpX7JoOMWZVsZ50oaVeq4UIk8yMeV7gYZNypKjgDOBJZJqkmWXR8R/FjEma5kLgalJYXgd+Pb2DsB5Ym1AqnmSicvMzczMNpWVLj4zM7ONuECZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUBlgKQfS5qQYnv9JdUkw430SavdRu3PkVSddrtmW+M8aX9coErTaOCRiBgUEf9V7GDMMsp5knEuUEUi6QfJvD6PA/2SZf8q6UVJL0n6taTOkrpKeiMZAgZJu0iqlVQmqUrSc5IWS3pY0u6SRgHjge8kc+tMlHRRsu0Nkp5M7g+vH2ZF0ghJz0paKOmhZCw0JB0q6Y+SFkh6LJkOofEx7CDpHkn/e7v94qxdcZ60by5QRSDpUHJDnQwC/gX4YvLUbyLiixHxz+SmGjg7mXZgDrkh+km2+3VErAf+HbgkIgYCS4AfJd+6vx24ISKOBZ4ChiTbVgNdkiQ+GpgrqRvwQ+ArETEYmA9cnKzzc+DUiDgUuBv4P40OoyMwFVgeET9M8ddjBjhPLCNDHbVDQ4CHI+IjAEn146kdnLzL2g3oQm5IG8jNHTMRmEFu6JB/lbQrsFtE/DFZ5x7goSb2tQA4VLkJ4D4BFpJLwCHARcCXgAHA07mh0NgReJbcu9WDgVnJ8g7Aykbt3gE8GBGNk9EsTc6Tds4FqniaGmNqCrkZRF+SNA4YChART0vqLenLQIeIeDlJvOZ3ErFeudGTvw08AywGjgX6kHv32QeYFRHfaLydpEOApRGxpSmbnwGOlfR/I2JdPrGYbQPnSTvmLr7ieAo4WdJOyTu2E5LlXYGVSbfBGZts8+/ANOCXABHxV+Avkuq7Jc4E/kjTngImJD/nAucCNZEbiPE54ChJ+wMk/fkHAH8Cuks6IlleJumgRm3eBfwn8JAkv9GxQnCetHMuUEWQTGv9AFBDbu6buclT/4vcDKKzgFc32WwqsDu55Ks3FrhO0mKgCvjJFnY5FygHno2Id4F19fuMiNXAOGBa0s5zQP9kSvFTgWskvZTEeuQmx3E9ua6QX0nya8lS5Twxj2beRkg6FTgpIs4sdixmWeU8KS0+5WwDJP0c+BowqtixmGWV86T0+AzKzMwyyf2hZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSf8feZ3K8s9z83MAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x216 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df['dayofweek'] = df['effective_date'].dt.dayofweek\n",
    "bins = np.linspace(df.dayofweek.min(), df.dayofweek.max(), 10)\n",
    "g = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\n",
    "g.map(plt.hist, 'dayofweek', bins=bins, ec=\"k\")\n",
    "g.axes[-1].legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "We see that people who get the loan at the end of the week dont pay it off, so lets use Feature binarization to set a threshold values less then day 4 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "      <th>weekend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>male</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>female</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30     2016-09-08   \n",
       "1           2             2     PAIDOFF       1000     30     2016-09-08   \n",
       "2           3             3     PAIDOFF       1000     15     2016-09-08   \n",
       "3           4             4     PAIDOFF       1000     30     2016-09-09   \n",
       "4           6             6     PAIDOFF       1000     30     2016-09-09   \n",
       "\n",
       "    due_date  age             education  Gender  dayofweek  weekend  \n",
       "0 2016-10-07   45  High School or Below    male          3        0  \n",
       "1 2016-10-07   33              Bechalor  female          3        0  \n",
       "2 2016-09-22   27               college    male          3        0  \n",
       "3 2016-10-08   28               college  female          4        1  \n",
       "4 2016-10-08   29               college    male          4        1  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['weekend'] = df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Convert Categorical features to numerical values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets look at gender:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Gender  loan_status\n",
       "female  PAIDOFF        0.865385\n",
       "        COLLECTION     0.134615\n",
       "male    PAIDOFF        0.731293\n",
       "        COLLECTION     0.268707\n",
       "Name: loan_status, dtype: float64"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby(['Gender'])['loan_status'].value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "86 % of female pay there loans while only 73 % of males pay there loan\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets convert male to 0 and female to 1:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "      <th>weekend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30     2016-09-08   \n",
       "1           2             2     PAIDOFF       1000     30     2016-09-08   \n",
       "2           3             3     PAIDOFF       1000     15     2016-09-08   \n",
       "3           4             4     PAIDOFF       1000     30     2016-09-09   \n",
       "4           6             6     PAIDOFF       1000     30     2016-09-09   \n",
       "\n",
       "    due_date  age             education  Gender  dayofweek  weekend  \n",
       "0 2016-10-07   45  High School or Below       0          3        0  \n",
       "1 2016-10-07   33              Bechalor       1          3        0  \n",
       "2 2016-09-22   27               college       0          3        0  \n",
       "3 2016-10-08   28               college       1          4        1  \n",
       "4 2016-10-08   29               college       0          4        1  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## One Hot Encoding  \n",
    "#### How about education?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "education             loan_status\n",
       "Bechalor              PAIDOFF        0.750000\n",
       "                      COLLECTION     0.250000\n",
       "High School or Below  PAIDOFF        0.741722\n",
       "                      COLLECTION     0.258278\n",
       "Master or Above       COLLECTION     0.500000\n",
       "                      PAIDOFF        0.500000\n",
       "college               PAIDOFF        0.765101\n",
       "                      COLLECTION     0.234899\n",
       "Name: loan_status, dtype: float64"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby(['education'])['loan_status'].value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Feature before One Hot Encoding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>education</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>High School or Below</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>Bechalor</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>college</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>college</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>college</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender             education\n",
       "0       1000     30   45       0  High School or Below\n",
       "1       1000     30   33       1              Bechalor\n",
       "2       1000     15   27       0               college\n",
       "3       1000     30   28       1               college\n",
       "4       1000     30   29       0               college"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[['Principal','terms','age','Gender','education']].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Use one hot encoding technique to conver categorical varables to binary variables and append them to the feature Data Frame "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>weekend</th>\n",
       "      <th>Bechalor</th>\n",
       "      <th>High School or Below</th>\n",
       "      <th>college</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n",
       "0       1000     30   45       0        0         0                     1   \n",
       "1       1000     30   33       1        0         1                     0   \n",
       "2       1000     15   27       0        0         0                     0   \n",
       "3       1000     30   28       1        1         0                     0   \n",
       "4       1000     30   29       0        1         0                     0   \n",
       "\n",
       "   college  \n",
       "0        0  \n",
       "1        0  \n",
       "2        1  \n",
       "3        1  \n",
       "4        1  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Feature = df[['Principal','terms','age','Gender','weekend']]\n",
    "Feature = pd.concat([Feature,pd.get_dummies(df['education'])], axis=1)\n",
    "Feature.drop(['Master or Above'], axis = 1,inplace=True)\n",
    "Feature.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Changing variables in loan_status to binary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "      <th>weekend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1  loan_status  Principal  terms effective_date  \\\n",
       "0           0             0            1       1000     30     2016-09-08   \n",
       "1           2             2            1       1000     30     2016-09-08   \n",
       "2           3             3            1       1000     15     2016-09-08   \n",
       "3           4             4            1       1000     30     2016-09-09   \n",
       "4           6             6            1       1000     30     2016-09-09   \n",
       "\n",
       "    due_date  age             education  Gender  dayofweek  weekend  \n",
       "0 2016-10-07   45  High School or Below       0          3        0  \n",
       "1 2016-10-07   33              Bechalor       1          3        0  \n",
       "2 2016-09-22   27               college       0          3        0  \n",
       "3 2016-10-08   28               college       1          4        1  \n",
       "4 2016-10-08   29               college       0          4        1  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['loan_status'].replace(to_replace=['PAIDOFF','COLLECTION'], value=[1,0],inplace=True)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Feature selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets defind feature sets, X:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "X = Feature\n",
    "X[0:5]\n",
    "X.replace([np.inf, -np.inf], np.nan, inplace=True)\n",
    "X.fillna(X.mean(), inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "What are our lables?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 1, 1, 1, 1])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = df['loan_status'].values\n",
    "y[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Normalize Data "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Data Standardization give data zero mean and unit variance (technically should be done after train test split )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/preprocessing/data.py:645: DataConversionWarning: Data with input dtype int64 were all converted to float64 by StandardScaler.\n",
      "  return self.partial_fit(X, y)\n",
      "/opt/conda/envs/Python36/lib/python3.6/site-packages/ipykernel/__main__.py:1: DataConversionWarning: Data with input dtype int64 were all converted to float64 by StandardScaler.\n",
      "  if __name__ == '__main__':\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[ 0.51578458,  0.92071769,  2.33152555, -0.42056004, -1.20577805,\n",
       "        -0.38170062,  1.13639374, -0.86968108],\n",
       "       [ 0.51578458,  0.92071769,  0.34170148,  2.37778177, -1.20577805,\n",
       "         2.61985426, -0.87997669, -0.86968108],\n",
       "       [ 0.51578458, -0.95911111, -0.65321055, -0.42056004, -1.20577805,\n",
       "        -0.38170062, -0.87997669,  1.14984679],\n",
       "       [ 0.51578458,  0.92071769, -0.48739188,  2.37778177,  0.82934003,\n",
       "        -0.38170062, -0.87997669,  1.14984679],\n",
       "       [ 0.51578458,  0.92071769, -0.3215732 , -0.42056004,  0.82934003,\n",
       "        -0.38170062, -0.87997669,  1.14984679]])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X= preprocessing.StandardScaler().fit(X).transform(X)\n",
    "X[0:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_confusion_matrix(cm, classes,\n",
    "                          normalize=False,\n",
    "                          title='Confusion matrix',\n",
    "                          cmap=plt.cm.Blues):\n",
    "    \"\"\"\n",
    "    This function prints and plots the confusion matrix.\n",
    "    Normalization can be applied by setting `normalize=True`.\n",
    "    \"\"\"\n",
    "    if normalize:\n",
    "        cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n",
    "        print(\"Normalized confusion matrix\")\n",
    "    else:\n",
    "        print('Confusion matrix, without normalization')\n",
    "\n",
    "    print(cm)\n",
    "\n",
    "    plt.imshow(cm, interpolation='nearest', cmap=cmap)\n",
    "    plt.title(title)\n",
    "    plt.colorbar()\n",
    "    tick_marks = np.arange(len(classes))\n",
    "    plt.xticks(tick_marks, classes, rotation=45)\n",
    "    plt.yticks(tick_marks, classes)\n",
    "\n",
    "    fmt = '.2f' if normalize else 'd'\n",
    "    thresh = cm.max() / 2.\n",
    "    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n",
    "        plt.text(j, i, format(cm[i, j], fmt),\n",
    "                 horizontalalignment=\"center\",\n",
    "                 color=\"white\" if cm[i, j] > thresh else \"black\")\n",
    "\n",
    "    plt.tight_layout()\n",
    "    plt.ylabel('True label')\n",
    "    plt.xlabel('Predicted label')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Classification "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Now, it is your turn, use the training set to build an accurate model. Then use the test set to report the accuracy of the model\n",
    "You should use the following algorithm:\n",
    "- K Nearest Neighbor(KNN)\n",
    "- Decision Tree\n",
    "- Support Vector Machine\n",
    "- Logistic Regression\n",
    "\n",
    "\n",
    "\n",
    "__ Notice:__ \n",
    "- You can go above and change the pre-processing, feature selection, feature-extraction, and so on, to make a better model.\n",
    "- You should use either scikit-learn, Scipy or Numpy libraries for developing the classification algorithms.\n",
    "- You should include the code of the algorithm in the following cells."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.25, random_state=0)\n",
    "print ('Train set:', X_train.shape,  y_train.shape)\n",
    "print ('Test set:', X_test.shape,  y_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# K Nearest Neighbor(KNN)\n",
    "Notice: You should find the best k to build the model with the best accuracy.  \n",
    "**warning:** You should not use the __loan_test.csv__ for finding the best k, however, you can split your train_loan.csv into train and test to find the best __k__.\n",
    "\n",
    "# Found the best k value to be 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier(algorithm='auto', leaf_size=35, metric='minkowski',\n",
       "           metric_params=None, n_jobs=None, n_neighbors=5, p=2,\n",
       "           weights='uniform')"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.metrics import f1_score\n",
    "k = 5\n",
    "#Train Model and Predict  \n",
    "neigh = KNeighborsClassifier(n_neighbors = k, leaf_size = 35).fit(X, y)\n",
    "neigh"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Decision Tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeClassifier(class_weight=None, criterion='entropy', max_depth=4,\n",
       "            max_features=None, max_leaf_nodes=None,\n",
       "            min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "            min_samples_leaf=1, min_samples_split=2,\n",
       "            min_weight_fraction_leaf=0.0, presort=False, random_state=None,\n",
       "            splitter='best')"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DT = DecisionTreeClassifier(criterion=\"entropy\", max_depth = 4)\n",
    "DT.fit(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Support Vector Machine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVC(C=10, cache_size=200, class_weight=None, coef0=0.0,\n",
       "  decision_function_shape='ovr', degree=3, gamma=0.01, kernel='rbf',\n",
       "  max_iter=-1, probability=False, random_state=0, shrinking=True,\n",
       "  tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf_svm = svm.SVC(kernel='rbf', C = 10, gamma = 0.01, random_state = 0)\n",
    "clf_svm.fit(X, y) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=0.01, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=200, multi_class='warn',\n",
       "          n_jobs=None, penalty='l2', random_state=None, solver='liblinear',\n",
       "          tol=0.0001, verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "LR = LogisticRegression(C=0.01, solver='liblinear',max_iter = 200).fit(X,y)\n",
    "LR"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model Evaluation using Test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import jaccard_similarity_score\n",
    "from sklearn.metrics import f1_score\n",
    "from sklearn.metrics import log_loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, download and load the test set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2020-04-07 13:38:22--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv\n",
      "Resolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196\n",
      "Connecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 3642 (3.6K) [text/csv]\n",
      "Saving to: ‘loan_test.csv’\n",
      "\n",
      "100%[======================================>] 3,642       --.-K/s   in 0s      \n",
      "\n",
      "2020-04-07 13:38:22 (408 MB/s) - ‘loan_test.csv’ saved [3642/3642]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget -O loan_test.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Load Test set for evaluation "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>10/7/2016</td>\n",
       "      <td>50</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>9/9/2016</td>\n",
       "      <td>9/15/2016</td>\n",
       "      <td>35</td>\n",
       "      <td>Master or Above</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>21</td>\n",
       "      <td>21</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/10/2016</td>\n",
       "      <td>10/9/2016</td>\n",
       "      <td>43</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/10/2016</td>\n",
       "      <td>10/9/2016</td>\n",
       "      <td>26</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>9/11/2016</td>\n",
       "      <td>9/25/2016</td>\n",
       "      <td>29</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           1             1     PAIDOFF       1000     30       9/8/2016   \n",
       "1           5             5     PAIDOFF        300      7       9/9/2016   \n",
       "2          21            21     PAIDOFF       1000     30      9/10/2016   \n",
       "3          24            24     PAIDOFF       1000     30      9/10/2016   \n",
       "4          35            35     PAIDOFF        800     15      9/11/2016   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0  10/7/2016   50              Bechalor  female  \n",
       "1  9/15/2016   35       Master or Above    male  \n",
       "2  10/9/2016   43  High School or Below  female  \n",
       "3  10/9/2016   26               college    male  \n",
       "4  9/25/2016   29              Bechalor    male  "
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df = pd.read_csv('loan_test.csv')\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(54, 10)"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(54, 12)"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df['due_date'] = pd.to_datetime(test_df['due_date'])\n",
    "test_df['effective_date'] = pd.to_datetime(test_df['effective_date'])\n",
    "test_df['dayofweek'] = test_df['effective_date'].dt.dayofweek\n",
    "test_df['weekend'] = test_df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\n",
    "test_df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\n",
    "test_df['loan_status'].replace(to_replace=['PAIDOFF','COLLECTION'], value=[1,0],inplace=True)\n",
    "test_df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>weekend</th>\n",
       "      <th>Bechalor</th>\n",
       "      <th>High School or Below</th>\n",
       "      <th>college</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>50</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>35</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>43</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>26</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n",
       "0       1000     30   50       1        0         1                     0   \n",
       "1        300      7   35       0        1         0                     0   \n",
       "2       1000     30   43       1        1         0                     1   \n",
       "3       1000     30   26       0        1         0                     0   \n",
       "4        800     15   29       0        1         1                     0   \n",
       "\n",
       "   college  \n",
       "0        0  \n",
       "1        0  \n",
       "2        0  \n",
       "3        1  \n",
       "4        0  "
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Feature_test = test_df[['Principal','terms','age','Gender','weekend']]\n",
    "Feature_test = pd.concat([Feature_test,pd.get_dummies(test_df['education'])], axis=1)\n",
    "Feature_test.drop(['Master or Above'], axis = 1,inplace=True)\n",
    "Feature_test.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_test = Feature_test\n",
    "#np.where(X_testdata.values >= np.finfo(np.float64).max)\n",
    "X_test.replace([np.inf, -np.inf], np.nan, inplace=True)\n",
    "X_test.fillna(X_test.mean(), inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 1, 1, 1, 1])"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_test = test_df['loan_status'].values\n",
    "y_test[0:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/preprocessing/data.py:645: DataConversionWarning: Data with input dtype int64 were all converted to float64 by StandardScaler.\n",
      "  return self.partial_fit(X, y)\n",
      "/opt/conda/envs/Python36/lib/python3.6/site-packages/ipykernel/__main__.py:1: DataConversionWarning: Data with input dtype int64 were all converted to float64 by StandardScaler.\n",
      "  if __name__ == '__main__':\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(54, 8)"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_test = preprocessing.StandardScaler().fit(X_test).transform(X_test)\n",
    "X_test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Predicting on all models at once"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "y_pred_knn = neigh.predict(X_test)\n",
    "y_pred_svm = clf_svm.predict(X_test)\n",
    "y_pred_lr = LR.predict(X_test)\n",
    "y_pred_dt = DT.predict(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## KNN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "jaccard_knn = jaccard_similarity_score(y_test, y_pred_knn)\n",
    "f1_score_knn = f1_score(y_pred_knn, y_test)\n",
    "accuracy_knn = metrics.accuracy_score(y_test, y_pred_knn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.50      0.36      0.42        14\n",
      "           1       0.80      0.88      0.83        40\n",
      "\n",
      "   micro avg       0.74      0.74      0.74        54\n",
      "   macro avg       0.65      0.62      0.62        54\n",
      "weighted avg       0.72      0.74      0.73        54\n",
      "\n",
      "Confusion matrix, without normalization\n",
      "[[ 5  9]\n",
      " [ 5 35]]\n",
      "The F1-score is 0.83 and Jaccard similarity is 0.74 for KNN classifier\n",
      "Test set Accuracy:  0.74\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVMAAAEmCAYAAADfpHMGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3XmcneP9//HXe2ZCkNgaSyhiV9UKCV+11FKC1t7F1pZSqRatVpUvvi1FbW31V7XFXmqtBrWLpUFtERFJg5TaI4ugQYQkn98f13U4YuacMzNn5pwz8372cT9yzn3u+7qvMzWfufZLEYGZmXVOU60zYGbWEziYmplVgYOpmVkVOJiamVWBg6mZWRU4mJqZVYGDqdWUpEUk/V3S25Ku60Q6+0q6s5p5qxVJW0h6ptb5sPaRx5laJSTtA/wMWAeYBYwDTo6IBzqZ7neAw4BNI2JupzNa5yQFsGZE/LvWebHqcsnUypL0M+APwG+A5YCVgXOAXauQ/CrAs70hkFZCUkut82AdFBE+fLR5AEsA7wDfLHHNwqRg+1o+/gAsnD/bCngFOAKYBkwBvpc/OwH4APgwP+NA4HjgiqK0BwEBtOT3+wPPk0rH/wH2LTr/QNF9mwKPAW/nfzct+uw+4ETgwZzOncCANr5bIf+/KMr/bsBXgWeBmcAxRddvDDwEvJWv/ROwUP5sdP4u7+bvu2dR+kcBrwOXF87le1bPz9gwv18BmAFsVev/Nnx88nDJ1Mr5EtAXGFnimmOBTYDBwPqkgHJc0efLk4LyiqSAebakpSLiV6TS7jUR0S8iLiqVEUmLAX8EdoyI/qSAOa6V65YGbsnXfgb4PXCLpM8UXbYP8D1gWWAh4OclHr086WewIvBL4ALg28AQYAvgl5JWy9fOA34KDCD97L4C/AggIr6cr1k/f99ritJfmlRKH1784Ih4jhRo/yJpUeAS4NKIuK9Efq0GHEytnM8AM6J0NXxf4NcRMS0ippNKnN8p+vzD/PmHEXErqVS2dgfzMx9YT9IiETElIia2cs3XgMkRcXlEzI2Iq4CngZ2LrrkkIp6NiNnAtaQ/BG35kNQ+/CFwNSlQ/r+ImJWfPxH4IkBEPB4RD+fnvgCcD2xZwXf6VUTMyfn5hIi4AJgMPAIMJP3xsjrjYGrlvAEMKNOWtwLwYtH7F/O5j9JYIBi/B/Rrb0Yi4l1S1fhgYIqkWyStU0F+Cnlasej96+3IzxsRMS+/LgS7qUWfzy7cL2ktSTdLel3Sf0kl7wEl0gaYHhHvl7nmAmA94KyImFPmWqsBB1Mr5yHgfVI7YVteI1VRC1bO5zriXWDRovfLF38YEXdExHakEtrTpCBTLj+FPL3awTy1x7mkfK0ZEYsDxwAqc0/JITWS+pHaoS8Cjs/NGFZnHEytpIh4m9ROeLak3SQtKqmPpB0lnZ4vuwo4TtIykgbk66/o4CPHAV+WtLKkJYD/LXwgaTlJu+S20zmk5oJ5raRxK7CWpH0ktUjaE1gXuLmDeWqP/sB/gXdyqfmHC3w+FVjtU3eV9v+AxyPi+6S24PM6nUurOgdTKysifk8aY3ocMB14GTgUuCFfchIwBhgPPAWMzec68qy7gGtyWo/zyQDYRBoV8Bqph3tLcufOAmm8AeyUr32D1BO/U0TM6Eie2unnpM6tWaRS8zULfH48cJmktyR9q1xiknYFdiA1bUD6/2FDSftWLcdWFR60b2ZWBS6ZmplVgYOpmfVakvpKelTSk5ImSjohn79U0n8kjctHqaFzAHjqmpn1ZnOAbSLiHUl9gAck3ZY/OzIi/lppQg6mZtZrReo0eie/7ZOPDnUkuQOqmw0YMCBWWWVQrbPRa30wb36ts9BrvfryS8x8Y0a5MbcVa158lYi5n5ow9gkxe/pE0jjpghERMaL4GknNpJEjawBnR8RRki4lTQeeA9wNHF1usoRLpt1slVUG8eAjY2qdjV7r1Zmlf/ms6+w2bLOqphdzZ7Pw2qVHl70/7uz3I2JoyXTS7LbBkpYERkpajzS++XXSug0jSOsj/LpUOu6AMrPGJEFTc+mjHSLiLdKKYjvkdR8il0YvIS3eU5KDqZk1LjWVPsrdnmbtLZlfLwJsCzwtaWA+J9JU6gnl0nI138walzrdBDuQNCOtmVS4vDYibpZ0j6RlSOsqjOPjGWhtcjA1swaldlflFxQR44ENWjm/TXvTcjA1s8YkKqrKdxcHUzNrUKpGNb9qHEzNrHF1sppfTQ6mZtag5Gq+mVmnCVfzzcw6T9BUPyGsfnJiZtZeTS6Zmpl1jodGmZlVQ+cH7VeTg6mZNS53QJmZVYGr+WZmnSRX883MqsPVfDOzzvIMKDOzzhOu5puZdZ5LpmZm1eE2UzOzKnA138ysk+RqvplZdbiab2bWOQKamlwyNTPrHOWjTjiYmlmDEqqjan79lJHNzNqpqamp5FGOpL6SHpX0pKSJkk7I51eV9IikyZKukbRQ2bxU4fuYmdWEpJJHBeYA20TE+sBgYAdJmwCnAWdGxJrAm8CB5RJyMDWzxqQKjjIieSe/7ZOPALYB/prPXwbsVi4tB1Mza0hClVTzB0gaU3QM/1Q6UrOkccA04C7gOeCtiJibL3kFWLFcftwBZWYNq4Kq/IyIGFrqgoiYBwyWtCQwEvhca5eVe5CDqZk1rGr25kfEW5LuAzYBlpTUkkunnwVeK3e/q/lm1pgEalLJo2wS0jK5RIqkRYBtgUnAvcA38mX7ATeWS8slUzNrSKrOONOBwGWSmkmFy2sj4mZJ/wKulnQS8ARwUbmEHEzNrGF1NphGxHhgg1bOPw9s3J60HEzNrDHlan69cDA1s4ZVT9NJHUzNrCEVxpnWCwdTq8jaawyif7/+NDc309LSwoOPjKl1lnqVS0eczTVXXEIQ7Lnv9/jeDw6tdZbqQ/0UTB1MrXK3j7qXAQMG1Dobvc6zkyZyzRWX8LfbR9NnoYU4YK9d2Xq7HRi02hq1zlptqb6q+fVTRjazVv178jMMHrIRiyy6KC0tLWy86ebceetNtc5WXejsqlFVzUu3Ps0aliR23nEYm248hIsuGFHr7PQqa62zLo89/CBvznyD2e+9x32j7mDKq6/UOlv1oZMLnVRTlwZTSctLulrSc5L+JelWSWu1ce0gSRPy660k3dzBZx4uadGi97cWZjh0ML0NJF2YX0vSHyX9W9J4SRvm88tIur2jz2gE9/zjQR56bCw33Hwb5597Ng/cP7rWWeo11lhrHYYf+jP2+9ZOHLD3rnzu81+gucUtdFCVJfiqpsuCqdI3GQncFxGrR8S6wDHAcl31zOxw4KNgGhFfjYi3OpHeMcBZ+fWOwJr5GA6cm58xHZgiabNOPKeurbDCCgAsu+yy7LLb7jz22KM1zlHv8q199+emUQ9x1Y13scSSSzFotdVrnaWakypaNarbdOXTtgY+jIjzCiciYhzwgKQzJE2Q9JSkPUslImkxSRdLekzSE5J2zeebJf02pzFe0mGSfgysANwr6d583QuSBuSS7yRJF+QVte/Mc3GRtFFO46FC3vL5/sAXI+LJnJ1dgT/nNRAfJi2GMDB/dgOwb7V+ePXk3XffZdasWR+9HnXXnXz+8+vVOFe9yxvTpwHw2isvc+etN7Hz7t+qcY7qQz2VTLuyrrAe8Hgr5/cgrWi9PjAAeExSqTrjscA9EXFArq4/KmkU8F1gVWCDiJgraemImCnpZ8DWETGjlbTWBPaOiIMkXQt8HbgCuAQYHhH/lHRq0fVDgQlF71cEXi56X1jncAowBjiptS+Q11AcDrDSyiuX+Kr1adrUqez5jd0BmDtvLnvutQ/Dtt+hxrnqXQ45cB/efHMmfVr6cPwpZ7LEkkvVOkv1oX4682syNGpz4Kq8huBUSf8ANgLGt3H9MGAXST/P7/sCK5NWdzmvsIBrRMys4Nn/yaVjSIF+UA7Q/SPin/n8lcBO+fVAYHrR/a39X1dY53AaqVT86QsiRgAjAIYMGVp2XcR6s+pqq/Ho2CfLX2hd5uqbRtU6C/VHvWer54l8vIRVsfb+LRHw9Yh45hMnUxm+vYFpTtHrecAiZfIzmxS8C14BVip6X7zOYd98vZl1AwF1NMy0S9tM7wEWlnRQ4YSkjUibU+2Z2zyXAb4MlOrNuAM4LAdPJBVWeLkTOFhSSz6/dD4/C+hfaSYj4k1gVt5EC2Cvoo8nAcUjo28Cvpt79TcB3o6IKfmztfhkk4CZdanS7aU9pjc/IgLYHdguD42aCBxPqkaPB54kBdxfRMTrJZI6kbTJ1fjcMXRiPn8h8FI+/ySwTz4/Arit0AFVoQOBEZIeIv3Bezt/h6eBJXJHFMCtwPPAv4ELgB8VpbE1cEs7nmlmndTUpJJHd1KKeb2bpH6FHQolHQ0MjIif5Pc/BWZFxIVl0hgN7JpLum0aMmRoeF577bw60y0xtbLbsM14atzYqkW4vgPXikH7nVXymmdO2+HxcntAVUv9tN7W1tckjcsl3y34ZK/8uXyyrfVTcnPF78sFUjOrHlFfJVNPowAi4hrgmjY+ex+4vMz900njTM2sG3V3wCzFwdTMGpPqqzffwdTMGlIaGlU/0dTB1MwaVPe3i5biYGpmDcslUzOzzqqzNlMPjTKzhlSNoVGSVpJ0b15RbqKkwvjy4yW9modMjpP01XJpuWRqZg2rCtX8ucARETE2z3R8XNJd+bMzI+K3lSbkYGpmDauzsTSvrTElv54laRJpWc12czXfzBqSVFE1f4CkMUXH8LbT0yBgA+CRfOrQvGj8xZLKLiDrYGpmDaqiVaNmRMTQoqPV3SAl9QOuBw6PiP+SppGvTlrIfgrwu3K5cTA1s4YllT4qS0N9SIH0LxHxN4CImBoR8yJiPmmFuI3LpeM2UzNrTOr83Py8TvJFwKSI+H3R+YFFaxXvTgVrFTuYmllDqtJ00s2A7wBPSSpsaXQMsLekwaTdPF4AflAuIQdTM2tYnQ2mEfEArW9ddGt703IwNbOG5bn5ZmadVWfTSR1Mzawhie7fNK+UNoOppMVL3ZjHYpmZ1Uxzg1TzJ5J6sopzW3gfwMpdmC8zs7LqqGDadjCNiJW6MyNmZu2RBubXTzStaAaUpL0kHZNff1bSkK7NlplZec1NKnl0p7LBVNKfgK1JA1sB3gPO68pMmZlVohrTSaulkt78TSNiQ0lPAETETEkLdXG+zMxKEqlHv15UEkw/lNRE6nRC0meA+V2aKzOzctT9VflSKmkzPZu0osoykk4AHgBO69JcmZlVoKGq+RHxZ0mPA9vmU9+MiLIrqJiZdSUBTXXUm1/pDKhm4ENSVd9roJpZXainufmV9OYfC1wFrAB8FrhS0v92dcbMzEopV8Wvu2o+8G1gSES8ByDpZOBx4JSuzJiZWTmNVs1/cYHrWoDnuyY7ZmaVa4hgKulMUhvpe8BESXfk98NIPfpmZjWTOqBqnYuPlSqZFnrsJwK3FJ1/uOuyY2ZWITXIEnwRcVF3ZsTMrL3qqTe/bJuppNWBk4F1gb6F8xGxVhfmy8yspHqr5lcyZvRS4BJS3ncErgWu7sI8mZlVRLmq39bRnSoJpotGxB0AEfFcRBxHWkXKzKxmJGiWSh7dqZKhUXOUQvxzkg4GXgWW7dpsmZmVV0f9TxWVTH8K9AN+DGwGHAQc0JWZMjOrRGer+ZJWknSvpEmSJkr6ST6/tKS7JE3O/y5VLq1KFjp5JL+cxccLRJuZ1ZSoyhJ8c4EjImKspP7A45LuAvYH7o6IUyUdDRwNHFUqoVKD9keS1zBtTUTs0ZGcm5lVRRXm30fEFGBKfj1L0iRgRWBXYKt82WXAfXQ0mAJ/6lw2zerPetsfWess9Fpznn2l6mlWUJUfIGlM0fsRETGijbQGARsAjwDL5UBLREyRVLafqNSg/bvL3WxmViuCSnrsZ0TE0LJpSf1Ii+AfHhH/7ciwKq9NamYNq0mlj0pI6kMKpH+JiL/l01MlDcyfDwSmlc1Lx76CmVntdTaY5mGfFwGTIuL3RR/dBOyXX+8H3FgurUpX2kfSwhExp9Lrzcy6kkQ1evM3I41SekrSuHzuGOBU4FpJBwIvAd8sl1Alc/M3JkXuJYCVJa0PfD8iDutg5s3MqqIKvfkPQJv7RX+lPWlVUs3/I7AT8EZ++JN4OqmZ1VhhQ71SR3eqpJrfFBEvLtC7Na+L8mNmVrHmOppOWkkwfTlX9UNSM3AY8GzXZsvMrDTVoPRZSiXB9Iekqv7KwFRgVD5nZlZTzXU0HqmSufnTgL26IS9mZhUrtJnWi0p68y+glTn6ETG8S3JkZlahOoqlFVXzRxW97gvsDrzcNdkxM6uQKppO2m0qqeZfU/xe0uXAXV2WIzOzCtTbHlAVz4AqsiqwSrUzYmbWXg0VTCW9ycdtpk3ATNJCqWZmNSOqMp20akoG07wIwPqkfZ8A5kdEmwtGm5l1myosDl1NJUdp5cA5MiLm5cOB1MzqRj1NJ61kyOujkjbs8pyYmbVDquaXPrpTqT2gWiJiLrA5cJCk54B3Sd8hIsIB1sxqSDS1ueBT9yvVZvoosCGwWzflxcysYqK+2kxLBVMBRMRz3ZQXM7PKCVoapDd/GUk/a+vDBZb4NzPrVo1UMm0G+tH2KtRmZjXVKAudTImIX3dbTszM2iFt9VzrXHysbJupmVldUlogul6UCqbt2kzKzKy71U8oLRFMI2Jmd2bEzKw9UjW/fsJpR1aNMjOrC3UUSyuaTmpmVoeEVPoom4J0saRpkiYUnTte0quSxuXjq5XkxsHUzBpSoZpf6qjApcAOrZw/MyIG5+PWShJyMDWzhqUyRzkRMZq0RnOnOZiaWWPKQ6PKVPMHSBpTdFS6EeihksbnZoClKrnBHVBm1pAq7M2fERFD25n0ucCJpB1GTgR+BxxQ7iaXTM2sYXW2mt+aiJiaF8OfD1wAbFzJfQ6mZtawpNJHx9LUwKK3uwMT2rq2mKv5ZtaQqjFoX9JVwFakttVXgF8BW0kaTKrmvwD8oJK0HEzNrEEJdXJCaUTs3crpizqSloOpmTWsepoB5WBqZg1J8tx8M7OqqKNY6mBqlVl7jUH079ef5uZmWlpaePCRMbXOUo+28EItjLrocBZaqIWW5mZGjnqCk867lREnfJsthqzB2++8D8DwX17O+GdfrXFua6ezbabV5GBqFbt91L0MGDCg1tnoFeZ8MJcdhv+Rd2d/QEtLE/dc/DPufPBfABzzhxsYOWpcjXNYe16Cz8wq8u7sDwDo09JMS0szEVHjHNWfOoqlHrRvlZHEzjsOY9ONh3DRBSNqnZ1eoalJPHz10bx096nc8/DTPDbhRQCOP2RnHr3mfzn9iD1YqE/vLg+pzP+6U82DqaR5ec3ACZKuk7RometvlbRkK+ePl/TzNu4ZLunpfDwqafOiz7aQNDHnYRFJZ+T3Z+TPD5f03fz6m/mz+ZKGFqXxBUmXdvBH0BDu+ceDPPTYWG64+TbOP/dsHrh/dK2z1OPNnx9ssteprLH9cQxdbxXWXX0gvzzrJtbf/UQ2//YZLLXEYhzxvW1rnc2aEaWX3+vuJoCaB1Ngdl4zcD3gA+DgUhdHxFcj4q1KE5e0E2kGw+YRsU5O/0pJy+dL9gV+m/MwO1+7YUQcKamFtMDBlfnaCcAewCciSUQ8BXxW0sqV5qvRrLDCCgAsu+yy7LLb7jz22KM1zlHv8fY7sxk9ZjLDNl2X12f8F4APPpzLn298mKGfH1TbzNVSmamk3d0EUA/BtNj9wBoAkm6Q9HguCX60bJakFyQNyK+PlfSMpFHA2m2keRRwZETMAIiIscBlwCGSvg98C/ilpL9IuglYDHhE0p7ANsDYiJib750UEc+08Zy/A3t17uvXp3fffZdZs2Z99HrUXXfy+c+vV+Nc9WwDlurHEv0WAaDvwn3Y5n/W5pkXprL8gMU/umaXrb/Iv557rVZZrAtdsdBJR9VNg0suBe4I3J5PHRARMyUtAjwm6fqIeKPo+iGk4LUB6XuMBR5vJenPt3J+DLBfRPxfrvLfHBF/zem+ExGD8+sT2kizNWOAo4HTW/luw4HhACut3HiF12lTp7LnN3YHYO68uey51z4M2761xcmtWpYfsDgX/Po7NDc10dQkrr9rLLfdP4Hbzj+MAUv1R4Lxz7zCYSdfXeus1ox78z9tEUmFcR738/G82B9L2j2/XglYE3ij6L4tgJER8R5ALlVWSqRFDMoZCEyqMM1pwAqtfRARI4ARAEOGDG24LtlVV1uNR8c+Wets9CoTJr/Gl/Y+7VPnd/zBWTXITR2rn1haF8F0dqEkWCBpK2Bb4EsR8Z6k+4C+rdxbSWD6FzAEuKfo3Ib5fNm8tfHc1vTN15tZN6mnQfv11mZasATwZg6k6wCbtHLNaGD33APfH9i5jbROB06T9BmAvLTW/sA5FeRjErkNtwJrUeG6h2ZWHU0qfXSneiiZtuZ24GBJ44FngIcXvCAixkq6BhgHvEhqIviUiLhJ0orAPyUFMAv4dkRMqSAftwGXF97kZoezgGWAWySNi4jt88dbA7dU+gXNrArqp2Ba+2AaEf1aOTeH1BnV2vWDil6fDJxcwTPOJe3r0tpn+7eVn4h4UdIbktaMiMkRMRIYuWAakhYGhgKHl8uLmVVH6rGvn2har9X8enI0qSOqlJWBowtDqMysG5Sp4ruaX2fyuNK2xpYWrpkMTO6eHJnZR+qnYOpgamaNqvvn35fiYGpmDUl0f1W+FAdTM2tcDqZmZp3nar6ZWRXUUzXfQ6PMrDGVWzKqgkAr6WJJ0yRNKDq3tKS7JE3O/y5VSXYcTM2sYVVhpf1LgQWXQDsauDsi1gTuzu/LcjA1s4ZU6M3vzKD9iBgNzFzg9K6kNY/J/+5WSX7cZmpmjat8wBwgqXhf8hF5ScxSlius3RERUyQtW0lWHEzNrGFVUJWfERFDy11UDa7mm1nD6qK5+VMlDQTI/06rKC8dfpyZWa11zSZQNwH75df7ATdWcpODqZk1pMISfJ3pzZd0FfAQsLakVyQdCJwKbCdpMrBdfl+W20zNrDFVYZm9iNi7jY++0t60HEzNrHHV0QwoB1Mza1CiyVs9m5l1Tuf6mKrPwdTMGlcdRVMHUzNrWK7mm5lVQf2EUgdTM2tUgjoqmDqYmlljEqA6iqYOpmbWsOonlDqYmlkDq6OCqYOpmTUuV/PNzKqgfkKpg6mZNSi5N9/MrDpczTczq4L6CaUOpmbWwOqoYOpgamaNSXW2BJ+3LTEzqwKXTM2sYdVRwdTB1MwalLwEn5lZp3mlfTOzaqmjaOpgamYNqxrVfEkvALOAecDciBjakXQcTM2sYVWxYLp1RMzoTAIOpmbWuOqomu9xpmbWkESq5pc6KhTAnZIelzS8w/mJiI7eax0gaTrwYq3z0QkDgE5Vh6zDGv1nv0pELFOtxCTdTvqZlNIXeL/o/YiIGLFAOitExGuSlgXuAg6LiNHtzo+DqbWHpDEdbaC3zvHPvutJOh54JyJ+2957Xc03s15L0mKS+hdeA8OACR1Jyx1QZtabLQeMzOuitgBXRsTtHUnIwdTaa0T5S6yL+GdfZRHxPLB+NdJym6mZWRW4zdTMrAocTM3MqsDB1KyXkOTf9y7kH65ZDydpL0mrRsT8WuelJ3MwtW4jaVlJv5G0vKQl8rk6ml3d8+Sf82eBcyQdJmmHWuepp3JvvnUbSX2B3wBzgcWAyyLiUUlNLjV1LUlDgfWAbwK3RMQ5Nc5Sj+Ngal1O0sKkdSLn5fdrAlsCxwHfj4hRkhT+j7FqJH0B+FfhZ57PLQRsBJwOXBgRl9Qqfz2Rq/nWpfIv9e3AhZJOB4iIyRFxIXAU8GdJX3IgrR5JNwCjga9I+mhiTkR8EBEPAicAm0tat1Z57IkcTK3LSFoaOAf4M3ASsLGksyStmkui1wDHAodKWr6Wee0pJG0OvAmcDBzOAgE1Gwe8CqzUzdnr0RxMrSv1IS1/dl9EPAfsCPQDfpo/A7gTeA7oX5Mc9iCSliIFylPyqkd/A44kBdS+hesiYhrwFHCEpH41yWwP5GBqVSdpBUkDI2Iq8AiwmaT+ETEbOAT4HHAqQES8Stp753s1y3APIGk7YFhEvAO8BJCbUq4CfgEMydf9KH92HXAG8GFNMtwDOZhaV9gR+Ep+PQnYFRgiqV9EvAfsB6wqaXGAiDgBuFlSn1ZTs5JyqbMPMFxSS0S8X6jaR8RFwAXArySNB75UdOvoiJjT/TnumRxMrSu8B2wKEBF/AR4EDgN2kLQiadzjZ4Gmolk5D0eES0ntJOl84OsRcSvwKPBDSc0RMVdSM0BEXA18ARgXEd8p3OtAWl0eGmVdQtIo4P5c6kTSfqRxjkOAJUntetfVMIsNLw/A/0lE7Cjpf4CVgU2AG4EHImJ+LqFuDewWEYfk+zyutws4mFpVSVopIl6WtAHwE+AfhfGMeUXzJYGFI+LfHlvacZK2JDWhTCZ14J0F/AX4JbAocDdwZ0R8mNurZ+X7HEi7iKv51mmFKaGS1gHulrRPRDwB3AFsKekogIiYFREvR8S/83sH0g6QdCnw09wr/09gHWBqRHxA6lR6nVQaPStvEvdO4V4H0q7jkqlVhaSvAruTtoFYDTg5Iq6StBnwf6Qe5suACRHxdu1y2tjyPkW/Jg3KHwr8h1Qy/TtwcERcmTuklgcOAFYklVgfyMHWuoiDqXWapOWAe0m/vJNI7XanA7/JA/ORdDQwHZgJ3OBSacdJOpY0VveuiNg7n/saKWgOj4hri65dGXjdgbTreQ8o67CiNs95pNLRYxExL3c+3QCcmK+5OiJOzfe0OJC2n6SBwOIR8Qzwb1LJdJqkLYCxEXGLpL2BWyT9J5+bFxEv1TDbvYrbTK3dipbN+wxARMwgDYe6Lr+fBzwJ3AbsLWlQ0T3zsHaRdDFwLnCHpDOBtSJiD+BF4Lukabp9I+I2YHBEPFa8wIl1DwdTa5dCaTS3kY6SdHZ+fSAwR9J9kg4mzQ2/kVS1/7BQGnWptH1yIF0qInYDtiJ1OK0r6UjgTOB50rJ6W0vqExHj833+3e5m/oFbxYoC6RBgH+AYUgfI9qTOp32AkaS1Sr9Fmpc/GHAA7QBJA4ClSD8lEpq8AAAJnklEQVRLIuIF0h+o84EvAhsAp5F+zssWT3pwr333c5uplSVpdaA5Ip6VtAxwBWlA/q2SRgN7kNbJ7AOcnWffbEJaMeq7EfFazTLf2N4FFgeGSnqEVLD/IL/eF9guIsZKOsIl/tpzydQq8SVg6dwuN51Uvfy6pB3ywhrXAeNJs5sG5nv+C3ytUO20DpkPvEJqB50PkKvys0mrbfWDj5tOvAVMbblkamVFxBV59tJESXtFxAhJs4HTJRERt0v6C3BbXgWKiPhXTTPdA0TEnDz3/u+SPsiLlhSq8t8BHl7gepdOa8jjTK1VeZ3LtXI1civgCWA46Zd4v4h4Ig/F+Q1wSF5owzpB0jDSViOv5FJmUx5qtiVwMWl0RACrAjMj4rs1zK4twCVTa0tf4CRJU4GNgf0j4gxJ84Ercwn1qryQxqya5rQHyGsZ7A2Ml3R1REyRND+vAPUPSTsCa5Bml42JiMvyfZ5rXydcMrVPKfyCStoFuBy4PCIOLerN/zlpS4xdI+LxfI8XLekkSTsD2wIvAH+OiDdyCVWtBUwH0vrikql9Qg6K85U2wluINIbxD7nH+HcAEfFbSTMp2mrEgbRjFvgj9E3SUKjt80dX55EQrf5sHUjri3vz7RNyyXMX4CLg5Yi4E9iLtIr7wZK2k3QbcEVE3Oce5M4p6on/HTA9InYGTiEtUPItSZ+pZf6scg6m9gmSViXtGXRwRDyS2+zGA98AdsmfXVhYOMMl0o6R9BtJXy46tQjwBkBuD50E/BA4RGmjPKtzDqa2oPmkMaIv5/cCiIinSHs57RUR17tE2mnnRcRoSTvl9zcAg/KShYXN8J4DnomIN2uVSaucg2kvV7Sw88KSFgamkUpIn5O0cJ7NtJmkU0gr5BdKTy6RdoCyiHhJ0prAdZIOIu3i+gqpOeVESdcCk4qWMPQfrzrn3nxD0q6kufWLkeZ6b5aPZ4HXgCOAQ/OqRNZBuclkXn7dJ9KWIhsBV5IWfB4JbEja3fXNiDg9X+uREg3AwbSXk7Q58DtgN1Kn04cRsavSYsNrkjpCbo+Iu2uYzYZXCKR5Naff59M3RsS9kjYk7W//+4g4f4H7PPypQTiY9jKSVgC2j483ufs+afX7ecBRwLcj4nlJi0TEbP8yV0+e4PA30sLO80mdeUdHxKU5oN5H2rb5rtrl0jrK40x7n88B++X20POAN4GDSFX8QiDdi7TH/Q+AuTXMa8PLQfIl0qZ2W5IWhDmXVKW/FfiRpEUj4hxJ63ll/MblYNpLSBoErJMXJWkh/RK/D1xNmnP/EDA3zwM/FjgqIubUKr89QV6kZAPSmq8TgVNJqz2dAoyOiJMknQYcJWlsRDyc73NtoAE5mPYenwMukHRwRNyc2+4OIa2ZuTepHe8U0lYk/5vXKnXHRwdJuoI0xOx/gJ1ICzx/OSJGSZoDrJAvXQg4pxBIwTObGpWDaS8REbdJ+hFwSi753JRH2xyaPo79ASQtExHTHUg7TtKKpF0HDs0/w79L2h1YKV9yB6lm8CTwaESclu/zz7yBOZj2cMW/oDmAtgAn5/M35lWgfiFp+Yj4EzAjX+tf6g6KiFclbQzcLundPKNpbdLAfIBHgadIiz7fDw6kPYGDaQ9WtMrTBqRq5csR8TelhZ1PkzQ/Iv4uqRmYCg6i1RIRY/KyeXdK+i1weP5j1hQRc0nLFhYCqdtIewAH0x5I0kLkHUEl7UDqPb4V2FLS2cD1pM3wzsq/yDfWMLs9VkQ8luffj+bjURGfmsnkQNozOJj2MJLWJs1YGinpaeAA4MCIuEfSNqSN2KZHxF/z9NGZNcxujxcRT0najlRCXTkizqh1nqxreG5+DyJpXeCvwATg8Yj4DylYrpdn4NwD3AP8WGlzvOsj4n7P++5aETGG1KO/eq3zYl3HJdMeQtLiwFnAmRFxcVGAHEPaM2gT4EFSx8frFFU33U7a9SLin8A/wZ1NPZVLpj3HbOBVUnsofPyH8nrS1MWfS7qStOf91ZG2C7YacCDtmTw3v4eQtCSpd/joiLglnyusTLQSsB2pyv9ipJ1FXToyqyKXTHuIiHiLVM3/uqTB+XShl3gwaUm92yLiiXy9A6lZFTmY9iwjgSnAwbnnfn5euf004HrPtTfrOq7m9zCSliPNA/8RMJbUg3xKnu3kqr1ZF3Ew7aFyUJ1P2mrkFQdSs67lYGpmVgVuMzUzqwIHUzOzKnAwNTOrAgdTM7MqcDA1M6sCB1PrMpLmSRonaYKk6yQt2om0tpJ0c369i6SjS1y7ZN6ipb3POF7Szys9v8A1l0r6RjueNUjShPbm0eqXg6l1pdkRMTgi1gM+AA4u/lBJu/8bjIibIuLUEpcsSZq0YNZtHEytu9wPrJFLZJMknUOaobWSpGGSHpI0Npdg+wFI2kHS05IeAPYoJCRpf0l/yq+XkzRS0pP52JS0pfLquVR8Rr7uSEmPSRov6YSitI6V9IykUaR9mkqSdFBO50lJ1y9Q2t5W0v2SnpW0U76+WdIZRc/+QWd/kFafHEyty+VN/HYkraUKKWj9OSI2IG01fRywbURsSFp/9WeS+gIXADsDWwDLt5H8H4F/RMT6wIak/emPBp7LpeIjJQ0D1gQ2Ji36MkTSlyUNAfYi7W2/B7BRBV/nbxGxUX7eJODAos8GAVsCXwPOy9/hQODtiNgop3+QpFUreI41GC8ObV1pEUnj8uv7gYtIG/u9WLRP/CbAusCDeT3rhYCHgHWA/0TEZPhoH/rhrTxjG+C7ABExD3hb0lILXDMsH0/k9/1IwbU/MDIi3svPuKmC77SepJNITQn9SNs2F1yb93OaLOn5/B2GAV8sak9dIj/72QqeZQ3EwdS60uyIGFx8IgfMd4tPAXdFxN4LXDcYqNZcZ5EWezl/gWcc3oFnXArsFhFPStof2KroswXTivzswyKiOOgiaVA7n2t1ztV8q7WHgc0krQEgaVFJawFPA6tKKuybtHcb998N/DDf25y3b5lFKnUW3AEcUNQWu6KkZUm7hu4uaRFJ/UlNCuX0B6ZI6kPanLDYNyU15TyvBjyTn/3DfD2S1pK0WAXPsQbjkqnVVERMzyW8q5R2SwU4LiKelTQcuEXSDOABYL1WkvgJMELSgcA84IcR8ZCkB/PQo9tyu+nngIdyyfgd4NsRMVbSNcA44EXyPvZl/B/wSL7+KT4ZtJ8B/gEsBxwcEe9LupDUljpW6eHTgd0q++lYI/GqUWZmVeBqvplZFTiYmplVgYOpmVkVOJiamVWBg6mZWRU4mJqZVYGDqZlZFfx/iAiXpP62M5cAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "cnf_matrix_knn = confusion_matrix(y_test, y_pred_knn, labels=[0,1])\n",
    "np.set_printoptions(precision=2)\n",
    "print (classification_report(y_test, y_pred_knn))\n",
    "plt.figure()\n",
    "plot_confusion_matrix(cnf_matrix_knn, classes=['Collecting(0)','Paid Off(1)'],normalize= False,  title='Confusion matrix')\n",
    "print(\"The F1-score is %.2f and Jaccard similarity is %.2f for KNN classifier\" %(f1_score_knn,jaccard_knn))\n",
    "print(\"Test set Accuracy: \",accuracy_knn.round(2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Decision Trees"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "jaccard_dt = jaccard_similarity_score(y_test, y_pred_dt)\n",
    "f1_scores_dt = f1_score(y_pred_dt, y_test)\n",
    "accuracy_dt = metrics.accuracy_score(y_test, y_pred_dt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.75      0.21      0.33        14\n",
      "           1       0.78      0.97      0.87        40\n",
      "\n",
      "   micro avg       0.78      0.78      0.78        54\n",
      "   macro avg       0.77      0.59      0.60        54\n",
      "weighted avg       0.77      0.78      0.73        54\n",
      "\n",
      "Confusion matrix, without normalization\n",
      "[[ 3 11]\n",
      " [ 1 39]]\n",
      "The F1-score is 0.87 and Jaccard similarity is 0.78 for Decision Tree classifier\n",
      "Test set Accuracy:  0.78\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVMAAAEmCAYAAADfpHMGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3XecXFX9//HXO4UECD2UUGIQKSJfCUnoHQFBUYqiIAp8RWJUUBQUBH4KSgfBr0iRXqWJAaR3A0pLIITE0DsESGhSQkg2n98f5wwMy+7M7M7Mzszm/fRxH5m59865n1nZz55y7zmKCMzMrDp9Gh2AmVlv4GRqZlYDTqZmZjXgZGpmVgNOpmZmNeBkamZWA06m1lCS5pf0D0lvS7qiinJ2k3RzLWNrFEkbS3qs0XFY18j3mVolJH0H+AWwGvAOMBE4MiLurrLc7wH7AhtExJyqA21ykgJYOSKebHQsVluumVpZkn4B/BE4ClgaGAqcCmxfg+I/Azw+LyTSSkjq1+gYrJsiwpu3TjdgEeBdYOcS5wwgJduX8/ZHYEA+thnwIrA/8BowDfjffOxw4ENgdr7GXsBhwEVFZQ8DAuiX3+8JPE2qHT8D7Fa0/+6iz20APAC8nf/doOjYncDvgX/lcm4GBnfy3Qrx/6oo/h2ArwCPA28ABxedvw5wD/BWPvfPwHz52Lj8Xd7L3/fbReUfCLwCXFjYlz+zUr7GiPx+WWAGsFmj/9vw9snNNVMrZ31gIDC2xDmHAOsBw4E1SQnl0KLjy5CS8nKkhHmKpMUi4rek2u5lETEoIs4uFYikBYE/AdtGxEKkhDmxg/MWB67L5y4BnAhcJ2mJotO+A/wvsBQwH3BAiUsvQ/oZLAf8BjgT+C4wEtgY+I2kz+Zz24CfA4NJP7svAT8GiIhN8jlr5u97WVH5i5Nq6aOLLxwRT5ES7cWSFgDOBc6LiDtLxGsN4GRq5SwBzIjSzfDdgN9FxGsRMZ1U4/xe0fHZ+fjsiLieVCtbtZvxzAXWkDR/REyLiCkdnPNV4ImIuDAi5kTEJcCjwNeKzjk3Ih6PiJnA5aQ/BJ2ZTeofng1cSkqU/xcR7+TrTwG+CBAREyLi3nzdZ4G/AJtW8J1+GxGzcjyfEBFnAk8A9wFDSH+8rMk4mVo5rwODy/TlLQs8V/T+ubzvozLaJeP3gUFdDSQi3iM1jccA0yRdJ2m1CuIpxLRc0ftXuhDP6xHRll8Xkt2rRcdnFj4vaRVJ10p6RdJ/STXvwSXKBpgeER+UOedMYA3g5IiYVeZcawAnUyvnHuADUj9hZ14mNVELhuZ93fEesEDR+2WKD0bETRGxFamG9igpyZSLpxDTS92MqStOI8W1ckQsDBwMqMxnSt5SI2kQqR/6bOCw3I1hTcbJ1EqKiLdJ/YSnSNpB0gKS+kvaVtJx+bRLgEMlLSlpcD7/om5eciKwiaShkhYBfl04IGlpSV/PfaezSN0FbR2UcT2wiqTvSOon6dvA6sC13YypKxYC/gu8m2vNP2p3/FXgs5/6VGn/B0yIiB+Q+oJPrzpKqzknUysrIk4k3WN6KDAdeAHYB7gqn3IEMB6YBDwCPJj3dedatwCX5bIm8MkE2Id0V8DLpBHuTcmDO+3KeB3YLp/7OmkkfruImNGdmLroANLg1jukWvNl7Y4fBpwv6S1J3ypXmKTtgW1IXRuQ/n8YIWm3mkVsNeGb9s3MasA1UzOzGnAyNTOrASdTM7MacDI1M6sBT6rQw5YYPDhWGNr+FkjrKR982NGdVNYTXnnpBd568/Vy99xWrO/Cn4mY86kHxj4hZk6/KSK2qdU1S3Ey7WErDP0Mt911X6PDmGdNeem/jQ5hnjV6py1qWl7MmcmAVUvfXfbBxFPKPX1WM06mZtaaJOjTt9FRfMTJ1Mxal5pn2MfJ1Mxal2rWBVs1J1Mza1Fu5puZVU+4mW9mVj25mW9mVhNu5puZVUtu5puZVU24mW9mVj1Bn+ZJYc0TiZlZV/VxzdTMrDq+NcrMrBZ8076ZWW14AMrMrAaaqJnfPJGYmXVFYQq+UlvZIjRQ0v2SHpY0RdLhef95kp6RNDFvw8uV5ZqpmbWu6pv5s4AtIuJdSf2BuyXdkI/9MiL+VmlBTqZm1qKqfwIqIgJ4N7/tn7foTllu5ptZaxJVN/MBJPWVNBF4DbglIgrrCh0paZKkkyQNKFeOk6mZtahcMy21wWBJ44u20e1LiYi2iBgOLA+sI2kN4NfAasDawOLAgeWicTPfzFpX+T7TGRExqpKiIuItSXcC20TECXn3LEnnAgeU+7xrpmbWuqofzV9S0qL59fzAlsCjkobkfQJ2ACaXK8s1UzNrTarJFHxDgPMl9SVVLi+PiGsl3S5pSVLP7ERgTLmCnEzNrHVVeWtUREwC1upg/xZdLcvJ1MxakoA+fZqnp9LJ1Mxak/LWJJxMzaxFCXmiEzOz6rmZb2ZWA66ZmplVy32mZmbVE3Iz38ysFtzMNzOrASdTM7NqCeSlns3MqiPfZ2pmVhtOpmZm1XIz38ysNlwzNTOrUrPdZ9o8kVjT+uCDD9hq0/XZdL0RbDhqTY454vBGh9TrHfPrfdl+/VXZc7sNP9p3xw1Xs8dXN2Cz1Qbz6CMPNTC6JqIyWw9yMrWyBgwYwNjrbuGf9z7InfeM5/Zbb2L8/fc2OqxebdudduX4sy7/xL4VV1mN3598PmuuvUGDomoySs38UlvZIqSBku6X9LCkKZIOz/tXlHSfpCckXSZpvnJlOZlaWZIYNGgQALNnz2b27NlN1VfVG6259gYstMhin9g3bKVVGfrZlRsUUXPq06dPya0Cs4AtImJNYDiwjaT1gGOBkyJiZeBNYK+ysVTxPWwe0tbWxmbrj+TzKy7LZltsyci11210SGZVN/MjeTe/7Z+3ALYA/pb3n09aVK+kuiZTSctIulTSU5L+I+l6Sat0cu4wSZPz680kXdvNa+4naYGi99cXVh/sZnlrSTorv5akP0l6UtIkSSPy/iUl3djda7SCvn37cuc9E5j02LM8OP4Bpk4pu1ijWd1V28zPZfSVNBF4DbgFeAp4KyLm5FNeBJYrV07dkmleInUscGdErBQRqwMHA0vX65rZfsBHyTQivhIRb1VR3sHAyfn1tsDKeRsNnJavMR2YJmnDDkvoRRZZdFE23HhTbrv15kaHYvM4SZU08wdLGl+0jW5fTkS0RcRwYHlgHeDzHVwuysVTz5rp5sDsiDj9o2giJgJ3Szpe0mRJj0j6dqlCJC0o6RxJD0h6SNL2eX9fSSfkMiZJ2lfST4FlgTsk3ZHPe1bS4FzznSrpzNzRfHNeJxtJa+cy7inElvcvBHwxIh7O4WwPXJCbBvcCixbW1wauAnar1Q+vmcyYPp2330p/j2bOnMm4O25j5VVWbXBUZhXVTGdExKii7YzOysqVrjuB9Ui/24VbR5cHXi4XSz2T6RrAhA7270Tq6F0T2BI4vighdeQQ4PaIWJuUoI+XtCCpZrgisFZEfBG4OCL+RPrSm0fE5h2UtTJwSkR8AXgL+Ebefy4wJiLWB9qKzh8FFLdnlwNeKHpfXP0fD2zc0ReQNLrwl/H1GTNKfNXm9Oqr09jhK1uyybprsdUm67PpFlvy5W2/2uiwerXDf7E3P95lG55/5km+uckaXHfFRYy75Vq+uckaTHnoAQ764a4csNc3Gx1m41XZZ5q76BbNr+cn5aSpwB1A4Qe8B3B1ubIacdP+RsAlEdEGvCrpn8DawKROzt8a+LqkA/L7gcBQ0pc+vdCvERFvVHDtZ3LtGFKiH5Z/kAtFxL/z/r8C2+XXQ4DpRZ/v6P+eQvX/NVKt+NMnpL+GZwAMHzGybHOh2XxhjS9yx7/HNzqMecpvTzyzw/2bbLVdh/vnSarJGlBDgPMl9SVVLi+PiGsl/Qe4VNIRwEPA2eUKqmcyncLHmb1YV++pEfCNiHjsEztTHb6riWlW0es2YP4y8cwkJe+CF4EVit4XV/8H5vPNrAcIqPYOvYiYBKzVwf6nSf2nFatnM/92YICkvQs7JK1Numfr27nPc0lgE+D+EuXcBOybkyeSCl/8ZmBMoV9D0uJ5/zvAQpUGGRFvAu/ke8sAdik6PBX4XNH7a4Dd86j+esDbETEtH1uFT3YJmFldle4v7el7oeuWTCMigB2BrfKtUVOAw0jN6EnAw6SE+6uIeKVEUb8n3fs1KQ8M/T7vPwt4Pu9/GPhO3n8GcENhAKpCewFnSLqH9Afv7fwdHgUWyQNRANcDTwNPAmcCPy4qY3Pgui5c08yq1KePSm49SSnnzdskDSrcuCvpIGBIRPwsv/858E5EnFWmjHHA9rmm26nhI0bGbXfdV6PIraumvPTfRocwzxq90xY8OnlizTLcwCGrxLA9Ti55zmPHbjMhIkbV6pql+Amo5KuSJuaa78bAEUXHTuOTfa2fkrsrTiyXSM2sdkRz1Uw9BR8QEZcBl3Vy7APgwjKfn066z9TMelBPJ8xSnEzNrDWp+tH8WnIyNbOWlG6Nap5s6mRqZi2q5/tFS3EyNbOW5ZqpmVm13GdqZla9wq1RzcLJ1Mxalpv5ZmY10ES51MnUzFqT5Ga+mVkN9PzMUKU4mZpZy2qiXOpkamYtqsma+Z41ysxaUuFx0momh5a0gqQ78mKbUyQVpt48TNJLeTa5iZK+Uq4s10zNrGXVoM90DrB/RDyYJ4GfIOmWfOykiDih0oKcTM2sZVXbzM/LDk3Lr9+RNJWPVxzuWixVRWJm1ij5cdJSGzC4sMx63kZ3Wpw0jLS4XmEpjH0kTZJ0jqTFyoXjZGpmLUmVLag3IyJGFW1ndFiWNAi4EtgvIv5LWmFjJWA4qeb6h3LxdNrMl7RwqQ/mC5qZNUzfGozmS+pPSqQXR8TfASLi1aLjZwLXliunVJ/pFNK69MXRFt4HMLTrYZuZ1U614095CfmzgakRcWLR/iFFy7jvSAXLuHeaTCNiherCNDOrn9QvWnXNdEPge8AjkibmfQcDu0oaTqo4Pgv8sFxBFY3mS9oF+GxEHCVpeWDpiJjQncjNzGql2mZ+RNzNJ1vfBdd3tayyA1CS/gxsTsreAO8Dp3f1QmZmtVbBaH6PqaRmukFEjJD0EEBEvCFpvjrHZWZWkkgj+s2ikmQ6W1IfUt8BkpYA5tY1KjOzcqSajObXSiX3mZ5Cum1gSUmHA3cDx9Y1KjOzCrRUMz8iLpA0Adgy79o5IsreJmBmVk8C+jTRHHyVPpvfF5hNaur7qSkzawotNQWfpEOAS4BlgeWBv0r6db0DMzMrpVwTv+ma+cB3gZER8T6ApCOBCcDR9QzMzKycVmvmP9fuvH7A0/UJx8ysci2RTCWdROojfR+YIumm/H5r0oi+mVnDpAGoRkfxsVI108KI/RTguqL999YvHDOzClW4NElPKTXRydk9GYiZWVc102h+2T5TSSsBRwKrAwML+yNilTrGZWZWUrM18yu5Z/Q84FxS7NsClwOX1jEmM7OKVLs6aS1VkkwXiIibACLiqYg4lDSLlJlZw0jQVyq59aRKbo2alWejfkrSGOAlYKn6hmVmVl4TjT9VVDP9OTAI+ClpVuq9ge/XMygzs0pU28yXtIKkOyRNlTRF0s/y/sUl3SLpifxv2dVJK5nopLDs6Tt8PEG0mVlDiZpMwTcH2D8iHpS0EDBB0i3AnsBtEXGMpIOAg4ADSxVU6qb9seQ5TDsSETt1J3Izs5qowfP3edG8afn1O5KmAssB2wOb5dPOB+6ku8kU+HN1YVpH+kosOKDSybqs1r787d80OoR51qynX655mRU05QdLGl/0/oyIOKOTsoYBawH3kda5KyTZaZLKjhOVumn/tnIfNjNrFEElI/YzImJU2bKkQaRJ8PeLiP9257Yqz01qZi2rj0pvlZDUn5RIL46Iv+fdr0oako8PAV4rG0v3voKZWeNVm0zzbZ9nA1Mj4sSiQ9cAe+TXewBXlyur4s47SQMiYlal55uZ1ZNELUbzNyTdpfSIpIl538HAMcDlkvYCngd2LldQJc/mr0PK3IsAQyWtCfwgIvbtZvBmZjVRg9H8u6HT9aK/1JWyKmnm/wnYDng9X/xh/DipmTVYYUG9UltPqqSZ3ycinms3utVWp3jMzCrWt4keJ60kmb6Qm/ohqS+wL/B4fcMyMytNDah9llJJMv0Rqak/FHgVuDXvMzNrqL5NdD9SJc/mvwbs0gOxmJlVrNBn2iwqGc0/kw6e0Y+I0XWJyMysQk2USytq5t9a9HogsCPwQn3CMTOrkCp6nLTHVNLMv6z4vaQLgVvqFpGZWQWabQ2o7kxftCLwmVoHYmbWVS2VTCW9ycd9pn2AN0gTpZqZNYyoyeOkNVMymeZJANYkrfsEMDciOp0w2sysx9RgcuhaKnmXVk6cYyOiLW9OpGbWNJrpcdJKbnm9X9KIukdiZtYFqZlfeutJpdaA6hcRc4CNgL0lPQW8R/oOERFOsGbWQKJPpxM+9bxSfab3AyOAHXooFjOzionm6jMtlUwFEBFP9VAsZmaVE/RrkdH8JSX9orOD7ab4NzPrUbWomUo6hzRf82sRsUbedxiwNzA9n3ZwRFxfrqxSybQvMIjOZ6E2M2uoGozYn0da1v6CdvtPiogTulJQqWQ6LSJ+18XAzMx6RFrquboyImKcpGE1CKfkrVGukZpZ81KaILrUBgyWNL5oq3S2u30kTZJ0jqTFKvlAqWTapcWkzMx6mspswIyIGFW0nVFBsacBKwHDgWnAHyqJpdNmfkS8UUkBZmaNkJr5tW9AR8SrH10jzed8bSWfa6JJ/83MukYqvXWvTA0persjMLmSz3VnCj4zsybwUb9o90uQLgE2I/Wtvgj8FthM0nDSbHnPAj+spCwnUzNrSbVo5kfErh3sPrs7ZTmZmlnLaqZbjpxMzaw15VujmoWTqZm1pHqN5neXk6mZtazmSaVOpmbWwpqoYupkamatyc18M7OaEGqihr6TqZm1rCaqmDqZmllrktzMNzOriSbKpU6mVpkf/uD73HD9tSy51FJMmFjRvA9WhQHz9ePWs/djvvn60a9vX8be+hBHnH49m669Ckf/fEfm69+Xh6a+wJjDL6atbW6jw22YZuoz9axRVpHv7bEnV197Y6PDmGfM+nAO24z+E+t++xjW3eVott5gddZbc0XO+t332P2gcxm181E8P+0Nvvu1dRsdasMURvNLbT3JydQqstHGm7D44os3Oox5ynszPwSgf7++9OvXl7a2ucz6cA5PPv8aALff+yg7fGl4I0NsuHpMwdddTqZmTapPH3HvpQfx/G3HcPu9j/LA5Ofo378vI1YfCsCOWw5n+aUrWlGj11KZ//WkhidTSW2SJkqaLOkKSQuUOf96SYt2sP8wSQd08pnRkh7N2/2SNio6trGkKTmG+SUdn98fn4/vJ2n3/HrnfGyupFFFZfyPpPO6+SMw69DcucF6uxzD5758KKPW+AyrrzSE3Q86l+P234m7LjyAd96bxZy2tkaH2TCidBO/p5v5zTAANTMihgNIuhgYA5zY2ckR8ZWuFC5pO9LkrhtFxAxJI4CrJK0TEa8AuwEnRMS5+fwfAktGxCxJ/YDvAyNycZOBnYC/tIvpEUnLSxoaEc93JT6zct5+dybjxj/B1huszh8vvI0t9/ojAF9abzVW/sxSDY6ugRrQlC+l4TXTdu4CPgcg6SpJE3JN8KMVBSU9K2lwfn2IpMck3Qqs2kmZBwK/jIgZABHxIHA+8BNJPwC+BfxG0sWSrgEWBO6T9G1gC+DBiJiTPzs1Ih7r5Dr/AHap7uubJYMXG8Qig+YHYOCA/myx7qo89uyrLLnYIADm69+P/ffcijP/dncjw2y4ChbUK/35tProa5ImF+1bXNItkp7I/1a9OmmPyrXAbYFH8q7vR8RIYBTwU0lLtDt/JCl5rUWqLa7dSdFfACa02zce+EJEnAVcQ0q2u0XE18k15Yi4DNiwg892ZjywcSffbXRhqdnpM6ZXWFxz2f27u7LZxuvz+GOPsdKw5TnvnG5NRm4VWmbwwtx45k+5/7Jfc/dFv+S2+x7lhrsm8/M9tuShKw/lgct/zfXjHuGfDzze6FAbpkaj+ecB27TbdxBwW0SsDNyW35fVDM38+SVNzK/v4uMlA34qacf8egVgZeD1os9tDIyNiPcBcq2yUiKt71LOEGBqhWW+Bizb0YG8vOwZACNHjqrkuk3ngosuaXQI85TJT7zM+rse+6n9B//xKg7+41UNiKhJVdnMj4hxkoa12709aV0oSK3YO0kt3JKaIZl+1GdaIGkzYEtg/Yh4X9KdwMAOPltJYvoPMBK4vWjfiLy/bGydXLcjA/P5ZtZDKhixHyxpfNH7M3LlppSlI2IaQERMk1RRx3TTNPPbWQR4MyfS1YD1OjhnHLBjHoFfCPhaJ2UdBxxb6CbIqw7uCZxaQRxTyX24FViFCpeENbPa6KPSGzAjIkYVbeUSabc1Q820IzcCYyRNAh4D7m1/QkQ8KOkyYCLwHKmL4FMi4hpJywH/lhTAO8B3C395yrgBuLDwJnc7nAwsCVwnaWJEfDkf3hy4rtIvaGY1UJ/R/FclDcm10iGkLryyGp5MI2JQB/tmkQajOjp/WNHrI4EjK7jGacBpnRzbs7N4IuI5Sa9LWjkinoiIscDY9mVIGkAaKNuvXCxmVhtpxL4u2fQaYA/gmPzv1ZV8qFmb+c3kINJAVClDgYMKt1CZWQ8o08TvU0GelXQJcA+wqqQXJe1FSqJbSXoC2Cq/L6vhNdNml+8r7eze0sI5TwBP9ExEZvaR6kfzd+3k0Je6WpaTqZm1KC9bYmZWNVFZU76nOJmaWetyMjUzq56b+WZmNeBmvplZtSqdGqqHOJmaWctyM9/MrEoezTczqxUnUzOz6rmZb2ZWA27mm5nVgpOpmVl16jgFX7c4mZpZa6pwmr2e4mRqZq3LydTMrFqiT2XLOfcIJ1Mza0m1eppU0rOkteHagDkRMao75TiZmlnrql3FdPOImFFNAU6mZtaymqmZ7wX1zKxlqcxWoQBuljRB0ujuxuKaqZm1JkEFFdPBksYXvT8jIs5od86GEfGypKWAWyQ9GhHjuhqOk6mZtSQBKp9NZ5QbUIqIl/O/r0kaC6wDdDmZuplvZi2r2ma+pAUlLVR4DWwNTO5OLK6ZmlnLqsH409LA2FzD7Qf8NSJu7E5BTqZm1rIqaOaXFBFPA2vWIhYnUzNrWc1zY5STqZm1KFU2mt9jnEzNrGVV28yvJSdTM2tZzZNKnUzNrIU1UcXUydTMWpOabAo+37RvZlYDrpmaWctqooqpk6mZtSg11xR8TqZm1pJqNdN+rTiZmlnraqJs6mRqZi3LzXwzsxponlTqZGpmrayJsqmTqZm1JNFczXxFRKNjmKdImg481+g4qjAYqGpJXOu2Vv/ZfyYilqxVYZJuJP1MSpkREdvU6pol43Eyta6QNL7cmjpWH/7ZNzc/TmpmVgNOpmZmNeBkal3Vfs1x6zn+2Tcx95mamdWAa6ZmZjXgZGpmVgNOpmbzCEn+fa8j/3DNejlJu0haMSLmNjqW3szJ1HqMpKUkHSVpGUmL5H3N8zxgL5R/zssDp0raV1KPPA00L/JovvUYSQOBo4A5wILA+RFxv6Q+rjXVl6RRwBrAzsB1EXFqg0PqdZxMre4kDQDmRERbfr8ysClwKPCDiLhVksL/MdaMpP8B/lP4med98wFrA8cBZ0XEuY2KrzdyM9/qKv9S3wicJek4gIh4IiLOAg4ELpC0vhNp7Ui6ChgHfEnSRzPDRcSHEfEv4HBgI0mrNyrG3sjJ1OpG0uLAqcAFwBHAOpJOlrRiroleBhwC7CNpmUbG2ltI2gh4EzgS2I92CTWbCLwErNDD4fVqTqZWT/2BD4A7I+IpYFtgEPDzfAzgZuApYKGGRNiLSFqMlCiPjogTgL8DvyQl1IGF8yLiNeARYH9JgxoSbC/kZGo1J2lZSUMi4lXgPmBDSQtFxEzgJ8DngWMAIuIloA3434YF3AtI2grYOiLeBZ4HyF0plwC/Akbm836cj10BHA/MbkjAvZCTqdXDtsCX8uupwPbASEmDIuJ9YA9gRUkLA0TE4cC1kvp3WJqVlGud/YHRkvpFxAeFpn1EnA2cCfxW0iRg/aKPjouIWT0fce/kZGr18D6wAUBEXAz8C9gX2EbScqT7HpcH+hQ9lXNvRLiW1EWS/gJ8IyKuB+4HfiSpb0TMkdQXICIuBf4HmBgR3yt81om0tnxrlNWFpFuBu3KtE0l7kO5zHAksSurXu6KBIba8fAP+zyJiW0nrAkOB9YCrgbsjYm6uoW4O7BARP8mf8329deBkajUlaYWIeEHSWsDPgH8W7meUtBApkQ6IiCd9b2n3SdqU1IXyBGkA72TgYuA3wALAbcDNETE791e/kz/nRFonbuZb1QqPhEpaDbhN0nci4iHgJmBTSQcCRMQ7EfFCRDyZ3zuRdoOk84Cf51H5fwOrAa9GxIekQaVXSLXRkyUtBbxb+KwTaf24Zmo1IekrwI7A0sBngSMj4hJJGwL/jzTCfD4wOSLeblykrU3SgsDvSDfljwKeIdVM/wGMiYi/5gGpZYDvA8uRaqx352RrdeJkalWTtDRwB+mXdyqp3+444Kh8Yz6SDgKmA28AV7lW2n2SDiHdq3tLROya932VlDRHR8TlRecOBV5xIq2/9k9GmFWsqM+zjVQ7eiAi2vLg01XA7/M5l0bEMfkz/ZxIu07SEGDhiHgMeJJUM31N0sbAgxFxnaRdgeskPZP3tUXE8w0Me57iPlPrsqJp85YAiIgZpNuhrsjv24CHgRuAXSUNK/pMG9Ylks4BTgNuknQSsEpE7AQ8B+xOekx3YETcAAyPiAeKJzixnuFkal1SqI3mPtJbJZ2SX+8FzJJ0p6QxpGfDryY17WcXaqOulXZNTqSLRcQOwGakAafVJf0SOAl4mjSt3uaS+kfEpPw5/273MP/ArWJFiXQk8B3gYNIAyJdJg0/fAcaS5ir9Fum5/OGAE2g3SBoMLEb6WRIRz5L+QP0F+CKwFnAs6ee8VPFDDx6173nuM7WyJK0E9I2IxyUtCVxEuiH/eknjgJ0QzMuoAAAJcElEQVRI82T2B07JT9+sR5oxaveIeLlhwbe294CFgVGS7iNV7D/Mr3cDtoqIByXt7xp/47lmapVYH1g898tNJzUvvyFpmzyxxhXAJNLTTUPyZ/4LfLXQ7LRumQu8SOoHnQuQm/IzSbNtDYKPu068BExjuWZqZUXERfnppSmSdomIMyTNBI6TRETcKOli4IY8CxQR8Z+GBt0LRMSs/Oz9PyR9mCctKTTlvwfc2+58104byPeZWofyPJer5GbkZsBDwGjSL/EeEfFQvhXnKOAneaINq4KkrUlLjbyYa5l98q1mmwLnkO6OCGBF4I2I2L2B4Vo7rplaZwYCR0h6FVgH2DMijpc0F/hrrqFekifSeKehkfYCeS6DXYFJki6NiGmS5uYZoP4paVvgc6Sny8ZHxPn5c37Wvkm4ZmqfUvgFlfR14ELgwojYp2g0/wDSkhjbR8SE/BlPWlIlSV8DtgSeBS6IiNdzDVUdJUwn0ubimql9Qk6Kc5UWwpuPdA/jH/OI8R8AIuIESW9QtNSIE2n3tPsjtDPpVqgv50OX5jshOvzZOpE2F4/m2yfkmufXgbOBFyLiZmAX0izuYyRtJekG4KKIuNMjyNUpGon/AzA9Ir4GHE2aoORbkpZoZHxWOSdT+wRJK5LWDBoTEfflPrtJwDeBr+djZxUmznCNtHskHSVpk6Jd8wOvA+T+0KnAj4CfKC2UZ03OydTam0u6R/SF/F4AEfEIaS2nXSLiStdIq3Z6RIyTtF1+fxUwLE9ZWFgM7yngsYh4s1FBWuWcTOdxRRM7D5A0AHiNVEP6vKQB+WmmDSUdTZohv1B7co20G5RFxPOSVgaukLQ3aRXXF0ndKb+XdDkwtWgKQ//xanIezTckbU96tn5B0rPeG+btceBlYH9gnzwrkXVT7jJpy6/7R1pSZG3gr6QJn8cCI0iru74ZEcflc32nRAtwMp3HSdoI+AOwA2nQaXZEbK802fDKpIGQGyPitgaG2fIKiTTP5nRi3n11RNwhaQRpffsTI+Iv7T7n259ahJPpPEbSssCX4+NF7n5Amv2+DTgQ+G5EPC1p/oiY6V/m2skPOPydNLHzXNJg3kERcV5OqHeSlm2+pXFRWnf5PtN5z+eBPXJ/6OnAm8DepCZ+IZHuQlrj/ofAnAbG2vJyknyetKjdpqQJYU4jNemvB34saYGIOFXSGp4Zv3U5mc4jJA0DVsuTkvQj/RJ/AFxKeub+HmBOfg78EODAiJjVqHh7gzxJyVqkOV+nAMeQZns6GhgXEUdIOhY4UNKDEXFv/pxbAy3IyXTe8XngTEljIuLa3Hf3E9KcmbuS+vGOJi1F8us8V6kHPrpJ0kWkW8zWBbYjTfC8SUTcKmkWsGw+dT7g1EIiBT/Z1KqcTOcREXGDpB8DR+eazzX5bpt90uHYE0DSkhEx3Ym0+yQtR1p1YJ/8M/yHpB2BFfIpN5FaBg8D90fEsflz/pm3MCfTXq74FzQn0H7AkXn/1XkWqF9JWiYi/gzMyOf6l7qbIuIlSesAN0p6Lz/RtCrpxnyA+4FHSJM+3wVOpL2Bk2kvVjTL01qkZuULEfF3pYmdj5U0NyL+Iakv8Co4idZKRIzP0+bdLOkEYL/8x6xPRMwhTVtYSKTuI+0FnEx7IUnzkVcElbQNafT4emBTSacAV5IWwzs5/yJf3cBwe62IeCA/fz+Oj++K+NSTTE6kvYOTaS8jaVXSE0tjJT0KfB/YKyJul7QFaSG26RHxt/z46BsNDLfXi4hHJG1FqqEOjYjjGx2T1Yefze9FJK0O/A2YDEyIiGdIyXKN/ATO7cDtwE+VFse7MiLu8nPf9RUR40kj+is1OharH9dMewlJCwMnAydFxDlFCXI8ac2g9YB/kQY+XqGouel+0vqLiH8D/wYPNvVWrpn2HjOBl0j9ofDxH8orSY8uHiDpr6Q17y+NtFywNYATae/kZ/N7CUmLkkaHD4qI6/K+wsxEKwBbkZr8z0VaWdS1I7Macs20l4iIt0jN/G9IGp53F0aJh5Om1LshIh7K5zuRmtWQk2nvMhaYBozJI/dz88ztxwJX+ll7s/pxM7+XkbQ06TnwHwMPkkaQj85PO7lpb1YnTqa9VE6qc0lLjbzoRGpWX06mZmY14D5TM7MacDI1M6sBJ1MzsxpwMjUzqwEnUzOzGnAytbqR1CZpoqTJkq6QtEAVZW0m6dr8+uuSDipx7qJ5iZauXuMwSQdUur/dOedJ+mYXrjVM0uSuxmjNy8nU6mlmRAyPiDWAD4ExxQeVdPm/wYi4JiKOKXHKoqSHFsx6jJOp9ZS7gM/lGtlUSaeSntBaQdLWku6R9GCuwQ4CkLSNpEcl3Q3sVChI0p6S/pxfLy1prKSH87YBaUnllXKt+Ph83i8lPSBpkqTDi8o6RNJjkm4lrdNUkqS9czkPS7qyXW17S0l3SXpc0nb5/L6Sji+69g+r/UFac3IytbrLi/htS5pLFVLSuiAi1iItNX0osGVEjCDNv/oLSQOBM4GvARsDy3RS/J+Af0bEmsAI0vr0BwFP5VrxLyVtDawMrEOa9GWkpE0kjQR2Ia1tvxOwdgVf5+8RsXa+3lRgr6Jjw4BNga8Cp+fvsBfwdkSsncvfW9KKFVzHWownh7Z6ml/SxPz6LuBs0sJ+zxWtE78esDrwrzyf9XzAPcBqwDMR8QR8tA796A6usQWwO0BEtAFvS1qs3Tlb5+2h/H4QKbkuBIyNiPfzNa6p4DutIekIUlfCINKyzQWX5/WcnpD0dP4OWwNfLOpPXSRf+/EKrmUtxMnU6mlmRAwv3pET5nvFu4BbImLXducNB2r1rLNIk738pd019uvGNc4DdoiIhyXtCWxWdKx9WZGvvW9EFCddJA3r4nWtybmZb412L7ChpM8BSFpA0irAo8CKkgrrJu3ayedvA36UP9s3L9/yDqnWWXAT8P2ivtjlJC1FWjV0R0nzS1qI1KVQzkLANEn9SYsTFttZUp8c82eBx/K1f5TPR9Iqkhas4DrWYlwztYaKiOm5hneJ0mqpAIdGxOOSRgPXSZoB3A2s0UERPwPOkLQX0Ab8KCLukfSvfOvRDbnf9PPAPblm/C7w3Yh4UNJlwETgOfI69mX8P+C+fP4jfDJpPwb8E1gaGBMRH0g6i9SX+qDSxacDO1T207FW4lmjzMxqwM18M7MacDI1M6sBJ1MzsxpwMjUzqwEnUzOzGnAyNTOrASdTM7Ma+P+LY5f0ge4IPQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "cnf_matrix_dt = confusion_matrix(y_test, y_pred_dt, labels=[0,1])\n",
    "np.set_printoptions(precision=2)\n",
    "print (classification_report(y_test, y_pred_dt))\n",
    "plt.figure()\n",
    "plot_confusion_matrix(cnf_matrix_dt, classes=['Collecting(0)','Paid Off(1)'],normalize= False,  title='Confusion matrix')\n",
    "print(\"The F1-score is %.2f and Jaccard similarity is %.2f for Decision Tree classifier\" %(f1_scores_dt,jaccard_dt))\n",
    "print(\"Test set Accuracy: \",accuracy_dt.round(2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Support Vector Machines"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/metrics/classification.py:1143: UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no predicted samples.\n",
      "  'precision', 'predicted', average, warn_for)\n"
     ]
    }
   ],
   "source": [
    "f1_svm = f1_score(y_test, y_pred_svm, average='weighted') \n",
    "jaccard_svm = jaccard_similarity_score(y_test, y_pred_svm)\n",
    "accuracy_svm = metrics.accuracy_score(y_test, y_pred_svm)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The F1-score is 0.63 and Jaccard similarity is 0.74 for SVM classifier \n",
      "Test set Accuracy:  0.74\n"
     ]
    }
   ],
   "source": [
    "print('The F1-score is %.2f and Jaccard similarity is %.2f for SVM classifier '%(f1_svm, jaccard_svm))\n",
    "print(\"Test set Accuracy: \", (accuracy_svm.round(2)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "yhat_prob = LR.predict_proba(X_test)\n",
    "jaccard_lr = jaccard_similarity_score(y_test, y_pred_lr)\n",
    "log_loss_lr = log_loss(y_test, yhat_prob)\n",
    "f1_score_lr = f1_score(y_pred_lr, y_test)\n",
    "accuracy_lr = metrics.accuracy_score(y_test, y_pred_lr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The log-loss is 0.557, Jaccard similarity is 0.74 and F1-score is 0.851 for Logistic Regression classifier \n",
      "Test set Accuracy:  0.741\n"
     ]
    }
   ],
   "source": [
    "print('The log-loss is %.3f, Jaccard similarity is %.2f and F1-score is %.3f for Logistic Regression classifier ' %( log_loss_lr, jaccard_lr, f1_score_lr))\n",
    "print(\"Test set Accuracy: \", accuracy_lr.round(3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Report\n",
    "The final report with all important evaluation metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| Algorithm          | Jaccard    | F1-score    | LogLoss  |  Accuracy  |\n",
    "|--------------------|------------|-------------|----------|------------|\n",
    "| KNN                | 0.74       | 0.83        |  NA      |  0.74      |\n",
    "| Decision Tree      | 0.87       | 0.78        |  NA      |  0.78      |\n",
    "| SVM                | 0.74       | 0.63        |  NA      |  0.740     |\n",
    "| LogisticRegression | 0.74       | 0.85        | 0.55     |  0.741     |"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.6",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}