metal_icon_prediction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prediction of the metal ion content from multi-parameter data\n",
    "The Water_data.csv dataset(found in dataset folder) is used, which is a multi-parameter dataset consisting of 201 samples obtained from 67 mixtures of Cadmium, Lead, and tap water.  Three features (attributes) were measured for each sample (Mod1, Mod2, Mod3). Use K-Nearest Neighbor Regression to predict total metal concentration (c_total), concentration of Cadmium (Cd) and concentration of Lead (Pb), for each sample using number of neighbors k = 1, 2, 3, 4 and 5."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 150,
   "metadata": {},
   "outputs": [],
   "source": [
    "#import necessary libraries\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import operator\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 151,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>c_total</th>\n",
       "      <th>Cd</th>\n",
       "      <th>Pb</th>\n",
       "      <th>Mod1</th>\n",
       "      <th>Mod2</th>\n",
       "      <th>Mod3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>196</th>\n",
       "      <td>2.918376</td>\n",
       "      <td>3.941121</td>\n",
       "      <td>0.654578</td>\n",
       "      <td>-0.843990</td>\n",
       "      <td>-0.534095</td>\n",
       "      <td>-1.420835</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>197</th>\n",
       "      <td>2.918376</td>\n",
       "      <td>3.941121</td>\n",
       "      <td>0.654578</td>\n",
       "      <td>-0.845365</td>\n",
       "      <td>-0.555321</td>\n",
       "      <td>-1.422472</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>198</th>\n",
       "      <td>2.918376</td>\n",
       "      <td>5.036636</td>\n",
       "      <td>-0.440936</td>\n",
       "      <td>-0.841559</td>\n",
       "      <td>-0.289483</td>\n",
       "      <td>0.567578</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>199</th>\n",
       "      <td>2.918376</td>\n",
       "      <td>5.036636</td>\n",
       "      <td>-0.440936</td>\n",
       "      <td>-0.841333</td>\n",
       "      <td>-0.294660</td>\n",
       "      <td>0.841408</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>200</th>\n",
       "      <td>2.918376</td>\n",
       "      <td>5.036636</td>\n",
       "      <td>-0.440936</td>\n",
       "      <td>-0.834399</td>\n",
       "      <td>-0.314160</td>\n",
       "      <td>0.777797</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      c_total        Cd        Pb      Mod1      Mod2      Mod3\n",
       "196  2.918376  3.941121  0.654578 -0.843990 -0.534095 -1.420835\n",
       "197  2.918376  3.941121  0.654578 -0.845365 -0.555321 -1.422472\n",
       "198  2.918376  5.036636 -0.440936 -0.841559 -0.289483  0.567578\n",
       "199  2.918376  5.036636 -0.440936 -0.841333 -0.294660  0.841408\n",
       "200  2.918376  5.036636 -0.440936 -0.834399 -0.314160  0.777797"
      ]
     },
     "execution_count": 151,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# load dataset and normalize data with z-score\n",
    "from scipy.stats import zscore\n",
    "df = pd.read_csv('Water_data.csv')\n",
    "\n",
    "# apply zscore to every every columns\n",
    "df[df.columns] = df[df.columns].apply(zscore) \n",
    "df.tail()\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 152,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.56000976, -0.44093638, -0.44093638, -0.97172094, -0.66981829,\n",
       "        -0.3350763 ],\n",
       "       [-0.56000976, -0.44093638, -0.44093638, -0.96309341, -0.66999086,\n",
       "        -0.16082967],\n",
       "       [-0.56000976, -0.44093638, -0.44093638, -0.96282669, -0.66973201,\n",
       "         0.09487507],\n",
       "       ...,\n",
       "       [ 2.9183762 ,  5.03663569, -0.44093638, -0.8415591 , -0.28948263,\n",
       "         0.56757761],\n",
       "       [ 2.9183762 ,  5.03663569, -0.44093638, -0.84133341, -0.29465961,\n",
       "         0.84140779],\n",
       "       [ 2.9183762 ,  5.03663569, -0.44093638, -0.83439856, -0.31415958,\n",
       "         0.77779679]])"
      ]
     },
     "execution_count": 152,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#convert pandas dataframe into numpy array\n",
    "data = df.values\n",
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "==== Knn Regression  implementation ====="
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 153,
   "metadata": {},
   "outputs": [],
   "source": [
    "#calculates eculidan distance between two data instances\n",
    "def eculidan_dist(data1, data2):\n",
    "    \n",
    "    distance = 0\n",
    "    length = len(data2)\n",
    "        \n",
    "    # loop through all  input variables (except target variables)\n",
    "    for x in range(3, length):\n",
    "        distance += np.power((data1[x]- data2[x]),2)\n",
    "    return np.sqrt(distance)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "metadata": {},
   "outputs": [],
   "source": [
    "# finds k nearest  neighbors of  test instance \n",
    "\n",
    "def get_neighbours(trainingSet, testInstance, k):\n",
    "        distance = []\n",
    "        \n",
    "        # find distance  between  a test instance to all training instances\n",
    "        for x in range(len(trainingSet)):\n",
    "            dist = eculidan_dist(trainingSet[x], testInstance) # call eculidan_dist function\n",
    "            distance.append((trainingSet[x], dist)) \n",
    "        \n",
    "        # sort the distance in ascending order\n",
    "        distance.sort(key=operator.itemgetter(1))\n",
    "        \n",
    "    \n",
    "        #store the  k neighbours\n",
    "        neighbours = []\n",
    "        for x in  range(k):\n",
    "            # takes only neighbours from distance and append \n",
    "            neighbours.append(distance[x][0]) \n",
    "        return neighbours\n",
    "    \n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {},
   "outputs": [],
   "source": [
    "# finds average of target variables of k neighbours\n",
    "def get_predic_val(neighbors):\n",
    "        total_neig = len(neighbors)\n",
    "        predict_array = np.array([0.0,0.0,0.0]) # initilize predict values for c_total, Cd and Pb \n",
    "        \n",
    "        #iterates over K neighbours\n",
    "        for x in range(total_neig):\n",
    "            predict_array += neighbors[x][0:3] # take c_total, Cd and Pb\n",
    "            \n",
    "        return  predict_array/total_neig # take average \n",
    "    \n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " ==== cross validation ====="
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "metadata": {},
   "outputs": [],
   "source": [
    "''' this function divides the data into test and train set based on the given 'leave' number. For example, if 'leave' \n",
    "    is given as 1, then test set will 1 data instance and rest will be train dataset and keep doing until all instance\n",
    "    become testset. Thus, there is n-tests and n-train set in n-samples.  \n",
    "'''\n",
    "\n",
    "# cross validation for leave-one-out method\n",
    "def cross_validation( dataset, leav_out, k):\n",
    "    \n",
    "    '''dataset = data used for cv, \n",
    "       leav_out = no. of instance for test set, \n",
    "       k = parameter for knn'''\n",
    "    \n",
    "    predictions = [] # store prediction\n",
    "    \n",
    "    i = 0 \n",
    "    while i <= len(dataset): #iterates as the no. of samples\n",
    "        \n",
    "        # separate test and training dataset\n",
    "        test = dataset[i: i+leav_out] # test set\n",
    "        train = np.concatenate([dataset[0:i], dataset[i+leav_out:]]) # train set\n",
    "        \n",
    "        # each element in test data\n",
    "        for x in range(len(test)):\n",
    "            neighbor = get_neighbours(train, test[x], k) # call get_neighbours function\n",
    "            predic_val = get_predic_val(neighbor) # get predicted value\n",
    "            predictions.append(predic_val)\n",
    "        \n",
    "        i = i + leav_out # increament i values \n",
    "        \n",
    "    return np.array(predictions)\n",
    "        \n",
    "\n",
    "    \n",
    "   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.5405308 , -0.43582398, -0.41537437],\n",
       "       [-0.53357402, -0.43889142, -0.40135179],\n",
       "       [-0.54887892, -0.43450936, -0.42983516],\n",
       "       [-0.54609621, -0.42925089, -0.43071157],\n",
       "       [-0.54748757, -0.44093638, -0.42121712],\n",
       "       [-0.54609621, -0.43655432, -0.42340815],\n",
       "       [-0.55351677, -0.43786894, -0.43377901],\n",
       "       [-0.55212541, -0.43553184, -0.43392508],\n",
       "       [-0.54400918, -0.43392508, -0.42275084],\n",
       "       [-0.53682052, -0.44093638, -0.40441923],\n",
       "       [-0.55351677, -0.43889142, -0.43275653],\n",
       "       [-0.53913944, -0.43838018, -0.41062714],\n",
       "       [-0.53913944, -0.42231263, -0.42669469],\n",
       "       [-0.53218267, -0.43838018, -0.399672  ],\n",
       "       [-0.53913944, -0.42085194, -0.42815537],\n",
       "       [-0.53913944, -0.42596435, -0.42304297],\n",
       "       [-0.53102321, -0.438015  , -0.39821131],\n",
       "       [-0.54261783, -0.425234  , -0.42925089],\n",
       "       [-0.49739881, -0.44093638, -0.34234008],\n",
       "       [-0.52429833, -0.43071157, -0.39492477],\n",
       "       [-0.5405308 , -0.43071157, -0.42048677],\n",
       "       [-0.54887892, -0.44093638, -0.42340815],\n",
       "       [-0.54887892, -0.44093638, -0.42340815],\n",
       "       [-0.54748757, -0.43158799, -0.43056551],\n",
       "       [ 2.9183762 ,  3.94112128,  0.65457804],\n",
       "       [-0.54748757, -0.4277902 , -0.43436329],\n",
       "       [-0.54261783, -0.42815537, -0.42632952],\n",
       "       [-0.53241456, -0.43684646, -0.40157089],\n",
       "       [-0.54261783, -0.4354588 , -0.41902609],\n",
       "       [-0.54261783, -0.4354588 , -0.41902609],\n",
       "       [-0.55351677, -0.43889142, -0.43275653],\n",
       "       [-0.54887892, -0.43889142, -0.42545311],\n",
       "       [-0.54748757, -0.43363295, -0.42852055],\n",
       "       [-0.53913944, -0.43838018, -0.41062714],\n",
       "       [-0.54748757, -0.43509363, -0.42705986],\n",
       "       [-0.5405308 , -0.42508793, -0.42611041],\n",
       "       [-0.54748757, -0.42954303, -0.43261047],\n",
       "       [-0.5405308 , -0.43020033, -0.42099801],\n",
       "       [-0.53566105, -0.42304297, -0.42048677],\n",
       "       [-0.54887892, -0.4329026 , -0.43144192],\n",
       "       [-0.53705241, -0.43633522, -0.40938556],\n",
       "       [-0.54261783, -0.43253743, -0.42194746],\n",
       "       [-0.54400918, -0.43889142, -0.4177845 ],\n",
       "       [-0.54400918, -0.43100371, -0.42567221],\n",
       "       [-0.54609621, -0.4277902 , -0.43217226],\n",
       "       [-0.54261783, -0.43253743, -0.42194746],\n",
       "       [-0.53913944, -0.42085194, -0.42815537],\n",
       "       [-0.54400918, -0.42289691, -0.43377901],\n",
       "       [-0.54400918, -0.42545311, -0.43122281],\n",
       "       [-0.54540054, -0.42611041, -0.43275653],\n",
       "       [-0.53913944, -0.42340815, -0.42559917],\n",
       "       [-0.5159502 , -0.42705986, -0.38543031],\n",
       "       [-0.52406643, -0.41720023, -0.40807094],\n",
       "       [-0.52870428, -0.40624509, -0.42632952],\n",
       "       [-0.51131235, -0.41026197, -0.39492477],\n",
       "       [-0.53218267, -0.40697543, -0.43107675],\n",
       "       [-0.53566105, -0.40514957, -0.43838018],\n",
       "       [-0.52893618, -0.43889142, -0.39404836],\n",
       "       [-0.52893618, -0.43889142, -0.39404836],\n",
       "       [-0.5405308 , -0.43531274, -0.41588561],\n",
       "       [-0.49044204, -0.41902609, -0.35329522],\n",
       "       [ 1.77050884,  3.21443005, -0.42632952],\n",
       "       [-0.53241456, -0.42662165, -0.41179569],\n",
       "       [-0.52429833, -0.43173405, -0.39390229],\n",
       "       [ 1.77050884,  3.21808176, -0.42998123],\n",
       "       [ 1.77050884,  3.21808176, -0.42998123],\n",
       "       [-0.52870428, -0.41610472, -0.41646989],\n",
       "       [-0.5405308 , -0.42107105, -0.4301273 ],\n",
       "       [-0.53566105, -0.41391369, -0.42961606],\n",
       "       [ 0.13566744,  0.65457804, -0.44093638],\n",
       "       [-0.08463034,  0.30401343, -0.43728466],\n",
       "       [-0.30029027, -0.05093324, -0.42194746],\n",
       "       [-0.52058805, -0.38908203, -0.43071157],\n",
       "       [-0.52058805, -0.38908203, -0.43071157],\n",
       "       [-0.52058805, -0.38908203, -0.43071157],\n",
       "       [-0.53357402, -0.43889142, -0.40135179],\n",
       "       [-0.55676326, -0.43582398, -0.44093638],\n",
       "       [-0.54400918, -0.43392508, -0.42275084],\n",
       "       [-0.5372843 , -0.43071157, -0.41537437],\n",
       "       [-0.52406643, -0.42450366, -0.40076751],\n",
       "       [-0.52406643, -0.42450366, -0.40076751],\n",
       "       [-0.51942859, -0.40770577, -0.41026197],\n",
       "       [-0.51942859, -0.40770577, -0.41026197],\n",
       "       [-0.51942859, -0.40770577, -0.41026197],\n",
       "       [-0.30029027, -0.04582084, -0.42705986],\n",
       "       [-0.30029027, -0.04582084, -0.42705986],\n",
       "       [-0.41623647, -0.22475486, -0.43071157],\n",
       "       [-0.49739881, -0.35694694, -0.42632952],\n",
       "       [-0.45102033, -0.3036319 , -0.40661026],\n",
       "       [-0.48116634, -0.32189048, -0.43582398],\n",
       "       [-0.50435558, -0.35329522, -0.44093638],\n",
       "       [-0.50435558, -0.35329522, -0.44093638],\n",
       "       [-0.50435558, -0.35329522, -0.44093638],\n",
       "       [-0.51038478, -0.43582398, -0.36790208],\n",
       "       [-0.51038478, -0.43582398, -0.36790208],\n",
       "       [-0.53032753, -0.43071157, -0.40441923],\n",
       "       [-0.50203666, -0.4226778 , -0.36790208],\n",
       "       [-0.50203666, -0.4226778 , -0.36790208],\n",
       "       [-0.04984648,  0.30401343, -0.38250894],\n",
       "       [-0.4672528 , -0.3971158 , -0.33868836],\n",
       "       [-0.4672528 , -0.3971158 , -0.33868836],\n",
       "       [-0.4672528 , -0.3971158 , -0.33868836],\n",
       "       [-0.49044204, -0.36790208, -0.40441923],\n",
       "       [-0.49044204, -0.36790208, -0.40441923],\n",
       "       [-0.49044204, -0.36790208, -0.40441923],\n",
       "       [-0.49044204, -0.34599179, -0.42632952],\n",
       "       [-0.49044204, -0.33868836, -0.43363295],\n",
       "       [-0.49739881, -0.34964351, -0.43363295],\n",
       "       [-0.49044204, -0.36790208, -0.40441923],\n",
       "       [-0.49044204, -0.34599179, -0.42632952],\n",
       "       [-0.49739881, -0.34964351, -0.43363295],\n",
       "       [-0.44406356, -0.43363295, -0.26565407],\n",
       "       [-0.44406356, -0.43363295, -0.26565407],\n",
       "       [-0.44406356, -0.43363295, -0.26565407],\n",
       "       [-0.44406356, -0.3971158 , -0.30217122],\n",
       "       [-0.44406356, -0.3971158 , -0.30217122],\n",
       "       [-0.44406356, -0.3971158 , -0.30217122],\n",
       "       [-0.42087432, -0.33868836, -0.32408151],\n",
       "       [-0.42087432, -0.33868836, -0.32408151],\n",
       "       [-0.42087432, -0.33868836, -0.32408151],\n",
       "       [-0.42087432, -0.32408151, -0.33868836],\n",
       "       [-0.42087432, -0.32408151, -0.33868836],\n",
       "       [-0.42087432, -0.32408151, -0.33868836],\n",
       "       [-0.42087432, -0.25104721, -0.41172266],\n",
       "       [-0.42087432, -0.25104721, -0.41172266],\n",
       "       [-0.45102033, -0.28902504, -0.42121712],\n",
       "       [-0.42087432, -0.25104721, -0.41172266],\n",
       "       [-0.42087432, -0.23644035, -0.42632952],\n",
       "       [-0.42087432, -0.23644035, -0.42632952],\n",
       "       [-0.09622496, -0.44093638,  0.28940657],\n",
       "       [ 0.01972124, -0.44093638,  0.4719923 ],\n",
       "       [ 0.01972124, -0.44093638,  0.4719923 ],\n",
       "       [-0.21217116, -0.25835064, -0.0757649 ],\n",
       "       [-0.21217116, -0.25835064, -0.0757649 ],\n",
       "       [-0.21217116, -0.25835064, -0.0757649 ],\n",
       "       [-0.21217116, -0.1487992 , -0.18531635],\n",
       "       [-0.21217116, -0.11228205, -0.22183349],\n",
       "       [-0.21217116, -0.18531635, -0.1487992 ],\n",
       "       [-0.21217116, -0.1487992 , -0.18531635],\n",
       "       [-0.21217116, -0.1487992 , -0.18531635],\n",
       "       [-0.21217116, -0.1487992 , -0.18531635],\n",
       "       [-0.21217116, -0.0757649 , -0.25835064],\n",
       "       [-0.21217116, -0.0757649 , -0.25835064],\n",
       "       [-0.28173888, -0.13419234, -0.30947465],\n",
       "       [-0.41623647, -0.22840658, -0.42705986],\n",
       "       [-0.20057654,  0.12142769, -0.43728466],\n",
       "       [-0.41159862, -0.22767624, -0.42048677],\n",
       "       [ 0.25161363, -0.44093638,  0.83716377],\n",
       "       [-0.09622496, -0.44093638,  0.28940657],\n",
       "       [ 0.83134463,  0.43547515,  0.87368092],\n",
       "       [ 0.13566744, -0.0757649 ,  0.28940657],\n",
       "       [ 0.13566744, -0.0757649 ,  0.28940657],\n",
       "       [ 0.13566744, -0.0757649 ,  0.28940657],\n",
       "       [ 0.13566744, -0.0757649 ,  0.28940657],\n",
       "       [ 0.13566744,  0.07030368,  0.14333798],\n",
       "       [ 0.13566744,  0.07030368,  0.14333798],\n",
       "       [ 0.13566744,  0.07030368,  0.14333798],\n",
       "       [ 0.36755983,  0.14333798,  0.43547515],\n",
       "       [ 0.13566744, -0.0757649 ,  0.28940657],\n",
       "       [ 0.13566744,  0.28940657, -0.0757649 ],\n",
       "       [ 0.13566744,  0.28940657, -0.0757649 ],\n",
       "       [ 0.13566744,  0.28940657, -0.0757649 ],\n",
       "       [-0.08463034,  0.30401343, -0.43728466],\n",
       "       [-0.08463034,  0.30401343, -0.43728466],\n",
       "       [-0.30029027, -0.04582084, -0.42705986],\n",
       "       [ 0.59945223, -0.44093638,  1.38492098],\n",
       "       [ 0.59945223, -0.44093638,  1.38492098],\n",
       "       [ 0.59945223, -0.44093638,  1.38492098],\n",
       "       [ 0.83134463,  0.14333798,  1.1658181 ],\n",
       "       [ 0.83134463,  0.14333798,  1.1658181 ],\n",
       "       [ 0.83134463,  0.14333798,  1.1658181 ],\n",
       "       [ 0.59945223,  0.14333798,  0.80064663],\n",
       "       [ 0.59945223,  0.14333798,  0.80064663],\n",
       "       [ 0.59945223,  0.14333798,  0.80064663],\n",
       "       [ 0.83134463,  1.01974951,  0.28940657],\n",
       "       [ 0.83134463,  1.01974951,  0.28940657],\n",
       "       [ 0.83134463,  1.01974951,  0.28940657],\n",
       "       [ 0.83134463,  1.1658181 ,  0.14333798],\n",
       "       [ 0.83134463,  1.1658181 ,  0.14333798],\n",
       "       [ 0.83134463,  1.1658181 ,  0.14333798],\n",
       "       [ 0.39074907,  1.02705294, -0.41172266],\n",
       "       [ 0.39074907,  1.02705294, -0.41172266],\n",
       "       [ 0.3837923 ,  1.02997431, -0.42559917],\n",
       "       [ 2.9183762 , -0.0757649 ,  4.67146422],\n",
       "       [ 2.9183762 , -0.0757649 ,  4.67146422],\n",
       "       [ 2.9183762 ,  0.28940657,  4.30629275],\n",
       "       [ 2.9183762 ,  0.28940657,  4.30629275],\n",
       "       [ 2.9183762 ,  0.28940657,  4.30629275],\n",
       "       [ 2.9183762 ,  0.28940657,  4.30629275],\n",
       "       [ 2.9183762 ,  2.48043539,  2.11526392],\n",
       "       [ 2.9183762 ,  2.48043539,  2.11526392],\n",
       "       [ 2.22269901,  1.01974951,  2.48043539],\n",
       "       [ 2.9183762 ,  3.21077833,  1.38492098],\n",
       "       [ 2.9183762 ,  3.21077833,  1.38492098],\n",
       "       [ 2.9183762 ,  3.21077833,  1.38492098],\n",
       "       [ 2.9183762 ,  3.21077833,  1.38492098],\n",
       "       [ 2.9183762 ,  3.21077833,  1.38492098],\n",
       "       [ 2.9183762 ,  3.21077833,  1.38492098],\n",
       "       [ 1.77050884,  3.21443005, -0.42632952],\n",
       "       [ 0.62264147,  1.39952784, -0.41902609],\n",
       "       [ 0.62264147,  1.39952784, -0.41902609]])"
      ]
     },
     "execution_count": 157,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#prediction from knn scratch\n",
    "prediction_from_scratch = cross_validation(data,1,3)\n",
    "prediction_from_scratch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "===== Evaluation of Model ========="
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 158,
   "metadata": {},
   "outputs": [],
   "source": [
    "# implementation  of C_index\n",
    "def c_index (true_labels, predictions):\n",
    "    n = 0\n",
    "    h_sum =0\n",
    "    \n",
    "    for i in range(len(true_labels)):\n",
    "        t = true_labels[i]\n",
    "        p = predictions[i]\n",
    "        \n",
    "        \n",
    "        for j in range(i+1, len(true_labels)):\n",
    "            nt = true_labels[j]\n",
    "            np = predictions[j]\n",
    "            \n",
    "            if( t!=nt):\n",
    "                n += 1\n",
    "                \n",
    "                if(p < np and t <  nt) or (p > np and t > nt):\n",
    "                    h_sum += 1\n",
    "                \n",
    "                elif(p == np):\n",
    "                    h_sum += 0.5\n",
    "                \n",
    "    c_index = h_sum/n\n",
    "    return c_index\n",
    "    \n",
    "    \n",
    "    \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 159,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.75"
      ]
     },
     "execution_count": 159,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# testing c-index implementation\n",
    "true_labels = [-1, 1, 1, -1, 1]\n",
    "predictions = [0.60, 0.80, 0.75, 0.75, 0.70]\n",
    "c_index(true_labels, predictions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "                  ==== Compare Knn  from scratch  with scikit learn =====\n",
    "Now here, we want to compare knn from scratch with sklearn, whether it works as correctly as sklearn knn regression or not. Thus, the c-index from knn scratch is calcualted here. Here, leave-one-out is used for both (knn from scratch and \n",
    "knn from sklearn). This is just testing purpose between  knn from scratch and knn from sklearn."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C-index  of 0 is 0.90\n",
      "C-index  of 1 is 0.88\n",
      "C-index  of 2 is 0.85\n"
     ]
    }
   ],
   "source": [
    "'''==== find the c-index of usign knn from scratch ====''' \n",
    "\n",
    "#find c-index of 'c_total =0', 'Cd =1', 'Pb =2'\n",
    "for x in range(len(prediction_from_scratch.T)): # tranpose the result array and loop thorugh it\n",
    "    true_labels = (df.values).T[x] # true values \n",
    "    predictions = prediction_from_scratch.T[x] # predicted values\n",
    "    c_val = c_index(true_labels , predictions ) # call c_index function\n",
    "    print('C-index  of',x,  'is {0:.2f}'.format(c_val))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " === finding c-index from sklearn knn ===="
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 161,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import necessary libraries\n",
    "from  sklearn.neighbors import KNeighborsRegressor\n",
    "from sklearn.multioutput import MultiOutputRegressor\n",
    "\n",
    "#define function\n",
    "def cross_validation_sklearn( dataset, leav_out, k):\n",
    "    sklearn_prediction = [] # store prediction\n",
    "    \n",
    "    i = 0 \n",
    "    while i < len(dataset): #loop until it reachs all samples\n",
    "        \n",
    "        # separate test and training dataset\n",
    "        test = dataset[i: i+leav_out] # test set\n",
    "        y_test, x_test = test[:,0:3], test[:,3:6] # separate input features(x_test) and target (y_test)\n",
    "        \n",
    "        train = np.concatenate([dataset[0:i], dataset[i+leav_out:]]) # train set\n",
    "        y_train, x_train = train[:,0:3], train[:,3:6] # separate input features(x_train) and target (y_train)\n",
    "       \n",
    "    \n",
    "\n",
    "        # intialize knn regression\n",
    "        knn= KNeighborsRegressor(n_neighbors=k, metric='euclidean' )\n",
    "        multi_regr =MultiOutputRegressor(knn) #intialize multioutput regression (since we have multi target ouputs)\n",
    "        multi_regr.fit(x_train, y_train) # train with the dataset\n",
    "        respon_val = multi_regr.predict(x_test) # get prediction\n",
    "        sklearn_prediction.append(respon_val) # append prediction\n",
    "        \n",
    "        \n",
    "        \n",
    "        \n",
    "        i = i + leav_out # increament i value\n",
    "    result = [instance for item in sklearn_prediction for instance in item ] # change 3d into 2d list\n",
    "    return np.array(result)\n",
    "        \n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.5405308 , -0.53357402, -0.54887892, -0.54609621, -0.54748757,\n",
       "        -0.54609621, -0.55351677, -0.55212541, -0.54400918, -0.53682052,\n",
       "        -0.55351677, -0.53913944, -0.53913944, -0.53218267, -0.53913944,\n",
       "        -0.53913944, -0.53102321, -0.54261783, -0.49739881, -0.52429833,\n",
       "        -0.5405308 , -0.54887892, -0.54887892, -0.54748757,  2.9183762 ,\n",
       "        -0.54748757, -0.54261783, -0.53241456, -0.54261783, -0.54261783,\n",
       "        -0.55351677, -0.54887892, -0.54748757, -0.53913944, -0.54748757,\n",
       "        -0.5405308 , -0.54748757, -0.5405308 , -0.53566105, -0.54887892,\n",
       "        -0.53705241, -0.54261783, -0.54400918, -0.54400918, -0.54609621,\n",
       "        -0.54261783, -0.53913944, -0.54400918, -0.54400918, -0.54540054,\n",
       "        -0.53913944, -0.5159502 , -0.52406643, -0.52870428, -0.51131235,\n",
       "        -0.53218267, -0.53566105, -0.52893618, -0.52893618, -0.5405308 ,\n",
       "        -0.49044204,  1.77050884, -0.53241456, -0.52429833,  1.77050884,\n",
       "         1.77050884, -0.52870428, -0.5405308 , -0.53566105,  0.13566744,\n",
       "        -0.08463034, -0.30029027, -0.52058805, -0.52058805, -0.52058805,\n",
       "        -0.53357402, -0.55676326, -0.54400918, -0.5372843 , -0.52406643,\n",
       "        -0.52406643, -0.51942859, -0.51942859, -0.51942859, -0.30029027,\n",
       "        -0.30029027, -0.41623647, -0.49739881, -0.45102033, -0.48116634,\n",
       "        -0.50435558, -0.50435558, -0.50435558, -0.51038478, -0.51038478,\n",
       "        -0.53032753, -0.50203666, -0.50203666, -0.04984648, -0.4672528 ,\n",
       "        -0.4672528 , -0.4672528 , -0.49044204, -0.49044204, -0.49044204,\n",
       "        -0.49044204, -0.49044204, -0.49739881, -0.49044204, -0.49044204,\n",
       "        -0.49739881, -0.44406356, -0.44406356, -0.44406356, -0.44406356,\n",
       "        -0.44406356, -0.44406356, -0.42087432, -0.42087432, -0.42087432,\n",
       "        -0.42087432, -0.42087432, -0.42087432, -0.42087432, -0.42087432,\n",
       "        -0.45102033, -0.42087432, -0.42087432, -0.42087432, -0.09622496,\n",
       "         0.01972124,  0.01972124, -0.21217116, -0.21217116, -0.21217116,\n",
       "        -0.21217116, -0.21217116, -0.21217116, -0.21217116, -0.21217116,\n",
       "        -0.21217116, -0.21217116, -0.21217116, -0.28173888, -0.41623647,\n",
       "        -0.20057654, -0.41159862,  0.25161363, -0.09622496,  0.83134463,\n",
       "         0.13566744,  0.13566744,  0.13566744,  0.13566744,  0.13566744,\n",
       "         0.13566744,  0.13566744,  0.36755983,  0.13566744,  0.13566744,\n",
       "         0.13566744,  0.13566744, -0.08463034, -0.08463034, -0.30029027,\n",
       "         0.59945223,  0.59945223,  0.59945223,  0.83134463,  0.83134463,\n",
       "         0.83134463,  0.59945223,  0.59945223,  0.59945223,  0.83134463,\n",
       "         0.83134463,  0.83134463,  0.83134463,  0.83134463,  0.83134463,\n",
       "         0.39074907,  0.39074907,  0.3837923 ,  2.9183762 ,  2.9183762 ,\n",
       "         2.9183762 ,  2.9183762 ,  2.9183762 ,  2.9183762 ,  2.9183762 ,\n",
       "         2.9183762 ,  2.22269901,  2.9183762 ,  2.9183762 ,  2.9183762 ,\n",
       "         2.9183762 ,  2.9183762 ,  2.9183762 ,  1.77050884,  0.62264147,\n",
       "         0.62264147],\n",
       "       [-0.43582398, -0.43889142, -0.43450936, -0.42925089, -0.44093638,\n",
       "        -0.43655432, -0.43786894, -0.43553184, -0.43392508, -0.44093638,\n",
       "        -0.43889142, -0.43838018, -0.42231263, -0.43838018, -0.42085194,\n",
       "        -0.42596435, -0.438015  , -0.425234  , -0.44093638, -0.43071157,\n",
       "        -0.43071157, -0.44093638, -0.44093638, -0.43158799,  3.94112128,\n",
       "        -0.4277902 , -0.42815537, -0.43684646, -0.4354588 , -0.4354588 ,\n",
       "        -0.43889142, -0.43889142, -0.43363295, -0.43838018, -0.43509363,\n",
       "        -0.42508793, -0.42954303, -0.43020033, -0.42304297, -0.4329026 ,\n",
       "        -0.43633522, -0.43253743, -0.43889142, -0.43100371, -0.4277902 ,\n",
       "        -0.43253743, -0.42085194, -0.42289691, -0.42545311, -0.42611041,\n",
       "        -0.42340815, -0.42705986, -0.41720023, -0.40624509, -0.41026197,\n",
       "        -0.40697543, -0.40514957, -0.43889142, -0.43889142, -0.43531274,\n",
       "        -0.41902609,  3.21443005, -0.42662165, -0.43173405,  3.21808176,\n",
       "         3.21808176, -0.41610472, -0.42107105, -0.41391369,  0.65457804,\n",
       "         0.30401343, -0.05093324, -0.38908203, -0.38908203, -0.38908203,\n",
       "        -0.43889142, -0.43582398, -0.43392508, -0.43071157, -0.42450366,\n",
       "        -0.42450366, -0.40770577, -0.40770577, -0.40770577, -0.04582084,\n",
       "        -0.04582084, -0.22475486, -0.35694694, -0.3036319 , -0.32189048,\n",
       "        -0.35329522, -0.35329522, -0.35329522, -0.43582398, -0.43582398,\n",
       "        -0.43071157, -0.4226778 , -0.4226778 ,  0.30401343, -0.3971158 ,\n",
       "        -0.3971158 , -0.3971158 , -0.36790208, -0.36790208, -0.36790208,\n",
       "        -0.34599179, -0.33868836, -0.34964351, -0.36790208, -0.34599179,\n",
       "        -0.34964351, -0.43363295, -0.43363295, -0.43363295, -0.3971158 ,\n",
       "        -0.3971158 , -0.3971158 , -0.33868836, -0.33868836, -0.33868836,\n",
       "        -0.32408151, -0.32408151, -0.32408151, -0.25104721, -0.25104721,\n",
       "        -0.28902504, -0.25104721, -0.23644035, -0.23644035, -0.44093638,\n",
       "        -0.44093638, -0.44093638, -0.25835064, -0.25835064, -0.25835064,\n",
       "        -0.1487992 , -0.11228205, -0.18531635, -0.1487992 , -0.1487992 ,\n",
       "        -0.1487992 , -0.0757649 , -0.0757649 , -0.13419234, -0.22840658,\n",
       "         0.12142769, -0.22767624, -0.44093638, -0.44093638,  0.43547515,\n",
       "        -0.0757649 , -0.0757649 , -0.0757649 , -0.0757649 ,  0.07030368,\n",
       "         0.07030368,  0.07030368,  0.14333798, -0.0757649 ,  0.28940657,\n",
       "         0.28940657,  0.28940657,  0.30401343,  0.30401343, -0.04582084,\n",
       "        -0.44093638, -0.44093638, -0.44093638,  0.14333798,  0.14333798,\n",
       "         0.14333798,  0.14333798,  0.14333798,  0.14333798,  1.01974951,\n",
       "         1.01974951,  1.01974951,  1.1658181 ,  1.1658181 ,  1.1658181 ,\n",
       "         1.02705294,  1.02705294,  1.02997431, -0.0757649 , -0.0757649 ,\n",
       "         0.28940657,  0.28940657,  0.28940657,  0.28940657,  2.48043539,\n",
       "         2.48043539,  1.01974951,  3.21077833,  3.21077833,  3.21077833,\n",
       "         3.21077833,  3.21077833,  3.21077833,  3.21443005,  1.39952784,\n",
       "         1.39952784],\n",
       "       [-0.41537437, -0.40135179, -0.42983516, -0.43071157, -0.42121712,\n",
       "        -0.42340815, -0.43377901, -0.43392508, -0.42275084, -0.40441923,\n",
       "        -0.43275653, -0.41062714, -0.42669469, -0.399672  , -0.42815537,\n",
       "        -0.42304297, -0.39821131, -0.42925089, -0.34234008, -0.39492477,\n",
       "        -0.42048677, -0.42340815, -0.42340815, -0.43056551,  0.65457804,\n",
       "        -0.43436329, -0.42632952, -0.40157089, -0.41902609, -0.41902609,\n",
       "        -0.43275653, -0.42545311, -0.42852055, -0.41062714, -0.42705986,\n",
       "        -0.42611041, -0.43261047, -0.42099801, -0.42048677, -0.43144192,\n",
       "        -0.40938556, -0.42194746, -0.4177845 , -0.42567221, -0.43217226,\n",
       "        -0.42194746, -0.42815537, -0.43377901, -0.43122281, -0.43275653,\n",
       "        -0.42559917, -0.38543031, -0.40807094, -0.42632952, -0.39492477,\n",
       "        -0.43107675, -0.43838018, -0.39404836, -0.39404836, -0.41588561,\n",
       "        -0.35329522, -0.42632952, -0.41179569, -0.39390229, -0.42998123,\n",
       "        -0.42998123, -0.41646989, -0.4301273 , -0.42961606, -0.44093638,\n",
       "        -0.43728466, -0.42194746, -0.43071157, -0.43071157, -0.43071157,\n",
       "        -0.40135179, -0.44093638, -0.42275084, -0.41537437, -0.40076751,\n",
       "        -0.40076751, -0.41026197, -0.41026197, -0.41026197, -0.42705986,\n",
       "        -0.42705986, -0.43071157, -0.42632952, -0.40661026, -0.43582398,\n",
       "        -0.44093638, -0.44093638, -0.44093638, -0.36790208, -0.36790208,\n",
       "        -0.40441923, -0.36790208, -0.36790208, -0.38250894, -0.33868836,\n",
       "        -0.33868836, -0.33868836, -0.40441923, -0.40441923, -0.40441923,\n",
       "        -0.42632952, -0.43363295, -0.43363295, -0.40441923, -0.42632952,\n",
       "        -0.43363295, -0.26565407, -0.26565407, -0.26565407, -0.30217122,\n",
       "        -0.30217122, -0.30217122, -0.32408151, -0.32408151, -0.32408151,\n",
       "        -0.33868836, -0.33868836, -0.33868836, -0.41172266, -0.41172266,\n",
       "        -0.42121712, -0.41172266, -0.42632952, -0.42632952,  0.28940657,\n",
       "         0.4719923 ,  0.4719923 , -0.0757649 , -0.0757649 , -0.0757649 ,\n",
       "        -0.18531635, -0.22183349, -0.1487992 , -0.18531635, -0.18531635,\n",
       "        -0.18531635, -0.25835064, -0.25835064, -0.30947465, -0.42705986,\n",
       "        -0.43728466, -0.42048677,  0.83716377,  0.28940657,  0.87368092,\n",
       "         0.28940657,  0.28940657,  0.28940657,  0.28940657,  0.14333798,\n",
       "         0.14333798,  0.14333798,  0.43547515,  0.28940657, -0.0757649 ,\n",
       "        -0.0757649 , -0.0757649 , -0.43728466, -0.43728466, -0.42705986,\n",
       "         1.38492098,  1.38492098,  1.38492098,  1.1658181 ,  1.1658181 ,\n",
       "         1.1658181 ,  0.80064663,  0.80064663,  0.80064663,  0.28940657,\n",
       "         0.28940657,  0.28940657,  0.14333798,  0.14333798,  0.14333798,\n",
       "        -0.41172266, -0.41172266, -0.42559917,  4.67146422,  4.67146422,\n",
       "         4.30629275,  4.30629275,  4.30629275,  4.30629275,  2.11526392,\n",
       "         2.11526392,  2.48043539,  1.38492098,  1.38492098,  1.38492098,\n",
       "         1.38492098,  1.38492098,  1.38492098, -0.42632952, -0.41902609,\n",
       "        -0.41902609]])"
      ]
     },
     "execution_count": 162,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# prediction from sklearn knn regression\n",
    "prediction_from_sklearn = cross_validation_sklearn( data, 1, 3)\n",
    "prediction_from_sklearn.T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C-index  of 0 is 0.90\n",
      "C-index  of 1 is 0.88\n",
      "C-index  of 2 is 0.85\n"
     ]
    }
   ],
   "source": [
    "''' === c-index from sklearn prediction ==== '''\n",
    "\n",
    "# find c-index of 'c_total =0', 'Cd =1', 'Pb =2'\n",
    "for x in range(len(prediction_from_sklearn.T)): # tranpose the result array and loop thorugh it\n",
    "    true_label = (df.values).T[x] # true values \n",
    "    prediction = prediction_from_sklearn.T[x] # predicted values\n",
    "    c_val = c_index(true_label , prediction) # call c_index function\n",
    "    print('C-index  of',x,  'is {0:.2f}'.format(c_val))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: c-indexs from both i.e knn from scratch and sklearn are same, which indicates that our knn implementation is working correctly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now find the c-index from two CV, 1)leave-one-out and 2) leave-three-out using 'k' for knn from 1 to 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "When k is  1\n",
      "C-index  of 0 is 0.90\n",
      "C-index  of 1 is 0.90\n",
      "C-index  of 2 is 0.87\n",
      "\n",
      "When k is  2\n",
      "C-index  of 0 is 0.91\n",
      "C-index  of 1 is 0.90\n",
      "C-index  of 2 is 0.87\n",
      "\n",
      "When k is  3\n",
      "C-index  of 0 is 0.90\n",
      "C-index  of 1 is 0.88\n",
      "C-index  of 2 is 0.85\n",
      "\n",
      "When k is  4\n",
      "C-index  of 0 is 0.89\n",
      "C-index  of 1 is 0.85\n",
      "C-index  of 2 is 0.85\n",
      "\n",
      "When k is  5\n",
      "C-index  of 0 is 0.88\n",
      "C-index  of 1 is 0.83\n",
      "C-index  of 2 is 0.83\n"
     ]
    }
   ],
   "source": [
    "'''======== c-index at leave-one-out ======'''\n",
    "    \n",
    "#cross validation (leave one out)\n",
    "data =np.array(df)\n",
    "\n",
    "# k from 1 to 5\n",
    "for k in range(1, 6):\n",
    "    leave_one_out_result = cross_validation(data,  1, k)\n",
    "   \n",
    "    \n",
    "    print('\\nWhen k is ', k)\n",
    "    # find c-index of 'c_total =0', 'Cd =1', 'Pb =2'\n",
    "    for x in range(len(leave_one_out_result.T)): # tranpose the result array and loop thorugh it\n",
    "        true_labels = (df.values).T[x] # true values \n",
    "        predictions = leave_one_out_result.T[x] # predicted values\n",
    "        c_val = c_index(true_labels , predictions ) # call c_index function\n",
    "        print('C-index  of',x,  'is {0:.2f}'.format(c_val))\n",
    "\n",
    "   \n",
    "   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "=======  C-index at leave-3-out ======="
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "When k is  1\n",
      "C-index of  0 is 0.82\n",
      "C-index of  1 is 0.74\n",
      "C-index of  2 is 0.74\n",
      "\n",
      "When k is  2\n",
      "C-index of  0 is 0.82\n",
      "C-index of  1 is 0.75\n",
      "C-index of  2 is 0.75\n",
      "\n",
      "When k is  3\n",
      "C-index of  0 is 0.82\n",
      "C-index of  1 is 0.74\n",
      "C-index of  2 is 0.75\n",
      "\n",
      "When k is  4\n",
      "C-index of  0 is 0.82\n",
      "C-index of  1 is 0.72\n",
      "C-index of  2 is 0.76\n",
      "\n",
      "When k is  5\n",
      "C-index of  0 is 0.82\n",
      "C-index of  1 is 0.72\n",
      "C-index of  2 is 0.76\n"
     ]
    }
   ],
   "source": [
    "# cross validation (leave-3-out)\n",
    "data = np.array(df)\n",
    "\n",
    "# when k = 1,5\n",
    "for k in range(1,6):\n",
    "    leave_3_out_result = cross_validation(data,  3, k) # call cross validation function\n",
    "    \n",
    "    \n",
    "    print('\\nWhen k is ', k)\n",
    "    # find c-index of 'c_total = 0', 'Cd =1', 'Pb =2'\n",
    "    for x in range(len(leave_3_out_result.T)):\n",
    "        true_labels = (df.values).T[x] # true values\n",
    "        predictions = leave_3_out_result.T[x] # predicted values\n",
    "        c_val = c_index(true_labels , predictions )\n",
    "        print('C-index of ', x,'is {0:.2f}'.format(c_val))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Report:\n",
    "After obeserving the results from both the cross validations, it is found that the result from the leave-1-out CV gives\n",
    "more percentage of c-index than the result from leave-3-out CV. This is because  some replicas of the data instance\n",
    "(test sample) are in training set when performing leave-1-out CV. Thus, the information is leaked in the learning set and regressior algorithm predict very well.  Eventhough the predicted result (c-index) in leave-3-out CV is less, it is the realistic result because the leave-3-out CV in this dataset maintains the Independent and identically distributed (IID) by taking 3- replicas as testset and rest as training set.\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}