dm_lab_2_fm.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.5"
    },
    "colab": {
      "name": "dm_lab-2_fm.ipynb",
      "provenance": [],
      "include_colab_link": true
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/foxtrotmike/CS909/blob/master/dm_lab_2_fm.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H5YY8bbnZI9T"
      },
      "source": [
        "# Exercise-2\n",
        "## by Fayyaz Minhas\n",
        "## Gradient Descent and Perceptron\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U1YPXeG3ZI9X"
      },
      "source": [
        "# Understanding gradient Descent\n",
        "\n",
        "We discussed how gradient descent works for solving optimization problems: By taking steps in a direction opposite to the gradient. Below is a very simple implementation for this method. Based on this, can you:\n",
        "1. Try to understand how Gradient Descent Works\n",
        "2. Can you find the  minima of the following function: $f(x)=2x^2-x$. You will need to update the implementation of the function and its derivative in the code for this.\n",
        "3. What happens when you use: $f(x)=sin(3x)$? Is this function convex?\n",
        "4. What do you think are the limitations of gradient descent for non-convex functions?\n",
        "5. What happens when you change the learning rate parameter (step size)?\n",
        "You can also look into questions posted as part of the self assessment exercise.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6vNkYFhYZI9Y",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "outputId": "02c1eeae-c2c8-4367-b5b3-7b9078889313"
      },
      "source": [
        "import numpy as np\n",
        "\n",
        "def gd(f,df,x0=0.0,lr = 0.01,eps=1e-4,nmax=1000, history = True):\n",
        "    \"\"\"\n",
        "    Implementation of a gradient descent solver.\n",
        "        f: function, f(x) returns value of f(x) for a given x\n",
        "        df: gradient function df(x) returns the gradient at x\n",
        "        x0: initial position [Default 0.0]\n",
        "        lr: learning rate [0.001]\n",
        "        eps: min step size threshold [1e-4]\n",
        "        nmax: maximum number of iters [1000]\n",
        "        history: whether to store history of x or not [True]\n",
        "    Returns:\n",
        "        x: argmin_x f(x)\n",
        "        converged: True if the final step size is less than eps else false\n",
        "        H: history\n",
        "    \"\"\"\n",
        "    H = []\n",
        "    x = x0\n",
        "    if history:\n",
        "        H = [[x,f(x)]]\n",
        "    for i in range(nmax):\n",
        "        dx = -lr*df(x) #gradient step\n",
        "        if np.linalg.norm(dx)<eps: # if the step taken is too small, we have converged\n",
        "            break\n",
        "        if history:\n",
        "            H.append([x+dx,f(x+dx)])\n",
        "        x = x+dx #gradient update\n",
        "    converged = np.linalg.norm(dx)<eps        \n",
        "    return x,converged,np.array(H)\n",
        "    \n",
        "if __name__=='__main__':\n",
        "    import matplotlib.pyplot as plt\n",
        "    def f(x):\n",
        "        y = x**2 \n",
        "        return y\n",
        "    def df(x):\n",
        "        dy = 2*x\n",
        "        return dy\n",
        "    \n",
        "\n",
        "    z = np.linspace(-3,3,100)\n",
        "    #select random initial point in the range\n",
        "    x0 = np.min(z)+(np.max(z)-np.min(z))*np.random.rand()\n",
        "    \n",
        "    x,c,H = gd(f,df,x0=x0,lr = 0.01,eps=1e-4,nmax=1000, history = True) \n",
        "    \n",
        "    plt.plot(z,f(z)); plt.plot(z,df(z));\n",
        "    plt.legend(['f(x)','df(x)'])\n",
        "    plt.xlabel('x');plt.ylabel('value')\n",
        "    s = 'Convergence in '+str(len(H))+' steps'\n",
        "    if not c:\n",
        "        s = 'No '+s\n",
        "    plt.title(s)\n",
        "    plt.plot(H[0,0],H[0,1],'ko',markersize=10)\n",
        "    plt.plot(H[:,0],H[:,1],'r.-')\n",
        "    plt.plot(H[-1,0],H[-1,1],'k*',markersize=10)    \n",
        "    plt.grid(); plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEWCAYAAABv+EDhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3hU1dbA4d9KIQmEJiVUAbHRhSASQCSAKCpgw15RUGxcxavX8llQLIgNhSsgqFyRqKgIdoHQpCNNQHoLvZNA+qzvjzPBACGEZJKTzKz3ec7jzJy2NhP3mrP3PvuIqmKMMSbwBLkdgDHGGHdYAjDGmABlCcAYYwKUJQBjjAlQlgCMMSZAWQIwxpgAZQnAmEIkIkkico7bcRiTE0sAJkcicpuILPRWYDtE5GcRaed2XCWNqkaq6oYz3U9EKovIHyKyT0QOisgcEWl7wjaPi8hOETksIqNFJOyE9f1EZKOIHBGRVSJy/hnGUFdEVERCzjR+UzJYAjAnEZEngPeA14Ao4GxgGNDDzbiyC4BKKQnoBVQBKgJvApOyyi0iVwD/AToBdYBzgJezdhaR+4H7gKuBSOAaYG8Rxm9KAlW1xZZjC1Aep/Lpmcs2YTgJYrt3eQ8I867rACQA/YHdwA7gXu+6S4CdQHC2Y10HLPO+DsKp1NYD+4CvgLO86+oCilOpbQFmAMHA2zgV20bgEe82IdnKMsobwzbg1axzA/cAs4DBwAHv/l2zxXUW8Im3fAeACdnWXQMsAQ4Cs4GmufxbKXCu9/WnwFDgRyARmAfUz8N3EgR08x6rqvezL4DXsm3TCdiZbfutQKc8fuetgIXAYWAX8I738y3ecyZ5lxjv572AVd5/l1+BOieU9zFgg/d7eQsI8q47F5gOHPKu+9Ltv/dAX1wPwJbitQBXAhlZlegpthkAzAWq4vxCnQ284l3Xwbv/ACAUuAo4ClT0rl8PXJ7tWF8D//G+7uc9bi2cJDMcGOddV9dbuYwBygARwIPASu/2FYHJHJ8AvvMeo4w31vnAA9519wDpQG+cRNLXW9mLd/2PwJfe44YCl3k/b46T2C7x7nc3sAlvAszh3+rEBLDPW+GGAGOBuNN8H8uANO9xRmb7fClwc7b3lb3bVMK5YlPvv+dWnOT2clZFnMM55gB3el9HAq1P+DcPybZtD2Ad0MBbhueB2SeUNx4ngZ4NrAHu964bBzyHk6DCgXZu/70H+uJ6ALYUrwW4He8vyVy2WQ9cle39FcAm7+sOQPIJlcbubJXKq8Bo7+uywBG8vyBxflV2yrZfdW8lHZKtMjon2/qpWRW6933nrAoLp+kqFYjItv5WIN77+h5gXbZ1pb37VvOe14M3aZ1Q9v/iTXbZPluNN0HksP2JCeDjbOuuAv7Ow3cS7o397hO+gyuzvQ/1nqsu0Mb7+keggvezNUDvUxx/Bk6CqHzC5zklgJ+B+7K9D8JJ8HWylTd7XA8BU7yvxwAjgFpu/53b4izWB2BOtA+ofJo29hrA5mzvN3s/O3YMVc3I9v4ozi9LcJourvd2WF4P/KmqWceqA3zn7fQ8iJMQMnEq8yxbT4hj6ynW1cGpFHdkO95wnCuBLDuzXqjqUe/LSKA2sF9VD+RQ9jpA/6xjeo9b+4Ty52ZnttfZ/11OSVVTVHUc8B8Raeb9OAkol22zrNeJOAkYYJCqHlTVTThlv+oUp7gPOB/4W0QWiMg1uYRTB3g/W9n3AwLUzLZN9u8h+9/GU95t54vIChHplct5TBGwBGBONAfnl/O1uWyzHaciyHK297PTUtWVOJVCV+A2nISQZStOO3yFbEu4qm7Lfohsr3fgNP9kqX3CsVJxftVmHaucqjbKQ5hbgbNEpMIp1g08IcbS3gq6sIXidPYCrACaZVvXDNilqvtwrkiymo2ynHLaX1Vdq6q34iTHN4HxIlLmFPtsxbnqyl7+CFWdnW2b7N/Dsb8NVd2pqr1VtQbwADBMRM49fbFNYbEEYI6jqoeAF4ChInKtiJQWkVAR6Soig7ybjQOeF5EqIlLZu/3nZ3CaL3Dap9vj9AFk+QgYKCJ1ALzHz23k0VdAPxGp6a2sn85Wjh3Ab8DbIlJORIJEpL6IXHa64Lz7/oxTQVX0lr+9d/VI4EERuUQcZUTkahEpm/fin56ItBaRdiJSSkQiRORpnCuhed5NxgD3iUhDb9mfx2liyrqa+RJ4SkTKikgtoA/wwynOdYeIVFFVD07HNjhNYHu8/81+H8NHwDMi0si7b3kR6XnCIf/t/XerjfM9f+ndtqc3FnA6kNV7fOMWt9ugbCmeC05fwEKcNvqdOO3JbbzrwoEhOL/Ad3hfh3vXdQASTjjWJqBztvdn4/yP/+MJ2wUBT+D8gk3Eaed+zbuuLie3R4cA7+I0W20EHsfpM8jqyC2P02afgDPyZDFwi3fdPcCsE86fvb3+LOAznFExB4Bvs213JbAAp7LcgZPEyp7i3/HEPoBXs6076d8q27rLcDp6E3GaWaYD7U/Y5glvfIdxRiyFZVtXDojz7r8VJ0nLKc71OU4/TRLOlcW12dYNwEkEB/mnH+dOYLn3vFvx9ulkK2/WKKB9OKO0skZeDcIZjZXk/W77uP13HuhL1v8oxpR4ItIV+EhV65x2Y1MoRESB81R1nduxmNOzJiBTYnmbRq4SkRARqQm8iDP00xiTB5YATEkmOMMXD+A076zCaeowxuSBNQEZY0yAsisAY4wJUCVqQq3KlStr3bp187XvkSNHKFOmjG8DcomVpfjxl3KAlaW4KkhZFi1atFdVq5z4eYlKAHXr1mXhwoX52nfatGl06NDBtwG5xMpS/PhLOcDKUlwVpCwisjmnz60JyBhjApQlAGOMCVCWAIwxJkBZAjDGmABlCcAYYwKUJQBjjAlQlgCMMSZABUQCWLR5Pz9uSHM7DGOMOWMp6Zm8NHEFB1N9/+iEgEgAPyzbwddr0pm/cb/boRhjzBl5b/JaPp29ie1Jvp+3LSASwJNdLqByhPCfb5aRkp7pdjjGGJMnyxMOMXLmBm5uWZuGlYJ9fvyASABlwkK4p1EYG/YeYciUtW6HY4wxp5We6eGpb5ZxVplSPHtVg0I5R0AkAIDGlYPpGV2L4TM28Ne2Q26HY4wxuRoxYwOrdhzmlR6NKV86tFDOETAJAOD5qxtSsXQpnv5mGemZ9ixqY0zxtG53Eu9PXstVTapxZeNqhXaegEoA5UuH8uq1jVix/TDDp693OxxjjDlJpkf59/ilRJQK5qXujQr1XAGVAACubFydq5tW5/0pa1m9M9HtcIwx5jif/LGRxVsO8nL3RlQtG16o5wq4BAAwoHsjyoaH8u/xS8mwpiBjTDGxYU8Sb/26ms4NqtLjohqFfr6ATACVIsMY0KMRyxIOMXLmRrfDMcYYPB7l6W+WERYSxMDrmiAihX7OgEwAAFc3qU7XxtV4d/Ia1u6ypiBjjLs+mb2JBZsO8EK3RkSVK9ymnyyuJgAReVxEVojIXyIyTkSKptTOuXnl2sZEhoXQ/2trCjLGuGf9niQG/fI3HS+syg0tahbZeV1LACJSE3gMaKmqjYFg4JaijKFyZBivXtuYZQmH+O80GxVkjCl6mR7lya+XEh4azBvXF03TTxa3m4BCgAgRCQFKA9uLOoCrmlSnW7MaDJm6lpXbDxf16Y0xAW7EjA0s3nKQAT0aUbWImn6yiKrvJxjK88lF+gEDgWTgN1W9PYdt+gB9AKKioqLj4uLyda6kpCQiIyNzXpemPPdHMuVKCS/GhBMSVHQZOD9yK0tJ4y9l8ZdygJWlKG1L9PDi7GSaVQ3mkYvCcv31X5CyxMbGLlLVlietUFVXFqAiMBWoAoQCE4A7ctsnOjpa8ys+Pj7X9b+v2Kl1nv5BB/2yKt/nKCqnK0tJ4i9l8ZdyqFpZikpqeqZe9f4MbT7gN92TmHLa7QtSFmCh5lCnutkE1BnYqKp7VDUd+BZo41owDaO4qWUt/jttPYs227TRxpjCNWTKWlZsP8zr1zehcmSYKzG4mQC2AK1FpLQ41z2dgFUuxsP/XdOQGhUieOKrpRxJzXAzFGOMH/tzywGGTVvHjdG1uKJR4c31czquJQBVnQeMB/4ElntjGeFWPABlw0N5u2cztuw/ysCfXM1Fxhg/dTQtgye+XEL18hG82K2hq7G4OgpIVV9U1QtVtbGq3qmqqW7GA3DJOZXofek5fDFvC1P/3uV2OMYYPzPwx1Vs3n+Ut29qRtnwwpnmOa/cHgZaLPXvcj4XVivLU+OXsTfJ9ZxkjPETU1btYuy8LfS+9Bxan1PJ7XAsAeQkLCSY929pzuGUDJ4evyxr1JIxxuTbnsRUnhq/jIbVy9G/y/luhwNYAjilC6qV5T9XXsiUv3czdt4Wt8MxxpRgqspT45eSlJrB+7dcRFiI75/vmx+WAHJxT5u6XHpeZV79cSXrdie5HY4xpoT6fO5m4lfv4ZmuF3JeVFm3wznGEkAugoKEwT2bEREaTL+4xaRmZLodkjGmhFm9M5FXf1zFZedX4e42dd0O5ziWAE4jqlw4b93YjBXbD/PWL6vdDscYU4KkpGfy2LjFlA0PZXDPZkU60VteWALIg84No7grpg4fz9rItNW73Q7HGFNCvP7TKlbvSmRwz6ZUKevO3b65sQSQR89e1YALosry5NdL2ZNoQ0ONMbmbvHIXn83ZzH3t6tHhgqpuh5MjSwB5FB4azJBbm5OYksETXy3B47GhocaYnO04lMyT45fSsHo5nrryArfDOSVLAGfggmpleaFbQ2au3cvwGRvcDscYUwxlZHroN24JaRkePrytebEZ8pkTSwBn6LZWZ3N10+oM/m01izYfcDscY0wxM2TKWuZv2s/A6xpzTpXi+ywCsARwxkSE169vQs0KETw2bjGHjqa7HZIxppiYvW4vH8Q7s3xe17yW2+GcliWAfCgXHsoHtzZnd2IKT45falNFGGPYnZjCY3FLOKdyGQb0aOR2OHliCSCfmtWuwDNdG/D7yl2MmrXR7XCMMS7K9Cj9xi0hKTWdYbdHU7pUiNsh5YklgAK4t21drmxUjTd+/tueImZMAHt/8hrmbNjHKz0ac0G14jPVw+lYAigAEWFQz6bUqBDBI18sZv+RNLdDMsYUsRlr9vBB/Dp6RteiZ8vabodzRiwBFFC58FCG3d6CfUlp/OvLJWTa/QHGBIztB5P515dLOL9qWQb0aOx2OGfMEoAPNK5Znpe6N2LGmj0MmbLW7XCMMUUgLcPDQ2P/JC3Dw7A7WhBRqviO9z8VVxOAiFQQkfEi8reIrBKRGDfjKYhbW9Xmhha1GDJ1LfE2X5Axfu/VH1eyZOtB3rqxKfWL+Xj/U3H7CuB94BdVvRBoBpTYJ7GLCK9e25gLosryr7glbN1/1O2QjDGFZMLibYyZs5nel9aja5PqhX/Co4UzyMS1BCAi5YH2wCgAVU1T1YNuxeMLEaWC+eiOaDyqPPj5IlLS7fkBxvibVTsO859vl9Gq7lk8deWFhXuyo/vh9xfgnYaUP7jS54cXt25iEpGLgBHASpxf/4uAfqp65ITt+gB9AKKioqLj4uLydb6kpCQiI4vmMm3J7gze+zOVtjVCuL9JKZ/PAV6UZSls/lIWfykHWFlyPV6a8vKcZNI98FKbcCqEFc5v6OCMo9RKmEjtrd8TnJnM7qrt+SvqWoIrnZOv48XGxi5S1ZYnrVBVVxagJZABXOJ9/z7wSm77REdHa37Fx8fne9/8ePf31Vrn6R/00z82+vzYRV2WwuQvZfGXcqhaWU4lI9Ojd42ap+c++6Mu2rzfZ8c9TuoR1Vnvqb5RV/XFcqrjblPduUJVC1YWYKHmUKe62QeQACSo6jzv+/FACxfj8anHOp5H5wZVeeWHlXw7bREPPfQQ5cqVIygoiHLlyvHQQw+xfv16t8M0xuTRO7+vZvqaPbzcvTEtzq7o24NnpMH8kTCkudPkU6M59I6HW8ZCVEPfnisb1xKAqu4EtopI1mTZnXCag/xCUJDwzs0XUWb3cnp2acvIjz8mMTERVSUxMZGPP/6Ypk2b8vPPP7sdqjHmNH5ctoOh8eu55eLa3HbJ2b47cGYGLP4cPoiGn56Es86Be3+GO7+FmoX/e9jtCSseBcaKSClgA3Cvy/H41J5tW1j7xQA86al4TliXnp5Oeno6N954I8uWLaN+/fquxGiMyd2K7Yd48uulRNepyMu+muTN44GV30H867BvrfOLv9u7UL8TFOFzg11NAKq6BKcvwC+9/fbbZGT8M110a6ADMA2Y6/0sPT2dd999lw8//LDI4zPG5G5fUip9xiyifEQo/72jRcEf7qIKa36BqQNh13Ko0gBu/hwuvKZIK/4sbt8H4Nc+//xz0tOdBNAamA68CkzxvgcnAfzvf/9zJ0BjzCll3em7NymVEXdFU7VseMEOuGE6jLocxt0CaUlw/cfQ9w9o0M2Vyh/cbwLya0lJScded8D5xw4CSnnfz81hO2OM+1SVFyf+xbyN+3n35mY0rVUh/wfbOh+mDIBNM6FcTej2Plx0OwSH+i7gfLIEUIgiIyNJTEwEnGafFCACCAaica4C5nq3M8YUH6P/2MS4+Vt5OLZ+/p/stWOp09Sz9lcoUwWufAOi74XQAl5J+JA1ARWiO+64g9BQJ8vPxRnm9BmgwA3AVKBdcDB33nmnazEaY44X//duBv64kisaRdH/8gtOv8OJ9qyBr+6G4e1h61zo9AI8tgRa9y1WlT9YAihU/fv3P5YAwEkCqwEPIEA4cKUIjz/+uDsBGmOOs3pnIo+OW8yF1crx7s0XERR0Bm3zBzbBd31h2CWwbjK0/zf0WwaX9oew4nmVbwmgENWvX5/x48dTunTpY4lgGpCGcws0wG0VqlBvXBzMmeNSlMYYcJ7p2+vTBZQJC2bUPS3z/ljHwzvghyfgg5bw1zfQ+iHotxQ6Pg8RBeg7KAKWAApZ165dWbZsGX369KFcuXLMDwqie+nS/NS6NSvadKTe3h3I/z0PnTpZEjDGJclpmdz/2UIOHE1j1N0XU718xOl3OrIXfn0OhlwEf34GLe6EfkvgioFQpnLhB+0DlgCKQP369fnwww85dOgQmZmZTD5yhO5z5tDo6k4oTnOQJifDSy9ZEjCmiGV6lH99uZi/th1iyC3NaVyzfO47pBxyOnffbwZzh0Gj6+CRhXDNu1CuRtEE7SM2CshFEhuLRkQ4lT+gv/+OzJwJU6ZATIl9No4xJYaq8soPK/l1xS5euKYhnRtGnXrjtCMwbzj88T6kHISGPSD2OaiSj47iYsKuANwUE4NMmUJmh1gARBVNTYVp09yNy5gAMWrWRj6dvYn72tWjV7t6OW+UkQpzP4L3L4IpL0PtVtBnOtw0pkRX/mAJwH0xMYS8NhDCw1EAj4cjCxdbU5AxhWzS0u28+uMqrmpSjeeuanDyBpkZsOgzGNICfnnaqex7/Qq3fw01Lir6gAuBJYDiICYGmTqVxB43AFD626/Rjh0tCRhTSOZu2Ef/r5Zycd2KvHPTCcM9PR5Y9jUMvRgmPQZlo+DOCXD3JDi79akPWgJZAiguYmIod0k0BAUhACkpZDz7nCUBY3xs5fbD9P5sIWdXKs2IO1sSHuqd4E0VVv0AH7WDb++HkAi4ZRzcPwXqx7o2X09hsk7g4qRDByQsDE1JAVWCp8WjnToh1ilsjE9s3X+Uuz+ZT2R4CGN6taJimVJOxb8hHqa8Atv/hLPqww2joNH1EOTfv5H9u3QlTUwMTJmCXH45iDhXAsnJeKZOdTsyY0q8vUmp3DV6PmkZHj7r1YoaFSJg8xz49Gr433VwZA90/xAeng9NbvT7yh/sCqD4iYmBl15CZs7Ek5JCkCobv5xEPREkNtauBIzJh8Mp6dw9ej47DiUz9v5LOD9zPXz+Kqz7HcpUha6DIPoeCAlzO9QiZQmgOMq6EoiPZ33cROovn4cunw8R4c49AsaYPEtOy+S+TxewemciY68tT/TcfrBqIoRXgM4vQas+UKqM22G6wvUEICLBwEJgm6pe43Y8xUZMDBITQz1VdPk8BEWTk5GXXqJc9+7QoYPbERpT7GV4lL5jF7Fr89/EnxtP7Z8mQalIuOw/EPMQhJ/mrl8/53oCAPoBq4BybgdSHAV17HjsbmEB9LffaTZ9OrRoYc1BxuQiI9PDV4t3cPmB8dwaPp2gHaHQ5lFo9ziUPsvt8IoFV3s5RKQWcDXwsZtxFGveu4U9nTp75w1SJDXV5g0yJheexN3MGtqHDw4/xi2h0wlqea8zUVuXV6zyz0ZU1b2Ti4wHXgfKAk/m1AQkIn2APgBRUVHRcXFx+TpXUlJSiX7yVrkVK2j2RH+C0lKPfeYJC2Pp229zuFEjFyMrmJL+vWTxl3JAyS5LSHoStbZOoNqWiYRqGvMj2iPNbiclIpc5fkqIgnwvsbGxi1S15UkrVNWVBbgGGOZ93QH44XT7REdHa37Fx8fne99iY/ZsTY/tqB5n5LJ6RFS7dFGdPdvtyPLNL74X9Z9yqJbQsqQkqk4fpJ7Xa6u+WE4nPt9FR373S8ksyykUpCzAQs2hTnWzCagt0F1ENgFxQEcR+dzFeIq/mBhCBr6Kp1QpZ94gVfT33+1ZAiZwpafAnKHO1MxTX2VNWBO6pr7Ostbvcl+PLm5HV+y5lgBU9RlVraWqdYFbgKmqeodb8ZQYMTEsfecdMi7rAHhnEE1JsT4BE1gy02HhaBjSHH59Fo1qxMcXjOCKXQ/Rpm0Hnr2qAeKHUzf4WnEYBWTO0OFGjQht0QLt2PHYtBH2LAETEDyZsPxrmPa68wzeWq3Q6z7ilRVVGP3HRu5tW5fnr7bKP6+Kxb3OqjpN7R6AM+OdQfS4K4FkuxIwfkoVVn4P/20D3z0AYWXhtq/w3PsrLyyrxOg/NnJPm7q8cE1Dq/zPQLFIACafYmIIff21f54lgPUJGD+jCmsnw4gO8NVdoB7o+Sn0mYHn3C48O+Ev/jd3M33an8OL3azyP1OWAEo675XAcU8VS06BMWNcDsyYAtr0B3zSFcbeAMn74dr/Qt850Og6MhSeHL+UuAVbeST2XJ7peqFV/vlgfQD+wPtUMe3QAU1LAxTPiBFOdr/rLusTMCXLtkUw9VVYPxUiq8HVb0PzuyCkFACpGZn0G7eEX1bspP/l5/Nop/NcDrjksisAfxETg/TqdWwa6SCPB/3oI2sOMiXHrpUQdzuM7Ajbl8Dlrzh37158/7HKPzktk95jFvHLip28cE1Dq/wLyBKAP7nrLiQ8HBXxThsBmpyMvviiJQFTfO1bD9/c73TwbpwBHZ6Ffkuh7WMQGnFss0NH07lz1Dxmrt3DoBuanvoh7ibPrAnIn2RNIz1mDPrJJ3hSUwkC9Pff0Vmz7Mlipng5uBVmDILFYyG4FLTt5yw5zNWz63AKd42az4a9SXx4awuublrdhYD9j10B+JuYGPjvf5H4eKRLl2NXAiQnk/n883YlYNyXtBt+fho+aAFL45wmnn5L4fKXc6z8N+xJ4vphs0k4cJRP7mlllb8P2RWAv4qJQV56CWbORJOTAQiaOhXt0MHpK7DOYVPUju6H2UNg3nDISIWLboPLnoYKtU+5y+ItB7jvs4UAjOvTmqa1KhRVtAHBrgD8WVaTUJcu/zxjOC0NHT7cOodN0UlNhOmDnPl6Zr0HF1zlPHe3x4e5Vv6/r9zFrSPnEhkWwvgHY6zyLwR2BeDvsj1jOGvaiKz5g2TMGLsKMIUnPRkWfAyz3oWj++CCq6HjcxB1+unL/zdnEy9OXEGTWhUYdXdLKkcG1rN6i4olgECQrXPYM/JjyMwAVTwjR9q9Asb3MtJg8RiYMRgSd8A5sdDx/6BW9Gl3zfQoA39cxeg/NtK5QVWG3Nqc0qWsmios9i8bKGJiICbGGRU0fDiiCpmZzr0Cn31mI4RMwXkyYdmXMO0NOLgZareGGz6Guu3ytHtSagb9xi1myt+76dW2Hs9d3YDgILu7tzBZH0CgyXavAGQfIfR/1idg8sfjgRXfwbDWMKEvRFSE27+BXr/kufJPOHCUG/87m2lr9vDKtY15oVtDq/yLgCWAQJPVHPTAAxAWhgcnCQRNnYK2bw8jRrgdoSkpVGHNrzDiMvj6HpAguOl/0GcanNcZ8jg3z/yN++n+4R9sO5jM6Hsu5s7WdQozapONJYBAlO1egaDs9wpkZKB9+0LfvnY1YHK3cQaM6gJf3ASph+G6EdB3NjTsnueKH+CLeVu4beRcKpQO5fuH23LZ+VUKMWhzIksAgSxrhFBIyD9JIGsOIbsaMDnZugA+6w6fdYPD2+Ca9+CRhdDsZggKzvNhUjMyeebbZTz73XLanluZ7x5qyzlVSuaD6Esy6wQOdDExMHQo8sgjaIYzOkjAef3ww0iTJtY5bGDncmeGzjW/QOnKcMXr0LIXhIaf8aG2H0ym7+eLWJpwiIdj6/PE5RdYe79LTnsFICJRIjJKRH72vm8oIvcV9MQiUltE4kVkpYisEJF+BT2myac+fWD6dKdfIDj4uCah1CeetOagQLZ3rdO+/1E72DLHGc7ZbynEPJSvyn/Gmj1c88Es1u85wog7o/n3FRda5e+ivDQBfQr8CtTwvl8D/MsH584A+qtqQ6A18LCINPTBcU1+ZPULDBuGhIZ6nzAGpebOxtPuUmsOCjQHNsOEh2FoK1jzG1z6JPRbBu2fhLAzb6rJ9Chv/7aauz+ZT5XIML5/pC1dGlUrhMDNmchLE1BlVf1KRJ4BUNUMEcks6IlVdQeww/s6UURWATWBlQU9timAPn2gSRPkpZfQ335HUMSTieeBB9AFCwnuda81CfmzxJ2ct2Y4zPjdGdVzSV9o9zhE5r9zdtfhFP4Vt4Q5G/bRM7oWA3o0JqJU3vsLTOERVc19A5FpwA3A76raQkRaA2+q6mU+C0KkLjADaKyqh09Y1wfoAxAVFRUdFxeXr3MkJSURGekfnUxFUZZyK1ZwUb9+SGYmWRfoCmhQMGv/1Y8d3br55Dz+8r2U9HKEph2m9tZvqLntJ0Qz2VmtM5vr3ERqeOUCHXfJ7gw+Xp5KmgfubFCKS2uF+ijivCnp30t2BSlLbGzsIlVtedIKVc11ATvnfsAAAB/jSURBVFoAfwCHvP9dAzQ93X55XYBIYBFw/em2jY6O1vyKj4/P977FTZGVZfhw1dBQVRH1OKO+1QOaKaL6wAOqs2cX+BT+8r2U2HIkH1Kd+prqwJqqL5ZX/aa3zvnpi4IfNi1DX/z+L63z9A/a9b0ZunZXog+CPXMl9nvJQUHKAizUHOrU0zYBqeqfInIZcAFO3+BqVU3PVxo6gYiEAt8AY1X1W18c0/iQtzmIMWOQkSNR79WAqDozio4ahQwd6mxnSpa0ozB/BPzxHiQfgAbdIPY5qNqAlGnTCnTov7Yd4vEvl7B2dxK92tbj6a4XEBZiTT7F0WkTgIjcdcJHLUQEVR1TkBOLiACjgFWq+k5BjmUKkXcOIZo3z3GoqOfBB52RBJYESoaMVFj0GcwcDEm74NzO0PF5qNG84IfO9PDR9PW8N3ktlSJLMaZXK9rbjV3FWl46gS/O9joc6AT8CRQoAQBtgTuB5SKyxPvZs6r6UwGPawpDVufwmDFwwtWA54EHyJwwgdA6dWxm0eIqMwOWjoPpb8KhrVCnLfT8DOr45rtavTORp8YvZWnCIa5pWp1Xr21MhdKlfHJsU3jy0gT0aPb3IlIByF9P7PHHnQXYAOCSJPvVQN++qMfjJAEg5OefnfsHRo6EYcPsiqC48Hhgxbcw7XXYtw5qtIBu70P9jmc0ZcOppGd6GD59PUOmrCMyPIQPbm1Ot2Y1Tr+jKRbycyfwEaCerwMxJYi3cpdHHoFsTUIAmpkJDz2ELF5sVwNuUnXu2p36Kuz6C6o2hJvHwoVX+6TiB+dxjc98u5y/dyZyddPqDOjeiEr24JYSJS99AJPg2H1BQUBD4KvCDMqUANk7iEeNQtOdcQGCkwT0o49g5EikWzd46ilLBEVFFTZMcyr+bQvhrHPg+o+h8fVnNFdPbg6npDP419X8b+5mosqGM/zOaK6wm7pKpLxcAQzO9joD2KyqCYUUjylJspqE7roLGTQIJk1CPZ5/OokzM9EJE5BJk6xZqChsmQdTX4FNM6FcLeg2xHnwerBvxt6rKhOWbGPgj3+z70gqd8fUpX+X8ykbXrRj+43v5KUPYHpRBGJKsJgY+O47mDMHGTMGzdZJDN5moQcfRH7+2a4GCsOOpc4v/rW/QZkqcOWbEH1PvubqOZWV2w/z0sQVzN+0n2a1yjPq7pY0q20PaS/pTpkARCSRf5p+jlsFqKqWK7SoTMnkvSKQ5s3hoYecip9jfzDohAnw44/IffdZ/4Av7FkN8QNh5fcQXh46vQiXPAClyvjsFLsTU3j71zV8tWgrFSJCeeP6JtzUsjZBNoGbXzhlAlDVskUZiPEjWUNGBw2C779Hs3cSp6cf1z9QrnNn6NDBzWhLngObYNqbsCwOQktD+6cg5mGI8N0v8qNpGYyetZH/TltPWqaHXm3r8VjH8yhf2pp7/EmeRwGJSFWc+wAAUNUthRKR8Q9ZzUIjRiDZrgaA4/oHLvr+e5g82ZqG8uLwDpjxFvw5xunQbf2QM1FbmYLN15NdeqaHKVvSeXLWNPYmpdKlYRTPXNWAepV9d1Vhio+8jALqDryNMx30bqAOsApoVLihGb+Q/QaynTvRSZOO6x/A2zQkEydC9+6WCHJyZB/MegcWfAyeDGhxN7T/N5Sr7rNTpGd6+G7xNj6cuo4t+9NoVfcsht/Zgug6Z/nsHKb4ycsVwCs48/VPVtXmIhIL3FG4YRm/kjVaCJARI07qHwCc0UOWCI6XfBDmDIW5wyD9KDS7FS57CirW9dkp0jI8TFiyjaHx69i87yhNapbnXy3C6NezNeKj+wVM8ZWXB8Kkq+o+IEhEglQ1Hjh5WlFj8qJPH5g5E7n2WiQoyJliGo7dUaweDzphAtq2LVx3XWA+jSztCMx8G95vBjMGOfP1PDQPrh3ms8o/q42/w1vxPDV+GWXDQ/j4rpZMfKQtF1UNsco/QOTlCuCgiEQCM4GxIrIb525gY/In27DRvf/+N1XmzHGuAMh2RZA1amjiRKRdO2jY0P9HDqWnwKJPnMr/yB447wro+BxUb+azU+w8lMKYOZv4Yv4WDh5Np1W9s3jt+iZcdn4Vq/QDUF4SQDxQHuiH0/RTHhhQmEGZABETw4pXX6VDWNhJI4ayNw3pjBkwYwby0UfQvj288YZ/JYLMdFgyFqYPgsPboO6lzrQNZ1/ik8OrKgs3H+DzuZv5cdkOPKp0aViN3u3rWRt/gMtLAggBfgP2A18CX3qbhIzxjewjhvr2BY/n2A0o2X+TKsCMGdCmDVK3Llx0UcnuK/Bkwl/fQPxrcGAj1GzpNPOc08Enhz94NI2JS7czdu4WVu9KpGxYCHfF1OXetnWpfVZpn5zDlGx5uRP4ZeBlEWkK3AxMF5EEVe1c6NGZwJJ9fqGVK2HWrByTgQK6aRNs2oRMmAAlLRmowt8/wNSBsGcVRDWGW+Pg/CsLPFFbeqaHWev2Mn5RAr+v2EVapocmNcvz5g1N6NasBqVL5Wf+R+OvzuSvYTewE9gHVC2ccEzAyzZiiDlznGQwdy4sWXLqq4LsyaBaNWjdungmA1VYP8WZtmH7Yqh0Ltw4GhpeB0F5GY+Rs0yPsmDTfiYt3c7Pf+1k/5E0KpYO5bZLzqZny1o0qlHeh4Uw/iQv9wE8BNwEVAG+Bnqr6srCDsyY45LBiBHIe+/BqlUAp04GO3fChAnOkNKzznISQr9+7k9Et3k2THkFtsyG8mdDj6HQ9BYIzt8v8uS0TOZs2Muvf+1i8qpd7DuSRnhoEJ0bRNGtWQ06XFDFHsNoTisvf321gX+p6pLTbmlMYenTx1nmzIFBg5znDWzeDJw6Gej+/bB/PzzwAPLkk1CmDISH+7y56NChQ/zf//0fP/zwA+XLn/Bre9ufzi/+9VMgMgquGgwt7oKQM5s33+NRVu9KZPb6fUxbvZt5G/eTluEhMiyE2Aur0qVhFB0vrEqZMGviMXmXlz6AZ4oiEGPyJKvDGP5JBnPnOr/8yTkZAGhiIiQmOm82bXKuEEp7O0JVISoKnnkmX1cKEydOZNasWUyaNIk77vDeI7l7lVPx//0DRFSEywfAxb2hVN46X1PSM1mx/RCLtxxk4aYDzNu4jwNHnWcu1K9ShjsuqcNlF1Sh9Tln2S99k2+u/lwQkSuB94Fg4GNVfcPNeEwJkz0ZjBgB772H7Nrl/Orn+GRwUkIA9OjRfz7YtMm5UnjwQac93vtcg2MiIuDRR+HNN08KY/To0cf+e0fXGOe5u8u+glKR0OEZZ86e8Jwnz1VVdiemsn53Eqt3JbJy+2FWbD/M2t2JpGc6569VMYJODaJofU4lYupXomaFiDP7dzLmFFxLACISDAwFLgcSgAUiMtH6F0y+ZDURwT/JICHhn1/95J4Qjm2jCtkmrjsmORkGDUIHDaILMCXbqqxHn/8RH49UPvfY5x0ua8rYh/px+GA6+4/sZU9iKnsSU9lxKIWEA0dJOJDMln1HSUzNOLZP5chSNKxRnvbnV6H52RVofnYFqpb13bz+xmTn5hVAK2Cdqm4AEJE4oAdgCcAUTA7JgAMHkLS0Y1cHWU5sMjrdIEwBngPmAFnXD2kn/BdAQsJYVaMrbd6YetIxwkODqF2xNLUqRtCyTkXqV43k3CqRnFs1kiplw+yOXFNkRDWnZ74UwYlFbgSuVNX7ve/vBC5R1UdO2K4P0AcgKioqOi4uLl/nS0pKIjIysmBBFxNWlvwrt2IF9YYPp8zGjYjHg6SlEZyRcdr9TqyS44Fr+CcJZBdSKozuj77CuY0uonQIlA4VyoQKFcKE8mFC6RCKdSVvf1/FU0HKEhsbu0hVT57DTVVdWYAbcdr9s97fCXyY2z7R0dGaX/Hx8fnet7ixsvjY7Nmq7durRkSoBgc7i4iq0wtwymUSaDjH5rNTvO8nTZrkdokKpFh8Jz5iZXEACzWHOjX/d58U3DacIaZZank/M6ZoxcTA9Olw9ChkZDhLViewqjNkNOTk1tKDOG2oQUCE978hkZEcPHiwaOM3Jp/cTAALgPNEpJ6IlAJuASa6GI8xJ8tIgxvPhzfOhRfLwafdYMt8UGVUhw4cDQqiWfPmDHjrLZo1b87Ro0ePjQoyprhzLQGoagbwCPArzhPGvlLVFW7FY8xxPJmwZBx82BJ+fALK14K7JsLdE6H2xQCUL1+et956i4ULF9KyZUsWLFjAoEGDKFcu5yGfxhQ3rt4HoKo/AT+5GYMxx/F4YNVEZ4bOvauhWlO47Ss4r8tJE7VNmDDhuPfBwcH079+f/v37F2XExuSb3TduDDht/Wt/h6mvwM5lUPl86PkZNOheoInajCnOLAEYs3GmM23D1rlQoQ5c+xE0vQmCbIoF498sAZjAlbAIpg6ADdOgbHW4+m1ofheElDrtrsb4A0sAJvDs/Mtp41/9I5SuBF1ehYvvh1CbY8cEFksAJnDsW+9U/H99A2FlIfY5aN3XeW1MALIEYPzfwS3ODJ1Lxjnz8Ld7HNo8CqXtgegmsFkCMP4rcRfMHAyLPnXet+oDlz4BkfZEU2PAEoDxR0f3wx/vwbwRkJkGze+Ay55ybuYyxhxjCcD4j5TDMHcYzBkKqYnQpCd0+A9Uqu92ZMYUS5YATMmXngzzR8KsdyF5P1x4jdPBG9XQ7ciMKdYsAZiSKyMN/vwMZgyGpJ1QvxN0fA5qRrsdmTElgiUAU/JkZsCyL2H6G84In7PbwI2joW5btyMzpkSxBGBKDo8HVk6Aaa/D3jVQ/SK4+l04t9NJE7UZY07PEoAp/lRhza/OfD27lkOVBnDz505bv1X8xuSbJQBTvG2Y7szQmbAAKtaF60ZAkxttojZjfMASgCmeti5wJmrbOAPK1YRr3nPG8weHuh2ZMX7DEoApXnYsg/iBsOYXKF0ZrngdWvaC0HC3IzPG77iSAETkLaAbkAasB+5VVXuSdiDbu9ap+Fd8B+HloeP/wSUPQlik25EZ47fcugL4HXhGVTNE5E3gGeBpl2IxLgpP3gUTHoKl4yAkAi590pmoLaKC26EZ4/dcSQCq+lu2t3OBG92Iw7jo8A6YOZhWCz91OnQv6evM0hlZxe3IjAkYoqruBiAyCfhSVT8/xfo+QB+AqKio6Li4uHydJykpichI/2hOKMllCU07TO2t31Bz20+IZrK18mVsr387qeGV3Q6tQEryd3IiK0vxVJCyxMbGLlLVlietUNVCWYDJwF85LD2ybfMc8B3eRHS6JTo6WvMrPj4+3/sWNyWyLMkHVacOVB1YU/WlCqrfPqC6b0PJLEsO/KUcqlaW4qogZQEWag51aqE1Aalq59zWi8g9wDVAJ2+Axh+lHYX5I5zpmZMPQMMe0OFZqHqhd4PNroZnTCBzaxTQlcBTwGWqetSNGEwhy0iFRZ85D2RJ2gXndXFm6KxxkduRGWO83BoF9CEQBvwuzq38c1X1QZdiMb6UmQFLv4Dpg+DQVqjTDm4aA2e3djsyY8wJ3BoFdK4b5zWFyOOBFd86D13fv96Zkrn7EDgn1ubrMaaYsjuBTcGowuqfYOpA2L0CqjaCW8bBBV2t4jemmLMEYPJHFTbEOzN0blsEZ9WHG0ZBo+shKMjt6IwxeWAJwJy5LXNhyiuweRaUqwXdP4Bmt0Gw/TkZU5LY/7Em77YvcX7xr/sdylSFroMg+h4ICXM7MmNMPlgCMKe3+29norZVEyGiInR+GVr1gVKl3Y7MGFMAlgDMqe3fCNPegOVfQWhpuOxpiHnYma3TGFPiWQIwJzu83RnHv/h/EBTiVPptH4cyldyOzBjjQ5YAzD+O7IVZ78L8kaAep33/0iehXHW3IzPGFAJLAAaSD8LsD2DufyEjGZrd6jT3VKzjdmTGmEJkCSCQpSbBvI9g9hBIOQSNrnMmaqtyvtuRGWOKgCWAQJSeAgtHw6x34MgeOO8K6Pg8VG/qdmTGmCJkCSCQZKbD4s9hxltweBvUaw8dv4DardyOzBjjAksAgcCTCcvHw7TX4cBGqHUxXDsMzungdmTGGBdZAvBnqrBqkjND555VENUEbv0Szr/CJmozxlgC8EuqsG4KxL8K2xdDpfPgxk+g4bU2UZsx5hhLAP5m82xnorYts6H82dBjGDS92SZqM8acxGoFf7HtT5j6CqyfCpHV4KrB0OJuCCnldmTGmGLK1QQgIv2BwUAVVd3rZiwl1q6VzkRtf/8AEWfB5a9Aq94QGuF2ZMaYYs61BCAitYEuwBa3YijR9q33TtT2NYSVdR643rqv89oYY/LAzSuAd4GngO9djKHkOZTA+auHwvQpzjz87f4FbR6D0me5HZkxxVJ6ejoJCQmkpKS4HUqBlC9fnlWrVuW6TXh4OLVq1SI0NDRPx3QlAYhID2Cbqi4VG46YN0m7YebbsHA01TweuPh+uLQ/lI1yOzJjirWEhATKli1L3bp1Kcn1TWJiImXLnvoKX1XZt28fCQkJ1KtXL0/HFFX1VXzHH1hkMlAth1XPAc8CXVT1kIhsAlqeqg9ARPoAfQCioqKi4+Li8hVPUlISkZGR+drXTSHpidTe+h21En4gyJPOzmodWVGlGyGV6rodmk+U1O/lRP5SDvC/stSsWZP69euX6MofIDMzk+Dg4Fy3UVXWr1/PoUOHjvs8NjZ2kaq2zHGHolyAJsBuYJN3ycDpB6h2un2jo6M1v+Lj4/O9rytSDqtOe1P1tVqqL5ZX/bqX6p61qloCy5ILfymLv5RD1f/KsnLlSrfD8InDhw/nabucygss1Bzq1CJvAlLV5UDVrPenuwIIOOnJsOBjZ17+o/vggquh43MQ1cjtyIwxfsZuCy0uMtKcin9Ic/jteajWFO6fCrd+YZW/MSXckCFDaNCgAbfffjsTJkxgwIABuW7/5JNPMnXq1EKPy/UbwVS1rtsxuMqTCcu+dIZ0HtwMtVvD9SOh3qVuR2aM8ZFhw4YxefJkatWqRZs2bZg4cWKu2z/66KP07t2bjh07FmpcrieAgOXxwKrvnYna9q5xfvHfPh7O7WwTtRlTSF6etIKV2w/79JgNa5TjxW6nvkp/8MEH2bBhA127duWOO+4gLCyMypUrA9CjRw9uuOEG7rrrLoYPH86MGTMYO3YsderUYd++fezcuZNq1XIaS+MblgCKmiqs/Q2mvgo7l0HlC+CmMdCgu1X8xvihjz76iF9++YX4+HgmTZpEixYtjq0bMWIEbdu2pV69erz99tvMnTv32LoWLVrwxx9/cMMNNxRabJYAitLGmc58PVvnQcW6cO1H0PQmCMp9aJcxxjdy+6VeFHbs2EGVKlWOvY+KimLAgAHExsby3XffcdZZ/9zQWbVqVbZv316o8VgCKAoJC52Kf8M0KFsDrnkXmt8JwXm7W88Y4x8iIiJOGqO/fPlyKlWqdFJln5KSQkRE4c7pZaOACtPOv2DcrfBxJ9i5HLoMhMf+hJa9rPI3JgA1aNCAdevWHXs/f/58fv75ZxYvXszgwYPZuHHjsXVr1qyhcePGhRqPJYDCsHcdjO8FH7WDTX9A7PPQbym0ecRm6TQmgLVv357FixejqqSmptK7d29Gjx5NjRo1ePvtt+nVqxeqSnp6OuvWraNly5Nv3vUlawLypYNbYPqbsGQchITDpU9Am0choqLbkRljXLRp06Zjrzt37syUKVPo3LkzS5cuPfZ59+7d6d69OwA//PADN954IyEhhVtFWwLwhcRdMHMwLPzEGcnTqo9T+UdWPf2+xpiA8uyzzzJv3rxct8nIyKB///6FHoslgII4uh/+eA/mjYDMNGh+B1z2FJSv5XZkxphiKioq6tgv/VPp2bNnkcRiCSA/Ug7D3GEwZyikJkKTntDhP1CpvtuRGWNMnlkCOBNpR2HBSJj1HiTvhwbdoMOzENXQ7ciMMeaMWQLIi4w0+PMzmDEYknY60zV0fB5qNHc7MmOMyTdLALnJzIBlcTDtTTi0Beq0hZ6fQJ02bkdmjDEFZvcB5MTjgb++hWGt4fuHoUwluONbuOdHq/yNMQX20ksvMXjwYP7++28uuugimjdvzvr160lOTuayyy4jMzPzlPsuX76ce+65xydxWALIThVW/wzD28P4e527dW8eC73j4dxONlmbMcanJkyYwI033sjixYupX78+o0eP5vrrr8/10Y9NmjQhISGBLVu2FPj81gSUZcM0Z4bOhAVw1jnOnPyNb7CJ2ozxJz//x5mWxZeqNYGub5x2s4EDB/LZZ59RtWpVateuTYMGDRg2bBjBwcFMmTKF+Ph4xo4dyxdffAHAd999x4cffsjkyZPZuXMnl156KbNmzaJatWp069aNuLg4nnrqqQKFblcAW+bBp9fAmB5weDt0GwIPz7dZOo0xPrNo0SLi4uJYsmQJP/30EwsWLKB06dI8+OCDPP7448THx5OWlsaGDRuoW7cuANdddx3Vq1dn6NCh9O7dm2efffbYswFatmzJzJkzCxxX4F4B7FgKUwfC2l+hTBW48g2IvhdCw92OzBhTWPLwS70wzJw5k+uuu47SpUsD5Hgj2N69e6lQocJxn33wwQc0btyY1q1bH3dzmK+minYtAYjIo8DDQCbwo6oW7Fomr/ashviBsPJ7CC8PnV6ASx6EUmWK5PTGGJOTiIgIUlJSjvssISGBoKAgdu3ahcfjOfa5r6aKdqUJSERigR5AM1VtBAwu7HOGJ++C7/o6I3vWTYH2/4Z+y+DS/lb5G2MKVfv27ZkwYQLJyckkJiYyadKkk7apWLEimZmZx5JARkYGvXr1Yty4cTRo0IAPP/zw2La+mirarSuAvsAbqpoKoKq7C/Vs09+i1fw3IDgEWj8E7R6HMpUL9ZTGGJOlRYsW3HzzzTRr1oyqVaty8cUX57hdly5dmDVrFp07d+a1117j0ksvpV27djRr1ozo6Giuv/56GjRoQHx8PFdffXWB4xJVLfBBzvikIkuA74ErgRTgSVVdcIpt+wB9AKKioqLj4uLO+HxRO6cRsW8528+9jbSwSvkPvJhISkoiMjLS7TB8wl/K4i/lAP8rS82aNTn33HPdDiVPlixZwtChQxk5cuRJ6zIzMwkODiY1NZWuXbvy22+/5Thd9Lp160566lhsbOwiVT354QKqWigLMBn4K4elh/e/HwACtAI24k1GuS3R0dGaX/Hx8fnet7ixshQ//lIOVf8ry8qVK90O44yMGjVKMzIyTvr88OHDqqq6Zs2aXL+jnMoLLNQc6tRCawJS1c6nWicifYFvvYHNFxEPUBnYU1jxGGNMSdCrV69c15933nmcd955PjmXW/cBTABiAUTkfKAUsNelWIwxfk5daOp2w5mW060EMBo4R0T+AuKAuzVQviFjTJEKDw9n3759fp8EVJV9+/YRHp73e5lcGQWkqmnAHW6c2xgTWGrVqkVCQgJ79pTsFuaUlJTTVu7h4eHUqpX3JxIG7p3AxpiAEBoaSr169dwOo8CmTZtG8+a+fQaJzQVkjDEByhKAMcYEKEsAxhgToFy5Ezi/RGQPsDmfu1fGf4aaWlmKH38pB1hZiquClKWOqlY58cMSlQAKQkQWak63QpdAVpbix1/KAVaW4qowymJNQMYYE6AsARhjTIAKpAQwwu0AfMjKUvz4SznAylJc+bwsAdMHYIwx5niBdAVgjDEmG0sAxhgToAIqAYjIKyKyTESWiMhvIlLD7ZjyS0TeEpG/veX5TkQquB1TfohITxFZISIeESmRw/VE5EoRWS0i60TkP27Hk18iMlpEdntn6S2xRKS2iMSLyErv31Y/t2PKLxEJF5H5IrLUW5aXfXr8QOoDEJFyqnrY+/oxoKGqPuhyWPkiIl2AqaqaISJvAqjq0y6HdcZEpAHgAYbjPBp0ocshnRERCQbWAJcDCcAC4FZVXelqYPkgIu2BJGCMqhb8ieMuEZHqQHVV/VNEygKLgGtL6HciQBlVTRKRUGAW0E9V5/ri+AF1BZBV+XuVAUps9lPV31Q1w/t2LpD3OWCLEVVdpaqr3Y6jAFoB61R1g3ea8zicx56WOKo6A9jvdhwFpao7VPVP7+tEYBVQ092o8sf7RMck79tQ7+KzeiugEgCAiAwUka3A7cALbsfjI72An90OIkDVBLZme59ACa1s/JGI1AWaA/PcjST/RCRYRJYAu4HfVdVnZfG7BCAik0XkrxyWHgCq+pyq1gbGAo+4G23uTlcW7zbPARk45SmW8lIOY3xNRCKBb4B/nXD1X6KoaqaqXoRzld9KRHzWPOd3D4TJ7WH0JxgL/AS8WIjhFMjpyiIi9wDXAJ2K8yM1z+A7KYm2AbWzva/l/cy4yNte/g0wVlW/dTseX1DVgyISD1wJ+KSj3u+uAHIjIudle9sD+NutWApKRK4EngK6q+pRt+MJYAuA80SknoiUAm4BJrocU0DzdpyOAlap6jtux1MQIlIla4SfiETgDDbwWb0VaKOAvgEuwBl1shl4UFVL5K81EVkHhAH7vB/NLYkjmkTkOuADoApwEFiiqle4G9WZEZGrgPeAYGC0qg50OaR8EZFxQAecaYd3AS+q6ihXg8oHEWkHzASW4/y/DvCsqv7kXlT5IyJNgc9w/raCgK9UdYDPjh9ICcAYY8w/AqoJyBhjzD8sARhjTICyBGCMMQHKEoAxxgQoSwDGGBOgLAEYY0yAsgRgjDEByhKAMQUgIhd7n8kQLiJlvHO2l9iplE1gsRvBjCkgEXkVCAcigARVfd3lkIzJE0sAxhSQdw6gBUAK0EZVM10OyZg8sSYgYwquEhAJlMW5EjCmRLArAGMKSEQm4jwJrB7OowiL9XMmjMnid88DMKYo/X97d2gEIBQDUfCnNQqhPMpA0hTiMHgk4nYriHuTicjM7GutO8nx/ge+ZmZLcv49G3yxAQCUcgMAKCUAAKUEAKCUAACUEgCAUgIAUEoAAEo9PJI5LyT9uvwAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xeDlmiHrPRkI"
      },
      "source": [
        "## Importing plotit\r\n",
        "\r\n",
        "To use the code in the following exercises you will need to execute the cell below\r\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pWhbEdVpPQDk"
      },
      "source": [
        "from numpy.random import randn #importing randn\r\n",
        "import numpy as np #importing numpy\r\n",
        "import matplotlib.pyplot as plt #importing plotting module\r\n",
        "import itertools\r\n",
        "import warnings\r\n",
        "\r\n",
        "def plotit(X,Y=None,clf=None,  conts = None, ccolors = ('b','k','r'), colors = ('c','y'), markers = ('s','o'), hold = False, transform = None,extent = None,**kwargs):\r\n",
        "    \"\"\"\r\n",
        "    A function for showing data scatter plot and classification boundary\r\n",
        "    of a classifier for 2D data\r\n",
        "        X: nxd  matrix of data points\r\n",
        "        Y: (optional) n vector of class labels\r\n",
        "        clf: (optional) classification/discriminant function handle\r\n",
        "        conts: (optional) contours (if None, contours are drawn for each class boundary)\r\n",
        "        ccolors: (optional) colors for contours   \r\n",
        "        colors: (optional) colors for each class (sorted wrt class id)\r\n",
        "            can be 'scaled' or 'random' or a list/tuple of color ids\r\n",
        "        markers: (optional) markers for each class (sorted wrt class id)\r\n",
        "        hold: Whether to hold the plot or not for overlay (default: False).\r\n",
        "        transform: (optional) a function handle for transforming data before passing to clf\r\n",
        "        kwargs: any keyword arguments to be passed to clf (if any)        \r\n",
        "    \"\"\"\r\n",
        "    if clf is not None and X.shape[1]!=2:\r\n",
        "        warnings.warn(\"Data Dimensionality is not 2. Unable to plot.\")\r\n",
        "        return\r\n",
        "    if markers is None:\r\n",
        "        markers = ('.',)\r\n",
        "    eps=1e-6\r\n",
        "    d0,d1 = (0,1)\r\n",
        "    if extent is None:\r\n",
        "        minx, maxx = np.min(X[:,d0])-eps, np.max(X[:,d0])+eps\r\n",
        "        miny, maxy = np.min(X[:,d1])-eps, np.max(X[:,d1])+eps\r\n",
        "        extent = [minx,maxx,miny,maxy]\r\n",
        "    else:\r\n",
        "        [minx,maxx,miny,maxy] = extent\r\n",
        "    if Y is not None:\r\n",
        "        classes = sorted(set(Y))\r\n",
        "        if conts is None or len(conts)<2:\r\n",
        "            #conts = list(classes)\r\n",
        "            vmin,vmax = classes[0]-eps,classes[-1]+eps\r\n",
        "        else:            \r\n",
        "            vmin,vmax= np.min(conts)-eps,np.max(conts)+eps\r\n",
        "        \r\n",
        "    else:\r\n",
        "        vmin,vmax=-2-eps,2+eps\r\n",
        "        if conts is None or len(conts)<2:            \r\n",
        "            conts = sorted([-1+eps,0,1-eps])\r\n",
        "        else:\r\n",
        "            vmin,vmax= np.min(conts)-eps,np.max(conts)+eps\r\n",
        "        \r\n",
        "    if clf is not None:\r\n",
        "        npts = 150\r\n",
        "        x = np.linspace(minx,maxx,npts)\r\n",
        "        y = np.linspace(miny,maxy,npts)\r\n",
        "        t = np.array(list(itertools.product(x,y)))\r\n",
        "        if transform is not None:\r\n",
        "            t = transform(t)\r\n",
        "        z = clf(t,**kwargs)\r\n",
        "        \r\n",
        "        z = np.reshape(z,(npts,npts)).T        \r\n",
        "        \r\n",
        "        \r\n",
        "        plt.contour(x,y,z,conts,linewidths = [2],colors=ccolors,extent=extent, label='f(x)=0')\r\n",
        "        #plt.imshow(np.flipud(z), extent = extent, cmap=plt.cm.Purples, vmin = -2, vmax = +2); plt.colorbar()\r\n",
        "        plt.pcolormesh(x, y, z,cmap=plt.cm.Purples,vmin=vmin,vmax=vmax);plt.colorbar()\r\n",
        "        plt.axis(extent)\r\n",
        "    \r\n",
        "    if Y is not None:        \r\n",
        "        for i,y in enumerate(classes):\r\n",
        "            \r\n",
        "            if colors is None or colors=='scaled':\r\n",
        "                cc = np.array([[i,i,i]])/float(len(classes))\r\n",
        "            elif colors =='random':\r\n",
        "                cc = np.array([[np.random.rand(),np.random.rand(),np.random.rand()]])\r\n",
        "            else:\r\n",
        "                cc = colors[i%len(colors)]\r\n",
        "            \r\n",
        "            mm = markers[i%len(markers)]\r\n",
        "            plt.scatter(X[Y==y,d0],X[Y==y,d1], marker = mm,c = cc, s = 50)     \r\n",
        "         \r\n",
        "    else:\r\n",
        "        plt.scatter(X[:,d0],X[:,d1],marker = markers[0], c = 'k', s = 5)\r\n",
        "    plt.xlabel('$x_1$')\r\n",
        "    plt.ylabel('$x_2$')   \r\n",
        "    if not hold:\r\n",
        "        plt.grid()        \r\n",
        "        plt.show()\r\n",
        "    return extent"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RpKV63D0ZI9e"
      },
      "source": [
        "## Trying out Perceptron!\n",
        "\n",
        "Let's try a single Perceptron on the same toy problem. More help on using Perceptron in sk-learn is available [here](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Perceptron.html). Here are some questions for you:\n",
        "\n",
        "1. What are your observations about the line that is drawn by this predictor? \n",
        "2. What happens to the line if you change the data?\n",
        "3. Is this line optimal? \n",
        "4. Can you predict the score and class label for a new test point, say [2,0]? Is this point classified correctly?\n",
        "5. You can use clf.coef_ and clf.intercept_ to get the weight vector and bias learned by the method. What do these weights and their signs tell you about your data? \n",
        "6. Try changing the data and observe how that impacts the weight vector. Solve the AND, OR and XOR problems.\n",
        "7. What happens when your data is not linearly separable?\n",
        "\n",
        "You can also look into questions posted as part of the self assessment exercise related to this.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "b1eqqLKMZI9e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 358
        },
        "outputId": "6b4dace1-6b1a-443f-b729-07aa8e26d0c1"
      },
      "source": [
        "from sklearn.linear_model import Perceptron\n",
        "clf = Perceptron()\n",
        "X = np.array([[-1,0],[0,1],[4,4],[2,3]])\n",
        "y = np.array([-1,-1,+1,+1])\n",
        "clf.fit(X,y)\n",
        "plotit(X,y,clf=clf.decision_function,conts=[0.0])\n",
        "print(\"Learned Weights: \",clf.coef_)\n",
        "print(\"Bias: \",clf.intercept_)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:63: UserWarning: The following kwargs were not used by contour: 'label'\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEMCAYAAADJQLEhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3df5hdVX3v8fcnQ0JoASE/rqYEBB+5iagPwXJRLz8uBhWkPsF7iwppFVpo1CutVdMWyr14S+tzsTetttVbnSIVKz+LWlObPlwEJEkrSPghEn5opK0kxaYkBDOZH+fMnO/9Y6+T7DnMzNmTOWfPzDmf1/PsZ6+99tprrR2G9T37tyICMzPrXnOmuwNmZja9HAjMzLqcA4GZWZdzIDAz63IOBGZmXc6BwMysy5UWCCT1SHpE0jfHWHeopNskbZP0gKTjy+qXmVnZJN0gaaekx8dZL0l/msbExyS9IbfuEkk/TNMlrehPmUcEHwGeHGfdZcALEfFq4NPAp0rrlZlZ+b4EnDfB+ncAJ6ZpDfDnAJIWAJ8A3gicBnxC0tFT7UwpgUDSUuAXgOvHKXIBcGNK3wGcI0ll9M3MrGwRsRHYPUGRC4AvR+Z+4ChJS4BzgbsiYndEvADcxcQBpZCyjgg+A/w2UBtn/THAswARMQy8CCwsp2tmZjPO/jEx2Z7yxsufkkOmWkEzkt4J7IyIhySdPcW61pAdJjF//vyfP+6441rQw5mlVqsxZ84MuIYfMP7LR7I1Y72d5EBelvjRj14GwKte9WLj5rn6X9pYYz2jijS2mwqX/bKUQ+aK4WpnvaKlU/ap55AReg458Ltz587tvPhiTOkswwK9Oqr0Ny3Xx3NbgcFcVm9E9E6l7XZreyAATgdWSTofmA8cKekrEfHLuTI7gGOB7ZIOAV4G7GqsKP1j9gIsW7Ysntg63iWH2WvT5o2cecZZbW+n/o6pkeHsf5bhkdqo5epwjUplBIBKNZtXq9m6wcFhAIaGhhlI6cGBalqX5v0VAN793pMBuOq3vsPAYFbPUNqmvm2lv0qlbyhL78u2r+xNy31pub+yP13tr44uu+9A/kh1vIPO1lu9bjk3r32qtPbK0Cn79IbzH+HcK+7k0MOyv40PfGDqdVbp51T9WtNy345rByPi1Ck2Vx8T65amvB3A2Q35355iW+0/NRQRV0XE0og4HrgIuKchCACsB+pXvy9MZWb/zxIzmxaP33sS1NpwmVEFptZYD7w/3T30JuDFiHgOuBN4u6Sj00Xit6e8KZm2cxCSrpW0Ki1+EVgoaRvwMeDK6eqXmc1+lYFDuenKixnaN4+hgbktqVOCOT1qOhWrS7cA3wGWSdou6TJJH5T0wVRkA/AMsA34C+C/A0TEbuD3gQfTdG3Km5IyTg3tFxHfJh3GRMQ1ufxB4N1l9sXMOtuPHz+OP7rwo7x25Vb27PnmT1pSaYtuZoyIi5usD+DD46y7AbihJR1JZsBVSTOz9qgMzuORDaewcyc7WlGf1HyajUo9IjAzm72E5szSkb4JBwIzsyLE7P3J34QDgZlZQR0aBxwIzMyKEPjUkJlZV5MDgZlZ1/OpITOzbtehkcCBwMysoA6NAw4EZmaFSKinM5/BdSAwMyuggx8jcCAwMyuqUz+c6EBgZlZUZ8YBBwIzs0L8HIGZmfmIwMysq2lmfE+8DRwIzMyKEB37BRcHAjOzArLbRzvz3FDb45uk+ZK+K+l7krZK+r0xylwq6d8lPZqmy9vdLzOzyWrVF8oknSfpaUnbJL3kG+2SPp0bD38gaU9u3Uhu3fpW7FcZRwRDwMqI6JM0F9gs6e8j4v6GcrdFxBUl9MfM7OC04IhAUg/wOeBtwHbgQUnrI+KJepmI+Giu/K8Dp+SqGIiIFVPuSE7bjwgi05cW56Yp2t2umVlLFTgaKBgnTgO2RcQzEVEBbgUumKD8xcAtU9+B8ZVy6UNSj6RHgZ3AXRHxwBjFflHSY5LukHRsGf0yM5sM9cxpOhVwDPBsbnl7yntpe9IrgROAe3LZ8yVtkXS/pHcd7L7klXKxOCJGgBWSjgK+Lul1EfF4rsjfArdExJCkDwA3Aisb65G0BlgDsHjxYjZt3lhC78vV19dXzn5FfZYlIhryI6jl0vky9eVaLajV19UO5AH0zK1XeDIARy3czZH7y2Zr8tvGSKq7lv1J1kZ6RtWblRm7rXyZAzvSfguWzmf1uuWltVeGTtwngPvWTr2OSbxraJGkLbnl3ojoPchmLwLuSGNo3SsjYoekVwH3SPp+RPzoIOsHSr5rKCL2SLoXOA94PJe/K1fseuAPx9m+F+gFWLZsWZx5xllt7O302LR5I2XsV30wHxnORuXhkdqo5epwjUol+9urVLN5tZqtGxwcBmBoaJiBlB4cqKZ12bw6VBnV3p5dCxgYzOoZStvUt630V6n0DWXpfSlvb1ruq6Yylf3pan/K2zd6Xu2vMpL6WIbV65Zz89qnSmuvDJ24Ty0jFb1r6PmIOHWC9TuA/FmPpSlvLBcBH85nRMSONH9G0rfJrh9MKRCUcdfQ4nQkgKTDyC6QPNVQZklucRXwZLv7ZWY2aXMKTM09CJwo6QRJ88gG+5fc/SNpOXA08J1c3tGSDk3pRcDpwBON205WGUcES4Ab05XyOcDtEfFNSdcCWyJiPfAbklYBw8Bu4NIS+mVmNimteI4gIoYlXQHcCfQAN0TE1oYxEbIAcWvEqPOdrwG+IKlGNp5el7/b6GC1PRBExGOMvvWpnn9NLn0VcFW7+2JmdtAEc3pa80BZRGwANjTkXdOw/L/G2O4fgde3pBM5frLYzKyoDn2y2IHAzKwAf6HMzKzb+XsEZmbdbhIvE5plHAjMzArq0DjgQGBmVogo+gqJWceBwMysIB8RmJl1MeGLxWZm1plxwIHAzKwQde6nKh0IzMwKkU8NmZl1OwcCM7Nu5ieLzcy6m981ZGZmHRsJHAjMzAryXUNmZt1MQi36MM1M40BgZlZQhx4QtP/j9WadZt5hQ7zh/Ec4YtFe3nD+I8w7bGi6u2RlSHcNNZtmo7YHAknzJX1X0vckbZX0e2OUOVTSbZK2SXpA0vHt7pfZwTjudT/m43/9Gc694k6OWNjHuVfcycf/+jMc97ofT3fXrAxS86lQNTpP0tNpzLtyjPWXSvp3SY+m6fLcuksk/TBNl7Rit8o4IhgCVkbEycAK4DxJb2oocxnwQkS8Gvg08KkS+mU2KfMOG+KXrruFQ3+2wqGHVQE49LAqh/5shV+67hbmza9Mcw+tneq3j041DkjqAT4HvAM4CbhY0kljFL0tIlak6fq07QLgE8AbgdOAT0g6eqr71vZAEJm+tDg3TdFQ7ALgxpS+AzhHnXp53mat173lCZjT+KebzAleu3JruR2y0rXo1NBpwLaIeCYiKsCtZGNgEecCd0XE7oh4AbgLOO+gdianlIvFKQI+BLwa+FxEPNBQ5BjgWYCIGJb0IrAQeL6hnjXAGoDFixezafPGdne9dH19feXsV9RnWSKiIT+CWi6dL1NfrtWCWn1d7UAeQM/ceoUnA3DUwt0cub9stia/bYykumvZn2RtpGdUvVmZsdvKlzmwI613xKJjqI6cTTX9rBkZWUpf37r96097z+G8ZuURbWu/DAuWzmf1uuXT3Y2Wu29tCypR4YF+kaQtueXeiOjNLe8f75LtZL/wG/2ipLOAHwAfjYhnx9n2mCKdmkgpgSAiRoAVko4Cvi7pdRHx+EHU0wv0AixbtizOPOOsFvd0+m3avJEy9qs+mI8MZ6Py8Eht1HJ1uEalko3OlWo2r1azdYODwwAMDQ0zkNKDA9W0LptXh0afJtmzawEDg1k9Q2mb+raV/iqVvuyCa2VfytublvuqqUxlf7ran/L2jZ5X+6uMpD62wxvOf4Rzr7hz/2mhvr51HH54NsIMDczlu7efyyMbTmlb+2VYvW45N699arq7MWMVDATPR8SpU2zqb4FbImJI0gfIzpisnGKd4yr1rqGI2APcy0sPZXYAxwJIOgR4GbCrzL6ZNfP4vSdBbZyBoCa23vPacjtk5UqvoW42FbB/vEuWprz9ImJXRNRvR7se+Pmi2x6MMu4aWpyOBJB0GPA2oPEnx3qgfvX7QuCeiDYe45sdhMrAodx05cUM7ZvH0MBcIDsSGNo3j5uuvJjK4Lxp7qG1nQpMzT0InCjpBEnzgIvIxsADzUhLcourgCdT+k7g7ZKOTheJ357ypqSMU0NLgBvTdYI5wO0R8U1J1wJbImI98EXgryRtA3aT/cOYzTg/fvw4/ujCj/LalVs57T2H893bz2XrPa91EOgCrfpUZboOegXZAN4D3BARWxvGxN+QtAoYJhsTL03b7pb0+2TBBODaiNg91T61PRBExGPAS06cRsQ1ufQg8O5298WsFSqD83hkwym8ZuURs/6agE1Oq25mjIgNwIaGvPyYeBVw1Tjb3gDc0JKOJH7FhJlZEX7XkJmZderjTQ4EZmZFCJil7xJqxoHAzKygDj0gcCAwMyuiVXcNzUQOBGZmRUhoTme+ud+BwMysIHVmHHAgMDMryncNmZl1OwcCM7MuJp8aMjPratkXynxEYGbWvSTU05mHBA4EZmYFdegBgQOBmVlhfqDMzKy7+RqBmVk3k08NmZmZTw2ZmXUvSczp0LuGyvh4/bGS7pX0hKStkj4yRpmzJb0o6dE0XTNWXWZm06o1H69H0nmSnpa0TdKVY6z/WBozH5N0t6RX5taN5MbK9Y3bHowyjgiGgY9HxMOSjgAeknRXRDzRUG5TRLyzhP6YmR2UVlwsltQDfA54G7AdeFDS+oYx8RHg1Ijol/Qh4A+B96Z1AxGxYsodyWn7EUFEPBcRD6f0XuBJ4Jh2t2tm1mqao6ZTAacB2yLimYioALcCF+QLRMS9EdGfFu8HlrZ0RxqUeo1A0vHAKcADY6x+s6TvAf8KrI2IrWNsvwZYA7B48WI2bd7Yvs5Ok76+vnL2K+qzLBHRkB9BLZfOl6kv12pBrb6udiAPoGduvcKTAThq4W6O3F82W5PfNkZS3bXsT7I20jOq3qzM2G3lyxzYkfZbsHQ+q9ctL629MnTiPgHct7YFlajwEcEiSVtyy70R0ZtbPgZ4Nre8HXjjBPVdBvx9bnl+qn8YuC4i/qZIpyZSWiCQdDjwVeA3I+KnDasfBl4ZEX2Szgf+BjixsY70j9kLsGzZsjjzjLPa3Ovybdq8kTL2qz6Yjwxno/LwSG3UcnW4RqWSjc6VajavVrN1g4PDAAwNDTOQ0oMD1bQum1eHKqPa27NrAQODWT1DaZv6tpX+KpW+oSy9L+XtTct91VSmsj9d7U95+0bPq/1VRlIfy7B63XJuXvtUae2VoRP3qVUm8YWy5yPi1Ja0Kf0ycCrwX3LZr4yIHZJeBdwj6fsR8aOptFPKJXBJc8mCwE0R8bXG9RHx04joS+kNwFxJi8rom5lZUVLzqYAdwLG55aUpr6EtvRW4GlgVEUP1/IjYkebPAN8mO8syJWXcNSTgi8CTEfHH45R5RSqHpNNSv3a1u29mZpPRokDwIHCipBMkzQMuAkbd/SPpFOALZEFgZy7/aEmHpvQi4HSg8cabSSvj1NDpwPuA70t6NOX9LnAcQER8HrgQ+JCkYWAAuCiixJO9ZmbNSC25aygihiVdAdwJ9AA3RMRWSdcCWyJiPfB/gMOBv05t/jgiVgGvAb4gqUb2g/m6Me7AnLS2B4KI2EyTu2sj4rPAZ9vdFzOzqWjVKybSKfANDXnX5NJvHWe7fwRe35peHOAni83MCvCHaczMjDl+15CZWRfz20fNzExFXyY0yzgQmJkVkF0jmO5etIcDgZlZQQ4EZmZdzheLzcy62SQeHZ5tHAjMzArq0DjgQGBmVoQfKDMzMx8RmJl1Ox8RmJl1M0GH3jTkQGBmVoSvEZiZma8RmJl1Ox8RmJl1uQ6NAw4EZmaFtOhTlTNR4Y/XS3qbpL+QtCItrym43bGS7pX0hKStkj4yRhlJ+lNJ2yQ9JukNxXfBzKz9BMyZ03wqVJd0nqSn05h35RjrD5V0W1r/gKTjc+uuSvlPSzq3Ffs2mSOCXwU+BPwPSQuAFQW3GwY+HhEPSzoCeEjSXQ0fXH4HcGKa3gj8eZqbmc0YrTgikNQDfA54G7AdeFDS+oYx8TLghYh4taSLgE8B75V0EnAR8Frg54BvSfqPETEylT4VPiIA9kbEnohYC7wd+E9FNoqI5yLi4ZTeCzwJHNNQ7ALgy5G5HzhK0pJJ9M3MrO3q752baCrgNGBbRDwTERXgVrIxMO8C4MaUvgM4R1kUugC4NSKGIuKfgG2pvimZzBHB39UTEXGlpF+fbGPp8OYU4IGGVccAz+aWt6e85xq2XwOsAVi8eDGbNm+cbBdmvL6+vnL2K+qzLBHRkB9BLZfOl6kv12pBrb6udiAPoGduvcKTAThq4W6O3F82W5Pftv57JmrZn2RtpGdUvVmZsdvKlzmwI+23YOl8Vq9bXlp7ZejEfQK4b20LKlHhI4JFkrbklnsjoje3PNZ413gGZH+ZiBiW9CKwMOXf37Bt4w/rSWsaCCT9CfCbEfGNfH5E/NlkGpJ0OPDVVNdPJ9XLA232Ar0Ay5YtizPPOOtgqpnRNm3eSBn7VR/MR4azUXl4pDZquTpco1LJRudKNZtXq9m6wcFhAIaGhhlI6cGBalqXzatDlVHt7dm1gIHBrJ6htE1920p/lUrfUJbel/L2puW+aipT2Z+u9qe8faPn1f4qI6mPZVi9bjk3r32qtPbK0In71FLFfvE/HxGntrknLVXk1NBeYL2knwGQdK6kf5hMI5LmkgWBmyLia2MU2QEcm1temvLMzGaE7GLxnKZTAUXGu/1lJB0CvAzYVXDbSWva64j4H8AtwH0pAHwMeMlV7vGk81pfBJ6MiD8ep9h64P3p7qE3AS9GxHPjlDUzmxYtukbwIHCipBMkzSO7+Lu+ocx64JKUvhC4J7LD+PXARemuohPIbrD57lT3q8ipoXOAXwP2AUuAX42IpyfRxunA+4DvS3o05f0ucBxARHwe2ACcT3bhox/4lUnUb2bWfi16jiCd878CuBPoAW6IiK2SrgW2RMR6sh/PfyVpG7CbLFiQyt0OPEF2R+aHp3rHEBS7WHw18D8jYrOk1wO3SfpYRNxTpIGI2EyTM2sp0n24SH1mZtMhe+lca+qKiA1kP4Dzedfk0oPAu8fZ9pPAJ1vTk0zTQBARK3Pp70t6B9n5/v/cyo6Ymc10nfpk8aRfMRERz6XTRWZmXcWBICciBlrdETOzGU0wp0O/TOOXzpmZFSBADgRmZt2tQ88MORCYmRXSwa+hdiAwMyuoQ+OAA4GZWVE+IjAz62LZu4YcCMzMulcrHy2eYRwIzMwK8qkhM7Mu16FxwIHAzKwY+YEyM7NuJr9iwszMfI3AzKzLdWgccCAwMyvKRwRmZl0s+yZxZwaCph+vnypJN0jaKenxcdafLelFSY+m6ZqxypmZTbcWfby+SRtaIOkuST9M86PHKLNC0nckbZX0mKT35tZ9SdI/5cbUFc3abHsgAL4EnNekzKaIWJGma0vok5nZJIk5c+Y0nVrgSuDuiDgRuDstN+oH3h8RryUbXz8j6ajc+t/KjamPNmuw7YEgIjYCu9vdjplZu5VxRABcANyY0jcC72osEBE/iIgfpvS/AjuBxQfb4Ey5RvBmSd8D/hVYGxFbxyokaQ2wBmDx4sVs2ryxxC6Wo6+vr5z9ivosS0Q05EdQy6XzZerLtVpQq6+rHcgD6Jlbr/BkAI5auJsj95fN1uS3jZFUdy37k6yN9IyqNyszdlv5Mgd2pP0WLJ3P6nXLS2uvDJ24TwD3rZ16HVLhL5QtkrQlt9wbEb2TaOrlEfFcSv8EePnE/dJpwDzgR7nsT6bT7HcDV0bE0ER1zIRA8DDwyojok3Q+8DfAiWMVTP+YvQDLli2LM884q7xelmTT5o2UsV/1wXxkOBuVh0dqo5arwzUqlWx0rlSzebWarRscHAZgaGiYgZQeHKimddm8OlQZ1d6eXQsYGMzqGUrb1Let9Fep9GV/p5V9KW9vWu6rpjKV/elqf8rbN3pe7a8ykvpYhtXrlnPz2qdKa68MnbhPrVTwF//zEXHqxPXoW8Arxlh1dX4hIkLSuL9uJC0B/gq4JKL+E4uryALIPLLx8neACU+5T3sgiIif5tIbJP1fSYsi4vnp7JeZWSPRmnM/EfHWcduQ/k3Skoh4Lg30O8cpdyTwd8DVEXF/ru760cSQpL8Emh4PlXGxeEKSXqF0T1Y6xJkD7JreXpmZjUEFpqlbD1yS0pcA33hJN6R5wNeBL0fEHQ3rlqS5yK4vjHnHZl7bjwgk3QKcTXbebDvwCWAuQER8HrgQ+JCkYWAAuCiixBO9ZmZFlPeuoeuA2yVdBvwL8B4ASacCH4yIy1PeWcBCSZem7S5NdwjdJGlx1mMeBT7YrMG2B4KIuLjJ+s8Cn213P8zMpkKU8/H6iNgFnDNG/hbg8pT+CvCVcbZfOdk2p/0agZnZbNGhDxY7EJiZFdWpr5hwIDAzK6hD44ADgZlZEf4wjZmZ+YjAzKzbteqBspnGgcDMrCAfEZiZdTnfNWRm1sVa+JrpGceBwMysEPmuITOzbudTQ2Zm3a4z44ADgZlZIfIRgZlZVxO+WGxm1vXmdGgkcCAwMyvIp4bMzLpch8YBBwIzsyKkcr5QNh3K+GbxDcA7gZ0R8box1gv4E+B8oJ/su5sPt7tfM8XR3/kH9o6M7F9e19fHWzZv3L98RE8PL7z59Onompk1KCMOSFoA3AYcD/wz8J6IeGGMciPA99PijyNiVco/AbgVWAg8BLwvIioTtTmnVZ2fwJeA8yZY/w7gxDStAf68hD7NGPkgcDDrzaw89ddMTDS1wJXA3RFxInB3Wh7LQESsSNOqXP6ngE9HxKuBF4DLmjXY9kAQERuB3RMUuQD4cmTuB46StKTd/TIzm6w5c9R0aoELgBtT+kbgXUU3TGdYVgJ3TGb7Mo4ImjkGeDa3vD3lmZnNGNKB6wQTTS3w8oh4LqV/Arx8nHLzJW2RdL+k+mC/ENgTEcNpudB4OqsuFktaQ3b6iMWLF7Mpdy59tlrX1zdqeenIyEvy2rKfUZ9liYiG/AhquXS+TH25Vgtq9XW1A3kAPXPrFZ4MwFELd3Pk/rLZmvy2kc6ARS37k6yN9IyqNyszdlv5Mgd2pP0WLJ3P6nXLS2uvDJ24TwD3rS21uUWStuSWeyOiN19A0reAV4yx7dX5hYgISeP9Ub8yInZIehVwj6TvAy8eTIdnQiDYARybW16a8l4i/WP2AixbtizOPOOs9veuzd7SMMiv6+tj7eGHj8obbsN+1gfzkeFsVB4eqY1arg7XqFSy0blSzebVarZucDD7sTE0NMxASg8OVNO6bF4dGn1tas+uBQwMZvUMpW3q21b6q1T6hrL0vpS3Ny33VVOZyv50tT/l7Rs9r/ZXGUl9LMPqdcu5ee1TpbVXhk7cp1Yq+Iv/+Yg4daICEfHWCdr4N0lLIuK5dJp85zh17EjzZyR9GzgF+CrZ6fVD0lHBuONp3kw4NbQeeL8ybwJezB0WmZnNGCVdLF4PXJLSlwDfeGk/dLSkQ1N6EXA68ERkv/DuBS6caPtGZdw+egtwNtnh0nbgE8BcgIj4PLCB7NbRbWS3j/5Ku/tkZjaDXQfcLuky4F+A9wBIOhX4YERcDrwG+IKkGtkP+usi4om0/e8At0r6A+AR4IvNGmx7IIiIi5usD+DD7e7HTHVET8+Et4ge0dNTYm/MbCJlvGsoInYB54yRvwW4PKX/EXj9ONs/A5w2mTZnwjWCrtb4sNimzRvbck3AzFqgMx8sdiAwMyvC3yw2M+t6Qh16SOBAYGZWVGfGAQcCM7Oi/GEaM7Nu15lxwIHAzKyoDo0DDgRmZkVkH6/vzFAwE14xYWZm08hHBGZmRfg5AjMz69RTQw4EZmYFdWgccCAwMyvGTxabmVlnxgEHAjOzIrLbR6e7F+3hQGBmVlCHxgEHAjOzQuS7hszMul6HxoFyniyWdJ6kpyVtk3TlGOsvlfTvkh5N0+Vl9MvMbFJK+Hq9pAWS7pL0wzQ/eowyb8mNl49KGpT0rrTuS5L+KbduRbM22x4IJPUAnwPeAZwEXCzppDGK3hYRK9J0fbv7ZWY2WSowtcCVwN0RcSJwd1oeJSLurY+XwEqgH/h/uSK/lRtPH23WYBlHBKcB2yLimYioALcCF5TQrplZy9TvGmrzAQFk4+ONKX0j8K4m5S8E/j4i+g+2wTICwTHAs7nl7Smv0S9KekzSHZKOLaFfZmaTIqnp1AIvj4jnUvonwMublL8IuKUh75NpPP20pEObNThTLhb/LXBLRAxJ+gBZFFzZWEjSGmANwOLFi9m0eWO5vSxBX19fOfsV9VmWiGjIj6CWS+fL1JdrtaBWX1c7kAfQM7de4ckAHLVwN0fuL5utyW8bI6nuWvYnWRvpGVVvVmbstvJlDuxI+y1YOp/V65aX1l4ZOnGfAO5bW2pziyRtyS33RkRvvoCkbwGvGGPbq/MLERGSxv2jlrQEeD1wZy77KrIAMg/oBX4HuHaiDpcRCHYA+V/4S1PefhGxK7d4PfCHY1WU/jF7AZYtWxZnnnFWa3s6A2zavJEy9qs+mI8MZ6Py8Eht1HJ1uEalko3OlWo2r1azdYODwwAMDQ0zkNKDA9W0LptXhyqj2tuzawEDg1k9Q2mb+raV/iqVvqEsvS/l7U3LfdVUprI/Xe1PeftGz6v9VUZSH8uwet1ybl77VGntlaET96llip/6eT4iTp2oQES8ddxmpH+TtCQinksD/c4JqnoP8PWIqObqrh9NDEn6S6BpGCzj1NCDwImSTpA0j+wwZn2+QNrZulXAkyX0y8xskkq5XLweuCSlLwG+MUHZi2k4LVQfT5Wdp3oX8HizBtt+RBARw5KuIDt06QFuiIitkq4FtkTEeuA3JK0ChoHdwKXt7peZ2WSV9BzBdcDtki4D/oXsVz+STgU+GBGXp+Xjyc623New/U2SFpNFpUeBDzZrsJRrBBGxAdjQkHdNLn0V2XktM7Oulk6VnzNG/kO8av8AAAXYSURBVBbg8tzyPzPGjTcR8ZLrq83MlIvFZmYzmmjZXUEzjr9ZbGbW5XxEYGZWUIceEPiIwMys2/mIwMysiA7+Mo0DgZlZQZ0ZBhwIzMwK69ADAgcCM7PCOjQS+GKxmVmX8xGBmVlBnXk84EBgZlZch0YCBwIzswKyu0c7MxI4EJiZFdSZYcCBwMysmBZ+nX6mcSAwMyusMyOBA4GZWUGdGQYcCMzMiuvQSOBAYGZWSOd+mGbWBoL+Wo1DNm/cv3xETw8vvPn0aeyRmdnsVMorJiSdJ+lpSdskXTnG+kMl3ZbWP5A+yjwpe0dGWtFVM7NxSc2nqbehd0vaKqmWPlg/Xrkxx1VJJ6RxdFsaV+c1a7PtgUBSD/A54B3AScDFkk5qKHYZ8EJEvBr4NPCpdvfLzGyGehz4b8DG8Qo0GVc/BXw6jacvkI2vEyrjiOA0YFtEPBMRFeBW4IKGMhcAN6b0HcA56tSTcWY2K9WfLG42TVVEPBkRTzcpNua4msbNlWTjKGTj6ruatVnGNYJjgGdzy9uBN45XJiKGJb0ILASezxeStAZYA8DcufCBD4yq5JAf/OChVnZ8miyiYb87wKJf+1DH7RP3re28/1aduE/JsqlW8NDDD915yNyeRQWKzpe0JbfcGxG9U22/wXjj6kJgT0QM5/KPaVbZrLpYnP4xewEkbYmnnx73/NlsJWlLRHTUfnXiPkFn7lcn7hNk+zXVOiLivFb0BUDSt4BXjLHq6oj4RqvaKaqMQLADODa3vDTljVVmu6RDgJcBu0rom5lZ6SLirVOsYrxxdRdwlKRD0lHBWOPtS5RxjeBB4MR0JXsecBGwvqHMeuCSlL4QuCciooS+mZnNRmOOq2ncvJdsHIVsXG16hNH2QJCi0hXAncCTwO0RsVXStZJWpWJfBBZK2gZ8DHjJLaZjaPU5t5miE/erE/cJOnO/OnGfYBbtl6T/Kmk78Gbg7yTdmfJ/TtIGGH9cTVX8DvCxNJ4uJBtfJ27TP7zNzLqbv1lsZtblHAjMzLrcrA4ERR/Fng2avYZjNpJ0g6Sdkh6f7r60iqRjJd0r6Yn0t/eR6e5TK0iaL+m7kr6X9uv3prtPrSKpR9Ijkr453X2ZqWZ1IKDAo9izQcHXcMxGXwJadu/1DDEMfDwiTgLeBHy4Q/5bDQErI+JkYAVwnqQ3TXOfWuUjZBdUbRyzOhAUfBR7NijyGo5ZJyI2Arunux+tFBHPRcTDKb2XbIBp+uTmTBeZvrQ4N02z/k4SSUuBXwCun+6+zGSzOhB0kLEeF5/1g0unS2/JPQV4YHp70hrpFMqjwE7grojohP36DPDbQG26OzKTzfhAIOlbkh4fY5r1v5ht9pJ0OPBV4Dcj4qfT3Z9WiIiRiFhB9jTqaZJeN919mgpJ7wR2RkQnvIOsrWb8u4Za8Cj2bFDkNRw2Q0iaSxYEboqIr013f1otIvZIupfs+s5svtB/OrBK0vnAfOBISV+JiF+e5n7NODP+iKBLFHkNh80A6TW/XwSejIg/nu7+tIqkxZKOSunDgLcBT01vr6YmIq6KiKURcTzZ/1P3OAiMbVYHgvEexZ5tmjwuPmtJugX4DrBM0nZJTT+QMQucDrwPWCnp0TSdP92daoElwL2SHiP7YXJXRPh2yy7hV0yYmXW5WX1EYGZmU+dAYGbW5RwIzMy6nAOBmVmXcyAwM+tyDgRmZl3OgcBmtfRK6Lel9B9I+rPp7pPZbDPjXzFh1sQngGsl/QeyF8CtalLezBr4gTKb9STdBxwOnB0ReyW9CrgaeFlEXDi9vTOb+XxqyGY1Sa8nez1CJX0fgPRdh054nYVZKRwIbNaStAS4iewjPn2SOu1raGalcCCwWUnSzwBfI/ts5JPA75NdLzCzSfI1Aus4khYCnyR7lfL1EfG/p7lLZjOaA4GZWZfzqSEzsy7nQGBm1uUcCMzMupwDgZlZl3MgMDPrcg4EZmZdzoHAzKzLORCYmXU5BwIzsy73/wG3mbo6GScthQAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        },
        {
          "output_type": "stream",
          "text": [
            "Learned Weights:  [[2. 0.]]\n",
            "Bias:  [-2.]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "97krRNHtZI9g"
      },
      "source": [
        "## Classifying digits\n",
        "Below is a code for classifying between MNIST digits (7 vs 9) using nearest neighbor and Perceptron classifiers. Please try to understand the code and answer the following questions."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DdYgrb4lZI9g",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 864
        },
        "outputId": "307804a8-af7b-454f-9473-b3f3fcb5a9a7"
      },
      "source": [
        "import numpy as np\n",
        "from sklearn.datasets import load_digits\n",
        "import random\n",
        "from sklearn.neighbors import KNeighborsClassifier\n",
        "\n",
        "import matplotlib.pyplot as plt\n",
        "digN,digP = 7,9\n",
        "X,y = load_digits(n_class=10, return_X_y=True) #load the digits dataset\n",
        "idx = (y==digP) + (y==digN) #Just focus on 7 and 9\n",
        "X,y = X[idx,:], y[idx]\n",
        "y[y==digN]=-1.0 #set (negative class)\n",
        "y[y==digP]=+1.0 #set (positive class)\n",
        "idx = list(range(len(y)))\n",
        "random.shuffle(idx) # Just shuffle the data\n",
        "X,y = X[idx,:], y[idx]\n",
        "plt.gray();plt.matshow(X[0].reshape((8,-1))) \n",
        "# show an example. we need to reshape to show as an image.\n",
        "\n",
        "Ntr = 15 #select number of training examples\n",
        "Xtr,Ytr = X[:Ntr], y[:Ntr] #get training examples\n",
        "Xtt,Ytt = X[Ntr:], y[Ntr:] #get validation (test) examples\n",
        "\n",
        "clf = KNeighborsClassifier(n_neighbors=1)\n",
        "clf = clf.fit(Xtr, Ytr)\n",
        "Z = clf.predict(Xtt)\n",
        "print('Accuracy of kNN:',np.mean(Z==Ytt))\n",
        "plt.gray();plt.matshow(Xtt[0].reshape((8,-1))); plt.title('Predicted:'+str(Z[0]))\n",
        "error_idx = np.nonzero(Ytt!=np.sign(Z))[0][0] # show one error example\n",
        "plt.gray();plt.matshow(Xtt[error_idx].reshape((8,-1))); plt.title('Misclassified: Predicted as '+str(Z[error_idx])) # show a misclassification\n",
        "\n",
        "\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Accuracy of kNN: 0.9622093023255814\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0.5, 1.05, 'Misclassified: Predicted as -1')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 0 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPoAAAECCAYAAADXWsr9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAALxklEQVR4nO3d34tc9RnH8c/HNcFUYxaqETHiWigBEZoECRVF04RIrJLmoheJKKy0pBetGFoQ7U3NPyDbiyKEqBGMEY0uFGmtARNEaLX5sdGYjUXDignqKhJXRRpMnl7MiWyXtHt2Pd+zs/u8XzBkZnZ2nmc2fOacM3POeRwRAjC3XTDTDQAoj6ADCRB0IAGCDiRA0IEECDqQQFcE3fY62+/Yftf2g4VrPW571PaRknXG1bva9l7bR22/bfv+wvUusv2G7cNVva0l61U1e2wfsv1i6VpVvRHbb9kesr2/cK1e27ttH7M9bPvGgrWWVq/p3GXM9pZGnjwiZvQiqUfSe5J+IGm+pMOSritY7xZJKyQdaen1XSlpRXV9oaR/FX59lnRJdX2epNcl/bjwa/ytpKclvdjS33RE0mUt1XpS0i+r6/Ml9bZUt0fSR5KuaeL5umGJvlLSuxFxPCJOS3pG0s9KFYuIVyV9Vur5z1Pvw4g4WF3/QtKwpKsK1ouI+LK6Oa+6FNsryvYSSXdI2l6qxkyxvUidBcNjkhQRpyPiVEvl10h6LyLeb+LJuiHoV0n6YNztEyoYhJlku0/ScnWWsiXr9NgekjQqaU9ElKw3IOkBSWcL1pgoJL1s+4DtzQXrXCvpE0lPVJsm221fXLDeeBsl7Wrqyboh6CnYvkTS85K2RMRYyVoRcSYilklaImml7etL1LF9p6TRiDhQ4vn/j5sjYoWk2yX92vYthepcqM5m3qMRsVzSV5KKfoYkSbbnS1ov6bmmnrMbgn5S0tXjbi+p7pszbM9TJ+Q7I+KFtupWq5l7Ja0rVOImSettj6izybXa9lOFan0rIk5W/45KGlRn86+EE5JOjFsj2q1O8Eu7XdLBiPi4qSfshqD/U9IPbV9bvZNtlPTnGe6pMbatzjbecEQ80kK9y233VtcXSFor6ViJWhHxUEQsiYg+df7fXomIu0vUOsf2xbYXnrsu6TZJRb5BiYiPJH1ge2l11xpJR0vUmmCTGlxtlzqrJjMqIr6x/RtJf1Pnk8bHI+LtUvVs75K0StJltk9I+kNEPFaqnjpLvXskvVVtN0vS7yPiL4XqXSnpSds96ryRPxsRrXzt1ZIrJA123j91oaSnI+KlgvXuk7SzWggdl3RvwVrn3rzWSvpVo89bfZQPYA7rhlV3AIURdCABgg4kQNCBBAg6kEBXBb3w7owzVot61Jvpel0VdElt/jFb/Y+jHvVmsl63BR1AAUV2mLE9p/fCWbx48ZR/5+uvv9aCBQumVa+3t3fKv/P5559r0aJF06p35syZKf/O2NiYLr300mnVGxkZmfLvnD17VhdcML3l1HRe32wSEZ5434zvAjsb3XXXXa3W27BhQ6v1Tp1q65Drjv7+/lbrtf36ugGr7kACBB1IgKADCRB0IAGCDiRA0IEECDqQAEEHEqgV9DZHJgFo3qRBr04y+Cd1TkF7naRNtq8r3RiA5tRZorc6MglA8+oEPc3IJGCuauyglupA+baP2QVQQ52g1xqZFBHbJG2T5v5hqsBsU2fVfU6PTAIymHSJ3vbIJADNq7WNXs0JKzUrDEBh7BkHJEDQgQQIOpAAQQcSIOhAAgQdSICgAwkQdCABJrVMQ9uTU6Yzsui7GBwcbLXe0NBQq/X6+vpardcNWKIDCRB0IAGCDiRA0IEECDqQAEEHEiDoQAIEHUiAoAMJEHQggTojmR63PWr7SBsNAWhenSX6DknrCvcBoKBJgx4Rr0r6rIVeABTCNjqQALPXgAQaCzqz14Duxao7kECdr9d2Sfq7pKW2T9j+Rfm2ADSpzpDFTW00AqAcVt2BBAg6kABBBxIg6EACBB1IgKADCRB0IAGCDiQwJ2av9ff3t1qvt7e31XrLli1rtd7AwECr9dr+e2bEEh1IgKADCRB0IAGCDiRA0IEECDqQAEEHEiDoQAIEHUiAoAMJ1Dk55NW299o+avtt2/e30RiA5tTZ1/0bSb+LiIO2F0o6YHtPRBwt3BuAhtSZvfZhRBysrn8haVjSVaUbA9CcKW2j2+6TtFzS6yWaAVBG7cNUbV8i6XlJWyJi7Dw/Z/Ya0KVqBd32PHVCvjMiXjjfY5i9BnSvOp+6W9JjkoYj4pHyLQFoWp1t9Jsk3SNpte2h6vLTwn0BaFCd2WuvSXILvQAohD3jgAQIOpAAQQcSIOhAAgQdSICgAwkQdCABgg4kMCdmr/X19bVab9++fa3Wa/v1HTp0qNV6W7dubbVeRizRgQQIOpAAQQcSIOhAAgQdSICgAwkQdCABgg4kQNCBBAg6kECds8BeZPsN24er2WvsrwjMMnX2df+3pNUR8WV1fvfXbP81Iv5RuDcADalzFtiQ9GV1c151YUADMIvU2ka33WN7SNKopD0Rwew1YBapFfSIOBMRyyQtkbTS9vUTH2N7s+39tvc33SSA72ZKn7pHxClJeyWtO8/PtkXEDRFxQ1PNAWhGnU/dL7fdW11fIGmtpGOlGwPQnDqful8p6UnbPeq8MTwbES+WbQtAk+p86v6mpOUt9AKgEPaMAxIg6EACBB1IgKADCRB0IAGCDiRA0IEECDqQgDtHoTb8pHarh7H29va2WU47duxotd6pU6fmdL22Z8v19/e3Wq/tv2dEeOJ9LNGBBAg6kABBBxIg6EACBB1IgKADCRB0IAGCDiRA0IEECDqQQO2gV0McDtnmxJDALDOVJfr9koZLNQKgnLojmZZIukPS9rLtACih7hJ9QNIDks4W7AVAIXUmtdwpaTQiDkzyOGavAV2qzhL9JknrbY9IekbSattPTXwQs9eA7jVp0CPioYhYEhF9kjZKeiUi7i7eGYDG8D06kECdIYvfioh9kvYV6QRAMSzRgQQIOpAAQQcSIOhAAgQdSICgAwkQdCABgg4kMCdmr2F2GxoaarXewMBAq/XantXH7DUgKYIOJEDQgQQIOpAAQQcSIOhAAgQdSICgAwkQdCABgg4kUOuccdWpnr+QdEbSN5zSGZhdpnJyyJ9ExKfFOgFQDKvuQAJ1gx6SXrZ9wPbmkg0BaF7dVfebI+Kk7cWS9tg+FhGvjn9A9QbAmwDQhWot0SPiZPXvqKRBSSvP8xhmrwFdqs401YttLzx3XdJtko6UbgxAc+qsul8hadD2ucc/HREvFe0KQKMmDXpEHJf0oxZ6AVAIX68BCRB0IAGCDiRA0IEECDqQAEEHEiDoQAIEHUiA2WuYcW3PJuvr62u13qpVq1qtx+w1ICmCDiRA0IEECDqQAEEHEiDoQAIEHUiAoAMJEHQgAYIOJFAr6LZ7be+2fcz2sO0bSzcGoDl1Bzj8UdJLEfFz2/Mlfa9gTwAaNmnQbS+SdIukfkmKiNOSTpdtC0CT6qy6XyvpE0lP2D5ke3s1yOG/2N5se7/t/Y13CeA7qRP0CyWtkPRoRCyX9JWkByc+iJFMQPeqE/QTkk5ExOvV7d3qBB/ALDFp0CPiI0kf2F5a3bVG0tGiXQFoVN1P3e+TtLP6xP24pHvLtQSgabWCHhFDktj2BmYp9owDEiDoQAIEHUiAoAMJEHQgAYIOJEDQgQQIOpBA3T3jMM7DDz/car22Z3fdeuutrdY7fPhwq/U2bNjQar1uwBIdSICgAwkQdCABgg4kQNCBBAg6kABBBxIg6EACBB1IYNKg215qe2jcZcz2ljaaA9CMSXeBjYh3JC2TJNs9kk5KGizcF4AGTXXVfY2k9yLi/RLNAChjqkHfKGlXiUYAlFM76NU53ddLeu5//JzZa0CXmsphqrdLOhgRH5/vhxGxTdI2SbIdDfQGoCFTWXXfJFbbgVmpVtCrMclrJb1Qth0AJdQdyfSVpO8X7gVAIewZByRA0IEECDqQAEEHEiDoQAIEHUiAoAMJEHQgAYIOJOCI5o8/sf2JpOkcs36ZpE8bbqcbalGPem3VuyYiLp94Z5GgT5ft/RFxw1yrRT3qzXQ9Vt2BBAg6kEC3BX3bHK1FPerNaL2u2kYHUEa3LdEBFEDQgQQIOpAAQQcSIOhAAv8BpyaSVY48AIEAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 288x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPoAAAEMCAYAAADtUKuNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAPN0lEQVR4nO3de4xc9XnG8e+TtR3MLasYB+ELLA2JVYraNUKkiJQYKCkEgqO0UqAlwqStGymlsXpJk6pKqJpI/FGlRkpLSoHYbbiEOBhFqNAQYUStEFLbbFKwoQG6gB3AhhQbUFTL5u0f5yxaNuvs2fX5nZ3d9/lII5+ZOTPvO2M9e65zfooIzGx2e9t0N2Bm5TnoZgk46GYJOOhmCTjoZgk46GYJOOizmKR1kr5YT/+GpCc6qhuSTumiljXjoPcAScOSfibpNUkv1gE9us0aEfEfEbGsQS+rJG1us/YE9eZJ2lB/ByFpRVe1M3HQe8eHI+Jo4HTgDOCvRz8pac60dNWNzcAVwAvT3chs5aD3mIjYBdwDnFYv4T4l6cfAjwEkXSJpSNIrkr4n6VdHXitpuaRtkl6V9A3giFHPrZC0c9T9pZLulLRH0suSviLpl4GvAmfVaxev1PO+XdLfSXq2XuP4qqT5o97rLyQ9L+knkj4xyc+7PyLWRsRm4OCUvjSbkIPeYyQtBT4EPFI/9BHgfcCpkpYDNwN/BCwA/gn4dh3EecBdwL8C7wS+Cfz2IWr0AXcDzwADwGLg9ojYAXwSeCgijo6I/vol1wLvBQaBU+r5P1+/14XAnwMXAO8BfnNMrd+V9KPD+EqsDRHh2zTfgGHgNeAVqvD9IzAfCOC8UfNdD/ztmNc+AXwAOAf4CaBRz30P+GI9vQLYWU+fBewB5ozTyypg86j7Al4H3j3qsbOA/6mnbwauHfXce+u+T5nC97ATWDHd/x+z8Tabt/tmmo9ExHdHPyAJ4LlRD50EXCnp6lGPzQMWUYVrV9SJqT1ziFpLgWci4kCDvhYCRwJb636gCn9fPb0I2NqgJpJOBLaP3I9qn4R1wEHvfaOD+xzwpYj40tiZJH0AWCxJo8J+IvDUOO/5HHCipDnjhH3szxlfAn4G/EpU+w/Gep7qD8eIEw/5QSKeBRzuaeBt9Jnln4FPSnqfKkdJuljSMcBDwAHgTyTNlfRR4MxDvM8PqAJ6bf0eR0g6u37uRWBJvc1PRLxR1/17Se8CkLRY0m/V898BrJJ0qqQjgS9M9kPV+xhGdhzOq/vRL3yRTYqDPoNExBbgD4GvAP8LPEm1TU1E7Ac+Wt//KfAx4M5DvM9B4MNUO9aepdo2/lj99P3AY8ALkl6qH/vLutb3Je0Dvgssq9/rHmBt/bon63/fJOn3JD02wUd7gmqtYTHw7/X0SRO8xiZBb92km6Ymqj2311Ft990YEdcWrHUzcAmwOyJOK1VnVL2lwL8Ax1OtFt8QEdcVrHcE8CDwdqpNsw0RMeml7CRr9gFbqPYRXFKyVl1vGHiV6nDcgYg4o2CtfuBG4DSq/79PRMRDhWotA74x6qFfAj4fEWsP+82ne28gVbifqj/UPOCHwKkF651DdVLKox19vhOA0+vpY4D/Lvz5BBxdT88FHgZ+vfBn/FPgVuDujr7TYeC4jmqtB/6gnp4H9HdUt4/qBKKT2ni/Xlh1PxN4MiKejmr183ZgZaliEfEg1aptJyLi+YjYVk+/CuygWkUtVS8i4rX67tz6Vmy1TdIS4GKqpd6sIukdVAuGm+DNk3te6aj8+cBTEXHIoxiT0QtBX8xbDyHtpGAQppOkAWA51VK2ZJ0+SUPAbuC+iChZby3wGeCNgjXGCuA7krZKWl2wzslU5xt8TdIjkm6UdFTBeqNdBtzW1pv1QtBTqH+k8i1gTUTsK1krIg5GxCCwBDhTUpF9EZJG9nVsnXDmdr0/Ik4HLgI+JemcQnXmUG3mXR8Ry6lOHPpsoVpvqo94XEp1dmMreiHou3jrcdgl9WOzhqS5VCG/JSLG3RNeQr2auQm4sFCJs4FL651jtwPnSfp6oVpvivp4fkTsBjZy6MOIh2sn1dmEI2tEG6iCX9pFwLaIeLGtN+yFoP8n8B5JJ9d/yS4Dvj3NPbWmPh58E7AjIr7cQb2F9Z5i6h+eXAA8XqJWRHwuIpZExADV/9v9EXFFiVoj6uP+x4xMAx8EHi1RKyJeAJ6r94ZDtd28/Re8pC2X0+JqO/TAmXERcUDSH1MdP+0Dbo6IiY67Tpmk26jO+z6u/jXXFyLiplL1qJZ6Hwf+q95uBviriPi3QvVOANbXh7zeBtwREXcXqjUdjgc21ufTzAFujYh7C9a7GrilXgg9DVxVsNbIH68LqH641N771rvyzWwW64VVdzMrzEE3S8BBN0vAQTdLwEE3S6Cngl74dMZpq+V6rjfd9Xoq6ECXX2an/3Gu53rTWa/Xgm5mBRQ5YUbSrD4LZ/78+RPPNMaBAweYM2dqJyIuWrRo0q/Zt28fxx577JTq9ff3TzzTGHv27GHhwoVTqnfw4OQv5/7yyy+zYMGCKdXbvn3yZ7EePHiQvr6+iWccx/79+6f0uqmKiJ+7DNe0nwI7Ey1bNuHIRq265pprOq23cmWxywGMa+/evZ3WGxwc7LTe8PBwp/XG41V3swQcdLMEHHSzBBx0swQcdLMEHHSzBBx0swQcdLMEGgVd0oWSnpD0pKTil7s1s3ZNGPT6IoP/QHUJ2lOByyWdWroxM2tPkyV6p0MmmVn7mgQ9zZBJZrNVaz9qqX8o3/Vvds2sgSZBbzRkUkTcANwAs/9nqmYzTZNV91k9ZJJZBhMu0bseMsnM2tdoG70eJ6zUWGFmVpjPjDNLwEE3S8BBN0vAQTdLwEE3S8BBN0vAQTdLwEE3S2BWjNSyYsWKTutt2rSp03rr16/vtN65557bab2uv8+BgYFO63mkFjPrhINuloCDbpaAg26WgINuloCDbpaAg26WgINuloCDbpaAg26WQJMhmW6WtFvSo100ZGbta7JEXwdcWLgPMytowqBHxIPATzvoxcwK8Ta6WQIee80sgdaC7rHXzHqXV93NEmhyeO024CFgmaSdkn6/fFtm1qYmgyxe3kUjZlaOV93NEnDQzRJw0M0ScNDNEnDQzRJw0M0ScNDNEnDQzRKYFWOvDQ4Odlrvqquu6rTeunXrOq23cuXKTut1bWhoaLpb6JyX6GYJOOhmCTjoZgk46GYJOOhmCTjoZgk46GYJOOhmCTjoZgk46GYJNLk45FJJmyRtl/SYpE930ZiZtafJue4HgD+LiG2SjgG2SrovIrYX7s3MWtJk7LXnI2JbPf0qsANYXLoxM2vPpLbRJQ0Ay4GHSzRjZmU0/pmqpKOBbwFrImLfOM977DWzHtUo6JLmUoX8loi4c7x5PPaaWe9qstddwE3Ajoj4cvmWzKxtTbbRzwY+Dpwnaai+fahwX2bWoiZjr20G1EEvZlaIz4wzS8BBN0vAQTdLwEE3S8BBN0vAQTdLwEE3S8BBN0tAEe2flu5z3We2rsd6GxgY6LTeihUrOq3XtYj4uRPcvEQ3S8BBN0vAQTdLwEE3S8BBN0vAQTdLwEE3S8BBN0vAQTdLwEE3S6DJVWCPkPQDST+sx177my4aM7P2NLmu+/8B50XEa/X13TdLuicivl+4NzNrSZOrwAbwWn13bn3zj1bMZpBG2+iS+iQNAbuB+yLCY6+ZzSCNgh4RByNiEFgCnCnptLHzSFotaYukLW03aWaHZ1J73SPiFWATcOE4z90QEWdExBltNWdm7Wiy132hpP56ej5wAfB46cbMrD1N9rqfAKyX1Ef1h+GOiLi7bFtm1qYme91/BCzvoBczK8Rnxpkl4KCbJeCgmyXgoJsl4KCbJeCgmyXgoJsl4KCbJdDkzDibZmvXru203pVXXtlpva51PbbcqlWrOq03Hi/RzRJw0M0ScNDNEnDQzRJw0M0ScNDNEnDQzRJw0M0ScNDNEnDQzRJoHPR6EIdHJPnCkGYzzGSW6J8GdpRqxMzKaTok0xLgYuDGsu2YWQlNl+hrgc8AbxTsxcwKaTJSyyXA7ojYOsF8HnvNrEc1WaKfDVwqaRi4HThP0tfHzuSx18x614RBj4jPRcSSiBgALgPuj4grindmZq3xcXSzBCZ1KamIeAB4oEgnZlaMl+hmCTjoZgk46GYJOOhmCTjoZgk46GYJOOhmCTjoZgl47LUZYGhoqNN6e/fu7bTemjVrOq131113dVqvF3iJbpaAg26WgINuloCDbpaAg26WgINuloCDbpaAg26WgINuloCDbpZAo1Ng60s9vwocBA74ks5mM8tkznU/NyJeKtaJmRXjVXezBJoGPYDvSNoqaXXJhsysfU1X3d8fEbskvQu4T9LjEfHg6BnqPwD+I2DWgxot0SNiV/3vbmAjcOY483jsNbMe1WQ01aMkHTMyDXwQeLR0Y2bWniar7scDGyWNzH9rRNxbtCsza9WEQY+Ip4Ff66AXMyvEh9fMEnDQzRJw0M0ScNDNEnDQzRJw0M0ScNDNEnDQzRJQRLT/plL7b5pY12OvPfDAA53W63rstdkuIjT2MS/RzRJw0M0ScNDNEnDQzRJw0M0ScNDNEnDQzRJw0M0ScNDNEnDQzRJoFHRJ/ZI2SHpc0g5JZ5VuzMza03QAh+uAeyPidyTNA44s2JOZtWzCoEt6B3AOsAogIvYD+8u2ZWZtarLqfjKwB/iapEck3VgP5PAWklZL2iJpS+tdmtlhaRL0OcDpwPURsRx4Hfjs2Jk8JJNZ72oS9J3Azoh4uL6/gSr4ZjZDTBj0iHgBeE7Ssvqh84HtRbsys1Y13et+NXBLvcf9aeCqci2ZWdsaBT0ihgBve5vNUD4zziwBB90sAQfdLAEH3SwBB90sAQfdLAEH3SwBB90sAY+9NgX9/f2d1hseHu603uDgYKf1uv58s53HXjNLykE3S8BBN0vAQTdLwEE3S8BBN0vAQTdLwEE3S8BBN0vAQTdLYMKgS1omaWjUbZ+kNV00Z2btmPDikBHxBDAIIKkP2AVsLNyXmbVosqvu5wNPRcQzJZoxszKaXtd9xGXAbeM9IWk1sPqwOzKz1jVeoteDN1wKfHO85z32mlnvmsyq+0XAtoh4sVQzZlbGZIJ+OYdYbTez3tYo6PV46BcAd5Ztx8xKaDr22uvAgsK9mFkhPjPOLAEH3SwBB90sAQfdLAEH3SwBB90sAQfdLAEH3SyBUmOv7QGm8lPW44CXWm6nF2q5nut1Ve+kiFg49sEiQZ8qSVu6+vVbl7Vcz/Wmu55X3c0ScNDNEui1oN8wS2u5nutNa72e2kY3szJ6bYluZgU46GYJOOhmCTjoZgk46GYJ/D8m3wmG2PzdNwAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 288x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPoAAAEMCAYAAADtUKuNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAASrUlEQVR4nO3df7BcZX3H8feHmwBCIFchIJJAFJQxoiYMg7UoRH5YfgmMdVrSQYyCyNSqGaNWHeuPalvaWpvMWKX8MqgIYoDUYQTBgRQZQUlCbIFAhRhMIpCAJiSMFQPf/vE8t26u9949Nznn3L15Pq+ZTPbuOXu+z57dz57n/Nh9FBGY2a5tt7FugJk1z0E3K4CDblYAB92sAA66WQEcdLMC7PJBl3SJpL/ZicfPlXRXnW0atPybJb2r4+8vSHpK0hOSDpG0VVLfDix3uqSQNKHeFo+epEWSvpBvv1nSwy3VDUmHt1Gr143boEtaI+k5SfsPuv++/AJPB4iIiyLi82PRxioi4tSIuApA0iHAfGBGRLw0In4REZMi4vmm25HX52/yB8uTOZyT6q4TET+MiCMqtKfRD9imSHqLpDskbZa0ZqzbM2DcBj37OTBn4A9JrwX2Grvm7LRDgKcjYsMY1X9bREwCjgKOBj41eIZe6CH0uGeBK4GPjnVDOo33oH8DOK/j73cBX++cYVC3cX9JN0naJOlXkn4oabc8bZqkGyRtlPS0pC8PVVDSQklrJT0jabmkN3dMO0bSsjztSUlfyvfvKembebmbJN0r6cA8bamkCySdBNwGvCxvVRcN7n5LmizpCkmPS1qfu/l9eVqfpC/mbv9q4PQdXakRsR64GTgyLzskvV/Sz4Cf5fvOkLQyP58fSXpdx3qYJWmFpC2Svg3s2TFttqR1HX//wXqX9GrgEuCNeV1syvPukZ/jL/L6vUTSizqW9dG8bn4p6T0jPUdJ75a0KrdxtaT3dUwb9n1SYd39JCK+AayuMn9bxnvQ7wH2lfTq/IY/B/jmCPPPB9YBU4ADgU8CkR97E/AYMB04GLh2mGXcC8wEXgJ8C/iOpIE38kJgYUTsCxwGXJfvfxcwGZgG7AdcBPymc6ER8QPgVOCXubs+d4jai4BtwOHALOCtwAV52nuBM/L9RwPv6HygpI9LummY57QdSdOA04D7Ou4+G3gDMEPSLNJW6335+fw78N0cxN2BJaQP4ZcA3wH+dJg6Q673iFhFWkd353XRnx9yMfAq0vo/PM//6bysU4CPACcDrwRO6vI0N5DW177Au4F/lXRUnjbk+6TL8npbRIzLf8Aa0ov5KeAfgFNIW8QJpBdlep5vEfCFfPtvgf8ADh+0rDcCG4EJQ9SZC9w1Qjt+Dbw+374T+Byw/6B53gP8CHjdEI9fClyQb88G1nVMm56fywTSG+63wIs6ps8B7si3bwcu6pj21oHHjmJ9bgU2kYL3lYFaeTkndMz7VeDzgx7/MHA8cBzwS0Ad037U8Rr8/3MczXoHROoWHzbodft5vn0lcHHHtFfldh9e8fkvAT400vtklO/Pk4A1Y52TgX/jfYsOacvxF6Q3xtdHnpV/Bh4Bbs3dtY/n+6cBj0XEtm7FJH0kd/k25y7lZGDggOD5pDfYQ7l7fkZHG78PXJu7lf8kaeIoniPAocBE4PHcpdxE2pIekKe/DFjbMf9jo1w+wNkR0R8Rh0bEX0ZEZ6+jc9mHAvMH2pHbMi234WXA+sjv9i5tqbzeSVvXvYDlHTVvyffDKJ+/pFMl3ZO75ptIPZiB13G498ngZXwy71pslXRJhecwZsZ90CPiMdJBudOAG7rMuyUi5kfEK4AzgQ9LOpH0BjlEXQ405f3xjwF/Brw4UpdyM2lrQ0T8LCLmkML3j8BiSXtHxO8i4nMRMQP4Y1KX8bwhiwxvLWmLvn8OY39E7BsRr8nTHycFZ8Aho1x+N53BXQv8XUc7+iNir4i4JrfjYEmq0JaR1vvgrvJTpN2d13TUnBzp4CGM4vlL2gO4HvgicGB+Hb/H71/H4d4n2zcw4u8j7VpMioiLhqvXC8Z90LPzSV3LZ0eaKR9AOjy/CTcDzwMvAD8hvVEulrR3Pnh27BCL2Ie0j7wRmCDp06R9vIHlnytpSkS8QOoCA7ygdMrltXmf9Bngd7luZRHxOHAr8C+S9pW0m6TDJB2fZ7kO+KCkqZJeDAy5FarJZcBFkt6gZG9Jp0vaB7ibtI4+KGmipLcDxwyznJHW+5PA1LzPT16nl5H2pQ8AkHSwpD/J818HzJU0Q9JewGdGaP/uwB6k13GbpFNJuzrk5Q73Pukqvy57knpfys9p9yqPbdIuEfSIeDQillWY9ZXAD0j7oncDX4mIOyKdp34b6QDPL0gHYv58iMd/n9Rd/B9S1/B/2b67eArwgKStpANz5+Tu70uBxaSQrwL+k9SdH63zSG/SB0nHBhYDB+Vpl+X2/RRYwaDeTe5m3rwDNf9AXtfvBb6c2/EIadeJiHgOeHv++1ek9ThkT6vLer8deAB4QtJT+b6/zrXukfQM6bU8Ii/rZmBBftwj+f/h2r8F+CDpw+HXpF2/73bMMuT7pMtqGXAcqefxPVKv4jekD+gxpe13pcaoEemI6UKgD7g8Ii5usNaVpK7zhog4sqk6HfWmkY4dHEjqjl4aEQsbrLcn6aDgHqSDeIsjYqStWx01+4BlpH3zM7rNX0O9NcAW0pZ2W0Qc3WCtfuBy0qnGAN4TEXc3VOsI4Nsdd70C+HRELNjphY/10UBSuB/NT2p30hZpRoP1jiNdEHJ/S8/vIOCofHsfUm+gyecnYFK+PRH4MfBHDT/HD5NONd7U0jpdw6AzGw3WuorfnxXZHehvqW4f8ARwaB3L64Wu+zHAIxGxOlK371rgrKaKRcSdpC5lKyLi8YhYkW9vIXXdD26wXkTE1vznxPyvsW6bpKmki3Mub6rGWJE0mbRhuALSbklEbBr5UbU5EXg00sHmndYLQT+Y7fdz19FgEMaS0vX3s0hb2Sbr9ElaSboo5LaIaLLeAtKZiFEdXNxJQTr1tVzShQ3WeTnpgN3XlL5DcbmkvRus1+kc4Jq6FtYLQS+C0hdErgfmRcQzTdaKiOcjYiYwFThGUiPHIvJ1AhsiYnkTyx/BmyLiKNKVhO+XdFxDdSaQdvO+GhGzSBfsNHk2A4B8lP5M0lWFteiFoK9n+/OfU/N9u4x8ccz1wNURMeK5/jrlbuYdpLMBTTgWODMfHLsWOEHSSJcg1yLStfhE+vLPjQx/+m5nrSNdxTfQI1pMCn7TTgVWRMSTdS2wF4J+L/BKSS/Pn2TnsP2pjnEtn4u9AlgVEV9qod6UfKQYpS98nAw81EStiPhEREyNiOmk1+32iDi3iVoD8vn2fQZuk85/399ErYh4Alibj4ZD2m9+sIlag8yhxm47pK7JmIqIbZL+inQOuA+4MiIeaKqepGtI11vvr/Qtqs9ExBVN1SNt9d4J/Hfebwb4ZER8r6F6BwFX5VNeuwHXRUSlL7OMEwcCN+YL7yYA34qIWxqs9wHg6rwRWk36Akxj8ofXyaQvDNW33Hwo38x2Yb3QdTezhjnoZgVw0M0K4KCbFcBBNytATwW94csZx6yW67neWNfrqaADba7MVl8413O9sazXa0E3swY0csGMJF+FU6Np06Z1n2mQrVu3MmnSjg20st9++436MU8//fQOPQ6gr2/UI06xceNGpkyZ0n3GIWzYMPrxMXZmfa5du7b7TDWKCA2+b8wvgbXu5s+f32q9uXPntlpv8uTJrdZbuLCxH/gZ0rx581qtNxR33c0K4KCbFcBBNyuAg25WAAfdrAAOulkBHHSzAjjoZgWoFHRJp0h6WNIjww0ha2a9q2vQ848M/hvpJ2hnAHMkzWi6YWZWnypb9FaHTDKz+lUJejFDJpntqmr7Ukv+onzb39k1swqqBL3SkEkRcSlwKfhrqma9pkrXfZceMsmsBF236G0PmWRm9au0j57HCWtqrDAza5ivjDMrgINuVgAH3awADrpZARx0swI46GYFcNDNCuCgmxXAI7XsgEWLFrVab+bMma3WO/vss1ut99nPfrbVemvWrGm1Xi/wFt2sAA66WQEcdLMCOOhmBXDQzQrgoJsVwEE3K4CDblYAB92sAA66WQGqDMl0paQNku5vo0FmVr8qW/RFwCkNt8PMGtQ16BFxJ/CrFtpiZg3xPrpZATz2mlkBagu6x14z613uupsVoMrptWuAu4EjJK2TdH7zzTKzOlUZZHFOGw0xs+a4625WAAfdrAAOulkBHHSzAjjoZgVw0M0K4KCbFcBBNyvALjH22llnndVqvdmzZ7dar+2x1/r7+1ut1/bzmzdvXqv1eoG36GYFcNDNCuCgmxXAQTcrgINuVgAH3awADrpZARx0swI46GYFcNDNClDlxyGnSbpD0oOSHpD0oTYaZmb1qXKt+zZgfkSskLQPsFzSbRHxYMNtM7OaVBl77fGIWJFvbwFWAQc33TAzq8+o9tElTQdmAT9uojFm1ozKX1OVNAm4HpgXEc8MMd1jr5n1qEpBlzSRFPKrI+KGoebx2GtmvavKUXcBVwCrIuJLzTfJzOpWZR/9WOCdwAmSVuZ/pzXcLjOrUZWx1+4C1EJbzKwhvjLOrAAOulkBHHSzAjjoZgVw0M0K4KCbFcBBNyuAg25WgF1i7LXNmzePdRMatWTJklbrHX/88a3Wa/v1W7lyZav1eoG36GYFcNDNCuCgmxXAQTcrgINuVgAH3awADrpZARx0swI46GYFcNDNClDlV2D3lPQTST/NY699ro2GmVl9qlzr/lvghIjYmn/f/S5JN0fEPQ23zcxqUuVXYAPYmv+cmP95gAazcaTSPrqkPkkrgQ3AbRHhsdfMxpFKQY+I5yNiJjAVOEbSkYPnkXShpGWSltXdSDPbOaM66h4Rm4A7gFOGmHZpRBwdEUfX1Tgzq0eVo+5TJPXn2y8CTgYearphZlafKkfdDwKuktRH+mC4LiJuarZZZlanKkfd/wuY1UJbzKwhvjLOrAAOulkBHHSzAjjoZgVw0M0K4KCbFcBBNyuAg25WgF1i7LWlS5e2Wm/mzJmt1mtb2+tz0aJFrdYrkbfoZgVw0M0K4KCbFcBBNyuAg25WAAfdrAAOulkBHHSzAjjoZgVw0M0KUDnoeRCH+yT5hyHNxpnRbNE/BKxqqiFm1pyqQzJNBU4HLm+2OWbWhKpb9AXAx4AXGmyLmTWkykgtZwAbImJ5l/k89ppZj6qyRT8WOFPSGuBa4ARJ3xw8k8deM+tdXYMeEZ+IiKkRMR04B7g9Is5tvGVmVhufRzcrwKh+SioilgJLG2mJmTXGW3SzAjjoZgVw0M0K4KCbFcBBNyuAg25WAAfdrAAOulkBFBH1L1Sqf6EFmz17dqv1lixZ0mq96dOnt1pv06ZNrdZrW0Ro8H3eopsVwEE3K4CDblYAB92sAA66WQEcdLMCOOhmBXDQzQrgoJsVwEE3K0Cl34zLP/W8BXge2OafdDYbX0bz45BviYinGmuJmTXGXXezAlQNegC3Slou6cImG2Rm9avadX9TRKyXdABwm6SHIuLOzhnyB4A/BMx6UKUtekSsz/9vAG4EjhliHo+9ZtajqoymurekfQZuA28F7m+6YWZWnypd9wOBGyUNzP+tiLil0VaZWa26Bj0iVgOvb6EtZtYQn14zK4CDblYAB92sAA66WQEcdLMCOOhmBXDQzQrgoJsVYDTfR7cxMm/evFbrLViwoNV6u/pYaL3AW3SzAjjoZgVw0M0K4KCbFcBBNyuAg25WAAfdrAAOulkBHHSzAjjoZgWoFHRJ/ZIWS3pI0ipJb2y6YWZWn6rXui8EbomId0jaHdirwTaZWc26Bl3SZOA4YC5ARDwHPNdss8ysTlW67i8HNgJfk3SfpMvzQA7bkXShpGWSltXeSjPbKVWCPgE4CvhqRMwCngU+PngmD8lk1ruqBH0dsC4ifpz/XkwKvpmNE12DHhFPAGslHZHvOhF4sNFWmVmtqh51/wBwdT7ivhp4d3NNMrO6VQp6RKwEvO9tNk75yjizAjjoZgVw0M0K4KCbFcBBNyuAg25WAAfdrAAOulkBPPbaONDf399qvaVLl7Zaz5rnLbpZARx0swI46GYFcNDNCuCgmxXAQTcrgINuVgAH3awADrpZARx0swJ0DbqkIySt7Pj3jKR5bTTOzOrR9Vr3iHgYmAkgqQ9YD9zYcLvMrEaj7bqfCDwaEY810Rgza8Zov712DnDNUBMkXQhcuNMtMrPaVd6i58EbzgS+M9R0j71m1rtG03U/FVgREU821Rgza8Zogj6HYbrtZtbbKgU9j4d+MnBDs80xsyZUHXvtWWC/httiZg3xlXFmBXDQzQrgoJsVwEE3K4CDblYAB92sAA66WQEcdLMCKCLqX6i0EdiRr7LuDzxVc3N6oZbruV5b9Q6NiCmD72wk6DtK0rK2vv3WZi3Xc72xrueuu1kBHHSzAvRa0C/dRWu5nuuNab2e2kc3s2b02hbdzBrgoJsVwEE3K4CDblYAB92sAP8HubnWgfDyDtUAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 288x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aPWp7xpDZI9i"
      },
      "source": [
        "1. What is the relationship between accuracy over test examples and training set size for the classifier? How can you design an experiment to do this?\n",
        "2. What is the impact of changing the value of k in the nearest neighbor classifier? Which value of k gives you the best validation performance?\n",
        "3. Here, we are using each pixel as a feature. Can you extract better features that allow you to do better classification? "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B4mgVXMNZI9i"
      },
      "source": [
        "## Using Perceptron for Digit Classification\n",
        "\n",
        "Use the codes provided above and compare the predictive performance of a perceptron for this problem. Compare the performance of the perceptron to the k-NN classifier. Which one is better and what is your intuition as to why that is the case.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vChQd0jQTUk-"
      },
      "source": [
        "# Your code goes here"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vP8LOePATY6K"
      },
      "source": [
        "Once you have trained and evaluated your perceptron, execute the following cell to show the internal representation of the weights of the perceptron (clf). What is your intuition about this?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6G73hKgKTn6Z"
      },
      "source": [
        "plt.gray();plt.matshow(clf.coef_.reshape((8,-1)))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LcvbJV_jTYDE"
      },
      "source": [
        ""
      ]
    }
  ]
}