Problem Set 3-ML-Preethi.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Problem Set 3- Question 1\n",
    "By Preethi Susan Abraham"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Generate a quadratic model with y=x+x2+e. x and e are both vectors which are normal mean 0, variance 1. Generate a monte-carlo sample from this model with length m = 20. Use the functions train_test_split() to get training and test sets where the test set is 1/2 of the full sample size, m. Fit three different models, a linear model, a quadratic model, and a 7 degree polynomial. (Use PolynomialFeatures() function for this.) Fit the three models to the training set and report their performance with R-squared on both the train and test sets. Hint: In this problem you are only generating one sample of length 20. Similar to the example from class, except now you have a quadratic term too."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LinearRegression\n",
    "\n",
    "np.random.seed(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def rsquared(y,yhat,ytrain):\n",
    "    RSS = np.sum( (y-yhat)**2)\n",
    "    yhatNaive = np.mean(ytrain)\n",
    "    naiveRSS = np.sum( (y-yhatNaive)**2)\n",
    "    rs = 1. - RSS/naiveRSS\n",
    "    return rs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model 1: Linear Model (y=X+e)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Intercept, coefficient:\n",
      "[array([0.25995179]), array([[0.70384945]])]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEGCAYAAABsLkJ6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXxV9Z3/8dcnIYFAkH13wQVQNoXgUtcgFXCpIq0zrdbitDNOZ6bza3/OMKPjTJ1f+7Mbv2lnOu38uhdqUVpHpNa2olWotS6VsK8KrhAUBAMEAoTkM398byTELDfJvfec3PN+Ph55JDl3OW8OcD7f+z3f8/2auyMiIslTEHUAERGJhgqAiEhCqQCIiCSUCoCISEKpAIiIJFS3qAO0x8CBA33kyJEdeu3Bgwfp1atXZgNlSFyzxTUXxDdbXHNBfLPFNRfEN1t7c1VUVLzj7oPe94C7d5mvsrIy76hly5Z1+LXZFtdscc3lHt9scc3lHt9scc3lHt9s7c0FrPBmzqnqAhIRSSgVABGRhFIBEBFJKBUAEZGEUgEQEUmoLjUMVEQk25as2sG8pVuorKqhfw/jX/rsYNakEVHHygoVABGRlCWrdnDX4nXU1NYBsOewc9fidQB5WQTUBSQikjJv6Zb3Tv4NamrrmLd0S0SJsksFQEQkpbKqpl3buzoVABGRlOF9S9q1vatTARARSZk7YwwlRYUnbCspKmTujDERJcouXQQWEUlpuNB7wiigGybk5QVgUAEQETnBrEkj3jvhL1++nPI8PfmDuoBERBJLBUBEJKFUAEREEkoFQEQkoVQAREQSSgVARCShVABERBJKBUBEJKF0I5hIBjTMIb+jqoYRzz/F3Blj8vbuUckfKgAindR0DvkdVTV5PYe85A91AYl0UtLmkJf8oQIg0klJm0Ne8ocKgEgnJW0OeckfKgAinZS0OeQlf+gisEgnNZ5DfkdVDSP6lmgUkHQJKgAiGdAwh/zy5cspLy+POo5IWtQFJCKSUCoAIiIJpQIgIpJQKgAiIgmlAiAiklAqACIiCaUCICKSUCoAIiIJFVkBMLNTzGyZmW0ysw1m9tmosoiIJFGUdwIfA/7O3VeaWW+gwsyecPeNEWYSEUmMyD4BuPtOd1+Z+vkAsAnQ5CkiIjli7h51BsxsJPA0MN7d9zd57HbgdoAhQ4aULVq0qEP7qK6uprS0tHNBsySu2eKaC+KbLa65IL7Z4poL4putvbmmTp1a4e5T3veAu0f6BZQCFcDstp5bVlbmHbVs2bIOvzbb4potrrnc45strrnc45strrnc45utvbmAFd7MOTXSUUBmVgQ8BCx098VRZhERSZooRwEZ8ENgk7t/PaocIiJJFeUngEuAW4ErzWx16uuaCPOIiCRKZMNA3f0ZwKLav4hI0ulOYBGRhFIBEBFJKK0JLCIsWbWDeUu3UFlVw3Atap8YKgAiCbdk1Q7uWryOmto6AHZU1XDX4nUAKgJ5Tl1AIgk3b+mW907+DWpq65i3dEtEiSRXVABEEq6yqqZd2yV/qACIJNzwviXt2i75QwVAJOHmzhhDSVHhCdtKigqZO2NMRIkkV3QRWCThGi70ahRQ8qgAiAizJo3QCT+uaqpg3YNw7sege2anplYBEBGJG3d443lYuQA2PAzHDkOvQTBuVkZ3owIgIhIXh/bCmgegYgG8swWKe8N5N8PkOTD8vIzvTgVAJEZ0R24CucNrz4TW/sZfQN1RGDEFrv8WjLsx490+jakAiORIWyd33ZGbMNW7Yc39obW/dxt07wNlt4XW/tDxOYmgAiCSA+mc3Fu7I1cFIE/U18Orv4OK+bD5V1BfC6d+AC6fC2NvgOKeOY2jAiCSA+mc3HVHbh478BasXhha+1WvQ0k/uOAvQmt/8NmRxVIBEMmBdE7uw/uWsKOZ5+mO3C6qvg62PRVa+1t+A14HIy+DaZ+Hs6+Doh5RJ1QBEMmFdE7uc2eMOaGbCHRHbpe0bwes+imsug/2vQk9B8IH/ia09geeFXW6E6gAiORAOid33ZHbhdUdg61PhC6el5eC18MZU2H6F2HMtdCtOOqEzVIBEMmBdE/uuiO3i6l6I7T2V94HByqhdAhc8jmYfCv0PyPqdG1SARDJEZ3c80RdLbz0WOjb3/pk2HbWNLjmazB6JhQWRRqvPVQARETSsfcVWPkTWLUQDu6C3sPhin+ASR+HvqdGna5DVABERFpg9bWwfnFo7b/6O7ACGDUj3LB11gehsGufQrt2ehGRbHhnK6yczwdeXAC1+6HPKTD1bjjvFuiTP914KgAiIgC1h2HTL0Nr//VnoKAb+/qfz6AZfw9nToWCwjbfoqtRARCRZNu1OUzEtuYBqHkX+o2EaffAebewoWIT5aPKo06YNSoAIpI8Rw/BxiVh3P6bz0NBEZxzXbhZ6/QroKBhtdxNkcbMNhUAEUmOt9aFk/7an8ORfTDgLLjqi2HO/V4Do06XcyoAIpLfjlTDhsXhxL9jBRR2DzNvls2B0y4Bs6gTRkYFQETyU+Xq0Le/9kE4egAGnQ0zvgznfhR69o86XSyoAIhI/ji8H9b/d2jt71wN3XrAuNlh3P4pFyS6td8cFQAR6drcYUdFGL65fjHUHoTB4+DqeTDxpjD3vjRLBUBEuqaaqnAxd+UCeHs9FPWE8bOh7M9gRJla+2lQARCRrsMd3nwhtPY3LIFjNTDsXLjuGzD+I9DjpKgTdikqACISf4f2wppFobW/ezMU9w4Xc8vmwPBJUafrsiItAGb2I+A6YJe7j48yi4jEjDu89kw46W98BOqOwIgpcP1/hgu73UujTtjltVkAzGwR8GPgcXf3DO9/PvAt4CcZfl8R6aqqd8Oa+8PUy3u2Qvc+oaU/eQ4MVTsxk9L5BDAf+CTwLTP7GTDf3bdmYufu/rSZjczEe0n8LVm1Q8sdSvPq68N0yysXwKZHob4WTrkILvv7cNNWcc+oE+YlS7dRb2b9gFuAfwReBb4PPODuxzoVIBSAR1vqAjKz24HbAYYMGVK2aNGiDu2nurqa0tJ4fmSMa7ZM5nq2spb5649ytP74tuICuG18MRcPb/8KSkk4ZpkWx2zFR96l3xu/ZuSepyk5/Ba13Xrz1tBydg6bzqFe0S+yEsdjBu3PNXXq1Ap3n9J0e1oFIHXyvxn4BPAOcD9wKTDK3T+Ydorm33skrRSAxqZMmeIrVqzo0H6WL19OeXl5h16bbXHNlslcl3zlKXZU1bxv+4i+Jfzhzivb/X5JOGaZFpts9XWwbRlU/DgsrVh/DEZeFrp4zvkQFPWIOuF7YnPMmmhvLjNrtgCkcw3g58AEwkn/w+6+PfXQQjNblXYCSbTKZk7+rW2XPLS/8vgC6vvegJ4D4KK/4oVjZ3PhNR+POl0ipXMN4AfAE81dAHZ3jb+StAzvW9LsJ4DhfUsiSCM5U3cMtv42jNt/eSl4PZxRDlf9Hzj7WujWnZrly6PNmGBtFgB3fzxbOzezB4ByYKCZbQfucfcfZmt/Ep25M8Zw1+J11NTWvbetpKiQuTPGRJhKsqbqjeOt/QOVUDoELvkcTL4V+p8RdTpJifQ+AHf/WJT7l9xpGO2jUUB5rK429OlXLAitfoCzpsE1X4PRM6Gw/Rf7Jbt0J7DkzKxJI3TCz0d7Xw1j9lcvhOq3ofdwuHxuaO33jX4kj7QsrQJgZr3c/WDD92yHEpGYO3YUNj8axu2/shysAEZND9Mun3UVFKpt2RWk+7f0e2Byo+8ikkTvbA0n/dX3w6F3oM8pMPVuOO8W6KNPd11Ne8u05lcVSZraw6G1XzEfXvs9WCGMuTq09s+8EgoKo04oHaTPaSLSvN1bwgXdNfdDzbvQbyRM+3xo7fceGnU6yQAVABE5rrYmzLO/cgG88RwUFIXx+mW3welXQEFB1Aklg9pbADI9G6iIZFiHJt17e0Po4ln7Mzi8D/qfCVd9Ac69GUoH5SR3e2lywc5LtwBYk+8iEkNLVu044Ya7HVU13LV4HQB9mz756MGwhm7FfNixAgqL4ZzrQ2t/5KWxXlKxtT+nikD60i0ANzf5LiIxNG/plhPutgaoqa1j3tIt3HtRqvumcnXo4ln7IBw9AAPHwIwvwcSPQq8BEaRuv9b+nCoA6UurALj7psbfRSSeWppcb1/VXoZVVsB374Gdq6FbDxh3Y5iB89SLYt3ab44mF8wMXQQWySMnTrrnTLRX+FjhU9zQ7Tl6vnQYBo+Dq+fBxJugpF+kWRtrb3++JhfMDBUAkTwyd8YY7l38AjPqn+bmwqcYW/A6h7w7u069jj0DL6TsQ7fHrrX/bGUt9z3Zvv58TS6YGW2O6TKz2elsE5EIucMbLzDr9Xt5rviv+L9FP6YO42vdPs1T1/6ekZ/8EQdOGhO7kz/AQy/Vttif35JZk0bw5dkTGNG3BCMsLPTl2RPU/99O6XwC+GdgcZNtdzezTURy7dDeMHSzYgHs3gTFpXQ772NQNocJwycxIep8adhzuPnR5W3152tywc5rsQCY2QxgJjDCzL7e6KGTgPrmXyUiWecOrz8bhm9u/AXUHYERU+BD34TxH4bu8VvDtjUDelizRUD9+dnX2ieAXcB64DCwodH2A8Cd2QwlEqXY3mB08B1Y80Bo7e95Gbr3gbI5YSTP0DaX1I6tD48u4r5NderPj0CLBcDdVwGrzGwhocV/qrtvzVkykQjE7gaj+np47enQ2t/0KNTXwikXwWV3wNhZUNwz95ky7OLhRYw9Z2w8i26eS+cawDTg60AxcLqZnUdYuvHGrCYTiUBsbjA68HZYYGXlAnj3NejRFy74C5j8CRh8Tu5y5Ij686ORTgH4AnAhsAzA3Veb2VlZTSUSkUhvMKqvg23LYOV82PIbqD/GShvHgqN/w9rul/PZIROYNVgnScmcdApArbtX2YnDxzQpnOSlSG4w2l95fAH1fW9AzwG8fMat/K8tE9lUOyQ8Z1+d5rqRjEtnbtdNZvYnQIGZnW5m/w48n+VcIpGYO2MMJUUnLnCSlQuSdcdgy2Nw/0fhG+Ng2b3Q/3T4yI/hjk3ctv364yf/lLbGxou0VzqfAD4DfJ5wIfhhYCnwT9kMJRKVhtZ11i5IVr0Jq+4LLf79O6DXYLjks6Fvv/8Z7z1Nc91ILrRZAFKLwP9j6ksk72X8gmRdLby0NFzQffmJsO2saTDzK2FpxcKi971Ec91ILrRZAMzsYd7f578PWAF8392PZiOYSJf37muw8iewaiFUvwW9h8Hlc2HSx6Hfaa2+VHPdSC6k0wX0JjAUeCD1+58Ce4GJwPeBOdmJJtL1WH1tWFKxYj68sgysAEZNDzdrjZoOhenNv5j1rigR0isA57r7FQ2/mNkS4HfufrmZbcxeNJEuZM82qJjPB15cALX7oM8pUP5PobXfp2MnbY2Nl2xLpwAMMbOT3X176vfhQMMioUeyE0ukC6g9DJsfDa39134PVsj+/lMYOHMunHklFBS2+RYiUUqnAPwD8JyZbSasCTwa+IyZ9QIWZjOcSCzt3hLm41nzANTshb6nwZX/ApM+zvqKzZSPKo8sWmznMZJYarUAmFkB8DbhpD+WUAA2uHvD8IT/l914IjFRWxNm3qxYAG88CwVFcPa1YTK208uhoOGWms2RRYzdPEYSe60WAHevN7P/cPeLgIocZRKJj7c3hi6etYvg8D7ofyZc9QU492YoHdTmy3MpNvMYSZeRThfQE2Z2g7v/IutpROLg6EHY8HA48W9/EQqL4ZzrQ2t/5GWxXFULdPOYtF+6dwL3MbMjQA2hG8jdvX9Wk4nk2s614aS/7kE4sh8GjoEZX4KJH4VeA6JO1ybdPCbtlU4BGJj1FCJROXIA1j8UTvyVq6BbjzDPftltcOpFsW3tN0c3j0l7pTMVRJ2Z9QHOBHo0eujZrKUSySZ3qFyZau0/BLUHYfBYuPprMPFPoKRf1Ak7RDePSXulMxXEp4A7gBHAOuB8wmyg5Z3duZnNBP4DKAR+4O5f6ex7irTo8D5Y+/MwJ89b66CoJ4ybHfr2Tz6/S7X2W6Kbx6Q90ukC+hwwBXjO3S8zs3HAP3d2x2ZWCHwbuArYDrxoZo+4u+4ulsxxDxdyK+bD+sVwrAaGToBr/w0m3AQ9+kSdUCQy6RSAw+5eY2aYWbG7bzCzszOw7wuAre7+CoCZLQJuAFQApPMO7YW1Pwvj9ndvguJSOPdPw5w8wyflRWtfpLPMvfnFvcysm7sfM7NHgE8AfwdcSpgIrpe7z+zUjs0+Asx09z9P/X4rcKG7f6bJ824HbgcYMmRI2aJFizq0v+rqakpLSzsTOWvimi2uuaCFbO702beRYTsfZ/CuP1DgtezvPYqdw6aza/Bl1HXL/miYLnfMYiCuuSC+2dqba+rUqRXuPuV9D7h7s1/Ayma2TQNmA91bel26X8BNhH7/ht9vBf6ztdeUlZV5Ry1btqzDr822uGaLay73Jtmqd7v/4Zvu3yxzv+ck9y+d7P7oHe6Va6LNFTNxzRbXXO7xzdbeXMAKb+ac2loX0Ps+I7v7k2mXnLZtB05p9PvJQGUG31/ymdfDK78Lffubfgn1tXDKhXDZ/w/DOIt7Rp1QJPZaKwCDzOyOlh509693ct8vAqPM7HRgB/BR4OZOvqfku+pdsHohF/zxu/C7ndCjL5z/qTBuf/A5UacT6VJaKwCFQCnNfBLIBA/XFz5DWGO4EPiRu2/Ixr6ki6uvD4urVMyHLb+G+mMc7TOOnld/IUzRUNSjzbcQkfdrrQDsdPcvZHPn7v5r4NfZ3Id0Yft3wuqfhmUVq96AngPgwk/D5Dms3lBJ+cTyqBOKdGntugYgknX1dbD1t2H45kuPgdfB6ZfDtHvgnA9Bt+6pJ3bdy0Was1/iorUCMC1nKUSq3oRVPw1f+7dDr0Fw8d/C5E/AgDOjTpcxmrNf4qTFAuDue3MZRBKo7hi8vDT07W/9bbhr98wrYeaXYPTV0K046oQZpzn7JU7SuRNYJLPefQ1W3hda+9VvQelQuPQOmHwr9BsZdboOW7JqB19cfoi9j/2qxa4dzdkvcaICILlx7GgYwbNyAWxbFqZiOOsqKPsGjJoOhV37n+Lxrp1wZ31LXTuas1/ipGv/r5P427MtnPRX3w8Hd8NJJ0P5nTDp49Dn5KjTZUy6XTuas1/iRAVAMu/YkXB3bsV8eO33YIUwema4WeusaVBQGHXCjEu3a0dz9kucqABI5ux+6Xhrv2Yv9D0VrvwXOO8WOGlY1Omyqj1dO5qzX+JCBUA6p7YGNj4STvyv/wEKusHZ14Zpl8+YCgUFUSfMCXXtSFekAiAd8/bGcNJfswgOV0H/M+CD/xpa+6WDo06Xcw0t+i/+Yg17D7u6dqRLUAGQ9B09CBseDnfpbv8jFBaHu3Mnz4GRlyWmtd+SWZNG0Hffy5SXl0cdRSQtKgDStp1rUwuoPwhH9sPA0TD9Xjj3Y9BrQNTpRKSDVACkeUcOMKzycfjev0LlKijsDuNmhZE8p35ASyqK5AEVADnOHSpXhi6e9Q8x5mg1DB4LM78KE/8EevaPOqGIZJAKgMDhfbD25+Gi7lvroKgnjJvNSpvA5Ov/Uq19kTylApBU7rD9xdC3v34xHKuBoRPg2n+DCTdBjz7sX75cJ3+RPKYCkDSH9h5v7e/aCEW9QvdO2W0wfJJO+CIJogKQBO7wxnOhtb9hCdQdCSf7D/0HjP8wdO8ddUIRiYAKQD47uAfWPBBa+++8BN1PCpOwlc2BYedGnU5EIqYCkG/cwwRsFQtg0yNQdxROPh9u+DaMuxGKe0WdUCSntARny1QA8kX1bli9MCygvncb9OgDUz4Z7tIdMjbqdCKR0BKcrVMB6Mrq6+GVZaGLZ/OvoP4YnHoxXPEPMPYGKNIiI5JsWoKzdSoAXdH+nbD6p2FZxarXoaQ/XPjp0NofNDrqdCKxoSU4W6cC0FXU18HWJ8NInpceA6+D0y+HaZ8PE7J1697mW6gvVJJGS3C2TgUg7vZtD4unr7wP9m+HXoPg4r+FyZ+AAWem/TbqC5Uk0joNrVMBiKO6Y/Dy46G1v/WJMLLnzKkw80sw+mroVtzut1RfqCSRluBsnQpAnLz7Oqy6L7T4D+yE0qFw6R0w+VboN7JTb62+UEkqLcHZMhWAqNXVMnD3s3DfN2HbU2EqhrM+GObkGTUDCjPzV9TRvlBdNxDJXyoAUdmzLYzZX72Q8Qd3w0kjoPzOcKdun5MzvruO9IU+W1nLfU/quoFIvlIByKVjR2Dzo6Fv/9WnwQph9EzWFk9m4o3/GwoKs7brjvSFPvRSLTW1fsI2XTcQyR8qALnwzsvhpL/mATi0B/qeClf+M5z3cThpGHuXL8/qyb9Be/tC9xz2ZrfruoFIflAByJbaw2EunooF8PozUNANxlwTpl0+Y2qXWEB9QA9rtghoDLVIflAByLRdm8JJf80DcLgK+p0O0+4Jffulg6NO1y4fHl3EfZvqNIZaJE+pAGTC0UOw4eHQzbP9j1BYDGdfF1r7Iy/rEq395lw8vIix54zVKCCRPKUC0Bk714aJ2Nb+HI7shwGjYPq9cO7HoNeAqNNlhMZQi+SvSAqAmd0E/CtwDnCBu6+IIkeHHKmG9Q+F1n7lSijsHmbeLLsNTrtYSyqKSJcR1SeA9cBs4LsR7b993KFyVWoB9YfgaDUMOgdmfjWsp9uzf9QJRbJGNwPmr0gKgLtvArC4t5YP74N1D4aLum+thW4lMH52aO2ffL5a+5L3NIlgfjP35sd652TnZsuBv2+tC8jMbgduBxgyZEjZokWLOrSv6upqSktL236iOyftf4lhO5cyeNczFNYfobrX6VQOn87bQ66grlvml1RMO1uOxTUXxDdbXHNBx7L93fJDzQ4FHtDD+LfynpHlypW4ZmtvrqlTp1a4+5Sm27P2CcDMfgsMbeahu939F+m+j7t/D/gewJQpU7y8vLxDeZYvX06rr615N1zMrZgPuzZCUS8470+h7DZKh09mtBnZWmqlzWwRiWsuiG+2uOaCjmXb+9ivmt9+2DP258y3Y5YLmcqVtQLg7h/M1ntnjDu88Vzo4tm4BI4dhuGT4Lp/hwkfge69o04oEiktqJLfkjkM9NDecKNWxXx45yXofhKcdwuUzYFh50adTiQ2tKBKfotqGOiNwH8Cg4Bfmdlqd5+R1Z26hwnYKhaEKRrqjsLJF8AN34ZxN0Jx5vv2Rbo6LaiS36IaBfQw8HDOdrjqp1zwx3vhd5XQow+U/Vlo7Q8Zl7MIIl2VbgbMX8noAjrwFkeL+9Jz5j3hpq0i9V+KiCSjAFx6B6vrp1B+bnnUSUREYqNrzlLWXl10MjYRkWxKxieAmNAt9SISJyoAOdLaLfV9owwmIomlvpEcmbd0ywljqeH4+roiIlFQAciRltbR1fq6IhIVFYAcaenWed1SLyJRUQHIkbkzxlBSVHjCNt1SLyJRUgHIkVmTRvDl2RMY0bcEA0b0LeHLsydoFJCIREajgHJIt9SLSJzoE4CISEKpAIiIJJQKgIhIQqkAiIgklAqAiEhCqQCIiCSUCoCISEKpAIiIJJQKgIhIQqkAiIgklAqAiEhCqQCIiCSUCoCISEKpAIiIJJQKgIhIQmk9gARbsmoH85ZuobKqhuF9S5g7Y4zWKxBJEBWAhFqyagd3LV5HTW0dADuqarhr8ToAFQGRhFAXUELNW7rlvZN/g5raOuYt3RJRIhHJNRWAhKqsqmnXdhHJPyoACTW8b0m7totI/lEBSKi5M8ZQUlR4wraSokLmzhgTUSIRyTVdBE6ohgu9GgUkklwqAAk2a9IInfBFEiySLiAzm2dmm81srZk9bGZ9o8ghIpJkUV0DeAIY7+4TgZeAuyLKISKSWJEUAHd/3N2PpX59Hjg5ihwiIklm7h5tALNfAj9z95+28PjtwO0AQ4YMKVu0aFGH9lNdXU1paWmHc2ZTXLPFNRfEN1tcc0F8s8U1F8Q3W3tzTZ06tcLdp7zvAXfPyhfwW2B9M183NHrO3cDDpApRW19lZWXeUcuWLevwa7Mtrtnimss9vtnimss9vtnimss9vtnamwtY4c2cUyP7BGBmc4BPA9Pc/VCar9kNvN7BXQ4E3unga7Mtrtnimgvimy2uuSC+2eKaC+Kbrb25TnP3QU03RlIAzGwm8HXgCnffnaN9rvDmPgLFQFyzxTUXxDdbXHNBfLPFNRfEN1umckU1CuhbQG/gCTNbbWbfiSiHiEhiRXIjmLufFcV+RUTkuCTNBfS9qAO0Iq7Z4poL4pstrrkgvtnimgvimy0juSIfBioiItFI0icAERFpRAVARCSh8rYApDvhnJnNNLMtZrbVzO7MUbabzGyDmdWbWYtDuczsNTNblxoptSJGuaI4Zv3N7Akzezn1vV8Lz6tLHa/VZvZIFvO0egzMrLuZ/Sz1+AtmNjJbWdqZ6zYz293oGP15LnKl9v0jM9tlZutbeNzM7Jup7GvNbHJMcpWb2b5Gx+zzOcp1ipktM7NNqf+Xn23mOZ07Zs3dHZYPX8B0oFvq568CX23mOYXANuAMoBhYA4zNQbZzgDHAcmBKK897DRiYw2PWZq4Ij9nXgDtTP9/Z3N9n6rHqHGRp8xgAfw18J/XzRwnTncQh123At3L1b6rJvi8HJgPrW3j8GuA3gAEXAS/EJFc58GgEx2sYMDn1c2/CxJlN/z47dczy9hOApzfh3AXAVnd/xd2PAouAG3KQbZO7x2719TRzRXLMUvtYkPp5ATArB/tsSTrHoHHe/wammZnFIFdk3P1pYG8rT7kB+IkHzwN9zWxYDHJFwt13uvvK1M8HgE1A0wU8OnXM8rYANPFJQpVsagTwZqPft/P+AxwlBx43s4rUpHhxENUxG+LuOyH8xwAGt/C8Hma2wsyeN7NsFYl0jsF7z0k1RPYBA7KUpz25AD6c6i74bzM7JcuZ2iPO/x8/YGZrzOw3ZjYu1ztPdSFOAl5o8lCnjlmXXhHMzH4LDG3mobvd/Rep59wNHAMWNvcWzWzLyLjYdLKl4RJ3rzSzwYS7pjenWitR5orkmLXjbU5NHbMzgKfMbJ27b8tEvkbSOQZZO06tSGefvwQecPcjZvZpwqeUKw/OTSgAAARySURBVLOcK11RHLN0rCTMpVNtZtcAS4BRudq5mZUCDwGfc/f9TR9u5iVpH7MuXQDc/YOtPZ6acO46woRzzR2U7UDjFtDJQGUusqX5HpWp77vM7GHCR/xOFYAM5IrkmJnZ22Y2zN13pj7i7mrhPRqO2StmtpzQasp0AUjnGDQ8Z7uZdQP6kP1uhjZzufueRr9+n3B9LC6y9m+rMxqfdN3912b2X2Y20N2zPkmcmRURTv4L3X1xM0/p1DHL2y4gCxPO/SNwvbc82+iLwCgzO93MigkX67I2cqQ9zKyXmfVu+JlwUbvZUQo5FtUxewSYk/p5DvC+Tytm1s/Muqd+HghcAmzMQpZ0jkHjvB8BnmqhEZLTXE36h68n9CvHxSPAJ1IjWy4C9jV0+0XJzIY2XL8xswsI5809rb8qI/s14IfAJnf/egtP69wxy/WV7Vx9AVsJfWOrU18NIzKGA79u9LxrCFfXtxG6QXKR7UZC5T4CvA0sbZqNMJJjTeprQy6ypZMrwmM2AHgSeDn1vX9q+xTgB6mfLwbWpY7ZOuBTWczzvmMAfIHQ4ADoATyY+nf4R+CMHB2ntnJ9OfXvaQ2wDDg7F7lS+34A2AnUpv6dfYowJfynU48b8O1U9nW0MkIux7k+0+iYPQ9cnKNclxK6c9Y2Oo9dk8ljpqkgREQSKm+7gEREpHUqACIiCaUCICKSUCoAIiIJpQIgIpJQKgCSl5rMCrq6I7NxmllfM/vrzKcTiQcNA5W8ZGbV7l7ayfcYSZgFcnw7X1fo7nWd2bdILugTgCSGmRVaWCfixdRkaH+Z2l5qZk+a2UoL6y80zKD5FeDM1CeIeal54R9t9H7fMrPbUj+/ZmafN7NngJvM7Ewzeyw1kd/vzezsTmY/zcJaCAPNrCD1ntM7854iXXouIJFWlJjZ6tTPr7r7jYQ7PPe5+/mpKSP+YGaPE+4Yv9Hd96emkHjewmIydwLj3f08CAuDtLHPw+5+aeq5TxLu1nzZzC4E/osmk66Z2VTgG828zyF3v7jxBnd/3cy+CnyHMCPkRnd/PM1jIdIsFQDJVzUNJ+5GpgMTzewjqd/7EGZ13A58ycwuB+oJ0+kO6cA+fwbvzd54MfBgoyUAujd9srsvA5pmbJG7/8DMbiJMBZD260RaogIgSWLA37r70hM2hm6cQUCZu9ea2WuEuXyaOsaJ3aZNn3Mw9b0AqGqmAJ0Yph2fAFLP78nxhY1KgQOtvb9IW1QAJEmWAn9lZk+lTvSjgR2ETwK7UtumAqelnn+AsBRfg9eBsanuox7ANOCZpjtJdSW9amY3ufuDqVkdJ7r7mibPa9cnAMLUzQtTOb5PmOpcpMN0EViS5AeE6aFXWlgA/LuERtBCYIqZrQBuATbDe3Pn/8HM1pvZPHd/E/g5YXbGhcCqVvZ1C/ApM2uYzbVTSzOa2RXA+YS1kBcCR83szzrzniIaBioiklD6BCAiklAqACIiCaUCICKSUCoAIiIJpQIgIpJQKgAiIgmlAiAiklD/AwJ2mpF9GIzKAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Linear Model (y=X+e)\n",
    "\n",
    "m=20\n",
    "n=1\n",
    "\n",
    "X = np.random.normal(size=(m,n),loc=0,scale=1.)\n",
    "e = np.random.normal(size=(m,1),loc=0,scale=1.)\n",
    "y = X + e #Linear Model\n",
    "\n",
    "lr = LinearRegression(fit_intercept=True)#Linear Regression\n",
    "lr.fit(X, y) #fitting the model\n",
    "\n",
    "xfeature = np.arange(start=-2.0,stop=2.0,step=0.05)\n",
    "xf = xfeature.reshape(len(xfeature),1) #converting to matrix form\n",
    "\n",
    "#Prediction\n",
    "yhat = lr.predict(xf)\n",
    "# plot data\n",
    "plt.plot(X,y,\"o\")\n",
    "# plot regression line (target predictions)\n",
    "plt.plot(xf,yhat)\n",
    "plt.xlabel(\"Feature = x\")\n",
    "plt.ylabel(\"Target = y\")\n",
    "plt.grid()\n",
    "\n",
    "print(\"Intercept, coefficient:\")\n",
    "print([lr.intercept_,lr.coef_])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Full sample R-square:\n",
      "0.2416695155657037\n"
     ]
    }
   ],
   "source": [
    "#R squared value\n",
    "print(\"Full sample R-square:\")\n",
    "print(lr.score(X,y))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.2416695155657037\n"
     ]
    }
   ],
   "source": [
    "#Same as above but with created function 'rsquared'\n",
    "\n",
    "yhat = lr.predict(X)\n",
    "# Send in y itself as training data \n",
    "# In this case test = train\n",
    "rsq = rsquared(y,yhat,y)\n",
    "print(rsq)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train/test R-squares: (scikit-learn version)\n",
      "0.508655237064873\n",
      "-0.3844957310512862\n",
      "Train/test R-squares: (scikit-learn versions)\n",
      "0.508655237064873\n",
      "-0.3844957310512862\n"
     ]
    }
   ],
   "source": [
    "#Training and testing\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.5)\n",
    "# Now run regression on training data \n",
    "lr.fit(X_train, y_train)\n",
    "\n",
    "\n",
    "# First rsquared by hand with function\n",
    "yhat_train = lr.predict(X_train)\n",
    "yhat_test  = lr.predict(X_test)\n",
    "\n",
    "# R-squared (my version using training data for sample mean)\n",
    "rsq_train = rsquared(y_train,yhat_train,y_train)\n",
    "rsq_test  = rsquared(y_test, yhat_test, y_train)\n",
    "\n",
    "# R-squared (scikit learn version, using test data for sample mean)\n",
    "rsq_train = rsquared(y_train,yhat_train,y_train)\n",
    "rsq_test  = rsquared(y_test, yhat_test, y_test)\n",
    "\n",
    "print(\"Train/test R-squares: (scikit-learn version)\")\n",
    "print(rsq_train)\n",
    "print(rsq_test)\n",
    "\n",
    "# Print R-square values on training and test data\n",
    "print(\"Train/test R-squares: (scikit-learn versions)\")\n",
    "print(lr.score(X_train,y_train))\n",
    "print(lr.score(X_test,y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model 2: Quadratic Model (y=X+X^2+e)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import PolynomialFeatures"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X dimensions\n",
      "(20, 2)\n",
      "R squared for quadratic model= 0.2994469137718183\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEGCAYAAABsLkJ6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3de3hU5bn38e+dkIRAkMjBIKAiiuCxRfBQDzWIirXuiqg92aq7ttT6tm93tbRare7W993W4ra7rbtbq7W21opaEa0nVCSeUYOAgMhJRQkICAYIBMjh3n+siYSQhJlkZtbKrN/nuuaayZqZte5nBp571rOeg7k7IiISP3lhByAiIuFQAhARiSklABGRmFICEBGJKSUAEZGY6hZ2AKno16+fDxkyJG3727JlCz179kzb/sKickRLrpQDcqcscS/H7NmzP3b3/i23d6kEMGTIECorK9O2v4qKCsrLy9O2v7CoHNGSK+WA3ClL3MthZita264mIBGRmFICEBGJKSUAEZGYUgIQEYkpJQARkZjqUr2ARETiZtqcKiZPX8yq6lr6dDd+3ruK8SMHpWXfSgAiIhE1bU4VV0+dT21dAwDrtzlXT50PkJYkoCYgEZGImjx98aeVf5PaugYmT1+clv0rAYiIRNSq6tqUtqdKCUBEJKIGlhantD1VSgAiIhE1adxwigvyd9lWXJDPpHHD07J/XQQWEYmopgu9u/QCOudI9QISEYmD8SMHfVrhV1RUUJ6myh/UBCQiEltKACIiMaUEICISU0oAIiIxpQQgIhJTSgAiIjGlBCAiElNKACIiMaWBYCJp0jRve1V1LYNmPcekccPTNmJTJBOUAETSoOW87VXVtWmdt10kE9QEJJIGmZ63XSQTlABE0iDT87aLZIISgEgaZHredpFMUAIQSYNMz9sukgm6CCySBs3nba+qrmVQabF6AUnkKQGIpEnTvO0VFRWUl5eHHY7IHqkJSEQkppQARERiSglARCSmlABERGJKCUBEJKaUAEREYkoJQEQkppQARERiKrQEYGb7mdlMM1tkZgvN7IdhxSIiEkdhjgSuB6509zfNrBcw28yecfe3Q4xJRCQ2QjsDcPfV7v5m4vFmYBGgiVNERLLE3D3sGDCzIcALwBHuvqnFcxOBiQBlZWWjpkyZkrbj1tTUUFJSkrb9hUXliJZcKQfkTlniXo4xY8bMdvfRuz3h7qHegBJgNjBhT68dNWqUp9PMmTPTur+wqBzRkivlcM+dssS9HEClt1KnhtoLyMwKgIeAe919apixiIjETZi9gAz4E7DI3W8JKw4RkbgK8wzgROCbwKlmNjdxOyvEeEREYiW0bqDu/hJgYR1fRCTuNBJYRCSmlABERGJKawKLSKdNm1PF5OmLWVVdy8DSYiaNG874kRrXGXVKACLSKdPmVHH11PnU1jUAUFVdy9VT5wMoCUScmoBEpFMmT1/8aeXfpLaugcnTF4cUkSRLCUBEOmVVdW1K2yU6lABEpFMGlhantF2iQwlARDpl0rjhFBfk77KtuCCfSeOGhxSRJEsXgUWkU5ou9KoXUNejBCAinTZ+5CBV+F2QmoBERGJKCUBEJKbUBCTSBWikrWSCEoBICFKp0DXSVjJFTUAiWdZUoVdV1+LsrNCnzalq9fUaaSuZogQgkmWpVugaaSuZogQgkmWpVugaaSuZogQgkmWpVugaaSuZogQgkmWpVujjRw7ixglHMqi0GAMGlRZz44QjdQFYOk29gESyrCNTJ2ikrWSCEoBICFShSxSoCUhEJKaUAEREYkoJQEQkppQARERiSglARCSmlABERGJKCUBEJKaUAEREYkoJQEQkppQARERiSglARCSmlABERGJKCUBEJKaUAEREYirUBGBmd5nZWjNbEGYcIiJxtMcEYGZTzGycmVkGjn83cGYG9isiInuQzIIwdwPfAm41s/uBu919WToO7u4vmNmQdOxLMm/anKqUVrESkWgzd0/uhWZ7AxcCPwXeA+4A7nP3+k4FECSAx9z9iDaenwhMBCgrKxs1ZcqUzhxuFzU1NZSUlKRtf2HJRjleWVXH3Qt2sKNx57bCPLjkiEJOGFiQlmPo+4ieXClL3MsxZsyY2e4+uuX2pBJAovL/OnAR8DHwd+AkYJi7n5ZyNLvuewjtJIDmRo8e7ZWVlZ053C4qKiooLy9P2/7Cko1ynPir56iqrt1t+6DSYl6+6tS0HEPfR/TkSlniXg4zazUB7LEJyMweAI4kqPTPc/eViafuNbM5KUciXdKqVir/9raLSPQlcw3gTuAZb+VUwd1Hpj8kiaKBpcWtngEMLC0OIRoRSYc99gJy96dbq/zTwczuA14FhpvZSjO7NBPHkc6bNG44xQX5u2wrLshn0rjhIUUkIp2VzBlAxrj718I8viSvqbePegFJ2jTUwbZNsGMzbK+BHVtgR01wq9sG9dugYUdwX78NGuqhsfmtAWj+2zTRUz0vH/K6Bbf8AsjLZ/CHVfD6UujWHQqKg/vCHlDYCwp7QlEJFJZA0V6QH2q1mFXxKal02viRg1ThS+vcYftmqFkDmz8K7rd8DFs/3nm/dQPUVsO2ati2MajoU2XNKve8fGganvRpHvAgMTTWQ2MdeNBt7WCA5Ukeo2gv6F4Kxb2D+x59oGd/6NEPeiZuJWXBrde+QSLpopJKAGbW0923NN1nOigRiZi6bVD9AVSvgI0fwsYq2LgSNiXua9ZA3dbd32d5UNwnqDR79IU+BwaVavfeO29FLX6FF/aEgh7QrSj4pd6tCPKLgl/zqY5HbWyExjperJjByccfA/W1UL8d6mqDeLfXNDsDqQkSU1OSqq2G2k9g7SLY8kLwuDVFvaFXGew1EHoPhr0GB/e9B0Pp/tB7P+hWmPpnngXJngG8CBzd7F5Ecs32zbB+OaxfBhvehfXLGfn+PKj8BGo+2vW1lh/8+u09CAaODCq/T38Vl0HJgOBXc3Fp8Es9LHl5kFdEQ7ceUNK/c/tqqIfaDbBlXeJMZw1sXh2c8WxeDZtWwbIZwd/Nm6YsL0gKex8Q3PoMhb4HB7c+Q4MmqZCk2gSUiekgRCSbatbBuneC28dLEo+X7F7J7zWYxrxSGHpaovIaAqUHBL9sew0It2IPQ343KNknuJUd3vbr6nfA5lXBmVH1B/DJ+ztvS58JksenLPg8+w2D/iOa3YYHyTPDdA1AJFfV74B1i2DNwsRtQXC/Zd3O1xT2Ciqbg8fu/FXa9yDY+0Ao7MG8HBlAlVXdCoNkufeQ1p9vfqa1fjmsXxok4so/B01UTQYfA99+NrOhZnTvIpId9dvhowWwem7iNg/WvB1cCIWgLb3/CBg2DsoO2/lLc6+BqberS+cU9YKBnw1uzTU2BGcM6xYHiTsv89VzqkfIyHgAEUmBe/DLsaoSqmbDykr4aP7Oyr54b9j3M/C5y2HAUcGtz9BWuzdqgr/UZPTzyssPLpL3ORCGZ2eS5GQTgLW4F5FsqauFVXPgw9fgg9eC+9oNwXOFJcFF2M9dDoNGBY9775fUr/ppc6q4eup8ausaAKiqruXqqfMBlARakYufV7IJ4Ost7kUkU7bXwIezYMUr8P7Lwa/8pl/3fYfB8LNgv2ODNuL+wzt8MXby9MWfVmZNausamDx9cZet0DIpFz+vpBKAuy9qfi8iaVRXCx/Mgveeh/degFVzwRuCNuCmX/f7fw4GHws9+6btsJrgLzW5+HnpIrBItjU2wuo5sPw5ePd5+PB1aNgeVPiDRsNJP4IhJwYVflHm5rCP2wR/nW2/z8XPSwlAJBs2fxRU+MtmBPdNbfhlR8Kx34EDT4EDTshohd/SpHHDd2nThtyd4O+VVXXcM6Nz7fe5+Hklsx7ABHefuqdtItJMYyOsehOWTIclT8FHbwXbe+4Dw84I+t0PHdP50amdEKcJ/h5aUkdt3a6dGFNtv8/FzyuZM4BrgZaV/TWtbBOJtx1bYNkMRiz6M7x+aTABmuXBfsfB2Ovg4NOh7IhgeoKIiMsEf+u3td6DPdX2+1z7vNpMAGY2DjgTGGRmtzR7ai+gsfV3icRMzTpY8iS88zi8WwH12+jbrQRGnAmHnBn80u/RJ+woY69vd2s1CXTl9vt0aO8MYC2wANgGLGy2fTNwVSaDEsm2lC4QbloFi/4Jbz8SdNXEoff+MOoSGPFFXnlvB6ec2qmlsiXNzjukgHsWNeRU+306tJkA3H0OMMfM7iX4xb+/uy/LWmQiWZLUAJ+NK4MK/+1HgoFYAP0PhVN+AiPOhgFHfjr4yldUZLsIsgcnDCzgsEMPy6n2+3RI5hrAWOAWoBA40Mw+C1zv7udmNDKRLGlrgM+dT73G+Lr1sOAh+ODV4IkBR8Kp18Kh50D/Q0KIVjoq19rv0yGZBPBL4DhgJoC7zzWzgzMalUgWNb8QWMJWzsx/g3PyXuaEbQvhCQ8mTRtzLRwxIZgpMwI0h4+kQzIJoM7dq23XuUU0KZzkjP16F3DQ5teZkP8ip+fNprvVsaJxH+7pdh6XTLwymD0zQnJxThoJRzIJYJGZfRnIM7MDgR8CszIblkgWrH4L5v6dZ/x+igo3sMFLeKChnGkNJ7Ko2whuPPsoKItehZqLc9JIOJJJAN8HriO4EPwwMB34WSaDEsmYLR/D/Adh7r3BFMr5hRQN/wKzep3BT+f254ON9QwsLebGCDep5OKcNBKOPSaAxCLwP03cRLqexgZYPhPe/AssfjKYWXPgSDjrZjjiPOjRh+OB578QdqDJycU5aSQcyUwF8TC7t/lvBCqBO9x9RyYCE+m0jVUw528w5x7Y+CEU94FjJ8LIC9tf0zXicnFOGglHMk1AHwIDgPsSf38F2AAcBdwBXJyZ0EQ6oLEhmGztjT/B0ungjTC0HE7/JYz4InQrCjvCTsvFOWkkHMkkgM+4+ylNf5jZNOB5d/+8mb2dudBEUlCzLvilP/vPwbqqPfcJplU++qK2F+fuwtSnXdIhmQRQZmaD3X1l4u+BQNMUhtszE5ZIklZWwmu3w8KHg7b9ISfDab8IRud2Kww7OpFISyYB/AR41czeIVgT+BDg+2bWE7g3k8GJtKp+e1Dhv3Z7MOVyYS8Y/S045tJgiURJigaTSbsJwMzygDUElf5hBAlgobs3dUG4ObPhiTRTsxbeuBMq74It66DfIUFPns98FYp6hR1dl6LBZAJ7SADu3mhmv3X344HZWYpJZFdrFsKrf4D5D0DDDhg2Do6/LFhQZdcR6pIkDSYTSK4J6BkzO8fdH8l4NCJN3GHZs/DqrcE8+wU9ggu6x30P+mkqqs7SYDKB5EcC9zaz7UAtQTOQu7tWuZD0q98RzL75yu9h7ULotS+MvT6Ya18Lq6SNBpMJJJcA+mU8CpHtmxn84TT43eWwqSqYa3/8bcFIXfXmSTsNJhNIbiqIBjPrDRwEdG/21CsZi0riY8t6eO02eP12Dt62MejGefZ/wbDT1b6fQRpMJpDcVBCXAlcAg4D5wDEEs4GWd/bgZnYm8FsgH7jT3X/V2X1KF7FpFbxyazBwq24rjDib2T1OYdSXJoYdWWxoMJkk0wT0b8Bo4FV3P9nMDgeu7eyBzSwf+G/gdGAl8IaZPeruGl2cyz5ZAS/dAnPuDaZpOOrLcOK/wT4j2FxREXZ0IrGSTALY5u61ZoaZFbr7QjMbkYZjHwssc/d3AcxsCnAOoASQi9YvDyr+eVPA8oIePSf+EPY+IOzIRGLL3Ftf3MvMurl7vZk9ClwEXAmcRDARXE93P7NTBzY7HzjT3b+d+PubwHHu/v0Wr5sITAQoKysbNWXKlM4cdhc1NTWUlJSkbX9hiXI5irdWccCKByhb8wKNed1Yve8ZfLD/BHYU9d3ttVEuRypypRyQO2WJeznGjBkz291H7/aEu7d6A95sZdtYYAJQ1Nb7kr0BFxC0+zf9/U3g9+29Z9SoUZ5OM2fOTOv+whLJcqxf7j71Mvd/L3W/ocz9qZ+5b1rd7lsiWY4OyJVyuOdOWeJeDqDSW6lT22sC2q0LhrvPSDn1tG0lsF+zvwcDq9K4fwnDJyvghckw9++QXwDHXx409ZTsE3ZkItJCewmgv5ld0daT7n5LJ4/9BjAssc5wFfBV4Oud3KeEZdPqoOJ/8y9g+XDsd4LpmHsNCDsyEWlDewkgHyihlTOBdPDg+sL3CdYYzgfucveFmTiWZNDWDfDSb+D1P0JjPRx9MZx8JfRW90KRqGsvAax2919m8uDu/gTwRCaPIRmyfTPM+p9gyobtm4MZOU/5KfQ5MOzIRCRJKV0DEKF+R9DM8/xNwZTMI86GU6+FfQ4NO7IuS/PyS1jaSwBjsxaFRJ97sAjLjF/CJ+/BASfB16bA4N17lknyNC+/hKnNBODuG7IZiETYey/CMz+HVXNgn8Pg6w9qrp400bz8EqZkRgJLXK1bAs9cB0uehL0Gw/j/gaO+Ann5YUcWWdPmVHFDxVY2PPV4Us05mpdfwqQEILvb8jFU3AiVfw4WYhl7PRz/PSjQXPHt2dmcE4yuT6Y5R/PyS5jywg5AIqR+O7z0X/C7kUHlP+oS+L9z4OQrVPknob3mnLZMGjec4oJdz6g0L79ki84AJLjA+87j8PS1wQXeYePgjBugvyqhVHSkOUfz8kuYlADibs1CeOoqeO8F6D8CvjEVDlYHsI7oaHOO5uWXsKgJKK62boDHroDbToKP5sMXJsNlL6vy7wQ150hXozOAuGlsgNl3w3M3wLZNcMy3ofxqLbieBk2/4m94ZB4btrmacyTylADi5INZ8MQk+OitYCDXWb+GssPDjiqnjB85iNKNSykvLw87FJE9UgKIg5q18PTP4a0psNcgOP8uOHyCBnKJxJwSQC5rbIDKu2DGDcHC6yddEczUWdT1V0YSkc5TAshVKyvh8Stg9TwYWg5n3Qz9hoUdlYhEiBJArqn9BJ79RXCht6RMzT0i0iYlgFzhDgseCvr0b10fTN1QfjV03yvsyEQkopQAckD32tXwtwmw/DkYeDR84yHY9zNhhyUiEacE0JU11MHLv+WYN26CbkXBYK5jLtVsnSKSFCWArmrlbHj0B7B2IRv6fY7+F/0Z9to37KhEpAtRAuhqttfAzP8Pr90GJQPgq39n4Uc9KVflLxJJUV7yUwmgK1n2LPzzR7Dxg2AKh7HXBxd5P6oIOzIRaUXUl/zUZHBdQe0nMO1y+Nt5UNAd/vUp+OJ/qoePSMR1ZI2IbNIZQNS98zg89qNgla6Tfwyn/CS44CsikRf1JT+VACKg1TbCQ4rgyZ8EffvLjoQLH1TXTpEuJupLfqoJKGRNbYRV1bU4QRvhc1PvZPtvj4G3H4Ux18DEmar8RbqgqK8RoTOAkDVvIyxlM78o+Avn5L/CO3UHMeKyx6HssJAjFJGOivqSn0oAIWtqCzwtbzY3FtxJb2q4ue4Cbm/4F5aq8hfp8qK85KcSQMgO6d3IxK23cV7+S7zdeAAX1V3FIj+AQSm2EUa5r7GIRJMSQJjereBh+zGFeWv5bf253Fp/LnV0S7mN8JVVddwzI7p9jUUkmnQROAw7tsKTP4W/nkOPnr146ZT7eKDkIurpxqDSYm6ccGRKFfdDS+oi3ddYRKJJZwDZVjUbpn4X1i+F4y6DsddTXtiDl0/t+C7Xb/NWt0elr7GIRJMSQLY01MNLv4GKG6HXALjokWClrjTo291aTQJR6WssItGkJqBs+OR9uPuLMPP/weHnwvdeSVvlD3DeIQWR7mssItGkM4BMcod5U+CJScGSjBPuhKMuSPthThhYwGGHHqZeQCKSEiWATKmtDubwWTgV9j8BJtwOpftn7HBR7mssItEUShOQmV1gZgvNrNHMRocRQ0Z9MAtuOwkWPQpjr4NLHsto5S8i0hFhnQEsACYAt4d0/MxoqIcXb4bnbwoq/G89DYNHhR2VSCRosGL0hJIA3H0RgJmFcfjMqP4Qpn4HPngVjvoKnHWz5usXSYj6wihxZe6t9yHPysHNKoAfu3tlO6+ZCEwEKCsrGzVlypS0Hb+mpoaSkpJO76ffulcZvvj3mDeydNhlrBlQ3vngUpCucoRN5YiedJXlyoqtrXZV7tvd+M/yHp3e/57kynfS0XKMGTNmtrvv1tyesTMAM3sWGNDKU9e4+yPJ7sfd/wj8EWD06NFeXl6engCBiooKOrW/um3w9LWw8A4YOBLOv4tD+wzl0LRFmJxOlyMiVI7oSVdZNjz1eOvbt3lWPqtc+U7SXY6MJQB3Py1T+46Ej5fBPy6Bj+bD8f8HTvt36FYYclAi0RT1hVHiSgPBOmLe/XD752FjFXztfjjzP1T5i7Qj6gujxFUoF4HN7Fzg90B/4HEzm+vu48KIJSV1tcGgrjn3BH37z7sTeusClsieRH1hlLgKqxfQw8DDYRy7wz5eCg9cDGsXwslXQvnPIF/j6ESSpcGK0aMaLBnz/wH//CHkF8KFD8Gw3L68ISLxoATQnrptMP1nUPkn2O84OP8u6D047KhERNJCCaAtn6yABy6C1XPhhB/A2OshvyDsqERE0ibnE0CHhp8veToY1euN8JV74dCzsxOsiEgW5XQCSHn4eWMDVPwKXvg1lB0BX/4r9D0omyGLiGRNTo8DmDx9cfJr5W5ZD387L6j8P/sN+PazqvxFJKfl9BlAW2vi7rZ91Ry4/5tQsxb+5Xcw6uIsRCciEq6cPgNoa5j5Ltvn3At/Ghes3vWtp1T5i0hs5HQCaHf4ef0Ohi25DR65HPY/Hr77PAw6OqRIRUSyL6ebgNocfn5IEdx9FoNWvQEn/hBOvU6jekUkdnK+1mt1+HndNigoZuFhP+Hw068JJzARkZDldBNQmwq6w0WPsm6fE8OOREQkNPFMAAC5tByliEgHxDcBiIjEnBKAiEhMKQGIiMSUEoCISEwpAYiIxJQSgIhITCkBiIjElBKAiEhMKQGIiMSUEoCISEwpAYiIxJQSgIhITCkBiIjEVM6vB9CVTZtTtftiNi3XNhAR6SAlgIiaNqeKq6fOp7auAYCq6lqunjofQElARNJCTUARNXn64k8r/ya1dQ1Mnr44pIhEJNcoAUTUquralLaLiKRKCSCiBpYWp7RdRCRVSgARNWnccIoL8nfZVlyQz6Rxw0OKSERyjS4CR1TThV71AhKRTFECiLDxIwepwheRjAmlCcjMJpvZO2b2lpk9bGalYcQhIhJnYV0DeAY4wt2PApYAV4cUh4hIbIWSANz9aXevT/w5CxgcRhwiInFm7h5uAGb/BO5397+18fxEYCJAWVnZqClTpqTt2DU1NZSUlKRtf2FROaIlV8oBuVOWuJdjzJgxs9199G5PuHtGbsCzwIJWbuc0e801wMMkEtGebqNGjfJ0mjlzZlr3FxaVI1pypRzuuVOWuJcDqPRW6tTQzgDM7GLgMmCsu29N8j3rgBVpDKMf8HEa9xcWlSNacqUckDtliXs5DnD3/i03hpIAzOxM4BbgFHdfl/UAdsZR6a2dFnUxKke05Eo5IHfKonK0LqxeQLcCvYBnzGyumd0WUhwiIrEVykAwdz84jOOKiMhOcZ8L6I9hB5AmKke05Eo5IHfKonK0IvRuoCIiEo64nwGIiMSWEoCISEzFKgGY2QVmttDMGs2sza5UZva+mc1P9FCqzGaMyUihHGea2WIzW2ZmV2UzxmSYWR8ze8bMlibu927jdQ2J72KumT2a7TjbsqfP18yKzOz+xPOvmdmQ7Ee5Z0mU4xIzW9fsO/h2GHHuiZndZWZrzWxBG8+bmf0uUc63zOzobMeYjCTKUW5mG5t9H9d1+GCtjQ7L1RtwKDAcqABGt/O694F+YcfbmXIA+cByYChQCMwDDgs79hYx/hq4KvH4KuCmNl5XE3asHfl8gcuB2xKPv0ow5UnosXegHJcAt4YdaxJl+TxwNLCgjefPAp4EDDgeeC3smDtYjnLgsXQcK1ZnAO6+yN27/KrqSZbjWGCZu7/r7juAKcA5mY8uJecAf0k8/gswPsRYUpXM59u8fP8AxpqZZTHGZHSFfydJcfcXgA3tvOQc4K8emAWUmtm+2YkueUmUI21ilQBS4MDTZjY7MRldVzQI+LDZ3ysT26KkzN1XAyTu92njdd3NrNLMZplZVJJEMp/vp6/xYPbbjUDfrESXvGT/nZyXaDb5h5ntl53Q0q4r/J9I1ufMbJ6ZPWlmh3d0Jzm3IpiZPQsMaOWpa9z9kSR3c6K7rzKzfQhGK7+TyMpZk4ZytPZLM+t9ftsrRwq72T/xfQwFnjOz+e6+PD0Rdlgyn28kvoM9SCbGfwL3uft2M7uM4Kzm1IxHln5d4ftIxpsEc/vUmNlZwDRgWEd2lHMJwN1PS8M+ViXu15rZwwSnyVlNAGkox0qg+S+1wcCqTu4zZe2Vw8zWmNm+7r46cSq+to19NH0f75pZBTCSoN06TMl8vk2vWWlm3YDeZOnUPgV7LIe7r2/25x3ATVmIKxMi8X+is9x9U7PHT5jZH8ysn7unPEmcmoBaMLOeZtar6TFwBsE01l3NG8AwMzvQzAoJLkJGpgdNwqPAxYnHFwO7ndmY2d5mVpR43A84EXg7axG2LZnPt3n5zgee88RVvAjZYzlatJN/CViUxfjS6VHgokRvoOOBjU1NkF2JmQ1oupZkZscS1OPr239XG8K+4p3lq+vnEvwK2A6sAaYntg8Enkg8HkrQE2IesJCgySX02FMtR+LvswiW3Fwe0XL0BWYASxP3fRLbRwN3Jh6fAMxPfB/zgUvDjru9zxf4JfClxOPuwIPAMuB1YGjYMXewHDcm/i/MA2YCI8KOuY1y3AesBuoS/z8uJZhy/rLE8wb8d6Kc82mnJ2DEy/H9Zt/HLOCEjh5LU0GIiMSUmoBERGJKCUBEJKaUAEREYkoJQEQkppQARERiSglAclKLGUTndmQmTjMrNbPL0x+dSDSoG6jkJDOrcfeSTu5jCMGsi0ek+L58d2/ozLFFskFnABIbZpZvZpPN7I3ExGbfTWwvMbMZZvamBetANM2G+SvgoMQZxOTEPOyPNdvfrWZ2SeLx+2Z2nZm9BFxgZgeZ2VOJCQVfNLMRnYz9AAvWTehnZnmJfZ7RmX2K5NxcQCIJxWY2N/H4PXc/l2TkWyQAAAG+SURBVGBE5UZ3PyYxvcTLZvY0wQyR57r7psR0E7MsWHjmKuAId/8sBAtx7OGY29z9pMRrZxCM3FxqZscBf6DFBGpmNgb4TSv72eruJzTf4O4rzOwm4DbgNeBtd386yc9CpFVKAJKrapsq7mbOAI4ys/MTf/cmmEVxJfAfZvZ5oJFgiuCyDhzzfgjOKAimsHiw2fT/RS1f7O4zgZYxtsnd7zSzCwimBUj6fSJtUQKQODHgB+4+fZeNQTNOf2CUu9eZ2fsE8/i0VM+uzaYtX7MlcZ8HVLeSgHYNJoUzgMTrexDMYAlQAmxub/8ie6IEIHEyHfiemT2XqOgPAaoIzgTWJraNAQ5IvH4z0KvZ+1cAhyWaj7oDY4GXWh4k0ZT0npld4O4PJmZuPMrd57V4XUpnAATTMN+biOMO4OwU3iuyG10Elji5k2Aq6TctWHD7doIfQfcCo82sErgQeAc+nQf/ZTNbYGaT3f1D4AHgrcR75rRzrAuBS82saVbZTi2zaGanAMcQrJt8L7DDzP61M/sUUTdQEZGY0hmAiEhMKQGIiMSUEoCISEwpAYiIxJQSgIhITCkBiIjElBKAiEhM/S+r4xltVb1aZQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "poly = PolynomialFeatures(degree=2,include_bias=False)\n",
    "Xquadratic = poly.fit_transform(X)\n",
    "print(\"X dimensions\")\n",
    "print(Xquadratic.shape)\n",
    "# Fit regression\n",
    "lr.fit(Xquadratic,y)\n",
    "# Set X's for forecasting\n",
    "xfeature = np.arange(start=-1.5,stop=1.5,step=0.05)\n",
    "# Turn them into a matrix\n",
    "xf = xfeature.reshape(len(xfeature),1)\n",
    "# Add powers of X\n",
    "xfeaturePoly = poly.fit_transform(xf)\n",
    "yhat = lr.predict(xfeaturePoly)\n",
    "# plot data\n",
    "plt.plot(X,y,'o')\n",
    "# plot features\n",
    "plt.plot(xfeature,yhat)\n",
    "plt.xlabel(\"Feature = x\")\n",
    "plt.ylabel(\"Target = y\")\n",
    "plt.grid()\n",
    "# Report in sample performance of polynomial\n",
    "print('R squared for quadratic model=',lr.score(Xquadratic,y))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train R-square for Quadratic Model= 0.40603267365784756\n",
      "Test R-square for Quadratic Model -0.13976744342212855\n"
     ]
    }
   ],
   "source": [
    "#Training and Testing\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.5)\n",
    "Xquad_train = poly.fit_transform(X_train)\n",
    "Xquad_test  = poly.fit_transform(X_test)\n",
    "lr.fit(Xquad_train,y_train)\n",
    "print('Train R-square for Quadratic Model=',lr.score(Xquad_train,y_train))\n",
    "print('Test R-square for Quadratic Model',lr.score(Xquad_test,y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model 3: Polynomial Model (degree 7)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X dimensions\n",
      "(20, 7)\n",
      "R squared for polynomial model of degree 7= 0.5438824894853835\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEGCAYAAABsLkJ6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXxU1f3/8dcHCBAImyBRggouoCgqqwouiVShahVxaa3V2mpdql0t/Uptv/Zn+y22uNTaBXfriq0iWpfiAohoRQJBFhG1ikLYBAwQCBCS8/vjTDBAlplkZu6due/n4zGPSSYz937OJLmfueee8znmnENERKKnRdABiIhIMJQAREQiSglARCSilABERCJKCUBEJKJaBR1AIrp16+Z69eqVtO1t2bKF9u3bJ217QVE7wiVb2gHZ05aot2Pu3LnrnHP77vl4RiWAXr16UVxcnLTtzZgxg8LCwqRtLyhqR7hkSzsge9oS9XaY2ad1Pa4uIBGRiFICEBGJKCUAEZGIUgIQEYkoJQARkYjKqFFAIiJRM6WklAlTl7KyrIJ92hq/6lTK6AEFSdm2EoCISEhNKSll3OSFVFRWAbB+m2Pc5IUASUkC6gISEQmpCVOX7jr416iorGLC1KVJ2b4SgIhISK0sq0jo8UQpAYiIhFSPzrkJPZ4oJQARkZAaO7IvuTktd3ssN6clY0f2Tcr2dRFYRCSkai707jYK6Jz+GgUkIhIFowcU7Drgz5gxg8IkHfxBXUAiIpGlBCAiElFKACIiEaUEICISUUoAIiIRpQQgIhJRSgAiIhGlBCAiElGaCCaSJDV120vLKih4expjR/ZN2oxNkVRQAhBJgj3rtpeWVSS1brtIKqgLSCQJUl23XSQVlABEkiDVddtFUkEJQCQJUl23XSQVlABEkiDVddtFUkEXgUWSoHbd9tKyCgo652oUkISeEoBIktTUbZ8xYwaFhYVBhyPSKHUBiYhElBKAiEhEKQGIiESUEoCISEQpAYiIRJQSgIhIRCkBiIhElBKAiEhEBZYAzOwAM5tuZkvMbLGZ/SioWEREoijImcA7geudc/PMrAMw18xecc69F2BMIiKREdgZgHNulXNuXuzrzcASQIVTRETSxJxzQceAmfUCZgJHOec27fGzK4ErAfLz8wdNmjQpafstLy8nLy8vadsLitoRLtnSDsietkS9HUVFRXOdc4P3+oFzLtAbkAfMBcY09txBgwa5ZJo+fXpStxcUtSNcsqUdzmVPW6LeDqDY1XFMDXQUkJnlAE8DjznnJgcZi4hI1AQ5CsiA+4Elzrnbg4pDRCSqgjwDGA5cApxqZvNjtzMCjEdEJFICGwbqnJsFWFD7FxGJOs0EFhGJKCUAEZGI0prAItJsU0pKmTB1KSvLKujROZexI/syeoDmdYadEoCINMuUklLGTV5IRWUVAKVlFYybvBBASSDk1AUkIs0yYerSXQf/GhWVVUyYujSgiCReSgAi0iwryyoSelzisPApKH4g5btRAhCRZunROTehxyUOJY9AyaMp340SgIg0y9iRfcnNabnbY7k5LRk7sm9AEWWBDR/DPgenfDdKACLSLKMHFDB+TH8KOudiQEHnXMaP6a8LwE21cztsXJGWBKBRQCLSbKMHFOiAnyxln4Gr1hmAiEjkbPjY3ysBiIhETBoTgLqARDKAZtpGyIaPoU1HaNc15btSAhAJQCIHdM20jZgNH8M+vcFSXyxZXUAiaVZzQC8tq8Dx5QF9Sklpnc/XTNuISdMQUFACEEm7RA/ommkbIVWVfhSQEoBIdkr0gK6ZthGycTlU71QCEMlWiR7QNdM2QtI4AgiUAETSLtEDumbaRsiGT/x9mhKARgGJpFnNgTuRYZ2aaRsRGz6BnHaQl5+W3SkBiARAB3SpU80IoDQMAQV1AYmIhEfNHIA0UQIQEQmD6ir44pO09f+DuoAk0zkHm0ph00po1RZat/d9qK3bQes8aNGy8W2IhMGmlVC1QwlApF6bVsFHr8CaxbB6EaxZBNvK6n5uq7bQcwgceAIcNAwOGOoThEgYpXkIKCgBSCZwDpbPhtl3w5Ln/ESZnPaQ3w+OPBfyj4TOB8HObVC5FXZs8fcbS+Gz/8Abt8LMamjRCgoGw8BL4Mgx/ixBJCyUAERq2bkdFv7TH/hXL4A2neC4q2HAJdCtD7SI8xLWtk2w4h349C1Y8jw8ey1M/QUc800Y/F3Yt09q2yESjw0fQ8s20KFH2napBCDhVDoPnrka1i2FfY+As+6Ao7/etC6cth3h0K/426m/gk/fhDn3w5z7YPbfoPcp/vEDhiS/HSLxqhkBFO8HmyRQApBw2bkDZk6AN27zk2EuehL6jEzeuGgz6HWiv5WvhZJH4O2JcP9X4IivwYiboNthydmXSCI2pHcEEGgYqIRI+/JlcN8ImPkHOPpC+P5/oO+o1E2KyesOJ10PPyyBwnHw0TT4y3Hw/E9g85rU7FOkLs6ltQx0DSUACYfiBxg093rYvAq+/hicOxFyO6dn323yoPAG+NF8GPwdmPcw3DXQX3uormr89SLNtXk17KxI6yQwUAKQMHjjNnj+J3zR5Rj4/ttwxFnBxJHXHc68Da59Bw44Dl76OTwwCtYuCSYeiY4ARgCBEoAEyTl45SZ47WbofyGLjvoFtO8WdFTQ9RD41tNw7j2w/iOYeBJMH+9HJYmkQk0C6KIzAImC6irf1/7mH/1QzHPvxrUI0ZgEMzjm63DdHD/X4PVb4O6TYeX8oCOTbLThYz9PpdMBad2tEoCkX1UlPHMVzH0QTvwJnHl7Woe+JaR9NzjvXrj4KT+f4L4RvstK1wYkmTZ87Ccztkzvh6BA/+vM7AEzW2tmi4KMQ9LIOT++f+E//ZDLr/w6baVvm+Ww0+CaN/1Q0dduhofOhC+WBR2VZIsARgBBHAnAzCaZ2UizlPyXPgSMSsF2Jaxe/z0sesof/E/6adDRJKbdPnD+g/7awJrF8LcTYf7jPqmJNJVzgcwBgPjOAB4Cvgt8YGa/NbNDk7Vz59xMYEOytiepNaWklOG3TKP3DS8w/JZpTCkpTWwDi56GGeN9CYYTf5KaIFOt5trANW/C/kfDlGvg6cth28agI5NMtWUd7NgcSAIwF+enFzPrAlwM/A/wCXAv8IRzbmezAjDrBTzvnDuqnp9fCVwJkJ+fP2jSpEnN2d1uysvLycvLS9r2gpKOdry1spKHFu1gR/WXj7VuAZcd1ZphPXIafX2HTR9w7Pwb2dzhUN495mZci71fk3G/D1fFgZ9Npvcnj7OtbTeWHHE9mzodnnntaEC2tCXM7ei4cQkDS25gQf9fsaHr4Aaf29R2FBUVzXXO7bXxuBJA7OD/TeBSYB3wOHAicJhz7isJR7P7tnvRQAKobfDgwa64uLg5u9vNjBkzKCwsTNr2gpKOdgy/ZRqlZRV7PV7QOZc3bzi14RdvXAH3nurLM39vWr1DPTP297H8HX8WsLEUisYxo2oghUUjgo4qKTL2d7KHULdj/hMw5Wq4bi50a7iDpantMLM6E0Cjl5zN7B9Af/xB/zzn3IrYjx4zs5KEI5GMtLKOg39Dj++yYws88Q3YsRUufTYc4/yT7YChcPUsP6x12m85pvNRMPBw6KQ1fyUOGz4GawGdD0z7ruO5BnAf0M8595taB38AnHMDUhOWhE2PzrkJPb7Lv37kL5he8CB0PyIFkYVE205w3v1wzl/puOkj+NsweO/ZoKOSTLDhYz/+v1XrtO+60QTgnHvZxXuhIEFm9gTwH6Cvma0ws8tTsR9pvrEj+5Kbs/vyirk5LRk7sm/9L1r4lB/uWTjOD6PMdmYw4GKKB9/hL+j941J49jrYXh50ZBJmqxcE9uEo0KmXzrmLgty/xG/0AN+dMWHqUlaWVdCjcy5jR/bd9fheNq6A538KPYfCiRk23LOZKtr1gMtf9iOe3rjdL0Rz3r1QMCjo0CRsKr6AdR/46rcBCNHcewm70QMK6j/g11Zd7Sd7Ve+EMXenfXZjKLTMgRH/C4ecCpOvgvtPh5N+5stPB3CqLyFVOtff9xwayO7jmglsZu1r34s0aPbfYNkbMGp8IGObQ6XXiXDNLL8G8eu3wH2nwuqFQUclYbGiGDAoGBjI7uMtBfHGHvcidVvzHrz6/6DvGTDw0qCjCYfcLr4L6BuP+4Vm7imC1//gayJJtC1/B7r3gzYdAtl9orWAMqBoiwRm53aY/D2/Bu/X/pQZNX7S6fAz4drZ0O8cmP5/vrBc6bygo5KgVFdDaTH0bHjyVyqFtASjZKQZ42HNIjj7z5C3b9DRhFO7feD8++HCR/wqUPee6ucPbFVFlMhZ/5EvIXJAMP3/oAQgybJ2Cbx1Fxx7sV/HVxrW72y/1sDx18Dcv8Ndg/x9dXXjr5XssGKOv+85JLAQEh2eobKHsjfn4IWfQes8OO3moKPJHG07+QvlA77l379//RDm/d2XyO59clpCmFJSGv/QXknu+7XiHf830PWw5AaZgHjPAGyPe5EvLfgHfDrLH7iysdRDquUfCd950ZeZ3rQS/v41eOgs+PQ/Kd3tlJJSxk1eSGlZBQ4oLatg3OSFiVd5jYikv18riqFgcKCLIcW752/ucS/iVZTBy7/0k5wGfjvoaDJXTZnpH5bAqFvg86Xw4Ch4eLQfKZICE6YupaJy95XNKiqrmDB1aUr2l+mS+n5t3wxr3wu0+wfiTADOuSW170V2mf5/sHUdnHlbeJd1zCQ5uf66wI/ehdN/68sE3H+av1g87xFfXC9JmlzgL6KS+n6tLAFXnRkJQKROK+fDnPtgyBXQQ3UBk6p1Oxj2A/jRAn9GsL0cnrsObjvcXy9Y3fxVVJtc4K8pqquhclugq6c1d0GjpL5fNWd1PYMtDxLBOfqSFNXV8ML10K4bFN0YdDTZq02ePyM47mr47D9Q/CDMexjm3OtnWR92Ohx6GvQa7s8eEjB2ZF/GTV64W7dGowX+ajjnV7L6Ypm/bSr1w1o3r4zdr/ZnKzu3w85tUB2b9NaiFeTu468Vtevqh8V27+fHwhcMhtzOCbUhXm+trOSR175sa03/PRD3RdxmvV97WlEM3fr4SYIBimc9gDHOucmNPSYRU/Kwn8Ry7j0p+6eVWszgoGH+9tXf++U1P3zFDx2dPRFa5fqyEz0GQH4/f1Dd55AG6zA1WuCvchvttqzw+6k50H+xDL74FL74BHbsUeW0dQfosB903B8OOM4nr1ZtoVUbf98yx5/JbF3/5W31InjvOXYNMNz3cJ8M+p7pk1uS6kg9/UElFZW7n33U9N/HmwASLohYH+f8ENA+IxN7XQrE8+7+EtjzYH9jHY9JVGzfDNN+CwcOC6yKYaS12weGfs/fKitg2Zvw4cvw8Qz472u+bxmgZWv/KbP9vj5Jt+0cu+/kn1NVyeiqHYweuMN/Ut+yDt79HN5cA+VrYNtGhgLEhqvTqq1ftKRLb3/G0aWX/7pLL7/4TVPLGWzbBCvnwfI5/sC45HkoeRTy8uGYi3xJka6HNOstW7+t7q6nRPvv4y6I2JAvlvnrZgHOAK5RbwIws5HAKKDAzG6v9aOOgGarRNlbd8GWz+GiJ1XuIWg5uXDYV/wNfD/7uqV+Yt6axb7U8Nb1vjx3xRewrcxXaa3RopVPFC1b+y6ZvHxfm/7gQmjfnSWrtnDEsFH+IN++e2ou9Lft6Pd3cKH/vqrSn3XMe9j/rb35R/9hY/iP/KfmJvzNdW1rdSaBlFzvaEwIJoDVaOgMYC2wCNgGLK71+GbghlQGJSG2aZX/pzxyTOAXsJIpayZE5bSF/Y/xt7o4588aWrSEFjmNHtDXzJjBEQcen4JAG9AyBw4/w982r4b5j/sJck98HXqfAiN/B/s1uoT4bs7rk8MjS6qS03/fXCvmQE57300XsHoTgHOuBCgxs8fwn/gPdM59lLbIJJxmjPef0Eb8KuhIkqZmgk9zLhBmDDM/wihTdNgPTvqpHxFV/ID/+7v7JD97uuiX0CE/rs0M65FDvyP6hSPJr5jjyz+3aNn4c1MsnmsAI4DbgdZAbzM7FrjJOXduSiOT8Fn7PpQ8AkOvyqo6/w1N8Mm6BJCpWubAcVf5a06vT4B37oZFk2HETf5aSBzdQknpv2+uygq/HsSwHwYbR0w8HXo3A8cBZQDOufnAoakMSkLq1V/7kR6n/DzoSJIqEydENXdMe8bK7QKjfgfXvgMHHg8vjYVJF2dONdWV8/01mBD0/0N8CaDSOVe2x2MqChc1y2bBBy/BST/xo1CySFonRCWBavjgRwVd/BSMHO9HQE08KeW1k5IiRBeAIb4EsMTMLgRamFlvM/sj8HaK45IwcQ5e/hV0LPATkrLM2JF9yc3ZvT82sAuEcVANnxgzOOH7cMUrvovooTN891B1VeOvDcry2X5EVUjWy4gnAVwHDMJfCH4G2A78OJVBScgsfsaP0z71lwnPNs0EowcUMH5Mfwo652JAQedcxo/pH3x/cT0yscsqpXoMgKtmwlHnwfTfwhPf8BPOwqbiCz+89dDTgo5kl0YvAjvntgD/E7tJ1FTt9JO+uh8JR3896GhSJhQXCOPUo3MupXUc7MPaZZUWbTvCmHvhwBPgxbH+bOCb//CjiMJi4VNQtd2PYAqJRs8AzOwZM5u8x+1BM7vWzFqnI0gJ0IInYcN/4dQbQzFsTTKvyyptzGDI5XDRJFj3Edx3mh+5FhYlj0J+//rnaAQgni6g5cBO4JHYbQewATgauDd1oUngqirh9d/D/sdC3zOCjkZiMq3LKu36nA7fecF/2n7gdD+AIWirF8Gq+f7Tf4hmz8czD+AY59wpNd+Y2RTgdefcyWb2XupCk8DNfxzKPoUzbg3VH61kVpdVIHoMgMtfgccugIdH073vD4HC4OKZ/5gvtxGy2lnxnAHkm1nPWt/3AGouYW9PfkgSCju3w8wJvkTvYeG5aCUSty4HweVToecQjlhyu68tFISdO+DdSdD3q6EbQh3PGcDPgf+Y2fv4NYH7ANeZWXvgsVQGJwEqeQQ2Loev3alP/1kqa+ofNSS3C3zraTZMPJOuz/3Ar1Fw/DXpjeGDl6BiAwy4JL37jUODCcDMWgBr8Af9fvgEsNg5VzME4dbUhieBqNwGM2/zIyoOOTXoaCQFIlX/qHU7Fh31C05Z+xD8+wafBE7+Wfr2X/IodOgRyv+lBruAnHPVwJ3OuQrn3FznXHGtg79kq7kP+ZWdin6hT/9ZKmqTyVyLHLjg734o87Tf+LIm6ViectMq+OhVOPaiUI6ii6cL6BUzO8c592zKo5Hg7dgKs26HXidB75ODjkZSJJKTyVq2gtETIacdzLrDTxb76h9Ss8ZBjXef8IvvHHtx6vbRDPEkgOuATma2HajAdwM551y4rmZIchQ/4FeDuuChoCORFIrsZLIWLeCsO/xylW/d5RfIGf03X0oi2Zzz3T8HDmv2imapEk/q6wbkAHn40T/d+HIUkGSTygq/+tLBhX7tWclakZ5MZgan/QZG/C8s/KevJlqZgjOfz972kyhDNPN3T/GUgqgys07AIUDbWj96K2VRSTDm/t0v9XiKqn5ku6QtcJ6pzOCk6/06yS9cD4+eBxc94ddLTpY590HrPOh3TvK2mWSNJgAzuxz4KVAALASG4KuBFjZ352Y2CrgTaAnc55y7pbnblCbauR3evBMOOlGf/iNCk8nwpSNyO8PkK+Ghs+Bbk5NTqXPeI7DoKRj+Y9/dFFLxdAH9GBgMLHPOnYSvDLqquTs2s5bAX4Cv4oeYXmRmwS+SGVUlj/qRP+kcHicSBkedF6sf9CHcWwQripu3vc/ehud/4rtSTw330qnxJIBtNUM/zay1c24xcHgS9j0U+Mg597FzbgcwCQjvuVI2q6qEWX/0i1QcXBh0NCLpd9hp8J0XfdfQA6Pg7YlNGyZathye/BZ0PgDOf9CPPAoxc/U00sxaOed2mtlzwKXA9cCJ+EJw7Z1zo5q1Y7PzgVHOuSti318CHOecu26P510JXAmQn58/aNKkSc3Z7W7Ky8vJywvv6Vm8mtuO/Va9yuFL72JB/1+xoevgJEaWGP0+widb2hJvO1pVlnP4+3+k2/o5rN13OEv7XkdVq3Zx7aNF1TYGlNxAbsUa5g38A1vbH9DcsPfS1N9HUVHRXOfc3v/czrk6b8C8Oh4bAYwB2tT3unhvwAX4fv+a7y8B7mroNYMGDXLJNH369KRuLyjNasfOSufuPNa5iSc5V12dtJiaQr+P8MmWtiTUjqoq5964w7lfd3HuzgHOLXur8f+N6mrnnrzEuZs6Obd0arNibUhTfx9AsavjmNrQ+cleU0Cdc68lnHrqtwKonSJ7AiuTuH2Jx+LJsOFj+PqjmvUrAn6uwIk/9l2iT30XHhwF3fvBwEv9TOLaBd0qvoDPZvtV8957Fk672ZejzhANJYB9zeyn9f3QOXd7M/c9BzjMzHoDpcA3gG82c5uSiOpqmHmr/+Pue2bQ0YiES6/h8INiWPS0HyL97xvglZvgiK9Bu67w6VuwZhHgoEUODL0Khv0w6KgT0lACaImf/JWSj4XOX1+4Dpga29cDzl9glnRZ8iysWwrnP5Da6fAimapNBxh0mb+tXuhLSi940g+c6DkECsf5YdMFg6B1fNcKwqShBLDKOXdzKnfunHsReDGV+5B6OOcrfnY9FPqNDjoakfDbrz+cMQFG/s5/n4ryEWmW0DUAySIfTIU1C30dlBBWKYySSNTlzyZZcOCv0VACGJG2KCS9nPOrfXU+EPpfEHQ0kRapuvwSOvV2/DrnNqQzEEmjT16H0mIY/qOs+jSTiaJWl1/CJdzT1CQ1Zt4KefvBseGtUpipppSU8psZW9nw7xfi6s6JZF1+CQ0N/Yia5e/Asjdg2HWQ07bx50vcarpz1m9zOL7szplSUlrva+qrv5/1dfklFJQAombmrZC7Dwz6TtCRZJ2mdOdEui6/BE5dQFGyagF8OBWKfhnqErWZqindOZGvyy+BUgKIkjdugzYdYej3go4kKzV1mUXV5ZegqAsoKj7/wNcqGXKFXwBDkk7dOZJpdAYQFbPugFZt4YRrg44ka9V8iv/Ns++yYZtTd46EnhJAFGz4xNcvOe4qaN8t6Giy2ugBBXTe+CGFhYVBhyLSKHUBRcGs26FFq4yrVCgiqaUEkO3KlsP8J2DgJdBx/6CjEZEQUQLIdm/+0d8P/3GwcYhI6CgBZLNNq2DeI3DsN/0i1SIitSgBZLO3/gTVO+HEnwQdiYiEkBJAtipfC8UP+jVM9+kddDQiEkJKANnqrbugajucdH3QkYhISCkBZKMt62HO/XDkGOh2aNDRiEhIaSJYNnr7r1C5FU7+WdCRiERemJf8VALINlvWw+y7od/Z0P2IoKMRibSwL/mpLqBs89adsKMcCscFHYlI5IV9yU8lgGyyeQ3Mvscv9K5P/yKBC/uSn+oCCoGk9RHOugOqdkDhDckPUkQS1tQ1ItJFZwABq+kjLC2riHsd2TptXAHF98OxF0HXQ1ISq4gkJuxrRCgBBCxpfYQzbwXn4OSfJzE6EWmO0QMKGD+mPwWdczGgoHMu48f0D8UFYFAXUOCS0UfYtmINlDwCgy6DLgclKTIRSYYwL/mpM4CA1dcXmEgf4UGfTqKKFpyz4Hh63/ACw2+ZlngXkohEjhJAwJrdR7juQ/JXz+CRnafx7sZ2zbuOICKRogQQsGb3EU7/HdvJ4a4dZ+32cJjGGotIOOkaQAg0uY9wRTEsnsx9O0eznk57/TgsY41FJJx0BpCpnIOpv4D23flHq7PrfEpYxhqLSDgpAWSqxc/A8tlw6i/5ap+OoR5rLCLhpASQiSq3was3Qf5RMOBbDOuRE+qxxiISTroGkIlmT4Syz+CSKdDCf/IP81hjEQmnQM4AzOwCM1tsZtVmNjiIGDJW+efwxm3QZxQcUhR0NCKSwYI6A1gEjAHuDmj/mWvGeNixBU77TdCRiCQkzAujRFUgCcA5twTAzILYfeZauwTmPghDroB9+wQdjUjcwr4wSlSZcy64nZvNAH7mnCtu4DlXAlcC5OfnD5o0aVLS9l9eXk5eXl7StpdSznH0gl/TcdOHzD5uIpWtO+76UUa1owFqR/gkqy3Xz9jK+m17H2u6tjVuK2zX7O03Jlt+J01tR1FR0Vzn3F7d7Sk7AzCzV4H96vjRjc65Z+PdjnPuHuAegMGDB7vCwsLkBAjMmDGDZG4vpd59Er6YD6N+z/Djdx/3n1HtaIDaET7JasuGf79Q9+PbXFreq2z5nSS7HSlLAM65r6Rq25FT/jn8+3+g51AY+r2goxFJWNgXRokqzQPIBC/93F/4PfuuXcM+RTJJ2BdGiaqghoGea2YrgBOAF8xsahBxZIT3X4TFk+HksdD98KCjEWmSsC+MElVBjQJ6BngmiH1nlG0b4YWfQvcjYfiPg45GpFk0WTF8NBM4zF65CcrXwDceg1atg45GRLKMrgGE1Sdv+DH/J1wLBYOCjkZEspASQBht3QDPXgtdekPhL4KORkSyVNZ3AWXc9PPqKnj6Cti0Er7zErRO/SQZEYmmrE4AGTn9fNpv4b+vwdfuhAOGBB2NiGSxrO4CmjB16a6Df41Qr5X73rMw63YYdJm/iYikUFYngPrWxF1ZVgGb16Q5mkasXQLPXAM9h8BX/xB0NCISAVmdAOqbZt6v03b46/EcvuQOqPgizVHVoaIMJl0MrdvDhY9AqzZBRyQiEZDVCaC+6edXnj4Ahn6P/DUz4S/HwwcBTkSurICnvgtln8KFD0PH/YOLRUQiJasTQH3Tz88Z1BuKfsHcQbdCu67w+IUw5Vr/STydtm2ER8+H/06Ds/4IB52Q3v2LSKRl9SggaHj6eXmHQ+DK6fD6H2DWHf5A/LU7oc/pqQ+s/HN4dAysfQ/Ouw/6n5/6fYqI1JLVZwBxadUGRvwKrngV2naCxy/wXTLla1O3z7LP4IGRsO5DuOhJHfxFJBBKADUKBsJVM6HoRljyL/jzECh5FJK9Ytra9+H+kbB1HVz6LBymZRNEJBhKALW1ag2n/ByufhO6H+HLMTx8Nqxe1Pxt79gC08fDvUXgquCyF+HA45q/XRGRJlICqJky6hQAAAkrSURBVMu+ffwB+qw7YOV8mDgcHh4NH72a+BlBdZU/k/jTQHj9FugzEq54DfY7KjWxi4jEKesvAjdZixYw+LvQb7Svyjn7Hnj0PNj3CF+h8/Azod0+9b9+82pY/o6/wLxmoZ/gdeHD+tQvIqGhBNCYdvvASdfDCT+ARU/BW3+G567zt3bdoFsf6HaYv21dD6sXwqoFsCV2EbnTgXD+A3DkGDALti0iIrUoAcSrVWs49ptwzEXw6VuwsgTWfeBv7z/vD/4tWvkzhMNOg/2Ohv36Q8/BmtkrIqGkBJAoM+g13N9q27rBl3LQwV5EMoQSQLI0dD1ARCSENApIRCSilABERCJKCUBEJKKUAEREIkoJQEQkopQAREQiSsNAQ2xKSSkTpi5lZVkFPTrnMnZk33rXNhARSZQSQEhNKSll3OSFVFRWAVBaVsG4yQsBlAREJCnUBRRSE6Yu3XXwr1FRWcWEqUsDikhEso0SQEitLKtI6HERkUQpAYRUj865CT0uIpIoJYCQGjuyL7k5LXd7LDenJWNH9g0oIhHJNroIHFI1F3o1CkhEUkUJIMRGDyjQAV9EUiaQLiAzm2Bm75vZAjN7xsw6BxGHiEiUBXUN4BXgKOfc0cAHwLiA4hARiaxAEoBz7mXn3M7Yt28DPYOIQ0Qkysw5F2wAZv8CnnTOPVrPz68ErgTIz88fNGnSpKTtu7y8nLy8vKRtLyhqR7hkSzsge9oS9XYUFRXNdc4N3usHzrmU3IBXgUV13M6p9ZwbgWeIJaLGboMGDXLJNH369KRuLyhqR7hkSzucy562RL0dQLGr45ga2BmAmX0buBoY4ZzbGudrPgc+TWIY3YB1SdxeUNSOcMmWdkD2tCXq7TjIObfvng8GkgDMbBRwO3CKc+7ztAfwZRzFrq7TogyjdoRLtrQDsqctakfdghoF9GegA/CKmc03s4kBxSEiElmBTARzzh0axH5FRORLUa8FdE/QASSJ2hEu2dIOyJ62qB11CHwYqIiIBCPqZwAiIpGlBCAiElGRSgBmdoGZLTazajOrdyiVmS0zs4WxEUrF6YwxHgm0Y5SZLTWzj8zshnTGGA8z28fMXjGzD2P3Xep5XlXsdzHfzJ5Ld5z1aez9NbM2ZvZk7OezzaxX+qNsXBztuMzMPq/1O7giiDgbY2YPmNlaM1tUz8/NzP4Ua+cCMxuY7hjjEUc7Cs1sY63fx/82eWd1zQ7L1htwBNAXmAEMbuB5y4BuQcfbnHYALYH/AgcDrYF3gX5Bx75HjH8Aboh9fQPw+3qeVx50rE15f4HvAxNjX38DX/Ik8Nib0I7LgD8HHWscbTkZGAgsqufnZwAvAQYcD8wOOuYmtqMQeD4Z+4rUGYBzbolzLuNXVY+zHUOBj5xzHzvndgCTgHNSH11CzgH+Hvv678DoAGNJVDzvb+32PQWMMDNLY4zxyIS/k7g452YCGxp4yjnAw857G+hsZvunJ7r4xdGOpIlUAkiAA142s7mxYnSZqABYXuv7FbHHwiTfObcKIHbfvZ7ntTWzYjN728zCkiTieX93Pcf56rcbga5piS5+8f6dnBfrNnnKzA5IT2hJlwn/E/E6wczeNbOXzOzIpm4k61YEM7NXgf3q+NGNzrln49zMcOfcSjPrjp+t/H4sK6dNEtpR1yfNtI/5bagdCWzmwNjv42BgmpktdM79NzkRNlk8728ofgeNiCfGfwFPOOe2m9nV+LOaU1MeWfJlwu8jHvPwtX3KzewMYApwWFM2lHUJwDn3lSRsY2Xsfq2ZPYM/TU5rAkhCO1YAtT+p9QRWNnObCWuoHWa2xsz2d86tip2Kr61nGzW/j4/NbAYwAN9vHaR43t+a56wws1ZAJ9J0ap+ARtvhnFtf69t7gd+nIa5UCMX/RHM55zbV+vpFM/urmXVzziVcJE5dQHsws/Zm1qHma+B0fBnrTDMHOMzMeptZa/xFyNCMoIl5Dvh27OtvA3ud2ZhZFzNrE/u6GzAceC9tEdYvnve3dvvOB6a52FW8EGm0HXv0k58NLEljfMn0HHBpbDTQ8cDGmi7ITGJm+9VcSzKzofjj+PqGX1WPoK94p/nq+rn4TwHbgTXA1NjjPYAXY18fjB8J8S6wGN/lEnjsibYj9v0Z+CU3/xvSdnQFXgM+jN3vE3t8MHBf7OthwMLY72MhcHnQcTf0/gI3A2fHvm4L/BP4CHgHODjomJvYjvGx/4V3genA4UHHXE87ngBWAZWx/4/L8SXnr4793IC/xNq5kAZGAoa8HdfV+n28DQxr6r5UCkJEJKLUBSQiElFKACIiEaUEICISUUoAIiIRpQQgIhJRSgCSlfaoIDq/KZU4zayzmX0/+dGJhIOGgUpWMrNy51xeM7fRC1918agEX9fSOVfVnH2LpIPOACQyzKylmU0wszmxwmZXxR7PM7PXzGye+XUgaqph3gIcEjuDmBCrw/58re392cwui329zMz+18xmAReY2SFm9u9YQcE3zOzwZsZ+kPl1E7qZWYvYNk9vzjZFsq4WkEhMrpnNj339iXPuXPyMyo3OuSGx8hJvmtnL+AqR5zrnNsXKTbxtfuGZG4CjnHPHgl+Io5F9bnPOnRh77mv4mZsfmtlxwF/Zo4CamRUBd9Sxna3OuWG1H3DOfWpmvwcmArOB95xzL8f5XojUSQlAslVFzYG7ltOBo83s/Nj3nfBVFFcAvzOzk4FqfIng/Cbs80nwZxT4Ehb/rFX+v82eT3bOTQf2jLFezrn7zOwCfFmAuF8nUh8lAIkSA37gnJu624O+G2dfYJBzrtLMluHr+OxpJ7t3m+75nC2x+xZAWR0JaPdgEjgDiD2/Hb6CJUAesLmh7Ys0RglAomQqcI2ZTYsd6PsApfgzgbWxx4qAg2LP3wx0qPX6T4F+se6jtsAIYNaeO4l1JX1iZhc45/4Zq9x4tHPu3T2el9AZAL4M82OxOO4FzkrgtSJ70UVgiZL78KWk55lfcPtu/Iegx4DBZlYMXAy8D7vq4L9pZovMbIJzbjnwD2BB7DUlDezrYuByM6upKtusZRbN7BRgCH7d5MeAHWb2neZsU0TDQEVEIkpnACIiEaUEICISUUoAIiIRpQQgIhJRSgAiIhGlBCAiElFKACIiEfX/ASsEzfBo7yJVAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "poly1 = PolynomialFeatures(degree=7,include_bias=False)\n",
    "Xpoly = poly1.fit_transform(X)\n",
    "print(\"X dimensions\")\n",
    "print(Xpoly.shape)\n",
    "# Fit regression\n",
    "lr.fit(Xpoly,y)\n",
    "# Set X's for forecasting\n",
    "xfeature = np.arange(start=-1.5,stop=1.5,step=0.05)\n",
    "# Turn them into a matrix\n",
    "xf = xfeature.reshape(len(xfeature),1)\n",
    "# Add powers of X\n",
    "xfeaturePoly = poly1.fit_transform(xf)\n",
    "yhat = lr.predict(xfeaturePoly)\n",
    "# plot data\n",
    "plt.plot(X,y,'o')\n",
    "# plot features\n",
    "plt.plot(xfeature,yhat)\n",
    "plt.xlabel(\"Feature = x\")\n",
    "plt.ylabel(\"Target = y\")\n",
    "plt.grid()\n",
    "# Report in sample performance of polynomial\n",
    "print('R squared for polynomial model of degree 7=',lr.score(Xpoly,y))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train R-square for Polynomial Model of degree 7= 0.9514279455516594\n",
      "Test R-square for Polynomial Model of degree 7= -763328.7563570915\n"
     ]
    }
   ],
   "source": [
    "#Training and Testing:\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.5)\n",
    "Xpoly_train = poly1.fit_transform(X_train)\n",
    "Xpoly_test  = poly1.fit_transform(X_test)\n",
    "lr.fit(Xpoly_train,y_train)\n",
    "print('Train R-square for Polynomial Model of degree 7=',lr.score(Xpoly_train,y_train))\n",
    "print('Test R-square for Polynomial Model of degree 7=',lr.score(Xpoly_test,y_test))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}