3.7_Boston_housing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Boston Housing Price\n",
    "A regression example.\\\n",
    "Predict the *median price of homes* in a given Boston suburb in the mid-1970s, given the crime rate, the local property tax rate, and so on.\n",
    "\n",
    "Only 506 samples, split into 404 training samples and 102 test samples. 13 features, and each feature has a different numerical range. For example, feature One takes values between 0 and 1; feature Two takes values between 1 and 12...\n",
    "\n",
    "\n",
    "Deep Learning with Python by Francois Chollet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.datasets import boston_housing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "(train_data, train_targets), (test_data, test_targets) = boston_housing.load_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preparing the data\n",
    "Need to normalize the heterogeneous data.\\\n",
    "The feature should be centered around 0 and has a unit standard deviation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "mean = train_data.mean(axis=0)\n",
    "train_data -= mean\n",
    "std = train_data.std(axis=0)\n",
    "train_data /= std\n",
    "\n",
    "test_data -= mean\n",
    "test_data /= std"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Build the network\n",
    "The network ends with a single unit and no activation (ie. a linear layer). This is a typica lsetup for scalar regression (a regression where you're trying to predict a single continuous value).\n",
    "\n",
    "Monitoring the metric: mean absolute error (MAE)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras import models\n",
    "from keras import layers\n",
    "\n",
    "def build_model():\n",
    "    model = models.Sequential()\n",
    "    model.add(layers.Dense(64, activation='relu', input_shape=(train_data.shape[1],)))\n",
    "    model.add(layers.Dense(64, activation='relu'))\n",
    "    model.add(layers.Dense(1))\n",
    "    model.compile(optimizer='rmsprop', loss='mse', metrics=['mae'])\n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Validating using K-fold validation\n",
    "Because there are few samples, splitting them into training, validation, and testing sets is not a good idea.\\\n",
    "Save the metrics for each epoch and for each fold."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "processing fold # 1 / 4\n",
      "processing fold # 2 / 4\n",
      "processing fold # 3 / 4\n",
      "processing fold # 4 / 4\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "k = 4\n",
    "num_val_samples = len(train_data) // k\n",
    "num_epochs = 500\n",
    "all_mae_histories = []\n",
    "\n",
    "for i in range(k):\n",
    "    print('processing fold #', i+1, '/',k)\n",
    "    [kstrt, kend] = [i*num_val_samples, (i+1)*num_val_samples]\n",
    "    \n",
    "    # Prepares validation data from partition k.\n",
    "    val_data = train_data[kstrt:kend]\n",
    "    val_targets = train_targets[kstrt:kend]\n",
    "    \n",
    "    # prepares training data from all other partitions.\n",
    "    training_data = np.concatenate([train_data[:kstrt], \n",
    "                                    train_data[kend:]], axis=0)\n",
    "    training_targets = np.concatenate([train_targets[:kstrt], \n",
    "                                       train_targets[kend:]], axis=0)\n",
    "    \n",
    "    model = build_model()   # <--------- Build the model\n",
    "    \n",
    "    history = model.fit(training_data, training_targets,\n",
    "                 epochs=num_epochs, batch_size=1, verbose=False)\n",
    "    \n",
    "    # save MAE for partition k.\n",
    "    all_mae_histories.append(history.history['mae'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Mean k-fold validation scores\n",
    "average_mae_history = [ np.mean([x[i] for x in all_mae_histories]) for i in range(num_epochs) ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAgXElEQVR4nO3deXScd33v8fd3Fu376j2y8RKc1YlDQkLJRoBCSIALhdD2ppDe3IaUQsuhDfeelrYXei4tl0LuZTNrgBAKJDQhJRtOIEkTktiOncVZbMf7KtnWLo1m+d4/5pEsa2xZSDMaaebzOkdnZp4ZzfP9yfJnfvo9v+f3mLsjIiLFI5TvAkREZHop+EVEioyCX0SkyCj4RUSKjIJfRKTIRPJdwEQ0NTV5W1tbvssQEZlV1q9f3+HuzWO3z4rgb2trY926dfkuQ0RkVjGznSfarqEeEZEio+AXESkyCn4RkSKj4BcRKTIKfhGRIqPgFxEpMgp+EZEiU9DBf9eGPdz+1AmnsYqIFK2CDv57Nu3j357Zne8yRERmlIIOfgN0nRkRkeMVdPCHzHCU/CIioxV08JtBKpXvKkREZpaCDn4w9fdFRMYo6OA3A11MXkTkeIUd/PkuQERkBirs4DfN6hERGaugg1+zekREMhV08JtBSrkvInKcwg5+TAd3RUTGyFnwm9l3zOyQmb0waluDmT1kZluC2/pc7T+9QzTQIyIyRi57/N8D3j5m2y3AWndfBqwNHueMgZJfRGSMnAW/uz8KHBmz+VrgtuD+bcC7c7V/ADOdwCUiMtZ0j/G3uvv+4P4BoPVkLzSzG81snZmta29vn9TO0ou0KfpFREbL28FdTyfySVPZ3de4+2p3X93c3DypfYQ0xi8ikmG6g/+gmc0FCG4P5XJnZkZKPX4RkeNMd/DfA1wf3L8euDuXO9N6/CIimXI5nfMO4ElghZntMbMbgP8NXGVmW4C3BI9zR0s2iIhkiOTqjd39upM8dWWu9jmWaZk2EZEMhX3mrpZlFhHJUNDBr1k9IiKZCjr4Dc3qEREZq7CDXwd3RUQyFH7w57sIEZEZpqCDH0w9fhGRMQo6+E3Lc4qIZCjo4A9pjF9EJENBB79m9YiIZCrs4NfBXRGRDIUd/GioR0RkrMIOftPF1kVExiro4AcN9YiIjFXQwR/SIL+ISIaCDn4zNKtHRGSMwg5+1OEXERmrsINfJ3CJiGQo8OA3XH1+EZHjFHbwox6/iMhYBR38aFKPiEiGgg5+TecUEclU0MFvaDqniMhYhR386vCLiGQo7OBHa/WIiIxV2MGvHr+ISIbCDn40nVNEZKzCDv70RXc13CMiMkqBB3/6VrkvInJMYQc/QY8/z3WIiMwkhR38Iz1+Rb+IyLC8BL+Z/aWZvWhmL5jZHWZWlpP9BLeKfRGRY6Y9+M1sPvAXwGp3PxMIAx/Mzb7St+rwi4gck6+hnghQbmYRoALYl4udjMzqUZ9fRGTEtAe/u+8FvgDsAvYDXe7+4NjXmdmNZrbOzNa1t7dPal/q8YuIZMrHUE89cC2wGJgHVJrZH419nbuvcffV7r66ubl5cvsantWj4BcRGZGPoZ63ANvdvd3d48BdwMW52NFIj19DPSIiI/IR/LuAi8yswtKD8FcCL+ViRyOzepT7IiIj8jHG/xTwM2AD8HxQw5pc7OtYj19ERIZF8rFTd/8M8Jlc7yektXpERDIU9Jm7w1LKfRGREQUd/KaxHhGRDIUd/MGtZvWIiBxT2MGvE7hERDIUdvAHt8p9EZFjThr8ZvaTUfc/P+a5jCUWZiJdgUtEJNN4Pf5lo+5fNea5ya2hMM1COrYrIpJhvOAfLy9nR5YGPf6UevwiIiPGO4GrwsxWkf5wKA/uW/BVPh3FTdXwGP8s+ZgSEZkW4wX/fuCLwf0Do+4PP57xNI1fRCTTSYPf3S8/2XNmFs1NOdmlZZlFRDJNeDqnpV1pZt8G9uSwpqzRsswiIplOGfxmdpGZ3QrsBO4GHgVOz3Vh2RDSCVwiIhnGm8f/T2a2Bfgc8BywCmh399vc/eh0FTgVw0M9mtUjInLMeAd3/xR4Ffga8At3j5nZ7EpQ9fhFRDKMN9QzF/gs8C5gm5n9gPS0zrys4T8ZduqXiIgUnfFm9SSB+4H7zawUuJr0/P29ZrbW3T80TTVO2rElG/JciIjIDDKh3ru7x4A7gTvNrBp4T06ryhItyywikumkwW9mfzWdheRCKBjIUo9fROSY8Xr8XwA2AvcBMY4fMp8VUapZPSIimcYL/lXAdcA7gfXAHcBan0VrHGvJBhGRTCed1ePum9z9Fnc/F/g2cC2w2cyuma7ismX2fFSJiOTeRM7cbSbd+z+L9FINh3JdVLaMXGxdfX4RkRHjHdz9CPAHQBnwM+AP3H3WhD6MmtWj3BcRGTHeGP+3gBdIr9HzNuCtx3rQ4O4zfshHY/wiIpnGC/6TLss8W4R0BS4RkQzjnbn7m+ksJBc01CMikmnC6/HPRqZF2kREMhR08A/3+bVkg4jIMQUd/Orxi4hkOuUibWa2HPgUcNro17v7FZPdqZnVkZ41dCbpSTcfcfcnJ/t+J91Ptt9QRKQATGR1zp8CXwe+CSSztN8vA/e7+/vMrASoyNL7HiekZZlFRDJMJPgT7v61bO3QzGqBNwN/AuDuQ8BQtt7/+H2lbzWdU0TkmImM8f/CzD5qZnPNrGH4awr7XAy0A981s2fN7FtmVjmF9zspncAlIpJpIsF/Pekx/idIr9K5Hlg3hX1GgPOAr7n7KqAPuGXsi8zsRjNbZ2br2tvbJ7Wj4WWZZ9GCoiIiOXfKoR53X5zlfe4B9rj7U8Hjn3GC4Hf3NcAagNWrV08uudXjFxHJMJFZPVHgJtLj8gC/Br7h7vHJ7NDdD5jZbjNb4e6vAFcCmyfzXqeiM3dFRDJN5ODu14Ao8NXg8R8H2/50Cvv9GHB7MKPnNeDDU3ivkwppWWYRkQwTCf4L3P2cUY8fNrNNU9mpu28EVk/lPSbi2KyeXO9JRGT2mMjB3aSZvW74gZktIXvz+XPq2MHdPBciIjKDTKTH/yngETN7jfSw+WnkaGgm244t2aDkFxEZNpFZPWvNbBmwItj0irvHcltWdmiEX0Qk03iXXrzC3R82s/eOeWqpmeHud+W4tqnTIm0iIhnG6/FfCjwMvOsEzzkw44N/ZK0e9flFREaMdwWuzwR3/9Hdt49+zsyyfVJXTmgev4hIponM6rnzBNt+lu1CcsG0OqeISIbxxvhPB84AaseM89cAZbkuLBuOLdKm5BcRGTbeGP8K4GqgjuPH+XuA/5bDmrJGQz0iIpnGG+O/G7jbzN6Yi6tjTQctyywikmkiJ3A9a2Y3kx72GRnicfeP5KyqrNGyzCIiY03k4O4PgDnA24DfAAtID/fMeCHN4xcRyTCR4F/q7n8L9Ln7bcA7gQtzW1Z2mObxi4hkmEjwD6+732lmZwK1QEvuSsoeHdwVEck0kTH+NWZWD/wtcA9QBfxdTqvKEtNQj4hIhoks0vat4O5vgCW5LSe7RpZlznMdIiIzyXgncP3VeN/o7l/MfjnZpWWZRUQyjdfjrw5uVwAXkB7mgfTJXE/nsqhs0Tx+EZFM453A9Q8AZvYocJ679wSP/x74j2mpbopM8/hFRDJMZFZPKzA06vFQsG3G08FdEZFME5nV833gaTP7efD43cD3clVQNmmoR0Qk00Rm9XzOzO4Dfi/Y9GF3fza3ZWWHLrYuIpJpvFk9Ne7ebWYNwI7ga/i5Bnc/kvvypkbLMouIZBqvx/8j0ssyr+f40RILHs/4Of1aq0dEJNN4s3quDm5nxWUWTyyd/Cklv4jIiPGGes4b7xvdfUP2y8mu4aEeERE5Zryhnv8zznMOXJHlWrJOi7SJiGQab6jn8uksJBe0LLOISKaJzOMnWI55Jcdfgev7uSoqW9TjFxHJdMrgN7PPAJeRDv5fAr8PPE76xK4ZLWSaxy8iMtZElmx4H3AlcMDdPwycQ/piLDPe8MFdzeoRETlmIsE/4O4pIGFmNcAhYOFUd2xmYTN71szunep7nYpiX0TkmImM8a8zszrgm6RP5uoFnszCvj8OvATUZOG9TmhkOqeSX0RkxEl7/Gb2FTO7xN0/6u6d7v514Crg+mDIZ9LMbAHpi7Z/61SvneJ+AM3qEREZbbwe/6vAF8xsLvAT4I4sLs72JeCvOXaxlwxmdiNwI8CiRYsmtRPN6hERyXTSHr+7f9nd3whcChwGvmNmL5vZZ8xs+WR3aGZXA4fcff14r3P3Ne6+2t1XNzc3T3JfwXtN6rtFRArTKQ/uuvtOd/+8u68CriO9Hv9LU9jnJcA1ZrYD+DFwhZn9cArvd1LD0zk1q0dE5JhTBr+ZRczsXWZ2O3Af8Arw3snu0N0/7e4L3L0N+CDwsLv/0WTfbzylkXTzhhKpXLy9iMisNN4ibVeR7uG/g/TF1X8M3OjufdNU25SVl4QB6B9K5rkSEZGZY7yDu58mvSb/J939aC527u6/Bn6di/cGKAmHiISMvlgiV7sQEZl1xlukbcavvnkqZkZFSVg9fhGRUSZy5u6sVlkaoX9IPX4RkWEFH/wVJWH61OMXERlRBMEfoV9j/CIiI4og+NXjFxEZreCDv7I0woCCX0RkRMEHf7rHr6EeEZFhRRH8/TH1+EVEhhVB8EfU4xcRGaXgg7+yNMzAUBLXQm0iIkARBH91WZREynX2rohIoOCDv7GyBIAjfUN5rkREZGYo+OBvqioFoKM3ludKRERmhoIP/saqdI//cK96/CIiUBTBn+7xH+5Tj19EBIoh+IMx/g71+EVEgCII/rJomKrSiIZ6REQCBR/8AK01pezt7M93GSIiM0JRBP/y1mq2HOzNdxkiIjNC0QT/jsN9DMZ1EpeISNEEf8pRr19EhCIJ/rPm1wKwcU9nfgsREZkBiiL4FzaU01RVwrM7j+a7FBGRvCuK4DczVp/WwK9fbaerP57vckRE8qoogh/gz69YytH+Ib77xPZ8lyIikldFE/xnzq/lDW0N/Mdz+/NdiohIXhVN8ANce+58thzqZe1LB/NdiohI3hRV8L/v/AUsaarkph9uYNPuznyXIyKSF0UV/CWREHfceBFl0RAf+uZveX5PV75LEhGZdkUV/ACtNWV84i3L6RtK8pc/2agrc4lI0Zn24DezhWb2iJltNrMXzezj013DR960mH96z1lsPdTLm//5EZ7dpfn9IlI88tHjTwCfdPeVwEXAzWa2crqL+NCFi/j5Ry+mqjTCe776BDf9cD0PvHhgussQEZl20x787r7f3TcE93uAl4D5010HwKpF9Xzjj8/nshXN3PfCAf77D9Zz8+0b2KC/AESkgJm752/nZm3Ao8CZ7t495rkbgRsBFi1adP7OnTtzWstgPMm/PPAKP3pqF2bw3T+5gLamSlprynK6XxGRXDGz9e6+OmN7voLfzKqA3wCfc/e7xnvt6tWrfd26ddNS16HuQd7/jSfZebifaNi486aLOWt+LWY2LfsXEcmWGRX8ZhYF7gUecPcvnur10xn8AO09Mb70q1e5/aldACyoL+ctr2/lrStbuXhp07TVISIyFTMm+C3ddb4NOOLun5jI90x38A97fEsHT20/zFd/vY1kygmHjPqKEkojIa5a2crfX3PGtNckIjJRMyn43wQ8BjwPpILN/8Pdf3my78lX8A/bfaSfaDjEtx9/jQPdMX6xaR8An333mcyvK+f3ljURCRfdKREiMsPNmOCfjHwH/1h9sQQfWPMkL+xNH48uj4a5/PRmrji9lTPn13D6nJo8VygicvLgj+SjmNmusjTCnTddzMZdnWw51Mtzezp5aPNBfvn8AcIhY/Vp9Wze182FSxr5X+8+g0goRHN1ab7LFhEB1OPPmkQyxSsHe/jhb3fx0OaDdPTGRp4ri6aD/0NvOI0/u3SJZgiJyLTQUM80cnc27eli/c6j/HTdbpqqSnl8awcA4ZBRVx5lxZxqqssiLKyvYHlrNe89b76OE4hIVin486x/KMGd6/dwoHuQfZ2DPLalg8rSMDsP9wNw5vwahhIpDnQN8v7VCymPhrnx0iXUlEXzXLmIzFYK/hkqlXK+/+QObntyJ7uP9JNIHfv3CBk0VJbw3vMWsLSlirm1ZaxaVE9VqQ7NiMipKfhngVgiCcBP1+0hkUzx+NYO+oeSPLHt8MhrKkrClEfDxJMplrdWc9aCWi5f0UJ5SZgL2hryVbqIzEAK/lmsvSdG/1CCnYf7uXPDHh7afJD+oWTG6xbUl3Pl6S2UlYTpGUxw2fJmhpIpLlrSSFOVZhWJFBsFf4HpiyVIJJ3n93ZxsHuQ5/d28ZN1u0/4gQCwpLmS6tIIqxbVs2pRHUtbqvj249tZtbCOS5e3UBoN0RdLMKe2jJJwSAeaRQqAgr9IHO6NUV0WZffRfh57tZ2XD/TQ0Rvj8a0dRMMhegYTGd9jBqN/DZY0V3LzZUs5Y34Nc2rK2Nc5yMKGcqp1oFlkVlHwF7lUyukbSvDYlvRxg8e2tHP12fOoLAnz9Udf49FX28f9/rm1ZcyrK2d5axV7jg6worWai5Y0csHiBuLJFOt2HOUNixuoLY+y43Afr2uuwt11zoJIHin4ZVw9g3FCZuztHGBZSxVPbDvMi/u6ePlAD4PxJK8c6GFbex8AjZUl9MQSDCVSmEHIjGTKaa0pZUF9Bet3HmVxUyW7jvTzlte38C/vP4easijxZIpIyIglUpRFw3lusUjhU/DLlHUNxNl6qJdVC+sYSqbYuLuT3752mMF4ipXzavjef25ny8FeemLHDyfNryuneyB+3Pb5deVctqKZXUf6Of+0emrLo6Q8fSD7bWe08vq5NfpwEJkiBb9Mm4GhJHc9u4fLV7Sw5tHX2LDrKLXlUTbu6mRRYwUL6yu4fwLXNz7/tHp6BxMc7otRGgmzoL6cNy9vZnlrNYubKjnSN8TipkrqKqJEdTBaJIOCX2aMVMp5dvdRlrVWs6Ojj6FEiubqUqpKI0RCIW7+0QZeOdjDnJoyWmvKiIRs3A8KM2iqKqW5qpTO/iEuXNLInNoy1u88yrzaMpIOyVSKC9oaOHtBHb2xBE9s6+Dqs+bxupZKKkoiuHvGENQrB3oIGSxrrZ6OH4tI1in4ZVZ7aX83dRVRYvEUu4/2c6RviMF4kv1dg/TFEhzqibH36ADrdh79nd43GjbeunIO2zv62Ly/m0uWNtJaXcY9m/aNnEV9zTnzeF1zFd2DcUojIRqrSrlwcQNnzKth4+5OmqvTxzZEZhoFvxSFVw+mp6+eMa+WypIwsUSKZ3YcoTQSZmFDOb/YtJ9kKsU158xnzWPb+PHTu2msKqGuvIQLlzRwz6Z99MeS1JRHj1thdayQwVnza9m0pwuA1ppS3rysmb2dA7x6sJfGyhLKSsJcuLiBRQ0VJJIpdhzuZ8/RAd559hwaKkvZeqiXZS1VOHDeojo27u7k+b1dXHl6Kyvm6K8MmToFv8gJDCVSRMM2Mu00lkjiDqWREId6YhhwqCdGLJGiL5agsjTMgvoK1jz6Gk9uO8zChnKe2n6EsKXfoyRs1FaUEIsn2Xmkn2RqYv+/asoixJPOQDyJGVy4uIFD3TGO9A8RT6SoLY+yr2uQ+ooo1547n9Vt9aQcmipLiEZCPPjiAWKJFK01ZSyoL+eSpU3c98IBOvuG+NiVywB4YW8X8+rKGYwnaaku1Ul6RUDBL5Ij8WTqhAeX93cNUB4NMxhPEUskSaacqtIIP12/h0Pdg3QPJugaiNNYWUIy5bx6qId/vPZM7nt+Pw9uPkg4ZKxoraY0EqJ/KMmDmw+OW0dpJEQskTrhcyvn1rB5f/fI49c1V3Lp8hYefvkgF7Q10DUQ56ntRzh3Yfovj3ecNYez5tfxn9s6qCqJ0FhVwgMvHuCyFS0sbamivSfG6XOq6RyI01RVQioFvbEE559Wz87D/ZyzsJYNuzqZU1NGfUWUspIwNWVRBuNJUu5UlGihwemg4BeZ5fYc7ScSClEWDfHS/h5qy6N09g/RORBnUUMFS1uq+NVLB6kpi/L9J3fQXF068ldDIuksa6liW0cf7d2DRCOhkSXBAeoqoixrqeKZHccfI2msLKF/KMlAPMn8unL2dQ0wmcgIGbxhcQMvH+ihsz9OW2MFsUSKwXiS1poyGipLaO+JkXRnTk0Z4ZDRUl3G9o5eVrc18OK+LhJJp76ihMaqEqpKI1zQ1kB7MBw3v66cRCrFwFCKipJw+vtrSqmvKOHJbYc5b1E9ZdEQ1WVR9nUNsOtwP6+fW0NDZQklkRCD8STxZIrq4MOpNBLCzEZOQhx7MmL3YJyKaHoosXIGr5ar4BeR4/TFEkTDIRwnEgoRMnh2dyd15VHm1ZVzoGuQBfXldA7EeWxLO+88ax4HugbpjydYUF/BS/u7KY+GeXDzQZa2VLGkqZJb124hnkzx3J4u3r96IZGQ8VpHL22Nldz73H66B+MMJVL0DyVZ0lTJaY0VHOyOURoN0VBRwm9fO0zfUJLyaJhI2JhbW8bWQ70Mj5jVV0Q52h+fdJtDBqNH38zSHxrtPekPkBVzqnluTxfLW6uYW1vOhl1HSaWcwUR6NdyewThNVaW8uK+L+ooSOnpjvOucedRXlPD09iM4cPqcasIho7Y8yqKG9EH/7R19hEPpEx2Xt1bzzI4jnD6nmm3tvZwxr5ZLljax83AfOw/3U1eRvlDT3qMDrJxXM6WJAwp+EZk2qZQTCh2/XIe7k0g5nUFwn+g61MmUEx7zff1DCdxhW3svZ86r5btP7GBubRlVpREG40mWtlRxoHuQ9p4YLdVl7O8aYDCe4mj/EF0DcebWlpFMOTsP9+M4bY2V1JRHiSVS7Ozo49ndnZw5r4bXOvrY3tHHhYsb2dc5QOdAPBjWGiSZcnoGEzRXl/L83i46++O8fm4N3QNxDnYPkkg5S1uq2HqoF0j/BTUwlDzp0Ft1aYSeWCLjg+hE7vnzSzh7Qd0Ef/LH08XWRWTajA19ADMjGrYTBv6wsaEPjBwPGA6/G960OOM1S5qrJlnp7y6RTJFI+cg5H92DceKJFI1VpSRTTntPjObqUtydzoE4sUSKknCIRCrFvZv2M7eujHecOZeDPYM0VJZw3/MHSKacObVltFSX0j2Y4O6Ne3nT0iZ2HennjHm1WW+DevwiIgXqZD1+zecSESkyCn4RkSKj4BcRKTIKfhGRIqPgFxEpMgp+EZEio+AXESkyCn4RkSIzK07gMrN2YOckvrUJ6MhyOTOd2lwc1ObiMNU2n+buzWM3zorgnywzW3eis9YKmdpcHNTm4pCrNmuoR0SkyCj4RUSKTKEH/5p8F5AHanNxUJuLQ07aXNBj/CIikqnQe/wiIjKGgl9EpMgUbPCb2dvN7BUz22pmt+S7nmwxs++Y2SEze2HUtgYze8jMtgS39cF2M7Nbg5/Bc2Z2Xv4qnzwzW2hmj5jZZjN70cw+Hmwv2HabWZmZPW1mm4I2/0OwfbGZPRW07d/MrCTYXho83ho835bXBkySmYXN7Fkzuzd4XNDtBTCzHWb2vJltNLN1wbac/m4XZPCbWRj4CvD7wErgOjNbmd+qsuZ7wNvHbLsFWOvuy4C1wWNIt39Z8HUj8LVpqjHbEsAn3X0lcBFwc/DvWcjtjgFXuPs5wLnA283sIuDzwL+6+1LgKHBD8PobgKPB9n8NXjcbfRx4adTjQm/vsMvd/dxRc/Zz+7vt7gX3BbwReGDU408Dn853XVlsXxvwwqjHrwBzg/tzgVeC+98ArjvR62bzF3A3cFWxtBuoADYAF5I+izMSbB/5PQceAN4Y3I8Er7N81/47tnNBEHJXAPcCVsjtHdXuHUDTmG05/d0uyB4/MB/YPerxnmBboWp19/3B/QNAa3C/4H4OwZ/0q4CnKPB2B8MeG4FDwEPANqDT3RPBS0a3a6TNwfNdQOO0Fjx1XwL+GkgFjxsp7PYOc+BBM1tvZjcG23L6ux2ZbKUyM7m7m1lBztE1syrgTuAT7t5tZiPPFWK73T0JnGtmdcDPgdPzW1HumNnVwCF3X29ml+W5nOn2Jnffa2YtwENm9vLoJ3Pxu12oPf69wMJRjxcE2wrVQTObCxDcHgq2F8zPwcyipEP/dne/K9hc8O0GcPdO4BHSQx11ZjbcYRvdrpE2B8/XAoent9IpuQS4xsx2AD8mPdzzZQq3vSPcfW9we4j0B/wbyPHvdqEG/zPAsmBGQAnwQeCePNeUS/cA1wf3ryc9Bj68/b8GMwEuArpG/fk4a1i6a/9t4CV3/+Kopwq23WbWHPT0MbNy0sc0XiL9AfC+4GVj2zz8s3gf8LAHg8Czgbt/2t0XuHsb6f+vD7v7H1Kg7R1mZpVmVj18H3gr8AK5/t3O94GNHB4weQfwKulx0f+Z73qy2K47gP1AnPT43g2kxzbXAluAXwENwWuN9OymbcDzwOp81z/JNr+J9Djoc8DG4Osdhdxu4Gzg2aDNLwB/F2xfAjwNbAV+CpQG28uCx1uD55fkuw1TaPtlwL3F0N6gfZuCrxeHsyrXv9taskFEpMgU6lCPiIichIJfRKTIKPhFRIqMgl9EpMgo+EVEioyCX4qWmSWDFRGHv7K2iquZtdmoFVRFZhIt2SDFbMDdz813ESLTTT1+kTGC9dH/OVgj/WkzWxpsbzOzh4N10Nea2aJge6uZ/TxYO3+TmV0cvFXYzL4ZrKf/YHAGLmb2F5a+tsBzZvbjPDVTipiCX4pZ+Zihng+Meq7L3c8C/h/pVSMB/i9wm7ufDdwO3BpsvxX4jafXzj+P9BmYkF4z/SvufgbQCfyXYPstwKrgff4sN00TOTmduStFy8x63b3qBNt3kL4IymvB4nAH3L3RzDpIr30eD7bvd/cmM2sHFrh7bNR7tAEPefpCGpjZ3wBRd/+smd0P9AL/Dvy7u/fmuKkix1GPX+TE/CT3fxexUfeTHDum9k7S662cBzwzavVJkWmh4Bc5sQ+Mun0yuP8E6ZUjAf4QeCy4vxa4CUYunlJ7sjc1sxCw0N0fAf6G9HLCGX91iOSSehpSzMqDK1wNu9/dh6d01pvZc6R77dcF2z4GfNfMPgW0Ax8Otn8cWGNmN5Du2d9EegXVEwkDPww+HAy41dPr7YtMG43xi4wRjPGvdveOfNcikgsa6hERKTLq8YuIFBn1+EVEioyCX0SkyCj4RUSKjIJfRKTIKPhFRIrM/wcmfyS6WDc+/AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'\\nMAE did not hit bottom. More epochs might lead to lower MAE. \\nUsually we want to pick an epoch number when MAE is at the lowest.\\n'"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.plot(range(1, len(average_mae_history) + 1), average_mae_history)\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Validation MAE')\n",
    "plt.show()\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "MAE did not hit bottom. More epochs might lead to lower MAE. \n",
    "Usually we want to pick an epoch number when MAE is at the lowest.\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}