notebooks/1b - Simple Neural Network with Keras.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Single Layer Neural Network Example\n",
    "\n",
    "Builds a simple neural network with Keras ([code](../examples/mnist/mnist0.py))\n",
    "\n",
    "### MNIST Dataset Overview\n",
    "\n",
    "The MNIST database is a large dataset of handwritten digits used for training image processing algorthms. This data set has 50,000 training example images, and 10,000 testing images. Each image has been flatted from an array of size (28,28) to a 1-D numpy array with 784 features (28 * 28). \n",
    "\n",
    "### Code Overview:\n",
    "1. Import libraries\n",
    "2. Import data for training and testing \n",
    "3. Define Model\n",
    "4. Define Loss function\n",
    "5. Define Training procedure\n",
    "6. Run Model code \n",
    "    a. Train model\n",
    "    b. Test model\n",
    "7. Plot accuracy of results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example Code\n",
    "\n",
    "import libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from __future__ import absolute_import, division, print_function\n",
    "import os\n",
    "os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'\n",
    "\n",
    "import time \n",
    "import tensorflow as tf\n",
    "import matplotlib.pyplot as plt\n",
    "mnist = tf.keras.datasets.mnist"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set training and network parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Training Parameters\n",
    "epochs = 10\n",
    "\n",
    "# Network Parameters\n",
    "NUM_INPUTS = 784\n",
    "NUM_OUTPUTS = 10 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import mnist data and preprocess data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import Dataset\n",
    "(x_train, y_train),(x_test, y_test) = mnist.load_data()\n",
    "x_train, x_test = x_train / 255.0, x_test / 255.0\n",
    "y_train = tf.keras.utils.to_categorical(y_train)\n",
    "y_test = tf.keras.utils.to_categorical(y_test)\n",
    "x_train = x_train.reshape(-1, NUM_INPUTS)\n",
    "x_test = x_test.reshape (-1, NUM_INPUTS)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = tf.keras.models.Sequential()\n",
    "model.add(tf.keras.layers.Dense(NUM_OUTPUTS, activation='softmax', input_dim=NUM_INPUTS))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define optimizer, loss\n",
    "Compile model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense (Dense)                (None, 10)                7850      \n",
      "=================================================================\n",
      "Total params: 7,850\n",
      "Trainable params: 7,850\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.compile(optimizer='adam', \n",
    "              loss='categorical_crossentropy', \n",
    "              metrics=['categorical_accuracy'])\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Train model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 48000 samples, validate on 12000 samples\n",
      "Epoch 1/10\n",
      "48000/48000 [==============================] - 5s 107us/sample - loss: 0.5057 - categorical_accuracy: 0.8695 - val_loss: 0.3161 - val_categorical_accuracy: 0.9150\n",
      "Epoch 2/10\n",
      "48000/48000 [==============================] - 5s 97us/sample - loss: 0.3179 - categorical_accuracy: 0.9117 - val_loss: 0.2841 - val_categorical_accuracy: 0.9209\n",
      "Epoch 3/10\n",
      "48000/48000 [==============================] - 5s 95us/sample - loss: 0.2929 - categorical_accuracy: 0.9184 - val_loss: 0.2752 - val_categorical_accuracy: 0.9243\n",
      "Epoch 4/10\n",
      "48000/48000 [==============================] - 5s 102us/sample - loss: 0.2817 - categorical_accuracy: 0.9210 - val_loss: 0.2711 - val_categorical_accuracy: 0.9249\n",
      "Epoch 5/10\n",
      "48000/48000 [==============================] - 5s 103us/sample - loss: 0.2735 - categorical_accuracy: 0.9233 - val_loss: 0.2708 - val_categorical_accuracy: 0.9259\n",
      "Epoch 6/10\n",
      "48000/48000 [==============================] - 5s 104us/sample - loss: 0.2686 - categorical_accuracy: 0.9240 - val_loss: 0.2650 - val_categorical_accuracy: 0.9270\n",
      "Epoch 7/10\n",
      "48000/48000 [==============================] - 5s 104us/sample - loss: 0.2637 - categorical_accuracy: 0.9263 - val_loss: 0.2653 - val_categorical_accuracy: 0.9268\n",
      "Epoch 8/10\n",
      "48000/48000 [==============================] - 5s 102us/sample - loss: 0.2607 - categorical_accuracy: 0.9268 - val_loss: 0.2633 - val_categorical_accuracy: 0.9282\n",
      "Epoch 9/10\n",
      "48000/48000 [==============================] - 5s 102us/sample - loss: 0.2577 - categorical_accuracy: 0.9278 - val_loss: 0.2631 - val_categorical_accuracy: 0.9281\n",
      "Epoch 10/10\n",
      "48000/48000 [==============================] - 5s 100us/sample - loss: 0.2552 - categorical_accuracy: 0.9286 - val_loss: 0.2652 - val_categorical_accuracy: 0.9272\n"
     ]
    }
   ],
   "source": [
    "history = model.fit(x_train, y_train, epochs=epochs, validation_split=0.2, shuffle=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Test Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "evaluation = model.evaluate(x_test, y_test, verbose=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot Model Accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXxU1d348c83k8lKVrYAgSQqLhGtQACXtm5oXYtWrVptFa3bI612ffTRKrW/uvx+9uevLdaWtvBoa9FWH1tsRetCq31UEnYB9RFJgAhIyGSBrDOT7++PexOGZAJDyHAzyff9es0r9565c+ebgZzvnHPuOVdUFWOMMaa7JK8DMMYYMzBZgjDGGBOVJQhjjDFRWYIwxhgTlSUIY4wxUSV7HUB/GTFihBYXF3sdhjHGJJQVK1bsUtWR0Z4bNAmiuLiY5cuXex2GMcYkFBHZ3Ntzce1iEpHzRORDEdkoIndFeb5IRF4XkbUi8g8RKYwoXyEiq0VkvYjcGs84jTHG9BS3BCEiPuBx4HygFLhaREq7HfYo8JSqngg8ADzklm8HTlXVk4AZwF0iMjZesRpjjOkpni2I6cBGVd2kqu3AM8CsbseUAq+720s7n1fVdlVtc8tT4xynMcaYKOI5BjEO2BqxX43TGoi0BrgM+ClwKZAlIsNVtVZExgN/A44Cvqeq2w42gGAwSHV1Na2trX36BYw30tLSKCwsxO/3ex2KMUNaPBOERCnrvvDTd4F5InI98CbwCRACUNWtwIlu19KfReQ5Vf10nzcQuRm4GWDChAk93qy6upqsrCyKi4sRiRaOGWhUldraWqqrqykpKfE6HGOGtHh23VQD4yP2C4F9WgGquk1Vv6Sqk4F73LKG7scA64HPdX8DVZ2vqmWqWjZyZM+rtFpbWxk+fLglhwQiIgwfPtxafcYMAPFMEBXARBEpEZEU4CpgceQBIjJCRDpjuBtY4JYXiki6u50HnAZ82JcgLDkkHvs3M2ZgiFsXk6qGRGQO8ArgAxao6noReQBYrqqLgTOAh0REcbqYbndffhzwE7dcgEdV9b14xWqMMQNduENpaAkSaGqnrrnd+dnUTqC5nZx0P9fMKOr394zrRDlVfQl4qVvZfRHbzwHPRXndq8CJ8YztcKitreXss88GYMeOHfh8Pjq7wsrLy0lJSTngOWbPns1dd93FMcccc1DvfeGFF9LY2Mhbb7118IEbY+JKVdndFnIq+K4KP0igqY1AU7Cr4o/8Wd8SpLfb90yekJt4CWKoGz58OKtXrwZg7ty5DBs2jO9+97v7HKOqqCpJSdF7+xYuXHjQ71tbW8t7771HWloaW7ZsiTqA3x9CoRDJyfZfyMSBKrTUQV0l1FVB3Wb3p/vY416vIkkRD+m27zvA8+4jydf7cz1e3+2cyalo5kiC6SNo8ufTmJRPQHKo0Rx2hLOpafN1+8Yf7KrwQx3Ra3u/T8jLSCE/03kcNyab/IwU8jJTyM/wOz8zU7qOyctIIT3FF5d/Bvvr9sDGjRu55JJL+OxnP8uyZcv461//yg9/+ENWrlxJS0sLV155Jffd5zS0PvvZzzJv3jwmTZrEiBEjuPXWW1myZAkZGRn85S9/YdSoUT3O/9xzz3HJJZeQk5PDs88+y/e+9z3AacXccsstVFZWIiLMnz+fGTNmsHDhQh577DFEhClTprBw4UKuvfZaLr/8ci655BIAhg0bxp49e3jttdd4+OGHGTFiBOvXr+e9997j4osvZtu2bbS2tvKtb32Lr3/96wD87W9/4wc/+AHhcJjRo0ezZMkSjjnmGMrLy8nPzyccDjNx4kSWL19Ofn7+Yfr0zYARaoP6LftW/J3JoH4ztDXue3zGCMgrhsJpkD3GqaC1w0km2rHvoyMcsR/l+a5HeL/Ph8JhgsEg7cF2gqEQwXCYUChEKBwmFA6THG4lt6OePNlDCpAHRH6P36Np1EkuDb5cmpLzaU0dTjB/JB2FI5Fho0jOHk1a3hgy88eSm5NLXqafYanJA2YcbsgkiB++uJ4N2xoPfOBBKB2bzf0XH9+n127YsIGFCxfyy1/+EoCHH36Y/Px8QqEQZ555JpdffjmlpftOPG9oaOD000/n4Ycf5tvf/jYLFizgrrt6rGDCokWLeOihh8jJyeHaa6/tShC3334755xzDnPmzCEUCtHc3MyaNWt45JFHePvtt8nPzycQCBww9nfffZcNGzZ0tUyefPJJ8vPzaW5upqysjMsuu4y2tjZuu+023nrrLYqKiggEAvh8Pq6++mr+8Ic/MGfOHF555RWmTZtmyWGwUnW+6Xf/9l9X5SSAxm3sc+V7chrkFjlJoOhUyHO384qd8tRh/RZaKNxBzZ42djS08mljKzsaWtnR2Na1/Wmj82hqD/d4bU66n4LsNEbnpDEqK5XhmSkMTxfGJO9hZFIjw6knJ1xHVihAZlstw5pqGN+0E/bshD0fQGMvf2P+TBg2EjJHwTD3kTkqomz03u1+/Cz2Z8gkiIHmyCOPZNq0aV37ixYt4re//S2hUIht27axYcOGHgkiPT2d888/H4CpU6dGHV/45JNP2LJlCyeffDIiQjgc5oMPPuDYY4/lH//4B8888wwAycnJZGdn88Ybb3DllVd2VdKxVNannHLKPt1Wjz32GIsXOxeoVVdX8/HHH7N161bOPPNMioqK9jnvjTfeyBVXXMGcOXNYsGBBV2vDHEDbHugIQpIfkpLdh9s14qX2pp4JoH7z3pZAqGXf47PGOhV+yen7JoC8Yqfi66WrNVaqSmNriJ2NreyIqOyd7bau7V172nr05/t9wqisNApy0jhuTDZnHDOKgpxURmenUZDtlI/KSjv07pxwEJp2QVfS2Olu17g/P4Xaj2HLO9AcoOf0McCfAZkj3aQxCgpOhDP+/dDiimLIJIi+ftOPl8zMzK7tjz76iJ/+9KeUl5eTm5vLtddeG3UeQOSgts/nIxQK9Tjm2Wefpba2tmuSWUNDA8888wxz584Fel5CqqpRm7PJycl0dHQAEHab1dFif+2113jzzTd59913SU9P57Of/Sytra29nre4uJi8vDyWLl3KqlWrOPfcc6N+PkNGRwc074LGT6BxO+ze5nyzbtzulO3e7uy374n++q5k4SaM7gnE120/KTniGHff54/9eXDi6UwGTTX7xpOS5VT2w4+Co2bumwByxoM/LeaPRlVpCYZpagvT0h6mqT1Ec3uY5vYQu1tDXZX9pw3uz0anRdAS7PmtPzfD/dafnUbpmGxG53RW+qldSSE/I4WkpMOQcH1+p4sse8yBjw2HnP8f+ySSnc7nvsdNJoFNcQt1yCSIgayxsZGsrCyys7PZvn07r7zyCuedd16fzrVo0SJee+21rtbJRx99xEUXXcTcuXM588wz+eUvf8mcOXMIh8M0NTUxc+ZMvvzlL/PNb36zq4spPz+f4uJiVqxYwZe+9CVeeOEFwuGef3TgJKD8/HzS09NZv349FRUVAJx22mnceeedbN68uauLKbIVcc011zB79uxeB+cHhVD73go+asW/3fnZEdz3deKDLLcCGXksHHk2ZBWALwU6Qu4j7Lxun/2Q8+00cr8j2PvzoTbo2NPt+M5jIvY7Iva1w4ktrxiOuQDyiujILaZt2HiaMsfTlJRFU3sHLcEQTW1hp0LfHaKpNkxL+ydOZR8M09S2t7J3fkaWOeUtwXCvV+10SvElMSo7lYLsNErHZnPWsaO6un9GZ6VSkOMkhTR/fAZx486X7PzbZxV48vaWIAaAKVOmUFpayqRJkzjiiCM47bTT+nSejz/+mB07dlBWVtZVNnHiRFJTU1mxYgXz5s3jpptu4le/+hXJycn86le/Yvr06Xz/+9/n85//PMnJyUydOpXf/va33HLLLcyaNYtXX32Vc889l9TU1KjveeGFFzJ//nw+85nPcOyxxzJjhrPc1ujRo3niiSeYNWsWqsrYsWNZsmQJAJdeeik33HAD119/fZ9+zwGhtbFnxd+17T6ad/V8nT8Dssc6lWzRqU4SyBrrlGWPgexxTtdB0sCp0D5tbGXF5jpWbK7jvU8aaGwJ0vxpmOatoa4KH2rcx4Gl+31kpPjISPWR4U8mI9VHZkoy+ZkpZKb4SE9JJjPFR0ZqMhkpvl7KkhmdnUp+ZsqAGdAdjEQPlKITRFlZmXa/YdD777/Pcccd51FEpjfvvvsud999N0uXLu31GM/+7UJtTrN996ewZwfs3rG3FdD52L09epdPxnC3sh/jJoFuFX/WGEjL8X7cYD9C4Q4+2LG7KyGs2FzHJ/XOOEJKchKTxmYzYliqW8En71t5p/jISEkmMzWyLHlvMkhJJt3vw3c4unFMzERkhaqWRXvOWhDmsPrxj3/M/PnzuwbLD5tga0SFv8NNAtu7JYId0BLlCpPILp/RpU7fevbYvS2Bzp8H0b8+UDQ0B1m5ZW8yWFNdT7N75c6orFTKivOYfVoxU4vyOH5sDinJg7hL0PRgCcIcVvfccw/33HNP/52wvSmi0t/hVPi7t7sJIKK8taHna5OSnatAsgogrwQmnOxU9J1lWQUwrAAyRwyoLp++UlU27WpixeY6VroJ4aOdTkvIlyQcNyaLK6YWMqUoj6lFeYzLTbfumyHOEoQZeFSdq3t2fdSt8o/89u9ut+/u+XpfilOxZ42GkUdDyeed7awxe8uzxkB6/iFfVjmQtbSHWVNdvzchbKmjvtkZEM9J9zNlQi6zThrLlKI8PlOYS2aqVQdmX/Y/wsRH52zWjrAzW7XrKphwt/JQz+M0DI074U9f3vecyelO5T6sAEYfD0ed3fPbflYBpOcN6H7+eNlW39LVVbRySx0btjV2Ledw1KhhnFs6mqlu6+CIEcMOzyWdJqFZgjC96+jYtxLfp3KPqNi7V/bqLnOwX+4aOEk+p4/f54ek9L376UG4dH5E5T96wA/wHk7BcAcbtjU6CWGL00LY3uDMnUn3+zhpfC63nn4kU4vymDwhl9yMAy8MaUx3liCMoyPsrH3TUu/063eEiDqDM1LngmhdE6pSwR+x3/WcW+l3zf71HbhrJ7Uejruy3369RNYe6iDQ1M66TxpY4Q4or62upzXoJOFxuelMK87vah0cW5BFsm/wdp2Zw8cSRBz1x3LfAAsWLOCCCy6goCD6ZJn29nYKCgq4/fbb+dGPfhR7gF1JoQ5adwMdTiWemuXOpu2tcu8st0roYHQuA1Hf3E5dc5C65nZnuylIfbOznHNdc9B9fm955HpAfp9w/NgcrplRxNSiPKZMyKMgJ/GunjKJwRJEHMWy3HcsFixYwJQpU3pNEC+//DKlpaU8++yzB04QHWHnip7WemeyF+pU/hn5kJ4LKcN67cax5b33ag919Kzou7ad9fwjK/v65iD1LUHCvSzxLOIMHOdlpJCb4WdUVhpHj84iLyOFvAw/uRkpHFOQxQnjchJ3VrBJOPbX7pEnn3ySxx9/nPb2dk499VTmzZtHR0cHs2fPZvXq1agqN998M6NHj2b16tVceeWVpKenR215LFq0iG9/+9s89thjVFRUdC2zsWzZMu68806am5tIS/Gz9L+eJEVb+d6PHuPVt8pJ8iVz6y0382/fuJPC8eNZt24duanCu+++y7333strr73GvffeS01NDZs2baKgoIC5c+dy/fXXs2fPHpKSkvjFL37RNXv6wQcfZNGiRSQlJXHRRRfxta99ja9+9auUl5cDzuS36667rmt/INu1p41VW+pZtaWOLYFmp9LvrOi7favvLjU5qauiz3Mr9ly3onfK91b6nWXZ6X6bQGYGnKGTIJbcBTv6+a6lBSfA+Q8f9MvWrVvHCy+8wNtvv01ycjI333wzzzzzDEceeSS7du3ivfecOOvr68nNzeXnP/858+bN46STTupxrqamJv75z3+ycOFCduzYwaJFi5g2bRqtTXu46sov8/yCnzHl2CIaGhtJTQrzi6eXsK2+lTXr3seXnOws732Agd9Vq1bx5ptvkpaWRnNzM6+++ippaWl88MEHXHfddSxbtowXX3yRJUuWUF5eTnp6etfaS2lpaaxbt45JkyaxcOFCZs+efdCfV7x1zh5e6Q72rtpaz+baZsDp0hmXm05eZgqjs9M4ZvTeyj43M7LS97vf9uN38xZjDrehkyAGkNdee42KioquNZNaWloYP348X/jCF/jwww+54447uOCCC2Ja6XTx4sWcc845pKWlccWXLqVs+nQevftW3l+1kgljRjDl+KMgLZecERPBn8Fr/30fd955Jz63qyiW5b1nzZpFWprTz93W1sacOXNYs2YNycnJfPzxx12/0w033EB6evo+573xxhtZuHAhjzzyCH/6059YtWrVwX9g/ax2Txsrt9R3JYS11Q1dK4COykplyoQ8rpkxgSkT8phkXTpmCBs6CaIP3/TjRVW54YYboo4XrF27liVLlvCzn/2M559/nvnz5+/3XIv+8DTLlpVTPKEQtIOduwK8+dZ/kz18DJKSAaNK92khxLK8d/elxiOX9/7JT37C+PHj+f3vf08wGGTYsGH7Pe8VV1zBgw8+yGmnncYpp5xCbm7ufn+f/tbZOli1pa4rKXS2DpKThOPHZnPltPFMKcpjyoRcmz1sTIShkyAGkJkzZ3L55Zdzxx13MGLECGpra2lqaiI9Pd1pCVxxBSUlJdx6660AZGVlsXt3xIzhcBBa66nbvpll775D9fIl+NMyIT2PXz/9Aotefoef/fznbN5yGytXrWLKlCk0NjaSmZnJueeeyxNPPMHnPvc5fD5fj+W9zznnHJ5//vleY29oaOCoo45CRHjyySfpXOzx3HPP5ZFHHukaK+k8b0ZGBmeddRZz5szhySefjOvnCk7rYFVn62CL0zqIXFtoyoQ8vjJ9AlOK8mzA15gDsAThgRNOOIH777+fmTNn0tHRgd/v55e//CU+n48bb7yx69v4I488AsDs2bP5+tdvJD01hfKXnyWFdgCeX/wS55x1Bv6xk5xZxiJccvmXuef+HzLv8cdZtGgRt912G62traSnp/PGG29wyy238NFHH3HiiSeSnJzMbbfdxq233srcuXO56aabKCgoYPr06b3GPmfOHC6//HIWLVrEzJkzu5YBv+iii1izZg1lZWX4/X4uvvjirhbSNddcw0svvdR1yW9/iaV18OUyax0Y01e23PdAFmp3L0d1J6+Bc9/etFznktTktISYWfzwww/T1tbG/fffH/Nrov3b7a91MDIrlSkTcpkyIc9aB8YcBFvuO5F0JoWWeghGJIWsAicx+NO9je8gXXzxxWzdupU33njjoF6nCuu3NbBySz2r3LWFqiJaB6Vu62CymxQK86x1YEx/swQxULQ2OstUB51K0FmYboybFBJ3puyLL74Y03GqSmuog6bWEHvaQmxvaOHGp/4FwIhhTuvgqunOlUUnjMuxS0mNOQzimiBE5Dzgp4AP+I2qPtzt+SJgATASCADXqmq1iJwEPAFkA2Hgx6r6bF9i6O3qmgEj1AoNnzhLXvhS3GWo3e6jQUxVaQ91sKfNSQhNbWFC7lVUKT4hPcXHT686yVoHxngobglCRHzA48A5QDVQISKLVXVDxGGPAk+p6pMichbwEPBVoBn4mqp+JCJjgRUi8oqq1h9MDGlpadTW1jJ8+PCBV8F0hJ37GuzZ6YwjZI2FYSMH9fpGnQmhyU0KwbCTEPy+JLLSksl0b2G5u6GO1PxsTi0Z53HExgxt8WxBTAc2quomABF5BpgFRCaIUuBb7vZS4M8Aqvo/nQeo6jYR2YnTyjioBFFYWEh1dTU1NbHdTP2waW9y1kPqCEFKptONVB/AaUQNHuEOpS3UQVsoTFuwo+veBD6BVL+P1OQkUpOTwJfEHqDzLs9paWkUFhZ6FrcxxhHPBDEO2BqxXw3M6HbMGuAynG6oS4EsERmuqrWdB4jIdCAF+Lj7G4jIzcDNABMmTOgRgN/vp6Sk5NB+i/60bTUs+T5sXQZjJ8P5/wfGT/M6qn5T39zOu5sCvPPxLt7ZVMv/fOpU+VlpyZx8xHBOPXI4px45gqNHDxt4LTpjTA/xTBDRaoDu19R+F5gnItcDbwKfAKGuE4iMAX4HXKfa8w40qjofmA/OZa79E3YcNO2C1x+AlU9BxnD44jw46ZqEv91lU1uI8qoA73xcy9sf72L9tkZUnRvWTC/J57IphZxy5HCOH5tjC9EZk4DimSCqgfER+4XAtsgDVHUb8CUAERkGXKaqDe5+NvA34F5VfTeOccZPOAgVv4GlDzmXrJ78b3D6951B6ATUGgyzckudmxBqWbO1nlCHkuJLYkpRLt+aeTSnHjmcEwtzSUlO7ORnjIlvgqgAJopICU7L4CrgK5EHiMgIIOC2Du7GuaIJEUkBXsAZwP5THGOMn03/gCX/DjUfwJFnwXkPw8hjvI7qoATDHaytruftjU5CWLGljvZQB74k4cTCHG45/QhOPXIEU4vybFKaMYNQ3BKEqoZEZA7wCs5lrgtUdb2IPAAsV9XFwBnAQyKiOF1Mt7sv/zLweWC42/0EcL2qro5XvP2mbjP8/R54/0XIK4ar/gDHXJAQM54BAk3tPLdiK29/XEt5ZYDm9jAiUDomm+tOKeLUI0dQVpxHVprf61CNMXE2qJfaOKzam+Ffj8HbP3MuVf3cd+CUOQk1ye2l97bzgz+vo7apnaNGDXMHlYczo2Q4eZl203tjBiNbaiOeVGH9C/D3H0BjNUy6HM55AHIS5xr+2j1t3Ld4PX9bu50TC3N4+qYZHFuQ7XVYxhiPWYI4FDvWOeMMm//l3F3usl9D0aleR3VQOlsNja1BvveFY7jl80eQ7LMBZmOMJYi+aQ7A0h/D8gXOJLeLHoMp10FS4gzURrYaThiXwx+uOJljCrK8DssYM4BYgjgY4RCsWOgkh9ZGmHYTnHk3pOd5HdlBWfLedu61VoMx5gAsQcSq6l9Od9Kn66D4c3D+IzD6eK+jOiiBpnZ+8Jd11mowxsTEEsSBNFQ7A9Dr/wtyxsOXn4Ljvpgwl612imw1fPfco7nl9CPxW6vBGLMfliB6E2yBt38Ob/1fQOGMu+HUb0JKhteRHZRAUzv3/WUdf127nUnjsnn6CrtCyRgTG0sQ3anCB3+FV/4D6rdA6SVw7o8gt+digAPdy+ucVkNDS5DvnHM0t55hrQZjTOwsQUTa+b4zzlD5TxhVCte9CCWf9zqqgxZoauf+xet5cc02Jo3L5nc3zuC4MdZqMMYcHEsQ4Nz/+R8PQ/l8SM2CCx6FqbPBl3gfj7UajDH9JfFqwP5W+zH89hxnbkPZbDjzXsgc7nVUBy2y1XD8WGs1GGMOnSWIvBI4/lJnotuYE72Opk9eXreDe//8Hg0tQb59ztHcZq0GY0w/sASRlAQX/sTrKPqkrqmd+6zVYIyJE0sQCcpaDcaYeLMEkWDq3LGGxWu2UTrGWg3GmPixBJFAXlm/g3teWEd9czvfmnk0/3amtRqMMfFjCSIB1DW1M/fF9fxltdNqeOqG6ZSOtVaDMSa+LEEMcJGthjtnTuT2M4+yVoMx5rCwBDFAWavBGOM1SxAD0N/X7+A/rNVgjPGYJYgBpL65nbmL1/Pn1ds4bkw2T94wjePH5ngdljFmiLIEMUBsb2jhi/P+m7qmdu4422k1pCRbq8EY4x1LEAPE39d/Ss3uNp6/7RSmFuV7HY4xxmBfUQeI8qoAY3PSmDIhse5vbYwZvOKaIETkPBH5UEQ2ishdUZ4vEpHXRWStiPxDRAojnntZROpF5K/xjHEgUFXKKwNMK8lHEuxWpsaYwStuCUJEfMDjwPlAKXC1iJR2O+xR4ClVPRF4AHgo4rn/A3w1XvENJJtrm6nZ3ca0YutaMsYMHPFsQUwHNqrqJlVtB54BZnU7phR43d1eGvm8qr4O7I5jfANGeWUAgBklliCMMQNHPBPEOGBrxH61WxZpDXCZu30pkCUiiXe3nkNUXhUgL8PPUaOGeR2KMcZ0iWeCiNaZrt32vwucLiKrgNOBT4BQzG8gcrOILBeR5TU1NX2P1GMVVQGmFdv4gzFmYIlngqgGxkfsFwLbIg9Q1W2q+iVVnQzc45Y1xPoGqjpfVctUtWzkyJH9EfNh92ljK5trm5lu3UvGmAEmngmiApgoIiUikgJcBSyOPEBERohIZwx3AwviGM+A1Dn+YAnCGDPQxC1BqGoImAO8ArwP/FFV14vIAyLyRfewM4APReR/gNHAjztfLyJvAX8CzhaRahH5Qrxi9VJ5ZYDMFB+ldtMfY8wAE9eZ1Kr6EvBSt7L7IrafA57r5bWfi2dsA0VFVYApRXkk22J8xpgBxmolD9U3t/Php7uZbvMfjDEDkCUIDy2vqkMVptn4gzFmALIE4aGKqgApviROGp/rdSjGGNODJQgPLasMcGJhDml+n9ehGGNMD5YgPNLcHmLdJw3WvWSMGbAsQXhk9ZZ6Qh1q8x+MMQOWJQiPLKsMIAJTi+z+D8aYgckShEcqqgKUjskmO83vdSjGGBOVJQgPtIc6WLmlzu7/YIwZ0CxBeGDdtgZagx02/mCMGdAsQXigwl2gz1oQxpiB7IAJQkTmiIiNpPaj8soAR4zIZGRWqtehGGNMr2JpQRQAFSLyRxE5T+yuNoeko0OpqApY95IxZsA7YIJQ1XuBicBvgeuBj0TkQRE5Ms6xDUoffrqbxtaQdS8ZYwa8mMYgVFWBHe4jBOQBz4nI/45jbINSRZXdIMgYkxgOeD8IEfkmcB2wC/gN8D1VDbp3gvsI+H58QxxcllUGGJOTRmFeutehGGPMfsVyw6ARwJdUdXNkoap2iMhF8QlrcFJVKioDnHzEcGwoxxgz0MXSxfQSEOjcEZEsEZkBoKrvxyuwwWhLoJmdu9use8kYkxBiSRBPAHsi9pvcMnOQllXa+IMxJnHEkiDEHaQGnK4l4nwv68GqojJAXoafo0YO8zoUY4w5oFgSxCYR+aaI+N3HHcCmeAc2GJVXBSgrzicpycYfjDEDXywJ4lbgVOAToBqYAdwcz6AGo52NrWyubWa6zX8wxiSIA3YVqepO4KrDEMugVm7zH4wxCSaWeRBpwI3A8UBaZ7mq3hDHuAad8soAGVHmk8QAABN0SURBVCk+jh+b7XUoxhgTk1i6mH6Hsx7TF4B/AoXA7ngGNRiVVwaYWpRHss8W0DXGJIZYaqujVPUHQJOqPglcCJwQy8ndxf0+FJGNInJXlOeLROR1EVkrIv8QkcKI564TkY/cx3Wx/kIDUUNzkA8/3W3rLxljEkosCSLo/qwXkUlADlB8oBeJiA94HDgfKAWuFpHSboc9CjylqicCDwAPua/NB+7HGRCfDtyfyEuOL98cQNXGH4wxiSWWBDHfrZzvBRYDG4BHYnjddGCjqm5S1XbgGWBWt2NKgdfd7aURz38BeFVVA6paB7wKnBfDew5I5VUB/D7hpPG5XodijDEx22+CcBfka1TVOlV9U1WPUNVRqvqrGM49DtgasV/tlkVaA1zmbl8KZInI8Bhfi4jcLCLLRWR5TU1NDCF5o7wywImFuaT5fV6HYowxMdtvgnBnTc/p47mjzQbTbvvfBU4XkVXA6ThzLUIxvhZVna+qZapaNnLkyD6GGV8t7WHeq26w7iVjTMKJpYvpVRH5roiMF5H8zkcMr6sGxkfsFwLbIg9Q1W2q+iVVnQzc45Y1xPLaRLFqSx2hDrUJcsaYhBPLmkqd8x1ujyhT4IgDvK4CmCgiJTgtg6uAr0QeICIjgIDbUrkbWOA+9QrwYMTA9Lnu8wmnvCqACEwtTtgxdmPMEBXLTOqSvpxYVUMiMgensvcBC1R1vYg8ACxX1cXAGcBDIqLAm7hJSFUDIvIjnCQD8ICqBnq8SQKoqApwXEE22Wl+r0MxxpiDEstM6q9FK1fVpw70WlV9Ced+EpFl90VsPwc818trF7C3RZGQguEOVm6u58pp4w98sDHGDDCxdDFNi9hOA84GVgIHTBBD3bpPGmgJhm2A2hiTkGLpYvpG5L6I5OAsv2EOoNy9QZDNoDbGJKK+LAzUDEzs70AGo4qqAEeMyGRkVqrXoRhjzEGLZQziRfbOQUjCmf38x3gGNRh0dCgVVXWcd3yB16EYY0yfxDIG8WjEdgjYrKrVcYpn0PifnbtpaAkyzcYfjDEJKpYEsQXYrqqtACKSLiLFqloV18gSXIU7/jDDEoQxJkHFMgbxJ6AjYj/slpn9WFYZoCA7jcK8dK9DMcaYPoklQSS7q7EC4G6nxC+kxKeqVFQFmF6Sj0i0ZaWMMWbgiyVB1IjIFzt3RGQWsCt+ISW+LYFmPm1ss/EHY0xCi2UM4lbgaRGZ5+5XA1FnVxtHuY0/GGMGgVgmyn0MnCwiwwBRVbsf9QFUVAXIzfBz1MhhXodijDF9dsAuJhF5UERyVXWPqu4WkTwR+V+HI7hEVV4ZoKwon6QkG38wxiSuWMYgzlfV+s4d9xagF8QvpMS2s7GVqtpm614yxiS8WBKET0S61ooQkXTA1o7oRXmVu/6SJQhjTIKLZZD698DrIrLQ3Z8NPBm/kBJbRWWAjBQfx4/N9joUY4w5JLEMUv9vEVkLzMS5V/TLQFG8A0tU5VV1TJmQh9/Xl3UQjTFm4Ii1FtuBM5v6Mpz7Qbwft4gSWENLkA92NNry3saYQaHXFoSIHI1zH+mrgVrgWZzLXM88TLElnBWbA6hiNwgyxgwK++ti+gB4C7hYVTcCiMi3DktUCWpZZQC/T5g8IdfrUIwx5pDtr4vpMpyupaUi8msRORtnDML0oqIywImFuaT5fV6HYowxh6zXBKGqL6jqlcCxwD+AbwGjReQJETn3MMWXMFraw7z3SYONPxhjBo0DDlKrapOqPq2qFwGFwGrgrrhHlmBWba0jGFabIGeMGTQO6lpMVQ2o6q9U9ax4BZSoKirrEIEpRXleh2KMMf3CLtbvJ+VVtRxbkE1Out/rUIwxpl/ENUGIyHki8qGIbBSRHt1SIjJBRJaKyCoRWSsiF7jlKSKyUETeE5E1InJGPOM8VMFwBys311v3kjFmUIlbghARH/A4cD5QClwtIqXdDrsX+KOqTsaZc/ELt/wmAFU9ATgH+ImIDNjWzvptjbQEwzZAbYwZVOJZ6U4HNqrqJvc2pc8As7odo0DnokU5wDZ3uxR4HUBVdwL1QFkcYz0k5ZW1AEwrsfEHY8zgEc8EMQ7YGrFf7ZZFmgtcKyLVwEvAN9zyNcAsEUkWkRJgKjC++xuIyM0islxEltfU1PR3/DErr6yjZEQmo7LSPIvBGGP6WzwTRLRJddpt/2rgP1W1EOceE79zu5IW4CSU5cD/A94GQj1OpjpfVctUtWzkyJH9GnysOjqUiqoA04qt9WCMGVxiWe67r6rZ91t/IXu7kDrdCJwHoKrviEgaMMLtVupa1kNE3gY+imOsffbRzj00tASZXjLc61CMMaZfxbMFUQFMFJESEUnBGYRe3O2YLTirwyIixwFpQI2IZIhIplt+DhBS1Q1xjLXPOscfptsAtTFmkIlbC0JVQyIyB3gF8AELVHW9iDwALFfVxcB3gF+7iwAqcL2qqoiMAl4RkQ7gE+Cr8YrzUJVX1VGQncb4/HSvQzHGmH4Vzy4mVPUlnMHnyLL7IrY3AKdFeV0VcEw8Y+sPqkpFZYBpJfmI2DqGxpjBZcDOLUgEWwMt7GhsZboNUBtjBiFLEIegvCoAYAPUxphByRLEISivrCUn3c/EUcO8DsUYY/qdJYhDUFFVx7TifJKSbPzBGDP4WILoo527W6nc1cR0W17DGDNIWYLoo4rKOsDGH4wxg5cliD6qqAqQ7vdx/NjsAx9sjDEJyBJEHy2rDDClKBe/zz5CY8zgZLVbHzS0BPlgRyPTi617yRgzeFmC6IOVm+tQtfs/GGMGN0sQfbCsMoDfJ0webwnCGDN4WYLog4qqACeMyyE9xed1KMYYEzeWIA5SazDM2up6ppXY8t7GmMHNEsRBWrWlnmBYmWEJwhgzyFmCOEgVVQFEYGqRJQhjzOBmCeIglVcGOLYgm5x0v9ehGGNMXFmCOAjBcAcrt9TZ/R+MMUOCJYiDsH5bI83tYVt/yRgzJFiCOAgVlc4NgmyCnDFmKLAEcRCWVQYoHp7BqKw0r0Mxxpi4swQRo44OZfnmANPt8lZjzBBhCSJGG2v2UN8cZFqxJQhjzNBgCSJGy9zxhxk2QG2MGSIsQcSoojLA6OxUxuenex2KMcYcFpYgYqCqlFcGmFacj4h4HY4xxhwWcU0QInKeiHwoIhtF5K4oz08QkaUiskpE1orIBW65X0SeFJH3ROR9Ebk7nnEeSHVdCzsaW239JWPMkBK3BCEiPuBx4HygFLhaREq7HXYv8EdVnQxcBfzCLb8CSFXVE4CpwC0iUhyvWA+kvGv+gyUIY8zQEc8WxHRgo6puUtV24BlgVrdjFMh2t3OAbRHlmSKSDKQD7UBjHGPdr/LKADnpfo4eleVVCMYYc9jFM0GMA7ZG7Fe7ZZHmAteKSDXwEvANt/w5oAnYDmwBHlXVQPc3EJGbRWS5iCyvqanp5/D3qqgKMK04j6QkG38wxgwd8UwQ0WpT7bZ/NfCfqloIXAD8TkSScFofYWAsUAJ8R0SO6HEy1fmqWqaqZSNHjuzf6F07d7eyaVeTzX8wxgw58UwQ1cD4iP1C9nYhdboR+COAqr4DpAEjgK8AL6tqUFV3Av8NlMUx1l4tr6oDsBnUxpghJ54JogKYKCIlIpKCMwi9uNsxW4CzAUTkOJwEUeOWnyWOTOBk4IM4xtqr8soA6X4fk8blePH2xhjjmbglCFUNAXOAV4D3ca5WWi8iD4jIF93DvgPcJCJrgEXA9aqqOFc/DQPW4SSahaq6Nl6x7k95ZYApRbn4fTZlxBgztCTH8+Sq+hLO4HNk2X0R2xuA06K8bg/Opa6eamwN8v6ORu44e6LXoRhjzGFnX4v3Y0VVHao2/mCMGZosQexHeVUAv0+YPN5uEGSMGXosQexHRWWASeNySE/xeR2KMcYcdpYgetEaDLOmut66l4wxQ5YliF6s3lpPMKxMtwlyxpghyhJEL8orA4hAWZElCGPM0GQJohcVVQGOGZ1FTobf61CMMcYTliCiCIU7WLG5zsYfjDFDmiWIKNZva6S5PWwJwhgzpFmCiKKiyllZ3AaojTFDmSWIKJZVBigensGo7DSvQzHGGM9Yguimo0NZXhWw+z8YY4Y8SxDdbKzZQ11z0MYfjDFDniWIbsor3fEHSxDGmCHOEkQ3FVUBRmWlMiE/w+tQjDHGU5YgIqgq5ZUBppfkIxLtltrGGDN0WIKIUF3XwvaGVuteMsYYLEHsw8YfjDFmL0sQESqqAuSk+zl6VJbXoRhjjOcsQUQorwpQVpRHUpKNPxhjjCUIV83uNjbVNFn3kjHGuCxBuJa76y9NswRhjDGAJYguyyoDpPt9TBqb43UoxhgzIFiCcFVUBZg8IZeUZPtIjDEG4pwgROQ8EflQRDaKyF1Rnp8gIktFZJWIrBWRC9zya0RkdcSjQ0ROilecja1B3t/eaAv0GWNMhLglCBHxAY8D5wOlwNUiUtrtsHuBP6rqZOAq4BcAqvq0qp6kqicBXwWqVHV1vGJdsbmODoUZNv5gjDFd4tmCmA5sVNVNqtoOPAPM6naMAtnudg6wLcp5rgYWxS1KoKIyQHKSMHlCXjzfxhhjEkpyHM89DtgasV8NzOh2zFzg7yLyDSATmBnlPFfSM7EAICI3AzcDTJgwoc+BllcGOKEwh/QUX5/PYYwxg008WxDRZptpt/2rgf9U1ULgAuB3ItIVk4jMAJpVdV20N1DV+apapqplI0eO7FOQrcEwa6sb7PaixhjTTTwTRDUwPmK/kJ5dSDcCfwRQ1XeANGBExPNXEefupcbWIOdNKuD0o/uWYIwxZrCKZxdTBTBRREqAT3Aq+690O2YLcDbwnyJyHE6CqAFwWxJXAJ+PY4yMykrjZ1dPjudbGGNMQopbC0JVQ8Ac4BXgfZyrldaLyAMi8kX3sO8AN4nIGpyWwvWq2tkN9XmgWlU3xStGY4wxvZO99XFiKysr0+XLl3sdhjHGJBQRWaGqZdGes2nDxhhjorIEYYwxJipLEMYYY6KyBGGMMSYqSxDGGGOisgRhjDEmqkFzmauI1ACbD+EUI4Bd/RROorPPYl/2eezLPo+9BsNnUaSqUZeSGDQJ4lCJyPLergUeauyz2Jd9Hvuyz2Ovwf5ZWBeTMcaYqCxBGGOMicoSxF7zvQ5gALHPYl/2eezLPo+9BvVnYWMQxhhjorIWhDHGmKgsQRhjjIlqyCcIETlPRD4UkY0icpfX8XhJRMaLyFIReV9E1ovIHV7H5DUR8YnIKhH5q9exeE1EckXkORH5wP0/corXMXlJRL7l/p2sE5FFIpLmdUz9bUgnCBHxAY8D5wOlwNUiUuptVJ4KAd9R1eOAk4Hbh/jnAXAHzg2vDPwUeFlVjwU+wxD+XERkHPBNoExVJwE+nLtmDipDOkEA04GNqrpJVduBZ4BZHsfkGVXdrqor3e3dOBXAOG+j8o6IFAIXAr/xOhaviUg2zl0efwugqu2qWu9tVJ5LBtJFJBnIALZ5HE+/G+oJYhywNWK/miFcIUYSkWJgMrDM20g89f+A7wMdXgcyAByBc7/4hW6X229EJNProLyiqp8AjwJbgO1Ag6r+3duo+t9QTxASpWzIX/crIsOA54E7VbXR63i8ICIXATtVdYXXsQwQycAU4AlVnQw0AUN2zE5E8nB6G0qAsUCmiFzrbVT9b6gniGpgfMR+IYOwmXgwRMSPkxyeVtX/8joeD50GfFFEqnC6Hs8Skd97G5KnqoFqVe1sUT6HkzCGqplAparWqGoQ+C/gVI9j6ndDPUFUABNFpEREUnAGmRZ7HJNnRERw+pjfV9X/63U8XlLVu1W1UFWLcf5fvKGqg+4bYqxUdQewVUSOcYvOBjZ4GJLXtgAni0iG+3dzNoNw0D7Z6wC8pKohEZkDvIJzFcICVV3vcVheOg34KvCeiKx2y/5DVV/yMCYzcHwDeNr9MrUJmO1xPJ5R1WUi8hywEufqv1UMwmU3bKkNY4wxUQ31LiZjjDG9sARhjDEmKksQxhhjorIEYYwxJipLEMYYY6KyBGHMAYhIWERWRzz6bQaxiBSLyLr+Op8x/WlIz4MwJkYtqnqS10EYc7hZC8KYPhKRKhF5RETK3cdRbnmRiLwuImvdnxPc8tEi8oKIrHEfnUsz+ETk1+69Bf4uIunu8d8UkQ3ueZ7x6Nc0Q5glCGMOLL1bF9OVEc81qup0YB7O6q+420+p6onA08DP3PKfAf9U1c/grGPUOWt/IvC4qh4P1AOXueV3AZPd89war1/OmN7YTGpjDkBE9qjqsCjlVcBZqrrJXeRwh6oOF5FdwBhVDbrl21V1hIjUAIWq2hZxjmLgVVWd6O7/O+BX1f8lIi8De4A/A39W1T1x/lWN2Ye1IIw5NNrLdm/HRNMWsR1m79jghTh3PJwKrHBvTGPMYWMJwphDc2XEz3fc7bfZe/vJa4B/uduvA7dB172us3s7qYgkAeNVdSnOTYtygR6tGGPiyb6RGHNg6RGr24JzX+bOS11TRWQZzpetq92ybwILROR7OHdh61z19A5gvojciNNSuA3nbmTR+IDfi0gOzo2tHrNbfJrDzcYgjOkjdwyiTFV3eR2LMfFgXUzGGGOishaEMcaYqKwFYYwxJipLEMYYY6KyBGGMMSYqSxDGGGOisgRhjDEmqv8PhAUxPoGUonwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot Accuracy \n",
    "plt.plot(history.history[\"categorical_accuracy\"])\n",
    "plt.plot(history.history[\"val_categorical_accuracy\"])\n",
    "plt.ylabel(\"Accuracy\")\n",
    "plt.xlabel(\"Epochs\")\n",
    "plt.legend([\"Train Accuracy\", \"Test Accuracy\"], loc=\"upper left\")\n",
    "plt.show();"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}