intro-to-pytorch/Part 4 - Fashion-MNIST (Exercises).ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Part 4 - Fashion-MNIST (Exercises).ipynb",
      "version": "0.3.2",
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/agungsantoso/deep-learning-v2-pytorch/blob/master/intro-to-pytorch/Part%204%20-%20Fashion-MNIST%20(Exercises).ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "metadata": {
        "id": "Ah0exj2oC_pm",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Classifying Fashion-MNIST\n",
        "\n",
        "Now it's your turn to build and train a neural network. You'll be using the [Fashion-MNIST dataset](https://github.com/zalandoresearch/fashion-mnist), a drop-in replacement for the MNIST dataset. MNIST is actually quite trivial with neural networks where you can easily achieve better than 97% accuracy. Fashion-MNIST is a set of 28x28 greyscale images of clothes. It's more complex than MNIST, so it's a better representation of the actual performance of your network, and a better representation of datasets you'll use in the real world.\n",
        "\n",
        "<img src='https://github.com/agungsantoso/deep-learning-v2-pytorch/blob/master/intro-to-pytorch/assets/fashion-mnist-sprite.png?raw=1' width=500px>\n",
        "\n",
        "In this notebook, you'll build your own neural network. For the most part, you could just copy and paste the code from Part 3, but you wouldn't be learning. It's important for you to write the code yourself and get it to work. Feel free to consult the previous notebooks though as you work through this.\n",
        "\n",
        "First off, let's load the dataset through torchvision."
      ]
    },
    {
      "metadata": {
        "id": "weiUhfThDW5j",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# http://pytorch.org/\n",
        "from os.path import exists\n",
        "from wheel.pep425tags import get_abbr_impl, get_impl_ver, get_abi_tag\n",
        "platform = '{}{}-{}'.format(get_abbr_impl(), get_impl_ver(), get_abi_tag())\n",
        "cuda_output = !ldconfig -p|grep cudart.so|sed -e 's/.*\\.\\([0-9]*\\)\\.\\([0-9]*\\)$/cu\\1\\2/'\n",
        "accelerator = cuda_output[0] if exists('/dev/nvidia0') else 'cpu'\n",
        "\n",
        "!pip install -q http://download.pytorch.org/whl/{accelerator}/torch-0.4.1-{platform}-linux_x86_64.whl torchvision\n",
        "import torch"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "qxSFfSQMDn7x",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "from torch import nn, optim\n",
        "from torch.autograd import Variable\n",
        "\n",
        "\n",
        "def test_network(net, trainloader):\n",
        "\n",
        "    criterion = nn.MSELoss()\n",
        "    optimizer = optim.Adam(net.parameters(), lr=0.001)\n",
        "\n",
        "    dataiter = iter(trainloader)\n",
        "    images, labels = dataiter.next()\n",
        "\n",
        "    # Create Variables for the inputs and targets\n",
        "    inputs = Variable(images)\n",
        "    targets = Variable(images)\n",
        "\n",
        "    # Clear the gradients from all Variables\n",
        "    optimizer.zero_grad()\n",
        "\n",
        "    # Forward pass, then backward pass, then update weights\n",
        "    output = net.forward(inputs)\n",
        "    loss = criterion(output, targets)\n",
        "    loss.backward()\n",
        "    optimizer.step()\n",
        "\n",
        "    return True\n",
        "\n",
        "\n",
        "def imshow(image, ax=None, title=None, normalize=True):\n",
        "    \"\"\"Imshow for Tensor.\"\"\"\n",
        "    if ax is None:\n",
        "        fig, ax = plt.subplots()\n",
        "    image = image.numpy().transpose((1, 2, 0))\n",
        "\n",
        "    if normalize:\n",
        "        mean = np.array([0.485, 0.456, 0.406])\n",
        "        std = np.array([0.229, 0.224, 0.225])\n",
        "        image = std * image + mean\n",
        "        image = np.clip(image, 0, 1)\n",
        "\n",
        "    ax.imshow(image)\n",
        "    ax.spines['top'].set_visible(False)\n",
        "    ax.spines['right'].set_visible(False)\n",
        "    ax.spines['left'].set_visible(False)\n",
        "    ax.spines['bottom'].set_visible(False)\n",
        "    ax.tick_params(axis='both', length=0)\n",
        "    ax.set_xticklabels('')\n",
        "    ax.set_yticklabels('')\n",
        "\n",
        "    return ax\n",
        "\n",
        "\n",
        "def view_recon(img, recon):\n",
        "    ''' Function for displaying an image (as a PyTorch Tensor) and its\n",
        "        reconstruction also a PyTorch Tensor\n",
        "    '''\n",
        "\n",
        "    fig, axes = plt.subplots(ncols=2, sharex=True, sharey=True)\n",
        "    axes[0].imshow(img.numpy().squeeze())\n",
        "    axes[1].imshow(recon.data.numpy().squeeze())\n",
        "    for ax in axes:\n",
        "        ax.axis('off')\n",
        "        ax.set_adjustable('box-forced')\n",
        "\n",
        "def view_classify(img, ps, version=\"MNIST\"):\n",
        "    ''' Function for viewing an image and it's predicted classes.\n",
        "    '''\n",
        "    ps = ps.data.numpy().squeeze()\n",
        "\n",
        "    fig, (ax1, ax2) = plt.subplots(figsize=(6,9), ncols=2)\n",
        "    ax1.imshow(img.resize_(1, 28, 28).numpy().squeeze())\n",
        "    ax1.axis('off')\n",
        "    ax2.barh(np.arange(10), ps)\n",
        "    ax2.set_aspect(0.1)\n",
        "    ax2.set_yticks(np.arange(10))\n",
        "    if version == \"MNIST\":\n",
        "        ax2.set_yticklabels(np.arange(10))\n",
        "    elif version == \"Fashion\":\n",
        "        ax2.set_yticklabels(['T-shirt/top',\n",
        "                            'Trouser',\n",
        "                            'Pullover',\n",
        "                            'Dress',\n",
        "                            'Coat',\n",
        "                            'Sandal',\n",
        "                            'Shirt',\n",
        "                            'Sneaker',\n",
        "                            'Bag',\n",
        "                            'Ankle Boot'], size='small');\n",
        "    ax2.set_title('Class Probability')\n",
        "    ax2.set_xlim(0, 1.1)\n",
        "\n",
        "    plt.tight_layout()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "uugOn1IkC_pn",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 122
        },
        "outputId": "0065f1b3-88ab-4240-89ca-549510bea280"
      },
      "cell_type": "code",
      "source": [
        "import torch\n",
        "from torchvision import datasets, transforms\n",
        "\n",
        "# Define a transform to normalize the data\n",
        "transform = transforms.Compose([transforms.ToTensor(),\n",
        "                                transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])\n",
        "# Download and load the training data\n",
        "trainset = datasets.FashionMNIST('~/.pytorch/F_MNIST_data/', download=True, train=True, transform=transform)\n",
        "trainloader = torch.utils.data.DataLoader(trainset, batch_size=64, shuffle=True)\n",
        "\n",
        "# Download and load the test data\n",
        "testset = datasets.FashionMNIST('~/.pytorch/F_MNIST_data/', download=True, train=False, transform=transform)\n",
        "testloader = torch.utils.data.DataLoader(testset, batch_size=64, shuffle=True)"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-images-idx3-ubyte.gz\n",
            "Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-labels-idx1-ubyte.gz\n",
            "Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-images-idx3-ubyte.gz\n",
            "Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-labels-idx1-ubyte.gz\n",
            "Processing...\n",
            "Done!\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "i4BHeTdjC_pt",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Here we can see one of the images."
      ]
    },
    {
      "metadata": {
        "id": "8zRppFeWC_pu",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 330
        },
        "outputId": "babda3b5-ee37-480c-baa3-2f0b2b9bfe0d"
      },
      "cell_type": "code",
      "source": [
        "image, label = next(iter(trainloader))\n",
        "imshow(image[0,:]);"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAATkAAAE5CAYAAADr4VfxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAADixJREFUeJzt3V2MXOdZB/AzszOzH/Y63virpDGS\njdukTZFTropqKcWqUooESBQkQFEFoiCQGikXjdqKUkopaqv0IlIqgaBVUBUBEuKChl4RjAQWvWgr\n4SKSixg7ddwmsRM71o7X3t354IabnZ0zPCe7c7z79Pe7y/s+ec+Zndm/z86ZZ97GcDgcFgBJNe/0\nCQBMk5ADUhNyQGpCDkhNyAGpCTkgtdakydOnTo4d//o3/qH4nY9+ZCondKfsxMe0sLAQrv3AQw9t\nGvut332s+Ou/enLD2NWrV8Nr7t2zN1zbaMTqlrvd8Jqrq6tjxz/5R18svvynnx45fuwEVlZWwsd/\n8fz5cO1W7cTX31bV+ZjOnD1XOveWruSOHT/xlk9mp8r4mA4eftudPoWp+Il7jt7pU9h2GV9/O+Ux\n+XMVSE3IAakJOSC1xqTe1YsXzu+Yv6sB3oqJIVd2d/XM2XOlc7vVTnxMW727+ok//FLxlT/71Iax\nDHdXn/zzZ4rH/uCRkePv7rurO/H1t1V1PqZtv7sKsFsIOSA1IQekJuSA1Ca2dXFn/dTx4+Hao2+/\nNzS+uHcxvOaBAwfCtXNzs6G6G2/eCK/Z6/VK59578sEN/71yK3ZDYW+Fx//qq6+Ga6vcUCm7STI6\n7ku7t4crOSA1IQekJuSA1IQckJqQA1ITckBqQg5ITcgBqQk5IDUdDzvYO9/xznBt2Sf+R8erfIr+\nhz+8HK7df9f+UN31N6+H15x0qteub1wn/FVLt66Ej3/gwMFwbZWOB+rlSg5ITcgBqQk5IDUhB6Qm\n5IDUhByQmpADUhNyQGpCDkhNyAGpaeuqWbT9qCiKotWaCdeura2HxldWbobXrLKNyuJibIOY7s34\n8ZvN8p/V6MY5/f4gtGbvdvnmOKOWlmKtakVRFC/9IFxa2lpn45rpcCUHpCbkgNSEHJCakANSE3JA\nakIOSE3IAakJOSA1IQekJuSA1LR11extR46Ea2dnZ///ov9z8+b43bp6vY1tTI1G/N+14TDWKlUU\nRXHt2rVQ3cxMvFWtyvHb7Xaort/vb/ua7Gyu5IDUhByQmpADUhNyQGpCDkhNyAGpCTkgNSEHpCbk\ngNR0PNRs7969U1l3MBjfHTA6PmlzmE3/by++scrt1dVQ3d69e+Jr3i5fc7RzI7o/UJWOi6WlpXDt\nTDN+vdAvea6YDldyQGpCDkhNyAGpCTkgNSEHpNYYDoelt9AuXjhfHDt+os7zAdhWE0Pu9KmTY8fP\nnD1XOrdb1fWY3nEi/o/Gz77vfeHa5eXuprFHH/+T4qkn/njD2GAQ/9LI0S/cnGR2di5Utx0fIfn9\nxz5T/MWTX9gwFv1ozNraevj4N1duhmufe+65cO24j5D4ndr6scr4cxVITcgBqQk5IDVtXTXbv39/\nuHbC26WblLU1jY5XWTNeWRRzc7H35Ko8/jfeeKN0rt3e+NJdX4+919bvx99nbLfivx6NCm1dhbau\nWrmSA1ITckBqQg5ITcgBqQk5IDUhB6Qm5IDUhByQmpADUhNyQGraumrWbrensm5Zt9bo+NraWnjN\nffvuCte+duW1UN38fKz9qyiKotPphOd6veBXSEW39SqKotmM7+zVbLhe2Kk8M0BqQg5ITcgBqQk5\nIDUhB6Qm5IDUhByQmpADUhNyQGo6HmpWZSOZfvRT/BUMKhy/SnfC8y+8EKo7fOhQeM19i/tK5zrt\njR0PN4croTWrdCYMGvENZ4aVtv2hTq7kgNSEHJCakANSE3JAakIOSE3IAakJOSA1IQekJuSA1IQc\nkJq2rprtWVgI1zZnKrQgDce3II2ONyts5DI/Nx+ufe3VV0N1ly69HF7zwQdPls612htfujPBn1Wv\nFz580WrFfz2aTdcLO5VnBkhNyAGpCTkgNSEHpCbkgNSEHJCakANSE3JAakIOSE3IAalp66pZpzMb\nrq2ys9dwML52dLzT6YytG2e9tx6uje4Cdu369fCarZnyl+foXKOItas1KuzWFW0VKwptXTuZZwZI\nTcgBqQk5IDUhB6TWGE54d/vihfPFseMn6jwfgG01MeROnxr/pYVnzp4rndut6npMv/DzHw7XLi3t\nD9cuL3c3jT32qS8UT37pMyOj8Tu2hw4dCtd+/emnQ3Xvuv9d4TUf/uAHx47/8q9/rPjHv/vahrGr\nV6+G1lxdWwsfv8rd1We/9a1w7a1btzaN+Z3a+rHK+HMVSE3IAakJOSA1HQ81q9TFEC8Nm5nQRTDq\nlVde2fbj/+iVH4VrJ3VcjM41dBxQwisDSE3IAakJOSA1IQekJuSA1IQckJqQA1ITckBqQg5ITcgB\nqWnrqlmjEdtwpXptbLzVmgmveenlK+HaqBs3boRr9ywshOe63c1fNTVOlba6dju+6Y+NbHYuzwyQ\nmpADUhNyQGpCDkhNyAGpCTkgNSEHpCbkgNSEHJCakANS09ZVs0k7UI1qNuNtXUVRVjs6Hl/z8uXL\nFY6//S6+9FJ4bmlpKbRmt3szfPzuzVirWFFUa8GjXq7kgNSEHJCakANSE3JAakIOSE3IAakJOSA1\nIQekJuSA1HQ81Oz2rdtTWbesO2J0vFeh4+K1K9u/kU0V3/ne98JzH374Q6E1Z2bi/673B/1wbavl\nV2mnciUHpCbkgNSEHJCakANSE3JAakIOSE3IAakJOSA1IQekJuSA1PSi1GymNXNHj7+4uHhHj19F\nd3k5PDccDkNrVtlwpkptp9MJ11IvV3JAakIOSE3IAakJOSC1xnDCO7YXL5wvjh0/Uef5AGyriSF3\n+tTJseNnzp4rndut6npMH3jooXDt0XvvDdd2u91NY48+/vniqSc+u2Gsyt3VbzzzTLh2Gg7cfffY\n8b//5r8Wv/ZLP7dh7Fd/5SOhNa+/eT18/LX1+BeM/se3vx2uvTLmy0j9Tm39WGX8uQqkJuSA1IQc\nkJqQA1LT1lWzKrs6VWkrupNrTsvCwkJ4rhnchavZjP+73mzEa1dXV8O11MuVHJCakANSE3JAakIO\nSE3IAakJOSA1IQekJuSA1IQckJqQA1LT1lWz5Qk7UI0aHDkSrm212qHx3dR+NBgMwnP9fj+05mxn\nNnz84AZglY5P/VzJAakJOSA1IQekJuSA1IQckJqQA1ITckBqQg5ITcgBqel4qNnsbPwT99PYdObW\n7dvbvua0TNp0ZnSu1+uF1lxfXw8fv9MZ30UyztL+/eHabrcbrmXrXMkBqQk5IDUhB6Qm5IDUhByQ\nmpADUhNyQGpCDkhNyAGpCTkgNW1dNXv7PfdMZd3BYPxGKqPjnXa8VanT6YRr19bWwrVR8/Pz4blW\nK/ZS7vVj7V9FURSNCW1lo+67775w7cuXL4dr2TpXckBqQg5ITcgBqQk5IDUhB6Qm5IDUhByQmpAD\nUhNyQGpCDkhNW1fN2hXaqgaDQbh2OIyNt1vx49+1b1+49urrr4droyb9rEbnojubtdvxVrXbFXY2\nW5hfCNdSL1dyQGpCDkhNyAGpCTkgtcZwWPaWdVFcvHC+OHb8RJ3nA7CtJobc6VMnx46fOXuudG63\nqusxffSRR8K1Ve6urqzc2jT28U98rvjqVz63YWxudja85j//y3Ph2mncXX3g3e8eO/7UX/5t8ejv\n/caGsVPvf39ozW73Zvj4Ve6urq+vh2u/+U/PbhrzO7X1Y5Xx5yqQmpADUhNyQGo6Hmo2MxP/ka+v\nb36frUzZJ/5HxxvNWGdAURTF3NxcuHYayt8t3jzXKGKPa1jhfc6imHACI+5eWqqwLnVyJQekJuSA\n1IQckJqQA1ITckBqQg5ITcgBqQk5IDUhB6Qm5IDUtHVtk+hGKvPz8Vap27ertHXFxoeDeKtS9DHt\nBINhrF1rdW0tvGarVeHXY/f8qH7suJIDUhNyQGpCDkhNyAGpCTkgNSEHpCbkgNSEHJCakANSE3JA\natq6tsmhgwdDdVVahaI7UBVF+S5go+OdTie85p02P2G3sNG5meZMaM2Zmfi/6ysr8ba6ZtP1wk7l\nmQFSE3JAakIOSE3IAakJOSA1IQekJuSA1IQckJqQA1LT8bBNFhcXQ3WDfmzDlaIoikYz3vHQLul4\naLc3jjcrfOJ/bkLHQR06s+XdGZvmgj+q2dnZ8PH7FZ6rKo7ee29o/OXLl6dy/B83ruSA1IQckJqQ\nA1ITckBqQg5ITcgBqQk5IDUhB6Qm5IDUhByQmraubXLXvrtCdcvLy1M5/mAwvgVpdLzX64XXnJub\n39I5bdW+ffvCc9HH1WjEW+X27FkI166srIRryza9sRnOdPipAqkJOSA1IQekJuSA1BrD4XBYNnnx\nwvni2PETdZ4PwLaaGHKnT50cO37m7LnSud1qq4/pZx58b6jup9/znvCavX78Tui4p/FjH/908bWv\nfnHDWKsVv6H+n+e+H679/n/Fa6M+9PDDY8c/+dknii9//vENYz959GhozdXV1S2f1zhV7q5+57vf\n3TT29N88W/z2b/7ihrEfXLq05fO6k+rMiTNnz5XO+XMVSE3IAakJOSA1IQekpq1rm9x//32husOH\nD4XXvPr66+Hasram0Vah+fl4q9bS/v3h2mk4eOBgeG7Pwp7QmlXauvr9frj2yOEj4doHHnggNL7b\nbzzsFK7kgNSEHJCakANSE3JAakIOSE3IAakJOSA1IQekJuSA1HQ8bJN/+/ezobrDh+IdD7Nzs+Ha\nsq8Qev6FFzauORtf88UXXwzXTsOFC/8Tnut2YxsEXbl6NXz89fX1qdS+eePG2PH/fv758BrEuZID\nUhNyQGpCDkhNyAGpCTkgNSEHpCbkgNSEHJCakANSE3JAao3huK3XAZJwJQekJuSA1IQckJqQA1IT\nckBqQg5I7X8BojsaD6sYCuQAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7fb93f1f2a58>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "S6lFLAozC_py",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Building the network\n",
        "\n",
        "Here you should define your network. As with MNIST, each image is 28x28 which is a total of 784 pixels, and there are 10 classes. You should include at least one hidden layer. We suggest you use ReLU activations for the layers and to return the logits or log-softmax from the forward pass. It's up to you how many layers you add and the size of those layers."
      ]
    },
    {
      "metadata": {
        "id": "QJ6opWgfC_pz",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# network architecture\n",
        "model = nn.Sequential(nn.Linear(784, 128),\n",
        "                      nn.ReLU(),\n",
        "                      nn.Linear(128, 64),\n",
        "                      nn.ReLU(),\n",
        "                      nn.Linear(64, 10),\n",
        "                      nn.LogSoftmax(dim=1))\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "1of9BcNEC_p3",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Train the network\n",
        "\n",
        "Now you should create your network and train it. First you'll want to define [the criterion](http://pytorch.org/docs/master/nn.html#loss-functions) ( something like `nn.CrossEntropyLoss`) and [the optimizer](http://pytorch.org/docs/master/optim.html) (typically `optim.SGD` or `optim.Adam`).\n",
        "\n",
        "Then write the training code. Remember the training pass is a fairly straightforward process:\n",
        "\n",
        "* Make a forward pass through the network to get the logits \n",
        "* Use the logits to calculate the loss\n",
        "* Perform a backward pass through the network with `loss.backward()` to calculate the gradients\n",
        "* Take a step with the optimizer to update the weights\n",
        "\n",
        "By adjusting the hyperparameters (hidden units, learning rate, etc), you should be able to get the training loss below 0.4."
      ]
    },
    {
      "metadata": {
        "id": "wyNFHvSLC_p3",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Create the network, define the criterion and optimizer\n",
        "criterion = nn.NLLLoss()\n",
        "optimizer = optim.Adam(model.parameters(), lr=0.003)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "tU6zwmtAC_p7",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 105
        },
        "outputId": "ea95102a-9fbb-4b35-c5ae-5343e7876ce1"
      },
      "cell_type": "code",
      "source": [
        "# Train the network\n",
        "epochs = 5\n",
        "for e in range(epochs):\n",
        "    running_loss = 0\n",
        "    for images, labels in trainloader:\n",
        "        # Flatten MNIST images into a 784 long vector\n",
        "        images = images.view(images.shape[0], -1)\n",
        "    \n",
        "        # Clear the gradients, do this because gradients are accumulated\n",
        "        optimizer.zero_grad()\n",
        "\n",
        "        # Forward pass, then backward pass, then update weights\n",
        "        output = model.forward(images)\n",
        "        loss = criterion(output, labels)\n",
        "        loss.backward()\n",
        "        optimizer.step()\n",
        "        \n",
        "        running_loss += loss.item()\n",
        "    else:\n",
        "        print(f\"Training loss: {running_loss/len(trainloader)}\")"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Training loss: 0.49647286418340864\n",
            "Training loss: 0.3888452152834772\n",
            "Training loss: 0.3532598616281298\n",
            "Training loss: 0.3316112327883874\n",
            "Training loss: 0.3128692433794043\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "S8vs43-1C_p-",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 227
        },
        "outputId": "d55902c0-64fd-4f07-e95e-363e844e4980"
      },
      "cell_type": "code",
      "source": [
        "%matplotlib inline\n",
        "%config InlineBackend.figure_format = 'retina'\n",
        "\n",
        "import torch.nn.functional as F\n",
        "\n",
        "# Test out your network!\n",
        "\n",
        "dataiter = iter(testloader)\n",
        "images, labels = dataiter.next()\n",
        "img = images[0]\n",
        "# Convert 2D image to 1D vector\n",
        "img = img.resize_(1, 784)\n",
        "\n",
        "# Calculate the class probabilities (softmax) for img\n",
        "with torch.no_grad():\n",
        "    logits = model.forward(img)\n",
        "ps = F.softmax(logits, dim=1)\n",
        "\n",
        "# Plot the image and probabilities\n",
        "view_classify(img.resize_(1, 28, 28), ps, version='Fashion')"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1EAAAGkCAYAAADQRa8pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAWJQAAFiUBSVIk8AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xu8pmO9+PHPasbMYMY4DaYiYvct\nleSQQ05JItsm5Udoo2hvlRCFlEY6IDo4dJDDFAlJ2TpQDDbb+ZCQvhmScV6YYWYYw8z6/XHfD4+n\ntWata2ad5/N+vdbrWc99f+/rvp5rhrW+872u627r6OhAkiRJktQzrxvoDkiSJEnSUGISJUmSJEkF\nTKIkSZIkqYBJlCRJkiQVMImSJEmSpAImUZIkSZJUwCRKkiRJkgqYREmSJElSAZMoSZIkSSpgEiVJ\nkiRJBUyiJEmSJKmASZQkSZIkFTCJkiRJkqQCIwe6A5IkSYNNRGwCfBzYEng9MAZ4DHgI+DVwbmY+\n3XLN6sA/ADKzrT/725siYjKwdxenXwaeBm4Dzs7Mi/q5T8dk5qQ+vteDwJuA92Xm1T28ZhLwVeCn\nmblPfWx1Ovn7EBFXU/292jczJ/dSt9XPTKIkSZJqETEWOAvYtT70F+BSYCbwBmBrql+Aj42IvTPz\n1wPS0f5xFzCl5diyQAAfAj4UEb8EPpaZ8/q7c4PMjcD3gZt7EHsR8Gfgr80HI+JbwBFDOQFfnJhE\nSZIkARExGrgSeA/VL7j7ZubNLTHLA5OAA4FfRcRumfnL/u5rP7k+Mw/u7EREbAn8lirZvA04vj87\nNthk5mXAZT2MPbWLUxv2Xo/U11wTJUmSVPkaVQL1D2DL1gQKIDOfyczPAd8D2oCTI2Jc/3Zz4GXm\nNcAx9dsDBrIvw0FEtAEbDHQ/1HNWoiRJ0mIvIpYDPlO/PTgzn+rmkq9QTfH7ZWbO7EH7KwGHAtsD\nawKjgHbg/4ATMvOWTq5ZF/gisBmwMjAHeISqAvS9zHy0JX4r4CBgI2AC8BwwDbgYODkzZ3TXz0JX\n169viohlG+1HREd9fGXgKGAP4JnMjKa+rkg1HjsCqwMjgEepKoEnZubfu7ppRLwR+DrwAWBF4Eng\nf4CjW9ep1fE7AftTJSkrAC8C9wEXAN/NzBcXcK/3Al+mSq6XBh4Azq6ve7kpbhIta6IW0ObVNK2J\nal2D1jR+/w38CJgPrJaZj3TR3n3AWsBemfnzBd1bvcdKlCRJUrXGZ2mqX+R/211wZs7KzKMz867u\nYiPi9cDtVAnRMlS/vJ9FtUnFR4HrI2Lblms2p1pnszvVL+6TgV9R/e72BeDGiFitKX53qvVLO1Ct\n4zoD+B2wPFXF6JqIWKa7vhZqTsrGdHL+M8CewCXAb5r6+m/AHcARwNi6nxcAs6mSnTsiYusu7rkC\ncBPV1LffAucDSwKfpvqMSzUHR8TR9b0/WN/zJ/V1qwPfAi6LiK5+H94CuAoYXffv91QJ8AnA6V1c\nszD+SJWYNXy//roSuI7qz/w/O7swItanSqCeo0qW1U+sREmSJMEm9eu1mTm/l9s+iGpTiluALTJz\nTuNERBxLVek4CXhn0zVHUf3y/t+Z+eOm+DaqX7APBA4GPl+fmkQ1vXDHzLy8KX4JqgTgw8A+wMm9\n+LnWql9nA51V7j4OrJ+Z/2w5/lPgjcCFwJ4tFZ1vAF8CzomIN3dSJfovqurMQZnZUV+zMtW6rLdT\nJW7fro+PpxpHgN0y85Uko77mHmArqrH5VSf9PwrYo3kHwohoJFb7RsQPO6sglsrM8yLiemDf+v0r\n69Ai4kyqSuQ+VElfq93r1wsy84VF7Yt6zkqUJElSleRAvSV1L/sD8Fng0OYEqtZIkN4REcs2HV+j\nfr2+ObhOHI4CNgWO60H8S1TTwt4DnLOwH6ALjemPv2tOhJpc2ZpA1ZWTTYCXgM90ct0kqmmOrwf+\nvZM2ZwOHNxIogMx8Ajilfvv/mmJHAp8CDqPalp6WaxrVsc07+3DA1a1buGfm/1JVjlrv1Vd+STVt\n9C311MJX1Al1ow9nt16ovmUlSpIkqZpWBtUv6b2qftbQ1V2cbl7nMp5Xp8gl8BbglIjYPzPva2pv\nJnBD622oKllnRsQhzetnMvNJqnVDiywiRlFVoA6iSnKeo6qkdeaqTo5tUb/e3Nm6s8x8KSKuoZrm\nuAn/WiG6uouKSyN5fGdEjMjMefX6qJ8u4OM0xmh8F+d/38Xx64HtgHUX0HavyMzZEXE+1TTHfanW\n0DVsDKxWhWXr3wf1MZMoSZKkKhmAas1Sr6u3BP80sD7Vpg+d3af5+UCH1LFbAn+PiL9QrZG5DLiq\nrjA12w/4E9WW4x+JiFua4v9vIaco/ldE/NcCzt8LfLw5wWvRWeLWqJgtqOL3YP26WifnHujimkZC\nNBpYjnp6Yb0O7HNUCd+bqDaiaP39t6vnMnV3r4ldnO9tZ1IlUf8vIj6Xmc/XxxtT+Sb3Uz/UxCRK\nkiQJGtPO3tbbDUfEZ6nWIrVR/QL+B6pf8htT2Q5qvSYz74+IdaiSqY8B69RfhwDtEXFsZp7SFH9z\nRLyDatOJj1Dt0LcR1fqif0bE4Zl5QWHXO3vY7svA01SbXlzTTXI2q5NjjY0fFrR+pzHlcclOznVV\nKWyeJjkGXnmm1/VUDwd+maqK83uq6XFQVXI2WkA/urtXZ5tp9LrMvCki7qFa8/VR4Gf1Zhi7AvOA\nn/VHP/RaJlGSJEnVL9uHAJtHxJI9WaQfEaMyc243MROoNo1oo9oC+9jm9Tz1L8P/kkQB1NPRvgx8\nOSLWArYFdqOaEndyPW3te03x06iqLp+rE6rtqLYXfzdwfkS8mJm/ab3PAnT5sN1F0EhMllpATONc\nZ0lYV4lL8/FGpeYrVAnUP6me+9W6PutrLDiJ6u5ez3dxvi+cCXwH2IsqadqcqhJ2WetW9+ofbiwh\nSZJUVYdmUK2P2ae74IgYAdwUEedFxIKmdW1C9UyoZ4FvNCdQtTf3pHOZOTUzf5CZW1LtUAfVZhVd\nxd+dmSdm5nq8uqtbl/H9qDFFbkGfu3Gusyl/nU3xA1i1fp0NTK+/37J+PaWTHQK760NP7vVwN9f3\npnOAucD76meO7Vkfd0OJAWISJUmSFnuZORs4sX77zYh4SzeXfJNqY4HN6Lxi0tCY9fNcZs7r5PwB\nTd+3QfXg34jYNSK266LN8+rXN9TxEyNir4jYtCfxA+zq+nXDepvx14iIMVTbjjfHNntfvW17q83q\n1zuaEtXG2E9vDY6IVYCd67ddrYn6YBfHG/e6rYvzi6Tede816k04LqH6THtRTeubXh/TADCJkiRJ\nqhxHtQZoWeB/I2Kn1oCImBARP6Z6cO5sqo0VZrbGNbm3fn1jRLxmN7eI+G+qKXft9aFGhWNlqmc7\n/Swi3tpJm7vVr3+uX9ehqlSc3UVVrDV+wGTmnVRjPJJq58ERjXN18nAC1fjfA1zRSRPLAsc2H6gf\nZtyosp3XdKox9ju0xL8R+B+qByDDq+PearuI+FDLte8H3g/Mp3rQb29p/jv0pi5izqxfv0q1ecYv\nOnmOlvqJa6IkSZKAzJwXETsAPwT2Bn4TEf+g2k58BlUlZxtgaWAasHNm3t5Ve3Wb90bEpcCOVInZ\nJcCLwHuBVer2vgZ8CDgjIq7MzAMi4ltUm0L8JSKuppra9jqqjS82pUrgDq3vcXlE/JxqitfUiJhC\nNdVsFNV6qHdTJWpHL/oo9YpPANdQbYywXkTcQFUN2oBqDdOTwMe62LTiFODjEbE91Z/LUlRJ0vLA\nzcAZTbHfpqrY7BIRN1ElTa8HPgBcVLd1M7BVRJwHnJeZv226fhLw64j4X+Dv9bU71H39Xmbes4jj\n8IrMfDoiHgRWp/p7cjdwSfODlql2X5zGq0nf5N66v8pZiZIkSapl5pzM3BfYEDiVahe5Hage2rop\n1S/unwaiuwSqyX9SPVR3NlVV6ANUu9u9JzNvA46k2gnvTdTreDLzKGAXqi3K167b+DiwEnA68O7M\nbH6w7sepkpMbqJKRT1BtgT2KakOCdy1gK/J+Va9PWp+q8vci1W6CuwAdVJtwvCsz7+ri8ieoNoO4\nE9iJ6jPOovqMH2je+j0zb6WasncTVbVuD6rNGA4G9s7MW4DvUq1X24FqbJtN4dWq08eA7amex3Ug\n1QN8e9t/An+lqkSuR/VA4lfUSWVjDdQ9df81QNo6OlrXN0qSJEkabCLiBKpt7D+bmacNdH8WZyZR\nkiRJ0iAXEStQTeucB7yx3gxFA8TpfJIkSdIgFhFLAj8HxgEnmUANPDeWkCRJkgahiPhPqi3VP0C1\n6cT/AccPZJ9UsRIlSZIkDU6bAPsDY4HTgO2bN8/QwHFNlCRJkiQVsBIlSZIkSQVMoiRJkiSpwHDb\nWMK5iZJUpm2gOyBJ0lBjJUqSJEmSCphESZIkSVIBkyhJkiRJKjDc1kRJkrS462hvnznQfRj0JkwY\nB4Bj1TOOVxnHq8xAjteECeMWam2wlShJkiRJKmASJUmSJEkFTKIkSZIkqYBJlCRJkiQVMImSJEmS\npAImUZIkSZJUwCRKkiRJkgqYREmSJElSAZMoSZIkSSpgEiVJkiRJBUyiJEmSJKmASZQkSZIkFTCJ\nkiRJkqQCIwe6A5IkqffseOglA92FImcdsfVAd0GSilmJkiRJkqQCJlGSJEmSVMAkSpIkSZIKmERJ\nkiRJUgGTKEmSJEkqYBIlSZIkSQXc4lyS1Oci4mjgGODCzNxtIduYBHwVWDIz53RyfnXgH8ABmfmj\nRejrVsBVLYefA/4JnAt8NzNfWtj2JUlDn5UoSVKfiog2YF/gL8BOEbH8AHepp/YAJtZfGwHnAN8C\nvt0XN4uINSKioy/aliT1LitRkqS+tg2wOrABcB2wJ3DKQHaoh6Zn5uP1948Df4uIrYH/AA7ug/tt\n2gdtSpL6gEmUJKmv7QfckJm3RcTFwCdoSaIi4mpgBtV0ua8BawD3A1/IzD901XBEnATsD7wPeLqT\n82+hqh5tBKwA3AMck5mXLsLneablHm8Djge2AJas+/3DzDylKWY0MAn4GPB6YDrwB+CLmflk01RF\n6mrUTzNzn0XooySpDzmdT5LUZyJiBWBn4Mz60FnAuhHx7k7C30E17W9P4D3AbODciFiqi7YPBj4D\n7JyZt3Vx7/8F3gzsDqwHXAv8JiLeV/g5RkfEHlRVte80HV+pvscKwIfqz3AO8P2I+FxTEz8BPg0c\nDbwN2Icq8ft9Pd3xRODUOnYicFBJ/yRJ/ctKlCSpL30cmAtcUL+fQrX5wyeAA1ti3whsnJlPAUTE\nacBPgbWo1lO9IiI+CpwA7J6ZU7q4937ASsB7M/P++tgh9cYRR/Kvm0e0+nVEzKu/Xwp4mapKdn5T\nzCeB5YGPZuZj9bFvRcSmwOeAkyPiDcBewBGZ+bM65v6I+DxwUd2/6yJiNkDTFMLFwoQJ4xbr+w81\njlcZx6vMUBovK1GSpL70Saod+WYBZGYHcDawRz3FrdnURgJVa69fl2uJ25xq2t+BmXnxAu69EXB/\nUwLVMIWqKtWdTwPr1l/voqoe7QFcERGj6pgN634/1nLt9cCaEbEMsD7QRlUFa42hh32RJA0iVqIk\nSX0iIjammt72joj4RCchO/NqhQpgVsv5xk51bS3HLwKWoJr2tiDLAG+OiNZ2lwBGRcSozJy7gOsf\ny8ypTe/vioibgfuoksMf1vd4tpNrn6tfx9UxdBLXHLPYam+fOSD3bfyL90Ddf6hxvMo4XmUGcrwW\ntvplEiVJ6iufBP4OdPZcqO9STem7oJNz3TkcWBo4PiKmZOb/dhE3HXgA2L6L8y+X3jgzp0bEs8A7\n60MzgFU7CR1fvz5bxzQfa42ZgSRpSHE6nySp10XE0lTJ0y8y88+tX1SbL2wTEZ0lIN2ZTLW5w5XA\nefUGEp25kSrBeS4zpza+qJKnJzJz/kJ8rtdTVY4eqQ/dRDVt7/UtoZsB99bTGG8F5lPt3tcaA3BL\nyz1aK2+SpEHGSpQkqS/sTpVsnN/F+V8DP6JaZ3RsaeOZ2RERewN3Uu34t1MnYWcDhwIXRcSRVInP\ne6h2wTsX+Hw3t1kuIlapv18CeAvwTeAh4PSmexwGXFBvFPEs1e6C2wN71319PCImA0dGxENUidc7\nqKpxV2XmzXVb0+vXnSPir5mZPRkLSVL/sxIlSeoLnwTuzMy/dXYyM6cDfwT2WdjKS72L3T7Aji3b\niTfOP0O1CcWTwG+p1jJ9C/ge8IUe3OI84LH6636qbdpvB7bMzPb6Hk8BW1ElT1cAd1Gt9frPpp34\nAA6gShqPB7L+/jd1bMPP6+svrPspSRqk2jo6OrqPGjqG1YeRpH7g1LFhZsdDLxlSPwvPOmLrAbmv\nC//LOF5lHK8yA7yxxEL9HLQSJUmSJEkFTKIkSZIkqYBJlCRJkiQVMImSJEmSpAImUZIkSZJUwOdE\nSZI0jFx60k7uCCZJfcxKlCRJkiQVMImSJEmSpAImUZIkSZJUwCRKkiRJkgqYREmSJElSAZMoSZIk\nSSpgEiVJkiRJBUyiJEmSJKmASZQkSZIkFTCJkiRJkqQCJlGSJEmSVMAkSpIkSZIKjBzoDkiSNNhF\nxGRg76ZD84F24HbgxMycMhD9kiQNDCtRkiT1TDswsf5aDdi5Pv6niFh/wHolSep3VqIkSeqZ+Zn5\neNP7RyJiX+BxYAfgtoHpliSpv5lESZK06J4BiIiRwNHAXsCqwNPAdcBhmflgIzgidga+BawBTAW+\nCBwEjM7Mrfqz45KkciZRkiQthIhYGfgu8Ahwfn34S8ARwB7ATcAqwA+AXwHr19e9HfglcDmwG7AC\n8D1gOeCB3ujbhAnjeqOZxYJjVcbxKuN4lRlK42USJUlSz6wUEbPq70cAY4AHgd0y86n6+A+ACzIz\n6/fTIuJM4IcRMSEz26kSLICPZ+Z0gIjYC/gLvZRESZL6lkmUJEk98zSwSdP7FYFtqTaWODQzfwjM\nAfaqp+u9ARjFqz9rV6DanGJNYGojgQLIzLsi4qHe6mh7+8zeamrYavyLt2PVM45XGcerzECO18JW\nv0yiJEnqmXmZObXp/VTgxogYBXwnIs4HJgMfBA4HrgKeB3YBjm+6bgWgs98U2vui05Kk3mcSJUnS\normVamrfO4EdgeMz8/uNkxExoiX+RWCpTtpZAZjVyXFJ0iDjc6IkSVo0b61f/wm00VRRqhOoveq3\nbfXrfcC/RcRyTXHrA6v3eU8lSb3CSpQkST3zuohYpen9OOB9wFeA0zLznxFxH7BPRPyJavOJbwDX\nAmsDW0TEw8CFwMHAaRHxdWACcCJVEiZJGgKsREmS1DMTgMeavm4DDqBa/3RQHbMnMBe4GbgIuBj4\nHPB/wMnArpl5A7A/sGndxonA56kqWHP66bNIkhaBlShJkrqRmfsA+/Qg7hZgg05Obdby/hLgp5n5\nErzykN41qJIvSdIgZxIlSVI/ioi3AncB50bECUAHcAjVw3bPHMi+SZJ6xul8kiT1o8z8G/DvwFuA\nm4AbqXb22yEzbx/IvkmSesZKlCRJ/SwzLwcuH+h+SJIWjpUoSZIkSSpgEiVJkiRJBUyiJEmSJKmA\nSZQkSZIkFTCJkiRJkqQCJlGSJEmSVMAkSpIkSZIK+JwoSZKGkR0PvWSguyBpgJx1xNYD3YXFhpUo\nSZIkSSpgEiVJkiRJBUyiJEmSJKmAa6IkoKOjY6C78BptbW192v71119fFD9t2rSi+A022KAoftll\nl+1x7JgxY4raHjmy7H9zo0aNKorv6z8rSZI0+FiJkiRJkqQCJlGSJEmSVMDpfJKkISEiNgYOBjYB\nVgHmAvcCZ2Tm6QPUpw7g+Mw8YiDuL0kaGFaiJEmDXkRsBVwHzAN2BdYCtgJuAn4cEZ8fsM5JkhY7\nVqIkSUPBAcAjwF6Z2dgJZhpwW0QsCaw3YD2TJC12TKIkSUPBksAIYAmqaXyvyMz9Gt9HxIPAb4C/\nA4cBKwP3AJ/NzJub4rYDjgTeSfWz8Drg0My8tyXmy8C7gY66za9n5sVddTIi1gSuB/4EfDwzO7q7\nV0TsA5wNfAg4DXg6MzcsGRxJUv8yiZIkDQV/AHYEro2IE4ArMvPZLmK3A5av45cGzgN+BrwVICK2\nBH4HXEhV4VoKOB64JiLWzsyn6mTof+rr9gXmA/sBF0bEhpl5R+tNI2ICcBlwK7BPnUB1e6+mJr4E\nfAL428IMkCRNmDBuoLuwSIZS/02iJElDwY+AlYDDgYuA+RFxB/BHYHJm/r0pdhlgv8ycCxARPwOO\niYhlMvM54AjgQaqpgfPqmD2AfwKfAr5JNXVwHWBaZs6uY46pr90GeE0SFRFLAZcCTwC7ZubL9ame\n3Kvhgsy8euGHSJLUX0yiJEmDXr0O6piI+B6wPbAl1cYSRwKHR8SBmfmDOvz2RgJVa69flwOeAzYC\nftVIaur2n4iIe6jXVmXmnIh4O3BaRLwNGAc0nqy8Qkv3RgC/oEre3puZzzed6/ZeTW7t2WhIUufa\n22cOdBcWSqMCNRD9X9jql0mUJGnIqKfwnV9/ERHrAecC342Ii+qwWS2XNTaiaCRBywB7R8THWuLG\nAHPqdj9MVfH6JfD/qCpMHcB9nXTrU8BYqmRtVMu5bu/VZEYnbUuSBiG3OJckDXoRMSYixrQez8zb\nqapRo6jXPPXAdKo1Suu2fL0V2L2O2YtqSt/umXldZt4HzO6ivQeAd1BteHFORLQ1nevJvSRJQ4yV\nKA2Ijo6O7oOatLW1dR/UjwZbf2bNav2H9wU79dRT+6gnlbPPPrvP2l5++eWL4sePH18UP3/+/KL4\nqVOnFsVvvPHGfdr+YYcdVhS/4YaDfxO4iJgI/AP4NvCVTkLWqF8f6WGTNwJrZ+ZrBjci3kq1fgmq\npOyZzGz+C7FP/dr6P4DLM/PeiNgTuAr4ItXmET29lyRpiDGJkiQNapn5WEScBhwVEUtTVXYeA8ZT\n7cQ3CTgnM++PiJ40eTxwdUT8gGrDiheAXYBv1K//A9wAfCgidgNuBj5Ctb5pGrBeREzMzMda+nlt\nRHwD+HpEXJ2ZN/XwXpKkIcbpfJKkQS8zD6WaYvcu4GJgKnAtsDNwKNU25D1t6zqq5GsdqkrRXVRJ\n0m6Z2Uhqvg/8nCrxuY3qGU8fB74HvLc+15mvUSVdv4iI8T28lyRpiGkrnVY1yA2rDzOcDbbpfIOt\nP6VKp/N96lOf6qOeVJ555pk+a9vpfAu2ENP5BtdfZi2yHQ+9xJ+F0mLqrCO2HuguLJQB3p1voX4O\nWomSJEmSpAImUZIkSZJUwCRKkiRJkgqYREmSJElSAbc4lyRpGLn0pJ0GZHH2UDOQC9mHIserjOM1\n/FmJkiRJkqQCJlGSJEmSVMAkSpIkSZIKuCZKA6L0YbVD/WG4jz76aFH8I488UhR/4oknFsX/+c9/\nLopfc801i+LHjBlTFF/y5zVjxoyitmfOLJuPPnbs2KL4VVZZpSi+vb29KP7BBx8sih9sf/clSRqO\nrERJkiRJUgGTKEmSJEkqYBIlSZIkSQVMoiRJkiSpgEmUJEmSJBUwiZIkSZKkAiZRkiRJklTAJEqS\nJEmSCviwXUmSOhERGwMHA5sAqwBzgXuBMzLz9DrmQeDqzNxnAe1MBrbLzLInM0uSBi0rUZIktYiI\nrYDrgHnArsBawFbATcCPI+LzBc0dBLyzB/e8JiL2Ke2rJKn/WYmSJOlfHQA8AuyVmR31sWnAbRGx\nJLBeTxvKzGe7i4mIkcAGwNkL0VdJUj8zidKw1NHR0X1Qk7a2tqL4GTNmFMWffvrpRfHbbrttUfyI\nESOK4tdcc82i+JVXXrko/qGHHiqKX3XVVXscO2HChKK2S/+spk+fXhRfOvZz5swpil9hhRWK4tdZ\nZ52ieHVpSWAEsATVNL5XZOZ+rcERsRcwCVgNuA/YPzOvr89Npmk6Xz0F8BJgHLA7cDhwct3U2RFx\ndmaW/U9JktSvTKIkSfpXfwB2BK6NiBOAKxZQUdqIKuHaGRgL/BQ4LyLenJnzu7hmB+B3VNP8Hgfu\nAK6lWoN1waJ2fsKEcYvaxGLDsSrjeJVxvMoMpfEyiZIk6V/9CFiJqkp0ETA/Iu4A/ghMzsy/N8WO\nAz6ZmXMBIuJM4HiqzSge7aL9ccAhjSQrIp6qjz+bmY/39oeRJPUukyhJklrU66COiYjvAdsDW1Jt\nLHEkcHhEHJiZP6jDb28kULX2+nVB/6R6xwKqVIusvX1mXzU9bDT+xdux6hnHq4zjVWYgx2thq18m\nUZIkdaGewnd+/UVErAecC3w3Ii6qw55vuayxKHNB65rKFutJkgYVtziXJKlFRIyJiDGtxzPzdqpq\n1Cjgrf3eMUnSoGASJUlSk4iYSFUpOqqLkDXq10f64PbuyidJQ4DT+SRJapKZj0XEacBREbE0cCHw\nGDAe2I5qK/NzMvP+iOit2zb21t8qIm4H/p6ZL/RW45Kk3mUlSpKkFpl5KLAX8C7gYmAq1RbkOwOH\nAvv28v2eAH4A/D9gCrBib7YvSepdVqIkSepEZp4HnNdNzOqdHJsMTG56v09319THPwN8prSfkqT+\nZyVKkiRJkgqYREmSJElSAafzLUY6Ojq6D1pIbW2L14ZSY8b8y87HCzR27Nii+ClTphTFr7POOkXx\nSy21VFH8Cy+UrW/fYIMNiuJXWGGFHse2t7d3H9Rk5ZVXLoqfM2dOUfyTTz5ZFD9zZtmDBFdcsWxp\nzKhRo4riJUlSOStRkiRJklTAJEqSJEmSCphESZIkSVIBkyhJkiRJKmASJUmSJEkFTKIkSZIkqYBJ\nlCRJkiQVMImSJEmSpAImUZIkDSM7HnrJQHdBkoY9kyhJkiRJKmASJUmSJEkFRg50B3pTR0dHUXxb\nW1sf9WToKx3L0vhSr3td3+b77e3tRfErrbRSUfxHP/rRovgPfvCDRfF33nlnUfzMmTOL4ldZZZWi\n+JEjy/7Xcvfdd/c4ds6cOUVtT5s2rSh+9OjRRfGlfxdGjRpVFD927NiieEmS1PesREmSJElSAZMo\nSZIkSSpgEiVJkiRJBYbVmihJkiJiY+BgYBNgFWAucC9wRmae3k99OB/YODNXL7hmEvBVYMnMLFv8\nJ0nqV1aiJEnDRkRsBVwHzAPb0uZjAAAgAElEQVR2BdYCtgJuAn4cEZ8fsM5JkoYNK1GSpOHkAOAR\nYK/MbGwbOg24LSKWBNYbsJ5JkoYNkyhJ0nCyJDACWIJqGt8rMnO/xvcRMRb4JvARYCXgCeBy4IuZ\n+XQdM4lqWuDmwA+oErCngdMy8/imtt4LnAKsDTwOnNjaqZ7cT5I0dJhESZKGkz8AOwLXRsQJwBWZ\n+WwncScDOwG7A38D1gAmA6dTJToNS1AlSF8DHgAOB46LiGsy88aIWB74LdWaq02B+cBRwGbAiwtx\nv14xYcK43m5y2HKsyjheZRyvMkNpvEyiJEnDyY+oKj2HAxcB8yPiDuCPwOTM/HsddxTw9cx8oH4/\nLSIuBD4bEW1NUwGXAk7MzD8BRMTXgf2B9wA3ArsAywKfzMx765g9gUd5bRLV0/tJkoYAkyhJ0rBR\nJyPHRMT3gO2BLak2ljgSODwiDszMH1BVjA6MiO2pdvAbCYyiqjyNBpp3x7ux6fv2+nW5+vUdwPON\nBKruw9yIuBV4a9N1JfdbZO3tM3uzuWGp8S/ejlXPOF5lHK8yAzleC1v9MomSJA079RS+8+svImI9\n4FzguxHxK6r1SKsBnwdupUpiPgcc2Elzs5q+b1SM2urXccDsTq555TeBiGgrvJ8kaZAziSrQ0VE2\n26I0vlRbW1v3QQsZX9p2X3vxxRe7D2ry5JNPFsVPmjSpKP7oo48uin/ooYeK4r/zne8Uxe+7775F\n8csvv3xR/NSpU4viH3nkkaL4kr9v7e3t3Qc1mT9/flF86X+3I0aMKIofObLsf7tz587tPkiviIgx\nAK3PWcrM2yPiSOA3VBWidwH/nZmTm64dvRC3nE015a/Vsk3fv6MX7ydJGgR8TpQkaViIiInADKr1\nR51Zo35tVJZeycgjYhmq9U3wapWpJ+4Flo6Itze1tSSwQVPMqF68nyRpELASJUkaFjLzsYg4DTgq\nIpYGLgQeA8YD2wGTgHOodsebDnwmIu4Glge+Q1Wl2g94X0Rc3cPbXgx8Gzg1Ig6pj32V104B7M37\nSZIGAStRkqRhIzMPBfaimj53MTAVuBbYGTgU2DczZwN7AK8H7gR+AhxHtfnEX6l29XtvD+/3RN32\nCsDNwCXAFOB3TTG9dj9J0uBgJUqSNKxk5nnAed3EXAZc1smptzd9/yeq6lXzdXNomX6XmVcA6/TF\n/SRJg5OVKEmSJEkqYBIlSZIkSQVMoiRJkiSpgEmUJEmSJBUwiZIkaRi59KSdBroLkjTsmURJkiRJ\nUgGTKEmSJEkq4HOiCrS1tXUftAjxg8kzzzxTFP/ggw8WxT/88MNF8bfffntR/JgxY4ri11hjjaL4\nVVddtSh+rbXWKoofMWJEUfypp55aFL/BBhsUxT/66KNF8aX933zzzXscu+aaaxa1XWrevHlF8bNn\nzy6KnzlzZlH86NGji+IlSVLfsxIlSZIkSQVMoiRJkiSpgEmUJEmSJBUwiZIkSZKkAiZRkiRJklTA\nJEqSJEmSCphESZIkSVIBkyhJkiRJKmASJUmSJEkFRg50ByRJGowiYlvgQGAjYDzwBHAjcHJmXjeQ\nfZMkDSwrUZIktYiIY4HfA/cDOwABfBIYC1wTEZ/qg3v+NCIm9Xa7kqTeN6wqUW1tbX3a/vPPP18U\n/9hjjxXFv/DCC0Xx7e3tRfEPP/xwj2PvvvvuoraXX375ovgNN9ywKH7VVVctil9qqaWK4pdYYomi\n+Hnz5hXFz5w5syh+8803L4rfZpttiuIvv/zyovjSz7vMMssUxZeM/8orr1zU9lNPPVUUX/pnVfp3\n/6WXXiqKL/3/jhZdRGwPfBn4bGae1nTqwYi4ArgQOD4ifpmZ03vx1psC/+jF9iRJfWRYJVGSJPWC\nw4C/Az9oPZGZHRHxX8D8zJwREW11/H7AGsAs4Crg8Myc2rguIvYEDgXWBl4E7gKOysxr6vMddehX\nI+KrwBqZ+WAffT5J0iIyiZIkqRYRI4H3Aj/MzI7OYjLzmaa3xwCHA1+kmv63CnAqMCUi1s7MWRGx\nBXAu8HXgI8Ao4AjgtxERmfkosDrwIHAScCJQNtWgxYQJ4xbl8sWKY1XG8SrjeJUZSuNlEiVJ0qtW\nBEZTJTQLFBGjgIOBMzPz+/Xh+yLiE8CtwM5UydNtwDuAzMyX62uPB/ahSth+SbVpBcCszHy8tz6M\nJKlvmERJkvSqRvWpJ4ts3wqMA65tOX4HMAdYDzg3M2dHxCbA6RGxFrB0U/srLHqX/1V7e9navsVR\n41+8HauecbzKOF5lBnK8Frb65e58kiS96ingeeDfehDb2EHl2eaDmTmfam3UOICIOAT4CXAnsCOw\nLvChXuqvJGkAmERJklTLzHnANcB/1Ouj/kVELBcR+wMz6kPjW86/jiqBapzfC7gxMz+dmTfXG07M\n7ZMPIEnqFyZRkiS91onAG4GvtJ6od+M7FfgOVQXqWWCLlrANqNZV3VK/H8W/bhSxT/3aOm2wb5/V\nIUnqFa6JkiSpSWZOqbcZPyYiVgd+DDxCtYX5F4H3AR/LzGkRcRLwlYi4C7iMape9U4G/AZfUTd4A\n7BER21BtWLE/1T9izgM2iohfAk8DLwCbRMQ6wEOZ2ahkSZIGGStRkiS1yMyvAdsAy1ElQwmcRbWL\n3vqZ+Zs69OvAUcAhdcwFVLvxbZ2ZL9YxXwamABcD1wNLAJ8GTgF2B75db6d+LLAJ1UYVb+vjjyhJ\nWgRWoiRJ6kRmXglc2U1MB/Dt+qurmCeB/+jk1CH1VyPuW8C3FqqzkqR+ZSVKkiRJkgos1pWoq666\nqij+nnvuKYqfMGFCUfwSSyxRFD9ixIii+PHjx3cfVNtjjz2K2n7b28pmnlxxxRVF8c8++2z3QU2W\nWmqpovgnn3yyKH7OnDlF8fPnzy+KHzNmTFH8rFmziuI33XTTovh58+YVxU+fPr0ovq2t52vpS9t+\n6KGHiuInTpxYFF869i+++GL3QU3GjSt7fsXzzz9fFF/634okSbISJUmSJElFTKIkSZIkqYBJlCRJ\nkiQVMImSJEmSpAImUZIkSZJUwCRKkiRJkgqYREmSJElSAZMoSZIkSSpgEiVJkiRJBUyiJEmSJKmA\nSZQkSZIkFRg50B3oTfPmzSuKj4ii+NVWW60o/umnny6Kf+qpp4ri77333qL42bNn9zh27ty5RW1f\ndtllRfH33HNPUfxKK61UFJ+ZRfGlf1YvvvhiUfxLL71UFN/W1lYUX9qf0j/fFVdcsSh+zJgxRfEl\n/+1OmzatqO2JEycWxT/33HNF8aV/VuPGjSuKf+ihh4riH3/88aL4N7/5zUXxkiTJSpQkSZIkFTGJ\nkiRJkqQCJlGSJEmSVGBYrYmSJKk7ETEZ2Lvp0FzgCeAG4MeZOWUg+iVJGjqsREmSFkftwMT66y3A\nPsBM4E8RcdIA9kuSNARYiZIkLY7mZ2bzVob/BKZExJXAeRFxT2aeNUB9kyQNciZRkiTVMvMXEfFJ\n4EjgrIh4ELgEGAfsDuyWmZdGxCrACcDmVNWs+4GTmhOviPgw8CWg8TyNu4CvZuYV9fl1gOOADYGx\nwD+AH2bmKX39OSVJi8YkSpKk17oEODkiGg8H3AH4HfBO4PGIGAVMAcYAB1AlUB8BzoiIlzPzZxHx\nFuBC4Cv16yjg08DvI2LNzJwG/Ba4HtgKmA18ADgtIp7MzAsW5QNMmFD2PLLFmWNVxvEq43iVGUrj\nZRIlSdJrNZ5w3HhS8zjgkMycDxARuwFvA7bOzKvqmOMiYmPgy8DPgHWpfsaenZlP1NcdAvwCmB4R\nKwGrAr/OzMbTx38SEbcCj/Xpp5MkLTKTKEmSXmuJ+vXl+vWORgJV24hqR79rWq67EtgpIsZSVZja\ngWsi4nTgisz8C9UOgETE7Pr7H0bEu4DLgesz847e+ADt7TN7o5lhrfEv3o5VzzheZRyvMgM5Xgtb\n/XJ3PkmSXuvfgA6qzSYAZrScX4Zqet5zETGr8QV8uz4/MTMfBt5DlVgdAtwZEQ/W663IzA5gW+Ak\nqumCVwPtEXFiPV1QkjSIDatK1OzZs4vil1pqqaL4iRMndh/U5E1velNR/IgRI4rit91226L4WbNm\n9Th23rx5RW2/613vKorfc889i+JLPf/880XxL7/8cvdBTWbO7Nt/KZk/f373Qf3Y/siRffu/ipK/\n+6V9WWKJJboParL00ksXxZf+f2TJJZcsip8zZ05R/KhR/v7dCz4K3JaZT0VEZ+enAy9QTdnrzEMA\nmfkg8BngMxHxduBAqnVT/8jMKZk5C/gG8I2ImAjsCRwLPA8c3YufR5LUy6xESZJUi4gDgfWAby4g\n7EZgSWDpzJza+KJKfqZn5tyIWDcitm5ckJn3ZOZ/A88B60bEG+q1VY3zj2XmicAf6To5kyQNEsOq\nEiVJUg+9rt6mHKp/UHwTsC+wH/CtzPz1Aq69FLgb+HlEHAzcB6wNnAL8BdgF2Bg4sU7KGmunPky1\nlfl1wLJUz6NaFziH6kG/6wObseAETpI0CJhESZIWRxN4dRe8DuAZ4CZgu8z844IurCtN7weOB84D\nlgcep9p576t1zI8iYmngi8BpVBtR/BXYNTNvBoiIHal28/sM1c/jB4ETge/22qeUJPWJto6OjoHu\nQ6957rnnij5M6bqQ8ePHF8WXrisqXRNV2n5frol65plniuJL14WUck1U77bvmqiuDfU1UePHj28r\nukBDQYc7gnXP3dPKOF5lHK8yA7w730L9HHRNlCRJkiQVMImSJEmSpAImUZIkSZJUwCRKkiRJkgqY\nREmSJElSAZMoSZIkSSpgEiVJkiRJBYbVw3ZLn9cyffr0oviHH364KH706NFF8W1tZdvUlz7/piS+\n9Fk8q666alF86fPJSsemNL70uVil/S+N7+vnRPW10mcVvfTSS33Uk75/Xltp30ue17Yw7b/wwgtF\n8aXPv5MkSVaiJEmSJKmISZQkSZIkFTCJkiRJkqQCJlGSJEmSVMAkSpIkSZIKmERJkiRJUgGTKEmS\nJEkqYBIlSZIkSQVMoiRJkiSpgEmUJEmSJBUYOdAdkCSpJyJiMrB3y+GXgAeAnwPHZ+bcHrY1Cfgq\nsGRmzomIq4Exmblxr3VYkjRsDaskauTIso8zYcKEPupJZe7cHv0sf8XLL79cFP/SSy8VxZf0Z86c\nOUVtz5s3ryj+da8rK4KWtl86lh0dHUXxpX/X5s+fXxRf2p++VvrnVdr/kvFpa2srars0fsSIEX3a\nfulYlv5dGz16dFH8ENQOrNP0fllgG+B4YG3gYwPRKUnS4mVYJVGSpGFvfmY+3vT+ceBvETEBODoi\nvpCZDw9Q3yRJiwmTKEnScHBn/bpaRJxLy9S8iNgKuArYPjMv666xiBgNTKKqbL0emA78AfhiZj4Z\nEccCXwAmZObMpus2Bm4AdsnMX0fEKsAJwObAROB+4KTMPKvpmg7gSGAzqqrahpl510KNgiSpX5hE\nSZKGg3+rX6f1Uns/AXYCDgT+D3gL8CPg9xGxIdUarC8DOwLnNV23G/AM8LuIGAVMAcYAB1AlUB8B\nzoiIlzPzZ03X7QecAXwWeGxROz9hwrhFbWKx4ViVcbzKOF5lhtJ4mURJkoasiFgCeB9wGHBxZk6L\niEVt8w3AXsARTYnO/RHxeeAi4L2ZeV1E3A7sSp1ERURb/f7CzJwbEbsBbwO2zsyr6naOq6tVXwaa\nk6gZmXncInVcktRvTKIkSUPJShExq+n9GKod+s4BPt9L91gfaAOubTl+ff26HnAdVfL09YgYm5mz\nqKbjvaHuC8BGwFzgmpZ2rgR2aroO4NZe6jsA7e0zuw9azDX+xdux6hnHq4zjVWYgx2thq18mUZKk\noeRpYJOm9y8Bj/V0a/MeWqZ+fbbl+HP1a+Mn7i+o1jv9O3A+1VS++zOzkWwtA4wCnmupjjV+9k4E\n7qu/n9ErPZck9QuTKEnSUDIvM6d2E9NBVUlqNrbgHo2EZnzL8fHN5zPz0fr5UrtGxC+Bj1Ktm2qY\nDrwArNvFfR4q6JMkaRApe2CJJEmD3wyg9UGAJQ/RvRWYD2zRcnyz+vWWpmPnAdsBHwBWBs5tOncj\nsCSwdGZObXwBzwPTe7l6JknqR1aiJEnDzc3AzhGxP9X6oy2AHXp6cWY+HhGTgSMj4iHgJuAdwHeB\nqzLz5qbwXwGnAd8Ebmipkl0K3A38PCIOppq6tzZwCvAXYJeF+3iSpIFmJUqSNNycTLUF+fHAn6nW\nLB1Y2MYBVFPzjgey/v43wM7NQZk5A/gd8G5eW4WirjS9n6pydR7VFuc/AX5NtfufJGmIauvo6Bjo\nPvSmYfVhJKkftK4d0tDX4Y5g3XP3tDKOVxnHq8wA7863UD8HrURJkiRJUgGTKEmSJEkqYBIlSZIk\nSQVMoiRJkiSpgEmUJEmSJBUwiZIkSZKkAiZRkiRJklTAJEqSJEmSCphESZIkSVIBkyhJkiRJKmAS\nJUmSJEkFTKIkSZIkqYBJlCRJkiQVMImSJEmSpAImUZIkSZJUwCRKkiRJkgqYREmSJElSgZED3QFJ\nkkpExGRg727CrsnMrfq+N5KkxZFJlCRpqDkIOKLp/Y+A9YENm47N7dceSZIWKyZRkqQhJTOfBZ5t\nvI+IOcC8zHx84HolSVqcmERJkoaliLgamAHcBRwMHJmZp0bEeOA4YCdgReAJ4GLgS5k5u+naMZm5\ncVN7WwFXAdtn5mURsRzwbWD7up0ngV8Dh2fmC/U1qwAnAJsDE4H7gZMy86ymdjuAI4HNgG2ADTPz\nrj4YEklSLzGJkiQNZ+8E5gAbAI1K1aXAmsCngb/U534ErArsUtD2ycC6wEeBh4G3A6cDSwAHRMQo\nYAowBjiAKoH6CHBGRLycmT9rams/4Azgs8BjxZ+yxYQJ4xa1icWGY1XG8SrjeJUZSuNlEiVJGs5W\nBTbIzOkAEbEJVVVot8z8TR3zQESsCpwYEatm5rQetr0+1QYWN9Tvp0XE+3h159sPA28Dts7Mq+pj\nx0XExsCXgeYkakZmHrcwH1CS1P9MoiRJw9kDjQSq1th84tqWuOvr13cDPU2iLgG+GBGjgf8BrsrM\n+5vOb0S1wcU1LdddCewUEWMzc1Z97NYe3rNH2ttn9mZzw1LjX7wdq55xvMo4XmUGcrwWtvplEiVJ\nGs5mtLxfpn59tuX4c/VryU/TLwF/BT4JXAgQEZcCn8vMR+p7jQKei4jm6xo/eycC93XRT0nSIGYS\nJUlanDSSlfHA803Hx7ec7wDaWq4d2/wmMzuAc4BzImIs8CGqjSZ+AWwBTAdeoFo31ZmHFqL/kqRB\nwCRKkrQ4ual+3QK4oOn4ZsB84Pb6/QzgTS3XNu/UtxTwH8BlmTmjnpZ3YUS8ATimDrsROAxYOjPv\nbLr29cCLmemzrCRpiDKJkiQtNjLzloiYApwUEbOAe4FNgaOAn2ZmY2e8m4GdI2J/qjVMWwA7NDX1\nElXVac+IOJZqR71Vgb14dQ3UpcDdwM8j4mCqqXtrA6dQ7QpYshOgJGkQeV33IZIkDSsfBn4LnAkk\n8HWqxOa/mmJOBn4OHA/8Gfh34MDGycx8CXg/VfXq98BUqml8twD71DFz65hbgPOotjj/CdWzpPbq\no88mSeoHbR0dHQPdh940rD6MJPWD1nU/Gvo63BGse+6eVsbxKuN4lRng3fkW6ueglShJkiRJKmAS\nJUmSJEkFTKIkSZIkqYBJlCRJkiQVMImSJEmSpAImUZIkSZJUwCRKkiRJkgqYREmSJElSAZMoSZIk\nSSpgEiVJkiRJBUyiJEmSJKmASZQkSZIkFTCJkiRJkqQCJlGSJEmSVMAkSpIkSZIKmERJkiRJUgGT\nKEmSJEkqMHKgOyBJWnxExGRg727CrsnMrRay/Q7gmMyctICYq4ExmbnxQt7jImBGZu63MNdLkoY+\nK1GSpP50EDCx6esS4OGWY7v0cR92AT7UXVBE/DMitmo5NgJ4P3B5/f6nETGpD/ooSRrErERJkvpN\nZj4LPNt4HxFzgHmZ+Xg/9uGZ7mIi4g3Aap2c2ghYBriifr8p8I/e650kaSgwiZIkDQkR0QYcCewD\nrArMAq4FDsvMB1piDwUOASYAdwD7Zua99bmraZrOV08BPBLYDNgG+AJwct3UVRHxz8z/397dB9tR\n1gcc/15xeLG8iBLRihDoyC9GYeSlBARqSm1Ax1pfwJaBtrxZqpaiQdG+WIrg1LZBBVOkFhBRqWMV\nokOBdhQCtQKCjJaC/IyFkDg4GjBpSEJQ9PaPZw8uJ+ec3L3n5J5zku9n5s7es8+z++w+89y9+zu7\n+9ucXX0+DrgrM1dXywGcFxHnAftm5vKIOAK4EDiM8n/2u8DfZ+bnq/ZmUwKv04DDgROAHYD/AM7M\nzB/33VmSpC3KIEqSNC5OA/4COIkSGO0J/B1wPTC3Vu8E4CZKQLQX8Dngk8DRPdZ9BnA58KfAD4FH\ngWuAt1ACtZZjq3UDzAaWAxcBi4BVETEXuJkSEL0aeAL4E+BfIuKJzPxybV3nU4K1ecAc4NPVz2s3\n3xW9zZq1S7+r2GbYV83YX83YX82MU38ZREmSxsUhwIpaILIiIt4K7BMRz8rMX1Tz12fmOdXvD0TE\nl4DTN7PuNZn54daHiFhd/fqTzFxVzXsecCiwsCr7UTVd17odMSLOpgROv5eZG6vysyPiGOAsyjNg\nLfdl5qLq92URsRj4q4h4fmY+trnOkCQNj0GUJGlcXA/8cUR8FfgMcHNmrqRcNaq7o+3zKmD7iNgh\nM5/ssu67p9D+AuBx4M4edX6dcrvfxrb536BcIav7z7bP9wATwD5AX0HUqlWP97P4NqH1jbd9NTX2\nVzP2VzPD7K/pXv0yO58kaaRExN4Rsa72cx9AZt4AzAfWABdTrkTdERFHtq1iQ9vn1rNLEz2aXTOF\nTTsW+FpmPtWjzq7UEmfUrAXa/1O3t7mumj53CtsiSRoigyhJ0qh5BHhl7efpdOSZ+fXMPB7YA/hN\nYCNwY0TMROCxgPKsUy9rgN06zN+NTYOm9qCq9Xk1kqSR5u18kqSRUl3p+X77/Ig4FliZmfdXdZZG\nxLspt8HtV00HbaJq+0DgV6neD9WpTuVO4MSI2LF1S1+VVfBI4K625eZTEmO0HAL8FHgQSdJIM4iS\nJI2LU4GDIuIs4AHKrXMLgR9T0ogPUutq0IKIWEPJ9Pe9zFxeq/MkJYnEEVWQtYKSbe9U4Joq7fnP\nKS8YnkPJ/Fd3QES8D7i2Kn8nsKR6l5YkaYR5O58kaVy8DfgacAWwrPp9V2BBZj4x4LbuomTSW0hJ\naf462m7ly8xJ4ALgCEqSiJdlZgK/BexOSXDxLeBA4PWZeUtbGxcB+1KuXn0eWEoJpCRJI25icnJy\n87XGx1a1M5I0A3olW9AWUHvZ7tsz87It0MSkGcE2z+xpzdhfzdhfzQw5O9+0/g96JUqSJEmSGjCI\nkiRJkqQGTCwhSdIMqpJTeBulJI0xr0RJkiRJUgMGUZIkSZLUgEGUJEmSJDVgECVJkiRJDRhESZIk\nSVIDBlGSJEmS1IBBlCRJkiQ1YBAlSZIkSQ0YREmSJElSAwZRkiRJktSAQZQkSZIkNWAQJUmSJEkN\nGERJkiRJUgMGUZIkSZLUgEGUJEmSJDVgECVJkiRJDUxMTk4OexsGaavaGUmaARPD3gBJksaNV6Ik\nSZIkqQGDKEmSJElqwCBKkiRJkhp49rA3YMC8t1+SJEnSFuWVKEmSJElqwCBKkiRJkhowiJIkSZKk\nBgyiJEmSJKkBgyhJkiRJasAgSpIkSZIaMIiSJEmSpAYMoiRJkiSpAYMoSZIkSWrAIEqSJEmSGjCI\nkiRJkqQGnj3sDZAkSUVEPA84D3gj8CLgUeAG4AOZ+cMpLP8q4APA4cBOwPeAfwYWZ+ZkW925wAeB\nVwO7Ag8DnwU+nJk/HdQ+bUkD6K+jquUPA3YEVgJfAi7IzHW1esuBfXqs6qDM/Pb09mJm9NNXETHZ\nqxzYPTPX1Opvs2MrIk4BPrWZ1d+amfOr+ssZ87HVEhHbAxcC7wFua+3jFJcdu2OXQZQkSSMgInYC\nlgJzgMXA3cBLKSckx0TEIZm5usfyxwA3UgKBvwF+AvwucAnwa8C7anVfDnwDeAJYBPwAmF8tdzDl\nxHGkDaC/TqKceCXlZHkt8HrgXODoiDgqM39RW2QV8I4uq3uov73Zsvrtq8r9lH7qZH2trW19bN0C\nnNClbC/go8B9bfPHdmy1REQA1wD7AxMNlx3LY5dBlCRJo+FdwAHAOzPz0tbMiPgOcB3lW9qFPZa/\nFNgIHF37pvwzEbEE+LOI+FRmfqea/xFgZ+CozLy3mve5iFgPnB0Rb8jMrwxsz7aMafdXROwAfIJy\n0jYvM/+vKroyIq6jnIgdR7ny0LIhM7848L2YGf2OLYBVU9z/bXpsZebDlCsjm6j+Fh8D/rqtaJzH\nFhGxO3APsAw4FHig4SrG8tjlM1GSJI2GP6R8o39F2/wvU75tPTkiOn7DGxHzgAC+0OFWo8WUb4ZP\nruq+CPht4ObaSUi9LsAfTHcnZtC0+wt4IXAt8Le1AKqlFTgdOKgNHQH99NWUOba6i4g3Ua6uvC8z\nHxvIVo6O7YGrgcMzM5ssOM7HLoMoSZKGLCJ2pdw6dE9mPlkvq54H+CYwC9i3yyoOq6a3dyi7s5rO\nq6aHUk5MNqmbmd+n3Eozr71slPTbX5n5cGaekpmf6FC8WzVd26P95wwi6JgJAxhb7eubiIhf6VK8\nzY+tLuvcAbi4WvbKzdQdm7HVkpk/ysy3Z+bGaSw+tscugyhJkoav9WD5D7qUr6im+3Upn91t+cx8\nHFhTW7Zr3VpbL4mIUb7lv9/+6qh6MP40YAOwpK14p4i4JCJWU65SbIiIJRExp0kbQzCovtojIq4G\nHgfWRcTaiLg6Il5cqzN7Cm1ti2PrbcBLgHPbkyRUxnVsDcLsajp2xy6DKEmShm+XarqhS/n6tnrT\nWX6XBnV7tTUKBr4PESrDORYAAARpSURBVPEsSjawl1EysD3SVuUFlJO4M4E3AZ+kJKK4IyL2n2o7\nQzCovppbTU+mJE64jnLr1O0RsceA2xqmge5DdRXq/ZRsdbd2qTauY2sQxvbYNcrfBEiSJG1xVTa2\naygJJf4xMz/SVuWPgJ9n5tdr85ZExL2UwOt84MQZ2djheC0lscS3avO+GBErgb8EzgH+fChbNvpO\nAV4MnNWlfFsfW2PLIEqSpOFrPX/T7VmTndvqTWf5tQ3qQrlta1T1219Pi4hZwFco76e5IDPbM6fR\n4wrClcDHgddsrp0h6ruvMvOmLkWXUoKo11CCKMfWps6gZOS7vlPhmI+tQRjbY5e380mSNHwPAZOU\n98h00npOY1mX8ger6SbLR8RulGQJyzZXt9bWQ5n5VK8NHrJ++wuAiNgT+C/KA+undgqgeqneI/Uo\n5YWfo2ogfdXFqmrdrf13bNVExGzK2Pq3zPxZk40Yk7E1CGN77DKIkiRpyDJzPfDfwMERsWO9LCK2\nA14FrMzMFZ2Wp7x8EuDIDmVHV9PW7ULfBJ7qVDciXgE8t1Z3JA2gv1pZ2G4C9gbekJlXdam3X0Sc\nXvVNe9nOlFu1urYzbP32VUQcEBFnRsTeHYpfSsmW1lrWsfVMx1bTmzsVjvvYGpCxPXYZREmSNBqu\nAJ5Debi87mTKg+eXt2ZExJyIeDrFcmZ+m/KyyxMiYq9avQng3cDPgE9XdR+l3L42PyIOamvrnGp6\nOaNv2v1VuRh4JXBiZt7Yo509q3V9tEPq6fdTgohrm2/+jOqnr14BXMamL4iFXz4HdS04tjo4pJr+\nT5fyrWFsNbI1Hbt8JkqSpNFwGXASsCgi9gHuBl4OLATuBRbV6n4XSMr7bFreAdwC3BYRH6OkBv59\n4BhKtrn/rdV9L/AbwL9HxCLgEeC4qv0rMvO2we/ewE27vyLiQMoD/fcD20XE8R3Wvyozb83M2yPi\nKkqCgKUR8QXgScpVhuOrtj408L0brH7G1r9S0r6fXmXhuwHYDngz5Xmdr1ISILRs02OrTSuz3vJO\njWwlY4uImMsvsze2zGr7u7ohMzewFR27vBIlSdIIqJ6ZWEB5mPwtwFWUE/3LgfnVCUiv5e+knFw8\nAHwQ+CfghcBpmXlhW90HKbclLQXOpXzzfjDwHjb99n0k9dlfB1O+5Z9LCRI6/Zxfq39G9bML8A/A\nYsoVmguBIzNzqkkGhqKfvqqeL/kdysnr/sAlwEWUF86+F3hd/RkUx9Yz7F5NeyU6GOuxVXkrz/zb\ngU3/tl7QbeFxPXZNTE52eueXJEmSJKkTr0RJkiRJUgMGUZIkSZLUgEGUJEmSJDVgECVJkiRJDRhE\nSZIkSVIDBlGSJEmS1IBBlCRJkiQ1YBAlSZIkSQ0YREmSJElSAwZRkiRJktSAQZQkSZIkNWAQJUmS\nJEkNGERJkiRJUgMGUZIkSZLUgEGUJEmSJDVgECVJkiRJDRhESZIkSVIDBlGSJEmS1IBBlCRJkiQ1\nYBAlSZIkSQ38P08mXQs1rNnCAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7fb93bac8358>"
            ]
          },
          "metadata": {
            "tags": [],
            "image/png": {
              "width": 424,
              "height": 210
            }
          }
        }
      ]
    }
  ]
}