Autoencoders/Autoencoders-Review.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Autoencoders"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Table of contents:\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "<p><a href=\"#ref1\">- Introduction</a></p>\n",
    "<p><a href=\"#ref2\">- Feature Extraction and Dimensionality Reduction</a></p>\n",
    "<p><a href=\"#ref3\">- Autoencoder Structure</a></p>\n",
    "<p><a href=\"#ref4\">- Performance</a></p>\n",
    "<p><a href=\"#ref5\">- Training: Loss Function</a></p>\n",
    "<p><a href=\"#ref6\">- Code</a></p>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"ref1\"></a>\n",
    "# Introduction\n",
    "An autoencoder, also known as autoassociator or diabolo network, is an artificial neural network employed to recreate the given input.\n",
    "It takes a set of **unlabeled** inputs, encodes them and then tries to extract the most valuable information from them.\n",
    "Autoencoders are used for feature extraction, learning generative models of data, dimensionality reduction and can be used for compression. \n",
    "\n",
    "A 2006 paper named Reducing the Dimensionality of Data with Neural Networks, done by G. E. Hinton and R. R. Salakhutdinov, showed better results than years of refining other types of networks, and was a breakthrough in the field of Neural Networks.\n",
    "\n",
    "Now, autoencoders, based on Restricted Boltzmann Machines, are employed in some of the largest deep learning applications. They are the building blocks of Deep Belief Networks (DBN).\n",
    "\n",
    "<img src=\"https://ibm.box.com/shared/static/xlkv9v7xzxhjww681dq3h1pydxcm4ktp.png\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"ref2\"></a>\n",
    "# Feature Extraction and Dimensionality Reduction\n",
    "\n",
    "An example given by Nikhil Buduma in [KdNuggets](http://www.kdnuggets.com/2015/03/deep-learning-curse-dimensionality-autoencoders.html) can explain the utility of this type of Neural Network with excellence.  \n",
    "Say that you want to extract what feeling the person in a photography is feeling. Using as an example the following 256x256 grayscale picture:\n",
    "\n",
    "<img src=\"https://ibm.box.com/shared/static/r5knpow4bk2farlvxia71e9jp2f2u126.png\"/>\n",
    "\n",
    "This image being 256x256 correspond with an input vector of 65536 dimensions! If we used an image produced with convential cellphone camera, that generates images of 4000 x 3000 pixels, we would have 12 million dimensions to analyse.\n",
    "\n",
    "\n",
    "This bottleneck is further problematized as the difficulty of a machine learning problem is increased as more dimensions are involved. According to a 1982 [study](http://www-personal.umich.edu/~jizhu/jizhu/wuke/Stone-AoS82.pdf) by C.J. Stone, the time to fit a model, at best, is:\n",
    "\n",
    "\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "<center><font size=6><strong>$m^{-p/(2p+d)}$</strong></font></center>\n",
    "<br>\n",
    "Where:\n",
    "<br>\n",
    "m: Number of data points\n",
    "<br>\n",
    "d: Dimensionality of the data\n",
    "<br>\n",
    "p: Parameter that depends on the model\n",
    "</div>\n",
    "\n",
    "As you can see, it increases exponentially!\n",
    "Returning to the example, we don't need to use all of the 65,536 dimensions to classify an emotion. A human identifies emotions according to some specific facial expression, some **key features**, like the shape of the mouth and eyebrows.\n",
    "\n",
    "<img src=\"https://ibm.box.com/shared/static/m8urvuqujkt2vt1ru1fnslzh24pv7hn4.png\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"ref3\"></a>\n",
    "# Autoencoder Structure\n",
    "\n",
    "<img src=\"https://ibm.box.com/shared/static/no7omt2jhqvv7uuls7ihnzikyl9ysnfp.png\" style=\"width: 400px;\"/>\n",
    "\n",
    "An autoencoder can be divided in two parts, the **encoder** and the **decoder**.\n",
    "\n",
    "The encoder needs to compress the representation of an input. In this case we are going to compress the face of the person, that consists of 2000 dimensional data to only 30 dimensions, taking some steps between this compression.\n",
    "\n",
    "The decoder is a reflection of the encoder network. It works to recreate the input, as closely as possible. It has an important role during training, to force the autoencoder to select the most important features in the compressed representation.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"ref4\"></a>\n",
    "# Performance\n",
    "\n",
    "After the training, you can use the encoded data as a reliable dimensionally-reduced data, applying it to any problem that a dimensionality reduction problem seems to fit.\n",
    "\n",
    "<img src=\"https://ibm.box.com/shared/static/yt3xyon4g2jyw1w9qup1mvx7cgh28l64.png\"/>\n",
    "\n",
    "This image was extracted from the Hinton paper comparing the two-dimensional reduction for 500 digits of the MNIST, with PCA on the left and autoencoder on the right. We can see that the autoencoder provided us with a better separation of data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"ref5\"></a>\n",
    "# Training: Loss function\n",
    "\n",
    "An autoencoder uses the Loss function to properly train the network. The Loss function will calculate the differences between our output and the expected results. After that, we can minimize this error doing gradient descent. There are more than one type of Loss functions, it depends on the type of data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Binary Values:\n",
    "$$l(f(x)) = - \\sum_{k} (x_k log(\\hat{x}_k) + (1 - x_k) \\log (1 - \\hat{x}_k) \\ )$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For binary values, we can use an equation based on the sum of Bernoulli's cross-entropy. \n",
    "\n",
    "$x_k$ is one of our inputs and $\\hat{x}_k$ is the respective output.\n",
    "\n",
    "We use this function so that if $x_k$ equals to one, we want to push $\\hat{x}_k$ as close as possible to one. The same if $x_k$ equals to zero.\n",
    "\n",
    "If the value is one, we just need to calculate the first part of the formula, that is, $- x_k log(\\hat{x}_k)$. Which, turns out to just calculate $- log(\\hat{x}_k)$.\n",
    "\n",
    "And if the value is zero, we need to calculate just the second part, $(1 - x_k) \\log (1 - \\hat{x}_k) \\ )$ - which turns out to be $log (1 - \\hat{x}_k) $.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Real Values:\n",
    "$$l(f(x)) = - 1/2\\sum_{k} (\\hat{x}_k- x_k \\ )^2$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As the above function would behave badly with inputs that are not 0 or 1, we can use the sum of squared differences for our Loss function. If you use this loss function, it's necessary that you use a linear activation function for the output layer.\n",
    "\n",
    "As it was with the above example, $x_k$ is one of the inputs and $\\hat{x}_k$ is the respective output, and we want to make the output as similar as possible to the input."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loss Gradient:\n",
    "\n",
    "$$\\nabla_{\\hat{a}(x^{(t)})} \\ l( \\ f(x^{(t)}))  = \\hat{x}^{(t)} - x^{(t)} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use the gradient descent to reach the local minumum of our function $l( \\ f(x^{(t)})$, taking steps towards the negative of the gradient of the function in the current point.\n",
    "\n",
    "Our function is about the preactivation of the output layer $\\nabla_{\\hat{a}(x^{(t)})}$ of the loss $l( \\ f(x^{(t)})$.\n",
    "\n",
    "It's actually a simple formula, it just calculates the difference between the output $\\hat{x}^{(t)}$ and the input $x^{(t)}$.\n",
    "\n",
    "Then our network just backpropagates the gradient $\\nabla_{\\hat{a}(x^{(t)})} \\ l( \\ f(x^{(t)}))$ through the network using **backpropagation**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"ref6\"></a>\n",
    "# Code\n",
    "\n",
    "We are going to use the MNIST dataset for the example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting /tmp/data/train-images-idx3-ubyte.gz\n",
      "Extracting /tmp/data/train-labels-idx1-ubyte.gz\n",
      "Extracting /tmp/data/t10k-images-idx3-ubyte.gz\n",
      "Extracting /tmp/data/t10k-labels-idx1-ubyte.gz\n"
     ]
    }
   ],
   "source": [
    "from __future__ import division, print_function, absolute_import\n",
    "\n",
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "# Import MNIST data.\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "mnist = input_data.read_data_sets(\"/tmp/data\", one_hot=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "learning_rate = 0.01\n",
    "training_epochs = 20\n",
    "batch_size = 256\n",
    "display_step = 1\n",
    "examples_to_show = 10\n",
    "\n",
    "# Network Parameters.\n",
    "n_hidden_1 = 256  # 1st layer num features.\n",
    "n_hidden_2 = 128  # 2nd layer num features.\n",
    "n_input = 784  # MNIST data input (img shape: 28*28).\n",
    "\n",
    "X = tf.placeholder(\"float\", [None, n_input])\n",
    "\n",
    "weights = {\n",
    "    'encoder_h1': tf.Variable(tf.random_normal([n_input, n_hidden_1])),\n",
    "    'encoder_h2': tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2])),\n",
    "    'decoder_h1': tf.Variable(tf.random_normal([n_hidden_2, n_hidden_1])),\n",
    "    'decoder_h2': tf.Variable(tf.random_normal([n_hidden_1, n_input])),\n",
    "}\n",
    "biases = {\n",
    "    'encoder_b1': tf.Variable(tf.random_normal([n_hidden_1])),\n",
    "    'encoder_b2': tf.Variable(tf.random_normal([n_hidden_2])),\n",
    "    'decoder_b1': tf.Variable(tf.random_normal([n_hidden_1])),\n",
    "    'decoder_b2': tf.Variable(tf.random_normal([n_input])),\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we need to create the encoder. For this, we are going to use sigmoidal functions. Sigmoidal functions continue to deliver great results with this type of networks. This is due to having a good derivative that is well-suited to backpropagation. We can create the encoder using the sigmoidal function like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Building the encoder.\n",
    "def encoder(x):\n",
    "    # Encoder first layer with sigmoid activation #1.\n",
    "    layer_1 = tf.nn.sigmoid(tf.add(tf.matmul(x, weights['encoder_h1']),\n",
    "                                   biases['encoder_b1']))\n",
    "    \n",
    "    # Encoder second layer with sigmoid activation #2.\n",
    "    layer_2 = tf.nn.sigmoid(tf.add(tf.matmul(layer_1, weights['encoder_h2']),\n",
    "                                   biases['encoder_b2']))\n",
    "    return layer_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And the decoder:\n",
    "\n",
    "You can see that the `layer_1` in the encoder is the `layer_2` in the decoder and vice-versa."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Building the decoder.\n",
    "def decoder(x):\n",
    "    # Decoder first layer with sigmoid activation #1.\n",
    "    layer_1 = tf.nn.sigmoid(tf.add(tf.matmul(x, weights['decoder_h1']),\n",
    "                                   biases['decoder_b1']))\n",
    "    \n",
    "    # Decoder second layer with sigmoid activation #2.\n",
    "    layer_2 = tf.nn.sigmoid(tf.add(tf.matmul(layer_1, weights['decoder_h2']),\n",
    "                                   biases['decoder_b2']))\n",
    "    return layer_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's construct the model.\n",
    "In the variable `cost` we have the loss function and in the `optimizer` variable we have the gradient used for backpropagation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Construct model.\n",
    "encoder_op = encoder(X)\n",
    "decoder_op = decoder(encoder_op)\n",
    "\n",
    "# Prediction.\n",
    "y_pred = decoder_op\n",
    "# Targets (Labels) are the input data.\n",
    "y_true = X\n",
    "\n",
    "# Define loss and optimizer, minimize the squared error.\n",
    "cost = tf.reduce_mean(tf.pow(y_true - y_pred, 2))\n",
    "optimizer = tf.train.RMSPropOptimizer(learning_rate).minimize(cost)\n",
    "\n",
    "# Initializing the variables.\n",
    "init = tf.global_variables_initializer()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The training will run for 20 epochs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0001 cost= 0.206316501\n",
      "Epoch: 0002 cost= 0.173048288\n",
      "Epoch: 0003 cost= 0.156645894\n",
      "Epoch: 0004 cost= 0.143489093\n",
      "Epoch: 0005 cost= 0.138170794\n",
      "Epoch: 0006 cost= 0.134937942\n",
      "Epoch: 0007 cost= 0.130655780\n",
      "Epoch: 0008 cost= 0.126598656\n",
      "Epoch: 0009 cost= 0.122495979\n",
      "Epoch: 0010 cost= 0.118640684\n",
      "Epoch: 0011 cost= 0.119143337\n",
      "Epoch: 0012 cost= 0.114796378\n",
      "Epoch: 0013 cost= 0.115012594\n",
      "Epoch: 0014 cost= 0.114606217\n",
      "Epoch: 0015 cost= 0.114076257\n",
      "Epoch: 0016 cost= 0.110001050\n",
      "Epoch: 0017 cost= 0.112542860\n",
      "Epoch: 0018 cost= 0.109700687\n",
      "Epoch: 0019 cost= 0.107055821\n",
      "Epoch: 0020 cost= 0.104846880\n",
      "Optimization Finished!\n"
     ]
    }
   ],
   "source": [
    "# Launch the graph.\n",
    "# Using InteractiveSession.\n",
    "sess = tf.InteractiveSession()\n",
    "sess.run(init)\n",
    "\n",
    "total_batch = int(mnist.train.num_examples/batch_size)\n",
    "# Training cycle.\n",
    "for epoch in range(training_epochs):\n",
    "    # Loop over all batches.\n",
    "    for i in range(total_batch):\n",
    "        batch_xs, batch_ys = mnist.train.next_batch(batch_size)\n",
    "        # Run optimization op (backprop) and cost op (to get loss value).\n",
    "        _, c = sess.run([optimizer, cost], feed_dict={X: batch_xs})\n",
    "    # Display logs per epoch step.\n",
    "    if epoch % display_step == 0:\n",
    "        print(\"Epoch:\", '%04d' % (epoch+1),\n",
    "              \"cost=\", \"{:.9f}\".format(c))\n",
    "\n",
    "print(\"Optimization Finished!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's apply encode and decode."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Applying encode and decode over test set.\n",
    "encode_decode = sess.run(\n",
    "    y_pred, feed_dict={X: mnist.test.images[:examples_to_show]})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's simply visualize the graphs!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAACNCAYAAACT6v+eAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzsnXd4VFX6xz9nZtILJHQCJJRQlSIKKBZsVF17V3RX17W76rq6/nTdoq66q7tWXFZdu6uCvaGi2GiCIEjvEAg9kJA+M+f3x3vvnZlkEtImmQzn+zx5ZnLnzr3nnfOec9/zfctRWmsMDAwMDAwMDAwaBldLN8DAwMDAwMDAoDXDGFMGBgYGBgYGBo2AMaYMDAwMDAwMDBoBY0wZGBgYGBgYGDQCxpgyMDAwMDAwMGgEjDFlYGBgYGBgYNAIGGPKwMDAwMDAwKARaJQxpZQar5RapZRaq5S6s6kaFU0wMrZ+xLp8YGSMFcS6jLEuHxgZD1lorRv0B7iBdUAvIB74CRjY0OtF45+RsfX/xbp8RsaWb5uR0chnZIwtGRvyp6wfp95QSh0N/ElrPc76/w+Wcfa3mr4TrxJ0IikNul9LwIeXckpJJo0yiqmk4i44tGVszfIBFFFQDNx/KPchGBmjEWYsVkdrltHoaQCtTcZglFFMhS5XBzvP04h7ZAFbgv7PA0ZWPUkpdTVwNUAiyYxUJzfils2LHTqPPWxnoDqSeXomlVQckjLGinwAX+hpOxHdDUGsyHgo6ynEvoyxIh+YsYiRsVVgnp5Zp/MiHoCutZ6qtT5Sa31kHAmRvl2LINZljHX5wMgYK4h1GWNdPjAyxgoOBRmD0RhjaivQPej/btaxmEECSZRRGnzIyNjKEEa+eGJIPoj9PgQjYyzAjMXYwKEgY0PQGDffD0CuUqon8kNeCFzcJK2KEqSTQSkHKNXFaDQ0k4wb7zsaAF+ixLN1GLSLOUOmh5zT+8tfkjY/CYBOj89u8L1aSsbmQrB8CSQBZALvt3CzmhSx3odgZIwFmLEYGzgUZGwIGmxMaa29SqkbgBlIdP/zWutlTdayKIBLueinh7KIbymlBOBNI2PrQrB81sDfG0vyQez3IRgZYwFmLMYGDgUZG4IGZ/M1BOkqU7fmILRCvfegEf2NkbHgo1wAvh/6vzqd/2pRFwDePOsEAHwr1jTovjbqImNz9qEaPgiAj95/GYDDn7kBgO5/bTgT94WetlBrfWRt50RCRnfbNqx6shcAK098FoC7dw5n6SV9AfAtX90k92kOPW1pGBkFrVk+aLmx2FyIZj31dO4EQEVu12qfxa0Wj92qP/Si7XJpfuaKMgBc3y4KOTeaZWwq1FVGUwHdwMDAwMDAwKARaEzMlEETouCj3BoZqWf29eLROacCkJO9C4DPBr7NJWn5ANx/RXsAet3ROGYq2rDzqHQAvPgASN7WfCxqU8PfsxtLx/wbgEpLjPs6LmTIWccA0L2JmKnmhO/EIwC4YeqbAEzJ7VOv7xddMIq2i3fLtVatbdrGNTP2TZY4x3kPTgFg4FPXAdDjoflor7fF2lUTPNmSO9TxjX0AfL1wIAD9n96Hb9mqBl3T3aEDAHsm9CHjjR8B0OXljW2qQRNh/6Wj2DNRGKY7h30KwOT0j6ud99z+HgCcnfYOGeclhnx2WtbwCLey9cIYUy0M78minF8OeQqIA+BfBeL6+eoCiwHftpO+BQsAcCWKcj8w73Duar9UrpERfZN1U6BgsBhReV6ZkNs9N6clm9MgeLp3A6Dn1NZtLITDpnGS7pzpPtCg72+fVEHlZUKOZ57WZM1qdniyuvLXPz4bcmz59U8DMOHx49BFRS3RrBrh6dyJv8yShJZ+cX4ATtrTGQDfsvovyGwj6pLvxIAalfgO1y/9jXy4KHpCadzt2wGw6p89GJMrcm49oRKIPaPPNWQAK2+UIpnfjv0XAB3cP+CqgzPqyjabrXeJtZ5nEArj5jMwMDAwMDAwaASikpna82uhzHtcJqv5lTs7UVEurE3W6/KanCerYf/i5S3QwqbDgax4AFy4HEZq1i8OB8C3vjrdvvbPwwB4LfMRsAqhdfs09mxiPXoo3572KAAnfHMjAH1YVNtXogqb/yjuu+HjRT8f7vJt2PNSjxG37ZZ75Pz2S4RlTHpvfqSb2GCoONHZk05a3KjrpC1K5Pwrvwbgq7bC4Pn27W9c41oAO8dlMza5MuTYEQsuAKDDgehx33q6SbHxNm+UMDjeDUC/L64BIPfyHxt83RX35QBwfqq4jo741+/puqjhSSJNjZ03yNi69+aXAJiU/Jnz2ZntTwfAu3Vb8zcsgijumcbqCVOs/5Lq9J1n9kmCzKubjqrxnDZEF8PuGiru6bLOwsJtPFNx7ogfAKjUouNfvTwCgC5f70dHkCmNvaewgYGBgYGBgUEzIiqZqd/f/hoA56QUyIHeQR+OkZeN3hIAHtt1Yr2vP39nNgApj7QBwDNzYYPa2RRo+5LEAZ274FJUQSEA3vyNNZ5/1cQvAEh1xXZ5/r0Dk+jiTgYga1pcC7em/ljymycAqNS+Ws+bNeRVeTNEXt4plnIXzxediefLltPL2lB0lgSeP54lMg54V0pW5DKvXtcpz9DclLESgFlpA+RgK2KmXMmin+Nu+q7aZwn/y5A3zVh65mAoGC1B5+/mPOUcG3D3TgAaGnWpjx7C2tMkseKEpecB0P35ldSu9c0Dd195cDx7m8QMDY2Xx50/6Jz8KbLpcpffdMabv71Z29dYeLplseIOYXQ7zZbM/fTX5wLgKtesrqwAYIu3LQDdPfu44ufLAShYIfFjnX4Q/Ww7ewv6gHh72uyLLvapKvTooQCsvx5eO/o/AAy3mNawuF1Y/tLfVTB1nzBZT/8k5YRyr1yBv6ysSdoVlcbU43ddCMAfBwtxlrFCUzBAlCV+sGSfPHzY2wD8s8s8PipJBWBScvVA2FItCjWvXGjAMYmV0EUm/T4XSJBk37rtYxhRHKzO0Mb7xfV5Zdt/WEcSuS1/FABpX6yQa0Ssdc2Pk6+bw7vFMgmkzhJ3Z2uQL26WGENxqpbBbWFRhZ+NlRK8e1bKXgDOT5WH2/kvT43KzBk9eihPPfQYAK8UyqKk/92iu/Xtn6PH/tyUTWt2lB8jBuB9HZ9zjpX4Zb5Jf21ui7QpHOzMvV1nBB4aR/5DXOedtzTMHaePFuv/7ldfdI4d+EiC2FP2rG/QNZsaK+4Ug3ZwLQ/aecNl4b56TgVnv3wrAL3ul3CCpnrINjXcbYUEGPHRBt5tLwXkRy+4IeSchE9+4PZJVwA42ZnuAblkrloHQKY/9HkTzSlM/mPFeNooCbJ8NFoWA709SUi9cPi8VFyZdy0/k32b5bnx85my2LtnhzwnH+68gCFJmwB4dMQbAPzhlivo9remcUkbN5+BgYGBgYGBQSMQlcxUyrR51mvgWHqVc57oPAaA+0bnkP610JIPj6le58ZTKqRuyhKpydTum+kcHm8FsW9sHe6jfZcdzfeThZFq45J01TnlbhbfJ8HoSYXRG6xcX7gH9QPggY6v81xh6wpKLj1zBL/s8hYQcO+Fc/MdNlOCfjvMTCBhv3z+hzGyrll63uPOeXl/kMDZplo5NQUK/lBCN4+sY2+9cRIAcQX1c0d6ugiD8d8en1KpW+96bsPZ1RmPc9ecab2LnoDmLY8Jc79mxAsA3L1zKFn/lUDchrK9W8cI0z86wc9hs8V11OOJ6NFT98C+fHHyv6z/hLV4aI8wiQv29eCN3p+GnN83Lp7/XCIB2w89fwYA/g2bmqexdYRdFqd8mjBTd7X/kn5vC13T/53q/Vm1Xlhjd8hoCax/bSivVnPlSX9etOFUfljZE4D+N4t3pkPxKjpYZ10z/BQAdt4kDPotU9zc3WkWAN+Wigdh8Q1PcOYr0t/eLXmNamvrnckMDAwMDAwMDKIAUclM1QXe7TsASJm+w7HGU6btqfH8HVdJzNGgeA//2CvsR85/xbcfzf5igN1HaIeRsnH5rKvo+27sMFI2tp7aznm/sCjbelfaMo2pI2w27b5Hp3JkfIV9NOScd4q7cPdX5wAw4PcSdO0rLHQ+77dGymLM/4X084iEMj659mEAxib+HoCcBxa2WHFBu1zJW4f/nZf2DwYg7ouGBcgv/4vE8FRqH5dvlNWjb+euJmhl82LSUT857/f7RUcr/yR7nrmiiJnSWuJNbZZ03p4c3KU763UNV5oEaq+6XwJ43/2FlC3xE0eP85Y2VVObDLtHtCPHIwkCV285HoC8URJT60opYfg1EjP2u19L9f5L0nZyvDXFfjBdilYunyQMajQEprszMlj5V5kjVg2QgrALy6H/X+QZFjyXtGa4UoTxXPMXKQ+04oSncFlz6Q/lEix/yXvXA9Dvzyvou0+KWfurXgg4PE32GPzcI+zVgr8Pp92j4vU6M2WfddZBt9yre9ub7EoGBgYGBgYGBocgWi0zVVfYmSxP3vUkIFlWbz0mq+F2+dG9PUnF58LMzOn/CHZp/yFzJD5hwG3rWkV2W31RODBQ/HDxk5LF0Zbo7ie/lXIdYKUC+NWm8QAUXZBE3zxhEsP1m53Ned0LEk+14Df/ootbYgN+vFJiP855+3L0TyuatO11hetM2UOvqyeB514TmbpRvxgZm8F75WRJpS/XlWx+VFbbKeX1K6vQkiifKEUNn8z6j3Msz6K3XV9Hf2HZj/u/y5WzpKTM5iKJHal4rnON528/TjNxpBRofb/r09ZRiTcdvfhCMoi+WBxfAvgRJmPJv4XlyLTmEX9xMV0eEd1983Tpy4vSPgQt/MaOcmHhdFn0bDGz7dIBrDpLstPeL5YsxedOOxXfrnUt2awmxz6rYPWX50mMsItkZpZKGaAHr5NnX5/PJFM23DyqPB5c/aySGO9mAvD3lyTj9PD4nYCwlW4lPNLh8y4ma2fT/IYxb0ytvEWq/h6VIHTesopSMpeXtGSTDgpPrxwA/tpHgpkzXIkstMZ19l9FhXwFBS3RtIihfIJMau+NlQnjL7uHkzl9CRCewo123LVD9lUsvErclr68uj1wcqaL0XLPmaN4sPMPkWlcPWDvu3Z334+cY90eaFig8crrJGX5yATR4acKBpIyvfUYUTZ2HFU9ceX0D38L1L/WVnOg4xNilH81VRZkJyaV8VyPrwBwWW4O/6M118NyoRzDxMbrReLObHeXJyrHZ9o5+c77/eOKAcj8b/Xz/pj9vvUu4KT5dlF/APoWRE8YRdHIQKjDYxtOBiBpdWwZUgBW0XLKdMD9VuQX/d0+UnZeKD1bKpr3yQ3q4zLR7fOyf+T6ti8DsKBCzh+dYGtosnP+92VyLOs+1WShE8bNZ2BgYGBgYGDQCMQsM1U+SZiOH8/9p3VEqMJrb76ZpNnRs+IIh95vSuDcsPiArXuRlU7f96eWZysigbyTRBUHx8sK4/KNh9OxeGVLNqneCC7UueQIeyVfTxeIkhWZx+WvVvhz25+h85nhvhQ5qGTpj3HJUp5ixA+T6UzDXI3tc/aG/P/qhiNpT/TsXVdXxA8LZYVXVJTQ/3FhFKPR9W5X0n/s2JMA+OsxOeSNFf1ce/ozAMwvF7279LNrqn0/96VyPnrr+ZBjDy8fB0DWT5Hb66wxKJreBQbJ+ysGClv4zVHCaOwaloo+TXTxsDh5FqyorGSQtefkOxOEHb9j1K/lAnOXNFeza8Tro6dicx/TBr4CwNGP3kbP9yW0wD2r4XsrRhMy3hN9unryJQC80v8VfpEiunrOteJi9ukAF1quxb+eoIJNGXkfYKQEXnyMWSIFwTOvl5Gq1zed/hpmysDAwMDAwMCgEYhZZmrzBLETU5UwUhdtOBWA5E9/Inp2ywpFweWSfv7nTo9YR6Ttl288hQG/l8Kk0bjybQp0OExSte1Vh+e9jJZsTr2w6lrxxR9sH766YOPZEmM1rcN8Z9dz+7pd723++DH/Xkkh/usu2Y/v4t4L+KaLBHjWNWXcTgL5fuj/rCMyNkvntodWxkyVnTaCBUdNsf6T/llV2RFfK4hfscvJJL+9g76yGxcTrzki5Jy+VGftXYP7O7FV9+0+DIDsm4WpjNayMp3f38DqPwhrc3u75QDc8a4wqsHxXxesk8KzpTd14KzXZwHwy/QtAKy7SfS0dxTsDDQiIc6ZBzKsMjkrL3iKyvPlmF0IuM0P8tmBbpp0a1ef9kuKnevsHiylBzrNsubbKNNbf1ERAAlj5fXqTmez4k85AIwdLiU4Vu/vCMCmre1xx4v8v+gn7OHDnRfUeO2BX11Nv9vE6+PdUb/SIHVBTBpTrrQ0LjtONh8t9Mv+Sjsf6AVAQnl0usk8WV057iaho6tuYjxneR/6FkRnu5sCnp7Z/KOfBNv/Z788eDOfj+4MvmDcfdwHDf6up7tUeS8a3hWAZ375dLVz5pfLBKkqmv/RZU9un22VoNxvh75G/odSgfnbfx9d4/f2DZQHVmrOfkZ13SjXqmIKqmhd1dSC0vbuau7X3y88m560vCsoUth8r9sxQD67X2o2pW6JAgujFnjzt3P17ZIU8N9/SE2svnFiSKD99PlMXHj9b5BQAn/xch788nQArjzTqoR+pFiczw6ZhL+Fsmht9Pzg16w+7Zlqx21dXHWKlVl6St2uN/9OMY5/u9xye50WnYsa346d9L1WDJ+N1rF4pDJ9LoEK9Z+9I/XPgo2pjV5JNDvzCanTl/uv+fi8kZtDjZvPwMDAwMDAwKARiElmas2fBvFhe1nhn7FGqk4nfBzdzM6Ku7rzbudQhuPEpecBMOD3a2PWvQew5jddGWWRcb/+UerfdOfnFmxR82H5n6W+z7KxT1b7bPqB9gBM+Z3oQeKKlkucyPizsGMn/Oki3jnsBQAeurdm9nBBuayYfbiC6m+FVhvu8cTSqEyrrw3lZ+5z3q+okJVvt2dbxx6f9cXuq4V5XDLqKTZ6JTU/aVf1WmrRitS3hOn/JbcCsPd86a+y/QkMuF3cW77igAus353iDjw592wAPh80HYB773WRdXbztLkm9Lt+EePeuhqAyU/KcyLZVc5pybJzQFW29GAYkSBM43fDXgVg0N9vovftrccbYGPDA6KjPx5lJ5rFO5+d+7AwUl2fklIukSbCDTNlYGBgYGBgYNAIxBQztf/SUQAsueBx1nmlkvaBhyQmJYH8Gr8XDVj4i39iB5zbaHOdrNu9MVagsyr83cuc96X7Ems5M7YQN6sLf+syvcbPX9h6DACJH0RBKY/5EvzZZiJcNuYmAPblJtR4erv/BFa5W9+WHPWFI18IOceOx2oNcPeVoPsFR72CHXj+yQEJxm7oHoXRjpJTDzjvz118FQAdv2p9Kfg2Q5X6VuBYOKbf1sfCd6Rf7dIKDw2eztNdxgAtt0+f9nodPXu9f1fn+OPnSsyTL05Y32N+J3NFXQv+uiw+pduQ6H4+hsO2249hxiWyf2mSChTkfKygDwCd/ytV+5uL/T6oMaWU6g68BHRCmLKpWuvHlFKZwBtADhIbdr7WulU+9ct0Ccv4gQrKAEUWPemhcqnUFSxlLqWUUEEZSqmM1ihjXeRLIhnd6pwuAdRVRqruQNyKEOt6CrEvoxmLZiy2FhwKMjYl6sJMeYHbtNY/KqXSgIVKqc+BK4CZWusHlVJ3AncCd0SuqTXDkyWW+m/veQOQAl4X/nQZAB0+ObiFrlDkMph0lYFXVzKfmWTqTuSzkUw6kqP6843+CB/eZpWxspNkTcVVZIX93LdLCgXa5fBVgjAF7g7tA+d0aIursojEC3Jo0zMLf2k56+9+igO3jOHA1z/iShnG8Z9lsW7/fDa30B5bT498xXmf9UnD5te69OFGvZK97Kx5E7IGwq3kwRcct1B48aiQc/78l+c4Maks5FiccgeVU6gutz5pa8j/0aKndoHAdrPqdn7pRtnrjJGhx/XooajvF4ccixYZq2LHiZKOHdzHT34l5Vbqs4VMXfW0pcZiMP49XLblyPeV0O5fyQc5O4CWHItNgQ7/FnZn5ISLAZg3/DVu/l0OAL1vE2YqWvQ0ZVqo7n0wRGKIHrzsB0q0xLcN/+ZaALKfdbP7JokbE4a1dkSLjFVROVa26nr3hofp4QnVy83eEt6/Q7bbSShp3jjpgxpTWut8EB+Z1rpIKbUCyALOAMZYp70IzKIFjCnl8TDkwzwAzkvdA8CrRR3pdI/Ql3VZ3yWoJBKQ/X88Ko5knUY5pexiG8M5AYA44qmg9EyaUcaPpj1f6+fHLLoIgN070gHI6CA09bzhr9X6vXPfLOWaHm9wy8o9fDa9MyePGEvW7TtYx/ImaHXdUXa6VCQ+NnE+jfU416UPu5DNWn5u8gJWD75xLgDnWxsSA3zz96eA0NpTlWEiIGuqTXXYzGvIJdSlEq16elBYceeuKiGaVQ0piF4ZyzIDwfMLy+UhNeAhmXfqk2xdVz1t7rEYjLw/iHt5dILo39zyZNz1cO+15FhsEvhlTLZ7RB7Uu18uZcWFMp5Pf20yAAkLl0WlnvaYYe0zdxkkKwnGXnHCc3Io+1Q+zplhnRk6FjdvzyTXKT4giNaxuPE0WdDkBBlS+T4xEif/9jaSP2qZ/THrFYCulMoBhgHzgE6WoQWwHXEDtnqU6mKK2EcbMqmgnAQlyqTkidDqZdy4pZLFP5cz4ogEdu720aWTGDHxJKKjtpxp/VBTH8aTCDESJxjregqxL2NtemrGYutBrOspHBoyNhZ1VmalVCowHfit1rpQqcBKTWutlQpfgk8pdTVwNUAidaeK64wh/fhrx5dDDj31wHm0/an+aZ5e7WUJc+jHUDwqLiSX0lKaiMl4xvJLmHnYtHp9Z/aw12v8zKZ4K4P2MRo372JW3/Eq6WMu5Zjpgymq+D8Oe+EGsr7zopSqMXc0Un24+RdywwTl4S+7Dwcg9T0Jsmzoo6TWPoyQjL3eEHfr/EsTGZFQdpCzQ2EX5Jy6XVZ6BdeJ56P/hprLYbSknjYIVmuqFu2sDdEmY8cgl+v7hcOAgJu9IWgJPa0rLrloJhCoFH7lgivIRhIQ3O0y5aSOUqnft6Jmd2Q0y1gXuL5eBMCYF29n+a+EmSq6X0pEpJ+Xhr+oKOr0NG6B9MeoHy9i7hGhz4eXcz7H5k/KtSRonWYV7ex/07qon29s3Vt0tu0BCCTAjPnuBgB6v9MyrBTUkZlSSsUhhtSrWmtrIwJ2KKW6WJ93AcLWZ9daT9VaH6m1PjKOmrN/Whp+7WcJc+hMDzoqiVGKJ4FyLYPHehC0WhkrKzXr73+bzDGDSD1sMADu1DS8hYUAlOtSe2BUQ2uQDw7eh9ZrWK9MrMjY2vUUYl/GuuipGYutX8bWrqdwaMjYVKhLNp8CngNWaK0fDfrofeBy4EHr9b2ItLAGuAf2BeDq/wVuO/D56wHIebl+Wx1orVnOAlJII1v1dY53oCv5bCKH/lRSARGUMWncBgY9INa1DtMraf1ll/Nw8VCDvv2lfG9zinOs1zQrrXn+UrTWLOMH2uCm36IMQFi7Sp2B58HpJKj+bGQTHpqn+KA7XWK87hj9sXPstU9km4pe3oYVjqtLH+bL9gP7arxIA+FbLlsx/PHWq9hyurAvqyf8u07fve552VOr+/2zrSM1J8VEg542BP7EUEZql6+8xnOjTUY7qeOMrj85x/ZUpEpby2uWoybUVU+bayzWBX6fi503SBzVpKu+BeDd9V0AwhazbMmxGAn0mbqFl88Txvibw8V7MG7wL1nx3X+iRk9t2OUdOt+YwenP/wKAu3I+AuDoBJ9TCPj/Pr4AgD63yLMyHCsVLWPRnSGhdb+dJ7pn77cL8NCeAQDk/loYuZbMgVVa1+5QUUodC3wLLCXQ1ruQuKk3gR7AJqQ0wt7arpWuMvVIdXJj2wzAmiclNWjNWVOcY2Mny35Lnpn1q/uyT+9mAbNIpY1zrA+HkU4mS5lLGaVUUIYPb7vmlLGpUBf5kkimgnKK9L7wS2ILTSGf/YDq+72o09KCriSdI0aEz2LK6ou6yriXnYu11sNqu1ZTyFh4kWTzxV0hm8t+OugNxv4slLr/BckM0woyFkvSRG3uEhutVU//uF6Cl1OUuBYueuEWAHr8eXa1c6NNRuWRlc3m/8nehMuPeYXD5l4CQNbZy+p9vWgbi+FwwhJhHewNgl0ox+U36JtfSZv/JJXDfavWVvt+tI3FpoB7QC4AH3whGeMDXjqWdXe+FDV6Wht23CSGcNFRpfS/W1zT3k1bDvq9aBmLBZdLhuL8v8mz3hcUunLCTZKpmDI9cu69eXomhXpvrWMR6pbN9x1V94EIoOW1vAnQVrXnFM4N+5mdtWD9oLUqTLSiLvKByNhaUVcZv9DTWu3OPLGupxD7MpqxGBtjMWVgD05RsaunEPtjsanR6rIp7HT6mac/Yh1puQBFg4bBdo+sknIhxLMp5vYeTH/dcjVbMaBnMYIU1lufrnfOizW5w+EvG8TdUPy0xFz0mF6dkYpWaGuX+Zw7hYkZ8LfLUIvTWrJJEceM/5MH5fI/iCtvzrz+9H9sGwC9t68CwFdWv0SL1g6bOb5g/VgAPhj2LFeOuk4+nLukpZpVJ3R6XMZbJ+pXxiNacM7vvgBCGSmAPh9cQ98IMlL1hdmbz8DAwMDAwMCgEWh1zNS20VKwK7jy6atFVnXiQikHEBsVWgwMYgQnS3HLFPJauCENh2/tBgB6nNfCDWkG2HtB7vpA/u/D3FbJaEQCJWfJ02Xe7K4U9JOEn4z65TsZ1BNDkjYD4FbC/cwtEz5/4MM7o0ovDTNlYGBgYGBgYNAItDpmqir+tmcgc8blAKDzl7ZsYwwMDAwMYha+3ZJ9O7VvLzJoWBkXg/rht69eCcDKXz8NwK+evxGA7uujK/ay1RlTve4UBZ545xFBR7e3TGMMDAwMDAwMIobse8VoGnfvUAC6E11GlA3j5jMwMDAwMDAwaAQOWrSzSW+m1C6gGGj4plbNh/aEtjNba93hYF9SShUBqyLWqqZFvWVs5X0IsS9jXfX0UJDRjMXogRmLNeAQkTGmxyI0szEFoJRaoLU+sllv2gA0tJ2tRT6IfRkb004jY/Qg1vUUYl9Go6eR+25zItb1FBreVuPmMzAwMDDp8IxyAAAgAElEQVQwMDBoBIwxZWBgYGBgYGDQCLSEMTW1Be7ZEDS0na1FPoh9GRvTTiNj9CDW9RRiX0ajp5H7bnMi1vUUGtjWZo+ZMjAwMDAwMDCIJRg3n4GBgYGBgYFBI2CMKQMDAwMDAwODRqDZjCml1Hil1Cql1Fql1J3Ndd+DQSnVXSn1lVJquVJqmVLqZuv4n5RSW5VSi62/iXW4lpGxhdBUMkarfBD7Mho9NTJWuU5My2d9x8jYQmhKGQHQWkf8D3AD64BeQDzwEzCwOe5dh7Z1AY6w3qcBq4GBwJ+A3xkZDx0Zo1m+Q0FGo6dGxkNFPiNj7Mho/zWKmaqHxTkCWKu1Xq+1rgD+B5zRmHs3FbTW+VrrH633RcAKIMv+3MgYgtYqY7/WLh/EvoxGTw8JGY2eBmBkbEEcTMb6osHGlFLKDTwFTECsuYuUUgNrOD0L2BL0fx6NaHSkoJTKAYYB86xDNwDvAUuAYzAytkYZFXAFoIHvgEtbu3wQ+zIegnoKsS+j0dNQGBmjBOFkVEotUUo9r5TKqMs1GsNMRa3F2RAopVKB6cBvtdaFwBTgEmAWQv89iJEx6hFGxnnAl4jBvxUooxXLB7Ev4yGqp4eCjEZPWxkOYRl7A0OBfOCROl3H8hc2pAHnAuO11ldZ/18GjNRa31DlvKuBW4CubtzpyaQ36H4tgUoq8FFJIimUUUwlFZM5BGW05LsaSHHj7t9a5QMoosAPTInxPswD3otxGc1YNGMx6mD0NIDWLGMwyiimQperg53niXRDtNZTlVLPA6uTSU8fqU4Oe96MbYsBGNd1aP1vokTOGVsXOdewr2cj+Lp1vdcOnccetjNQHck8PZNKKsKe1ywyhrlGc8motZ4KTFVKeZJJr2yt8gF8oaeVhjsvxvowL9x5MSZj2PNiXcZmGYvNMJ/CITMWw55nZAzfvoaiqWUMxjw9s07nNcaY2gp0D/q/m3WsGrTWXqXUDcBHNV2sMT8kYdi12q5X13slkEQZIeM9cjIqFVaO2tASMqarzEbfs66IkHxFtJSe1vN6jZDxK0TOaoiEjCouXq5dGX5SbXVjsQGIqbHYfPPpoTAWW4eeKhX7MkbgXsFoTMzUD0CuUqqnUioeuBB4v6aTtdYfN+JeLYJ0MijlAKW6GI0GI2OrQ7B8fu0HaEPs9+EkYl9GMxZbGQ7RsXjI62lrlLEhaLAxpbX2IlH9M5CUwje11svqe50Z2xZXo+Xqc37w+3Fdhzap5elSLt58JZ1FfEsxhRBJGYNWg3WR0d22TaCdiYm4EhOd/z3du9V6K1dKCq4UiVtoChlbSx/OYQbA3mjRU5vRAYSZVKr6/+qgrvrm1dMw5+vKCnRlRVgZXcnJzjHl8aA8VchwW0aXW/5qQEvLGPy/DUdX69BHdYEZi3VDrPeh3camltGVluYc06OHokcH+tbdJr3VzDfB/9uoi666kpPrLGPV69cFjaozpbX+WGvdV2vdW2t9f2OuFa2YeHIKx6jxpNIGI2PrhC3faDUBYHtLtycSiPU+BCNjLMCMxdjAoSBjfdHgbL6GIF1l6pqC0OoLJ5Asa1jgYB1lqS1ArSbM0zMp1HsPatI2WEaXG/w+ADb/8RgAVlzzNADXbR3FZ6sHyHn5CQDEZRfT4dUkABL2VALgKSqXS+3ez0fzQ13UTSVjQ+VTHg/a6w05ZvfD+J4j0eXSdpUg8jGwD64NVhy1dUzvLwTAX17uBMfaqOsK+gs9baHW+sjazmkSPa0SxHvSFVex5ZQ4+aybxBskJFaSfd1uAHx7CoDQuKOo1NPg2L4qMk46+nTnmE4WplS7XKi8fAB8Vv/ZbJUuL+fTzQtCLh8VMoZByHxTRX53Whr+0jI55Jb1qb9CxiTa3yBdjeRYDAdbvomDTsR/oFgOiiuu2rit6bs2xvc48qDfgSYeizbbYPWNiotHeytD22n1w/jsEQHZ/FZfWnNvTYjKsRgEm+X+dNN8ACYdMQ5dIvPM9ksGAeBNVuhj9wHQ7c/yvfzj2wLQ9YvdfPzFmyHXjBoZq/Rt2CBym9WurR9dbmbkLQw51JQymo2ODQwMDAwMDAwagYiXRmgI7JWrv6Sk+oeWlWozUjO2LmJ8z5Ehp9gsx8FgW6Uzti2OaJR/OPiPk/ZvHi8r+A7Dd3BEeykUW7xb2O9PS4SReTprLj0XDAag7zBhay7JmsvTV4+R831iE3e6VVaDvh27nPu0pIxQQ6ZXtT6cF2ibT1YWJdkpFJwiK6r4QlmRdHxliZwTxECGla/KSiZisFdD1io3+H4ui00be+7lAHw57Vn+vrc3AEclbQDg0byx7B2dA0Cb+cJaefO2VrtWS/chELrys957uncF4KTJRwDw5ZznmHjyeXKeV/rRVVSIr6fE8Lm3iF6rtFS5VP4O5/KOLmxb1GIy2nGH/rKyoIMia6APgtqnrLVoQgLkSmLz7iFSS6fDOysB8BUUOJcKK2Nz6So48Wra5ws6Jno3YeLFAHyy7DUmjL9QPttsMYr79tfp+oHfaEHz96H9+1n9FU7G8dkjAGFvJvQaJaenSuyob+++g7JTEB1j0ZMl4867Ld+R25Zj7Dky33z244uc+KtfA7BvlDwPLx7yA4OTNwOw9zUZg/9YNBYA9X5A56NhLIagytgI2we19J2j90FsaST6MWqMqWDqzjGirIlGeeICD2Prh3X36wPAqRcMJS7HygLfuUdO8Xgcgyoc3dxSChIso+tbi/ofdzQA+rkOrHprPQApyOu9k38FwOML9jBg2yoAdp4jVfsXXLuDaYe9INctlt/irVShUbXPV7uMdaFEGwDHbZc9wukvp9+C3UNV3CTBbVXxYnztOMrNixc9AcBViyfLZ29bwczFxbXK50oS92dYY7yRCKGYa/n97Afy1jEyyd2xYyhntRGKeUm5PHhP67iEWbeKnub9PReAVMsV5isqigo9dWAZjCou3ulT7yYx/kuOld0hTr7sSsoOkymlzTdiMGqfD3e+uDIdd22S5crtk8PEQdJXqMLICFMDwsnoGFHBumr3cRWjCsCVKHLsntiHu+9+EYCFJT0B+HazPKzjvlgY9J3mC6morZ6Ou317fLtkwWX35eorJTh58CPXkeUSV5C2XZUuN8pluW+rzqcHS6mPoLEYbr6x9dSVkOD0p/1Zydmy6D7ir8Mp+Jf0a7sfpF8zVpfhWShzbLh5I5rGonfrNnnjcoMWOWyj3bNHymWMPedytLUJyviBywG4t8NiXEh/bPXJou2fHnlm+Hfurl1Pm6Ef6/sbj8saBi5rQWONU33MEDw7rXCQtjL3uvNE1707dzOu23A5X/mrX7CRz0Xj5jMwMDAwMDAwaASihpkKZ5Xm3yKsTdev9uNrI6vAvJPkNfuPcwBYf/cwEpd3BEB75HX5tU8z+B/XAZC8QyzQzE9XA+A/UFzNDTiu69CIsTXV7lMF7X+S9qWvKWLtQyJv77uFwWj7ksjog0Cwq7VYfLzrD+z2ybExyWsBeMtlBfiFkSEk/TdCMgbkq+7SQ+tA4HGFxVqFYQ39pRI0ueqXU9jtk5XlYZ3E3VDoqlldg9PwI8FIBd+nGoJkdLeTleG+U/oC0O2B2QCcddVC4pT87olK5J+cvpsDPnEtjX5A+vCZ508HoPt/V+Lbs7f6vasG2oahsBuLsDLa9xvQC1YIc7r2RWFJe188Vz4bPoj4uRsB8FkB2TPyFjIhd7R8brGOWMHp28d2psMiGc+eVVuce0dCpqqotR/BmQ+U23IbhSlIauvx/L9N4YBf5B2asBOA733CggQzecH3jrSM4eRT/YXBLurbhrRv5dj+l4WRyh0v+7u6hg6EldK/dhD9jLyFzoredturOGm/K6OtuJvA0ZHm78PA7+uU4vD5nLbuuVTanvlfmU+vWLGdzRXtAPi4m4QSfH7Emxz2mDwzejy/Ri5hj78a5lO7PE1d3aA1QqmwemLfp9rp9m/r104bbBbRt3qd/H/cMHb8UnRyakep4B2nUqi0mKztPhl3XV6SV39Z9dCYUGY6cqxqbYxUcOKSnZxkP79dycnO+FTthIb7aNqLDJpziXw+V1zubdcKQ5W6JgOVL+NTW/NTcImamooO1xWGmTIwMDAwMDAwaASihplSRx4GwNaT0jkwQCzPZCGT2HI33D9Y0jb7xYll+dMFEqexoqySviMksNWNsDybvQf48pa/A3BJd1kV22uLnTccQ9f3JAjPly/f037trD4iuqIKigFztxcGo83nVqBq4QF62fHVYVZCtgVe2Euuke89QBuXWNSXWIGVcJC6ac1YBiPcPW3GyA70Dfcbu63Ccp+VxHFUgshaMHpvtfNsOKuVSq9zfbtoqXdL2C3qmg5V4wiUomyYxMy0v24jAPc9JKxNiT+OF/eILs7KE4YgfdAHnJEmnX5d9rEAdEWYrMJzR5L6rjCUTjCt1tUDbSO48gcCxV0zJIV62/GZjH3R0tm9EqPo7iuB9XrlRqcPbH0dnz0CXNJGf7GVcm/Fd/gndqawl8RMtdsguuzO7YVvzfqIylQN4eJBrBW8K1Pk9u3eU+1rrt45AOR7Z9HRLYzIr3tIrJSbH+UyNdzH7jebyazKQjYWDoMUH4erg7AwlekWuz9Wc9ztMseuWJENwIC2Ip/vpxXONewkivE9R+JOlfe+QolHsVfxrvQ0J8A7eGUfab0EQgrA2gUpHdaiQ3t2jJfYxAfueBaAXXdaTIW72Ple6XftAZjauytvXyfPjBsfGl2Hm6vGM1I2tK4XK+L8tko5JTkcWPNC/Iad+Ff1AOCq9EsB+GDQa5RYz5Z7esp8k8APzrWqXiOSXpoaUaWgpis5GTqK/vrbyBjzpYkuViS68Xvk/EH3yjz6TRl8PWIqAJecE9qPheeNpGSMjLcD3WRkZn3jJeHjKr9BA5+TUWNMuVaLgZMwdBBdfykPEfWlGEw39pjJ4336A3DlaunoGQVifO0pT6FHR5kIrmwjxtGGSmjvkYfAgIUi4sQ2PwGw3ZvH8+vPAiDZoul9O3Y67YjoJGB1kq6swGsZck4HKletylt+imRYrLhaak/t9imSLWNq05uHA9D5BTFSHOWoCS05WJA6UTXh45XfAFDiryDZJYNn/6XygGrzmiVXULttV0vwAIioERU82KsMOnf79vCF6G6n+1NCPktzVfDDLpnclox4HYBLN47hzJxZADyz6Tu5hnX5327sQNlXMvn79loZYUG1yJqj71RcvGMA7TtDdOzmG6bxen/JJtr4mkxyfSst46ii0slws+tHTehzDJ+sFQNx0hHjAFhxdw4A542ezfvvSU21tqs6AKAX/FxLg+q/f+VBcZBr+veH2eLQGj8ffzUNgAN+D25L7tvWyoLmX0ePAXACvYGw92lqI8q5lW3sZLbFu1Hm1t0TZD4dMmA9O44Wo2hAe0kU8BeXBtpoyffJelkITDr6dD6a8wEQyPrbPEGMzMS9mvb/mR8RGQ6GYPePHYDtzhB3z7qrepB9r+jd/BvF2H99jbj7Srem0nG+DLSfH5b59PB5F3P1SAnsPmqxjK2ZD8vDOP31edX7riUWpjasOSg4yN4JoagMPL/SxePHN796B4DcV25lzaVTAMi/TcZd18ek70Keey30XHAlJjry2IuMHef0o/1Ucc+u/eeokPOHHbmW/RWyGHs6S3T1mX1ZHJ8owfU7rxcZ9x9l1TD0VPDIyLcAmLJ5DACeJ/w4kjeyT42bz8DAwMDAwMCgEYgaZspOm26/uMihxg9UyLFZhQOoGC+Fcs9PlTTKF06TYPMDA9vxWqGsuK585TkAxr98O+5Sy6VmZUCOv3IpAMclbeSRqw8AkHKNzQqpQM2Y5rbKnXIBgftWTRXd+6ujuf3O10K+9sftJzM3Xyj6Ti+JdZ6yXNgu78FW8M0pY5hVlF2JOKQkQpU929xKMbdM2tl2mbADWodJZw12e0VQrlrdvzZb1TGTkuN6AfCf7v8GoFzLKv+L0jS258lqiyHysnrqAHoeLy6/W0d+DsBZacJsXNHlex4/XOr9xM0V11ltjF6TwmImXClJ+PYJw7FjtPzOxyRt4C8vSJD8+jHWeMuz3MzaX60P/KWlTvCya6CwBredJPuenpG6gi7nSRr+589KILAvPj6QIFKVdo8EGxDmmsHBwHZFaUdXXW7HhRkMOwD98bwJctmigmrnNAds9567s8yPdigDQFqetcPC/rZ0HiCs58czZaUeSBlXuJIC+3wCVHRvx4QJFwHgT5Jx4LYI4f19oL09LpuxZlbI/YIPpQhDU9Hex9Y7hZm4u72wT+88eSIAPWYXoNcII8fD8jKscx75XnkudIsXtjDjMysQPTKtbzhsD4fP78xLNvtr11wsOawr5Zmhv0/u31Yy4Y/C7nT1il7bSSGutDR8+/aFXD8iTHAtCJ7flFU367Trv2HehTkArOv/DAC93roGgE37M9mzzppTrQ1CHll8Ku/+U3qsc6XIs7+fFTpyxiOkWSU+7tgt3+vWL474AjmvsYlLhpkyMDAwMDAwMGgEooeZsgPwFq90DqWMl0DUJUBSW4lGD1SxnQ5YKaqdZBU29rwrAOi1cJFjceseEt/xxhnyvW2jiih8Vqxe73Yr2FKpFvMTh0NwdVb5H477k/iB8yxipLAykcrvJGZll8V0JH9pFUcMt5oIt8qI0EoyZB8+69rBVaWryzfU6a+JqyYC8Gbu23x5QOLiWL724O30+yK6Mg4O+qx2D+u9b9kquk8V9mW3T2KN2rtF1x7v05/c0ZK+POEpiTvJdBeyP1cYgpc3WuxOjrx8OCiD9U9KYG/uLOu3CyrSF1FYYyE4uDX3ekmdv/H60bT7lfTVpHuFofp0k8TTBDONoQUArUKK+4Rh7OyRwN0rexzL3g+lhERmnoxvFRffvAxHGEZTV1ZUKygboqsWc3P3Tokju6fDj+z1yarae5useHVZfuTbHgZOQVU7bjCIvUn8YL71CspK0pjQ7zgAZuRJrYRxXYc6yQaTRp8BQHzZdir6dAGgpIuV8GEtw3vfPqf5GSkbTgxqgC22dxDIvX4re66SUjMnX3YlAAtflnihcV2HOokrk475BQBlz0KJ1fy/L5Sq4LlFy0PuE20IDlof30M8N59ulnE6rms53ZfKs2/co9b+tbogNEaXQIKJb9cu572TKNLcCI57tQoCzx0SR9wsmT96fSFFrNefJwzVuK5DaWcxsRPvPQGAXPdm/IXCMLqsJK+eA+X/67KP5e8bJbYq5wIJWHe3b4eviUrpGGbKwMDAwMDAwKARiBpmyk4zdSUn486SVZCTQZI72tnJ3E4Tty1xlA/fTsmYcRcdsM7xO3EX9rrs0g7iU966MoMHXxFfup1O6z9wIFJiHRS17YA94q5rATgwrYh/7BK7NytB/LsLt3YndZdY8m3XSCaOs0Krq687Qiuu4Lii2rYKCHfskq6yslpQnsz7D0mMQ5vyuXW8ceRXkE5h0KDVmy3jyZddycPdHwMgWcmK6dKNYwBwd4rHP8/KVLNWxUprOs0X9m1Hn6SQ+zyx6Xv2+iQz8MZf3QBAh/dXh03RjxSU2836B2V1v3qyrOov3HAS5fdJ/2pr65u6bgOx7UyJ8TslWbLgTtiSz1XrOwFQ2Vlefd07wg9Lm0iCOsBfQ6xilRIUTkwRPqdA4t5KWcnv91dwxRprP7uf1zZDo+sHO8Pt42VfATD+jMvwV1qlYKxV+aSjhBF2px/A200Y79WThYEb+LdtxOdJHNHOIyQ+td2ySuv8dKdcQotB+525/JNVwrCNPe8KUreJnrq8wmxMHCxFjVVckbMHqD9VxvOWBRlMbSflArKmCSNsx475mitWsY5w9LTbcCd+z27rxDHnAKA8mwLbzgTD3o6tnbCP62+Qbay0qw99/itsqn9DSci5kUZwPKrNjn2y5nsAzl9/MtuLRbZub1uZ+TedYH1zv8PO+fZXOm32WHPJij/IfLNiwJMAVOb5WF4pv5e7g2QPE2bP2FZfGsGGv7QUdsvAndj/eOtYMTPy5MFib8JoG1Uzti5yDCu7evaMrYscinrtPfJQm1koAa6vzxlF/8esybrS6gDlah73SRBsBQp+ENmDZNLw8QCUdJLOXX7MKxy/VMo5fP+yuINWPPwMR84QY2tfrlWrZ26YAO0wQZrNiXDyVX34zti2mImnnA9A9zgxeo9PhPQNpfW7WTO4G0IocOtBa+vp9hsS6OaRDURnlspn656Ukh5zFj3j6C7W5Pzp5gWMO1sC0OPiRP/6JcgEuLKiAyV+qzr4eVbpjg+al0hWHg9HHidu9/eL5aGTlbiP/z33HyDYwBCE2zR0xrbFTBwkRvEdN0pJCL/VPx8XZ3PgXnk4J1j149TSNegWKt0RTlftuUX7tXN84kCZzL/aKKUuns6ay+43pKZR+8qWce/VCK2x96s7+VJxdyUU7nfKOjiB5dYeZ5/+8DGjl0jtpaTvRJc/mvchA2dLrSLvWvkd4opl/ko4EMYlFOFkkHDwF4sBYId6oDWznhU9PepumSc7bhQd/nTJTGeOLRgkRsXqyVPYabnmb7hdjMkD31VPNGhJbPk/IQHGZdn1Ct1OgsTEw0+Sk3ZKqMenm6tvNj1j22JH7o1XSKKMN0V0o8NC0K5AHUQA7a0MfDmCc6q9+HYlJzvz64DvLwPgsn7zuauX7Jk4cbkYitqq1j5j22JnTrWNyk83L+ChPWIgvpHxHgCbrE3X49DcfuNNAKSU2/UdgxYCpjSCgYGBgYGBgUHLocWYqRrdP1o7FKwviAWoaUdr2a/MWgVZQXWTjvkFK37XGYDbDpc07PcHymoj/RaPQwVXv3b4djZ0x/DaXFzhUuwn9JK01YJzcgAYeoYEQB4+72LaPyOrqvRCYWt6vf0bOh+Q3yLjOwnW89rXdLmZsXVhyLXH9zgS7H3GqtDWDZWxvrt913TehNzRrHxU9pjqbFUnfmZfX3yJop7hlNS+d8i1a1hZRKoP7RW/vboZMmkFeVZ69Y2LJH03Z4YEVo97bSi27tqs6sT+x6P6yTHPVyL/I2cLgzp51RYuSROX3oVDJNliYqVNb4dvZ1PL6CssZO66IwDYM1pS/Y/5qYyjfyeytXFbVb6t3yHk+xZLOPH4s1jzlMjWwSMFWS/sLivsNY+PZP1r/w6556Th49G7ZHVdtRJGxPoxDBzWzWasLd0anz0CV47MJfcNkZXvc/s7E3/ALt9QnR0Oq6u1tLOpx6KydhzwfGnNCX17M+5MWflXdBb2c/soYfBP+M3VbD9J5tE0q8D3xRtO5P8O/wSAWd2Fad18pzU3K9Us8tnfDXt9rXGnWYHU38s55ROOos9XvwSg8/k7AFjfXdjD3JevJetFYRA9Wj4r8VfQziUM/3VdxB36tz2D5fpKOeVcbITbJzOiMgLdH5gTcj9XmzSO+Iuwbu37WrtLVMgzQJ4lQYkrwOGPXodriozjEZ3FO7NtlCSF7Jt8NB9/807I/Sb0GuXsz1jVcxMJGf0lJXhyhO3tcZ60750Ph/Djfum3fUeIa67sJEk4G3PV4SQNlvl25bXi5t3t+5rfZa6yrig6fWO2zGGrpx7FhqlTQ+45LmtYkz03DDNlYGBgYGBgYNAItBgzVaPF53KHpNHXCGtVMGProoDf1PL5ejdu5qrjxTq9IE18o/mLhfVZeOlexj8pcUfh9kOqajU31Pqu93ddbiqPHgjArfdKbEmWR1YRo3NcTLzrPAD8azcBkDunAk+2WOw2k2fveecvL3dKSCi32MvuTplOQGJTydjg36ZKeu4na77n0b2yUuxg9eE7AzuQmCOJBd4wK0An/ijMqqLZ+rDKvUdnrKXILzL5VspKyVew32mTfS3bv+8rLKS8o/RZXLFc67CF8v0N5R04LV/6cO3Xst9fL39gq5XmkNGdnk7u5B9Djp2S9jNpd8n4fL67xF90mS1s6edv/JcTf/VrAPb2k7iLzo/N5tWjZRugHI+Mt7OWS7++dpdm0kMS+OzNF4YAf6DIpCOj1dfNNhaDS6U4+2nKVPnppvkcu0TKChxhxbddk30sKSeKbHbfBjPPB2Nqgs9pqrEYnFzj3b4j5DP/+k14ughz714jDFPPdRI79PE373DMrcI8FkseEHtGFzBog8iamCFMxT/PkVirlLfnN4t8tX5XqWpB8HEHvLjc0ocP9H0bgH8lnwrA230+58INEmN0TedZAJzVbQRv5gnzMyRe9Hvr28ISd79sozOfhjwzdCCOLrh9zTXflA/O4fRrhe0dnixFSG9753IA1lw61+nHvQNkTsm+dzZP3yjbVm3xSkmWZ74fI9e6z+vEAtoJFv6yQMmA5pBRJSQ4Wx/ZSHw+g8XHSIkDz9miq9cPEplvzNhEz0+vAuCao4RNvKT7aD60vDKVFpu2+S0pYTLwd9uZeIfEb/qs5JnaimXXV8boCUC3qEjlUuGY8uqwFEsEDlWyyrFH8vVg2Rvs0s3y2X0dhTacuL0jfl/NwZGNUZLGwN0mnR4PigFoBx5Pfu86ALI/8RK/RjraDoTF5XZqcQQbliAPH3sjZV+WBJOSvxfXUDHWxnWNrCz1xajbr+Hrh58AoNIqYLP20VH0uVWy+OwMOrvKb20uPefz5kAVI6+Dp5AiLUZE+yWhbp9xXYc6xq6rrbi9vDmdSPhIDI3NT0nl4ke6iPHyQmFH+qWJ3Afmy8PbFxTs6xiTRCAw1JYrqxN5N0i2YbcHZjsfn58uNVo+Gydlh/tbbpTvy/zc/dTzAKS55IHU7tZyZxPn6XnSn9e0lVpA720owm/vTRccsGzXeKrFYI4ogu+nQsn7I++5lrfvlQ1x21t7Y26Zdhjdz5V+U1Y2kr03YW0GQK2fNxL+ojB7ClrQXq9Tj8mBnZnZbTjpLml7m4GSHLH9hmO4Q+x5/rxeHlTfPSHu2XFvD6M2NIueal1tLHqKyun2rLjtLi8UA797tjOeP/0AABlkSURBVLiPJ286ngGpYrRf84oYHBf9NIvzu0nm6nObxeD4edSrAIwfcCnu3fL7+LdLokRI3bxIyBguCaOKjOsvcnFPqtRK3OOTZIHzTpUsuGUVpbz60D8AaGslF+y9wu+MxW5z5fz/9fwSgAEj+tNrkZW9GGYD57B6qlTTdqtfV5M7fd4WkvPEre7eJ8bdJ1pc8J8eGAD3yenXZ8jz/datcZyWJZ/nzJf+XzH6ZQAm7B4dqHKuqjvlGtuPxs1nYGBgYGBgYNAIRA8zZVmi2k/tKe5VU/3DnBP32QKHsttsse2D/yEsT9aBH6tR+C1Z4dZmK7ZeMYCXusqKN86ymjecJIHy335ydOAL4eriWBa1TUW7kjx4syVIb+sYWYGkbkmlPEOu2zE0XrT5UMW9584QhuasO2fiQVYk75UIk9bn1rmB9HRrv6mWYg3DQincbYQq33aZuAPi1VrauSS4f+cRImubL629oyoqUe2k3s+BQVIHJfHD+Y6MT1i73tsB7MclHeDBJeMA6LVG2Buf3+dUbq6aRNCksCtLb95G129Ff9Y/LDr4VH4Gf+0m9d+m9nkj5GuJSpGqZMAtqZD+vC77WEfGAxbj/ESB1H9RJeWBGj5B4zqcq6zZ4Qqt3+PqYAWd/+F5stzCGBZa+/F1P/fnaqUUatXVFigdUA3B+69BSOC87flQqzcC0HHJSke+6QdEhyeNPtE6O9QtU+N9IoEwuxA4Ho71efisYPm4vXJs+x5xbSZ+2IbdO6Q/exVIGMjse+MdGZ/bL4zc3MLeAKw7N40+90t4hZ3E40pMDARnR6Ivw1zTZdens1y4E4YuJdmab7p6hIk8IUnCJTJcSfgthmVOubT5/l5DHRmXVIjuziwVJrUitxRdLi7M4HCRWvesbeK+1d7KQFkG65hvx06wQgB8VdqgEhJAdws5dlrWcEfGI/4qwfnjnrFZxJKIymOYKQMDAwMDAwODRiB6mCkLyuMJG29g74bu7OJu+zeDVrT+o4QhuPnlN5zvnr9CfOMdf7Ss07g4p2iig+CVYjOxVbb1PKGPpIl3/udsEm+TFUSqS1YGeyqEFYjfVeqk0ztQKhDQnGDtFN9eVlvrr8yhIleCgm8fLunbT64cg9crtrNnmjAjVQNTmxJh01/tgE0rQPCUS2SvpSvbfgCIf/uOWVK8s6/rx6Dv1oGFUSqwqgmTWNDUUG43Hy//GoBTL5ICeFNy+/DCZtG3NVbF8FM/kfTsuCUbKeslrFtRN+nnzVNGOPFr7+SJXruVrD7XVnpp84H0P7tknzUVFx+o7B9UNThSUPHxfP7GfwGcwpt7Cgo4730p/vjjkcJMjVp8LgADMnfw/UYJrqksFDl4Rjky2oGhaW7RzZLcdiRultgdXVHh3NPdWVhVnxWfElEWjtrT0e1KzMMeEGY7zVWKFxmLrxRKzJi7bZug7x68P5RL4WojDI/PYiQjibDyVY2zDLMDg71fZuFZQ50+vH6NlPrQnloKWtZ1B4YmRFUZfYWFpC6WoPnV//4QCJSe8VdU4neFsuSenB6OjIN/lM+u6ShBzddc8BW/WXMzAB3flbhW//5C3KlWOQY7ljHCbOMn6yXm0C7bse4oH103y9iwiwWX60rr1csnFst/z8tScDXrhFJHxve3SqxmtkeY8KTkCiqGyNj1fB9IdHHFy5yqugub7FuzvsnksZ/pznytdeD5bvVjcPx01T52ZXdj6Xipbm4/Mx/aMI9xXcWT0cE137luAHUobB2pCuhKqe7AS0AnhH2bqrV+TCmVCbyBbM26EThfax35mSECKNMlLNNzqaAMtCJL9aIHfajUFSxlLqWUUEEZSqmM1ihjQX4Zm+55EW9BMShwF/cjJ2VIiHxJJKPDKVorQZkuYVnlD1RoMZqz6EkPlVtNRiC6yhrXA2W6mJ/1bNHTSkU3V2+60ztm9BRExmXb36LcVwJeH93cfehOz5iRsUyXsMy7gIp9FaAgS2eH1dNWPxb1fNFTVOyOxQOzZL7x+8lSvekRc2OxhGXl34iMGz10bzOEbmTGlIxNibowU17gNq31j0qpNGChUupz4Apgptb6QaXUncCdwB1N0Sjb8laewOrBZmYcNiZOLGa0H3cHa/uDh2WV285VzB/WScbR7ff/BoD2C8XaDtmHz7JAlfaTy2DSVQZeXcl8PZPps/by4puFbH26GzmqP9/oj/DhbRoZlXJW+jsvE7/+1DsfI8naz82WcfIq2edrWafBJFoZbSrRWvH7NVjsxM4zxcdfeJJkKkwZ8W+6eyT75Mbs0ZTrUro8ejg9b9sn8jGTtoXJHHvdfMa1dfPl38azUa9kM2saLZoNhzl0Bfmoq2Qwdfxe0nnbWKsKgPhdnsD59YBCkasPD/QhM5n+VUFIH27UK9nLzs6NEMuB9nqdvb423yztf+OFeSRaK11bxhOWSLr1goJsru4qTM7IBGGvXto/jEs2CzMQp4SZs1eWf847jXafCAsQYC98oXrqn8m0WXsipqf+kpLAKjAtsOVDlxtFz8Ztks8y+8o4yk/pQZ9yOz5BXkt6tuGZTZIdVaLl2fm/bUcBkLStGFV1/7PySvrQLyCjdybTZu2K3FgkiJUJYqXtMiv2Z4WPinGToirZZm2z8fyUSQB03DebukKhpA91W7x+0dNM3SmyY7GW+K1aY7usGB1fgqLTHIkPfGidlMNI3xooYVGVJVWaUD2N8FgMJ4fyePBuzQ/5TMUFxYVZWdHuVNE/vW8/ZadJzOkPu4URHZW6DoD1FR1IKrAK9FrZp66EOHLLBomM2M+MyI3FYDncbYWFKjqpP/+09mz9ebi07z9WJuI+v4d7lvwCgDbr5LPtI5K4/1mZU1xWhE+lRf0U70whPk9KlvgspkhpyNWHiYwVlczfNZO3ZmU2mYzVPAhKBYrlBgcg+as8+229nFLuMFI2rrrvt7RTc0O+F3rTMKxTmKKrwferKw5qTGmt84F8632RUmoFkAWcAYyxTnsRmEUTKE2w20JbCxd3epLjUrEFdPYkGjaW5X+WekvvdBPKb2hCArdvHwJA+wXyINLWvn1oHaAXrUlx1rZVznU9Ko5kncbkMdmsYhHDEXozjngqKD2zKWREa6fORedPxYXDneCukq5pV8D+8k8rsFnpr77pB8DaS6Zw3Vahrc9Jlcq1hydKqYR+cV7iEGPzaetB1jsulfF3HEkcHpIr0yinlKef9jKc0SQo6EI261jeaNGCZZRXa0PVhAQ+3SCbGNu/9Ws9v7JOdrOiQh7QvR4Uo/eT4Oqzdq0fOzDZr52BYgdLzlq/+KB92IVs1vJzRlOJaE+sfaZaJSouhQwrONnG3e0lwLWy3TJc1rbbftul2W4N5VomhgIrmPkv26X+TfFFSfiLdoVca9a2lY6R6iGCemq5eWzXGwT2PnNntnU2ILd/749nWRXaB53obEhuK+ysL+bx1D5xhx2XLAbCqg1SwKhHNxfJq+39Ma3zm3ssBiMoiLlqqMG6C58BYGG54r58q77W69K3H4erlBwcLmC7zdxuEoln1iapzWTLF/GxWBVBFb3tdod7gKgU0dP5D0zh1SIJIfjbexcA0DZTjF//tvJAUoQ1d8/KX3VQPW3qsVgV2q8D+7lWeWaM7znSMQDtgPKPl33F4H9ImMhZnURPi/3ynEh2VeAptuYba1H75drZzTMWw4Sd+IvlWZa6er9TSmUcImMPy91XXFGCWigGcHyR9MvSW55xElzssIoS67rJmz1QZI9d+U1m5TX/czGQ/RBwvVXVTfs5IgtPec4tq5DfZMFfpzDuuVqMoCq/Z7gq5w1NdKpXzJRSKgcYBswDOlmGFsB2xA0Y7jtXA1cDJJIc7pSoQqkupoh9tCGTCspJsBgDJQ/CVi9jqT4QVr54EtE11NdoTfJBzX0YTyLUoPOxImOs6CnEvoy16akZi61fxljRUzg0ZGws6mxMKaVSgenAb7XWhSrIctRaa6VU2NGvtZ4KTAVIV5kNiuzy7dvvBGqDrJDtFOSyU7O5YqQEiXb12KxWAtO/E8q27yqr2KXNeCnlrJpt2PssebWX0sO/5JWbM/j3r+NCandZStN0MlrMineLBEne1XME/RZYrktktW6zNY91+4K9fmn/7ed/Zp2Twj+6SCXYEss1VGZZ2+Va8cw+2Vdq6hfiiupzy1y8uoyf+I5Xnm3D2ZOWkdAlSL5aCrDVWb5agvd1ebnTZ3aQri1fn7gETv/+enlfujTMdYWxc9kBn/sLAytLC/ZqotY+bGoZLTl9+eLyuKPnSCav2hJyus+i0eNUUHhIUFRlnldW+E/sHgPAhgukU3w780OYoYCMOvJ6Go4et475du8JUPFWILZdukKp0gB1HzQ/fLBddNHfSfoxdZWs+JO27kcly6Rsc7J16semHothZK1awK/ne1cD8NDJbzD7E5GnR8G8oEZVCWK1Weagw26rNMa4rkPxai9LmBO5sVgbtK6+GncSeoIYYIsBASjyWfv7WeRFZbbsk+ZJjEd5Q/cvrJOeRlrGMH1o6627TeDBHuwJcVse6tm7JaEkL0X668slAxj4k5SA0Na8M67rUHC58OrKZtdTe4zpn1cy6ejTraMy73xUIv1U5k9HWdNM8qZAdfhtPmER9/rlmXHBC78HIL4YtOUtcaUkB2Qk8mOxWiB66EWYdNREu/VAoHxMN0+qU+X82pslQeCbKVOrXSL0ZjIulbt60dyGuvds1Kk0glIqDjGkXtVav20d3qGU6mJ93gXY2aAWRAn82s8SPZuLz07l7ElWFh0JlGuZUPwSENpqZfRrP0uYQ2d6hJWvXJfaA6PVwpaxpj60XluweFHjcTAZW7ueQuzLaMZiDI1Ff+w+MyD2x2JTQumDpAEqoaBeBPZqrX8bdPzvwJ6gAPRMrfXva7tWusrUI9XJIccaag3azIQrtycXviOxN8clbQTg5o3nUvE7WTnphVbsQbjVtrWa1H4/y/iBOOLppwLtWKOXEEe8E2hXQenfm1zGoHgKm3mxmbM9EyU+Kq7Ez30P/wfAKdJW5E+kUstvcPNCiWPIuUCC7gsvHsX+ntZ+TP9cjNaapaXfEKfjapRvo17JJtZQof+/vfOPreos4/jnvbc/1kL5WTLLL1tKNx1s7gcwtpAQE4KKS7YlxhizaaYzMWyZ0yzBSTSLuj/QyKL/EN3UxG3q3FwczrEmLJsZjvFLGTAISwcodAUsBSmUFnr7+sfznntvL7f31zn33NvT55PctD3tPef93vd5T5/zvM/7vIM57+IF6UtfGp0ld6RmtuSdmufk0ensYAPTHpAo1XBPKrHVI/PJJXbNNVJQDog1SGQjMSAriGqp4/pYahuSTI1dHDhlrc2Z+Fqqnda0fIwrbRLx7lkhffmjb/wOgCX1J5nknope6O8AYNPTd9OyTYrtxY/L/ShXuQprbfh2mhltzLbsPVtE0h0bvGspN/7gPQC+0iz5e99vkwT0mnlzsV6OlYvWJfovcGB4e2XGYha8fKCLn5e/P7cwzrU7JYRRu1NypkYGBlL3I7dVUKJX8h1jjY2MuHzNWH29jMXBt3PqC3QslkByX79PtgIw9xdHuX+WJNm/fFaiO4eXSGQjPm0qZrL8k7Vucc/wuf+lxmIOjeUci3lx9hlzbY9NncKV+bKQad5GSTw/sVz0XHh9AVPWiR3Ezkj05sqJ7rwaQ7FTT4e7D8amuL5L2xLGtEt+0+k7ZvDgdzYD8JM37wKg4yGJro6suJn4Ltmaxl5xPm4sxoEr/whXo7edjLsfmHg8WfKn6yGJGO554CkAPkok+MyWbwNw/cOSO5arVIyprUv9D3HbPo0MDOQthbDDvsF525f36aYQZ2oF8Dawn1SRhu8heVN/AuYD/0ZKI/TlOle2DzTPxeVr+t5Ljrgb8D33L+ara18D4BP1ksL1+Mav0/IHMYzEuXOpczi8TvS4vWUuu3mLyUylfZHc+Mz7S5nCDGpWbGX7tjiXGSTB8MySNAZQu8pLfDSzr02ec2i+hKHr942uQpw405dMvgTYtuMSK+/pZjJyo29fdIkfPz6Tn963hP28y8y2c/Qdnc5lhui354q+gV9Fnjoz3iarPfdIXZPYZZj1vCTEek5Sep/HZ0oVca9C70h/f0l92EAjfZzea63NuaFY0XaajtdmVx3duAT5syvbaDwt7a/bI6s0Excupqb8slTmDd1Og8TdFHsfXMa6x34PwLSYOMxrX5E6VR3r9yZr53iErjGPrcanyxjr+bIk0Q/MtkyVBVHMeksWjyS6e5KJzPEZLqfanXP45KlR/Rj6WCwB7+Hl5DdlWn7t2r+wvEHqC/3wuPwT/s8z8kAw4+AFXn/l2VHvr5qxWADefXVkcTsffE0cktZ2eaBpqpN70aln2tixYdOo942rseg5XPX1nHjkVgAaTot9Nv9V6mYl+s4mFyV4VFJj+grRxKelzbdtFIdp9RRJBXnj/CL2rpwm7c/Y6DrzHPIBxEf9X4Tc03xecnqhzlQhq/m2wZgx5zJaQHhMM82sQgoPdm71PlBJYuh8cU76B5rTYKqVFbc3sMp8Ifmzp/Hnpp7bWEnnOymjGa8U0ocAW+1LFd7Ho3SibqcQfY06FnUsjhcmgsYgCbUC+nU3DdDZmT18matitqmtS5Yx8Kpcm2aJVtx43wGWN0hY9re9siP25JNjj9FsSyGDJKvGtGWYyeNFRquSu113HU0eq+mScyTV5ljuGRRF96FHLJ6s3ZPolbE365ey9N/U1jDihZazfB7JKRMX5alIHzryht+9pHSXzIn72vRib2oqK2utk9E2W9UaCyA+RaZP4kPw5EFJIL10SJ4iO56XciVbjrxbnRpj8WRfeSVVWjol6t2zuoXm3WKP9qybSonH8RbkeNO03j5qVTkW8+DVshty20q+evomBpolWrX/7xKRWrhV9qr7267XqrMP8+GVW3H9RMzQvFOii8fc4jQzLH9zZMOm8anRw91vRoaGmP+CWyDjFg0MOxvu7P5XVWn0okk18+by0SNuWt1Im88k5N6y94sLGRnIsjekl76TMeXXeWJPWTXq3nyKoiiKoig+yJszFSS55k2zeaejjrkcjPiC+QAkXISm67lbWHWdJIIeW+ZWinxuKfVbZO+hoPYwK3Te1JfGYhgrydsHhWgMQl9yV/LBQQoiwP0St9qX9lhrl+T6m4I1OpvMLA4IqSdeb1+5WFNTsvBlOffwqjY7NUsWY3dLIdZ4hySQ+t3fK1SNXkJs+t6duexwnI3FbGTeM7vX3cmcDZKA7uU7ZlsoUiyBjsUcx7KS0U/2zk9h3pGFEseevAOA1vXb8yjITbWNxXTbzVmOoAjC0Nj1lBSnfvXejQA82iplkuIzZ5Doc7tDBOnHZNhGoRo1MqUoiqIoiuKD8CNT8dVln7usWdDK8JFjY/4+c5VUIW0p1gMvNFciqMiZnCzlUZdLY7H6ykUp+qC4p+HxqLFcduoRa2xM5e9d9cv4VVG32KRJjFy8OOb5qlFj0Iz3sRhramKkv3/M30dhLJZDo9ppiomgMdQEdEAq0wZYdTRb5+RypPxcqxgK1ViIE1WwAaY5xuXWWO4+LOb65UI1Xs2YjhTZEzxzOVL5rhUUUe/HcuvL5WT4uVYxqMbiqEY7zbxG1DTqNJ+iKIqiKIoPQp3mM8b8F7gI9IZ20dJpZnQ7P26tnZXvTcaYfuBw2VoVLEVrHOd9CNHXWKidTgSNOharBx2LYzBBNEZ6LELIzhSAMWZ3vnnyaqDUdo4XfRB9jX7aqRqrh6jbKURfo9pp+d4bJlG3Uyi9rTrNpyiKoiiK4gN1phRFURRFUXxQCWfqVxW4ZimU2s7xog+ir9FPO1Vj9RB1O4Xoa1Q7Ld97wyTqdgoltjX0nClFURRFUZQoodN8iqIoiqIoPgjNmTLGfNYYc9gY02WM+W5Y182HMWaeMeZNY8xBY8z7xphvueNPGGO6jTF73WtNAedSjRUiKI3Vqg+ir1HtVDVmnCfS+tx7VGOFCFIjANbasr+AOPAhsACoA94Dbgjj2gW0rQW41X3fBHwA3AA8ATymGieOxmrWNxE0qp2qxomiTzVGR6P3CisytQzostYesdZeBv4I3B3StXNire2x1v7Tfd8PHALmlHAq1VhBAtJYtfog+hrVTosi6hqjrg9UY0UJUCMQ3jTfHOB42s8n8NHocmGMaQVuAXa4Qw8bY/YZY35jjJme5+2qsUrwoXFc6IPoa1Q7nfAao64PVGPV4FMjoAnoSYwxk4E/A49aa88Dm4B24GagB/hZBZsXCKpRNY4Hoq4PVCMR0Bh1faAaKUJjWM5UNzAv7ee57lhVYIypRT7M5621LwNYa09ZaxPW2hHgaSRcmQvVWGEC0FjV+iD6GtVOVaMj6vpANVacgDQC4TlTu4AOY0ybMaYO+BKwOaRr58QYY4BfA4estRvTjrek/dm9wIE8p1KNFSQgjVWrD6KvUe00iWqMvj5QjRUlQI1CsRnrpb6ANUi2/IfA+rCuW0C7VgAW2Afsda81wLPAfnd8M9CiGqOvsVr1TQSNaqeqcSLpU43R0Wit1QroiqIoiqIoftAEdEVRFEVRFB+oM6UoiqIoiuIDdaYURVEURVF8oM6UoiiKoiiKD9SZUhRFURRF8YE6U4qiKIqiKD5QZ0pRFEVRFMUH6kwpiqIoiqL44P83OKdEeDN9FwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x144 with 20 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Compare original images with their reconstructions.\n",
    "f, a = plt.subplots(2, 10, figsize=(10, 2))\n",
    "for i in range(examples_to_show):\n",
    "    a[0][i].imshow(np.reshape(mnist.test.images[i], (28, 28)))\n",
    "    a[1][i].imshow(np.reshape(encode_decode[i], (28, 28)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, the reconstructions were successful. It can be seen that some noise was added to the image."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### References:\n",
    "- https://en.wikipedia.org/wiki/Autoencoder\n",
    "- http://ufldl.stanford.edu/tutorial/unsupervised/Autoencoders/\n",
    "- https://cs.stanford.edu/~quocle/tutorial2.pdf\n",
    "- https://gist.github.com/hussius/1534135a419bb0b957b9\n",
    "- http://www.kdnuggets.com/2015/03/deep-learning-curse-dimensionality-autoencoders.html/\n",
    "- https://www.youtube.com/watch?v=xTU79Zs4XKY\n",
    "- Reducing the Dimensionality of Data with Neural Networks, G. E. Hinton, R. R. Salakhutdinov, Science  28 Jul 2006, Vol. 313, Issue 5786, pp. 504-507, DOI: 10.1126/science.1127647 - http://science.sciencemag.org/content/313/5786/504.full"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  },
  "widgets": {
   "state": {},
   "version": "1.1.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}