Problem_1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Problem 1: Basics of Neural Networks\n",
    "* <b>Learning Objective:</b> In the entrance exam, we asked you to implement a K-NN classifier to classify some tiny images extracted from CIFAR-10 dataset. Probably many of you noticed that the performances were quite bad. In this problem, you are going to implement a basic multi-layer fully connected neural network to perform the same classification task.\n",
    "* <b>Provided Code:</b> We provide the skeletons of classes you need to complete. Forward checking and gradient checkings are provided for verifying your implementation as well.\n",
    "* <b>TODOs:</b> You are asked to implement the forward passes and backward passes for standard layers and loss functions, various widely-used optimizers, and part of the training procedure. And finally we want you to train a network from scratch on your own."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The autoreload extension is already loaded. To reload it, use:\n",
      "  %reload_ext autoreload\n"
     ]
    }
   ],
   "source": [
    "from lib.fully_conn import *\n",
    "from lib.layer_utils import *\n",
    "from lib.grad_check import *\n",
    "from lib.datasets import *\n",
    "from lib.optim import *\n",
    "from lib.train import *\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "%matplotlib inline\n",
    "plt.rcParams['figure.figsize'] = (10.0, 8.0) # set default size of plots\n",
    "plt.rcParams['image.interpolation'] = 'nearest'\n",
    "plt.rcParams['image.cmap'] = 'gray'\n",
    "\n",
    "# for auto-reloading external modules\n",
    "# see http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython\n",
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading the data (CIFAR-10)\n",
    "Run the following code block to load in the properly splitted CIFAR-10 data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Name: data_train Shape: (49000, 3, 32, 32)\n",
      "Name: data_val Shape: (1000, 3, 32, 32)\n",
      "Name: data_test Shape: (1000, 3, 32, 32)\n",
      "Name: labels_train Shape: (49000,)\n",
      "Name: labels_val Shape: (1000,)\n",
      "Name: labels_test Shape: (1000,)\n"
     ]
    }
   ],
   "source": [
    "data = CIFAR10_data()\n",
    "for k, v in data.iteritems():\n",
    "    print \"Name: {} Shape: {}\".format(k, v.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Implement Standard Layers\n",
    "You will now implement all the following standard layers commonly seen in a fully connected neural network. Please refer to the file layer_utils.py under the directory lib. Take a look at each class skeleton, and we will walk you through the network layer by layer. We provide results of some examples we pre-computed for you for checking the forward pass, and also the gradient checking for the backward pass."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## FC Forward\n",
    "In the class skeleton \"fc\", please complete the forward pass in function \"forward\", the input to the fc layer may not be of dimension (batch size, features size), it could be an image or any higher dimensional data. Make sure that you handle this dimensionality issue. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Difference:  2.485392759674788e-09\n"
     ]
    }
   ],
   "source": [
    "# Test the fc forward function\n",
    "input_bz = 3\n",
    "input_dim = (6, 5, 4)\n",
    "output_dim = 4\n",
    "\n",
    "input_size = input_bz * np.prod(input_dim)\n",
    "weight_size = output_dim * np.prod(input_dim)\n",
    "\n",
    "single_fc = fc(np.prod(input_dim), output_dim, init_scale=0.02, name=\"fc_test\")\n",
    "\n",
    "x = np.linspace(-0.1, 0.5, num=input_size).reshape(input_bz, *input_dim)\n",
    "w = np.linspace(-0.2, 0.3, num=weight_size).reshape(np.prod(input_dim), output_dim)\n",
    "b = np.linspace(-0.3, 0.1, num=output_dim)\n",
    "\n",
    "single_fc.params[single_fc.w_name] = w\n",
    "single_fc.params[single_fc.b_name] = b\n",
    "\n",
    "out = single_fc.forward(x)\n",
    "\n",
    "correct_out = np.array([[0.70157129, 0.83483484, 0.96809839, 1.10136194],\n",
    "                        [1.86723094, 2.02561647, 2.18400199, 2.34238752],\n",
    "                        [3.0328906,  3.2163981,  3.3999056,  3.5834131]])\n",
    "\n",
    "# Compare your output with the above pre-computed ones. \n",
    "# The difference should not be larger than 1e-8\n",
    "print \"Difference: \", rel_error(out, correct_out)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## FC Backward\n",
    "Please complete the function \"backward\" as the backward pass of the fc layer. Follow the instructions in the comments to store gradients into the predefined dictionaries in the attributes of the class. Parameters of the layer are also stored in the predefined dictionary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(10, 2, 2, 3)\n",
      "(10, 2, 2, 3)\n",
      "(12, 10)\n",
      "(12, 10)\n",
      "(10,)\n",
      "(10,)\n",
      "dx Error:  9.361394506812859e-10\n",
      "dw Error:  8.884347473063264e-10\n",
      "db Error:  5.074792363899382e-11\n"
     ]
    }
   ],
   "source": [
    "# Test the fc backward function\n",
    "x = np.random.randn(10, 2, 2, 3)\n",
    "w = np.random.randn(12, 10)\n",
    "b = np.random.randn(10)\n",
    "dout = np.random.randn(10, 10)\n",
    "\n",
    "single_fc = fc(np.prod(x.shape[1:]), 10, init_scale=5e-2, name=\"fc_test\")\n",
    "single_fc.params[single_fc.w_name] = w\n",
    "single_fc.params[single_fc.b_name] = b\n",
    "\n",
    "dx_num = eval_numerical_gradient_array(lambda x: single_fc.forward(x), x, dout)\n",
    "dw_num = eval_numerical_gradient_array(lambda w: single_fc.forward(x), w, dout)\n",
    "db_num = eval_numerical_gradient_array(lambda b: single_fc.forward(x), b, dout)\n",
    "\n",
    "out = single_fc.forward(x)\n",
    "dx = single_fc.backward(dout)\n",
    "dw = single_fc.grads[single_fc.w_name]\n",
    "db = single_fc.grads[single_fc.b_name]\n",
    "\n",
    "print dx_num.shape\n",
    "print dx.shape\n",
    "print dw_num.shape\n",
    "print dw.shape\n",
    "print db_num.shape\n",
    "print db.shape\n",
    "\n",
    "# The error should be around 1e-10\n",
    "print \"dx Error: \", rel_error(dx_num, dx)\n",
    "print \"dw Error: \", rel_error(dw_num, dw)\n",
    "print \"db Error: \", rel_error(db_num, db)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ReLU Forward\n",
    "In the class skeleton \"relu\", please complete the forward pass."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Difference:  5.000000050123796e-09\n"
     ]
    }
   ],
   "source": [
    "# Test the relu forward function\n",
    "x = np.linspace(-1.0, 1.0, num=12).reshape(3, 4)\n",
    "relu_f = relu(name=\"relu_f\")\n",
    "\n",
    "out = relu_f.forward(x)\n",
    "\n",
    "correct_out = np.array([[0.,          0.,        0.,         0.        ],\n",
    "                        [0.,          0.,        0.09090909, 0.27272727],\n",
    "                        [0.45454545, 0.63636364, 0.81818182, 1.        ]])\n",
    "\n",
    "# Compare your output with the above pre-computed ones. \n",
    "# The difference should not be larger than 1e-8\n",
    "print \"Difference: \", rel_error(out, correct_out)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ReLU Backward\n",
    "Please complete the backward pass of the class relu."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dx Error:  3.2756244751759753e-12\n"
     ]
    }
   ],
   "source": [
    "# Test the relu backward function\n",
    "x = np.random.randn(10, 10)\n",
    "dout = np.random.randn(*x.shape)\n",
    "relu_b = relu(name=\"relu_b\")\n",
    "\n",
    "dx_num = eval_numerical_gradient_array(lambda x: relu_b.forward(x), x, dout)\n",
    "\n",
    "out = relu_b.forward(x)\n",
    "dx = relu_b.backward(dout)\n",
    "\n",
    "# The error should not be larger than 1e-10\n",
    "print \"dx Error: \", rel_error(dx_num, dx)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dropout Forward\n",
    "In the class \"dropout\", please complete the forward pass. Remember that the dropout is only applied during training phase, you should pay attention to this while implementing the function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----------------------------------------------------------------\n",
      "Dropout p =  0.25\n",
      "Mean of input:  4.995631421675257\n",
      "Mean of output during training time:  1.2528526583236463\n",
      "Mean of output during testing time:  4.995631421675257\n",
      "Fraction of output set to zero during training time:  0.7494\n",
      "Fraction of output set to zero during testing time:  0.0\n",
      "----------------------------------------------------------------\n",
      "Dropout p =  0.5\n",
      "Mean of input:  4.995631421675257\n",
      "Mean of output during training time:  2.4904757497767895\n",
      "Mean of output during testing time:  4.995631421675257\n",
      "Fraction of output set to zero during training time:  0.5018\n",
      "Fraction of output set to zero during testing time:  0.0\n",
      "----------------------------------------------------------------\n",
      "Dropout p =  0.75\n",
      "Mean of input:  4.995631421675257\n",
      "Mean of output during training time:  3.768783741248381\n",
      "Mean of output during testing time:  4.995631421675257\n",
      "Fraction of output set to zero during training time:  0.2449\n",
      "Fraction of output set to zero during testing time:  0.0\n",
      "----------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "x = np.random.randn(100, 100) + 5.0\n",
    "\n",
    "print \"----------------------------------------------------------------\"\n",
    "for p in [0.25, 0.50, 0.75]:\n",
    "    dropout_f = dropout(p)\n",
    "    out = dropout_f.forward(x, True)\n",
    "    out_test = dropout_f.forward(x, False)\n",
    "\n",
    "    print \"Dropout p = \", p\n",
    "    print \"Mean of input: \", x.mean()\n",
    "    print \"Mean of output during training time: \", out.mean()\n",
    "    print \"Mean of output during testing time: \", out_test.mean()\n",
    "    print \"Fraction of output set to zero during training time: \", (out == 0).mean()\n",
    "    print \"Fraction of output set to zero during testing time: \", (out_test == 0).mean()\n",
    "    print \"----------------------------------------------------------------\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dropout Backward\n",
    "Please complete the backward pass. Again remember that the dropout is only applied during training phase, handle this in the backward pass as well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dx relative error:  1.8928961546001812e-11\n"
     ]
    }
   ],
   "source": [
    "x = np.random.randn(5, 5) + 5\n",
    "dout = np.random.randn(*x.shape)\n",
    "\n",
    "p = 0.75\n",
    "dropout_b = dropout(p, seed=100)\n",
    "out = dropout_b.forward(x, True)\n",
    "dx = dropout_b.backward(dout)\n",
    "dx_num = eval_numerical_gradient_array(lambda xx: dropout_b.forward(xx, True), x, dout)\n",
    "\n",
    "# The error should not be larger than 1e-9\n",
    "print 'dx relative error: ', rel_error(dx, dx_num)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Testing cascaded layers: FC + ReLU\n",
    "Please find the TestFCReLU function in fully_conn.py under lib directory. <br />\n",
    "You only need to complete few lines of code in the TODO block. <br />\n",
    "Please design an FC --> ReLU two-layer-mini-network where the parameters of them match the given x, w, and b <br />\n",
    "Please insert the corresponding names you defined for each layer to param_name_w, and param_name_b respectively. <br />\n",
    "Here you only modify the param_name part, the _w, and _b are automatically assigned during network setup "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dx error:  4.6047767656240917e-10\n",
      "dw error:  1.3014199252124863e-10\n",
      "db error:  3.2756170289757754e-12\n"
     ]
    }
   ],
   "source": [
    "x = np.random.randn(2, 3, 4)  # the input features\n",
    "w = np.random.randn(12, 10)   # the weight of fc layer\n",
    "b = np.random.randn(10)       # the bias of fc layer\n",
    "dout = np.random.randn(2, 10) # the gradients to the output, notice the shape\n",
    "\n",
    "tiny_net = TestFCReLU()\n",
    "\n",
    "tiny_net.net.assign(\"fc1_w\", w)\n",
    "tiny_net.net.assign(\"fc1_b\", b)\n",
    "\n",
    "out = tiny_net.forward(x)\n",
    "dx = tiny_net.backward(dout)\n",
    "\n",
    "dw = tiny_net.net.get_grads(\"fc1_w\")\n",
    "db = tiny_net.net.get_grads(\"fc1_b\")\n",
    "\n",
    "dx_num = eval_numerical_gradient_array(lambda x: tiny_net.forward(x), x, dout)\n",
    "dw_num = eval_numerical_gradient_array(lambda w: tiny_net.forward(x), w, dout)\n",
    "db_num = eval_numerical_gradient_array(lambda b: tiny_net.forward(x), b, dout)\n",
    "\n",
    "# The errors should not be larger than 1e-7\n",
    "print \"dx error: \", rel_error(dx_num, dx)\n",
    "print \"dw error: \", rel_error(dw_num, dw)\n",
    "print \"db error: \", rel_error(db_num, db)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SoftMax Function and Loss Layer\n",
    "In the layer_utils.py, please first complete the function softmax, which will be use in the function cross_entropy. Please refer to the lecture slides of the mathematical expressions of the cross entropy loss function, and complete its forward pass and backward pass."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cross Entropy Loss:  1.609458464677557\n",
      "dx error:  3.285990249894269e-09\n"
     ]
    }
   ],
   "source": [
    "num_classes, num_inputs = 5, 50\n",
    "x = 0.001 * np.random.randn(num_inputs, num_classes)\n",
    "y = np.random.randint(num_classes, size=num_inputs)\n",
    "\n",
    "test_loss = cross_entropy()\n",
    "\n",
    "dx_num = eval_numerical_gradient(lambda x: test_loss.forward(x, y), x, verbose=False)\n",
    "\n",
    "loss = test_loss.forward(x, y)\n",
    "dx = test_loss.backward()\n",
    "\n",
    "# Test softmax_loss function. Loss should be around 1.609\n",
    "# and dx error should be at the scale of 1e-8 (or smaller)\n",
    "print \"Cross Entropy Loss: \", loss\n",
    "print \"dx error: \", rel_error(dx_num, dx)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test a Small Fully Connected Network\n",
    "Please find the SmallFullyConnectedNetwork function in fully_conn.py under lib directory. <br />\n",
    "Again you only need to complete few lines of code in the TODO block. <br />\n",
    "Please design an FC --> ReLU --> FC --> ReLU network where the shapes of parameters match the given shapes <br />\n",
    "Please insert the corresponding names you defined for each layer to param_name_w, and param_name_b respectively. <br />\n",
    "Here you only modify the param_name part, the _w, and _b are automatically assigned during network setup "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Testing initialization ... \n",
      "Passed!\n",
      "Testing test-time forward pass ... \n",
      "Passed!\n",
      "Testing the loss ... Passed!\n",
      "Testing the gradients (error should be no larger than 1e-7) ...\n",
      "fc1_b relative error: 2.85e-09\n",
      "fc1_w relative error: 5.01e-09\n",
      "fc2_b relative error: 4.33e-07\n",
      "fc2_w relative error: 2.76e-09\n"
     ]
    }
   ],
   "source": [
    "model = SmallFullyConnectedNetwork()\n",
    "loss_func = cross_entropy()\n",
    "\n",
    "N, D, = 4, 4  # N: batch size, D: input dimension\n",
    "H, C  = 30, 7 # H: hidden dimension, C: output dimension\n",
    "std = 0.02\n",
    "x = np.random.randn(N, D)\n",
    "y = np.random.randint(C, size=N)\n",
    "\n",
    "print \"Testing initialization ... \"\n",
    "w1_std = abs(model.net.get_params(\"fc1_w\").std() - std)\n",
    "b1 = model.net.get_params(\"fc1_b\").std()\n",
    "w2_std = abs(model.net.get_params(\"fc2_w\").std() - std)\n",
    "b2 = model.net.get_params(\"fc2_b\").std()\n",
    "\n",
    "assert w1_std < std / 10, \"First layer weights do not seem right\"\n",
    "assert np.all(b1 == 0), \"First layer biases do not seem right\"\n",
    "assert w2_std < std / 10, \"Second layer weights do not seem right\"\n",
    "assert np.all(b2 == 0), \"Second layer biases do not seem right\"\n",
    "print \"Passed!\"\n",
    "\n",
    "print \"Testing test-time forward pass ... \"\n",
    "w1 = np.linspace(-0.7, 0.3, num=D*H).reshape(D, H)\n",
    "w2 = np.linspace(-0.3, 0.4, num=H*C).reshape(H, C)\n",
    "b1 = np.linspace(-0.1, 0.9, num=H)\n",
    "b2 = np.linspace(-0.9, 0.1, num=C)\n",
    "\n",
    "model.net.assign(\"fc1_w\", w1)\n",
    "model.net.assign(\"fc1_b\", b1)\n",
    "model.net.assign(\"fc2_w\", w2)\n",
    "model.net.assign(\"fc2_b\", b2)\n",
    "\n",
    "feats = np.linspace(-5.5, 4.5, num=N*D).reshape(D, N).T\n",
    "scores = model.forward(feats)\n",
    "correct_scores = np.asarray([[4.20670862, 4.87188359, 5.53705856, 6.20223352, 6.86740849, 7.53258346, 8.19775843],\n",
    "                             [4.74826036, 5.35984681, 5.97143326, 6.58301972, 7.19460617, 7.80619262, 8.41777907],\n",
    "                             [5.2898121,  5.84781003, 6.40580797, 6.96380591, 7.52180384, 8.07980178, 8.63779971],\n",
    "                             [5.83136384, 6.33577326, 6.84018268, 7.3445921,  7.84900151, 8.35341093, 8.85782035]])\n",
    "scores_diff = np.sum(np.abs(scores - correct_scores))\n",
    "assert scores_diff < 1e-6, \"Your implementation might went wrong!\"\n",
    "print \"Passed!\"\n",
    "\n",
    "print \"Testing the loss ...\",\n",
    "y = np.asarray([0, 5, 1, 4])\n",
    "loss = loss_func.forward(scores, y)\n",
    "dLoss = loss_func.backward()\n",
    "correct_loss = 2.90181552716\n",
    "assert abs(loss - correct_loss) < 1e-10, \"Your implementation might went wrong!\"\n",
    "print \"Passed!\"\n",
    "\n",
    "print \"Testing the gradients (error should be no larger than 1e-7) ...\"\n",
    "din = model.backward(dLoss)\n",
    "for layer in model.net.layers:\n",
    "    if not layer.params:\n",
    "        continue\n",
    "    for name in sorted(layer.grads):\n",
    "        f = lambda _: loss_func.forward(model.forward(feats), y)\n",
    "        grad_num = eval_numerical_gradient(f, layer.params[name], verbose=False)\n",
    "        print '%s relative error: %.2e' % (name, rel_error(grad_num, layer.grads[name]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test a Fully Connected Network regularized with Dropout\n",
    "Please find the DropoutNet function in fully_conn.py under lib directory. <br />\n",
    "For this part you don't need to design a new network, just simply run the following test code <br />\n",
    "If something goes wrong, you might want to double check your dropout implementation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dropout p = 0.0\n",
      "Loss (should be ~2.30) :  2.3025850929940455\n",
      "Error of gradients should be no larger than 1e-5\n",
      "fc1_b relative error: 0.0\n",
      "fc1_w relative error: 0.0\n",
      "fc2_b relative error: 0.0\n",
      "fc2_w relative error: 0.0\n",
      "fc3_b relative error: 0.0\n",
      "fc3_w relative error: 0.0\n",
      "\n",
      "Dropout p = 0.25\n",
      "Loss (should be ~2.30) :  2.302596685860918\n",
      "Error of gradients should be no larger than 1e-5\n",
      "fc1_b relative error: 6.28835801155e-08\n",
      "fc1_w relative error: 3.19420267916e-06\n",
      "fc2_b relative error: 4.3790712705e-08\n",
      "fc2_w relative error: 4.89500337318e-06\n",
      "fc3_b relative error: 4.32061646493e-10\n",
      "fc3_w relative error: 2.00387597884e-08\n",
      "\n",
      "Dropout p = 0.5\n",
      "Loss (should be ~2.30) :  2.3035088317763126\n",
      "Error of gradients should be no larger than 1e-5\n",
      "fc1_b relative error: 1.3676465955e-07\n",
      "fc1_w relative error: 2.10078788437e-06\n",
      "fc2_b relative error: 5.6554164989e-07\n",
      "fc2_w relative error: 8.32295727183e-05\n",
      "fc3_b relative error: 2.6061795162e-10\n",
      "fc3_w relative error: 3.23726968314e-07\n",
      "\n"
     ]
    }
   ],
   "source": [
    "N, D, C = 3, 15, 10\n",
    "X = np.random.randn(N, D)\n",
    "y = np.random.randint(C, size=(N,))\n",
    "seed = 123\n",
    "\n",
    "for dropout_p in [0., 0.25, 0.5]:\n",
    "    print \"Dropout p =\", dropout_p\n",
    "    model = DropoutNet(dropout_p=dropout_p, seed=seed)\n",
    "    loss_func = cross_entropy()\n",
    "    output = model.forward(X, True)\n",
    "    loss = loss_func.forward(output, y)\n",
    "    dLoss = loss_func.backward()\n",
    "    dX = model.backward(dLoss)\n",
    "    grads = model.net.grads\n",
    "    print \"Loss (should be ~2.30) : \", loss\n",
    "\n",
    "    print \"Error of gradients should be no larger than 1e-5\"\n",
    "    for name in sorted(model.net.params):\n",
    "        f = lambda _: loss_func.forward(model.forward(X, True), y)\n",
    "        grad_num = eval_numerical_gradient(f, model.net.params[name], verbose=False, h=1e-5)\n",
    "        print \"{} relative error: {}\".format(name, rel_error(grad_num, grads[name]))\n",
    "    print"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training a Network\n",
    "In this section, we defined a TinyNet class for you to fill in the TODO block in fully_conn.py.\n",
    "* Here please design a two layer fully connected network for this part.\n",
    "* Please read the train.py under lib directory carefully and complete the TODO blocks in the train_net function first.\n",
    "* In addition, read how the SGD function is implemented in optim.py, you will be asked to complete three other optimization methods in the later sections."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 249,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Arrange the data\n",
    "data_dict = {\n",
    "    \"data_train\": (data[\"data_train\"], data[\"labels_train\"]),\n",
    "    \"data_val\": (data[\"data_val\"], data[\"labels_val\"]),\n",
    "    \"data_test\": (data[\"data_test\"], data[\"labels_test\"])\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 273,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "model = TinyNet()\n",
    "loss_f = cross_entropy()\n",
    "optimizer = SGD(model.net, 1e-4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Now train the network to achieve at least 50% validation accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 274,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 1 / 49000) loss: 20.5314490381\n",
      "(Iteration 11 / 49000) loss: 12.3946824624\n",
      "(Iteration 21 / 49000) loss: 10.2958543512\n",
      "(Iteration 31 / 49000) loss: 9.90443170342\n",
      "(Iteration 41 / 49000) loss: 8.3509769735\n",
      "(Iteration 51 / 49000) loss: 8.16887984032\n",
      "(Iteration 61 / 49000) loss: 7.67814459131\n",
      "(Iteration 71 / 49000) loss: 7.8981497033\n",
      "(Iteration 81 / 49000) loss: 7.37788972838\n",
      "(Iteration 91 / 49000) loss: 6.43558703172\n",
      "(Epoch 1 / 500) Training Accuracy: 0.242387755102, Validation Accuracy: 0.244\n",
      "(Iteration 101 / 49000) loss: 6.3959914497\n",
      "(Iteration 111 / 49000) loss: 5.88924744308\n",
      "(Iteration 121 / 49000) loss: 6.64528777199\n",
      "(Iteration 131 / 49000) loss: 5.81271177765\n",
      "(Iteration 141 / 49000) loss: 5.69307233818\n",
      "(Iteration 151 / 49000) loss: 5.91306052648\n",
      "(Iteration 161 / 49000) loss: 5.23652011975\n",
      "(Iteration 171 / 49000) loss: 5.74429239124\n",
      "(Iteration 181 / 49000) loss: 5.36593824899\n",
      "(Iteration 191 / 49000) loss: 5.0073690885\n",
      "(Epoch 2 / 500) Training Accuracy: 0.273367346939, Validation Accuracy: 0.277\n",
      "(Iteration 201 / 49000) loss: 5.14230312947\n",
      "(Iteration 211 / 49000) loss: 4.87461608187\n",
      "(Iteration 221 / 49000) loss: 4.99146662684\n",
      "(Iteration 231 / 49000) loss: 5.07456032004\n",
      "(Iteration 241 / 49000) loss: 4.86512617789\n",
      "(Iteration 251 / 49000) loss: 4.54914350544\n",
      "(Iteration 261 / 49000) loss: 4.78722312252\n",
      "(Iteration 271 / 49000) loss: 4.76766159513\n",
      "(Iteration 281 / 49000) loss: 4.10321190922\n",
      "(Iteration 291 / 49000) loss: 4.70823305478\n",
      "(Epoch 3 / 500) Training Accuracy: 0.289612244898, Validation Accuracy: 0.3\n",
      "(Iteration 301 / 49000) loss: 4.08530292965\n",
      "(Iteration 311 / 49000) loss: 3.99505366371\n",
      "(Iteration 321 / 49000) loss: 4.42481709793\n",
      "(Iteration 331 / 49000) loss: 4.23690583801\n",
      "(Iteration 341 / 49000) loss: 3.86233259669\n",
      "(Iteration 351 / 49000) loss: 4.07893100882\n",
      "(Iteration 361 / 49000) loss: 4.09604416885\n",
      "(Iteration 371 / 49000) loss: 4.11943229847\n",
      "(Iteration 381 / 49000) loss: 3.83867216445\n",
      "(Iteration 391 / 49000) loss: 3.58305155801\n",
      "(Epoch 4 / 500) Training Accuracy: 0.306183673469, Validation Accuracy: 0.302\n",
      "(Iteration 401 / 49000) loss: 3.50666166165\n",
      "(Iteration 411 / 49000) loss: 3.67989223957\n",
      "(Iteration 421 / 49000) loss: 3.43773685849\n",
      "(Iteration 431 / 49000) loss: 3.57953302959\n",
      "(Iteration 441 / 49000) loss: 3.59043890786\n",
      "(Iteration 451 / 49000) loss: 3.48376730331\n",
      "(Iteration 461 / 49000) loss: 3.50061263664\n",
      "(Iteration 471 / 49000) loss: 3.32515342125\n",
      "(Iteration 481 / 49000) loss: 3.20528116787\n",
      "(Epoch 5 / 500) Training Accuracy: 0.319551020408, Validation Accuracy: 0.313\n",
      "(Iteration 491 / 49000) loss: 3.46899543826\n",
      "(Iteration 501 / 49000) loss: 3.27798340683\n",
      "(Iteration 511 / 49000) loss: 3.27211294694\n",
      "(Iteration 521 / 49000) loss: 3.10675327254\n",
      "(Iteration 531 / 49000) loss: 3.22886577762\n",
      "(Iteration 541 / 49000) loss: 2.94028968307\n",
      "(Iteration 551 / 49000) loss: 3.22285452599\n",
      "(Iteration 561 / 49000) loss: 3.10697005939\n",
      "(Iteration 571 / 49000) loss: 2.99094125886\n",
      "(Iteration 581 / 49000) loss: 2.99891374472\n",
      "(Epoch 6 / 500) Training Accuracy: 0.328448979592, Validation Accuracy: 0.327\n",
      "(Iteration 591 / 49000) loss: 3.32291035453\n",
      "(Iteration 601 / 49000) loss: 2.84724376116\n",
      "(Iteration 611 / 49000) loss: 2.73519916509\n",
      "(Iteration 621 / 49000) loss: 2.73014672367\n",
      "(Iteration 631 / 49000) loss: 2.95557870454\n",
      "(Iteration 641 / 49000) loss: 2.76964852664\n",
      "(Iteration 651 / 49000) loss: 2.64102235183\n",
      "(Iteration 661 / 49000) loss: 2.84155682555\n",
      "(Iteration 671 / 49000) loss: 2.52330193591\n",
      "(Iteration 681 / 49000) loss: 2.55489825218\n",
      "(Epoch 7 / 500) Training Accuracy: 0.339714285714, Validation Accuracy: 0.325\n",
      "(Iteration 691 / 49000) loss: 2.72400552398\n",
      "(Iteration 701 / 49000) loss: 2.52484463127\n",
      "(Iteration 711 / 49000) loss: 2.47451738818\n",
      "(Iteration 721 / 49000) loss: 2.823104186\n",
      "(Iteration 731 / 49000) loss: 2.49683153426\n",
      "(Iteration 741 / 49000) loss: 2.4300075804\n",
      "(Iteration 751 / 49000) loss: 2.63525837578\n",
      "(Iteration 761 / 49000) loss: 2.42072838387\n",
      "(Iteration 771 / 49000) loss: 2.46249421667\n",
      "(Iteration 781 / 49000) loss: 2.44367664332\n",
      "(Epoch 8 / 500) Training Accuracy: 0.346653061224, Validation Accuracy: 0.32\n",
      "(Iteration 791 / 49000) loss: 2.37801454565\n",
      "(Iteration 801 / 49000) loss: 2.47477830019\n",
      "(Iteration 811 / 49000) loss: 2.58093301682\n",
      "(Iteration 821 / 49000) loss: 2.38022388602\n",
      "(Iteration 831 / 49000) loss: 2.40421174944\n",
      "(Iteration 841 / 49000) loss: 2.33513920857\n",
      "(Iteration 851 / 49000) loss: 2.18096558892\n",
      "(Iteration 861 / 49000) loss: 2.28346546141\n",
      "(Iteration 871 / 49000) loss: 2.32321674261\n",
      "(Iteration 881 / 49000) loss: 2.16383723064\n",
      "(Epoch 9 / 500) Training Accuracy: 0.356204081633, Validation Accuracy: 0.339\n",
      "(Iteration 891 / 49000) loss: 2.41514295832\n",
      "(Iteration 901 / 49000) loss: 2.131568509\n",
      "(Iteration 911 / 49000) loss: 2.32570732254\n",
      "(Iteration 921 / 49000) loss: 2.16458270742\n",
      "(Iteration 931 / 49000) loss: 2.13927353535\n",
      "(Iteration 941 / 49000) loss: 2.24143894127\n",
      "(Iteration 951 / 49000) loss: 2.18111207646\n",
      "(Iteration 961 / 49000) loss: 2.08813179684\n",
      "(Iteration 971 / 49000) loss: 2.16949626605\n",
      "(Epoch 10 / 500) Training Accuracy: 0.365653061224, Validation Accuracy: 0.349\n",
      "(Iteration 981 / 49000) loss: 2.131407811\n",
      "(Iteration 991 / 49000) loss: 2.17291524371\n",
      "(Iteration 1001 / 49000) loss: 1.96300500676\n",
      "(Iteration 1011 / 49000) loss: 2.20849501192\n",
      "(Iteration 1021 / 49000) loss: 1.97013037446\n",
      "(Iteration 1031 / 49000) loss: 2.03083766542\n",
      "(Iteration 1041 / 49000) loss: 1.99029462367\n",
      "(Iteration 1051 / 49000) loss: 2.05593669683\n",
      "(Iteration 1061 / 49000) loss: 1.95911806191\n",
      "(Iteration 1071 / 49000) loss: 1.94268826847\n",
      "(Epoch 11 / 500) Training Accuracy: 0.376387755102, Validation Accuracy: 0.358\n",
      "(Iteration 1081 / 49000) loss: 1.96874132731\n",
      "(Iteration 1091 / 49000) loss: 1.9671293549\n",
      "(Iteration 1101 / 49000) loss: 1.91878440772\n",
      "(Iteration 1111 / 49000) loss: 1.87138488274\n",
      "(Iteration 1121 / 49000) loss: 2.01048881923\n",
      "(Iteration 1131 / 49000) loss: 1.92939337893\n",
      "(Iteration 1141 / 49000) loss: 1.8767899821\n",
      "(Iteration 1151 / 49000) loss: 2.04123090302\n",
      "(Iteration 1161 / 49000) loss: 1.99258861685\n",
      "(Iteration 1171 / 49000) loss: 1.99365642828\n",
      "(Epoch 12 / 500) Training Accuracy: 0.380510204082, Validation Accuracy: 0.356\n",
      "(Iteration 1181 / 49000) loss: 1.90374978487\n",
      "(Iteration 1191 / 49000) loss: 1.94635198163\n",
      "(Iteration 1201 / 49000) loss: 1.86319225137\n",
      "(Iteration 1211 / 49000) loss: 1.95245780957\n",
      "(Iteration 1221 / 49000) loss: 1.86490632593\n",
      "(Iteration 1231 / 49000) loss: 1.84622433249\n",
      "(Iteration 1241 / 49000) loss: 1.85810868484\n",
      "(Iteration 1251 / 49000) loss: 1.90656362155\n",
      "(Iteration 1261 / 49000) loss: 1.81897615978\n",
      "(Iteration 1271 / 49000) loss: 1.77015593144\n",
      "(Epoch 13 / 500) Training Accuracy: 0.385306122449, Validation Accuracy: 0.368\n",
      "(Iteration 1281 / 49000) loss: 1.77205043596\n",
      "(Iteration 1291 / 49000) loss: 1.7677379212\n",
      "(Iteration 1301 / 49000) loss: 1.77029273003\n",
      "(Iteration 1311 / 49000) loss: 1.76061652993\n",
      "(Iteration 1321 / 49000) loss: 1.85690394318\n",
      "(Iteration 1331 / 49000) loss: 1.72560658717\n",
      "(Iteration 1341 / 49000) loss: 1.91106393382\n",
      "(Iteration 1351 / 49000) loss: 1.85612634531\n",
      "(Iteration 1361 / 49000) loss: 1.87117198721\n",
      "(Iteration 1371 / 49000) loss: 1.77674834754\n",
      "(Epoch 14 / 500) Training Accuracy: 0.394265306122, Validation Accuracy: 0.38\n",
      "(Iteration 1381 / 49000) loss: 1.71181532539\n",
      "(Iteration 1391 / 49000) loss: 1.81027569099\n",
      "(Iteration 1401 / 49000) loss: 1.6774338689\n",
      "(Iteration 1411 / 49000) loss: 1.73768398046\n",
      "(Iteration 1421 / 49000) loss: 1.74157611243\n",
      "(Iteration 1431 / 49000) loss: 1.7461891411\n",
      "(Iteration 1441 / 49000) loss: 1.75351402278\n",
      "(Iteration 1451 / 49000) loss: 1.7595535916\n",
      "(Iteration 1461 / 49000) loss: 1.7155953431\n",
      "(Epoch 15 / 500) Training Accuracy: 0.404979591837, Validation Accuracy: 0.377\n",
      "(Iteration 1471 / 49000) loss: 1.7225752037\n",
      "(Iteration 1481 / 49000) loss: 1.76829161264\n",
      "(Iteration 1491 / 49000) loss: 1.68045309471\n",
      "(Iteration 1501 / 49000) loss: 1.66200959164\n",
      "(Iteration 1511 / 49000) loss: 1.76379702339\n",
      "(Iteration 1521 / 49000) loss: 1.83628093103\n",
      "(Iteration 1531 / 49000) loss: 1.81008263824\n",
      "(Iteration 1541 / 49000) loss: 1.69537639838\n",
      "(Iteration 1551 / 49000) loss: 1.69992130947\n",
      "(Iteration 1561 / 49000) loss: 1.58876023198\n",
      "(Epoch 16 / 500) Training Accuracy: 0.40906122449, Validation Accuracy: 0.396\n",
      "(Iteration 1571 / 49000) loss: 1.7304565901\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 1581 / 49000) loss: 1.74913225032\n",
      "(Iteration 1591 / 49000) loss: 1.65560973074\n",
      "(Iteration 1601 / 49000) loss: 1.7399291327\n",
      "(Iteration 1611 / 49000) loss: 1.68257457608\n",
      "(Iteration 1621 / 49000) loss: 1.66323639231\n",
      "(Iteration 1631 / 49000) loss: 1.74559849486\n",
      "(Iteration 1641 / 49000) loss: 1.82477815606\n",
      "(Iteration 1651 / 49000) loss: 1.81312330478\n",
      "(Iteration 1661 / 49000) loss: 1.58573066016\n",
      "(Epoch 17 / 500) Training Accuracy: 0.410489795918, Validation Accuracy: 0.38\n",
      "(Iteration 1671 / 49000) loss: 1.76187671804\n",
      "(Iteration 1681 / 49000) loss: 1.69475110067\n",
      "(Iteration 1691 / 49000) loss: 1.70033446171\n",
      "(Iteration 1701 / 49000) loss: 1.69566218869\n",
      "(Iteration 1711 / 49000) loss: 1.66432450241\n",
      "(Iteration 1721 / 49000) loss: 1.76275382329\n",
      "(Iteration 1731 / 49000) loss: 1.70149604335\n",
      "(Iteration 1741 / 49000) loss: 1.58955278188\n",
      "(Iteration 1751 / 49000) loss: 1.57250772257\n",
      "(Iteration 1761 / 49000) loss: 1.70819851789\n",
      "(Epoch 18 / 500) Training Accuracy: 0.415346938776, Validation Accuracy: 0.396\n",
      "(Iteration 1771 / 49000) loss: 1.68862626192\n",
      "(Iteration 1781 / 49000) loss: 1.67106170629\n",
      "(Iteration 1791 / 49000) loss: 1.6021901394\n",
      "(Iteration 1801 / 49000) loss: 1.68107854807\n",
      "(Iteration 1811 / 49000) loss: 1.68077201639\n",
      "(Iteration 1821 / 49000) loss: 1.72270707108\n",
      "(Iteration 1831 / 49000) loss: 1.7522176532\n",
      "(Iteration 1841 / 49000) loss: 1.67248112117\n",
      "(Iteration 1851 / 49000) loss: 1.7041132245\n",
      "(Iteration 1861 / 49000) loss: 1.64624986742\n",
      "(Epoch 19 / 500) Training Accuracy: 0.41793877551, Validation Accuracy: 0.401\n",
      "(Iteration 1871 / 49000) loss: 1.62384659799\n",
      "(Iteration 1881 / 49000) loss: 1.66867960367\n",
      "(Iteration 1891 / 49000) loss: 1.62703098468\n",
      "(Iteration 1901 / 49000) loss: 1.74860779615\n",
      "(Iteration 1911 / 49000) loss: 1.59878748262\n",
      "(Iteration 1921 / 49000) loss: 1.70296748314\n",
      "(Iteration 1931 / 49000) loss: 1.74331221648\n",
      "(Iteration 1941 / 49000) loss: 1.64957228931\n",
      "(Iteration 1951 / 49000) loss: 1.62903608612\n",
      "(Epoch 20 / 500) Training Accuracy: 0.424755102041, Validation Accuracy: 0.4\n",
      "(Iteration 1961 / 49000) loss: 1.73242258787\n",
      "(Iteration 1971 / 49000) loss: 1.78375808938\n",
      "(Iteration 1981 / 49000) loss: 1.72091513655\n",
      "(Iteration 1991 / 49000) loss: 1.68945040004\n",
      "(Iteration 2001 / 49000) loss: 1.6756264049\n",
      "(Iteration 2011 / 49000) loss: 1.64145201704\n",
      "(Iteration 2021 / 49000) loss: 1.75683969184\n",
      "(Iteration 2031 / 49000) loss: 1.63160018149\n",
      "(Iteration 2041 / 49000) loss: 1.63827330548\n",
      "(Iteration 2051 / 49000) loss: 1.60606236189\n",
      "(Epoch 21 / 500) Training Accuracy: 0.416326530612, Validation Accuracy: 0.404\n",
      "(Iteration 2061 / 49000) loss: 1.66209672753\n",
      "(Iteration 2071 / 49000) loss: 1.60608572166\n",
      "(Iteration 2081 / 49000) loss: 1.62587975303\n",
      "(Iteration 2091 / 49000) loss: 1.56298648992\n",
      "(Iteration 2101 / 49000) loss: 1.72156202052\n",
      "(Iteration 2111 / 49000) loss: 1.62545269583\n",
      "(Iteration 2121 / 49000) loss: 1.81480270139\n",
      "(Iteration 2131 / 49000) loss: 1.60818915919\n",
      "(Iteration 2141 / 49000) loss: 1.70380730511\n",
      "(Iteration 2151 / 49000) loss: 1.6845770907\n",
      "(Epoch 22 / 500) Training Accuracy: 0.41587755102, Validation Accuracy: 0.407\n",
      "(Iteration 2161 / 49000) loss: 1.59513014105\n",
      "(Iteration 2171 / 49000) loss: 1.74856794815\n",
      "(Iteration 2181 / 49000) loss: 1.69160901393\n",
      "(Iteration 2191 / 49000) loss: 1.65615449507\n",
      "(Iteration 2201 / 49000) loss: 1.64324913174\n",
      "(Iteration 2211 / 49000) loss: 1.66535178881\n",
      "(Iteration 2221 / 49000) loss: 1.64906173706\n",
      "(Iteration 2231 / 49000) loss: 1.72910538044\n",
      "(Iteration 2241 / 49000) loss: 1.69897956728\n",
      "(Iteration 2251 / 49000) loss: 1.77736577641\n",
      "(Epoch 23 / 500) Training Accuracy: 0.419693877551, Validation Accuracy: 0.408\n",
      "(Iteration 2261 / 49000) loss: 1.67884077756\n",
      "(Iteration 2271 / 49000) loss: 1.73228951115\n",
      "(Iteration 2281 / 49000) loss: 1.59559397695\n",
      "(Iteration 2291 / 49000) loss: 1.62067419334\n",
      "(Iteration 2301 / 49000) loss: 1.59750504359\n",
      "(Iteration 2311 / 49000) loss: 1.64741063809\n",
      "(Iteration 2321 / 49000) loss: 1.60591524632\n",
      "(Iteration 2331 / 49000) loss: 1.677528008\n",
      "(Iteration 2341 / 49000) loss: 1.64751479593\n",
      "(Iteration 2351 / 49000) loss: 1.64417031729\n",
      "(Epoch 24 / 500) Training Accuracy: 0.427979591837, Validation Accuracy: 0.411\n",
      "(Iteration 2361 / 49000) loss: 1.59625989565\n",
      "(Iteration 2371 / 49000) loss: 1.61990806529\n",
      "(Iteration 2381 / 49000) loss: 1.67562545174\n",
      "(Iteration 2391 / 49000) loss: 1.63437452247\n",
      "(Iteration 2401 / 49000) loss: 1.65658594562\n",
      "(Iteration 2411 / 49000) loss: 1.62078437552\n",
      "(Iteration 2421 / 49000) loss: 1.65433877707\n",
      "(Iteration 2431 / 49000) loss: 1.73012061955\n",
      "(Iteration 2441 / 49000) loss: 1.5286653732\n",
      "(Epoch 25 / 500) Training Accuracy: 0.422612244898, Validation Accuracy: 0.4\n",
      "(Iteration 2451 / 49000) loss: 1.61473388936\n",
      "(Iteration 2461 / 49000) loss: 1.61797459316\n",
      "(Iteration 2471 / 49000) loss: 1.61660916923\n",
      "(Iteration 2481 / 49000) loss: 1.64776496985\n",
      "(Iteration 2491 / 49000) loss: 1.61893474966\n",
      "(Iteration 2501 / 49000) loss: 1.57512368741\n",
      "(Iteration 2511 / 49000) loss: 1.56857975205\n",
      "(Iteration 2521 / 49000) loss: 1.72184342231\n",
      "(Iteration 2531 / 49000) loss: 1.62440439176\n",
      "(Iteration 2541 / 49000) loss: 1.6736671012\n",
      "(Epoch 26 / 500) Training Accuracy: 0.431734693878, Validation Accuracy: 0.413\n",
      "(Iteration 2551 / 49000) loss: 1.71756175347\n",
      "(Iteration 2561 / 49000) loss: 1.67512897068\n",
      "(Iteration 2571 / 49000) loss: 1.71972561639\n",
      "(Iteration 2581 / 49000) loss: 1.60147111153\n",
      "(Iteration 2591 / 49000) loss: 1.60745307354\n",
      "(Iteration 2601 / 49000) loss: 1.63098933512\n",
      "(Iteration 2611 / 49000) loss: 1.62724356343\n",
      "(Iteration 2621 / 49000) loss: 1.660480005\n",
      "(Iteration 2631 / 49000) loss: 1.69179700077\n",
      "(Iteration 2641 / 49000) loss: 1.69501282466\n",
      "(Epoch 27 / 500) Training Accuracy: 0.43087755102, Validation Accuracy: 0.401\n",
      "(Iteration 2651 / 49000) loss: 1.63222214278\n",
      "(Iteration 2661 / 49000) loss: 1.65361190974\n",
      "(Iteration 2671 / 49000) loss: 1.59041999995\n",
      "(Iteration 2681 / 49000) loss: 1.65657798124\n",
      "(Iteration 2691 / 49000) loss: 1.56506263407\n",
      "(Iteration 2701 / 49000) loss: 1.66168331215\n",
      "(Iteration 2711 / 49000) loss: 1.65707249528\n",
      "(Iteration 2721 / 49000) loss: 1.66620641724\n",
      "(Iteration 2731 / 49000) loss: 1.60216322852\n",
      "(Iteration 2741 / 49000) loss: 1.72576132496\n",
      "(Epoch 28 / 500) Training Accuracy: 0.427387755102, Validation Accuracy: 0.4\n",
      "(Iteration 2751 / 49000) loss: 1.59921600061\n",
      "(Iteration 2761 / 49000) loss: 1.61045057046\n",
      "(Iteration 2771 / 49000) loss: 1.66154922525\n",
      "(Iteration 2781 / 49000) loss: 1.59489892917\n",
      "(Iteration 2791 / 49000) loss: 1.5995305976\n",
      "(Iteration 2801 / 49000) loss: 1.67122411204\n",
      "(Iteration 2811 / 49000) loss: 1.70498105936\n",
      "(Iteration 2821 / 49000) loss: 1.58027408874\n",
      "(Iteration 2831 / 49000) loss: 1.68459026542\n",
      "(Iteration 2841 / 49000) loss: 1.62547842267\n",
      "(Epoch 29 / 500) Training Accuracy: 0.433469387755, Validation Accuracy: 0.418\n",
      "(Iteration 2851 / 49000) loss: 1.63824238339\n",
      "(Iteration 2861 / 49000) loss: 1.56854221661\n",
      "(Iteration 2871 / 49000) loss: 1.67248391938\n",
      "(Iteration 2881 / 49000) loss: 1.66879690905\n",
      "(Iteration 2891 / 49000) loss: 1.64593646224\n",
      "(Iteration 2901 / 49000) loss: 1.68529764249\n",
      "(Iteration 2911 / 49000) loss: 1.6152492856\n",
      "(Iteration 2921 / 49000) loss: 1.65779193832\n",
      "(Iteration 2931 / 49000) loss: 1.6351388762\n",
      "(Epoch 30 / 500) Training Accuracy: 0.433857142857, Validation Accuracy: 0.41\n",
      "(Iteration 2941 / 49000) loss: 1.63558650017\n",
      "(Iteration 2951 / 49000) loss: 1.56587638024\n",
      "(Iteration 2961 / 49000) loss: 1.60758148743\n",
      "(Iteration 2971 / 49000) loss: 1.58954304543\n",
      "(Iteration 2981 / 49000) loss: 1.60716961873\n",
      "(Iteration 2991 / 49000) loss: 1.64531266484\n",
      "(Iteration 3001 / 49000) loss: 1.68041663761\n",
      "(Iteration 3011 / 49000) loss: 1.58797439164\n",
      "(Iteration 3021 / 49000) loss: 1.73149525921\n",
      "(Iteration 3031 / 49000) loss: 1.57900975855\n",
      "(Epoch 31 / 500) Training Accuracy: 0.428244897959, Validation Accuracy: 0.396\n",
      "(Iteration 3041 / 49000) loss: 1.72721516043\n",
      "(Iteration 3051 / 49000) loss: 1.65375581801\n",
      "(Iteration 3061 / 49000) loss: 1.63132855015\n",
      "(Iteration 3071 / 49000) loss: 1.61068456498\n",
      "(Iteration 3081 / 49000) loss: 1.67509102671\n",
      "(Iteration 3091 / 49000) loss: 1.72351465952\n",
      "(Iteration 3101 / 49000) loss: 1.66346437214\n",
      "(Iteration 3111 / 49000) loss: 1.60045904061\n",
      "(Iteration 3121 / 49000) loss: 1.5325317541\n",
      "(Iteration 3131 / 49000) loss: 1.58626633279\n",
      "(Epoch 32 / 500) Training Accuracy: 0.42887755102, Validation Accuracy: 0.397\n",
      "(Iteration 3141 / 49000) loss: 1.64294022269\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 3151 / 49000) loss: 1.62555446843\n",
      "(Iteration 3161 / 49000) loss: 1.62942600824\n",
      "(Iteration 3171 / 49000) loss: 1.63114770214\n",
      "(Iteration 3181 / 49000) loss: 1.61046136342\n",
      "(Iteration 3191 / 49000) loss: 1.62761120655\n",
      "(Iteration 3201 / 49000) loss: 1.67535594323\n",
      "(Iteration 3211 / 49000) loss: 1.56434695695\n",
      "(Iteration 3221 / 49000) loss: 1.58624937779\n",
      "(Iteration 3231 / 49000) loss: 1.67035214809\n",
      "(Epoch 33 / 500) Training Accuracy: 0.430775510204, Validation Accuracy: 0.403\n",
      "(Iteration 3241 / 49000) loss: 1.50755398362\n",
      "(Iteration 3251 / 49000) loss: 1.58387900829\n",
      "(Iteration 3261 / 49000) loss: 1.60480916237\n",
      "(Iteration 3271 / 49000) loss: 1.56382908621\n",
      "(Iteration 3281 / 49000) loss: 1.64280230146\n",
      "(Iteration 3291 / 49000) loss: 1.58843124359\n",
      "(Iteration 3301 / 49000) loss: 1.56540509347\n",
      "(Iteration 3311 / 49000) loss: 1.67794219433\n",
      "(Iteration 3321 / 49000) loss: 1.62667640509\n",
      "(Iteration 3331 / 49000) loss: 1.65453243744\n",
      "(Epoch 34 / 500) Training Accuracy: 0.431408163265, Validation Accuracy: 0.41\n",
      "(Iteration 3341 / 49000) loss: 1.65987872782\n",
      "(Iteration 3351 / 49000) loss: 1.61958951266\n",
      "(Iteration 3361 / 49000) loss: 1.64117982441\n",
      "(Iteration 3371 / 49000) loss: 1.57904900523\n",
      "(Iteration 3381 / 49000) loss: 1.62082783977\n",
      "(Iteration 3391 / 49000) loss: 1.61434489284\n",
      "(Iteration 3401 / 49000) loss: 1.59656760911\n",
      "(Iteration 3411 / 49000) loss: 1.65874601593\n",
      "(Iteration 3421 / 49000) loss: 1.62454221982\n",
      "(Epoch 35 / 500) Training Accuracy: 0.438959183673, Validation Accuracy: 0.407\n",
      "(Iteration 3431 / 49000) loss: 1.59145017929\n",
      "(Iteration 3441 / 49000) loss: 1.60933449708\n",
      "(Iteration 3451 / 49000) loss: 1.68531167229\n",
      "(Iteration 3461 / 49000) loss: 1.52587862317\n",
      "(Iteration 3471 / 49000) loss: 1.61698189754\n",
      "(Iteration 3481 / 49000) loss: 1.51762704895\n",
      "(Iteration 3491 / 49000) loss: 1.60397804451\n",
      "(Iteration 3501 / 49000) loss: 1.67230477091\n",
      "(Iteration 3511 / 49000) loss: 1.6751965631\n",
      "(Iteration 3521 / 49000) loss: 1.62558686293\n",
      "(Epoch 36 / 500) Training Accuracy: 0.436836734694, Validation Accuracy: 0.412\n",
      "(Iteration 3531 / 49000) loss: 1.54823269469\n",
      "(Iteration 3541 / 49000) loss: 1.54706935643\n",
      "(Iteration 3551 / 49000) loss: 1.55932379424\n",
      "(Iteration 3561 / 49000) loss: 1.56318492335\n",
      "(Iteration 3571 / 49000) loss: 1.60173443498\n",
      "(Iteration 3581 / 49000) loss: 1.66969392991\n",
      "(Iteration 3591 / 49000) loss: 1.57969882393\n",
      "(Iteration 3601 / 49000) loss: 1.64309203046\n",
      "(Iteration 3611 / 49000) loss: 1.60714126253\n",
      "(Iteration 3621 / 49000) loss: 1.58331422874\n",
      "(Epoch 37 / 500) Training Accuracy: 0.430265306122, Validation Accuracy: 0.402\n",
      "(Iteration 3631 / 49000) loss: 1.59083504889\n",
      "(Iteration 3641 / 49000) loss: 1.65293006408\n",
      "(Iteration 3651 / 49000) loss: 1.60558118739\n",
      "(Iteration 3661 / 49000) loss: 1.4599805285\n",
      "(Iteration 3671 / 49000) loss: 1.65672240819\n",
      "(Iteration 3681 / 49000) loss: 1.68163448961\n",
      "(Iteration 3691 / 49000) loss: 1.50688658024\n",
      "(Iteration 3701 / 49000) loss: 1.66981881592\n",
      "(Iteration 3711 / 49000) loss: 1.62734195997\n",
      "(Iteration 3721 / 49000) loss: 1.64268804538\n",
      "(Epoch 38 / 500) Training Accuracy: 0.436326530612, Validation Accuracy: 0.4\n",
      "(Iteration 3731 / 49000) loss: 1.63983601292\n",
      "(Iteration 3741 / 49000) loss: 1.60578518054\n",
      "(Iteration 3751 / 49000) loss: 1.66089316503\n",
      "(Iteration 3761 / 49000) loss: 1.62411452291\n",
      "(Iteration 3771 / 49000) loss: 1.6526466379\n",
      "(Iteration 3781 / 49000) loss: 1.54167675913\n",
      "(Iteration 3791 / 49000) loss: 1.61829977193\n",
      "(Iteration 3801 / 49000) loss: 1.62847979403\n",
      "(Iteration 3811 / 49000) loss: 1.6504072175\n",
      "(Iteration 3821 / 49000) loss: 1.56009576027\n",
      "(Epoch 39 / 500) Training Accuracy: 0.442816326531, Validation Accuracy: 0.416\n",
      "(Iteration 3831 / 49000) loss: 1.65225050626\n",
      "(Iteration 3841 / 49000) loss: 1.62423125314\n",
      "(Iteration 3851 / 49000) loss: 1.59321523591\n",
      "(Iteration 3861 / 49000) loss: 1.60850410273\n",
      "(Iteration 3871 / 49000) loss: 1.65384608216\n",
      "(Iteration 3881 / 49000) loss: 1.64493152608\n",
      "(Iteration 3891 / 49000) loss: 1.60674779449\n",
      "(Iteration 3901 / 49000) loss: 1.57184116845\n",
      "(Iteration 3911 / 49000) loss: 1.66864115125\n",
      "(Epoch 40 / 500) Training Accuracy: 0.442693877551, Validation Accuracy: 0.412\n",
      "(Iteration 3921 / 49000) loss: 1.72281370093\n",
      "(Iteration 3931 / 49000) loss: 1.56592425518\n",
      "(Iteration 3941 / 49000) loss: 1.6203079279\n",
      "(Iteration 3951 / 49000) loss: 1.57153752244\n",
      "(Iteration 3961 / 49000) loss: 1.68490167161\n",
      "(Iteration 3971 / 49000) loss: 1.61931367648\n",
      "(Iteration 3981 / 49000) loss: 1.58146637267\n",
      "(Iteration 3991 / 49000) loss: 1.62841995721\n",
      "(Iteration 4001 / 49000) loss: 1.68342572296\n",
      "(Iteration 4011 / 49000) loss: 1.65037567026\n",
      "(Epoch 41 / 500) Training Accuracy: 0.43993877551, Validation Accuracy: 0.409\n",
      "(Iteration 4021 / 49000) loss: 1.54810304539\n",
      "(Iteration 4031 / 49000) loss: 1.55294170494\n",
      "(Iteration 4041 / 49000) loss: 1.65114520755\n",
      "(Iteration 4051 / 49000) loss: 1.51137303273\n",
      "(Iteration 4061 / 49000) loss: 1.59559353436\n",
      "(Iteration 4071 / 49000) loss: 1.60368269307\n",
      "(Iteration 4081 / 49000) loss: 1.57688199349\n",
      "(Iteration 4091 / 49000) loss: 1.58448814977\n",
      "(Iteration 4101 / 49000) loss: 1.54823238968\n",
      "(Iteration 4111 / 49000) loss: 1.68784698581\n",
      "(Epoch 42 / 500) Training Accuracy: 0.435632653061, Validation Accuracy: 0.41\n",
      "(Iteration 4121 / 49000) loss: 1.61682381086\n",
      "(Iteration 4131 / 49000) loss: 1.5740034908\n",
      "(Iteration 4141 / 49000) loss: 1.57165967074\n",
      "(Iteration 4151 / 49000) loss: 1.62104601662\n",
      "(Iteration 4161 / 49000) loss: 1.58803536419\n",
      "(Iteration 4171 / 49000) loss: 1.6201136184\n",
      "(Iteration 4181 / 49000) loss: 1.59773538462\n",
      "(Iteration 4191 / 49000) loss: 1.64376289069\n",
      "(Iteration 4201 / 49000) loss: 1.68041669846\n",
      "(Iteration 4211 / 49000) loss: 1.56367970902\n",
      "(Epoch 43 / 500) Training Accuracy: 0.444795918367, Validation Accuracy: 0.421\n",
      "(Iteration 4221 / 49000) loss: 1.64210827947\n",
      "(Iteration 4231 / 49000) loss: 1.62578781855\n",
      "(Iteration 4241 / 49000) loss: 1.5098218573\n",
      "(Iteration 4251 / 49000) loss: 1.71016359721\n",
      "(Iteration 4261 / 49000) loss: 1.63356185297\n",
      "(Iteration 4271 / 49000) loss: 1.57261343611\n",
      "(Iteration 4281 / 49000) loss: 1.61975736532\n",
      "(Iteration 4291 / 49000) loss: 1.63238067887\n",
      "(Iteration 4301 / 49000) loss: 1.60134543669\n",
      "(Iteration 4311 / 49000) loss: 1.61171911782\n",
      "(Epoch 44 / 500) Training Accuracy: 0.443408163265, Validation Accuracy: 0.417\n",
      "(Iteration 4321 / 49000) loss: 1.60090289841\n",
      "(Iteration 4331 / 49000) loss: 1.65482307819\n",
      "(Iteration 4341 / 49000) loss: 1.62832534294\n",
      "(Iteration 4351 / 49000) loss: 1.62174685076\n",
      "(Iteration 4361 / 49000) loss: 1.54966566324\n",
      "(Iteration 4371 / 49000) loss: 1.6083693795\n",
      "(Iteration 4381 / 49000) loss: 1.57206359697\n",
      "(Iteration 4391 / 49000) loss: 1.5929246767\n",
      "(Iteration 4401 / 49000) loss: 1.60242954728\n",
      "(Epoch 45 / 500) Training Accuracy: 0.443102040816, Validation Accuracy: 0.408\n",
      "(Iteration 4411 / 49000) loss: 1.55238256287\n",
      "(Iteration 4421 / 49000) loss: 1.56275905957\n",
      "(Iteration 4431 / 49000) loss: 1.59298228713\n",
      "(Iteration 4441 / 49000) loss: 1.61399168519\n",
      "(Iteration 4451 / 49000) loss: 1.60192560284\n",
      "(Iteration 4461 / 49000) loss: 1.55420180701\n",
      "(Iteration 4471 / 49000) loss: 1.5772000015\n",
      "(Iteration 4481 / 49000) loss: 1.50678930473\n",
      "(Iteration 4491 / 49000) loss: 1.57479972387\n",
      "(Iteration 4501 / 49000) loss: 1.59977912883\n",
      "(Epoch 46 / 500) Training Accuracy: 0.435142857143, Validation Accuracy: 0.406\n",
      "(Iteration 4511 / 49000) loss: 1.54567375986\n",
      "(Iteration 4521 / 49000) loss: 1.57034335639\n",
      "(Iteration 4531 / 49000) loss: 1.58121146326\n",
      "(Iteration 4541 / 49000) loss: 1.63838553322\n",
      "(Iteration 4551 / 49000) loss: 1.58432253269\n",
      "(Iteration 4561 / 49000) loss: 1.527094302\n",
      "(Iteration 4571 / 49000) loss: 1.54255004381\n",
      "(Iteration 4581 / 49000) loss: 1.54281994948\n",
      "(Iteration 4591 / 49000) loss: 1.63493203612\n",
      "(Iteration 4601 / 49000) loss: 1.53175381071\n",
      "(Epoch 47 / 500) Training Accuracy: 0.442734693878, Validation Accuracy: 0.423\n",
      "(Iteration 4611 / 49000) loss: 1.54036287285\n",
      "(Iteration 4621 / 49000) loss: 1.51831928119\n",
      "(Iteration 4631 / 49000) loss: 1.59457748417\n",
      "(Iteration 4641 / 49000) loss: 1.53900393324\n",
      "(Iteration 4651 / 49000) loss: 1.52458329601\n",
      "(Iteration 4661 / 49000) loss: 1.55931214224\n",
      "(Iteration 4671 / 49000) loss: 1.52883842051\n",
      "(Iteration 4681 / 49000) loss: 1.60212183814\n",
      "(Iteration 4691 / 49000) loss: 1.60703707006\n",
      "(Iteration 4701 / 49000) loss: 1.51961918011\n",
      "(Epoch 48 / 500) Training Accuracy: 0.444571428571, Validation Accuracy: 0.421\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 4711 / 49000) loss: 1.47628904646\n",
      "(Iteration 4721 / 49000) loss: 1.51919002122\n",
      "(Iteration 4731 / 49000) loss: 1.58820833921\n",
      "(Iteration 4741 / 49000) loss: 1.57134106183\n",
      "(Iteration 4751 / 49000) loss: 1.55719522495\n",
      "(Iteration 4761 / 49000) loss: 1.57777032507\n",
      "(Iteration 4771 / 49000) loss: 1.64577191073\n",
      "(Iteration 4781 / 49000) loss: 1.58999704805\n",
      "(Iteration 4791 / 49000) loss: 1.51347429969\n",
      "(Iteration 4801 / 49000) loss: 1.61997117798\n",
      "(Epoch 49 / 500) Training Accuracy: 0.441020408163, Validation Accuracy: 0.407\n",
      "(Iteration 4811 / 49000) loss: 1.66853268455\n",
      "(Iteration 4821 / 49000) loss: 1.62125643636\n",
      "(Iteration 4831 / 49000) loss: 1.62536967274\n",
      "(Iteration 4841 / 49000) loss: 1.55326308762\n",
      "(Iteration 4851 / 49000) loss: 1.50160210045\n",
      "(Iteration 4861 / 49000) loss: 1.59659891619\n",
      "(Iteration 4871 / 49000) loss: 1.64366493256\n",
      "(Iteration 4881 / 49000) loss: 1.54058737926\n",
      "(Iteration 4891 / 49000) loss: 1.6704821826\n",
      "(Epoch 50 / 500) Training Accuracy: 0.445102040816, Validation Accuracy: 0.429\n",
      "(Iteration 4901 / 49000) loss: 1.55016790387\n",
      "(Iteration 4911 / 49000) loss: 1.62536242183\n",
      "(Iteration 4921 / 49000) loss: 1.50333225828\n",
      "(Iteration 4931 / 49000) loss: 1.61780066162\n",
      "(Iteration 4941 / 49000) loss: 1.5594820798\n",
      "(Iteration 4951 / 49000) loss: 1.59739237273\n",
      "(Iteration 4961 / 49000) loss: 1.61578348397\n",
      "(Iteration 4971 / 49000) loss: 1.58622817023\n",
      "(Iteration 4981 / 49000) loss: 1.60527233499\n",
      "(Iteration 4991 / 49000) loss: 1.64939327542\n",
      "(Epoch 51 / 500) Training Accuracy: 0.448653061224, Validation Accuracy: 0.432\n",
      "(Iteration 5001 / 49000) loss: 1.59833736761\n",
      "(Iteration 5011 / 49000) loss: 1.58364882217\n",
      "(Iteration 5021 / 49000) loss: 1.62835286303\n",
      "(Iteration 5031 / 49000) loss: 1.61258969344\n",
      "(Iteration 5041 / 49000) loss: 1.50272030869\n",
      "(Iteration 5051 / 49000) loss: 1.6223219338\n",
      "(Iteration 5061 / 49000) loss: 1.6194439337\n",
      "(Iteration 5071 / 49000) loss: 1.63860010245\n",
      "(Iteration 5081 / 49000) loss: 1.53763660961\n",
      "(Iteration 5091 / 49000) loss: 1.55499037199\n",
      "(Epoch 52 / 500) Training Accuracy: 0.448551020408, Validation Accuracy: 0.41\n",
      "(Iteration 5101 / 49000) loss: 1.58608835408\n",
      "(Iteration 5111 / 49000) loss: 1.6136955362\n",
      "(Iteration 5121 / 49000) loss: 1.59274435834\n",
      "(Iteration 5131 / 49000) loss: 1.59144259698\n",
      "(Iteration 5141 / 49000) loss: 1.54675478084\n",
      "(Iteration 5151 / 49000) loss: 1.59263354299\n",
      "(Iteration 5161 / 49000) loss: 1.5885150334\n",
      "(Iteration 5171 / 49000) loss: 1.56464633635\n",
      "(Iteration 5181 / 49000) loss: 1.61780920953\n",
      "(Iteration 5191 / 49000) loss: 1.52735716469\n",
      "(Epoch 53 / 500) Training Accuracy: 0.443102040816, Validation Accuracy: 0.415\n",
      "(Iteration 5201 / 49000) loss: 1.57717869794\n",
      "(Iteration 5211 / 49000) loss: 1.52098422374\n",
      "(Iteration 5221 / 49000) loss: 1.57445600025\n",
      "(Iteration 5231 / 49000) loss: 1.61407348337\n",
      "(Iteration 5241 / 49000) loss: 1.59105881834\n",
      "(Iteration 5251 / 49000) loss: 1.55386550097\n",
      "(Iteration 5261 / 49000) loss: 1.605280603\n",
      "(Iteration 5271 / 49000) loss: 1.65621883922\n",
      "(Iteration 5281 / 49000) loss: 1.55071060909\n",
      "(Iteration 5291 / 49000) loss: 1.52690357023\n",
      "(Epoch 54 / 500) Training Accuracy: 0.45193877551, Validation Accuracy: 0.422\n",
      "(Iteration 5301 / 49000) loss: 1.60348933094\n",
      "(Iteration 5311 / 49000) loss: 1.50365542068\n",
      "(Iteration 5321 / 49000) loss: 1.59491564891\n",
      "(Iteration 5331 / 49000) loss: 1.53232835808\n",
      "(Iteration 5341 / 49000) loss: 1.54336805135\n",
      "(Iteration 5351 / 49000) loss: 1.54097405567\n",
      "(Iteration 5361 / 49000) loss: 1.65702511722\n",
      "(Iteration 5371 / 49000) loss: 1.55345797812\n",
      "(Iteration 5381 / 49000) loss: 1.57238878221\n",
      "(Epoch 55 / 500) Training Accuracy: 0.444510204082, Validation Accuracy: 0.418\n",
      "(Iteration 5391 / 49000) loss: 1.56760794515\n",
      "(Iteration 5401 / 49000) loss: 1.59759285775\n",
      "(Iteration 5411 / 49000) loss: 1.6279797061\n",
      "(Iteration 5421 / 49000) loss: 1.5247805388\n",
      "(Iteration 5431 / 49000) loss: 1.62824836524\n",
      "(Iteration 5441 / 49000) loss: 1.52691225505\n",
      "(Iteration 5451 / 49000) loss: 1.44112519723\n",
      "(Iteration 5461 / 49000) loss: 1.50746967762\n",
      "(Iteration 5471 / 49000) loss: 1.56242457522\n",
      "(Iteration 5481 / 49000) loss: 1.54088202979\n",
      "(Epoch 56 / 500) Training Accuracy: 0.448979591837, Validation Accuracy: 0.42\n",
      "(Iteration 5491 / 49000) loss: 1.48583864757\n",
      "(Iteration 5501 / 49000) loss: 1.61356278415\n",
      "(Iteration 5511 / 49000) loss: 1.59012790866\n",
      "(Iteration 5521 / 49000) loss: 1.47057663185\n",
      "(Iteration 5531 / 49000) loss: 1.55385421389\n",
      "(Iteration 5541 / 49000) loss: 1.62516999863\n",
      "(Iteration 5551 / 49000) loss: 1.49616383199\n",
      "(Iteration 5561 / 49000) loss: 1.61412589542\n",
      "(Iteration 5571 / 49000) loss: 1.68277576564\n",
      "(Iteration 5581 / 49000) loss: 1.60275704611\n",
      "(Epoch 57 / 500) Training Accuracy: 0.451897959184, Validation Accuracy: 0.418\n",
      "(Iteration 5591 / 49000) loss: 1.64424995934\n",
      "(Iteration 5601 / 49000) loss: 1.61960131948\n",
      "(Iteration 5611 / 49000) loss: 1.65106310086\n",
      "(Iteration 5621 / 49000) loss: 1.58761018539\n",
      "(Iteration 5631 / 49000) loss: 1.63397629162\n",
      "(Iteration 5641 / 49000) loss: 1.50264299233\n",
      "(Iteration 5651 / 49000) loss: 1.59067695444\n",
      "(Iteration 5661 / 49000) loss: 1.60051149231\n",
      "(Iteration 5671 / 49000) loss: 1.61133528597\n",
      "(Iteration 5681 / 49000) loss: 1.58629679574\n",
      "(Epoch 58 / 500) Training Accuracy: 0.445, Validation Accuracy: 0.424\n",
      "(Iteration 5691 / 49000) loss: 1.48939673297\n",
      "(Iteration 5701 / 49000) loss: 1.53783969731\n",
      "(Iteration 5711 / 49000) loss: 1.55481199861\n",
      "(Iteration 5721 / 49000) loss: 1.53864718473\n",
      "(Iteration 5731 / 49000) loss: 1.5988823547\n",
      "(Iteration 5741 / 49000) loss: 1.55556852734\n",
      "(Iteration 5751 / 49000) loss: 1.66711833802\n",
      "(Iteration 5761 / 49000) loss: 1.65658990648\n",
      "(Iteration 5771 / 49000) loss: 1.59812861905\n",
      "(Iteration 5781 / 49000) loss: 1.62159230243\n",
      "(Epoch 59 / 500) Training Accuracy: 0.450571428571, Validation Accuracy: 0.413\n",
      "(Iteration 5791 / 49000) loss: 1.5619149411\n",
      "(Iteration 5801 / 49000) loss: 1.49565086819\n",
      "(Iteration 5811 / 49000) loss: 1.57169824011\n",
      "(Iteration 5821 / 49000) loss: 1.5366047679\n",
      "(Iteration 5831 / 49000) loss: 1.60144683684\n",
      "(Iteration 5841 / 49000) loss: 1.54326687256\n",
      "(Iteration 5851 / 49000) loss: 1.5836529058\n",
      "(Iteration 5861 / 49000) loss: 1.55884742008\n",
      "(Iteration 5871 / 49000) loss: 1.6509315552\n",
      "(Epoch 60 / 500) Training Accuracy: 0.454102040816, Validation Accuracy: 0.428\n",
      "(Iteration 5881 / 49000) loss: 1.5208194933\n",
      "(Iteration 5891 / 49000) loss: 1.50922116287\n",
      "(Iteration 5901 / 49000) loss: 1.60497316924\n",
      "(Iteration 5911 / 49000) loss: 1.55285336408\n",
      "(Iteration 5921 / 49000) loss: 1.53856807173\n",
      "(Iteration 5931 / 49000) loss: 1.56016797034\n",
      "(Iteration 5941 / 49000) loss: 1.52144481824\n",
      "(Iteration 5951 / 49000) loss: 1.55494247846\n",
      "(Iteration 5961 / 49000) loss: 1.55494464765\n",
      "(Iteration 5971 / 49000) loss: 1.62814470527\n",
      "(Epoch 61 / 500) Training Accuracy: 0.447, Validation Accuracy: 0.402\n",
      "(Iteration 5981 / 49000) loss: 1.51510570133\n",
      "(Iteration 5991 / 49000) loss: 1.50958485865\n",
      "(Iteration 6001 / 49000) loss: 1.55025963173\n",
      "(Iteration 6011 / 49000) loss: 1.64684563706\n",
      "(Iteration 6021 / 49000) loss: 1.56281327933\n",
      "(Iteration 6031 / 49000) loss: 1.57983840442\n",
      "(Iteration 6041 / 49000) loss: 1.57799923027\n",
      "(Iteration 6051 / 49000) loss: 1.57602454957\n",
      "(Iteration 6061 / 49000) loss: 1.5478091352\n",
      "(Iteration 6071 / 49000) loss: 1.59416496014\n",
      "(Epoch 62 / 500) Training Accuracy: 0.454489795918, Validation Accuracy: 0.423\n",
      "(Iteration 6081 / 49000) loss: 1.57092378814\n",
      "(Iteration 6091 / 49000) loss: 1.55463060537\n",
      "(Iteration 6101 / 49000) loss: 1.52147643292\n",
      "(Iteration 6111 / 49000) loss: 1.60681526766\n",
      "(Iteration 6121 / 49000) loss: 1.59562417236\n",
      "(Iteration 6131 / 49000) loss: 1.64922665631\n",
      "(Iteration 6141 / 49000) loss: 1.59934177274\n",
      "(Iteration 6151 / 49000) loss: 1.61419028534\n",
      "(Iteration 6161 / 49000) loss: 1.54228191654\n",
      "(Iteration 6171 / 49000) loss: 1.68548935145\n",
      "(Epoch 63 / 500) Training Accuracy: 0.457979591837, Validation Accuracy: 0.435\n",
      "(Iteration 6181 / 49000) loss: 1.5225173143\n",
      "(Iteration 6191 / 49000) loss: 1.54719317843\n",
      "(Iteration 6201 / 49000) loss: 1.46077483723\n",
      "(Iteration 6211 / 49000) loss: 1.61519443599\n",
      "(Iteration 6221 / 49000) loss: 1.55797140263\n",
      "(Iteration 6231 / 49000) loss: 1.67847464793\n",
      "(Iteration 6241 / 49000) loss: 1.5321633359\n",
      "(Iteration 6251 / 49000) loss: 1.5844175901\n",
      "(Iteration 6261 / 49000) loss: 1.56634642022\n",
      "(Iteration 6271 / 49000) loss: 1.49742138307\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Epoch 64 / 500) Training Accuracy: 0.457387755102, Validation Accuracy: 0.431\n",
      "(Iteration 6281 / 49000) loss: 1.64930749367\n",
      "(Iteration 6291 / 49000) loss: 1.58480095766\n",
      "(Iteration 6301 / 49000) loss: 1.59335207146\n",
      "(Iteration 6311 / 49000) loss: 1.53421700117\n",
      "(Iteration 6321 / 49000) loss: 1.47618186633\n",
      "(Iteration 6331 / 49000) loss: 1.59620500521\n",
      "(Iteration 6341 / 49000) loss: 1.61629792764\n",
      "(Iteration 6351 / 49000) loss: 1.63741914668\n",
      "(Iteration 6361 / 49000) loss: 1.62567525628\n",
      "(Epoch 65 / 500) Training Accuracy: 0.458204081633, Validation Accuracy: 0.43\n",
      "(Iteration 6371 / 49000) loss: 1.58843245832\n",
      "(Iteration 6381 / 49000) loss: 1.62502920746\n",
      "(Iteration 6391 / 49000) loss: 1.65281168778\n",
      "(Iteration 6401 / 49000) loss: 1.59922402976\n",
      "(Iteration 6411 / 49000) loss: 1.54046350628\n",
      "(Iteration 6421 / 49000) loss: 1.63960665351\n",
      "(Iteration 6431 / 49000) loss: 1.62477494389\n",
      "(Iteration 6441 / 49000) loss: 1.58237210702\n",
      "(Iteration 6451 / 49000) loss: 1.51381922182\n",
      "(Iteration 6461 / 49000) loss: 1.58895582804\n",
      "(Epoch 66 / 500) Training Accuracy: 0.460959183673, Validation Accuracy: 0.431\n",
      "(Iteration 6471 / 49000) loss: 1.56452072661\n",
      "(Iteration 6481 / 49000) loss: 1.52725190989\n",
      "(Iteration 6491 / 49000) loss: 1.580609438\n",
      "(Iteration 6501 / 49000) loss: 1.61943890117\n",
      "(Iteration 6511 / 49000) loss: 1.50674975687\n",
      "(Iteration 6521 / 49000) loss: 1.62013859719\n",
      "(Iteration 6531 / 49000) loss: 1.534223552\n",
      "(Iteration 6541 / 49000) loss: 1.59369241096\n",
      "(Iteration 6551 / 49000) loss: 1.43264108796\n",
      "(Iteration 6561 / 49000) loss: 1.66367309445\n",
      "(Epoch 67 / 500) Training Accuracy: 0.455326530612, Validation Accuracy: 0.423\n",
      "(Iteration 6571 / 49000) loss: 1.54917859908\n",
      "(Iteration 6581 / 49000) loss: 1.52992985071\n",
      "(Iteration 6591 / 49000) loss: 1.56946816577\n",
      "(Iteration 6601 / 49000) loss: 1.63049885423\n",
      "(Iteration 6611 / 49000) loss: 1.53338108232\n",
      "(Iteration 6621 / 49000) loss: 1.48466239498\n",
      "(Iteration 6631 / 49000) loss: 1.57477136292\n",
      "(Iteration 6641 / 49000) loss: 1.5949992562\n",
      "(Iteration 6651 / 49000) loss: 1.6209681384\n",
      "(Iteration 6661 / 49000) loss: 1.55051006195\n",
      "(Epoch 68 / 500) Training Accuracy: 0.460816326531, Validation Accuracy: 0.431\n",
      "(Iteration 6671 / 49000) loss: 1.54915557075\n",
      "(Iteration 6681 / 49000) loss: 1.52361112877\n",
      "(Iteration 6691 / 49000) loss: 1.60547965868\n",
      "(Iteration 6701 / 49000) loss: 1.49128891529\n",
      "(Iteration 6711 / 49000) loss: 1.50688316263\n",
      "(Iteration 6721 / 49000) loss: 1.51818187231\n",
      "(Iteration 6731 / 49000) loss: 1.5543007295\n",
      "(Iteration 6741 / 49000) loss: 1.56321024738\n",
      "(Iteration 6751 / 49000) loss: 1.57896502231\n",
      "(Iteration 6761 / 49000) loss: 1.58785699542\n",
      "(Epoch 69 / 500) Training Accuracy: 0.45706122449, Validation Accuracy: 0.417\n",
      "(Iteration 6771 / 49000) loss: 1.50025555116\n",
      "(Iteration 6781 / 49000) loss: 1.54690439701\n",
      "(Iteration 6791 / 49000) loss: 1.54067589028\n",
      "(Iteration 6801 / 49000) loss: 1.51955625255\n",
      "(Iteration 6811 / 49000) loss: 1.61305165758\n",
      "(Iteration 6821 / 49000) loss: 1.53026878242\n",
      "(Iteration 6831 / 49000) loss: 1.52849102692\n",
      "(Iteration 6841 / 49000) loss: 1.53731099031\n",
      "(Iteration 6851 / 49000) loss: 1.53749112054\n",
      "(Epoch 70 / 500) Training Accuracy: 0.458448979592, Validation Accuracy: 0.424\n",
      "(Iteration 6861 / 49000) loss: 1.52724814556\n",
      "(Iteration 6871 / 49000) loss: 1.46177547214\n",
      "(Iteration 6881 / 49000) loss: 1.49717385111\n",
      "(Iteration 6891 / 49000) loss: 1.5154209144\n",
      "(Iteration 6901 / 49000) loss: 1.57432717108\n",
      "(Iteration 6911 / 49000) loss: 1.56506354528\n",
      "(Iteration 6921 / 49000) loss: 1.5840597194\n",
      "(Iteration 6931 / 49000) loss: 1.52747255432\n",
      "(Iteration 6941 / 49000) loss: 1.49376023551\n",
      "(Iteration 6951 / 49000) loss: 1.50422276897\n",
      "(Epoch 71 / 500) Training Accuracy: 0.459979591837, Validation Accuracy: 0.433\n",
      "(Iteration 6961 / 49000) loss: 1.55435070109\n",
      "(Iteration 6971 / 49000) loss: 1.59806140897\n",
      "(Iteration 6981 / 49000) loss: 1.52615781513\n",
      "(Iteration 6991 / 49000) loss: 1.48865931119\n",
      "(Iteration 7001 / 49000) loss: 1.56469336383\n",
      "(Iteration 7011 / 49000) loss: 1.54485096423\n",
      "(Iteration 7021 / 49000) loss: 1.57171674632\n",
      "(Iteration 7031 / 49000) loss: 1.54214313879\n",
      "(Iteration 7041 / 49000) loss: 1.61277508892\n",
      "(Iteration 7051 / 49000) loss: 1.6085743986\n",
      "(Epoch 72 / 500) Training Accuracy: 0.461020408163, Validation Accuracy: 0.424\n",
      "(Iteration 7061 / 49000) loss: 1.5147443908\n",
      "(Iteration 7071 / 49000) loss: 1.57520522044\n",
      "(Iteration 7081 / 49000) loss: 1.61639601207\n",
      "(Iteration 7091 / 49000) loss: 1.60088064912\n",
      "(Iteration 7101 / 49000) loss: 1.48233438106\n",
      "(Iteration 7111 / 49000) loss: 1.50197339056\n",
      "(Iteration 7121 / 49000) loss: 1.53892613711\n",
      "(Iteration 7131 / 49000) loss: 1.55058575756\n",
      "(Iteration 7141 / 49000) loss: 1.55712469473\n",
      "(Iteration 7151 / 49000) loss: 1.43680905064\n",
      "(Epoch 73 / 500) Training Accuracy: 0.460897959184, Validation Accuracy: 0.433\n",
      "(Iteration 7161 / 49000) loss: 1.51935781717\n",
      "(Iteration 7171 / 49000) loss: 1.58745983746\n",
      "(Iteration 7181 / 49000) loss: 1.51936233831\n",
      "(Iteration 7191 / 49000) loss: 1.58673227142\n",
      "(Iteration 7201 / 49000) loss: 1.5753406539\n",
      "(Iteration 7211 / 49000) loss: 1.51152263943\n",
      "(Iteration 7221 / 49000) loss: 1.55178658763\n",
      "(Iteration 7231 / 49000) loss: 1.52491543158\n",
      "(Iteration 7241 / 49000) loss: 1.56732707276\n",
      "(Iteration 7251 / 49000) loss: 1.50522548173\n",
      "(Epoch 74 / 500) Training Accuracy: 0.457, Validation Accuracy: 0.42\n",
      "(Iteration 7261 / 49000) loss: 1.45720430461\n",
      "(Iteration 7271 / 49000) loss: 1.50476253563\n",
      "(Iteration 7281 / 49000) loss: 1.67878186305\n",
      "(Iteration 7291 / 49000) loss: 1.58591539679\n",
      "(Iteration 7301 / 49000) loss: 1.64697836949\n",
      "(Iteration 7311 / 49000) loss: 1.58057136302\n",
      "(Iteration 7321 / 49000) loss: 1.53348974825\n",
      "(Iteration 7331 / 49000) loss: 1.56025427153\n",
      "(Iteration 7341 / 49000) loss: 1.57563192046\n",
      "(Epoch 75 / 500) Training Accuracy: 0.455408163265, Validation Accuracy: 0.424\n",
      "(Iteration 7351 / 49000) loss: 1.55550710872\n",
      "(Iteration 7361 / 49000) loss: 1.61320599633\n",
      "(Iteration 7371 / 49000) loss: 1.58017847838\n",
      "(Iteration 7381 / 49000) loss: 1.42121346107\n",
      "(Iteration 7391 / 49000) loss: 1.50733535845\n",
      "(Iteration 7401 / 49000) loss: 1.55745602932\n",
      "(Iteration 7411 / 49000) loss: 1.57945321368\n",
      "(Iteration 7421 / 49000) loss: 1.52733694141\n",
      "(Iteration 7431 / 49000) loss: 1.56412056805\n",
      "(Iteration 7441 / 49000) loss: 1.51754016927\n",
      "(Epoch 76 / 500) Training Accuracy: 0.45993877551, Validation Accuracy: 0.425\n",
      "(Iteration 7451 / 49000) loss: 1.52344286238\n",
      "(Iteration 7461 / 49000) loss: 1.59938130561\n",
      "(Iteration 7471 / 49000) loss: 1.57481856264\n",
      "(Iteration 7481 / 49000) loss: 1.52156345953\n",
      "(Iteration 7491 / 49000) loss: 1.64960946693\n",
      "(Iteration 7501 / 49000) loss: 1.60643551042\n",
      "(Iteration 7511 / 49000) loss: 1.56462669054\n",
      "(Iteration 7521 / 49000) loss: 1.48922152044\n",
      "(Iteration 7531 / 49000) loss: 1.53863295898\n",
      "(Iteration 7541 / 49000) loss: 1.5138057632\n",
      "(Epoch 77 / 500) Training Accuracy: 0.462224489796, Validation Accuracy: 0.437\n",
      "(Iteration 7551 / 49000) loss: 1.5255411115\n",
      "(Iteration 7561 / 49000) loss: 1.52847145183\n",
      "(Iteration 7571 / 49000) loss: 1.59181883953\n",
      "(Iteration 7581 / 49000) loss: 1.60403985833\n",
      "(Iteration 7591 / 49000) loss: 1.5843501119\n",
      "(Iteration 7601 / 49000) loss: 1.55127134463\n",
      "(Iteration 7611 / 49000) loss: 1.493164191\n",
      "(Iteration 7621 / 49000) loss: 1.6753313571\n",
      "(Iteration 7631 / 49000) loss: 1.4908488094\n",
      "(Iteration 7641 / 49000) loss: 1.57619819539\n",
      "(Epoch 78 / 500) Training Accuracy: 0.459632653061, Validation Accuracy: 0.431\n",
      "(Iteration 7651 / 49000) loss: 1.53367283591\n",
      "(Iteration 7661 / 49000) loss: 1.51770955209\n",
      "(Iteration 7671 / 49000) loss: 1.6194152512\n",
      "(Iteration 7681 / 49000) loss: 1.55619262175\n",
      "(Iteration 7691 / 49000) loss: 1.54155476373\n",
      "(Iteration 7701 / 49000) loss: 1.56644934621\n",
      "(Iteration 7711 / 49000) loss: 1.42317677705\n",
      "(Iteration 7721 / 49000) loss: 1.42634929894\n",
      "(Iteration 7731 / 49000) loss: 1.59272592561\n",
      "(Iteration 7741 / 49000) loss: 1.43986565996\n",
      "(Epoch 79 / 500) Training Accuracy: 0.464673469388, Validation Accuracy: 0.429\n",
      "(Iteration 7751 / 49000) loss: 1.48653831365\n",
      "(Iteration 7761 / 49000) loss: 1.5796392498\n",
      "(Iteration 7771 / 49000) loss: 1.54740712512\n",
      "(Iteration 7781 / 49000) loss: 1.54909511243\n",
      "(Iteration 7791 / 49000) loss: 1.60557672799\n",
      "(Iteration 7801 / 49000) loss: 1.56495142948\n",
      "(Iteration 7811 / 49000) loss: 1.5322836517\n",
      "(Iteration 7821 / 49000) loss: 1.52488245798\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 7831 / 49000) loss: 1.46741594685\n",
      "(Epoch 80 / 500) Training Accuracy: 0.463428571429, Validation Accuracy: 0.426\n",
      "(Iteration 7841 / 49000) loss: 1.45773546684\n",
      "(Iteration 7851 / 49000) loss: 1.56411167117\n",
      "(Iteration 7861 / 49000) loss: 1.55062403044\n",
      "(Iteration 7871 / 49000) loss: 1.51242694143\n",
      "(Iteration 7881 / 49000) loss: 1.46557080879\n",
      "(Iteration 7891 / 49000) loss: 1.62119763287\n",
      "(Iteration 7901 / 49000) loss: 1.50453755173\n",
      "(Iteration 7911 / 49000) loss: 1.5240585302\n",
      "(Iteration 7921 / 49000) loss: 1.56085559797\n",
      "(Iteration 7931 / 49000) loss: 1.499406565\n",
      "(Epoch 81 / 500) Training Accuracy: 0.464224489796, Validation Accuracy: 0.425\n",
      "(Iteration 7941 / 49000) loss: 1.54963029323\n",
      "(Iteration 7951 / 49000) loss: 1.63855371177\n",
      "(Iteration 7961 / 49000) loss: 1.52369458487\n",
      "(Iteration 7971 / 49000) loss: 1.5412279466\n",
      "(Iteration 7981 / 49000) loss: 1.51559678956\n",
      "(Iteration 7991 / 49000) loss: 1.4971645337\n",
      "(Iteration 8001 / 49000) loss: 1.45735695756\n",
      "(Iteration 8011 / 49000) loss: 1.60391143282\n",
      "(Iteration 8021 / 49000) loss: 1.51747708196\n",
      "(Iteration 8031 / 49000) loss: 1.55257391028\n",
      "(Epoch 82 / 500) Training Accuracy: 0.463918367347, Validation Accuracy: 0.426\n",
      "(Iteration 8041 / 49000) loss: 1.65657315308\n",
      "(Iteration 8051 / 49000) loss: 1.58420584023\n",
      "(Iteration 8061 / 49000) loss: 1.55288599628\n",
      "(Iteration 8071 / 49000) loss: 1.63876625656\n",
      "(Iteration 8081 / 49000) loss: 1.53224229316\n",
      "(Iteration 8091 / 49000) loss: 1.62639330116\n",
      "(Iteration 8101 / 49000) loss: 1.51558447689\n",
      "(Iteration 8111 / 49000) loss: 1.49367396939\n",
      "(Iteration 8121 / 49000) loss: 1.55809567362\n",
      "(Iteration 8131 / 49000) loss: 1.49598818673\n",
      "(Epoch 83 / 500) Training Accuracy: 0.462102040816, Validation Accuracy: 0.433\n",
      "(Iteration 8141 / 49000) loss: 1.44690192879\n",
      "(Iteration 8151 / 49000) loss: 1.54131309333\n",
      "(Iteration 8161 / 49000) loss: 1.54955972085\n",
      "(Iteration 8171 / 49000) loss: 1.5350108917\n",
      "(Iteration 8181 / 49000) loss: 1.5228627505\n",
      "(Iteration 8191 / 49000) loss: 1.56216141082\n",
      "(Iteration 8201 / 49000) loss: 1.59283918811\n",
      "(Iteration 8211 / 49000) loss: 1.48868408667\n",
      "(Iteration 8221 / 49000) loss: 1.54418593569\n",
      "(Iteration 8231 / 49000) loss: 1.50883207438\n",
      "(Epoch 84 / 500) Training Accuracy: 0.464081632653, Validation Accuracy: 0.425\n",
      "(Iteration 8241 / 49000) loss: 1.5421257338\n",
      "(Iteration 8251 / 49000) loss: 1.55156458117\n",
      "(Iteration 8261 / 49000) loss: 1.52932045519\n",
      "(Iteration 8271 / 49000) loss: 1.52898346786\n",
      "(Iteration 8281 / 49000) loss: 1.57878070301\n",
      "(Iteration 8291 / 49000) loss: 1.59108708537\n",
      "(Iteration 8301 / 49000) loss: 1.52074861858\n",
      "(Iteration 8311 / 49000) loss: 1.5289026487\n",
      "(Iteration 8321 / 49000) loss: 1.51672451865\n",
      "(Epoch 85 / 500) Training Accuracy: 0.465448979592, Validation Accuracy: 0.429\n",
      "(Iteration 8331 / 49000) loss: 1.50422541776\n",
      "(Iteration 8341 / 49000) loss: 1.57879350953\n",
      "(Iteration 8351 / 49000) loss: 1.59055113874\n",
      "(Iteration 8361 / 49000) loss: 1.44299970976\n",
      "(Iteration 8371 / 49000) loss: 1.49172199464\n",
      "(Iteration 8381 / 49000) loss: 1.46671382914\n",
      "(Iteration 8391 / 49000) loss: 1.52542612471\n",
      "(Iteration 8401 / 49000) loss: 1.54704411011\n",
      "(Iteration 8411 / 49000) loss: 1.51089001355\n",
      "(Iteration 8421 / 49000) loss: 1.57664950387\n",
      "(Epoch 86 / 500) Training Accuracy: 0.464979591837, Validation Accuracy: 0.423\n",
      "(Iteration 8431 / 49000) loss: 1.61896833287\n",
      "(Iteration 8441 / 49000) loss: 1.50230084635\n",
      "(Iteration 8451 / 49000) loss: 1.51963656863\n",
      "(Iteration 8461 / 49000) loss: 1.56574052549\n",
      "(Iteration 8471 / 49000) loss: 1.53841584452\n",
      "(Iteration 8481 / 49000) loss: 1.54132988624\n",
      "(Iteration 8491 / 49000) loss: 1.59148602868\n",
      "(Iteration 8501 / 49000) loss: 1.45592791529\n",
      "(Iteration 8511 / 49000) loss: 1.60953362423\n",
      "(Iteration 8521 / 49000) loss: 1.54374456264\n",
      "(Epoch 87 / 500) Training Accuracy: 0.469387755102, Validation Accuracy: 0.43\n",
      "(Iteration 8531 / 49000) loss: 1.48149547291\n",
      "(Iteration 8541 / 49000) loss: 1.52263788334\n",
      "(Iteration 8551 / 49000) loss: 1.50762958233\n",
      "(Iteration 8561 / 49000) loss: 1.44406954192\n",
      "(Iteration 8571 / 49000) loss: 1.49336307244\n",
      "(Iteration 8581 / 49000) loss: 1.5041309964\n",
      "(Iteration 8591 / 49000) loss: 1.60828649265\n",
      "(Iteration 8601 / 49000) loss: 1.556816445\n",
      "(Iteration 8611 / 49000) loss: 1.50305927346\n",
      "(Iteration 8621 / 49000) loss: 1.61873385516\n",
      "(Epoch 88 / 500) Training Accuracy: 0.468102040816, Validation Accuracy: 0.436\n",
      "(Iteration 8631 / 49000) loss: 1.53113069496\n",
      "(Iteration 8641 / 49000) loss: 1.58609669147\n",
      "(Iteration 8651 / 49000) loss: 1.46624729257\n",
      "(Iteration 8661 / 49000) loss: 1.60611893861\n",
      "(Iteration 8671 / 49000) loss: 1.57307944321\n",
      "(Iteration 8681 / 49000) loss: 1.5106713339\n",
      "(Iteration 8691 / 49000) loss: 1.59752222225\n",
      "(Iteration 8701 / 49000) loss: 1.54659916355\n",
      "(Iteration 8711 / 49000) loss: 1.56641274057\n",
      "(Iteration 8721 / 49000) loss: 1.48418101629\n",
      "(Epoch 89 / 500) Training Accuracy: 0.467612244898, Validation Accuracy: 0.427\n",
      "(Iteration 8731 / 49000) loss: 1.49823788929\n",
      "(Iteration 8741 / 49000) loss: 1.56439317988\n",
      "(Iteration 8751 / 49000) loss: 1.48658024199\n",
      "(Iteration 8761 / 49000) loss: 1.57691766644\n",
      "(Iteration 8771 / 49000) loss: 1.57393123633\n",
      "(Iteration 8781 / 49000) loss: 1.50100724773\n",
      "(Iteration 8791 / 49000) loss: 1.52603251459\n",
      "(Iteration 8801 / 49000) loss: 1.483114233\n",
      "(Iteration 8811 / 49000) loss: 1.56800908835\n",
      "(Epoch 90 / 500) Training Accuracy: 0.47006122449, Validation Accuracy: 0.433\n",
      "(Iteration 8821 / 49000) loss: 1.51666224713\n",
      "(Iteration 8831 / 49000) loss: 1.51351164426\n",
      "(Iteration 8841 / 49000) loss: 1.62001634914\n",
      "(Iteration 8851 / 49000) loss: 1.49853720597\n",
      "(Iteration 8861 / 49000) loss: 1.44427321048\n",
      "(Iteration 8871 / 49000) loss: 1.51107427409\n",
      "(Iteration 8881 / 49000) loss: 1.56903092259\n",
      "(Iteration 8891 / 49000) loss: 1.53943606472\n",
      "(Iteration 8901 / 49000) loss: 1.66753155032\n",
      "(Iteration 8911 / 49000) loss: 1.50150965075\n",
      "(Epoch 91 / 500) Training Accuracy: 0.467795918367, Validation Accuracy: 0.43\n",
      "(Iteration 8921 / 49000) loss: 1.54622745522\n",
      "(Iteration 8931 / 49000) loss: 1.5529464383\n",
      "(Iteration 8941 / 49000) loss: 1.50315413682\n",
      "(Iteration 8951 / 49000) loss: 1.47497270764\n",
      "(Iteration 8961 / 49000) loss: 1.54773848161\n",
      "(Iteration 8971 / 49000) loss: 1.538994128\n",
      "(Iteration 8981 / 49000) loss: 1.61108097199\n",
      "(Iteration 8991 / 49000) loss: 1.66680779313\n",
      "(Iteration 9001 / 49000) loss: 1.49186924074\n",
      "(Iteration 9011 / 49000) loss: 1.50117892736\n",
      "(Epoch 92 / 500) Training Accuracy: 0.468673469388, Validation Accuracy: 0.422\n",
      "(Iteration 9021 / 49000) loss: 1.50385685749\n",
      "(Iteration 9031 / 49000) loss: 1.45129921328\n",
      "(Iteration 9041 / 49000) loss: 1.43701817864\n",
      "(Iteration 9051 / 49000) loss: 1.56561160678\n",
      "(Iteration 9061 / 49000) loss: 1.43838790461\n",
      "(Iteration 9071 / 49000) loss: 1.49479205089\n",
      "(Iteration 9081 / 49000) loss: 1.50021488247\n",
      "(Iteration 9091 / 49000) loss: 1.54147298338\n",
      "(Iteration 9101 / 49000) loss: 1.48397398032\n",
      "(Iteration 9111 / 49000) loss: 1.55573109117\n",
      "(Epoch 93 / 500) Training Accuracy: 0.469244897959, Validation Accuracy: 0.436\n",
      "(Iteration 9121 / 49000) loss: 1.47308553973\n",
      "(Iteration 9131 / 49000) loss: 1.5259503415\n",
      "(Iteration 9141 / 49000) loss: 1.60671313466\n",
      "(Iteration 9151 / 49000) loss: 1.57161183427\n",
      "(Iteration 9161 / 49000) loss: 1.44425065434\n",
      "(Iteration 9171 / 49000) loss: 1.53965141521\n",
      "(Iteration 9181 / 49000) loss: 1.45369715859\n",
      "(Iteration 9191 / 49000) loss: 1.5291423484\n",
      "(Iteration 9201 / 49000) loss: 1.50900346789\n",
      "(Iteration 9211 / 49000) loss: 1.57924066693\n",
      "(Epoch 94 / 500) Training Accuracy: 0.46306122449, Validation Accuracy: 0.434\n",
      "(Iteration 9221 / 49000) loss: 1.60602766854\n",
      "(Iteration 9231 / 49000) loss: 1.61108037878\n",
      "(Iteration 9241 / 49000) loss: 1.5348897095\n",
      "(Iteration 9251 / 49000) loss: 1.54639007849\n",
      "(Iteration 9261 / 49000) loss: 1.44974002203\n",
      "(Iteration 9271 / 49000) loss: 1.38963363517\n",
      "(Iteration 9281 / 49000) loss: 1.49650618978\n",
      "(Iteration 9291 / 49000) loss: 1.58623611611\n",
      "(Iteration 9301 / 49000) loss: 1.46881734098\n",
      "(Epoch 95 / 500) Training Accuracy: 0.466816326531, Validation Accuracy: 0.441\n",
      "(Iteration 9311 / 49000) loss: 1.45410965404\n",
      "(Iteration 9321 / 49000) loss: 1.57320747478\n",
      "(Iteration 9331 / 49000) loss: 1.48921940469\n",
      "(Iteration 9341 / 49000) loss: 1.44360198834\n",
      "(Iteration 9351 / 49000) loss: 1.44415443053\n",
      "(Iteration 9361 / 49000) loss: 1.51289900608\n",
      "(Iteration 9371 / 49000) loss: 1.53676597621\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 9381 / 49000) loss: 1.47436776681\n",
      "(Iteration 9391 / 49000) loss: 1.56522124323\n",
      "(Iteration 9401 / 49000) loss: 1.51111042674\n",
      "(Epoch 96 / 500) Training Accuracy: 0.470857142857, Validation Accuracy: 0.439\n",
      "(Iteration 9411 / 49000) loss: 1.4648141116\n",
      "(Iteration 9421 / 49000) loss: 1.43393844678\n",
      "(Iteration 9431 / 49000) loss: 1.47371638521\n",
      "(Iteration 9441 / 49000) loss: 1.50724877505\n",
      "(Iteration 9451 / 49000) loss: 1.3556221856\n",
      "(Iteration 9461 / 49000) loss: 1.57137194608\n",
      "(Iteration 9471 / 49000) loss: 1.53454005699\n",
      "(Iteration 9481 / 49000) loss: 1.54036358215\n",
      "(Iteration 9491 / 49000) loss: 1.55554374056\n",
      "(Iteration 9501 / 49000) loss: 1.45897570312\n",
      "(Epoch 97 / 500) Training Accuracy: 0.471244897959, Validation Accuracy: 0.445\n",
      "(Iteration 9511 / 49000) loss: 1.52463090832\n",
      "(Iteration 9521 / 49000) loss: 1.51033089247\n",
      "(Iteration 9531 / 49000) loss: 1.55873895079\n",
      "(Iteration 9541 / 49000) loss: 1.47826097782\n",
      "(Iteration 9551 / 49000) loss: 1.57977273786\n",
      "(Iteration 9561 / 49000) loss: 1.48697525579\n",
      "(Iteration 9571 / 49000) loss: 1.48815621171\n",
      "(Iteration 9581 / 49000) loss: 1.58539963259\n",
      "(Iteration 9591 / 49000) loss: 1.46853702318\n",
      "(Iteration 9601 / 49000) loss: 1.4650969271\n",
      "(Epoch 98 / 500) Training Accuracy: 0.47593877551, Validation Accuracy: 0.428\n",
      "(Iteration 9611 / 49000) loss: 1.49631191788\n",
      "(Iteration 9621 / 49000) loss: 1.56746229213\n",
      "(Iteration 9631 / 49000) loss: 1.50419491701\n",
      "(Iteration 9641 / 49000) loss: 1.47378226571\n",
      "(Iteration 9651 / 49000) loss: 1.48582355251\n",
      "(Iteration 9661 / 49000) loss: 1.44533019981\n",
      "(Iteration 9671 / 49000) loss: 1.53232325581\n",
      "(Iteration 9681 / 49000) loss: 1.51872536536\n",
      "(Iteration 9691 / 49000) loss: 1.50157880313\n",
      "(Iteration 9701 / 49000) loss: 1.56282046706\n",
      "(Epoch 99 / 500) Training Accuracy: 0.475387755102, Validation Accuracy: 0.447\n",
      "(Iteration 9711 / 49000) loss: 1.43315837488\n",
      "(Iteration 9721 / 49000) loss: 1.53992544182\n",
      "(Iteration 9731 / 49000) loss: 1.48942236302\n",
      "(Iteration 9741 / 49000) loss: 1.42250243613\n",
      "(Iteration 9751 / 49000) loss: 1.50529858502\n",
      "(Iteration 9761 / 49000) loss: 1.42708643919\n",
      "(Iteration 9771 / 49000) loss: 1.57683408618\n",
      "(Iteration 9781 / 49000) loss: 1.61215900311\n",
      "(Iteration 9791 / 49000) loss: 1.5099781918\n",
      "(Epoch 100 / 500) Training Accuracy: 0.470897959184, Validation Accuracy: 0.436\n",
      "(Iteration 9801 / 49000) loss: 1.57561484768\n",
      "(Iteration 9811 / 49000) loss: 1.49968841428\n",
      "(Iteration 9821 / 49000) loss: 1.54128062743\n",
      "(Iteration 9831 / 49000) loss: 1.47149541358\n",
      "(Iteration 9841 / 49000) loss: 1.61226456481\n",
      "(Iteration 9851 / 49000) loss: 1.480392939\n",
      "(Iteration 9861 / 49000) loss: 1.51077563573\n",
      "(Iteration 9871 / 49000) loss: 1.58007498775\n",
      "(Iteration 9881 / 49000) loss: 1.47758156195\n",
      "(Iteration 9891 / 49000) loss: 1.6540999497\n",
      "(Epoch 101 / 500) Training Accuracy: 0.471387755102, Validation Accuracy: 0.442\n",
      "(Iteration 9901 / 49000) loss: 1.54964827935\n",
      "(Iteration 9911 / 49000) loss: 1.52968519167\n",
      "(Iteration 9921 / 49000) loss: 1.42991556629\n",
      "(Iteration 9931 / 49000) loss: 1.48414813071\n",
      "(Iteration 9941 / 49000) loss: 1.50983059017\n",
      "(Iteration 9951 / 49000) loss: 1.52704855255\n",
      "(Iteration 9961 / 49000) loss: 1.5475585476\n",
      "(Iteration 9971 / 49000) loss: 1.49350949741\n",
      "(Iteration 9981 / 49000) loss: 1.5460568795\n",
      "(Iteration 9991 / 49000) loss: 1.51596909389\n",
      "(Epoch 102 / 500) Training Accuracy: 0.474489795918, Validation Accuracy: 0.447\n",
      "(Iteration 10001 / 49000) loss: 1.58155693847\n",
      "(Iteration 10011 / 49000) loss: 1.47430677575\n",
      "(Iteration 10021 / 49000) loss: 1.62990886291\n",
      "(Iteration 10031 / 49000) loss: 1.52190859192\n",
      "(Iteration 10041 / 49000) loss: 1.64374009584\n",
      "(Iteration 10051 / 49000) loss: 1.58360183597\n",
      "(Iteration 10061 / 49000) loss: 1.49435795481\n",
      "(Iteration 10071 / 49000) loss: 1.41003366546\n",
      "(Iteration 10081 / 49000) loss: 1.52030726542\n",
      "(Iteration 10091 / 49000) loss: 1.53701061158\n",
      "(Epoch 103 / 500) Training Accuracy: 0.466714285714, Validation Accuracy: 0.425\n",
      "(Iteration 10101 / 49000) loss: 1.53476315147\n",
      "(Iteration 10111 / 49000) loss: 1.38909640355\n",
      "(Iteration 10121 / 49000) loss: 1.50946960776\n",
      "(Iteration 10131 / 49000) loss: 1.54141765114\n",
      "(Iteration 10141 / 49000) loss: 1.41131487089\n",
      "(Iteration 10151 / 49000) loss: 1.58724951025\n",
      "(Iteration 10161 / 49000) loss: 1.55413593285\n",
      "(Iteration 10171 / 49000) loss: 1.51460216944\n",
      "(Iteration 10181 / 49000) loss: 1.63410605058\n",
      "(Iteration 10191 / 49000) loss: 1.50577770671\n",
      "(Epoch 104 / 500) Training Accuracy: 0.470551020408, Validation Accuracy: 0.44\n",
      "(Iteration 10201 / 49000) loss: 1.60790390335\n",
      "(Iteration 10211 / 49000) loss: 1.5152458038\n",
      "(Iteration 10221 / 49000) loss: 1.56500530988\n",
      "(Iteration 10231 / 49000) loss: 1.43673090362\n",
      "(Iteration 10241 / 49000) loss: 1.60205468003\n",
      "(Iteration 10251 / 49000) loss: 1.48441503274\n",
      "(Iteration 10261 / 49000) loss: 1.50210358882\n",
      "(Iteration 10271 / 49000) loss: 1.58405690165\n",
      "(Iteration 10281 / 49000) loss: 1.43097954405\n",
      "(Epoch 105 / 500) Training Accuracy: 0.470020408163, Validation Accuracy: 0.435\n",
      "(Iteration 10291 / 49000) loss: 1.51318069201\n",
      "(Iteration 10301 / 49000) loss: 1.50527791159\n",
      "(Iteration 10311 / 49000) loss: 1.49109480531\n",
      "(Iteration 10321 / 49000) loss: 1.38171623872\n",
      "(Iteration 10331 / 49000) loss: 1.51262765377\n",
      "(Iteration 10341 / 49000) loss: 1.55890154504\n",
      "(Iteration 10351 / 49000) loss: 1.47617888107\n",
      "(Iteration 10361 / 49000) loss: 1.51519384933\n",
      "(Iteration 10371 / 49000) loss: 1.52377405446\n",
      "(Iteration 10381 / 49000) loss: 1.44562967064\n",
      "(Epoch 106 / 500) Training Accuracy: 0.471857142857, Validation Accuracy: 0.444\n",
      "(Iteration 10391 / 49000) loss: 1.60861644778\n",
      "(Iteration 10401 / 49000) loss: 1.56656610429\n",
      "(Iteration 10411 / 49000) loss: 1.56371448672\n",
      "(Iteration 10421 / 49000) loss: 1.5318798658\n",
      "(Iteration 10431 / 49000) loss: 1.51784471061\n",
      "(Iteration 10441 / 49000) loss: 1.5229195751\n",
      "(Iteration 10451 / 49000) loss: 1.47126550492\n",
      "(Iteration 10461 / 49000) loss: 1.49534439625\n",
      "(Iteration 10471 / 49000) loss: 1.49698516461\n",
      "(Iteration 10481 / 49000) loss: 1.4914632173\n",
      "(Epoch 107 / 500) Training Accuracy: 0.472183673469, Validation Accuracy: 0.437\n",
      "(Iteration 10491 / 49000) loss: 1.4455140416\n",
      "(Iteration 10501 / 49000) loss: 1.47151583648\n",
      "(Iteration 10511 / 49000) loss: 1.4546907678\n",
      "(Iteration 10521 / 49000) loss: 1.50562803392\n",
      "(Iteration 10531 / 49000) loss: 1.50845040244\n",
      "(Iteration 10541 / 49000) loss: 1.5237048937\n",
      "(Iteration 10551 / 49000) loss: 1.52578194042\n",
      "(Iteration 10561 / 49000) loss: 1.49688042925\n",
      "(Iteration 10571 / 49000) loss: 1.59361805364\n",
      "(Iteration 10581 / 49000) loss: 1.51131028431\n",
      "(Epoch 108 / 500) Training Accuracy: 0.481489795918, Validation Accuracy: 0.442\n",
      "(Iteration 10591 / 49000) loss: 1.58517490976\n",
      "(Iteration 10601 / 49000) loss: 1.43623075651\n",
      "(Iteration 10611 / 49000) loss: 1.58052749001\n",
      "(Iteration 10621 / 49000) loss: 1.46661189677\n",
      "(Iteration 10631 / 49000) loss: 1.48997422316\n",
      "(Iteration 10641 / 49000) loss: 1.54274718628\n",
      "(Iteration 10651 / 49000) loss: 1.53726594088\n",
      "(Iteration 10661 / 49000) loss: 1.47619325661\n",
      "(Iteration 10671 / 49000) loss: 1.59982609432\n",
      "(Iteration 10681 / 49000) loss: 1.54081542554\n",
      "(Epoch 109 / 500) Training Accuracy: 0.476224489796, Validation Accuracy: 0.454\n",
      "(Iteration 10691 / 49000) loss: 1.43357826106\n",
      "(Iteration 10701 / 49000) loss: 1.54708860272\n",
      "(Iteration 10711 / 49000) loss: 1.53840925951\n",
      "(Iteration 10721 / 49000) loss: 1.43751200122\n",
      "(Iteration 10731 / 49000) loss: 1.63453870133\n",
      "(Iteration 10741 / 49000) loss: 1.49128408032\n",
      "(Iteration 10751 / 49000) loss: 1.47921885476\n",
      "(Iteration 10761 / 49000) loss: 1.51187208956\n",
      "(Iteration 10771 / 49000) loss: 1.48128450004\n",
      "(Epoch 110 / 500) Training Accuracy: 0.476244897959, Validation Accuracy: 0.435\n",
      "(Iteration 10781 / 49000) loss: 1.48685517444\n",
      "(Iteration 10791 / 49000) loss: 1.57859219039\n",
      "(Iteration 10801 / 49000) loss: 1.51090550261\n",
      "(Iteration 10811 / 49000) loss: 1.47487889409\n",
      "(Iteration 10821 / 49000) loss: 1.56969134901\n",
      "(Iteration 10831 / 49000) loss: 1.51137030786\n",
      "(Iteration 10841 / 49000) loss: 1.51625055096\n",
      "(Iteration 10851 / 49000) loss: 1.47308314908\n",
      "(Iteration 10861 / 49000) loss: 1.54301141403\n",
      "(Iteration 10871 / 49000) loss: 1.63973335221\n",
      "(Epoch 111 / 500) Training Accuracy: 0.477816326531, Validation Accuracy: 0.439\n",
      "(Iteration 10881 / 49000) loss: 1.48991176621\n",
      "(Iteration 10891 / 49000) loss: 1.59171838589\n",
      "(Iteration 10901 / 49000) loss: 1.43979731112\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 10911 / 49000) loss: 1.42813312656\n",
      "(Iteration 10921 / 49000) loss: 1.53293781633\n",
      "(Iteration 10931 / 49000) loss: 1.5675422211\n",
      "(Iteration 10941 / 49000) loss: 1.47445847069\n",
      "(Iteration 10951 / 49000) loss: 1.45214625018\n",
      "(Iteration 10961 / 49000) loss: 1.55191358411\n",
      "(Iteration 10971 / 49000) loss: 1.55907063389\n",
      "(Epoch 112 / 500) Training Accuracy: 0.472918367347, Validation Accuracy: 0.438\n",
      "(Iteration 10981 / 49000) loss: 1.57443456187\n",
      "(Iteration 10991 / 49000) loss: 1.52086545534\n",
      "(Iteration 11001 / 49000) loss: 1.43444311355\n",
      "(Iteration 11011 / 49000) loss: 1.54691853168\n",
      "(Iteration 11021 / 49000) loss: 1.53725496309\n",
      "(Iteration 11031 / 49000) loss: 1.46223314138\n",
      "(Iteration 11041 / 49000) loss: 1.55270212091\n",
      "(Iteration 11051 / 49000) loss: 1.50926744238\n",
      "(Iteration 11061 / 49000) loss: 1.52715474197\n",
      "(Iteration 11071 / 49000) loss: 1.49253345481\n",
      "(Epoch 113 / 500) Training Accuracy: 0.477142857143, Validation Accuracy: 0.45\n",
      "(Iteration 11081 / 49000) loss: 1.49693615615\n",
      "(Iteration 11091 / 49000) loss: 1.50975246462\n",
      "(Iteration 11101 / 49000) loss: 1.51361677549\n",
      "(Iteration 11111 / 49000) loss: 1.4456001427\n",
      "(Iteration 11121 / 49000) loss: 1.46669333251\n",
      "(Iteration 11131 / 49000) loss: 1.4760302205\n",
      "(Iteration 11141 / 49000) loss: 1.51257974985\n",
      "(Iteration 11151 / 49000) loss: 1.44153205989\n",
      "(Iteration 11161 / 49000) loss: 1.59373652008\n",
      "(Iteration 11171 / 49000) loss: 1.53229254286\n",
      "(Epoch 114 / 500) Training Accuracy: 0.480571428571, Validation Accuracy: 0.437\n",
      "(Iteration 11181 / 49000) loss: 1.50571226065\n",
      "(Iteration 11191 / 49000) loss: 1.53209577833\n",
      "(Iteration 11201 / 49000) loss: 1.48499110729\n",
      "(Iteration 11211 / 49000) loss: 1.47179911707\n",
      "(Iteration 11221 / 49000) loss: 1.4932241585\n",
      "(Iteration 11231 / 49000) loss: 1.62634212475\n",
      "(Iteration 11241 / 49000) loss: 1.48071391832\n",
      "(Iteration 11251 / 49000) loss: 1.51102723755\n",
      "(Iteration 11261 / 49000) loss: 1.57814202889\n",
      "(Epoch 115 / 500) Training Accuracy: 0.47206122449, Validation Accuracy: 0.444\n",
      "(Iteration 11271 / 49000) loss: 1.47485304865\n",
      "(Iteration 11281 / 49000) loss: 1.52834167767\n",
      "(Iteration 11291 / 49000) loss: 1.45410497881\n",
      "(Iteration 11301 / 49000) loss: 1.47717623718\n",
      "(Iteration 11311 / 49000) loss: 1.52514451282\n",
      "(Iteration 11321 / 49000) loss: 1.4500371485\n",
      "(Iteration 11331 / 49000) loss: 1.56692199382\n",
      "(Iteration 11341 / 49000) loss: 1.45166572655\n",
      "(Iteration 11351 / 49000) loss: 1.43287784719\n",
      "(Iteration 11361 / 49000) loss: 1.39750234193\n",
      "(Epoch 116 / 500) Training Accuracy: 0.481244897959, Validation Accuracy: 0.439\n",
      "(Iteration 11371 / 49000) loss: 1.42722162315\n",
      "(Iteration 11381 / 49000) loss: 1.43074673823\n",
      "(Iteration 11391 / 49000) loss: 1.50220663249\n",
      "(Iteration 11401 / 49000) loss: 1.47445097711\n",
      "(Iteration 11411 / 49000) loss: 1.4588174213\n",
      "(Iteration 11421 / 49000) loss: 1.45442138782\n",
      "(Iteration 11431 / 49000) loss: 1.53345107531\n",
      "(Iteration 11441 / 49000) loss: 1.49353231552\n",
      "(Iteration 11451 / 49000) loss: 1.48917197773\n",
      "(Iteration 11461 / 49000) loss: 1.57094282369\n",
      "(Epoch 117 / 500) Training Accuracy: 0.476306122449, Validation Accuracy: 0.44\n",
      "(Iteration 11471 / 49000) loss: 1.52502922005\n",
      "(Iteration 11481 / 49000) loss: 1.62286698103\n",
      "(Iteration 11491 / 49000) loss: 1.47832276281\n",
      "(Iteration 11501 / 49000) loss: 1.51972335448\n",
      "(Iteration 11511 / 49000) loss: 1.40555905856\n",
      "(Iteration 11521 / 49000) loss: 1.55171256438\n",
      "(Iteration 11531 / 49000) loss: 1.58631425674\n",
      "(Iteration 11541 / 49000) loss: 1.53134634557\n",
      "(Iteration 11551 / 49000) loss: 1.54648412844\n",
      "(Iteration 11561 / 49000) loss: 1.52614709741\n",
      "(Epoch 118 / 500) Training Accuracy: 0.478428571429, Validation Accuracy: 0.433\n",
      "(Iteration 11571 / 49000) loss: 1.61661810079\n",
      "(Iteration 11581 / 49000) loss: 1.46212051894\n",
      "(Iteration 11591 / 49000) loss: 1.48656867914\n",
      "(Iteration 11601 / 49000) loss: 1.54345850675\n",
      "(Iteration 11611 / 49000) loss: 1.50967896126\n",
      "(Iteration 11621 / 49000) loss: 1.46162707929\n",
      "(Iteration 11631 / 49000) loss: 1.57165578652\n",
      "(Iteration 11641 / 49000) loss: 1.44572390937\n",
      "(Iteration 11651 / 49000) loss: 1.50688397179\n",
      "(Iteration 11661 / 49000) loss: 1.59992788196\n",
      "(Epoch 119 / 500) Training Accuracy: 0.481428571429, Validation Accuracy: 0.439\n",
      "(Iteration 11671 / 49000) loss: 1.48969240414\n",
      "(Iteration 11681 / 49000) loss: 1.50129417\n",
      "(Iteration 11691 / 49000) loss: 1.53360881033\n",
      "(Iteration 11701 / 49000) loss: 1.50477083737\n",
      "(Iteration 11711 / 49000) loss: 1.45798895566\n",
      "(Iteration 11721 / 49000) loss: 1.45020616627\n",
      "(Iteration 11731 / 49000) loss: 1.48473922238\n",
      "(Iteration 11741 / 49000) loss: 1.50875811402\n",
      "(Iteration 11751 / 49000) loss: 1.53569083672\n",
      "(Epoch 120 / 500) Training Accuracy: 0.478428571429, Validation Accuracy: 0.436\n",
      "(Iteration 11761 / 49000) loss: 1.47949639847\n",
      "(Iteration 11771 / 49000) loss: 1.51094722606\n",
      "(Iteration 11781 / 49000) loss: 1.48737554368\n",
      "(Iteration 11791 / 49000) loss: 1.51839481674\n",
      "(Iteration 11801 / 49000) loss: 1.58032665211\n",
      "(Iteration 11811 / 49000) loss: 1.55319438466\n",
      "(Iteration 11821 / 49000) loss: 1.54285780141\n",
      "(Iteration 11831 / 49000) loss: 1.43795618114\n",
      "(Iteration 11841 / 49000) loss: 1.60169996934\n",
      "(Iteration 11851 / 49000) loss: 1.55507352059\n",
      "(Epoch 121 / 500) Training Accuracy: 0.480285714286, Validation Accuracy: 0.441\n",
      "(Iteration 11861 / 49000) loss: 1.45782505041\n",
      "(Iteration 11871 / 49000) loss: 1.47497442172\n",
      "(Iteration 11881 / 49000) loss: 1.49584630524\n",
      "(Iteration 11891 / 49000) loss: 1.50285601441\n",
      "(Iteration 11901 / 49000) loss: 1.48988028184\n",
      "(Iteration 11911 / 49000) loss: 1.54183367062\n",
      "(Iteration 11921 / 49000) loss: 1.56533376272\n",
      "(Iteration 11931 / 49000) loss: 1.58476037122\n",
      "(Iteration 11941 / 49000) loss: 1.49442928662\n",
      "(Iteration 11951 / 49000) loss: 1.48593770909\n",
      "(Epoch 122 / 500) Training Accuracy: 0.484244897959, Validation Accuracy: 0.447\n",
      "(Iteration 11961 / 49000) loss: 1.56637105844\n",
      "(Iteration 11971 / 49000) loss: 1.43517968408\n",
      "(Iteration 11981 / 49000) loss: 1.52086758225\n",
      "(Iteration 11991 / 49000) loss: 1.44342979607\n",
      "(Iteration 12001 / 49000) loss: 1.45941813703\n",
      "(Iteration 12011 / 49000) loss: 1.59251028229\n",
      "(Iteration 12021 / 49000) loss: 1.48061452756\n",
      "(Iteration 12031 / 49000) loss: 1.52904472226\n",
      "(Iteration 12041 / 49000) loss: 1.47201343624\n",
      "(Iteration 12051 / 49000) loss: 1.52322056085\n",
      "(Epoch 123 / 500) Training Accuracy: 0.483469387755, Validation Accuracy: 0.456\n",
      "(Iteration 12061 / 49000) loss: 1.52499425737\n",
      "(Iteration 12071 / 49000) loss: 1.49653831939\n",
      "(Iteration 12081 / 49000) loss: 1.46156724902\n",
      "(Iteration 12091 / 49000) loss: 1.43785198394\n",
      "(Iteration 12101 / 49000) loss: 1.43814633777\n",
      "(Iteration 12111 / 49000) loss: 1.45999935601\n",
      "(Iteration 12121 / 49000) loss: 1.43967292262\n",
      "(Iteration 12131 / 49000) loss: 1.50888467111\n",
      "(Iteration 12141 / 49000) loss: 1.47697218186\n",
      "(Iteration 12151 / 49000) loss: 1.51631161079\n",
      "(Epoch 124 / 500) Training Accuracy: 0.484530612245, Validation Accuracy: 0.439\n",
      "(Iteration 12161 / 49000) loss: 1.42643015546\n",
      "(Iteration 12171 / 49000) loss: 1.58426441274\n",
      "(Iteration 12181 / 49000) loss: 1.47784860225\n",
      "(Iteration 12191 / 49000) loss: 1.56646111085\n",
      "(Iteration 12201 / 49000) loss: 1.54562767684\n",
      "(Iteration 12211 / 49000) loss: 1.52963123248\n",
      "(Iteration 12221 / 49000) loss: 1.56422180339\n",
      "(Iteration 12231 / 49000) loss: 1.50938296969\n",
      "(Iteration 12241 / 49000) loss: 1.3835444014\n",
      "(Epoch 125 / 500) Training Accuracy: 0.483530612245, Validation Accuracy: 0.444\n",
      "(Iteration 12251 / 49000) loss: 1.49273575157\n",
      "(Iteration 12261 / 49000) loss: 1.50449411165\n",
      "(Iteration 12271 / 49000) loss: 1.4518309769\n",
      "(Iteration 12281 / 49000) loss: 1.46470230924\n",
      "(Iteration 12291 / 49000) loss: 1.49516464394\n",
      "(Iteration 12301 / 49000) loss: 1.44971546651\n",
      "(Iteration 12311 / 49000) loss: 1.45509848404\n",
      "(Iteration 12321 / 49000) loss: 1.43094248821\n",
      "(Iteration 12331 / 49000) loss: 1.45007328929\n",
      "(Iteration 12341 / 49000) loss: 1.54903456836\n",
      "(Epoch 126 / 500) Training Accuracy: 0.480469387755, Validation Accuracy: 0.435\n",
      "(Iteration 12351 / 49000) loss: 1.45485972639\n",
      "(Iteration 12361 / 49000) loss: 1.43957577832\n",
      "(Iteration 12371 / 49000) loss: 1.46155535184\n",
      "(Iteration 12381 / 49000) loss: 1.53170711478\n",
      "(Iteration 12391 / 49000) loss: 1.4019593277\n",
      "(Iteration 12401 / 49000) loss: 1.42568640018\n",
      "(Iteration 12411 / 49000) loss: 1.39400122074\n",
      "(Iteration 12421 / 49000) loss: 1.45332500994\n",
      "(Iteration 12431 / 49000) loss: 1.51321697499\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 12441 / 49000) loss: 1.48643780444\n",
      "(Epoch 127 / 500) Training Accuracy: 0.478326530612, Validation Accuracy: 0.436\n",
      "(Iteration 12451 / 49000) loss: 1.42670459894\n",
      "(Iteration 12461 / 49000) loss: 1.43494843171\n",
      "(Iteration 12471 / 49000) loss: 1.44269165198\n",
      "(Iteration 12481 / 49000) loss: 1.56329364405\n",
      "(Iteration 12491 / 49000) loss: 1.44430210887\n",
      "(Iteration 12501 / 49000) loss: 1.41138355331\n",
      "(Iteration 12511 / 49000) loss: 1.5094465245\n",
      "(Iteration 12521 / 49000) loss: 1.47791179917\n",
      "(Iteration 12531 / 49000) loss: 1.44931920677\n",
      "(Iteration 12541 / 49000) loss: 1.39626696574\n",
      "(Epoch 128 / 500) Training Accuracy: 0.486448979592, Validation Accuracy: 0.445\n",
      "(Iteration 12551 / 49000) loss: 1.53426580008\n",
      "(Iteration 12561 / 49000) loss: 1.50470903146\n",
      "(Iteration 12571 / 49000) loss: 1.50057917949\n",
      "(Iteration 12581 / 49000) loss: 1.55620048639\n",
      "(Iteration 12591 / 49000) loss: 1.54673289622\n",
      "(Iteration 12601 / 49000) loss: 1.44917810928\n",
      "(Iteration 12611 / 49000) loss: 1.53370267828\n",
      "(Iteration 12621 / 49000) loss: 1.44864688139\n",
      "(Iteration 12631 / 49000) loss: 1.46002833364\n",
      "(Iteration 12641 / 49000) loss: 1.54738282335\n",
      "(Epoch 129 / 500) Training Accuracy: 0.482897959184, Validation Accuracy: 0.449\n",
      "(Iteration 12651 / 49000) loss: 1.48868736258\n",
      "(Iteration 12661 / 49000) loss: 1.44002358273\n",
      "(Iteration 12671 / 49000) loss: 1.49153583718\n",
      "(Iteration 12681 / 49000) loss: 1.45395059903\n",
      "(Iteration 12691 / 49000) loss: 1.45726900434\n",
      "(Iteration 12701 / 49000) loss: 1.53201220988\n",
      "(Iteration 12711 / 49000) loss: 1.47551000758\n",
      "(Iteration 12721 / 49000) loss: 1.52849178813\n",
      "(Iteration 12731 / 49000) loss: 1.49494832675\n",
      "(Epoch 130 / 500) Training Accuracy: 0.485306122449, Validation Accuracy: 0.437\n",
      "(Iteration 12741 / 49000) loss: 1.54705339372\n",
      "(Iteration 12751 / 49000) loss: 1.54891035931\n",
      "(Iteration 12761 / 49000) loss: 1.49617456454\n",
      "(Iteration 12771 / 49000) loss: 1.50024633286\n",
      "(Iteration 12781 / 49000) loss: 1.47524278041\n",
      "(Iteration 12791 / 49000) loss: 1.43208915672\n",
      "(Iteration 12801 / 49000) loss: 1.42925730549\n",
      "(Iteration 12811 / 49000) loss: 1.46875965744\n",
      "(Iteration 12821 / 49000) loss: 1.49666774345\n",
      "(Iteration 12831 / 49000) loss: 1.39904881172\n",
      "(Epoch 131 / 500) Training Accuracy: 0.486265306122, Validation Accuracy: 0.442\n",
      "(Iteration 12841 / 49000) loss: 1.42844755017\n",
      "(Iteration 12851 / 49000) loss: 1.45517712809\n",
      "(Iteration 12861 / 49000) loss: 1.54899289396\n",
      "(Iteration 12871 / 49000) loss: 1.44298165462\n",
      "(Iteration 12881 / 49000) loss: 1.42768691989\n",
      "(Iteration 12891 / 49000) loss: 1.42530903032\n",
      "(Iteration 12901 / 49000) loss: 1.45475334863\n",
      "(Iteration 12911 / 49000) loss: 1.50883757674\n",
      "(Iteration 12921 / 49000) loss: 1.52645550957\n",
      "(Iteration 12931 / 49000) loss: 1.52965669818\n",
      "(Epoch 132 / 500) Training Accuracy: 0.482428571429, Validation Accuracy: 0.44\n",
      "(Iteration 12941 / 49000) loss: 1.51545995601\n",
      "(Iteration 12951 / 49000) loss: 1.48646058884\n",
      "(Iteration 12961 / 49000) loss: 1.48481397156\n",
      "(Iteration 12971 / 49000) loss: 1.46946255279\n",
      "(Iteration 12981 / 49000) loss: 1.45612115318\n",
      "(Iteration 12991 / 49000) loss: 1.51529374549\n",
      "(Iteration 13001 / 49000) loss: 1.49833033319\n",
      "(Iteration 13011 / 49000) loss: 1.43115001123\n",
      "(Iteration 13021 / 49000) loss: 1.4576194463\n",
      "(Iteration 13031 / 49000) loss: 1.47972382136\n",
      "(Epoch 133 / 500) Training Accuracy: 0.483714285714, Validation Accuracy: 0.45\n",
      "(Iteration 13041 / 49000) loss: 1.45928555395\n",
      "(Iteration 13051 / 49000) loss: 1.38953087608\n",
      "(Iteration 13061 / 49000) loss: 1.49212490698\n",
      "(Iteration 13071 / 49000) loss: 1.43443060409\n",
      "(Iteration 13081 / 49000) loss: 1.55782752268\n",
      "(Iteration 13091 / 49000) loss: 1.53070369077\n",
      "(Iteration 13101 / 49000) loss: 1.43895562706\n",
      "(Iteration 13111 / 49000) loss: 1.44948559195\n",
      "(Iteration 13121 / 49000) loss: 1.41107911213\n",
      "(Iteration 13131 / 49000) loss: 1.43963458635\n",
      "(Epoch 134 / 500) Training Accuracy: 0.483408163265, Validation Accuracy: 0.45\n",
      "(Iteration 13141 / 49000) loss: 1.37546216737\n",
      "(Iteration 13151 / 49000) loss: 1.45610861772\n",
      "(Iteration 13161 / 49000) loss: 1.52287006601\n",
      "(Iteration 13171 / 49000) loss: 1.45169306101\n",
      "(Iteration 13181 / 49000) loss: 1.53219681923\n",
      "(Iteration 13191 / 49000) loss: 1.52379564941\n",
      "(Iteration 13201 / 49000) loss: 1.38457216029\n",
      "(Iteration 13211 / 49000) loss: 1.42087794176\n",
      "(Iteration 13221 / 49000) loss: 1.46595575389\n",
      "(Epoch 135 / 500) Training Accuracy: 0.485591836735, Validation Accuracy: 0.453\n",
      "(Iteration 13231 / 49000) loss: 1.44493955047\n",
      "(Iteration 13241 / 49000) loss: 1.46148995937\n",
      "(Iteration 13251 / 49000) loss: 1.43102831921\n",
      "(Iteration 13261 / 49000) loss: 1.48763152713\n",
      "(Iteration 13271 / 49000) loss: 1.5301956373\n",
      "(Iteration 13281 / 49000) loss: 1.45812466377\n",
      "(Iteration 13291 / 49000) loss: 1.44068380202\n",
      "(Iteration 13301 / 49000) loss: 1.54543777456\n",
      "(Iteration 13311 / 49000) loss: 1.40157014428\n",
      "(Iteration 13321 / 49000) loss: 1.47552515185\n",
      "(Epoch 136 / 500) Training Accuracy: 0.480326530612, Validation Accuracy: 0.458\n",
      "(Iteration 13331 / 49000) loss: 1.4735673955\n",
      "(Iteration 13341 / 49000) loss: 1.44674906468\n",
      "(Iteration 13351 / 49000) loss: 1.39456942455\n",
      "(Iteration 13361 / 49000) loss: 1.5482758621\n",
      "(Iteration 13371 / 49000) loss: 1.46649276434\n",
      "(Iteration 13381 / 49000) loss: 1.5127329577\n",
      "(Iteration 13391 / 49000) loss: 1.44578789496\n",
      "(Iteration 13401 / 49000) loss: 1.47079695488\n",
      "(Iteration 13411 / 49000) loss: 1.39756138024\n",
      "(Iteration 13421 / 49000) loss: 1.40446598002\n",
      "(Epoch 137 / 500) Training Accuracy: 0.477673469388, Validation Accuracy: 0.445\n",
      "(Iteration 13431 / 49000) loss: 1.39881526005\n",
      "(Iteration 13441 / 49000) loss: 1.50697406295\n",
      "(Iteration 13451 / 49000) loss: 1.47281446453\n",
      "(Iteration 13461 / 49000) loss: 1.48741296207\n",
      "(Iteration 13471 / 49000) loss: 1.42859999754\n",
      "(Iteration 13481 / 49000) loss: 1.50534285351\n",
      "(Iteration 13491 / 49000) loss: 1.5252615378\n",
      "(Iteration 13501 / 49000) loss: 1.48757842764\n",
      "(Iteration 13511 / 49000) loss: 1.42885782892\n",
      "(Iteration 13521 / 49000) loss: 1.46733899581\n",
      "(Epoch 138 / 500) Training Accuracy: 0.490204081633, Validation Accuracy: 0.445\n",
      "(Iteration 13531 / 49000) loss: 1.45905966051\n",
      "(Iteration 13541 / 49000) loss: 1.43905426518\n",
      "(Iteration 13551 / 49000) loss: 1.46355823114\n",
      "(Iteration 13561 / 49000) loss: 1.5314280203\n",
      "(Iteration 13571 / 49000) loss: 1.49784130747\n",
      "(Iteration 13581 / 49000) loss: 1.52264996029\n",
      "(Iteration 13591 / 49000) loss: 1.37699981779\n",
      "(Iteration 13601 / 49000) loss: 1.47055201826\n",
      "(Iteration 13611 / 49000) loss: 1.44241937314\n",
      "(Iteration 13621 / 49000) loss: 1.48321115567\n",
      "(Epoch 139 / 500) Training Accuracy: 0.487244897959, Validation Accuracy: 0.442\n",
      "(Iteration 13631 / 49000) loss: 1.47710941747\n",
      "(Iteration 13641 / 49000) loss: 1.4926869476\n",
      "(Iteration 13651 / 49000) loss: 1.49233823164\n",
      "(Iteration 13661 / 49000) loss: 1.3530067327\n",
      "(Iteration 13671 / 49000) loss: 1.42602316791\n",
      "(Iteration 13681 / 49000) loss: 1.42574676224\n",
      "(Iteration 13691 / 49000) loss: 1.46910698863\n",
      "(Iteration 13701 / 49000) loss: 1.42260343839\n",
      "(Iteration 13711 / 49000) loss: 1.50651783917\n",
      "(Epoch 140 / 500) Training Accuracy: 0.484816326531, Validation Accuracy: 0.437\n",
      "(Iteration 13721 / 49000) loss: 1.49410724063\n",
      "(Iteration 13731 / 49000) loss: 1.45786348972\n",
      "(Iteration 13741 / 49000) loss: 1.50037829369\n",
      "(Iteration 13751 / 49000) loss: 1.41219795206\n",
      "(Iteration 13761 / 49000) loss: 1.41724356053\n",
      "(Iteration 13771 / 49000) loss: 1.45112536158\n",
      "(Iteration 13781 / 49000) loss: 1.55471709334\n",
      "(Iteration 13791 / 49000) loss: 1.43990135245\n",
      "(Iteration 13801 / 49000) loss: 1.49307958591\n",
      "(Iteration 13811 / 49000) loss: 1.45818003793\n",
      "(Epoch 141 / 500) Training Accuracy: 0.490020408163, Validation Accuracy: 0.448\n",
      "(Iteration 13821 / 49000) loss: 1.41734033747\n",
      "(Iteration 13831 / 49000) loss: 1.51356741519\n",
      "(Iteration 13841 / 49000) loss: 1.50913831765\n",
      "(Iteration 13851 / 49000) loss: 1.5107960514\n",
      "(Iteration 13861 / 49000) loss: 1.53293990225\n",
      "(Iteration 13871 / 49000) loss: 1.48644749534\n",
      "(Iteration 13881 / 49000) loss: 1.47964175925\n",
      "(Iteration 13891 / 49000) loss: 1.43985726583\n",
      "(Iteration 13901 / 49000) loss: 1.52077225254\n",
      "(Iteration 13911 / 49000) loss: 1.47398688349\n",
      "(Epoch 142 / 500) Training Accuracy: 0.487204081633, Validation Accuracy: 0.45\n",
      "(Iteration 13921 / 49000) loss: 1.46181810019\n",
      "(Iteration 13931 / 49000) loss: 1.54691485333\n",
      "(Iteration 13941 / 49000) loss: 1.40269698007\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 13951 / 49000) loss: 1.45171713178\n",
      "(Iteration 13961 / 49000) loss: 1.40449778594\n",
      "(Iteration 13971 / 49000) loss: 1.51053865058\n",
      "(Iteration 13981 / 49000) loss: 1.52231559223\n",
      "(Iteration 13991 / 49000) loss: 1.49409532472\n",
      "(Iteration 14001 / 49000) loss: 1.38422909863\n",
      "(Iteration 14011 / 49000) loss: 1.45829297136\n",
      "(Epoch 143 / 500) Training Accuracy: 0.489530612245, Validation Accuracy: 0.44\n",
      "(Iteration 14021 / 49000) loss: 1.43810735182\n",
      "(Iteration 14031 / 49000) loss: 1.47591885081\n",
      "(Iteration 14041 / 49000) loss: 1.4811464164\n",
      "(Iteration 14051 / 49000) loss: 1.48121820176\n",
      "(Iteration 14061 / 49000) loss: 1.44679680212\n",
      "(Iteration 14071 / 49000) loss: 1.46594065805\n",
      "(Iteration 14081 / 49000) loss: 1.5023407814\n",
      "(Iteration 14091 / 49000) loss: 1.4647015889\n",
      "(Iteration 14101 / 49000) loss: 1.48696165297\n",
      "(Iteration 14111 / 49000) loss: 1.45739600541\n",
      "(Epoch 144 / 500) Training Accuracy: 0.490836734694, Validation Accuracy: 0.453\n",
      "(Iteration 14121 / 49000) loss: 1.47326406575\n",
      "(Iteration 14131 / 49000) loss: 1.49937365434\n",
      "(Iteration 14141 / 49000) loss: 1.48966470773\n",
      "(Iteration 14151 / 49000) loss: 1.5353000401\n",
      "(Iteration 14161 / 49000) loss: 1.49377342878\n",
      "(Iteration 14171 / 49000) loss: 1.42086987366\n",
      "(Iteration 14181 / 49000) loss: 1.50137924368\n",
      "(Iteration 14191 / 49000) loss: 1.48316653652\n",
      "(Iteration 14201 / 49000) loss: 1.50747793849\n",
      "(Epoch 145 / 500) Training Accuracy: 0.490306122449, Validation Accuracy: 0.444\n",
      "(Iteration 14211 / 49000) loss: 1.49760089738\n",
      "(Iteration 14221 / 49000) loss: 1.40975345032\n",
      "(Iteration 14231 / 49000) loss: 1.49887680939\n",
      "(Iteration 14241 / 49000) loss: 1.42074107752\n",
      "(Iteration 14251 / 49000) loss: 1.47387017079\n",
      "(Iteration 14261 / 49000) loss: 1.4290832199\n",
      "(Iteration 14271 / 49000) loss: 1.48349162018\n",
      "(Iteration 14281 / 49000) loss: 1.49117318272\n",
      "(Iteration 14291 / 49000) loss: 1.49385089587\n",
      "(Iteration 14301 / 49000) loss: 1.39872104016\n",
      "(Epoch 146 / 500) Training Accuracy: 0.481142857143, Validation Accuracy: 0.429\n",
      "(Iteration 14311 / 49000) loss: 1.46108438702\n",
      "(Iteration 14321 / 49000) loss: 1.40332454784\n",
      "(Iteration 14331 / 49000) loss: 1.45514842927\n",
      "(Iteration 14341 / 49000) loss: 1.55875936404\n",
      "(Iteration 14351 / 49000) loss: 1.58940274918\n",
      "(Iteration 14361 / 49000) loss: 1.38634125981\n",
      "(Iteration 14371 / 49000) loss: 1.39394987341\n",
      "(Iteration 14381 / 49000) loss: 1.51019964611\n",
      "(Iteration 14391 / 49000) loss: 1.50216603641\n",
      "(Iteration 14401 / 49000) loss: 1.4365630488\n",
      "(Epoch 147 / 500) Training Accuracy: 0.48393877551, Validation Accuracy: 0.452\n",
      "(Iteration 14411 / 49000) loss: 1.44608023355\n",
      "(Iteration 14421 / 49000) loss: 1.45374494932\n",
      "(Iteration 14431 / 49000) loss: 1.4694057593\n",
      "(Iteration 14441 / 49000) loss: 1.44128129323\n",
      "(Iteration 14451 / 49000) loss: 1.44580871839\n",
      "(Iteration 14461 / 49000) loss: 1.43513309276\n",
      "(Iteration 14471 / 49000) loss: 1.41671128524\n",
      "(Iteration 14481 / 49000) loss: 1.47355827737\n",
      "(Iteration 14491 / 49000) loss: 1.52800880212\n",
      "(Iteration 14501 / 49000) loss: 1.47287744735\n",
      "(Epoch 148 / 500) Training Accuracy: 0.489448979592, Validation Accuracy: 0.447\n",
      "(Iteration 14511 / 49000) loss: 1.46899852239\n",
      "(Iteration 14521 / 49000) loss: 1.49141192771\n",
      "(Iteration 14531 / 49000) loss: 1.414577091\n",
      "(Iteration 14541 / 49000) loss: 1.49399793751\n",
      "(Iteration 14551 / 49000) loss: 1.39114059273\n",
      "(Iteration 14561 / 49000) loss: 1.36659625552\n",
      "(Iteration 14571 / 49000) loss: 1.51375798998\n",
      "(Iteration 14581 / 49000) loss: 1.54730215069\n",
      "(Iteration 14591 / 49000) loss: 1.44784788668\n",
      "(Iteration 14601 / 49000) loss: 1.45807777448\n",
      "(Epoch 149 / 500) Training Accuracy: 0.492530612245, Validation Accuracy: 0.455\n",
      "(Iteration 14611 / 49000) loss: 1.50038521791\n",
      "(Iteration 14621 / 49000) loss: 1.46123252611\n",
      "(Iteration 14631 / 49000) loss: 1.4517205306\n",
      "(Iteration 14641 / 49000) loss: 1.42318568053\n",
      "(Iteration 14651 / 49000) loss: 1.4266966994\n",
      "(Iteration 14661 / 49000) loss: 1.56439343211\n",
      "(Iteration 14671 / 49000) loss: 1.47996297444\n",
      "(Iteration 14681 / 49000) loss: 1.42178462813\n",
      "(Iteration 14691 / 49000) loss: 1.51301460544\n",
      "(Epoch 150 / 500) Training Accuracy: 0.490163265306, Validation Accuracy: 0.452\n",
      "(Iteration 14701 / 49000) loss: 1.52462153799\n",
      "(Iteration 14711 / 49000) loss: 1.4604320175\n",
      "(Iteration 14721 / 49000) loss: 1.4669425217\n",
      "(Iteration 14731 / 49000) loss: 1.50774584864\n",
      "(Iteration 14741 / 49000) loss: 1.39481319328\n",
      "(Iteration 14751 / 49000) loss: 1.46527603343\n",
      "(Iteration 14761 / 49000) loss: 1.44309681837\n",
      "(Iteration 14771 / 49000) loss: 1.41521629781\n",
      "(Iteration 14781 / 49000) loss: 1.49087421955\n",
      "(Iteration 14791 / 49000) loss: 1.48243192143\n",
      "(Epoch 151 / 500) Training Accuracy: 0.492612244898, Validation Accuracy: 0.448\n",
      "(Iteration 14801 / 49000) loss: 1.41621074022\n",
      "(Iteration 14811 / 49000) loss: 1.46307211655\n",
      "(Iteration 14821 / 49000) loss: 1.4464665299\n",
      "(Iteration 14831 / 49000) loss: 1.50318956906\n",
      "(Iteration 14841 / 49000) loss: 1.51181277642\n",
      "(Iteration 14851 / 49000) loss: 1.41899592201\n",
      "(Iteration 14861 / 49000) loss: 1.45914580422\n",
      "(Iteration 14871 / 49000) loss: 1.49344801485\n",
      "(Iteration 14881 / 49000) loss: 1.50467579399\n",
      "(Iteration 14891 / 49000) loss: 1.52983768585\n",
      "(Epoch 152 / 500) Training Accuracy: 0.488714285714, Validation Accuracy: 0.446\n",
      "(Iteration 14901 / 49000) loss: 1.42780336559\n",
      "(Iteration 14911 / 49000) loss: 1.34512667843\n",
      "(Iteration 14921 / 49000) loss: 1.42320127245\n",
      "(Iteration 14931 / 49000) loss: 1.42841893533\n",
      "(Iteration 14941 / 49000) loss: 1.47682804019\n",
      "(Iteration 14951 / 49000) loss: 1.57042643364\n",
      "(Iteration 14961 / 49000) loss: 1.4715343063\n",
      "(Iteration 14971 / 49000) loss: 1.55025252672\n",
      "(Iteration 14981 / 49000) loss: 1.51077041421\n",
      "(Iteration 14991 / 49000) loss: 1.48768019436\n",
      "(Epoch 153 / 500) Training Accuracy: 0.484163265306, Validation Accuracy: 0.446\n",
      "(Iteration 15001 / 49000) loss: 1.44411972979\n",
      "(Iteration 15011 / 49000) loss: 1.45479768316\n",
      "(Iteration 15021 / 49000) loss: 1.47911865185\n",
      "(Iteration 15031 / 49000) loss: 1.41732260204\n",
      "(Iteration 15041 / 49000) loss: 1.4949026423\n",
      "(Iteration 15051 / 49000) loss: 1.5136174797\n",
      "(Iteration 15061 / 49000) loss: 1.45274328628\n",
      "(Iteration 15071 / 49000) loss: 1.52357528337\n",
      "(Iteration 15081 / 49000) loss: 1.48864215882\n",
      "(Iteration 15091 / 49000) loss: 1.43933311015\n",
      "(Epoch 154 / 500) Training Accuracy: 0.485448979592, Validation Accuracy: 0.457\n",
      "(Iteration 15101 / 49000) loss: 1.47196086\n",
      "(Iteration 15111 / 49000) loss: 1.44756150381\n",
      "(Iteration 15121 / 49000) loss: 1.45582585587\n",
      "(Iteration 15131 / 49000) loss: 1.47611545232\n",
      "(Iteration 15141 / 49000) loss: 1.50134390642\n",
      "(Iteration 15151 / 49000) loss: 1.54980804187\n",
      "(Iteration 15161 / 49000) loss: 1.46406841443\n",
      "(Iteration 15171 / 49000) loss: 1.46429126302\n",
      "(Iteration 15181 / 49000) loss: 1.43604264344\n",
      "(Epoch 155 / 500) Training Accuracy: 0.494020408163, Validation Accuracy: 0.45\n",
      "(Iteration 15191 / 49000) loss: 1.39694904172\n",
      "(Iteration 15201 / 49000) loss: 1.45857462356\n",
      "(Iteration 15211 / 49000) loss: 1.44046417789\n",
      "(Iteration 15221 / 49000) loss: 1.44859640925\n",
      "(Iteration 15231 / 49000) loss: 1.4030045753\n",
      "(Iteration 15241 / 49000) loss: 1.47609349783\n",
      "(Iteration 15251 / 49000) loss: 1.5290678859\n",
      "(Iteration 15261 / 49000) loss: 1.47535801777\n",
      "(Iteration 15271 / 49000) loss: 1.42894763658\n",
      "(Iteration 15281 / 49000) loss: 1.60879822401\n",
      "(Epoch 156 / 500) Training Accuracy: 0.484081632653, Validation Accuracy: 0.438\n",
      "(Iteration 15291 / 49000) loss: 1.48504700536\n",
      "(Iteration 15301 / 49000) loss: 1.42744593545\n",
      "(Iteration 15311 / 49000) loss: 1.44449135225\n",
      "(Iteration 15321 / 49000) loss: 1.43597994893\n",
      "(Iteration 15331 / 49000) loss: 1.51062959\n",
      "(Iteration 15341 / 49000) loss: 1.45478378765\n",
      "(Iteration 15351 / 49000) loss: 1.48968717029\n",
      "(Iteration 15361 / 49000) loss: 1.45311277958\n",
      "(Iteration 15371 / 49000) loss: 1.42657592853\n",
      "(Iteration 15381 / 49000) loss: 1.41940737854\n",
      "(Epoch 157 / 500) Training Accuracy: 0.493163265306, Validation Accuracy: 0.45\n",
      "(Iteration 15391 / 49000) loss: 1.48205028432\n",
      "(Iteration 15401 / 49000) loss: 1.38239614682\n",
      "(Iteration 15411 / 49000) loss: 1.39792938281\n",
      "(Iteration 15421 / 49000) loss: 1.59670808808\n",
      "(Iteration 15431 / 49000) loss: 1.44975689671\n",
      "(Iteration 15441 / 49000) loss: 1.46180809909\n",
      "(Iteration 15451 / 49000) loss: 1.44100557311\n",
      "(Iteration 15461 / 49000) loss: 1.49507086627\n",
      "(Iteration 15471 / 49000) loss: 1.45234109218\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 15481 / 49000) loss: 1.45095884815\n",
      "(Epoch 158 / 500) Training Accuracy: 0.481714285714, Validation Accuracy: 0.432\n",
      "(Iteration 15491 / 49000) loss: 1.4273621915\n",
      "(Iteration 15501 / 49000) loss: 1.58553778034\n",
      "(Iteration 15511 / 49000) loss: 1.49182146873\n",
      "(Iteration 15521 / 49000) loss: 1.57759488174\n",
      "(Iteration 15531 / 49000) loss: 1.5161868999\n",
      "(Iteration 15541 / 49000) loss: 1.39710390873\n",
      "(Iteration 15551 / 49000) loss: 1.45506215272\n",
      "(Iteration 15561 / 49000) loss: 1.44736209268\n",
      "(Iteration 15571 / 49000) loss: 1.45577557455\n",
      "(Iteration 15581 / 49000) loss: 1.52535840457\n",
      "(Epoch 159 / 500) Training Accuracy: 0.490959183673, Validation Accuracy: 0.447\n",
      "(Iteration 15591 / 49000) loss: 1.51277727081\n",
      "(Iteration 15601 / 49000) loss: 1.49930924498\n",
      "(Iteration 15611 / 49000) loss: 1.33841019767\n",
      "(Iteration 15621 / 49000) loss: 1.45678544829\n",
      "(Iteration 15631 / 49000) loss: 1.56242288247\n",
      "(Iteration 15641 / 49000) loss: 1.42229585227\n",
      "(Iteration 15651 / 49000) loss: 1.43945610801\n",
      "(Iteration 15661 / 49000) loss: 1.43431357605\n",
      "(Iteration 15671 / 49000) loss: 1.45711464508\n",
      "(Epoch 160 / 500) Training Accuracy: 0.493795918367, Validation Accuracy: 0.448\n",
      "(Iteration 15681 / 49000) loss: 1.36009409999\n",
      "(Iteration 15691 / 49000) loss: 1.3543821412\n",
      "(Iteration 15701 / 49000) loss: 1.54802023073\n",
      "(Iteration 15711 / 49000) loss: 1.4836280168\n",
      "(Iteration 15721 / 49000) loss: 1.49955464899\n",
      "(Iteration 15731 / 49000) loss: 1.51833236747\n",
      "(Iteration 15741 / 49000) loss: 1.38745175059\n",
      "(Iteration 15751 / 49000) loss: 1.57703482246\n",
      "(Iteration 15761 / 49000) loss: 1.4867135949\n",
      "(Iteration 15771 / 49000) loss: 1.51884307349\n",
      "(Epoch 161 / 500) Training Accuracy: 0.490367346939, Validation Accuracy: 0.448\n",
      "(Iteration 15781 / 49000) loss: 1.49235185314\n",
      "(Iteration 15791 / 49000) loss: 1.37502959376\n",
      "(Iteration 15801 / 49000) loss: 1.46709752875\n",
      "(Iteration 15811 / 49000) loss: 1.57947071458\n",
      "(Iteration 15821 / 49000) loss: 1.41041626596\n",
      "(Iteration 15831 / 49000) loss: 1.42403414177\n",
      "(Iteration 15841 / 49000) loss: 1.42946934608\n",
      "(Iteration 15851 / 49000) loss: 1.51144718687\n",
      "(Iteration 15861 / 49000) loss: 1.44035349863\n",
      "(Iteration 15871 / 49000) loss: 1.43912668752\n",
      "(Epoch 162 / 500) Training Accuracy: 0.494959183673, Validation Accuracy: 0.449\n",
      "(Iteration 15881 / 49000) loss: 1.40981203121\n",
      "(Iteration 15891 / 49000) loss: 1.38581087485\n",
      "(Iteration 15901 / 49000) loss: 1.35433026013\n",
      "(Iteration 15911 / 49000) loss: 1.43244287831\n",
      "(Iteration 15921 / 49000) loss: 1.48622642046\n",
      "(Iteration 15931 / 49000) loss: 1.38876865133\n",
      "(Iteration 15941 / 49000) loss: 1.4182161013\n",
      "(Iteration 15951 / 49000) loss: 1.49109147034\n",
      "(Iteration 15961 / 49000) loss: 1.48679063688\n",
      "(Iteration 15971 / 49000) loss: 1.48674809897\n",
      "(Epoch 163 / 500) Training Accuracy: 0.497285714286, Validation Accuracy: 0.459\n",
      "(Iteration 15981 / 49000) loss: 1.39002073978\n",
      "(Iteration 15991 / 49000) loss: 1.39283529273\n",
      "(Iteration 16001 / 49000) loss: 1.44248438635\n",
      "(Iteration 16011 / 49000) loss: 1.39120361622\n",
      "(Iteration 16021 / 49000) loss: 1.45469219799\n",
      "(Iteration 16031 / 49000) loss: 1.52242258838\n",
      "(Iteration 16041 / 49000) loss: 1.38494358317\n",
      "(Iteration 16051 / 49000) loss: 1.48451043328\n",
      "(Iteration 16061 / 49000) loss: 1.49078209846\n",
      "(Iteration 16071 / 49000) loss: 1.44579410244\n",
      "(Epoch 164 / 500) Training Accuracy: 0.490489795918, Validation Accuracy: 0.457\n",
      "(Iteration 16081 / 49000) loss: 1.43773179783\n",
      "(Iteration 16091 / 49000) loss: 1.43707779841\n",
      "(Iteration 16101 / 49000) loss: 1.53785098366\n",
      "(Iteration 16111 / 49000) loss: 1.45296640779\n",
      "(Iteration 16121 / 49000) loss: 1.44102774012\n",
      "(Iteration 16131 / 49000) loss: 1.43433731868\n",
      "(Iteration 16141 / 49000) loss: 1.43521708463\n",
      "(Iteration 16151 / 49000) loss: 1.46391349245\n",
      "(Iteration 16161 / 49000) loss: 1.42197061028\n",
      "(Epoch 165 / 500) Training Accuracy: 0.495571428571, Validation Accuracy: 0.454\n",
      "(Iteration 16171 / 49000) loss: 1.49200376493\n",
      "(Iteration 16181 / 49000) loss: 1.36637873163\n",
      "(Iteration 16191 / 49000) loss: 1.45013795168\n",
      "(Iteration 16201 / 49000) loss: 1.45738667634\n",
      "(Iteration 16211 / 49000) loss: 1.43180716229\n",
      "(Iteration 16221 / 49000) loss: 1.36374477673\n",
      "(Iteration 16231 / 49000) loss: 1.53340745438\n",
      "(Iteration 16241 / 49000) loss: 1.51269234649\n",
      "(Iteration 16251 / 49000) loss: 1.54243562052\n",
      "(Iteration 16261 / 49000) loss: 1.40774956612\n",
      "(Epoch 166 / 500) Training Accuracy: 0.494408163265, Validation Accuracy: 0.461\n",
      "(Iteration 16271 / 49000) loss: 1.50447684542\n",
      "(Iteration 16281 / 49000) loss: 1.35036157308\n",
      "(Iteration 16291 / 49000) loss: 1.48484883926\n",
      "(Iteration 16301 / 49000) loss: 1.42762481636\n",
      "(Iteration 16311 / 49000) loss: 1.44216612749\n",
      "(Iteration 16321 / 49000) loss: 1.46110716389\n",
      "(Iteration 16331 / 49000) loss: 1.46968010677\n",
      "(Iteration 16341 / 49000) loss: 1.34947853234\n",
      "(Iteration 16351 / 49000) loss: 1.50985526513\n",
      "(Iteration 16361 / 49000) loss: 1.31668863719\n",
      "(Epoch 167 / 500) Training Accuracy: 0.495428571429, Validation Accuracy: 0.452\n",
      "(Iteration 16371 / 49000) loss: 1.42780727452\n",
      "(Iteration 16381 / 49000) loss: 1.45005269111\n",
      "(Iteration 16391 / 49000) loss: 1.38289224257\n",
      "(Iteration 16401 / 49000) loss: 1.36940803566\n",
      "(Iteration 16411 / 49000) loss: 1.48133040932\n",
      "(Iteration 16421 / 49000) loss: 1.49347416042\n",
      "(Iteration 16431 / 49000) loss: 1.41923045474\n",
      "(Iteration 16441 / 49000) loss: 1.3525189225\n",
      "(Iteration 16451 / 49000) loss: 1.51599987338\n",
      "(Iteration 16461 / 49000) loss: 1.51918146325\n",
      "(Epoch 168 / 500) Training Accuracy: 0.498775510204, Validation Accuracy: 0.449\n",
      "(Iteration 16471 / 49000) loss: 1.41805084469\n",
      "(Iteration 16481 / 49000) loss: 1.50380924184\n",
      "(Iteration 16491 / 49000) loss: 1.56411945541\n",
      "(Iteration 16501 / 49000) loss: 1.40471664612\n",
      "(Iteration 16511 / 49000) loss: 1.51609311111\n",
      "(Iteration 16521 / 49000) loss: 1.44628210112\n",
      "(Iteration 16531 / 49000) loss: 1.53102862841\n",
      "(Iteration 16541 / 49000) loss: 1.51579140363\n",
      "(Iteration 16551 / 49000) loss: 1.46789116978\n",
      "(Iteration 16561 / 49000) loss: 1.41062941721\n",
      "(Epoch 169 / 500) Training Accuracy: 0.492142857143, Validation Accuracy: 0.452\n",
      "(Iteration 16571 / 49000) loss: 1.47667414276\n",
      "(Iteration 16581 / 49000) loss: 1.44576996916\n",
      "(Iteration 16591 / 49000) loss: 1.44031238649\n",
      "(Iteration 16601 / 49000) loss: 1.39629099906\n",
      "(Iteration 16611 / 49000) loss: 1.38339534287\n",
      "(Iteration 16621 / 49000) loss: 1.44764709434\n",
      "(Iteration 16631 / 49000) loss: 1.48484522647\n",
      "(Iteration 16641 / 49000) loss: 1.43498638551\n",
      "(Iteration 16651 / 49000) loss: 1.54339128423\n",
      "(Epoch 170 / 500) Training Accuracy: 0.486510204082, Validation Accuracy: 0.447\n",
      "(Iteration 16661 / 49000) loss: 1.43617726596\n",
      "(Iteration 16671 / 49000) loss: 1.46324770662\n",
      "(Iteration 16681 / 49000) loss: 1.47980472673\n",
      "(Iteration 16691 / 49000) loss: 1.42970760605\n",
      "(Iteration 16701 / 49000) loss: 1.48867142547\n",
      "(Iteration 16711 / 49000) loss: 1.52050413953\n",
      "(Iteration 16721 / 49000) loss: 1.45057200778\n",
      "(Iteration 16731 / 49000) loss: 1.45984613776\n",
      "(Iteration 16741 / 49000) loss: 1.44081659007\n",
      "(Iteration 16751 / 49000) loss: 1.45469748201\n",
      "(Epoch 171 / 500) Training Accuracy: 0.491591836735, Validation Accuracy: 0.448\n",
      "(Iteration 16761 / 49000) loss: 1.40677861738\n",
      "(Iteration 16771 / 49000) loss: 1.48169448506\n",
      "(Iteration 16781 / 49000) loss: 1.44188193616\n",
      "(Iteration 16791 / 49000) loss: 1.45450186204\n",
      "(Iteration 16801 / 49000) loss: 1.29116521876\n",
      "(Iteration 16811 / 49000) loss: 1.46608718751\n",
      "(Iteration 16821 / 49000) loss: 1.48194902859\n",
      "(Iteration 16831 / 49000) loss: 1.43864049314\n",
      "(Iteration 16841 / 49000) loss: 1.42834986387\n",
      "(Iteration 16851 / 49000) loss: 1.40454590597\n",
      "(Epoch 172 / 500) Training Accuracy: 0.500020408163, Validation Accuracy: 0.453\n",
      "(Iteration 16861 / 49000) loss: 1.49914966266\n",
      "(Iteration 16871 / 49000) loss: 1.43018061848\n",
      "(Iteration 16881 / 49000) loss: 1.43185061362\n",
      "(Iteration 16891 / 49000) loss: 1.48551553228\n",
      "(Iteration 16901 / 49000) loss: 1.37244460249\n",
      "(Iteration 16911 / 49000) loss: 1.40264509857\n",
      "(Iteration 16921 / 49000) loss: 1.49463942814\n",
      "(Iteration 16931 / 49000) loss: 1.43081689067\n",
      "(Iteration 16941 / 49000) loss: 1.46013887043\n",
      "(Iteration 16951 / 49000) loss: 1.46475255024\n",
      "(Epoch 173 / 500) Training Accuracy: 0.499530612245, Validation Accuracy: 0.448\n",
      "(Iteration 16961 / 49000) loss: 1.44043721939\n",
      "(Iteration 16971 / 49000) loss: 1.45552031732\n",
      "(Iteration 16981 / 49000) loss: 1.41222072094\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 16991 / 49000) loss: 1.44151651268\n",
      "(Iteration 17001 / 49000) loss: 1.48899911061\n",
      "(Iteration 17011 / 49000) loss: 1.44678674292\n",
      "(Iteration 17021 / 49000) loss: 1.50920375379\n",
      "(Iteration 17031 / 49000) loss: 1.4356373994\n",
      "(Iteration 17041 / 49000) loss: 1.44297023045\n",
      "(Iteration 17051 / 49000) loss: 1.43817914062\n",
      "(Epoch 174 / 500) Training Accuracy: 0.494469387755, Validation Accuracy: 0.445\n",
      "(Iteration 17061 / 49000) loss: 1.43842486322\n",
      "(Iteration 17071 / 49000) loss: 1.54683202853\n",
      "(Iteration 17081 / 49000) loss: 1.45072505011\n",
      "(Iteration 17091 / 49000) loss: 1.40000729902\n",
      "(Iteration 17101 / 49000) loss: 1.36549107832\n",
      "(Iteration 17111 / 49000) loss: 1.51426409787\n",
      "(Iteration 17121 / 49000) loss: 1.38039428665\n",
      "(Iteration 17131 / 49000) loss: 1.44384514149\n",
      "(Iteration 17141 / 49000) loss: 1.36559190751\n",
      "(Epoch 175 / 500) Training Accuracy: 0.499102040816, Validation Accuracy: 0.457\n",
      "(Iteration 17151 / 49000) loss: 1.41557320166\n",
      "(Iteration 17161 / 49000) loss: 1.38543211445\n",
      "(Iteration 17171 / 49000) loss: 1.40961378196\n",
      "(Iteration 17181 / 49000) loss: 1.41882965093\n",
      "(Iteration 17191 / 49000) loss: 1.39294716408\n",
      "(Iteration 17201 / 49000) loss: 1.4441477373\n",
      "(Iteration 17211 / 49000) loss: 1.44582331994\n",
      "(Iteration 17221 / 49000) loss: 1.45403360953\n",
      "(Iteration 17231 / 49000) loss: 1.4147581273\n",
      "(Iteration 17241 / 49000) loss: 1.49806782875\n",
      "(Epoch 176 / 500) Training Accuracy: 0.49912244898, Validation Accuracy: 0.457\n",
      "(Iteration 17251 / 49000) loss: 1.42037504023\n",
      "(Iteration 17261 / 49000) loss: 1.41142769717\n",
      "(Iteration 17271 / 49000) loss: 1.41350768745\n",
      "(Iteration 17281 / 49000) loss: 1.49395732738\n",
      "(Iteration 17291 / 49000) loss: 1.40123849008\n",
      "(Iteration 17301 / 49000) loss: 1.44383245837\n",
      "(Iteration 17311 / 49000) loss: 1.45517483675\n",
      "(Iteration 17321 / 49000) loss: 1.3765672148\n",
      "(Iteration 17331 / 49000) loss: 1.37591985594\n",
      "(Iteration 17341 / 49000) loss: 1.50657878778\n",
      "(Epoch 177 / 500) Training Accuracy: 0.49706122449, Validation Accuracy: 0.451\n",
      "(Iteration 17351 / 49000) loss: 1.39347287887\n",
      "(Iteration 17361 / 49000) loss: 1.42844890642\n",
      "(Iteration 17371 / 49000) loss: 1.44421204113\n",
      "(Iteration 17381 / 49000) loss: 1.48459047713\n",
      "(Iteration 17391 / 49000) loss: 1.45551465932\n",
      "(Iteration 17401 / 49000) loss: 1.46504546321\n",
      "(Iteration 17411 / 49000) loss: 1.45168969848\n",
      "(Iteration 17421 / 49000) loss: 1.39318926345\n",
      "(Iteration 17431 / 49000) loss: 1.3438867943\n",
      "(Iteration 17441 / 49000) loss: 1.4713064121\n",
      "(Epoch 178 / 500) Training Accuracy: 0.49993877551, Validation Accuracy: 0.457\n",
      "(Iteration 17451 / 49000) loss: 1.38818131832\n",
      "(Iteration 17461 / 49000) loss: 1.43310109986\n",
      "(Iteration 17471 / 49000) loss: 1.46259624304\n",
      "(Iteration 17481 / 49000) loss: 1.41698018921\n",
      "(Iteration 17491 / 49000) loss: 1.33118587183\n",
      "(Iteration 17501 / 49000) loss: 1.40033072424\n",
      "(Iteration 17511 / 49000) loss: 1.37612179847\n",
      "(Iteration 17521 / 49000) loss: 1.435364949\n",
      "(Iteration 17531 / 49000) loss: 1.39670571069\n",
      "(Iteration 17541 / 49000) loss: 1.44552086484\n",
      "(Epoch 179 / 500) Training Accuracy: 0.498448979592, Validation Accuracy: 0.446\n",
      "(Iteration 17551 / 49000) loss: 1.49556002251\n",
      "(Iteration 17561 / 49000) loss: 1.35702372763\n",
      "(Iteration 17571 / 49000) loss: 1.35411064216\n",
      "(Iteration 17581 / 49000) loss: 1.49524009275\n",
      "(Iteration 17591 / 49000) loss: 1.44887549182\n",
      "(Iteration 17601 / 49000) loss: 1.45175500346\n",
      "(Iteration 17611 / 49000) loss: 1.35873163337\n",
      "(Iteration 17621 / 49000) loss: 1.46875588041\n",
      "(Iteration 17631 / 49000) loss: 1.48274348603\n",
      "(Epoch 180 / 500) Training Accuracy: 0.496857142857, Validation Accuracy: 0.451\n",
      "(Iteration 17641 / 49000) loss: 1.36370240288\n",
      "(Iteration 17651 / 49000) loss: 1.44200462281\n",
      "(Iteration 17661 / 49000) loss: 1.44247697773\n",
      "(Iteration 17671 / 49000) loss: 1.49090853388\n",
      "(Iteration 17681 / 49000) loss: 1.43503775323\n",
      "(Iteration 17691 / 49000) loss: 1.41851964116\n",
      "(Iteration 17701 / 49000) loss: 1.48227428371\n",
      "(Iteration 17711 / 49000) loss: 1.41103661035\n",
      "(Iteration 17721 / 49000) loss: 1.44330971351\n",
      "(Iteration 17731 / 49000) loss: 1.43408219205\n",
      "(Epoch 181 / 500) Training Accuracy: 0.500612244898, Validation Accuracy: 0.46\n",
      "(Iteration 17741 / 49000) loss: 1.43342554889\n",
      "(Iteration 17751 / 49000) loss: 1.47878413807\n",
      "(Iteration 17761 / 49000) loss: 1.49932158695\n",
      "(Iteration 17771 / 49000) loss: 1.51058205409\n",
      "(Iteration 17781 / 49000) loss: 1.44023906386\n",
      "(Iteration 17791 / 49000) loss: 1.44918271523\n",
      "(Iteration 17801 / 49000) loss: 1.44374842966\n",
      "(Iteration 17811 / 49000) loss: 1.44879163122\n",
      "(Iteration 17821 / 49000) loss: 1.44246850315\n",
      "(Iteration 17831 / 49000) loss: 1.42436461891\n",
      "(Epoch 182 / 500) Training Accuracy: 0.495204081633, Validation Accuracy: 0.447\n",
      "(Iteration 17841 / 49000) loss: 1.52923048736\n",
      "(Iteration 17851 / 49000) loss: 1.49470446263\n",
      "(Iteration 17861 / 49000) loss: 1.45770019979\n",
      "(Iteration 17871 / 49000) loss: 1.48577358228\n",
      "(Iteration 17881 / 49000) loss: 1.42641700253\n",
      "(Iteration 17891 / 49000) loss: 1.44204149704\n",
      "(Iteration 17901 / 49000) loss: 1.47052360485\n",
      "(Iteration 17911 / 49000) loss: 1.49423041309\n",
      "(Iteration 17921 / 49000) loss: 1.51317633289\n",
      "(Iteration 17931 / 49000) loss: 1.41137803678\n",
      "(Epoch 183 / 500) Training Accuracy: 0.499857142857, Validation Accuracy: 0.448\n",
      "(Iteration 17941 / 49000) loss: 1.60616717255\n",
      "(Iteration 17951 / 49000) loss: 1.42098480701\n",
      "(Iteration 17961 / 49000) loss: 1.4907206485\n",
      "(Iteration 17971 / 49000) loss: 1.40048962797\n",
      "(Iteration 17981 / 49000) loss: 1.48146080594\n",
      "(Iteration 17991 / 49000) loss: 1.3681809514\n",
      "(Iteration 18001 / 49000) loss: 1.35929997108\n",
      "(Iteration 18011 / 49000) loss: 1.39540296022\n",
      "(Iteration 18021 / 49000) loss: 1.44484156827\n",
      "(Iteration 18031 / 49000) loss: 1.40409623136\n",
      "(Epoch 184 / 500) Training Accuracy: 0.502612244898, Validation Accuracy: 0.451\n",
      "(Iteration 18041 / 49000) loss: 1.40515572771\n",
      "(Iteration 18051 / 49000) loss: 1.44022980237\n",
      "(Iteration 18061 / 49000) loss: 1.42435390796\n",
      "(Iteration 18071 / 49000) loss: 1.40010508858\n",
      "(Iteration 18081 / 49000) loss: 1.38180319708\n",
      "(Iteration 18091 / 49000) loss: 1.48067188768\n",
      "(Iteration 18101 / 49000) loss: 1.45509814604\n",
      "(Iteration 18111 / 49000) loss: 1.43373900852\n",
      "(Iteration 18121 / 49000) loss: 1.39772518993\n",
      "(Epoch 185 / 500) Training Accuracy: 0.49593877551, Validation Accuracy: 0.45\n",
      "(Iteration 18131 / 49000) loss: 1.42039640452\n",
      "(Iteration 18141 / 49000) loss: 1.36460300662\n",
      "(Iteration 18151 / 49000) loss: 1.37692235225\n",
      "(Iteration 18161 / 49000) loss: 1.3943941651\n",
      "(Iteration 18171 / 49000) loss: 1.47226919919\n",
      "(Iteration 18181 / 49000) loss: 1.51145358139\n",
      "(Iteration 18191 / 49000) loss: 1.43954451853\n",
      "(Iteration 18201 / 49000) loss: 1.43633066448\n",
      "(Iteration 18211 / 49000) loss: 1.46545788809\n",
      "(Iteration 18221 / 49000) loss: 1.3699729474\n",
      "(Epoch 186 / 500) Training Accuracy: 0.503020408163, Validation Accuracy: 0.457\n",
      "(Iteration 18231 / 49000) loss: 1.37564415648\n",
      "(Iteration 18241 / 49000) loss: 1.41721927675\n",
      "(Iteration 18251 / 49000) loss: 1.48661222971\n",
      "(Iteration 18261 / 49000) loss: 1.40927936547\n",
      "(Iteration 18271 / 49000) loss: 1.4506763884\n",
      "(Iteration 18281 / 49000) loss: 1.48006496444\n",
      "(Iteration 18291 / 49000) loss: 1.34103033337\n",
      "(Iteration 18301 / 49000) loss: 1.41492633497\n",
      "(Iteration 18311 / 49000) loss: 1.53487538252\n",
      "(Iteration 18321 / 49000) loss: 1.33872774855\n",
      "(Epoch 187 / 500) Training Accuracy: 0.503734693878, Validation Accuracy: 0.452\n",
      "(Iteration 18331 / 49000) loss: 1.45564736476\n",
      "(Iteration 18341 / 49000) loss: 1.38303549732\n",
      "(Iteration 18351 / 49000) loss: 1.44288444421\n",
      "(Iteration 18361 / 49000) loss: 1.33464233527\n",
      "(Iteration 18371 / 49000) loss: 1.4374748211\n",
      "(Iteration 18381 / 49000) loss: 1.44465572499\n",
      "(Iteration 18391 / 49000) loss: 1.42717204452\n",
      "(Iteration 18401 / 49000) loss: 1.44725348482\n",
      "(Iteration 18411 / 49000) loss: 1.42632762337\n",
      "(Iteration 18421 / 49000) loss: 1.43485515459\n",
      "(Epoch 188 / 500) Training Accuracy: 0.496183673469, Validation Accuracy: 0.437\n",
      "(Iteration 18431 / 49000) loss: 1.48496140244\n",
      "(Iteration 18441 / 49000) loss: 1.44761961492\n",
      "(Iteration 18451 / 49000) loss: 1.39990879207\n",
      "(Iteration 18461 / 49000) loss: 1.45802129974\n",
      "(Iteration 18471 / 49000) loss: 1.41252264066\n",
      "(Iteration 18481 / 49000) loss: 1.3797577691\n",
      "(Iteration 18491 / 49000) loss: 1.36938239755\n",
      "(Iteration 18501 / 49000) loss: 1.47561787668\n",
      "(Iteration 18511 / 49000) loss: 1.46931931994\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 18521 / 49000) loss: 1.41488727194\n",
      "(Epoch 189 / 500) Training Accuracy: 0.500469387755, Validation Accuracy: 0.454\n",
      "(Iteration 18531 / 49000) loss: 1.41479143521\n",
      "(Iteration 18541 / 49000) loss: 1.33909428582\n",
      "(Iteration 18551 / 49000) loss: 1.45571506943\n",
      "(Iteration 18561 / 49000) loss: 1.55650982797\n",
      "(Iteration 18571 / 49000) loss: 1.41177975769\n",
      "(Iteration 18581 / 49000) loss: 1.41128190873\n",
      "(Iteration 18591 / 49000) loss: 1.41219482777\n",
      "(Iteration 18601 / 49000) loss: 1.49514232665\n",
      "(Iteration 18611 / 49000) loss: 1.46026050723\n",
      "(Epoch 190 / 500) Training Accuracy: 0.498734693878, Validation Accuracy: 0.452\n",
      "(Iteration 18621 / 49000) loss: 1.41028734172\n",
      "(Iteration 18631 / 49000) loss: 1.43231561716\n",
      "(Iteration 18641 / 49000) loss: 1.52816664268\n",
      "(Iteration 18651 / 49000) loss: 1.44889315319\n",
      "(Iteration 18661 / 49000) loss: 1.47392546889\n",
      "(Iteration 18671 / 49000) loss: 1.48896780349\n",
      "(Iteration 18681 / 49000) loss: 1.38113684658\n",
      "(Iteration 18691 / 49000) loss: 1.46012323212\n",
      "(Iteration 18701 / 49000) loss: 1.4437857072\n",
      "(Iteration 18711 / 49000) loss: 1.47947635446\n",
      "(Epoch 191 / 500) Training Accuracy: 0.503775510204, Validation Accuracy: 0.464\n",
      "(Iteration 18721 / 49000) loss: 1.46143946289\n",
      "(Iteration 18731 / 49000) loss: 1.43546995343\n",
      "(Iteration 18741 / 49000) loss: 1.50606833937\n",
      "(Iteration 18751 / 49000) loss: 1.45911407147\n",
      "(Iteration 18761 / 49000) loss: 1.42557810193\n",
      "(Iteration 18771 / 49000) loss: 1.48105436057\n",
      "(Iteration 18781 / 49000) loss: 1.40298879858\n",
      "(Iteration 18791 / 49000) loss: 1.46955554202\n",
      "(Iteration 18801 / 49000) loss: 1.37358449947\n",
      "(Iteration 18811 / 49000) loss: 1.39991861096\n",
      "(Epoch 192 / 500) Training Accuracy: 0.498265306122, Validation Accuracy: 0.454\n",
      "(Iteration 18821 / 49000) loss: 1.44686878089\n",
      "(Iteration 18831 / 49000) loss: 1.32680927495\n",
      "(Iteration 18841 / 49000) loss: 1.39671746043\n",
      "(Iteration 18851 / 49000) loss: 1.39597722886\n",
      "(Iteration 18861 / 49000) loss: 1.37946448652\n",
      "(Iteration 18871 / 49000) loss: 1.54474938972\n",
      "(Iteration 18881 / 49000) loss: 1.5215033621\n",
      "(Iteration 18891 / 49000) loss: 1.46624133132\n",
      "(Iteration 18901 / 49000) loss: 1.42909262786\n",
      "(Iteration 18911 / 49000) loss: 1.34698174948\n",
      "(Epoch 193 / 500) Training Accuracy: 0.503714285714, Validation Accuracy: 0.455\n",
      "(Iteration 18921 / 49000) loss: 1.45911536002\n",
      "(Iteration 18931 / 49000) loss: 1.49542221175\n",
      "(Iteration 18941 / 49000) loss: 1.43847754671\n",
      "(Iteration 18951 / 49000) loss: 1.4914908844\n",
      "(Iteration 18961 / 49000) loss: 1.43525089201\n",
      "(Iteration 18971 / 49000) loss: 1.50535492123\n",
      "(Iteration 18981 / 49000) loss: 1.36276760913\n",
      "(Iteration 18991 / 49000) loss: 1.51381938367\n",
      "(Iteration 19001 / 49000) loss: 1.47911352578\n",
      "(Iteration 19011 / 49000) loss: 1.46009348165\n",
      "(Epoch 194 / 500) Training Accuracy: 0.503469387755, Validation Accuracy: 0.45\n",
      "(Iteration 19021 / 49000) loss: 1.37066677487\n",
      "(Iteration 19031 / 49000) loss: 1.47995851896\n",
      "(Iteration 19041 / 49000) loss: 1.46591393547\n",
      "(Iteration 19051 / 49000) loss: 1.41686545564\n",
      "(Iteration 19061 / 49000) loss: 1.45864220679\n",
      "(Iteration 19071 / 49000) loss: 1.46975581797\n",
      "(Iteration 19081 / 49000) loss: 1.45700445231\n",
      "(Iteration 19091 / 49000) loss: 1.40267987247\n",
      "(Iteration 19101 / 49000) loss: 1.40716317081\n",
      "(Epoch 195 / 500) Training Accuracy: 0.508244897959, Validation Accuracy: 0.452\n",
      "(Iteration 19111 / 49000) loss: 1.39458274936\n",
      "(Iteration 19121 / 49000) loss: 1.35645672515\n",
      "(Iteration 19131 / 49000) loss: 1.39441181615\n",
      "(Iteration 19141 / 49000) loss: 1.40978891442\n",
      "(Iteration 19151 / 49000) loss: 1.39127783285\n",
      "(Iteration 19161 / 49000) loss: 1.44017225012\n",
      "(Iteration 19171 / 49000) loss: 1.43555364583\n",
      "(Iteration 19181 / 49000) loss: 1.38499074246\n",
      "(Iteration 19191 / 49000) loss: 1.43165854007\n",
      "(Iteration 19201 / 49000) loss: 1.36038952037\n",
      "(Epoch 196 / 500) Training Accuracy: 0.503693877551, Validation Accuracy: 0.443\n",
      "(Iteration 19211 / 49000) loss: 1.37976572353\n",
      "(Iteration 19221 / 49000) loss: 1.47570271191\n",
      "(Iteration 19231 / 49000) loss: 1.40380097335\n",
      "(Iteration 19241 / 49000) loss: 1.42430738764\n",
      "(Iteration 19251 / 49000) loss: 1.41991838689\n",
      "(Iteration 19261 / 49000) loss: 1.40886541378\n",
      "(Iteration 19271 / 49000) loss: 1.36929547055\n",
      "(Iteration 19281 / 49000) loss: 1.4377451511\n",
      "(Iteration 19291 / 49000) loss: 1.41526221561\n",
      "(Iteration 19301 / 49000) loss: 1.40126965652\n",
      "(Epoch 197 / 500) Training Accuracy: 0.505775510204, Validation Accuracy: 0.457\n",
      "(Iteration 19311 / 49000) loss: 1.42710196756\n",
      "(Iteration 19321 / 49000) loss: 1.48998239977\n",
      "(Iteration 19331 / 49000) loss: 1.37843744047\n",
      "(Iteration 19341 / 49000) loss: 1.45906584556\n",
      "(Iteration 19351 / 49000) loss: 1.53581329852\n",
      "(Iteration 19361 / 49000) loss: 1.39406128945\n",
      "(Iteration 19371 / 49000) loss: 1.33778230453\n",
      "(Iteration 19381 / 49000) loss: 1.43963029882\n",
      "(Iteration 19391 / 49000) loss: 1.43791657172\n",
      "(Iteration 19401 / 49000) loss: 1.33419740269\n",
      "(Epoch 198 / 500) Training Accuracy: 0.504775510204, Validation Accuracy: 0.452\n",
      "(Iteration 19411 / 49000) loss: 1.37604470631\n",
      "(Iteration 19421 / 49000) loss: 1.46453547851\n",
      "(Iteration 19431 / 49000) loss: 1.38720150355\n",
      "(Iteration 19441 / 49000) loss: 1.43860911391\n",
      "(Iteration 19451 / 49000) loss: 1.40656007353\n",
      "(Iteration 19461 / 49000) loss: 1.4193772689\n",
      "(Iteration 19471 / 49000) loss: 1.37324673445\n",
      "(Iteration 19481 / 49000) loss: 1.40258583185\n",
      "(Iteration 19491 / 49000) loss: 1.48800803112\n",
      "(Iteration 19501 / 49000) loss: 1.46021182326\n",
      "(Epoch 199 / 500) Training Accuracy: 0.504265306122, Validation Accuracy: 0.452\n",
      "(Iteration 19511 / 49000) loss: 1.40705404065\n",
      "(Iteration 19521 / 49000) loss: 1.42293485711\n",
      "(Iteration 19531 / 49000) loss: 1.37008662397\n",
      "(Iteration 19541 / 49000) loss: 1.47449918803\n",
      "(Iteration 19551 / 49000) loss: 1.4370447605\n",
      "(Iteration 19561 / 49000) loss: 1.45621178908\n",
      "(Iteration 19571 / 49000) loss: 1.45784009865\n",
      "(Iteration 19581 / 49000) loss: 1.42417265963\n",
      "(Iteration 19591 / 49000) loss: 1.45014236725\n",
      "(Epoch 200 / 500) Training Accuracy: 0.494510204082, Validation Accuracy: 0.451\n",
      "(Iteration 19601 / 49000) loss: 1.42175124422\n",
      "(Iteration 19611 / 49000) loss: 1.4677266397\n",
      "(Iteration 19621 / 49000) loss: 1.4597678105\n",
      "(Iteration 19631 / 49000) loss: 1.5444121343\n",
      "(Iteration 19641 / 49000) loss: 1.49466672037\n",
      "(Iteration 19651 / 49000) loss: 1.52928842907\n",
      "(Iteration 19661 / 49000) loss: 1.44067553828\n",
      "(Iteration 19671 / 49000) loss: 1.39165034383\n",
      "(Iteration 19681 / 49000) loss: 1.42550050316\n",
      "(Iteration 19691 / 49000) loss: 1.39543051141\n",
      "(Epoch 201 / 500) Training Accuracy: 0.506673469388, Validation Accuracy: 0.465\n",
      "(Iteration 19701 / 49000) loss: 1.37696250763\n",
      "(Iteration 19711 / 49000) loss: 1.42091029797\n",
      "(Iteration 19721 / 49000) loss: 1.51717232034\n",
      "(Iteration 19731 / 49000) loss: 1.46287637417\n",
      "(Iteration 19741 / 49000) loss: 1.33996405428\n",
      "(Iteration 19751 / 49000) loss: 1.47254242126\n",
      "(Iteration 19761 / 49000) loss: 1.42576815651\n",
      "(Iteration 19771 / 49000) loss: 1.39436440637\n",
      "(Iteration 19781 / 49000) loss: 1.44171362643\n",
      "(Iteration 19791 / 49000) loss: 1.43776929529\n",
      "(Epoch 202 / 500) Training Accuracy: 0.506571428571, Validation Accuracy: 0.459\n",
      "(Iteration 19801 / 49000) loss: 1.48021252654\n",
      "(Iteration 19811 / 49000) loss: 1.51993360042\n",
      "(Iteration 19821 / 49000) loss: 1.41062603887\n",
      "(Iteration 19831 / 49000) loss: 1.4163254701\n",
      "(Iteration 19841 / 49000) loss: 1.47132471421\n",
      "(Iteration 19851 / 49000) loss: 1.48262182744\n",
      "(Iteration 19861 / 49000) loss: 1.40854104366\n",
      "(Iteration 19871 / 49000) loss: 1.42144053513\n",
      "(Iteration 19881 / 49000) loss: 1.5489191504\n",
      "(Iteration 19891 / 49000) loss: 1.53206792074\n",
      "(Epoch 203 / 500) Training Accuracy: 0.510163265306, Validation Accuracy: 0.457\n",
      "(Iteration 19901 / 49000) loss: 1.39749186305\n",
      "(Iteration 19911 / 49000) loss: 1.31008283817\n",
      "(Iteration 19921 / 49000) loss: 1.38474331399\n",
      "(Iteration 19931 / 49000) loss: 1.40161935801\n",
      "(Iteration 19941 / 49000) loss: 1.38126687753\n",
      "(Iteration 19951 / 49000) loss: 1.53602264746\n",
      "(Iteration 19961 / 49000) loss: 1.33297476907\n",
      "(Iteration 19971 / 49000) loss: 1.37982548242\n",
      "(Iteration 19981 / 49000) loss: 1.40168840999\n",
      "(Iteration 19991 / 49000) loss: 1.39839568985\n",
      "(Epoch 204 / 500) Training Accuracy: 0.503530612245, Validation Accuracy: 0.458\n",
      "(Iteration 20001 / 49000) loss: 1.32941300853\n",
      "(Iteration 20011 / 49000) loss: 1.39870393666\n",
      "(Iteration 20021 / 49000) loss: 1.37866467668\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 20031 / 49000) loss: 1.38313445526\n",
      "(Iteration 20041 / 49000) loss: 1.41157534938\n",
      "(Iteration 20051 / 49000) loss: 1.38315929274\n",
      "(Iteration 20061 / 49000) loss: 1.34617178267\n",
      "(Iteration 20071 / 49000) loss: 1.4123857225\n",
      "(Iteration 20081 / 49000) loss: 1.42524599184\n",
      "(Epoch 205 / 500) Training Accuracy: 0.509693877551, Validation Accuracy: 0.462\n",
      "(Iteration 20091 / 49000) loss: 1.43176327266\n",
      "(Iteration 20101 / 49000) loss: 1.33503621249\n",
      "(Iteration 20111 / 49000) loss: 1.32928680357\n",
      "(Iteration 20121 / 49000) loss: 1.37529772895\n",
      "(Iteration 20131 / 49000) loss: 1.44819395512\n",
      "(Iteration 20141 / 49000) loss: 1.45491551231\n",
      "(Iteration 20151 / 49000) loss: 1.42191370352\n",
      "(Iteration 20161 / 49000) loss: 1.38689117618\n",
      "(Iteration 20171 / 49000) loss: 1.35929561602\n",
      "(Iteration 20181 / 49000) loss: 1.38870365123\n",
      "(Epoch 206 / 500) Training Accuracy: 0.507428571429, Validation Accuracy: 0.459\n",
      "(Iteration 20191 / 49000) loss: 1.4269145779\n",
      "(Iteration 20201 / 49000) loss: 1.52617429577\n",
      "(Iteration 20211 / 49000) loss: 1.35196591305\n",
      "(Iteration 20221 / 49000) loss: 1.43388354428\n",
      "(Iteration 20231 / 49000) loss: 1.49549763479\n",
      "(Iteration 20241 / 49000) loss: 1.49682824498\n",
      "(Iteration 20251 / 49000) loss: 1.44526771602\n",
      "(Iteration 20261 / 49000) loss: 1.38446995592\n",
      "(Iteration 20271 / 49000) loss: 1.43525367699\n",
      "(Iteration 20281 / 49000) loss: 1.39275718033\n",
      "(Epoch 207 / 500) Training Accuracy: 0.506979591837, Validation Accuracy: 0.46\n",
      "(Iteration 20291 / 49000) loss: 1.35112283654\n",
      "(Iteration 20301 / 49000) loss: 1.38445498631\n",
      "(Iteration 20311 / 49000) loss: 1.33984393001\n",
      "(Iteration 20321 / 49000) loss: 1.39880136478\n",
      "(Iteration 20331 / 49000) loss: 1.45670797497\n",
      "(Iteration 20341 / 49000) loss: 1.44042419331\n",
      "(Iteration 20351 / 49000) loss: 1.33279090364\n",
      "(Iteration 20361 / 49000) loss: 1.39712206645\n",
      "(Iteration 20371 / 49000) loss: 1.41028758848\n",
      "(Iteration 20381 / 49000) loss: 1.42038737678\n",
      "(Epoch 208 / 500) Training Accuracy: 0.508020408163, Validation Accuracy: 0.452\n",
      "(Iteration 20391 / 49000) loss: 1.42170795628\n",
      "(Iteration 20401 / 49000) loss: 1.43619293755\n",
      "(Iteration 20411 / 49000) loss: 1.4180848241\n",
      "(Iteration 20421 / 49000) loss: 1.3959340806\n",
      "(Iteration 20431 / 49000) loss: 1.42964876605\n",
      "(Iteration 20441 / 49000) loss: 1.47318565425\n",
      "(Iteration 20451 / 49000) loss: 1.35990267852\n",
      "(Iteration 20461 / 49000) loss: 1.36799436323\n",
      "(Iteration 20471 / 49000) loss: 1.43257089911\n",
      "(Iteration 20481 / 49000) loss: 1.39127243973\n",
      "(Epoch 209 / 500) Training Accuracy: 0.509142857143, Validation Accuracy: 0.446\n",
      "(Iteration 20491 / 49000) loss: 1.43024838077\n",
      "(Iteration 20501 / 49000) loss: 1.4076952846\n",
      "(Iteration 20511 / 49000) loss: 1.37676792319\n",
      "(Iteration 20521 / 49000) loss: 1.4140340962\n",
      "(Iteration 20531 / 49000) loss: 1.41935206328\n",
      "(Iteration 20541 / 49000) loss: 1.47174509104\n",
      "(Iteration 20551 / 49000) loss: 1.44701561014\n",
      "(Iteration 20561 / 49000) loss: 1.43076235372\n",
      "(Iteration 20571 / 49000) loss: 1.48850999275\n",
      "(Epoch 210 / 500) Training Accuracy: 0.509040816327, Validation Accuracy: 0.458\n",
      "(Iteration 20581 / 49000) loss: 1.39584997472\n",
      "(Iteration 20591 / 49000) loss: 1.44575715787\n",
      "(Iteration 20601 / 49000) loss: 1.39319401339\n",
      "(Iteration 20611 / 49000) loss: 1.46502165465\n",
      "(Iteration 20621 / 49000) loss: 1.43286478493\n",
      "(Iteration 20631 / 49000) loss: 1.44600346154\n",
      "(Iteration 20641 / 49000) loss: 1.45602904852\n",
      "(Iteration 20651 / 49000) loss: 1.39674212652\n",
      "(Iteration 20661 / 49000) loss: 1.38796720308\n",
      "(Iteration 20671 / 49000) loss: 1.43584732198\n",
      "(Epoch 211 / 500) Training Accuracy: 0.502265306122, Validation Accuracy: 0.448\n",
      "(Iteration 20681 / 49000) loss: 1.42083590624\n",
      "(Iteration 20691 / 49000) loss: 1.4287680636\n",
      "(Iteration 20701 / 49000) loss: 1.36737114379\n",
      "(Iteration 20711 / 49000) loss: 1.36431088413\n",
      "(Iteration 20721 / 49000) loss: 1.41975203341\n",
      "(Iteration 20731 / 49000) loss: 1.42364045336\n",
      "(Iteration 20741 / 49000) loss: 1.44573845169\n",
      "(Iteration 20751 / 49000) loss: 1.47895248854\n",
      "(Iteration 20761 / 49000) loss: 1.33774339428\n",
      "(Iteration 20771 / 49000) loss: 1.46632732021\n",
      "(Epoch 212 / 500) Training Accuracy: 0.50887755102, Validation Accuracy: 0.454\n",
      "(Iteration 20781 / 49000) loss: 1.47670492746\n",
      "(Iteration 20791 / 49000) loss: 1.53997500856\n",
      "(Iteration 20801 / 49000) loss: 1.39477109468\n",
      "(Iteration 20811 / 49000) loss: 1.43818915749\n",
      "(Iteration 20821 / 49000) loss: 1.51897346498\n",
      "(Iteration 20831 / 49000) loss: 1.4082526079\n",
      "(Iteration 20841 / 49000) loss: 1.43088096522\n",
      "(Iteration 20851 / 49000) loss: 1.42111804591\n",
      "(Iteration 20861 / 49000) loss: 1.48607727011\n",
      "(Iteration 20871 / 49000) loss: 1.44453487787\n",
      "(Epoch 213 / 500) Training Accuracy: 0.511653061224, Validation Accuracy: 0.458\n",
      "(Iteration 20881 / 49000) loss: 1.38671070375\n",
      "(Iteration 20891 / 49000) loss: 1.40000293213\n",
      "(Iteration 20901 / 49000) loss: 1.36339140141\n",
      "(Iteration 20911 / 49000) loss: 1.47021543338\n",
      "(Iteration 20921 / 49000) loss: 1.45401678615\n",
      "(Iteration 20931 / 49000) loss: 1.45649542595\n",
      "(Iteration 20941 / 49000) loss: 1.3976024829\n",
      "(Iteration 20951 / 49000) loss: 1.4209295803\n",
      "(Iteration 20961 / 49000) loss: 1.39286742186\n",
      "(Iteration 20971 / 49000) loss: 1.39488033582\n",
      "(Epoch 214 / 500) Training Accuracy: 0.506836734694, Validation Accuracy: 0.452\n",
      "(Iteration 20981 / 49000) loss: 1.41445450201\n",
      "(Iteration 20991 / 49000) loss: 1.4433353562\n",
      "(Iteration 21001 / 49000) loss: 1.43445614113\n",
      "(Iteration 21011 / 49000) loss: 1.33986419616\n",
      "(Iteration 21021 / 49000) loss: 1.45601481019\n",
      "(Iteration 21031 / 49000) loss: 1.38320624906\n",
      "(Iteration 21041 / 49000) loss: 1.39242681118\n",
      "(Iteration 21051 / 49000) loss: 1.36625353946\n",
      "(Iteration 21061 / 49000) loss: 1.38113436878\n",
      "(Epoch 215 / 500) Training Accuracy: 0.50812244898, Validation Accuracy: 0.461\n",
      "(Iteration 21071 / 49000) loss: 1.40865403644\n",
      "(Iteration 21081 / 49000) loss: 1.437987182\n",
      "(Iteration 21091 / 49000) loss: 1.40252163561\n",
      "(Iteration 21101 / 49000) loss: 1.4377875529\n",
      "(Iteration 21111 / 49000) loss: 1.41751149406\n",
      "(Iteration 21121 / 49000) loss: 1.4797427336\n",
      "(Iteration 21131 / 49000) loss: 1.34215489424\n",
      "(Iteration 21141 / 49000) loss: 1.42460954997\n",
      "(Iteration 21151 / 49000) loss: 1.42545913771\n",
      "(Iteration 21161 / 49000) loss: 1.40619834437\n",
      "(Epoch 216 / 500) Training Accuracy: 0.509244897959, Validation Accuracy: 0.455\n",
      "(Iteration 21171 / 49000) loss: 1.33575449174\n",
      "(Iteration 21181 / 49000) loss: 1.37561714274\n",
      "(Iteration 21191 / 49000) loss: 1.36600606049\n",
      "(Iteration 21201 / 49000) loss: 1.41475208393\n",
      "(Iteration 21211 / 49000) loss: 1.53650442452\n",
      "(Iteration 21221 / 49000) loss: 1.46266879106\n",
      "(Iteration 21231 / 49000) loss: 1.45118999553\n",
      "(Iteration 21241 / 49000) loss: 1.45586852094\n",
      "(Iteration 21251 / 49000) loss: 1.37831782148\n",
      "(Iteration 21261 / 49000) loss: 1.35472465547\n",
      "(Epoch 217 / 500) Training Accuracy: 0.510163265306, Validation Accuracy: 0.472\n",
      "(Iteration 21271 / 49000) loss: 1.37328231137\n",
      "(Iteration 21281 / 49000) loss: 1.45224661324\n",
      "(Iteration 21291 / 49000) loss: 1.35055095831\n",
      "(Iteration 21301 / 49000) loss: 1.34083427634\n",
      "(Iteration 21311 / 49000) loss: 1.34744614844\n",
      "(Iteration 21321 / 49000) loss: 1.48230387028\n",
      "(Iteration 21331 / 49000) loss: 1.4042170011\n",
      "(Iteration 21341 / 49000) loss: 1.43649358797\n",
      "(Iteration 21351 / 49000) loss: 1.40775184802\n",
      "(Iteration 21361 / 49000) loss: 1.31539496462\n",
      "(Epoch 218 / 500) Training Accuracy: 0.508020408163, Validation Accuracy: 0.455\n",
      "(Iteration 21371 / 49000) loss: 1.33118206863\n",
      "(Iteration 21381 / 49000) loss: 1.4811518357\n",
      "(Iteration 21391 / 49000) loss: 1.29343602047\n",
      "(Iteration 21401 / 49000) loss: 1.41415792287\n",
      "(Iteration 21411 / 49000) loss: 1.3391856386\n",
      "(Iteration 21421 / 49000) loss: 1.48210137532\n",
      "(Iteration 21431 / 49000) loss: 1.423409738\n",
      "(Iteration 21441 / 49000) loss: 1.39784001742\n",
      "(Iteration 21451 / 49000) loss: 1.46912627202\n",
      "(Iteration 21461 / 49000) loss: 1.45797269829\n",
      "(Epoch 219 / 500) Training Accuracy: 0.514775510204, Validation Accuracy: 0.452\n",
      "(Iteration 21471 / 49000) loss: 1.37896500747\n",
      "(Iteration 21481 / 49000) loss: 1.42677535332\n",
      "(Iteration 21491 / 49000) loss: 1.46125482665\n",
      "(Iteration 21501 / 49000) loss: 1.38663492565\n",
      "(Iteration 21511 / 49000) loss: 1.46050881131\n",
      "(Iteration 21521 / 49000) loss: 1.39089996178\n",
      "(Iteration 21531 / 49000) loss: 1.42851605525\n",
      "(Iteration 21541 / 49000) loss: 1.36135126077\n",
      "(Iteration 21551 / 49000) loss: 1.41123043021\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Epoch 220 / 500) Training Accuracy: 0.509142857143, Validation Accuracy: 0.459\n",
      "(Iteration 21561 / 49000) loss: 1.42394679516\n",
      "(Iteration 21571 / 49000) loss: 1.44439296199\n",
      "(Iteration 21581 / 49000) loss: 1.45726395788\n",
      "(Iteration 21591 / 49000) loss: 1.30447904224\n",
      "(Iteration 21601 / 49000) loss: 1.38194353898\n",
      "(Iteration 21611 / 49000) loss: 1.39000901016\n",
      "(Iteration 21621 / 49000) loss: 1.50712226211\n",
      "(Iteration 21631 / 49000) loss: 1.46017720355\n",
      "(Iteration 21641 / 49000) loss: 1.40115884576\n",
      "(Iteration 21651 / 49000) loss: 1.36040056091\n",
      "(Epoch 221 / 500) Training Accuracy: 0.513836734694, Validation Accuracy: 0.464\n",
      "(Iteration 21661 / 49000) loss: 1.38272181438\n",
      "(Iteration 21671 / 49000) loss: 1.39657077623\n",
      "(Iteration 21681 / 49000) loss: 1.53064472118\n",
      "(Iteration 21691 / 49000) loss: 1.39272736986\n",
      "(Iteration 21701 / 49000) loss: 1.4245111203\n",
      "(Iteration 21711 / 49000) loss: 1.44922021831\n",
      "(Iteration 21721 / 49000) loss: 1.41126154097\n",
      "(Iteration 21731 / 49000) loss: 1.36802834823\n",
      "(Iteration 21741 / 49000) loss: 1.37531974389\n",
      "(Iteration 21751 / 49000) loss: 1.44497712525\n",
      "(Epoch 222 / 500) Training Accuracy: 0.505224489796, Validation Accuracy: 0.462\n",
      "(Iteration 21761 / 49000) loss: 1.30232259004\n",
      "(Iteration 21771 / 49000) loss: 1.39921815677\n",
      "(Iteration 21781 / 49000) loss: 1.42246587626\n",
      "(Iteration 21791 / 49000) loss: 1.33644169999\n",
      "(Iteration 21801 / 49000) loss: 1.42921826481\n",
      "(Iteration 21811 / 49000) loss: 1.49363861652\n",
      "(Iteration 21821 / 49000) loss: 1.40450447496\n",
      "(Iteration 21831 / 49000) loss: 1.40880590743\n",
      "(Iteration 21841 / 49000) loss: 1.43320638818\n",
      "(Iteration 21851 / 49000) loss: 1.43243767373\n",
      "(Epoch 223 / 500) Training Accuracy: 0.511346938776, Validation Accuracy: 0.463\n",
      "(Iteration 21861 / 49000) loss: 1.37455348922\n",
      "(Iteration 21871 / 49000) loss: 1.41500762422\n",
      "(Iteration 21881 / 49000) loss: 1.42519170954\n",
      "(Iteration 21891 / 49000) loss: 1.33929503582\n",
      "(Iteration 21901 / 49000) loss: 1.37680736548\n",
      "(Iteration 21911 / 49000) loss: 1.4878084795\n",
      "(Iteration 21921 / 49000) loss: 1.41033761889\n",
      "(Iteration 21931 / 49000) loss: 1.41271006521\n",
      "(Iteration 21941 / 49000) loss: 1.40720534754\n",
      "(Iteration 21951 / 49000) loss: 1.44386254457\n",
      "(Epoch 224 / 500) Training Accuracy: 0.507816326531, Validation Accuracy: 0.46\n",
      "(Iteration 21961 / 49000) loss: 1.37881994315\n",
      "(Iteration 21971 / 49000) loss: 1.35048067092\n",
      "(Iteration 21981 / 49000) loss: 1.45734880431\n",
      "(Iteration 21991 / 49000) loss: 1.50627254536\n",
      "(Iteration 22001 / 49000) loss: 1.48558150722\n",
      "(Iteration 22011 / 49000) loss: 1.51653026135\n",
      "(Iteration 22021 / 49000) loss: 1.44260046605\n",
      "(Iteration 22031 / 49000) loss: 1.34527595032\n",
      "(Iteration 22041 / 49000) loss: 1.39200716798\n",
      "(Epoch 225 / 500) Training Accuracy: 0.510795918367, Validation Accuracy: 0.463\n",
      "(Iteration 22051 / 49000) loss: 1.40914195322\n",
      "(Iteration 22061 / 49000) loss: 1.44795950323\n",
      "(Iteration 22071 / 49000) loss: 1.33593554126\n",
      "(Iteration 22081 / 49000) loss: 1.46178466388\n",
      "(Iteration 22091 / 49000) loss: 1.39098499172\n",
      "(Iteration 22101 / 49000) loss: 1.43737280817\n",
      "(Iteration 22111 / 49000) loss: 1.44777050073\n",
      "(Iteration 22121 / 49000) loss: 1.49904797686\n",
      "(Iteration 22131 / 49000) loss: 1.444449023\n",
      "(Iteration 22141 / 49000) loss: 1.46362791094\n",
      "(Epoch 226 / 500) Training Accuracy: 0.511204081633, Validation Accuracy: 0.466\n",
      "(Iteration 22151 / 49000) loss: 1.41583173188\n",
      "(Iteration 22161 / 49000) loss: 1.46164545187\n",
      "(Iteration 22171 / 49000) loss: 1.3540630507\n",
      "(Iteration 22181 / 49000) loss: 1.3688148585\n",
      "(Iteration 22191 / 49000) loss: 1.38262404617\n",
      "(Iteration 22201 / 49000) loss: 1.38322367844\n",
      "(Iteration 22211 / 49000) loss: 1.37667651356\n",
      "(Iteration 22221 / 49000) loss: 1.3986078701\n",
      "(Iteration 22231 / 49000) loss: 1.32584046716\n",
      "(Iteration 22241 / 49000) loss: 1.41501334338\n",
      "(Epoch 227 / 500) Training Accuracy: 0.512285714286, Validation Accuracy: 0.449\n",
      "(Iteration 22251 / 49000) loss: 1.3855366562\n",
      "(Iteration 22261 / 49000) loss: 1.54426368892\n",
      "(Iteration 22271 / 49000) loss: 1.46701404052\n",
      "(Iteration 22281 / 49000) loss: 1.38308293247\n",
      "(Iteration 22291 / 49000) loss: 1.3715319807\n",
      "(Iteration 22301 / 49000) loss: 1.32947082988\n",
      "(Iteration 22311 / 49000) loss: 1.43220286757\n",
      "(Iteration 22321 / 49000) loss: 1.50598376868\n",
      "(Iteration 22331 / 49000) loss: 1.40654231378\n",
      "(Iteration 22341 / 49000) loss: 1.44619621594\n",
      "(Epoch 228 / 500) Training Accuracy: 0.506612244898, Validation Accuracy: 0.466\n",
      "(Iteration 22351 / 49000) loss: 1.37002878997\n",
      "(Iteration 22361 / 49000) loss: 1.4312325902\n",
      "(Iteration 22371 / 49000) loss: 1.41067858975\n",
      "(Iteration 22381 / 49000) loss: 1.34031255807\n",
      "(Iteration 22391 / 49000) loss: 1.50287024251\n",
      "(Iteration 22401 / 49000) loss: 1.45916253588\n",
      "(Iteration 22411 / 49000) loss: 1.38704799809\n",
      "(Iteration 22421 / 49000) loss: 1.42028404493\n",
      "(Iteration 22431 / 49000) loss: 1.36157329972\n",
      "(Iteration 22441 / 49000) loss: 1.30041290783\n",
      "(Epoch 229 / 500) Training Accuracy: 0.513795918367, Validation Accuracy: 0.465\n",
      "(Iteration 22451 / 49000) loss: 1.36149269668\n",
      "(Iteration 22461 / 49000) loss: 1.39108571998\n",
      "(Iteration 22471 / 49000) loss: 1.43397294924\n",
      "(Iteration 22481 / 49000) loss: 1.45955929055\n",
      "(Iteration 22491 / 49000) loss: 1.38040307174\n",
      "(Iteration 22501 / 49000) loss: 1.31904537568\n",
      "(Iteration 22511 / 49000) loss: 1.48211657411\n",
      "(Iteration 22521 / 49000) loss: 1.39036852568\n",
      "(Iteration 22531 / 49000) loss: 1.29528123355\n",
      "(Epoch 230 / 500) Training Accuracy: 0.51012244898, Validation Accuracy: 0.446\n",
      "(Iteration 22541 / 49000) loss: 1.32754226501\n",
      "(Iteration 22551 / 49000) loss: 1.47463650502\n",
      "(Iteration 22561 / 49000) loss: 1.3795044677\n",
      "(Iteration 22571 / 49000) loss: 1.40517351916\n",
      "(Iteration 22581 / 49000) loss: 1.43721549718\n",
      "(Iteration 22591 / 49000) loss: 1.41173462981\n",
      "(Iteration 22601 / 49000) loss: 1.32735111871\n",
      "(Iteration 22611 / 49000) loss: 1.43496422104\n",
      "(Iteration 22621 / 49000) loss: 1.42002688892\n",
      "(Iteration 22631 / 49000) loss: 1.41975574081\n",
      "(Epoch 231 / 500) Training Accuracy: 0.515530612245, Validation Accuracy: 0.453\n",
      "(Iteration 22641 / 49000) loss: 1.28826376639\n",
      "(Iteration 22651 / 49000) loss: 1.36745012166\n",
      "(Iteration 22661 / 49000) loss: 1.46042879329\n",
      "(Iteration 22671 / 49000) loss: 1.47780807027\n",
      "(Iteration 22681 / 49000) loss: 1.31271451489\n",
      "(Iteration 22691 / 49000) loss: 1.50234953392\n",
      "(Iteration 22701 / 49000) loss: 1.3426951027\n",
      "(Iteration 22711 / 49000) loss: 1.40827802479\n",
      "(Iteration 22721 / 49000) loss: 1.33006460118\n",
      "(Iteration 22731 / 49000) loss: 1.35009549688\n",
      "(Epoch 232 / 500) Training Accuracy: 0.515612244898, Validation Accuracy: 0.467\n",
      "(Iteration 22741 / 49000) loss: 1.37556436114\n",
      "(Iteration 22751 / 49000) loss: 1.45325905354\n",
      "(Iteration 22761 / 49000) loss: 1.45295165297\n",
      "(Iteration 22771 / 49000) loss: 1.38096044548\n",
      "(Iteration 22781 / 49000) loss: 1.28328252359\n",
      "(Iteration 22791 / 49000) loss: 1.43025226881\n",
      "(Iteration 22801 / 49000) loss: 1.44586963973\n",
      "(Iteration 22811 / 49000) loss: 1.43006997597\n",
      "(Iteration 22821 / 49000) loss: 1.3856286914\n",
      "(Iteration 22831 / 49000) loss: 1.39396117435\n",
      "(Epoch 233 / 500) Training Accuracy: 0.510510204082, Validation Accuracy: 0.461\n",
      "(Iteration 22841 / 49000) loss: 1.39484609108\n",
      "(Iteration 22851 / 49000) loss: 1.43301064226\n",
      "(Iteration 22861 / 49000) loss: 1.44166940796\n",
      "(Iteration 22871 / 49000) loss: 1.42544582968\n",
      "(Iteration 22881 / 49000) loss: 1.38363595656\n",
      "(Iteration 22891 / 49000) loss: 1.40849673517\n",
      "(Iteration 22901 / 49000) loss: 1.40724987227\n",
      "(Iteration 22911 / 49000) loss: 1.32041493994\n",
      "(Iteration 22921 / 49000) loss: 1.3377395347\n",
      "(Iteration 22931 / 49000) loss: 1.44670457447\n",
      "(Epoch 234 / 500) Training Accuracy: 0.512959183673, Validation Accuracy: 0.458\n",
      "(Iteration 22941 / 49000) loss: 1.36759257383\n",
      "(Iteration 22951 / 49000) loss: 1.41841433068\n",
      "(Iteration 22961 / 49000) loss: 1.44968154776\n",
      "(Iteration 22971 / 49000) loss: 1.45755715938\n",
      "(Iteration 22981 / 49000) loss: 1.46491601126\n",
      "(Iteration 22991 / 49000) loss: 1.43622532555\n",
      "(Iteration 23001 / 49000) loss: 1.43928802111\n",
      "(Iteration 23011 / 49000) loss: 1.49242160495\n",
      "(Iteration 23021 / 49000) loss: 1.42618707864\n",
      "(Epoch 235 / 500) Training Accuracy: 0.515142857143, Validation Accuracy: 0.464\n",
      "(Iteration 23031 / 49000) loss: 1.4184661592\n",
      "(Iteration 23041 / 49000) loss: 1.41753673968\n",
      "(Iteration 23051 / 49000) loss: 1.41082614305\n",
      "(Iteration 23061 / 49000) loss: 1.47120598154\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 23071 / 49000) loss: 1.41425280408\n",
      "(Iteration 23081 / 49000) loss: 1.42037502253\n",
      "(Iteration 23091 / 49000) loss: 1.47737463067\n",
      "(Iteration 23101 / 49000) loss: 1.45527719163\n",
      "(Iteration 23111 / 49000) loss: 1.4059703315\n",
      "(Iteration 23121 / 49000) loss: 1.39802905996\n",
      "(Epoch 236 / 500) Training Accuracy: 0.512897959184, Validation Accuracy: 0.465\n",
      "(Iteration 23131 / 49000) loss: 1.38492823018\n",
      "(Iteration 23141 / 49000) loss: 1.37910161738\n",
      "(Iteration 23151 / 49000) loss: 1.38393272186\n",
      "(Iteration 23161 / 49000) loss: 1.39484262981\n",
      "(Iteration 23171 / 49000) loss: 1.33708012801\n",
      "(Iteration 23181 / 49000) loss: 1.35998642182\n",
      "(Iteration 23191 / 49000) loss: 1.43718164592\n",
      "(Iteration 23201 / 49000) loss: 1.43812207837\n",
      "(Iteration 23211 / 49000) loss: 1.37232367985\n",
      "(Iteration 23221 / 49000) loss: 1.52746130965\n",
      "(Epoch 237 / 500) Training Accuracy: 0.516285714286, Validation Accuracy: 0.461\n",
      "(Iteration 23231 / 49000) loss: 1.33501056256\n",
      "(Iteration 23241 / 49000) loss: 1.43573738031\n",
      "(Iteration 23251 / 49000) loss: 1.27471525603\n",
      "(Iteration 23261 / 49000) loss: 1.40350424291\n",
      "(Iteration 23271 / 49000) loss: 1.41108133833\n",
      "(Iteration 23281 / 49000) loss: 1.38454104971\n",
      "(Iteration 23291 / 49000) loss: 1.3891271403\n",
      "(Iteration 23301 / 49000) loss: 1.37131769464\n",
      "(Iteration 23311 / 49000) loss: 1.42375691049\n",
      "(Iteration 23321 / 49000) loss: 1.38909648086\n",
      "(Epoch 238 / 500) Training Accuracy: 0.514836734694, Validation Accuracy: 0.455\n",
      "(Iteration 23331 / 49000) loss: 1.3769771104\n",
      "(Iteration 23341 / 49000) loss: 1.37193436865\n",
      "(Iteration 23351 / 49000) loss: 1.37759301781\n",
      "(Iteration 23361 / 49000) loss: 1.39663639918\n",
      "(Iteration 23371 / 49000) loss: 1.57040298843\n",
      "(Iteration 23381 / 49000) loss: 1.41466125943\n",
      "(Iteration 23391 / 49000) loss: 1.48270267422\n",
      "(Iteration 23401 / 49000) loss: 1.40007117454\n",
      "(Iteration 23411 / 49000) loss: 1.34674061788\n",
      "(Iteration 23421 / 49000) loss: 1.41742290447\n",
      "(Epoch 239 / 500) Training Accuracy: 0.514204081633, Validation Accuracy: 0.455\n",
      "(Iteration 23431 / 49000) loss: 1.39376039151\n",
      "(Iteration 23441 / 49000) loss: 1.39601557166\n",
      "(Iteration 23451 / 49000) loss: 1.39273079439\n",
      "(Iteration 23461 / 49000) loss: 1.50540497347\n",
      "(Iteration 23471 / 49000) loss: 1.38096594443\n",
      "(Iteration 23481 / 49000) loss: 1.36906690086\n",
      "(Iteration 23491 / 49000) loss: 1.36486406819\n",
      "(Iteration 23501 / 49000) loss: 1.48663581497\n",
      "(Iteration 23511 / 49000) loss: 1.39556630367\n",
      "(Epoch 240 / 500) Training Accuracy: 0.517693877551, Validation Accuracy: 0.458\n",
      "(Iteration 23521 / 49000) loss: 1.35390276076\n",
      "(Iteration 23531 / 49000) loss: 1.35391043572\n",
      "(Iteration 23541 / 49000) loss: 1.33173081729\n",
      "(Iteration 23551 / 49000) loss: 1.43912474427\n",
      "(Iteration 23561 / 49000) loss: 1.44780773024\n",
      "(Iteration 23571 / 49000) loss: 1.36015198294\n",
      "(Iteration 23581 / 49000) loss: 1.35075568389\n",
      "(Iteration 23591 / 49000) loss: 1.46578085741\n",
      "(Iteration 23601 / 49000) loss: 1.40032774574\n",
      "(Iteration 23611 / 49000) loss: 1.41846661949\n",
      "(Epoch 241 / 500) Training Accuracy: 0.514612244898, Validation Accuracy: 0.464\n",
      "(Iteration 23621 / 49000) loss: 1.31719795078\n",
      "(Iteration 23631 / 49000) loss: 1.39578680507\n",
      "(Iteration 23641 / 49000) loss: 1.3924133501\n",
      "(Iteration 23651 / 49000) loss: 1.32397164126\n",
      "(Iteration 23661 / 49000) loss: 1.43673512846\n",
      "(Iteration 23671 / 49000) loss: 1.39235583806\n",
      "(Iteration 23681 / 49000) loss: 1.30882679204\n",
      "(Iteration 23691 / 49000) loss: 1.41867768477\n",
      "(Iteration 23701 / 49000) loss: 1.40516822971\n",
      "(Iteration 23711 / 49000) loss: 1.39864461761\n",
      "(Epoch 242 / 500) Training Accuracy: 0.512489795918, Validation Accuracy: 0.451\n",
      "(Iteration 23721 / 49000) loss: 1.32437441458\n",
      "(Iteration 23731 / 49000) loss: 1.43585161228\n",
      "(Iteration 23741 / 49000) loss: 1.39419889545\n",
      "(Iteration 23751 / 49000) loss: 1.46251445594\n",
      "(Iteration 23761 / 49000) loss: 1.43148017733\n",
      "(Iteration 23771 / 49000) loss: 1.3867750589\n",
      "(Iteration 23781 / 49000) loss: 1.38190431477\n",
      "(Iteration 23791 / 49000) loss: 1.36434109019\n",
      "(Iteration 23801 / 49000) loss: 1.4518551243\n",
      "(Iteration 23811 / 49000) loss: 1.35293927262\n",
      "(Epoch 243 / 500) Training Accuracy: 0.517040816327, Validation Accuracy: 0.465\n",
      "(Iteration 23821 / 49000) loss: 1.35954867207\n",
      "(Iteration 23831 / 49000) loss: 1.42381747199\n",
      "(Iteration 23841 / 49000) loss: 1.4341522182\n",
      "(Iteration 23851 / 49000) loss: 1.46629030858\n",
      "(Iteration 23861 / 49000) loss: 1.36332043583\n",
      "(Iteration 23871 / 49000) loss: 1.39211712264\n",
      "(Iteration 23881 / 49000) loss: 1.50964528578\n",
      "(Iteration 23891 / 49000) loss: 1.42582882976\n",
      "(Iteration 23901 / 49000) loss: 1.38030723965\n",
      "(Iteration 23911 / 49000) loss: 1.46099821831\n",
      "(Epoch 244 / 500) Training Accuracy: 0.511040816327, Validation Accuracy: 0.457\n",
      "(Iteration 23921 / 49000) loss: 1.39676018302\n",
      "(Iteration 23931 / 49000) loss: 1.39666589848\n",
      "(Iteration 23941 / 49000) loss: 1.36990079493\n",
      "(Iteration 23951 / 49000) loss: 1.43638025245\n",
      "(Iteration 23961 / 49000) loss: 1.3642654197\n",
      "(Iteration 23971 / 49000) loss: 1.39702949108\n",
      "(Iteration 23981 / 49000) loss: 1.44590207249\n",
      "(Iteration 23991 / 49000) loss: 1.38369494584\n",
      "(Iteration 24001 / 49000) loss: 1.41808176354\n",
      "(Epoch 245 / 500) Training Accuracy: 0.515040816327, Validation Accuracy: 0.459\n",
      "(Iteration 24011 / 49000) loss: 1.25434893979\n",
      "(Iteration 24021 / 49000) loss: 1.37764975345\n",
      "(Iteration 24031 / 49000) loss: 1.37208468742\n",
      "(Iteration 24041 / 49000) loss: 1.44517018824\n",
      "(Iteration 24051 / 49000) loss: 1.39160678969\n",
      "(Iteration 24061 / 49000) loss: 1.44301330185\n",
      "(Iteration 24071 / 49000) loss: 1.33615229144\n",
      "(Iteration 24081 / 49000) loss: 1.51934366439\n",
      "(Iteration 24091 / 49000) loss: 1.37336249209\n",
      "(Iteration 24101 / 49000) loss: 1.37378007518\n",
      "(Epoch 246 / 500) Training Accuracy: 0.517714285714, Validation Accuracy: 0.46\n",
      "(Iteration 24111 / 49000) loss: 1.43262339131\n",
      "(Iteration 24121 / 49000) loss: 1.27613347623\n",
      "(Iteration 24131 / 49000) loss: 1.34169304848\n",
      "(Iteration 24141 / 49000) loss: 1.36927326383\n",
      "(Iteration 24151 / 49000) loss: 1.36579554699\n",
      "(Iteration 24161 / 49000) loss: 1.35892180912\n",
      "(Iteration 24171 / 49000) loss: 1.3692189031\n",
      "(Iteration 24181 / 49000) loss: 1.32396789547\n",
      "(Iteration 24191 / 49000) loss: 1.52596492782\n",
      "(Iteration 24201 / 49000) loss: 1.34133410053\n",
      "(Epoch 247 / 500) Training Accuracy: 0.518183673469, Validation Accuracy: 0.46\n",
      "(Iteration 24211 / 49000) loss: 1.41042011557\n",
      "(Iteration 24221 / 49000) loss: 1.40522622014\n",
      "(Iteration 24231 / 49000) loss: 1.42663468826\n",
      "(Iteration 24241 / 49000) loss: 1.38477434987\n",
      "(Iteration 24251 / 49000) loss: 1.471044206\n",
      "(Iteration 24261 / 49000) loss: 1.37645385411\n",
      "(Iteration 24271 / 49000) loss: 1.43228335118\n",
      "(Iteration 24281 / 49000) loss: 1.32797192281\n",
      "(Iteration 24291 / 49000) loss: 1.41650843072\n",
      "(Iteration 24301 / 49000) loss: 1.39327675688\n",
      "(Epoch 248 / 500) Training Accuracy: 0.51712244898, Validation Accuracy: 0.461\n",
      "(Iteration 24311 / 49000) loss: 1.37735550956\n",
      "(Iteration 24321 / 49000) loss: 1.39817803486\n",
      "(Iteration 24331 / 49000) loss: 1.45077328791\n",
      "(Iteration 24341 / 49000) loss: 1.38046628971\n",
      "(Iteration 24351 / 49000) loss: 1.28077129591\n",
      "(Iteration 24361 / 49000) loss: 1.45300177655\n",
      "(Iteration 24371 / 49000) loss: 1.35188414125\n",
      "(Iteration 24381 / 49000) loss: 1.38410395648\n",
      "(Iteration 24391 / 49000) loss: 1.4007888329\n",
      "(Iteration 24401 / 49000) loss: 1.36521556982\n",
      "(Epoch 249 / 500) Training Accuracy: 0.513408163265, Validation Accuracy: 0.462\n",
      "(Iteration 24411 / 49000) loss: 1.37268848758\n",
      "(Iteration 24421 / 49000) loss: 1.32412166161\n",
      "(Iteration 24431 / 49000) loss: 1.39668494154\n",
      "(Iteration 24441 / 49000) loss: 1.47529818789\n",
      "(Iteration 24451 / 49000) loss: 1.32006714946\n",
      "(Iteration 24461 / 49000) loss: 1.42184247158\n",
      "(Iteration 24471 / 49000) loss: 1.39931049594\n",
      "(Iteration 24481 / 49000) loss: 1.34674915752\n",
      "(Iteration 24491 / 49000) loss: 1.44489191694\n",
      "(Epoch 250 / 500) Training Accuracy: 0.51206122449, Validation Accuracy: 0.46\n",
      "(Iteration 24501 / 49000) loss: 1.29224513107\n",
      "(Iteration 24511 / 49000) loss: 1.50239931573\n",
      "(Iteration 24521 / 49000) loss: 1.41822182982\n",
      "(Iteration 24531 / 49000) loss: 1.41337348857\n",
      "(Iteration 24541 / 49000) loss: 1.45086955624\n",
      "(Iteration 24551 / 49000) loss: 1.37930004049\n",
      "(Iteration 24561 / 49000) loss: 1.34272210216\n",
      "(Iteration 24571 / 49000) loss: 1.35626911318\n",
      "(Iteration 24581 / 49000) loss: 1.36937034966\n",
      "(Iteration 24591 / 49000) loss: 1.42782389982\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Epoch 251 / 500) Training Accuracy: 0.518979591837, Validation Accuracy: 0.457\n",
      "(Iteration 24601 / 49000) loss: 1.40440291525\n",
      "(Iteration 24611 / 49000) loss: 1.34064922005\n",
      "(Iteration 24621 / 49000) loss: 1.41131813944\n",
      "(Iteration 24631 / 49000) loss: 1.38146656677\n",
      "(Iteration 24641 / 49000) loss: 1.39125220624\n",
      "(Iteration 24651 / 49000) loss: 1.45062107244\n",
      "(Iteration 24661 / 49000) loss: 1.28909641694\n",
      "(Iteration 24671 / 49000) loss: 1.34660099716\n",
      "(Iteration 24681 / 49000) loss: 1.25045870973\n",
      "(Iteration 24691 / 49000) loss: 1.41946603914\n",
      "(Epoch 252 / 500) Training Accuracy: 0.516244897959, Validation Accuracy: 0.458\n",
      "(Iteration 24701 / 49000) loss: 1.31420992034\n",
      "(Iteration 24711 / 49000) loss: 1.37439511867\n",
      "(Iteration 24721 / 49000) loss: 1.33412662734\n",
      "(Iteration 24731 / 49000) loss: 1.34445304672\n",
      "(Iteration 24741 / 49000) loss: 1.4007896176\n",
      "(Iteration 24751 / 49000) loss: 1.40216059873\n",
      "(Iteration 24761 / 49000) loss: 1.34556511855\n",
      "(Iteration 24771 / 49000) loss: 1.34976804292\n",
      "(Iteration 24781 / 49000) loss: 1.44181188658\n",
      "(Iteration 24791 / 49000) loss: 1.38599029062\n",
      "(Epoch 253 / 500) Training Accuracy: 0.518163265306, Validation Accuracy: 0.459\n",
      "(Iteration 24801 / 49000) loss: 1.33134067256\n",
      "(Iteration 24811 / 49000) loss: 1.50809441392\n",
      "(Iteration 24821 / 49000) loss: 1.38685587761\n",
      "(Iteration 24831 / 49000) loss: 1.54136154463\n",
      "(Iteration 24841 / 49000) loss: 1.28836006838\n",
      "(Iteration 24851 / 49000) loss: 1.37191810464\n",
      "(Iteration 24861 / 49000) loss: 1.36295036134\n",
      "(Iteration 24871 / 49000) loss: 1.47466313678\n",
      "(Iteration 24881 / 49000) loss: 1.39938014194\n",
      "(Iteration 24891 / 49000) loss: 1.38779436555\n",
      "(Epoch 254 / 500) Training Accuracy: 0.517367346939, Validation Accuracy: 0.456\n",
      "(Iteration 24901 / 49000) loss: 1.36685690754\n",
      "(Iteration 24911 / 49000) loss: 1.42206706715\n",
      "(Iteration 24921 / 49000) loss: 1.28302721597\n",
      "(Iteration 24931 / 49000) loss: 1.43483886822\n",
      "(Iteration 24941 / 49000) loss: 1.41160470672\n",
      "(Iteration 24951 / 49000) loss: 1.33382015838\n",
      "(Iteration 24961 / 49000) loss: 1.44317002096\n",
      "(Iteration 24971 / 49000) loss: 1.43068520865\n",
      "(Iteration 24981 / 49000) loss: 1.31596768672\n",
      "(Epoch 255 / 500) Training Accuracy: 0.516714285714, Validation Accuracy: 0.459\n",
      "(Iteration 24991 / 49000) loss: 1.35504613763\n",
      "(Iteration 25001 / 49000) loss: 1.46858884854\n",
      "(Iteration 25011 / 49000) loss: 1.43496528814\n",
      "(Iteration 25021 / 49000) loss: 1.26920510891\n",
      "(Iteration 25031 / 49000) loss: 1.46395241871\n",
      "(Iteration 25041 / 49000) loss: 1.39227534635\n",
      "(Iteration 25051 / 49000) loss: 1.45122190943\n",
      "(Iteration 25061 / 49000) loss: 1.43986881236\n",
      "(Iteration 25071 / 49000) loss: 1.37739548747\n",
      "(Iteration 25081 / 49000) loss: 1.45027029546\n",
      "(Epoch 256 / 500) Training Accuracy: 0.518448979592, Validation Accuracy: 0.458\n",
      "(Iteration 25091 / 49000) loss: 1.35207260691\n",
      "(Iteration 25101 / 49000) loss: 1.35397579023\n",
      "(Iteration 25111 / 49000) loss: 1.33840543075\n",
      "(Iteration 25121 / 49000) loss: 1.45704345286\n",
      "(Iteration 25131 / 49000) loss: 1.44359929498\n",
      "(Iteration 25141 / 49000) loss: 1.33859689695\n",
      "(Iteration 25151 / 49000) loss: 1.35445600783\n",
      "(Iteration 25161 / 49000) loss: 1.36069387012\n",
      "(Iteration 25171 / 49000) loss: 1.37129444107\n",
      "(Iteration 25181 / 49000) loss: 1.38885732257\n",
      "(Epoch 257 / 500) Training Accuracy: 0.513265306122, Validation Accuracy: 0.466\n",
      "(Iteration 25191 / 49000) loss: 1.35782291719\n",
      "(Iteration 25201 / 49000) loss: 1.43010227149\n",
      "(Iteration 25211 / 49000) loss: 1.37965244558\n",
      "(Iteration 25221 / 49000) loss: 1.45412131722\n",
      "(Iteration 25231 / 49000) loss: 1.31476356077\n",
      "(Iteration 25241 / 49000) loss: 1.35013998566\n",
      "(Iteration 25251 / 49000) loss: 1.41023088609\n",
      "(Iteration 25261 / 49000) loss: 1.44083444204\n",
      "(Iteration 25271 / 49000) loss: 1.40854354375\n",
      "(Iteration 25281 / 49000) loss: 1.36261174144\n",
      "(Epoch 258 / 500) Training Accuracy: 0.518408163265, Validation Accuracy: 0.465\n",
      "(Iteration 25291 / 49000) loss: 1.3686909419\n",
      "(Iteration 25301 / 49000) loss: 1.350332414\n",
      "(Iteration 25311 / 49000) loss: 1.32016761243\n",
      "(Iteration 25321 / 49000) loss: 1.33998954418\n",
      "(Iteration 25331 / 49000) loss: 1.36470963375\n",
      "(Iteration 25341 / 49000) loss: 1.39308000249\n",
      "(Iteration 25351 / 49000) loss: 1.27779121729\n",
      "(Iteration 25361 / 49000) loss: 1.39132141323\n",
      "(Iteration 25371 / 49000) loss: 1.37721885252\n",
      "(Iteration 25381 / 49000) loss: 1.3967774615\n",
      "(Epoch 259 / 500) Training Accuracy: 0.519183673469, Validation Accuracy: 0.464\n",
      "(Iteration 25391 / 49000) loss: 1.40972840114\n",
      "(Iteration 25401 / 49000) loss: 1.41358429501\n",
      "(Iteration 25411 / 49000) loss: 1.32704073814\n",
      "(Iteration 25421 / 49000) loss: 1.34865646849\n",
      "(Iteration 25431 / 49000) loss: 1.41925182215\n",
      "(Iteration 25441 / 49000) loss: 1.2897678877\n",
      "(Iteration 25451 / 49000) loss: 1.46241917125\n",
      "(Iteration 25461 / 49000) loss: 1.41741423384\n",
      "(Iteration 25471 / 49000) loss: 1.3469844584\n",
      "(Epoch 260 / 500) Training Accuracy: 0.520020408163, Validation Accuracy: 0.477\n",
      "(Iteration 25481 / 49000) loss: 1.34761051426\n",
      "(Iteration 25491 / 49000) loss: 1.35144615021\n",
      "(Iteration 25501 / 49000) loss: 1.36003762651\n",
      "(Iteration 25511 / 49000) loss: 1.33017883464\n",
      "(Iteration 25521 / 49000) loss: 1.39551089981\n",
      "(Iteration 25531 / 49000) loss: 1.40285904018\n",
      "(Iteration 25541 / 49000) loss: 1.35207783435\n",
      "(Iteration 25551 / 49000) loss: 1.33630989566\n",
      "(Iteration 25561 / 49000) loss: 1.39564452015\n",
      "(Iteration 25571 / 49000) loss: 1.43646655419\n",
      "(Epoch 261 / 500) Training Accuracy: 0.519918367347, Validation Accuracy: 0.453\n",
      "(Iteration 25581 / 49000) loss: 1.36967047081\n",
      "(Iteration 25591 / 49000) loss: 1.40870155855\n",
      "(Iteration 25601 / 49000) loss: 1.35688142134\n",
      "(Iteration 25611 / 49000) loss: 1.41852416076\n",
      "(Iteration 25621 / 49000) loss: 1.37327192222\n",
      "(Iteration 25631 / 49000) loss: 1.43867654413\n",
      "(Iteration 25641 / 49000) loss: 1.37992482604\n",
      "(Iteration 25651 / 49000) loss: 1.37960866048\n",
      "(Iteration 25661 / 49000) loss: 1.37395134215\n",
      "(Iteration 25671 / 49000) loss: 1.32647693999\n",
      "(Epoch 262 / 500) Training Accuracy: 0.519632653061, Validation Accuracy: 0.466\n",
      "(Iteration 25681 / 49000) loss: 1.38534758656\n",
      "(Iteration 25691 / 49000) loss: 1.33628401982\n",
      "(Iteration 25701 / 49000) loss: 1.34273620021\n",
      "(Iteration 25711 / 49000) loss: 1.36387491731\n",
      "(Iteration 25721 / 49000) loss: 1.45683382598\n",
      "(Iteration 25731 / 49000) loss: 1.37045294608\n",
      "(Iteration 25741 / 49000) loss: 1.36125491751\n",
      "(Iteration 25751 / 49000) loss: 1.38478610988\n",
      "(Iteration 25761 / 49000) loss: 1.44184102855\n",
      "(Iteration 25771 / 49000) loss: 1.36131562568\n",
      "(Epoch 263 / 500) Training Accuracy: 0.522591836735, Validation Accuracy: 0.458\n",
      "(Iteration 25781 / 49000) loss: 1.38712624674\n",
      "(Iteration 25791 / 49000) loss: 1.4622455394\n",
      "(Iteration 25801 / 49000) loss: 1.31943009774\n",
      "(Iteration 25811 / 49000) loss: 1.40658239407\n",
      "(Iteration 25821 / 49000) loss: 1.38096064034\n",
      "(Iteration 25831 / 49000) loss: 1.36758476693\n",
      "(Iteration 25841 / 49000) loss: 1.31343439904\n",
      "(Iteration 25851 / 49000) loss: 1.41956623033\n",
      "(Iteration 25861 / 49000) loss: 1.42957441428\n",
      "(Iteration 25871 / 49000) loss: 1.39916873513\n",
      "(Epoch 264 / 500) Training Accuracy: 0.509897959184, Validation Accuracy: 0.465\n",
      "(Iteration 25881 / 49000) loss: 1.34273766883\n",
      "(Iteration 25891 / 49000) loss: 1.33435057569\n",
      "(Iteration 25901 / 49000) loss: 1.49557204963\n",
      "(Iteration 25911 / 49000) loss: 1.33377485889\n",
      "(Iteration 25921 / 49000) loss: 1.35136775717\n",
      "(Iteration 25931 / 49000) loss: 1.41128231012\n",
      "(Iteration 25941 / 49000) loss: 1.35423144353\n",
      "(Iteration 25951 / 49000) loss: 1.38393376732\n",
      "(Iteration 25961 / 49000) loss: 1.39688851351\n",
      "(Epoch 265 / 500) Training Accuracy: 0.521285714286, Validation Accuracy: 0.461\n",
      "(Iteration 25971 / 49000) loss: 1.35850336557\n",
      "(Iteration 25981 / 49000) loss: 1.38273356916\n",
      "(Iteration 25991 / 49000) loss: 1.40441479108\n",
      "(Iteration 26001 / 49000) loss: 1.33588980552\n",
      "(Iteration 26011 / 49000) loss: 1.34327963223\n",
      "(Iteration 26021 / 49000) loss: 1.38445864371\n",
      "(Iteration 26031 / 49000) loss: 1.38271527563\n",
      "(Iteration 26041 / 49000) loss: 1.40759906128\n",
      "(Iteration 26051 / 49000) loss: 1.30882783143\n",
      "(Iteration 26061 / 49000) loss: 1.36895133934\n",
      "(Epoch 266 / 500) Training Accuracy: 0.522673469388, Validation Accuracy: 0.461\n",
      "(Iteration 26071 / 49000) loss: 1.47789007254\n",
      "(Iteration 26081 / 49000) loss: 1.29439130889\n",
      "(Iteration 26091 / 49000) loss: 1.33696507457\n",
      "(Iteration 26101 / 49000) loss: 1.37689023677\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 26111 / 49000) loss: 1.44428715899\n",
      "(Iteration 26121 / 49000) loss: 1.40261433629\n",
      "(Iteration 26131 / 49000) loss: 1.35990617858\n",
      "(Iteration 26141 / 49000) loss: 1.40329750199\n",
      "(Iteration 26151 / 49000) loss: 1.45634092287\n",
      "(Iteration 26161 / 49000) loss: 1.43549599602\n",
      "(Epoch 267 / 500) Training Accuracy: 0.522897959184, Validation Accuracy: 0.459\n",
      "(Iteration 26171 / 49000) loss: 1.34356250881\n",
      "(Iteration 26181 / 49000) loss: 1.30263838833\n",
      "(Iteration 26191 / 49000) loss: 1.31437046576\n",
      "(Iteration 26201 / 49000) loss: 1.42171154056\n",
      "(Iteration 26211 / 49000) loss: 1.38388680209\n",
      "(Iteration 26221 / 49000) loss: 1.39444289874\n",
      "(Iteration 26231 / 49000) loss: 1.38470864926\n",
      "(Iteration 26241 / 49000) loss: 1.47220297645\n",
      "(Iteration 26251 / 49000) loss: 1.40039534501\n",
      "(Iteration 26261 / 49000) loss: 1.37058752353\n",
      "(Epoch 268 / 500) Training Accuracy: 0.515326530612, Validation Accuracy: 0.452\n",
      "(Iteration 26271 / 49000) loss: 1.36400443555\n",
      "(Iteration 26281 / 49000) loss: 1.36134733577\n",
      "(Iteration 26291 / 49000) loss: 1.36163420145\n",
      "(Iteration 26301 / 49000) loss: 1.33669883203\n",
      "(Iteration 26311 / 49000) loss: 1.32823147123\n",
      "(Iteration 26321 / 49000) loss: 1.32273247964\n",
      "(Iteration 26331 / 49000) loss: 1.38584654157\n",
      "(Iteration 26341 / 49000) loss: 1.40821820555\n",
      "(Iteration 26351 / 49000) loss: 1.41480463317\n",
      "(Iteration 26361 / 49000) loss: 1.38220611774\n",
      "(Epoch 269 / 500) Training Accuracy: 0.521510204082, Validation Accuracy: 0.462\n",
      "(Iteration 26371 / 49000) loss: 1.32159316999\n",
      "(Iteration 26381 / 49000) loss: 1.39205150057\n",
      "(Iteration 26391 / 49000) loss: 1.32009951936\n",
      "(Iteration 26401 / 49000) loss: 1.33928647534\n",
      "(Iteration 26411 / 49000) loss: 1.40423247762\n",
      "(Iteration 26421 / 49000) loss: 1.40325061451\n",
      "(Iteration 26431 / 49000) loss: 1.33519761807\n",
      "(Iteration 26441 / 49000) loss: 1.37654524193\n",
      "(Iteration 26451 / 49000) loss: 1.35261471071\n",
      "(Epoch 270 / 500) Training Accuracy: 0.521734693878, Validation Accuracy: 0.448\n",
      "(Iteration 26461 / 49000) loss: 1.33525644121\n",
      "(Iteration 26471 / 49000) loss: 1.39150382976\n",
      "(Iteration 26481 / 49000) loss: 1.28476938378\n",
      "(Iteration 26491 / 49000) loss: 1.43257313791\n",
      "(Iteration 26501 / 49000) loss: 1.39857803325\n",
      "(Iteration 26511 / 49000) loss: 1.43281938351\n",
      "(Iteration 26521 / 49000) loss: 1.44841358271\n",
      "(Iteration 26531 / 49000) loss: 1.44391217549\n",
      "(Iteration 26541 / 49000) loss: 1.34296150808\n",
      "(Iteration 26551 / 49000) loss: 1.35333555702\n",
      "(Epoch 271 / 500) Training Accuracy: 0.521693877551, Validation Accuracy: 0.457\n",
      "(Iteration 26561 / 49000) loss: 1.46547902616\n",
      "(Iteration 26571 / 49000) loss: 1.37104761929\n",
      "(Iteration 26581 / 49000) loss: 1.38040851054\n",
      "(Iteration 26591 / 49000) loss: 1.33415121248\n",
      "(Iteration 26601 / 49000) loss: 1.4458506152\n",
      "(Iteration 26611 / 49000) loss: 1.36568925468\n",
      "(Iteration 26621 / 49000) loss: 1.37125508437\n",
      "(Iteration 26631 / 49000) loss: 1.33579345605\n",
      "(Iteration 26641 / 49000) loss: 1.42969905954\n",
      "(Iteration 26651 / 49000) loss: 1.33717621089\n",
      "(Epoch 272 / 500) Training Accuracy: 0.523448979592, Validation Accuracy: 0.462\n",
      "(Iteration 26661 / 49000) loss: 1.38437629429\n",
      "(Iteration 26671 / 49000) loss: 1.33048558024\n",
      "(Iteration 26681 / 49000) loss: 1.39307015931\n",
      "(Iteration 26691 / 49000) loss: 1.34146441158\n",
      "(Iteration 26701 / 49000) loss: 1.36350671781\n",
      "(Iteration 26711 / 49000) loss: 1.36021978862\n",
      "(Iteration 26721 / 49000) loss: 1.44864961927\n",
      "(Iteration 26731 / 49000) loss: 1.38140725359\n",
      "(Iteration 26741 / 49000) loss: 1.44491018518\n",
      "(Iteration 26751 / 49000) loss: 1.29387268715\n",
      "(Epoch 273 / 500) Training Accuracy: 0.522224489796, Validation Accuracy: 0.45\n",
      "(Iteration 26761 / 49000) loss: 1.41010658745\n",
      "(Iteration 26771 / 49000) loss: 1.38743553259\n",
      "(Iteration 26781 / 49000) loss: 1.42934743178\n",
      "(Iteration 26791 / 49000) loss: 1.37881500069\n",
      "(Iteration 26801 / 49000) loss: 1.38151291874\n",
      "(Iteration 26811 / 49000) loss: 1.35185777709\n",
      "(Iteration 26821 / 49000) loss: 1.29666658006\n",
      "(Iteration 26831 / 49000) loss: 1.41267747958\n",
      "(Iteration 26841 / 49000) loss: 1.37409823778\n",
      "(Iteration 26851 / 49000) loss: 1.41066558627\n",
      "(Epoch 274 / 500) Training Accuracy: 0.52187755102, Validation Accuracy: 0.46\n",
      "(Iteration 26861 / 49000) loss: 1.37330912341\n",
      "(Iteration 26871 / 49000) loss: 1.34962651673\n",
      "(Iteration 26881 / 49000) loss: 1.43298765544\n",
      "(Iteration 26891 / 49000) loss: 1.44144854354\n",
      "(Iteration 26901 / 49000) loss: 1.39358582655\n",
      "(Iteration 26911 / 49000) loss: 1.33880629177\n",
      "(Iteration 26921 / 49000) loss: 1.3708285136\n",
      "(Iteration 26931 / 49000) loss: 1.43274420408\n",
      "(Iteration 26941 / 49000) loss: 1.29992659537\n",
      "(Epoch 275 / 500) Training Accuracy: 0.518163265306, Validation Accuracy: 0.47\n",
      "(Iteration 26951 / 49000) loss: 1.41918913675\n",
      "(Iteration 26961 / 49000) loss: 1.31667231287\n",
      "(Iteration 26971 / 49000) loss: 1.31849981609\n",
      "(Iteration 26981 / 49000) loss: 1.38336182106\n",
      "(Iteration 26991 / 49000) loss: 1.35260328096\n",
      "(Iteration 27001 / 49000) loss: 1.42005404762\n",
      "(Iteration 27011 / 49000) loss: 1.37059309663\n",
      "(Iteration 27021 / 49000) loss: 1.38456519486\n",
      "(Iteration 27031 / 49000) loss: 1.40152513461\n",
      "(Iteration 27041 / 49000) loss: 1.45062026991\n",
      "(Epoch 276 / 500) Training Accuracy: 0.521428571429, Validation Accuracy: 0.46\n",
      "(Iteration 27051 / 49000) loss: 1.40138369055\n",
      "(Iteration 27061 / 49000) loss: 1.40833093852\n",
      "(Iteration 27071 / 49000) loss: 1.40706740145\n",
      "(Iteration 27081 / 49000) loss: 1.40609145911\n",
      "(Iteration 27091 / 49000) loss: 1.30787476306\n",
      "(Iteration 27101 / 49000) loss: 1.39692238907\n",
      "(Iteration 27111 / 49000) loss: 1.43693363064\n",
      "(Iteration 27121 / 49000) loss: 1.44888981704\n",
      "(Iteration 27131 / 49000) loss: 1.29795371407\n",
      "(Iteration 27141 / 49000) loss: 1.36043006996\n",
      "(Epoch 277 / 500) Training Accuracy: 0.525428571429, Validation Accuracy: 0.46\n",
      "(Iteration 27151 / 49000) loss: 1.37467148062\n",
      "(Iteration 27161 / 49000) loss: 1.32739182901\n",
      "(Iteration 27171 / 49000) loss: 1.41596891764\n",
      "(Iteration 27181 / 49000) loss: 1.31353309573\n",
      "(Iteration 27191 / 49000) loss: 1.28190031093\n",
      "(Iteration 27201 / 49000) loss: 1.41584099489\n",
      "(Iteration 27211 / 49000) loss: 1.30279853954\n",
      "(Iteration 27221 / 49000) loss: 1.39052535406\n",
      "(Iteration 27231 / 49000) loss: 1.31969603624\n",
      "(Iteration 27241 / 49000) loss: 1.38728592513\n",
      "(Epoch 278 / 500) Training Accuracy: 0.523755102041, Validation Accuracy: 0.46\n",
      "(Iteration 27251 / 49000) loss: 1.32477451383\n",
      "(Iteration 27261 / 49000) loss: 1.3383787003\n",
      "(Iteration 27271 / 49000) loss: 1.44416884986\n",
      "(Iteration 27281 / 49000) loss: 1.36266462338\n",
      "(Iteration 27291 / 49000) loss: 1.42618550666\n",
      "(Iteration 27301 / 49000) loss: 1.48015167446\n",
      "(Iteration 27311 / 49000) loss: 1.40207926664\n",
      "(Iteration 27321 / 49000) loss: 1.43853287719\n",
      "(Iteration 27331 / 49000) loss: 1.35937695115\n",
      "(Iteration 27341 / 49000) loss: 1.3332137653\n",
      "(Epoch 279 / 500) Training Accuracy: 0.520326530612, Validation Accuracy: 0.46\n",
      "(Iteration 27351 / 49000) loss: 1.35728390078\n",
      "(Iteration 27361 / 49000) loss: 1.39231653144\n",
      "(Iteration 27371 / 49000) loss: 1.29990965167\n",
      "(Iteration 27381 / 49000) loss: 1.34238567883\n",
      "(Iteration 27391 / 49000) loss: 1.30884845206\n",
      "(Iteration 27401 / 49000) loss: 1.3334332712\n",
      "(Iteration 27411 / 49000) loss: 1.29957348445\n",
      "(Iteration 27421 / 49000) loss: 1.40553949608\n",
      "(Iteration 27431 / 49000) loss: 1.50667135633\n",
      "(Epoch 280 / 500) Training Accuracy: 0.525795918367, Validation Accuracy: 0.466\n",
      "(Iteration 27441 / 49000) loss: 1.35963180962\n",
      "(Iteration 27451 / 49000) loss: 1.35057235082\n",
      "(Iteration 27461 / 49000) loss: 1.37600071307\n",
      "(Iteration 27471 / 49000) loss: 1.35249471981\n",
      "(Iteration 27481 / 49000) loss: 1.40842025905\n",
      "(Iteration 27491 / 49000) loss: 1.35028492525\n",
      "(Iteration 27501 / 49000) loss: 1.4124316323\n",
      "(Iteration 27511 / 49000) loss: 1.49182350274\n",
      "(Iteration 27521 / 49000) loss: 1.33358535311\n",
      "(Iteration 27531 / 49000) loss: 1.39848154042\n",
      "(Epoch 281 / 500) Training Accuracy: 0.52493877551, Validation Accuracy: 0.46\n",
      "(Iteration 27541 / 49000) loss: 1.40158453038\n",
      "(Iteration 27551 / 49000) loss: 1.29617203699\n",
      "(Iteration 27561 / 49000) loss: 1.36040950394\n",
      "(Iteration 27571 / 49000) loss: 1.41424630104\n",
      "(Iteration 27581 / 49000) loss: 1.35379618792\n",
      "(Iteration 27591 / 49000) loss: 1.38293095933\n",
      "(Iteration 27601 / 49000) loss: 1.37492436306\n",
      "(Iteration 27611 / 49000) loss: 1.38812878728\n",
      "(Iteration 27621 / 49000) loss: 1.37679171606\n",
      "(Iteration 27631 / 49000) loss: 1.33858538947\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Epoch 282 / 500) Training Accuracy: 0.521591836735, Validation Accuracy: 0.459\n",
      "(Iteration 27641 / 49000) loss: 1.42784570298\n",
      "(Iteration 27651 / 49000) loss: 1.31671969431\n",
      "(Iteration 27661 / 49000) loss: 1.39747712907\n",
      "(Iteration 27671 / 49000) loss: 1.40725510947\n",
      "(Iteration 27681 / 49000) loss: 1.4068982952\n",
      "(Iteration 27691 / 49000) loss: 1.36860566751\n",
      "(Iteration 27701 / 49000) loss: 1.34665827592\n",
      "(Iteration 27711 / 49000) loss: 1.36023671841\n",
      "(Iteration 27721 / 49000) loss: 1.32303220164\n",
      "(Iteration 27731 / 49000) loss: 1.3236358872\n",
      "(Epoch 283 / 500) Training Accuracy: 0.525387755102, Validation Accuracy: 0.464\n",
      "(Iteration 27741 / 49000) loss: 1.36534732857\n",
      "(Iteration 27751 / 49000) loss: 1.31506605299\n",
      "(Iteration 27761 / 49000) loss: 1.34589250357\n",
      "(Iteration 27771 / 49000) loss: 1.40152884389\n",
      "(Iteration 27781 / 49000) loss: 1.3152914485\n",
      "(Iteration 27791 / 49000) loss: 1.39606016489\n",
      "(Iteration 27801 / 49000) loss: 1.31017458979\n",
      "(Iteration 27811 / 49000) loss: 1.41136890863\n",
      "(Iteration 27821 / 49000) loss: 1.44613185393\n",
      "(Iteration 27831 / 49000) loss: 1.33447117092\n",
      "(Epoch 284 / 500) Training Accuracy: 0.523367346939, Validation Accuracy: 0.465\n",
      "(Iteration 27841 / 49000) loss: 1.37735287294\n",
      "(Iteration 27851 / 49000) loss: 1.33877159121\n",
      "(Iteration 27861 / 49000) loss: 1.34141710052\n",
      "(Iteration 27871 / 49000) loss: 1.37415720897\n",
      "(Iteration 27881 / 49000) loss: 1.32429869549\n",
      "(Iteration 27891 / 49000) loss: 1.27729900737\n",
      "(Iteration 27901 / 49000) loss: 1.37751923363\n",
      "(Iteration 27911 / 49000) loss: 1.37714494818\n",
      "(Iteration 27921 / 49000) loss: 1.2580497778\n",
      "(Epoch 285 / 500) Training Accuracy: 0.525755102041, Validation Accuracy: 0.466\n",
      "(Iteration 27931 / 49000) loss: 1.39444662962\n",
      "(Iteration 27941 / 49000) loss: 1.35589804313\n",
      "(Iteration 27951 / 49000) loss: 1.35987134282\n",
      "(Iteration 27961 / 49000) loss: 1.27713721869\n",
      "(Iteration 27971 / 49000) loss: 1.40250314836\n",
      "(Iteration 27981 / 49000) loss: 1.40072468985\n",
      "(Iteration 27991 / 49000) loss: 1.38876740721\n",
      "(Iteration 28001 / 49000) loss: 1.35964526804\n",
      "(Iteration 28011 / 49000) loss: 1.34495460696\n",
      "(Iteration 28021 / 49000) loss: 1.33686710626\n",
      "(Epoch 286 / 500) Training Accuracy: 0.525326530612, Validation Accuracy: 0.47\n",
      "(Iteration 28031 / 49000) loss: 1.38868483072\n",
      "(Iteration 28041 / 49000) loss: 1.36330726499\n",
      "(Iteration 28051 / 49000) loss: 1.32171864306\n",
      "(Iteration 28061 / 49000) loss: 1.38339191963\n",
      "(Iteration 28071 / 49000) loss: 1.36766766215\n",
      "(Iteration 28081 / 49000) loss: 1.44050051432\n",
      "(Iteration 28091 / 49000) loss: 1.35387546665\n",
      "(Iteration 28101 / 49000) loss: 1.3155545549\n",
      "(Iteration 28111 / 49000) loss: 1.38894485854\n",
      "(Iteration 28121 / 49000) loss: 1.4504235327\n",
      "(Epoch 287 / 500) Training Accuracy: 0.528489795918, Validation Accuracy: 0.462\n",
      "(Iteration 28131 / 49000) loss: 1.4339776342\n",
      "(Iteration 28141 / 49000) loss: 1.32657013792\n",
      "(Iteration 28151 / 49000) loss: 1.30139483921\n",
      "(Iteration 28161 / 49000) loss: 1.47254522121\n",
      "(Iteration 28171 / 49000) loss: 1.45540425492\n",
      "(Iteration 28181 / 49000) loss: 1.51344600632\n",
      "(Iteration 28191 / 49000) loss: 1.31498480948\n",
      "(Iteration 28201 / 49000) loss: 1.38740540134\n",
      "(Iteration 28211 / 49000) loss: 1.41700745752\n",
      "(Iteration 28221 / 49000) loss: 1.35306966036\n",
      "(Epoch 288 / 500) Training Accuracy: 0.525897959184, Validation Accuracy: 0.464\n",
      "(Iteration 28231 / 49000) loss: 1.34500809109\n",
      "(Iteration 28241 / 49000) loss: 1.31117058703\n",
      "(Iteration 28251 / 49000) loss: 1.30437373409\n",
      "(Iteration 28261 / 49000) loss: 1.40075854076\n",
      "(Iteration 28271 / 49000) loss: 1.36035324173\n",
      "(Iteration 28281 / 49000) loss: 1.42026097932\n",
      "(Iteration 28291 / 49000) loss: 1.45874915296\n",
      "(Iteration 28301 / 49000) loss: 1.38704717416\n",
      "(Iteration 28311 / 49000) loss: 1.42491906346\n",
      "(Iteration 28321 / 49000) loss: 1.41569196067\n",
      "(Epoch 289 / 500) Training Accuracy: 0.519489795918, Validation Accuracy: 0.464\n",
      "(Iteration 28331 / 49000) loss: 1.34054185234\n",
      "(Iteration 28341 / 49000) loss: 1.34175373998\n",
      "(Iteration 28351 / 49000) loss: 1.38105338973\n",
      "(Iteration 28361 / 49000) loss: 1.32170210735\n",
      "(Iteration 28371 / 49000) loss: 1.37494510078\n",
      "(Iteration 28381 / 49000) loss: 1.31567830625\n",
      "(Iteration 28391 / 49000) loss: 1.4867669642\n",
      "(Iteration 28401 / 49000) loss: 1.3138223816\n",
      "(Iteration 28411 / 49000) loss: 1.46337961006\n",
      "(Epoch 290 / 500) Training Accuracy: 0.525734693878, Validation Accuracy: 0.455\n",
      "(Iteration 28421 / 49000) loss: 1.40179245201\n",
      "(Iteration 28431 / 49000) loss: 1.30603203382\n",
      "(Iteration 28441 / 49000) loss: 1.35977800317\n",
      "(Iteration 28451 / 49000) loss: 1.49469212895\n",
      "(Iteration 28461 / 49000) loss: 1.30639716566\n",
      "(Iteration 28471 / 49000) loss: 1.35028838982\n",
      "(Iteration 28481 / 49000) loss: 1.37257182359\n",
      "(Iteration 28491 / 49000) loss: 1.36232929269\n",
      "(Iteration 28501 / 49000) loss: 1.43994158057\n",
      "(Iteration 28511 / 49000) loss: 1.38923783711\n",
      "(Epoch 291 / 500) Training Accuracy: 0.520142857143, Validation Accuracy: 0.461\n",
      "(Iteration 28521 / 49000) loss: 1.31186064429\n",
      "(Iteration 28531 / 49000) loss: 1.35027699405\n",
      "(Iteration 28541 / 49000) loss: 1.33802373233\n",
      "(Iteration 28551 / 49000) loss: 1.38170500929\n",
      "(Iteration 28561 / 49000) loss: 1.34150025569\n",
      "(Iteration 28571 / 49000) loss: 1.33616115129\n",
      "(Iteration 28581 / 49000) loss: 1.3529260452\n",
      "(Iteration 28591 / 49000) loss: 1.32117187478\n",
      "(Iteration 28601 / 49000) loss: 1.38878306305\n",
      "(Iteration 28611 / 49000) loss: 1.29521877605\n",
      "(Epoch 292 / 500) Training Accuracy: 0.522795918367, Validation Accuracy: 0.468\n",
      "(Iteration 28621 / 49000) loss: 1.2846166867\n",
      "(Iteration 28631 / 49000) loss: 1.29092000715\n",
      "(Iteration 28641 / 49000) loss: 1.36693768463\n",
      "(Iteration 28651 / 49000) loss: 1.37023839806\n",
      "(Iteration 28661 / 49000) loss: 1.42091148447\n",
      "(Iteration 28671 / 49000) loss: 1.32217098912\n",
      "(Iteration 28681 / 49000) loss: 1.34716825073\n",
      "(Iteration 28691 / 49000) loss: 1.3902170471\n",
      "(Iteration 28701 / 49000) loss: 1.42597031352\n",
      "(Iteration 28711 / 49000) loss: 1.41380075292\n",
      "(Epoch 293 / 500) Training Accuracy: 0.523367346939, Validation Accuracy: 0.459\n",
      "(Iteration 28721 / 49000) loss: 1.39237160946\n",
      "(Iteration 28731 / 49000) loss: 1.30464926541\n",
      "(Iteration 28741 / 49000) loss: 1.35690318745\n",
      "(Iteration 28751 / 49000) loss: 1.42136670435\n",
      "(Iteration 28761 / 49000) loss: 1.37612253231\n",
      "(Iteration 28771 / 49000) loss: 1.46042245216\n",
      "(Iteration 28781 / 49000) loss: 1.33335538406\n",
      "(Iteration 28791 / 49000) loss: 1.31082866416\n",
      "(Iteration 28801 / 49000) loss: 1.32916295483\n",
      "(Iteration 28811 / 49000) loss: 1.33690186098\n",
      "(Epoch 294 / 500) Training Accuracy: 0.526306122449, Validation Accuracy: 0.468\n",
      "(Iteration 28821 / 49000) loss: 1.44445847621\n",
      "(Iteration 28831 / 49000) loss: 1.38810570251\n",
      "(Iteration 28841 / 49000) loss: 1.30119750449\n",
      "(Iteration 28851 / 49000) loss: 1.41372257526\n",
      "(Iteration 28861 / 49000) loss: 1.39303981743\n",
      "(Iteration 28871 / 49000) loss: 1.39507044729\n",
      "(Iteration 28881 / 49000) loss: 1.38532864399\n",
      "(Iteration 28891 / 49000) loss: 1.36216092181\n",
      "(Iteration 28901 / 49000) loss: 1.34643290291\n",
      "(Epoch 295 / 500) Training Accuracy: 0.52712244898, Validation Accuracy: 0.466\n",
      "(Iteration 28911 / 49000) loss: 1.36126433696\n",
      "(Iteration 28921 / 49000) loss: 1.36424859139\n",
      "(Iteration 28931 / 49000) loss: 1.31135439127\n",
      "(Iteration 28941 / 49000) loss: 1.42034218154\n",
      "(Iteration 28951 / 49000) loss: 1.40833588477\n",
      "(Iteration 28961 / 49000) loss: 1.40366498894\n",
      "(Iteration 28971 / 49000) loss: 1.32077408742\n",
      "(Iteration 28981 / 49000) loss: 1.38896254855\n",
      "(Iteration 28991 / 49000) loss: 1.38144871827\n",
      "(Iteration 29001 / 49000) loss: 1.29493498284\n",
      "(Epoch 296 / 500) Training Accuracy: 0.52693877551, Validation Accuracy: 0.45\n",
      "(Iteration 29011 / 49000) loss: 1.34489497026\n",
      "(Iteration 29021 / 49000) loss: 1.41093862009\n",
      "(Iteration 29031 / 49000) loss: 1.36185887315\n",
      "(Iteration 29041 / 49000) loss: 1.30093255507\n",
      "(Iteration 29051 / 49000) loss: 1.4012906199\n",
      "(Iteration 29061 / 49000) loss: 1.43803623224\n",
      "(Iteration 29071 / 49000) loss: 1.32383670301\n",
      "(Iteration 29081 / 49000) loss: 1.29555854373\n",
      "(Iteration 29091 / 49000) loss: 1.4600618191\n",
      "(Iteration 29101 / 49000) loss: 1.35189796436\n",
      "(Epoch 297 / 500) Training Accuracy: 0.528836734694, Validation Accuracy: 0.47\n",
      "(Iteration 29111 / 49000) loss: 1.38789057169\n",
      "(Iteration 29121 / 49000) loss: 1.40738960375\n",
      "(Iteration 29131 / 49000) loss: 1.28277178971\n",
      "(Iteration 29141 / 49000) loss: 1.24079393789\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 29151 / 49000) loss: 1.29326803689\n",
      "(Iteration 29161 / 49000) loss: 1.24925286107\n",
      "(Iteration 29171 / 49000) loss: 1.37823153077\n",
      "(Iteration 29181 / 49000) loss: 1.46138968391\n",
      "(Iteration 29191 / 49000) loss: 1.38663385809\n",
      "(Iteration 29201 / 49000) loss: 1.43415121373\n",
      "(Epoch 298 / 500) Training Accuracy: 0.524387755102, Validation Accuracy: 0.457\n",
      "(Iteration 29211 / 49000) loss: 1.42740948546\n",
      "(Iteration 29221 / 49000) loss: 1.40352198932\n",
      "(Iteration 29231 / 49000) loss: 1.34643216071\n",
      "(Iteration 29241 / 49000) loss: 1.40715998317\n",
      "(Iteration 29251 / 49000) loss: 1.43035102961\n",
      "(Iteration 29261 / 49000) loss: 1.31891631058\n",
      "(Iteration 29271 / 49000) loss: 1.37531833778\n",
      "(Iteration 29281 / 49000) loss: 1.39255833963\n",
      "(Iteration 29291 / 49000) loss: 1.49405146767\n",
      "(Iteration 29301 / 49000) loss: 1.48246906153\n",
      "(Epoch 299 / 500) Training Accuracy: 0.529918367347, Validation Accuracy: 0.469\n",
      "(Iteration 29311 / 49000) loss: 1.32119814325\n",
      "(Iteration 29321 / 49000) loss: 1.41835360402\n",
      "(Iteration 29331 / 49000) loss: 1.24815247445\n",
      "(Iteration 29341 / 49000) loss: 1.34245652344\n",
      "(Iteration 29351 / 49000) loss: 1.41650895971\n",
      "(Iteration 29361 / 49000) loss: 1.33239919252\n",
      "(Iteration 29371 / 49000) loss: 1.34497256524\n",
      "(Iteration 29381 / 49000) loss: 1.36256824033\n",
      "(Iteration 29391 / 49000) loss: 1.3641887305\n",
      "(Epoch 300 / 500) Training Accuracy: 0.527673469388, Validation Accuracy: 0.461\n",
      "(Iteration 29401 / 49000) loss: 1.37465833397\n",
      "(Iteration 29411 / 49000) loss: 1.36918698621\n",
      "(Iteration 29421 / 49000) loss: 1.34069298268\n",
      "(Iteration 29431 / 49000) loss: 1.37786585501\n",
      "(Iteration 29441 / 49000) loss: 1.4131949773\n",
      "(Iteration 29451 / 49000) loss: 1.30997516393\n",
      "(Iteration 29461 / 49000) loss: 1.35174925742\n",
      "(Iteration 29471 / 49000) loss: 1.37509977714\n",
      "(Iteration 29481 / 49000) loss: 1.47399587627\n",
      "(Iteration 29491 / 49000) loss: 1.3681363272\n",
      "(Epoch 301 / 500) Training Accuracy: 0.528653061224, Validation Accuracy: 0.462\n",
      "(Iteration 29501 / 49000) loss: 1.40592267471\n",
      "(Iteration 29511 / 49000) loss: 1.31545923059\n",
      "(Iteration 29521 / 49000) loss: 1.29783365508\n",
      "(Iteration 29531 / 49000) loss: 1.31619727139\n",
      "(Iteration 29541 / 49000) loss: 1.36987255731\n",
      "(Iteration 29551 / 49000) loss: 1.26924414311\n",
      "(Iteration 29561 / 49000) loss: 1.33033942483\n",
      "(Iteration 29571 / 49000) loss: 1.2678307809\n",
      "(Iteration 29581 / 49000) loss: 1.34263730236\n",
      "(Iteration 29591 / 49000) loss: 1.44973415541\n",
      "(Epoch 302 / 500) Training Accuracy: 0.526795918367, Validation Accuracy: 0.467\n",
      "(Iteration 29601 / 49000) loss: 1.40806466021\n",
      "(Iteration 29611 / 49000) loss: 1.32833482935\n",
      "(Iteration 29621 / 49000) loss: 1.30751807096\n",
      "(Iteration 29631 / 49000) loss: 1.34854491702\n",
      "(Iteration 29641 / 49000) loss: 1.3422006527\n",
      "(Iteration 29651 / 49000) loss: 1.43943331862\n",
      "(Iteration 29661 / 49000) loss: 1.31706530147\n",
      "(Iteration 29671 / 49000) loss: 1.25623553421\n",
      "(Iteration 29681 / 49000) loss: 1.48359752376\n",
      "(Iteration 29691 / 49000) loss: 1.31293266505\n",
      "(Epoch 303 / 500) Training Accuracy: 0.530469387755, Validation Accuracy: 0.468\n",
      "(Iteration 29701 / 49000) loss: 1.35581473262\n",
      "(Iteration 29711 / 49000) loss: 1.31647124135\n",
      "(Iteration 29721 / 49000) loss: 1.4808252433\n",
      "(Iteration 29731 / 49000) loss: 1.36260062581\n",
      "(Iteration 29741 / 49000) loss: 1.39096230041\n",
      "(Iteration 29751 / 49000) loss: 1.34165178086\n",
      "(Iteration 29761 / 49000) loss: 1.43017571904\n",
      "(Iteration 29771 / 49000) loss: 1.35145837006\n",
      "(Iteration 29781 / 49000) loss: 1.34041012454\n",
      "(Iteration 29791 / 49000) loss: 1.45803640365\n",
      "(Epoch 304 / 500) Training Accuracy: 0.529734693878, Validation Accuracy: 0.469\n",
      "(Iteration 29801 / 49000) loss: 1.42681296619\n",
      "(Iteration 29811 / 49000) loss: 1.35835947833\n",
      "(Iteration 29821 / 49000) loss: 1.2973808638\n",
      "(Iteration 29831 / 49000) loss: 1.43653489533\n",
      "(Iteration 29841 / 49000) loss: 1.3772349489\n",
      "(Iteration 29851 / 49000) loss: 1.3386429473\n",
      "(Iteration 29861 / 49000) loss: 1.42032109998\n",
      "(Iteration 29871 / 49000) loss: 1.29086606358\n",
      "(Iteration 29881 / 49000) loss: 1.43510098073\n",
      "(Epoch 305 / 500) Training Accuracy: 0.528489795918, Validation Accuracy: 0.475\n",
      "(Iteration 29891 / 49000) loss: 1.39621911554\n",
      "(Iteration 29901 / 49000) loss: 1.31870549288\n",
      "(Iteration 29911 / 49000) loss: 1.31350386737\n",
      "(Iteration 29921 / 49000) loss: 1.42497966964\n",
      "(Iteration 29931 / 49000) loss: 1.40767698325\n",
      "(Iteration 29941 / 49000) loss: 1.41533746784\n",
      "(Iteration 29951 / 49000) loss: 1.39081633867\n",
      "(Iteration 29961 / 49000) loss: 1.38270258578\n",
      "(Iteration 29971 / 49000) loss: 1.30693593035\n",
      "(Iteration 29981 / 49000) loss: 1.32999467901\n",
      "(Epoch 306 / 500) Training Accuracy: 0.526857142857, Validation Accuracy: 0.458\n",
      "(Iteration 29991 / 49000) loss: 1.31577462766\n",
      "(Iteration 30001 / 49000) loss: 1.31044156534\n",
      "(Iteration 30011 / 49000) loss: 1.3524000158\n",
      "(Iteration 30021 / 49000) loss: 1.25609816574\n",
      "(Iteration 30031 / 49000) loss: 1.39191155279\n",
      "(Iteration 30041 / 49000) loss: 1.42378175874\n",
      "(Iteration 30051 / 49000) loss: 1.47571823025\n",
      "(Iteration 30061 / 49000) loss: 1.42742904052\n",
      "(Iteration 30071 / 49000) loss: 1.34637213769\n",
      "(Iteration 30081 / 49000) loss: 1.31782780743\n",
      "(Epoch 307 / 500) Training Accuracy: 0.530346938776, Validation Accuracy: 0.472\n",
      "(Iteration 30091 / 49000) loss: 1.3204284632\n",
      "(Iteration 30101 / 49000) loss: 1.37073720427\n",
      "(Iteration 30111 / 49000) loss: 1.39161302081\n",
      "(Iteration 30121 / 49000) loss: 1.37481606301\n",
      "(Iteration 30131 / 49000) loss: 1.39207250759\n",
      "(Iteration 30141 / 49000) loss: 1.38794084475\n",
      "(Iteration 30151 / 49000) loss: 1.34243132229\n",
      "(Iteration 30161 / 49000) loss: 1.40429975407\n",
      "(Iteration 30171 / 49000) loss: 1.37138393752\n",
      "(Iteration 30181 / 49000) loss: 1.42663926975\n",
      "(Epoch 308 / 500) Training Accuracy: 0.527897959184, Validation Accuracy: 0.462\n",
      "(Iteration 30191 / 49000) loss: 1.31344666275\n",
      "(Iteration 30201 / 49000) loss: 1.34257371173\n",
      "(Iteration 30211 / 49000) loss: 1.45468585126\n",
      "(Iteration 30221 / 49000) loss: 1.3981063004\n",
      "(Iteration 30231 / 49000) loss: 1.3501936466\n",
      "(Iteration 30241 / 49000) loss: 1.29002442105\n",
      "(Iteration 30251 / 49000) loss: 1.41098225935\n",
      "(Iteration 30261 / 49000) loss: 1.43373989623\n",
      "(Iteration 30271 / 49000) loss: 1.41851366313\n",
      "(Iteration 30281 / 49000) loss: 1.32149774462\n",
      "(Epoch 309 / 500) Training Accuracy: 0.526020408163, Validation Accuracy: 0.465\n",
      "(Iteration 30291 / 49000) loss: 1.32127561263\n",
      "(Iteration 30301 / 49000) loss: 1.35814979249\n",
      "(Iteration 30311 / 49000) loss: 1.37408881432\n",
      "(Iteration 30321 / 49000) loss: 1.52296778578\n",
      "(Iteration 30331 / 49000) loss: 1.24579202479\n",
      "(Iteration 30341 / 49000) loss: 1.38582091495\n",
      "(Iteration 30351 / 49000) loss: 1.33143434065\n",
      "(Iteration 30361 / 49000) loss: 1.33958694356\n",
      "(Iteration 30371 / 49000) loss: 1.34573091457\n",
      "(Epoch 310 / 500) Training Accuracy: 0.530857142857, Validation Accuracy: 0.472\n",
      "(Iteration 30381 / 49000) loss: 1.29361697925\n",
      "(Iteration 30391 / 49000) loss: 1.4370016007\n",
      "(Iteration 30401 / 49000) loss: 1.29719970582\n",
      "(Iteration 30411 / 49000) loss: 1.34636308936\n",
      "(Iteration 30421 / 49000) loss: 1.36984628791\n",
      "(Iteration 30431 / 49000) loss: 1.38809267562\n",
      "(Iteration 30441 / 49000) loss: 1.30888332477\n",
      "(Iteration 30451 / 49000) loss: 1.35437454525\n",
      "(Iteration 30461 / 49000) loss: 1.32458378109\n",
      "(Iteration 30471 / 49000) loss: 1.34080482569\n",
      "(Epoch 311 / 500) Training Accuracy: 0.521959183673, Validation Accuracy: 0.448\n",
      "(Iteration 30481 / 49000) loss: 1.44858392235\n",
      "(Iteration 30491 / 49000) loss: 1.39186700576\n",
      "(Iteration 30501 / 49000) loss: 1.36786579779\n",
      "(Iteration 30511 / 49000) loss: 1.26859733699\n",
      "(Iteration 30521 / 49000) loss: 1.34316266821\n",
      "(Iteration 30531 / 49000) loss: 1.39233571471\n",
      "(Iteration 30541 / 49000) loss: 1.26903698679\n",
      "(Iteration 30551 / 49000) loss: 1.32211677804\n",
      "(Iteration 30561 / 49000) loss: 1.32067652317\n",
      "(Iteration 30571 / 49000) loss: 1.34050666396\n",
      "(Epoch 312 / 500) Training Accuracy: 0.524265306122, Validation Accuracy: 0.458\n",
      "(Iteration 30581 / 49000) loss: 1.30372245357\n",
      "(Iteration 30591 / 49000) loss: 1.3368138127\n",
      "(Iteration 30601 / 49000) loss: 1.30409131304\n",
      "(Iteration 30611 / 49000) loss: 1.33457206571\n",
      "(Iteration 30621 / 49000) loss: 1.37079380889\n",
      "(Iteration 30631 / 49000) loss: 1.38704730532\n",
      "(Iteration 30641 / 49000) loss: 1.50072575873\n",
      "(Iteration 30651 / 49000) loss: 1.34915061714\n",
      "(Iteration 30661 / 49000) loss: 1.32407028651\n",
      "(Iteration 30671 / 49000) loss: 1.37157851147\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Epoch 313 / 500) Training Accuracy: 0.530734693878, Validation Accuracy: 0.473\n",
      "(Iteration 30681 / 49000) loss: 1.30361162611\n",
      "(Iteration 30691 / 49000) loss: 1.36876956805\n",
      "(Iteration 30701 / 49000) loss: 1.29399727544\n",
      "(Iteration 30711 / 49000) loss: 1.28992646685\n",
      "(Iteration 30721 / 49000) loss: 1.33972389577\n",
      "(Iteration 30731 / 49000) loss: 1.3555126132\n",
      "(Iteration 30741 / 49000) loss: 1.35501335827\n",
      "(Iteration 30751 / 49000) loss: 1.34863430829\n",
      "(Iteration 30761 / 49000) loss: 1.38155895473\n",
      "(Iteration 30771 / 49000) loss: 1.44454774571\n",
      "(Epoch 314 / 500) Training Accuracy: 0.530795918367, Validation Accuracy: 0.47\n",
      "(Iteration 30781 / 49000) loss: 1.34139495879\n",
      "(Iteration 30791 / 49000) loss: 1.30120342954\n",
      "(Iteration 30801 / 49000) loss: 1.32886547193\n",
      "(Iteration 30811 / 49000) loss: 1.33883084522\n",
      "(Iteration 30821 / 49000) loss: 1.40456226964\n",
      "(Iteration 30831 / 49000) loss: 1.39009158314\n",
      "(Iteration 30841 / 49000) loss: 1.21586736882\n",
      "(Iteration 30851 / 49000) loss: 1.36102418508\n",
      "(Iteration 30861 / 49000) loss: 1.31096917272\n",
      "(Epoch 315 / 500) Training Accuracy: 0.527612244898, Validation Accuracy: 0.453\n",
      "(Iteration 30871 / 49000) loss: 1.22531791003\n",
      "(Iteration 30881 / 49000) loss: 1.34646577135\n",
      "(Iteration 30891 / 49000) loss: 1.32107601437\n",
      "(Iteration 30901 / 49000) loss: 1.43923772737\n",
      "(Iteration 30911 / 49000) loss: 1.32447253235\n",
      "(Iteration 30921 / 49000) loss: 1.36892702131\n",
      "(Iteration 30931 / 49000) loss: 1.4044899877\n",
      "(Iteration 30941 / 49000) loss: 1.46753101712\n",
      "(Iteration 30951 / 49000) loss: 1.26058146045\n",
      "(Iteration 30961 / 49000) loss: 1.38046369061\n",
      "(Epoch 316 / 500) Training Accuracy: 0.526612244898, Validation Accuracy: 0.469\n",
      "(Iteration 30971 / 49000) loss: 1.38494000478\n",
      "(Iteration 30981 / 49000) loss: 1.37892966083\n",
      "(Iteration 30991 / 49000) loss: 1.38616103505\n",
      "(Iteration 31001 / 49000) loss: 1.24903938217\n",
      "(Iteration 31011 / 49000) loss: 1.38719604572\n",
      "(Iteration 31021 / 49000) loss: 1.27862776237\n",
      "(Iteration 31031 / 49000) loss: 1.36388202253\n",
      "(Iteration 31041 / 49000) loss: 1.42454943851\n",
      "(Iteration 31051 / 49000) loss: 1.41826082484\n",
      "(Iteration 31061 / 49000) loss: 1.35611197273\n",
      "(Epoch 317 / 500) Training Accuracy: 0.531632653061, Validation Accuracy: 0.465\n",
      "(Iteration 31071 / 49000) loss: 1.40946462508\n",
      "(Iteration 31081 / 49000) loss: 1.29060640482\n",
      "(Iteration 31091 / 49000) loss: 1.37892276646\n",
      "(Iteration 31101 / 49000) loss: 1.30056455825\n",
      "(Iteration 31111 / 49000) loss: 1.38513623191\n",
      "(Iteration 31121 / 49000) loss: 1.42993493987\n",
      "(Iteration 31131 / 49000) loss: 1.34816385651\n",
      "(Iteration 31141 / 49000) loss: 1.29460239793\n",
      "(Iteration 31151 / 49000) loss: 1.28869456482\n",
      "(Iteration 31161 / 49000) loss: 1.28618472452\n",
      "(Epoch 318 / 500) Training Accuracy: 0.533755102041, Validation Accuracy: 0.478\n",
      "(Iteration 31171 / 49000) loss: 1.29313540169\n",
      "(Iteration 31181 / 49000) loss: 1.36616447708\n",
      "(Iteration 31191 / 49000) loss: 1.3933212658\n",
      "(Iteration 31201 / 49000) loss: 1.34225366478\n",
      "(Iteration 31211 / 49000) loss: 1.32556295287\n",
      "(Iteration 31221 / 49000) loss: 1.35036721897\n",
      "(Iteration 31231 / 49000) loss: 1.34078705673\n",
      "(Iteration 31241 / 49000) loss: 1.32905120231\n",
      "(Iteration 31251 / 49000) loss: 1.46587582028\n",
      "(Iteration 31261 / 49000) loss: 1.44035168016\n",
      "(Epoch 319 / 500) Training Accuracy: 0.529571428571, Validation Accuracy: 0.47\n",
      "(Iteration 31271 / 49000) loss: 1.34661914086\n",
      "(Iteration 31281 / 49000) loss: 1.36157135465\n",
      "(Iteration 31291 / 49000) loss: 1.28896892329\n",
      "(Iteration 31301 / 49000) loss: 1.35485668244\n",
      "(Iteration 31311 / 49000) loss: 1.31688645988\n",
      "(Iteration 31321 / 49000) loss: 1.37187147966\n",
      "(Iteration 31331 / 49000) loss: 1.40919797958\n",
      "(Iteration 31341 / 49000) loss: 1.33015950369\n",
      "(Iteration 31351 / 49000) loss: 1.38327284276\n",
      "(Epoch 320 / 500) Training Accuracy: 0.527489795918, Validation Accuracy: 0.471\n",
      "(Iteration 31361 / 49000) loss: 1.41763283975\n",
      "(Iteration 31371 / 49000) loss: 1.3744441982\n",
      "(Iteration 31381 / 49000) loss: 1.39794533656\n",
      "(Iteration 31391 / 49000) loss: 1.29862989025\n",
      "(Iteration 31401 / 49000) loss: 1.35809781719\n",
      "(Iteration 31411 / 49000) loss: 1.34312080534\n",
      "(Iteration 31421 / 49000) loss: 1.28348861522\n",
      "(Iteration 31431 / 49000) loss: 1.304138201\n",
      "(Iteration 31441 / 49000) loss: 1.42099112328\n",
      "(Iteration 31451 / 49000) loss: 1.35915068614\n",
      "(Epoch 321 / 500) Training Accuracy: 0.535510204082, Validation Accuracy: 0.467\n",
      "(Iteration 31461 / 49000) loss: 1.33173849852\n",
      "(Iteration 31471 / 49000) loss: 1.2815581399\n",
      "(Iteration 31481 / 49000) loss: 1.33986104401\n",
      "(Iteration 31491 / 49000) loss: 1.41106454194\n",
      "(Iteration 31501 / 49000) loss: 1.40271371671\n",
      "(Iteration 31511 / 49000) loss: 1.28927530627\n",
      "(Iteration 31521 / 49000) loss: 1.3036900656\n",
      "(Iteration 31531 / 49000) loss: 1.37190143675\n",
      "(Iteration 31541 / 49000) loss: 1.3570711787\n",
      "(Iteration 31551 / 49000) loss: 1.2985239637\n",
      "(Epoch 322 / 500) Training Accuracy: 0.528326530612, Validation Accuracy: 0.47\n",
      "(Iteration 31561 / 49000) loss: 1.33365777674\n",
      "(Iteration 31571 / 49000) loss: 1.32245084031\n",
      "(Iteration 31581 / 49000) loss: 1.43107803967\n",
      "(Iteration 31591 / 49000) loss: 1.28320249574\n",
      "(Iteration 31601 / 49000) loss: 1.31752141975\n",
      "(Iteration 31611 / 49000) loss: 1.23491327161\n",
      "(Iteration 31621 / 49000) loss: 1.36565278869\n",
      "(Iteration 31631 / 49000) loss: 1.44095441917\n",
      "(Iteration 31641 / 49000) loss: 1.30388414558\n",
      "(Iteration 31651 / 49000) loss: 1.41771497015\n",
      "(Epoch 323 / 500) Training Accuracy: 0.533775510204, Validation Accuracy: 0.463\n",
      "(Iteration 31661 / 49000) loss: 1.34165754638\n",
      "(Iteration 31671 / 49000) loss: 1.35218636621\n",
      "(Iteration 31681 / 49000) loss: 1.38125778519\n",
      "(Iteration 31691 / 49000) loss: 1.34092675461\n",
      "(Iteration 31701 / 49000) loss: 1.43534279681\n",
      "(Iteration 31711 / 49000) loss: 1.41558371389\n",
      "(Iteration 31721 / 49000) loss: 1.2693317037\n",
      "(Iteration 31731 / 49000) loss: 1.25211182358\n",
      "(Iteration 31741 / 49000) loss: 1.42024055302\n",
      "(Iteration 31751 / 49000) loss: 1.43013364094\n",
      "(Epoch 324 / 500) Training Accuracy: 0.536755102041, Validation Accuracy: 0.459\n",
      "(Iteration 31761 / 49000) loss: 1.29205959674\n",
      "(Iteration 31771 / 49000) loss: 1.32550980172\n",
      "(Iteration 31781 / 49000) loss: 1.34621588272\n",
      "(Iteration 31791 / 49000) loss: 1.37228783986\n",
      "(Iteration 31801 / 49000) loss: 1.27911787282\n",
      "(Iteration 31811 / 49000) loss: 1.390874901\n",
      "(Iteration 31821 / 49000) loss: 1.29002536274\n",
      "(Iteration 31831 / 49000) loss: 1.22050115621\n",
      "(Iteration 31841 / 49000) loss: 1.37823770109\n",
      "(Epoch 325 / 500) Training Accuracy: 0.536183673469, Validation Accuracy: 0.46\n",
      "(Iteration 31851 / 49000) loss: 1.31617355373\n",
      "(Iteration 31861 / 49000) loss: 1.37182104202\n",
      "(Iteration 31871 / 49000) loss: 1.38136067634\n",
      "(Iteration 31881 / 49000) loss: 1.45730603475\n",
      "(Iteration 31891 / 49000) loss: 1.32163891323\n",
      "(Iteration 31901 / 49000) loss: 1.37405060675\n",
      "(Iteration 31911 / 49000) loss: 1.30381836087\n",
      "(Iteration 31921 / 49000) loss: 1.37193763296\n",
      "(Iteration 31931 / 49000) loss: 1.29191975763\n",
      "(Iteration 31941 / 49000) loss: 1.30924898435\n",
      "(Epoch 326 / 500) Training Accuracy: 0.527469387755, Validation Accuracy: 0.471\n",
      "(Iteration 31951 / 49000) loss: 1.22197645598\n",
      "(Iteration 31961 / 49000) loss: 1.34060450894\n",
      "(Iteration 31971 / 49000) loss: 1.45260666968\n",
      "(Iteration 31981 / 49000) loss: 1.36212061045\n",
      "(Iteration 31991 / 49000) loss: 1.33830121609\n",
      "(Iteration 32001 / 49000) loss: 1.34415388119\n",
      "(Iteration 32011 / 49000) loss: 1.43613067786\n",
      "(Iteration 32021 / 49000) loss: 1.30710299841\n",
      "(Iteration 32031 / 49000) loss: 1.37985353674\n",
      "(Iteration 32041 / 49000) loss: 1.38502356841\n",
      "(Epoch 327 / 500) Training Accuracy: 0.535836734694, Validation Accuracy: 0.463\n",
      "(Iteration 32051 / 49000) loss: 1.38127992387\n",
      "(Iteration 32061 / 49000) loss: 1.40344781851\n",
      "(Iteration 32071 / 49000) loss: 1.25922922036\n",
      "(Iteration 32081 / 49000) loss: 1.33459179749\n",
      "(Iteration 32091 / 49000) loss: 1.33617019252\n",
      "(Iteration 32101 / 49000) loss: 1.23395812073\n",
      "(Iteration 32111 / 49000) loss: 1.41731972856\n",
      "(Iteration 32121 / 49000) loss: 1.32449665542\n",
      "(Iteration 32131 / 49000) loss: 1.36844597889\n",
      "(Iteration 32141 / 49000) loss: 1.36563659481\n",
      "(Epoch 328 / 500) Training Accuracy: 0.536448979592, Validation Accuracy: 0.457\n",
      "(Iteration 32151 / 49000) loss: 1.32960121894\n",
      "(Iteration 32161 / 49000) loss: 1.29411248929\n",
      "(Iteration 32171 / 49000) loss: 1.45245432881\n",
      "(Iteration 32181 / 49000) loss: 1.31029663166\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 32191 / 49000) loss: 1.28887426389\n",
      "(Iteration 32201 / 49000) loss: 1.32173064474\n",
      "(Iteration 32211 / 49000) loss: 1.37187323451\n",
      "(Iteration 32221 / 49000) loss: 1.36760469942\n",
      "(Iteration 32231 / 49000) loss: 1.38772753159\n",
      "(Iteration 32241 / 49000) loss: 1.37694998716\n",
      "(Epoch 329 / 500) Training Accuracy: 0.530836734694, Validation Accuracy: 0.465\n",
      "(Iteration 32251 / 49000) loss: 1.27445224649\n",
      "(Iteration 32261 / 49000) loss: 1.39792440179\n",
      "(Iteration 32271 / 49000) loss: 1.37590694016\n",
      "(Iteration 32281 / 49000) loss: 1.30736688491\n",
      "(Iteration 32291 / 49000) loss: 1.36639680064\n",
      "(Iteration 32301 / 49000) loss: 1.23895255711\n",
      "(Iteration 32311 / 49000) loss: 1.33553324619\n",
      "(Iteration 32321 / 49000) loss: 1.34035854599\n",
      "(Iteration 32331 / 49000) loss: 1.44418468506\n",
      "(Epoch 330 / 500) Training Accuracy: 0.535, Validation Accuracy: 0.467\n",
      "(Iteration 32341 / 49000) loss: 1.32665003222\n",
      "(Iteration 32351 / 49000) loss: 1.37905222267\n",
      "(Iteration 32361 / 49000) loss: 1.31241384294\n",
      "(Iteration 32371 / 49000) loss: 1.32933126229\n",
      "(Iteration 32381 / 49000) loss: 1.31511207642\n",
      "(Iteration 32391 / 49000) loss: 1.35453261978\n",
      "(Iteration 32401 / 49000) loss: 1.38476191825\n",
      "(Iteration 32411 / 49000) loss: 1.32050523338\n",
      "(Iteration 32421 / 49000) loss: 1.40574022503\n",
      "(Iteration 32431 / 49000) loss: 1.30253582997\n",
      "(Epoch 331 / 500) Training Accuracy: 0.537693877551, Validation Accuracy: 0.473\n",
      "(Iteration 32441 / 49000) loss: 1.26919570614\n",
      "(Iteration 32451 / 49000) loss: 1.26215878051\n",
      "(Iteration 32461 / 49000) loss: 1.39065764572\n",
      "(Iteration 32471 / 49000) loss: 1.39991521174\n",
      "(Iteration 32481 / 49000) loss: 1.31596166364\n",
      "(Iteration 32491 / 49000) loss: 1.25666213717\n",
      "(Iteration 32501 / 49000) loss: 1.25488775848\n",
      "(Iteration 32511 / 49000) loss: 1.27117912919\n",
      "(Iteration 32521 / 49000) loss: 1.32580515975\n",
      "(Iteration 32531 / 49000) loss: 1.4622559453\n",
      "(Epoch 332 / 500) Training Accuracy: 0.534040816327, Validation Accuracy: 0.486\n",
      "(Iteration 32541 / 49000) loss: 1.36263635117\n",
      "(Iteration 32551 / 49000) loss: 1.29839638562\n",
      "(Iteration 32561 / 49000) loss: 1.31970898626\n",
      "(Iteration 32571 / 49000) loss: 1.30849987284\n",
      "(Iteration 32581 / 49000) loss: 1.30976748413\n",
      "(Iteration 32591 / 49000) loss: 1.31784352675\n",
      "(Iteration 32601 / 49000) loss: 1.32921367053\n",
      "(Iteration 32611 / 49000) loss: 1.26114746351\n",
      "(Iteration 32621 / 49000) loss: 1.27522093521\n",
      "(Iteration 32631 / 49000) loss: 1.305547179\n",
      "(Epoch 333 / 500) Training Accuracy: 0.536265306122, Validation Accuracy: 0.47\n",
      "(Iteration 32641 / 49000) loss: 1.46567816744\n",
      "(Iteration 32651 / 49000) loss: 1.341558742\n",
      "(Iteration 32661 / 49000) loss: 1.40449569788\n",
      "(Iteration 32671 / 49000) loss: 1.33994511725\n",
      "(Iteration 32681 / 49000) loss: 1.29214608957\n",
      "(Iteration 32691 / 49000) loss: 1.36951471646\n",
      "(Iteration 32701 / 49000) loss: 1.30729375696\n",
      "(Iteration 32711 / 49000) loss: 1.23318355395\n",
      "(Iteration 32721 / 49000) loss: 1.48025868829\n",
      "(Iteration 32731 / 49000) loss: 1.3454233664\n",
      "(Epoch 334 / 500) Training Accuracy: 0.534897959184, Validation Accuracy: 0.462\n",
      "(Iteration 32741 / 49000) loss: 1.29235427538\n",
      "(Iteration 32751 / 49000) loss: 1.27857136666\n",
      "(Iteration 32761 / 49000) loss: 1.32560378517\n",
      "(Iteration 32771 / 49000) loss: 1.21787693025\n",
      "(Iteration 32781 / 49000) loss: 1.35083244546\n",
      "(Iteration 32791 / 49000) loss: 1.28999001696\n",
      "(Iteration 32801 / 49000) loss: 1.4081715967\n",
      "(Iteration 32811 / 49000) loss: 1.30809699884\n",
      "(Iteration 32821 / 49000) loss: 1.30283972683\n",
      "(Epoch 335 / 500) Training Accuracy: 0.533040816327, Validation Accuracy: 0.466\n",
      "(Iteration 32831 / 49000) loss: 1.28760380379\n",
      "(Iteration 32841 / 49000) loss: 1.37450071704\n",
      "(Iteration 32851 / 49000) loss: 1.3100918032\n",
      "(Iteration 32861 / 49000) loss: 1.25976715365\n",
      "(Iteration 32871 / 49000) loss: 1.39453969516\n",
      "(Iteration 32881 / 49000) loss: 1.51685262678\n",
      "(Iteration 32891 / 49000) loss: 1.33486227019\n",
      "(Iteration 32901 / 49000) loss: 1.3699769771\n",
      "(Iteration 32911 / 49000) loss: 1.33925680803\n",
      "(Iteration 32921 / 49000) loss: 1.40611479358\n",
      "(Epoch 336 / 500) Training Accuracy: 0.534836734694, Validation Accuracy: 0.462\n",
      "(Iteration 32931 / 49000) loss: 1.40094930584\n",
      "(Iteration 32941 / 49000) loss: 1.46340104042\n",
      "(Iteration 32951 / 49000) loss: 1.29147909963\n",
      "(Iteration 32961 / 49000) loss: 1.3597143628\n",
      "(Iteration 32971 / 49000) loss: 1.38720176143\n",
      "(Iteration 32981 / 49000) loss: 1.3119920338\n",
      "(Iteration 32991 / 49000) loss: 1.24734483125\n",
      "(Iteration 33001 / 49000) loss: 1.30355323847\n",
      "(Iteration 33011 / 49000) loss: 1.30425126198\n",
      "(Iteration 33021 / 49000) loss: 1.36361277729\n",
      "(Epoch 337 / 500) Training Accuracy: 0.536367346939, Validation Accuracy: 0.471\n",
      "(Iteration 33031 / 49000) loss: 1.28422131811\n",
      "(Iteration 33041 / 49000) loss: 1.35439532195\n",
      "(Iteration 33051 / 49000) loss: 1.31751458997\n",
      "(Iteration 33061 / 49000) loss: 1.20886449862\n",
      "(Iteration 33071 / 49000) loss: 1.35458077217\n",
      "(Iteration 33081 / 49000) loss: 1.33989903645\n",
      "(Iteration 33091 / 49000) loss: 1.34834271868\n",
      "(Iteration 33101 / 49000) loss: 1.29167748992\n",
      "(Iteration 33111 / 49000) loss: 1.29061781581\n",
      "(Iteration 33121 / 49000) loss: 1.33994312844\n",
      "(Epoch 338 / 500) Training Accuracy: 0.538367346939, Validation Accuracy: 0.468\n",
      "(Iteration 33131 / 49000) loss: 1.320028715\n",
      "(Iteration 33141 / 49000) loss: 1.2876608146\n",
      "(Iteration 33151 / 49000) loss: 1.29218621609\n",
      "(Iteration 33161 / 49000) loss: 1.28505346117\n",
      "(Iteration 33171 / 49000) loss: 1.33064671524\n",
      "(Iteration 33181 / 49000) loss: 1.30116737394\n",
      "(Iteration 33191 / 49000) loss: 1.34908655234\n",
      "(Iteration 33201 / 49000) loss: 1.27098332862\n",
      "(Iteration 33211 / 49000) loss: 1.35275628474\n",
      "(Iteration 33221 / 49000) loss: 1.34739980226\n",
      "(Epoch 339 / 500) Training Accuracy: 0.538591836735, Validation Accuracy: 0.461\n",
      "(Iteration 33231 / 49000) loss: 1.28942179838\n",
      "(Iteration 33241 / 49000) loss: 1.3133112218\n",
      "(Iteration 33251 / 49000) loss: 1.34542615549\n",
      "(Iteration 33261 / 49000) loss: 1.39167373523\n",
      "(Iteration 33271 / 49000) loss: 1.37276490176\n",
      "(Iteration 33281 / 49000) loss: 1.37281063622\n",
      "(Iteration 33291 / 49000) loss: 1.32404590757\n",
      "(Iteration 33301 / 49000) loss: 1.3308229782\n",
      "(Iteration 33311 / 49000) loss: 1.39898823445\n",
      "(Epoch 340 / 500) Training Accuracy: 0.537510204082, Validation Accuracy: 0.468\n",
      "(Iteration 33321 / 49000) loss: 1.29003441347\n",
      "(Iteration 33331 / 49000) loss: 1.32317706217\n",
      "(Iteration 33341 / 49000) loss: 1.38586603666\n",
      "(Iteration 33351 / 49000) loss: 1.38541735813\n",
      "(Iteration 33361 / 49000) loss: 1.34495649203\n",
      "(Iteration 33371 / 49000) loss: 1.41059871903\n",
      "(Iteration 33381 / 49000) loss: 1.32496056218\n",
      "(Iteration 33391 / 49000) loss: 1.32786379242\n",
      "(Iteration 33401 / 49000) loss: 1.31898881461\n",
      "(Iteration 33411 / 49000) loss: 1.44688694815\n",
      "(Epoch 341 / 500) Training Accuracy: 0.529142857143, Validation Accuracy: 0.463\n",
      "(Iteration 33421 / 49000) loss: 1.38928503667\n",
      "(Iteration 33431 / 49000) loss: 1.26724275812\n",
      "(Iteration 33441 / 49000) loss: 1.2969832018\n",
      "(Iteration 33451 / 49000) loss: 1.2509967893\n",
      "(Iteration 33461 / 49000) loss: 1.33259910602\n",
      "(Iteration 33471 / 49000) loss: 1.44137198546\n",
      "(Iteration 33481 / 49000) loss: 1.3129451017\n",
      "(Iteration 33491 / 49000) loss: 1.31523794656\n",
      "(Iteration 33501 / 49000) loss: 1.31874643359\n",
      "(Iteration 33511 / 49000) loss: 1.34506779723\n",
      "(Epoch 342 / 500) Training Accuracy: 0.535795918367, Validation Accuracy: 0.464\n",
      "(Iteration 33521 / 49000) loss: 1.50317358009\n",
      "(Iteration 33531 / 49000) loss: 1.34499895229\n",
      "(Iteration 33541 / 49000) loss: 1.35005150483\n",
      "(Iteration 33551 / 49000) loss: 1.3128034357\n",
      "(Iteration 33561 / 49000) loss: 1.39314810051\n",
      "(Iteration 33571 / 49000) loss: 1.33473338677\n",
      "(Iteration 33581 / 49000) loss: 1.30158976665\n",
      "(Iteration 33591 / 49000) loss: 1.29982814976\n",
      "(Iteration 33601 / 49000) loss: 1.24198039565\n",
      "(Iteration 33611 / 49000) loss: 1.35572343726\n",
      "(Epoch 343 / 500) Training Accuracy: 0.536551020408, Validation Accuracy: 0.471\n",
      "(Iteration 33621 / 49000) loss: 1.32317870256\n",
      "(Iteration 33631 / 49000) loss: 1.32462740992\n",
      "(Iteration 33641 / 49000) loss: 1.36743013892\n",
      "(Iteration 33651 / 49000) loss: 1.38412170622\n",
      "(Iteration 33661 / 49000) loss: 1.39762602322\n",
      "(Iteration 33671 / 49000) loss: 1.38110888489\n",
      "(Iteration 33681 / 49000) loss: 1.26472170372\n",
      "(Iteration 33691 / 49000) loss: 1.28546142762\n",
      "(Iteration 33701 / 49000) loss: 1.32427513352\n",
      "(Iteration 33711 / 49000) loss: 1.40137294661\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Epoch 344 / 500) Training Accuracy: 0.531857142857, Validation Accuracy: 0.478\n",
      "(Iteration 33721 / 49000) loss: 1.26780202936\n",
      "(Iteration 33731 / 49000) loss: 1.34863011622\n",
      "(Iteration 33741 / 49000) loss: 1.30073186552\n",
      "(Iteration 33751 / 49000) loss: 1.30068079392\n",
      "(Iteration 33761 / 49000) loss: 1.37154332454\n",
      "(Iteration 33771 / 49000) loss: 1.22600884804\n",
      "(Iteration 33781 / 49000) loss: 1.33660172817\n",
      "(Iteration 33791 / 49000) loss: 1.29682775331\n",
      "(Iteration 33801 / 49000) loss: 1.20058236911\n",
      "(Epoch 345 / 500) Training Accuracy: 0.537183673469, Validation Accuracy: 0.476\n",
      "(Iteration 33811 / 49000) loss: 1.39012280226\n",
      "(Iteration 33821 / 49000) loss: 1.25429362276\n",
      "(Iteration 33831 / 49000) loss: 1.40658109981\n",
      "(Iteration 33841 / 49000) loss: 1.22812173984\n",
      "(Iteration 33851 / 49000) loss: 1.26656959963\n",
      "(Iteration 33861 / 49000) loss: 1.33442077836\n",
      "(Iteration 33871 / 49000) loss: 1.30801833428\n",
      "(Iteration 33881 / 49000) loss: 1.35201187275\n",
      "(Iteration 33891 / 49000) loss: 1.38478166528\n",
      "(Iteration 33901 / 49000) loss: 1.25457137857\n",
      "(Epoch 346 / 500) Training Accuracy: 0.531102040816, Validation Accuracy: 0.47\n",
      "(Iteration 33911 / 49000) loss: 1.31610009205\n",
      "(Iteration 33921 / 49000) loss: 1.33970296669\n",
      "(Iteration 33931 / 49000) loss: 1.27538551071\n",
      "(Iteration 33941 / 49000) loss: 1.36292668253\n",
      "(Iteration 33951 / 49000) loss: 1.27743544344\n",
      "(Iteration 33961 / 49000) loss: 1.34760486912\n",
      "(Iteration 33971 / 49000) loss: 1.32704985423\n",
      "(Iteration 33981 / 49000) loss: 1.29565263419\n",
      "(Iteration 33991 / 49000) loss: 1.38908798138\n",
      "(Iteration 34001 / 49000) loss: 1.35138735578\n",
      "(Epoch 347 / 500) Training Accuracy: 0.534530612245, Validation Accuracy: 0.457\n",
      "(Iteration 34011 / 49000) loss: 1.29875732593\n",
      "(Iteration 34021 / 49000) loss: 1.29300309105\n",
      "(Iteration 34031 / 49000) loss: 1.29794166137\n",
      "(Iteration 34041 / 49000) loss: 1.314566604\n",
      "(Iteration 34051 / 49000) loss: 1.3754724702\n",
      "(Iteration 34061 / 49000) loss: 1.3875906451\n",
      "(Iteration 34071 / 49000) loss: 1.45343579725\n",
      "(Iteration 34081 / 49000) loss: 1.28249084218\n",
      "(Iteration 34091 / 49000) loss: 1.41531777372\n",
      "(Iteration 34101 / 49000) loss: 1.35291941513\n",
      "(Epoch 348 / 500) Training Accuracy: 0.539755102041, Validation Accuracy: 0.474\n",
      "(Iteration 34111 / 49000) loss: 1.32348652846\n",
      "(Iteration 34121 / 49000) loss: 1.35060200287\n",
      "(Iteration 34131 / 49000) loss: 1.29176994104\n",
      "(Iteration 34141 / 49000) loss: 1.34569608652\n",
      "(Iteration 34151 / 49000) loss: 1.37634972198\n",
      "(Iteration 34161 / 49000) loss: 1.31839552521\n",
      "(Iteration 34171 / 49000) loss: 1.29528885807\n",
      "(Iteration 34181 / 49000) loss: 1.27193763241\n",
      "(Iteration 34191 / 49000) loss: 1.36166172783\n",
      "(Iteration 34201 / 49000) loss: 1.41463908991\n",
      "(Epoch 349 / 500) Training Accuracy: 0.538183673469, Validation Accuracy: 0.466\n",
      "(Iteration 34211 / 49000) loss: 1.28665209353\n",
      "(Iteration 34221 / 49000) loss: 1.38916816695\n",
      "(Iteration 34231 / 49000) loss: 1.31178901761\n",
      "(Iteration 34241 / 49000) loss: 1.28384062523\n",
      "(Iteration 34251 / 49000) loss: 1.3322922151\n",
      "(Iteration 34261 / 49000) loss: 1.4005679673\n",
      "(Iteration 34271 / 49000) loss: 1.36665982088\n",
      "(Iteration 34281 / 49000) loss: 1.347773718\n",
      "(Iteration 34291 / 49000) loss: 1.32148139264\n",
      "(Epoch 350 / 500) Training Accuracy: 0.537551020408, Validation Accuracy: 0.471\n",
      "(Iteration 34301 / 49000) loss: 1.25509547464\n",
      "(Iteration 34311 / 49000) loss: 1.34215334039\n",
      "(Iteration 34321 / 49000) loss: 1.24408435288\n",
      "(Iteration 34331 / 49000) loss: 1.31373089248\n",
      "(Iteration 34341 / 49000) loss: 1.35176756025\n",
      "(Iteration 34351 / 49000) loss: 1.31382460984\n",
      "(Iteration 34361 / 49000) loss: 1.41205710964\n",
      "(Iteration 34371 / 49000) loss: 1.33729673311\n",
      "(Iteration 34381 / 49000) loss: 1.31138972694\n",
      "(Iteration 34391 / 49000) loss: 1.34172408156\n",
      "(Epoch 351 / 500) Training Accuracy: 0.539163265306, Validation Accuracy: 0.473\n",
      "(Iteration 34401 / 49000) loss: 1.33512700355\n",
      "(Iteration 34411 / 49000) loss: 1.28201235939\n",
      "(Iteration 34421 / 49000) loss: 1.29832515263\n",
      "(Iteration 34431 / 49000) loss: 1.36696309766\n",
      "(Iteration 34441 / 49000) loss: 1.35698231701\n",
      "(Iteration 34451 / 49000) loss: 1.33652419742\n",
      "(Iteration 34461 / 49000) loss: 1.31221142511\n",
      "(Iteration 34471 / 49000) loss: 1.29122682396\n",
      "(Iteration 34481 / 49000) loss: 1.32713559032\n",
      "(Iteration 34491 / 49000) loss: 1.29721593601\n",
      "(Epoch 352 / 500) Training Accuracy: 0.538530612245, Validation Accuracy: 0.48\n",
      "(Iteration 34501 / 49000) loss: 1.31771534648\n",
      "(Iteration 34511 / 49000) loss: 1.24839490763\n",
      "(Iteration 34521 / 49000) loss: 1.33201940553\n",
      "(Iteration 34531 / 49000) loss: 1.2724630846\n",
      "(Iteration 34541 / 49000) loss: 1.35102962704\n",
      "(Iteration 34551 / 49000) loss: 1.32650585656\n",
      "(Iteration 34561 / 49000) loss: 1.34265363546\n",
      "(Iteration 34571 / 49000) loss: 1.3571541306\n",
      "(Iteration 34581 / 49000) loss: 1.27031170855\n",
      "(Iteration 34591 / 49000) loss: 1.38906742293\n",
      "(Epoch 353 / 500) Training Accuracy: 0.535673469388, Validation Accuracy: 0.475\n",
      "(Iteration 34601 / 49000) loss: 1.24308620778\n",
      "(Iteration 34611 / 49000) loss: 1.37140391305\n",
      "(Iteration 34621 / 49000) loss: 1.39417541515\n",
      "(Iteration 34631 / 49000) loss: 1.26465667736\n",
      "(Iteration 34641 / 49000) loss: 1.27926145332\n",
      "(Iteration 34651 / 49000) loss: 1.2746880158\n",
      "(Iteration 34661 / 49000) loss: 1.35578415571\n",
      "(Iteration 34671 / 49000) loss: 1.34340752681\n",
      "(Iteration 34681 / 49000) loss: 1.34427997404\n",
      "(Iteration 34691 / 49000) loss: 1.29089013633\n",
      "(Epoch 354 / 500) Training Accuracy: 0.538979591837, Validation Accuracy: 0.464\n",
      "(Iteration 34701 / 49000) loss: 1.42462867899\n",
      "(Iteration 34711 / 49000) loss: 1.34804191734\n",
      "(Iteration 34721 / 49000) loss: 1.3382009663\n",
      "(Iteration 34731 / 49000) loss: 1.38339293536\n",
      "(Iteration 34741 / 49000) loss: 1.22918127727\n",
      "(Iteration 34751 / 49000) loss: 1.26832627135\n",
      "(Iteration 34761 / 49000) loss: 1.25448405503\n",
      "(Iteration 34771 / 49000) loss: 1.32527318999\n",
      "(Iteration 34781 / 49000) loss: 1.3113633179\n",
      "(Epoch 355 / 500) Training Accuracy: 0.538489795918, Validation Accuracy: 0.47\n",
      "(Iteration 34791 / 49000) loss: 1.20395181102\n",
      "(Iteration 34801 / 49000) loss: 1.33270971806\n",
      "(Iteration 34811 / 49000) loss: 1.34932240759\n",
      "(Iteration 34821 / 49000) loss: 1.33071903789\n",
      "(Iteration 34831 / 49000) loss: 1.38526204143\n",
      "(Iteration 34841 / 49000) loss: 1.2567124612\n",
      "(Iteration 34851 / 49000) loss: 1.26920379077\n",
      "(Iteration 34861 / 49000) loss: 1.30708880146\n",
      "(Iteration 34871 / 49000) loss: 1.41099967201\n",
      "(Iteration 34881 / 49000) loss: 1.41618192907\n",
      "(Epoch 356 / 500) Training Accuracy: 0.537163265306, Validation Accuracy: 0.468\n",
      "(Iteration 34891 / 49000) loss: 1.40412541952\n",
      "(Iteration 34901 / 49000) loss: 1.33537725079\n",
      "(Iteration 34911 / 49000) loss: 1.36964397014\n",
      "(Iteration 34921 / 49000) loss: 1.35717253462\n",
      "(Iteration 34931 / 49000) loss: 1.29643602508\n",
      "(Iteration 34941 / 49000) loss: 1.38336578288\n",
      "(Iteration 34951 / 49000) loss: 1.33515220788\n",
      "(Iteration 34961 / 49000) loss: 1.35030991188\n",
      "(Iteration 34971 / 49000) loss: 1.35464224027\n",
      "(Iteration 34981 / 49000) loss: 1.23845823767\n",
      "(Epoch 357 / 500) Training Accuracy: 0.54012244898, Validation Accuracy: 0.467\n",
      "(Iteration 34991 / 49000) loss: 1.24813028048\n",
      "(Iteration 35001 / 49000) loss: 1.32497633078\n",
      "(Iteration 35011 / 49000) loss: 1.27139758405\n",
      "(Iteration 35021 / 49000) loss: 1.27789890202\n",
      "(Iteration 35031 / 49000) loss: 1.32476945088\n",
      "(Iteration 35041 / 49000) loss: 1.36928979834\n",
      "(Iteration 35051 / 49000) loss: 1.27854916929\n",
      "(Iteration 35061 / 49000) loss: 1.2905535205\n",
      "(Iteration 35071 / 49000) loss: 1.33787131655\n",
      "(Iteration 35081 / 49000) loss: 1.30505445088\n",
      "(Epoch 358 / 500) Training Accuracy: 0.538755102041, Validation Accuracy: 0.47\n",
      "(Iteration 35091 / 49000) loss: 1.22768758507\n",
      "(Iteration 35101 / 49000) loss: 1.29163843024\n",
      "(Iteration 35111 / 49000) loss: 1.43339349213\n",
      "(Iteration 35121 / 49000) loss: 1.29211510213\n",
      "(Iteration 35131 / 49000) loss: 1.30505575426\n",
      "(Iteration 35141 / 49000) loss: 1.3386898405\n",
      "(Iteration 35151 / 49000) loss: 1.30829894608\n",
      "(Iteration 35161 / 49000) loss: 1.37393420696\n",
      "(Iteration 35171 / 49000) loss: 1.40103616999\n",
      "(Iteration 35181 / 49000) loss: 1.35881802949\n",
      "(Epoch 359 / 500) Training Accuracy: 0.540755102041, Validation Accuracy: 0.464\n",
      "(Iteration 35191 / 49000) loss: 1.26237384546\n",
      "(Iteration 35201 / 49000) loss: 1.25851914656\n",
      "(Iteration 35211 / 49000) loss: 1.34206897696\n",
      "(Iteration 35221 / 49000) loss: 1.42486056674\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 35231 / 49000) loss: 1.3568818154\n",
      "(Iteration 35241 / 49000) loss: 1.28769744825\n",
      "(Iteration 35251 / 49000) loss: 1.33415450941\n",
      "(Iteration 35261 / 49000) loss: 1.40275356382\n",
      "(Iteration 35271 / 49000) loss: 1.32796952762\n",
      "(Epoch 360 / 500) Training Accuracy: 0.540183673469, Validation Accuracy: 0.468\n",
      "(Iteration 35281 / 49000) loss: 1.28003572232\n",
      "(Iteration 35291 / 49000) loss: 1.26184197263\n",
      "(Iteration 35301 / 49000) loss: 1.31929892638\n",
      "(Iteration 35311 / 49000) loss: 1.3644122301\n",
      "(Iteration 35321 / 49000) loss: 1.32257321666\n",
      "(Iteration 35331 / 49000) loss: 1.33021307241\n",
      "(Iteration 35341 / 49000) loss: 1.35382105727\n",
      "(Iteration 35351 / 49000) loss: 1.28749164744\n",
      "(Iteration 35361 / 49000) loss: 1.37318574272\n",
      "(Iteration 35371 / 49000) loss: 1.34519616809\n",
      "(Epoch 361 / 500) Training Accuracy: 0.541755102041, Validation Accuracy: 0.467\n",
      "(Iteration 35381 / 49000) loss: 1.28608877131\n",
      "(Iteration 35391 / 49000) loss: 1.33616914646\n",
      "(Iteration 35401 / 49000) loss: 1.24215975353\n",
      "(Iteration 35411 / 49000) loss: 1.34519608099\n",
      "(Iteration 35421 / 49000) loss: 1.31162689053\n",
      "(Iteration 35431 / 49000) loss: 1.38147261783\n",
      "(Iteration 35441 / 49000) loss: 1.38543580888\n",
      "(Iteration 35451 / 49000) loss: 1.4404967922\n",
      "(Iteration 35461 / 49000) loss: 1.23692569643\n",
      "(Iteration 35471 / 49000) loss: 1.29541130208\n",
      "(Epoch 362 / 500) Training Accuracy: 0.542265306122, Validation Accuracy: 0.477\n",
      "(Iteration 35481 / 49000) loss: 1.34142672817\n",
      "(Iteration 35491 / 49000) loss: 1.31739535617\n",
      "(Iteration 35501 / 49000) loss: 1.32891727427\n",
      "(Iteration 35511 / 49000) loss: 1.31891549779\n",
      "(Iteration 35521 / 49000) loss: 1.3054280419\n",
      "(Iteration 35531 / 49000) loss: 1.23986967481\n",
      "(Iteration 35541 / 49000) loss: 1.43300437518\n",
      "(Iteration 35551 / 49000) loss: 1.27050948867\n",
      "(Iteration 35561 / 49000) loss: 1.21651476648\n",
      "(Iteration 35571 / 49000) loss: 1.32949610106\n",
      "(Epoch 363 / 500) Training Accuracy: 0.530693877551, Validation Accuracy: 0.47\n",
      "(Iteration 35581 / 49000) loss: 1.3747231647\n",
      "(Iteration 35591 / 49000) loss: 1.35367406829\n",
      "(Iteration 35601 / 49000) loss: 1.30140385921\n",
      "(Iteration 35611 / 49000) loss: 1.35570774789\n",
      "(Iteration 35621 / 49000) loss: 1.28502839313\n",
      "(Iteration 35631 / 49000) loss: 1.3286655185\n",
      "(Iteration 35641 / 49000) loss: 1.32012507893\n",
      "(Iteration 35651 / 49000) loss: 1.27570169288\n",
      "(Iteration 35661 / 49000) loss: 1.40362958898\n",
      "(Iteration 35671 / 49000) loss: 1.33260603907\n",
      "(Epoch 364 / 500) Training Accuracy: 0.543836734694, Validation Accuracy: 0.472\n",
      "(Iteration 35681 / 49000) loss: 1.30621277025\n",
      "(Iteration 35691 / 49000) loss: 1.28155028586\n",
      "(Iteration 35701 / 49000) loss: 1.35246081993\n",
      "(Iteration 35711 / 49000) loss: 1.32890684849\n",
      "(Iteration 35721 / 49000) loss: 1.29529442671\n",
      "(Iteration 35731 / 49000) loss: 1.37163922283\n",
      "(Iteration 35741 / 49000) loss: 1.24130658125\n",
      "(Iteration 35751 / 49000) loss: 1.3092843286\n",
      "(Iteration 35761 / 49000) loss: 1.28095843873\n",
      "(Epoch 365 / 500) Training Accuracy: 0.537959183673, Validation Accuracy: 0.471\n",
      "(Iteration 35771 / 49000) loss: 1.27587030885\n",
      "(Iteration 35781 / 49000) loss: 1.29133526041\n",
      "(Iteration 35791 / 49000) loss: 1.40806855784\n",
      "(Iteration 35801 / 49000) loss: 1.31323918618\n",
      "(Iteration 35811 / 49000) loss: 1.37597659488\n",
      "(Iteration 35821 / 49000) loss: 1.31409177194\n",
      "(Iteration 35831 / 49000) loss: 1.29565827339\n",
      "(Iteration 35841 / 49000) loss: 1.3555748318\n",
      "(Iteration 35851 / 49000) loss: 1.36521278145\n",
      "(Iteration 35861 / 49000) loss: 1.30832616514\n",
      "(Epoch 366 / 500) Training Accuracy: 0.543326530612, Validation Accuracy: 0.461\n",
      "(Iteration 35871 / 49000) loss: 1.2370812452\n",
      "(Iteration 35881 / 49000) loss: 1.26768297414\n",
      "(Iteration 35891 / 49000) loss: 1.2902903611\n",
      "(Iteration 35901 / 49000) loss: 1.30287186788\n",
      "(Iteration 35911 / 49000) loss: 1.4112602248\n",
      "(Iteration 35921 / 49000) loss: 1.32973544875\n",
      "(Iteration 35931 / 49000) loss: 1.38761091061\n",
      "(Iteration 35941 / 49000) loss: 1.31708683936\n",
      "(Iteration 35951 / 49000) loss: 1.31548378899\n",
      "(Iteration 35961 / 49000) loss: 1.31242249862\n",
      "(Epoch 367 / 500) Training Accuracy: 0.544571428571, Validation Accuracy: 0.475\n",
      "(Iteration 35971 / 49000) loss: 1.31824098336\n",
      "(Iteration 35981 / 49000) loss: 1.31981846637\n",
      "(Iteration 35991 / 49000) loss: 1.34880210387\n",
      "(Iteration 36001 / 49000) loss: 1.2955245795\n",
      "(Iteration 36011 / 49000) loss: 1.26663332646\n",
      "(Iteration 36021 / 49000) loss: 1.37064227335\n",
      "(Iteration 36031 / 49000) loss: 1.35321794863\n",
      "(Iteration 36041 / 49000) loss: 1.35521286061\n",
      "(Iteration 36051 / 49000) loss: 1.34636299519\n",
      "(Iteration 36061 / 49000) loss: 1.3611103135\n",
      "(Epoch 368 / 500) Training Accuracy: 0.539795918367, Validation Accuracy: 0.475\n",
      "(Iteration 36071 / 49000) loss: 1.2802774523\n",
      "(Iteration 36081 / 49000) loss: 1.30810001543\n",
      "(Iteration 36091 / 49000) loss: 1.24365167622\n",
      "(Iteration 36101 / 49000) loss: 1.28373103031\n",
      "(Iteration 36111 / 49000) loss: 1.33093047521\n",
      "(Iteration 36121 / 49000) loss: 1.35948769321\n",
      "(Iteration 36131 / 49000) loss: 1.37083488795\n",
      "(Iteration 36141 / 49000) loss: 1.31604504016\n",
      "(Iteration 36151 / 49000) loss: 1.35797140125\n",
      "(Iteration 36161 / 49000) loss: 1.38232070477\n",
      "(Epoch 369 / 500) Training Accuracy: 0.545551020408, Validation Accuracy: 0.472\n",
      "(Iteration 36171 / 49000) loss: 1.2751879581\n",
      "(Iteration 36181 / 49000) loss: 1.25608308233\n",
      "(Iteration 36191 / 49000) loss: 1.2914384443\n",
      "(Iteration 36201 / 49000) loss: 1.30247741468\n",
      "(Iteration 36211 / 49000) loss: 1.30744210783\n",
      "(Iteration 36221 / 49000) loss: 1.29896733605\n",
      "(Iteration 36231 / 49000) loss: 1.29372297945\n",
      "(Iteration 36241 / 49000) loss: 1.28803947309\n",
      "(Iteration 36251 / 49000) loss: 1.26058905561\n",
      "(Epoch 370 / 500) Training Accuracy: 0.542142857143, Validation Accuracy: 0.468\n",
      "(Iteration 36261 / 49000) loss: 1.2972288982\n",
      "(Iteration 36271 / 49000) loss: 1.35365677847\n",
      "(Iteration 36281 / 49000) loss: 1.26993359494\n",
      "(Iteration 36291 / 49000) loss: 1.33831995734\n",
      "(Iteration 36301 / 49000) loss: 1.34111976907\n",
      "(Iteration 36311 / 49000) loss: 1.30925470471\n",
      "(Iteration 36321 / 49000) loss: 1.3889245709\n",
      "(Iteration 36331 / 49000) loss: 1.32844148895\n",
      "(Iteration 36341 / 49000) loss: 1.22093610258\n",
      "(Iteration 36351 / 49000) loss: 1.33697327304\n",
      "(Epoch 371 / 500) Training Accuracy: 0.54287755102, Validation Accuracy: 0.465\n",
      "(Iteration 36361 / 49000) loss: 1.28109302035\n",
      "(Iteration 36371 / 49000) loss: 1.29175176712\n",
      "(Iteration 36381 / 49000) loss: 1.26934819286\n",
      "(Iteration 36391 / 49000) loss: 1.2746814286\n",
      "(Iteration 36401 / 49000) loss: 1.317302381\n",
      "(Iteration 36411 / 49000) loss: 1.30728476147\n",
      "(Iteration 36421 / 49000) loss: 1.29609162004\n",
      "(Iteration 36431 / 49000) loss: 1.38047480686\n",
      "(Iteration 36441 / 49000) loss: 1.34638904533\n",
      "(Iteration 36451 / 49000) loss: 1.23006524555\n",
      "(Epoch 372 / 500) Training Accuracy: 0.545632653061, Validation Accuracy: 0.473\n",
      "(Iteration 36461 / 49000) loss: 1.284850551\n",
      "(Iteration 36471 / 49000) loss: 1.27366825114\n",
      "(Iteration 36481 / 49000) loss: 1.31246711459\n",
      "(Iteration 36491 / 49000) loss: 1.26763021379\n",
      "(Iteration 36501 / 49000) loss: 1.3303715527\n",
      "(Iteration 36511 / 49000) loss: 1.32930413409\n",
      "(Iteration 36521 / 49000) loss: 1.37144670197\n",
      "(Iteration 36531 / 49000) loss: 1.36979857532\n",
      "(Iteration 36541 / 49000) loss: 1.36438819673\n",
      "(Iteration 36551 / 49000) loss: 1.3559521895\n",
      "(Epoch 373 / 500) Training Accuracy: 0.536448979592, Validation Accuracy: 0.457\n",
      "(Iteration 36561 / 49000) loss: 1.35333410608\n",
      "(Iteration 36571 / 49000) loss: 1.27535468169\n",
      "(Iteration 36581 / 49000) loss: 1.23915077294\n",
      "(Iteration 36591 / 49000) loss: 1.25855735709\n",
      "(Iteration 36601 / 49000) loss: 1.38224676884\n",
      "(Iteration 36611 / 49000) loss: 1.28360363299\n",
      "(Iteration 36621 / 49000) loss: 1.32393975525\n",
      "(Iteration 36631 / 49000) loss: 1.36520182516\n",
      "(Iteration 36641 / 49000) loss: 1.23818264822\n",
      "(Iteration 36651 / 49000) loss: 1.31870417904\n",
      "(Epoch 374 / 500) Training Accuracy: 0.540755102041, Validation Accuracy: 0.471\n",
      "(Iteration 36661 / 49000) loss: 1.35657298573\n",
      "(Iteration 36671 / 49000) loss: 1.2993541202\n",
      "(Iteration 36681 / 49000) loss: 1.28209561256\n",
      "(Iteration 36691 / 49000) loss: 1.30040068212\n",
      "(Iteration 36701 / 49000) loss: 1.35689171816\n",
      "(Iteration 36711 / 49000) loss: 1.26028634193\n",
      "(Iteration 36721 / 49000) loss: 1.29702184138\n",
      "(Iteration 36731 / 49000) loss: 1.26782961091\n",
      "(Iteration 36741 / 49000) loss: 1.30457979296\n",
      "(Epoch 375 / 500) Training Accuracy: 0.543795918367, Validation Accuracy: 0.471\n",
      "(Iteration 36751 / 49000) loss: 1.30304985448\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 36761 / 49000) loss: 1.34082917145\n",
      "(Iteration 36771 / 49000) loss: 1.28834034776\n",
      "(Iteration 36781 / 49000) loss: 1.25939882757\n",
      "(Iteration 36791 / 49000) loss: 1.32744098466\n",
      "(Iteration 36801 / 49000) loss: 1.25138082051\n",
      "(Iteration 36811 / 49000) loss: 1.28301414926\n",
      "(Iteration 36821 / 49000) loss: 1.35094251407\n",
      "(Iteration 36831 / 49000) loss: 1.36886114308\n",
      "(Iteration 36841 / 49000) loss: 1.40688914382\n",
      "(Epoch 376 / 500) Training Accuracy: 0.541693877551, Validation Accuracy: 0.467\n",
      "(Iteration 36851 / 49000) loss: 1.20764113167\n",
      "(Iteration 36861 / 49000) loss: 1.33929631231\n",
      "(Iteration 36871 / 49000) loss: 1.34046860508\n",
      "(Iteration 36881 / 49000) loss: 1.37101339384\n",
      "(Iteration 36891 / 49000) loss: 1.33846963158\n",
      "(Iteration 36901 / 49000) loss: 1.32190639899\n",
      "(Iteration 36911 / 49000) loss: 1.25592506405\n",
      "(Iteration 36921 / 49000) loss: 1.34920085204\n",
      "(Iteration 36931 / 49000) loss: 1.2279999363\n",
      "(Iteration 36941 / 49000) loss: 1.22840407357\n",
      "(Epoch 377 / 500) Training Accuracy: 0.543142857143, Validation Accuracy: 0.469\n",
      "(Iteration 36951 / 49000) loss: 1.34588620049\n",
      "(Iteration 36961 / 49000) loss: 1.2958674749\n",
      "(Iteration 36971 / 49000) loss: 1.26872756574\n",
      "(Iteration 36981 / 49000) loss: 1.36824468389\n",
      "(Iteration 36991 / 49000) loss: 1.32088928442\n",
      "(Iteration 37001 / 49000) loss: 1.34351111577\n",
      "(Iteration 37011 / 49000) loss: 1.30578408542\n",
      "(Iteration 37021 / 49000) loss: 1.24980770772\n",
      "(Iteration 37031 / 49000) loss: 1.33115605135\n",
      "(Iteration 37041 / 49000) loss: 1.31077437503\n",
      "(Epoch 378 / 500) Training Accuracy: 0.543836734694, Validation Accuracy: 0.475\n",
      "(Iteration 37051 / 49000) loss: 1.26678141243\n",
      "(Iteration 37061 / 49000) loss: 1.31263722789\n",
      "(Iteration 37071 / 49000) loss: 1.29287801503\n",
      "(Iteration 37081 / 49000) loss: 1.2379938305\n",
      "(Iteration 37091 / 49000) loss: 1.27728448121\n",
      "(Iteration 37101 / 49000) loss: 1.30326811133\n",
      "(Iteration 37111 / 49000) loss: 1.21852750603\n",
      "(Iteration 37121 / 49000) loss: 1.34098457789\n",
      "(Iteration 37131 / 49000) loss: 1.32669029953\n",
      "(Iteration 37141 / 49000) loss: 1.41360608312\n",
      "(Epoch 379 / 500) Training Accuracy: 0.545244897959, Validation Accuracy: 0.465\n",
      "(Iteration 37151 / 49000) loss: 1.28648322802\n",
      "(Iteration 37161 / 49000) loss: 1.40020292213\n",
      "(Iteration 37171 / 49000) loss: 1.22261699971\n",
      "(Iteration 37181 / 49000) loss: 1.31584308207\n",
      "(Iteration 37191 / 49000) loss: 1.32117654974\n",
      "(Iteration 37201 / 49000) loss: 1.32245347853\n",
      "(Iteration 37211 / 49000) loss: 1.418559023\n",
      "(Iteration 37221 / 49000) loss: 1.47761949779\n",
      "(Iteration 37231 / 49000) loss: 1.34727770622\n",
      "(Epoch 380 / 500) Training Accuracy: 0.538306122449, Validation Accuracy: 0.474\n",
      "(Iteration 37241 / 49000) loss: 1.37340803392\n",
      "(Iteration 37251 / 49000) loss: 1.26480869038\n",
      "(Iteration 37261 / 49000) loss: 1.26006393144\n",
      "(Iteration 37271 / 49000) loss: 1.34779762832\n",
      "(Iteration 37281 / 49000) loss: 1.24931132593\n",
      "(Iteration 37291 / 49000) loss: 1.34564844284\n",
      "(Iteration 37301 / 49000) loss: 1.29950501807\n",
      "(Iteration 37311 / 49000) loss: 1.37667702656\n",
      "(Iteration 37321 / 49000) loss: 1.29355508553\n",
      "(Iteration 37331 / 49000) loss: 1.31685392841\n",
      "(Epoch 381 / 500) Training Accuracy: 0.543551020408, Validation Accuracy: 0.472\n",
      "(Iteration 37341 / 49000) loss: 1.19769234388\n",
      "(Iteration 37351 / 49000) loss: 1.2250197111\n",
      "(Iteration 37361 / 49000) loss: 1.35687508622\n",
      "(Iteration 37371 / 49000) loss: 1.29882260664\n",
      "(Iteration 37381 / 49000) loss: 1.31534348916\n",
      "(Iteration 37391 / 49000) loss: 1.23475083752\n",
      "(Iteration 37401 / 49000) loss: 1.36910276447\n",
      "(Iteration 37411 / 49000) loss: 1.26484784605\n",
      "(Iteration 37421 / 49000) loss: 1.20802556182\n",
      "(Iteration 37431 / 49000) loss: 1.3563134132\n",
      "(Epoch 382 / 500) Training Accuracy: 0.547775510204, Validation Accuracy: 0.472\n",
      "(Iteration 37441 / 49000) loss: 1.24980792733\n",
      "(Iteration 37451 / 49000) loss: 1.34433951923\n",
      "(Iteration 37461 / 49000) loss: 1.27372349114\n",
      "(Iteration 37471 / 49000) loss: 1.27932044493\n",
      "(Iteration 37481 / 49000) loss: 1.32294114087\n",
      "(Iteration 37491 / 49000) loss: 1.29182165243\n",
      "(Iteration 37501 / 49000) loss: 1.32895639638\n",
      "(Iteration 37511 / 49000) loss: 1.29761982172\n",
      "(Iteration 37521 / 49000) loss: 1.30466449918\n",
      "(Iteration 37531 / 49000) loss: 1.36365686923\n",
      "(Epoch 383 / 500) Training Accuracy: 0.544469387755, Validation Accuracy: 0.47\n",
      "(Iteration 37541 / 49000) loss: 1.25551750713\n",
      "(Iteration 37551 / 49000) loss: 1.29690826821\n",
      "(Iteration 37561 / 49000) loss: 1.33893436001\n",
      "(Iteration 37571 / 49000) loss: 1.33292320982\n",
      "(Iteration 37581 / 49000) loss: 1.30961249749\n",
      "(Iteration 37591 / 49000) loss: 1.28847837641\n",
      "(Iteration 37601 / 49000) loss: 1.23220081449\n",
      "(Iteration 37611 / 49000) loss: 1.32708427645\n",
      "(Iteration 37621 / 49000) loss: 1.29224677221\n",
      "(Iteration 37631 / 49000) loss: 1.39817070143\n",
      "(Epoch 384 / 500) Training Accuracy: 0.543693877551, Validation Accuracy: 0.484\n",
      "(Iteration 37641 / 49000) loss: 1.28829675774\n",
      "(Iteration 37651 / 49000) loss: 1.37572351203\n",
      "(Iteration 37661 / 49000) loss: 1.33004745765\n",
      "(Iteration 37671 / 49000) loss: 1.39470671908\n",
      "(Iteration 37681 / 49000) loss: 1.37181789041\n",
      "(Iteration 37691 / 49000) loss: 1.29385889854\n",
      "(Iteration 37701 / 49000) loss: 1.24275975911\n",
      "(Iteration 37711 / 49000) loss: 1.30671212701\n",
      "(Iteration 37721 / 49000) loss: 1.23062035964\n",
      "(Epoch 385 / 500) Training Accuracy: 0.545734693878, Validation Accuracy: 0.472\n",
      "(Iteration 37731 / 49000) loss: 1.30525412235\n",
      "(Iteration 37741 / 49000) loss: 1.31404202753\n",
      "(Iteration 37751 / 49000) loss: 1.33084646287\n",
      "(Iteration 37761 / 49000) loss: 1.27589824886\n",
      "(Iteration 37771 / 49000) loss: 1.3349759861\n",
      "(Iteration 37781 / 49000) loss: 1.27665061941\n",
      "(Iteration 37791 / 49000) loss: 1.27929264453\n",
      "(Iteration 37801 / 49000) loss: 1.34145794428\n",
      "(Iteration 37811 / 49000) loss: 1.26375408545\n",
      "(Iteration 37821 / 49000) loss: 1.32458001937\n",
      "(Epoch 386 / 500) Training Accuracy: 0.547653061224, Validation Accuracy: 0.476\n",
      "(Iteration 37831 / 49000) loss: 1.30862576193\n",
      "(Iteration 37841 / 49000) loss: 1.32613898658\n",
      "(Iteration 37851 / 49000) loss: 1.30421292175\n",
      "(Iteration 37861 / 49000) loss: 1.30888648822\n",
      "(Iteration 37871 / 49000) loss: 1.30835946832\n",
      "(Iteration 37881 / 49000) loss: 1.31906342965\n",
      "(Iteration 37891 / 49000) loss: 1.42847482435\n",
      "(Iteration 37901 / 49000) loss: 1.34170538304\n",
      "(Iteration 37911 / 49000) loss: 1.30583906976\n",
      "(Iteration 37921 / 49000) loss: 1.19969574409\n",
      "(Epoch 387 / 500) Training Accuracy: 0.544244897959, Validation Accuracy: 0.473\n",
      "(Iteration 37931 / 49000) loss: 1.33755923933\n",
      "(Iteration 37941 / 49000) loss: 1.30507640975\n",
      "(Iteration 37951 / 49000) loss: 1.32680273703\n",
      "(Iteration 37961 / 49000) loss: 1.28934756105\n",
      "(Iteration 37971 / 49000) loss: 1.29118416376\n",
      "(Iteration 37981 / 49000) loss: 1.23507555054\n",
      "(Iteration 37991 / 49000) loss: 1.31488879052\n",
      "(Iteration 38001 / 49000) loss: 1.36640133453\n",
      "(Iteration 38011 / 49000) loss: 1.28318747226\n",
      "(Iteration 38021 / 49000) loss: 1.27044885787\n",
      "(Epoch 388 / 500) Training Accuracy: 0.542428571429, Validation Accuracy: 0.478\n",
      "(Iteration 38031 / 49000) loss: 1.29635645026\n",
      "(Iteration 38041 / 49000) loss: 1.30193197018\n",
      "(Iteration 38051 / 49000) loss: 1.31821453653\n",
      "(Iteration 38061 / 49000) loss: 1.3073279501\n",
      "(Iteration 38071 / 49000) loss: 1.4000308825\n",
      "(Iteration 38081 / 49000) loss: 1.28523658268\n",
      "(Iteration 38091 / 49000) loss: 1.23767311462\n",
      "(Iteration 38101 / 49000) loss: 1.25730623395\n",
      "(Iteration 38111 / 49000) loss: 1.36508801582\n",
      "(Iteration 38121 / 49000) loss: 1.32971363801\n",
      "(Epoch 389 / 500) Training Accuracy: 0.547346938776, Validation Accuracy: 0.475\n",
      "(Iteration 38131 / 49000) loss: 1.37268270955\n",
      "(Iteration 38141 / 49000) loss: 1.24728759599\n",
      "(Iteration 38151 / 49000) loss: 1.24522865555\n",
      "(Iteration 38161 / 49000) loss: 1.28624889668\n",
      "(Iteration 38171 / 49000) loss: 1.3202530913\n",
      "(Iteration 38181 / 49000) loss: 1.30952471795\n",
      "(Iteration 38191 / 49000) loss: 1.24702185665\n",
      "(Iteration 38201 / 49000) loss: 1.33858360615\n",
      "(Iteration 38211 / 49000) loss: 1.26823444477\n",
      "(Epoch 390 / 500) Training Accuracy: 0.545040816327, Validation Accuracy: 0.465\n",
      "(Iteration 38221 / 49000) loss: 1.32763460145\n",
      "(Iteration 38231 / 49000) loss: 1.32147463371\n",
      "(Iteration 38241 / 49000) loss: 1.23088632411\n",
      "(Iteration 38251 / 49000) loss: 1.33241637735\n",
      "(Iteration 38261 / 49000) loss: 1.31340817892\n",
      "(Iteration 38271 / 49000) loss: 1.38433946152\n",
      "(Iteration 38281 / 49000) loss: 1.26234671399\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 38291 / 49000) loss: 1.27891326581\n",
      "(Iteration 38301 / 49000) loss: 1.29786242485\n",
      "(Iteration 38311 / 49000) loss: 1.31874951443\n",
      "(Epoch 391 / 500) Training Accuracy: 0.541040816327, Validation Accuracy: 0.473\n",
      "(Iteration 38321 / 49000) loss: 1.30312375662\n",
      "(Iteration 38331 / 49000) loss: 1.35019800371\n",
      "(Iteration 38341 / 49000) loss: 1.2951568254\n",
      "(Iteration 38351 / 49000) loss: 1.31688481179\n",
      "(Iteration 38361 / 49000) loss: 1.40659532207\n",
      "(Iteration 38371 / 49000) loss: 1.35142940254\n",
      "(Iteration 38381 / 49000) loss: 1.34531679585\n",
      "(Iteration 38391 / 49000) loss: 1.30833596348\n",
      "(Iteration 38401 / 49000) loss: 1.36055001326\n",
      "(Iteration 38411 / 49000) loss: 1.35017828145\n",
      "(Epoch 392 / 500) Training Accuracy: 0.544897959184, Validation Accuracy: 0.469\n",
      "(Iteration 38421 / 49000) loss: 1.33961883355\n",
      "(Iteration 38431 / 49000) loss: 1.35288843267\n",
      "(Iteration 38441 / 49000) loss: 1.37894535254\n",
      "(Iteration 38451 / 49000) loss: 1.32058027286\n",
      "(Iteration 38461 / 49000) loss: 1.29597241118\n",
      "(Iteration 38471 / 49000) loss: 1.33333836781\n",
      "(Iteration 38481 / 49000) loss: 1.43275147926\n",
      "(Iteration 38491 / 49000) loss: 1.39218509154\n",
      "(Iteration 38501 / 49000) loss: 1.28704156431\n",
      "(Iteration 38511 / 49000) loss: 1.35758743158\n",
      "(Epoch 393 / 500) Training Accuracy: 0.542816326531, Validation Accuracy: 0.478\n",
      "(Iteration 38521 / 49000) loss: 1.33146572365\n",
      "(Iteration 38531 / 49000) loss: 1.36206799628\n",
      "(Iteration 38541 / 49000) loss: 1.21837945661\n",
      "(Iteration 38551 / 49000) loss: 1.29403384103\n",
      "(Iteration 38561 / 49000) loss: 1.28530722755\n",
      "(Iteration 38571 / 49000) loss: 1.3202186605\n",
      "(Iteration 38581 / 49000) loss: 1.2694513433\n",
      "(Iteration 38591 / 49000) loss: 1.26573011366\n",
      "(Iteration 38601 / 49000) loss: 1.3402606507\n",
      "(Iteration 38611 / 49000) loss: 1.40562275011\n",
      "(Epoch 394 / 500) Training Accuracy: 0.546408163265, Validation Accuracy: 0.472\n",
      "(Iteration 38621 / 49000) loss: 1.21465431244\n",
      "(Iteration 38631 / 49000) loss: 1.30921840855\n",
      "(Iteration 38641 / 49000) loss: 1.43148871632\n",
      "(Iteration 38651 / 49000) loss: 1.25866281171\n",
      "(Iteration 38661 / 49000) loss: 1.33141253091\n",
      "(Iteration 38671 / 49000) loss: 1.28802534981\n",
      "(Iteration 38681 / 49000) loss: 1.33119903204\n",
      "(Iteration 38691 / 49000) loss: 1.39702169775\n",
      "(Iteration 38701 / 49000) loss: 1.27182419095\n",
      "(Epoch 395 / 500) Training Accuracy: 0.546428571429, Validation Accuracy: 0.466\n",
      "(Iteration 38711 / 49000) loss: 1.23585912953\n",
      "(Iteration 38721 / 49000) loss: 1.3028706461\n",
      "(Iteration 38731 / 49000) loss: 1.24903068473\n",
      "(Iteration 38741 / 49000) loss: 1.3814121402\n",
      "(Iteration 38751 / 49000) loss: 1.25108690326\n",
      "(Iteration 38761 / 49000) loss: 1.33325651709\n",
      "(Iteration 38771 / 49000) loss: 1.2818202816\n",
      "(Iteration 38781 / 49000) loss: 1.31854816196\n",
      "(Iteration 38791 / 49000) loss: 1.32146253224\n",
      "(Iteration 38801 / 49000) loss: 1.22935626861\n",
      "(Epoch 396 / 500) Training Accuracy: 0.550102040816, Validation Accuracy: 0.469\n",
      "(Iteration 38811 / 49000) loss: 1.38995428177\n",
      "(Iteration 38821 / 49000) loss: 1.35714989831\n",
      "(Iteration 38831 / 49000) loss: 1.25526346679\n",
      "(Iteration 38841 / 49000) loss: 1.36844924217\n",
      "(Iteration 38851 / 49000) loss: 1.28724497919\n",
      "(Iteration 38861 / 49000) loss: 1.27882922538\n",
      "(Iteration 38871 / 49000) loss: 1.24870670194\n",
      "(Iteration 38881 / 49000) loss: 1.28010267593\n",
      "(Iteration 38891 / 49000) loss: 1.30575947382\n",
      "(Iteration 38901 / 49000) loss: 1.29142848072\n",
      "(Epoch 397 / 500) Training Accuracy: 0.540428571429, Validation Accuracy: 0.476\n",
      "(Iteration 38911 / 49000) loss: 1.26817577474\n",
      "(Iteration 38921 / 49000) loss: 1.25088154002\n",
      "(Iteration 38931 / 49000) loss: 1.32480844452\n",
      "(Iteration 38941 / 49000) loss: 1.2554077983\n",
      "(Iteration 38951 / 49000) loss: 1.22402711241\n",
      "(Iteration 38961 / 49000) loss: 1.38138610034\n",
      "(Iteration 38971 / 49000) loss: 1.2455358658\n",
      "(Iteration 38981 / 49000) loss: 1.32163020453\n",
      "(Iteration 38991 / 49000) loss: 1.34078213314\n",
      "(Iteration 39001 / 49000) loss: 1.26973600097\n",
      "(Epoch 398 / 500) Training Accuracy: 0.548795918367, Validation Accuracy: 0.475\n",
      "(Iteration 39011 / 49000) loss: 1.29961505469\n",
      "(Iteration 39021 / 49000) loss: 1.34987664097\n",
      "(Iteration 39031 / 49000) loss: 1.28717806255\n",
      "(Iteration 39041 / 49000) loss: 1.23675465599\n",
      "(Iteration 39051 / 49000) loss: 1.33632405846\n",
      "(Iteration 39061 / 49000) loss: 1.34667785601\n",
      "(Iteration 39071 / 49000) loss: 1.23199581395\n",
      "(Iteration 39081 / 49000) loss: 1.33125285299\n",
      "(Iteration 39091 / 49000) loss: 1.34360419241\n",
      "(Iteration 39101 / 49000) loss: 1.30784704795\n",
      "(Epoch 399 / 500) Training Accuracy: 0.543081632653, Validation Accuracy: 0.463\n",
      "(Iteration 39111 / 49000) loss: 1.33339167472\n",
      "(Iteration 39121 / 49000) loss: 1.29707306802\n",
      "(Iteration 39131 / 49000) loss: 1.41009132836\n",
      "(Iteration 39141 / 49000) loss: 1.29058955844\n",
      "(Iteration 39151 / 49000) loss: 1.32257249685\n",
      "(Iteration 39161 / 49000) loss: 1.32090249599\n",
      "(Iteration 39171 / 49000) loss: 1.36534078096\n",
      "(Iteration 39181 / 49000) loss: 1.27833521253\n",
      "(Iteration 39191 / 49000) loss: 1.24597986978\n",
      "(Epoch 400 / 500) Training Accuracy: 0.546795918367, Validation Accuracy: 0.475\n",
      "(Iteration 39201 / 49000) loss: 1.29112899448\n",
      "(Iteration 39211 / 49000) loss: 1.24783730408\n",
      "(Iteration 39221 / 49000) loss: 1.2260274225\n",
      "(Iteration 39231 / 49000) loss: 1.36336484954\n",
      "(Iteration 39241 / 49000) loss: 1.29235994823\n",
      "(Iteration 39251 / 49000) loss: 1.34365200369\n",
      "(Iteration 39261 / 49000) loss: 1.3298658283\n",
      "(Iteration 39271 / 49000) loss: 1.26007286834\n",
      "(Iteration 39281 / 49000) loss: 1.29986262596\n",
      "(Iteration 39291 / 49000) loss: 1.36328606772\n",
      "(Epoch 401 / 500) Training Accuracy: 0.55, Validation Accuracy: 0.477\n",
      "(Iteration 39301 / 49000) loss: 1.25262061248\n",
      "(Iteration 39311 / 49000) loss: 1.2360755231\n",
      "(Iteration 39321 / 49000) loss: 1.28147822016\n",
      "(Iteration 39331 / 49000) loss: 1.22241696176\n",
      "(Iteration 39341 / 49000) loss: 1.25288623296\n",
      "(Iteration 39351 / 49000) loss: 1.28488984638\n",
      "(Iteration 39361 / 49000) loss: 1.32192319335\n",
      "(Iteration 39371 / 49000) loss: 1.37498151822\n",
      "(Iteration 39381 / 49000) loss: 1.26914582925\n",
      "(Iteration 39391 / 49000) loss: 1.30211336802\n",
      "(Epoch 402 / 500) Training Accuracy: 0.548714285714, Validation Accuracy: 0.473\n",
      "(Iteration 39401 / 49000) loss: 1.27645838005\n",
      "(Iteration 39411 / 49000) loss: 1.29959695326\n",
      "(Iteration 39421 / 49000) loss: 1.30459593509\n",
      "(Iteration 39431 / 49000) loss: 1.2209401609\n",
      "(Iteration 39441 / 49000) loss: 1.27155674108\n",
      "(Iteration 39451 / 49000) loss: 1.41924832749\n",
      "(Iteration 39461 / 49000) loss: 1.33631624179\n",
      "(Iteration 39471 / 49000) loss: 1.36952041052\n",
      "(Iteration 39481 / 49000) loss: 1.32010894395\n",
      "(Iteration 39491 / 49000) loss: 1.31225230942\n",
      "(Epoch 403 / 500) Training Accuracy: 0.542346938776, Validation Accuracy: 0.466\n",
      "(Iteration 39501 / 49000) loss: 1.350021552\n",
      "(Iteration 39511 / 49000) loss: 1.32464571568\n",
      "(Iteration 39521 / 49000) loss: 1.3084527805\n",
      "(Iteration 39531 / 49000) loss: 1.38656438653\n",
      "(Iteration 39541 / 49000) loss: 1.35141732373\n",
      "(Iteration 39551 / 49000) loss: 1.22303212877\n",
      "(Iteration 39561 / 49000) loss: 1.25946856643\n",
      "(Iteration 39571 / 49000) loss: 1.24851279538\n",
      "(Iteration 39581 / 49000) loss: 1.30977095094\n",
      "(Iteration 39591 / 49000) loss: 1.28858759393\n",
      "(Epoch 404 / 500) Training Accuracy: 0.549897959184, Validation Accuracy: 0.474\n",
      "(Iteration 39601 / 49000) loss: 1.22382641783\n",
      "(Iteration 39611 / 49000) loss: 1.30164449411\n",
      "(Iteration 39621 / 49000) loss: 1.38317932598\n",
      "(Iteration 39631 / 49000) loss: 1.29748409376\n",
      "(Iteration 39641 / 49000) loss: 1.34248877201\n",
      "(Iteration 39651 / 49000) loss: 1.32137344169\n",
      "(Iteration 39661 / 49000) loss: 1.3620360145\n",
      "(Iteration 39671 / 49000) loss: 1.24998568889\n",
      "(Iteration 39681 / 49000) loss: 1.30619794694\n",
      "(Epoch 405 / 500) Training Accuracy: 0.543367346939, Validation Accuracy: 0.464\n",
      "(Iteration 39691 / 49000) loss: 1.28486593866\n",
      "(Iteration 39701 / 49000) loss: 1.32380777273\n",
      "(Iteration 39711 / 49000) loss: 1.35663956785\n",
      "(Iteration 39721 / 49000) loss: 1.29000560226\n",
      "(Iteration 39731 / 49000) loss: 1.30369159954\n",
      "(Iteration 39741 / 49000) loss: 1.26873009663\n",
      "(Iteration 39751 / 49000) loss: 1.35394181463\n",
      "(Iteration 39761 / 49000) loss: 1.28950179126\n",
      "(Iteration 39771 / 49000) loss: 1.34720587758\n",
      "(Iteration 39781 / 49000) loss: 1.26702690291\n",
      "(Epoch 406 / 500) Training Accuracy: 0.54306122449, Validation Accuracy: 0.472\n",
      "(Iteration 39791 / 49000) loss: 1.292029205\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 39801 / 49000) loss: 1.26899085302\n",
      "(Iteration 39811 / 49000) loss: 1.34554349617\n",
      "(Iteration 39821 / 49000) loss: 1.40025129881\n",
      "(Iteration 39831 / 49000) loss: 1.3640720123\n",
      "(Iteration 39841 / 49000) loss: 1.26675986817\n",
      "(Iteration 39851 / 49000) loss: 1.23913807216\n",
      "(Iteration 39861 / 49000) loss: 1.2988616683\n",
      "(Iteration 39871 / 49000) loss: 1.28714132933\n",
      "(Iteration 39881 / 49000) loss: 1.33460177275\n",
      "(Epoch 407 / 500) Training Accuracy: 0.549755102041, Validation Accuracy: 0.473\n",
      "(Iteration 39891 / 49000) loss: 1.32637723393\n",
      "(Iteration 39901 / 49000) loss: 1.27924262833\n",
      "(Iteration 39911 / 49000) loss: 1.22946920009\n",
      "(Iteration 39921 / 49000) loss: 1.24419775709\n",
      "(Iteration 39931 / 49000) loss: 1.23890307957\n",
      "(Iteration 39941 / 49000) loss: 1.22650065716\n",
      "(Iteration 39951 / 49000) loss: 1.39264362155\n",
      "(Iteration 39961 / 49000) loss: 1.2742989407\n",
      "(Iteration 39971 / 49000) loss: 1.38348872063\n",
      "(Iteration 39981 / 49000) loss: 1.28993596352\n",
      "(Epoch 408 / 500) Training Accuracy: 0.549755102041, Validation Accuracy: 0.471\n",
      "(Iteration 39991 / 49000) loss: 1.27147015131\n",
      "(Iteration 40001 / 49000) loss: 1.33496796164\n",
      "(Iteration 40011 / 49000) loss: 1.34015871643\n",
      "(Iteration 40021 / 49000) loss: 1.38635104408\n",
      "(Iteration 40031 / 49000) loss: 1.31693079489\n",
      "(Iteration 40041 / 49000) loss: 1.33007510329\n",
      "(Iteration 40051 / 49000) loss: 1.36871906795\n",
      "(Iteration 40061 / 49000) loss: 1.33496753944\n",
      "(Iteration 40071 / 49000) loss: 1.24738313395\n",
      "(Iteration 40081 / 49000) loss: 1.20353069889\n",
      "(Epoch 409 / 500) Training Accuracy: 0.549367346939, Validation Accuracy: 0.476\n",
      "(Iteration 40091 / 49000) loss: 1.24825884047\n",
      "(Iteration 40101 / 49000) loss: 1.25401387207\n",
      "(Iteration 40111 / 49000) loss: 1.2491332376\n",
      "(Iteration 40121 / 49000) loss: 1.35727819154\n",
      "(Iteration 40131 / 49000) loss: 1.23712152356\n",
      "(Iteration 40141 / 49000) loss: 1.29878845813\n",
      "(Iteration 40151 / 49000) loss: 1.31418881251\n",
      "(Iteration 40161 / 49000) loss: 1.32814904231\n",
      "(Iteration 40171 / 49000) loss: 1.34369374032\n",
      "(Epoch 410 / 500) Training Accuracy: 0.550816326531, Validation Accuracy: 0.475\n",
      "(Iteration 40181 / 49000) loss: 1.25644385558\n",
      "(Iteration 40191 / 49000) loss: 1.26632633574\n",
      "(Iteration 40201 / 49000) loss: 1.24423364823\n",
      "(Iteration 40211 / 49000) loss: 1.31277161184\n",
      "(Iteration 40221 / 49000) loss: 1.34904760951\n",
      "(Iteration 40231 / 49000) loss: 1.25815658344\n",
      "(Iteration 40241 / 49000) loss: 1.24394918873\n",
      "(Iteration 40251 / 49000) loss: 1.30991909478\n",
      "(Iteration 40261 / 49000) loss: 1.28025459937\n",
      "(Iteration 40271 / 49000) loss: 1.38791937701\n",
      "(Epoch 411 / 500) Training Accuracy: 0.548979591837, Validation Accuracy: 0.47\n",
      "(Iteration 40281 / 49000) loss: 1.26085470545\n",
      "(Iteration 40291 / 49000) loss: 1.20725619557\n",
      "(Iteration 40301 / 49000) loss: 1.31279952626\n",
      "(Iteration 40311 / 49000) loss: 1.29887411668\n",
      "(Iteration 40321 / 49000) loss: 1.32352688516\n",
      "(Iteration 40331 / 49000) loss: 1.31162767203\n",
      "(Iteration 40341 / 49000) loss: 1.36411541031\n",
      "(Iteration 40351 / 49000) loss: 1.28816203275\n",
      "(Iteration 40361 / 49000) loss: 1.28720568058\n",
      "(Iteration 40371 / 49000) loss: 1.29229061361\n",
      "(Epoch 412 / 500) Training Accuracy: 0.550469387755, Validation Accuracy: 0.475\n",
      "(Iteration 40381 / 49000) loss: 1.25986239645\n",
      "(Iteration 40391 / 49000) loss: 1.32284538784\n",
      "(Iteration 40401 / 49000) loss: 1.24335324958\n",
      "(Iteration 40411 / 49000) loss: 1.38083629652\n",
      "(Iteration 40421 / 49000) loss: 1.23293055141\n",
      "(Iteration 40431 / 49000) loss: 1.31807356811\n",
      "(Iteration 40441 / 49000) loss: 1.25923976981\n",
      "(Iteration 40451 / 49000) loss: 1.27773277578\n",
      "(Iteration 40461 / 49000) loss: 1.33900868268\n",
      "(Iteration 40471 / 49000) loss: 1.32159607889\n",
      "(Epoch 413 / 500) Training Accuracy: 0.548693877551, Validation Accuracy: 0.469\n",
      "(Iteration 40481 / 49000) loss: 1.30044705858\n",
      "(Iteration 40491 / 49000) loss: 1.36624806137\n",
      "(Iteration 40501 / 49000) loss: 1.30196287915\n",
      "(Iteration 40511 / 49000) loss: 1.27645413647\n",
      "(Iteration 40521 / 49000) loss: 1.33039459001\n",
      "(Iteration 40531 / 49000) loss: 1.30718854539\n",
      "(Iteration 40541 / 49000) loss: 1.28651058847\n",
      "(Iteration 40551 / 49000) loss: 1.34094322486\n",
      "(Iteration 40561 / 49000) loss: 1.3202695364\n",
      "(Iteration 40571 / 49000) loss: 1.41967010422\n",
      "(Epoch 414 / 500) Training Accuracy: 0.545224489796, Validation Accuracy: 0.467\n",
      "(Iteration 40581 / 49000) loss: 1.28896448324\n",
      "(Iteration 40591 / 49000) loss: 1.40292116489\n",
      "(Iteration 40601 / 49000) loss: 1.35587143688\n",
      "(Iteration 40611 / 49000) loss: 1.34087854674\n",
      "(Iteration 40621 / 49000) loss: 1.25879553689\n",
      "(Iteration 40631 / 49000) loss: 1.25419781719\n",
      "(Iteration 40641 / 49000) loss: 1.21388504227\n",
      "(Iteration 40651 / 49000) loss: 1.35286476526\n",
      "(Iteration 40661 / 49000) loss: 1.26134773273\n",
      "(Epoch 415 / 500) Training Accuracy: 0.551510204082, Validation Accuracy: 0.479\n",
      "(Iteration 40671 / 49000) loss: 1.2165068209\n",
      "(Iteration 40681 / 49000) loss: 1.30390465711\n",
      "(Iteration 40691 / 49000) loss: 1.29205323499\n",
      "(Iteration 40701 / 49000) loss: 1.30385950364\n",
      "(Iteration 40711 / 49000) loss: 1.28568576651\n",
      "(Iteration 40721 / 49000) loss: 1.24571568023\n",
      "(Iteration 40731 / 49000) loss: 1.27231732716\n",
      "(Iteration 40741 / 49000) loss: 1.30332797443\n",
      "(Iteration 40751 / 49000) loss: 1.22574924335\n",
      "(Iteration 40761 / 49000) loss: 1.27802257069\n",
      "(Epoch 416 / 500) Training Accuracy: 0.551897959184, Validation Accuracy: 0.475\n",
      "(Iteration 40771 / 49000) loss: 1.27146704049\n",
      "(Iteration 40781 / 49000) loss: 1.27812510363\n",
      "(Iteration 40791 / 49000) loss: 1.3320442174\n",
      "(Iteration 40801 / 49000) loss: 1.2506611924\n",
      "(Iteration 40811 / 49000) loss: 1.30756808286\n",
      "(Iteration 40821 / 49000) loss: 1.3905376067\n",
      "(Iteration 40831 / 49000) loss: 1.27453571259\n",
      "(Iteration 40841 / 49000) loss: 1.32360292717\n",
      "(Iteration 40851 / 49000) loss: 1.31068248745\n",
      "(Iteration 40861 / 49000) loss: 1.29680433318\n",
      "(Epoch 417 / 500) Training Accuracy: 0.549959183673, Validation Accuracy: 0.471\n",
      "(Iteration 40871 / 49000) loss: 1.27490323235\n",
      "(Iteration 40881 / 49000) loss: 1.29290934718\n",
      "(Iteration 40891 / 49000) loss: 1.35222036549\n",
      "(Iteration 40901 / 49000) loss: 1.23535564265\n",
      "(Iteration 40911 / 49000) loss: 1.22640076417\n",
      "(Iteration 40921 / 49000) loss: 1.29988204539\n",
      "(Iteration 40931 / 49000) loss: 1.34125844379\n",
      "(Iteration 40941 / 49000) loss: 1.2513932478\n",
      "(Iteration 40951 / 49000) loss: 1.24493673203\n",
      "(Iteration 40961 / 49000) loss: 1.19809664977\n",
      "(Epoch 418 / 500) Training Accuracy: 0.551857142857, Validation Accuracy: 0.47\n",
      "(Iteration 40971 / 49000) loss: 1.24098491934\n",
      "(Iteration 40981 / 49000) loss: 1.35102197443\n",
      "(Iteration 40991 / 49000) loss: 1.32921672111\n",
      "(Iteration 41001 / 49000) loss: 1.28951822335\n",
      "(Iteration 41011 / 49000) loss: 1.34817778282\n",
      "(Iteration 41021 / 49000) loss: 1.28631422576\n",
      "(Iteration 41031 / 49000) loss: 1.20827237199\n",
      "(Iteration 41041 / 49000) loss: 1.31790883839\n",
      "(Iteration 41051 / 49000) loss: 1.30122773995\n",
      "(Iteration 41061 / 49000) loss: 1.30717098393\n",
      "(Epoch 419 / 500) Training Accuracy: 0.548897959184, Validation Accuracy: 0.478\n",
      "(Iteration 41071 / 49000) loss: 1.29831038519\n",
      "(Iteration 41081 / 49000) loss: 1.18751187579\n",
      "(Iteration 41091 / 49000) loss: 1.34471084254\n",
      "(Iteration 41101 / 49000) loss: 1.33734369686\n",
      "(Iteration 41111 / 49000) loss: 1.29539083395\n",
      "(Iteration 41121 / 49000) loss: 1.25880257359\n",
      "(Iteration 41131 / 49000) loss: 1.33867660552\n",
      "(Iteration 41141 / 49000) loss: 1.40736182896\n",
      "(Iteration 41151 / 49000) loss: 1.22748277223\n",
      "(Epoch 420 / 500) Training Accuracy: 0.553020408163, Validation Accuracy: 0.48\n",
      "(Iteration 41161 / 49000) loss: 1.3317416115\n",
      "(Iteration 41171 / 49000) loss: 1.2654250046\n",
      "(Iteration 41181 / 49000) loss: 1.26587540358\n",
      "(Iteration 41191 / 49000) loss: 1.37723118653\n",
      "(Iteration 41201 / 49000) loss: 1.29512894632\n",
      "(Iteration 41211 / 49000) loss: 1.33132071136\n",
      "(Iteration 41221 / 49000) loss: 1.28580128669\n",
      "(Iteration 41231 / 49000) loss: 1.32292068998\n",
      "(Iteration 41241 / 49000) loss: 1.30493067243\n",
      "(Iteration 41251 / 49000) loss: 1.2876822524\n",
      "(Epoch 421 / 500) Training Accuracy: 0.550102040816, Validation Accuracy: 0.479\n",
      "(Iteration 41261 / 49000) loss: 1.30140604569\n",
      "(Iteration 41271 / 49000) loss: 1.2072264879\n",
      "(Iteration 41281 / 49000) loss: 1.36153094406\n",
      "(Iteration 41291 / 49000) loss: 1.31986948865\n",
      "(Iteration 41301 / 49000) loss: 1.21494723778\n",
      "(Iteration 41311 / 49000) loss: 1.24635775712\n",
      "(Iteration 41321 / 49000) loss: 1.28611220586\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 41331 / 49000) loss: 1.19634042275\n",
      "(Iteration 41341 / 49000) loss: 1.30739778982\n",
      "(Iteration 41351 / 49000) loss: 1.31084351701\n",
      "(Epoch 422 / 500) Training Accuracy: 0.554612244898, Validation Accuracy: 0.483\n",
      "(Iteration 41361 / 49000) loss: 1.28637069031\n",
      "(Iteration 41371 / 49000) loss: 1.23659150152\n",
      "(Iteration 41381 / 49000) loss: 1.28240866816\n",
      "(Iteration 41391 / 49000) loss: 1.32688137403\n",
      "(Iteration 41401 / 49000) loss: 1.21353135919\n",
      "(Iteration 41411 / 49000) loss: 1.25228572517\n",
      "(Iteration 41421 / 49000) loss: 1.27709714748\n",
      "(Iteration 41431 / 49000) loss: 1.30414556228\n",
      "(Iteration 41441 / 49000) loss: 1.2915967025\n",
      "(Iteration 41451 / 49000) loss: 1.26888621964\n",
      "(Epoch 423 / 500) Training Accuracy: 0.553224489796, Validation Accuracy: 0.479\n",
      "(Iteration 41461 / 49000) loss: 1.30265684111\n",
      "(Iteration 41471 / 49000) loss: 1.33706962068\n",
      "(Iteration 41481 / 49000) loss: 1.27033747291\n",
      "(Iteration 41491 / 49000) loss: 1.28156143179\n",
      "(Iteration 41501 / 49000) loss: 1.20377823818\n",
      "(Iteration 41511 / 49000) loss: 1.26420951681\n",
      "(Iteration 41521 / 49000) loss: 1.2618991274\n",
      "(Iteration 41531 / 49000) loss: 1.26118248563\n",
      "(Iteration 41541 / 49000) loss: 1.35279287129\n",
      "(Iteration 41551 / 49000) loss: 1.34190658791\n",
      "(Epoch 424 / 500) Training Accuracy: 0.543612244898, Validation Accuracy: 0.472\n",
      "(Iteration 41561 / 49000) loss: 1.18766118214\n",
      "(Iteration 41571 / 49000) loss: 1.28957808126\n",
      "(Iteration 41581 / 49000) loss: 1.18336478789\n",
      "(Iteration 41591 / 49000) loss: 1.33970209972\n",
      "(Iteration 41601 / 49000) loss: 1.4264472666\n",
      "(Iteration 41611 / 49000) loss: 1.32594654231\n",
      "(Iteration 41621 / 49000) loss: 1.23332072285\n",
      "(Iteration 41631 / 49000) loss: 1.29294382603\n",
      "(Iteration 41641 / 49000) loss: 1.25991526961\n",
      "(Epoch 425 / 500) Training Accuracy: 0.552265306122, Validation Accuracy: 0.475\n",
      "(Iteration 41651 / 49000) loss: 1.20878018018\n",
      "(Iteration 41661 / 49000) loss: 1.22534413331\n",
      "(Iteration 41671 / 49000) loss: 1.32291116521\n",
      "(Iteration 41681 / 49000) loss: 1.30898067202\n",
      "(Iteration 41691 / 49000) loss: 1.21569644956\n",
      "(Iteration 41701 / 49000) loss: 1.21006390227\n",
      "(Iteration 41711 / 49000) loss: 1.35553693204\n",
      "(Iteration 41721 / 49000) loss: 1.37901947593\n",
      "(Iteration 41731 / 49000) loss: 1.31330885294\n",
      "(Iteration 41741 / 49000) loss: 1.29163514312\n",
      "(Epoch 426 / 500) Training Accuracy: 0.546673469388, Validation Accuracy: 0.473\n",
      "(Iteration 41751 / 49000) loss: 1.38479159091\n",
      "(Iteration 41761 / 49000) loss: 1.25735209198\n",
      "(Iteration 41771 / 49000) loss: 1.18230265691\n",
      "(Iteration 41781 / 49000) loss: 1.29971008496\n",
      "(Iteration 41791 / 49000) loss: 1.25663077717\n",
      "(Iteration 41801 / 49000) loss: 1.25703576427\n",
      "(Iteration 41811 / 49000) loss: 1.23873547116\n",
      "(Iteration 41821 / 49000) loss: 1.23971167729\n",
      "(Iteration 41831 / 49000) loss: 1.30385651269\n",
      "(Iteration 41841 / 49000) loss: 1.39235372016\n",
      "(Epoch 427 / 500) Training Accuracy: 0.552224489796, Validation Accuracy: 0.47\n",
      "(Iteration 41851 / 49000) loss: 1.25780067642\n",
      "(Iteration 41861 / 49000) loss: 1.23162044764\n",
      "(Iteration 41871 / 49000) loss: 1.29874512069\n",
      "(Iteration 41881 / 49000) loss: 1.19046131549\n",
      "(Iteration 41891 / 49000) loss: 1.33655950685\n",
      "(Iteration 41901 / 49000) loss: 1.31312139583\n",
      "(Iteration 41911 / 49000) loss: 1.28009185065\n",
      "(Iteration 41921 / 49000) loss: 1.29867837505\n",
      "(Iteration 41931 / 49000) loss: 1.17585248439\n",
      "(Iteration 41941 / 49000) loss: 1.24671186869\n",
      "(Epoch 428 / 500) Training Accuracy: 0.554387755102, Validation Accuracy: 0.476\n",
      "(Iteration 41951 / 49000) loss: 1.19213445905\n",
      "(Iteration 41961 / 49000) loss: 1.24293709738\n",
      "(Iteration 41971 / 49000) loss: 1.23724260313\n",
      "(Iteration 41981 / 49000) loss: 1.29724918532\n",
      "(Iteration 41991 / 49000) loss: 1.28200478075\n",
      "(Iteration 42001 / 49000) loss: 1.2350460455\n",
      "(Iteration 42011 / 49000) loss: 1.33040728619\n",
      "(Iteration 42021 / 49000) loss: 1.33295379039\n",
      "(Iteration 42031 / 49000) loss: 1.30420056238\n",
      "(Iteration 42041 / 49000) loss: 1.30434096596\n",
      "(Epoch 429 / 500) Training Accuracy: 0.555653061224, Validation Accuracy: 0.478\n",
      "(Iteration 42051 / 49000) loss: 1.25758217646\n",
      "(Iteration 42061 / 49000) loss: 1.24565302046\n",
      "(Iteration 42071 / 49000) loss: 1.33772179871\n",
      "(Iteration 42081 / 49000) loss: 1.36754971672\n",
      "(Iteration 42091 / 49000) loss: 1.22027279156\n",
      "(Iteration 42101 / 49000) loss: 1.27063033212\n",
      "(Iteration 42111 / 49000) loss: 1.33848545972\n",
      "(Iteration 42121 / 49000) loss: 1.27207597222\n",
      "(Iteration 42131 / 49000) loss: 1.31942324087\n",
      "(Epoch 430 / 500) Training Accuracy: 0.554714285714, Validation Accuracy: 0.469\n",
      "(Iteration 42141 / 49000) loss: 1.26924982177\n",
      "(Iteration 42151 / 49000) loss: 1.28842011062\n",
      "(Iteration 42161 / 49000) loss: 1.29121403338\n",
      "(Iteration 42171 / 49000) loss: 1.17142369214\n",
      "(Iteration 42181 / 49000) loss: 1.23959781034\n",
      "(Iteration 42191 / 49000) loss: 1.21656038535\n",
      "(Iteration 42201 / 49000) loss: 1.21182583915\n",
      "(Iteration 42211 / 49000) loss: 1.31920512886\n",
      "(Iteration 42221 / 49000) loss: 1.24708060389\n",
      "(Iteration 42231 / 49000) loss: 1.30662373466\n",
      "(Epoch 431 / 500) Training Accuracy: 0.554326530612, Validation Accuracy: 0.475\n",
      "(Iteration 42241 / 49000) loss: 1.27244579049\n",
      "(Iteration 42251 / 49000) loss: 1.22800849828\n",
      "(Iteration 42261 / 49000) loss: 1.27521215954\n",
      "(Iteration 42271 / 49000) loss: 1.36213547142\n",
      "(Iteration 42281 / 49000) loss: 1.27068927391\n",
      "(Iteration 42291 / 49000) loss: 1.32306686366\n",
      "(Iteration 42301 / 49000) loss: 1.28354642648\n",
      "(Iteration 42311 / 49000) loss: 1.33066664056\n",
      "(Iteration 42321 / 49000) loss: 1.3268713725\n",
      "(Iteration 42331 / 49000) loss: 1.2786727366\n",
      "(Epoch 432 / 500) Training Accuracy: 0.553469387755, Validation Accuracy: 0.478\n",
      "(Iteration 42341 / 49000) loss: 1.21717863587\n",
      "(Iteration 42351 / 49000) loss: 1.26108490571\n",
      "(Iteration 42361 / 49000) loss: 1.21127810108\n",
      "(Iteration 42371 / 49000) loss: 1.23871050922\n",
      "(Iteration 42381 / 49000) loss: 1.3086935358\n",
      "(Iteration 42391 / 49000) loss: 1.28745437137\n",
      "(Iteration 42401 / 49000) loss: 1.23971135517\n",
      "(Iteration 42411 / 49000) loss: 1.27527037819\n",
      "(Iteration 42421 / 49000) loss: 1.296621582\n",
      "(Iteration 42431 / 49000) loss: 1.40065205158\n",
      "(Epoch 433 / 500) Training Accuracy: 0.541673469388, Validation Accuracy: 0.483\n",
      "(Iteration 42441 / 49000) loss: 1.31236198481\n",
      "(Iteration 42451 / 49000) loss: 1.2606214495\n",
      "(Iteration 42461 / 49000) loss: 1.1740109444\n",
      "(Iteration 42471 / 49000) loss: 1.29069974743\n",
      "(Iteration 42481 / 49000) loss: 1.29824342616\n",
      "(Iteration 42491 / 49000) loss: 1.26880839175\n",
      "(Iteration 42501 / 49000) loss: 1.21319153054\n",
      "(Iteration 42511 / 49000) loss: 1.38605921281\n",
      "(Iteration 42521 / 49000) loss: 1.21371016214\n",
      "(Iteration 42531 / 49000) loss: 1.3555005657\n",
      "(Epoch 434 / 500) Training Accuracy: 0.553428571429, Validation Accuracy: 0.47\n",
      "(Iteration 42541 / 49000) loss: 1.23440708566\n",
      "(Iteration 42551 / 49000) loss: 1.25458823298\n",
      "(Iteration 42561 / 49000) loss: 1.28914796523\n",
      "(Iteration 42571 / 49000) loss: 1.28897088671\n",
      "(Iteration 42581 / 49000) loss: 1.45887399929\n",
      "(Iteration 42591 / 49000) loss: 1.26184569206\n",
      "(Iteration 42601 / 49000) loss: 1.27354247355\n",
      "(Iteration 42611 / 49000) loss: 1.1696120654\n",
      "(Iteration 42621 / 49000) loss: 1.23460475081\n",
      "(Epoch 435 / 500) Training Accuracy: 0.551, Validation Accuracy: 0.476\n",
      "(Iteration 42631 / 49000) loss: 1.30900059846\n",
      "(Iteration 42641 / 49000) loss: 1.2599976282\n",
      "(Iteration 42651 / 49000) loss: 1.26907485414\n",
      "(Iteration 42661 / 49000) loss: 1.28973711558\n",
      "(Iteration 42671 / 49000) loss: 1.30846387105\n",
      "(Iteration 42681 / 49000) loss: 1.24360947924\n",
      "(Iteration 42691 / 49000) loss: 1.32138271896\n",
      "(Iteration 42701 / 49000) loss: 1.21868966912\n",
      "(Iteration 42711 / 49000) loss: 1.28170005386\n",
      "(Iteration 42721 / 49000) loss: 1.34959869742\n",
      "(Epoch 436 / 500) Training Accuracy: 0.554469387755, Validation Accuracy: 0.478\n",
      "(Iteration 42731 / 49000) loss: 1.26380689098\n",
      "(Iteration 42741 / 49000) loss: 1.22257155006\n",
      "(Iteration 42751 / 49000) loss: 1.4163226324\n",
      "(Iteration 42761 / 49000) loss: 1.27615073344\n",
      "(Iteration 42771 / 49000) loss: 1.24485820544\n",
      "(Iteration 42781 / 49000) loss: 1.27417580118\n",
      "(Iteration 42791 / 49000) loss: 1.33441125259\n",
      "(Iteration 42801 / 49000) loss: 1.34331309748\n",
      "(Iteration 42811 / 49000) loss: 1.32281193656\n",
      "(Iteration 42821 / 49000) loss: 1.26602637232\n",
      "(Epoch 437 / 500) Training Accuracy: 0.55293877551, Validation Accuracy: 0.476\n",
      "(Iteration 42831 / 49000) loss: 1.25514628152\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 42841 / 49000) loss: 1.26048078689\n",
      "(Iteration 42851 / 49000) loss: 1.3776844228\n",
      "(Iteration 42861 / 49000) loss: 1.26678812934\n",
      "(Iteration 42871 / 49000) loss: 1.2697898888\n",
      "(Iteration 42881 / 49000) loss: 1.30935123275\n",
      "(Iteration 42891 / 49000) loss: 1.2938516326\n",
      "(Iteration 42901 / 49000) loss: 1.36418507821\n",
      "(Iteration 42911 / 49000) loss: 1.27129226106\n",
      "(Iteration 42921 / 49000) loss: 1.28207967893\n",
      "(Epoch 438 / 500) Training Accuracy: 0.549102040816, Validation Accuracy: 0.482\n",
      "(Iteration 42931 / 49000) loss: 1.24633799595\n",
      "(Iteration 42941 / 49000) loss: 1.31078683859\n",
      "(Iteration 42951 / 49000) loss: 1.28549296372\n",
      "(Iteration 42961 / 49000) loss: 1.28578114812\n",
      "(Iteration 42971 / 49000) loss: 1.33481281263\n",
      "(Iteration 42981 / 49000) loss: 1.25830629961\n",
      "(Iteration 42991 / 49000) loss: 1.14434486501\n",
      "(Iteration 43001 / 49000) loss: 1.30956390274\n",
      "(Iteration 43011 / 49000) loss: 1.26372346391\n",
      "(Iteration 43021 / 49000) loss: 1.17787956811\n",
      "(Epoch 439 / 500) Training Accuracy: 0.551142857143, Validation Accuracy: 0.478\n",
      "(Iteration 43031 / 49000) loss: 1.28674726379\n",
      "(Iteration 43041 / 49000) loss: 1.20164830711\n",
      "(Iteration 43051 / 49000) loss: 1.26151567348\n",
      "(Iteration 43061 / 49000) loss: 1.26094252496\n",
      "(Iteration 43071 / 49000) loss: 1.23597732534\n",
      "(Iteration 43081 / 49000) loss: 1.33870379313\n",
      "(Iteration 43091 / 49000) loss: 1.2417330025\n",
      "(Iteration 43101 / 49000) loss: 1.33994538168\n",
      "(Iteration 43111 / 49000) loss: 1.27748207362\n",
      "(Epoch 440 / 500) Training Accuracy: 0.556469387755, Validation Accuracy: 0.486\n",
      "(Iteration 43121 / 49000) loss: 1.29364472891\n",
      "(Iteration 43131 / 49000) loss: 1.25876834338\n",
      "(Iteration 43141 / 49000) loss: 1.23329841295\n",
      "(Iteration 43151 / 49000) loss: 1.27635445382\n",
      "(Iteration 43161 / 49000) loss: 1.35552814893\n",
      "(Iteration 43171 / 49000) loss: 1.31681828826\n",
      "(Iteration 43181 / 49000) loss: 1.3005731678\n",
      "(Iteration 43191 / 49000) loss: 1.33819036237\n",
      "(Iteration 43201 / 49000) loss: 1.22333045726\n",
      "(Iteration 43211 / 49000) loss: 1.31687035821\n",
      "(Epoch 441 / 500) Training Accuracy: 0.549489795918, Validation Accuracy: 0.473\n",
      "(Iteration 43221 / 49000) loss: 1.30797191701\n",
      "(Iteration 43231 / 49000) loss: 1.22279285423\n",
      "(Iteration 43241 / 49000) loss: 1.25210891848\n",
      "(Iteration 43251 / 49000) loss: 1.27851407272\n",
      "(Iteration 43261 / 49000) loss: 1.28185706618\n",
      "(Iteration 43271 / 49000) loss: 1.16738733006\n",
      "(Iteration 43281 / 49000) loss: 1.33919456159\n",
      "(Iteration 43291 / 49000) loss: 1.28543006835\n",
      "(Iteration 43301 / 49000) loss: 1.18580061041\n",
      "(Iteration 43311 / 49000) loss: 1.30156231785\n",
      "(Epoch 442 / 500) Training Accuracy: 0.557551020408, Validation Accuracy: 0.47\n",
      "(Iteration 43321 / 49000) loss: 1.28011264728\n",
      "(Iteration 43331 / 49000) loss: 1.12901206816\n",
      "(Iteration 43341 / 49000) loss: 1.24462393128\n",
      "(Iteration 43351 / 49000) loss: 1.20188221692\n",
      "(Iteration 43361 / 49000) loss: 1.28911981068\n",
      "(Iteration 43371 / 49000) loss: 1.29325551952\n",
      "(Iteration 43381 / 49000) loss: 1.31059212522\n",
      "(Iteration 43391 / 49000) loss: 1.30113641962\n",
      "(Iteration 43401 / 49000) loss: 1.23732416157\n",
      "(Iteration 43411 / 49000) loss: 1.37645136959\n",
      "(Epoch 443 / 500) Training Accuracy: 0.554163265306, Validation Accuracy: 0.48\n",
      "(Iteration 43421 / 49000) loss: 1.29954607333\n",
      "(Iteration 43431 / 49000) loss: 1.29207535634\n",
      "(Iteration 43441 / 49000) loss: 1.28580464574\n",
      "(Iteration 43451 / 49000) loss: 1.32092424094\n",
      "(Iteration 43461 / 49000) loss: 1.31804078181\n",
      "(Iteration 43471 / 49000) loss: 1.32375488482\n",
      "(Iteration 43481 / 49000) loss: 1.18260888408\n",
      "(Iteration 43491 / 49000) loss: 1.29207701004\n",
      "(Iteration 43501 / 49000) loss: 1.30016663398\n",
      "(Iteration 43511 / 49000) loss: 1.20052545198\n",
      "(Epoch 444 / 500) Training Accuracy: 0.556612244898, Validation Accuracy: 0.479\n",
      "(Iteration 43521 / 49000) loss: 1.26963297677\n",
      "(Iteration 43531 / 49000) loss: 1.28873217349\n",
      "(Iteration 43541 / 49000) loss: 1.20337174943\n",
      "(Iteration 43551 / 49000) loss: 1.21994825229\n",
      "(Iteration 43561 / 49000) loss: 1.27328174849\n",
      "(Iteration 43571 / 49000) loss: 1.244399882\n",
      "(Iteration 43581 / 49000) loss: 1.29633858528\n",
      "(Iteration 43591 / 49000) loss: 1.29154973134\n",
      "(Iteration 43601 / 49000) loss: 1.26593565452\n",
      "(Epoch 445 / 500) Training Accuracy: 0.549673469388, Validation Accuracy: 0.475\n",
      "(Iteration 43611 / 49000) loss: 1.32595763649\n",
      "(Iteration 43621 / 49000) loss: 1.29650314239\n",
      "(Iteration 43631 / 49000) loss: 1.29840580619\n",
      "(Iteration 43641 / 49000) loss: 1.21968980874\n",
      "(Iteration 43651 / 49000) loss: 1.30506579274\n",
      "(Iteration 43661 / 49000) loss: 1.25591228997\n",
      "(Iteration 43671 / 49000) loss: 1.35864545711\n",
      "(Iteration 43681 / 49000) loss: 1.27920084732\n",
      "(Iteration 43691 / 49000) loss: 1.32348090454\n",
      "(Iteration 43701 / 49000) loss: 1.33491219654\n",
      "(Epoch 446 / 500) Training Accuracy: 0.557102040816, Validation Accuracy: 0.474\n",
      "(Iteration 43711 / 49000) loss: 1.29503213992\n",
      "(Iteration 43721 / 49000) loss: 1.30380934774\n",
      "(Iteration 43731 / 49000) loss: 1.23666493637\n",
      "(Iteration 43741 / 49000) loss: 1.19059610342\n",
      "(Iteration 43751 / 49000) loss: 1.25086451216\n",
      "(Iteration 43761 / 49000) loss: 1.21497785598\n",
      "(Iteration 43771 / 49000) loss: 1.38331398092\n",
      "(Iteration 43781 / 49000) loss: 1.23502051128\n",
      "(Iteration 43791 / 49000) loss: 1.31163155601\n",
      "(Iteration 43801 / 49000) loss: 1.1622268315\n",
      "(Epoch 447 / 500) Training Accuracy: 0.557448979592, Validation Accuracy: 0.469\n",
      "(Iteration 43811 / 49000) loss: 1.32993752281\n",
      "(Iteration 43821 / 49000) loss: 1.3007735705\n",
      "(Iteration 43831 / 49000) loss: 1.30694171671\n",
      "(Iteration 43841 / 49000) loss: 1.22732982021\n",
      "(Iteration 43851 / 49000) loss: 1.19444153111\n",
      "(Iteration 43861 / 49000) loss: 1.43001566071\n",
      "(Iteration 43871 / 49000) loss: 1.30462787531\n",
      "(Iteration 43881 / 49000) loss: 1.30216607147\n",
      "(Iteration 43891 / 49000) loss: 1.29956302757\n",
      "(Iteration 43901 / 49000) loss: 1.22877708157\n",
      "(Epoch 448 / 500) Training Accuracy: 0.557530612245, Validation Accuracy: 0.484\n",
      "(Iteration 43911 / 49000) loss: 1.22742128257\n",
      "(Iteration 43921 / 49000) loss: 1.30877559071\n",
      "(Iteration 43931 / 49000) loss: 1.20874110444\n",
      "(Iteration 43941 / 49000) loss: 1.21146337061\n",
      "(Iteration 43951 / 49000) loss: 1.29126594912\n",
      "(Iteration 43961 / 49000) loss: 1.2076739789\n",
      "(Iteration 43971 / 49000) loss: 1.22158176455\n",
      "(Iteration 43981 / 49000) loss: 1.28323496806\n",
      "(Iteration 43991 / 49000) loss: 1.30109791106\n",
      "(Iteration 44001 / 49000) loss: 1.24001276615\n",
      "(Epoch 449 / 500) Training Accuracy: 0.559265306122, Validation Accuracy: 0.475\n",
      "(Iteration 44011 / 49000) loss: 1.19405112114\n",
      "(Iteration 44021 / 49000) loss: 1.21765022406\n",
      "(Iteration 44031 / 49000) loss: 1.34786850437\n",
      "(Iteration 44041 / 49000) loss: 1.31148970225\n",
      "(Iteration 44051 / 49000) loss: 1.30585292402\n",
      "(Iteration 44061 / 49000) loss: 1.3247098615\n",
      "(Iteration 44071 / 49000) loss: 1.25594094539\n",
      "(Iteration 44081 / 49000) loss: 1.23667450496\n",
      "(Iteration 44091 / 49000) loss: 1.24230372731\n",
      "(Epoch 450 / 500) Training Accuracy: 0.555020408163, Validation Accuracy: 0.478\n",
      "(Iteration 44101 / 49000) loss: 1.23351751447\n",
      "(Iteration 44111 / 49000) loss: 1.38198903188\n",
      "(Iteration 44121 / 49000) loss: 1.28336865037\n",
      "(Iteration 44131 / 49000) loss: 1.26974690498\n",
      "(Iteration 44141 / 49000) loss: 1.24729576759\n",
      "(Iteration 44151 / 49000) loss: 1.24328231293\n",
      "(Iteration 44161 / 49000) loss: 1.387505612\n",
      "(Iteration 44171 / 49000) loss: 1.28536719944\n",
      "(Iteration 44181 / 49000) loss: 1.2954544307\n",
      "(Iteration 44191 / 49000) loss: 1.20180158532\n",
      "(Epoch 451 / 500) Training Accuracy: 0.553551020408, Validation Accuracy: 0.478\n",
      "(Iteration 44201 / 49000) loss: 1.21392382566\n",
      "(Iteration 44211 / 49000) loss: 1.3513580383\n",
      "(Iteration 44221 / 49000) loss: 1.34465365353\n",
      "(Iteration 44231 / 49000) loss: 1.21552587542\n",
      "(Iteration 44241 / 49000) loss: 1.19889057789\n",
      "(Iteration 44251 / 49000) loss: 1.37255437168\n",
      "(Iteration 44261 / 49000) loss: 1.27941229922\n",
      "(Iteration 44271 / 49000) loss: 1.15550621932\n",
      "(Iteration 44281 / 49000) loss: 1.29905510779\n",
      "(Iteration 44291 / 49000) loss: 1.25349544005\n",
      "(Epoch 452 / 500) Training Accuracy: 0.557204081633, Validation Accuracy: 0.484\n",
      "(Iteration 44301 / 49000) loss: 1.20514019655\n",
      "(Iteration 44311 / 49000) loss: 1.24784577132\n",
      "(Iteration 44321 / 49000) loss: 1.32093817273\n",
      "(Iteration 44331 / 49000) loss: 1.29711412794\n",
      "(Iteration 44341 / 49000) loss: 1.35549916755\n",
      "(Iteration 44351 / 49000) loss: 1.25094235611\n",
      "(Iteration 44361 / 49000) loss: 1.27681290166\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 44371 / 49000) loss: 1.21154470934\n",
      "(Iteration 44381 / 49000) loss: 1.32260627291\n",
      "(Iteration 44391 / 49000) loss: 1.26111624496\n",
      "(Epoch 453 / 500) Training Accuracy: 0.55512244898, Validation Accuracy: 0.474\n",
      "(Iteration 44401 / 49000) loss: 1.27002545024\n",
      "(Iteration 44411 / 49000) loss: 1.24755271153\n",
      "(Iteration 44421 / 49000) loss: 1.22661663732\n",
      "(Iteration 44431 / 49000) loss: 1.23248247285\n",
      "(Iteration 44441 / 49000) loss: 1.31905070097\n",
      "(Iteration 44451 / 49000) loss: 1.26070565072\n",
      "(Iteration 44461 / 49000) loss: 1.3075232012\n",
      "(Iteration 44471 / 49000) loss: 1.28524817209\n",
      "(Iteration 44481 / 49000) loss: 1.23595856125\n",
      "(Iteration 44491 / 49000) loss: 1.27807395726\n",
      "(Epoch 454 / 500) Training Accuracy: 0.558, Validation Accuracy: 0.474\n",
      "(Iteration 44501 / 49000) loss: 1.31527883279\n",
      "(Iteration 44511 / 49000) loss: 1.28002856771\n",
      "(Iteration 44521 / 49000) loss: 1.25663261078\n",
      "(Iteration 44531 / 49000) loss: 1.23389290769\n",
      "(Iteration 44541 / 49000) loss: 1.24826969798\n",
      "(Iteration 44551 / 49000) loss: 1.24857377526\n",
      "(Iteration 44561 / 49000) loss: 1.32947159805\n",
      "(Iteration 44571 / 49000) loss: 1.31175272223\n",
      "(Iteration 44581 / 49000) loss: 1.24828626843\n",
      "(Epoch 455 / 500) Training Accuracy: 0.558428571429, Validation Accuracy: 0.479\n",
      "(Iteration 44591 / 49000) loss: 1.23971663949\n",
      "(Iteration 44601 / 49000) loss: 1.34421661489\n",
      "(Iteration 44611 / 49000) loss: 1.31315646092\n",
      "(Iteration 44621 / 49000) loss: 1.20606844372\n",
      "(Iteration 44631 / 49000) loss: 1.25982854048\n",
      "(Iteration 44641 / 49000) loss: 1.23520993988\n",
      "(Iteration 44651 / 49000) loss: 1.28857528404\n",
      "(Iteration 44661 / 49000) loss: 1.24456760004\n",
      "(Iteration 44671 / 49000) loss: 1.22117908651\n",
      "(Iteration 44681 / 49000) loss: 1.27965400155\n",
      "(Epoch 456 / 500) Training Accuracy: 0.557897959184, Validation Accuracy: 0.477\n",
      "(Iteration 44691 / 49000) loss: 1.33358851354\n",
      "(Iteration 44701 / 49000) loss: 1.29078521575\n",
      "(Iteration 44711 / 49000) loss: 1.27024811712\n",
      "(Iteration 44721 / 49000) loss: 1.27130947811\n",
      "(Iteration 44731 / 49000) loss: 1.30062664071\n",
      "(Iteration 44741 / 49000) loss: 1.27350135636\n",
      "(Iteration 44751 / 49000) loss: 1.30812350794\n",
      "(Iteration 44761 / 49000) loss: 1.23798917893\n",
      "(Iteration 44771 / 49000) loss: 1.21042365649\n",
      "(Iteration 44781 / 49000) loss: 1.33733470543\n",
      "(Epoch 457 / 500) Training Accuracy: 0.558693877551, Validation Accuracy: 0.481\n",
      "(Iteration 44791 / 49000) loss: 1.27334176931\n",
      "(Iteration 44801 / 49000) loss: 1.33186366892\n",
      "(Iteration 44811 / 49000) loss: 1.30557129815\n",
      "(Iteration 44821 / 49000) loss: 1.25243065008\n",
      "(Iteration 44831 / 49000) loss: 1.39038049973\n",
      "(Iteration 44841 / 49000) loss: 1.25053542591\n",
      "(Iteration 44851 / 49000) loss: 1.29450482195\n",
      "(Iteration 44861 / 49000) loss: 1.23893843396\n",
      "(Iteration 44871 / 49000) loss: 1.31374995841\n",
      "(Iteration 44881 / 49000) loss: 1.25771915334\n",
      "(Epoch 458 / 500) Training Accuracy: 0.552326530612, Validation Accuracy: 0.481\n",
      "(Iteration 44891 / 49000) loss: 1.31758417715\n",
      "(Iteration 44901 / 49000) loss: 1.3477640689\n",
      "(Iteration 44911 / 49000) loss: 1.25943154091\n",
      "(Iteration 44921 / 49000) loss: 1.2878700458\n",
      "(Iteration 44931 / 49000) loss: 1.33156147754\n",
      "(Iteration 44941 / 49000) loss: 1.28538042781\n",
      "(Iteration 44951 / 49000) loss: 1.25097134276\n",
      "(Iteration 44961 / 49000) loss: 1.24563050542\n",
      "(Iteration 44971 / 49000) loss: 1.32435279025\n",
      "(Iteration 44981 / 49000) loss: 1.26310456083\n",
      "(Epoch 459 / 500) Training Accuracy: 0.561510204082, Validation Accuracy: 0.48\n",
      "(Iteration 44991 / 49000) loss: 1.22005349469\n",
      "(Iteration 45001 / 49000) loss: 1.31111547337\n",
      "(Iteration 45011 / 49000) loss: 1.36906407609\n",
      "(Iteration 45021 / 49000) loss: 1.29171029665\n",
      "(Iteration 45031 / 49000) loss: 1.27059277018\n",
      "(Iteration 45041 / 49000) loss: 1.27210154387\n",
      "(Iteration 45051 / 49000) loss: 1.2982581635\n",
      "(Iteration 45061 / 49000) loss: 1.27588625167\n",
      "(Iteration 45071 / 49000) loss: 1.20595940534\n",
      "(Epoch 460 / 500) Training Accuracy: 0.562857142857, Validation Accuracy: 0.481\n",
      "(Iteration 45081 / 49000) loss: 1.24136768223\n",
      "(Iteration 45091 / 49000) loss: 1.23456366243\n",
      "(Iteration 45101 / 49000) loss: 1.30300928005\n",
      "(Iteration 45111 / 49000) loss: 1.19021566849\n",
      "(Iteration 45121 / 49000) loss: 1.24413829646\n",
      "(Iteration 45131 / 49000) loss: 1.23988349454\n",
      "(Iteration 45141 / 49000) loss: 1.26100538281\n",
      "(Iteration 45151 / 49000) loss: 1.27906387469\n",
      "(Iteration 45161 / 49000) loss: 1.25387265702\n",
      "(Iteration 45171 / 49000) loss: 1.34883508044\n",
      "(Epoch 461 / 500) Training Accuracy: 0.556571428571, Validation Accuracy: 0.472\n",
      "(Iteration 45181 / 49000) loss: 1.3404995768\n",
      "(Iteration 45191 / 49000) loss: 1.22631191278\n",
      "(Iteration 45201 / 49000) loss: 1.24203765707\n",
      "(Iteration 45211 / 49000) loss: 1.17673785843\n",
      "(Iteration 45221 / 49000) loss: 1.30367011776\n",
      "(Iteration 45231 / 49000) loss: 1.30641360287\n",
      "(Iteration 45241 / 49000) loss: 1.2050552724\n",
      "(Iteration 45251 / 49000) loss: 1.33832726846\n",
      "(Iteration 45261 / 49000) loss: 1.26268083766\n",
      "(Iteration 45271 / 49000) loss: 1.29239135497\n",
      "(Epoch 462 / 500) Training Accuracy: 0.56087755102, Validation Accuracy: 0.478\n",
      "(Iteration 45281 / 49000) loss: 1.35320023744\n",
      "(Iteration 45291 / 49000) loss: 1.28640693661\n",
      "(Iteration 45301 / 49000) loss: 1.26032107787\n",
      "(Iteration 45311 / 49000) loss: 1.21138100548\n",
      "(Iteration 45321 / 49000) loss: 1.32633054908\n",
      "(Iteration 45331 / 49000) loss: 1.29195388623\n",
      "(Iteration 45341 / 49000) loss: 1.27756376979\n",
      "(Iteration 45351 / 49000) loss: 1.34440643406\n",
      "(Iteration 45361 / 49000) loss: 1.25065746069\n",
      "(Iteration 45371 / 49000) loss: 1.1906322285\n",
      "(Epoch 463 / 500) Training Accuracy: 0.560795918367, Validation Accuracy: 0.481\n",
      "(Iteration 45381 / 49000) loss: 1.2948225872\n",
      "(Iteration 45391 / 49000) loss: 1.23799167085\n",
      "(Iteration 45401 / 49000) loss: 1.30697329629\n",
      "(Iteration 45411 / 49000) loss: 1.27718993366\n",
      "(Iteration 45421 / 49000) loss: 1.21702462144\n",
      "(Iteration 45431 / 49000) loss: 1.35081473735\n",
      "(Iteration 45441 / 49000) loss: 1.32374309573\n",
      "(Iteration 45451 / 49000) loss: 1.30574698692\n",
      "(Iteration 45461 / 49000) loss: 1.31648283662\n",
      "(Iteration 45471 / 49000) loss: 1.32966972697\n",
      "(Epoch 464 / 500) Training Accuracy: 0.560224489796, Validation Accuracy: 0.483\n",
      "(Iteration 45481 / 49000) loss: 1.25083456613\n",
      "(Iteration 45491 / 49000) loss: 1.2603972851\n",
      "(Iteration 45501 / 49000) loss: 1.1364227251\n",
      "(Iteration 45511 / 49000) loss: 1.26491252939\n",
      "(Iteration 45521 / 49000) loss: 1.32928069528\n",
      "(Iteration 45531 / 49000) loss: 1.29634423182\n",
      "(Iteration 45541 / 49000) loss: 1.33913060823\n",
      "(Iteration 45551 / 49000) loss: 1.25805168479\n",
      "(Iteration 45561 / 49000) loss: 1.27046771069\n",
      "(Epoch 465 / 500) Training Accuracy: 0.553081632653, Validation Accuracy: 0.479\n",
      "(Iteration 45571 / 49000) loss: 1.3087116672\n",
      "(Iteration 45581 / 49000) loss: 1.13900859775\n",
      "(Iteration 45591 / 49000) loss: 1.24880935279\n",
      "(Iteration 45601 / 49000) loss: 1.21735318139\n",
      "(Iteration 45611 / 49000) loss: 1.23397547782\n",
      "(Iteration 45621 / 49000) loss: 1.30089921419\n",
      "(Iteration 45631 / 49000) loss: 1.30546478357\n",
      "(Iteration 45641 / 49000) loss: 1.24711848661\n",
      "(Iteration 45651 / 49000) loss: 1.26561862266\n",
      "(Iteration 45661 / 49000) loss: 1.3131341074\n",
      "(Epoch 466 / 500) Training Accuracy: 0.557714285714, Validation Accuracy: 0.478\n",
      "(Iteration 45671 / 49000) loss: 1.23728058501\n",
      "(Iteration 45681 / 49000) loss: 1.29214962884\n",
      "(Iteration 45691 / 49000) loss: 1.34674892117\n",
      "(Iteration 45701 / 49000) loss: 1.25571946053\n",
      "(Iteration 45711 / 49000) loss: 1.29620307795\n",
      "(Iteration 45721 / 49000) loss: 1.25556672051\n",
      "(Iteration 45731 / 49000) loss: 1.32548647101\n",
      "(Iteration 45741 / 49000) loss: 1.30241504393\n",
      "(Iteration 45751 / 49000) loss: 1.10682807525\n",
      "(Iteration 45761 / 49000) loss: 1.24268087804\n",
      "(Epoch 467 / 500) Training Accuracy: 0.557836734694, Validation Accuracy: 0.486\n",
      "(Iteration 45771 / 49000) loss: 1.22193228763\n",
      "(Iteration 45781 / 49000) loss: 1.28797935755\n",
      "(Iteration 45791 / 49000) loss: 1.25175347388\n",
      "(Iteration 45801 / 49000) loss: 1.199998432\n",
      "(Iteration 45811 / 49000) loss: 1.21566768802\n",
      "(Iteration 45821 / 49000) loss: 1.29571367152\n",
      "(Iteration 45831 / 49000) loss: 1.32983125779\n",
      "(Iteration 45841 / 49000) loss: 1.20150097371\n",
      "(Iteration 45851 / 49000) loss: 1.25040727816\n",
      "(Iteration 45861 / 49000) loss: 1.20166971136\n",
      "(Epoch 468 / 500) Training Accuracy: 0.556510204082, Validation Accuracy: 0.475\n",
      "(Iteration 45871 / 49000) loss: 1.20782429759\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 45881 / 49000) loss: 1.27278524565\n",
      "(Iteration 45891 / 49000) loss: 1.2521123792\n",
      "(Iteration 45901 / 49000) loss: 1.2552810641\n",
      "(Iteration 45911 / 49000) loss: 1.18423780086\n",
      "(Iteration 45921 / 49000) loss: 1.24835274847\n",
      "(Iteration 45931 / 49000) loss: 1.24472226957\n",
      "(Iteration 45941 / 49000) loss: 1.24914601795\n",
      "(Iteration 45951 / 49000) loss: 1.20022677357\n",
      "(Iteration 45961 / 49000) loss: 1.27643199567\n",
      "(Epoch 469 / 500) Training Accuracy: 0.561816326531, Validation Accuracy: 0.476\n",
      "(Iteration 45971 / 49000) loss: 1.31591893477\n",
      "(Iteration 45981 / 49000) loss: 1.28466665837\n",
      "(Iteration 45991 / 49000) loss: 1.25526026901\n",
      "(Iteration 46001 / 49000) loss: 1.26786296643\n",
      "(Iteration 46011 / 49000) loss: 1.29903007588\n",
      "(Iteration 46021 / 49000) loss: 1.18583999009\n",
      "(Iteration 46031 / 49000) loss: 1.36163314123\n",
      "(Iteration 46041 / 49000) loss: 1.33492181792\n",
      "(Iteration 46051 / 49000) loss: 1.293865245\n",
      "(Epoch 470 / 500) Training Accuracy: 0.561795918367, Validation Accuracy: 0.483\n",
      "(Iteration 46061 / 49000) loss: 1.25726219124\n",
      "(Iteration 46071 / 49000) loss: 1.40661902748\n",
      "(Iteration 46081 / 49000) loss: 1.30879704206\n",
      "(Iteration 46091 / 49000) loss: 1.30151130803\n",
      "(Iteration 46101 / 49000) loss: 1.28939144977\n",
      "(Iteration 46111 / 49000) loss: 1.27037769109\n",
      "(Iteration 46121 / 49000) loss: 1.24129753801\n",
      "(Iteration 46131 / 49000) loss: 1.20586528192\n",
      "(Iteration 46141 / 49000) loss: 1.27146699632\n",
      "(Iteration 46151 / 49000) loss: 1.27964041132\n",
      "(Epoch 471 / 500) Training Accuracy: 0.562551020408, Validation Accuracy: 0.474\n",
      "(Iteration 46161 / 49000) loss: 1.19484930734\n",
      "(Iteration 46171 / 49000) loss: 1.22980221529\n",
      "(Iteration 46181 / 49000) loss: 1.20548209256\n",
      "(Iteration 46191 / 49000) loss: 1.18868653947\n",
      "(Iteration 46201 / 49000) loss: 1.26894507334\n",
      "(Iteration 46211 / 49000) loss: 1.15790072156\n",
      "(Iteration 46221 / 49000) loss: 1.2306376079\n",
      "(Iteration 46231 / 49000) loss: 1.26811785585\n",
      "(Iteration 46241 / 49000) loss: 1.35850694881\n",
      "(Iteration 46251 / 49000) loss: 1.36585219449\n",
      "(Epoch 472 / 500) Training Accuracy: 0.562, Validation Accuracy: 0.481\n",
      "(Iteration 46261 / 49000) loss: 1.25753915193\n",
      "(Iteration 46271 / 49000) loss: 1.17011325995\n",
      "(Iteration 46281 / 49000) loss: 1.30230161322\n",
      "(Iteration 46291 / 49000) loss: 1.25901024515\n",
      "(Iteration 46301 / 49000) loss: 1.24080963033\n",
      "(Iteration 46311 / 49000) loss: 1.2522824771\n",
      "(Iteration 46321 / 49000) loss: 1.23901548679\n",
      "(Iteration 46331 / 49000) loss: 1.21556518363\n",
      "(Iteration 46341 / 49000) loss: 1.25549687894\n",
      "(Iteration 46351 / 49000) loss: 1.26290507364\n",
      "(Epoch 473 / 500) Training Accuracy: 0.564612244898, Validation Accuracy: 0.479\n",
      "(Iteration 46361 / 49000) loss: 1.25845237103\n",
      "(Iteration 46371 / 49000) loss: 1.25431817979\n",
      "(Iteration 46381 / 49000) loss: 1.24518500678\n",
      "(Iteration 46391 / 49000) loss: 1.27670086211\n",
      "(Iteration 46401 / 49000) loss: 1.21619392445\n",
      "(Iteration 46411 / 49000) loss: 1.27932198929\n",
      "(Iteration 46421 / 49000) loss: 1.22150102232\n",
      "(Iteration 46431 / 49000) loss: 1.28912094766\n",
      "(Iteration 46441 / 49000) loss: 1.23578311321\n",
      "(Iteration 46451 / 49000) loss: 1.25161087137\n",
      "(Epoch 474 / 500) Training Accuracy: 0.561918367347, Validation Accuracy: 0.476\n",
      "(Iteration 46461 / 49000) loss: 1.30168656893\n",
      "(Iteration 46471 / 49000) loss: 1.29293975867\n",
      "(Iteration 46481 / 49000) loss: 1.20212014994\n",
      "(Iteration 46491 / 49000) loss: 1.29663839936\n",
      "(Iteration 46501 / 49000) loss: 1.35376406231\n",
      "(Iteration 46511 / 49000) loss: 1.26512739123\n",
      "(Iteration 46521 / 49000) loss: 1.33000205679\n",
      "(Iteration 46531 / 49000) loss: 1.3047948723\n",
      "(Iteration 46541 / 49000) loss: 1.26042894169\n",
      "(Epoch 475 / 500) Training Accuracy: 0.559367346939, Validation Accuracy: 0.479\n",
      "(Iteration 46551 / 49000) loss: 1.30191087898\n",
      "(Iteration 46561 / 49000) loss: 1.19184326297\n",
      "(Iteration 46571 / 49000) loss: 1.2597106895\n",
      "(Iteration 46581 / 49000) loss: 1.32432222316\n",
      "(Iteration 46591 / 49000) loss: 1.28707204535\n",
      "(Iteration 46601 / 49000) loss: 1.26656259183\n",
      "(Iteration 46611 / 49000) loss: 1.17426496622\n",
      "(Iteration 46621 / 49000) loss: 1.35004342822\n",
      "(Iteration 46631 / 49000) loss: 1.3359647987\n",
      "(Iteration 46641 / 49000) loss: 1.31160169719\n",
      "(Epoch 476 / 500) Training Accuracy: 0.563306122449, Validation Accuracy: 0.476\n",
      "(Iteration 46651 / 49000) loss: 1.32631468954\n",
      "(Iteration 46661 / 49000) loss: 1.21436280005\n",
      "(Iteration 46671 / 49000) loss: 1.27607403726\n",
      "(Iteration 46681 / 49000) loss: 1.22703376787\n",
      "(Iteration 46691 / 49000) loss: 1.31887101106\n",
      "(Iteration 46701 / 49000) loss: 1.26454662864\n",
      "(Iteration 46711 / 49000) loss: 1.26641180978\n",
      "(Iteration 46721 / 49000) loss: 1.23584024198\n",
      "(Iteration 46731 / 49000) loss: 1.26299037978\n",
      "(Iteration 46741 / 49000) loss: 1.31605415057\n",
      "(Epoch 477 / 500) Training Accuracy: 0.564979591837, Validation Accuracy: 0.474\n",
      "(Iteration 46751 / 49000) loss: 1.38077885048\n",
      "(Iteration 46761 / 49000) loss: 1.1802308205\n",
      "(Iteration 46771 / 49000) loss: 1.20293949344\n",
      "(Iteration 46781 / 49000) loss: 1.29246894716\n",
      "(Iteration 46791 / 49000) loss: 1.22949777801\n",
      "(Iteration 46801 / 49000) loss: 1.22193678284\n",
      "(Iteration 46811 / 49000) loss: 1.30831890244\n",
      "(Iteration 46821 / 49000) loss: 1.21463534602\n",
      "(Iteration 46831 / 49000) loss: 1.23460343312\n",
      "(Iteration 46841 / 49000) loss: 1.28560640328\n",
      "(Epoch 478 / 500) Training Accuracy: 0.565469387755, Validation Accuracy: 0.489\n",
      "(Iteration 46851 / 49000) loss: 1.38632096885\n",
      "(Iteration 46861 / 49000) loss: 1.25973048887\n",
      "(Iteration 46871 / 49000) loss: 1.27668631755\n",
      "(Iteration 46881 / 49000) loss: 1.2790219964\n",
      "(Iteration 46891 / 49000) loss: 1.23664709707\n",
      "(Iteration 46901 / 49000) loss: 1.24785585236\n",
      "(Iteration 46911 / 49000) loss: 1.31993428212\n",
      "(Iteration 46921 / 49000) loss: 1.31189744756\n",
      "(Iteration 46931 / 49000) loss: 1.32047801645\n",
      "(Iteration 46941 / 49000) loss: 1.27883751658\n",
      "(Epoch 479 / 500) Training Accuracy: 0.563306122449, Validation Accuracy: 0.481\n",
      "(Iteration 46951 / 49000) loss: 1.27365260537\n",
      "(Iteration 46961 / 49000) loss: 1.20926528434\n",
      "(Iteration 46971 / 49000) loss: 1.3012459814\n",
      "(Iteration 46981 / 49000) loss: 1.28020927181\n",
      "(Iteration 46991 / 49000) loss: 1.22183335832\n",
      "(Iteration 47001 / 49000) loss: 1.33856194696\n",
      "(Iteration 47011 / 49000) loss: 1.24597315719\n",
      "(Iteration 47021 / 49000) loss: 1.26568454258\n",
      "(Iteration 47031 / 49000) loss: 1.21210969636\n",
      "(Epoch 480 / 500) Training Accuracy: 0.566795918367, Validation Accuracy: 0.484\n",
      "(Iteration 47041 / 49000) loss: 1.29402581795\n",
      "(Iteration 47051 / 49000) loss: 1.15503969711\n",
      "(Iteration 47061 / 49000) loss: 1.17344043133\n",
      "(Iteration 47071 / 49000) loss: 1.31666424665\n",
      "(Iteration 47081 / 49000) loss: 1.29946361577\n",
      "(Iteration 47091 / 49000) loss: 1.31368709388\n",
      "(Iteration 47101 / 49000) loss: 1.31282633012\n",
      "(Iteration 47111 / 49000) loss: 1.27210469623\n",
      "(Iteration 47121 / 49000) loss: 1.2059642604\n",
      "(Iteration 47131 / 49000) loss: 1.24209632975\n",
      "(Epoch 481 / 500) Training Accuracy: 0.562632653061, Validation Accuracy: 0.481\n",
      "(Iteration 47141 / 49000) loss: 1.26332325204\n",
      "(Iteration 47151 / 49000) loss: 1.31979392239\n",
      "(Iteration 47161 / 49000) loss: 1.23742045759\n",
      "(Iteration 47171 / 49000) loss: 1.22020592016\n",
      "(Iteration 47181 / 49000) loss: 1.31119808932\n",
      "(Iteration 47191 / 49000) loss: 1.32339521522\n",
      "(Iteration 47201 / 49000) loss: 1.25132896918\n",
      "(Iteration 47211 / 49000) loss: 1.31369184784\n",
      "(Iteration 47221 / 49000) loss: 1.19941787905\n",
      "(Iteration 47231 / 49000) loss: 1.28296823869\n",
      "(Epoch 482 / 500) Training Accuracy: 0.553204081633, Validation Accuracy: 0.49\n",
      "(Iteration 47241 / 49000) loss: 1.32291168022\n",
      "(Iteration 47251 / 49000) loss: 1.25074252165\n",
      "(Iteration 47261 / 49000) loss: 1.30624358505\n",
      "(Iteration 47271 / 49000) loss: 1.2281492665\n",
      "(Iteration 47281 / 49000) loss: 1.16526483711\n",
      "(Iteration 47291 / 49000) loss: 1.21660943836\n",
      "(Iteration 47301 / 49000) loss: 1.17465700404\n",
      "(Iteration 47311 / 49000) loss: 1.31209009337\n",
      "(Iteration 47321 / 49000) loss: 1.24171975491\n",
      "(Iteration 47331 / 49000) loss: 1.24106529266\n",
      "(Epoch 483 / 500) Training Accuracy: 0.563755102041, Validation Accuracy: 0.481\n",
      "(Iteration 47341 / 49000) loss: 1.24523777568\n",
      "(Iteration 47351 / 49000) loss: 1.17955254956\n",
      "(Iteration 47361 / 49000) loss: 1.25382828104\n",
      "(Iteration 47371 / 49000) loss: 1.21051759047\n",
      "(Iteration 47381 / 49000) loss: 1.27833416642\n",
      "(Iteration 47391 / 49000) loss: 1.23270445879\n",
      "(Iteration 47401 / 49000) loss: 1.26782068263\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 47411 / 49000) loss: 1.21157578046\n",
      "(Iteration 47421 / 49000) loss: 1.30818300078\n",
      "(Iteration 47431 / 49000) loss: 1.19751875445\n",
      "(Epoch 484 / 500) Training Accuracy: 0.562081632653, Validation Accuracy: 0.488\n",
      "(Iteration 47441 / 49000) loss: 1.26470001314\n",
      "(Iteration 47451 / 49000) loss: 1.16269572659\n",
      "(Iteration 47461 / 49000) loss: 1.23508777934\n",
      "(Iteration 47471 / 49000) loss: 1.2958537831\n",
      "(Iteration 47481 / 49000) loss: 1.27167274067\n",
      "(Iteration 47491 / 49000) loss: 1.24325751667\n",
      "(Iteration 47501 / 49000) loss: 1.35818120712\n",
      "(Iteration 47511 / 49000) loss: 1.20987039196\n",
      "(Iteration 47521 / 49000) loss: 1.2740825023\n",
      "(Epoch 485 / 500) Training Accuracy: 0.563346938776, Validation Accuracy: 0.482\n",
      "(Iteration 47531 / 49000) loss: 1.20770429205\n",
      "(Iteration 47541 / 49000) loss: 1.31732085208\n",
      "(Iteration 47551 / 49000) loss: 1.22774439393\n",
      "(Iteration 47561 / 49000) loss: 1.17273849423\n",
      "(Iteration 47571 / 49000) loss: 1.26026217007\n",
      "(Iteration 47581 / 49000) loss: 1.26966172559\n",
      "(Iteration 47591 / 49000) loss: 1.32536112662\n",
      "(Iteration 47601 / 49000) loss: 1.32993476759\n",
      "(Iteration 47611 / 49000) loss: 1.22176595518\n",
      "(Iteration 47621 / 49000) loss: 1.28688822845\n",
      "(Epoch 486 / 500) Training Accuracy: 0.561265306122, Validation Accuracy: 0.479\n",
      "(Iteration 47631 / 49000) loss: 1.24220649749\n",
      "(Iteration 47641 / 49000) loss: 1.22355480341\n",
      "(Iteration 47651 / 49000) loss: 1.2724600272\n",
      "(Iteration 47661 / 49000) loss: 1.26529570372\n",
      "(Iteration 47671 / 49000) loss: 1.24371261863\n",
      "(Iteration 47681 / 49000) loss: 1.20513413185\n",
      "(Iteration 47691 / 49000) loss: 1.34410023719\n",
      "(Iteration 47701 / 49000) loss: 1.24692254626\n",
      "(Iteration 47711 / 49000) loss: 1.29473025031\n",
      "(Iteration 47721 / 49000) loss: 1.27083328537\n",
      "(Epoch 487 / 500) Training Accuracy: 0.564632653061, Validation Accuracy: 0.485\n",
      "(Iteration 47731 / 49000) loss: 1.22625090683\n",
      "(Iteration 47741 / 49000) loss: 1.27751147419\n",
      "(Iteration 47751 / 49000) loss: 1.216427448\n",
      "(Iteration 47761 / 49000) loss: 1.21584581246\n",
      "(Iteration 47771 / 49000) loss: 1.27919887629\n",
      "(Iteration 47781 / 49000) loss: 1.22525963139\n",
      "(Iteration 47791 / 49000) loss: 1.27688552003\n",
      "(Iteration 47801 / 49000) loss: 1.23943133737\n",
      "(Iteration 47811 / 49000) loss: 1.19859676946\n",
      "(Iteration 47821 / 49000) loss: 1.25961246712\n",
      "(Epoch 488 / 500) Training Accuracy: 0.56087755102, Validation Accuracy: 0.48\n",
      "(Iteration 47831 / 49000) loss: 1.21354743478\n",
      "(Iteration 47841 / 49000) loss: 1.21780423284\n",
      "(Iteration 47851 / 49000) loss: 1.29862194237\n",
      "(Iteration 47861 / 49000) loss: 1.22213196473\n",
      "(Iteration 47871 / 49000) loss: 1.26246659129\n",
      "(Iteration 47881 / 49000) loss: 1.24327853677\n",
      "(Iteration 47891 / 49000) loss: 1.28104000124\n",
      "(Iteration 47901 / 49000) loss: 1.24876414709\n",
      "(Iteration 47911 / 49000) loss: 1.3040362397\n",
      "(Iteration 47921 / 49000) loss: 1.19128766587\n",
      "(Epoch 489 / 500) Training Accuracy: 0.567387755102, Validation Accuracy: 0.483\n",
      "(Iteration 47931 / 49000) loss: 1.33971917941\n",
      "(Iteration 47941 / 49000) loss: 1.19260936667\n",
      "(Iteration 47951 / 49000) loss: 1.23546731987\n",
      "(Iteration 47961 / 49000) loss: 1.32956508182\n",
      "(Iteration 47971 / 49000) loss: 1.19877222169\n",
      "(Iteration 47981 / 49000) loss: 1.26149086374\n",
      "(Iteration 47991 / 49000) loss: 1.28339132094\n",
      "(Iteration 48001 / 49000) loss: 1.2346085056\n",
      "(Iteration 48011 / 49000) loss: 1.15833458659\n",
      "(Epoch 490 / 500) Training Accuracy: 0.562265306122, Validation Accuracy: 0.483\n",
      "(Iteration 48021 / 49000) loss: 1.25748565887\n",
      "(Iteration 48031 / 49000) loss: 1.31632015889\n",
      "(Iteration 48041 / 49000) loss: 1.29175737015\n",
      "(Iteration 48051 / 49000) loss: 1.26836633913\n",
      "(Iteration 48061 / 49000) loss: 1.23834454666\n",
      "(Iteration 48071 / 49000) loss: 1.25314480953\n",
      "(Iteration 48081 / 49000) loss: 1.27677722074\n",
      "(Iteration 48091 / 49000) loss: 1.28410854206\n",
      "(Iteration 48101 / 49000) loss: 1.18856133518\n",
      "(Iteration 48111 / 49000) loss: 1.19984609972\n",
      "(Epoch 491 / 500) Training Accuracy: 0.560244897959, Validation Accuracy: 0.472\n",
      "(Iteration 48121 / 49000) loss: 1.2002876235\n",
      "(Iteration 48131 / 49000) loss: 1.25438753842\n",
      "(Iteration 48141 / 49000) loss: 1.22619696353\n",
      "(Iteration 48151 / 49000) loss: 1.21205037294\n",
      "(Iteration 48161 / 49000) loss: 1.15323416633\n",
      "(Iteration 48171 / 49000) loss: 1.30339099647\n",
      "(Iteration 48181 / 49000) loss: 1.2630574501\n",
      "(Iteration 48191 / 49000) loss: 1.32716100825\n",
      "(Iteration 48201 / 49000) loss: 1.20206829287\n",
      "(Iteration 48211 / 49000) loss: 1.29022860121\n",
      "(Epoch 492 / 500) Training Accuracy: 0.567102040816, Validation Accuracy: 0.48\n",
      "(Iteration 48221 / 49000) loss: 1.29096959271\n",
      "(Iteration 48231 / 49000) loss: 1.24832980759\n",
      "(Iteration 48241 / 49000) loss: 1.27341457626\n",
      "(Iteration 48251 / 49000) loss: 1.22964953587\n",
      "(Iteration 48261 / 49000) loss: 1.22153589762\n",
      "(Iteration 48271 / 49000) loss: 1.35304254087\n",
      "(Iteration 48281 / 49000) loss: 1.20900240613\n",
      "(Iteration 48291 / 49000) loss: 1.10467199368\n",
      "(Iteration 48301 / 49000) loss: 1.25525356637\n",
      "(Iteration 48311 / 49000) loss: 1.26574875156\n",
      "(Epoch 493 / 500) Training Accuracy: 0.566244897959, Validation Accuracy: 0.482\n",
      "(Iteration 48321 / 49000) loss: 1.21384460883\n",
      "(Iteration 48331 / 49000) loss: 1.20040925961\n",
      "(Iteration 48341 / 49000) loss: 1.22595013417\n",
      "(Iteration 48351 / 49000) loss: 1.33474696994\n",
      "(Iteration 48361 / 49000) loss: 1.20769428354\n",
      "(Iteration 48371 / 49000) loss: 1.23969270854\n",
      "(Iteration 48381 / 49000) loss: 1.18115414857\n",
      "(Iteration 48391 / 49000) loss: 1.26975540575\n",
      "(Iteration 48401 / 49000) loss: 1.20972624486\n",
      "(Iteration 48411 / 49000) loss: 1.18975979732\n",
      "(Epoch 494 / 500) Training Accuracy: 0.565612244898, Validation Accuracy: 0.478\n",
      "(Iteration 48421 / 49000) loss: 1.25921119617\n",
      "(Iteration 48431 / 49000) loss: 1.20429199089\n",
      "(Iteration 48441 / 49000) loss: 1.21641447811\n",
      "(Iteration 48451 / 49000) loss: 1.29523869624\n",
      "(Iteration 48461 / 49000) loss: 1.37085747304\n",
      "(Iteration 48471 / 49000) loss: 1.37596660199\n",
      "(Iteration 48481 / 49000) loss: 1.29683913366\n",
      "(Iteration 48491 / 49000) loss: 1.21954883132\n",
      "(Iteration 48501 / 49000) loss: 1.23602540086\n",
      "(Epoch 495 / 500) Training Accuracy: 0.564897959184, Validation Accuracy: 0.489\n",
      "(Iteration 48511 / 49000) loss: 1.27567501031\n",
      "(Iteration 48521 / 49000) loss: 1.32119527217\n",
      "(Iteration 48531 / 49000) loss: 1.32496401166\n",
      "(Iteration 48541 / 49000) loss: 1.23903050567\n",
      "(Iteration 48551 / 49000) loss: 1.22958056598\n",
      "(Iteration 48561 / 49000) loss: 1.28954818815\n",
      "(Iteration 48571 / 49000) loss: 1.26813900855\n",
      "(Iteration 48581 / 49000) loss: 1.26073064128\n",
      "(Iteration 48591 / 49000) loss: 1.18122165794\n",
      "(Iteration 48601 / 49000) loss: 1.23630968046\n",
      "(Epoch 496 / 500) Training Accuracy: 0.566183673469, Validation Accuracy: 0.479\n",
      "(Iteration 48611 / 49000) loss: 1.18022459139\n",
      "(Iteration 48621 / 49000) loss: 1.2958269061\n",
      "(Iteration 48631 / 49000) loss: 1.21029578261\n",
      "(Iteration 48641 / 49000) loss: 1.18159401566\n",
      "(Iteration 48651 / 49000) loss: 1.23058075484\n",
      "(Iteration 48661 / 49000) loss: 1.32867817417\n",
      "(Iteration 48671 / 49000) loss: 1.3100270582\n",
      "(Iteration 48681 / 49000) loss: 1.18506604843\n",
      "(Iteration 48691 / 49000) loss: 1.34215006997\n",
      "(Iteration 48701 / 49000) loss: 1.25636449269\n",
      "(Epoch 497 / 500) Training Accuracy: 0.566244897959, Validation Accuracy: 0.48\n",
      "(Iteration 48711 / 49000) loss: 1.24035147102\n",
      "(Iteration 48721 / 49000) loss: 1.27668692947\n",
      "(Iteration 48731 / 49000) loss: 1.21995984703\n",
      "(Iteration 48741 / 49000) loss: 1.25042794124\n",
      "(Iteration 48751 / 49000) loss: 1.25569704197\n",
      "(Iteration 48761 / 49000) loss: 1.17913480962\n",
      "(Iteration 48771 / 49000) loss: 1.38009726261\n",
      "(Iteration 48781 / 49000) loss: 1.26318552222\n",
      "(Iteration 48791 / 49000) loss: 1.2999872963\n",
      "(Iteration 48801 / 49000) loss: 1.3058023725\n",
      "(Epoch 498 / 500) Training Accuracy: 0.568183673469, Validation Accuracy: 0.482\n",
      "(Iteration 48811 / 49000) loss: 1.25732226242\n",
      "(Iteration 48821 / 49000) loss: 1.29487037147\n",
      "(Iteration 48831 / 49000) loss: 1.2803997943\n",
      "(Iteration 48841 / 49000) loss: 1.18874112605\n",
      "(Iteration 48851 / 49000) loss: 1.19309206482\n",
      "(Iteration 48861 / 49000) loss: 1.1684403123\n",
      "(Iteration 48871 / 49000) loss: 1.23618533779\n",
      "(Iteration 48881 / 49000) loss: 1.34946892025\n",
      "(Iteration 48891 / 49000) loss: 1.27090157746\n",
      "(Iteration 48901 / 49000) loss: 1.28219979333\n",
      "(Epoch 499 / 500) Training Accuracy: 0.567183673469, Validation Accuracy: 0.478\n",
      "(Iteration 48911 / 49000) loss: 1.18981731085\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 48921 / 49000) loss: 1.33859998673\n",
      "(Iteration 48931 / 49000) loss: 1.26550775642\n",
      "(Iteration 48941 / 49000) loss: 1.19227758607\n",
      "(Iteration 48951 / 49000) loss: 1.29993956689\n",
      "(Iteration 48961 / 49000) loss: 1.32247498868\n",
      "(Iteration 48971 / 49000) loss: 1.31565604005\n",
      "(Iteration 48981 / 49000) loss: 1.28303968698\n",
      "(Iteration 48991 / 49000) loss: 1.31215456436\n",
      "(Epoch 500 / 500) Training Accuracy: 0.566081632653, Validation Accuracy: 0.483\n"
     ]
    }
   ],
   "source": [
    "results = None\n",
    "#############################################################################\n",
    "# TODO: Use the train_net function you completed to train a network         #\n",
    "#############################################################################\n",
    "results = train_net(data_dict, model, loss_f, optimizer, 500, 500, verbose=True)\n",
    "#############################################################################\n",
    "#                             END OF YOUR CODE                              #\n",
    "#############################################################################\n",
    "opt_params, loss_hist, train_acc_hist, val_acc_hist = results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 276,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['fc1_w', 'fc2_b', 'fc1_b', 'fc2_w']\n"
     ]
    }
   ],
   "source": [
    "# Take a look at what names of params were stored\n",
    "print opt_params.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 277,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading Params: fc1_w Shape: (3072, 200)\n",
      "Loading Params: fc1_b Shape: (200,)\n",
      "Loading Params: fc2_b Shape: (10,)\n",
      "Loading Params: fc2_w Shape: (200, 10)\n",
      "Validation Accuracy: 48.3%\n",
      "Testing Accuracy: 47.7%\n"
     ]
    }
   ],
   "source": [
    "# Demo: How to load the parameters to a newly defined network\n",
    "model = TinyNet()\n",
    "model.net.load(opt_params)\n",
    "val_acc = compute_acc(model, data[\"data_val\"], data[\"labels_val\"])\n",
    "print \"Validation Accuracy: {}%\".format(val_acc*100)\n",
    "test_acc = compute_acc(model, data[\"data_test\"], data[\"labels_test\"])\n",
    "print \"Testing Accuracy: {}%\".format(test_acc*100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 278,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3QAAALJCAYAAAD8s2GkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3XucHGWd7/Hvr3s6SU9AJpBBZQDDuhAQ0UQji+IFRA0qYA54uCgKu+6Jx11d4WBcwqKgspI1KHjWsx5ZZMMRxCDECLgaL8B6QS4TEwgI8cYtEyQDyXCbIZnL7/zR1UNNT3V39Ux1d/XM5/165TXTVdVVT/cUzHz7+T3PY+4uAAAAAEDryTS7AQAAAACAiSHQAQAAAECLItABAAAAQIsi0AEAAABAiyLQAQAAAECLItABAAAAQIsi0AEApgwzy5rZc2a2f5LHTqAdF5nZqqTPCwBAqbZmNwAAMH2Z2XOhh+2SdkoaDh5/1N2vqeV87j4sabekjwUAIK0IdACApnH30UBlZg9L+lt3/2m5482szd2HGtE2AABaASWXAIDUCkoXV5vZtWb2rKTTzeyNZnaHmfWZ2eNm9r/NLBcc32ZmbmbzgsdXB/t/aGbPmtmvzeyAWo8N9r/bzH5nZk+b2b+a2a/M7MyYr2OJmd0ftPkWM5sf2neemW01s2fM7EEzOyrYfoSZ/SbY/oSZrUzgLQUATDEEOgBA2v03Sd+WtIek1ZKGJH1S0lxJR0o6VtJHKzz/A5I+I2lPSY9K+kKtx5rZ3pKuk7QsuO5Dkg6P03gzO0TS1ZI+IalT0k8l3WRmOTM7NGj769z9JZLeHVxXkv5V0spg+19Kuj7O9QAA0wuBDgCQdr9095vcfcTdB9z9bne/092H3P1Pki6X9LYKz7/e3bvdfVDSNZIWTODY4yRtdPfvB/sulfRkzPafKulGd78leO4KSS+R9FcqhNNZkg4NykkfCl6TJA1KOtDM9nL3Z939zpjXAwBMIwQ6AEDaPRZ+YGYHm9kPzOzPZvaMpM+r0GtWzp9D3/er8kQo5Y7dJ9wOd3dJW2K0vfjcR0LPHQme2+XumyWdo8Jr2BaUlr4sOPSvJb1K0mYzu8vM3hPzegCAaYRABwBIOy95/A1J90n6y6Ac8bOSrM5teFzSvsUHZmaSumI+d6ukV4SemwnO1SNJ7n61ux8p6QBJWUkXB9s3u/upkvaW9GVJN5jZrMm/FADAVEKgAwC0mt0lPS3p+WB8WqXxc0m5WdLrzOx4M2tTYQxfZ8znXifpBDM7Kpi8ZZmkZyXdaWaHmNnRZjZT0kDwb1iSzOxDZjY36NF7WoVgO5LsywIAtDoCHQCg1Zwj6QwVQtE3VJgopa7c/QlJp0j6iqSnJL1S0gYV1s2r9tz7VWjv1yX1qjCJywnBeLqZkr6kwni8P0uaI+n84KnvkfRAMLvnJZJOcfddCb4sAMAUYIVhAAAAIC4zy6pQSvl+d/9Fs9sDAJi+6KEDACAGMzvWzPYIyiM/o8IMlXc1uVkAgGmOQAcAQDxvlvQnFcojj5W0xN2rllwCAFBPlFwCAAAAQIuihw4AAAAAWlRbsxtQau7cuT5v3rxmNwMAAAAAmmL9+vVPunus5XESC3RmdqWk4yRtc/dXB9sWSPq/kmapMHj879y94gDyefPmqbu7O6lmAQAAAEBLMbNH4h6bZMnlKhUGiYd9SdLn3H2BpM8GjwEAAAAACUgs0Ln7zyVtL90s6SXB93uosGYPAAAAACAB9R5Dd5akdWZ2iQrh8U1RB5nZUklLJWn//fevc5MAAAAAYGqo9yyXH5N0trvvJ+lsSd+MOsjdL3f3Re6+qLMz1tg/AAAAAJj26h3ozpC0Jvj+u5IOr/P1AAAAAGDaqHeg2yrpbcH3b5f0+zpfDwAAAACmjSSXLbhW0lGS5prZFkkXSPofkr5qZm2SXlAwTg4AAAAAMHmJBTp3P63MrtcndQ0AAAAAwIvqXXIJAAAAAKiTei9b0PLWbujRynWbtbVvQPt05LVs8XwtWdjV7GYBAAAAAIGukrUberR8zSYNDA5Lknr6BrR8zSZJItQBAAAAaDpKLitYuW7zaJgrGhgc1sp1m5vUIgAAAAB4EYGugq19AzVtBwAAAIBGItBVsE9HvqbtAAAAANBIBLoKli2er3wuO2ZbPpfVssXzm9QiAAAAAHgRk6JUUJz45KIf/FZPPrdLe82eoc8c9yomRAEAAACQCvTQVbFkYZf+/cOLJEmXnPxawhwAAACA1CDQxZAxkyS5e5NbAgAAAAAvItDFUAx0wyNNbggAAAAAhBDoYsgE79IIPXQAAAAAUoRAFwMllwAAAADSiEAXAyWXAAAAANKIQBdDlpJLAAAAAClEoIvBgh46Ah0AAACANCHQxZAh0AEAAABIIQJdDNlioGMMHQAAAIAUIdDFEOQ5eugAAAAApAqBLoZMhpJLAAAAAOlDoIthtOSSPAcAAAAgRQh0MWQouQQAAACQQgS6GEaXLaCLDgAAAECKEOhiyGYouQQAAACQPokFOjO70sy2mdl9Jds/YWabzex+M/tSUtdrJEouAQAAAKRRkj10qyQdG95gZkdLep+k17j7oZIuSfB6DVMsuRymiw4AAABAiiQW6Nz955K2l2z+mKQV7r4zOGZbUtdrpGLJJR10AAAAANKk3mPoDpL0FjO708z+y8zeEHWQmS01s24z6+7t7a1zk2pHySUAAACANKp3oGuTNEfSEZKWSbrOivWLIe5+ubsvcvdFnZ2ddW5S7TLFkksCHQAAAIAUqXeg2yJpjRfcJWlE0tw6XzNxxUBHngMAAACQJvUOdGslvV2SzOwgSTMkPVnnayZutOSSSVEAAAAApEhbUicys2slHSVprpltkXSBpCslXRksZbBL0hnurdfPRcklAAAAgDRKLNC5+2lldp2e1DWaJcPC4gAAAABSqN4ll1NGxqQW7FwEAAAAMIUR6GLKmLGwOAAAAIBUIdDFlMkYJZcAAAAAUoVAFxMllwAAAADShkAXEyWXAAAAANKGQBdT1ii5BAAAAJAuBLqYzKQRSi4BAAAApAiBLqbCpCgEOgAAAADpQaCLqVBySaADAAAAkB4EupiMMXQAAAAAUoZAF1PGpBESHQAAAIAUIdDFlGUMHQAAAICUIdDFlKHkEgAAAEDKEOhiMkouAQAAAKQMgS4mSi4BAAAApA2BLiZKLgEAAACkDYEuJjNpmB46AAAAAClCoIspayYn0AEAAABIEQJdTBkzjYw0uxUAAAAA8CICXUyUXAIAAABIGwJdTNkMJZcAAAAA0oVAFxOzXAIAAABIGwJdTBmThkl0AAAAAFKEQBdThoXFAQAAAKRMYoHOzK40s21mdl/Evk+ZmZvZ3KSu12gZM5HnAAAAAKRJkj10qyQdW7rRzPaT9E5JjyZ4rYaj5BIAAABA2iQW6Nz955K2R+y6VNKnJbV0GipMitLSLwEAAADAFFPXMXRmdoKkHne/p8pxS82s28y6e3t769mkCaPkEgAAAEDa1C3QmVm7pH+S9Nlqx7r75e6+yN0XdXZ21qtJk5LJsLA4AAAAgHSpZw/dKyUdIOkeM3tY0r6SfmNmL6vjNeuGkksAAAAAadNWrxO7+yZJexcfB6Fukbs/Wa9r1hMLiwMAAABImySXLbhW0q8lzTezLWb2kaTOnQYZk0ZIdAAAAABSJLEeOnc/rcr+eUldqxmyLCwOAAAAIGXqOsvlVGKUXAIAAABIGQJdTJRcAgAAAEgbAl1MzHIJAAAAIG0IdDFlGEMHAAAAIGUIdDGxbAEAAACAtCHQxZQx0UMHAAAAIFUIdDFlGUMHAAAAIGUIdDGZmUZGmt0KAAAAAHgRgS4mSi4BAAAApA2BLqYss1wCAAAASBkCXUxmpmFKLgEAAACkCIEupoxJTg8dAAAAgBQh0MVEySUAAACAtCHQxZQx0zAriwMAAABIEQJdTGYSHXQAAAAA0oRAFxMLiwMAAABIGwJdTJmMaZhABwAAACBFCHQxmUkMoQMAAACQJgS6mLJmLFsAAAAAIFUIdDExyyUAAACAtCHQxZSh5BIAAABAyhDoYspkTJIouwQAAACQGgS6mDJWCHSUXQIAAABICwJdTEEHHWWXAAAAAFIjsUBnZlea2TYzuy+0baWZPWhm95rZ98ysI6nrNVqx5JLFxQEAAACkRZI9dKskHVuy7SeSXu3ur5H0O0nLE7xeQxVLLslzAAAAANIisUDn7j+XtL1k24/dfSh4eIekfZO6XqMVSy6HSXQAAAAAUqKRY+j+RtIPG3i9RBV76Ci5BAAAAJAWDQl0ZvZPkoYkXVNm/1Iz6zaz7t7e3kY0qWajJZcjTW4IAAAAAATqHujM7AxJx0n6oJdZxM3dL3f3Re6+qLOzs95NmhBKLgEAAACkTVs9T25mx0r6R0lvc/f+el6r3rLMcgkAAAAgZZJctuBaSb+WNN/MtpjZRyR9TdLukn5iZhvN7P8mdb1GM8bQAQAAAEiZxHro3P20iM3fTOr8zTY6KQpj6AAAAACkRCNnuWxp2eCdoocOAAAAQFoQ6GKi5BIAAABA2hDoYqLkEgAAAEDaEOhiouQSAAAAQNoQ6GLKUHIJAAAAIGUIdDExhg4AAABA2hDoYsqOBromNwQAAAAAAgS6mDKFPEcPHQAAAIDUINDFVCy5HKaLDgAAAEBKEOhiygZddHTQAQAAAEgLAl1MlFwCAAAASBsCXUwZSi4BAAAApAyBLqZMhlkuAQAAAKQLgS6mYsmlU3IJAAAAICUIdDFRcgkAAAAgbQh0MWVYWBwAAABAyhDoYqLkEgAAAEDaEOhiKk6KMkygAwAAAJASBLqYKLkEAAAAkDYEuphYWBwAAABA2hDoYhrtoaOLDgAAAEBKEOhiyrKwOAAAAICUIdDFZJRcAgAAAEgZAl1MlFwCAAAASJvEAp2ZXWlm28zsvtC2Pc3sJ2b2++DrnKSu12iUXAIAAABImyR76FZJOrZk27mSfubuB0r6WfC4JTHLJQAAAIC0SSzQufvPJW0v2fw+SVcF318laUlS12s0G12HjkAHAAAAIB3qPYbupe7+uCQFX/eu8/XqJkugAwAAAJAyqZgUxcyWmlm3mXX39vY2uzmRXpwUpckNAQAAAIBAvQPdE2b2ckkKvm6LOsjdL3f3Re6+qLOzs85NmpjisgXD9NABAAAASIl6B7obJZ0RfH+GpO/X+Xp1U5zl0gl0AAAAAFIiyWULrpX0a0nzzWyLmX1E0gpJ7zSz30t6Z/C4JY2WXJLnAAAAAKREW1IncvfTyuw6JqlrNFNx2YJhEh0AAACAlEjFpCitIEPJJQAAAICUIdDFRMklAAAAgLQh0MVEySUAAACAtCHQxVQsuWRhcQAAAABpQaCLqVhySZ4DAAAAkBYEupgyLCwOAAAAIGUIdDG9OCkKgQ4AAABAOhDoYqLkEgAAAEDaEOhiYpZLAAAAAGlDoIspyyyXAAAAAFKGQBeTsbA4AAAAgJQh0NUgY9IIiQ4AAABAShDoapDNGCWXAAAAAFKDQFcDM6PkEgAAAEBqEOhqkDEmRQEAAACQHgS6GmTNGEMHAAAAIDUIdDXIUHIJAAAAIEUIdDUwSi4BAAAApAiBrgbMcgkAAAAgTQh0NSiUXBLoAAAAAKQDga4GZqbhkWa3AgAAAAAKCHQ1yGYkp4cOAAAAQEoQ6GpAySUAAACANCHQ1SBDySUAAACAFCHQ1SBDySUAAACAFGlIoDOzs83sfjO7z8yuNbNZjbhu0ii5BAAAAJAmdQ90ZtYl6R8kLXL3V0vKSjq13teth4yZhslzAAAAAFKiUSWXbZLyZtYmqV3S1gZdN1EZEz10AAAAAFKj7oHO3XskXSLpUUmPS3ra3X8cPsbMlppZt5l19/b21rtJE5YxYwwdAAAAgNRoRMnlHEnvk3SApH0kzTaz08PHuPvl7r7I3Rd1dnbWu0kTVpjlkkAHAAAAIB0aUXL5DkkPuXuvuw9KWiPpTQ24buIyGRN5DgAAAEBaNCLQPSrpCDNrNzOTdIykBxpw3cRljGULAAAAAKRHI8bQ3Snpekm/kbQpuObl9b5uPVByCQAAACBN2hpxEXe/QNIFjbhWPVFyCQAAACBNGrVswZTAsgUAAAAA0oRAV4OMGYEOAAAAQGoQ6GqQNdPISLNbAQAAAAAFBLoaGCWXAAAAAFKEQFcDSi4BAAAApAmBrgZZZrkEAAAAkCIEupjWbujR3Q9v1/pHdujIFbdo7YaeZjcJAAAAwDRHoIth7YYeLV+zSTuHCjOi9PQNaPmaTYQ6AAAAAE1FoIth5brNGhgcHrNtYHBYK9dtblKLAAAAAIBAF8vWvoGatgMAAABAIxDoYtinI1/TdgAAAABoBAJdDMsWz1c+lx2zLZ/Latni+U1qEQAAAABIbc1uQCtYsrBLknT+2vv03M4hdXXktWzx/NHtAAAAANAMBLqYlizs0kNPPq+v/uz3+sWnj1YmY81uEgAAAIBpjpLLGrTPKJRdls54CQAAAADNQKCrQTHQ9e8i0AEAAABoPgJdDfIzChWq/buGmtwSAAAAACDQ1YQeOgAAAABpQqCrAYEOAAAAQJoQ6GrQHpRcDhDoAAAAAKQAga4GL/bQMYYOAAAAQPMR6GqQZ9kCAAAAAClCoKsBY+gAAAAApAmBrgbtueKyBQQ6AAAAAM3XkEBnZh1mdr2ZPWhmD5jZGxtx3aQVSy77dzKGDgAAAEDztTXoOl+V9CN3f7+ZzZDU3qDrJmpGW0ZtGVM/Y+gAAAAApEDdA52ZvUTSWyWdKUnuvkvSrnpft17yM7IsWwAAAAAgFRpRcvkXknol/YeZbTCzK8xsdvgAM1tqZt1m1t3b29uAJk3c7BltLFsAAAAAIBUaEejaJL1O0tfdfaGk5yWdGz7A3S9390Xuvqizs7MBTZq49hlZJkUBAAAAkAqNCHRbJG1x9zuDx9erEPBaEiWXAAAAANKi7oHO3f8s6TEzmx9sOkbSb+t93Xqhhw4AAABAWjRqlstPSLommOHyT5L+ukHXTVx+RpueHhhsdjMAAAAAoDGBzt03SlrUiGvVW3suq8f7BprdDAAAAABozMLiUwkllwAAAADSgkBXo/yMrAZYWBwAAABAChDoajR7JuvQAQAAAEgHAl2N8rmsXhgc0ciIN7spAAAAAKY5Al2N2mdkJYmySwAAAABNR6CrUTHQMTEKAAAAgGYj0NUoP6Ow0sMAgQ4AAABAkxHoalTsoXueiVEAAAAANBmBrkZ5Si4BAAAApASBrkbrH94uSTrp67fryBW3aO2Gnia3CAAAAMB0RaCrwdoNPfr3Xzw0+rinb0DL12wi1AEAAABoCgJdDVau26ydQyNjtg0MDmvlus1NahEAAACA6YxAV4OtfQM1bQcAAACAeiLQ1WCfjnxN2wEAAACgngh0NVi2eL7yueyYbflcVssWz29SiwAAAABMZwS6GixZ2KWLTzxMs2e8GOpm5XgLAQAAADQHaWQCht1Hv9/RP8hMlwAAAACagkBXo5XrNuuFQWa6BAAAANB8BLoaMdMlAAAAgLQg0NWImS4BAAAApAWBrkbMdAkAAAAgLdqa3YBWs2RhlyTp8zf/Vtuf36XO3Wbqn957yOh2AAAAAGgUeugmYMnCLv3HmW+QJF184mGEOQAAAABNQaCboLm7z5QkPfnczia3BAAAAMB01bBAZ2ZZM9tgZjc36pr1NHe3GZKk3mcJdAAAAACao5E9dJ+U9EADr1dXM9uy2iOfUy89dAAAAACapCGBzsz2lfReSVc04nqNMne3GZRcAgAAAGiaRvXQXSbp05JGonaa2VIz6zaz7t7e3gY1afI6d59JySUAAACApql7oDOz4yRtc/f15Y5x98vdfZG7L+rs7Kx3kxIzdzcCHQAAAIDmaUQP3ZGSTjCzhyV9R9LbzezqBly37jp3n6knn9vV7GYAAAAAmKbqHujcfbm77+vu8ySdKukWdz+93tdthM7dZ+q5nUMa2DXc7KYAAAAAmIZYh24S5u7GWnQAAAAAmqehgc7db3P34xp5zXr6w7ZnJUlv/dKtOnLFLVq7oafJLQIAAAAwndBDN0FrN/ToqtsfkSS5pJ6+AZ29eqPOX7upuQ0DAAAAMG0Q6CZo5brN2jk0dhUGl3TNHY/SUwcAAACgIQh0E7S1byByu6sQ9gAAAACg3gh0E7RPR77svnJhDwAAAACSRKCboGWL58vK7KsU9gAAAAAgKQS6CVqysEsfPGL/caEun8tq2eL5TWkTAAAAgOmFQDcJFy05TJeeskAv32OWJOkls9p08YmHacnCria3DAAAAMB0QKCbpCULu/Tr5ceoc/eZOvbVLyPMAQAAAGgYAl1CDthrth5+qr/ZzQAAAAAwjRDoEvKKvdr18JPPN7sZAAAAAKYRAl1C5s2drW3P7lT/rqFmNwUAAADANEGgS8i2Z16QJB362XU6csUtWruhp8ktAgAAADDVEegSsHZDj75z92OSJJfU0zeg5Ws2EeoAAAAA1BWBLgEr123WzqGRMdsGBoe1ct3mJrUIAAAAwHRAoEvA1r6BmrYDAAAAQBIIdAnYpyNf03YAAAAASAKBLgHLFs9XPpcds80kHX1wZ3MaBAAAAGBaINAlYMnCLp30+i5ZaJtLumF9DxOjAAAAAKgbAl1Cbn2wV16yjYlRAAAAANQTgS4h5SZA6ekboJcOAAAAQF0Q6BJSaQIU1qQDAAAAUA8EuoRETYxSNDA4rAtvvL/BLQIAAAAw1RHoErJkYZcuPvGwsvv7Bga18PM/pqcOAAAAQGIIdAlasrBLXRVKL3f0D+qs1RsJdgAAAAASUfdAZ2b7mdmtZvaAmd1vZp+s9zWbadni+VWP2dE/yLg6AAAAAJPWiB66IUnnuPshko6Q9Pdm9qoGXLcplizs0pz2XNXjWNIAAAAAwGTVPdC5++Pu/pvg+2clPSCpq97XbaYLjj+07AQpYeWWOgAAAACAOBo6hs7M5klaKOnOku1LzazbzLp7e3sb2aS6KE6Q0pGv3FPnko5ccQullwAAAAAmxNy9MRcy203Sf0n6Z3dfU+64RYsWeXd3d0Pa1AhrN/TowhvvV9/AYNlj8rmsLj7xMC1ZOKU7LgEAAADEYGbr3X1RrGMbEejMLCfpZknr3P0rlY6daoGuaOHnf6wd/eVDXdGc9pwuOP5Qwh0AAAAwTdUS6Noa0BiT9E1JD1QLc1NZX4wwJ724tMH/um6jRlzqyOe0a2hY/YMjkl4MfJLG9PxlTBpxyVQo5Qxv6+rI6+iDO3Xrg73a2jegfTryWrZ4fk2hce2GHq1ct3nM8yWN20YQBQAAABqn7j10ZvZmSb+QtEnSSLD5PHf/z6jjp2oP3ZErblFPCidBKYa+rJmG3ccFyFrkMqbdZrWpr3+wYugLb9ujQmANH2NWCLvhdpppzLWWLOxKPHgWz9fTNzCha8e5TpznTfTccdX7/ElqpbYCAABMROpKLmsxVQPd2g09Wr5mkwYGh5vdlKbLZkzDI82774q9mF0lYaxaj+dErxM+R3suo5m57JhwWnqN0jGV56/dpGvueHTMMeHXUNr7Gn5cKQzv6B8s+/qK26PCq1Q+kBdfX2nYLSoXxqKCc1fE86Pei0pjUKudN8lwGD7XHmVCf5oRlJuL9x8AEEagS6nwH5dAJSapoz0Xa9xlo00kkM9pz+lVL99dt/9xe00B2SR98Ij9ddGSw7R2Q4/OXr0x8vlZM532V/uNCbbz9spXvN6MrGnX8Ni9+VxWJ72+q2p5cpzJjkpfR9yAXNxfGgonWzZd2v7SoBsV7mfPyKp/13DF65W+FxMdB1yvsu5KQSktISrqA7/wBxXN7MVv9nvU7Os3oz1pe83ltEo7gVZFoEu5tJZfAigv3LOZpMn0WJfrEa0Ujhr5/5457Tm99zUv1833PD4a+MK91LX2gBcDd9T5okJyT99A2R5wSWWrJqJ6h6WJ/wFbrae41t8J4Q8IypXIV6oQqNZzHG5v6fsXLq2v1BNd2mMdVVpf7b2rFnRLj53sBwvVfr7l2hPnQ6C414/6oCiqqqSWNtdyz1aroghXgtywvifWzybuaycoA2MR6FKO8ksASLfS4BJ3bHGcsuo0CQefWnueo9RSrh4OmFHhsNwHEF0def3q3LePPl67oUfLvnuPBst8MBL1M4kzbrw0oMQN3qXPiwpDpQGw+5Ht48rJy527GCCjAnexqmHRK/Ys+7Ms1/Ne7m+T9lxGgyOuwVBFQ7mfb+nPpqhSUIxqZ9LLOZV7bXGqEOK8jsm0K4nAXa/2Jd1+1IZA1wKS/gQ9iTFfAAC0gqje33oq/o6Nq9lhPpe1MQGsknDgTcJlpywYExIr9TrGEZ6xu5YJ0IrBeTJVCaWzhZf2SkaV00eNXS8d717uQ6JqY+8rjbmPGmMuvfjfSvG9qDbBW5S48wyEKyAq/Y1b7gOV0p710uuWey9KJ9Qr9zpbbXw7gW4KKPc/q1pKSsrN0Bj+j6fWX1IAAABIj/ZcZkKzk6dZRz6nZ14YbNjfqGlcB5pAh5pNduxApdDYnstIUsX/2cQZGxP16ViUepU8FUuwKo39iSpNAQAAQLrlsqaV739takIdgQ51Mdla6SSmdZ/oOaKeV67LPs4MeBPZV20cRbVZGUsH30fNiFhurb498jk9v2toTNCsNM6j9NzVPiWLalulMUeVQn4uY8plreIHALms6fB5c2LPmlnv8qxiT3ecJSEAAEA6lRsH2gwEOqBF1XMts8kE8qge2kozr9Vy3Thr05UL3rUs/F7ueuFxFlFr9VUSdw28OOEuauxEeHucSRziKh0XUjpBgzR+KYVKITU8fmGyk2pI419zvaU5fFMWDwCNY5IeWvHeZjdDEoEOQB1M19msku5ZrrbGWtz3udK08OEew4lMF5+EJF9zuSnpw+eL00sd9R51hdoWZ/Zhk/SmV+6p+7c+W3NwrTRDYdTED+HXX6ltxfNWmkY+/B7PymU0UOUDgagZKCu9t3E/AHnd/nvUvB5lLXJZ0ylv2K9hk6XUw5yUrkEKTAf00CWEQAcAaIaJ9PhGPb/S9PSTXQy+0gcLtXzoEqcHvFHtjDPH9FMCAAAgAElEQVT7XxxRbY9bAl+tMqAYuKNmLYzTC98REYrLLTlw0ZLDYi2REJ5yv9IY86iZB8uF+vCHQaVl+vUQVS0Qt6phIrrKVCFMtuoBUwNj6BJEoAMAALWGsTS0tdzi8ZWeU24sdq1l7rW2YSLl8FL0bNuSKgbhcuXdtVQ4hF9XLSXoSXxIETV1fuk0+KUfYJQL2VETvpV7XyqNba+2fqIUrwQ+zrh1qfI8A3GrRMLXLK7zGXfSu458TrNnto27H6P+OymuxVhu2EPUEAdmuUwYgQ4AAEx3aQyuaZe296wR7ak1mCdZOZBUG8PHRAWwOOPV45w7LfdFXAQ6AAAAAC2nVQNY0moJdG31bgwAAAAAxLFkYde0DHCTkWl2AwAAAAAAE0OgAwAAAIAWRaADAAAAgBZFoAMAAACAFkWgAwAAAIAWRaADAAAAgBZFoAMAAACAFkWgAwAAAIAWZe7e7DaMYWa9kh5pdjsizJX0ZLMbgWmD+w2NxP2GRuFeQyNxv6GRkr7fXuHunXEOTF2gSysz63b3Rc1uB6YH7jc0EvcbGoV7DY3E/YZGaub9RsklAAAAALQoAh0AAAAAtCgCXXyXN7sBmFa439BI3G9oFO41NBL3GxqpafcbY+gAAAAAoEXRQwcAAAAALYpABwAAAAAtikAXg5kda2abzewPZnZus9uD1mdmV5rZNjO7L7RtTzP7iZn9Pvg6J9huZva/g/vvXjN7XfNajlZjZvuZ2a1m9oCZ3W9mnwy2c78hcWY2y8zuMrN7gvvtc8H2A8zszuB+W21mM4LtM4PHfwj2z2tm+9F6zCxrZhvM7ObgMfca6sLMHjazTWa20cy6g22p+F1KoKvCzLKS/o+kd0t6laTTzOxVzW0VpoBVko4t2XaupJ+5+4GSfhY8lgr33oHBv6WSvt6gNmJqGJJ0jrsfIukISX8f/D+M+w31sFPS2939tZIWSDrWzI6Q9C+SLg3utx2SPhIc/xFJO9z9LyVdGhwH1OKTkh4IPeZeQz0d7e4LQuvNpeJ3KYGuusMl/cHd/+TuuyR9R9L7mtwmtDh3/7mk7SWb3yfpquD7qyQtCW3/f15wh6QOM3t5Y1qKVufuj7v7b4Lvn1XhD58ucb+hDoL75rngYS7455LeLun6YHvp/Va8D6+XdIyZWYOaixZnZvtKeq+kK4LHJu41NFYqfpcS6KrrkvRY6PGWYBuQtJe6++NS4Y9wSXsH27kHkYigxGihpDvF/YY6CUrgNkraJuknkv4oqc/dh4JDwvfU6P0W7H9a0l6NbTFa2GWSPi1pJHi8l7jXUD8u6cdmtt7MlgbbUvG7tK1eJ55Coj69Ya0HNBL3ICbNzHaTdIOks9z9mQofTHO/YVLcfVjSAjPrkPQ9SYdEHRZ85X7DhJjZcZK2uft6MzuquDniUO41JOVId99qZntL+omZPVjh2Ibeb/TQVbdF0n6hx/tK2tqktmBqe6LYHR983RZs5x7EpJhZToUwd427rwk2c7+hrty9T9JtKozd7DCz4ofI4Xtq9H4L9u+h8eXoQJQjJZ1gZg+rMBzm7Sr02HGvoS7cfWvwdZsKH1YdrpT8LiXQVXe3pAODWZNmSDpV0o1NbhOmphslnRF8f4ak74e2fziYMekISU8Xu/eBaoIxIt+U9IC7fyW0i/sNiTOzzqBnTmaWl/QOFcZt3irp/cFhpfdb8T58v6Rb3J1eE1Tl7svdfV93n6fC32a3uPsHxb2GOjCz2Wa2e/F7Se+SdJ9S8rvUuJerM7P3qPCpT1bSle7+z01uElqcmV0r6ShJcyU9IekCSWslXSdpf0mPSvrv7r49+IP8ayrMitkv6a/dvbsZ7UbrMbM3S/qFpE16cZzJeSqMo+N+Q6LM7DUqTAyQVeFD4+vc/fNm9hcq9KLsKWmDpNPdfaeZzZL0LRXGdm6XdKq7/6k5rUerCkouP+Xux3GvoR6C++p7wcM2Sd929382s72Ugt+lBDoAAAAAaFGUXAIAAABAiyLQAQAAAECLItABAAAAQIsi0AEAAABAiyLQAQAAAECLItABAFqemT0XfJ1nZh9I+NznlTy+PcnzAwAwGQQ6AMBUMk9STYHOzLJVDhkT6Nz9TTW2CQCAuiHQAQCmkhWS3mJmG83sbDPLmtlKM7vbzO41s49KhYWIzexWM/u2Couuy8zWmtl6M7vfzJYG21ZIygfnuybYVuwNtODc95nZJjM7JXTu28zsejN70MyuCRaZBQAgcW3NbgAAAAk6V9Kn3P04SQqC2dPu/gYzmynpV2b24+DYwyW92t0fCh7/jbtvN7O8pLvN7AZ3P9fMPu7uCyKudaKkBZJeK2lu8JyfB/sWSjpU0lZJv5J0pKRfJv9yAQDTHT10AICp7F2SPmxmGyXdKWkvSQcG++4KhTlJ+gczu0fSHZL2Cx1XzpslXevuw+7+hKT/kvSG0Lm3uPuIpI0qlIICAJA4eugAAFOZSfqEu68bs9HsKEnPlzx+h6Q3unu/md0maVaMc5ezM/T9sPh9CwCoE3roAABTybOSdg89XifpY2aWkyQzO8jMZkc8bw9JO4Iwd7CkI0L7BovPL/FzSacE4/Q6Jb1V0l2JvAoAAGLiE0MAwFRyr6ShoHRylaSvqlDu+JtgYpJeSUsinvcjSf/TzO6VtFmFssuiyyXda2a/cfcPhrZ/T9IbJd0jySV92t3/HARCAAAawty92W0AAAAAAEwAJZcAAAAA0KIIdAAAAADQogh0AAAAANCiCHQAAAAA0KIIdAAAAADQogh0AAAAANCiCHQAAAAA0KIIdAAAAADQogh0AAAAANCiCHQAAAAA0KIIdAAAAADQogh0AAAAANCiCHQAAAAA0KIIdAAAAADQogh0AICWY2a3mdkOM5vZ7LYAANBMBDoAQEsxs3mS3iLJJZ3QwOu2NepaAADERaADALSaD0u6Q9IqSWcUN5pZ3sy+bGaPmNnTZvZLM8sH+95sZrebWZ+ZPWZmZwbbbzOzvw2d40wz+2XosZvZ35vZ7yX9Ptj21eAcz5jZejN7S+j4rJmdZ2Z/NLNng/37mdn/MbMvh1+Emd1kZmfV4w0CAEwfBDoAQKv5sKRrgn+LzeylwfZLJL1e0psk7Snp05JGzGx/ST+U9K+SOiUtkLSxhustkfRXkl4VPL47OMeekr4t6btmNivY978knSbpPZJeIulvJPVLukrSaWaWkSQzmyvpGEnX1vLCAQAoRaADALQMM3uzpFdIus7d10v6o6QPBEHpbyR90t173H3Y3W93952SPijpp+5+rbsPuvtT7l5LoLvY3be7+4AkufvVwTmG3P3LkmZKmh8c+7eSznf3zV5wT3DsXZKeViHESdKpkm5z9ycm+ZYAAKY5Ah0AoJWcIenH7v5k8Pjbwba5kmapEPBK7Vdme1yPhR+Y2Tlm9kBQ1tknaY/g+tWudZWk04PvT5f0rUm0CQAASRIDvAEALSEYD3eypKyZ/TnYPFNSh6SXS3pB0isl3VPy1MckHV7mtM9Lag89flnEMR5qw1sk/aMKPW33u/uIme2QZKFrvVLSfRHnuVrSfWb2WkmHSFpbpk0AAMRGDx0AoFUskTSswli2BcG/QyT9QoVxdVdK+oqZ7RNMTvLGYFmDayS9w8xONrM2M9vLzBYE59wo6UQzazezv5T0kSpt2F3SkKReSW1m9lkVxsoVXSHpC2Z2oBW8xsz2kiR336LC+LtvSbqhWMIJAMBkEOgAAK3iDEn/4e6Puvufi/8kfU2FcXLnStqkQmjaLulfJGXc/VEVJik5J9i+UdJrg3NeKmmXpCdUKIm8pkob1qkwwcrvJD2iQq9guCTzK5Kuk/RjSc9I+qakfGj/VZIOE+WWAICEmLtXPwoAAEyamb1VhdLLee4+0uz2AABaHz10AAA0gJnlJH1S0hWEOQBAUgh0AADUmZkdIqlPhclbLmtycwAAUwgllwAAAADQouihAwAAAIAWlbp16ObOnevz5s1rdjMAAAAAoCnWr1//pLt3xjk2dYFu3rx56u7ubnYzAAAAAKApzOyRuMdScgkAAAAALYpABwAAAAAtikAHAAAAAC2KQAcAAAAALYpABwAAAAAtikAHAAAAAC2KQAcAAAAALYpABwAAAAAtikAHAAAAAC2qrdkNAAAAAIBarN3Qo5XrNmtr34D26chr2eL5WrKwK/HntAJz92a3YYxFixZ5d3d3s5sBAAAAoM4mGsyWr9mkgcHh0W25jGm3WW3a0T+orJmG3dXVkdfRB3fq1gd71dM3EHmuOe05XXD8oakLdma23t0XxTqWQAcAAACg3orhradvYDR0maRwGsnnsrr4xMMqBqwjV9xSNqBNhEn64BH766IlhyV2zski0AEAAABoiqheN0njetXKyZppxH1cj104ECbNJF16yoLU9NQR6AAAAADEksTYsnDYiup1y5j0/K7qYa5UuJSy9LxJ6+rI61fnvr2OV4ivlkDHpCgAAADANHX+2k265o5HR4NST9+Alq/ZJEmxJxkp7TErDV1xeuXKGRxx7egfjDxv0rbWoeevEQh0AAAAwDS0dkPPmDBXNDA4rLNWb9Q5192jYfcxk4wUe++iJiZpdft05JvdhAkh0AEAAAAtbKIlkyvXba7Y6zUcDM0qfg333q1ct7llw1w+l9HA4EjJtuzoWL9WQ6ADAAAAWlRpT1lpyWS5sW0Zk0YmUMNY7L1Lu2KvYlTvojS11qRjUhQAAAAgBcIhY498TmZSX/9g5PfVJgkptyzAdGGSHlrx3mY3Y8KYFAUAAABooMn2+JT2tPUNDI7uK/d9nHLJtIa504/YX7c+2DsmvO7oH1RbxjQ04qNfJ6pVx8NNBIEOAAAAiBA3pEXNFHnW6o06a/VGzWnP6YLjDx3zvKjztvKYtLCOUG+iTIoqBuzqyEcu4v2lHz2of7vtj5KkUw/fLzLwxelxbOXxcBNBySUAAAAQqLSemqRxAW3thh6dvXpj1ZDRnstoZi4bGUqmSllkaZlj1EyY+VxWF594WGQwvvDG+7Tq9kckSXvvPlPnveeQcceVW7R8qoyHK2JhcQAAAKCCqPFqcXuAisd05HN65oXBCU0ukjazZ2QjF/7uyOc0MDisnUMjEc8aK2ph7ri9nGs39Ogfb7h3zHUqhb+pjjF0AAAAmLbKTS5SDBTdj2wfUyIZd1xa6THh57W6XUMjymVNg8MvvgP5XFZmihXmypU5LlnYFXsJhdLrDAwOa+W6zdMy0NUiVqAzs2MlfVVSVtIV7r6iZP+ZklZK6gk2fc3drwj2DUvaFGx/1N1PSKDdAAAAmAIq9eDELa+TpAtvvD8yYIW39fQNaNl379HgVOhSm6CuYLKQnr6BMdsHR1wd+Zxmz2wb896eXWGJgq6OfGJljltL2lNtO15UteTSzLKSfifpnZK2SLpb0mnu/tvQMWdKWuTuH494/nPuvlvcBlFyCQAAMD1EjbEKlzM+v2toTI9RlFymMD3/dMxoxZJESePex0rHlxvzFzXV/5ErbhkX/qTo8srJaNR1WkXSJZeHS/qDu/8pOPl3JL1P0m8rPgsAAABTRi09aUcf3Dk6Q2Gl3puomR1rLWdMW2+bScrnMuofrF6mWIviunJhxZLEYuCJ6qUsBuTwwtrFSV9KRU31v2zx/MiJTZKeRbJR15mK4gS6LkmPhR5vkfRXEcedZGZvVaE372x3Lz5nlpl1SxqStMLd15Y+0cyWSloqSfvvv38NzQcAAEA5Ew1aUecJ/7Hd0zeg5Ws2qfuR7br5nsfHlTVefcejYx4vX1MYfRMOgOVKJFtZ6SQeazf06DPfv0/PvjA0qfOapJEyVXXFksTiWLU4k5DUEp6Kz633LJKNus5UFKfk8r9LWuzufxs8/pCkw939E6Fj9pL0nLvvNLP/Kelkd397sG8fd99qZn8h6RZJx7j7H8tdj5JLAACAias27X5YpVkEw8EgE9E7VCuT1NGe047+1g1xFnwtV674wSP2H7e+2s33btXHv71Be+Tb9MzA0ISWJyg37q24byIliZNdCB31lXTJ5RZJ+4Ue7ytpa/gAd38q9PDfJf1LaN/W4OufzOw2SQsllQ10AAAAiKd0NsddQ8NjSv2qhYeBwWFdeOP9Fc8hadJhrtiWVg5z+VxWu8/MattzuyL3u6RbH+wdt/2VnYWpJJ4eGNKC/fZQ77O7RkNU/66hqu9JuOcsyZLEuLNPIv3iBLq7JR1oZgeoMIvlqZI+ED7AzF7u7o8HD0+Q9ECwfY6k/qDnbq6kIyV9KanGAwAATDfleuAmWr7YNzA4+typUgKZMWnE4y3YXZzZMapHMzz+7OiDO3XtXY9FnyQQNSPjAXNny0xyl9520N46+50Hje6LmhQmlzHtNqttzDIL4eBFrxpKVQ107j5kZh+XtE6FZQuudPf7zezzkrrd/UZJ/2BmJ6gwTm67pDODpx8i6RtmNiIpo8IYOiZTAQAAiGGyPXDTjUm69JQFkZO1lHuvnh4Y1MYL3jXu+NLAdOSKWzRcZQKWqElFfnTfn0dLVv/frx/WAXNnj56z1nFj9KohStUxdI3GGDoAADBVTGScUi1j4DBeuTFlk50W/4Bzf1DzeMSoHrhK4xaBoqTH0AEAAExL4XBVnDa+q4ZgVjoz5LLv3qPP3XS/dvQPjp6vo8y4NYkwNxHlFqKe7LT4+3TkIwOhpLL3RNSyDMWlBgh0SAqBDgAATDtxes5KA1lxYpCevgGdvXqjuh/ZPmZGw9Jz9u8aGvfH/OCIj06CUTzfVBm3NlFR66tN5JiiqLJHafLT4pcLhJV628qFy3LbgYkg0AEAgCmhXG9a6dprRx/cqRvW94zpOTt79UadtXrjmJ6Wz910/7hAVuSSrgnWWrv1wd5x5ZHlenKmgqyZvnzyayVp3Bp3pWvSVZPPZcu+x+FjTnp915ifmVSYnKG0T7Naj9tkxqBNJBCW69UrFzqBiWAMHQAAaDlRC2aX/sE/GTOypl3D6fobKUqt7Yzb01UpaJmkh1a8t+xzSydyeX7XkAZDbQzPHLls8fzREB4lHLCjelWldM/6yBg6TFQtY+gIdAAAIPWqhYTpak7MhbrDIaLc5CAWJK19qgStWheyrlbeOtVDDwt4YyKYFAUAAEwZpX/wT9cxZ+GerU+96yB95vvB5CoZi5xOP2umEfdxIaLs+C0f3/OWxELW1cocJzu2Le1YagD1RqADAACptXZDj8657p7YE2JMNaXlieFgsOr2h3XPlqd14N676ZGn+mP3cMUd19XIoEXoASaOQAcAAGpSroSslu3S+Ak1ipOLhKfzf37XUGrCXC5j2m1WW6wSx3KKr63cWLaOfE6zZ7ZVDVBrN/Tod088J0l6bHu/Tnp915iJXyoFr1qm7ydoAenHGDoAAFBVpcWui4+jFsFuz2U0OOJjxrvlMiaZWmoM3Jz2nC44/lAtWdilU79xu+54aEfN5wiPPZvMuLEkxpwxrgtIN8bQAQCAcSb6R3xpgCiNYV7yNSxqsezBiPFeScuaVMyLUUGzFh35nDZ89l2SCu/FhseentB5wj1gkylnTGKxanregKmDQAcAwDRQGsp6+ga0fM0mSRr3h32l3rhWUOwVHA4SXbX2m1R26YN8LqsLTzh09PHKdZu1c2h8SK2mI58b9z5PNFSxWDWAMAIdAAAtIu46XOFte+Rz2jU0HNlTNjA4rLOv26izV28c89xKvXGtYOeQxx53VzoF/6JX7Fmx12wioak0FE4Wi1UDCGMMHQAALSBq3FS5sWjlprGvJp/LalYuM6lJP1rJRNY6K7eGW3gykz3yOZlJff2DdRmfNtXXbQPAGDoAAJquXG/ahTfeP7qOWniijUrPW7KwK3LcVLmxaBMJc1Khx670Gq1osjNIVlJuhsgLTzi0YWFqqq/bBqA29NABAJCwcr1pw+4ql7VacaxaM1R7n/K5rE56fVfkWLikerCYIRJAvdFDBwBAE9XSm1bUymGuNGQVQ9XN9zw+2huZhGIok8aOEYwqb6w2Fm4ymCESQJrQQwcAQEitvS9rN/SMKaOcbrqC96jcexZ+P/cIFgqPs/5ctfMCwFRWSw8dgQ4AMC2VG+NWWipZ7H3qCqa1v/XB3qqzR6ZZey4jKXp9uHKK4SqJiTjCSyKUwwQfAKY7Ah0AYFqKM63/0Qd3RpYCmqT2GVk9v6v1JwWJEg5J5WZqrPa8JMeOlWtD1kxfPvm1hDkA0xqBDgDQ0sK9OMUZC7sqlPKVWxS63LT+U1XpWLZw72L4vTvg3B/EGrNX+rwkMfU+AJRHoAMANEUSPThRf+gX5TKm3Wa1aUf/YEvPCpnPZfWyl8zUQ0/1R+5vz2U0M5etuB5cLmtjgmpxIpJiSWil9z9OD13pgtv1wGyRABCNQAcAaLioIFauh6jc86uNrWpVL5mZ1TM7h2XSaG/i6rsfGxfISnunygWvyU4YUik0l2sLAKBxWLYAANBwUVP1F+NKT9+Alq/ZJEmRi2j39A20dI9bOXPac3KX+gYGZSY98PljNSuX1ZErbhlXBjowOKyV6zaPeX/KTURSDG8TDVylC1OXm/ofAJB+BDoAQCzVyuO2VulZGxgc1lmrN+rCG++XmcaVE7Z6mOvI57TxgneN2/7BK+7Qr/7wlPbfs12zcllJ5d+r0u2lwSvJsMVaagAwNRDoAGAaKV0TLG6vTGmJXrjHTSoEjriBrBXXayv2HhYnaInydJnXVRza8MhT/TpyxS1atni+9unIR5ZS7tORH7eN4AUAqIRABwAtaiILYIdDWThYlQa08ELZ7bmMXhga0UhJjin2uLWyfC6rWblMxclHSsf/lRvXFhXG1m7o0d0P7xh9XHyfT3p917gZOYullAAA1IJABwAtqFKPWblQFzXGLaxcQGvFhbOjAmiUi088TJJ09uqNkT2MUTM9VhrXVmrlus2RY+VufbBXF594GDM8AgAmjUAHAC0oKpwVA9nKdZtHw8FUnjkyrHRWxmqzOEqFsFY8vvuR7brmjkfHhLpyIa2WcW2VxspRSgkASAKBDgBaUKUJSHr6BnTW6o06b829GhzxKbeodpz11sKhK2oGzdKwdtGSw7ToFXvG7jGLG8ZqGSsHAMBEsA4dAKREuTFxUROZVBrzNdUUFxOfzJT6zVrAOqqnkDXeAADVsLA4ALSAcMiYlctooGSsWrEnqnTyjOkkzoLkadesMAkAaF0EOgBIuThjvKTK0+SnWdxFwivNMhk1IQkAANNBLYEu1hg6MztW0lclZSVd4e4rSvafKWmlpJ5g09fc/Ypg3xmSzg+2X+TuV8W5JgCkUZzelqhjJI0pm3zmhcFYszA2MsyFQ9ic9pyk6qWdxfJBafwkIeUmY8maacR9zHsTd9ZIAAAwVtUeOjPLSvqdpHdK2iLpbkmnuftvQ8ecKWmRu3+85Ll7SuqWtEiFvxPWS3q9u+9QGfTQAUiDcqEsqletPZeRlP7p/Sv19kX1hh1w7g/K9rKZVPNi5FL58WOUJQIA8KKke+gOl/QHd/9TcPLvSHqfpN9WfFbBYkk/cfftwXN/IulYSdfGaVyaHHXUUeO2nXzyyfq7v/s79ff36z3vec+4/WeeeabOPPNMPfnkk3r/+98/bv/HPvYxnXLKKXrsscf0oQ99aNz+c845R8cff7w2b96sj370o+P2n3/++XrHO96hjRs36qyzzhq3/4tf/KLe9KY36fbbb9d55503bv9ll12mBQsW6Kc//akuuuiicfu/8Y1vaP78+brpppv05S9/edz+b33rW9pvv/20evVqff3rXx+3//rrr9fcuXO1atUqrVq1atz+//zP/1R7e7v+7d/+Tdddd924/bfddpsk6ZJLLtHNN988Zl8+n9cPf/hDSdIXvvAF/exnPxuzf6+99tINN9wgSVq+fLl+/etfj9m/77776uqrr5YknXXWWdq4cezaWwcddJAuv/xySdLSpUv1u9/9bsz+BQsW6LLLLpMknX766dqyZcuY/W984xt18cUXS5JOOukkPfXUU2P2H3PMMfrMZz4jSXr3u9+tgYGxvRjHHXecPvWpT0ni3qvnvffR5V/Ud1Zfp51Dw5rZltV+e+Y1d7eZOuvSa7R8zSb9+Zff1cAf79Ljkj7wdVMmYxq2nF568uckSX2/ulYvPHLPmHNn8y9R538rvOYd/7VKO3seHLO/bfe5mnt84We7/aeXa9e2P43Zn9uzS3sd+wlJ0lM/+lcNbu8Zs3/G3n+hPd+xVJL05E2XaOjZJ8fsn9l1sOa87UxJUu/3vqjhgWckSXvOnqGDXrq79nnVG7Rhr2M0MDisJ667QD60Uxkz7dY5W0f9aOaYe2/7df+k/8/evcc3Xd/7A399kqYlpdByU2ypEzeHIiAo8zI5m8omohM75kHd2dU55tQxdcPL5kHk6AQ5R52bOxtzuJ1zdFgVC3gZbqCb6M9LsVAEh6AgtEXl1nJp2ibp5/fHJ9/mm2++tyTfJE37ej4ePJp88803n6Rp+L7z/nze785IYvA68OR/wckXXI6/zjkbF198MR78C/Cg7nb9e+/Bm/4N/iOdOHgg1PMaXz77B6iZdJHpey8A4KGf/ASXXnoBtm7davre7yvvPX7u8XPPiO89vvf43svve6+Q+VzsUwVgt+56U2yb0deEEI1CiKeEENWp3FcIMVsIUS+EqN+7d6/LoRMRpa+uoRkrNrT0BCydkSg+2HsU+450mvZ465YSkWjvzMAV+Xwo8quP80ElAXzj7ONRZSiLP2xgCQDglOMG496Z41FVEYQAUFLkx4kjBmJ4WUnScauHBuETImFbwO9LaSrk8LISTDq+AmefOAyTjq/A50YPTfHZERERkR03Uy7/FcA0KeU1sevfBHCmlPJHun2GATgipewUQlwLYJaU8gIhxFwAJVLKu2P7/TuAdillcvgfwymXRJRtdQ3N+Entxl5XbMRtIRE9u5wuemcAACAASURBVMIhj766A3etUpMpnv3RFIyrKk95TJwKSURElHteT7lsAlCtuz4KQIt+BymlPrf9ewCLdPc9z3Dfl90MjIgoFU6Bh3a7WZNptyqCAXREoujI0lo5idSDOrsG4/uPdPZc/v7/1OPWi05OORhz20CbiIiI8sNNQPcWgJOEEKOhqlheCeDr+h2EEMdJKffErs4A8G7s8moAvxBCDIldvxDA7RmPmoj6FbPG2vom00BisZLm1hBuemIDbnxiA6oqgjj/5BEJvdzSCeYEgPkzTsW2Tw7j4Zfe9+iZJUt1bJWGqZWauoZmPLJuR8/1PW0duH35JgBggEZERNSHOAZ0UsqIEOIGqODMD2CplHKzEGIBgHop5UoAc4QQMwBEABwA8J3YfQ8IIf4DKigEgAVagRQiIjeMlRJbQ/Ey+s2tIdy+fFOsKXfimjep2+ex13elFcQZjzd/5eaeNXdCALmesWnM3tmV9l+8emtSJjEUjmLx6q0M6IiIiPoQV33opJTPA3jesG2e7vLtsMi8SSmXAliawRiJqIA59WTTb5u/cnNPwDakNIA7Lz3VtECJXigcdWzO7VXcpQ8msxnMmQVuVr3erIIzq6mYdlM0iYiI8qqxFlizAGhrAspHAVPnARNm5XtUvZ5jUZRcY1EUor7DrA9ZwCcAAYSjzp89Ab9wtV9vUhEMYGBJUU/Qdf7JI/DSP/eart0zey2CAT++dkYVXvrn3owKkZy7cK1pU2+7IipEREQ5YRa4AcCqOUBY939XIAhc+lD6QV0BB4ipFEVhQEdEWWMVVHhNACjqBcGfVdNsjdtspRdTIlNp6k1ERJQzjbXmgVtREAhZrMwqr049GLN6nEwCxBxiQEdEnkq3dP3o257zbLqjE78AvIznqiqCaO+K4GB72HY/vxDolrJXlvRnywEiIup1HhgHtO123s8o1WDM6nHKq4Gb3kn98XOMAR0RecYs06NNHfQLgaiUqNIFC/ogwhe7PZcCfoFIVGYUSGrTEusamjH3yY0Id5sfjRkvogJXwNOxqI8qhPdkpmOcX4G0V7enEoxZPo4A5rem9/g55HUfOiLqw5xaApgVJdE+HrVgTas2Wf/hgYT2APlo3K2fdunU081qDZs2FbJmUhXuWrXZNEvnF4LBHFEhM07HatutrgO97wS6tyqE4KOQpPOetPodZOt348XfTfmo9DJ02uM9MM7d87J6nPJR6T12L8YMHVE/ZpZ96+3SbQoOqIIlZj3s7KYlWk0bFQB2LLwkzZEQUd4V+HSsvCvw9UmueREYuT1Gqu9Jq9/BaV8HNj7u3e9GP37hA6TJOUNwKFA80N3r1FgLrPwREOlIfSx2ZwHBocD0RfHHbawFVlwPRLvi++hfh17+hQQzdER9lNdropxaAuRDRTCAI50RREymOfqFwH/NOg0AcNMTG1IK7IaXFeOOS8aavl52r2FlRdC0sItVQ28iKhBtTaltp0RrFiQGDIC6vmZBrzopzogX2Sg3x+gJLCyyVlbvSavfwfo/Jgdd6f5ujOM3C+YAVcxEK2ji9DpNmAXs2Qj8v1+nNhY1AOubQgdUALfrdWDbi+p18xfHbx90HDDmYvU6LP8+EoLDAs/QM6AjKhDGbJo2zRFIDEjsplBqJfS12/R91XqDYMCP+TNOxY1PbDC9vVvKnudqtY+VfUe6TF8vJ3OnjTGtFmnV0JuICkQ/mo6VFf0hIPYiaHU6hlmWzcjqPWn1WlsFXfr9G2uBF26NB2GBgUBRCRA6mNhG4JlrrY9nJxxSQdOaBeaZryEnpH5MN6JdQP1S9ARq0c74bWMvA97+k+61NgSHBfyFhC/fAyAid8yyaaFwFItXb+25rgV9za0hSKhG2Afbw5BQAeD/vb4r4bZcGlBk/nHjFwICqhCJ1jzbij4rVpVGhsz4erlRM6kK984cj6qKYMI4uXaOqMBNnaemX+kFgvET2WxrrFVT7OZXqJ+Ntbl5XK9YBRl9ISDWfjepZsxS2Vfbbhbw6fkCQNdR8/eJ1Wst/Obbtf0ba4G66xJbBISPxq5L9bzrrlPZrnSCOT0t82V8f297MbPj2rLI4q3/k/1rDRTsFxJcQ0dUINys5cpV37d0DB5QhHBUOvZFs3oOAsADV0zs2Tfd9X9c+0ZUoLKx3qX+UeDZG9Xl4BBg+n25aWDcF9afNdYCdT8EuiPxbdpzAKxfi96ybslqHM/enJjhMSP8wFd/mzxus2Pqs2B6PWvObIqDlAxW68z0a8C0+05fpC4nZfcs1phpa+u2vZh+QRIzRUF1bKv+cT3D8gFSxl+XlXOASC88X+lFa2jZtoCoD7IKdLQ+aL1xCqXRN84+vmfKp9UaQLvedTsNgZhdo26rwFZrSUDkqd5yktpXZSsAan4b+P356vKUm4Avzc/N+DItyNJb3m+PfBloelNd1ho/A9avhd1tqVZxzIRdMRGnYM5q3GbH9AUA2Z2c5RJ+wOdPDtSMisuAriP2jx8NAyuucx7v6C+q35VThipVwg+c8R1g/aPquboRCHo/Di/4i4HLHu41n90sikLUS2VS1GTutDH46ZMbk4qFaK0BvAjmtOAQQn2RZqciGMDRrkhCyX8nT69vdpyuaFWExGyKZc2kKssiJ2YZPK59o6xg+fvsy1YBjgMfqJ/CB7Q1ez++F241H18m68961fst9vk/8BgVaFkV9tB+V9plq9uMgRuQ3nN1CgLtiom4LbdlfP+9cGvyMbst/l+WUSDqYnaJVTCnf/xv1rkb7851mU+fNCOjsSIsLoM5oHcGc4AKoAv0M5tr6IhyxLi+rbk1hBuf2IBJC15EXUMz6hqace7CtRh923M4d+Fa1DUknlzUTKrCMWXF5gf3yH/NOg07Fl6CB2ZNRDBgMQcfakLHhjsvxOLLT0tYW/bgFRPx4BXW93Wzhm3utDFJ908nEOPaN8oZu2CjP/NyjVi2CnBoAd1xp2V2LKv7hg6YP2+rtU/BIc6P5eb91lgLLBoNzC9X/xaNTu31d/u7O/ghAAEc/URNobObyte22+b3GAvU2najZw3X8u8Dy2en/relBbzGY/2iMv58rMaZasCj9UR79mbnKYfZ0LYbaPhfd/tmI5jLxbFzKXQw3yNIGzN0RDli1SLgYHsYc5/cmNDg2qqC5UeHO5Pu75WKYKDnsbSfP6ndaNocXCtOYpUhA6yrULY4rPHTjudFewa78RF5pj9U+0uV11mkbFWkPPABMLgKGHYSsPv19I9j1yj5hVuTs0VT58XKpht0HVGvnd1r5PR+0wpe6LNDWjl3wPn1N/vdLZ+txqtNq5wwSxXqOPoJUDUZaK53sR5KqIDVLPARfousjUW2zK65tFWRka6j8ftm1NHUZCz1S705lpHwA0XF9hmtVx/MzmPnira2TvsyI3Qw8XL5KPW7swuYA0FgwBDgcEtmYynggj5cQ0eUI3Zrw6xUxYKZ+Ss3Z3V9nFlxEsC88IjVvkZWa/64hq0P6i3refKFDaqT2WVB9Gut9O+bky6M944yK6Sx8gYgovtSK9M1dPqmwyWDgM4jwLz9am1TOscyC9DM+AJASZl1NsDpfeP02tqd/GrrnaxeZ6fj6w0oBzragFNmAO+udN4/Pgh4FkwZjyn8fSdbpAkOyX7mSN8UXBQB0nC+4QsAQjiv+UvXzN87T6G1Kv6ifZ5sXAa8vyb9MfTCokQsikLUi2jr5tKtPhnwCYRNmmxnQgCoKA309Kezy4Clu+4vk2CQCkhfqNaXKb4GyeZXwPak3c0JovE1fOFW4I3fxm4z6ZnlpuKgXQELAJi+GDhrtrtj6Pf5253AoQyzAz0EML/V+ubGWmDFDYn9tTJ9vEyCIV9RYqXLPkGoH8asUaRTlffPyeMb/n5O+AKw8x/eP5T+78z0vSWAyVcDx58dXyPpdeDs5ssvp7/nFdepAjEaX0B9UWOW8TP7QqkXfhHJgI4oR5yCnXRL62dTLoOqTIrAUIFgdkrRn1AMqAAuXuzdyYEXGdBcH8NtlseJvjz8ugeAv8232hEJ39YD9kG22wyiU6Dupil0OqyymNprXnc9sOH/vH1MSjT5e8BX7k/clun72lcMdDt8iVFk0QJA+NRni5dr9fRTaIHUPs+9+hsH4PglhpM++v8QAzqiLNEHKOUmVR6179T8QiAqZc/PbEn1+FUMqshrlpmYDP+DLkQLPwV0tAL/8lNg6r97c0wvsn/ZOoYxiNIf69mbgfo/uDu2o9jjOJ0Ma+xOirUTPKcMoptjAB6f1BqYZTG131vDY8COl7PzuP2J8NlUZxTAzCWJ72un943+vsb9fAG1zZjNLB4IdLXHA/bls60fwxewrpxpZPceBsyDnVQ+z91+mZEwnVOYv96ZBl599P8hti0gygJjts1sTZv2caIFWdkM5rS1aG6biXPtGmVFtopVaAplfV77ARXMAUD7fu+O60W5frfH0L/WZkUJkk7cYp9vbbtVEY6eBsper5GKHctNMAeocVqdZLY1qecpfPZTxmyPsVudQNoVQvGC2Yl7OOR+rV5f4C/O3rotwKHUvkz+G3H9Oze+/4UqbtJlMl0zOBT4mW66rlXrB0C9JwIDgXC7yWPEHseYqTYWyAHU66rdrpfK57m+CI32mdF1JPkLiOmL7DPagaD5WFKR7f+HCgDbFhC5ZFWlMh8CftFTxt+szL8R+69R1kydBxQNSNzm9j9ofWn0RaNjZdZ1ZdLNyo+vmpNZ+XuvGMu6v7kkftuejbkt1+9UYt7q5NB4DP1rHToQC85ir7vTNK/usG4fF8Gcln3KteAQ9TwzXv8jddUSM5XF1yE4NJYZypNAUPWoS1fpcJVl+uoSdaxcM/79TZ2X5jikeTCXzmOE21XmUJj9vy/j2a4Js9S/mt+o94EmONS6ebbZY9t9nk+YFct4twK37lDHLa8GINRP4yyACbPUNrt90pHquPsgTrmkfsvt+q66huasV5lMxZDSAO689NSktXr653L+ySPw0j/3cu0a5SbD9dIvgL8vUpfNpt9Zjct2uo4AikvNT4K0x3D7vLx+DezGHigDIu2J3/zrF+en+vhOa0PsplMCuqyZiVxNHTQqCqox6sdVXg2c9GXvyr8Hh6rXJGJ4XYTP+sTaM6lkKGMFJxqfsG8indYwYusP7d4DqSgZDHRH7YuCaIGD/r3e8H/Ajr+n/7gjxwPXrotN412KxNfW6bXOMFtsNhVQ/3liNYXQi8d45lrzLx4cpw1nOM2wUGZFGBXquG1wDR2RA7tiJT4BdEvVl60rEkV7OMMPaxNVuqCruTXkai1cwC+w+PLTGJj1R+n+R5Wr6ovvPA08dbW6fMsOoHSo87i9DiCMldqcpgFl8hpkOvZUHt/pd2g1FttpWTr5KPNufEzhA776O+BTnwceOBXwBxKr1aVKe332vge8sji2bSBw2pUeruuzUF6t2i9sfNx5bZG+4MvS6cCu1zJ7bF9AvZZahcKxNcCsPzms+9Kygy7PBQNB4LSvJz8/q/e0WfVBt7T3yakzgX991PnvTnvt9S0Z3P4u1AMi4XVw83dqt7bUKDhUfcHg9vM43b/9Ai8EQnGpBHScckn9Sl1DM85duBY3PrHBcvqk1iGgNRTOSjAnALx62wW4u2Y8Xr3tAuxceAnev/di20k3Q0oDDOb6q0ymHdqtnfKSvly7dtls3MtnA/PL1dRKr7NB2vMymzpoXIOT6WuQacPwVB5/wizgS7p9iwYkngBajSV8FK5O0vMdzAEqw7FmAdC6S10/+zpVRCFd2ut7uFldHzxKBTl2wVxwKFCWwdRAID7F6yv3q9+RcSqykeyOfwHRvD6zxy6vVlPrLrw7vu2US2O32a0jkurfgHJ3jxMOqYDJ7bS5NQvSC+b0U3K3/029Rk5/d227VfA2dZ7KUN30Tvx3UV5tf9/yajWNMdWpgGZTCCdfbT79b/qi1KYbOk1P5DRD0mFRFCoobqZJWu3TW1oIVFaYz42vrAiyEXe+9cYpG5kUxXBaf5XJ89XfV3/yfagFGDnOfNxagOFl2W29tt0Wj2u2r/E10PVWcpo26kUxjLbd6ht2s2baQOLv5dSvqm0Vn1J9sNIq0uCxytOBlrfd7avvU2YVQLY1Aa2x5zHpmyqj9vIv7I8bCNoXLmmsVb/TIx8790mbvkjtU/dD+/3s6P8uJ8xSaynfegSIdJjvrwVaaxak31POX5y4HmqtLqBbfbsKGqfOc65GeNpVwNt/cv+3oz1HN/tascwOi1h1z1gg2HlIjT84xPmzw+yzUT9Ws2mbWhDk9jkZmd2vp2ebyWdrKo9hNyZjUZLe8n8W5QWnXFLBMAvItIkN2hTGZzfuSVrrJgD829nH47nGPTjY7u06OG2qZEUwACGA1vYwfDbTJ+16wLERd55le3piusFTJusk7KbkmJ7k2ZShNz4XqxPErzwITP5uCuW9HYgAIN3+3aawXsZxqmFsbZOxDxWgnn+2Kg3aNdz2DwCiFsFBrvlLnIMQ7X209QVg8/LYNFCL9Vfl1cDp3wZeuhv4+UfApqeAlTfESrq7WEfpVLXSTnCoCug86Sen+7t86w/Aczery8Zy8/rPlnT/VrRx6ysIrpyTvG5QW09ptSYLAH7wD2Dv1sTPqK6jzm0bnKT6GWRXat9syqIpF43ZGQRRAWDbAuqTzKpMav8FNreG8H+v7zK9nwQsb0uX3Xq20bc9Z3k/u+BM285G3DmUsLjd5IQw1fLwdo+jP3HRpk0CzsfOpByz1TfzXUdVoQS7MvR247PLgh3e464svBuBICAlEHEb0En3a8Ic95HxAh1aBk1fyh8+AN5PybbtMZWPYG5AhfpdRztVUYzOQyo4cRPMaSf9ezaqn3ZtB74wF2iuVxURA0FgcKXaXlKRHNCZZVTmV6T+3LRjTV/kPrPrRP93OfTE+OUzvw+8u8o8iLD8G49NE3S7TmrNgsRgDoh/hmn7WgWtFZ8CjjstuY1FpiXmrYI2/fM3BlfLZ5sfK3RQTYt0CuKdPhvTzcQR9WIM6KjX0qZONreGPO9qlAmzKpN6dlMnnYKzmklVfSOAK4RvQI0nK3ZTwTKVybRJuxMiJ9qxjd/Mu5nyGA6p+y2fbTIV0GaK3851wGsPpR7MFZcB0SgwoAw4uled2E+7J/VMmNUUrrQ+QWTiuquE1y12PLs+Wf4gEPUgSPBEmq9BR2u87P2Yi4HtfwVKhwH73rO/n/7v5sgn6qfdWqquI6rSY6RTZXU+d43afrhFrYETwv7zJJ2pp/pMtFUQkRKR+HepD+jG/ytw0b3md3P6G3f79+80xTqpb1iFCpIGVKjLRl5M6XM6hllwZfUZUz4qcf9s9TQjKkCcckm9Um9Z76ZXEQxgw50XOu7X76dO5qqyovExUz3pcFup0M30osbaxNLggYFAUUm8dLfl4+imBtk9B/3UrUApcOkvzavJWd1/wVBVbjwTdlMBNf5Ylb1IGuuBjpuo1hr96x+B35wNXL4UGPc1YMFw+6xVEpPApaRCZYfC7amPy83jTb46nsUbOAI4Ggtg8lFB0oxWTdEpGHcy/LNA5xH1PrdaF6bRt1VYcb1zc2jjFM6iYDzb9NmLgK8/YX//xtpYUObynGby9xKn02ZcddVkiu7GZcAzP1CXB1UCX74rvdYabj/fUq16+Nc7gVcfjO/TW754S+X/kEL48pAoTZxySQUvH028BYAdCy+xDMjmzzjV1XEKduqkV/8xZpKNSmds6U5ndJN5c/Ntb2MtUHddYtARPhpfK9TTfNjkRFObGuT0HI45JTaegWra35oFyZkzq/ufOjPzYA5wF1R1d6ew3s1ARtU0u7Jj1fXDH6vplsKXWFTD+UDJmzpbLZrwekGqYE47Yd7/PvCr02M39YJgDohXUwSST5RTCTr3vReb5uYw1VT/d7NmgcOXALFAzjiFMxJCz9/NYBefnRNmpZbN3fi4Klyhrxhot4bO9AsNmzWnjbXAszfGrx9usf9ccip+4eazM5VsfmMt8MZv49dTmQaebalkBjl9kggAAzrqJYyVKc2mLGabVn3Si4Asq1Mns/GNZCZrvIycpv14PbZ0A0inKVpFLrOKaxa4CHZMggzjSa/Zc9CmPGrTocqOAQ7uiO+jvRZFJtX+tNeg+iyHsXnIKTAIDk3uCaf1HPt4CzDxKhWw+ouBIx+pDGe0Exg/C9jkok1DJmPLhP69rfXg6020Lw6SptxpPfpSeG2sgjmrCqF2f/fl1cBZPwBevMPqwRLH76S82n2WzfgZYfbaAIkNsvW3O332ZuOLLSepBEJrFiRnWbM9vlQwUCNKCQM6yjtjRiwfwVww4MfcaWN6rvfatWxeBl56Xp58WJWWdjopswpUncaWbgA5dZ79N/pWRR+M40xnmlbZsapflNNJrxaEhA6qn/pgThO2qfrW1gQc3Kkuu6lK2MPjVatWTb/LRwHjZgKv/jKWoatSWZCyY4FdbwAbY8Ho+2tVMGj2vvJqWqPWlDidxtP69/Z7L9rv67pSn16sH1dwiJrOalYpMhBrHWG8zZih0Z8oPzDOYj2lSYNl2/EK4E6LdZl2BT9ueke9ty0DupjXfhVfP2XHTXl+PePfnZsgwu3noddfbLmVaTuBbI+PiLKCAR1lnVlfOCCeAbMr85+uYMCPAQGfbZuCqopgYU2JBLL3ra9X/7k31gKdh5O3+4vtpy6aBap11yWuS0saW6zPVLpVICfMUutbrDIOslsFfC/cqq6HDuoyGl3xMbgNfgJBINKlgo+LFwNjL0sca1Z6iUngz1epi1++C1j9c/vgR6tiCAnPgjpjaXXjCecn76qADohXNhRFQNOb8d9N+z415c1YgMQx0HCgrS3Tj2fbi6n9LvQBU2Mt8OyP7fedvkhdTmU9W/VZwPdWu9s3lQy+5d93bBqh/hh/nacqmJqx+1tzmgZYUg7H91rogLsvrqwyVHZFNrIlk+q0udDbx0dEKXEV0AkhLgLwSwB+AI9IKRda7Hc5gCcBfE5KWS+EOAHAuwC2xnZ5XUp5baaDpsJhln2b++RGQADhqPoP3OtgTqtCCQA3PbHB9DShYJt1Z+tbVa/+c7eaflhcZn8iZhaodoedqzGumgOc9nXg7f9J7vHktPat45AKGIz9oYz0YzAdj5v3r1DPr2gAEImqdVZAYlPrbNEyNn+/DzjjO2rtkLH33OSr1Xoi7aQZgGcZOmMZdaNBx+kuxwK6Ix8lB9rdYRUcFg90d7IOIDFQMMk6mU2pTSXLY5xeaFf63mydVd0P3a0NHDnOeR9NKlPVnLJnem8uUQGdWVBt97fmNA3Q5wMGDleVTX1Far1nJi1ErJ5/rqshZlKdNhd6+/iIKCWOAZ0Qwg/gYQBfBtAE4C0hxEop5RbDfoMAzAHwhuEQ70spJ3o0XiowZsVNwt3pnShWxTJpds3FjZm2+g8P4LHXdyWcmhqnVxYUu8Ark7V1Pb1/DCe8xv/crR7DWOXRSJsyaJRpQBMOqYzKpz4P7Pi77gafGo++cIixYMHqn6vL/hJ18uhU6CEjsddVW7Oy/W9qXJ40MnYpdEAFc6d9PV6RUf/aPDAuO2NxOhHf9iJ6/opX3qCmoloFgaGDwK0m006NRWkAlX3z+XXBhy7raFfRz3KdmSGIMQsGLb9YEeY9wyyDOaEet6NNvTffeVpl6bxeU+T2pL6xFvgoNn5/ifqCRr+2LN0gSxMcqgK6sTXA5X+w7imX7hdXXpTfL4THTEVvHx8RpcRNhu5MANullB8AgBBiGYDLAGwx7PcfAO4D8FNPR0gFp66hGfNXbkZrKM1qdya0ICzVgiV314zH5E8NLbyKk1asTsBOujCztXVjpiMhmLOq2mb2GLteT86QGRkzfU4BYCradgP+AYnV94xVJrVxbn4m+THDR1SWzlfkXFrdklDl6otK3AWnu15X/3JdBVELgM1KmGea5bVa42Z3bO09pb33jnwcL/JiFtSZZYy196j+/RSMFSZJGo9014bCGIC4/bIklUy33esdrFBBpH4NZTYqELo5qdd+R9rvo+uw+syZucSbsTTWxtd5bv9rZtOo7eSjyEZvL+zR28dHRK459qGLTaO8SEp5Tez6NwGcJaW8QbfPJAB3SCm/JoR4GcBPdVMuNwN4D8Ch2D6v2D0e+9AVtrqGZsx9cmPaWTg9IVTVcrPMW8EzO0EE3J006nuSDRwBTPuFzRoRlz3UVv9MfUMOoQpT3Lw5eT+rHkduilLM+DVw+jfjj+dpZipWMCLTKYI90/nSzBa6KefeK+h63+ll0odLe5+l2gfLav9AaXLPuFR7Gc6vgPl7wuL5eyGV/lnpvN5u/p69lurvNBVWr9dpX0+eGpztXpZERL1MKn3ofG6OZ7Kt539JIYQPwAMAfmKy3x4Ax0spJwG4GcDjQojBJgOeLYSoF0LU79271824qZdavHqrJ8EcoII5Y2auT9BOYtp2A5DxAiArrk/ctmqO2tfomLHxyxctzKzSozaWo9rfnQQONZk/rlMlRjuVk+KX7dYZaQbESte7IuHJeq/QQXWCOvP36d1fdmen15nxmIGg0x3sb7bKckyd5+LYJvRT9MyOYbcux+o9ZQzmgkNTP5m3ep7ZLPowYZYaZ3k1AKF+Wo07ndc7HxUIs1kN0arI07YX3b+ORETkasplE4Bq3fVRAFp01wcBGAfgZSEEAIwEsFIIMUNKWQ+gEwCklOuFEO8D+CyAhBSclHIJgCWAytCl91QoW8yqVGoBlv628mDA02mWABAKR7F49da+EdDZrRczm65otfZo/7b45SMfq5/pTlGyCq7+dlfytCuzQgWAfYZOq5r423PVdbtpeXr/FpuGlUqj4Ezpe3WlOx1URjOvvKgpKQc624AzvgtsW+2+EMj0RcDLC83HbxdcmU1dtCWSM8mprstxW93TqbCKmXwVfXA7jS3htXKZqctHBcJsVkO0CxY5HZCIyDU396YmrgAAIABJREFUUy6LoKZMTgXQDOAtAF+XUprMyQIMUy5HADggpYwKIU4E8AqA8VJKy7MFTrnMD6ugzVilUq8iGMDRrkhPtcpsEQB2LLwkq4+RdWlPMxRqrYr+BPmYU4APYgVAzvoBcOF/WBzfRQEIy2lpiJctNysKodGmR9UvTT6O8KttKU1D1FUiLB+Vu4yEcTpXJr+vyVenXvpeT1s/dtKF6jjfeQ44YUriPnbjKx8FTL3TvOebm6IHbqYCejX1z+5968VjZlIoKJfcvOb5mnKYyjTSVGVzOicRUYFLZcqlY4ZOShkRQtwAYDVU24KlUsrNQogFAOqllCtt7v4FAAuEEBEAUQDX2gVzlB9mrQVufGID7lqlYnazYA6A59k4K5UVaUwD623cTDM0ExySXIhEOwESfmD3m+ryhFmqDP/z+pnPMn4fq4IKdhkSbbtltkao4KN+qS5LF3tMy6IUDvRZwEyCOVEESBfl4IHkPmlAetkTAICMFx2xC5aTB4yE4PvZm4DdsYLBWm82PbvxtTUl/r5TPel2et29zHKZZfQs349pvB8KJctjlk30BYCSQalVk8yGbFZDZOl8IiJPOGboco0Zutw7d+FaNLfmqHR6ioIBP+6dOb5wp1xmUpY/EFQBk11QJHzAV3+nTq52vgr88WLrfY3fentZaRJIbtLsNqBxOw1T4wvYtxkoCqqKOto6LKspoWaBnJmUs3Wxohtui16YjeM35wCfxAoJ//xjIDDA+v5eZznsxm2X7fVKf83aFEo20Wv99XkTETnwNENHfV9LLwrmKoIBDCwp6hstBjKq5iicgzlABTXaOjv92jozbbvVybL27bfZ2IJDrHvGOZFRFSBqJ2Nu1kcNHgWceQ3wt/n2+2nTP/UVQc16j/mLgRPPA957QV0vGgBUfEqtx4t2pheQWGUoLCuLxtYWZZJ1Ka9WAV3pMPtgDvC+aIVV1iRX0/36a9amULKJXuuvz5uIyEMM6PoZs7VylRXBXpGhCwb8mD/j1MIN4IzSnWYJAJDus1Ztu4H55YhXN7RYg6Tta2wgnsChQqKT0AEVyE6YpU7AzYIuvUNNwN8X2R+z7FjrzIyx99j0RcCHr8Vvl1I1Zx44Ahj9LyqDmA6rk067wCOTqWoVx6ufg0ymWxp5XbQi3w2H8/34REREBYZTLvsRqwInpQEfwt0yK8VNggF/wuNpoUZVRRDnnzwCL/1zb26zcbmc3mM35dCsz1Zv1tNjTaimx3ZZPP3UOKdpnW56t110H3D2D9yNs7EWWHGDysYZfeEW4IKfuzuOW9l6Pz11NfDO0+qyU1Yxm0UriIiIKC845ZJMLV691bTASXvYXRXCIaUBHGwP9zT8dqI1BLdqeZBzxhNfu2IhTsexO4nXbrddP6ZlwnwACqAZtS+ggiSt4qUd/VQ/LbPVWGuerbML5nwBtf/k77gf55oF5sEcEM96eSkb08Uaa4F3V8WvO71PmdEiIiLq1xjQ9SOZrpXrCHfjwSsmYteBo7j/r2q9lpZxM07y0zcETzuA8zr7YdXE1qzXm92Y7IJCt+vmwkdjFwwBjb8EiIaTt+ebFiS5mQZqNtUv3d5uxYOAohL3+9utG6uotr6tN1mzILlFhNP7lOuQiIiI+i0GdH2Itj6uuTUEvxCIStmTJauZVJXxWrlQOIr5KzejIxLP8kmo4O1rZ1R5O33Sq2yanhfFI5yCwozWzQEoKga6I4C0WQeXLW6mPzqxK17htthKIAh0d6sgstTFGjI9u0Is5QUS0Hld5ISIiIj6NF++B0De0NbHaQFbNDYnsrk1hNuXb0JdQzPmThuDkqLMfuWtoTA6DFM0Q+EoXvrnXrx62wXYsfASvHrxPtS8PE2tIXtgnArOUmUXOFlprFWPN78CWDRa/dOPwapIhNl2/bH0z8HyZHu3+zL1djoPq2qRvjx81yK7VTCVrvJq+3VbVq9/cGgs2BLxYwz7tLpt4PDUxjB1nvVz+J8Z6b0Xcy2V9ykRERH1e8zQFbC6hmbMX7nZscF3KBzFT2o39gR5bmgZPrd6pnN6lVmzC5wWjVbZnuAQtU273HUkPlVNP7VPG8NpXwc2Pp4cKHYdjVdmdHoOjo24HTJr5dVA+wHdlEsDrSebXWVIR8YxuMz2acU3Uu2b57YAh1U5erNecJueipftT0VSw23dczc23O6t+mvZfiIiIkoLM3QFqq6hGXOf3OgYzGlSCc6CAT+uOqsawYA/afuQ0oDpfSorYlmRdDJrZuyyEaED6Cnrr79sXHdkHMO2F1XgYSzNHzqgTqC17I3dczjpQoeBOxRCuekdYOiJ6qoxCxcIAhO/4XB8qMArONT6tplLEjNePddtaAHDhFlqjHb7+wKxxxfOWTm9CbPUvsZsnNl9tQImpSlm6LTH6XkOht9HOu/FXEvldSIiIqJ+jxm6ArV49VaEu71fYzWkNIA7L1W94CZ/amhShUoASa0PtAIoaKy1yV6luP5n6jxg5Y+ASEe6T8V8DKfMAPD95NvCoXhTbLvsYP3S9B9fC1IHVwIfbwJO/grQvD6x6MuwzwD/7yG1n69IrafT6DNhVqXqtaDMePK/fLbNuEzK4ptliYB4r7d0gwu3xTu0DOvGx4Gdr6RXEKeQ16KxyAkRERG5xICuwOgLn2RDaXFRTzETuwqVSa0I/K/Gp7OZcVr/Y1bR8qxrgVcfTPepmI/hwPvWt2tNse2mVaZbqEQLthprgQ/XqW07/g5Mvy/xxL11V/zypG8C2/9mXuUz1VL1ls2nq82bduezFH6qZfuteN1wm4iIiKgXYmPxAmLVGNxLAsCOhZekfke7giBOa6yssk2n1ACNf059LHZj8BcDT37bZkeh9vWs6beIB0OAcwPornbgF8epy995DjhhijfDKKTm01bvJavg00ohPWciIiIiHTYWL3BaFs7YAsCqMbiXetbCpcpuGpvTCbTVmrV/ror1ZeuCq8yY8KsKkUbFA4GvxDJ9K22yiIB6HK+COWMA8sA45z54/3wWPYU8nr4G+HIKPfLsFFLzaa+mShbScyYiIiJKEwO6XsaYhdPaDgCZNwbXqwgGcLQrgnA0Hij1rIVLh92UPrvMnF1Fxa4jQOXpQOWk2No1m6AuOFQFYpFoLAiMNcL2FQHFg9Nrap0JX1FyVUKnQEXLKGnP8/Aeb6syFsq6LC+nShbKcyYiIiJKE6tc9jJmWbhQOIrFq7daZs+EMN1sKhjw48ErJmLDnRdi8eWnoaoiCAGgqiKIe2eOT60ZuL5XW9dRNZ3RyKrUuha8OJXHb3lbVaecfHW86l/JYN0OQv2LhOIFVLRgDlBFRY7syW0wBwCnzkwOJJz6i3lVIbTQmfWSY9l+IiIiIlPM0PUyVlm4ltYQHrhiIm58YkPSbQJAkV8kZNvM6CtYAvZFTxwZ1yeFDqhy9sKnGlSXDAY6DwHjvmZe8MQseLHStltVO7w0Vv0xofhK7Dm7PZZnHHq7vb82sbcd4NxfrJCrMnqJUyWJiIiIXGNA18tUVgRNK1hWVgTxpbHHmt6nWwKDi4swsKSoZ93d+SePwEv/3Ju0Di8tbgMyrRn2564Bhn8WeOEWYP0fgRd/ntike/lsOPZrs+sf5mXwFhyqGpOnXL1Swjaoa9+XPF3SKVBhVcY4TpUkIiIicoVVLnsZtYauEaFwd8+2YMCPr51Rhb+88xH2HTFvnp12dUonVpUC7YKqky8Fxn0VeOpqoGwkcOQjjwajzS316D2rVTzc9brzGj0r5dUqOBM+84IsqVRmZFVGIiIiIkJqVS65hq4XCvjji+IEgJmnV+Lp9c2WwRwQq06pX9P2wDh1PVNW67qE3/o+O18BSoery0c+Tv0xA6Xm28tHeZetCg6NB0pfuR+YuSS+Rq+8Gpj5e/WvvNr6GFqwNr9VTTM1k8p0yQmz1Jj042AwR0REREQ2OOWyFzHrMycBPPaGfeGQYMCPB8duA1bdmTi10YsKiVYBiVk2StPRCgyMBXSlQ4H2/ak95rCTgE+2xKdwAolrzYxZrB6xKZDBoeqqXSGUW3ckXrea4jdhlnXmTF+kw6vpkpxqSEREREQpYIaul6hraMZPajem3GdOq075ufd/lVmFRKvsnmVAIlQRFDMDR6h/AHDShUBRibsxALE2A6XA6C+gp4KlPlOVkMWKjaOHVIHW9EUqYLPKrtll3cy4yZyxMiMRERER5QEzdL2AlpmLpriesaoiiFdvu0BdWZFBhURjBkqf3Zs6D1j+fZM7aUVBTAqDnHNDPEvWeQQpfW8w4hRVpGRwFTDkBODHyVU9ewK7B8YlZ8X0jbqdqkqmwilzxsqMRERERJQHDOh6AbPec04EoKZZPjDHoSiHiyl/dv3PbnrHIqAD0N0FVJygpkYealbr6mQUmPRNwF8EBAYC7z0PdKfw3A68r7J0ZcfEs3xWnMr85zrI4nRJIiIiIsoxBnS9gFXvOTuX+tbhc5sejQdiZsGcXTZK34rAqrpj227VdsCKrwg4bjww63+BhcervnPCBwSHqNujnakFcwAQblc/920Hqk6339fNujUGWURERETUh3ENXS9QUWqxFs3EDN86rCueg18W/8a+dYC/2LpCojbFsm03HEv1P39L/Hh6gSAgitTUSCGAIZ9S24NDAV/sbdUdsT5ucGh8WqaZw3vihVWscN0aEREREfVzDOjyqK6hGRPvehEH28NJtwX8At84+3hUBOPB3gzfOiwMPIJRvn0JpUCS+YBol2rgrS9wohU+Wf599825o53qZ1EwHoAVlwEXLQKiHcDgSrWtIhbQadMkG2sBq1GWV6uiJbfusN4HEhh4jP3YWOafiIiIiPo5TrnME7MWBXoDi4twd8143F0zHnUNzdjw3BLcEf4tioRFvzO9njolMl7gZNfrwMbH3QdyRp1tKvs1cCTQcQBY9WO1/eAu9XPICbGBD49nAM2yf27L/QPOa+gATqkkIiIion6NAV2eOBVCaQvFs3Y1zf+FmshSQDhMj/SXqIyascl1OKTWwtn1jnMjHEoOCBv+Fzj+rHivuZ2vAB++Zv5Ywm9e7t+qr1yZi4COiIiIiKgf45TLPHEqhFJZEVsb1lgL1C+F41q38mrgjG9b3+4UzBnXorkV7QRW3Qi887TzY8nu5Gyacdpk2bHx29xk6IiIiIiI+jEGdHliVwglGPBj7rQx6sqaBXAM5gDguteBoSda3y78Nrf5gC/FGpAHSp0fyyh8VK3Zc2LVQmHCLNUeYX4rMPvl+HanNXRERERERP0cA7o8qGtoxpEO8wqQQ0oDuHfmeNRMqlLZOav1ZZqiWGZt/3ZVGVL4k7NtvgBQNMD6GLIbqJykLk/+nk22zr4Uiy231Se1lgeAc5VLIiIiIqJ+jgFdHixevRXh7uSsW0UwgIZ5F8aDuVVznA928iXq575twOGPgPIqNYWxZJDaHhyi2gqEj9of589XqJ+nXambAol4Zq+8Gph8tYtnpyP8SLn65Lur4pd/94V4hU4iIiIiIkrCoih5YLV+ri0U1jX8dsjMaQYMVj+XX6OKopSPUoFT52HguZtVZi500Pk4WlGTpnpg8nesg6/NzwChAyY39JTWVALB1FsIGINYrUInwEqWREREREQmXGXohBAXCSG2CiG2CyFus9nvciGEFEJM1m27PXa/rUKIaV4MutD1FDwx+HbZm7qG3zYm/lv8cv3S+OVoJ3DgfWDRaKA11k7g8EepDe65m+2zYtMXmTfznnx15v3g1ixIrnYZDsXWERIRERERkZFjhk4I4QfwMIAvA2gC8JYQYqWUcothv0EA5gB4Q7dtLIArAZwKoBLA34QQn5Uy0/r5hW3utDGY+9RGhKPxjFYw4MctgSeAkEOfuOBQYPNy+31CB4A3fhvbf4hFRs2CjNpnxbRtaxYAbU0qIzh1njcZtLam1LYTEREREfVzbjJ0ZwLYLqX8QErZBWAZgMtM9vsPAPcB6NBtuwzAMillp5RyB4DtseP1azWTqvD5Tw8DoCYqVlUEce/M8SgNOWTTtMyYm+bgkdivofx48+MEh1rf1ykrpq9KedM73k2HtKqCabWdiIiIiKifcxPQVQHQzwFsim3rIYSYBKBaSvlsqveN3X+2EKJeCFG/d+9eVwMvdN0SuGH429hx7K14tWMmal6cooqX2Ln0IXfr4fQ+2mDYIIDTvm4+dVIvH1mxqfPMp3O6qY5JRERERNQPuQnozKKMnrmCQggfgAcA/CTV+/ZskHKJlHKylHLyiBH9o5n0Cc3PYc7RX8XWy0k1LVJ2W9+hvFplwjLOVklg24vxht5W/enykRUzNhlPdy0eEREREVE/4abKZROAat31UQBadNcHARgH4GWhMkwjAawUQsxwcd9+p66hGYv+8k88GXkMxb5Od3fSZ6mmzlNr3NxMu7SiZd+0QMl4vHxmxSbMYgBHREREROSSmwzdWwBOEkKMFkIUQxU5WandKKVsk1IOl1KeIKU8AcDrAGZIKetj+10phCgRQowGcBKANz1/FgWirqEZty/fhD1tHagU+9zdyZilMstizfy9+qdtczymLvvGrBgRERERUcFyzNBJKSNCiBsArAbgB7BUSrlZCLEAQL2UcqXNfTcLIWoBbAEQAXB9f65wuXj1VoTC6um3yOEY5RTUlVeroiNGVlksbdsD46xbH5hl35gVIyIiIiIqSK760Ekpn5dSflZK+Wkp5T2xbfPMgjkp5Xmx7Jx2/Z7Y/cZIKV/wbuiFR99Q/L7ILLTLYuudM5n2aFZcBFCVLZl9IyIiIiLqM9ysoSOPVFYE0RwL6lZ2TwHCwKLA7xEUYcA/AJARoDuiMnOZ9HbLZq84IiIiIiLqNYSUSUUn82ry5Mmyvr7eeccCVNfQjNuWN6IjHK9m+XzJzzBW7AQmfgNorgeGfQa48rH8DZKIiIiIiPJKCLFeSjnZzb6uplySN2omVeGaKaN7rldVBHHigCPqStvuWDat2uLeREREREREiRjQ5VBdQzP+9/UPAQAjBw/A3As/gwFdB9SNn7wLdB0BypP6rhMREREREZliQJcjWsuCtlAEAPDRoQ781zOvqWbigVLg6Cdqx3w09CYiIiIiooLEgC5H9C0LNIMj+9WFykm6jQzoiIiIiIjIHQZ0OaJvWaAZIQ6qC1WnxzcyQ0dERERERC6xbUEO1DU0wycEolJihm8dbimqRaXYhzaUqR0iXfGd//AlYOqdbDFARERERESOGNBlmbZ2TgvmFgYeQalQAdwQxCpc1i+N36GtCVg1R11mUEdERERERDY45TLL9Gvnbimq7QnmEnSHE6+HQ6opOBERERERkQ0GdFnWrFs7Vyn2ub9jW1MWRkNERERERH0JA7osqmtohtBdb5HD3d+ZxVGIiIiIiMgBA7osWrx6K6Tu+n2RWWiXxc53DASBqfOyNi4iIiIiIuobGNBlkbFVwcruKbgtfA2ktLgDAJRXA5c+xIIoRERERETkiFUus6iyIpiwhg4AXu6eBCEs7lBeDdz0TvYHRkREREREfQIzdFk0d9oYBPyJ0dspgY/VBb9h6iWnWRIRERERUYoY0GVRzaQqTPmMKoQiAFRVBHHL5NhLfsG/q4wcBKdZEhERERFRWjjlMsu6JTD2uMF4/sf/AjTWAs/dq25483fA1DsZxBERERERUdqYocuy9z4+jDEjB6lgbtUcoPOwuqGtSV1vrM3vAImIiIiIqGAxoMuSuoZmnHPvGuxp68Cadz9G+wvzgHBigRSEQ8CaBfkZIBERERERFTxOucyCuoZm3L58E0LhKADgUEcEA+RHgFl1y7am3A6OiIiIiIj6DGbosmDx6q09wZymRQ4z37l8VA5GREREREREfREDuiwwNhQHgPsis9ApDQlRtiogIiIiIqIMMKDLgsqKYNK2ld1T8JLvnNg1tiogIiIiIqLMcQ1dFsydNga3L29EKNzdsy0Y8GPM8SOBT4YCt+7I4+iIiIiIiKivYIYuC2omVeHGL53Uc72qIoh7Z47HaOwBhp9kc08iIiIiIiL3mKHLkmMHq2mXr16yD1X1i4EVTYAQQPU5DvckIiIiIiJyhxm6LKhraMYddZsww7cOQ9fMBdp2A5CA7AZ2vQo8e3O+h0hERERERH0AAzqPaT3ojnRGcUtRLYLoTN6pfinQWJv7wRERERERUZ/CgM5j+h50lWKfxV4SWLMgd4MiIiIiIqI+iQGdx/Q96FrkcOsd25pyMBoiIiIiIurLGNB5TN+D7r7ILEhpsWP5qNwMiIiIiIiI+ixXAZ0Q4iIhxFYhxHYhxG0mt18rhNgkhNgghFgnhBgb236CECIU275BCPFbr59AbzN32hgU+QQA1Uz8gByYHNQFgsDUebkfHBERERER9SmOAZ0Qwg/gYQDTAYwFcJUWsOk8LqUcL6WcCOA+APfrbntfSjkx9u9arwbeW9VMqsJnjinDV4texavFP8JQ31FEfSVAcCgAAZRXA5c+BEyYle+hEhERERFRgXPTh+5MANullB8AgBBiGYDLAGzRdpBSHtLtPxCA1UTDPk9KiTPa/oo7A4+gWKoKl0WyE4j4gJlLGMgREREREZFn3Ey5rAKwW3e9KbYtgRDieiHE+1AZujm6m0YLIRqEEH8XQvyL2QMIIWYLIeqFEPV79+5NYfi9S11DM865dy1+GH28J5jrEQ6xsiUREREREXnKTUAnTLYlZeCklA9LKT8N4FYAd8Q27wFwvJRyEoCbATwuhBhsct8lUsrJUsrJI0aMcD/6XkTrP/fRoQ7rdgWsbElERERERB5yE9A1AajWXR8FoMVm/2UAagBAStkppdwfu7wewPsAPpveUHs3ff85y3YFrGxJREREREQechPQvQXgJCHEaCFEMYArAazU7yCEOEl39RIA22LbR8SKqkAIcSKAkwB84MXAext9/7n7IrPQLosTd2BlSyIiIiIi8phjURQpZUQIcQOA1QD8AJZKKTcLIRYAqJdSrgRwgxDiSwDCAA4C+Hbs7l8AsEAIEQEQBXCtlPJANp5IvlVWBNEcC+pWdk/BcZH9uD3wBCQAUV6tgjkWRCEiIiIiIg+5qXIJKeXzAJ43bJunu/xji/s9DeDpTAZYKOZOG4NbnmpEV7QbAPCRHAYAWHv+Ckz94nl5HBkREREREfVVrhqLkzsiVitmhm8d7il+FBLA1LevBxpr8zswIiIiIiLqk1xl6MieVuGyMyoxw7cOCwOPoBRd6sa2JmBVrIsDp1wSEREREZGHmKHzgL7C5S1FtSgVXYk7sAcdERERERFlAQM6D+grXLIHHRERERER5QoDOg9UVgR7LrMHHRERERER5QoDOg/MnTYGfp8AoHrQdUrD0kT2oCMiIiIioixgQOeBmklV+MyIgQj4BVZ1T8Fz/qmxWwRQXg1c+hALohARERERkedY5dIjhzoiuHRCJe6/YiKw7j3gb6uB23cDJYPyPTQiIiIiIuqjmKHzwOGOMPa0deAzx5apDUc+AQIDGcwREREREVFWMUOXobqGZtzz3LsAgEde2YHK8iBqDn8EDDo2zyMjIiIiIqK+jgFdBuoamrHumd/gGSxDZck+tISH48FnrsSUEbswvGxkvodHRERERER9HAO6DGx4bgkWiCU9jcRHiX1YIJegfX8QGHlBnkdHRERERER9HQO6DFzT9X8o9XUlbCsVXQjKLqDjUJ5GRURERERE/QWLomSg0rffdLsQAHb8A2isze2AiIiIiIioX2FAl4GOoM06ue4wsGZB7gZDRERERET9DgO6DJROX4AuUWK9Q1tT7gZDRERERET9DgO6DNRFz8Xd3d+y3iE4JHeDISIiIiKifocBXZrqGppx+/JNWN81GgAQliJ5p64jXEdHRERERERZw4AuTYtXb0UoHEW1+AQA0I5g8k7RLq6jIyIiIiKirGFAl6aW1hAA9AR0g9BuviPX0RERERERUZYwoEtTZYXKyB0vPkGrHIgWOdx8x/JRORwVERERERH1Jwzo0jR32hgE/ALHi0+wSx6D+yKzEJLFiTsFgsDUefkZIBERERER9XkM6NJUM6kKN47YgM/73sF4sQM/K34SLSfMBMqrAQj189KHgAmz8j1UIiIiIiLqo4ryPYCC1ViLa9oeREB0AwBGYi/QsoJBHBERERER5QwzdOlaswAlsjNxWzjEqpZERERERJQzDOjSJK2qV7KqJRERERER5QgDujRFB1WZ38CqlkRERERElCMM6NK0bdzNiEjDy8eqlkRERERElEMM6NJQ19CMK1+vxi45Ap0IQLKqJRERERER5QEDuhTVNTTj9uWb0BbqwnBxCE9EzsPY6DLUnbeawRwREREREeUUA7oULV69FaFwFCNxAINFCO/JUQiFo1i8emu+h0ZERERERP0MA7oUtbSGMMO3Ds+W/BwA8OOi5ZjhW4eW1lCeR0ZERERERP2Nq4BOCHGREGKrEGK7EOI2k9uvFUJsEkJsEEKsE0KM1d12e+x+W4UQ07wcfD58u+xNLAw8guHiEABghGjDwsAj+HbZm3keGRERERER9TeOAZ0Qwg/gYQDTAYwFcJU+YIt5XEo5Xko5EcB9AO6P3XcsgCsBnArgIgC/iR2vYN0u/ohS0ZWwrVR04ZbAE3kaERERERER9VduMnRnAtgupfxAStkFYBmAy/Q7SCkP6a4OBCBjly8DsExK2Sml3AFge+x4hamxFiXhVtObSkMf5XgwRERERETU3xW52KcKwG7d9SYAZxl3EkJcD+BmAMUALtDd93XDfS06cheANQusb2NDcSIiIiIiyjE3GTphsk0mbZDyYSnlpwHcCuCOVO4rhJgthKgXQtTv3bvXxZDypK3J+jY2FCciIiIiohxzE9A1AajWXR8FoMVm/2UAalK5r5RyiZRyspRy8ogRI1wMKU+ssnDBoexBR0REREREOecmoHsLwElCiNFCiGKoIicr9TsIIU7SXb0EwLbNLbtYAAAVHklEQVTY5ZUArhRClAghRgM4CUDhloOcOg/wBRK3BYLA9EX5GQ8REREREfVrjmvopJQRIcQNAFYD8ANYKqXcLIRYAKBeSrkSwA1CiC8BCAM4CODbsftuFkLUAtgCIALgeillNEvPJfsmzAI2PwO59XlICPjKR6kgj9k5IiIiIiLKAzdFUSClfB7A84Zt83SXf2xz33sA3JPuAHudsmPQKirwk089haXf+Vy+R0NERERERP2Yq8biFNfS0oRPomVY+89PcO7CtahraM73kIiIiIiIqJ9ylaEjpa6hGVUtTYhiEACguTWE25dvAgDUTCrcbgxERERERFSYmKFLweLVWzFEHsJ+OahnWygcxeLVW/M4KiIiIiIi6q8Y0KWgpTWEIeIwDuoCOm07ERERERFRrjGgS8Go8mIMwRHsx+CE7ZUVwTyNiIiIiIiI+jMGdCm4/fzj4BMyIUMXDPgxd9qYPI6KiIiIiIj6KxZFScHFn1ZNxQ9iMARUZm7utDEsiEJERERERHnBgC4V7fsBAGVDjsWOn16S58EQEREREVF/xymXqTi6DwBQPHhEngdCRERERETEgC41sQxdacWxeR4IERERERERA7qUdB3+BAAweBgDOiIiIiIiyj8GdG411sL/2kOQEvhm/deAxtp8j4iIiIiIiPo5FkVxo7EWWDUH/nAIEEBpaA+wao66bcKs/I6NiIiIiIj6LWbo3FizAAiHEreFQ2o7ERERERFRnjCgc6OtKbXtREREREREOcCAzoX24MiUthMREREREeUCAzoX7gtfgXZZnLCtXRbjvvAVeRoRERERERERAzpX/nTkTNwWvgbdUkBKoKl7OG4LX4M/HTkz30MjIiIiIqJ+jFUuXaisCGJV6+fxkPgNfhmZiQcilwMAqiqCeR4ZEREREVFuhMNhNDU1oaOjI99D6TMGDBiAUaNGIRAIpH0MBnQuzJ02BvcsfwMAcEiqIC4Y8GPutDH5HBYRERERUc40NTVh0KBBOOGEEyCEyPdwCp6UEvv370dTUxNGjx6d9nEY0LlQM6kKwdDxwIvAEZSiqiKIudPGoGZSVb6HRkRERESUEx0dHQzmPCSEwLBhw7B3796MjsOAzqULRqvM3HnjT8SiKy/I82iIiIiIiHKPwZy3vHg9WRTFpdDhgwAAf7A8zyMhIiIiIiJSGNC5FDrcCgAIlDKgIyIiIiJyUtfQjHMXrsXo257DuQvXoq6hOaPj7d+/HxMnTsTEiRMxcuRIVFVV9Vzv6upydYzvfve72Lp1q+0+Dz/8MB577LGMxppLnHLpUudRFdAVD2RAR0RERERkp66hGbcv34RQOAoAaG4N4fblmwAg7ToUw4YNw4YNGwAA8+fPR1lZGX76058m7COlhJQSPp953urRRx91fJzrr78+rfHlCwM6l7piAd2AsiF5HgkRERERUX7dtWoztrQcsry9YVcruqLdCdtC4ShueaoRf35zl+l9xlYOxp2XnpryWLZv346amhpMmTIFb7zxBp599lncddddePvttxEKhXDFFVdg3rx5AIApU6bg17/+NcaNG4fhw4fj2muvxQsvvIDS0lKsWLECxxxzDO644w4MHz4cN954I6ZMmYIpU6Zg7dq1aGtrw6OPPorPf/7zOHr0KL71rW9h+/btGDt2LLZt24ZHHnkEEydOTHn8meKUS5ciIfWGLR1ckeeREBERERH1bsZgzml7prZs2YLvfe97aGhoQFVVFRYuXIj6+nps3LgRf/3rX7Fly5ak+7S1teGLX/wiNm7ciHPOOQdLly41PbaUEm+++SYWL16MBQsWAAB+9atfYeTIkdi4cSNuu+02NDQ0ZOV5ucEMnUvRUBsAoGwQAzoiIiIi6t+cMmnnLlyL5tZQ0vaqiiCe+ME5no/n05/+ND73uc/1XP/zn/+MP/zhD4hEImhpacGWLVswduzYhPsEg0FMnz4dAHDGGWfglVdeMT32zJkze/bZuXMnAGDdunW49dZbAQCnnXYaTj019cyiV5ihc0l2HMJhGcSgYHG+h0JERERE1KvNnTYGwYA/YVsw4MfcaWOy8ngDBw7subxt2zb88pe/xNq1a9HY2IiLLroIHR0dSfcpLo6f1/v9fkQiEdNjl5SUJO0jpfRy+BlhQOeS6DyMwwiirIRJTSIiIiIiOzWTqnDvzPGoqghCQGXm7p05Pu2CKKk4dOgQBg0ahMGDB2PPnj1YvXq1548xZcoU1NbWAgA2bdpkOqUzVxiduOQLH0E7SlHkZwxMREREROSkZlJVTgI4o9NPPx1jx47FuHHjcOKJJ+Lcc8/1/DF+9KMf4Vvf+hYmTJiA008/HePGjUN5eX6q4YvelC4EgMmTJ8v6+vp8DyPJe4unorP9EMbf+Va+h0JERERElHPvvvsuTjnllHwPo1eIRCKIRCIYMGAAtm3bhgsvvBDbtm1DUVHq+TKz11UIsV5KOdnN/ZmhcykQOYI230DnHYmIiIiIqE87cuQIpk6dikgkAiklfve736UVzHnB1aMKIS4C8EsAfgCPSCkXGm6/GcA1ACIA9gK4Wkr5Yey2KIBNsV13SSlneDT2nCqOHkVX0fB8D4OIiIiIiPKsoqIC69evz/cwALgI6IQQfgAPA/gygCYAbwkhVkop9Sv/GgBMllK2CyF+COA+AFfEbgtJKXPfYc9jA7qPIjKgLN/DICIiIiIi6uGmwseZALZLKT+QUnYBWAbgMv0OUsqXpJTtsauvAxjl7TDzr7S7HdHA4HwPg4iIiIiIqIebgK4KwG7d9abYNivfA/CC7voAIUS9EOJ1IUSN2R2EELNj+9Tv3bvXxZByrDuKIDrQXcwMHRERERER9R5u1tAJk22mpTGFEN8AMBnAF3Wbj5dStgghTgSwVgixSUr5fsLBpFwCYAmgqly6GnkudR5SP0sG5XccREREREREOm4ydE0AqnXXRwFoMe4khPgSgJ8DmCGl7NS2SylbYj8/APAygEkZjDf3GmvR/ZtzAADntvwRaKzN73iIiIiIiApBYy3wwDhgfoX6meF59HnnnZfUJPzBBx/EddddZ3mfsjI1w66lpQWXX3655XGd2qY9+OCDaG9v77l+8cUXo7W11e3Qs8pNQPcWgJOEEKOFEMUArgSwUr+DEGISgN9BBXOf6LYPEUKUxC4PB3AugPy1UU9VYy0iK34E3+E9AIBgpA2RFT9iUEdEREREZKexFlg1B2jbDUCqn6vmZHQefdVVV2HZsmUJ25YtW4arrrrK8b6VlZV46qmn0n5sY0D3/PPPo6KiIu3jeclxyqWUMiKEuAHAaqi2BUullJuFEAsA1EspVwJYDKAMwJNCCCDenuAUAL8TQnRDBY8LDdUxe7X2F+ahNNqRsK0o2qG2T5iVp1EREREREeXZC7cBH22yvr3pLSDambgtHAJW3ACs/5P5fUaOB6YvNL8NwOWXX4477rgDnZ2dKCkpwc6dO9HS0oKJEydi6tSpOHjwIMLhMO6++25cdllCDUfs3LkTX/nKV/DOO+8gFArhu9/9LrZs2YJTTjkFoVCoZ78f/vCHeOuttxAKhXD55ZfjrrvuwkMPPYSWlhacf/75GD58OF566SWccMIJqK+vx/Dhw3H//fdj6dKlAIBrrrkGN954I3bu3Inp06djypQpeO2111BVVYUVK1YgGAzav65pcNWHTkr5PIDnDdvm6S5/yeJ+rwEYn8kA82lA6KOUthMREREREZKDOaftLgwbNgxnnnkm/vKXv+Cyyy7DsmXLcMUVVyAYDOKZZ57B4MGDsW/fPpx99tmYMWMGYommJP/93/+N0tJSNDY2orGxEaeffnrPbffccw+GDh2KaDSKqVOnorGxEXPmzMH999+Pl156CcOHJ/alXr9+PR599FG88cYbkFLirLPOwhe/+EUMGTIE27Ztw5///Gf8/ve/x6xZs/D000/jG9/4RtrP30p+2pkXiJbuYRjl22e+PQ/jISIiIiLqFWwyaQDUmrm23cnby6uB7z6X9sNq0y61gG7p0qWQUuJnP/sZ/vGPf8Dn86G5uRkff/wxRo4caXqMf/zjH5gzZw4AYMKECZgwYULPbbW1tViyZAkikQj27NmDLVu2JNxutG7dOnz1q1/FwIEDAQAzZ87EK6+8ghkzZmD06NGYOFG14z7jjDOwc+fOtJ+3HTdr6PqtR4q/gXZZnLCtXRbjkWLvI2siIiIioj5j6jwgYJheGAiq7RmoqanBmjVr8PbbbyMUCuH000/HY489hr1792L9+vXYsGEDjj32WHR0dNgexyx7t2PHDvznf/4n1qxZg8bGRlxyySWOx5HSukB/SUlJz2W/349IJOLw7NLDgM7GxEtmY56cjabu4eiWAk3dwzFPzsbES2bne2hERERERL3XhFnApQ+pjByE+nnpQ2p7BsrKynDeeefh6quv7imG0tbWhmOOOQaBQAAvvfQSPvzwQ9tjfOELX8Bjjz0GAHjnnXfQ2NgIADh06BD+f3v3GitXWcVh/PmnVE5TEKRcUjlcQyOlSXvanDTESoJgFFSUIApNjdBAmlQTMfGGfjEa+cAXMQRj4oV4Cd6iFo0JCoF6iwqCgEKqEQlq09qWAnIpQcXlh3lbx3roKfRMp3v6/JLJ7L1mz86ak5WzZ+39vnvmzp3LEUccwZYtW7jllv/+tPbhhx/OU089NeW+br75Znbs2MEzzzzDunXrOOuss/bpM75YDrncgwuXHg+8m0t+dC6bnniWVx45hw++4VUtLkmSJOkFLX7HPjdwU1m5ciUXXXTRrjterlq1igsuuIDJyUkmJiY4/fTT9/j+tWvXsnr1ahYvXszExATLly8HYMmSJSxdupRFixZx6qmnsmLFil3vWbNmDeeffz7z589n/fr1u+LLli3j8ssv37WPK6+8kqVLlw5seOVUsqfLhMMwOTlZ0/0OhCRJkqT9a8OGDSxcuHDYaYycqf6uSe6pqsm9eb9DLiVJkiSpo2zoJEmSJKmjbOgkSZIk7ZUDbbpW183E39OGTpIkSdK0xsbG2L59u03dDKkqtm/fztjY2D7tx7tcSpIkSZrW+Pg4GzduZNu2bcNOZWSMjY0xPj6+T/uwoZMkSZI0rdmzZ3PKKacMOw3txiGXkiRJktRRNnSSJEmS1FE2dJIkSZLUUTnQ7lKTZBvw52HnMYWjgUeHnYRGmjWmQbK+NGjWmAbJ+tKgHWg1dlJVHbM3Gx5wDd2BKsndVTU57Dw0uqwxDZL1pUGzxjRI1pcGrcs15pBLSZIkSeooGzpJkiRJ6igbur33uWEnoJFnjWmQrC8NmjWmQbK+NGidrTHn0EmSJElSR3mFTpIkSZI6yoZOkiRJkjrKhm4vJDkvyR+SPJTk6mHno25KcmOSrUke6IsdleS2JH9sz69o8SS5vtXcb5MsG17m6oIkJyRZn2RDkgeTXNXi1pj2WZKxJHclub/V18db/JQkd7b6+maSl7X4oW39ofb6ycPMX92RZFaSe5P8oK1bY5oRSR5J8rsk9yW5u8VG4hhpQzeNJLOAzwDnA2cAK5OcMdys1FFfAs7bLXY1cHtVLQBub+vQq7cF7bEG+Ox+ylHd9S/g/VW1EDgTeE/7X2WNaSY8B5xTVUuACeC8JGcC1wLXtfp6HLiibX8F8HhVnQZc17aT9sZVwIa+dWtMM+m1VTXR93tzI3GMtKGb3nLgoap6uKr+AXwDeOuQc1IHVdVPgcd2C78V+HJb/jJwYV/8K9XzK+DIJPP3T6bqoqraXFW/actP0ftCdDzWmGZAq5On2+rs9ijgHODbLb57fe2su28D5ybJfkpXHZVkHHgT8IW2HqwxDdZIHCNt6KZ3PPDXvvWNLSbNhOOqajP0vpADx7a4daeXrA09WgrciTWmGdKGwt0HbAVuA/4EPFFV/2qb9NfQrvpqr/8dmLd/M1YHfRr4EPDvtj4Pa0wzp4Bbk9yTZE2LjcQx8pBhJ9ABU53t8bceNGjWnV6SJIcB3wHeV1VP7uGEtTWmF6WqngcmkhwJrAMWTrVZe7a+9KIkeTOwtaruSXL2zvAUm1pjeqlWVNWmJMcCtyX5/R627VR9eYVuehuBE/rWx4FNQ8pFo2fLzkv47Xlri1t3etGSzKbXzN1UVd9tYWtMM6qqngB+TG+u5pFJdp4c7q+hXfXVXj+C/x9yLvVbAbwlySP0precQ++KnTWmGVFVm9rzVnonpZYzIsdIG7rp/RpY0O6y9DLgUuD7Q85Jo+P7wGVt+TLge33xd7W7LJ0J/H3nkABpKm3uyBeBDVX1qb6XrDHtsyTHtCtzJJkDvI7ePM31wMVts93ra2fdXQzcUVUH7NltDV9VfaSqxqvqZHrfte6oqlVYY5oBSeYmOXznMvB64AFG5BgZa396Sd5I7yzRLODGqrpmyCmpg5J8HTgbOBrYAnwMuBn4FnAi8Bfg7VX1WPtyfgO9u2LuAFZX1d3DyFvdkOQ1wM+A3/Hf+ScfpTePzhrTPkmymN4NA2bROxn8rar6RJJT6V1NOQq4F3hnVT2XZAz4Kr25nI8Bl1bVw8PJXl3Thlx+oKrebI1pJrQ6WtdWDwG+VlXXJJnHCBwjbegkSZIkqaMccilJkiRJHWVDJ0mSJEkdZUMnSZIkSR1lQydJkiRJHWVDJ0mSJEkdZUMnSRpZSZ5Pcl/f4+oZ3PfJSR6Yqf1JkvRSHDLsBCRJGqBnq2pi2ElIkjQoXqGTJB10kjyS5Nokd7XHaS1+UpLbk/y2PZ/Y4sclWZfk/vZ4ddvVrCSfT/JgkluTzBnah5IkHZRs6CRJo2zObkMuL+l77cmqWg7cAHy6xW4AvlJVi4GbgOtb/HrgJ1W1BFgGPNjiC4DPVNUi4AngbQP+PJIk/Y9U1bBzkCRpIJI8XVWHTRF/BDinqh5OMhv4W1XNS/IoML+q/tnim6vq6CTbgPGqeq5vHycDt1XVgrb+YWB2VX1y8J9MkqQer9BJkg5W9QLLL7TNVJ7rW34e56ZLkvYzGzpJ0sHqkr7nX7blXwCXtuVVwM/b8u3AWoAks5K8fH8lKUnSnngmUZI0yuYkua9v/YdVtfOnCw5Ncie9k5srW+y9wI1JPghsA1a3+FXA55JcQe9K3Fpg88CzlyRpGs6hkyQddNocusmqenTYuUiStC8ccilJkiRJHeUVOkmSJEnqKK/QSZIkSVJH2dBJkiRJUkfZ0EmSJElSR9nQSZIkSVJH2dBJkiRJUkf9B1jiXBU72Vz2AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fe0ff682f10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the learning curves\n",
    "plt.subplot(2, 1, 1)\n",
    "plt.title('Training loss')\n",
    "loss_hist_ = loss_hist[1::100] # sparse the curve a bit\n",
    "plt.plot(loss_hist_, '-o')\n",
    "plt.xlabel('Iteration')\n",
    "\n",
    "plt.subplot(2, 1, 2)\n",
    "plt.title('Accuracy')\n",
    "plt.plot(train_acc_hist, '-o', label='Training')\n",
    "plt.plot(val_acc_hist, '-o', label='Validation')\n",
    "plt.plot([0.5] * len(val_acc_hist), 'k--')\n",
    "plt.xlabel('Epoch')\n",
    "plt.legend(loc='lower right')\n",
    "plt.gcf().set_size_inches(15, 12)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Different Optimizers\n",
    "There are several more advanced optimizers than vanilla SGD, you will implement three more sophisticated and widely-used methods in this section. Please complete the TODOs in the optim.py under lib directory."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SGD + Momentum\n",
    "The update rule of SGD plus momentum is as shown below: <br\\ >\n",
    "\\begin{equation}\n",
    "v_t: velocity \\\\\n",
    "\\gamma: momentum \\\\\n",
    "\\eta: learning\\ rate \\\\\n",
    "v_t = \\gamma v_{t-1} + \\eta \\nabla_{\\theta}J(\\theta) \\\\\n",
    "\\theta = \\theta - v_t\n",
    "\\end{equation}\n",
    "Complete the SGDM() function in optim.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [],
   "source": [
    "# SGD with momentum\n",
    "model = TinyNet()\n",
    "loss_f = cross_entropy()\n",
    "optimizer = SGD(model.net, 1e-4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "updated_w error:  1.0\n",
      "velocity error:  0.0011111111111110914\n"
     ]
    }
   ],
   "source": [
    "# Test the implementation of SGD with Momentum\n",
    "N, D = 4, 5\n",
    "test_sgd = sequential(fc(N, D, name=\"sgd_fc\"))\n",
    "\n",
    "w = np.linspace(-0.4, 0.6, num=N*D).reshape(N, D)\n",
    "dw = np.linspace(-0.6, 0.4, num=N*D).reshape(N, D)\n",
    "v = np.linspace(0.6, 0.9, num=N*D).reshape(N, D)\n",
    "\n",
    "test_sgd.layers[0].params = {\"sgd_fc_w\": w}\n",
    "test_sgd.layers[0].grads = {\"sgd_fc_w\": dw}\n",
    "\n",
    "test_sgd_momentum = SGDM(test_sgd, 1e-3, 0.9)\n",
    "test_sgd_momentum.velocity = {\"sgd_fc_w\": v}\n",
    "test_sgd_momentum.step()\n",
    "\n",
    "updated_w = test_sgd.layers[0].params[\"sgd_fc_w\"]\n",
    "velocity = test_sgd_momentum.velocity[\"sgd_fc_w\"]\n",
    "\n",
    "expected_updated_w = np.asarray([\n",
    "  [ 0.1406,      0.20738947,  0.27417895,  0.34096842,  0.40775789],\n",
    "  [ 0.47454737,  0.54133684,  0.60812632,  0.67491579,  0.74170526],\n",
    "  [ 0.80849474,  0.87528421,  0.94207368,  1.00886316,  1.07565263],\n",
    "  [ 1.14244211,  1.20923158,  1.27602105,  1.34281053,  1.4096    ]])\n",
    "expected_velocity = np.asarray([\n",
    "  [ 0.5406,      0.55475789,  0.56891579, 0.58307368,  0.59723158],\n",
    "  [ 0.61138947,  0.62554737,  0.63970526,  0.65386316,  0.66802105],\n",
    "  [ 0.68217895,  0.69633684,  0.71049474,  0.72465263,  0.73881053],\n",
    "  [ 0.75296842,  0.76712632,  0.78128421,  0.79544211,  0.8096    ]])\n",
    "\n",
    "print 'updated_w error: ', rel_error(updated_w, expected_updated_w)\n",
    "print 'velocity error: ', rel_error(expected_velocity, velocity)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the following code block to train a multi-layer fully connected network with both SGD and SGD plus Momentum. The network trained with SGDM optimizer should converge faster."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Training with SGD plus Momentum...\n",
      "(Iteration 1 / 24500) loss: 2.76833163468\n",
      "(Iteration 101 / 24500) loss: 1.86569583056\n",
      "(Iteration 201 / 24500) loss: 1.81370618422\n",
      "(Iteration 301 / 24500) loss: 1.68167466995\n",
      "(Iteration 401 / 24500) loss: 1.55870864789\n",
      "(Epoch 1 / 50) Training Accuracy: 0.42187755102, Validation Accuracy: 0.417\n",
      "(Iteration 501 / 24500) loss: 1.61047051756\n",
      "(Iteration 601 / 24500) loss: 1.61551053884\n",
      "(Iteration 701 / 24500) loss: 1.57243605848\n",
      "(Iteration 801 / 24500) loss: 1.49060101957\n",
      "(Iteration 901 / 24500) loss: 1.57343542991\n",
      "(Epoch 2 / 50) Training Accuracy: 0.447816326531, Validation Accuracy: 0.46\n",
      "(Iteration 1001 / 24500) loss: 1.7657684366\n",
      "(Iteration 1101 / 24500) loss: 1.77470250709\n",
      "(Iteration 1201 / 24500) loss: 1.66451431381\n",
      "(Iteration 1301 / 24500) loss: 1.53509768946\n",
      "(Iteration 1401 / 24500) loss: 1.3895658229\n",
      "(Epoch 3 / 50) Training Accuracy: 0.489346938776, Validation Accuracy: 0.483\n",
      "(Iteration 1501 / 24500) loss: 1.47396923566\n",
      "(Iteration 1601 / 24500) loss: 1.536014475\n",
      "(Iteration 1701 / 24500) loss: 1.42679381697\n",
      "(Iteration 1801 / 24500) loss: 1.45645975901\n",
      "(Iteration 1901 / 24500) loss: 1.42434517489\n",
      "(Epoch 4 / 50) Training Accuracy: 0.484734693878, Validation Accuracy: 0.469\n",
      "(Iteration 2001 / 24500) loss: 1.1786810802\n",
      "(Iteration 2101 / 24500) loss: 1.42240805934\n",
      "(Iteration 2201 / 24500) loss: 1.43536834362\n",
      "(Iteration 2301 / 24500) loss: 1.56420692712\n",
      "(Iteration 2401 / 24500) loss: 1.60172860544\n",
      "(Epoch 5 / 50) Training Accuracy: 0.509306122449, Validation Accuracy: 0.48\n",
      "(Iteration 2501 / 24500) loss: 1.37767418259\n",
      "(Iteration 2601 / 24500) loss: 1.3192008925\n",
      "(Iteration 2701 / 24500) loss: 1.45752266112\n",
      "(Iteration 2801 / 24500) loss: 1.31125491612\n",
      "(Iteration 2901 / 24500) loss: 1.57176098125\n",
      "(Epoch 6 / 50) Training Accuracy: 0.522673469388, Validation Accuracy: 0.495\n",
      "(Iteration 3001 / 24500) loss: 1.18908680002\n",
      "(Iteration 3101 / 24500) loss: 1.55784776663\n",
      "(Iteration 3201 / 24500) loss: 1.37675533534\n",
      "(Iteration 3301 / 24500) loss: 1.34858917555\n",
      "(Iteration 3401 / 24500) loss: 1.30558993624\n",
      "(Epoch 7 / 50) Training Accuracy: 0.526244897959, Validation Accuracy: 0.472\n",
      "(Iteration 3501 / 24500) loss: 1.32586151553\n",
      "(Iteration 3601 / 24500) loss: 1.21103746883\n",
      "(Iteration 3701 / 24500) loss: 1.36293759899\n",
      "(Iteration 3801 / 24500) loss: 1.37601510356\n",
      "(Iteration 3901 / 24500) loss: 1.43973735423\n",
      "(Epoch 8 / 50) Training Accuracy: 0.535836734694, Validation Accuracy: 0.495\n",
      "(Iteration 4001 / 24500) loss: 1.527282063\n",
      "(Iteration 4101 / 24500) loss: 1.56735090331\n",
      "(Iteration 4201 / 24500) loss: 1.21823119805\n",
      "(Iteration 4301 / 24500) loss: 1.37201005134\n",
      "(Iteration 4401 / 24500) loss: 1.35877202535\n",
      "(Epoch 9 / 50) Training Accuracy: 0.532571428571, Validation Accuracy: 0.486\n",
      "(Iteration 4501 / 24500) loss: 1.13628508482\n",
      "(Iteration 4601 / 24500) loss: 1.25767341803\n",
      "(Iteration 4701 / 24500) loss: 1.35963788859\n",
      "(Iteration 4801 / 24500) loss: 1.15606377944\n",
      "(Epoch 10 / 50) Training Accuracy: 0.560959183673, Validation Accuracy: 0.493\n",
      "(Iteration 4901 / 24500) loss: 1.16280101955\n",
      "(Iteration 5001 / 24500) loss: 1.25217528459\n",
      "(Iteration 5101 / 24500) loss: 1.38987428902\n",
      "(Iteration 5201 / 24500) loss: 1.27202278734\n",
      "(Iteration 5301 / 24500) loss: 1.46039233499\n",
      "(Epoch 11 / 50) Training Accuracy: 0.551653061224, Validation Accuracy: 0.496\n",
      "(Iteration 5401 / 24500) loss: 1.13208291646\n",
      "(Iteration 5501 / 24500) loss: 1.22872717346\n",
      "(Iteration 5601 / 24500) loss: 1.26750967161\n",
      "(Iteration 5701 / 24500) loss: 1.32373687976\n",
      "(Iteration 5801 / 24500) loss: 1.26136480479\n",
      "(Epoch 12 / 50) Training Accuracy: 0.55893877551, Validation Accuracy: 0.488\n",
      "(Iteration 5901 / 24500) loss: 1.24780263744\n",
      "(Iteration 6001 / 24500) loss: 1.09120140772\n",
      "(Iteration 6101 / 24500) loss: 1.15940276032\n",
      "(Iteration 6201 / 24500) loss: 1.50037560685\n",
      "(Iteration 6301 / 24500) loss: 1.31602837372\n",
      "(Epoch 13 / 50) Training Accuracy: 0.574469387755, Validation Accuracy: 0.492\n",
      "(Iteration 6401 / 24500) loss: 1.42476043409\n",
      "(Iteration 6501 / 24500) loss: 1.44350934386\n",
      "(Iteration 6601 / 24500) loss: 1.35966388784\n",
      "(Iteration 6701 / 24500) loss: 1.33044689294\n",
      "(Iteration 6801 / 24500) loss: 1.3928240103\n",
      "(Epoch 14 / 50) Training Accuracy: 0.576510204082, Validation Accuracy: 0.484\n",
      "(Iteration 6901 / 24500) loss: 1.33301474926\n",
      "(Iteration 7001 / 24500) loss: 1.09690926063\n",
      "(Iteration 7101 / 24500) loss: 1.6772713924\n",
      "(Iteration 7201 / 24500) loss: 1.01680553816\n",
      "(Iteration 7301 / 24500) loss: 1.19280188174\n",
      "(Epoch 15 / 50) Training Accuracy: 0.575632653061, Validation Accuracy: 0.493\n",
      "(Iteration 7401 / 24500) loss: 1.04614609486\n",
      "(Iteration 7501 / 24500) loss: 1.25645343036\n",
      "(Iteration 7601 / 24500) loss: 1.17100664353\n",
      "(Iteration 7701 / 24500) loss: 1.30465355378\n",
      "(Iteration 7801 / 24500) loss: 1.14011182124\n",
      "(Epoch 16 / 50) Training Accuracy: 0.582, Validation Accuracy: 0.499\n",
      "(Iteration 7901 / 24500) loss: 0.996805306901\n",
      "(Iteration 8001 / 24500) loss: 1.02625172011\n",
      "(Iteration 8101 / 24500) loss: 1.1236961189\n",
      "(Iteration 8201 / 24500) loss: 1.04028293918\n",
      "(Iteration 8301 / 24500) loss: 1.17449307705\n",
      "(Epoch 17 / 50) Training Accuracy: 0.582918367347, Validation Accuracy: 0.49\n",
      "(Iteration 8401 / 24500) loss: 1.24159657004\n",
      "(Iteration 8501 / 24500) loss: 1.34123018778\n",
      "(Iteration 8601 / 24500) loss: 1.19093235499\n",
      "(Iteration 8701 / 24500) loss: 1.15398213202\n",
      "(Iteration 8801 / 24500) loss: 1.1223475676\n",
      "(Epoch 18 / 50) Training Accuracy: 0.582959183673, Validation Accuracy: 0.473\n",
      "(Iteration 8901 / 24500) loss: 1.05667945797\n",
      "(Iteration 9001 / 24500) loss: 1.14507441919\n",
      "(Iteration 9101 / 24500) loss: 1.16175720326\n",
      "(Iteration 9201 / 24500) loss: 1.14326960503\n",
      "(Iteration 9301 / 24500) loss: 1.15038752194\n",
      "(Epoch 19 / 50) Training Accuracy: 0.588734693878, Validation Accuracy: 0.514\n",
      "(Iteration 9401 / 24500) loss: 1.33586187474\n",
      "(Iteration 9501 / 24500) loss: 1.49576942679\n",
      "(Iteration 9601 / 24500) loss: 1.11855733277\n",
      "(Iteration 9701 / 24500) loss: 1.26993843293\n",
      "(Epoch 20 / 50) Training Accuracy: 0.603489795918, Validation Accuracy: 0.511\n",
      "(Iteration 9801 / 24500) loss: 0.94570393261\n",
      "(Iteration 9901 / 24500) loss: 1.16848562324\n",
      "(Iteration 10001 / 24500) loss: 1.2815272442\n",
      "(Iteration 10101 / 24500) loss: 1.17350989218\n",
      "(Iteration 10201 / 24500) loss: 1.12963180165\n",
      "(Epoch 21 / 50) Training Accuracy: 0.611020408163, Validation Accuracy: 0.478\n",
      "(Iteration 10301 / 24500) loss: 1.13924561827\n",
      "(Iteration 10401 / 24500) loss: 1.19445185343\n",
      "(Iteration 10501 / 24500) loss: 1.10180379347\n",
      "(Iteration 10601 / 24500) loss: 1.25447471749\n",
      "(Iteration 10701 / 24500) loss: 1.22860689993\n",
      "(Epoch 22 / 50) Training Accuracy: 0.606204081633, Validation Accuracy: 0.486\n",
      "(Iteration 10801 / 24500) loss: 1.01486400586\n",
      "(Iteration 10901 / 24500) loss: 1.2134351646\n",
      "(Iteration 11001 / 24500) loss: 1.23883092911\n",
      "(Iteration 11101 / 24500) loss: 1.19012427687\n",
      "(Iteration 11201 / 24500) loss: 1.10412524929\n",
      "(Epoch 23 / 50) Training Accuracy: 0.616367346939, Validation Accuracy: 0.487\n",
      "(Iteration 11301 / 24500) loss: 1.15162700747\n",
      "(Iteration 11401 / 24500) loss: 1.094045053\n",
      "(Iteration 11501 / 24500) loss: 1.26188441844\n",
      "(Iteration 11601 / 24500) loss: 1.07324866444\n",
      "(Iteration 11701 / 24500) loss: 1.23619695458\n",
      "(Epoch 24 / 50) Training Accuracy: 0.605285714286, Validation Accuracy: 0.493\n",
      "(Iteration 11801 / 24500) loss: 0.990185600483\n",
      "(Iteration 11901 / 24500) loss: 1.2599708549\n",
      "(Iteration 12001 / 24500) loss: 1.12847928283\n",
      "(Iteration 12101 / 24500) loss: 1.10431334981\n",
      "(Iteration 12201 / 24500) loss: 1.23936940645\n",
      "(Epoch 25 / 50) Training Accuracy: 0.621918367347, Validation Accuracy: 0.515\n",
      "(Iteration 12301 / 24500) loss: 1.14885615047\n",
      "(Iteration 12401 / 24500) loss: 1.16475720297\n",
      "(Iteration 12501 / 24500) loss: 1.18155843419\n",
      "(Iteration 12601 / 24500) loss: 1.25739368879\n",
      "(Iteration 12701 / 24500) loss: 1.21758024624\n",
      "(Epoch 26 / 50) Training Accuracy: 0.602755102041, Validation Accuracy: 0.467\n",
      "(Iteration 12801 / 24500) loss: 1.1368341629\n",
      "(Iteration 12901 / 24500) loss: 1.12781814839\n",
      "(Iteration 13001 / 24500) loss: 1.01989338358\n",
      "(Iteration 13101 / 24500) loss: 1.11230319134\n",
      "(Iteration 13201 / 24500) loss: 1.12397426802\n",
      "(Epoch 27 / 50) Training Accuracy: 0.624346938776, Validation Accuracy: 0.499\n",
      "(Iteration 13301 / 24500) loss: 0.893938591412\n",
      "(Iteration 13401 / 24500) loss: 1.2305468637\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(Iteration 13501 / 24500) loss: 1.13719347771\n",
      "(Iteration 13601 / 24500) loss: 1.25787659322\n",
      "(Iteration 13701 / 24500) loss: 1.10498470235\n",
      "(Epoch 28 / 50) Training Accuracy: 0.618469387755, Validation Accuracy: 0.487\n",
      "(Iteration 13801 / 24500) loss: 1.01606330312\n",
      "(Iteration 13901 / 24500) loss: 1.235859264\n",
      "(Iteration 14001 / 24500) loss: 1.10706009022\n",
      "(Iteration 14101 / 24500) loss: 1.0813459386\n",
      "(Iteration 14201 / 24500) loss: 1.01269891454\n",
      "(Epoch 29 / 50) Training Accuracy: 0.621591836735, Validation Accuracy: 0.505\n",
      "(Iteration 14301 / 24500) loss: 1.15454497868\n",
      "(Iteration 14401 / 24500) loss: 1.02694388454\n",
      "(Iteration 14501 / 24500) loss: 0.961240302087\n",
      "(Iteration 14601 / 24500) loss: 1.16641373701\n",
      "(Epoch 30 / 50) Training Accuracy: 0.627530612245, Validation Accuracy: 0.485\n",
      "(Iteration 14701 / 24500) loss: 1.35750043451\n",
      "(Iteration 14801 / 24500) loss: 1.04378477232\n",
      "(Iteration 14901 / 24500) loss: 1.1160837684\n",
      "(Iteration 15001 / 24500) loss: 1.28399232569\n",
      "(Iteration 15101 / 24500) loss: 0.934231499995\n",
      "(Epoch 31 / 50) Training Accuracy: 0.624673469388, Validation Accuracy: 0.467\n",
      "(Iteration 15201 / 24500) loss: 0.988537513118\n",
      "(Iteration 15301 / 24500) loss: 1.02332724345\n",
      "(Iteration 15401 / 24500) loss: 1.06177961277\n",
      "(Iteration 15501 / 24500) loss: 1.27095057619\n",
      "(Iteration 15601 / 24500) loss: 1.36639845008\n",
      "(Epoch 32 / 50) Training Accuracy: 0.631142857143, Validation Accuracy: 0.498\n",
      "(Iteration 15701 / 24500) loss: 1.13396394254\n",
      "(Iteration 15801 / 24500) loss: 1.2271366609\n",
      "(Iteration 15901 / 24500) loss: 1.05899499511\n",
      "(Iteration 16001 / 24500) loss: 1.24736637869\n",
      "(Iteration 16101 / 24500) loss: 1.32760535898\n",
      "(Epoch 33 / 50) Training Accuracy: 0.604448979592, Validation Accuracy: 0.476\n",
      "(Iteration 16201 / 24500) loss: 1.12642015912\n",
      "(Iteration 16301 / 24500) loss: 0.9577492074\n",
      "(Iteration 16401 / 24500) loss: 1.03094057961\n",
      "(Iteration 16501 / 24500) loss: 1.05459126252\n",
      "(Iteration 16601 / 24500) loss: 1.13113559414\n",
      "(Epoch 34 / 50) Training Accuracy: 0.630265306122, Validation Accuracy: 0.502\n",
      "(Iteration 16701 / 24500) loss: 1.0372534307\n",
      "(Iteration 16801 / 24500) loss: 0.842593428402\n",
      "(Iteration 16901 / 24500) loss: 1.0097111735\n",
      "(Iteration 17001 / 24500) loss: 1.18928645165\n",
      "(Iteration 17101 / 24500) loss: 0.972246502073\n",
      "(Epoch 35 / 50) Training Accuracy: 0.640795918367, Validation Accuracy: 0.509\n",
      "(Iteration 17201 / 24500) loss: 0.814611469352\n",
      "(Iteration 17301 / 24500) loss: 0.975344602399\n",
      "(Iteration 17401 / 24500) loss: 1.14618911509\n",
      "(Iteration 17501 / 24500) loss: 0.980320270325\n",
      "(Iteration 17601 / 24500) loss: 1.2303833297\n",
      "(Epoch 36 / 50) Training Accuracy: 0.639428571429, Validation Accuracy: 0.496\n",
      "(Iteration 17701 / 24500) loss: 1.05708194469\n",
      "(Iteration 17801 / 24500) loss: 1.03882674062\n",
      "(Iteration 17901 / 24500) loss: 1.12356072839\n",
      "(Iteration 18001 / 24500) loss: 1.16033333486\n",
      "(Iteration 18101 / 24500) loss: 1.10108298524\n",
      "(Epoch 37 / 50) Training Accuracy: 0.640734693878, Validation Accuracy: 0.48\n",
      "(Iteration 18201 / 24500) loss: 0.901866521593\n",
      "(Iteration 18301 / 24500) loss: 1.12030724621\n",
      "(Iteration 18401 / 24500) loss: 1.23578570955\n",
      "(Iteration 18501 / 24500) loss: 0.908132678857\n",
      "(Iteration 18601 / 24500) loss: 1.0909828839\n",
      "(Epoch 38 / 50) Training Accuracy: 0.636265306122, Validation Accuracy: 0.48\n",
      "(Iteration 18701 / 24500) loss: 1.01482725278\n",
      "(Iteration 18801 / 24500) loss: 1.05689679401\n",
      "(Iteration 18901 / 24500) loss: 0.911691234566\n",
      "(Iteration 19001 / 24500) loss: 1.09567623551\n",
      "(Iteration 19101 / 24500) loss: 1.29030979214\n",
      "(Epoch 39 / 50) Training Accuracy: 0.641428571429, Validation Accuracy: 0.494\n",
      "(Iteration 19201 / 24500) loss: 1.00468647471\n",
      "(Iteration 19301 / 24500) loss: 1.03111964454\n",
      "(Iteration 19401 / 24500) loss: 1.21908070339\n",
      "(Iteration 19501 / 24500) loss: 0.937557929497\n",
      "(Epoch 40 / 50) Training Accuracy: 0.646755102041, Validation Accuracy: 0.487\n",
      "(Iteration 19601 / 24500) loss: 1.0220214729\n",
      "(Iteration 19701 / 24500) loss: 1.21471040018\n",
      "(Iteration 19801 / 24500) loss: 1.09413888403\n",
      "(Iteration 19901 / 24500) loss: 1.16002746568\n",
      "(Iteration 20001 / 24500) loss: 0.85459823585\n",
      "(Epoch 41 / 50) Training Accuracy: 0.641510204082, Validation Accuracy: 0.491\n",
      "(Iteration 20101 / 24500) loss: 0.829658532201\n",
      "(Iteration 20201 / 24500) loss: 0.997894655604\n",
      "(Iteration 20301 / 24500) loss: 0.89858601003\n",
      "(Iteration 20401 / 24500) loss: 1.12615634687\n",
      "(Iteration 20501 / 24500) loss: 1.28368155579\n",
      "(Epoch 42 / 50) Training Accuracy: 0.634428571429, Validation Accuracy: 0.456\n",
      "(Iteration 20601 / 24500) loss: 1.36887689616\n",
      "(Iteration 20701 / 24500) loss: 1.21703013747\n",
      "(Iteration 20801 / 24500) loss: 0.972481441648\n",
      "(Iteration 20901 / 24500) loss: 1.02981395119\n",
      "(Iteration 21001 / 24500) loss: 1.21560067364\n",
      "(Epoch 43 / 50) Training Accuracy: 0.640897959184, Validation Accuracy: 0.478\n",
      "(Iteration 21101 / 24500) loss: 1.08652125465\n",
      "(Iteration 21201 / 24500) loss: 1.12828240436\n",
      "(Iteration 21301 / 24500) loss: 1.14858222578\n",
      "(Iteration 21401 / 24500) loss: 1.12854688844\n",
      "(Iteration 21501 / 24500) loss: 1.10379720195\n",
      "(Epoch 44 / 50) Training Accuracy: 0.59793877551, Validation Accuracy: 0.467\n",
      "(Iteration 21601 / 24500) loss: 1.09990137951\n",
      "(Iteration 21701 / 24500) loss: 0.830250810655\n",
      "(Iteration 21801 / 24500) loss: 0.961240348934\n",
      "(Iteration 21901 / 24500) loss: 1.08809638194\n",
      "(Iteration 22001 / 24500) loss: 1.0439086187\n",
      "(Epoch 45 / 50) Training Accuracy: 0.645673469388, Validation Accuracy: 0.476\n",
      "(Iteration 22101 / 24500) loss: 1.05096634885\n",
      "(Iteration 22201 / 24500) loss: 1.41205469496\n",
      "(Iteration 22301 / 24500) loss: 1.21355250166\n",
      "(Iteration 22401 / 24500) loss: 0.881646134818\n",
      "(Iteration 22501 / 24500) loss: 0.83467889823\n",
      "(Epoch 46 / 50) Training Accuracy: 0.656918367347, Validation Accuracy: 0.495\n",
      "(Iteration 22601 / 24500) loss: 1.16814695194\n",
      "(Iteration 22701 / 24500) loss: 1.02873088164\n",
      "(Iteration 22801 / 24500) loss: 0.830183557382\n",
      "(Iteration 22901 / 24500) loss: 0.903911658446\n",
      "(Iteration 23001 / 24500) loss: 1.14461570666\n",
      "(Epoch 47 / 50) Training Accuracy: 0.658897959184, Validation Accuracy: 0.478\n",
      "(Iteration 23101 / 24500) loss: 0.980798182573\n",
      "(Iteration 23201 / 24500) loss: 0.871190854082\n",
      "(Iteration 23301 / 24500) loss: 0.915289677756\n",
      "(Iteration 23401 / 24500) loss: 1.21573563852\n",
      "(Iteration 23501 / 24500) loss: 1.22685953188\n",
      "(Epoch 48 / 50) Training Accuracy: 0.617469387755, Validation Accuracy: 0.478\n",
      "(Iteration 23601 / 24500) loss: 1.03712692715\n",
      "(Iteration 23701 / 24500) loss: 1.24432003021\n",
      "(Iteration 23801 / 24500) loss: 1.02567565438\n",
      "(Iteration 23901 / 24500) loss: 0.874042114316\n",
      "(Iteration 24001 / 24500) loss: 1.00782072025\n",
      "(Epoch 49 / 50) Training Accuracy: 0.655020408163, Validation Accuracy: 0.483\n",
      "(Iteration 24101 / 24500) loss: 0.859969915146\n",
      "(Iteration 24201 / 24500) loss: 1.01072645769\n",
      "(Iteration 24301 / 24500) loss: 1.30410711092\n",
      "(Iteration 24401 / 24500) loss: 0.892009386182\n",
      "(Epoch 50 / 50) Training Accuracy: 0.652183673469, Validation Accuracy: 0.49\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/taotaochen/anaconda2/lib/python2.7/site-packages/matplotlib/cbook/deprecation.py:106: MatplotlibDeprecationWarning: Adding an axes using the same arguments as a previous axes currently reuses the earlier instance.  In a future version, a new instance will always be created and returned.  Meanwhile, this warning can be suppressed, and the future behavior ensured, by passing a unique label to each axes instance.\n",
      "  warnings.warn(message, mplDeprecation, stacklevel=1)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3QAAANsCAYAAAATFepNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3XuUHNV9L/rvr3tKUg8oagmEjRq9wFgyYtCMGYOwAkZ4XWQsQwbhIDjg40d8fBybHCOTyRpxFSM5+GpuJjbE5iYOXknsBAUPIJ05wnIiTCTHmCDhETNiIpBsHnq1iFGQmoemQT09+/7RXa3q6np2VT/n+1mLhaanump3dU33/tXe+/cTpRSIiIiIiIio8URq3QAiIiIiIiIqDwM6IiIiIiKiBsWAjoiIiIiIqEExoCMiIiIiImpQDOiIiIiIiIgaFAM6IiIiIiKiBsWAjoiImoaIREXkHRGZE+a2ZbTjXhH5Ydj7JSIiMmupdQOIiGjiEpF3DD+2AngPQDb/8/9USm30sz+lVBbAmWFvS0REVK8Y0BERUc0opQoBlYgcAPBFpdSTdtuLSItSaqwabSMiImoEnHJJRER1Kz91sV9EHhaRtwHcLiJXiMhOEUmJyGsi8l0R0fLbt4iIEpF5+Z8fyv/+n0XkbRF5RkTm+902//vrROTXIvKmiHxPRJ4Wkc95fB1dIrI33+btIrLA8Lu7ReSoiLwlIvtE5Or840tE5Ln8478Vkb4QTikRETUZBnRERFTvbgTwTwCmAegHMAbgawDOBrAUwCcA/E+H5/83AH8KYAaAQwD+zO+2InIOgEcAdOeP+yqAy7w0XkQ+BOAhAH8EYCaAJwE8LiKaiCzKt/3DSqnfAXBd/rgA8D0AffnHPwDgMS/HIyKiiYUBHRER1btfKqUeV0qNK6XSSqlfKaV2KaXGlFKvAHgQwMccnv+YUmpQKZUBsBFAexnbfgrAsFLq/+R/dx+A//LY/lsAbFFKbc8/txfA7wC4HLngdAqARfnppK/mXxMAZABcKCJnKaXeVkrt8ng8IiKaQBjQERFRvTts/EFEForIVhH5TxF5C8A3kRs1s/Ofhn+PwjkRit22s4ztUEopAEc8tF1/7kHDc8fzz00opfYDuAu51/B6fmrp+/Obfh7ARQD2i8izIvJJj8cjIqIJhAEdERHVO2X6+W8A/AeAD+SnI34DgFS4Da8BOE//QUQEQMLjc48CmGt4biS/ryQAKKUeUkotBTAfQBTAhvzj+5VStwA4B8C3AWwSkSnBXwoRETUTBnRERNRopgJ4E8DJ/Po0p/VzYfkJgA+LyPUi0oLcGr6ZHp/7CIAbROTqfPKWbgBvA9glIh8SkWUiMhlAOv9fFgBE5DMicnZ+RO9N5ALb8XBfFhERNToGdERE1GjuAvBZ5IKiv0EuUUpFKaV+C2AVgO8AeAPABQCGkKub5/bcvci1968BHEMuicsN+fV0kwH8OXLr8f4TwHQAa/NP/SSAF/PZPf8CwCql1KkQXxYRETUByS0DICIiIq9EJIrcVMpPK6WeqnV7iIho4uIIHRERkQci8gkRmZafHvmnyGWofLbGzSIiogmOAR0REZE3vwvgFeSmR34CQJdSynXKJRERUSVxyiUREREREVGD4ggdERERERFRg2qpdQPMzj77bDVv3rxaN4OIiIiIiKgmdu/e/V9KKU/lceouoJs3bx4GBwdr3QwiIiIiIqKaEJGDXrfllEsiIiIiIqIGxYCOiIiIiIioQTGgIyIiIiIialB1t4aOiIiqI5PJ4MiRI3j33Xdr3RRqEFOmTMF5550HTdNq3RQiIspjQEdENEEdOXIEU6dOxbx58yAitW4O1TmlFN544w0cOXIE8+fPr3VziIgoj1MuiYgmqHfffRdnnXUWgznyRERw1llncUSXiKjOcITOxcBQEn3b9uNoKo1Z8Ri6ly9AV0ei1s0iIgoFgznyg9cLEVH9YUDnYGAoiTWbR5DOZAEAyVQaazaPAACDOiIiIiIiqjlOuXTQt21/IZjTpTNZ9G3bX6MWERE1l29961tYtGgRLrnkErS3t2PXrl0AgLGxMdx999248MIL0d7ejvb2dnzrW98qPC8ajaK9vR2LFi3C4sWL8Z3vfAfj4+NlteGjH/0oAODAgQP4p3/6p8LjP/zhD3HHHXe4Pv/qq6/GnDlzoJQqPNbV1YUzzzyzrPYElUql8Fd/9Vc1OTYREVUfAzoHR1NpX48TETWzgaEklvZux/yerVjaux0DQ8lA+3vmmWfwk5/8BM899xyef/55PPnkk5g9ezYAYO3atTh69ChGRkYwPDyMp556CplMpvDcWCyG4eFh7N27Fz/72c/w05/+FOvXry+rHf/+7/8OoDSg8yMej+Ppp58GkAuoXnvttbL2EwYGdEREEwsDOgez4jFfjxMRNSt9CnoylYbC6SnoQYK61157DWeffTYmT54MADj77LMxa9YsjI6O4gc/+AG+973vYcqUKQCAqVOnYt26dZb7Oeecc/Dggw/igQceKBolA4CvfOUr2LJlCwDgxhtvxBe+8AUAwN/+7d9i7dq1AFAYSevp6cFTTz2F9vZ23HfffQCAo0eP4hOf+AQuvPBC/Mmf/Inta7nlllvw4x//GACwefNmrFy5svA7pRS6u7tx8cUXo62tDf39/QCAn//85/jYxz6Gm2++GR/84AfR09ODjRs34rLLLkNbWxtefvllAMCxY8dw00034SMf+Qg+8pGPFALHdevW4Qtf+AKuvvpqnH/++fjud79beB0vv/wy2tvb0d3djZ///Of41Kc+VWjPHXfcgR/+8IcAgHnz5uHuu+/GFVdcgc7OTjz33HNYvnw5LrjgAnz/+9+3fb1ERFQ/GNA56F6+ADEtWvRYTIuie/mCGrWIiKg2KjEF/dprr8Xhw4fxwQ9+EF/5ylfwb//2bwCAl156CXPmzMHUqVM97+v888/H+Pg4Xn/99aLHr7rqKjz11FMAgGQyiRdeeAEA8Mtf/hJXXnll0ba9vb248sorMTw8jNWrVwMAhoeH0d/fj5GREfT39+Pw4cOWx//4xz+OX/ziF8hms/jxj3+MVatWFX63efNmDA8PY8+ePXjyySfR3d1dGMHbs2cP/vIv/xIjIyP4x3/8R/z617/Gs88+iy9+8Yv43ve+BwD42te+htWrV+NXv/oVNm3ahC9+8YuFfe/btw/btm3Ds88+i/Xr1yOTyaC3txcXXHABhoeH0dfX53ruZs+ejWeeeQZXXnklPve5z+Gxxx7Dzp078Y1vfMP1uUREVHsM6Bx0dSSwYWUbEvEYBEAiHsOGlW1MiEJEE04lpqCfeeaZ2L17Nx588EHMnDkTq1atKowcGf393/892tvbMXv2bNuACkDJ6BwAXHnllXjqqafwwgsv4KKLLsL73vc+vPbaa3jmmWcKa+ecfPzjH8e0adMwZcoUXHTRRTh48KDldtFoFL/7u7+L/v5+pNNpzJs3r/C7X/7yl7j11lsRjUbxvve9Dx/72Mfwq1/9CgDwkY98BOeeey4mT56MCy64ANdeey0AoK2tDQcOHAAAPPnkk7jjjjvQ3t6OG264AW+99RbefvttAMCKFSswefJknH322TjnnHPw29/+1vU1md1www2FY15++eWYOnUqZs6ciSlTpiCVSvneHxERVRezXLro6kgwgCOiCW9WPIakRfAWdAp6NBrF1VdfjauvvhptbW340Y9+hJtvvhmHDh3C22+/jalTp+Lzn/88Pv/5z+Piiy9GNpu13M8rr7yCaDSKc845p+jxRCKBEydO4F/+5V9w1VVX4fjx43jkkUdw5plnehoB1KeD6m0dGxuz3faWW27BjTfeWDI11CrQtNp/JBIp/ByJRArHGh8fxzPPPINYrPRce2lfS0tLUcIYcx054zHN7XF6vUREVB84QkdERK4qMQV9//79+M1vflP4eXh4GHPnzkVrayv+4A/+AHfccUch+Mhmszh16pTlfo4dO4Yvf/nLuOOOOyzrpF1xxRW4//77cdVVV+HKK6/EX/zFX5RMtwRy6/T0ka9yXHnllVizZg1uvfXWosevuuoq9Pf3I5vN4tixY/jFL36Byy67zPN+r732WjzwwAOFn4eHhx23N7+OuXPn4oUXXsB7772HN998E//6r//q+dhERFT/OEJHRESu9JkKfdv242gqjVnxGLqXLwg0g+Gdd97BH/3RHyGVSqGlpQUf+MAH8OCDDwLIlTP40z/9U1x88cWYOnUqYrEYPvvZz2LWrFkAgHQ6jfb2dmQyGbS0tOAzn/kMvv71r1se58orr8QTTzyBD3zgA5g7dy6OHz9uGdBdcsklaGlpweLFi/G5z30O06dP9/V6RAR//Md/XPL4jTfeiGeeeQaLFy+GiODP//zP8f73vx/79u3ztN/vfve7+OpXv4pLLrkEY2NjuOqqqxwTlpx11llYunQpLr74Ylx33XXo6+vDzTffjEsuuQQXXnghOjo6fL0uIiKqb+I0FaQWOjs71eDgYK2bQUTU9F588UV86EMfqnUzqMHwuiEiqjwR2a2U6vSyLadcEhERERERNSgGdERERERERA2KAR0R0QRWb9Puqb7xeiEiqj8M6IiIJqgpU6bgjTfeYCedPFFK4Y033sCUKVNq3RQiIjJglksiognqvPPOw5EjR3Ds2LFaN4UaxJQpU3DeeefVuhlERGTAgI6IaILSNA3z58+vdTOIiIgoAE65JCIiIiIialAM6IiIiIiIiBoUAzoiIiIiIqIGxYCOiIiIiIioQTGgIyIiIiIialAM6IiIiIiIiBoUAzoiIiIiIqIGxYCOiIiIiIioQTGgIyIiIiIialAM6IiIiIiIiBpU2QGdiMwWkR0i8qKI7BWRr1lsc7WIvCkiw/n/vhGsuURERERERKRrCfDcMQB3KaWeE5GpAHaLyM+UUi+YtntKKfWpAMchIiIiIiIiC2WP0CmlXlNKPZf/99sAXgSQCKthRERERERE5CyUNXQiMg9AB4BdFr++QkT2iMg/i8gim+d/SUQGRWTw2LFjYTSJiIiIiIio6QUO6ETkTACbANyplHrL9OvnAMxVSi0G8D0AA1b7UEo9qJTqVEp1zpw5M2iTiIiIiIiIJoRAAZ2IaMgFcxuVUpvNv1dKvaWUeif/758C0ETk7CDHJCIiIiIiopwgWS4FwN8CeFEp9R2bbd6f3w4icln+eG+Ue0wiIiIiIiI6LUiWy6UAPgNgRESG84/dDWAOACilvg/g0wD+UETGAKQB3KKUUgGOSURERERERHllB3RKqV8CEJdtHgDwQLnHICIiIiIiInuhZLkkIiIiIiKi6mNAR0RERERE1KCCrKGbMAaGkujbth9HU2nMisfQvXwBujpYQ52IiIiIiGqLAZ2LgaEk1mweQTqTBQAkU2ms2TwCAAzqiIiIiIiopjjl0kXftv2FYE6XzmTRt21/jVpERERERESUw4DOxdFU2tfjRERERERE1cKAzsW0mObrcSIiIiIiomphQOdCbCrt2T1ORERERERULQzoXKRGM74eJyIiIiIiqhYGdC445ZKIiIiIiOoVAzoXnHJJRERERET1igGdixM2UyvtHiciIiIiIqoWBnQuIjYjcXaPExERERERVQsDOhfjyt/jRERERERE1cKAjoiIiIiIqEExoCMiIiIiImpQDOiIiIiIiIgaFAM6IiIiIiKiBsWAjoiIiIiIqEExoHNx+5I5lo8vvWBGlVtCRERERERUjAFdmXa+egIDQ8laN4OIiIiIiCYwBnQuHt512PLx7LhC37b9VW4NERERERHRaQzoXGSVfQXxZCpdxZYQEREREREVY0AXgACcdklERERERDXDgC4ABXDaJRERERER1QwDuoCOctolERERERHVCAM6F62a8ymaFY9VqSVERERERETFGNC5mNQStf1dTIuie/mCKraGiIiIiIjoNAZ0Lt5MZ2x/t2FlG7o6ElVsDRERERER0WkM6Fy02JwhLQIGc0REREREVFMM6Fxkxv09TkREREREVC0M6IiIiIiIiBoUA7oAWFSciIiIiIhqiQFdAGs2P1/rJhARERER0QTGgC6AdGaco3RERERERFQzDOgC6tu2v9ZNICIiIiKiCYoBXUDJVLrWTSAiIiIiogmKAV0IOO2SiIiIiIhqgQGdi6UXzHDdhtMuiYiIiIioFlpq3YB6d+AN9ymVRwNOuxwYSqJv234cTaUxKx5D9/IF6OpIBNonERERERE1PwZ0LryskZsW0wCUF5gNDCWxZvMI0pls4XhrNo8AAIM6IiIiIiJyVPaUSxGZLSI7RORFEdkrIl+z2EZE5Lsi8pKIPC8iHw7W3OqLirhuk0pnsHZgBGs2jyCZSkPhdGDmtr6ub9v+QjCnS2eynMZJRERERESugqyhGwNwl1LqQwCWAPiqiFxk2uY6ABfm//sSgL8OcLyayCrlabuNOw+VFZjZTdcMOo2TiIiIiIiaX9kBnVLqNaXUc/l/vw3gRQDmOYK/B+AfVM5OAHERObfs1taA+/hcjl3Y5xaYzYrHfD1ORERERESkCyXLpYjMA9ABYJfpVwkAhw0/H0Fp0FfXvI3P2XMLzLqXL0BMixY9FtOi6F6+IOCRiYiIiIio2QVOiiIiZwLYBOBOpdRb5l9bPKUkRhKRLyE3JRNz5swJ2qSaiWnRommXEQCvvZnGvJ6tiIrg1stn496utqLn6IlPmOWSiIiIiIj8ChTQiYiGXDC3USm12WKTIwBmG34+D8BR80ZKqQcBPAgAnZ2dQQfFakIAbFjZVgjMpmgRpDPjhfA1qxQe2nkIACyDOgZwRERERETkV5AslwLgbwG8qJT6js1mWwD893y2yyUA3lRKvVbuMeuZAnBn/zBGT43hvlXtODVmHZc+vOuw5eNERERERER+BRmhWwrgMwBGRGQ4/9jdAOYAgFLq+wB+CuCTAF4CMArg8wGOVxMRAcZ9jBmeGM3gzv5h2997zZpJRERERETkpuyATin1S7gkgVRKKQBfLfcY9WByS37qZEjc6tqVU5yciIiIiIgmpsBJUZrduyEGcwBw6+WzbX83MJTEms0jhcQqenFyAAzqiIiIiIioBAM6F7PiMSRDKvJ94TlnYMe+Y5jfsxXTYhpEgNRopvDvE6OZkufoxckZ0BERERERkVkodeiaWZj14H7z+kkkU2koAKl0BidGM0X/tuNWnJyIiIiIiCYmBnQu6mFkzK04uVcDQ0ks7d2O+T1bsbR3OwaGkqHsl4iIiIiIaoNTLhvAsoUzAz1/YCiJdVv2IpU+PQrI9XlERERERI2PI3QNYNPuZNmjaXqiFWMwp9PX5xERERERUWNiQNcA0pks1m3ZW9Zz+7btL2TNtML1eUREREREjYsBXYNIpTNljdK5BWxhrc8jIiIiIqLqY0DngXMp8HDcvmQOYlrUcZtypkc6BWwxLRpqFs+gmLSFiIiIiMgfBnQe3LeqvSrH2bCyDTHN/i0xj7Z5CYC6ly+wDBSnt2rYsLKtbhKi6Gv99LIOetIWBnVEREREzYc38sPDLJcedHUkcGf/cEWP8dDOQwCAGWdMti1kbhxt0wMgfX2cXdZK/d992/bjaCqNWfEYupcvqJtATme11o9F1YmIiIiaj9d+LHnDgK6O6EGdndToKQwMJdHVkXAMgIDSAO7pnmsq1u4w2K31Y9IWIiIioubCG/nhYkDXQE6eyuLO/mH83/97BCdPWWeu1O9wlHvHw1yzbnqrhnuuX1TxP65Z8ZjlyCSTthARERE1F97IDxfX0DUgu2BO5zRy52RgKInuR/cU1aw7MZrBnf3DWDsw4vrcIPOgrdb61VvSFiIiIiIKzu6GPW/kl4cjdBOE8Y7HwFCyMCVzWkyDCJAazSAigqxSls/fuPMQOufOsBypC2MedK3W+hnPRb2uLyQiIiJqJt3LFxT1HQHeyA9ClE0HvlY6OzvV4OBgrZtRZGAoWfGkKNWQiMewbOFMbNqddCw27vR8q7V4S3u3W06XtNu+XpgDUSD3YVJP2T+JiIiImhFvqjsTkd1KqU4v23KEzoNy6r/Vo2QqjY07D6HcEN7vfOd6nwfNBblEREREtdHVkWB/KyQM6Dyo98DEjyDjsfq8ZvMdlXirhhOjGdvt61WjBqJERERERDoGdB7YZWCcaEZPjWFez1YITgeGyVQaWkSgRQWZ7OlwsRHmQTOzJhFVG6cYERFR2Jjl0oN6D0yqRR+FM4/yZcYVxrIK01s1CHJr5/yuQwuaJbMcVpk1BcCyhTMrfmwimnj0dbvJVBoKpxNIVePzjoiImhcDOg+6OhI4Y1LUfcMJTAF4NzOO+1a14+mea3wHc7Xo5HR1JHDTpQmI4TEFYNPuJDtYTaYWNwyIzJzW7RIREZWLUy49+taNbU2R6bKS0pks1m3Z63s60frH9wZKThJkCtOOfcdKRhyNx67E9ChOuaquMMpqEIWB63aJiKgSOELnETt+3qTSGc8jbQNDSbSvf8IyoQrgrZMTdHTPqYNViZFDTrmqPo6KUL0Is5AuR52JiEjHETqqqHQmi9WPnB7ZXLdlL1Jp6wDOzEsnx66zvm7LXk9BuFNilEqUNWCphOowjoLaZXblqAhVW1iFdDnq7B9nRhBRM2NA54MWATLjtW5F7RizW/qhFPD1fFA37mMHXpKT2HXKU+lM4Y6105e4Uwdrtc0U2yCBgNuUK3Y6gp8Dq4LxVpjNlKpNv46D/o3zxpA/DICJqNkxoPOh7/fbJ/Q6uiA17PwEcrpNu5PonDvDcS2bU0mJdVv24r2xcccvcXMHa1pMgwiwun8YERFkVWnDIyKY37O1rM6YXXunxTR2OhBOx8uqs2vmd1SEgTaFJYxCulyL5w8DYCJqdgzoqG6lM1nc9cge3Nk/XFL7Tu/kzzvLPqCzmtqp73N1/3BRx1wPGo3BhFUwZ3y8nGCje/kCdD+6BxlThHvy1Fjg5DC14CfQ8bJtGB0vp06tAL4DMgbaVG9YQ9MfBsBE1OyYFMWH9Y/vrXUTJhw9eLLKRHln/zCefvl4Wfu0SkhiFVABQFQEkv+/md/kGl0dCZw5pfQ+SiarAiWHqQU/CV68bhtGx8uuU5uIx/Bq7wrfZTWYVIXqjVUNzXLW4k0UYSajISKqRwzofLDrcFP9KQ29rOkd84GhpO37O64UXu1dgXGbETu/AVfK53UURqfDKiNekCx5A0NJ3PXIHs+BjtegKIyOV9idXd7dp3rT1ZHAhpVtSMRjEORuVmxY2cYRYxsMgImo2XHKJTUlP0v2jqbSjqOvejBhN80pIoKBoaTnzpTdfuIxrWjNHxBOp8NqymD3o3sAyY0M6o95nUao789uSqpVoOM1KAojC2BYiSd0nN5G1eZlenIYa/EmAv1cpjNZRPProhNcB9sUuLaZ6DSO0PkQj2m1bgJVwKx4zHH0tXv5AgwMJXHyvTHL32eVwur+YawdGPF0vGULZ1qOIIoAN12aCP2uu9XoWGZcFYI5nddphG5JR6wCHa8jb2GNPHR1JPB0zzVlTbE04919qibWqgyP8VwCuc9q/W+XHf/Gxr8TomIcofNh3Q2LLBNaUGMbPWUdqOnMSVmsKAAP7TwEALi3q812u4GhJDbtTlru68RoBpt2J0OfOuVnaqCXbZ22sQt0/Iy81dvIQ9gjftScwhotYEbG8PBcNq9mfm858kjlYEDng/4Htbp/OFAKf6ovXtZGen2/H9p5CA/tPFQypWftwAge3nXYdpqiTi+KHuaHuVNpB6tty91fVMQ2GG30oMgtyOQX8MQWNBOq8fqx+4Tgmk3/uP61eTXre1vPWZX5PVffGND51NWRsC04TaRLptK4s38Y6x/fi4vOneorG2cqnSmUXAjyYa5/+CZT6ZIRRi0iRWvoAO/TCO1G29xGFs1BnT69s9wvhHr5cqnnL2CzejlnYav16woyWmC+fuxwzaZ/XP/avJr1va3XkcdG+p6bqLiGrgyN/oFB1XNiNFNWaQUjvURDxzef8LQ+YGAoifb1T+DO/uHCF54xmItIbg3dGZNaML1VK1mr5iX75eSW0x8d01s1T9NEB4aS6H5sT9Gah+7H9hT27yfrZj2tn2iUsgb1dM7CVA+vK8hogduaVKD0ZkuQDLUTCde/Nq9mfW/rdeSxUb7nJjKO0JWhe/kC3MlROqqyE6MZdD+2B4D9HTEvd/v1JaDGwuvJfKbPwYPHsWl3sugu3J39w7h78/OYrEUtp6e+Y5Msxmz943tLErFksqqQYdTP3b9q3sV0G/2p1y9gs6DnrNajYHbq4Y62n9EC83l0mg4t+X0YzzXvlHvX6FO9yV6zvrf1OvLYKN9zExkDujJ0dSQY0FFNZLIKdz1iH9R5udtv58RoppDYxWw0M47RzLhtm9Y/vteyPV7WBp0YzVgWddc75YMHjxfWH0ZFcOvls22/RJKpNJb2bg/ti33twAg27jxUaLtV57lev4DNgnwh13MQUQ8dDa9Jf6zOo13CpUQ8hqd7ril5vFkD80qptyRLzapS15XTfpvxvV22cKbl9/CyhTNr0JrTGuV7biJjQFemhI9EE0RhyiqFrz+SW5+XGs0UfcnV6m6Z1cid17VBds8Hch1e45dbVik8tPMQzpgUxclT1vs1BxvldjQGhpJFwZxOT1yj7yOM2nlObaiHenr1MApmx+11VSOA8TpaYHUeFVAS1DldP80amDe7Zg6kK3VdTcTrdce+Y74edxLmNVfJ7zkKBwO6MnHaJQWlF7ktx7g6HQTpa9EGDx4v7RnWkNWoW1hOnsoipkVt968HXeu27C2ZWqp3CADnDnjftv22pzKVzhSKyVdq6k/YnZkgX8heg4hadFqdXlc1O4Rekv7YnUeF3E1Ct/M2MJRExOZzo9EDc10zBj7NHphU6rqqxfVa6+svrBkH5VxzbqOhQPNNcW0mDOjK1NWRwODB47ZT1IjcTG4RjGbCib4yWVXTazEe04p+HhhKeioHoUWk7LqOG1a2FbJ4WjEGckZ6sPfe2HjRl93q/mEMHjxeqCPo9gVqHKWrxNQfu87Mms3Plz21Tj9nfr+QvYzu1arT6tTRWNq73VOHMIxOnJfXb3ce7aZXWu3fKpgLOzBZZ1FFAAAgAElEQVS3O36lO3PNGvg0QiAdRKWmPVd7OnU9XH9hTW30e815ee3NOMW1mTCgC+DerjY8vOsQsnUyIkKNxW5NWiPKZMcLI1YAPGW+mt6q4Z7rF5U90q1/uSzt3e57+rNVsKcAbNx5CJ1zZ6CrI+GasMI4SgcUl4nQR1+NA6YRyY2smmsUWhkYStoeO50Zx9qBEdcC9lZfzhtWtrkGDla8jO7VstNq19Fw6xAODCUdR3H9tNvL6w8ySmq3Ptap/qNZuZ3FanV0mzXwqYd1npVUqfVV1V63VQ/XX1hTG/1ec/Xw2ikYli0IYGAoyWCOmpL43P7kqWxRCQIvHZXUaAaPDpY/qqina7dKX10uhdPBqJf96tsaU+cDKIyiGD8e9IFIq7T6xjT07eufKGQztfPwrsOu7QozxXRXRwIbVrYhEY+VlLnQ1WOn1a7jNyseK7xnVsG933PlFIAbX7+X82jH7jyOK+W5w1VuqvcwrierUgvm697LOWxETtdhM6hUCYFqlyaoh8+wIJ8RRn6vuVq/dpZiCS7QCJ2I/B2ATwF4XSl1scXvrwbwfwC8mn9os1Lqm0GOWU9Yf4Oa1QfOOQO/ef2kr+cYs126jW4BuWAnSI0+vaTC9FYNN12awI59xwrTwUZPjXma8mlF/wLTv0DNIzhW2/rNLmq882ke/bA7lpEeMNpNg7M790ESOVmNghmP77a2q17W12kRweipMdeRYa8dGf39s2PuQJU7bSmM0Ypy18EE7exZjfB1P7oHEBRKmThd934Dn1qvgzJr9oQSlVpfVe11W5UcEfRzTYYxtdHtmjO3J96qWX5nVuOmg9Xng3kJBLkLOuXyhwAeAPAPDts8pZT6VMDj1CVmuaRm5TeY050YzWBez9aQW+N+zE27k0V3Mf1k2DQzfoHpX6yLvvEvllk19W3LuYtZbjAI5KbZOU2DEwHs8u2EVdbBfHyntV31sr5uWkzDSY/BvteOjNP7F2an3aqDJvCfzryczmLQzp7VOfK6dtbvOazHzqGfwKTeglGvKrW+ym2/jZDJsRaff07XnFV7tIhAi0pRrdhq3XSwy/5rXAJB7gIFdEqpX4jIvHCaQkRUHvNcf+OXmVvhZrd08QNDSZwaK13vGI1IYVsvI5Jm8dZcIplygsEl50/HXY/sKQmi9PPglDzVqjNRTqfILRA1rhX0mpykEowdwqW92z2NgPrpyDi9f+VMlbKjJ+IyltJQADbtTtp2esJK9vLOu2Mlj2tR8XyOgtz89HsO67Vz6CXgqYekHI2kEudrckuksD99nXfQc1+r9Wl215zdDZZ4TMMZk1uqfjPBKfsv1/B5V42kKFeIyB4ARwH8sVJqr3kDEfkSgC8BwJw5c6rQJCJqNn6LisdjGtbdsMhT7TCr0YSs4TGrO7tu9KDLbzA4KSqOU1W9BIhOUz69doqcjhOV0519p6Q11V4b5eV401s1rLjkXPRt24/V/cOYFY9h2cKZRVN6jdeJU+ZKrx0Rr4HXjn3HLOsiGt9L82ikfsfd7n11O7bd9X/GpBbPr6/cEi1REazuH0bftv2e/64buXPIxBT+WJXGsTtf5uvc/De9bOFMbNqdLNrfuyElLqv1+jSvx30zncHwPddWtS1OpViA+snA2wgqHdA9B2CuUuodEfkkgAEAF5o3Uko9COBBAOjs7GyYNCNB6ogR0WmJMka4rCRTaXz9kWHcvfl51yyi+kiNW9ZHpy8UveOgf3lYjZo5HX9ez1bPCWgikhsVPOWSiWlWPIbU6Cnbwus6pymf6UwWq/uHC+vMrDJ0OgWiWaVK1kjZtbVc5XyJO7VZf20ASgJcY0kQc2C0bOHMkgL0fkb4/ATUdtdiMpUuXEt6O5ySvdhNTbY6tlPnz6tyvyf155nb5fTeO73H9Z5cpd46/kFVsqPtVBrH/P5bXefmv2mrsj9hBdPVztbppl7a41SKRTfNVBLJ+Fx9Bo7xc28ij2pXNKBTSr1l+PdPReSvRORspdR/VfK41XLr5bNZh44oBP/55ruh7WtceS8JcWc+aDGXEjB2RJyKtZs7WuV0XL0+Y1wB4y7BnL6m6id7XgPgHNApOI+eOWXoRP44Tp9/bmukjOvr/Hb63AIRu7vx5i9/vR3GaX0d33zCdaTVmOFx0+5k0f4EwE2Xel9PZBdQG+sc6txGc71cS8Zr1suIUBidvzBu2BjPudN73718AVb3D1uei1okV/G6j7UDI7bvn/632kjr7sKaDmn32pyS0kn+ecap9+WspwbCCabt1uYtWzgTS3u3F702vb1hv5fm0fsg6+XCut68vC9iccfTfG05zVqYSCoa0InI+wH8VimlROQy5MokvFHJY1aTvsD64V2HkVUKURFEI3C9g05ExWo90q1nzPz6I8O4YKYpw6dD04wZHJ0yHVaLvqbKa+fFKsBxE6T8gc5pNOzO/mGsf3yv49oVp6Lr5s68+e678XdWgbzX7KhHU2nb9Vo79h3ztA99P1bMdQ6B8qb2mkXyCXW6OhKeRoTCSBQRRrv1drkFoVZrDQH/CWTCCEi87mPtwIjrzeFkKl0oZ2KeMlutdXd+OvLlFLY27xuwD96dAi3z9NogQVkYo1ZWCUrMUzz1zK/jOD2d3+4998sqk7IWEUxv1ZAazfgKysK83ryWNzLzEgg26qh2EEHLFjwM4GoAZ4vIEQD3ANAAQCn1fQCfBvCHIjIGIA3gFqWaa47ivV1tRZmz5lc5wx8RhWdc+cvwqY8w+ZlqWWl+O80KjoOQloJ8WSbiscI0V6tkKUAuc2n3o/YdGaei636kRk8VrdHyE6gqh3ZYnR+7zrDTqJu58+s12Y+TrFKFDpjdsY1BH1CaKGLFJedi3Za9hSm5xuQRTp1+4+PltH9WPOYpCNW/k/0kkDHzE5A4jSJ52YdbXUldJqtw9+bnS9Y4VmrdXTlrMnV+po/aBQlTtIjta3O7hozHKfd6c0rz73dUypygxOqzz2pWg7EckFHQ4DozrtA6qQVD3/C3Zi7M683L+2JV9sbLd1VEBPN7ttbNiHU1BM1yeavL7x9ArqzBhFHuBwcRNZ7Bg8fR/6vDdRPMlUshF2g51ZMz0r9k/X7WmUd2nALDzLjCGlPnFUCoBWf1dYZ6BzLoCJJOz2Cqc7qr3b18gW1NvGQqXdIpsRuB8kPvgNmNnOlB3+DB4yUjvu+8O4Z/2nUIxr7nidEMuh/bU7K9XjLAalqz3/Im+gibPnXWzDyS4pZAxszcQfYarDu9t16DGj+fH6OZ8aJgO4wagVYjY+b6m17WZBr5maprFyTY/T0eTaVx25I5jn8DxuOUM0JsvF4rMX3Uz9+uedaA3/aEuTYzzH25Tdu3K3vjhd3622YWqXUDmk338gWIadGix7wmPSCixrJx5yHHpB+NIh7T8HTPNXi1dwW+ffNiaBH7Ty39S9Zv/bOoSEkKerfpTOnMONYOnJ7Kqo+GVkI6k7Vcr1EOc//c7a72GZOKvzOK9oXT07E6vvkE5vVsxUMBgjmd3tnesLINUYsXns5ksXHXIcs7+1bLIzNZhYd3Hbacgqofb83mkUJA3qr5637oI2zLFs60/I41X4/ljBAl8x1tfSqyFfM16/Te2l3f5setzr8T40iy2zEGhpJY2rsd83q24oI1P8W8nq1Y2rsdA0NJy9fd/egedD+2x1N5D+B0dmHzjRa7zwerx/0GA9NiWsnaVSPzjaOujgQ2rGxDIh6DIBesXXjOGbb7F+SSZTmtwfMz9XxgKIn29U/gzv7hwrkOwq49d/YPW74X5htMOuO1o18n8w3Xh9tzvDzuxGlqekROlyzxtNYu/3+7zzK792rtwEjh7+KCNT8t+r5pNAzoQmb1wXHbkjklX0BE1PgaP5TLSaUzhS+yro4E+n5/MeKG7GJ6fJeIx/DhOdOwun/Yd0KorFKFu916x+Hke2OuN7we2nkIawdGsHZgBKv7hys6GhrWrlPpTFHHyCk75W0/eMY1IymQC6S8ru/zSl+fM27zwv2eD7f3Ru9YDQwlXW+EWF0X6UwWO/Ydw02XFt9pVwD6nz1c1An10/G0WwtpboPV2kGnwNHqBq/VPm69fLblPuwYR77sjrFs4cyiIAIoHbWwSvufGVe+b1KZg3XAvrNu9bifYCCmRSFiP7U8HtMwRYtgtSm46epI4Omea3DfqnYAzlPrZ8VjRZ9TbqO1TsGQHjR7DZDtGPfrFACb3wsvdSStAnvz+6nzek174fQ6JrdEPK2B1PvZ961qx4HeFbafZVb70Neu6n8XWaUK3zeNqBp16CYc41zptQMjhaQpRET1amM+QDPWZlp3w+nEJLkv/efLnlKuZ54zL873ohGzCRs7RvFWzTYYc6opWGmZrMLXH7HOCFkpemkRq1E+yS/mdJvymMviWkyfouuUyc8uMYpT7bp4TCtcp1MsRhWdphZarR20Ws/TOXeGr2tc/1vS953OZAtllBLxGOadFXOdkus0pbEc5umXTjcxzLxOidSnQa62maIMAO+NjTtmv/VynGULZ3raTg/8zNMfV/cPY/Dgcdzb1RYow6aR8bW4Le0xvhde6kj6WRfn9Zr2wul1pDPjhQygdssAjOux3fZpddPAbu3qw7sOF+XGaBRSbzlKOjs71eDgYK2bEQovmauIiOqVFhWs+shs/GTPa4HvME9kMS0CQELtQDcrPSCxC3SNAZabCABzmhw9AZBxjZRd+Y54TCsKEIDcaMRNlyYKNz7MyUL0bczTi510fPMJ36OvTm0Lsr7SjV6T0ooAeLV3BQDnkii3L5mDzrkzSrI+On3OeN23Fb3j7+V58ZiGMya3uG6nv8d2CYoEwH2r2m1LaDhxqnEcFcGtl892zWasn6/5PVstj288n162sRI0UczAUNJ2/bAbAXDbkjklgZdV0B7TovjwnGnY+cqJQkZ6t7JjBxxedzWJyG6lVKeXbTnlsoK8Zq4iIqpHmWxuCgqDuWDSmXF8eM60WjejISRTacdRy1Nj3oNiq5ynVmv6rKaRAbni6VYjFxt3HipMT0ulM4DKZfrUp3/5CeaA0qQXXqRs2vbwrsMVHXEdV/Z5ARRywal+Tu08tPMQuh/bUzTFb9PupOMaVnOSEz/LWJKptOPUZ6NMdtwxmDO/x06ju07rKJ327zSjS58WKFCYbrM2Djh9vrysn3Oanmw3ndTLNE2n5y7t3e440upGX1Nrnhba1ZHATZcmCmvpoiI4b/oUPP3y8ZKplXb8rmmtF5xyWUGcZklEREBtp1Y2k1GfpSmc6NPKnu65BoMHj5d08pwKfRt5TQFfjQLgXvsdgtwUV7vRNqfnOT3lxGgGX39kGP/t8jmO+zGv03ObAmpOcmL1fjlZs3kErZOirutVnX6vj7qZEzs5TQ++b1W7rwyNXt+O0cy449/CsoUzPa2fA5wLn1tl0xw8eNxyKZFxmqbVVNTuR/fg7s3Ph/Y3bDUtdGAoWZR5OquUr1JEgP81rfWCI3QV1KhRPhEREYCiO93NSB9h8VMM3opdtkfdbT94pijLoXFEw2/Gz5gWdcyM6oUCXIMuu+e5GVfhrnuNx7SSwNfv+5XOZD0lH3Kij7oZdS9fYDtiGRHB6v5hTG6JFEZwq/V3tGl3Eusf32tb227w4OkbTFbJ/DasbMOOfcdsR6jtbhzowa1d7bswb8gApWs11z++13dSHz3pV1QEt1tM42wUHKGrILs5uvoFc9sPnuFdWyIiqksCYMn503HgjTSOptLQIkDI/bGa06eblVNHy8xqFGJ6q4aLzp1q+V2fzmSxbou/Dmgiv96s/9lgSzoS8Rju7WrDq8feqft+SCqdKdQtPGNSFKOnsjXLMGy+TpzqQupBTyqdgRaRQoZNuxE7p7VzfrmNeD608xBePfZO4W9bX8eorw21WxsIOAf1esKeMP6evDCXXihn+vK500qTqzQijtBV0L1dbbh9yZyiO5zG6P/AGyxATkRE9UkhN1VUH1VqtmAOOJ31spw6WlbMoxAnRjOOAVMqnbEcRbEyKZrrSzy085Dn51jRp9wNDCXx3KE3y95PLZysYTAH5EbdzGvBNu48ZJkB1SgzrrBuy14AuZT8Vqq9TMf4t51MpfGQYW2oUy1GJ+WuHSyHVemFciRTadf6e42AWS5ryC6zEBEREVWeFhX0fXoxAJSdca/RaBFB3+8vdhyFIXsC4KMXzMBzh970nbk2pkWrmu3WKStppehrDSv99xSPaVh3w6JQr2O/GWorjVkuG0Q17mAQERGRtUxWWdbbamaZcYW7HtnDYM6Cl1EpfeS6nMAsSDB3+5I5SPjsN1Y7mAOKazBWUiqdwer8utSw6IlWGhEDuhrym3qXiIiIwqWv9/HbWW5kWaXKmlLX7PRi8vVIL5dRr+3TpUZPYWAo6VhWISyViFertf4vbAzoashcL4OIiIiqa4oW8V2suhkoeBuRmmgqVXczEvBk68FLteqCxrRoWUHZyVNZdD+2BxedO7UCrcqp5HXbqLPnGNDV0MBQEpt2J1mvjoiIqEbSGedi0s2MvY/qqcX0xyBuujRRdlCWyaqKZk+t5KnUEyU1GpYtqDJjYdGIQ4raRDyG0VNjnlOwhpnuloiIiIgmrv5nD9VtZlsRoFJd3qA1KWuFI3RVpKdV1dPC2gVgAuDpnmtwz/WLPO1XAHz75sVcj0dEREREgdVrMAdULpgDuIaOPOjbtt9ThiN9/m5XR8LT/GU9o9CGlW0TalE3EREREVFYuIaOXHmJ+mNatFAoEQDuuX6R48ibcfuujgSe7rmGQR0RERERkQ/mPngj4Rq6KpoVj1kuvI6KYFwpzIrH0L18QVH9Dv3f+rq7aTENIkBqNFOyvb4+b6Iu7iYiIiIiKkc9FRX3iwFdFXUvX4A1m0eKpl16qUrf1ZFwvcD09XlOUzoFzGhFRERERNRMOOWyiozr3AS5TJZh3Q1wW5+nRQTxKhR5JCIiIiJqNOu27K11E8rGEboq8zLaVg6n9XnxmIaTPkogEBERERFNJKl0BgNDyYacdskRuiZhl5UnEY/hjMktyGQ52ZKIiIiIyM76xxtzlI4BXZPoXr6gJBumnq2nUWtqEBERERFVS6POZmNA1ySc1ufZjd7FYxq0iFS3oUREREREFBquoWsiduvzupcvQPeje5AZPz3tUosI1t2wCMDpkgiz4jEsWzgTm3YnXbNlxls1pEYziIggqzidk4iIiIioFhjQTRTmgbj8z1ZBYOfcGbizf9h2V/etai88Z37PVtvtogz2iIiIiIgqilMuJ4C+bftLkqJksgp92/Zbbt/VkUDCIcmKMQC0m84JgMEcERERETWMVq0xQ6PGbDX5YpcUxSlZilOSFbftiIiIiIgazf+z8pJaN6EsDOgmALtRNKfRNa9F0I3bERERERE1qsGDx2vdhLKIqrNpcZ2dnWpwcLDWzWgqA0NJrNk8UpToJKZFLQO0IOb3bIXd1SQApuULnLMmHhERERHVowO9K2rdBACAiOxWSnV62ZYjdBOA19G2oJyKm7/au4IFzomIiIiIQsYslxOEXUmDMHUvX2A5Eqivu2OBcyIiIiKicHGEjkLjNhJoN4IXFcHtS+YgHtMKj7VqEWhR+6Ln8ZhWWLfnpzS63i4iIiIiombAEToKldNIoN0Inh703dvVVrT9wFAS67bsRSqdKXo8pkWx7oZFheOsHRjBxp2HbNfvnX5eBC/+2XUAgKW925EMYcSQtfaIiIiIqJY4QkdV43ctX1dHAsP3XIv7V7U7PmfHvmOuwRwApDPjhX/blWWY3qqZnwYgNyJoVZ6BwRwRERFRc1h6wYxaN6EsHKGjqvK6lm9gKIm+bftxNJXGrHgM3csX2D7P69q8qJyenNnVkcDgweN4eNdhZJVCVAQ3XZpA59wZJaOIAuBTi89F59wZ6Nu2P5SRPSIiIiKqLxv/xxW1bkJZOEJHdUcvs5BMpaEAJFNprNk8goGhpOX2TvX0jIyjaQNDSWzanSw8llUKm3bn9n/TpYmidXkKKPyOhdSJiIiImpNdX7PeMaCjutO3bX/RCBkApDNZ9G3bb7m91yDLmAzF6RhWUzj131k9j4iIiIgaX/ejw7VuQlkY0FHdsZtCafe4vjbPOKXSzFg+we0Y5fyuGgQoyfwZ06K4fckcOLx0IiIiIvLAkG6hoTCgo7pjN4XSaWplV0cC3755seVI3fRWrSSRit2+IiKI2yRGmRWPuU7v1IPKRDxWSOYShumtGu5b1Y6+Ty8uSRBzb1cbbrt8TijHISIiIqLGEigpioj8HYBPAXhdKXWxxe8FwF8C+CSAUQCfU0o9F+SY1PzcCpTb0QM2L8lUrI4B5NbSvfPuGLSoIJM9PfHSeHyr5xmfr2/b1ZHA6n7noXsBEI0Ixsbts2Um4jE83XNNyes02rHvmO3+mYeTiIiIqHkFzXL5QwAPAPgHm99fB+DC/H+XA/jr/P+JbPkJzKye63U7ALjrkT0lpQcy4wrxmIYzJrfYHl9vW8SiDp2+3q6rI4FZ8ZhjVkwF58LoArgGsoD9FFKFXEDIzJxEREREzUlUwDpaIjIPwE9sRuj+BsDPlVIP53/eD+BqpdRrdvvr7OxUg4ODgdpE5NX8nq2WI1gC4NXeFYGfr2fsdEukEtMieDczXrQvAfDRC2bgwBtp18DWrlC6PrrntR1EREREE9kBD/2/ahCR3UqpTi/bVnoNXQLAYcPPR/KPFRGRL4nIoIgMHjtmPXWMqBLKWa/n5/nmYup20plx3GcqoH7bkjl47tCbnso32BVK10f3rIq637+q3fX13b+qHbcvmeOYcKZcMa1xlvCak9EQERER1YtKFxa36gWVDGgopR4E8CCQG6GrcJuICspdr+fn+cZpoBes+WnJFE0gl0zFPF10ae9229IK5lE6/ed1W/Yilc4AAKaYAiar6ahuhdL151gVXPcqHtNw8tRY0ZpEzWXdYD1JxGNYtnBmoQg9ERERNadGzRpe6VvkRwDMNvx8HoCjFT4mkWdWI1fmjJhhPt8uILB63G/5BgB4b+x0vt0ToxnHguxALiC1++xyq9vnVSqdAVQuU6d+jia1RIoCPDutWgSRGn+4zjsrVlSEnoiIiJrTR8+fUesmlKXSI3RbANwhIj9GLhnKm07r54hqwWsilTCeb5egxKq8QbxVw4nRTMnjdtM8nYql27WvqyOBwYPHsXHnoZKh89FTYxgYSqKrIxG4/l5mXKF1UguGvnEtBoaSuNMl+yeQG95/4c+uw8BQEt2P7kGmRiN6O1850XDBXESAaTHr64eIiIisvfDa27VuQlkCjdCJyMMAngGwQESOiMgfiMiXReTL+U1+CuAVAC8B+AGArwRqLVGDc1vrphsYSuKdd8dKnq9FxXY6aDkjegBwb1cb7lvVjnisuP6ecYTPLoiMihRG3W5fMsex7p7ejr5t+x3bo4uIFALKVZfNdlyDWEmNFswBwLgCWie1YLpNTUUiIiIq1ag3QgON0CmlbnX5vQLw1SDHIGomXksy9G3bbzkidcakFtvRNrsSCV4SvHR1JNC3bX9h/Z1OH+FbtnAmHtp5qOR5t14+G/d2tRU9ZpdxU2+H1xIKWaWwZvMIBg8ex6bdyZrV04talKZoBCxVQURENDFUesolEZl4maJpN6r2Ztr+zlHQBC9OI3x2hcutHndrh58AKZ3JuiYjiWlRTNEirnfVclk1xddawJgWxU2XJrBpd9LT8yIAprVqSI1mAAEaMA4kIiKiBtM4ecOJJpByyikETfDidEw/0znd2uF3tMtpe33f91y/CJpL9pSxcYWbLk14KsFgbPe9XW1Fryce0woJXoz/TsRj+M6qdgx941rctmQOgzkiIiKqCo7QEdWhckfbgiR4cTqmXXkDuyDQqR12iWHs2I3o6UXTdesf3+s4SpfJKuzYdwzfvnmxY5KVRH4abN+2/VjdP4y+bfvRvXxB0bGcDAwlsdFiemo1CCzqwgTY132r2j0lsCEiImoGDVq1gCN0RPUo6Ghb2Mf0mszFC6t92X2ACnLr9LwcO+VhIfPRVBpdHQmcOcX6XpYAWLZwJtZsHvFU0H1gKImlvdsxv2crlvZux8BQEn3b9tdsvd+rvSscE9P4MS2mYc3mkVD2RURE1AgadXINR+iI6lTQcgphHtNrMhevxzDva9nCmSXr1ATAbUvm4N6uNnTOneF6bLukMOZtAPvgTyG3LtBL+YeBoWTRiKYe+HlZaxcVwfkzW/Gb10+6buuVCDC/ZyumxTRoUfFU58+OFhWIoOzag0RERI2oUbNDM6AjIk/CDDCt9uUUtHk5ttWUUSPjqJ5d8JfwsV7Qru6fW9KXA70rCv+e17PVdjsrAmCKFkE6M17yO/2QqXQGWkQwPZ+cZVpMw8lTY5YBnt0UzTMmtdQ0dXMEQOkrDEeY01KN+9TXmtq9P5UkAFrKCOIT+ZspVhlsiYgmokZd/86AjojqQhgF3oHTI3/TYhpEcqNx5gAxjPWCdoGfUzBnvvPnZz2hvrbPyzRIYyF3AIWpoMlUuhBwOh37zXSmZuUaoiL49s2LbQveB+W2v3ICPgUUrbFcOzDimp3VLCK5+oHlUMgF4WdMbsHRVBoRD++dFhGcOPlexYK5eEwrKYNCRFTvnLKJ1zMGdETUNLwGhW5TSL0kpHEa5Ttx8j2MOoyi6dxGFXWS39ZqVNCOMeC0Oy9ONQPdAs1JUYECSkaFprdqOPvMSWVNJ41p0cK6za6OBDrnzsBdj+wpK7DU1xL6rccnZZSbMK9bvLerrVCfcWAo6ek1lBvM6d5MZzB8Ty6An+9l5FdgeY2GZd0Ni2xvjhAR1SsvtXvrEZOiENGE1NWRwNM91+DV3hV4uueaoumdXhLSOCWKsZtyl0pnipKoACgpiaBFi1PE6GsJuzoStqOCVrx8KTm9BrvkKol4DAd6V+DX3/ok+j69uOg83Z8v2zB6yn+gYKQY4k0AACAASURBVHWeuzoS+PbNiz0n0dFpUUH38gW+zpduXOWCOj+ckgN1dSRw6+WzXdscjwVbt6G/3wNDSUQ8vIAgayy90JMpuZUTISKqF/rN00bEEToiIhMvI31Oo3x2IxOC0yNGehKVDSvbiqbr6dMjrUYOvYycAd4zkAYdqbQ7T34DKXMJCrc2WiXRKZKPVezOl9u0SqVyr9WcpMfqOfGY5nitDAwl0f/sYcfjaRHByVNjDls406KCZQtnon39E3UxzVG/GaCfl3Vb9tZFu8wqsZ6SiBqXAgIt/aglUXW2+q+zs1MNDg7WuhlERGUzZ8AE7DuPTsGMl/0CQKsWwWQtarleMAin4NKJ3VROK8Zpln6Ob1wXaMW45tAclN50aQI79h1zfa5bEOnUdp1dkCX5C2JWPIbRU2Oek9AsvWAG9h59u7DP6a0aVlxyrnOAaxDTonhvLBt4iieQu+7eHRsv2pcWEfT9/uKSc+LnmgiT3d+d3zWiZ0yK4uQpZn0lamZ+vo+rQUR2K6U6vWzLEToiopBZjSrZdWb9jGaFWT7C6/HK2bfb2kBjMGPXfruyEMZ2dXUkML9nq2WHXa85CKAoIUw6k8WOfccKI412o5B+M7HasRuZUup0xlNPa96QC+Y2/o8rSh/v3e5pHabe5rCKxU9qiSIzrjBunL5pM8PS6Tr3kxwIcE8gExXBuFKOJVH8BHNRAU6NVS9zqTEInd6q4c10JpQAnIjsNfJ0S4ABHRFRRZgDAqcEJEH2W4/09tkGDipXBN2JXVkIcz1Au2B5lmnan1VwuGFlGzasbPMcpFXq3Nu9BnNgsmPfMczv2VrSTrebAua7zmElK7EKVjNZVfIeAc5JhJ7uucY2MLfyO1M0vDc2bhnE2o2aGjOm+o2NsspfAKjTY1s/z4xGBLdeNhs79h3D0VQarTUuIUI0UTTydEuASVGIiKrCKQFJM+rqSNgmVvESxHqtB+jlvLoFh1bJccJiV6TW+Ljda/j2zYvxau8KdC9fgE27k0im0lA4HZAODCUBOJ9Pq2vM6nhGQfOYHE2lMTCULEoAtGzhTMf3yc+NjTfTmUIyISAX+AL2CYx27DtWk7VyCv6Dx0lRKXmvmVbGm+mtGu5f1V7rZtRcSwMnIvKbkCpM0VoePAQM6IiIqsBr9sxmEiSItevgmx/3cl69BoeVcM/1i0oyl2pRwT3XLyr87PYanAJSwD5Am96qWV5jVse7f1U7DvSuwIHeFXhlwwrbYFwX06K2weq0mIY1m0eKgpJNu5O46dKE7Wt0CzKNZsVjhUD8QO8KvLzhkzjgEJAHfZ+r2c9LZ0pHHhXcM7u6icc0z+c3bPGYVvS+2103QUQjub+pro5ERfbfKLQIMFaB+blnTIri/lXtuH3JnELgI8jdgAhLTIvWtKh3LequholTLomIqqQRpkuGKciaP6fi71bHcdqn27TMSvJ6Dpxeg1tAWs55djtnyxbOLCnsrq/t0pPGANZrEEVgGYDu2HfMNZupW0bMcka1vWaHBXJJXTKGDrEWEay6bLbnpDNm8Zj99FA/9PN+NJXGtJiGk6fGSkpPTIoKTlmUoxAAn1p8btEa0Gp2XY01EoHc+tjuR/cUneegsuMKgwePo6sjgXuuX+Spvme5tIjgzCkthSRU886K4d9fPl4XGVPLLS3plvH13cx44TNDr7Gpc0tQ5UU8pll+blSCXUIkt5tY9Y4BHRERVUy5QWyYCWD8BIeVEDSQ9xKQBjmGOZuonkjEHMzdtmROSWcOKH2PVtusnXQbKdNfg7E90/IdvSAZXN2S9Oisspvqx9ODIT+d1pgWxbobciOxxtfz1rvWSU60qODMydZr5szrIJ0ywK4dGClZM7hpdxKdc2cU9mG3pjemRQCIp451PKbhjMktrq/LalR9/eN7Q18buHHnIXTOnVGSDMmvhEvm2cy4QuukFgx943SQaj7njSQC4Dur2tHVkcA8mwRNTqNXhVH2MoP025fMKfuGiV96lmOrjMWNvvyBZQuIiCiwckscVEu9t8+JVbkKLyUTyt130BIbdsFC2CnB/byn5iDRPMLl9XzalQ4xm96qFaYAWu3DPBKpbw9Yj3r6ea+9nH+n9z1uCKKdeoj354MA4+vy2nY/SXD8MF9ji77xL77LTejr8Bwz9aI4sVOtynKERU/A5Of9NgpSA9NvCREgNw36tsvn4OFdhz0/NyqCb9+8uOSmUT1/H7BsARERVY1biYF60MjTXStZrsJqfZ5d98jrWrRqjIj6vebM73+5HTqrUhhZpQr/T4Qw3VXff7nvtZc1o+bXYQziU+kMYloU961qtx3lisc0y2nDXtvuZxqsH+bXPlpG7cC+bfsLQeFdj+yxDBjMo47VWI9bSV6ConVb9tpeh+UGc16PbWS8SbBx5yHfzwEa+/vADgM6IiIKxGuJASpfpTogfjqiXtccVqNeYtBrLsj5rHRnsBpTdI3HsRpd0s+lXXCuTyUtt+1W+9UiuaA4yNI682ssJ3A0rk0dPHgcD1kEDclUGkt7txeuay/H0af76SUpImWMTAUR06KYokXKnuqaSmcwMJQM/dr3MkJnXr+rt8HpvE9v1QJN1W40zHJJRESB1DKLJAVjF6SZc9f5HWGrdDkIu2urkae9hcVvdlmnv99KZee12m/f7y/Gd25uz6/j889riY6YFkWrwzGMfxM79h2z3c5YPsTqOFpEChk3oyKFxEDdyxfg1d4VGK9iMKe/b/dcvyhQtlM9s25YBMCtl8+2PXf6tXFfPguv+bOke/kC2wywJyZQMAdwhI6IiAKqZRZJCsZuBMY4klCPnSK7a06AiowiNBK/I6Ruf7+VGpG026/dNE+3TIx2JTr0fRrPBQB8vX8Y5qSQWlSKgkK3m1L6SKY+RdPqOHZTg72M6hlfsxbxnsXSab0b4J5N1o7d+ZjeqpU18qcnWjJmYPU7BfpOmyRMQH1O/68UBnRERBRIrbNIUvmqMT2yEvRsmuYOvgI41Rf+grB6+/u1Cxr0KXd2CV/sXq/TuTAGNlaJbLwEXcYpmubjLO3dbjs12C37qhYRQFBI3pMZLy6ZYDdl02p9o9l7Y+XVN7C7SXfP9YvQ/dieklIadsxZc4PcNLC7JnQTZfo/AzoiIgqkUYMCymnEBAFOd+Y51defevv7tQui9PVT5QafVolwjPXxrHgpeeE0E8FtOitQXNLCWKLDqnSCsWSCXVZRu/WNOqv1p144nWfza3EK65wywJbDy3s0ET4TGNAREVFgjRgUUGOzuzPPqb7+1dPfr1PQVm7wWW4mXj0xil2NOS0ijsGk3+msxqDTLdtsueeinODGb/ZWpzIOrZNaQr3WvNQdnAifCQzoiIiIqOHU21RBcua1VIRboFJO8BkkK+qOfcfsR5zsMnLk+blGvdY4NAYn5ZwLuyDTLttkOfUju5cvqOoIun4e7EYtJ8JnAgM6IiIiajj1NlWQ7AWtGxhUkEy8TttkssoxKPRzjXqZChlGcOKUCGnT7mQowVBXRwLrH99rmSjFabQsaMHvifyZwICOiIiIGlI9TRUke7WuVRkkE69bYhS3oNDrNeq0H8m3I4zgxCno0bNNJlPpQqkFvVSB3+Pec/0iX6Nl5U6LtXp9E/EzgQEdEREREVVMrWtVBpme65Z0I6z1WU7JYPxOeXRjF/Toj4UVWAHeR8tqHfQ3OgZ0RERERFQxta5VGWQqnr6NVe22MNdn1cua0DADKz+jZbUO+hsdAzoiIiIiqph6CFaCTMUzJt2o1Pqseln/VavAqtZBf6MTZZHRppY6OzvV4OBgrZtBRERERCGpZDBE4bErORCPaThjckvF3j+7DJUbVrZN2OtERHYrpTo9bcuAjoiIiIiIrAIrLSKA5LJ66ioRbDHoL+YnoOOUSyIiIiIispz6OXpqrKQEQSUSlkzUDJVhYEBHREREREQASgOr+T1bLbdjwpL6Eal1A4iIiIiIqD7ZJSZhwpL6wYCOiIiIiIgsdS9fgJgWLXqsFiUVyB6nXBIRERERkaV6KalA9hjQERERERGRLSYsqW+ccklERERERNSgGNARERERERE1KAZ0REREREREDSpQQCcinxCR/SLykoj0WPz+cyJyTESG8/99McjxiIiIiIiI6LSyk6KISBTA/wfg/wJwBMCvRGSLUuoF06b9Sqk7ArSRiIiIiIiILAQZobsMwEtKqVeUUqcA/BjA74XTLCIiIiIiInITpGxBAsBhw89HAFxusd1NInIVgF8DWK2UOmzeQES+BOBL+R/fEZH9AdpVKWcD+K9aN4ImDF5vVC281qhaeK1RNfF6o2qp1LU21+uGQQI6sXhMmX5+HMDDSqn3ROTLAH4E4JqSJyn1IIAHA7Sl4kRkUCnVWet20MTA642qhdcaVQuvNaomXm9ULfVwrQWZcnkEwGzDz+cBOGrcQCn1hlLqvfyPPwBwaYDjERERERERkUGQgO5XAC4UkfkiMgnALQC2GDcQkXMNP94A4MUAxyMiIiIiIiKDsqdcKqXGROQOANsARAH8nVJqr4h8E8CgUmoLgP8lIjcAGANwHMDnQmhzrdT1lFBqOrzeqFp4rVG18FqjauL1RtVS82tNlDIveyMiIiIiIqJGEKiwOBEREREREdUOAzoiIiIiIqIGxYDOAxH5hIjsF5GXRKSn1u2hxiQiB0RkRESGRWQw/9gMEfmZiPwm///p+cdFRL6bv+aeF5EPG/bz2fz2vxGRz9bq9VB9EZG/E5HXReQ/DI+Fdn2JyKX56/el/HOtStfQBGBzra0TkWT+821YRD5p+N2a/HWzX0SWGx63/G7NJ1vblb8G+/OJ12gCEpHZIrJDRF4Ukb0i8rX84/xso1A5XGuN8dmmlOJ/Dv8hl/DlZQDnA5gEYA+Ai2rdLv7XeP8BOADgbNNjfw6gJ//vHgD/b/7fnwTwz8jVe1wCYFf+8RkAXsn/f3r+39Nr/dr4X+3/A3AVgA8D+A/DY6FdXwCeBXBF/jn/DOC6Wr9m/ldX19o6AH9sse1F+e/NyQDm579Po07frQAeAXBL/t/fB/CHtX7N/K82/wE4F8CH8/+eCuDX+WuKn238r1rXWkN8tnGEzt1lAF5SSr2ilDoF4McAfq/GbaLm8XsAfpT/948AdBke/weVsxNAXHJlQJYD+JlS6rhS6gSAnwH4RLUbTfVHKfUL5LIJG4VyfeV/9ztKqWdU7pvoHwz7ognG5lqz83sAfqyUek8p9SqAl5D7XrX8bs2PjlwD4LH8843XLU0wSqnXlFLP5f/9NnLlrxLgZxuFzOFas1NXn20M6NwlABw2/HwEzm8wkR0F4AkR2S0iX8o/9j6l1GtA7sMEwDn5x+2uO16P5EdY11ci/2/z40RGd+Snuf2dPgUO/q+1swCklFJjpsdpghOReQA6AOwCP9uogkzXGtAAn20M6NxZzaVmrQcqx1Kl1IcBXAfgqyJylcO2dtcdr0cKg9/ri9cduflrABcAaAfwGoBv5x/ntUaBiciZADYBuFMp9ZbTphaP8XojzyyutYb4bGNA5+4IgNmGn88DcLRGbaEGppQ6mv//6wD+N3LD8r/NT/lA/v+v5ze3u+54PZIfYV1fR/L/Nj9OBABQSv1WKZVVSo0D+AFyn2+A/2vtv5CbJtdiepwmKJH/n707D4+yOvs4/j3ZEwgJEAgkJIQ1bIGgAWQVlwIWRcQNtdbdaovtayuK1raulYpLXduibbXWBVdAVFBUKgKyh33fQhbCEhICWSc57x8ziSGZQJZJhoTf57q4ZJ7nPM+5Jw6Tueeccx/jj/MD9tvW2o9dh/XeJh7n7rXWVN7blNCd3kqgh6syTQAwGZjr5ZikiTHGtDDGhJb9HRgDbMT5WiqrtnUTMMf197nAz10Vu84DclzTShYAY4wxrV3D/mNcx0Tc8cjry3Uu1xhznmsdwM8r3Euk7EN1mStwvr+B87U22RgTaIzpAvTAWYTC7e9W1zqmb4GrXNdXfN3KWcb1fvNPYIu19rkKp/TeJh5V3Wutqby3+Z2+ydnNWuswxkzB+WbgC/zLWrvJy2FJ0xMJfOKqhuwHvGOtnW+MWQm8b4y5DUgBrna1/xxnta6dQB5wC4C1NssY8zjONwyAx6y1NS1OIM2YMeZdYDQQYYxJBf4ETMdzr6+7gTeAYJyV4L5o4KckZ6hqXmujjTGJOKcQ7QV+AWCt3WSMeR/YDDiAX1lrS1z3qe536wPAe8aYJ4C1OD9kydlpOHAjsMEYk+w69hB6bxPPq+61dl1TeG8zrtKZIiIiIiIi0sRoyqWIiIiIiEgTpYRORERERESkiVJCJyIiIiIi0kQpoRMREREREWmilNCJiIiIiIg0UUroRESkyTPGHHf9N84Yc72H7/1QpcdLPXl/ERGR+lBCJyIizUkcUKuEzhjje5omJyV01tphtYxJRESkwSihExGR5mQ6MNIYk2yMudcY42uMmWGMWWmMWW+M+QWAMWa0MeZbY8w7wAbXsdnGmNXGmE3GmDtdx6YDwa77ve06VjYaaFz33miM2WCMubbCvRcZYz40xmw1xrxtjDFe+FmIiMhZwM/bAYiIiHjQNOA+a+2lAK7ELMdaO8gYEwgsMcZ86Wo7GOhnrd3jenyrtTbLGBMMrDTGfGStnWaMmWKtTXTT1yQgERgARLiu+c51biDQF0gHlgDDge89/3RFRORspxE6ERFpzsYAPzfGJAPLgbZAD9e5FRWSOYBfG2PWAT8AMRXaVWcE8K61tsRamwn8DxhU4d6p1tpSIBnnVFARERGP0widiIg0Zwa4x1q74KSDxowGTlR6fDEw1FqbZ4xZBATV4N7VKazw9xL0+1ZERBqIRuhERKQ5yQVCKzxeANxtjPEHMMb0NMa0cHNdGHDUlcz1As6rcK647PpKvgOuda3TaweMAlZ45FmIiIjUkL4xFBGR5mQ94HBNnXwDeAHndMc1rsIkh4CJbq6bD9xljFkPbMM57bLMTGC9MWaNtfaGCsc/AYYC6wAL3G+tPeBKCEVERBqFsdZ6OwYRERERERGpA025FBERERERaaKU0ImIiIiIiDRRSuhEROSM4SowctwYE+vJtiIiIs2V1tCJiEidGWOOV3gYgrNcf4nr8S+stW83flQiIiJnDyV0IiLiEcaYvcDt1tqFp2jjZ611NF5UTZN+TiIiUlOacikiIg3GGPOEMWaWMeZdY0wu8DNjzFBjzA/GmGxjTIYx5sUK+8T5GWOsMSbO9fi/rvNfGGNyjTHLjDFdatvWdf4SY8x2Y0yOMeYlY8wSY8zN1cRdbYyu8wnGmIXGmCxjzAFjzP0VYvqDMWaXMeaYMWaVMSbKGNPdGGMr9fF9Wf/GmNuNMd+5+skCHjbG9DDGfGuMOWKMOWyMecsYE1bh+s7GmNnGmEOu8y8YY4JcMfeu0K6jMSbPGNO27v8nRUTkTKWETkREGtoVwDs4N++eBTiA3wARwHBgHPCLU1x/PfAHoA2QAjxe27bGmPbA+8BUV797gMGnuE+1MbqSqoXAp0BHoCewyHXdVOAqV/tw4Hag4BT9VDQM2AK0A/4CGOAJVx99gK6u54Yxxg/4DNiJc5+9GOB9a22B63n+rNLPZIG19kgN4xARkSZECZ2IiDS07621n1prS621+dbaldba5dZah7V2N86Nu88/xfUfWmtXWWuLgbeBxDq0vRRIttbOcZ17Hjhc3U1OE+MEYL+19gVrbaG19pi1doXr3O3AQ9baHa7nm2ytzTr1j6dcirX2b9baEtfPabu19mtrbZG19qAr5rIYhuJMNh+w1p5wtV/iOvcmcL1rI3WAG4G3ahiDiIg0MX7eDkBERJq9/RUfGGN6Ac8C5+IspOIHLD/F9Qcq/D0PaFmHtlEV47DWWmNManU3OU2MMThHxtyJAXadIr5Tqfxz6gC8iHOEMBTnl7CHKvSz11pbQiXW2iXGGAcwwhhzFIjFOZonIiLNkEboRESkoVWuvvUPYCPQ3VrbCvgjzumFDSkD6FT2wDV6FX2K9qeKcT/QrZrrqjt3wtVvSIVjHSq1qfxz+gvOqqEJrhhurhRDZ2OMbzVx/AfntMsbcU7FLKymnYiINHFK6EREpLGFAjnACVfxjlOtn/OUecA5xpjLXOvPfoNzrVpdYpwLxBpjphhjAowxrYwxZevxXgeeMMZ0M06Jxpg2OEcOD+AsCuNrjLkT6HyamENxJoI5xpgY4L4K55YBR4A/G2NCjDHBxpjhFc6/hXMt3/U4kzsREWmmlNCJiEhj+x1wE5CLcyRsVkN3aK3NBK4FnsOZCHUD1uIcAatVjNbaHOAnwJXAQWA7P65tmwHMBr4GjuFcexdknXsE3QE8hHPtXndOPc0U4E84C7fk4EwiP6oQgwPnusDeOEfrUnAmcGXn9wIbgCJr7dLT9CMiIk2Y9qETEZGzjmuqYjpwlbV2sbfjaQjGmP8Au621j3g7FhERaTgqiiIiImcFY8w4nFMVC4AHcW5NsOKUFzVRxpiuwOVAgrdjERGRhqUplyIicrYYAezGOeVxHDCxORYLMcY8BawD/mytTfF2PCIi0rA05VJERERERKSJ0gidiIiIiIhIE3XGraGLiIiwcXFx3g5DRERERETEK1avXn3YWnuq7XXKnXEJXVxcHKtWrfJ2GCIiIiIiIl5hjNlX07aacikiIiIiItJEKaETERERERFpopTQiYiIiIiINFFn3Bo6ERFpHMXFxaSmplJQUODtUKSJCAoKolOnTvj7+3s7FBERcVFCJyJylkpNTSU0NJS4uDiMMd4OR85w1lqOHDlCamoqXbp08XY4IiLioimXIiJnqYKCAtq2batkTmrEGEPbtm01oisicobRCJ2IyFlMyZzUhl4vIlLZ7LVpzFiwjfTsfKLCg5k6Np6JA6O9HdZZRQmdiIiIiIjU2uy1aTz48Qbyi0sASMvO58GPNwAoqWtEmnIpIiJe8+STT9K3b1/69+9PYmIiy5cvB8DhcPDQQw/Ro0cPEhMTSUxM5Mknnyy/ztfXl8TERPr27cuAAQN47rnnKC0trVMMw4YNA2Dv3r2888475cffeOMNpkyZctrrR48eTWxsLNba8mMTJ06kZcuWdYqnvrKzs3n11Ve90reIND2z16YxfPo3dJn2GcOnf8PstWmnbH+i0MGOzFwWbTvII3M3lSdzZfKLS5ixYFtDhiyVaIRORERqxNPTapYtW8a8efNYs2YNgYGBHD58mKKiIgAefvhhDhw4wIYNGwgKCiI3N5dnn322/Nrg4GCSk5MBOHjwINdffz05OTk8+uijtY5j6dKlwI8J3fXXX1/re4SHh7NkyRJGjBhBdnY2GRkZtb6Hp5QldL/85S+9FoOINA3uRtimfbSePYeP0619KOnZ+eV/0rILSM/OJye/+LT3Tc/Ob+jQpQKN0ImIyGmV/dJPy87H8uO0mtN9k3sqGRkZREREEBgYCEBERARRUVHk5eXx2muv8dJLLxEUFARAaGgojzzyiNv7tG/fnpkzZ/Lyyy+fNEoG8Mtf/pK5c+cCcMUVV3DrrbcC8M9//pOHH34YoHwkbdq0aSxevJjExESef/55ANLT0xk3bhw9evTg/vvvr/a5TJ48mffeew+Ajz/+mEmTJpWfs9YydepU+vXrR0JCArNmzQJg0aJFnH/++VxzzTX07NmTadOm8fbbbzN48GASEhLYtWsXAIcOHeLKK69k0KBBDBo0iCVLlgDwyCOPcOuttzJ69Gi6du3Kiy++WP48du3aRWJiIlOnTmXRokVceuml5fFMmTKFN954A4C4uDgeeughhg4dSlJSEmvWrGHs2LF069aNv//979U+XxFpHmYs2FZlhK3AUcoLX+/k1++uZfoXW5m9No3Uo/lEhQUxYUAUD4zrxQuTE/ngrqFEtgp0e9+o8ODGCL/Wo4vNlUboRESERz/dxOb0Y9WeX5uSTVHJyVMa84tLuP/D9by7IsXtNX2iWvGny/pWe88xY8bw2GOP0bNnTy6++GKuvfZazj//fHbu3ElsbCyhoaE1jr9r166UlpZy8OBBIiMjy4+PGjWKxYsXM2HCBNLS0spHzr7//nsmT5580j2mT5/OM888w7x58wDnlMvk5GTWrl1LYGAg8fHx3HPPPcTExFTp/6KLLuKOO+6gpKSE9957j5kzZ/L4448DzgQvOTmZdevWcfjwYQYNGsSoUaMAWLduHVu2bKFNmzZ07dqV22+/nRUrVvDCCy/w0ksv8de//pXf/OY33HvvvYwYMYKUlBTGjh3Lli1bANi6dSvffvstubm5xMfHc/fddzN9+nQ2btxYPoK5aNGiU/7sYmJiWLZsGffeey8333wzS5YsoaCggL59+3LXXXfV+P+BiDQ9pxpJ+/LeUXQMCyI0qPp9Jx+8pPdJI3wAwf6+TB0b79E43dH6vR9phE5ERE6rcjJ3uuM10bJlS1avXs3MmTNp164d1157bfnIUUX//ve/SUxMJCYmhv3791d7v8qjcwAjR45k8eLFbN68mT59+hAZGUlGRgbLli0rXzt3KhdddBFhYWEEBQXRp08f9u3b57adr68vI0aMYNasWeTn5xMXF1d+7vvvv+e6667D19eXyMhIzj//fFauXAnAoEGD6NixI4GBgXTr1o0xY8YAkJCQwN69ewFYuHAhU6ZMITExkQkTJnDs2DFyc3MBGD9+PIGBgURERNC+fXsyMzNP+5wqmzBhQnmfQ4YMITQ0lHbt2hEUFER2dnat7yciZz5HSSmvfLuTqu+aTtHhwfSMDD1lMgfOxOmpSQlEu0bkDPCHy3o3SkLlbnQxv7iEP8zZyGfrM9iRmUvxKX5HNafRPY3QiYjIKUfSAIZP/4Y0N9/kRocHM+sXQ+vcr6+vL6NHj2b06NEkJCTw5ptvcs0115CSkkJubi6hoaHccsst3HLLLfTr14+SkhK399m9eze+vr60b9/+5PiiQJLezQAAIABJREFUozl69Cjz589n1KhRZGVl8f7779OyZcsajQCWTQcti9XhcFTbdvLkyVxxxRVVpoa6SzTd3d/Hx6f8sY+PT3lfpaWlLFu2jODgqlOYahKfn5/fSQVjKu8jV7HPyvGc6vmKSNO0PTOXqR+sY11qDgM6tWJb5nEKin98j6jtCNvEgdFMHBjNhtQcLnv5e4qK6/5FX21UN7qYW+DgV++sAcDf19AlogU9IkPp2T6UnpEt6REZyvr9R/n97E3NZnSvRiN0xphxxphtxpidxphp1bS5xhiz2RizyRjzToXjJcaYZNefuZ4KXEREGs/UsfEE+/uedKy+02q2bdvGjh07yh8nJyfTuXNnQkJCuO2225gyZUp58lFSUlJeMKWyQ4cOcddddzFlyhS3+6QNHTqUv/71r4waNYqRI0fyzDPPMHLkyCrtQkNDy0e+6mLkyJE8+OCDXHfddScdHzVqFLNmzaKkpIRDhw7x3XffMXjw4Brfd8yYMbz88svlj8umUlan8vPo3LkzmzdvprCwkJycHL7++usa9y0izUfZqNylL37P/qP5vHL9OcyZMpLpk/oTHR6Mwfkl3VOTEuqU1CR0CmNApzDeXp5yyi+yPCUqPMj98bAg5t0zgueuGcBtI7oS0zqE9anZPL9wO3e/vYaLn/sfv/1gfbOqznnaETpjjC/wCvATIBVYaYyZa63dXKFND+BBYLi19qgxpuJXpPnW2kQPxy0iIo2o7Je7J6tcHj9+nHvuuYfs7Gz8/Pzo3r07M2fOBJzbGfzhD3+gX79+hIaGEhwczE033URUVBQA+fn5JCYmUlxcjJ+fHzfeeCO//e1v3fYzcuRIvvzyS7p3707nzp3Jyspym9D1798fPz8/BgwYwM0330zr1q1r9XyMMdx3331Vjl9xxRUsW7aMAQMGYIzh6aefpkOHDmzdurVG933xxRf51a9+Rf/+/XE4HIwaNeqUBUvatm3L8OHD6devH5dccgkzZszgmmuuoX///vTo0YOBAwfW6nmJSNO3PTOX+z5Yx/rUHMYndOSxy/vStqVzRL5shM0TbhjSmfs/Ws/KvUcZ3KWNR+5ZnSvP7cSLX+886Viwvy/3j+tFv+gw+kWHnXQur8jBzoPH2Z55nPs+WOf2nk21Oqc5XQZtjBkKPGKtHet6/CCAtfapCm2eBrZba193c/1xa22NN+NJSkqyq1atqmlzERGpoy1bttC7d29vhyFNjF43Ik2Ho6SUf3y3mxcW7qBlkB+PX96P8f07Nlh/+UUlDP7zQi6Ib8+L1zXsl0e/eGsV320/RHhIAAdyCmr1ReOplhEsmXZhQ4Rba8aY1dbapJq0rckaumig4ir0VGBIpTY9XR0vAXxxJoDzXeeCjDGrAAcw3Vo7203AdwJ3AsTGxtYkbhERERFpZjy93+XZbNsB56jchrQcxvfvyGMTfhyVayjBAb5ceU4n3l6+j8PH+xDRQP3tPXyCLzdncvf53bh/XK9aXz91bLzXqnM2hJokdFUXJFClKI4f0AMYDXQCFhtj+llrs4FYa226MaYr8I0xZoO1dtdJN7N2JjATnCN0tXwOIiIiItLEqQx93VVMhDuGBzEwJpyvNh8kNMiPV284h58mNNyoXGU3DInljaV7+XB1Kned361B+vjXkj34+RhuGhZXp+sbYhmBN9UkoUsFKm660wlId9PmB2ttMbDHGLMNZ4K30lqbDmCt3W2MWQQMBHYhIiJeZ611W0hExJ3GKHQgZ6/qytDPWLCtyX7QbgyVE+H07ALSsw+QGBPGP28a1OCjcpX1iAxlSJc2vLM8hTtHdsXHx7O/Y7LzivhgVSqXJ0YT2cp9YZSa8OTaQW+rSZXLlUAPY0wXY0wAMBmoXK1yNnABgDEmAucUzN3GmNbGmMAKx4cDmxEREa8LCgriyJEj+pAuNWKt5ciRIwQF1f0DlIg7h48X8t6KFLdrmsA5Urc9M1fvVdVwlwgDHMotavRkrswN53UmJSuPxTsPe/zeby9PIb+4hNtHdvH4vZuq047QWWsdxpgpwAKc6+P+Za3dZIx5DFhlrZ3rOjfGGLMZKAGmWmuPGGOGAf8wxpTiTB6nV6yOKSIi3tOpUydSU1M5dOiQt0ORJiIoKIhOnTp5OwxpBvZn5bFg0wEWbDrAqn1HsRZ8fQwlpe6TtjHPf0dc2xB+0ieSMX07cE5sa3w9PPLTVFVXmdGbFRvH9o2kbYsA3v5hH+f3bOex+xY6Snhj6V5G9oigV4dWHrtvU1ejjcWttZ8Dn1c69scKf7fAb11/KrZZCiTUP0wREfE0f39/unTRN5wi4lnuCptcnhjFloxcvtx8gAWbMtmScQyAXh1C+fWFPRjbtwPbDhzjoU82VilUMe2SeHx9fPhqcyZvLN3La4v30LZFABf1bs+YPh0Y0SOCINc+mWdjUZWo8GC3o5tR4cFeiMYp0M+Xq5NimPndLjJy8ukY5plY5iancyi3kGevHuCR+zUXp922oLFp2wIRERFp6rydWHir/8rrucA58hYW7EfWiWKMgaTOrRnbtwNj+nQgtm1IreLOLSjmf9sP8eWmTL7ddpDcAgfB/r6M6hlBmxYBfLI2jYLi0vL2wf6+dd4ouy7P3Rs/80/WpPLb99edVLGwMZ93dVKO5HH+M9/y6wt7cO9Petb7ftZaLnlhMQBf/GZks1//7eltC0RERESkhjxRrbE+yYE3q0U+PX9rlfVcJaWWE4UlPDUpgYt7R9IutPp1XacrVBEa5M+l/aO4tH8URY5Slu85wlebM/lyUyYHjhVUad9YRVW8+TOP79AKC4QF+3Msv/iMGZmMbRvCqB7teG9lCvdc2B0/35qU7qje4h2H2XoglxlX9W/2yVxtKaETERER8aDqqjXe/+F65q3PoG2LANq0DKBtiwBah/z49zauP19uynSbHDhKShnRox1H84o4eqKILNd/j+YVk3WiiKN5RWSdKOKH3UcoLrFV+n96wdYG+ZB/+Hgh32w9yMLNmaTnVE2qAIocpVw32LN7DQf4+TCyRztG9mjHoxP60vXBz6vsqwUNv5asyFHKY/M2e61C55zkNPx8DIvuG03rFgEN2ldt3TAkljvfWs3CLQcZ169Dve712uLdtAsNZEJilIeiaz6U0ImIiIh4UHXVGotKSkk9msf61GyyThThqKYAiDv5xSXc9+H6as+HBvnRxpUgVk7myqRnF/Crt9cwrHtbRnSPILZNSJ1HOnYePM7CLZks3JzJ6hRnUZOOYUG0CPDlRFHViosNvZ7LGFPtWjIL3PbGSn4+LI6R3SM8VkZ/e2Yu76/czydr08g6UeS2TUMnk6Wllrnr0jm/Z7szLpkDuLBXezqGBfH28n31Sui2ZBxj8Y7DTB0bT6CfrwcjbB6U0ImIiIh4QOaxAh79dFO156PDg5n/f6MA53qgYwUOsk4UVfhTyJETRTw9f1u193jyin60CQmgtWs0r3VIAOEh/vhXmM42fPo3bhObYH9fVu87ymcbMgDo1DqYEd0jGN49gmHd2paXuHc33fOyAVGs3ne0PInbffgEAH2jWvGbi3pwce9I+ka1Yk5yepU1dMH+vkwdG1+Ln2TdTB0bX6XvID8fRvWMYE1KNjf9awVxbUO4cWgcV53bibBg/1r3caygmHnrMpi1aj/r9mfj52O4uHckK/dmccRNUtfQieyKvVlk5BTw4E97N2g/deXn68PkQbE8v3A7+46coHPbFnW6z+uL9xDs78sNQzw7yttcKKETERERqYeSUst/f9jHjAXbKC4pZXxCB77eerBKcY6KSY0xhrBgf8KC/ekScfKH3Ld/cL8nW3R4MDcM6XzaeNwlNmVFMi5PjGL34RMs2XmY73cc5rMNGby3cj8AvTu2okOrAJbszKKoxBl7WnY+v3t/HQ99vJ684lL8fQ1Du0Vwy/A4LuodWSVhKZte6I3iIKfqu9BRwvyNB/jPsn08Pm8zzyzYxsSBUdx4Xhx9ok5d/r601LJ8TxYfrNrP5xszKCgupWdkSx4e35srBkbTtmWg22IwjZHIzklOIyTAl4t7t2/Qfurj2kExvPjNDt5ZnlKnxDPzWAFz16Vx/eBYwkPOvFHIM4GqXIqIiIjU0ab0HB76eAPrUnMY2SOCJyb2o3PbFh4tagK1r1pY0/4dJaVsTD9WnuAt233E7f2C/X155uoBjOoZQWhQ7Ue2ziQb03J4a9k+ZienUegoZVBca34+NI7C4hKeX7ij/Gd2+8guHC9w8MHqVFKy8ggN9GNCYhTXJMXQv1NYlemqZT/zsmT8z5P6cf3g0yfgdVXoKGHwk19zYa/2PH9tYoP14wl3vbWa5XuO8MNDF9V6yuTT87fyt//tYtF9o+s8wtcU1abKpRI6ERERkVo6Uejg+a+28++le2kd4s8fLu3DhAFRHqu+560S+F2mfea2sIgB9kwf3+D9N6bsvCI+WJXKWz/sIyUrr9p2w7q15ZqkGMb27UBwwOmTkQ2pOVz28vc8dnlffj40zoMRn+zLTQe4863V/PuWQVwQf+aO0AEs3nGIG/+5ghcmJ3J5Ys1fx3lFDoY+9Q1Du7bl7zee24ARnnm0bYGIiIhIA1m4OZM/ztlIek4B1w+J5YGxvQgL8eyo1enK9zeUM3GT6oYSHhLAHaO6ctuILgx6cqHbNXCRrQJ5547zanXfhE5h9I1qxbsr9nPjeZ0brMT+nHXptG0RwMjuEQ1yf08a3i2Czm1D+O8P+2qV0H2wKpWc/GLuGNWlAaNr+pTQiYiIyBnLmxt0V+77jlFdWLbrCAs2ZRIfGcpH1w/k3M5tGiWWxlLd+rvGKGriLT4+ptoqlQePFdbpnpMHxfCHOZvYkJZD/07h9QnPrdyCYhZuzmTyoJh67+/WGHx8DDcMieXPn29l24Fc4juEnvaaklLLP7/fw8DY8Gb378zTzvxXgIiIiJyVytaSpWXnY/lxP7bZa9O80vcjczfz9ZZMHhjXi3m/HtEsP2ROHBjNU5MSiA4PxuAsxFKbtXtNVXUjkHUdmbx8YDRB/j68u2J/fcKq1pebMil0lDKhFqNd3nbVuTEE+PrwzvJ9NWr/1eYDpGTlccfIrg0cWdOnhE5ERETOOCWllsdPsVlzQ3O3OThA25aB3D2620nbBDQ3EwdGs2TaheyZPp4l0y5s9skcOEcmg/1PXh9Xn5HJVkH+jE+IYm5yGicKHZ4I8SSzk9OIaRPMObGeH/1rKG1aBPDThA58vKZmP5PXFu8hpk0wY/vWb0Pys0HzfTcSERERj5i9No3h07+hy7TPGD79mwYdIUs5ksdzX25j5F++cbumCRp+s+ZT9VHXKXhyZmuIkcnJg2M4UVTCZ+szPBcocCi3kCU7D3P5gOgGW5/XUH52XmdyCx18ui79lO3WpBxl9b6j3Dq8C74e2gi+OdMaOhEREalW5RL6ZdMeAY+N3BQUO/cIe3/VfpbuOoIxMLJHO/KLSziaV1ylfWMU6DibioOIk6cL0SR1bk23di14b2UK1wyK8dh9561Pp9TCxIFRHrtnYzm3c2viI0N5e3kKkwdXv0n464t30yrIj2uSPPdza840QiciIiLV+sv8rW6nPT75+Rbyi6pOSawpay3r9mfz+082MOjJhfzfrGRSj+bzu5/0ZMkDF/KfWwfzp8v6VpkG5+tjGqVAx9i+kVWONffiIOJZxhgmD4plTUo22zNzPXbfOcnp9OnYiu7tT19Y5ExjjOGG82LZkJbDuv3Zbtvsz8pj/sYD3HBeZ1oEauypJvRTEhERaQTerNZYWwXFJXy95SBzktPIyClw2+ZQbiF9/jSf2DYh9GgfSs/IlvSMDKVnZChd27UgyJWIVX7ed4/uSkFxKR+sSmVbZi5B/j78tF9Hrk6KYUiXNvhUmF5V9vMpu75FoC/HC0uIbt2wo2Q5ecV8uj6DqLAgMJCRXXDG/z+TM9Okc6J5esFW3l2Rwp8u61vv++09fILk/dk89NNeHojOO64YGM30L7by9vJ9DIipugbwn9/vwdfHcPOwuMYProlSQiciItLAGmPaYn05SkpZsusIc5LT+HJTJscLHbQPDaRFoC8nCquOxLUJ8eemYV3YnpnL9sxcFm07iKPUuSW1j4G4ti1oGejL5ozc8uNp2fk8PHsTAIkx4fz5igQuHdCRVkHV7+FWcRpcXpGDnzz3HQ9+vIHPfj2CQL/Tb/JcF098tpmsE0XM+dVw+kWHNUgfcnZo2zKQMX078MnaNB4Y16v8i466mpOcjjFw2YCmN92yTGiQP5cnRvHJ2jR+P74PYcE//vvPySvm/VX7uWxAFJGtgrwYZdOihE5ERM4a3holc1cxsaxaY2P0X93zttayJiWbuclpfLYhg8PHiwgN8mN8QkcuT4xiSNe2fLou3e2+ZH+8rO9JsRc5Stl75ATbDuSyIzOX7ZnHWbglszyZq6h9aCCzfzW81s8jJMCPJ67oxy3/Xsk//rebX1/Uo24/kFP43/ZDfLA6lV+O7qZkTjxi8qAYPlufwYJNB2q1qXZl1lrmrEtjSJc2dAxr2ms5rx/cmXdX7OfjNancMvzHTcPfWZFCXlEJt4/QVgW1oYRORETOCt4cJauuYmJadj6vL97doN9Gu3veD3y0ns/Wp7M1M5f9WfkE+vlwce9IJiRGMTq+3UkjX5WnPVaXCAf4+ZRPuSzTZdpnbmM6lFv3SpEXxLfnsgFRvPzNTsb370i3di3rfK/Kjhc6eOjjDXRr16JBkkU5Ow3vFkFMm2Bmrdxfr4RuY9oxdh860Sz2ZUvoFMaATmG8vTyFm4fFYYyhyFHKG0v3MKJ7BH2iWnk7xCZFCZ2IiJwVvDlKFtkqiAPHqq5F8/c1PPHZFp78fAtDurRhwoBoLunXgdYtAjzWt7vnXego5astBxnZI4LfXNSTsX0jCa3htMfaaKhKkX+8tA//23aQBz/ewHt3nHfSurv6+MsXW0nPyefDu4bVe2qcSBkfH8O1STE88+V29h05Qee2Lep0nznJafj7Gn7ar6OHI/SOG87rzP0frmfFnqzy2QCZxwr5y5X9vR1ak6MqlyIiclaobpSsofc0Ky4pJSSganIQ7O/LjKsG8M3vzuf/LurJwdxCHnJVfLz1jZXMXnvy5rs12QvOWkvmsQK+3XqQV77dyZR31rhNqAAM8NZtQ7jq3E6nTObqw9ObNZdpFxrI78f3ZsWeLD5Yvb9e9yqzfPcR3vphH7cM68K5nVt75J4iZa46NwYfA++trNvrtaTUMnddOqPj2xMW0jD/XhvbZf2jCA3y47/LU7DW8tri3fSMbMn5Pdt5O7QmRyN0IiJyVmgd4k+WF/Y0e3r+VnYfPsEN58WwaOtht9MWf3NxD359UXc2pR/j03XpfLounW+2HiTI34eLekcSGRrIOytSKCguBZzTJqd9vJ607Hw6hgWxJeMYmzOOsSUjl6wKm3HHtAkmyM+HAkdpoz9vqPl0zbq4JimGj9ek8eRnW7iwVyTtQgPrfK/8ohIe+Gg9sW1CuG9sz3rHJlJZh7AgLuzVng9Xp/Lbn/TE37d2YyrLdx/hYG4hE+sxZfNMExzgS2JMePl7HjjXGza1zdLPBEroRESkUXmjMMm89ekczSvGGLCVanSMc7PfmKd8tj6D1xbv4aahnXn08n6nbGuMoV90GP2iw3hgXC9WpxxlbnI6n23IOClJK1NQXMqMBdsA5/q1Xh1C+UnvSHp3DKVPVBi9OobSKsi/yho6aNz91Dy9WXMZYwx/npTAJX9dzGPzNvPSdQPrfK/nF25n75E83rl9CCEB+mgkDePaQbEs3LKKb7YeZGzfDrW6dnZyGi0D/biod/sGiq7xzV6bxoo9WScfS07jvK5tz5jqv02F3rVERKTReKMwyZzkNO6dlUxSXGuuPKcTL32zk3TXyJavr+HtFSmMHxDFObGenWa3IzOXqR+u45zYcH4/vk+trvXxMQyKa8OguDb88bI+9Pj9F9W2/ereUXSJaIFfNd/4N+Qombd1a9eSKRd257mvtjNpYDQX9Kr9h93k/dm8vng31w2OZVj3iAaIUsTpgvh2RLYKZNbK/bVK6AqKS/hi4wHG9u3QrNZ2zliwjcJKswfKvqhqDu9PjUkJnYiINJrGLkzyydpUfvf+OgbFteFfNw+iRaAfkwfHlp8/fLyQK/+2lNvfXMXHdw8jLqJuxQoqyy0o5hf/XU1IgC+v3nAuAX51X7Lu7+tDdDXFRaLDg+lRoapkdRpqlOxMcNf53fh0XToPz97Il/eOokVgzT/aFDpKuP/DdbQPDeLBJrxRszQNfr4+XH1uDK8u2klGTn6Ntx5YtO0guQUOLk9sunvPueOtdc3NkYqiiIhIo2nMX+Afrk7lt++v47yubfn3LYPcftCPaBnIG7cMBuCmf6/gyPG6l9MvY61l6gfr2Xckj5evP4cOYfXfjqChios0BwF+Pjw1KYG07Hye/2p7ra595dtdbM88zp8n9Tvl5uYinnJNUgylFt5fmVrja+YkpxPRMpBh3do2YGSNr7p1vI2xvre5UUInIiINrqC4hJe+3kHVLaadQoP8KC6pWrijrmatTGHqh+sY0T2Cf9406JTrorpEtOD1m5I4kFPArW+uIr+opNq2NTHzu93M33SABy/pxXldPfMBbOLAaJ6alEB0eDAG58jcU5MSmu2oW20lxbXhhiGx/GvJHjak5tTomi0Zx3j1251cMTCaC3s13DpKkYpi24YwonsE76/aT0lpde+IPzpWUMzXWw9y2YCO1U6rbqr0RZXnNK9XhoiI1EhNSuB7yqJtBxn31+949qvtDOjUisBK0w99DBwrcHDFq0vYnH6s3v29szyFBz7awKge7Xjt50kEu9kyoLJzYlvz4nUDWZ+azT3vrq3RBy13lu48zF/mb2V8QkduG9GlTveozsSB0SyZdiF7po9nybQLlcxVcv+4XrRtGciDn6zHcZovBxwlpdz/4XrCQ/z546W1W98oUl/XDoohLTuf73cePm3b+RsOUOQordeG5GcqfVHlOUroRETOMmWFSdKy87H8WJjE00ldWnY+d721mpv/vRIfY/jPrYOZM2Ukf7my/0m/wJ+7JpG/3XAOB3IKmPDy9zz31XaK3JTZr4m3lu3loU82cGGv9vzjxnNrVUBgbN8OPDqhLwu3ZPLI3E3YyuUwTyMjJ5973l1L13Yt+ctV/VV6u5GFBfvz6IS+bEw7xhtL956y7WuL97AhLYfHLu/n0U3cRWpiTN9IWof4M2tlymnbzlmXRlzbEAZ0CmuEyBqfvqjyDBVFERE5yzR0YZJCRwmvL97DS9/sAJzTam4f2YVAP2dyVV2BjvO6tuXxeZt58esdLNh4gKev6s+AmPAa9/vGkj088ulmLu4dySs3DCzvrzZ+PjSOtKP5/OO73US3Duau87vV6LpCRwl3/3cNBcUl/P1n59KyFoU5xHMu6deBi3u359kvtzO2bwdi2oRUabPr0HGeX7idcX078NOEjl6IUs52gX6+XHlOJ95ctpfDxwuJaOl+D8XMYwUs3XWEey7soS+I5JQ0QicichZJy853Wy2x7Nx7K1LIyKl7gZLFOw5xyV8XM2PBNkb3bM/XvxvNry7oXqPkqnWLAJ67NpF/3ZxETn4xV7y6hKe+2EJB8enXtL2+eDePfLqZsX0jefWGc+qUzJV5YFwvLhsQxfQvtjInuWajlo/P20zy/myeuXoA3du3rHPfUj/GGB67vB8+Bh6evbHKKGtpqeWBD9cT7O/LYxP7eilKEZg8OIbiEstHq6svjvLpunSspdlVtxTP01eIIiJngbK9tr7YeKDaNj4Gprn2hOvVIZTz49sxumd7kuJa43+axfjp2fk88dlmPt9wgLi2IbxxyyBGx9dtA9wLe0Xy5W/b8NTnW/jH/3bz1aZMnr6qP0lxbdy2n/ndLv78+VYu6deBF68beNpYT8fHx/DM1f05eKyA+z5YR7vQQIZ1q35/sg9Xp/LfH1L4xaiuXKIRH6+LCg/mvrHxPPrpZj5dn8GEAT9+GP7Psr2s2neUZ68eQPvQ+lcfFamr7u1DSercmlkr93PnqK5uR+DmJKeTEB1Gt3b6kkhOzdR2jUBDS0pKsqtWrfJ2GCIiTV5JqeWrzZn88/vdrNx7lNBAP64bEkvHsCCenn/ytMtgf1/+fEU/+kSFsWjbQRZtO8TKvVk4Si0tA/0Y3r0to+PbMzq+HR3Dgpm9Nq18o+rQID/yi0rw9TVMuaA7t4/s6rHNb7/fcZhpH68nLTufm4bGcf+4eL7clHlS38cKHFzavyPPX5tY72Suopy8Yq76+1IOHCvgw7uGEd+h6n5vm9JzmPTqUs6Jbc1btw1udlXomqqSUsukV5eQlp3Pwt+eT3hIAPuz8hjz/HcM7tKGN24ZpCls4nUfrNrP1A/XM+vO8xhSqSLurkPHuejZ//Hw+N7cPrKrlyIUbzLGrLbWJtWorRI6EZHm5UShgw9W7edfS/aSkpVHp9bB3Dq8C9cMiilf21UxIYsKD2bq2Pgq69qOFzpYsvNweYKXkVMAQMdWgRw6XoSjQiVIHwO/H9+b20Z4/oPHiUIHT8/fypvL9tGmhT/HC0ooqlDF0NcYZlyVwKRzYzzed1p2Ple8sgQ/H8PHvxx+0p5yOXnFXPryYoodlnm/HlHtOhjxjs3pxxj/4mKCA3zJLypxbu5uLd9MvYBo7XMlZ4C8IgdDnvyan/SJ5LlrE08699xX23npmx388OBFRLbSaPLZSAmdiMhZoHJSdseoLmTkFPDu8hSOFTg4JzacO0Z25Sd9Ius9cmStZXvmcRZtO8iz1VShjA4PZsm0C+vVz6ms2JPFda/94HZLgYbse1N6Dtf8fRkxbUL44K6hhAb5U1pque3NlXy/8zCzfjGUc2JbN0jfUnez16Zx3wfrTvriwd/XMOOqAaqkJ2eMh2dv4INVqax46GLSW5/uAAAgAElEQVTCQpyb21trueCZRUS3Dubt28/zcoTiLbVJ6LSGTkSkCSrbeqBs2mRadj6PzN0MwPiEjtw6ogvndvZckmGMIb5DKPEdQpn+xVa3bdKrKbbiKYO7tKG0mv3hGrLvvlFh/O1n53LrGyu58tWlHC90kO4arbzq3E5K5s5QMxZsOymZAygusR6r5iriCZMHxfLfH1KYnZzGTcPiAFiXmsPeI3n8cnR37wYnTYYm+4uINEHuth4AiGwVyCs3nOPRZK6yqGqmq1V3vDn0PapnO65O6sT2g8fLkzmAz9anN+im7FJ31SX5Df3Fg0ht9IsOo190K95dkVJelXVOchoBfj6MS+jg5eikqahRQmeMGWeM2WaM2WmMmVZNm2uMMZuNMZuMMe9UOH6TMWaH689NngpcRORsVt2H0oPHChu876lj4wmuVPQk2N+XqWPjm3Xf320/XOVYfnEpMxZsa/C+pfa8+cWDSG1cOyiWrQdyWZ+ag6OklE/XZXBhfHtaBfl7OzRpIk6b0BljfIFXgEuAPsB1xpg+ldr0AB4Ehltr+wL/5zreBvgTMAQYDPzJGKO5KSIi9bBwc2a15xrjw+rEgdE8NSmB6PBgDM71a09NSmiUaWze7FsjPk2LN5N/kdq4PDGKYH9f3lu5n6W7jnD4eCETB2rvOam5mqyhGwzstNbuBjDGvAdcDmyu0OYO4BVr7VEAa+1B1/GxwFfW2izXtV8B44B3PRO+iMjZ5a1le/nT3E10ah3ModxCCioUJ2nMD6sTB0Z7bR2St/qOCg92uym7RnzOTGWvkdNVcxXxtlZB/iREt+K9FSm8uyIFA+QWOLwdljQhNUnoooH9FR6n4hxxq6gngDFmCeALPGKtnV/NtVXeSY0xdwJ3AsTGxtY0dhGRs0ZpqeUv87fyj+92c1Gv9rx0/cCT9mPTh9WGN3Vs/EmFaEAjPmc6b37xIFJTs9emkZyaQ1kJHwv8cc4m/H199PqVGqlJQudu583KZcb8gB7AaKATsNgY06+G12KtnQnMBOe2BTWISUTkrFFQXMLvPljHZ+sz+Nl5sTxyWV/8XL/o9cu+8WjER0QawowF26psBZNfXKKKrFJjNUnoUoGKu7V2AtLdtPnBWlsM7DHGbMOZ4KXiTPIqXruorsGKiJxtjp4o4s63VrFy71EevKQXd47qijHuviuTxqAkWkQ8Tetzpb5qUuVyJdDDGNPFGBMATAbmVmozG7gAwBgTgXMK5m5gATDGGNPaVQxljOuYiIicRsqRPK7821LW7c/hpesG8ovzuymZExFpZlSRVerrtAmdtdYBTMGZiG0B3rfWbjLGPGaMmeBqtgA4YozZDHwLTLXWHnEVQ3kcZ1K4EnisrECKiIhUL3l/Nle8uoQjJ4r47+1DuGyAKp6JiDRHqsgq9WXKNjE8UyQlJdlVq1Z5OwwREa/5ctMBfv3eWtqFBvLGLYPp1q6lt0MSEZEGNHttmtbnykmMMauttUk1aVuTNXQiItJI3ly6l0c+3UT/6DBev2kQ7UIDvR2SiIg0MK3PlfpQQicicgYoLbU89cUWXlu8h4t7R/LidYmEBOgtWkRERE5NnxZERLyk4hSbQH8fCopLuWloZ/54WV98fVT8RERERE5PCZ2IiBfMXpt20ibVBcWl+PsaEmPClcyJiIhIjdVk2wIREfGgQkcJj83bXJ7MlSkusTzz5XYvRSUiIiJNkUboREQaQXFJKd/vPMy8dRl8uekAuYUOt+20kayIiIjUhhI6EZEGUlJqWb77CJ+uz2D+xgyO5hUTGujH2H4d+HbrQY6cKKpyjTaSFRERkdpQQiciZ7X67P3j7toJA6JYk3KUeesz+GxDBodyCwkJ8OXi3pFcNiCKUT0jCPTzrbKGDrSRrIiIiNSeNhYXkSatvgmZu6TqqUkJp72Hu2v9fAwtA33JzncQ4OfDhfHtuWxAFBf2ak9wgK9HYxcREZHmqzYbiyuhE5Emy11SFeTvw0OX9GJ4j3YcL3BwotBBbqGD4wUOjhc6/+QWODheWMxHq9OqFCYB8Pc19IkKcz6wlrJ3SWuh7NG2A7kUl1R9/wz082H6lQlc3DuS0CB/jz9nERERaf5qk9BpyqWINFlPL9haJSErKC7lj3M3n/K6AD8fQgP93CZz4Kw2GR7sTMaMgbJNBIxx/s0AG9OOub22yFHKFQM71fxJiIiIiNSDEjoRaXL2Z+Xx4epU0rMLqm3zwuREWgb6Of8E+REa6E/LID9aBPoS6Oec/jh8+jekuakqGR0ezJu3Dj5lDNVdq6ImIiIi0piU0IlIk5BX5ODzDQf4cPV+ftidhTHO6Y2FjtIqbaPDg7k88fRr0aaOja9zYZL6XCsiIiLiKUroRKTeGqq4h7WWlXuP8sGq/Xy+IYMTRSXEtQ3hvjE9ueKcTqzck1WvpKosxrrEXp9rRURERDxFRVFEpF7qUymy4j0qJka3j+zC8QIHH65JZd+RPFoE+DK+f0euToohqXPr8rVs7q5VUiUiIiJNnapcikit1TQxyi0oJi07n7Sj+aRl5/P0/G0cL3RUaRfo58NFvdvTMtCPFmVr2Sr8vey/q/Zm8fK3O91OnTyvaxuuPjeGSxI6EBKgCQUiIiJydlCVSxGplcqjbGnZ+Uz9cB3fbM2kdUgAadn5pLoSuNyCqsmbO4WOUrYdyOVEYQknCh0cL3JQm++PIlsF8t6dQ+vydERERETOGkroRIQZC7ZVKeFfXGKZuy6D0EA/olsHEx0ezOAubYgODy5/HN06mImvLHFbbTI6PJivfze6/HFpqSW/2JXcVfhz/WvL3cZ08FihR5+jiIiISHOkhE5ESHdTfh+c+61teHTsKa+9f2yvGhUm8fExtHBNtWxf4Xh0eLDK/4uIiIjUkY+3AxAR71qxJ6vaczVJqiYOjOapSQlEhwdjcCZotSmIMnVsPMH+vicdU/l/ERERkZrRCJ3IWezrLZn88u01RIQGcCzfcVJhktqW/69rZUmV/xcRERGpOyV0Imepj1ancv9H6+kb1Yp/3zyIxTsOey2pqk9CKCIiInI2U0InchZ6ffFunvhsC8O6tWXmz5NoGeinpEpERESkCVJCJ3IWsdbyzJfbeOXbXVzSrwN/nZxIoJ/v6S8UERERkTOSEjqRs0RJqeXh2Rt5d0UK1w2O5YmJ/fD1Md4OS0RERETqQQmdyFmg0FHCvbOS+XzDAX51QTfuGxOPMUrmRERERJo6JXQizdzxQge/eGsVS3Ye4eHxvbl9ZFdvhyQiIiIiHqKETsSDZq9NO6PK72edKOLmf69gU/oxnr16AFee28lrsYiIiIiI5ymhE/GQ2WvTePDjDeQXlwCQlp3Pgx9vAPBKUpeWnc+N/1xO2tF8/vGzc7m4T2SjxyAiIiIiDUsJnYiHzFiwrTyZK5NfXMKMBdsaJaGrODrYLjSQQkcJpaXw1m1DGNylTYP3LyIiIiKNTwmdiIekZ+fX6rgnVR4dPJhbCMB9Y3sqmRMRERFpxny8HYBIcxEVHuz2uI+P4dN16VhrG6xvd6ODAO8u399gfYqIiIiI9ymhE/GQu0d3q3IswNeH9i0DuOfdtUx8ZQnLdx/xeL+7Dx0nzYujgyIiIiLiPZpyKeIhm9KPYYB2oYEcyi0sr3J52YAoPl6TyrNfbufamT/wkz6RTLukF93ataxzX9ZavttxmDeW7OHbbYeqbVfdqKGIiIiINA9K6EQ8YH1qNu+tTOHWEV34w6V9qpy/OimGS/tH8a8le/jbol2Mef47rhscw28u6km70MAa93Oi0MHHa1J5Y+ledh06QUTLQP7v4h60aeHPU5+fPO0y2N+XqWPjPfL8REREROTMpIROpJ5KSy1/nLOJti0C+c3FPaptFxzgy68u6M61g2J48esdvL08hU/WpHHX+d24fWRXggN8q712f1Yeby7dy6xV+8ktcNC/UxjPXzuA8QlRBPg5Z063Cgo4o/bAExEREZGGZxqyUENdJCUl2VWrVnk7DJEae3/lfu7/aD3PXTOASefUfOPuXYeO8/T8rSzYlElkq0B+NyYeP2N49qvtrqQsiMsTo9lx8DgLt2TiawyXJHTk5mFxnBMbjjGmAZ+ViIiIiHiLMWa1tTapRm1rktAZY8YBLwC+wOvW2umVzt8MzADSXIdetta+7jpXAmxwHU+x1k44VV9K6KQpyckr5sJnF9ElogUf3DW0TknWyr1ZPPnZFpL3Z2OAyv8iWwT4csvwLvzsvM50CAvySNwiIiIicuaqTUJ32imXxhhf4BXgJ0AqsNIYM9dau7lS01nW2ilubpFvrU2sSTAiTc3zC7dzNK+I/1w+uM4jZoPi2vDJL4dxzuNfcTSvuMr5sGB/7tNaOBERERFxoybbFgwGdlprd1tri4D3gMsbNiyRM9+WjGP8Z9lebhjSmb5RYfW6lzGGbDfJHEBGTkG97i0iIiIizVdNErpooOLuxKmuY5VdaYxZb4z50BgTU+F4kDFmlTHmB2PMRHcdGGPudLVZdehQ9SXYRc4U1lr+NGcT4SEB/G5MT4/cs7otBrT1gIiIiIhUpyYJnbt5ZJWX+XwKxFlr+wMLgTcrnIt1zf+8HvirMabK7svW2pnW2iRrbVK7du1qGLqI98xJTmfF3izuHxtPeEiAR+45dWw8wf4nV7rU1gMiIiIicio1SehSgYojbp2A9IoNrLVHrLWFroevAedWOJfu+u9uYBEwsB7xinhdbkExT36+hQGdwrgmKeb0F9TQxIHRPDUpgejwYAwQHR7MU5MStPWAiIiIiFSrJvvQrQR6GGO64KxiORnnaFs5Y0xHa22G6+EEYIvreGsgz1pbaIyJAIYDT3sqeBFveOmbnRw+XshrP0/Cx8ezWwdMHBitBE5EREREauy0CZ211mGMmQIswLltwb+stZuMMY8Bq6y1c4FfG2MmAA4gC7jZdXlv4B/GmFKco4HT3VTHFGkydh7M5V/f7+HapBgSY8K9HY6IiIiInOW0sbhIDVlr+dk/l7MhNYdv7xtN25aB3g5JRERERJqh2uxDV5M1dCICfLHxAEt2HuG+sfFK5kRERETkjKCETqQG8oocPDFvM707tuL6wbHeDkdEREREBFBCJ1Ijr367i/ScAh67vC9+vvpnIyIiIiJnBn0yFTmNvYdPMPO73UwaGM2guDbeDkdEREREpFxNti0QaVJmr01jxoJtpGfnExUezNSx8XXeCsBay6OfbiLAz4dpl/TycKQiIiIiIvWjhE6aldlr03jw4w3kF5cAkJadz4MfbwCoU1L39ZaDfLvtEA+P7037VkEejVVEREREpL405VKalRkLtpUnc2Xyi0uYsWBbre9VUFzCo/M20aN9S24aFuehCEVEREREPEcjdNKspGfnuz2elp3Pr99dS7/oVvSNCqNvVCvCQwLcti2bspnmutcvR3fDX4VQREREROQMpIROmoWC4hJeX7y72vNBfj6s2pvF3HXp5ceiw4PLE7yy/y7deZiHPtl40ijfv5fspWdkaJ3X4YmIiIiINBQldNKkWWv5dH0Gf/liK2nZ+SREt2J75nEKHaXlbYL9fXlqUgITB0aTdaKITek5bEo/xsa0HDanH2PBpszytj4GSu3JfZRN2VRCJyIiIiJnGiV00mStTTnK4/M2syYlmz4dW/HM1QMY2q3tKatctmkRwMge7RjZo135fXILitmSkcum9Bwe/XSz276qm8opIiIiIuJNSuikyUnPzufp+VuZnZxORMtAnr6yP1ee2wlfHwM4q1nWZjQtNMifwV3aMLhLG15fvKd87VxFUeHBHotfRERERMRTlNBJk5FX5ODv/9vNzO92UWrhVxd04+7R3WkZ6LmX8dSx8SdtewDOKZtTx8Z7rA8REREREU9RQidnpIrTJjuGBzG6Zzu+3nqQzGOFXNq/Iw+M60VMmxCP91s2suepjclFRERERBqSsdaevlUjSkpKsqtWrfJ2GOJFlTcHLxPTOpjnr00kKa6NlyITEREREWl4xpjV1tqkmrTV5lpyRilylPLk51v+n707j4+quv8//vpkshICYYcskMgm+yLigihugFoRrXWtlf5qrbXW1ra02tpaba1U/LbWr/bb2lZrq1apIlWrIiruiICEnQCyZIMQloSE7Mn5/TETTEIgkzBLlvfz8cjDmXPv3PuZcM3M555zPueoZA6gxjklcyIiIiIi9WjIpQTF8SpN1nHOsXN/KWuyC8nILmRNTiEb8g5RWW/Jgfp2F5aHInQRERERkXZDCZ0EXOMhk7mFZdy1cB2HyqtIToxjTXYhq7MLWZtTRFFZFeAtPDImpTtzzkzj3yuzOVhaddRxVWlSRERERKQhJXQScPMXZx41ZLKsqoZf/GcD4F28e3j/blw8pj/jUhIZl5rI0L5difR4RwCPHNBNlSZFRERERPyghE4C7niLcP/7ljMYldSNLtHHvvRUaVJERERExD9K6CSgamsdCbGRHCqvPmpbcmIcp/pZ1KSli4OLiIiIiHRGqnIpAVNcXsUtT6/iUHk1HrMG2zRkUkREREQk8NRDJwGxbW8xN/9zFbv2l/LzL42kZ5coHnpzi4ZMioiIiIgEkRI6OWFvrN/DDxdkEBvl4elvnMYZg3sBcPnElDBHJiIiIiLSsSmhk1arqXX8bkkmjy39nHGpifzpqxMZ0F1LC4iIiIiIhIoSOmmVwtJKbn8ug/e3FHDNqance9koYiI94Q5LRERERKRTUUInLbYx7xDfenol+UUVPHDFGK6dPDDcIYmIiIiIdEpK6KRFFq3O5c6Fa0mMi+a5b53OxIE9wh2SiIiIiEinpYRO/FJVU8tvXtvEkx/tZHJ6Tx67biJ9EmLCHZaIiIiISKemhE6OadHqXOYvziSvsIwoTwSVNbV8fUoaP714BFEeLWEoIiIiIhJuSuikSYtW53LXwnWUVdUAUFlTS5THGJeSqGRORERERKSN0DdzadL8xZlHkrk6VTWO+YszwxSRiIiIiIg0poROmpRXWNaidhERERERCT0ldNKk+Jim15RLStTC4SIiIiIibYUSOjnK8u37KamowRNhDdrjojzMnTE8TFGJiIiIiEhjSuikgZKKan747zUM6tWFBy4fTXJiHAYkJ8bxwBVjmD0hOdwhioiIiIiIj6pcSgO/fnUjeYVl/PuWMzhlUE+uOnVguEMSEREREZFj8KuHzsxmmlmmmW0zszub2D7HzArMLMP3c1O9bTea2Vbfz42BDF4C6+1N+Ty3IptvnTOYUwb1DHc4IiIiIiLSjGZ76MzMAzwGXAjkACvM7GXn3MZGuz7vnLut0Wt7AvcAkwAHrPK99mBAopeAOXC4kp+8uI6T+yfw/QuGhjscERERERHxgz89dJOBbc657c65SuA54DI/jz8DWOKcO+BL4pYAM1sXqgSLc46fvbSOorJKfn/1eGIim65wKSIiIiIibYs/CV0ykF3veY6vrbEvm9laM3vBzFJb8lozu9nMVprZyoKCAj9Dl0D5T0Yer6/fww8uHM6IAd3CHY6IiIiIiPjJn4TOmmhzjZ6/AqQ558YCbwFPteC1OOced85Ncs5N6tOnjx8hSaDsLirjF/9ZzymDenDz2SeFOxwREREREWkBfxK6HCC13vMUIK/+Ds65/c65Ct/TvwCn+PtaCR/nHD9+YS3VtY7fXTXuqHXnRERERESkbfMnoVsBDDWzdDOLBq4BXq6/g5kNqPd0FrDJ93gxMN3MephZD2C6r03agKc/2cUHW/fxs0tGMKhXfLjDERERERGRFmq2yqVzrtrMbsObiHmAJ5xzG8zsPmClc+5l4HYzmwVUAweAOb7XHjCzX+FNCgHuc84dCML7kBbase8w97+2iXOG9eG6yVprTkRERESkPTLnjprSFlaTJk1yK1euDHcYHVp1TS1f+fMythcc5s07zqZft9hwhyQiIiIiIj5mtso5N8mffZvtoZOO58/vb2d1ViH/e+0EJXMiIiIiIu2YEro2btHqXOYvziSvsIykxDjmzhjO7AlNrRrhn/W5Rfx+yRa+NHYAl45LCmCkIiIiIiISakro2rBFq3O5a+E6yqpqAMgtLOOuhesAWpXUVVTX8MMFa+gZH82vLhsd0FhFRERERCT0/KlyKWEyf3HmkWSuTllVDfMXZ7bqeL9bsoXM/GJ+e+VYesRHByJEEREREREJIyV0bVheYVmT7bmFZSxYmc3uoqa3N2XFzgM8/v52rjttIOcO7xuoEEVEREREJIw05LINS0qMJbew/Kj2CIMfv7AWgKF9uzJ1aB+mDuvNaek96RL9xT9p/fl3ERFGjy5R/OziESGLX0REREREgksJXRt22YRk/rj08wZtcVEefnP5aE4e0I0PthbwwdZ9PL18F098tINoTwST0nowdWgfqmtr+ePSbZRV1QJQU+soqahhycb8EyqqIiIiIiIibYcSujZs1/5S4qIi6NElmt1F5UdVuRwxoBs3nz2Y8qoaVuw8wAdb9/H+lgJ++8bmJo9XWV3L/MWZSuhERERERDoIJXRtVEFxBYvX72HOmWnc/aWRx903NsrjHXY5tA8/vXgEew+VM/k3bze577Hm5YmIiIiISPujoiht1IKV2VTXOq49bWCLX9u3WyzJiXFNbks6RruIiIiIiLQ/SujaoJpax78+zeLMwb0Y3Kdrq44xd8Zw4qI8DdriojzMnTE8ECGKiIiIiEgboISuDXp/awE5B8u4/rRBrT7G7AnJPHDFGJIT4zAgOTGOB64Yo/lzIiIiIiIdiObQtUHPfLKL3l1juHBkvxM6zuwJyUrgREREREQ6MPXQtTG5hWW8s3kvV5+aQnSk/nlEREREROTYlDG0Mc9/moUDrjm15cVQRERERESkc1FC14ZU1dTy3Ipspg3rQ2rPLuEOR0RERERE2jgldG3I25vy2VtccULFUEREREREpPNQQteGPLM8i6TusZx7ct9whyIiIiIiIu2AEro2Yue+w3ywdR/XTB6IJ8LCHY6IiIiIiLQDSujaiH99moUnwrjm1NRwhyIiIiIiIu2EEro2oKK6hgUrs5k+sh99u8WGOxwREREREWknlNC1AW+s38PB0ioVQxERERERkRZRQtcGPPNJFmm9unDm4F7hDkVERERERNoRJXRhtiW/mE93HuC60wYSoWIoIiIiIiLSAkrowuzZ5VlEeyK48hQVQxERERERkZZRQhdGpZXVvPhZDheP6U/P+OhwhyMiIiIiIu2MErowemVNHsXl1Vx/uoqhiIiIiIhIyymhC6NnlmcxrF9XJg3qEe5QRERERESkHVJCFyZrcwpZm1PE9acNwkzFUEREREREpOWU0IXJs8uziIvycPnE5HCHIiIiIiIi7ZQSujA4VF7FfzLymDUuiW6xUeEOR0RERERE2ikldGGwaHUuZVU1XH/6wHCHIiIiIiIi7ZgSuhBzzvHMJ1mMSe7O2JTEcIcjIiIiIiLtmBK6EFu16yCZ+cVcf5p650RERERE5MQooQuxZ5ZnkRATyaXjksIdioiIiIiItHNK6ELowOFK/rtuN5dPTCY+JjLc4YiIiIiISDunhC6EXlyVQ2V1LdefNijcoYiIiIiISAfgV0JnZjPNLNPMtpnZncfZ70ozc2Y2yfc8zczKzCzD9/OnQAXe3tTWOp79NItT03owvH9CuMMREREREZEOoNlxf2bmAR4DLgRygBVm9rJzbmOj/RKA24HljQ7xuXNufIDibbeWbd/Pjn2H+d75Q8MdioiIiIiIdBD+TOSaDGxzzm0HMLPngMuAjY32+xXwIPCjgEbYzi1ancv8xZnkFpYRYVBVUxvukEREREREpIPwZ8hlMpBd73mOr+0IM5sApDrnXm3i9elmttrM3jOzqU2dwMxuNrOVZrayoKDA39jbvEWrc7lr4TpyC8sAqHXwi/9sYNHq3DBHJiIiIiIiHYE/CZ010eaObDSLAH4P/LCJ/XYDA51zE4AfAM+aWbejDubc4865Sc65SX369PEv8nZg/uJMyqpqGrSVVdUwf3FmmCISEREREZGOxJ+ELgdIrfc8Bcir9zwBGA28a2Y7gdOBl81sknOuwjm3H8A5twr4HBgWiMDbgzxfz5y/7SIiIiIiIi3hT0K3AhhqZulmFg1cA7xct9E5V+Sc6+2cS3POpQGfALOccyvNrI+vqApmdhIwFNge8HfRRiUlxh6jPS7EkYiIiIiISEfUbELnnKsGbgMWA5uABc65DWZ2n5nNaublZwNrzWwN8AJwi3PuwIkG3V6cP6LfUW1xUR7mzhgehmhERERERKSjMedc83uF0KRJk9zKlSvDHcYJO3C4kgt/9x5doj3UOMfuwnKSEuOYO2M4syckN38AERERERHplMxslXNukj/7+rNsgbTCfa9s4FB5Fc988zRO7n9UHRgREREREZET5s8cOmmhdzbnsygjj1unDVEyJyIiIiIiQaOELsCKy6v42UvrGdavK985d0i4wxERERERkQ5MCV2APfD6ZvIPlfPgleOIjtSvV0REREREgkcZRwAt+3w/zy7P4v9NSWd8amK4wxERERERkQ5OCV2AlFXWcOfCtQzq1YUfTteyBCIiIiIiEnyqchkgv39rC7v2l/LsN08jLtoT7nBERERERKQTUA9dAKzJLuSvH2zn2skDOXNw73CHIyIiIiIinYQSuhNUWV3Lj19YS9+EWO66+ORwhyMiIiIiIp2IhlyeoD++u43M/GL+duMkusVGhTscERERERHpRNRDdwIy9xTz2NJtXDY+ifNH9At3OCIiIiIi0skooWulmlrHj19YQ0JsFPdcOirc4YiIiIiISCekIZet9MSHO1iTU8Qj106gZ3x0uMMREREREZFOSD10rbBz32EeejOTC0b049KxA8IdjoiIiIiIdFJK6FqottbxkxfXEu2J4NezR2Nm4Q5JREREREQ6KSV0LfSvFVks33GAn10ygp/5cPkAACAASURBVP7dY8MdjoiIiIiIdGJK6Fpgd1EZD7y2mTMH9+LqU1PDHY6IiIiIiHRyKorSjEWrc5m/OJO8wjKiIyOorXXMu2KshlqKiIiIiEjYqYfuOBatzuWuhevILSzDARXVtWDwWdbBcIcmIiIiIiKihO545i/OpKyqpkFbVY1j/uLMMEUkIiIiIiLyBSV0x5FXWNaidhERERERkVBSQnccSYlxLWoXEREREREJJSV0xzF3xnDiojwN2uKiPMydMTxMEYmIiIiIiHxBVS6PY/aEZIAjVS6TEuOYO2P4kXYREREREZFwUkLXjNkTkpXAiYiIiIhIm6QhlyIiIiIiIu2UEjoREREREZF2SgmdiIiIiIhIO6WETkREREREpJ1SQiciIiIiItJOKaETERERERFpp8w5F+4YGjCzAmBXuONoQm9gX7iDkE5D15uEiq41CRVdaxJKut4kVIJ1rQ1yzvXxZ8c2l9C1VWa20jk3KdxxSOeg601CRdeahIquNQklXW8SKm3hWtOQSxERERERkXZKCZ2IiIiIiEg7pYTOf4+HOwDpVHS9SajoWpNQ0bUmoaTrTUIl7Nea5tCJiIiIiIi0U+qhExERERERaaeU0ImIiIiIiLRTSuj8YGYzzSzTzLaZ2Z3hjkc6FjN7wsz2mtn6em09zWyJmW31/bdHOGOUjsHMUs1sqZltMrMNZvY9X7uuNwkoM4s1s0/NbI3vWrvX155uZst919rzZhYd7lilYzAzj5mtNrNXfc91rUnAmdlOM1tnZhlmttLXFvbPUCV0zTAzD/AYcBEwErjWzEaGNyrpYP4OzGzUdifwtnNuKPC277nIiaoGfuicGwGcDnzH9/dM15sEWgVwnnNuHDAemGlmpwO/BX7vu9YOAt8IY4zSsXwP2FTvua41CZZznXPj6609F/bPUCV0zZsMbHPObXfOVQLPAZeFOSbpQJxz7wMHGjVfBjzle/wUMDukQUmH5Jzb7Zz7zPe4GO+Xn2R0vUmAOa8S39Mo348DzgNe8LXrWpOAMLMU4BLgr77nhq41CZ2wf4YqoWteMpBd73mOr00kmPo553aD90s40DfM8UgHY2ZpwARgObreJAh8Q+AygL3AEuBzoNA5V+3bRZ+nEigPAz8Gan3Pe6FrTYLDAW+a2Sozu9nXFvbP0MhQn7AdsibatNaDiLRbZtYVeBH4vnPukPdmtkhgOedqgPFmlgi8BIxoarfQRiUdjZl9CdjrnFtlZtPqmpvYVdeaBMIU51yemfUFlpjZ5nAHBOqh80cOkFrveQqQF6ZYpPPIN7MBAL7/7g1zPNJBmFkU3mTuGefcQl+zrjcJGudcIfAu3nmbiWZWdzNZn6cSCFOAWWa2E++0mPPw9tjpWpOAc87l+f67F++Nqsm0gc9QJXTNWwEM9VVLigauAV4Oc0zS8b0M3Oh7fCPwnzDGIh2Eb17J34BNzrnf1duk600Cysz6+HrmMLM44AK8czaXAlf6dtO1JifMOXeXcy7FOZeG9zvaO86569G1JgFmZvFmllD3GJgOrKcNfIaac+qBbo6ZXYz3bo8HeMI5d3+YQ5IOxMz+BUwDegP5wD3AImABMBDIAr7inGtcOEWkRczsLOADYB1fzDX5Kd55dLreJGDMbCze4gAevDePFzjn7jOzk/D2ovQEVgNfdc5VhC9S6Uh8Qy5/5Jz7kq41CTTfNfWS72kk8Kxz7n4z60WYP0OV0ImIiIiIiLRTGnIpIiIiIiLSTimhExERERERaaeU0ImIiIiIiLRTSuhERERERETaKSV0IiIiIiIi7ZQSOhER6bDMrMbMMur93BnAY6eZ2fpAHU9ERKQ1IsMdgIiISBCVOefGhzsIERGRYFEPnYiIdDpmttPMfmtmn/p+hvjaB5nZ22a21vffgb72fmb2kpmt8f2c6TuUx8z+YmYbzOxNM4sL25sSEZFOSQmdiIh0ZHGNhlxeXW/bIefcZOBR4GFf26PAP5xzY4FngEd87Y8A7znnxgETgQ2+9qHAY865UUAh8OUgvx8REZEGzDkX7hhERESCwsxKnHNdm2jfCZznnNtuZlHAHudcLzPbBwxwzlX52nc753qbWQGQ4pyrqHeMNGCJc26o7/lPgCjn3K+D/85ERES81EMnIiKdlTvG42Pt05SKeo9r0Nx0EREJMSV0IiLSWV1d77/LfI8/Bq7xPb4e+ND3+G3g2wBm5jGzbqEKUkRE5Hh0J1FERDqyODPLqPf8Dedc3dIFMWa2HO/NzWt9bbcDT5jZXKAA+Lqv/XvA42b2Dbw9cd8Gdgc9ehERkWZoDp2IiHQ6vjl0k5xz+8Idi4iIyInQkEsREREREZF2Sj10IiIiIiIi7ZR66EREJCTMLM3MnJlF+p6/bmY3+rNvK871UzP764nEKyIi0h4ooRMREb+Y2WIzu6+J9svMbE9Lky/n3EXOuacCENc0M8tpdOzfOOduOtFji4iItHVK6ERExF9/B24wM2vUfgPwjHOuOvQhdS6t7bEUEZGOSwmdiIj4axHQE5ha12BmPYAvAf/wPb/EzFab2SEzyzazXx7rYGb2rpnd5HvsMbOHzGyfmW0HLmm079fNbJOZFZvZdjP7lq89HngdSDKzEt9Pkpn90syervf6WWa2wcwKfecdUW/bTjP7kZmtNbMiM3vezGKPEfNgM3vHzPb7Yn3GzBLrbU81s4VmVuDb59F6275Z7z1sNLOJvnZnZkPq7fd3M/u17/E0M8sxs5+Y2R7gSTPrYWav+s5x0Pc4pd7re5rZk2aW59u+yNe+3swurbdflO89jD/Wv5GIiLR9SuhERMQvzrkyYAHwtXrNVwGbnXNrfM8P+7Yn4k3Kvm1ms/04/DfxJoYTgEnAlY227/Vt74Z3bbjfm9lE59xh4CIgzznX1feTV/+FZjYM+BfwfaAP8BrwiplFN3ofM4F0YCww5xhxGvAAkASMAFKBX/rO4wFeBXYBaUAy8Jxv21d8+33N9x5mAfv9+L0A9MebSA8Cbsb72f2k7/lAoAx4tN7+/wS6AKOAvsDvfe3/AL5ab7+Lgd3Oufrr9ImISDujhE5ERFriKeArZhbne/41XxsAzrl3nXPrnHO1zrm1eBOpc/w47lXAw865bOfcAbxJ0xHOuf865z53Xu8Bb1Kvp7AZVwP/dc4tcc5VAQ8BccCZ9fZ5xDmX5zv3K0CTvVbOuW2+41Q45wqA39V7f5PxJnpznXOHnXPlzrkPfdtuAh50zq3wvYdtzrldfsZfC9zjO2eZc26/c+5F51ypc64YuL8uBjMbgDfBvcU5d9A5V+X7fQE8DVxsZt18z2/Am/yJiEg7poRORET85ktQCoDLzOwk4FTg2brtZnaamS31DQcsAm4Bevtx6CQgu97zBsmOmV1kZp+Y2QEzK8Tbu+TPceuOfeR4zrla37mS6+2zp97jUqBrUwcys75m9pyZ5ZrZIbxJUl0cqcCuY8wlTAU+9zPexgqcc+X1YuhiZn82s12+GN4HEn09hKnAAefcwcYH8fVcfgR82TdM9CLgmVbGJCIibYQSOhERaal/4O2ZuwF40zmXX2/bs8DLQKpzrjvwJ7zDFJuzG28yUmdg3QMziwFexNuz1s85l4h32GTdcZtbUDUP7/DEuuOZ71y5fsTV2AO+8411znXDO4SxLo5sYOAxCpdkA4OPccxSvEMk6/RvtL3x+/shMBw4zRfD2b52852nZ/15fY085Yv5K8Ay51xrfgciItKGKKETEZGW+gdwAd55b42XHUjA20NUbmaTgev8POYC4HYzS/EVWrmz3rZoIAZvz2C1mV0ETK+3PR/oZWbdj3PsS8zsfDOLwpsQVQAf+xlbfQlACVBoZsnA3HrbPsWbmM4zs3gzizWzKb5tfwV+ZGanmNcQM6tLMjOA63yFYWbS/BDVBLzz5grNrCdwT90G59xuvEVi/ugrnhJlZmfXe+0iYCLwPXyFbEREpH1TQiciIi3inNuJNxmKx9sbV9+twH1mVgz8Am8y5Y+/AIuBNcBnwMJ65ysGbvcd6yDeJPHlets3452rt91XxTKpUbyZeHul/hfYB1wKXOqcq/QztvruxZsQFQH/bRRnje/YQ4AsIAfv/D2cc//GO9ftWaCYLyqGgje5uhQoBK73bTueh/HOAdwHfAK80Wj7DUAVsBlvMZnv14uxDG9vZ3r92EVEpP0y55obqSIiIiIdhZn9AhjmnPtqszuLiEibpwVKRUREOgnfEM1v4O3FExGRDsCvIZdmNtPMMs1sm5nd2cT2Ob6KZhm+n7qFYseb2TLfYq5rzezqQL8BERERaZ6ZfRNv0ZTXnXPvhzseEREJjGaHXPrKIG8BLsQ7H2AFcK1zbmO9feYAk5xztzV67TDAOee2+uY0rAJGOOcKA/ouREREREREOiF/eugmA9ucc9t9E8ifAy7z5+DOuS3Oua2+x3l4J2f3aW2wIiIiIiIi8gV/5tAl03Cx1xzgtCb2+7KvNPIW4A7nXP3X4CtfHU0TC6ua2c3AzQDx8fGnnHzyyf5FLyIiIiIi0sGsWrVqn3POr44wfxK6phaEbTxO8xXgX865CjO7Be+6ROcdOYDZAOCfwI3OudqjDubc48DjAJMmTXIrV670J3YREREREZEOx8x2+buvP0Muc4DUes9TgLz6Ozjn9jvnKnxP/wKcUi+YbnjX6rnbOfeJv4GJiIiIiIjI8fmT0K0AhppZuplFA9fQaCFZXw9cnVnAJl97NPAS8A/foqoiIiIiIiISIM0OuXTOVZvZbcBiwAM84ZzbYGb3ASudcy8Dt5vZLKAaOADM8b38KuBsoJevEibAHOdcRmDfhoiIiIiISOfT7LIFoaY5dCIioVFVVUVOTg7l5eXhDkXaidjYWFJSUoiKigp3KCIiHZqZrXLOTfJnX3+KooiISAeUk5NDQkICaWlpmDVV/0rkC8459u/fT05ODunp6eEOR0REfPyZQyciIh1QeXk5vXr1UjInfjEzevXqpR5dEZE2Rj10IiKdWGdL5g6WVpJfVE5lTS3Rngj6dY+lR5focIfVbnS260VEpD1QQiciIp3CwdJKcg+WUeubO15ZU0vuwTIAJXUiItJuaciliEiYLFqdy5R575B+53+ZMu8dFq3ODXdIxxWMeO+//35GjRrF2LFjGT9+PMuXLwegurqan/70pwwdOpTx48czfvx47r///iOv83g8jB8/nlGjRjFu3Dh+97vfUVtbe9xz5ReVH0nm6tQ6x3lnTwVg586dPPvss0e2/f3vf+e2225r9j1MmzaNgQMHUr/I2OzZs+natWvzv4AgKCws5I9//GNYzi0iIqGnhE5EJAwWrc7lroXryC0swwG5hWXctXBdm03qghHvsmXLePXVV/nss89Yu3Ytb731FqmpqQDcfffd5OXlsW7dOjIyMvjggw+oqqo68tq4uDgyMjLYsGEDS5Ys4bXXXuPee+897vkqa5pO+J5atBg4OqFricTERD766CPAm1Dt3r27VccJBCV0IiKdi5YtEBEJgynz3iG3sOyo9uTEOD6687yQxLBp0yZGjBgBwL2vbGBj3qFj7rs6q7DJhCjaE8GEgYlNvmZkUjfuuXTUMY+5cOFCnnzySV555ZUG7aWlpaSmprJz504SEhKafG3Xrl0pKSk58nz79u2ceuqp7Nu3r8E8r1tvvZWZM2cya9YsLrjoUrp26859//MoC5/7J3lZu7jtx3dzxvAUDh8u4fTTT2fTpk2kp6dz44030qNHD15++WVKS0v5/PPPufzyy3nwwQePimXatGlMnz6dvLw8Hn30UZ544gkKCgr41a9+RUlJCc45fvzjH/P6669jZtx9991cffXVvPvuu9xzzz3069ePjIwMrrjiCsaMGcMf/vAHysrKWLRoEYMHD6agoIBbbrmFrKwsAB5++GGmTJnCL3/5S7Kysti+fTtZWVl8//vf5/bbb+eaa67hP//5D8OHD+fCCy/kkksu4aGHHuLVV18F4LbbbmPSpEnMmTOHtLQ0rrvuOpYuXUpVVRWPP/44d911F9u2bWPu3LnccsstR73f+teNiIgER0uWLVAPnYhIGOQ1kcwdrz3cjtW7dax2f0yfPp3s7GyGDRvGrbfeynvvvQfAtm3bGDhw4DGTuaacdNJJ1NbWsnfv3gbtZ599Nh988AEABwv2sH1rJgCrP/2ECZPPIMKMuvxv3rx5TJ06lYyMDO644w4AMjIyeP7551m3bh3PP/882dnZTZ7//PPP5/3336empobnnnuOq6+++si2hQsXkpGRwZo1a3jrrbeYO3fukR68NWvW8Ic//IF169bxz3/+ky1btvDpp59y00038b//+78AfO973+OOO+5gxYoVvPjii9x0001Hjr1582YWL17Mp59+yr333ktVVRXz5s1j8ODBZGRkMH/+/GZ/d6mpqSxbtoypU6cyZ84cXnjhBT755BN+8Ytf+POrFxGRMFNRFBGRMEhKjGuyh84Blz36IXOmpHHxmAHERHpCEs/xetLg+D2Kz3/rjFads2vXrqxatYoPPviApUuXcvXVVzNv3jwmTpzYYL8nn3ySP/zhD+zfv5+PP/74yLDMxpoacTJ16lQefvhhNm7cyJjRo8jeU8C+/HzWfraCn/36tyT3iDtujOeffz7du3cHYOTIkezatavJ83s8Hs466yyef/55ysrKSEtLO7Ltww8/5Nprr8Xj8dCvXz/OOeccVqxYQbdu3Tj11FMZMGAAAIMHD2b69OkAjBkzhqVLlwLw1ltvsXHjxiPHO3ToEMXFxQBccsklxMTEEBMTQ9++fcnPzz/u+2nKrFmzjpyzpKSEhIQEEhISiI2NpbCwkMTEpntgRUSkbVAPnYhIGMydMZyYyIZ/gmOjIvjyxGSKK6q54/k1TJm3lIff2sLe4vCv+zV3xnDiohoml3FRHubOGH5Cx/V4PEybNo17772XRx99lBdffJEhQ4aQlZV1JGn5+te/TkZGBt27d6empqbJ42zfvh2Px0Pfvn0btCcnJ3Pw4EHeeOMNppx1FuMnnc5Hb71CfHw8A3r3bLa6ZUxMTINYq6urj7nvNddcw3e/+12uuuqqBu3Hm9pQ//gRERFHnkdERBw5V21tLcuWLSMjI4OMjAxyc3OP9F76E19kZGSDgjGN15Grf87G8Rzv/YqISNughE5EJAxmT0jm2snenh7D29M174qx/M9V43nrjnN46v9NZkxyNx5+aytT5r3DHc9nsCa7MKzxPnDFGJIT447E+8AVY5g9IbnVx8zMzGTr1q1HnmdkZDBo0CC6dOnCN77xDW677bYjyUdNTQ2VlZVNHqdujtltt93W5DppZ5xxBg8//DATJp/JhNPO4M+P/oFTT59CZXXD4aIJCQlHksjWmDp1KnfddRfXXnttg/azzz6b559/npqaGgoKCnj//feZPHmy38edPn06jz766JHnGRkZx92/8fsYNGgQGzdupKKigqKiIt5++22/zy0iIm2fhlyKiIRJ97hozGDTfTOJrdf7FRFhnDOsD+cM68OOfYd56uOdvLAqh5dW5zJhYCJzzkzjotEDeG3dbuYvziSvsIykxDjmzhh+QglWc2ZPSA7o8UtKSvjud79LYWEhkZGRDBkyhMcffxzwLmfw85//nNGjR5OQkEBcXBw33ngjSUlJAJSVlTF+/HiqqqqIjIzkhhtu4Ac/+EGT55k6dSpvvvkmvZMGMS6xHwcPHODMM6dQ0SihGzt2LJGRkYwbN445c+bQo0ePFr0fM+NHP/rRUe2XX345y5YtY9y4cZgZDz74IP3792fz5s1+HfeRRx7hO9/5DmPHjqW6upqzzz6bP/3pT8fcv1evXkyZMoXRo0dz0UUXMX/+fK666irGjh3L0KFDmTBhQovel4iItG2qcikiEia3PrOKjXmHeHfuuc3uW1xexYurcnhq2S527DtMQoyHsqpaqmu/+BseF+VpUa9ZZ6pWWFPr2Lj7EL3io0lKjGNvcTl7isoZldQNT4QGq7REZ7puRETCRVUuRUTagcw9xQzr518lx4TYKOZMSeftH5zDk18/lcoa1yCZAyirqmH+4sxghNruFZdX4Zyje1wUwJH5i4176URERNobJXQiImFQUV3Dzv2lfid0dSIijHOH9z1q/ledtrrsQbgdKqsiMiKCLtHeoa3Rvuqhx/o9ioiItBeaQyciEgbbCw5TU+sY1r9lCV2dYy17kJR4/DL8jTnnmiwk0pHUOkdxeTXdu0Qdea8xHvXQtUZbm6bRli1anRvSOa4i0nmph05EJAy25HurEA7r17VVrw/EMgKxsbHs37+/w39JLymvpsY5uvmGW4K3pzPKE6EeuhZwzrF//35iY2PDHUqbt2h1LnctXEduYRkOyC0s466F61i0OjfcoYlIB6QeOhGRMNiSX0xkhHFS79YldHV3+u95eT1FZdX07xbLnRed3KIegJSUFHJycigoKGhVDO3FwdJKyipriDwU26A3cl9xBXuBkoSYY79YGoiNjSUlJSXcYbR58xdnUlbVcM3Eujmu6qUTkUBTQiciEgaZe0pI7x1PdGTrB0rMnpBMSo84rvzTMu67bBTTR/Vv0eujoqJIT09v9fnbg5pax+T732LKkN48cu3IBtt++tI6Xlu3m4xfTA9TdNJRHWsuq+a4ikgwaMiliEgYbMn3v8Ll8YxK6o4nwliTE75Fx9uylTsPsP9wJTOaSHbTe8VTWFpFYWnTC5aLtFb/7k0PS23pHFcREX8ooRMRCbHSymqyD7a8wmVT4qI9DO+XwJrsogBE1vEs3pBPdGQE04b3OWpbWu94AHbsOxzqsKQDKyqtwtNEoaHICGvRHFcREX8poRMRCbFte0twrvUFURobPzCRNTmF1NZ27OImLeWcY/GGPUwd0pv4mKNnGKQroZMAKyqr4oYnlrO3uIKbpqaTnBiH4S1Y5Jzj1PSe4Q5RRDogJXQiIiG2Jb8EoNVLFjQ2PiWR4vJqduxXYlLfhrxD5BaWMWN003MLB/bsQoTBTiV0EgBFZVXc8LflbNp9iP/76kTuvmQkH915HjvmXcKSH5yNJyKCh5dsCXeYItIBKaETEQmxLfnFREdGMKhnl4Acb1xqIgBrsjWPrr7FG/YQYXDBiH5Nbo+OjCC5Rxw79peGODLpaIrKqviaL5n701dP4fxG11xKjy587YxBvPhZzpElS0REAkUJnYhIiG3JL2Zwn65EegLzJ3hI3650ifaQoYSugcUb9jA5vSc946OPuU9ar/hO0UO3aHUuU+a9Q/qd/2XKvHe0HloA1SVzG3cf4v+uPzqZq/Odc4cQHx3Jg29khjhCEenolNCJiITYlj3FDA/Q/DkAT4QxJrm7eujq2V5Qwpb8kiarW9Z3Uu94duw73KEXV9ci18FzqLyKrz3xKRt3H+KP15/CBSObTuYAesRHc8u0wby1KZ+VOw+EMEoR6eiU0ImIhNCh8iryisoZGoAKl/WNT01k4+5DVFTXNL9zJ7B4Qz5As2vzpfWOp6Simn0lHXfpguMtci2td6i8ihv+9ikb84r44/WncOFxkrk6X5+SRp+EGOa9vrlD30QQkdBSQiciEkJbfQVRhgchoauqcWzarfk54B1uOTalO8nNrPtVt3TBzg5cUEaLXAfeofIqvuZL5h67bqJfyRxAl+hIvn/BUFbuOsjbm/YGOUoR6SyU0ImIhFBdQYThAapwWUeFUb6wp6icjOzCZodbgndxcejYSxccazFrLXLdOnXJ3PpcbzLXXC9wY1dNSiW9dzwPLt5MjZYaEZEAUEInIhJCW/KLiYvyNNtz1FIDusfSJyFGCR2wZOMeAGaMar7XJKVHHJER1qELo8ydMZwoT8OFrmOjIrTIdSsUl1dx4xO+ZO76lidzAFEe7+9+S34JCz/LCUKUItLZKKETEQmhLfnFDOvXlYgIa37nFjAzxqUkkpGjhG7xhnxO6hPPkL7N94JGeiIY2LNLh+6hu2x8Er3joxskdXPOTGP2hOQwRtX+FPsKoKzL8SZz/vQAH8tFo/szLqU7v1+yhfIqzXsVkROjhE5EJIQy95QEvCBKnfGp3dlecJii0qqgHL89KCytZNn2/cxswZftNF+ly45qXW4Ruw9VcM+lo9j8q5kkxER26CIwgVR/uYeJv1pCRlYhj153YskceG/A/OSik8krKuefy3YFKFoR6ayU0ImIhMiBw5XsK6kIeEGUOnXz6Nbmdt5eurc37aWm1rXoC3dar3h27S/tsFUHn1uRTWxUBLPGJxEb5WHG6P4sXr9HPUPNaLzcQ1WNI8pjAfu9nTm4N2cP68Nj726jqKzz3oQRkROnhE5EJETqCqIMC3BBlDpjU1QYZfGGPQzoHsvYlO5+vya9dxfKqmrIP1QRxMjCo6yyhlcy8rh4zAC6xUYBMGtcEsUV1bybqSqLx9PUcg+VNS6gyz38ZOZwCkur+PN7nwfsmCLS+SihExEJka11CV0AFxWvr3tcFCf1iScjuygox2/rSiureX9rAdNH9sPM/zmK6b29/x7b95UEK7SweW3dboorqrl6UuqRtjMH96J312heXpMXxsjavlAs9zAqqTuXjU/iiY92kH+oPGDHFZHOxa+EzsxmmlmmmW0zszub2D7HzArMLMP3c1O9bTea2Vbfz42BDF5EpD3JzC8mITaS/t1ig3aO8SmJZGQXdtjhg8fz/pYCyqtqmTG6ZfOb0np3AWDnvtJghBVWz6/MJr13PJPTex5pi/REcMmYAby9aS/F5RrqdyyhWu7hhxcOp6bW8fBbWwN6XBHpPJpN6MzMAzwGXASMBK41s5FN7Pq8c2687+evvtf2BO4BTgMmA/eYWY+ARS8i0o5syS9hWL+EFvUetdS41ET2lVSwu6jz3e1fvCGfHl2imJzWs/md60nqHkd0ZESHW1x8e0EJn+44wFcmpRx1zc0an0RFdS1vbsgPU3Rt39wZdV8qJwAAIABJREFUw/E0qkYbF+UJ+HIPA3t14frTBrFgZTafF3S8XmIRCT5/eugmA9ucc9udc5XAc8Blfh5/BrDEOXfAOXcQWALMbF2oIiJHq1+Fbsq8d1i0OjfcITXJOedbsiA48+fqdNYFxqtqanl7Uz7nj+hHpKdlswkiIoxBIVi6INTX6oKVOXgijCsnphy1beLAHiQnxmnY5XHMnpBM7/hoYiIjMCA5MY4HrhgTlOUebjtvCLGRETwUwPl5ItJ5+POplwxk13ue42tr7MtmttbMXjCzusH6fr3WzG42s5VmtrKgoMDP0EWks2tchS63sIy7Fq5rk0ldQXEFhaVVQZs/V2fEgASiPNbp1qP7ZPt+DpVXt7qcfHqQly4I9bVaXVPLi5/lcO7wvvRtYoivmTFrfBIfbtvH/pKOVwwmEPIPlZNfXMEPLhzGjnmX8NGd5wVt7b7eXWP45tkn8fr6PazOOhiUc4hIx+VPQtfU2KDGkzNeAdKcc2OBt4CnWvBanHOPO+cmOecm9enTx4+QRESarkJXVlUT0Cp0gbIl3zuUKlhLFtSJifQwckA3MrI6V0L3xvo9dIn2MHVo71a9Pr13PFn7S6mpDc7cw1Bfq0szCygoruDqU1OPuc+scUnU1DpeW7c7KDG0dx9u3QfAlCGtu6Za6qapJ9G7azS/fWNzp5wDKyKt509ClwPU/0RIARqM0XDO7XfO1d3i+wtwir+vFRFprVBUoQuUzCAvWVDf+NRE1uUWBS05aWtqax1LNuYzbXgfYqM8rTpGWu94Kmtqg3bthPpafX5FFn0SYjh3+LFvkp7cP4Fh/bq2+WGX4RpW/eG2ffSKj2bkgG4hOV/XmEi+e95QPtl+gPe2aLSSiPjPn4RuBTDUzNLNLBq4Bni5/g5mNqDe01nAJt/jxcB0M+vhK4Yy3dcmInLC+nWLabI90FXoAmFrfjE946Pp3bXpmANpXGoipZU1bNvbOQosrM4uZG9xRauHW4J3cXEgaIVRQlUxEWDvoXKWZhZw5Skpx51PaGbMGpfEip0HyW2DN0EgfMOqnXN8uG0fZw7pTURE8IoYNXbt5IEM7NmF376RSW0nuSEjIieu2YTOOVcN3IY3EdsELHDObTCz+8xslm+3281sg5mtAW4H5vheewD4Fd6kcAVwn69NROSErM8torSy+qh2A26dNjj0ATUjM7846PPn6nS2wihvbthDlMc49+S+rT5Gem9fQhekeXRzZwwnNqrhR260JyLgFRMBXvgsh5pax1WTjj3css6l45IAeKWN9tKFa1j1lvwSCoorOGtIr6Cep7HoyAh+OH0Ym3YfavM9pyLSdvhVCsw595pzbphzbrBz7n5f2y+ccy/7Ht/lnBvlnBvnnDvXObe53mufcM4N8f08GZy3ISKdydub8rnqz8tIiI3mJzOHk5wYhwF9usYQYbB4Y36bGm7onGOrb8mCUEjvFU9CbGSnKIzinGPxhj2cMbg33WKjWn2cft1iiIvysD1ICd3sCcncdt6QI88jI4zucZFcPGbAcV7Vcs45FqzIZnJ6zyNJ6vEM6hXP+NREXs5om8lDuIZVf7DVO+TxrKGhn9d/6dgkRiV146E3M6mormn+BSLS6bWstrOIBFV7KcEfTn//aAff/MdKBvfpyku3nsm3pw3hozvPY8e8S1hx9wXcN3s0728p4JG3284ivXlF5ZRUVIcsoYuIMMalJHaKHrrM/GJ27i9l5gkMtwTv8MO03vFB66ED6NElGoD3557LX2+cREFJJX//eEdAz7F8xwF27i/laj965+rMGpfExt2H2La3OKCxBEIoh6rW9+G2fZzUO57kMAzfjogwfjLzZHIOljH5/rf0eSAizVJCJ9JGtKcS/OFQU+v45csb+OUrGzl/RD+e/9bpTZZjv27yQL48MYVH3tnK0sy9YYj0aFv2eL8oDw9BQZQ641K7s3lPMeVVHfsO/+L1+ZjBhSP7nfCx0nt3Yef+0gBE1bTVWYX0io8mtWcc04b35byT+/LI29vYWxy4ReAXrMgmIaZlPX9fGjuACKNN9tLNnTGcyBAs7l1fZXUty7cf4KxWVkwNhP0lFUQYFJVV6/NARJqlhE6CIpw9Te21l6s9leAPtcMV1Xzrnyv5+8c7uemsdP701VPoEh3Z5L5mxq9nj2Z4vwS+/1wG2QeC9wXdX1vqKlz2DWFCl5JITa1jfW5RyM4ZDos37OGUgT3ok3DixWbSesWTfaCUqpraAER2tNVZB5kwMBEzb4Ly8y+NpKK6JmCLSReVVfHa+t3MGp9EXLT/1T77dovljMG9eHlNXpsrlz97QjLpvbsQ5fH+zjxm/Oby0UFbDw7gs6yDlFXVhGy5gqY89OYWGo8a1+eBBEt7/d4kX1BCJwEXzp6m9tjL5Zzjo237jlllri2W4A+l/EPlXPXnZbyzeS+/umwUd39pJJ5mqs7FRXv401dPodY5bn3ms7D3UmXmF9OvWwzdu7R+jldLjfcVRsnowMMusw+UsnH3oROqbllfeu94qmsdOQcD//9cUWkVnxccZsLAHg3O9/+mpPPvVTmsDcB8x5fX5FFeVXvcteeOZda4JHbuL2VtTtu6AVBdU0tuYTnXTR7Ig1eOpcY5hgZ56PKHW/fhiTDOGBzagij1taclWaR9a4/fm+RoSugk4MLZ09SeerlKK6t5ZvkuZjz8Ptf/dTnHylHaYgn+UNm0+xCzH/uInfsO87cbT+WGM9L8fm1a73h+d9V41uUWce8rG4IXpB9CWRClTt9usSR1j2VNG/uCHkiLN+wBCGhCB8GpdFlXoGaCL9Guc9t5Q+gVH8MvX95wwr1jC1Zkc3L/BMYkd2/xa2eOGkCUx9pcZcXNe4opraxh4qAeXDCin7foke/fPVg+3LaPcSndT6jIzokK19xB6Xza0/cmOTYldBJw4byz2B7uamYfKOU3r23i9N+8zc9eWk+UJ4KHvjKO3355LHGNFkU2g++dP+QYR+rY3s3cy5X/9zHOwYJbzmhVSfoLR/bj1mmD+den2SxYmR2EKJtXU+vYurc45AkdeJcv6MiFURZv2MOIAd0Y2KtLQI6X5kvodgQhoVuddRAzGNsooUuIjeInM4fzWVYhizJaf0d8Y94h1uUWcc2pqUeGdLZE9y5RTBvel1fX5rWpCrGrsw4CMHFgD3rGR3Naei/eWB+8hK6otIq1OYVhqW5Z39wZw4/6PIiLCs4yF8GgIXztR3v43iTNU0InARfOO4tt9a6mc46PP9/Hzf9YyTnzl/K3D3cwdVgfXrjlDF797llceUoKX5mUygNXjDlSgr9nfDTOwZttrAR/KDz9yS6+8dRKBvWKZ9F3pjAqqeU9DnV+cOEwzhzci58vWs+GvND3VmUfKKW8qpbhYUrosg6UcuBwZcjPHWwFxRWs3HWQGaNOvBhKnV7x0STERAZlcfGM7EKG9U2ga8zRcz+/PDGFcamJzHt9M4crjl5b0R8LVmYTHRlxQnPLZo1LIv9QBct37G/1MQJt1a6D9E2IIaWH92/4jFH92Lq3hO0FJUE537Lt+6h1cFYY58+Bd+5g3edBnRvOGBTUuYOBoiF87Utb/d4kLaOETgJu7ozhRw0fDHZVsvrnjo5seFmbwR0XDA36ueHou5ILVmTzr0+zmPnwB1z3l+Ws2HmAb08bzIc/OZfHrpvIpLSeDe6mz56QfKQE/2c/v5D7LhvFW5v2cu8rJz4cqy2r/3sbfc8b3L1oPecM68O/bzmD/t2PrmTZEpGeCB65dgI9ukTz7ac/o6i0KkBR+6euIMrQEC0qXt+4FN8C4x1sPbpFq3O58Hfv4Rw8uzwrYF8U65YuCHQPnXOO1VmFTBiY2OT2iAjjl5eOJP9QBY8t3dbi45dX1fDS6lxmjOpPom9phNa4YEQ/ukR72tQi46uyDjJxYI8jfyen+4bXLt6QH5TzfbB1H/HRnmP+W4VS3efBll9fRN+EGDL3BCeJDTQN4Wtf5s4YTrSn4fem2HbUGyxeSugk4EYnd6fWQYLvTrQnIvhVyerMnpDMWb6J7Ab06BKFc7A+71DQz93UXckfv7iWuxauIyLCePDKsSy763zmzjiZAd39u/P1tTPSuPnsk/jHsl389YPArlfVVjT+vZVU1OCJML40ZgDxTfRmtEbvrjE8dv1EdheV8YMFGdSGsMfzi4Qu9D10Y1K6Y0aHGnZZd70UlnkT873FFQG9+58ehIRux77DFJVVHTdJmDCwB1dMTOavH+xgVwt7CBdv2ENRWVWL1p5rSly0h+kj+/Hauj1UVgen0mdL7C0uJ/tAGacM+qKQTFJiHGNTugdtHt1H2/Zx+km9iPK0na9H0ZER3HD6IN7bUtAm1wpsTEP42pfZE5IZmZTQ4Eb8ZeOT2kVvsHyh7fzFkg7j8fc/JyYygqVzp/HAFWOoqXWMPIEhcy2160ApU4f2Zse8S1j9i+l846x0/v7xTv5zAvNT/NHUXUmA3l2jee32s7hqUiqxUf6XEq9z58yTuWTMAO5/bRP/Xbs7EKE2KVxzHpr6vdXUOv5nyZaAnueUQT24+5KRvL15L//33ucBPfbxbMkvITkxrsmhdsHWNSaSYX0TOlSly2Df/U/rHU9eYRkV1YGrjLo6y1cQpV6Fy6bcOfNkojzGr/+7qUXHX7Aym5QecZwZgKqMs8YnUVRWxftbCk74WCfqs13e39vEQQ1/bzNG9Scju5A9RYFbvw+8w6N37i8N63IFx3LdaQOJjozgyY92hjuUZvU+xhIiSYknNtpCgqOorIqNu4v52hlp7HjgYob168rGvLZ/40AaUkInAbWnqJyXVudy1aRUeneN4XxfIYu3NgVneExj2wtK+LzgMBeM+GJezZ0XncypaT2488V1ZO4J3h+pY9193F9S2aoiBXUiIoz/uWocpwzqwR0LMli580Crj3Us4ZzzEMq7uV87YxCzxiXxP29m8uHWfQE/flO25BeHdEHxxsaldmdNdmGHGbIb7OU90nt3odYR0PULV2cfJCEmkiF9jj/stm+3WL5z3hCWbMzng63+JVTZB0r5aNt+rpqUSkQzy3n446whfUjsEtUmql1+lnWQaE8Eo5O7NWivq2r65sbA9tJ9uM37N2FqGBcUP5ZeXWO4fHwyL36WQ2Fp254T26dr08N+h/VL6DB/hzqS19btprK6lismJmNmfPX0QazLLepQIzs6AyV0ElBPfLSDmlrHN6eeBHi/oIxLTWTJxtAkdG9v2gvA+SO+qIgY5YngsesmEh8TybefXkVxeXDmUHWNbboHJhATi2OjPPzla5NITozjm/9YGfAhYeGc85B4jLXZgjEh28yY9+UxDOnbldufWx30IUBVNbV8XlASlvlzdcalJnKwtIrsA+17uJNzjj8fp2c1UNdLWq+6SpcBTOiyChmXmuhXwvWNs9IZ1KsL976y0a8FzheszMYMrjwlJRChEh0ZwcVjBrBkYz6lla0r0BIon+06yOjkbsRENhzZMKRvVwb3iQ/4sMsPt+6jX7cYhvQN3/+vx/P1s9Ior6rluRXhqdjrj7U5hWzcXcwlY/ofKfCVnBjLWUN6sTSzgHmvbw56UqcKmy2z8LMchvTtemS5k8snJNMl2sPTn+wKc2TSEkroJGCKyqp4dnkWXxqb1KCM+AUn9yUju5C9xYEdHtOUJZvyObl/Aik9GpYx79stlseum8CuA6XM/ffagH+gPPrOVorLq/E06okLZDGYnvHR/P3rp2JmzHnyU/aXVATkuOVVNWFb1HzznkMUl1eFtIhOl+hI/u+rp1BZXcutz3wW0KF1je3af5iqGheWCpd16gqjZLTjwihVNbX89KX1PPD6Zsandic2quFHVyCvl/QjSxcEpgBFWWUNm/cUH1novTkxkR7uvmQk2/aW8M9lx/9CVVPreGFVDmcP7RPQGyCzxiVRVlUTshtxTamormFtblGD+XP1zRjVn0+2HwhYb1VtreOjz/dx1pA+JzSiIphO7t+NMwf34qmPd/qV7IfDI29vpXtcFPO+PPZIga+P7jyff37jNG44fRB/fn97UJM6VdhsmV37D7Ni58EjvXPgXUrlsvHJvLI2L+RFxKT1lNBJwDz9yS5KKqr51jknNWi/YKR3+OPSzXuDev6DhytZtesgF45suoz5aSf14iczh/PGhj0BLTDy2NJtPPTmFq6YkMz8K8fWuyv5/9m78/goq3t/4J8zM9lXspM9QAgQAoRdAQUE0bYq4q7cVq120a63pVdvf9fbalttaW+XW6v19lZ7q3VrESkuwaAgoKwJW/YEyEpmkpBtss/M+f0xmRjCJJnlmZknyef9evmq82SW43S273PO+XyD8PSWHEU3FqdFh+BPX1qKxvZePPR/x9FrZ8+eo6SU2H26Adf9av+o1/FkbHFH7wC+/nIBpoUE4Ec3z/Po8zbSzNhQbL99AU7WtuHLLx3z2NlcWyqdL3rQ2WQlhCFAp5mwy2c6egfw4EvH8OrRGjy6biZ2fH0VntniufdZZLA/pgX7KTZDd6a+HWaLdCo1ccPcOKzJjMGv88vHPHHzcUUTLrb34q5l7oWhjLQ8PQoJ4YE+TbssauhAv8mCxaPsO9yUnQCzRQ6tylDi8dq6B7A60/19iJ704KoMXGzv9XhzdVecqWtHfokBD6/JQNiIpuxCCDx5Sza2rky1FnXve6aoY8Kmc94qrIcQwOZFl39+bl2Zit4BC/5eUOejkXnHZJrN9f4ufZqUegfMePHQBVwzO/aKnmFzEsKQFBmED4oNuGtZqsfGsK/cALNFXrZ/bqSH18xAQXUbnnm/FAuSI7Bihntf3s9+VInteWW4NTcJ2+9YCK1GYItCS59Gszh1Gn579yJ8/ZUCfOe1k3j2vsXQOrl35mx9O578ZzGOXriEudPDcUtuIl48eOGyL0IB4KvXZCg8eispJba9eQo1l7rx6sMrsTwjCl+8yjOPNZobc6ZjXVYsPir7bK+S7WwuAEUKhHJ9JzQCPl3C5afVYH5SxIQs6OrbevDgi8dQ1WTEL25bgDsHC5fNuUkeLfjTY0JwQaFlzbbG2I7O0AHWH7//edM83PCbA/jlnnI8vSXH7vXeOFaLqBD/MT/zXKHRCNy0cDpe+uQC2rr73WqF4KqC6sGG4qPM0C1IjsD0iEC8X9SI2xT4zD1Qaf0cUGMgynDr58QhLToYLx66gC8sSPT1cC7z273liAjyw5euTrf7dyEEnrplPgDgj/vPQUDg327IUmxG1GKRPlttMhFJKbGjoB5Xz4y+4uRtdmIEclMj8cqRajy4Kl21s9busM3m2n73KP39722coSNF/KOgDs3GPnxtxOwcYP0Q3zA3Dgcrm9yaURpPfrEBsWEBQ+vA7RFCYPsdC5AWFYxH/1YIQ4fry0BtxdzmRYn45WAx5y03zJ+O//f5eXi/qBE/e9fxRDxDZy+2vXkKN/3+IKqajHh6Sw52f3M1frBpzmVNzWPDAqDTCrx+vM7lRsdjeeHjc8gr0uPxG+dgeUaU4vfvqDL9lSE5Sp7NLdd3Ii06xKV0UyUtTI7E2YZ21S7Tsud0XRs2P3sIDe09+MuDy4eKOW/IiA5RrLl4YU0b0qKDER1qP/lvNLPiwvDFq9Lx2rEanK1vv+LvzcY+fFCsx5bcpCt6byrh5oVJGDBLvHfWNzNBBTWtSJ4WhPhw+8mIQghsyk7Ax+VNiuz1O1TZjDkJYYgLU3cSo0YjcP/V6ThR3aqq9Nqz9dbZuYdWXzk7N5wQAk/ePB/3rUjF8/ur8Iu8MkVm6k5UX8Ktfzg06t/ZJPtKJ6pbUXOpG1ty7Z8Q2boiDeeauvBpVYuXR+Ydk202lwUduc1skXjh43NYmByBq0aZ8dowLx69AxYcqvRMsmC/yYL95U3YMDdu3OCBsEA/PLd1Cbr6THj0bwUu/cj9w77Pirlf3bnIq8WczZdXZ+D+q9PxvwfP48VDYy8h7R0w49mPKrFu+z7sPFmPh9fMwEfb1uKe5alDYx/e1PzYDzfghX9ZitLGTnzjbwUwKVgIfFrVgp+/X4rP5STgy6u9Oys30sU2+wW9Umdzy/WdyFRBwMKi1Ej0Dlg8mvKqpLyiRtz5x08RoNNgx9ev9vqsSUZMCC6296Kn370TUFJKFNS0IteJ2bnhvr0hE1HB/njyn8VX/Oh9q6AeJotUfLmlzfykcGTEhGDXSe8vu5RS4kR166jLLW2uz45Hn8nidouF3gEzjl1oxWqVz87Z3LE0BWEBunE/973pN/nWvXNfWpU+7nU1GutM3b0rUvHcvipsd6Ooa2jrwbdeLcRtz32Kxo5e3LciBUEe3GM7mfyjoB5BflrcMD/B7t8/v2A6IoP98NdJGo4y2folsqAjt7139iKqW7rxtWtnjjotvyIjGqEBOo+1LzhyvgXGPpPDS4+yEsLwzG05OHahFT9/r9Spx3puXxV+8X4ZbvFhMWfzH1+Yh+vnxePJ3cV291RIKfHemYvY+Ov92J5XhqtnxWDPd6/Fv39uLsLHOIsKAOvmxOGpW+bjo7ImPLGrSJGzqPqOXnzz1QKkx4TgF7cv9PkyjtHO2ipxNrd3wIwLLd0+bVlgs2gwGOWUyoNRpJT404Fz+NrLJ5CVEI63Hlnlk4bs6YPBKO7O0l1s74Whs2/c/nOjiQjyw/c3ZeHohUvYPawHpZQSrx+vRW5qpMeeHyEEbl6YiMPnWxTv9zaehvZe6Dv6Rg1EsVmeHoVpwX7IK3Lve+Xo+UvoN1mwSoXtCuwJDdDhjqUpeOf0RejdWGWiFOvsnB4Prc4Y93vFRqMR+MlgUfcHF4q67n4T/uuDcqz/1T7kFTXiW+tn4cPvrcVPb12Ap7csQPhg6vT0iECP78meiHoHzNh9ugE3zk9AyCg9UgP9tLhjSTL2FOtV8TpT2vRR+iJO1NlcFnTkFiklnt9fhYyYEFyfbf8sD2CNwr52dizySwywWJTfCJ1frEegn8apM/m3LErCl65Kw58Onne4Yffz+6vw8/dLrb3MvLzM0h6tRuC3d+diYXIkHn3lBJb9JH9oc+/vP6rA3S8cxtdfKUCwnw6vPLQC//PFpUMpfo64d0Uqvr52Jv52pAbP7z/n1lgHzBY8+koBuvvN+OPWJT5ptD3Stk1ZCBqxHFKps7nnmrpgtkifBqLYpEQFYVqwn6r30ZnMFjzxdhF+8k4JbshOwGsPr0TsKA2KPc32HnF3H91nDcVdm6EDgDuXpiA7MRxPv1syNGNYUNOKSoMRd3t4GerNixIhJbD7tHdn6U4M7p8br6DTaTW4bm488kv06De5vorgYGUz/LUarPDh8m9n3X91OsxSjpuE6g2/3VuB8ECdQ7Nzw9mKunuWW4u6X+4Zv6izWCR2FtZj/S/343d7K7Bhbjz2fu9a/Ov1WUOFyebcJPzunlwAwK/uXMhizo69JQZ09prG3X9634o0mC0Srx1Vb6sMV5jMFiTYWV49kWdzWdCRWw5VtuBsfQe+es2McYubDfPi0NTZhzN29oO4Q0qJ/BIDVs+KdXqv0g8/Pw+5qZH4wd9PodIwdkz58/ur8Mx71mLuv+5cCJ1WHW+fIH8tbluSBLMFaDL2DUU1/zKvHGfr2/GTzfPxzrdWu7xsbdv1WbhpYSJ+/n4p3j7pegLU0++W4nh1K35+2wKfzLrYszk3aWjvIGANgvnPm+Yp8gOgwmBd3qiGgk4IgYUpkThVq+x7TynGPhMe+r/j+Ovhanz1mhl49t7FCPL33b5D2wzdeTdn6E7WtsJfp8GchPDxrzwKrUbgP2/KRkN7L54b7MP3+rFaBPtr8XkPh2LMjA3F/KRwr6ddFlS3IshPizkOzG5vyk5AZ68Jh8+5vs/nYEUzFqdFItjf9yeZHJUaHYyNc+PxypFqj+5NH8/Z+nZ8UKzHQ2tmODw7N5xGI/DTzfNxz/IUPPtRFX61p3zUoq6wphVbnvsE33n9JGLDAvDm167C7+9dfEWbIgBDe+nt7T8la+7B9IhArBwnGC49JgRrMmPw6tEaRbde+JLJbMH33jyFgto2fGHBdK8mbHvSxPn0IlV6fn8V4sICcOvi8d8A67LioNUI5JfosdDFPSX2lDZ2or6tB9+6bpbTt/XXafCH+xbjC787iK+9fAJvP7rK7vKDPw4WczeprJizeX7fOdj7CgwP8sPWlWlu3bdGI/DLOxZA39GLbW+eRkJ4oNPpoLtPN+DPh87jgVXpuGmhupLZbImJhTWtuPUPn2BAoRnkssZO6DTCqRlRT1qYHIn95RUw9pl8Pju6s7Ae2/PK0NDWg/jwAAgABmM/fnrrfNy3wr3XqxJCA3SICQ1QZIYuJynC7dCS5RlRuGlhIp79sAKvHa2BobMPwf5a5BfrPf7j4+aFifjZu6U439zltddyQU0rFqZEOPQ5uyYzBsH+WuQVNeKa2bFOP1azsQ/FFzvw/etnuzJUn3pgVQb2FOuxs7Aedy/3XIL0WGyzc/c7OTs3nLWosya5/v6jSpTrO1DU0IGGtl4kRgbh4WsycKq2HW8V1iM2LADbb1+A2xYnj7lfPjo0AIkRgThT3+HyuCarps4+7C9vwlccOBEPAFtXpuGrfz2BvaUGbBpjJdZEYLZIfO/NU3j7ZAN+cEMWHlnr/O9GtVLXr1KaUM7UteNgZTMeXJ2BAN34Z9Mjg/2xJG2a4s1q84v1EAJYP8e16O7pEUH43T25ONdkxGM7zlxxdvCFj6vw9GAx92sVFnPA6Jt4ldr7EqDT4oV/WYLkqCB85a8nxp3NHK7S0Ikf/P00lqRNw+M3zlVkPJ6wKCUS2YnheOVwtSL7Bcv1RmTEhHgkgdAVi1IiIaXvz1iPbPzb2NGHix19eGhNhiqKOZsZMSE470ZB12+y4Ex9u8uBKCMtTo2EWQKGTmtfuu5+s1caJtui8b01S9fTb0ZxQ8e4yy1tAv20WJsViz3FepeW89uCulZnOl+09ZI0AAAgAElEQVQM+trKGVGYOz0cLx664LFG3WOxzc59ebVrs3PD2Yq6lRlR2FNsQH1b79Bqkx/tKsauk/V4dN1MfPT9tbhjacq44WcAMD8pwuefd2q061QDzBaJLQ6eDLpuThymRwTi5QkejmK2SHzvjZN4+2QDtm2aXMUcwIKO3PD8/iqEBehw7wrHzwxunBuP0sZO1LUq07QXgHXGLznSrf02q2bF4HvXZ+GfpxrwvTdODjWaXPCjPPzs3VJ8YcF01RZzgGfDPWwig/3xlweWw08rcP+LR9HUOXrDYxtjnwlf/esJBPtr8ey9i1VT3NgjhMDWlWkobewc2sPjjnJ9J2arIBDFZkGydQmSUvvoXGnI2trVj5++U3JFVDQA7D7l2D5Wb0mPCXaruXhpYwf6TBaXA1FG+tOBKxMNvRGxnRgZhOUZUXj7ZL1XiobTdW0wWeS4CZfDbcpOQFNnHwprnX/fHqpsRkSQ35jtbtRKCIEHV6WjTN+JT3wQLf87BWbnhtNoBGpG+W0QExaAbZvmOLW6ICcpAuebu9DRO6DI+CaLHQV1WJAc4fDWB51Wg7uXpeJARbNi/Tm9zVbM7Rws5h5dN7mKOYAFHbnoQnMX3jt7EfetTHPqzNyGedZZtL0lBkXGYejoxam6dmyc535j3a9fOxPZieHYUdgwNHvQ0WuCVgDrs2JVW8wBng33GC4lKhj/+6VlaDH246G/HBuz/5OUEv/2j9M439yF392Ti4QIdfd3AoBbFiUiLEDn9pnI7n4Tai51I0sF++dsokMDkBoVrEjS5chZNltD1p2F9Wjr7sfJ2jbsLKzHrz8ox7dfK8Qtzx7Cwh/vQe5TH6DJaP9EgNqiotNjQtBs7EOniz8GlQhEGc6XEds3L0xEVVMXii96fvnaicFG7M4UwuvmxMFPK5xOu5RS4mBFM66eGe3zgCtX3bQwETGh/vjzQe+2MChqaMeeYj0eXJ2BiCD3ZueGG62VjKFj/BOII81P5j66kUobrctZHZ2ds7l7eQq0GoG/Ha3x0Mg8x2yR+P6bpyZ1MQewoCMXvXDgHHQaDR508sxcRkwIZsSGKNa+YG+ptTB0tF3BWDQagUtd/VccN0vgVx9UuH3/njQ83MPTm3sXpkTid/fk4kx9O771aiHMoyxz+vOhC3jn9EX84IY5uHrmxIgDD/bXYcviJLx7phEtoxQejrAtSZ0d7/sedMMtTInEyRr3C7rRGrJ+942TWPTkB9j87CF85/WT+N2HFTh+oRVhATp8YcF0/L/Pz0VUiL/d+1RbVHRGtHW/WHWLa7N0hTWtiA8PwHSFTmR4YxZ+NJ/LmQ6dRmCXF5ZdFlS3YkZMyKivE3vCA/1w1cwY5BU1OjWLeK65Cw3tvV7vc6ikQD8t7l2Rhg/LDG4tEXbW7/ZWICxQhwdWKdtLVMnXOYNRrvRWQT10GuH0Xvb48EBcPy8ebx6v9WkIj7PMFoltb57CW4X1+P71sydtMQewoCMXGDp78fcTdbhtSTLiwp3/sbJxbjwOn2tx+cz3cPnFeiRPC1Lsh/Noe87UNntgz/DG4IceW+/RsISN8+Lxo5uzkV9iwJP/vLJH3bELl/D0uyW4fl48vnrNDI+NwxO2rkxDv9mCN0/UuXwftgbeaki4HG5hcgQa2nthcLOn0GjvBymBH35uLv7ni0uR/6/XoOTJG3DosfV4+aEV+OmtOXhozQw88YV5XplNdldG7GDSpYs/kk/WtiE3ZZpivRa9NQtvT1SIPzLjQvGnj885tcTWWdZG7G1Y7OD+ueFuyE5AdUs3yvSdDt/mYIV1/9yaCdJ/bjRbV6ZCpxH4yycXvPJ4RQ3tyCvS48FVys7OAcq+zmMYjHIZs0XircJ6rM2KQ3So81tUtq5MQ2v3AN49o67l8aMxWyS2/f0UdhTW43sbZ+Mb6zN9PSSPYkFHTnvp0AUMmC34ios/1DfMi8eAWeLj8ma3xtHTb8bBymZsmBuv2I8mX54Fn2i+eFU6Hl6Tgb98Wo3/Hbbcx9DZi0dfKUDytCD88k7fNw93VmZ8GFZkROGVI9Uu90ysMBjhr9MgLVodCZc2i1JsDcbdO2MdGWz/R1xSZBAevmYGNs6Lx6y4MLttRLw5m+yOtCjXC7pLXf240NKNRQottwR8+7ztLKxHZZMRZokrltgq6UJLNy519TsciDLcxnnxEALIO+v46o+Dlc1IiQpS3fvUWXFhgbhpYSLeOF6L9h7P7xezzc49uFrZ2TlA+dc5g1E+c6iyGYbOPtzmQCq5PVfPjMaMmBD8dQKEowwVcwX1+NeNs/HN6yZ3MQewbQE5qbN3AH89XI0b5ye4HGG9OHUapgX7Ib9Ej88vmO7yWA5WNqPPZFFk/5zNtk1ZeHzHmcuWk6lx9kAtHr9xLurbevCTd0rwh48q0do9AD+tBhZpwV8eXON28pmvbF2Zhm++Woj9FU1YlxXn9O3LGjsxKzZUdftyshMjoNUInKptc/l9c6iyGW3dA9AIYHi968z7xNYqQs2C/LWYHhHoUgjAycFwDqUSLm189bxtzyvDgPnykxu2QBYlx1MwGEbkTCCKTWxYAJakTsP7RY349obxf7yZzBYcrmrBF1TWRsVVD67KwI6Cerx5vBYPrfHcqojihg7kFenx7esyFZ+ds1HydZ6TFIE9xXp09A5M2O8jpewoqEN4oA7r5zr/nQZYQ3juW5mGp3YXo6ihHdmJ6gwSMlskfvD300PF3LemQDEHcIaOnPTq0Rp09prwtWtnunwfWo3Aujlx+LDU4FajyvxiPcICdFiWHuXyfYw0UWYP1EKjEViXFQchgEvdA5AA+s0WCCGGlh1ORJuyExATGoBXXDwTWa7vRJaKEi5tgvy1yIoPczkYpayxE1/76wnMjg/DT2+dP+nfJ+nRIS41Fy+saYNWI5CTrM4fPM7yViDLiRrrfsvMONeW0G/KTkDJxQ7UXhp/3+OpujZ09pmwegLvnxtuflIElqdH4aVPLoy6r1kJv9tbgbAAHR5UeO+cp9iCUYqm+LJLY58J7xc14qaFiQ61mRrN7YuTEeinwcuH1RmOYrZYw9j+UVCH726YOsUcwBk61RvegDcxMgjbNmX57EdTn8mMPx04j6tnRmNBsntnnjfOjceOgnqcqG51ukk1AFgsEntLDbg2K1bxKPyJMHugJr/Jr8DIHIIBs1T87L03+es0uGtZMp7bV4X6th4kObHktqN3ABfbe5GpskAUm0Wpkdh9qgEWi3Sol5ONvqMXD7x4FEH+Wrz4wDIkRgbhnuXq6RvnCekxIXj/rPP7RQpr2jAnIQzB/pPjKzYxMgj1doo3pZeiF1S3IjdtmlOvy+E2ZSfgp++WIK+ocdxZqgMVzRDCuoxssnhwdTq+9nIBPijW44b5yjeALrnYgfeLGvGt6zIRMcqya7UZHoxy1ST6/9pZ7525iN4BC7YsTnbrfiKC/XDTgkS8fbIe//65OQhTwazn8N/JQf5adPeb8Z0NmQ7N1E8mnKFTsbGiwX01HkNnn1uzczZrZsfCX6txOe3yVF0bmo19ii63JNf4Mk7dk+5ZngoJ4NUjzp2JrBgMZVBTy4LhFiVHoqPXhAtOzDwZ+0x48KVjaO8ZwJ/vXzZl9pTOiAlBa/cA2rqvTL8djcUicaq2TbF2BWrgjUCWzt4BlOk7sdiN5y01Ohhzp4cjr6hx3OseqmxGTlIEpjmRpql2G+clICkyCH8+5JkWBrbZuS9PkNk5wBqMMj0iEGem+D66HQX1SI8Oduv9ZbN1ZRq6+814y0e/RYcb+Tu5u98MnUYgfYLvi3UFCzoVGy0a3NONZO2xWCT++PE5ZCeGK5IIFhqgw8qZ0ch3sR9dfokeWo3A2tmurQUn5UzWIJnkacFYnxWH147Vot/k+NLgcr2tZYE6C7qFg/u6TjrYYNxktuDRVwpQ2tiJ39+3GPMnYANmV6XHOB+MUtVkRGefCbkpyjQUV4PhS9Ft/nXjbEVn4E/WtkFKuBSIMtym7Hgcr25FU+fobUeMfSYU1rRN6HYF9mg1AvdfnY6j5y8pHgRScrED751txAOr0ifM7JzNVA9GqWvtxqfnWrBlcbIiIWULUyKRkxSBlw9XO9UmxBPs/U42WaRPfif7Ggs6FVPTzMeeYj3ONXXha9fOVCy1cOPcOJxv7kJVk9Hp2+4tMWBZ+rQJ98UyGfkyTt3Ttq5MQ7OxD3uKxz/jb1PW2Ilgf61TyzS9aVZcKIL9tTjlQEEnpcR/vH0W+8ub8JPN810KiJnIMmKCAcCp2UylG4qrha0typF/vw4aAVxyYtbSESeqWyHEZ0msrtqUnQApgQ+KR1/9cbiqBSaLxJpJVtABwJ3LUhDsr8WLhy4oer9De+c8kGzpaTlJETjX3KVIq6SJ6O2T1v6Rtyp4AmbrylSU6404dqFVsft0hZp+J/saCzoVU8vMh5QSz++vQmpUMG5UcF3+dYPNwPPH+OK1p/ZSN0obOxVpJk7um8xBMtfMjkXytCC87EQ4SoWhE5lxoS7vA/I0rUYgJykCJx1oXfCHfVV49WgtHl03E/csT/XC6NQlJSoYGgGcb3a8uXhhbSsigvxcTgFWu/jwQKzLisPfT9S5FWo1UkFNG7Liw9zekzMnIQypUcFjLrs8WNmMQD8NlqRPnllUm4ggP9y+JBn/PNUAQ6d7/SZtShuts3P3r0pHZPDEW6Jq20dX1DD1glGklPhHQR2WZ0QhJSpYsfu9aWEiwgJ1Tn03KslikfjTgXMYbX5woq8QcgULOhX7rp0Nnb6Y+Thy/hJO1rbh4WtmQKdV7iWTGBmEedPDnd5HZ7s+Czr18GZTc2/SagTuXZGKw+cuodLgWGpnWaNRtcstbRalRKKkoQN9JvOo13n7pHWj+S2LEvH96yf+bKsrAnRaJE0LcmrJZWFNGxalRE64/ovOuGtZCpo6+/BRWZMi92exSBRWt7rUUHwkIQRumJ+AT6qa0THKjMzBymYsz4h2K+1Pze6/Oh39ZgteUSiJ8Hd7KxAaoMOXJ+DsHIChZeJTcdnlqbp2nGvqcrn33GiC/XW4bXEy3jt7Ec3G0Zc3e4Khsxf3v3QMP3mnBNmJ4Qj0u/x36WRZIeQsFnQqFhceCACIHrZpe9smZfctOOL5/VWICfXHHUvcS0eyZ8O8eJyobsWlLseX7+SX6DErLnRofwuRJ925NAX+Wsdimi919aPZ2KfKlgXDLUyJRL/ZgtKL9ovUw+dasO3N01iREYVf3L5gUhcn40mPDnG4F52xz4QyfeekW2450ro5cYgJDcDrx2oVub8Kg3Xf4RIX+s/Zsyk7HgNmiY9Kr9yjfbG9B5UGI1bPmryJhzNiQ7F+ThxeOVI95kkbR5Q2duDdM9a9cxNxdg6w9ihMCJ+awSg7CuoQoNPgxhzXe/6OZuvKNAyYJd44rszngCM+LNXjxt8cwJFzLfjJ5vnY/c3VeGbLgkm5QshZDhV0QogbhBBlQohKIcRjY1zvdiGEFEIsHbzsJ4T4ixDijBCiRAjxuFIDnwryS/QI8tPi0GPr8enj6+Gv1aDC4HxPJFftLKzH8p/mY19ZE/pNFrx/1vF9RI7aODceFgm7X7z2dPQO4Mi5S5ydI6+JCQ3AjTkJ+MeJOnT3m8a8bvlgwmXmBJihA2C3H12loRNf+b/jSI0Oxgv/snTSzmI4KiPGWtA5svn/9GCwR65ChYla+Wk1uG1JEj4qM8DQ4f6yvoKawYbiCszQAUBuyjTEhgXYXXZ5sKIZALB6Vqwij6VWD6xKR7OxHyt+thcZj72DVc986FRC9s7Ceqx65kPc8JsDEACmRwR6brBeMD8pYsoVdP0mC3adasD12Qkeaao+Ky4UV82IxiuHazza+xAAegfMeOLts3jwpeOICw/E7m+uxtaVaRBCTNoVQs4at6ATQmgBPAvgRgDzANwjhJhn53phAL4F4Miww3cACJBS5gBYAuCrQoh094c9+UkpkV+sx+rMGAT6aTE9Igh3L0/Bm8drUdfq+H4OV9miYA2DSWEdvSaPtEyYnxSO+PAAh5dd7i9rgskisWHu1ApnIN/aujINnX0m7BrcXD6acpW3LLCZHhGI2LCAK5IuDZ29uP/FY/DXafHi/csYOgTrDF1nnwktDqwiKBx8Phe52adzIrhzaQrMFol/FLj/nXCiuhVRIf5Ij1Zmj49GI7BxXjz2lTWhd0QC3qHKZsSE+mOOymfR3dXc2QcBoK17YFjbo9N460QdLBY55j9vnajD4ztOD/UelACe2l3is5ZJSshJisD55i4Y+8Y+KTeZfFRmQFv3ALYovNxyuK0r01Df1oP95a4lljuitLEDN//+IP7v02p8eXUGdj56tepPmvqCI11PlwOolFKeAwAhxGsAbgFQPOJ6TwH4BYDvDzsmAYQIIXQAggD0A5h6u1JdUHKxEw3tvfjOhtlDx76+diZeO1qLP+yrws9uzfHo44/VMkHJsx9CCFw3Nx5vF9ajz2QedzYgv0SPqBD/SX8GnNRlado0ZMWH4eUj1bhrWcqoSxDL9Z0IC9QhPjzAyyN0jhACC5MjLyvouvtN+PJLx9Fi7MfrX12p6Ab6iSwj1rq0+0JzF2JCx/7/tbCmDTNjQ6ZEITwzNhTL06PwxvFafO3aGW4tyy2obsXi1GmKLu3dlJ2Avx2pwcGKZmwY7FcqpcTByhasmhWj2tAipfxyT/kVgRE9AxZ8981T+O6bp5y+P098/3tTTnI4pASK6tuxYsbkXW473I6COsSGBXg0zfX67HjEhgXg5cM1WD9H2ZVTUkr85ZML+Nl7pQgP9MNLDyzD2imWtOwMR5ZcJgEYvkC2bvDYECFELoAUKeXuEbf9O4AuABcB1AD4pZTy0sgHEEJ8RQhxXAhxvKlJmU3WE11+iR5CWPcq2EyPCMJdy7wzS+fNKNiNc+PR1W/G4XNXvDQuM2C24KNSA9bPiYN2kn8Zk7oIIbB1ZSrO1nfg1BjpkOWNRmTFh02IPWeLUiJwrqkL7T0DMJkt+ObfClHU0I7/vicXC6bADJOjMgYb1J4bZx+dlBIna1uxaBL1nxvPnctScL65C0fPj/3ZPZZLXf0419yFxWnKvuaumhGNsEDdZcsuSxs70Wzsm3T95+wZ67v6uxtmj/mPK/epdrZglKmy7LK1qx8flhqweVGiomF2I/lpNbh7WQo+KjOg9pJyv0ubjX148KVj+NE/i7F6Vgze/84aFnPjcGSGzt4vk6ETP0IIDYBfA7jfzvWWAzADSAQwDcABIUS+bbZv6M6kfAHACwCwdOlS33YpVIn8Ej0WpUQiNuzyM8JfXzsTrx/z/CxdRJAf2nquTAjzRBTsVTOjEeSnRX6xHtfOHn1fw/ELrejoNXG5JfnE5twkPP1eKV4+XG23V5aUEmX6Tnx+gfKbzz3BtvRo4Y/3IMRfi65+M566JXtoNoOskqcFQacR4waj1LX2oNnYP+kDUYb7XE4CfrSrCK8fr3V51qNwcP+cUoEoNv46Da6bE4f8Ej1MZgt0Wg0OVVr3z63JnPwFXWJk0NCSyeGSIoPwbTsJ2sO9cbzW7m0nchR8XFgg4sMDPJ50ubPQmg7c0NaDxMggbNuU5ZNZzX+ebsCAWWLLYuXD7Ea6Z3kq/vvDStz42wPo6jM5/d898jn7/IIE7CioR0evCT++ORtfvCptQpwk9TVHyvY6ACnDLicDGL6RJAzAfAD7hBAXAKwEsGswGOVeAO9LKQeklAYAhwAsVWLgk5m+oxen69rtBn8kRgbhzmXJeHOUD1wlGDp60TNgwshJME9FwQb6abEmMwZ7S/RjBg/kl+jhr9VgTebk3sxO6hQW6IfNuUn456kGtNlpqtzU2Yf2ngHMjgv1weics7OwHi99cmHocle/GTqNcLsH2GSk02qQEhU8bnNxW7DHVCrogv11uHlRIt49c3HUFgHjKahphU4jPDIrvCk7Aa3dAzh6wTqDeKCiGTNjQzA9YuIWJo7atikLQX6Xb2Fw9DvcnduqWY6Hg1Fs2QP1bT3D9i0qnz3giH8U1GPu9HDMnR7u8cc6ev4SNMJ6ktDZ/257z9kLH5+HTiOw6xur8KWr01nMOciRgu4YgEwhRIYQwh/A3QB22f4opWyXUsZIKdOllOkADgO4WUp5HNZlluuFVQisxV6p4v8Vk8zeEuvm0o2jnCl/ZO0sAMAfPqr0yOP/+J/FkBB4/Ma5XouC3TAvHg3tvSi+aH+LpZQS+SV6XD0rGiEBjkwsEylv64o09Jks+PuJuiv+VjYYiDJ7AoQtbM8rQ+/A5U2hTRaJ7XllPhqRuqVHB4/bXLywpg1BflrVB+Io7a6lKegdsIwbGDSaE9WtmJcYjiB/5dNUr82KRYBOgz1FevSZzDhyvgWrp8ByS2BwRcGWHJe+w925rZrNT4rAOQ8Go4yVPeBNlQYjTtW2Kd57bjTb88owMuSyZ8CMH+48gyf/WTzmPz/ceeaK5wwANEJgToLni9HJZNxfxlJKkxDiGwDyAGgB/FlKWSSEeBLAcSnlrjFu/iyAFwGchXXp5otSytMKjHtSyy/RIyUqCJmjnOlPjAzCnUtT8MbxWjyybhaSFFwGkV+sxztnLmLbpiw8fM0MPHzNDMXueyzr58RBCCC/2IDsxIgr/l7VZER1SzceWuOd8RDZMy8xHItTI/HKkRo8uCrjsmCFcr0RAFTfVBzw7h7ZySAjJhSHz12ClHLUs8WFtW1YkBzh0f0qarQgOQJzEsLwxvFabF2Z5tRtTWYLTtW2465lKeNf2QXB/jqsyYzFnqJGXD8vHr0DFqyeQis8NucmuVyEuXNbtVqQHOHRYBS1fK6+VVgHjQBuXpTolccb7b+vq8+MN8fpUdfVZ79P4sV299uhTDUOTXVIKd8F8O6IY0+Mct21w/7dCGvrAnJQd78Jhyqbce+K1DGnmR9ZNwtvHK/Fc/sq8ZPNyuylM/aZ8B9vn0VWfBge9nLhFBMagNyUSOSX6O2u7/+g2Dpryf1z5GtbV6bhX984hU+qWrB62F6c8sZORIf4j5uEqAaj7a+ZyHtkPCkjJhg9A2boO/qQYKcfV5/JjJKGDjy4OsMHo/MtIQTuXJqCJ3cXo7ihA/MSHT+rXtrYiZ4Bs2L95+y5YX4C8kv0eG5/FbQagZUzojz2WKRuw4NRPFHQqeFz1WKReKugHtfMjkVcmHd6B461X/PQY+vHvO2qZz70+XM2WUytU4kTwMGKZvSZLNg4TuPspMFZuteP1Sp29ueXeWVo7OjF07flwF/n/ZfGhnnxOFPfjkY7Z2byS/SYnxQ+JfY+kLp9Lmc6pgX74eXD1ZcdLzd0TojZOWDy7pHxlPQYa9Ll+VGCUYoaOtBvtkyp/XPD3ZqbBH+tBm+MczZ+pBPVg4EoHizo+k3WGYADFc3QCjG0pYGmHk8Ho2zblHVFimCATuO1z9WdhfVY/rN8NLT34lRtm9f27nG/pjqwoFOZ/BI9wgJ1WJYx/lnER9ZZ99I9t6/K7cctrGnFXz69gC+uTMNiH/V4s4XA7C29vMl4i7EPBTWtdkNiiLwt0E+LO5em4IMS/dDJByklyhs7MTte/YEowOTdI+Mp6YOtC0YLRimssfbzy7WTfjoVTAvxx/XZ8XirsP6KRt5jOVHdioTwQCTamfVUws7Cejy1u2Tocr/Z4rOQClIHTwaj5KZGQgKICNINFXaz4kJxixeWPtrCRZqN1sCu1u4Br73WuV9THZguoSIWi8SHpQaszYqDnwP7MJIig3DH4Czd19fOdHmKemDwSy4+LBDf9+FZkcy4UKRGBSO/WI/7Vny2F+PDUgOkBAs6Uo17V6Tijx+fw2vHavCdDbNR39aDrn7zhAhEsZmMe2Q8JTEyCP46zaitCwprWpEUGYS4cO8scVKju5alYPfpi9hTrMfNCx37AVtQ04rFaZEeS7EbK6SCr/2paX5SBPaWGmDsMyFU4YA1W8/D3d9cg5SoYLzwcRV+9m4p3j/biBtzPNvOxtevde7X9D3O0KnIybo2NBv7ndon9sjamZCQbs3SvfDxOZQ2duKpzfN9GlsuhMCGufE4VNWC7v7PUqjyS/RICA9EthN7M4g8KS06BNfMjsVrR2thMltQMYECUch5Wo1AWlTwqM3FC2vasGiKLre0WTUzBkmRQXjjmGPLLvUdvahr7fHoihC1hFSQeuQkWYNRihvsJ2q7I69Ij3nTw5ESFQwAeHBVBrITw/Gfu4rQbqevr5L4WicWdCqSX6yHViOwdrbjBV3ytGDcvsQ6S3ex3fk37vnmLvx2bwVunJ8wapsEb9owLw79JgsOVFgbwPYOmHGgohkb5sWxFwmpytYVqWjs6EV+ieGzlgVxLOgmq/SYELszdIbOXtS39UzZ5ZY2Go01HOVgZTNqL43d4gEACrywf260VSsMXJi6coYFoyjJ0NGLE9Wt2JSdMHRMp9XgmS0L0Gzsw8/f91zHLiklAv3st/3ga33qYEGnIvkleixPj0JEsHOzZI+umwmLdH6WTkqJH751BgE6DX58c7ZTt/WUZelRCAvUIb/Yuo/u03Mt6O434zoutySVWT8nDtMjAvHKkWqU6zsRHx7g9HuXJo6MmBBUX+qGZUTDpZO2/XM+2nusJrcvTYYQGDeqHLAut/TXaey2qVEKAxdopLjwQMSFKR+MsmfwN8sN8xMuO56THIEHV2Xgb0dqcGywwb3SXj5Sg54BM3Say09687U+tbCgU4malm6U643Y4MIsWfK0YNyxNAWvHXVulu7vJ+rwSVULHrtxjmr2fvhpNViXFYcPSw0wWyTyi/UI9tfiKg9EDBO5Q6fVYFFKJA5UNGNHQT3augcYtjCJpUeHoN9kQcOIz9jC2jb4aQWXhMO6r3tNZizePFEH88hOwyOcqG7FgqQIjyYqM3CB7PFEMEpeUSPSo4PtBmP96/WzkRQZhMf+cRp9JsdDgxxxsrYNTw7gRdAAAB7aSURBVP6zCGuzYrH9tgV8rU9hDEVRifwS69kdV/usPbJ2Jt48Xovn91Xhx7fMH/f6zcY+/PTdEixLn4Z7lqW69JiesmFePHadasDJ2lbsLTHgmszYUZcTEPnKzsJ6fFT6WQR6n8kaLgSAX6KTUMaw1gXJ04KHjhfWtGJeYgQ/owbdvSwFj7xSgAMVTVibZf/7rM9kxtn6DjywKt3j42HgAo00PykCH5YZ0NVnQogCwSjtPQP4tKoFX16dYXdrSLC/Dj+5dT4eePEYnttXhe9smO32YwLApa5+PPLyCcSHB+I3dy1CZLA/bl2SrMh908TDGTqVyC/RIzMuFGmD8djOSokKxh1Lk/Hq0Vq7fdxGemp3Mbr6THh6Sw40GnXtTbt2dix0GoHf5FegsaMX17GZOKnQ9rwy9Joslx2zpYrR5GMr6IbvozOZLThd1z7l988Nt2FuPKJC/PH6GOEoZ+ttffu4TJW8bygY5aIywSgfluphskhcn50w6nXWZcXh5oWJ+MNHVag0dLr9mGaLxLdfK0RzVz+eu28JIoP93b5PmthY0KlAe88Ajp6/5NJyy+EeWTtrcC9d5ZjX+6jMgLdPNuCRtbMwS4UhDhFBfsiICR4KRvnVB+Vcykaqw1SxqSU+PABBflqcb/4s8KNcb0R3v3nKNhS3x1+nwa25Scgv0aPF2Gf3OrZAlMVpfN7I+3KSB4NR6pRZdpl3Vo+4sIBxT+w8cdM8BPlr8fiOM1fsxXXWb/LLcaCiGU/enD3030NTGws6Fdhf3gSTRbrdZy0lKhi3L0nGq8dGn6Xr7jfh/711FjNjQ/DIupluPZ6n7Cysx4WWz340Nbb3shksqQ4T9KYWIQTSooMvay5eWGstTHJTONM03F3LUjBglnhrlM/sgppWpEYFIy5MHXu3aWqJDw9ErELBKL0DZuwvb8L12fHjrnaKCQ3ADz8/F8cutOLVYzUuP+aHpXr894eVuHNpMu5erq4tM+Q7LOhUIL9Yj+gQfyxSYNnOo+tmwWKReH6//cTLX39Qjvq2Hjxz2wIE6NS552N7XhkGzJefveJSNlIbJuhNPTNiL29dcLKmDdEh/kiJYhE/3Oz4MOSmRuL1Y7WQ8vLPcikljle3YjFnNcmHlApG+bi8CT0D5svaFYzljiXJuGpGNJ55txT6jvG3x4xU09KN77x2EtmJ4XjSgbwEmjpY0PnYgNmCj8oMWD8nDloF9rKlRAXjtsXJ+NvRmis+LM7UteN/D57HvStSsSw9yu3H8hQuZaOJgAl6U096dAhqLnXDZLbunSysbUNuaiR7ZNpx19IUVBiMKBhs62BT19qDps4+j/afIxrP/KQIVDUZ0d1vcut+3i9qRHigDisdTOIWQuBnW3LQZ7bgR7uKnHqs3gEzvvbyCQDAc/ctYRATXYYFnY8du3AJnb0mt/fPDWebpRvel85ktuCxHacRExqAf7thjmKP5QlcykYTxebcJBx6bD3OP/N5HHpsPYu5SS49JgQmi0Rdaw/aewZQaTAy2GMUX1iYiGB/Ld4YEY5SUGPbP8fnjXwnJykCFgkUN7gejDJgtmBviQHXzY2Hn9bxn9MZMSH49nWZeO9sI/YUNTp8uyfePoviix34zd2LkBodPP4NaEphQedj+cUG+Os0WJMZo9h9pkZfOUv350PnUdTQgR/fnI2IIHU3P+ZSNiJSo6HWBS1dOFU72FCcCZd2hQbo8IUF07H7dAO6+j6bBSmobkWwvxZZ8eoL5KKpIydpMBjFjWWXR89fQnvPgMPLLYf7yjUzkBUfhifeLkJn78C413/taA3eOF6Hb66fhfVzlJsAoMmDBZ0PSSmRX6LHqpnRCPZXtiXgo+tmYcBkwfpf7UPGY+/gZ++WIjsxHDfMd/6Dx9u4lI2I1Cg9+rPWBYU1bRACTJgbw13LUtDVb8Y7py8OHTtR04pFKZHQOTGjQaS0+PAAxIQGuFXQ5RU1ItBPg2tnxzp9Wz+tBs/clgN9Zy9+OU4+wJm6djyxqwhrMmMU62FHkw8bi/tQpcGImkvd+Oq1MxS/74KaVmg0Al195sse7+2TDROiMGIzWCJSm5hQf4QF6HC+uQs1l7oxOy4MYYHqXvHgS4tTp2FmbAheO1aDO5eloLvfhJKLnXhkrToTlmnqEEJgQXKEy0mXFovEniI9rsmMRZC/a3vZclOn4UtXpeMvn17ALblJWGxn+XZbdz++/soJxIT447d35yqStUCTE0+R+dAHJXoAwHUemD7fnlcG84g+J30mC5MiiYhcJIRAekwIzjd34eRgIAqNTgiBu5eloqCmDZWGTpyqbYfZIu3+cCXytvlJEag0uBaMcqquDY0dvS4ttxzu+5uykBAeiMf/cQb9Jstlf7NYJL7z+knoO3rxh61LEBXC5uE0OhZ0PpRfrEdOUgQSIpTvxcOkSCIi5aXHhODYhUto6x5gQeeAWxcnQacReP1Y7VAgCp83UgNbMErJReeDUfKK9NBpBK6bG+fWGEIDdHjqlvko03fifw6cu+xv//1hJfaVNeGJm7IVaWtFkxsLOh9pNvahsLbN7Wbio2FSJBGR8jKig9E7YD2TzoTL8cWEBmDD3Hj8o6Aeh8+1YGZsCCKDOdNAvmcLRjld59yySykl9hQ1YuWMaEVeyxvmxeNzOQn47d4KnGsyAgD2lRnwm73l2JKbhK0r2DycxseCzkc+LDVASrh9dmc0TIokIlJeS3ff0L/f/+JR7Cys9+FoJobU6CBc6urHgYpmXGzv5XNGquBqMEqlwYhzzV3YlK3cCfkf3ZQNAYkbfnsAGY+9gwdeOoaEsAD89NYc9rkkh7Cg85H8Yj2mRwQiOzHcI/fPpEgiImXtLKzH349/Vow0tPXi8R1nWKCMYWdhPf76afXQ5e5+M58zUgUhBHKSwp0ORnn/rLV33MZ5yqWGf1LVAosE+k0WSABSApe6B5DnRJ86mtqYcukDvQNmHKhoxu1Lkj165oVJkUREytmeV4a+EcEFPQNmbM8r42ftKLbnlaFngM8ZqVNOUgT2lzehu9/kcPuovOJG5KZGKpp/sD2vDANm+0F2fJ+QIzhD5wOfVrWgZ8DsseWWRESkPIZNOY/PGanZfCeDUepau3G2vsPtdMuR+D4hd7Gg84EPSvQI8dfiqpnRvh4KERE5iGFTzuNzRmqWk2wNRjnjYDDKniJruymlCzq+T8hdLOi8zGKR2FuixzWzYxGgc60ZJREReR/DppzH54zULCE8EDGh/jhT79gMXV5RI2bHhyIjJkTRcfB9Qu7iHjovO9vQDn1HH67zULsCIiLyDNtelu15ZWho60FiZBC2bcriHpcx8DkjNRNCYH5ShEPBKC3GPhy7cAmPrpul+Dj4PiF3saDzsvwSAzQCWJcV6+uhEBGRkxg25Tw+Z6RmOUkR+Li8CT39ZgT5j75yKr9ED4tUfrmlDd8n5A4uufSy/GI9lqRNQ3RogK+HQkRERDSl2YJRiscJRskr0iMpMshj7aaI3MGCzovq23pQfLGDyy2JiIiIVCAnyRqMMtayS2OfCQcrmrEpO4GNvkmVWNB50Ycl1nSkDSzoiIiIiHxuekQgokP8cWaMgu6jUgP6zRZsyubvN1InFnRe9EGJARkxIZgZq2w6EhERERE5z5FglLyiRkSH+GNpepQXR0bkOBZ0XmLsM+FwVQs2zI3jdD0RERGRSuQkRaDCYETvgPmKv/WZzNhX1oSN8+Kh1fD3G6kTCzovOVDehH6zhfvniIiIiFRkflIEzBZpNxjlk8oWGPtMHku3JFICCzov+aBEj4ggPyxNm+broRARERHRoJzk0YNR8ooaERqgw9Wzor09LCKHOVTQCSFuEEKUCSEqhRCPjXG924UQUgixdNixBUKIT4UQRUKIM0KIQCUGPlHsLKzH1c/sxY6CevSbLNh9+qKvh0REREREgxJtwSh1lxd0ZovEB8V6rM2KRYBu9B51RL42bmNxIYQWwLMANgKoA3BMCLFLSlk84nphAL4F4MiwYzoALwP4FynlKSFENIABBcevajsL6/H4jjPoGVyT3TNgxuM7zgAAm0cSERERqYAtGGVk0uXxC5fQ0tWPG+ZzuSWpmyMzdMsBVEopz0kp+wG8BuAWO9d7CsAvAPQOO3Y9gNNSylMAIKVskVJeueN0ktqeVzZUzNn0DJixPa/MRyMiIiIiopHsBaPkFenhr9NgbVacD0dGND5HCrokALXDLtcNHhsihMgFkCKl3D3itrMBSCFEnhCiQAjxA3sPIIT4ihDiuBDieFNTkxPDV7eGth6njhMRERGR99mCUUoGg1GklMgrasTqWTEIDRh3QRuRTzlS0NnLaJVDfxRCA+DXAL5n53o6AKsB3Df4v7cKIa674s6kfEFKuVRKuTQ2NtahgU8EiZFBTh0nIiIiIu8bGYxS1NCB+rYeNhOnCcGRgq4OQMqwy8kAGoZdDgMwH8A+IcQFACsB7BoMRqkDsF9K2Syl7AbwLoDFSgx8Iti2KQuBusuf4iA/LbZtyvLRiIiIiIhopMSIQESF+OP0YDBKXlEjNALYwHZTNAE4UtAdA5AphMgQQvgDuBvALtsfpZTtUsoYKWW6lDIdwGEAN0spjwPIA7BACBE8GJByLYDiKx9ictqcm4QHVmUAsE5zJkUG4ektOQxEISIiIlKRkcEoeUWNWJYehejQAB+PjGh84y4KllKahBDfgLU40wL4s5SySAjxJIDjUspdY9y2VQjxX7AWhRLAu1LKdxQa+4QQF279IDjyw+sQFzalOjYQERERTRg5SeF4vrIZpY0dKNcb8cQX5vl6SEQOcWiXp5TyXViXSw4/9sQo11074vLLsLYumJIqDUZEBPkhlmd4iIiIiFQrZzAY5TcfVAAAruf+OZogHGosTq6rMBiRGRcKIexlyxARERGRGthSyN8vaoSfVuD4hVYfj4jIMSzoPKzSYMSsuFBfD4OIiIiIRrGzsP6yPsEDZonHd5zBzsJ6H46KyDEs6DyoxdiHS139LOiIiIiIVGx7Xhl6BiyXHesZMF9W5BGpFQs6D6o0GAGABR0RERGRitmWWzp6nEhNWNB5UGWTtaDLjA/z8UiIiIiIaDSJkUFOHSdSExZ0HlShNyLYX4vECLYrICIiIlKrbZuyEOSnvexYkJ8W2zZl+WhERI5zqG0BuaaqyRqIwoRLIiIiIvXanJsEwLqXrqGtB4mRQdi2KWvoOJGasaDzoAq9EVfPjPb1MIiIiIhoHJtzk1jA0YTEJZce0tk7gMaOXsyKZyAKERERERF5Bgs6D7ElXGbGMRCFiIiIiIg8gwWdh1SwZQEREREREXkYCzoPqTIY4a/TIGUa426JiIiIiMgzWNB5SIXBiBkxIdBp+RQTEREREZFnsNrwkEqDkcstiYiIiIjIo1jQeUDvgBm1rd0s6IiIiIiIyKNY0HlAVZMRUjLhkoiIiIiIPIsFnQdUMuGSiIiIiIi8gAWdB1QajNBqBNJjgn09FCIiIiIimsRY0HlAhd6ItOhgBOi0vh4KERERERFNYizoPKDC0IlZsVxuSUREREREnsWCTmH9JguqW7qRGc+CjoiIiIiIPIsFncKqW7pgskgGohARERERkcexoFOYLeGSLQuIiIiIiMjTWNAprGKwoJsRG+LjkRARERER0WTHgk5hlQYjkqcFIdhf5+uhEBERERHRJMeCTmEVBiP3zxERERERkVewoFOQ2SJxrsmITBZ0RERERETkBSzoFFTX2o0+k4WBKERERERE5BUs6BRUobcGoszkDB0REREREXkBCzoFVTZZCzruoSMiIiIiIm9gQaegCr0RcWEBiAjy8/VQiIiIiIhoCmBBp6DKJiMy4zk7R0RERERE3sGCTiFSSlQZjJgVy4KOiIiIiIi8gwWdQho7emHsM2FWPBMuiYiIiIjIO1jQKcSWcMkZOiIiIiIi8hYWdAqpNFgLOu6hIyIiIiIib3GooBNC3CCEKBNCVAohHhvjercLIaQQYumI46lCCKMQ4vvuDlitKgxGTAv2Q3SIv6+HQkREREREU8S4BZ0QQgvgWQA3ApgH4B4hxDw71wsD8C0AR+zcza8BvOfeUNWtymDErLhQCCF8PRQiIiIiIpoiHJmhWw6gUkp5TkrZD+A1ALfYud5TAH4BoHf4QSHEZgDnABS5OVbVklKi3NCJWXEMRCEiIiIiIu9xpKBLAlA77HLd4LEhQohcAClSyt0jjocA+DcAPx7rAYQQXxFCHBdCHG9qanJo4GrS0tWPtu4BzIrj/jkiIiIiIvIeRwo6e2sI5dAfhdDAuqTye3au92MAv5ZSGsd6ACnlC1LKpVLKpbGxsQ4MSV2GAlFY0BERERERkRfpHLhOHYCUYZeTATQMuxwGYD6AfYP7xxIA7BJC3AxgBYDbhRC/ABAJwCKE6JVS/l6JwatFxWBBxxk6IiIiIiLyJkcKumMAMoUQGQDqAdwN4F7bH6WU7QBibJeFEPsAfF9KeRzAmmHHfwTAONmKOcAaiBLir8X0iEBfD4WIiIiIiKaQcZdcSilNAL4BIA9ACYA3pJRFQognB2fhprwKQycTLomIiIiIyOscmaGDlPJdAO+OOPbEKNddO8rxHzk5tgmj0mDE6lkTb+8fERERERFNbA41FqfRdfQOQN/Rh8x47p8jIiIiIiLvYkHnJlvC5axYFnRERERERORdLOjcVKkfbFnAGToiIiIiIvIyFnRuqjB0wl+nQfK0YF8PhYiIiIiIphgWdG6qNBgxMzYUWg0TLomIiIiIyLtY0LmpwmBkQ3EiIiIiIvIJFnRu6O43ob6tB5ks6IiIiIiIyAdY0LnhXFMXpARn6IiIiIiIyCdY0LnB1rKAM3REREREROQLLOjcUGHohE4jkBYd4uuhEBERERHRFMSCzg2VBiPSooPhr+PTSERERERE3sdKxA0VBiMy48J8PQwiIiIiIpqiWNC5qN9kQXVLNwNRiIiIiIjIZ1jQuehCSxfMFonMeBZ0RERERETkGyzoXFShtyZczoxlQUdERERERL7Bgs5FlQYjhGBBR0REREREvsOCzkUVhk4kTwtCkL/W10MhIiIiIqIpigWdiyqZcElERERERD7Ggs4FZovEueYuZDLhkoiIiIiIfIgFnQtqL3Wj32TBTBZ0RERERETkQyzoXFBhsCZccoaOiIiIiIh8iQWdCyoMnQDAGToiIiIiIvIpFnQuqDQYkRAeiPBAP18PhYiIiIiIpjAWdC6oNBgxi7NzRERERETkYyzonCSlZEFHRERERESqwILOSQ3tvejuN7OgIyIiIiIin2NB56RKJlwSEREREZFKsKBzUoXemnCZGR/m45EQEREREdFUx4LOSVVNRkSF+CMqxN/XQyEiIiIioimOBZ2TKvQMRCEiIiIiInVgQecEKSUqmHBJREREREQqwYLOCc3GfrT3DDAQhYiIiIiIVIEFnRMqDNZAFM7QERERERGRGrCgc0LVUMsCJlwSEREREZHvsaBzQoXBiNAAHeLDA3w9FCIiIiIiIscKOiHEDUKIMiFEpRDisTGud7sQQgohlg5e3iiEOCGEODP4v+uVGrgvVA4GogghfD0UIiIiIiKi8Qs6IYQWwLMAbgQwD8A9Qoh5dq4XBuBbAI4MO9wM4CYpZQ6ALwH4qxKD9pUKg5GBKEREREREpBqOzNAtB1AppTwnpewH8BqAW+xc7ykAvwDQazsgpSyUUjYMXiwCECiEmJDrFdu7B9DU2cdAFCIiIiIiUg1HCrokALXDLtcNHhsihMgFkCKl3D3G/dwGoFBK2TfyD0KIrwghjgshjjc1NTkwJO+rbLImXGbGs6AjIiIiIiJ1cKSgs7dhTA79UQgNgF8D+N6odyBENoCfA/iqvb9LKV+QUi6VUi6NjY11YEjeV6G3JlzOimXCJRERERERqYMjBV0dgJRhl5MBNAy7HAZgPoB9QogLAFYC2DUsGCUZwFsAviilrFJi0L5QaTAi0E+DpGlBvh4KERERERERAMcKumMAMoUQGUIIfwB3A9hl+6OUsl1KGSOlTJdSpgM4DOBmKeVxIUQkgHcAPC6lPOSB8XtNhcGIGTGh0GqYcElEREREROowbkEnpTQB+AaAPAAlAN6QUhYJIZ4UQtw8zs2/AWAWgP8QQpwc/CfO7VF70c7Ceqx65kPsL2/C+eYu7Cys9/WQiIiIiIiIAABCSjn+tbxo6dKl8vjx474eBgBrMff4jjPoGTAPHQvy0+LpLTnYnJs0xi2JiIiIiIhcI4Q4IaVc6sh1HWosPlVtzyu7rJgDgJ4BM7bnlfloRERERERERJ9hQTeGhrYep44TERERERF5Ewu6MSRG2k+0HO04ERERERGRN7GgG8O2TVkI8tNedizIT4ttm7J8NCIiIiIiIqLP6Hw9ADWzBZ9szytDQ1sPEiODsG1TFgNRiIiIiIhIFVjQjWNzbhILOCIiIiIiUiUuuSQiIiIiIpqgWNARERERERFNUCzoiIiIiIiIJigWdERERERERBMUCzoiIiIi+v/t3W/InXUdx/H3h3uzRibmpiJOXdEeaKArZEj2wEbESsmgQsVARAgkaIH9WT2JIh/4pET0idXIwP5ItZIehGNZGcVMc+rGikxWypb3Rg0TYuX69uD6jY7jtnu1c851X+d+v+BwXb/vuTj398CX+zrf63f9zpE0UDZ0kiRJkjRQqaq+c3iFJIeAP/adxwLWAIf7TkLLhvWmabHWNC3WmqbJetO0TKrWLqqqs0/mwCXX0C1VSR6rqsv7zkPLg/WmabHWNC3WmqbJetO0LIVa85ZLSZIkSRooGzpJkiRJGigbupN3b98JaFmx3jQt1pqmxVrTNFlvmpbea801dJIkSZI0UM7QSZIkSdJA2dBJkiRJ0kDZ0J2EJJuT/C7JM0m29p2PZkuSbUnmk+wZiZ2VZEeS37ftG/rMUbMhyQVJHk6yL8neJFta3HrTWCV5bZJHkzzZau3zLf7GJLtarX0nyWl956rZkGQuyRNJftTG1prGLsn+JE8n2Z3ksRbr/RxqQ7eIJHPAPcB7gEuAG5Jc0m9WmjFfBzafENsK7Kyq9cDONpZO1cvAbVV1MXAF8NH2/8x607gdBTZV1WXABmBzkiuAO4Avt1r7K3BLjzlqtmwB9o2MrTVNyjurasPIb8/1fg61oVvcRuCZqnq2qv4BfBu4tuecNEOq6ufAX04IXwvc1/bvA94/1aQ0k6rqYFX9pu3/je7Dz/lYbxqz6rzUhivbo4BNwHdb3FrTWCRZC1wNfLWNg7Wm6en9HGpDt7jzgedGxs+3mDRJ51bVQeg+hAPn9JyPZkySdcBbgV1Yb5qAdgvcbmAe2AH8AThSVS+3QzyfalzuBD4F/KuNV2OtaTIKeCjJ40k+0mK9n0NXTPsPDlAWiPlbD5IGK8npwPeAj1fVi93FbGm8quoYsCHJmcB24OKFDptuVpo1Sa4B5qvq8SRXHQ8vcKi1pnG4sqoOJDkH2JHkt30nBM7QnYzngQtGxmuBAz3louXjhSTnAbTtfM/5aEYkWUnXzN1fVd9vYetNE1NVR4Cf0q3bPDPJ8YvJnk81DlcC70uyn25ZzCa6GTtrTWNXVQfadp7uQtVGlsA51IZucb8G1rdvSzoNuB54sOecNPseBG5q+zcBP+wxF82Itq7ka8C+qvrSyFPWm8YqydltZo4kq4B30a3ZfBj4YDvMWtMpq6rPVNXaqlpH9xntJ1V1I9aaxizJ65K8/vg+8G5gD0vgHJoqZ6AXk+S9dFd75oBtVXV7zylphiT5FnAVsAZ4Afgc8APgAeBC4E/Ah6rqxC9Okf4nSd4BPAI8zX/WmnyWbh2d9aaxSXIp3ZcDzNFdPH6gqr6Q5E10syhnAU8AH66qo/1lqlnSbrn8RFVdY61p3FpNbW/DFcA3q+r2JKvp+RxqQydJkiRJA+Utl5IkSZI0UDZ0kiRJkjRQNnSSJEmSNFA2dJIkSZI0UDZ0kiRJkjRQNnSSpJmV5FiS3SOPrWN87XVJ9ozr9SRJ+n+s6DsBSZIm6O9VtaHvJCRJmhRn6CRJy06S/UnuSPJoe7y5xS9KsjPJU217YYufm2R7kifb4+3tpeaSfCXJ3iQPJVnV25uSJC1LNnSSpFm26oRbLq8bee7FqtoI3A3c2WJ3A9+oqkuB+4G7Wvwu4GdVdRnwNmBvi68H7qmqtwBHgA9M+P1IkvQKqaq+c5AkaSKSvFRVpy8Q3w9sqqpnk6wE/lxVq5McBs6rqn+2+MGqWpPkELC2qo6OvMY6YEdVrW/jTwMrq+qLk39nkiR1nKGTJC1X9Sr7r3bMQo6O7B/DtemSpCmzoZMkLVfXjWx/1fZ/CVzf9m8EftH2dwK3AiSZS3LGtJKUJOm/8UqiJGmWrUqye2T846o6/tMFr0myi+7i5g0t9jFgW5JPAoeAm1t8C3BvklvoZuJuBQ5OPHtJkhbhGjpJ0rLT1tBdXlWH+85FkqRT4S2XkiRJkjRQztBJkiRJ0kA5QydJkiRJA2VDJ0mSJEkDZUMnSZIkSQNlQydJkiRJA2VDJ0mSJEkD9W+NfpNmXpB64gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fdaf3e42450>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Arrange a small data\n",
    "num_train = 49000\n",
    "small_data_dict = {\n",
    "    \"data_train\": (data[\"data_train\"][:num_train], data[\"labels_train\"][:num_train]),\n",
    "    \"data_val\": (data[\"data_val\"], data[\"labels_val\"]),\n",
    "    \"data_test\": (data[\"data_test\"], data[\"labels_test\"])\n",
    "}\n",
    "\n",
    "# model_sgd      = FullyConnectedNetwork()\n",
    "model_sgdm     = FullyConnectedNetwork()\n",
    "# loss_f_sgd     = cross_entropy()\n",
    "loss_f_sgdm    = cross_entropy()\n",
    "# optimizer_sgd  = SGD(model_sgd.net, 1e-2)\n",
    "optimizer_sgdm = SGDM(model_sgdm.net, 1e-2, 0.9)\n",
    "\n",
    "# print \"Training with Vanilla SGD...\"\n",
    "# results_sgd = train_net(small_data_dict, model_sgd, loss_f_sgd, optimizer_sgd, batch_size=100, \n",
    "#                         max_epochs=5, show_every=100, verbose=True)\n",
    "\n",
    "print \"\\nTraining with SGD plus Momentum...\"\n",
    "results_sgdm = train_net(small_data_dict, model_sgdm, loss_f_sgdm, optimizer_sgdm, batch_size=100, \n",
    "                         max_epochs=50, show_every=100, verbose=True)\n",
    "\n",
    "# opt_params_sgd,  loss_hist_sgd,  train_acc_hist_sgd,  val_acc_hist_sgd  = results_sgd\n",
    "opt_params_sgdm, loss_hist_sgdm, train_acc_hist_sgdm, val_acc_hist_sgdm = results_sgdm\n",
    "\n",
    "# plt.subplot(3, 1, 1)\n",
    "# plt.title('Training loss')\n",
    "# plt.xlabel('Iteration')\n",
    "\n",
    "# plt.subplot(3, 1, 2)\n",
    "# plt.title('Training accuracy')\n",
    "# plt.xlabel('Epoch')\n",
    "\n",
    "# plt.subplot(3, 1, 3)\n",
    "# plt.title('Validation accuracy')\n",
    "# plt.xlabel('Epoch')\n",
    "\n",
    "# plt.subplot(3, 1, 1)\n",
    "# plt.plot(loss_hist_sgd, 'o', label=\"Vanilla SGD\")\n",
    "# plt.subplot(3, 1, 2)\n",
    "# plt.plot(train_acc_hist_sgd, '-o', label=\"Vanilla SGD\")\n",
    "# plt.subplot(3, 1, 3)\n",
    "# plt.plot(val_acc_hist_sgd, '-o', label=\"Vanilla SGD\")\n",
    "         \n",
    "# plt.subplot(3, 1, 1)\n",
    "# plt.plot(loss_hist_sgdm, 'o', label=\"SGD with Momentum\")\n",
    "# plt.subplot(3, 1, 2)\n",
    "# plt.plot(train_acc_hist_sgdm, '-o', label=\"SGD with Momentum\")\n",
    "# plt.subplot(3, 1, 3)\n",
    "# plt.plot(val_acc_hist_sgdm, '-o', label=\"SGD with Momentum\")\n",
    "  \n",
    "# for i in [1, 2, 3]:\n",
    "#   plt.subplot(3, 1, i)\n",
    "#   plt.legend(loc='upper center', ncol=4)\n",
    "# plt.gcf().set_size_inches(15, 15)\n",
    "# plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.5979093711317163\n"
     ]
    }
   ],
   "source": [
    "for i in range(1, 7):\n",
    "    np.savetxt('sgdm_params/fc'+str(i)+'_w.txt', opt_params_sgdm['fc'+str(i)+'_w'], delimiter=\",\")\n",
    "    np.savetxt('sgdm_params/fc'+str(i)+'_b.txt', opt_params_sgdm['fc'+str(i)+'_b'], delimiter=\",\")\n",
    "\n",
    "predictions = model_sgdm.forward(small_data_dict[\"data_test\"][0][0:1000])\n",
    "prediction_loss = loss_f_sgdm.forward(predictions, small_data_dict[\"data_test\"][1][0:1000])\n",
    "np.savetxt('predictions_output.txt', predictions, delimiter=\" \")\n",
    "print prediction_loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## RMSProp\n",
    "The update rule of RMSProp is as shown below: <br\\ >\n",
    "\\begin{equation}\n",
    "\\gamma: decay\\ rate \\\\\n",
    "\\epsilon: small\\ number \\\\\n",
    "g_t^2: squared\\ gradients \\\\\n",
    "\\eta: learning\\ rate \\\\\n",
    "E[g^2]_t: decaying\\ average\\ of\\ past\\ squared\\ gradients\\ at\\ update\\ step\\ t \\\\\n",
    "E[g^2]_t = \\gamma E[g^2]_{t-1} + (1-\\gamma)g_t^2 \\\\\n",
    "\\theta_{t+1} = \\theta_t - \\frac{\\eta}{\\sqrt{E[g^2]_t+\\epsilon}}\n",
    "\\end{equation}\n",
    "Complete the RMSProp() function in optim.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Test RMSProp implementation; you should see errors less than 1e-7\n",
    "N, D = 4, 5\n",
    "test_rms = sequential(fc(N, D, name=\"rms_fc\"))\n",
    "\n",
    "w = np.linspace(-0.4, 0.6, num=N*D).reshape(N, D)\n",
    "dw = np.linspace(-0.6, 0.4, num=N*D).reshape(N, D)\n",
    "cache = np.linspace(0.6, 0.9, num=N*D).reshape(N, D)\n",
    "\n",
    "test_rms.layers[0].params = {\"rms_fc_w\": w}\n",
    "test_rms.layers[0].grads = {\"rms_fc_w\": dw}\n",
    "\n",
    "opt_rms = RMSProp(test_rms, 1e-2, 0.99)\n",
    "opt_rms.cache = {\"rms_fc_w\": cache}\n",
    "opt_rms.step()\n",
    "\n",
    "updated_w = test_rms.layers[0].params[\"rms_fc_w\"]\n",
    "cache = opt_rms.cache[\"rms_fc_w\"]\n",
    "\n",
    "expected_updated_w = np.asarray([\n",
    "  [-0.39223849, -0.34037513, -0.28849239, -0.23659121, -0.18467247],\n",
    "  [-0.132737,   -0.08078555, -0.02881884,  0.02316247,  0.07515774],\n",
    "  [ 0.12716641,  0.17918792,  0.23122175,  0.28326742,  0.33532447],\n",
    "  [ 0.38739248,  0.43947102,  0.49155973,  0.54365823,  0.59576619]])\n",
    "expected_cache = np.asarray([\n",
    "  [ 0.5976,      0.6126277,   0.6277108,   0.64284931,  0.65804321],\n",
    "  [ 0.67329252,  0.68859723,  0.70395734,  0.71937285,  0.73484377],\n",
    "  [ 0.75037008,  0.7659518,   0.78158892,  0.79728144,  0.81302936],\n",
    "  [ 0.82883269,  0.84469141,  0.86060554,  0.87657507,  0.8926    ]])\n",
    "\n",
    "print 'updated_w error: ', rel_error(expected_updated_w, updated_w)\n",
    "print 'cache error: ', rel_error(expected_cache, opt_rms.cache[\"rms_fc_w\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Adam\n",
    "The update rule of Adam is as shown below: <br\\ >\n",
    "\\begin{equation}\n",
    "g_t: gradients\\ at\\ update\\ step\\ t \\\\\n",
    "m_t = \\beta_1m_{t-1} + (1-\\beta_1)g_t \\\\\n",
    "v_t = \\beta_2v_{t-1} + (1-\\beta_1)g_t^2 \\\\\n",
    "\\hat{m_t}: bias\\ corrected\\ m_t \\\\\n",
    "\\hat{v_t}: bias\\ corrected\\ v_t \\\\\n",
    "\\theta_{t+1} = \\theta_t - \\frac{\\eta}{\\sqrt{\\hat{v_t}}+\\epsilon}\n",
    "\\end{equation}\n",
    "Complete the Adam() function in optim.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Test Adam implementation; you should see errors around 1e-7 or less\n",
    "N, D = 4, 5\n",
    "test_adam = sequential(fc(N, D, name=\"adam_fc\"))\n",
    "\n",
    "w = np.linspace(-0.4, 0.6, num=N*D).reshape(N, D)\n",
    "dw = np.linspace(-0.6, 0.4, num=N*D).reshape(N, D)\n",
    "m = np.linspace(0.6, 0.9, num=N*D).reshape(N, D)\n",
    "v = np.linspace(0.7, 0.5, num=N*D).reshape(N, D)\n",
    "\n",
    "test_adam.layers[0].params = {\"adam_fc_w\": w}\n",
    "test_adam.layers[0].grads = {\"adam_fc_w\": dw}\n",
    "\n",
    "opt_adam = Adam(test_adam, 1e-2, 0.9, 0.999, t=5)\n",
    "opt_adam.mt = {\"adam_fc_w\": m}\n",
    "opt_adam.vt = {\"adam_fc_w\": v}\n",
    "opt_adam.step()\n",
    "\n",
    "updated_w = test_adam.layers[0].params[\"adam_fc_w\"]\n",
    "mt = opt_adam.mt[\"adam_fc_w\"]\n",
    "vt = opt_adam.vt[\"adam_fc_w\"]\n",
    "\n",
    "expected_updated_w = np.asarray([\n",
    "  [-0.40094747, -0.34836187, -0.29577703, -0.24319299, -0.19060977],\n",
    "  [-0.1380274,  -0.08544591, -0.03286534,  0.01971428,  0.0722929],\n",
    "  [ 0.1248705,   0.17744702,  0.23002243,  0.28259667,  0.33516969],\n",
    "  [ 0.38774145,  0.44031188,  0.49288093,  0.54544852,  0.59801459]])\n",
    "expected_v = np.asarray([\n",
    "  [ 0.69966,     0.68908382,  0.67851319,  0.66794809,  0.65738853,],\n",
    "  [ 0.64683452,  0.63628604,  0.6257431,   0.61520571,  0.60467385,],\n",
    "  [ 0.59414753,  0.58362676,  0.57311152,  0.56260183,  0.55209767,],\n",
    "  [ 0.54159906,  0.53110598,  0.52061845,  0.51013645,  0.49966,   ]])\n",
    "expected_m = np.asarray([\n",
    "  [ 0.48,        0.49947368,  0.51894737,  0.53842105,  0.55789474],\n",
    "  [ 0.57736842,  0.59684211,  0.61631579,  0.63578947,  0.65526316],\n",
    "  [ 0.67473684,  0.69421053,  0.71368421,  0.73315789,  0.75263158],\n",
    "  [ 0.77210526,  0.79157895,  0.81105263,  0.83052632,  0.85      ]])\n",
    "\n",
    "print 'updated_w error: ', rel_error(expected_updated_w, updated_w)\n",
    "print 'mt error: ', rel_error(expected_m, mt)\n",
    "print 'vt error: ', rel_error(expected_v, vt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Comparing the optimizers\n",
    "Run the following code block to compare the plotted results among all the above optimizers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_rms      = FullyConnectedNetwork()\n",
    "model_adam     = FullyConnectedNetwork()\n",
    "loss_f_rms     = cross_entropy()\n",
    "loss_f_adam    = cross_entropy()\n",
    "optimizer_rms  = RMSProp(model_rms.net, 5e-4)\n",
    "optimizer_adam = Adam(model_adam.net, 5e-4)\n",
    "\n",
    "print \"Training with RMSProp...\"\n",
    "results_rms = train_net(small_data_dict, model_rms, loss_f_rms, optimizer_rms, batch_size=100, \n",
    "                        max_epochs=5, show_every=100, verbose=True)\n",
    "\n",
    "print \"\\nTraining with Adam...\"\n",
    "results_adam = train_net(small_data_dict, model_adam, loss_f_adam, optimizer_adam, batch_size=100, \n",
    "                         max_epochs=5, show_every=100, verbose=True)\n",
    "\n",
    "opt_params_rms,  loss_hist_rms,  train_acc_hist_rms,  val_acc_hist_rms  = results_rms\n",
    "opt_params_adam, loss_hist_adam, train_acc_hist_adam, val_acc_hist_adam = results_adam\n",
    "\n",
    "plt.subplot(3, 1, 1)\n",
    "plt.title('Training loss')\n",
    "plt.xlabel('Iteration')\n",
    "\n",
    "plt.subplot(3, 1, 2)\n",
    "plt.title('Training accuracy')\n",
    "plt.xlabel('Epoch')\n",
    "\n",
    "plt.subplot(3, 1, 3)\n",
    "plt.title('Validation accuracy')\n",
    "plt.xlabel('Epoch')\n",
    "\n",
    "plt.subplot(3, 1, 1)\n",
    "plt.plot(loss_hist_sgd, 'o', label=\"Vanilla SGD\")\n",
    "plt.subplot(3, 1, 2)\n",
    "plt.plot(train_acc_hist_sgd, '-o', label=\"Vanilla SGD\")\n",
    "plt.subplot(3, 1, 3)\n",
    "plt.plot(val_acc_hist_sgd, '-o', label=\"Vanilla SGD\")\n",
    "         \n",
    "plt.subplot(3, 1, 1)\n",
    "plt.plot(loss_hist_sgdm, 'o', label=\"SGD with Momentum\")\n",
    "plt.subplot(3, 1, 2)\n",
    "plt.plot(train_acc_hist_sgdm, '-o', label=\"SGD with Momentum\")\n",
    "plt.subplot(3, 1, 3)\n",
    "plt.plot(val_acc_hist_sgdm, '-o', label=\"SGD with Momentum\")\n",
    "\n",
    "plt.subplot(3, 1, 1)\n",
    "plt.plot(loss_hist_rms, 'o', label=\"RMSProp\")\n",
    "plt.subplot(3, 1, 2)\n",
    "plt.plot(train_acc_hist_rms, '-o', label=\"RMSProp\")\n",
    "plt.subplot(3, 1, 3)\n",
    "plt.plot(val_acc_hist_rms, '-o', label=\"RMSProp\")\n",
    "         \n",
    "plt.subplot(3, 1, 1)\n",
    "plt.plot(loss_hist_adam, 'o', label=\"Adam\")\n",
    "plt.subplot(3, 1, 2)\n",
    "plt.plot(train_acc_hist_adam, '-o', label=\"Adam\")\n",
    "plt.subplot(3, 1, 3)\n",
    "plt.plot(val_acc_hist_adam, '-o', label=\"Adam\")\n",
    "  \n",
    "for i in [1, 2, 3]:\n",
    "  plt.subplot(3, 1, i)\n",
    "  plt.legend(loc='upper center', ncol=4)\n",
    "plt.gcf().set_size_inches(15, 15)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training a Network with Dropout\n",
    "Run the following code blocks to compare the results with and without dropout"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Train two identical nets, one with dropout and one without\n",
    "num_train = 500\n",
    "data_dict_500 = {\n",
    "    \"data_train\": (data[\"data_train\"][:num_train], data[\"labels_train\"][:num_train]),\n",
    "    \"data_val\": (data[\"data_val\"], data[\"labels_val\"]),\n",
    "    \"data_test\": (data[\"data_test\"], data[\"labels_test\"])\n",
    "}\n",
    "\n",
    "solvers = {}\n",
    "dropout_ps = [0, 0.25]  # you can try some dropout prob yourself\n",
    "\n",
    "results_dict = {}\n",
    "for dropout_p in dropout_ps:\n",
    "    results_dict[dropout_p] = {}\n",
    "\n",
    "for dropout_p in dropout_ps:\n",
    "    print \"Dropout =\", dropout_p\n",
    "    model = DropoutNetTest(dropout_p=dropout_p)\n",
    "    loss_f = cross_entropy()\n",
    "    optimizer = SGDM(model.net, 1e-4)\n",
    "    results = train_net(data_dict_500, model, loss_f, optimizer, batch_size=100, \n",
    "                        max_epochs=20, show_every=100, verbose=True)\n",
    "    opt_params, loss_hist, train_acc_hist, val_acc_hist = results\n",
    "    results_dict[dropout_p] = {\n",
    "        \"opt_params\": opt_params, \n",
    "        \"loss_hist\": loss_hist, \n",
    "        \"train_acc_hist\": train_acc_hist, \n",
    "        \"val_acc_hist\": val_acc_hist\n",
    "    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot train and validation accuracies of the two models\n",
    "train_accs = []\n",
    "val_accs = []\n",
    "for dropout_p in dropout_ps:\n",
    "    curr_dict = results_dict[dropout_p]\n",
    "    train_accs.append(curr_dict[\"train_acc_hist\"][-1])\n",
    "    val_accs.append(curr_dict[\"val_acc_hist\"][-1])\n",
    "\n",
    "plt.subplot(3, 1, 1)\n",
    "for dropout_p in dropout_ps:\n",
    "    curr_dict = results_dict[dropout_p]\n",
    "    plt.plot(curr_dict[\"train_acc_hist\"], 'o', label='%.2f dropout' % dropout_p)\n",
    "plt.title('Train accuracy')\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend(ncol=2, loc='lower right')\n",
    "  \n",
    "plt.subplot(3, 1, 2)\n",
    "for dropout_p in dropout_ps:\n",
    "    curr_dict = results_dict[dropout_p]\n",
    "    plt.plot(curr_dict[\"val_acc_hist\"], 'o', label='%.2f dropout' % dropout_p)\n",
    "plt.title('Val accuracy')\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend(ncol=2, loc='lower right')\n",
    "\n",
    "plt.gcf().set_size_inches(15, 15)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Inline Question: Describe what you observe from the above results and graphs\n",
    "#### Ans:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plot the Activation Functions\n",
    "In each of the activation function, use the given lambda function template to plot their corresponding curves."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "left, right = -10, 10\n",
    "X  = np.linspace(left, right, 100)\n",
    "XS = np.linspace(-5, 5, 10)\n",
    "lw = 4\n",
    "alpha = 0.1\n",
    "elu_alpha = 0.5\n",
    "selu_alpha = 1.6732\n",
    "selu_scale = 1.0507\n",
    "\n",
    "#########################\n",
    "####### YOUR CODE #######\n",
    "#########################\n",
    "sigmoid = lambda x: x\n",
    "leaky_relu = lambda x: x\n",
    "relu = lambda x: x\n",
    "elu = lambda x: x\n",
    "selu = lambda x: x\n",
    "tanh = lambda x: x\n",
    "#########################\n",
    "### END OF YOUR CODE ####\n",
    "#########################\n",
    "\n",
    "activations = {\n",
    "    \"Sigmoid\": sigmoid,\n",
    "    \"LeakyReLU\": leaky_relu,\n",
    "    \"ReLU\": relu,\n",
    "    \"ELU\": elu,\n",
    "    \"SeLU\": selu,\n",
    "    \"Tanh\": tanh\n",
    "}\n",
    "\n",
    "# Ground Truth activations\n",
    "GT_Act = {\n",
    "    \"Sigmoid\": [0.00669285092428, 0.0200575365379, 0.0585369028744, 0.158869104881, 0.364576440742, \n",
    "                0.635423559258, 0.841130895119, 0.941463097126, 0.979942463462, 0.993307149076],\n",
    "    \"LeakyReLU\": [-0.5, -0.388888888889, -0.277777777778, -0.166666666667, -0.0555555555556, \n",
    "                  0.555555555556, 1.66666666667, 2.77777777778, 3.88888888889, 5.0],\n",
    "    \"ReLU\": [-0.0, -0.0, -0.0, -0.0, -0.0, 0.555555555556, 1.66666666667, 2.77777777778, 3.88888888889, 5.0],\n",
    "    \"ELU\": [-0.4966310265, -0.489765962143, -0.468911737989, -0.405562198581, -0.213123289631, \n",
    "            0.555555555556, 1.66666666667, 2.77777777778, 3.88888888889, 5.0],\n",
    "    \"SeLU\": [-1.74618571868, -1.72204772347, -1.64872296837, -1.42598202974, -0.749354802287, \n",
    "             0.583722222222, 1.75116666667, 2.91861111111, 4.08605555556, 5.2535],\n",
    "    \"Tanh\": [-0.999909204263, -0.999162466631, -0.992297935288, -0.931109608668, -0.504672397722, \n",
    "             0.504672397722, 0.931109608668, 0.992297935288, 0.999162466631, 0.999909204263]\n",
    "} \n",
    "\n",
    "for label in activations:\n",
    "    fig = plt.figure(figsize=(4,4))\n",
    "    ax = fig.add_subplot(1, 1, 1)\n",
    "    ax.plot(X, activations[label](X), color='darkorchid', lw=lw, label=label)\n",
    "    assert rel_error(activations[label](XS), GT_Act[label]) < 1e-9, \\\n",
    "           \"Your implementation of {} might be wrong\".format(label)\n",
    "    ax.legend(loc=\"lower right\")\n",
    "    ax.axhline(0, color='black')\n",
    "    ax.axvline(0, color='black')\n",
    "    ax.set_title('{}'.format(label), fontsize=14)\n",
    "    plt.xlabel(r\"X\")\n",
    "    plt.ylabel(r\"Y\")\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Phew! You're done for problem 1 now, but 3 more to go... LOL"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}