plot-sagemaker-hpo-obj-value.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook demonstrates how to use SageMaker SDK to plot all the mAP values of two completed HPO jobs that tuned the SSD object detection algorithm.\n",
    "\n",
    "The SSD algorithm permits only one backend per HPO job. Hence, two HPO jobs were run to completion: one with `vgg16` backend, and another with `resnet50` backend.\n",
    "\n",
    "In this particular example, the much simpler `vgg16` backend surprisingly scores higher mAP values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sagemaker as sm\n",
    "\n",
    "sess = sm.Session()\n",
    "\n",
    "# Attach to existing HPO jobs\n",
    "hpo_vgg16 = sm.tuner.HyperparameterTuner.attach('hpo-vgg16', sagemaker_session=sess)\n",
    "hpo_resnet50 = sm.tuner.HyperparameterTuner.attach('hpo-resnet50', sagemaker_session=sess)\n",
    "\n",
    "# We're interested only in the analytics.\n",
    "#\n",
    "# Quote from https://sagemaker.readthedocs.io/en/stable/tuner.html:\n",
    "#     An instance of HyperparameterTuningJobAnalytics for this latest tuning job of this tuner.\n",
    "#     Analytics olbject gives you access to tuning results summarized into a pandas dataframe.\n",
    "analytics_vgg16 = hpo_vgg16.analytics()\n",
    "analytics_resnet50 = hpo_resnet50.analytics()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "See [SageMaker Analytics](https://sagemaker.readthedocs.io/en/stable/analytics.html) on detail API reference. Important methods are `dataframe()` and possibly `export_csv()`.\n",
    "\n",
    "This notebook will want to work with the dataframes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_vgg16 = analytics_vgg16.dataframe()\n",
    "df_resnet50 = analytics_resnet50.dataframe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Display the dataframes. If an HPO job has, say 40 training jobs, than the dataframe will have 40 rows. We'll want the final objective value which contains our mAP values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>FinalObjectiveValue</th>\n",
       "      <th>TrainingElapsedTimeSeconds</th>\n",
       "      <th>TrainingEndTime</th>\n",
       "      <th>TrainingJobName</th>\n",
       "      <th>TrainingJobStatus</th>\n",
       "      <th>TrainingStartTime</th>\n",
       "      <th>learning_rate</th>\n",
       "      <th>mini_batch_size</th>\n",
       "      <th>optimizer</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.996949</td>\n",
       "      <td>1481.0</td>\n",
       "      <td>2019-08-01 09:32:57+00:00</td>\n",
       "      <td>hpo-vgg16-040-d112ffae</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-08-01 09:08:16+00:00</td>\n",
       "      <td>6.160778e-04</td>\n",
       "      <td>5.0</td>\n",
       "      <td>adam</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.894325</td>\n",
       "      <td>1609.0</td>\n",
       "      <td>2019-08-01 09:05:22+00:00</td>\n",
       "      <td>hpo-vgg16-039-e295f827</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-08-01 08:38:33+00:00</td>\n",
       "      <td>8.827578e-04</td>\n",
       "      <td>4.0</td>\n",
       "      <td>adam</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.994909</td>\n",
       "      <td>1711.0</td>\n",
       "      <td>2019-08-01 08:32:06+00:00</td>\n",
       "      <td>hpo-vgg16-038-0f26a12f</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-08-01 08:03:35+00:00</td>\n",
       "      <td>1.000000e-03</td>\n",
       "      <td>3.0</td>\n",
       "      <td>rmsprop</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.999718</td>\n",
       "      <td>1904.0</td>\n",
       "      <td>2019-08-01 08:00:07+00:00</td>\n",
       "      <td>hpo-vgg16-037-5fa021be</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-08-01 07:28:23+00:00</td>\n",
       "      <td>8.707713e-04</td>\n",
       "      <td>2.0</td>\n",
       "      <td>rmsprop</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.965522</td>\n",
       "      <td>1946.0</td>\n",
       "      <td>2019-08-01 07:25:32+00:00</td>\n",
       "      <td>hpo-vgg16-036-149b7c18</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-08-01 06:53:06+00:00</td>\n",
       "      <td>1.472494e-04</td>\n",
       "      <td>2.0</td>\n",
       "      <td>sgd</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.989455</td>\n",
       "      <td>1943.0</td>\n",
       "      <td>2019-08-01 06:50:26+00:00</td>\n",
       "      <td>hpo-vgg16-035-37a69cce</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-08-01 06:18:03+00:00</td>\n",
       "      <td>3.300026e-04</td>\n",
       "      <td>2.0</td>\n",
       "      <td>rmsprop</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.993801</td>\n",
       "      <td>1917.0</td>\n",
       "      <td>2019-08-01 06:15:03+00:00</td>\n",
       "      <td>hpo-vgg16-034-f07a25aa</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-08-01 05:43:06+00:00</td>\n",
       "      <td>8.661307e-04</td>\n",
       "      <td>2.0</td>\n",
       "      <td>rmsprop</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.868692</td>\n",
       "      <td>1618.0</td>\n",
       "      <td>2019-08-01 05:40:25+00:00</td>\n",
       "      <td>hpo-vgg16-033-362dc66d</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-08-01 05:13:27+00:00</td>\n",
       "      <td>7.286375e-04</td>\n",
       "      <td>4.0</td>\n",
       "      <td>sgd</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.554022</td>\n",
       "      <td>2335.0</td>\n",
       "      <td>2019-08-01 05:10:33+00:00</td>\n",
       "      <td>hpo-vgg16-032-169a5129</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-08-01 04:31:38+00:00</td>\n",
       "      <td>1.386004e-07</td>\n",
       "      <td>2.0</td>\n",
       "      <td>adam</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0.970208</td>\n",
       "      <td>1719.0</td>\n",
       "      <td>2019-08-01 04:25:37+00:00</td>\n",
       "      <td>hpo-vgg16-031-c9d7751c</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-08-01 03:56:58+00:00</td>\n",
       "      <td>4.119370e-04</td>\n",
       "      <td>3.0</td>\n",
       "      <td>rmsprop</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0.879229</td>\n",
       "      <td>1344.0</td>\n",
       "      <td>2019-08-01 03:49:47+00:00</td>\n",
       "      <td>hpo-vgg16-030-468d3009</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-08-01 03:27:23+00:00</td>\n",
       "      <td>7.707714e-04</td>\n",
       "      <td>16.0</td>\n",
       "      <td>sgd</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>0.962034</td>\n",
       "      <td>2260.0</td>\n",
       "      <td>2019-08-01 03:22:38+00:00</td>\n",
       "      <td>hpo-vgg16-029-0f44f4ae</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-08-01 02:44:58+00:00</td>\n",
       "      <td>6.515197e-07</td>\n",
       "      <td>2.0</td>\n",
       "      <td>adadelta</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>0.727122</td>\n",
       "      <td>1997.0</td>\n",
       "      <td>2019-08-01 02:36:00+00:00</td>\n",
       "      <td>hpo-vgg16-028-0702ab81</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-08-01 02:02:43+00:00</td>\n",
       "      <td>1.888173e-05</td>\n",
       "      <td>3.0</td>\n",
       "      <td>adadelta</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>0.995995</td>\n",
       "      <td>1492.0</td>\n",
       "      <td>2019-08-01 01:57:15+00:00</td>\n",
       "      <td>hpo-vgg16-027-7c220d1c</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-08-01 01:32:23+00:00</td>\n",
       "      <td>1.000000e-03</td>\n",
       "      <td>5.0</td>\n",
       "      <td>adam</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>0.937932</td>\n",
       "      <td>1950.0</td>\n",
       "      <td>2019-08-01 01:29:59+00:00</td>\n",
       "      <td>hpo-vgg16-026-6e8d4f8e</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-08-01 00:57:29+00:00</td>\n",
       "      <td>4.532468e-04</td>\n",
       "      <td>3.0</td>\n",
       "      <td>adadelta</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>0.999787</td>\n",
       "      <td>1933.0</td>\n",
       "      <td>2019-08-01 00:54:17+00:00</td>\n",
       "      <td>hpo-vgg16-025-e56515d9</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-08-01 00:22:04+00:00</td>\n",
       "      <td>8.112999e-04</td>\n",
       "      <td>2.0</td>\n",
       "      <td>rmsprop</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>0.971635</td>\n",
       "      <td>2237.0</td>\n",
       "      <td>2019-08-01 00:17:11+00:00</td>\n",
       "      <td>hpo-vgg16-024-46ff6696</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-07-31 23:39:54+00:00</td>\n",
       "      <td>2.783369e-06</td>\n",
       "      <td>2.0</td>\n",
       "      <td>adadelta</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>0.860642</td>\n",
       "      <td>1357.0</td>\n",
       "      <td>2019-07-31 23:33:20+00:00</td>\n",
       "      <td>hpo-vgg16-023-4ecc9f6f</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-07-31 23:10:43+00:00</td>\n",
       "      <td>4.975791e-04</td>\n",
       "      <td>11.0</td>\n",
       "      <td>sgd</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>0.491700</td>\n",
       "      <td>2119.0</td>\n",
       "      <td>2019-07-31 23:03:23+00:00</td>\n",
       "      <td>hpo-vgg16-022-b3bd9cc3</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-07-31 22:28:04+00:00</td>\n",
       "      <td>2.662771e-06</td>\n",
       "      <td>3.0</td>\n",
       "      <td>sgd</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>0.928263</td>\n",
       "      <td>2235.0</td>\n",
       "      <td>2019-07-31 22:23:29+00:00</td>\n",
       "      <td>hpo-vgg16-021-1a15079e</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-07-31 21:46:14+00:00</td>\n",
       "      <td>1.000000e-07</td>\n",
       "      <td>2.0</td>\n",
       "      <td>adadelta</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>0.973251</td>\n",
       "      <td>1723.0</td>\n",
       "      <td>2019-07-31 21:40:00+00:00</td>\n",
       "      <td>hpo-vgg16-020-97cc349e</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-07-31 21:11:17+00:00</td>\n",
       "      <td>7.173868e-04</td>\n",
       "      <td>3.0</td>\n",
       "      <td>sgd</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>0.990345</td>\n",
       "      <td>1945.0</td>\n",
       "      <td>2019-07-31 21:08:32+00:00</td>\n",
       "      <td>hpo-vgg16-019-59b89b3d</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-07-31 20:36:07+00:00</td>\n",
       "      <td>4.987543e-04</td>\n",
       "      <td>2.0</td>\n",
       "      <td>sgd</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>0.850995</td>\n",
       "      <td>1331.0</td>\n",
       "      <td>2019-07-31 20:33:19+00:00</td>\n",
       "      <td>hpo-vgg16-018-e6825396</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-07-31 20:11:08+00:00</td>\n",
       "      <td>3.131052e-04</td>\n",
       "      <td>7.0</td>\n",
       "      <td>adam</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>0.937704</td>\n",
       "      <td>2367.0</td>\n",
       "      <td>2019-07-31 20:08:29+00:00</td>\n",
       "      <td>hpo-vgg16-017-34903c42</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-07-31 19:29:02+00:00</td>\n",
       "      <td>1.782630e-04</td>\n",
       "      <td>2.0</td>\n",
       "      <td>adadelta</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>0.974323</td>\n",
       "      <td>2085.0</td>\n",
       "      <td>2019-07-31 19:21:53+00:00</td>\n",
       "      <td>hpo-vgg16-016-492021ff</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-07-31 18:47:08+00:00</td>\n",
       "      <td>3.240865e-05</td>\n",
       "      <td>2.0</td>\n",
       "      <td>sgd</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>0.899255</td>\n",
       "      <td>1660.0</td>\n",
       "      <td>2019-07-31 18:39:37+00:00</td>\n",
       "      <td>hpo-vgg16-015-3cac7c2a</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-07-31 18:11:57+00:00</td>\n",
       "      <td>9.553078e-05</td>\n",
       "      <td>4.0</td>\n",
       "      <td>adam</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>0.981587</td>\n",
       "      <td>1907.0</td>\n",
       "      <td>2019-07-31 18:07:21+00:00</td>\n",
       "      <td>hpo-vgg16-014-29fb96de</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-07-31 17:35:34+00:00</td>\n",
       "      <td>9.948679e-04</td>\n",
       "      <td>2.0</td>\n",
       "      <td>sgd</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>0.998390</td>\n",
       "      <td>1949.0</td>\n",
       "      <td>2019-07-31 17:33:04+00:00</td>\n",
       "      <td>hpo-vgg16-013-70be2058</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-07-31 17:00:35+00:00</td>\n",
       "      <td>8.895727e-04</td>\n",
       "      <td>2.0</td>\n",
       "      <td>rmsprop</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>0.802672</td>\n",
       "      <td>1452.0</td>\n",
       "      <td>2019-07-31 16:55:14+00:00</td>\n",
       "      <td>hpo-vgg16-012-9590cc42</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-07-31 16:31:02+00:00</td>\n",
       "      <td>1.059627e-05</td>\n",
       "      <td>11.0</td>\n",
       "      <td>rmsprop</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>0.833478</td>\n",
       "      <td>1334.0</td>\n",
       "      <td>2019-07-31 16:28:33+00:00</td>\n",
       "      <td>hpo-vgg16-011-152b6e93</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-07-31 16:06:19+00:00</td>\n",
       "      <td>7.733755e-04</td>\n",
       "      <td>12.0</td>\n",
       "      <td>rmsprop</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>0.503813</td>\n",
       "      <td>1723.0</td>\n",
       "      <td>2019-07-31 15:59:11+00:00</td>\n",
       "      <td>hpo-vgg16-010-f7fee935</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-07-31 15:30:28+00:00</td>\n",
       "      <td>1.858163e-06</td>\n",
       "      <td>5.0</td>\n",
       "      <td>rmsprop</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>0.884186</td>\n",
       "      <td>1340.0</td>\n",
       "      <td>2019-07-31 15:23:32+00:00</td>\n",
       "      <td>hpo-vgg16-009-b95a70c2</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-07-31 15:01:12+00:00</td>\n",
       "      <td>2.767307e-04</td>\n",
       "      <td>16.0</td>\n",
       "      <td>rmsprop</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>0.970930</td>\n",
       "      <td>1940.0</td>\n",
       "      <td>2019-07-31 14:57:54+00:00</td>\n",
       "      <td>hpo-vgg16-008-23d1e032</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-07-31 14:25:34+00:00</td>\n",
       "      <td>5.835997e-05</td>\n",
       "      <td>2.0</td>\n",
       "      <td>rmsprop</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>0.972325</td>\n",
       "      <td>1948.0</td>\n",
       "      <td>2019-07-31 14:22:51+00:00</td>\n",
       "      <td>hpo-vgg16-007-d26ab378</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-07-31 13:50:23+00:00</td>\n",
       "      <td>6.399043e-05</td>\n",
       "      <td>2.0</td>\n",
       "      <td>rmsprop</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>0.812450</td>\n",
       "      <td>1368.0</td>\n",
       "      <td>2019-07-31 13:44:02+00:00</td>\n",
       "      <td>hpo-vgg16-006-5c646d49</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-07-31 13:21:14+00:00</td>\n",
       "      <td>2.370264e-04</td>\n",
       "      <td>14.0</td>\n",
       "      <td>adam</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>0.550683</td>\n",
       "      <td>1476.0</td>\n",
       "      <td>2019-07-31 13:16:28+00:00</td>\n",
       "      <td>hpo-vgg16-005-ab85f6cc</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-07-31 12:51:52+00:00</td>\n",
       "      <td>3.564221e-04</td>\n",
       "      <td>8.0</td>\n",
       "      <td>adadelta</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>0.751024</td>\n",
       "      <td>2047.0</td>\n",
       "      <td>2019-07-31 12:43:52+00:00</td>\n",
       "      <td>hpo-vgg16-004-2dc96f2e</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-07-31 12:09:45+00:00</td>\n",
       "      <td>4.312492e-04</td>\n",
       "      <td>2.0</td>\n",
       "      <td>adam</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>0.992321</td>\n",
       "      <td>1483.0</td>\n",
       "      <td>2019-07-31 12:04:56+00:00</td>\n",
       "      <td>hpo-vgg16-003-4ccf8ee7</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-07-31 11:40:13+00:00</td>\n",
       "      <td>3.309315e-04</td>\n",
       "      <td>5.0</td>\n",
       "      <td>adam</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>2019-07-31 11:35:50+00:00</td>\n",
       "      <td>hpo-vgg16-002-75e671a3</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-07-31 11:10:50+00:00</td>\n",
       "      <td>1.000000e-07</td>\n",
       "      <td>11.0</td>\n",
       "      <td>sgd</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>0.795947</td>\n",
       "      <td>1440.0</td>\n",
       "      <td>2019-07-31 11:05:31+00:00</td>\n",
       "      <td>hpo-vgg16-001-89e023da</td>\n",
       "      <td>Completed</td>\n",
       "      <td>2019-07-31 10:41:31+00:00</td>\n",
       "      <td>1.025624e-06</td>\n",
       "      <td>11.0</td>\n",
       "      <td>adam</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    FinalObjectiveValue  TrainingElapsedTimeSeconds           TrainingEndTime  \\\n",
       "0              0.996949                      1481.0 2019-08-01 09:32:57+00:00   \n",
       "1              0.894325                      1609.0 2019-08-01 09:05:22+00:00   \n",
       "2              0.994909                      1711.0 2019-08-01 08:32:06+00:00   \n",
       "3              0.999718                      1904.0 2019-08-01 08:00:07+00:00   \n",
       "4              0.965522                      1946.0 2019-08-01 07:25:32+00:00   \n",
       "5              0.989455                      1943.0 2019-08-01 06:50:26+00:00   \n",
       "6              0.993801                      1917.0 2019-08-01 06:15:03+00:00   \n",
       "7              0.868692                      1618.0 2019-08-01 05:40:25+00:00   \n",
       "8              0.554022                      2335.0 2019-08-01 05:10:33+00:00   \n",
       "9              0.970208                      1719.0 2019-08-01 04:25:37+00:00   \n",
       "10             0.879229                      1344.0 2019-08-01 03:49:47+00:00   \n",
       "11             0.962034                      2260.0 2019-08-01 03:22:38+00:00   \n",
       "12             0.727122                      1997.0 2019-08-01 02:36:00+00:00   \n",
       "13             0.995995                      1492.0 2019-08-01 01:57:15+00:00   \n",
       "14             0.937932                      1950.0 2019-08-01 01:29:59+00:00   \n",
       "15             0.999787                      1933.0 2019-08-01 00:54:17+00:00   \n",
       "16             0.971635                      2237.0 2019-08-01 00:17:11+00:00   \n",
       "17             0.860642                      1357.0 2019-07-31 23:33:20+00:00   \n",
       "18             0.491700                      2119.0 2019-07-31 23:03:23+00:00   \n",
       "19             0.928263                      2235.0 2019-07-31 22:23:29+00:00   \n",
       "20             0.973251                      1723.0 2019-07-31 21:40:00+00:00   \n",
       "21             0.990345                      1945.0 2019-07-31 21:08:32+00:00   \n",
       "22             0.850995                      1331.0 2019-07-31 20:33:19+00:00   \n",
       "23             0.937704                      2367.0 2019-07-31 20:08:29+00:00   \n",
       "24             0.974323                      2085.0 2019-07-31 19:21:53+00:00   \n",
       "25             0.899255                      1660.0 2019-07-31 18:39:37+00:00   \n",
       "26             0.981587                      1907.0 2019-07-31 18:07:21+00:00   \n",
       "27             0.998390                      1949.0 2019-07-31 17:33:04+00:00   \n",
       "28             0.802672                      1452.0 2019-07-31 16:55:14+00:00   \n",
       "29             0.833478                      1334.0 2019-07-31 16:28:33+00:00   \n",
       "30             0.503813                      1723.0 2019-07-31 15:59:11+00:00   \n",
       "31             0.884186                      1340.0 2019-07-31 15:23:32+00:00   \n",
       "32             0.970930                      1940.0 2019-07-31 14:57:54+00:00   \n",
       "33             0.972325                      1948.0 2019-07-31 14:22:51+00:00   \n",
       "34             0.812450                      1368.0 2019-07-31 13:44:02+00:00   \n",
       "35             0.550683                      1476.0 2019-07-31 13:16:28+00:00   \n",
       "36             0.751024                      2047.0 2019-07-31 12:43:52+00:00   \n",
       "37             0.992321                      1483.0 2019-07-31 12:04:56+00:00   \n",
       "38             0.000000                      1500.0 2019-07-31 11:35:50+00:00   \n",
       "39             0.795947                      1440.0 2019-07-31 11:05:31+00:00   \n",
       "\n",
       "                            TrainingJobName TrainingJobStatus  \\\n",
       "0   hpo-vgg16-040-d112ffae         Completed   \n",
       "1   hpo-vgg16-039-e295f827         Completed   \n",
       "2   hpo-vgg16-038-0f26a12f         Completed   \n",
       "3   hpo-vgg16-037-5fa021be         Completed   \n",
       "4   hpo-vgg16-036-149b7c18         Completed   \n",
       "5   hpo-vgg16-035-37a69cce         Completed   \n",
       "6   hpo-vgg16-034-f07a25aa         Completed   \n",
       "7   hpo-vgg16-033-362dc66d         Completed   \n",
       "8   hpo-vgg16-032-169a5129         Completed   \n",
       "9   hpo-vgg16-031-c9d7751c         Completed   \n",
       "10  hpo-vgg16-030-468d3009         Completed   \n",
       "11  hpo-vgg16-029-0f44f4ae         Completed   \n",
       "12  hpo-vgg16-028-0702ab81         Completed   \n",
       "13  hpo-vgg16-027-7c220d1c         Completed   \n",
       "14  hpo-vgg16-026-6e8d4f8e         Completed   \n",
       "15  hpo-vgg16-025-e56515d9         Completed   \n",
       "16  hpo-vgg16-024-46ff6696         Completed   \n",
       "17  hpo-vgg16-023-4ecc9f6f         Completed   \n",
       "18  hpo-vgg16-022-b3bd9cc3         Completed   \n",
       "19  hpo-vgg16-021-1a15079e         Completed   \n",
       "20  hpo-vgg16-020-97cc349e         Completed   \n",
       "21  hpo-vgg16-019-59b89b3d         Completed   \n",
       "22  hpo-vgg16-018-e6825396         Completed   \n",
       "23  hpo-vgg16-017-34903c42         Completed   \n",
       "24  hpo-vgg16-016-492021ff         Completed   \n",
       "25  hpo-vgg16-015-3cac7c2a         Completed   \n",
       "26  hpo-vgg16-014-29fb96de         Completed   \n",
       "27  hpo-vgg16-013-70be2058         Completed   \n",
       "28  hpo-vgg16-012-9590cc42         Completed   \n",
       "29  hpo-vgg16-011-152b6e93         Completed   \n",
       "30  hpo-vgg16-010-f7fee935         Completed   \n",
       "31  hpo-vgg16-009-b95a70c2         Completed   \n",
       "32  hpo-vgg16-008-23d1e032         Completed   \n",
       "33  hpo-vgg16-007-d26ab378         Completed   \n",
       "34  hpo-vgg16-006-5c646d49         Completed   \n",
       "35  hpo-vgg16-005-ab85f6cc         Completed   \n",
       "36  hpo-vgg16-004-2dc96f2e         Completed   \n",
       "37  hpo-vgg16-003-4ccf8ee7         Completed   \n",
       "38  hpo-vgg16-002-75e671a3         Completed   \n",
       "39  hpo-vgg16-001-89e023da         Completed   \n",
       "\n",
       "           TrainingStartTime  learning_rate  mini_batch_size optimizer  \n",
       "0  2019-08-01 09:08:16+00:00   6.160778e-04              5.0      adam  \n",
       "1  2019-08-01 08:38:33+00:00   8.827578e-04              4.0      adam  \n",
       "2  2019-08-01 08:03:35+00:00   1.000000e-03              3.0   rmsprop  \n",
       "3  2019-08-01 07:28:23+00:00   8.707713e-04              2.0   rmsprop  \n",
       "4  2019-08-01 06:53:06+00:00   1.472494e-04              2.0       sgd  \n",
       "5  2019-08-01 06:18:03+00:00   3.300026e-04              2.0   rmsprop  \n",
       "6  2019-08-01 05:43:06+00:00   8.661307e-04              2.0   rmsprop  \n",
       "7  2019-08-01 05:13:27+00:00   7.286375e-04              4.0       sgd  \n",
       "8  2019-08-01 04:31:38+00:00   1.386004e-07              2.0      adam  \n",
       "9  2019-08-01 03:56:58+00:00   4.119370e-04              3.0   rmsprop  \n",
       "10 2019-08-01 03:27:23+00:00   7.707714e-04             16.0       sgd  \n",
       "11 2019-08-01 02:44:58+00:00   6.515197e-07              2.0  adadelta  \n",
       "12 2019-08-01 02:02:43+00:00   1.888173e-05              3.0  adadelta  \n",
       "13 2019-08-01 01:32:23+00:00   1.000000e-03              5.0      adam  \n",
       "14 2019-08-01 00:57:29+00:00   4.532468e-04              3.0  adadelta  \n",
       "15 2019-08-01 00:22:04+00:00   8.112999e-04              2.0   rmsprop  \n",
       "16 2019-07-31 23:39:54+00:00   2.783369e-06              2.0  adadelta  \n",
       "17 2019-07-31 23:10:43+00:00   4.975791e-04             11.0       sgd  \n",
       "18 2019-07-31 22:28:04+00:00   2.662771e-06              3.0       sgd  \n",
       "19 2019-07-31 21:46:14+00:00   1.000000e-07              2.0  adadelta  \n",
       "20 2019-07-31 21:11:17+00:00   7.173868e-04              3.0       sgd  \n",
       "21 2019-07-31 20:36:07+00:00   4.987543e-04              2.0       sgd  \n",
       "22 2019-07-31 20:11:08+00:00   3.131052e-04              7.0      adam  \n",
       "23 2019-07-31 19:29:02+00:00   1.782630e-04              2.0  adadelta  \n",
       "24 2019-07-31 18:47:08+00:00   3.240865e-05              2.0       sgd  \n",
       "25 2019-07-31 18:11:57+00:00   9.553078e-05              4.0      adam  \n",
       "26 2019-07-31 17:35:34+00:00   9.948679e-04              2.0       sgd  \n",
       "27 2019-07-31 17:00:35+00:00   8.895727e-04              2.0   rmsprop  \n",
       "28 2019-07-31 16:31:02+00:00   1.059627e-05             11.0   rmsprop  \n",
       "29 2019-07-31 16:06:19+00:00   7.733755e-04             12.0   rmsprop  \n",
       "30 2019-07-31 15:30:28+00:00   1.858163e-06              5.0   rmsprop  \n",
       "31 2019-07-31 15:01:12+00:00   2.767307e-04             16.0   rmsprop  \n",
       "32 2019-07-31 14:25:34+00:00   5.835997e-05              2.0   rmsprop  \n",
       "33 2019-07-31 13:50:23+00:00   6.399043e-05              2.0   rmsprop  \n",
       "34 2019-07-31 13:21:14+00:00   2.370264e-04             14.0      adam  \n",
       "35 2019-07-31 12:51:52+00:00   3.564221e-04              8.0  adadelta  \n",
       "36 2019-07-31 12:09:45+00:00   4.312492e-04              2.0      adam  \n",
       "37 2019-07-31 11:40:13+00:00   3.309315e-04              5.0      adam  \n",
       "38 2019-07-31 11:10:50+00:00   1.000000e-07             11.0       sgd  \n",
       "39 2019-07-31 10:41:31+00:00   1.025624e-06             11.0      adam  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_vgg16"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl4VOXd//H3mSV7QrbJhB0CIeyCgiIoaABpi7QqYKvi0opW61Zp3StWWqlLraK2KNIHn4rtr1aLVnisCy4oqIgsAWTHQMieyZ7Mes79+2OSgQAhIdsMme/runJNZs6ZmW+G8OHmPveiKaUUQgghznimYBcghBCiY0igCyFENyGBLoQQ3YQEuhBCdBMS6EII0U1IoAshRDchgS6EEN2EBLoQx6mrq2Ps2LHMnz//hGPZ2dmMHj2asWPHMnHiRO6//37q6uqCUKUQJ5JAF+I477//PhEREWzYsIHS0tITjr/44ots2bKFVatWsWPHDpYuXRqEKoU4kQS66Pays7NZvnw5s2bNYsyYMTz44IOUlZUxf/58xo4dyw033EBVVVXg/FWrVvGTn/yErKws/vOf/zT7una7nQsvvJB9+/Z1xY8hRIsk0EVYeP/991mxYgXvvfceH3/8MTfddBMLFizgyy+/xDAMXn31VQDy8/PZuHEjs2bNYtasWbz11lvNvmZhYSHr1q1j2LBhXfVjCHFKlmAXIERXmDdvHqmpqQCMGzeO5ORkhg8fDsD06dP54osvAHj77bfJyspi8ODBxMfH89RTT/Htt98GzgW47bbbMJvNxMfHM2XKFG655Zau/4GEOAkJdBEWGsMcIDIyssn9qKgo6uvrAX+gz507F/B3qYwfP55Vq1Y1CfQ///nPTJw4sYsqF6L1pMtFiAabN28mNzeXZcuWMWnSJCZNmkROTg6rV6/G5/MFuzwhWiQtdCEavPXWW0yaNIknnngi8JjL5eKHP/wh69atIzs7O4jVCdEyaaEL0eDdd99l3rx52Gy2wFffvn350Y9+dMqLo0KECk02uBBCiO5BWuhCCNFNSKALIUQ3IYEuhBDdhAS6EEJ0E0Ebtujz6VRU1Afr7U9LUlKM1NoJpNbOIbV2jlCp1WaLb/ZY0FroFos5WG992qTWziG1dg6ptXOcCbVKl4sQQnQTEuhCCNFNSKALIUQ3IYEuhBDdhAS6EEJ0Ey0G+gMPPMD555/PpZdeetLjSil+//vfM336dGbNmsXOnTs7vEghhBAtazHQr7jiCpYvX97s8XXr1pGbm8v777/P7373O3772992ZH1CCCFaqcWJRePHj+fIkSPNHl+7di2XXXYZmqYxZswYqqurKSkpIS0trUMLFUKIUKKUAl0Hrxd8PvB6wOUCjwfVcIvbBW43uN0otxs8bnB7/LceT8NjHv85Hrf/tQBU4E1o+gDw0p+brandM0WLi4tJT08P3E9PT6e4uLhVgX6qGU+h5nRqPXDgAA8++CA7d+7k7rvv5sYbbwwcq66u5je/+Q179+5F0zQWL17M2LFjg1ZrsEmtnaM716p8PlR9Paq+HqOuHlwu1LFfbvcJ9/F4UB4Pyuv13/d6/fc9XvC4/Y97PODxohru+7/3oLz+xwu9Xv/3Xl/DcU8nfSIt6cRAb4/S0ppgvn2r2Wzxp1Wrrlu47ba7WbfuE2pr3U2e+/vf/5YxY8bz8MOP4fV6cblcHfo5nG6twSS1do5QrlXV1aH270Pt24M6cIAo5cVV6wSfD+XzNbR2G1q8Pp+/5ep0ourrwVkPTqf/sc4WEeH/slrBGgEWC5aEBHTNBFYrmsXScMwKjd9brGhRURARCZFHv7TISIiMgsiIwDEtIsJ/PCIy8F5aZKT/dTStoQityc3Rx5vX7kC32+0UFRUF7hcVFWG329v7skGxdOnzpKXZmT37SgD++teXiIqKorKyjPXrN5CWZsdisTBz5g+5+OJpfPHF5zz//DNERUUzevRZFBTk8+STz5KUlExSUjIbNnze5PVra2vZtm0LDz30WwCsVitWq7Wrf0whOp0yDMg/gtq3F7V3D8bePah9eyAvr8l5La6MYrVCTIz/KzkZLTrW/310NFrDLVFRDYF5bHhGQlTU0eC0RhwX0lY0q/Xo/cZbiz+gtZOEZyj/Q9mo3YGenZ3NypUrmTlzJtu2bSM+Pr5D+s99f3oK44P32v06xzJNn4FlwT3NHp86dTrPPfenQKB//PGHzJt3Azt3bmPlyn9RUVHONdfMZebMH+J2u3nqqT/wwgvL6NWrN4888mCL719YmE9iYiKLFz/K/v17ycoaxl13/Zro6OgO+xmF6ErKMKCwEHVwP+rAftSBA/7b7w74W9PHSkxEO/c8tMwstCFZaIMzSemXTnmNOxCkR1u7lmaDVTSvxUBfsGABGzdupKKigsmTJ3PHHXcEdkC/6qqrmDJlCp9++inTp08nOjqaxYsXd3rRnWXIkKFUVJRTVlZKRUUF8fHx7Nmzm+9973uYTCZSUlI5++xxABw+nEuvXr3p1as3ANOnz+A//1l1ytfXdZ29e/fwy1/ey4gRI3n22T+ycuUr3HTTrZ3+s4nuKXBhTtf9XRS6D93kQZVWge475nEdpev+x7wN3Rk+b8P3/i4O1djd0fhYYz+x1+s/5vV/j9cLtXWo7w6iDu4/MbitVrSBGWiDBqMNGYo2ZAhaZhbYbCcEtMUWjxbird4zSYuB/qc//emUxzVN45FHHumwghpZFtwDp2hNd5aLL57Gxx+vpbzcQXb2JRQU5HfYa9tsadhsaYwYMbLhvaaycuUrHfb6IvQplwtKilHFxaiSYv/3DfcpK/OHpq6jfF7wNQS17guEMj7dH7aNIa7rJ7xH0Unet1NYrWgDBvqDu/ErYxD06evvYxZdTj7142RnT+fJJx+jsrKSF15YRk7OVt5//79ccME0Kisr2LLlG6ZPn0G/fv0pKMinsLCAnj17sXbtBy2+dkpKKmlpdg4fzqVfvwFs2rSRAQMyuuCnEp1J6Trq4AFwlKEqK6GiHFVRARUVUFnR8H05qqwUqqqafyGLxd+Xa7GA2QIWM5jN/u8jo9DMDfctloZba8NtwzkNxyNjo/HoKnD/hNcymwPdGlrgol5E0+4Oq9Xf/9zYr3xsP7O14eJfz14S3CFG/jSOk5ExiPr6Omw2G6mpqVx0UTY7d25h3ry5pKXZGTJkKHFxcURGRrFgwX386ld3EBUVzbBhwwOv4XCUMX/+ddTV1WEyafzrX/9g5crXiY2N4+677+HRRx/G5/PSq1dvHnig4/93I7qO8cUG9GeeQu3dc+oTExIgJRVt6HA0ux3s6Wh2O1qaHdLsaPZ0fx9zB/QZp5wBF+9E59CUUqrl0zrHmfJLFxNjor7eoKqqkptuup6lS/9KSkoq9fX1xMTEoJTi6aefoG/fvvz4x9cEtdYz4Up8ozO5VmP/Pn+Qr/8cNA3TtEvQBmdCYhJaUhIkJUFSMlpiIvRI9LeEg1RrKJNa21ZHc6SF3gq33HIL5eWV+HxebrhhPikpqQC8884q3n13DT6fl8zMLH70o9lBrlR0NlVaiv6X5zHe/jcYBtq552FecA+mocNbfrIQnUxa6K0QKv8yt4bU2jlSYs2UPP0c+v/+DzidaBmDMN/9a7QLJofc0Loz6XOVWttWR3OkhS7ClnK5ULu+RRUcAUP5180wDED5l84wDP9jNdUU/7+VGMUlkJyC+Vf3YbrsCrkgKEKO/EaKsKCUgsOHMHbkoHJyUNu3+S9kNsypaIkWFYXp5lsxX/8ztNjYTq5WiLaRQBfdlqqtxfjn3zG2bEbtyIHKyqMHrVa04SPQRo5Gy8jwD+fTAM3kXzPDZGpYQ0MDsxnb9CmUmyXIRWiTQBfdkrFhPb5FD0PjOkN9+mKaMAlt9Gi0UWehZQ31j7NuJbMtHkKg/1SEL6UUxdVu6UMPVTU1NXzwwX+54oq5gccmTz6XjIzBgH/hsyeeeAaAgoJ8HnnkQaqrq8jKGsbDDy+Shb1OQtXWoj/9JMaqN8BiwfTzX2D+8VVoySnBLk2I06KU4nC5kx0FVezIr2ZHQTWV9V6+fHRGs8+RQD8FpRSdOQiotraGVav+1STQIyMjeeWVv59w7tKlz/PjH1/NtGkzeOqpxaxe/TaXXz6n02o7ExlfbMD36G+gqAhtSBbmRYsxDR0W7LKEaBWlFLmOerbnV7Mjv4qdBdVUu45e40mOsTI5M/WUryGBfpzCwgIWLLid4cNHsmfPbq655jrWrHmL+nonvXr14cEHHyEmJoalS59n/fp1mM1mxo+fwO23/5LHHvstsbGx7N69C4fDwS9+cQcXXzwNgL///W989NGHeL0eJk++mBtv/Dkvvvg8+fn53HDD1Ywffx633XbXSWtSSrF589c88sjvAfj+9y/lf/5nmQR6A1Vbi/7MUxhv/svfKr/5Vsw3/RzN2vouFSGCoTHEP91bxmf7yiipObrWe2pcBBdl2RjZK4FRvRPo2SOqxSGyIRvo/7M+l/X7HR36mpMGp/CzSQNaPO/IkTweeuhR+vTpy0MP3cMrr6ygrk5n5cpX+Oc/X+OKK+aybt3H/P3vb6JpGjU1R/tWy8rK+MtflnPoUC7337+Aiy+exsaNX5KXl8fLL/8vSinuv38BW7du5pZb7uDgwQNNWuQej4cbb7wWs9nMvHk3MHnyRVRVVREXF4+lYZiczZZGaWlJh342Zyrjyw34Hn0YCgvRModg/t1imeQjQl5hlYt1e0v5dF8ZeeX+1SqjrWYuGpLKWX0TGdk7AXt85GnPcQjZQA+m9PSejBw5ivXrPyM39yBXXXUVPp+Bz+dlxIhRxMbGERERyR/+sIhJky5k4sQLA8+dPPkiTCYTAwdmUF5eDsDGjV/y9ddf8tOf+pcFcDrrOXLkMHZ7+gnv/cYb72CzpZGff4S77rqVQYMGExsb1zU/eIhSHg8UF6GKilBFhVBUiCouQuXlob76AsxmaZWLkFde5+GzfWV8ureMfSW1AFjNGhMHJTM508a4AYlEWszteo+QDfSfTRrQqtZ0Z4iKigL8/x0aN+48/vKX50+YIfbyy//LN99s5OOP1/Lmm6/z3HMvAhx3oVIFXmfevBu47LKmSwMUFhac8N42m39zkN69+zB27Dns3bubiy6aSm1tDT6fD4vFQmlpSeC87kQpBQUFGFs3o7ZuRu3c6Q/w8ub/p6ZlDcX8yO8wDR/RhZUKcXr2FdfywKoduH0GJg3O7pfI5MxUJmQkExvZcTEcsoEeCkaMGMWf/vQEhw4dIiYmGafTSWlpCampNtxuF+effwGjRo3hyit/dMrXOe+883n55aVccsn3iYmJobS0BIvFQkxMDPX1Rzfhqq6uJioqioiICCorK9m+fRtXX30dmqYxduw4PvlkLdOmzeDdd1dzwQVTOvvH73RK1zF27/KH95bNGFs2Q0nx0RMiIiC9J9rgwWj2npCejmZPR0tP9z9uT0eLP3M2Qxbhqd6j89T7e/H4DH46sT/ZQ20kxnTO/yQl0E8hKSmJhx76LQsWLMDpdAFw0023EhMTywMPLMDj8aCU4o477j7l65x77gRyc7/jllt+CkB0dAwLF/6O3r37MGrUWVx77ZVMmDCJyZMv4qmnFqNpJpQymDfvegYO9K+Xfuutd/Db3z7Iyy8vJTMzi0svPfU/IqFC1dQ07SYpLPR3nxQWUrhnF6q29ujJySloU6djGnM22tiz/WPFZWimOMMtW3eQwioXs8/uxRVn9+7U95LFuVohVBblaY1g1aqUgkO5GN9sQm3ehNq9yx/kdXXNPseSkYFx1li0MWdjGjMW+vUPuYWuGsnvQOfo7rWu21vGU+/vZbAtlifnjMJqNnVIHc2RFrpoE2UYqP37UN983RDi3zTt646LQ+vdx7+RQ0NXCT17otl7+rtM0uyk9U45Y/4yC3G6iqtd/OWTA0RZTdwzY0iHhHlLJNBFi5RhwJE81N49GHt2+1vfW7dATfXRk2xpmL43E+2cczCdMx4GZoRsa1uIzqYbiqff30edR+euqYPplRjdJe8rgS6aUC4Xat8e1J49qD27UXv3oPbtgWMu3gL+tVGyp6KdPQ7TOeOgdx8JcCEa/PPrPHYV1XBhZgpTh9q67H0l0MNYYJhgzlZUzlbUtm2ovbubLilrNqMNGOi/QDkkCy1rGFpWlqyNIkQzdhZU889NR0iLj+QXFw3q0oaOBHoYUYaBytmG2roZI2cbKmcrlJUdPcFiQRs2HG3ESLShw9CGDEUbNBgtMjJ4RQtxBql1+Xj6g70A/OqSTOI6cIx5a0ighwFVVorx1r/R//0GFOQfPWBLQ5t2CabRZ6GNHuMPcwlvIdpEKcWfPzlAaY2Hq8/ty/CeCV1egwR6N6UMA7XxS/Q3Xkd98pG/GyUq2r912vmTMJ01Bi29Z7DLFKLbWLurhM/3OxjWM54rx/UJSg0S6N2M7nCgr3gV/d+vQ14eANqQLEyzr8T0g0tlZqUQbVBZ78Fb4cRR5cJQyr/9LP5bpRRVTh8vffYdsRFmfj09E7MpOAMEJNC7CWP/PowVyyl6/7/g9UJkJKZZl2Ga+2O0UaNlBIoQp8nt01m/38F/dxazq7B18yXunTGEtISoTq6seRLoZzhj17foy19Crf0AAEtmJuryOZgu/SFaQo8gVyfEmedweT3v7Szmo92l1Lp9aMDoPj3okxKL2+3FpGmYNA1Na9h+tuH7IfZ4LmxhA4rOJoF+hjK2bUV/+UXU5+sA0EaOwnzTLaTNnkVZWW0LzxZCHKuxNf7ezmK+bWiNJ8VYmXtOby4ZYSc9IeqMWKZAAv0MopRCbdroD/KNXwGgnTMO8/xb0Cacj6Zp0rUiRAO3T2fzoUq+LazGqysMpTAM0JXCMPz3daXw6Yrt+dXUuv3zL8b2TeR7I+2cOyAJSxdM1+9IEughTnm9/tmaOVsx/vsuatsWALSJF2CefzOms8cFuUIhQofHZ7D5cCWf7y9j43flOL1Gq56X2NgaH24nvUfw+sDbq1WBvm7dOh577DEMw2Du3LncfPPNTY4XFBRw3333UVNTg67r/PrXv2bKlDN/ve5gUBUV/vDetgW1bStq5w5wuQLHtYsuxjz/FkwjRwWxSiFCh1dvCPF9ZXz1XQVOrw5AekIkM0elMn5gEjFWM2aTv+/bZNIwa2BqvK9BfJQ1aCNTOlKLga7rOosWLWLFihXY7XbmzJlDdnY2gwcPDpyzdOlSvv/973P11Vezf/9+br75Zj766KNOLfxMpzwe1OFDkPsd6ruDqO8OYuzYDocPHT1J09Ayh6CNPgvtrLGYzj7Hv4KhEGGg1uWjtNZNrdtHvUenzu2jzq1T7/FR13C/2uUj50gV9R5/iKfFR/KDUXYuGJzKIFts2HVBthjoOTk59O/fn759+wIwc+ZM1q5d2yTQNU2jtmGjgpqaGtLSut/2aO2hHGUYn3yMyj2Iys1F5R6E/HwwjvvvYFwc2sQL/JN+Ro/xDzeMC+/9REX35tMNCiqdHKnwf+VXOsmvdJFf4aTS6W3Va9jiI5gxws4Fg1PITIsLuxA/VouBXlxcTHr60c2M7XY7OTk5Tc65/fbbufHGG1m5ciVOp5MVK1a06s1PtVB7qGlrrb68PMqu/Ql6wdH9Q00pKVjGnYNl8GAsgwZhGTQI66BBmAf0RzN17gL4oUZq7RyhWGt5rZt9RTXsLaphX1E1+4pqyCuvx6c33WPHpEGvpGiG9+lBr6Ro4qOsxEVZiAvcWoiLtBAfZSU2ykJybESXhXgofq7H6pCLomvWrOHyyy/nZz/7GVu2bOHee+9l9erVmFoIp1AfAtSorcOVVHER3p9dBwUFmG64EdNF2WgDBqIlJgLga/gKcDS/u09n1xoMUmvnCHatSimKq93sK6nlu7I6DpbV8V1pHeX1TVvcMRFmsnomYI+NoE9SNL2ToumdGE2vxKjWbQahDAynhzKnp5N+kqaC/bkeW0dzWgx0u91OUVFR4H5xcTF2u73JOW+88QbLly8HYOzYsbjdbioqKkhJCd8lVlVZKd6bfwb5RzDfchvmW24LdklCdAqfbnCwrI5vC2vYVVjNrsIaKo4L79S4CM4dkMRAWyyDUmMZaIvFHh9JWlpCSIRkd9FioI8aNYrc3Fzy8vKw2+2sWbOGp59+usk5PXv25IsvvuCKK67gwIEDuN1ukpOTO63oUKcqKvD9/EY4lIvpp/Mx/fwXwS5JCAAMpXB6dHyGf/y1bhj4DIVuqGMeU4F1So7XuAWx06uzp6iWbwur2Vtci9t39HpQcoyVSYNSyEqPI8MWx8CUGBKiZbPvrtBioFssFhYuXMj8+fPRdZ3Zs2eTmZnJkiVLGDlyJFOnTuX+++/nN7/5Da+88gqapvH444+H7YUJVV2F75YbUQf2Y7r6Wsx33h22n4XoOLqhKK52Uen0ousKr3FcGOuNgWyAtYzCslqqXT5qXF6qnf7RINUuL3VuH0YHbguvAf2SYxjeK55hPRMY1jMee3yk/M4HiabUyf4d7hpnyn+1Wtt3pmpr/WG+YzumOVdifuiRLv/FDpV+vtaQWptSSlFR7/WP9Kjwj/YoaBj1UVTtQm9jEvvHWVtIiLISH2UhNtKCxaRhNZswmzQsJs1/a2783sSxQ7Ibf4cDD2lgNWkMSotjaHp8uzZxkN+BttXRHJkp2kGUsx7fHbf4w/yHl2F+cKG0UroBpRTldV6qjRpKymrx6QqvYeDV/a1hr67wHXPfZyi8uhFoMXuPOcfjM3B5Ddw+veHWwOXVA7f1Hr1J10WjuEgLg22x9E6MJiUuwh+6ZhPWxiAOBLIJi0mjV1o8yu31h3i0lZgIMyb5XQwLEugdQLlc+O66DbVlM6bv/QDzI7/rkOGHouu5vDoHSuvYU1TD7qIa9hTXUl7XOaMozCaNKIuJSIuJKKuZpNgI0hMiG0Z6RNOnYcTH6fY/h0pLUnQ9CfR2UjU1+O7/FWrjV2jZ0zD/7g9oZnOwyxKt4NMNCqpcgQDfU1TDd476Jl0bSTFWJmQk0zslFp/Hh8Vswmr2t4atDa1iq1lr8v3xxywmExazRmRDeEdazURZTGfcwk8i9Emgt5HyejDeeB192VKoqEC7cAqWJ/6IZpWr+aFGNxRFVS4Ol9dzqLzef+uop6DShe+Y8LaYNAbbYhmaHk9Ww5ctzj9pRVq94kwggX6alFKoD9/H99wzkHcYYmMx3/FLTNfegGaNCHZ5YaOoysWGAw4KqlyBPmt/f3VjP7b/vsurU1jlwnvcbMRoq4lBtlj6JccwIDWGLHs8GbbY1k1oESJESaCfBmPrFvQ/PYXK2QoWC6arrsF8061oYTzmvisVVrn4fH8Z6/c7OFDa8qxaq1kjwmyiX3IM/ZNj6JcSQ7/kaPonx2CToXWiG5JAbwXvgYN4H/19YJs3bdolWO74JVr/AcEtLAwUVDpZv9/B5wccHGwIcbNJ4+x+iUwanMKw9HisFhNW0zH91w3D7ySwRbiRQG+B/spfKXlhCfh8aKPHYF5wD6YxY4NdVreklKKwysXe4lr2ldSyI7+ag2VHQ/yc/olcMCiF8zKSiY+SaxVCHE8C/RSU242+9AVMiYmY7v8N2tTp0urrQCXVLr486GBfQ4DvK6mlzq0HjltMGuP6JzJpcCoTBiYTFyW/rkKcivwNOQW1PQfcbqKvnYd32iXBLqdbKKlxs3ZXCWt3l1Bc7W5yrFePKMb1TyIzLY5MexwZqbFEWWUIqBCtJYF+CmrTRgAiJ55P65baFyfj1Q2+PFjOB9+WsDWvEoV/lMmUoWn0T4wi0x7HYFuctMCFaCf5G3QKxqavQdOIPPdcan0tny+ayi2r44NdJXy8p5Qal/8DHJYez/ThaVwwOJV+vRNlbLcQHUgCvRnK7UblbEUbkoUpKQkkeE7KUIqKOi+OOjelNR7Kat2U1brZWVDDvhL/toSJ0VYuH9uL6cPS6JscE+SKhei+JNCbobbngMeDNu7cYJfSpcrrPBRUOnF6dZweg3qPr+F7PfBYncdHeZ2HsloPjjrPSVcBNGkwrn8SlwxPY/yAJJnmLkQXkEBvhvH1VwCYxnfvQHfUethRUMX2/Gp25FeRX+lq1fNMGiTFRDDYFktqfCS2uAhS4vy3qXGR9Oxx+otKCSHaRwK9Gaqh/1w7+5xgl9KhThXg0VYz4/onMjA1lthIC9FWM9ER5sBtzDH3e0RbpNUtRIiRQD8J5Xajtm9DyxqKltAj2OW0idunk1fuJLesjlyHf1Gq3LJ6Kp1Hx+s0Bvio3j0Y2TuBQbY4zCYZZy/EmUoC/SRUzraQ7z93eXUq6r1U1HuoqPPfltd5cDh97CmoorDKdcJWY2nxkZw3MInhPRMkwIXohiTQT8JoGH9uGjc+yJX4p8PvL6njq+/K2VlQ3RDcXpxevdnnxEVaGNYzgQEpMfRPiWFgSiz9UqKJiZA/biG6M/kbfhJq08aG/vNxQXl/n26wo6CaLw+W89V35ZTV+nfM0YAe0VbsCZEkx0aQGGMlOSaCpFgrSTERJMVYGTEwFdweWaJAiDAkgX4c5XKhcrahDR2GlpDQZe9b79HZfLiCLw+Wsym3gjqPvwUeF2nh4iwbEzKSGds3keiIU0+Ft/WIorRU5rUKEY4k0I+jtm8DrxetC7pb6j0+Nn5Xwef7HWw+XBHYhMEWH0H20DQmZCQzvGe8jCYRQrSKBPpxjK8b+88754JocyHePzmG8wclMyEjmYzUWOkyEUKcNgn046hvvgaTCW1sx4w/V0pRXudhe371SUN80uAULhicIlPihRDtJoF+jPb0n9e5feRXOimodHGk0klBpZP8ShcFlU5cXiNwnoS4EKKzSKAfQ+VsbXX/uVKKA6V1rN1dwoYD5ZTXeU44J8JiondiFL0SoxmYGsvEjGQJcSFEp5FAP4ax6Wvg1P3n5XUePtlTyke7SzlUXg9AQpSFc/on0jsxuuHLH+IpcRGYpC9cCNFFJNCPoTZtPGn/udur89m+MtbuLmHL4UoM5d8ebeKgZKYOTePsfokyEkUIEXQS6A2U04nanoM2dDhafDx1bh9b86r45lAFX35XHtigITMtjqlDbVyYmSqrCQo4WxA9AAAWL0lEQVQhQooEegNj2xYOxaWx5dzL2PLvHewqqgms850aH8klw9PIHppGP+kDF0KEqFYF+rp163jssccwDIO5c+dy8803n3DO//3f//HCCy+gaRpDhw7l6aef7vBiO5puKDbmlvNNbiWbvvXgmP0oAFpBNZn2OM7pl8g5/ZM4f0RPyh21Qa5WCCFOrcVA13WdRYsWsWLFCux2O3PmzCE7O5vBgwcHzsnNzWXZsmX84x//oEePHjgcjk4tuiO4fTqPv7uXTYcqAIj36Vx46CvGXT+bc7J60uOY7hRZkVAIcSZoMdBzcnLo378/ffv2BWDmzJmsXbu2SaC//vrrXHPNNfTo4V87PCUlpZPK7Rj1Hp3fr9nF9vxqxvTtwdVj7AyYMxVLZhbWMXcHuzwhhGiTFgO9uLiY9PT0wH273U5OTk6Tc3JzcwH4yU9+gmEY3H777UyePLnFN7fZ4k+z3Parqvdw36rNfJtfzcXD7Tw6ezTGFxtweL3ETrmAHs3UFIxa20pq7RxSa+eQWjtOh1wU1XWdQ4cO8eqrr1JUVMS8efN45513SGhhtmVpaU1HvH2rVdR7WPj2t+Q66snOsnHnRRlUVdTh+/ATAFzDz8Jzkppstvgur7WtpNbOIbV2Dqm1bXU0p8XB03a7naKiosD94uJi7Hb7CedkZ2djtVrp27cvAwYMCLTaQ0VJjZv739xBrqOemaPSuWva4EDfuNrUseu3CCFEMLQY6KNGjSI3N5e8vDw8Hg9r1qwhOzu7yTnTpk1j40b/KoXl5eXk5uYG+txDQUGlk/ve3E5BlYu55/Tm55MHBmZwKmc9akcO2rARaHFxQa5UCCHarsUuF4vFwsKFC5k/fz66rjN79mwyMzNZsmQJI0eOZOrUqVx44YWsX7+eH/zgB5jNZu69916SkpK6ov4W5ZbV8fB/vqWy3st1E/oxd1yfJsfVtq3g86GND939Q4UQojU0pZRq+bTO0dn9UXuLa3jkP7uodfu4ZfJAZo7uecI5vheWYCx/CcsLL2K64OQXckOl76w1pNbOIbV2Dqm1bXU0p9vOFPXqBo++s4t6j49fTh3M1GFpJz1Pfb0RzGa0MWd3cYVCCNGxuu2KUlVOL9UuHxMHpTQf5s561M7taMOGS/+5EOKM121b6NVO/2JaSTERKKXAWQ8VlajKCqiqRFVUoPbs8vefd9J2c0II0ZW6baBXlVcDEPvPv+H91b/B6232XNOkC7qqLCGE6DTdMtCV00nlsmUwcDrxPida1lBITEJLTDzmNhEtMQl69kIbPiLYJQshRLt1u0BXXi++exdQ5TDBQEj61QKsQ07ehy6EEN1Jt7ooqgwDfeGDqM8+pSbT3+ruERMZ5KqEEKJrdJtAV0qhP7kY4901aGeNpXbyNMC/36cQQoSDbhPo+ot/xvh/f0fLHILl+b9Q7fXPl5Jt4oQQ4aJbBLr+91cxXvoL9OmL5S8voyX0oLphD9B4aaELIcLEGR/o+ur/oD/5B0hNxfricjSbDYBqp5eYCDNW8xn/IwohRKuc0WlnfPox+iMPQXwClqXL0focXeGx2uWT/nMhRFg5IxNP5edjfPYJ+jN/BKsVy/NLMWUOOXpcKaqdXjJssUGsUgghutYZEejK6UR98zXGhvUYGz6H3O/8BywWLEv+jGnM2CbnO70GPkNJC10IEVZCNvHUoVyMTz/G2LAetXkTeDz+A9HRaJMvwjRxEqbJF6H16n3Cc6td/mn+MsJFCBFOQjLQ1eFDeC+/FAwDAG1IFtrECzBNnIQ25my0iIhTPr/a2RDoURLoQojwEZqBnpsLhoHph5dhvuPuwMiV1mocspgQHZI/nhBCdIqQHOWiyh0AaOeMP+0wB2mhCyHCU0gGOo2BnpzcpqdLC10IEY5CMtCVwx/opKS26fnSQhdChKOQDHRpoQshxOkLyUBv7EMnOaVNz5cWuhAiHIVkoFNeDvEJLQ5PbE61y4cGxEVKC10IET5CMtCVwwFt7G4Bfws9LsqC2aR1YFVCCBHaQi7Qla5DZQVaStu6W8A/U1S6W4QQ4SbkAp2qSv8M0Tb2nxtKUePyyQVRIUTYCblAbxyyqLUx0OvcPgwlF0SFEOEn5AI9MGSxjV0u1U4ZsiiECE8hF+iBSUVtHoPuH7LYQ1roQogwE3KBTmOXS1tnicqkIiFEmAq5QFcVDS30pDa20GVSkRAiTLUq0NetW8eMGTOYPn06y5Yta/a89957j6ysLLZv3972ihzt7EOXFroQIky1GOi6rrNo0SKWL1/OmjVrWL16Nfv37z/hvNraWv72t79x1llntasgVV7u/0YW5hJCiNPSYqDn5OTQv39/+vbtS0REBDNnzmTt2rUnnLdkyRJuuukmIiMj21eRwwGRkRAT06anH22hS6ALIcJLi/0SxcXFpKenB+7b7XZycnKanLNz506Kioq46KKL+Otf/9rqN7fZ4k94rKiyHGw20tISWv06x3IZCoCMPonEdWAr/WS1hiqptXNIrZ1Dau047e5oNgyDxx9/nD/84Q+n/dzS0pom95VS6GVlaJlZJxxr9WtWOjGbNOqrnThrXG16jePZbPFtrqerSa2dQ2rtHFJr2+poTotdLna7naKiosD94uJi7HZ74H5dXR179+7luuuuIzs7m61bt3Lrrbe27cJobS14PNCudVx8JERZ0DRZmEsIEV5abKGPGjWK3Nxc8vLysNvtrFmzhqeffjpwPD4+nq+++ipw/9prr+Xee+9l1KhRp19NO2eJgn9iUWpsO/vxhRDiDNRioFssFhYuXMj8+fPRdZ3Zs2eTmZnJkiVLGDlyJFOnTu2wYgIbW7RxDLpPN6hz62SkypBFIUT4aVXyTZkyhSlTpjR57K677jrpua+++mrbq2nnLNEad8MIFxmyKIQIQyE1UzQwBr2t67jIwlxCiDAWWoHuKAPaM0tUJhUJIcJXSAU67d4cWiYVCSHCV0gFemOXS1s3tzjaQpcuFyFE+AmpQMfhAJMJEhPb9HSZ9i+ECGchFeiq3AFJSWimtpV1dGEuaaELIcJPSAU65Y42D1kEaaELIcJbyAS6crv9U//b2H8O0kIXQoS3kAn0wLT/No5BB/9F0QiLiSiruaOqEkKIM0bIBHpgc+j2rOPi9EnrXAgRtkIm0AkMWWxPH7pX+s+FEGErZAJdlftnibZ12r/bp+PyGtJCF0KErZAJ9PZuDl3jkoW5hBDhLWQC/ejCXO2d9i8tdCFEeAqZQG9vC10W5hJChLuQCXRV0bgwlyydK4QQbREygY7DAfEJaNaINj29sYXeQ0a5CCHCVMgEunI42jkGXbpchBDhLSQCXek6VFa0c5Zo4ygX6XIRQoSnkAh0KitAqfat49J4UVS6XIQQYSokAj2wsUV7VlpsuCgaLy10IUSYColAp3Ev0XYuzBUTYcZqDo0fSQghulpIpJ8szCWEEO0XEoFOO/cSVUrJwlxCiLAXEoGuyhsnFbUt0J1eA6+upIUuhAhrIRHogT50mfYvhBBtFhKB3v6FuRqHLEoLXQgRvkIi0Cl3QFQUxMS06enVsnSuEEKERqCrcgckJaNpWpuef7SFLoEuhAhfQQ90pRQ4HG3uP4djWujS5SKECGOtCvR169YxY8YMpk+fzrJly044vmLFCn7wgx8wa9Ysrr/+evLz81tfQU0NeL3QrlmiclFUCCFaDHRd11m0aBHLly9nzZo1rF69mv379zc5Z9iwYbz55pu88847zJgxg6eeeqr1FVQ0jkHvgIW5pIUuhAhjLQZ6Tk4O/fv3p2/fvkRERDBz5kzWrl3b5JwJEyYQHR0NwJgxYygqKmp1AcrRuDm0LJ0rhBDt0WKTtri4mPT09MB9u91OTk5Os+e/8cYbTJ48uVVvbrPF4/Q5KQfi+/Uizhbfqucdz6UrNA0G9knCbGrbhdWW2NpYWzBIrZ1Dau0cUmvH6dA+irfffpsdO3awcuXKVp1fWlqD/t0RAGoj43CW1rTpfctqXMRFWih31Lbp+S2x2eIpbWNtXU1q7RxSa+eQWttWR3NaDHS73d6kC6W4uBi73X7CeRs2bODFF19k5cqVRES0fhs51c5ZogA1Tp90twghwl6LfeijRo0iNzeXvLw8PB4Pa9asITs7u8k53377LQsXLmTp0qWknG4wN1wUJaltF0WNwMJcckFUCBHeWkxBi8XCwoULmT9/PrquM3v2bDIzM1myZAkjR45k6tSpPPnkk9TX13PXXXcB0LNnT1588cVWFdC4dG5bN7eod+sYSi6ICiFEq5q1U6ZMYcqUKU0eawxvgFdeeaXtFZSXg9kMPXq06elHt56TFroQIrwFf6aoowySktBMbSulSoYsCiEEEAKBTrmjzRtbwLELc0kLXQgR3oIa6Mrlgrq6jplUJAtzCSHCXHBb6OWNF0SlhS6EEO0V3BZ648YW7doc2t9C7yEtdCFEmAuNFnpSRyydK4EuhAhvwW2hN4xB74gWunS5CCHCXWi00Nu5dK7ZpBETYe6oqoQQ4owUIi309m1ukRBlafP2dUII0V2ESAu9fX3oMqlICCGC3UJvCHSSk9r0fN1Q1Lp9Mu1fCCEIdgvd4YCEBDRr65fbPVaNS6b9CyFEo+CPQ++Iaf/SQhdCiOAFuvL5oLKifbNEZWEuIYQICFqgG+XloFS7LohWOaWFLoQQjYIX6GWNF0Tb0+UiLXQhhGgUtEDXy0oBWZhLCCE6Sgi00NsxS1SWzhVCiIDgtdBLG1roHTLKRQJdCCGC2EIv83/Tzmn/IF0uQggBIRDo7VuYy0uExUSUVRbmEkKIIF4UbWyht2ccuk9a50II0SC4LfSoKIiOafNr1Li80n8uhBANghfopWWQnNLmZW89PgOn15AWuhBCNAhel4vD0a4x6LIwlxBCNBW8xbk8HlmYSwghOlBQV1ts1xh0WZhLCCGaCO566O3cSxSkhS6EEI2C20KXpXOFEKLDBLeF3o5ZolWyjosQQjQR1P6K1s4S9ekG5fVeymrdlNV4KKt18/WhCkCm/QshRKNWpeG6det47LHHMAyDuXPncvPNNzc57vF4uPfee9m5cyeJiYk888wz9OnTp8XXPf6iaI3Ly/6SOvaX1nKwtI6SGjdltR4q6jyokzw/NsKMPSGyNT+CEEJ0ey0Guq7rLFq0iBUrVmC325kzZw7Z2dkMHjw4cM6//vUvEhIS+OCDD1izZg1//OMfefbZZ0/5urURMRxyR3HgmyOBEC+udjctzqSREhfB8F4J2OIiSImLDNymxkXQKzGKmAhpoQshBLQi0HNycujfvz99+/YFYObMmaxdu7ZJoH/00UfcfvvtAMyYMYNFixahlDrlLNDrr3sOPjocuJ8QZeHsfokMssWSmRbHIFssqfGRmNo4k1QIIcJNi4FeXFxMenp64L7dbicnJ+eEc3r27Ol/QYuF+Ph4KioqSD5FH/l4UxXDJ41laK8EhvZKIL1HVJuXAegKNlt8sEtoNam1c0itnUNq7ThB6694/pErKS2t8d/x+igrqw1WKS2y2eKP1hripNbOIbV2Dqm1bXU0p8Vhi3a7naKiosD94uJi7Hb7CecUFhYC4PP5qKmpISkpqa31CiGEaIMWA33UqFHk5uaSl5eHx+NhzZo1ZGdnNzknOzubVatWAfDee+8xYcKEkO4+EUKI7qjFLheLxcLChQuZP38+uq4ze/ZsMjMzWbJkCSNHjmTq1KnMmTOHe+65h+nTp9OjRw+eeeaZrqhdCCHEMTSl1MmGeHeJUOiPao1Q6TtrDam1c0itnUNqbVsdzQnu1H8hhBAdRgJdCCG6CQl0IYToJiTQhRCimwjqRVEhhBAdR1roQgjRTUigCyFENyGBLoQQ3YQEuhBCdBMS6EII0U1IoAshRDchgS6EEN1EUDa4aGnT6VCSnZ1NbGwsJpMJs9nMv//972CXFPDAAw/wySefkJKSwurVqwGorKzk7rvvJj8/n969e/Pss8/So0ePIFd68lqff/55Xn/99cDOVgsWLGDKlCnBLJPCwkLuvfdeHA4HmqZx5ZVXcv3114fk59pcraH4ubrdbq655ho8Hg+6rjNjxgzuvPNO8vLyWLBgAZWVlYwYMYInn3ySiIiIkKz1/vvvZ+PGjcTH+xfHevzxxxk2bFhQaz2B6mI+n09NnTpVHT58WLndbjVr1iy1b9++ri6j1S6++GLlcDiCXcZJbdy4Ue3YsUPNnDkz8NgTTzyhXnrpJaWUUi+99JJ68skng1VeEyer9bnnnlPLly8PYlUnKi4uVjt27FBKKVVTU6MuueQStW/fvpD8XJurNRQ/V8MwVG1trVJKKY/Ho+bMmaO2bNmi7rzzTrV69WqllFIPP/yweu2114JZplKq+Vrvu+8+9e677wa5ulPr8i6XYzedjoiICGw6LU7f+PHjT2glrl27lssuuwyAyy67jA8//DAYpZ3gZLWGorS0NEaMGAFAXFwcGRkZFBcXh+Tn2lytoUjTNGJjYwH/rmY+nw9N0/jyyy+ZMWMGAJdffnlIZEFztZ4JujzQT7bpdKj+Eja68cYbueKKK/jnP/8Z7FJa5HA4SEtLA8Bms+FwOIJc0am99tprzJo1iwceeICqqqpgl9PEkSNH2LVrF2eddVbIf67H1gqh+bnqus6PfvQjJk6cyMSJE+nbty8JCQlYLP6e3/T09JDJguNrbfxcn3nmGWbNmsXixYvxeDxBrvJEclG0Bf/4xz9YtWoVL7/8Mq+99hpff/11sEtqNU3TQrplcdVVV/HBBx/w9ttvk5aWxuOPPx7skgLq6uq48847efDBB4mLi2tyLNQ+1+NrDdXP1Ww28/bbb/Ppp5+Sk5PDwYMHg11Ss46vde/evSxYsID//ve/vPnmm1RVVbFs2bJgl3mCLg/01mw6HUoaa0tJSWH69Onk5OQEuaJTS0lJoaSkBICSkpLAhbFQlJqaitlsxmQyMXfuXLZv3x7skgDwer3ceeedzJo1i0suuQQI3c/1ZLWG6ufaKCEhgfPOO4+tW7dSXV2Nz+cDoKioKOSyoLHWzz77jLS0NDRNIyIigiuuuCLkPlcIQqC3ZtPpUFFfX09tbW3g+/Xr15OZmRnkqk4tOzubt956C4C33nqLqVOnBrmi5jUGJMCHH34YEp+tUoqHHnqIjIwMfvrTnwYeD8XPtblaQ/FzLS8vp7q6GgCXy8WGDRsYNGgQ5513Hu+99x4Aq1atCoksOFmtGRkZgc9VKRUyn+vxgrJ87qeffsrixYsDm07feuutXV1Cq+Tl5XHbbbcB/j61Sy+9NKRqXbBgARs3bqSiooKUlBTuuOMOpk2bxi9/+UsKCwvp1asXzz77LImJicEu9aS1bty4kd27dwPQu3dvFi1aFOinDpZNmzZxzTXXMGTIEEwmf3tnwYIFjB49OuQ+1+ZqXb16dch9rrt37+b+++9H13WUUnzve9/j9ttvJy8vj7vvvpuqqiqGDRvGH//4x6APW2yu1uuuu46KigqUUgwdOpRHH300cPE0VMh66EII0U3IRVEhhOgmJNCFEKKbkEAXQohuQgJdCCG6CQl0IYToJiTQhRCim5BAF0KIbuL/A985e/wzXXmkAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "sns.set()\n",
    "sns.set_palette(\"Set1\")\n",
    "\n",
    "mAP = pd.concat([df_vgg16['FinalObjectiveValue'].rename('vgg16').sort_values().reset_index(drop=True),\n",
    "                 df_resnet50['FinalObjectiveValue'].rename('resnet50').sort_values().reset_index(drop=True)],\n",
    "                axis=1)\n",
    "mAP.plot(title='mAP');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}