Chap06-Deep_learning_for_text_and_sequences/6.3-advanced-usage-of-recurrent-neurl-networks.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6.3 순환 신경망의 고급 사용법\n",
    "\n",
    "이 절에서는 순환 신경망의 성능과 일반화 능력을 향상시키기 위한 세 가지의 고급 기술을 살펴보겠습니다. 이 절을 마칠 때면 케라스에서 순환 신경망을 사용하는 대부분의 방법을 알게 될 것입니다. \n",
    "\n",
    "온도 예측 문제로 세 가지 개념을 모두 시연해 보겠습니다. \n",
    "- 이 시계열 데이터는 건물 옥상에 설치된 센서에서 취득한 온도, 기압, 습도 같은 데이터입니다. \n",
    "- 이 데이터를 사용하여 마지막 데이터 포인트에서부터 24시간 이후의 온도를 예측하겠습니다. \n",
    "- 이 문제는 시계열 데이터에서 일반적으로 나타나는 여러 가지 어려운 점을 가지고 있습니다. 전형적이고 꽤 도전적인 문제입니다.\n",
    "\n",
    "\n",
    "다음 기법들을 적용하겠습니다.\n",
    "- **순환 드롭아웃(recurrent dropout)** : 순환 층에서 과대적합을 방지하기 위해 케라스에 내장되어 있는 드롭아웃을 사용합니다.\n",
    "- **순환 층 스태킹(stacking recurrent layer)** : 네트워크의 표현 능력을 증가시킵니다(대신 계산 비용이 많이 듭니다).\n",
    "- **양방향 순환 층(bidirectional recurrent layer)** : 순환 네트워크에 같은 정보를 다른 방향으로 주입하여 정확도를 높이고 기억을 좀 더 오래 유지시킵니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.3.1 기온 예측 문제\n",
    "\n",
    "지금까지 다룬 시퀀스 데이터는 IMDB 데이터셋이나 로이터 데이터셋과 같은 텍스트 데이터입니다. 시퀀스 데이터는 이런 언어 처리 분야뿐만 아니라 훨씬 많은 문제에서 등장합니다. \n",
    "\n",
    "이 절에 있는 모든 예제는 날씨 시계열 데이터셋을 사용합니다. \n",
    "- 이 데이터는 독일 예나 시에 있는 막스 플랑크 생물지구화학 연구소( http://www.bgc-jena.mpg.de/wetter/ )의 지상 관측소에서 수집한 것입니다.\n",
    "- 이 데이터셋에는 수년간에 걸쳐 (기온, 기압, 습도, 풍향 등과 같은) 14개의 관측치가 10분마다 기록되어 있습니다. \n",
    "- 원본 데이터는 2003년부터 기록되어 있지만 이 예제에서는 2009~2016년 사이의 데이터만 사용합니다. \n",
    "- 이 데이터셋은 시계열 수치 데이터를 다루는 법을 익히는 데 안성맞춤입니다. \n",
    "\n",
    "최근 데이터(몇 일치 데이터 포인트)를 입력으로 사용해 모델을 만들고 24시간 이후의 기온을 예측하겠습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['\"Date Time\"', '\"p (mbar)\"', '\"T (degC)\"', '\"Tpot (K)\"', '\"Tdew (degC)\"', '\"rh (%)\"', '\"VPmax (mbar)\"', '\"VPact (mbar)\"', '\"VPdef (mbar)\"', '\"sh (g/kg)\"', '\"H2OC (mmol/mol)\"', '\"rho (g/m**3)\"', '\"wv (m/s)\"', '\"max. wv (m/s)\"', '\"wd (deg)\"']\n",
      "420551\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "\n",
    "data_dir = '../datasets/jena_climate/'\n",
    "fname = os.path.join(data_dir, 'jena_climate_2009_2016.csv')\n",
    "\n",
    "with open(fname) as f:\n",
    "    data = f.read()\n",
    "    \n",
    "lines = data.split('\\n')\n",
    "header = lines[0].split(',')\n",
    "lines = lines[1:]\n",
    "\n",
    "print(header)\n",
    "print(len(lines))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 420551 entries, 0 to 420550\n",
      "Data columns (total 15 columns):\n",
      "Date Time          420551 non-null object\n",
      "p (mbar)           420551 non-null float64\n",
      "T (degC)           420551 non-null float64\n",
      "Tpot (K)           420551 non-null float64\n",
      "Tdew (degC)        420551 non-null float64\n",
      "rh (%)             420551 non-null float64\n",
      "VPmax (mbar)       420551 non-null float64\n",
      "VPact (mbar)       420551 non-null float64\n",
      "VPdef (mbar)       420551 non-null float64\n",
      "sh (g/kg)          420551 non-null float64\n",
      "H2OC (mmol/mol)    420551 non-null float64\n",
      "rho (g/m**3)       420551 non-null float64\n",
      "wv (m/s)           420551 non-null float64\n",
      "max. wv (m/s)      420551 non-null float64\n",
      "wd (deg)           420551 non-null float64\n",
      "dtypes: float64(14), object(1)\n",
      "memory usage: 48.1+ MB\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "None"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date Time</th>\n",
       "      <th>p (mbar)</th>\n",
       "      <th>T (degC)</th>\n",
       "      <th>Tpot (K)</th>\n",
       "      <th>Tdew (degC)</th>\n",
       "      <th>rh (%)</th>\n",
       "      <th>VPmax (mbar)</th>\n",
       "      <th>VPact (mbar)</th>\n",
       "      <th>VPdef (mbar)</th>\n",
       "      <th>sh (g/kg)</th>\n",
       "      <th>H2OC (mmol/mol)</th>\n",
       "      <th>rho (g/m**3)</th>\n",
       "      <th>wv (m/s)</th>\n",
       "      <th>max. wv (m/s)</th>\n",
       "      <th>wd (deg)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>01.01.2009 00:10:00</td>\n",
       "      <td>996.52</td>\n",
       "      <td>-8.02</td>\n",
       "      <td>265.40</td>\n",
       "      <td>-8.90</td>\n",
       "      <td>93.3</td>\n",
       "      <td>3.33</td>\n",
       "      <td>3.11</td>\n",
       "      <td>0.22</td>\n",
       "      <td>1.94</td>\n",
       "      <td>3.12</td>\n",
       "      <td>1307.75</td>\n",
       "      <td>1.03</td>\n",
       "      <td>1.75</td>\n",
       "      <td>152.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>01.01.2009 00:20:00</td>\n",
       "      <td>996.57</td>\n",
       "      <td>-8.41</td>\n",
       "      <td>265.01</td>\n",
       "      <td>-9.28</td>\n",
       "      <td>93.4</td>\n",
       "      <td>3.23</td>\n",
       "      <td>3.02</td>\n",
       "      <td>0.21</td>\n",
       "      <td>1.89</td>\n",
       "      <td>3.03</td>\n",
       "      <td>1309.80</td>\n",
       "      <td>0.72</td>\n",
       "      <td>1.50</td>\n",
       "      <td>136.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>01.01.2009 00:30:00</td>\n",
       "      <td>996.53</td>\n",
       "      <td>-8.51</td>\n",
       "      <td>264.91</td>\n",
       "      <td>-9.31</td>\n",
       "      <td>93.9</td>\n",
       "      <td>3.21</td>\n",
       "      <td>3.01</td>\n",
       "      <td>0.20</td>\n",
       "      <td>1.88</td>\n",
       "      <td>3.02</td>\n",
       "      <td>1310.24</td>\n",
       "      <td>0.19</td>\n",
       "      <td>0.63</td>\n",
       "      <td>171.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>01.01.2009 00:40:00</td>\n",
       "      <td>996.51</td>\n",
       "      <td>-8.31</td>\n",
       "      <td>265.12</td>\n",
       "      <td>-9.07</td>\n",
       "      <td>94.2</td>\n",
       "      <td>3.26</td>\n",
       "      <td>3.07</td>\n",
       "      <td>0.19</td>\n",
       "      <td>1.92</td>\n",
       "      <td>3.08</td>\n",
       "      <td>1309.19</td>\n",
       "      <td>0.34</td>\n",
       "      <td>0.50</td>\n",
       "      <td>198.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>01.01.2009 00:50:00</td>\n",
       "      <td>996.51</td>\n",
       "      <td>-8.27</td>\n",
       "      <td>265.15</td>\n",
       "      <td>-9.04</td>\n",
       "      <td>94.1</td>\n",
       "      <td>3.27</td>\n",
       "      <td>3.08</td>\n",
       "      <td>0.19</td>\n",
       "      <td>1.92</td>\n",
       "      <td>3.09</td>\n",
       "      <td>1309.00</td>\n",
       "      <td>0.32</td>\n",
       "      <td>0.63</td>\n",
       "      <td>214.3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             Date Time  p (mbar)  T (degC)  Tpot (K)  Tdew (degC)  rh (%)  \\\n",
       "0  01.01.2009 00:10:00    996.52     -8.02    265.40        -8.90    93.3   \n",
       "1  01.01.2009 00:20:00    996.57     -8.41    265.01        -9.28    93.4   \n",
       "2  01.01.2009 00:30:00    996.53     -8.51    264.91        -9.31    93.9   \n",
       "3  01.01.2009 00:40:00    996.51     -8.31    265.12        -9.07    94.2   \n",
       "4  01.01.2009 00:50:00    996.51     -8.27    265.15        -9.04    94.1   \n",
       "\n",
       "   VPmax (mbar)  VPact (mbar)  VPdef (mbar)  sh (g/kg)  H2OC (mmol/mol)  \\\n",
       "0          3.33          3.11          0.22       1.94             3.12   \n",
       "1          3.23          3.02          0.21       1.89             3.03   \n",
       "2          3.21          3.01          0.20       1.88             3.02   \n",
       "3          3.26          3.07          0.19       1.92             3.08   \n",
       "4          3.27          3.08          0.19       1.92             3.09   \n",
       "\n",
       "   rho (g/m**3)  wv (m/s)  max. wv (m/s)  wd (deg)  \n",
       "0       1307.75      1.03           1.75     152.3  \n",
       "1       1309.80      0.72           1.50     136.1  \n",
       "2       1310.24      0.19           0.63     171.6  \n",
       "3       1309.19      0.34           0.50     198.0  \n",
       "4       1309.00      0.32           0.63     214.3  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv(fname, header=0)\n",
    "display(df.info())\n",
    "display(df.head())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "420,551개 데이터 전체를 넘파이 배열로 바꿉니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(420551, 14)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "float_data = np.zeros((len(lines), len(header) - 1))\n",
    "for i, line in enumerate(lines):\n",
    "    values = [float(x) for x in line.split(',')[1:]]\n",
    "    float_data[i, :] = values\n",
    "    \n",
    "float_data.shape  # == df.values[:, 1:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "예를 들어 다음은 시간에 따른 기온(섭씨) 그래프입니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD8CAYAAAB0IB+mAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJztnXd4FVX6x79vGjUhlFADBhAIRWqkKIqiKAqW9Ye76K5rQV13bbvqKpa1F3SLa11X17qKC1YQBKWqKNJ7CYReAgmdAEkIOb8/7tzk5ua2mTkz59y57+d58mTutPPec2feOfOet5AQAgzDMIz3SVItAMMwDOMOrPAZhmESBFb4DMMwCQIrfIZhmASBFT7DMEyCwAqfYRgmQWCFzzAMkyBIU/hElExEy4hoivG5PREtIKICIppARGmy2mIYhmHMI3OEfzeAdQGfnwfwohDidAAHAYyR2BbDMAxjEpIRaUtE2QDeB/AMgHsAXAagGEBLIUQFEQ0C8LgQ4uJI52nWrJnIycmxLQ/DMEwisWTJkn1CiKxo+6VIau+fAO4HkG58bgrgkBCiwvi8E0CbUAcS0a0AbgWAdu3aYfHixZJEYhiGSQyIaFss+9k26RDRSABFQoglVo4XQrwphMgTQuRlZUV9QDEMwzAWkTHCPxvA5UR0KYC6ADIAvAQgk4hSjFF+NoBdEtpiGIZhLGJ7hC+EeFAIkS2EyAEwGsBsIcSvAcwBMMrY7XoAk+y2xTAMw1jHST/8BwDcQ0QF8Nn033awLYZhGCYKsiZtAQBCiLkA5hrLmwH0l3l+hmEYxjocacswDJMgsMJnGIZJEFjhMwyjNdv3H8cPG4tVi+EJpNrwGYZhZHPuX+cAALaOG6FYkviHR/gMw8QFpSdPqRYh7mGFzzBMXPHOvC3YdeiEajHiElb4DMPEDXuPlOLJKWtx47sLVYsSl7DCZxKKP3y0BB/M36paDMYipyp92X2PllZE2ZMJBSt8JqH4etUePDppjWoxAABHS0/i9bkFqKy0n6I8UeCesgcrfIZRxLNfr8cL0/Px7do9qkVhEgRW+HFESVkF7hi/FPtLylSLwkjgeLnPLFF6slKxJEyiwAo/jpiwaAemrCzEK7MLVIvCSCCJCABQKaHqXKJBqgWIU1jhxxF+W29KEl/u8ci6wiPo+NDXVS6Fhr7HPRNXKJQqfsj9y3Qs3LJftRgAgK9W7MbQv88NOf9yyweLMX7BdgVSRYcVfhxRYVxcW/Ydg4xaxIy7jF+wHacqBWat2wsAIB6nmmbG2r2qRQAA3PvJCmwuPobyU7XNcTPW7sVDX6xSIFV0WOHHERXGxTVrfRHe+mGzYmkYu5AG+n7b/mPo8+S32HHgeNW6nQeP4/CJkwqlCk/gOOfDn7fh0UmrlchRXhGf8y6s8OOIqasKq5ZX7DysUBJGBjpY5iYs2oGDx09i0vLqCqSDn5+D4f/8XqFU4fE/JA8cL8cjX67GB/Njqt3NGLDCj5HvNhRXBX2oYv2eo0rbN8PqXYcx6l8/cf6TCCRpMMT3X9EUJEvh4VL3hYmBg8d8bx7s2WQNVvgxMDe/CNe/sxCvz9HHO0a9qojMo5NWY/G2g1izW/2byNvztuCyV+apFqMKv1kiWMmqYPUu3++zufiYYkliY/5mPSZt/cTbVBor/BgoOurze98WYOdUzYlyvUfOS7cfAgAtbMFPTVmLVbsOo6RMr3B8DfQ9urXKAAB0yGqgWBLGDVjhxymz1hepFiEmdDJDLdl2ULUIAKoVvQb6Xou3DMY9WOGbQKfXt8t6tVYtQkyocj0UQmBTcUmQLHrgv450sOFrIEJI9pWUYcNefQYL4RBB2X1OhnDT1AlW+Iwn+XTJTlzw9+9qlMZT/bwOVq46mQh14/y/zcVFL+rpKRRI8CCQFb6HUDka0v1CCoeqPvNPRm4qKomypzq+31D9MFKVH2mfMT/18+b9+HpVIe4Yv1SJHMF4If1xztipmKOZ6dW2wieiukS0kIhWENEaInrCWN+eiBYQUQERTSCiNPviJi4Hj5erFsESKvT98fIKLNbEXh9I0RGfcl27+0itbTPXqYkgXbrd108/bNyHP3y0FFNWFkY5ggkk2lvjje8tckWOWJExwi8DMFQI0QtAbwDDiWgggOcBvCiEOB3AQQBjJLSlFJU2/Mr4HOA7PsL/eOF2fLywZt6SeyeuwBpDqQb+ZKrTUczJ9432JizeUWubqsnTfSXxOZBQweOT12D66pqprJ/9eh1G/eunqnQZumNb4Qsf/vfmVONPABgK4FNj/fsArrTblip+3qTe93fW+poXlKZzba7z4Oer8ODnNfOWrAkxggb09jVPVqTwdXCbjRfe+2krbvtwSY114xdsx+JtBzHm/cWKpDKHFBs+ESUT0XIARQBmANgE4JAQwm+I2wmgjYy2VPD5Ml/Y+WdLd+KF6euVyFBxquboVPUEZKyo8NIJVGKBrQfOg+wvKUPhYXcLYVdEiNRO4tk0T3Bc8/gYKZeZEOKUEKI3gGwA/QHkxnosEd1KRIuJaHFxcXH0AxTz+txNrra3eOsBTF9dWMs0onIyqOJUZcyBX24NXAOTWQUq/EAVG3gz9nt6JgY9N9sN0aqIlJpDBxfNYG7SzP4cD6zYcUi1CBGROq4QQhwCMAfAIACZRJRibMoGsCvMMW8KIfKEEHlZWVkyxfEEo96Yj9s+rO054Y8anbJyNxZuOeCqTLePX4quj053tc1odH5kGoqP1vZ02XskPqqDJRuZ1F6etRFLtrn7e4ZjtmYeJvGAjg/uQGR46WQRUaaxXA/AMADr4FP8o4zdrgcwyW5biUy4wtt3jF+GX/57vquyfLMm8gRVQYArpJuTkf7CIoG88Z27b2RW8dvw/zFjA/7vX+7+njoTd8n39Nb3SIm+S1RaAXifiJLhe4BMFEJMIaK1AP5HRE8DWAbgbQltMXHA8wHzHDpd/y/N2qhahLBwioPQ9HtqhmoRzKH55JoML52VQog+QoieQogeQognjfWbhRD9hRCnCyGuFkLEx7s1Y4oJi7YjZ+zUqoLcQM2qRE9OWYtVLuXuf2rKWggh0KxhHVfak8kR9pYJyTHNJ0GDCU61oBvsG8BYZsKi7VWT2LPWhbf3XvaqO6mJl2w7iKKjZaiXFn+Xdd20ZNUihGRdYWgXV53QKXOs7mUr4+/OcJE5+UXIGTtVtRja8sBn1f7vd368DAAwe73aAJQXpudrleQuVu76eBkWaJbrHQAueekHV9s7eKzcdIDc0VI93o4Kikr0smGGgBV+ELsOnUDO2KlYtv0gbnxXb7e0fA1SDwd6JQghcNN7agNQPlu6U2n7sXIkhJIKFYHrFHsOl2o3mCk8fAJ9npph2vVZl+f7ze8vwgc/bVUtRkQSXuFXVgq8Ontj1Q04z8iuGByur4JoI51Rb/zkkiTh2bKvOnrVX/RENfEw/7lgc23Xy2+jeD/J5O/f5rvWVqz4yyoGzgHFgi5vdFv3H8ecfL1jiRJe4c9aX4S/fbsBT0xeW2O9Dv7b0UroBkffqkc3efQlVAFzNyty9Wqb6VpbTlOpi8aPAxJe4fsjNE+c9N1s/mvnuw3qn9TRLuQT2vkox8HQWhNUB+ik15Xhka0HrO5jJ+EVfjD//XmbahGqiBSKryPxYErRhaOK6+uWVcRp+tUQVMbZfaKShFf4uw7VrDqkUw3WeHtT3R8h1W7RkVLsMWy0TqOfqas2dxleTap4+4ctStsPhX+8oPuv102ztCJmSGiFX3S0FM9+XTP7ZSyj6pU73ZmcjDfb5C0fhPfQ6f/sLAx8bpYrchRaeLBMXrEbz369TrosF/7jO7z1/Wbp57VLvob1YquijU1e927fJrpnxIxEQiv8Q8et+e9OdMl97lScKfx45q6Pl+FNSYp5X0lZVcnCgqISPOPAg8Rtxry3CG/P0++tANA/ulUnElrhB2LmqV14yB3ThPCOmTWhyHt6Jvo9PVO1GFKZtb4IT01ZG31HBfC4KHYSWuEHzjEu3hp7DdRZLqWN1c2kc8/E5RjwrLcUGWOO6asL8edPVjhybn8+JrNXvV53id4ktMLXkZyxU/H4ZF8qZN0U/udLd9mOT1i2Xb/i4kzs3PbhUnyyxJlo5mvfWgDA/Ihdt/tEZ1jhG+jkUvie5uHZdliyTS+F/2PBPqnn23vEHXOflzlm0mWV9X3ssMI3OFpage9NBFuVe8iP2U108//+9X8WSD1f4HUho/hK8dEyXPf2Ahw8Ft7l1Wts3meu2Px1by/Quhh7/2dmuhpFHYmEVvjBo/rfvrMw5mP/8FHtsoMq2VciPxXE+AXb0euJb01nL2R8jJtW7fJrNYjuP/M244eN+/DxIvW5nZyi8yPT8MRXoSu6xULh4VL8d/5WafIEs2rnYVuFWIqOlmH1LndqQkQj4RT+oePlKKvwe+RYt+PMXKc2DXAwny3ZKb2c30NfrNJ65KQj4Z6NVn38/SkYdHvmykyDXV5RiXd/3GrrHH/7doMcYUJw2avzsN/mG5YuFoGEU/i9n5yBm97TO+0x4Eufa+Yifm7a+hojSl3R8W1hc3FJ9J1iJJxP+NSVhZbOl1QVi2Sv3w5YyDMfCdVpsOONOil6qFo9pHCZHwv0KzQRTM/Hv7WUotmJHOca6mipyCwCH66v9liczPVXULKTLmZzcQn6PjUD71t0BvhJ8sQ2o46EUvgLt1TnIA+swWqVnLFTsXhr7bzmXuNoWQWWbJPzPXUs1n2sTF6ovOxnY5K1bAM12LrfNwlqNQPstWEmtn/YqD6jbLygy5gpoRR+4Eju/k9XSpk5/3ihvTQL5RWV2rvy9XriW/zfv+SMgu2YFY6VVeCBT1dKn1eQGZof7vulJlt80JF/hK+LyqjmurcXolS7FN1MJBJK4QcyZWUh7pm43PZ51uy2N/t+/6crMODZWSirOIU5LkXwxisfzN+GCYt34F8mS+BFo/Sk8xNqJy1m8JSRQdKfNsQJ10AdH0RMeBJW4QPA5mJz/r6h6Noqw9bx3xrl3CpOCbw6p8C2PACqEnfpiB2TTjwoF/kmHfs2Hb8HzKKtB/HVit1Y5FEzZPdHp2tRuEhnElrhy0Bm5aJG9VKlnMdKemC3kOEpUu1WKx+7xTT2HZX7sPXb8O2IFRgDcOfHy3D1G/ImqS9/9Ues2KFHLeNj5afwjxly3DNPnqqUWlhFlyItthU+EbUlojlEtJaI1hDR3cb6JkQ0g4g2Gv8b2xeXiQWdB8J2ZPMHr9j12Y6E3QI4squU+ccTG2zkr1+7+4gkaWpTUFSCp6fqmUXTDp0enoYx78tz3z7pFYUPoALAvUKIbgAGAridiLoBGAtglhCiE4BZxmcmCL99VSAxKsLaeSE6Uqp/EJjs29pvAvOb/swyYdF2lJ/SI+gnFNKroEkc7czJl2ce0uXetq3whRCFQoilxvJRAOsAtAFwBYD3jd3eB3Cl3baY+Efntw/AvseO7O9n12L4wGer5AjiENKDIDV0+wU86pZJRDkA+gBYAKCFEMIfXrgHQAuZbTHh0bkCkB3J3HhY2G1Ddt/LnCNyCrIxfj143JmkcEIIbdIZAPpEmEtT+ETUEMBnAP4ohKhhNBS+bxvyGxPRrUS0mIgWFxcn7gy7zAtCk2tLOl79XpHQX93rhX8C+YVv8tH5kWnaxAmMm7YeQ/8+V7UYchQ+EaXCp+w/EkJ8bqzeS0StjO2tAIR0MhdCvCmEyBNC5GVlZckQx1VkjehOSLwwr3jtR5zU2G5rFZ3fXPzIfijZGeF/6lChknjAn5bkhIWC406MxtfvOSrFDdwuMrx0CMDbANYJIf4RsGkygOuN5esBTLLblo6s2ikn7em9E+WWjdNlZCOTwPvwL1+uVidIBGSrCjsutvc5VIpQJk6/wVj5PYo1jmOxi4wR/tkArgMwlIiWG3+XAhgHYBgRbQRwofHZc2wskpNp8YeN+7BboseCrmNhO5kpA7/Tf3/ehsLDJ+wLJBnZo8N3ftwi9XyOYENry7zmAzllRDZb+j10vXkkkGL3BEKIeQj/k19g9/yJxLpCef7Sutq7v1y+G/8c3cfSsYHJ7wBg0HOzZYhUA/uTtnI4VSmQnBQnFnwNr7WjNtJINKxrWy1qC0faepTAAKDyikptvAR0x/Y8gaRu7vjQ15i2yloOfbdZqFmqhsBrPd9mIJ2TUd0qYIXvUfoaJdkOHCtH50em4a0fNiuWqJob3l2IxydbL2mXKHy9eo9qERyj4lSl9HkrP4E1cf/2bb6tc41f4K3SkqzwbaL7W7c/kvHzpbsUS1LN3PxivGexGIfuyPQk+mrFbmnn0o1Vuw7js6XOeBEFvt0u3W4vz4/XvN1Y4dtE98CYL5b5biq26LhDpbf0Q0Li5XuFFb5NdFb4S7YdwFs/+Lw88m0k30okvHyzM+Z59mv960SbgRW+Tezo+92HnHUrDCzd175ZA0fbssKcfP0KvtjV9/y8iM5PBfuqyi46QTzUTVCFd/2PXMLOCH+WwxWuAkWrk6Lfs/212QU4v0tz1WJIRXZ6ZC8SrkauLKautOfdtP3AcUmS6Id+WiDOCJy0nbJyN56eEntucF2KIqiiU4uGqkWohRX31eKAoid/meRcBHDe0zMdO7cTzM0vwsqd7hdHsRtlvnWfc28fj01abSndgyxY4dskKUDj3zF+Gf4zL/bISNm1WYO57u2Flo6bk19UNdnLRGbexn0485mZ+HaNz4WyWHLFq0D2xVnI/w3vLsLlr/6oWgzTD3Enh2Hvz9+GD+ZvdbCFyLDCt4mdKds9R9wrRXjgWOxpaG98dxH+NMH5PCxeMLWuMEawyzQp88fUZv6m/ab2d/q6PKXwwmeFbxOdqwkFUuTgyNMqk5br52du9Vb0wsPLK/g90/yYvUe9POnLCt8mpSfjQ+HriMyU0LIIda//sLE4bMSlf2I8HlI3Jyr+MpFCiJjs+04rfJXPE1b4ijBjYmHUct3bC/HQF6FLBdqp9uQUQgjXU1cUFOkb5+GfZpuwaAdy/zId2xx0CdUdVvgKWL3rcFWuGyY2HnXQ+yUUy3ccwupdJmodaDTAn7muyPXUFRf+43vsPKinO6Pfhj/2c99De3MULxwPW3RY4dulVaO6po/ZwFGvpvlg/jaXWvLd7Ve+9iNGvjIv6t7VJh19+N9CNQm/Bj8/R0m70Xg9yBsuOUrsjNPmubKTp5TVcmCFb5M9R0pRWSlquH69Fw9FKxjLPPv1OkzWJLHZkdKTuH38UhwyioEfLT2JnQf1Kwyjmn/M2FC1HK3OgNP5kF6eXYBBz83G0dKTzjYUAlb4NmndqB46PPQ1npqyrmrdhMXsw+5l3vx+M+76eJlqMbDncCmembIOU1cW4sUZG1BxqhJnPP6t8rxJJTaKjzjFy7M2Vi1HU/hu2fgnKtATrPBtssvIhxNYii7aNJ6XbYRmOV5egTvGL0WRizEJkTD72/h/a7cLzAghMPC5WZiweAcAX0CPDi7Ca3YfRo/HvtHmDSgUfoX/wfytyP3LNJSUVeAbI3Bu1c7DeHl2gStyqChKxArfATROoFmD3YdOKJ9om7x8N6asLLRdqEIWZm9BVdkxHvhsZa11OmTqWLPbV6bz+w3FiiUJjz//1aOT1qD0ZCXu/3QFfvffJSgoKsFlr0aft5HFrkMnkDN2KlbtNOEcYBPPK/zDJ066/iSNF4V/1rjZtSbatu939wHg/2V0dG+MhVdn+0wFS7YdtHT8/24daOm4UOYAnRK36fwWG2zS8SdLO17urilqtpE8cbyLk+yeVvi7D51Arye+db28Xzwqr237j2HvkVLXMwX6FYP/IfnPmRvMuUM6JE+sHDMSYe09Yi2SeWCHppaOC4XMgU26xULe8XDlHzxWXiNuQNX9quKh6GmF7/dWmLF2r6vtxssIP5Ahf52LAc/OsnyjW8UfbVut8DfWcof8qWCfqzJZQYci8TIH+H8d1VPeyTTjySlrUXi4es7omMsjez8q0jB7WuGrYqWLNjmrBCqoJ75SV1D8qap00uGfkveHsFc7xbyNxY6mx3USmSadobktbB2vc6qJLfuO1eirzcX2f+9bz+1g+diPF25Ht0en25YhFljhJyjv/rg15LLKt5Nwo2Q3B88vzy7AeX+b616DJon0JiHzLSMtJQntmtQ3fdy+El88wHf5+k7aEgEVp2r3lZ3uS0u2p0qPu5QjX4rCJ6J3iKiIiFYHrGtCRDOIaKPxv7GMtqxw4Fh5Vb5yHdBh7PPZUr1iBYj0nugDIqfZdUv09XvC+9jLnrOtn5Zs+hi/3tuvca6olCRCRYjoqlnr3DX9qkDWCP89AMOD1o0FMEsI0QnALOOzEjYVH8Ot/12iqvlavDbHHT/fQC7uXvMV3e8+F0x+BIXiJEkUXmnqkq72mrd+DrvNLRFPRvC1l9VPf764CwDgP9fnmT62eXrkVCMqqz35ISJ8t6H2vNDhE+5HvrqNlBk6IcT3RJQTtPoKAOcZy+8DmAvgARntuUV6nRQcdSBqcIvGNuKfNx9Q0i6BwpokIik5XXDLZn3kRPjrUZbCv/380wEA2Y3Nm3TCmQRLyiow5r1FWLBFzfUVCAFYtr22Gy3ZsGfqPGcRiJM2/BZCCH814T0A7M0CuchVfdpg67gRqGfhlTbekXHhZqXXMX1MSVlF2Jb9dmGdcWuE//PmCGYljXXOzLV7tVD2fsor9B9EOIErk7bCN3QLeTkS0a1EtJiIFhcX6zHR88sz2wKojsizgpOeHh2zGpg+JmZfYwlK4/Pfn2X6mE3FJVorrGhYEf2nsUNNH/NqBHOg04FXuS3To+7jZE3fQPq2y7R8bFlFJZqFGJSkJsehP7VJnFT4e4moFQAY/4tC7SSEeFMIkSeEyMvKypLScMWpSpSUVVj2OEk1Zp7OyG5kWQYnPT38r9xOIMMsYKXficiR1+K7Lugk/ZyyaJ1Zr8bnJ6/obut8Kmul+nl66rroO0nggzEDcFmv1paPb1Qvtda64NKIXsRJhT8ZwPXG8vUAJjnYVg3u/WQFejz2jeXjW2T4nv7ndpbzAJLN0Nzmpo+JVQmbHSSO7NkqRFvmNf6KHYfw+pxN0Xc0yT3DOks/pwzO6lg7wrZ320z8boh1f+5Kh0f4fU+z7mi3R3JyvIZ1UtC+mfk3XT92XFh/ldfW8rGqkeWW+TGA+QC6ENFOIhoDYByAYUS0EcCFxmdX8BfHtvqbtszweRqMPrMt/nRhZyz7yzBZotmmd9tMZNZPw9ZxI/DVHYOln//gcXP28vS6vpHS/cO7VK2z+mL8UkAK23jD7LUW7k0qFt/3cGaT37y9wJwQJmlSP83ScfM37cdb38tLb9LHMOfYMcDUS7Pur/LAJbmYec8QBKbkaWyxb9xGlpfONWE2XSDj/KpITU7C3RfqYxJonl4HX95+dtXnOqmxP69jLRj+w0ZraQwC5zvszH3EK2UmC7IH2tvPaNMIq4z8Qdf2b4eHv4hczvGdMAV2rObziRWrJqP5ESaazbJ13Iiq5V5trZtcv7KRvrlJgzQ0aVCt4MdddQZG9ct2zZxlB09H2h44Zu0GsOOeFcjqXYel+vYGi9W5RTrujtFGPTe/GGUVTvhA11YCLTLq4Mazc2yd9RMjz3u8YNZ9t2mD0J5MsVx7/5or3/QVC1bndwj2fQH+cF5HvPXbmnEBbS24jdplQPsmVcv+32pUv2yk2Iy0dYv4kNIiVivKhKqIM7x7S9PnGfnKPPz6P+GDdcySWa/2a+N1g06L+XgnXNGqsl0GrCMiPDA819Z5//ypL3+O6upJ+0pqDhrspi84p1MzAEDThrV/S9Vzrs/+4gxsee7SsNuzGpp3t5XF/cNzMaxbTc/uHBs2fKvkNHW/TZl4WuGXlJpXFqFm7wGgcYPQ66OxelfoiFYrhBr8mVESTphazjcmkHu3zcR/x/TH36/uJfX8h0zOKfjJCzHB+Mo1ffDyNX3w4ZgBMZ/n4he/r/HZruvj8B6+gUPgaXSxgI3qlx3xDePKPm1clKaaaXefE3J9qoJR9UOXdq21LtQVccNZOY7LYgV3c+G6zBELRYLD++KqvytDvXmYcWWUrVj89tSVj1+EjLo1H4iy2gqV5CoW/jtmAA6d8D0svrz9bGTUTUGHrIYAgMPHY78ugnPCBKbVtYL/oRv4pvDPX/XGG99tQvfWGbbObYc5952HtBRnFKjda0H1m08gDQPSh0f6Wo9f3h3v/bTVcXnM4qkRfklZBd6ZVz2hFSnRlFnCjfzdwG8GuOWcEC57ikf4AGopewBIltRWhcURdb20ZLRq5PNz7902s0rZA/aiiVfsPFS1bKU8ZOcWvuClvu2q30A6ZDXEC6N6KbUD23FxjMbeI6WoiIP0GLEQeFVf0NX3dhvuvqprwqnCLTw1wn/yqzUSKsGH/vHaNvEpj6v6tsHnS3fZbMMcdVJ8KR5CZS/MUPggioQM5TVv4z40S9fL3S3Qg2bj3hLTx/c7rTF+HDsUrRtFTjKmGy/+yrqpbtv+4zhiwbzqx3/v6cZLo/tgX0lZrTfvib8bBADomZ2JhRqlkwA8NsI/ZOJVPRzhBqb+1AR1HHrtjUz4EWnd1Njz/ehiK46V9XuO4KgNReEEgV5XZr1W/C61bTLrSfMEcxq/Lru8VxvLphU7zgJbx42oivXQjbqpySETzPU3PHkGBnj0xELh4RNS5IqEpxS+DK7qG3piyn9/2rUnmvXymHLn4IC6r/aURLzV2j1wrBxz80Nm5NACs9dC77bW87+o4o3f9EP/nCZG+mprF7+TKR/m3Hce5tx3nmPnDyYpxDyaLKasKIy+k008ZdKRMWga0il0OgW/R0CKzQRLb88LHTQTjh5tGlXdZvGlru3z+txNuLK39Xwp4fCbyOyi0VyiLSKl6rioe0tcZMElORCrE++x4OTcg9vslZx+IhSeGuFbnaQNTFFw1unNQu5zRe/W+N2QDrjfhn/51JWFtqLx7D7Q4sSKUIOTDuSHqZeWjPO62M+TFGzSOSCpytOKRy+Scp5YOaON9YjVWLA68R73mLzhgmM+nMBTCn/bfmtV4P0+9m0yw08OpSYn4cFLuob0SInG8XKfHfr28UstydfP8CkBYjLHAAAZQklEQVRv01ifyatBHWon/3ICp7w7WkSpzBQLwZaKvk/NsH1OAGhU3z2b9dS7BseeUdSi3j4VopygF/nmj+fiqYCMp51bNIywd22+XG493UOseEbhl5rMZeLntWv7SpakNnYnk38/pCPm3Hcecluq89O2wrO/OAOf3DbI1jm+WaNzndH4H7l2b90oZHxHKAK/bXrd2K3BVqNTu7WKr+u9S8t0XDcop+rzyJ6twwaNqcIzCt/qa2O7JvVNT779dVRPS22ZxR+Ik5RE0myVa8PUsnWCawe0w5k5TdBVwxv3novsp02Od0uF1VoBzdPrmDJtLt5Wu5xgLJze3NwIWUd0u/Y9o/CtYsXz4Oq8tnhkRO0Qa92ZvHw3Ln35B0xfbd8b4BdhvJlCoePUQYsM+yYdq2+VumCnVsBlIeoghMPq3EacP09r0NxC2U8nSHiFD/iKKQDAQBN26ZtDRb2GQcCacrAyyfr0lT3Cblu/xze631Rsr/zi1nEj8EsTRSBCFfvwAvdMXBHzvgsfjutM4QCATGNu4c8Xd0GmQ/nfT2ta7dcuqyi7LK7pb63wSf7TwzHvAfPlLJ3AU26ZVhACaNwgDbPuHYJshyZFx01bbyn/dqx+82ufvBhLth3EOYZL6SNfhs6n7jdBfLNmj2lZ7DD2klz8dlAOzv3rHFfbdYILcptj1npzsQG/ymuL5hImiVVTJyW5Rj56JwhMydG5efQaum7y3FXWTLl+N+DUZMJJB11UYyHhFb6fjlnO2QvtFFuIhfppKVXKPhL5htvqyp2HHZUnmJTkJLRr6lzu8jaZ9bDrkC9K8eFLu6LcwbwtVjI0OlGr17MEjHHuGOpc7WYV5LbMqCp0o4qEN+kk0q24ZZ89U46uBE7u3XJuh5iLvM974HzM+NO5ptrKjNFl8qs7BuM3A9uZOjdTTd5pjWP2HooXdIiD8YzC16AvpWOnhFsoYi1zGG9YfWhnN66PTi3MmQ1itSufkd0IN53dHoAvDw0TG8O6+oqcjDQxKewGHSR4yemQP8kzJp1j5Xol2bLLPcM647YhHaWeU1YkqG64ORA044rZIauh4zZvrzH2klz8aVhnRUkKw1O/jv10HOsK3XOJDodevWqDVSbs0qufuBgdsvTOwdG9dYZjBSm8xnUDfWUenU4RAACfLrGbfpuJBBGhbmqyFqPhQGQ4DI04Q/1bS0JqlIZ1UpBuuGLarVHqFJqKpSX1jBTRfvdahpGNjOJBt54buyu3U3hG4bOCZJjE5U8X2o+cjkSXlnq5iFrFMwrfa+R4KO2r0+Qa4es3DW6vWBJGFakpzpqALj3DXopoIEG8dIhoOBHlE1EBEY11ur2Y0aH3w7DkkQs9kUfELZo0SMPWcSMwrFsLS8d/fdc5GNLZfrpkp2gVZ+UQVeD0G368FQ8Kh6MKn4iSAbwG4BIA3QBcQ0TdnGjLbBh2isY+vo0dCluXQYMQdXXjnW6tM/D+Tf1VixGSK3q3RuFh5wtjWEGnOZPKOMhkp0OxFqdH+P0BFAghNgshygH8D8AVDrcZEy+N7o0xg9ujV7a8snMdNff8sUrBM5eoFiFhSUtO0jYXkU5DJsdNoBK+rKxKa3ZwWuG3AbAj4PNOY10VRHQrES0mosXFxcWWG9pXYs7HPLtxffxlZDdbNSqDE2JNuVNO7mvdrE0pyUlV2f66u+D6yFTXVu7aKkP69XDbkI74+JaB0s734q96STuXVUb2bIUrJJfDbFQvFecapj7NbknLKJ+0FUK8KYTIE0LkZWVZt6O+86O5WrEyCE6IVU+SuUM3H2QAGGSMMq1mDASAM3MayxLH89w5tBPaZNZzJOJ07CW5Vb+nHfyX6dBca3MnwTx+mXVrLxFVVYaThdU5IZ1xWuHvAhCoIbKNddIpKCpx4rRxSZ921Waq3Jbp2gSZxZJSt2kDfecv3KR9swb4cexQNM+oq+2E4cTbBuG2IR2RYaL6VSRuOFuOl9U1/eXlMPLH6eg4CLOC0wp/EYBORNSeiNIAjAYw2eE2E54v/nB21fJzV52BNAsZHoORcbnHYj3j6OLa6KprcltmYOwludopQ1nBlBRmOZ5x9O4SQlQAuAPANwDWAZgohFjjZJtMTdItFF2PhJ17KZZoRacDaKwSb/VVg0lN9orKCo//G4Z7ALXJNFfvol0T51J6q8Lx4ZQQ4mshRGchREchxDNOt8cEI6SEhcsYxcUix4AOTWy34wSf/f4sZW33z9GzTwK5WaOgNxke1+/ecCZ+f57c5IU6wO/PkrjhrBzVIoQlScKv7FfWdvR+LMemSDA/OYGsCXkr3H7+6XjvxjMtHfvqtX0AOB849MhIe+E1umUVPT+3OVKSk/DCqJ64buBp2rrGmkXPu8sky7YfVC0CHr+8u2oRavDyNb4bvXVmPSk2/IdHdMV1A0/DiDOsu751Dsg9H2pytlPzhqZfuxOBpCTCeV2aWzr2wq7e8zRxk1aN6uGpK3toOxAxiye+xR4TkYhtmzirUK7ul2352E4S0ylc3qs1to4bgfppKXjl2r6Wz+O3/TZpkIanruxha1LVn8YYAOqm1h4xWxlF3XeRnjZ/J1jzxMUx7feK8bAHar5VvXNDHt74jfVrwSk+HDNAzokkTR6/a/FtKh7QJzbaBtF+598MbIcPf94OAJh+t7mSdmZ5/v96Yl7BPlPh8PcM64yGdVIwun9bHCuTX5XKzqhZZvHtxgGj+l5tG1XVofUTyyiqe+sMvHJNHwz9+3cAgDuGdpImn+40CJHKYOFDF6D/s7NqrLusV2t0yGqAsopKpBr2vDuHni7NX14m79/UH4M7NZNyLv8kq/9NsmmDNOy3UPTnfItvU7HQsE4KSsrUFWvyiMKPrPHP6tisSuGHumlkkpREuG1IRzw2OXZnpLsuqFZa9dP0+klm3zdE6vnSUpJQXlEZcgI3y4jmjUTfdo3RwcGC8/FG84zQD+TurasjonWzjwNAchLhVKWQWo9iSOcsfHn72eiV3QiDOjZF0wZp6Pf0zKrtL/6qN3757/nS2rOCal8pT5h0onl/yPBSMcM3a/a42p6TyM7/4f8lQhWoPrdT9Ejri7o7P0p9+Zo+cWH7XvjQBdF30pSzT/eN6mWnPOvdNhNEhM4t0tG0YfUA4qXRvdG/vXpvp5G91Fa98oTCj6TOW2bUDalcnOS3g06LvlOCkxz0EN46bgS6tY7s67513AicE8NDwS6X92qN4T3s5z93mnCje1U8fGnXmPft0sL3luZ0ZPUt5/jcRc/qGJvZ6NPbBjkpDp66ooej54+GXvYDi1RUVobdJiBiMhVY5aObB6BF0I2XIsMPUgMCUzTI4rPfn4WvVuw2bcec/seaien88x5OkQiBSrKJtdB3wzopuH94Li7o2gI9JWarDcXDI7rh4RGxu4zmORzzoNrbxxMK/6MF28NuO1UJ9MpuhBvPzsGNZ8kPDvG/mgaSLim3iErO7ZxV5cMtkx5tGqFHm0aYsXZvxN8tmNyWNUf/gfMeutEiw7kBRjxSNzUJpSerB2XT7j4HqclJGNjBfd/2RvVScfjESdfb1QVPDEVLT4b3bBFCgIjw2GXd0a6pO6HSAxRcyLK5bUgHZEhOyxDIsG4t8JiN7Ig6M+e+8xxvo3uA+WtwiEGHCkIFd3108wDMvKd64r9BWjLaKkxZcCqoUMov+rQJs6c38YTCjxRFqH8dHE1xoeNuDJMdUYeb0E4qCTc8rabcObhq+cObJfmx2yRUl519ejNkN9YnJ03w/NrVedbjZuKR+Lc9ILIfvtnSh4lGbst0rN9ztNZ6t3ot+HUfCO3B4zZtMvWaEA0m3AOpV1tnbeKRuLxXa3yXX4zpEbzU7ru4i4sS1WZkz9Z4fe6mqs9ndWyGreNGoLyiEiIBhofeGOFH0A/ZjTlUPxJdWqaHXO/Wc/Lru87BX0f1rLFOvboH+p2m3oXPCir7rkGdFLxxXb+qz6HuPdXmp26tM7DluUtrrU9LSVJeglBWXYFIeELhR/Kz79oyvtPaqsKt0U6HrIa4Oq9mFa1RNtJT2KFzCw7oksGDl+QC8AVCBSMzctsquuXvX/zIha615UmFP+3uahc+zX7buEGlJWxAh6b46o7B0XeUzKTbB2OJyZvvzYARrRtEc5XV4Xqvb2QWDSVLo/rOOQLEKylJ/ky0zv94nlD4wf3UNaBYha7l4XQhnGJXbc1UobjqpSXXiM6Mha5BhVFkJsALxUc3D8C8B853tA2nSHc4rUm88G+XBwmBeOIXcDt1gpcIp9hl5jjxMsEuhrILaQdTPy1Fu3xLsfDxLQOR00wfbx2VXNxdXRR3/F05IWB9Lx/V+l6nh/j4W2J3e1QRTBSIPr1Wk0EeKSDiBI3qpeLmwe1xVV/n5668YdKJtE3XO0ATwo3kVbuo6eCa6cdMANrpDpt0oqHbhCQTmm/+WJ2mnYjwyMhuUXNJycATCj/SaJCv/8iEN+m4KkYtdPzd6isscxiPqL6GdCacO7TTeELhz1pfpFoE7Rl9ZtuQ68PpVdU3q076PrhQCxMFHZ/WDACPKHwmOuHuwTPaNKq17ld5bTGki/NpiCOhk844YFRN0kiksOggY2Y9nwnMrMcT4zyemLRt16Q+th84HmarDreADoTuh/ohXOWeD4p8VYEOtmgi35tOcHRo5OtNLe0UJibzM7JnK5RVVOLyXtYL3jPO4IkRfqjc5TeenQNAr5FiKNyK7gzXD7p2jw5ytTCiQlOCrq+pd4UPClNlCrt5sC8RnSrbcCBEhFH9sm0VvE8EVjx6EVY8epGrbdr6RYjoaiJaQ0SVRJQXtO1BIiogonwiutiemJEJ5dHRUfO6p/6AnWv6t3OlvXBOL7o+EDPq6RORqXo+www6ubMykWlUP9X1yGO7j+DVAK4C8H3gSiLqBmA0gO4AhgN4nYgcc3GIp4v8zBxnA3PCES7iWNdI5GYa2n91MDOFw5/mXWMRGQ2wpfCFEOuEEPkhNl0B4H9CiDIhxBYABQD622krEqFeHf2DMt2uf1UK9quVu0Oub2WkAb6e6/DGTKQBv5PlNCOhOm6CiQ+cMrK1AbAj4PNOY10tiOhWIlpMRIuLi4stNfbatX1rrzTew3Ud8biduuDQ8dpl3a7ul43zuzTH+JsH4LHLursqTzwQfO34zWJCAC+N7o0/B+V2vyC3OVo2UpMNcli3FgA4opWJTFQvHSKaCSBU8oeHhRCT7AoghHgTwJsAkJeXZ0kLqiyZZhqNHkD+erxnaVIiT1f8F+XE2wbhy2W7kVE3BVf0rj1+UVl7wV/Ig2EiEVXhCyGsJGveBSAw0ifbWMdoBFcDM0duywyMvaR2+Ht6nRQcLavAzed0UCAVw8SOUyadyQBGE1EdImoPoBOAhQ61VYM2mcYoy3gf121CVydpWN9HpvBwKQCg9OSpiPvVSfXdRnVTOfUCoze2Aq+I6BcAXgGQBWAqES0XQlwshFhDRBMBrAVQAeB2IUTku0YS/jD4q/tlI3/PEdw7TG0NzWioVLqs72Njf0k5OqoNPGYYKdhS+EKILwB8EWbbMwCesXN+O9RNTcbTV56hqvm4gHPex0azhmmqRWAYKXguFE6H0PJY+KVRx9XvXaEC1veR8U/CpibHdpuwaySjO57IpROI7ils/VMKua3SlXtV8KRtZHJbpmPnwRNVNvrw+P01HReJYWzhOYWvczSkStpk1quV5pf1U2T+OboPVu08jObpanzrGUY2njPpxI26d1nbhnoOnsVBOhFpWCeFA5kYT+E5ha87qlIrhHJP5WyGcuCXSiZe4DveZbbtPwYAKKuodLXdUNkyK9mmIwV/7IdOdXgZJhQetOGrliAyu41gnoVbD+D83OautRtqhM9umXJ4+/o8zN+8nys8MdrjuRG+7lV2OmY1AOC+/TxUFCjrezk0bVgHI3vqfd0xNXnqyh7KUpWrxHMKv3WmugRWsdAiw+fx4XbKh1vO9VVEat+sQdU6VvhMonLdwNPwyW1nqRbDdTyn8HU36YwxStHlulyKrnF9X7RoduN6aNvE91AMDhRq0oAjShnGy3jPhq+ZY+ZXdwzGZa/Oq/p8QdcWSgOuiKiqj4JH+DPvGYKDx8sVSMUwjBvwCN9herTJwL3DOqsWowohRHUhj6BtTRqkaV8LmGEY63hP4asWIAgiwp0XdFItRo0I5PS6vsLJuvUVwzDO4j2TDmuxkLTI8LkM9sxuhN8MPA2Tl+/GaU3jI9EcwzBy8JzC73daE9UiaEluywxMvWswcltmIDmJ8LshHVWLxDCMy3hO4UfPbJi4dG/dSLUIDMMohLUjwzBMguA5hc8mfIZhmNB4TuH7PVAYhmGYmnhO4etCA80rbzEMk3h4btJWF1Y+fjHKA1Igv/GbvpibX6xQIoZhEh1W+A6RnESoFzDKH96jFYb3aKVQIoZhEh026TAMwyQIthQ+Ef2ViNYT0Uoi+oKIMgO2PUhEBUSUT0QX2xeVYRiGsYPdEf4MAD2EED0BbADwIAAQUTcAowF0BzAcwOtExLOYDMMwCrGl8IUQ3wohKoyPPwPINpavAPA/IUSZEGILgAIA/e20xTAMw9hDpg3/JgDTjOU2AHYEbNtprGMY0zTjWrEMI4WoXjpENBNAyxCbHhZCTDL2eRhABYCPzApARLcCuBUA2rVrZ/ZwJgG4Oi87+k4Mw0QlqsIXQlwYaTsR3QBgJIALhKiqobQLQNuA3bKNdaHO/yaANwEgLy+Pq6wyteAgNoaRg10vneEA7gdwuRDieMCmyQBGE1EdImoPoBOAhXbaYhKXikoeBzCMDOza8F8FkA5gBhEtJ6I3AEAIsQbARABrAUwHcLsQ4pTNtpgE44VRPQEAvbIzo+zJMEws2Iq0FUKcHmHbMwCesXN+JrH5ZV5bnN+lObLSedKWYWTAkbaM1rCyZxh5sMJnGIZJEFjhMwzDJAis8BmGYRIEVvgMwzAJgmfy4X9wU38cKT2pWgyGYRht8YzCP7dzlmoRGIZhtIZNOgzDMAkCK3yGYZgEgRU+wzBMgsAKn2EYJkFghc8wDJMgsMJnGIZJEFjhMwzDJAis8BmGYRIEqq5KqB4iKgawzeLhzQDskyiOF+E+igz3T3S4jyKjqn9OE0JEjT7VSuHbgYgWCyHyVMuhM9xHkeH+iQ73UWR07x826TAMwyQIrPAZhmESBC8p/DdVCxAHcB9FhvsnOtxHkdG6fzxjw2cYhmEi46URPsMwDBMBTyh8IhpORPlEVEBEY1XLIxsieoeIiohodcC6JkQ0g4g2Gv8bG+uJiF42+mIlEfUNOOZ6Y/+NRHR9wPp+RLTKOOZlIqJIbegGEbUlojlEtJaI1hDR3cZ67iMDIqpLRAuJaIXRR08Y69sT0QLje00gojRjfR3jc4GxPSfgXA8a6/OJ6OKA9SHvw3Bt6AgRJRPRMiKaYnz2Vv8IIeL6D0AygE0AOgBIA7ACQDfVckn+jucC6AtgdcC6FwCMNZbHAnjeWL4UwDQABGAggAXG+iYANhv/GxvLjY1tC419yTj2kkht6PYHoBWAvsZyOoANALpxH9XoIwLQ0FhOBbDA+D4TAYw21r8B4PfG8h8AvGEsjwYwwVjuZtxjdQC0N+695Ej3Ybg2dPwDcA+A8QCmRJI9XvtHeQdL+IEGAfgm4PODAB5ULZcD3zMHNRV+PoBWxnIrAPnG8r8BXBO8H4BrAPw7YP2/jXWtAKwPWF+1X7g2dP8DMAnAMO6jsP1TH8BSAAPgCxJKMdZX3UsAvgEwyFhOMfaj4PvLv1+4+9A4JmQbuv0ByAYwC8BQAFMiyR6v/eMFk04bADsCPu801nmdFkKIQmN5D4AWxnK4/oi0fmeI9ZHa0Bbj1boPfCNY7qMADHPFcgBFAGbAN+I8JISoMHYJ/F5VfWFsPwygKcz3XdMIbejGPwHcD6DS+BxJ9rjsHy8o/IRH+IYGjrpbudGGXYioIYDPAPxRCHEkcBv3ESCEOCWE6A3fSLY/gFzFImkDEY0EUCSEWKJaFifxgsLfBaBtwOdsY53X2UtErQDA+F9krA/XH5HWZ4dYH6kN7SCiVPiU/UdCiM+N1dxHIRBCHAIwBz7zQSYRpRibAr9XVV8Y2xsB2A/zfbc/Qhs6cTaAy4loK4D/wWfWeQke6x8vKPxFADoZM91p8E2gTFYskxtMBuD3IrkePru1f/1vDU+UgQAOGyaHbwBcRESNDU+Si+CzFRYCOEJEAw3Pk98GnStUG1phyP02gHVCiH8EbOI+MiCiLCLKNJbrwTfHsQ4+xT/K2C24j/zfaxSA2cYbzGQAow0vlfYAOsE3oR3yPjSOCdeGNgghHhRCZAshcuCTfbYQ4tfwWv+oniiRNNlyKXyeGZsAPKxaHge+38cACgGchM/GNwY+298sABsBzATQxNiXALxm9MUqAHkB57kJQIHxd2PA+jwAq41jXkV1QF7INnT7AzAYPlPKSgDLjb9LuY9q9FFPAMuMPloN4FFjfQf4FFIBgE8A1DHW1zU+FxjbOwSc62GjH/JheCsZ60Peh+Ha0PUPwHmo9tLxVP9wpC3DMEyC4AWTDsMwDBMDrPAZhmESBFb4DMMwCQIrfIZhmASBFT7DMEyCwAqfYRgmQWCFzzAMkyCwwmcYhkkQ/h+tMc2Y94nYyAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "temp = float_data[:, 1]  # 온도(섭씨)\n",
    "plt.plot(range(len(temp)), temp)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이 그래프에서 온도에 주기성이 있다는 것을 잘 볼 수 있습니다.\n",
    "\n",
    "다음은 기간을 좁혀서 처음 10일간의 온도 데이터를 나타낸 그래프입니다. \n",
    "\n",
    "> 2009년 1월 1일 0시 10분부터 1월 11일 0시 0분까지 데이터입니다.\n",
    "\n",
    "10분마다 데이터가 기록되므로 하루에 총 144개의 데이터 포인트가 있습니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD8CAYAAAB0IB+mAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJztnXecG+W193+P+kqr7X29613baxv3jjGYanonkEC4JATyEkhP7hsuxDf3hpZwA0kuIQVIIOSlhtBCaAbbgGm2sbFx721tr7c3Sav+vH/MjHakVdeM6vl+PvtBmpE0h7H0mzPnOYVxzkEQBEHkP5pMG0AQBEGkBxJ8giCIAoEEnyAIokAgwScIgigQSPAJgiAKBBJ8giCIAoEEnyAIokAgwScIgigQSPAJgiAKBF2mDZBTVVXFW1paMm0GQRBETrFx48Yeznl1rNepLviMsQsAPARAC+AvnPP7I722paUFGzZsUNskgiCIvIIxdjie16ka0mGMaQH8AcCFAKYBuI4xNk3NYxIEQRDhUTuGvwjAPs75Ac65G8DzAC5X+ZgEQRBEGNQW/EYA7bLnR8VtBEEQRJrJeJYOY+wWxtgGxtiG7u7uTJtDEASRt6gt+McANMmejxO3BeCcP8Y5X8A5X1BdHXORmSAIgkgStQX/MwBtjLFWxpgBwLUAXlP5mARBEEQYVE3L5Jx7GWPfBbACQlrmE5zz7WoekyAIggiP6nn4nPM3Abyp9nGI3GTtgV5YDDrMHFeaaVMIIu/J+KItkb8c6Lah5Y43cO1jn4bdv7dzGNc+thaX/v6jNFtGEIUJCT6BT/b34LvPfg6/X7mB9g63F195bC0AYO2BPnA++tmrdnbiLx8ewLm/XRPY5vb6FTs2QRDhyapeOkRm+Oqf1wEA7rtyJkqL9Ip85n+8tBXdw67A88ERD8rMBvTb3bj5b2PbZ+zoGMKcpjJFjk0QRHjIwycCONxeRT7H5fVh1c5O1JeacPfl0wEAB3rseGFDOx54Z3fQa3/z5dnQMGD1zk5Fjk0QRGTIwycC2F3KCP76g31wuH14+Lq50GsFn+KnL2/FrhPDgddcMqset54xETMaS/HKpmN4ZfMx/OjcyWCMKWIDQRBjIQ+fCGB3+RT5nNW7umDUabBkYhXqSk0AgF0nhlFbYgQAXDGnAb//6jzMaBQyc66c24j2vhFsONyvyPEJgggPefgFzmeH+gKPE/HwP9zbDZNei4UtFYFtnHO89sVx/L9PD+P0tioUGbRoqymGXsvg8XHcf9UsLJlUCZ0m2M84d1otAOCaRz7FM988GadOqgIgZPHUlJjwnWc+h9Wkw+IJlfjygiYUGbQAgG3HBjGh2gKzgb7GBBEP9EspcK55ZDRl0han4Ds9Ptzw+HoAwKH7Lw5sf2XTMfz4hS8AAF87pQUAwBjD419fiB0dQzh9cjW0mrEhG6tJj7nNZdh0ZADX/2Udbj1jIqwmHR5YERzvf2vbCfz3a9vx1M2L0O/w4PvPbcKZU6rx5DcWJfT/TBCFCgl+AeP0BIdwHO74Qjrv7+4as83n57jrXzsAAI/dMB9nTa0J7Dt9cjVOnxy9T9Jz/2cxVu/qwj2v78AjH+wPbJ8/vhxtNcXw+DhKinT468eHAhcbwZZufH6kH/Oay8E5h58j7EWFIAgS/IJmwOEJeh6vh//p/t7AY6fHB52G4dE1BzA44sGyk2px3vS6hG0x6bW4aGY9zp5ag0se/gj7umx4+uaTcVpbVdDrbj1jIt7a2oEV2zvx04tOwk1/+wxff3w9/s/pE/D3z9rh83O89r1TUWM1JWwDQeQ7JPgFzMCIO+h5PDF8zjne3n4i8HzY6cXb208Ewi+//+rclGwy6bVY+eMzIu6vLTHhxlNbceOprQCAfz93Mu54eSt+8+6ewGve29WFryxsTskOgshHSPALmFAP3x5HSGflzi50DrnQUmnGoV4HbC4vth8bBADcsHg8THqtKrZG4tpFzZhYU4wKiwGtlRbMvvsdbDk6iK8sTKsZBJETUFpmluD1+bHxcD/67e7YL1aIMYIfh4e/UUyd/Mn5UwEAZz34Pp7/rB0nt1bgnitmKG9kHCxsqcDE6mJoNAyzxpViq3gBIggiGBL8LGDbsUFMWv4WvvSnT/Dd5z5P23E3tQfnve+WFUZF4kifHROqLKgqNgRtP21SVYR3pJeZjWXY2TEEl1eZmgKCSJY9nbF/T+mGBD8LuOTh0W6Ru0/Y0nbcRz84EPT8o3098MVooHakz4GmCjOspuCeO5fMblDcvmSYPa4UHh8P3IkQRCZ4ZdNRnPfbNXgvTEZbJiHBzzJ8fn9QZ8l00VxhBgB0DTsjvsbv5zjYbUdrlQUNZcFZMC2VZlXti5czp9SgzKzHSxuPxX4xQajE+oOCw9He58iwJcGQ4GeYUE+03+FJm3dq0o/+89+weDyA6O0VDvXaYXf7MK2+BGVmA17/3mmBfdnSA6fIoMW0+hIc6EnfnRJBhCKlOBelOYkhFiT4GeZfXxwPPK62GqFhwJq9PWk5tvzLWCv2vIm0cOvzc/zkxS0AEOiBM6OxFP996TTcuKRFXUMTpLnCnHWeFVFYSL+jeIsZ0wWlZWaYL44OYHZTGa5b2ISFrRX4wfObsHZ/L/gyrrrXLP8ySouw4QTf4/Pjpic/w8bD/WiuMOOkemtg3zfEfPhsoqnCjB6bG8NOz5i1BoJIB34xLLvrxFCGLQmGPPwM0jXkxKYjA5jZWCLkk1cX4+wpNVh/qA8X/+4jeHzqTYHy+zlcXj8unlmPe6+YgWKjcO0Pl4v/6f5efLi3BwvGl+Ol25ZkTfgmEtMbSgAAD6/el2FLiEJFmuC2pzO7Qosk+Bnk6XVHAAAXzxzNcPnu2W246dRW7OgYwn1v7FTt2CNiH52Z40rxb4vHw6gTwjvhRg1K6ZqP3DAf1VajajYphdRt87E1Byg9k8gILvF3lG2hRRL8DOHzc/xu1V4AwIzGksB2g06Dn140FUsmVuLJTw7hk33qxPMlwTeLrYYNOuGr4PaNFciNh/vRVFGEquLsF3sA0Gs1+MNX5wEA/t8nhzNsDVGISI0Ju4ZdcfeoSgck+Bli+/HRatDQOLNOq8Gfv7YAFoMWj390EJxzeBUO74yIoRupFYJRFHyXJ/g4nHNsONyPheMrkEucOqkSAHD/27uCZusSRDqQd6J9VNb9NdOQ4GeIj0TPfcN/Lgu732LU4QfL2rBqVxfm3vMuJi1/Cx/s6Vbs+JE9/GDBf+LjQ+ixuTC3ObcGjJeZDXjl20vg83MsvG8lOgZHMm0SUUC4vH5cNFPoGhsuTJopSPAzxK6OYTSWRQ+T3HzaBCyeUBHoefPTl7fCH6MSNl4kD78oxMMP/XJ+fkSoCTj7pFpFjptO5jaX467LhCHqT6+l0A6RPpweP0qLDCgt0o+ZO5FJSPAzxM6OoaD0xnBoNQzPfnMx1i8/B/deMQPHBkbwx/eVyTyRPHxJ8CUP3xUi+CUmPaqKjWgsK1LkuOnm60tasLClHG9uPZFVPzwiPQw5Papmu0XC5fXBqNPApNfA6SEPv6Bxenw40GPH1LqSmK/VaBhqrCZcf3IzJtcW4/3dyoR1Ah6+FNLRSjH8YFEcGvGgtCi3yzW+vqQFB3vsuPThjzDs9MR+A5E3zPr5O/i3v6xL+3FdHj9Mei1Mei2cWZQpRoKfAfZ12eDzc5xUH1vwJRhjOGVCJXZ2DCkS1gl4+KLgM8ZQYtJhcEQQxPY+B1rueANvbO3I+eKlS2Y14OeXTsPeLhvW7ElPFTORPaw72JfW4w05PXD7/DDpNTDptAHnKhsgwU8zT3x0EP/3H8Kg76kxQjqhTG8ohd3tw6Fee8p2hMbwAaDKakSPzQ2X14cbHh/1iqSLQC7z1ZPHQ6dhQdlRRH6j1HpXoiz79QcAAKNOK4R0aNG2MOkYHMHdr+/ArhPDmNlYitZKS0LvnyZWkG47nnq5tiPEwweAqmIjemwuvL3tBA71jhaM3JehwSZKYtBp0Fplwd6u7Kp8zGc8Pj96bZlLiQ1dj0oXXWIasF7LYNBp4KaQTu5jc3nj+jLbXd5AP411B4Rby6+fMh6P37gAGk1iLQom11pRZtbjHdlM2WRxhvPwiw3osbmwamcXKi0GvP3Dpdh+1/lYkiXDTVJlYnUx3t3RSYu3aWL5K1sx/96VGUtLzMS/s3yehJ9zGHVaSsvMB770x08w/96VgeeHe+342avbxowovO2Zz3HB/36Ifrsbz647guYKM/770umosZpCPzImBp0GJ7dWKDLCzxFG8KuLjdjfbcdb2zpw5pQaTK0rgcWY2wu2cuaPLwcA/PE96rGTDl7ZJMwkONRrx4ZDfTGH6yhNJhZL5VW1Bq1G8PAzkCUUCRL8JNktji+78KEPMeBw44wH3sdTaw/jzAffx9efWI9+uxt+P8casVhq7j3vYv2hPlx/cnPCnr2cqmIjDvc6sGpnZ0r2j3h8MGg10GlHvwJN4hAUj4/jsjnZMcFKSW46rRXFRh3e2ZHauSPiQ+rP9PfP2nH1I5/irx8fTOvxtx5N/3rNx7JWKBajDgathjz8XGfj4dFV/50dQ5hz97uB54MjHnywpxsvbzqGtQd6x7z3xlNbUjr2dYuaAQAPrNid0uc4Pb6gASgAMF62piBNwMontBqGH507GbtODONgT+oL30R0SouE7K7HPxKE/nBv+hqJcc7xz82jsyb2d6dn7ebbzwgzqU16DS6Z1SDG8Enwc5rVu8bOqSw26vDFf52HSovQV35fly0wueqTO87GnRdOxfa7zg94Pckyo7EUd18+HbtODKfUo8Ph9gYt2ALBYworLIbQt+QFF82sg4YBL208mmlT8p6SouB03mJT+sKDq3d14Y2tHYHn54iZM+niL19biCKDtnAEnzH2c8bYMcbYZvHvIrWOlW42Hu7HrHGlWPHD0wPbHr1hPkrNerz/kzNxUn0Jnlt/BL9+dw8AoK7EhG+dMVGxePiVcxsBAH/f0J70Z4x4/DAbgu1pknn1JWn8caaT+tIinNxamXXDpfOREXdwl0i9Nn3+ZV/IWlq6MYp3z4UWw/8t53yO+PemysdSFafHB7+f45VNR7H2QB9mNpZiSp0VL922BN86YwJObhW6SVpNenxDNvKvtcqSUsw+HFaTHt8+cyKO9DqSLhsfcfsCnTIlTHotrp4/Dg9eMzvrh5ykwqLWCuzoGELXUOSB7UTqhI73sznT1ybYGGaWbDqydqRiynnNQoKAQavJWHpoOPLTjVMYl9eHsx58Hx2DowIh5cTPH18eyP6QWDatFnhJePzgNbNUsWlSTTG8fo7DvQ5MqilO+P0jHm+gU6acB6+ZrYR5Wc350+vw0Kq9eH93N768sCnT5uQtDrcvKKTROeSEx+dPi6cvNQOUE87JUZoKix4LxpdDKzp5xkIJ6Yh8lzG2hTH2BGOsPPbLs5Mf//2LILEHgMtmR85iqbAYAs3G2moTq6aNlwbx8zuT9FJH3L6glMxCYkqdFUV6LXaJk7wI5eGcw+H2okq2FvTG1g7c+tTGtBw/3P1pOkIrTo8/EM4BRkM6nGem6jeUlDx8xthKAHVhdi0H8CcA9wDg4n9/DeCmMJ9xC4BbAKC5uTkVc1ThaL8jsPhz1dxGNFeacdNprTH7yzx/y2LotAwlKvWhkRaHe5OMVTrcPlRYcmOCldJoNQxT6qzYKLZ+JpTH5fXDz4HKYiOOy5ylVWESHtTA4xsrsKHDfdTA5fUFspMAIaTDOeD1c+i1mQ+TpiT4nPPw0ztCYIz9GcDrET7jMQCPAcCCBQuy4zIIId3y75+148lPDgEALp3dgAeumR24VYtFk8ppjVIWzbPrDuPimfVx2yXh9PjChnQKhbOm1OC3K/dgyOlR7aJcyEi9miqLM5PtFW5tKx3zjV0ef1A4ySCbM5HORetIqJmlUy97eiWAbWodS2ke/WA/Lnzow4DYXz1/HB6+bm7CoqomZWbhh7T2QF/AzkQY8RRuSAcA5ogTvLYpULVMjEXq1VRfmpk5CvK4ufS7TcfiqcvrD1onMEQYLJQp1Lzk/IoxtpUxtgXAWQB+pOKxFOWXb+0Kep6NC5nyi8+hJIqIHG7fmDz8QmJmYykAYEsGqjELAYfYYmDxhAr858Un4a83LgQAzGlKz6hMebz+CfHY6fDwnR5feA8/S1IzVcvS4ZzfoNZnq8kBsSKv0mLAv753Gvodmc3njca1C5vw/Gft8CexIOT0FLbgV1gMmFpnxaubjuFbp0/I6zTUTCClZFoMOly+VKgbObm1AumK2cpDOsYI09zUwOUNCeloC8fDzym8YivXFzYIFZj/ceFUNJQVYXpDaYYti8z9X5qFKbVW7Euw5a/H54fHxws6pAMAX5o3DrtODCe98E1ERhJ8s3H0O2bSa8dMVFOLzAl+cOqnVGwZWpOQKSgPX+SOl7fiRbHcflx5Ea6Y05hhi+LjlImVeHb9Efj9PO4Cr9B5toXKrHHCxXz1zi7Kx1eYEY8Q0pFXc5v06StCkjzquhJToJ2J2lk6nHMhLVPm4ReLgp8tozXJwxd5UdZb5W83LQrE3rKdttpiuL1+nEggH7/PJni05XnaLydeFrVWwGrUYcuxgUybkncEPHxZ2NCo06atR71bTMv86D/OCvyW1Y7hSw3aqktGW59bTZLgp6/KOBq5oWoqE1oUMbE68crVTNEidrhccv9qPLxqb1zv6RYHt1RbCzMPX4IxIR9/NxVgKU64eQsmvQbONOTCA0D3sBNaDYNOq0lbSOfEoPC7miIrtpTqdYZdJPhZw5B49f3+OW04+Mvc6vE2Xtbh8tfv7sEf3tsX8/axRxzBVl1c2IIPAPNbyvH5kQEMZPHifC4yEsHDT0emjM3lxXPr2wMDV6TKV7UF3y42i7PI1i1GPXwK6WQN0pSqlkpzzmVrhOY5P7BiN/768aGo75E8/CprYYd0AOCMydXw+Tm+oPRMRVi1sxM3PflZYPKTvENsujz80KlzUgxf7UwZhyT4snULCulkIX2O3I1physGCzd4RU7PsAsaBlQWaGsFOSfVCU3w9lBYRxFu/tsGrN7VhWMDI2AsuImZSS94+Gr3lRkcCfamjWmK4dtdY+9qivRaaDUsrZ1Co0GCj1GPoMKce4IfjliThbptLlRYDFlVOZwpyi0GVFuNgZGVhDIc7rXDrNcG3TEbdRr4efg+N0oyFEnwVb67kDx8s+yuhjGGsiJ91qT+kuADONAtVKrWlyU+WDwbWNpWBQC45fQJuP7kZhwfHImaDdE97EYVxe8DTKm1Yg8JvqJ8vK93TNtuKT9d7eHiLjEH/6Fr5wAQRNegUz8lVPLwQ9OdG8uLcLQ/feMdo0GCDyEEMqHKghprbgr+ozfMx4e3n4WfXnQS5jaXg3OMaecsp9/hRnme3M0owYzGUmw5Oogv2ik9MxX8/mDPXT4jGUifp+0RhV1+wTFqNaqHdBxubyCEI6epwoyj/SOqHjteCl7wOefY1D4wZohJLmE26ALdOcvE1qyht7VyBhxulFuoQ6TEl+YJRXYPr96XYUtym1DPPXQusjSFSu1cfClkZJB1pzSmoejL7vYFZehINJWbcbTfEcgayiQFK/jSwtHR/hH02d2YnaamTmojDY4eipIGNuDwBLptEsKQmoUt5egappGHqTAS0j7gtElVQc+lkI7awiu1VZC3IzbqtOpn6bi8Y+ZEA0BTRRE8Pp70sCIlKUjB55zj3N+uwSm/XIVN4m387HH5IfjS8IUBR3jB55xjYMSDcjN5+HJaKi1Z8YPMZUZCPPdl02qDnkshHbU9fKkzpV4nF3z1Pfx+R/jfVVO5cPfd3pf5OH5BCn7nkAv7umzoGHTiHxvaYdBpMKVOnVGE6UZaeD42ED5mOOT0wufnKCsiD19OtdWIHps7a0bR5SKxhFxKV1S7kdiohz8aSzfoNKo3buu1u1AZJhlCCrdmQ2JAQQr+EdmV9sO9PVg8oTJneufEosSkR4XFgMO94XvkD4qefxl5+EGUFunh83PYs6SrYS4y4haE9qwp1fj0zrPH7JdaeXSLld5qIS3aBsfwtap7+L02d2D0qJzxFWbUlhix9mCfqsePh/xQuQQ5HuL9zmgoyZAl6tBcYY6Yiy/196csnWBK41jszkc459h9YliRBUUppHPzaRPCTrqqFbPg1A6dSYu2wTF8dbN0OOeC4Ifx8DUahpZKC7qyIGRYkIK/WYzbf/O0VgCCQOYTLZVxCD5l6QQhLWL32rKjQCZd3P7iFpz/v2vw+pbjKX9WoO12hME6ZWY9DDqN6oLvDrtoq1F10XbY5YXb50dVhBm+daWmhDraqkVeCf5d/9qOxz86GPU1244NBmbA3n7BVNx/1Ux8af64NFiXPpoqzDg+OBJ2kPNAIKRDHr6cSTVCzvi+7szHWdOF0+PDP8S24Pu7Ex+TGcqIWGkaac4CYwy1JUb1Bd87Noav9qKt1JAw0tD2uhITOodcGV8jyivB/+vHh3DP6zuivkY+Hcqg0+DaRc1ZMU1eSepLi8B5+Fip1BVSytcnBMZXWmDQarCrgHrq/HPzMUU/T2oQJjUMC0etVX1P1+PzQ69lIW0d1I3hS60TIvWnqi0xwe31oz9C9ly6yBulizc+J1WgPn3zyWqak1HqS4VY6XPiJCw50heulAQ/CL1WgxmNJVh3IPMLa+mia2jUIfjdqr2wp9izXeqQWWyMIvilpqDjqoEg+MHSpnYMP1YyRJ34mzwRpQI+HeSF4NtcXry3qzvwPNptU3u/A2VmPU5rq4r4mlxH+nI9vHof3tlxImhfv8ONEpMOujy7q1GCpW3V+OLoQNSitXzi+c/ag56vP5TaxU7y8ItjePjpWLQNFXyLUYehEfU6VkpVxpHCWbUl6VmwjkVe/Or3ddlw69MbA88j9dwednrw7LojaK2yhN2fL0gePgBsPNwftK/H5kJVgU+6isS88UIfom0F0htfqtV45pvC3a4nxZCHzeWFSa+JGiKtKzXC7vapOhDEHcbDbygrwuCIJ+W7mEhIVcamCIIvOWEk+ApQEuJRDLvCf5mkBd1FLRWq25RJ5OGaQyHZOj026pQZiVmNwlDzHR1DGbYkfVw8sz6QpTaQYkrqsNOLYmP0UOGop6teWMfj9cOgDW5g1lgupIlGKkhMFanoLJLg11iNKDbq8MXRzDboyw/BD4lHR5ouIy3Y/nDZZNVtyiSMMdyweDyAsXnlPTYXjTaMQLnFgGKjLms6G6pJn7jIOL2xZLT/UoqCb3N5oy7YAqOCf6Qv9aygSHh8/qC2CgAwThT8WLMikkWKKkRKSdVrNZhQbYnaxTYd5IXgh37JIk2X2dtpwzlTayL+o+QT91wxA8tOqg3M6wWAXpsLB7rtEVPHCCEc1jGY/4J/7WOfAhDG8VmNOjAGPPnJoZT+34ednqgLtgAwp6kMJr0Ga/b0JH2cWISL4U+ts0KrYVizpzvCu1JDqkEwRanYtxh0cLgyW8mdF4IvzayUsIWJ03l9fhzosaGtNj965sRDmVmPPrtw6+z2+jH/3pUAhNtLIjwNZUU4PpD5Ahk18fs59nQKd7vnT6+DRsPAudA59o6Xtib9uTZnbA/fpNdiUWsl3tl+IurrUiFcDN9s0MHn53hq7WEc6lH+7sLh9sGg00RNhrAYtYFB55kiLwQfAJ64cQH+65JpAICfv7Z9TGe6w30OeHwcbSFTePKZceVF6Bxy4em1hzEwMlpB+m9iuIcYS11JdlREqolUa/A/X5oZWEyU8PqTX7i1ubwxPXwAWNRSjuODTtW6Znp8Y2P4wOjalhpxfIfbGzTLNhxmg071xnGxyBvBP3tqLc4V27Hu7bLhO89+HrRfGmM4sYAE/ysLmwAA248PBeKzv7tuLlXZRqHKakCf3T2mfiGfkLo2zh8/mrzw0m1LAETOcIuHYac3akqmREOZEE8P7WmlFOHy8IHRbKQ+FebL2l0+WML0wpdjMWrDRh/SSd4IPjDajQ8AthwdxEMr9waerxBvIcN1s8tX6kuLcFJ9CbqHnRgcoYKreKgqNsLn54GeQ/lIj00I88l/L/PHl+PCGXUpLdwOOz2wxuHhN5apmzHj8Y6N4QOjocxUs5HCEbeHT4KvHCa9Fl9Z0BSYtPPblXsACM3SXhR7hlji+ELmEzVWIzqHXLCJi0XFYUawEaNIKas9edxErXvYBYNWMyadubRIH3AMEsXp8WHI6Y0r5TeQIqlSNpQ7TJYOAFhNgrOjRg2A3e2DOYa2WIw6ODxCDUKy5zlV8krwAeB/rp6Fp25ehAXijNr2PkfQP3C4mZP5TI3ViK5hp6yxVWFd8BJFEqxem7rl/5mk2+ZCtdUY1GsGENKbk60ylgqKQtcEwlFbYoKGqejhR4jhm/Qa6DQsYtp2KjhcXlhiePgWgxacA2c9+D5m3/WO4jbEQ94JPiDkoS+/+CQAwM6OoaA0zdCMnnyntsSE7uFRDz/WbWehU20VQn7deSz4QvHd2NBmaZEeTo8/qZ4z+7uFrJ8J1bGr2PVaDcrMBtXCZm5v+Bg+YwxWk049Dz9GDF+6A5DuHr1hutmqTV4KPjC6OPvomgOqxOxyhZoSI/x8dJ4mCX50CiWkEy70IoV4Lnrow4Q/UyooaiyLb7aEILzqxLMjLdoCQvPAp9ceUbyJmcPtjRk9qAhJlshEWCdvBb9EjNdtPNyPA922GK/OX2rEKUMPrRIWsAuh6CwVSovSM6QjU/TYXNjZMRQ05lOiSPRQ93fbE+7bLol3SVF8IUM1BT+Shy9n1a5OxY7HOYfNGXvRNrTgMROtkvNW8OX8+UOhh85vvzI7w5akn5qSYE8u1m1nocMYQ1N50Zg6jnxBai+ytK16zD6jbKEz0d7xw04PtBoWsVtkKKVF+sBsBiUZHPHgxJAzZjW5klm3HYNO9NrdaKuJXtQZGkZT4/8/FikJPmPsGsbYdsaYnzG2IGTfnYyxfYyx3Yyx81MzMzk++MmZQc+vnJtfk63iQepdIqHVjF3MIoJprjCH9YDzgS5xKM61i5rG7DPIBD/RrpI2p1B0FboQHInGsiJVehb12Fzwc2B6hDnVPzl/CoDE//+iIU1wV38kAAAgAElEQVSRayiLvmBdETIcZSAHPfxtAK4CsEa+kTE2DcC1AKYDuADAHxljaY8ljK+0BHKNI30B8h1qlJY4zRVmHOl1ZHwcnRpIg7SlgeJy5IIfb0Woz88x5PRgOI62CnLqS4vQNewKO4YzFWwxpm7ddsZEAFC0yldqlxAr5Tt0ylwmaj1SEnzO+U7O+e4wuy4H8Dzn3MU5PwhgH4BFqRwrWWaKLW+/c9akTBw+4xiiNHMiwtNUYcawy5sRD0xtuoZdMOo0YWPt8pBOvD1fnvr0EGb9/B1sPTYYyHOPB2kmQ7/CVa9SJWukqleNhsGg1aRUURzpmLHCpRoNw8vfXoI1PzkLQH4t2jYCkI/TOSpuSzu3njERzRXmgvXwAeCdH52eaRNyCqk/fD6GdZ5ddwSMIWzoJUjw4+zquGpXFwChnYkugXBhlVjxrnQ2VGDMYpS7DZNeo6iHL3XAjKfGZ15zeaDw7KXPlZ0pHA8x78EYYysB1IXZtZxz/s9UDWCM3QLgFgBobm5O9ePGsKi1AmtuP0vxz80lJtdacfsFUzAhzyd9KUVzpSD4h3rtmN1UlmFrlMPm8kbt5aLVyEM68Xn48swUTQKCXykVuNmVrXeQQjrRmriZ9FpF59tKrZHNcRY1SutoOzuGcLDHntYJfDEt5JwvS+JzjwGQrwqNE7eF+/zHADwGAAsWLMi/oGmW8O0zCzOklQwTq4th0Gmw9eggLp+TkRtTVZDaAi+NMM+5raYYDaUmHB90xu3h9zs8mNtchnnN5bhybvznSsqi6VXLw48i+EUGraJdK6W7BaM+8YDJns7htAq+WiGd1wBcyxgzMsZaAbQBWK/SsQhCUfRaDWY0lGR8HJ3SSKmmd1w4Nex+i1GH525ZDCB+D7/H5kJ9qQk/u2QaZojrZfFQZZEK3BT28F2xF1DLivSK5sAHxhsmUcWf7vTfVNMyr2SMHQVwCoA3GGMrAIBzvh3ACwB2AHgbwHc455ltBE0QCTC3uRxfHB0M5K3nA1Ia5LjyyNWw0sKjPU4PuCdC1W4sSoqEKVvPrjsCn4JJ8TaXF3otC1qPCKXCYggMBlICqWYhEQ9fWlc72j+ClzYexeJfrEpLS+5Us3Re4ZyP45wbOee1nPPzZfvu45xP5JxP4Zy/lbqpBJE+bj6tFUadBr9fvTf2i3OEo/0OlJh0UVtkSwuPnx/ux81Pfha1iZzLG3+HzFAYE6ZsHeix461tHQm/PxJ2V+x6gHKzQdEMLKfHB8YQ9SITyuRaK6bWWdHe58C//+MLnBhywpaGaViUs0cQYWgoK8IpEyqx5ehgpk1RjB6bO6gHfjiksMQrm45h1a4unPObDyK+VhokkozgA8B4cXH8QwXn29qc3pj58BZxUL1Sg1CcHh+MOk3cRWcSDWVFQUPNlV7PCAcJPkFEYEZjKQ702BWtyswkwy5vzFz50EybaJ5wz7Ak+MkNFXrj+0txcmsFPj3Qm9T7wzEcx5hF6YJwyi9XKXJMp8cPU5wtJeTUlhgDlc8AcNOTnyliTzRI8AkiAlKrXyXy8f+5+Rjm3/OuanNc48Hm9CRUDQsA0+oj169IC65VMe4aIlFs1GF2UxlODDkVq2q2xyH40hCgRPsFRcLp8SW1YGs16WFzjV5Qp9RG78WjBCT4BBGB8RWC4B/uTV3wf/D8ZvTa3dgtDhDPBPEOGb//qpmBx5ooCiHNDKiyJN++o7rYCLfXr1hM3eaKPVc30dBLLJxeP0xJpGSaDdqgit+7L5+upFlhIcEniAhIBVhH+uyKfeYtT21Q7LMSRWpwFour5o3Dr740CxfOqIuYr845x+0vbgEgDH5PFqmwbd3BvqQ/Q048/4/lZmXnWjs9vqRCOnI7b1g8HjUlsaeFpQoJPkFEoLRIj2KjDscHlOuN3zmUuUlaw67YC5qA0H/pywubYDXpAm0DQjnYM3oRTKXl9qxxpWAMit35xHMXc80CoWtunUIC6/T4YExC8OXnLZEMn1QgwSeIKNSXmtAxqM7s1XTCOYfNlVhHS7NBF7EAa82ebgDAFXMaUrLLpNeirEiPbpsyF9V4BF+v1eC6RU3wKbRu4PL4YUpCsKWeOgCgcJQpIiT4BBGF+pDUuVzF4faB8+gtB0IpMmgDfWIkPD4/3F4/Ooac0GsZfvPlOSnbVllsRPdw6nc+Pj+Hw+2L6y7GpNfCqVB7Bac3uZDOnHGjfZqK0jSYiASfIKJQX2JSRPDl/WvSUVEZSjxdJEOxGLTw+Dh6ba5ARs7Zv34fM3++Ap2DTtRYTQk1TIuEw+XFiu2d+POaAyl9jtTSOZ67GLNBi2GXV5EWxS5Pcou2pWY9Plu+DN86fQK+ubQ1ZTvigQSfIKJQX2ZCj80Fd4opfH5Z+MCpYKfGeBmOo4tkKJLXOf/elVhw70oAQHvfCFxePzqHXKgrVSYGftflMwAA/7tyT0rpmVKnzHg8fGkU4+y73kl5fnGyHj4AVFuNuPOikwIzuNWGBJ8golBfagLnSFkUvL5RIVOyU2O8SB5+IjH8UPGVP+8cciq26HnutFrceeFU2N2+lM6NPY5OmRJygV4t9vRPlmTz8DMBCT5BRKG+VFhYSzWsI28Q9uqmY2kfn/jqJqE7ebExfk/ytJA2ykPO0QXcAz32MfOSU0EKNUXr1x+L4QTCVkWyPv7v7+5K6d/DmWRIJxPkhpUEkSGkwdSpZup4ZYJ/7xs78erm9E072nCoD09+cgiAUM4fL1Prgqtsh0Li3XWlys1LlrzyYWfygh/P8BOJxrLRDJkV2zuxYvuJpI7p9vox5PQkHdJJNyT4BBGFOtHDTzUX3+v3Q76+qUT1brwckh1LumOJF4vMEx5yBgu+kh6+FMMedia/iDogXpDiEfxTJ1XhR8smB57vSrIOYPeJYXAOtKWhLYISkOATRBSKjTpUWgw42JNaX3yvj6NE1pZYr03fT0/e+z3Rofavf38prp4vFCr124PFeFx5YhePaCgR0lkrNmGrsMSupNVrNfjBsrbA8xNJhuwkexsUWsBWGxJ8gojBtIYSbD8+lNJn+Pw8aMF0JI0Lt32iUN+4pCXh97ZWWQLvO9ov3ClIGS5T6iI3VksU6dykEtJxe/0oMemSuvM4NpBcyG7EI9grXxPIZkjwCSIG0xpKsLfTllJqptfPUS3rG29Pw7ALCafHh9IiPX5+WXLNuaSBKVLY474rZ2Drz89LKMUzFtJn2VIQfKfHl3Bv/q+e3AwA+HBvD95OYhCLlFWUSnuJdEKCTxAxmNFQCrfPj10nkvfyXR4fJlQX41dXzwKQXg/f5fWl1KtFCkVJC79lZn3MvvqJUiY2NOtzJD8EJJkmZr+4ciZ+epEw4/fWpz9P+Jijgk8ePkHkBQtbKgAIY/+SxeX1w6jT4MsLmtBaZYl7ZqwSuLz+hGP3ckJHIpYWKdttEhA8fLNBm3QsHUg+PfJrp7QEHh/qSawzqnThppAOQeQJtSVGlJh02JvCQHNB8AVRMBu0cKRxipZ0sUmFryxoCjyONhM3FaY3lOD1LR3w+JILnQ2OJJceadJr8dTNiwAAP35hc0LvJQ+fIPIMxhgm11qxtzMVwffBKHqfFoMurdW2Ls/oxSZZGmR562VmdQT/qnnj0GNzJeXlbzrSj63HBrExybuwWY1CI7PPjwwk9L4RcS2GKm0JIo9oq7ViT9dwUhWZPj+Hx8cDXrbZqI3YdlgNXF5fSiEdQOhbL6GWhy8VQ51Ioo3Fh3uFQejJji0slV3EEvk3drh9KNJrFWkilw5I8AkiDuY2lWHA4UkqPVPK7pG87BKTPqXFyWSOn2pI58wp1YHHatUQ1JdKVc2JC75LgYZ0P7tkGoDog9tDcXh8ORPOAUjwCSIulk2rhVbD8Pa2xEvwJW++SAzpjK80o71vJOnwQ6K4vP6kJjLJYYxh/fJz8NJtpyhk1Vik7pvHk8iJl0Iq8gtTokgXnOMJtNEYcftyZsEWIMEniLiosBhwcmsF3koiV3u0U6UQNrhmvrAAunpXp3IGRsHl9cOggFdeYzVh/vgKBSwKj9WkR22JEXuSaHMgtSr64/XzUji+kEufyPqKw+2FJUdy8AESfIKIm4tm1mN/tx3bjw8m9D6pelQSlOZKM+pKTOhK03xbt2zBONuZVl+CHR1JhM18PmhYagVQUgVxYoJPHj5B5CXnTa8FADzx0aGEFvYCw0dkrRUqLAb02dMTx3d5/TCmsXdPKkxrKMG+LlvCMXl3irUGwGgu/doDvXH/+464KYZPEHlJjdWEW06fgJc+P4pN7fGn7wVCOrJe9MUmXUqNwhJBiOHnxk99Wn0pvH6ecAqsx8dTDltJHv6f3t+Pf24+Htd7HCT4BJG/fPfsSTDoNHj8w4Nxe4FSy1958zSrMX2C7/amnoefLqY1CA3Zth5LLGwmVBOn9v8oDwf98O+b4/r3HfH40jaAXAlI8AkiAUpMenzj1Ba8sbUj7iybcAPE0+nhOz2p5+Gni/EVZpSZ9XgrwWwoJVJPK4uDW0asO9gX8z0OtxfmHBl+ApDgE0TC3Hr6RACj/ddjEbpoCwi9Y1LpDBkvXp8fLq8/Z8IOGg3DOVNrsTPBhVu3zw+9NrXip9D6ggPdsfvq0KItQeQ55RYDptZZsUas7ozFsNMLg1YTFFZJl4cvNWlTspWx2kyuLUb3sAuDI/EXQLkVqCYGRgeZMBZfxS8t2hJEAVBfasL6g334YE93zNfaXJ4xg7WtRh1cXn9KPfbjwe6Kf85rtjCpphgAsC+BZnUeH1dE8N/98RnY9LNzUWLS43er9uLuf+2I+Fq31w+vn5PgE0S+871zhPF4b26JXYj1wZ7uQAaIRGDgh8peviT4lhwS/Cl1wnzYN7fGX+TmVqi4zGLUodxiCNxdPPHxQZz14PvwhungOdoaOXfOLQk+QSTBvOZyLG2rwpYY2STtfQ60942MGaEnDfzoV7mnji0HPfxx5WZct6gJf/34IDribHOgRB5+JA722HG0f6wdDnG8IXn4BFEAzG0qw+4TQwEvOhybxXz9Hy2bHLRdGsXXM6xuta0tBz18ALjl9Inwc2BFnNk6Lp9f0aZuD107J+j5rjDtHuyu3OqFD6Qo+Iyxaxhj2xljfsbYAtn2FsbYCGNss/j3SOqmEkR2Mbe5HH4ePWf8e89tAgB8c2lr0PYqq+Dh99jU9fBHQzq5I0oA0FIppGfuiTOO71EgLVPO5XMasX75OfjPi08CANz69Eb4/cF5+YGQTgGlZW4DcBWANWH27eeczxH/bk3xOASRdUytF2LNkSZhOT2CIDSWFY3xsCUP//63d6poIWBz5V6WDiB055xUXRz3wq3bp3xIp8ZqwjeXTgg8f293F/7y4QEAwIDDjUt//xGA3BlgDqQo+JzznZzz3UoZQxC5RF2JCWaDFge6w4tStxiuue3MiWP2lYsx/Pa+xFsBJ0KfXbCh3KL8HFq1aa40Y/3BPmw6ErvAzenxqTZ16pdXzQQA3Py3Dbj3jZ1wuL040ucI7Kc8fIFWxtgmxtgHjLGlkV7EGLuFMbaBMbahuzt2ihtBZAuMMbRWWcZU3Do9Pkxe/haW/uo9AOFHAmo1DEvbqgBgTKhASbqGXCjSa2HNMQ8fGD0v33jys5ivHXB4gqZWKcnscWVBz/d12YJ69udSDD/mt4AxthJAXZhdyznn/4zwtg4AzZzzXsbYfACvMsamc87HlM9xzh8D8BgALFiwQL1vPkGowIDDg2MDIzjUY0dLlQWA4Nm7ZWl8oYIhsbStCh/u7cGIx6faomrnsAs1JUYwlhsj+ORoNYI/6otxQXR7/bC5vKgwq3MXE9py4bLffxz0PJcEP6aHzzlfxjmfEeYvktiDc+7inPeKjzcC2A9gcqTXE0SucvsFUwAAX3tifWDbiGe0te+an5yFpgpz2PdK+dt2Fefbdg05UWs1qfb5avLts4RQ2HSxoVokhsTmdCUqzdotj3EhKfiQDmOsmjGmFR9PANAG4IAaxyKITHL5nEYYdRoc6XMEqmaHZT1ymivDiz2AQNOtkQQGbiRK17AL1SVG1T5fTSZWF+O8abXot0dvsSCdP7XukmItBufSgniqaZlXMsaOAjgFwBuMsRXirtMBbGGMbQbwIoBbOeexW88RRA7y4DWzAQB7OoVcbSn3/dLZDVHfJ4UCEpmwlCi57OEDQE2JEd226LUK0h2SmqGVr57cjBpr+AtnLmXppGQp5/wVAK+E2f4SgJdS+WyCyBWkkE3nkBMzGktxqEfosnj7+VOivq8oIPjqhHScHh/sbt+YGHQuUWLSo8/uxtF+B8aVh79bcgRaHKgn+L+4ciaumT8OV/7xk8C2529ZjMayItWOqQZUaUsQKSJ5fl1iGub7u7tQbtbHFAMpBKGWh5+LjdNCkdZr7309cr2CQ6w1UHuYuNyTf/17p2HxhMqI6zPZCgk+QaSItFgoTbY61OvAkolV0GiiZ8ZIFZqJjvOLF+lCkktZJKF87+xJAKL/PzjSENIBgitqZzSWqnostSDBJ4gUkRZfbS4f2vscONhjD4zqi4Y0EOXu1yO34E0Fh8qLmenAYtShtcqClzcdwzvbT6DX5sJja/YH1S5IWVGqC34OXzglSPAJIkU0GgazQQuHy4t/bj4GxoAr5jbGfF+zLBwQ73zcREjHYmY6qBeHkvzlo4O45amN+MWbu7BfrG5+bv0R/OD5zQDUXzyVLtA/XNam6nHUhASfIBTAYtTB7vZiZ8cwWiotcS3myYuhXCoMQgnEtnPYwweAh6+bCwBoqynG0X6hpUHHoBNOjw//+eq2wOvU9sBNei323XchfnAOCT5BFDTFRh1sLh86BkfQUBZ/GuQ9V8wAAAwlMM4vXvLFw68sNmLB+HJsbh+ATqy+/doT6/G1J9YHFWWl4/9Tp9XkZNWyBAk+QSiAxSiEdEY8/oRCC6Xigq9ULaok0mKm2tkr6WDe+PIxXUnXH+xDbcnoxVXJfvj5Cp0hglAAs0EYSu70+GBKoD+6FJ/e0TF2wEaqBAZ05Fgv/HBMqimG2+sfMzksRiIUEQIJPkEoQLFRh84hp9imN/6f1fzmctSWGPHO9vgmOyVCPnn4beJg81CcHmHt4+M7zk6nOTkLCT5BKMDkWisO9TrQMehMaPFQo2GY3lAa96CPRJA8/FyayBSJtlpr2O0f7OlGW01xzlW8ZgoSfIJQgFMnVQYeJxLSAYRwxYEee8w2wInicHthNmhjFoDlAsVGHdYvPyfsvkgTx4ixkOAThAIsaq1I+r2TqoX4dLtsipIS2N2+nM/QkVNjNeHgLy8as31ShHAPMRYSfIJQAKNOiy/+6zxcNrsBNy5pSei9E0XBUjqsY3d586I6VE64lMhXvr0kA5bkJiT4BKEQpWY9fnfdXDQkGE+WPNT9EWbjJsux/hHUl+RfbPvsqTUAgBuXtOCFb50Cq0mdwSf5SO4v3xNEjlNapEeZWY/2fmVDOkf7R3CaODc3n/jj9fPQZ3cnfGElyMMniKygqdyM9r6R2C9MgCGnB2Uqjf3LJCa9lsQ+SUjwCSILGFdeFOgTowQenx8Ot0+1Oa9EbkKCTxBZgCD4I4p1zZTm6kodHgkCIMEniKxgXLkZLq8fv3hzZ1Cv92TJh2lXhPKQ4BNEFtBUIcSk//zhQby7szPlz5M6ZZLgE3JI8AkiC5AP6B5RYMbtaOM0EnxiFBJ8gsgC5L1glGixIIV0LHlWeEWkBgk+QWQB8qlUSgh+oFMmefiEDBJ8gsgyDvfZU3q/2+vH8QEngPxojUwoBwk+QWQJL912CgBg27GhlD7ntqc34u7XdwDIj+EnhHKQ4BNEljB/fAWunj8O248PppSPv2pXV+AxZekQckjwCSKLaKspRo/NDZu46JoqxgSmbxH5D30bCCKLqCkxAgA6h5xJf4ZBFHmrURe2nTBRuJDgE0QW0VAqpGeuloVlEmVqnTAOcFihuwQifyDBJ4gsQpqc1e/wJP0ZNVYTACAPJhsSCkOCTxBZBGMM1VYj+mzupD/DLy74vvH9pUqZReQJJPgEkWVUWgzotScv+B6fH/Oay3BSfYmCVhH5AAk+QWQZe7tsWLmzM+mh5i6vH3ot/bSJsdC3giCyDKm1wtZjg0m93+PzBzJ1CEIOfSsIIst44VtCxW2yufgenx8G8vCJMKT0rWCMPcAY28UY28IYe4UxVibbdydjbB9jbDdj7PzUTSWIwqCtphgAYHMmKfheTiEdIiypfiveBTCDcz4LwB4AdwIAY2wagGsBTAdwAYA/MsaoqQdBxEGxOJYwFQ9fTyEdIgwpfSs45+9wzqVv5VoA48THlwN4nnPu4pwfBLAPwKJUjkUQhYJeq4FJr8GwM7lcfLfPD72WkvCJsSjpBtwE4C3xcSOAdtm+o+K2MTDGbmGMbWCMbeju7lbQHILIXawmPcXwCcWJ+a1gjK1kjG0L83e57DXLAXgBPJOoAZzzxzjnCzjnC6qrqxN9O0HkJVajDsNiDN/r8yc02NxNaZlEBGL2TuWcL4u2nzF2I4BLAJzDR3u6HgPQJHvZOHEbQRBxUGwaFfxJy9/CnKYyvPqdU+N6r8dHi7ZEeFLN0rkAwO0ALuOcy6tEXgNwLWPMyBhrBdAGYH0qxyKIQsJq0sHm8qJPrLjd3D4Q93vdlIdPRCDV6Qi/B2AE8K7YhnUt5/xWzvl2xtgLAHZACPV8h3PuS/FYBFEwFBt1+HhfJ+bd825C7+OcizF8WrQlxpKS4HPOJ0XZdx+A+1L5fIIoVKwmfVLv8/k5OAeFdIiw0LeCILKQcnNygu/xCctolIdPhIO+FQSRhYyvtIzZtu3YYNRYvsfnx9JfrQZAHj4RHvpWEEQWcumshjHbLnn4I1zxh48jvqfH5kKP2EefYvhEOEjwCSILKU0ipNMn66FPHj4RDvpWEESW8rebEutG8vvV+wKPKS2TCAd9KwgiSzljcjWaKorifn2vjTx8Ijr0rSCILOapm04e4617fP6g58tf2YrH1uxHt80V2EaCT4Qj1cIrgiBUpKXKgh+fOxn3v7UrsG3E4wsIusfnxzPrjgAQirUkjBTSIcJA3wqCyHKK9MGjJJzu0aJ1u6yjpry7plFPP21iLPStIIgs5ysLm/CtMybgZ5dMAwA43D48v/4InB5fxBbKoRcJggAopEMQWY9Jr8WdF56EN7d2AADe2NqBB1bsxhdHBzB/fEXE9xBEKCT4BJEjSF57x+AIAOC59e14bn172NeS4BPhoJAOQeQIkojbXbEbz5oohk+EgTx8gsgRigyC4L+yKfIsoaVtVegYdKLWakqXWUQOQYJPEDlCPAuxt505EUsmVqXBGiIXofs+gsgR4hF8yr8nokHfDoLIEUyG8D/X1qrRVspGHS3WEpGhkA5B5AhlRYbA4z9dPw8XzqwH58LAk9Y73wQATKsvyYhtRG5Agk8QOYK8p05NibAoK86SxvqfngM/BzQa6oNPRIYEnyBykLrS4Cwc6QJAENGgGD5B5CC1VmOmTSByEPLwCSKHePrmk3F8YAQ6an9MJAEJPkHkEKe1UY49kTzkJhAEQRQIJPgEQRAFAgk+QRBEgUCCTxAEUSCQ4BMEQRQIJPgEQRAFAgk+QRBEgUCCTxAEUSAwqdteNsAY6wZwOIWPqALQo5A5akJ2Kkuu2Ankjq1kp/Koaet4znl1rBdlleCnCmNsA+d8QabtiAXZqSy5YieQO7aSncqTDbZSSIcgCKJAIMEnCIIoEPJN8B/LtAFxQnYqS67YCeSOrWSn8mTc1ryK4RMEQRCRyTcPnyAIgohAXgg+Y+wCxthuxtg+xtgdGbaliTH2HmNsB2NsO2PsB+L2CsbYu4yxveJ/y8XtjDH2O9H2LYyxeWm2V8sY28QYe1183soYWyfa83fGmEHcbhSf7xP3t6TZzjLG2IuMsV2MsZ2MsVOy8Zwyxn4k/rtvY4w9xxgzZcs5ZYw9wRjrYoxtk21L+Bwyxr4uvn4vY+zrabLzAfHffgtj7BXGWJls352inbsZY+fLtquqC+HslO37d8YYZ4xVic8zdj6D4Jzn9B8ALYD9ACYAMAD4AsC0DNpTD2Ce+NgKYA+AaQB+BeAOcfsdAP5HfHwRgLcAMACLAaxLs70/BvAsgNfF5y8AuFZ8/AiA28TH3wbwiPj4WgB/T7OdfwPwTfGxAUBZtp1TAI0ADgIokp3LG7PlnAI4HcA8ANtk2xI6hwAqABwQ/1suPi5Pg53nAdCJj/9HZuc08TdvBNAqaoE2HboQzk5xexOAFRBqiqoyfT6DbEvHD0HlL/EpAFbInt8J4M5M2yWz558AzgWwG0C9uK0ewG7x8aMArpO9PvC6NNg2DsAqAGcDeF38MvbIfliBcyt+gU8RH+vE17E02VkqCikL2Z5V5xSC4LeLP16deE7Pz6ZzCqAlREgTOocArgPwqGx70OvUsjNk35UAnhEfB/3epXOaLl0IZyeAFwHMBnAIo4Kf0fMp/eVDSEf6kUkcFbdlHPEWfS6AdQBqOecd4q4TAGrFx5m0/38B3A7ALz6vBDDAOfeGsSVgp7h/UHx9OmgF0A3gr2L46S+MMQuy7Jxyzo8BeBDAEQAdEM7RRmTnOZVI9Bxmw+/tJgjeMqLYkxE7GWOXAzjGOf8iZFdW2JkPgp+VMMaKAbwE4Iec8yH5Pi5cyjOaHsUYuwRAF+d8YybtiBMdhFvnP3HO5wKwQwg/BMiSc1oO4HIIF6gGABYAF2TSpkTIhnMYC8bYcgBeAM9k2pZQGGNmAD8F8F+ZtiUS+SD4xyDEzCTGidsyBmNMD0Hsn+Gcvyxu7mSM1Yv76wF0idszZf+pAC5jjB0C8DyEsM5DAMoYY9Jwe7ktATvF/aUAetNgJyB4PUc55+vE5y9CuABk2zldBuAg57ybc+4B8DKE85yN51Qi0XOYsd8bY+xGAJcAuF68OCGKPZmwcyKEi/0X4iJYuiYAAAGnSURBVO9qHIDPGWN12WJnPgj+ZwDaxEwIA4TFr9cyZQxjjAF4HMBOzvlvZLteAyCtwH8dQmxf2v41cRV/MYBB2S22anDO7+Scj+Oct0A4Z6s559cDeA/A1RHslOy/Wnx9WrxBzvkJAO2MsSnipnMA7ECWnVMIoZzFjDGz+D2Q7My6cyoj0XO4AsB5jLFy8Y7mPHGbqjDGLoAQfryMc+4Isf9aMeOpFUAbgPXIgC5wzrdyzms45y3i7+oohASOE8iW86nW4kA6/yCsgO+BsCq/PMO2nAbhtngLgM3i30UQYrOrAOwFsBJAhfh6BuAPou1bASzIgM1nYjRLZwKEH8w+AP8AYBS3m8Tn+8T9E9Js4xwAG8Tz+iqEjIasO6cA7gKwC8A2AE9ByB7JinMK4DkIawseCGJ0czLnEEIMfZ/494002bkPQqxb+k09Inv9ctHO3QAulG1XVRfC2Rmy/xBGF20zdj7lf1RpSxAEUSDkQ0iHIAiCiAMSfIIgiAKBBJ8gCKJAIMEnCIIoEEjwCYIgCgQSfIIgiAKBBJ8gCKJAIMEnCIIoEP4/qncalqwrNZAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(range(1440), temp[:1440])  # 24(시간) * 60(분) / 10(분) * 10(일)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이 그래프에서 일별 주기성을 볼 수 있습니다. 특히 마지막 4일간을 보면 확실합니다. 이 데이터는 아주 추운 겨울 중 10일입니다.\n",
    "\n",
    "지난 몇 달간의 데이터를 사용해 다음 달의 평균 온도를 예측하는 문제는 쉬운 편입니다. 연간 데이터 주기성은 안정적이기 때문입니다. 하지만 하루 하루 데이터를 살펴보면 온도 변화는 매우 불안정합니다. 일자별 수준의 시계열 데이터를 예측할 수 있을까요? 직접 확인해 보겠습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.3.2 데이터 준비\n",
    "\n",
    "이 문제의 정확한 정의는 다음과 같습니다. `lookback` 타임스텝(하나의 타임스텝은 10분입니다)만큼 이전으로 돌아가서 매 `steps` 타임스텝마다 샘플링합니다. 이 데이터를 바탕으로 `delay` 타임스텝 이후의 온도를 예측할 수 있을까요? 사용할 변수는 다음과 같습니다.\n",
    "\n",
    "- **lookback** = 1440 : 10일 전 데이터로 돌아갑니다.\n",
    "- **steps** = 6 : 한 시간마다 데이터 포인트 하나를 샘플링합니다.\n",
    "- **delay** = 144 : 24시간이 지난 데이터가 타깃이 됩니다.\n",
    "\n",
    "\n",
    "시작하기 전에 두 가지 작업을 처리해야 합니다:\n",
    "\n",
    "- 신경망에 주입할 수 있는 형태로 데이터를 전처리합니다. 데이터가 이미 수치형이므로 추가적인 벡터화가 필요하지 않습니다. 하지만 데이터에 있는 각 시계열 특성의 범위가 서로 다릅니다(예를 들어 온도는 일반적으로 -20도에서 +30도 사이이고, 밀리바(mb)로 측정된 기압은 1,000 근처의 값입니다). 각 시계열 **특성을 개별적으로 정규화**하여 **비슷한 범위를 가진 작은 값**으로 바꾸겠습니다.\n",
    "\n",
    "- float_data 배열을 받아 과거 데이터의 배치와 미래 타깃 온도를 추출하는 파이썬 제너레이터를 만듭니다. 이 데이터셋에 있는 샘플은 중복이 많습니다(샘플 N과 샘플 N + 1은 대부분 타임스텝이 비슷합니다). 모든 샘플을 각기 메모리에 적재하는 것은 낭비가 심하므로 대신에 원본 데이터를 사용해 그때 그때 배치를 만들겠습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Data Normalization\n",
    "\n",
    "각 시계열 특성에 대해 평균을 빼고 표준 편차로 나누어 전처리합니다. 처음 200,000개 타임스텝을 훈련 데이터로 사용할 것이므로 전체 데이터에서 200,000개만 사용하여 평균과 표준 편차를 계산합니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "mean = float_data[:200000].mean(axis=0)\n",
    "float_data -= mean\n",
    "std = float_data[:200000].std(axis=0)\n",
    "float_data /= std"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### generator 함수 만들기\n",
    "\n",
    "다음은 여기서 사용할 제너레이터입니다. 이 제너레이터 함수는 `(samples, targets)` 튜플을 반복적으로 반환합니다. `samples`는 입력 데이터로 사용할 배치이고 `targets`은 이에 대응되는 타깃 온도의 배열입니다. 이 제너레이터 함수는 다음과 같은 매개변수가 있습니다:\n",
    "\n",
    "- `data` : 코드 6-32에서 정규화한 부동 소수 데이터로 이루어진 원본 배열\n",
    "- `lookback` : 입력으로 사용하기 위해 거슬러 올라갈 타임스텝\n",
    "- `delay` : 타깃으로 사용할 미래의 타임스텝\n",
    "- `min_index`와 `max_index` : 추출할 타임스텝의 범위를 지정하기 위한 data 배열의 인덱스. 검증 데이터와 테스트 데이터를 분리하는 데 사용합니다.\n",
    "- `shuffle` : 샘플을 섞을지 시간 순서대로 추출할지 결정합니다.\n",
    "- `batch_size` : 배치의 샘플 수\n",
    "- `step` : 데이터를 샘플링할 타임스텝 간격. 한 시간에 하나의 데이터 포인트를 추출하기 위해 6으로 지정하겠습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generator(data, lookback, delay, min_index, max_index, \n",
    "              shuffle=False, batch_size=128, step=6):\n",
    "    if max_index is None:\n",
    "        max_index = len(data) - delay - 1\n",
    "    i = min_index + lookback\n",
    "    while 1:\n",
    "        if shuffle:\n",
    "            rows = np.random.randint(\n",
    "                min_index + lookback, max_index, size=batch_size)\n",
    "        else:\n",
    "            if i + batch_size >= max_index:\n",
    "                i = min_index + lookback\n",
    "            rows = np.arange(i, min(i + batch_size, max_index))\n",
    "            i += len(rows)\n",
    "            \n",
    "        samples = np.zeros((len(rows), \n",
    "                            lookback // step, \n",
    "                            data.shape[-1]))\n",
    "        targets = np.zeros((len(rows), ))\n",
    "        for j, row in enumerate(rows):\n",
    "            indices = range(rows[j] - lookback, rows[j], step)\n",
    "            samples[j] = data[indices]\n",
    "            targets[j] = data[rows[j] + delay][1]\n",
    "        yield samples, targets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이제 `generator` 함수를 사용해 훈련용, 검증용, 테스트용으로 세 개의 제너레이터를 만들어 보죠. 각 제너레이터는 원본 데이터에서 다른 시간대를 사용합니다. 훈련 제너레이터는 처음 200,000개 타임스텝을 사용하고 검증 제너레이터는 그 다음 100,000개를 사용하고 테스트 제너레이터는 나머지를 사용합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "lookback = 1440  # 10일\n",
    "step = 6  # 1시간 단위\n",
    "delay = 144  # 24시간 (하루)\n",
    "batch_size = 128\n",
    "\n",
    "train_gen = generator(float_data,\n",
    "                      lookback=lookback, \n",
    "                      delay=delay,\n",
    "                      min_index=0,\n",
    "                      max_index=200000,\n",
    "                      shuffle=True,\n",
    "                      step=step,\n",
    "                      batch_size=batch_size)\n",
    "val_gen = generator(float_data, \n",
    "                    lookback=lookback, \n",
    "                    delay=delay,\n",
    "                    min_index=200001,\n",
    "                    max_index=300000,\n",
    "                    step=step, \n",
    "                    batch_size=batch_size)\n",
    "test_gen = generator(float_data,\n",
    "                     lookback=lookback,\n",
    "                     delay=delay,\n",
    "                     min_index=300001,\n",
    "                     max_index=None,\n",
    "                     step=step,\n",
    "                     batch_size=batch_size)\n",
    "\n",
    "# 전체 검증 세트를 순회하기 위해 val_gen에서 추출할 횟수\n",
    "val_steps = (300000 - 200001 - lookback) // batch_size\n",
    "\n",
    "# 전체 테스트 세트를 순회하기 위해 test_gen에서 추출할 횟수\n",
    "test_steps = (len(float_data) - 300001 - lookback) // batch_size"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.3.3 상식 수준의 기준점(baseline)\n",
    "\n",
    "블랙 박스 같은 딥러닝 모델을 사용해 온도 예측 문제를 풀기 전에 간단한 상식 수준의 해법을 시도해 보겠습니다. 이는 정상 여부 확인을 위한 용도이고 고수준 머신 러닝 모델이라면 뛰어 넘어야 할 **기준점(baseline)**을 만듭니다. \n",
    "\n",
    "이런 상식 수준의 해법은 알려진 해결책이 없는 새로운 문제를 다루어야 할 때 유용합니다. \n",
    "- 일부 클래스가 월등히 많아 불균형한 분류 문제가 고전적인 예입니다. \n",
    "- 데이터셋에 클래스 A의 샘플이 90%, 클래스 B의 샘플이 10%가 있다면, 이 분류 문제에 대한 상식 수준의 접근법은 새로운 샘플을 항상 클래스 'A'라고 예측하는 것입니다. \n",
    "- 이 분류기는 전반적으로 90%의 정확도를 낼 것입니다. \n",
    "- 머신 러닝 기반의 모델이라면 90% 이상을 달성해야 유용하다고 볼 수 있습니다. \n",
    "- 이따금 이런 기본적인 기준점을 넘어서기가 아주 어려운 경우가 있습니다.\n",
    "\n",
    "이 경우 온도 시계열 데이터는 연속성이 있고 일자별로 주기성을 가진다고 가정할 수 있습니다(오늘 온도는 내일 온도와 비슷할 가능성이 높습니다). 그렇기 때문에 상식 수준의 해결책은 지금으로부터 24시간 후의 온도는 지금과 동일하다고 예측하는 것입니다. 이 방법을 평균 절댓값 오차(MAE)로 평가해 보겠습니다:\n",
    "\n",
    "```python\n",
    "np.mean(np.abs(preds - targets))\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "다음은 평가를 위한 반복 루프입니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.2897359729905486\n"
     ]
    }
   ],
   "source": [
    "def evaluate_naive_method():\n",
    "    batch_maes = []\n",
    "    for step in range(val_steps):\n",
    "        samples, targets = next(val_gen)\n",
    "        preds = samples[:, -1, 1]\n",
    "        mae = np.mean(np.abs(preds - targets))\n",
    "        batch_maes.append(mae)\n",
    "    print(np.mean(batch_maes))\n",
    "    \n",
    "evaluate_naive_method()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.5672247338393395"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "0.29 * std[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "출력된 MAE는 0.29입니다. 이 온도 데이터는 평균이 0이고 표준 편차가 1이므로 결괏값이 바로 와 닿지는 않습니다. 평균 절댓값 오차 0.29에 표준 편차를 곱하면 섭씨 2.57°C가 됩니다. 평균 절댓값 오차가 상당히 크네요. 이제 딥러닝 모델이 더 나은지 시도해 봅시다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.3.4 기본적인 머신러닝 방법\n",
    "\n",
    "머신 러닝 모델을 시도하기 전에 상식 수준의 기준점을 세워 놓았습니다. 비슷하게 RNN처럼 복잡하고 연산 비용이 많이 드는 모델을 시도하기 전에 간단하고 손쉽게 만들 수 있는 머신 러닝 모델(예를 들면 소규모의 완전 연결 네트워크)을 먼저 만드는 것이 좋습니다. 이를 바탕으로 더 복잡한 방법을 도입하는 근거가 마련되고 실제적인 이득도 얻게 될 것입니다.\n",
    "\n",
    "다음 코드는 데이터를 펼쳐서 두 개의 Dense 층을 통과시키는 완전 연결 네트워크를 보여줍니다. 전형적인 회귀 문제이므로 마지막 Dense 층에 활성화 함수를 두지 않았습니다. 손실 함수는 MAE입니다. 상식 수준의 방법에서 사용한 것과 동일한 데이터와 지표를 사용했으므로 결과를 바로 비교해볼 수 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "500/500 [==============================] - 7s 14ms/step - loss: 1.6410 - val_loss: 0.7259\n",
      "Epoch 2/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.5548 - val_loss: 0.3111\n",
      "Epoch 3/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.3126 - val_loss: 0.3228\n",
      "Epoch 4/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.2747 - val_loss: 0.3247\n",
      "Epoch 5/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.2613 - val_loss: 0.3353\n",
      "Epoch 6/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.2512 - val_loss: 0.3135\n",
      "Epoch 7/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.2429 - val_loss: 0.3098\n",
      "Epoch 8/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.2380 - val_loss: 0.3259\n",
      "Epoch 9/20\n",
      "500/500 [==============================] - 6s 12ms/step - loss: 0.2313 - val_loss: 0.3198\n",
      "Epoch 10/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.2261 - val_loss: 0.3250\n",
      "Epoch 11/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.2212 - val_loss: 0.3380\n",
      "Epoch 12/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.2175 - val_loss: 0.3485\n",
      "Epoch 13/20\n",
      "500/500 [==============================] - 6s 12ms/step - loss: 0.2170 - val_loss: 0.3730\n",
      "Epoch 14/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.2126 - val_loss: 0.3322\n",
      "Epoch 15/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.2113 - val_loss: 0.3526\n",
      "Epoch 16/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.2067 - val_loss: 0.3334\n",
      "Epoch 17/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.2070 - val_loss: 0.3345\n",
      "Epoch 18/20\n",
      "500/500 [==============================] - 5s 11ms/step - loss: 0.2040 - val_loss: 0.3454\n",
      "Epoch 19/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.2021 - val_loss: 0.3420\n",
      "Epoch 20/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.2010 - val_loss: 0.3507\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras import layers\n",
    "from keras.optimizers import RMSprop\n",
    "from keras import backend as K\n",
    "\n",
    "K.clear_session()\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.Flatten(input_shape=(lookback // step, float_data.shape[-1])))\n",
    "model.add(layers.Dense(32, activation='relu'))\n",
    "model.add(layers.Dense(1))\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae')\n",
    "history = model.fit_generator(train_gen,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=20,\n",
    "                              validation_data=val_gen,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3X2YE+W9//H3l+X5QZ4VBWXxmeUZV9QfUkD50ZVWPbTUilirraJetdZaz68c9aiHluuotdZqrS3tUVtFqEeP1lYUPZWW2lZhsQgiIqigKwgLCoKAuOz398edLGFJNtlNNtmd/byuK1cmk3tm7kySz9y5ZzJj7o6IiERLq0JXQEREck/hLiISQQp3EZEIUriLiESQwl1EJIIU7iIiEaRwl6TMrMjMdprZUbksW0hmdqyZ5fzYXzObYGbrEh6vNrMxmZRtwLJ+bWbXN3T6Oub7QzN7MNfzlcJpXegKSG6Y2c6Ehx2BT4F9sceXu/uc+szP3fcBnXNdtiVw9xNyMR8zuxS40N3HJcz70lzMW6JP4R4R7l4TrrGW4aXu/r+pyptZa3evykfdRCT/1C3TQsR+dv/OzOaa2Q7gQjM7zcxeMrNtZrbRzO42szax8q3NzM2sOPb44djzz5jZDjP7h5kNqG/Z2PNnmdmbZrbdzO4xs7+Z2cUp6p1JHS83s7Vm9pGZ3Z0wbZGZ/cTMtprZ20BZHevnBjObV2vcvWZ2Z2z4UjNbFXs9b8Va1anmVWFm42LDHc3soVjdVgIn1Sp7o5m9HZvvSjM7JzZ+CPAzYEysy2tLwrq9JWH6K2KvfauZPWlmh2eybtIxs8mx+mwzsxfM7ISE5643sw1m9rGZvZHwWk81s1di4zeZ2Y8yXZ40AnfXLWI3YB0woda4HwJ7gbMJG/UOwMnAKYRfcEcDbwJXxcq3Bhwojj1+GNgClAJtgN8BDzeg7KHADuDc2HPXAp8BF6d4LZnU8fdAV6AY+DD+2oGrgJVAP6AnsCh85JMu52hgJ9ApYd6bgdLY47NjZQw4A9gNDI09NwFYlzCvCmBcbPgO4M9Ad6A/8HqtsucBh8fekwtidTgs9tylwJ9r1fNh4JbY8MRYHYcD7YGfAy9ksm6SvP4fAg/GhgfG6nFG7D26HlgdGx4ErAf6xMoOAI6ODS8BpsaGuwCnFPq70JJvarm3LC+6+x/cvdrdd7v7End/2d2r3P1tYDYwto7pH3P3cnf/DJhDCJX6lv0isMzdfx977ieEDUFSGdbxP919u7uvIwRpfFnnAT9x9wp33wrcWsdy3gZeI2x0AP4v8JG7l8ee/4O7v+3BC8CfgKQ7TWs5D/ihu3/k7usJrfHE5T7q7htj78kjhA1zaQbzBZgG/Nrdl7n7HmAGMNbM+iWUSbVu6nI+8JS7vxB7j24lbCBOAaoIG5JBsa69d2LrDsJG+jgz6+nuO9z95QxfhzQChXvL8l7iAzM70cyeNrMPzOxjYCbQq47pP0gY3kXdO1FTlT0isR7u7oSWblIZ1jGjZRFanHV5BJgaG74g9jhejy+a2ctm9qGZbSO0mutaV3GH11UHM7vYzF6NdX9sA07McL4QXl/N/Nz9Y+AjoG9Cmfq8Z6nmW014j/q6+2rge4T3YXOsm69PrOglQAmw2swWm9mkDF+HNAKFe8tS+zDAXxJaq8e6+yHATYRuh8a0kdBNAoCZGQeGUW3Z1HEjcGTC43SHaj4KTDCzvoQW/COxOnYAHgP+k9Bl0g14LsN6fJCqDmZ2NHAfcCXQMzbfNxLmm+6wzQ2Erp74/LoQun/ez6Be9ZlvK8J79j6Auz/s7qMJXTJFhPWCu6929/MJXW8/Bh43s/ZZ1kUaSOHesnUBtgOfmNlA4PI8LPOPwEgzO9vMWgPfAXo3Uh0fBa4xs75m1hP4fl2F3f0D4EXgQWC1u6+JPdUOaAtUAvvM7IvAmfWow/Vm1s3C/wCuSniuMyHAKwnbucsILfe4TUC/+A7kJOYC3zSzoWbWjhCyf3X3lL+E6lHnc8xsXGzZ/0rYT/KymQ00s/Gx5e2O3aoJL+BrZtYr1tLfHntt1VnWRRpI4d6yfQ/4OuGL+0vCjs9G5e6bgK8CdwJbgWOAfxKOy891He8j9I2vIOzseyyDaR4h7CCt6ZJx923Ad4EnCDslpxA2Upm4mfALYh3wDPDbhPkuB+4BFsfKnAAk9lM/D6wBNplZYvdKfPpnCd0jT8SmP4rQD58Vd19JWOf3ETY8ZcA5sf73dsDthP0kHxB+KdwQm3QSsMrC0Vh3AF91973Z1kcaxkKXp0hhmFkRoRtgirv/tdD1EYkKtdwl78ysLNZN0Q74d8JRFosLXC2RSFG4SyGcDrxN+Mn/eWCyu6fqlhGRBlC3jIhIBKnlLiISQQU7cVivXr28uLi4UIsXEWmWli5dusXd6zp8GChguBcXF1NeXl6oxYuINEtmlu6f1oC6ZUREIknhLiISQQp3EZEI0pWYRFqIzz77jIqKCvbs2VPoqkgG2rdvT79+/WjTJtWpheqmcBdpISoqKujSpQvFxcWEk3FKU+XubN26lYqKCgYMGJB+giSaVbfMnDlQXAytWoX7OfW65LNIy7Znzx569uypYG8GzIyePXtm9Sur2bTc58yB6dNh167weP368BhgWtbnwRNpGRTszUe271WzabnfcMP+YI/btSuMFxGRAzWbcH/33fqNF5GmZevWrQwfPpzhw4fTp08f+vbtW/N4797MTvt+ySWXsHr16jrL3HvvvczJUZ/t6aefzrJly3Iyr3xrNt0yRx0VumKSjReR3JszJ/wyfvfd8D2bNSu7LtCePXvWBOUtt9xC586due666w4o4+64O61aJW93PvDAA2mX861vfavhlYyQZtNynzULOnY8cFzHjmG8iORWfB/X+vXgvn8fV2McxLB27VpKSkqYNm0agwYNYuPGjUyfPp3S0lIGDRrEzJkza8rGW9JVVVV069aNGTNmMGzYME477TQ2b94MwI033shdd91VU37GjBmMGjWKE044gb///e8AfPLJJ3z5y1+mpKSEKVOmUFpamraF/vDDDzNkyBAGDx7M9ddfD0BVVRVf+9rXasbffffdAPzkJz+hpKSEoUOHcuGFF+Z8nWUibcvdzO4HvghsdvfBKcqMA+4C2gBb3H1sLisJ+1sMuWxJiEhyde3jaozv3BtvvMFvf/tbSktLAbj11lvp0aMHVVVVjB8/nilTplBSUnLANNu3b2fs2LHceuutXHvttdx///3MmDHjoHm7O4sXL+app55i5syZPPvss9xzzz306dOHxx9/nFdffZWRI0fWWb+KigpuvPFGysvL6dq1KxMmTOCPf/wjvXv3ZsuWLaxYsQKAbdu2AXD77bezfv162rZtWzMu3zJpuT9IuIZiUmbWDfg54RqLg4Cv5KZqB5s2Ddatg+rqcK9gF2kc+d7Hdcwxx9QEO8DcuXMZOXIkI0eOZNWqVbz++usHTdOhQwfOOussAE466STWrVuXdN5f+tKXDirz4osvcv755wMwbNgwBg0aVGf9Xn75Zc444wx69epFmzZtuOCCC1i0aBHHHnssq1ev5uqrr2bBggV07doVgEGDBnHhhRcyZ86cBv8JKVtpw93dFxEuCpzKBcD/uPu7sfKbc1Q3ESmQVPuyGmsfV6dOnWqG16xZw09/+lNeeOEFli9fTllZWdLjvdu2bVszXFRURFVVVdJ5t2vXLm2ZhurZsyfLly9nzJgx3HvvvVx++eUALFiwgCuuuIIlS5YwatQo9u3bl9PlZiIXfe7HA93N7M9mttTMLkpV0Mymm1m5mZVXVlbmYNEi0hgKuY/r448/pkuXLhxyyCFs3LiRBQsW5HwZo0eP5tFHHwVgxYoVSX8ZJDrllFNYuHAhW7dupaqqinnz5jF27FgqKytxd77yla8wc+ZMXnnlFfbt20dFRQVnnHEGt99+O1u2bGFX7T6uPMjF0TKtgZOAM4EOwD/M7CV3f7N2QXefDcwGKC0t1fX9RJqoQu7jGjlyJCUlJZx44on079+f0aNH53wZ3/72t7nooosoKSmpucW7VJLp168fP/jBDxg3bhzuztlnn80XvvAFXnnlFb75zW/i7pgZt912G1VVVVxwwQXs2LGD6upqrrvuOrp06ZLz15BORtdQNbNi4I/Jdqia2Qygg7vfHHv8X8Cz7v7fdc2ztLTUdbEOkfxZtWoVAwcOLHQ1moSqqiqqqqpo3749a9asYeLEiaxZs4bWrZvW0eHJ3jMzW+rupSkmqZGLV/J74Gdm1hpoC5wC/CQH8xURaRQ7d+7kzDPPpKqqCnfnl7/8ZZML9mxlcijkXGAc0MvMKoCbCYc84u6/cPdVZvYssByoBn7t7q81XpVFRLLTrVs3li5dWuhqNKq04e7uUzMo8yPgRzmpkYiIZK3Z/ENVREQyp3AXEYkghbuISAQp3EUkL8aPH3/QH5Luuusurrzyyjqn69y5MwAbNmxgypQpScuMGzeOdIdW33XXXQf8mWjSpEk5Oe/LLbfcwh133JH1fHJN4S4ieTF16lTmzZt3wLh58+YxdWraYzYAOOKII3jssccavPza4T5//ny6devW4Pk1dQp3EcmLKVOm8PTTT9dcmGPdunVs2LCBMWPG1Bx3PnLkSIYMGcLvf//7g6Zft24dgweH/1Hu3r2b888/n4EDBzJ58mR2795dU+7KK6+sOV3wzTffDMDdd9/Nhg0bGD9+POPHjweguLiYLVu2AHDnnXcyePBgBg8eXHO64HXr1jFw4EAuu+wyBg0axMSJEw9YTjLLli3j1FNPZejQoUyePJmPPvqoZvnxUwDHT1j2l7/8peZiJSNGjGDHjh0NXrfJROuofRHJyDXXQK4vMDR8OMRyMakePXowatQonnnmGc4991zmzZvHeeedh5nRvn17nnjiCQ455BC2bNnCqaeeyjnnnJPyOqL33XcfHTt2ZNWqVSxfvvyAU/bOmjWLHj16sG/fPs4880yWL1/O1VdfzZ133snChQvp1avXAfNaunQpDzzwAC+//DLuzimnnMLYsWPp3r07a9asYe7cufzqV7/ivPPO4/HHH6/z/OwXXXQR99xzD2PHjuWmm27iP/7jP7jrrru49dZbeeedd2jXrl1NV9Add9zBvffey+jRo9m5cyft27evx9pOTy13EcmbxK6ZxC4Zd+f6669n6NChTJgwgffff59NmzalnM+iRYtqQnbo0KEMHTq05rlHH32UkSNHMmLECFauXJn2pGAvvvgikydPplOnTnTu3JkvfelL/PWvfwVgwIABDB8+HKj7tMIQzi+/bds2xo4Nl7P4+te/zqJFi2rqOG3aNB5++OGaf8KOHj2aa6+9lrvvvptt27bl/B+yarmLtEB1tbAb07nnnst3v/tdXnnlFXbt2sVJJ50EwJw5c6isrGTp0qW0adOG4uLipKf5Teedd97hjjvuYMmSJXTv3p2LL764QfOJi58uGMIpg9N1y6Ty9NNPs2jRIv7whz8wa9YsVqxYwYwZM/jCF77A/PnzGT16NAsWLODEE09scF1rU8tdRPKmc+fOjB8/nm984xsH7Ejdvn07hx56KG3atGHhwoWsT3bB5ASf+9zneOSRRwB47bXXWL58ORBOF9ypUye6du3Kpk2beOaZZ2qm6dKlS9J+7TFjxvDkk0+ya9cuPvnkE5544gnGjBlT79fWtWtXunfvXtPqf+ihhxg7dizV1dW89957jB8/nttuu43t27ezc+dO3nrrLYYMGcL3v/99Tj75ZN544416L7MuarmLSF5NnTqVyZMnH3DkzLRp0zj77LMZMmQIpaWlaVuwV155JZdccgkDBw5k4MCBNb8Ahg0bxogRIzjxxBM58sgjDzhd8PTp0ykrK+OII45g4cKFNeNHjhzJxRdfzKhRowC49NJLGTFiRJ1dMKn85je/4YorrmDXrl0cffTRPPDAA+zbt48LL7yQ7du34+5cffXVdOvWjX//939n4cKFtGrVikGDBtVcVSpXMjrlb2PQKX9F8kun/G1+sjnlr7plREQiSOEuIhJBCneRFqRQ3bBSf9m+Vwp3kRaiffv2bN26VQHfDLg7W7duzeqPTTpaRqSF6NevHxUVFVRWVha6KpKB9u3b069fvwZPr3AXaSHatGnDgAEDCl0NyZO03TJmdr+ZbTazOq+LamYnm1mVmSU/J6eIiORNJn3uDwJldRUwsyLgNuC5HNRJRESylDbc3X0R8GGaYt8GHgc256JSIiKSnayPljGzvsBk4L4Myk43s3IzK9dOHRGRxpOLQyHvAr7v7tXpCrr7bHcvdffS3r1752DRIiKSTC6OlikF5sVOqt8LmGRmVe7+ZA7mLSIiDZB1uLt7zbFVZvYg8EcFu4hIYaUNdzObC4wDeplZBXAz0AbA3X/RqLUTEZEGSRvu7p7ZpclD2Yuzqo2IiOSEzi0jIhJBCncRkQhSuIuIRJDCXUQkghTuIiIRpHAXEYkghbuISAQp3EVEIkjhLiISQQp3EZEIUriLiESQwl1EJIIU7iIiEaRwFxGJIIW7iEgEKdxFRCJI4S4iEkEKdxGRCFK4i4hEUNpwN7P7zWyzmb2W4vlpZrbczFaY2d/NbFjuqykiIvWRScv9QaCsjuffAca6+xDgB8DsHNRLRESy0DpdAXdfZGbFdTz/94SHLwH9sq+WiIhkI9d97t8Enkn1pJlNN7NyMyuvrKzM8aJFRCQuZ+FuZuMJ4f79VGXcfba7l7p7ae/evXO1aBERqSVtt0wmzGwo8GvgLHffmot5iohIw2Xdcjezo4D/Ab7m7m9mXyUREclW2pa7mc0FxgG9zKwCuBloA+DuvwBuAnoCPzczgCp3L22sCouISHqZHC0zNc3zlwKX5qxGIiKSNf1DVUQkghTuIiIRpHAXEYkghbuISAQp3EVEIkjhLiISQQp3EZEIUriLiESQwl1EJIIU7iIiEaRwFxGJIIW7iEgEKdxFRCJI4S4iEkEKdxGRCFK4i4hEkMJdRCSCFO4iIhGUNtzN7H4z22xmr6V43szsbjNba2bLzWxk7qspIiL1kUnL/UGgrI7nzwKOi92mA/dlXy0REclG2nB390XAh3UUORf4rQcvAd3M7PBcVVBEROovF33ufYH3Eh5XxMYdxMymm1m5mZVXVlbmYNEiIpJMXneouvtsdy9199LevXvnc9EiIi1KLsL9feDIhMf9YuNERKRAchHuTwEXxY6aORXY7u4bczBfERFpoNbpCpjZXGAc0MvMKoCbgTYA7v4LYD4wCVgL7AIuaazKiohIZtKGu7tPTfO8A9/KWY1ERCRr+oeqiEgEKdxFRCJI4S4iEkEKdxGRCFK4i4hEkMJdRCSCFO4iIhGkcBcRiSCFu4hIBCncRUQiSOEuIhJBCncRkQhSuIuIRJDCXUQkghTuIiIRpHAXEYkghbuISAQp3EVEIkjhLiISQRmFu5mVmdlqM1trZjOSPH+UmS00s3+a2XIzm5T7qoqISKbShruZFQH3AmcBJcBUMyupVexG4FF3HwGcD/w81xUVEZHMZdJyHwWsdfe33X0vMA84t1YZBw6JDXcFNuSuiiIiUl+ZhHtf4L2ExxWxcYluAS40swpgPvDtZDMys+lmVm5m5ZWVlQ2oroiIZCJXO1SnAg+6ez9gEvCQmR00b3ef7e6l7l7au3fvHC1aRERqyyTc3weOTHjcLzYu0TeBRwHc/R9Ae6BXLiooIiL1l0m4LwGOM7MBZtaWsMP0qVpl3gXOBDCzgYRwV7+LiEiBpA13d68CrgIWAKsIR8WsNLOZZnZOrNj3gMvM7FVgLnCxu3tjVVpEROrWOpNC7j6fsKM0cdxNCcOvA6NzWzUREWko/UNVRCSCFO4iIhHULMN9375C10BEpGlrduH+5JPQuzds0H9gRURSanbhPmAAfPQRPPdcoWsiItJ0NbtwHzoU+vSBZ58tdE1ERJquZhfuZlBWFlru6nsXEUmu2YU7hHD/6CNYsqTQNRERaZqaZbhPmACtWqlrRkQklWYZ7j17wqhRCncRkVSaZbhD6JpZvBi2bi10TUREmp5mHe7u8Pzzha6JiEjT02zDvbQUevRQ14yISDLNNtyLimDixBDu1dWFro2ISNPSbMMdQtfMpk2wfHmhayIi0rQ063CfODHcq2tGRORAzTrcDz8chg9XuIuI1Naswx1C18zf/gYff1zomoiINB0ZhbuZlZnZajNba2YzUpQ5z8xeN7OVZvZIbquZWlkZVFXBn/6UryWKiDR9acPdzIqAe4GzgBJgqpmV1CpzHPBvwGh3HwRc0wh1Teq006BLF3XNiIgkyqTlPgpY6+5vu/teYB5wbq0ylwH3uvtHAO6+ObfVTK1tWzjzzBDu7vlaqohI05ZJuPcF3kt4XBEbl+h44Hgz+5uZvWRmZclmZGbTzazczMorKysbVuMkysrg3XfhjTdyNksRkWYtVztUWwPHAeOAqcCvzKxb7ULuPtvdS929tHfv3jlaNHz+8+FeXTMiIkEm4f4+cGTC436xcYkqgKfc/TN3fwd4kxD2eVFcDCeeqHAXEYnLJNyXAMeZ2QAzawucDzxVq8yThFY7ZtaL0E3zdg7rmVZZGfzlL7BrV+oyc+aEDUGrVuF+zpx81U5EJL/Shru7VwFXAQuAVcCj7r7SzGaa2TmxYguArWb2OrAQ+Fd3z+vJeMvK4NNPQ8AnM2cOTJ8O69eHHa/r14fHCngRiSLzAh1iUlpa6uXl5Tmb3+7d4SyR06fDT3968PPFxSHQa+vfH9aty1k1REQalZktdffSdOWa/T9U4zp0gHHjUve7v/tu/caLiDRnkQl3CF0zb74Jbyfp7T/qqOTTpBovItKcRS7cARYsOPi5WbOgY8cDx3XsGMaLiERNpML9+OND33qyrplp02D27NDHbhbuZ88O40VEoqZ1oSuQS2ah9f7QQ7B3bzg1QaJp0xTmItIyRKrlDiHcP/kknAZYRKSlily4n3EGtGmjf6uKSMsWuXDv0gVOP13hLiItW+TCHULXzPLlsGFDoWsiIlIYkQ13SH5IpIhISxDJcB8yJFw8W10zItJSRTLc44dEPv98uL6qiEhLE8lwhxDuH30ES5YUuiYiIvkX2XCfMCGct11dMyLSEkU23Hv0gFNOUbiLSMsU2XCH0DWzZAls2VLomoiI5Ffkw9097FgVEWlJIh3uJ50EPXuqa0ZEWp6Mwt3MysxstZmtNbMZdZT7spm5maW9BFQ+FBXBxInhz0zV1YWujYhI/qQNdzMrAu4FzgJKgKlmVpKkXBfgO8DLua5kNsrKYNMmePXVQtdERCR/Mmm5jwLWuvvb7r4XmAecm6TcD4DbgD05rF/WJk4M9+qaEZGWJJNw7wu8l/C4IjauhpmNBI5096dzWLec6NMHRoxQuItIy5L1DlUzawXcCXwvg7LTzazczMorKyuzXXTGysrg73+H7dvztkgRkYLKJNzfB45MeNwvNi6uCzAY+LOZrQNOBZ5KtlPV3We7e6m7l/bu3bvhta6nsrJwjpkXXsjbIkVECiqTcF8CHGdmA8ysLXA+8FT8SXff7u693L3Y3YuBl4Bz3L28UWrcAKedFi7ioa4Zac5274Z//AN+9Sv45z/DfzhEUkl7gWx3rzKzq4AFQBFwv7uvNLOZQLm7P1X3HAqvTZtwrplnnw1fCLNC10ikbvv2weuvw+LF4V/WixfDihUHnuX0hBPgggtg6lQ47rjC1VWaJvMCbf5LS0u9vDx/jfvZs+Hyy8MXZuDAvC1WJC13WLduf4gvXgyvvBIu9A7QtSucfDKMGhXuS0rgz3+GuXPhL38J05eWhpD/6lehb9+6libNnZktdfe0/yVqMeG+fj0UF8Odd8J3v5u3xYocoLoaNm8O4Z0Y5vHzH7VrF47uiof5qFFw7LHhDKfJvP8+/O538MgjsHRp+FU6dmxo0X/5y+EEetlyhw8+CL8cVqyA1avh0ENDI6mkJPyC6Ngx++VEUXV1OJBj69Zw27Il3A8cGN7jhlC4J1FSAkceqcvvSW5VV4cv7KZNIQQ3bTrwljhu8+bQ5QIhiEtK9of4ySeHq4i1bduwerz5ZmjNP/JIGG7TJhxMMHUqnHMOdOqUfh4ffwwrV+4P8hUr4LXXwuuL69kzBFa8i8gMBgwIryXxNnAgdO7csNeSrerqcD2HDz44+LZ5c9hgtWsXbu3bH3ifajhx3M6d+wO7rtuHHyb/d/x118GPftSw16ZwT+Laa+HnPw8rvLm1NPbtC1v9TZvCF+v44+Gwwwpdq2hzD5+Vigp4771wH7/Fg2LTJqis3B/Yidq2De9R7VufPjBsGIwcGXb0N0a9//nPEPLz5oXWfceO8C//EoI+/se+1asPDPAVK8Iv3LjOnWHw4LDBSbzv3Rv27oW1a0M3Z+Jt9erwXNxRRx0c+ieeGDY0+/aF4Kuurt/wvn0hOBPfg9oBvmlT8quwtWsXfnUUFcGePfDpp+G2Z092pyjp0CFs9DK99enT8Pde4Z7Ec8/B5z8P8+fDWWflddFJJX5Ik7XyEocrKw/+8B12GAwfHm7DhoX7448PH1ypm3tY9/GwTgzvxOHduw+crqgoXJ/38MMPDOtkId6tW+F33ldXw1//GoL+scfCxqpLl/C64uHXunUI3HiAx0O8f//U3UGpVFXB228fHPqrVoUAbQxFRSGw+/TZf4u/L7XHde2a+j2pqjo48GsPx+87dz4wrDt0aJzXlozCPYk9e0If5GWXwU9/mr68O2zbBhs2hNvjj4f+zW3bwhfk5JPDz9GqqnDbt2//cF23Tz8NPw2TBTaE1kViYNQe7tw5tJCWLQu3lSvhs8/CtB06hC9mPPSHDw9f1sZoIdbl00/DTsK33jrwtnZtCM+2beGQQ0K94vephmuP69QpzH/nzrDTcefOA4fT3e/YEd7P2mFTVBR2RvbrF7rv+vU7eLhPn+a78dy7NzRw/vCHEEjxMD/hhIZ3BWVq377wq+D11+GNN8L7V1QUNh6tWqUfrj2uR4/9od2zZ/03Qs2Zwj2FSZNCwCxeHL7gGzfuD+/ajzduTN/a6N49hE7r1pndiorCF6l37+TBfdhhIcDq0+Lbuzfjz/qAAAAIi0lEQVR8YZYtCydIi4f+hx/uL3Pssftb+EOGhBZM+/bJbx06hDqmq8PHHx8c3okBnvjR6tQJjjkm3Pr3DxujHTvCPBLv48M7d2Z3HHeHDmGZnTsffN+5c2h5x0M7fn/YYc03uKXlyDTc0x7nHjVlZfCd74RQrq1LFzjiiPDFP+20MBx/fM01oXuktkMOCS3UQmrbFoYODbc499CtkBj2y5aFn+aZShX+RUXw7rvhl0eiXr3CRmTMmP1BfswxYdyhh9Zvg1VdDbt2HRz6O3aEFnj79qnDu2NHhbRIi2u5b98OP/5xaLnGgzt+X1fXRatWyVuSZs3rXPE7doRW/iefhF8lyW67d6d+bs+e0Oru2zeEdmKIH3JIoV+dSPSp5Z5C164wc2b9pzvqqAOPJEgc35zE9xWISLS1oN0Q2Zk16+DDJzt2DOMzNWdO+CNVq1bhfs6cXNZQRGQ/hXuGpk0LpzDo3z90xfTvHx5Pm5bZ9HPmwPTpofXvHu6nT1fAi0jjULjXw7RpYedpdXW4zzTYAW64IewgTLRrVxifKbX8RSRTCvc8effd+o2vLVctf20gRFoGhXuepNrxmukO2Vy1/LPdQGjjINI8KNzzJNsdstm2/CH7DYQ2DiLNh8I9T7LdIZttyx+y30BEYeOgjYu0GO5ekNtJJ53kkrmHH3bv2NE9xGK4dewYxmeqf/8Dp4/f+vfPbHqz5NOb5Wf52a6DXKzDhx8O9TUL9/WZViQXCFfAS5uxCvdmJNtgyTbcmvvGodAbl/g8sn0PCzm9FJ7CXZLK5svd3DcOhd64FPqXhzZO0ZDTcAfKgNXAWmBGkuevBV4HlgN/Avqnm6fCvXlqzhuHQm9cCl1/bZwKv3HJxcYpZ+EOFAFvAUcDbYFXgZJaZcYDHWPDVwK/SzdfhXvLVMiNQ6E3LoX+5aGNU/PfOLnnNtxPAxYkPP434N/qKD8C+Fu6+SrcpSEK2XIq9Mah0NMXeuPS0jdOcbkM9ynArxMefw34WR3lfwbcmOK56UA5UH7UUUfV7xWJNAHN+ZeHNk7Ne+MUV5BwBy4EXgLapZuvWu7SEhW6z1Ybp+a7cYrLe7cMMAFYBRyayYIV7iLNjzZOhZs+Lpfh3hp4GxiQsEN1UK0yI2I7XY/LZKGucBeRAmjOG6e4TMM9o8vsmdkk4K7YkTP3u/ssM5sZW8hTZva/wBBgY2ySd939nLrmWajL7ImINGc5vcyeu88H5tcad1PC8IR611BERBqNThwmIhJBCncRkQhSuIuIRJDCXUQkgjI6WqZRFmxWCawvyMLT6wVsKXQl6tDU6wdNv46qX3ZUv+xkU7/+7t47XaGChXtTZmblmRxqVChNvX7Q9Ouo+mVH9ctOPuqnbhkRkQhSuIuIRJDCPbnZha5AGk29ftD066j6ZUf1y06j10997iIiEaSWu4hIBCncRUQiqMWGu5kdaWYLzex1M1tpZt9JUmacmW03s2Wx203J5tWIdVxnZitiyz7oFJoW3G1ma81suZmNzGPdTkhYL8vM7GMzu6ZWmbyvPzO738w2m9lrCeN6mNnzZrYmdt89xbRfj5VZY2Zfz2P9fmRmb8TewyfMrFuKaev8PDRi/W4xs/cT3sdJKaYtM7PVsc/jjDzW73cJdVtnZstSTNuo6y9VphTs85fJeYGjeAMOB0bGhrsAb3Lwhb/HAX8sYB3XAb3qeH4S8AxgwKnAywWqZxHwAeHPFQVdf8DngJHAawnjbgdmxIZnALclma4H4boFPYDuseHuearfRKB1bPi2ZPXL5PPQiPW7Bbgug8/AW8DR7L/uQ0k+6lfr+R8DNxVi/aXKlEJ9/lpsy93dN7r7K7HhHYSrSPUtbK3q7Vzgtx68BHQzs8MLUI8zgbfcveD/OHb3RcCHtUafC/wmNvwb4F+STPp54Hl3/9DdPwKeB8ryUT93f87dq2IPXwL65Xq5mUqx/jIxCljr7m+7+15gHmG951Rd9TMzA84D5uZ6uZmoI1MK8vlrseGeyMyKCVeTejnJ06eZ2atm9oyZDcprxcCB58xsqZlNT/J8X+C9hMcVFGYDdT6pv1CFXH9xh7l7/EIyHwCHJSnTVNblNwi/xpJJ93loTFfFuo3uT9Gt0BTW3xhgk7uvSfF83tZfrUwpyOevxYe7mXUGHgeucfePaz39CqGrYRhwD/Bknqt3uruPBM4CvmVmn8vz8tMys7bAOcB/J3m60OvvIB5+AzfJ43/N7AagCpiTokihPg/3AccAwwlXW/txnpZbX1Opu9Wel/VXV6bk8/PXosPdzNoQ3oQ57v4/tZ9394/dfWdseD7Qxsx65at+7v5+7H4z8AThp2+i94EjEx73i43Lp7OAV9x9U+0nCr3+EmyKd1fF7jcnKVPQdWlmFwNfBKbFAuAgGXweGoW7b3L3fe5eDfwqxXILvf5aA18CfpeqTD7WX4pMKcjnr8WGe6x/7r+AVe5+Z4oyfWLlMLNRhPW1NU/162RmXeLDhJ1ur9Uq9hRwUeyomVOB7Qk///IlZWupkOuvlqeA+NEHXwd+n6TMAmCimXWPdTtMjI1rdGZWBvw/4Bx335WiTCafh8aqX+J+nMkplrsEOM7MBsR+zZ1PWO/5MgF4w90rkj2Zj/VXR6YU5vPXWHuOm/oNOJ3w82g5sCx2mwRcAVwRK3MVsJKw5/8l4P/ksX5Hx5b7aqwON8TGJ9bPgHsJRymsAErzvA47EcK6a8K4gq4/woZmI/AZod/ym0BP4E/AGuB/gR6xsqXArxOm/QawNna7JI/1W0vob41/Dn8RK3sEML+uz0Oe6vdQ7PO1nBBUh9euX+zxJMIRIm/ls36x8Q/GP3cJZfO6/urIlIJ8/nT6ARGRCGqx3TIiIlGmcBcRiSCFu4hIBCncRUQiSOEuIhJBCncRkQhSuIuIRND/Bxl5OvYxqLcUAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss) + 1)\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.4.5 첫 번째 순환 신경망\n",
    "\n",
    "첫 번째 완전 연결 네트워크는 잘 작동하지 않았습니다. 그렇다고 이 문제에 머신러닝이 적합하지 않다는 뜻은 아닙니다. **앞선 모델은 시계열 데이터를 펼쳤기 때문에 입력 데이터에서 시간 개념을 잃어버렸습니다.** 대신 인과 관계와 순서가 의미가 있는 시퀀스 데이터 그대로 사용해 보겠습니다. 이런 시퀀스 데이터에 아주 잘 들어맞는 순환 시퀀스 처리 모델을 시도해 보겠습니다. 이 모델은 앞선 모델과 달리 데이터 포인터의 **시간 순서**를 사용합니다.\n",
    "\n",
    "이전 절에서 소개한 LSTM 층 대신에 2014년에 정준영 등이 개발한 GRU 층을 사용하겠습니다. \n",
    "- GRU 층은 LSTM과 같은 원리로 작동하지만 조금 더 간결하고 그래서 계산 비용이 덜 듭니다(LSTM 만큼 표현 학습 능력이 높지는 않을 수 있습니다). \n",
    "- 계산 비용과 표현 학습 능력 사이의 트레이드오프(trade-off)는 머신 러닝 어디에서나 등장합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "gru_1 (GRU)                  (None, 32)                4512      \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              (None, 1)                 33        \n",
      "=================================================================\n",
      "Total params: 4,545\n",
      "Trainable params: 4,545\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Epoch 1/20\n",
      "500/500 [==============================] - 78s 155ms/step - loss: 0.3042 - val_loss: 0.2692\n",
      "Epoch 2/20\n",
      "500/500 [==============================] - 77s 154ms/step - loss: 0.2822 - val_loss: 0.2636\n",
      "Epoch 3/20\n",
      "500/500 [==============================] - 77s 154ms/step - loss: 0.2773 - val_loss: 0.2639\n",
      "Epoch 4/20\n",
      "500/500 [==============================] - 77s 153ms/step - loss: 0.2716 - val_loss: 0.2634\n",
      "Epoch 5/20\n",
      "500/500 [==============================] - 77s 153ms/step - loss: 0.2665 - val_loss: 0.2635\n",
      "Epoch 6/20\n",
      "500/500 [==============================] - 77s 153ms/step - loss: 0.2624 - val_loss: 0.2632\n",
      "Epoch 7/20\n",
      "500/500 [==============================] - 77s 154ms/step - loss: 0.2554 - val_loss: 0.2671\n",
      "Epoch 8/20\n",
      "500/500 [==============================] - 77s 153ms/step - loss: 0.2509 - val_loss: 0.2664\n",
      "Epoch 9/20\n",
      "500/500 [==============================] - 77s 153ms/step - loss: 0.2471 - val_loss: 0.2732\n",
      "Epoch 10/20\n",
      "500/500 [==============================] - 77s 153ms/step - loss: 0.2427 - val_loss: 0.2726\n",
      "Epoch 11/20\n",
      "500/500 [==============================] - 77s 153ms/step - loss: 0.2388 - val_loss: 0.2725\n",
      "Epoch 12/20\n",
      "500/500 [==============================] - 77s 154ms/step - loss: 0.2330 - val_loss: 0.2789\n",
      "Epoch 13/20\n",
      "500/500 [==============================] - 77s 154ms/step - loss: 0.2292 - val_loss: 0.2819\n",
      "Epoch 14/20\n",
      "500/500 [==============================] - 77s 154ms/step - loss: 0.2253 - val_loss: 0.2831\n",
      "Epoch 15/20\n",
      "500/500 [==============================] - 77s 154ms/step - loss: 0.2216 - val_loss: 0.2904\n",
      "Epoch 16/20\n",
      "500/500 [==============================] - 77s 154ms/step - loss: 0.2184 - val_loss: 0.2913\n",
      "Epoch 17/20\n",
      "500/500 [==============================] - 77s 154ms/step - loss: 0.2130 - val_loss: 0.2934\n",
      "Epoch 18/20\n",
      "500/500 [==============================] - 77s 154ms/step - loss: 0.2119 - val_loss: 0.3024\n",
      "Epoch 19/20\n",
      "500/500 [==============================] - 77s 153ms/step - loss: 0.2084 - val_loss: 0.3047\n",
      "Epoch 20/20\n",
      "500/500 [==============================] - 77s 153ms/step - loss: 0.2044 - val_loss: 0.2987\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras import layers\n",
    "from keras.optimizers import RMSprop\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.GRU(32, input_shape=(None, float_data.shape[-1])))  # (None, 14)\n",
    "model.add(layers.Dense(1))\n",
    "model.summary()\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae')\n",
    "history = model.fit_generator(train_gen,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=20,\n",
    "                              validation_data=val_gen,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XmYFNW5x/Hvy46CgKBiGFlcLjgoCo6oUQSUKLiAGDQgiOtFvRg16k24ookaTRSMGpVEibugxCWoMRJicBRcUIZtZBFBBR1EQCIgsg68949TA804Sw/T0z09/fs8Tz/dVXWq+u2anrdPnTp1ytwdERHJDLVSHYCIiCSPkr6ISAZR0hcRySBK+iIiGURJX0Qkgyjpi4hkECV9qRAzq21mG8ysdSLLppKZHWpmCe+7bGa9zGxpzPQiM+sWT9k9eK9HzeymPV2/jO3eYWZPJnq7kjp1Uh2AVC0z2xAzuRewBdgeTV/h7uMrsj133w40SnTZTODu7ROxHTO7HBji7j1itn15IrYtNZ+Sfg3n7juTblSTvNzd/11aeTOr4+6FyYhNRJJPzTsZLjp8/6uZPWdm3wFDzOwEM5tuZmvNbIWZPWBmdaPydczMzaxtND0uWj7JzL4zs/fNrF1Fy0bL+5jZJ2a2zsweNLN3zeziUuKOJ8YrzGyJmX1rZg/ErFvbzO4zszVm9hnQu4z9M9LMJhSbN8bM7o1eX25mC6PP82lUCy9tWwVm1iN6vZeZPRPFNh84pljZm83ss2i7882sbzT/SOAhoFvUdPZNzL69NWb9K6PPvsbMXjazA+PZN+Uxs/5RPGvN7E0zax+z7CYz+8rM1pvZxzGf9XgzmxXNX2lmo+N9P6kC7q5HhjyApUCvYvPuALYCZxMqAQ2BY4HjCEeCBwOfAFdH5esADrSNpscB3wA5QF3gr8C4PSi7P/Ad0C9adj2wDbi4lM8ST4yvAE2AtsB/ij47cDUwH8gCmgNTw79Cie9zMLAB2Dtm26uAnGj67KiMAacAm4BO0bJewNKYbRUAPaLX9wBvAc2ANsCCYmXPBw6M/iYXRDEcEC27HHirWJzjgFuj16dFMR4NNAD+BLwZz74p4fPfATwZvT48iuOU6G90E7Aoet0RWAa0jMq2Aw6OXs8ABkWvGwPHpfp/IZMfqukLwDvu/nd33+Hum9x9hrt/4O6F7v4ZMBboXsb6L7p7nrtvA8YTkk1Fy54FzHH3V6Jl9xF+IEoUZ4y/d/d17r6UkGCL3ut84D53L3D3NcBdZbzPZ8A8wo8RwE+Ab909L1r+d3f/zIM3gSlAiSdrizkfuMPdv3X3ZYTae+z7Pu/uK6K/ybOEH+ycOLYLMBh41N3nuPtmYATQ3cyyYsqUtm/KMhB41d3fjP5GdxF+OI4DCgk/MB2jJsLPo30H4cf7MDNr7u7fufsHcX4OqQJK+gLwZeyEmXUws3+Y2ddmth64HWhRxvpfx7zeSNknb0sr+6PYONzdCTXjEsUZY1zvRaihluVZYFD0+oJouiiOs8zsAzP7j5mtJdSyy9pXRQ4sKwYzu9jM5kbNKGuBDnFuF8Ln27k9d18PfAu0iilTkb9ZadvdQfgbtXL3RcANhL/Dqqi5sGVU9BIgG1hkZh+a2Rlxfg6pAkr6AuFwP9YjhNrtoe6+D/BrQvNFVVpBaG4BwMyM3ZNUcZWJcQVwUMx0eV1Knwd6mVkrQo3/2SjGhsCLwO8JTS9NgX/FGcfXpcVgZgcDfwauAppH2/04ZrvldS/9itBkVLS9xoRmpOVxxFWR7dYi/M2WA7j7OHc/kdC0U5uwX3D3Re4+kNCE9wfgJTNrUMlYZA8p6UtJGgPrgO/N7HDgiiS852tAFzM728zqANcC+1VRjM8D15lZKzNrDvyqrMLu/jXwDvAksMjdF0eL6gP1gNXAdjM7Czi1AjHcZGZNLVzHcHXMskaExL6a8Pv334SafpGVQFbRiesSPAdcZmadzKw+IflOc/dSj5wqEHNfM+sRvff/Es7DfGBmh5tZz+j9NkWPHYQPcKGZtYiODNZFn21HJWORPaSkLyW5AbiI8A/9COGEa5Vy95XAz4B7gTXAIcBswnUFiY7xz4S2948IJxlfjGOdZwknZnc27bj7WuAXwETCydABhB+vePyGcMSxFJgEPB2z3XzgQeDDqEx7ILYd/A1gMbDSzGKbaYrW/yehmWVitH5rQjt/pbj7fMI+/zPhB6k30Ddq368PjCKch/macGQxMlr1DGChhd5h9wA/c/etlY1H9oyFplOR6sXMahOaEwa4+7RUxyNSU6imL9WGmfWOmjvqA7cQen18mOKwRGoUJX2pTk4CPiM0HZwO9Hf30pp3RGQPqHlHRCSDqKYvIpJBqt2Aay1atPC2bdumOgwRkbQyc+bMb9y9rG7OQDVM+m3btiUvLy/VYYiIpBUzK+/KckDNOyIiGUVJX0Qkgyjpi4hkECV9EZEMoqQvIpJBlPRFRDJIjUn648dD27ZQq1Z4Hj8+1RGJiFQ/1a6f/p4YPx6GDYONG8P0smVhGmBwpQeUFREp25dfwhtvQFYWnHZaqqMpW42o6Y8cuSvhF9m4McwXEUm0LVtgyhT43/+FI46A1q3hssvg9NPhf/4HNm9OdYSlqxE1/S++qNh8EZGK+uwz+Oc/w+PNN+H776FePejWDS65BH7yExg3DkaPhnffheefh/btUx31D9WIpN+6dWjSKWm+iMie2LQJ3nprV6L/5JMwv107uOgi6NMHevSARjG3lB81KswbOhSOOQYefhiGDElB8GWoEUn/zjt3b9MH2GuvMF9EJB7uIbFPmhSS/Ntvh2aaBg2gZ08YPhx694bDDgOz0rdzxhkwZw5ccAFceGE4KnjwQdh77+R9lrLUiKRfdLJ25MjQpNO6dUj4OokrIvHYsgXOOSckewjNMldcEWrzJ58MDRtWbHtZWSHZ33ZbyEUffBCaezp2THzsFVXtbqKSk5PjGmVTRJJlxw4YOBBeeAHuuCPU0Nu1S9z2//3v0MSzfn2o8V96adlHCnvKzGa6e0555eLqvRPdu3SRmS0xsxElLL/SzD4yszlm9o6ZZccs+79ovUVmdnrFPoaISNVxh+uuCwn/nntCa0EiEz5Ar16huefHP4bLLw8/AN99l9j3qIhyk76Z1QbGAH2AbGBQbFKPPOvuR7r70cAo4N5o3WxgINAR6A38KdqeiEjKjRoVat/XXw833FB179OyJUyeHI4kJkyALl1g9uyqe7+yxFPT7woscffP3H0rMAHoF1vA3dfHTO4NFLUZ9QMmuPsWd/8cWBJtT0QkpZ56CkaMgEGDQjfLqla7djiSyM0NPYOOPx7GjAlHG8kUT9JvBXwZM10QzduNmQ03s08JNf1rKrKuiEgyTZoULqbq1QuefDIM35IsJ58cmnt69YKrr4bzzoO1a5P3/gn7qO4+xt0PAX4F3FyRdc1smJnlmVne6tWrExWSiMgPfPghDBgAnTrBSy+FC6ySrUUL+PvfwxHGK69A584hrmSIJ+kvBw6Kmc6K5pVmAnBORdZ197HunuPuOfvtV+59fUVE9sgnn8CZZ8IBB8Drr8M++6Qullq14MYbYdq00MRz4olw771V39wTT9KfARxmZu3MrB7hxOyrsQXM7LCYyTOBxdHrV4GBZlbfzNoBhwFJ+j0TEdnl66/D2DgQTqq2bJnaeIocf3w4qXv22aG9v6qVe3GWuxea2dXAZKA28Li7zzez24E8d38VuNrMegHbgG+Bi6J155vZ88ACoBAY7u7bq+iziIiUaP36cKHVqlVhaIXDDit3laRq1iw0NW3aVDV9+GPp4iwRqdG2bAlNOm+/HdrRe/dOdURVI96Ls2rEMAwiIiXZsQMuvjgMg/z00zU34VdEjRhPX0SkOPdwwdWECXD33WHwM1HSF5Ea6g9/gPvvh2uvDTc7kUBJX0RqnHHjQqI///zQDbKqT46mEyV9EalR/vWvcCernj1DO34yr7ZNB9odIlJj5OXBueeGcesnToT69VMdUfWjpC8iNcInn4S7VrVoEcbWadIk1RFVT+qyKSJpaeNGeOed0B1zyhSYNQv23TdcbXvggamOrvpS0heRtLBtWxiUrCjJv/9+mFe3bhjK4De/CbdIPfTQVEdavSnpi0i1tGMH5OfvSvJTp8L334eeOJ07hztenXoqnHRS9bnpeDpQ0heRasEdlizZleRzc2HNmrCsfXu46KKQ5Hv0CM04smeU9EWkyrnDN9/A8uXhUVCw63XsdNHNRLKy4KyzQpI/5RRopVsvJYySvogkxJdfwvTpuyf0otdffRUGPotVq1YY3rhVqzDqZc+eoavlqaeGdnldUFU1lPRFpNL+/W/o3x82bAjTDRqE2nqrVnDCCbtet2q163XLllBHGSjptMtFpFKefTaMZNm+PTzxBBx8cBgfXjX16klJX0T22B/+EG751707vPwyNG2a6oikPLoiV0QqbMeOMGzxjTeGm4z/859K+OlCSV9EKmTLFhgyJIxe+fOfh/HqGzRIdVQSLzXviEjc1q8PA5pNmQJ33QW//KXa7tONkr6IxGXFijCg2bx58NRTMHRoqiOSPaGkLyLl+uQTOP10WL26Zt9cPBMo6YtImT74IFwdaxaGRjj22FRHJJWhE7kiUqp//CMMg7DPPvDee0r4NYGSvoiU6IknoF8/6NAhJHwNWVwzKOmLyG7c4c474dJLwzg4b70FBxyQ6qgkUdSmLyI7bd8O11wDf/pT6Iv/2GNQr16qo5JEUk1fRADYvBnOPz8k/F/+MnTLVMKveVTTj4wfDyNHwhdfQOvW4fB28OBURyVSNdasgY8/hkWLwvPHH8OcOWEo5Pvvh2uvTXWEUlWU9AkJf9iwcKNlgGXLwjQo8Uv6KiyEzz//YXJftCjc0KRI/frwX/8Fxx0HDz0EffumLmapeubuqY5hNzk5OZ6Xl5fU92zbNiT64tq0gaVLkxqKyB7ZsgUmTQo3Di9K7kuWhBuHF9l//9ATp+jRvn14btMGatdOXeySGGY2091zyiunmj6hSaci80Wqgx074J13YNw4eOGFcKvBOnVC18r27UONPTbBN2uW6oilOogr6ZtZb+CPQG3gUXe/q9jy64HLgUJgNXCpuy+Llo0CziScNH4DuNar2eFF69Yl1/Rbt05+LCLlWbgwJPrx48P3du+9wyBoQ4aEWw7WrZvqCKU6K7f3jpnVBsYAfYBsYJCZZRcrNhvIcfdOwIvAqGjdHwMnAp2AI4Bjge4Jiz5B7rwT9tpr93l77RXmi1QHK1bAfffBMcdAdjbcfXd4Hj8eVq6Ep5+G005TwpfyxVPT7woscffPAMxsAtAPWFBUwN1zY8pPB4YULQIaAPUAA+oCKysfdmIVnaxV7x2pTjZsCHejeuaZcA/aHTsgJyf0rhk4UBdMyZ6JJ+m3Ar6MmS4Ajiuj/GXAJAB3f9/McoEVhKT/kLsvLL6CmQ0DhgG0TlGbyuDBSvKSeoWFIcGPGwcTJ4YeZW3awP/9X2i+6dAh1RFKukvoiVwzGwLkEDXhmNmhwOFAVlTkDTPr5u7TYtdz97HAWAi9dxIZk0g62LYtHF0+/HBormnWDC68MCT6H/8YaukySkmQeL5Ky4GDYqazonm7MbNewEigr7tviWb3B6a7+wZ330A4AjihciFXT+PHh66ftWqF5/HjUx2RpIuVK6FXL7jtttBX/m9/C234Dz8MJ52khC+JFc/XaQZwmJm1M7N6wEDg1dgCZtYZeISQ8FfFLPoC6G5mdcysLuEI4AfNO+mu6OKuZcvCYFVFF3cp8Ut5pk+HLl1gxozQdv/KK9C/f7hgSqQqlJv03b0QuBqYTEjYz7v7fDO73cyKrt0bDTQCXjCzOWZW9KPwIvAp8BEwF5jr7n9P9IdItZEjd13NW2TjxjBfpCTuoSZ/8snhpuLvvx+ackSqmq7ITYBatcI/cXFmoceFSKxNm2D48DBefZ8+4YhQF05JZcV7Ra5aCxOgtA5HurhLilu2LLTTP/EE/PrX8NprSviSXEr6CaCLuyQeb7wRLq769FN49dVw4lYnaSXZ9JVLgMGDYezY0J/aLDyPHat+/xK4w113Qe/ecOCB4aTt2WenOirJVBpwLUF0cZeUZP16uPjicKHVwIHw6KNhrByRVFHSF6kiCxeG7pdLloRxc669NhwJiqSSkr5IFXjppVDD32svmDIFule7YQYlUynpS43x1luwfDk0bw777huemzeHJk2SV8MuLISbbw6jYB5/fBjnPiur/PVEkkVJX9Lejh3hQri77ip5ee3aoVtk0Y9A8R+F2OmGDUP52EedOj+cV9Ly776Diy4KA6ZdeWUYDVNX1kp1o6RfTejG7Hvm++/DwGQTJ8J//zdcfz385z/hsWbNrkfs9BdfhJuAr1nzwyupK6t+fXj8cbjkksRuVyRRlPSrAd2Yfc8UFIRbAs6du+cnSjdv3v1HYfNm2L5916OwcPfpkh5FZXbsCFfYHnlk1XxekUTQMAzVgG7MXnF5eSHhb9gAzz0HZ56Z6ohEUkvDMKQR3Zi9Yl58MQxUVq8evPeeEr5IRSjpVwMauyc+7nDHHXDeedC5M3z4IRxxRKqjEkkvSvrVgMbuKd/mzWHo4VtuCc9TpsD++6c6KpH0U2OS/vbtoZvcRx+lOpKK09g9ZVu5Ek45BZ59NvwQPv10GINeRCquxpzIXbw4DFm7dm1oArj++tB3WtLbRx/BWWfB6tXhzlI//WmqIxKpnjLuRO5hh8G8eeGk3i9/CT17wuefpzoqqYzXXgs3BS8shGnTlPBFEqHGJH2A/fYLY5489VTou92pEzz2WMl3tZLqyx3uvTd0yWzfPpywPeaYVEclUjPUqKQPoU186FDIz4djj4XLL4d+/UK7cE02fnzo71+rVnhO9k3Z3eHtt2HSJJg5M1w4tWVLxbezdWu4MO2GG+Dcc2HqVGjVKvHximSqGntFbps2YQyUBx6AESNC176xY8NQtzVNKq/o3b49HF397nfh6Kq4Jk3ggANCT5v99y/79fbtMGBAGDjtppvgt7/VnaVEEq3GnMgty4IFYXyWWbPCgFh//GNIRjVFKq7o3boVxo0Lg5wtXhyaYX71q/C8alV4rFy563Xs9Jo1pW+3Xr3QJDdkSNXELVJTxXsit8bW9GNlZ8P774dePb/7HeTmwpNPhpO9NUEyr+jduDHc/Wn06NCE07lzGD64f//4e0tt2wbffFPyj8E554RmORGpGhmR9CHUIG+/PfTuufDC0O/7F78I/b4bNkx1dJXTunXJNf1EXtG7bh386U9hYLPVq0P32L/8BU4/veKDnNWtG+4Ve+CBiYtPROKTcS2mxx0Hs2fD8OEhgeXkhGafdFaVV/SuXh2GfG7dOrSzH3NMOLk6bVq40bdu/yeSXjIu6UO4MfVDD8HkyeFiruOOC00/hYWV226qTo9UxRW9X34J110XtvX738Npp4VeOZMmQbduiYtdRJIrI07kluXbb0Ot/7nnoGvX0M6/efOePbZsCTXspk0r/mjWLJxcrlVr1/jsxZ9Lmld8vPe6dcMQBUWPhg3Dc/368dXKFy8Ot/p7+unwIzZkSDhB26FD1f8tRGTPxXsiN+OTfpG//jXchGPdut2TZlHCLD6vpEe9erBpU/ghWbu25Mf27Un/aDvVr7/rR6D4j8K6dfDpp+GHywx69Qpt9m3apC5eEYmfeu9U0M9+Fh5VyT3c3q+0H4S1a0OZ2PuuFn8ub9m2bbuOPDZt2v1IpLTpzz+Hjz8Od34qivPdd+Gdd5T0RWoaJf0kMoNGjcIjKyvV0ezStu2uhF9k48ZwAlcjfYrULBl5Ild2pzt3iWSOuJK+mfU2s0VmtsTMRpSw/HozW2Bm+WY2xczaxCxrbWb/MrOFUZm2iQtfEkF37hLJHOUmfTOrDYwB+gDZwCAzyy5WbDaQ4+6dgBeBUTHLngZGu/vhQFdgVSICl8TRnbtEMkc8Nf2uwBJ3/8zdtwITgH6xBdw9192j4b6YDmQBRD8Oddz9jajchphyUk3ozl0imSOeE7mtgC9jpguA48oofxkwKXr9X8BaM/sb0A74NzDC3XfruGhmw4BhAK3VppASgwcryYtkgoSeyDWzIUAOMDqaVQfoBtwIHAscDFxcfD13H+vuOe6es99++yUyJBERiRFP0l8OHBQznRXN242Z9QJGAn3dvej2GQXAnKhpqBB4GehSuZBFRGRPxZP0ZwCHmVk7M6sHDARejS1gZp2BRwgJf1WxdZuaWVH1/RRgQeXDFhGRPVFu0o9q6FcDk4GFwPPuPt/MbjezvlGx0UAj4AUzm2Nmr0brbic07Uwxs48AA/5SBZ9DRETioLF3RERqgHjH3tEVuZIQqb4xu4jER2PvSKWl8sbsIlIxqulLpY0cuSvhFykasE1Eqhclfak0Ddgmkj6U9KXSNGCbSPpQ0pdK04BtIulDSV8qTQO2iaQP9d6RhNCAbSLpQTV9EZEMoqQv1YIu7hJJDjXvSMrp4i6R5FFNX1JOF3eJJI+SvqScLu4SSR4lfUk5XdwlkjxK+pJyurhLJHmU9CXldHGXSPKo945UC7q4SyQ5VNOXGkH9/EXio5q+pD318xeJn2r6kvbUz18kfkr6kvbUz18kfkr6kvbUz18kfkr6kvbUz18kfkr6kvbUz18kfuq9IzWC+vmLxEc1fRHUz18yh2r6kvHUz18yiWr6kvHUz18yiZK+ZDz185dMoqQvGU/9/CWTKOlLxlM/f8kkcSV9M+ttZovMbImZjShh+fVmtsDM8s1sipm1KbZ8HzMrMLOHEhW4SKKon79kknJ775hZbWAM8BOgAJhhZq+6+4KYYrOBHHffaGZXAaOAn8Us/y0wNXFhiySW+vlLpoinpt8VWOLun7n7VmAC0C+2gLvnuntR/4fpQFbRMjM7BjgA+FdiQhapftTPX9JFPEm/FfBlzHRBNK80lwGTAMysFvAH4May3sDMhplZnpnlrV69Oo6QRKqPon7+y5aB+65+/kr8Uh0l9ESumQ0BcoDR0az/AV5394Ky1nP3se6e4+45++23XyJDEqly6ucv6SSeK3KXAwfFTGdF83ZjZr2AkUB3d98SzT4B6GZm/wM0AuqZ2QZ3/8HJYJF0pX7+kk7iSfozgMPMrB0h2Q8ELogtYGadgUeA3u6+qmi+uw+OKXMx4WSvEr7UKK1bhyadkuaLVDflNu+4eyFwNTAZWAg87+7zzex2M+sbFRtNqMm/YGZzzOzVKotYpJpRP39JJ+buqY5hNzk5OZ6Xl5fqMEQqZPz40Ib/xRehhn/nneoCKsllZjPdPae8chplUyQB1M9f0oWGYRARySBK+iIiGURJX0Qkgyjpi4hkECV9kWpAY/dIsqj3jkiK6R69kkyq6YukmMbukWRS0hdJMY3dI8mkpC+SYrpHrySTkr5IimnsHkkmJX2RFNM9eiWZlPRFqoHBg2HpUtixIzxXNOGry6fES102RdKcunxKRaimL5Lm1OVTKkJJXyTNqcunVISSvkiaU5dPqQglfZE0py6fUhFK+iJpTl0+pSLUe0ekBtDtGiVequmLiPr5ZxDV9EUynPr5ZxbV9EUynPr5ZxYlfZEMp37+mUVJXyTDqZ9/ZlHSF8lw6uefWZT0RTKc+vlnFiV9EdHQzhlEXTZFpFLU5TO9qKYvIpWiLp/pJa6kb2a9zWyRmS0xsxElLL/ezBaYWb6ZTTGzNtH8o83sfTObHy37WaI/gIiklrp8ppdyk76Z1QbGAH2AbGCQmWUXKzYbyHH3TsCLwKho/kZgqLt3BHoD95tZ00QFLyKppy6f6SWemn5XYIm7f+buW4EJQL/YAu6e6+5FB3jTgaxo/ifuvjh6/RWwCtgvUcGLSOqpy2d6iSfptwK+jJkuiOaV5jJgUvGZZtYVqAd8WsKyYWaWZ2Z5q1evjiMkEakuEtHlU71/kiehvXfMbAiQA3QvNv9A4BngInffUXw9dx8LjAXIycnxRMYkIlWvMkM7q/dPcsVT018OHBQznRXN242Z9QJGAn3dfUvM/H2AfwAj3X165cIVkZpGvX+SK56a/gzgMDNrR0j2A4ELYguYWWfgEaC3u6+KmV8PmAg87e4v7mmQ27Zto6CggM2bN+/pJiSJGjRoQFZWFnXr1k11KJIG1PsnucpN+u5eaGZXA5OB2sDj7j7fzG4H8tz9VWA00Ah4wcwAvnD3vsD5wMlAczO7ONrkxe4+pyJBFhQU0LhxY9q2bUu0famm3J01a9ZQUFBAu3btUh2OpIHWrUOTTknzJfHiatN399eB14vN+3XM616lrDcOGFeZAAE2b96shJ8mzIzmzZujE/ISrzvv3L1NH9T7pyqlzRW5SvjpQ38rqQgN+JZcGntHRFJON3ZPnrSp6VdEovv8rlmzhqOPPpqjjz6ali1b0qpVq53TW7dujWsbl1xyCYsWLSqzzJgxYxifoA7KJ510EnPmVOjUiYhkgBpX06+KPr/NmzffmUBvvfVWGjVqxI033rhbGXfH3alVq+Tf0SeeeKLc9xk+fPieBSgiEqcaV9NPZp/fJUuWkJ2dzeDBg+nYsSMrVqxg2LBh5OTk0LFjR26//fadZYtq3oWFhTRt2pQRI0Zw1FFHccIJJ7BqVejlevPNN3P//ffvLD9ixAi6du1K+/btee+99wD4/vvv+elPf0p2djYDBgwgJyen3Br9uHHjOPLIIzniiCO46aabACgsLOTCCy/cOf+BBx4A4L777iM7O5tOnToxZMiQhO8zEUmtGpf0k93n9+OPP+YXv/gFCxYsoFWrVtx1113k5eUxd+5c3njjDRYsWPCDddatW0f37t2ZO3cuJ5xwAo8//niJ23Z3PvzwQ0aPHr3zB+TBBx+kZcuWLFiwgFtuuYXZs2eXGV9BQQE333wzubm5zJ49m3fffZfXXnuNmTNn8s033/DRRx8xb948hg4dCsCoUaOYM2cO+fn5PPTQQ5XcOyLJoWEc4lfjkn6yR/w75JBDyMnJ2Tn93HPP0aVLF7p06cLChQtLTPoNGzakT58+ABxzzDEsXbq0xG2fe+65PyjzzjvvMHDgQACOOuooOnbsWGZ8H3zwAaeccgrxGeUvAAANnElEQVQtWrSgbt26XHDBBUydOpVDDz2URYsWcc011zB58mSaNGkCQMeOHRkyZAjjx4/XxVWSFoqadJctA/ddTbpK/CWrcUk/2SP+7b333jtfL168mD/+8Y+8+eab5Ofn07t37xKvIq5Xr97O17Vr16awsLDEbdevX7/cMnuqefPm5Ofn061bN8aMGcMVV1wBwOTJk7nyyiuZMWMGXbt2Zfv27Ql9X5FES0STbiYdKdS4pJ/KPr/r16+ncePG7LPPPqxYsYLJkycn/D1OPPFEnn/+eQA++uijEo8kYh133HHk5uayZs0aCgsLmTBhAt27d2f16tW4O+eddx633347s2bNYvv27RQUFHDKKacwatQovvnmGzYW/28SqWYq26SbaUcKNa73DqSuz2+XLl3Izs6mQ4cOtGnThhNPPDHh7/Hzn/+coUOHkp2dvfNR1DRTkqysLH7729/So0cP3J2zzz6bM888k1mzZnHZZZfh7pgZd999N4WFhVxwwQV899137NixgxtvvJHGjRsn/DOIJFJlh3Eo60ihJl47YO7VayTjnJwcz8vL223ewoULOfzww1MUUfVSWFhIYWEhDRo0YPHixZx22mksXryYOnWq1++3/maSLMW7aUNo0o33CL9WrVDDL84MdvxgIPjqy8xmuntOeeWqV6aQcm3YsIFTTz2VwsJC3J1HHnmk2iV8kWQqSuwjR4Ymndatwzm8eGvpmTbgm7JFmmnatCkzZ85MdRgi1UplmnQzbcC3GnciV0SkIjJtwDclfRHJeIMHw9KloQ1/6dKKJ/x06vKp5h0RkUpIt3v8qqYvIlIJ6XaPXyX9OPTs2fMHF1rdf//9XHXVVWWu16hRIwC++uorBgwYUGKZHj16ULyLanH333//bhdJnXHGGaxduzae0Mt06623cs8991R6OyKZLN3u8aukH4dBgwYxYcKE3eZNmDCBQYMGxbX+j370I158cY/vC/+DpP/666/TtGnTPd6eiCROssf7qqy0a9O/7jpI9L1Bjj4aohGNSzRgwABuvvlmtm7dSr169Vi6dClfffUV3bp1Y8OGDfTr149vv/2Wbdu2cccdd9CvX7/d1l+6dClnnXUW8+bNY9OmTVxyySXMnTuXDh06sGnTpp3lrrrqKmbMmMGmTZsYMGAAt912Gw888ABfffUVPXv2pEWLFuTm5tK2bVvy8vJo0aIF9957785ROi+//HKuu+46li5dSp8+fTjppJN47733aNWqFa+88goNGzYs9TPOmTOHK6+8ko0bN3LIIYfw+OOP06xZMx544AEefvhh6tSpQ3Z2NhMmTODtt9/m2muvBcKtEadOnaordyVjpVuXT9X047DvvvvStWtXJk2aBIRa/vnnn4+Z0aBBAyZOnMisWbPIzc3lhhtuoKyrnP/85z+z1157sXDhQm677bbd+tzfeeed5OXlkZ+fz9tvv01+fj7XXHMNP/rRj8jNzSU3N3e3bc2cOZMnnniCDz74gOnTp/OXv/xl51DLixcvZvjw4cyfP5+mTZvy0ksvlfkZhw4dyt13301+fj5HHnkkt912GwB33XUXs2fPJj8/n4cffhiAe+65hzFjxjBnzhymTZtW5o+JSE2Xbl0+066mX1aNvCoVNfH069ePCRMm8NhjjwFhzPubbrqJqVOnUqtWLZYvX87KlStp2bJliduZOnUq11xzDQCdOnWiU6dOO5c9//zzjB07lsLCQlasWMGCBQt2W17cO++8Q//+/XeO9Hnuuecybdo0+vbtS7t27Tj66KOBsodvhjC+/9q1a+nevTsAF110Eeedd97OGAcPHsw555zDOeecA4RB366//noGDx7MueeeS1ZWVjy7UKTGSqd7/KqmH6d+/foxZcoUZs2axcaNGznmmGMAGD9+PKtXr2bmzJnMmTOHAw44oMThlMvz+eefc8899zBlyhTy8/M588wz92g7RYqGZYbKDc38j3/8g+HDhzNr1iyOPfZYCgsLGTFiBI8++iibNm3ixBNP5OOPP97jOEUkuf38lfTj1KhRI3r27Mmll1662wncdevWsf/++1O3bl1yc3NZVtIgHjFOPvlknn32WQDmzZtHfn4+EIZl3nvvvWnSpAkrV67c2ZQE0LhxY7777rsfbKtbt268/PLLbNy4ke+//56JEyfSrVu3Cn+2Jk2a0KxZM6ZNmwbAM888Q/fu3dmxYwdffvklPXv25O6772bdunVs2LCBTz/9lCOPPJJf/epXHHvssUr6IpWQ7KGd0655J5UGDRpE//79d+vJM3jwYM4++2yOPPJIcnJy6NChQ5nbuOqqq7jkkks4/PDDOfzww3ceMRx11FF07tyZDh06cNBBB+02LPOwYcPo3bv3zrb9Il26dOHiiy+ma9euQDiR27lz5zKbckrz1FNP7TyRe/DBB/PEE0+wfft2hgwZwrp163B3rrnmGpo2bcott9xCbm4utWrVomPHjjvvAiYiFZfsoZ01tLJUCf3NROKTqKGd4x1aWc07IiIplOx+/kr6IiIplOz7eqdN0q9uzVBSOv2tROKX7H7+aXEit0GDBqxZs4bmzZtjZqkOR8rg7qxZs4YGDRqkOhSRtJHMfv5pkfSzsrIoKChg9erVqQ5F4tCgQQNdsCVSTcWV9M2sN/BHoDbwqLvfVWz59cDlQCGwGrjU3ZdFyy4Cbo6K3uHuT1U0yLp169KuXbuKriYiIsWU26ZvZrWBMUAfIBsYZGbZxYrNBnLcvRPwIjAqWndf4DfAcUBX4Ddm1ixx4YuISEXEcyK3K7DE3T9z963ABGC3YSTdPdfdiy4vmA4UHdufDrzh7v9x92+BN4DeiQldREQqKp6k3wr4Mma6IJpXmsuAojEE4lrXzIaZWZ6Z5andXkSk6iT0RK6ZDQFygO4VWc/dxwJjo22sNrOyB7BJrRbAN6kOogyKr3IUX+UovsqpTHxt4ikUT9JfDhwUM50VzduNmfUCRgLd3X1LzLo9iq37Vllv5u77xRFTyphZXjyXOqeK4qscxVc5iq9ykhFfPM07M4DDzKydmdUDBgKvxhYws87AI0Bfd18Vs2gycJqZNYtO4J4WzRMRkRQot6bv7oVmdjUhWdcGHnf3+WZ2O5Dn7q8Co4FGwAvRxVNfuHtfd/+Pmf2W8MMBcLu7/6dKPomIiJQrrjZ9d38deL3YvF/HvO5VxrqPA4/vaYDV0NhUB1AOxVc5iq9yFF/lVHl81W5oZRERqTppM+CaiIhUnpK+iEgGUdIvxswOMrNcM1tgZvPN7NoSyvQws3VmNid6/LqkbVVxnEvN7KPo/fNKWG5m9oCZLTGzfDPrksTY2sfsmzlmtt7MritWJqn70MweN7NVZjYvZt6+ZvaGmS2OnkscIsTMLorKLI7GkkpWfKPN7OPo7zfRzJqWsm6Z34UqjO9WM1se8zc8o5R1e5vZoui7OCKJ8f01JralZjanlHWTsf9KzCsp+Q66ux4xD+BAoEv0ujHwCZBdrEwP4LUUx7kUaFHG8jMIV0YbcDzwQYrirA18DbRJ5T4ETga6APNi5o0CRkSvRwB3l7DevsBn0XOz6HWzJMV3GlAnen13SfHF812owvhuBW6M4+//KXAwUA+YW/z/qariK7b8D8CvU7j/SswrqfgOqqZfjLuvcPdZ0evvgIWUPexEddUPeNqD6UBTMzswBXGcCnzq0airqeLuU4Hi3YX7AUWjvj4FnFPCqkkZP6qk+Nz9X+5eGE3GjmmVdKXsv3iUO3ZXIpQVn4V+5OcDzyX6feNVRl5J+ndQSb8MZtYW6Ax8UMLiE8xsrplNMrOOSQ0scOBfZjbTzIaVsLyiYyZVlYGU/s+W6n14gLuviF5/DRxQQpnqsh8vZdeYVsWV912oSldHzU+Pl9I0UR32XzdgpbsvLmV5UvdfsbyS9O+gkn4pzKwR8BJwnbuvL7Z4FqG54ijgQeDlZMcHnOTuXQhDXg83s5NTEEOZoiu4+wIvlLC4OuzDnTwcR1fL/stmNpJwr4rxpRRJ1Xfhz8AhwNHACkITSnU0iLJr+Unbf2XllWR9B5X0S2BmdQl/mPHu/rfiy919vbtviF6/DtQ1sxbJjNHdl0fPq4CJhMPoWHGNmVTF+gCz3H1l8QXVYR8CK4uavKLnVSWUSel+NLOLgbOAwVFS+IE4vgtVwt1Xuvt2d98B/KWU9031/qsDnAv8tbQyydp/peSVpH8HlfSLidr/HgMWuvu9pZRpGZXDzLoS9uOaJMa4t5k1LnpNOOE3r1ixV4GhUS+e44F1MYeRyVJqDSvV+zDyKlDUE+Ii4JUSyqRs/CgLd6z7JWFMq42llInnu1BV8cWeI+pfyvuWO3ZXFesFfOzuBSUtTNb+KyOvJP87WJVnrNPxAZxEOMTKB+ZEjzOAK4ErozJXA/MJPRGmAz9OcowHR+89N4pjZDQ/NkYj3PHsU+Ajwp3Nkhnj3oQk3iRmXsr2IeHHZwWwjdAmehnQHJgCLAb+Dewblc0h3Ba0aN1LgSXR45IkxreE0JZb9D18OCr7I+D1sr4LSYrvmei7lU9IXgcWjy+aPoPQW+XTZMYXzX+y6DsXUzYV+6+0vJL076CGYRARySBq3hERySBK+iIiGURJX0Qkgyjpi4hkECV9EZEMoqQvIpJBlPRFRDLI/wN7qoYVwPeAxwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss) + 1)\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "훨씬 좋군요! 상식 수준의 모델을 크게 앞질렀습니다. 이 그림은 시퀀스를 펼쳐서 처리하는 완전 연결 네트워크에 비해서 순환 네트워크가 이런 종류의 작업에 훨씬 뛰어나다는 것과 머신 러닝의 가치를 보여줍니다.\n",
    "\n",
    "새로운 검증 MAE는 0.265 이하(크게 과대적합되기 시작하는 곳)이고 정규화되기 전인 섭씨로 복원하면 MAE는 2.35°C입니다. 초기 에러 2.57°C보다는 확실히 낫지만 더 개선할 수 있을 것 같습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.3.6 과대적합을 감소하기 위해 순환 드롭아웃 사용하기\n",
    "\n",
    "훈련 손실과 검증 손실 곡선을 보면 모델이 과대적합인지 알 수 있습니다. 몇 번의 에포크 이후에 훈련 손실과 검증 손실이 현저하게 벌어지기 시작합니다. 이런 현상을 해결하기 위해 잘 알려진 드롭아웃 기법을 이미 보았습니다. 훈련 데이터를 층에 주입할 때 데이터에 있는 우연한 상관 관계를 깨뜨리기 위해 입력 층의 유닛을 랜덤하게 끄는 기법입니다. \n",
    "\n",
    "순환 신경망에 드롭아웃을 올바르게 적용하는 방법은 간단하지 않습니다. 순환 층 이전에 드롭아웃을 적용하면 규제에 도움이 되는 것보다 학습에 더 방해되는 것으로 오랫동안 알려졌습니다. 2015년 야린 갈이 베이지안 딥러닝에 관한 박사 논문에서 순환 네트워크에 적절하게 드롭아웃을 사용하는 방법을 알아 내었습니다. \n",
    "\n",
    "- 타임스텝마다 랜덤하게 드롭아웃 마스크를 바꾸는 것이 아니라 동일한 드롭아웃 마스크(동일한 유닛의 드롭 패턴)를 모든 타임스텝에 적용해야 합니다. \n",
    "\n",
    "- GRU이나 LSTM 같은 순환 게이트에 의해 만들어지는 표현을 규제하려면 순환 층 내부 계산에 사용된 활성화 함수에 타임스텝마다 동일한 드롭아웃 마스크를 적용해야 합니다(순환 드롭 아웃 마스크). \n",
    "\n",
    "- 모든 타임스텝에 동일한 드롭아웃 마스크를 적용하면 네트워크가 학습 오차를 타임스템에 걸쳐 적절하게 전파시킬 것입니다. 타임스텝마다 랜덤한 드롭아웃 마스크를 적용하면 오차 신호가 전파되는 것을 방해하고 학습 과정에 해를 끼칩니다.\n",
    "\n",
    "야린 갈은 케라스를 사용해 연구를 하였고 케라스 순환 층에 이 기능을 구현하는 데 도움을 주었습니다. \n",
    "\n",
    "케라스에 있는 모든 순환 층은 두 개의 드롭아웃 매개변수를 가지고 있습니다. \n",
    "\n",
    "- **dropout**은 층의 입력에 대한 드롭아웃 비율을 정하는 부동 소수 값입니다. \n",
    "\n",
    "- **recurrent_dropout**은 순환 상태의 드롭아웃 비율을 정합니다. \n",
    "\n",
    "GRU 층에 드롭아웃과 순환 드롭아웃을 적용해 과대적합에 어떤 영향을 미치는지 살펴보겠습니다. 드롭아웃으로 규제된 네트워크는 언제나 완전히 수렴하는 데 더 오래 걸립니다. 에포크를 두 배 더 늘려 네트워크를 훈련하겠습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "gru_1 (GRU)                  (None, 32)                4512      \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 1)                 33        \n",
      "=================================================================\n",
      "Total params: 4,545\n",
      "Trainable params: 4,545\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Epoch 1/40\n",
      "500/500 [==============================] - 88s 175ms/step - loss: 0.3355 - mean_absolute_error: 0.3355 - val_loss: 0.2781 - val_mean_absolute_error: 0.2781\n",
      "Epoch 2/40\n",
      "500/500 [==============================] - 87s 175ms/step - loss: 0.3145 - mean_absolute_error: 0.3145 - val_loss: 0.2778 - val_mean_absolute_error: 0.2778\n",
      "Epoch 3/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.3079 - mean_absolute_error: 0.3079 - val_loss: 0.2721 - val_mean_absolute_error: 0.2721\n",
      "Epoch 4/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.3040 - mean_absolute_error: 0.3040 - val_loss: 0.2713 - val_mean_absolute_error: 0.2713\n",
      "Epoch 5/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.3011 - mean_absolute_error: 0.3011 - val_loss: 0.2677 - val_mean_absolute_error: 0.2677\n",
      "Epoch 6/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2989 - mean_absolute_error: 0.2989 - val_loss: 0.2661 - val_mean_absolute_error: 0.2661\n",
      "Epoch 7/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2956 - mean_absolute_error: 0.2956 - val_loss: 0.2634 - val_mean_absolute_error: 0.2634\n",
      "Epoch 8/40\n",
      "500/500 [==============================] - 87s 175ms/step - loss: 0.2953 - mean_absolute_error: 0.2953 - val_loss: 0.2676 - val_mean_absolute_error: 0.2676\n",
      "Epoch 9/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2908 - mean_absolute_error: 0.2908 - val_loss: 0.2669 - val_mean_absolute_error: 0.2669\n",
      "Epoch 10/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2913 - mean_absolute_error: 0.2913 - val_loss: 0.2672 - val_mean_absolute_error: 0.2672\n",
      "Epoch 11/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2894 - mean_absolute_error: 0.2894 - val_loss: 0.2685 - val_mean_absolute_error: 0.2685\n",
      "Epoch 12/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2894 - mean_absolute_error: 0.2894 - val_loss: 0.2612 - val_mean_absolute_error: 0.2612\n",
      "Epoch 13/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2863 - mean_absolute_error: 0.2863 - val_loss: 0.2651 - val_mean_absolute_error: 0.2651\n",
      "Epoch 14/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2868 - mean_absolute_error: 0.2868 - val_loss: 0.2625 - val_mean_absolute_error: 0.2625\n",
      "Epoch 15/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2869 - mean_absolute_error: 0.2869 - val_loss: 0.2631 - val_mean_absolute_error: 0.2631\n",
      "Epoch 16/40\n",
      "500/500 [==============================] - 87s 175ms/step - loss: 0.2851 - mean_absolute_error: 0.2851 - val_loss: 0.2650 - val_mean_absolute_error: 0.2650\n",
      "Epoch 17/40\n",
      "500/500 [==============================] - 87s 175ms/step - loss: 0.2834 - mean_absolute_error: 0.2834 - val_loss: 0.2637 - val_mean_absolute_error: 0.2637\n",
      "Epoch 18/40\n",
      "500/500 [==============================] - 87s 175ms/step - loss: 0.2841 - mean_absolute_error: 0.2841 - val_loss: 0.2624 - val_mean_absolute_error: 0.2624\n",
      "Epoch 19/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2817 - mean_absolute_error: 0.2817 - val_loss: 0.2640 - val_mean_absolute_error: 0.2640\n",
      "Epoch 20/40\n",
      "500/500 [==============================] - 87s 175ms/step - loss: 0.2810 - mean_absolute_error: 0.2810 - val_loss: 0.2621 - val_mean_absolute_error: 0.2621\n",
      "Epoch 21/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2798 - mean_absolute_error: 0.2798 - val_loss: 0.2650 - val_mean_absolute_error: 0.2650\n",
      "Epoch 22/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2804 - mean_absolute_error: 0.2804 - val_loss: 0.2617 - val_mean_absolute_error: 0.2617\n",
      "Epoch 23/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2784 - mean_absolute_error: 0.2784 - val_loss: 0.2598 - val_mean_absolute_error: 0.2598\n",
      "Epoch 24/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2770 - mean_absolute_error: 0.2770 - val_loss: 0.2629 - val_mean_absolute_error: 0.2629\n",
      "Epoch 25/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2772 - mean_absolute_error: 0.2772 - val_loss: 0.2685 - val_mean_absolute_error: 0.2685\n",
      "Epoch 26/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2779 - mean_absolute_error: 0.2779 - val_loss: 0.2629 - val_mean_absolute_error: 0.2629\n",
      "Epoch 27/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2763 - mean_absolute_error: 0.2763 - val_loss: 0.2632 - val_mean_absolute_error: 0.2632\n",
      "Epoch 28/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2761 - mean_absolute_error: 0.2761 - val_loss: 0.2618 - val_mean_absolute_error: 0.2618\n",
      "Epoch 29/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2739 - mean_absolute_error: 0.2739 - val_loss: 0.2629 - val_mean_absolute_error: 0.2629\n",
      "Epoch 30/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2759 - mean_absolute_error: 0.2759 - val_loss: 0.2627 - val_mean_absolute_error: 0.2627\n",
      "Epoch 31/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2744 - mean_absolute_error: 0.2744 - val_loss: 0.2610 - val_mean_absolute_error: 0.2610\n",
      "Epoch 32/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2744 - mean_absolute_error: 0.2744 - val_loss: 0.2635 - val_mean_absolute_error: 0.2635\n",
      "Epoch 33/40\n",
      "500/500 [==============================] - 87s 175ms/step - loss: 0.2736 - mean_absolute_error: 0.2736 - val_loss: 0.2616 - val_mean_absolute_error: 0.2616\n",
      "Epoch 34/40\n",
      "500/500 [==============================] - 87s 175ms/step - loss: 0.2719 - mean_absolute_error: 0.2719 - val_loss: 0.2653 - val_mean_absolute_error: 0.2653\n",
      "Epoch 35/40\n",
      "500/500 [==============================] - 87s 175ms/step - loss: 0.2737 - mean_absolute_error: 0.2737 - val_loss: 0.2599 - val_mean_absolute_error: 0.2599\n",
      "Epoch 36/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2708 - mean_absolute_error: 0.2708 - val_loss: 0.2675 - val_mean_absolute_error: 0.2675\n",
      "Epoch 37/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2708 - mean_absolute_error: 0.2708 - val_loss: 0.2604 - val_mean_absolute_error: 0.2604\n",
      "Epoch 38/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2714 - mean_absolute_error: 0.2714 - val_loss: 0.2697 - val_mean_absolute_error: 0.2697\n",
      "Epoch 39/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2706 - mean_absolute_error: 0.2706 - val_loss: 0.2711 - val_mean_absolute_error: 0.2711\n",
      "Epoch 40/40\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2697 - mean_absolute_error: 0.2697 - val_loss: 0.2636 - val_mean_absolute_error: 0.2636\n",
      "CPU times: user 1h 20min 41s, sys: 3min 24s, total: 1h 24min 6s\n",
      "Wall time: 58min 9s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "from keras import backend as K\n",
    "from keras.models import Sequential\n",
    "from keras import layers\n",
    "from keras.optimizers import RMSprop\n",
    "\n",
    "K.clear_session()\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.GRU(32, \n",
    "                     dropout=0.2, \n",
    "                     recurrent_dropout=0.2,\n",
    "                     input_shape=(None, float_data.shape[-1])))\n",
    "model.add(layers.Dense(1))\n",
    "model.summary()\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae', metrics=['mae'])\n",
    "history = model.fit_generator(train_gen,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=40,\n",
    "                              validation_data=val_gen,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl4VOXZ+PHvTSDsgixWZEdRCIssAfSHlEVUlAqiVEFQsFKUSq1S+4qCG0pVal1LfbVWWwWk1JXWIlalRX0VCcoqIghBgwgBZQchyf374zmTTMJk5mRmkplk7s91zZU55zznnGdOknue82xHVBVjjDGpoVqiM2CMMabiWNA3xpgUYkHfGGNSiAV9Y4xJIRb0jTEmhVjQN8aYFGJB35SJiKSJyAERaRXPtIkkIqeJSNz7LovIYBHJDlreICL9/KSN4lzPiMjt0e4f5rj3ichf4n1ckzjVE50BU75E5EDQYh3gByDfW75OVeeW5Xiqmg/Ui3faVKCqZ8TjOCIyARirqgOCjj0hHsc2VZ8F/SpOVQuDrleSnKCqb5eWXkSqq2peReTNGFPxrHonxXm3738TkRdFZD8wVkTOFpGPRGSPiGwXkcdFpIaXvrqIqIi08ZbneNsXich+EflQRNqWNa23/UIR+UJE9orIEyLygYiMLyXffvJ4nYhsEpHvReTxoH3TROQREdktIpuBIWGuzzQRmV9i3WwRedh7P0FE1nuf50uvFF7asXJEZID3vo6IvODlbR3Qs0Ta6SKy2TvuOhEZ5q3vAvwB6OdVne0KurZ3B+1/vffZd4vIayLSzM+1iURERnj52SMi74rIGUHbbheRb0Rkn4h8HvRZzxKRT7z1O0Tkd37PZ8qBqtorRV5ANjC4xLr7gKPAxbhCQG2gF9AHdyfYDvgCmOylrw4o0MZbngPsAjKBGsDfgDlRpD0J2A8M97ZNAY4B40v5LH7y+DrQAGgDfBf47MBkYB3QAmgMLHX/CiHP0w44ANQNOvZOINNbvthLI8Ag4DDQ1ds2GMgOOlYOMMB7/xDwH+BEoDXwWYm0lwPNvN/JlV4efuRtmwD8p0Q+5wB3e+/P9/LYDagF/BF418+1CfH57wP+4r3v6OVjkPc7uh3Y4L3vBGwFTvbStgXaee+XA6O99/WBPon+X0jll5X0DcD7qvoPVS1Q1cOqulxVl6lqnqpuBp4G+ofZ/yVVzVLVY8BcXLApa9qfACtV9XVv2yO4L4iQfObxflXdq6rZuAAbONflwCOqmqOqu4EHwpxnM7AW92UEcB7wvapmedv/oaqb1XkXeAcI2VhbwuXAfar6vapuxZXeg8+7QFW3e7+Tebgv7EwfxwUYAzyjqitV9QgwFegvIi2C0pR2bcIZBSxU1Xe939EDuC+OPkAe7gumk1dFuMW7duC+vNuLSGNV3a+qy3x+DlMOLOgbgK+DF0Skg4i8ISLfisg+YAbQJMz+3wa9P0T4xtvS0p4SnA9VVVzJOCSfefR1LlwJNZx5wGjv/ZXeciAfPxGRZSLynYjswZWyw12rgGbh8iAi40VklVeNsgfo4PO44D5f4fFUdR/wPdA8KE1ZfmelHbcA9ztqrqobgF/jfg87verCk72k1wAZwAYR+VhELvL5OUw5sKBvwN3uB3sKV7o9TVVPAO7EVV+Up+246hYAREQoHqRKiiWP24GWQcuRupQuAAaLSHNciX+el8fawEvA/biql4bAWz7z8W1peRCRdsCTwCSgsXfcz4OOG6l76Te4KqPA8erjqpG2+chXWY5bDfc72wagqnNUtS+uaicNd11Q1Q2qOgpXhfd74GURqRVjXkyULOibUOoDe4GDItIRuK4CzvlPoIeIXCwi1YFfAU3LKY8LgJtEpLmINAZuDZdYVb8F3gf+AmxQ1Y3epppAOpAL5IvIT4Bzy5CH20WkobhxDJODttXDBfZc3Pffz3El/YAdQItAw3UILwLXikhXEamJC77vqWqpd05lyPMwERngnfs3uHaYZSLSUUQGeuc77L0KcB/gKhFp4t0Z7PU+W0GMeTFRsqBvQvk1MA73D/0UrsG1XKnqDuAK4GFgN3Aq8CluXEG88/gkru59Da6R8SUf+8zDNcwWVu2o6h7gZuBVXGPoSNyXlx934e44soFFwPNBx10NPAF87KU5AwiuB/83sBHYISLB1TSB/d/EVbO86u3fClfPHxNVXYe75k/ivpCGAMO8+v2awCxcO8y3uDuLad6uFwHrxfUOewi4QlWPxpofEx1xVafGJBcRScNVJ4xU1fcSnR9jqgor6ZukISJDvOqOmsAduF4fHyc4W8ZUKRb0TTI5B9iMqzq4ABihqqVV7xhjomDVO8YYk0KspG+MMSkk6SZca9KkibZp0ybR2TDGmEplxYoVu1Q1XDdnIAmDfps2bcjKykp0NowxplIRkUgjywGr3jHGmJRiQd8YY1KIBX1jjEkhSVenb4ypWMeOHSMnJ4cjR44kOivGh1q1atGiRQtq1Cht6qXwLOgbk+JycnKoX78+bdq0wU1uapKVqrJ7925ycnJo27Zt5B1CqDLVO3PnQps2UK2a+zm3TI/7NiZ1HTlyhMaNG1vArwREhMaNG8d0V1YlSvpz58LEiXDokFveutUtA4yJeW5BY6o+C/iVR6y/qypR0p82rSjgBxw65NYbY4wpUiWC/ldflW29MSZ57N69m27dutGtWzdOPvlkmjdvXrh89Ki/afevueYaNmzYEDbN7NmzmRunet9zzjmHlStXxuVYFa1KVO+0auWqdEKtN8bE19y57i76q6/c/9jMmbFVozZu3LgwgN59993Uq1ePW265pVgaVUVVqVYtdDn1ueeei3ieG264IfpMViFVoqQ/cybUqVN8XZ06br0xJn4C7Wdbt4JqUftZeXSc2LRpExkZGYwZM4ZOnTqxfft2Jk6cSGZmJp06dWLGjBmFaQMl77y8PBo2bMjUqVM588wzOfvss9m5cycA06dP59FHHy1MP3XqVHr37s0ZZ5zB//3f/wFw8OBBLrvsMjIyMhg5ciSZmZkRS/Rz5syhS5cudO7cmdtvvx2AvLw8rrrqqsL1jz/+OACPPPIIGRkZdO3albFjx8b9mvlRJUr6gVJGPEsfxpjjhWs/K4//t88//5znn3+ezMxMAB544AEaNWpEXl4eAwcOZOTIkWRkZBTbZ+/evfTv358HHniAKVOm8OyzzzJ16tTjjq2qfPzxxyxcuJAZM2bw5ptv8sQTT3DyySfz8ssvs2rVKnr06BE2fzk5OUyfPp2srCwaNGjA4MGD+ec//0nTpk3ZtWsXa9asAWDPnj0AzJo1i61bt5Kenl64rqJViZI+uD+47GwoKHA/LeAbE38V3X526qmnFgZ8gBdffJEePXrQo0cP1q9fz2effXbcPrVr1+bCCy8EoGfPnmRnZ4c89qWXXnpcmvfff59Ro0YBcOaZZ9KpU6ew+Vu2bBmDBg2iSZMm1KhRgyuvvJKlS5dy2mmnsWHDBm688UYWL15MgwYNAOjUqRNjx45l7ty5UQ+uilWVCfrGmPJXWjtZebWf1a1bt/D9xo0beeyxx3j33XdZvXo1Q4YMCdlfPT09vfB9WloaeXl5IY9ds2bNiGmi1bhxY1avXk2/fv2YPXs21113HQCLFy/m+uuvZ/ny5fTu3Zv8/Py4ntcPC/rGGN8S2X62b98+6tevzwknnMD27dtZvHhx3M/Rt29fFixYAMCaNWtC3kkE69OnD0uWLGH37t3k5eUxf/58+vfvT25uLqrKT3/6U2bMmMEnn3xCfn4+OTk5DBo0iFmzZrFr1y4OlawrqwBVok7fGFMxEtl+1qNHDzIyMujQoQOtW7emb9++cT/HL3/5S66++moyMjIKX4GqmVBatGjBvffey4ABA1BVLr74YoYOHconn3zCtddei6oiIjz44IPk5eVx5ZVXsn//fgoKCrjllluoX79+3D9DJEn3jNzMzEy1h6gYU3HWr19Px44dE52NpJCXl0deXh61atVi48aNnH/++WzcuJHq1ZOrfBzqdyYiK1Q1s5RdCiXXJzHGmAQ6cOAA5557Lnl5eagqTz31VNIF/FhVrU9jjDExaNiwIStWrEh0NsqVr4ZcERkiIhtEZJOIHNfhVUSuF5E1IrJSRN4XkQxvfW9v3UoRWSUiI+L9AYwxxvgXMeiLSBowG7gQyABGB4J6kHmq2kVVuwGzgIe99WuBTG/9EOApEbG7C2OMSRA/Jf3ewCZV3ayqR4H5wPDgBKq6L2ixLqDe+kOqGugAWyuw3hhjTGL4KXU3B74OWs4B+pRMJCI3AFOAdGBQ0Po+wLNAa+CqoC8BY4wxFSxug7NUdbaqngrcCkwPWr9MVTsBvYDbRKRWyX1FZKKIZIlIVm5ubryyZIypBAYOHHjcQKtHH32USZMmhd2vXr16AHzzzTeMHDkyZJoBAwYQqQv4o48+WmyQ1EUXXRSXeXHuvvtuHnrooZiPE29+gv42oGXQcgtvXWnmA5eUXKmq64EDQOcQ255W1UxVzWzatKmPLBljqorRo0czf/78Yuvmz5/P6NGjfe1/yimn8NJLL0V9/pJB/1//+hcNGzaM+njJzk/QXw60F5G2IpIOjAIWBicQkfZBi0OBjd76toGGWxFpDXQAsuOQb2NMFTFy5EjeeOONwgemZGdn880339CvX7/CfvM9evSgS5cuvP7668ftn52dTefOrix5+PBhRo0aRceOHRkxYgSHDx8uTDdp0qTCaZnvuusuAB5//HG++eYbBg4cyMCBAwFo06YNu3btAuDhhx+mc+fOdO7cuXBa5uzsbDp27MjPf/5zOnXqxPnnn1/sPKGsXLmSs846i65duzJixAi+//77wvMHploOTPT23//+t/AhMt27d2f//v1RX9tQItbpq2qeiEwGFgNpwLOquk5EZgBZqroQmCwig4FjwPfAOG/3c4CpInIMKAB+oaq74voJjDFxc9NNEO8HQnXrBl68DKlRo0b07t2bRYsWMXz4cObPn8/ll1+OiFCrVi1effVVTjjhBHbt2sVZZ53FsGHDSn1O7JNPPkmdOnVYv349q1evLjY18syZM2nUqBH5+fmce+65rF69mhtvvJGHH36YJUuW0KRJk2LHWrFiBc899xzLli1DVenTpw/9+/fnxBNPZOPGjbz44ov86U9/4vLLL+fll18OOz/+1VdfzRNPPEH//v258847ueeee3j00Ud54IEH2LJlCzVr1iysUnrooYeYPXs2ffv25cCBA9SqdVyNeEx81emr6r9U9XRVPVVVZ3rr7vQCPqr6K1XtpKrdVHWgqq7z1r8QtL6Hqr4W19wbY6qE4Cqe4KodVeX222+na9euDB48mG3btrFjx45Sj7N06dLC4Nu1a1e6du1auG3BggX06NGD7t27s27duoiTqb3//vuMGDGCunXrUq9ePS699FLee+89ANq2bUu3bt2A8NM3g5vff8+ePfTv3x+AcePGsXTp0sI8jhkzhjlz5hSO/O3bty9Tpkzh8ccfZ8+ePXEfEWx95o0xhcKVyMvT8OHDufnmm/nkk084dOgQPXv2BGDu3Lnk5uayYsUKatSoQZs2bUJOpxzJli1beOihh1i+fDknnngi48ePj+o4AYFpmcFNzRypeqc0b7zxBkuXLuUf//gHM2fOZM2aNUydOpWhQ4fyr3/9i759+7J48WI6dOgQdV5LsqmVjTEJV69ePQYOHMjPfvazYg24e/fu5aSTTqJGjRosWbKEraEehh3kxz/+MfPmzQNg7dq1rF69GnDTMtetW5cGDRqwY8cOFi1aVLhP/fr1Q9ab9+vXj9dee41Dhw5x8OBBXn31Vfr161fmz9agQQNOPPHEwruEF154gf79+1NQUMDXX3/NwIEDefDBB9m7dy8HDhzgyy+/pEuXLtx666306tWLzz//vMznDMdK+saYpDB69GhGjBhRrCfPmDFjuPjii+nSpQuZmZkRS7yTJk3immuuoWPHjnTs2LHwjuHMM8+ke/fudOjQgZYtWxablnnixIkMGTKEU045hSVLlhSu79GjB+PHj6d3794ATJgwge7du4etyinNX//6V66//noOHTpEu3bteO6558jPz2fs2LHs3bsXVeXGG2+kYcOG3HHHHSxZsoRq1arRqVOnwqeAxYtNrWxMirOplSufWKZWtuodY4xJIRb0jTEmhVjQN8aQbNW8pnSx/q4s6BuT4mrVqsXu3bst8FcCqsru3btjGrBlvXeMSXEtWrQgJycHm+ywcqhVqxYtWrSIen8L+sakuBo1atC2bdtEZ8NUEKveMcaYFGJB3xhjUogFfWOMSSEW9I0xJoVY0DfGmBRiQd8YY1KIBX1jjEkhFvSNMSaF+Ar6IjJERDaIyCYRmRpi+/UiskZEVorI+yKS4a0/T0RWeNtWiMigeH8AY4wx/kUM+iKSBswGLgQygNGBoB5knqp2UdVuwCzgYW/9LuBiVe2Ce1j6C3HLuTHGmDLzU9LvDWxS1c2qehSYDwwPTqCq+4IW6wLqrf9UVb/x1q8DaotITYwxxiSEn7l3mgNfBy3nAH1KJhKRG4ApQDoQqhrnMuATVf0hxL4TgYkArVq18pElY4wx0YhbQ66qzlbVU4FbgenB20SkE/AgcF0p+z6tqpmqmtm0adN4ZckYY0wJfoL+NqBl0HILb11p5gOXBBZEpAXwKnC1qn4ZTSaNMcbEh5+gvxxoLyJtRSQdGAUsDE4gIu2DFocCG731DYE3gKmq+kF8smyMMSZaEYO+quYBk4HFwHpggaquE5EZIjLMSzZZRNaJyEpcvf64wHrgNOBOrzvnShE5Kf4fwxhjjB+SbI9Iy8zM1KysrERnwxhjKhURWaGqmZHS2YhcY4xJIRb0jTEmhVjQN8aYFGJB3xhjUogFfWOMSSEpEfTnzoU2baBaNfdz7txE58gYYxLDz9w7ldrcuTBxIhw65Ja3bnXLAGPGJC5fxhiTCFW+pD9tWlHADzh0yK03xphUU+WD/ldflW29McZUZVU+6Jc2U7PN4GyMSUVVPujPnAl16hRfV6eOW2+MMammygf9MWPg6aehdWsQcT+fftoacY0xqanK994BF+AtyBtjTAqU9I0xxhSxoG+MMSnEgr4xxqQQC/rGGJNCfAV9ERkiIhtEZJOITA2x/XoRWeM9DvF9Ecnw1jcWkSUickBE/hDvzBtjjCmbiEFfRNKA2cCFQAYwOhDUg8xT1S6q2g2YBTzsrT8C3AHcEr8sG2OMiZafkn5vYJOqblbVo8B8YHhwAlXdF7RYF1Bv/UFVfR8X/I0xxiSYn376zYGvg5ZzgD4lE4nIDcAUIB0YVJZMiMhEYCJAK5sfwRhjyk3cGnJVdbaqngrcCkwv475Pq2qmqmY2bdo0XlkyxhhTgp+gvw1oGbTcwltXmvnAJbFkqqLZQ1aMManCT9BfDrQXkbYikg6MAhYGJxCR9kGLQ4GN8cti+Qo8ZGXrVlAtesiKBX5jTFUUMeirah4wGVgMrAcWqOo6EZkhIsO8ZJNFZJ2IrMTV648L7C8i2bjePONFJCdEz5+EsoesGGNSiahqovNQTGZmpmZlZVXY+apVcyX8kkSgoKDCsmGMMTERkRWqmhkpXcqPyLWHrBhjUknKB317yIoxJpWkfNC3h6wYY1JJSjxEJRJ7yIoxJlWkfEnfGGNSiQV9Y4xJIRb0fbARu8aYqsLq9CMIjNgNDOAKjNgFawcwxlQ+VtKPwEbsGmOqEgv6EXz1VdnWG2NMMrOgH4GN2DXGVCUW9COwEbvGmKrEgn4ENmLXGFOVWO8dH2zErjGmqrCSfhxYP35jTGVhJf0YWT9+Y0xlYiX9GFk/fmNMZeIr6IvIEBHZICKbRGRqiO3Xi8gaEVkpIu8HPxJRRG7z9tsgIhfEM/PJwPrxG2Mqk4hBX0TSgNnAhUAGMDrEc27nqWoXVe0GzMI9Excv3SigEzAE+KN3vCrD+vEbYyoTPyX93sAmVd2sqkeB+cDw4ASqui9osS4QeOrscGC+qv6gqluATd7xqgzrx2+MqUz8BP3mwNdByzneumJE5AYR+RJX0r+xjPtOFJEsEcnKzc31m/ek4Kcfv/XuMcYki7g15KrqbFU9FbgVmF7GfZ9W1UxVzWzatGm8slRhxoyB7GwoKHA/Swb8iRNdrx7Vot49FviNMYngJ+hvA1oGLbfw1pVmPnBJlPtWOda7xxiTTPwE/eVAexFpKyLpuIbZhcEJRKR90OJQYKP3fiEwSkRqikhboD3wcezZrjysd48xJplEHJylqnkiMhlYDKQBz6rqOhGZAWSp6kJgsogMBo4B3wPjvH3XicgC4DMgD7hBVfPL6bMkpVatXJVOqPXGGFPRRFUjp6pAmZmZmpWVlehsxE3JEbvgevfYpG3GmHgSkRWqmhkpnY3ILWc2S6cxJpnY3DsVwGbpNMYkCyvpJ5j14TfGVCQL+gnkpw+/fSkYY+LJgn4CRerDbwO7jDHxZkE/gSL14beBXcaYeLOgn0CRZui0gV3GmHizoJ9AkWbotGmbjTHxZkE/gSL14bdpm40x8Wb99BMsXB/+wPpp01yVTqtWLuBbn39jTLQs6Cc5G9hljIknq94xxpgUYkHfGGNSiAV9Y4xJIRb0qzibxsEYE8yCfiUXLqjbNA7GmJIs6FdikYK6TeNgjCnJV9AXkSEiskFENonI1BDbp4jIZyKyWkTeEZHWQdseFJG13uuKeGY+1UUK6jaNgzGmpIhBX0TSgNnAhUAGMFpEMkok+xTIVNWuwEvALG/foUAPoBvQB7hFRE6IX/ZTW6SgbtM4GGNK8lPS7w1sUtXNqnoUmA8MD06gqktUNVDm/Aho4b3PAJaqap6qHgRWA0Pik3UTKaj7mcYhUkOvNQQbU7X4CfrNga+DlnO8daW5FljkvV8FDBGROiLSBBgItCy5g4hMFJEsEcnKzc31l3MTMahHmtsnUpuANQQbU/WIqoZPIDISGKKqE7zlq4A+qjo5RNqxwGSgv6r+4K2bBvwUyAV2AstV9dHSzpeZmalZWVlRfpzUM3du9HPztGnjAnlJrVtDdnbk7caY5CEiK1Q1M1I6P3PvbKN46byFt67kCQcD0wgK+ACqOhOY6aWZB3zh45zGp1jm5onUJmANwcZUPX6qd5YD7UWkrYikA6OAhcEJRKQ78BQwTFV3Bq1PE5HG3vuuQFfgrXhl3sQmUpuANQQbU/VEDPqqmoerslkMrAcWqOo6EZkhIsO8ZL8D6gF/F5GVIhL4UqgBvCcinwFPA2O945kkEKlNwObzN6YKUtWkevXs2VNNxZkzR7V1a1UR93POnLJtT9SxjTHFAVnqI8ZGbMitaNaQWzUEev4EDx6rU6eo91Ck7caYsvHbkGvTMJiohevDH2m0sE0RYUxi2JOzTFRKltQDffjBldStZ5AxyclK+iYqkUrq1jPImORkQd9EJVJJPR49g2wKCGPiz4K+iUqkknqkKSBinSLCGBMd671jolLevW9inQIilukpjKmMrPeOKVeRSuqxiqWh1+4SjCmdBX0TtTFjXKm7oMD9jGdJ2k9Db2l1/tYd1JjSWZdNk5RmzgxdfRRo6A3XZdS6gxpTOivpm6QUqfooXGneuoMaUzoL+iZphas+Cleat4nijCmdBX1TKYUrzZd3I7MxlZkFfVMp+XlUZLhGZhv4ZVKVBX1TKcVSmvfTpdO+FExVZYOzTMqJNPDLpn02lZENzjKmFJG6dMajn7/dKZhk5Svoi8gQEdkgIptEZGqI7VNE5DMRWS0i74hI66Bts0RknYisF5HHRUTi+QGMKatIXTpj7edvI4JNMosY9EUkDZgNXAhkAKNFJKNEsk+BTFXtCrwEzPL2/X9AX9wD0TsDvYD+ccu9MVGI1Agcy2hgsBHBJrn5Ken3Bjap6mZVPQrMB4YHJ1DVJaoa+DP/CGgR2ATUAtKBmrgHpe+IR8aNiVakRuBIXwqRSvI2ItgkMz9BvznwddByjreuNNcCiwBU9UNgCbDdey1W1fUldxCRiSKSJSJZubm5fvNuTNTCdemMZTQw2Ihgk9zi2pArImOBTOB33vJpQEdcyb85MEhE+pXcT1WfVtVMVc1s2rRpPLNkTFSiHQ0M/u4UrJHXJIqfoL8NaBm03MJbV4yIDAamAcNU9Qdv9QjgI1U9oKoHcHcAZ8eWZWMSK5YHyFgjr0k0P0F/OdBeRNqKSDowClgYnEBEugNP4QL+zqBNXwH9RaS6iNTANeIeV71jTGXiZ26f0u4UKqKR1+4kTDgRp1ZW1TwRmQwsBtKAZ1V1nYjMALJUdSGuOqce8HevR+ZXqjoM15NnELAG16j7pqr+ozw+yOHDMH48NGkCTZuG/tmkCaSnl8fZTSoJDuBlfTJXPBp5wz0VLNyU0zawzACgqkn16tmzp0bj229VTz9dtVEjVXfjfPyrQQPVTz+N6vDGxEXr1qH/Nlu3LkozZ45bFnE/58wpvq1OneL71qlTlMbP8U3VhCuER4yxVXIahrw8+O47yM2FXbvcz9xcuOMO6NULFi2KU2aNKaNIUzxE2h5pColq1VyYL0nEVTWZqsvvNAwJL9mXfEVb0vdj1ixX6lm6tNxOYUxE4UrykUrqIqG3i/jbP9L5w20zyQ2fJf2EB/mSr/IM+gcPqjZrptqvn2pBQbmdxpioxRrUI1X/hNseaV8/7EsjcSzol2L2bPep33yzXE9jTFRiDeqBNNHcScTaHhCPLw0TPb9Bv0rW6Ydz9CiccQY0bgzLl7u6TmOShZ9pncP13okkXJ0/xNYeEKm9wZQvm1q5FOnpcPfdsGIFvPpqonNjTHF+Hg4T6alg4YQbWBbr9BE251DlkHJBH2DsWOjQwfXmyc9PdG6MKS6WoB5JuIFlsT5QPtbZSf1sN3Hgpw6oIl/lXacfsGCBq3N84YUKOZ0xSSOW3juxjCGIdbsJD2vIDS8/X7V7d9V27VSPHq2QUxpTqZVnI7Kf7bH2DKrqPYss6PvwxhvuCvzv/1bYKY2ptGLt3ROpO2q47bHeBaTCXYTfoJ9yvXeCqcI557geB5s2Qa1aFXJaYyqlWEf7RurdE247xNYzKBV6FlnvHR9E4Le/hW3b4MknE512wgNgAAAV3klEQVQbY5JbrL17IjUUh9sea88g61lUJKWDPkD//nDeeS7479+f6NwYk7xi7d0TqTtquO2x9gyyp5kF8VMHVJGviqzTD1i2zNXx3XtvhZ/amEolUY2h1jMoMqwht2wuuUS1Rg3V889X/f3vVdessfl5jEkmsfQMirR/eeatoljQL6Nvv1W96SbVjh2L/mBOOUV1/HjVefNUd+5MSLaMMT5E6hnkR7RjFJLlLsKCfgy++kr1mWdUL7+86KEsIqqXXWZ9+o1JRuU9WVy47X7vMk4+uWh9eXwhxDXoA0OADcAmYGqI7VOAz4DVwDtAa2/9QGBl0OsIcEm4cyVD0A+Wl6f68ceqv/mNu1oTJ1q1jzHJJtbSdiwDx8LdZRQUqN5+u2q1auV/JxC3oI97Lu6XQDsgHVgFZJRIMxCo472fBPwtxHEaAd8F0pX2SragH+y229wVe/jhROfEGFNSLPXqsQwcK+0LoWlT1bPPDr2tPNob/AZ9P102ewObVHWzqh4F5gPDS/QAWqKqgclgPwJahDjOSGBRULpK57774LLL4Ne/hn+Uy+PdjTHRKq/ZRyNtD9WVVcQ9ovWbb0o/Z2CMQGA67a1b3ddB4GH25TXZnJ+g3xz4Omg5x1tXmmuBUE+hHQW8GGoHEZkoIlkikpWbm+sjS4lRrRo8/zz06AGjR8OqVYnOkTEmHmIZOBYYX3DKKUXbmjVzsWLjxqIRxSUFvkimTSv+/ARwy9OmRf95wop0K4AroT8TtHwV8IdS0o7FlfRrlljfDMgFakQ6XzJX7wRs26baooVqy5aq33yT6NwYY+IhlhlGVVVvvVW1enXVV15xEzoG71eyvaFataL949HzSNV/9U7EuXdE5GzgblW9wFu+zfuyuL9EusHAE0B/Vd1ZYtuvgE6qOjHSl1BFzr0Ti5Ur3bw9GRnwn/8cXwowxqQOVTj9dGjbFt566/jtwU87O+EE2LcPNm92I4fjNS9QPOfeWQ60F5G2IpKOq6ZZWOJk3YGngGElA75nNKVU7VRW3brBvHmQlQXjxvmbcCoZbdrk2io2b050TkxZzJ1rT35LJmvXuv+lyy4LvT24vWH1alfn/8wzblus01uUmZ/bAeAi4AtcL55p3roZuCAP8Dawg6KumQuD9m0DbAOq+TlXZajeCfbQQ+5WbNq0ROfEv/x8N630kCFFt5KjRiU6V8avTZtcNUKdOqrbtyc6N0ZV9a67XHWM39/H0KGqzZoVjfupyN47voJ+Rb4qW9AvKFCdMMFdyQceUP3hh/if4+BB1SVLVPfti+04333nppho187lt1kz1XvuUb3mGjcFhQWQymH0aNXatV3g/8UvEp2bquXYsejG4XTpotqvn//0r7/u/gdffbXs5yqNBf0KdPSo6rBh7mqedprq3/8evwFcBQVuJDCopqWp9u7tGowWLVLdvz9yvrZsUX3nHdWf/9wFClA95xzV+fOLShkbNrj1990Xnzyb8rNiRdGd5S9+4QL/xo2JzlXV8MMPrkB0++1l2++LL9zv5NFH/e9z7Jib5uXCC8t2rnAs6FewggJXZdKpk7uqZ52lunRp7Mf985/d8W68UXX6dNW+fV2pPPAlcNZZbtDYH//ofl55pUvTsmXxUYC1a7s7kk8/DX2ewYPdPnl5sefZlJ/zz1dt3Fh1zx53Z1anjuoVVyQ6V1XDSy+5/5UGDSIXqILdf7/b76uvyna+O+5w1TnZ2WXbrzQW9BMkL88F6lNOcVd3+HDV9eujO9YXX6jWras6cGDxLmAHDqi+9ZYrkZx9tivtgfvZtq3qgAGqV1/t/qj+9CeX9rvvwp/rlVfcMV5/Pbq8mvL39tt63Ijw6dPduqysxOWrqrjgAvf/BmV7hGqvXu5VVtnZLujfcUfZ9w3Fgn6CHTyoOnOmav36rkQ+caLq7t3+9z961P0hnXhi5BLE/v2qOTmxldKPHVNt3tz94Zvkk5+v2rOnaqtWqocPF63fu9eV/M87L3F5qwq2bHEB+M47Vbt3V+3c2V8V7datWtieF42LLnIFxGPHots/mN+gn/JPziovderA7bfDl1/CDTfAs89C377+H892zz2wfDk89RS0bBk+bb160Lw5pKVFn9/q1d3Q78WLXdczk1xeeglWrIB77y3+LOcTToDp0+Hf/4a3305c/iq7Z591P3/2M/jlL10XzKVLI+/3yivuZ2ldNSOZONFN1fDGG9HtHxU/3wwV+aoqJf2Sli5VbdjQfauvXh0+7X//60od11xTMXkL2LbNVRHdckvFnteEd/So6yDQpUvou7kjR1w3v549i1cDpqIjR9xdb1kE7nKHDHHLhw65KdVHjoy87znnqHbtWvZ8Bp+7WTNX4o8VVtJPLv36wXvvFb0vrRSxZw9cdRW0awePPVZx+QM3d8gll7hSz+HD/vZZuRJuugm+/z7++SkocCXYZLzzUIX580OPpIy3Z55x1+D++0PfzdWs6e4AVqxwdwSp6tgxuOACOOMMyMnxv9+bb8K2bfDzn7vl2rVhwgQ3+C3ccb79Fj74AC69NPo8V68O114LixZV4EPa/XwzVOSrqpb0A7KzVTt0UK1Z0zWeBisocIOk0tJUP/ooMfl7911XR/mXv0ROu2OHm4MI3BPHtmyJXz5WriyallbEDWZZvDg5SrIFBe4pa4HpcbdtK79z7d+v+qMfqf74x+HrmPPy3J3Aaael7oN+Jk/Wwg4NV1/tf79hw9w1Dr5ugTr+6dNL3+/JJ9351qyJOsuqWtSge+edsR0Ha8hNXrt2ua6W1aoV7yXwwgua8P7yBQXuS6lPn/Dpjh1zvYpq1VKdPds1OJ90knvgTCz27nUBNS1NtUkTd33uusv9U4LqGWeoPvFE7APVolVQUBRcrrxStV49d3u/Z0/5nO/ee925Pvwwctp//tOl/eMfyycvySzQtfnXv3bjWMCNaYhk2zb3t3brrcdvGzbM/U0fORJ638GDVU8/PT5jci680FUxxdKga0E/yR086Eqv4ILal1+6nj7nnJP4vvKPPaYRuwEG/rGee84tr1/vuovWrq362mtlP2dBgerf/ubaPERUr7uueG+nI0fc0PTevd1569d3Yxe++KLs51J1A5rGjVP9yU9U1671t09+vur117vz33KLy/Nbb7mS5YABxXvVxMPOne5zjhjhL31Bgbsj+NGPytbPvLL78EPV9HTXg+nYMfcF3KSJ+51ECsj33ed+n6EGuL31ltv2wgvHb9u9231ZTJ0an8/w6qsac5dpC/qVwNGjrrEWXMPRCSfEb6BGLL7/3g36ufba0NsDffqvu674+h073B2CSNlGJ27Y4P5hQbVHD9Vly8Kn/+gj1TFjigapXXCB+6LxU0rautWNTk5Lc19QjRq549xzT/gpNPLzi6bbmDq1eDCZM8etHzkyvl/YN93k7gY/+8z/Ph9+6PJy773xy0dF2r5dddIk15nBj23bXENou3bFCwmzZ7vrEK4Akp/vCioDB5a+/YwzQt/1PvecO/7y5f7yGcnRo+5zDB0a/TEs6FcSBQVuJK2I6rx5ic5NkcC0Dd9/X3z9hg2u9NmrV+jb3oMHXck0MIo4VBAsKHB1pi++6ErO6eluFOQf/lC2oLl9uwvWzZu787Vs6UpuoeYQ2r7d5Sc93b1++Uu3budON5cNuL7Zoaqn8vJUx493ae64I3Tp8fe/d9snT47P7f6WLS6fEyaUfd9LLnG/ow8+cFNwLFjg6p/vu0/15ptdffdll6k+/3z5zBUVrbVriz96cNIkV91XmsOHXUCuW/f4evVjx1w15emnl97GESjJh/u/e/xxl6bk38VPfuLyGs/nZT/xhJvAMdpjWtCvZMqrTjhan3yix80ncuCAm2aicWNXYi5NXp7qlClaOCJ5505XcnvwQReQTj656B+7dm0XhGKZ7O3YMXf3EbhbqF7dTU3wn/+49pNbb3V3Lmlp7sssVN4XLnRfHtWquXrhgweLPsvYse6499wTPh+//rVL99vfRv9ZVF0wGzDAtZeUtfuhqrszKPkg7sCrbl335RhogG/WzA0i3LUr8nF37nRfHv37u7+B005z1W0XXOC+OG+4wX0pPvJI5Lu1kt5+233xn3yy6958882uINSypZvepKSCgqK75JIdIgICbRyPPx56+09/6j5HuGq5vXtdu824cUXr9u1zX8g33+z741UIC/omZmed5W5vCwrca/RoF0z+/W9/+//hD8cHn/btVa+6yt1+f/JJfEYiBtuwwf0znniiFs5PJOICd6SJyfbscVVWoHrqqa4kGLgLmDkz8rnz8121U3BbR1kdPap68cVaal2yX0uXqr78svviW73aVYMEB7eCAjdp3/nnF335Xn+96uefFz/O3r2qf/2r68OelubSdujgRpiPHu0Cfq9e7no1alT8KVBXXBG+cBDw7LPui7pz5+LpP/xQNSPDHWvsWNXc3KJtgRJ4uB4vBQWq557rAnvJO9adO121np/A/YtfuN52O3e65RdfdOd+773I+1YkC/omZs8/7/5C3nmnqHG3rKXYd95RvftuV+oK/qctb4cOucD7q1/5b6gNWLLEBbFA8HrwQf/7/vCDu+NIS3OfuSzy812PIHBfihVlzRrXflOzpjv30KGqTz+teumlRevatHFtGatWha9+yM93wfGOO9ydSq1a7v2BA8enLSgomjvovPNC3+0eOeICe/Xqqk2busb+d99113f48MhdeFeudF9EJQcc/u537rx+2kvWrXNp77/fLY8c6e5IkqH7cDAL+iZmhw+7UlK3bu6fbtiw5PtDLy8HD7peVc88U/Z99+1zo2Nr11adO9dfHW1BgavDjkf1ULR27HBVWCed5PJx8smuHeTDD6OrZ9661Y07AVd19sILRX8/R44UfcFNmBB5bMGqVaqZmS59jRpuXEi4+v5g11zjqmO+/NItFxS4O9i+ff1/lkGD3LxH+/e7qsJJk/zvW1Es6Ju4+M1v3F/Jaacdf4tsSvftt66REVzVSKSBa1OnurT/8z/xbRyMxuHDroQcr55I779fFLD79FF98033wJHAF5zfz3vsmOqsWe5YZemqu22bC9Q//alb/u9/1fcAxIBAj7Vx49zPt9/2v29FiWvQB4YAG4BNwNQQ26cAnwGrgXeA1kHbWgFvAeu9NG3CncuCfnLJyXGNr7GOOkxFeXmuWqxuXRd0Hn44dCANzMd+3XWJD/jlJT/fBdlmzdxnrVnTPcinotx1lzvvBx+49oEGDYoa6/04dsw1Kge6VyfjqOe4BX0gDfds3HZAOrAKyCiRZiBQx3s/Cfhb0Lb/AOd57+sF0pX2sqBvqpqtW4sG4vXq5UrRAYGh/KNHJ35QXkXYv9/Vp1f0NCMHDrgvnO7dXTtDNI+Z/O1v3e+qoidC9CueQf9sYHHQ8m3AbWHSdwc+8N5nAO/7yUjgZUHfVEUFBa5ke9JJrn3ktttcrxUR1+c7GUuOVc2zz2ph43xpT5ALJzfXdVFN1LxYkfgN+tV9zMnWHPg6aDkH6BMm/bXAIu/96cAeEXkFaAu87VUP5QfvICITgYkArVq18pElYyoXEbjiCjjvPLjlFjdjJsCAAbBgAdSokdDspYSrr4bZsyE9Hbp1K/v+TZrAsmXxz1dFi+vUyiIyFsgEfuetqg70A24BeuGqiMaX3E9Vn1bVTFXNbNq0aTyzZExSadTITV399ttuSuqFC91Uvqb8paXBkiXuQUGpzE9JfxsQ/OymFt66YkRkMDAN6K+qP3irc4CVqrrZS/MacBbw51gybUxld+657mUqVv36ic5B4vkp6S8H2otIWxFJB0YBC4MTiEh34ClgmKruLLFvQxEJFN8H4XrwGGOMSYCIQV9V84DJwGJct8sFqrpORGaIyDAv2e9wPXP+LiIrRWSht28+rmrnHRFZAwjwp3L4HMYYY3wQ1+ibPDIzMzUrKyvR2TDGmEpFRFaoamakdPaMXGOMSSEW9I0xJoVY0DfGmBRiQd8YY1KIBX1jjEkhSdd7R0Ryga1hkjQBdlVQdsrK8hYdy1t0LG/Rqap5a62qEac0SLqgH4mIZPnplpQIlrfoWN6iY3mLTqrnzap3jDEmhVjQN8aYFFIZg/7Tic5AGJa36FjeomN5i05K563S1ekbY4yJXmUs6RtjjImSBX1jjEkhlSboi8gQEdkgIptEZGqi81OSiGSLyBpvaumEThMqIs+KyE4RWRu0rpGI/FtENno/T0yivN0tItu8a7dSRC5KQL5aisgSEflMRNaJyK+89Qm/bmHylgzXrZaIfCwiq7y83eOtbysiy7z/1795z+JIlrz9RUS2BF23KB6eGLc8ponIpyLyT2+5/K+bnwfpJvoFpAFf4h63mA6sAjISna8SecwGmiQ6H15efgz0ANYGrZuFez4xwFTgwSTK293ALQm+Zs2AHt77+sAXQEYyXLcweUuG6yZAPe99DWAZ7ul4C4BR3vr/BSYlUd7+AoxM5HULyuMUYB7wT2+53K9bZSnp9wY2qepmVT0KzAeGJzhPSUtVlwLflVg9HPir9/6vwCUVmilPKXlLOFXdrqqfeO/34x4Y1JwkuG5h8pZw6hzwFmt4L8U9Je8lb32irltpeUsKItICGAo84y0LFXDdKkvQbw58HbScQ5L80QdR4C0RWSEiExOdmRB+pKrbvfffAj9KZGZCmCwiq73qn4RUPQWISBugO65kmFTXrUTeIAmum1dFsRLYCfwbd1e+R91T9yCB/68l86aqges207tuj4hIzUTkDXgU+B+gwFtuTAVct8oS9CuDc1S1B3AhcIOI/DjRGSqNunvHpCnxAE8CpwLdgO3A7xOVERGpB7wM3KSq+4K3Jfq6hchbUlw3Vc1X1W5AC9xdeYdE5COUknkTkc7Abbg89gIaAbdWdL5E5CfATlVdUdHnrixBfxvQMmi5hbcuaajqNu/nTuBV3B9/MtkhIs0AvJ87I6SvMKq6w/vnLMA9Qzkh105EauCC6lxVfcVbnRTXLVTekuW6BajqHmAJcDbQUESqe5sS/v8alLchXnWZquoPwHMk5rr1BYaJSDauunoQ8BgVcN0qS9BfDrT3WrbTgVHAwgTnqZCI1BWR+oH3wPnA2vB7VbiFwDjv/Tjg9QTmpZhAUPWMIAHXzqtP/TOwXlUfDtqU8OtWWt6S5Lo1FZGG3vvawHm4NoclwEgvWaKuW6i8fR70JS64OvMKv26qepuqtlDVNrh49q6qjqEirluiW6/L0Mp9Ea7XwpfAtETnp0Te2uF6FK0C1iU6f8CLuNv9Y7h6wWtx9YXvABuBt4FGSZS3F4A1wGpckG2WgHydg6u6WQ2s9F4XJcN1C5O3ZLhuXYFPvTysBe701rcDPgY2AX8HaiZR3t71rttaYA5eD59EvYABFPXeKffrZtMwGGNMCqks1TvGGGPiwIK+McakEAv6xhiTQizoG2NMCrGgb4wxKcSCvjHGpBAL+sYYk0L+P9FvJuZylVjjAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss) + 1)\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "성공이네요! 30번째 에포크까지 과대적합이 일어나지 않았습니다. 평가 점수는 안정적이지만 이전보다 더 나아지진 않았습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.3.7 스태킹 순환 층\n",
    "\n",
    "과대적합은 더 이상 없지만 성능상의 병목이 있는 것 같으므로 네트워크의 용량을 늘려야 합니다. 일반적인 머신 러닝 작업 흐름을 기억하세요. \n",
    "\n",
    "1. 드롭아웃 등을 사용하여 과대적합을 줄이는 기본 단계를 거쳤다고 가정하고,\n",
    "\n",
    "2. 과대적합이 일어날 때까지 네트워크의 용량을 늘리는 것이 좋습니다. 너무 많이 과대적합되지 않는 한 아직 충분한 용량에 도달한 것이 아닙니다.\n",
    "\n",
    "네트워크의 용량을 늘리려면 일반적으로 층에 있는 유닛의 수를 늘리거나 층을 더 많이 추가합니다. 순환 층 스태킹은 더 강력한 순환 네트워크를 만드는 고전적인 방법입니다. 예를 들어 구글 번역 알고리즘의 현재 성능은 7개의 대규모 LSTM 층을 쌓은 대규모 모델에서 나온 것입니다.\n",
    "\n",
    "케라스에서 순환 층을 차례대로 쌓으려면 모든 중간 층은 마지막 타임스텝 출력만이 아니고 전체 시퀀스(3D 텐서)를 출력해야 합니다. `return_sequences=True`로 지정하면 됩니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "gru_1 (GRU)                  (None, None, 32)          4512      \n",
      "_________________________________________________________________\n",
      "gru_2 (GRU)                  (None, 64)                18624     \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 1)                 65        \n",
      "=================================================================\n",
      "Total params: 23,201\n",
      "Trainable params: 23,201\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Epoch 1/10\n",
      "500/500 [==============================] - 180s 360ms/step - loss: 0.3363 - val_loss: 0.2794\n",
      "Epoch 2/10\n",
      "500/500 [==============================] - 178s 356ms/step - loss: 0.3130 - val_loss: 0.2711\n",
      "Epoch 3/10\n",
      "500/500 [==============================] - 178s 356ms/step - loss: 0.3094 - val_loss: 0.2701\n",
      "Epoch 4/10\n",
      "500/500 [==============================] - 178s 356ms/step - loss: 0.3038 - val_loss: 0.2674\n",
      "Epoch 5/10\n",
      "500/500 [==============================] - 178s 356ms/step - loss: 0.2964 - val_loss: 0.2685\n",
      "Epoch 6/10\n",
      " 81/500 [===>..........................] - ETA: 1:51 - loss: 0.2997"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras import layers\n",
    "from keras.optimizers import RMSprop\n",
    "\n",
    "K.clear_session()\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.GRU(32,\n",
    "                     dropout=0.1,\n",
    "                     recurrent_dropout=0.5,\n",
    "                     return_sequences=True,\n",
    "                     input_shape=(None, float_data.shape[-1])))\n",
    "model.add(layers.GRU(64, activation='relu',\n",
    "                     dropout=0.1, \n",
    "                     recurrent_dropout=0.5))\n",
    "model.add(layers.Dense(1))\n",
    "model.summary()\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae')\n",
    "history = model.fit_generator(train_gen,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=40,\n",
    "                              validation_data=val_gen,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xt8FdW5//HPw10ucpPWQlRQOUK4CBgBSxFQtCAVpKUKgnelWqltqT1StNbScqqUg1bL8Ug9apUoh+KxUpXSXyst0ioSLIKAFETAcFFAQQEVQ57fH2sSdmIum7CTvcl8369XXtkze83Ms3fgmTVrzVpj7o6IiMRDnXQHICIiNUdJX0QkRpT0RURiRElfRCRGlPRFRGJESV9EJEaU9OWImFldM9tnZiensmw6mdnpZpbye5fNbLCZbUpYXmdm/ZMpW4VjPWxmk6u6fQX7/bmZPZbq/Ur61Et3AFK9zGxfwmJj4FPgULT8LXfPPZL9ufshoGmqy8aBu5+Riv2Y2fXAOHcfmLDv61Oxb6n9lPRrOXcvTrpRTfJ6d/9zeeXNrJ67F9REbCJS89S8E3PR5fv/mtlTZvYRMM7MzjGzV8xsj5ltN7P7zax+VL6embmZtY+WZ0fvLzCzj8zsZTPrcKRlo/eHmtm/zGyvmT1gZn83s6vLiTuZGL9lZhvM7AMzuz9h27pmdq+Z7TazjcCQCr6f281sTql1M81sRvT6ejNbG32et6JaeHn7yjezgdHrxmb2RBTbauCsUmXvMLON0X5Xm9nwaH034NdA/6jpbFfCd3tXwvY3Rp99t5n93sy+lMx3UxkzGxnFs8fMXjSzMxLem2xm28zsQzN7M+Gz9jWz16L175rZL5M9nlQDd9dPTH6ATcDgUut+DhwELiZUAo4Dzgb6EK4ETwX+BUyIytcDHGgfLc8GdgE5QH3gf4HZVSj7BeAjYET03kTgM+Dqcj5LMjE+CzQH2gPvF312YAKwGsgCWgOLw3+FMo9zKrAPaJKw7/eAnGj54qiMAecBHwPdo/cGA5sS9pUPDIxeTwf+CrQETgHWlCp7KfCl6G9yeRTDF6P3rgf+WirO2cBd0esLoxh7AI2A/wJeTOa7KePz/xx4LHrdOYrjvOhvNBlYF73uAmwGTozKdgBOjV4vA8ZEr5sBfdL9fyHOP6rpC8ASd/+Duxe6+8fuvszdl7p7gbtvBGYBAyrYfp6757n7Z0AuIdkcadmvASvc/dnovXsJJ4gyJRnjL9x9r7tvIiTYomNdCtzr7vnuvhu4u4LjbATeIJyMAC4APnD3vOj9P7j7Rg9eBP4ClNlZW8qlwM/d/QN330yovSced667b4/+Jk8STtg5SewXYCzwsLuvcPdPgEnAADPLSihT3ndTkdHAfHd/Mfob3U04cfQBCggnmC5RE+Hb0XcH4eTd0cxau/tH7r40yc8h1UBJXwDeSVwws05m9ryZ7TCzD4EpwAkVbL8j4fUBKu68La9s28Q43N0JNeMyJRljUsci1FAr8iQwJnp9ebRcFMfXzGypmb1vZnsIteyKvqsiX6ooBjO72sxej5pR9gCdktwvhM9XvD93/xD4AGiXUOZI/mbl7beQ8Ddq5+7rgB8Q/g7vRc2FJ0ZFrwGygXVm9qqZXZTk55BqoKQvEC73Ez1EqN2e7u7HA3cSmi+q03ZCcwsAZmaUTFKlHU2M24GTEpYru6V0LjDYzNoRavxPRjEeB8wDfkFoemkB/CnJOHaUF4OZnQo8CNwEtI72+2bCfiu7vXQbocmoaH/NCM1IW5OI60j2W4fwN9sK4O6z3b0foWmnLuF7wd3XuftoQhPefwJPm1mjo4xFqkhJX8rSDNgL7DezzsC3auCYzwG9zOxiM6sHfBdoU00xzgW+Z2btzKw1cFtFhd19B7AEeAxY5+7ro7caAg2AncAhM/sacP4RxDDZzFpYGMcwIeG9poTEvpNw/ruBUNMv8i6QVdRxXYangOvMrLuZNSQk35fcvdwrpyOIebiZDYyO/UNCP8xSM+tsZoOi430c/RQSPsAVZnZCdGWwN/pshUcZi1SRkr6U5QfAVYT/0A8ROlyrlbu/C1wGzAB2A6cB/ySMK0h1jA8S2t5XEToZ5yWxzZOEjtniph133wN8H3iG0Bk6inDySsZPCFccm4AFwOMJ+10JPAC8GpU5A0hsB/9/wHrgXTNLbKYp2v6PhGaWZ6LtTya08x8Vd19N+M4fJJyQhgDDo/b9hsA0Qj/MDsKVxe3RphcBay3cHTYduMzdDx5tPFI1FppORTKLmdUlNCeMcveX0h2PSG2hmr5kDDMbEjV3NAR+TLjr49U0hyVSqySV9KP/jOuiwRyTynj/RjNbZWYrzGyJmWWXev/kaCDJrakKXGqlrwAbCU0HXwVGunt5zTsiUgWVNu9El9n/ItyfnM/hgRZrEsocH90WRjRy8NvuPiTh/XmEzpul7j495Z9CRESSkkxNvzewIRqAchCYw+GBKkDxfcBFmpBwS5mZXQK8TRgBKSIiaZTMhGvtKDmIJJ8wAq8EM7uZMHS+AWGYNmbWlHA73AVAuU07ZjYeGA/QpEmTszp16lReURERKcPy5ct3uXtFtzkDKZxl091nAjPN7HLgDsKtXXcRhrvvC2Ntyt12FmEYPTk5OZ6Xl5eqsEREYsHMKhtZDiSX9LdScuRg8Qi8cswh3McL4YpglJlNA1oAhWb2ibv/utytRUSk2iST9JcRJkvqQEj2ownzjxQzs44JoxSHEQaO4O79E8rcBexTwhcRSZ9Kk767F5jZBGAhYT6NR9x9tZlNAfLcfT4wwcwGE+6r/oDQtCMiIhkm40bkqk1fpGZ99tln5Ofn88knn6Q7FElCo0aNyMrKon79klMvmdlyd690+m09LlEk5vLz82nWrBnt27enohsuJP3cnd27d5Ofn0+HDh0q36AMtWYahtxcaN8e6tQJv3OP6HHfIvH1ySef0Lp1ayX8Y4CZ0bp166O6KqsVNf3cXBg/Hg4cCMubN4dlgLFHPbegSO2nhH/sONq/Va2o6d9+++GEX+TAgbBeREQOqxVJf8uWI1svIplj9+7d9OjRgx49enDiiSfSrl274uWDB5Obdv+aa65h3bp1FZaZOXMmuSlq9/3KV77CihUrUrKvmlYrmndOPjk06ZS1XkRSKzc3XEVv2RL+j02denTNqK1bty5OoHfddRdNmzbl1ltLztri7rg7deqUXU999NFHKz3OzTffXPUga5FaUdOfOhUaNy65rnHjsF5EUqeo/2zzZnA/3H9WHTdObNiwgezsbMaOHUuXLl3Yvn0748ePJycnhy5dujBlypTiskU174KCAlq0aMGkSZM488wzOeecc3jvvfcAuOOOO7jvvvuKy0+aNInevXtzxhln8I9//AOA/fv3841vfIPs7GxGjRpFTk5OpTX62bNn061bN7p27crkyZMBKCgo4Iorrihef//99wNw7733kp2dTffu3Rk3blzKv7Nk1IqaflEtI5W1DxH5vIr6z6rj/9ubb77J448/Tk5OuP387rvvplWrVhQUFDBo0CBGjRpFdnaJx3ewd+9eBgwYwN13383EiRN55JFHmDTpc48Bwd159dVXmT9/PlOmTOGPf/wjDzzwACeeeCJPP/00r7/+Or169aowvvz8fO644w7y8vJo3rw5gwcP5rnnnqNNmzbs2rWLVatWAbBnzx4Apk2bxubNm2nQoEHxuppWK2r6EP7BbdoEhYXhtxK+SOrVdP/ZaaedVpzwAZ566il69epFr169WLt2LWvWrPncNscddxxDhw4F4KyzzmLTpk1l7vvrX//658osWbKE0aNHA3DmmWfSpUuXCuNbunQp5513HieccAL169fn8ssvZ/HixZx++umsW7eOW265hYULF9K8eXMAunTpwrhx48jNzf3c4KqaUmuSvohUv/L6yaqr/6xJkybFr9evX8+vfvUrXnzxRVauXMmQIUPKvF+9QYMGxa/r1q1LQUFBmftu2LBhpWWqqnXr1qxcuZL+/fszc+ZMvvWtbwGwcOFCbrzxRpYtW0bv3r05dOhQSo+bDCV9EUlaOvvPPvzwQ5o1a8bxxx/P9u3bWbhwYcqP0a9fP+bOnQvAqlWryrySSNSnTx8WLVrE7t27KSgoYM6cOQwYMICdO3fi7nzzm99kypQpvPbaaxw6dIj8/HzOO+88pk2bxq5duzhQuq2sBtSKNn0RqRnp7D/r1asX2dnZdOrUiVNOOYV+/fql/Bjf+c53uPLKK8nOzi7+KWqaKUtWVhY/+9nPGDhwIO7OxRdfzLBhw3jttde47rrrcHfMjHvuuYeCggIuv/xyPvroIwoLC7n11ltp1qxZyj9DZTThmkjMrV27ls6dO6c7jIxQUFBAQUEBjRo1Yv369Vx44YWsX7+eevUyq35c1t9ME66JiByhffv2cf7551NQUIC789BDD2Vcwj9atevTiIgchRYtWrB8+fJ0h1Gt1JErIhIjSvoiIjGipC8iEiNJJX0zG2Jm68xsg5l9bjyzmd1oZqvMbIWZLTGz7Gh972jdCjN73cxGpvoDiIhI8ipN+mZWF5gJDAWygTFFST3Bk+7ezd17ANOAGdH6N4CcaP0Q4CEzU+exiBQbNGjQ5wZa3Xfffdx0000Vbte0aVMAtm3bxqhRo8osM3DgQCq7Bfy+++4rMUjqoosuSsm8OHfddRfTp08/6v2kWjI1/d7ABnff6O4HgTnAiMQC7v5hwmITwKP1B9y9aHxzo6L1IiJFxowZw5w5c0qsmzNnDmPGjElq+7Zt2zJv3rwqH7900n/hhRdo0aJFlfeX6ZJJ+u2AdxKW86N1JZjZzWb2FqGmf0vC+j5mthpYBdyYcBIQEWHUqFE8//zzxQ9M2bRpE9u2baN///7F98336tWLbt268eyzz35u+02bNtG1a1cAPv74Y0aPHk3nzp0ZOXIkH3/8cXG5m266qXha5p/85CcA3H///Wzbto1BgwYxaNAgANq3b8+uXbsAmDFjBl27dqVr167F0zJv2rSJzp07c8MNN9ClSxcuvPDCEscpy4oVK+jbty/du3dn5MiRfPDBB8XHL5pquWiit7/97W/FD5Hp2bMnH330UZW/27KkrKnF3WcCM83scuAO4Kpo/VKgi5l1Bn5rZgvcvcQsSWY2HhgPcLKefCKSNt/7HqT6gVA9ekCUL8vUqlUrevfuzYIFCxgxYgRz5szh0ksvxcxo1KgRzzzzDMcffzy7du2ib9++DB8+vNznxD744IM0btyYtWvXsnLlyhJTI0+dOpVWrVpx6NAhzj//fFauXMktt9zCjBkzWLRoESeccEKJfS1fvpxHH32UpUuX4u706dOHAQMG0LJlS9avX89TTz3Fb37zGy699FKefvrpCufHv/LKK3nggQcYMGAAd955Jz/96U+57777uPvuu3n77bdp2LBhcZPS9OnTmTlzJv369WPfvn00atToCL7tyiVT098KnJSwnBWtK88c4JLSK919LbAP6FrGe7PcPcfdc9q0aZNESCJSmyQ28SQ27bg7kydPpnv37gwePJitW7fy7rvvlrufxYsXFyff7t2707179+L35s6dS69evejZsyerV6+udDK1JUuWMHLkSJo0aULTpk35+te/zksvvQRAhw4d6NGjB1Dx9M0Q5vffs2cPAwYMAOCqq65i8eLFxTGOHTuW2bNnF4/87devHxMnTuT+++9nz549KR8RnMzelgEdzawDIdmPBi5PLGBmHd19fbQ4DFgfre8AvOPuBWZ2CtAJ2JSi2EUkxSqqkVenESNG8P3vf5/XXnuNAwcOcNZZZwGQm5vLzp07Wb58OfXr16d9+/ZlTqdcmbfffpvp06ezbNkyWrZsydVXX12l/RQpmpYZwtTMlTXvlOf5559n8eLF/OEPf2Dq1KmsWrWKSZMmMWzYMF544QX69evHwoUL6dSpU5VjLa3Smn7UBj8BWAisBea6+2ozm2Jmw6NiE8xstZmtACYSNe0AXwFej9Y/A3zb3XelLHoRqRWaNm3KoEGDuPbaa0t04O7du5cvfOEL1K9fn0WLFrG5rIdhJzj33HN58sknAXjjjTdYuXIlEKZlbtKkCc2bN+fdd99lwYIFxds0a9aszHbz/v378/vf/54DBw6wf/9+nnnmGfr373/En6158+a0bNmy+CrhiSeeYMCAARQWFvLOO+8waNAg7rnnHvbu3cu+fft466236NatG7fddhtnn302b7755hEfsyJJXTe4+wvAC6XW3Znw+rvlbPcE8MTRBCgi8TBmzBhGjhxZ4k6esWPHcvHFF9OtWzdycnIqrfHedNNNXHPNNXTu3JnOnTsXXzGceeaZ9OzZk06dOnHSSSeVmJZ5/PjxDBkyhLZt27Jo0aLi9b169eLqq6+md+/eAFx//fX07Nmzwqac8vz2t7/lxhtv5MCBA5x66qk8+uijHDp0iHHjxrF3717cnVtuuYUWLVrw4x//mEWLFlGnTh26dOlS/BSwVNHUyiIxp6mVjz1HM7WypmEQEYkRJX0RkRhR0hcRMq2ZV8p3tH8rJX2RmGvUqBG7d+9W4j8GuDu7d+8+qgFbmvxMJOaysrLIz89n586d6Q5FktCoUSOysrKqvL2SvkjM1a9fnw4dOqQ7DKkhat4REYkRJX0RkRhR0hcRiRElfRGRGFHSFxGJESV9EZEYUdIXEYkRJX0RkRhR0hcRiRElfRGRGFHSFxGJESV9EZEYSSrpm9kQM1tnZhvMbFIZ799oZqvMbIWZLTGz7Gj9BWa2PHpvuZmdl+oPICIiyas06ZtZXWAmMBTIBsYUJfUET7p7N3fvAUwDZkTrdwEXu3s34Cr0kHQRkbRKpqbfG9jg7hvd/SAwBxiRWMDdP0xYbAJ4tP6f7r4tWr8aOM7MGh592CIiUhXJzKffDngnYTkf6FO6kJndDEwEGgBlNeN8A3jN3T8tY9vxwHiAk08+OYmQRESkKlLWkevuM939NOA24I7E98ysC3AP8K1ytp3l7jnuntOmTZtUhSQiIqUkk/S3AiclLGdF68ozB7ikaMHMsoBngCvd/a2qBCkiIqmRTNJfBnQ0sw5m1gAYDcxPLGBmHRMWhwHro/UtgOeBSe7+99SELCIiVVVp0nf3AmACsBBYC8x199VmNsXMhkfFJpjZajNbQWjXv6poPXA6cGd0O+cKM/tC6j+GiIgkw9w93TGUkJOT43l5eekOQ0TkmGJmy909p7JyGpErIhIjSvoiIjGipC8iEiNK+iIiMaKkLyISI0r6IiIxoqQvIhIjSvoiIjGipC8iEiNK+iIiMaKkn0K5udC+PdSpE37n5qY7IhGRkpJ5iIokITcXxo+HAwfC8ubNYRlg7Nj0xSUikkg1/RS5/fbDCb/IgQNhvYhIplDST5EtW45svYhIOijpp0h5j/bVI39FJJMo6afI1KnQuHHJdY0bh/UiIplCST9Fxo6FWbPglFPALPyeNUuduCKSWXT3TgqNHaskLyKZLamavpkNMbN1ZrbBzCaV8f6NZrYqegbuEjPLjta3NrNFZrbPzH6d6uBFROTIVJr0zawuMBMYCmQDY4qSeoIn3b2bu/cApgEzovWfAD8Gbk1dyCIiUlXJ1PR7AxvcfaO7HwTmACMSC7j7hwmLTQCP1u939yWE5C81RCODRaQ8ybTptwPeSVjOB/qULmRmNwMTgQbAeSmJTo6YRgaLSEVSdveOu89099OA24A7jmRbMxtvZnlmlrdz585UhRRLGhksIhVJJulvBU5KWM6K1pVnDnDJkQTh7rPcPcfdc9q0aXMkm0opGhksIhVJJukvAzqaWQczawCMBuYnFjCzjgmLw4D1qQtRjoRGBotIRSpN+u5eAEwAFgJrgbnuvtrMppjZ8KjYBDNbbWYrCO36VxVtb2abCHfzXG1m+WXc+SMppJHBIlIRc/d0x1BCTk6O5+XlpTuMY1pubmjD37Il1PCnTlUnrkhtZ2bL3T2nsnIakVsLaWSwiJRHc++IiMSIkr6ISIwo6YuIxIiSvohIjCjpi4jEiJK+iEiMKOmLiMSIkr6ISIwo6Uu10bz+IplHI3KlWmhef5HMpJq+VAvN6y+SmZT0pVpoXn+RzKSkL9VC8/qLZCYlfakWmtdfJDMp6Uu1GDsWZs2CU04Bs/B71ix14oqkm+7ekWqjef1FMo9q+iIiMaKkLyISI0klfTMbYmbrzGyDmU0q4/0bzWyVma0wsyWJDz83sx9F260zs6+mMngRETkylSZ9M6sLzASGAtnAmMSkHnnS3bu5ew9gGjAj2jYbGA10AYYA/xXtT0RE0iCZmn5vYIO7b3T3g8AcYERiAXf/MGGxCeDR6xHAHHf/1N3fBjZE+xMRkTRI5u6ddsA7Ccv5QJ/ShczsZmAi0AA4L2HbV0pt266MbccD4wFO1ugdEZFqk7KOXHef6e6nAbcBdxzhtrPcPcfdc9q0aZOqkEREpJRkkv5W4KSE5axoXXnmAJdUcVsREalGyST9ZUBHM+tgZg0IHbPzEwuYWceExWHA+uj1fGC0mTU0sw5AR+DVow9bRESqotI2fXcvMLMJwEKgLvCIu682sylAnrvPByaY2WDgM+AD4Kpo29VmNhdYAxQAN7v7oWr6LCIiUglz98pL1aCcnBzPy8tLdxgiIscUM1vu7jmVldOIXBGRGFHSl1ovU57VmylxSLxplk2p1TLlWb2ZEoeI2vSlVmvfPiTY0k45BTZtil8cUnupTV+EzHlWb6bEIaKkL7VapjyrN1PiAPUtxJ2SvtRqmfKs3kyJo6hvYfNmcD/ct6DEHx9K+lKrZcqzejMljttvP9yZXOTAgbBe4kEduSIxUqdOqOGXZgaFhTUfj6SOOnJF5HMyqW9B0kNJXyRGMqVvQdJHSV8kRjKlb0HSRyNyRWJm7Fgl+ThTTV9EJEaU9EVEYkRJX0RqnEYFp4/a9EWkRmnG0fRSTV9EapRGBaeXkr6I1CjNOJpeSSV9MxtiZuvMbIOZTSrj/YlmtsbMVprZX8zslIT37jGzN6Kfy1IZvIgcezQqOL0qTfpmVheYCQwFsoExZpZdqtg/gRx37w7MA6ZF2w4DegE9gD7ArWZ2fOrCF5FjjUYFp1cyNf3ewAZ33+juB4E5wIjEAu6+yN2LWuleAbKi19nAYncvcPf9wEpgSGpCF5FjkUYFp1cySb8d8E7Ccn60rjzXAQui168DQ8yssZmdAAwCTiq9gZmNN7M8M8vbuXNncpGLyDFr7NjwmMjCwvBbCb/mpPSWTTMbB+QAAwDc/U9mdjbwD2An8DJwqPR27j4LmAVhauVUxiQiIoclU9PfSsnaeVa0rgQzGwzcDgx390+L1rv7VHfv4e4XAAb86+hCFhGRqkom6S8DOppZBzNrAIwG5icWMLOewEOEhP9ewvq6ZtY6et0d6A78KVXBi4jIkak06bt7ATABWAisBea6+2ozm2Jmw6NivwSaAr8zsxVmVnRSqA+8ZGZrCM0346L9iYikXRyng9DjEkUklkpPBwHh1tFj9U4iPS5RRKQCcZ0OQklfRGIprtNBKOmLSCzFdToIJX0RiaW4TgehpC8isRTX6SD0EBURia04PiReNX0RkTSryfECqumLiKRRTT8+UjV9EZE0qunxAkr6IiJpVNPjBZT0RUTSqKbHCyjpi4ikUU2PF1DSFxFJo5oeL6C7d0RE0qwmxwuopi8iEiNK+iIiMaKkLyISI0r6IiIxklTSN7MhZrbOzDaY2aQy3p9oZmvMbKWZ/cXMTkl4b5qZrTaztWZ2v5lZKj+AiIgkr9Kkb2Z1gZnAUCAbGGNm2aWK/RPIcffuwDxgWrTtl4F+QHegK3A2MCBl0YuIyBFJpqbfG9jg7hvd/SAwBxiRWMDdF7l70ewRrwBZRW8BjYAGQEOgPvBuKgIXEZEjl0zSbwe8k7CcH60rz3XAAgB3fxlYBGyPfha6+9rSG5jZeDPLM7O8nTt3Jhv757z3XpU3FRGJhZR25JrZOCAH+GW0fDrQmVDzbwecZ2b9S2/n7rPcPcfdc9q0aVOlY2/eDB07ws03w/79Vf4IIiK1WjJJfytwUsJyVrSuBDMbDNwODHf3T6PVI4FX3H2fu+8jXAGcc3Qhl+0LX4DrroMHH4Qzz4S//706jiIicmxLJukvAzqaWQczawCMBuYnFjCznsBDhISf2MiyBRhgZvXMrD6hE/dzzTupcNxxMGMGLFoEhw5B//5w223wySfVcTQRkWNTpUnf3QuACcBCQsKe6+6rzWyKmQ2Piv0SaAr8zsxWmFnRSWEe8BawCngdeN3d/5DqD5FowABYuRJuuAGmTYOcHHjtteo8oojIscPcPd0xlJCTk+N5eXkp2deCBaHJZ+dO+PGP4Uc/gvr1U7JrEZGMYmbL3T2nsnK1ekTu0KHwxhtw2WXwk5/Al78Ma9akOyoRkfSp1UkfoFUrmD0bfvc7ePtt6NUL/vM/Q7u/iEjc1PqkX2TUKFi9Gr76Vbj1Vhg0CDZuTHdUIiI1KzZJH+CLX4Tf/x4eewxefx26d4eHHoIM69YQEak2sUr6EB5HdtVVoa3/nHPgxhtD2//Wz408EBGpfWKX9IucdBIsXAgzZ8JLL0HXrqHtX7V+EanNYpv0AerUgW9/G1asgOxsuOKK0PZ/FNP/iIhktFgn/SIdO8LixWEw13PPQZcuoe1fJJX+8he48EL4xS9095ikj5J+pG5d+OEPYflyyMqCkSPhyithz550RybHurw8uOACGDwYXn0VJk+GgQNh06Z0RyZxpKRfSteu8MorcOed8OSTYflPf0p3VHIsWrcOvvlNOPts+Oc/4d57YccOeOKJMFVI9+7w+OPqR5KapaRfhgYN4Kc/hZdfhmbNwr393/427NuX7sjkWJCfD+PHh2bCBQtCBWLjRvje96BRIxg3Ltwy3KNHuJPsssvg/ffTHbXEhZJ+Bc4+O0zWNnEi/Pd/hymblyxJd1SSqd5/H/7930Mf0WOPhWc7bNwYKhDHH1+ybPv2YUbYX/wCnnkGunWDP/85HVFL3CjpV+K448K0DX/9a7gMP/fc0PavKZulyP79IXmfeipMnx6adNatg1/9KjznoTx168KkSaE58fhSvGRtAAALAUlEQVTjQ7v/xIn6tyXVS0k/SeeeG9phx48P/7HPOit0+kp8ffZZeGjP6aeHztlzzw3NNo8/Dh06JL+fon9LN98c2v3PPjv8WxOpDkr6R6Bp09DMs2BBuKunTx+4667wn1/io7AQ5syBzp1DX89pp4UBfvPnh2aaqmjcGH79a3j++TBO5Oyzw0OBCgtTG7uIkn4VDBkSpnEYPTq01/btGyZzk9rNHf74x1AzHzMmJOrnngsJ/ytfSc0xLroIVq0KU4P84AehySc/PzX7FgEl/Spr2TJM2zBvHmzZEhLB9OkadFNbvfJKmJl16NBwlffEE+E2zGHDwnxOqdSmTejc/c1vwnG7dYO5c1N7DIkvJf2j9I1vhFr+0KGhg3fgQHjrrXRHJamyZk0YqHfOObB2LTzwQOikHTcudMRWFzO4/vowRci//Vu4rfPKK2Hv3uo7psRDrX5cYk1yDzX/73wHDhwIE7qdcEKotSX+Lmtdixapry3K0dmyJTxt7fHHoUmTcEL//vdDv05N++wzmDoVfvaz8O/qiSegf/+aj0MyW7KPS0wq6ZvZEOBXQF3gYXe/u9T7E4HrgQJgJ3Ctu282s0HAvQlFOwGj3b3cmW2O1aRf5J13Qm1w61bYtSt0yhX9Lu9WvHr1oHXrz58MKjphNGxYs58rLnbtgv/4jzD7KsCECeHZyieckN64IAwWHDcuPAFu0qRwE0GDBumOSjJFypK+mdUF/gVcAOQDy4Ax7r4mocwgYKm7HzCzm4CB7n5Zqf20AjYAWe5+oLzjHetJvzzu4Qqg6CRQ+oRQ1u/33y9/iH6zZhWfINq0gbZtw1B/JYbK7dsX7paZPj3cd3/VVSGpnnxyuiMr6aOPwhXH//xPePRnbi506pTuqCQTJJv06yWxr97ABnffGO14DjACKE767r4oofwrwLgy9jMKWFBRwq/NzEIzQZMmYTRmMg4dCom/shPFjh3hjo+yriYaNgzJoW/f0C7dt2+YUE7NScGnn8KsWaHpZOfO0H7/85+HqbYzUbNm8PDDoQP5hhvC33b6dLjpJv1NJTnJJP12wDsJy/lAnwrKXwcsKGP9aGBGWRuY2XhgPMDJmVa1SqO6dUONvU2b5LfZv//wyWDTJli6NDQLPPhgGPgD4Qqg6ATQt2+48+i446rlI2SsQ4fChHp33hm+p4EDw6javn3THVlyRo4MsV5zTRjU9dxz8MgjcOKJ6Y5MMl0yzTujgCHufn20fAXQx90nlFF2HDABGODunyas/xKwEmjr7hUOZaqtzTvpdvBgGOX58svhNsBXXjn8YPh69cLkX0UngXPOCSNKa2PN0T0MgJo8OVwd9ewZkv2FFx6bn9c99D/88Iehk/nhh2HEiHRHJemQbPNOMrdsbgVOSljOitaVPuBg4HZgeGLCj1wKPFNZwpfq06AB5OSEu4tyc8NtpTt2wLPPHk4Yjz4aOgpPOy08RH748JAQFy2qHTOMLlkS7nq5+GL4+OMwqjYvL8yieiwmfAhxT5hw+DkQl1wSmn1qw99LqkcyNf16hI7c8wnJfhlwubuvTijTE5hHuCJYX8Y+XgF+VKrtv0yq6adPQUEYc5B4NbBuXXivTp0wSCjxaqBjx7A+3QoLYfduePfd8n+2bg1XOl/6UrgV89proX79dEeeWgcPhuaqadPCiXv27DBViMRDqm/ZvAi4j3DL5iPuPtXMpgB57j7fzP4MdAO2R5tscffh0bbtgb8DJ7l7pTOJKOlnlt27w9Oeik4CS5ceHiDUsmVIKkX9A717hzEHqVBQEPomKkrkRT87d5Y9Erp+/TDL5Re/GH4GDgy14saNUxNjpvrb38JArq1bw0lg8uTQhCe1W0qTfk1S0s9shYXw5pslrwZWrz58a2nnziU7ibOzD49c/ewzeO+95BL5rl1l367aqNHhJF7ZT5wHve3ZE05wubnh7zB7dqj9S+2lpC81Zu9eWLbs8EnglVfCFQKEWwzbtg3J/oMPyt6+SZPkE3mzZvFN5FXx1FPhds6CgjC//7XX6vurrZT0JW3cYcOGkPxffjkk/IoSeZMm6Y64dtuyJQw2++tfQ0fvb36TGSOMJbWU9EWkWGFhGHE8eXKY8uPhh0Ozz8GDodkt8SfZdakoC2FsQdu20K7d4d/t2oUKQW3rbK9OqRyRKyLHuDp14NZbw/z8Y8fC175WfceqWzck6/r1w63CRa9L/zRoEE5Ga9fC9u2hCSqRWUj8iSeC0ieHtm2hVSs1WR0JJX2RGDnzzND/8uSTYS6oshJxRUm6svX16lXtNt7CwnAX1tatsG1b+J34evNm+Mc/DvcVJWrUKCT/sq4WEtfFbdR5edS8IyLHjE8+CVcFiSeE0r+3bg2D70pr2bLsq4V27UITU+vW4aqhefPMGH9ypNS8IyK1TqNGYYqQih487x7uKCvvqmHbtvC40x07yn4GsVk4QbRqVfKn6KRQ3k+LFsfGeIhjIEQRkeSZhQTcogV06VJ+uUOHDo/W3rEj3FL8/vslf3bvDmNG/vWvsLxnT8XHbtGi4hNDWSeOli1rtsNaSV9EYqlu3cN9Ack6dCgk/rJODqXXvf9+eODN+++HE0pZVxVFmjULJ4BzzgljK6qTkr6ISJLq1g219datj2y7wsLQ5FTWiSHxJyureuJOpKQvIlLN6tQJzTgtW6Z/OoxjsI9aRESqSklfRCRGlPRFRGJESV9EJEaU9EVEYkRJX0QkRpT0RURiRElfRCRGMm6WTTPbCWxOdxxH6QRgV7qDyCD6PkrS93GYvouSjub7OMXd21RWKOOSfm1gZnnJTHEaF/o+StL3cZi+i5Jq4vtQ846ISIwo6YuIxIiSfvWYle4AMoy+j5L0fRym76Kkav8+1KYvIhIjqumLiMSIkr6ISIwo6aeQmZ1kZovMbI2ZrTaz76Y7pnQzs7pm9k8zey7dsaSbmbUws3lm9qaZrTWzc9IdUzqZ2fej/ydvmNlTZtYo3THVJDN7xMzeM7M3Eta1MrP/Z2bro98tU31cJf3UKgB+4O7ZQF/gZjPLTnNM6fZdYG26g8gQvwL+6O6dgDOJ8fdiZu2AW4Acd+8K1AVGpzeqGvcYMKTUuknAX9y9I/CXaDmllPRTyN23u/tr0euPCP+p26U3qvQxsyxgGPBwumNJNzNrDpwL/A+Aux909z3pjSrt6gHHmVk9oDGwLc3x1Ch3Xwy8X2r1COC30evfApek+rhK+tXEzNoDPYGl6Y0kre4D/h0oTHcgGaADsBN4NGruetjMmqQ7qHRx963AdGALsB3Y6+5/Sm9UGeGL7r49er0D+GKqD6CkXw3MrCnwNPA9d/8w3fGkg5l9DXjP3ZenO5YMUQ/oBTzo7j2B/VTDpfuxImqrHkE4GbYFmpjZuPRGlVk83E+f8nvqlfRTzMzqExJ+rrv/X7rjSaN+wHAz2wTMAc4zs9npDSmt8oF8dy+68ptHOAnE1WDgbXff6e6fAf8HfDnNMWWCd83sSwDR7/dSfQAl/RQyMyO02a519xnpjied3P1H7p7l7u0JHXQvuntsa3LuvgN4x8zOiFadD6xJY0jptgXoa2aNo/835xPjju0E84GrotdXAc+m+gBK+qnVD7iCUKtdEf1clO6gJGN8B8g1s5VAD+A/0hxP2kRXPPOA14BVhFwUqykZzOwp4GXgDDPLN7PrgLuBC8xsPeFq6O6UH1fTMIiIxIdq+iIiMaKkLyISI0r6IiIxoqQvIhIjSvoiIjGipC8iEiNK+iIiMfL/AUj04YDc/AlqAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss) + 1)\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "층을 추가하여 성능을 조금 향상시켰지만 크지는 않습니다. 여기서 두 가지 결론을 얻을 수 있습니다:\n",
    "\n",
    "- 아직 충분히 과대적합을 만들지 못했기 때문에 검증 손실을 향상하기 위해서 층의 크기를 늘릴 수 있습니다. 하지만 적지 않은 계산 비용이 추가됩니다.\n",
    "\n",
    "- 층을 추가한 만큼 도움이 되지 않았으므로 여기서는 네트워크의 용량을 늘리는 것이 도움이 되지 않는다고 볼 수 있습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.3.8 양방향 RNN(Bidirectional RNN) 사용하기\n",
    "\n",
    "이 절에서 소개할 마지막 기법은 **양방향 RNN**(Bidirectional RNN)입니다. 양방향 RNN은 RNN의 한 변종이고 특정 작업에서 기본 RNN 보다 훨씬 좋은 성능을 냅니다. 자연어 처리에서는 맥가이버 칼이라고 할 정도로 즐겨 사용됩니다.\n",
    "\n",
    "**RNN은 특히 순서 또는 시간에 민감**합니다. 즉, 입력 시퀀스의 타입스텝 순서대로 처리합니다. 타입스텝을 섞거나 거꾸로 하면 RNN이 시퀀스에서 학습하는 표현을 완전히 바꾸어 버립니다. 이는 온도 예측과 같이 순서에 의미가 있는 문제에 잘 맞는 이유이기도 합니다. \n",
    "\n",
    "양방향 RNN은 RNN이 순서에 민감하다는 성질을 사용합니다. 앞서 보았던 GRU나 LSTM 같은 RNN 두 개를 사용합니다. 각 RNN은 입력 시퀀스를 한 방향(시간의 순서나 반대 순서)으로 처리한 다음 각 표현을 합칩니다. 시퀀스를 양쪽 방향으로 처리하기 때문에 양방향 RNN은 단방향 RNN이 놓치기 쉬운 패턴을 감지할 수 있습니다.\n",
    "\n",
    "\n",
    "![](./images/rnn-vs-brnn.png)\n",
    "\n",
    "\n",
    "놀랍게도 이 절에 있는 RNN 층이 시간의 순서대로 (오래된 타임스텝이 먼저 나오도록) 시퀀스를 처리하는 것은 근거 없는 결정입니다. 적어도 이 결정을 궁금해하지 않았습니다. 시간의 반대 방향으로 (최근 타임스텝이 먼저 나오도록) 입력 시퀀스를 처리하면 만족할만한 RNN 성능을 낼 수 있을까요? 실제 이렇게 해 보고 결과가 어떤지 확인해 보죠. 해야 할 일은 입력 시퀀스를 시간 차원을 따라 거꾸로 생성하는 데이터 제너레이터 만드는 것 뿐입니다(제너레이터 함수의 마지막 줄을 yield samples[:, ::-1, :], targets로 바꿉니다). 이 절의 첫 번째 예제와 동일하게 하나의 GRU 층을 가진 네트워크를 훈련합니다:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 순서를 뒤집어서 학습시키기 - 온도"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "def reverse_order_generator(data, lookback, delay, min_index, max_index,\n",
    "                            shuffle=False, batch_size=128, step=6):\n",
    "    if max_index is None:\n",
    "        max_index = len(data) - delay - 1\n",
    "    i = min_index + lookback\n",
    "    while 1:\n",
    "        if shuffle:\n",
    "            rows = np.random.randint(\n",
    "                min_index + lookback, max_index, size=batch_size)\n",
    "        else:\n",
    "            if i + batch_size >= max_index:\n",
    "                i = min_index + lookback\n",
    "            rows = np.arange(i, min(i + batch_size, max_index))\n",
    "            i += len(rows)\n",
    "\n",
    "        samples = np.zeros((len(rows),\n",
    "                           lookback // step,\n",
    "                           data.shape[-1]))\n",
    "        targets = np.zeros((len(rows),))\n",
    "        for j, row in enumerate(rows):\n",
    "            indices = range(rows[j] - lookback, rows[j], step)\n",
    "            samples[j] = data[indices]\n",
    "            targets[j] = data[rows[j] + delay][1]\n",
    "        yield samples[:, ::-1, :], targets\n",
    "        \n",
    "train_gen_reverse = reverse_order_generator(\n",
    "    float_data,\n",
    "    lookback=lookback,\n",
    "    delay=delay,\n",
    "    min_index=0,\n",
    "    max_index=200000,\n",
    "    shuffle=True,\n",
    "    step=step, \n",
    "    batch_size=batch_size)\n",
    "val_gen_reverse = reverse_order_generator(\n",
    "    float_data,\n",
    "    lookback=lookback,\n",
    "    delay=delay,\n",
    "    min_index=200001,\n",
    "    max_index=300000,\n",
    "    step=step,\n",
    "    batch_size=batch_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "500/500 [==============================] - 78s 156ms/step - loss: 0.4822 - val_loss: 0.4829\n",
      "Epoch 2/20\n",
      "500/500 [==============================] - 77s 153ms/step - loss: 0.4418 - val_loss: 0.4620\n",
      "Epoch 3/20\n",
      "500/500 [==============================] - 77s 153ms/step - loss: 0.4005 - val_loss: 0.4752\n",
      "Epoch 4/20\n",
      "500/500 [==============================] - 77s 153ms/step - loss: 0.3627 - val_loss: 0.4679\n",
      "Epoch 5/20\n",
      "500/500 [==============================] - 76s 152ms/step - loss: 0.3316 - val_loss: 0.4753\n",
      "Epoch 6/20\n",
      "500/500 [==============================] - 76s 152ms/step - loss: 0.3149 - val_loss: 0.4861\n",
      "Epoch 7/20\n",
      "500/500 [==============================] - 77s 154ms/step - loss: 0.3008 - val_loss: 0.4571\n",
      "Epoch 8/20\n",
      "500/500 [==============================] - 77s 154ms/step - loss: 0.2879 - val_loss: 0.4453\n",
      "Epoch 9/20\n",
      "500/500 [==============================] - 77s 154ms/step - loss: 0.2765 - val_loss: 0.4394\n",
      "Epoch 10/20\n",
      "500/500 [==============================] - 78s 155ms/step - loss: 0.2716 - val_loss: 0.4456\n",
      "Epoch 11/20\n",
      "500/500 [==============================] - 78s 156ms/step - loss: 0.2630 - val_loss: 0.4211\n",
      "Epoch 12/20\n",
      "500/500 [==============================] - 77s 155ms/step - loss: 0.2564 - val_loss: 0.4339\n",
      "Epoch 13/20\n",
      "500/500 [==============================] - 77s 154ms/step - loss: 0.2539 - val_loss: 0.4335\n",
      "Epoch 14/20\n",
      "500/500 [==============================] - 78s 155ms/step - loss: 0.2490 - val_loss: 0.4357\n",
      "Epoch 15/20\n",
      "500/500 [==============================] - 78s 155ms/step - loss: 0.2448 - val_loss: 0.4274\n",
      "Epoch 16/20\n",
      "500/500 [==============================] - 78s 155ms/step - loss: 0.2418 - val_loss: 0.4347\n",
      "Epoch 17/20\n",
      "500/500 [==============================] - 76s 153ms/step - loss: 0.2377 - val_loss: 0.4344\n",
      "Epoch 18/20\n",
      "500/500 [==============================] - 75s 151ms/step - loss: 0.2362 - val_loss: 0.4472\n",
      "Epoch 19/20\n",
      "500/500 [==============================] - 76s 151ms/step - loss: 0.2327 - val_loss: 0.4412\n",
      "Epoch 20/20\n",
      "500/500 [==============================] - 76s 151ms/step - loss: 0.2281 - val_loss: 0.4298\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras import layers\n",
    "from keras.optimizers import RMSprop\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.GRU(32, input_shape=(None, float_data.shape[-1])))\n",
    "model.add(layers.Dense(1))\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae')\n",
    "history = model.fit_generator(train_gen_reverse,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=20,\n",
    "                              validation_data=val_gen_reverse,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VNX5x/HPA7LIoqxWZQtaF1YBI2qRAkot4EKp1IKx7kX8aa3VtlJxRW0RLUUsPyv2p7YSQatVcSu2FbW2FQmIICqySDSICCiIgkCS5/fHuYEhZJkwk5kk832/XvPK3Dvn3vvMnclzz5x77rnm7oiISGaol+4AREQkdZT0RUQyiJK+iEgGUdIXEckgSvoiIhlESV9EJIMo6UuVmFl9M/vSzDoms2w6mdk3zSzpfZfNbLCZrY6ZXmZm/eMpuw/b+qOZXbevy1ew3tvM7KFkr1fSZ790ByDVy8y+jJlsAmwHiqLpS909tyrrc/cioFmyy2YCdz8qGesxs0uAc919YMy6L0nGuqXuU9Kv49x9V9KNapKXuPs/yitvZvu5e2EqYhOR1FPzToaLfr4/amYzzWwLcK6ZnWhmr5vZJjNba2ZTzaxBVH4/M3Mzy4qmZ0Svv2BmW8zsv2bWuaplo9eHmtn7ZrbZzO4xs3+b2QXlxB1PjJea2Qoz+9zMpsYsW9/MfmdmG81sFTCkgv0z3sxmlZo3zcwmR88vMbN3o/ezMqqFl7euAjMbGD1vYmYPR7EtBY4tVfZ6M1sVrXepmZ0Zze8B/B7oHzWdbYjZtzfHLD82eu8bzewpMzsknn1TGTMbEcWzycxeMrOjYl67zsw+NrMvzOy9mPd6gpktjOavM7M7492eVAN31yNDHsBqYHCpebcBO4AzCJWA/YHjgOMJvwQPA94HrojK7wc4kBVNzwA2ANlAA+BRYMY+lD0I2AIMj167GtgJXFDOe4knxqeBA4Es4LOS9w5cASwF2gOtgVfDv0KZ2zkM+BJoGrPuT4HsaPqMqIwBJwPbgJ7Ra4OB1THrKgAGRs/vAl4GWgKdgHdKlT0bOCT6TM6JYvhG9NolwMul4pwB3Bw9PzWKsRfQGPhf4KV49k0Z7/824KHoeZcojpOjz+g6YFn0vBuQDxwcle0MHBY9nw+Mjp43B45P9/9CJj9U0xeA19z9GXcvdvdt7j7f3ee5e6G7rwKmAwMqWP5xd89z951ALiHZVLXs6cAid386eu13hANEmeKM8TfuvtndVxMSbMm2zgZ+5+4F7r4RmFjBdlYBbxMORgDfAT5397zo9WfcfZUHLwH/BMo8WVvK2cBt7v65u+cTau+x233M3ddGn8kjhAN2dhzrBcgB/ujui9z9a2AcMMDM2seUKW/fVGQUMNvdX4o+o4mEA8fxQCHhANMtaiL8INp3EA7eR5hZa3ff4u7z4nwfUg2U9AXgo9gJMzvazJ4zs0/M7AtgAtCmguU/iXm+lYpP3pZX9tDYONzdCTXjMsUZY1zbItRQK/IIMDp6fk40XRLH6WY2z8w+M7NNhFp2RfuqxCEVxWBmF5jZW1Ezyibg6DjXC+H97Vqfu38BfA60iylTlc+svPUWEz6jdu6+DLiG8Dl8GjUXHhwVvRDoCiwzszfMbFic70OqgZK+QPi5H+s+Qu32m+5+AHAjofmiOq0lNLcAYGbGnkmqtERiXAt0iJmurEvpY8BgM2tHqPE/EsW4P/A48BtC00sL4MU44/ikvBjM7DDgXuAyoHW03vdi1ltZ99KPCU1GJetrTmhGWhNHXFVZbz3CZ7YGwN1nuHs/QtNOfcJ+wd2XufsoQhPeb4EnzKxxgrHIPlLSl7I0BzYDX5lZF+DSFGzzWaCPmZ1hZvsBPwXaVlOMjwFXmVk7M2sNXFtRYXf/BHgNeAhY5u7Lo5caAQ2B9UCRmZ0OnFKFGK4zsxYWrmO4Iua1ZoTEvp5w/PsxoaZfYh3QvuTEdRlmAhebWU8za0RIvv9y93J/OVUh5jPNbGC07V8QzsPMM7MuZjYo2t626FFMeAM/MrM20S+DzdF7K04wFtlHSvpSlmuA8wn/0PcRTrhWK3dfB/wQmAxsBA4H3iRcV5DsGO8ltL0vIZxkfDyOZR4hnJjd1bTj7puAnwFPEk6GjiQcvOJxE+EXx2rgBeDPMetdDNwDvBGVOQqIbQf/O7AcWGdmsc00Jcv/jdDM8mS0fEdCO39C3H0pYZ/fSzggDQHOjNr3GwGTCOdhPiH8shgfLToMeNdC77C7gB+6+45E45F9Y6HpVKRmMbP6hOaEke7+r3THI1JXqKYvNYaZDYmaOxoBNxB6fbyR5rBE6hQlfalJTgJWEZoOvguMcPfymndEZB+oeUdEJIOopi8ikkFq3IBrbdq08aysrHSHISJSqyxYsGCDu1fUzRmogUk/KyuLvLy8dIchIlKrmFllV5YDat4REckoSvoiIhlESV9EJIPUuDZ9EUmtnTt3UlBQwNdff53uUCQOjRs3pn379jRoUN7QSxVT0hfJcAUFBTRv3pysrCzC4KZSU7k7GzdupKCggM6dO1e+QBnUvCOS4b7++mtat26thF8LmBmtW7dO6FeZkr6IKOHXIol+Vkr6AsBXX8H06bBqVeVlRaT2qjNJPzcXsrKgXr3wNzc33RHVDtu3w9SpcPjhcOmlcPrpsHVruqOSTLJx40Z69epFr169OPjgg2nXrt2u6R074ht2/8ILL2TZsmUVlpk2bRq5SUoMJ510EosWLUrKulIu3XdmL/049thjvapmzHBv0sQddj+aNAnzpWw7drjff797hw5hfw0c6D5liruZ+6WXpjs6SaV33nmnSuVnzHDv1Cl8Vzp1Su7/2U033eR33nnnXvOLi4u9qKgoeRtKUL9+/fzNN99M2/bL+syAPI8jx9aJmv748XvXTrduDfNlT8XF8Mgj0LUr/PjHcOih8I9/wEsvwU9/Cr/4Bdx3Hzz5ZLojlZooNxfGjIH8/FC9ys8P09Xxy3rFihV07dqVnJwcunXrxtq1axkzZgzZ2dl069aNCRMm7CpbUvMuLCykRYsWjBs3jmOOOYYTTzyRTz/9FIDrr7+eKVOm7Co/btw4+vbty1FHHcV//vMfAL766ivOOussunbtysiRI8nOzq60Rj9jxgx69OhB9+7due666wAoLCzkRz/60a75U6dOBeB3v/sdXbt2pWfPnpx77rlJ32fxqBNdNj/8sGrzM5E7PP003HADvP029OwJs2eH5pzY80K33hoOAJdcAscdB+3bl79OyTwVVbByEr4h497ee+89/vznP5OdnQ3AxIkTadWqFYWFhQwaNIiRI0fStWvXPZbZvHkzAwYMYOLEiVx99dU88MADjBs3bq91uztvvPEGs2fPZsKECfztb3/jnnvu4eCDD+aJJ57grbfeok+fPhXGV1BQwPXXX09eXh4HHngggwcP5tlnn6Vt27Zs2LCBJUuWALBp0yYAJk2aRH5+Pg0bNtw1L9XqRE2/Y8ey53fokNo4aiJ3mDMH+vaFESNgxw6YNQvefBPOOGPPhA/QsGH4JbB9O/zoR1BUlJ64pWZKdQXr8MMP35XwAWbOnEmfPn3o06cP7777Lu+8885ey+y///4MHToUgGOPPZbVq1eXue7vf//7e5V57bXXGDVqFADHHHMM3bp1qzC+efPmcfLJJ9OmTRsaNGjAOeecw6uvvso3v/lNli1bxpVXXsmcOXM48MADAejWrRvnnnsuubm5+3xxVaLqRNK//XZo0mTv+X36hKSXqV57DQYOhCFDYP16eOABWLoUfvjDcMK7PEccAb//Pbz8MtxxR6qildqgvApWefMT1bRp013Ply9fzt13381LL73E4sWLGTJkSJn91Rs2bLjref369SksLCxz3Y0aNaq0zL5q3bo1ixcvpn///kybNo1LL70UgDlz5jB27Fjmz59P3759KUpDrapOJP2cnNDdsFOnUHPt2BEGDICnnoKbbsq8xL9gAQwdCv37w/vvhwS+bBlceCHsF2eD3vnnw6hRcOONMG9e9cYrtUdZFawmTcL86vbFF1/QvHlzDjjgANauXcucOXOSvo1+/frx2GOPAbBkyZIyf0nEOv7445k7dy4bN26ksLCQWbNmMWDAANavX4+784Mf/IAJEyawcOFCioqKKCgo4OSTT2bSpEls2LCBrWnoKlcn2vQhJP7YNsXi4nCC6dZbw/Qtt+zdlJFshYVQv371b6c8b78dDnJ//Su0agWTJsHll5f9K6gyZnDvvfDf/8Lo0bBoERxwQPJjltql5H9s/PjQpNOxY0j41dGeX1qfPn3o2rUrRx99NJ06daJfv35J38ZPfvITzjvvPLp27brrUdI0U5b27dtz6623MnDgQNydM844g9NOO42FCxdy8cUX4+6YGXfccQeFhYWcc845bNmyheLiYn7+85/TvHnzpL+HSsXTxSeVj33pslmeoiL3iy8OXRJvuMG9uDhpq97D9u3ut9/u3rix+2GHuV93nfvbb1fPtkr7+GP3yZPds7PD+2ze3P3mm903b07O+v/9b/d69dzPPTc565Oap6pdNuuynTt3+rZt29zd/f333/esrCzfuXNnmqPaWyJdNutMTb8s9eqFZh+ovhr/f/4TLmp6+20YPhy2bYOJE+HXv4YePUItedQo2Mexkcq0eXOozT/ySOhpU1wMxx4Lv/1taJZp3Tp52/rWt8Kvh5tugu9+F9LUy0wkJb788ktOOeUUCgsLcXfuu+8+9ou3TbS2iOfIkMpHMmv6Jaqjxv/55+5jx4YLVDp0cJ89e/drn3zifs897t/61u6LxU44wX3q1PDavvj6a/e//tV95Ej3Ro3COg8/3P3GG93ffTfx91ORwkL3/v3Dr4gVK6p3W5J6qunXPonU9NOe5Es/qiPpuycv8RcXu//lL+6HHBKaPa66yn3LlvLLf/CB+8SJ7j17hm3Xq+c+eLD7//1fOHBUFvNLL4W4DzwwLH/QQe5XXun++uvV11xVlvx89xYt3Pv2DVfzSt2hpF/7KOnHKdHEn5/vfvrpYfnevd3z8qq2/NKl7tdfH2ro4N6wofv3vuf+6KPuX30VyhQXuy9c6H7NNe7t2oVyzZq5n3ee+9/+5p7O5sW//CXEc9116YtBkk9Jv/ZR0q+CfUn8O3eGk6VNm4YxfX7728SSb3Gx+xtvuP/sZ+6HHro7sY8c6d6lS5hu0MD9zDPdZ83afUCoCS65JDRpvfRSuiORZFHSr310IrcKqnpyd+HCMEbNwoUwbBhMmxZG8UyEWRji4Ljj4M474V//Cidln3kGjjwyjIEzcmRyT8gmy5QpId4f/Qjeeqtmxigi5asTF2dVVUniv/jikPjLuoDryy/hmmtCYl6zBh59FJ59NvGEX1r9+uGq2enTYe1aeOWV0BuopibTpk1h5kz49NMwPk/p/SZSVYMGDdrrQqspU6Zw2WWXVbhcs2bNAPj4448ZOXJkmWUGDhxIXl5eheuZMmXKHhdJDRs2LCnj4tx8883cddddCa8n2TIy6UPFif+556BbN5g8OdTy33sPzj47fRdd1TS9e4duqU89FUbkFEnE6NGjmTVr1h7zZs2axejRo+Na/tBDD+Xxxx/f5+2XTvrPP/88LVq02Of11XQZm/Rh78T/y1+G5H766dCsWWjG+MMfoA5//vvsqqtCv/2f/QwquVJdpEIjR47kueee23XDlNWrV/Pxxx/Tv3//Xf3m+/TpQ48ePXj66af3Wn716tV0794dgG3btjFq1Ci6dOnCiBEj2LZt265yl1122a5hmW+66SYApk6dyscff8ygQYMYNGgQAFlZWWzYsAGAyZMn0717d7p3775rWObVq1fTpUsXfvzjH9OtWzdOPfXUPbZTlkWLFnHCCSfQs2dPRowYweeff75r+yVDLZcM9PbKK6/suolM79692bJlyz7v27JkXJt+abFt/HfdBY0a7T4AxIzbJKXUqwcPPQTHHBMuPnvjDWjcON1RSaKuuioMuZFMvXqFc0HladWqFX379uWFF15g+PDhzJo1i7PPPhszo3Hjxjz55JMccMABbNiwgRNOOIEzzzyz3PvE3nvvvTRp0oR3332XxYsX7zE08u23306rVq0oKirilFNOYfHixVx55ZVMnjyZuXPn0qZNmz3WtWDBAh588EHmzZuHu3P88cczYMAAWrZsyfLly5k5cyb3338/Z599Nk888USF4+Ofd9553HPPPQwYMIAbb7yRW265hSlTpjBx4kQ++OADGjVqtKtJ6a677mLatGn069ePL7/8ksZJ/sfK6Jp+iZLE/+CDsGQJXH+9En48Dj549z679tp0RyO1WWwTT2zTjrtz3XXX0bNnTwYPHsyaNWtYt25duet59dVXdyXfnj170rNnz12vPfbYY/Tp04fevXuzdOnSSgdTe+211xgxYgRNmzalWbNmfP/73+df//oXAJ07d6ZXr15AxcM3Qxjff9OmTQwYMACA888/n1dffXVXjDk5OcyYMWPXlb/9+vXj6quvZurUqWzatCnpVwRnfE2/RL16cMEF6Y6i9hk2LPQ2uvtuOPVUOO20dEckiaioRl6dhg8fzs9+9jMWLlzI1q1bOfbYYwHIzc1l/fr1LFiwgAYNGpCVlVXmcMqV+eCDD7jrrruYP38+LVu25IILLtin9ZQoGZYZwtDMlTXvlOe5557j1Vdf5ZlnnuH2229nyZIljBs3jtNOO43nn3+efv36MWfOHI4++uh9jrU01fQlYXfcEZp5LrgAPv443dFIbdSsWTMGDRrERRddtMcJ3M2bN3PQQQfRoEED5s6dS35+foXr+fa3v80jjzwCwNtvv83ixYuBMCxz06ZNOfDAA1m3bh0vvPDCrmWaN29eZrt5//79eeqpp9i6dStfffUVTz75JP3796/yezvwwANp2bLlrl8JDz/8MAMGDKC4uJiPPvqIQYMGcccdd7B582a+/PJLVq5cSY8ePbj22ms57rjjeO+996q8zYqopi8Ja9QodOM89tgwyNzNN8PYsZCmGwNJLTV69GhGjBixR0+enJwczjjjDHr06EF2dnalNd7LLruMCy+8kC5dutClS5ddvxiOOeYYevfuzdFHH02HDh32GJZ5zJgxDBkyhEMPPZS5c+fumt+nTx8uuOAC+vbtC8All1xC7969K2zKKc+f/vQnxo4dy9atWznssMN48MEHKSoq4txzz2Xz5s24O1deeSUtWrTghhtuYO7cudSrV49u3brtugtYspjXsI7W2dnZXlm/WqmZliwJvXn++U84+uhwYnzYMHV1reneffddunTpku4wpArK+szMbIG7Z5ezyC5q3pGk6dED/v73cMP14uLQ9fXUU8PBQERqhriSvpkNMbNlZrbCzPa+rfzucmeZmZtZdjSdZWbbzGxR9PhDsgKXmsks3HD97bfDyd0FC0KXvUsvhQo6XYhIilSa9M2sPjANGAp0BUabWdcyyjUHfgqUvqPqSnfvFT3GJiFmqQUaNIArr4QVK8LfBx4IN1yfOBES6DQh1aSmNfNK+RL9rOKp6fcFVrj7KnffAcwChpdR7lbgDkD/0rJLq1bwu9/B0qUwaBD86lehvf/RRzVuT03RuHFjNm7cqMRfC7g7GzduTOiCrXh677QDPoqZLgCOjy1gZn2ADu7+nJn9otTync3sTeAL4Hp3/1fpDZjZGGAMQMeOHasQvtQWRx4JTz8dbu949dXhKt6pU8MBIeocIWnSvn17CgoKWL9+fbpDkTg0btyY9u3b7/PyCXfZNLN6wGTggjJeXgt0dPeNZnYs8JSZdXP3L2ILuft0YDqE3juJxiQ118knh3b+hx6C8ePh+OPhnHPgN78BHe/To0GDBnRO5k2ca6Gvvgr3mF6/HoYMCb9KmzRJd1TVI57mnTVAh5jp9tG8Es2B7sDLZrYaOAGYbWbZ7r7d3TcCuPsCYCVwZDICl9qrfv0wyN3y5SHx//WvcNRRcMMNsHFjemLauBEeewxWrUrP9iV9nn02jKp7003h3NPpp4ehzYcNg9//vg5+Jyq7ywrh18AqoDPQEHgL6FZB+ZeB7Oh5W6B+9PwwwsGiVUXbq+47Z5Vnxgz3Tp3CXaE6dQrTkhr5+e7nnOO77iF8wgnuN9/sPm9euNNZdSgudl+yJNy/+KSTwnbBvWNH97Vrq2ebUrPk54fblYJ7167ur7zi/vXX7i++GO59feSR4TVwP/po96uvdv/HP9y3b0935GUjmbdLBIYB7xNq6uOjeROAM8soG5v0zwKWAouAhcAZlW0rHUl/xoxwG8SSDxjCtBJ/ai1a5H7TTe7HHx8OvuDepo17To77ww+7f/ppYuvfts39hRfcL788HNhLPuvevcOtM2fNCp/7ccfVrFtUSnLt2OF+553h9qf77x8O/OUl8uXL3e++2/3UU8M9rUtubTpihPv997uvWZPa2CuS1KSfykc6kn5sAoh9dOqU8lAksn69e26u+7nnurdtGz4Ps5CQb7zR/T//cS8srHw9a9a4T58e7jdccmDff/8wPX26e0HBnuWfeips56yzqu9XhqTPv//t3qNH+B6cfrr7Bx/Ev+yXX7rPnu1+6aXuHTrszhO9erlfd11YdzzfyeqipF8FJbXK0g+zlIciZSgqcp8/333CBPcTT9zdFNOqlfuoUe5/+pP7J5/sLvvGG+HA0KfP7s+yY8dQw3/+efetWyve3uTJYZlrr63+9yapsWGD+yWXhM+1fXv3J58MTXz7KrZ58Nvfdq9fP6y7c2f3mTPTU2FQ0q8C1fRrl40bwz/W+ee7f+Mbe9a4Sqbr1XPv18/9N78J/5xV+QcvLnYfOzas549/rLa3ISlQXOz+0EOhmbB+ffdrrnHfsiX52/nsM/dHHnHv2TN8b4491v2f/0z+diqipF8FatOvvYqK3BcscL/tNvdBg9x/+MPwuW3YkNh6d+4M7bj77Zf6f15JjqVL3QcMCP/PJ57o/tZb1b/NwsLwy7Ok+WfoUPfFi6t/u+5K+lWm3jtS2qZN7t26ubdo4f7uu+mORuL11Vfuv/pVOGC3bBnO3aS6uWXbNvdJk8J3x8z9ggvcP/ywercZb9LX0MoiFcjPD1cMN2sGr78ObdumO6J95w6bN8OaNeFmN2vW7Pn844/h8MPDNRQnnxzuJpdKhYXhIqn99guP+vXDoypDcz/3HFxxBaxeDeefD5MmwUEHVVvIlfrsM/j1r+Gee8L+/OlPYdw4aNEi+duKd2hlJX2RSsybBwMHQp8+4V4B1XED+OLicHHYypXh/swNGux+VGV68+ayE3rJ861b9952y5bQrl245/GCBfD553DYYSH5X3ghHHJI8t9viaIiePnlcBOeJ56A6N7geyhJ/rEHg7Keu4f9d/TRcO+94TOrKfLzw8WHM2aE/T1+PFx+ebgBUbIo6Ysk0eOPww9+AKNHQ25ucm8Ms3hxuNPYf/+bvHVCSCiHHhoSesnfkkfs/P33373M11+HK6Tvvz8k4/r1w32Pf/zjMDxBMu7R7R5+Nc2cGQ5069aFX1IjRoRhuIuKQq2/5G/JI3a6rOc7d4ZfZVdeGQ6ENdGiRXDttfDii5CVBbfdFr5TyfhVpaQvkmQTJ4ZRQm+8EW65JfH1bdkSbi15992h9nfXXWEgusJC2LEjJLGdO+N7XjLdvPnuxN6qVWIHp+XL4Y9/DOMkffppWOdFF4VfAJ06VW1d7uHgNnMmzJoVar6NGoUhD0aNCgeW2INPXff3v8MvfxkOAr17h2aowYMTW2e8ST/tJ25LP9J1IlekMsXF7hddFLo/PPxwYut5/HH3du3CusaMCd1Qa6odO9yfeMJ9yJBwUtLM/bvfdf/LXyofkmDZMvdbbgnDGEDoNjl0aOjhsnlzauKvqYqKdncggdBb7M039319qPeOSPJt3x66hjZs6P7qq1VffuVK92HDwn9ez57hyuLaJD8/DJXRvn14Dwcd5P6LX4TkHltm0qTdF8eZha6Tf/hDuNJa9vT11+6//W3oadS9+75fNBZv0lfzjkgVff45nHhiGIZ33jz45jcrX2b79tB8c9ttoV18wgT4yU+S00aeDkVFMGdOaPt/5pkwPWBA+Pvaa6FM376h6ebss0PTkFRs0yYoKIDu3fdtebXpi1SjlSvDvQBatw4nYFu1Kr/s3LnwP/8D770HZ50FU6ZAAvfAqHE++SS0+z/0UDiB+sMfhmR/+OHpjiyzKOmLVLPXXoNTTgm1/hdf3LvHyLp18POfh256nTvDtGkwdGh6YpW6L96kn+LLL0TqjpNOggcfhFdegTFjdt/zt6go9BMvuRfw9deHewQr4UtNUEtbFEVqhnPOCV0bb7453Ad46NDQ5/6NN8It9/73f0PyF6kplPRFEnTjjbBiRbjK8oYboE2b0KRzzjnJvYhLJBmU9EUSZBYuYtq5M4zNM2FCuNhKpCZS0hdJgkaNwpWmIjWdTuSKiGQQJX0RkQyipC8ikkGU9EVEMoiSvohIBlHSFxHJIEr6IiIZRElfRCSDKOmLiGQQJX0RkQyipC8ikkHiSvpmNsTMlpnZCjMbV0G5s8zMzSw7Zt6vouWWmdl3kxF0TZSbC1lZUK9e+Jubm+6IRET2VumAa2ZWH5gGfAcoAOab2Wx3f6dUuebAT4F5MfO6AqOAbsChwD/M7Eh3L0reW0i/3NxwE42tW8N0fn6YBsjJSV9cIiKlxVPT7wuscPdV7r4DmAUML6PcrcAdwNcx84YDs9x9u7t/AKyI1lenjB+/O+GX2Lo1zBcRqUniSfrtgI9ipguiebuYWR+gg7s/V9Vlo+XHmFmemeWtX78+rsBrkg8/rNp8EZF0SfhErpnVAyYD1+zrOtx9urtnu3t227ZtEw0p5Tp2rNp8EZF0iSfprwE6xEy3j+aVaA50B142s9XACcDs6GRuZcvWCbffDk2a7DmvSZMwX0SkJokn6c8HjjCzzmbWkHBidnbJi+6+2d3buHuWu2cBrwNnunteVG6UmTUys87AEcAbSX8XaZaTA9OnQ6dO4dZ5nTqFaZ3EFZGaptLeO+5eaGZXAHOA+sAD7r7UzCYAee4+u4Jll5rZY8A7QCFweV3ruVMiJ0dJXkRqPnP3dMewh+xLgqqAAAAN3UlEQVTsbM/Ly0t3GCIitYqZLXD37MrK6YpcEZEMoqQvIpJBlPRFRDKIkr6ISAZR0hcRySBK+iIiGURJX0Qkgyjpi4hkECV9EZEMoqQvIpJBlPRFRDKIkr6ISAZR0hcRySBK+iIiGURJX0Qkgyjpi4hkECV9EZEMoqQvIpJBlPRFRDKIkr6ISAZR0q8hcnMhKwvq1Qt/c3PTHZGI1EX7pTsACQl+zBjYujVM5+eHaYCcnPTFJSJ1j2r6NcD48bsTfomtW8N8EZFkUtKvAT78sGrzRUT2lZJ+DdCxY9Xmi4jsKyX9GuD226FJkz3nNWkS5ouIJJOSfg2QkwPTp0OnTmAW/k6frpO4IpJ86r1TQ+TkKMmLSPVTTV9EJIPElfTNbIiZLTOzFWY2rozXx5rZEjNbZGavmVnXaH6WmW2L5i8ysz8k+w2IiEj8Km3eMbP6wDTgO0ABMN/MZrv7OzHFHnH3P0TlzwQmA0Oi11a6e6/khi0iIvsinpp+X2CFu69y9x3ALGB4bAF3/yJmsingyQtRRESSJZ6k3w74KGa6IJq3BzO73MxWApOAK2Ne6mxmb5rZK2bWP6FoRUQkIUk7kevu09z9cOBa4Ppo9lqgo7v3Bq4GHjGzA0ova2ZjzCzPzPLWr1+frJBERKSUeJL+GqBDzHT7aF55ZgHfA3D37e6+MXq+AFgJHFl6AXef7u7Z7p7dtm3beGMXEZEqiifpzweOMLPOZtYQGAXMji1gZkfETJ4GLI/mt41OBGNmhwFHAKuSEbiIiFRdpb133L3QzK4A5gD1gQfcfamZTQDy3H02cIWZDQZ2Ap8D50eLfxuYYGY7gWJgrLt/Vh1vREREKmfuNaujTXZ2tufl5aU7DBGRWsXMFrh7dmXldEWuiEgGUdIXEckgSvoiIhlESV9EJIMo6YuIZBAlfRGRDKKkX0fk5kJWFtSrF/7m5qY7IhGpiXTnrDogNxfGjIGtW8N0fn6YBt2NS0T2pJp+HTB+/O6EX2Lr1jBfRCSWkn4d8OGHVZsvIplLSb8O6NixavNFJHMp6dcBt98OTZrsOa9JkzBfRCSWkn4dkJMD06dDp05gFv5On66TuCKyN/XeqSNycpTkRaRyqumLiGQQJX0RkQyipC8ikkGU9EVEMoiSvgAau0ckU6j3jmjsHpEMopq+aOwekQyipC8au0ckgyjpi8buEckgSvqisXtEMoiSvmjsHpEMot47AmjsHpFMoZq+iEgGUdIXEckgSvoiIhlESV+SQsM4iNQOcSV9MxtiZsvMbIWZjSvj9bFmtsTMFpnZa2bWNea1X0XLLTOz7yYzeKkZSoZxyM8H993DOCjxi9Q85u4VFzCrD7wPfAcoAOYDo939nZgyB7j7F9HzM4H/cfchUfKfCfQFDgX+ARzp7kXlbS87O9vz8vISe1eSUllZIdGX1qkTrF6d6mhEMpOZLXD37MrKxVPT7wuscPdV7r4DmAUMjy1QkvAjTYGSI8lwYJa7b3f3D4AV0fqkDtEwDiK1RzxJvx3wUcx0QTRvD2Z2uZmtBCYBV1Zx2TFmlmdmeevXr483dqkhNIyDSO2RtBO57j7N3Q8HrgWur+Ky0909292z27Ztm6yQJEU0jINI7RFP0l8DdIiZbh/NK88s4Hv7uKzUQhrGQaT2iCfpzweOMLPOZtYQGAXMji1gZkfETJ4GLI+ezwZGmVkjM+sMHAG8kXjYUtPk5ISTtsXF4a8SvkjNVOnYO+5eaGZXAHOA+sAD7r7UzCYAee4+G7jCzAYDO4HPgfOjZZea2WPAO0AhcHlFPXdERKR6VdplM9XUZVNEpOqS2WVTpNrpil6R1NDQypJ2ujG7SOqopi9ppxuzi6SOkr6kna7oFUkdJX1JO13RK5I6SvqSdrqiVyR1lPQl7XRFr0jqqPeO1Ai6MbtIaqimLyKSQZT0pU7QxV0i8VHzjtR6urhLJH6q6Uutp4u7ROKnpC+1ni7uEomfkr7Uesm4uEvnBCRTKOlLrZfoxV0l5wTy88F99zkBJX6pi5T0pdZL9OIunROQTKKbqEjGq1cv1PBLMwu3fxSpDXQTFZE4acA3ySRK+pLxNOCbZBIlfcl4yRjwTb1/pLbQFbkiJDbgm64IltpENX2RBKn3j9QmSvoiCdIVwVKbKOmLJEi9f6Q2UdIXSVAyev/oRLCkipK+SIIS7f2jYSAklZT0RZIgJwdWrw5X8K5eXbVeO8k4EaxfChIvddkUSbNETwSry6hUhWr6ImmW6IlgdRmVqogr6ZvZEDNbZmYrzGxcGa9fbWbvmNliM/unmXWKea3IzBZFj9nJDF6kLkj0RLC6jEpVVJr0zaw+MA0YCnQFRptZ11LF3gSy3b0n8DgwKea1be7eK3qcmaS4ReqMRE8E6yYyUhXx1PT7AivcfZW77wBmAcNjC7j7XHcv+YH5OtA+uWGK1G2JnAjWTWSkKuJJ+u2Aj2KmC6J55bkYeCFmurGZ5ZnZ62b2vbIWMLMxUZm89evXxxGSiJTQTWSkKpJ6ItfMzgWygTtjZneKBvY/B5hiZoeXXs7dp7t7trtnt23bNpkhiWSERH4pJOOcgJqHao94kv4aoEPMdPto3h7MbDAwHjjT3beXzHf3NdHfVcDLQO8E4hWRJEv0nICah2qXeJL+fOAIM+tsZg2BUcAevXDMrDdwHyHhfxozv6WZNYqetwH6Ae8kK3gRSVyi5wR0cVntUunFWe5eaGZXAHOA+sAD7r7UzCYAee4+m9Cc0wz4i5kBfBj11OkC3GdmxYQDzER3V9IXqUFKmoLGjw9NOh07hoQfbxORLi6rXXRjdBFJSFZWSNSldeoUzi9U9/IS6MboIpISurisdlHSF5GE6OKy2kVJX0QSVtsvLsukg4aSvoikVbovLsu0LqdK+iKSdum8uCzTupwq6YtIrZboOYFkdTmtLb8UlPRFpFZL9JxApt3PQElfRGq1RM8JZFqXUyV9Ean1EjknkGldTpX0RSTj1fYup1WhpC8ikoB0dzmtKo29IyKSRvXqhRp+aWbhl0e8NPaOiEgtkIxzAlWhpC8ikkaJnhOoKiV9EZE0SvScQFVVehMVERGpXjk5qbthjGr6IiIZRElfRCSDKOmLiGQQJX0RkQyipC8ikkFq3BW5ZrYeyE93HBVoA2xIdxAVUHyJUXyJUXyJSSS+Tu7etrJCNS7p13RmlhfPpc7povgSo/gSo/gSk4r41LwjIpJBlPRFRDKIkn7VTU93AJVQfIlRfIlRfImp9vjUpi8ikkFU0xcRySBK+iIiGURJvxQz62Bmc83sHTNbamY/LaPMQDPbbGaLoseNaYhztZktiba/163GLJhqZivMbLGZ9UlhbEfF7JtFZvaFmV1VqkxK96GZPWBmn5rZ2zHzWpnZ381sefS3ZTnLnh+VWW5m56cwvjvN7L3o83vSzFqUs2yF34VqjO9mM1sT8xkOK2fZIWa2LPoujkthfI/GxLbazBaVs2wq9l+ZeSUt30F31yPmARwC9ImeNwfeB7qWKjMQeDbNca4G2lTw+jDgBcCAE4B5aYqzPvAJ4cKRtO1D4NtAH+DtmHmTgHHR83HAHWUs1wpYFf1tGT1vmaL4TgX2i57fUVZ88XwXqjG+m4Gfx/H5rwQOAxoCb5X+f6qu+Eq9/lvgxjTuvzLzSjq+g6rpl+Lua919YfR8C/Au0C69Ue2T4cCfPXgdaGFmh6QhjlOAle6e1qus3f1V4LNSs4cDf4qe/wn4XhmLfhf4u7t/5u6fA38HhqQiPnd/0d0Lo8nXgfbJ3m68ytl/8egLrHD3Ve6+A5hF2O9JVVF8ZmbA2cDMZG83XhXklZR/B5X0K2BmWUBvYF4ZL59oZm+Z2Qtm1i2lgQUOvGhmC8xsTBmvtwM+ipkuID0Hr1GU/8+W7n34DXdfGz3/BPhGGWVqyn68iPDLrSyVfReq0xVR89MD5TRN1IT91x9Y5+7Ly3k9pfuvVF5J+XdQSb8cZtYMeAK4yt2/KPXyQkJzxTHAPcBTqY4POMnd+wBDgcvN7NtpiKFCZtYQOBP4Sxkv14R9uIuH39E1sv+ymY0HCoHccoqk67twL3A40AtYS2hCqYlGU3EtP2X7r6K8kqrvoJJ+GcysAeGDyXX3v5Z+3d2/cPcvo+fPAw3MrE0qY3T3NdHfT4EnCT+jY60BOsRMt4/mpdJQYKG7ryv9Qk3Yh8C6kiav6O+nZZRJ6340swuA04GcKCnsJY7vQrVw93XuXuTuxcD95Ww33ftvP+D7wKPllUnV/isnr6T8O6ikX0rU/vd/wLvuPrmcMgdH5TCzvoT9uDGFMTY1s+Ylzwkn/N4uVWw2cF7Ui+cEYHPMz8hUKbeGle59GJkNlPSEOB94uowyc4BTzaxl1HxxajSv2pnZEOCXwJnuvrWcMvF8F6orvthzRCPK2e584Agz6xz98htF2O+pMhh4z90LynoxVfuvgryS+u9gdZ6xro0P4CTCT6zFwKLoMQwYC4yNylwBLCX0RHgd+FaKYzws2vZbURzjo/mxMRowjdBzYgmQneIYmxKS+IEx89K2DwkHn7XATkKb6MVAa+CfwHLgH0CrqGw28MeYZS8CVkSPC1MY3wpCW27J9/APUdlDgecr+i6kKL6Ho+/WYkLyOqR0fNH0MEJvlZWpjC+a/1DJdy6mbDr2X3l5JeXfQQ3DICKSQdS8IyKSQZT0RUQyiJK+iEgGUdIXEckgSvoiIhlESV9EJIMo6YuIZJD/B/fVnoJrhNksAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss) + 1)\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "순서를 뒤집은 GRU는 상식 수준의 기준점보다도 성능이 낮습니다. 이 경우에는 시간 순서대로 처리하는 것이 중요한 역할을 합니다. 사실 이는 당연합니다. \n",
    "\n",
    "기본적인 GRU 층은 먼 과거보다 최근 내용을 잘 기억합니다. 또한, 최근에 가까운 날씨 데이터 포인트일수록 오래된 데이터 포인트보다 예측에 유용합니다(상식 수준의 기준점이 꽤 강력한 이유입니다). 시간 순서대로 처리하는 네트워크가 거꾸로 처리하는 것보다 성능이 높아야만 합니다. \n",
    "\n",
    "하지만 자연어 처리를 포함하여 다른 많은 문제에서는 그렇지 않습니다. 문장을 이해하는데 있어서 단어의 중요성은 단어가 문장 어디에 놓여 있는지에 따라 결정되지 않습니다. 같은 기법을 이전 절의 LSTM IMDB 예제에 적용해 보죠:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 순서를 뒤집어서 학습시키기 - Imdb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 20000 samples, validate on 5000 samples\n",
      "Epoch 1/10\n",
      "20000/20000 [==============================] - 51s 3ms/step - loss: 0.4867 - acc: 0.7703 - val_loss: 0.3891 - val_acc: 0.8488\n",
      "Epoch 2/10\n",
      "20000/20000 [==============================] - 51s 3ms/step - loss: 0.3152 - acc: 0.8792 - val_loss: 0.3758 - val_acc: 0.8378\n",
      "Epoch 3/10\n",
      "20000/20000 [==============================] - 51s 3ms/step - loss: 0.2924 - acc: 0.8990 - val_loss: 0.3251 - val_acc: 0.8734\n",
      "Epoch 4/10\n",
      "20000/20000 [==============================] - 51s 3ms/step - loss: 0.2212 - acc: 0.9200 - val_loss: 0.3678 - val_acc: 0.8810\n",
      "Epoch 5/10\n",
      "20000/20000 [==============================] - 51s 3ms/step - loss: 0.1993 - acc: 0.9298 - val_loss: 0.3741 - val_acc: 0.8770\n",
      "Epoch 6/10\n",
      "20000/20000 [==============================] - 51s 3ms/step - loss: 0.1777 - acc: 0.9371 - val_loss: 0.3487 - val_acc: 0.8786\n",
      "Epoch 7/10\n",
      "20000/20000 [==============================] - 50s 3ms/step - loss: 0.1523 - acc: 0.9482 - val_loss: 0.3690 - val_acc: 0.8692\n",
      "Epoch 8/10\n",
      "20000/20000 [==============================] - 51s 3ms/step - loss: 0.1395 - acc: 0.9535 - val_loss: 0.4202 - val_acc: 0.8572\n",
      "Epoch 9/10\n",
      "20000/20000 [==============================] - 51s 3ms/step - loss: 0.1301 - acc: 0.9560 - val_loss: 0.3788 - val_acc: 0.8434\n",
      "Epoch 10/10\n",
      "20000/20000 [==============================] - 51s 3ms/step - loss: 0.1250 - acc: 0.9598 - val_loss: 0.3953 - val_acc: 0.8492\n"
     ]
    }
   ],
   "source": [
    "from keras.datasets import imdb\n",
    "from keras.preprocessing import sequence\n",
    "from keras import layers\n",
    "from keras.models import Sequential\n",
    "\n",
    "# 특성으로 사용할 단어의 수\n",
    "max_features = 10000\n",
    "# 사용할 텍스트의 길이(가장 빈번한 max_features 개의 단어만 사용합니다)\n",
    "maxlen = 500\n",
    "\n",
    "# 데이터 로드\n",
    "(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)\n",
    "\n",
    "# 시퀀스를 뒤집습니다\n",
    "x_train = [x[::-1] for x in x_train]\n",
    "x_test = [x[::-1] for x in x_test]\n",
    "\n",
    "# 시퀀스에 패딩을 추가합니다\n",
    "x_train = sequence.pad_sequences(x_train, maxlen=maxlen)\n",
    "x_test = sequence.pad_sequences(x_test, maxlen=maxlen)\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.Embedding(max_features, 128))\n",
    "model.add(layers.LSTM(32))\n",
    "model.add(layers.Dense(1, activation='sigmoid'))\n",
    "\n",
    "model.compile(optimizer='rmsprop',\n",
    "              loss='binary_crossentropy',\n",
    "              metrics=['acc'])\n",
    "history = model.fit(x_train, y_train,\n",
    "                    epochs=10,\n",
    "                    batch_size=128,\n",
    "                    validation_split=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "시간 순서로 훈련한 LSTM과 거의 동일한 성능을 얻을 수 있습니다. 놀랍게도 이런 **텍스트 데이터셋에는 순서를 뒤집어 처리하는 것이 시간 순서대로 처리하는 것과 거의 동일하게 잘 작동**합니다. \n",
    "\n",
    "- **이는 언어를 이해하는 데 단어의 순서가 중요하지만 결정적이지는 않다는 가정을 뒷받침합니다.**\n",
    "\n",
    "거꾸로 된 시퀀스에서 훈련한 RNN은 원래 시퀀스에서 훈련한 것과는 다른 표현을 학습합니다. 이와 비슷하게 시작할 때 죽고 마지막 날 태어나는 삶처럼 실제 세상의 시간이 거꾸로 흘러간다면 우리의 정신 세계가 달라질 것입니다. \n",
    "\n",
    "머신 러닝에서 다른 표현이 유용하다면 항상 사용할 가치가 있습니다. 이 표현이 많이 다를수록 더 좋습니다. 이 표현이 데이터를 바라보는 새로운 시각을 제공하고 다른 방식에서는 놓칠 수 있는 데이터의 특징을 잡아냅니다. 이런 표현은 작업의 성능을 올리는 데 도움을 줍니다. 이것이 다음 장에서 살펴볼 앙상블의 개념입니다.\n",
    "\n",
    "양방향 RNN은 이 아이디어를 사용하여 시간 순서대로 처리하는 RNN의 성능을 향상시킵니다. **입력 시퀀스를 양쪽 방향으로 바라보기 때문에, 드러나지 않은 다양한 표현을 얻어 시간 순서대로 처리할 때 놓칠 수 있는 패턴을 잡아 냅니다.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 케라스의 `Bidirectional` 레이어를 이용해 학습시키기 - Imdb\n",
    "\n",
    "케라스에서는 [Bidirectional](https://keras.io/layers/wrappers/#bidirectional) 층을 사용하여 양방향 RNN을 만듭니다. \n",
    "\n",
    "- 이 클래스는 첫 번째 매개변수로 순환 층의 객체를 전달받습니다. \n",
    "\n",
    "- `Bidirectional` 클래스는 전달받은 순환 층으로 새로운 두 번째 객체를 만듭니다. \n",
    "\n",
    "- 하나는 시간 순서대로 입력 시퀀스를 처리하고 다른 하나는 반대 순서로 입력 시퀀스를 처리합니다. \n",
    "\n",
    "IMDB 감성 분석 문제에 이를 적용해 보죠:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_1 (Embedding)      (None, None, 32)          320000    \n",
      "_________________________________________________________________\n",
      "bidirectional_1 (Bidirection (None, 64)                16640     \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 1)                 65        \n",
      "=================================================================\n",
      "Total params: 336,705\n",
      "Trainable params: 336,705\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Train on 20000 samples, validate on 5000 samples\n",
      "Epoch 1/10\n",
      "20000/20000 [==============================] - 101s 5ms/step - loss: 0.5609 - acc: 0.7162 - val_loss: 0.4003 - val_acc: 0.8404\n",
      "Epoch 2/10\n",
      "20000/20000 [==============================] - 100s 5ms/step - loss: 0.3285 - acc: 0.8740 - val_loss: 0.3192 - val_acc: 0.8856\n",
      "Epoch 3/10\n",
      "20000/20000 [==============================] - 100s 5ms/step - loss: 0.2642 - acc: 0.9026 - val_loss: 0.5006 - val_acc: 0.8592\n",
      "Epoch 4/10\n",
      "20000/20000 [==============================] - 99s 5ms/step - loss: 0.2253 - acc: 0.9198 - val_loss: 0.3010 - val_acc: 0.8844\n",
      "Epoch 5/10\n",
      "20000/20000 [==============================] - 99s 5ms/step - loss: 0.2072 - acc: 0.9268 - val_loss: 0.3541 - val_acc: 0.8834\n",
      "Epoch 6/10\n",
      "20000/20000 [==============================] - 99s 5ms/step - loss: 0.1836 - acc: 0.9370 - val_loss: 0.3862 - val_acc: 0.8754\n",
      "Epoch 7/10\n",
      "20000/20000 [==============================] - 98s 5ms/step - loss: 0.1712 - acc: 0.9403 - val_loss: 0.3818 - val_acc: 0.8816\n",
      "Epoch 8/10\n",
      "20000/20000 [==============================] - 98s 5ms/step - loss: 0.1566 - acc: 0.9476 - val_loss: 0.5140 - val_acc: 0.8104\n",
      "Epoch 9/10\n",
      "20000/20000 [==============================] - 98s 5ms/step - loss: 0.1435 - acc: 0.9498 - val_loss: 0.4125 - val_acc: 0.8410\n",
      "Epoch 10/10\n",
      "20000/20000 [==============================] - 98s 5ms/step - loss: 0.1312 - acc: 0.9561 - val_loss: 0.3673 - val_acc: 0.8728\n"
     ]
    }
   ],
   "source": [
    "from keras import backend as K\n",
    "K.clear_session()\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.Embedding(max_features, 32))\n",
    "model.add(layers.Bidirectional(layers.LSTM(32)))\n",
    "model.add(layers.Dense(1, activation='sigmoid'))\n",
    "model.summary()\n",
    "\n",
    "model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['acc'])\n",
    "history = model.fit(x_train, y_train, epochs=10, batch_size=128, validation_split=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "여기서 얻은 검증 정확도는 88% 정도로 이전 절에서 사용했던 일반 LSTM보다 조금 더 성능이 높습니다. 이 모델은 조금 더 일찍 과대적합되는 것 같습니다. 양방향 순환 층이 단방향 LSTM보다 모델 파라미터가 두 배 많기 때문에 놀라운 일은 아닙니다. 규제를 조금 추가한다면 양항뱡 순환 층을 사용하는 것이 이 작업에 더 적합해 보입니다.\n",
    "\n",
    "이제 동일한 방식을 온도 예측 문제에 적용해 보죠:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 케라스의 `Bidirectional` 레이어를 이용해 학습시키기 - 온도"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "bidirectional_3 (Bidirection (None, 64)                9024      \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              (None, 1)                 65        \n",
      "=================================================================\n",
      "Total params: 9,089\n",
      "Trainable params: 9,089\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Epoch 1/20\n",
      "500/500 [==============================] - 145s 290ms/step - loss: 0.2911 - val_loss: 0.2698\n",
      "Epoch 2/20\n",
      "500/500 [==============================] - 144s 288ms/step - loss: 0.2734 - val_loss: 0.2688\n",
      "Epoch 3/20\n",
      "500/500 [==============================] - 144s 288ms/step - loss: 0.2663 - val_loss: 0.2656\n",
      "Epoch 4/20\n",
      "500/500 [==============================] - 144s 289ms/step - loss: 0.2604 - val_loss: 0.2713\n",
      "Epoch 5/20\n",
      "500/500 [==============================] - 144s 288ms/step - loss: 0.2514 - val_loss: 0.2656\n",
      "Epoch 6/20\n",
      "500/500 [==============================] - 144s 288ms/step - loss: 0.2462 - val_loss: 0.2766\n",
      "Epoch 7/20\n",
      "500/500 [==============================] - 144s 288ms/step - loss: 0.2415 - val_loss: 0.2797\n",
      "Epoch 8/20\n",
      "500/500 [==============================] - 144s 288ms/step - loss: 0.2361 - val_loss: 0.2858\n",
      "Epoch 9/20\n",
      "500/500 [==============================] - 144s 288ms/step - loss: 0.2308 - val_loss: 0.2864\n",
      "Epoch 10/20\n",
      "500/500 [==============================] - 144s 288ms/step - loss: 0.2244 - val_loss: 0.2865\n",
      "Epoch 11/20\n",
      "500/500 [==============================] - 144s 288ms/step - loss: 0.2201 - val_loss: 0.2934\n",
      "Epoch 12/20\n",
      "500/500 [==============================] - 144s 288ms/step - loss: 0.2124 - val_loss: 0.2968\n",
      "Epoch 13/20\n",
      "500/500 [==============================] - 144s 288ms/step - loss: 0.2086 - val_loss: 0.2944\n",
      "Epoch 14/20\n",
      "500/500 [==============================] - 144s 289ms/step - loss: 0.2038 - val_loss: 0.3047\n",
      "Epoch 15/20\n",
      "500/500 [==============================] - 144s 288ms/step - loss: 0.2000 - val_loss: 0.3062\n",
      "Epoch 16/20\n",
      "500/500 [==============================] - 144s 288ms/step - loss: 0.1955 - val_loss: 0.3028\n",
      "Epoch 17/20\n",
      "500/500 [==============================] - 144s 287ms/step - loss: 0.1917 - val_loss: 0.3101\n",
      "Epoch 18/20\n",
      "500/500 [==============================] - 144s 288ms/step - loss: 0.1890 - val_loss: 0.3084\n",
      "Epoch 19/20\n",
      "500/500 [==============================] - 144s 288ms/step - loss: 0.1849 - val_loss: 0.3202\n",
      "Epoch 20/20\n",
      "500/500 [==============================] - 144s 288ms/step - loss: 0.1805 - val_loss: 0.3112\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras import layers\n",
    "from keras.optimizers import RMSprop\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.Bidirectional(\n",
    "    layers.GRU(32), input_shape=(None, float_data.shape[-1])))\n",
    "model.add(layers.Dense(1))\n",
    "model.summary()\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae')\n",
    "history = model.fit_generator(train_gen,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=20,\n",
    "                              validation_data=val_gen,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XmYFNXZ9/HvDbLIoiDiBjggGmWVZRw0BBFFBFQIxhgQIhoN6hti1EcjEY2GaKJgDGrQSBJjIiiP0ceIIiJRFDWCrIJsYRF0lMCAAiKKDtzvH6dmaIZZepie7unp3+e6+pquqlPVdy9z9+lTp84xd0dERDJDjVQHICIiyaOkLyKSQZT0RUQyiJK+iEgGUdIXEckgSvoiIhlESV/KxcxqmtlOMzs+kWVTycxONLOE9102s95mtj5meZWZ9Yin7EE81p/N7NaD3b+U495lZo8n+riSOoekOgCpXGa2M2axHrAb2BMtX+3uk8tzPHffAzRIdNlM4O4nJ+I4ZnYVMMzdz4o59lWJOLZUf0r61Zy7FybdqCZ5lbv/q6TyZnaIu+cnIzYRST4172S46Of7/5rZU2b2OTDMzM4wszlmts3MNprZg2ZWKyp/iJm5mbWMlidF26eb2edm9o6ZtSpv2Wh7PzP7j5ltN7OHzOxtM7u8hLjjifFqM1tjZp+Z2YMx+9Y0s9+b2VYzWwf0LeX1GW1mU4qsm2Bm90f3rzKzFdHzWRvVwks6Vq6ZnRXdr2dmT0SxLQO6Fil7m5mti467zMwGROs7AH8AekRNZ1tiXts7Y/a/JnruW83sn2Z2bDyvTVnMbFAUzzYze83MTo7ZdquZfWJmO8xsZcxzPd3MFkbrN5nZuHgfTyqBu+uWITdgPdC7yLq7gK+BCwmVgEOB04BuhF+CJwD/AUZG5Q8BHGgZLU8CtgDZQC3gf4FJB1H2KOBzYGC07UbgG+DyEp5LPDE+DxwOtAQ+LXjuwEhgGdAcaALMDv8KxT7OCcBOoH7MsTcD2dHyhVEZA84GvgQ6Rtt6A+tjjpULnBXdvw94HWgMZAHLi5S9BDg2ek8ujWI4Otp2FfB6kTgnAXdG9/tEMXYC6gIPA6/F89oU8/zvAh6P7reJ4jg7eo9uBVZF99sBG4BjorKtgBOi+/OAIdH9hkC3VP8vZPJNNX0BeMvdX3D3ve7+pbvPc/e57p7v7uuAiUDPUvZ/xt3nu/s3wGRCsilv2QuAxe7+fLTt94QviGLFGeNv3X27u68nJNiCx7oE+L2757r7VuCeUh5nHfA+4csI4FzgM3efH21/wd3XefAa8CpQ7MnaIi4B7nL3z9x9A6H2Hvu4T7v7xug9eZLwhZ0dx3EBhgJ/dvfF7v4VMAroaWbNY8qU9NqUZjAw1d1fi96jewhfHN2AfMIXTLuoifCD6LWD8OV9kpk1cffP3X1unM9DKoGSvgB8FLtgZqeY2TQz+6+Z7QDGAEeWsv9/Y+7vovSTtyWVPS42Dnd3Qs24WHHGGNdjEWqopXkSGBLdvzRaLojjAjOba2afmtk2Qi27tNeqwLGlxWBml5vZe1EzyjbglDiPC+H5FR7P3XcAnwHNYsqU5z0r6bh7Ce9RM3dfBfwP4X3YHDUXHhMVvQJoC6wys3fNrH+cz0MqgZK+QPi5H+tRQu32RHc/DPglofmiMm0kNLcAYGbG/kmqqIrEuBFoEbNcVpfSp4HeZtaMUON/MorxUOAZ4LeEppdGwCtxxvHfkmIwsxOAR4BrgSbRcVfGHLes7qWfEJqMCo7XkNCM9HEccZXnuDUI79nHAO4+yd27E5p2ahJeF9x9lbsPJjTh/Q541szqVjAWOUhK+lKchsB24AszawNcnYTHfBHoYmYXmtkhwM+AppUU49PA9WbWzMyaALeUVtjd/wu8BTwOrHL31dGmOkBtIA/YY2YXAOeUI4ZbzayRhesYRsZsa0BI7HmE778fE2r6BTYBzQtOXBfjKeBKM+toZnUIyfdNdy/xl1M5Yh5gZmdFj30z4TzMXDNrY2a9osf7MrrtJTyBH5rZkdEvg+3Rc9tbwVjkICnpS3H+BxhO+Id+lHDCtVK5+ybgB8D9wFagNbCIcF1BomN8hND2vpRwkvGZOPZ5knBitrBpx923ATcAzxFOhl5M+PKKxx2EXxzrgenA32OOuwR4CHg3KnMyENsOPhNYDWwys9hmmoL9XyY0szwX7X88oZ2/Qtx9GeE1f4TwhdQXGBC179cBxhLOw/yX8MtidLRrf2CFhd5h9wE/cPevKxqPHBwLTaciVYuZ1SQ0J1zs7m+mOh6R6kI1fakyzKxv1NxRB7id0Ovj3RSHJVKtKOlLVfIdYB2h6eA8YJC7l9S8IyIHQc07IiIZRDV9EZEMUuUGXDvyyCO9ZcuWqQ5DRCStLFiwYIu7l9bNGaiCSb9ly5bMnz8/1WGIiKQVMyvrynJAzTsiIhlFSV9EJIMo6YuIZJAq16ZfnG+++Ybc3Fy++uqrVIcicahbty7NmzenVq2ShoYRkVRJi6Sfm5tLw4YNadmyJWHwRamq3J2tW7eSm5tLq1atyt5BRJIqruad6PL4VdH0aqOK2X6NmS01s8Vm9paZtY3Wn2tmC6JtC8zs7IMJ8quvvqJJkyZK+GnAzGjSpIl+lYlUUWUm/WjgqwlAP8JECEMKknqMJ929g7t3Ioy0d3+0fgtwobt3IIzO98TBBqqEnz70XolUXfHU9HOANdGUcF8DU9g3dRxQODNPgfpEkzy4+yJ3/yRavww4NBpMS0SkWnn9dZgxI9VRlC2epN+M/ad1y6WYGY3M7CdmtpZQ07+umON8D1hY3ABaZjbCzOab2fy8vLz4Ik+irVu30qlTJzp16sQxxxxDs2bNCpe//jq+YcGvuOIKVq1aVWqZCRMmMHny5ESEzHe+8x0WL16ckGOJSOleeAHOPRf69oVbboE9e1IdUSnKmjmdMDHEn2OWfwj8oZTylwJ/K7KuHbAWaF3W43Xt2tWLWr58+QHrSjNpkntWlrtZ+DtpUrl2L9Udd9zh48aNO2D93r17fc+ePYl7oArq3r27L1q0KGWPX973TCRdvfKKe+3a7qed5n711e7gfu657lu2JDcOYL6XkV/dPa6a/sfsP5dn4ZyYJZgCfLdgwcyaE2bwuczd18b3VXTwJk+GESNgwwZwD39HjAjrE23NmjW0bduWoUOH0q5dOzZu3MiIESPIzs6mXbt2jBkzprBsQc07Pz+fRo0aMWrUKE499VTOOOMMNm/eDMBtt93G+PHjC8uPGjWKnJwcTj75ZP79738D8MUXX/C9732Ptm3bcvHFF5OdnV1mjX7SpEl06NCB9u3bc+uttwKQn5/PD3/4w8L1Dz74IAC///3vadu2LR07dmTYsGEJf81EqpPZs2HgQDjlFHj5ZfjjH+Evf4E33oDsbKiKP7bjSfrzgJPMrJWZ1QYGA1NjC5jZSTGL5xOmcsPMGgHTgFHu/nZiQi7d6NGwa9f+63btCusrw8qVK7nhhhtYvnw5zZo145577mH+/Pm89957zJw5k+XLlx+wz/bt2+nZsyfvvfceZ5xxBo899lixx3Z33n33XcaNG1f4BfLQQw9xzDHHsHz5cm6//XYWLVpUany5ubncdtttzJo1i0WLFvH222/z4osvsmDBArZs2cLSpUt5//33ueyyywAYO3YsixcvZsmSJfzhD3+o4KsjUn3NnQvnnw9ZWTBzJhxxRFj/ox/Bm29Cfj58+9uVU+GsiDKTvrvnEyZtngGsAJ5292VmNsbMBkTFRprZMjNbDNxI6KlDtN+JwC+j7pyLzeyoxD+NfT78sHzrK6p169ZkZ2cXLj/11FN06dKFLl26sGLFimKT/qGHHkq/fv0A6Nq1K+vXry/22BdddNEBZd566y0GDx4MwKmnnkq7du1KjW/u3LmcffbZHHnkkdSqVYtLL72U2bNnc+KJJ7Jq1Squu+46ZsyYweGHHw5Au3btGDZsGJMnT9bFVSIlWLQotN8ffTS8+iocVSSr5eTAggXh77BhcP318M03qYm1qLj66bv7S+7+LXdv7e53R+t+6e5To/s/c/d27t7J3Xt5mEAZd7/L3etH6wtumyvv6cDxx5dvfUXVr1+/8P7q1at54IEHeO2111iyZAl9+/Yttr967dq1C+/XrFmT/Pz8Yo9dp06dMsscrCZNmrBkyRJ69OjBhAkTuPrqqwGYMWMG11xzDfPmzSMnJ4c9VfqMlEjyLVsGffpAw4Yh4R93XPHljjoq/AL42c/ggQfCid7NlZr94lPtxt65+26oV2//dfXqhfWVbceOHTRs2JDDDjuMjRs3MqMS+m91796dp59+GoClS5cW+0siVrdu3Zg1axZbt24lPz+fKVOm0LNnT/Ly8nB3vv/97zNmzBgWLlzInj17yM3N5eyzz2bs2LFs2bKFXUXbykQy2OrV0Ls31KoFr70WmnZKU6sWjB8PTzwRmoO6doV585ITa0nSYhiG8hg6NPwdPTo06Rx/fEj4BesrU5cuXWjbti2nnHIKWVlZdO/ePeGP8dOf/pTLLruMtm3bFt4KmmaK07x5c379619z1lln4e5ceOGFnH/++SxcuJArr7wSd8fMuPfee8nPz+fSSy/l888/Z+/evdx00000bNgw4c9BJB2tXw/nnBPa6t94A048Mf59hw2Ddu1g0CDo0QMefji0/adClZsjNzs724tOorJixQratGmTooiqlvz8fPLz86lbty6rV6+mT58+rF69mkMOqVrf33rPpDr5+GM480z49FOYNQs6dTq442zZAkOGwL/+BddeG34FxLT2VoiZLXD37LLKVa1MIWXauXMn55xzDvn5+bg7jz76aJVL+CLVyaZNoYaflxeS9cEmfIAjj4Tp00NLxNix8N578MwzcOyxiYu3LMoWaaZRo0YsWLAg1WGIZIRPPw0nYD/8MAyxkJNT8WMecgjce29o37/iivD3mWdC985kqHYnckVEEmH7djjvPPjPf2Dq1NAWn0iXXAJz5oSOJmedFS7sSkZru5K+iEgRO3dC//7hitpnngk9dipDhw6hN0/v3qGN/6qrYO/eynmsAkr6IlItrF0bhkXYtKliNeYvvwxDK8yZA089BRdckLgYi9O4cRiwbfRoqF8falRyVlabvoiktd274a674J57QndKgEaN4OSTw5g4p5yy737r1qX3ltm9G773vdBD5+9/h4svTs5zqFkzPAc171QRvXr1OuBCq/Hjx3PttdeWul+DBg0A+OSTT7i4hE/PWWedRdEuqkWNHz9+v4uk+vfvz7Zt2+IJvVR33nkn9913X4WPI5Iqb78detPcdRcMHgzTpoVukEOGhLbymTPhF7+Aiy6Ctm3Dum99CwYMgJtvDoOjvfVW6EqZnx/2mz49tK+nYrzBZMw/pJp+HIYMGcKUKVM477zzCtdNmTKFsWPHxrX/cccdxzPPPHPQjz9+/HiGDRtGvehS45deeumgjyVSHezYEZL5ww+HCzCnTw9j4ZRUdtWqcFu5ct/fV14JNfsC9evDF1+EL40RI5LzPFJBNf04XHzxxUybNq1wwpT169fzySef0KNHj8J+8126dKFDhw48//zzB+y/fv162rdvD8CXX37J4MGDadOmDYMGDeLLL78sLHfttdcWDst8xx13APDggw/yySef0KtXL3r16gVAy5Yt2bJlCwD3338/7du3p3379oXDMq9fv542bdrw4x//mHbt2tGnT5/9Hqc4ixcv5vTTT6djx44MGjSIzz77rPDxC4ZaLhjo7Y033iicRKZz5858/vnnB/3aipTXtGnh6tZHHoHrrgtj4ZSU8AEOOwxOOy3U3O+6C/7xD1i6NCT4tWvD8e6/P2x//PEwVk51lnY1/euvT/wY1Z06hW/3khxxxBHk5OQwffp0Bg4cyJQpU7jkkkswM+rWrctzzz3HYYcdxpYtWzj99NMZMGBAifPEPvLII9SrV48VK1awZMkSunTpUrjt7rvv5ogjjmDPnj2cc845LFmyhOuuu47777+fWbNmceSRR+53rAULFvDXv/6VuXPn4u5069aNnj170rhxY1avXs1TTz3Fn/70Jy655BKeffbZUsfHv+yyy3jooYfo2bMnv/zlL/nVr37F+PHjueeee/jggw+oU6dOYZPSfffdx4QJE+jevTs7d+6kbt265Xi1RQ5OXl74/3/yydBU8/TTcMYZB3+8mjXhhBPCrX//xMVZ1ammH6eCJh4ITTtDhgwBwpj3t956Kx07dqR37958/PHHbNq0qcTjzJ49uzD5duzYkY4dOxZue/rpp+nSpQudO3dm2bJlZQ6m9tZbbzFo0CDq169PgwYNuOiii3jzzTcBaNWqFZ2iSwdLG74Zwvj+27Zto2fPngAMHz6c2bNnF8Y4dOhQJk2aVHjlb/fu3bnxxht58MEH2bZtm64IzkD5+fDBB2GUyYkTYdSo0O985Eh46aXQAyZR3GHSJGjTJtTS77gDFi6sWMLPZGn331pajbwyDRw4kBtuuIGFCxeya9cuunbtCsDkyZPJy8tjwYIF1KpVi5YtWxY7nHJZPvjgA+677z7mzZtH48aNufzyyw/qOAUKhmWGMDRzWc07JZk2bRqzZ8/mhRde4O6772bp0qWMGjWK888/n5deeonu3bszY8YMTjnllIOOVaqm7dth3brQBLJu3b7b2rVhRrrYUbdr1QojTn7yCUyYAIceGoYuuOCCMNFI8+YHF8OGDaH/+vTp0K1bOPFaxhQSUgbV9OPUoEEDevXqxY9+9KPCWj6EWvJRRx1FrVq1mDVrFhs2bCj1OGeeeSZPPvkkAO+//z5LliwBwrDM9evX5/DDD2fTpk1Mnz69cJ+GDRsW227eo0cP/vnPf7Jr1y6++OILnnvuOXocxGWDhx9+OI0bNy78lfDEE0/Qs2dP9u7dy0cffUSvXr2499572b59Ozt37mTt2rV06NCBW265hdNOO42VK1eW+zGl6nn88dADJicHmjQJ3R67dIHvfz9M9v3ss+GLICcnLP/lL6Fr44YNoWa/ejVs3RqmDbzyytDWfs010KJFaEIdPRreeSe+ScP37oWHHgoJfvbsMB79228r4SdC2tX0U2nIkCEMGjSosJkHYOjQoVx44YV06NCB7OzsMmu81157LVdccQVt2rShTZs2hb8YTj31VDp37swpp5xCixYt9huWecSIEfTt25fjjjuOWbNmFa7v0qULl19+OTnRgCBXXXUVnTt3LrUppyR/+9vfuOaaa9i1axcnnHACf/3rX9mzZw/Dhg1j+/btuDvXXXcdjRo14vbbb2fWrFnUqFGDdu3aFc4CJulp7174+c/hd78LtfWTTw5NNa1b72vzbtUKShnBu1DdumHogvPOgwcfhBUrwonSadPCeDO/+U0YdKxfv/AL4LzzwpdLrOXLw5Wp77wTtv/xj9CyZaU89YykoZWlUug9Sw/ffBNq5U88AT/9aWg+rawrQj/7LHSTfPHF0FyzdWs4mfqd74QvgL594bnnwvwXDRqEWIYNS07f9epAQyuLSKm++CI03UyfHroy3npr5SbYxo3hBz8Itz17wkxS06aFL4Gf/zzcIDQxPfDAgfPOSmIo6YtkoK1bw0nWd98NvW9+/OPkPn7NmmEo4W9/O9TsP/ooDF3csmXlDW4mQVw/5Mysr5mtMrM1ZjaqmO3XmNlSM1tsZm+ZWduYbb+I9ltlZucV3TdeVa0ZSkqm96pq++ijMEzwokXh5GyyE35xWrQI7fhK+JWvzKRvZjWBCUA/oC0wJDapR5509w7u3gkYC9wf7dsWGAy0A/oCD0fHK5e6deuydetWJZM04O5s3bpVF2xVUcuXh9r1J5+E9vXvfjfVEUmyxdO8kwOscfd1AGY2BRgIFF455O47YsrXBwqy80BgirvvBj4wszXR8d4pT5DNmzcnNzeXvLy88uwmKVK3bl2aH2zHbKk077wTTpjWqRO6QcZcFygZJJ6k3wz4KGY5F+hWtJCZ/QS4EagNnB2z75wi+zYrZt8RwAiA448//oAAatWqRatWreIIVUSK89JLYZjgZs1CDV//TpkrYZ2z3H2Cu7cGbgFuK+e+E909292zmzZtmqiQRIQwLvyAAWG8mrffVsLPdPEk/Y+BFjHLzaN1JZkCFLQUlndfEUmg++6D4cPDHKyzZqkbpMSX9OcBJ5lZKzOrTTgxOzW2gJmdFLN4PrA6uj8VGGxmdcysFXAS8G7FwxapXjZtCjXyp54K49tUtM/C3r1hkpCbbw5X106bBg0bJiZWSW9ltum7e76ZjQRmADWBx9x9mZmNAea7+1RgpJn1Br4BPgOGR/suM7OnCSd984GfuHscI2+IVG/uYUz3F14It3ff3T/RN20aBhjr1g1OPz2MBx/PMAgQrrK96qrwJTJyZLjQqbLnXZX0kRbDMIhUB7t3w+uv70v0H34Y1p92Glx4YbhYyixMyD13bvhbMJadWZjj9fTT930ZtG8PRUe1/uKLULN/6SX49a/DIGcaxiAzxDsMg5K+SCXKywtNKy+8EHrN7NwZhh0+99yQ6M8/H449tuT9t22DefP2fRHMnRvmc4Uw32t29r4vgjZtwjg6c+eGWaWq85R/ciAlfZEUcA9DChfU5ufMCeuaNQs1+QsvhLPPDon/YI+/bt2+L4A5c8KVtd98E7bXqRNmlrroosQ9J0kPGnBNJIk+/xzuuSck3IKRrbt2DbM8XXghdO6cmGYWszDkcevWcOmlYd3u3WEK0fnzw2xSMTNwihyg2pzemTw5DNZUo0b4O3lyqiOSTOAOU6aE9vbf/jb0hX/0UcjNDUn4jjtCEq7MdvU6dULzzk9+ooQvZasWNf3Jk0P75a5dYXnDhn3tmUOHpi4uqd5Wrgy9Y159NSTb554Ls0qJVGXVoqY/evS+hF9g166wXiTRvvgCfvGLMHbNggVhTth331XCl/RQLWr6BV3f4l0vcjDc4Z//hOuvD5+t4cNh7Fhd5SrppVrU9IsZo63U9SLltWZN6F550UXhIqk33wwTiSvhS7qpFkn/7rtDn+VY9eqF9SIV8eWX4WRs+/bw1ltw//2wcGGY11UkHVWLpD90aJjyLSsr9JLIygrLOokrFTFtWkj2Y8aEGv7KlXDDDQdeBSuSTqrNx3foUCV5SYwNG+BnP4Pnnw9dMV99NVxQJVIdVIuavkgi7N4Nv/lNGM5g5sxwsdV77ynhS/VSbWr6Un24h7b0HTvCbfv2ffeLLhfdtnt3GFZ4z559t9jl0rbt3g1ffRWacn7/e3UEkOpJSV9Szn3fqJCrV4fknZ9f9n5164aeNIcdFm4NG0KDBlCzZrgyu2bN+O8X3Pr2hT59Kv85i6SKkr6k1L//DaNGhS6QrVvDkCH7J/LDDtt/ueB+w4ZQu3aqoxdJP0r6khLLlsGtt8LUqXDMMfDww2Hij1q1Uh2ZSPWmE7mSVBs2wOWXQ4cOYUKRu+8OFz5de60SvkgyqKYvSbFlS+gZM2FCuJbixhvD+DVNmqQ6MpHMoqQvlWrnThg/HsaNC/eHD4c771TPGJFUiat5x8z6mtkqM1tjZqOK2X6jmS03syVm9qqZZcVsG2tmy8xshZk9aKYZOzPB11+HWv2JJ8Ltt4e+7kuXwmOPKeGLpFKZSd/MagITgH5AW2CImbUtUmwRkO3uHYFngLHRvt8GugMdgfbAaUDPhEUvVc7evfDUU+ECp5Ej4eSTQw+d554LE4yISGrFU9PPAda4+zp3/xqYAgyMLeDus9y9YET7OUDzgk1AXaA2UAeoBWxKROBStezaFYYdzs4O0/g1aBDGrnn99TCFn4hUDfG06TcDPopZzgW6lVL+SmA6gLu/Y2azgI2AAX9w9xVFdzCzEcAIgOP12z8tuMP778OMGfDKKzB7driitVUrmDQp9Levob5hIlVOQk/kmtkwIJuoCcfMTgTasK/mP9PMerj7m7H7uftEYCJAdna2JzImSZwtW8KYNAWJfuPGsL5t29Dl8rzzQtu9LpoSqbriSfofAy1ilptH6/ZjZr2B0UBPd98drR4EzHH3nVGZ6cAZwJtF95eq55tv4J13QpKfMSOMI+8OjRvDueeG4Qr69IEWLco+lohUDfEk/XnASWbWipDsBwOXxhYws87Ao0Bfd98cs+lD4Mdm9ltC805PYHwiApfKsXbtviT/2muhm2XNmnD66fCrX4Ukn50d1olI+ikz6bt7vpmNBGYANYHH3H2ZmY0B5rv7VGAc0AD4R9Qj80N3H0DoyXM2sJRwUvdld3+hcp6KVERuLgwcGGrzAC1bhvkJzjsPevWCRo1SGp6IJIi5V60m9OzsbJ8/f36qw8go69eHtvitW8NIl/36hf71uqJCJH2Y2QJ3zy6rnPpXRCZPDrXbGjXC38mTUx1RcqxZA2eeCZ99Bv/6F1x3HZx0khK+SHWlYRgICX7EiNDXHMKgYCNGhPvVeQrGlStDDf/rr2HWLOjUKdURiUhlU00fGD16X8IvsGtXWF9dvf8+9OwZrqB9/XUlfJFMoaQPfPhh+danu0WL4Kyz4JBD4I03oH37VEckIsmipE/JA4BVx4uD3303NOnUrx+uoj355FRHJCLJpKRPmMijXr3919WrF9Yni3voLvnpp5X3GG+/Db17wxFHhITfunXlPZaIVE1K+oSTtRMnQlZW6LWSlRWWk3ES1x2mTw8XP3XtGnoO3XZb6D6ZSLNmhT73xx4bmnSyssreR0Sqn2qT9PfuDclsxYpQWy7v5QdDh4b+6nv3hr+VnfBjk33//rB5MzzwQOgj/5vfhOR/661hvJuKeuWV8BhZWeE1at687H1EpHqqNhdnbd4MRx+9b7lWLWjaNKw7+mg46qiS7zdtGk5qJoM7vPxymD3q3XdDIr7tNrjssn0DlS1bFi6Sevrp0Mw0ciT8z/+EOMvrxRfhe98L49vPnHlwxxCRqi/ei7OqTdL/6qswWcemTftumzcfeH/37gP3NQtztR59NHTrFmrbvXsnduiBeJJ9UcuXw113wZQpcOih8JOfwE03hS+qePzf/8HgwXDqqWEsnSOOSNjTEZEqJuOSfjzcYceOkr8QcnPhzTdh+/Z9g4z17RtuXbqUPj785MlMMX5NAAARNklEQVShX/+HH4ZeP3ffHZqIDibZF7VyZUj+Tz0FdeuGYYxvvnn/XzZFTZkCw4ZBTk5oRjr88HK9VCKSZuJN+rh7lbp17drVU+mbb9zfest99Gj3rl3dQ9p2b9rUfdgw90mT3Ddv3n+fSZPc69XbVxbcDz3U/eab3XNywnJWlvvEie67dx98bCtXuv/wh+41aoTj33CD+8aNB5Z7/PFQ5swz3XfsOPjHE5H0QRgAs8wcm1E1/YOxeXM4Efryy6GJZMuW0BzUtWv4BdCvX5glqqQLubKywi+A4cMTN7nI6tXhl8SkSeHcxdVXwy23hJ45EyfCNdfAOefA888f2BVVRKonNe9Ugr17Q1/6l18OTSZz5oR1pdm9u/JmklqzJvT0+fvfw4novn1Dou/fH559NjQFiUhmUNJPgoKRKa+4Ar744sDtWVmh+2dlW7cuJP+//Q0uuCC059epU/mPKyJVh4ZWToLGjeH734dHHw29a2Il84reE06AP/8Z8vJCjx0lfBEpiZJ+AgwdCn/6U2qu6I3VqJHGwReR0mk8/QQZOrR6j70vItWDavoiIhkkrqRvZn3NbJWZrTGzUcVsv9HMlpvZEjN71cyyYrYdb2avmNmKqEzLxIVffWTqdI0iklxlJn0zqwlMAPoBbYEhZta2SLFFQLa7dwSeAcbGbPs7MM7d2wA5wOZEBF6dFEzXuGFDuLSrYLpGJX4RSbR4avo5wBp3X+fuXwNTgIGxBdx9lrsXTDg4B2gOEH05HOLuM6NyO2PKSSQTp2sUkdSIJ+k3Az6KWc6N1pXkSmB6dP9bwDYz+z8zW2Rm46JfDvsxsxFmNt/M5ufl5cUbe7WRadM1ikjqJPRErpkNA7KBcdGqQ4AewE3AacAJwOVF93P3ie6e7e7ZTTNw7N9Mmq5RRFIrnqT/MdAiZrl5tG4/ZtYbGA0McPeCAYxzgcVR01A+8E+gS8VCrn6qwnSNIpIZ4kn684CTzKyVmdUGBgNTYwuYWWfgUULC31xk30ZmVlB9PxtYXvGwq5dUTtcoIpmlzIuz3D3fzEYCM4CawGPuvszMxhCG8pxKaM5pAPzDwiWhH7r7AHffY2Y3Aa9a2LAA+FNlPZl0pou7RCQZNOCaiEg1oAHXRETkAEr6IiIZRElfRCSDKOmLiGQQJf1qQgO2iUg8NJ5+NVAwYFvB+D0FA7aBuoGKyP5U068GNGCbiMRLSb8a0IBtIhIvJf1qQAO2iUi8lPSrAQ3YJiLxUtKvBjRgm4jES0m/mhg6FNavh717w9/yJnx1+RTJDOqyKeryKZJBVNMXdfkUySBK+qIunyIZRElf1OVTJIMo6Yu6fIpkECV9UZdPkQyipC+AunyKZAp12ZQKU5dPkfQRV03fzPqa2SozW2Nmo4rZfqOZLTezJWb2qpllFdl+mJnlmtkfEhW4VB3q8imSPspM+mZWE5gA9APaAkPMrG2RYouAbHfvCDwDjC2y/dfA7IqHK1WRunyKpI94avo5wBp3X+fuXwNTgIGxBdx9lrsX1PXmAM0LtplZV+Bo4JXEhCxVjbp8iqSPeJJ+M+CjmOXcaF1JrgSmA5hZDeB3wE2lPYCZjTCz+WY2Py8vL46QpCpRl0+R9JHQ3jtmNgzIBsZFq/4f8JK755a2n7tPdPdsd89u2rRpIkOSJFCXT5H0EU/S/xhoEbPcPFq3HzPrDYwGBrj77mj1GcBIM1sP3AdcZmb3VChiqZLU5VMkPcTTZXMecJKZtSIk+8HApbEFzKwz8CjQ1903F6x396ExZS4nnOw9oPePZDZ1+RRJnjJr+u6eD4wEZgArgKfdfZmZjTGzAVGxcUAD4B9mttjMplZaxFLtqMunSPKYu6c6hv1kZ2f7/PnzUx2GJFGNGlDcx9AsNBeJSNnMbIG7Z5dVTsMwSMqpy6dI8ijpS8qpy6dI8ijpS8qpy6dI8mjANakShg5VkhdJBtX0pVpQP3+R+KimL2lP/fxF4qeavqQ99fMXiZ+SvqQ9De0sEj8lfUl76ucvEj8lfUl76ucvEj8lfUl76ucvEj/13pFqQf38ReKjmr6ISAZR0hdBF3dJ5lDzjmQ8XdwlmUQ1fcl4urhLMomSvmQ8XdwlmURJXzKeLu6STKKkLxlPF3dJJokr6ZtZXzNbZWZrzGxUMdtvNLPlZrbEzF41s6xofScze8fMlkXbfpDoJyBSUYm4uEu9fyRdlDkxupnVBP4DnAvkAvOAIe6+PKZML2Cuu+8ys2uBs9z9B2b2LcDdfbWZHQcsANq4+7aSHk8To0u6Kdr7B8IvBV0VLMmUyInRc4A17r7O3b8GpgADYwu4+yx3L/jIzwGaR+v/4+6ro/ufAJuBpvE/DZGqT71/JJ3Ek/SbAR/FLOdG60pyJTC96EozywFqA2uL2TbCzOab2fy8vLw4QhKpOtT7R9JJQk/kmtkwIBsYV2T9scATwBXuvrfofu4+0d2z3T27aVP9EJD0ot4/kk7iSfofAy1ilptH6/ZjZr2B0cAAd98ds/4wYBow2t3nVCxckapHvX8kncST9OcBJ5lZKzOrDQwGpsYWMLPOwKOEhL85Zn1t4Dng7+7+TOLCFqk6NLSzpJMyx95x93wzGwnMAGoCj7n7MjMbA8x396mE5pwGwD/MDOBDdx8AXAKcCTQxs8ujQ17u7osT/1REUkdDO0u6KLPLZrKpy6aISPklssumiIhUE0r6IiIZRElfpArQMA6SLJpERSTFNImLJJNq+iIppmEcJJmU9EVSTMM4SDIp6YukmIZxkGRS0hdJsUQM46ATwRIvJX2RFKvoMA4FJ4I3bAD3fSeClfilOLoiVyTNtWwZEn1RWVmwfn2yo5FU0RW5IhlCJ4KlPJT0RdKcTgRLeSjpi6Q5jecv5aGkL5LmNJ6/lIeSvkg1MHRoOGm7d2/4W96Ery6fmUNj74hkOI39k1lU0xfJcBr7J7Mo6YtkOHX5zCxK+iIZTl0+M0tcSd/M+prZKjNbY2ajitl+o5ktN7MlZvaqmWXFbBtuZquj2/BEBi8iFaexfzJLmUnfzGoCE4B+QFtgiJm1LVJsEZDt7h2BZ4Cx0b5HAHcA3YAc4A4za5y48EWkojT2T2aJp6afA6xx93Xu/jUwBRgYW8DdZ7l7wamgOUDz6P55wEx3/9TdPwNmAn0TE7qIJEpFunzqRHB6iSfpNwM+ilnOjdaV5Epgenn2NbMRZjbfzObn5eXFEZKIVBU6EZxeEnoi18yGAdnAuPLs5+4T3T3b3bObNm2ayJBEpJLpRHB6iSfpfwy0iFluHq3bj5n1BkYDA9x9d3n2FZH0pbF/0ks8SX8ecJKZtTKz2sBgYGpsATPrDDxKSPibYzbNAPqYWePoBG6faJ2IVBMa+ye9lJn03T0fGElI1iuAp919mZmNMbMBUbFxQAPgH2a22MymRvt+Cvya8MUxDxgTrRORakRj/6QPzZwlIilVdOwfCM1D+rVQPpo5S0TSgrp8JpeSvoiklLp8JpeSvoiklLp8JpeSvoiklMb+SS4lfRFJKY39k1zqvSMiaa1ly5Doi8rKCt1HM4V674hIRtCJ4PJR0heRtJaIE8GZdE5ASV9E0lpFTwRn2jkBJX0RSWsVPRGcaReH6USuiGS0GjVCDb8oszCWULrQiVwRkThk2sVhSvoiktEybT4AJX0RyWiJmA8gnXr/HJLqAEREUm3o0IMfxrno0NAFvX8KjlvVqKYvIlIB6db7R0lfRKQC0u2KYCV9EZEKSLfeP0r6IiIVkG69f+JK+mbW18xWmdkaMxtVzPYzzWyhmeWb2cVFto01s2VmtsLMHjQzS1TwIiKplm69f8rsvWNmNYEJwLlALjDPzKa6+/KYYh8ClwM3Fdn320B3oGO06i2gJ/B6RQMXEakq0qn3Tzw1/Rxgjbuvc/evgSnAwNgC7r7e3ZcARS9adqAuUBuoA9QCNlU4ahGRaiLZvX/iSfrNgI9ilnOjdWVy93eAWcDG6DbD3VeUN0gRkeoq2b1/KvVErpmdCLQBmhO+KM42sx7FlBthZvPNbH5eXl5lhiQiUqUku/dPPEn/Y6BFzHLzaF08BgFz3H2nu+8EpgNnFC3k7hPdPdvds5s2bRrnoUVE0l+ye//Ek/TnASeZWSszqw0MBqbGefwPgZ5mdoiZ1SKcxFXzjohIJBG9f8ojrvH0zaw/MB6oCTzm7neb2RhgvrtPNbPTgOeAxsBXwH/dvV3U8+dh4EzCSd2X3f3G0h5L4+mLiJRfvOPpaxIVEZFqQJOoiIjIAZT0RUQyiJK+iEgGUdIXEckgVe5ErpnlARtSHUcpjgS2pDqIUii+ilF8FaP4KqYi8WW5e5kXOlW5pF/Vmdn8eM6Qp4riqxjFVzGKr2KSEZ+ad0REMoiSvohIBlHSL7+JqQ6gDIqvYhRfxSi+iqn0+NSmLyKSQVTTFxHJIEr6IiIZREm/CDNrYWazzGx5NKH7z4opc5aZbTezxdHtlymIc72ZLY0e/4AR6ix4MJrMfomZdUlibCfHvDaLzWyHmV1fpExSX0Mze8zMNpvZ+zHrjjCzmWa2OvrbuIR9h0dlVpvZ8CTGN87MVkbv33Nm1qiEfUv9LFRifHea2ccx72H/Evbta2aros/iqCTG978xsa03s8Ul7JuM16/YvJKSz6C76xZzA44FukT3GwL/AdoWKXMW8GKK41wPHFnK9v6ESWsMOB2Ym6I4awL/JVw4krLXkDC8dxfg/Zh1Y4FR0f1RwL3F7HcEsC762zi63zhJ8fUBDonu31tcfPF8FioxvjuBm+J4/9cCJxDmyn6v6P9TZcVXZPvvgF+m8PUrNq+k4jOomn4R7r7R3RdG9z8nTPoS15zAVcxA4O8ezAEamdmxKYjjHGCtu6f0Kmt3nw18WmT1QOBv0f2/Ad8tZtfzgJnu/qm7fwbMBPomIz53f8Xd86PFOYRZ61KihNcvHjnAGndf5+5fA1MIr3tClRafmRlwCfBUoh83XqXklaR/BpX0S2FmLYHOwNxiNp9hZu+Z2XQza5fUwAIHXjGzBWY2opjtBz2hfYINpuR/tlS/hke7+8bo/n+Bo4spU1Vexx8RfrkVp6zPQmUaGTU/PVZC00RVeP16AJvcfXUJ25P6+hXJK0n/DCrpl8DMGgDPAte7+44imxcSmitOBR4C/pns+IDvuHsXoB/wEzM7MwUxlMrC9JoDgH8Us7kqvIaFPPyOrpL9l81sNJAPTC6hSKo+C48ArYFOwEZCE0pVNITSa/lJe/1KyyvJ+gwq6RfDwny+zwKT3f3/im539x0eJnrH3V8CapnZkcmM0d0/jv5uJkxVmVOkSEUmtE+UfsBCd99UdENVeA2BTQVNXtHfzcWUSenraGaXAxcAQ6OkcIA4PguVwt03ufsed98L/KmEx03163cIcBHwvyWVSdbrV0JeSfpnUEm/iKj97y/ACne/v4Qyx0TlMLMcwuu4NYkx1jezhgX3CSf83i9SbCpwWdSL53Rge8zPyGQpsYaV6tcwMhUo6AkxHHi+mDIzgD5m1jhqvugTrat0ZtYX+DkwwN13lVAmns9CZcUXe45oUAmPOw84ycxaRb/8BhNe92TpDax099ziNibr9SslryT/M1iZZ6zT8QZ8h/ATawmwOLr1B64BronKjASWEXoizAG+neQYT4ge+70ojtHR+tgYDZhA6DmxFMhOcoz1CUn88Jh1KXsNCV8+G4FvCG2iVwJNgFeB1cC/gCOistnAn2P2/RGwJrpdkcT41hDacgs+h3+Myh4HvFTaZyFJ8T0RfbaWEJLXsUXji5b7E3qrrE1mfNH6xws+czFlU/H6lZRXkv4Z1DAMIiIZRM07IiIZRElfRCSDKOmLiGQQJX0RkQyipC8ikkGU9EVEMoiSvohIBvn/q3XDRlAae10AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss) + 1)\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이 네트워크는 일반 GRU 층과 비슷한 성능을 냅니다. 이유는 쉽게 이해할 수 있습니다. 모든 예측 성능은 시간 순서대로 처리하는 네트워크의 절반에서 옵니다. 시간 반대 순서로 처리하는 절반은 이런 작업에 성능이 매우 좋지 않기 때문입니다(최근의 정보가 오래 전의 정보보다 훨씬 더 중요합니다)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.3.9 더 나아가서\n",
    "\n",
    "온도 예측 문제의 성능을 향상하기 위해 시도해 볼 수 있는 것들이 많이 있습니다.\n",
    "\n",
    "- 스태킹한 각 순환 층의 유닛 수를 조정합니다. 지금 설정은 대부분 임의로 한 것이라 최적화가 덜 되었을 것입니다.\n",
    "\n",
    "- RMSprop 옵티마이저가 사용한 학습률을 조정합니다.\n",
    "\n",
    "- GRU 대신 LSTM 층을 사용합니다.\n",
    "\n",
    "- 순환 층 위에 용량이 큰 완전 연결된 회귀 층을 사용합니다. 즉 유닛 수가 많은 Dense 층이나 Dense 층을 스태킹합니다.\n",
    "\n",
    "- 최종적으로 (검증 MAE 기준으로 봤을 때) 최선의 모델을 테스트 세트에서 확인해야 합니다. 이를 잊으면 검증 세트에 과대적합된 네트워크 구조를 만들게 될 것입니다.\n",
    "\n",
    "늘 그렇듯이 딥러닝은 과학보다는 예술에 가깝습니다. 어떤 문제에 적합하거나 그렇지 않은 가이드라인은 제시할 수 있지만 결국 모든 문제는 다릅니다. 경험을 바탕으로 다른 전략들을 시도해 봐야 합니다. 현재는 문제를 해결하는 최선의 방법을 미리 알 수 있는 이론이 없습니다. 반복해서 시도해야 합니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.3.10 정리 \n",
    "\n",
    "다음은 이번 절에서 배운 것들입니다.\n",
    "\n",
    "- 4장에서 처음 배웠던 것처럼 새로운 문제를 해결할 때는 선택한 지표에서 상식 수준의 기준점을 설정하는 것이 좋습니다. 기준점을 가지고 있지 않으면 실제 향상이 되었는지 알 수 없습니다.\n",
    "\n",
    "- 계산 비용을 추가할지 판단하기 위해서 비용이 비싼 모델 전에 간단한 모델을 시도합니다.\n",
    "\n",
    "- 시간 순서가 중요한 데이터가 있다면 순환 층이 적합합니다. 시계열 데이터를 펼쳐서 처리하는 모델의 성능을 쉽게 앞지를 것입니다.\n",
    "\n",
    "- 순환 네트워크에 드롭아웃을 사용하려면 타임스텝 동안 일정한 드롭아웃 마스크와 순환 드롭아웃 마스크를 사용해야 합니다. 둘 다 케라스 순환 층에 포함되어 있습니다. 순환 층에 있는 `dropout`과 `recurrent_dropout` 매개변수를 사용하면 됩니다.\n",
    "\n",
    "- 스태킹 RNN은 단일 RNN 층보다 더 강력한 표현 능력을 제공합니다. 하지만 계산 비용이 많이 들기 때문에 항상 시도할 가치가 있지는 않습니다. (기계 번역 같은) 복잡한 문제에서 확실히 도움이 되지만 작고 간단한 문제에서는 항상 그렇지 않습니다.\n",
    "\n",
    "- 양쪽 방향으로 시퀀스를 바라보는 양방향 RNN은 자연어 처리 문제에 유용합니다. 하지만 최근의 정보가 오래된 것보다 훨씬 의미 있는 시퀀스 데이터에는 잘 작동하지 않습니다(온도 예측).\n",
    "\n",
    "여기서 자세히 다루지 않은 두 가지 중요한 개념이 있습니다. 순환 어텐션과 시퀀스 마스킹입니다. 둘 다 자연어 처리에 깊게 관련되어 있고 온도 예측 문제에는 적합하지 않습니다. 이 책을 끝내고 앞으로 공부할 목록으로 남겨 두겠습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "tensorflow",
   "language": "python",
   "name": "tensorflow"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}