Chap06-Deep_learning_for_text_and_sequences/6.4-sequence-processing-with-convnets.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6.4 컨브넷을 사용한 시퀀스 처리\n",
    "\n",
    "- 컴퓨터 비전에서 뛰어난 컨브넷(Convnet)의 특징이 시퀀스 처리와도 깊게 관련되어 있다.\n",
    "\n",
    "- 시간을 2D 이미지의 높이와 너비 같은 공간의 차원으로 다룰 수 있다.\n",
    "\n",
    "- 텍스트 분류나 시계열 예측 같은 간단한 문제에서 작은 1D 컨브넷이 RNN을 대신하여 빠르게 처리할 수 있다고 알려져 있다. \n",
    "\n",
    "- **1D 컨브넷**(1D Convnet)은 특정 시퀀스 처리 문제에서 RNN과 비슷한 성능을 낸다.\n",
    "    - 일반적으로 RNN에 비해 계산 비용이 훨씬 싸다.\n",
    "    - 1D 컨브넷은 전형적으로 팽창된 커널(dilated kernel)과 함께 사용된다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.4.1 시퀀스 데이터를 위한 1D 합성곱 이해하기\n",
    "\n",
    "- 1D-Convolution은 시퀀스에서 1D 패치(patch, 부분 시퀀스)를 추출하여 1D-합성곱을 적용한다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](./images/1d-conv.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 이런 1D 합성곱 층은 시퀀스에 있는 **지역 패턴**을 인식할 수 있다.\n",
    "\n",
    "- 동일한 변환이 시퀀스에 있는 모든 패치에 적용되기 때문에 특정 위치에서 학습한 패턴을 나중에 다른 위치에서 인식할 수 있다.\n",
    "\n",
    "- 이것은 1D 컨브넷에 시간의 이동에 대한 **이동 불변성**(translation invariant)을 제공한다.\n",
    "\n",
    "    - 예를 들어 크기 5인 윈도우를 사용하여 문자 시퀀스를 처리하는 1D 컨브넷은 5개 이하의 단어나 단어의 부분을 학습한다.\n",
    "    \n",
    "    - 이 컨브넷은 이 단어가 입력 시퀀스의 어느 문장에 있더라도 인식할 수 있다. \n",
    "    \n",
    "- 따라서, 문자 수준의 1D 컨브넷은 **단어의 형태학(word morphology)**에 관해 학습할 수 있다. ([[링크](https://arxiv.org/abs/1610.03017)] 참고)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.4.2 시퀀스 데이터를 위한 1D 풀링\n",
    "\n",
    "- 1D 풀링(pooling) 연산은 2D 풀링 연산과 동일하다.\n",
    "\n",
    "- 입력에서 1D 패치(부분 시퀀스)를 추출하고 최대값(최대 풀링)을 출력하거나 평균값(평균 풀링)을 출력한다.\n",
    "\n",
    "- 2D 컨브넷과 마찬가지로 1D 입력의 길이를 줄이기 위해 사용한다(서브샘플링(subsamplig)).\n",
    "\n",
    "![](./images/1D-pooling.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.4.3 1D 컨브넷 구현\n",
    "\n",
    "케라스에서 1D 컨브넷은 [Conv1D](https://keras.io/layers/convolutional/#conv1d) 층을 사용하여 구현합니다. `Conv1D`는 `Conv2D`와 인터페이스가 비슷합니다. \n",
    "\n",
    "- `(samples, time, features)` 크기의 3D 텐서를 입력받고 비슷한 형태의 3D 텐서를 반환합니다. \n",
    "\n",
    "- 합성곱 윈도우는 시간 축의 1D 윈도우입니다. 즉, 입력 텐서의 두 번째 축입니다.\n",
    "\n",
    "간단한 두 개 층으로 된 1D 컨브넷을 만들어 익숙한 IMDB 감성 분류 문제에 적용해 보죠.\n",
    "\n",
    "기억을 되살리기 위해 데이터를 로드하고 전처리하는 코드를 다시 보겠습니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'2.2.4'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import keras \n",
    "\n",
    "keras.__version__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "데이터 로드...\n",
      "25000 훈련 시퀀스\n",
      "25000 테스트 시퀀스\n",
      "시퀀스 패딩 (samples x time)\n",
      "train_x.shape : (25000, 500)\n",
      "test_x.shape : (25000, 500)\n"
     ]
    }
   ],
   "source": [
    "from keras.datasets import imdb\n",
    "from keras.preprocessing import sequence\n",
    "\n",
    "max_features = 10000  # 특성으로 사용할 단어의 수\n",
    "max_len = 500  # 사용할 텍스트의 길이 (가장 빈번한 max_features 개의 단어만 사용)\n",
    "\n",
    "print('데이터 로드...')\n",
    "(train_x, train_y), (test_x, test_y) = imdb.load_data(num_words=max_features)\n",
    "print(len(train_x), '훈련 시퀀스')\n",
    "print(len(test_x), '테스트 시퀀스')\n",
    "\n",
    "print('시퀀스 패딩 (samples x time)')\n",
    "train_x = sequence.pad_sequences(train_x, maxlen=max_len)  # padding='pre'\n",
    "test_x = sequence.pad_sequences(test_x, maxlen=max_len)\n",
    "print('train_x.shape :', train_x.shape)\n",
    "print('test_x.shape :', test_x.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1D 컨브넷은 5장에서 사용한 2D 컨브넷과 비슷한 방식으로 구성합니다. \n",
    "\n",
    "- `Conv1D`와 `MaxPooling1D` 층을 쌓고 **전역 풀링 층**([`GlobalMaxPooling1D`](https://keras.io/layers/pooling/#globalmaxpooling1d))이나 `Flatten` 층으로 마칩니다. \n",
    "\n",
    "- 이 구조는 **3D 입력을 2D 출력으로 바꾸므로** 분류나 회귀를 위해 모델에 하나 이상의 `Dense` 층을 추가할 수 있습니다.\n",
    "\n",
    "\n",
    "![](./images/GAP_GMP.png)\n",
    "\n",
    "\n",
    "한 가지 다른 점은 **1D 컨브넷에 큰 합성곱 윈도우를 사용할 수 있다는 것**입니다. \n",
    "\n",
    "- 2D 합성곱 층에서 3 × 3 합성곱 윈도우는 3 × 3 = 9 특성을 고려합니다. 하지만 1D 합성곱 층에서 크기 3인 합성곱 윈도우는 3개의 특성만 고려합니다. \n",
    "\n",
    "- 그래서 1D 합성곱에 크기 7이나 9의 윈도우를 사용할 수 있습니다.\n",
    "\n",
    "다음은 IMDB 데이터셋을 위한 1D 컨브넷의 예입니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_1 (Embedding)      (None, 500, 128)          1280000   \n",
      "_________________________________________________________________\n",
      "conv1d_1 (Conv1D)            (None, 494, 32)           28704     \n",
      "_________________________________________________________________\n",
      "max_pooling1d_1 (MaxPooling1 (None, 98, 32)            0         \n",
      "_________________________________________________________________\n",
      "conv1d_2 (Conv1D)            (None, 92, 32)            7200      \n",
      "_________________________________________________________________\n",
      "global_max_pooling1d_1 (Glob (None, 32)                0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 1)                 33        \n",
      "=================================================================\n",
      "Total params: 1,315,937\n",
      "Trainable params: 1,315,937\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Train on 20000 samples, validate on 5000 samples\n",
      "Epoch 1/10\n",
      "20000/20000 [==============================] - 2s 121us/step - loss: 0.8337 - acc: 0.5091 - val_loss: 0.6874 - val_acc: 0.5654\n",
      "Epoch 2/10\n",
      "20000/20000 [==============================] - 1s 51us/step - loss: 0.6699 - acc: 0.6389 - val_loss: 0.6641 - val_acc: 0.6572\n",
      "Epoch 3/10\n",
      "20000/20000 [==============================] - 1s 51us/step - loss: 0.6235 - acc: 0.7522 - val_loss: 0.6081 - val_acc: 0.7438\n",
      "Epoch 4/10\n",
      "20000/20000 [==============================] - 1s 50us/step - loss: 0.5259 - acc: 0.8080 - val_loss: 0.4847 - val_acc: 0.8054\n",
      "Epoch 5/10\n",
      "20000/20000 [==============================] - 1s 52us/step - loss: 0.4113 - acc: 0.8486 - val_loss: 0.4278 - val_acc: 0.8306\n",
      "Epoch 6/10\n",
      "20000/20000 [==============================] - 1s 51us/step - loss: 0.3499 - acc: 0.8681 - val_loss: 0.4148 - val_acc: 0.8362\n",
      "Epoch 7/10\n",
      "20000/20000 [==============================] - 1s 53us/step - loss: 0.3088 - acc: 0.8623 - val_loss: 0.4495 - val_acc: 0.8184\n",
      "Epoch 8/10\n",
      "20000/20000 [==============================] - 1s 51us/step - loss: 0.2795 - acc: 0.8480 - val_loss: 0.4285 - val_acc: 0.8034\n",
      "Epoch 9/10\n",
      "20000/20000 [==============================] - 1s 51us/step - loss: 0.2552 - acc: 0.8275 - val_loss: 0.4532 - val_acc: 0.7826\n",
      "Epoch 10/10\n",
      "20000/20000 [==============================] - 1s 52us/step - loss: 0.2329 - acc: 0.8041 - val_loss: 0.4929 - val_acc: 0.7608\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras import layers\n",
    "from keras.optimizers import RMSprop\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.Embedding(max_features, 128, input_length=max_len))\n",
    "model.add(layers.Conv1D(32, 7, activation='relu'))\n",
    "model.add(layers.MaxPool1D(5))\n",
    "model.add(layers.Conv1D(32, 7, activation='relu'))\n",
    "model.add(layers.GlobalMaxPooling1D())\n",
    "model.add(layers.Dense(1))\n",
    "model.summary()\n",
    "\n",
    "model.compile(optimizer=RMSprop(lr=1e-4),\n",
    "              loss='binary_crossentropy',\n",
    "              metrics=['acc'])\n",
    "history = model.fit(train_x, train_y,\n",
    "                    epochs=10,\n",
    "                    batch_size=128,\n",
    "                    validation_split=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "그림 6-27과 6-28은 훈련과 검증 결과를 보여줍니다. 검증 정확도는 LSTM보다 조금 낮지만 CPU나 GPU에서 더 빠르게 실행됩니다(속도 향상은 환경에 따라 많이 다릅니다). 여기에서 적절한 에포크 수(4개)로 모델을 다시 훈련하고 테스트 세트에서 확인할 수 있습니다. \n",
    "\n",
    "이 예는 단어 수준의 감성 분류 작업에 순환 네트워크를 대신하여 빠르고 경제적인 1D 컨브넷을 사용할 수 있음을 보여줍니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd4VFX6wPHvSxQQBAxNlBiCiPSeBRUEBVHAFWw/BXFXUGEtWLDsouCCIIgV1LWQRdQVhEVXXVSKoCgqFsJKDYJIM4AYioiCQsL7++PcJJOQMoFJ7pT38zzzZG6beefO5L3nnnPuuaKqGGOMiQ3l/A7AGGNM2bGkb4wxMcSSvjHGxBBL+sYYE0Ms6RtjTAyxpG+MMTHEkn4MEpE4EflFRBJDua6fROQMEQl5/2MRuUBENgVMrxWRc4NZ9yjea7KI3H+02xsTjOP8DsAUT0R+CZisBPwOZHnTf1HVaSV5PVXNAk4M9bqxQFUbheJ1RORG4FpVPS/gtW8MxWsbUxRL+hFAVXOSrleSvFFVFxS2vogcp6qZZRGbMcWx32N4seqdKCAiD4nIv0VkuojsA64VkbNF5AsR+UlEtovI0yJyvLf+cSKiIpLkTU/1ls8RkX0i8rmI1C/put7yniKyTkT2isgzIvKZiAwoJO5gYvyLiKwXkT0i8nTAtnEiMkFEdonIBqBHEftnuIjMyDfvWRF50nt+o4is8T7Pd14pvLDXSheR87znlUTkVS+21UC7fOuOEJEN3uuuFpHe3vwWwD+Ac72qs50B+3ZUwPY3eZ99l4i8LSKnBLNvSrKfs+MRkQUisltEfhCRvwa8zwPePvlZRFJF5NSCqtJE5NPs79nbn4u899kNjBCRhiKy0HuPnd5+qxawfT3vM2Z4y58SkYpezE0C1jtFRPaLSI3CPq8phqraI4IewCbggnzzHgIOApfgDuQnAH8AOuDO5k4H1gFDvPWPAxRI8qanAjuBZOB44N/A1KNYtzawD+jjLbsLOAQMKOSzBBPjf4FqQBKwO/uzA0OA1UACUANY5H7OBb7P6cAvQOWA1/4RSPamL/HWEaArcABo6S27ANgU8FrpwHne88eBj4B4oB6Qlm/dq4BTvO/kGi+Gk71lNwIf5YtzKjDKe36hF2NroCLwHPBhMPumhPu5GrADuAOoAFQF2nvL7gOWAw29z9AaqA6ckX9fA59mf8/eZ8sEbgbicL/HM4FuQHnvd/IZ8HjA51nl7c/K3vodvWUpwNiA97kbeMvv/8NIfvgegD1K+IUVnvQ/LGa7e4DXvecFJfIXAtbtDaw6inWvBz4JWCbAdgpJ+kHGeFbA8jeBe7zni3DVXNnLeuVPRPle+wvgGu95T2BtEeu+C9zqPS8q6W8J/C6AWwLXLeB1VwEXe8+LS/qvAOMCllXFteMkFLdvSrif/wQsKWS977LjzTc/mKS/oZgYrsx+X+Bc4AcgroD1OgIbAfGmlwGXh/r/KpYeVr0TPb4PnBCRxiLynne6/jMwGqhZxPY/BDzfT9GNt4Wte2pgHOr+S9MLe5EgYwzqvYDNRcQL8BrQz3t+jTedHccfReRLr+rhJ1wpu6h9le2UomIQkQEistyrovgJaBzk64L7fDmvp6o/A3uAugHrBPWdFbOfT8Ml94IUtaw4+X+PdURkpohs9WJ4OV8Mm9R1GshDVT/DnTV0EpHmQCLw3lHGZLA6/WiSv7viJFzJ8gxVrQr8HVfyLk3bcSVRAEREyJuk8juWGLfjkkW24rqUzgQuEJG6uOqn17wYTwDeAB7GVb2cBLwfZBw/FBaDiJwOPI+r4qjhve43Aa9bXPfSbbgqo+zXq4KrRtoaRFz5FbWfvwcaFLJdYct+9WKqFDCvTr518n++R3C9zlp4MQzIF0M9EYkrJI5/AdfizkpmqurvhaxngmBJP3pVAfYCv3oNYX8pg/d8F2grIpeIyHG4euJapRTjTOBOEanrNer9raiVVfUHXBXEy7iqnW+9RRVw9cwZQJaI/BFX9xxsDPeLyEnirmMYErDsRFziy8Ad/wbhSvrZdgAJgQ2q+UwHbhCRliJSAXdQ+kRVCz1zKkJR+3kWkCgiQ0SkgohUFZH23rLJwEMi0kCc1iJSHXew+wHXYSBORAYTcIAqIoZfgb0ichquiinb58AuYJy4xvETRKRjwPJXcdVB1+AOAOYYWNKPXncD1+EaVifhGlxLlaruAK4GnsT9EzcAvsaV8EId4/PAB8BKYAmutF6c13B19DlVO6r6EzAUeAvXGHol7uAVjJG4M45NwBwCEpKqrgCeAb7y1mkEfBmw7XzgW2CHiARW02RvPxdXDfOWt30i0D/IuPIrdD+r6l6gO3AF7kC0DujiLX4MeBu3n3/GNapW9KrtBgH34xr1z8j32QoyEmiPO/jMAv4TEEMm8EegCa7UvwX3PWQv34T7nn9X1cUl/Owmn+zGEWNCzjtd3wZcqaqf+B2PiVwi8i9c4/Aov2OJdHZxlgkpEemB6ylzANfl7xCutGvMUfHaR/oALfyOJRpY9Y4JtU7ABlxd9kXAZdbwZo6WiDyMu1ZgnKpu8TueaGDVO8YYE0OspG+MMTEk7Or0a9asqUlJSX6HYYwxEWXp0qU7VbWoLtJAGCb9pKQkUlNT/Q7DGGMiiogUd1U6YNU7xhgTUyzpG2NMDLGkb4wxMSTs6vQLcujQIdLT0/ntt9/8DsUUoWLFiiQkJHD88YUNJ2OM8VtEJP309HSqVKlCUlISbuBGE25UlV27dpGenk79+vWL38AY44uIqN757bffqFGjhiX8MCYi1KhRw87GijBtGiQlQbly7u+0Et3O3pjQiIiSPmAJPwLYd1S4adNg8GDYv99Nb97spgH6H+3YmcYchYgo6RsT6YYPz0342fbvd/ONKUuW9IOwa9cuWrduTevWralTpw5169bNmT548GBQrzFw4EDWrl1b5DrPPvss0+ycPyptKWSosMLmlyarZoptEVO9UxLTprkS1JYtkJgIY8ce2yl0jRo1WLZsGQCjRo3ixBNP5J577smzTs5Nh8sVfBx96aWXin2fW2+99eiDNGEtMdFV6RQ0vyxZNZOJupJ+9o9682ZQzf1Rl0ZpZv369TRt2pT+/fvTrFkztm/fzuDBg0lOTqZZs2aMHj06Z91OnTqxbNkyMjMzOemkkxg2bBitWrXi7LPP5scffwRgxIgRTJw4MWf9YcOG0b59exo1asTixe6GQb/++itXXHEFTZs25corryQ5OTnngBRo5MiR/OEPf6B58+bcdNNNZI+mum7dOrp27UqrVq1o27YtmzZtAmDcuHG0aNGCVq1aMdzqHEJu7FioVCnvvEqV3PyyZNVMJuqSfln/qL/55huGDh1KWloadevWZfz48aSmprJ8+XLmz59PWlraEdvs3buXLl26sHz5cs4++2ymTJlS4GurKl999RWPPfZYzgHkmWeeoU6dOqSlpfHAAw/w9ddfF7jtHXfcwZIlS1i5ciV79+5l7ty5APTr14+hQ4eyfPlyFi9eTO3atXnnnXeYM2cOX331FcuXL+fuu+8O0d4x2fr3h5QUqFcPRNzflJSyL12HUzWT8UfUJf2y/lE3aNCA5OTknOnp06fTtm1b2rZty5o1awpM+ieccAI9e/YEoF27djml7fwuv/zyI9b59NNP6du3LwCtWrWiWbNmBW77wQcf0L59e1q1asXHH3/M6tWr2bNnDzt37uSSSy4B3MVUlSpVYsGCBVx//fWccMIJAFSvXr3kO8IUq39/2LQJDh92f/2oTimsOqmsq5mMf6Iu6Zf1j7py5co5z7/99lueeuopPvzwQ1asWEGPHj0K7Ldevnz5nOdxcXFkZmYW+NoVKlQodp2C7N+/nyFDhvDWW2+xYsUKrr/+eus/b4DwqWYy/om6pO/nj/rnn3+mSpUqVK1ale3btzNv3ryQv0fHjh2ZOXMmACtXrizwTOLAgQOUK1eOmjVrsm/fPv7zn/8AEB8fT61atXjnnXcAd9Hb/v376d69O1OmTOHAgQMA7N69O+Rxm/AQLtVMxj9BJX0R6SEia0VkvYgMK2B5oogsFJGvRWSFiPTy5ieJyAERWeY9Xgj1B8jPzx9127Ztadq0KY0bN+bPf/4zHTt2DPl73HbbbWzdupWmTZvy4IMP0rRpU6pVq5ZnnRo1anDdddfRtGlTevbsSYcOHXKWTZs2jSeeeIKWLVvSqVMnMjIy+OMf/0iPHj1ITk6mdevWTJgwIeRxm/ARDtVMxkfZXQ0LewBxwHfA6UB53E2Km+ZbJwW42XveFNjkPU8CVhX3HoGPdu3aaX5paWlHzItVhw4d0gMHDqiq6rp16zQpKUkPHTrkc1S5wvG7mjpVtV49VRH3d+pUvyMy9p2EHpCqQeTYYPrptwfWq+oGABGZAfQBAusVFKjqPa8GbDv6w5Apyi+//EK3bt3IzMxEVZk0aRLHHReVl1uEhPVLDz/2nfhL1Ou/XegKIlcCPVT1Rm/6T0AHVR0SsM4pwPtAPFAZuEBVl4pIErAaWAf8DIxQ1U8KeI/BwGCAxMTEdpvzXcWyZs0amjRpcpQf0ZSlcPuukpIKviiqXj1XtWHKnn0npUNElqpqcnHrhaohtx/wsqomAL2AV0WkHLAdSFTVNsBdwGsiUjX/xqqaoqrJqppcq1ax9/U1JmjWLz382Hfir2CS/lbgtIDpBG9eoBuAmQCq+jlQEaipqr+r6i5v/lJc28CZxxq0McGyfunhx74TfwWT9JcADUWkvoiUB/oCs/KtswXoBiAiTXBJP0NEaolInDf/dKAhsCFUwRtTHOuXHn7sO/FXsUlfVTOBIcA8YA0wU1VXi8hoEentrXY3MEhElgPTgQFea3JnYIWILAPeAG5SVesEbsqM9UsPP/ad+CyYLj5l+QjHLpvnnXeezp07N8+8CRMm6E033VTkdpUrV1ZV1a1bt+oVV1xR4DpdunTRJUuWFPk6EyZM0F9//TVnumfPnrpnz55gQi9zfn9XxsQqguyyGXVX5JaGfv36MWPGjDzzZsyYQb9+/YLa/tRTT+WNN9446vefOHEi+wNGkZs9ezYnnXTSUb+eMSZ2WdIPwpVXXsl7772Xc8OUTZs2sW3bNs4999ycfvNt27alRYsW/Pe//z1i+02bNtG8eXPADZHQt29fmjRpwmWXXZYz9AHAzTffnDMs88iRIwF4+umn2bZtG+effz7nn38+AElJSezcuROAJ598kubNm9O8efOcYZk3bdpEkyZNGDRoEM2aNePCCy/M8z7Z3nnnHTp06ECbNm244IIL2LFjB+CuBRg4cCAtWrSgZcuWOcM4zJ07l7Zt29KqVSu6desWkn0bLQ4dgm3b4H//gzlz4OWXYfx4GDoUrrkGunaFZs2gdm3o3NktW7HCDf9tTFmKuKt67rwTChg+/pi0bg1evixQ9erVad++PXPmzKFPnz7MmDGDq666ChGhYsWKvPXWW1StWpWdO3dy1lln0bt370LvF/v8889TqVIl1qxZw4oVK2jbtm3OsrFjx1K9enWysrLo1q0bK1as4Pbbb+fJJ59k4cKF1KxZM89rLV26lJdeeokvv/wSVaVDhw506dKF+Ph4vv32W6ZPn84///lPrrrqKv7zn/9w7bXX5tm+U6dOfPHFF4gIkydP5tFHH+WJJ55gzJgxVKtWjZUrVwKwZ88eMjIyGDRoEIsWLaJ+/foxMT5PZiZkZMAPP8COHe5R2HPvGHyEypWhTh04+WRo1AjOPtsdGO67zz0SEqBXL/fo1g1OPLFsP6OJPRGX9P2SXcWTnfRffPFFwLWJ3H///SxatIhy5cqxdetWduzYQZ06dQp8nUWLFnH77bcD0LJlS1q2bJmzbObMmaSkpJCZmcn27dtJS0vLszy/Tz/9lMsuuyxnpM/LL7+cTz75hN69e1O/fn1at24NFD58c3p6OldffTXbt2/n4MGD1K9fH4AFCxbkqc6Kj4/nnXfeoXPnzjnrROrwy5mZLkEHm8gLKolXqpSbyM88E8491z3Pnhf4PGAQ1jy2bYO5c+G992D6dNeQWb68Owvo1QsuvhgaNnQNncaEUsQl/aJK5KWpT58+DB06lP/973/s37+fdu3aAW4As4yMDJYuXcrxxx9PUlLSUQ1jvHHjRh5//HGWLFlCfHw8AwYMOKbhkLOHZQY3NHNB1Tu33XYbd911F7179+ajjz5i1KhRR/1+4erQIXj7bZdUly8vPJGfcEJuoj7jDOjYsfBEHorS+KmnwvXXu8fBg/DZZzB7tjsI3HWXezRokHsWcN55ULHisb+vySvUt1aNBBGX9P1y4okncv7553P99dfnacDdu3cvtWvX5vjjj2fhwoXkH0Iiv86dO/Paa6/RtWtXVq1axYoVKwA3LHPlypWpVq0aO3bsYM6cOZx33nkAVKlShX379h1RvXPuuecyYMAAhg0bhqry1ltv8eqrrwb9mfbu3UvdunUBeOWVV3Lmd+/enWeffTanjWDPnj2cddZZ3HLLLWzcuDGneiecS/vff+8S/eTJrvSemAiXXuoSd2GJ3K9SdfnycP757vHYY7Bxo2sXmD3bxf/MM+6g1K1b7kGgXj1/Yo0msToGkCX9EujXrx+XXXZZnqqP/v37c8kll9CiRQuSk5Np3Lhxka9x8803M3DgQJo0aUKTJk1yzhhatWpFmzZtaNy4MaeddlqeYZkHDx5Mjx49OPXUU1m4cGHO/LZt2zJgwADat28PwI033kibNm0KvRNXfqNGjeL//u//iI+Pp2vXrmzcuBFw9+q99dZbad68OXFxcYwcOZLLL7+clJQULr/8cg4fPkzt2rWZP39+UO9TVg4fhvnz4fnn4Z13XIm+Z0+4+Wb3Ny7O7wiDU78+3HKLexw4AB99lHsW8O67bp2mTV0VUK9e7qzk+ON9DTkiFXVr1WhO+sUOuFbWkpOTNTU1Nc+8cBvEyxTOj+9q50546SWYNAm++w5q1YIbbnClNq8JIiqowrp1LvnPng2LFrnqq6pVoXt3dwDo2RNOOcXvSCNDuXIFV/WJuAJEpAl2wDUr6ZuIpAqff+5K9a+/Dr//7hpUx4yByy+HgCaNqCHiegA1auTq/Pftgw8+cAeA2bPB61lL27a51UDt20fOGU5ZS0wseLTPaB8DyPrpm4iybx+88ILrZtuxI/z3v65Uv3KlK/n26xedCb8gVaq4doqUFNeGsWwZjBvneheNGwfnnOPaK669Fl57DXbt8jvi8BKrYwBFTElfVQvt+27CQ2lWFa5c6Ur1U6e6xN+qlUv+/ftb33ZwZwGtWrnHfffB7t3w/vvuDGDOHNdoWa4cdOiQexbQurWbF6uy6+1jrfdORNTpb9y4kSpVqlCjRg1L/GFKVdm1axf79u3L6ct/rH7/3VVZPPec69JYoQJcdZVrmD3rLOvDHqysLEhNza0Gyv73io93VWKdO0OXLu4gYDdhi1zB1ulHRNI/dOgQ6enpx9Rv3ZS+ihUrkpCQwPHH2JVk40bXKDtlirsitkEDuOkmGDgQatQIUbAxbMcOmDcPPv7YVYmtX+/mV6niqsyyDwLJya47qYkMUZX0TfTLynKl0Oefd1eqikDv3q5Uf8EFsV0NUdq2boVPPsk9CKR5d78+4QQ3bET2QaBDBzfPhCdL+sZ3wVzt+MMP8OKLrjFyyxbX3XDQIPdISPAn7liXkZH3ILB8uestVb686w3UpYs7EJxzjrWnhBNL+sZX+a92BNczIiXFjTr58ceuVP/mm248nG7dXKm+d2+70Cjc7Nnj2lSyDwJLl7ozs7g4aNcu9yDQqRPYiN/+saRvfJWUVHAf6Ph4N/TBmjXu+YAB8Je/uL7nJjLs2+eukcg+CHz1lRs/KLsHUfZBoHNnyDdyiClEKMYAsqRvfFXY1Y7gqghuvhmuvtrqiKPBgQPw5Ze5B4HPP3fzwA0X0aVL7oHArhY+UlFnxSVJ/Jb0ja8KK+nXqQPbt5d5OKYMHTzouoVmHwQ+/RR++cUta9gwt2G4c2cbOA4K/1+pVw+CHEYLCD7pB9UnQkR6iMhaEVkvIsMKWJ4oIgtF5GsRWSEivQKW3edtt1ZELgr+I5hINnr0kZf/V6oEjz/uTzym7JQv7xp577vPXRi2Zw8sWeK++yZNXDvOn//skt3pp8M997jlYVb+LDNbtpRs/rEqNumLSBzwLNATaAr0E5Gm+VYbAcxU1TZAX+A5b9um3nQzoAfwnPd6JoplZrp/9qws17An4kotJT1dNdHhuONcn/+773bDZuzc6XoEPf20q/55+mlX5degAQwbBl9/HVsHgMLG+imtMYCCKem3B9ar6gZVPQjMAPrkW0eBqt7zasA273kfYIaq/q6qG4H13uuZKJWZ6UpxM2a4+8Du2eNGLNy0yRK+ccqVg5Yt4bbb3FDRO3a4C/EaNYInnnADxjVqBCNGuOE3ov0AUNZjAAWT9OsC3wdMp3vzAo0CrhWRdGA2cFsJtkVEBotIqoikZmRkBBm6CTfZCX/6dJfw//Y3vyMykSA+3l1tPWeOu24jJcWdGT78sDs4NGsGDz7oenxFo/79cz9zWZwVh+o6x37Ay6qaAPQCXhWRoF9bVVNUNVlVk2vVqhWikExZysyE665zCf/hhy3hm6NTo4a7MG/+fNfg/9xzULu2S/pNm7qDwNix8O23fkcaWv37u7PhsjgrDiYxbwVOC5hO8OYFugGYCaCqnwMVgZpBbmsiXFaWS/ivveaG9B12RFO/MSVXu7br2vvRR26oiKefdjeMGTHC3ZC+bVt45BE3VpMJXjBJfwnQUETqi0h5XMPsrHzrbAG6AYhIE1zSz/DW6ysiFUSkPtAQ+CpUwRv/BSb8sWNdjw1jQu2UU1wbwKefunsHPPmk6yU0bJjrAdS+vWsP+P774l8r1hWb9FU1ExgCzAPW4HrprBaR0SLS21vtbmCQiCwHpgMD1FmNOwNIA+YCt6pqVml8EFP2srLcFbXTpsFDD8H99/sdkYkFCQkwdCh88YUr5T/6qKsWuece1+PlnHPgqadg27biXysW2cVZ5qhkJ/ypU13CHz7c74hMrPvuO5g5E/79b9clVMSNB3T11XDlle4uYtEspBdnGRMoK8v1tpg61d2T1hK+CQcNGrjqxWXL4JtvXOPv7t0wZAiceqob1G/SJHedQCyzpG9KJCsLrr8eXn3VXXU7YoTfERlzpEaN4IEHYNUq9xg+HNLT3c146tSBiy5y1wbs3u13pGXPkr4JWlaWuwn5v/7lSlEPPOB3RMYUr1kzV0D55ht3FvDXv7q7hd1wgzsAXHyx+03v2+d3pGXDkr4JSlYW3HgjvPKKS/h//7vfERlTMtlDP48b55J+airceSesXu16oNWu7er/Z81yg8ZFK0v6pljZCf/ll2HUKEv4JvKJuBvAPPqo6wG0eLEr+X/4IfTp47qI3nSTu4PY4cN+RxtalvRNkQ4fdldIvvwyjBzpHsZEExF3L+B//MN183zvPejRw7Vbde4M9eu7BuJVq/yONDQs6ZtCHT7sSvgvveRK96NG+R2RMaXr+OOhVy937cmOHa6HWrNm8Nhj0KKFqx569NHIvgjMkr4pUHYJ3xK+iVUnnujGwJk9250BPPMMVK7sxpVKTHQ3gklJibweQJb0zREOH3a3b5syxfXQGTXKnQIbE6tq13b9/RcvdheBjRkDP/7o7u9cpw5ceim8/nrubSLDmSV9k0d2wn/xRdcH/8EHLeEbE+j0093/RloaLF3qxgT66iu46ip31e/AgbBggesAEY4s6Zschw+7ksuLL7qLWUaPtoRvTGFE3Eif2QO9LVjghnt4803o3j13jKDU1PC6EYwlfQO4hH/TTTB5skv4Y8ZYwjcmWHFxbpiHKVNcA/Abb7geQc89B3/4g7s38OjR7voAv1nSNxw+7MYt/+c/3UiZlvCNOXoVK8IVV7gS/w8/uP+rU05xbWMNG8JZZ7l7A+zY4U98lvRj3OHDcMstrhfCffe5ETMt4RsTGvHxrtvzwoWwZYvr7vnbb3DHHVC3bu71AGU5BIQl/RiWnfAnTXI3oxg71hK+MaUlIQHuvdeN/7NqlRsD6Jtv3H2lTz4Z+vVzF4aVNkv6MerwYbj11tyEP26cJXxjykqzZu5/bsMGN9TDgAHuvsDjxpX+ex9X+m9hwo2q63P8wgvuQhNL+Mb4o1w5d6OXTp1g4kTXBlDq71n6b2HK2rRpkJTkflBJSW46m6or4T//vDu9fPhhS/jGhIPy5d2VvqUtqKQvIj1EZK2IrBeRYQUsnyAiy7zHOhH5KWBZVsCy/DdUNyE2bZq7uGrzZpfgN29209Om5Zbwn3/e1S2OH28J35hYU2z1jojEAc8C3YF0YImIzFLVtOx1VHVowPq3AW0CXuKAqrYOXcimKMOHw/79eeft3++6Yi5e7PoN33MPPPKIJXxjYlEwJf32wHpV3aCqB4EZQJ8i1u8HTA9FcKbktmwpfH52wn/0UUv4xsSqYJJ+XSBwINF0b94RRKQeUB/4MGB2RRFJFZEvROTSQrYb7K2TmpGREWTopiBF1QnefbclfGNiXagbcvsCb6hq4FBD9VQ1GbgGmCgiDfJvpKopqpqsqsm1atUKcUixZexYqFTpyPk9e7oxwS3hGxPbgkn6W4HTAqYTvHkF6Uu+qh1V3er93QB8RN76fhNi/fu7q2sDS/w9eriLPizhG2OCSfpLgIYiUl9EyuMS+xG9cESkMRAPfB4wL15EKnjPawIdgbT825rQ6t/fDZ4GbpS/2bMt4RtjnGJ776hqpogMAeYBccAUVV0tIqOBVFXNPgD0BWao5hlEtAkwSUQO4w4w4wN7/ZjS8dVX7uYnV1/thn21hG+MySYaTgM9A8nJyZqamup3GBHrl1+gTRs4eBCWL4eTTvI7ImNMWRCRpV77aZFsGIYoc+ed7nZuCxdawjfGHMmGYYgib77p7no1bJi7abMxxuRnST9KbN0KgwZBu3buZg3GGFMQS/pR4PBhNzTrb7+5MXbKl/c7ImNMuLI6/SgwcaK7KfOkSdCokd/RGGPCmZX0I9zy5e42h336uOodY4wpiiX9CHbgAFxzDVSvDpMnW398Y0zxrHongv3zMkWWAAARmElEQVTtb5CWBvPmQc2afkdjjIkEVtKPUHPmwDPPuH75F17odzTGmEhhST8C/fgjDBwILVq42x0aY0ywrHonwqjCDTfATz/B/PlQsaLfERljIokl/QgzaRK8+67rptmihd/RGGMijVXvRJA1a+Cuu+Cii+C22/yOxhgTiSzpR4iDB904+ZUrw0svQTn75owxR8GqdyLEAw/A11/D22/DKaf4HY0xJlJZeTECLFzo7m87eLC78tYYY46WJf0wt3s3/OlP0LAhPPmk39EYYyKdVe+EMVV3r9sdO+CLL1x9vjHGHAtL+mHsX/+C1193F2C1a+d3NMaYaBBU9Y6I9BCRtSKyXkSGFbB8gogs8x7rROSngGXXici33uO6UAYfzb77DoYMgc6d4d57/Y7GGBMtii3pi0gc8CzQHUgHlojILFVNy15HVYcGrH8b0MZ7Xh0YCSQDCiz1tt0T0k8RZTIz4dprIS4OXn3V/TXGmFAIpqTfHlivqhtU9SAwAyiqD0k/YLr3/CJgvqru9hL9fKDHsQQcCx56yNXhv/ACJCb6HY0xJpoEk/TrAt8HTKd7844gIvWA+sCHJdlWRAaLSKqIpGZkZAQTd9RavBjGjHE9dvr29TsaY0y0CXWXzb7AG6qaVZKNVDVFVZNVNblWrVohDily/Pyzq9ZJTIR//MPvaIwx0SiYpL8VOC1gOsGbV5C+5FbtlHTbmHf77bB5M0ydClWr+h2NMSYaBZP0lwANRaS+iJTHJfZZ+VcSkcZAPPB5wOx5wIUiEi8i8cCF3jyTz8yZ8MorMHw4dOzodzTGmGhVbO8dVc0UkSG4ZB0HTFHV1SIyGkhV1ewDQF9ghqpqwLa7RWQM7sABMFpVd4f2I0S+77+Hv/wFOnRwY+wYY0xpkYAcHRaSk5M1NTXV7zDKTFYWdOsGqamwbBmccYbfERljIpGILFXV5OLWsytyffb44/DxxzBliiV8Y0zpswHXfLR0qavOueIKGDDA72iMMbHAkr5P9u93N0WpXRtSUkDE74iMMbHAqnd8cvfdsHYtLFgA1av7HY0xJlZYSd8Hs2a5IRbuucc14hpjTFmxpF/GfvgBbrgBWrd2Y+wYY0xZsqRfhlRh4ED45ReYNg0qVPA7ImNMrLE6/TL0j3/A3Lnub9OmfkdjjIlFVtIvI6tWuZuh9OoFt9zidzTGmFhlSb8M/Pab655Ztaq7CMu6Zxpj/GLVO2Xg/vthxQp49104+WS/ozHGxDIr6Zey+fNhwgRXpXPxxX5HY4yJdZb0S9HOnXDdddCkCTz2mN/RGGOMVe+UGlUYPNgl/tmzoVIlvyMyxhhL+qXmxRfhrbdcCb91a7+jMcYYx6p3SsG6dXDHHdC1K9x1l9/RGGNMLkv6IXbokOueWaGCu/1hOdvDxpgwYtU7ITZunLsL1uuvQ0KC39EYY0xeQZVDRaSHiKwVkfUiMqyQda4SkTQRWS0irwXMzxKRZd7jiBuqR5Nly9wgatdcA1de6Xc0xhhzpGJL+iISBzwLdAfSgSUiMktV0wLWaQjcB3RU1T0iUjvgJQ6oatQ3ZR486Lpn1qgBTz/tdzTGGFOwYKp32gPrVXUDgIjMAPoAaQHrDAKeVdU9AKr6Y6gDDXdjx7qrbt9+2yV+Y4wJR8FU79QFvg+YTvfmBToTOFNEPhORL0SkR8CyiiKS6s2/tKA3EJHB3jqpGRkZJfoA4eDrr11d/rXXQp8+fkdjjDGFC1VD7nFAQ+A8IAFYJCItVPUnoJ6qbhWR04EPRWSlqn4XuLGqpgApAMnJyRqimMpEdrVOzZrw1FN+R2OMMUULpqS/FTgtYDrBmxcoHZilqodUdSOwDncQQFW3en83AB8BbY4x5rAyZgysXOlubm73ujXGhLtgkv4SoKGI1BeR8kBfIH8vnLdxpXxEpCauumeDiMSLSIWA+R3J2xYQ0ZYuhYcfhj/9CS65xN0NKynJ9c1PSnLTxhgTToqt3lHVTBEZAswD4oApqrpaREYDqao6y1t2oYikAVnAvaq6S0TOASaJyGHcAWZ8YK+fSPb77zBgANSu7ap1pk1zY+3s3++Wb97spsFdrGWMMeFAVMOrCj05OVlTU1P9DqNYI0a4HjvvvuuGTE5Kcok+v3r1YNOmso7OGBNrRGSpqiYXt54NEnAUUlNh/HjXgJs9Rv6WLQWvW9h8Y4zxgyX9Esqu1jn5ZJg4MXd+YmLB6xc23xhj/GBJv4QefBBWr4Z//hNOOil3/tixR46ZX6mSm2+MMeHCkn4JLFkCjzwCAwdCr155l/Xv77pt1qvnbnxer56btkZcY0w4sYbcIP32G7RtCz//DKtW5S3lG2OM34JtyLWhlYM0ahSsWQNz5ljCN8ZELqveCcKXX7rbHt5wA/ToUfz6xhgTrizpF+O331xvnVNPhSee8DsaY4w5Nla9U4y//x2++QbmzoVq1fyOxhhjjo2V9IvwxReudD9oEFx0kd/RGGPMsbOkX4gDB1y1TkICPP6439EYY0xoWPVOIf7+d1i7Ft5/H6pW9TsaY4wJDSvpF2DxYletM3gwdO/udzTGGBM6lvTzOXDAXXGbmGjVOsaY6GPVO/mMGAHr1sGCBVClit/RGGNMaFlJP8Bnn8GECXDTTdCtm9/RGGNM6FnS9+zf73rrJCbCo4/6HY0xxpQOq97xDB8O69fDBx9YtY4xJnpZSR/45BN3n9tbboGuXf2OxhhjSk9QSV9EeojIWhFZLyLDClnnKhFJE5HVIvJawPzrRORb73FdqAIPlV9/heuvd/e4feQRv6MxxpjSVWz1jojEAc8C3YF0YImIzFLVtIB1GgL3AR1VdY+I1PbmVwdGAsmAAku9bfeE/qMcnfvvd9U6CxfCiSf6HY0xxpSuYEr67YH1qrpBVQ8CM4A++dYZBDybncxV9Udv/kXAfFXd7S2bD4TN4MSLFsHTT8OQIXDeeX5HY4wxpS+YpF8X+D5gOt2bF+hM4EwR+UxEvhCRHiXYFhEZLCKpIpKakZERfPTH4Ndf3UVYp58O48eXyVsaY4zvQtV75zigIXAekAAsEpEWwW6sqilACrjbJYYopiLddx9s2AAffQSVK5fFOxpjjP+CKelvBU4LmE7w5gVKB2ap6iFV3Qiswx0Egtm2zH30ETzzDNx2G3Tp4nc0xhhTdoJJ+kuAhiJSX0TKA32BWfnWeRtXykdEauKqezYA84ALRSReROKBC715vvnlF9dbp0EDePhhPyMxxpiyV2z1jqpmisgQXLKOA6ao6moRGQ2kquoscpN7GpAF3KuquwBEZAzuwAEwWlV3l8YHCdawYbBpE3z8sVXrGGNij6iWSRV60JKTkzU1NbVUXnvhQnfx1R13wMSJpfIWxhjjCxFZqqrJxa0XM1fkZlfrnHEGjBvndzTGGOOPmBl7569/hc2bXd/8SpX8jsYYY/wREyX9Dz+E55+HO++ETp38jsYYY/wT9Ul/3z5XrdOwITz0kN/RGGOMv6K+eufee2HLFvj0U6vWMcaYqC7pL1gAkybBXXfBOef4HY0xxvgvapP+zz/DDTdAo0YwZozf0RhjTHiI2uqde++F9HRXrXPCCX5HY4wx4SEqS/rvvw8pKXD33XD22X5HY4wx4SPqkv7evXDjjdC4MYwe7Xc0xhgTXqKueueee2DrVli8GCpW9DsaY4wJL1FV0p83DyZPdom/Qwe/ozHGmPATNUk/u1qnSRN48EG/ozHGmPAUNUn/wAFo0QJeftmqdYwxpjBRU6dfpw7Mnu13FMYYE96ipqRvjDGmeJb0jTEmhljSN8aYGBJU0heRHiKyVkTWi8iwApYPEJEMEVnmPW4MWJYVMD//DdWNMcaUoWIbckUkDngW6A6kA0tEZJaqpuVb9d+qOqSAlzigqq2PPVRjjDHHKpiSfntgvapuUNWDwAygT+mGZYwxpjQEk/TrAt8HTKd78/K7QkRWiMgbInJawPyKIpIqIl+IyKUFvYGIDPbWSc3IyAg+emOMMSUSqobcd4AkVW0JzAdeCVhWT1WTgWuAiSLSIP/GqpqiqsmqmlyrVq0QhWSMMSa/YJL+ViCw5J7gzcuhqrtU9XdvcjLQLmDZVu/vBuAjoM0xxGuMMeYYBJP0lwANRaS+iJQH+gJ5euGIyCkBk72BNd78eBGp4D2vCXQE8jcAG2OMKSPF9t5R1UwRGQLMA+KAKaq6WkRGA6mqOgu4XUR6A5nAbmCAt3kTYJKIHMYdYMYX0OvHGGNMGRFV9TuGPJKTkzU1NdXvMIwxJqKIyFKv/bRIdkWuMcbEEEv6xhgTQyzpG2NMDLGkb4wxMcSSvjHGxBBL+sYYE0Ms6RtjTAyxpG+MMTHEkr4xxsQQS/rGGBNDLOkbY0wMsaRvjDExxJK+McbEEEv6xhgTQ6Im6U+bBklJUK6c+zttmt8RGWNM+Cn2JiqRYNo0GDwY9u9305s3u2mA/v39i8sYY8JNVJT0hw/PTfjZ9u93840xxuSKiqS/ZUvJ5htjTKyKiqSfmFiy+cYYE6uCSvoi0kNE1orIehEZVsDyASKSISLLvMeNAcuuE5Fvvcd1oQw+29ixUKlS3nmVKrn5xhhjchXbkCsiccCzQHcgHVgiIrNUNS3fqv9W1SH5tq0OjASSAQWWetvuCUn0nuzG2uHDXZVOYqJL+NaIa4wxeQXTe6c9sF5VNwCIyAygD5A/6RfkImC+qu72tp0P9ACmH124hevf35K8McYUJ5jqnbrA9wHT6d68/K4QkRUi8oaInFaSbUVksIikikhqRkZGkKEbY4wpqVA15L4DJKlqS2A+8EpJNlbVFFVNVtXkWrVqhSgkY4wx+QWT9LcCpwVMJ3jzcqjqLlX93ZucDLQLdltjjDFlJ5ikvwRoKCL1RaQ80BeYFbiCiJwSMNkbWOM9nwdcKCLxIhIPXOjNM8YY44NiG3JVNVNEhuCSdRwwRVVXi8hoIFVVZwG3i0hvIBPYDQzwtt0tImNwBw6A0dmNusYYY8qeqKrfMeQhIhnAZr/jOEY1gZ1+BxFGbH/kZfsjl+2LvI5lf9RT1WIbRcMu6UcDEUlV1WS/4wgXtj/ysv2Ry/ZFXmWxP6JiGAZjjDHBsaRvjDExxJJ+6UjxO4AwY/sjL9sfuWxf5FXq+8Pq9I0xJoZYSd8YY2KIJX1jjIkhlvRDSEROE5GFIpImIqtF5A6/Y/KbiMSJyNci8q7fsfhNRE7yBiT8RkTWiMjZfsfkJxEZ6v2frBKR6SJS0e+YypKITBGRH0VkVcC86iIy37v/yHxvJIOQsqQfWpnA3araFDgLuFVEmvock9/uIHdYjlj3FDBXVRsDrYjh/SIidYHbgWRVbY672r+vv1GVuZdxQ80HGgZ8oKoNgQ+86ZCypB9CqrpdVf/nPd+H+6cuaBjqmCAiCcDFuEH4YpqIVAM6Ay8CqOpBVf3J36h8dxxwgogcB1QCtvkcT5lS1UW4YWsC9SF3lOJXgEtD/b6W9EuJiCQBbYAv/Y3EVxOBvwKH/Q4kDNQHMoCXvOquySJS2e+g/KKqW4HHgS3AdmCvqr7vb1Rh4WRV3e49/wE4OdRvYEm/FIjIicB/gDtV9We/4/GDiPwR+FFVl/odS5g4DmgLPK+qbYBfKYVT90jh1VX3wR0MTwUqi8i1/kYVXtT1pw95n3pL+iEmIsfjEv40VX3T73h81BHoLSKbgBlAVxGZ6m9IvkoH0lU1+8zvDdxBIFZdAGxU1QxVPQS8CZzjc0zhYEf2UPXe3x9D/QaW9ENIRARXZ7tGVZ/0Ox4/qep9qpqgqkm4BroPVTVmS3Kq+gPwvYg08mZ1I7j7TEerLcBZIlLJ+7/pRgw3bAeYBVznPb8O+G+o38CSfmh1BP6EK9Uu8x69/A7KhI3bgGkisgJoDYzzOR7feGc8bwD/A1biclFMDckgItOBz4FGIpIuIjcA44HuIvIt7mxofMjf14ZhMMaY2GElfWOMiSGW9I0xJoZY0jfGmBhiSd8YY2KIJX1jjIkhlvSNMSaGWNI3xpgY8v8dPbyCvkZ4zQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xt4VNXZ/vHvQzjJGYHWCkrwTMKZiFqKgFqLJygWEYwHrEr1raK1tlK1Vmlp0VKl+uPVWqu2EkGqtWI90L6VFk9VAiqKgCAGDaACCqJoMeT5/bEmySTmMISZ7GTm/lxXrszs2bP3mgncs2btZ69t7o6IiKSXZlE3QEREkk/hLiKShhTuIiJpSOEuIpKGFO4iImlI4S4ikoYU7lItM8sys0/M7MBkrhslMzvEzJJe+2tmJ5hZUdz91WY2LJF167Gvu83smvo+v5bt/sLM7kv2diU6zaNugCSHmX0Sd7cN8F9gd+z+99y9YE+25+67gXbJXjcTuPvhydiOmV0InO3uI+K2fWEyti3pT+GeJty9PFxjPcML3f3/alrfzJq7e0lDtE1EGp6GZTJE7Gv3g2Y218x2AGeb2TFm9h8z22Zmm8zsNjNrEVu/uZm5mWXH7s+JPf6kme0wsxfMrNeerht7/CQze9PMtpvZ7Wb2nJlNqqHdibTxe2a21sw+MrPb4p6bZWa3mtlWM1sHjKrl/bnWzOZVWTbbzG6J3b7QzFbGXs9bsV51TdsqNrMRsdttzOz+WNtWAIOrrHudma2LbXeFmY2OLe8L/D9gWGzIa0vce3tD3PMvjr32rWb2VzP7WiLvTV3MbGysPdvM7GkzOzzusWvMbKOZfWxmq+Je69Fmtiy2/H0z+3Wi+5MUcHf9pNkPUAScUGXZL4BdwGmED/V9gCOBowjf4A4C3gQuja3fHHAgO3Z/DrAFyANaAA8Cc+qx7leAHcCY2GNXAl8Ak2p4LYm08VGgI5ANfFj22oFLgRVAD6ALsDj8k692PwcBnwBt47b9AZAXu39abB0DjgM+A/rFHjsBKIrbVjEwInZ7JvAvoDPQE3ijyrrjga/F/iZnxdrw1dhjFwL/qtLOOcANsdsnxto4AGgN/C/wdCLvTTWv/xfAfbHbvWPtOC72N7oGWB27nQusB/aLrdsLOCh2ewkwMXa7PXBU1P8XMvlHPffM8qy7P+bupe7+mbsvcfcX3b3E3dcBdwHDa3n+Q+5e6O5fAAWEUNnTdU8FXnH3R2OP3Ur4IKhWgm38lbtvd/ciQpCW7Ws8cKu7F7v7VmBGLftZB7xO+NAB+CbwkbsXxh5/zN3XefA08E+g2oOmVYwHfuHuH7n7ekJvPH6/8919U+xv8gDhgzkvge0C5AN3u/sr7v45MBUYbmY94tap6b2pzQRggbs/HfsbzSB8QBwFlBA+SHJjQ3tvx947CB/Sh5pZF3ff4e4vJvg6JAUU7pnl3fg7ZnaEmT1uZu+Z2cfANKBrLc9/L+72Tmo/iFrTuvvHt8PdndDTrVaCbUxoX4QeZ20eACbGbp8Vu1/WjlPN7EUz+9DMthF6zbW9V2W+VlsbzGySmb0aG/7YBhyR4HYhvL7y7bn7x8BHQPe4dfbkb1bTdksJf6Pu7r4a+CHh7/BBbJhvv9iq5wM5wGoze8nMTk7wdUgKKNwzS9UywN8RequHuHsH4HrCsEMqbSIMkwBgZkblMKpqb9q4CTgg7n5dpZrzgRPMrDuhB/9ArI37AA8BvyIMmXQC/p5gO96rqQ1mdhBwB3AJ0CW23VVx262rbHMjYainbHvtCcM/GxJo155stxnhb7YBwN3nuPtQwpBMFuF9wd1Xu/sEwtDbb4CHzaz1XrZF6knhntnaA9uBT82sN/C9Btjn34BBZnaamTUHLge6paiN84ErzKy7mXUBrq5tZXd/D3gWuA9Y7e5rYg+1AloCm4HdZnYqcPwetOEaM+tk4TyAS+Mea0cI8M2Ez7mLCD33Mu8DPcoOIFdjLnCBmfUzs1aEkH3G3Wv8JrQHbR5tZiNi+/4R4TjJi2bW28xGxvb3WeynlPACzjGzrrGe/vbYayvdy7ZIPSncM9sPgfMI/3F/RzjwmVLu/j5wJnALsBU4GHiZUJef7DbeQRgbf41wsO+hBJ7zAOEAafmQjLtvA34APEI4KDmO8CGViJ8RvkEUAU8Cf4rb7nLgduCl2DqHA/Hj1P8A1gDvm1n88ErZ858iDI88Env+gYRx+L3i7isI7/kdhA+eUcDo2Ph7K+BmwnGS9wjfFK6NPfVkYKWFaqyZwJnuvmtv2yP1Y2HIUyQaZpZFGAYY5+7PRN0ekXShnrs0ODMbFRumaAX8lFBl8VLEzRJJKwp3icI3gHWEr/zfAsa6e03DMiJSDxqWERFJQ+q5i4ikocgmDuvatatnZ2dHtXsRkSZp6dKlW9y9tvJhIMJwz87OprCwMKrdi4g0SWZW15nWgIZlRETSksJdRCQNKdxFRNKQrsQkkiG++OILiouL+fzzz6NuiiSgdevW9OjRgxYtappaqHYKd5EMUVxcTPv27cnOziZMximNlbuzdetWiouL6dWrV91PqEaTGpYpKIDsbGjWLPwu2KNLPotkts8//5wuXboo2JsAM6NLly579S2ryfTcCwpg8mTYuTPcX78+3AfI3+t58EQyg4K96djbv1WT6blfe21FsJfZuTMsFxGRyppMuL/zzp4tF5HGZevWrQwYMIABAwaw33770b179/L7u3YlNu37+eefz+rVq2tdZ/bs2RQkacz2G9/4Bq+88kpSttXQmsywzIEHhqGY6paLSPIVFIRvxu+8E/6fTZ++d0OgXbp0KQ/KG264gXbt2nHVVVdVWsfdcXeaNau+33nvvffWuZ/vf//79W9kGmkyPffp06FNm8rL2rQJy0UkucqOca1fD+4Vx7hSUcSwdu1acnJyyM/PJzc3l02bNjF58mTy8vLIzc1l2rRp5euW9aRLSkro1KkTU6dOpX///hxzzDF88MEHAFx33XXMmjWrfP2pU6cyZMgQDj/8cJ5//nkAPv30U77zne+Qk5PDuHHjyMvLq7OHPmfOHPr27UufPn245pprACgpKeGcc84pX37bbbcBcOutt5KTk0O/fv04++yzk/6eJaLJ9NzLegzJ7EmISPVqO8aViv9zq1at4k9/+hN5eXkAzJgxg3333ZeSkhJGjhzJuHHjyMnJqfSc7du3M3z4cGbMmMGVV17JPffcw9SpU7+0bXfnpZdeYsGCBUybNo2nnnqK22+/nf3224+HH36YV199lUGDBtXavuLiYq677joKCwvp2LEjJ5xwAn/729/o1q0bW7Zs4bXXXgNg27ZtANx8882sX7+eli1bli9raE2m5w7hH1VREZSWht8KdpHUaOhjXAcffHB5sAPMnTuXQYMGMWjQIFauXMkbb7zxpefss88+nHTSSQAMHjyYoqKiard9+umnf2mdZ599lgkTJgDQv39/cnNza23fiy++yHHHHUfXrl1p0aIFZ511FosXL+aQQw5h9erVTJkyhYULF9KxY0cAcnNzOfvssykoKKj3SUh7q0mFu4g0jJqOZaXqGFfbtm3Lb69Zs4bf/va3PP300yxfvpxRo0ZVW+/dsmXL8ttZWVmUlJRUu+1WrVrVuU59denSheXLlzNs2DBmz57N9773PQAWLlzIxRdfzJIlSxgyZAi7d+9O6n4ToXAXkS+J8hjXxx9/TPv27enQoQObNm1i4cKFSd/H0KFDmT9/PgCvvfZatd8M4h111FEsWrSIrVu3UlJSwrx58xg+fDibN2/G3TnjjDOYNm0ay5YtY/fu3RQXF3Pcccdx8803s2XLFnZWHeNqAE1mzF1EGk6Ux7gGDRpETk4ORxxxBD179mTo0KFJ38dll13GueeeS05OTvlP2ZBKdXr06MHPf/5zRowYgbtz2mmnccopp7Bs2TIuuOAC3B0z46abbqKkpISzzjqLHTt2UFpaylVXXUX79u2T/hrqktA1VM1sFPBbIAu4291nVHn8QOCPQKfYOlPd/YnatpmXl+e6WIdIw1m5ciW9e/eOuhmNQklJCSUlJbRu3Zo1a9Zw4oknsmbNGpo3b1z93er+Zma21N3zanhKuTpfiZllAbOBbwLFwBIzW+Du8d9jrgPmu/sdZpYDPAFkJ/4SREQazieffMLxxx9PSUkJ7s7vfve7RhfseyuRVzMEWOvu6wDMbB4wBogPdwc6xG53BDYms5EiIsnUqVMnli5dGnUzUiqRA6rdgXfj7hfHlsW7ATjbzIoJvfbLqtuQmU02s0IzK9y8eXM9misiIolIVrXMROA+d+8BnAzcb2Zf2ra73+Xuee6e161bnRfvFhGRekok3DcAB8Td7xFbFu8CYD6Au78AtAa6JqOBIiKy5xIJ9yXAoWbWy8xaAhOABVXWeQc4HsDMehPCXeMuIiIRqTPc3b0EuBRYCKwkVMWsMLNpZjY6ttoPgYvM7FVgLjDJE6mxFJGMMXLkyC+dkDRr1iwuueSSWp/Xrl07ADZu3Mi4ceOqXWfEiBHUVVo9a9asSicTnXzyyUmZ9+WGG25g5syZe72dZEtozN3dn3D3w9z9YHefHlt2vbsviN1+w92Hunt/dx/g7n9PZaNFpOmZOHEi8+bNq7Rs3rx5TJw4MaHn77///jz00EP13n/VcH/iiSfo1KlTvbfX2Gn6ARFpEOPGjePxxx8vvzBHUVERGzduZNiwYeV154MGDaJv3748+uijX3p+UVERffr0AeCzzz5jwoQJ9O7dm7Fjx/LZZ5+Vr3fJJZeUTxf8s5/9DIDbbruNjRs3MnLkSEaOHAlAdnY2W7ZsAeCWW26hT58+9OnTp3y64KKiInr37s1FF11Ebm4uJ554YqX9VOeVV17h6KOPpl+/fowdO5aPPvqofP9lUwCXTVj273//u/xiJQMHDmTHjh31fm+rk15V+yKSkCuugGRfYGjAAIjlYrX23XdfhgwZwpNPPsmYMWOYN28e48ePx8xo3bo1jzzyCB06dGDLli0cffTRjB49usbriN5xxx20adOGlStXsnz58kpT9k6fPp19992X3bt3c/zxx7N8+XKmTJnCLbfcwqJFi+jatXKtx9KlS7n33nt58cUXcXeOOuoohg8fTufOnVmzZg1z587l97//PePHj+fhhx+udX72c889l9tvv53hw4dz/fXXc+ONNzJr1ixmzJjB22+/TatWrcqHgmbOnMns2bMZOnQon3zyCa1bt96Dd7tu6rmLSIOJH5qJH5Jxd6655hr69evHCSecwIYNG3j//fdr3M7ixYvLQ7Zfv37069ev/LH58+czaNAgBg4cyIoVK+qcFOzZZ59l7NixtG3blnbt2nH66afzzDPPANCrVy8GDBgA1D6tMIT55bdt28bw4cMBOO+881i8eHF5G/Pz85kzZ075mbBDhw7lyiuv5LbbbmPbtm1JP0NWPXeRDFRbDzuVxowZww9+8AOWLVvGzp07GTx4MAAFBQVs3ryZpUuX0qJFC7Kzs6ud5rcub7/9NjNnzmTJkiV07tyZSZMm1Ws7ZcqmC4YwZXBdwzI1efzxx1m8eDGPPfYY06dP57XXXmPq1KmccsopPPHEEwwdOpSFCxdyxBFH1LutVannLiINpl27dowcOZLvfve7lQ6kbt++na985Su0aNGCRYsWsb66CybHOfbYY3nggQcAeP3111m+fDkQpgtu27YtHTt25P333+fJJ58sf0779u2rHdceNmwYf/3rX9m5cyeffvopjzzyCMOGDdvj19axY0c6d+5c3uu///77GT58OKWlpbz77ruMHDmSm266ie3bt/PJJ5/w1ltv0bdvX66++mqOPPJIVq1atcf7rI167iLSoCZOnMjYsWMrVc7k5+dz2mmn0bdvX/Ly8urswV5yySWcf/759O7dm969e5d/A+jfvz8DBw7kiCOO4IADDqg0XfDkyZMZNWoU+++/P4sWLSpfPmjQICZNmsSQIUMAuPDCCxk4cGCtQzA1+eMf/8jFF1/Mzp07Oeigg7j33nvZvXs3Z599Ntu3b8fdmTJlCp06deKnP/0pixYtolmzZuTm5pZfVSpZEpryNxU05a9Iw9KUv03P3kz5q2EZEZE0pHAXEUlDCneRDKJZQZqOvf1bKdxFMkTr1q3ZunWrAr4JcHe2bt26Vyc2qVpGJEP06NGD4uJidKGcpqF169b06NGj3s9XuItkiBYtWtCrV6+omyENRMMyIiJpSOEuIpKGFO4iImlI4S4ikoYU7iIiaUjhLiKShhTuIiJpSOEuIpKGFO4iImlI4S4ikoYU7iIiaSihcDezUWa22szWmtnUah6/1cxeif28aWbbkt9UERFJVJ0Th5lZFjAb+CZQDCwxswXu/kbZOu7+g7j1LwMGpqCtIiKSoER67kOAte6+zt13AfOAMbWsPxGYm4zGiYhI/SQS7t2Bd+PuF8eWfYmZ9QR6AU/X8PhkMys0s0LNKS0ikjrJPqA6AXjI3XdX96C73+Xuee6e161btyTvWkREyiQS7huAA+Lu94gtq84ENCQjIhK5RMJ9CXComfUys5aEAF9QdSUzOwLoDLyQ3CZW9vrrMGcOvP026FKQIiLVq7Naxt1LzOxSYCGQBdzj7ivMbBpQ6O5lQT8BmOcpvvruQw/BjTeG2/vtB1//esXPoEHQqlUq9y4i0jRYVFdCz8vL88LCwj1+3u7doff+/PMVP+vWhcdatoS8vIqwP+aY8AEgIpIuzGypu+fVuV5TC/fqvPcevPBCRdgXFsKuXeGxgw6q3Lvv0weyspKyWxGRBpdR4V7Vf/8Ly5ZVhP1zz8H774fH2rWDo4+uCPujjoJOnfZs+wUFcO218M47cOCBMH065Ocn/3WIiFSV0eFelTsUFVUeylm+HEpLwQxycyv37g85JCyvTkEBTJ4MO3dWLGvTBu66SwEvIqmncK/Djh3w0ksVYf/CC7B9e3isa9fKYZ+XB/vsEx7Lzob167+8vZ49wweIiEgqJRrudVbLpKv27eH448MPhF78ypWVe/cLYnVALVrAwIEh6KsLdghDNCIijUXGhntVzZqF4ZncXLjoorBs8+bKB2rvvLPm5x94YMO0U0QkEZrPvRbdusHo0TBjBixeHIZtbrwx9OTjNW8eDrCKiDQWCvc90LIlXH893HtvGGOHcDC1pASuuQZuu62iBFNEJEoK93rIzw8HT93h009DXX2/fnD55dC7N8yfr6kRRCRaCvckGDwY/u//4MknoW1bOPPMUD//739H3TIRyVQK9yQxg1Gj4OWX4b77YNMmGDECTjsNVqyIunUikmkU7kmWlQXnnQdvvhkOxD7zTBiyufBC2FDTRMkiIkmmcE+RffaBq6+Gt94KY/H33w+HHhqqaspOlhIRSRWFe4p16QK33AKrVsHYsfDLX8LBB6uyRkRSS+HeQHr1CvPSFBZC//6qrBGR1FK4N7CaKmv+9a+oWyYi6UThHoHqKmtGjoRTT92zypqCgjCRWbNm4XdBQYoaLCJNjsI9QvGVNTfdBM8+GyprLrig7sqasqmH168Pwzrr14f7CngRAYV7o7DPPvDjH4fKmiuuCBcAr6uy5tprK88pD+G+5rgREVC4NypdusBvfgOrV8Ppp9deWVPTFMOaelhEQOHeKGVnh957YSEMGFBRWfPggxWVNTVNMayph0UEFO6N2uDB8I9/wFNPhWu/TphQUVkzfXqYkTJemzZhuYiIwr2RM4NvfStc8Pu+++C990Jlzdy5Yfrhnj3DOj176jquIlJB4d5ElFXWrF5dUVlzzTXhMoEbNoQpiBXsIlImoXA3s1FmttrM1prZ1BrWGW9mb5jZCjN7ILnNlDLVVdYceyx8+GHULRORxqTOcDezLGA2cBKQA0w0s5wq6xwK/AQY6u65wBUpaKvEKausWbQoVMiccQZ88UXUrRKRxiKRnvsQYK27r3P3XcA8YEyVdS4CZrv7RwDu/kFymyk1+frX4e674emnYcoUzVMjIkEi4d4deDfufnFsWbzDgMPM7Dkz+4+ZjapuQ2Y22cwKzaxw8+bN9WuxfMk554Tphe+8E2bPjro1ItIYJOuAanPgUGAEMBH4vZl1qrqSu9/l7nnuntetW7ck7VognPA0enQYh//736NujYhELZFw3wAcEHe/R2xZvGJggbt/4e5vA28Swl4aSLNm4eBqbi6MHx/mjxeRzJVIuC8BDjWzXmbWEpgALKiyzl8JvXbMrCthmGZdEtspCWjfHhYsgFatwrVbVUEjkrnqDHd3LwEuBRYCK4H57r7CzKaZ2ejYaguBrWb2BrAI+JG7b01Vo6VmPXvCI4+ogkYk05lHVF6Rl5fnhYWFkew7E/zpT+Gkp4svhv/933AWq4g0fWa21N3z6lqveUM0RhreuefCG2+Es1lzc+HSS6NukYg0JE0/kMbiK2j+8Y+oWyMiDUnhnsbKKmhycsL4uypoRDKHwj3NtW8Pjz0GLVuqgkYkkyjcM4AqaEQyj8I9QwwdCr//fZiD5vLLo26NiKSaqmUyyLnnwooVcPPNoYLm+9+PukUikirquWeYX/4yjL1ffrkqaETSmcI9w2RlQUFBRQXN6tVRt0hEUkHhnoFUQSOS/hTuGaqsgmb9+jCLpCpoRNKLwj2DDR0Kd90F//ynKmhE0o2qZTLceeeFOWhUQSOSXtRzF1XQiKQhhbuogkYkDSncBai4ipMqaETSg8JdymVnw1/+sucVNAUF4bnNmoXfBQUpbKSIJEThLpV84xsVFTRXXFH3+gUFMHly+EBwD78nT1bAi0RN4S5fct558KMfhcvzzZ5d+7rXXgs7d1ZetnNnWC4i0VG4S7V+9Ss49dS6K2jeeWfPlotIw1C4S7WysuCBB6B37zD+/uab1a934IF7tlxEGobCXWpUNgdNixahF//RR19eZ/p0aNOm8rI2bcJyEYmOwl1qVVZBU1RU/VWc8vPDAdiePcEs/L7rrrBcRKKjcJc61VVBk58fwr+0NPxWsItEL6FwN7NRZrbazNaa2dRqHp9kZpvN7JXYz4XJb6pEadIkuOqqxCpoRCR6dU4cZmZZwGzgm0AxsMTMFrj7G1VWfdDdL01BG6WRmDEDVq0KFTSHHQbf/GbULRKRmiTScx8CrHX3de6+C5gHjElts6QxSrSCRkSil0i4dwfejbtfHFtW1XfMbLmZPWRmB1S3ITObbGaFZla4efPmejRXolZWQdO8eZiDproKGhGJXrIOqD4GZLt7P+AfwB+rW8nd73L3PHfP69atW5J2LQ0tOztcxentt6uvoBGR6n36Kdx3X5imI9USCfcNQHxPvEdsWTl33+ru/43dvRsYnJzmSWO1p3PQiGQqd/jPf+Cii2C//eD88+HPf079fhO5EtMS4FAz60UI9QnAWfErmNnX3H1T7O5oYGVSWymN0qRJsGIFzJwZruL0P/8TdYtEGo/Nm+H+++EPfwhXO2vTJnzTveCC0DlKtTrD3d1LzOxSYCGQBdzj7ivMbBpQ6O4LgClmNhooAT4EJqWwzdKIlFXQTJkS5oL/7nfD1L8imWj3bli4EO65J1wf4Ysv4KijwrfcM8+EDh0ari3m7g23tzh5eXleWFgYyb4luT7+GE4+GZ57DgYMCD3544+PulUiDWfduhDo990HGzZA165wzjmhs9OnT3L3ZWZL3T2vrvXUx5K91qEDLF4cyiQ/+ghOOCGE/euvR90ykdT57LNw3YLjjoODDw4zqfbrF8bTN2yAW25JfrDvCYW7JEWzZjBxYhii+fWv4fnnoX//cOGOTZvqfr5IU+AOS5eG40tf+xqcfXaYcuPnPw8VME88AePGhSHKqCncJalatw7TFLz1Flx2Gdx7Lxx6KNx4YygDE2mKPvwQbr8dBg6EvLzw7/rUU0O12Nq1cN110KNH1K2sTOEuKdGlC8yaBStXwkknwQ03hJD/wx/CQSeRxq60NFyoZsKE0EufMiWcpT17NmzcCHPmhCGZxlpA0EibJenikEPCGORzz4XpgC+8MBx0feqp8BVXpLF5553wTfOgg+DEE+Hvfw/Diy+/XDEk07lz1K2sm8JdGsTXvx7G4f/853CN1ZNOgm99C159NeqWicB//wsPPhj+TWZnV3zTnDs39NJvvz10SpoShbs0GLNwsOmNN+DWW6GwMIxhnn8+FBdH3TrJRMuXh1lO998/DL+sXAk//WmYWqNsSKZ166hbWT8Kd2lwrVqFKQveegt++MNQQnnYYeE/1Y4dUbdO0t22bXDnnXDkkaGi6847Q/nuwoUh1G+8MfTemzqFu0Smc+dQNrlqFYwZA7/4RRijv/NOKCmJunWSTj79NIT3OeeEg6OXXBKGYmbNCsMuDz4YxtezsqJuafLoDFVpNF56KZRRPvMMHHFECP5TTgnDObLndu8Oxzfat4+6JQ3vww/h2WfDv6XFi2HZstBh6NABzjorzO8yeHDT/LeV6BmqiUwcJtIghgyBf/8bHn0Urr46zBc/YkSYzmCw5hlNSGlpqEx68EF46CF4//1w9mReXvg58kgYNCj9An/jxoogX7y44uzoli3Dv6sf/QiOPTb8tGkTbVsbinru0ih98UWYbOmGG2DLlnAm4PTpcOCBUbes8SktDVPKlgX6xo2wzz7hW0+/fqEiacmSUOIHobd6+OEh6MsCv3//phN67uF4TVmYP/NMuA/Qtm2ozCoL8iOPDO9FOkm0565wl0Zt+3a46aZQXeMeDsT+5CfQsWPULYuWexjGevDBUF5aXBwOVJ98crgE4qmnQrt2lZ/zwQehTnvJklCptGQJvPdeeCwrK0zbHB/4ffs2jtPoS0vD1NJlQb54ccWUFl26hOlzjz0Whg0L1VfN03w8QuEuaeWdd8Ip3vffH/5D/+xncPHF0KJF1C1rOGXzmpQF+vr1IXxHjQqBftppezalrHvo5ZcFfdnvDz8Mj7dsGXr+ZYGflwc5OakPzy++CGPkZUH+7LMVl3Ps3r2iVz5sWLieb2M9QzRVFO6SlpYtCwddFy0KJ5ncdBN8+9tN88BYItzhlVdCoM+fH0r1WrQIlR3jx8Po0dCpU3L3V1RUOfCXLg3TOkMY4hg4sHLgH3bY3gXszp3w4otMJEHFAAAJnElEQVQVYf7CC2EZhL9xfJhnZ6fv3zpRCndJW+5h9r0f/zicEDV0KPzmN+GiCOnAHV57LYT5gw+GiamyskIt9plnhg+zhjz9vbQU1qwJQV8W+suWhSlvIRycHTy4cuD36lVzCG/fHg76lh38LCwMvXWz8E2hLMiHDQuXpZPKFO6S9kpKwgUSrr8+VIWceSb88pdhTpCmaMWKEOjz54fa/2bNwsRU48fD2LHhAhCNRUlJOJszPvBffRV27QqP77tv5QqdkpKKMfNXXw0fYC1ahMfLwnzo0OR+C0lXCnfJGDt2hJr4mTNDuBx8cPj6np0depBlt7Oz4atfbVxf61etqgj0FStCoA8fHgL99NPhK1+JuoWJ27UrlCDGj9+//nrFLKBt2sAxx1SE+VFHNZ0KncZE4S4ZZ8MGuOMOePPNMG5cVBQuUhyvdevKYV/1A6Bbt9SH/5o1FYG+fHnY37BhIdC/8530Gor47LPQU2/WLIzVZ9IB8FRRuIsAn3wSqkrKwv7ttytuFxXB1q2V12/Tpvbw79KlfuG/bl1FoL/8clg2dGgI9HHjwsRVIonQGaqScQoK4NprQ9nkgQeGk57y80P9dm5u9c/5+OOK8K8a/M8/HyaZite2bfXDPWX3O3euCP+iolCyOH9+GKYAOProcG3NM85ofFfukfSicJe0UFAQLqhQVkK3fn24DyHga9KhQzhZp2/f6h/ftq3m8F+8uKJEsEz79iHkmzev6KEfeWQ4JnDGGeGCJSINQcMykhays0MIV9WzZwjiVHAP4V/dkM/HH4cLkpxxRtOt3pHGScMyklHK5k1JdHkymIVhmM6dw8FCkcYkofPKzGyUma02s7VmNrWW9b5jZm5mdX6qiCRTTROKaaIxyVR1hruZZQGzgZOAHGCimeVUs1574HLgxWQ3UqQu06d/uWa6TZuwXCQTJdJzHwKsdfd17r4LmAeMqWa9nwM3AZ8nsX0iCcnPD1ME9+wZhkt69gz3azuYKpLOEgn37sC7cfeLY8vKmdkg4AB3f7y2DZnZZDMrNLPCzVXPLhHZS/n54WBmaWn4rWCXTLbXk2WaWTPgFuCHda3r7ne5e56753Xr1m1vdy0iIjVIJNw3AAfE3e8RW1amPdAH+JeZFQFHAwt0UFVEJDqJhPsS4FAz62VmLYEJwIKyB919u7t3dfdsd88G/gOMdncVsYuIRKTOcHf3EuBSYCGwEpjv7ivMbJqZjU51A0VEZM8ldBKTuz8BPFFl2fU1rDti75slIiJ7I8OuPigikhkU7iIiaUjhLiKShhTuIiJpSOEuIpKGFO4iImlI4S4ikoYU7iJJVFAQrgrVrFn4XVAQdYskU+lKTCJJUt/ruIqkgnruIkly7bUVwV5m586wXKShKdxFkiSK67iK1EThLpIkuo6rNCYKd5Ek0XVcpTFRuIskia7jKo2JqmVEkig/X2EujYN67iIiaUjhLiKShhTuIiJpSOEuIpKGFO4iImlI4S4ikoYU7iJpSLNTiurcRdKMZqcUUM9dJO1odkqBBMPdzEaZ2WozW2tmU6t5/GIze83MXjGzZ80sJ/lNFZFEaHZKgQTC3cyygNnASUAOMLGa8H7A3fu6+wDgZuCWpLdURBKi2SkFEuu5DwHWuvs6d98FzAPGxK/g7h/H3W0LePKaKCJ7QrNTCiQW7t2Bd+PuF8eWVWJm3zeztwg99ynVbcjMJptZoZkVbt68uT7tFZE6aHZKgSQeUHX32e5+MHA1cF0N69zl7nnuntetW7dk7VpEqsjPh6IiKC0NvxXsmSeRcN8AHBB3v0dsWU3mAd/em0aJiMjeSSTclwCHmlkvM2sJTAAWxK9gZofG3T0FWJO8JoqIyJ6qM9zdvQS4FFgIrATmu/sKM5tmZqNjq11qZivM7BXgSuC8lLVYRJoMnSkbHXOPprAlLy/PCwsLI9m3iKRe1TNlIVTt6ODu3jGzpe6eV9d6OkNVRFJCZ8pGS+EuIimhM2WjpXAXkZTQmbLRUriLSEroTNloKdxFJCV0pmy0NJ+7iKRMfr7CPCrquYtI2svEenv13EUkrWXqlanUcxeRtJap9fYKdxFJa5lab69wF5G0lqn19gp3EUlrmVpvr3AXkbSWqfX2CncRSXuN5cpUDVmSqVJIEZEG0NAlmeq5i4g0gIYuyVS4i4g0gIYuyVS4i4g0gIYuyVS4i4g0gIYuyVS4i4g0gIYuyVS1jIhIA2nIKZDVcxcRSUMKdxGRNKRwFxFJQwp3EZE0pHAXEUlD5u7R7NhsM7A+kp0nT1dgS9SNaET0flTQe1GZ3o/K9ub96Onu3epaKbJwTwdmVujueVG3o7HQ+1FB70Vlej8qa4j3Q8MyIiJpSOEuIpKGFO57566oG9DI6P2ooPeiMr0flaX8/dCYu4hIGlLPXUQkDSncRUTSkMK9HszsADNbZGZvmNkKM7s86jZFzcyyzOxlM/tb1G2Jmpl1MrOHzGyVma00s2OiblOUzOwHsf8nr5vZXDNrHXWbGoqZ3WNmH5jZ63HL9jWzf5jZmtjvzqnYt8K9fkqAH7p7DnA08H0zy4m4TVG7HFgZdSMaid8CT7n7EUB/Mvh9MbPuwBQgz937AFnAhGhb1aDuA0ZVWTYV+Ke7Hwr8M3Y/6RTu9eDum9x9Wez2DsJ/3u7Rtio6ZtYDOAW4O+q2RM3MOgLHAn8AcPdd7r4t2lZFrjmwj5k1B9oAGyNuT4Nx98XAh1UWjwH+GLv9R+Dbqdi3wn0vmVk2MBB4MdqWRGoW8GOgNOqGNAK9gM3AvbFhqrvNrG3UjYqKu28AZgLvAJuA7e7+92hbFbmvuvum2O33gK+mYicK971gZu2Ah4Er3P3jqNsTBTM7FfjA3ZdG3ZZGojkwCLjD3QcCn5Kir91NQWw8eQzhQ29/oK2ZnR1tqxoPD7XoKalHV7jXk5m1IAR7gbv/Jer2RGgoMNrMioB5wHFmNifaJkWqGCh297Jvcg8Rwj5TnQC87e6b3f0L4C/A1yNuU9TeN7OvAcR+f5CKnSjc68HMjDCmutLdb4m6PVFy95+4ew93zyYcKHva3TO2Z+bu7wHvmtnhsUXHA29E2KSovQMcbWZtYv9vjieDDzDHLADOi90+D3g0FTtRuNfPUOAcQi/1ldjPyVE3ShqNy4ACM1sODAB+GXF7IhP7BvMQsAx4jZA5GTMVgZnNBV4ADjezYjO7AJgBfNPM1hC+2cxIyb41/YCISPpRz11EJA0p3EVE0pDCXUQkDSncRUTSkMJdRCQNKdxFRNKQwl1EJA39fyNsA4YWRazfAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "acc = history.history['acc']\n",
    "val_acc = history.history['val_acc']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(acc) + 1)\n",
    "\n",
    "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.legend()\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.4.4 CNN과 RNN을 연결하여 긴 시퀀스를 처리하기\n",
    "\n",
    "1D 컨브넷이 입력 패치를 독립적으로 처리하기 때문에 RNN과 달리 (합성곱 윈도우 크기의 범위를 넘어선) 타임스텝의 순서에 민감하지 않습니다. 물론 장기간 패턴을 인식하기 위해 많은 합성곱 층과 풀링 층을 쌓을 수 있습니다. 상위 층은 원본 입력에서 긴 범위를 보게 될 것입니다. 이런 방법은 순서를 감지하기엔 부족합니다. \n",
    "\n",
    "온도 예측 문제에 1D 컨브넷을 적용하여 이를 확인해 보겠습니다. 이 문제는 순서를 감지해야 좋은 예측을 만들어 낼 수 있습니다. 다음은 이전에 정의한 float_data, train_gen, val_gen, val_steps를 다시 사용합니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "\n",
    "data_dir = '../datasets/jena_climate/'\n",
    "fname = os.path.join(data_dir, 'jena_climate_2009_2016.csv')\n",
    "\n",
    "f = open(fname)\n",
    "data = f.read()\n",
    "f.close()\n",
    "\n",
    "lines = data.split('\\n')\n",
    "header = lines[0].split(',')\n",
    "lines = lines[1:]\n",
    "\n",
    "float_data = np.zeros((len(lines), len(header) - 1))\n",
    "for i, line in enumerate(lines):\n",
    "    values = [float(x) for x in line.split(',')[1:]]\n",
    "    float_data[i, :] = values\n",
    "    \n",
    "mean = float_data[:200000].mean(axis=0)\n",
    "float_data -= mean\n",
    "std = float_data[:200000].std(axis=0)\n",
    "float_data /= std\n",
    "\n",
    "def generator(data, lookback, delay, min_index, max_index,\n",
    "              shuffle=False, batch_size=128, step=6):\n",
    "    if max_index is None:\n",
    "        max_index = len(data) - delay - 1\n",
    "    i = min_index + lookback\n",
    "    while 1:\n",
    "        if shuffle:\n",
    "            rows = np.random.randint(\n",
    "                min_index + lookback, max_index, size=batch_size)\n",
    "        else:\n",
    "            if i + batch_size >= max_index:\n",
    "                i = min_index + lookback\n",
    "            rows = np.arange(i, min(i + batch_size, max_index))\n",
    "            i += len(rows)\n",
    "\n",
    "        samples = np.zeros((len(rows),\n",
    "                           lookback // step,\n",
    "                           data.shape[-1]))\n",
    "        targets = np.zeros((len(rows),))\n",
    "        for j, row in enumerate(rows):\n",
    "            indices = range(rows[j] - lookback, rows[j], step)\n",
    "            samples[j] = data[indices]\n",
    "            targets[j] = data[rows[j] + delay][1]\n",
    "        yield samples, targets\n",
    "        \n",
    "lookback = 1440\n",
    "step = 6\n",
    "delay = 144\n",
    "batch_size = 128\n",
    "\n",
    "train_gen = generator(float_data,\n",
    "                      lookback=lookback,\n",
    "                      delay=delay,\n",
    "                      min_index=0,\n",
    "                      max_index=200000,\n",
    "                      shuffle=True,\n",
    "                      step=step, \n",
    "                      batch_size=batch_size)\n",
    "val_gen = generator(float_data,\n",
    "                    lookback=lookback,\n",
    "                    delay=delay,\n",
    "                    min_index=200001,\n",
    "                    max_index=300000,\n",
    "                    step=step,\n",
    "                    batch_size=batch_size)\n",
    "test_gen = generator(float_data,\n",
    "                     lookback=lookback,\n",
    "                     delay=delay,\n",
    "                     min_index=300001,\n",
    "                     max_index=None,\n",
    "                     step=step,\n",
    "                     batch_size=batch_size)\n",
    "\n",
    "# 전체 검증 세트를 순회하기 위해 val_gen에서 추출할 횟수\n",
    "val_steps = (300000 - 200001 - lookback) // batch_size\n",
    "\n",
    "# 전체 테스트 세트를 순회하기 위해 test_gen에서 추출할 횟수\n",
    "test_steps = (len(float_data) - 300001 - lookback) // batch_size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv1d_1 (Conv1D)            (None, None, 32)          2272      \n",
      "_________________________________________________________________\n",
      "max_pooling1d_1 (MaxPooling1 (None, None, 32)          0         \n",
      "_________________________________________________________________\n",
      "conv1d_2 (Conv1D)            (None, None, 32)          5152      \n",
      "_________________________________________________________________\n",
      "max_pooling1d_2 (MaxPooling1 (None, None, 32)          0         \n",
      "_________________________________________________________________\n",
      "conv1d_3 (Conv1D)            (None, None, 32)          5152      \n",
      "_________________________________________________________________\n",
      "global_max_pooling1d_1 (Glob (None, 32)                0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 1)                 33        \n",
      "=================================================================\n",
      "Total params: 12,609\n",
      "Trainable params: 12,609\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Epoch 1/20\n",
      "500/500 [==============================] - 6s 12ms/step - loss: 0.4201 - val_loss: 0.4404\n",
      "Epoch 2/20\n",
      "500/500 [==============================] - 6s 12ms/step - loss: 0.3641 - val_loss: 0.4432\n",
      "Epoch 3/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.3427 - val_loss: 0.4461\n",
      "Epoch 4/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.3238 - val_loss: 0.4448\n",
      "Epoch 5/20\n",
      "500/500 [==============================] - 6s 12ms/step - loss: 0.3121 - val_loss: 0.4543\n",
      "Epoch 6/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.3043 - val_loss: 0.4445\n",
      "Epoch 7/20\n",
      "500/500 [==============================] - 6s 12ms/step - loss: 0.2950 - val_loss: 0.4745\n",
      "Epoch 8/20\n",
      "500/500 [==============================] - 6s 12ms/step - loss: 0.2897 - val_loss: 0.4788\n",
      "Epoch 9/20\n",
      "500/500 [==============================] - 6s 12ms/step - loss: 0.2832 - val_loss: 0.4560\n",
      "Epoch 10/20\n",
      "500/500 [==============================] - 6s 12ms/step - loss: 0.2775 - val_loss: 0.4641\n",
      "Epoch 11/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.2748 - val_loss: 0.4544\n",
      "Epoch 12/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.2692 - val_loss: 0.5006\n",
      "Epoch 13/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.2651 - val_loss: 0.4566\n",
      "Epoch 14/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.2613 - val_loss: 0.4522\n",
      "Epoch 15/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.2592 - val_loss: 0.4604\n",
      "Epoch 16/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.2565 - val_loss: 0.4775\n",
      "Epoch 17/20\n",
      "500/500 [==============================] - 6s 12ms/step - loss: 0.2519 - val_loss: 0.4842\n",
      "Epoch 18/20\n",
      "500/500 [==============================] - 6s 12ms/step - loss: 0.2510 - val_loss: 0.4404\n",
      "Epoch 19/20\n",
      "500/500 [==============================] - 6s 12ms/step - loss: 0.2488 - val_loss: 0.4508\n",
      "Epoch 20/20\n",
      "500/500 [==============================] - 6s 11ms/step - loss: 0.2474 - val_loss: 0.4615\n"
     ]
    }
   ],
   "source": [
    "from keras import backend as K\n",
    "from keras.models import Sequential\n",
    "from keras import layers\n",
    "from keras.optimizers import RMSprop\n",
    "\n",
    "K.clear_session()\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.Conv1D(32, 5, activation='relu',\n",
    "                        input_shape=(None, float_data.shape[-1])))\n",
    "model.add(layers.MaxPooling1D(3))\n",
    "model.add(layers.Conv1D(32, 5, activation='relu'))\n",
    "model.add(layers.MaxPooling1D(3))\n",
    "model.add(layers.Conv1D(32, 5, activation='relu'))\n",
    "model.add(layers.GlobalMaxPooling1D())\n",
    "model.add(layers.Dense(1))\n",
    "model.summary()\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae')\n",
    "history = model.fit_generator(train_gen,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=20,\n",
    "                              validation_data=val_gen,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "다음은 훈련 MAE와 검증 MAE입니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl4FFX28PHvCfsmKKAoS4LIKKssEWEQAdcoCgIuaBjcEV+RcdRR3EbF4RkRRh0QEUbRcQgio4MyLoMbKow/kAAhQBBZRA0iBpA1ICQ57x+3Ap3YSTrpTi/p83meftJddavqdKX79K1bt26JqmKMMSY+JEQ6AGOMMeFjSd8YY+KIJX1jjIkjlvSNMSaOWNI3xpg4YknfGGPiiCV9Uy4iUk1E9otIq1CWjSQROU1EQt53WUQuEJEtPq/Xi0ifQMpWYFsvisiDFV2+lPX+WUReCfV6TeRUj3QApnKJyH6fl3WBX4B87/VtqppWnvWpaj5QP9Rl44Gqnh6K9YjILcBwVe3ns+5bQrFuU/VZ0q/iVPVo0vVqkreo6kcllReR6qqaF47YjDHhZ807cc47fH9dRF4TkX3AcBHpJSJLRGS3iGwTkckiUsMrX11EVESSvNezvPnvi8g+Efk/EWld3rLe/EtE5GsR2SMiU0TkfyJyQwlxBxLjbSKyUUR+FpHJPstWE5FnRGSniGwGUkrZPw+JyJxi06aKyNPe81tEZJ33fjZ5tfCS1pUtIv2853VF5J9ebGuB7sXKPiwim731rhWRgd70TsBzQB+v6WyHz759zGf5Ud573ykib4nIyYHsm7KIyGAvnt0i8omInO4z70ER+UFE9orIVz7vtaeIrPCmbxeRiYFuz1QCVbVHnDyALcAFxab9GTgMXI6rBNQBzgLOxh0Jngp8DYz2ylcHFEjyXs8CdgDJQA3gdWBWBcqeCOwDBnnz7gaOADeU8F4CifFtoCGQBOwqfO/AaGAt0AJoDHzuvgp+t3MqsB+o57Pun4Bk7/XlXhkBzgMOAp29eRcAW3zWlQ30855PAj4FjgcSgaxiZa8GTvb+J9d5MZzkzbsF+LRYnLOAx7znF3kxdgFqA88DnwSyb/y8/z8Dr3jP23lxnOf9jx4E1nvPOwDfAs28sq2BU73ny4BrvecNgLMj/V2I54fV9A3AYlX9j6oWqOpBVV2mqktVNU9VNwMzgL6lLP+Gqqar6hEgDZdsylv2MiBDVd/25j2D+4HwK8AY/6Kqe1R1Cy7BFm7rauAZVc1W1Z3Ak6VsZzOwBvdjBHAh8LOqpnvz/6Oqm9X5BPgY8HuytpirgT+r6s+q+i2u9u673bmqus37n8zG/WAnB7BegFTgRVXNUNVDwFigr4i08ClT0r4pzTBgvqp+4v2PnsT9cJwN5OF+YDp4TYTfePsO3I93WxFprKr7VHVpgO/DVAJL+gbge98XInKGiLwrIj+KyF5gHNCklOV/9HmeS+knb0sqe4pvHKqquJqxXwHGGNC2cDXU0swGrvWeX+e9LozjMhFZKiK7RGQ3rpZd2r4qdHJpMYjIDSKyymtG2Q2cEeB6wb2/o+tT1b3Az0BznzLl+Z+VtN4C3P+ouaquB+7B/R9+8poLm3lFbwTaA+tF5EsRuTTA92EqgSV9A+5w39d0XO32NFU9DvgTrvmiMm3DNbcAICJC0SRVXDAxbgNa+rwuq0vpXOACEWmOq/HP9mKsA7wB/AXX9NII+CDAOH4sKQYRORWYBtwONPbW+5XPesvqXvoDrsmocH0NcM1IWwOIqzzrTcD9z7YCqOosVe2Na9qphtsvqOp6VR2Ga8L7K/CmiNQOMhZTQZb0jT8NgD3AARFpB9wWhm2+A3QTkctFpDrwe6BpJcU4F7hLRJqLSGPg/tIKq+qPwGLgFWC9qm7wZtUCagI5QL6IXAacX44YHhSRRuKuYxjtM68+LrHn4H7/bsXV9AttB1oUnrj24zXgZhHpLCK1cMl3kaqWeORUjpgHikg/b9t/xJ2HWSoi7USkv7e9g96jAPcGficiTbwjgz3eeysIMhZTQZb0jT/3ANfjvtDTcSdcK5WqbgeuAZ4GdgJtgJW46wpCHeM0XNv7atxJxjcCWGY27sTs0aYdVd0N/AGYhzsZeiXuxysQj+KOOLYA7wOv+qw3E5gCfOmVOR3wbQf/ENgAbBcR32aawuX/i2tmmect3wrXzh8UVV2L2+fTcD9IKcBAr32/FvAU7jzMj7gji4e8RS8F1onrHTYJuEZVDwcbj6kYcU2nxkQXEamGa064UlUXRToeY6oKq+mbqCEiKV5zRy3gEVyvjy8jHJYxVYolfRNNzgE245oOLgYGq2pJzTvGmAqw5h1jjIkjVtM3xpg4EnUDrjVp0kSTkpIiHYYxxsSU5cuX71DV0ro5A1GY9JOSkkhPT490GMYYE1NEpKwrywFr3jHGmLhiSd8YY+KIJX1jjIkjlvSNMSaOWNI3xpg4YknfGGPiSEBJ3xsTZb13T82xfubfICI5IpLhPW7xmXe9iGzwHteHMnhjjDHlU2Y/fW+0w6m428RlA8tEZL6qZhUr+rqqji627Am4IWSTcWNoL/eW/Tkk0RtThbz3HrRoAZ07RzoSU5UFUtPvAWz07gN6GJjDsfuFluVi4ENV3eUl+g9xY3AbY3z8+CMMHgypqWDDYZnKFEjSb07Re3lm4/82dkNFJFNE3hCRwtvABbSsiIwUkXQRSc/JyQkwdGOqjmnT4PBhWLPG1fiNqSyhOpH7HyBJVTvjavP/KM/CqjpDVZNVNblp0zKHjjCmSjl4EJ5/HlJSoGVLmDAh0hGZqiyQpL+VojdwPnoj5EKqutNn3PMXge6BLmtMvEtLgx074L774J57YNEi+OKLSEdlqqpAkv4yoK2ItBaRmsAwYL5vARE52eflQGCd93wBcJGIHC8ixwMXedOMMbj2+6efhi5doF8/uOUWOOEEq+2bylNm0lfVPGA0LlmvA+aq6loRGSciA71iY0RkrYisAsYAN3jL7gKewP1wLAPGedOMMcCCBbBuHdx9N4hAvXowejTMnw9ZxfvHGRMCUXfnrOTkZLWhlU28uPhiWL0atmyBmjXdtB07oFUruOYaePnliIZnYoiILFfV5LLK2RW5xkTImjXwwQeuZl+Y8AGaNHHNPLNmwfffl7y8MRVhSd+YCHn2WahTB2677dfz7r7btfc/80z44zJVmyV9YyJg+3ZXk7/+emjc+Nfzk5Lg2mthxgzYZWfBTAhZ0jcmAqZNg19+gbvuKrnMfffBgQMwdWr44jJVnyV9Y8Ls0CF3MdaAAXD66SWX69QJLr0UJk+G3NzwxWeqNkv6xoRZWhrk5Lh2+7KMHet681gvnpIVFMD//gf33guvvRbpaKKfddk0JoxUXQ2+WjXIyHB988sq37s3bNsGGzZA9TLHxY0P+fmweDG88Qa8+abbPwDHHQfffQcNG0Y2vkiwLpvGRKEPP4S1a49djFUWEbj/ftePf+7cSg8vquXlwUcfwahRcMop7grml16CXr1g9mz47DPYuxdeeCHSkUY3q+kbE0aXXOJq+Fu2QK1agS1TUAAdO7q+/CtXBvZjUVUcPgyffOJq9G+9BTt3uquWL7sMrrzS7c969Y6VT0k5tn9r145Y2BFhNX0TM44cgRUr4NVXITs70tFUnrVr4b//hTvuCDzhAyQkuJ48q1a5YRuqul9+gXfegRtugJNOcol97lyX0OfNc+dD5sxxSd834YM7B7J9O/yjXOP8xher6ZuwKiiAr7+GZcuOPVaudF90cEML/9//QXN/d2yIcbfeeuwq2yZNyrfs4cPQpo17fPpppYQXcYsXu6aZ//zHNdM0agSDBrnkfuGFgf1QqkLPnu7k9/r18XUOJNCafhztEhNuqq7m/uWXxxJ8err7QgPUrQvdu7ua71lnuS/51Ve7Gt3nn8Pxx0c2/lDKyYF//tNdjFXehA+uaefuu91j6VI4++zQxxgpqvC3v7lhpRs1gquucon+vPOKDk8RCBFX2x8yxJ3gveaayok5lllN34TMnj2ulu6b5Ldvd/Nq1HD3fj3rLPfo0QPatXO9WHx99JHrm96zpxuXpqq0y44bB48+6kbObNeuYuvYv98NxNavH/z73yENL2IOH3Y/+i++6BL1q6/+usmmvAoKoH17N8TFihXxcw4k0Jo+qhpVj+7du6uJPdnZqk2bqoKqiGq7dqojRqhOmaK6ZInqwYOBr+u119x6hgxRzcurvJjD5eBB1RNPVL3kkuDX9cgjbv+uWxf8uiItJ0f13HPd//qRR1Tz80O37pkz3Xr/+9/QrTPaAekaQI6NeJIv/rCkH3sKClQvvli1bl3Vd99V3bMn+HU+84z7dN5+u1t/LCtMQB9+GPy6fvpJtU4d1ZtuCn5dkbRmjWrr1qq1a7sf+VD75RfV5s1V+/cP/bqjVaBJ33rvmKBNn+56lUyc6Jpmjjsu+HXedZfrsTJtGowfH/z6IqVwpMxOneD884NfX9OmcNNN7vzA1hi98ei777q+9YcOub71w4aFfhs1a7pzBAsXunMgxkcgvwzhfFhNP7Zs3Khar57qhReGvkaen6/6u9+5WvLf/x7adYfLhx+6+GfODN06N29WrVZN9Z57QrfOcCgoUJ040TVPdeum+v33lbu9fftUjz9edfDgyt1OtMBq+qay5ee73ijVq8PMmaE/YZaQ4K64TElxY87Pn1/2MtHm6addX/PrrgvdOlu3dr1Spk+Hn38O3Xor0y+/wM03wx//6HrmLFoELVpU7jbr13c3qJk3z92S0jiW9E2FPf20G+hqypTK+wLXqAH/+pfr2nnNNfDFF5Wzncqwbh28/z78v/9XvouxAnHffa43z7RpoV1vZfjpJ7jgAjdo3KOPugur6tYNz7bvvNP14pk4MTzbiwmBHA6E82HNO7Fh9WrVmjXdoXM4TrT+9JNq27bucD0rq/K3FwojR6rWquVirwyXXOJ6TOXmVs76QyEzUzUx0Z2wff31yMRw552qNWqofvddZLYfLljzjqkshw/DiBFuJMPp08PTD7ppU3eyuFYtdzPxaB+uYccO1+d8xAgXe2W4/3530dcrr1TO+oM1fz789rdumI1Fi9yFd5Fwzz2u777detKxpG/K7c9/dkMnzJhReQnNn9at4b33YPduNx5LqNqzs7PhiSfcRVMpKa7HhwZ5zeILL7jeKaXdGStY557rrsydNMmNQBktVGHCBLjiCrdPly2D5LIvGao0iYnunIrdetITyOFAOB/WvBPdvvzS9RwZMSJyMXz8sTtc79On4k0bR46ovv226mWXqSYkuB42ffuqnnSSe3722arz5lXsgqFDh9x6UlIqFlt5zJvn4q2Mvu4VcfCg+2yA6rBh0dP0tHq1i2ncuEhHUrL8fHfBWkVhF2eZUMvNVT3jDNUWLVR//jmyscyZ47r+DR5cvqt2N21SffBB1ZNPdp/+Zs1UH3jAdT1Vde/x+efdhUPgrix+5RXVw4cD38Yrr7hlFywo33uqiPx89z/p0iXyF7H9+KNqr17Hkmuk4ynusstUGzdW3b8/0pEUVVCg+s47qmeeqdqvX8X3myV9E3J33aUhu7I0FP72NxfPqFGlf1EOHXI/Euef78onJKgOGKD61lslJ/MjR1TT0lQ7dXLLtGqlOnmy6oEDpcdUUKDaubNqhw7hS3ovvRS+Hxl/Vqxw/4MGDdxV2W+8EZk4yrJ4sdtPkydHOpJjPv1U9be/dXG1aeM+c5b0TVRYuNB9Wu64I9KRFHX//VriYXtWlurdd6s2aXIscT/+ePl6cRTWws45x62jSRPVJ55Q3bXLf/mPPnLlXnqpYu+nIg4dckMOnHde+La5d6/qjBmqycnu/daurXr99a4ZJZqdc477HJTnyK0ypKe7oUtA9ZRTVF94IfiYLOmbkNmzx3W7O+206Dw0LmxDnjHD1cRfeUW1d283rXp11aFD3cBbwQ7etmiRO0IA1fr1Ve+9V3Xr1qJlBgxwg6uVZ4C5UJg0ycW1ZEnlbmf5ctXbbnPvH1Q7dnSD6pX0Ixht3nnHxf3qq5HZflaW+zyCa2qaNCl05z0s6ZuQuflm1yTyxReRjsS/w4ddn/WEBNXjjnOf6rZtVSdMcO3MobZqlep117nt1aypeuutqhs2uJEvQfWxx0K/zbLs3euuYUhIcE1St9zihq7IzAz+x27vXtXp01W7d3fvr04d1RtucJ+HaGu3L0tBgfuh6tAhtKN6luWbb9w+S0hwP5iPPhqagQl9WdKPM0eOuF4tob4AZf589ykZOza06w21/ftVr7hCNTXVtZOGIxlt2uTasmvVcl/mpCT3fPv2yt+2P2vWuGSSkqJ6wgnu/wZubKS+fVXvu0/1zTcDG/OmoEB12TL3g1avnltPp06qzz0X+ZP4wZo1y72f+fMrf1vbtqmOHu16m9Wq5cZLCqaHTmks6ceJwt4mp57q/ps1aria+YYNwa87J8d1Pezc2bUbG/+2bXPnFho0cF/waFBQoPr11y7B3Xmnao8e7qik8IfglFNcz6cnn1T95BNXm1d1tc9p01S7dnXl6tZ1wzgvWRJ7tfqSHDnimit/+9vK28auXa5XWN26rovzbbdV/gBzIU36QAqwHtgIjC2l3FBAgWTvdRJwEMjwHi+UtS1L+oHZudOdUCy8cUmPHu4LPnq0O6mWkOD6SWdmVmz9BQWqV13lfkQyMkIbe1V15Eh4mwzK69Ah1aVLXe+V1FR3jqbwRyAhQbV9e5ekwHUffP551d27Ix115Zgyxb3PRYtCu959+1THj1dt2NB1Kb7uutBUwAIRsqQPVAM2AacCNYFVQHs/5RoAnwNLiiX9NYEEUviwpF+6b791XScLD7kvvfTXzRnbtrlD+cKTbQMHlv8E3+zZbtnx40Mbv4kuO3aovv++Ow8xYIBrzvnyy6pTqy/JgQOuJ9aAAaFZ38GDrgvxiSce+86tWhWadQcqlEm/F7DA5/UDwAN+yj0LDAA+taQfeqtXu7Hlq1d3h4vDh5ddi9+503VRLGzfPf98dyhf1hd661Z3UrBnT1d7NaYqeuIJ970IJjkfOqQ6darrMgvu4qpIdXgIZdK/EnjR5/XvgOeKlekGvOk9L570DwArgc+APiVsYySQDqS3atUqLDsoFhQUqH72mavNF7av/v73qlu2lG89+/a5rmHNmrn19Oyp+p//+E/+BQXuRGCdOqrr14fmfRgTjXbtckfDqanlX/aXX1yPppYt3XfqnHNchSqSwpb0cYO2fQok6a+Tfi2gsfe8O/A9cFxp27OavmsX/ve/XXIuvCBo3Dh3KB6MgwddO21ioltv587uSlXfLn3Tp7t5U6YEty1jYsE997gj582bAyt/+LC78C4p6VgF6oMPoqM5LNCkL65syUSkF/CYql7svX4AQFX/4r1u6LX57/cWaQbsAgaqanqxdX0K3Ft8uq/k5GRNTy9xdsSpws6dsGkTbNzo/hY+37IFqlWDRo3csMO+fwOZJuLufTpxInz9tRtV8t574YYbQnvTiSNH4LXX4C9/ga++gt/8BsaOdcPgdu8OPXvCBx+4O1cZU5Vt3eq+ZyNHwnPPlVwuLw9mz4Zx49z3PTnZPU9JCc/Q4oEQkeWqWuZ4poEk/erA18D5wFZgGXCdqq4tofyneIldRJoCu1Q1X0ROBRYBnVS1xAFOoyHpFxTADz8UTey+CX7PnqLlW7SA005zH56CAjd/9+5f/y0oKH27CQmuTNeubqz0oUPdrQgr833Om+duPL5ypdt+/fqwZg20bFl52zUmmtx8s0vo334LJ55YdF5+Prz+Ojz+uKuIde3qnl92WfQk+0KBJv0yU4qq5onIaGABrifPTFVdKyLjcIcTpd259FxgnIgcAQqAUaUl/HA6csTVzH1r6oXPN292Y6EXql7dJfQ2baBXL5fg27Rxj9at3e3YyqLqbm9X0g/Cnj2wbx+cdx6cf354PlAJCe6HZcgQd4OS556Dm26yhG/iyx//6G7lOGWKu68CuArRG2/AY4+521527uwqSIMGRV+yL68ya/rhFsqa/oED/pP6pk3uV9235l237rFE7pvUTzvNJcHKrHEbYyJr6FD45BOXFz76yN3Ld80aaN/e1eyHDIn+5s6Q1fRjxe7dMHly0SS/fXvRMo0bu0TesycMH34ssbdpA82axf4vuDGmYu6/H/79b1fJy8mB0093572uusqdp6tKqkzSF3GHYs2buyR+2WVFk3qbNu6EqTHGFNejh2u6WbcO/vpXd3vFqpbsC1WZpN+wIeTmQu3akY7EGBOL3nor0hGER5S3UpWPJXxjjCldlUr6xhhjSmdJ3xhj4oglfWOMiSOW9I0xJo5Y0jfGmDhiSd8YY+KIJX1jjIkjlvSNMSaOWNI3xpg4YknfGGPiSJVJ+mlpkJTkhj9NSnKvjTHGFFUlBlxLS3O3O8vNda+//da9BkhNjVxcxhgTbapETf+hh44l/EK5uW66McaYY6pE0v/uu/JNN8aYeFUlkn6rVuWbbowx8apKJP3x4909bn3VreumG2OMOaZKJP3UVJgxAxIT3W0TExPdazuJa4wxRVWJ3jvgErwleWOMKV2VqOkbY4wJjCV9Y4yJI5b0jTEmjljSN8aYOGJJ3xhj4oglfWOMiSOW9I0xJo5Y0jfGmDgSUNIXkRQRWS8iG0VkbCnlhoqIikiyz7QHvOXWi8jFoQjaGGNMxZR5Ra6IVAOmAhcC2cAyEZmvqlnFyjUAfg8s9ZnWHhgGdABOAT4Skd+oan7o3oIxxphABVLT7wFsVNXNqnoYmAMM8lPuCWACcMhn2iBgjqr+oqrfABu99RljjImAQJJ+c+B7n9fZ3rSjRKQb0FJV3y3vst7yI0UkXUTSc3JyAgrcGGNM+QV9IldEEoCngXsqug5VnaGqyaqa3LRp02BDMsYYU4JARtncCrT0ed3Cm1aoAdAR+FREAJoB80VkYADLGmOMCaNAavrLgLYi0lpEauJOzM4vnKmqe1S1iaomqWoSsAQYqKrpXrlhIlJLRFoDbYEvQ/4ujDHGBKTMmr6q5onIaGABUA2YqaprRWQckK6q80tZdq2IzAWygDzgDuu5Y4wxkSOqGukYikhOTtb09PRIh2GMMTFFRJaranJZ5eyKXGOMiSOW9I0xJo5Y0jfGmDhiSd8YY+KIJX1jjIkjlvSNMSaOWNI3xpg4YknfGGPiiCV9Y4yJI5b0jTEmjljSN8aYOGJJ3xhj4oglfWOMiSOW9I0xJo5Y0jfGmDhiSd8YY+KIJX1jjIkjlvSNMSaOWNI3xpg4Yknfk5YGSUmQkOD+pqVFOiJjjAm96pEOIBqkpcHIkZCb615/+617DZCaGrm4jDEm1KymDzz00LGEXyg31003xpiqxJI+8N135ZtujDGxypI+0KpV+aYbY0yssqQPjB8PdesWnVa3rptujDFViSV93MnaGTMgMRFE3N8ZM+wkrjGm6rHeO57UVEvyxpiqz2r6xhgTRyzpG2NMHLGkb4wxcSSgpC8iKSKyXkQ2ishYP/NHichqEckQkcUi0t6bniQiB73pGSLyQqjfgDHGmMCVeSJXRKoBU4ELgWxgmYjMV9Usn2KzVfUFr/xA4GkgxZu3SVW7hDZsY4wxFRFITb8HsFFVN6vqYWAOMMi3gKru9XlZD9DQhWiMMSZUAkn6zYHvfV5ne9OKEJE7RGQT8BQwxmdWaxFZKSKfiUgffxsQkZEiki4i6Tk5OeUI3xhjTHmE7ESuqk5V1TbA/cDD3uRtQCtV7QrcDcwWkeP8LDtDVZNVNblp06ahCskYY0wxgST9rUBLn9ctvGklmQNcAaCqv6jqTu/5cmAT8JuKhWqMMSZYgST9ZUBbEWktIjWBYcB83wIi0tbn5QBggze9qXciGBE5FWgLbA5F4NHGbsJijIkFZfbeUdU8ERkNLACqATNVda2IjAPSVXU+MFpELgCOAD8D13uLnwuME5EjQAEwSlV3VcYbiSS7CYsxJlaIanR1tElOTtb09PRIh1EuSUku0ReXmAhbtoQ7GmNMPBKR5aqaXFY5uyI3BOwmLMaYWGFJPwTsJizGmFhhST8E7CYsxphYYUk/BOwmLMaYWGE3UQkRuwmLMSYWWE3fGGPiiCV9Y4yJI5b0jTEmjljSN8aYOGJJ3xhj4oglfWOMiSOW9KOEjdJpjAkH66cfBWyUTmNMuFhNPwo89NCxhF8oN9dNN8aYULKkHwVslE5jTLhY0o8CNkqnMSZcLOlHARul0xgTLpb0o4CN0mmMCRfrvRMlbJROY0w4WE3fGGPiiCX9KsIu7jLGBMKad6oAu7jLGBMoq+lXAXZxlzEmUJb0qwC7uMsYEyhL+lWAXdxljAmUJf0qwC7uMsYEypJ+FWAXdxljAmVJv4pITYUtW6CgwP0tb8K3Lp/GxAfrsmmsy6cxccRq+sa6fBoTRwJK+iKSIiLrRWSjiIz1M3+UiKwWkQwRWSwi7X3mPeAtt15ELg5l8CY0rMunMfGjzKQvItWAqcAlQHvgWt+k7pmtqp1UtQvwFPC0t2x7YBjQAUgBnvfWZ6KIdfk0Jn4EUtPvAWxU1c2qehiYAwzyLaCqe31e1gPUez4ImKOqv6jqN8BGb30miliXT2PiRyBJvznwvc/rbG9aESJyh4hswtX0x5Rz2ZEiki4i6Tk5OYHGbkLEunwaEz9CdiJXVaeqahvgfuDhci47Q1WTVTW5adOmoQrJlIN1+TQmPgTSZXMr0NLndQtvWknmANMquKyJQdbl05jYEUhNfxnQVkRai0hN3InZ+b4FRKStz8sBwAbv+XxgmIjUEpHWQFvgy+DDNtHEunwaEzvKrOmrap6IjAYWANWAmaq6VkTGAemqOh8YLSIXAEeAn4HrvWXXishcIAvIA+5Q1fxKei8mQqzLpzGxQ1S17FJhlJycrOnp6ZEOw5RDUpJr0ikuMdGdHwhEWpo7MvjuO9dVdPx4axoypjxEZLmqJpdVzq7INUELtstn4TmBb78F1WPnBOxksDGhZ0nfBC3YLp92TsCY8LHmHRNxCQmuhl+ciOtCaowpmzXvmJhhw0AYEz6W9E3EhWIYCLs4zJjAWNI3ERfsOQE7EWxoGW2oAAAQ0ElEQVRM4KxN38S8UHQZNSbWBdqmHxN3zjpy5AjZ2dkcOnQo0qGYANSuXZsWLVpQo0aNsGzPLg4zJnAxkfSzs7Np0KABSUlJiEikwzGlUFV27txJdnY2rVu3Dss2W7XyX9Mvz4lguzjMxIuYaNM/dOgQjRs3toQfA0SExo0bh/WozC4OMyZwMZH0AUv4MSTc/yu7OMyYwMVM0jemNMHcDyAU5wSsy6iJFVUy6Yf6C7hz5066dOlCly5daNasGc2bNz/6+vDhwwGt48Ybb2T9+vWllpk6dSppIcoW55xzDhkZGSFZV1UX7MVh1jxkYoqqRtWje/fuWlxWVtavppVk1izVunVV3dfPPerWddND4dFHH9WJEyf+anpBQYHm5+eHZiMh0Lt3b125cmXEtl+e/1mkBfuZSUwsumzhIzGxMqM2pijcUPdl5tgqV9MPZ/vsxo0bad++PampqXTo0IFt27YxcuRIkpOT6dChA+PGjTtatrDmnZeXR6NGjRg7dixnnnkmvXr14qeffgLg4Ycf5tlnnz1afuzYsfTo0YPTTz+dL774AoADBw4wdOhQ2rdvz5VXXklycnKZNfpZs2bRqVMnOnbsyIMPPghAXl4ev/vd745Onzx5MgDPPPMM7du3p3PnzgwfPjzk+ywaBXtOwLqMmlhS5ZJ+uL+AX331FX/4wx/IysqiefPmPPnkk6Snp7Nq1So+/PBDsrKyfrXMnj176Nu3L6tWraJXr17MnDnT77pVlS+//JKJEyce/QGZMmUKzZo1Iysri0ceeYSVK1eWGl92djYPP/wwCxcuZOXKlfzvf//jnXfeYfny5ezYsYPVq1ezZs0aRowYAcBTTz1FRkYGmZmZPPfcc0HundgRzDmBUIwdZOcETLhUuaQf7sG72rRpQ3LysYvgXnvtNbp160a3bt1Yt26d36Rfp04dLrnkEgC6d+/OlhIuGx0yZMivyixevJhhw4YBcOaZZ9KhQ4dS41u6dCnnnXceTZo0oUaNGlx33XV8/vnnnHbaaaxfv54xY8awYMECGjZsCECHDh0YPnw4aWlpYbu4KtZZl1ETS6pc0g/F4F3lUa9evaPPN2zYwN/+9jc++eQTMjMzSUlJ8dtfvWbNmkefV6tWjby8PL/rrlWrVpllKqpx48ZkZmbSp08fpk6dym233QbAggULGDVqFMuWLaNHjx7k59vdLctiXUZNLKlyST/YL2Aw9u7dS4MGDTjuuOPYtm0bCxYsCPk2evfuzdy5cwFYvXq13yMJX2effTYLFy5k586d5OXlMWfOHPr27UtOTg6qylVXXcW4ceNYsWIF+fn5ZGdnc9555/HUU0+xY8cOcotnI+OXdRk1sSImhmEor9TUyFxC361bN9q3b88ZZ5xBYmIivXv3Dvk27rzzTkaMGEH79u2PPgqbZvxp0aIFTzzxBP369UNVufzyyxkwYAArVqzg5ptvRlURESZMmEBeXh7XXXcd+/bto6CggHvvvZcGDRqE/D2YooIdRqKweajw97mweQhsKAnzazExyua6deto165dhCKKLnl5eeTl5VG7dm02bNjARRddxIYNG6hePbp+v+1/FrjiSRtck2SgR6g2yqgBu3NWlbV//3569+7NmWeeydChQ5k+fXrUJXxTPtHQZdSah+KHZYsY06hRI5YvXx7pMEyIBdMkac1Dpjyspm9MjAu2x1ooeg/ZkULssKRvTIyLdPOQXWcQWyzpG1MFRPKKYjtSiC2W9I2Jc8E2D9mRQmyxpB+A/v37/+pCq2effZbbb7+91OXq168PwA8//MCVV17pt0y/fv0o60bwzz77bJGLpC699FJ2794dSOileuyxx5g0aVLQ6zGxLdjmoWg4UjCBs6QfgGuvvZY5c+YUmTZnzhyuvfbagJY/5ZRTeOONNyq8/eJJ/7333qNRo0YVXp8xxQXTPBTpIwWw5qHyiLmkf9dd0K9faB933VX6Nq+88krefffdozdM2bJlCz/88AN9+vRh//79nH/++XTr1o1OnTrx9ttv/2r5LVu20LFjRwAOHjzIsGHDaNeuHYMHD+bgwYNHy91+++1Hh2V+9NFHAZg8eTI//PAD/fv3p3///gAkJSWxY8cOAJ5++mk6duxIx44djw7LvGXLFtq1a8ett95Khw4duOiii4psx5+MjAx69uxJ586dGTx4MD///PPR7RcOtVw40Ntnn3129CYyXbt2Zd++faXvQFOlRfpIIRTNQ/H0oxFQ0heRFBFZLyIbRWSsn/l3i0iWiGSKyMcikugzL19EMrzH/FAGHy4nnHACPXr04P333wdcLf/qq69GRKhduzbz5s1jxYoVLFy4kHvuuYfSrnKeNm0adevWZd26dTz++ONF+tyPHz+e9PR0MjMz+eyzz8jMzGTMmDGccsopLFy4kIULFxZZ1/Lly3n55ZdZunQpS5Ys4e9///vRoZY3bNjAHXfcwdq1a2nUqBFvvvlmqe9xxIgRTJgwgczMTDp16sTjjz8OwJNPPsnKlSvJzMzkhRdeAGDSpElMnTqVjIwMFi1aRJ06dcq/U02VEskjhWCbh+LtnEKZF2eJSDVgKnAhkA0sE5H5quo70tdKIFlVc0XkduAp4Bpv3kFV7RKqgL3KbNgVNvEMGjSIOXPm8NJLLwFuzPsHH3yQzz//nISEBLZu3cr27dtp1qyZ3/V8/vnnjBkzBoDOnTvTuXPno/Pmzp3LjBkzyMvLY9u2bWRlZRWZX9zixYsZPHjw0ZE+hwwZwqJFixg4cCCtW7emSxe320sbvhnc+P67d++mb9++AFx//fVcddVVR2NMTU3liiuu4IorrgDcoG933303qampDBkyhBYtWgSyC43xq/AH4qGHXJNOq1Yu4Yery2lpPxqBxpCWVvH4wy2Qmn4PYKOqblbVw8AcYJBvAVVdqKqFu20JUOWywKBBg/j4449ZsWIFubm5dO/eHYC0tDRycnJYvnw5GRkZnHTSSX6HUy7LN998w6RJk/j444/JzMxkwIABFVpPocJhmSG4oZnfffdd7rjjDlasWMFZZ51FXl4eY8eO5cUXX+TgwYP07t2br776qsJxGgOR7XIaDb2Pwtm8FEjSbw587/M625tWkpuB931e1xaRdBFZIiJX+FtAREZ6ZdJzcnICCCn86tevT//+/bnpppuKnMDds2cPJ554IjVq1GDhwoV86+96eB/nnnsus2fPBmDNmjVkZmYCbljmevXq0bBhQ7Zv3360KQmgQYMGftvN+/Tpw1tvvUVubi4HDhxg3rx59OnTp9zvrWHDhhx//PEsWrQIgH/+85/07duXgoICvv/+e/r378+ECRPYs2cP+/fvZ9OmTXTq1In777+fs846y5K+iahgm4ci3fso3M1LIR17R0SGA8lAX5/Jiaq6VUROBT4RkdWqusl3OVWdAcwAN8pmKGMKpWuvvZbBgwcX6cmTmprK5ZdfTqdOnUhOTuaMM84odR233347N954I+3ataNdu3ZHjxjOPPNMunbtyhlnnEHLli2LDMs8cuRIUlJSjrbtF+rWrRs33HADPXr0AOCWW26ha9eupTbllOQf//gHo0aNIjc3l1NPPZWXX36Z/Px8hg8fzp49e1BVxowZQ6NGjXjkkUdYuHAhCQkJdOjQ4ehdwIyJhGCbh8aP9z/Kabh6H4WiealcyrpzOtALWODz+gHgAT/lLgDWASeWsq5XgCtL21737t1/dZf3rKysit8i3kSE/c9MLJk1SzUxUVXE/Z01K/BlExNVXR296CMxMbDlRfwvL1K+9wCkaxn5XFUDat5ZBrQVkdYiUhMYBhTphSMiXYHpwEBV/cln+vEiUst73gToDZR+qydjjAmzSPY+Cvd9vctM+qqaB4wGFuBq8nNVda2IjBORgV6xiUB94F/Fuma2A9JFZBWwEHhSi/b6McaYmBbsdQrhvq93QG36qvoe8F6xaX/yeX5BCct9AXQKJkCfdSEioViVqWQaZXdjM6ayBXM/hGDPSZRXTNxEpXbt2uzcuZPGjRtb4o9yqsrOnTupXbt2pEMxJmaE877eMZH0W7RoQXZ2NtHandMUVbt2bbtgy5goFRNJv0aNGrRu3TrSYRhjTMyLuQHXjDHGVJwlfWOMiSOW9I0xJo5ItHWvE5EcoPQBbCKrCbAj0kGUwuILjsUXHIsvOMHEl6iqTcsqFHVJP9qJSLqqJkc6jpJYfMGx+IJj8QUnHPFZ844xxsQRS/rGGBNHLOmX34xIB1AGiy84Fl9wLL7gVHp81qZvjDFxxGr6xhgTRyzpG2NMHLGkX4yItBSRhSKSJSJrReT3fsr0E5E93r0DMkTkT/7WVclxbhGR1d720/3MFxGZLCIbRSRTRLqFMbbTffZNhojsFZG7ipUJ6z4UkZki8pOIrPGZdoKIfCgiG7y/x5ew7PVemQ0icn0Y45soIl95/795ItKohGVL/SxUYnyPichWn//hpSUsmyIi673P4tgwxve6T2xbRCSjhGXDsf/85pWIfAYDub1WPD2Ak4Fu3vMGwNdA+2Jl+gHvRDjOLUCTUuZfirtBvQA9gaURirMa8CPuwpGI7UPgXKAbsMZn2lPAWO/5WGCCn+VOADZ7f4/3nh8fpvguAqp7zyf4iy+Qz0IlxvcYcG8A//9NwKlATWBV8e9TZcVXbP5fgT9FcP/5zSuR+AxaTb8YVd2mqiu85/twdwtrHtmoKmQQ8Ko6S4BGInJyBOI4H9ikqhG9ylpVPwd2FZs8CPiH9/wfwBV+Fr0Y+FBVd6nqz8CHQEo44lPVD9TduQ5gCRCx8apL2H+B6AFsVNXNqnoYmIPb7yFVWnzibsJxNfBaqLcbqFLyStg/g5b0SyEiSUBXYKmf2b1EZJWIvC8iHcIamKPAByKyXERG+pnfHPje53U2kfnxGkbJX7ZI78OTVHWb9/xH4CQ/ZaJlP96EO3Lzp6zPQmUa7TU/zSyhaSIa9l8fYLuqbihhflj3X7G8EvbPoCX9EohIfeBN4C5V3Vts9gpcc8WZwBTgrXDHB5yjqt2AS4A7ROTcCMRQKhGpCQwE/uVndjTsw6PUHUdHZf9lEXkIyAPSSigSqc/CNKAN0AXYhmtCiUbXUnotP2z7r7S8Eq7PoCV9P0SkBu4fk6aq/y4+X1X3qup+7/l7QA0RaRLOGFV1q/f3J2Ae7jDa11agpc/rFt60cLoEWKGq24vPiIZ9CGwvbPLy/v7kp0xE96OI3ABcBqR6SeFXAvgsVApV3a6q+apaAPy9hO1Gev9VB4YAr5dUJlz7r4S8EvbPoCX9Yrz2v5eAdar6dAllmnnlEJEeuP24M4wx1hORBoXPcSf81hQrNh8Y4fXi6Qns8TmMDJcSa1iR3oee+UBhT4jrgbf9lFkAXCQix3vNFxd50yqdiKQA9wEDVTW3hDKBfBYqKz7fc0SDS9juMqCtiLT2jvyG4fZ7uFwAfKWq2f5mhmv/lZJXwv8ZrMwz1rH4AM7BHWJlAhne41JgFDDKKzMaWIvribAE+G2YYzzV2/YqL46HvOm+MQowFddzYjWQHOYY6+GSeEOfaRHbh7gfn23AEVyb6M1AY+BjYAPwEXCCVzYZeNFn2ZuAjd7jxjDGtxHXllv4OXzBK3sK8F5pn4UwxfdP77OViUteJxePz3t9Ka63yqZwxudNf6XwM+dTNhL7r6S8EvbPoA3DYIwxccSad4wxJo5Y0jfGmDhiSd8YY+KIJX1jjIkjlvSNMSaOWNI3xpg4YknfGGPiyP8HzTRPR96HpHIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss) + 1)\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "검증 MAE는 0.40 대에 머물러 있습니다. 작은 컨브넷을 사용해서 상식 수준의 기준점을 넘지 못 했습니다. **이는 컨브넷이 입력 시계열에 있는 패턴을 보고 이 패턴의 시간 축의 위치(시작인지 끝 부분인지 등)를 고려하지 않기 때문입니다. 최근 데이터 포인트일수록 오래된 데이터 포인트와는 다르게 해석해야 하기 때문에 컨브넷이 의미 있는 결과를 만들지 못합니다.** 이런 컨브넷의 한계는 IMDB 데이터에서는 문제가 되지 않습니다. 긍정 또는 부정적인 감성과 연관된 키워드 패턴의 중요성은 입력 시퀀스에 나타난 위치와 무관하기 때문입니다.\n",
    "\n",
    "**컨브넷의 속도와 경량함을 RNN의 순서 감지 능력과 결합하는 한가지 전략은 1D 컨브넷을 RNN 이전에 전처리 단계로 사용하는 것**입니다. 수천 개의 스텝을 가진 시퀀스 같이 RNN으로 처리하기엔 현실적으로 너무 긴 시퀀스를 다룰 때 특별히 도움이 됩니다. 컨브넷이 긴 입력 시퀀스를 더 짧은 고수준 특성의 (다운 샘플된) 시퀀스로 변환합니다. 추출된 특성의 시퀀스는 RNN 파트의 입력이 됩니다.\n",
    "\n",
    "\n",
    "![](./images/1d-cnn-rnn.png)\n",
    "\n",
    "\n",
    "이 기법이 연구 논문이나 실전 애플리케이션에 자주 등장하지는 않습니다. 아마도 널리 알려지지 않았기 때문일 것입니다. 이 방법은 효과적이므로 많이 사용되기를 바랍니다. 온도 예측 문제에 적용해 보죠. \n",
    "\n",
    "이 전략은 훨씬 긴 시퀀스를 다룰 수 있으므로 더 오래전 데이터를 바라보거나(데이터 제너레이터의 lookback 매개변수를 증가시킵니다), 시계열 데이터를 더 촘촘히 바라볼 수 있습니다(제너레이터의 step 매개변수를 감소시킵니다). 여기서는 그냥 step을 절반으로 줄여서 사용하겠습니다. 온도 데이터가 30분마다 1 포인트씩 샘플링되기 때문에 결과 시계열 데이터는 두 배로 길어집니다. 앞서 정의한 제너레이터 함수를 다시 사용합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 이전에는 6이었습니다(시간마다 1 포인트); 이제는 3 입니다(30분마다 1 포인트)\n",
    "step = 3\n",
    "lookback = 1440  # 변경 안 됨\n",
    "delay = 144 # 변경 안 됨\n",
    "\n",
    "train_gen = generator(float_data,\n",
    "                      lookback=lookback,\n",
    "                      delay=delay,\n",
    "                      min_index=0,\n",
    "                      max_index=200000,\n",
    "                      shuffle=True,\n",
    "                      step=step)\n",
    "val_gen = generator(float_data,\n",
    "                    lookback=lookback,\n",
    "                    delay=delay,\n",
    "                    min_index=200001,\n",
    "                    max_index=300000,\n",
    "                    step=step)\n",
    "test_gen = generator(float_data,\n",
    "                     lookback=lookback,\n",
    "                     delay=delay,\n",
    "                     min_index=300001,\n",
    "                     max_index=None,\n",
    "                     step=step)\n",
    "val_steps = (300000 - 200001 - lookback) // 128\n",
    "test_steps = (len(float_data) - 300001 - lookback) // 128"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이 모델은 두 개의 Conv1D 층 다음에 GRU 층을 놓았습니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv1d_1 (Conv1D)            (None, None, 32)          2272      \n",
      "_________________________________________________________________\n",
      "max_pooling1d_1 (MaxPooling1 (None, None, 32)          0         \n",
      "_________________________________________________________________\n",
      "conv1d_2 (Conv1D)            (None, None, 32)          5152      \n",
      "_________________________________________________________________\n",
      "gru_1 (GRU)                  (None, 32)                6240      \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 1)                 33        \n",
      "=================================================================\n",
      "Total params: 13,697\n",
      "Trainable params: 13,697\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Epoch 1/10\n",
      "500/500 [==============================] - 64s 128ms/step - loss: 0.3327 - val_loss: 0.2921\n",
      "Epoch 2/10\n",
      "500/500 [==============================] - 64s 127ms/step - loss: 0.3041 - val_loss: 0.2892\n",
      "Epoch 3/10\n",
      "500/500 [==============================] - 63s 127ms/step - loss: 0.2934 - val_loss: 0.2730\n",
      "Epoch 4/10\n",
      "500/500 [==============================] - 63s 127ms/step - loss: 0.2875 - val_loss: 0.2701\n",
      "Epoch 5/10\n",
      "500/500 [==============================] - 63s 127ms/step - loss: 0.2803 - val_loss: 0.2853\n",
      "Epoch 6/10\n",
      "500/500 [==============================] - 63s 127ms/step - loss: 0.2725 - val_loss: 0.2736\n",
      "Epoch 7/10\n",
      "500/500 [==============================] - 63s 127ms/step - loss: 0.2678 - val_loss: 0.2797\n",
      "Epoch 8/10\n",
      "500/500 [==============================] - 64s 127ms/step - loss: 0.2627 - val_loss: 0.2790\n",
      "Epoch 9/10\n",
      "500/500 [==============================] - 64s 127ms/step - loss: 0.2588 - val_loss: 0.2776\n",
      "Epoch 10/10\n",
      "500/500 [==============================] - 63s 127ms/step - loss: 0.2526 - val_loss: 0.2805\n"
     ]
    }
   ],
   "source": [
    "K.clear_session()\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.Conv1D(32, 5, activation='relu',\n",
    "                        input_shape=(None, float_data.shape[-1])))\n",
    "model.add(layers.MaxPooling1D(3))\n",
    "model.add(layers.Conv1D(32, 5, activation='relu'))\n",
    "model.add(layers.GRU(32, dropout=0.1, recurrent_dropout=0.5))\n",
    "model.add(layers.Dense(1))\n",
    "\n",
    "model.summary()\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae')\n",
    "history = model.fit_generator(train_gen,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=10,\n",
    "                              validation_data=val_gen,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VOXZ//HPxS6LoIC1ghBcIchqBC2lLKJFqVCUKhjcqkWtS1vq75GKrUrLU6s+iliePlKXthJFqlWpG9pKRVtFgkUQkEKRJeICVDZBIXD9/rjPhEnMMgmTnJD5vl+vvDLnzH3OXDOB65y5z32u29wdERHJDPXiDkBERGqOkr6ISAZR0hcRySBK+iIiGURJX0Qkgyjpi4hkECV9qRQzq29mO8ysQzrbxsnMjjOztI9dNrMhZrYmaXmFmfVPpW0VXusBM7upqtuXs99fmNnv0r1fiU+DuAOQ6mVmO5IWmwJfAHuj5SvdPa8y+3P3vUDzdLfNBO5+Yjr2Y2ZXAGPdfWDSvq9Ix76l7lPSr+PcvSjpRmeSV7j7X8pqb2YN3L2wJmITkZqn7p0MF319f9zMHjOz7cBYMzvNzN40sy1m9qGZTTWzhlH7BmbmZpYVLc+Inn/BzLab2Rtm1qmybaPnzzKzf5nZVjO7z8z+bmaXlhF3KjFeaWarzOxTM5uatG19M7vHzDab2WpgaDmfz0Qzm1li3TQzuzt6fIWZLY/ez7+js/Cy9lVgZgOjx03N7JEotqXAySXa3mxmq6P9LjWz4dH6bsCvgf5R19mmpM/21qTtr4re+2Yze9rMvprKZ1MRMxsZxbPFzF4xsxOTnrvJzDaY2TYzey/pvZ5qZm9H6z82sztTfT2pBu6unwz5AdYAQ0qs+wWwGziHcBJwCHAK0JfwTfAY4F/AtVH7BoADWdHyDGATkAM0BB4HZlSh7RHAdmBE9Nx4YA9waRnvJZUYnwFaAlnAfxLvHbgWWAq0B1oD88J/hVJf5xhgB9Asad+fADnR8jlRGwMGA7uA7tFzQ4A1SfsqAAZGj+8C/gYcBnQElpVoez7w1ehvcmEUw1ei564A/lYizhnArdHjM6MYewJNgP8FXknlsynl/f8C+F30uEsUx+Dob3QTsCJ63BVYCxwZte0EHBM9XgCMiR63APrG/X8hk390pi8Ar7v7n919n7vvcvcF7j7f3QvdfTUwHRhQzvZPuHu+u+8B8gjJprJtvwUscvdnoufuIRwgSpVijL90963uvoaQYBOvdT5wj7sXuPtm4PZyXmc18C7hYARwBvCpu+dHz//Z3Vd78ArwV6DUi7UlnA/8wt0/dfe1hLP35Ned5e4fRn+TRwkH7JwU9guQCzzg7ovc/XNgAjDAzNontSnrsynPaGC2u78S/Y1uJxw4+gKFhANM16iL8P3os4Nw8D7ezFq7+3Z3n5/i+5BqoKQvAOuTF8yss5k9Z2Yfmdk2YBLQppztP0p6vJPyL96W1fao5Djc3QlnxqVKMcaUXotwhlqeR4Ex0eMLo+VEHN8ys/lm9h8z20I4yy7vs0r4ankxmNmlZvZO1I2yBeic4n4hvL+i/bn7NuBToF1Sm8r8zcra7z7C36idu68Afkz4O3wSdRceGTW9DMgGVpjZW2Z2dorvQ6qBkr5A+Lqf7H7C2e1x7n4o8DNC90V1+pDQ3QKAmRnFk1RJBxLjh8DRScsVDSmdBQwxs3aEM/5HoxgPAZ4AfknoemkFvJRiHB+VFYOZHQP8BrgaaB3t972k/VY0vHQDocsosb8WhG6kD1KIqzL7rUf4m30A4O4z3L0foWunPuFzwd1XuPtoQhfe/wBPmlmTA4xFqkhJX0rTAtgKfGZmXYAra+A1nwV6m9k5ZtYA+AHQtppinAX80MzamVlr4MbyGrv7R8DrwO+AFe6+MnqqMdAI2AjsNbNvAadXIoabzKyVhfsYrk16rjkhsW8kHP++RzjTT/gYaJ+4cF2Kx4DLzay7mTUmJN/X3L3Mb06ViHm4mQ2MXvv/Ea7DzDezLmY2KHq9XdHPPsIbuMjM2kTfDLZG723fAcYiVaSkL6X5MXAJ4T/0/YQLrtXK3T8GLgDuBjYDxwL/JNxXkO4Yf0Poe19CuMj4RArbPEq4MFvUtePuW4AfAU8RLoaOIhy8UnEL4RvHGuAF4A9J+10M3Ae8FbU5EUjuB38ZWAl8bGbJ3TSJ7V8kdLM8FW3fgdDPf0DcfSnhM/8N4YA0FBge9e83Bu4gXIf5iPDNYmK06dnAcgujw+4CLnD33Qcaj1SNha5TkdrFzOoTuhNGuftrcccjUlfoTF9qDTMbGnV3NAZ+Shj18VbMYYnUKUr6Upt8HVhN6Dr4JjDS3cvq3hGRKlD3johIBtGZvohIBql1BdfatGnjWVlZcYchInJQWbhw4SZ3L2+YM1ALk35WVhb5+flxhyEiclAxs4ruLAfUvSMiklGU9EVEMoiSvohIBql1ffoiUrP27NlDQUEBn3/+edyhSAqaNGlC+/btadiwrNJL5VPSF8lwBQUFtGjRgqysLEJxU6mt3J3NmzdTUFBAp06dKt6gFHWmeycvD7KyoF698DuvUtN9i2Suzz//nNatWyvhHwTMjNatWx/Qt7I6caaflwfjxsHOnWF57dqwDJB7wLUFReo+JfyDx4H+rerEmf7EifsTfsLOnWG9iIjsVyeS/rp1lVsvIrXH5s2b6dmzJz179uTII4+kXbt2Rcu7d6dWdv+yyy5jxYoV5baZNm0aeWnq9/3617/OokWL0rKvmlYnunc6dAhdOqWtF5H0yssL36LXrQv/xyZPPrBu1NatWxcl0FtvvZXmzZtzww03FGvj7rg79eqVfp768MMPV/g611xzTdWDrEPqxJn+5MnQtGnxdU2bhvUikj6J62dr14L7/utn1TFwYtWqVWRnZ5Obm0vXrl358MMPGTduHDk5OXTt2pVJkyYVtU2ceRcWFtKqVSsmTJhAjx49OO200/jkk08AuPnmm5kyZUpR+wkTJtCnTx9OPPFE/vGPfwDw2Wefcd5555Gdnc2oUaPIycmp8Ix+xowZdOvWjZNOOombbroJgMLCQi666KKi9VOnTgXgnnvuITs7m+7duzN27Ni0f2apqBNn+omzjHSefYjIl5V3/aw6/r+99957/OEPfyAnJweA22+/ncMPP5zCwkIGDRrEqFGjyM7OLrbN1q1bGTBgALfffjvjx4/noYceYsKECV/at7vz1ltvMXv2bCZNmsSLL77Ifffdx5FHHsmTTz7JO++8Q+/evcuNr6CggJtvvpn8/HxatmzJkCFDePbZZ2nbti2bNm1iyZIlAGzZsgWAO+64g7Vr19KoUaOidTUtpTP9aEajFWa2ysy+9OmZ2VVmtsTMFpnZ62aWHa3vE61bZGbvmNnIdL+BhNxcWLMG9u0Lv5XwRdKvpq+fHXvssUUJH+Cxxx6jd+/e9O7dm+XLl7Ns2bIvbXPIIYdw1llnAXDyySezZs2aUvd97rnnfqnN66+/zujRowHo0aMHXbt2LTe++fPnM3jwYNq0aUPDhg258MILmTdvHscddxwrVqzg+uuvZ86cObRs2RKArl27MnbsWPLy8qp8c9WBqjDpR3OVTgPOArKBMYmknuRRd+/m7j0JkyPfHa1/F8iJ1g8F7jezOvHtQiQTlXWdrLqunzVr1qzo8cqVK7n33nt55ZVXWLx4MUOHDi11vHqjRo2KHtevX5/CwsJS9924ceMK21RV69atWbx4Mf3792fatGlceeWVAMyZM4errrqKBQsW0KdPH/bu3ZvW101FKmf6fYBV7r46msF+JjAiuYG7b0tabAZ4tH6nuyc+zSaJ9SJycIrz+tm2bdto0aIFhx56KB9++CFz5sxJ+2v069ePWbNmAbBkyZJSv0kk69u3L3PnzmXz5s0UFhYyc+ZMBgwYwMaNG3F3vvOd7zBp0iTefvtt9u7dS0FBAYMHD+aOO+5g06ZN7CzZV1YDUjnrbgesT1ouAPqWbGRm1wDjgUbA4KT1fYGHgI7ARUkHgeRtxwHjADpoyI1IrRXn9bPevXuTnZ1N586d6dixI/369Uv7a1x33XVcfPHFZGdnF/0kumZK0759e37+858zcOBA3J1zzjmHYcOG8fbbb3P55Zfj7pgZv/rVrygsLOTCCy9k+/bt7Nu3jxtuuIEWLVqk/T1UpMI5cs1sFDDU3a+Ili8C+rr7tWW0vxD4prtfUmJ9F+D3wDfcvcx7iHNyclyTqIjUnOXLl9OlS5e4w6gVCgsLKSwspEmTJqxcuZIzzzyTlStX0qBB7eqVLu1vZmYL3T2njE2KpPJOPgCOTlpuH60ry0zgNyVXuvtyM9sBnAQoq4tIrbNjxw5OP/10CgsLcXfuv//+WpfwD1Qq72YBcLyZdSIk+9HAhckNzOx4d18ZLQ4DVkbrOwHr3b3QzDoCnYE1aYpdRCStWrVqxcKFC+MOo1pVmPSjhH0tMAeoDzzk7kvNbBKQ7+6zgWvNbAiwB/gUSHTtfB2YYGZ7gH3A9919U3W8ERERqVhK31vc/Xng+RLrfpb0+AdlbPcI8MiBBCgiIulTJ8owiIhIapT0RUQyiJK+iMRq0KBBX7rRasqUKVx99dXlbte8eXMANmzYwKhRo0ptM3DgQCoaAj5lypRiN0mdffbZaamLc+utt3LXXXcd8H7STUlfRGI1ZswYZs6cWWzdzJkzGTNmTErbH3XUUTzxxBNVfv2SSf/555+nVatWVd5fbaekLyKxGjVqFM8991zRhClr1qxhw4YN9O/fv2jcfO/evenWrRvPPPPMl7Zfs2YNJ510EgC7du1i9OjRdOnShZEjR7Jr166idldffXVRWeZbbrkFgKlTp7JhwwYGDRrEoEGDAMjKymLTpjDI8O677+akk07ipJNOKirLvGbNGrp06cL3vvc9unbtyplnnlnsdUqzaNEiTj31VLp3787IkSP59NNPi14/UWo5Uejt1VdfLZpEplevXmzfvr3Kn21p6tZdByJyQH74Q0j3hFA9e0KUL0t1+OGH06dPH1544QVGjBjBzJkzOf/88zEzmjRpwlNPPcWhhx7Kpk2bOPXUUxk+fHiZ88T+5je/oWnTpixfvpzFixcXK408efJkDj/8cPbu3cvpp5/O4sWLuf7667n77ruZO3cubdq0KbavhQsX8vDDDzN//nzcnb59+zJgwAAOO+wwVq5cyWOPPcZvf/tbzj//fJ588sly6+NffPHF3HfffQwYMICf/exn3HbbbUyZMoXbb7+d999/n8aNGxd1Kd11111MmzaNfv36sWPHDpo0aVKJT7tiOtMXkdgld/Ekd+24OzfddBPdu3dnyJAhfPDBB3z88cdl7mfevHlFybd79+5079696LlZs2bRu3dvevXqxdKlSysspvb6668zcuRImjVrRvPmzTn33HN57bXXAOjUqRM9e/YEyi/fDKG+/5YtWxgwYAAAl1xyCfPmzSuKMTc3lxkzZhTd+duvXz/Gjx/P1KlT2bJlS9rvCNaZvogUKe+MvDqNGDGCH/3oR7z99tvs3LmTk08+GYC8vDw2btzIwoULadiwIVlZWaWWU67I+++/z1133cWCBQs47LDDuPTSS6u0n4REWWYIpZkr6t4py3PPPce8efP485//zOTJk1myZAkTJkxg2LBhPP/88/Tr1485c+bQuXPnKsdaks70RSR2zZs3Z9CgQXz3u98tdgF369atHHHEETRs2JC5c+eytrTJsJN84xvf4NFHHwXg3XffZfHixUAoy9ysWTNatmzJxx9/zAsvvFC0TYsWLUrtN+/fvz9PP/00O3fu5LPPPuOpp56if//+lX5vLVu25LDDDiv6lvDII48wYMAA9u3bx/r16xk0aBC/+tWv2Lp1Kzt27ODf//433bp148Ybb+SUU07hvffeq/Rrlkdn+iJSK4wZM4aRI0cWG8mTm5vLOeecQ7du3cjJyanwjPfqq6/msssuo0uXLnTp0qXoG0OPHj3o1asXnTt35uijjy5WlnncuHEMHTqUo446irlz5xat7927N5deeil9+vQB4IorrqBXr17lduWU5fe//z1XXXUVO3fu5JhjjuHhhx9m7969jB07lq1bt+LuXH/99bRq1Yqf/vSnzJ07l3r16tG1a9eiWcDSpcLSyjVNpZVFapZKKx98DqS0srp3REQyiJK+iEgGUdIXEWpbN6+U7UD/Vkr6IhmuSZMmbN68WYn/IODubN68+YBu2Epp9I6ZDQXuJUyi8oC7317i+auAa4C9wA5gnLsvM7MzgNsJk6XvBv6fu79S5WhFJO3at29PQUEBGzdujDsUSUGTJk1o3759lbevMOmbWX1gGnAGUAAsMLPZ7p58O9uj7v5/UfvhwN3AUGATcI67bzCzkwizb7WrcrQiknYNGzakU6dOcYchNSSV7p0+wCp3X+3uuwkTn49IbuDu25IWmwEerf+nu2+I1i8FDjGzxoiISCxS6d5pB6xPWi4A+pZsZGbXAOMJXTmDS9nPecDb7v5FFeIUEZE0SNuFXHef5u7HAjcCNyc/Z2ZdgV8BV5a2rZmNM7N8M8tXv6KISPVJJel/ABydtNw+WleWmcC3Ewtm1h54CrjY3f9d2gbuPt3dc9w9p23btimEJCIiVZFK0l8AHG9mncysETAamJ3cwMyOT1ocBqyM1rcCngMmuPvf0xOyiIhUVYVJ390LgWsJI2+WA7PcfamZTYpG6gBca2ZLzWwRoV//ksR64DjgZ2a2KPo5Iv1vQ0REUqGCayIidYAKromIyJco6YuIZBAlfRGRDKKkLyKSQZT0RUQyiJK+iEgGUdIXEckgSvoiIhlESV9EJIMo6YuIZBAlfRGRDKKkLyKSQZT0RUQyiJK+iEgGUdIXEckgSvoiIhkkpaRvZkPNbIWZrTKzCaU8f5WZLYlmxnrdzLKj9a3NbK6Z7TCzX6c7eBERqZwKk76Z1QemAWcB2cCYRFJP8qi7d3P3nsAdwN3R+s+BnwI3pC9kERGpqlTO9PsAq9x9tbvvBmYCI5IbuPu2pMVmgEfrP3P31wnJX0REYtYghTbtgPVJywVA35KNzOwawqTojYDBlQnCzMYB4wA6dOhQmU1FRKQS0nYh192nufuxwI3AzZXcdrq757h7Ttu2bdMVkoiIlJBK0v8AODppuX20riwzgW8fSFAiIlI9Ukn6C4DjzayTmTUCRgOzkxuY2fFJi8OAlekLUURE0qXCPn13LzSza4E5QH3gIXdfamaTgHx3nw1ca2ZDgD3Ap8Alie3NbA1wKNDIzL4NnOnuy9L/VkREpCLm7nHHUExOTo7n5+fHHYaIyEHFzBa6e05F7XRHrohIBlHSFxHJIEr6IiIZRElfRCSDKOmLiGQQJX0RkQyipJ9GeXmQlQX16oXfeXlxRyQiUlwqBdckBXl5MG4c7NwZlteuDcsAubnxxSUikkxn+mkyceL+hJ+wc2dYLyJSWyjpp8m6dZVbLyISByX9NClrGgBNDyAitYmSfppMngxNmxZf17RpWC8iUlso6adJbi5Mnw4dO4JZ+D19ui7iikjtotE7aZSbqyQvIrWbzvRFRDKIkr6ISAZJKemb2VAzW2Fmq8xsQinPX2VmS8xskZm9bmbZSc/9JNpuhZl9M53Bi4hI5VSY9M2sPjANOAvIBsYkJ/XIo+7ezd17AncAd0fbZhPm1O0KDAX+N9qfiIjEIJUz/T7AKndf7e67gZnAiOQG7r4tabEZkJiDcQQw092/cPf3gVXR/kREJAapjN5pB6xPWi4A+pZsZGbXAOOBRsDgpG3fLLFtu1K2HQeMA+igu5lERKpN2i7kuvs0dz8WuBG4uZLbTnf3HHfPadu2bbpCEhGRElJJ+h8ARyctt4/WlWUm8O0qbisiItUolaS/ADjezDqZWSPChdnZyQ3M7PikxWHAyujxbGC0mTU2s07A8cBbBx62iIhURYV9+u5eaGbXAnOA+sBD7r7UzCYB+e4+G7jWzIYAe4BPgUuibZea2SxgGVAIXOPue6vpvYiISAXM3StuVYNycnI8Pz8/7jBERA4qZrbQ3XMqaqc7ckVEMoiSvohIBlHSFxHJIEr6IiIZRElfRCSDKOmLiGQQJX0RkQyipF8H5eVBVhbUqxd+5+XFHZGI1BaaI7eOycuDceNg586wvHZtWAbN3ysiOtOvcyZO3J/wE3buDOtFRJT065h16yq3XkQyi5J+HVPWHDSam0ZEoI4l/WeeCWe0tayGXI2aPBmaNi2+rmnTsF5EpM5cyC0ogG9HU7cceST07Qt9+oTfOTnQsmW88dWUxMXaiRPDAbBDh5DwdRFXRKAOlVbeswcWLYL588PPW2/Bv/4VnjODzp33HwT69oVu3aBhwzQHLyISk1RLK6eU9M1sKHAvYRKVB9z99hLPjweuIEyUshH4rruvjZ77FWE2LYCfu/vj5b1WOuvp/+c/kJ+//0Awfz5s2hSea9IEevcu/o0gKyscIEREDjZpS/pmVh/4F3AGUECYPnGMuy9LajMImO/uO83samCgu19gZsOAHwJnAY2BvwGnu/u2sl6vOidRcYc1a8K3gMRB4O234fPPw/Nt2+4/APTpE34OO6xaQhERSatUk34qffp9gFXuvjra8UxgBGEKRADcfW5S+zeBsdHjbGCeuxcChWa2GBgKzErpXaSZGXTqFH4uuCCs27MHliwpfiB4/vn9F4OPP35/l1CfPtCjBzRuHEf0IiIHLpWk3w5Yn7RcAPQtp/3lwAvR43eAW8zsf4CmwCCSDha1QcOGoZund2+46qqwbuvW0C2UOBD85S8wY0Z4rlEj6NWr+DeC445Tt5CIHBzSOnrHzMYCOcAAAHd/ycxOAf5B6Ot/A/jSxOhmNg4YB9ChFgwob9kSTj89/EA46y8o2H+BeP58ePBBuO++8Pzhh+/vDkocCNq0iS9+EZGypNKnfxpwq7t/M1r+CYC7/7JEuyHAfcAAd/+kjH09Csxw9+fLer2DZWL0wkJYtqz4aKGlS2HfvvDtYdas/UNIRUSqWzonRl8AHG9mncysETAamF3ixXoB9wPDkxO+mdU3s9bR4+5Ad+Cl1N9G7dWgAXTvDt/7HjzwACxeHLqF/va30P2TmxsuEouI1CYVJv3oIuy1wBxgOTDL3Zea2SQzGx41uxNoDvzRzBaZWeKg0BB4zcyWAdOBsdH+6qTmzWHAgHBncJs2cM458MEHcUcVH5V4Fql96szNWbXN4sXQr18Y/fPaa9CsWdwR1aySJZ4hlIOYPl13B4tUh3R270gVdO8Ojz8O77wDY8eGvv5MohLPIrWTkn41OvtsuOceePppmDAh7mhqlko8i9ROSvrV7Lrr4Oqr4c47wzDPTKESzyK1k5J+NTODqVPhzDPDzV9z51a8TV2gEs8itZOSfg1o0CCM2z/hBDjvvP3VP+uy3Nxw0bZjx3Dg69hRF3FFagON3qlBq1eHO3ZbtYI334TWreOOKHM8+2wYUjtwYNyRiFQPjd6phY45JlzUXbcunPHv3h13RJlh1iwYPjyU1bj33syeWU1ESb+G9esHDz0Er74a+viVgKrX3Llw0UXwta+FxP/DH4aL64V19hZBkfLVmekSDya5ubBiBfz852FGr//6r7gjqpsWLYIRI0IV1NmzQ7fajTfCXXeFrraZM+HQQ+OOUqRm6Uw/JrfdFmr6T5gATz0VdzR1z/vvw1lnhUQ/Z06ohFqvXhg6e//98NJL8PWv674ByTxK+jExg4cfDmWYc3Nh4cK4I6o7Nm6Eb34TvvgiJPz27Ys/P24cvPACrF0bPv8FC+KJUyQOSvoxOuSQUJytbdvQ35zJxdnSZccOGDYszH/w7LPQpUvp7c44A954I/wNBgyAJ5+s2ThF4qKkH7OvfCUkp+3bQ1XOHTvijujgtXs3jBoVSlo//ni4eFue7OwwF0KPHmG7O+7QhXWp+5T0a4Fu3cJFxURxtr1fmltMKrJvH1x+eejOuf/+cABNxRFHwCuvhOsrN94Y5kfYs6d6YxWJk5J+LZEozvbMM/CTn8QdzcHnxhvDPMa/+EVI/pVxyCHw6KNw882hPtLQofDpp9UTp0jclPRrkeuug+9/P4wweeCBuKM5ePzP/4RhmNdcAzfdVLV91KsXhtD+7ndh/oOvfS0M6xSpa1JK+mY21MxWmNkqM/tSkWAzG29my8xssZn91cw6Jj13h5ktNbPlZjbVzCydb6AuMQt3jJ55ZqjM+corcUdU+82YATfcEPrk7703fIYlVWYGr0sugZdfho8/DiUz/vGP6opcJCbuXu4PUB/4N3AM0Ah4B8gu0WYQ0DR6fDXwePT4a8Dfo33UB94ABpb3eieffLJnui1b3LOz3Vu1cl+xIu5oaq8XX3Rv0MB94ED3XbtKbzNjhnvTpu7hEm34ado0rC/PihXuxx3n3rix+6OPpj92EXf3wkL3JUvcH3zQfdw49xtuqPq+gHyvIJ+7e0pn+n2AVe6+2t13AzOBESUOHHPdPTFP0ptAYmS0A02ig0Vjwpy5H1fyuJRxWrYMI3oaNgzDDzdvjjui2mfBglC/qGvXUM+oSZPS21V1Bq8TTghF8fr2hQsvhEmTNLJHDox7uBnwiSfCXfgDB4b/6926hetQjz8OH31U/XGkUoahHbA+abkA6FtO+8uBFwDc/Q0zmwt8CBjwa3dfXnIDMxsHjAPooFk2AOjUKSSzQYNCcnvpJWjUKO6oaoeVK8OF77Ztw01WLVuW3fZAZvBq3Tp87uPGwS23hNd94AFo3LhqcdcGu3eHkWKPPBKS0KGHQosW4SfxuKzficfNmoWuMinfli3h5OStt/b/JJJ6o0bQsydcdlm4QbBPnzCfdk18rmmtvWNmY4EcYEC0fBzQhf1n/i+bWX93fy15O3efDkyHUFo5nTEdzL72tVCcbezYUJztwQdL77POJB99FO62hTA886tfLb99hw7hztvS1qeiceNwcfeEE8LonjVrQtmMNm0qE3X8Nm8OQ1l//Wv48MPwftq0Cdcutm+HbdvC71QK0ZmFMtWpHCAqatO8ed04gHzxRRhynUju8+cXnzfjxBPDDYF9+4YE3717fCcPqST9D4Cjk5bbR+uKMbMhwET9qvO0AAAQT0lEQVRggLt/Ea0eCbzp7juiNi8ApwGvldxeSpebG/7xTJoU/uHceGPcEcVn27ZQT+eTT0L1zBNOqHibyZPDmXpyF09lZ/AyC91Bxx4Ll14Kp54Kzz0X/h613cqVMGVKKPmxa1c4YP7udyEBlTyBcIfPPw/JP/lAUNbvkus++aT4cioHkHr14Mgj4aijwk+7dvsfJy8ffnjtOeHZty98ronk/tZbobhf4v6OI48Myf2SS0KCz8kJNaBqiwonUTGzBsC/gNMJyX4BcKG7L01q0wt4Ahjq7iuT1l8AfA8YSujeeRGY4u5/Luv16vIkKlXlHvqVZ84M5QLOPTfuiGreF1+ELp1588L1jsTZfiry8kLSXrcunOFPnlz1GbzeeCNU7iwshD/9qXZOyuIePqe774Y//zlcGxo7Fn70IzjppJqL4YsvKj54bNkSvnls2BDKkGzYUPo1rEaNyj8oJH5atEj/e/nww+JdNAsWwNat4bnmzUNST3TR9OkTaj3FcYBKdRKVlGbOMrOzgSmEETgPuftkM5tEuFo828z+AnQj9N0DrHP34WZWH/hf4BuEi7ovuvv48l5LSb90u3bB4MHhK+Rrr8HJJ8cdUc3Ztw/GjAmToTzySEhgcVq9Gr71LVi1KnSZXHZZvPEk7NkDf/xjSPYLF4bum+9/P/x85StxR5e6zz8P3XiJg0Dip+Ty9u1f3rZFi/IPCu3ahS7BsrpWtm8Pn11ykl8fXdGsXz90yySSe9++oTR6/frV91lURlqTfk1S0i9bYuz47t3hH2PJ6pF1kTv84Adw333hprUbbog7omDLFvjOd+Avfwl3UP/iF/H1TW/ZAr/9LUydGgrNde4czuovuijcbVxXbd9e/kEhsVzaDHWtWxc/KOzbF87gly3bP0rr2GOLn8H37Bm6BmsrJf066t13wwXeY48NZ/zNm8cdUfX65S/DXbbjx4c7b2uTPXvCXcC//W04APz+9zWbZFevDjekPfggfPZZ+Cb44x+HMhJ14eJoOrjDf/5T8YGhsLB4N80ppxx8F+tTTfoVDuSv6R/dnFWx5593r1fPffjwcHNHXfXQQ+Fmqtxc9717446mdPv2ud95p7uZe9++7h99VP2v9/e/u593Xvg30LCh+8UXu//zn9X7ulL7kcabs6SWOeusMCJj9uww81Zd9OyzoeLlmWeGYau19czVLHQ5PfkkLF4cut+WLq14u8oqLAzXNE47Lcyz/MorYSTXmjXhG0bPnul/Tambaul/JanIddeFroW77qp7xdneeAPOPx969Qp3Lx4MN6WNHBlGzHzxReh+e+ml9Ox327ZQffW440L5582bYdq0cHHxv/879EeLVIaS/kFsypQwdLEuFWdbvjyMjGnXLoyFr44heNUlJydcYM/KCsNL77+/6vtauzb0z7dvH65ndOwY7tB+770wGqdZs7SFLRlGSf8g1qBBqNdxwgmhVMOKFXFHdGAKCsJBrGHDcLftEUfEHVHlHX00vP56eB9XXRUSd2UmxXnrLRg9Olyov/fecABcsABefTXcH1BbhgfKwUtJ/yBXV4qzffppGHWyZUuop3PMMXFHVHUtWoTJcK67LoyZP/fc8qfB3Ls3lHbo3z9cE3jxxXB2//77YXKXnIrHY4ikTEm/DkgUZysoCAmmtHHJtdmuXWFi+JUrw/vo1SvuiA5cgwZh3PzUqeGg/I1vfHni+x07wv0HJ5wQ/m4FBaHLbv36MF/v0UeXvm+RA6GkX0ckirPNmwdXXnnwlAEuLAwlJv7+93C37eDBcUeUXtddF0ZZrVwZzuIXLQrJfcKEkNSvvz7cLfvEE6HND35wcF3HkINPWqtsSrwuvDAUZ7vttlAMrLYP53QPI5CefjqcEZ9/ftwRVY9hw0I//7e+FQ7Oe/aEO0DPOy/cOXvaaXFHKJlEZ/p1zC23hAuBP/lJGDtem912G0yfHmK97rq4o6lePXrsr5BaWBgqMY4YoYQvNU9n+nWMWSiju2ZNmDc2KyvcUp64tbx379rRffB//xeS/mWXVa7M8cEqLy8k/V27wvKGDaHkM1S94qdIVaj2Th316aehJkuiFOyaNWG9GXTpsv8gcMopNT+hw5/+FA5Iw4aFUSsNMuDUIyur9MlcOnbc/7cRORAquCbFbNwI+fn7DwILFoRJLyDc8dqjx/6DQJ8+4ZpAdYwJf/XVMIa9d+9QobI2Vy1Mp3r1Sr+4bhb690UOlJK+lMs9DA1MPgjk5++vUd68eajZn3wg6NjxwCaHWLIkjEU/6qhQIbR16/S8l4OBzvSluqWa9DPgi7WUxizMItWhQ+hqgXDGuWLF/oPAW2+FUTWJcf9t2hS/PnDKKanfNbt2bbj5qnnzcPNRJiV8SM+0jSLpkFLSN7OhwL2EmbMecPfbSzw/HrgCKAQ2At9197VmNgi4J6lpZ2C0uz+djuAlverVC/39XbrAxReHdbt3hzP0xEFgwYJQIiHRJdGhQ/GDwMknh4mvk23aFLp0du4MZ/ipTkpelyQu1qZr2kaRqkpljtz6hDlyzwAKCHPkjnH3ZUltBgHz3X2nmV0NDHT3C0rs53BgFdDe3XdSBnXv1H47dsA//1m8a2j16vCcWZi5KXEQ6N07lBRYtAhefjl074hI+qWze6cPsMrdV0c7ngmMAIqSvrvPTWr/JlDaLKajgBfKS/hycGjePCTv5AS+aVO4JpA4CMyZA3/4Q3iuXr1wz4ASvkj8Ukn67YD1ScsFQN9y2l8OvFDK+tHA3aVtYGbjgHEAHTLxu38d0KZN6LMfOjQsu4dyAwsWhAu3p54ab3wiEqT1Qq6ZjQVygAEl1n8V6AbMKW07d58OTIfQvZPOmCQeZqG2jIqGidQuqZRh+ABI/q/bPlpXjJkNASYCw939ixJPnw885e57qhqoiKRHXl4YQlqvXvidlxd3RFKTUkn6C4DjzayTmTUidNPMTm5gZr2A+wkJ/5NS9jEGeOxAgxWRA5OXF4aOrl0buuDWrg3LSvyZo8Kk7+6FwLWErpnlwCx3X2pmk8xseNTsTqA58EczW2RmRQcFM8sifFN4Nc2xi0glTZxY/F4BCMsTJ8YTj9Q83ZErkkFUDqLuSnXIpkori2SQsgbHadBc5lDSF8kgkyd/ucidykFkFiV9kQySmxsmrkkUz+vYMSyrHETmUME1kQyTm6skn8l0pi8ikkGU9EVEMoiSvohIBlHSFxHJIEr6IiIZRElfRGqcir7FR0M2RaRGJYq+JWoAJYq+gYaS1gSd6YtIjVLRt3gp6YtIjVq3rnLrJb2U9EWkRqnoW7yU9EWkRqnoW7xSSvpmNtTMVpjZKjObUMrz481smZktNrO/mlnHpOc6mNlLZrY8apOVvvBF5GCjom/xqnASFTOrD/wLOAMoIEyfOMbdlyW1GQTMd/edZnY1MNDdL4ie+xsw2d1fNrPmwD5331nydRI0iYqISOWlcxKVPsAqd1/t7ruBmcCI5AbuPjcpkb9JmDwdM8sGGrj7y1G7HeUlfBERqV6pJP12wPqk5YJoXVkuB16IHp8AbDGzP5nZP83szuibg4iIxCCtF3LNbCyQQ5goHcLNX/2BG4BTgGOAS0vZbpyZ5ZtZ/saNG9MZkohImTLxzuBUkv4HwNFJy+2jdcWY2RBgIjDc3b+IVhcAi6KuoULgaaB3yW3dfbq757h7Ttu2bSv7HkREKi1xZ/DatWGy+MSdwXU98aeS9BcAx5tZJzNrBIwGZic3MLNewP2EhP9JiW1bmVkikw8GliEiErNMvTO4wqQfnaFfC8wBlgOz3H2pmU0ys+FRszuB5sAfzWyRmc2Ott1L6Nr5q5ktAQz4bTW8DxGRSsnUO4NTKrjm7s8Dz5dY97Okx0PK2fZloHtVAxQRqQ4dOoQundLW12W6I1dEMlKm3hmspC8iGSlT7wxWPX0RyVi5uXU/yZekM30RkQyipC8ikkGU9EVEMoiSvohIBlHSFxHJIEr6IiIZRElfRCSDKOmLiMSsJks86+YsEZEYJUo8Jyp+Jko8Q/XcOKYzfRGRGNV0iWclfRGRGNV0iWclfRGRGJVVyrm6Sjwr6YuIxKimSzynlPTNbKiZrTCzVWY2oZTnx5vZMjNbbGZ/NbOOSc/tjWbTKppRS0REgpou8WzuXn4Ds/rAv4AzCBOdLwDGuPuypDaDgPnuvtPMrgYGuvsF0XM73L15qgHl5OR4fn5+5d+JiEgGM7OF7p5TUbtUzvT7AKvcfbW77wZmAiOSG7j7XHdPXH9+E2hf2YBFRKT6pZL02wHrk5YLonVluRx4IWm5iZnlm9mbZvbt0jYws3FRm/yNGzemEJKIiFRFWm/OMrOxQA4wIGl1R3f/wMyOAV4xsyXu/u/k7dx9OjAdQvdOOmMSEZH9UjnT/wA4Omm5fbSuGDMbAkwEhrv7F4n17v5B9Hs18Deg1wHEKyIiByCVpL8AON7MOplZI2A0UGwUjpn1Au4nJPxPktYfZmaNo8dtgH7AMkREJBYVjt4BMLOzgSlAfeAhd59sZpOAfHefbWZ/AboBH0abrHP34Wb2NcLBYB/hADPF3R+s4LU2Amur/I5qhzbApriDqEX0eRSnz2M/fRbFHcjn0dHd21bUKKWkL5VjZvmpDJ3KFPo8itPnsZ8+i+Jq4vPQHbkiIhlESV9EJIMo6VeP6XEHUMvo8yhOn8d++iyKq/bPQ336IiIZRGf6IiIZRElfRCSDKOmnkZkdbWZzozLTS83sB3HHFDczq29m/zSzZ+OOJW5m1srMnjCz98xsuZmdFndMcTKzH0X/T941s8fMrEncMdUkM3vIzD4xs3eT1h1uZi+b2cro92Hpfl0l/fQqBH7s7tnAqcA1ZpYdc0xx+wGwPO4gaol7gRfdvTPQgwz+XMysHXA9kOPuJxFu/Bwdb1Q17nfA0BLrJgB/dffjgb9Gy2mlpJ9G7v6hu78dPd5O+E9dXkXSOs3M2gPDgAfijiVuZtYS+AbwIIC773b3LfFGFbsGwCFm1gBoCmyIOZ4a5e7zgP+UWD0C+H30+PdAqZWJD4SSfjUxsyxCcbn58UYSqynAfxHKcGS6TsBG4OGou+sBM2sWd1BxiQox3gWsI5Rv2eruL8UbVa3wFXdPlLP5CPhKul9ASb8amFlz4Engh+6+Le544mBm3wI+cfeFccdSSzQAegO/cfdewGdUw1f3g0XUVz2CcDA8CmgWlWaXiIfx9GkfU6+kn2Zm1pCQ8PPc/U9xxxOjfsBwM1tDmG1tsJnNiDekWBUABe6e+Ob3BOEgkKmGAO+7+0Z33wP8CfhazDHVBh+b2VcBot+fVNC+0pT008jMjNBnu9zd7447nji5+0/cvb27ZxEu0L3i7hl7JufuHwHrzezEaNXpZHaZ8XXAqWbWNPp/czoZfGE7yWzgkujxJcAz6X4BJf306gdcRDirXRT9nB13UFJrXAfkmdlioCfw3zHHE5voG88TwNvAEkIuyqiSDGb2GPAGcKKZFZjZ5cDtwBlmtpLwbej2tL+uyjCIiGQOnemLiGQQJX0RkQyipC8ikkGU9EVEMoiSvohIBlHSFxHJIEr6IiIZ5P8Dyzkt84aLahUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss) + 1)\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "검증 손실로 비교해 보면 이 설정은 규제가 있는 GRU 모델만큼 좋지는 않습니다. 하지만 훨씬 빠르기 때문에 데이터를 두 배 더 많이 처리할 수 있습니다. 여기서는 큰 도움이 안 되었지만 다른 데이터셋에서는 중요할 수 있습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.4.5 정리\n",
    "\n",
    "다음은 이번 절에서 배운 것들입니다.\n",
    "\n",
    "- 2D 컨브넷이 2D 공간의 시각적 패턴을 잘 처리하는 것과 같이 1D 컨브넷은 시간에 따른 패턴을 잘 처리합니다. 1D 컨브넷은 특정 자연어 처리 같은 일부 문제에 RNN을 대신할 수 있는 빠른 모델입니다.\n",
    "\n",
    "- 전형적으로 1D 컨브넷은 컴퓨터 비전 분야의 2D 컨브넷과 비슷하게 구성합니다. Conv1D 층과 Max-Pooling1D 층을 쌓고 마지막에 전역 풀링 연산이나 Flatten 층을 둡니다.\n",
    "\n",
    "- RNN으로 아주 긴 시퀀스를 처리하려면 계산 비용이 많이 듭니다. 1D 컨브넷은 비용이 적게 듭니다. 따라서 1D 컨브넷을 RNN 이전의 전처리 단계로 사용하는 것은 좋은 생각입니다. 시퀀스 길이를 줄이고 RNN이 처리할 유용한 표현을 추출해 줄 것입니다.\n",
    "\n",
    "유용하고 중요한 개념이지만 여기서 다루지 않은 것은 팽창 커널을 사용한 1D 합성곱입니다."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "tensorflow",
   "language": "python",
   "name": "tensorflow"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}