17100050_SentimentAnalysis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "490FWZrsRT4e"
   },
   "source": [
    "# Sentiment Analysis Using Simple RNN, LSTM, GRU\n",
    "\n",
    "\n",
    "---\n",
    "\n",
    "\n",
    "Submitted by :- Saurabh Roy\n",
    "\n",
    "---\n",
    "Guided by :- Dr. Muneendra Ojha\n",
    "\n",
    "\n",
    "---\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_F-ZEaTpBhOx"
   },
   "source": [
    "#Enabling and Checking for the GPU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 110
    },
    "colab_type": "code",
    "id": "M-eFuJp_BrYa",
    "outputId": "b2e85827-090e-4a90-f5de-e9d73a8e26c8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From <ipython-input-1-a32398aaaac2>:3: is_gpu_available (from tensorflow.python.framework.test_util) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use `tf.config.list_physical_devices('GPU')` instead.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 1,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "tf.test.is_gpu_available(\n",
    "    cuda_only=False, min_cuda_compute_capability=None\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "AUoIppa_RiiF"
   },
   "source": [
    "## Installing\n",
    "- Graphviz and Hierarchymagic for drawing graphs\n",
    "- Keras\n",
    "- Bokeh for visualizations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "colab_type": "code",
    "id": "fsjcKzx8Rpfn",
    "outputId": "eb4dc8ff-3ceb-4408-e9b9-6e4fcf9b74d1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading package lists...\n",
      "Building dependency tree...\n",
      "Reading state information...\n",
      "graphviz is already the newest version (2.40.1-2).\n",
      "0 upgraded, 0 newly installed, 0 to remove and 25 not upgraded.\n",
      "Requirement already satisfied: keras in /usr/local/lib/python3.6/dist-packages (2.2.5)\n",
      "Requirement already satisfied: numpy>=1.9.1 in /usr/local/lib/python3.6/dist-packages (from keras) (1.18.2)\n",
      "Requirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from keras) (2.10.0)\n",
      "Requirement already satisfied: pyyaml in /usr/local/lib/python3.6/dist-packages (from keras) (3.13)\n",
      "Requirement already satisfied: keras-preprocessing>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from keras) (1.1.0)\n",
      "Requirement already satisfied: keras-applications>=1.0.8 in /usr/local/lib/python3.6/dist-packages (from keras) (1.0.8)\n",
      "Requirement already satisfied: scipy>=0.14 in /usr/local/lib/python3.6/dist-packages (from keras) (1.4.1)\n",
      "Requirement already satisfied: six>=1.9.0 in /usr/local/lib/python3.6/dist-packages (from keras) (1.12.0)\n",
      "Requirement already satisfied: bokeh in /usr/local/lib/python3.6/dist-packages (1.4.0)\n",
      "Requirement already satisfied: numpy>=1.7.1 in /usr/local/lib/python3.6/dist-packages (from bokeh) (1.18.2)\n",
      "Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.6/dist-packages (from bokeh) (2.8.1)\n",
      "Requirement already satisfied: PyYAML>=3.10 in /usr/local/lib/python3.6/dist-packages (from bokeh) (3.13)\n",
      "Requirement already satisfied: six>=1.5.2 in /usr/local/lib/python3.6/dist-packages (from bokeh) (1.12.0)\n",
      "Requirement already satisfied: tornado>=4.3 in /usr/local/lib/python3.6/dist-packages (from bokeh) (4.5.3)\n",
      "Requirement already satisfied: pillow>=4.0 in /usr/local/lib/python3.6/dist-packages (from bokeh) (7.0.0)\n",
      "Requirement already satisfied: Jinja2>=2.7 in /usr/local/lib/python3.6/dist-packages (from bokeh) (2.11.1)\n",
      "Requirement already satisfied: packaging>=16.8 in /usr/local/lib/python3.6/dist-packages (from bokeh) (20.3)\n",
      "Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.6/dist-packages (from Jinja2>=2.7->bokeh) (1.1.1)\n",
      "Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.6/dist-packages (from packaging>=16.8->bokeh) (2.4.6)\n",
      "Collecting git+https://github.com/tla/ipython-hierarchymagic\n",
      "  Cloning https://github.com/tla/ipython-hierarchymagic to /tmp/pip-req-build-jyph04v6\n",
      "Requirement already satisfied: sphinx in /usr/local/lib/python3.6/dist-packages (from ipython-hierarchymagic==0.1) (1.8.5)\n",
      "Requirement already satisfied: Pygments>=2.0 in /usr/local/lib/python3.6/dist-packages (from sphinx->ipython-hierarchymagic==0.1) (2.1.3)\n",
      "Requirement already satisfied: setuptools in /usr/local/lib/python3.6/dist-packages (from sphinx->ipython-hierarchymagic==0.1) (46.0.0)\n",
      "Requirement already satisfied: docutils>=0.11 in /usr/local/lib/python3.6/dist-packages (from sphinx->ipython-hierarchymagic==0.1) (0.15.2)\n",
      "Requirement already satisfied: imagesize in /usr/local/lib/python3.6/dist-packages (from sphinx->ipython-hierarchymagic==0.1) (1.2.0)\n",
      "Requirement already satisfied: Jinja2>=2.3 in /usr/local/lib/python3.6/dist-packages (from sphinx->ipython-hierarchymagic==0.1) (2.11.1)\n",
      "Requirement already satisfied: snowballstemmer>=1.1 in /usr/local/lib/python3.6/dist-packages (from sphinx->ipython-hierarchymagic==0.1) (2.0.0)\n",
      "Requirement already satisfied: babel!=2.0,>=1.3 in /usr/local/lib/python3.6/dist-packages (from sphinx->ipython-hierarchymagic==0.1) (2.8.0)\n",
      "Requirement already satisfied: alabaster<0.8,>=0.7 in /usr/local/lib/python3.6/dist-packages (from sphinx->ipython-hierarchymagic==0.1) (0.7.12)\n",
      "Requirement already satisfied: sphinxcontrib-websupport in /usr/local/lib/python3.6/dist-packages (from sphinx->ipython-hierarchymagic==0.1) (1.2.1)\n",
      "Requirement already satisfied: packaging in /usr/local/lib/python3.6/dist-packages (from sphinx->ipython-hierarchymagic==0.1) (20.3)\n",
      "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.6/dist-packages (from sphinx->ipython-hierarchymagic==0.1) (1.12.0)\n",
      "Requirement already satisfied: requests>=2.0.0 in /usr/local/lib/python3.6/dist-packages (from sphinx->ipython-hierarchymagic==0.1) (2.21.0)\n",
      "Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.6/dist-packages (from Jinja2>=2.3->sphinx->ipython-hierarchymagic==0.1) (1.1.1)\n",
      "Requirement already satisfied: pytz>=2015.7 in /usr/local/lib/python3.6/dist-packages (from babel!=2.0,>=1.3->sphinx->ipython-hierarchymagic==0.1) (2018.9)\n",
      "Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.6/dist-packages (from packaging->sphinx->ipython-hierarchymagic==0.1) (2.4.6)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests>=2.0.0->sphinx->ipython-hierarchymagic==0.1) (2019.11.28)\n",
      "Requirement already satisfied: urllib3<1.25,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests>=2.0.0->sphinx->ipython-hierarchymagic==0.1) (1.24.3)\n",
      "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests>=2.0.0->sphinx->ipython-hierarchymagic==0.1) (3.0.4)\n",
      "Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests>=2.0.0->sphinx->ipython-hierarchymagic==0.1) (2.8)\n",
      "Building wheels for collected packages: ipython-hierarchymagic\n",
      "  Building wheel for ipython-hierarchymagic (setup.py): started\n",
      "  Building wheel for ipython-hierarchymagic (setup.py): finished with status 'done'\n",
      "  Created wheel for ipython-hierarchymagic: filename=ipython_hierarchymagic-0.1-cp36-none-any.whl size=5025 sha256=19c6ce9968e75f8f4c32fdae2be36fe7764ba0eca9baae229e4511dfe2999d75\n",
      "  Stored in directory: /tmp/pip-ephem-wheel-cache-hmsxujkk/wheels/32/77/29/9bb9c3340e84bcdaa481575d1d18e60857fc184ba89a34e5f6\n",
      "Successfully built ipython-hierarchymagic\n",
      "Installing collected packages: ipython-hierarchymagic\n",
      "Successfully installed ipython-hierarchymagic-0.1\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "WARNING: apt does not have a stable CLI interface. Use with caution in scripts.\n",
      "\n",
      "  Running command git clone -q https://github.com/tla/ipython-hierarchymagic /tmp/pip-req-build-jyph04v6\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "apt install graphviz\n",
    "pip install keras\n",
    "pip install bokeh\n",
    "pip install git+https://github.com/tla/ipython-hierarchymagic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "tSBL163Gnc9M"
   },
   "outputs": [],
   "source": [
    "%load_ext hierarchymagic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "63OYi2oCR4XD"
   },
   "source": [
    "### Checking Bokeh\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "U8mszQ7qR0gs"
   },
   "outputs": [],
   "source": [
    "from bokeh.plotting import figure, show\n",
    "from bokeh.io import output_notebook, push_notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "NmOsAGEWSWBz"
   },
   "outputs": [],
   "source": [
    "output_notebook()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "IIc2fG6MDpG8"
   },
   "source": [
    "###Importing Numpy, Pandas, Keras"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "tbj23rw5SXfm",
    "outputId": "d63f79b4-2a6a-40d3-e77f-ef454378f5c8"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import keras"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "m_OHiSh5Sl4x"
   },
   "source": [
    "### Sentiment Analysis Data\n",
    "Keras IMDB dataset is used here.This dataset consist of 25,000 movies reviews from IMDB, labeled by sentiment (positive/negative).[Dataset Link](https://keras.io/datasets/#imdb-movie-reviews-sentiment-classification)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "eUZXDcDYUd3b"
   },
   "outputs": [],
   "source": [
    "from keras.datasets import imdb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "cellView": "form",
    "colab": {},
    "colab_type": "code",
    "id": "zg9XOElTUiu5"
   },
   "outputs": [],
   "source": [
    "#@title ### Dataset Parameters\n",
    "#@markdown ---\n",
    "#@markdown #### Number of words to use in classification\n",
    "num_classification_words = 20000 #@param {type: \"number\"}\n",
    "#@markdown #### The word limit for each entry\n",
    "words_limit = 100 #@param {type: \"number\"}\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 53
    },
    "colab_type": "code",
    "id": "s4nASiKQUxnn",
    "outputId": "c063deba-0824-4048-d2ca-56d39f4f0c0e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://s3.amazonaws.com/text-datasets/imdb.npz\n",
      "17465344/17464789 [==============================] - 0s 0us/step\n"
     ]
    }
   ],
   "source": [
    "(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=num_classification_words)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "9DxwpnHsGPJV"
   },
   "source": [
    "###Dataset Instances"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 164
    },
    "colab_type": "code",
    "id": "m43gz-YGuxaN",
    "outputId": "f7bb8154-a756-4d13-c046-19ec0de043b5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://s3.amazonaws.com/text-datasets/imdb_word_index.json\n",
      "1646592/1641221 [==============================] - 0s 0us/step\n",
      "REVIEW 1 \t <START> this film was just brilliant casting location scenery story direction everyone's really suited the part they played and you could just imagine being there robert <UNK> is an amazing actor and now the same being director <UNK> father came from the same scottish island as myself so i loved the fact there was a real connection with this film the witty remarks throughout the film were great it was just brilliant so much that i bought the film as soon as it was released for retail and would recommend it to everyone to watch and the fly fishing was amazing really cried at the end it was so sad and you know what they say if you cry at a film it must have been good and this definitely was also congratulations to the two little boy's that played the <UNK> of norman and paul they were just brilliant children are often left out of the praising list i think because the stars that play them all grown up are such a big profile for the whole film but these children are amazing and should be praised for what they have done don't you think the whole story was so lovely because it was true and was someone's life after all that was shared with us all\n",
      "REVIEW 2 \t <START> big hair big boobs bad music and a giant safety pin these are the words to best describe this terrible movie i love cheesy horror movies and i've seen hundreds but this had got to be on of the worst ever made the plot is paper thin and ridiculous the acting is an abomination the script is completely laughable the best is the end showdown with the cop and how he worked out who the killer is it's just so damn terribly written the clothes are sickening and funny in equal measures the hair is big lots of boobs bounce men wear those cut tee shirts that show off their <UNK> sickening that men actually wore them and the music is just <UNK> trash that plays over and over again in almost every scene there is trashy music boobs and <UNK> taking away bodies and the gym still doesn't close for <UNK> all joking aside this is a truly bad film whose only charm is to look back on the disaster that was the 80's and have a good old laugh at how bad everything was back then\n",
      "REVIEW 3 \t <START> this has to be one of the worst films of the 1990s when my friends i were watching this film being the target audience it was aimed at we just sat watched the first half an hour with our jaws touching the floor at how bad it really was the rest of the time everyone else in the theatre just started talking to each other leaving or generally crying into their popcorn that they actually paid money they had <UNK> working to watch this feeble excuse for a film it must have looked like a great idea on paper but on film it looks like no one in the film has a clue what is going on crap acting crap costumes i can't get across how <UNK> this is to watch save yourself an hour a bit of your life\n",
      "REVIEW 4 \t <START> the scots excel at storytelling the traditional sort many years after the event i can still see in my mind's eye an elderly lady my friend's mother retelling the battle of <UNK> she makes the characters come alive her passion is that of an eye witness one to the events on the <UNK> heath a mile or so from where she lives br br of course it happened many years before she was born but you wouldn't guess from the way she tells it the same story is told in bars the length and <UNK> of scotland as i discussed it with a friend one night in <UNK> a local cut in to give his version the discussion continued to closing time br br stories passed down like this become part of our being who doesn't remember the stories our parents told us when we were children they become our invisible world and as we grow older they maybe still serve as inspiration or as an emotional reservoir fact and fiction blend with <UNK> role models warning stories <UNK> magic and mystery br br my name is <UNK> like my grandfather and his grandfather before him our protagonist introduces himself to us and also introduces the story that stretches back through generations it produces stories within stories stories that evoke the impenetrable wonder of scotland its rugged mountains shrouded in <UNK> the stuff of legend yet <UNK> is rooted in reality this is what gives it its special charm it has a rough beauty and authenticity tempered with some of the finest gaelic singing you will ever hear br br <UNK> angus visits his grandfather in hospital shortly before his death he burns with frustration part of him yearns to be in the twenty first century to hang out in <UNK> but he is raised on the western <UNK> among a gaelic speaking community br br yet there is a deeper conflict within him he yearns to know the truth the truth behind his <UNK> ancient stories where does fiction end and he wants to know the truth behind the death of his parents br br he is pulled to make a last fateful journey to the <UNK> of one of <UNK> most <UNK> mountains can the truth be told or is it all in stories br br in this story about stories we revisit bloody battles poisoned lovers the folklore of old and the sometimes more treacherous folklore of accepted truth in doing so we each connect with angus as he lives the story of his own life br br <UNK> the <UNK> pinnacle is probably the most honest unpretentious and genuinely beautiful film of scotland ever made like angus i got slightly annoyed with the pretext of hanging stories on more stories but also like angus i <UNK> this once i saw the <UNK> picture ' forget the box office pastiche of braveheart and its like you might even <UNK> the justly famous <UNK> of the wicker man to see a film that is true to scotland this one is probably unique if you maybe <UNK> on it deeply enough you might even re evaluate the power of storytelling and the age old question of whether there are some truths that cannot be told but only experienced\n",
      "REVIEW 5 \t <START> worst mistake of my life br br i picked this movie up at target for 5 because i figured hey it's sandler i can get some cheap laughs i was wrong completely wrong mid way through the film all three of my friends were asleep and i was still suffering worst plot worst script worst movie i have ever seen i wanted to hit my head up against a wall for an hour then i'd stop and you know why because it felt damn good upon bashing my head in i stuck that damn movie in the <UNK> and watched it burn and that felt better than anything else i've ever done it took american psycho army of darkness and kill bill just to get over that crap i hate you sandler for actually going through with this and ruining a whole day of my life\n"
     ]
    }
   ],
   "source": [
    "word_to_id = keras.datasets.imdb.get_word_index()\n",
    "INDEX_FROM=3\n",
    "word_to_id = {k:(v+INDEX_FROM) for k,v in word_to_id.items()}\n",
    "word_to_id[\"<PAD>\"] = 0     #Padding\n",
    "word_to_id[\"<START>\"] = 1   #Start Index\n",
    "word_to_id[\"<UNK>\"] = 2     #Unkown Words\n",
    "id_to_word = {value:key for key,value in word_to_id.items()}\n",
    "for i in range(5):\n",
    "  print(\"REVIEW\",str(i+1),\"\\t\",' '.join(id_to_word[id] for id in x_train[i] ))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "pZY-T6fnfkrL"
   },
   "source": [
    "#### Data Preprocessing\n",
    "The RNN will take sequences of constant length. This length is the `words_limit` which is defined to be 100. Since the reviews differ heavily in terms of lengths, I will trim each review to its first 100 words. If reviews are shorter than 100 words I will pad them with zeros."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "G5rn3G_dmRN_"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import random\n",
    "import json\n",
    "from six.moves import range\n",
    "import six"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "1xGzfrLNfj5R"
   },
   "outputs": [],
   "source": [
    "def pad_sequences(sequences, maxlen=None):\n",
    "    \n",
    "    dtype='int32';padding='pre';truncating='pre';value=0.\n",
    "    num_samples = len(sequences)\n",
    "\n",
    "    lengths = []\n",
    "    sample_shape = ()\n",
    "    flag = True\n",
    "\n",
    "    # take the sample shape from the first non empty sequence\n",
    "    # checking for consistency in the main loop below.\n",
    "\n",
    "    for x in sequences:\n",
    "        try:\n",
    "            lengths.append(len(x))\n",
    "            if flag and len(x):\n",
    "                sample_shape = np.asarray(x).shape[1:]\n",
    "                flag = False\n",
    "        except TypeError:\n",
    "            raise ValueError('`sequences` must be a list of iterables. '\n",
    "                             'Found non-iterable: ' + str(x))\n",
    "\n",
    "    if maxlen is None:\n",
    "        maxlen = np.max(lengths)\n",
    "\n",
    "    is_dtype_str = np.issubdtype(dtype, np.str_) or np.issubdtype(dtype, np.unicode_)\n",
    "\n",
    "    x = np.full((num_samples, maxlen) + sample_shape, value, dtype=dtype)\n",
    "    for idx, s in enumerate(sequences):\n",
    "        if not len(s):\n",
    "            continue  # empty list/array was found\n",
    "        if truncating == 'pre':\n",
    "            trunc = s[-maxlen:]\n",
    "        elif truncating == 'post':\n",
    "            trunc = s[:maxlen]\n",
    "        trunc = np.asarray(trunc, dtype=dtype)\n",
    "\n",
    "        if padding == 'post':\n",
    "            x[idx, :len(trunc)] = trunc\n",
    "        elif padding == 'pre':\n",
    "            x[idx, -len(trunc):] = trunc\n",
    "    return x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 53
    },
    "colab_type": "code",
    "id": "eDWse0BSfj_E",
    "outputId": "67449bd0-7dd2-4f78-c84b-7d6498db9fff"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train shape: (25000, 100)\n",
      "test shape: (25000, 100)\n"
     ]
    }
   ],
   "source": [
    "x_train_seq = pad_sequences(x_train, maxlen=words_limit)\n",
    "x_test_seq = pad_sequences(x_test, maxlen=words_limit)\n",
    "\n",
    "print('train shape:', x_train_seq.shape)\n",
    "print('test shape:', x_test_seq.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 144
    },
    "colab_type": "code",
    "id": "tlkZTs3kvvT7",
    "outputId": "b070266e-a1b2-4483-80ca-13c2769ab434"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1415,   33,    6, ...,   19,  178,   32],\n",
       "       [ 163,   11, 3215, ...,   16,  145,   95],\n",
       "       [1301,    4, 1873, ...,    7,  129,  113],\n",
       "       ...,\n",
       "       [  11,    6, 4065, ...,    4, 3586,    2],\n",
       "       [ 100, 2198,    8, ...,   12,    9,   23],\n",
       "       [  78, 1099,   17, ...,  204,  131,    9]], dtype=int32)"
      ]
     },
     "execution_count": 23,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_train_seq"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "vnFjkUXIVjU9"
   },
   "source": [
    "Keras Models and Layers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "LZKhJIEhO_p9"
   },
   "outputs": [],
   "source": [
    "from tensorflow.keras.models import Sequential, Model\n",
    "from tensorflow.keras.layers import Embedding, SimpleRNN, Dense, Dropout, Activation, Input, LSTM, GRU"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Bhs9axrqxfAf"
   },
   "source": [
    "## RNN Model for Sentiment Analysis\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 184
    },
    "colab_type": "code",
    "id": "knaAJVhEqE30",
    "outputId": "20c46bb8-2df3-4ad0-e833-86026c570c0c"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA5gAAACnCAYAAAB0DPNYAAAABmJLR0QA/wD/AP+gvaeTAAAgAElE\nQVR4nOzdd1gUV/s38O9Sl7a79C6gFAUEFSwUTUxQsaSAUXkeezRi1KjRJBo1hhiMMRpNfGOJwVgf\n0aigxqgRbFEUVBCQXgREaaKUhV3qnvcPf8wlERUQWMD7c11zuWVmzn1kd2fuOWV4jDEGQgghhBBC\nCCHkFSnIOwBCCCGEEEIIId0DJZiEEEIIIYQQQtoEJZiEEEIIIYQQQtqEkrwDIIQQQjoLmUyGsrIy\nlJaWoqysDLW1tSgrK+Per66uhkQi4Z7z+Xyoqalxz7W0tKCsrAyRSASRSASBQAAlJTrUEkIIeX3Q\nUY8QQki3V1hYiIyMDDx48AD5+fkoKChAXl4e929JSQnKy8shFovbvGwNDQ0IBAJoa2vDyMgIJiYm\nMDQ0hKmpKQwNDWFubg4rKyuYmpqCx+O1efmEEEJIR+LRLLKEEEK6g+rqaiQmJiI+Ph6pqanIyMhA\nRkYGMjMzucRRUVERBgYGzyR62traEAqFEAgEEIlE3OOG1siGxE9ZWRmamppcmRKJBNXV1dzz0tJS\n1NXVoby8nEtay8vLUVZWhpKSEi6hLSwsxIMHD1BUVITa2loAT1pDra2t0atXL/Tq1Qt2dnZwcnKC\no6NjozIJIYSQzowSTEIIIV2OVCrFzZs3cf36dcTFxXFJZV1dHdTU1GBnZwdra2suYWv418TEBIqK\nivIOv5H8/HxkZmZyCXFDUpyamgqxWAwFBQX07NkT/fr1g5OTEwYPHgw3NzdoaWnJO3RCCCHkGZRg\nEkII6fQeP36My5cv4+rVq7h27Rqio6NRW1sLU1NT9O/fH05OTnB2doaTkxNsbGw6XRLZGowx3L17\nl0ug4+PjERsbi6ysLCgqKqJv377w9PSEu7s7hg8fDiMjI3mHTAghhFCCSQghpPORyWS4ffs2wsPD\nER4ejsuXL0Mmk8HOzg4uLi7w9PSEh4cHHBwc5B1qhyssLMSNGzcQERGBq1ev4tatW6iurkbPnj0x\nbtw4vPPOOxg2bBhUVFTkHSohhJDXECWYhBBCOgWZTIbLly/j0KFDCA0NxcOHD2FmZgZvb294e3vD\ny8sLQqFQ3mF2OhKJBJcuXcKZM2dw9uxZZGRkQCgUYty4cfDz88PIkSMp2SSEENJhKMEkhBAiV9HR\n0Thw4AD++OMP5OXloV+/fpg0aRLGjh2Lvn37yju8LicjIwOnT5/GH3/8gWvXrkFbWxu+vr6YPHky\n3njjDZqplhBCSLuiBJMQQkiHq66uxsmTJ7Fz506Eh4fD0tISkyZNwvTp09GnTx95h9dt3L9/H8eO\nHcORI0cQEREBW1tbfPjhh5gzZw60tbXlHR4hhJBuiBJMQgghHaa4uBibNm3Cb7/9hvLycvj4+GDe\nvHkYNmyYvEPr9mJjY7Ft2zYcPHgQCgoKmDZtGpYtWwZzc3N5h0YIIaQboQSTEEJIuyspKcGPP/6I\nLVu2QE1NDQsWLMBHH31EM5/KQWlpKfbs2YOff/4Z+fn5mD17Nr788kuYmprKOzRCCCHdACWYhBBC\n2k1tbS1++uknrF27FkpKSvjss8/wySefQENDQ96hvfZqamrw+++/47vvvsPDhw+xaNEirF69Gurq\n6vIOjRBCSBdGCSYhhJB2cf36dcydOxdpaWlYtmwZli5dCi0tLXmHRf6luroav/76K77++mtoa2tj\n69atGD16tLzDIoQQ0kUpyDsAQggh3UttbS2WLFkCT09PGBgY4M6dOwgICKDkspNSVVXFwoULkZyc\njEGDBmHMmDGYOnUqKisr5R0aIYSQLohaMAkhhLSZgoICTJw4ETExMdi2bRumTZsm75BIC505cwbT\np0+HoaEhQkJCYGNjI++QCCGEdCHUgkkIIaRNxMXFwdXVFQUFBYiMjKTksosaPXo0oqOjoaamhoED\nB+L8+fPyDokQQkgXQgkmIYSQV5aUlIQRI0bAzs4ON27cgKOjo7xDIq/A3NwcV65cwdixY/Huu+/i\nn3/+kXdIhBBCughKMAkhhLySzMxMeHl5oXfv3jh58iREIpG8Q+LMnDkTfD4fPB4PVVVV8g6nTc2e\nPRtaWlrg8XiIjY1t8/2rqqpi3759GDduHMaNG4cbN260eRmEEEK6HxqDSQghpNXq6+vh4eGB2tpa\nXLp0qVNO5LNq1SqsXbsWUqkUfD5f3uG0qUOHDuE///kPbt++jX79+rVLGbW1tXjvvfeQkpKCuLi4\nTvk3JoQQ0nlQCyYhhJBWW79+PWJjY7Fv375OkXhIpVK4u7vLO4xuRVlZGfv27YNEIsFnn30m73AI\nIYR0cpRgEkIIaZUHDx7g22+/xbfffgsHBwd5hwMA2LVrF4qKipp8j8fjdXA07a+j6qSnp4dffvkF\nv/32G27evNkhZRJCCOmaKMEkhBDSKr/88gt0dXWxaNGidi1n27Zt0NDQgLq6Ok6cOIHRo0dDIBDA\nzMwMwcHB3HqLFy/G0qVLkZmZCR6PB2tra+49BQUF/PXXXxg9ejSEQiGMjY3x+++/tzgWV1dX8Hg8\n8Hg8ODk5ITc3t8n1AgICoKOjAz6fj8DAQAAAYwybNm1Cnz59oKqqCm1tbbz//vtISUnhtvvhhx+g\nrq4OLS0tFBUVYenSpTA1NUVqaioYY9iwYQPs7OygqqoKoVCIzz//vMV1aK0PPvgArq6u+PHHHzus\nTEIIIV0PJZiEEEJa5Y8//sC0adOgoqLSruXMmzcPn376KaRSKbS0tHD48GFkZmaiZ8+e+Oijj1Bb\nWwsA+Omnn/DOO++gV69eYIwhIyOD24dMJoNIJMKhQ4eQnZ2NAQMGYN68eZBIJC2K5datW/Dw8IC5\nuTni4uJgbm4OADh16hTs7Oy49QICAvDNN99g9erVWLVqFffal19+iZUrV6KoqAj//PMPcnNzMXTo\nUBQWFgIAvvjiCyxZsgQVFRX47rvvYGVlhSFDhoAxhq+++grLli2Dv78/CgsLUVBQgOXLl7/S/21L\nzZ49G3/++WeL/98IIYS8PijBJIQQ0mJ5eXm4e/cuRowY0aHluru7QyAQQF9fH35+fqisrMS9e/ea\nva1QKIS2tjb8/PxQXV2NrKysFscwY8YM5Obm4tKlS9xre/fuRVpaGq5fv869dvjwYUyZMgXAk7Gh\nmzZtgq+vL6ZMmQKhUIi+fftix44dKC4uxs6dO58p5/vvv8eCBQtw7NgxWFhYYPPmzfDy8sKSJUsg\nEomgpqYGHR2dFsf/KkaOHAmJRILbt293aLmEEEK6DkowCSGEtNjdu3cBoFGrXUdraDltaMFsCWVl\n5VZvO2nSJKirq2Pfvn0AgJKSEmRmZnK39QCA7OxsqKiooEePHgCAxMREVFRUwNXVtdG+Bg4cCBUV\nFURFRb2wzIyMDEgkErz99tstjrctWVpags/nIzMzU65xEEII6bwowSSEENJiUqkUAKCmpibnSDqe\nlpYWfH19cezYMUgkEgQHB2PWrFl45513cPjwYVRXVyM4OJhrvQSA0tJSAICmpuYz+xOJRBCLxS8s\n8/79+wAAfX39NqxJ66irq1MXWUIIIc9FCSYhhJAW09bWBgA8fvxYzpHIx8yZMyEWixEaGorg4GD4\n+flh5syZKCkpwalTp3D8+HF88MEH3PoikQgAmkwkS0tLYWZm9sLyGu7fWV1d3Ya1aLmamhqUl5dD\nV1dXrnEQQgjpvCjBJIQQ0mJ9+vSBoqIioqOj5R2KXAwfPhwWFhYIDAyEgYEBdHV1MWrUKBgbG+Pr\nr7+GlZUVBAIBt76joyM0NTVx69atRvuJiopCTU0NXFxcXlieo6MjFBQUcPny5XapT3PFxcWhrq4O\njo6Oco2DEEJI50UJJiGEkBbT0NCAu7s7Tpw4Ie9QGtHR0UFeXh6ys7MhFotbNcayOXg8HqZPn46U\nlBRMnz4dAKCoqIipU6ciMTERU6dObbQ+n8/H0qVLERISggMHDqC8vBx37tzBxx9/DGNjY/j7+7+w\nPH19fYwfPx5Hjx7Frl27UF5ejvj4+CYnB2pPx48fR48ePdC7d+8OLZcQQkgXwgghhJBW2LVrF1NR\nUWG5ubntWs7WrVuZuro6A8BsbGxYZmYm27lzJxMIBAwAs7CwYGlpaYwxxmJiYpiFhQVTU1Njnp6e\nbMmSJUxNTa3RtgcOHGDa2toMADMzM2MJCQmtiuvu3bvMwMCA1dTUcK8lJyczAwMDVltb+8z6MpmM\nbdiwgdnY2DBlZWWmra3NfHx8WGpqKrfO+vXruXjNzc3Z/v37uffEYjGbPXs209XVZZqamszT05Ot\nXr2aq0dcXFyr6tFclZWVzMDAgK1atapdyyGEENK18RhjTJ4JLiGEkK6ppqYGffr0gZOTE0JDQ+Ud\nDmlnn376Kfbu3YvU1NROMdkQIYSQzom6yBJCCGkVFRUV7Nu3DydPnsSBAwfkHQ5pRxEREdiyZQu2\nbNlCySUhhJAXogSTEEJIq3l4eGDhwoWYN28eIiMj5R1Oq6SkpIDH47108fPzk3eocpGeno4JEybg\nvffea3TrFUIIIaQp1EWWEELIK6mvr8d//vMfnDt3DuHh4XB1dZV3SKSN5ObmYtiwYTA0NERYWBi0\ntLTkHRIhhJBOjlowCSGEvBJFRUXs378fQ4YMwahRoxAWFibvkEgbiImJwdChQyESiXDmzBlKLgkh\nhDQLJZiEEEJemaqqKkJDQzF69GiMHj0a69atA3WQ6br27NkDd3d3WFtbIzw8HNra2vIOiRBCSBdB\nXWQJeQ2VlpZCKpVCKpWipKQEAFBeXo76+npuHcYYSktLn9lWJBKBx+NxzxUVFbkbymtra0NNTQ1q\namoQiUTtXAvSWf3888/4/PPP4eXlhe3bt8PCwkLeIZFmevjwIZYuXYoDBw5AUVERfD4fzs7OcHBw\ngL29PVxcXDBo0CCoqKjIO1TSQpWVlRCLxRCLxSgvL0dNTQ0qKyu59//9XEFBAUKhkHve8FuvoqIC\nLS0tCIVCCIVCKChQWwXpvGQyGcrKylBRUYHKykpUVlaitrYWFRUVjdb79zkQgGcurKmpqYHP54PP\n50NdXR0ikQiamppQVlZu93p0NZRgEtKFPXz4EHl5ecjLy8OjR4+eWYqLi1FcXAyxWIyysjJIpVJI\nJJIOi+/pZFNLSwu6urrQ19eHrq5uo0VHRwcmJiYwNTWlGSq7iYiICMyePRu5ubkICAjA4sWLoaSk\nJO+wyHMwxrB371589tlnUFNTw7Zt22BiYoKYmBhER0cjJiYGd+7cQVVVFdTV1eHk5IQBAwZgwIAB\ncHFxgYODA51kdbC6ujrcu3cPOTk5yMvLw8OHD1FQUICCggLucUlJCUpLS5s8eW4rGhoa0NLSgkgk\ngr6+PgwNDWFkZAQDAwMYGRnB0NAQZmZmsLKyopZw8krq6+uRn5+PnJwcFBUV4eHDh3j48CF3rlNc\nXIyHDx+ipKQEZWVlkEgkkEql7R6XsrIyNDU1uXMdfX19GBgYQE9PD7q6utDT04O+vj6MjIxgamoK\nMzMzqKqqtntc8kQJJiGdVG1tLXJycpCZmYmMjAzcu3cPeXl53L/3799HVVUVt76KisoziVtDMicQ\nCCASiRpddXv6MY/Hg7q6+jM/eAKBAIqKitzzhiuBT6uuroZEIuFaPBtaRktLSyGRSFBVVYXS0lKI\nxWLuAPDvRLi6uprbH5/Ph6mpKUxMTNCjRw+YmprC3Nwc1tbWsLa2hoWFBZ3IdhHV1dVYv3491q1b\nh169emHNmjXw8fFp1AJO5O/ChQv46quvEBUVhfnz5yMwMLDJ8ZZ1dXVITU1FdHQ0t8TGxqKyshLK\nysqwsbGBi4tLo0VNTU0ONeo+6uvrkZmZiYSEBKSkpCArKwt3795FVlYWcnNzUVdXBwBQUlKCgYEB\nDA0NYWxszJ3M6ujoQCQSQSAQQEtLi1uEQmGj3icN+3j67/7vVp6GFs7q6mquFbS0tBQVFRUQi8Uo\nLS1FUVERCgoKUFhYiKKiIuTl5UEsFnP7EIlE6NmzJ6ysrGBlZQVra2s4ODjAwcGBkk8C4MnEYmlp\naUhPT0d2djZyc3Nx79497tyn4TMPAEKhkEvkGhZ9fX3o6OhAKBRCXV2dO8/R0NCAuro6tLS0wOPx\nnull9e9zoPr6epSXlzdap6KiArW1tdzF+rKyMojFYkgkElRWVnLnOg3J79OJb3FxMTdshMfjwcjI\nCObm5txiaWkJGxsb2NrawtLSsstfkKUEkxA5Ky4uRlxcHBISEpCeno6MjAxkZmYiOzub+yHV09Nr\nlGyZmJjA3NycS8TMzMy69AQcFRUVuH//Ppc4NzzOzc3FgwcPkJOTg+LiYgBPToIsLCy4hNPGxgaO\njo5wdnaGnp6enGtCmpKeno6VK1fi2LFjcHJywjfffIN33nmHEk05++eff7B69WpcvnwZI0eOxHff\nfQcXF5cW7aO+vh45OTlITEzkks7r16/j0aNHUFJSgq2tbaOEs3///tDQ0GinGnVtZWVluHHjBqKj\no3Hnzh0kJSUhOTkZ1dXVUFBQgIWFBZecPZ2kWVhYwMjISN7hP5dUKsW9e/eQlZX1zJKens6dxJuY\nmMDBwQGOjo5wcnLC4MGD0bt3b/qd6IZkMhkyMzNx+/Zt3LlzB+np6UhLS0NaWhrXTVskEsHKygrm\n5uawsLBolIxZWFjAwMCgS3XVl8lkKCoqQm5uLrfk5OTg/v37yM3NRVZWFgoLCwE8aTDo2bMnevfu\nDRsbG9jb26Nfv35dqqcIJZiEdBCZTIbk5GTExsYiPj4ecXFxuHPnDvLy8gA8SSLt7Oy4xOnphcYz\nPhk3mpGRwSXgGRkZSE9PR2pqKpd8Ghsbw8nJiVv69esHe3t7GiPUSSQmJuKbb77B0aNHYWNjgw8/\n/BBz5syhlosOVFNTgxMnTmDnzp0IDw+Hh4cHvv32WwwfPrxNy8nLy2vU0hkVFYWHDx9CUVERFhYW\n3HhOFxcXuLu7Q1dXt03L7+wYY0hISMC1a9cQGRmJGzduICUlBTKZDObm5nB0dISjoyPXumdvbw91\ndXV5h90ucnJykJSUhISEBCQmJiIxMREJCQmoqqqCUCjEoEGDMGTIEAwaNAienp50POxiZDIZEhIS\ncOvWLdy+fRuxsbGIi4uDWCyGkpISbGxsuETK1tYWdnZ2sLW1hYGBgbxD73Dl5eVcop2amso9Tk5O\nhlQqhYqKChwdHdG/f39ucXFx6ZTdbSnBJKSdlJeX48aNG7h69Wqjq/oNXcmenjTDwcEBPXv2lHfI\nXVZJSQnXgpKUlITExETExMRAKpVCU1MTzs7OcHFxgaenJ954443X8sDVmcTFxeGXX37BwYMHoaio\niKlTp2L27Nno37+/vEPrttLS0rB7927s2rULpaWl8PHxwSeffAJPT88Oi+HfSeetW7dQUFAA4MnF\noadbOocMGdJu47GPHz+OkSNHdnjClp+fj6tXryI8PBynT5/G/fv3G/0+ubi4YNiwYbC0tOzQuDqj\np7tjR0RE4OrVq0hJSQGPx0O/fv3g5eUFLy8veHp6gs/nyztc8pTa2lrEx8fj6tWriIiIwPnz5/H4\n8eMmu9EPGDCg2144aUtN9RRpuGinpKQEZ2dneHh4wNPTE2+99VanuGBHCSYhbaS0tBQXL15EWFgY\nLly4gNTUVACAra0thgwZgsGDB8PNzQ2Ojo5dpotDV1ZXV4eEhARcv34dkZGRiIqKQlpaGhhjsLW1\nxVtvvQUvLy+89dZb1IImJyUlJdi9eze2b9+OjIwM2NnZwc/PD35+fujdu7e8w+vy7t27h8OHD+PQ\noUOIiYmBiYkJPvroI/j7+8PY2Fje4QFonHQ2XBxKSkoC8GzSOWjQIBgaGr5ymaamppDJZPj5558x\nceLEV97f8zDGEBkZiZCQEBw/fhwZGRlQV1eHp6cn3n77bXh5ecHZ2bnROHfyfMXFxbh06RLOnz+P\n8PBwZGRkQE1NDW+99RZ8fX3x3nvvdYoT69cNYwyxsbE4c+YMzpw5gxs3bqCmpgZmZmYYOnQo3N3d\nMXToUDg6OtJnvY1lZGTg2rVruHLlCiIiIriLMI6OjvD29oa3tzc8PT3lcs5JCSYhrSSTyXDt2jX8\n/fffCAsLw61bt8AYg4uLC7y8vODh4YEhQ4bQAa8Tefz4MSIjIxEREYHw8HBER0cDAPc3GzVqFDw8\nPOgg2MEYY4iKisKhQ4fwxx9/ID8/H87OzhgzZgxGjx4NNze3Lj/hQUeQyWSIiYnB2bNn8ddffyEq\nKgra2trw9fWFn58f3nzzzS7x2X66R0LDkpycDMbYM0lnS3t/FBcXQ19fnxvX5+7ujm3btsHJyalN\nYmeM4cqVKzhy5AhCQ0Px4MEDWFtbw9fXF97e3nB3d++U3dm6opycHISHh+PEiRMICwtDXV0d3njj\nDfj6+mLChAk0I3k7qqysxOnTp3HmzBmcPXsW+fn5MDIywujRo/H2229j6NCh6NGjh7zDfO0UFxfj\n2rVruHjxIk6fPo20tDQIBAJ4eXnB29sb7777bptcpGsOSjAJaYH6+npcv34dR44cwdGjR5GXl4ee\nPXty3XU6S9cE0jwVFRWIjIxEeHg4l3Dq6upizJgxmDBhAkaNGtWlJhHoDmQyGS5fvoxjx47h7Nmz\nyMzMhEgkgpeXF0aOHAlPT0+a+OMpd+/eRUREBMLCwvD333+jqKgIpqam8Pb2xvvvv4+RI0d2i89w\nWVkZ7ty50yjpbBizqK2t3WhMp4uLC+zt7Zv8jPz999/w9vbmnispKaG+vh6TJ0/Gpk2bWp2U5Ofn\nY9++fQgKCkJGRgbs7e3xzjvvYNy4cR3aDfl1JZVKER4ejiNHjuDEiROoqqrCyJEjMW3aNPj4+NAF\nqjZQXV2Nc+fOcRdPpFIp11153Lhx8PDwoN/lTiYrKwthYWEIDw/H2bNnUVlZCTc3N0yYMAH//e9/\n2/UiDCWYhLxEwxXp3bt34+TJk3j8+DH69+8PX19f+Pr6wt7eXt4hkjaSkpKCkJAQhISEIDo6Gtra\n2nj33XcxY8YMvPHGG3TwlIP09HSu69WVK1dQWVkJXV1duLm5wcPDA+7u7ujXr1+j2y10V5WVlbhz\n5w6uX7+OiIgIXLt2Dfn5+VBVVYWbmxtGjx4Nb2/vNmuN6+zKy8sRHx/fZNIpFArh6OjYKOns06cP\n1q9fj4CAANTU1DTal7KyMtTU1LBmzRosWLCg2S29Fy9exObNm3HmzBkIBAJMmTIFs2bNem3+Bp2R\nRCLB0aNHERQUhKtXr8LMzAxz5szBvHnzoKOjI+/wupzIyEjs2LEDx48fh1gsxtChQ+Hn54cPPviA\nZm7vQqRSKU6dOoVDhw7h9OnTkMlk8Pb2xqxZszB27Ng2791CCSYhz/Hw4UPs3bsXQUFBSE1Nhaur\nKyZNmgRfX1+akOc1kJ2djZCQEBw6dAg3b96Era0tZs2ahenTp3dYFxPSWF1dHW7fvo1r164hIiIC\nERERyMvLA4/Hg5WVFZydneHk5ARnZ2fY29vDysqqS7be1dXVIScnB8nJyYiPj+dmnk5PT4dMJoOe\nnh7c3d3h4eEBDw8PuLi40EQn/0csFiM2NhYxMTHckpKSgrq6OohEIhgbGyMtLQ319fVNbq+goIDe\nvXtjx44dGDp0aJPrMMbw559/Yt26dYiMjMSbb74Jf39/+Pj4UPfXTiYtLQ27du1CUFAQamtr4e/v\nj08//RQmJibyDq1Tk0qlOHToELZu3Yro6Gj0798f06ZNw8SJE+n/rhsoLy/H8ePH8b///Q/h4eHo\n0aMH5s6di1mzZrXZRQNKMAn5lzt37uD777/H0aNHoa6ujsmTJ2P27Nno16+fvEMjchIXF4egoCAc\nOHAAlZWV8PX1xZdffglnZ2d5h/bay83NbXTrn7i4OGRkZEAmk0FRURHm5ubo1asXrK2t0atXL5ia\nmsLMzAwGBgYwMzODpqZmh8cskUiQl5eHgoIC5Ofn4/79+9ytdzIzM5GTk4Pa2loAgKWlJZc0Ozs7\no1+/fujVq1eHx9yVSSQSxMfHIyYmBqtXr8ajR49euL6ioiLq6+sxZswYbN++vdFYsjNnzuCLL75A\nYmIi3n33XSxfvhxDhgxp7yqQVyQWi7Fz505s2rQJjx49wpw5cxAQEEAtmv9SVlaGzZs345dffoFY\nLMYHH3yABQsWwM3NTd6hkXaSnp6O7du3Y/fu3aiqqsLUqVOxcuVKWFhYvNJ+KcEk5P/cvHkTa9eu\nxcmTJ+Hk5IRPP/0UEydOhJqamrxDI52EVCrFkSNHsHnzZsTFxWHcuHFYuXIlBg8eLO/QyFMqKyuR\nlpaGzMzMRolbZmYm8vPzueQNANTV1WFqagptbW0IBAJoa2tDKBRCIBBAIBCAz+eDz+c3+h14etbh\nsrIyyGQyAE/GKEkkEtTU1KC8vBzl5eUoKSlBeXk5ysrKUFpairy8PO7G8sCTFjMjIyP06tXrmcXW\n1pbu+deGSktLoaOjg+ae9igrK0NRUREBAQEYO3Ysli9fjr/++gsffPABAgIC4ODg0M4Rk7ZWXV2N\nPXv2ICAgALW1tVizZg38/f27xORX7amyshJbtmzBxo0bwRjDokWLMHfuXOqt8xqprKzEgQMHsH79\nejx48ABz5szBihUrWj3jOCWY5LWXnp6OxYsX4/Tp0xg8eDBWrlyJcePG0Xg78lyMMfz1119Yu3Yt\nIiMj4e3tjZ9++gl2dnbyDo28BGMMRUVFKCwsxP3791FUVIT79++jtLQU5eXlKC0t5R6XlZWhpqYG\nlZWV3Ji9+vr6RgmipqYmNwW8kpIStLS0oKSkBKFQCKFQCJFIxCWsIpEIJiYmMDIygomJCYyNjWFg\nYPDan9x2lIsXL+Ktt95q9voNLZkAwOPxYGFhgd27d+PNN99spwhJRykvL0dgYCB+/vln9OnTB/v3\n70ffvn3lHZZc7N27F1988QWkUikWL16MJUuW0IWt11htbS127dqFtWvX4tnjzPYAACAASURBVPHj\nx/j888+xYsWKlg83YYS8pqRSKVu9ejVTVVVlzs7O7Ny5c/IOiXRBYWFhrH///kxVVZWtWrWKSSQS\neYdECGnCxo0bmaKiIgPALQoKCkxZWZnxeDzuNW1tbebq6somTpzILC0tmYqKCvviiy9YSUmJvKtA\n2lhqaiobOnQoU1NTY9u3b5d3OB0qJyeHeXt7MwUFBbZw4UL28OFDeYdEOhGpVMo2btzINDQ0mKOj\nI4uKimrR9tSCSV5LkZGRmDJlCoqKivDNN9/gk08+oWnMSavV19dj69atWL16NXR0dLB//354eHjI\nOyxCyFMmT56MgwcPAnjSzdna2hoODg6wtraGtbU1bGxsYG1tDYFAgPPnz2PSpEkwMDDA4cOHX9vW\nrddBXV0d1qxZg++++w7jx4/Hvn37uv1kTfv378f8+fNhZmaGXbt20RhL8lxZWVnw9/fHhQsX8MUX\nXyAwMBAKCgov3Y4STPLaCQoKwoIFC/D2229j586dMDU1lXdIpJvIz8/H3LlzcfbsWfz888+YO3eu\nvEMihPyfCxcuQCgUwtraGkKh8LnrNYyzfO+997Br1y5oaGh0YJREXi5evIjx48dj4MCBCA0Nhbq6\nurxDanOMMaxatQrr1q3DkiVLEBgYSDNQk5dijGHXrl345JNP4O3tjQMHDrz8d7HN21QJ6aTq6urY\nxx9/zHg8Hvvqq69YfX29vEPq1GbNmsU0NTUZAHb79m15h9NqHV0PmUzG1qxZwxQUFNhHH33Eamtr\n271MQkjbCA0NZSoqKmzWrFntcow4evQos7KyatRNFwBTVVVllpaWbObMmezu3bsv3GbKlCnP7HfE\niBFMU1OTKSgoMHt7exYdHd3q7VojJSWFLViwgNnb2zNNTU2mqKjIBAIBs7GxYWPGjGHXrl1r1n42\nbNjA9PX1GQCuy2pTr7WXmJgYpqenx958801WWVnZrmV1tOrqajZp0iSmoqLCdu/e3WHlhoWFsfHj\nxzMzMzOmoqLCNDQ0mL29PVu8eDHLzs7usDjkpbt8NxhjLCIigunr6zMXFxdWWFj4wnUpwSSvhfr6\nejZ9+nSmpqbGQkJC5B1OlxEcHNzlE0zG5FOPkydPMg0NDTZ58mS6mEFIF5Camso0NTWZv78/k8lk\n7VpWr169mFAoZIw9OT4VFhayffv2MXV1dWZgYMCKi4ub3EZXV5cBYKdOnXrm/TNnzrD33nuvzbZr\nrqCgIKasrMyGDRvGzp49y0pKSlhVVRXLzMxkhw4dYu7u7uzXX39t9v7S09OfOWFu6rX2kpCQwHR1\nddmcOXPavayOUl9fz/z8/JhAIGAXL17ssHKXLVvGALCZM2ey27dvM6lUysrKytjZs2eZi4sLEwgE\n7Pz58x0WT0frbt8NxhjLzMxkvXr1Yh4eHqy6uvq56728Ey0h3cB3332H4OBghIaGwsfHR97hkNfA\nO++8g+PHj+Po0aP45ptv5B0OIeQFamtrMXXqVNjZ2WHLli0dOou4goICDAwMMHXqVCxYsABFRUUI\nDw9vct0tW7ZAQUEB/v7+KCsra3YZrd3uZSIjI+Hv74+hQ4fi/PnzGDVqFEQiEVRVVdGzZ09MmjQJ\nq1ev5mZh7gocHBywd+9e/Pbbbzh06JC8wwHwZNbbioqKVm+/ceNGhISEIDQ0tMNmQT5x4gTWr1+P\nOXPm4Pfff0e/fv3A5/MhEAgwatQoXLp0CUZGRpg4ceJL703bFKlUCnd393aIvG3K6I7fDQDo2bMn\n/vzzTyQkJGDevHnPXY8STNLt3bx5EwEBAfjxxx8xatQoeYfTpXSXW7XIqx5eXl74+eefERgYiOvX\nr8slBkLIy23cuBFJSUk4dOhQy6fjb0PW1tYAgIKCgibfd3d3x+LFi/HgwQN89tlnzd5va7d7mcDA\nQNTX12PdunXPnShv1KhRWLBgQZuV2RHGjh2LuXPnYsGCBY1uSyQvsbGx0NfXx8SJE3HixAlUV1c3\ne9vU1FR89dVXCAwMbNFtel7Vxo0bAQCrVq1q8n1NTU0sWbIEjx49QlBQUIv3v2vXLhQVFb1SjO1Z\nRnf9bgBAnz59cPDgQezevRt//PFHk+tQgkm6vcWLF2PYsGGYP39+h5d9+fJlDBo0COrq6hAIBOjb\nty93sKqvr8fq1avRo0cPqKmpwcnJCYcPH260/f79++Hq6go+nw8NDQ1YWlri22+/BfBk0PWmTZvQ\np08fqKqqQltbG++//z5SUlK47bdt2wYNDQ2oq6vjxIkTGD16NAQCAczMzBAcHNyoLMYYNmzYADs7\nO6iqqkIoFOLzzz9vVb1dXV3B4/HA4/Hg5OSE3NzcJtcLCAiAjo4O+Hw+AgMDm12vH374Aerq6tDS\n0kJRURGWLl0KU1NTpKamtmk92oK/vz+8vLywaNEiucVACHm+qqoq/Pjjj1iyZAmX4MlLeno6AMDZ\n2fm56wQGBsLW1hZBQUHPbelsy+2ep6amBufPn4euri4GDRrU7O2ac+zrDNauXYu6ujrs2LFD3qEA\nePI5DQkJgY+PD/T09PDhhx/i/Pnz3L1an2flypXo3bs3lixZ0kGRAhKJBJGRkejRowfMzc2fu17D\n7LVhYWEAgIULF0JFRQVGRkbcOvPnz4eGhgZ4PB6Ki4sBPDmvW7p0KTIzM8Hj8WBtbY0tW7aAz+fD\nwMAAc+fOhbGxMfh8Ptzd3REVFcXt71XKaK7u/t0AgDFjxmDatGn48ssvm/4MdkiHXULk5ObNmwwA\nu3r1aoeXXVFRwQQCAVu/fj2TSqWsoKCA+fr6cvea+uyzz5iqqio7evQoKykpYStWrGAKCgrs5s2b\njDHGNm/ezACwdevWsUePHrHHjx+zX3/9lU2ePJkxxtjq1auZiooK279/PystLWXx8fFswIABTE9P\njxUUFHBxrFy5kgFg58+fZ2VlZayoqIgNHTqUaWhosJqamkbr8Xg89uOPP7KSkhImkUjY1q1bWz12\n0cPDg5mbmzcay/Tnn38yW1vbRutt2bKFrV27lnve0notWrSI/b//9/+Yr68vS05ObvN6tIWoqCgG\ngEVERMilfELI84WEhDBFRUWWn5/fYWU+PQaTMcZKSkrYnj17mLq6Ohs7duxzt8nKymKMMXbt2jWm\noKDALC0tWUVFBWPsxWMwW7Pdy6SlpTEAbMiQIS3a7mXHvs4wzqzBvHnzWN++fTu0zKZcvnz5mYmh\nVFRUGAAmFArZnDlz2JUrV54ZO5yfn8+UlJTYwYMHOzTe5ORkBoC5urq+cL3CwkIGgFlZWXGvTZ48\nmRkaGjZab8OGDQxAo3t1jh8/nvXq1avRev7+/kxDQ4MlJSWxqqoqlpiYyAYOHMi0tLTYvXv32qSM\n5ngdvhuMPaknj8djp0+ffuY9asEk3dqpU6dgZWUll3sSZmdno7y8HA4ODuDz+TA0NMSxY8egp6eH\nqqoqbNu2DT4+Phg/fjxEIhFWrVoFZWVl7N69G7W1tfjmm28wfPhwLF++HDo6OtDW1sasWbMwcOBA\nSKVSbNq0Cb6+vpgyZQqEQiH69u2LHTt2oLi4GDt37nwmHnd3dwgEAujr68PPzw+VlZW4d+8egCfj\nDDZv3gwvLy8sWbIEIpEIampq0NHRaXX9Z8yYgdzcXFy6dIl7be/evUhLS2vUXfTw4cOYMmUKF0dL\n6/X9999jwYIFOHbsGCwsLNq8Hm1h0KBBsLOzw6lTp+QaByHkWZcuXcKAAQMatWh0hLKyMq6nh7a2\nNmbOnIkVK1YgNDT0pdu6ubnh008/RXZ2NpYvX97sMlu7XVMaeuNoamo2e5uXHfs6m7FjxyIhIYFr\n1epMGsbulZWVYffu3Rg6dChMTU2xaNEi3L59G8CTW/MoKCjgvffe69DYGsaLCgSCF64nEokAAGKx\nuM3KVlJS4npA2dvbY9u2bRCLxR36+XodvhsAYGNjg/79++Pvv/9+5j26szzp1pKSkuDq6iqXsnv2\n7AkDAwNMmTIFixYtwowZM2BpaQngyZgIiUQCR0dHbn01NTUYGRkhJSUF8fHxKC0tfWbMqKKiIhYt\nWoRbt26hoqLimboNHDgQKioqjbqDNKVhjFFtbS0AICMjAxKJBG+//farVpszadIkLFq0CPv27cPw\n4cNRUlKCzMxMqKqqYt++fXBzc0N2djZUVFTQo0cPAEBiYuIr1as96tFWXF1dkZiYKO8wCCH/kpGR\nAXt7+w4vVygUorS0FADwxRdfYMOGDRAKhVBWVm7W9oGBgTh16hS2bt2KSZMmNbvc1m73bw0nzxKJ\npNnbvOzY19k4ODiAMYadO3fCxsZGbnEkJye/8P2GY3l+fj62b9+OLVu2oE+fPjAxMUHPnj07/J6e\nWlpaAMB9vp/n8ePHAF6eiL4KV1dXqKurd+jn63X4bjTo379/k59PSjBJt1ZRUdHhV6UbqKmp4cKF\nC1i+fDnWrl2LNWvWYOLEidi9ezcqKysBPBn8/u8B8MbGxtzVr4are//W8KPd1NUxkUjU4quB9+/f\nBwDo6+u3aLsX0dLSgq+vL44dO4atW7ciODgYs2bNwqVLl3D48GH89NNPCA4O5lovgVevV3vUo61o\naWkhLy9P3mEQQv5FIpG8/Kbh7eyrr77C/v37sWLFCrz33nsvHLfWgM/nY/fu3fD09MSHH36I9evX\nN6us1m73b5aWluDz+UhLS2v2Ni879nU2DceilStXyjmS5mtINpOTk5GcnAwtLS3cuHGjRWMBX5WF\nhQWUlZVRWFj4wvUaJrNq7+RdVVUVDx8+bNcynvY6fDcaaGlpNTnDMXWRJd2avr4+8vPz5Va+g4MD\n/vzzT+Tl5WHZsmU4fPgwNm7cyCVAmzdvBntyP1puuX79OkxMTADgud1yXtStpLS0FGZmZi2Kk8/n\nA0CLZqZrjpkzZ0IsFiM0NBTBwcHw8/PDzJkzUVJSglOnTuH48eP44IMPuPVftV7tVY+2kJeXB0ND\nQ3mHQQj5Fx0dnQ49+WyKlpYWvv/+e4jF4hdO/f9vbm5uWLJkCdLT07kJ4Npzu6epqqpi1KhRKC4u\nRkRExHPXe/z4MWbPng0ALz32dTYNM4jGxsY+E29HLpcvX25WvA29k6ytrfH1119j6dKl0NPT69Dk\nEnhyLB46dCgePHiArKys56539epVAGjXGf5ra2tbdV70Kl6H70aD/Px8GBgYPPM6JZikWxsyZAiu\nXbuGqqqqDi87Ly8PSUlJAJ78cKxbtw4DBgxAUlISzM3NwefzERsb2+S2lpaW0NHRwblz55p839HR\nEZqamrh161aj16OiolBTUwMXF5cWxero6AgFBYVmH8Saa/jw4bCwsEBgYCAMDAygq6uLUaNGwdjY\nGF9//TWsrKwadY151Xq1Vz1eVU1NDa5evYohQ4bIOxRCyL84Ozvjxo0b8g4D06ZNw+DBg3Hq1Knn\nTv3flG+//Ra9e/fmxt2193ZPCwgIgKqqKpYsWQKpVNrkOgkJCdxtGl527Otsbty4AVVVVfTu3Vve\noTxXQ1JpaGiIuXPn4sqVK0hPT0dAQABGjRqFrKys587k3p4axviuWbOmyffLy8uxefNmGBgY4MMP\nP+ReV1JS4lph28KlS5fAGGt0/G3rMprS3b8bACCTyXD16lUMHDjwmfcowSTdmq+vL6qrq3HgwIEO\nLzsvLw9z585FSkoKampqcPv2beTk5GDIkCHg8/mYOXMmgoODsW3bNpSXl6O+vh73799Hfn4+VFVV\nsWLFCvzzzz9YuHAhHjx4AJlMBrFYjKSkJPD5fCxduhQhISE4cOAAysvLcefOHXz88ccwNjaGv79/\ni2LV19fH+PHjcfToUezatQvl5eWIj49vclKdluDxeJg+fTpSUlIwffp0AE/GkU6dOhWJiYmYOnVq\no/VftV7tVY9XFRwcDLFY3Ki1lhDSOYwZMwY5OTmIjIyUaxw8Hg9btmwBj8fDwoULUVJS0qztGrq8\nKioqtqi81m73tH79+uF///sfEhISMHToUJw+fRplZWWora1FVlYWfvvtN8yaNYsbV/qyY19nc/jw\nYXh5eUFVVVXeoTTSkJQIBAJMmjQJYWFhyM/Px88//wxPT09uvWHDhkFPTw+///57h8c4YsQIrFu3\nDnv37sWMGTMQFxeHqqoqlJeX49y5c9zcDEePHoVQKOS2s7a2xuPHj3H8+HHU1tbi4cOHyMnJeWb/\nOjo6yMvLQ3Z2NsRiMZcwymQylJSUoK6uDvHx8Vi8eDF69OiBGTNmtFkZzdHdvxsAcPbsWeTl5cHX\n1/fZN9tmolpCOq+PP/6YGRsbs5KSkg4tNzs7m7m7uzNtbW2mqKjITExM2MqVK1ldXR1jjLHq6mq2\nbNky1qNHD6akpMT09fXZ+PHjWWJiIrePX375hfXt25fx+XzG5/NZ//792datWxljjMlkMrZhwwZm\nY2PDlJWVmba2NvPx8WGpqanc9lu3bmXq6uoMALOxsWGZmZls586dTCAQMADMwsKCpaWlMcYYE4vF\nbPbs2UxXV5dpamoyT09Ptnr1agaAmZmZsbi4uFb9P9y9e5cZGBg0uiVKcnIyMzAwYLW1tc+s35x6\nrV+/nqmpqTEAzNzcnO3fv597r73q0Vrl5eXMzMyMzZ49u0PLJYQ034ABA1p1q46WioiIYLa2ttyt\nJkxMTNjcuXMbrTNjxgwGgIlEIrZu3ToWEhLCevXqxQAwPT09tmDBgib3/fnnnzeqQ2u3a4179+6x\nzz77jPXt25dpamoyRUVFJhKJWP/+/dmsWbMa3aLpRce+H3/8kRkaGjIATENDg/n6+jb5Wke4ffs2\n4/F47OTJkx1S3os8fZsSdXV1Nm3aNHb27Nkmj6H/FhAQwIRCYaPbfHWk69evs//+97+sR48eTEVF\nhWloaDBHR0e2dOlSdv/+/WfWf/ToERs+fDjj8/nMysqKffLJJ+zzzz9nAJi1tTV3u5GYmBhmYWHB\n1NTUmKenJysoKGD+/v5MWVmZmZqaMiUlJSYQCNj777/PMjMz26yMluqO3w3GGKurq2P9+/d/7m2V\neIwx1t4ZLiHy9PjxYzg6OmLQoEEIDQ0Fj8eTd0jkNcEYw4QJE3DlyhUkJCR0ysmHCCHA33//jdGj\nRyM4OPiVZlYl3UN1dTXc3NzA5/MREREh9/OGqKgo/PDDD/jvf/+LsWPHcvMNNEdlZSUcHR25OSHk\nXZf2NHfuXBw5cgSPHj2Sdyjd3urVq7Fhwwbcvn27yS7k1EWWdHs6Ojr4448/cObMGcyfPx90TYV0\nlEWLFuHkyZM4fPgwJZeEdGKjRo3C/Pnz4e/v/8JJScjrYdmyZUhPT8eePXs6RUI2ePBgHDt2DOPH\nj29RcgkAGhoaOHjwIMLCwvD555+3U4SdR319vbxD6Pb+97//ITAwED/99NNzxydTgkleC56enjh8\n+DCCgoLwn//8h5sOmjRfSkoKd1PwFy1+fn7yDlXupFIpJk+ejO3bt+PgwYN488035R0SIeQlNmzY\nAEtLS4wdO/a1vaUQ/c4DP/30E7Zs2YLffvsNtra28g6nTbi5uWHPnj3YvHkzFixYQElYK9B344md\nO3dixowZWLp06QvnxaAusuS1cuXKFUyYMAGGhoY4fvw4rKys5B0S6WYePHgAHx8fZGZm4tChQxgx\nYoS8QyKENFNhYSFGjBgBsViMCxcu0DHiNfPDDz9g+fLl2LRpExYvXizvcNpcaGgopkyZAk9PTxw5\ncqTRLO5d3YoVK/Djjz+ipqYGlpaW2LBhA02s14YYY/jmm2+wZs0arF69Gl9//fULW/cpwSSvnezs\nbPj4+CAnJwfr16/HrFmzoKBAjfnk1TDGsHv3bixbtgzGxsY4fvw4evbsKe+wCCH/Jzs7G7Nnz4aN\njQ1sbGxgbW0NW1tbWFlZNZoltKioCCNHjsSjR48QHBzcaFZO0j1VVVVh6dKl2L59O7Zu3YqPP/5Y\n3iG1mxs3buDdd9+Fjo4Odu3aBTc3N3mHRDq5nJwczJkzB5cvX8auXbswefLkl25DCSZ5LUkkEnz1\n1VfYsmULXF1dsX37dvTr10/eYZEuKj4+HvPmzUNUVBTmz5+PwMBAaGpqyjssQshT6uvroaGhgdra\nWigqKnK3HFBQUIChoSF69+6N3r17w8bGBsbGxggKCsLly5fx9ddfY8WKFXQhsptKTk6Gn58f7t27\nh6CgIIwfP17eIbW73NxczJ49G+Hh4Vi4cCECAwOhoaEh77BIJyOTybB161asWLECFhYW+P333zFo\n0KBmbUsJJnmtNSQGkZGRmDZtGr788kvY2NjIOyzSRWRkZOD777/H3r17MXDgQGzfvh3Ozs7yDou8\ngvLycojFYojFYlRWVkIqlaKqqop7XyKRoLq6mnuurKzc6GKCiooKNDQ0oKamBi0tLWhpaUEkEnVo\nHcjzubi4ICYm5rnvKysrg8fjoaamBgC4LmBCoRDe3t744YcfYG5u3iGxkvZVXV2NzZs349tvv4Wz\nszMOHjwIS0tLeYfVofbs2YMlS5ZAKBQiICAAU6ZMeaX7opLuIywsDCtXrkRsbCyWL1+OlStXtuh+\nsJRgktceYwwHDhxAYGAgMjMzMWnSJKxYsQIODg7yDo10UomJiVi3bh0OHjwIAwMDzJs3D8uWLet0\nN+MmT7q+ZWdnIycnB4WFhSgqKkJ+fj6KiopQWFiIgoIClJaWQiwWo7S0tN3iEAgEXLJpYGAAY2Nj\nGBgYcI/19fVhYWGBnj17Ql1dvd3ieN3Nnz8fv/32W4tumA4AmpqakEgk+OijjxAYGAg9Pb12ipB0\nhJMnT2Lp0qXIz8/Hl19+iWXLlkFJSUneYclFQUEBVq1ahb1798La2hoBAQGYMGECtdi/pq5evYpV\nq1bh8uXLGD16NNavX4++ffu2eD+UYBLyf2QyGf744w989913SExMxKhRozB79my88847UFZWlnd4\nRM5qa2tx6tQp/Pbbb/j7779hZ2eHqqoq5Ofno6qqCsrKyrCzs4OTkxOcnZ3h7OwMJycnGBsbyzv0\nbq+6uhopKSlISkpCamoq7t69i6ysLNy9exf5+fncrYn4fD709fVhYmLCJXcmJiYQCoXQ0tKCtrY2\n1+qopaUFTU1NrkWygaqqaqMEsKqqClKplHve0OIpkUi4ltCysjKUlZWhoqICpaWlKCws5JLcgoIC\nFBYWQiKRcPswNDSElZUVt9ja2sLBwQF9+vShbmyvKCgoCHPnzm3WLJpKSkoQiURYu3YtZs2aheDg\nYCxfvhyVlZX45JNPsHDhQko0u5iwsDB89913uHz5Mvz8/PDDDz/AzMxM3mF1ChkZGQgICEBwcDB6\n9+6NBQsWYOrUqTTc4zVQV1eHEydOYOvWrbh48SLefPNNBAYGwsPDo9X7pASTkH9hjOHPP//Ejh07\n8Pfff0NfXx/Tp0/nJocgr5eMjAzs2rULe/bsQVFREUaMGIG5c+fi3XffhYKCAurr65GTk4PExERE\nR0cjOjoaSUlJyMrKAmMM2trasLe3h4uLCxwcHLjHampq8q5al1RQUICoqCjExMQgMTERCQkJyMzM\nRF1dHZSVldGrVy/07NmzUYJmZWUFS0tLaGtryzv85xKLxcjOzkZWVha3NCTKGRkZqKqqgoKCAiwt\nLeHo6Ah7e3v0798fQ4YMQY8ePeQdfqeVl5fHfS+jo6Nx9erVl7ZUKysrQyaTYf78+fj2228bzbRZ\nUVGBzZs3Y8uWLZBKpZg9ezaWLl1K3WY7MZlMhtDQUKxbtw7R0dEYOXIkVq9e/Uonz91ZYmIiNm3a\nhODgYCgrK2P69OmYP38+7Ozs5B0aaWOFhYUICgrCjh07kJeXhzFjxuDTTz/FW2+99cr7pgSTkBfI\ny8vD/v37sWPHDmRnZ8Pe3h4TJkzApEmT0KdPH3mHR9pJdnY2Tpw4gSNHjuDatWswMjLCtGnTMGfO\nnGbPDFtaWoqEhAQu4UxMTERMTAykUimUlJS4VqmGhNPV1ZVaO/+lrq4ON27cQFRUFCIjIxEVFYWc\nnBzweDzY2Nigb9++sLe35xIuOzu7btnboL6+Hnfv3sWdO3eQlJSEhIQEJCYmIjk5GfX19TA2Nsag\nQYMwePBguLm5YciQIS2+GXtXV1dXh6SkJMTExHBLbGwsKisroaKigr59+2LAgAFwdnbGp59+2mQX\nWQUFBchkMrz55pvYunUr7O3tn1tedXU19u7di8DAQOTl5WH48OGYM2cO3n///W75GeyKGo7fO3fu\nRHZ2NsaMGYNVq1Zh8ODB8g6tSygrK8Phw4fx008/ITk5mTv/mT59Ot2+pwuTSCT466+/sG/fPpw7\ndw5qamqYPn06Fi9e3KZ/V0owCWmG+vp6nD9/HkePHsWJEydQVFQER0dH+Pr6Yty4cRgwYAANjO/C\nZDIZYmJi8NdffyEkJATx8fHQ09PD+++/j/Hjx2PEiBFt8vetq6vDvXv3nmntvHv3LgA0au1saPF0\ncHB4rZKFu3fvIjw8HOHh4QgLC0NpaSmEQiEGDhwIDw8PuLi4wN3dHbq6uvIOVe4qKytx+/btRi10\nSUlJUFJSgrOzM7y8vODl5YU33nijWyU9dXV1SE1NbVTv27dvQyKRQFlZGTY2Ntx3qOHizdPfIWdn\nZ8THxzfap6KiIkxNTbFx40ZMmDCh2bHU1NTg2LFjCAoKwsWLF2FsbIwZM2Zg8uTJL0xQSftoOHne\nvXs3zp07B11dXUydOhVz5syBra2tvMPrkmQyGcLDwxEcHIzQ0FCIxWIMGzYMkyZNwrhx46iLcRdQ\nWlqKc+fO4fDhwzh9+jRkMhm8vb3h5+cHHx+fdjnHoASTkBaqr6/HlStXEBISgtDQUNy/fx86OjoY\nPnw4vLy8MGLECPTq1UveYZKXeDqRuXDhAh49egQTExP4+PjA19cXb7zxRoddNCgpKeGSzobWzujo\naG5sp42NTaPWzoEDB8LIyKjd4jl48CBcXV075ISspqYGFy5cQEhIsGr0xgAAIABJREFUCE6ePInC\nwkLu+/T222/j7bffphPDFrh37x7Onz/PLQUFBdDW1sbo0aMxfvx4eHt7d6lJhGpra5GWltYomWz4\nbmhqasLOzq7RRZmBAwe+dLItf39/7N69G7W1tVBWVoaCggKWL1+O5cuXv9KJVmZmJn7//Xfs3bsX\nDx48gJ2dHXx9feHr6wtXV9dW75e8WGlpKU6dOoWQkBCcPXsWNTU1GDFiBGbNmoV3330XKioq8g6x\n26iursaZM2dw6NAh/Pnnn5BIJHBycoK3tze8vb3h6enZrS5mdVWMMcTGxuLMmTM4e/Ysrl+/DsYY\nhg8fDj8/P/j6+rb7kBFKMAl5RQkJCVyicvnyZVRUVMDS0hKenp4YMmQI3Nzc4OTk9NrOUNcZ1NXV\n4c6dO7h+/ToiIyNx9epVZGVlQUNDA8OGDeNaevr27cvdlkDeGlppnk44o6OjkZ+fD6Dp1k5HR8c2\nmcnWzc0NN2/exMKFC/HVV1+1+YGopqYGZ86cwdGjR3Hq1CmUlZXBxcUFPj4+GDlyJAYMGEAzGLaR\nhIQEhIWF4fjx44iIiICqqiq8vb0xfvx4vP/++50q2ayoqEBqamqjz/utW7dQXV0NLS0tODk5NWqZ\n7N27d6suAv3666+YO3cuAMDPzw8bN26Eqalpm9VDJpPh2rVr3EXI/9/enYfXdO7//39mknFnRyIj\nQoIgkzmooYYIx1AVdKRKtKpVVHU4Leqc6qSKarV6TK22H1pFlU4SqgiiQkQQESISmUn2TmTeWb8/\n+s36SU1Bkp3wflzXujKte633jmx7v9Z9r/s+f/48np6eDBkyhIEDBzJgwACZHOguVI44qXzd3bNn\nDyYmJgwYMIDQ0FBGjhyJi4uLscu85xUXF/Pnn3/y66+/8uuvv5KQkIBGo+HBBx+kd+/e9OrVq1oX\nfMTdMxgMxMXFsXfvXiIjI9m9ezcZGRm4ubkxZMgQ/vWvfzFo0KA6nYdAAqYQNaisrIwDBw6wa9cu\nDhw4QFRUFDqdDhsbG7p27UqPHj3o0qULgYGBtG7dWkJnLTAYDJw5c4bjx48THR3NgQMHiI6O5sqV\nK2i1WoKCgujZsycDBgygZ8+eDe7q9tW9nVf3eF7d23n1hEJBQUG4urpW+/iKoqDRaLhy5Qrm5ubY\n2Njw7rvvMmXKlLv+ez19+jRr167lyy+/JDs7m06dOjF8+HCefPJJmUCrDly6dImff/6ZjRs3smPH\nDqysrBg5ciRPPfUUwcHBdVpLfn4+x44dq9Irefr0aQwGA1qtFn9//yphsn379jV20eGvv/7imWee\n4dNPP6V37941csybiY6OZuvWrYSHh3P48GEqKirUIcx9+vS57efo/aasrEy9QPjHH3/wxx9/cPny\nZVxdXRk4cCBDhw5l2LBhst6skZ07d47ffvuN3bt3s2/fPtLT07GysqJr16707t2bbt260alTJ7l/\nswZkZmYSExPD4cOHiYyMJDIyEr1ej1arpVevXvTp04fBgwfTsWNHo100l4ApRC2qqKggPj6egwcP\nqtupU6coLy/HysoKX19fAgMD8ff3JzAwEB8fH5o3by69N9VQUVFBamoqCQkJxMbGcvz4cY4fP66G\nLXNzc9q1a1dl4pOafJNan1QOI7y6t/Pw4cNkZGQA4O7uXmWI7c16f5KSkq6ZyKhy9tKPP/6Y4cOH\n31ZtBoOBjRs38sknn7B//35atmzJxIkTmThxosy8aUTZ2dl88803rFq1ipMnTxIYGMjzzz/P008/\nXeM9DjqdTr3gU7nFx8dTUVGBg4MDfn5+VcKkr69vrb4pqlyixBj3zet0Ov78808iIiLYuXMnp06d\nQlEUWrRoQY8ePQgKCqJbt24EBATcl4GpckKrI0eOEBUVxaFDh9TJ0ezt7enTpw/BwcEMHDgQf3//\nejPiRFzr7NmzREZGqr1qp0+fVp/zHTt2pFOnTnTs2FF971OfRlPUF2VlZeoEb0ePHiUmJoaYmBjS\n0tIA8PT0pFevXmqo9Pf3rzfvcSRgClHHSkpKOHHihBqIYmNjiY2NJTMzE/h7nb1WrVrRpk0bWrdu\nrX709PSkWbNm99XyFkVFRaSmppKSksKZM2dITEwkMTGRM2fOcPbsWYqLi4G/1w0MCAggMDBQ/ejr\n63tfTY5zPdfr7YyLi6OkpOS6vZ09evQgMjKSUaNGXXOsq2fYXLZs2S0XXi4pKWHdunUsXLiQpKQk\nQkNDeeaZZxg4cGC9eQEUfztw4ACrV6/mm2++wdHRkVmzZjFlyhQ0Gs1tH+vq2ZMrt8oQ5e7ufs1y\nPX5+frXwiBqOvLw8dabkyo/Z2dkANG/eHF9fXwICAtQJv7y8vO6J4bWlpaVcuHBBHW1SueTQqVOn\n1Jm2/f396d69O927dycoKOievUB4vygoKCA2NpaYmBiOHj3K0aNH1dcjExMTmjdvjo+PDz4+PrRr\n1462bdvSsmVLPD097+nX8rKyMtLS0jh//jwJCQkkJCQQHx/P6dOnSUpKory8HDMzM9q2bUvHjh3V\ncN6pU6d6PdmdBEwh6ons7GzOnDlTJUhVblev2+bo6IiHhweenp54eHjQtGlT3N3dcXZ2xsnJiSZN\nmuDk5ISTk1O9vNm+rKyMS5cuVdmys7NJT08nNTWVtLQ0UlJSSEtL49KlS2o7rVZL69atq2xt2rTB\nx8cHZ2dnIz6ihqW4uJgTJ05w7Ngx9eJGTEwMubm5wN/3dl65coXS0tLrtrewsMBgMBAWFsaCBQuu\nudepoqKCVatW8Z///IdLly4xYcIEXnnlFVq3bl3rj03cnfT0dBYvXswXX3yBhYUFr776KjNnzrxh\nj2blGpNX95yfPHkSQA2TV0++U5sTU91LkpOTiYuLU5ejOXHiBCdPnlQvqGk0mmvWeXVzc1NfB9zd\n3dFqtUarv6ysjKysLDIzM0lPTyc7O5vk5OQqa7xevHiRiooKAJo2bYqfn58apP39/fHz85MerftA\neXk5iYmJxMfHXxOucnJy1P2cnZ1p3rw5zZo1o0WLFjRv3hw3NzeaNGlCkyZNcHV1pUmTJvXqb6ak\npIScnBxycnLIysoiOztbfS6kpqaSmppKcnIyGRkZ6nNBo9HQtm1bNWRfHbgbWueCBEwhGoCcnBxS\nUlLU/5QqQ9jFixe5ePEi6enp11083N7eniZNmtC4cWPs7OywsrLC3t7+ms8rg6iDg0OVIUfW1tZV\nrhwWFxdTVFRU5RyVwaSsrIyCggLy8/MpKiqioKAAvV5PcXExBQUF5OXlkZ2djV6vv6ZOrVarhmUP\nDw+aN2+ufmzatCnNmzeXEFnLUlJSiI2N5Y033iAuLk59wbsRCwsLrKysePPNN9UQsnfvXmbMmEFc\nXBxTp07ltddew8PDo44egagpubm5fPzxxyxatAh3d3cWL15Mly5drpnJtXLCqX+Gye7du8skKzXM\nYDBw/vz5KiGtcktOTiYzM5Or385ZWVnh4uKCs7MzWq0WjUaDRqPBzs4OrVaLVqvF1NQUW1vbKveh\nX/0aUFRUpIZa+Ht4b0VFBYWFheTn55Ofn49Op0On05Gfn49er1ffSF/NxsaG5s2b07Jly2uCcatW\nrep04hHRcFy+fJnk5GRSUlLUUJaSksKFCxdISUkhMzOTkpKSKm1sbGzU0GlnZ4eNjQ0ajQZ7e3ts\nbGywsbFR/96srKyqhDYLCwvs7OzUr//5fqfyPQ78fQ95YWEhV65cIS8vT/1cr9ej0+nIzMwkPz+/\nSm3m5uZVgnLz5s1p0aJFlc/vpbWwJWAKcY8oLy+/pmfw0qVL5OTkkJuby5UrVygqKkKv11NQUEBR\nUZH6JqG8vByDwXBN+CsoKKiyILm5ufk1w+bs7e0xMzNTf6bRaLCyslI/t7a2xs7ODgcHB7WX9Z+b\nTHZUf3h6epKSklLt/U1MTPD09MTb25vdu3cTEhLCkiVLaN++fS1WKepCSkoKr776Kt999x2mpqYY\nDIZrwmTPnj3viSGbDZ3BYFDDXVpaGllZWWRlZZGTk6MGwMpNr9erFwb1er16T2pFRQU6nU49pqWl\nZZUeocowam1trf7/7uDggL29PRqNBq1Wi7OzMx4eHri4uODi4oK7u3uVN+1C1KT8/Hy1Z7Cyt7By\nKygoUC+G6HQ6NQRW/o3/c6TOPwPlPwOnqampOjKgMrxWvrexsbHB1tYWe3t77O3t1R7Vys3Z2RlH\nR8c6+q3UDxIwhRBCAH9fULC3t6c6LwuNGjWioqKC8vJy4O+gOWbMGNavX2+UyVNE7dm7dy9PPvkk\nJSUlfP3114SEhBi7JCGEEPWY3C0thBACgOPHj1cJlyYmJlhYWFQZNu3q6kr//v2ZMmUKY8eOxdzc\nnD59+pCWlsb3338v4fIe1KdPH+Li4ggODuZf//oX8+fPN3ZJQggh6jEZlyaEEAKAY8eOAX8Phfby\n8iIgIID27dvTvn17dVa/yiFDixYt4tNPP2XevHnMmzdPZne8x9nb2/Ptt9/Sr18/nn/+eXJzc1m6\ndKksEyGEEOIa8o5ACCEEAMHBwSQkJFBUVERCQgKbNm1iwYIFPPnkk3Tp0kUNl++++y6vvvoqS5Ys\nYf78+TUeLjdt2oS3tzcmJiZVNisrK7y8vJg0aRJJSUk3bTN+/PhrjhsSEoJGo8HMzAw/Pz+OHDly\nx+3uxOnTp3nxxRfx8/NDo9Fgbm6OVqvFx8eHYcOGceDAgWodZ9GiRbi4uGBiYsKKFStu+L3a8Mwz\nz7BhwwZWrFjB888/X2vnEUII0YApQgghRDVt375dMTExUT777LNaP1erVq0UrVarKIqiGAwGJTMz\nU1m3bp1iY2OjuLi4KDk5Oddt4+TkpADK9u3br/n5r7/+qowcObLG2lXXqlWrFAsLC6Vv377Kb7/9\npuTm5irFxcXK2bNnlQ0bNigPPPCA8sUXX1T7eGfOnFEA5fPPP7/p92rLTz/9pJiZmdXJ34EQQoiG\nRXowhRBCVEtmZiZhYWFMmDCBqVOn1um5TU1NcXFxYfz48UybNo2srCwiIiKuu++yZcswNTVlypQp\nVWbFvJU7bXcrBw8eZMqUKfTp04edO3cyePBgHBwcsLS0xNvbm0cffZR58+bdcO3R+mjEiBHMmTOH\nWbNmqUOrhRBCCJAhskIIIapp+vTpaDQaPvnkE6PW0bp1awAyMjKu+/MHHniAmTNncvHiRWbPnl3t\n495pu1tZsGABBoOB995774ZL8gwePJhp06bV2Dnrwty5c+nSpQtTpkwxdilCCCHqEQmYQgghbikh\nIYEffviBDz74wOjr2p05cwaADh063HCfBQsW4OPjw6pVq27Y01mT7W6ktLSUnTt34uTkRFBQULXb\nGQwG5s2bh6enJ9bW1gQGBvLdd9/ddT01yczMjKVLl3Lo0CHCw8ONXY4QQoh6QgKmEEKIW/r2229p\n2rQpDz/8sNFqyMvL46uvvmL58uUMGzaMfv363XBfa2trvvzyS0xNTXnmmWe4cuVKtc5xp+1uJDk5\nmeLiYtq0aXNb7V5//XUWLlzIkiVLSE9PZ8SIETzxxBMcPnz4ruqpaV27dqVHjx58/fXXxi5FCCFE\nPSEBUwghxC3t3r2bIUOG1PlyJDqdTp3htXHjxkycOJE33niDLVu23LJtz549eemllzh//jyvv/56\ntc95p+2uR6/XA9xWr29xcTGfffYZo0aNYvTo0Tg4ODBnzhwsLCxYu3btXdVTG4YPH86ff/5p7DKE\nEELUExIwhRBC3FJiYiK+vr51fl6tVouiKCiKwiuvvIKiKGi1WiwsLKrVfsGCBbRt25bly5ezb9++\nap/3Ttv9U2WwLCwsrHab06dPU1hYiL+/v/o9a2tr3NzciI+Pv+Naaoufnx8XLlygpKTE2KUIIYSo\nByRgCiGEuKXCwkJsbGyMWsPcuXNxc3PjjTfeICUlpVptrKysWLt2LSYmJkyaNImioqJabfdPLVu2\nxMrKioSEhGq3qRyWO2fOnCrrgCYnJ99WUK0rtra2wO2FaCGEEPcuCZhCCCFuydHRkZycHKPWoNFo\neP/998nPz+f555+vdruePXsya9Yszpw5w9tvv13r7a5maWnJ4MGDycnJITIy8ob7Xb58mcmTJwPg\n7OwMwJIlS9Te28rtwIEDd1RHbcrKysLc3BytVmvsUoQQQtQDEjCFEELcUocOHTh06JCxy+Cpp56i\ne/fubN++ne+//77a7d5++23atWvH0aNHb+t8d9ruavPnz8fS0pJZs2bdsCc0Li5OXcKkefPmWFlZ\nERMTc8fnrEuHDh0iICCgzu/PFUIIUT/Jq4EQQohbGjp0KL///jt5eXlGrcPExIRly5ZhYmLC9OnT\nyc3NrVa7yiGvZmZmt3W+O213tY4dO/Ltt98SFxdHnz59+OWXX9DpdJSVlZGUlMTKlSsJCwtT7yu1\nsrJi4sSJrF+/ns8++wy9Xo/BYCA1NZX09PQ7rqM2lJeXs2nTJoYNG2bsUoQQQtQTEjCFEELc0iOP\nPIKVlRVLly6t9XPt37+ftm3bcvbsWXQ6HU2bNmXq1Knqz4OCgpgwYQKZmZl4e3vz/vvvs2XLFlq3\nbs3Zs2fp1q0bL7744jXH7dGjB7NmzaryvTttd7tGjx5NfHw8/fv35/XXX6dZs2ZYW1vTuXNnPv/8\ncx588EEef/xxdf+lS5fy0ksv8cEHH+Dk5IS7uzszZ84kNzeXxYsX07t3bwBmz57N6NGjr/u9uvDN\nN9+QmZnJpEmT6uR8Qggh6j8TRVEUYxchhBCi/lu4cCFvvfUWBw8epEOHDsYuRxjZxYsX6dixI2PH\njuWzzz4zdjlCCCHqCQmYQgghqqWiooJBgwaRkpLCkSNHbmttR3FvqaioICQkhAsXLhAdHY1GozF2\nSUIIIeoJGSIrhBCiWkxNTfnqq6+4fPkyY8eOveOlOxq6+Pj4KsuH3Gh77LHHjF1qrTAYDEyePJnI\nyEg2btwo4VIIIUQV0oMphBACgOjoaL7//nvatm2Lr68vbdu2pXHjxtfsd/ToUUJCQvD19WX79u0S\nMO4jBoOBSZMm8d1337Fx40ZGjBhh7JKEEELUMxIwhRBCAHDq1Cl8fX0xMTGh8qXB0dGR9u3b4+/v\nT7t27Wjfvj1t27ZFp9MREhKCp6cnGzZsoFWrVkauXtS2rKwsnnrqKfbt28dPP/3EgAEDjF2SEEKI\nekgCphBCCODv3ikbGxtKS0uv+VnlGo3l5eUAWFpa0qxZMy5fvkxhYSHPPfcckyZNol27djRq1KhO\n6xa1LyIigvHjx2NjY8OGDRvo1q2bsUsSQghRT92XAVOn05Gfn49er6eoqIiCggLKysqq/LyiokL9\nWqPRqG+uALRaLebm5mi1WhwcHLC3t6/ycyEaisrnQn5+PoWFhRQWFlJSUqL+/MqVK1XCRqNGjbC1\ntVW/trS0xMbGBhsbGzQaDRqNBq1WW6ePQdSsDh06EBsbe8dtN2/ejLe3dw1XJYzl8uXLzJ07lxUr\nVvDII4+wYsUKeY4LIYS4qXsiFVVUVJCamsr58+dJTU0lKyuL9PR0MjIyyMrKIi0tjby8PPR6fa0t\nEl75BtvBwQEXFxfc3d1xc3PDxcUFDw8PXFxcaNmyJd7e3lhbW9dKDUIUFxdz7tw5kpOTycrKIjMz\nU30eZGRkkJmZiU6nQ6/Xo9Ppaq0OrVarPh9cXV1xc3PD2dkZNzc39XnRokULvLy85PlQz3Tr1o2T\nJ0+qPZU3Y25ujsFg4Mknn2TIkCG89dZbBAYG8vrrrzN79mysrKzqoGJRGwwGA1988QXz5s3DwsKC\nr776inHjxhm7LCGEEA1Ag+rBzMjIIC4ujri4OBITEzl37hznzp0jKSlJ7WUxNzfHxcUFV1dXPDw8\ncHZ2pmnTpjRu3BiNRkPjxo2xt7dHo9Fgb2+PjY0N1tbWVd4I2dnZYWFhoX59dY9mRUUFOp2O8vJy\n8vPzyc3NRa/Xqz2ieXl5aqitfHOfnp5OYWGhejwPDw+8vb3x9vamVatW6v1Nbdq0kZ5QcUvFxcWc\nOnWKEydOkJCQQFJSkvo8SE9PV/ezsrLC1dUVd3d39Tnh5uaGg4ODGv4qnwsajQZbW1u1R/LqY1wd\nAIuKiiguLla/ruzxvHLlitoTWvk8KCgoIC8vj4yMDDXkpqenk5WVVWX2UXd3d7y8vNTNx8cHPz8/\n2rdvL+GzDmRnZ3Ps2DGOHTtGbGws+/fvJzk5ucqojn8yNTWloqKCPn36sGzZMjp27AhASUkJS5Ys\n4Z133sHJyYlXXnmFSZMmyb9jA1JeXs6GDRt47733SExMZMaMGcydO1cmchJCCFFt9TJgKopCfHw8\nBw8eJCYmhri4OGJjY8nJyQHA2dkZHx8fNaBVhjVvb2/c3NwwMTEx8iO4Vn5+vhoEzp07x9mzZ6t8\nNBgMWFpa4uvri7+/P4GBgQQFBdG1a9cqb/jF/eXixYscOnSI6OhoTp48SVxcnPr30qhRoyp//1eH\ntJYtW9brYWw6nY7k5OQq4bjy88TEREpLSzEzM8PLywt/f398fX3p0qUL3bt3p2nTpsYuv0EqLy8n\nPj6e2NhYYmNjiYmJITY2Vr0o4erqSocOHXB0dGTDhg03PI65uTlNmzblww8/ZOzYsdfdJy0tjXff\nfZc1a9Zgb2/PjBkzeP755+v13+T9rqioiLVr17Jo0SJSUlJ4/PHHmTNnDj4+PsYuTQghRANTLwJm\nYWEhkZGRHDhwgIMHD3Lw4EFyc3OxsbEhICCAgIAA/P391eDl7Oxs7JJrVHFxsRoe4uLiOH78OMeO\nHSM9PR1zc3M6dOhAz5496dGjBw8++CDNmjUzdsmiFpSWlnLo0CH1ORAVFUVqaiqmpqb4+Pjg7++P\nn58ffn5+93SPd3l5OYmJicTFxXHixAlOnDhBXFwcCQkJGAwGmjZtSvfu3enRowc9evQgKCgIS0tL\nY5ddr+h0Oo4fP87Jkyc5ceIE0dHRHD16lMLCQszNzdVe4srg3rVrV9zd3QHIy8vD0dGRf740WFhY\n0KhRI+bOncvMmTOr9TvPzs5m+fLlLFu2jKKiIkaMGMGzzz7LwIED6+WFwPvRqVOn+Oqrr1izZg16\nvZ5HHnmEuXPn0qZNG2OXJoQQooEyWsA8d+4cERERbNu2jYiICIqLi3F3d6dLly707t2bXr160a1b\nt/v6jWNaWhrR0dFERkayb98+Dh8+TElJCd7e3gQHBxMcHMzgwYOxt7c3dqniDlU+DyIiItixYwc6\nnQ43Nze6du1Kly5d6NKlC7169cLR0dHYpRpdQUEBMTExREdHEx0dzd69ezl//jzW1tb06tVLfU50\n6tQJU1NTY5dbJwwGA8nJyWqIrOzpTkpKQlEUHBwc8PPzo0uXLlUC5a2GrLq5uZGZmQlUvc/yo48+\nwsXF5bbr1Ol0rFu3jtWrV3Ps2DF8fX0JCwvjkUcekQtmRpCTk8OmTZtYs2YNhw4donXr1kyaNIlJ\nkybh6upq7PKEEEI0cHUWMMvLy/njjz/YuHEjW7duJSsrCxcXF4KDgwkJCSE4OFiGvt1CUVER+/bt\nY8eOHYSHhxMbG4uFhQX9+vVjzJgxjBo1iiZNmhi7THETxcXFhIeHs3nzZn7++Weys7Np0qQJAwYM\nYODAgQwcOFDWE7wN586dY9euXURERLBr1y719zls2DBCQ0MJCQm5ZyaauVWvpKenpxogKwPlnc7m\nOnToUH777TcURWHAgAF8/PHH+Pv718jjOHz4MKtWrWLDhg3o9XqCgoIIDQ0lNDSU1q1b18g5xLUu\nXrzIli1b2Lx5M3v27KFRo0aEhoYSFhZGv379pEdZCCFEjanVgFlRUcHu3bvZsGEDW7ZsIScnhy5d\nujBmzBiGDBlChw4d5EXtLmRmZhIeHs6PP/7IL7/8QllZGf3792fs2LGMHTsWBwcHY5co+Hvik23b\ntrFp0yZ+/vlnCgoK6N69O6NGjWLQoEF06NDhvulxq02KohAbG8uOHTv48ccfOXjwIDY2NgwdOpTR\no0fz0EMPNZiwWTl6obJH8sSJE5w6dQpFUdBqtepw6cpA2blz5xq9V3vOnDls2LCBjz/+mGHDhtXY\nca9WUlJCeHg4W7ZsYevWrVy6dAl/f3/1gmPfvn2rLIkjbk9JSQn79+9n586dhIeH89dff2FnZ8fQ\noUMJDQ1l6NCh2NnZGbtMIYQQ96BaCZiZmZl8+eWXrFy5krNnz+Lr68vYsWN54oknZMKAWlJUVERE\nRAQbN25ky5YtlJeXq/c7BQcHG7u8+9LV9zZdunSJnj17MnbsWEaPHi3DAutATk4Ov/zyCxs3buT3\n33/HxsaGRx99lClTptC5c2djlweAXq/nzJkzVYa4xsTEcOXKFczMzGjRooUaIisDpa+vb61fmEtL\nS8PZ2bnKbNq1qby8nD179vDTTz8RERHBiRMnaNSoET169GDgwIHqLRNyO8CNFRYWcuTIETVU7tu3\nj8LCQlq1akVwcDDDhw8nODi4wVxkEUII0XDVaMD866+/+PDDD/nxxx+xs7Pjqaee4tlnn8XX17em\nTiGqQa/X8+233/K///2PmJgYAgMDmT59OuPHj6dRo0bGLu+eVlZWxvr16/n000/566+/1HubJkyY\ngIeHh7HLu29lZGSoYT8hIYEuXbrwwgsvMG7cuDoLUZW9klcPcY2Pj6eiogJ7e3sCAgKq9Ep26tTp\nvu3By8jIICIigp07d7Jr1y4uXLiAqakp7dq1o3v37gQFBdGtWzd8fX3vyyVQSktLiY+PJzo6Wp0Y\nLC4ujvLyclxdXenfvz8DBw4kODiYli1bGrtcIYQQ95kaCZj79+/n7bff5rfffiMoKIhp06YxduxY\nuVJaD0RFRfHFF1/w7bff4u7uzmuvvcakSZPu68mTakNRURGrV69m0aJFpKWl8cgjjzB58mQefPBB\nGQZejyiKwr59+1i5ciUbNmzA3d2dl19+mcmTJ9fYENP8/Hw1tofvAAAddUlEQVQSEhKq9EoeO3aM\ngoICAHUys7rulWzI0tLSOHToEFFRUURFRXH48GHy8/MxNTXF29ubgIAAfH19CQgIoF27dnh7e98T\n6zYWFRVx7tw54uPjOXnyJMePH1fXvy0vL8fa2prOnTsTFBRE9+7d6d69uwRKIYQQRndXAfPEiRO8\n9NJLhIeH07t3b+bMmcPgwYNrsj5RQy5cuMDChQtZvXo1Tk5OLFiwgAkTJsib2rtUXl7O559/zoIF\nC8jPzycsLIzZs2fTokULY5cmbuHChQt89NFHrFq1CltbW9544w1eeOGF2+rRvFWvZJs2bapMvHM/\n90rWJIPBQGJiohq4Kpd4SkxMpLy8HAAnJ6cqa8N6enrStGlTXFxccHV1xd3d3ahrDBcXF5OVlUVa\nWhpZWVmkp6dz4cIFdU3Y8+fPk5GRAYCpqam6Jqyfn5/a292uXbs664EXQgghquuOAmZ+fj7z58/n\nk08+oWPHjnzwwQf079+/NuoTNaxyAfQVK1bQvXt3li9fTseOHY1dVoMUERHBzJkzSUxMZPr06cye\nPfuOlnAQxpWdnc3ixYtZunQpXl5eLF26lJCQkCr7lJaWcubMmSoT78TExJCTkwP8/72SVw9xbd++\nvUzeVMdKS0tJTExUQ9rV24ULF7h8+XKV/W1tbXF3d8fBwQEHBwfs7e3RaDRoNBrs7OzUidLs7e0x\nMzMD/g57Wq1WPUZBQQFlZWXA3z3keXl5wN+vk1dveXl56PV69Ho9aWlp6PX6KrXY29vj6empBmJv\nb2+8vLxo2bIlrVu3NmoYFkIIIW7HbQfMHTt2MHHiREpKSnj33XeZPHmyvIlqgGJiYnjhhReIiori\npZde4t1335Ur4dV0+fJlnnvuOTZu3MjIkSP56KOPZGmRe8C5c+eYPXs2W7ZsYfDgwXTu3JkzZ85w\n7Ngxzp49S0VFBRqNhoCAAAIDA+nYsSOBgYEEBATIbJwNRElJCdnZ2aSnp5OZmUl2drYa9q4OgJWh\nsDIE5ubmqscoKytThzsD2NjYVLnlQKvVYmpqip2dnRpWNRoNjRs3xt7eHnt7e9zc3HB1dcXFxQV3\nd3dcXFzuy3tJhRBC3JuqHTANBgP//e9/WbBgAY8++iiffPIJTk5OtV2fqEWKorBmzRpmzJhBhw4d\n2LBhA82bNzd2WfVaZGQkTzzxBBUVFaxevfqani7R8IWHhzNu3DguX75Mz5496d+/Px06dKBDhw54\ne3vLsHIhhBBCiJuoVtdjfn4+Q4YMYeHChSxfvpz/+7//k3B5DzAxMSEsLIy//voLnU5H586d2bt3\nr7HLqreWLFlCv3796NChAzExMRIu71GDBg3i1KlTjBgxgv3792Nra0toaCitWrWScCmEEEIIcQu3\nDJh6vZ4hQ4YQFxdHZGQkzz33XF3UJepQ+/btiYqKol+/fvzrX//izz//NHZJ9c78+fN5+eWXeffd\nd9m6datcYLnHOTo6snnzZhYtWsS///1v5syZY+yShBBCCCEahJsGzIKCAoYMGUJSUhJ//PFHrS1O\nXlJSwowZM3Bzc8PGxobffvuNX375Ba1Wy7Zt22rlnA3J5MmT0Wg0mJiYEBMTUyvnsLW1Zf369Qwf\nPpxhw4axZ8+eWjlPQ/Tvf/+bt99+m5UrV/LKK6/UeC/Wpk2b1KGXV29WVlZ4eXkxadIkkpKSbtpm\n/Pjx1xw3JCQEjUaDmZkZfn5+HDly5I7b3YnTp0/z4osv4ufnh0ajwdzcHK1Wi4+PD8OGDePAgQPV\nOs6iRYtwcXHBxMSEFStW3PB7tWHmzJmsXbuW999/n1deeaXWziOEEEIIcc9QbmLy5MlKkyZNlPj4\n+JvtdtfeeecdxcfHR8nNzVW++OILZePGjcr27dsVe3t75aeffqrVczcU69evVwDl6NGjtXqesrIy\nJTQ0VHFzc1OysrJq9VwNwYYNGxQTExNl3bp1tX6uVq1aKVqtVlEURTEYDEpmZqaybt06xcbGRnFx\ncVFycnKu28bJyUkBlO3bt1/z819//VUZOXJkjbWrrlWrVikWFhZK3759ld9++03Jzc1ViouLlbNn\nzyobNmxQHnjgAeWLL76o9vHOnDmjAMrnn39+0+/VlvXr1ysmJibK119/XevnEkIIIYRoyG7Yg7l1\n61ZWrVrF559/Ttu2bWs15P7444907doVBwcHnn32WcaMGcOwYcPQ6XSMGDHiro5dVFTEAw88UEOV\n3vvMzc1Zs2YNVlZWPPPMM8Yu565dPRPk7UpJSWHq1KlMmzbtuj19tcnU1BQXFxfGjx/PtGnTyMrK\nIiIi4rr7Llu2DFNTU6ZMmYJOp6v2Oe603a0cPHiQKVOm0KdPH3bu3MngwYNxcHDA0tISb29vHn30\nUebNm0dpaWmNnbO2PfbYY7z00ktMnTqVhIQEY5cjhBBCCFFvXTdgFhUVMXXqVMLCwhgzZkytF5Ga\nmlprS2SsXr2arKysWjl2XarLyUW0Wi3r1q1j27ZtbNmypc7OWxtOnTqFi4sLo0ePZvPmzRQXF1e7\n7ZQpU2jWrBkLFy6sxQpvrXXr1gDqouv/9MADDzBz5kwuXrzI7Nmzq33cO213KwsWLMBgMPDee+9h\nbm5+3X0GDx7MtGnTauycdeG9996jTZs2PP/888YuRQghhBCi3rpuwPzyyy/R6XS88847tXry8PBw\nWrduTXp6Ol999RUmJibY2dmxb98+PD09MTEx4dNPPwVg4cKF2NjYoNFoyMrK4uWXX6Zp06acPn2a\nP//8k6CgIGxsbLC3tycgIAC9Xs/MmTN5+eWXOXv2LCYmJuob9ero2rWrep9aYGAgKSkp191v/vz5\nODo6YmVlxYIFC4C/l/9YvHgx7du3x9LSksaNG/Pwww8THx+vtrvZ41EUhQ8//JC2bdtiaWmJVqut\n8/u/+vTpQ2hoKB988EGdnrc2lJSUsHXrVsaMGYOTkxMTJkxgx44dGAyGG7Y5cuQIv/76Kx999BFW\nVlZ1WO21zpw5A0CHDh1uuM+CBQvw8fFh1apVN+zprMl2N1JaWsrOnTtxcnIiKCio2u0MBgPz5s3D\n09MTa2trAgMD+e677+66nprUqFEjFi9ezM6dOzl48KCxyxFCCCGEqJeuGzDXr19PaGgorq6utXry\nQYMGkZiYiKurKxMmTEBRFAoKCujduzf79++vsu+rr77KrFmzKCgo4N1338XLy4sePXpQUFDAQw89\nxJgxY7h8+TJnzpzBx8eH0tJSli5dyogRI2jVqhWKopCYmFjt2g4fPkyvXr1o3rw5x44dU9eH3L59\ne5Uhw/Pnz+c///kP8+bNU2eanD9/Pv/+97958803ycrKYs+ePaSkpNCnTx8yMzNv+ngURWHu3Lm8\n9tprTJkyhczMTDIyMnj99dfv9td926ZOnUpUVNRt/d7qK4PBgKIoFBYW8n//938MHjwYR0dHpkyZ\nwr59+1D+sRzsN998Q7t27Rg0aJCRKoa8vDy++uorli9fzrBhw+jXr98N97W2tubLL7/E1NSUZ555\nhitXrlTrHHfa7kaSk5MpLi6mTZs2t9Xu9ddfZ+HChSxZsoT09HRGjBjBE088weHDh++qnprWr18/\n/P39+frrr41dihBCCCFEvXRNwCwpKSEqKoohQ4YYo55qef/995k2bRqbNm3CysoKvV6Pn58fVlZW\nuLq6smnTJpo0aXLX53n66adJSUlh9+7d6ve++uorEhISqsyA+d133zFu3Djg7+HFixcvJjQ0lHHj\nxqHVagkICGDFihXk5OTwv//976aPp0WLFixZsoTg4GBmzZqFg4MD1tbWODo63vXjuV19+/bFxsbm\nnptRtry8HPh7CZ4vv/ySPn364O7uzowZM4iOjgZg9+7dDB06tM5r0+l0as9548aNmThxIm+88Ua1\nhir37NmTl156ifPnz9/WBYk7bXc9lfe72tnZVbtNcXExn332GaNGjWL06NE4ODgwZ84cLCwsWLt2\n7V3VUxuGDx8uS/kIIYQQQtzANTdIpaSkUFpaSvv27Y1Rz23z9vbGxcWFcePGMWPGDJ5++mlatmxZ\nI8d+9NFHmTFjBuvWraN///7k5uZy9uxZLC0tWbduHT179uT8+fM0atQIT09PAE6cOEFBQQFdu3at\ncqxu3brRqFEjoqKibnrOxMRECgsLGThwYI08hrthbm6Oj48Pu3fvRqPRGLucO3Kr3tfKiWYyMzP5\n/PPPWbZsGW3btuX8+fM8+uijdVFiFVqtlry8PODvXu4PP/wQrVZb7XuUFyxYwPbt21m+fPlt1X+n\n7f6pMlgWFhZWu83p06cpLCzE399f/Z61tTVubm5VhpXXF35+fixevNjYZQghhBBC1EvXBMyioiLg\n7zd4DYG1tTW7du3i9ddf55133uG///0vjzzyCGvXrr3rx6DRaAgNDWXTpk0sX76c9evXExYWxu7d\nu/nuu+9YunQp69evV3svATUcXK8Hx8HBgfz8/JueMzU1FQBnZ+e7qr2m2NracuDAgftiSGBZWRnw\nd+ABWLVqFX369DHaLMRz587l66+/5o033mDkyJHqMO2bsbKyYu3atfTu3ZtJkyZV+x7aO233Ty1b\ntsTKyuq2ZlqtHJY7Z84cdZh5JXd39zuqozbZ2tpSWlpKWVlZrU1OJoQQQgjRUF0zRLZyKOalS5fq\nvJg75efnx7Zt20hLS+O1117ju+++Y9GiRTVy7IkTJ5Kfn8+WLVtYv349jz32GBMnTiQ3N5ft27fz\n448/Vplp18HBAeC6QTIvL49mzZrd9HyVE8qUlJTUSP13Kzs7m6eeegpFURrkdqse40qNGjUCoFWr\nVrz11ls4Ojoyffp0oy5xo9FoeP/998nPz7+tmUt79uzJrFmzOHPmDG+//Xatt7uapaUlgwcPJicn\nh8jIyBvud/nyZSZPngz8/xdTlixZcs2/39VD0euLrKys2+pVFkIIIYS4n1wTMD08PHBycqp3k2vc\nSFpaGidPngT+fqP63nvv0blzZ/V7d6t///60aNGCBQsW4OLigpOTE4MHD8bd3Z233noLLy8v7O3t\n1f39/f2xs7O75vcXFRVFaWkpXbp0uen5/P39MTU1rRf3eOXm5pKYmEhgYKCxS6kVlaHS2dmZ5557\njr1795KYmMj8+fPp3Lkzhw4dMnKF8NRTT9G9e3e2b9/O999/X+12b7/9Nu3atePo0aO3db47bXe1\n+fPnY2lpyaxZs9QREf8UFxenLmHSvHlzrKysiImJueNz1qVDhw7ddEZfIYQQQoj72TUB08TEhMGD\nB7Np0yZj1HPb0tLSeO6554iPj6e0tJSjR4+SnJxMjx49gL97ZNPS0jh//jz5+fnqMMjqMjExYcKE\nCcTHxzNhwgQAzMzMGD9+PCdOnGD8+PFV9reysuLll19m8+bNfPPNN+j1eo4fP87UqVNxd3dnypQp\nNz2fs7Mzo0eP5ocffmD16tXo9XpiY2OvOzlQbdu8eTONGjW66eylDU1lr5OdnR2PPvoo4eHhZGRk\n8PHHH9O7d291v6FDh7Jt27bbupewNpiYmLBs2TJMTEyYPn06ubm51WpXOeTVzMzsts53p+2u1rFj\nR7799lvi4uLo06cPv/zyCzqdjrKyMpKSkli5ciVhYWHqv4WVlRUTJ05k/fr1fPbZZ+j1egwGA6mp\nqaSnp99xHbWhuLiYrVu3MmzYMGOXIoQQQghRPynXsXPnTgVQ9u/ff70f15jz588rnTp1UgDF3Nxc\n6dy5s/LDDz8on3zyieLm5qYAio2NjfLQQw8pH3zwgWJtba0ASvPmzZWvv/5aPcYDDzygNG7cWDEz\nM1M8PDyUN998UykvL1cURVGOHDmitGjRQrG2tlZ69+6tZGRk3Had586dU1xcXJTS0lL1e6dOnVJc\nXFyUsrKya/avqKhQPvzwQ6VNmzaKhYWF0rhxY2XUqFHK6dOn1X1u9HgURVHy8/OVyZMnK05OToqd\nnZ3Su3dvZd68eQqgNGvWTDl27NhtP4bbZTAYFD8/P2X8+PG1fq7aFBUVpQAKoFhbWytPPvmk8vPP\nP1f5t7yezMxMxdbWVlm0aFGt1xgZGan4+PiodXp4eCjPPfdclX2efvppBVAcHByU9957T9m8ebPS\nqlUrBVCaNGmiTJs27brHfuWVV5SRI0eqX99puztx4cIFZfbs2UpAQIBiZ2enmJmZKQ4ODkqnTp2U\nsLAwJTIyUt23pKREee211xRPT0/F3NxccXZ2VkaPHq2cOHFC+eijjxRXV1cFUGxtbZXQ0NDrfq8u\nLFu2TLG2tlbS09Pr5HxCCCGEEA2NiaL8YwHA/2fAgAHk5OTw119/YWlpWetBV9Qv77//PvPnz+fY\nsWNV1v1saGJiYpg/fz6PP/44Dz300G1N/DR37lyWLl1KdHQ0Pj4+tVilaAjOnTtHp06deO655+54\nEiQhhBBCiHvdDQNmamoqgYGBPP300zIl/33myJEj9OzZk/fff5+XXnrJ2OUYTVlZGX379qW0tJT9\n+/fLhZb7WHl5OX379qWgoIBDhw6pk3EJIYQQQoiqrrkHs1KzZs1YtmwZS5cuZcWKFXVZU62Jj49X\nF7G/2fbYY48Zu1SjOXfuHA8//DB9+/ZlxowZxi7HqCwsLPjmm29ITEzkiSeeUNfMvN/c78+bsrIy\nxo0bx/Hjx/nhhx8kXAohhBBC3MQNezArffjhh7z22mt88sknvPDCC3VVlzCC5ORk+vXrh6OjI+Hh\n4eqSNfe7yMhIhg4dSq9evdi0aVODWSNW3L3S0lIef/xxfv/9d3766ScGDBhg7JKEEEIIIeq1G/Zg\nVnrllVd4++23efHFF3nrrbcwGAx1UZeoY/v27aNXr140adKEnTt3Sri8Sq9evQgPD+fgwYMMGjSI\nlJQUY5ck6sDFixcJCQlh586dhIeHS7gUQgghhKiGWwZMgDfffJMVK1awcOFCQkJCyMzMrO26RB1R\nFIWFCxfSv39/unbtSkREBA4ODsYuq94JCgpi79695Obm0rFjR7Zu3WrskkQt2rZtGx07diQzM5M9\ne/bQs2dPY5ckhBBCCNEgVCtgAjz77LPs37+f5ORkOnbsyPr162uzLlEHEhISCAkJYc6cOXzwwQds\n2bIFrVZr7LLqLT8/P/766y9CQ0MZNWoUEydOJCMjw9hliRqUlZVFWFgYI0eOZMSIERw+fJjAwEBj\nlyWEEEII0WBUO2ACdOrUiejoaEaMGMG4ceMYMGAAJ0+erK3aRC0pLCzkzTffJDAwkOzsbPbu3cus\nWbMwMTExdmn1no2NDStXruSHH35g165dtG3blkWLFt23EwDdK8rKyli8eDE+Pj6Eh4fz/fffs2bN\nGmxtbY1dmhBCCCFEg3JbARNAq9Xyv//9jwMHDpCfn0/Hjh2ZMmUKSUlJtVGfqEElJSV89tlntGvX\njuXLl7Nw4UIOHz5M9+7djV1agxMaGsqpU6d46aWXmDdvHr6+vqxcuZKSkhJjlyZuQ2lpKWvWrMHP\nz48333yT6dOnEx8fz5gxY4xdmhBCCCFEg3TbAbNSUFAQUVFRrFixgoiICNq2bcvEiRM5c+ZMTdYn\nakBhYSFLly7F29ubl19+mZEjRxIfH8/06dMxNzc3dnkNlo2NDfPnzyc+Pp4BAwbw4osv0qpVKz76\n6CMKCgqMXZ64iStXrrB06VJatWrF1KlT6du3L6dOneK///0vNjY2xi5PCCGEEKLBuuUyJdVRUVHB\npk2bmDdvHgkJCQwYMIBnn32Whx9+GAsLi5qoU9yBhIQE1qxZw+rVqykqKiIsLIxXX32Vpk2bGru0\ne1JmZiaff/45S5cuRVEURo4cyVNPPUVwcLCxSxP/z4kTJ/j6669ZtWoVxcXFhIWFMXv2bJo3b27s\n0oQQQggh7gk1EjArGQwGtm7dyhdffEFERATu7u5MnjyZcePG0bp165o6jbiJgoICtm7dysqVK9mz\nZw9eXl5MnjyZyZMn4+zsbOzy7guXL1/myy+/ZPXq1Zw8eZIOHToQFhbGmDFjcHd3N3Z5953MzEx+\n+OEHVq9ezdGjR2nXrh1hYWFMnDgRJycnY5cnhBBCCHFPqdGAebVz586xatUq1q5dS0ZGBp06dWLs\n2LGMGTOGNm3a1MYp71v5+fls27aNH374gd9++43y8nKGDx/OlClTGDRoEKamdzwSWtyl/fv3s3r1\nar7//nsKCwvp2bMnoaGhhIaG0rJlS2OXd89KTk5my5YtbN68mcjISKytrRk7dixhYWH07t3b2OUJ\nIYQQQtyzai1gVjIYDPz5559s3LiRzZs3k5WVRYcOHRg8eDAhISH06tULKyur2izhnnTq1Cl27NjB\njh072LVrFwaDgYEDBzJmzBgefvhh6ZmpZ4qKivj999/ZvHkz27dvJzc3l06dOjFo0CCCg4Pp3bs3\n1tbWxi6zwSoqKiIyMpKIiAjCw8M5cuQIDg4ODB8+nFGjRjFkyBC5t1IIIYQQog7UesC8msFgYM+e\nPfz444/s2LGD+Ph4bGxs6Nu3L4MGDeKBBx6gc+fONGrUqK5KajDOnj3LgQMH+OOPP9ixYwepqak0\nbtyYgQMHMnToUEaOHImjo6OxyxTVUFZWxh9//MG2bduIiIggPj4eS0tLevXqxYABA+jVqxddu3bF\nzs7O2KXWWwUFBURHRxMZGcmuXbuIjIykuLiYtm3bEhwczPDhwxkwYID8XyKEEEIIUcfqNGD+04UL\nF9ReuD/++IOcnBwsLS3p0qULPXr0oGfPnnTu3BkvL6/7ao3G3Nxcjh07xsGDBzlw4AAHDx4kKysL\nCwsLgoKCCAkJISQkhG7dumFmZmbscsVdunjxIhEREezcuZNdu3Zx8eJFzMzM8PX1JSgoiO7du9O1\na1fat29/X/b2l5SUcPLkSaKjo4mKiiIqKoqTJ09iMBjw8PBgwIABDBw4kODgYJo1a2bscoUQQggh\n7mtGDZj/lJCQwMGDB9VgFRcXR3l5ORqNBj8/PwICAggICMDf3582bdrQtGnTBh08dTodZ8+e5eTJ\nkxw/flzdUlNTAWjatKkatHv06EGXLl3uy4Bxv0lNTVWDVFRUFNHR0Vy5cgUzMzNatWpFQEAAfn5+\n+Pv7065dO7y8vO6J3s4rV66QlJREfHw8cXFxnDhxguPHj3P27FnKy8uxtbWlc+fOdO/eXd1k9lch\nhBBCiPqlXgXMf7py5QpxcXHExsaq4Ss2NpbLly8DYGlpiZeXF97e3urm4eGBu7s7Li4uNG3aFI1G\nY5TaS0tLycrKIi0tjczMTNLS0khOTubcuXPqdunSJQAaNWpE+/bt1QAdGBhIQECALCciACgvLych\nIUENXVcHL4PBAICzszNeXl7q5unpiYeHBy4uLri5ueHm5mbUezyLiorIzMwkPT1dfV5cuHCBpKQk\nzp8/T1JSEllZWQCYmZnh7e1dJUj7+fnRtm1bWbdVCCGEEKKeq9cB80bS09NJTEysEtbOnTtHUlIS\nmZmZVFRUqPtaW1vj7u6Oo6MjDg4O2NvbY29vj0ajUT8CNG7cWG1jZWWlvhk3GAzo9Xr1Z4WFhZSU\nlFBSUkJ+fj56vZ7c3Fz1c51OR2ZmJjk5OVVqtre3p0WLFlXCsLe3N61atcLb21vWCxW3rbi4mLNn\nz5KUlKT+/VeGtQsXLpCbm1tlfzs7Ozw8PNBqteqm0Wiws7NDo9Gg1WoB0Gq16szDZmZm2Nvbq8fI\nz8+nvLwc+Hv9W51OB4Beryc/P1/ddDodOp0OvV7PxYsXKSgoqFKLg4MDnp6eVUJx5da6dWvpqRdC\nCCGEaKAaZMC8GYPBQFZWltpLkpWVRXp6Orm5ueTl5aHT6dQwqNfrKSgouGGIrHR1+LS0tMTGxoZG\njRqpb8qvDq729vZq76mLiwvu7u5G7z0S96eSkhL17z8zM1P9vPJCyNXPhcpgqCgKeXl56jFKS0u5\ncuWK+rWtrW2ViXMcHBwwMTFBo9FU2SqfE1qtVh1R4OLigoeHB87OzhIghRBCCCHuUfdcwBRCCCGE\nEEIIYRymxi5ACCGEEEIIIcS9QQKmEEIIIYQQQogaIQFTCCGEEEIIIUSNMAc2GrsIIYQQQgghhBAN\n3/8HFgujotwCIfYAAAAASUVORK5CYII=\n"
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%dot\n",
    "digraph G {\n",
    "    rankdir=LR;\n",
    "    node1 [label=\"RNN_Cell\"];\n",
    "    node2 [label=\"RNN_Cell\"];\n",
    "    node3 [label=\"RNN_Cell\"];\n",
    "    first_word -> node1;\n",
    "    second_word -> node2;\n",
    "    nth_word -> node3;\n",
    "    node1 -> node2;\n",
    "    node2 -> node3;\n",
    "    node3 -> Output_Cell;\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "g7fpxHE3iY0W"
   },
   "outputs": [],
   "source": [
    "rnn_input = Input(shape=(100,))\n",
    "embedding = Embedding(num_classification_words, 128, input_length=words_limit)(rnn_input)\n",
    "simple_rnn = SimpleRNN(128)(embedding)\n",
    "dropout = Dropout(0.4)(simple_rnn)\n",
    "dense = Dense(1)(dropout)\n",
    "activation = Activation('sigmoid')(dense)\n",
    "model = Model(rnn_input, activation)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 380
    },
    "colab_type": "code",
    "id": "yrqP4S8_kq_H",
    "outputId": "f662361e-8504-43cd-8928-71eff8c37947"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input_1 (InputLayer)         [(None, 100)]             0         \n",
      "_________________________________________________________________\n",
      "embedding (Embedding)        (None, 100, 128)          2560000   \n",
      "_________________________________________________________________\n",
      "simple_rnn (SimpleRNN)       (None, 128)               32896     \n",
      "_________________________________________________________________\n",
      "dropout (Dropout)            (None, 128)               0         \n",
      "_________________________________________________________________\n",
      "dense (Dense)                (None, 1)                 129       \n",
      "_________________________________________________________________\n",
      "activation (Activation)      (None, 1)                 0         \n",
      "=================================================================\n",
      "Total params: 2,593,025\n",
      "Trainable params: 2,593,025\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "xZ6xspYRPpG5"
   },
   "source": [
    "Optimizer: Adam and Loss Model: Binary CrossEntropy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "utnMKHdjkuDQ"
   },
   "outputs": [],
   "source": [
    "model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 126
    },
    "colab_type": "code",
    "id": "m4ZgxJIalJrr",
    "outputId": "35243812-86f4-45dd-b49a-86b01a45eff1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/3\n",
      "782/782 [==============================] - 79s 101ms/step - loss: 0.6375 - accuracy: 0.6354 - val_loss: 0.6657 - val_accuracy: 0.5817\n",
      "Epoch 2/3\n",
      "782/782 [==============================] - 79s 101ms/step - loss: 0.6258 - accuracy: 0.6465 - val_loss: 0.8133 - val_accuracy: 0.5606\n",
      "Epoch 3/3\n",
      "782/782 [==============================] - 79s 101ms/step - loss: 0.5792 - accuracy: 0.6939 - val_loss: 0.6133 - val_accuracy: 0.6598\n"
     ]
    }
   ],
   "source": [
    "history=model.fit(x_train_seq, y_train, batch_size=32, epochs=3, validation_data=(x_test_seq, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "wZFRvw5RUQp-"
   },
   "source": [
    "Loss Score and Accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 72
    },
    "colab_type": "code",
    "id": "f-SiWS3jRY95",
    "outputId": "bb1e0d9f-fd95-436d-e602-459e416c9185"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "782/782 [==============================] - 6s 8ms/step - loss: 0.6133 - accuracy: 0.6598\n",
      "Score:  0.6132510900497437\n",
      "Accuracy:  0.6598399877548218\n"
     ]
    }
   ],
   "source": [
    "score, acc = model.evaluate(x_test_seq, y_test, batch_size=32)\n",
    "print(\"Score: \",score)\n",
    "print(\"Accuracy: \",acc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "zHIEepZ1Tw3k",
    "outputId": "405a6ab5-5fc1-4c33-d014-4e942b68c746"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['loss', 'accuracy', 'val_loss', 'val_accuracy'])"
      ]
     },
     "execution_count": 21,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "history.history.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 71
    },
    "colab_type": "code",
    "id": "zIwtTxhCT2FQ",
    "outputId": "bf9362da-f268-49c1-8875-aaf8a9eafae9"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "BokehDeprecationWarning: 'legend' keyword is deprecated, use explicit 'legend_label', 'legend_field', or 'legend_group' keywords instead\n",
      "BokehDeprecationWarning: 'legend' keyword is deprecated, use explicit 'legend_label', 'legend_field', or 'legend_group' keywords instead\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div style=\"display: table;\"><div style=\"display: table-row;\"><div style=\"display: table-cell;\"><b title=\"bokeh.models.renderers.GlyphRenderer\">GlyphRenderer</b>(</div><div style=\"display: table-cell;\">id&nbsp;=&nbsp;'1051', <span id=\"1063\" style=\"cursor: pointer;\">&hellip;)</span></div></div><div class=\"1062\" style=\"display: none;\"><div style=\"display: table-cell;\"></div><div style=\"display: table-cell;\">data_source&nbsp;=&nbsp;ColumnDataSource(id='1048', ...),</div></div><div class=\"1062\" style=\"display: none;\"><div style=\"display: table-cell;\"></div><div style=\"display: table-cell;\">glyph&nbsp;=&nbsp;Line(id='1049', ...),</div></div><div class=\"1062\" style=\"display: none;\"><div style=\"display: table-cell;\"></div><div style=\"display: table-cell;\">hover_glyph&nbsp;=&nbsp;None,</div></div><div class=\"1062\" style=\"display: none;\"><div style=\"display: table-cell;\"></div><div style=\"display: table-cell;\">js_event_callbacks&nbsp;=&nbsp;{},</div></div><div class=\"1062\" style=\"display: none;\"><div style=\"display: table-cell;\"></div><div style=\"display: table-cell;\">js_property_callbacks&nbsp;=&nbsp;{},</div></div><div class=\"1062\" style=\"display: none;\"><div style=\"display: table-cell;\"></div><div style=\"display: table-cell;\">level&nbsp;=&nbsp;'glyph',</div></div><div class=\"1062\" style=\"display: none;\"><div style=\"display: table-cell;\"></div><div style=\"display: table-cell;\">muted&nbsp;=&nbsp;False,</div></div><div class=\"1062\" style=\"display: none;\"><div style=\"display: table-cell;\"></div><div style=\"display: table-cell;\">muted_glyph&nbsp;=&nbsp;None,</div></div><div class=\"1062\" style=\"display: none;\"><div style=\"display: table-cell;\"></div><div style=\"display: table-cell;\">name&nbsp;=&nbsp;None,</div></div><div class=\"1062\" style=\"display: none;\"><div style=\"display: table-cell;\"></div><div style=\"display: table-cell;\">nonselection_glyph&nbsp;=&nbsp;Line(id='1050', ...),</div></div><div class=\"1062\" style=\"display: none;\"><div style=\"display: table-cell;\"></div><div style=\"display: table-cell;\">selection_glyph&nbsp;=&nbsp;None,</div></div><div class=\"1062\" style=\"display: none;\"><div style=\"display: table-cell;\"></div><div style=\"display: table-cell;\">subscribed_events&nbsp;=&nbsp;[],</div></div><div class=\"1062\" style=\"display: none;\"><div style=\"display: table-cell;\"></div><div style=\"display: table-cell;\">tags&nbsp;=&nbsp;[],</div></div><div class=\"1062\" style=\"display: none;\"><div style=\"display: table-cell;\"></div><div style=\"display: table-cell;\">view&nbsp;=&nbsp;CDSView(id='1052', ...),</div></div><div class=\"1062\" style=\"display: none;\"><div style=\"display: table-cell;\"></div><div style=\"display: table-cell;\">visible&nbsp;=&nbsp;True,</div></div><div class=\"1062\" style=\"display: none;\"><div style=\"display: table-cell;\"></div><div style=\"display: table-cell;\">x_range_name&nbsp;=&nbsp;'default',</div></div><div class=\"1062\" style=\"display: none;\"><div style=\"display: table-cell;\"></div><div style=\"display: table-cell;\">y_range_name&nbsp;=&nbsp;'default')</div></div></div>\n",
       "<script>\n",
       "(function() {\n",
       "  var expanded = false;\n",
       "  var ellipsis = document.getElementById(\"1063\");\n",
       "  ellipsis.addEventListener(\"click\", function() {\n",
       "    var rows = document.getElementsByClassName(\"1062\");\n",
       "    for (var i = 0; i < rows.length; i++) {\n",
       "      var el = rows[i];\n",
       "      el.style.display = expanded ? \"none\" : \"table-row\";\n",
       "    }\n",
       "    ellipsis.innerHTML = expanded ? \"&hellip;)\" : \"&lsaquo;&lsaquo;&lsaquo;\";\n",
       "    expanded = !expanded;\n",
       "  });\n",
       "})();\n",
       "</script>\n"
      ],
      "text/plain": [
       "GlyphRenderer(id='1051', ...)"
      ]
     },
     "execution_count": 22,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p = figure(title=\"Loss History\", x_axis_label='Time', y_axis_label='Loss')\n",
    "range(len(history.history['val_loss']))\n",
    "p.line(range(len(history.history['val_loss'])), history.history['val_loss'], \n",
    "       legend=\"Val. Loss\", line_width=2, line_color='orange')\n",
    "p.line(range(len(history.history['loss'])), history.history['loss'], \n",
    "       legend=\"Loss\", line_width=2, line_color='blue')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 617
    },
    "colab_type": "code",
    "id": "IuDFkyRnT6uQ",
    "outputId": "e5df5fd5-5038-475d-f56a-46bbc36e0021"
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "\n",
       "(function(root) {\n",
       "  function now() {\n",
       "    return new Date();\n",
       "  }\n",
       "\n",
       "  var force = true;\n",
       "\n",
       "  if (typeof root._bokeh_onload_callbacks === \"undefined\" || force === true) {\n",
       "    root._bokeh_onload_callbacks = [];\n",
       "    root._bokeh_is_loading = undefined;\n",
       "  }\n",
       "\n",
       "  var JS_MIME_TYPE = 'application/javascript';\n",
       "  var HTML_MIME_TYPE = 'text/html';\n",
       "  var EXEC_MIME_TYPE = 'application/vnd.bokehjs_exec.v0+json';\n",
       "  var CLASS_NAME = 'output_bokeh rendered_html';\n",
       "\n",
       "  /**\n",
       "   * Render data to the DOM node\n",
       "   */\n",
       "  function render(props, node) {\n",
       "    var script = document.createElement(\"script\");\n",
       "    node.appendChild(script);\n",
       "  }\n",
       "\n",
       "  /**\n",
       "   * Handle when an output is cleared or removed\n",
       "   */\n",
       "  function handleClearOutput(event, handle) {\n",
       "    var cell = handle.cell;\n",
       "\n",
       "    var id = cell.output_area._bokeh_element_id;\n",
       "    var server_id = cell.output_area._bokeh_server_id;\n",
       "    // Clean up Bokeh references\n",
       "    if (id != null && id in Bokeh.index) {\n",
       "      Bokeh.index[id].model.document.clear();\n",
       "      delete Bokeh.index[id];\n",
       "    }\n",
       "\n",
       "    if (server_id !== undefined) {\n",
       "      // Clean up Bokeh references\n",
       "      var cmd = \"from bokeh.io.state import curstate; print(curstate().uuid_to_server['\" + server_id + \"'].get_sessions()[0].document.roots[0]._id)\";\n",
       "      cell.notebook.kernel.execute(cmd, {\n",
       "        iopub: {\n",
       "          output: function(msg) {\n",
       "            var id = msg.content.text.trim();\n",
       "            if (id in Bokeh.index) {\n",
       "              Bokeh.index[id].model.document.clear();\n",
       "              delete Bokeh.index[id];\n",
       "            }\n",
       "          }\n",
       "        }\n",
       "      });\n",
       "      // Destroy server and session\n",
       "      var cmd = \"import bokeh.io.notebook as ion; ion.destroy_server('\" + server_id + \"')\";\n",
       "      cell.notebook.kernel.execute(cmd);\n",
       "    }\n",
       "  }\n",
       "\n",
       "  /**\n",
       "   * Handle when a new output is added\n",
       "   */\n",
       "  function handleAddOutput(event, handle) {\n",
       "    var output_area = handle.output_area;\n",
       "    var output = handle.output;\n",
       "\n",
       "    // limit handleAddOutput to display_data with EXEC_MIME_TYPE content only\n",
       "    if ((output.output_type != \"display_data\") || (!output.data.hasOwnProperty(EXEC_MIME_TYPE))) {\n",
       "      return\n",
       "    }\n",
       "\n",
       "    var toinsert = output_area.element.find(\".\" + CLASS_NAME.split(' ')[0]);\n",
       "\n",
       "    if (output.metadata[EXEC_MIME_TYPE][\"id\"] !== undefined) {\n",
       "      toinsert[toinsert.length - 1].firstChild.textContent = output.data[JS_MIME_TYPE];\n",
       "      // store reference to embed id on output_area\n",
       "      output_area._bokeh_element_id = output.metadata[EXEC_MIME_TYPE][\"id\"];\n",
       "    }\n",
       "    if (output.metadata[EXEC_MIME_TYPE][\"server_id\"] !== undefined) {\n",
       "      var bk_div = document.createElement(\"div\");\n",
       "      bk_div.innerHTML = output.data[HTML_MIME_TYPE];\n",
       "      var script_attrs = bk_div.children[0].attributes;\n",
       "      for (var i = 0; i < script_attrs.length; i++) {\n",
       "        toinsert[toinsert.length - 1].firstChild.setAttribute(script_attrs[i].name, script_attrs[i].value);\n",
       "      }\n",
       "      // store reference to server id on output_area\n",
       "      output_area._bokeh_server_id = output.metadata[EXEC_MIME_TYPE][\"server_id\"];\n",
       "    }\n",
       "  }\n",
       "\n",
       "  function register_renderer(events, OutputArea) {\n",
       "\n",
       "    function append_mime(data, metadata, element) {\n",
       "      // create a DOM node to render to\n",
       "      var toinsert = this.create_output_subarea(\n",
       "        metadata,\n",
       "        CLASS_NAME,\n",
       "        EXEC_MIME_TYPE\n",
       "      );\n",
       "      this.keyboard_manager.register_events(toinsert);\n",
       "      // Render to node\n",
       "      var props = {data: data, metadata: metadata[EXEC_MIME_TYPE]};\n",
       "      render(props, toinsert[toinsert.length - 1]);\n",
       "      element.append(toinsert);\n",
       "      return toinsert\n",
       "    }\n",
       "\n",
       "    /* Handle when an output is cleared or removed */\n",
       "    events.on('clear_output.CodeCell', handleClearOutput);\n",
       "    events.on('delete.Cell', handleClearOutput);\n",
       "\n",
       "    /* Handle when a new output is added */\n",
       "    events.on('output_added.OutputArea', handleAddOutput);\n",
       "\n",
       "    /**\n",
       "     * Register the mime type and append_mime function with output_area\n",
       "     */\n",
       "    OutputArea.prototype.register_mime_type(EXEC_MIME_TYPE, append_mime, {\n",
       "      /* Is output safe? */\n",
       "      safe: true,\n",
       "      /* Index of renderer in `output_area.display_order` */\n",
       "      index: 0\n",
       "    });\n",
       "  }\n",
       "\n",
       "  // register the mime type if in Jupyter Notebook environment and previously unregistered\n",
       "  if (root.Jupyter !== undefined) {\n",
       "    var events = require('base/js/events');\n",
       "    var OutputArea = require('notebook/js/outputarea').OutputArea;\n",
       "\n",
       "    if (OutputArea.prototype.mime_types().indexOf(EXEC_MIME_TYPE) == -1) {\n",
       "      register_renderer(events, OutputArea);\n",
       "    }\n",
       "  }\n",
       "\n",
       "  \n",
       "  if (typeof (root._bokeh_timeout) === \"undefined\" || force === true) {\n",
       "    root._bokeh_timeout = Date.now() + 5000;\n",
       "    root._bokeh_failed_load = false;\n",
       "  }\n",
       "\n",
       "  var NB_LOAD_WARNING = {'data': {'text/html':\n",
       "     \"<div style='background-color: #fdd'>\\n\"+\n",
       "     \"<p>\\n\"+\n",
       "     \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n",
       "     \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n",
       "     \"</p>\\n\"+\n",
       "     \"<ul>\\n\"+\n",
       "     \"<li>re-rerun `output_notebook()` to attempt to load from CDN again, or</li>\\n\"+\n",
       "     \"<li>use INLINE resources instead, as so:</li>\\n\"+\n",
       "     \"</ul>\\n\"+\n",
       "     \"<code>\\n\"+\n",
       "     \"from bokeh.resources import INLINE\\n\"+\n",
       "     \"output_notebook(resources=INLINE)\\n\"+\n",
       "     \"</code>\\n\"+\n",
       "     \"</div>\"}};\n",
       "\n",
       "  function display_loaded() {\n",
       "    var el = document.getElementById(null);\n",
       "    if (el != null) {\n",
       "      el.textContent = \"BokehJS is loading...\";\n",
       "    }\n",
       "    if (root.Bokeh !== undefined) {\n",
       "      if (el != null) {\n",
       "        el.textContent = \"BokehJS \" + root.Bokeh.version + \" successfully loaded.\";\n",
       "      }\n",
       "    } else if (Date.now() < root._bokeh_timeout) {\n",
       "      setTimeout(display_loaded, 100)\n",
       "    }\n",
       "  }\n",
       "\n",
       "\n",
       "  function run_callbacks() {\n",
       "    try {\n",
       "      root._bokeh_onload_callbacks.forEach(function(callback) {\n",
       "        if (callback != null)\n",
       "          callback();\n",
       "      });\n",
       "    } finally {\n",
       "      delete root._bokeh_onload_callbacks\n",
       "    }\n",
       "    console.debug(\"Bokeh: all callbacks have finished\");\n",
       "  }\n",
       "\n",
       "  function load_libs(css_urls, js_urls, callback) {\n",
       "    if (css_urls == null) css_urls = [];\n",
       "    if (js_urls == null) js_urls = [];\n",
       "\n",
       "    root._bokeh_onload_callbacks.push(callback);\n",
       "    if (root._bokeh_is_loading > 0) {\n",
       "      console.debug(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n",
       "      return null;\n",
       "    }\n",
       "    if (js_urls == null || js_urls.length === 0) {\n",
       "      run_callbacks();\n",
       "      return null;\n",
       "    }\n",
       "    console.debug(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n",
       "    root._bokeh_is_loading = css_urls.length + js_urls.length;\n",
       "\n",
       "    function on_load() {\n",
       "      root._bokeh_is_loading--;\n",
       "      if (root._bokeh_is_loading === 0) {\n",
       "        console.debug(\"Bokeh: all BokehJS libraries/stylesheets loaded\");\n",
       "        run_callbacks()\n",
       "      }\n",
       "    }\n",
       "\n",
       "    function on_error() {\n",
       "      console.error(\"failed to load \" + url);\n",
       "    }\n",
       "\n",
       "    for (var i = 0; i < css_urls.length; i++) {\n",
       "      var url = css_urls[i];\n",
       "      const element = document.createElement(\"link\");\n",
       "      element.onload = on_load;\n",
       "      element.onerror = on_error;\n",
       "      element.rel = \"stylesheet\";\n",
       "      element.type = \"text/css\";\n",
       "      element.href = url;\n",
       "      console.debug(\"Bokeh: injecting link tag for BokehJS stylesheet: \", url);\n",
       "      document.body.appendChild(element);\n",
       "    }\n",
       "\n",
       "    for (var i = 0; i < js_urls.length; i++) {\n",
       "      var url = js_urls[i];\n",
       "      var element = document.createElement('script');\n",
       "      element.onload = on_load;\n",
       "      element.onerror = on_error;\n",
       "      element.async = false;\n",
       "      element.src = url;\n",
       "      console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n",
       "      document.head.appendChild(element);\n",
       "    }\n",
       "  };\n",
       "\n",
       "  function inject_raw_css(css) {\n",
       "    const element = document.createElement(\"style\");\n",
       "    element.appendChild(document.createTextNode(css));\n",
       "    document.body.appendChild(element);\n",
       "  }\n",
       "\n",
       "  \n",
       "  var js_urls = [\"https://cdn.pydata.org/bokeh/release/bokeh-1.4.0.min.js\", \"https://cdn.pydata.org/bokeh/release/bokeh-widgets-1.4.0.min.js\", \"https://cdn.pydata.org/bokeh/release/bokeh-tables-1.4.0.min.js\", \"https://cdn.pydata.org/bokeh/release/bokeh-gl-1.4.0.min.js\"];\n",
       "  var css_urls = [];\n",
       "  \n",
       "\n",
       "  var inline_js = [\n",
       "    function(Bokeh) {\n",
       "      Bokeh.set_log_level(\"info\");\n",
       "    },\n",
       "    function(Bokeh) {\n",
       "    \n",
       "    \n",
       "    }\n",
       "  ];\n",
       "\n",
       "  function run_inline_js() {\n",
       "    \n",
       "    if (root.Bokeh !== undefined || force === true) {\n",
       "      \n",
       "    for (var i = 0; i < inline_js.length; i++) {\n",
       "      inline_js[i].call(root, root.Bokeh);\n",
       "    }\n",
       "    } else if (Date.now() < root._bokeh_timeout) {\n",
       "      setTimeout(run_inline_js, 100);\n",
       "    } else if (!root._bokeh_failed_load) {\n",
       "      console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n",
       "      root._bokeh_failed_load = true;\n",
       "    } else if (force !== true) {\n",
       "      var cell = $(document.getElementById(null)).parents('.cell').data().cell;\n",
       "      cell.output_area.append_execute_result(NB_LOAD_WARNING)\n",
       "    }\n",
       "\n",
       "  }\n",
       "\n",
       "  if (root._bokeh_is_loading === 0) {\n",
       "    console.debug(\"Bokeh: BokehJS loaded, going straight to plotting\");\n",
       "    run_inline_js();\n",
       "  } else {\n",
       "    load_libs(css_urls, js_urls, function() {\n",
       "      console.debug(\"Bokeh: BokehJS plotting callback run at\", now());\n",
       "      run_inline_js();\n",
       "    });\n",
       "  }\n",
       "}(window));"
      ],
      "application/vnd.bokehjs_load.v0+json": "\n(function(root) {\n  function now() {\n    return new Date();\n  }\n\n  var force = true;\n\n  if (typeof root._bokeh_onload_callbacks === \"undefined\" || force === true) {\n    root._bokeh_onload_callbacks = [];\n    root._bokeh_is_loading = undefined;\n  }\n\n  \n\n  \n  if (typeof (root._bokeh_timeout) === \"undefined\" || force === true) {\n    root._bokeh_timeout = Date.now() + 5000;\n    root._bokeh_failed_load = false;\n  }\n\n  var NB_LOAD_WARNING = {'data': {'text/html':\n     \"<div style='background-color: #fdd'>\\n\"+\n     \"<p>\\n\"+\n     \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n     \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n     \"</p>\\n\"+\n     \"<ul>\\n\"+\n     \"<li>re-rerun `output_notebook()` to attempt to load from CDN again, or</li>\\n\"+\n     \"<li>use INLINE resources instead, as so:</li>\\n\"+\n     \"</ul>\\n\"+\n     \"<code>\\n\"+\n     \"from bokeh.resources import INLINE\\n\"+\n     \"output_notebook(resources=INLINE)\\n\"+\n     \"</code>\\n\"+\n     \"</div>\"}};\n\n  function display_loaded() {\n    var el = document.getElementById(null);\n    if (el != null) {\n      el.textContent = \"BokehJS is loading...\";\n    }\n    if (root.Bokeh !== undefined) {\n      if (el != null) {\n        el.textContent = \"BokehJS \" + root.Bokeh.version + \" successfully loaded.\";\n      }\n    } else if (Date.now() < root._bokeh_timeout) {\n      setTimeout(display_loaded, 100)\n    }\n  }\n\n\n  function run_callbacks() {\n    try {\n      root._bokeh_onload_callbacks.forEach(function(callback) {\n        if (callback != null)\n          callback();\n      });\n    } finally {\n      delete root._bokeh_onload_callbacks\n    }\n    console.debug(\"Bokeh: all callbacks have finished\");\n  }\n\n  function load_libs(css_urls, js_urls, callback) {\n    if (css_urls == null) css_urls = [];\n    if (js_urls == null) js_urls = [];\n\n    root._bokeh_onload_callbacks.push(callback);\n    if (root._bokeh_is_loading > 0) {\n      console.debug(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n      return null;\n    }\n    if (js_urls == null || js_urls.length === 0) {\n      run_callbacks();\n      return null;\n    }\n    console.debug(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n    root._bokeh_is_loading = css_urls.length + js_urls.length;\n\n    function on_load() {\n      root._bokeh_is_loading--;\n      if (root._bokeh_is_loading === 0) {\n        console.debug(\"Bokeh: all BokehJS libraries/stylesheets loaded\");\n        run_callbacks()\n      }\n    }\n\n    function on_error() {\n      console.error(\"failed to load \" + url);\n    }\n\n    for (var i = 0; i < css_urls.length; i++) {\n      var url = css_urls[i];\n      const element = document.createElement(\"link\");\n      element.onload = on_load;\n      element.onerror = on_error;\n      element.rel = \"stylesheet\";\n      element.type = \"text/css\";\n      element.href = url;\n      console.debug(\"Bokeh: injecting link tag for BokehJS stylesheet: \", url);\n      document.body.appendChild(element);\n    }\n\n    for (var i = 0; i < js_urls.length; i++) {\n      var url = js_urls[i];\n      var element = document.createElement('script');\n      element.onload = on_load;\n      element.onerror = on_error;\n      element.async = false;\n      element.src = url;\n      console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n      document.head.appendChild(element);\n    }\n  };\n\n  function inject_raw_css(css) {\n    const element = document.createElement(\"style\");\n    element.appendChild(document.createTextNode(css));\n    document.body.appendChild(element);\n  }\n\n  \n  var js_urls = [\"https://cdn.pydata.org/bokeh/release/bokeh-1.4.0.min.js\", \"https://cdn.pydata.org/bokeh/release/bokeh-widgets-1.4.0.min.js\", \"https://cdn.pydata.org/bokeh/release/bokeh-tables-1.4.0.min.js\", \"https://cdn.pydata.org/bokeh/release/bokeh-gl-1.4.0.min.js\"];\n  var css_urls = [];\n  \n\n  var inline_js = [\n    function(Bokeh) {\n      Bokeh.set_log_level(\"info\");\n    },\n    function(Bokeh) {\n    \n    \n    }\n  ];\n\n  function run_inline_js() {\n    \n    if (root.Bokeh !== undefined || force === true) {\n      \n    for (var i = 0; i < inline_js.length; i++) {\n      inline_js[i].call(root, root.Bokeh);\n    }\n    } else if (Date.now() < root._bokeh_timeout) {\n      setTimeout(run_inline_js, 100);\n    } else if (!root._bokeh_failed_load) {\n      console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n      root._bokeh_failed_load = true;\n    } else if (force !== true) {\n      var cell = $(document.getElementById(null)).parents('.cell').data().cell;\n      cell.output_area.append_execute_result(NB_LOAD_WARNING)\n    }\n\n  }\n\n  if (root._bokeh_is_loading === 0) {\n    console.debug(\"Bokeh: BokehJS loaded, going straight to plotting\");\n    run_inline_js();\n  } else {\n    load_libs(css_urls, js_urls, function() {\n      console.debug(\"Bokeh: BokehJS plotting callback run at\", now());\n      run_inline_js();\n    });\n  }\n}(window));"
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "\n",
       "\n",
       "\n",
       "\n",
       "\n",
       "  <div class=\"bk-root\" id=\"e57dd8ef-2959-4696-b87a-b3ef686bf277\" data-root-id=\"1001\"></div>\n"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "application/javascript": [
       "(function(root) {\n",
       "  function embed_document(root) {\n",
       "    \n",
       "  var docs_json = {\"65d5e7a4-c2b1-43a1-96d7-3365e75a2186\":{\"roots\":{\"references\":[{\"attributes\":{\"below\":[{\"id\":\"1012\",\"type\":\"LinearAxis\"}],\"center\":[{\"id\":\"1016\",\"type\":\"Grid\"},{\"id\":\"1021\",\"type\":\"Grid\"},{\"id\":\"1046\",\"type\":\"Legend\"}],\"left\":[{\"id\":\"1017\",\"type\":\"LinearAxis\"}],\"renderers\":[{\"id\":\"1038\",\"type\":\"GlyphRenderer\"},{\"id\":\"1051\",\"type\":\"GlyphRenderer\"}],\"title\":{\"id\":\"1002\",\"type\":\"Title\"},\"toolbar\":{\"id\":\"1028\",\"type\":\"Toolbar\"},\"x_range\":{\"id\":\"1004\",\"type\":\"DataRange1d\"},\"x_scale\":{\"id\":\"1008\",\"type\":\"LinearScale\"},\"y_range\":{\"id\":\"1006\",\"type\":\"DataRange1d\"},\"y_scale\":{\"id\":\"1010\",\"type\":\"LinearScale\"}},\"id\":\"1001\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"source\":{\"id\":\"1035\",\"type\":\"ColumnDataSource\"}},\"id\":\"1039\",\"type\":\"CDSView\"},{\"attributes\":{\"axis_label\":\"Time\",\"formatter\":{\"id\":\"1044\",\"type\":\"BasicTickFormatter\"},\"ticker\":{\"id\":\"1013\",\"type\":\"BasicTicker\"}},\"id\":\"1012\",\"type\":\"LinearAxis\"},{\"attributes\":{\"callback\":null,\"data\":{\"x\":[0,1,2],\"y\":[0.6374900341033936,0.6258423328399658,0.579218864440918]},\"selected\":{\"id\":\"1073\",\"type\":\"Selection\"},\"selection_policy\":{\"id\":\"1072\",\"type\":\"UnionRenderers\"}},\"id\":\"1048\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"1042\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{},\"id\":\"1013\",\"type\":\"BasicTicker\"},{\"attributes\":{},\"id\":\"1044\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"ticker\":{\"id\":\"1013\",\"type\":\"BasicTicker\"}},\"id\":\"1016\",\"type\":\"Grid\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#1f77b4\",\"line_width\":2,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1037\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"1060\",\"type\":\"Selection\"},{\"attributes\":{\"bottom_units\":\"screen\",\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"lightgrey\"},\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":{\"value\":1.0},\"line_color\":{\"value\":\"black\"},\"line_dash\":[4,4],\"line_width\":{\"value\":2},\"render_mode\":\"css\",\"right_units\":\"screen\",\"top_units\":\"screen\"},\"id\":\"1045\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"axis_label\":\"Loss\",\"formatter\":{\"id\":\"1042\",\"type\":\"BasicTickFormatter\"},\"ticker\":{\"id\":\"1018\",\"type\":\"BasicTicker\"}},\"id\":\"1017\",\"type\":\"LinearAxis\"},{\"attributes\":{\"label\":{\"value\":\"Loss\"},\"renderers\":[{\"id\":\"1051\",\"type\":\"GlyphRenderer\"}]},\"id\":\"1061\",\"type\":\"LegendItem\"},{\"attributes\":{},\"id\":\"1018\",\"type\":\"BasicTicker\"},{\"attributes\":{\"dimension\":1,\"ticker\":{\"id\":\"1018\",\"type\":\"BasicTicker\"}},\"id\":\"1021\",\"type\":\"Grid\"},{\"attributes\":{\"line_color\":\"blue\",\"line_width\":2,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1049\",\"type\":\"Line\"},{\"attributes\":{\"active_drag\":\"auto\",\"active_inspect\":\"auto\",\"active_multi\":null,\"active_scroll\":\"auto\",\"active_tap\":\"auto\",\"tools\":[{\"id\":\"1022\",\"type\":\"PanTool\"},{\"id\":\"1023\",\"type\":\"WheelZoomTool\"},{\"id\":\"1024\",\"type\":\"BoxZoomTool\"},{\"id\":\"1025\",\"type\":\"SaveTool\"},{\"id\":\"1026\",\"type\":\"ResetTool\"},{\"id\":\"1027\",\"type\":\"HelpTool\"}]},\"id\":\"1028\",\"type\":\"Toolbar\"},{\"attributes\":{\"source\":{\"id\":\"1048\",\"type\":\"ColumnDataSource\"}},\"id\":\"1052\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"1059\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"1073\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"1022\",\"type\":\"PanTool\"},{\"attributes\":{\"data_source\":{\"id\":\"1048\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"1049\",\"type\":\"Line\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"1050\",\"type\":\"Line\"},\"selection_glyph\":null,\"view\":{\"id\":\"1052\",\"type\":\"CDSView\"}},\"id\":\"1051\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_color\":\"orange\",\"line_width\":2,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1036\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"1023\",\"type\":\"WheelZoomTool\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#1f77b4\",\"line_width\":2,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1050\",\"type\":\"Line\"},{\"attributes\":{\"overlay\":{\"id\":\"1045\",\"type\":\"BoxAnnotation\"}},\"id\":\"1024\",\"type\":\"BoxZoomTool\"},{\"attributes\":{\"callback\":null,\"data\":{\"x\":[0,1,2],\"y\":[0.6656894683837891,0.813258171081543,0.6132510900497437]},\"selected\":{\"id\":\"1060\",\"type\":\"Selection\"},\"selection_policy\":{\"id\":\"1059\",\"type\":\"UnionRenderers\"}},\"id\":\"1035\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"1025\",\"type\":\"SaveTool\"},{\"attributes\":{\"label\":{\"value\":\"Val. Loss\"},\"renderers\":[{\"id\":\"1038\",\"type\":\"GlyphRenderer\"}]},\"id\":\"1047\",\"type\":\"LegendItem\"},{\"attributes\":{},\"id\":\"1026\",\"type\":\"ResetTool\"},{\"attributes\":{},\"id\":\"1072\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"data_source\":{\"id\":\"1035\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"1036\",\"type\":\"Line\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"1037\",\"type\":\"Line\"},\"selection_glyph\":null,\"view\":{\"id\":\"1039\",\"type\":\"CDSView\"}},\"id\":\"1038\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"text\":\"Loss History\"},\"id\":\"1002\",\"type\":\"Title\"},{\"attributes\":{},\"id\":\"1027\",\"type\":\"HelpTool\"},{\"attributes\":{\"items\":[{\"id\":\"1047\",\"type\":\"LegendItem\"},{\"id\":\"1061\",\"type\":\"LegendItem\"}]},\"id\":\"1046\",\"type\":\"Legend\"},{\"attributes\":{\"callback\":null},\"id\":\"1004\",\"type\":\"DataRange1d\"},{\"attributes\":{\"callback\":null},\"id\":\"1006\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"1008\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"1010\",\"type\":\"LinearScale\"}],\"root_ids\":[\"1001\"]},\"title\":\"Bokeh Application\",\"version\":\"1.4.0\"}};\n",
       "  var render_items = [{\"docid\":\"65d5e7a4-c2b1-43a1-96d7-3365e75a2186\",\"roots\":{\"1001\":\"e57dd8ef-2959-4696-b87a-b3ef686bf277\"}}];\n",
       "  root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
       "\n",
       "  }\n",
       "  if (root.Bokeh !== undefined) {\n",
       "    embed_document(root);\n",
       "  } else {\n",
       "    var attempts = 0;\n",
       "    var timer = setInterval(function(root) {\n",
       "      if (root.Bokeh !== undefined) {\n",
       "        clearInterval(timer);\n",
       "        embed_document(root);\n",
       "      } else {\n",
       "        attempts++;\n",
       "        if (attempts > 100) {\n",
       "          clearInterval(timer);\n",
       "          console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\");\n",
       "        }\n",
       "      }\n",
       "    }, 10, root)\n",
       "  }\n",
       "})(window);"
      ],
      "application/vnd.bokehjs_exec.v0+json": ""
     },
     "metadata": {
      "application/vnd.bokehjs_exec.v0+json": {
       "id": "1001"
      },
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "output_notebook()\n",
    "show(p)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 653
    },
    "colab_type": "code",
    "id": "Wr3jPKEsalgg",
    "outputId": "f52a0ec6-5d75-428a-b5f9-505604bdf827"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "BokehDeprecationWarning: 'legend' keyword is deprecated, use explicit 'legend_label', 'legend_field', or 'legend_group' keywords instead\n",
      "BokehDeprecationWarning: 'legend' keyword is deprecated, use explicit 'legend_label', 'legend_field', or 'legend_group' keywords instead\n"
     ]
    },
    {
     "data": {
      "application/javascript": [
       "\n",
       "(function(root) {\n",
       "  function now() {\n",
       "    return new Date();\n",
       "  }\n",
       "\n",
       "  var force = true;\n",
       "\n",
       "  if (typeof root._bokeh_onload_callbacks === \"undefined\" || force === true) {\n",
       "    root._bokeh_onload_callbacks = [];\n",
       "    root._bokeh_is_loading = undefined;\n",
       "  }\n",
       "\n",
       "  var JS_MIME_TYPE = 'application/javascript';\n",
       "  var HTML_MIME_TYPE = 'text/html';\n",
       "  var EXEC_MIME_TYPE = 'application/vnd.bokehjs_exec.v0+json';\n",
       "  var CLASS_NAME = 'output_bokeh rendered_html';\n",
       "\n",
       "  /**\n",
       "   * Render data to the DOM node\n",
       "   */\n",
       "  function render(props, node) {\n",
       "    var script = document.createElement(\"script\");\n",
       "    node.appendChild(script);\n",
       "  }\n",
       "\n",
       "  /**\n",
       "   * Handle when an output is cleared or removed\n",
       "   */\n",
       "  function handleClearOutput(event, handle) {\n",
       "    var cell = handle.cell;\n",
       "\n",
       "    var id = cell.output_area._bokeh_element_id;\n",
       "    var server_id = cell.output_area._bokeh_server_id;\n",
       "    // Clean up Bokeh references\n",
       "    if (id != null && id in Bokeh.index) {\n",
       "      Bokeh.index[id].model.document.clear();\n",
       "      delete Bokeh.index[id];\n",
       "    }\n",
       "\n",
       "    if (server_id !== undefined) {\n",
       "      // Clean up Bokeh references\n",
       "      var cmd = \"from bokeh.io.state import curstate; print(curstate().uuid_to_server['\" + server_id + \"'].get_sessions()[0].document.roots[0]._id)\";\n",
       "      cell.notebook.kernel.execute(cmd, {\n",
       "        iopub: {\n",
       "          output: function(msg) {\n",
       "            var id = msg.content.text.trim();\n",
       "            if (id in Bokeh.index) {\n",
       "              Bokeh.index[id].model.document.clear();\n",
       "              delete Bokeh.index[id];\n",
       "            }\n",
       "          }\n",
       "        }\n",
       "      });\n",
       "      // Destroy server and session\n",
       "      var cmd = \"import bokeh.io.notebook as ion; ion.destroy_server('\" + server_id + \"')\";\n",
       "      cell.notebook.kernel.execute(cmd);\n",
       "    }\n",
       "  }\n",
       "\n",
       "  /**\n",
       "   * Handle when a new output is added\n",
       "   */\n",
       "  function handleAddOutput(event, handle) {\n",
       "    var output_area = handle.output_area;\n",
       "    var output = handle.output;\n",
       "\n",
       "    // limit handleAddOutput to display_data with EXEC_MIME_TYPE content only\n",
       "    if ((output.output_type != \"display_data\") || (!output.data.hasOwnProperty(EXEC_MIME_TYPE))) {\n",
       "      return\n",
       "    }\n",
       "\n",
       "    var toinsert = output_area.element.find(\".\" + CLASS_NAME.split(' ')[0]);\n",
       "\n",
       "    if (output.metadata[EXEC_MIME_TYPE][\"id\"] !== undefined) {\n",
       "      toinsert[toinsert.length - 1].firstChild.textContent = output.data[JS_MIME_TYPE];\n",
       "      // store reference to embed id on output_area\n",
       "      output_area._bokeh_element_id = output.metadata[EXEC_MIME_TYPE][\"id\"];\n",
       "    }\n",
       "    if (output.metadata[EXEC_MIME_TYPE][\"server_id\"] !== undefined) {\n",
       "      var bk_div = document.createElement(\"div\");\n",
       "      bk_div.innerHTML = output.data[HTML_MIME_TYPE];\n",
       "      var script_attrs = bk_div.children[0].attributes;\n",
       "      for (var i = 0; i < script_attrs.length; i++) {\n",
       "        toinsert[toinsert.length - 1].firstChild.setAttribute(script_attrs[i].name, script_attrs[i].value);\n",
       "      }\n",
       "      // store reference to server id on output_area\n",
       "      output_area._bokeh_server_id = output.metadata[EXEC_MIME_TYPE][\"server_id\"];\n",
       "    }\n",
       "  }\n",
       "\n",
       "  function register_renderer(events, OutputArea) {\n",
       "\n",
       "    function append_mime(data, metadata, element) {\n",
       "      // create a DOM node to render to\n",
       "      var toinsert = this.create_output_subarea(\n",
       "        metadata,\n",
       "        CLASS_NAME,\n",
       "        EXEC_MIME_TYPE\n",
       "      );\n",
       "      this.keyboard_manager.register_events(toinsert);\n",
       "      // Render to node\n",
       "      var props = {data: data, metadata: metadata[EXEC_MIME_TYPE]};\n",
       "      render(props, toinsert[toinsert.length - 1]);\n",
       "      element.append(toinsert);\n",
       "      return toinsert\n",
       "    }\n",
       "\n",
       "    /* Handle when an output is cleared or removed */\n",
       "    events.on('clear_output.CodeCell', handleClearOutput);\n",
       "    events.on('delete.Cell', handleClearOutput);\n",
       "\n",
       "    /* Handle when a new output is added */\n",
       "    events.on('output_added.OutputArea', handleAddOutput);\n",
       "\n",
       "    /**\n",
       "     * Register the mime type and append_mime function with output_area\n",
       "     */\n",
       "    OutputArea.prototype.register_mime_type(EXEC_MIME_TYPE, append_mime, {\n",
       "      /* Is output safe? */\n",
       "      safe: true,\n",
       "      /* Index of renderer in `output_area.display_order` */\n",
       "      index: 0\n",
       "    });\n",
       "  }\n",
       "\n",
       "  // register the mime type if in Jupyter Notebook environment and previously unregistered\n",
       "  if (root.Jupyter !== undefined) {\n",
       "    var events = require('base/js/events');\n",
       "    var OutputArea = require('notebook/js/outputarea').OutputArea;\n",
       "\n",
       "    if (OutputArea.prototype.mime_types().indexOf(EXEC_MIME_TYPE) == -1) {\n",
       "      register_renderer(events, OutputArea);\n",
       "    }\n",
       "  }\n",
       "\n",
       "  \n",
       "  if (typeof (root._bokeh_timeout) === \"undefined\" || force === true) {\n",
       "    root._bokeh_timeout = Date.now() + 5000;\n",
       "    root._bokeh_failed_load = false;\n",
       "  }\n",
       "\n",
       "  var NB_LOAD_WARNING = {'data': {'text/html':\n",
       "     \"<div style='background-color: #fdd'>\\n\"+\n",
       "     \"<p>\\n\"+\n",
       "     \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n",
       "     \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n",
       "     \"</p>\\n\"+\n",
       "     \"<ul>\\n\"+\n",
       "     \"<li>re-rerun `output_notebook()` to attempt to load from CDN again, or</li>\\n\"+\n",
       "     \"<li>use INLINE resources instead, as so:</li>\\n\"+\n",
       "     \"</ul>\\n\"+\n",
       "     \"<code>\\n\"+\n",
       "     \"from bokeh.resources import INLINE\\n\"+\n",
       "     \"output_notebook(resources=INLINE)\\n\"+\n",
       "     \"</code>\\n\"+\n",
       "     \"</div>\"}};\n",
       "\n",
       "  function display_loaded() {\n",
       "    var el = document.getElementById(null);\n",
       "    if (el != null) {\n",
       "      el.textContent = \"BokehJS is loading...\";\n",
       "    }\n",
       "    if (root.Bokeh !== undefined) {\n",
       "      if (el != null) {\n",
       "        el.textContent = \"BokehJS \" + root.Bokeh.version + \" successfully loaded.\";\n",
       "      }\n",
       "    } else if (Date.now() < root._bokeh_timeout) {\n",
       "      setTimeout(display_loaded, 100)\n",
       "    }\n",
       "  }\n",
       "\n",
       "\n",
       "  function run_callbacks() {\n",
       "    try {\n",
       "      root._bokeh_onload_callbacks.forEach(function(callback) {\n",
       "        if (callback != null)\n",
       "          callback();\n",
       "      });\n",
       "    } finally {\n",
       "      delete root._bokeh_onload_callbacks\n",
       "    }\n",
       "    console.debug(\"Bokeh: all callbacks have finished\");\n",
       "  }\n",
       "\n",
       "  function load_libs(css_urls, js_urls, callback) {\n",
       "    if (css_urls == null) css_urls = [];\n",
       "    if (js_urls == null) js_urls = [];\n",
       "\n",
       "    root._bokeh_onload_callbacks.push(callback);\n",
       "    if (root._bokeh_is_loading > 0) {\n",
       "      console.debug(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n",
       "      return null;\n",
       "    }\n",
       "    if (js_urls == null || js_urls.length === 0) {\n",
       "      run_callbacks();\n",
       "      return null;\n",
       "    }\n",
       "    console.debug(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n",
       "    root._bokeh_is_loading = css_urls.length + js_urls.length;\n",
       "\n",
       "    function on_load() {\n",
       "      root._bokeh_is_loading--;\n",
       "      if (root._bokeh_is_loading === 0) {\n",
       "        console.debug(\"Bokeh: all BokehJS libraries/stylesheets loaded\");\n",
       "        run_callbacks()\n",
       "      }\n",
       "    }\n",
       "\n",
       "    function on_error() {\n",
       "      console.error(\"failed to load \" + url);\n",
       "    }\n",
       "\n",
       "    for (var i = 0; i < css_urls.length; i++) {\n",
       "      var url = css_urls[i];\n",
       "      const element = document.createElement(\"link\");\n",
       "      element.onload = on_load;\n",
       "      element.onerror = on_error;\n",
       "      element.rel = \"stylesheet\";\n",
       "      element.type = \"text/css\";\n",
       "      element.href = url;\n",
       "      console.debug(\"Bokeh: injecting link tag for BokehJS stylesheet: \", url);\n",
       "      document.body.appendChild(element);\n",
       "    }\n",
       "\n",
       "    for (var i = 0; i < js_urls.length; i++) {\n",
       "      var url = js_urls[i];\n",
       "      var element = document.createElement('script');\n",
       "      element.onload = on_load;\n",
       "      element.onerror = on_error;\n",
       "      element.async = false;\n",
       "      element.src = url;\n",
       "      console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n",
       "      document.head.appendChild(element);\n",
       "    }\n",
       "  };\n",
       "\n",
       "  function inject_raw_css(css) {\n",
       "    const element = document.createElement(\"style\");\n",
       "    element.appendChild(document.createTextNode(css));\n",
       "    document.body.appendChild(element);\n",
       "  }\n",
       "\n",
       "  \n",
       "  var js_urls = [\"https://cdn.pydata.org/bokeh/release/bokeh-1.4.0.min.js\", \"https://cdn.pydata.org/bokeh/release/bokeh-widgets-1.4.0.min.js\", \"https://cdn.pydata.org/bokeh/release/bokeh-tables-1.4.0.min.js\", \"https://cdn.pydata.org/bokeh/release/bokeh-gl-1.4.0.min.js\"];\n",
       "  var css_urls = [];\n",
       "  \n",
       "\n",
       "  var inline_js = [\n",
       "    function(Bokeh) {\n",
       "      Bokeh.set_log_level(\"info\");\n",
       "    },\n",
       "    function(Bokeh) {\n",
       "    \n",
       "    \n",
       "    }\n",
       "  ];\n",
       "\n",
       "  function run_inline_js() {\n",
       "    \n",
       "    if (root.Bokeh !== undefined || force === true) {\n",
       "      \n",
       "    for (var i = 0; i < inline_js.length; i++) {\n",
       "      inline_js[i].call(root, root.Bokeh);\n",
       "    }\n",
       "    } else if (Date.now() < root._bokeh_timeout) {\n",
       "      setTimeout(run_inline_js, 100);\n",
       "    } else if (!root._bokeh_failed_load) {\n",
       "      console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n",
       "      root._bokeh_failed_load = true;\n",
       "    } else if (force !== true) {\n",
       "      var cell = $(document.getElementById(null)).parents('.cell').data().cell;\n",
       "      cell.output_area.append_execute_result(NB_LOAD_WARNING)\n",
       "    }\n",
       "\n",
       "  }\n",
       "\n",
       "  if (root._bokeh_is_loading === 0) {\n",
       "    console.debug(\"Bokeh: BokehJS loaded, going straight to plotting\");\n",
       "    run_inline_js();\n",
       "  } else {\n",
       "    load_libs(css_urls, js_urls, function() {\n",
       "      console.debug(\"Bokeh: BokehJS plotting callback run at\", now());\n",
       "      run_inline_js();\n",
       "    });\n",
       "  }\n",
       "}(window));"
      ],
      "application/vnd.bokehjs_load.v0+json": "\n(function(root) {\n  function now() {\n    return new Date();\n  }\n\n  var force = true;\n\n  if (typeof root._bokeh_onload_callbacks === \"undefined\" || force === true) {\n    root._bokeh_onload_callbacks = [];\n    root._bokeh_is_loading = undefined;\n  }\n\n  \n\n  \n  if (typeof (root._bokeh_timeout) === \"undefined\" || force === true) {\n    root._bokeh_timeout = Date.now() + 5000;\n    root._bokeh_failed_load = false;\n  }\n\n  var NB_LOAD_WARNING = {'data': {'text/html':\n     \"<div style='background-color: #fdd'>\\n\"+\n     \"<p>\\n\"+\n     \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n     \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n     \"</p>\\n\"+\n     \"<ul>\\n\"+\n     \"<li>re-rerun `output_notebook()` to attempt to load from CDN again, or</li>\\n\"+\n     \"<li>use INLINE resources instead, as so:</li>\\n\"+\n     \"</ul>\\n\"+\n     \"<code>\\n\"+\n     \"from bokeh.resources import INLINE\\n\"+\n     \"output_notebook(resources=INLINE)\\n\"+\n     \"</code>\\n\"+\n     \"</div>\"}};\n\n  function display_loaded() {\n    var el = document.getElementById(null);\n    if (el != null) {\n      el.textContent = \"BokehJS is loading...\";\n    }\n    if (root.Bokeh !== undefined) {\n      if (el != null) {\n        el.textContent = \"BokehJS \" + root.Bokeh.version + \" successfully loaded.\";\n      }\n    } else if (Date.now() < root._bokeh_timeout) {\n      setTimeout(display_loaded, 100)\n    }\n  }\n\n\n  function run_callbacks() {\n    try {\n      root._bokeh_onload_callbacks.forEach(function(callback) {\n        if (callback != null)\n          callback();\n      });\n    } finally {\n      delete root._bokeh_onload_callbacks\n    }\n    console.debug(\"Bokeh: all callbacks have finished\");\n  }\n\n  function load_libs(css_urls, js_urls, callback) {\n    if (css_urls == null) css_urls = [];\n    if (js_urls == null) js_urls = [];\n\n    root._bokeh_onload_callbacks.push(callback);\n    if (root._bokeh_is_loading > 0) {\n      console.debug(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n      return null;\n    }\n    if (js_urls == null || js_urls.length === 0) {\n      run_callbacks();\n      return null;\n    }\n    console.debug(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n    root._bokeh_is_loading = css_urls.length + js_urls.length;\n\n    function on_load() {\n      root._bokeh_is_loading--;\n      if (root._bokeh_is_loading === 0) {\n        console.debug(\"Bokeh: all BokehJS libraries/stylesheets loaded\");\n        run_callbacks()\n      }\n    }\n\n    function on_error() {\n      console.error(\"failed to load \" + url);\n    }\n\n    for (var i = 0; i < css_urls.length; i++) {\n      var url = css_urls[i];\n      const element = document.createElement(\"link\");\n      element.onload = on_load;\n      element.onerror = on_error;\n      element.rel = \"stylesheet\";\n      element.type = \"text/css\";\n      element.href = url;\n      console.debug(\"Bokeh: injecting link tag for BokehJS stylesheet: \", url);\n      document.body.appendChild(element);\n    }\n\n    for (var i = 0; i < js_urls.length; i++) {\n      var url = js_urls[i];\n      var element = document.createElement('script');\n      element.onload = on_load;\n      element.onerror = on_error;\n      element.async = false;\n      element.src = url;\n      console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n      document.head.appendChild(element);\n    }\n  };\n\n  function inject_raw_css(css) {\n    const element = document.createElement(\"style\");\n    element.appendChild(document.createTextNode(css));\n    document.body.appendChild(element);\n  }\n\n  \n  var js_urls = [\"https://cdn.pydata.org/bokeh/release/bokeh-1.4.0.min.js\", \"https://cdn.pydata.org/bokeh/release/bokeh-widgets-1.4.0.min.js\", \"https://cdn.pydata.org/bokeh/release/bokeh-tables-1.4.0.min.js\", \"https://cdn.pydata.org/bokeh/release/bokeh-gl-1.4.0.min.js\"];\n  var css_urls = [];\n  \n\n  var inline_js = [\n    function(Bokeh) {\n      Bokeh.set_log_level(\"info\");\n    },\n    function(Bokeh) {\n    \n    \n    }\n  ];\n\n  function run_inline_js() {\n    \n    if (root.Bokeh !== undefined || force === true) {\n      \n    for (var i = 0; i < inline_js.length; i++) {\n      inline_js[i].call(root, root.Bokeh);\n    }\n    } else if (Date.now() < root._bokeh_timeout) {\n      setTimeout(run_inline_js, 100);\n    } else if (!root._bokeh_failed_load) {\n      console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n      root._bokeh_failed_load = true;\n    } else if (force !== true) {\n      var cell = $(document.getElementById(null)).parents('.cell').data().cell;\n      cell.output_area.append_execute_result(NB_LOAD_WARNING)\n    }\n\n  }\n\n  if (root._bokeh_is_loading === 0) {\n    console.debug(\"Bokeh: BokehJS loaded, going straight to plotting\");\n    run_inline_js();\n  } else {\n    load_libs(css_urls, js_urls, function() {\n      console.debug(\"Bokeh: BokehJS plotting callback run at\", now());\n      run_inline_js();\n    });\n  }\n}(window));"
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "\n",
       "\n",
       "\n",
       "\n",
       "\n",
       "  <div class=\"bk-root\" id=\"31a42f3f-55b6-4e30-80a7-fb43e9f17ce0\" data-root-id=\"1134\"></div>\n"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "application/javascript": [
       "(function(root) {\n",
       "  function embed_document(root) {\n",
       "    \n",
       "  var docs_json = {\"f8b425c7-7a3d-497a-bcc8-02acbad18bc2\":{\"roots\":{\"references\":[{\"attributes\":{\"below\":[{\"id\":\"1145\",\"type\":\"LinearAxis\"}],\"center\":[{\"id\":\"1149\",\"type\":\"Grid\"},{\"id\":\"1154\",\"type\":\"Grid\"},{\"id\":\"1179\",\"type\":\"Legend\"}],\"left\":[{\"id\":\"1150\",\"type\":\"LinearAxis\"}],\"renderers\":[{\"id\":\"1171\",\"type\":\"GlyphRenderer\"},{\"id\":\"1184\",\"type\":\"GlyphRenderer\"}],\"title\":{\"id\":\"1135\",\"type\":\"Title\"},\"toolbar\":{\"id\":\"1161\",\"type\":\"Toolbar\"},\"x_range\":{\"id\":\"1137\",\"type\":\"DataRange1d\"},\"x_scale\":{\"id\":\"1141\",\"type\":\"LinearScale\"},\"y_range\":{\"id\":\"1139\",\"type\":\"DataRange1d\"},\"y_scale\":{\"id\":\"1143\",\"type\":\"LinearScale\"}},\"id\":\"1134\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{},\"id\":\"1143\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"1193\",\"type\":\"Selection\"},{\"attributes\":{\"bottom_units\":\"screen\",\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"lightgrey\"},\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":{\"value\":1.0},\"line_color\":{\"value\":\"black\"},\"line_dash\":[4,4],\"line_width\":{\"value\":2},\"render_mode\":\"css\",\"right_units\":\"screen\",\"top_units\":\"screen\"},\"id\":\"1178\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#1f77b4\",\"line_width\":2,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1183\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"1141\",\"type\":\"LinearScale\"},{\"attributes\":{\"data_source\":{\"id\":\"1181\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"1182\",\"type\":\"Line\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"1183\",\"type\":\"Line\"},\"selection_glyph\":null,\"view\":{\"id\":\"1185\",\"type\":\"CDSView\"}},\"id\":\"1184\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"active_drag\":\"auto\",\"active_inspect\":\"auto\",\"active_multi\":null,\"active_scroll\":\"auto\",\"active_tap\":\"auto\",\"tools\":[{\"id\":\"1155\",\"type\":\"PanTool\"},{\"id\":\"1156\",\"type\":\"WheelZoomTool\"},{\"id\":\"1157\",\"type\":\"BoxZoomTool\"},{\"id\":\"1158\",\"type\":\"SaveTool\"},{\"id\":\"1159\",\"type\":\"ResetTool\"},{\"id\":\"1160\",\"type\":\"HelpTool\"}]},\"id\":\"1161\",\"type\":\"Toolbar\"},{\"attributes\":{\"line_color\":\"blue\",\"line_width\":2,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1182\",\"type\":\"Line\"},{\"attributes\":{\"label\":{\"value\":\"Acc\"},\"renderers\":[{\"id\":\"1184\",\"type\":\"GlyphRenderer\"}]},\"id\":\"1194\",\"type\":\"LegendItem\"},{\"attributes\":{},\"id\":\"1192\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"1146\",\"type\":\"BasicTicker\"},{\"attributes\":{},\"id\":\"1158\",\"type\":\"SaveTool\"},{\"attributes\":{\"line_color\":\"orange\",\"line_width\":2,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1169\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"1214\",\"type\":\"Selection\"},{\"attributes\":{\"text\":\"Accuracy History\"},\"id\":\"1135\",\"type\":\"Title\"},{\"attributes\":{\"dimension\":1,\"ticker\":{\"id\":\"1151\",\"type\":\"BasicTicker\"}},\"id\":\"1154\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"1155\",\"type\":\"PanTool\"},{\"attributes\":{\"items\":[{\"id\":\"1180\",\"type\":\"LegendItem\"},{\"id\":\"1194\",\"type\":\"LegendItem\"}]},\"id\":\"1179\",\"type\":\"Legend\"},{\"attributes\":{},\"id\":\"1151\",\"type\":\"BasicTicker\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#1f77b4\",\"line_width\":2,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1170\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"1177\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"callback\":null,\"data\":{\"x\":[0,1,2],\"y\":[0.5817199945449829,0.5605999827384949,0.6598399877548218]},\"selected\":{\"id\":\"1193\",\"type\":\"Selection\"},\"selection_policy\":{\"id\":\"1192\",\"type\":\"UnionRenderers\"}},\"id\":\"1168\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"callback\":null,\"data\":{\"x\":[0,1,2],\"y\":[0.6354399919509888,0.6465200185775757,0.693880021572113]},\"selected\":{\"id\":\"1214\",\"type\":\"Selection\"},\"selection_policy\":{\"id\":\"1213\",\"type\":\"UnionRenderers\"}},\"id\":\"1181\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"ticker\":{\"id\":\"1146\",\"type\":\"BasicTicker\"}},\"id\":\"1149\",\"type\":\"Grid\"},{\"attributes\":{\"label\":{\"value\":\"Val. Acc\"},\"renderers\":[{\"id\":\"1171\",\"type\":\"GlyphRenderer\"}]},\"id\":\"1180\",\"type\":\"LegendItem\"},{\"attributes\":{},\"id\":\"1160\",\"type\":\"HelpTool\"},{\"attributes\":{\"axis_label\":\"Time\",\"formatter\":{\"id\":\"1177\",\"type\":\"BasicTickFormatter\"},\"ticker\":{\"id\":\"1146\",\"type\":\"BasicTicker\"}},\"id\":\"1145\",\"type\":\"LinearAxis\"},{\"attributes\":{\"callback\":null},\"id\":\"1137\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"1213\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"axis_label\":\"Loss\",\"formatter\":{\"id\":\"1175\",\"type\":\"BasicTickFormatter\"},\"ticker\":{\"id\":\"1151\",\"type\":\"BasicTicker\"}},\"id\":\"1150\",\"type\":\"LinearAxis\"},{\"attributes\":{\"callback\":null},\"id\":\"1139\",\"type\":\"DataRange1d\"},{\"attributes\":{\"overlay\":{\"id\":\"1178\",\"type\":\"BoxAnnotation\"}},\"id\":\"1157\",\"type\":\"BoxZoomTool\"},{\"attributes\":{\"source\":{\"id\":\"1181\",\"type\":\"ColumnDataSource\"}},\"id\":\"1185\",\"type\":\"CDSView\"},{\"attributes\":{\"source\":{\"id\":\"1168\",\"type\":\"ColumnDataSource\"}},\"id\":\"1172\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"1159\",\"type\":\"ResetTool\"},{\"attributes\":{\"data_source\":{\"id\":\"1168\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"1169\",\"type\":\"Line\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"1170\",\"type\":\"Line\"},\"selection_glyph\":null,\"view\":{\"id\":\"1172\",\"type\":\"CDSView\"}},\"id\":\"1171\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"1156\",\"type\":\"WheelZoomTool\"},{\"attributes\":{},\"id\":\"1175\",\"type\":\"BasicTickFormatter\"}],\"root_ids\":[\"1134\"]},\"title\":\"Bokeh Application\",\"version\":\"1.4.0\"}};\n",
       "  var render_items = [{\"docid\":\"f8b425c7-7a3d-497a-bcc8-02acbad18bc2\",\"roots\":{\"1134\":\"31a42f3f-55b6-4e30-80a7-fb43e9f17ce0\"}}];\n",
       "  root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
       "\n",
       "  }\n",
       "  if (root.Bokeh !== undefined) {\n",
       "    embed_document(root);\n",
       "  } else {\n",
       "    var attempts = 0;\n",
       "    var timer = setInterval(function(root) {\n",
       "      if (root.Bokeh !== undefined) {\n",
       "        clearInterval(timer);\n",
       "        embed_document(root);\n",
       "      } else {\n",
       "        attempts++;\n",
       "        if (attempts > 100) {\n",
       "          clearInterval(timer);\n",
       "          console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\");\n",
       "        }\n",
       "      }\n",
       "    }, 10, root)\n",
       "  }\n",
       "})(window);"
      ],
      "application/vnd.bokehjs_exec.v0+json": ""
     },
     "metadata": {
      "application/vnd.bokehjs_exec.v0+json": {
       "id": "1134"
      },
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "p = figure(title=\"Accuracy History\", x_axis_label='Time', y_axis_label='Loss')\n",
    "p.line(range(len(history.history['val_accuracy'])), history.history['val_accuracy'], \n",
    "       legend=\"Val. Acc\", line_width=2, line_color='orange')\n",
    "p.line(range(len(history.history['accuracy'])), history.history['accuracy'], \n",
    "       legend=\"Acc\", line_width=2, line_color='blue')\n",
    "output_notebook()\n",
    "show(p)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Sz5DDjQlE1ZG"
   },
   "source": [
    "# Conclusion\n",
    "RNN and its kind are a great way to do sentiment analysis with minimum amount of workflow. The validation accuracy is going down and the model accuracy is going up [overfitting]. Since the change is negligible, we can ignore this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "VxPkViEi0VFa"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "SentimentAnalysis1.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}