Sentiment_Analysis.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Sentiment_Analysis.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3",
      "language": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/akalgreadis/deep-learning-playground/blob/master/Sentiment_Analysis.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "metadata": {
        "id": "6KxVXIWjohky",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Deep learning Sentiment Analysis\n",
        "\n",
        "In this hands-on exercise we are going to learn to use Recurrent Neural Network (RNN) to perform sentiment analysis on movie reviews. \n",
        "\n",
        "The tools we are going to use for this exercise are :\n",
        "\n",
        "* **Tensorflow** - an open-source software library for dataflow programming across a range of tasks. It is a symbolic math library, and is also used for machine learning applications such as neural networks.\n",
        "\n",
        "\n",
        "* **Keras** - a heavyweight wrapper for Tensorflow. It’s minimalistic, modular, and awesome for rapid experimentation. This is our favorite Python library for deep learning and the best place to start for beginners.\n",
        "\n",
        "\n",
        "* **Google CoLab** - an implementation of the widely used open-source Jupyter Notebook that has most popular libraries used for machine learning already preinstalled. Its a perfect tool for rapid prototyping and experimentation.\n"
      ]
    },
    {
      "metadata": {
        "trusted": true,
        "id": "Wa-WVdcyohk_",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 218
        },
        "outputId": "2183e3f2-aa39-4594-bde3-3ec89a32d24f"
      },
      "cell_type": "code",
      "source": [
        "from IPython.display import Image\n",
        "Image('https://preview.ibb.co/bC0Kfq/sentiment-Analysis-Workshop.jpg')"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "image/jpeg": "/9j/4AAQSkZJRgABAQEAYABgAAD//gA7Q1JFQVRPUjogZ2QtanBlZyB2MS4wICh1c2luZyBJSkcg\nSlBFRyB2NjIpLCBxdWFsaXR5ID0gOTAK/9sAQwADAgIDAgIDAwMDBAMDBAUIBQUEBAUKBwcGCAwK\nDAwLCgsLDQ4SEA0OEQ4LCxAWEBETFBUVFQwPFxgWFBgSFBUU/9sAQwEDBAQFBAUJBQUJFA0LDRQU\nFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQU/8AAEQgAyQKA\nAwEiAAIRAQMRAf/EAB8AAAEFAQEBAQEBAAAAAAAAAAABAgMEBQYHCAkKC//EALUQAAIBAwMCBAMF\nBQQEAAABfQECAwAEEQUSITFBBhNRYQcicRQygZGhCCNCscEVUtHwJDNicoIJChYXGBkaJSYnKCkq\nNDU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6g4SFhoeIiYqSk5SVlpeYmZqi\no6Slpqeoqaqys7S1tre4ubrCw8TFxsfIycrS09TV1tfY2drh4uPk5ebn6Onq8fLz9PX29/j5+v/E\nAB8BAAMBAQEBAQEBAQEAAAAAAAABAgMEBQYHCAkKC//EALURAAIBAgQEAwQHBQQEAAECdwABAgMR\nBAUhMQYSQVEHYXETIjKBCBRCkaGxwQkjM1LwFWJy0QoWJDThJfEXGBkaJicoKSo1Njc4OTpDREVG\nR0hJSlNUVVZXWFlaY2RlZmdoaWpzdHV2d3h5eoKDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKz\ntLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uLj5OXm5+jp6vLz9PX29/j5+v/aAAwDAQACEQMRAD8A\n/VLrSUtJQAUtFJQAvWkpaSgApaKSgBetJS0lABS0UlAC9aSlpKAClopKAF60lLSUAFLRSUAL1pKW\nkoAKWikoAXrSUtJQAUtFJQAvWkpaSgApaKSgBetJS0lABS0UlAC9aSlpKAClopKAF60lLSUAFLRS\nUAL1pKWkoAKWikoAXrSUtJQAUtFJQAvWkpaSgApaKSgBetJS0lABS0UlAC9aSlpKAClopKAF60lG\nQO9GaAClpCR60ZGetAC9aSgfWigApaKSgBetJS0lABS0UlAC9aSlpKAClopKACilpKACjFFFABRS\n0lABRiiigAopaSgAoxRRQAUE4FLSEZoA5fxj8UvB/wAPHt08UeJ9J8PvcgmFdTu44DIB1K7iM4yK\n5r/hpz4Rf9FM8Kfjq8H/AMVXlfxa8G6F44/bY+HGmeItGsdcsD4P1aUW1/As0Ydbi2w21gRkZIz7\nmvVx+zT8KP8AonPhg/8AcKh/+JoAb/w058Iv+imeFP8Awbwf/FUf8NOfCL/opnhT/wAG8H/xVP8A\n+GafhP8A9E48Mf8Agqh/+Jo/4Zp+E/8A0Tjwx/4Kof8A4mgBn/DTvwj/AOimeFP/AAbwf/FUf8NO\nfCL/AKKZ4U/8G8H/AMVT/wDhmr4T/wDROPDH/gqh/wDiaP8Ahmr4T/8AROPDH/gqh/8AiaAGf8NO\nfCL/AKKZ4U/8G8H/AMVR/wANOfCL/opnhT/wbwf/ABVP/wCGafhP/wBE48Mf+CqH/wCJo/4Zp+E/\n/ROPDH/gqh/+JoAZ/wANO/CP/opnhT/wbwf/ABVH/DTnwi/6KZ4U/wDBvB/8VT/+GavhP/0Tjwx/\n4Kof/iaP+GavhP8A9E48Mf8Agqh/+JoAZ/w058Iv+imeFP8Awbwf/FUf8NOfCL/opnhT/wAG8H/x\nVP8A+GafhP8A9E48Mf8Agqh/+Jo/4Zp+E/8A0Tjwx/4Kof8A4mgBn/DTvwj/AOimeFP/AAbwf/FU\nf8NOfCL/AKKZ4U/8G8H/AMVT/wDhmr4T/wDROPDH/gqh/wDiaP8Ahmr4T/8AROPDH/gqh/8AiaAG\nf8NOfCL/AKKZ4U/8G8H/AMVR/wANOfCL/opnhT/wbwf/ABVP/wCGafhP/wBE48Mf+CqH/wCJo/4Z\np+E//ROPDH/gqh/+JoAZ/wANO/CP/opnhT/wbwf/ABVH/DTnwi/6KZ4U/wDBvB/8VT/+GavhP/0T\njwx/4Kof/iaP+GavhP8A9E48Mf8Agqh/+JoAZ/w058Iv+imeFP8Awbwf/FV3ug+INM8UaTbapo9/\nbapptyu6C7tJVkikGSMqw4PIP5VwNz+zZ8KEt5WHw48MZCk/8gqH0/3a4H/gnxEsf7JHgVUUIi/b\ngqgYAAvZ8CgD6LoxRRQAUUtJQAUYoooAKKWkoAKMUUUAFFLSUAFGKKKACilpKACjFFFABRS0lABR\niiigAopaSgAoxRRQAUUtJQAZxWJ4s8ZaN4H0abVdcv4tPsouskmSWPoqjlj7AE1P4k1+z8L6Hf6t\nqEqwWVnE00sjdgBn86/Ob4vfFvVfi14mkv7x3isIiVs7LPywp64/vHua+Wz7PaeTUlZc1SWy/V+X\n5nxfE3ElLIKCsuarL4V+r8vzPYviD+2zqt/NLb+ENOTTbUEhb2+USTP7hB8q/ju/CvINS+PnxB1S\ndpZvFeooSc7YZfLUfQDiuAo7Zr8TxWeZjjJc1Ss/ROy+5H87Y3iTNsfNzrV5LyT5V9ysei6R+0L8\nRNFnEsPim+mx/BcsJV/Jsivbvhx+2zunis/GmnKkZ+U6lp6n5fd4+/1X/vmvkzFKOD6VeDz7McFL\nmp1W12buvx/Q0wHE+bZdNSpV212k+ZP7/wBLM/VnRdasPEGnQahpt3Fe2M6ho54GDow9iKvg56V+\neX7P3xxvPhT4iit7ud5PDd04W6tzyISf+WqjsR3x1FfoPZ3Ed3bxzwuJIpFDq69CDyDX7hkmc0s4\noe0irSXxLt/wGf0dw7xBRz/De1guWcdJR7PuvJ9CajFFFfRH1YUUtJQAUYoooAKKWkoAKMUUUALR\nnNFFABRRSE4FAC0ZzTd1Z3iTxNpfg/Qb7WtavYdN0mxiM9zd3DbY4kHVmPYUAadFc7qfxC8N6N4e\n/t291qyttI2wv9rklATEzBYj/wADLKB65rU1nWrLw9o99qupXMdlp1jA9zc3MpwkUSKWd2PYBQSf\npQBeozms/QtesPE+jWOr6VdxX+mX0CXNtdQtuSaNwCrqe4IINX85oAWiiigAozmig0AfO3jH/k+7\n4bf9iXrH/pRbV9EDvXzv4x/5Pu+G3/Yl6x/6UWtfRAoAWjOaKKACiiigAozmiigAooooAKM5oooA\nKKKKACjOaKKACiiigCK7/wCPWb/cb+VfPX/BPr/k0nwP9b7/ANLp6+hbv/j1m/3D/Kvnr/gn2f8A\njErwP/vX3/pdPQB9FUUVyXxF+KGhfCux0u/8RXBstOv9Qh0wXjD93DLLkRmQ/wAKlgF3HgFhQB1t\nIWpqyB1DKQwIyCO9eeftFfEq9+DvwP8AGvjfTrWC9vtC0yW+ht7kkRyMoyFbGDj6UAei5pc5r81d\nJ/b0/aO0P4I2Pxt8SeA/BOq/DOeZEmXSZJ7e9hR5hErHfLIPvEL93qR0r768IfFLQfFfw70LxkL6\nHTdI1ezjvInv5Vi2B1B2sScZGcde1AHYUZzVTT9UtNXtI7uxuoL22kGUnt5BIjD2YEg1TtfF2iX2\npyadbaxYXGoR/ftIrpGlT6oDkflQBq7qdXz14I/aE8SWj/FnU/iLp2g6L4b8KT7tNn0vUknmuYAH\nP71d52uSoCj5ckkY4q78Ff2sdC/aI+DWu+NvBlulveWUd4LfStZuooZGaFSUeXax8uNmx8x6DJ7U\nAe7g5pc5rx34C/GDVvFfwv0PVviQ/h3wz4pv5ZIjY2GpRyQuQ+FCEucsQR8oJPPvXp974m0nTdRt\ndPvNTsrW/uv9Raz3CJLNzj5FJy3PoKANOisy68S6TZajFp9xqllb383+rtZbhFlf/dUnJrR3UAOo\nzmsa28ZaDeqrW+t6dOHmNspiu423Sj/lmMH73+z1qa88S6Tp1/DY3Wp2Vtez/wCqt5rhEkf/AHVJ\nyfwoA06QtzWY/ifSI9Zj0h9Vsk1aQbksGuEE7DBORHncRgE9Ogqnq/i2wthqdlZahp1xrtrayXC6\nc94iSZVcjeM5Vc4yxGBmgDfDc0uc15L+zN8RPGPxM+FsOt+PNM0fRtf+1TQyW2iXi3NuI1xtO4M2\nGweRn0PevR9O8TaTq8lwljqllevbnEy29wkhiPfcAePxoA06KxrXxjoV7p13qFvrWnT2FoStxdRX\ncbRQkDJDsDheo60668XaHY2Nre3Os6fb2V0QLe4luo1jmJ6BGJw34UAa9Gc0gOTS0AFFFFABRnNF\nFAHy3+3D43ksdD0bwrby7Pt8hvLoL1MaHCKfYvlvrGK+Oa9+/bVuJJfi5BG5OyLTolX2BZif1NeA\n1/O3E+IliM1q832dF6L/AIN2fylxlip4rO6/M9ItRXol/ndnNfErxxD8OvBeoa7LELhoAqwwk4Ek\njHABPp/hVX4T/EOP4oeCbbXFt1tZzI9vc26nISRcHg+hBBrM+Png+88cfC/UrDT42mvYnS5ihXrJ\ntPKj3wao/s2+C77wP8MIbfVIWtr69upLxreTho1IVVBHYkLnHvWSoYJ5M67f77ntvra3bt1v3MI4\nbL/7AliG/wDaPaW315bdu3W9t9Llz43fF5PhDoWn3MdmmoX99OY44ZGKqEUAuxx9QB9a7Tw5rsHi\nfw9pur2uRBewLOqnkrkcg/Q5FeO/tW/DzVvGug6De6NayX89hNJDLbwjLbJMYbHfBXB+teo/Dzw9\nL4T8CaFpE5/0i0tVWUA5w55I/DOKMTQwccooVqb/AHzlJS11t6fdb1DF4bL4ZFhq9J/v5Skpa62V\n7XXS2lvU6IV96fse+OJPFXwqXT7mTzLvRZzZ5JyTDgNGT9ASv/AK+Cq+q/2EbhxqHjCAH90YrZyP\ncGQf1r0eEMRKjmkILaaaf3X/ADR6vAeKnQzqFOO01JP7rr8UfXwOaWmr0p1fvp/T4UZzRRQAUUUU\nAFGc0UUAFFFFABRmijrQAnWuR+MD6wnwk8bN4e+0nxANEvTpwsgTP9p8h/K8sDkvv24x3xXXE4Fc\nj4l+LHhTwj4u0Pwtq2sxWviDW939n6eI3kkmAIBYhVO1ckDc2Bk4zmgD4j8EaJ+0XpsNtquoeI/H\n19c6frPh3Zp15bZiure6tom1LzF8sFlhkLr1wmDnnJrkvFsX7Q/jPwFL4L1nw/47vXttH8UWery3\nNqXtdTmkctpwjdSTKBHhVJAGeBmvte8/a5+EFjYa3ezeOtP+y6LeJYX0saSuIpndkUDah3KXVl3r\nlcqRnIrmvEf7dHwo0Sw8G6hZa1JremeJtafRIruytpStrNGoMnnKUDoV3J8pXcQ4IGOaAPk3xD8K\nfi/4M1z4gz6Hpnjaa71bRfDEmm/ZImmsgIZ7ZbyFhg7Zo1LbVxwglPamaF4o+NXxWvfiFZaPdeMv\nEekCTxbpWqQ3sayaUbcQTJZw2pHzNceaUXb6dOK+0z+2L8G00nW9Tbx3YrYaL5RvZmhmARZZRDG6\njZmRDIQm9Nyg8EisjwX+0F8AfBXgW81zw14l0vT9BvNYljmFrDOZp9QkO+QCDYZWds7uF6c9KAPl\nz4f2v7TXhbxl8KdBg0vV9I8KafpGg2yp5UzWaQrbIt7HdIkbBZd+8AyFdu1cda/SEdK8hvf2tfhD\npuqXmnXnjnT7O6s7c3M4uEljSNQgkKl2QL5gVgTFneM8rXc+AfiJoPxP8Ox654bvHvtMd2jWWW1l\nt2LDqNkqK3cc4wc8UAdLmlpMZpelABQTRSGgD538Y/8AJ93w2/7EvWP/AEota+iRXzt4x/5Pu+G3\n/Yl6x/6UWtfRAoAWjNFHWgBOtLRR0oAKM0UdaAE60tFHSgAozRR1oATrS0UdKACjNFHWgBOtLRR0\noAiuubWb/cP8q/O39mPw1+1BqfwJ8Jz/AAx8XeA9J8IMlyLe1122ma6V/tc3mbisLggtkjnp2r9E\nbv8A49Zv9xv5V89/8E+x/wAYleBz6tff+l09AHBf8IV+3B/0UD4Wf+Adx/8AI1eOfte+GP2pdO/Z\n08aTfErxr8NdQ8IC1Vbq1sbOf7TKxkURrFmBRv37cHIxX6UmuZ8cfDvQ/iNb6XbeILMahZ6ffxal\nFayH9280efLLr/EATnB4yB6UAfIn/BNjRP2idJ8A2Z+JlzbR+CTAP7KstXV31dUx8hBH3I8dBJls\nYwAOa9s/boOf2QPi1/2L9z/KvdAMcVznxH+HukfFXwJrnhDX45ZdF1m1ezu0gkMbmNuuGHQ+9AH5\nffs4/sxfG79p39k7wj4TvvH3h3w78GrmQTGxs7WSXU51SfftkyiqMOuRhyM4JBwK6HUfh/oPjD9v\n3SPgt4+mmPw08L+F0j0TRry7MUF3IiR4ZgCA5bMjEH+7X6JfCH4S+H/gh4A0zwb4Xinh0TTVZbdL\nmYyyAFixyx68muR+On7J3w1/aJks7jxjojy6nZrtt9UsZ2truJc5wJF5x/iaAPiT9nGy1Lw/+09+\n0Z8H/ht4jk0/wDFpUn9nTvctLb6XctCgBRiTtCu7rweAvtWP+x7YWP7NP7QnhH4ffFn4crb/ABG1\nOa6Oj+PrDVDdLqYkLqPNUMQVwdoJ5GFyoOTX3v8AD/8AZS+Gfwx+H2t+DfD3h5bPSNbiaLUnMjNc\nXYYYO+U/MeCfzNYPwp/Yh+FXwe8ZReKtF0u+vddt1Mdrd6vfyXjWiH+GLefkHJ4HTJoA+B/2e/A2\nj/ER/wBuHSdbshf2aGC7SHcR++hkvpIm49HVT+Fbf7DHgDwfaf8ABPX4w+L7W3tx40u/DviCyvZ1\nmJlNutrIY1ZM8Drg4r77+Fn7LHgH4P8AiDx1rHh+yuvtXjR1fWFvLkzRy7TIQFU/dH758geormPA\nf7CHwi+Gg8bp4c0a706Hxhpk2k6lCl45T7PKrK6xg/c4c4I6UAfnNpnwtuvFH/BL/wAL+OdHjH9v\n+BvEc+rQyAHIi81Q/Trg7Tj2r2n4N+Jv+GtP2zNW+MFnbnUNB+Hvguz/ALOQHcJL6W2MwTbjhhJL\nPx1zGM+lfcHw7/Zp8DfDD4OXnww0ayuW8I3aTxzW13cGV2Eww/znn6elN/Z7/Zl8C/sxeGtT0PwL\nY3FrZajd/bblry4M8jvsVANx/hAXge59aAPyN8G2XjT9onwH481eP4Xal4v+Id9q0rw+NT4ritZt\nFkVwVhS2YAhQBgjcN2e2BX7A/s6zeLLj4HeC28c2z2fi1dNjj1CKSVZG8xfl3F1JDFgA2Qe9eYeN\nf+Cd3wT8c+MbjxLcaBe6XqF1J5t0mkX8lrDctnJMiKcHPP5mvofQtDs/DWiWGkafEYbGxt47W3jy\nTsjRQqjJ64AFAH5R/wDBOP4Jad8UfG3xH8W6zf6pcN4G8RyXekaPbzlLc3MhlYyMo+8SIwv415F4\nOTxt+0hoHxR1l/hnf+O/iDe6zKtr4wHieOyfQmVVMccdq4ztHOSGUMML/Ca/XH4Ffsy+CP2dG8TN\n4Ntr23PiK7F9f/a7ppt0o3YK5+6PnbiuE+IH/BPf4L/EbxrdeKr7QrzTdVvJDLeHSL+S0jumJBJk\nRDg5x+p9aAPjr44aT4y+CM37Lv7QHjS3e18RaPJDoPiuVmEjlCG2sxTKktCbhSwPZetdv+yj4bb4\nlS/tOfHi+RZP7dXU9L0mbdvBtoonBZT6HanT0r7W+Iv7PHgr4pfB/wD4Vnr2nyzeFBHBEkMUzJLG\nImDRlX6gjaOfTPrVr4f/AAM8J/DL4Sx/DfQbSa38MLazWnlvKWlZJQwcl+pY7jz9KAPyZ+G/xI8S\nfDb/AIJPa/P4annsrjUfGU9hcXluSHiheKDdgjpnGM+9d98Fvhx8RLL4z/CPxN8PvhBd+ANAjRbb\nxDc/8JTBqceuW0gXM0i/Kd23eScMTkdNtfoJ8Of2Sfhr8Mfg/qfwx03RpL7wdqM8tzc2OqTG4LvI\nqhvmPI+4uPTFc38Kf2DPhN8GfGtt4o8NafqkOoWjtJaxXOpSy29sWyD5cZOAMMRQB+eniCC/+Flz\n+0B+zlp8j2l54z8Z6Ta6QYwZGMN5I7ysR6eVCi/8DxV34Xmb42+Mv2ePgddiaaLwD4g1SfVI3cH/\nAEezuP3fmD/aKBcY6PxX6S+IP2Ufh54m+POkfGC+02dvGuloqQTpOVhO1GRWePozANwT6D0pfAf7\nKfw8+G/xj8SfE7RNNuIvFmvqy3c8twXjAYgtsQ8LkqM4oA9eHWnZpMUvWgBOtLRR0oAKM0UdaAPj\nD9uXw5JbeLtA1wL+4u7RrZmHZ42zz9Vcfka+YLqSS3tJ5Yrd7qWONnS3jIDykAkIpPGTjAz61+lH\nx3+Gi/FT4eX+kpgahERc2Tn+GZQcD6MCVP8AvV+cd9Y3Gl3k1rdQvbXVu5jlifhkYHBBr8I4uwEs\nLmDxFrwqa/Pqv1+Z/NHHWWTwWaPF8t4VbP5rRr9fmYnhbxJZ+LvD9lrGnlhBcrkxv9+Fxw0bjsyn\nINahYk5JyfWuM1Twlqeh61d674Rlt457xvMvtGuzttrt+8ikf6uT36HvUQ+K0Gnnyte8NeItDuQM\nsPsDXUR/3ZI8givmp4L2758H70X0+0vJrd27rR+Wx8hPL3iH7TAe9F/Zv78fJrd2/mSafk9F3Ckg\n5BwfWsXxZ4pt/CVhbTSwS3t1d3CWtpY2+PNuZGOMLngADkk1gD4mz6sTF4b8Ka3q0xwBPe2xsbVM\n92eTnHsBV3w54PvU1j/hIvEl3HqOv+WY4Y7cEW1gh6pCDySe7nk0o4RYf95i9Evs395+Vlql3btp\ntqKGBWF/eY/RL7F/ek+istYru3bTa7OtdAjsuQwB4I7+9fZH7C/huS18MeJNckG1L26jtYsjGREp\nLEeozJj/AICa+S/C3hnUPGWv2Wj6XCZ727kEaKBwPVj7Ack1+l/w48F2vw+8F6X4ftOYrKIIz45k\nc8u5+rEn8a+w4Ly+dbFvGSXuwTS829Pyufe+HuVzr494+S9ymmk+8mrfgr/gdIB+NLRR0r9tP6KC\njNFHWgBOtLRR0oAKM0UdaAE60tFHSgBKKDS0AI3NeRfFL9ny3+KHxO8C+MJteutIl8K3IuYobGFF\nluMNkxtP98Rt0ZOQwr100YoA+PfDv/BNjwf4UstRtNN8RahHDcarZ6jbvLF5ktvFb3JuFt9xfDKX\nP3sA/rXR+I/2F9H1231povFmo6ff6h42n8ax3kFuha3mmhEMkAUnDKUyNx5yc19PNXw9qP8AwUof\nSPEM/hu78BMniO3lurCe2S/LRR6gl1NDBbF/K6yxwtJnHA7HrQBe0n/gmH4T0fQNb0uPxZqUyX0V\nrbW9zcQb5bSCG9iuxECXwyloUHQY5I6mu61z9iWwv/HOo+NNM8Zalo3imTxE/iKxvobaOQWcj2wt\n5ItjHDqyDqcEZrxHwx+1r8Q2n+FOnaTY40vxPrXiKx1OTX9YFzdCS13ny45/KTYqfeQbcnCpkda7\nP9g/9pnxT4y03wd4E8Z6fc3mo3/h6fW7PxJc6n9rmvoo7kxP5qkbkIJAGScgdqANDU/+CbnhnUvG\nmteI5fFF3dTapPNfyW2oWa3MKXky4mmVGcJ8x3HaV4zgHgV7p+zt8DLX9nn4enwpZa1e63bm9mvV\nkvOFg8zH7qFMny4l28KCeSx716iBxRj2oAXvSUtFACUdqDR2oA+d/GP/ACfd8Nv+xL1j/wBKLWvo\ngV87+Mf+T7vht/2Jesf+lFtX0RQAUUHrS0AHekpaKAEooNLQAd6SlooASig0tAB3pKWigBKKDS0A\nHekoPAooAiu/+PWb/cb+VfPf/BPr/k0nwP8A719/6XT19CXf/HrN/uN/Kvnv/gn1/wAmk+B/96+/\n9Lp6APorvSYFLRQAmKKDS0AHekpaKAEooNLQAd6SlooASig0tAB3pKWigBKKDS0AHekpaKAEooNL\nQAd6SlooASig0tAB3pKWigBKKDS0AJjnpXgvx+/ZotPia8mtaK0en+IgvzBhiK6wOA3o3bd+de90\nhA9BXDjcFQx9F0MRG8X/AFdHm5hl2GzTDvDYqPNF/h5p9Gflj4r8F654H1FrLXdNuNNuASB5q4V/\ndW6MPoax0mki+47J/unFfqxq+habr9m9rqVhbahbOMNFcxLIp/Ag15vqf7Lvwx1WdppPC0ULseRa\n3E0C/wDfKOAPyr8sxfA1ZSvhaqa/vXX4pO/3H4tjvDavzt4KvFx7Sun96Tv9yPzueV5PvuzH3NdR\n4F+GXiP4j3yW2habLdKWw1yw2wx/7znj8Otfd+j/ALNPw10OcSweFbaWQd7ySS5H5SMw/SvR7HT7\nXTbdILS2itYEGFjhQKoHsBWmD4GnzJ4yqrdo/wCbS/I1y/w3qKaePrLl7Qvr82lb7meX/Av4BaV8\nIbBp3ZdQ1+dNs96VwEH9yMdh79TXrIAFGB7UtfqeFwtHB0o0KEeWK6H7Tg8HQy+hHD4aPLCOy/rq\nHekpaK6jtEooNLQAd6SlooASig0tAB3pKWigBKKKoeIYL660HUYdMmW31KS2kS2mcZVJSpCMfocG\ngC/RX55x/BL46r4G8PQXWg+JLvUrXUJn8Uwx+LI0k152hkWCe2k8zEEUcrBmjO0kEcHFaWl/s5/H\nW7g8VXXiXV9Zv9ct/h6tjpE1jrQit7jWGgnjYFN4+ZQ8Y3sANwznjNAH3yTxXE3XwV8B3mrT6pP4\nR0iXUZ9Qj1WS6e1UyPdopVJyccuAxAPua+G/G37NP7Q+lW/w6g8Latrd1bJo1pLqMs2srLc6frDF\nGuZHDzIsiZG0KpcABsDnNcT8TfCPxw+Fnw+8aXniFteisdS1rS0s9QutbUajPLJfqHhXyZWRYyjY\nDEIeQMdaAP0Vm+BXw+n06wsJPB2kPZWF8+p2kJtV2wXTks8y+jMSST3q74a+Efgzwde6deaH4Y03\nSrrTrN9Ps5rW3CNBbs+9olI6KW5x618VSfBD4+J4Q05ZLPXLvwyviTUbxvBqeJ0XVI7OS2VLMNel\ntrCOcSSMu7kOOuMVh6x+zX+0fcaHr+pXuq65qXjLTdF0U6HNYa6sNvNfRz5ugyGRVYiLAJcBWI4z\nQB92/EL4ueFvhXJo0fiXUWsH1id7axVbeSUyyJG0jL8inHyIx5xnGOtdB4b8RWPi3w/p2taZK0+n\nahAlzbyPG0ZaNwCpKsAVOD0IBr88vih+zV8cvFPxcvdWm0bUdZktfEmp6jY67Hr8a2wsJbSSO0hi\ntWdTHJGzKC3HPPvTNA/Zu/aDt9L1DVriTxHF4m06DwoNIQeIUMTyRQxpqhdPNKtjDZ3feOSN2c0A\nff8A8Q/iFoHwq8H6j4p8U6gmlaFp6q1zdurMIwzBBwoJOWYDgd68M/4eQ/s7f9FFtv8AwDuf/jdf\nL3wvPi7xh+2hoHhjXpNS8Tmz1rXJ/FMpvnutJmtPIYWavbEbIgjEKAw+ZsEZwDX6Ff8ACn/Af/Qk\n+Hf/AAUwf/EUAeKn/gpD+ztj/kott/4BXP8A8br548D/APBTfwX8O/j94p8K6x4hfxN8MtXvf7S0\nTxLDHIZNMMwDSWs0bKGMSyb9pAJUEdQcL94n4P8AgPH/ACJPh3/wUwf/ABFfNng/9hTw34r/AGgv\nFnxU8e+HdO8kXgtPDfhyCFFtba0gARLiVFAUvIwZ9pzgMM88AA6e+8Qab4r/AG0vhXrGj3sWo6Ze\n+B9Wmt7qBt0ciG4tSCDX01ivnHxNaQ2P7cvwzt7eJIIY/BWrqkUShVUfaLXgAcCvo6gA6kUUd6Wg\nBK/Ob/grp8bvH3wdi+Hp8E+LtW8Lm9a5Fz/Zly0Pm7duN2OuMmv0Zr8r/wDgt/8A6n4X/wC9d/yS\ngD9UK/Mz4x/tI/Gn9pH9rfX/AIHfCHxXZfDzTNCea1udSncxT3EkJAmcNtL5DZVVQcgZJxyP0zr4\nc/az/wCCbcHxo+IFz8Sfh74pl8D+PJMPO6bhDcSqAA+5CGjcgAEjIPUg0Adz+zd8Gv2h/ANj450n\n4l/E618X291p6Q+H7+KWR5YJyJN8kheMOuP3eMFu/pXwf+1/p/7UH7H2j+G9R1z9oHV9ej1y4mt4\nk0++uEMZjVWJbfjruHT0r6D/AOCef7TvxQuPjn4r+AnxVuhrmraFbzvDqhcPJG8Dojxs4++pD5Dd\nRj3rA/4Lg/8AIjfCr/sI33/oqKgD2b9lr4B/H/4d+MLLxl8Q/jVL458LvpUkh0SW4uGzI8YZGO8b\nfl9a+F/2a/2xfjHpX7QXhLVvF3jjX9Z8A3/iKXRJ4b+9d7YMcfKQeMqJEI+ntX6/3eoJpPwRmvnZ\nES28OmZmlbCgLbZJJ7DivxS8JeArjxH+wB4y8a6eN974Y+Iiaihj5wn2eJGP0/eK3/AaAPtv/gqj\n+0V408A3PgvwP8N/EV74f8QTW174g1G4024MMwtIIXIUkfwkJMxHrGteq/8ABMD4m+Jfiv8AsvW2\ns+LdevvEetrq13DJe6hKZZSgK7V3HsBXyL8P9Ri/a3+LXxq+J15jUNB8MfDd9HtiyHZ9pmszG+0n\n/ZW5z/10FfQH/BHC9Wf9ly+tw4Ywa3cZX+7kKaAPvCiiloATtR1oooAiu/8Aj1m/3G/lXz1/wT6Y\nN+yR4HwQfmvxx/1/XFfQt1/x6zf7h/lXzr/wTytY7X9kfwSkQ2q0uoyHvy1/cMf1JoA+j6KKKACi\niloASiiigAoopaAEooooAKKKWgBKKKKACiiloASiiigAoopaAEooooAKKKWgBKKKKACiiloASiii\ngAoopaAExRRRQAUUUtACUUUUAFFFLQAlFFFABRRS0AJRRRQAtJR+NHtQAUtJR+NAHO/EjXbrwv8A\nD3xPrNlsN5p2l3V5D5q7l3xxM65HcZUcV89/Dz9qrwZ8RPCvws8P/EO0trjxd4w0mz1SG3n09ZLG\nS5eMyIEDFiCGUkHHBA5zX0n4n8P2/ivw5quiXjyJaalaS2czQsA4SRCjFSQQDhjjINfPfh3/AIJ/\nfDXwt4z8MeJrC98Rrf8Ah9LBLeOS/R4pDZwtDCzqY+uxiCFKgnnAOTQBhfs6ftzaL430P7H48uG0\nXxHv1e4S4Ni8FjPbWUz7/KkJId0iUMwHoat/8Noab4+8d/CKz+H9ylzoviTXbzSdXTULUpcRCO0M\n8ZUZ+XdwQecg9q2/Cn7BXwy8MX08s0mveILRrXUbSDTdY1Iy2tmt9n7Y0KKq7GkDFScng8YPNWvh\n7+w78O/hnc+EbjR5dWEvhjUptUs2lkgBklkg8giYpCpdVTgZ+b1JoA+hBRRjFHtQA3B60+kzR+NA\nC0lH40UAfO/jH/k+74bf9iXrH/pRbV9ECvnfxj/yfd8Nv+xL1j/0otq+iKAFpKPxo9qACvkL9vv9\niPXP2wk8JLo/iXT/AA//AGKZjJ9ugeTzN+3GNvTGK+vaPxoA+AfB37FH7Tmh+L9D1LVP2kJdS0yz\nvoLi6sibj/SIkkVnj54+ZQRz61p/Fr9g74w+Ivij4n8V+B/j9qvhm18QXr3M+ms0yRQIx4SNUfbl\nQAA2AePevuv8aPagD5e/ZB/YX0P9lzU9a8S3Ou3njDxtrMZhu9avh8wjLB2Vckn5mCliTk7RVH9v\nv9jTWv2w9A8H6do3iKx8PPod1cXEj30LyCQSIigDb0xs/Wvq+j8aAOM8V+BrvXvhBq3g+3uYYby9\n0OXSVupFJjVngMW8gc45zXzd+zr+wXN8J/2VfHvwg8S65Za2/ii9ubk39nC6JEklvDEgIbnKtEW/\n4FX2J+NHtQB8vfsw/sVQfs4/Abxd4Cj1mLWNQ8RNcPPqLwlUy8QjQFfRQP1NWf2EP2VNe/ZJ+Heu\neGNb8QWHiD7ZqH2y3lsIHjEa7ApVtx5ORmvpij8aAFpKPxo9qAClpKPxoAiu/wDj1m/3G/lXz3/w\nT7/5NJ8D/W+/9Lp6+hLv/j1m/wBxv5V89/8ABPv/AJNJ8D/W+/8AS6egD6JJoBoPHWkzigB1JQDn\nvQTigApabntS/jQAtJR+NHtQAUtJR+NAC0lB4oBzQAUtJR+NAC0lH40e1ABS0lH40ALSUfjSZ4oA\nWlpKPxoAWko/Gj2oAKWko/GgBaSj8aPagApaSs3xD4k0vwnpFzqusX8Gm6dbIXlubhwqIB70m0ld\nlwhKpJQgrt7JdTS3D1ozXwt8Xf8AgpLb2FzPYfD7RkvymVGqapuERPqsQIJH1I/GvnfWf23vjPrF\ny8o8YPYRk5EFlZW6Iv0JQt+ZNeBWzzCUXypuXp/SP13K/CviLMaSq1Ixop/ztp/dFSa+dj9cc+9G\nR61+Seg/ty/GbQ7pJX8VjU4lPNvf2UDo31Korfkwr6T+Dv8AwUe0rXbu307x7pS6HPIQn9qWJL22\nT3ZD8yD8WHvToZ3hK8uVtxfmRmvhbxFllN1oQjWit+Rtv7mk38rn2yDnvRVPStWs9Z0+C+sLqK8s\n51DxTwMHR1PcEVcBz0r3k76o/I2nFuMlZoKWko/GmIWko/Gj2oAKWko/GgBaSj8aPagApaSj8aAC\nilpKACignFc9efETwtYeJ7fw3deJNItvEVwnmQ6PNfxJdyr/AHlhLbyPcCgDoaKTePSsu+8U6Zpu\nvaZotxc+XqepJK9pB5bHzBGAZPmA2jAI6kZ7ZoA1aKQvj61kX/jDSdM8SaXoFzdeXq2pxTTWlv5b\nnzUi2+YdwG1cb16kZzxmgDYopN+O2KN2KAA5pa5TRfij4e1/xvq3hC1nvE8Q6XCtzc2l5pt1bAws\n7IsscksapMhZGAaNmGVPNdZ+FACUUtJ2oA+d/GP/ACfd8Nv+xL1j/wBKLWvoivnfxj/yfd8Nv+xL\n1j/0ota+iBQAUUtJQAUUUv4UAJRS0lABRRS/hQAlFLSUAFFFL+FACUUtJQAUUUv4UAQ3f/HrN/uN\n/Kvnv/gn3/yaV4HHvff+l09fQt3/AMes3+4f5V+dv7MfwO+Nfjj4EeFNW8FfG9/BPh+VLlYdGGkr\nceUy3cwc794zuIJ6cZoA/RauC+MXxf0j4JeHbDxB4hSWPQZdQgsLy/TGyxWUlVml9Iw+1WPbcD0B\nr52/4Zf/AGlx/wA3Nyf+CFf/AI5Xj37XPwQ+N3gn9nzxdf8AjD4+yeKtGlgS1OhLoSh9QlkkVY4l\nw5O4sQeASMZHSgD9H7O+gv7WG5tpo7i3mUPHLEwZXU9CCOCK8m/aw/aDH7MPwV1X4gHQT4kFjPbw\nf2eLz7Lv82VY8+Z5b4xuz905x2rwX/gnF8Avjd8H/BNs3xD8UNb6DNDus/B9wgmmswRkFpD/AKs/\n9MxnHQ4PA2P+CsIx+xR4q/6/9O/9Ko6AOp+GP7bujfFT9lTxT8Y9O0ExXfhywvLq/wDDT3wLxywR\ntJ5Rm8vgOACG2dGzjtXp/wCzj8Zx+0B8GfDfj4aR/YQ1mFpv7PNz9o8rDsuPM2Ju6f3RX5f/ABL0\ne8/ZQv8AVEWVrP4afGrwFNZPKwJgtNUNmQpIHQ72Hp8svX5a67WP2g/EfwQ/4JufBTTfCt3d6Xqv\nimd7BtRsYfNnghWVjJ5QyMyHcAB35HFAH6ubs0Zr8t/gD8YfE/w9/aN0m18Kaf8AFzVPhtqWnzjV\nIfHulygw3axvIJUkJYKp2+oGTgCub+GQ+IP7S/wW+MXx81P4reINA8QaDe3lnpulafceXZQxRwRy\nFWi9GEu0HqNpOc0AfrZuoDDPtX5g/CX416T4Z/4Jm+FdS8f+MvFdreapr1zaW9xolyZNUvZFuZWE\nCyOThSqkFieBjHJAOV+zn4/8f/C/9tLwr8PLyw8Z+HPCPjKwnxo3jHWFv7lSIJJBcKw5jOY8bDyM\n0AfoZ4N/aF8CfEH4m+JvAGgawdQ8UeG036parbyKtv8APsxvKhWOewJqn8E/jTqXxb1Xxraah4F1\nnwanh7Uzp8E+qgbdRUbv3sfA4G3kc/eXk54/Ob9ib9n6xP7evxW0r/hJtfC+C7kzwT/bW8y/2zBc\nXJ/5aAg5571p/szaZ4s+N/gf9sDw8vibW31O1v7eTSZRqExktpEe9dVjO7Kg7FGBjOBQB+rOaN3t\nX5DxfGzxf+0P8GP2afhHpWuarY+K9R16e3129gvXju0hsyULSMDuIKPI5B/55jrXW/tGfHu48U/t\nU698NNYufiPF4C8IWEVrHp/w9tXmvbi52qRNM4dW2DI5zyfxoA/UvdmgmvkL/gm18SfiB43+GPiT\nSPiBa+IPtWhan5On6j4jsHtbm7tHBMe4MPmZdpzjONwGTXzl8UdG+IXxg/4KY/Er4Z+HviHqvg/R\nb7S7L7VJbXDnyYFsLSVhAmdqMzDkjH3m9aAP1KzRuFflf+0Z8a9TuP2nE+D2t6p8Qp/AnhDQY0kg\n8AQNNqWo3eyLE0xVlYIC/Jz1Cj+I1zV7qvxi+Kv7E3xG/tO48Z6VqfgDVxqGj6jqSTaff3umch1l\n5XcUQ7zgnGwjmgD9dc15Z8NvjTqXj34o+PfCN14G1nw9Z+GZ0it9bvwPs+pgk/NFwMDjI5OQcnHS\nvibRfizrf7Yn7S3wAsdH1nULXQ9L8MQ+I/ESaddPHDLck4eN9rDdiWIrg+pFW/2X/inrmhftT/te\nX19qeoarY+H5tQubOwubp5IovLu5dqorEhRgAcY4oA/SHNAbPavxI0/9oL4g/EHwFdfEbSr/AOLW\no/FyTVTNY/2Rpsk3hxLUTANb/ISDhMnOwncNvQk19K678WvEHwm/a8+GvxG8VXmqab4c8ZeCJrjU\ndLuZpEtbS7htWmlHlMcK2YfTI3UAfpFmlHNfi94N/aF8f+AvgF8YtK8R+INUn8T+MdM0/X9FdbiV\n3gF5O6iOAk7gSOy8dh0r9Xf2d/Bl98Pvgh4J8P6ne3Oo6pZaXAt5d3crSSyzld0jFmJJ+Yt1NAHo\nlFFL+FACUUtJQBU1TVbXRdOur69mW2s7aNppppDhURRkk/gK/Jr9qf8Aab1P48eLJoLSaW18JWch\nSxs8lfNAP+tcd2PUZ6DFfXn/AAUX+J03hH4T2PhixlMV34kuTHMy9fssQBkH/AmaMe4LV+ZmPSvh\n8+x0ub6rB6df8j+qPCThaj9XefYqN5ttU79EtHL1bul2s+4ZpTxWd4g1mHw9o11qM43RwLnYOCx7\nCqvg/wAUQ+MNDj1CGMwtvMckROdrD39wQa+TVCp7J1kvdva/mf0HLM8HDHRy11F7aUedR68qdr9v\nlv12NrtS5wa53xv4xg8E6XFdSwtcSSyCOOIHGe5OfYVsaZqMOr6dbXtvnyZ0DrntntQ6NSNNVmvd\neiYU8zwlXG1MuhUTrQSlKPVJ7Pt/w67o+nf2Qf2p774NeJrfQdbunn8F6hKElSVs/YnPAlT0H94d\nMc9q/U61mjuYElicSROoZWU5BBGQRX4PDrX6r/sF/E6b4hfAq1s72YzaloE502R3bLNEAGhY/wDA\nW2++w19fkOOlJvCzfmv8j+cfFvhajRhHPsLGzbUaiXW+0vXo+90fR9FAOc0v4V9qfzEJRS0lABRR\nS/hQAlFLSUAFFFL+FACUtFJ+FACN0NfHXjG+sray+JnhjUZYm8ean40sLzR7DcPtk0YubUwTxL94\nxxhXJYcKFbNfYzdKwdT1/TtM8S6Npk9tM+oamsxt5o7YuiCIKW3yAYTIYYyeecdKAPIdQ8Jx+O/2\nmtfs9W1TWv7L0/w/YzwabZatcWkHmtNLmQrC6knCgdcHuDWd47+IHgvwh+1D4Lil8cWWmXk1ne22\nqafc+IisSsIozbiS2eXYjndlTtDNnvX0SI1Dl9o3HgtjnH1qCTT7WWUySWsLyZ5dkBPHvQB8PfED\nxiV8RzXGp+L9V034gp8SdOsbfRrfVZoVbSGvoVQfZVfYYXjIJk28k7S3O0+rfEH4heEfCX7WngiO\nTxvaaZNdabqEGqafceISkIkAg+zh7dpdiOcttO0McnrXqHib4KaX4w8W2WsatrGsXdjaXkOox6C8\nsJsTdQkNFKcx+b8rBWCCQR7gDszXTDWPD954pn0Qy2k2uwW6XklqUBlWFmKq5yOhKkfhQB8t2KX/\nAPwlXiTxR/b+uDUbP4orpMMTapObVbNreDdD5Bby9pLk8rwemOa5fx18Yb24+L1nf6Tq8ejalb+O\notAuNJuNZunv3tlmEbM9kG8iOCRTlXKZIIIYkivugW0WMiNeTuPyjk+v1phsLYymQ28ZkPJfYM/n\nQB4jqWsad4V/a9N1rOoWmk2+r+DrTT9PkvZ1hW6uUv7hmhjLEBpMSxnaOTu6V7rio5bWGZkaSJJG\nQ5UsoJU+3pUtACUUtJQB87+Mf+T7vht/2Jesf+lFrX0Rivnfxj/yfd8Nv+xL1j/0otq+iKAClopP\nwoAKMUtFACUtFJ+FABRilooASlopPwoAKMUtFACUtFJ+FABRilooAhu/+PWb/cP8q+e/+CfQ/wCM\nSfA/1vv/AEunr6Eu/wDj1m/3G/lXz3/wT7/5NJ8D/W+/9Lp6APok1ka/4R0jxTJpjatYw3/9m3aX\n9os67liuEBCSAdNy7jg9jz1ArYooAaqheleVftPfs/2P7Tnwf1PwBqOr3GiWt9NBM15axLI6GKRZ\nAArEDkrj8a9XpPwoA8N+N/7Jfhr47fAGx+F+tXk0MOnx2wstXjhVp7eWEBRIoPHzLuUj0Y1hn9hn\nwRqv7M2h/BnX7u81fS9G+e01ZQsF1FLuZhIuMgH5iMd6+j6KAPnT4G/sm6z8IPFC6pqvxg8YeO7K\nG3e2g0nWpgbZUYY+cZO8gYwT/Wvzz+N+m/Bbwjq/xP8ADPgjXPiTYa1qtxPb/wDCuLK2kjs726Dk\nLLxy8HXAxyOOlfstjNQNYWzS+abeIy/39g3fnQB8I/CP9gWy+JX7DXw/+H3jqW/8O6/p9xLrVvcw\nKPPsZ5ndtrKeDw4ypxyB6V6L4Q/YAsfDfx08FfFXUfiP4j8UeI/DsUkcjaxslF5uikiHPHlgLIeF\nB5FfWG3FLQB8v3v7B2gRftLn4yaD4v17w5qdzcrdalpVjJttr9gACr4IO04BKnIzzXR/syfsjaV+\nzR4h+Iuq6f4gu9cfxnexXs8V1AkYtijTHau0ncD556/3RXvtJ+FAHy38Ef2APB/wP/aE8QfFbTdX\nu7671FblbXSpoEWGw89wzlGByTgMvbhzWj8dP2INC+LnxCi8f6D4t134c+OBCLeXWPD8u0zxj+F0\nyA3Hv2HpX0pRQB5/8EvhXd/CHwZ/Yd/4v1rxvdvO1zLquuy+ZMzMqgqv91PlyFHAya4LRv2SNL0f\n9rfXvjwniG7k1PVrKOzfRzAohjC28UG4PncTiIHp3r32k/CgD5x+Pv7E+g/Gbx5a+PdG8Ua38O/H\nkEAtW13w/IEkniGPlkXjdwMcnpjrgV33wr+Bx8BfDrU/Cev+LNZ+IEepiVLu91+QSSSJImx0A5AU\ngnjpzXqNFAHzN+yB+wt4a/ZDvfEd5pOu3niK81hY4hPfQJG1vCpLeWpUnIJIP4VrfB/9jrRPhN8Y\nPin48XW7jW28fzTy3mmXVuiwwLLM0jICDlh8xHPavoOk/CgD47sf+Cctp4Q1+6fwJ8W/GvgbwvdX\nDXM3hvS7nFuGYknYcjYOccDtXfftW/sa6J+1X4Q8OaPqniG/0K80N2MOqWsYkmdXj2SK2SMhgOfx\n9a+h6KAPln4pf8E/PBfxN+Ifw58TNqVzpNv4NtLWyTS7WBDFew2zBoEkOQQFI4HPU19SIgRQF4AG\nAPSnUn4UAFGKWigBKWik/CgD83f+CmmoSy/Fjw1ZliYIdHEip2DNNICf/HV/KvjqvuH/AIKeeFZY\nfEXg3xIqsYZ7aXT3YDhWR965PuJG/wC+TXw/X5Zm6axtS/8AWiP748O6lOpwvg/Z9E0/VSd/xOZ+\nJWmTav4Lv4LZS8qgSBF5LAHmsj4MaNc6T4Ske6jeFrq4MiI4wdoAAOPcg13kkiwRtI7iNFGS7nAH\n41gS+O9K3lLc3GoMOCbSBnX/AL66VFKtWnhZYWnC6bvc6sfl2WYXPqWfYvEKFSNNwUW0r3e66uyb\nVkutzlvjhotzqWi6dcW0Ukwt5mV1jGcBgMHH1GPxrrvBWmy6P4T0u0nBWaOEFlPUEnOP1qO38daS\n8oimkmsJGOB9shaME+mTxW+HDqGUhlIyCDkGivWrRw0MLUhZJt37/wBXZWWZbltXPMTnuErqc6kI\nwcU0+W1te+vKtGuj76FfdP8AwS9v3Go+P7Tny3itJfbKmUf+zV8K1+g//BMLwtNb+GfGviF1xBdX\nUFjCfUxozv8Ah+9T8jW+Sxbx0LdL/kzxvE6pThwtilP7Tgl688X+SbPt8dKXFAGKWv08/hQSlopP\nwoAKMUtFACUtFJ+FABRilooAKSiloAa3SvmH4v8AjbwR4N/ai+Hmp3Oq2OmahFa6hDrNwZCpVDFF\n9nWY9MZ37c+9fT/XtRgelAHxd4z8LyJ4r+LPiuKC9Ov2HjrRU026DyEwwNDYpKIl6BWDyBsDB5z0\nrAa70q5+JHxSWynu5fifF8SYY9FWB5DIlr9msTIqj7oiwZfMHTBGe1fdzDGK57wf4C0jwVd+JbnS\n4pI5fEGqPrF95khcPctDFCSufujZBGMDjgnvQB8u/E7xkunXOt+FXubuPxFcfEjR7uCyRZN7WRvb\nVjKCOPK2ggnpk4NbereBvAXhj9smXVfEOkadZNq2k2T6Vd3UZAn1EXTfcboZOY+PpX1UAB2xRgel\nAAvIooAxS0AJilopKAFpDwKKOuc0AfKnx78cWvwp/aw+HvjLWtO1e40CLwvqdg9zpWmTXuyaSe3Z\nVIjUkZCMc10A/bw+Gvey8Yf+Erf/APxqvosADtRigD51/wCG8fhp/wA+XjH/AMJW/wD/AI1R/wAN\n4fDT/ny8Y/8AhKX/AP8AGq+iqWgD50/4bw+Gn/Pl4w/8JS//APjVH/DePw0/58vGP/hK3/8A8ar6\nLpKAPnX/AIbx+Gn/AD5eMf8Awlb/AP8AjVH/AA3h8NP+fLxj/wCEpf8A/wAar6KpaAPnT/hvD4af\n8+XjD/wlL/8A+NUf8N4/DT/ny8Y/+Erf/wDxqvoukoA+df8AhvH4af8APl4x/wDCVv8A/wCNUf8A\nDeHw0/58vGP/AISl/wD/ABqvoqloA+dP+G8Php/z5eMP/CUv/wD41R/w3j8NP+fLxj/4St//APGq\n+i6SgD51/wCG8fhp/wA+XjH/AMJW/wD/AI1R/wAN4fDT/ny8Y/8AhKX/AP8AGq+iqWgD50/4bw+G\nn/Pl4w/8JS//APjVH/DePw0/58vGP/hK3/8A8ar6LpKAPnKb9u74ayQyILHxjllI/wCRUv8A0/65\nVd/YM0y+0b9lXwRa6jZXOnXQF47W93EY5VDXkzLlTgjKsD+NfQFAAHagAxS0UlAC0lFLQAmKWiko\nAWkopaAExS0UlAC0lFLQAmKWikoAWkopaAExS0UlAC0lFLQAmKWikoAWkopaAExS0UlAC0lFLQB5\nD+1F8HV+Nvwi1XQoVUatARe6bI38NwgOB9GUsh9N2e1fj7qFjPpF3c2t/E9pc2zMk0cw2tGV+8GB\n6Ywa/d8qCeRXyJ+1v+xcnxTlufFng2OG38T7SbmwchI7/A7MeFftk8HvjrXy2c5bLEpV6SvJbruv\n8z948NONqWRylleYS5aM3eMntGWzv/deno9erZ+VVlaN43Yajfh10gN/odieBKB/y1kHfPZfSuoh\njS3jEcSrEgGAqAACtTxB4Z1Twhqk+laxptzpN9btsktbqIxOmP8AZPb07Vm18NXqSm+W1ktl2/4P\ndn9XZZgqOHp+2UlUqT1lU6y9HraP8sU7JbEdxbxXcTRTxpNEwwUkXcDXMTQt4GuY54GZ/D0rhJoC\nS32RjwHX/YJ6jtXV1s+E/BGueP8AWIdG0DSLnWr+c4W2tot+R6t2CjuzYA7mqoVJRfJbmT6d/Tz7\nMzzTB0asPrXOqdSnqqj+z5S2vB/aTdrdmk1X0DQL/wATa5ZaRpls93qN5KsEEEfVnJwB/wDXr9lP\ngJ8Lbf4NfCvQvC0LCSW1h33UwGPNuHO6RvpuJA9AAO1eQ/smfse2nwVjXxD4kMGpeMZkwvl/NFYq\neqoT95j3b8B6n6fA4r7vJstlhIurVXvP8EfyZ4lcaUuIa0MvwEr0Kbu3/PLa68kr273b7BjFLRSV\n9KfiAtJRS0AJilopKAFpKKWgBMUtFJQAtFJS8UAFFJRQAuM0UlLxQAUUlFAC0UlLxQAUUlFAC0Ul\nLxQAUUlFAC0UlLxQAUUlFAC0UlLxQAUUlFAC0UlLxQAUUlFAC0UlLxQAUUlFAC0UlLxQAUUlFAC0\nUlLxQAUUlFAC0UlLxQAUUlFAC0UlLxQAUUlFAC0UlLxQAUUlFAC0UlLxQAUUlFAC0UlLxQAUUlFA\nC0UlLxQAUmBRRQByXjz4V+EvidZC28UaBZawijCvPH+8Qf7LjDL+BrwjWf8Agnb8K9SunmtRq2mh\njnyobrcg+gZSf1r6ko4rkq4ShXd6sE36H0GX8QZtlUeTA4qdOPZSdvu2PmDQP+Cefwp0i6Sa6i1T\nVdp/1Vzd4jP1CgH9a978E/Dzw18O9P8AsHhvRLPRrbADLaxBWfHTc3Vj7kmukxRTpYWhQ1pQS+RO\nYZ9mubLlx2JnUXZydvu2DFLSUvFdR4IUUlFAC0UlLxQAUUlFAC0UlLxQAUUlFAC0ZopKAFozRRQA\nUZopKAFozRRQAUZopKAFozRRQAUZopKAFozRRQAUZopKAFozRRQAUZopKAFozRRQAUZopKAFozRR\nQAUZopKAFozRRQAUZopKAFozRRQAUZopKAFozRRQAUZopKAFozRRQAUZopKAFozRRQAUZopKAFoz\nRRQAUZopKAFozRRQAUZopKAFozRRQAUZopKAFozRRQAUZopKAFozRRQAUZopKAFozRRQAUZopKAF\nozRRQAUZopKAFozRRQAUUtJ6UAFFLRQAlFLSelABRS0UAJRS0npQAUUtFACUUtJ6UAFFLRQAlFLS\nelABRS0UAJRS0npQAUUtFACUUtJ6UAFFLRQAlFLSelABRS0UAJRS0npQAUUtFACUUtJ6UAFFLRQA\nlFLSelABRS0UAJRS0npQAUUtFACUUtJ6UAFFLRQAlFLSelABRS0UAJRS0npQAUUtFACUUtJ6UAFF\nLRQAlFLSelABRS0UAJRS0npQAUUtFACUUtJ6UAFFLRQAlFLSelABRS0UAf/Z\n",
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 1
        }
      ]
    },
    {
      "metadata": {
        "id": "C-2E5p2dohkz",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Step 1 - Import libraries and modules"
      ]
    },
    {
      "metadata": {
        "trusted": true,
        "id": "edIyyG81ohk1",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "import numpy \n",
        "import keras\n",
        "# Import the sequential model, which is a linear stack of layers\n",
        "from keras.models import Sequential \n",
        "# Import the core layers that are used in almost any neural netwrok\n",
        "from keras.layers import Dense, Dropout, Flatten\n",
        "# Import the LSTM layer which is commonly used in RNN's\n",
        "from keras.layers import LSTM \n",
        "from keras.layers.embeddings import Embedding \n",
        "from keras.callbacks import TensorBoard\n",
        "from numpy import array"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "n2jAjEVFohk4",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "#Step 2 - Import the IMDb dataset \n",
        "\n",
        "As a first step, we will load the Internet Movie Database (IMDB) dataset, which is already part of the Keras datasets.\n",
        "\n",
        "The dataset consists of 25,000 movies reviews from IMDb, labeled by sentiment (positive/negative). Reviews have been preprocessed, and each review is encoded as a sequence of word indexes (integers). For convenience, words are indexed by overall frequency in the dataset, so that for instance the integer \"3\" encodes the 3rd most frequent word in the data. This allows for quick filtering operations such as: \"only consider the top 10,000 most common words, but eliminate the top 20 most common words\"."
      ]
    },
    {
      "metadata": {
        "trusted": true,
        "id": "Y7ADou88ohk6",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "from keras.datasets import imdb \n",
        "# Define the n most common words we want to load\n",
        "top_words = 5000\n",
        "# Load the train and test datasets\n",
        "(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=top_words)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "Jw_JsUzjpWp8",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Step 3 - Preprocess input data"
      ]
    },
    {
      "metadata": {
        "id": "AnQ-VpT1mbKL",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Print movie review example \n",
        "\n",
        "Let's print an example of a movie review from the dataset.\n",
        "\n",
        "We only loaded the 5000 most common words from the dataset, therefore some words migh appear as < UNKNOWN >."
      ]
    },
    {
      "metadata": {
        "id": "mbmZd_C-msqi",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Load the word indices such that we can convert the integers representations of the words in the dataset to strings\n",
        "word_to_id = imdb.get_word_index()\n",
        "word_to_id = {k:(v+3) for k,v in word_to_id.items()}\n",
        "word_to_id[\"<PAD>\"] = 0\n",
        "word_to_id[\"<START>\"] = 1\n",
        "word_to_id[\"<UNKOWN>\"] = 2\n",
        "\n",
        "id_to_word = {value:key for key,value in word_to_id.items()}\n",
        "\n",
        "# Define the review to print, try changing this number to see the print for different reviews\n",
        "review_number = 0\n",
        "# Print the review text\n",
        "print(' '.join(id_to_word[id] for id in x_train[review_number] ))\n",
        "# Print the sentiment of the review, 0 represents a negative review, 1 represents a positive review\n",
        "print('sentiment = ', y_train[review_number] )"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "ojsaybHlmtYb",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Preprocess data\n",
        "Not all reviews contain the same number of words, we will truncate the reviews in both the training and test datasets to a maximum number of words of 500."
      ]
    },
    {
      "metadata": {
        "trusted": true,
        "id": "vzDlPC3FohlC",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Truncate and pad the review sequences\n",
        "from keras.preprocessing import sequence \n",
        "max_review_length = 500 \n",
        "x_train = sequence.pad_sequences(x_train, maxlen=max_review_length) \n",
        "x_test = sequence.pad_sequences(x_test, maxlen=max_review_length) \n",
        "\n",
        "# Print the shape of the datasets, which should have a size of: \n",
        "# total number of reviews X number of words in the reviews\n",
        "print('x_train shape:', x_train.shape)\n",
        "print('x_test shape:', x_test.shape)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "x6tNcy9t8nFB",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "##Print data sample\n",
        "\n",
        "If you run the cell below, we will print the first reviews.\n",
        "\n",
        "If you look at the data you will realize it has been already pre-processed. All words have been mapped to integers and the integers represent the words sorted by their frequency. It is very common in text analysis to represent a dataset like this. So 4 represents the 4th most used word, 5 the 5th most used word and so on"
      ]
    },
    {
      "metadata": {
        "trusted": true,
        "id": "5dNXowMWohlK",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Print the first reviews.\n",
        "pd.DataFrame(x_train).head()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "X-vecLYPohlO",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Step 4 - Design model architecture\n",
        "\n",
        "The next step is to create the model that will be trained. We will use the Keras Sequential model, which is a linear stack of layers. For more information see https://keras.io/getting-started/sequential-model-guide/.\n",
        "\n",
        "* The first layer is an embedding layer. https://keras.io/layers/embeddings/\n",
        "\n",
        "* The second layer is a Long Short Term Memory (LSTM) layer, which is a type of recurrent layer. \n",
        "\n",
        "* The third layer is a fully connected layer.\n",
        "\n",
        "The last step is to compile the model, for which we need to select both a loss function and an optimizer. Keras offer a wide selection and you can find more information here: \n",
        "\n",
        "* Loss functions : https://keras.io/losses/\n",
        "* Optimizers: https://keras.io/optimizers/\n"
      ]
    },
    {
      "metadata": {
        "trusted": true,
        "id": "kgjTChkwohlP",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "embedding_vector_length = 32  #@param {type:\"slider\", min:16, max:64, step:16}\n",
        "number_recurrents = 4  #@param {type:\"slider\", min:2, max:10, step:1}\n",
        "\n",
        "# Create the model and add layers\n",
        "model = Sequential() \n",
        "model.add(Embedding(top_words, embedding_vector_length, input_length=max_review_length)) \n",
        "model.add(LSTM(number_recurrents))\n",
        "model.add(Dense(1, activation='sigmoid')) \n",
        "# Compile the model with the selected loss function and optimizer\n",
        "model.compile(loss='binary_crossentropy',optimizer='adam', metrics=['accuracy']) \n",
        "print(model.summary())"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "fdWFgGBfohlY",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Step 5 - Train the model\n",
        "\n",
        "Before we are ready to fit our model we need to select two more parameters, batch size and number of epochs.\n",
        "\n",
        "* **Batch size** : defines the number of samples that will be propagated through the network. Advantages of using a batch size smaller that your total samples is memory usage and efficiency in training.\n",
        "\n",
        "* **Epochs** : The number of epochs is the number of times the model will cycle through the data. The more epochs we run, the more the model will improve, up to a certain point. After that point, the model will stop improving during each epoch.\n",
        "\n",
        "We will then fit the model which we defined in the previous step. This should take a little over 2 minutes per epoch. Notice that the training loss decreases and the accuracy increases over time."
      ]
    },
    {
      "metadata": {
        "scrolled": true,
        "trusted": true,
        "id": "pbJJs61pohla",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "#@title Define model parameter values\n",
        "batch_size = 64 #@param {type:\"slider\", min:0, max:128, step:32}\n",
        "epochs = 3 #@param {type: \"slider\", min: 1, max: 8}\n",
        "\n",
        "early_stopping = keras.callbacks.EarlyStopping(monitor='val_loss', patience=10, verbose=1)\n",
        "checkpoint = keras.callbacks.ModelCheckpoint(filepath=\"best_weights.hdf5\", monitor='val_loss', verbose=1, save_best_only=True)\n",
        "callbacks = [early_stopping, checkpoint]\n",
        "\n",
        "convergence_history = model.fit(x_train, y_train, validation_data=(x_test, y_test), epochs=epochs, batch_size=batch_size, callbacks=callbacks)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "1BpNKjEnGDm-",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "#Step 6 - Validation"
      ]
    },
    {
      "metadata": {
        "id": "4X2-ab2X0EZi",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "# Plot training & validation accuracy values\n",
        "plt.figure(figsize=(12,8))\n",
        "plt.plot(convergence_history.history['acc'])\n",
        "plt.plot(convergence_history.history['val_acc'])\n",
        "plt.title('Model accuracy')\n",
        "plt.ylabel('Accuracy')\n",
        "plt.xlabel('Epoch')\n",
        "plt.legend(['Train', 'Test'], loc='upper left')\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "B6hp9x4dMR4G",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Plot training & validation loss values\n",
        "plt.figure(figsize=(12,8))\n",
        "plt.plot(convergence_history.history['loss'], label=\"Training loss\")\n",
        "plt.plot(convergence_history.history['val_loss'], label=\"Validation loss\")\n",
        "plt.grid(\"on\")\n",
        "plt.xlabel(\"Epoch\")\n",
        "plt.ylabel(\"Loss\")\n",
        "plt.legend()\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "FxPmtZR9ohle",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Step 7 - Evaluate the model\n",
        "\n",
        "Use the test data to calculate the accuracy of the trained model."
      ]
    },
    {
      "metadata": {
        "trusted": true,
        "id": "BHsubm8Xohlg",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "scores = model.evaluate(x_test, y_test, verbose=1) \n",
        "print(\"Accuracy: %.2f%%\" % (scores[1]*100))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "u-ytA_TWohlq",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Step 8 - Use the trained model\n",
        "Predict sentiment from the reviews:\n",
        "\n",
        "\"this movie was terrible and bad\"\n",
        "\"i really liked the movie and had fun\""
      ]
    },
    {
      "metadata": {
        "trusted": true,
        "id": "eHbsaAIPohlr",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "bad = \"this movie was terrible and bad\"\n",
        "good = \"i really liked the movie and had fun\"\n",
        "for review in [good,bad]:\n",
        "    tmp = []\n",
        "    for word in review.split(\" \"):\n",
        "        tmp.append(word_to_id[word])\n",
        "    tmp_padded = sequence.pad_sequences([tmp], maxlen=max_review_length) \n",
        "    print(\"%s. Sentiment: %s\" % (review,model.predict(array([tmp_padded][0]))[0][0]))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "ifF9Bp9QCqnQ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Now predict your own sentences with the trained model"
      ]
    },
    {
      "metadata": {
        "id": "StOHJEDKST_z",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import string\n",
        "#@title Define your review and predict the sentiment\n",
        "badReview = \"\" #@param {type:\"string\"}\n",
        "goodReview = \"\" #@param {type:\"string\"}\n",
        "\n",
        "if badReview == \"\" or goodReview == \"\":\n",
        "  print(\"Please provide a good and a bad review.\")\n",
        "else :\n",
        "  exclude = set(string.punctuation)\n",
        "  for review in [badReview,goodReview]:\n",
        "      review = ''.join(ch for ch in review.lower() if ch not in exclude)\n",
        "      tmp = []\n",
        "      for word in review.split(\" \"):\n",
        "        if word in word_to_id:\n",
        "          if word_to_id[word] < top_words:\n",
        "            tmp.append(word_to_id[word])\n",
        "      tmp_padded = sequence.pad_sequences([tmp], maxlen=max_review_length) \n",
        "      print(\"%s. Sentiment: %s\" % (review, model.predict(array([tmp_padded][0]))[0][0]))"
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}