Deep_learning_for_Text_Classification.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# Using TextVectorization Layer"
      ],
      "metadata": {
        "id": "HgFNU35npuT1"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Deep learning models, being differentiable functions, can only process numeric tensors: they can’t take raw text as input. Vectorizing text is the process of transforming\n",
        "text into numeric tensors. Text vectorization processes come in many shapes and\n",
        "forms, but they all follow the same template :\n",
        "\n",
        " First, you standardize the text to make it easier to process, such as by converting\n",
        "it to lowercase or removing punctuation.\n",
        "\n",
        " You split the text into units (called tokens), such as characters, words, or groups\n",
        "of words. This is called tokenization.\n",
        "\n",
        " You convert each such token into a numerical vector. This will usually involve\n",
        "first indexing all tokens present in the data."
      ],
      "metadata": {
        "id": "48zRdoilp1x5"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Every step I’ve introduced so far would be very easy to implement in pure Python.\n"
      ],
      "metadata": {
        "id": "nRcwx9XNp9gd"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "RZQYkZF8prNT"
      },
      "outputs": [],
      "source": [
        "import string\n",
        "class Vectorizer:\n",
        " def standardize(self, text):\n",
        "    text = text.lower()\n",
        "    return \"\".join(char for char in text if char not in string.punctuation)\n",
        "\n",
        " def tokenize(self, text):\n",
        "    text = self.standardize(text)\n",
        "    return text.split()\n",
        "\n",
        " def make_vocabulary(self, dataset):\n",
        "    self.vocabulary = {\"\": 0, \"[UNK]\": 1}\n",
        "    for text in dataset:\n",
        "      text = self.standardize(text)\n",
        "      tokens = self.tokenize(text)\n",
        "      for token in tokens:\n",
        "        if token not in self.vocabulary:\n",
        "          self.vocabulary[token] = len(self.vocabulary)\n",
        "    self.inverse_vocabulary = dict((v, k) for k, v in self.vocabulary.items())\n",
        "\n",
        " def encode(self, text):\n",
        "    text = self.standardize(text)\n",
        "    tokens = self.tokenize(text)\n",
        "    return [self.vocabulary.get(token, 1) for token in tokens]\n",
        "\n",
        " def decode(self, int_sequence):\n",
        "    return \" \".join(self.inverse_vocabulary.get(i, \"[UNK]\") for i in int_sequence)"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "vectorizer = Vectorizer()\n",
        "dataset = [\n",
        " \"I write, erase, rewrite\",\n",
        " \"Erase again, and then\",\n",
        " \"A poppy blooms.\",\n",
        "]\n",
        "vectorizer.make_vocabulary(dataset)"
      ],
      "metadata": {
        "id": "Nl127awaq5cm"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(vectorizer.vocabulary)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SO7HIfQvsU2C",
        "outputId": "e8469828-ace6-47c3-d3c7-673c65c2115a"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{'': 0, '[UNK]': 1, 'i': 2, 'write': 3, 'erase': 4, 'rewrite': 5, 'again': 6, 'and': 7, 'then': 8, 'a': 9, 'poppy': 10, 'blooms': 11}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "test_sentence = \"I write, rewrite, and still rewrite again\""
      ],
      "metadata": {
        "id": "VuxJKndiq733"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "encoded_sentence = vectorizer.encode(test_sentence)\n",
        "print(encoded_sentence)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Pgq0snk_rB7e",
        "outputId": "1c986c62-c23f-415d-f678-f4a236a6815f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[2, 3, 5, 7, 1, 5, 6]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "decoded_sentence = vectorizer.decode(encoded_sentence)\n",
        "print(decoded_sentence)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8u4TV2cisIbK",
        "outputId": "f0cb812a-4792-4767-b867-c62cd11ffc72"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "i write rewrite and [UNK] rewrite again\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "However, using something like this wouldn’t be very performant. In practice, you’ll\n",
        "work with the Keras TextVectorization layer, which is fast and efficient and can be\n",
        "dropped directly into a tf.data pipeline or a Keras model."
      ],
      "metadata": {
        "id": "gCQZYtCGs7J_"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "[What are Tensors and why they are used?](https://chat.openai.com/share/99048956-5c7f-4744-9802-ff77e0241905)"
      ],
      "metadata": {
        "id": "gDKYPgviusLO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from tensorflow.keras.layers import TextVectorization\n",
        "text_vectorization = TextVectorization(output_mode=\"int\",) # Configures the layer to return sequences of words encoded as integer indices. There are several other output modes available"
      ],
      "metadata": {
        "id": "IHNRKXyKs8Ax"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "By default, the TextVectorization layer will use the setting “convert to lowercase and\n",
        "remove punctuation” for text standardization, and “split on whitespace” for tokenization."
      ],
      "metadata": {
        "id": "m2SwsjTEu4Hu"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "But importantly, you can provide custom functions for standardization and tokenization, which means the layer is flexible enough to handle any use case. Note that\n",
        "such custom functions should operate on tf.string tensors, not regular Python\n",
        "strings!"
      ],
      "metadata": {
        "id": "OxWCkkWyu4r7"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        " For instance, the default layer behavior is equivalent to the following:"
      ],
      "metadata": {
        "id": "DrL0R4wUu9lC"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import re\n",
        "import string\n",
        "import tensorflow as tf\n",
        "def custom_standardization_fn(string_tensor):\n",
        " lowercase_string = tf.strings.lower(string_tensor) # Convert strings to lowercase.\n",
        " return tf.strings.regex_replace(lowercase_string, f\"[{re.escape(string.punctuation)}]\", \"\") # Replace punctuation characters with the empty string.\n",
        "\n",
        "def custom_split_fn(string_tensor):\n",
        " return tf.strings.split(string_tensor) # Split strings on whitespace.\n",
        "\n",
        "\n",
        "text_vectorization = TextVectorization(\n",
        " output_mode=\"int\",\n",
        " standardize=custom_standardization_fn,\n",
        " split=custom_split_fn,\n",
        ")"
      ],
      "metadata": {
        "id": "rU0aNCUiu64_"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "To index the vocabulary of a text corpus, just call the adapt() method of the layer\n",
        "with a Dataset object that yields strings, or just with a list of Python strings"
      ],
      "metadata": {
        "id": "OQw14rtXvv6M"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "dataset = [\n",
        " \"I write, erase, rewrite\",\n",
        " \"Erase again, and then\",\n",
        " \"A poppy blooms.\",\n",
        "]\n",
        "text_vectorization.adapt(dataset)"
      ],
      "metadata": {
        "id": "mXC-3iWGvwXw"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Note that you can retrieve the computed vocabulary via get_vocabulary()—this can\n",
        "be useful if you need to convert text encoded as integer sequences back into words.\n",
        "The first two entries in the vocabulary are the mask token (index 0) and the OOV\n",
        "token (index 1). Entries in the vocabulary list are sorted by frequency, so with a realworld dataset, very common words like “the” or “a” would come first."
      ],
      "metadata": {
        "id": "T2sschSpv66r"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "text_vectorization.get_vocabulary()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "57VRODMuv7Yt",
        "outputId": "e899408a-75d6-448c-8f1a-3ee806aa434a"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['',\n",
              " '[UNK]',\n",
              " 'erase',\n",
              " 'write',\n",
              " 'then',\n",
              " 'rewrite',\n",
              " 'poppy',\n",
              " 'i',\n",
              " 'blooms',\n",
              " 'and',\n",
              " 'again',\n",
              " 'a']"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "vocabulary = text_vectorization.get_vocabulary()\n",
        "\n",
        "# Test sentence\n",
        "test_sentence = \"I write, rewrite, and still rewrite again\"\n",
        "\n",
        "# Encode the test sentence\n",
        "encoded_sentence = text_vectorization(test_sentence)\n",
        "\n",
        "# Print the encoded sentence\n",
        "print(encoded_sentence)\n",
        "\n",
        "# Create an inverse vocabulary mapping\n",
        "inverse_vocab = dict(enumerate(vocabulary))\n",
        "\n",
        "# Decode the encoded sentence\n",
        "decoded_sentence = \" \".join(inverse_vocab[int(i)] for i in encoded_sentence)\n",
        "\n",
        "# Print the decoded sentence\n",
        "print(decoded_sentence)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TwQPRedUxIhq",
        "outputId": "208fb0f7-f4f9-4a23-d0f3-7d7fc9520948"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tf.Tensor([ 7  3  5  9  1  5 10], shape=(7,), dtype=int64)\n",
            "i write rewrite and [UNK] rewrite again\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "[Word respresentations in ML](https://chat.openai.com/share/07c02eb3-c1ad-45c9-8bc7-094899f3e793)"
      ],
      "metadata": {
        "id": "Nw-7lfJi8tVB"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "[Word order in Transformners ](https://chat.openai.com/share/5c5cd13c-b8a0-41c5-a0b5-15863961bcac)"
      ],
      "metadata": {
        "id": "Axlm_YKoAeR9"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# IMDB Movie Review"
      ],
      "metadata": {
        "id": "vzWTzkGU37gg"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!curl -O https://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz\n",
        "!tar -xf aclImdb_v1.tar.gz"
      ],
      "metadata": {
        "id": "N_U_eZDz8yTU",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "03902981-3954-43ff-838e-c307bbde0031"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
            "                                 Dload  Upload   Total   Spent    Left  Speed\n",
            "100 80.2M  100 80.2M    0     0  32.1M      0  0:00:02  0:00:02 --:--:-- 32.1M\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "You’re left with a directory named aclImdb, with the following structure:\n",
        "\n",
        "aclImdb/\n",
        "\n",
        "...train/\n",
        "\n",
        "......pos/\n",
        "\n",
        "......neg/\n",
        "\n",
        "...test/\n",
        "\n",
        "......pos/\n",
        "\n",
        "......neg/"
      ],
      "metadata": {
        "id": "4OpedpSf4Ok7"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "For instance, the train/pos/ directory contains a set of 12,500 text files, each of which\n",
        "contains the text body of a positive-sentiment movie review to be used as training data.\n",
        "The negative-sentiment reviews live in the “neg” directories. In total, there are 25,000\n",
        "text files for training and another 25,000 for testing."
      ],
      "metadata": {
        "id": "m3r39IXa4cKC"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "There’s also a train/unsup subdirectory in there, which we don’t need. Let’s\n",
        "delete it:"
      ],
      "metadata": {
        "id": "IOKcJ7tA4hJf"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!rm -r aclImdb/train/unsup"
      ],
      "metadata": {
        "id": "aXx3ZsE94Uo6"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Take a look at the content of a few of these text files. Whether you’re working with\n",
        "text data or image data, remember to always inspect what your data looks like before\n",
        "you dive into modeling it. It will ground your intuition about what your model is actually doing:"
      ],
      "metadata": {
        "id": "gYL-VcpG4p9g"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!cat aclImdb/train/pos/4077_10.txt"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HxrnzcGm4r2H",
        "outputId": "9b7a1898-e5d4-40e8-e7e8-09de2fa6d3cb"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "I first saw this back in the early 90s on UK TV, i did like it then but i missed the chance to tape it, many years passed but the film always stuck with me and i lost hope of seeing it TV again, the main thing that stuck with me was the end, the hole castle part really touched me, its easy to watch, has a great story, great music, the list goes on and on, its OK me saying how good it is but everyone will take there own best bits away with them once they have seen it, yes the animation is top notch and beautiful to watch, it does show its age in a very few parts but that has now become part of it beauty, i am so glad it has came out on DVD as it is one of my top 10 films of all time. Buy it or rent it just see it, best viewing is at night alone with drink and food in reach so you don't have to stop the film.<br /><br />Enjoy"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Next, let’s prepare a validation set by setting apart 20% of the training text files in a\n",
        "new directory, aclImdb/val"
      ],
      "metadata": {
        "id": "4c-PYPbz5XFG"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import os, pathlib, shutil, random\n",
        "\n",
        "base_dir = pathlib.Path(\"aclImdb\")\n",
        "val_dir = base_dir / \"val\"\n",
        "train_dir = base_dir / \"train\"\n",
        "for category in (\"neg\", \"pos\"):\n",
        " os.makedirs(val_dir / category)\n",
        " files = os.listdir(train_dir / category)\n",
        " random.Random(1337).shuffle(files) # Shuffle the list of training files using a seed, to ensure we get the same validation set every time we run the code.\n",
        " num_val_samples = int(0.2 * len(files))\n",
        " val_files = files[-num_val_samples:]\n",
        " for fname in val_files:\n",
        "    shutil.move(train_dir / category / fname,val_dir / category / fname)"
      ],
      "metadata": {
        "id": "Lkph5VDn5Xi7"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "You can\n",
        "do the exact same thing for text files using the text_dataset_from_directory utility.\n",
        "Let’s create three Dataset objects for training, validation, and testing"
      ],
      "metadata": {
        "id": "fDa9FVzQ558Y"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from tensorflow import keras\n",
        "batch_size = 32\n",
        "train_ds = keras.utils.text_dataset_from_directory(\n",
        " \"aclImdb/train\", batch_size=batch_size\n",
        ")\n",
        "val_ds = keras.utils.text_dataset_from_directory(\n",
        " \"aclImdb/val\", batch_size=batch_size\n",
        ")\n",
        "test_ds = keras.utils.text_dataset_from_directory(\n",
        " \"aclImdb/test\", batch_size=batch_size\n",
        ")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "uAgBNYIS56be",
        "outputId": "45575042-30cc-47ec-a1c6-99db99ff1938"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Found 20000 files belonging to 2 classes.\n",
            "Found 5000 files belonging to 2 classes.\n",
            "Found 25000 files belonging to 2 classes.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "for inputs, targets in train_ds:\n",
        "    print(\"inputs.shape:\", inputs.shape)\n",
        "    print(\"inputs.dtype:\", inputs.dtype)\n",
        "    print(\"targets.shape:\", targets.shape)\n",
        "    print(\"targets.dtype:\", targets.dtype)\n",
        "    print(\"inputs[0]:\", inputs[0])\n",
        "    print(\"targets[0]:\", targets[0])\n",
        "    break"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TFa3uWgm68jv",
        "outputId": "a1b9adff-8d4f-4e9d-b920-af69e828bc87"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "inputs.shape: (32,)\n",
            "inputs.dtype: <dtype: 'string'>\n",
            "targets.shape: (32,)\n",
            "targets.dtype: <dtype: 'int32'>\n",
            "inputs[0]: tf.Tensor(b'Moe and Larry are newly henpecked husbands, having married Shemp\\'s demanding sisters. At his music studio, Shemp learns he will inherit a fortune if he marries someone himself! <br /><br />\"Husbands Beware\" is a remake of 1947\\'s \"Brideless Groom,\" widely considered by many to be one of the best Stooge films with Shemp. The remake contains most of the footage from that film. The new scenes, shot May 17, 1955, include the storyline of Moe and Larry marrying Shemp\\'s sisters, along with their cooking of a turkey laced with turpentine! A few new scenes are tacked onto the end of the film as well(a double for Dee Green was used; if you blink, you will miss the double\\'s appearance.) <br /><br />\"Husbands Beware\" would have made for a good film with just the plot line of marrying the sisters. Budget considerations, coupled with fewer bookings for two-reel comedies, influenced the decision to use older footage.<br /><br />Although completely new films were still being made by the Stooges, most of their releases by 1955-56 were made up of older films with a few new scenes tossed in. \"Husbands Beware,\" while one of these hybrids, is watchable and entertaining; we get to see most of \"Brideless Groom\" again, and the new scenes are funny enough to get the viewer through the film. This film is one of the last Stooge comedies to feature new footage of Shemp, and it was released six weeks after his death.<br /><br />7 out of 10.', shape=(), dtype=string)\n",
            "targets[0]: tf.Tensor(1, shape=(), dtype=int32)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Processing words as a set: The bag-of-words approach"
      ],
      "metadata": {
        "id": "j8dFUth17KGS"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "The simplest way to encode a piece of text for processing by a machine learning\n",
        "model is to discard order and treat it as a set (a “bag”) of tokens. You could either look\n",
        "at individual words (unigrams), or try to recover some local order information by\n",
        "looking at groups of consecutive token (N-grams)."
      ],
      "metadata": {
        "id": "Hh-4sXyg-Ee7"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# SINGLE WORDS (UNIGRAMS) WITH BINARY ENCODING\n",
        "\n",
        "\n",
        "If you use a bag of single words, the sentence “the cat sat on the mat” becomes\n",
        "{\"cat\", \"mat\", \"on\", \"sat\", \"the\"}\n",
        "\n",
        "The main advantage of this encoding is that you can represent an entire text as a single vector, where each entry is a presence indicator for a given word.\n",
        "\n",
        " For instance,\n",
        "using binary encoding (multi-hot), you’d encode a text as a vector with as many\n",
        "dimensions as there are words in your vocabulary—with 0s almost everywhere and\n",
        "some 1s for dimensions that encode words present in the text."
      ],
      "metadata": {
        "id": "mCgi3bWT-E9P"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "[Single Unigram , it's advantages and binary encoding](https://chat.openai.com/share/ef00b623-71a1-4c71-9760-af1515436016)"
      ],
      "metadata": {
        "id": "GUVaAIIk932V"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "First, let’s process our raw text datasets with a TextVectorization layer so that\n",
        "they yield multi-hot encoded binary word vectors. Our layer will only look at single\n",
        "words (that is to say, unigrams)."
      ],
      "metadata": {
        "id": "WjsKKoih-T21"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Limit the vocabulary to the 20,000 most frequent words. Otherwise we’d be indexing every word in the training data— potentially tens of thousands of terms that only occur once or\n",
        "# twice and thus aren’t informative. In general, 20,000 is the right vocabulary size for text classification.\n",
        "\n",
        "text_vectorization = TextVectorization(\n",
        " max_tokens=20000,\n",
        " output_mode=\"multi_hot\", # Encode the output tokens as multi-hot binary vectors.\n",
        ")"
      ],
      "metadata": {
        "id": "X96F0Kbm7LBN"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "text_only_train_ds = train_ds.map(lambda x, y: x) # Prepare a dataset that only yields raw text inputs (no labels)."
      ],
      "metadata": {
        "id": "Rpu7uurr-qhz"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Take one element from the dataset\n",
        "sample_data = next(iter(text_only_train_ds.take(1)))\n",
        "\n",
        "# Print the sample data\n",
        "print(sample_data)"
      ],
      "metadata": {
        "id": "FIkoyXTW-ySp",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b50b608c-9f70-42ad-bcce-b217cee935d0"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tf.Tensor(\n",
            "[b\"This film tells the story of a romance between Albert Einstien niece and a gas station attendant. In order to get the two together, Einstien agrees to help Ed(Hudsucker Proxy's Tim Robbins) learn to act more intelligent. This impresses Catherine (Meg Ryan). Unfortunately Einstien goes too far and Ed is considered to be a genius. Hilarity ensues. Not to be missed. Filmed in Mercer county New Jersey at Princeton University, Lawrenceville Prep School (doubling for Princeton University) as well as a beautiful vintage gas station in Hopewell.\"\n",
            " b'New York City houses one man above all others, the possibly immortal Dr. Anton Mordrid. Mordrid is the sworn protector of humanity, using his magical powers to keep his brother and rival, Kabal, chained up so that he may not enslave the human race. Well, wouldn\\'t you know it? A prophesy comes true and Kabal breaks free, and begins collecting elements (including platinum and uranium) for his alchemy experiments. With the help of a police woman named Sam, can Mordrid defeat his evil brother? \"Dr. Mordrid\" comes to me courtesy of Charles Band in the Full Moon Archive Collection. I had not heard of it, which is a bit odd given that I\\'m a big fan of Jeffrey Combs (Mordrid) and the film isn\\'t that old. But now it\\'s mine and I can enjoy it again and again. The film certainly is fun in the classic Full Moon style. Richard Band provides the music (which doesn\\'t differ much from all his other scores) and Brian Thompson plays the evil Kabal. We even have animated dinosaur bones! What more do you want? Of course, the cheese factor is high. I felt much of the film was a rip-off of the Dr. Strange comics. And the blue pantsuit was silly. And plot holes are everywhere (I could list at least five, but why bother). And why does the ancient symbol of Mordrid and Kabal look suspiciously like a hammer and sickle? Combs has never been a strong actor, so he fits right in with the cheese. These aren\\'t complaints. Full Moon fans have come to expect these things and devour them like crack-laced Grape Nuts. I\\'m guilty... I loved this film.<br /><br />If you\\'re not a Full Moon fan, or a Jeffrey Combs fan... you may want to look elsewhere. But if you like the early 1990s style of movie-making and haircuts, you\\'ll eat this up. Stallone and Schwarzenegger fans might like seeing Brian Thompson as a villain, looking as goony as ever and not being able to enunciate English beyond a third grade level. I did. I wish there was a \"Mordrid II\", but the company that makes a sequel to practically everything (is \"Gingerdead Man 3\" really necessary?) passed on this one.'\n",
            " b'PRC which was the lowest of the low actually struck gold with this moody little thriller. They did the same thing a year earlier with \"Detour\" which is probably one of the finest low-budget films ever made.<br /><br />\"Strangler\" is basically a one set film, filled with mist and shadows, a technique used by most poverty row studios to hide the sets, or lack thereof. But here, it works well. The ghost of Charles Middleton (better known as Ming the Merciless) lurches around the swamp killing those involved in his wrongful execution for murder and generates some sympathy from the viewer. His final victim is to be the daughter of the ferryman.....he concentrates his wrath not only on those directly involved in his fate but their relatives as well.<br /><br />Rosemary LaPlanche does her usual imitation of someone in a coma that passes for her acting style. She offers herself up to the strangler in order to put a stop to the killing but as a sop to the audience, the strangler sees the goodness of her gesture as a sign that his mission is complete and he returns to the hereafter, somewhat chastened. If Ulmer(who directed \"Detour\") has directed \"Strangler\" she would be hanging from the nearest tree and the strangler\\'s job would be done. But who\\'s complaining? It\\'s not the story that is the major attraction but the shrouded sets, lighting and the general moodiness of the piece. It stands, right behind \"Detour\", as PRC\\'s finest hour'\n",
            " b'This is what makes me proud to be British. This is by far the funniest thing on TV. The league consists of Jeremy dyson, Steve pemberton, mark gatiss and the lovely Reece shearsmith. Totally underrated, this horror-comedy is perfection. The characters are iconic and the catchphrases bizarre, \"Hello Dave\". It is a comedy that everyone simply must watch.<br /><br />The best thing about the league of gentlemen is that it is always fresh, and always pushing the boundaries. It does not need to rely on catchphrases(unlike little Britain) for it to be funny. the fact that the league are willing to kill off arguably their most famous and iconic characters, shows us that they\\'ve got balls of steel.'\n",
            " b'\"Women? They\\'re all scrubbers...!\" <br /><br />No, not a good translation; not at all! This lags behind the previous year\\'s \"Dad\\'s Army\", entirely missing the special, small-screen magic of the seminal television sitcom original, and failing to play interestingly at all with the big screen... you could just about say that this film well represents a Britain entering decline, and more precisely even than that, a *British film industry* entering decline. And that is hardly a recommendation, is it? To be an exemplar of saddening folly...<br /><br />All that remains after the subtlety of the TV original has been surgically stripped away, by Cliff Owen, Galton and Simpson are: endless, dilapidated musical cues, yawn, from the Ron Grainer theme... bolstered sentimentality (that shoddy, thick-eared ending... how much bolder does the second Steptoe film seem in comparison) an increased seediness - with director and writers seemingly detaching themselves completely - fully applicable to something like the \\'misbegotten monstrosity\\' (yours truly on this site) from 1973, \"The Mutations\". There is a strangely botched, cut-adrift tone about the scene where Harold is beaten up in a rugby club, that I partly hate and recoil it (so far, as a friend intimated, from the mood of the TV series...), but this at least seems an original slant, and emblematic of tensions just rising to the boil in the Britain of 1972... There is, however, an implied prostitute, aye of a \\'heart-of-gold\\' who turns loose woman-traitor \\'pon poor auld \\'Arold - and beyond-caricature writing of the \\'class\\' element; not to mention, surprisingly misjudged performances from the usually redoubtable leads. Brambell and Corbett collude with the script, and indeed fail to cure it of an essential ham. What would Anthony Aloysius Hancock have made of it all...? I will merely concede that a few moments just about work - chiefly those where G & S play things a little more carefully and B & C touch tenderer nerves - and it is not on the whole an unwatchable affair. <br /><br />But, and oh, how this pains me to say it: it is tiresome, boring, both wilfully detached from reality and what made the TV series great, and also fully in tune with the lazy, tawdry, misogynist \\'fuck it, that\\'ll do...\\' actuality of much of what was allowed to pass for mainstream film-making in the Britain of the time.'\n",
            " b\"I have a nice collection on movies going, and this one was added to it. Number 274 to be exact. <br /><br />The title had me going at first. Splatter University. I thought this would be a great horror movie. Was I ever wrong. Don't get me wrong, this is not the worst movie I have ever seen, but it could have been a lot better. I love all movies, but this one was one that was more of a laugh then a scare. <br /><br />Poor audio quality, poor acting, and poor shot arrangements are some of the areas that could have been improved. 3 out of 10 stars.<br /><br />Movie is ideal for a good laugh. If your looking for one of those movies to make fun of, then this is one!\"\n",
            " b\"The film had many fundamental values of family and love. It expressed human emotion and was an inspiring story. The script was clear( it was very easy to understand making it perfect for children)and was enjoyable and humorous at times. There were a few charged symbols to look for. The cinematography was acceptable. There was no sense of experimentation that a lot of cinematographers have been doing today(which quiet frankly is getting a little warn out). It was plainly filmed but had a nice soft quality to it. Although editing could have been done better I thought it was a nice movie for a family to enjoy. And the organization of information was just thrown at you which was something I didn't like either but in all it was a good movie.\"\n",
            " b'I lost my father at a very young age.So young in fact,that I have no recollection of him.Over the years I have learned many things about him. One of those things was that he loved westerns,and watching Bonanza every Sunday evening was an absolute ritual for him.I,myself, remember the tail end of the series\\' run,having been 8 years old when the show ceased production in 1973.Watching this show over the years somehow makes me closer to my long ago lost father.It has all the right elements to make a show successful;laughter,tears,edge of your seat suspense,and it even angered you at times.My most vivid memory of the show\\'s original run,came shortly after the death of our beloved \"Hoss\" Cartwright,Dan Blocker.One particular episode,and the end of the closing credits, flashed a picture of Blocker,and faded to black,and I can also recall my oldest sister with a tear in her eye at the sight of this.I can remember this as though it were yesterday.On behalf of my late father, who is not here to say so himself,we love Bonanza.Long live the Cartwrights.'\n",
            " b\"This is a most handsome film. The color photography is beautiful as it shows the lavishness of the Metropolitan Opera House in brilliant color. Other indoor scenes at various mansions, etc are equally brilliant. As for the music, what more can be said other than that Lanza's voice was at its' peak as he sang so many of the worlds' best known and beloved arias. The marvelous Dorothy Kirsten is also a joy as her soprano voice blends with that of Lanza in delightful harmony. Of course, Hollywood took their customary liberties with the life story of Caruso. There is precious little in the story line that relates to actual events. For example, the facts relating to his death are totally fabricated and bear no relationship to the truth. There are some very good web sites that tell the true story of Caruso and contain several pictures of him. These web sites can be located by using any good search engine. There are also several books available concerning his life history. But, the fictional story line does nothing to mar this beautiful film. The voices of Lanza, Kirsten, and the chorus members are the real stars of this movie. Enjoy, I know that I sure did.\"\n",
            " b'This very unfunny failed TV Pilot can be found as an extra on the 30th Annivesery DVD Special Edition \"Blazing Saddles\". Imagine the movie without the satire, humor, or writing skills. But with all the trappings of a typical lame \\'70\\'s sit-com show complete with obtrusive laugh track and you\\'ll still have no clue how sheer putrid this failed show was. What the hell was Lou Gossett Jr. thinking when he signed onto this disaster?? This was possibly the worst thing he\\'s been in (and yes I\\'m including the first \"Punisher\" movie and \"Iron Eagles 3\". Steve Landesberg, I understand as he can\\'t say no to crap.<br /><br />My Grade: F'\n",
            " b'What made the original Killer Tomatoes fun was it was made by people with no budget who were just being wacky for a couple of days...<br /><br />This was something with a budget, but it just wasn\\'t as much fun. John Astin of Adams Family fame is actually making an effort here to be comedic, but he is supported by lame actors, cheap special effects and unfunny gags.<br /><br />The plot. Dr. Gangrene (Astin) escapes from a French prison and decides he is going to put a pretender on the throne of France... The hero, his French girlfriend and the Gizmo-like \"Fuzzy Tomato\" decide they are going to stop him...<br /><br />Forgettable Direct to Video nonsense...'\n",
            " b\"Most horror movies are in fact horrible movies. They get to be same ol'-same ol'. Same ol' pack every minute with some cheap thrill (usually 'splatter') and nowadays they can pack every second with gaudy special effects. One of the goals of a really good horror flick is to suspend the sense of disbelief of the audience. For instance, I saw both of the recent Mummy movies and nearly got dizzy viewing ridiculous special effects every second. It probably costs a million dollars per second to make those movies and my sense of disbelief was never suspended, it grew roots.<br /><br />Subtlety can be more terrifying. Less is more. <br /><br />I first saw 'The Woman in Black' on the A&E channel. After flipping through the usual 987 channels of very bad television I stopped to watch it. This movie almost has the feel of a 'Masterpiece Theater' production. That was fine with me, I've always preferred British TV & movies anyway.<br /><br />Most viewers would find this to be too slowly paced. But the slow pacing helps give the story credibility. The special effects are few which lulls the viewer into thinking that this film is set in the real world thus making us a bit more uneasy. The makeup and costume for the ghost are kept simple and believable. Hollywood would have made her look like a she demon from hell with glowing eyes-fangs-claws etc. Hollywood would have done an overkill and turned this idea into a mediocrity.<br /><br />The woman only makes about five appearances in the film. Most of them are where she appears in the distance and even that creates a good fright. If she appeared too often, it could've cheapened the mood that gets set. However this movie is so well made that through much of the film we're led into sensing that she is there the whole time but not visible. The scene where she 'visits' Arthur Kidd late at night and we see her just a little too close is a masterpiece in horror.<br /><br />This is just an extraordinary film that I think should rate as one the finest horror films ever made. I have a copy of 'The Haunting', 'The Changling' and a zillion more. I haven't seen anything that tops 'The Woman in Black' yet although I'm still looking. This movie is so well made that it gives even the most hardened skeptic (like me) a moment where I almost had second thoughts about the non-existence of ghosts. I joke to people that I occasionally get brief fears that she could appear standing in the middle of the road or that I'd see her staring through my window, etc. Maybe she could be in a crowd at the mall glaring at me with her look of hate. This is how a really great horror film should be. Like a LaFanu novel, The Woman in Black very slowly pulls you in and wraps herself around your neck and before you realize it, she's squeezing the life out of you and then it's too late.<br /><br />Closest thing I have to a criticism is that this was made for the small screen... and it's a terrible shame that this is out of print. I just paid over $40 for my second copy of this movie. It's a major prize in my collection. Now I'm on a quest to find an even better horror movie that not only gives the chills but also qualifies a sound drama.\"\n",
            " b\"I do have the `guts' to inform you to please stay away from `Dahmer', the biographical film based on the real-life story of the grotesque serial killer. `Dahmer' strays more in relation to the mentality of its focused subject. Jeffrey Dahmer, who murdered over 15 young males and ate some of their body parts, was probably the most incongruous serial killer of our generation. However, the real sick individuals are the filmmakers of this awful spectacle who should have had their heads examined before deciding to greenlight this awful `dahm' project. This is not an easy film to digest, even though Jeffrey would have easily digested it with some fiery `brainsadillas' appetizers or even some real-life `Mr. Potato skins'. * Failure\"\n",
            " b'I read a couple of good reviews on this board for \"Mr. Scarface\", but for anyone uninitiated in the genre of Italian gangster films like myself, the picture will probably make very little sense. Indeed, after the initial setup involving the ten million lira scam, the picture devolved into a fairly routine revenge flick with a minor twist in the identity of Rick\\'s (Al Cliver) character. The whole gang war plot got muddied up for me with the inclusion of Vinchenzo Napoli (Vittorio Caprioli), but as most other viewers commented, he\\'s about the only one who gave this picture any life with his often ineffective attempts at violence. I found it somewhat unbelievable that Manzari\\'s goons who chased Tony through the streets didn\\'t actually stroke out before Tony even laid a hand on them. For all of his buildup as the title character, Jack Palance was wasted rather unceremoniously in an anticlimactic near finale, making the U.S. working title, \"Mr. Scarface\", rather moot. I\\'ve seen enough spaghetti Westerns to know that they don\\'t all work; I guess in this case, my first look at a spaghetti gangster flick didn\\'t quite make it either.'\n",
            " b'A visit by Hitler in Rome is the backdrop of this tender story of love, friendship, homosexuality and fascism. Sophia Loren plays the housewife and mother of six children who stays at home while her entire family go to the military parade in honor of Hitler and Mussolini. She has to stay at home since the family cannot afford a maid. She would have loved to go though as she along with the entire housing complex where she lives is an ardent admirer of Il Duce.<br /><br />There is one exception though. Across the yard sits Marcello Mastroianni on his chair contemplating suicide. The reason? He is homosexual and because of that has recently lost his job as a radio announcer. The film really takes off when these two people meet by chance. Mastroianni is in despair and badly in need of a friend. Loren, frustrated by her own cheating husband misunderstands Mastroianni and in a masterfully shot, directed and acted scene on the roof of the building complex offers her body to him only to be rejected. The initial chock is replaced soon afterwards by her hunger for this man, this anti fascist, this homosexual, this other world who is so willing to give her all that she longs for.<br /><br />This is a beautifully crafted movie with two of the most talented actors ever. Loren proves here that she is an actress of caliber when well directed. This is a simple but yet powerful film about fascism, love, ordinary people and most importantly the human condition. Despite its sad ending there is a glimpse of hope in the denouement, things will change, someone has understood.'\n",
            " b'I hired out Hybrid on the weekend. What a disappointment! A stupid lame attempt at a tele-movie. The guy they got for the lead was totally weak and when running {he did a lot} looked like he was eating those minty sweets...with his backside! The wolf contacts he wore were great, though I feel the actor relied on them too much, as there was nothing menacing about his acting at all. The wise native American Indian chick has to be one of the most stony hard faced hags ever seen. Talk about a sour cow! She smiled about once for the entire film, and I think that is because she had sex. The sex scene was lame too. They may as well have shown blowing curtains, if you can dig that.<br /><br />Last of all, and this is a big pet hate of mine, on the cover and the DVD menu, the losers digitally drew in cool sharp teeth on the guy. They were nowhere to be seen in the film. :('\n",
            " b\"I saw this movie as a very young girl (I'm 27 now) and it scared me witless for years. I had nightmares about every aspect of this film from the way it was drawn to the music to (obviously) the violence. My parents still argue about who allowed me to watch it and both of them say that they would never let me watch such a movie. I think they only say that knowing that I have such strong feelings about it ;0) I am currently reading the book (out of morbid curiosity and the fact that it's a classic) and it is really a great story. However I don't think that it should have been made into a cartoon. Ever. Well, maybe kids nowadays would find it quaint but it gave me nightmares for weeks and weeks and I still have a hard time seeing rabbits drawn in a similar way. Gives me a little heart palpitation every time. Yah I am a wuss but I strongly suggest that any parent looking to show this movie to their kids, read them the book instead or watch it first to make certain that they approve of the content. Not everyone finds it as disturbing as I did but we are out there ;0)\"\n",
            " b'Maybe one of the most entertaining Ninja-movies ever made. A hard-hitting action movie with lots of gore and slow motion (eehaaa!). Made in \\xc2\\xb483 and still the greatest swedish action movie made so far! And we can hardly wait to see the upcoming sequel, Ninja mission 2000 - The legacy of Markov!'\n",
            " b\"Remade today, this film would be a very creepy, very disturbing dark comedy. Stalking, obsession, and a web of lies and manipulations are given a 1948 gloss of aren't-they-cute harmlessness. Drake plays the stalker, an unabashed user of people, alternately pathetic and manipulative, Grant plays the stalking victim, alternately angry and oblivious.<br /><br />Vastly disturbing; I haven't been able to look at classic romances with the same suspension of disbelief since.<br /><br />\"\n",
            " b\"BASEketball is awesome! It's hilarious and so damned funny that you will wet your pants laughing. I have seen it so many times I have stopped counting. But everytime it gets funnier.<br /><br />Trust me on this one...BASEketball is a surefire hit and I loved it and will continue to love it. I hope one day there will be a special edition DVD brought out!!!<br /><br />Ten Thumbs Up!!!\"\n",
            " b'What a poor image of Professional Police Officers is displayed on the Television in the watching of this alleged Reality show. One can only hope that the actual reasonable suspicion that leads to probable cause that leads to the totality of the circumstances involved to make a \"stop\" , then the \"Pat Down\" of the outside of one\\'s Garment, then to be able to articulate why the officer went into someone\\'s pocket and retrieved contraband, was cut out of the scenes, because if it wasn\\'t, the arrest in most places are going to be tossed, should they even get passed a supervisor. A report of a warrant over the radio does not constitute the actual existence of the warrant unless the person dispatching has the original warrant in hand. If the dispatcher is reading from a computer printout, it is good enough for an arrest, but it does not necessarily mean the warrant is still in effect. Since I haven\\'t seen a Dis-claimer from CBS (I may have missed it), CBS could be in trouble.'\n",
            " b'Generically speaking, Fay Grim is a highly entertaining thriller featuring two of the most inexorably enjoyable names in American movies, unshakably beautiful and gracefully spunky Parker Posey and endlessly charismatic and unavoidably hilarious Jeff Goldblum. They have many scenes in the first half of the film in which we see these two insatiable presences volleying off of each other, even radiating with charm when Goldblum rolls off Hartley\\'s shamelessly epic info-dumps. Nevertheless, if one were to deconstruct Fay Grim, one would see many instances in which countless scenes could\\'ve been squeezed for much more benefit than they have resulted in being.<br /><br />This sort of filmed in-joke is the sequel to Hal Hartley\\'s Henry Fool, which was made ten years earlier. It has title character Posey forced by CIA agent Goldblum to track down the notebooks that were the precious possessions of her missing fugitive husband, the predecessor\\'s titular anti-hero. Available within them is information that could concede the safety of the United States. Fay first makes for Paris to get a hold of them but becomes engulfed in a bona fide celebration of espionage clich\\xc3\\xa9s featuring everything from car bombs to ambiguous helpers to Following the Girl to double-crosses to triple-crosses.<br /><br />The primary appeal of it all for me is that it\\'s such a novel approach to the sequel of a movie about a garbageman and a struggling novelist in a small town. In the original Henry Fool, Posey played a simple woman leading a very simple life. Hartley\\'s talents do not reach the heights of many of the other independent newbies from the 1990s, but I do admire his wild creativity in making an inadvertent Nearne sister out of her, giving her a terrific predicament, as he did to her character\\'s brother, played by James Urbaniak, in Henry Fool, as she is trapped between whether or not she may still love her overwhelming refugee husband and the problematic but forceful plans of Goldblum.<br /><br />Hartley, however, is simply riding on that fragmentary idea. His plot, though complex and labyrinthine, true to the form of the spy film, it seems as if to be entirely capricious. The reason I was not bored was mostly due to the pace at which the story unfolds, not to mention the presence of Posey and Goldblum. The problem with the remainder of Hartley\\'s cast is that I cannot seem to become fond of the rest of them. It has nothing to do with how obscure they are compared to the relative star power of the two said charm masters, but with how they don\\'t seem to hold their own alongside them, though Saffron Burrows certainly comes close. Most of the scenes not involving Posey or Goldblum are far too light on their feet, stringing us along with info-dumps we have no choice but to listen to or else be totally lost in the ensuing sequence of scenes. They are shot almost entirely in tiled angles, as if Hartley is compensating for that implacable feeling of a lack of material.<br /><br />Liam Aiken, however, playing the now teenage son of Fay and Henry, has a certain allure about him, seeming wise beyond his years, certainly much wiser than any of the adult characters. Perhaps Hartley intended that, or maybe it\\'s simply Aiken\\'s presence. The problem with a Hartley film is that you never quite know what was intended and what just happens to be there. As Scorsese said, \"Cinema is a matter of what\\'s in the frame and what\\'s out.\" One has to be able to trust that what we see is a conscious decision by the filmmaker to remain in the finished film.'\n",
            " b'I know that movies aren\\'t necessarily supposed to mirror reality, but this one got on my nerves. It perpetuates ignorant stereotypes about \"psychological trauma\" and mental illness. The \"psycho mom\" thing has been done too many times before (and usually done better) and much of the rest of the plot is far-fetched as well. The acting was not horrible but nothing to rave about.<br /><br />One highlight: I am a long-time fan of General Hospital and it was a trip to see one of the roles played by former GH regular, Jon Lindstrom.<br /><br />Anyway, if you can overlook the bogus psychoanalytical part of it, in the same way a person must suspend reality / judgment when watching a lot of movies, then this movie might be tolerable. If you have nothing better to do and fairly low standards.<br /><br />I\\'m sorry I spent my time watching it.'\n",
            " b\"OK so there's nudity, but hey, there's free porn on the internet for whomever likes it. And its just silly how they forced tits into every frame. I mean i was embarrassed, not from the nudity but from the far-fetchessness of the producers/writers of this piece of crap.<br /><br />The movie is NOT funny at all, its just extremely predictable all the time. There is no plot, no dramatic content at all. This is way waay worse then the other pie-films and they arnt that great either:) If you're really drunk or maybe a 13 year old buy who are really obsessed with tits this might be acceptable, otherwise not. <br /><br />May it forever roth align with crap of the same magnitude with regards Erik the questmaster flash MC\"\n",
            " b'I feel like I have some uber-rare disease that no one has heard of and I have finally come across a support group on the net! I finally found this title by asking for an answer on an \"experts\" site on the web. I too, saw this movie in my youth and was struck by the atmosphere and especially the ending. I have never forgotten it and have never seen it since. No one I know saw the film and I had almost given up on ever finding it\\'s title. Alas, even knowing the name, I shall probably never see the film again as it is impossible to find commercially. Small steps...<br /><br />G'\n",
            " b\"I generally LIKE watching Burt Lancaster's films--especially when he is needed to go nuts with his imposing screen presence like in Elmer Gantry. However, his greatest strength, his magnetism, was occasionally also his greatest weakness as he rarely, if ever, underplayed ANYTHING. And it is this lack of subtlety that really hinders The Rainmaker. Now I understand that his character was meant to be a sort of showman but how Katherine Hepburn could fall under his spell is completely inexplicable. She is supposed to be smart but doesn't seem so when Lancaster's blarney is being thrown about the screen! In addition to this, the story is perhaps one of the most stagy looking films I have ever seen and it is way too obvious that this is a movie based on a play. It just looks like it was mostly filmed in a sound stage instead of in the great wide open West like it was supposed to be.<br /><br />Overall, a very overrated film.\"\n",
            " b\"Clara Lago is wonderful as the title character of the film, essentially a film about a Spanish/American girl who moves to Spain with her mom at the time of the Spanish Civil War. It turns out, the mother goes home to die, and she is left with her grandfather. She also makes friends and experiences much in a short time. Tomiche (Juan Jose Ballestra) is at first a nuisance to her then they become close. The film is shot beautifully, bathed in soft colors mostly. Carol yearns for her dad, who is a pilot in the war, and you can feel the love sher has for him. While the war itself is kind of taken a back seat in this film, it envelops the character's lives. I think you'll like it. See it especially for Clara Lago, who does a great job as Carol. She is definitely one to watch.\"\n",
            " b'I know, that\\'s not what you expect from a film with this sort of<br /><br />lineage- it\\'s a direct descendant of The Best Years of Our Lives<br /><br />and The Men... films dealing with men who are in the hospital<br /><br />dealing with tragic circumstances. But this film is full of wonderful<br /><br />surprises and performances. It features stellar performances from<br /><br />Eric Stoltz and Helen Hunt (including a rather risque nude scene)<br /><br />and Wesley Snipes and William Forsythe. As Emanuel Levy wrote<br /><br />in his book Cinema of Outsiders (about the Independent film<br /><br />movement) \"The Waterdance is coherant, attentive to detail, and<br /><br />unsentimental with a wicked down to earth humor- it\\' s at once<br /><br />funny and sad, and the entire cast is impressive.\" I was<br /><br />extraordinarily moved by this film, it\\'s hard hitting yes, but also has<br /><br />very tender moments and laugh out loud moments. A rare gem.'\n",
            " b'To call a movie like \"Thinner\" bad is like calling the earth round or Pauly Shore un-talented. No news, but how they got that way is what people want to know.<br /><br />As far as this movie.... The book was good, even if it was a little derivative of other stories from the \"be careful what you wish for\" genre. Burke plays an overweight lawyer who kills the daughter of a gypsy and is cursed by her father (Constantine from TV\\'s \"Room 222\") to several pounds a day. <br /><br />Like I said, it starts out good, but why involve the mobster (Mantegna)? Why fire automatic weapons so much? Why turn it into something so heavily dependent on FX? I thought it would have been much more effective if it focused more on the subtle ramifications of weight loss crazes, diseases, death, gypsy lore and such. <br /><br />But no, it\\'s not to be. Remember, this is Stephen King we\\'re talking about.<br /><br />And the ending... almost the same as the book, but a little too talky. In fact the whole movie talks too much, feeling it has to explain every plot turn to us. Not that I expected \"The Dead Zone\", but I could have done without another \"Pet Sematary\", thanks anyway.<br /><br />One star for at least trying to do a halfway decent makeup job. However, the rest of the movie is left to be... say it with me... \"Thinner\".'\n",
            " b'Just the ultimate masterpiece in my opinion. Every line, every phrase, every picture is exactly in place and Lindsay Crouse and Joe Mantegna are just THE cool shrink and the sleazy con-man, so well cast. 10 out of 10!'\n",
            " b'I saw this film on TV many years ago and I saw this film when I got this on tape. I thought that this was reasonably well done. It was not the best of all movies, but it was good enough. The movie has enough talent to inspire many people, especially younger kids. The acting was good, with Danny Glover leading the cast. The plot line was not very believable, but the script was well written. This movie can also be the interest of avid baseball fans. It does not directly apply to a action-packed sports movie. It directly applies to a nice film that you can watch with your family and learn some messages that are hidden in this film. Overall, the film was good, but not great. I give this a movie a 7/10.'\n",
            " b'I just saw this movie for the first time ever and I liked it. Her dancing was very entertaining. I read somewhere that she got the part in this movie because she knew how to dance. The scenery was great too. Yvonne is such a talented woman and beautiful. WE laughed at the silly kissing scenes, but that is what is great about old movies! I grew up with her on The Munsters and I am enjoying watching her in her earlier movies. They may not all be the best out there but still worth watching to see her act and sing. I am slowly purchasing all her movies and watching them as I receive them. I have a large collection of her memorabilia.'], shape=(32,), dtype=string)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "text_vectorization.adapt(text_only_train_ds)"
      ],
      "metadata": {
        "id": "pISPdw3s_M6J"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Prepare processed versions of our training, validation, and test dataset.\n",
        "\n",
        "binary_1gram_train_ds = train_ds.map(\n",
        " lambda x, y: (text_vectorization(x), y),\n",
        " num_parallel_calls=4)\n",
        "binary_1gram_val_ds = val_ds.map(\n",
        " lambda x, y: (text_vectorization(x), y),\n",
        " num_parallel_calls=4)\n",
        "binary_1gram_test_ds = test_ds.map(\n",
        " lambda x, y: (text_vectorization(x), y),\n",
        " num_parallel_calls=4)"
      ],
      "metadata": {
        "id": "Q8WiwTTj_UmD"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "for inputs, targets in binary_1gram_train_ds:\n",
        "    print(\"inputs.shape:\", inputs.shape)\n",
        "    print(\"inputs.dtype:\", inputs.dtype)\n",
        "    print(\"targets.shape:\", targets.shape)\n",
        "    print(\"targets.dtype:\", targets.dtype)\n",
        "    print(\"inputs[0]:\", inputs[0])\n",
        "    print(\"targets[0]:\", targets[0])\n",
        "    break\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ztTj1jK2AKpz",
        "outputId": "036414d1-a0db-454d-b77d-51581c698ee2"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "inputs.shape: (32, 20000)\n",
            "inputs.dtype: <dtype: 'float32'>\n",
            "targets.shape: (32,)\n",
            "targets.dtype: <dtype: 'int32'>\n",
            "inputs[0]: tf.Tensor([1. 1. 1. ... 0. 0. 0.], shape=(20000,), dtype=float32)\n",
            "targets[0]: tf.Tensor(1, shape=(), dtype=int32)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from tensorflow import keras\n",
        "from tensorflow.keras import layers\n",
        "\n",
        "def get_model(max_tokens=20000, hidden_dim=16):\n",
        " inputs = keras.Input(shape=(max_tokens,))\n",
        " x = layers.Dense(hidden_dim, activation=\"relu\")(inputs)\n",
        " x = layers.Dropout(0.5)(x)\n",
        " outputs = layers.Dense(1, activation=\"sigmoid\")(x)\n",
        " model = keras.Model(inputs, outputs)\n",
        " model.compile(optimizer=\"rmsprop\",\n",
        " loss=\"binary_crossentropy\",\n",
        " metrics=[\"accuracy\"])\n",
        " return model"
      ],
      "metadata": {
        "id": "y5v3bqQQAcZo"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model = get_model()\n",
        "\n",
        "model.summary()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0Of0Zdo9Apxg",
        "outputId": "34c1f83b-c7ca-46a1-9f90-19332e13a5c9"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"model\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " input_1 (InputLayer)        [(None, 20000)]           0         \n",
            "                                                                 \n",
            " dense (Dense)               (None, 16)                320016    \n",
            "                                                                 \n",
            " dropout (Dropout)           (None, 16)                0         \n",
            "                                                                 \n",
            " dense_1 (Dense)             (None, 1)                 17        \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 320033 (1.22 MB)\n",
            "Trainable params: 320033 (1.22 MB)\n",
            "Non-trainable params: 0 (0.00 Byte)\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "callbacks = [\n",
        " keras.callbacks.ModelCheckpoint(\"binary_1gram.keras\",\n",
        " save_best_only=True)\n",
        "]\n",
        "\n",
        "\"\"\"\n",
        "We call cache() on the\n",
        "datasets to cache them in\n",
        "memory: this way, we will\n",
        "only do the preprocessing\n",
        "once, during the first\n",
        "epoch, and we’ll reuse the\n",
        "preprocessed texts for the\n",
        "following epochs. This can\n",
        "only be done if the data\n",
        "is small enough to fit in\n",
        "memory\n",
        "\"\"\"\n",
        "\n",
        "history = model.fit(binary_1gram_train_ds.cache(),\n",
        "validation_data=binary_1gram_val_ds.cache(),\n",
        " epochs=10,\n",
        " callbacks=callbacks)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RmbrMb-2AtuG",
        "outputId": "31437804-6054-4562-9405-12d7201524be"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "625/625 [==============================] - 12s 17ms/step - loss: 0.2316 - accuracy: 0.9213 - val_loss: 0.3004 - val_accuracy: 0.8934\n",
            "Epoch 2/10\n",
            "625/625 [==============================] - 3s 5ms/step - loss: 0.2176 - accuracy: 0.9298 - val_loss: 0.3131 - val_accuracy: 0.8954\n",
            "Epoch 3/10\n",
            "625/625 [==============================] - 3s 5ms/step - loss: 0.2124 - accuracy: 0.9305 - val_loss: 0.3259 - val_accuracy: 0.8852\n",
            "Epoch 4/10\n",
            "625/625 [==============================] - 3s 5ms/step - loss: 0.2058 - accuracy: 0.9346 - val_loss: 0.3350 - val_accuracy: 0.8916\n",
            "Epoch 5/10\n",
            "625/625 [==============================] - 4s 6ms/step - loss: 0.2050 - accuracy: 0.9361 - val_loss: 0.3439 - val_accuracy: 0.8908\n",
            "Epoch 6/10\n",
            "625/625 [==============================] - 3s 5ms/step - loss: 0.2039 - accuracy: 0.9387 - val_loss: 0.3562 - val_accuracy: 0.8848\n",
            "Epoch 7/10\n",
            "625/625 [==============================] - 3s 5ms/step - loss: 0.2109 - accuracy: 0.9390 - val_loss: 0.3619 - val_accuracy: 0.8866\n",
            "Epoch 8/10\n",
            "625/625 [==============================] - 3s 5ms/step - loss: 0.1971 - accuracy: 0.9416 - val_loss: 0.3617 - val_accuracy: 0.8870\n",
            "Epoch 9/10\n",
            "625/625 [==============================] - 3s 5ms/step - loss: 0.2036 - accuracy: 0.9384 - val_loss: 0.3655 - val_accuracy: 0.8864\n",
            "Epoch 10/10\n",
            "625/625 [==============================] - 3s 5ms/step - loss: 0.1992 - accuracy: 0.9410 - val_loss: 0.3738 - val_accuracy: 0.8880\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import matplotlib.pyplot as plt\n",
        "accuracy = history.history[\"accuracy\"]\n",
        "val_accuracy = history.history[\"val_accuracy\"]\n",
        "loss = history.history[\"loss\"]\n",
        "val_loss = history.history[\"val_loss\"]\n",
        "epochs = range(1, len(accuracy) + 1)\n",
        "plt.plot(epochs, accuracy, \"bo\", label=\"Training accuracy\")\n",
        "plt.plot(epochs, val_accuracy, \"b\", label=\"Validation accuracy\")\n",
        "plt.title(\"Training and validation accuracy\")\n",
        "plt.legend()\n",
        "plt.figure()\n",
        "plt.plot(epochs, loss, \"bo\", label=\"Training loss\")\n",
        "plt.plot(epochs, val_loss, \"b\", label=\"Validation loss\")\n",
        "plt.title(\"Training and validation loss\")\n",
        "plt.legend()\n",
        "plt.show()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 887
        },
        "id": "Ych_WoLlBie-",
        "outputId": "5ef52a9a-e5b7-405f-d1db-21ec2026bd6a"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiwAAAGzCAYAAAAMr0ziAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABXqklEQVR4nO3deVgT1/4/8HcASUA2FWRXFKm7oChWrUqVe1FbL65VaxW11bai1VLrUnf9Knazrtdqb6t1t1a0i62KVK37jrvWfaGCYhUEZEvO74/5EQkESBDIQN6v58mjmZzMfBIC887MOWcUQggBIiIiIhmzMHUBRERERMVhYCEiIiLZY2AhIiIi2WNgISIiItljYCEiIiLZY2AhIiIi2WNgISIiItljYCEiIiLZY2AhIiIi2WNgIbM0ZMgQ+Pj4lOi5M2bMgEKhKN2CZObWrVtQKBRYtWpVuW537969UCgU2Lt3r3aZoT+rsqrZx8cHQ4YMKdV1EpHxGFhIVhQKhUG3vDs0ohd16NAhzJgxA0+ePDF1KURUCCtTF0CU15o1a3Tur169GjExMQWWN2zY8IW2880330Cj0ZTouVOmTMHEiRNfaPtkuBf5WRnq0KFDmDlzJoYMGQInJyedx65cuQILC363IzI1BhaSlbfeekvn/pEjRxATE1NgeX7p6emwtbU1eDtVqlQpUX0AYGVlBSsr/uqUlxf5WZUGpVJp0u1XFGlpaahataqpy6BKjF8bqMIJDg5GkyZNcPLkSXTo0AG2trb45JNPAAA//fQTXnvtNXh4eECpVMLX1xezZ8+GWq3WWUf+fhG5/R+++OILrFixAr6+vlAqlWjVqhWOHz+u81x9fVgUCgVGjRqFbdu2oUmTJlAqlWjcuDF27NhRoP69e/eiZcuWUKlU8PX1xfLlyw3uF7N//3707dsXtWrVglKphLe3Nz788EM8e/aswOuzs7NDfHw8evToATs7O7i4uGDcuHEF3osnT55gyJAhcHR0hJOTE8LDww06NXLixAkoFAp8//33BR7buXMnFAoFfv31VwDA7du3MXLkSNSvXx82NjaoUaMG+vbti1u3bhW7HX19WAyt+ezZsxgyZAjq1q0LlUoFNzc3DBs2DI8ePdK2mTFjBj7++GMAQJ06dbSnHXNr09eH5caNG+jbty+qV68OW1tbvPzyy9i+fbtOm9z+OD/88APmzJkDLy8vqFQqdO7cGdeuXSv2dRvznj158gQffvghfHx8oFQq4eXlhcGDByMpKUnbJiMjAzNmzMBLL70ElUoFd3d39OrVC9evX9epN//pVn19g3I/X9evX0e3bt1gb2+PgQMHAjD8MwoAly9fxhtvvAEXFxfY2Nigfv36mDx5MgBgz549UCgU2Lp1a4HnrV+/HgqFAocPHy72faTKg18TqUJ69OgRunbtiv79++Ott96Cq6srAGDVqlWws7NDZGQk7Ozs8Mcff2DatGlISUnB559/Xux6169fj6dPn+Ldd9+FQqHAZ599hl69euHGjRvFftM/cOAAoqOjMXLkSNjb22PRokXo3bs37ty5gxo1agAATp8+jS5dusDd3R0zZ86EWq3GrFmz4OLiYtDr3rx5M9LT0/H++++jRo0aOHbsGBYvXox79+5h8+bNOm3VajVCQ0PRunVrfPHFF9i9eze+/PJL+Pr64v333wcACCEQFhaGAwcO4L333kPDhg2xdetWhIeHF1tLy5YtUbduXfzwww8F2m/atAnVqlVDaGgoAOD48eM4dOgQ+vfvDy8vL9y6dQvLli1DcHAwLl68aNTRMWNqjomJwY0bNzB06FC4ubnhwoULWLFiBS5cuIAjR45AoVCgV69e+Ouvv7BhwwZ89dVXcHZ2BoBCfyaJiYlo27Yt0tPT8cEHH6BGjRr4/vvv8Z///Ac//vgjevbsqdN+3rx5sLCwwLhx45CcnIzPPvsMAwcOxNGjR4t8nYa+Z6mpqWjfvj0uXbqEYcOGoUWLFkhKSsLPP/+Me/fuwdnZGWq1Gq+//jpiY2PRv39/jBkzBk+fPkVMTAzOnz8PX19fg9//XDk5OQgNDcUrr7yCL774QluPoZ/Rs2fPon379qhSpQpGjBgBHx8fXL9+Hb/88gvmzJmD4OBgeHt7Y926dQXe03Xr1sHX1xdt2rQxum6qwASRjEVERIj8H9OOHTsKAOLrr78u0D49Pb3AsnfffVfY2tqKjIwM7bLw8HBRu3Zt7f2bN28KAKJGjRrin3/+0S7/6aefBADxyy+/aJdNnz69QE0AhLW1tbh27Zp22ZkzZwQAsXjxYu2y7t27C1tbWxEfH69ddvXqVWFlZVVgnfroe31RUVFCoVCI27dv67w+AGLWrFk6bZs3by4CAwO197dt2yYAiM8++0y7LCcnR7Rv314AECtXriyynkmTJokqVarovGeZmZnCyclJDBs2rMi6Dx8+LACI1atXa5ft2bNHABB79uzReS15f1bG1Kxvuxs2bBAAxJ9//qld9vnnnwsA4ubNmwXa165dW4SHh2vvjx07VgAQ+/fv1y57+vSpqFOnjvDx8RFqtVrntTRs2FBkZmZq2y5cuFAAEOfOnSuwrbwMfc+mTZsmAIjo6OgC7TUajRBCiO+++04AEPPnzy+0jb73Xojnvxt539fcz9fEiRMNqlvfZ7RDhw7C3t5eZ1neeoSQPl9KpVI8efJEu+zBgwfCyspKTJ8+vcB2qHLjKSGqkJRKJYYOHVpguY2Njfb/T58+RVJSEtq3b4/09HRcvny52PX269cP1apV095v3749AOkUQHFCQkJ0vqk2a9YMDg4O2ueq1Wrs3r0bPXr0gIeHh7ZdvXr10LVr12LXD+i+vrS0NCQlJaFt27YQQuD06dMF2r/33ns699u3b6/zWn777TdYWVlpj7gAgKWlJUaPHm1QPf369UN2djaio6O1y3bt2oUnT56gX79+euvOzs7Go0ePUK9ePTg5OeHUqVMGbaskNefdbkZGBpKSkvDyyy8DgNHbzbv9oKAgvPLKK9pldnZ2GDFiBG7duoWLFy/qtB86dCisra219w39TBn6nm3ZsgX+/v4FjkIA0J5m3LJlC5ydnfW+Ry8yRD/vz0Bf3YV9Rh8+fIg///wTw4YNQ61atQqtZ/DgwcjMzMSPP/6oXbZp0ybk5OQU26+NKh8GFqqQPD09dXYCuS5cuICePXvC0dERDg4OcHFx0f5hS05OLna9+f945oaXx48fG/3c3OfnPvfBgwd49uwZ6tWrV6CdvmX63LlzB0OGDEH16tW1/VI6duwIoODrU6lUBU5r5K0HkPpJuLu7w87OTqdd/fr1DarH398fDRo0wKZNm7TLNm3aBGdnZ3Tq1Em77NmzZ5g2bRq8vb2hVCrh7OwMFxcXPHnyxKCfS17G1PzPP/9gzJgxcHV1hY2NDVxcXFCnTh0Ahn0eCtu+vm3ljly7ffu2zvKSfqYMfc+uX7+OJk2aFLmu69evo379+qXaWdzKygpeXl4FlhvyGc0Na8XV3aBBA7Rq1Qrr1q3TLlu3bh1efvllg39nqPJgHxaqkPJ+i8v15MkTdOzYEQ4ODpg1axZ8fX2hUqlw6tQpTJgwwaChsZaWlnqXCyHK9LmGUKvV+Ne//oV//vkHEyZMQIMGDVC1alXEx8djyJAhBV5fYfWUtn79+mHOnDlISkqCvb09fv75ZwwYMEBn5zh69GisXLkSY8eORZs2beDo6AiFQoH+/fuX6ZDlN954A4cOHcLHH3+MgIAA2NnZQaPRoEuXLmU+VDpXST8X5f2eFXakJX8n7VxKpbLAcG9jP6OGGDx4MMaMGYN79+4hMzMTR44cwZIlS4xeD1V8DCxUaezduxePHj1CdHQ0OnTooF1+8+ZNE1b1XM2aNaFSqfSOEDFk1Mi5c+fw119/4fvvv8fgwYO1y2NiYkpcU+3atREbG4vU1FSdIxZXrlwxeB39+vXDzJkzsWXLFri6uiIlJQX9+/fXafPjjz8iPDwcX375pXZZRkZGiSZqM7Tmx48fIzY2FjNnzsS0adO0y69evVpgncacFqldu7be9yf3lGPt2rUNXldRDH3PfH19cf78+SLX5evri6NHjyI7O7vQzuO5R37yrz//EaOiGPoZrVu3LgAUWzcA9O/fH5GRkdiwYQOePXuGKlWq6JxuJPPBU0JUaeR+k837zTUrKwv//e9/TVWSDktLS4SEhGDbtm34+++/tcuvXbuG33//3aDnA7qvTwiBhQsXlrimbt26IScnB8uWLdMuU6vVWLx4scHraNiwIZo2bYpNmzZh06ZNcHd31wmMubXnP6KwePHiQr+9l0bN+t4vAFiwYEGBdebOH2JIgOrWrRuOHTumM6Q2LS0NK1asgI+PDxo1amToSymSoe9Z7969cebMGb3Df3Of37t3byQlJek9MpHbpnbt2rC0tMSff/6p87gxvz+GfkZdXFzQoUMHfPfdd7hz547eenI5Ozuja9euWLt2LdatW4cuXbpoR3KReeERFqo02rZti2rVqiE8PBwffPABFAoF1qxZU2qnZErDjBkzsGvXLrRr1w7vv/8+1Go1lixZgiZNmiAuLq7I5zZo0AC+vr4YN24c4uPj4eDggC1bthjUv6Yw3bt3R7t27TBx4kTcunULjRo1QnR0tNH9O/r164dp06ZBpVLh7bffLnCq4PXXX8eaNWvg6OiIRo0a4fDhw9i9e7d2uHdZ1Ozg4IAOHTrgs88+Q3Z2Njw9PbFr1y69R9wCAwMBAJMnT0b//v1RpUoVdO/eXe9EaBMnTsSGDRvQtWtXfPDBB6hevTq+//573Lx5E1u2bCm1WXENfc8+/vhj/Pjjj+jbty+GDRuGwMBA/PPPP/j555/x9ddfw9/fH4MHD8bq1asRGRmJY8eOoX379khLS8Pu3bsxcuRIhIWFwdHREX379sXixYuhUCjg6+uLX3/9FQ8ePDC4ZmM+o4sWLcIrr7yCFi1aYMSIEahTpw5u3bqF7du3F/hdGDx4MPr06QMAmD17tvFvJlUO5T4uicgIhQ1rbty4sd72Bw8eFC+//LKwsbERHh4eYvz48WLnzp3FDpXNHbr5+eefF1gnAJ0hlIUNa46IiCjw3PxDYoUQIjY2VjRv3lxYW1sLX19f8b///U989NFHQqVSFfIuPHfx4kUREhIi7OzshLOzsxg+fLh2+HT+YadVq1Yt8Hx9tT969EgMGjRIODg4CEdHRzFo0CBx+vRpg4Y157p69aoAIACIAwcOFHj88ePHYujQocLZ2VnY2dmJ0NBQcfny5QLvjyHDmo2p+d69e6Jnz57CyclJODo6ir59+4q///67wM9UCCFmz54tPD09hYWFhc4QZ30/w+vXr4s+ffoIJycnoVKpRFBQkPj111912uS+ls2bN+ss1zdMWB9D37Pc92PUqFHC09NTWFtbCy8vLxEeHi6SkpK0bdLT08XkyZNFnTp1RJUqVYSbm5vo06ePuH79urbNw4cPRe/evYWtra2oVq2aePfdd8X58+cN/nwJYfhnVAghzp8/r/35qFQqUb9+fTF16tQC68zMzBTVqlUTjo6O4tmzZ0W+b1R5KYSQ0ddPIjPVo0cPXLhwQW//CiJzl5OTAw8PD3Tv3h3ffvutqcshE2EfFqJyln+K8qtXr+K3335DcHCwaQoikrlt27bh4cOHOh15yfzwCAtROXN3d9de3+b27dtYtmwZMjMzcfr0afj5+Zm6PCLZOHr0KM6ePYvZs2fD2dm5xJP9UeXATrdE5axLly7YsGEDEhISoFQq0aZNG8ydO5dhhSifZcuWYe3atQgICNC5+CKZJx5hISIiItljHxYiIiKSPQYWIiIikr1K04dFo9Hg77//hr29/QtdfZSIiIjKjxACT58+hYeHR5ETL1aawPL333/D29vb1GUQERFRCdy9e1fvFcBzVZrAYm9vD0B6wQ4ODiauhoiIiAyRkpICb29v7X68MJUmsOSeBnJwcGBgISIiqmCK687BTrdEREQkewwsREREJHsMLERERCR7laYPiyHUajWys7NNXQZRqbO0tISVlRWH9BNRpWU2gSU1NRX37t0Dr0RAlZWtrS3c3d1hbW1t6lKIiEqdWQQWtVqNe/fuwdbWFi4uLvwWSpWKEAJZWVl4+PAhbt68CT8/vyInXyIiqojMIrBkZ2dDCAEXFxfY2NiYuhyiUmdjY4MqVarg9u3byMrKgkqlMnVJRESlyqy+hvHIClVmPKpCRJWZWRxhISKiikOtBvbvB+7fB9zdgfbtAUtLU1dFpsbAQkREshEdDYwZA9y793yZlxewcCHQq5fp6iLT4zFkI6jVwN69wIYN0r9qtakrMp6Pjw8WLFhgcPu9e/dCoVDgyZMnZVYTEREghZU+fXTDCgDEx0vLo6NNUxfJAwOLgaKjAR8f4NVXgTfflP718Sm7XyCFQlHkbcaMGSVa7/HjxzFixAiD27dt2xb379+Ho6NjibZHRGQItVo6sqJv5oncZWPHVswvilQ6eErIALmpP/8vUm7q//HH0j9Uef/+fe3/N23ahGnTpuHKlSvaZXZ2dtr/CyGgVqthZVX8j9PFxcWoOqytreHm5mbUcyqLrKwszmlCVE727y94ZCUvIYC7d6V2wcHlVhZBPn2KeISlGKZK/W5ubtqbo6MjFAqF9v7ly5dhb2+P33//HYGBgVAqlThw4ACuX7+OsLAwuLq6ws7ODq1atcLu3bt11pv/lJBCocD//vc/9OzZE7a2tvDz88PPP/+sfTz/KaFVq1bByckJO3fuRMOGDWFnZ4cuXbroBKycnBx88MEHcHJyQo0aNTBhwgSEh4ejR48ehb7eR48eYcCAAfD09IStrS2aNm2KDRs26LTRaDT47LPPUK9ePSiVStSqVQtz5szRPn7v3j0MGDAA1atXR9WqVdGyZUscPXoUADBkyJAC2x87diyC8/zlCw4OxqhRozB27Fg4OzsjNDQUADB//nw0bdoUVatWhbe3N0aOHInU1FSddR08eBDBwcGwtbVFtWrVEBoaisePH2P16tWoUaMGMjMzddr36NEDgwYNKvT9IDI3ef6ElEo7Kh3lfXahKAwsxTAm9Ze3iRMnYt68ebh06RKaNWuG1NRUdOvWDbGxsTh9+jS6dOmC7t27486dO0WuZ+bMmXjjjTdw9uxZdOvWDQMHDsQ///xTaPv09HR88cUXWLNmDf7880/cuXMH48aN0z7+6aefYt26dVi5ciUOHjyIlJQUbNu2rcgaMjIyEBgYiO3bt+P8+fMYMWIEBg0ahGPHjmnbTJo0CfPmzcPUqVNx8eJFrF+/Hq6urgCkmYw7duyI+Ph4/Pzzzzhz5gzGjx8PjUZjwDv53Pfffw9ra2scPHgQX3/9NQBpuPCiRYtw4cIFfP/99/jjjz8wfvx47XPi4uLQuXNnNGrUCIcPH8aBAwfQvXt3qNVq9O3bF2q1WicEPnjwANu3b8ewYcOMqo2oMnN3L9129OJk16dIVBLJyckCgEhOTi7w2LNnz8TFixfFs2fPjF7v+vVCSLGk6Nv69aXxKvRbuXKlcHR01N7fs2ePACC2bdtW7HMbN24sFi9erL1fu3Zt8dVXX2nvAxBTpkzR3k9NTRUAxO+//66zrcePH2trASCuXbumfc7SpUuFq6ur9r6rq6v4/PPPtfdzcnJErVq1RFhYmKEvWQghxGuvvSY++ugjIYQQKSkpQqlUim+++UZv2+XLlwt7e3vx6NEjvY+Hh4cX2P6YMWNEx44dtfc7duwomjdvXmxdmzdvFjVq1NDeHzBggGjXrl2h7d9//33RtWtX7f0vv/xS1K1bV2g0mmK3ZYwX+ZwTmVpOjhBeXkIoFPr/xioUQnh7S+2o7OX+PArb55Xmz6Oo/XdePMJSDDmn/pYtW+rcT01Nxbhx49CwYUM4OTnBzs4Oly5dKvYIS7NmzbT/r1q1KhwcHPDgwYNC29va2sLX11d7393dXds+OTkZiYmJCAoK0j5uaWmJwMDAImtQq9WYPXs2mjZtiurVq8POzg47d+7U1n7p0iVkZmaic+fOep8fFxeH5s2bo3r16kVupzj66ty9ezc6d+4MT09P2NvbY9CgQXj06BHS09O12y6sLgAYPnw4du3ahfj4eADSabUhQ4ZwIkOiPCwtpaHLAJD/VyP3/oIFnI+lvMjx7AIDSzHat5fmAChs36JQAN7eUrvyVrVqVZ3748aNw9atWzF37lzs378fcXFxaNq0KbKysopcT5UqVXTuKxSKIk+l6GsvXvCikp9//jkWLlyICRMmYM+ePYiLi0NoaKi29uIuqVDc4xYWFgVq1Hfl7vzv6a1bt/D666+jWbNm2LJlC06ePImlS5cCgMG1NW/eHP7+/li9ejVOnjyJCxcuYMiQIUU+h8gc9eolDWLw9NRd7uVVNoMbqHBy7FPEwFKMipT6Dx48iCFDhqBnz55o2rQp3NzccOvWrXKtwdHREa6urjh+/Lh2mVqtxqlTp4p83sGDBxEWFoa33noL/v7+qFu3Lv766y/t435+frCxsUFsbKze5zdr1gxxcXGF9r1xcXHR6RgMSEdGinPy5EloNBp8+eWXePnll/HSSy/h77//LrDtwurK9c4772DVqlVYuXIlQkJC4O3tXey2icxRr17ArVvAnj3A+vXSvzdvMqyUNzmeXShRYFm6dCl8fHygUqnQunVrnY6R+WVnZ2PWrFnw9fWFSqWCv78/duzYUWj7efPmQaFQYOzYsSUprUxUlNTv5+eH6OhoxMXF4cyZM3jzzTeN7nRaGkaPHo2oqCj89NNPuHLlCsaMGYPHjx8XeQrEz88PMTExOHToEC5duoR3330XiYmJ2sdVKhUmTJiA8ePHY/Xq1bh+/TqOHDmCb7/9FgAwYMAAuLm5oUePHjh48CBu3LiBLVu24PDhwwCATp064cSJE1i9ejWuXr2K6dOn4/z588W+lnr16iE7OxuLFy/GjRs3sGbNGm1n3FyTJk3C8ePHMXLkSJw9exaXL1/GsmXLkJSUpG3z5ptv4t69e/jmm2/Y2ZaoGJaW0tDlAQOkf+XwhdDcyPHsgtGBZdOmTYiMjMT06dNx6tQp+Pv7IzQ0tNA+D1OmTMHy5cuxePFiXLx4Ee+99x569uyJ06dPF2h7/PhxLF++XKdPhVxUhNQ/f/58VKtWDW3btkX37t0RGhqKFi1alHsdEyZMwIABAzB48GC0adMGdnZ2CA0NLfIKwlOmTEGLFi0QGhqK4OBgbfjIa+rUqfjoo48wbdo0NGzYEP369dN+7qytrbFr1y7UrFkT3bp1Q9OmTTFv3jxY/v+/dKGhoZg6dSrGjx+PVq1a4enTpxg8eHCxr8Xf3x/z58/Hp59+iiZNmmDdunWIiorSafPSSy9h165dOHPmDIKCgtCmTRv89NNPOvPiODo6onfv3rCzsytyeDcRkRzI8uyCsb15g4KCREREhPa+Wq0WHh4eIioqSm97d3d3sWTJEp1lvXr1EgMHDtRZ9vTpU+Hn5ydiYmJEx44dxZgxY4yqq6xGCdGLU6vV4qWXXtIZjWSOOnXqJEaPHl1m6+fnnIhK25YtBUcLeXtLy0tLmYwSysrKwsmTJxESEqJdZmFhgZCQEO2h9/wyMzMLfLO2sbHBgQMHdJZFRETgtdde01l3UTIzM5GSkqJzI3m4ffs2vvnmG/z11184d+4c3n//fdy8eRNvvvmmqUszicePH2Pr1q3Yu3cvIiIiTF0OEZHB5HR2waip+ZOSkqBWq7WTdeVydXXF5cuX9T4nNDQU8+fPR4cOHeDr64vY2FhER0dDnWdq2I0bN+LUqVM6HTWLExUVhZkzZxpTPpUTCwsLrFq1CuPGjYMQAk2aNMHu3bvRsGFDU5dmEs2bN8fjx4/x6aefon79+qYuh4jIKLl9ikytzK8ltHDhQgwfPhwNGjSAQqGAr68vhg4diu+++w4AcPfuXYwZMwYxMTFF9nHIb9KkSYiMjNTeT0lJ4cgLmfD29sbBgwdNXYZslPdILTJfcrnmC1FZMOqUkLOzMywtLXVGbwBAYmJioRfIc3FxwbZt25CWlobbt2/j8uXLsLOzQ926dQFIw0YfPHiAFi1awMrKClZWVti3bx8WLVoEKysrnSMxeSmVSjg4OOjciIjMlZyu+UJUFowKLNbW1ggMDNSZc0Kj0SA2NhZt2rQp8rkqlQqenp7IycnBli1bEBYWBgDo3Lkzzp07h7i4OO2tZcuWGDhwIOLi4rSjPIiIyopaDezdC2zYIP1b2hczLWuyu+YLURkw+pRQZGQkwsPD0bJlSwQFBWHBggVIS0vD0KFDAQCDBw+Gp6endujn0aNHER8fj4CAAMTHx2PGjBnQaDTai8fZ29ujSZMmOtuoWrUqatSoUWA5EVFpi46Wrsied2fv5SUN6ZTTtAWFKe6K8gqFdEX5sDCeHipvPEVXuowOLP369cPDhw8xbdo0JCQkICAgADt27NB2xL1z5w4sLJ4fuMnIyMCUKVNw48YN2NnZoVu3blizZg2cnJxK7UUQEZVE7pGJ/Dv73CMTcpoYsjDGXPNFDh0nzUVFD8JypBDiBS8CIxMpKSlwdHREcnJygf4sGRkZuHnzJurUqWNUx16iioSfc+Oo1VIfj8J29gqFtIO5eVPe34o3bJD6rBRn/Xpp5lgqe4UF4dwJ1ypCEC5PRe2/8+K1hIjILMnxarQlIcdrvpiz4k7RAdIpuorWT0oOGFgqueDgYJ3rMvn4+GDBggVFPkehUGDbtm0vvO3SWg9RWZDj1WhLQo7XfDFnlSUIyxEDi0x1794dXbp00fvY/v37oVAocPbsWaPXe/z4cYwYMeJFy9MxY8YMBAQEFFh+//59dO3atVS3RVRaKsuRCVle88WMVZYgLEcMLDL19ttvIyYmBvf0RPWVK1eiZcuWJbpIpIuLC2xtbUujxGK5ublBqVSWy7bkJCsry9QlkAEq05GJinJFeXNQWYKwHJllYBECSEszzc3QLs6vv/46XFxcsGrVKp3lqamp2Lx5M95++208evQIAwYMgKenJ2xtbdG0aVNs2LChyPXmPyV09epVdOjQASqVCo0aNUJMTEyB50yYMAEvvfQSbG1tUbduXUydOhXZ2dkAgFWrVmHmzJk4c+YMFAoFFAqFtub8p4TOnTuHTp06wcbGBjVq1MCIESOQmpqqfXzIkCHo0aMHvvjiC7i7u6NGjRqIiIjQbkuf69evIywsDK6urrCzs0OrVq2we/dunTaZmZmYMGECvL29oVQqUa9ePXz77bfaxy9cuIDXX38dDg4OsLe3R/v27XH9+nUABU+pAUCPHj0wZMgQnfd09uzZGDx4MBwcHLRHsIp633L98ssvaNWqFVQqFZydndGzZ08AwKxZs/QO6w8ICMDUqVMLfT/IcJXtyIScrvlizipTEJabMp+aX47S0wE7O9NsOzUVqFq1+HZWVlYYPHgwVq1ahcmTJ0Px/z/9mzdvhlqtxoABA5CamorAwEBMmDABDg4O2L59OwYNGgRfX18EBQUVuw2NRoNevXrB1dUVR48eRXJycoGdMyDNlbNq1Sp4eHjg3LlzGD58OOzt7TF+/Hj069cP58+fx44dO7RBwdHRscA60tLSEBoaijZt2uD48eN48OAB3nnnHYwaNUonlO3Zswfu7u7Ys2cPrl27hn79+iEgIADDhw8v5P1MRbdu3TBnzhwolUqsXr0a3bt3x5UrV1CrVi0A0txAhw8fxqJFi+Dv74+bN28iKSkJABAfH48OHTogODgYf/zxBxwcHHDw4EHk5OQU+/7l9cUXX2DatGmYPn26Qe8bAGzfvh09e/bE5MmTsXr1amRlZeG3334DAAwbNgwzZ87E8ePH0apVKwDA6dOncfbsWURzFrBSk3tkQt/w0wULKt7OXi7XfDFnuUG4Tx8pnOT9kloRg7CslN4Fok2rqMtTP3v2TFy8eFE8e/ZMCCFEaqrupbLL85aaavhrunTpkgAg9uzZo13Wvn178dZbbxX6nNdee0189NFH2vsdO3YUY8aM0d6vXbu2+Oqrr4QQQuzcuVNYWVmJ+Ph47eO///67ACC2bt1a6DY+//xzERgYqL0/ffp04e/vX6Bd3vWsWLFCVKtWTaTmeQO2b98uLCwsREJCghBCiPDwcFG7dm2Rk5OjbdO3b1/Rr1+/QmvRp3HjxmLx4sVCCCGuXLkiAIiYmBi9bSdNmiTq1KkjsrKy9D6e//0TQoiwsDARHh6uvV+7dm3Ro0ePYuvK/761adNGDBw4sND2Xbt2Fe+//772/ujRo0VwcHCh7fN/zslwOTlC7NkjxPr10r95PoJEJbJlixBeXrp//729peWkq6j9d15meYTF1lY60mGqbRuqQYMGaNu2Lb777jsEBwfj2rVr2L9/P2bNmgUAUKvVmDt3Ln744QfEx8cjKysLmZmZBvdRuXTpEry9veHh4aFdpu8SC5s2bcKiRYtw/fp1pKamIicnx+hrN126dAn+/v6omufwUrt27aDRaHDlyhXtxIONGzfWuRyDu7s7zp07V+h6U1NTMWPGDGzfvh33799HTk4Onj17hjt37gCA9vIOHTt21Pv8uLg4tG/fHlWqVDHq9eTXsmXLAsuKe9/i4uIKPXIEAMOHD8ewYcMwf/58WFhYYP369fjqq69eqE7Sj0cmqLT16iXNLsyZbkuPWQYWhcKw0zJy8Pbbb2P06NFYunQpVq5cCV9fX+3O9/PPP8fChQuxYMECNG3aFFWrVsXYsWNLtdPn4cOHMXDgQMycOROhoaFwdHTExo0b8eWXX5baNvLKHxwUCgU0Gk2h7ceNG4eYmBh88cUXqFevHmxsbNCnTx/te2BjY1Pk9op73MLCAiJfxyN9fWqq5vtAGfK+Fbft7t27Q6lUYuvWrbC2tkZ2djb69OlT5HOISD4YhEuXWXa6rUjeeOMN7bfr1atXY9iwYdr+LAcPHkRYWBjeeust+Pv7o27duvjrr78MXnfDhg1x9+5d3M8zvu7IkSM6bQ4dOoTatWtj8uTJaNmyJfz8/HD79m2dNtbW1oVeVTvvts6cOYO0tDTtsoMHD8LCwgL169c3uOb8Dh48iCFDhqBnz55o2rQp3NzccOvWLe3jTZs2hUajwb59+/Q+v1mzZti/f3+hHXtdXFx03h+1Wo3z588XW5ch71uzZs10LiSan5WVFcLDw7Fy5UqsXLkS/fv3LzbkEBFVVgwsMmdnZ4d+/fph0qRJuH//vs7oFD8/P8TExODQoUO4dOkS3n33XSQmJhq87pCQELz00ksIDw/HmTNnsH//fkyePFmnjZ+fH+7cuYONGzfi+vXrWLRoEbZu3arTxsfHBzdv3kRcXBySkpKQmZlZYFsDBw6ESqVCeHg4zp8/jz179mD06NEYNGiQ9nRQSfj5+SE6OhpxcXE4c+YM3nzzTZ0jMj4+PggPD8ewYcOwbds23Lx5E3v37sUPP/wAABg1ahRSUlLQv39/nDhxAlevXsWaNWtw5coVAECnTp2wfft2bN++HZcvX8b777+PJ0+eGFRXce/b9OnTsWHDBkyfPh2XLl3CuXPn8Omnn+q0eeedd/DHH39gx44dGDZsWInfJyKiio6BpQJ4++238fjxY4SGhur0N5kyZQpatGiB0NBQBAcHw83NDT169DB4vRYWFti6dSuePXuGoKAgvPPOO5gzZ45Om//85z/48MMPMWrUKAQEBODQoUMFhtX27t0bXbp0wauvvgoXFxe9Q6ttbW2xc+dO/PPPP2jVqhX69OmDzp07Y8mSJca9GfnMnz8f1apVQ9u2bdG9e3eEhoaiRYsWOm2WLVuGPn36YOTIkWjQoAGGDx+uPdJTo0YN/PHHH0hNTUXHjh0RGBiIb775RntqatiwYQgPD8fgwYPRsWNH1K1bF6+++mqxdRnyvgUHB2Pz5s34+eefERAQgE6dOuHYsWM6bfz8/NC2bVs0aNAArVu3fpG3qkyo1cDevdL1bPbu5XTjRFR2ePFDIhkTQsDPzw8jR45EZGRkkW3L+3POq9ESUWngxQ+JKriHDx9iyZIlSEhIwNChQ01djo7cq9Hmn4g5Pl5azqliiKi0meUoIaKKoGbNmnB2dsaKFStQrVo1U5ejVdzVaBUK6Wq0YWEcwklEpYeBhUim5Hq21pir0XJIJxGVFp4SIiKj8Gq0RGQKZhVY5PqNlag0lNfnm1ejJSJTMIvAkjvVe2nOAEskN+np6QAKzhZc2ng1WiIyBbPow2JlZQVbW1s8fPgQVapUgYWFWeQ0MhNCCKSnp+PBgwdwcnLSuRZTWeDVaInIFMwisCgUCri7u+PmzZsFpkcnKm9CAJmZ0mgbS0tAqSz8aIUxnJyc4Obm9uIrMkCvXsCPP+qfh2XBAs7DQkSlzywmjsul0Wh4WohMatcuYO5cICHh+TI3N+CTT4B//7vk661SpUqZH1nRR63m1WiJ6MUYOnGcWQUWIlPKnWwt/29c7tGVH3/kkQkiMj+c6ZZIRoqbbA2QJlvjtXiIiPRjYCEqB8ZMtkZERAUxsBCVA062RkT0YhhYiMoBJ1sjInoxDCxE5YCTrRERvRgGFqJykDvZGlAwtHCyNSKi4jGwEJWT3MnWPD11l3t5cUgzEVFxzGKmWyK56NULCAvjZGtERMZiYCEqZ5aWQHCwqasgIqpYeEqIiIiIZI+BhYiIiGSPgYWIiIhkj4GFiIiIZI+BhYiIiGSPgYWIiIhkj4GFiIiIZI+BhYiIiGSPgYWIiIhkj4GFiIiIZI+BhYiIiGSPgYWIiIhkjxc/pApDreZVjomIzBUDC1UI0dHAmDHAvXvPl3l5AQsXAr16ma4uIiIqHzwlRLIXHQ306aMbVgAgPl5aHh1tmrqIiKj8MLCQrKnV0pEVIQo+lrts7FipHRERVV4lCixLly6Fj48PVCoVWrdujWPHjhXaNjs7G7NmzYKvry9UKhX8/f2xY8cOnTbLli1Ds2bN4ODgAAcHB7Rp0wa///57SUojPdRqYO9eYMMG6d+KtHPfv7/gkZW8hADu3pXaERFR5WV0YNm0aRMiIyMxffp0nDp1Cv7+/ggNDcWDBw/0tp8yZQqWL1+OxYsX4+LFi3jvvffQs2dPnD59WtvGy8sL8+bNw8mTJ3HixAl06tQJYWFhuHDhQslfGQGQTpf4+ACvvgq8+ab0r49PxTmNcv9+6bYjIqKKSSGEvoPthWvdujVatWqFJUuWAAA0Gg28vb0xevRoTJw4sUB7Dw8PTJ48GREREdplvXv3ho2NDdauXVvodqpXr47PP/8cb7/9tkF1paSkwNHREcnJyXBwcDDmJVVauX0/8v+EFQrp3x9/lH+H1b17pZBVnD17gODgsq6GiIhKm6H7b6OOsGRlZeHkyZMICQl5vgILC4SEhODw4cN6n5OZmQmVSqWzzMbGBgcOHNDbXq1WY+PGjUhLS0ObNm0KrSUzMxMpKSk6N3qusvT9aN9eGg2UG7LyUygAb2+pHRERVV5GBZakpCSo1Wq4urrqLHd1dUVCQoLe54SGhmL+/Pm4evUqNBoNYmJiEB0djfv5juGfO3cOdnZ2UCqVeO+997B161Y0atSo0FqioqLg6OiovXl7exvzUiq9ytL3w9JSGroMFAwtufcXLOB8LERElV2ZjxJauHAh/Pz80KBBA1hbW2PUqFEYOnQoLCx0N12/fn3ExcXh6NGjeP/99xEeHo6LFy8Wut5JkyYhOTlZe7t7925Zv5QKpTL1/ejVSzp95empu9zLq2Kc1iIiohdn1MRxzs7OsLS0RGJios7yxMREuLm56X2Oi4sLtm3bhoyMDDx69AgeHh6YOHEi6tatq9PO2toa9erVAwAEBgbi+PHjWLhwIZYvX653vUqlEkql0pjyzYq7e+m2M7VevYCwMM50S0Rkrow6wmJtbY3AwEDExsZql2k0GsTGxhbZ3wQAVCoVPD09kZOTgy1btiAsLKzI9hqNBpmZmcaUR3lUxr4flpZSx9oBA6R/GVaIiMyH0VPzR0ZGIjw8HC1btkRQUBAWLFiAtLQ0DB06FAAwePBgeHp6IioqCgBw9OhRxMfHIyAgAPHx8ZgxYwY0Gg3Gjx+vXeekSZPQtWtX1KpVC0+fPsX69euxd+9e7Ny5s5RepvnJ7fvRp48UTvJ2vmXfDyIiqmiMDiz9+vXDw4cPMW3aNCQkJCAgIAA7duzQdsS9c+eOTv+UjIwMTJkyBTdu3ICdnR26deuGNWvWwMnJSdvmwYMHGDx4MO7fvw9HR0c0a9YMO3fuxL/+9a8Xf4VmLLfvh75r8CxYwL4fRERUcRg9D4tccR6WwvEqx0REJFeG7r95tWYzkNv3g4iIqKLixQ+JiIhI9hhYiIiISPYYWIiIiEj2GFiIiIhI9hhYiIiISPYYWIiIiEj2GFiIiIhI9hhYiIiISPYYWIiIiEj2GFiIiIhI9hhYiIiISPYYWIiIiEj2GFiIiIhI9hhYiIiISPYYWIiIiEj2GFiIiIhI9hhYiIiISPYYWIiIiEj2GFiIiIhI9hhYiIiISPYYWIiIiEj2GFiIiIhI9hhYiIiISPYYWIiIiEj2GFiIiIhI9hhYiIiISPYYWIiIiEj2GFiIiIhI9hhYiIiISPYYWIiIiEj2GFiIiIhI9hhYiIiISPYYWIiIiEj2GFiIiIhI9hhYiIiISPYYWIiIiEj2GFiIiIhI9hhYiIiISPasTF2AnKnVwP79wP37gLs70L49YGlp6qqIiIjMDwNLIaKjgTFjgHv3ni/z8gIWLgR69TJdXUREROaIp4T0iI4G+vTRDSsAEB8vLY+ONk1dRERE5oqBJR+1WjqyIkTBx3KXjR0rtSMiIqLywcCSz/79BY+s5CUEcPeu1I6IiIjKBwNLPvfvl247IiIienEMLPm4u5duOyIiInpxDCz5tG8vjQZSKPQ/rlAA3t5SOyIiIiofDCz5WFpKQ5eBgqEl9/6CBZyPhYiIqDyVKLAsXboUPj4+UKlUaN26NY4dO1Zo2+zsbMyaNQu+vr5QqVTw9/fHjh07dNpERUWhVatWsLe3R82aNdGjRw9cuXKlJKWVil69gB9/BDw9dZd7eUnLOQ8LERFR+TI6sGzatAmRkZGYPn06Tp06BX9/f4SGhuLBgwd620+ZMgXLly/H4sWLcfHiRbz33nvo2bMnTp8+rW2zb98+RERE4MiRI4iJiUF2djb+/e9/Iy0treSv7AX16gXcugXs2QOsXy/9e/MmwwoREZEpKITQN+NI4Vq3bo1WrVphyZIlAACNRgNvb2+MHj0aEydOLNDew8MDkydPRkREhHZZ7969YWNjg7Vr1+rdxsOHD1GzZk3s27cPHTp0MKiulJQUODo6Ijk5GQ4ODsa8JCIiIjIRQ/ffRh1hycrKwsmTJxESEvJ8BRYWCAkJweHDh/U+JzMzEyqVSmeZjY0NDhw4UOh2kpOTAQDVq1cvtE1mZiZSUlJ0bkRERFQ5GRVYkpKSoFar4erqqrPc1dUVCQkJep8TGhqK+fPn4+rVq9BoNIiJiUF0dDTuFzKRiUajwdixY9GuXTs0adKk0FqioqLg6OiovXl7exvzUoiIiKgCKfNRQgsXLoSfnx8aNGgAa2trjBo1CkOHDoWFhf5NR0RE4Pz589i4cWOR6500aRKSk5O1t7t375ZF+URERCQDRgUWZ2dnWFpaIjExUWd5YmIi3Nzc9D7HxcUF27ZtQ1paGm7fvo3Lly/Dzs4OdevWLdB21KhR+PXXX7Fnzx54eXkVWYtSqYSDg4POjYiIiConowKLtbU1AgMDERsbq12m0WgQGxuLNm3aFPlclUoFT09P5OTkYMuWLQgLC9M+JoTAqFGjsHXrVvzxxx+oU6eOkS+DiIiIKjMrY58QGRmJ8PBwtGzZEkFBQViwYAHS0tIwdOhQAMDgwYPh6emJqKgoAMDRo0cRHx+PgIAAxMfHY8aMGdBoNBg/frx2nREREVi/fj1++ukn2Nvba/vDODo6wsbGpjReJxEREVVgRgeWfv364eHDh5g2bRoSEhIQEBCAHTt2aDvi3rlzR6d/SkZGBqZMmYIbN27Azs4O3bp1w5o1a+Dk5KRts2zZMgBAcHCwzrZWrlyJIUOGGP+qiIiIqFIxeh4WueI8LERERBVPmczDQkRERGQKDCxEREQkewwsREREJHsMLERERCR7DCxEREQkewwsREREJHsMLERERCR7DCxEREQkewwsREREJHsMLERERCR7DCxEREQkewwsREREJHsMLERERCR7DCxEREQkewwsREREJHsMLERERCR7DCxEREQkewwsREREJHsMLERERCR7DCxEREQkewwsREREJHsMLERERCR7DCxEREQkewwsREREJHsMLERERCR7DCxEREQkewwsREREJHsMLERERCR7DCxEREQkewwsREREJHsMLERERCR7DCxEREQkewwsREREJHsMLERERCR7DCxEREQkewwsREREJHsMLERERCR7DCxEREQkewwsREREJHsMLERERCR7DCxEREQkewwsREREJHsMLERERCR7DCxEREQkewwsREREJHsMLERERCR7DCxEREQkewwsREREJHslCixLly6Fj48PVCoVWrdujWPHjhXaNjs7G7NmzYKvry9UKhX8/f2xY8cOnTZ//vknunfvDg8PDygUCmzbtq0kZREREVElZXRg2bRpEyIjIzF9+nScOnUK/v7+CA0NxYMHD/S2nzJlCpYvX47Fixfj4sWLeO+999CzZ0+cPn1a2yYtLQ3+/v5YunRpyV8JERERVVoKIYQw5gmtW7dGq1atsGTJEgCARqOBt7c3Ro8ejYkTJxZo7+HhgcmTJyMiIkK7rHfv3rCxscHatWsLFqRQYOvWrejRo4dRLyQlJQWOjo5ITk6Gg4ODUc8lIiIi0zB0/23UEZasrCycPHkSISEhz1dgYYGQkBAcPnxY73MyMzOhUql0ltnY2ODAgQPGbFrvelNSUnRuREREVDkZFViSkpKgVqvh6uqqs9zV1RUJCQl6nxMaGor58+fj6tWr0Gg0iImJQXR0NO7fv1/yqgFERUXB0dFRe/P29n6h9REREZF8lfkooYULF8LPzw8NGjSAtbU1Ro0ahaFDh8LC4sU2PWnSJCQnJ2tvd+/eLaWKiYiISG6MSg3Ozs6wtLREYmKizvLExES4ubnpfY6Liwu2bduGtLQ03L59G5cvX4adnR3q1q1b8qoBKJVKODg46NyIiIiocjIqsFhbWyMwMBCxsbHaZRqNBrGxsWjTpk2Rz1WpVPD09EROTg62bNmCsLCwklVMREREZsfK2CdERkYiPDwcLVu2RFBQEBYsWIC0tDQMHToUADB48GB4enoiKioKAHD06FHEx8cjICAA8fHxmDFjBjQaDcaPH69dZ2pqKq5du6a9f/PmTcTFxaF69eqoVavWi75GIiIiquCMDiz9+vXDw4cPMW3aNCQkJCAgIAA7duzQdsS9c+eOTv+UjIwMTJkyBTdu3ICdnR26deuGNWvWwMnJSdvmxIkTePXVV7X3IyMjAQDh4eFYtWpVCV8aERERVRZGz8MiV5yHhYiIqOIpk3lYiIiIiEyBgYWIiIhkj4GFiIiIZI+BhYiIiGSPgYWIiIhkj4GFiIiIZI+BhYiIiGSPgYWIiIhkj4GFiIiIZI+BhYiIiGSPgYWIiIhkj4GFiIiIZI+BhYiIiGSPgYWIiIhkj4GFiIiIZI+BhYiIiGSPgYWIiIhkj4GFiIiIZI+BhYiIiGSPgYWIiIhkj4GFiIiIZI+BhYiIiGSPgYWIiIhkj4GFiIiIZI+BhYiIiGSPgYWIiIhkj4GFiIiIZI+BhYiIiGSPgYWIiIhkj4GFiIiIZI+BhYiIiGSPgYWIiIhkj4GFiIiIZI+BhYiIiGSPgYWIiIhkj4GFiIiIZI+BhYiIiGSPgYWIiIhkj4GFiIiIZI+BhYiIiGSPgYWIiIhkj4GFiIiIZI+BhYiIiGSPgYWIiIhkj4GFiIiIZI+BhYiIiGSPgYWIiIhkj4GFiIiIZI+BhYiIiGSvRIFl6dKl8PHxgUqlQuvWrXHs2LFC22ZnZ2PWrFnw9fWFSqWCv78/duzY8ULrJCIiIvNidGDZtGkTIiMjMX36dJw6dQr+/v4IDQ3FgwcP9LafMmUKli9fjsWLF+PixYt477330LNnT5w+fbrE6yQiIiLzohBCCGOe0Lp1a7Rq1QpLliwBAGg0Gnh7e2P06NGYOHFigfYeHh6YPHkyIiIitMt69+4NGxsbrF27tkTrBIDMzExkZmZq76ekpMDb2xvJyclwcHAw5iURERGRiaSkpMDR0bHY/bdRR1iysrJw8uRJhISEPF+BhQVCQkJw+PBhvc/JzMyESqXSWWZjY4MDBw6UeJ0AEBUVBUdHR+3N29vbmJdCREREFYhRgSUpKQlqtRqurq46y11dXZGQkKD3OaGhoZg/fz6uXr0KjUaDmJgYREdH4/79+yVeJwBMmjQJycnJ2tvdu3eNeSlERERUgZT5KKGFCxfCz88PDRo0gLW1NUaNGoWhQ4fCwuLFNq1UKuHg4KBzIyIiosrJqNTg7OwMS0tLJCYm6ixPTEyEm5ub3ue4uLhg27ZtSEtLw+3bt3H58mXY2dmhbt26JV4nERERmRejAou1tTUCAwMRGxurXabRaBAbG4s2bdoU+VyVSgVPT0/k5ORgy5YtCAsLe+F1EhERkXmwMvYJkZGRCA8PR8uWLREUFIQFCxYgLS0NQ4cOBQAMHjwYnp6eiIqKAgAcPXoU8fHxCAgIQHx8PGbMmAGNRoPx48cbvE4iIiIyb0YHln79+uHhw4eYNm0aEhISEBAQgB07dmg7zd65c0enf0pGRgamTJmCGzduwM7ODt26dcOaNWvg5ORk8DqJiIjIvBk9D4tcGTqOm4iIiOSjTOZhISIiIjIFBhYiIiKSPQYWIiIikj0GFiIiIpI9BhYiIiKSPQYWIiIikj0GFiIiIpI9BhYiIiKSPQYWIiIikj0GFiIiIpI9BhYiIiKSPQYWIiIikj0GFiIiIpI9BhYiIiKSPQYWIiIikj0GFiIiIpI9BhYiIiKSPQYWIiIikj0GFjOgVgNnzwJHjpi6EiIiopKxMnUBVPoyMoATJ4D9+4EDB4CDB4HkZOmxJUuAiAjT1kdERGQsBpZK4MkT4NCh5wHl2DEgK0u3jUolBZkxY4D69YGQEJOUSkREVCIMLBXQvXtSMMkNKOfOAULotnF1BV55BWjfXvrX3x94+21g9Wqgb18p1Pj5maZ+IiIiYzGwyJwQwKVLugHl1q2C7fz8dANKvXqAQqHbZvly4OpV4PBhoHt3qU+Lk1N5vAoiIqIXw8AiM1lZwKlTzwPKwYPAo0e6bSwsgObNnweUdu0AN7fi161SAdHRQFAQcOUK0K8fsH07YMVPARERyRx3VSb29Kl0xCM3oBw9Cjx7ptvGxgZ4+eXnAeXllwF7+5Jtz80N+PlnKeTs2gWMGwcsWPDCL4OIiKhMMbCUs8TE5+Fk/34gLg7QaHTb1KghhZPcgNK8OWBtXXo1BAQAa9YAvXsDCxcCjRsDw4eX3vqJiIhKGwNLGRICuHZNt//J1asF2/n4PO970r69NIrHooxnyOnVC5g9G5g6FRg5EnjpJaBjx7LdJhERUUkxsJSinBzgzBndgJKYqNtGoQCaNn0eUF55BfDyMk29kycDFy4AGzdKR1uOHQPq1jVNLUREREVhYHkB6elSn5PcgHL4MJCaqtvG2lrq5JobUNq2lc/IHIUC+O476SjQiRPSyKHDhwEHB1NXRkREpIuBxQiPHknhJDegnDwpHVXJy9FR6tCae3qnZUtpdI5c2dgAP/0EtGoFXLwIvPmmdN/S0tSVERERPcfAUoSsLGDTpucB5dKlgm08PXX7nzRuXPF29h4eUkhp314a5jxpEvDZZ6auioiI6DmFEPnnSK2YUlJS4OjoiOTkZDiU0jkNtVoasZN7HR4AaNhQN6DUrl1wgraKatMmoH9/6f8rVwJDhpi0HCIiMgOG7r95hKUIlpbAO+9IgeSVV6RTPc7Opq6q7PTrJ3XCnT0bePddafbcdu1MXRURERGPsFA+Gg3wxhvAli2Aiwtw/Lh0FImIiKgsGLr/LuPZPqiisbAAvv9emlzu4UPgP/8pOPKJiIiovDGwUAFVq0rT97u6AmfPAoMGFZyNl4iIqDwxsJBe3t7Atm2AUin9O3WqqSsiIiJzxsBChXr5ZeB//5P+P3cusH69aeshIiLzxcBCRXrrLWDiROn/w4ZJ0/cTERGVNwYWKtacOdK0/ZmZQFgYcO+eqSsiIiJzw8BCxbKwANatky7amJAghZb0dFNXRURE5oQTx5FB7O2lkUOtWgGnTkmz4G7cKIUZMt7q1cDy5YCtLeDmpntzd3/+/2rVKs9MykREL4KBhQzm4wNERwOdOwObN0vXTZo+3dRVVSzp6UBEBLBqlWHtra0LBpr8oSb3JueLbBIRvSjOdEtG++474O23pf//8APQt69p66koLl2S3qsLF6QjU5MnS5c/SEh4frt///n/Hz82bv1OToYFG2dnHhkjIvngtYSozAwbJu10588HwsOBunWBwEBTVyVv69cDI0YAaWlSaNiwAQgOLvo5mZlAYqJuiNEXbBISpLZPnki3y5eLXq+lpTQpYFGhJndZ1aql9AYQEb0gHmGhElGrpZFDv/8OeHpK1xxydzd1VfKTkQGMGQOsWCHdf/VVKby4uZXeNoSQrihuSLB5+NC4ddvZFR9svLyk604REZWEoftvBhYqseRkoE0b6VRHUBCwdy9gY2PqquTj2jXpFFBcnNRxdupUYNo06QiHqWRnAw8eFB9s7t83biTYm28CS5dKp6WIiIzBwELl4to1oHVr4J9/pJ3W2rUc1QIAP/4onTp7+lQ6+rB2LfDvf5u6KuM8fVp8sMm9CQHUqgWsWQN06GDqyomoIinTqzUvXboUPj4+UKlUaN26NY4VM/3pggULUL9+fdjY2MDb2xsffvghMjIytI8/ffoUY8eORe3atWFjY4O2bdvi+PHjJSmNylm9etLO2cpKOtUxb56pKzKtzEzggw+kIytPnwKvvAKcPl3xwgogDWX38wPat5dez+jR0iUavvsO+O03aXj7338Dhw4Bvr7AnTtSv5zJk6UjOUREpUoYaePGjcLa2lp899134sKFC2L48OHCyclJJCYm6m2/bt06oVQqxbp168TNmzfFzp07hbu7u/jwww+1bd544w3RqFEjsW/fPnH16lUxffp04eDgIO7du2dwXcnJyQKASE5ONvYlUSn4+mshpO/ZQmzdaupqTOPGDSFatXr+PkyYIER2tqmrKh8pKUIMHfr8tbdsKcSVK6auiogqAkP330YHlqCgIBEREaG9r1arhYeHh4iKitLbPiIiQnTq1ElnWWRkpGjXrp0QQoj09HRhaWkpfv31V502LVq0EJMnTza4LgYW0xs1StpZVa0qRFycqaspXz/9JISTk/T6q1UTIt/H2Wxs3iy9fkAIW1shVqwQQqMxdVVEJGeG7r+NOiWUlZWFkydPIiQkRLvMwsICISEhOHz4sN7ntG3bFidPntSeNrpx4wZ+++03dOvWDQCQk5MDtVoNVb5Zr2xsbHDgwIFCa8nMzERKSorOjUzrq6+AkBBp6O5//iN17qzssrOBceOkyxU8eSJd4TouDnjtNVNXZhp9+gBnzwKdOkmddkeMAHr1ApKSTF0ZEVV0RgWWpKQkqNVquLq66ix3dXVFQkKC3ue8+eabmDVrFl555RVUqVIFvr6+CA4OxieffAIAsLe3R5s2bTB79mz8/fffUKvVWLt2LQ4fPoz79+8XWktUVBQcHR21N29vb2NeCpUBKytpIjk/P6k/Q69eUp+OyuruXaBjR+DLL6X7H34I7NsndT41Z15eQEwM8PnnQJUqwLZtQLNmwK5dpq6MiCqyMp/vcu/evZg7dy7++9//4tSpU4iOjsb27dsxe/ZsbZs1a9ZACAFPT08olUosWrQIAwYMgEUR03FOmjQJycnJ2tvdu3fL+qWQAapVA375RRreevAg8N57Uq+Gyub334HmzYHDhwFHR+mSBfPnS1PpkzST7rhxwNGjQMOG0uii0FAp1OXpb09EZDCjAouzszMsLS2RmJioszwxMRFuhcyENXXqVAwaNAjvvPMOmjZtip49e2Lu3LmIioqCRqMBAPj6+mLfvn1ITU3F3bt3cezYMWRnZ6Nu3bqF1qJUKuHg4KBzI3moX1860mJpKV0zZ/58U1dUenJypFEw3boBjx4BLVpIo2V69jR1ZfLUvDlw4oR0/SQAWLBAmrPn/HmTlkVEFZBRgcXa2hqBgYGIjY3VLtNoNIiNjUWbNm30Pic9Pb3AkRLL/z9zlsj31btq1apwd3fH48ePsXPnToSFhRlTHsnIv/4l9WkBgI8/BrZvN209peH+famPzty50v2RI6WjSEXkaoJ0ReolS4BffwVq1gTOnQNatgQWLaqcR9+IqIwY25t348aNQqlUilWrVomLFy+KESNGCCcnJ5GQkCCEEGLQoEFi4sSJ2vbTp08X9vb2YsOGDeLGjRti165dwtfXV7zxxhvaNjt27BC///679nF/f3/RunVrkZWVZXBdHCUkPxqNECNGSCNG7O2FOH/e1BWV3O7dQtSsKb0WOzshNm40dUUVU0KCEN26PR/+3KWLEPfvm7oqIjKlMhvWLIQQixcvFrVq1RLW1tYiKChIHDlyRPtYx44dRXh4uPZ+dna2mDFjhvD19RUqlUp4e3uLkSNHisePH2vbbNq0SdStW1dYW1sLNzc3ERERIZ48eWJUTQws8pSVJURwsLRzqlNHiIcPTV2RcXJyhJg5UwiFQnoNTZtyfpEXpdEIsXSpECqV9J46O0vDwonIPBm6/+bU/FTmHj2S+i3cuCGNqtm1q2J0Tn3wABg4ENi9W7r/zjvSaQxeL6l0XLwovb9xcdL9d9+VRlzxCtFE5qVMp+YnMkaNGsDPP0tTve/bB4waJf++C3/+CQQESGHF1hb4/nvgm28YVkpTo0bAkSPSaCIAWL4cCAwETp40bV1EJE8MLFQuGjcGNm6ULoz4zTfA4sWmrkg/jUa6HlKnTlIn24YNgePHgcGDTV1Z5aRUSvO17N4NeHoCV65Ik+/Nmweo1aaujojkhIGFyk23btLOCZDm45DbRGKPHgHduwOTJkk7y0GDpLDSqJGpK6v8OneWZsjt3VsaOj5pkrTszh1TV0ZEcsHAQuUqMhIYOlQ6kvHGG9I3ajk4fFiaM+S33wCVSjoK9P337E9RnqpXBzZvlq4GXbWqdPqwWTNg0yZTV0ZkvlJSgAMHpKPipj6dz063VO4yM6VvzwcPStP4Hzki7axMQQhpMrPx46Vv9n5+0k7T39809ZDk2jXgrbekmXIB6WjXkiUAf7WJyoYQ0mnwuDjg9Onn/16/rtvu3j3p9G1pMnT/bVW6myUqnlIpTWUfFARcvSodafn9d+m6M+Xp8WNg2DDpWjcA0K8fsGIFd4pyUK8esH8/MHs2MGcOsGaN9C1v7VqgbVtTV0dUsanV0t/e/OHk4UP97b28pCPQAQHSDOamwiMsZDJnz0o7n7Q0aer2JUvKb9snTkhB6eZNaYj1V18B778vdQomeTl4UDracuuWdI2iKVOAqVOli20SUdHS06VLYeQNJ2fPSsvzs7AAGjR4Hk6aN5eONjs7l22Nhu6/GVjIpH76SboOjxDAf/8rhYaylLudyEggKwuoU0c6BRQYWLbbpReTnAyMHi0daQGA1q2BdesAX1/T1kUkJ0lJUiDJG04uX5b6DOZnayv1EcsbTpo0Mc3UDQwsVGFERQGffCIdaty1SxpSXBZSUoDhw6ULMwJAjx7AypXSlaWpYti4UboCeHIyYGcndQQMD+eRMTIvQkhHh/OHk3v39LevWfN5MMkNJ/Xqmfb0Tl4MLFRhCCF1qly3DqhWDTh2TPplKk1nzgB9+0rnba2spOHVY8ZwR1cR3bkjfV7+/FO636ePNOmcqTpuk2HUaiA1tejb06e691Uq6eeae6tRQ/e+nV3l/x3OypJmhc4fTlJS9LevV69gOHFzk/f7xMBCFUpGBhAcLI0KadBAGjnk6Pji6xUC+N//gA8+kLbh7S0dYXn55RdfN5mOWi2FzqlTpdFdnp7A6tVld3TO3OTkFB0kigsa+m7PnpV+nVZWhYeZopY7OMhzB56cLH25yhtOLlwAsrMLtrW2lk7h5A0n/v7SjOIVDQMLVTgJCUCrVtJhzS5dgF9+ebGOlampUp+YtWul+6+9Js2tUqNG6dRLpnfihHQ9or/+knZA48ZJI4uUSlNXZhrZ2dK38UePXixsZGaWXY2WltJO1c6u6FvVqlId//xT8Pbo0YvVaGkpHc0tLOQUFnQcHaWOqS9KCODvv3VH6MTFSddb08fJ6fnRktxw0rBh+Y+sLCsMLFQhnToFvPKK9G3sww+B+fNLtp4LF6RTQJcuSX+c5s6Vdmal8ceG5CUtTepEvWKFdD8gAFi/XvqDXpnl9mM4dkw6MnnsmPT7k5FRetuwti4+WNjZGRZAcm9KZekc3UhPLzzM6Fue+9iLHOmxsJCCjjEhp3p16UKq+cNJUpL+bdSqpRtOmjeXlsnxiFBpYWChCuvHH6WwAUinc95+27jnr14tHVlJTwc8PKSOmu3bl36dJC8//SR9Vh49kvo+fPll5Rqq/uiRdKmI3HBy7Jj+nZ6TkzRvRmmEi4pwVXVjPXsmzcFkbNBJSyvdOiwtpVCd/8iJOfbFYmChCm3WLGD6dOmQ5+7dQIcOxT/n2TNp6Ou330r3//Uv6XRQzZplWyvJx/370qUfdu6U7r/2mvR5cHU1bV3GysiQvoXnPXpy7VrBdtbW0k4uKEga6h0UJHW65JHE0peZWbKg8/SpdHrL3183nDRpIgVrYmAxdTn0goQA+veXOsg6O0t/sOvUKbz9lSvSUZlz56Rv1DNnPh8qTeZFo5EmIRw/XtrJ1KwpDV/v1s3Ulemn0Uij13KDydGjUsdLfR0t/fyeB5PWraWdoLn216kosrOlv0MMkYVjYKEKLz1dOrJy8qT0beTQIf094DdulOZXSU2Vdk7r10vXKiLzdu6c1CH33DnpfkSENLLIFBNj5ZWYqBtOjh+XRofk5+LyPJwEBUkd0s3xdAFVfgwsVCnEx0t/qO/fB7p3B7ZufX7UJCND6my5bJl0v2NHYMMGwN3ddPWSvGRkAJMmSRe4BKQ+A+vXS4fky0NamtQRNm9AuXOnYDuVSpptOe/Rk9q1K0//G6KiMLBQpXHsmHSkJTMTmDABmDdPGv7Xt6+0MwCAyZOBGTN4fRnSb9cuaUbchASpX9TcuVLYLc3D9Gq1NKQ4b7+T8+el5XkpFECjRrr9Tpo0qTxDVImMxcBClcr69dLhfUDqWLt6tXQYvUYNqWNtly6mrY/kLykJeOcdaTQRIJ02/P57adI5YwkhzReUN5ycOKF/JImnp244CQzkFcGJ8mJgoUpnyhRgzpzn99u2lfqveHubriaqWHJnPh47VuojVa0a8M03QO/eRT8vJUXqa5I3oNy/X7CdnZ10CjNvQClJICIyJwwsVOloNMCbb0ojh8aNk8ILD6NTSfz1l/RZOnlSuj9sGLBwoRQ4srOBs2efz3Vy9Kh0xdv8fyktLaWr3eZ2im3dWrqsBEemERmHgYUqJSGkb7ulcZ0hMm9ZWVK/p3nzpM9VnTrSReJOn9Y/W6yPj26n2ObNAVvb8q6aqPJhYCEiMsC+fdLVn+/efb7MyUn3tE5QECcgJCorhu6/OaaCiMxax47SKaB166Qjd0FB0gRtHFJMJC8MLERk9pycpInliEi+OFkwERERyR4DCxEREckeAwsRERHJHgMLERERyR4DCxEREckeAwsRERHJHgMLERERyR4DCxEREckeAwsRERHJHgMLERERyR4DCxEREckeAwsRERHJHgMLERERyV6luVqzEAIAkJKSYuJKiIiIyFC5++3c/XhhKk1gefr0KQDA29vbxJUQERGRsZ4+fQpHR8dCH1eI4iJNBaHRaPD333/D3t4eCoXC1OXITkpKCry9vXH37l04ODiYuhyzx5+H/PBnIi/8echLWf48hBB4+vQpPDw8YGFReE+VSnOExcLCAl5eXqYuQ/YcHBz4yy8j/HnID38m8sKfh7yU1c+jqCMrudjploiIiGSPgYWIiIhkj4HFTCiVSkyfPh1KpdLUpRD485Aj/kzkhT8PeZHDz6PSdLolIiKiyotHWIiIiEj2GFiIiIhI9hhYiIiISPYYWIiIiEj2GFiIiIhI9hhYKrmoqCi0atUK9vb2qFmzJnr06IErV66Yuiz6/+bNmweFQoGxY8eauhSzFR8fj7feegs1atSAjY0NmjZtihMnTpi6LLOkVqsxdepU1KlTBzY2NvD19cXs2bOLvSgelZ4///wT3bt3h4eHBxQKBbZt26bzuBAC06ZNg7u7O2xsbBASEoKrV6+WS20MLJXcvn37EBERgSNHjiAmJgbZ2dn497//jbS0NFOXZvaOHz+O5cuXo1mzZqYuxWw9fvwY7dq1Q5UqVfD777/j4sWL+PLLL1GtWjVTl2aWPv30UyxbtgxLlizBpUuX8Omnn+Kzzz7D4sWLTV2a2UhLS4O/vz+WLl2q9/HPPvsMixYtwtdff42jR4+iatWqCA0NRUZGRpnXxnlYzMzDhw9Rs2ZN7Nu3Dx06dDB1OWYrNTUVLVq0wH//+1/83//9HwICArBgwQJTl2V2Jk6ciIMHD2L//v2mLoUAvP7663B1dcW3336rXda7d2/Y2Nhg7dq1JqzMPCkUCmzduhU9evQAIB1d8fDwwEcffYRx48YBAJKTk+Hq6opVq1ahf//+ZVoPj7CYmeTkZABA9erVTVyJeYuIiMBrr72GkJAQU5di1n7++We0bNkSffv2Rc2aNdG8eXN88803pi7LbLVt2xaxsbH466+/AABnzpzBgQMH0LVrVxNXRgBw8+ZNJCQk6PzdcnR0ROvWrXH48OEy336luVozFU+j0WDs2LFo164dmjRpYupyzNbGjRtx6tQpHD9+3NSlmL0bN25g2bJliIyMxCeffILjx4/jgw8+gLW1NcLDw01dntmZOHEiUlJS0KBBA1haWkKtVmPOnDkYOHCgqUsjAAkJCQAAV1dXneWurq7ax8oSA4sZiYiIwPnz53HgwAFTl2K27t69izFjxiAmJgYqlcrU5Zg9jUaDli1bYu7cuQCA5s2b4/z58/j6668ZWEzghx9+wLp167B+/Xo0btwYcXFxGDt2LDw8PPjzIJ4SMhejRo3Cr7/+ij179sDLy8vU5ZitkydP4sGDB2jRogWsrKxgZWWFffv2YdGiRbCysoJarTZ1iWbF3d0djRo10lnWsGFD3Llzx0QVmbePP/4YEydORP/+/dG0aVMMGjQIH374IaKiokxdGgFwc3MDACQmJuosT0xM1D5WlhhYKjkhBEaNGoWtW7fijz/+QJ06dUxdklnr3Lkzzp07h7i4OO2tZcuWGDhwIOLi4mBpaWnqEs1Ku3btCgzz/+uvv1C7dm0TVWTe0tPTYWGhu1uytLSERqMxUUWUV506deDm5obY2FjtspSUFBw9ehRt2rQp8+3zlFAlFxERgfXr1+Onn36Cvb299jyjo6MjbGxsTFyd+bG3ty/Qf6hq1aqoUaMG+xWZwIcffoi2bdti7ty5eOONN3Ds2DGsWLECK1asMHVpZql79+6YM2cOatWqhcaNG+P06dOYP38+hg0bZurSzEZqaiquXbumvX/z5k3ExcWhevXqqFWrFsaOHYv/+7//g5+fH+rUqYOpU6fCw8NDO5KoTAmq1ADova1cudLUpdH/17FjRzFmzBhTl2G2fvnlF9GkSROhVCpFgwYNxIoVK0xdktlKSUkRY8aMEbVq1RIqlUrUrVtXTJ48WWRmZpq6NLOxZ88evfuM8PBwIYQQGo1GTJ06Vbi6ugqlUik6d+4srly5Ui61cR4WIiIikj32YSEiIiLZY2AhIiIi2WNgISIiItljYCEiIiLZY2AhIiIi2WNgISIiItljYCEiIiLZY2AhIiIi2WNgISIiItljYCEiIiLZY2AhIiIi2ft/cPLqriIKAR4AAAAASUVORK5CYII=\n"
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjUAAAGzCAYAAADXFObAAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABcVUlEQVR4nO3deVxU5eIG8GcGZABlRzZFQTSXElFRrppLOQppLrmBP83lmnbdkeyq10TUClxTc7t5S71uaIVmmqQilCmhaaS5kCZuBKgZIKAsM+/vj3MZHAFlUBg4PN/PZz4573nPOe+ByXl8z3veVyGEECAiIiKq4ZTGbgARERHR88BQQ0RERLLAUENERESywFBDREREssBQQ0RERLLAUENERESywFBDREREssBQQ0RERLLAUENERESywFBDVIXGjBkDDw+PCu0bFhYGhULxfBtUzVy7dg0KhQKbN2+u0vPGxcVBoVAgLi5OV1be31VltdnDwwNjxox5rscsj82bN0OhUODatWtVfm6iZ8VQQwRAoVCU6/Xolx7Rszpx4gTCwsKQkZFh7KYQyYKpsRtAVB1s3bpV7/1///tfHD58uER5y5Ytn+k8GzduhFarrdC+7733HmbPnv1M56fye5bfVXmdOHECCxYswJgxY2Bra6u3LSkpCUol/91JZAiGGiIAI0eO1Hv/448/4vDhwyXKH5ebmwtLS8tyn6dOnToVah8AmJqawtSU/8tWlWf5XT0PKpXKqOcnqon4zwCicurRowdeeuklnD59Gt26dYOlpSX+9a9/AQC++uor9O3bF25ublCpVPDy8sKiRYug0Wj0jvH4OI2i8RjLli3DJ598Ai8vL6hUKnTo0AGnTp3S27e0MTUKhQJTpkzB3r178dJLL0GlUuHFF19EdHR0ifbHxcXB19cX5ubm8PLywr///e9yj9M5duwYhg4dikaNGkGlUsHd3R0zZszAgwcPSlxfvXr1kJKSgoEDB6JevXqoX78+Zs6cWeJnkZGRgTFjxsDGxga2trYYPXp0uW7D/PTTT1AoFNiyZUuJbd9++y0UCgX2798PALh+/TomTZqE5s2bw8LCAg4ODhg6dGi5xouUNqamvG0+e/YsxowZgyZNmsDc3BwuLi74+9//jj///FNXJywsDO+++y4AwNPTU3eLs6htpY2puXr1KoYOHQp7e3tYWlrib3/7Gw4cOKBXp2h80O7du/HBBx+gYcOGMDc3R8+ePXHlypWnXndZ1q1bhxdffBEqlQpubm6YPHlyiWu/fPkyBg8eDBcXF5ibm6Nhw4YICgpCZmamrs7hw4fx8ssvw9bWFvXq1UPz5s11/x8RPSv+s4/IAH/++Sdee+01BAUFYeTIkXB2dgYgDa6sV68eQkJCUK9ePRw9ehShoaHIysrC0qVLn3rcHTt24P79+3j77behUCiwZMkSDBo0CFevXn1qj8EPP/yAqKgoTJo0CVZWVli9ejUGDx6MGzduwMHBAQDw888/IyAgAK6urliwYAE0Gg0WLlyI+vXrl+u6P//8c+Tm5mLixIlwcHDAyZMn8fHHH+PWrVv4/PPP9epqNBr4+/vDz88Py5Ytw5EjR7B8+XJ4eXlh4sSJAAAhBAYMGIAffvgB//jHP9CyZUvs2bMHo0ePfmpbfH190aRJE+zevbtE/V27dsHOzg7+/v4AgFOnTuHEiRMICgpCw4YNce3aNaxfvx49evTAhQsXDOplM6TNhw8fxtWrVzF27Fi4uLjg/Pnz+OSTT3D+/Hn8+OOPUCgUGDRoEH777Tfs3LkTH330ERwdHQGgzN9Jeno6OnfujNzcXEybNg0ODg7YsmUL+vfvjy+++AJvvPGGXv2IiAgolUrMnDkTmZmZWLJkCUaMGIGEhIRyX3ORsLAwLFiwAGq1GhMnTkRSUhLWr1+PU6dO4fjx46hTpw7y8/Ph7++PvLw8TJ06FS4uLkhJScH+/fuRkZEBGxsbnD9/Hq+//jq8vb2xcOFCqFQqXLlyBcePHze4TUSlEkRUwuTJk8Xj/3t0795dABAbNmwoUT83N7dE2dtvvy0sLS3Fw4cPdWWjR48WjRs31r1PTk4WAISDg4O4d++ervyrr74SAMTXX3+tK5s/f36JNgEQZmZm4sqVK7qyX375RQAQH3/8sa6sX79+wtLSUqSkpOjKLl++LExNTUscszSlXV94eLhQKBTi+vXretcHQCxcuFCvbtu2bUX79u117/fu3SsAiCVLlujKCgsLRdeuXQUAsWnTpie2Z86cOaJOnTp6P7O8vDxha2sr/v73vz+x3fHx8QKA+O9//6sri42NFQBEbGys3rU8+rsypM2lnXfnzp0CgPj+++91ZUuXLhUARHJycon6jRs3FqNHj9a9Dw4OFgDEsWPHdGX3798Xnp6ewsPDQ2g0Gr1radmypcjLy9PVXbVqlQAgzp07V+Jcj9q0aZNem27fvi3MzMxE7969decQQog1a9YIAOKzzz4TQgjx888/CwDi888/L/PYH330kQAg7ty588Q2EFUUbz8RGUClUmHs2LElyi0sLHR/vn//Pu7evYuuXbsiNzcXly5deupxAwMDYWdnp3vftWtXANLthqdRq9Xw8vLSvff29oa1tbVuX41GgyNHjmDgwIFwc3PT1WvatClee+21px4f0L++nJwc3L17F507d4YQAj///HOJ+v/4xz/03nft2lXvWr755huYmprqem4AwMTEBFOnTi1XewIDA1FQUICoqChd2aFDh5CRkYHAwMBS211QUIA///wTTZs2ha2tLc6cOVOuc1WkzY+e9+HDh7h79y7+9re/AYDB5330/B07dsTLL7+sK6tXrx4mTJiAa9eu4cKFC3r1x44dCzMzM917Qz5Tjzpy5Ajy8/MRHBysN3B5/PjxsLa21t3+srGxASDdAszNzS31WEWDob/66qtKH4RNtRNDDZEBGjRooPdFUeT8+fN44403YGNjA2tra9SvX183yPjR8QRladSokd77ooDz119/Gbxv0f5F+96+fRsPHjxA06ZNS9Qrraw0N27cwJgxY2Bvb68bJ9O9e3cAJa/P3Ny8xC2UR9sDSGNdXF1dUa9ePb16zZs3L1d72rRpgxYtWmDXrl26sl27dsHR0RGvvvqqruzBgwcIDQ2Fu7s7VCoVHB0dUb9+fWRkZJTr9/IoQ9p87949TJ8+Hc7OzrCwsED9+vXh6ekJoHyfh7LOX9q5ip7Iu379ul75s3ymHj8vUPI6zczM0KRJE912T09PhISE4D//+Q8cHR3h7++PtWvX6l1vYGAgunTpgrfeegvOzs4ICgrC7t27GXDoueGYGiIDPPov8CIZGRno3r07rK2tsXDhQnh5ecHc3BxnzpzBrFmzyvUXtomJSanlQohK3bc8NBoNevXqhXv37mHWrFlo0aIF6tati5SUFIwZM6bE9ZXVnuctMDAQH3zwAe7evQsrKyvs27cPw4cP13tCbOrUqdi0aROCg4PRqVMn2NjYQKFQICgoqFK/SIcNG4YTJ07g3XffhY+PD+rVqwetVouAgIAq+wKv7M9FaZYvX44xY8bgq6++wqFDhzBt2jSEh4fjxx9/RMOGDWFhYYHvv/8esbGxOHDgAKKjo7Fr1y68+uqrOHToUJV9dki+GGqInlFcXBz+/PNPREVFoVu3brry5ORkI7aqmJOTE8zNzUt98qU8T8OcO3cOv/32G7Zs2YJRo0bpyg8fPlzhNjVu3BgxMTHIzs7W6/lISkoq9zECAwOxYMECfPnll3B2dkZWVhaCgoL06nzxxRcYPXo0li9frit7+PBhhSa7K2+b//rrL8TExGDBggUIDQ3VlV++fLnEMQ2ZIbpx48al/nyKbm82bty43McyRNFxk5KS0KRJE115fn4+kpOToVar9eq3bt0arVu3xnvvvYcTJ06gS5cu2LBhA95//30AgFKpRM+ePdGzZ0+sWLECH374IebOnYvY2NgSxyIyFG8/ET2jon9dPvov4Pz8fKxbt85YTdJjYmICtVqNvXv34o8//tCVX7lyBQcPHizX/oD+9QkhsGrVqgq3qU+fPigsLMT69et1ZRqNBh9//HG5j9GyZUu0bt0au3btwq5du+Dq6qoXKova/njPxMcff1zi8fLn2ebSfl4AsHLlyhLHrFu3LgCUK2T16dMHJ0+eRHx8vK4sJycHn3zyCTw8PNCqVavyXopB1Go1zMzMsHr1ar1r+vTTT5GZmYm+ffsCALKyslBYWKi3b+vWraFUKpGXlwdAui33OB8fHwDQ1SF6FuypIXpGnTt3hp2dHUaPHo1p06ZBoVBg69atldrNb6iwsDAcOnQIXbp0wcSJE6HRaLBmzRq89NJLSExMfOK+LVq0gJeXF2bOnImUlBRYW1vjyy+/NHhsxqP69euHLl26YPbs2bh27RpatWqFqKgog8ebBAYGIjQ0FObm5hg3blyJGXhff/11bN26FTY2NmjVqhXi4+Nx5MgR3aPuldFma2trdOvWDUuWLEFBQQEaNGiAQ4cOldpz1759ewDA3LlzERQUhDp16qBfv366sPOo2bNnY+fOnXjttdcwbdo02NvbY8uWLUhOTsaXX35ZabMP169fH3PmzMGCBQsQEBCA/v37IykpCevWrUOHDh10Y8eOHj2KKVOmYOjQoXjhhRdQWFiIrVu3wsTEBIMHDwYALFy4EN9//z369u2Lxo0b4/bt21i3bh0aNmyoNwCaqKIYaoiekYODA/bv34933nkH7733Huzs7DBy5Ej07NlTN1+KsbVv3x4HDx7EzJkzMW/ePLi7u2PhwoW4ePHiU5/OqlOnDr7++mvd+Ahzc3O88cYbmDJlCtq0aVOh9iiVSuzbtw/BwcHYtm0bFAoF+vfvj+XLl6Nt27blPk5gYCDee+895Obm6j31VGTVqlUwMTHB9u3b8fDhQ3Tp0gVHjhyp0O/FkDbv2LEDU6dOxdq1ayGEQO/evXHw4EG9p88AoEOHDli0aBE2bNiA6OhoaLVaJCcnlxpqnJ2dceLECcyaNQsff/wxHj58CG9vb3z99de63pLKEhYWhvr162PNmjWYMWMG7O3tMWHCBHz44Ye6eZTatGkDf39/fP3110hJSYGlpSXatGmDgwcP6p786t+/P65du4bPPvsMd+/ehaOjI7p3744FCxbonp4iehYKUZ3+OUlEVWrgwIE4f/58qeM9iIhqGo6pIaolHl/S4PLly/jmm2/Qo0cP4zSIiOg5Y08NUS3h6uqqW4/o+vXrWL9+PfLy8vDzzz+jWbNmxm4eEdEz45gaoloiICAAO3fuRFpaGlQqFTp16oQPP/yQgYaIZIM9NURERCQLHFNDREREssBQQ0RERLJQa8bUaLVa/PHHH7CysjJoanIiIiIyHiEE7t+/Dzc3t6dOMllrQs0ff/wBd3d3YzeDiIiIKuDmzZto2LDhE+vUmlBjZWUFQPqhWFtbG7k1REREVB5ZWVlwd3fXfY8/Sa0JNUW3nKytrRlqiIiIapjyDB3hQGEiIiKSBYYaIiIikgWGGiIiIpKFWjOmpjyEECgsLIRGozF2U6iGMzExgampKacPICKqQgw1/5Ofn4/U1FTk5uYauykkE5aWlnB1dYWZmZmxm0JEVCsw1ECamC85ORkmJiZwc3ODmZkZ/4VNFSaEQH5+Pu7cuYPk5GQ0a9bsqRNGERHRs2OogdRLo9Vq4e7uDktLS2M3h2TAwsICderUwfXr15Gfnw9zc3NjN4mISPYq9M/HtWvXwsPDA+bm5vDz88PJkyfLrBsVFQVfX1/Y2tqibt268PHxwdatW/XqKBSKUl9Lly7V1fHw8CixPSIioiLNLxP/NU3PEz9PRERVy+Ceml27diEkJAQbNmyAn58fVq5cCX9/fyQlJcHJyalEfXt7e8ydOxctWrSAmZkZ9u/fj7Fjx8LJyQn+/v4AgNTUVL19Dh48iHHjxmHw4MF65QsXLsT48eN178szuyARERHVDgaHmhUrVmD8+PEYO3YsAGDDhg04cOAAPvvsM8yePbtE/R49eui9nz59OrZs2YIffvhBF2pcXFz06nz11Vd45ZVX0KRJE71yKyurEnWJiIiIAANvP+Xn5+P06dNQq9XFB1AqoVarER8f/9T9hRCIiYlBUlISunXrVmqd9PR0HDhwAOPGjSuxLSIiAg4ODmjbti2WLl2KwsLCMs+Vl5eHrKwsvVdl02iAuDhg507pvzXxyXAPDw+sXLmy3PXj4uKgUCiQkZFRaW0CgM2bN8PW1rZSz0FERDWbQT01d+/ehUajgbOzs165s7MzLl26VOZ+mZmZaNCgAfLy8mBiYoJ169ahV69epdbdsmULrKysMGjQIL3yadOmoV27drC3t8eJEycwZ84cpKamYsWKFaUeJzw8HAsWLDDk8p5JVBQwfTpw61ZxWcOGwKpVwGOX8lw87ems+fPnIywszODjnjp1CnXr1i13/c6dOyM1NRU2NjYGn4uIiOh5qpKnn6ysrJCYmIjs7GzExMQgJCQETZo0KXFrCgA+++wzjBgxosTTIiEhIbo/e3t7w8zMDG+//TbCw8OhUqlKHGfOnDl6+xSt8lkZoqKAIUMAIfTLU1Kk8i++eP7B5tFxSLt27UJoaCiSkpJ0ZfXq1dP9WQgBjUYDU9On/7rr169vUDvMzMx4S5CIqBbLywOOHgX27AEGDAD69jVeWwy6/eTo6AgTExOkp6frlaenpz/xi02pVKJp06bw8fHBO++8gyFDhiA8PLxEvWPHjiEpKQlvvfXWU9vi5+eHwsJCXLt2rdTtKpVKtyJ3Za7MrdFIPTSPBxqguCw4+PnfinJxcdG9bGxsoFAodO8vXboEKysrHDx4EO3bt4dKpcIPP/yA33//HQMGDICzszPq1auHDh064MiRI3rHffz2k0KhwH/+8x+88cYbsLS0RLNmzbBv3z7d9sdvPxXdJvr222/RsmVL1KtXDwEBAXohrLCwENOmTYOtrS0cHBwwa9YsjB49GgMHDjToZ7B+/Xp4eXnBzMwMzZs313uqTgiBsLAwNGrUCCqVCm5ubpg2bZpu+7p169CsWTOYm5vD2dkZQ4YMMejcRES1WVYWsGsXEBQE1K8P9OkDbNwIbNtm3HYZFGrMzMzQvn17xMTE6Mq0Wi1iYmLQqVOnch9Hq9UiLy+vRPmnn36K9u3bo02bNk89RmJiIpRKZalPXFWlY8f0bzk9Tgjg5k2pXlWbPXs2IiIicPHiRXh7eyM7Oxt9+vRBTEwMfv75ZwQEBKBfv364cePGE4+zYMECDBs2DGfPnkWfPn0wYsQI3Lt3r8z6ubm5WLZsGbZu3Yrvv/8eN27cwMyZM3XbFy9ejO3bt2PTpk04fvw4srKysHfvXoOubc+ePZg+fTreeecd/Prrr3j77bcxduxYxMbGAgC+/PJLfPTRR/j3v/+Ny5cvY+/evWjdujUA4KeffsK0adOwcOFCJCUlITo6uswxXkREJElPl4JLnz5SkAkKkoLN/fuAqyswcSLw9ttGbqQwUGRkpFCpVGLz5s3iwoULYsKECcLW1lakpaUJIYR48803xezZs3X1P/zwQ3Ho0CHx+++/iwsXLohly5YJU1NTsXHjRr3jZmZmCktLS7F+/foS5zxx4oT46KOPRGJiovj999/Ftm3bRP369cWoUaPK3e7MzEwBQGRmZpbY9uDBA3HhwgXx4MGDch+vyI4dQkjR5cmvHTsMPnS5bdq0SdjY2Ojex8bGCgBi7969T933xRdfFB9//LHufePGjcVHH32kew9AvPfee7r32dnZAoA4ePCg3rn++usvXVsAiCtXruj2Wbt2rXB2dta9d3Z2FkuXLtW9LywsFI0aNRIDBgwo9zV27txZjB8/Xq/O0KFDRZ8+fYQQQixfvly88MILIj8/v8SxvvzyS2FtbS2ysrLKPN/z8CyfKyKi6uD334VYtkyIl18WQqHQ/1574QUhZs0S4scfhdBoKq8NT/r+fpzBY2oCAwNx584dhIaGIi0tDT4+PoiOjtYNHr5x44bepGM5OTmYNGkSbt26BQsLC7Ro0QLbtm1DYGCg3nEjIyMhhMDw4cNLnFOlUiEyMhJhYWHIy8uDp6cnZsyYoTdmxlhcXZ9vvefJ19dX7312djbCwsJw4MABpKamorCwEA8ePHhqT423t7fuz3Xr1oW1tTVu375dZn1LS0t4eXnp3ru6uurqZ2ZmIj09HR07dtRtNzExQfv27aHVast9bRcvXsSECRP0yrp06YJVq1YBAIYOHYqVK1eiSZMmCAgIQJ8+fdCvXz+YmpqiV69eaNy4sW5bQECA7vYaEVFtJgTwyy/A3r3SGJmzZ/W3+/oCb7whvVq2NEoTn6hCA4WnTJmCKVOmlLotLi5O7/3777+P999//6nHnDBhQokvqSLt2rXDjz/+aHA7q0LXrtJTTikppY+rUSik7V27Vn3bHn+KaebMmTh8+DCWLVuGpk2bwsLCAkOGDEF+fv4Tj1OnTh299wqF4okBpLT6orQfTiVyd3dHUlISjhw5gsOHD2PSpElYunQpvvvuO1hZWeHMmTOIi4vDoUOHEBoairCwMJw6dYqPjRNRraPRACdOSCFm714gObl4m4kJ0L27FGIGDAAq6Xmb54bzuD8jExPpsW1ACjCPKnq/cqVUz9iOHz+OMWPG4I033kDr1q3h4uJS5kDrymJjYwNnZ2ecOnVKV6bRaHDmzBmDjtOyZUscP35cr+z48eNo1aqV7r2FhQX69euH1atXIy4uDvHx8Th37hwAwNTUFGq1GkuWLMHZs2dx7do1HD169BmujIio5nj4EDhwAHjrLelOQrduwEcfSYHGwgIYOBDYvFkaRxMTA0yZUv0DDcAFLZ+LQYOkx7ZLm6dm5crKmaemIpo1a4aoqCj069cPCoUC8+bNM+iWz/MydepUhIeHo2nTpmjRogU+/vhj/PXXXwatjP7uu+9i2LBhaNu2LdRqNb7++mtERUXpnubavHkzNBoN/Pz8YGlpiW3btsHCwgKNGzfG/v37cfXqVXTr1g12dnb45ptvoNVq0bx588q6ZCIio8vMBL75RuqROXgQyM4u3mZnB7z+utQj07s3YMB0ZdUKQ81zMmiQ1DV37BiQmiol365dq0cPTZEVK1bg73//Ozp37gxHR0fMmjWrSmZaftysWbOQlpaGUaNGwcTEBBMmTIC/vz9MDPhhDRw4EKtWrcKyZcswffp0eHp6YtOmTbq5j2xtbREREYGQkBBoNBq0bt0aX3/9NRwcHGBra4uoqCiEhYXh4cOHaNasGXbu3IkXX3yxkq6YiMg40tKAr76SbivFxAAFBcXbGjSQemTeeEPqqXls5ECNpBBVPdjBSLKysmBjY4PMzMwSc9Y8fPgQycnJ8PT0LDHpH1U+rVaLli1bYtiwYVi0aJGxm/Pc8HNFRMZw5UrxQN/4eP3xni1aFA/09fUtOWyiOnrS9/fj2FNDVe769es4dOgQunfvjry8PKxZswbJycn4v//7P2M3jYioxhECSEyUQsyePcCvv+pv79hRCjEDB0qhRs4YaqjKKZVKbN68GTNnzoQQAi+99BKOHDmCltXx+UAiomqosBA4frz4iaXr14u3mZoCPXpIIWbAAGl8Z23BUENVzt3dvcSTS0RE9GQPHwKHD0tB5uuvgbt3i7dZWAABAVKPzOuvSwN/ayOGGiIiomoqI0N69HrvXumJpZyc4m329kC/flKQ6dUL4PyhDDVERETVSmqq9MTSnj3S6teFhcXb3N2Ln1jq2lW61UTF+OMgIiIyssuXiwf6Pj6BfqtWxQN927evGU8sGQtDDRERURUQArh3T3rkuuh1+TJw5gxw8aJ+3b/9rbhH5oUXjNLcGomhhoiI6DkRArhzp2RwKfpzRkbp+5maAq+8UrzGkptblTZbNhhqiIiIDCCENFNvaaHlyhXg/v0n79+gAdCsGdC0qfR64QUp0HA93WfHUFPL9ejRAz4+Pli5ciUAwMPDA8HBwQgODi5zH4VCgT179mDgwIHPdO7ndZwnCQsLw969e5GYmFhp5yAi+dFqgT/+KDu45OaWva9CIQ3obdpUP7w0bQo0acKnlCoTQ00N1a9fPxQUFCA6OrrEtmPHjqFbt2745Zdf4O3tbdBxT506hbrPeSWzsoJFamoq7GrrZApEZHQajbQIcWnB5fffpXlhyqJUAo0blx5cPD0BroxiHAw1NdS4ceMwePBg3Lp1Cw0fmy5y06ZN8PX1NTjQAED9+vWfVxOfysXFpcrORUS1U2EhcONG6cHl6lUgP7/sfU1MpIBSWnDx8ADMzKrsMqiclMZuQHUlhDTJUVW/yru86Ouvv4769etj8+bNeuXZ2dn4/PPPMW7cOPz5558YPnw4GjRoAEtLS7Ru3Ro7d+584nE9PDx0t6IA4PLly+jWrRvMzc3RqlUrHD58uMQ+s2bNwgsvvABLS0s0adIE8+bNQ8H/loLdvHkzFixYgF9++QUKhQIKhULXZoVCgb179+qOc+7cObz66quwsLCAg4MDJkyYgOzsbN32MWPGYODAgVi2bBlcXV3h4OCAyZMn685VHlqtFgsXLkTDhg2hUqng4+Oj19uVn5+PKVOmwNXVFebm5mjcuDHCw8MBAEIIhIWFoVGjRlCpVHBzc8O0adPKfW4iqhwFBVJYOXgQ+PhjYNo0oE8faayKhQXg5QX4+wOTJwMrVwL79wOXLkmBpk4doHlzoG9fIDgYWLMGiI6WQs+DB8XHXb1a/7gMNNUTe2rKkJsL1KtX9efNzgbKc/fH1NQUo0aNwubNmzF37lwo/jdxweeffw6NRoPhw4cjOzsb7du3x6xZs2BtbY0DBw7gzTffhJeXFzp27PjUc2i1WgwaNAjOzs5ISEhAZmZmqWNtrKyssHnzZri5ueHcuXMYP348rKys8M9//hOBgYH49ddfER0djSNHjgAAbGxsShwjJycH/v7+6NSpE06dOoXbt2/jrbfewpQpU/SCW2xsLFxdXREbG4srV64gMDAQPj4+GD9+/NN/aABWrVqF5cuX49///jfatm2Lzz77DP3798f58+fRrFkzrF69Gvv27cPu3bvRqFEj3Lx5Ezdv3gQAfPnll/joo48QGRmJF198EWlpafjll1/KdV4iY0pJARISpNePP0o9FIA09qO8L6XSsPpV8Xr4ULpNdP26dCupLCqVFGxK63Fxd5d6ZEgmRC2RmZkpAIjMzMwS2x48eCAuXLggHjx4oCvLzhZC6jep2ld2dvmv6eLFiwKAiI2N1ZV17dpVjBw5ssx9+vbtK9555x3d++7du4vp06fr3jdu3Fh89NFHQgghvv32W2FqaipSUlJ02w8ePCgAiD179pR5jqVLl4r27dvr3s+fP1+0adOmRL1Hj/PJJ58IOzs7kf3ID+DAgQNCqVSKtLQ0IYQQo0ePFo0bNxaFhYW6OkOHDhWBgYFltuXxc7u5uYkPPvhAr06HDh3EpEmThBBCTJ06Vbz66qtCq9WWONby5cvFCy+8IPLz88s836NK+1wRVbbsbCG++06IJUuEGDxYiIYNjfN3WVW/LCyEaN1aiDfeEOLdd4X45BMhjh4V4sYNITQaY/9W6Fk86fv7ceypKYOlpdRrYozzlleLFi3QuXNnfPbZZ+jRoweuXLmCY8eOYeHChQAAjUaDDz/8ELt370ZKSgry8/ORl5cHy3Ke5OLFi3B3d4fbIxMmdOrUqUS9Xbt2YfXq1fj999+RnZ2NwsJCWFtbl/9C/neuNm3a6A1S7tKlC7RaLZKSkuDs7AwAePHFF2HyyD+rXF1dce7cuXKdIysrC3/88Qe6dOmiV96lSxddj8uYMWPQq1cvNG/eHAEBAXj99dfRu3dvAMDQoUOxcuVKNGnSBAEBAejTpw/69esHU85TTkai1QJJScU9MAkJwLlzJXstlEqgdWvAz0+a1O3FF6XeiadFBa228uPIs5yjTh3paaKmTQFXV860S7z9VCaFony3gYxt3LhxmDp1KtauXYtNmzbBy8sL3bt3BwAsXboUq1atwsqVK9G6dWvUrVsXwcHByH/SyDgDxcfHY8SIEViwYAH8/f1hY2ODyMhILF++/Lmd41F16tTRe69QKKDVap/b8du1a4fk5GQcPHgQR44cwbBhw6BWq/HFF1/A3d0dSUlJOHLkCA4fPoxJkyZh6dKl+O6770q0i6gy3LlTfBspIQE4eRLIzCxZz81NCi9FIaZ9+5rx9xnRs2KoqeGGDRuG6dOnY8eOHfjvf/+LiRMn6sbXHD9+HAMGDMDIkSMBSGNkfvvtN7Rq1apcx27ZsiVu3ryJ1NRUuLq6AgB+fGxRkhMnTqBx48aYO3euruz69et6dczMzKB50g3v/51r8+bNyMnJ0fXWHD9+HEqlEs2bNy9Xe5/G2toabm5uOH78uC74FZ3n0TFG1tbWCAwMRGBgIIYMGYKAgADcu3cP9vb2sLCwQL9+/dCvXz9MnjwZLVq0wLlz59CuXbvn0kaiInl5QGJicQ9MQkLxWJhHWVgAvr7FIcbPD3jsgUiiWoOhpoarV68eAgMDMWfOHGRlZWHMmDG6bc2aNcMXX3yBEydOwM7ODitWrEB6enq5Q41arcYLL7yA0aNHY+nSpcjKytILL0XnuHHjBiIjI9GhQwccOHAAe/bs0avj4eGB5ORkJCYmomHDhrCysoJKpdKrM2LECMyfPx+jR49GWFgY7ty5g6lTp+LNN9/U3Xp6Ht59913Mnz8fXl5e8PHxwaZNm5CYmIjt27cDAFasWAFXV1e0bdsWSqUSn3/+OVxcXGBra4vNmzdDo9HAz88PlpaW2LZtGywsLNC4cePn1j6qnYSQAsujg3kTE0t/3Lhly+Lw8re/AS+9xJWaiYrwfwUZGDduHD799FP06dNHb/zLe++9h6tXr8Lf3x+WlpaYMGECBg4ciMzS+qtLoVQqsWfPHowbNw4dO3aEh4cHVq9ejYCAAF2d/v37Y8aMGZgyZQry8vLQt29fzJs3D2FhYbo6gwcPRlRUFF555RVkZGRg06ZNeuELACwtLfHtt99i+vTp6NChAywtLTF48GCsWLHimX42j5s2bRoyMzPxzjvv4Pbt22jVqhX27duHZs2aAZCe5FqyZAkuX74MExMTdOjQAd988w2USiVsbW0RERGBkJAQaDQatG7dGl9//TUcHByeaxtJ/jIypFtHj95Kunu3ZD1Hx+Lw4ucHdOjAqfSJnkQhhBDGbkRVyMrKgo2NDTIzM0sMYn348CGSk5Ph6ekJc04DSc8JP1cESJO/nTunP5j30qWS9czMgLZt9XthPD05+JXoSd/fj2NPDRHRc3Trlv44mJ9+kiZxe1yTJvrjYHx8pPlUiKjiGGqIiCooOxs4fVo/xPzxR8l6NjZAx47FPTAdOwJVuCIJUa3BUENEVA5aLXDxov5g3l9/lcofZWIizQnzaC9M8+bSXDFEVLkYaoiInuDBA2DLFmDZMmlK/sc1bKg/DqZdO84JQ2QsDDWPqCVjpqmK8PNUs2VkAOvWAatWAbdvS2WWltITSEUhxs8PaNDAqM0kokcw1KB4ltrc3FxYWFgYuTUkF7m5uQBKzoJM1VtKirSS87//Ddy/L5U1agTMnAn8/e/shSGqzhhqAJiYmMDW1ha3//fPMUtLS92svESGEkIgNzcXt2/fhq2trd5aVVR9XboELF0KbN0KFBRIZS+9BMyaBQQGSusMEVH1xlDzPy4uLgCgCzZEz8rW1lb3uaLqKyEBWLwY2LtXmtkXALp2lcJMnz6cJ4aoJmGo+R+FQgFXV1c4OTmhoOifaUQVVKdOHfbQVGNCAN9+C0REAN99V1zev78UZjp3Nl7biKjiGGoeY2Jiwi8jIpkqLAR27waWLAF++UUqMzUFRo4E3n0XKOeyaERUTTHUEJHs5eYCmzZJj2VfuyaV1a0LTJgAzJgBuLsbtXlE9Jww1BCRbN27B6xdC6xeXbxgZP36wLRpwKRJgL29cdtHRM9Xhea4XLt2LTw8PGBubg4/Pz+cPHmyzLpRUVHw9fWFra0t6tatCx8fH2zdulWvzpgxY6BQKPRej64EDQD37t3DiBEjYG1tDVtbW4wbNw7Z2dkVaT4RydzNm0BIiPQodmioFGg8PIA1a6SemvfeY6AhkiODe2p27dqFkJAQbNiwAX5+fli5ciX8/f2RlJQEJyenEvXt7e0xd+5ctGjRAmZmZti/fz/Gjh0LJycn+Pv76+oFBARg06ZNuveqx1Z2GzFiBFJTU3H48GEUFBRg7NixmDBhAnbs2GHoJRCRTF24II2X2b5dGj8DAN7ewOzZwNCh0vgZIpIvhTBw2lM/Pz906NABa9asAQBotVq4u7tj6tSpmD17drmO0a5dO/Tt2xeLFi0CIPXUZGRkYO/evaXWv3jxIlq1aoVTp07B19cXABAdHY0+ffrg1q1bcHNze+o5DVm6nIhqlhMnpMey9+0rLuvRQ3qSyd+fj2UT1WSGfH8bdPspPz8fp0+fhlqtLj6AUgm1Wo34+Pin7i+EQExMDJKSktCtWze9bXFxcXByckLz5s0xceJE/Pnnn7pt8fHxsLW11QUaAFCr1VAqlUhISCj1XHl5ecjKytJ7EZF8CAEcOAB06wZ06SIFGoUCeOMNabHJ2FggIICBhqg2Magz9u7du9BoNHB2dtYrd3Z2xqVLl8rcLzMzEw0aNEBeXh5MTEywbt069OrVS7c9ICAAgwYNgqenJ37//Xf861//wmuvvYb4+HiYmJggLS2txK0tU1NT2NvbIy0trdRzhoeHY8GCBYZcHhHVAAUFQGSkdJvp11+lsjp1gFGjpMeymzc3bvuIyHiq5A6zlZUVEhMTkZ2djZiYGISEhKBJkybo0aMHACAoKEhXt3Xr1vD29oaXlxfi4uLQs2fPCp1zzpw5CAkJ0b3PysqCO5/bJKqxcnKATz8Fli8HbtyQyurVA/7xDyA4mAtLEpGBocbR0REmJiZIT0/XK09PT3/idPBKpRJNmzYFAPj4+ODixYsIDw/XhZrHNWnSBI6Ojrhy5Qp69uwJFxeXEssXFBYW4t69e2WeV6VSlRhsTEQ1z9270lNLa9YARXelnZykIDNxImBra8zWEVF1YtCYGjMzM7Rv3x4xMTG6Mq1Wi5iYGHTq1Kncx9FqtcjLyytz+61bt/Dnn3/C1dUVANCpUydkZGTg9OnTujpHjx6FVquFn5+fIZdARDXE9evA9OlA48bAggVSoGnSBFi/Xnose84cBhoi0mfw7aeQkBCMHj0avr6+6NixI1auXImcnByMHTsWADBq1Cg0aNAA4eHhAKSxLb6+vvDy8kJeXh6++eYbbN26FevXrwcAZGdnY8GCBRg8eDBcXFzw+++/45///CeaNm2qe+S7ZcuWCAgIwPjx47FhwwYUFBRgypQpCAoKKteTT0RUc/z6qzReZscOQKORytq2lR7LHjwY4ComRFQWg0NNYGAg7ty5g9DQUKSlpcHHxwfR0dG6wcM3btyAUlncAZSTk4NJkybh1q1bsLCwQIsWLbBt2zYEBgYCkNZaOnv2LLZs2YKMjAy4ubmhd+/eWLRokd7to+3bt2PKlCno2bMnlEolBg8ejNWrVz/r9RNRNfHDD9ICkwcOFJf17Ck9lq1W8ykmIno6g+epqak4Tw1R9aPVAvv3S3PMnDghlSkUUo/MrFnAI7M4EFEtZcj3N+fXJKIql58P7Nwp3Wa6cEEqMzMDRo8GZs4EXnjBuO0jopqJoYaIqkx2NrBxI7BiBXDrllRmbS09xTR9OvC/ZwOIiCqEoYaIKt2dO8DHH0uPZf/1l1Tm4iI9lv2PfwA2NkZtHhHJBEMNEVWKwkLgu++k2X+3bwcePJDKmzWTZv59803A3Ny4bSQieWGoIaLnRqMBvv8e2L0b+PJLqYemiK+vNPj3jTf4WDYRVQ6GGiJ6JhoNcPw4sGuXFGQenXDcwQEYNAgYMUJaeJKPZRNRZWKoISKDabXSI9i7dwNffAGkphZvs7OTgsywYcArr0iLTRIRVQWGGiIqF60W+PFHKch8/jnwxx/F22xtpdtKw4YBr74qPZ5NRFTVGGqIqExCAAkJxUGm6DFsQHoUe+BAKcj06sUgQ0TGx1BDRHqEAE6dKg4yN24Ub7OyAgYMkIJM797AIyuZEBEZHUMNEUEI4MwZKcjs3i2tgl2kXj2gf38pyPj78zFsIqq+GGqIaikhgMTE4iBz9Wrxtrp1gX79pCATEABYWBitmURE5cZQQ1SLCAGcPVscZK5cKd5maQm8/roUZF57TXpPRFSTMNQQyZwQwK+/FgeZ334r3mZuDvTtKwWZvn2lHhoiopqKoYZIps6fLw4yly4Vl6tUQJ8+UpB5/XVpzAwRkRww1BDJyKVLxUHm/PnicjMz6ZbSsGHSWBkrK+O1kYiosjDUENVwv/1WHGTOnSsur1NHelopMFAKMlwJm4jkjqGGqAa6cqU4yPzyS3G5qak0f8ywYdJ8Mra2RmsiEVGVY6ghqiGuXi0OMj//XFxuagqo1VKQGThQWnuJiKg2YqghqsauXZNm9d29G/jpp+JyExOgZ8/iIOPgYKwWEhFVHww1RNVMSooUYiIjgZMni8uVSmmxyGHDpMUjHR2N10YiouqIoYaoGrhzB/jySynIfP+9NLcMIAWZHj2Kg4yTk1GbSURUrTHUEBlJZiawdy+wcydw5Aig0RRv69IFGD4cGDwYcHExWhOJiGoUhhqiKpSbC+zfLwWZb74B8vOLt7VrJwWZYcOARo2M10YiopqKoYaokuXlAYcOSUFm3z4gJ6d4W8uWUpAJDAReeMF4bSQikgOGGqJKUFgIxMVJQSYqCsjIKN7m6QkEBUmv1q0BhcJYrSQikheGGqLnRKsFTpyQBvt+/jlw+3bxNldXqTcmKAjo2JFBhoioMjDUED0DIYAzZ6Qgs2sXcPNm8TYHB2DIEOn20ssvS3PLEBFR5WGoIaqACxekIBMZCVy+XFxuZSU9ej18uDQ5Xp06xmsjEVFtw1BDVE5Xr0q9MZGRwNmzxeUWFtKCkUFB0krY5ubGayMRUW3GUEP0BCkp0viYnTv1Z/etUwcICJCCTL9+Ug8NEREZF0MN0WPu3pVm9925s+Tsvq++KgWZN94A7O2N204iItLHUEOE4tl9IyOBw4dLzu4bFCQN+uXsvkRE1RdDDdVaRbP7RkZKs/vm5RVva9dOCjLDhgGNGxuvjUREVH4MNVSr5OcD334rBZmvvtKf3bdFC+mppaAgzu5LRFQTMdSQ7BXN7hsZKY2VeXR2Xw8PKcQMH87ZfYmIajplRXZau3YtPDw8YG5uDj8/P5x89LGQx0RFRcHX1xe2traoW7cufHx8sHXrVt32goICzJo1C61bt0bdunXh5uaGUaNG4Y8//tA7joeHBxQKhd4rIiKiIs2nWkCrBY4fB6ZOBRo0AHr1Aj79VAo0rq5AcDDw44/SY9rh4YC3NwMNEVFNZ3BPza5duxASEoINGzbAz88PK1euhL+/P5KSkuDk5FSivr29PebOnYsWLVrAzMwM+/fvx9ixY+Hk5AR/f3/k5ubizJkzmDdvHtq0aYO//voL06dPR//+/fHTTz/pHWvhwoUYP3687r0Vn6Olx6SlAatXA9u3AzduFJcXze4bFAR07crZfYmI5EghRNEDq+Xj5+eHDh06YM2aNQAArVYLd3d3TJ06FbNnzy7XMdq1a4e+ffti0aJFpW4/deoUOnbsiOvXr6NRo0YApJ6a4OBgBAcHG9JcnaysLNjY2CAzMxPW1tYVOgZVXzdvAkuWABs3Fg/4LZrdNygIUKs5uy8RUU1kyPe3Qbef8vPzcfr0aajV6uIDKJVQq9WIj49/6v5CCMTExCApKQndunUrs15mZiYUCgVsbW31yiMiIuDg4IC2bdti6dKlKCwsLPMYeXl5yMrK0nuR/Fy9CkyYAHh5AWvWSIGmUyfgiy+kBSW3bJFm+WWgISKSP4NuP929excajQbOzs565c7Ozrh06VKZ+2VmZqJBgwbIy8uDiYkJ1q1bh169epVa9+HDh5g1axaGDx+ul8imTZuGdu3awd7eHidOnMCcOXOQmpqKFStWlHqc8PBwLFiwwJDLoxrk0iVpLMz27cVzyrzyCjBvHtCjB8fHEBHVRlXy9JOVlRUSExORnZ2NmJgYhISEoEmTJujRo4devYKCAgwbNgxCCKxfv15vW0hIiO7P3t7eMDMzw9tvv43w8HCoVKoS55wzZ47ePllZWXB3d3++F0ZV7uxZ4IMPpKULim6cvvYaMHeuNEkeERHVXgaFGkdHR5iYmCA9PV2vPD09HS5PmGpVqVSiadOmAAAfHx9cvHgR4eHheqGmKNBcv34dR48efep9Mz8/PxQWFuLatWto3rx5ie0qlarUsEM1008/Ae+/L80tU2TgQCnM+PoarVlERFSNGDSmxszMDO3bt0dMTIyuTKvVIiYmBp06dSr3cbRaLfIemb61KNBcvnwZR44cgYODw1OPkZiYCKVSWeoTVyQfx49LPTEdOkiBRqEAAgOlHps9exhoiIiomMG3n0JCQjB69Gj4+vqiY8eOWLlyJXJycjB27FgAwKhRo9CgQQOEh4cDkMa2+Pr6wsvLC3l5efjmm2+wdetW3e2lgoICDBkyBGfOnMH+/fuh0WiQlpYGQHoc3MzMDPHx8UhISMArr7wCKysrxMfHY8aMGRg5ciTs7Oye18+CqgkhgNhYYNEiadI8QHoEe8QI4F//AkrpmCMiIjI81AQGBuLOnTsIDQ1FWloafHx8EB0drRs8fOPGDSiVxR1AOTk5mDRpEm7dugULCwu0aNEC27ZtQ2BgIAAgJSUF+/btAyDdmnpUbGwsevToAZVKhcjISISFhSEvLw+enp6YMWOG3pgZqvmEAKKjpdtMJ05IZXXqAGPGALNnA02aGLV5RERUzRk8T01NxXlqqi+tFti3Twozp09LZebmwPjxwLvvAhzfTURUexny/c21n8hoNBrpKaYPPgB+/VUqq1sXmDgReOcd4Aljz4mIiEpgqKEqV1AA7NgBfPgh8NtvUpm1tbROU3Aw4Oho1OYREVENxVBDVSYvT5rhNyICSE6WyuztpSAzdSrw2ATSREREBmGooUr34AHwn/9IazPduiWVOTkBM2cC//iHtEYTERHRs2KooUqTnQ2sXw8sXw4UzdfYoAHwz38Cb70FWFoat31ERCQvDDX03GVkSItLfvQRcO+eVObhIT2WPWYMwImeiYioMjDU0HNz9y6wahWwejVQtCh6s2bShHkjRnClbCIiqlwMNfTM0tKkW0zr1wM5OVLZiy9K6zINGybNBkxERFTZGGqowm7eBJYuBTZuBB4+lMratgXmzQMGDACUBq0sRkRE9GwYashgV68CixcDmzZJc84AwN/+JoWZ116TFp0kIiKqagw1VG5JSUB4OLBtmzQbMAB07y6FmVdfZZghIiLjYqihpzp3TlrKYPduadFJAPD3l8bMdO1q3LYREREVYaihMp0+LS0yuXdvcVn//lKY6djRaM0iIiIqFUMNlXDihBRmDh6U3isUwJAhUphp08a4bSMiIioLQw0BkG4rxcUBixYBsbFSmYkJ8H//B8yZA7RsadTmERERPRVDTS0nhNQj88EHUg8NIE2SN3q0NAOwl5dx20dERFReDDW1lFYL7NkjhZmff5bKVCpg3Dhg1iygUSPjto+IiMhQDDW1TGEhsGsX8OGHwIULUlndusDEiUBICODqatz2ERERVRRDTS2Rnw/897/SPDNXr0plNjbA1KnA9OmAo6Nx20dERPSsGGpk7sED4D//AZYsAW7dksocHKRemcmTpWBDREQkBww1MnX/vrTA5PLlwO3bUpmrK/Duu8CECdItJyIiIjlhqJGZv/4CVq8GVq2S/gwAjRtLTzKNGQOYmxu1eURERJWGoUYmbt8GPvoIWLtW6qUBgBdeAP71L2mumTp1jNs+IiKiysZQU8PdugUsWwZ88ok0fgYAvL2lMDNkiDSBHhERUW3AUFNDXb0KLF4MbNoEFBRIZR07Au+9B7z+OlfMJiKi2oehpoa5eFF6LHvHDkCjkcq6d5fWZVKrGWaIiKj2YqipIRITpdl/v/xSWtoAAAICpDDz8stGbRoREVG1wFBTzcXHS2HmwIHisoEDpTDj62u0ZhEREVU7DDXVUNGK2e+/Dxw9KpUplUBgoDQA+KWXjNo8IiKiaomhphopbcVsU1Ng1ChpnplmzYzbPiIiouqMoaYa0GqBvXulnplHV8x+6y1pBuDGjY3aPCIiohqBocaIuGI2ERHR88NQYwRcMZuIiOj5Y6ipQqWtmO3oCMyYwRWziYiInhVDTRUoWjF7xQogPV0q44rZREREzxdDTSXiitlERERVh6GmEnDFbCIioqqnrMhOa9euhYeHB8zNzeHn54eTJ0+WWTcqKgq+vr6wtbVF3bp14ePjg61bt+rVEUIgNDQUrq6usLCwgFqtxuXLl/Xq3Lt3DyNGjIC1tTVsbW0xbtw4ZGdnV6T5lebWLSA4GPDwACIipEDj7S094XThAjB6NAMNERFRZTE41OzatQshISGYP38+zpw5gzZt2sDf3x+3b98utb69vT3mzp2L+Ph4nD17FmPHjsXYsWPx7bff6uosWbIEq1evxoYNG5CQkIC6devC398fDx8+1NUZMWIEzp8/j8OHD2P//v34/vvvMWHChApc8vN39Srw9ttAkybSraYHD6QVs/ftk9ZsGjYMMDExdiuJiIhkThioY8eOYvLkybr3Go1GuLm5ifDw8HIfo23btuK9994TQgih1WqFi4uLWLp0qW57RkaGUKlUYufOnUIIIS5cuCAAiFOnTunqHDx4UCgUCpGSklKuc2ZmZgoAIjMzs9ztLI/Fi4UwMRFCmg9YiO7dhTh0SAit9rmehoiIqFYy5PvboJ6a/Px8nD59Gmq1WlemVCqhVqsRHx9fngCFmJgYJCUloVu3bgCA5ORkpKWl6R3TxsYGfn5+umPGx8fD1tYWvo+s4KhWq6FUKpGQkFDqufLy8pCVlaX3qgzt2gEajbRi9rFj0ppNvXoBCkWlnI6IiIjKYNBA4bt370Kj0cDZ2Vmv3NnZGZcuXSpzv8zMTDRo0AB5eXkwMTHBunXr0KtXLwBAWlqa7hiPH7NoW1paGpycnPQbbmoKe3t7XZ3HhYeHY8GCBYZcXoX07AmcO8dFJomIiIytQgOFDWVlZYXExEScOnUKH3zwAUJCQhAXF1ep55wzZw4yMzN1r5s3b1bKeRQKBhoiIqLqwKCeGkdHR5iYmCC9aAa5/0lPT4eLi0uZ+ymVSjRt2hQA4OPjg4sXLyI8PBw9evTQ7Zeeng7XRxY7Sk9Ph4+PDwDAxcWlxEDkwsJC3Lt3r8zzqlQqqFQqQy6PiIiIajCDemrMzMzQvn17xMTE6Mq0Wi1iYmLQqVOnch9Hq9UiLy8PAODp6QkXFxe9Y2ZlZSEhIUF3zE6dOiEjIwOnT5/W1Tl69Ci0Wi38/PwMuQQiIiKSKYMn3wsJCcHo0aPh6+uLjh07YuXKlcjJycHYsWMBAKNGjUKDBg0QHh4OQBrb4uvrCy8vL+Tl5eGbb77B1q1bsX79egCAQqFAcHAw3n//fTRr1gyenp6YN28e3NzcMHDgQABAy5YtERAQgPHjx2PDhg0oKCjAlClTEBQUBDc3t+f0oyAiIqKazOBQExgYiDt37iA0NBRpaWnw8fFBdHS0bqDvjRs3oFQWdwDl5ORg0qRJuHXrFiwsLNCiRQts27YNgYGBujr//Oc/kZOTgwkTJiAjIwMvv/wyoqOjYf7IOgLbt2/HlClT0LNnTyiVSgwePBirV69+lmsnIiIiGVEIIYSxG1EVsrKyYGNjg8zMTFhbWxu7OURERFQOhnx/V8nTT0RERESVjaGGiIiIZIGhhoiIiGSBoYaIiIhkgaGGiIiIZIGhhoiIiGSBoYaIiIhkgaGGiIiIZIGhhoiIiGSBoYaIiIhkgaGGiIiIZIGhhoiIiGSBoYaIiIhkgaGGiIiIZIGhhoiIiGSBoYaIiIhkgaGGiIiIZIGhhoiIiGSBoYaIiIhkgaGGiIiIZIGhhoiIiGSBoYaIiIhkgaGGiIiIZIGhhoiIiGSBoYaIiIhkgaGGiIiIZIGhhoiIiGSBoYaIiIhkgaGGiIiIZIGhhoiIiGSBoYaIiIhkgaGGiIiIZIGhhoiIiGSBoYaIiIhkgaGGiIiIZIGhhoiIiGSBoYaIiIhkoUKhZu3atfDw8IC5uTn8/Pxw8uTJMutu3LgRXbt2hZ2dHezs7KBWq0vUVygUpb6WLl2qq+Ph4VFie0REREWaT0RERDJkcKjZtWsXQkJCMH/+fJw5cwZt2rSBv78/bt++XWr9uLg4DB8+HLGxsYiPj4e7uzt69+6NlJQUXZ3U1FS912effQaFQoHBgwfrHWvhwoV69aZOnWpo84mIiEimFEIIYcgOfn5+6NChA9asWQMA0Gq1cHd3x9SpUzF79uyn7q/RaGBnZ4c1a9Zg1KhRpdYZOHAg7t+/j5iYGF2Zh4cHgoODERwcXK525uXlIS8vT/c+KysL7u7uyMzMhLW1dbmOQURERMaVlZUFGxubcn1/G9RTk5+fj9OnT0OtVhcfQKmEWq1GfHx8uY6Rm5uLgoIC2Nvbl7o9PT0dBw4cwLhx40psi4iIgIODA9q2bYulS5eisLCwzPOEh4fDxsZG93J3dy9X+4iIiKhmMjWk8t27d6HRaODs7KxX7uzsjEuXLpXrGLNmzYKbm5teMHrUli1bYGVlhUGDBumVT5s2De3atYO9vT1OnDiBOXPmIDU1FStWrCj1OHPmzEFISIjufVFPDREREcmTQaHmWUVERCAyMhJxcXEwNzcvtc5nn32GESNGlNj+aEDx9vaGmZkZ3n77bYSHh0OlUpU4jkqlKrWciIiI5Mmg20+Ojo4wMTFBenq6Xnl6ejpcXFyeuO+yZcsQERGBQ4cOwdvbu9Q6x44dQ1JSEt56662ntsXPzw+FhYW4du1audtPRERE8mVQqDEzM0P79u31BvBqtVrExMSgU6dOZe63ZMkSLFq0CNHR0fD19S2z3qeffor27dujTZs2T21LYmIilEolnJycDLkEIiIikimDbz+FhIRg9OjR8PX1RceOHbFy5Urk5ORg7NixAIBRo0ahQYMGCA8PBwAsXrwYoaGh2LFjBzw8PJCWlgYAqFevHurVq6c7blZWFj7//HMsX768xDnj4+ORkJCAV155BVZWVoiPj8eMGTMwcuRI2NnZVejCiYiISF4MDjWBgYG4c+cOQkNDkZaWBh8fH0RHR+sGD9+4cQNKZXEH0Pr165Gfn48hQ4boHWf+/PkICwvTvY+MjIQQAsOHDy9xTpVKhcjISISFhSEvLw+enp6YMWOG3jgbIiIiqt0MnqempjLkOXciIiKqHiptnhoiIiKi6oqhhoiIiGSBoYaIiIhkgaGGiIiIZIGhhoiIiGSBoYaIiIhkgaGGiIiIZIGhhoiIiGSBoYaIiIhkgaGGiIiIZIGhhoiIiGSBoYaIiIhkgaGGiIiIZIGhhoiIiGSBoYaIiIhkgaGGiIiIZIGhhoiIiGSBoYaIiIhkgaGGiIiIZIGhhoiIiGSBoYaIiIhkgaGGiIiIZIGhhoiIiGSBoYaIiIhkgaGGiIiIZIGhhoiIiGSBoYaIiIhkgaGGiIiIZIGhhoiIiGSBoYaIiIhkgaGGiIiIZIGhhoiIiGSBoYaIiIhkgaGGiIiIZIGhhoiIiGSBoYaIiIhkoUKhZu3atfDw8IC5uTn8/Pxw8uTJMutu3LgRXbt2hZ2dHezs7KBWq0vUHzNmDBQKhd4rICBAr869e/cwYsQIWFtbw9bWFuPGjUN2dnZFmk9EREQyZHCo2bVrF0JCQjB//nycOXMGbdq0gb+/P27fvl1q/bi4OAwfPhyxsbGIj4+Hu7s7evfujZSUFL16AQEBSE1N1b127typt33EiBE4f/48Dh8+jP379+P777/HhAkTDG0+ERERyZRCCCEM2cHPzw8dOnTAmjVrAABarRbu7u6YOnUqZs+e/dT9NRoN7OzssGbNGowaNQqA1FOTkZGBvXv3lrrPxYsX0apVK5w6dQq+vr4AgOjoaPTp0we3bt2Cm5vbU8+blZUFGxsbZGZmwtraupxXS0RERMZkyPe3QT01+fn5OH36NNRqdfEBlEqo1WrEx8eX6xi5ubkoKCiAvb29XnlcXBycnJzQvHlzTJw4EX/++aduW3x8PGxtbXWBBgDUajWUSiUSEhJKPU9eXh6ysrL0XkRERCRfBoWau3fvQqPRwNnZWa/c2dkZaWlp5TrGrFmz4ObmpheMAgIC8N///hcxMTFYvHgxvvvuO7z22mvQaDQAgLS0NDg5Oekdx9TUFPb29mWeNzw8HDY2NrqXu7u7IZdKRERENYxpVZ4sIiICkZGRiIuLg7m5ua48KChI9+fWrVvD29sbXl5eiIuLQ8+ePSt0rjlz5iAkJET3Pisri8GGiIhIxgzqqXF0dISJiQnS09P1ytPT0+Hi4vLEfZctW4aIiAgcOnQI3t7eT6zbpEkTODo64sqVKwAAFxeXEgORCwsLce/evTLPq1KpYG1trfciIiIi+TIo1JiZmaF9+/aIiYnRlWm1WsTExKBTp05l7rdkyRIsWrQI0dHReuNiynLr1i38+eefcHV1BQB06tQJGRkZOH36tK7O0aNHodVq4efnZ8glEBERkUwZ/Eh3SEgINm7ciC1btuDixYuYOHEicnJyMHbsWADAqFGjMGfOHF39xYsXY968efjss8/g4eGBtLQ0pKWl6eaYyc7Oxrvvvosff/wR165dQ0xMDAYMGICmTZvC398fANCyZUsEBARg/PjxOHnyJI4fP44pU6YgKCioXE8+ERERkfwZPKYmMDAQd+7cQWhoKNLS0uDj44Po6Gjd4OEbN25AqSzOSuvXr0d+fj6GDBmid5z58+cjLCwMJiYmOHv2LLZs2YKMjAy4ubmhd+/eWLRoEVQqla7+9u3bMWXKFPTs2RNKpRKDBw/G6tWrK3rdREREJDMGz1NTU3GeGiIiopqn0uapISIiIqquGGqIiIhIFhhqiIiISBYYaoiIiEgWGGqIiIhIFhhqiIiISBYYaoiIiEgWGGqIiIhIFhhqiIiISBYYaoiIiEgWGGqIiIhIFhhqiIiISBYYaoiIiEgWGGqIiIhIFhhqiIiISBYYaoiIiEgWGGqIiIhIFhhqiIiISBYYaoiIiEgWGGqIiIhIFhhqiIiISBYYaoiIiEgWGGqIiIhIFkyN3YCaTqMBjh0DUlMBV1ega1fAxMTYrSIiIqp9GGqeQVQUMH06cOtWcVnDhsCqVcCgQcZrFxERUW3E208VFBUFDBmiH2gAICVFKo+KMk67iIiIaiuGmgrQaKQeGiFKbisqCw6W6hEREVHVYKipgGPHSvbQPEoI4OZNqR4RERFVDYaaCkhNfb71iIiI6Nkx1FSAq+vzrUdERETPjqGmArp2lZ5yUihK365QAO7uUj0iIiKqGgw1FWBiIj22DZQMNkXvV67kfDVERERViaGmggYNAr74AmjQQL+8YUOpnPPUEBERVS1OvvcMBg0CBgzgjMJERETVAUPNMzIxAXr0MHYriIiIiLefiIiISBYYaoiIiEgWKhRq1q5dCw8PD5ibm8PPzw8nT54ss+7GjRvRtWtX2NnZwc7ODmq1Wq9+QUEBZs2ahdatW6Nu3bpwc3PDqFGj8Mcff+gdx8PDAwqFQu8VERFRkeYTERGRDBkcanbt2oWQkBDMnz8fZ86cQZs2beDv74/bt2+XWj8uLg7Dhw9HbGws4uPj4e7ujt69eyMlJQUAkJubizNnzmDevHk4c+YMoqKikJSUhP79+5c41sKFC5Gamqp7TZ061dDmExERkUwphChtWcay+fn5oUOHDlizZg0AQKvVwt3dHVOnTsXs2bOfur9Go4GdnR3WrFmDUaNGlVrn1KlT6NixI65fv45GjRoBkHpqgoODERwcXK525uXlIS8vT/c+KysL7u7uyMzMhLW1dbmOQURERMaVlZUFGxubcn1/G9RTk5+fj9OnT0OtVhcfQKmEWq1GfHx8uY6Rm5uLgoIC2Nvbl1knMzMTCoUCtra2euURERFwcHBA27ZtsXTpUhQWFpZ5jPDwcNjY2Ohe7u7u5WofERER1UwGPdJ99+5daDQaODs765U7Ozvj0qVL5TrGrFmz4ObmpheMHvXw4UPMmjULw4cP10tk06ZNQ7t27WBvb48TJ05gzpw5SE1NxYoVK0o9zpw5cxASEqJ7X9RTQ0RERPJUpfPUREREIDIyEnFxcTA3Ny+xvaCgAMOGDYMQAuvXr9fb9mhA8fb2hpmZGd5++22Eh4dDpVKVOJZKpSq1nIiIiOTJoNtPjo6OMDExQXp6ul55eno6XFxcnrjvsmXLEBERgUOHDsHb27vE9qJAc/36dRw+fPip9838/PxQWFiIa9euGXIJREREJFMGhRozMzO0b98eMTExujKtVouYmBh06tSpzP2WLFmCRYsWITo6Gr6+viW2FwWay5cv48iRI3BwcHhqWxITE6FUKuHk5GTIJRAREZFMGXz7KSQkBKNHj4avry86duyIlStXIicnB2PHjgUAjBo1Cg0aNEB4eDgAYPHixQgNDcWOHTvg4eGBtLQ0AEC9evVQr149FBQUYMiQIThz5gz2798PjUajq2Nvbw8zMzPEx8cjISEBr7zyCqysrBAfH48ZM2Zg5MiRsLOze14/CyIiIqrBDA41gYGBuHPnDkJDQ5GWlgYfHx9ER0frBg/fuHEDSmVxB9D69euRn5+PIUOG6B1n/vz5CAsLQ0pKCvbt2wcA8PHx0asTGxuLHj16QKVSITIyEmFhYcjLy4OnpydmzJihN86GiIiIajeD56mpqQx5zp2IiIiqh0qbp4aIiIioumKoISIiIlmo0nlqqPrSaIBjx4DUVMDVFejaFTAxMXariIiIyo+hhhAVBUyfDty6VVzWsCGwahUwaJDx2kVERGQI3n6q5aKigCFD9AMNAKSkSOVRUcZpFxERkaEYamoxjUbqoSnt+beisuBgqR4REVF1x1BTix07VrKH5lFCADdvSvWIiIiqO4aaWiw19fnWIyIiMiaGmlrM1fX51iMiIjImhpparGtX6SknhaL07QoF4O4u1SMiIqruGGpqMRMT6bFtoGSwKXq/ciXnqyEiopqBoaaWGzQI+OILoEED/fKGDaVyzlNDREQ1BSffIwwaBAwYII8ZhTkzMhFR7cVQQwCkL/4ePYzdimfDmZGJiGo33n4iWeDMyERPp9EAcXHAzp3SfzmxJskNQw3VeJwZmejpoqIADw/glVeA//s/6b8eHgz8JC8MNVTjcWZkoidjTybVFgw1VONxZmSisrEnk2oThhqq8TgzMlHZ2JNJtQlDDdV4nBmZqGzsyaTahKGGajzOjExUNvZkUm3CUEOywJmRiUrHnkyqTTj5HsmGnGZGJnpeinoyhwyRAsyjA4bZk0lyw1BDsiKHmZGJnreinszSZtxeuZI9mSQfDDVE1RDXsKLnjT2ZVBsw1BBVM1zDiioLezJJ7jhQmKgakdvMr1xriIiqEkMNUTUht5lfudYQEVU1hhqiakJOM7+yx4no6fi5ev4YaoiqCbnM/MoeJ6Kn4+eqcjDUEFUTcpn5lT1ORE/Gz1XlYaghqibkMvMre5yIysbPVeViqCGqJuSyhhV7nIjKxs9V5WKoIapG5LCGFXuciMrGz1Xl4uR7RNVMTZ/5VS5rDcmlx4mqF36uKpdCiNLu7MlPVlYWbGxskJmZCWtra2M3h0j2SpsZ2d295qw1pNFIT6OkpJQ+/kGhkHqkkpOrf0Cj6oOfK8MZ8v1dodtPa9euhYeHB8zNzeHn54eTJ0+WWXfjxo3o2rUr7OzsYGdnB7VaXaK+EAKhoaFwdXWFhYUF1Go1Ll++rFfn3r17GDFiBKytrWFra4tx48YhOzu7Is0noiowaBBw7RoQGwvs2CH9Nzm5ZgQaQD5jnKh64eeqchkcanbt2oWQkBDMnz8fZ86cQZs2beDv74/bt2+XWj8uLg7Dhw9HbGws4uPj4e7ujt69eyMlJUVXZ8mSJVi9ejU2bNiAhIQE1K1bF/7+/nj48KGuzogRI3D+/HkcPnwY+/fvx/fff48JEyZU4JKJqKoUrTU0fLj035r2F7UcxjhR9SPHz1W1mUhQGKhjx45i8uTJuvcajUa4ubmJ8PDwcu1fWFgorKysxJYtW4QQQmi1WuHi4iKWLl2qq5ORkSFUKpXYuXOnEEKICxcuCADi1KlTujoHDx4UCoVCpKSklOu8mZmZAoDIzMwsV30ioiKFhULExgqxY4f038JCY7eI5EAun6svvxSiYUMhpBtq0qthQ6n8eTDk+9ugnpr8/HycPn0aarVaV6ZUKqFWqxEfH1+uY+Tm5qKgoAD29vYAgOTkZKSlpekd08bGBn5+frpjxsfHw9bWFr6+vro6arUaSqUSCQkJpZ4nLy8PWVlZei8iooqo6T1OVD3J4XNV3SYSNCjU3L17FxqNBs7Oznrlzs7OSEtLK9cxZs2aBTc3N12IKdrvScdMS0uDk5OT3nZTU1PY29uXed7w8HDY2NjoXu7u7uVqHxERVW/V5lZHLVcdJxKs0nlqIiIiEBkZiT179sDc3LxSzzVnzhxkZmbqXjdv3qzU8xERUeXjmknVR3WcSNCgUOPo6AgTExOkp6frlaenp8PFxeWJ+y5btgwRERE4dOgQvL29deVF+z3pmC4uLiUGIhcWFuLevXtlnlelUsHa2lrvRURENVd1u9VR21XHiQQNCjVmZmZo3749YmJidGVarRYxMTHo1KlTmfstWbIEixYtQnR0tN64GADw9PSEi4uL3jGzsrKQkJCgO2anTp2QkZGB06dP6+ocPXoUWq0Wfn5+hlwCERHVQNXxVkdtVy0nEjR0FHJkZKRQqVRi8+bN4sKFC2LChAnC1tZWpKWlCSGEePPNN8Xs2bN19SMiIoSZmZn44osvRGpqqu51//59vTq2trbiq6++EmfPnhUDBgwQnp6e4sGDB7o6AQEBom3btiIhIUH88MMPolmzZmL48OHlbjeffiIiqrliY/WfrinrFRtr7JbWHoWF0lNOCkXpvwuFQgh392d/qsuQ72+Dl0kIDAzEnTt3EBoairS0NPj4+CA6Olo30PfGjRtQKos7gNavX4/8/HwMGTJE7zjz589HWFgYAOCf//wncnJyMGHCBGRkZODll19GdHS03rib7du3Y8qUKejZsyeUSiUGDx6M1atXG57iiIioxqmOtzpqu+q4JAqXSSAiomovLk4aFPw0sbHS49FUdSp7SRRDvr8ZaoiIqNrjmknVm0ZTeYvwGvL9zVW6iYio2quOtzqoWNFEgsZWpfPUEBERVZQc10yi54s9NUREVGMMGgQMGFB5tzqoZmOoISKiGqW63Oqg6oe3n4iIiEgWGGqIiIhIFhhqiIiISBYYaoiIiEgWGGqIiIhIFhhqiIiISBYYaoiIiEgWGGqIiIhIFhhqiIiISBZqzYzCRYuRZ2VlGbklREREVF5F39uitOXZH1NrQs39+/cBAO7u7kZuCRERERnq/v37sLGxeWIdhShP9JEBrVaLP/74A1ZWVlAUrVNPerKysuDu7o6bN2/C2tra2M2p9fj7qF74+6he+PuofirrdyKEwP379+Hm5gal8smjZmpNT41SqUTDhg2N3Ywawdramn9JVCP8fVQv/H1UL/x9VD+V8Tt5Wg9NEQ4UJiIiIllgqCEiIiJZYKghHZVKhfnz50OlUhm7KQT+Pqob/j6qF/4+qp/q8DupNQOFiYiISN7YU0NERESywFBDREREssBQQ0RERLLAUENERESywFBDREREssBQU8uFh4ejQ4cOsLKygpOTEwYOHIikpCRjN4v+JyIiAgqFAsHBwcZuSq2WkpKCkSNHwsHBARYWFmjdujV++uknYzerVtJoNJg3bx48PT1hYWEBLy8vLFq0qFyLHdKz+/7779GvXz+4ublBoVBg7969etuFEAgNDYWrqyssLCygVqtx+fLlKmsfQ00t991332Hy5Mn48ccfcfjwYRQUFKB3797IyckxdtNqvVOnTuHf//43vL29jd2UWu2vv/5Cly5dUKdOHRw8eBAXLlzA8uXLYWdnZ+ym1UqLFy/G+vXrsWbNGly8eBGLFy/GkiVL8PHHHxu7abVCTk4O2rRpg7Vr15a6fcmSJVi9ejU2bNiAhIQE1K1bF/7+/nj48GGVtI/z1JCeO3fuwMnJCd999x26detm7ObUWtnZ2WjXrh3WrVuH999/Hz4+Pli5cqWxm1UrzZ49G8ePH8exY8eM3RQC8Prrr8PZ2Rmffvqprmzw4MGwsLDAtm3bjNiy2kehUGDPnj0YOHAgAKmXxs3NDe+88w5mzpwJAMjMzISzszM2b96MoKCgSm8Te2pIT2ZmJgDA3t7eyC2p3SZPnoy+fftCrVYbuym13r59++Dr64uhQ4fCyckJbdu2xcaNG43drFqrc+fOiImJwW+//QYA+OWXX/DDDz/gtddeM3LLKDk5GWlpaXp/b9nY2MDPzw/x8fFV0oZas0o3PZ1Wq0VwcDC6dOmCl156ydjNqbUiIyNx5swZnDp1ythNIQBXr17F+vXrERISgn/96184deoUpk2bBjMzM4wePdrYzat1Zs+ejaysLLRo0QImJibQaDT44IMPMGLECGM3rdZLS0sDADg7O+uVOzs767ZVNoYa0pk8eTJ+/fVX/PDDD8ZuSq118+ZNTJ8+HYcPH4a5ubmxm0OQwr6vry8+/PBDAEDbtm3x66+/YsOGDQw1RrB7925s374dO3bswIsvvojExEQEBwfDzc2Nvw/i7SeSTJkyBfv370dsbCwaNmxo7ObUWqdPn8bt27fRrl07mJqawtTUFN999x1Wr14NU1NTaDQaYzex1nF1dUWrVq30ylq2bIkbN24YqUW127vvvovZs2cjKCgIrVu3xptvvokZM2YgPDzc2E2r9VxcXAAA6enpeuXp6em6bZWNoaaWE0JgypQp2LNnD44ePQpPT09jN6lW69mzJ86dO4fExETdy9fXFyNGjEBiYiJMTEyM3cRap0uXLiWmOfjtt9/QuHFjI7WodsvNzYVSqf/VZWJiAq1Wa6QWURFPT0+4uLggJiZGV5aVlYWEhAR06tSpStrA20+13OTJk7Fjxw589dVXsLKy0t33tLGxgYWFhZFbV/tYWVmVGM9Ut25dODg4cJyTkcyYMQOdO3fGhx9+iGHDhuHkyZP45JNP8Mknnxi7abVSv3798MEHH6BRo0Z48cUX8fPPP2PFihX4+9//buym1QrZ2dm4cuWK7n1ycjISExNhb2+PRo0aITg4GO+//z6aNWsGT09PzJs3D25ubronpCqdoFoNQKmvTZs2Gbtp9D/du3cX06dPN3YzarWvv/5avPTSS0KlUokWLVqITz75xNhNqrWysrLE9OnTRaNGjYS5ublo0qSJmDt3rsjLyzN202qF2NjYUr8zRo8eLYQQQqvVinnz5glnZ2ehUqlEz549RVJSUpW1j/PUEBERkSxwTA0RERHJAkMNERERyQJDDREREckCQw0RERHJAkMNERERyQJDDREREckCQw0RERHJAkMNERERyQJDDREREckCQw0RERHJAkMNERERycL/A2cDWPdCMLzoAAAAAElFTkSuQmCC\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model = keras.models.load_model(\"binary_1gram.keras\")\n",
        "print(f\"Test acc: {model.evaluate(binary_1gram_test_ds)[1]:.3f}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9_Z9mKa1AxD3",
        "outputId": "c27092b7-5d27-4024-ce50-0dee11940529"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "782/782 [==============================] - 4s 5ms/step - loss: 0.2995 - accuracy: 0.8872\n",
            "Test acc: 0.887\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "This gets us to a test accuracy of 88.7%: not bad! Note that in this case, since the dataset is a balanced two-class classification dataset (there are as many positive samples as\n",
        "negative samples), the “naive baseline” we could reach without training an actual model\n",
        "would only be 50%."
      ],
      "metadata": {
        "id": "rqGrRii-BIeI"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# BIGRAMS WITH BINARY ENCODING"
      ],
      "metadata": {
        "id": "xhMHzyZKCySN"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Of course, discarding word order is very reductive, because even atomic concepts can\n",
        "be expressed via multiple words: the term “United States” conveys a concept that is\n",
        "quite distinct from the meaning of the words “states” and “united” taken separately.\n",
        "For this reason, you will usually end up re-injecting local order information into your\n",
        "bag-of-words representation by looking at N-grams rather than single words (most\n",
        "commonly, bigrams).\n"
      ],
      "metadata": {
        "id": "MQST-pWUC047"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "With bigrams, our sentence becomes\n",
        "\n",
        "{\"the\", \"the cat\", \"cat\", \"cat sat\", \"sat\",\n",
        " \"sat on\", \"on\", \"on the\", \"the mat\", \"mat\"}"
      ],
      "metadata": {
        "id": "mkgXB3EzC4Lu"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "The TextVectorization layer can be configured to return arbitrary N-grams: bigrams,\n",
        "trigrams, etc. Just pass an **ngrams=N** argument as in the following listing."
      ],
      "metadata": {
        "id": "nJkPsPSLDA8Q"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "text_vectorization = TextVectorization(\n",
        " ngrams=2,\n",
        " max_tokens=20000,\n",
        " output_mode=\"multi_hot\",\n",
        ")"
      ],
      "metadata": {
        "id": "QQDBKA9eBJER"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "text_only_train_ds = train_ds.map(lambda x, y: x)\n",
        "text_vectorization.adapt(text_only_train_ds)\n",
        "binary_2gram_train_ds = train_ds.map(\n",
        " lambda x, y: (text_vectorization(x), y),\n",
        " num_parallel_calls=4)\n",
        "binary_2gram_val_ds = val_ds.map(\n",
        " lambda x, y: (text_vectorization(x), y),\n",
        " num_parallel_calls=4)\n",
        "binary_2gram_test_ds = test_ds.map(\n",
        " lambda x, y: (text_vectorization(x), y),\n",
        " num_parallel_calls=4)"
      ],
      "metadata": {
        "id": "8llau3nqDJOn"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model = get_model()\n",
        "model.summary()\n",
        "callbacks = [\n",
        " keras.callbacks.ModelCheckpoint(\"binary_2gram.keras\",\n",
        " save_best_only=True)\n",
        "]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Oxcw9b8pDRwC",
        "outputId": "25ab8876-5a2a-4d3e-c32a-df026657232f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"model_1\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " input_2 (InputLayer)        [(None, 20000)]           0         \n",
            "                                                                 \n",
            " dense_2 (Dense)             (None, 16)                320016    \n",
            "                                                                 \n",
            " dropout_1 (Dropout)         (None, 16)                0         \n",
            "                                                                 \n",
            " dense_3 (Dense)             (None, 1)                 17        \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 320033 (1.22 MB)\n",
            "Trainable params: 320033 (1.22 MB)\n",
            "Non-trainable params: 0 (0.00 Byte)\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model.fit(binary_2gram_train_ds.cache(),\n",
        " validation_data=binary_2gram_val_ds.cache(),\n",
        " epochs=10,\n",
        " callbacks=callbacks)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UESfs2VdDU3E",
        "outputId": "28a5523e-f6ce-4be8-8253-175d4d441d5b"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "625/625 [==============================] - 10s 15ms/step - loss: 0.3868 - accuracy: 0.8371 - val_loss: 0.2689 - val_accuracy: 0.8952\n",
            "Epoch 2/10\n",
            "625/625 [==============================] - 3s 5ms/step - loss: 0.2500 - accuracy: 0.9118 - val_loss: 0.2617 - val_accuracy: 0.9034\n",
            "Epoch 3/10\n",
            "625/625 [==============================] - 3s 5ms/step - loss: 0.2136 - accuracy: 0.9287 - val_loss: 0.2797 - val_accuracy: 0.9040\n",
            "Epoch 4/10\n",
            "625/625 [==============================] - 3s 5ms/step - loss: 0.2093 - accuracy: 0.9368 - val_loss: 0.2928 - val_accuracy: 0.9034\n",
            "Epoch 5/10\n",
            "625/625 [==============================] - 3s 5ms/step - loss: 0.1885 - accuracy: 0.9436 - val_loss: 0.3199 - val_accuracy: 0.9008\n",
            "Epoch 6/10\n",
            "625/625 [==============================] - 5s 8ms/step - loss: 0.1819 - accuracy: 0.9463 - val_loss: 0.3315 - val_accuracy: 0.8916\n",
            "Epoch 7/10\n",
            "625/625 [==============================] - 3s 5ms/step - loss: 0.1784 - accuracy: 0.9488 - val_loss: 0.3506 - val_accuracy: 0.8948\n",
            "Epoch 8/10\n",
            "625/625 [==============================] - 3s 5ms/step - loss: 0.1739 - accuracy: 0.9523 - val_loss: 0.3582 - val_accuracy: 0.8916\n",
            "Epoch 9/10\n",
            "625/625 [==============================] - 3s 5ms/step - loss: 0.1675 - accuracy: 0.9545 - val_loss: 0.3742 - val_accuracy: 0.8900\n",
            "Epoch 10/10\n",
            "625/625 [==============================] - 4s 6ms/step - loss: 0.1628 - accuracy: 0.9538 - val_loss: 0.3743 - val_accuracy: 0.8882\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x7e54deeca5f0>"
            ]
          },
          "metadata": {},
          "execution_count": 38
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model = keras.models.load_model(\"binary_2gram.keras\")\n",
        "print(f\"Test acc: {model.evaluate(binary_2gram_test_ds)[1]:.3f}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "U8WNBuKTDXto",
        "outputId": "592508bb-eb05-4a7d-a237-02245c11a209"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "782/782 [==============================] - 5s 6ms/step - loss: 0.2687 - accuracy: 0.8999\n",
            "Test acc: 0.900\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We’re now getting 90% test accuracy, a marked improvement! Turns out local order\n",
        "is pretty important."
      ],
      "metadata": {
        "id": "kNtae5NPDbxM"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# BIGRAMS WITH TF-IDF ENCODING"
      ],
      "metadata": {
        "id": "kwWGrdoDDm5u"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "You can also add a bit more information to this representation by counting how many\n",
        "times each word or N-gram occurs, that is to say, by taking the histogram of the words\n",
        "over the text:\n",
        "\n",
        "{\"the\": 2, \"the cat\": 1, \"cat\": 1, \"cat sat\": 1, \"sat\": 1,\n",
        " \"sat on\": 1, \"on\": 1, \"on the\": 1, \"the mat: 1\", \"mat\": 1}"
      ],
      "metadata": {
        "id": "hcKLTREuDn59"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "If you’re doing text classification, knowing how many times a word occurs in a sample\n",
        "is critical: any sufficiently long movie review may contain the word “terrible” regardless of sentiment, but a review that contains many instances of the word “terrible” is\n",
        "likely a negative one."
      ],
      "metadata": {
        "id": "cHOlelkQDwKy"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "text_vectorization = TextVectorization(\n",
        " ngrams=2,\n",
        " max_tokens=20000,\n",
        " output_mode=\"count\"\n",
        ")"
      ],
      "metadata": {
        "id": "95lGr1IFDcSA"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now, of course, some words are bound to occur more often than others no matter\n",
        "what the text is about. The words “the,” “a,” “is,” and “are” will always dominate your\n",
        "word count histograms, drowning out other words—despite being pretty much useless\n",
        "features in a classification context. How could we address this?"
      ],
      "metadata": {
        "id": "ysfPfu_GHuI4"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "You already guessed it: via normalization. We could just normalize word counts by\n",
        "subtracting the mean and dividing by the variance (computed across the entire training dataset). That would make sense. Except most vectorized sentences consist almost\n",
        "entirely of zeros (our previous example features 12 non-zero entries and 19,988 zero\n",
        "entries), a property called “sparsity.” That’s a great property to have, as it dramatically\n",
        "reduces compute load and reduces the risk of overfitting. If we subtracted the mean\n",
        "from each feature, we’d wreck sparsity. Thus, whatever normalization scheme we use\n",
        "should be divide-only. What, then, should we use as the denominator? The best practice is to go with something called TF-IDF normalization—TF-IDF stands for “term frequency, inverse document frequency.”"
      ],
      "metadata": {
        "id": "AdrTwpa-HuqY"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "TF-IDF is so common that it’s built into the TextVectorization layer. All you need\n",
        "to do to start using it is to switch the output_mode argument to \"tf_idf\"."
      ],
      "metadata": {
        "id": "9gDLCsUwHwap"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "text_vectorization = TextVectorization(\n",
        " ngrams=2,\n",
        " max_tokens=20000,\n",
        " output_mode=\"tf_idf\",\n",
        ")"
      ],
      "metadata": {
        "id": "4UaJEJHKH0ie"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "text_vectorization.adapt(text_only_train_ds) # The adapt() call will learn the TF-IDF weights in addition to the vocabulary.\n",
        "tfidf_2gram_train_ds = train_ds.map(\n",
        " lambda x, y: (text_vectorization(x), y),\n",
        " num_parallel_calls=4)\n",
        "tfidf_2gram_val_ds = val_ds.map(\n",
        " lambda x, y: (text_vectorization(x), y),\n",
        " num_parallel_calls=4)\n",
        "tfidf_2gram_test_ds = test_ds.map(\n",
        " lambda x, y: (text_vectorization(x), y),\n",
        " num_parallel_calls=4)"
      ],
      "metadata": {
        "id": "4C6GUTmCJuBo"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model = get_model()\n",
        "model.summary()\n",
        "callbacks = [\n",
        " keras.callbacks.ModelCheckpoint(\"tfidf_2gram.keras\",\n",
        " save_best_only=True)\n",
        "]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IRZ2qNA2J08w",
        "outputId": "4df4dd99-7b1e-419e-d2f2-07ee315b860d"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"model_2\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " input_3 (InputLayer)        [(None, 20000)]           0         \n",
            "                                                                 \n",
            " dense_4 (Dense)             (None, 16)                320016    \n",
            "                                                                 \n",
            " dropout_2 (Dropout)         (None, 16)                0         \n",
            "                                                                 \n",
            " dense_5 (Dense)             (None, 1)                 17        \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 320033 (1.22 MB)\n",
            "Trainable params: 320033 (1.22 MB)\n",
            "Non-trainable params: 0 (0.00 Byte)\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model.fit(tfidf_2gram_train_ds.cache(),\n",
        " validation_data=tfidf_2gram_val_ds.cache(),\n",
        " epochs=10,\n",
        " callbacks=callbacks)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GT3nE-xTJ36G",
        "outputId": "e084515c-6f6e-4a5b-9f0a-94175128bbe5"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "625/625 [==============================] - 10s 14ms/step - loss: 0.4763 - accuracy: 0.7796 - val_loss: 0.3043 - val_accuracy: 0.8774\n",
            "Epoch 2/10\n",
            "625/625 [==============================] - 4s 6ms/step - loss: 0.3338 - accuracy: 0.8615 - val_loss: 0.2818 - val_accuracy: 0.8902\n",
            "Epoch 3/10\n",
            "625/625 [==============================] - 3s 5ms/step - loss: 0.2975 - accuracy: 0.8776 - val_loss: 0.2878 - val_accuracy: 0.8982\n",
            "Epoch 4/10\n",
            "625/625 [==============================] - 3s 5ms/step - loss: 0.2698 - accuracy: 0.8906 - val_loss: 0.2983 - val_accuracy: 0.8838\n",
            "Epoch 5/10\n",
            "625/625 [==============================] - 3s 5ms/step - loss: 0.2494 - accuracy: 0.8962 - val_loss: 0.3247 - val_accuracy: 0.8716\n",
            "Epoch 6/10\n",
            "625/625 [==============================] - 3s 5ms/step - loss: 0.2404 - accuracy: 0.9011 - val_loss: 0.3231 - val_accuracy: 0.8758\n",
            "Epoch 7/10\n",
            "625/625 [==============================] - 3s 5ms/step - loss: 0.2282 - accuracy: 0.9035 - val_loss: 0.3343 - val_accuracy: 0.8772\n",
            "Epoch 8/10\n",
            "625/625 [==============================] - 3s 5ms/step - loss: 0.2256 - accuracy: 0.9093 - val_loss: 0.3524 - val_accuracy: 0.8650\n",
            "Epoch 9/10\n",
            "625/625 [==============================] - 4s 6ms/step - loss: 0.2224 - accuracy: 0.9125 - val_loss: 0.3407 - val_accuracy: 0.8710\n",
            "Epoch 10/10\n",
            "625/625 [==============================] - 3s 5ms/step - loss: 0.2124 - accuracy: 0.9154 - val_loss: 0.3507 - val_accuracy: 0.8772\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x7e54ded233d0>"
            ]
          },
          "metadata": {},
          "execution_count": 44
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model = keras.models.load_model(\"tfidf_2gram.keras\")\n",
        "print(f\"Test acc: {model.evaluate(tfidf_2gram_test_ds)[1]:.3f}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7lci8Ih8J56G",
        "outputId": "d130467b-f46c-4a4c-ddac-8c3578887e3c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "782/782 [==============================] - 6s 7ms/step - loss: 0.2849 - accuracy: 0.8866\n",
            "Test acc: 0.887\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "This gets us an 88.7% test accuracy on the IMDB classification task: it doesn’t seem to\n",
        "be particularly helpful in this case."
      ],
      "metadata": {
        "id": "o_OLMiU5KTfD"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "However, for many text-classification datasets, it\n",
        "would be typical to see a one-percentage-point increase when using TF-IDF compared\n",
        "to plain binary encoding."
      ],
      "metadata": {
        "id": "YsWYxtaDKXLH"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Processing words as a sequence: The sequence model approach"
      ],
      "metadata": {
        "id": "TOaDlC-kKsB7"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "To implement a sequence model, you’d start by representing your input samples as\n",
        "sequences of integer indices (one integer standing for one word). Then, you’d map\n",
        "each integer to a vector to obtain vector sequences. Finally, you’d feed these\n",
        "sequences of vectors into a stack of layers that could cross-correlate features from adjacent vectors, such as a 1D convnet, a RNN, or a Transformer"
      ],
      "metadata": {
        "id": "cKjYPwcdM02M"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "[How sequence models work with example and explanation of above thing](https://chat.openai.com/share/c011049a-80a3-491a-9f28-c01735eaad82)"
      ],
      "metadata": {
        "id": "uoFUebc6M1iT"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from tensorflow import keras\n",
        "from tensorflow.keras import layers\n",
        "import tensorflow as tf"
      ],
      "metadata": {
        "id": "rKM499ekNzp2"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "\"\"\"\n",
        "In order to keep a manageable\n",
        "input size, we’ll truncate the\n",
        "inputs after the first 600 words.\n",
        "This is a reasonable choice, since\n",
        "the average review length is 233\n",
        "words, and only 5% of reviews\n",
        "are longer than 600 words.\n",
        "\"\"\"\n",
        "\n",
        "\n",
        "max_length = 600\n",
        "max_tokens = 20000\n",
        "text_vectorization = layers.TextVectorization(\n",
        " max_tokens=max_tokens,\n",
        " output_mode=\"int\",\n",
        " output_sequence_length=max_length,\n",
        ")\n",
        "text_vectorization.adapt(text_only_train_ds)"
      ],
      "metadata": {
        "id": "KM_tey7dKZPt"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "[Max tokens and max length , meaning and usage](https://chat.openai.com/share/24714764-300b-48d2-a378-21ea65f1c522)"
      ],
      "metadata": {
        "id": "IrkzwcTnNp7z"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "int_train_ds = train_ds.map(\n",
        " lambda x, y: (text_vectorization(x), y),\n",
        " num_parallel_calls=4)\n",
        "int_val_ds = val_ds.map(\n",
        " lambda x, y: (text_vectorization(x), y),\n",
        " num_parallel_calls=4)\n",
        "int_test_ds = test_ds.map(\n",
        " lambda x, y: (text_vectorization(x), y),\n",
        " num_parallel_calls=4)"
      ],
      "metadata": {
        "id": "OKKMxjxcNvCX"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "The simplest way to convert our integer sequences to vector\n",
        "sequences is to one-hot encode the integers (each dimension would represent one\n",
        "possible term in the vocabulary). On top of these one-hot vectors, we’ll add a simple\n",
        "bidirectional LSTM."
      ],
      "metadata": {
        "id": "yjNzctT5Onbk"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import tensorflow as tf\n",
        "inputs = keras.Input(shape=(None,), dtype=\"int64\") # One input is a sequence of integers.\n",
        "embedded = tf.one_hot(inputs, depth=max_tokens) # Encode the integers into binary 20,000-dimensional vectors.\n",
        "x = layers.Bidirectional(layers.LSTM(32))(embedded)\n",
        "x = layers.Dropout(0.5)(x)\n",
        "outputs = layers.Dense(1, activation=\"sigmoid\")(x) # Finally, add a classification layer.\n",
        "model = keras.Model(inputs, outputs)"
      ],
      "metadata": {
        "id": "nkQkgon8On8y"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model.summary()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "d-ryiHRMPVM_",
        "outputId": "0b8dfa48-4c5d-4ee4-9a7e-64161513b07e"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"model\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " input_1 (InputLayer)        [(None, None)]            0         \n",
            "                                                                 \n",
            " tf.one_hot (TFOpLambda)     (None, None, 20000)       0         \n",
            "                                                                 \n",
            " bidirectional (Bidirection  (None, 64)                5128448   \n",
            " al)                                                             \n",
            "                                                                 \n",
            " dropout (Dropout)           (None, 64)                0         \n",
            "                                                                 \n",
            " dense (Dense)               (None, 1)                 65        \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 5128513 (19.56 MB)\n",
            "Trainable params: 5128513 (19.56 MB)\n",
            "Non-trainable params: 0 (0.00 Byte)\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model.compile(optimizer=\"rmsprop\",\n",
        " loss=\"binary_crossentropy\",\n",
        " metrics=[\"accuracy\"])\n",
        "\n",
        "callbacks = [\n",
        " keras.callbacks.ModelCheckpoint(\"one_hot_bidir_lstm.keras\",\n",
        " save_best_only=True)\n",
        "]\n",
        "model.fit(int_train_ds, validation_data=int_val_ds, epochs=10,\n",
        " callbacks=callbacks)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lw7InEhkPb31",
        "outputId": "86c9d734-951e-4e17-ed6d-d9c454500e95"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "625/625 [==============================] - 172s 263ms/step - loss: 0.5604 - accuracy: 0.7176 - val_loss: 0.3477 - val_accuracy: 0.8620\n",
            "Epoch 2/10\n",
            "625/625 [==============================] - 165s 265ms/step - loss: 0.3629 - accuracy: 0.8637 - val_loss: 0.3090 - val_accuracy: 0.8792\n",
            "Epoch 3/10\n",
            "625/625 [==============================] - 164s 263ms/step - loss: 0.2981 - accuracy: 0.8943 - val_loss: 0.2786 - val_accuracy: 0.8928\n",
            "Epoch 4/10\n",
            "625/625 [==============================] - 164s 262ms/step - loss: 0.2510 - accuracy: 0.9144 - val_loss: 0.3003 - val_accuracy: 0.8742\n",
            "Epoch 5/10\n",
            "625/625 [==============================] - 162s 259ms/step - loss: 0.2251 - accuracy: 0.9247 - val_loss: 0.3026 - val_accuracy: 0.8860\n",
            "Epoch 6/10\n",
            "625/625 [==============================] - 163s 261ms/step - loss: 0.2001 - accuracy: 0.9351 - val_loss: 0.3028 - val_accuracy: 0.8906\n",
            "Epoch 7/10\n",
            "625/625 [==============================] - 164s 262ms/step - loss: 0.1775 - accuracy: 0.9417 - val_loss: 0.3509 - val_accuracy: 0.8918\n",
            "Epoch 8/10\n",
            "625/625 [==============================] - 163s 261ms/step - loss: 0.1560 - accuracy: 0.9512 - val_loss: 0.3272 - val_accuracy: 0.8790\n",
            "Epoch 9/10\n",
            "625/625 [==============================] - 164s 262ms/step - loss: 0.1405 - accuracy: 0.9582 - val_loss: 0.3810 - val_accuracy: 0.8812\n",
            "Epoch 10/10\n",
            "625/625 [==============================] - 164s 262ms/step - loss: 0.1304 - accuracy: 0.9596 - val_loss: 0.3501 - val_accuracy: 0.8872\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x7b56b4345d20>"
            ]
          },
          "metadata": {},
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model = keras.models.load_model(\"one_hot_bidir_lstm.keras\")\n",
        "print(f\"Test acc: {model.evaluate(int_test_ds)[1]:.3f}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BSQo5QKPQt-7",
        "outputId": "b19b7e0e-0101-4500-b1ca-d17032e69028"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "782/782 [==============================] - 99s 124ms/step - loss: 0.2965 - accuracy: 0.8829\n",
            "Test acc: 0.883\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "A first observation: this model trains very slowly, especially compared to the lightweight model of the previous section.\n",
        "\n",
        " This is because our inputs are quite large: each\n",
        "input sample is encoded as a matrix of size (600, 20000) (600 words per sample,\n",
        "20,000 possible words). That’s 12,000,000 floats for a single movie review. Our bidirectional LSTM has a lot of work to do.\n",
        "\n",
        " Second, the model only gets to 88% test accuracy—it doesn’t perform nearly as well as our (very fast) binary unigram model."
      ],
      "metadata": {
        "id": "SfHIR3nSPxYk"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Clearly, using one-hot encoding to turn words into vectors, which was the simplest\n",
        "thing we could do, wasn’t a great idea. There’s a better way: word embeddings."
      ],
      "metadata": {
        "id": "hMhBVrWJP_3h"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let’s look at how to use such an embedding space in practice. There are two ways\n",
        "to obtain word embeddings:\n",
        "\n",
        "\n",
        " Learn word embeddings jointly with the main task you care about (such as document classification or sentiment prediction). In this setup, you start with random word vectors and then learn word vectors in the same way you learn the\n",
        "weights of a neural network.\n",
        "\n",
        "\n",
        " Load into your model word embeddings that were precomputed using a different machine learning task than the one you’re trying to solve. These are called\n",
        "pretrained word embeddings"
      ],
      "metadata": {
        "id": "hrQViXCdaGXF"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# LEARNING WORD EMBEDDINGS WITH THE EMBEDDING LAYER"
      ],
      "metadata": {
        "id": "5mRU0-i-aIqn"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "It’s thus reasonable to learn a new embedding space with every new task. Fortunately, backpropagation makes this easy, and Keras makes it even easier. It’s about\n",
        "learning the weights of a layer: the Embedding layer."
      ],
      "metadata": {
        "id": "ndUtfjeWSrV8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "embedding_layer = layers.Embedding(input_dim=max_tokens, output_dim=256)\n",
        "# The Embedding layer takes at least two arguments: the number of possible tokens and the dimensionality of the embeddings (here, 256)."
      ],
      "metadata": {
        "id": "3Mck74BYP4KO"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "The Embedding layer is best understood as a dictionary that maps integer indices\n",
        "(which stand for specific words) to dense vectors. It takes integers as input, looks up\n",
        "these integers in an internal dictionary, and returns the associated vectors. It’s effectively a dictionary lookup"
      ],
      "metadata": {
        "id": "r38che9ZVnuj"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "The Embedding layer takes as input a rank-2 tensor of integers, of shape (batch_size,\n",
        "sequence_length), where each entry is a sequence of integers. The layer then returns\n",
        "a 3D floating-point tensor of shape (batch_size, sequence_length, embedding_\n",
        "dimensionality)."
      ],
      "metadata": {
        "id": "3SR7Cl8iVpiy"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "When you instantiate an Embedding layer, its weights (its internal dictionary of\n",
        "token vectors) are initially random, just as with any other layer. During training, these\n",
        "word vectors are gradually adjusted via backpropagation, structuring the space into\n",
        "something the downstream model can exploit. Once fully trained, the embedding\n",
        "space will show a lot of structure—a kind of structure specialized for the specific problem for which you’re training your model"
      ],
      "metadata": {
        "id": "ALozUYWrVu3Y"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "inputs = keras.Input(shape=(None,), dtype=\"int64\")\n",
        "embedded = layers.Embedding(input_dim=max_tokens, output_dim=256)(inputs)\n",
        "x = layers.Bidirectional(layers.LSTM(32))(embedded)\n",
        "x = layers.Dropout(0.5)(x)\n",
        "outputs = layers.Dense(1, activation=\"sigmoid\")(x)\n",
        "model = keras.Model(inputs, outputs)\n",
        "model.compile(optimizer=\"rmsprop\",\n",
        " loss=\"binary_crossentropy\",\n",
        " metrics=[\"accuracy\"])"
      ],
      "metadata": {
        "id": "ujSfx7epVs5F"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model.summary()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7y8MTG72WHlq",
        "outputId": "abfeffcc-3539-4dd4-bee2-b5c085b54a8e"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"model\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " input_1 (InputLayer)        [(None, None)]            0         \n",
            "                                                                 \n",
            " embedding_1 (Embedding)     (None, None, 256)         5120000   \n",
            "                                                                 \n",
            " bidirectional (Bidirection  (None, 64)                73984     \n",
            " al)                                                             \n",
            "                                                                 \n",
            " dropout (Dropout)           (None, 64)                0         \n",
            "                                                                 \n",
            " dense (Dense)               (None, 1)                 65        \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 5194049 (19.81 MB)\n",
            "Trainable params: 5194049 (19.81 MB)\n",
            "Non-trainable params: 0 (0.00 Byte)\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "callbacks = [\n",
        " keras.callbacks.ModelCheckpoint(\"embeddings_bidir_gru.keras\",\n",
        " save_best_only=True)\n",
        "]\n",
        "model.fit(int_train_ds, validation_data=int_val_ds, epochs=10,\n",
        " callbacks=callbacks)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kwYt968eWU_m",
        "outputId": "837ad342-a798-42db-e00e-de8c78542d6b"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "625/625 [==============================] - 71s 99ms/step - loss: 0.5379 - accuracy: 0.7275 - val_loss: 0.3938 - val_accuracy: 0.8446\n",
            "Epoch 2/10\n",
            "625/625 [==============================] - 45s 73ms/step - loss: 0.3710 - accuracy: 0.8550 - val_loss: 0.3471 - val_accuracy: 0.8690\n",
            "Epoch 3/10\n",
            "625/625 [==============================] - 39s 63ms/step - loss: 0.2954 - accuracy: 0.8928 - val_loss: 0.3261 - val_accuracy: 0.8766\n",
            "Epoch 4/10\n",
            "625/625 [==============================] - 34s 55ms/step - loss: 0.2438 - accuracy: 0.9140 - val_loss: 0.3540 - val_accuracy: 0.8742\n",
            "Epoch 5/10\n",
            "625/625 [==============================] - 36s 57ms/step - loss: 0.2096 - accuracy: 0.9281 - val_loss: 0.5095 - val_accuracy: 0.8450\n",
            "Epoch 6/10\n",
            "625/625 [==============================] - 34s 54ms/step - loss: 0.1777 - accuracy: 0.9387 - val_loss: 0.3368 - val_accuracy: 0.8756\n",
            "Epoch 7/10\n",
            "625/625 [==============================] - 31s 50ms/step - loss: 0.1539 - accuracy: 0.9501 - val_loss: 0.3761 - val_accuracy: 0.8668\n",
            "Epoch 8/10\n",
            "625/625 [==============================] - 31s 49ms/step - loss: 0.1290 - accuracy: 0.9584 - val_loss: 0.4479 - val_accuracy: 0.8706\n",
            "Epoch 9/10\n",
            "625/625 [==============================] - 31s 50ms/step - loss: 0.1174 - accuracy: 0.9631 - val_loss: 0.4383 - val_accuracy: 0.8646\n",
            "Epoch 10/10\n",
            "625/625 [==============================] - 33s 52ms/step - loss: 0.1012 - accuracy: 0.9693 - val_loss: 0.4443 - val_accuracy: 0.8774\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x7b82f421bbe0>"
            ]
          },
          "metadata": {},
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model = keras.models.load_model(\"embeddings_bidir_gru.keras\")\n",
        "print(f\"Test acc: {model.evaluate(int_test_ds)[1]:.3f}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IxYIaLlOWYH7",
        "outputId": "e0f17745-6892-4419-f062-ed242851f201"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "782/782 [==============================] - 18s 21ms/step - loss: 0.3487 - accuracy: 0.8667\n",
            "Test acc: 0.867\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "It trains much faster than the one-hot model (since the LSTM only has to process\n",
        "256-dimensional vectors instead of 20,000-dimensional), and its test accuracy is comparable (87%). However, we’re still some way off from the results of our basic bigram\n",
        "model. Part of the reason why is simply that the model is looking at slightly less data:\n",
        "the bigram model processed full reviews, while our sequence model truncates sequences\n",
        "after 600 words."
      ],
      "metadata": {
        "id": "RYeeE_xHYUk1"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# UNDERSTANDING PADDING AND MASKING"
      ],
      "metadata": {
        "id": "zf__RIKZeSJ2"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "One thing that’s slightly hurting model performance here is that our input sequences\n",
        "are full of zeros. This comes from our use of the output_sequence_length=max_\n",
        "length option in TextVectorization (with max_length equal to 600): sentences longer than 600 tokens are truncated to a length of 600 tokens, and sentences shorter\n",
        "than 600 tokens are padded with zeros at the end so that they can be concatenated\n",
        "together with other sequences to form contiguous batches."
      ],
      "metadata": {
        "id": "GmHPbEw6eUOT"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        " We’re using a bidirectional RNN: two RNN layers running in parallel, with one\n",
        "processing the tokens in their natural order, and the other processing the same\n",
        "tokens in reverse. The RNN that looks at the tokens in their natural order will spend\n",
        "its last iterations seeing only vectors that encode padding—possibly for several hundreds of iterations if the original sentence was short. The information stored in the\n",
        "internal state of the RNN will gradually fade out as it gets exposed to these meaningless inputs."
      ],
      "metadata": {
        "id": "vYJuH3y_eWIV"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We need some way to tell the RNN that it should skip these iterations. There’s an\n",
        "API for that: masking."
      ],
      "metadata": {
        "id": "jv43vW5debBx"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        " The Embedding layer is capable of generating a “mask” that corresponds to its\n",
        "input data. This mask is a tensor of ones and zeros (or True/False booleans), of shape\n",
        "(batch_size, sequence_length), where the entry mask[i, t] indicates where timestep t of sample i should be skipped or not (the timestep will be skipped if mask[i, t]\n",
        "is 0 or False, and processed otherwise).\n",
        "\n",
        "\n",
        " By default, this option isn’t active—you can turn it on by passing mask_zero=True\n",
        "to your Embedding layer. You can retrieve the mask with the compute_mask() method:"
      ],
      "metadata": {
        "id": "1-8DFMC4eYSD"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "In practice, you will almost never have to manage masks by hand. Instead, Keras will\n",
        "automatically pass on the mask to every layer that is able to process it (as a piece of\n",
        "metadata attached to the sequence it represents). This mask will be used by RNN layers to skip masked steps. If your model returns an entire sequence, the mask will also\n",
        "be used by the loss function to skip masked steps in the output sequence."
      ],
      "metadata": {
        "id": "njV90pJiep4Z"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "inputs = keras.Input(shape=(None,), dtype=\"int64\")\n",
        "embedded = layers.Embedding(input_dim=max_tokens, output_dim=256, mask_zero=True)(inputs)\n",
        "x = layers.Bidirectional(layers.LSTM(32))(embedded)\n",
        "x = layers.Dropout(0.5)(x)\n",
        "outputs = layers.Dense(1, activation=\"sigmoid\")(x)\n",
        "model = keras.Model(inputs, outputs)\n",
        "model.compile(optimizer=\"rmsprop\",\n",
        " loss=\"binary_crossentropy\",\n",
        " metrics=[\"accuracy\"])\n",
        "model.summary()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NDfqD0StYVsx",
        "outputId": "d86e734c-e046-4526-bc49-3864de446102"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"model_1\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " input_2 (InputLayer)        [(None, None)]            0         \n",
            "                                                                 \n",
            " embedding_2 (Embedding)     (None, None, 256)         5120000   \n",
            "                                                                 \n",
            " bidirectional_1 (Bidirecti  (None, 64)                73984     \n",
            " onal)                                                           \n",
            "                                                                 \n",
            " dropout_1 (Dropout)         (None, 64)                0         \n",
            "                                                                 \n",
            " dense_1 (Dense)             (None, 1)                 65        \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 5194049 (19.81 MB)\n",
            "Trainable params: 5194049 (19.81 MB)\n",
            "Non-trainable params: 0 (0.00 Byte)\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "callbacks = [\n",
        " keras.callbacks.ModelCheckpoint(\"embeddings_bidir_gru_with_masking.keras\",\n",
        " save_best_only=True)\n",
        "]\n",
        "model.fit(int_train_ds, validation_data=int_val_ds, epochs=10,\n",
        " callbacks=callbacks)\n",
        "model = keras.models.load_model(\"embeddings_bidir_gru_with_masking.keras\")\n",
        "print(f\"Test acc: {model.evaluate(int_test_ds)[1]:.3f}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CbBcGxAae0Gz",
        "outputId": "8edadea5-a4b8-4766-d0ef-e6a0379ecd56"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "625/625 [==============================] - 76s 108ms/step - loss: 0.4647 - accuracy: 0.7724 - val_loss: 0.3108 - val_accuracy: 0.8778\n",
            "Epoch 2/10\n",
            "625/625 [==============================] - 48s 77ms/step - loss: 0.2853 - accuracy: 0.8853 - val_loss: 0.3897 - val_accuracy: 0.8476\n",
            "Epoch 3/10\n",
            "625/625 [==============================] - 42s 67ms/step - loss: 0.2185 - accuracy: 0.9172 - val_loss: 0.2834 - val_accuracy: 0.8866\n",
            "Epoch 4/10\n",
            "625/625 [==============================] - 39s 63ms/step - loss: 0.1663 - accuracy: 0.9374 - val_loss: 0.3080 - val_accuracy: 0.8814\n",
            "Epoch 5/10\n",
            "625/625 [==============================] - 38s 60ms/step - loss: 0.1279 - accuracy: 0.9564 - val_loss: 0.4021 - val_accuracy: 0.8544\n",
            "Epoch 6/10\n",
            "625/625 [==============================] - 36s 58ms/step - loss: 0.0993 - accuracy: 0.9663 - val_loss: 0.3949 - val_accuracy: 0.8776\n",
            "Epoch 7/10\n",
            "625/625 [==============================] - 38s 60ms/step - loss: 0.0795 - accuracy: 0.9724 - val_loss: 0.3836 - val_accuracy: 0.8722\n",
            "Epoch 8/10\n",
            "625/625 [==============================] - 38s 61ms/step - loss: 0.0615 - accuracy: 0.9798 - val_loss: 0.4093 - val_accuracy: 0.8730\n",
            "Epoch 9/10\n",
            "625/625 [==============================] - 36s 57ms/step - loss: 0.0432 - accuracy: 0.9859 - val_loss: 0.5195 - val_accuracy: 0.8806\n",
            "Epoch 10/10\n",
            "625/625 [==============================] - 36s 57ms/step - loss: 0.0402 - accuracy: 0.9866 - val_loss: 0.4993 - val_accuracy: 0.8752\n",
            "782/782 [==============================] - 23s 22ms/step - loss: 0.3043 - accuracy: 0.8770\n",
            "Test acc: 0.877\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# USING PRETRAINED WORD EMBEDDINGS"
      ],
      "metadata": {
        "id": "1Zd-RBtUjPjm"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Sometimes you have so little training data available that you can’t use your data alone\n",
        "to learn an appropriate task-specific embedding of your vocabulary. In such cases,\n",
        "instead of learning word embeddings jointly with the problem you want to solve, you\n",
        "can load embedding vectors from a precomputed embedding space that you know is\n",
        "highly structured and exhibits useful properties—one that captures generic aspects of\n",
        "language structure. The rationale behind using pretrained word embeddings in natural language processing is much the same as for using pretrained convnets in image\n",
        "classification: you don’t have enough data available to learn truly powerful features on\n",
        "your own, but you expect that the features you need are fairly generic—that is, common visual features or semantic features. In this case, it makes sense to reuse features\n",
        "learned on a different problem"
      ],
      "metadata": {
        "id": "M8Ov2fPWjR17"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        " Such word embeddings are generally computed using word-occurrence statistics\n",
        "(observations about what words co-occur in sentences or documents), using a variety\n",
        "of techniques, some involving neural networks, others not"
      ],
      "metadata": {
        "id": "QPq6rLg2jWrD"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "[Word2Vec and GIoVe](https://chat.openai.com/share/eb463cd6-a5f2-4efd-be3b-e8ad14ac2d49)"
      ],
      "metadata": {
        "id": "_kU_DZa4j5-3"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let’s look at how you can get started using GloVe embeddings in a Keras model.\n",
        "The same method is valid for Word2Vec embeddings or any other word-embedding\n",
        "database. We’ll start by downloading the GloVe files and parse them. We’ll then load\n",
        "the word vectors into a Keras Embedding layer, which we’ll use to build a new model."
      ],
      "metadata": {
        "id": "bXhRngSykMN4"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "irst, let’s download the GloVe word embeddings precomputed on the 2014\n",
        "English Wikipedia dataset. It’s an 822 MB zip file containing 100-dimensional embedding vectors for 400,000 words (or non-word tokens)"
      ],
      "metadata": {
        "id": "oAd7-ubvkR5k"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!wget http://nlp.stanford.edu/data/glove.6B.zip\n",
        "!unzip -q glove.6B.zip"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "s-m7BbWvjbOO",
        "outputId": "0da12242-9ab8-4431-fb01-3d9891caf5c2"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2023-12-29 17:13:48--  http://nlp.stanford.edu/data/glove.6B.zip\n",
            "Resolving nlp.stanford.edu (nlp.stanford.edu)... 171.64.67.140\n",
            "Connecting to nlp.stanford.edu (nlp.stanford.edu)|171.64.67.140|:80... connected.\n",
            "HTTP request sent, awaiting response... 302 Found\n",
            "Location: https://nlp.stanford.edu/data/glove.6B.zip [following]\n",
            "--2023-12-29 17:13:48--  https://nlp.stanford.edu/data/glove.6B.zip\n",
            "Connecting to nlp.stanford.edu (nlp.stanford.edu)|171.64.67.140|:443... connected.\n",
            "HTTP request sent, awaiting response... 301 Moved Permanently\n",
            "Location: https://downloads.cs.stanford.edu/nlp/data/glove.6B.zip [following]\n",
            "--2023-12-29 17:13:48--  https://downloads.cs.stanford.edu/nlp/data/glove.6B.zip\n",
            "Resolving downloads.cs.stanford.edu (downloads.cs.stanford.edu)... 171.64.64.22\n",
            "Connecting to downloads.cs.stanford.edu (downloads.cs.stanford.edu)|171.64.64.22|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 862182613 (822M) [application/zip]\n",
            "Saving to: ‘glove.6B.zip’\n",
            "\n",
            "glove.6B.zip        100%[===================>] 822.24M  5.02MB/s    in 2m 39s  \n",
            "\n",
            "2023-12-29 17:16:27 (5.17 MB/s) - ‘glove.6B.zip’ saved [862182613/862182613]\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let’s parse the unzipped file (a .txt file) to build an index that maps words (as strings)\n",
        "to their vector representation."
      ],
      "metadata": {
        "id": "PCfbeQhWkchD"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import numpy as np\n",
        "path_to_glove_file = \"glove.6B.100d.txt\"\n",
        "embeddings_index = {}\n",
        "with open(path_to_glove_file) as f:\n",
        " for line in f:\n",
        "    word, coefs = line.split(maxsplit=1)\n",
        "    coefs = np.fromstring(coefs, \"f\", sep=\" \")\n",
        "    embeddings_index[word] = coefs\n",
        "print(f\"Found {len(embeddings_index)} word vectors.\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-gwKnxOFkc_U",
        "outputId": "244d4b26-4697-42f3-e2ad-6897a6b0b4a5"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Found 400000 word vectors.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "embedding_dim = 100\n",
        "vocabulary = text_vectorization.get_vocabulary() # Retrieve the vocabulary indexed by our previous TextVectorization layer.\n",
        "word_index = dict(zip(vocabulary, range(len(vocabulary)))) # Use it to create a mapping from words to their index in the vocabulary.\n",
        "embedding_matrix = np.zeros((max_tokens, embedding_dim))\n",
        "for word, i in word_index.items():\n",
        "  # Fill entry i in the matrix with the word vector for index i. Words not found in the embedding index will be all zeros.\n",
        " if i < max_tokens:\n",
        "    embedding_vector = embeddings_index.get(word)\n",
        "    if embedding_vector is not None:\n",
        "        embedding_matrix[i] = embedding_vector"
      ],
      "metadata": {
        "id": "Y0UqlC91nuh0"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Finally, we use a Constant initializer to load the pretrained embeddings in an Embedding\n",
        "layer. So as not to disrupt the pretrained representations during training, we freeze\n",
        "the layer via trainable=False"
      ],
      "metadata": {
        "id": "WCBmDRd-pNxt"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "embedding_layer = layers.Embedding(\n",
        "max_tokens,\n",
        "embedding_dim,\n",
        "embeddings_initializer=keras.initializers.Constant(embedding_matrix),trainable=False,\n",
        "mask_zero=True,\n",
        ")"
      ],
      "metadata": {
        "id": "rHwwdlB0pOer"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "inputs = keras.Input(shape=(None,), dtype=\"int64\")\n",
        "embedded = embedding_layer(inputs)\n",
        "x = layers.Bidirectional(layers.LSTM(32))(embedded)\n",
        "x = layers.Dropout(0.5)(x)\n",
        "outputs = layers.Dense(1, activation=\"sigmoid\")(x)\n",
        "model = keras.Model(inputs, outputs)\n",
        "model.compile(optimizer=\"rmsprop\",\n",
        " loss=\"binary_crossentropy\",\n",
        " metrics=[\"accuracy\"])"
      ],
      "metadata": {
        "id": "2uAzVQHPpa1v"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model.summary()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fC9m9O0ipdjx",
        "outputId": "4516ae44-ae4a-44e3-b343-7ab5dbaa37cc"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"model_2\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " input_3 (InputLayer)        [(None, None)]            0         \n",
            "                                                                 \n",
            " embedding_3 (Embedding)     (None, None, 100)         2000000   \n",
            "                                                                 \n",
            " bidirectional_2 (Bidirecti  (None, 64)                34048     \n",
            " onal)                                                           \n",
            "                                                                 \n",
            " dropout_2 (Dropout)         (None, 64)                0         \n",
            "                                                                 \n",
            " dense_2 (Dense)             (None, 1)                 65        \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 2034113 (7.76 MB)\n",
            "Trainable params: 34113 (133.25 KB)\n",
            "Non-trainable params: 2000000 (7.63 MB)\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "callbacks = [\n",
        " keras.callbacks.ModelCheckpoint(\"glove_embeddings_sequence_model.keras\",\n",
        " save_best_only=True)\n",
        "]\n",
        "model.fit(int_train_ds, validation_data=int_val_ds, epochs=10,\n",
        " callbacks=callbacks)\n",
        "model = keras.models.load_model(\"glove_embeddings_sequence_model.keras\")\n",
        "print(f\"Test acc: {model.evaluate(int_test_ds)[1]:.3f}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "50Ct3h_cpfwC",
        "outputId": "a085a6d8-1cac-40a3-8627-67092f177071"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "625/625 [==============================] - 84s 114ms/step - loss: 0.5744 - accuracy: 0.6976 - val_loss: 0.5026 - val_accuracy: 0.7584\n",
            "Epoch 2/10\n",
            "625/625 [==============================] - 59s 95ms/step - loss: 0.4511 - accuracy: 0.7947 - val_loss: 0.4081 - val_accuracy: 0.8164\n",
            "Epoch 3/10\n",
            "625/625 [==============================] - 62s 100ms/step - loss: 0.4035 - accuracy: 0.8195 - val_loss: 0.4009 - val_accuracy: 0.8208\n",
            "Epoch 4/10\n",
            "625/625 [==============================] - 61s 97ms/step - loss: 0.3669 - accuracy: 0.8400 - val_loss: 0.3592 - val_accuracy: 0.8420\n",
            "Epoch 5/10\n",
            "625/625 [==============================] - 35s 55ms/step - loss: 0.3424 - accuracy: 0.8579 - val_loss: 0.4357 - val_accuracy: 0.8210\n",
            "Epoch 6/10\n",
            "625/625 [==============================] - 60s 95ms/step - loss: 0.3236 - accuracy: 0.8673 - val_loss: 0.3253 - val_accuracy: 0.8604\n",
            "Epoch 7/10\n",
            "625/625 [==============================] - 36s 57ms/step - loss: 0.3021 - accuracy: 0.8745 - val_loss: 0.3377 - val_accuracy: 0.8516\n",
            "Epoch 8/10\n",
            "625/625 [==============================] - 62s 100ms/step - loss: 0.2883 - accuracy: 0.8809 - val_loss: 0.3148 - val_accuracy: 0.8618\n",
            "Epoch 9/10\n",
            "625/625 [==============================] - 59s 95ms/step - loss: 0.2734 - accuracy: 0.8910 - val_loss: 0.3077 - val_accuracy: 0.8720\n",
            "Epoch 10/10\n",
            "625/625 [==============================] - 36s 57ms/step - loss: 0.2605 - accuracy: 0.8968 - val_loss: 0.3417 - val_accuracy: 0.8584\n",
            "782/782 [==============================] - 23s 22ms/step - loss: 0.3028 - accuracy: 0.8718\n",
            "Test acc: 0.872\n"
          ]
        }
      ]
    }
  ]
}