sentiment-analysis-llm-bert.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8f3d75e9",
   "metadata": {
    "papermill": {
     "duration": 0.014643,
     "end_time": "2024-03-23T04:44:22.699201",
     "exception": false,
     "start_time": "2024-03-23T04:44:22.684558",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "\n",
    "\n",
    "<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n",
    "    Sentiment Analysis || BERT || Transformers\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3908954f",
   "metadata": {
    "papermill": {
     "duration": 0.012706,
     "end_time": "2024-03-23T04:44:22.725727",
     "exception": false,
     "start_time": "2024-03-23T04:44:22.713021",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "![](https://calln.com/wp-content/uploads/2019/10/Sentiment-Analysis-Infographic-860x600-860x600.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "169fb521",
   "metadata": {
    "papermill": {
     "duration": 0.013367,
     "end_time": "2024-03-23T04:44:22.752564",
     "exception": false,
     "start_time": "2024-03-23T04:44:22.739197",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "\n",
    "- Sentiment analysis, also known as opinion mining, is a process that involves analyzing text to determine the sentiment expressed, such as positive, negative, or neutral.\n",
    "  \n",
    "- It uses natural language processing (NLP) and machine learning techniques to understand the emotions, opinions, and attitudes conveyed in textual data.\n",
    "\n",
    "- The goal of sentiment analysis is to extract subjective information from text, allowing businesses and organizations to gain insights into public opinion, customer feedback, social media trends, and more.\n",
    "\n",
    "- Sentiment analysis has widespread applications in various fields, including marketing, customer service, social media monitoring, market research, and product development.\n",
    "\n",
    "- Businesses use sentiment analysis to analyze customer reviews, social media posts, surveys, and other forms of textual data to understand customer sentiment, identify trends, and make data-driven decisions.\n",
    "\n",
    "- Sentiment analysis algorithms classify text into categories such as positive, negative, or neutral based on the words, phrases, and context used in the text.\n",
    "\n",
    "- Sentiment analysis can be performed at different levels, including document-level (analyzing entire documents), sentence-level (analyzing individual sentences), and aspect-level (analyzing specific aspects or features mentioned in the text).\n",
    "\n",
    "Overall, sentiment analysis helps businesses and organizations better understand and respond to customer sentiment, improve products and services, and enhance overall customer satisfaction."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a88f3c55",
   "metadata": {
    "papermill": {
     "duration": 0.013295,
     "end_time": "2024-03-23T04:44:22.779396",
     "exception": false,
     "start_time": "2024-03-23T04:44:22.766101",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n",
    "   RNN || TRANSFORMERS || BERT\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e73c822",
   "metadata": {
    "papermill": {
     "duration": 0.013605,
     "end_time": "2024-03-23T04:44:22.807029",
     "exception": false,
     "start_time": "2024-03-23T04:44:22.793424",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "![](https://publish-01.obsidian.md/access/fc5a2dd092cb06eefdd04ff307334d7a/_attachments/BERT%20Architecture.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adc66b22",
   "metadata": {
    "papermill": {
     "duration": 0.013443,
     "end_time": "2024-03-23T04:44:22.834586",
     "exception": false,
     "start_time": "2024-03-23T04:44:22.821143",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "BERT (Bidirectional Encoder Representations from Transformers) is a powerful pre-trained language model developed by Google AI, specifically designed for natural language processing (NLP) tasks. Here are some key points about BERT:\n",
    "\n",
    "- **Bidirectional Encoding:** BERT is bidirectional, meaning it can understand the context of a word based on both the words that come before and after it in a sentence. This bidirectional approach helps capture deeper contextual meanings and nuances in text.\n",
    "\n",
    "- **Transformer Architecture:** BERT is based on the Transformer architecture, which uses self-attention mechanisms to weigh the importance of different words in a sentence. This architecture allows BERT to efficiently process and understand long-range dependencies in text.\n",
    "\n",
    "- **Pre-trained Model:** BERT is pre-trained on a large corpus of text data, such as Wikipedia articles and books. During pre-training, BERT learns rich word embeddings and contextual representations, which are then fine-tuned for specific downstream NLP tasks.\n",
    "\n",
    "- **Fine-tuning:** After pre-training, BERT can be fine-tuned on smaller datasets for various NLP tasks such as text classification, named entity recognition, question answering, sentiment analysis, and more. Fine-tuning involves adapting BERT's pre-trained parameters to the specific task at hand, improving its performance and accuracy.\n",
    "\n",
    "- **Contextual Word Embeddings:** BERT generates contextual word embeddings, which means that the meaning of a word can vary based on its context within a sentence. This allows BERT to capture polysemy (multiple meanings of words) and disambiguate words based on their surrounding context.\n",
    "\n",
    "- **State-of-the-Art Performance:** BERT has achieved state-of-the-art performance on a wide range of NLP benchmarks and tasks, surpassing previous models in accuracy and efficiency. Its ability to capture complex linguistic patterns and semantic relationships makes it a popular choice for NLP researchers and practitioners.\n",
    "\n",
    "- **Open-Source Implementation:** BERT is available as an open-source implementation, allowing developers and researchers to use, modify, and fine-tune the model for their specific applications. This accessibility has contributed to its widespread adoption and impact in the NLP community.\n",
    "\n",
    "Overall, BERT has revolutionized the field of NLP by introducing advanced techniques for contextual understanding of text, leading to significant advancements in various NLP applications and tasks."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "521daf40",
   "metadata": {
    "papermill": {
     "duration": 0.014975,
     "end_time": "2024-03-23T04:44:22.863349",
     "exception": false,
     "start_time": "2024-03-23T04:44:22.848374",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n",
    "   IMPORTING LIBRARY\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "0776b32f",
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5",
    "execution": {
     "iopub.execute_input": "2024-03-23T04:44:22.891507Z",
     "iopub.status.busy": "2024-03-23T04:44:22.890793Z",
     "iopub.status.idle": "2024-03-23T04:44:32.477537Z",
     "shell.execute_reply": "2024-03-23T04:44:32.476196Z"
    },
    "papermill": {
     "duration": 9.603225,
     "end_time": "2024-03-23T04:44:32.479895",
     "exception": false,
     "start_time": "2024-03-23T04:44:22.876670",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/kaggle/input/bert-sentiment-analysis/Sentiment Analysis.csv\n"
     ]
    }
   ],
   "source": [
    "import numpy as np \n",
    "import pandas as pd \n",
    "#for pytorch\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "device = torch.device(\"cuda\")\n",
    "\n",
    "#for BERT\n",
    "import transformers\n",
    "from transformers import AutoModel, BertTokenizerFast\n",
    "from sklearn.model_selection import train_test_split\n",
    "from torch.utils.data import TensorDataset, DataLoader, RandomSampler, SequentialSampler\n",
    "# optimizer from hugging face transformers\n",
    "from transformers import AdamW\n",
    "import os\n",
    "for dirname, _, filenames in os.walk('/kaggle/input'):\n",
    "    for filename in filenames:\n",
    "        print(os.path.join(dirname, filename))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35b8b81a",
   "metadata": {
    "papermill": {
     "duration": 0.01444,
     "end_time": "2024-03-23T04:44:32.509225",
     "exception": false,
     "start_time": "2024-03-23T04:44:32.494785",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n",
    "   Reading The Data\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "567c2ef9",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:44:32.540179Z",
     "iopub.status.busy": "2024-03-23T04:44:32.539673Z",
     "iopub.status.idle": "2024-03-23T04:44:32.568728Z",
     "shell.execute_reply": "2024-03-23T04:44:32.567871Z"
    },
    "papermill": {
     "duration": 0.0472,
     "end_time": "2024-03-23T04:44:32.570886",
     "exception": false,
     "start_time": "2024-03-23T04:44:32.523686",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df=pd.read_csv('/kaggle/input/bert-sentiment-analysis/Sentiment Analysis.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "85f13bfe",
   "metadata": {
    "papermill": {
     "duration": 0.014286,
     "end_time": "2024-03-23T04:44:32.599894",
     "exception": false,
     "start_time": "2024-03-23T04:44:32.585608",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n",
    "   Sample Of Data\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "2190a8a3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:44:32.628949Z",
     "iopub.status.busy": "2024-03-23T04:44:32.628637Z",
     "iopub.status.idle": "2024-03-23T04:44:32.648887Z",
     "shell.execute_reply": "2024-03-23T04:44:32.648017Z"
    },
    "papermill": {
     "duration": 0.037332,
     "end_time": "2024-03-23T04:44:32.651173",
     "exception": false,
     "start_time": "2024-03-23T04:44:32.613841",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sentence</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Ok brokeback mountain is such a horrible movie.</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Brokeback Mountain was so awesome.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>friday hung out with kelsie and we went and sa...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>I am going to start reading the Harry Potter s...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Is it just me, or does Harry Potter suck?...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            sentence  label\n",
       "0    Ok brokeback mountain is such a horrible movie.      0\n",
       "1                 Brokeback Mountain was so awesome.      1\n",
       "2  friday hung out with kelsie and we went and sa...      0\n",
       "3  I am going to start reading the Harry Potter s...      1\n",
       "4       Is it just me, or does Harry Potter suck?...      0"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9969683",
   "metadata": {
    "papermill": {
     "duration": 0.014318,
     "end_time": "2024-03-23T04:44:32.680704",
     "exception": false,
     "start_time": "2024-03-23T04:44:32.666386",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n",
    "   Data describe\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "5a8b25f0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:44:32.712727Z",
     "iopub.status.busy": "2024-03-23T04:44:32.712355Z",
     "iopub.status.idle": "2024-03-23T04:44:32.733902Z",
     "shell.execute_reply": "2024-03-23T04:44:32.732919Z"
    },
    "papermill": {
     "duration": 0.041018,
     "end_time": "2024-03-23T04:44:32.736092",
     "exception": false,
     "start_time": "2024-03-23T04:44:32.695074",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>5668.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.565279</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.495764</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             label\n",
       "count  5668.000000\n",
       "mean      0.565279\n",
       "std       0.495764\n",
       "min       0.000000\n",
       "25%       0.000000\n",
       "50%       1.000000\n",
       "75%       1.000000\n",
       "max       1.000000"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "677c66e2",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:44:32.766785Z",
     "iopub.status.busy": "2024-03-23T04:44:32.766481Z",
     "iopub.status.idle": "2024-03-23T04:44:32.776409Z",
     "shell.execute_reply": "2024-03-23T04:44:32.775545Z"
    },
    "papermill": {
     "duration": 0.02755,
     "end_time": "2024-03-23T04:44:32.778476",
     "exception": false,
     "start_time": "2024-03-23T04:44:32.750926",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "lens=[len(i.split()) for i in df.sentence]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "5c06cd0a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:44:32.809386Z",
     "iopub.status.busy": "2024-03-23T04:44:32.809090Z",
     "iopub.status.idle": "2024-03-23T04:44:33.157159Z",
     "shell.execute_reply": "2024-03-23T04:44:33.156224Z"
    },
    "papermill": {
     "duration": 0.365837,
     "end_time": "2024-03-23T04:44:33.159341",
     "exception": false,
     "start_time": "2024-03-23T04:44:32.793504",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([2113., 1092.,  912.,  692.,  276.,  253.,  155.,   29.,  142.,\n",
       "           4.]),\n",
       " array([ 3. ,  6.7, 10.4, 14.1, 17.8, 21.5, 25.2, 28.9, 32.6, 36.3, 40. ]),\n",
       " <BarContainer object of 10 artists>)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjAAAAGdCAYAAAAMm0nCAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/xnp5ZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAArV0lEQVR4nO3df3RU9Z3/8ddAmOGHyYQAySRrDAFrMPJDRRtnlRQKmxBTqivdFUGJNQWlwRaiNqRrMeAek4Zd/LFlcT0V6Z4FRfcArtCyBJCklYgQnA2g5ggbDB4ywVWZgSAhkPv9oyf325EgBiZMPuH5OOeek/v5fO697w+fw8nr3Llz47AsyxIAAIBBekW6AAAAgM4iwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjBMV6QK6Sltbm44cOaLo6Gg5HI5IlwMAAL4Fy7J0/PhxJSUlqVev899n6bEB5siRI0pOTo50GQAA4CIcPnxYV1999Xn7e2yAiY6OlvTnf4CYmJgIVwMAAL6NYDCo5ORk+/f4+fTYANP+sVFMTAwBBgAAw1zo8Q8e4gUAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwTlSkCzDR0AUbI11Cpx0qy410CQAAhA13YAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACM06kAU1paqltvvVXR0dGKj4/X3Xffrbq6upAxp06dUkFBgQYNGqSrrrpKU6dOVVNTU8iYhoYG5ebmqn///oqPj9cTTzyhM2fOhIzZvn27br75ZrlcLl177bVauXLlxc0QAAD0OJ0KMJWVlSooKNC7776riooKtba2KisrS83NzfaY+fPn66233tIbb7yhyspKHTlyRPfcc4/df/bsWeXm5ur06dPasWOHfve732nlypVauHChPaa+vl65ubmaMGGCfD6f5s2bp5/85Cf67//+7zBMGQAAmM5hWZZ1sQd/9tlnio+PV2VlpTIzMxUIBDRkyBCtXr1aP/rRjyRJH330ka6//npVV1frtttu0x/+8Af94Ac/0JEjR5SQkCBJevHFF1VUVKTPPvtMTqdTRUVF2rhxo/bt22dfa9q0aTp27Jg2bdr0rWoLBoNyu90KBAKKiYm52Cl2aOiCjWE93+VwqCw30iUAAHBB3/b39yU9AxMIBCRJcXFxkqSamhq1trZq0qRJ9pgRI0bommuuUXV1tSSpurpao0aNssOLJGVnZysYDGr//v32mL88R/uY9nN0pKWlRcFgMGQDAAA900UHmLa2Ns2bN0+33367Ro4cKUny+/1yOp2KjY0NGZuQkCC/32+P+cvw0t7f3vdNY4LBoL766qsO6yktLZXb7ba35OTki50aAADo5i46wBQUFGjfvn167bXXwlnPRSsuLlYgELC3w4cPR7okAADQRaIu5qC5c+dqw4YNqqqq0tVXX223ezwenT59WseOHQu5C9PU1CSPx2OPee+990LO1/4tpb8c8/VvLjU1NSkmJkb9+vXrsCaXyyWXy3Ux0wEAAIbp1B0Yy7I0d+5crVu3Ttu2bVNqampI/9ixY9WnTx9t3brVbqurq1NDQ4O8Xq8kyev1au/evTp69Kg9pqKiQjExMUpPT7fH/OU52se0nwMAAFzZOnUHpqCgQKtXr9abb76p6Oho+5kVt9utfv36ye12Kz8/X4WFhYqLi1NMTIweffRReb1e3XbbbZKkrKwspaen64EHHlB5ebn8fr+efPJJFRQU2HdQHnnkEf3mN7/RL37xCz300EPatm2bXn/9dW3caN63fwAAQPh16g7M8uXLFQgENH78eCUmJtrbmjVr7DHPPvusfvCDH2jq1KnKzMyUx+PR2rVr7f7evXtrw4YN6t27t7xer+6//37NnDlTixcvtsekpqZq48aNqqio0JgxY/TP//zP+u1vf6vs7OwwTBkAAJjukt4D053xHphQvAcGAGCCy/IeGAAAgEggwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGKfTAaaqqkpTpkxRUlKSHA6H1q9fH9LvcDg63JYsWWKPGTp06Dn9ZWVlIeepra3VuHHj1LdvXyUnJ6u8vPziZggAAHqcTgeY5uZmjRkzRsuWLeuwv7GxMWRbsWKFHA6Hpk6dGjJu8eLFIeMeffRRuy8YDCorK0spKSmqqanRkiVLVFJSopdeeqmz5QIAgB4oqrMH5OTkKCcn57z9Ho8nZP/NN9/UhAkTNGzYsJD26Ojoc8a2W7VqlU6fPq0VK1bI6XTqhhtukM/n09KlSzV79uzOlgwAAHqYLn0GpqmpSRs3blR+fv45fWVlZRo0aJBuuukmLVmyRGfOnLH7qqurlZmZKafTabdlZ2errq5OX375ZYfXamlpUTAYDNkAAEDP1Ok7MJ3xu9/9TtHR0brnnntC2n/2s5/p5ptvVlxcnHbs2KHi4mI1NjZq6dKlkiS/36/U1NSQYxISEuy+gQMHnnOt0tJSLVq0qItmAgAAupMuDTArVqzQjBkz1Ldv35D2wsJC++fRo0fL6XTq4YcfVmlpqVwu10Vdq7i4OOS8wWBQycnJF1c4AADo1roswPzxj39UXV2d1qxZc8GxGRkZOnPmjA4dOqS0tDR5PB41NTWFjGnfP99zMy6X66LDDwAAMEuXPQPz8ssva+zYsRozZswFx/p8PvXq1Uvx8fGSJK/Xq6qqKrW2ttpjKioqlJaW1uHHRwAA4MrS6QBz4sQJ+Xw++Xw+SVJ9fb18Pp8aGhrsMcFgUG+88YZ+8pOfnHN8dXW1nnvuOf3P//yP/vd//1erVq3S/Pnzdf/999vhZPr06XI6ncrPz9f+/fu1Zs0aPf/88yEfEQEAgCtXpz9C2r17tyZMmGDvt4eKvLw8rVy5UpL02muvybIs3Xfffecc73K59Nprr6mkpEQtLS1KTU3V/PnzQ8KJ2+3W5s2bVVBQoLFjx2rw4MFauHAhX6EGAACSJIdlWVaki+gKwWBQbrdbgUBAMTExYT330AUbw3q+y+FQWW6kSwAA4IK+7e9v/hYSAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxOh1gqqqqNGXKFCUlJcnhcGj9+vUh/Q8++KAcDkfINnny5JAxX3zxhWbMmKGYmBjFxsYqPz9fJ06cCBlTW1urcePGqW/fvkpOTlZ5eXnnZwcAAHqkTgeY5uZmjRkzRsuWLTvvmMmTJ6uxsdHeXn311ZD+GTNmaP/+/aqoqNCGDRtUVVWl2bNn2/3BYFBZWVlKSUlRTU2NlixZopKSEr300kudLRcAAPRAUZ09ICcnRzk5Od84xuVyyePxdNj34YcfatOmTdq1a5duueUWSdK//Mu/6M4779Q//dM/KSkpSatWrdLp06e1YsUKOZ1O3XDDDfL5fFq6dGlI0AEAAFemLnkGZvv27YqPj1daWprmzJmjzz//3O6rrq5WbGysHV4kadKkSerVq5d27txpj8nMzJTT6bTHZGdnq66uTl9++WWH12xpaVEwGAzZAABAzxT2ADN58mT9+7//u7Zu3apf//rXqqysVE5Ojs6ePStJ8vv9io+PDzkmKipKcXFx8vv99piEhISQMe377WO+rrS0VG63296Sk5PDPTUAANBNdPojpAuZNm2a/fOoUaM0evRoDR8+XNu3b9fEiRPDfTlbcXGxCgsL7f1gMEiIAQCgh+ryr1EPGzZMgwcP1oEDByRJHo9HR48eDRlz5swZffHFF/ZzMx6PR01NTSFj2vfP92yNy+VSTExMyAYAAHqmLg8wn376qT7//HMlJiZKkrxer44dO6aamhp7zLZt29TW1qaMjAx7TFVVlVpbW+0xFRUVSktL08CBA7u6ZAAA0M11OsCcOHFCPp9PPp9PklRfXy+fz6eGhgadOHFCTzzxhN59910dOnRIW7du1V133aVrr71W2dnZkqTrr79ekydP1qxZs/Tee+/pnXfe0dy5czVt2jQlJSVJkqZPny6n06n8/Hzt379fa9as0fPPPx/yEREAALhydTrA7N69WzfddJNuuukmSVJhYaFuuukmLVy4UL1791Ztba1++MMf6rrrrlN+fr7Gjh2rP/7xj3K5XPY5Vq1apREjRmjixIm68847dccdd4S848Xtdmvz5s2qr6/X2LFj9dhjj2nhwoV8hRoAAEiSHJZlWZEuoisEg0G53W4FAoGwPw8zdMHGsJ7vcjhUlhvpEgAAuKBv+/ubv4UEAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADBOpwNMVVWVpkyZoqSkJDkcDq1fv97ua21tVVFRkUaNGqUBAwYoKSlJM2fO1JEjR0LOMXToUDkcjpCtrKwsZExtba3GjRunvn37Kjk5WeXl5Rc3QwAA0ON0OsA0NzdrzJgxWrZs2Tl9J0+e1J49e/SrX/1Ke/bs0dq1a1VXV6cf/vCH54xdvHixGhsb7e3RRx+1+4LBoLKyspSSkqKamhotWbJEJSUleumllzpbLgAA6IGiOntATk6OcnJyOuxzu92qqKgIafvNb36j7373u2poaNA111xjt0dHR8vj8XR4nlWrVun06dNasWKFnE6nbrjhBvl8Pi1dulSzZ8/ubMkAAKCH6fJnYAKBgBwOh2JjY0Pay8rKNGjQIN10001asmSJzpw5Y/dVV1crMzNTTqfTbsvOzlZdXZ2+/PLLDq/T0tKiYDAYsgEAgJ6p03dgOuPUqVMqKirSfffdp5iYGLv9Zz/7mW6++WbFxcVpx44dKi4uVmNjo5YuXSpJ8vv9Sk1NDTlXQkKC3Tdw4MBzrlVaWqpFixZ14WwAAEB30WUBprW1VX//938vy7K0fPnykL7CwkL759GjR8vpdOrhhx9WaWmpXC7XRV2vuLg45LzBYFDJyckXVzwAAOjWuiTAtIeXTz75RNu2bQu5+9KRjIwMnTlzRocOHVJaWpo8Ho+amppCxrTvn++5GZfLddHhBwAAmCXsz8C0h5ePP/5YW7Zs0aBBgy54jM/nU69evRQfHy9J8nq9qqqqUmtrqz2moqJCaWlpHX58BAAAriydvgNz4sQJHThwwN6vr6+Xz+dTXFycEhMT9aMf/Uh79uzRhg0bdPbsWfn9fklSXFycnE6nqqurtXPnTk2YMEHR0dGqrq7W/Pnzdf/999vhZPr06Vq0aJHy8/NVVFSkffv26fnnn9ezzz4bpmkDAACTOSzLsjpzwPbt2zVhwoRz2vPy8lRSUnLOw7ft3n77bY0fP1579uzRT3/6U3300UdqaWlRamqqHnjgARUWFoZ8BFRbW6uCggLt2rVLgwcP1qOPPqqioqJvXWcwGJTb7VYgELjgR1idNXTBxrCe73I4VJYb6RIAALigb/v7u9MBxhQEmFAEGACACb7t72/+FhIAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMExXpAnB5DF2wMdIlXJRDZbmRLgEA0A1xBwYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxul0gKmqqtKUKVOUlJQkh8Oh9evXh/RblqWFCxcqMTFR/fr106RJk/Txxx+HjPniiy80Y8YMxcTEKDY2Vvn5+Tpx4kTImNraWo0bN059+/ZVcnKyysvLOz87AADQI3U6wDQ3N2vMmDFatmxZh/3l5eV64YUX9OKLL2rnzp0aMGCAsrOzderUKXvMjBkztH//flVUVGjDhg2qqqrS7Nmz7f5gMKisrCylpKSopqZGS5YsUUlJiV566aWLmCIAAOhpHJZlWRd9sMOhdevW6e6775b057svSUlJeuyxx/T4449LkgKBgBISErRy5UpNmzZNH374odLT07Vr1y7dcsstkqRNmzbpzjvv1KeffqqkpCQtX75c//AP/yC/3y+n0ylJWrBggdavX6+PPvroW9UWDAbldrsVCAQUExNzsVPskKl/2dlE/DVqALiyfNvf32F9Bqa+vl5+v1+TJk2y29xutzIyMlRdXS1Jqq6uVmxsrB1eJGnSpEnq1auXdu7caY/JzMy0w4skZWdnq66uTl9++WWH125paVEwGAzZAABAzxTWAOP3+yVJCQkJIe0JCQl2n9/vV3x8fEh/VFSU4uLiQsZ0dI6/vMbXlZaWyu1221tycvKlTwgAAHRLPeZbSMXFxQoEAvZ2+PDhSJcEAAC6SFgDjMfjkSQ1NTWFtDc1Ndl9Ho9HR48eDek/c+aMvvjii5AxHZ3jL6/xdS6XSzExMSEbAADomcIaYFJTU+XxeLR161a7LRgMaufOnfJ6vZIkr9erY8eOqaamxh6zbds2tbW1KSMjwx5TVVWl1tZWe0xFRYXS0tI0cODAcJYMAAAM1OkAc+LECfl8Pvl8Pkl/fnDX5/OpoaFBDodD8+bN0z/+4z/qv/7rv7R3717NnDlTSUlJ9jeVrr/+ek2ePFmzZs3Se++9p3feeUdz587VtGnTlJSUJEmaPn26nE6n8vPztX//fq1Zs0bPP/+8CgsLwzZxAABgrqjOHrB7925NmDDB3m8PFXl5eVq5cqV+8YtfqLm5WbNnz9axY8d0xx13aNOmTerbt699zKpVqzR37lxNnDhRvXr10tSpU/XCCy/Y/W63W5s3b1ZBQYHGjh2rwYMHa+HChSHvigEAAFeuS3oPTHfGe2B6Bt4DAwBXloi8BwYAAOByIMAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABin0y+yAy4nE9+5w7trAKDrcQcGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOGEPMEOHDpXD4ThnKygokCSNHz/+nL5HHnkk5BwNDQ3Kzc1V//79FR8fryeeeEJnzpwJd6kAAMBQUeE+4a5du3T27Fl7f9++ffqbv/kb/d3f/Z3dNmvWLC1evNje79+/v/3z2bNnlZubK4/Hox07dqixsVEzZ85Unz599Mwzz4S7XAAAYKCwB5ghQ4aE7JeVlWn48OH63ve+Z7f1799fHo+nw+M3b96sDz74QFu2bFFCQoJuvPFGPf300yoqKlJJSYmcTme4SwYAAIbp0mdgTp8+rf/4j//QQw89JIfDYbevWrVKgwcP1siRI1VcXKyTJ0/afdXV1Ro1apQSEhLstuzsbAWDQe3fv/+812ppaVEwGAzZAABAzxT2OzB/af369Tp27JgefPBBu2369OlKSUlRUlKSamtrVVRUpLq6Oq1du1aS5Pf7Q8KLJHvf7/ef91qlpaVatGhR+CcBAAC6nS4NMC+//LJycnKUlJRkt82ePdv+edSoUUpMTNTEiRN18OBBDR8+/KKvVVxcrMLCQns/GAwqOTn5os8HAAC6ry4LMJ988om2bNli31k5n4yMDEnSgQMHNHz4cHk8Hr333nshY5qamiTpvM/NSJLL5ZLL5brEqgEAgAm67BmYV155RfHx8crNzf3GcT6fT5KUmJgoSfJ6vdq7d6+OHj1qj6moqFBMTIzS09O7qlwAAGCQLrkD09bWpldeeUV5eXmKivr/lzh48KBWr16tO++8U4MGDVJtba3mz5+vzMxMjR49WpKUlZWl9PR0PfDAAyovL5ff79eTTz6pgoIC7rAAAABJXRRgtmzZooaGBj300EMh7U6nU1u2bNFzzz2n5uZmJScna+rUqXryySftMb1799aGDRs0Z84ceb1eDRgwQHl5eSHvjQEAAFe2LgkwWVlZsizrnPbk5GRVVlZe8PiUlBT9/ve/74rSAABAD8DfQgIAAMYhwAAAAOMQYAAAgHG69EV2wJVo6IKNkS6h0w6VffPrDgCgu+EODAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjBP2AFNSUiKHwxGyjRgxwu4/deqUCgoKNGjQIF111VWaOnWqmpqaQs7R0NCg3Nxc9e/fX/Hx8XriiSd05syZcJcKAAAMFdUVJ73hhhu0ZcuW/3+RqP9/mfnz52vjxo1644035Ha7NXfuXN1zzz165513JElnz55Vbm6uPB6PduzYocbGRs2cOVN9+vTRM8880xXlAgAAw3RJgImKipLH4zmnPRAI6OWXX9bq1av1/e9/X5L0yiuv6Prrr9e7776r2267TZs3b9YHH3ygLVu2KCEhQTfeeKOefvppFRUVqaSkRE6nsytKBgAABumSZ2A+/vhjJSUladiwYZoxY4YaGhokSTU1NWptbdWkSZPssSNGjNA111yj6upqSVJ1dbVGjRqlhIQEe0x2draCwaD2799/3mu2tLQoGAyGbAAAoGcKe4DJyMjQypUrtWnTJi1fvlz19fUaN26cjh8/Lr/fL6fTqdjY2JBjEhIS5Pf7JUl+vz8kvLT3t/edT2lpqdxut70lJyeHd2IAAKDbCPtHSDk5OfbPo0ePVkZGhlJSUvT666+rX79+4b6crbi4WIWFhfZ+MBgkxAAA0EN1+deoY2Njdd111+nAgQPyeDw6ffq0jh07FjKmqanJfmbG4/Gc862k9v2Onqtp53K5FBMTE7IBAICeqcsDzIkTJ3Tw4EElJiZq7Nix6tOnj7Zu3Wr319XVqaGhQV6vV5Lk9Xq1d+9eHT161B5TUVGhmJgYpaend3W5AADAAGH/COnxxx/XlClTlJKSoiNHjuipp55S7969dd9998ntdis/P1+FhYWKi4tTTEyMHn30UXm9Xt12222SpKysLKWnp+uBBx5QeXm5/H6/nnzySRUUFMjlcoW7XAAAYKCwB5hPP/1U9913nz7//HMNGTJEd9xxh959910NGTJEkvTss8+qV69emjp1qlpaWpSdna1//dd/tY/v3bu3NmzYoDlz5sjr9WrAgAHKy8vT4sWLw10qAAAwlMOyLCvSRXSFYDAot9utQCAQ9udhhi7YGNbzAZF2qCw30iUAgKRv//ubv4UEAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGCcqHCfsLS0VGvXrtVHH32kfv366a//+q/161//WmlpafaY8ePHq7KyMuS4hx9+WC+++KK939DQoDlz5ujtt9/WVVddpby8PJWWlioqKuwlA1e8oQs2RrqETjtUlhvpEgBEUNjTQGVlpQoKCnTrrbfqzJkz+uUvf6msrCx98MEHGjBggD1u1qxZWrx4sb3fv39/++ezZ88qNzdXHo9HO3bsUGNjo2bOnKk+ffromWeeCXfJAADAMGEPMJs2bQrZX7lypeLj41VTU6PMzEy7vX///vJ4PB2eY/Pmzfrggw+0ZcsWJSQk6MYbb9TTTz+toqIilZSUyOl0hrtsAABgkC5/BiYQCEiS4uLiQtpXrVqlwYMHa+TIkSouLtbJkyftvurqao0aNUoJCQl2W3Z2toLBoPbv39/VJQMAgG6uSx8oaWtr07x583T77bdr5MiRdvv06dOVkpKipKQk1dbWqqioSHV1dVq7dq0kye/3h4QXSfa+3+/v8FotLS1qaWmx94PBYLinA6Ab4bkd4MrWpQGmoKBA+/bt05/+9KeQ9tmzZ9s/jxo1SomJiZo4caIOHjyo4cOHX9S1SktLtWjRokuqFwAAmKHLPkKaO3euNmzYoLfffltXX331N47NyMiQJB04cECS5PF41NTUFDKmff98z80UFxcrEAjY2+HDhy91CgAAoJsKe4CxLEtz587VunXrtG3bNqWmpl7wGJ/PJ0lKTEyUJHm9Xu3du1dHjx61x1RUVCgmJkbp6ekdnsPlcikmJiZkAwAAPVPYP0IqKCjQ6tWr9eabbyo6Otp+ZsXtdqtfv346ePCgVq9erTvvvFODBg1SbW2t5s+fr8zMTI0ePVqSlJWVpfT0dD3wwAMqLy+X3+/Xk08+qYKCArlcrnCXDAAADBP2OzDLly9XIBDQ+PHjlZiYaG9r1qyRJDmdTm3ZskVZWVkaMWKEHnvsMU2dOlVvvfWWfY7evXtrw4YN6t27t7xer+6//37NnDkz5L0xAADgyhX2OzCWZX1jf3Jy8jlv4e1ISkqKfv/734erLAAA0IPwt5AAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHHC/iZeAEDHhi7YGOkSOu1QWW6kSwA6xB0YAABgHAIMAAAwDh8hAQB6FD6quzJwBwYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYJyrSBQAAuq+hCzZGugSgQ936DsyyZcs0dOhQ9e3bVxkZGXrvvfciXRIAAOgGum2AWbNmjQoLC/XUU09pz549GjNmjLKzs3X06NFIlwYAACKs2waYpUuXatasWfrxj3+s9PR0vfjii+rfv79WrFgR6dIAAECEdctnYE6fPq2amhoVFxfbbb169dKkSZNUXV3d4TEtLS1qaWmx9wOBgCQpGAyGvb62lpNhPycA4MrVFb+rTNX+b2FZ1jeO65YB5v/+7/909uxZJSQkhLQnJCToo48+6vCY0tJSLVq06Jz25OTkLqkRAIBwcT8X6Qq6n+PHj8vtdp+3v1sGmItRXFyswsJCe7+trU1ffPGFBg0aJIfDEcHKOi8YDCo5OVmHDx9WTExMpMu5LJgzc+6prrQ5X2nzlZhzuOdsWZaOHz+upKSkbxzXLQPM4MGD1bt3bzU1NYW0NzU1yePxdHiMy+WSy+UKaYuNje2qEi+LmJiYK+Y/QzvmfGVgzj3flTZfiTmH0zfdeWnXLR/idTqdGjt2rLZu3Wq3tbW1aevWrfJ6vRGsDAAAdAfd8g6MJBUWFiovL0+33HKLvvvd7+q5555Tc3OzfvzjH0e6NAAAEGHdNsDce++9+uyzz7Rw4UL5/X7deOON2rRp0zkP9vZELpdLTz311DkfifVkzPnKwJx7vittvhJzjhSHdaHvKQEAAHQz3fIZGAAAgG9CgAEAAMYhwAAAAOMQYAAAgHEIMN1ESUmJHA5HyDZixIhIlxVWVVVVmjJlipKSkuRwOLR+/fqQfsuytHDhQiUmJqpfv36aNGmSPv7448gUGyYXmvODDz54zrpPnjw5MsWGSWlpqW699VZFR0crPj5ed999t+rq6kLGnDp1SgUFBRo0aJCuuuoqTZ069ZwXV5rk28x5/Pjx56z1I488EqGKL93y5cs1evRo+0VmXq9Xf/jDH+z+nrbG0oXn3NPW+OvKysrkcDg0b948uy2S60yA6UZuuOEGNTY22tuf/vSnSJcUVs3NzRozZoyWLVvWYX95ebleeOEFvfjii9q5c6cGDBig7OxsnTp16jJXGj4XmrMkTZ48OWTdX3311ctYYfhVVlaqoKBA7777rioqKtTa2qqsrCw1NzfbY+bPn6+33npLb7zxhiorK3XkyBHdc889Eaz60nybOUvSrFmzQta6vLw8QhVfuquvvlplZWWqqanR7t279f3vf1933XWX9u/fL6nnrbF04TlLPWuN/9KuXbv0b//2bxo9enRIe0TX2UK38NRTT1ljxoyJdBmXjSRr3bp19n5bW5vl8XisJUuW2G3Hjh2zXC6X9eqrr0agwvD7+pwty7Ly8vKsu+66KyL1XC5Hjx61JFmVlZWWZf15Xfv06WO98cYb9pgPP/zQkmRVV1dHqsyw+vqcLcuyvve971k///nPI1fUZTBw4EDrt7/97RWxxu3a52xZPXeNjx8/bn3nO9+xKioqQuYY6XXmDkw38vHHHyspKUnDhg3TjBkz1NDQEOmSLpv6+nr5/X5NmjTJbnO73crIyFB1dXUEK+t627dvV3x8vNLS0jRnzhx9/vnnkS4prAKBgCQpLi5OklRTU6PW1taQtR4xYoSuueaaHrPWX59zu1WrVmnw4MEaOXKkiouLdfLkyUiUF3Znz57Va6+9pubmZnm93itijb8+53Y9cY0LCgqUm5sbsp5S5P8vd9s38V5pMjIytHLlSqWlpamxsVGLFi3SuHHjtG/fPkVHR0e6vC7n9/sl6Zw3LSckJNh9PdHkyZN1zz33KDU1VQcPHtQvf/lL5eTkqLq6Wr179450eZesra1N8+bN0+23366RI0dK+vNaO53Oc/7Yak9Z647mLEnTp09XSkqKkpKSVFtbq6KiItXV1Wnt2rURrPbS7N27V16vV6dOndJVV12ldevWKT09XT6fr8eu8fnmLPXMNX7ttde0Z88e7dq165y+SP9fJsB0Ezk5OfbPo0ePVkZGhlJSUvT6668rPz8/gpWhK02bNs3+edSoURo9erSGDx+u7du3a+LEiRGsLDwKCgq0b9++Hvc81zc535xnz55t/zxq1CglJiZq4sSJOnjwoIYPH365ywyLtLQ0+Xw+BQIB/ed//qfy8vJUWVkZ6bK61PnmnJ6e3uPW+PDhw/r5z3+uiooK9e3bN9LlnIOPkLqp2NhYXXfddTpw4ECkS7ksPB6PJJ3z9HpTU5PddyUYNmyYBg8e3CPWfe7cudqwYYPefvttXX311Xa7x+PR6dOndezYsZDxPWGtzzfnjmRkZEiS0WvtdDp17bXXauzYsSotLdWYMWP0/PPP9+g1Pt+cO2L6GtfU1Ojo0aO6+eabFRUVpaioKFVWVuqFF15QVFSUEhISIrrOBJhu6sSJEzp48KASExMjXcplkZqaKo/Ho61bt9ptwWBQO3fuDPl8uaf79NNP9fnnnxu97pZlae7cuVq3bp22bdum1NTUkP6xY8eqT58+IWtdV1enhoYGY9f6QnPuiM/nkySj1/rr2tra1NLS0iPX+Hza59wR09d44sSJ2rt3r3w+n73dcsstmjFjhv1zRNe5yx8Txrfy2GOPWdu3b7fq6+utd955x5o0aZI1ePBg6+jRo5EuLWyOHz9uvf/++9b7779vSbKWLl1qvf/++9Ynn3xiWZZllZWVWbGxsdabb75p1dbWWnfddZeVmppqffXVVxGu/OJ905yPHz9uPf7441Z1dbVVX19vbdmyxbr55put73znO9apU6ciXfpFmzNnjuV2u63t27dbjY2N9nby5El7zCOPPGJdc8011rZt26zdu3dbXq/X8nq9Eaz60lxozgcOHLAWL15s7d6926qvr7fefPNNa9iwYVZmZmaEK794CxYssCorK636+nqrtrbWWrBggeVwOKzNmzdbltXz1tiyvnnOPXGNO/L1b1pFcp0JMN3EvffeayUmJlpOp9P6q7/6K+vee++1Dhw4EOmywurtt9+2JJ2z5eXlWZb1569S/+pXv7ISEhIsl8tlTZw40aqrq4ts0Zfom+Z88uRJKysryxoyZIjVp08fKyUlxZo1a5bl9/sjXfYl6Wi+kqxXXnnFHvPVV19ZP/3pT62BAwda/fv3t/72b//WamxsjFzRl+hCc25oaLAyMzOtuLg4y+VyWddee631xBNPWIFAILKFX4KHHnrISklJsZxOpzVkyBBr4sSJdnixrJ63xpb1zXPuiWvcka8HmEius8OyLKvr7/MAAACED8/AAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGCc/wc6r0uiINiy2QAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.hist(lens)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "228acd24",
   "metadata": {
    "papermill": {
     "duration": 0.014624,
     "end_time": "2024-03-23T04:44:33.188784",
     "exception": false,
     "start_time": "2024-03-23T04:44:33.174160",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n",
    "   Installing Transformers\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "b0890388",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:44:33.219521Z",
     "iopub.status.busy": "2024-03-23T04:44:33.219196Z",
     "iopub.status.idle": "2024-03-23T04:44:47.466519Z",
     "shell.execute_reply": "2024-03-23T04:44:47.465480Z"
    },
    "papermill": {
     "duration": 14.265637,
     "end_time": "2024-03-23T04:44:47.469364",
     "exception": false,
     "start_time": "2024-03-23T04:44:33.203727",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: transformers in /opt/conda/lib/python3.10/site-packages (4.38.2)\r\n",
      "Requirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from transformers) (3.13.1)\r\n",
      "Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in /opt/conda/lib/python3.10/site-packages (from transformers) (0.21.4)\r\n",
      "Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.10/site-packages (from transformers) (1.26.4)\r\n",
      "Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.10/site-packages (from transformers) (21.3)\r\n",
      "Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.10/site-packages (from transformers) (6.0.1)\r\n",
      "Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.10/site-packages (from transformers) (2023.12.25)\r\n",
      "Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from transformers) (2.31.0)\r\n",
      "Requirement already satisfied: tokenizers<0.19,>=0.14 in /opt/conda/lib/python3.10/site-packages (from transformers) (0.15.2)\r\n",
      "Requirement already satisfied: safetensors>=0.4.1 in /opt/conda/lib/python3.10/site-packages (from transformers) (0.4.2)\r\n",
      "Requirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.10/site-packages (from transformers) (4.66.1)\r\n",
      "Requirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers) (2024.3.0)\r\n",
      "Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers) (4.9.0)\r\n",
      "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/lib/python3.10/site-packages (from packaging>=20.0->transformers) (3.1.1)\r\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (3.3.2)\r\n",
      "Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (3.6)\r\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (1.26.18)\r\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (2024.2.2)\r\n"
     ]
    }
   ],
   "source": [
    "!pip install transformers"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4a63b205",
   "metadata": {
    "papermill": {
     "duration": 0.014781,
     "end_time": "2024-03-23T04:44:47.499625",
     "exception": false,
     "start_time": "2024-03-23T04:44:47.484844",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n",
    "   Train-Test-Split\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "5c640944",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:44:47.532085Z",
     "iopub.status.busy": "2024-03-23T04:44:47.531676Z",
     "iopub.status.idle": "2024-03-23T04:44:47.552163Z",
     "shell.execute_reply": "2024-03-23T04:44:47.551270Z"
    },
    "papermill": {
     "duration": 0.039773,
     "end_time": "2024-03-23T04:44:47.554353",
     "exception": false,
     "start_time": "2024-03-23T04:44:47.514580",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# split into train, validation and test sets in the ration 70 : 15 : 15\n",
    "train_text, temp_text, train_labels, temp_labels = train_test_split(df['sentence'], df['label'], \n",
    "                                                                    random_state=2021, \n",
    "                                                                    test_size=0.3, \n",
    "                                                                    stratify=df['label'])\n",
    "\n",
    "\n",
    "val_text, test_text, val_labels, test_labels = train_test_split(temp_text, temp_labels, \n",
    "                                                                random_state=2021, \n",
    "                                                                test_size=0.5, \n",
    "                                                                stratify=temp_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f984b04",
   "metadata": {
    "papermill": {
     "duration": 0.01461,
     "end_time": "2024-03-23T04:44:47.584222",
     "exception": false,
     "start_time": "2024-03-23T04:44:47.569612",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n",
    "   IMPORTING BERT PRE-TRAINED MODEL\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "10d67c8e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:44:47.657409Z",
     "iopub.status.busy": "2024-03-23T04:44:47.656773Z",
     "iopub.status.idle": "2024-03-23T04:44:52.438017Z",
     "shell.execute_reply": "2024-03-23T04:44:52.437038Z"
    },
    "papermill": {
     "duration": 4.840912,
     "end_time": "2024-03-23T04:44:52.440469",
     "exception": false,
     "start_time": "2024-03-23T04:44:47.599557",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "144eaf7d596a4613907d07e2cc728277",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "config.json:   0%|          | 0.00/570 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1a9f427d528547d6aa4b37a5d7853c08",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "model.safetensors:   0%|          | 0.00/440M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# import BERT-base pretrained model\n",
    "bert = AutoModel.from_pretrained('bert-base-uncased')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1f0aeb7",
   "metadata": {
    "papermill": {
     "duration": 0.01685,
     "end_time": "2024-03-23T04:44:52.474864",
     "exception": false,
     "start_time": "2024-03-23T04:44:52.458014",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n",
    "   IMPORTING BERT TOKENIZER\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "e17cc457",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:44:52.512707Z",
     "iopub.status.busy": "2024-03-23T04:44:52.511236Z",
     "iopub.status.idle": "2024-03-23T04:44:53.090009Z",
     "shell.execute_reply": "2024-03-23T04:44:53.089168Z"
    },
    "papermill": {
     "duration": 0.599629,
     "end_time": "2024-03-23T04:44:53.092384",
     "exception": false,
     "start_time": "2024-03-23T04:44:52.492755",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "76131eb1aea6496399f96fce1fc94c48",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "tokenizer_config.json:   0%|          | 0.00/48.0 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "66aed60cdd2d40ddabb3035d2b775f04",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "vocab.txt:   0%|          | 0.00/232k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ba028e93bc79411db699ea7a51fda4b0",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "tokenizer.json:   0%|          | 0.00/466k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Load the BERT tokenizer\n",
    "tokenizer = BertTokenizerFast.from_pretrained('bert-base-uncased')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "8af6d8dc",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:44:53.126636Z",
     "iopub.status.busy": "2024-03-23T04:44:53.126314Z",
     "iopub.status.idle": "2024-03-23T04:44:53.440742Z",
     "shell.execute_reply": "2024-03-23T04:44:53.439743Z"
    },
    "papermill": {
     "duration": 0.333641,
     "end_time": "2024-03-23T04:44:53.442965",
     "exception": false,
     "start_time": "2024-03-23T04:44:53.109324",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([1504.,  701.,  572.,  521.,  164.,  215.,  117.,   63.,  104.,\n",
       "           6.]),\n",
       " array([ 3. ,  6.5, 10. , 13.5, 17. , 20.5, 24. , 27.5, 31. , 34.5, 38. ]),\n",
       " <BarContainer object of 10 artists>)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjAAAAGdCAYAAAAMm0nCAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/xnp5ZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAoCUlEQVR4nO3df3CU9YHH8c+GkBCB3RAwu+yZhFQ9IArYgo1blWqTIUDKlTO9mprD1GbI1Sa0EEVILQjWGow9lXiUlP4QZxqq9abQiieaAyWtxgDhcmDEFDk0obiJ15hdE48QyHN/ODzThSA/3LD5Ju/XzM6wz/PdZ7/f3bV598nuxmFZliUAAACDREV6AgAAABeKgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgnOhIT6C/9Pb26ujRoxo9erQcDkekpwMAAM6DZVn66KOP5PV6FRV19vMsgzZgjh49qqSkpEhPAwAAXISWlhZdccUVZ90/aANm9OjRkj55AJxOZ4RnAwAAzkcwGFRSUpL9c/xsBm3AnPq1kdPpJGAAADDMud7+wZt4AQCAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgnOhIT8BEE5a/EOkpXLB312RHegoAAIQNZ2AAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGOeCA6ampkbz5s2T1+uVw+HQli1bzjr2O9/5jhwOh5544omQ7e3t7crLy5PT6VR8fLwKCgrU2dkZMmbfvn26+eabNWLECCUlJam8vPxCpwoAAAapCw6Yrq4uTZs2TevWrfvUcZs3b9Ybb7whr9d7xr68vDw1NjaqurpaW7duVU1NjQoLC+39wWBQs2bNUkpKiurr6/Xoo49q1apV2rBhw4VOFwAADELRF3qDOXPmaM6cOZ865i9/+YsWLVqkl156SdnZ2SH7Dhw4oG3btmn37t2aMWOGJOnJJ5/U3Llz9ZOf/ERer1dVVVU6fvy4fvWrXykmJkbXXHONGhoa9Nhjj4WEDgAAGJrC/h6Y3t5eLViwQEuXLtU111xzxv7a2lrFx8fb8SJJmZmZioqKUl1dnT1m5syZiomJscdkZWWpqalJH374YbinDAAADHPBZ2DO5ZFHHlF0dLS+973v9bnf7/crMTExdBLR0UpISJDf77fHpKamhoxxu932vjFjxpxx3O7ubnV3d9vXg8HgZ1oHAAAYuMJ6Bqa+vl5r167Vxo0b5XA4wnnocyorK5PL5bIvSUlJl/T+AQDApRPWgPnjH/+otrY2JScnKzo6WtHR0Xrvvfd0zz33aMKECZIkj8ejtra2kNudOHFC7e3t8ng89pjW1taQMaeunxpzutLSUgUCAfvS0tISzqUBAIABJKy/QlqwYIEyMzNDtmVlZWnBggW66667JEk+n08dHR2qr6/X9OnTJUk7duxQb2+v0tPT7TH333+/enp6NHz4cElSdXW1Jk6c2OevjyQpNjZWsbGx4VwOAAAYoC44YDo7O/XOO+/Y1w8fPqyGhgYlJCQoOTlZY8eODRk/fPhweTweTZw4UZI0efJkzZ49WwsXLlRlZaV6enpUXFys3Nxc+yPXd9xxh1avXq2CggItW7ZMb775ptauXavHH3/8s6wVAAAMEhccMHv27NGtt95qXy8pKZEk5efna+PGjed1jKqqKhUXFysjI0NRUVHKyclRRUWFvd/lcunll19WUVGRpk+frnHjxmnlypV8hBoAAEiSHJZlWZGeRH8IBoNyuVwKBAJyOp1hPfaE5S+E9XiXwrtrss89CACACDvfn9/8LSQAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxLjhgampqNG/ePHm9XjkcDm3ZssXe19PTo2XLlmnKlCkaOXKkvF6v7rzzTh09ejTkGO3t7crLy5PT6VR8fLwKCgrU2dkZMmbfvn26+eabNWLECCUlJam8vPziVggAAAadCw6Yrq4uTZs2TevWrTtj38cff6y9e/dqxYoV2rt3r373u9+pqalJ//AP/xAyLi8vT42NjaqurtbWrVtVU1OjwsJCe38wGNSsWbOUkpKi+vp6Pfroo1q1apU2bNhwEUsEAACDjcOyLOuib+xwaPPmzZo/f/5Zx+zevVtf/OIX9d577yk5OVkHDhxQWlqadu/erRkzZkiStm3bprlz5+rIkSPyer1av3697r//fvn9fsXExEiSli9fri1btujtt98+r7kFg0G5XC4FAgE5nc6LXWKfJix/IazHuxTeXZMd6SkAAHBO5/vzu9/fAxMIBORwOBQfHy9Jqq2tVXx8vB0vkpSZmamoqCjV1dXZY2bOnGnHiyRlZWWpqalJH374YZ/3093drWAwGHIBAACDU78GzLFjx7Rs2TJ985vftCvK7/crMTExZFx0dLQSEhLk9/vtMW63O2TMqeunxpyurKxMLpfLviQlJYV7OQAAYIDot4Dp6enRN77xDVmWpfXr1/fX3dhKS0sVCATsS0tLS7/fJwAAiIzo/jjoqXh57733tGPHjpDfYXk8HrW1tYWMP3HihNrb2+XxeOwxra2tIWNOXT815nSxsbGKjY0N5zIAAMAAFfYzMKfi5eDBg/rP//xPjR07NmS/z+dTR0eH6uvr7W07duxQb2+v0tPT7TE1NTXq6emxx1RXV2vixIkaM2ZMuKcMAAAMc8EB09nZqYaGBjU0NEiSDh8+rIaGBjU3N6unp0df//rXtWfPHlVVVenkyZPy+/3y+/06fvy4JGny5MmaPXu2Fi5cqF27dum1115TcXGxcnNz5fV6JUl33HGHYmJiVFBQoMbGRj377LNau3atSkpKwrdyAABgrAv+GPWrr76qW2+99Yzt+fn5WrVqlVJTU/u83SuvvKJbbrlF0idfZFdcXKznn39eUVFRysnJUUVFhUaNGmWP37dvn4qKirR7926NGzdOixYt0rJly857nnyMOhQfowYAmOB8f35/pu+BGcgImFAEDADABAPme2AAAADCjYABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgnAsOmJqaGs2bN09er1cOh0NbtmwJ2W9ZllauXKnx48crLi5OmZmZOnjwYMiY9vZ25eXlyel0Kj4+XgUFBers7AwZs2/fPt18880aMWKEkpKSVF5efuGrAwAAg9IFB0xXV5emTZumdevW9bm/vLxcFRUVqqysVF1dnUaOHKmsrCwdO3bMHpOXl6fGxkZVV1dr69atqqmpUWFhob0/GAxq1qxZSklJUX19vR599FGtWrVKGzZsuIglAgCAwcZhWZZ10Td2OLR582bNnz9f0idnX7xer+655x7de++9kqRAICC3262NGzcqNzdXBw4cUFpamnbv3q0ZM2ZIkrZt26a5c+fqyJEj8nq9Wr9+ve6//375/X7FxMRIkpYvX64tW7bo7bffPq+5BYNBuVwuBQIBOZ3Oi11inyYsfyGsx7sU3l2THekpAABwTuf78zus74E5fPiw/H6/MjMz7W0ul0vp6emqra2VJNXW1io+Pt6OF0nKzMxUVFSU6urq7DEzZ86040WSsrKy1NTUpA8//LDP++7u7lYwGAy5AACAwSmsAeP3+yVJbrc7ZLvb7bb3+f1+JSYmhuyPjo5WQkJCyJi+jvG393G6srIyuVwu+5KUlPTZFwQAAAakQfMppNLSUgUCAfvS0tIS6SkBAIB+EtaA8Xg8kqTW1taQ7a2trfY+j8ejtra2kP0nTpxQe3t7yJi+jvG393G62NhYOZ3OkAsAABicwhowqamp8ng82r59u70tGAyqrq5OPp9PkuTz+dTR0aH6+np7zI4dO9Tb26v09HR7TE1NjXp6euwx1dXVmjhxosaMGRPOKQMAAANdcMB0dnaqoaFBDQ0Nkj55425DQ4Oam5vlcDi0ePFiPfTQQ/rDH/6g/fv3684775TX67U/qTR58mTNnj1bCxcu1K5du/Taa6+puLhYubm58nq9kqQ77rhDMTExKigoUGNjo5599lmtXbtWJSUlYVs4AAAwV/SF3mDPnj269dZb7eunoiI/P18bN27Ufffdp66uLhUWFqqjo0M33XSTtm3bphEjRti3qaqqUnFxsTIyMhQVFaWcnBxVVFTY+10ul15++WUVFRVp+vTpGjdunFauXBnyXTEAAGDo+kzfAzOQ8T0wofgeGACACSLyPTAAAACXAgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOGEPmJMnT2rFihVKTU1VXFycrrzySv3oRz+SZVn2GMuytHLlSo0fP15xcXHKzMzUwYMHQ47T3t6uvLw8OZ1OxcfHq6CgQJ2dneGeLgAAMFDYA+aRRx7R+vXr9W//9m86cOCAHnnkEZWXl+vJJ5+0x5SXl6uiokKVlZWqq6vTyJEjlZWVpWPHjtlj8vLy1NjYqOrqam3dulU1NTUqLCwM93QBAICBHNbfnhoJg69+9atyu9365S9/aW/LyclRXFycfv3rX8uyLHm9Xt1zzz269957JUmBQEBut1sbN25Ubm6uDhw4oLS0NO3evVszZsyQJG3btk1z587VkSNH5PV6zzmPYDAol8ulQCAgp9MZziVqwvIXwnq8S+HdNdmRngIAAOd0vj+/w34G5ktf+pK2b9+uP//5z5Kk//7v/9af/vQnzZkzR5J0+PBh+f1+ZWZm2rdxuVxKT09XbW2tJKm2tlbx8fF2vEhSZmamoqKiVFdX1+f9dnd3KxgMhlwAAMDgFB3uAy5fvlzBYFCTJk3SsGHDdPLkSf34xz9WXl6eJMnv90uS3G53yO3cbre9z+/3KzExMXSi0dFKSEiwx5yurKxMq1evDvdyAADAABT2MzC//e1vVVVVpU2bNmnv3r16+umn9ZOf/ERPP/10uO8qRGlpqQKBgH1paWnp1/sDAACRE/YzMEuXLtXy5cuVm5srSZoyZYree+89lZWVKT8/Xx6PR5LU2tqq8ePH27drbW3VddddJ0nyeDxqa2sLOe6JEyfU3t5u3/50sbGxio2NDfdyAADAABT2MzAff/yxoqJCDzts2DD19vZKklJTU+XxeLR9+3Z7fzAYVF1dnXw+nyTJ5/Opo6ND9fX19pgdO3aot7dX6enp4Z4yAAAwTNjPwMybN08//vGPlZycrGuuuUb/9V//pccee0zf/va3JUkOh0OLFy/WQw89pKuvvlqpqalasWKFvF6v5s+fL0maPHmyZs+erYULF6qyslI9PT0qLi5Wbm7ueX0CCQAADG5hD5gnn3xSK1as0He/+121tbXJ6/XqX/7lX7Ry5Up7zH333aeuri4VFhaqo6NDN910k7Zt26YRI0bYY6qqqlRcXKyMjAxFRUUpJydHFRUV4Z4uAAAwUNi/B2ag4HtgQvE9MAAAE0Tse2AAAAD6GwEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA40ZGeAC6NCctfiPQULsq7a7IjPQUAwADEGRgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxumXgPnLX/6if/7nf9bYsWMVFxenKVOmaM+ePfZ+y7K0cuVKjR8/XnFxccrMzNTBgwdDjtHe3q68vDw5nU7Fx8eroKBAnZ2d/TFdAABgmLAHzIcffqgbb7xRw4cP14svvqi33npL//qv/6oxY8bYY8rLy1VRUaHKykrV1dVp5MiRysrK0rFjx+wxeXl5amxsVHV1tbZu3aqamhoVFhaGe7oAAMBADsuyrHAecPny5Xrttdf0xz/+sc/9lmXJ6/Xqnnvu0b333itJCgQCcrvd2rhxo3Jzc3XgwAGlpaVp9+7dmjFjhiRp27Ztmjt3ro4cOSKv13vOeQSDQblcLgUCATmdzvAtUNKE5S+E9Xg4u3fXZEd6CgCAS+h8f36H/QzMH/7wB82YMUP/9E//pMTERH3+85/Xz3/+c3v/4cOH5ff7lZmZaW9zuVxKT09XbW2tJKm2tlbx8fF2vEhSZmamoqKiVFdX1+f9dnd3KxgMhlwAAMDgFPaA+Z//+R+tX79eV199tV566SXdfffd+t73vqenn35akuT3+yVJbrc75HZut9ve5/f7lZiYGLI/OjpaCQkJ9pjTlZWVyeVy2ZekpKRwLw0AAAwQYQ+Y3t5efeELX9DDDz+sz3/+8yosLNTChQtVWVkZ7rsKUVpaqkAgYF9aWlr69f4AAEDkhD1gxo8fr7S0tJBtkydPVnNzsyTJ4/FIklpbW0PGtLa22vs8Ho/a2tpC9p84cULt7e32mNPFxsbK6XSGXAAAwOAU9oC58cYb1dTUFLLtz3/+s1JSUiRJqamp8ng82r59u70/GAyqrq5OPp9PkuTz+dTR0aH6+np7zI4dO9Tb26v09PRwTxkAABgmOtwHXLJkib70pS/p4Ycf1je+8Q3t2rVLGzZs0IYNGyRJDodDixcv1kMPPaSrr75aqampWrFihbxer+bPny/pkzM2s2fPtn/11NPTo+LiYuXm5p7XJ5AAAMDgFvaAuf7667V582aVlpbqwQcfVGpqqp544gnl5eXZY+677z51dXWpsLBQHR0duummm7Rt2zaNGDHCHlNVVaXi4mJlZGQoKipKOTk5qqioCPd0AQCAgcL+PTADBd8Dg0jhu2sA4OJF7HtgAAAA+hsBAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOOE/Y85AkOdiX8ri7/fBMA0nIEBAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgnH4PmDVr1sjhcGjx4sX2tmPHjqmoqEhjx47VqFGjlJOTo9bW1pDbNTc3Kzs7W5dddpkSExO1dOlSnThxor+nCwAADNCvAbN792797Gc/09SpU0O2L1myRM8//7yee+457dy5U0ePHtVtt91m7z958qSys7N1/Phxvf7663r66ae1ceNGrVy5sj+nCwAADNFvAdPZ2am8vDz9/Oc/15gxY+ztgUBAv/zlL/XYY4/pK1/5iqZPn66nnnpKr7/+ut544w1J0ssvv6y33npLv/71r3Xddddpzpw5+tGPfqR169bp+PHj/TVlAABgiH4LmKKiImVnZyszMzNke319vXp6ekK2T5o0ScnJyaqtrZUk1dbWasqUKXK73faYrKwsBYNBNTY29teUAQCAIaL746DPPPOM9u7dq927d5+xz+/3KyYmRvHx8SHb3W63/H6/PeZv4+XU/lP7+tLd3a3u7m77ejAY/CxLAAAAA1jYz8C0tLTo+9//vqqqqjRixIhwH/6sysrK5HK57EtSUtIlu28AAHBphT1g6uvr1dbWpi984QuKjo5WdHS0du7cqYqKCkVHR8vtduv48ePq6OgIuV1ra6s8Ho8kyePxnPGppFPXT405XWlpqQKBgH1paWkJ99IAAMAAEfaAycjI0P79+9XQ0GBfZsyYoby8PPvfw4cP1/bt2+3bNDU1qbm5WT6fT5Lk8/m0f/9+tbW12WOqq6vldDqVlpbW5/3GxsbK6XSGXAAAwOAU9vfAjB49Wtdee23ItpEjR2rs2LH29oKCApWUlCghIUFOp1OLFi2Sz+fTDTfcIEmaNWuW0tLStGDBApWXl8vv9+uHP/yhioqKFBsbG+4pAwAAw/TLm3jP5fHHH1dUVJRycnLU3d2trKws/fSnP7X3Dxs2TFu3btXdd98tn8+nkSNHKj8/Xw8++GAkpgsAAAYYh2VZVqQn0R+CwaBcLpcCgUDYf500YfkLYT0eEGnvrsmO9BQAQNL5//zmbyEBAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACME/aAKSsr0/XXX6/Ro0crMTFR8+fPV1NTU8iYY8eOqaioSGPHjtWoUaOUk5Oj1tbWkDHNzc3Kzs7WZZddpsTERC1dulQnTpwI93QBAICBosN9wJ07d6qoqEjXX3+9Tpw4oR/84AeaNWuW3nrrLY0cOVKStGTJEr3wwgt67rnn5HK5VFxcrNtuu02vvfaaJOnkyZPKzs6Wx+PR66+/rvfff1933nmnhg8frocffjjcUwZgoAnLX4j0FC7Yu2uyIz0FYNBwWJZl9ecdfPDBB0pMTNTOnTs1c+ZMBQIBXX755dq0aZO+/vWvS5LefvttTZ48WbW1tbrhhhv04osv6qtf/aqOHj0qt9stSaqsrNSyZcv0wQcfKCYm5pz3GwwG5XK5FAgE5HQ6w7omE/+HE/g0Jv5gNfG/QxMfZ+BSO9+f3/3+HphAICBJSkhIkCTV19erp6dHmZmZ9phJkyYpOTlZtbW1kqTa2lpNmTLFjhdJysrKUjAYVGNjY5/3093drWAwGHIBAACDU78GTG9vrxYvXqwbb7xR1157rSTJ7/crJiZG8fHxIWPdbrf8fr895m/j5dT+U/v6UlZWJpfLZV+SkpLCvBoAADBQ9GvAFBUV6c0339QzzzzTn3cjSSotLVUgELAvLS0t/X6fAAAgMsL+Jt5TiouLtXXrVtXU1OiKK66wt3s8Hh0/flwdHR0hZ2FaW1vl8XjsMbt27Qo53qlPKZ0ac7rY2FjFxsaGeRUAAGAgCvsZGMuyVFxcrM2bN2vHjh1KTU0N2T99+nQNHz5c27dvt7c1NTWpublZPp9PkuTz+bR//361tbXZY6qrq+V0OpWWlhbuKQMAAMOE/QxMUVGRNm3apN///vcaPXq0/Z4Vl8uluLg4uVwuFRQUqKSkRAkJCXI6nVq0aJF8Pp9uuOEGSdKsWbOUlpamBQsWqLy8XH6/Xz/84Q9VVFTEWRYAABD+gFm/fr0k6ZZbbgnZ/tRTT+lb3/qWJOnxxx9XVFSUcnJy1N3draysLP30pz+1xw4bNkxbt27V3XffLZ/Pp5EjRyo/P18PPvhguKcLAAAMFPaAOZ+vlRkxYoTWrVundevWnXVMSkqK/uM//iOcUwMAAIMEfwsJAAAYh4ABAADG6bePUQMwh4lfyw9gaOMMDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIzDnxIAgEvExD/Z8O6a7EhPAegTZ2AAAIBxCBgAAGAcfoUEABhU+FXd0MAZGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcvokXAHBWJn6rLYYGzsAAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4wzogFm3bp0mTJigESNGKD09Xbt27Yr0lAAAwAAwYAPm2WefVUlJiR544AHt3btX06ZNU1ZWltra2iI9NQAAEGEDNmAee+wxLVy4UHfddZfS0tJUWVmpyy67TL/61a8iPTUAABBhA/KPOR4/flz19fUqLS21t0VFRSkzM1O1tbV93qa7u1vd3d329UAgIEkKBoNhn19v98dhPyYAYOhKXvJcpKdwwd5cndUvxz31c9uyrE8dNyAD5n//93918uRJud3ukO1ut1tvv/12n7cpKyvT6tWrz9ielJTUL3MEAGAocz3Rv8f/6KOP5HK5zrp/QAbMxSgtLVVJSYl9vbe3V+3t7Ro7dqwcDkcEZ3ZxgsGgkpKS1NLSIqfTGenpRMRQfwxY/9Bev8RjMNTXLw3Nx8CyLH300Ufyer2fOm5ABsy4ceM0bNgwtba2hmxvbW2Vx+Pp8zaxsbGKjY0N2RYfH99fU7xknE7nkHnRns1QfwxY/9Bev8RjMNTXLw29x+DTzrycMiDfxBsTE6Pp06dr+/bt9rbe3l5t375dPp8vgjMDAAADwYA8AyNJJSUlys/P14wZM/TFL35RTzzxhLq6unTXXXdFemoAACDCBmzA3H777frggw+0cuVK+f1+XXfdddq2bdsZb+wdrGJjY/XAAw+c8WuxoWSoPwasf2ivX+IxGOrrl3gMPo3DOtfnlAAAAAaYAfkeGAAAgE9DwAAAAOMQMAAAwDgEDAAAMA4BM8CsWrVKDocj5DJp0qRIT6vf1NTUaN68efJ6vXI4HNqyZUvIfsuytHLlSo0fP15xcXHKzMzUwYMHIzPZfnKux+Bb3/rWGa+J2bNnR2ay/aCsrEzXX3+9Ro8ercTERM2fP19NTU0hY44dO6aioiKNHTtWo0aNUk5OzhlfdGmq81n/LbfccsZr4Dvf+U6EZhx+69ev19SpU+0va/P5fHrxxRft/YP5+ZfOvf7B/vxfLAJmALrmmmv0/vvv25c//elPkZ5Sv+nq6tK0adO0bt26PveXl5eroqJClZWVqqur08iRI5WVlaVjx45d4pn2n3M9BpI0e/bskNfEb37zm0s4w/61c+dOFRUV6Y033lB1dbV6eno0a9YsdXV12WOWLFmi559/Xs8995x27typo0eP6rbbbovgrMPnfNYvSQsXLgx5DZSXl0doxuF3xRVXaM2aNaqvr9eePXv0la98RV/72tfU2NgoaXA//9K51y8N7uf/olkYUB544AFr2rRpkZ5GREiyNm/ebF/v7e21PB6P9eijj9rbOjo6rNjYWOs3v/lNBGbY/05/DCzLsvLz862vfe1rEZlPJLS1tVmSrJ07d1qW9clzPnz4cOu5556zxxw4cMCSZNXW1kZqmv3m9PVblmV9+ctftr7//e9HblIRMGbMGOsXv/jFkHv+Tzm1fssams//+eAMzAB08OBBeb1efe5zn1NeXp6am5sjPaWIOHz4sPx+vzIzM+1tLpdL6enpqq2tjeDMLr1XX31ViYmJmjhxou6++2799a9/jfSU+k0gEJAkJSQkSJLq6+vV09MT8jqYNGmSkpOTB+Xr4PT1n1JVVaVx48bp2muvVWlpqT7++ONITK/fnTx5Us8884y6urrk8/mG3PN/+vpPGSrP/4UYsN/EO1Slp6dr48aNmjhxot5//32tXr1aN998s958802NHj060tO7pPx+vySd8e3Lbrfb3jcUzJ49W7fddptSU1N16NAh/eAHP9CcOXNUW1urYcOGRXp6YdXb26vFixfrxhtv1LXXXivpk9dBTEzMGX+cdTC+DvpavyTdcccdSklJkdfr1b59+7Rs2TI1NTXpd7/7XQRnG1779++Xz+fTsWPHNGrUKG3evFlpaWlqaGgYEs//2dYvDY3n/2IQMAPMnDlz7H9PnTpV6enpSklJ0W9/+1sVFBREcGaIlNzcXPvfU6ZM0dSpU3XllVfq1VdfVUZGRgRnFn5FRUV68803B/X7vj7N2dZfWFho/3vKlCkaP368MjIydOjQIV155ZWXepr9YuLEiWpoaFAgENC///u/Kz8/Xzt37oz0tC6Zs60/LS1tSDz/F4NfIQ1w8fHx+vu//3u98847kZ7KJefxeCTpjE8btLa22vuGos997nMaN27coHtNFBcXa+vWrXrllVd0xRVX2Ns9Ho+OHz+ujo6OkPGD7XVwtvX3JT09XZIG1WsgJiZGV111laZPn66ysjJNmzZNa9euHTLP/9nW35fB+PxfDAJmgOvs7NShQ4c0fvz4SE/lkktNTZXH49H27dvtbcFgUHV1dSG/Gx5qjhw5or/+9a+D5jVhWZaKi4u1efNm7dixQ6mpqSH7p0+fruHDh4e8DpqamtTc3DwoXgfnWn9fGhoaJGnQvAb60tvbq+7u7kH//J/NqfX3ZSg8/+eDXyENMPfee6/mzZunlJQUHT16VA888ICGDRumb37zm5GeWr/o7OwM+X8Rhw8fVkNDgxISEpScnKzFixfroYce0tVXX63U1FStWLFCXq9X8+fPj9ykw+zTHoOEhAStXr1aOTk58ng8OnTokO677z5dddVVysrKiuCsw6eoqEibNm3S73//e40ePdp+X4PL5VJcXJxcLpcKCgpUUlKihIQEOZ1OLVq0SD6fTzfccEOEZ//ZnWv9hw4d0qZNmzR37lyNHTtW+/bt05IlSzRz5kxNnTo1wrMPj9LSUs2ZM0fJycn66KOPtGnTJr366qt66aWXBv3zL336+ofC83/RIv0xKIS6/fbbrfHjx1sxMTHW3/3d31m333679c4770R6Wv3mlVdesSSdccnPz7cs65OPUq9YscJyu91WbGyslZGRYTU1NUV20mH2aY/Bxx9/bM2aNcu6/PLLreHDh1spKSnWwoULLb/fH+lph01fa5dkPfXUU/aY//u//7O++93vWmPGjLEuu+wy6x//8R+t999/P3KTDqNzrb+5udmaOXOmlZCQYMXGxlpXXXWVtXTpUisQCER24mH07W9/20pJSbFiYmKsyy+/3MrIyLBefvlle/9gfv4t69PXPxSe/4vlsCzLupTBBAAA8FnxHhgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBx/h/4Co8HxFNrGAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "train_lens=[len(i.split()) for i in train_text]\n",
    "plt.hist(train_lens)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "f3b0d896",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:44:53.478972Z",
     "iopub.status.busy": "2024-03-23T04:44:53.478641Z",
     "iopub.status.idle": "2024-03-23T04:44:53.482902Z",
     "shell.execute_reply": "2024-03-23T04:44:53.481958Z"
    },
    "papermill": {
     "duration": 0.024471,
     "end_time": "2024-03-23T04:44:53.484934",
     "exception": false,
     "start_time": "2024-03-23T04:44:53.460463",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "pad_len=17"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f1832c5",
   "metadata": {
    "papermill": {
     "duration": 0.016854,
     "end_time": "2024-03-23T04:44:53.519134",
     "exception": false,
     "start_time": "2024-03-23T04:44:53.502280",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n",
    "   Tokenizing The Data\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "e55a1fcf",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:44:53.555540Z",
     "iopub.status.busy": "2024-03-23T04:44:53.555212Z",
     "iopub.status.idle": "2024-03-23T04:44:53.761136Z",
     "shell.execute_reply": "2024-03-23T04:44:53.760117Z"
    },
    "papermill": {
     "duration": 0.226989,
     "end_time": "2024-03-23T04:44:53.763698",
     "exception": false,
     "start_time": "2024-03-23T04:44:53.536709",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/lib/python3.10/site-packages/transformers/tokenization_utils_base.py:2645: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "\n",
    "# tokenize and encode sequences \n",
    "tokens_train = tokenizer.batch_encode_plus(\n",
    "    train_text.tolist(),\n",
    "    max_length = pad_len,\n",
    "    pad_to_max_length=True,\n",
    "    truncation=True\n",
    ")\n",
    "\n",
    "tokens_val = tokenizer.batch_encode_plus(\n",
    "    val_text.tolist(),\n",
    "    max_length = pad_len,\n",
    "    pad_to_max_length=True,\n",
    "    truncation=True\n",
    ")\n",
    "\n",
    "tokens_test = tokenizer.batch_encode_plus(\n",
    "    test_text.tolist(),\n",
    "    max_length = pad_len,\n",
    "    pad_to_max_length=True,\n",
    "    truncation=True\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "3fe1c024",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:44:53.799446Z",
     "iopub.status.busy": "2024-03-23T04:44:53.799134Z",
     "iopub.status.idle": "2024-03-23T04:44:53.880021Z",
     "shell.execute_reply": "2024-03-23T04:44:53.879024Z"
    },
    "papermill": {
     "duration": 0.101138,
     "end_time": "2024-03-23T04:44:53.882199",
     "exception": false,
     "start_time": "2024-03-23T04:44:53.781061",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "train_seq = torch.tensor(tokens_train['input_ids'])\n",
    "train_mask = torch.tensor(tokens_train['attention_mask'])\n",
    "train_y = torch.tensor(train_labels.tolist())\n",
    "\n",
    "val_seq = torch.tensor(tokens_val['input_ids'])\n",
    "val_mask = torch.tensor(tokens_val['attention_mask'])\n",
    "val_y = torch.tensor(val_labels.tolist())\n",
    "\n",
    "test_seq = torch.tensor(tokens_test['input_ids'])\n",
    "test_mask = torch.tensor(tokens_test['attention_mask'])\n",
    "test_y = torch.tensor(test_labels.tolist())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "361675ab",
   "metadata": {
    "papermill": {
     "duration": 0.016719,
     "end_time": "2024-03-23T04:44:53.916052",
     "exception": false,
     "start_time": "2024-03-23T04:44:53.899333",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n",
    "   Defining Some Parameters\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "5dfb3c08",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:44:53.950186Z",
     "iopub.status.busy": "2024-03-23T04:44:53.949852Z",
     "iopub.status.idle": "2024-03-23T04:44:53.957506Z",
     "shell.execute_reply": "2024-03-23T04:44:53.956674Z"
    },
    "papermill": {
     "duration": 0.026917,
     "end_time": "2024-03-23T04:44:53.959377",
     "exception": false,
     "start_time": "2024-03-23T04:44:53.932460",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "#define a batch size\n",
    "batch_size = 64\n",
    "\n",
    "# wrap tensors\n",
    "train_data = TensorDataset(train_seq, train_mask, train_y)\n",
    "\n",
    "# sampler for sampling the data during training\n",
    "train_sampler = RandomSampler(train_data)\n",
    "\n",
    "# dataLoader for train set\n",
    "train_dataloader = DataLoader(train_data, sampler=train_sampler, batch_size=batch_size)\n",
    "\n",
    "# wrap tensors\n",
    "val_data = TensorDataset(val_seq, val_mask, val_y)\n",
    "\n",
    "# sampler for sampling the data during training\n",
    "val_sampler = SequentialSampler(val_data)\n",
    "\n",
    "# dataLoader for validation set\n",
    "val_dataloader = DataLoader(val_data, sampler = val_sampler, batch_size=batch_size)\n",
    "     \n",
    "\n",
    "# freeze the BERT architecture\n",
    "\n",
    "for param in bert.parameters():\n",
    "    param.requires_grad = False"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb696f3d",
   "metadata": {
    "papermill": {
     "duration": 0.016051,
     "end_time": "2024-03-23T04:44:53.992043",
     "exception": false,
     "start_time": "2024-03-23T04:44:53.975992",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n",
    "   BERT Architecture\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce893143",
   "metadata": {
    "papermill": {
     "duration": 0.01555,
     "end_time": "2024-03-23T04:44:54.023219",
     "exception": false,
     "start_time": "2024-03-23T04:44:54.007669",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "![](https://i0.wp.com/neptune.ai/wp-content/uploads/2022/10/Attention_diagram_transformer.png?ssl=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "96a924de",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:44:54.056616Z",
     "iopub.status.busy": "2024-03-23T04:44:54.056308Z",
     "iopub.status.idle": "2024-03-23T04:44:54.064224Z",
     "shell.execute_reply": "2024-03-23T04:44:54.063388Z"
    },
    "papermill": {
     "duration": 0.026857,
     "end_time": "2024-03-23T04:44:54.066133",
     "exception": false,
     "start_time": "2024-03-23T04:44:54.039276",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "class BERT_architecture(nn.Module):\n",
    "\n",
    "    def __init__(self, bert):\n",
    "      \n",
    "      super(BERT_architecture, self).__init__()\n",
    "\n",
    "      self.bert = bert \n",
    "      \n",
    "      # dropout layer\n",
    "      self.dropout = nn.Dropout(0.2)\n",
    "      \n",
    "      # relu activation function\n",
    "      self.relu =  nn.ReLU()\n",
    "\n",
    "      # dense layer 1\n",
    "      self.fc1 = nn.Linear(768,512)\n",
    "      \n",
    "      # dense layer 2 (Output layer)\n",
    "      self.fc2 = nn.Linear(512,2)\n",
    "\n",
    "      #softmax activation function\n",
    "      self.softmax = nn.LogSoftmax(dim=1)\n",
    "\n",
    "    #define the forward pass\n",
    "    def forward(self, sent_id, mask):\n",
    "\n",
    "      #pass the inputs to the model  \n",
    "      _, cls_hs = self.bert(sent_id, attention_mask=mask, return_dict=False)\n",
    "      \n",
    "      x = self.fc1(cls_hs)\n",
    "\n",
    "      x = self.relu(x)\n",
    "\n",
    "      x = self.dropout(x)\n",
    "\n",
    "      # output layer\n",
    "      x = self.fc2(x)\n",
    "      \n",
    "      # apply softmax activation\n",
    "      x = self.softmax(x)\n",
    "\n",
    "      return x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "5113deb5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:44:54.099304Z",
     "iopub.status.busy": "2024-03-23T04:44:54.098963Z",
     "iopub.status.idle": "2024-03-23T04:44:54.503091Z",
     "shell.execute_reply": "2024-03-23T04:44:54.502259Z"
    },
    "papermill": {
     "duration": 0.424009,
     "end_time": "2024-03-23T04:44:54.505464",
     "exception": false,
     "start_time": "2024-03-23T04:44:54.081455",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# pass the pre-trained BERT to our define architecture\n",
    "model = BERT_architecture(bert)\n",
    "\n",
    "# push the model to GPU\n",
    "model = model.to(device)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "3d3190e4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:44:54.541278Z",
     "iopub.status.busy": "2024-03-23T04:44:54.540905Z",
     "iopub.status.idle": "2024-03-23T04:44:54.550940Z",
     "shell.execute_reply": "2024-03-23T04:44:54.549835Z"
    },
    "papermill": {
     "duration": 0.030742,
     "end_time": "2024-03-23T04:44:54.553154",
     "exception": false,
     "start_time": "2024-03-23T04:44:54.522412",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/lib/python3.10/site-packages/transformers/optimization.py:429: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "# define the optimizer\n",
    "optimizer = AdamW(model.parameters(),lr = 1e-5)  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "b76778aa",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:44:54.588677Z",
     "iopub.status.busy": "2024-03-23T04:44:54.588375Z",
     "iopub.status.idle": "2024-03-23T04:44:54.597021Z",
     "shell.execute_reply": "2024-03-23T04:44:54.595923Z"
    },
    "papermill": {
     "duration": 0.029194,
     "end_time": "2024-03-23T04:44:54.599216",
     "exception": false,
     "start_time": "2024-03-23T04:44:54.570022",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "class weights are [1.14985507 0.88470116] for [0 1]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.utils.class_weight import compute_class_weight\n",
    "#compute the class weights\n",
    "class_weights = compute_class_weight(class_weight = \"balanced\",\n",
    "                                        classes = np.unique(train_labels),\n",
    "                                        y = train_labels \n",
    "                                     )\n",
    "print(\"class weights are {} for {}\".format(class_weights,np.unique(train_labels)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "76ebb25a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:44:54.635194Z",
     "iopub.status.busy": "2024-03-23T04:44:54.634659Z",
     "iopub.status.idle": "2024-03-23T04:44:54.652058Z",
     "shell.execute_reply": "2024-03-23T04:44:54.651137Z"
    },
    "papermill": {
     "duration": 0.036864,
     "end_time": "2024-03-23T04:44:54.654107",
     "exception": false,
     "start_time": "2024-03-23T04:44:54.617243",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/tmp/ipykernel_26/26686347.py:2: FutureWarning: pandas.value_counts is deprecated and will be removed in a future version. Use pd.Series(obj).value_counts() instead.\n",
      "  pd.value_counts(train_labels)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "label\n",
       "1    2242\n",
       "0    1725\n",
       "Name: count, dtype: int64"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#count of both the categories of training labels\n",
    "pd.value_counts(train_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "4e17cfda",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:44:54.689935Z",
     "iopub.status.busy": "2024-03-23T04:44:54.689613Z",
     "iopub.status.idle": "2024-03-23T04:44:54.697700Z",
     "shell.execute_reply": "2024-03-23T04:44:54.697007Z"
    },
    "papermill": {
     "duration": 0.027601,
     "end_time": "2024-03-23T04:44:54.699689",
     "exception": false,
     "start_time": "2024-03-23T04:44:54.672088",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "#wrap class weights in tensor\n",
    "weights= torch.tensor(class_weights,dtype=torch.float)\n",
    "\n",
    "# push weights to GPU\n",
    "weights = weights.to(device)\n",
    "\n",
    "# define loss function\n",
    "# add weights to handle the \"imbalance\" in the dataset\n",
    "cross_entropy  = nn.NLLLoss(weight=weights) \n",
    "\n",
    "# number of training epochs\n",
    "epochs = 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "030404d2",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:44:54.738880Z",
     "iopub.status.busy": "2024-03-23T04:44:54.738575Z",
     "iopub.status.idle": "2024-03-23T04:44:54.748724Z",
     "shell.execute_reply": "2024-03-23T04:44:54.747923Z"
    },
    "papermill": {
     "duration": 0.033555,
     "end_time": "2024-03-23T04:44:54.750801",
     "exception": false,
     "start_time": "2024-03-23T04:44:54.717246",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# function to train the model\n",
    "def train():\n",
    "  \n",
    "  model.train()\n",
    "\n",
    "  total_loss, total_accuracy = 0, 0\n",
    "  \n",
    "  # empty list to save model predictions\n",
    "  total_preds=[]\n",
    "  \n",
    "  # iterate over batches\n",
    "  for step,batch in enumerate(train_dataloader):\n",
    "    \n",
    "    # progress update after every 50 batches.\n",
    "    if step % 50 == 0 and not step == 0:\n",
    "      print('  Batch {:>5,}  of  {:>5,}.'.format(step, len(train_dataloader)))\n",
    "\n",
    "    # push the batch to gpu\n",
    "    batch = [r.to(device) for r in batch]\n",
    " \n",
    "    sent_id, mask, labels = batch\n",
    "\n",
    "    # clear previously calculated gradients \n",
    "    model.zero_grad()        \n",
    "\n",
    "    # get model predictions for the current batch\n",
    "    preds = model(sent_id, mask)\n",
    "\n",
    "    # compute the loss between actual and predicted values\n",
    "    loss = cross_entropy(preds, labels)\n",
    "\n",
    "    # add on to the total loss\n",
    "    total_loss = total_loss + loss.item()\n",
    "\n",
    "    # backward pass to calculate the gradients\n",
    "    loss.backward()\n",
    "\n",
    "    # clip the the gradients to 1.0. It helps in preventing the exploding gradient problem\n",
    "    torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)\n",
    "\n",
    "    # update parameters\n",
    "    optimizer.step()\n",
    "\n",
    "    # model predictions are stored on GPU. So, push it to CPU\n",
    "    preds=preds.detach().cpu().numpy()\n",
    "\n",
    "    # append the model predictions\n",
    "    total_preds.append(preds)\n",
    "\n",
    "  # compute the training loss of the epoch\n",
    "  avg_loss = total_loss / len(train_dataloader)\n",
    "  \n",
    "  # predictions are in the form of (no. of batches, size of batch, no. of classes).\n",
    "  # reshape the predictions in form of (number of samples, no. of classes)\n",
    "  total_preds  = np.concatenate(total_preds, axis=0)\n",
    "\n",
    "  #returns the loss and predictions\n",
    "  return avg_loss, total_preds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "6cdaa806",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:44:54.783543Z",
     "iopub.status.busy": "2024-03-23T04:44:54.783284Z",
     "iopub.status.idle": "2024-03-23T04:44:54.791280Z",
     "shell.execute_reply": "2024-03-23T04:44:54.790440Z"
    },
    "papermill": {
     "duration": 0.026452,
     "end_time": "2024-03-23T04:44:54.793303",
     "exception": false,
     "start_time": "2024-03-23T04:44:54.766851",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# function for evaluating the model\n",
    "def evaluate():\n",
    "  \n",
    "  print(\"\\nEvaluating...\")\n",
    "  \n",
    "  # deactivate dropout layers\n",
    "  model.eval()\n",
    "\n",
    "  total_loss, total_accuracy = 0, 0\n",
    "  \n",
    "  # empty list to save the model predictions\n",
    "  total_preds = []\n",
    "\n",
    "  # iterate over batches\n",
    "  for step,batch in enumerate(val_dataloader):\n",
    "    \n",
    "    # Progress update every 50 batches.\n",
    "    if step % 50 == 0 and not step == 0:\n",
    "      \n",
    "      # # Calculate elapsed time in minutes.\n",
    "      # elapsed = format_time(time.time() - t0)\n",
    "            \n",
    "      # Report progress.\n",
    "      print('  Batch {:>5,}  of  {:>5,}.'.format(step, len(val_dataloader)))\n",
    "\n",
    "    # push the batch to gpu\n",
    "    batch = [t.to(device) for t in batch]\n",
    "\n",
    "    sent_id, mask, labels = batch\n",
    "\n",
    "    # deactivate autograd\n",
    "    with torch.no_grad():\n",
    "      \n",
    "      # model predictions\n",
    "      preds = model(sent_id, mask)\n",
    "\n",
    "      # compute the validation loss between actual and predicted values\n",
    "      loss = cross_entropy(preds,labels)\n",
    "\n",
    "      total_loss = total_loss + loss.item()\n",
    "\n",
    "      preds = preds.detach().cpu().numpy()\n",
    "\n",
    "      total_preds.append(preds)\n",
    "\n",
    "  # compute the validation loss of the epoch\n",
    "  avg_loss = total_loss / len(val_dataloader) \n",
    "\n",
    "  # reshape the predictions in form of (number of samples, no. of classes)\n",
    "  total_preds  = np.concatenate(total_preds, axis=0)\n",
    "\n",
    "  return avg_loss, total_preds\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "412da82b",
   "metadata": {
    "papermill": {
     "duration": 0.017605,
     "end_time": "2024-03-23T04:44:54.826386",
     "exception": false,
     "start_time": "2024-03-23T04:44:54.808781",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n",
    "   Training Of BERT Model\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "636edacc",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:44:54.864654Z",
     "iopub.status.busy": "2024-03-23T04:44:54.864344Z",
     "iopub.status.idle": "2024-03-23T04:54:52.787108Z",
     "shell.execute_reply": "2024-03-23T04:54:52.786109Z"
    },
    "papermill": {
     "duration": 597.945015,
     "end_time": "2024-03-23T04:54:52.790816",
     "exception": false,
     "start_time": "2024-03-23T04:44:54.845801",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " Epoch 1 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.6942982173735096\n",
      "Validation Loss: 0.6865829910550799\n",
      "\n",
      " Epoch 2 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.6857223347310097\n",
      "Validation Loss: 0.6788262213979449\n",
      "\n",
      " Epoch 3 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.6839109784172427\n",
      "Validation Loss: 0.6731255437646594\n",
      "\n",
      " Epoch 4 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.6791575003054834\n",
      "Validation Loss: 0.6689361844744001\n",
      "\n",
      " Epoch 5 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.6742743522890152\n",
      "Validation Loss: 0.6644086156572614\n",
      "\n",
      " Epoch 6 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.6717363711326353\n",
      "Validation Loss: 0.6603528431483677\n",
      "\n",
      " Epoch 7 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.6698466875860768\n",
      "Validation Loss: 0.6578725874423981\n",
      "\n",
      " Epoch 8 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.6660159211004933\n",
      "Validation Loss: 0.6527729417596545\n",
      "\n",
      " Epoch 9 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.662779361970963\n",
      "Validation Loss: 0.6486780898911613\n",
      "\n",
      " Epoch 10 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.6598197404415377\n",
      "Validation Loss: 0.6446010300091335\n",
      "\n",
      " Epoch 11 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.6548963583284809\n",
      "Validation Loss: 0.6426378616264888\n",
      "\n",
      " Epoch 12 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.6516557480058363\n",
      "Validation Loss: 0.6366613166672843\n",
      "\n",
      " Epoch 13 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.6474175520481602\n",
      "Validation Loss: 0.633568001644952\n",
      "\n",
      " Epoch 14 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.6489210984399242\n",
      "Validation Loss: 0.6289856604167393\n",
      "\n",
      " Epoch 15 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.6435848051501859\n",
      "Validation Loss: 0.6250918124403272\n",
      "\n",
      " Epoch 16 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.641099565452145\n",
      "Validation Loss: 0.6216732817036765\n",
      "\n",
      " Epoch 17 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.635964923327969\n",
      "Validation Loss: 0.616686395236424\n",
      "\n",
      " Epoch 18 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.6310500358381579\n",
      "Validation Loss: 0.6134493436132159\n",
      "\n",
      " Epoch 19 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.627935225925138\n",
      "Validation Loss: 0.6078410957540784\n",
      "\n",
      " Epoch 20 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.6243196177867151\n",
      "Validation Loss: 0.6037045887538365\n",
      "\n",
      " Epoch 21 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.6192135512828827\n",
      "Validation Loss: 0.6025086130414691\n",
      "\n",
      " Epoch 22 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.6196879984871033\n",
      "Validation Loss: 0.595528381211417\n",
      "\n",
      " Epoch 23 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.6113341931373842\n",
      "Validation Loss: 0.589654837335859\n",
      "\n",
      " Epoch 24 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.6110077952184985\n",
      "Validation Loss: 0.5852612980774471\n",
      "\n",
      " Epoch 25 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.6064522641320382\n",
      "Validation Loss: 0.5814665683678218\n",
      "\n",
      " Epoch 26 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.6012718658293447\n",
      "Validation Loss: 0.5761027208396367\n",
      "\n",
      " Epoch 27 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.5960486194779796\n",
      "Validation Loss: 0.5714708396366665\n",
      "\n",
      " Epoch 28 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.5962948731837734\n",
      "Validation Loss: 0.5677633881568909\n",
      "\n",
      " Epoch 29 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.5861872694184703\n",
      "Validation Loss: 0.5624826507908958\n",
      "\n",
      " Epoch 30 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.5834341712536351\n",
      "Validation Loss: 0.5566096092973437\n",
      "\n",
      " Epoch 31 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.5787830045146327\n",
      "Validation Loss: 0.5531828573771885\n",
      "\n",
      " Epoch 32 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.5781004794182316\n",
      "Validation Loss: 0.5474902093410492\n",
      "\n",
      " Epoch 33 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.5722199486147973\n",
      "Validation Loss: 0.5444174706935883\n",
      "\n",
      " Epoch 34 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.5646072097363011\n",
      "Validation Loss: 0.5367657797677177\n",
      "\n",
      " Epoch 35 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.5635563365874752\n",
      "Validation Loss: 0.5326461302382606\n",
      "\n",
      " Epoch 36 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.5591898185591544\n",
      "Validation Loss: 0.5301684673343386\n",
      "\n",
      " Epoch 37 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.5534902282299534\n",
      "Validation Loss: 0.5222673373562949\n",
      "\n",
      " Epoch 38 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.5517468202498651\n",
      "Validation Loss: 0.5184752196073532\n",
      "\n",
      " Epoch 39 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.5455134477346174\n",
      "Validation Loss: 0.5138968697616032\n",
      "\n",
      " Epoch 40 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.5458887777020854\n",
      "Validation Loss: 0.5109868198633194\n",
      "\n",
      " Epoch 41 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.5366043179265915\n",
      "Validation Loss: 0.5038247832230159\n",
      "\n",
      " Epoch 42 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.5336355634274021\n",
      "Validation Loss: 0.4973089588539941\n",
      "\n",
      " Epoch 43 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.5307721538889792\n",
      "Validation Loss: 0.49432359848703655\n",
      "\n",
      " Epoch 44 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.5200251518718658\n",
      "Validation Loss: 0.488547499690737\n",
      "\n",
      " Epoch 45 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.5159870539942095\n",
      "Validation Loss: 0.482387512922287\n",
      "\n",
      " Epoch 46 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.5162637084722519\n",
      "Validation Loss: 0.4780220091342926\n",
      "\n",
      " Epoch 47 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.5074128248999196\n",
      "Validation Loss: 0.47242243800844463\n",
      "\n",
      " Epoch 48 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.5037246836769965\n",
      "Validation Loss: 0.46705089935234617\n",
      "\n",
      " Epoch 49 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.5028968491861897\n",
      "Validation Loss: 0.4644003361463547\n",
      "\n",
      " Epoch 50 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.4969498904482011\n",
      "Validation Loss: 0.460189442549433\n",
      "\n",
      " Epoch 51 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.49361482358747916\n",
      "Validation Loss: 0.45419577615601675\n",
      "\n",
      " Epoch 52 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.4933097828780451\n",
      "Validation Loss: 0.45021570154598783\n",
      "\n",
      " Epoch 53 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.4850744850212528\n",
      "Validation Loss: 0.44736901777131216\n",
      "\n",
      " Epoch 54 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.4804393144384507\n",
      "Validation Loss: 0.44093984152589527\n",
      "\n",
      " Epoch 55 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.47408184745619375\n",
      "Validation Loss: 0.4355577124016626\n",
      "\n",
      " Epoch 56 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.4719146629495005\n",
      "Validation Loss: 0.4307883232831955\n",
      "\n",
      " Epoch 57 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.4702314128798823\n",
      "Validation Loss: 0.4296093646969114\n",
      "\n",
      " Epoch 58 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.4633274121630576\n",
      "Validation Loss: 0.4219406843185425\n",
      "\n",
      " Epoch 59 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.46150586009025574\n",
      "Validation Loss: 0.4179697803088597\n",
      "\n",
      " Epoch 60 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.4548664597734328\n",
      "Validation Loss: 0.4134607655661447\n",
      "\n",
      " Epoch 61 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.45141335168192465\n",
      "Validation Loss: 0.41026296998773304\n",
      "\n",
      " Epoch 62 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.45208284643388563\n",
      "Validation Loss: 0.4053444947515215\n",
      "\n",
      " Epoch 63 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.4443150975050465\n",
      "Validation Loss: 0.40330822127205984\n",
      "\n",
      " Epoch 64 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.4404743289755237\n",
      "Validation Loss: 0.3972064086369106\n",
      "\n",
      " Epoch 65 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.4377694620240119\n",
      "Validation Loss: 0.3942490837403706\n",
      "\n",
      " Epoch 66 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.43546064774836263\n",
      "Validation Loss: 0.3889520743063518\n",
      "\n",
      " Epoch 67 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.4282716853964713\n",
      "Validation Loss: 0.3850984807525362\n",
      "\n",
      " Epoch 68 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.4243525752136784\n",
      "Validation Loss: 0.37956422354493824\n",
      "\n",
      " Epoch 69 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.425806519485289\n",
      "Validation Loss: 0.3765595725604466\n",
      "\n",
      " Epoch 70 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.4168210601614368\n",
      "Validation Loss: 0.3725129153047289\n",
      "\n",
      " Epoch 71 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.4158971477900782\n",
      "Validation Loss: 0.37134018753256115\n",
      "\n",
      " Epoch 72 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.4122866055657787\n",
      "Validation Loss: 0.3660777360200882\n",
      "\n",
      " Epoch 73 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.4060992622567761\n",
      "Validation Loss: 0.36212129039423807\n",
      "\n",
      " Epoch 74 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.40188192792477145\n",
      "Validation Loss: 0.3587786321129118\n",
      "\n",
      " Epoch 75 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.40260793557090147\n",
      "Validation Loss: 0.3556363454886845\n",
      "\n",
      " Epoch 76 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.40322241715846524\n",
      "Validation Loss: 0.3514011353254318\n",
      "\n",
      " Epoch 77 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.39615158880910567\n",
      "Validation Loss: 0.34940495448453085\n",
      "\n",
      " Epoch 78 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.3928338375783736\n",
      "Validation Loss: 0.34509769082069397\n",
      "\n",
      " Epoch 79 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.39304802735005656\n",
      "Validation Loss: 0.3423802724906376\n",
      "\n",
      " Epoch 80 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.38572738151396474\n",
      "Validation Loss: 0.339596546122006\n",
      "\n",
      " Epoch 81 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.3833875261968182\n",
      "Validation Loss: 0.3369562817471368\n",
      "\n",
      " Epoch 82 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.38097719030995525\n",
      "Validation Loss: 0.3317586375134332\n",
      "\n",
      " Epoch 83 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.3797598884951684\n",
      "Validation Loss: 0.32942646954740795\n",
      "\n",
      " Epoch 84 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.381297912328474\n",
      "Validation Loss: 0.3271199124200003\n",
      "\n",
      " Epoch 85 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.3726073316028041\n",
      "Validation Loss: 0.3230288880211966\n",
      "\n",
      " Epoch 86 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.3733979069417523\n",
      "Validation Loss: 0.3208583210195814\n",
      "\n",
      " Epoch 87 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.3691604594069143\n",
      "Validation Loss: 0.3179579994508198\n",
      "\n",
      " Epoch 88 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.3715422763939827\n",
      "Validation Loss: 0.31619096228054594\n",
      "\n",
      " Epoch 89 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.36070742386002697\n",
      "Validation Loss: 0.3131261680807386\n",
      "\n",
      " Epoch 90 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.359139395336951\n",
      "Validation Loss: 0.31163873842784334\n",
      "\n",
      " Epoch 91 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.3606533706188202\n",
      "Validation Loss: 0.3066916401897158\n",
      "\n",
      " Epoch 92 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.3523764927541056\n",
      "Validation Loss: 0.3042978910463197\n",
      "\n",
      " Epoch 93 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.3561380861266967\n",
      "Validation Loss: 0.30311791598796844\n",
      "\n",
      " Epoch 94 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.35511694608196137\n",
      "Validation Loss: 0.29880089525665554\n",
      "\n",
      " Epoch 95 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.35072464231521855\n",
      "Validation Loss: 0.29632898420095444\n",
      "\n",
      " Epoch 96 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.3504011486807177\n",
      "Validation Loss: 0.2931276953646115\n",
      "\n",
      " Epoch 97 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.34254682304397704\n",
      "Validation Loss: 0.2957861359630312\n",
      "\n",
      " Epoch 98 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.3407739989699856\n",
      "Validation Loss: 0.2896047532558441\n",
      "\n",
      " Epoch 99 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.3403243350405847\n",
      "Validation Loss: 0.28774319908448626\n",
      "\n",
      " Epoch 100 / 100\n",
      "  Batch    50  of     62.\n",
      "\n",
      "Evaluating...\n",
      "\n",
      "Training Loss: 0.33939016105667236\n",
      "Validation Loss: 0.2847051737563951\n"
     ]
    }
   ],
   "source": [
    "# set initial loss to infinite\n",
    "best_valid_loss = float('inf')\n",
    "\n",
    "# empty lists to store training and validation loss of each epoch\n",
    "train_losses=[]\n",
    "valid_losses=[]\n",
    "\n",
    "#for each epoch\n",
    "for epoch in range(epochs):\n",
    "     \n",
    "    print('\\n Epoch {:} / {:}'.format(epoch + 1, epochs))\n",
    "    \n",
    "    #train model\n",
    "    train_loss, _ = train()\n",
    "    \n",
    "    #evaluate model\n",
    "    valid_loss, _ = evaluate()\n",
    "    \n",
    "    #save the best model\n",
    "    if valid_loss < best_valid_loss:\n",
    "        best_valid_loss = valid_loss\n",
    "        torch.save(model.state_dict(), 'saved_weights.pt')\n",
    "    \n",
    "    # append training and validation loss\n",
    "    train_losses.append(train_loss)\n",
    "    valid_losses.append(valid_loss)\n",
    "    \n",
    "    print('\\nTraining Loss: {}'.format(train_loss))\n",
    "    print('Validation Loss: {}'.format(valid_loss))\n",
    "     "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b9451ae",
   "metadata": {
    "papermill": {
     "duration": 0.040172,
     "end_time": "2024-03-23T04:54:52.871304",
     "exception": false,
     "start_time": "2024-03-23T04:54:52.831132",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n",
    "   Training Graph\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "0241ea2f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:54:52.953762Z",
     "iopub.status.busy": "2024-03-23T04:54:52.953133Z",
     "iopub.status.idle": "2024-03-23T04:54:53.222924Z",
     "shell.execute_reply": "2024-03-23T04:54:53.222010Z"
    },
    "papermill": {
     "duration": 0.312774,
     "end_time": "2024-03-23T04:54:53.224920",
     "exception": false,
     "start_time": "2024-03-23T04:54:52.912146",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGdCAYAAADAAnMpAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/xnp5ZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABt7ElEQVR4nO3ddXyVdf/H8ddZM8YYORoEQUZ3g0qndDcIUgISKioISImUIkhIdyMtY3TnaBCkuxk1Nrbr98f1Y8oNyOLsnMX7+XicB2fXueJzvuyW931d37AYhmEgIiIiYicO9i5ARERE4jeFEREREbErhRERERGxK4URERERsSuFEREREbErhRERERGxK4URERERsSuFEREREbErJ3sXEB6hoaFcu3aNRIkSYbFY7F2OiIiIhINhGDx69Ig0adLg4PD2+x+xIoxcu3aN9OnT27sMERERiYTLly+TLl26t34eK8JIokSJAPPLeHp6Wu28wcHBrF+/nooVK+Ls7Gy188rr1Na2o7a2LbW37aitbcdabR0QEED69OnD/h1/m1gRRl4+mvH09LR6GHF3d8fT01O/2NFMbW07amvbUnvbjtradqzd1u/qYqEOrCIiImJXCiMiIiJiVwojIiIiYlexos+IiIjELSEhIQQHB0fomODgYJycnAgMDCQkJCSaKhMIf1s7Ojri5OQU5Wk3FEZERMSmHj9+zJUrVzAMI0LHGYZBqlSpuHz5suacimYRaWt3d3dSp06Ni4tLpK8XqTAybtw4fvrpJ27cuEHevHkZO3YsRYoUeeO+H330EVu2bHlte9WqVVm9enVkLi8iIrFUSEgIV65cwd3dnRQpUkQoVISGhvL48WM8PDz+cwItibrwtLVhGAQFBXH79m3Onz9P1qxZI/33EuEwsmDBAnr06MGECRMoWrQoY8aMoVKlSpw+fZqUKVO+tv/SpUsJCgoK+/nu3bvkzZuX+vXrR6pgERGJvYKDgzEMgxQpUpAgQYIIHRsaGkpQUBBubm4KI9EsvG2dIEECnJ2duXjxYtj+kRHhv81Ro0bRrl07WrduTY4cOZgwYQLu7u5MnTr1jfsnTZqUVKlShb18fX1xd3dXGBERicf0mCXusEYwjNCdkaCgIA4cOECfPn1eKaJ8+fLs2rUrXOeYMmUKjRo1ImHChG/d5/nz5zx//jzs54CAAMBM1BHt8PRfXp7LmueUN1Nb247a2rbU3hHz8s5IaGgooaGhETr2ZR+Tl8dL9IlIW4eGhmIYBsHBwTg6Or7yWXj/dxGhMHLnzh1CQkLw9vZ+Zbu3tzenTp165/F79+7l2LFjTJky5T/3Gzp0KAMGDHht+/r163F3d49IyeHi6+tr9XPKm6mtbUdtbVtq7/BxcnIiVapUPH78+JVH+BHx6NEjK1clbxOetg4KCuLZs2ds3bqVFy9evPLZ06dPw3Udm46mmTJlCrlz535rZ9eX+vTpQ48ePcJ+fjm3fcWKFa0+Hbyvry8VKlTQ1MLRTG1tO2pr21J7R0xgYCCXL1/Gw8Mjwv0LXq4AG9tXcM+cOTPdunWjW7duUT7X5s2bKVeuHHfv3sXLyyvqxf2/iLR1YGAgCRIkoEyZMq/9nb58svEuEQojyZMnx9HRkZs3b76y/ebNm6RKleo/j33y5Anz589n4MCB77yOq6srrq6ur213dnaOlv+xR9d55XVqa9tRW9uW2jt8QkJCsFgsODg4RLivwcvHBS+Pt6WPPvqIfPnyMWbMmCifa9++fSRMmNAq3+HlOSLTnv8lIm3t4OCAxWJ54/8Gwvu/iQhV7uLiQsGCBfHz83ulYD8/P4oXL/6fxy5atIjnz5/TrFmziFwyWll8fSk2YAAEBtq7FBERicUMw3jtEcXbpEiRIlq6HMRmEY5RPXr0YPLkycyYMYOTJ0/SsWNHnjx5QuvWrQFo0aLFKx1cX5oyZQq1atUiWbJkUa/aGp49w/HTT/E+dAiHwYPtXY2ISPxkGPDkiX1e4Zx0rVWrVmzZsoWff/4Zi8WCxWJh+vTpWCwW1q5dS8GCBXF1dWX79u38/fff1KxZE29vbzw8PChcuDAbNmx45XyZMmV65Q6LxWLh999/p3bt2ri7u5M1a1ZWrFgR6SZdsmQJOXPmxNXVlUyZMjFy5MhXPh8/fjxZs2bFzc0Nb29v6tWrF/bZ4sWLyZ07NwkTJiRz5sxUrFiRJ0+eRLqW8Ipwn5GGDRty+/Zt+vXrx40bN8iXLx/r1q0L69R66dKl127pnD59mu3bt7N+/XrrVG0NCRIQ8ssvONWvj8OIEdCoEeTLZ++qRETil6dPwcMjXLs6AF7WvPbjx/AfIztf+vnnn/nrr7/IlStXWFeD48ePA/D1118zYsQIMmfOTJIkSbh8+TJVq1Zl8ODBuLq6MnPmTGrUqMHp06fJkCHDW68xYMAAhg8fzk8//cTYsWNp2rQpFy9eJGnSpBH6SgcOHKBBgwb079+fhg0bsnPnTjp16kSyZMlo1aoV+/fvp2vXrsyaNYsSJUpw7949tm3bBsD169dp3Lgxw4cPp2bNmly/fh1/f/8Iz5QbKUYs8PDhQwMwHj58aNXzBgUFGVdKlDAMMIwCBQwjONiq55d/BAUFGcuXLzeCgoLsXUqcp7a2LbV3xDx79sw4ceKE8ezZM3PD48fmf4Pt8Xr8ONx1f/jhh0a3bt3Cft60aZMBGMuXL3/nsTlz5jTGjh0b9nPGjBmN0aNHh/0MGN99913Yz48fPzYAY+3ate8898s67t+/bxiGYTRp0sSoUKHCK/v07t3byJEjh2EYhrFkyRLD09PTCAgIeO1cBw4cMADjwoULRkhIiHH//n0jJCTknTW89nf6L+H99zveT2F3tH17jCRJ4OBBGDXK3uWIiMQv7u7mHYpwvEIDAnhw5QqhAQHhPuY/X1bot1GoUKFXfn78+DG9evXCx8cHLy8vPDw8OHnyJJcuXfrP8+TJkyfsfcKECfH09OTWrVsRrufkyZOULFnylW0lS5bkzJkzhISEUKFCBTJmzEjmzJlp3rw5c+bMCRt+mzdvXsqVK0fu3Llp0KABM2bM4P79+xGuITLifRh57uVFyE8/mT98/z389Zd9CxIRiU8sFvNRiT1eVhge/L8TePbq1Ytly5YxZMgQtm3bhr+/P7lz537nnCr/O+rEYrFEy8RuiRIl4uDBg8ybN4/UqVPTr18/8ubNy4MHD3B0dMTX15e1a9eSI0cOJk2ahI+PD+fPn7d6Hf8rXoeRh4EP8b3ri9G8OVSsaI6qadcONLOfiIj8i4uLCyEhIe/cb8eOHbRq1YratWuTO3duUqVKxYULF6K/wP/n4+PDjh07XqspW7ZsYbOjOjk5Ub58eYYPH86RI0e4cOECGzduBMwQVLJkSfr378/WrVtxcXFh2bJl0V63TSc9i0meBj+l6NSinHtwjpKnStJ44kTIlQu2boX+/aFHD7DiBDIiIhJ7ZcqUiT179nDhwgU8PDzeetcia9asLF26lBo1amCxWOjbt69Np67v2bMnhQsX5ocffqBhw4bs2rWLX3/9lfHjxwOwatUqzp07R5kyZUiSJAlr1qwhNDSUDz74gD179uDn50fFihVJnjw5mzdv5vbt2/j4+ER73fH2zoi7szv1cpjDmTqt7cSlJA4wdKj54Q8/QMqUULUqTJkCd+7YsVIREbG3Xr164ejoSI4cOUiRIsVb+4CMGjWKJEmSUKJECWrUqEGlSpUoUKCAzeosUKAACxcuZP78+eTKlYt+/foxcOBAWrVqBYCXlxdLly6lbNmy+Pj4MGHCBObNm0fOnDnx9PRk69atVK1alezZszN48GBGjBhBlSpVor1ui2HYYsxO1AQEBJA4cWIePnxo1engnwY+Jd/YfJx5eoYyGcuwsdkGHH/+BaZNg/8ftgWAiwuMGQMdO1rt2vFNcHAwa9asoWrVqpqlMpqprW1L7R0xgYGBnD9/nvfeey/C08GHhoYSEBCAp6enzWdgjW8i0tb/9Xca3n+/4/XfprOjMz0y9sDDxYOtF7fy466foGdPOHYMTp6EQYMgTx4ICoJOncxHN+F4ZigiIiLhF6/DCEBq19SMqTgGgH6b+rHnyh7zg+zZ4dtvwd8fXs7QOno01KljztwnIiISzTp06ICHh8cbXx06dLB3eVYTbzuw/lvz3M3xPe/LguMLaLK0Cf6f+ZPINZH5ocUC33wDWbJAy5awYgWUKQMrV0KaNPYtXERE4rSBAwfSq1evN35mzW4L9qYwgjmUaUL1Cey6sotz98/RflV75tSZg4PlXzeOGjaEDBmgZk1zgrSiRWHtWnMEjoiISDRImTIlKVOmtHcZ0S7eP6Z5ycvNi9m1Z+NocWT+sfl0Wt3p9fn4ixeH3bvNRzhXrkCpUrBpk30KFhERiSMURv6ldMbSzKw9EwsWJh6YSLd13V4PJJkzw44dULo0PHwIlSrBnDn2KVhERCQOUBj5H01yN2FqzakAjN07lt6+vV8PJEmTwvr10KABBAdDs2bmHCUxf5S0iIhIjKMw8gat8rViYvWJAIzcNZLvNn73eiBxc4N588yhwGB2ci1eHJYs0fBfERGRCFAYeYv2BdsztspYAIZsH8KnKz7lafDTV3dycIARI2DsWHB1hT17oF49s0/Jb7/Bs2d2qFxERCR2URj5D12KdGFMpTFYsDDVfyrFfi/G6Tun37BjF7h4Eb77DpIkgbNnzUnSUqeG1q1h3TrzcY6IiMRbmTJlYsyYMWE/WywWli9f/tb9L1y4gMViwd/f/53n3rx5MxaLhQcPHkS5TntQGHmHbsW6saHFBrwTenP01lEKTirI3KNzX9/R29tc0+byZfjlF8iUyezgOn06VKliBpPPPoMzZ2z9FUREJAa6fv26TdZ9iQ0URsKh7Htl8e/gz0eZPuJJ8BOaLm1Kx1UdCQoJen3nhAnh88/h779hyxZzPZsUKeDuXZg0CfLmhXHj1NlVRCSeS5UqFa6urvYuI0ZQGAmnVB6p2NB8A33L9MWChQkHJlBuZjluPbn15gMcHMyZWsePh2vXwNcXypY1+5F06WIOCb561bZfQkQkhjEMgydBT8L/Co7Avu94RWSd2EmTJpEmTRpCQ0Nf2V6zZk3atGnD33//Tc2aNfH29sbDw4PChQuzYcOG/zzn/z6m2bt3L/nz58fNzY1ChQpx6NChCLXl/1qyZAk5c+bE1dWVTJkyMXLkyFc+Hz9+PFmzZsXNzQ1vb2/q1asX9tnixYspUaIECRMmJFmyZJQvX54n0bgUimZgjQBHB0cGfjyQ4umK02hJI7Zf2k6hSYX4o9Ef5E+d/+0HOjlB+fJmGPn1V/jqKzOc5Mpl/tykiTntvIhIPPM0+CkeQz3scu3HfR6T0CVhuPatX78+n3/+OZs2baJcuXIA3Lt3j3Xr1rFmzRoeP35M1apVGTx4MK6ursycOZMaNWpw+vRpMmTI8O5aHj+mevXqVKhQgdmzZ3P+/Hm6desW6e924MABGjRoQP/+/WnYsCE7d+6kU6dOJEuWjFatWrF//366du3KrFmzKFGiBPfu3WPbtm2A+fioadOmDBgwgEaNGvHkyRO2bdsWofAWUbozEglVslZhz6d7yJYsG5cDLlNyakkWHFvw7gMdHKBrVzh0CAoVggcPzDlKypaFo0ejvW4REYmcJEmSUKVKFebO/afP4OLFi0mePDkff/wxefPm5bPPPiNXrlxkzZqVH374gSxZsrBixYpwnX/u3LmEhoYyZcoUcubMSfXq1endu3ek6x01ahTlypWjb9++ZMuWjVatWtGlSxd++uknAC5dukTChAmpXr06GTNmJH/+/HTt2hUww8iLFy+oXr06mTJlInfu3HTq1AkPj+gLjbozEknZk2dnz6d7aLykMevOrqPRkkbsvrKbYeWH4er0jmeA2bPDzp0wbBgMGQKbN0O+fGb/koEDzUnVRETiAXdndx73eRyufUNDQwl4FIBnIk8cHKL+/6Xdnd0jtH/Tpk1p164d48ePx9XVlTlz5tCoUSMcHBx4/Pgx/fv3Z/Xq1WH/mD979oxLly6F69wnT54kT548uLm5hW0rXrx4hOr73/PVrFnzlW0lS5ZkzJgxhISEUKFCBTJmzEjmzJmpXLkylStXpnbt2ri7u5M3b17KlStHqVKlqFixIpUqVaJevXokSZIk0vW8i+6MRIGXmxerGq+idwkzvY7ZM4YSU0tw5m44Rsw4O0PfvnDyJNStC6GhZsfWrFnN0TjPn0dz9SIi9mexWEjokjD8L+cI7PuOlyWCj8dr1KiBYRisXr2ay5cvs23bNpo2bQpAr169WLZsGUOGDGHbtm34+/uTO3dugoLeMNAhBkiUKBEHDx5k3rx5pE6dmn79+pE3b14ePHiAo6Mjf/75JwsXLiRHjhyMHTuWDz74gPPnz0dbPQojUeTo4MjwCsNZ0WgFyRIk4+D1gxSYVIBZh2eF7wSZMsHixeDnZ/YhuXcPunUz757Mnm2GFBERsTs3Nzfq1KnDnDlzmDdvHh988AEFChQAYMeOHbRq1YratWuTO3duUqVKxYULF8J9bh8fH44cOUJgYGDYtt27d0e6Vh8fH3bs2PHKth07dpAtWzYcHR0BcHJyonz58gwfPpwjR45w4cIFNm7cCJghsVixYvTv359Dhw7h4uLCsmXLIl3PuyiMWEmND2pwuMNhPsz4IY+DHtNieQsaLm7IsVvHwneCsmXNviQTJphzkly4AM2bQ/78sGZNtNYuIiLh07RpU1avXs3UqVPD7ooAZM2alaVLl+Lv78/hw4dp0qTJayNv/kuTJk2wWCy0a9eOEydOsGbNGkaMGBHpOnv27Imfnx8//PADf/31FzNmzODXX3+lV69eAKxatYpffvkFf39/Ll68yMyZMwkNDeWDDz5gz549DB06lEOHDnHp0iWWLl3K7du38fHxiXQ976IwYkVpPdPi18KPgR8NxMHiwMLjC8n9W24qz67M+r/Xv7snspOTOTHa2bPmwnuJE8ORI1CtGnz9te6SiIjYWdmyZUmaNCmnT5+mSZMmYdtHjRpFkiRJKFGiBDVq1KBSpUphd03Cw8PDg5UrV3L06FHy58/Pt99+y48//hjpOgsUKMDChQuZP38+uXLlol+/fgwcOJBWrVoB4OXlxdKlSylbtiw+Pj5MmDCBefPmkTNnTjw9Pdm6dSsNGjQge/bsfPfdd4wcOTJaJ2izGNE5VsdKAgICSJw4MQ8fPsTT09Nq5w0ODmbNmjVUrVoVZ2dnq50X4MC1AwzbMYylJ5cSapghImeKnIyrOo4PM30YvpPcu2fO6vpy+uCmTWHqVHBxsWqtthCdbS2vUlvblto7YgIDAzl//jzvvffeK501wyM0NJSAgAA8Pa3TgVXeLiJt/V9/p+H991t/m9GkYJqCLKq/iDOfn6Fb0W4kdE7I8dvHqTi7IktPLg3fSZImhdGjYdo0867JnDlQtao5zbyIiEgcoTASzTInycyYymO40uMKtbPXJigkiPqL6jP5wOTwn6RVK1i1Cjw8zI6uZcpo9lYRkXikQ4cOeHh4vPHVoUMHe5cXZZpnxEa83LxYVH8RHVZ14PdDv9N+VXvuPrvLVyW/Ct/wskqVzLVuqlUz+5HkymXOSdKxo3nXRERE4qyBAweGdT79X9bsvmAv+lfMhhwdHJlUYxIpEqZg6Pah9PHrw60nt/ipwk84Oji++wQFCsCuXVCnjjnypmtXc/G9n382R+OIiEiclDJlSlKmTGnvMqKNHtPYmMViYUi5IYysaC5YNHr3aMrPKs+1R9fCd4JMmWDfPvjtN0iWDI4dg3LloF49uHEj+goXEbGiWDB2QsLJGn+XCiN20qN4D+bVnYeHiwebL2wm74S8rD2zNnwHOzpChw7w11/mCsCOjrBkiTknydat0Vu4iEgUvJxwK6bOTCoR9/TpU4AojSbTYxo7apSrEQVSF6DR4kYcunGIqnOr0rN4T4aUG4KLYziG7yZNCmPHQrt25sq/x4+bj2uGDIFevcyF+V66dMn8vFy5WDk0WETiBicnJ9zd3bl9+zbOzs4RGqIbGhpKUFAQgYGBGtobzcLT1oZh8PTpU27duoWXl1dY0IwMhRE7y5YsG7va7qK3b2/G7h3LyF0j2XZpG/PqziNzkszhO0mePLBnj9mZddYs+Oor2L4d2rQxR9/4+sLp0+a+5cvD8uWQMHzLZouIWJPFYiF16tScP3+eixcvRuhYwzB49uwZCRIkiPC6MhIxEWlrLy8vUqVKFaXrKYzEAK5OrvxS5RfKvVeO1n+0Zu/VveSfmJ9J1SfRMFfD8J0kYUKYMQNKl4bPP4eVK83XSw4O5qibDRugcmVYvRriQA9sEYl9XFxcyJo1a4Qf1QQHB7N161bKlCmjCeaiWXjb2tnZOUp3RF5SGIlBamaviX9qf5osacKOyztotKQRfuf9GFN5TPiWurZYzEc2hQqZd0nu3TPvhFSoAB9/bK4QXKWKedekfHlYt8581CMiYmMODg4RnoHV0dGRFy9e4ObmpjASzWzd1nroFsNkSJyBza02813p77BgYfLByRSYWIBf9/7KrSe3wneS/Plh926zg+v48VC7Nnh5QfHisHGjOQpn3z6zf8nt29H6fURERN5FYSQGcnJw4oeyP7ChxQZSe6Tm9N3TfL72c9KMTEOVOVWYfWQ2T4OfRu7kBQrA5s3g7Q2HD0ORIuZKwU8jeT4REZEoUhiJwcq+V5ZjnY4xptIYCqcpTIgRwrqz62i+rDlFJhfh7tO7kTtxrlzmEOD06eHCBfORToYM0Lev5ioRERGbUxiJ4ZImSEq3Yt3Y224vf3X5i/4f9idlwpQcv32cqnOr8uj5o8idOFs2c6jvzz/De+/B3bswaBBkzAijRln3S4iIiPwHhZFYJGuyrHz/0fdsarmJZAmSsffqXmotqEXgi8DInTBRInNK+TNnYPFis09JUBD07GkOD9YMiSIiYgMKI7FQjhQ5WNt0LR4uHmw8v5EmS5rwIvRF5E/o6Ah168LOnfDjj+a24cOhfXsICbFO0SIiIm+hMBJLFU5bmD8a/YGLowvLTi2j/cr2hBqhUT/xl1/C5MnmvCS//w4NG8Lz51E/r4iIyFsojMRiZd8ry4J6C3CwODDNfxpFJhdhy4UtUT/xp5/CwoXmtPFLlkD16vAokn1TRERE3kFhJJarlb0Ws2vPxtPVkwPXD/DRjI+oOb8mp++cjtqJ69Y1Z2lNmNCctbVMGbh+3So1i4iI/JvCSBzQOHdjzn5+lk6FOuFocWTF6RXkHJ+TXut7ERwSHPkTly9vzkmSMiX4+0OxYuYsriIiIlakMBJHpEiYgnHVxnG041FqZKtBiBHCyF0jqTynMvee3Yv8iQsVgl27IGtWc+XfkiVh2zbrFS4iIvGewkgc45PChxWNV7C0wVISOidk4/mNFJlchJO3o3BHI3Nmc6RN8eJw/7651s2kSRppIyIiVqEwEkfV9qnNrra7yOSVib/v/02xKcVYc2ZN5E+YPLnZd6RWLXN0zWefQcGC4OdntZpFRCR+UhiJw3J752bvp3spnaE0Ac8DqD63Ot9v+j7y/Ujc3c3J0UaNMhfeO3zY7FdSowacOmXV2kVEJP5QGInjUiRMwYYWG2hXoB0GBgO3DqTUtFKcuXsmcid0dIQvvoCzZ83ZW52cYNUqyJ3bXNtGc5KIiEgEKYzEAy6OLkyqMYm5deaS2DUxe6/uJd/EfEw6MAkjslO+J0tmrmtz/Lh5Z+TFC3Ntm/z5zQ6vIiIi4aQwEo80zt2Yox2P8nGmj3ka/JTPVn1GjXk1uPTwUuRPmi0brFhhPr7x9jaH/pYsCd27w5MnVqtdRETiLoWReCZ94vRsaLGBERVG4OLowuozq/EZ58NPO36K2pwkdevCiRPQsqW5wN7PP5vzkly9ar3iRUQkTlIYiYccLA70LNGTg+0PUjpDaZ4GP+XLDV9SYFIBdlzaEfkTJ00K06fDunWQOjUcOwalSpmrAouIiLyFwkg8ljNlTra02sK0mtNIliAZx24do9S0Unzj903k+5IAVKpkzkvy/vtw4YIZSA4dslrdIiIStyiMxHMWi4VW+Vpxustp2uZvC8DQ7UNp9UerqD22yZQJduwwO7TeuoVThQokO3oU7t0zhwFv3Wouwqe7JiIi8Z7CiACQzD0Zv3/yO1M/mYqjxZGZh2fyyfxPeBz0OPInTZnSXNvmww+xBARQqm9fnFOlAh8f+PBDqFcPCheGK1es9j1ERCT2URiRV7TO35o/Gv1BAqcErDu7jrIzynL7ye3In9DTE9atI7Ru3X+2eXmZo3BSpYKHD83ZXKPyWEhERGI1hRF5TbVs1djUchPJEiRj37V9FJtSjPV/r4/8Cd3cCJk3jzWzZxP8+LG5vs3p07BxI7i6wpo1MHOm9b6AiIjEKgoj8kZF0xVlR5sdZPLKxLn756g0uxI159fk73t/R/qcwR4e4OLyzwYfHxgwwHzfrZuGAYuIxFMKI/JWHyT/gEOfHaJ70e44OTix4vQKcozPQZ8NfaLWl+TfevY0+43ocY2ISLylMCL/ycvNi9GVR3OkwxEqZqlIUEgQw3YM48PpH3L/2f2oX8DJCaZNM++YrF4Ns2ZF/ZwiIhKrKIxIuPik8GFd03X80egPUrin4OD1g1SYVYEHgQ+ifvKcOaF/f/N9t24wciT8/jssXAh//gk3bkT9GiIiEmMpjEi4WSwWPvngEza23Ehy9+QcuH6AirMq8jDwYdRP3rs3FCwIDx5Ar17Qrh00bAiVK0PmzLBgQdSvISIiMZLCiERYrpS52NhiY9hom0qzKxHwPCBqJ3VygpUrzVDSrBlUrw6lS5tB5NkzaNQIvvsOQkOt8yVERCTGUBiRSMntnRu/Fn4kTZCUPVf3UHl2Ze48vRO1k6ZODcOHm/1GVq40Z2n96y/zTgnA4MFQpw48ehT1LyAiIjGGwohEWt5UefFr4UcStyTsurKLfBPyse3iNutexNERfvrJnIfE1RX++ANKlDDXvBERkThBYUSiJF+qfGxtvZUPkn3A1UdX+WjGRwzeOphQw8qPU5o3hy1b/lkNuEQJOH7cutcQERG7UBiRKMuVMhf72++neZ7mhBqhfLfpOyrPrszZe2ejtvrv/ypaFPbtg1y54Pp1KFMG9u613vlFRMQuIhVGxo0bR6ZMmXBzc6No0aLsfcc/CA8ePKBz586kTp0aV1dXsmXLxpo1ayJVsMRMHi4ezKw9k2k1p5HAKQG+53zJOjYrqUempub8mgzbMYxTT05F/UJp05p3SIoWNVcALlfOnFZeRERirQiHkQULFtCjRw++//57Dh48SN68ealUqRK3bt164/5BQUFUqFCBCxcusHjxYk6fPs3kyZNJmzZtlIuXmKdVvlbsb7+fDzN+iJODEzef3GTF6RX029KPr898zfCdw6N+kaRJYcMGM4g8fgxVq5p9SUREJFaKcBgZNWoU7dq1o3Xr1uTIkYMJEybg7u7O1KlT37j/1KlTuXfvHsuXL6dkyZJkypSJDz/8kLx580a5eImZcqTIweZWmwn4OoAdbXYwquIoPsn2CQB9N/dl3dl1Ub+Ih4c5Y2vt2vD8OdStq9lbRURiKaeI7BwUFMSBAwfo06dP2DYHBwfKly/Prl273njMihUrKF68OJ07d+aPP/4gRYoUNGnShK+++gpHR8c3HvP8+XOeP38e9nNAgDmHRXBwMMHBwREp+T+9PJc1zyn/cMKJwqkKUzhVYT7L+xm1ptbC964vjZc0ZlfrXWRJkiVqF3BwgDlzcOzQAYeZM6FFC0IePCC0QwfrfIFYSr/XtqX2th21te1Yq63De3yEwsidO3cICQnB29v7le3e3t6cOvXm/gDnzp1j48aNNG3alDVr1nD27Fk6depEcHAw33///RuPGTp0KANerub6L+vXr8fd3T0iJYeLr6+v1c8pr2uftj0Xn13kr6d/UXlaZX7M+iNujm5RP3GtWuS6d48sq1bh2LUrp/fu5Uy9elE/byyn32vbUnvbjtradqLa1k+fPg3XfhYjAsMdrl27Rtq0adm5cyfFixcP2/7ll1+yZcsW9uzZ89ox2bJlIzAwkPPnz4fdCRk1ahQ//fQT169ff+N13nRnJH369Ny5cwdPT8/wlvtOwcHB+Pr6UqFCBZydna12Xnndy7bOWSwnpWaV4uaTm9T3qc/sWrOxWCxRv4Bh4DBgAI5DhgAQ0qsXoT/8AE+fwpMnZt+SRIngf4J0XKTfa9tSe9uO2tp2rNXWAQEBJE+enIcPH/7nv98RujOSPHlyHB0duXnz5ivbb968SapUqd54TOrUqXF2dn7lkYyPjw83btwgKCgIFxeX145xdXXF1dX1te3Ozs7R8gsYXeeV12VMmpHFDRbz8YyPWXRyEUXSFaFXiV7WOfngwWbn1l69cBwxAscRI1793MkJtm2DYsWsc70YTr/XtqX2th21te1Eta3De2yEOrC6uLhQsGBB/Pz8wraFhobi5+f3yp2SfytZsiRnz54l9F9rivz111+kTp36jUFE4r5SGUoxptIYAL70/ZLZR2Zb7+Q9e8KkSeZsrS9ZLODiAi9eQI8eYM25T0REJMoiPJqmR48eTJ48mRkzZnDy5Ek6duzIkydPaN26NQAtWrR4pYNrx44duXfvHt26deOvv/5i9erVDBkyhM6dO1vvW0is06lwJz4v8jkGBq2Wt2L5qeXWO3m7dnDnDty+bT6mCQmB8+fB3R127YKlS613LRERibIIh5GGDRsyYsQI+vXrR758+fD392fdunVhnVovXbr0Sl+Q9OnT8+eff7Jv3z7y5MlD165d6datG19//bX1voXEOhaLhTGVx9Ayb0tCjBAaLm6I799W7JTm4QHJk0OCBOadkTRpzLsmAF9/DUFB1ruWiIhESYT6jLzUpUsXunTp8sbPNm/e/Nq24sWLs3v37shcSuIwB4sDv3/yO4+DHrPk5BJqLaiFb3NfSqQvET0X7N0bJk6Es2fNPz//PHquIyIiEaK1acSunBycmFNnDpWyVOJp8FOqzqnKzss7o+diiRLByyHjAwfCw4fRcx0REYkQhRGxO1cnV5Y2XErpDKV5+PwhH03/iIn7J1p3kb2X2raFDz4w+5T8+KP1zy8iIhGmMCIxgruzO2uarqFejnoEhwbTYXUH2q1sR+CLQOteyNn5nxAyejRcvmzd84uISIQpjEiM4eHiwcJ6CxlWbhgOFgemHJpCmWlluPzQyoHhk0+gVCkIDIQWLeD4ceueX0REIkRhRGIUi8XCV6W+Yl3TdSRNkJR91/aRf2J+Fp9YbM2LwKhR5l2SzZshd25o0ACOHbPeNUREJNwURiRGqpClAgfaH6Bg6oLcfXaX+ovq03RpU+49u2edCxQuDPv3Q5065iRoixaZoaR+fbhxwzrXEBGRcFEYkRgrk1cmdrbdyXelv8PR4sjco3PJ/Vtu1p5Za50L5MkDS5bA4cPwcmG9xYuhSBFzm4iI2ITCiMRoLo4u/FD2B3a23ckHyT7g2qNrVJ1blb4b+1pvtE2ePOadkcOHzZE2ly9DyZLwxx/WOb+IiPwnhRGJFYqkLcLBzw7SrWg3AAZtG0TP9T2tO/w3Tx5zuvjy5c2VfmvXhuHDtZaNiEg0UxiRWMPd2Z0xlccwruo4AEbvHk3nNZ0JNULfcWQEJEkCa9ZAp05mCPnqKyheHNq0MSdMmzED9uxRQBERsaJITQcvYk+dCnfCzcmNT1d8ym/7fyPwRSCTa0zG0cHROhdwdoZx4yBHDujWzQwfe/a8uk/btjB5sjkyR0REokRhRGKlNvnb4OroSsvlLZnmP43AF4FMrzUdF0cX612kc2eoUAH27oULF+DiRfPPjRthyhRzevlRoxRIRESiSGFEYq2meZri5uRGoyWNmHdsHlcCrrCkwRJSJExhvYtky2a+/m36dGjdGsaMgcSJoX9/611PRCQeUp8RidXq5qjLqsar8HT1ZNulbRT9vSjHb0XzjKqtWsEvv5jvBwww746IiEikKYxIrFfp/UrsbrubzEkyc/7BeYpPKc6aM2ui96Kffw6DB5vve/Y0+4+IiEikKIxInOCTwoe9n+7lw4wf8ijoETXm1WDQ1kG8CH0RfRft08ccbQPQoQNs3x591xIRicMURiTOSOaejPXN19OuQDtCjVD6bupL8SnFo++xjcUCQ4dC06YQGgpNmsD9+9FzLRGROExhROIUF0cXJlafyMxaM/Fy82L/tf0UmFSAoduGRs9dEosFfvsN3n/fnLn10081B4mISAQpjEicY7FYaJ63Occ7Had6tuoEhQTxzcZvKDGlBFcDrlr/gokSwfz55vwkS5fCxInWv4aISBymMCJxVppEaVjRaAUzas3Ay82Lfdf2UWZ6GS48uGD9ixUsCMOGme+/+AKOHbP+NURE4iiFEYnTLBYLLfK24GD7g2ROkplz989RamopTt85bf2Lde8OlStDYCA0agSPHln/GiIicZDCiMQL7yV5j22tt5EjRQ6uPrpKmellOHLziHUv4uBgrl2TKhUcPw5eXpAzJzRrBiNGwMGD1r2eiEgcoTAi8UaaRGnY0moL+VPl59aTW3w4/UP2XNnz7gMjImVKWLgQ0qc3R9icOAFz5kDv3uajnO++g5AQ615TRCSWUxiReCW5e3I2ttxIifQleBD4gEqzK3Ho+iHrXqR0aXMdm6tXYfVqGDQIqlUzPxs82Hx/9651rykiEospjEi84+XmxZ/N/qR0htI8fP6QirMrcvL2SetexGKBNGmgalX49ltYtcq8Q5IgAfz5JxQqpMc2IiL/T2FE4iUPFw9WNl5JwdQFufP0DhVmVeD8/fPRe9EmTWD3bsiSxVz9t2RJWLAgeq8pIhILKIxIvJXYLTHrmq0L69RaflZ5rj26Fr0XzZMH9u0zH9UEBpoBZeHC6L2miEgMpzAi8Vpy9+T4NvcNG/ZrkzskSZLAihXQps0/08gvXx691xQRicEURiTeS5MoDRuabyBtorScuH2CnONzMnTbUIJCgqLvog4OMGmSOew3JAQaNDA7u4qIxEMKIyKY85Bsbb2VjzN9zLMXz/hm4zfkm5CPrRe3Rt9FHR1h2jQziAQHQ506sH599F1PRCSGUhgR+X+Zk2TGr4Ufs2rPIoV7Ck7eOcmH0z+k8+rOhIRG09wgTk4wezbUrg1BQWZfkqpVzbsmN25EzzVFRGIYhRGRf7FYLDTL04zTXU7zWcHPABi/fzztV7Yn1AiNnos6O5sL7dWpAy9ewNq18Nln5tDgkiXNYcEiInGYwojIGyRJkIQJ1SewoN4CHCwOTPWfSufVnTEMI3ou6OICixebM7YOHgyFC4NhwM6dUKsWrFsXPdcVEYkBFEZE/kODnA2YWWsmFixMODCB7uu6R18gsVjAxwe++Qb27oXLl6FhQ7ODa716cOBA9FxXRMTOFEZE3qFpnqZM+WQKAL/s/YXevr2jL5D8W7p0MHMmlC8PT56YfUnOnYv+64qI2JjCiEg4tM7fmonVJwIwctdIqs6tyrFbx6L/wi4usGQJ5MsHt25B5cpw5070X1dExIYURkTCqX3B9oyvOh4nByfWnV1H3gl5ab+yPdcfXY/eC3t6wpo1kDEjnDkDFSpAhw7mkOAKFcz+Jd98E701iIhEI4URkQjoWLgjJzqdoK5PXUKNUCYfnEzWsVkZsXNE9F44dWqzE2vSpODvDxMnwqJFsGED7N8PQ4eaC/CJiMRCTvYuQCS2yZosK4sbLGbHpR308u3F7iu76e3bm9QeqWmap2n0XTh7dti+HWbNAldXc1r5pEnB19fsW/L553D0qDm7q4hILKL/aolEUskMJdnZZifflDIfkXRc3ZG/7/0dvRf18YEhQ+D776FrV3M6+bFjIVUq8xHOyJHRe30RkWigMCISBRaLhQEfD6B0htI8CnpE4yWNo3dNmzfx9ISffjLfDxoEly7Z9voiIlGkMCISRU4OTsyuM5skbknYd20f/Tb1s30RTZtC6dLw7BmOvXvb/voiIlGgMCJiBRkSZ+D3T34H4McdP7Lh3AbbFmCxwK+/gqMjDsuWkcLf37bXFxGJAoURESup41MnbD2b5suac+vJLdsWkCcPdOkCQO7Jk82F90REYgGFERErGlVpFDlS5ODG4xuUnFqSQ9cP2baAAQMwvL1JdPUqjh9/DJs22fb6IiKRoDAiYkXuzu4sqr+I9J7pOXvvLMWmFGPc3nG2mT4eIHFiQn77jReurjjs2wdly5qztuqxjYjEYAojIlaWI0UODn12iBrZahAUEkSXtV2ot6geDwIf2OT6RvXqbJgwgZCOHcHJyZwMLX9+aNIE/vrLJjWIiESEwohINEjmnow/Gv3B6EqjcXZwZunJpeQan4tRu0YR8Dwg2q//PEkSQn/+GU6dgsaNzY3z5kGOHNC2LVy4EO01iIiEl8KISDSxWCx0L9adnW13kjlJZq4+ukrP9T1JNyodPf/sycUHF6O/iCxZYO5cOHQIqleHkBCYOhWyZYNOneDBg+ivQUTkHRRGRKJZoTSFONbxGJOqTyJ78uw8CnrEqN2jyPJLluhf0+alfPlg5UrYtQvKl4fgYPjtN2jXzjbXFxH5DwojIjaQwDkB7Qq243in46xpsoZy75UjxAjhqw1fsf3SdtsVUqyYuZbNunXm3CSLF8PBg7a7vojIGyiMiNiQg8WBKlmrsKHFBlrmbUmoEUrzZc1t0o/kFZUqmR1aAfrZYcZYEZF/URgRsZNfqvxCJq9MXHhwga5ru9q+gO+/B0dHWL3afHwjImInCiMiduLp6sms2rNwsDgw4/AMFh1fZNsCsmaFVq3M9999Z9tri4j8i8KIiB2VylCKPqX6APDZqs+4GnDVtgX07QvOzrBxo/kSEbEDhRERO/v+w+8plKYQ9wPv0+qPVrwIfWG7i2fMCO3bm+/79gVbzRQrIvIvCiMidubs6Mzs2rNJ4JSADec2UHJqSU7fOW27Ar79FtzcYOdOc5SNiIiNKYyIxAAfJP+AeXXnkdg1MXuv7iXfxHz8vPtnQo3Q6L946tRhq/3y7bcQGBj91xQR+ReFEZEYomb2mhzrdIyKWSoS+CKQ7n92p/zM8raZqfWrr8DDw5yptWhROHky+q8pIvL/FEZEYpB0nulY13Qdv1X7DXdndzZd2ESBSQXYdH5T9F44eXJYtgxSpIAjR6BgQfj9d/UhERGbUBgRiWEsFgsdCnXgSIcjFEpTiHvP7lFxdkUm7J8QvRcuX94MIhUqwLNn5lTxDRvCwoUwZQqMGQODBsH48eYaNyIiVuJk7wJE5M2yJM3C1lZbabuiLfOOzaPj6o4cvXmUMZXH4OzoHD0XTZXK7MQ6ciR88w0sWmS+/tfz5/DFF9FTg4jEO7ozIhKDJXBOwJw6cxhSdggWLIzfP57Kcypz/v756LuogwP07g07dkDlylCmDFSrBo0aQc2a5j59+8KlS9FXg4jEKwojIjGcxWKhT+k+LG+0HA8XDzae38gHv35A59Wduf7oevRduEgRWLsWtmyBVatg3jxYuhRKlYInT8wROOpTIiJWoDAiEkt88sEn7Pl0DxWzVCQ4NJjx+8eT5ZcsfOX7Ffee3bNNEQ4OMHGiOWvrypVmp1cRkShSGBGJRXKkyMGfzf5kU8tNFE9XnGcvnjF853AKTy5su0CSI4c5FBjg88/h4UPbXFdE4iyFEZFY6KNMH7GjzQ5WNl5JhsQZOHf/HM2WNrPNJGlgdm59/324dk2L7IlIlCmMiMRSFouF6tmqs6LRCtyc3Fh7di2Dtg6yzcUTJIAJ/z/UeNw42LPHNtcVkThJYUQklsubKi8TqpnBoP/m/vz595+2uXC5ctC8udmJtUYNWLLENtcVkThHYUQkDmiZryXtC7THwKDlipbcCrplmwuPGgW5c8Pt21CvnjlJ2u3btrm2iMQZCiMiccTPVX4Om7F1+PnhBL6wwYJ3yZPDvn3mAnuOjuZsrTlywJw5cPdu9F9fROKESIWRcePGkSlTJtzc3ChatCh79+59677Tp0/HYrG88nJzc4t0wSLyZm5Obiyuv5ikCZJy9tlZOq3thGGLeUBcXc1p4vfsMe+S3LkDzZqZQSV5cihRAlq1Mmd2FRF5gwiHkQULFtCjRw++//57Dh48SN68ealUqRK3br39trCnpyfXr18Pe128aINVSEXioYxeGZlVcxYOODD76GyGbh9qu4sXLAj798P330O6dOa2u3dh1y6YMQOqVoUFC2xXj4jEGhEOI6NGjaJdu3a0bt2aHDlyMGHCBNzd3Zk6depbj7FYLKRKlSrs5e3tHaWiReTtKmSuQLt07QD4duO3LDy+0HYXd3GB/v3h8mV4/Bj8/c1HNw0bmh1dmzfXHRIReU2EFsoLCgriwIED9OnTJ2ybg4MD5cuXZ9euXW897vHjx2TMmJHQ0FAKFCjAkCFDyJkz51v3f/78Oc+fPw/7OSAgAIDg4GCCg4MjUvJ/enkua55T3kxtbTvBwcFUSV4FJ28nxh0YR4tlLUiTMA1F0xa1bSEuLmb/kRw5oEYNHA0Dh4ULMerUIWTNGoySJW1bTzTR77btqK1tx1ptHd7jLUYEHipfu3aNtGnTsnPnTooXLx62/csvv2TLli3secNcA7t27eLMmTPkyZOHhw8fMmLECLZu3crx48dJ9/JW7v/o378/AwYMeG373LlzcXd3D2+5IvFaiBHCsPPD2Bewj8ROiRmedTjerva7K2l58YIiQ4eS6sABgt3d2T5oEAGZM9utHhGJfk+fPqVJkyY8fPgQT0/Pt+4X7WHkfwUHB+Pj40Pjxo354Ycf3rjPm+6MpE+fnjt37vznl4mo4OBgfH19qVChAs7O0bQkuwBqa1v6d1s/N57z8ayPOXzzMJm9MjO8/HBqZK2BxWKxT3FPn+JYvToO27djpEhByJIlGMWK2acWK9Hvtu2orW3HWm0dEBBA8uTJ3xlGIvSYJnny5Dg6OnLz5s1Xtt+8eZNUqVKF6xzOzs7kz5+fs2fPvnUfV1dXXF1d33hsdPwCRtd55XVqa9txdnbG3dmdVU1WUWJKCc49OEe9xfUokb4EP5b/kVIZStm+qMSJzRWAy5bFcvAgTmXKQJs2MGwYpEhh+3qsSL/btqO2tp2otnV4j41QB1YXFxcKFiyIn59f2LbQ0FD8/PxeuVPyX0JCQjh69CipU6eOyKVFJJLSeabjSMcj9CnVhwROCdh5eSelp5Xmk3mfcPGBHUa2JU4Mvr5mCAGYOhWyZYPx4yEkxPb1iIjdRXg0TY8ePZg8eTIzZszg5MmTdOzYkSdPntC6dWsAWrRo8UoH14EDB7J+/XrOnTvHwYMHadasGRcvXuTTTz+13rcQkf/k5ebFkHJDONv1LO0LtMfR4sjKv1by4fQPufzwsu0LSpoUpkyBnTshXz548AA6dzbvjrz/PhQoAB99BI0awYkTtq9PRGwqQo9pABo2bMjt27fp168fN27cIF++fKxbty5suO6lS5dwcPgn49y/f5927dpx48YNkiRJQsGCBdm5cyc5cuSw3rcQkXBJkygNE2tM5IviX/DJvE84c+8MZWeWZWurraROZIe7lcWLm3OTTJhgrv57/775+rd9++DwYfDwsH19ImITEQ4jAF26dKFLly5v/Gzz5s2v/Dx69GhGjx4dmcuISDTJnjw7fi38KDO9DGfvnaX8rPJsbrmZFAnt0G/D0dG8K9KqFZw/Dw8fQkCA+edXX8G5c+af48bZvjYRsQmtTSMST6VPnB6/Fn6kTZSWE7dPUHF2Re4/u//uA6NLwoSQKxeULAlVqpiPaKZMMT8bPx7+1VdNROIWhRGReCxzksz4tfAjZcKU+N/wp8qcKjwOemzvsv5Rvjx07Gi+b9PGvGMiInGOwohIPPdB8g/Y0HwDSRMkZc/VPdRbWI/gkBg0w+Xw4ZA5M1y6BD172rsaEYkGCiMiQm7v3KxpsgZ3Z3f+/PtP2q5oS6gRau+yTB4eMG2a+f7337W2jUgcpDAiIgAUTVeUxfUX42hxZNaRWXy94Wt7l/SPMmWgWzfzfdu2cPq0fesREatSGBGRMFWyVmHKJ2an0Z92/sToXTFoJNyQIeDjA9euQdGi5sRpIhInKIyIyCta5mvJsHLDAOixvgdTDk6xc0X/z90dNm+GEiXMYb9Vqmi4r0gcoTAiIq/5suSXdC3SFYBPV35KlzVdCAoJsnNVQMqUsHEjtGhhTh3fpYs5R4mWlBeJ1RRGROQ1FouF0ZVH07dMXwDG7RvHxzM+5tqja3auDHB1henTzcX1LBZzDpJGjbSujUgspjAiIm/kYHFg4McDWdFoBYldE7Pz8k4KTCzAtovb7F2aGUK++gqWLQMXF1i6FHr1sndVIhJJCiMi8p9qfFCDfe32kStlLm4+uUnZmWVZcmKJvcsy1awJM2aY78eMMV8iEusojIjIO2VNlpXdbXdTL0c9XoS+oNGSRiw7uczeZZkaNYIffzTf9+hh3iURkVhFYUREwiWhS0Lm151Pk9xNeBH6ggaLG/DHqT/sXZapd2/o0AEMA5o2hd277V2RiESAwoiIhJujgyMzas2gca7GvAh9Qf1F9Vl5eqW9yzL7kIwdC9WqQWAgVK0KAwfC5cv2rkxEwkFhREQixMnBiZm1Z9IwZ0OCQ4Opu7Auq/5aZe+ywMkJ5s+HwoXh/n34/nvImNGcj2TJEgiKAUOTReSNFEZEJMKcHJyYXWc29XPUJzg0mDoL6sSMPiQeHrB1K8yeDR99ZD62WbcO6tWD5MmhQQPzs3v37F2piPyLwoiIRIqTgxNz6swJu0NSf1F95h+bb++ywM3N7DeyaROcOQN9+kDq1PDoESxaBM2bm5OnVayoxzgiMYTCiIhEmrOjM3PqzKFF3haEGCE0XdqUGf4z7F3WP95/31zT5soVs1Prt99CnjzmBGm+vubQ4KdP7V2lSLynMCIiUeLo4Mi0mtNoV6AdoUYorf5oxaQDk+xd1qscHMzF9QYNgsOH4cQJSJECDh2C9u3NxzkiYjcKIyISZQ4WByZWn8jnRT4H4LNVn1FtbjV2XNph58rewscHFi4ER0eYMwdGx6DViUXiIYUREbEKi8XCz5V/5tvS3+JgcWDNmTWUmlaKj6Z/xPq/12PEtLsPH330Twjp3Rs2bLBrOSLxmcKIiFiNxWJhUNlBnOp8ik/zf4qzgzNbLm6h0uxKfDzjY64GXLV3ia/q0gVatYLQUGjYEM6ft3dFIvGSwoiIWF3WZFmZ/MlkznU7R7ei3UjglIAtF7eQf2J+fP/2tXd5/7BY4LffzLlJ7t2DAgWgTRtYswaeP7d3dSLxhsKIiESbdJ7pGFN5DIc7HCavd15uP71NpdmV6L+5PyGhIfYuz+TmZq5nkzUrPHgA06aZM7l6e5t3TW7csHeFInGewoiIRLusybKyq+0u2hdoj4HBgC0DqDS7Enef3rV3aaZ06eDkSdi82Xx0kzo1PHxorghcrRo8eWLvCkXiNIUREbGJBM4JmFhjIrNqz8Ld2R2/835UnlOZR88f2bs0k6MjfPihucbNlStmMEmRAg4ehJYtzX4lIhItFEZExKaa5WnGnk/3kNw9Ofuv7afm/JoEvgi0d1mvcnAwg8nSpeDiYq5t8/339q5KJM5SGBERm8uVMhdrm67Fw8WDTRc20XiJuQpwjFOqFEz6/wncBg2CuXPtW49IHKUwIiJ2UShNIVY0WoGroyvLTy2n/cr2MW8uEjAf0Xz1lfm+TRtzWnkRsSqFERGxm4/f+5j59ebjYHFgmv80uq/rHnNG2fzbkCHmOjbPn0PlyjBxovqQiFiRwoiI2FWt7LWY8skUAH7Z+wtlZ5bl0sNLdq7qfzg4wOzZUKKEOcqmQwfzEc7Ro/auTCROUBgREbtrla8Vc+rMIZFLIrZe3EreCXlZeHyhvct6lYcHbNkCY8aY73ftggIFcPjyS7L88QcOPXpA7drmxGlVq8Ljx/auWCTWUBgRkRihSe4m+Hfwp2jaojwIfEDDxQ1p80cbHgY+tHdp/3Bygm7dzDlJ6tSBFy9wHDOGXNOm4fjrr7B8ubkS8Nq10LWrvasViTUURkQkxsicJDPbWm/ju9LfYcHCNP9pvPfzewzdNpTHQTHoTkO6dOZw3xUrCK1alSulShHSsyeMGwfjx5vTzE+bBosW2btSkVhBYUREYhRnR2d+KPsDm1ttJnvy7NwPvM83G78h88+ZGbVrFM+Cn9m7xH/UqEHI8uUc6NWL0KFDoVMn6NgR+vQxP2/fHi7FsP4vIjGQwoiIxEhlMpbhWMdjzKw1kyxJsnD76W16ru9JjvE5uPjgor3L+2/9+0ORIuZaN82bQ0gMHCEkEoMojIhIjOXo4EjzvM052fkkv9f4nXSe6bjw4ALV5laLWX1J/pezM8yZY3Z03boVhg+3d0UiMZrCiIjEeM6OzrQt0JadbXaS2iM1x28fp96iegSHBNu7tLd7/31znRuAfv1g50771iMSgymMiEiskT5xelY1WUVC54RsOLeBjqs7xsxZW19q2RIaNIAXL6BsWRg5UpOlibyBwoiIxCoFUhcIm7V1yqEpDNs+zN4lvZ3FYq5tU62aOXtrr15mKLkYw/u8iNiYwoiIxDrVs1Xn58o/A/DNxm+YfWS2nSv6D4kTw8qV5hTyCROaE6flzg1Tp+ouicj/UxgRkVipS5EudC/aHYCWy1sy8/BM+xb0XywWc5ivvz8ULw6PHkHbtmYomTtXo20k3lMYEZFYa2SlkXya/1NCjVBaLW/F5AOT7V3Sf3v/fXN0zbBh5h2TEyegaVPw8YHp0yE4BnfIFYlGCiMiEms5WByYWGMinQt3xsCg/ar2/Lr3V3uX9d+cnOCrr+DCBRg0CJImhTNnoHVryJQJBg6E69ftXaWITSmMiEis5mBxYGyVsfQo1gOAz9d+zsidI+1cVTh4ecG335qdWYcPB29vuHYNvv8eMmSAhg1h+3Z7VyliEwojIhLrWSwWRlQcwTelvgGgl28veq/vTagRCzqIenhA797mtPFz50KpUuZQ4IULoXRpmBzDHz2JWIHCiIjECRaLhcHlBjOk7BAARuwaQf1F9Xka/NTOlYWTiws0bgzbtpkdXZs0Mbd36QL799u1NJHopjAiInFKn9J9mF17Ni6OLiw9uZSPpn/Ejcc37F1WxOTNC7NnQ82aEBQE9erBvXv2rkok2iiMiEic0zRPUzY030DSBEnZd20fxX4vxpoza3gQ+MDepYWfxWKOsMmSxexX0ry55iWROEthRETipNIZS7O77W7eT/o+Fx9epNrcaiT5MQnZf81Oi2UtmO4/nZDQGD6/h5cXLF4Mbm6wZg0MGWLvikSihcKIiMRZWZNlZXfb3Xya/1OyJMkCwOm7p5l1ZBat/2jNN37f2LnCcMiXD377zXzfrx+sX2/XckSig8KIiMRpydyTMfmTyZztepY7ve+wpskaehbvCcDwncPZcG6DnSsMh1atoF07MAyzH8mvv5rvReIIhRERiTeSuSejStYqjKg4gs8KfgZA82XNuf3ktp0rC4dffoGqVSEwED7/3Hx/43865gYHw969cP68fWoUiSSFERGJl0ZVGkWOFDm48fgGrf9ojRHT7zS4ucGqVWYocXWFdevMtW1mzIDRo6F6dUiWDIoWNUfjXLpk74pFwk1hRETiJXdnd+bXnY+royurz6xm7N6x9i7p3SwW867IgQNm4Lhzx3yE06MHrF5tLsBnsZh/dutm72pFwk1hRETirdzeuRlRcQQAvX17c/jGYTtXFE45c8KePfDll5AxI1SuDD/9BAcPwuHD5vo3y5fDihX2rlQkXBRGRCRe61y4MzWy1SAoJIjq86qz7eI2e5cUPq6u8OOP5oJ7a9dCr16QP7/56Kan2UGXzz+HJ0/sWqZIeCiMiEi8ZrFYmFpzKtmSZeNKwBU+mvERfTf2JTgk2N6lRV7fvuYdk0uXYMAAe1cj8k4KIyIS7yV3T87+dvtpla8VoUYog7YNovS00vx97297lxY5CROaw38BRo2Co0ftW4/IOyiMiIgAiVwTMa3mNObXnU9i18TsubqHfBPzsfjEYnuXFjnVq0Pt2hASAh06aCp5idEURkRE/qVhroYc6XiE0hlK8zjoMfUX1afvxr6EGrHwH/NffgEPD9i50+xP0qIFDBsGK1fCzZv2rk4kjMKIiMj/yJA4AxtbbqRHsR4ADNo2iNoLahPwPMDOlUVQunQwcqQ53PfIEZg1C/r0gU8+MfuUDB1qTpQmYmcKIyIib+Dk4MTISiOZWWsmro6urDi9gmK/F+PM3TP2Li1i2reHv/+GpUth0CBo3Bh8fOD5c/jmG3OStEOH7F2lxHMKIyIi/6F53uZsa72NNInScPLOSfJPzM+QbUMIfBFo79LC7733zP4j334Lc+fC8eMwcyYkTWoGkcKFzc8CY9F3kjhFYURE5B0Kpy3M/nb7KZOxDE+Cn/Dtxm/xGefDkhNLYv408m9isUDz5nDiBNSvb3ZyHTLE7PT6/Lm9q5N4SGFERCQcUidKzeaWm5lTZw7pPNNx4cEF6i2qR4U5Fbj2/Jq9y4scb29YuNB8hOPhAX5+5vTyGnkjNqYwIiISThaLhSa5m3Cq8yn6lemHm5MbWy9t5au/vmL3ld32Li/yatc2A4mTE8yfD71727siiWcURkREIiihS0IGfDyAU51PUSh1IR6FPKLi3Ir8ceoPe5cWeRUqwPTp5vtRo8xROCI2ojAiIhJJGb0y4tvUl0KehQh8EUidhXX4bd9v9i4r8po2NRfcA3Otmzlz7FuPxBsKIyIiUZDQJSF93utD23xtCTVC6bSmE1+s+4Lrj67bu7TI6dkTvvjCfN+qlTlxWmzspCuxisKIiEgUOVocGV9lPAM/GgjAmD1jSDc6HVXmVGHu0bk8DX5q5wojwGKBESOgZUt48QK6dYMGDeDhQ3tXJnFYpMLIuHHjyJQpE25ubhQtWpS9e/eG67j58+djsVioVatWZC4rIhJjWSwW+n7YlyUNllAifQlCjVDWnV1H06VNSTUiFRP3T7R3ieHn4ADTpsHPP4OzMyxeDIUKgb//P/s8fGj+vHy5uSjfV1+Zj3k+/hgGD7ZT4RJbOUX0gAULFtCjRw8mTJhA0aJFGTNmDJUqVeL06dOkTJnyrcdduHCBXr16Ubp06SgVLCISk9XxqUMdnzqcvXeWWYdnMevILM4/OE+H1R1I6JKQZnma2bvE8LFYoGtXc4bWBg3g7FkoVgxy5YLz5+Hevbcfu3kzpEkDrVvbrFyJ3SJ8Z2TUqFG0a9eO1q1bkyNHDiZMmIC7uztTp0596zEhISE0bdqUAQMGkDlz5igVLCISG7yf9H0GfDyAv7v+Tfei3QFo/Udr1p5Za9/CIqpoUTh4EKpWNSdEO3DgnyCSIoU5e2udOubjnJ9+MlcIBujY0dxXJBwidGckKCiIAwcO0KdPn7BtDg4OlC9fnl27dr31uIEDB5IyZUratm3Ltm3b3nmd58+f8/xfswAGBJiLUwUHBxNsxUWdXp7LmueUN1Nb247a2rbC097Dyg7j5uObzDs+j3qL6rG+yXqKpC1iqxKjztMTli7FsmEDPH+OkSmTOcW8h8fr+4aG4nj5Mg6rV2PUqcOL3bsheXKrlKHfbduxVluH9/gIhZE7d+4QEhKCt7f3K9u9vb05derUG4/Zvn07U6ZMwf/fzxrfYejQoQwYMOC17evXr8fd3T0iJYeLr6+v1c8pb6a2th21tW29q73rONbhVKJTHHp0iCqzqzAk6xDSu6W3UXVW5OgIly+br7dwatyYDw8exOPSJe5XqcKufv3M46xEv9u2E9W2fvo0fJ23I9xnJCIePXpE8+bNmTx5MskjkIz79OlDjx49wn4OCAggffr0VKxYEU9PT6vVFxwcjK+vLxUqVMDZ2dlq55XXqa1tR21tWxFp73JB5ag0txL7ru1j2NVh/FLpF6pnrY7FYrFRtTbk44NRujQpDx+m+p49hA4aFOVT6nfbdqzV1i+fbLxLhMJI8uTJcXR05ObNm69sv3nzJqlSpXpt/7///psLFy5Qo0aNsG2h/7/mgZOTE6dPnyZLliyvHefq6oqrq+tr252dnaPlFzC6ziuvU1vbjtratsLT3kmck7Cm6RpKTyvNqTunqLu4LiXSl2BYuWGUzhjHOvcXKABTpkDjxjgOH46jk5O5MrAV7m7rd9t2otrW4T02Qh1YXVxcKFiwIH5+fmHbQkND8fPzo3jx4q/tnz17do4ePYq/v3/Y65NPPuHjjz/G39+f9Olj4S1KEZEoSO6enF1td9GnVB8SOCVg5+WdlJlehmpzq3Hqzpsfd8dajRr9s87NkCHg42OugaNJ1OR/RPgxTY8ePWjZsiWFChWiSJEijBkzhidPntD6/4dwtWjRgrRp0zJ06FDc3NzIlSvXK8d7eXkBvLZdRCS+8HLzYki5IXQp0oUftvzA5IOTWXNmDTsv72Rt07UUS1fM3iVaz48/QvHi0L07XLoEdetCxYrQqRMEB8PTp+YrNBQaN4YkSexdsdhBhMNIw4YNuX37Nv369ePGjRvky5ePdevWhXVqvXTpEg4OmthVRORd0iRKw2/Vf6NH8R60+qMVOy/vpPzM8qxqsoqPMn1k7/Ksw2IxVwWuVAmGDoXhw2H9evP1vxYuhA0bzNWDJV6J1N94ly5d6NKlyxs/27x5838eO/3lqpAiIgJA1mRZWd9sPbUW1GLDuQ1UmVOFpQ2WUiVrFXuXZj3u7vDDD+Y08337wl9/QcKE5vYECcwQsmUL9O8PVujsKrGLbmGIiMQACV0SsrLxSmpkq0Hgi0Bqzq/JkhNL7F2W9b3/PsybZ06ItnUrrFsHy5bB77+bnw8eDGtj2cRwEmUKIyIiMYSbkxtLGiyhYc6GBIcG02BxA37Y8gMvQl/Yu7To17Ch2Y8EoFmz/5zHROIehRERkRjE2dGZOXXm8Gn+Twk1Qum3uR+lppbizN0z9i4t+o0aBQULmtPNN2gAQUH2rkhsRGFERCSGcXRwZFKNScyuPZvEronZc3UP+SbmY8L+CRhxeVisq6vZiTVxYti9G774QoEknlAYERGJgSwWC03zNOVox6OUfa8sT4Of0nF1R+ourMvzF8/ffYLYKnNmmDHDfD9+PGTIAP36wdWr9q1LopXCiIhIDJY+cXp8m/syutJoXB1dWXZqWdwPJDVrwqRJkCYN3LxpjsLJmBHHRo1Ievy4Jk2LgxRGRERiOAeLA92LdWd1k9W4Obmx+sxq6iysQ+CLQHuXFn3atYMLF8zHNmXKQEgIDkuXUvrbb3EqWtS8e/I8DgeyeEZhREQkliiXuRyrGq8igVMC1pxZQ50FcTyQODtD/frm/COHDxPati0hLi5Y/P2hVSvzEU6fPvDnn3D//qvH3r0L8+dD69ZQvboe88RwCiMiIrFIuczlWNXEDCRrz66l9oLaPA56bO+yol+ePIT89ht//v47IYMGQbp0cOsWDBsGlStD0qTm2jfNmkGRIpAihTm9/PTpsHo1tG+vxzsxmMKIiEgsU/a9smGBZN3ZdWQdm5UJ+ycQHBJs79KiXbCnJ6FffgnnzsGCBdC8uTmRGsCpUzBnDuzbZwaP3Lmha1dwcYE1a8z9JUZSGBERiYXKvleWP5v9SeYkmbnx+AYdV3ck12+5WHJiSdwe/vuSs7M5F8nMmXDmDNy+DatWwcCBMG2a+VjmyBH4+Wf49lvzmK5dzcc3EuMojIiIxFKlM5bmZOeT/FL5F5K7J+evu39Rb1E9Ppz+IRceXLB3ebaVPDlUq2aue9OqlTkS56Wvv4acOc3A0quX3UqUt1MYERGJxVwcXfi86Of83fVvviv9He7O7my7tI18E/Kx6Pgie5cXM7i4wOTJ5grC06ebi/JJjKIwIiISB3i6evJD2R841vEYxdIV4+HzhzRY3ID2K9vzNPipvcuzv+LFoXNn8/1nn8HT/2+TBw9g/37Yvh1exIM1gGIohRERkTjkvSTvsbXVVvqU6oMFC5MPTqbQpEKM2zuOfVf3ERQSj6dXHzLEHIVz7hzkyWM+2kmSBAoXhtKloUIFuHHD3lXGSwojIiJxjLOjM0PKDcG3uS+pPVJz8s5JuqztQpHfi5BoaCKK/V4s/qwG/G+JEsFvv5nv//77n86sqVKBuzts3gz585vzmvzbvXvw00/mEOJ9+2xacnzhZO8CREQkepTLXI7DHQ7z2/7f2H1lN3uv7uXus7vsubqHPVf3cOjGIebVnYerk6u9S7Wd6tXB19ecJC1rVnNYsIcHnD4N9erBsWNQtiwMGmTu++uvMGsWPHtmHn/tGvj7g4P+v7w1KYyIiMRhKRKmoN+H/QAwDINz98/x599/8sWfX7Ds1DI+mf8Jyxouw93Z3c6V2lD58q9v++AD2LMHOnUyp5r/5hvz9VLevHD+PBw9CosXm8OKxWoU7URE4gmLxUKWpFnoVLgTq5usxt3ZnfV/r6fS7Eo8DHxo7/Lsz93dnKPk99/BzQ0cHc27JVu3wqFD0KOHuV///hASYtdS4xqFERGReKh85vL4NvclsWtitl/aTrmZ5bjz9I69y7I/iwXatjX7lFy+DIsWmZ1bLRbo3t2cdv7kSZg3z96VxikKIyIi8VSJ9CXY1HITyd2Tc+D6AfJPzM+WC1vefWB8kCYNpE796rbEiaF3b/P9gAEQHPen37cVhRERkXgsf+r8bG21lfeTvs+VgCt8PONjvvH7Jl6scxMpXbqYi/CdPWtORS9WoTAiIhLP+aTw4dBnh2iTrw0GBkO3D6Xk1JKcuXvG3qXFPB4e5vTyAD/8AEHxeN4WK1IYERERPFw8mFJzCovqLyKJWxL2XdtHvon5+HH7j/F7orQ36djRfIRz8SJMmWLvauIEhREREQlTL0c9Dnc4zMeZPuZp8FO+9vuavBPy4nfOz96lxRwJEvwz7Pfbb6FnT3PuksBAc5thmEOAR40yJ0qrUAHuqHPwf1EYERGRV6RPnB6/Fn7MqDWDlAlTcurOKcrPKk/DxQ25EnDF3uXFDO3amSsB379vho6KFSFZMnPCtLRpzenme/aEP/80F+br0sXeFcdoCiMiIvIai8VCi7wtON3lNJ8X+RwHiwMLjy8k29hs9N/cnydBT+xdon25usLu3bBwIbRpY46+efoUNm2C69fNuyeVK0O/fuZ8JQsWmMOE5Y0URkRE5K283Lz4pcovHGh/gJLpS/LsxTMGbBlAtl+zMfPwTEKNUHuXaD8eHlC/vtlv5MoVOHIEJkwAPz9zPZu1a80hwH36mPt36gS3br1+ntBQM8CExt+2VBgREZF3ypcqH9tab2NBvQVk8srEtUfXaLm8JUV/L8rJ2yftXZ79WSyQOzd89pn5qMbN7Z/P+vY1H9vcuWMGEsP45zN/f8iXz7yz4ukJxYtDhw7mgn6XL9v6W9iNwoiIiISLxWKhQc4GnOx8kqHlhpLIJRH7r+2n8OTCzD06197lxVwuLjB9Ojg5wZIl5qOd4GBzaHDhwmZnV4AnT8xHPxMnmqElZ05YvtyelduMwoiIiESIm5MbX5f6mtNdTlP2vbI8CX5C06VN6biqI4EvAu1dXsyUP7858gbMoFGihNmf5MULqFPHXA34xAlzmvmvvzb3f/QIatc2j4vja+EojIiISKSkTpSa9c3W813p7wCYcGACJaeW5K+7f9m5shjqm2/MRzL37sH+/ZAkCcyZY64CnDo1+PhAo0YwdKi5gnD37uZxQ4ZA9ermcXGUk70LEBGR2MvRwZEfyv5AyQwlaba0GQevH+SDXz+gdIbSNMvTjPo56pMkQRJ7lxkzuLiYU8h/8okZSsaNM/uKvImzM4weDYUKmcOI160zjyle3Oxbkjix+apWDQoUsOW3iBYKIyIiEmWV36/Moc8O0X5Ve/48+yfbLm1j26VtfL72c6pnq86P5X/k/aTv27tM+8udG86fD//+TZtCrlzm45rz51/v1PrDD2Y/k1geSBRGRETEKtInTs/apmu5/PAy847NY/aR2Ry9dZSlJ5ey4dwGZtaaSc3sNe1dZuyTNy8cOgSrV8Pdu/Dwofnats18nNOkCRw8CO7u9q400tRnRERErCp94vR8WfJLjnQ8gv9n/pRMX5KA5wHUWlCLPhv68CL0hb1LjH0SJzZDx+efw3ffwU8/meEkTRo4fRp69bJ3hVGiMCIiItEmb6q8bGq5ie5FuwMwbMcwKs2uxK0nb5j8SyImWTJzyDCY85KsWmXXcqJCYURERKKVs6MzoyuPZn7d+SR0TsjG8xvxGefD4K2DeRj40N7lxW4VKkCPHub7Nm3g5k371hNJCiMiImITDXM1ZF+7feRMkZN7z+7x3abvyPRzJvpv7s/9Z/ftXV7sNWSIOcPr7dtmIPn3DK+xhMKIiIjYjE8KH/w7+DOnzhx8kvvwIPABA7YMIOOYjHy38TvuPYu7c2lEG1dXmDvX/HPNGihdGn7+GS5dsndl4aYwIiIiNuXk4EST3E041ukYC+stJI93Hh4FPWLwtsFkGpOJvhv7KpREVM6cMHasuUbOjh3mhGkZM5rTzX/5JYwcCTNmmGFl/3548MDeFb9CQ3tFRMQuHCwO1M9Zn7o56vLHqT/ov6U/R24eYdC2Qfy852e6Fu1K92LdSe6e3N6lxg7t2kHFiuZ6NkuXmkN/9+83X2+SNq0ZYl6+qlY1Z4K1A4URERGxKweLA7V9alMze02Wn1pO/839OXrrKIO3DWb07tG0L9CeniV64p3A296lxnwZM0K3bubr5k1YsQKOHzf7k9y5Y/554wZcvw5Xr5qv9evNYzdtUhgREZH4zcHiQB2fOtTKXotlJ5cxZPsQDl4/yJg9Yxi3bxzN8zSnZFBJe5cZe3h7m3dL3uThQ3NhvuPH/3nlzGnb+v5FYURERGIUB4sDdXPUpY5PHdb/vZ6h24ey5eIWpvpP5Q/nPyh8pzC5U+e2d5mxW+LE5jo3xYvbuxJAHVhFRCSGslgsVHq/EptbbWZHmx1kT5adu8F3KTurLAevH7R3eWJFCiMiIhLjlUhfgo3NN5IlQRbuPLvDxzM+ZtvFbfYuS6xEYURERGKF5O7J+eH9HyidvjQBzwOoNLsSa8+stXdZYgUKIyIiEmu4O7qzqtEqqmWtxrMXz6g2txrV51Zn7Zm1hBqh9i5PIklhREREYpUEzglY1nAZrfO1xsBg9ZnVVJ1blQ9+/YDRu0ZravlYSGFERERiHWdHZ6bWnMrpLqfpXrQ7iV0Tc/beWXqs70HaUWn5dMWn6uQaiyiMiIhIrJUtWTZGVx7N1R5XmVh9Inm88/DsxTOmHJpCwUkFKT6lOLOPzNYjnBhOYURERGK9hC4JaV+wPf6f+bOt9TYa52qMs4Mzu6/spvmy5lSeXZmbj2/au0x5C4URERGJMywWC6UylGJu3blc/uIyAz8aSAKnBPie8yXvhLxsOLfB3iXKGyiMiIhInOTt4U3fD/uyr90+cqbIyc0nN6k4qyLfbfyOF6Ev7F2e/IvCiIiIxGk5U+Zkb7u9tC/QHgODwdsGk+e3PIzZPYa7T+/auzxBYUREROIBd2d3JtaYyPy680nsmpiTd07yxZ9fkGZUGhotbsSGcxswDMPeZcZbCiMiIhJvNMzVkAvdLzC+6ngKpC5AUEgQC44voMKsChSbUgy/c372LjFeUhgREZF4xcvNi46FO3Kg/QEOtD9Ap0KdSOickL1X91J+VnnKzijLrsu77F1mvKIwIiIi8VaB1AUYV20cf3f9m25Fu+Hi6MKmC5soMbUEZWeUZcL+CVx/dN3eZcZ5CiMiIhLveXt4M6byGM58foZP83+Ko8WRTRc20XF1R9KOSkuJKSUYsXME957ds3epcZLCiIiIyP/LkDgDkz+ZzJnPz/Bj+R8plq4YBga7ruyit29vsvyShZE7R/L8xXN7lxqnKIyIiIj8j/eSvMeXJb9kV9tdXPniCuOqjiN3ytw8CHxAL99eZB+XnXlH52maeStRGBEREfkPaT3T0qlwJw59dohpNaeRJlEaLjy4QJOlTcjzWx6+8v2KDec28Cz4mb1LjbUURkRERMLB0cGRVvlacebzMwz6eBAeLh4cv32c4TuHU2FWBZIOT0qFWRXYcmGLvUuNdRRGREREIsDd2Z1vy3zLhW4XmFNnDq3ytSJtorQEvghkw7kNVJhVgTlH5ti7zFhFYURERCQSkrkno0nuJkyrOY3LX1zmZOeTNMjZgODQYJota8ZPO37SrK7hpDAiIiISRRaLhezJszOv7jy+KPYFAF9u+JLu67oTEhpi5+piPoURERERK3GwODCq0ihGVhwJwC97f6Heonr8fe9vO1cWsymMiIiIWFmP4j2YV3cezg7OLD+1nKxjs1JjXg0tyPcWCiMiIiLRoFGuRmxtvZUq71fBwGDVX6uoMKsCuX7LxfebvmfNmTXceXrH3mXGCJEKI+PGjSNTpky4ublRtGhR9u7d+9Z9ly5dSqFChfDy8iJhwoTky5ePWbNmRbpgERGR2KJYumKsabqGU51P0aVwFzxcPDhx+wQDtw6k2txqpPgpBVl+yUKbP9pw7dE1e5drNxEOIwsWLKBHjx58//33HDx4kLx581KpUiVu3br1xv2TJk3Kt99+y65duzhy5AitW7emdevW/Pnnn1EuXkREJDb4IPkHjK06litfXGFCtQk0z9OcD5J9AMC5++eY5j+NAhMLxNs5SiIcRkaNGkW7du1o3bo1OXLkYMKECbi7uzN16tQ37v/RRx9Ru3ZtfHx8yJIlC926dSNPnjxs3749ysWLiIjEJondEvNZoc+YWXsmp7qc4t6X91jbdC15vPNw88lNys0sx4idI+JdvxKniOwcFBTEgQMH6NOnT9g2BwcHypcvz65du955vGEYbNy4kdOnT/Pjjz++db/nz5/z/Pk/ixAFBAQAEBwcTHBwcERK/k8vz2XNc8qbqa1tR21tW2pv24mLbe3h5EG5jOXY2mIrndd2Zs6xOfT27c3OSzuZXH0ynq6edqnLWm0d3uMtRgTi17Vr10ibNi07d+6kePHiYdu//PJLtmzZwp49e9543MOHD0mbNi3Pnz/H0dGR8ePH06ZNm7dep3///gwYMOC17XPnzsXd3T285YqIiMQahmGw7u46plydwgvjBYmdElM1eVUqJ69MYqfE9i4vUp4+fUqTJk14+PAhnp5vD1YRujMSWYkSJcLf35/Hjx/j5+dHjx49yJw5Mx999NEb9+/Tpw89evQI+zkgIID06dNTsWLF//wyERUcHIyvry8VKlTA2dnZaueV16mtbUdtbVtqb9uJD21djWo0v9qc5n805/yD88y7MY9ld5bRNFdTPi/8OTlS5LBJHdZq65dPNt4lQmEkefLkODo6cvPmzVe237x5k1SpUr31OAcHB95//30A8uXLx8mTJxk6dOhbw4irqyuurq6vbXd2do6WX8DoOq+8Tm1tO2pr21J7205cb+uSmUpyustplpxcwshdI9l/bT9T/KcwxX8KzfM0Z1j5YaRJlOaVYwzDYN3Zdaw+s5ouRbqQPXl2q9QS1bYO77ER6sDq4uJCwYIF8fPzC9sWGhqKn5/fK49t3iU0NPSVPiEiIiLyD2dHZxrlasTeT/eyrfU2amWvBcCsI7PINjYbQ7YNIfBFICGhISw+sZiCkwpSdW5Vxu0bR7mZ5bj44KJ9v0AERfgxTY8ePWjZsiWFChWiSJEijBkzhidPntC6dWsAWrRoQdq0aRk6dCgAQ4cOpVChQmTJkoXnz5+zZs0aZs2axW+//WbdbyIiIhLHWCwWSmUoRakMpdh3dR9d13Vl95XdfLvxWyYfnIybkxun7pwCIKFzQpK5J+PSw0tUnlOZ7a23k8w9mZ2/QfhEOIw0bNiQ27dv069fP27cuEG+fPlYt24d3t7eAFy6dAkHh39uuDx58oROnTpx5coVEiRIQPbs2Zk9ezYNGza03rcQERGJ4wqnLczONjuZe3QuX234igsPLgDg5eZF1yJd6Vq0K89ePKP4lOKcunPKnH6+xQbcnWP+wI9IdWDt0qULXbp0eeNnmzdvfuXnQYMGMWjQoMhcRkRERP7FYrHQNE9TamWvxbh943B2cKZtgbavDAH+s9mflJpail1XdtFocSOWNlyKk4NNxqtEmtamERERiWUSuiTky5Jf8kXxL16biyRHihysbLwSNyc3Vv61kubLmnPi9gk7VRo+CiMiIiJxTMkMJZlXdx4OFgfmH5tPzvE5yTU+FwO3DOTk7ZP2Lu81CiMiIiJxUK3stVjbdC3Vs1XH2cGZ47eP8/3m78kxPgeVZlfi8I3D9i4xjMKIiIhIHFUxS0VWNl7Jrd63mF5zOlWzVsXJwYn1f68n/8T8tFzekksPL9m7TIURERGRuM7LzYuW+VqyuslqTnc5TaNcjTAwmHl4JtnGZuMr3694EPjAbvUpjIiIiMQjmZNkZl7deez9dC8fZvyQ5yHPGb5zOGvPrLVbTTF7rI+IiIhEi8JpC7Op5SbWnFnDvGPzaJjLfvN/KYyIiIjEUxaLhWrZqlEtWzW71qHHNCIiImJXCiMiIiJiVwojIiIiYlcKIyIiImJXCiMiIiJiVwojIiIiYlcKIyIiImJXCiMiIiJiVwojIiIiYlcKIyIiImJXCiMiIiJiVwojIiIiYlcKIyIiImJXsWLVXsMwAAgICLDqeYODg3n69CkBAQE4Oztb9dzyKrW17aitbUvtbTtqa9uxVlu//Hf75b/jbxMrwsijR48ASJ8+vZ0rERERkYh69OgRiRMnfuvnFuNdcSUGCA0N5dq1ayRKlAiLxWK18wYEBJA+fXouX76Mp6en1c4rr1Nb247a2rbU3rajtrYda7W1YRg8evSINGnS4ODw9p4hseLOiIODA+nSpYu283t6euoX20bU1rajtrYttbftqK1txxpt/V93RF5SB1YRERGxK4URERERsat4HUZcXV35/vvvcXV1tXcpcZ7a2nbU1ral9rYdtbXt2LqtY0UHVhEREYm74vWdEREREbE/hRERERGxK4URERERsSuFEREREbGreB1Gxo0bR6ZMmXBzc6No0aLs3bvX3iXFekOHDqVw4cIkSpSIlClTUqtWLU6fPv3KPoGBgXTu3JlkyZLh4eFB3bp1uXnzpp0qjhuGDRuGxWKhe/fuYdvUztZ19epVmjVrRrJkyUiQIAG5c+dm//79YZ8bhkG/fv1InTo1CRIkoHz58pw5c8aOFcdOISEh9O3bl/fee48ECRKQJUsWfvjhh1fWNlFbR87WrVupUaMGadKkwWKxsHz58lc+D0+73rt3j6ZNm+Lp6YmXlxdt27bl8ePHUS/OiKfmz59vuLi4GFOnTjWOHz9utGvXzvDy8jJu3rxp79JitUqVKhnTpk0zjh07Zvj7+xtVq1Y1MmTIYDx+/Dhsnw4dOhjp06c3/Pz8jP379xvFihUzSpQoYceqY7e9e/camTJlMvLkyWN069YtbLva2Xru3btnZMyY0WjVqpWxZ88e49y5c8aff/5pnD17NmyfYcOGGYkTJzaWL19uHD582Pjkk0+M9957z3j27JkdK499Bg8ebCRLlsxYtWqVcf78eWPRokWGh4eH8fPPP4fto7aOnDVr1hjffvutsXTpUgMwli1b9srn4WnXypUrG3nz5jV2795tbNu2zXj//feNxo0bR7m2eBtGihQpYnTu3Dns55CQECNNmjTG0KFD7VhV3HPr1i0DMLZs2WIYhmE8ePDAcHZ2NhYtWhS2z8mTJw3A2LVrl73KjLUePXpkZM2a1fD19TU+/PDDsDCidraur776yihVqtRbPw8NDTVSpUpl/PTTT2HbHjx4YLi6uhrz5s2zRYlxRrVq1Yw2bdq8sq1OnTpG06ZNDcNQW1vL/4aR8LTriRMnDMDYt29f2D5r1641LBaLcfXq1SjVEy8f0wQFBXHgwAHKly8fts3BwYHy5cuza9cuO1YW9zx8+BCApEmTAnDgwAGCg4Nfafvs2bOTIUMGtX0kdO7cmWrVqr3SnqB2trYVK1ZQqFAh6tevT8qUKcmfPz+TJ08O+/z8+fPcuHHjlfZOnDgxRYsWVXtHUIkSJfDz8+Ovv/4C4PDhw2zfvp0qVaoAauvoEp523bVrF15eXhQqVChsn/Lly+Pg4MCePXuidP1YsVCetd25c4eQkBC8vb1f2e7t7c2pU6fsVFXcExoaSvfu3SlZsiS5cuUC4MaNG7i4uODl5fXKvt7e3ty4ccMOVcZe8+fP5+DBg+zbt++1z9TO1nXu3Dl+++03evTowTfffMO+ffvo2rUrLi4utGzZMqxN3/TfFLV3xHz99dcEBASQPXt2HB0dCQkJYfDgwTRt2hRAbR1NwtOuN27cIGXKlK987uTkRNKkSaPc9vEyjIhtdO7cmWPHjrF9+3Z7lxLnXL58mW7duuHr64ubm5u9y4nzQkNDKVSoEEOGDAEgf/78HDt2jAkTJtCyZUs7Vxe3LFy4kDlz5jB37lxy5syJv78/3bt3J02aNGrrOCxePqZJnjw5jo6Or40suHnzJqlSpbJTVXFLly5dWLVqFZs2bSJdunRh21OlSkVQUBAPHjx4ZX+1fcQcOHCAW7duUaBAAZycnHBycmLLli388ssvODk54e3trXa2otSpU5MjR45Xtvn4+HDp0iWAsDbVf1Oirnfv3nz99dc0atSI3Llz07x5c7744guGDh0KqK2jS3jaNVWqVNy6deuVz1+8eMG9e/ei3PbxMoy4uLhQsGBB/Pz8wraFhobi5+dH8eLF7VhZ7GcYBl26dGHZsmVs3LiR995775XPCxYsiLOz8yttf/r0aS5duqS2j4By5cpx9OhR/P39w16FChWiadOmYe/VztZTsmTJ14ao//XXX2TMmBGA9957j1SpUr3S3gEBAezZs0ftHUFPnz7FweHVf5ocHR0JDQ0F1NbRJTztWrx4cR48eMCBAwfC9tm4cSOhoaEULVo0agVEqftrLDZ//nzD1dXVmD59unHixAmjffv2hpeXl3Hjxg17lxardezY0UicOLGxefNm4/r162Gvp0+fhu3ToUMHI0OGDMbGjRuN/fv3G8WLFzeKFy9ux6rjhn+PpjEMtbM17d2713BycjIGDx5snDlzxpgzZ47h7u5uzJ49O2yfYcOGGV5eXsYff/xhHDlyxKhZs6aGm0ZCy5YtjbRp04YN7V26dKmRPHly48svvwzbR20dOY8ePTIOHTpkHDp0yACMUaNGGYcOHTIuXrxoGEb42rVy5cpG/vz5jT179hjbt283smbNqqG9UTV27FgjQ4YMhouLi1GkSBFj9+7d9i4p1gPe+Jo2bVrYPs+ePTM6depkJEmSxHB3dzdq165tXL9+3X5FxxH/G0bUzta1cuVKI1euXIarq6uRPXt2Y9KkSa98HhoaavTt29fw9vY2XF1djXLlyhmnT5+2U7WxV0BAgNGtWzcjQ4YMhpubm5E5c2bj22+/NZ4/fx62j9o6cjZt2vTG/z63bNnSMIzwtevdu3eNxo0bGx4eHoanp6fRunVr49GjR1GuzWIY/5rWTkRERMTG4mWfEREREYk5FEZERETErhRGRERExK4URkRERMSuFEZERETErhRGRERExK4URkRERMSuFEZERETErhRGRERExK4URkRERMSuFEZERETErhRGRERExK7+D4hxUjn5vupbAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from matplotlib import pyplot as plt\n",
    "plt.plot(train_losses,color = 'red',label = 'train_loss')\n",
    "plt.plot(valid_losses,color = 'green',label = 'valid_loss')\n",
    "plt.grid()\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe633601",
   "metadata": {
    "papermill": {
     "duration": 0.039704,
     "end_time": "2024-03-23T04:54:53.306085",
     "exception": false,
     "start_time": "2024-03-23T04:54:53.266381",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n",
    "   Saving The Models\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "d8661a60",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:54:53.391668Z",
     "iopub.status.busy": "2024-03-23T04:54:53.390806Z",
     "iopub.status.idle": "2024-03-23T04:54:53.746493Z",
     "shell.execute_reply": "2024-03-23T04:54:53.745483Z"
    },
    "papermill": {
     "duration": 0.40336,
     "end_time": "2024-03-23T04:54:53.748747",
     "exception": false,
     "start_time": "2024-03-23T04:54:53.345387",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<All keys matched successfully>"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#load weights of best model\n",
    "path = 'saved_weights.pt'\n",
    "model.load_state_dict(torch.load(path))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d16265b",
   "metadata": {
    "papermill": {
     "duration": 0.039504,
     "end_time": "2024-03-23T04:54:53.831572",
     "exception": false,
     "start_time": "2024-03-23T04:54:53.792068",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n",
    "   Prediction\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "3739b020",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:54:53.917407Z",
     "iopub.status.busy": "2024-03-23T04:54:53.917009Z",
     "iopub.status.idle": "2024-03-23T04:54:54.701464Z",
     "shell.execute_reply": "2024-03-23T04:54:54.700647Z"
    },
    "papermill": {
     "duration": 0.830126,
     "end_time": "2024-03-23T04:54:54.703882",
     "exception": false,
     "start_time": "2024-03-23T04:54:53.873756",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# get predictions for test data\n",
    "with torch.no_grad():\n",
    "  preds = model(test_seq.to(device), test_mask.to(device))\n",
    "  preds = preds.detach().cpu().numpy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "2c9bbe15",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:54:54.788296Z",
     "iopub.status.busy": "2024-03-23T04:54:54.787563Z",
     "iopub.status.idle": "2024-03-23T04:54:54.792100Z",
     "shell.execute_reply": "2024-03-23T04:54:54.791169Z"
    },
    "papermill": {
     "duration": 0.048424,
     "end_time": "2024-03-23T04:54:54.794192",
     "exception": false,
     "start_time": "2024-03-23T04:54:54.745768",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import classification_report,accuracy_score"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71550578",
   "metadata": {
    "papermill": {
     "duration": 0.042277,
     "end_time": "2024-03-23T04:54:54.880545",
     "exception": false,
     "start_time": "2024-03-23T04:54:54.838268",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n",
    "   Evaluation\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "7aa2c227",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:54:54.972746Z",
     "iopub.status.busy": "2024-03-23T04:54:54.972385Z",
     "iopub.status.idle": "2024-03-23T04:54:55.003238Z",
     "shell.execute_reply": "2024-03-23T04:54:55.002165Z"
    },
    "papermill": {
     "duration": 0.083641,
     "end_time": "2024-03-23T04:54:55.005606",
     "exception": false,
     "start_time": "2024-03-23T04:54:54.921965",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification Report:\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.88      0.92      0.90       370\n",
      "           1       0.93      0.91      0.92       481\n",
      "\n",
      "    accuracy                           0.91       851\n",
      "   macro avg       0.91      0.91      0.91       851\n",
      "weighted avg       0.91      0.91      0.91       851\n",
      "\n",
      "Accuracy: 0.9106933019976499\n"
     ]
    }
   ],
   "source": [
    "pred = np.argmax(preds, axis = 1)\n",
    "# Calculate classification report and accuracy\n",
    "report = classification_report(test_y, pred, output_dict=True)\n",
    "accuracy = accuracy_score(test_y, pred)\n",
    "\n",
    "# Print classification report\n",
    "print(\"Classification Report:\")\n",
    "print(classification_report(test_y, pred))\n",
    "\n",
    "# Print accuracy\n",
    "print(\"Accuracy:\", accuracy)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "6352f16b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:54:55.102765Z",
     "iopub.status.busy": "2024-03-23T04:54:55.102023Z",
     "iopub.status.idle": "2024-03-23T04:54:55.118094Z",
     "shell.execute_reply": "2024-03-23T04:54:55.116741Z"
    },
    "papermill": {
     "duration": 0.064519,
     "end_time": "2024-03-23T04:54:55.120576",
     "exception": false,
     "start_time": "2024-03-23T04:54:55.056057",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Metrics DataFrame:\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>precision</th>\n",
       "      <th>recall</th>\n",
       "      <th>f1-score</th>\n",
       "      <th>support</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.882812</td>\n",
       "      <td>0.916216</td>\n",
       "      <td>0.899204</td>\n",
       "      <td>370.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.933619</td>\n",
       "      <td>0.906445</td>\n",
       "      <td>0.919831</td>\n",
       "      <td>481.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>accuracy</th>\n",
       "      <td>0.910693</td>\n",
       "      <td>0.910693</td>\n",
       "      <td>0.910693</td>\n",
       "      <td>0.910693</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>macro avg</th>\n",
       "      <td>0.908216</td>\n",
       "      <td>0.911331</td>\n",
       "      <td>0.909518</td>\n",
       "      <td>851.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>weighted avg</th>\n",
       "      <td>0.911529</td>\n",
       "      <td>0.910693</td>\n",
       "      <td>0.910863</td>\n",
       "      <td>851.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              precision    recall  f1-score     support\n",
       "0              0.882812  0.916216  0.899204  370.000000\n",
       "1              0.933619  0.906445  0.919831  481.000000\n",
       "accuracy       0.910693  0.910693  0.910693    0.910693\n",
       "macro avg      0.908216  0.911331  0.909518  851.000000\n",
       "weighted avg   0.911529  0.910693  0.910863  851.000000"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a DataFrame to store metrics\n",
    "metrics_df = pd.DataFrame(report).transpose()\n",
    "\n",
    "# Add accuracy to the DataFrame\n",
    "metrics_df.loc['accuracy'] = accuracy\n",
    "\n",
    "# Display the DataFrame\n",
    "print(\"\\nMetrics DataFrame:\")\n",
    "metrics_df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "53907e23",
   "metadata": {
    "papermill": {
     "duration": 0.042421,
     "end_time": "2024-03-23T04:54:55.206534",
     "exception": false,
     "start_time": "2024-03-23T04:54:55.164113",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"color:white; background-color:#5642C5; padding: 10px; border-radius: 15px; font-size: 150%; font-family: Verdana; text-align:center; -webkit-text-stroke-width: 1px; -webkit-text-stroke-color: black; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.7);\">\n",
    "   Prediction Sample\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "cdfde256",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-23T04:54:55.289177Z",
     "iopub.status.busy": "2024-03-23T04:54:55.288499Z",
     "iopub.status.idle": "2024-03-23T04:54:55.295118Z",
     "shell.execute_reply": "2024-03-23T04:54:55.294295Z"
    },
    "papermill": {
     "duration": 0.04921,
     "end_time": "2024-03-23T04:54:55.297007",
     "exception": false,
     "start_time": "2024-03-23T04:54:55.247797",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['negative', 'negative', 'True', 'True', 'negative', 'negative',\n",
       "       'True', 'True', 'True', 'negative', 'negative', 'negative', 'True',\n",
       "       'negative', 'negative', 'True', 'negative', 'True', 'negative',\n",
       "       'True', 'True', 'negative', 'negative', 'negative', 'True',\n",
       "       'negative', 'True', 'True', 'negative', 'True', 'negative',\n",
       "       'negative', 'negative', 'negative', 'negative', 'negative',\n",
       "       'negative', 'negative', 'True', 'True', 'negative', 'True', 'True',\n",
       "       'True', 'True', 'negative', 'negative', 'True', 'True', 'True'],\n",
       "      dtype='<U8')"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def convert_predictions(binary_predictions):\n",
    "    return np.where(binary_predictions == 0, \"negative\", \"True\")\n",
    "pred_mapped = convert_predictions(pred)\n",
    "pred_mapped[:50]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a149719d",
   "metadata": {
    "papermill": {
     "duration": 0.040348,
     "end_time": "2024-03-23T04:54:55.378252",
     "exception": false,
     "start_time": "2024-03-23T04:54:55.337904",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kaggle": {
   "accelerator": "nvidiaTeslaT4",
   "dataSources": [
    {
     "datasetId": 4652006,
     "sourceId": 7917046,
     "sourceType": "datasetVersion"
    }
   ],
   "dockerImageVersionId": 30674,
   "isGpuEnabled": true,
   "isInternetEnabled": true,
   "language": "python",
   "sourceType": "notebook"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 638.767468,
   "end_time": "2024-03-23T04:54:58.165001",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2024-03-23T04:44:19.397533",
   "version": "2.5.0"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {
     "02850ce23e3e43848632010fda5cc3c4": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "03f95cb754fc4596b4a3a750669cba85": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_fc8154a62739469695e3a1c7ff696443",
       "placeholder": "​",
       "style": "IPY_MODEL_f3c3ebdffa09448ab4427f1e43579934",
       "value": "tokenizer_config.json: 100%"
      }
     },
     "144eaf7d596a4613907d07e2cc728277": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_4a8e92192c67427a9885fa1227590560",
        "IPY_MODEL_3e4390338fc449d0bf28499d191bb6e7",
        "IPY_MODEL_948f825db2c04363be87741957ee21db"
       ],
       "layout": "IPY_MODEL_3da9aed258184e6a9773fa8028e31f1b"
      }
     },
     "16414b8e0bfe43d48f896da9c6d36b9b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "169cb9d3cd3343c9bc1009899014e4f4": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "17c634ac91934f62b240677da2ccd5c9": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "1921f23a960848b7897f7bf50acb5731": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_5d7759f9f82c445f9aefe4a6dcb1b2f5",
       "max": 48.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_6f794145184440b680fa284ef98ceca2",
       "value": 48.0
      }
     },
     "19238d8576b64b6a98f977da1b976471": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "1968654013844d00be0fdebdc11fa4ba": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "1a9f427d528547d6aa4b37a5d7853c08": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_242f3902a8ff4a0fabee6a4384099660",
        "IPY_MODEL_1d4e9866e08b4fc39b2895e0786d96b3",
        "IPY_MODEL_6c35ed73419244198f32c4a7b600ddec"
       ],
       "layout": "IPY_MODEL_64c00285c2cc40419b31bc97210e185e"
      }
     },
     "1c2113a50845402d89afb78818bc050e": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "1d4e9866e08b4fc39b2895e0786d96b3": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_02850ce23e3e43848632010fda5cc3c4",
       "max": 440449768.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_93b16daa1fa74084bcd1b89f6ac6fd8d",
       "value": 440449768.0
      }
     },
     "242f3902a8ff4a0fabee6a4384099660": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_e029547348f74eef8a944c32ddb277c6",
       "placeholder": "​",
       "style": "IPY_MODEL_43694033235f4e6a847039580c9e0f28",
       "value": "model.safetensors: 100%"
      }
     },
     "2a5ab49a6df64823b0883dc5bd06fb70": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "2b3c7b5d122f4af1903b46b26923cd89": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "38e75407756e487fa2c196720b337f74": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "3da9aed258184e6a9773fa8028e31f1b": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "3e4390338fc449d0bf28499d191bb6e7": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_2b3c7b5d122f4af1903b46b26923cd89",
       "max": 570.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_169cb9d3cd3343c9bc1009899014e4f4",
       "value": 570.0
      }
     },
     "43694033235f4e6a847039580c9e0f28": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "498caba0dc9449f5a2ca68bc39aaaab4": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "4a8e92192c67427a9885fa1227590560": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_1968654013844d00be0fdebdc11fa4ba",
       "placeholder": "​",
       "style": "IPY_MODEL_abf0be9bc134453095d151726790fabc",
       "value": "config.json: 100%"
      }
     },
     "50bb2e3d0c374dc08eeba5bf1485f664": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_e9181ed9f803470485fdc236b1bf7273",
       "max": 466062.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_16414b8e0bfe43d48f896da9c6d36b9b",
       "value": 466062.0
      }
     },
     "5a2d3c2219c64d96b69bd5d7779c4672": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "5d0ff3235895462a82ac442413782f4e": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "5d7759f9f82c445f9aefe4a6dcb1b2f5": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "5dc8c81b92824528866f5a99ad265907": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "63b2a90ca3a14c2aafd917ad01a57c75": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_8a98cd6254104ef1a2b809f7a1f728a3",
       "placeholder": "​",
       "style": "IPY_MODEL_b0185c3767e146a7aaf519af49138255",
       "value": " 466k/466k [00:00&lt;00:00, 25.9MB/s]"
      }
     },
     "64c00285c2cc40419b31bc97210e185e": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "66aed60cdd2d40ddabb3035d2b775f04": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_a2503dc3793c494c97bbbe28ba810c7f",
        "IPY_MODEL_fe7175d6f5d64068a5020007324680bf",
        "IPY_MODEL_8a0c442bdb304c11b5e0d3307ba58d72"
       ],
       "layout": "IPY_MODEL_d919a9b02b8a48ab98e5eac1beb04c34"
      }
     },
     "66cc46a40cb7426b95b2cc796ab47ff6": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "6c35ed73419244198f32c4a7b600ddec": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_17c634ac91934f62b240677da2ccd5c9",
       "placeholder": "​",
       "style": "IPY_MODEL_dcedf5e3b37a437db48d27ca97df468b",
       "value": " 440M/440M [00:02&lt;00:00, 200MB/s]"
      }
     },
     "6d440b650755409d863c9763821c9416": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "6f794145184440b680fa284ef98ceca2": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "71270715acf247ea93e6b7cc2079a308": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "76131eb1aea6496399f96fce1fc94c48": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_03f95cb754fc4596b4a3a750669cba85",
        "IPY_MODEL_1921f23a960848b7897f7bf50acb5731",
        "IPY_MODEL_cb1224af1a2f4b40ba0ca5662f1352df"
       ],
       "layout": "IPY_MODEL_b66e551d43f1446096384d215dfa9bbf"
      }
     },
     "8a0c442bdb304c11b5e0d3307ba58d72": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_5dc8c81b92824528866f5a99ad265907",
       "placeholder": "​",
       "style": "IPY_MODEL_2a5ab49a6df64823b0883dc5bd06fb70",
       "value": " 232k/232k [00:00&lt;00:00, 4.02MB/s]"
      }
     },
     "8a98cd6254104ef1a2b809f7a1f728a3": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "93b16daa1fa74084bcd1b89f6ac6fd8d": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "948f825db2c04363be87741957ee21db": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_1c2113a50845402d89afb78818bc050e",
       "placeholder": "​",
       "style": "IPY_MODEL_5a2d3c2219c64d96b69bd5d7779c4672",
       "value": " 570/570 [00:00&lt;00:00, 38.8kB/s]"
      }
     },
     "a2503dc3793c494c97bbbe28ba810c7f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_38e75407756e487fa2c196720b337f74",
       "placeholder": "​",
       "style": "IPY_MODEL_5d0ff3235895462a82ac442413782f4e",
       "value": "vocab.txt: 100%"
      }
     },
     "a912702fa7494b6a8e6b69631a66d798": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_f5bf791add914e37869014ab6bfd82c1",
       "placeholder": "​",
       "style": "IPY_MODEL_71270715acf247ea93e6b7cc2079a308",
       "value": "tokenizer.json: 100%"
      }
     },
     "a92ba54e480d4442b3fb05ab47203e99": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "abf0be9bc134453095d151726790fabc": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "b0185c3767e146a7aaf519af49138255": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "b66e551d43f1446096384d215dfa9bbf": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "ba028e93bc79411db699ea7a51fda4b0": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_a912702fa7494b6a8e6b69631a66d798",
        "IPY_MODEL_50bb2e3d0c374dc08eeba5bf1485f664",
        "IPY_MODEL_63b2a90ca3a14c2aafd917ad01a57c75"
       ],
       "layout": "IPY_MODEL_a92ba54e480d4442b3fb05ab47203e99"
      }
     },
     "cb1224af1a2f4b40ba0ca5662f1352df": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_66cc46a40cb7426b95b2cc796ab47ff6",
       "placeholder": "​",
       "style": "IPY_MODEL_6d440b650755409d863c9763821c9416",
       "value": " 48.0/48.0 [00:00&lt;00:00, 4.12kB/s]"
      }
     },
     "d919a9b02b8a48ab98e5eac1beb04c34": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "dcedf5e3b37a437db48d27ca97df468b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "e029547348f74eef8a944c32ddb277c6": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "e9181ed9f803470485fdc236b1bf7273": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "f3c3ebdffa09448ab4427f1e43579934": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "f5bf791add914e37869014ab6bfd82c1": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "fc8154a62739469695e3a1c7ff696443": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "fe7175d6f5d64068a5020007324680bf": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_498caba0dc9449f5a2ca68bc39aaaab4",
       "max": 231508.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_19238d8576b64b6a98f977da1b976471",
       "value": 231508.0
      }
     }
    },
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}