Fine_Tuning_Whisper_Small_Model.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZEBGrHWampEz"
      },
      "source": [
        "# **Hugging Face Whisper Small Model Train and Fine-Tuning**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "I2msSW9JmpE0"
      },
      "outputs": [],
      "source": [
        "'''\n",
        "prevent the Colab runtime from disconnecting & automatically click the connect button in Google Colab at regular intervals (every minute)\n",
        "\n",
        "copy below javascript --> in colab, right click and select Inspect --> paste in console\n",
        "\n",
        "\n",
        "function ConnectButton(){\n",
        "     console.log(\"Connect pushed\");\n",
        "     document.querySelector(\"#top-toolbar > colab-connect-button\").shadowRoot.querySelector(\"#connect\").click()\n",
        "}\n",
        "setInterval(ConnectButton,60000);\n",
        "'''"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bYQaJPUs4-C_"
      },
      "source": [
        "## **Hugging Face Login**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 145,
          "referenced_widgets": [
            "a1ac2cd10bb8496ea6fd27f4e7dfbbca",
            "4192c621af5443ac83121877d8c751e0",
            "0b6648bdddab4a21acc8a02f5c11cb8f",
            "6bb97fc329a34c5e8d6974560a1913a2",
            "b1944f08aa8a4acb92fa5ba1db3d686f",
            "22d626a6b2844c909f0b3ab7d42a5131",
            "a462186102f24b9089e1a018628c4a0e",
            "8d095f4f9e034a448d24a70d3dd94c96",
            "3861f55c51964554bcc0d58c36f1b87e",
            "71a940b0f38a4b11a268caa438863222",
            "99ee2015a2e24292b48a63ba8ffec91b",
            "59a768ed461b4f61b68f74ea0be0831e",
            "50ad94d747214dd68342a45cb6e72f25",
            "9687a6ee5ec5403a8859fee09d8a34da",
            "3b34b40c68f74a6f87ee9e37930bdbc9",
            "aa2fff010eb6408bb94054de5bf37493",
            "1ea4ec48037f43b39cc3b84f235cc6d3",
            "25ea8a8ffa4b4e8b99711eb90ef7350b",
            "32127b53a9714b2c8ee489646d1e394f",
            "8630ea5ec783498da1d7d1269627f645",
            "ca7278cf17e8473f81f4a9777af7eced",
            "490ab7da03ec45b38425cdb2c9652bfe",
            "c19ea6a137844b61964bcc7186fd6631",
            "fdf36fb6679542e491e2420b9adc9307",
            "0de61b8a9ede4ebc82176b876cb8c445",
            "d52f1bedfae84c0d85a3f7c55180510e",
            "839543a6249e4ee9b66a7a7ce8962348",
            "699e3b328cbd42e7b9dab8f0d2a2daba",
            "e13593a166cb43e198f2bd277d6e5474",
            "26f39e779686424b9eef35687c1b3c5b",
            "27e11dacb88148dbbb38498ddf1545c8",
            "a63eec544b9d4e3489678be65b14c888"
          ]
        },
        "id": "-epdQXEumpE1",
        "outputId": "4b9b8398-dc3e-4018-854d-71841ad6fb08"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "a1ac2cd10bb8496ea6fd27f4e7dfbbca",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "VBox(children=(HTML(value='<center> <img\\nsrc=https://huggingface.co/front/assets/huggingface_logo-noborder.sv…"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "# access and work with models that are stored on the Hugging Face Hub.\n",
        "# generate access token with 'Write' access from your Huggingface account & use it to login in colab.\n",
        "\n",
        "from huggingface_hub import notebook_login\n",
        "\n",
        "notebook_login()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uuGNTxzc4-DB"
      },
      "source": [
        "## **Install Required Libraries**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-KjV_vjrmpE2"
      },
      "outputs": [],
      "source": [
        "!pip install --upgrade pip\n",
        "!pip install --upgrade datasets transformers accelerate soundfile librosa evaluate jiwer tensorboard\n",
        "\n",
        "'''\n",
        "datasets - specifically designed for loading datasets from the Hugging Face Datasets Hub,\n",
        "           which is a centralized repository of various datasets for natural language processing tasks.\n",
        "\n",
        "Transformers  : Access to large collection of pre-trained models for various NLP tasks\n",
        "\n",
        "accelerate - it is a Python library developed by Hugging Face, purpose is to speed up the training and inference process of deep learning models.\n",
        "\n",
        "soundfile - helps in reading and writing sound files\n",
        "\n",
        "librosa - It can be helpful for extracting features from audio\n",
        "\n",
        "evaluate - evaluating and comparing models and reporting their performance easier and more standardized\n",
        "\n",
        "jiwer - calculating the Word Error Rate (WER)\n",
        "\n",
        "tensorboard - It is a visualization tool included with TensorFlow. It helps in monitoring and debugging your machine learning models.\n",
        "'''"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "qiiufpF2Aa5B"
      },
      "outputs": [],
      "source": [
        "import warnings\n",
        "warnings.filterwarnings('ignore')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DAgBgoZi4-DC"
      },
      "source": [
        "## **Data Collection**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KNiXl6QDmpE4",
        "outputId": "a4452e11-3ad0-46f2-9863-ca6b236d5a1b"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "DatasetDict({\n",
              "    \n",
              "})"
            ]
          },
          "execution_count": 45,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# DatasetDict - It is designed to organize and manage multiple datasets effectively - manage train and test dataset are clear\n",
        "\n",
        "from datasets import load_dataset, DatasetDict\n",
        "\n",
        "common_voice = DatasetDict()    # create empty DatasetDict\n",
        "\n",
        "common_voice"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GTVLNU1HmpE4",
        "outputId": "5b3e6fb2-6257-49e7-efed-420f5b87e76d"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "Dataset({\n",
              "    features: ['client_id', 'path', 'audio', 'sentence', 'up_votes', 'down_votes', 'age', 'gender', 'accent', 'locale', 'segment'],\n",
              "    num_rows: 3927\n",
              "})"
            ]
          },
          "execution_count": 46,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "common_voice[\"train\"] = load_dataset(\"mozilla-foundation/common_voice_11_0\", \"mr\", split=\"train+validation\", use_auth_token=True)\n",
        "common_voice[\"train\"]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hHq-XYKxmpE5",
        "outputId": "349299e7-67d9-4de3-f45e-71efa002ed67"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "Dataset({\n",
              "    features: ['client_id', 'path', 'audio', 'sentence', 'up_votes', 'down_votes', 'age', 'gender', 'accent', 'locale', 'segment'],\n",
              "    num_rows: 1816\n",
              "})"
            ]
          },
          "execution_count": 47,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "common_voice[\"test\"] = load_dataset(\"mozilla-foundation/common_voice_11_0\", \"mr\", split=\"test\", use_auth_token=True)\n",
        "common_voice[\"test\"]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YpdVSHcDmpE5",
        "outputId": "cf472af4-f31e-4267-a359-50c123d44679"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "DatasetDict({\n",
            "    train: Dataset({\n",
            "        features: ['client_id', 'path', 'audio', 'sentence', 'up_votes', 'down_votes', 'age', 'gender', 'accent', 'locale', 'segment'],\n",
            "        num_rows: 3927\n",
            "    })\n",
            "    test: Dataset({\n",
            "        features: ['client_id', 'path', 'audio', 'sentence', 'up_votes', 'down_votes', 'age', 'gender', 'accent', 'locale', 'segment'],\n",
            "        num_rows: 1816\n",
            "    })\n",
            "})\n"
          ]
        }
      ],
      "source": [
        "print(common_voice)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rRblXznlmpE6",
        "outputId": "14778b3f-953a-4b75-bc61-5fd83d8f5668"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "DatasetDict({\n",
              "    train: Dataset({\n",
              "        features: ['audio', 'sentence'],\n",
              "        num_rows: 3927\n",
              "    })\n",
              "    test: Dataset({\n",
              "        features: ['audio', 'sentence'],\n",
              "        num_rows: 1816\n",
              "    })\n",
              "})"
            ]
          },
          "execution_count": 49,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "common_voice = common_voice.remove_columns([\"accent\", \"age\", \"client_id\", \"down_votes\", \"gender\", \"locale\", \"path\", \"segment\", \"up_votes\"])\n",
        "common_voice"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6pLq1UZ_4-DI"
      },
      "source": [
        "## **Data preprocessing**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rEJnDOtLCeIR"
      },
      "source": [
        "### **Feature Extractor**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lPjIYpvb4-DJ",
        "outputId": "cd225999-d097-403c-92ca-626341e0d629"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "WhisperFeatureExtractor {\n",
              "  \"chunk_length\": 30,\n",
              "  \"feature_extractor_type\": \"WhisperFeatureExtractor\",\n",
              "  \"feature_size\": 80,\n",
              "  \"hop_length\": 160,\n",
              "  \"n_fft\": 400,\n",
              "  \"n_samples\": 480000,\n",
              "  \"nb_max_frames\": 3000,\n",
              "  \"padding_side\": \"right\",\n",
              "  \"padding_value\": 0.0,\n",
              "  \"processor_class\": \"WhisperProcessor\",\n",
              "  \"return_attention_mask\": false,\n",
              "  \"sampling_rate\": 16000\n",
              "}"
            ]
          },
          "execution_count": 50,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# WhisperFeatureExtractor - It takes raw audio input and transforms it into a format that the model can understand the patterns\n",
        "\n",
        "from transformers import WhisperFeatureExtractor\n",
        "\n",
        "feature_extractor = WhisperFeatureExtractor.from_pretrained(\"openai/whisper-small\")\n",
        "feature_extractor"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LHU5YtvhCpJX"
      },
      "source": [
        "### **Tokenizer - Text Encoding and Decoding**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "m5pQWDHYmpE8"
      },
      "outputs": [],
      "source": [
        "# WhisperTokenizer -  The process of splitting long text into small chunks, where each chunk is called a token\n",
        "#                     and is assigned a numerical representation, is essential for the computer to understand and process the information.\n",
        "\n",
        "from transformers import WhisperTokenizer\n",
        "\n",
        "tokenizer = WhisperTokenizer.from_pretrained(\"openai/whisper-small\", language=\"Marathi\", task=\"transcribe\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ymGi54PumpE9",
        "outputId": "c0da3d35-e523-41f4-c484-fd9ea211f311"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Text          : आईचे आजारपण वाढत चालले, तसतशी मथीही नीट खातपीतनाशी झाली.\n",
            "Encode Text   : [50258, 50320, 50359, 50363, 3941, 228, 3941, 230, 3941, 248, 21981, 8485, 228, 3941, 250, 17937, 25411, 3941, 103, 3941, 96, 8485, 113, 17937, 3941, 95, 36158, 8485, 248, 17937, 46758, 46758, 21981, 11, 8485, 97, 45938, 36158, 3941, 114, 31881, 48449, 3941, 98, 31881, 44500, 31881, 8485, 101, 31881, 3941, 253, 8485, 244, 17937, 36158, 3941, 103, 31881, 36158, 35082, 17937, 3941, 114, 31881, 8485, 251, 17937, 46758, 31881, 13, 50257]\n",
            "Decode s.token: <|startoftranscript|><|mr|><|transcribe|><|notimestamps|>आईचे आजारपण वाढत चालले, तसतशी मथीही नीट खातपीतनाशी झाली.<|endoftext|>\n",
            "Decoded Text  : आईचे आजारपण वाढत चालले, तसतशी मथीही नीट खातपीतनाशी झाली.\n"
          ]
        }
      ],
      "source": [
        "# input sentence in Marathi taken from the common voice dataset\n",
        "text = common_voice[\"train\"][0][\"sentence\"]\n",
        "\n",
        "# encode (machine readable format) --> text encoded to numerical numbers\n",
        "encoded_text = tokenizer(text).input_ids\n",
        "\n",
        "# decode (human readable format) --> token ID convert to original text with special token\n",
        "decoded_with_special = tokenizer.decode(token_ids = encoded_text,\n",
        "                                        skip_special_tokens = False)\n",
        "\n",
        "# decode the token ID convert to original text w/o special token\n",
        "decoded_text = tokenizer.decode(token_ids = encoded_text,\n",
        "                                skip_special_tokens = True)\n",
        "\n",
        "print(f'Text          : {text}')\n",
        "print(f'Encode Text   : {encoded_text}')\n",
        "print(f'Decode s.token: {decoded_with_special}')\n",
        "print(f'Decoded Text  : {decoded_text}')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Aj_QExIhTKJ1"
      },
      "source": [
        "### **Processor**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ONftOoO_mpE9"
      },
      "outputs": [],
      "source": [
        "# WhisperProcessor - prepares your Marathi audio data so that it can be understood and transcribed by the Whisper model.\n",
        "#                    It's like a translator between your raw audio and the model.\n",
        "\n",
        "from transformers import WhisperProcessor\n",
        "\n",
        "processor = WhisperProcessor.from_pretrained(\"openai/whisper-small\", language=\"Marathi\", task=\"transcribe\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Phc6UszCTPfW"
      },
      "source": [
        "### **Sampling Rate Conversion**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-eyDMYQKLXlg",
        "outputId": "fd5a708a-cd0b-438d-de12-6385d6fcb2a9"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'audio': {'path': '/root/.cache/huggingface/datasets/downloads/extracted/f7e1ef6a2d14f20194999aad5040c5d4bb3ead1377de3e1bbc6e9dba34d18a8a/mr_train_0/common_voice_mr_30585613.mp3',\n",
              "  'array': array([ 1.13686838e-13, -1.42108547e-13, -1.98951966e-13, ...,\n",
              "          4.83472422e-06,  3.54798703e-06,  1.63231743e-06]),\n",
              "  'sampling_rate': 48000},\n",
              " 'sentence': 'आईचे आजारपण वाढत चालले, तसतशी मथीही नीट खातपीतनाशी झाली.'}"
            ]
          },
          "execution_count": 55,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "common_voice[\"train\"][0]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "uCKEZqgFmpE_"
      },
      "outputs": [],
      "source": [
        "'''\n",
        "convert the audio data sampling rate from 48000 HZ to 16000 HZ, ensuring compatibility with downstream tasks.\n",
        "This standardization is essential for consistent processing and analysis in speech to text.\n",
        "'''"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cvZV4zPcmpE_",
        "outputId": "4e87782d-5d80-40e1-c733-c71003c5e159"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "DatasetDict({\n",
              "    train: Dataset({\n",
              "        features: ['audio', 'sentence'],\n",
              "        num_rows: 3927\n",
              "    })\n",
              "    test: Dataset({\n",
              "        features: ['audio', 'sentence'],\n",
              "        num_rows: 1816\n",
              "    })\n",
              "})"
            ]
          },
          "execution_count": 56,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from datasets import Audio\n",
        "\n",
        "common_voice = common_voice.cast_column(\"audio\", Audio(sampling_rate=16000))\n",
        "common_voice"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "d6nsQ9TkmpE_",
        "outputId": "c4949b7a-82ed-482c-e194-afb78a748df9"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{'audio': {'path': '/root/.cache/huggingface/datasets/downloads/extracted/f7e1ef6a2d14f20194999aad5040c5d4bb3ead1377de3e1bbc6e9dba34d18a8a/mr_train_0/common_voice_mr_30585613.mp3', 'array': array([-3.06954462e-12, -3.63797881e-12, -4.54747351e-12, ...,\n",
            "       -7.74800901e-06, -1.74738125e-06,  4.36312439e-06]), 'sampling_rate': 16000}, 'sentence': 'आईचे आजारपण वाढत चालले, तसतशी मथीही नीट खातपीतनाशी झाली.'}\n"
          ]
        }
      ],
      "source": [
        "print(common_voice[\"train\"][0])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ny9olRmrQHvc",
        "outputId": "2eb8b727-0ae5-44a2-87d0-d8a16145fefe"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{'path': '/root/.cache/huggingface/datasets/downloads/extracted/f7e1ef6a2d14f20194999aad5040c5d4bb3ead1377de3e1bbc6e9dba34d18a8a/mr_train_0/common_voice_mr_30585613.mp3', 'array': array([-3.06954462e-12, -3.63797881e-12, -4.54747351e-12, ...,\n",
            "       -7.74800901e-06, -1.74738125e-06,  4.36312439e-06]), 'sampling_rate': 16000}\n"
          ]
        }
      ],
      "source": [
        "print(common_voice[\"train\"][0]['audio'])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QdXUxupnQj3w",
        "outputId": "96e07c46-052d-45ef-cbc0-4f70f1df8196"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "train_path          = /root/.cache/huggingface/datasets/downloads/extracted/f7e1ef6a2d14f20194999aad5040c5d4bb3ead1377de3e1bbc6e9dba34d18a8a/mr_train_0/common_voice_mr_30585613.mp3\n",
            "train_array         = [-3.06954462e-12 -3.63797881e-12 -4.54747351e-12 ... -7.74800901e-06\n",
            " -1.74738125e-06  4.36312439e-06]\n",
            "train_sampling_rate = 16000\n",
            "train_sentence      = आईचे आजारपण वाढत चालले, तसतशी मथीही नीट खातपीतनाशी झाली.\n"
          ]
        }
      ],
      "source": [
        "print(f\"train_path          = {common_voice['train'][0]['audio']['path']}\")\n",
        "print(f\"train_array         = {common_voice['train'][0]['audio']['array']}\")\n",
        "print(f\"train_sampling_rate = {common_voice['train'][0]['audio']['sampling_rate']}\")\n",
        "print(f\"train_sentence      = {common_voice['train'][0]['sentence']}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IbO3InJIMadx"
      },
      "source": [
        "### **Dataset Preparation**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TT8dvBRRUY7e"
      },
      "outputs": [],
      "source": [
        "'''\n",
        "Log-Mel features (audio):\n",
        "\n",
        "Mel refers Mel-frequency, a perceptual scale that mimics how humans hear different pitches.\n",
        "\n",
        "Log-Mel features involve measuring the audio signal's pitches using Mel scale and taking 'logarithm' of those values.\n",
        "\n",
        "widely used in speech and audio processing, particularly in speech recognition.\n",
        "'''"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "OTqW07NNNXz4"
      },
      "outputs": [],
      "source": [
        "common_voice"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5Y5-fiKKmpFD"
      },
      "outputs": [],
      "source": [
        "# Audio and Text Dataset Preprocessing for Machine Learning\n",
        "\n",
        "def dataset_preparation(batch):\n",
        "\n",
        "    # raw audio data transform into log-Mel features\n",
        "    batch[\"input_features\"] = feature_extractor(batch[\"audio\"][\"array\"], sampling_rate=batch[\"audio\"][\"sampling_rate\"]).input_features[0]\n",
        "\n",
        "    # sentence encode to label ids\n",
        "    batch[\"labels\"] = tokenizer(batch[\"sentence\"]).input_ids\n",
        "\n",
        "    return batch"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 255,
          "referenced_widgets": [
            "513f27027f994231b4c4b9ba8bf472b2",
            "1c94523d19184d66bd9c39f930f6618e",
            "bc42c987ab944013a6a98aa60db62fd0",
            "dca1537812504f4d8ae7d5050c84ae31",
            "7278a1f847fc424ab2892842f9fdc06a",
            "a7918c0101af44878dcd28962859d21f",
            "88e4c2db421644a795b16a750bf48519",
            "866cfaace83f4a90ad61e54e9906aebf",
            "bc61ea6a66d74c5fbf2ef30f2020bccb",
            "c1e0a77fce7a4b278b73547b256e979f",
            "eed928a4257e44b8aee370c31a888588",
            "3ac8582c0a47452a940e9a7a068afc45",
            "d094b49bfa6940a585cfa50f1d6a5b9e",
            "ee6d4aee6a0d49c9a785c6e847538e9a",
            "c94ceacb8fed487eb97160bebc2f8df7",
            "46239d7829404095b0babe6da9e29cab",
            "fc124d9887dd42619e18bcd03d3e38a2",
            "e9d7a8432f7b444e92f42ef13eeba855",
            "2cb71c1d8225401d963cffd181fae5ea",
            "3f2e65b6a0ca4c8fa50573d70d6530b6",
            "8adba088bc4441c6a0f76a2ba6a0c61f",
            "ba3cd688de994cb4b29bf15d86723e93"
          ]
        },
        "id": "HWDUAxjVmpFG",
        "outputId": "95f66093-5de4-417d-f67c-058c3ba082ec"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "513f27027f994231b4c4b9ba8bf472b2",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Map:   0%|          | 0/3927 [00:00<?, ? examples/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "3ac8582c0a47452a940e9a7a068afc45",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Map:   0%|          | 0/1816 [00:00<?, ? examples/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/plain": [
              "DatasetDict({\n",
              "    train: Dataset({\n",
              "        features: ['input_features', 'labels'],\n",
              "        num_rows: 3927\n",
              "    })\n",
              "    test: Dataset({\n",
              "        features: ['input_features', 'labels'],\n",
              "        num_rows: 1816\n",
              "    })\n",
              "})"
            ]
          },
          "execution_count": 72,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "common_voice = common_voice.map(dataset_preparation, remove_columns=['audio', 'sentence'], num_proc=1)  # number of parallel processes = 1\n",
        "common_voice"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Q03crv0VxAFj"
      },
      "outputs": [],
      "source": [
        "\"\"\"\n",
        "step 1 [batch - raw audio input]:\n",
        "\n",
        "{'audio': {'path': '/root/.cache/......./mr_train_0/common_voice_mr_30585613.mp3',\n",
        "           'array': array([-3.06954462e-12, -3.63797881e-12, ... 4.36312439e-06]),\n",
        "           'sampling_rate': 16000},\n",
        "\n",
        "'sentence': 'आईचे आजारपण वाढत चालले, तसतशी मथीही नीट खातपीतनाशी झाली.'}\n",
        "\n",
        "===========================================================================================================================\n",
        "\n",
        "step 2 (batch['audio']):\n",
        "\n",
        "{'path': '/root/.cache/......./mr_train_0/common_voice_mr_30585613.mp3',\n",
        " 'array': array([-3.06954462e-12, -3.63797881e-12, ... 4.36312439e-06]),\n",
        " 'sampling_rate': 16000}\n",
        "\n",
        "===========================================================================================================================\n",
        "\n",
        "step 3 (feature extract from raw audio):\n",
        "\n",
        "{'input_features': [array([[-0.57933486, -0.57933486, ..., -0.57933486, -0.57933486],\n",
        "                           ...,\n",
        "                           [-0.57933486, -0.57933486, ..., -0.57933486, -0.57933486]],\n",
        "                   dtype=float32)]}\n",
        "\n",
        "===========================================================================================================================\n",
        "\n",
        "step 3.1 (batch['input_features']):\n",
        "\n",
        "{'audio': {'path': '/root/.cache/......./mr_train_0/common_voice_mr_30585613.mp3',\n",
        "           'array': array([-3.06954462e-12, -3.63797881e-12, ... 4.36312439e-06]),\n",
        "           'sampling_rate': 16000},\n",
        "\n",
        "'sentence': 'आईचे आजारपण वाढत चालले, तसतशी मथीही नीट खातपीतनाशी झाली.',\n",
        "\n",
        "'input_features': array([[-0.57933486, -0.57933486, ..., -0.57933486, -0.57933486],\n",
        "                           ...,\n",
        "                           [-0.57933486, -0.57933486, ..., -0.57933486, -0.57933486]],\n",
        "                   dtype=float32)}\n",
        "\n",
        "===========================================================================================================================\n",
        "\n",
        "step 4 (sentence encode to token ids):\n",
        "\n",
        "{'input_ids'     : [50258, 50320, 50359, 50363, 3941, 228, ... 31881, 8485, 251, 17937, 46758, 31881, 13, 50257],\n",
        " 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}\n",
        "\n",
        "===========================================================================================================================\n",
        "\n",
        "step 5 (batch['labels']):\n",
        "\n",
        "{'audio': {'path': '/root/.cache/......./mr_train_0/common_voice_mr_30585613.mp3',\n",
        "           'array': array([-3.06954462e-12, -3.63797881e-12, ... 4.36312439e-06]),\n",
        "           'sampling_rate': 16000},\n",
        "\n",
        "'sentence': 'आईचे आजारपण वाढत चालले, तसतशी मथीही नीट खातपीतनाशी झाली.',\n",
        "\n",
        "'input_features': array([[-0.57933486, -0.57933486, ..., -0.57933486, -0.57933486],\n",
        "                           ...,\n",
        "                         [-0.57933486, -0.57933486, ..., -0.57933486, -0.57933486]],\n",
        "                   dtype=float32),\n",
        "\n",
        "{'labels': [50258, 50320, 50359, 50363, 3941, 228, ... 31881, 8485, 251, 17937, 46758, 31881, 13, 50257]}\n",
        "\n",
        "===========================================================================================================================\n",
        "\n",
        "Finally columns ['audio', 'sentence'] are removed and only ['input_features', 'labels'] columns are in the custom_voice dataset\n",
        "\n",
        "\"\"\"\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f85LhzSTW8xp"
      },
      "source": [
        "### **Data Collector**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rBvS9EYzmpFI"
      },
      "outputs": [],
      "source": [
        "import torch\n",
        "\n",
        "from dataclasses import dataclass\n",
        "from typing import Any, Dict, List, Union\n",
        "\n",
        "@dataclass\n",
        "class DataCollatorSpeechSeq2SeqWithPadding:\n",
        "    processor: Any\n",
        "\n",
        "    def __call__(self, features: List[Dict[str, Union[List[int], torch.Tensor]]]) -> Dict[str, torch.Tensor]:\n",
        "\n",
        "        # split inputs and labels since they have to be of different lengths and need different padding methods\n",
        "        # first treat the audio inputs by simply returning torch tensors\n",
        "        input_features = [{\"input_features\": i[\"input_features\"]} for i in features]\n",
        "\n",
        "        batch = self.processor.feature_extractor.pad(input_features, return_tensors=\"pt\")\n",
        "\n",
        "        # get the tokenized label sequences\n",
        "        label_features = [{\"input_ids\": i[\"labels\"]} for i in features]\n",
        "\n",
        "        # pad the labels to max length\n",
        "        labels_batch = self.processor.tokenizer.pad(label_features, return_tensors=\"pt\")\n",
        "\n",
        "        # replace padding with -100 to ignore loss correctly\n",
        "        labels = labels_batch[\"input_ids\"].masked_fill(labels_batch.attention_mask.ne(1), -100)\n",
        "\n",
        "        # if bos token is appended in previous tokenization step,\n",
        "        # cut bos token here as it's append later anyways\n",
        "        if (labels[:, 0] == self.processor.tokenizer.bos_token_id).all().cpu().item():\n",
        "            labels = labels[:, 1:]\n",
        "\n",
        "        batch[\"labels\"] = labels\n",
        "\n",
        "        return batch\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7J5F7GPYmpFJ"
      },
      "outputs": [],
      "source": [
        "data_collator = DataCollatorSpeechSeq2SeqWithPadding(processor=processor)\n",
        "data_collator"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 414,
          "referenced_widgets": [
            "960202d69f16416693e57c49b70a9f3c",
            "8fd888770cf84ff38a24d4bc91f40297",
            "2f65ae1a49f34926a6fd9262c59124f7",
            "ff7082c3cb794968a14632843a28572d",
            "3e4d09c1c8ca4709849c7a31e8db058f",
            "1771084de59e4467888a4188e31d51d6",
            "9355ef2b4f8f45318bbe8a5a98214dd1",
            "6fecddbe341947f5be09638760a92072",
            "a6ca5a211b75421cb64618b2a57c1875",
            "498eef9c5b374ffca56a78745a969b65",
            "b0e08f220ac34f36befdb26ad6364497"
          ]
        },
        "id": "1R3A1aPnmpFK",
        "outputId": "18298d77-e737-4e5e-8d39-16906ff4e91a"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "960202d69f16416693e57c49b70a9f3c",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading builder script:   0%|          | 0.00/4.49k [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/plain": [
              "EvaluationModule(name: \"wer\", module_type: \"metric\", features: {'predictions': Value(dtype='string', id='sequence'), 'references': Value(dtype='string', id='sequence')}, usage: \"\"\"\n",
              "Compute WER score of transcribed segments against references.\n",
              "\n",
              "Args:\n",
              "    references: List of references for each speech input.\n",
              "    predictions: List of transcriptions to score.\n",
              "    concatenate_texts (bool, default=False): Whether to concatenate all input texts or compute WER iteratively.\n",
              "\n",
              "Returns:\n",
              "    (float): the word error rate\n",
              "\n",
              "Examples:\n",
              "\n",
              "    >>> predictions = [\"this is the prediction\", \"there is an other sample\"]\n",
              "    >>> references = [\"this is the reference\", \"there is another one\"]\n",
              "    >>> wer = evaluate.load(\"wer\")\n",
              "    >>> wer_score = wer.compute(predictions=predictions, references=references)\n",
              "    >>> print(wer_score)\n",
              "    0.5\n",
              "\"\"\", stored examples: 0)"
            ]
          },
          "execution_count": 73,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "import evaluate\n",
        "\n",
        "metric = evaluate.load(\"wer\")\n",
        "metric"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "pGyYjCqVmpFK"
      },
      "outputs": [],
      "source": [
        "def compute_metrics(pred):\n",
        "\n",
        "    pred_ids = pred.predictions\n",
        "    label_ids = pred.label_ids\n",
        "\n",
        "    # replace -100 with the pad_token_id\n",
        "    label_ids[label_ids == -100] = tokenizer.pad_token_id\n",
        "\n",
        "    # we do not want to group tokens when computing the metrics\n",
        "    pred_str = tokenizer.batch_decode(pred_ids, skip_special_tokens=True)\n",
        "    label_str = tokenizer.batch_decode(label_ids, skip_special_tokens=True)\n",
        "\n",
        "    wer = 100 * metric.compute(predictions=pred_str, references=label_str)\n",
        "\n",
        "    return {\"wer\": wer}\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wc8f6ZqlmpFO",
        "outputId": "2887c57d-f562-47ad-f0c9-c7ac4056885b"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "WhisperForConditionalGeneration(\n",
              "  (model): WhisperModel(\n",
              "    (encoder): WhisperEncoder(\n",
              "      (conv1): Conv1d(80, 768, kernel_size=(3,), stride=(1,), padding=(1,))\n",
              "      (conv2): Conv1d(768, 768, kernel_size=(3,), stride=(2,), padding=(1,))\n",
              "      (embed_positions): Embedding(1500, 768)\n",
              "      (layers): ModuleList(\n",
              "        (0-11): 12 x WhisperEncoderLayer(\n",
              "          (self_attn): WhisperAttention(\n",
              "            (k_proj): Linear(in_features=768, out_features=768, bias=False)\n",
              "            (v_proj): Linear(in_features=768, out_features=768, bias=True)\n",
              "            (q_proj): Linear(in_features=768, out_features=768, bias=True)\n",
              "            (out_proj): Linear(in_features=768, out_features=768, bias=True)\n",
              "          )\n",
              "          (self_attn_layer_norm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
              "          (activation_fn): GELUActivation()\n",
              "          (fc1): Linear(in_features=768, out_features=3072, bias=True)\n",
              "          (fc2): Linear(in_features=3072, out_features=768, bias=True)\n",
              "          (final_layer_norm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
              "        )\n",
              "      )\n",
              "      (layer_norm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
              "    )\n",
              "    (decoder): WhisperDecoder(\n",
              "      (embed_tokens): Embedding(51865, 768, padding_idx=50257)\n",
              "      (embed_positions): WhisperPositionalEmbedding(448, 768)\n",
              "      (layers): ModuleList(\n",
              "        (0-11): 12 x WhisperDecoderLayer(\n",
              "          (self_attn): WhisperAttention(\n",
              "            (k_proj): Linear(in_features=768, out_features=768, bias=False)\n",
              "            (v_proj): Linear(in_features=768, out_features=768, bias=True)\n",
              "            (q_proj): Linear(in_features=768, out_features=768, bias=True)\n",
              "            (out_proj): Linear(in_features=768, out_features=768, bias=True)\n",
              "          )\n",
              "          (activation_fn): GELUActivation()\n",
              "          (self_attn_layer_norm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
              "          (encoder_attn): WhisperAttention(\n",
              "            (k_proj): Linear(in_features=768, out_features=768, bias=False)\n",
              "            (v_proj): Linear(in_features=768, out_features=768, bias=True)\n",
              "            (q_proj): Linear(in_features=768, out_features=768, bias=True)\n",
              "            (out_proj): Linear(in_features=768, out_features=768, bias=True)\n",
              "          )\n",
              "          (encoder_attn_layer_norm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
              "          (fc1): Linear(in_features=768, out_features=3072, bias=True)\n",
              "          (fc2): Linear(in_features=3072, out_features=768, bias=True)\n",
              "          (final_layer_norm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
              "        )\n",
              "      )\n",
              "      (layer_norm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
              "    )\n",
              "  )\n",
              "  (proj_out): Linear(in_features=768, out_features=51865, bias=False)\n",
              ")"
            ]
          },
          "execution_count": 81,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from transformers import WhisperForConditionalGeneration\n",
        "\n",
        "model = WhisperForConditionalGeneration.from_pretrained(\"openai/whisper-small\")\n",
        "model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rJKcpGW0mpFP"
      },
      "outputs": [],
      "source": [
        "model.config.forced_decoder_ids = None\n",
        "model.config.suppress_tokens = []"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1Ei4Z6vsbknu"
      },
      "outputs": [],
      "source": [
        "\"\"\"\n",
        "PEFT - Parameter Efficient Fine-Tuning\n",
        "\n",
        "In this method certain layers of the model are freezed during fine-tuning in LLM.\n",
        "\n",
        "This approach updates only a small subset of the models parameters. Since other parameters are freezed,\n",
        "it requires less memory, resulting in faster training and making it more cost-effective compared to other methods.\n",
        "\n",
        "\n",
        "LoRA - Low-Rank Adapters\n",
        "\n",
        "This method calculates and print the\n",
        "  - total number of trainable parameters\n",
        "  - total number of all parameters\n",
        "  - percentage of trainable parameters\n",
        "\n",
        "This information provides an overview of the models complexity and resource requirements for training.\n",
        "\n",
        "\"\"\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zLU9WPEel7OT"
      },
      "outputs": [],
      "source": [
        "\"\"\"\n",
        "!pip install peft\n",
        "\n",
        "from peft import prepare_model_for_int8_training, LoraConfig, get_peft_model\n",
        "\n",
        "model = prepare_model_for_int8_training(model)\n",
        "\n",
        "def make_inputs_require_grad(module, input, output):\n",
        "      output.requires_grad_(True)\n",
        "\n",
        "model.model.encoder.conv1.register_forward_hook(make_inputs_require_grad)\n",
        "\n",
        "\"\"\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GCz2VRkwmFQe",
        "outputId": "c077b5f8-ee91-4edf-d947-4a304e7cfd6a"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "trainable params: 1,769,472 || all params: 243,504,384 || trainable%: 0.7266694631666262\n"
          ]
        }
      ],
      "source": [
        "# lora_config = LoraConfig(r=16, lora_alpha=32, target_modules=[\"q_proj\", \"v_proj\"], lora_dropout=0.05, bias=\"none\")\n",
        "\n",
        "# # purpose is to modify the original model based on the specified LORA configuration.\n",
        "# model = get_peft_model(model, lora_config)\n",
        "\n",
        "# model.print_trainable_parameters()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jsXUw0OTf3qa"
      },
      "source": [
        "**Note**:\n",
        "\n",
        "*IndexError: Invalid key: 3403 is out of bounds for size 0 -----> error faced during the model training.*\n",
        "\n",
        "*Thats why PEFT & LORA methods has beed skipped in the fine-tuning process.*"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TOcxmABa4-DR"
      },
      "source": [
        "## **Model Training**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TFbWuC8Z4-DR"
      },
      "outputs": [],
      "source": [
        "\"\"\"\n",
        "Note: Model Fine-tuning and training has not worked properly. Because during the training process, faced the issue of\n",
        "\n",
        "    - CUDA Setup failed despite GPU being available\n",
        "    - NotImplementedError: Cannot copy out of meta tensor; no data!\n",
        "\n",
        "So change the training args parameters and calculate the loss and error at different stpes [25, 50, 100].\n",
        "\n",
        "\"\"\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cUDLc0etmpFV",
        "outputId": "2a19bee4-53ba-47bf-f424-78dc2ee7f1fc"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "Seq2SeqTrainingArguments(output_dir='./whisper-small-mr', overwrite_output_dir=False, do_train=False, do_eval=True, do_predict=False, evaluation_strategy=<IntervalStrategy.STEPS: 'steps'>, prediction_loss_only=False, per_device_train_batch_size=8, per_device_eval_batch_size=8, per_gpu_train_batch_size=None, per_gpu_eval_batch_size=None, gradient_accumulation_steps=2, eval_accumulation_steps=None, eval_delay=0, learning_rate=0.02, weight_decay=0.0, adam_beta1=0.9, adam_beta2=0.999, adam_epsilon=1e-08, max_grad_norm=1.0, num_train_epochs=3.0, max_steps=100, lr_scheduler_type=<SchedulerType.LINEAR: 'linear'>, lr_scheduler_kwargs={}, warmup_ratio=0.0, warmup_steps=50, log_level='passive', log_level_replica='warning', log_on_each_node=True, logging_dir='./whisper-small-mr/runs/Dec30_10-49-56_0db79d94c419', logging_strategy=<IntervalStrategy.STEPS: 'steps'>, logging_first_step=False, logging_steps=25, logging_nan_inf_filter=True, save_strategy=<IntervalStrategy.STEPS: 'steps'>, save_steps=50, save_total_limit=None, save_safetensors=True, save_on_each_node=False, save_only_model=False, no_cuda=False, use_cpu=False, use_mps_device=False, seed=42, data_seed=None, jit_mode_eval=False, use_ipex=False, bf16=False, fp16=True, fp16_opt_level='O1', half_precision_backend='auto', bf16_full_eval=False, fp16_full_eval=False, tf32=None, local_rank=0, ddp_backend=None, tpu_num_cores=None, tpu_metrics_debug=False, debug=[], dataloader_drop_last=False, eval_steps=25, dataloader_num_workers=0, past_index=-1, run_name='./whisper-small-mr', disable_tqdm=False, remove_unused_columns=True, label_names=None, load_best_model_at_end=True, metric_for_best_model='wer', greater_is_better=False, ignore_data_skip=False, fsdp=[], fsdp_min_num_params=0, fsdp_config={'min_num_params': 0, 'xla': False, 'xla_fsdp_grad_ckpt': False}, fsdp_transformer_layer_cls_to_wrap=None, deepspeed=None, label_smoothing_factor=0.0, optim=<OptimizerNames.ADAMW_TORCH: 'adamw_torch'>, optim_args=None, adafactor=False, group_by_length=False, length_column_name='length', report_to=['tensorboard'], ddp_find_unused_parameters=None, ddp_bucket_cap_mb=None, ddp_broadcast_buffers=None, dataloader_pin_memory=True, dataloader_persistent_workers=False, skip_memory_metrics=True, use_legacy_prediction_loop=False, push_to_hub=True, resume_from_checkpoint=None, hub_model_id='gopiashokan/whisper-small-mr', hub_strategy=<HubStrategy.EVERY_SAVE: 'every_save'>, hub_token=None, hub_private_repo=False, hub_always_push=False, gradient_checkpointing=True, gradient_checkpointing_kwargs=None, include_inputs_for_metrics=False, fp16_backend='auto', push_to_hub_model_id='whisper-small-mr', push_to_hub_organization=None, push_to_hub_token=None, mp_parameters='', auto_find_batch_size=False, full_determinism=False, torchdynamo=None, ray_scope='last', ddp_timeout=1800, torch_compile=False, torch_compile_backend=None, torch_compile_mode=None, dispatch_batches=None, split_batches=False, include_tokens_per_second=False, include_num_input_tokens_seen=False, neftune_noise_alpha=None, sortish_sampler=False, predict_with_generate=True, generation_max_length=225, generation_num_beams=None, generation_config=None)"
            ]
          },
          "execution_count": 25,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from transformers import Seq2SeqTrainingArguments\n",
        "\n",
        "training_args = Seq2SeqTrainingArguments(\n",
        "                                          output_dir=\"./whisper-small-mr\",  # change to a repo name of your choice\n",
        "                                          per_device_train_batch_size=8,    # 8 samples pocessed in each training batch\n",
        "                                          gradient_accumulation_steps=2,    # increase by 2x for every 2x decrease in batch size\n",
        "                                          learning_rate=2e-2,               # learning_rate = 0.02\n",
        "                                          warmup_steps=50,                  # learning rate graduqally incresed from 0 to 50 and after 50 it should same learning rate 50-100 steps\n",
        "                                          max_steps=100,                    # Maximum number of training steps.\n",
        "                                          gradient_checkpointing=True,      # it store only necessary results to save memory\n",
        "                                          fp16=True,                        # 16-bit floating-point precision for certain operations & speeding up training\n",
        "                                          evaluation_strategy=\"steps\",\n",
        "                                          per_device_eval_batch_size=8,     # evaluation batch size\n",
        "                                          predict_with_generate=True,       # in evaluation will predict outcomes and also generate new sequences of text\n",
        "                                          generation_max_length=225,        # Maximum length of generated sequences during evaluation.\n",
        "                                          save_steps=50,                    # saving models in every 50 steps\n",
        "                                          eval_steps=25,                    # evaluate the model during training in each 25 steps\n",
        "                                          logging_steps=25,                 # logging information automatically stored by huggingface transformer library\n",
        "                                          report_to=[\"tensorboard\"],        # training logs should be reporte to TensorBoard\n",
        "                                          load_best_model_at_end=True,\n",
        "                                          metric_for_best_model=\"wer\",\n",
        "                                          greater_is_better=False,          # error metrics like Word Error Rate (WER) or similar, where lower values indicate better performance\n",
        "                                          push_to_hub=True,\n",
        "                                          push_to_hub_model_id=\"whisper-small-mr\"\n",
        "                                      )\n",
        "training_args"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6PNg1Dq8mpFV",
        "outputId": "056bd582-ca80-4765-d6d2-8f013ae26767"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<transformers.trainer_seq2seq.Seq2SeqTrainer at 0x7896cbefd420>"
            ]
          },
          "execution_count": 26,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# manage the training process of sequence-to-sequence model\n",
        "\n",
        "from transformers import Seq2SeqTrainer\n",
        "\n",
        "trainer = Seq2SeqTrainer(\n",
        "                          args=training_args,\n",
        "                          model=model,\n",
        "                          train_dataset=common_voice[\"train\"],\n",
        "                          eval_dataset=common_voice[\"test\"],\n",
        "                          data_collator=data_collator,\n",
        "                          compute_metrics=compute_metrics,\n",
        "                          tokenizer=processor.feature_extractor)\n",
        "trainer"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 254
        },
        "id": "nlaSYB4CmpFW",
        "outputId": "3a1e5f7e-3106-43e0-a50e-ee9617b88072"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "`use_cache = True` is incompatible with gradient checkpointing. Setting `use_cache = False`...\n"
          ]
        },
        {
          "data": {
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='51' max='100' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [ 51/100 40:52 < 40:52, 0.02 it/s, Epoch 0.20/1]\n",
              "    </div>\n",
              "    <table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              " <tr style=\"text-align: left;\">\n",
              "      <th>Step</th>\n",
              "      <th>Training Loss</th>\n",
              "      <th>Validation Loss</th>\n",
              "      <th>Wer</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>25</td>\n",
              "      <td>19.471000</td>\n",
              "      <td>35.392952</td>\n",
              "      <td>100.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table><p>\n",
              "    <div>\n",
              "      \n",
              "      <progress value='132' max='227' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [132/227 14:47 < 10:43, 0.15 it/s]\n",
              "    </div>\n",
              "    "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='51' max='100' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [ 51/100 40:52 < 40:52, 0.02 it/s, Epoch 0.20/1]\n",
              "    </div>\n",
              "    <table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              " <tr style=\"text-align: left;\">\n",
              "      <th>Step</th>\n",
              "      <th>Training Loss</th>\n",
              "      <th>Validation Loss</th>\n",
              "      <th>Wer</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>25</td>\n",
              "      <td>19.471000</td>\n",
              "      <td>35.392952</td>\n",
              "      <td>100.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table><p>\n",
              "    <div>\n",
              "      \n",
              "      <progress value='149' max='227' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [149/227 16:42 < 08:48, 0.15 it/s]\n",
              "    </div>\n",
              "    "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "trainer.train()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IHAoyWv6mpFX"
      },
      "source": [
        "### **Model Max_steps = 50**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jczi9M8ampFX",
        "outputId": "8ce9491d-66e6-43af-dd7b-a7c8945e48e5"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "Seq2SeqTrainingArguments(output_dir='./whisper-small-mr', overwrite_output_dir=False, do_train=False, do_eval=True, do_predict=False, evaluation_strategy=<IntervalStrategy.STEPS: 'steps'>, prediction_loss_only=False, per_device_train_batch_size=8, per_device_eval_batch_size=8, per_gpu_train_batch_size=None, per_gpu_eval_batch_size=None, gradient_accumulation_steps=2, eval_accumulation_steps=None, eval_delay=0, learning_rate=0.02, weight_decay=0.0, adam_beta1=0.9, adam_beta2=0.999, adam_epsilon=1e-08, max_grad_norm=1.0, num_train_epochs=3.0, max_steps=50, lr_scheduler_type=<SchedulerType.LINEAR: 'linear'>, lr_scheduler_kwargs={}, warmup_ratio=0.0, warmup_steps=25, log_level='passive', log_level_replica='warning', log_on_each_node=True, logging_dir='./whisper-small-mr/runs/Dec30_23-11-27_44c3f51c7637', logging_strategy=<IntervalStrategy.STEPS: 'steps'>, logging_first_step=False, logging_steps=50, logging_nan_inf_filter=True, save_strategy=<IntervalStrategy.STEPS: 'steps'>, save_steps=50, save_total_limit=None, save_safetensors=True, save_on_each_node=False, save_only_model=False, no_cuda=False, use_cpu=False, use_mps_device=False, seed=42, data_seed=None, jit_mode_eval=False, use_ipex=False, bf16=False, fp16=True, fp16_opt_level='O1', half_precision_backend='auto', bf16_full_eval=False, fp16_full_eval=False, tf32=None, local_rank=0, ddp_backend=None, tpu_num_cores=None, tpu_metrics_debug=False, debug=[], dataloader_drop_last=False, eval_steps=50, dataloader_num_workers=0, past_index=-1, run_name='./whisper-small-mr', disable_tqdm=False, remove_unused_columns=True, label_names=None, load_best_model_at_end=True, metric_for_best_model='wer', greater_is_better=False, ignore_data_skip=False, fsdp=[], fsdp_min_num_params=0, fsdp_config={'min_num_params': 0, 'xla': False, 'xla_fsdp_grad_ckpt': False}, fsdp_transformer_layer_cls_to_wrap=None, deepspeed=None, label_smoothing_factor=0.0, optim=<OptimizerNames.ADAMW_TORCH: 'adamw_torch'>, optim_args=None, adafactor=False, group_by_length=False, length_column_name='length', report_to=['tensorboard'], ddp_find_unused_parameters=None, ddp_bucket_cap_mb=None, ddp_broadcast_buffers=None, dataloader_pin_memory=True, dataloader_persistent_workers=False, skip_memory_metrics=True, use_legacy_prediction_loop=False, push_to_hub=True, resume_from_checkpoint=None, hub_model_id='gopiashokan/whisper-small-mr', hub_strategy=<HubStrategy.EVERY_SAVE: 'every_save'>, hub_token=None, hub_private_repo=False, hub_always_push=False, gradient_checkpointing=True, gradient_checkpointing_kwargs=None, include_inputs_for_metrics=False, fp16_backend='auto', push_to_hub_model_id='whisper-small-mr', push_to_hub_organization=None, push_to_hub_token=None, mp_parameters='', auto_find_batch_size=False, full_determinism=False, torchdynamo=None, ray_scope='last', ddp_timeout=1800, torch_compile=False, torch_compile_backend=None, torch_compile_mode=None, dispatch_batches=None, split_batches=False, include_tokens_per_second=False, include_num_input_tokens_seen=False, neftune_noise_alpha=None, sortish_sampler=False, predict_with_generate=True, generation_max_length=225, generation_num_beams=None, generation_config=None)"
            ]
          },
          "execution_count": 24,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from transformers import Seq2SeqTrainingArguments\n",
        "\n",
        "training_args = Seq2SeqTrainingArguments(\n",
        "                                          output_dir=\"./whisper-small-mr\",\n",
        "                                          per_device_train_batch_size=8,\n",
        "                                          gradient_accumulation_steps=2,\n",
        "                                          learning_rate=2e-2,\n",
        "                                          warmup_steps=25,\n",
        "                                          max_steps=50,\n",
        "                                          gradient_checkpointing=True,\n",
        "                                          fp16=True,\n",
        "                                          evaluation_strategy=\"steps\",\n",
        "                                          per_device_eval_batch_size=8,\n",
        "                                          predict_with_generate=True,\n",
        "                                          generation_max_length=225,\n",
        "                                          save_steps=50,\n",
        "                                          eval_steps=50,\n",
        "                                          logging_steps=50,\n",
        "                                          report_to=[\"tensorboard\"],\n",
        "                                          load_best_model_at_end=True,\n",
        "                                          metric_for_best_model=\"wer\",\n",
        "                                          greater_is_better=False,\n",
        "                                          push_to_hub=True,\n",
        "                                          push_to_hub_model_id=\"whisper-small-mr\"\n",
        "                                      )\n",
        "training_args"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "w6aOtycempFY",
        "outputId": "d743b7ed-606d-493f-c8a8-5bd5c8375b17"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<transformers.trainer_seq2seq.Seq2SeqTrainer at 0x7fa676d30eb0>"
            ]
          },
          "execution_count": 25,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from transformers import Seq2SeqTrainer\n",
        "\n",
        "trainer = Seq2SeqTrainer(\n",
        "                          args=training_args,\n",
        "                          model=model,\n",
        "                          train_dataset=common_voice[\"train\"],\n",
        "                          eval_dataset=common_voice[\"test\"],\n",
        "                          data_collator=data_collator,\n",
        "                          compute_metrics=compute_metrics,\n",
        "                          tokenizer=processor.feature_extractor)\n",
        "trainer"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 176
        },
        "id": "pty-n9LjmpFY",
        "outputId": "b620a749-8fdb-4816-b058-a1aab34192df"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "`use_cache = True` is incompatible with gradient checkpointing. Setting `use_cache = False`...\n"
          ]
        },
        {
          "data": {
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='50' max='50' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [50/50 41:45, Epoch 0/1]\n",
              "    </div>\n",
              "    <table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              " <tr style=\"text-align: left;\">\n",
              "      <th>Step</th>\n",
              "      <th>Training Loss</th>\n",
              "      <th>Validation Loss</th>\n",
              "      <th>Wer</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>50</td>\n",
              "      <td>72.229600</td>\n",
              "      <td>153.046295</td>\n",
              "      <td>100.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table><p>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "There were missing keys in the checkpoint model loaded: ['proj_out.weight'].\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "TrainOutput(global_step=50, training_loss=72.2296435546875, metrics={'train_runtime': 2513.6542, 'train_samples_per_second': 0.318, 'train_steps_per_second': 0.02, 'total_flos': 2.30868320256e+17, 'train_loss': 72.2296435546875, 'epoch': 0.2})"
            ]
          },
          "execution_count": 28,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "trainer.train()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kQDrHZw2mpFY"
      },
      "source": [
        "### **Model Max_steps = 100**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "eohSFtJCmpFZ",
        "outputId": "6b1c807d-e748-4dc5-ca00-fed65a3784fe"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "Seq2SeqTrainingArguments(output_dir='./whisper-small-mr', overwrite_output_dir=False, do_train=False, do_eval=True, do_predict=False, evaluation_strategy=<IntervalStrategy.STEPS: 'steps'>, prediction_loss_only=False, per_device_train_batch_size=8, per_device_eval_batch_size=8, per_gpu_train_batch_size=None, per_gpu_eval_batch_size=None, gradient_accumulation_steps=2, eval_accumulation_steps=None, eval_delay=0, learning_rate=0.02, weight_decay=0.0, adam_beta1=0.9, adam_beta2=0.999, adam_epsilon=1e-08, max_grad_norm=1.0, num_train_epochs=3.0, max_steps=100, lr_scheduler_type=<SchedulerType.LINEAR: 'linear'>, lr_scheduler_kwargs={}, warmup_ratio=0.0, warmup_steps=50, log_level='passive', log_level_replica='warning', log_on_each_node=True, logging_dir='./whisper-small-mr/runs/Dec30_13-29-34_b7eabe5028e2', logging_strategy=<IntervalStrategy.STEPS: 'steps'>, logging_first_step=False, logging_steps=100, logging_nan_inf_filter=True, save_strategy=<IntervalStrategy.STEPS: 'steps'>, save_steps=100, save_total_limit=None, save_safetensors=True, save_on_each_node=False, save_only_model=False, no_cuda=False, use_cpu=False, use_mps_device=False, seed=42, data_seed=None, jit_mode_eval=False, use_ipex=False, bf16=False, fp16=True, fp16_opt_level='O1', half_precision_backend='auto', bf16_full_eval=False, fp16_full_eval=False, tf32=None, local_rank=0, ddp_backend=None, tpu_num_cores=None, tpu_metrics_debug=False, debug=[], dataloader_drop_last=False, eval_steps=100, dataloader_num_workers=0, past_index=-1, run_name='./whisper-small-mr', disable_tqdm=False, remove_unused_columns=True, label_names=None, load_best_model_at_end=True, metric_for_best_model='wer', greater_is_better=False, ignore_data_skip=False, fsdp=[], fsdp_min_num_params=0, fsdp_config={'min_num_params': 0, 'xla': False, 'xla_fsdp_grad_ckpt': False}, fsdp_transformer_layer_cls_to_wrap=None, deepspeed=None, label_smoothing_factor=0.0, optim=<OptimizerNames.ADAMW_TORCH: 'adamw_torch'>, optim_args=None, adafactor=False, group_by_length=False, length_column_name='length', report_to=['tensorboard'], ddp_find_unused_parameters=None, ddp_bucket_cap_mb=None, ddp_broadcast_buffers=None, dataloader_pin_memory=True, dataloader_persistent_workers=False, skip_memory_metrics=True, use_legacy_prediction_loop=False, push_to_hub=True, resume_from_checkpoint=None, hub_model_id='gopiashokan/whisper-small-mr', hub_strategy=<HubStrategy.EVERY_SAVE: 'every_save'>, hub_token=None, hub_private_repo=False, hub_always_push=False, gradient_checkpointing=True, gradient_checkpointing_kwargs=None, include_inputs_for_metrics=False, fp16_backend='auto', push_to_hub_model_id='whisper-small-mr', push_to_hub_organization=None, push_to_hub_token=None, mp_parameters='', auto_find_batch_size=False, full_determinism=False, torchdynamo=None, ray_scope='last', ddp_timeout=1800, torch_compile=False, torch_compile_backend=None, torch_compile_mode=None, dispatch_batches=None, split_batches=False, include_tokens_per_second=False, include_num_input_tokens_seen=False, neftune_noise_alpha=None, sortish_sampler=False, predict_with_generate=True, generation_max_length=225, generation_num_beams=None, generation_config=None)"
            ]
          },
          "execution_count": 24,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from transformers import Seq2SeqTrainingArguments\n",
        "\n",
        "training_args = Seq2SeqTrainingArguments(\n",
        "                                          output_dir=\"./whisper-small-mr\",\n",
        "                                          per_device_train_batch_size=8,\n",
        "                                          gradient_accumulation_steps=2,\n",
        "                                          learning_rate=2e-2,\n",
        "                                          warmup_steps=50,\n",
        "                                          max_steps=100,\n",
        "                                          gradient_checkpointing=True,\n",
        "                                          fp16=True,\n",
        "                                          evaluation_strategy=\"steps\",\n",
        "                                          per_device_eval_batch_size=8,\n",
        "                                          predict_with_generate=True,\n",
        "                                          generation_max_length=225,\n",
        "                                          save_steps=100,\n",
        "                                          eval_steps=100,\n",
        "                                          logging_steps=100,\n",
        "                                          report_to=[\"tensorboard\"],\n",
        "                                          load_best_model_at_end=True,\n",
        "                                          metric_for_best_model=\"wer\",\n",
        "                                          greater_is_better=False,\n",
        "                                          push_to_hub=True,\n",
        "                                          push_to_hub_model_id=\"whisper-small-mr\"\n",
        "                                      )\n",
        "training_args"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zinxl2KdmpFZ",
        "outputId": "54462ba9-fd28-4b2a-d23f-725c6b97a014"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<transformers.trainer_seq2seq.Seq2SeqTrainer at 0x783ac8299510>"
            ]
          },
          "execution_count": 25,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from transformers import Seq2SeqTrainer\n",
        "\n",
        "trainer = Seq2SeqTrainer(\n",
        "                          args=training_args,\n",
        "                          model=model,\n",
        "                          train_dataset=common_voice[\"train\"],\n",
        "                          eval_dataset=common_voice[\"test\"],\n",
        "                          data_collator=data_collator,\n",
        "                          compute_metrics=compute_metrics,\n",
        "                          tokenizer=processor.feature_extractor)\n",
        "trainer"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "COpJIC3qmpFa"
      },
      "outputs": [],
      "source": [
        "processor.save_pretrained(training_args.output_dir)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 176
        },
        "id": "hDsRUWaTmpFe",
        "outputId": "39793eff-4c75-458d-91eb-7e99f2dc3b96"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "`use_cache = True` is incompatible with gradient checkpointing. Setting `use_cache = False`...\n"
          ]
        },
        {
          "data": {
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='100' max='100' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [100/100 45:54, Epoch 0/1]\n",
              "    </div>\n",
              "    <table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              " <tr style=\"text-align: left;\">\n",
              "      <th>Step</th>\n",
              "      <th>Training Loss</th>\n",
              "      <th>Validation Loss</th>\n",
              "      <th>Wer</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>100</td>\n",
              "      <td>84.853600</td>\n",
              "      <td>110.923683</td>\n",
              "      <td>100.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table><p>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "There were missing keys in the checkpoint model loaded: ['proj_out.weight'].\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "TrainOutput(global_step=100, training_loss=84.853642578125, metrics={'train_runtime': 2762.6552, 'train_samples_per_second': 0.579, 'train_steps_per_second': 0.036, 'total_flos': 4.61736640512e+17, 'train_loss': 84.853642578125, 'epoch': 0.41})"
            ]
          },
          "execution_count": 26,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "trainer.train()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2DQzq63n4-Dp"
      },
      "source": [
        "## **Trained Model Push to Hugging Face Hub**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 81
        },
        "id": "Nw46hr7rmpFf",
        "outputId": "3dc60f99-93d1-4b1b-c58a-262ab5cfe5b8"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAABHoAAAAdCAIAAABaE5zTAABN8klEQVR4nO19Z7gkVbX22qFy59MnzpkcSJJzUkEJAso1gICAZEUkB0VJol7hqgRJgoCIIElBwgURvFdBEJCMk/OcmZM7V64dvh+7z5kzgOHeD9THW+/Tz54+3dXVu6pr1trvWu9ahXbYczcihc6BIwgpRRKySQwATV0HgEwSYwkBoRyDzgFJkAgDAJEMADjoAECAISkAQCLMgQIAgRgAOKISAEsBSEjAMAH1HAEAgPrgxHtCAJYIS8BIEgAOiAEIJCmAAMQApAQNACaeEwBovwsCAAAoAACKAQCEufG5eh3ExLe3vxC1PyVgI9SBbHwFSQp/MyQSAoFEAkkAACzVHvDkiCUGAIGw2hoAiJQSAcMAIKgAQIxhIRCANLEUGAUgMUMWSEIgARASqEACEJOAQRqUYycRGBIPt7ihM8jjmBZiDenRMFlDspQGFuHG334IKVKk+BtBBDYYpgIRAQJJgSGiIqDAMOiYJF5Q0BxIuBCSGcTHKBYsAxhBIqjHERYiK6WJhIFRiEgVowALKqUBPMcxSOxxwkFqIDERmEiEBcaQAGkJJKU0pTQFaBgFCFeJlIh1gLAYEQCCQAIADGkAgGWCQQAIgTBDU+wzUACgAgCwAI1jAcQVgDErIBAEV5EUmBcEABBX4ISDAVJDQiMSU5kAMIk5gFBOIcJEIEwExW2TzjQOCDgnMUM6FwVAzJJVQDKCgpAGlTGgBHAgEAWeEQhj5AHiSGgAQKQQCARQANBkLAE4mBIBlZt4Fona3gQDAwCGdPUcSYEAJGAOGgDoMgEADtrExoklPIFEhPWQGBG2dZGUoqYmo4jgkJgRFCSSpqwilEipSTBAWG2vgSLAAaAYQILUEHdAagAYUIxQBMDVn1K5N4mhbebbmOpxEAiBgCMQWOicqwtKAhWAsQQqGQAAYA6UYQIARHIETCIBiGMBEgFHBACo4ADAkAHQ/pQAXSAgAgAJkBiQmHCgACBBEgAMkgIgAAwgAHEABogDQPvdFClSpEjxnkKt+zfyn7dDTrW84u3vTtCSqR4F0CabyXff71/FO74LYNM5Tj5/x5YSt30MetedvK/4HzoqJCUCMXEoAomJ57h9XG2SBhPP8eTxKkY3sR9hWRYhRABnQkiMACFqaURTv6xAMh3TMR3f41EgEVMWakmkJZHGIhrHNGaEcczAAGoRwBxhKQQHIQhFtvWugQ+p/o+jdnQGTRouJN9t86nm5s9vINp7ntwCv3P7qdYKgQSJAVA7etW2NrI9P0BTPvJ2uyo2mmMB8DYXgCenKlD7kximxrPePvOpr6DJABma/PSU2UzBu58JJDFI8Y5N1CsSweQxvm3H79j/OwNz7xmwbHtJOfHn23zb20+RRJNedeIHRW8LGmL5dkecIkWKFCn+gaB4SuJJLSMm2Ff7ySYr/glMMgRAQkgANMEDkNjkXQCYSGdNYqrXVG5DtHc+kXSadPEAGyOF7XjhlP38tbXIPwpI/lnSJaa6QKQyXfBOL47VMgkJkGjqOgm1T7UAqTJgAiY+HiZhTDAiDtEJMOwngR8HQeR34V4Vb4Z0TMd0fE9HjoWrcYEEFQAgJAKGZUyQBGg0qxlkYKFlqKmB5FgELBIMm4AwgJSAFAORgBV/k4AAY4lAAldvAQAglf9HAMoObDStbWq0CSZW4VJMicvApgYZQzvjsalRVgdFsAQE7byKBDxhrzBIigAkCACJpLJDABIJhDFqT3DSR0xaZokmqBoSMEEqkHgXaoTVHqWiDZPqg78VSkww5bNYTv1n8ihho85CAiheo06+QCAlBomRxAhJqXYl1Qbq5Ex4q/YZQco3oSmWGeBdIo9/7igmZtLen0TwZygjqBTi23c+5egkAqxm+Dbe+xeABEjytpf+ykdSpEiRIsX/FhRNUh0JSEXFAKDtMjd6vikQEsFE1ksATDohxYsETHUkf8mF/DUgMTWZM/Ub4V34zCau6888/1/gf5atQhKwWqO0JRz4z3z1Jm9JBG3WqdyzxBIpugsSKGz8LRAARlIgEBIBlsAnvxUE54lABFEpkOCIJ5hlu4oEqNv0iEbebQ4pUqT4/wLDItAkx0ITMKHvgoQAkjjfa2eEmYy2OMaAMUJYciYBAaIT//2VsE0CCEACTSS4JCjNtgCJsdyEmkwYijYmNsYIQCDAKv6i7DOAsjAIQCLcNvIqQaRIhmyTk0mGhqUQbb0cxxIkkPY0ACZUZwipYBmCjSEyAAFImSPRZk3tOQssAEAAlYCVpFzAZLJ9qlUUqL23jSZRTPij9vMJzjP1+ZTPC7wxJiXQFN41mU4Tk1xrgvip1JZEgNsTwBwhAIzbprt91EiZZflnPYhE7b3/77wMApASi7bnxW12LdtHKhGWIDBggUCAgI0OCU0enUQTFwUSIBVvfVfHDW1njCYCqeraS5EiRYoU7z8ongjvYTQpaKECAZ5wMBPuS0xN2rTjlBuja/ht/AEmdvu34O2Cw7drUdSfU2u0/jLUPNmU51P2DfBXd/IX0lN/C7BSMwKAxJPro/Y52cgeJ1+fJIQbZ6XSjLKtsFfLI+V6gUgBIARIicSEvxQAwjRNATJGLGaJDpJmzfm7bCZKjGR0nqpKUqR4H6CqVZFEugAiAAA4Bo4wERhaCaqxRb/9Y+zFIuaGmdUpRkSDmAEAkhgDFgASBAKEpcBtiYBiNkICxhKLiRjThD0iEk2stAHUfqYyMIFAIjmZQscg2kLFtvFR1qOd7CISBBISqAQAxEBiPFkliwDkxOvtPI9SAAokxRQ9IQYQUpEsiQm0y1aRbEv0BFIMDxNQFowCCAICgAMgMaWM9i+fZICJnNs7LTPaSMMmo0oSMJIY0CSvm1Ks207pKAKDQSICAkvJMSBBQRIkMQYhJCiXCCCQlCBBbuI18Kb7VURIijYx+6sHNanjEBOcmIoJjaUitCrQNiEVZBiAKyfbLkub8rMr+ckmgU7YJFS6kUDCu7y7cUqpm0iRIkWK9wsUT7pm2W6AIYDChKREAn0bP1HxUd4uuWYIQKhya4SxBAxMQrvEmfwZwvOumGRcE5kc9apqfcEmnmNAMSCx6esAG932xJbKu0xu+T+Zyf8nkJxIEE6RYr5b2Zs6ve2VEN64DMLqF8HABAIB7SJpAMknSuxUCFnCJjkrzplAEjQpkODAGEkynTnWlQwnlYTy9+toU6T4Pw0MUiOCaFySdoEl4ghTQbt68lhLEswkkhJjiZEQAMDb/2knpIAIFNESSGKlTwMJfKKKByRWBEkiMUU7hifSFyrV035VbPxXcESxBGjb8I1bokkLCYCAEYEZnvgUgnabhHbLBFVmFgOSE9WwDIFQVp0rSyUBJOUYABRpa+9q0s5KwBxhJDGSApBkiGCJkBRYCtXmASZiSYCAA+BJpd+UhkwMKIAgEtRxTb4uJg9fdVcCAGBIKQIQACDUpojAp/gjDgTaKkfMMQbAWOkFAAmEsdBBEiQFYHXUBLfjWepnUKmkjUr6jc//l9J2rBgXbzdPUgITAYAZAoHaWUEkxUSZ2cYPglTnDFTjJSJUsxAK0G4cIgEm+mFMnds7iZZ8Nz6WIkWKFCneS1AMMOFNJ+u1lDhkQlMOSLkEiZTUQcXeNsYORbs9FFI5sbYCvx1cBAnKF25K2CY8wERuB9oUA4CAQBIkZhMrEjnBxKYEddXfSs6u/pjsjTFJcvAkZ8OAJhUUeKoQRb5DbT/RRRDets3/CBNanU0KoAE2ymwQCLUFnxDYUFVH3i4MxxhE25UCxRIBJJPFG1iqELZaPYn2CUSSJxEjCCFJdQIStWKvKdxGq8kyiL3NvaZIkeK9gEBCQoIg4RxIWzqNBGDO+VBzLMc1N3HzWg4LKjCJkgQJaWG1dEdIYpUmQiAQcCzb2S0l9kMgJEJYUgSxalwhVTgGtbNgysggiQBt/O8tkQAkJAKJEEdS9aYTKtujqJ2QsJEsSbVngdv1YxMKQ9X1FAvUNjJYUFVFhkBg4AIJladSITCOAU9UoDGk6AEAAEdYIiwQxhJjkCpIJ0GYkiHJOCITHGmjwRWoTduUy5hSaoWneI1JleDGAxdA1AQwCA4gAGHACCSRAiTmGGCiCq7dnUhOui0gAggAxwCCYCBYYgxcTjQLmfBciiJOfiPedGxvM6Gl36SAalNvsjEG11aMt7fDACrPJhDICWEmEmjSFyCBACSmEhDgCTqaSIRZ+4RMnkNMRTvjN1kZoKh7m0iD8kTv4F0p6UqRIkWK9w3tLucSIVCLeBCyrUVXaSY0aZQlACAgILky/YABZDt8iBESmADHwAVgoQqJJWCQEwZ+qk96O8Tb1IlIICllW2SPNqnXUgKcjX5CTlGiT/45pavGpgXlaFOxxV9GOy0l/2eMa6PA8p07ax8dm/SMEzofrNy5BCowpgITIQArlQ4RgNshz0mJ5rs5RcuyYhELEExKQQgQQR0NDATwvw28pkiR4i8CI5FgAQBcbiyJURkY3dAp1gXFgiMhgSCMqaHrOkSu+uhExkkiEBOd+rAqxwIkAEi7oYUEBJIjIZFo929AICQQwO1E94TKTfVaQFIgBBy1kzZISolBAGr32gEGE4t1JABUe4iJw0GKbkkBCLhiI+36K0X8OAFAkmEkJcIgCUgsERaAVPcMIhlTsro23QKu8vNoIg8DmkQcQUSAM2lMGmYsVWHS2/VsWEqBhGj34ZAAIDDe1BxjAIYlZhi3m460a1yVX+NYKuqyMYCo6sQ4BoEwRwQkxsCRFAxjiagmKZaAIAIQAAa0WxNxkIwjohJxqsXipuKFd8W7WGk02YRQFVxJjFTBlcr1SYwAiGQcUQFIAMWQAAgMYuNPrCApAMMAQrYjbuoSEhOEEMHfYPTbV+yU9Jf8m+SdKVKkSJHifwpKEHhBaFt5CRxJYRpWGBEhBJbAOdc0jXGpUdx0W4aetS0j8QMnm2s2I51qxWxmdGw4xMjIFLgQIEUSh5bpINMybeu8U08gUnznez9gXAa+j5DkCTNMLZPNjo4OX3755Zlc9mtfv5hLxASnhikFwlgSiVR4L06kYFw3DCDI9QMCtqZpFGNKIInwCccf/+EP7hLE4ZXf/e6K5WswQnHEDEtSqrUacTabJRqrVqtZJ0upXq2PO46dyWSq47WsY/m+r2maHwT5YoFzjimJ4zgIgv7+/g0D6y3LyuVy1fFxQ9OjJNZNExGcJIlpmlLKSqWSz+c551JK27Zd12WMlcvlarVqGAZPmGpLwaX0wyBfKLmuazmZhMUAYNt2o1bJOMb5557XbDZuv/WWWMgE0c032+yiM89kLG6AvPSSbzbWj9uGDiT84Q9vZri/1XR/ctNla9esbLXi3Xfb4/QTjsg6dOGKpaeff365f/bcabO+cvKp111z5ZurXtezFjGtMPS5ZEhHgQg4cI0jIxXlp0jxPoBhAAqiXbAkJ9pUCEAIaSjyY82yvUpsk5zEhp9EBxx0wOc/cwiRwvXjS79xeaPZlELwhF/9ve8VsxnT0F977bVvfuvb02b3XnjRZVa2iKVwSBRGnsTa17/xzf0O/Pjue+2xet2bl118SZZqSSzz2dzXL76s1MOee+kPN9/0gGmYH9hyq2NPOvGCS6/YafsdjvvMwRnbHKpFN95626I/LTQNjbXqt9x8K7c6OOfVdcu+9rWvacXcDddfb9CEIipjM+MYNXfNt6+84qP7Hbn9zrusWPvKd6/8D4IJcGxb2W9dekFH0Xjm+Wduvfv+RAhKaJIkW+240+mnfrELa7/99a/+45ZrL7roou0XbAmA1oyOnH/JpcSyeBLvvtNOp3zxCyNIA5lccfpJIgwTwBIJkJIAIhjOOOOsPfbZc2D96q5C6Zabb/3v379gaIaN4aqrr3WRbVnW5eedWqvVQLeHR8fm9OSbtVq+3DEyNq5R0EydgbbF5gsu+OKxBqb1CC791pWVsXEM0JHPXXrZZVppGhP8O5dcsGrlcqlTJoVpkiBKSM7cYsEW5590XMbWayL56jlfiwcDJFlnX+k7379ybMSf0dd72ddPHRsaGq+6tmNSTBt+07AxwbjebHZ19MY+4lxQghdstuDii8666Yc3/P6FN+KEn3n6aVtv9QHLsgiGs04/w3XdMAyz+WKt1jBtq7dn2hdOPXXG9FmCxyysf/Wii9eNe7vsuvtJR326M2teeN6ZG8YqxClwgY47/th999qlMrT6wq9ddMyxx+6514cswTRCGdIolj+9+ZrfPft7Jriu6zttt/1hhx3279fdtmLlqqIh87lMM2RBxD52wMGf+tSnnIz+s5/d+cSvHsMYpORCAMFvY1Z/RvSeIkWKFCneC2CMMSFEYsKlFCyJkyhmEjDBBJyM1fB8iQgTvKe3OxG83mwYGmFxKDgIIUK/Wc7n8/k8IVqukNc0LWMYUdAaGRkZHRnvKpcatcrw4JCmGdls1nGcTCYjhEiSKJvNMsYwxlJKCVzTDIRxzJJEcM/zkBS+75ZKBYI1TTPCJOzt7RFCapoehD4imGC87TbbPf3001/64ilvvv4659xxspZl+YEbBJ5p2Ece8bmrr/l+d09nq+URrNm2HQReq9VynGwUhLZpOZbd293N4iSO47GxMSklADQaDU3TAs8fHx01dSOTyZi6Ecdxq9XSNK1er4dh2NfXxxijlFJKK5UKY6y7u3t4eFiRMV3XGWMZ23Esw9B0EJIQ4vt+4IeMsXq9aVnWV776tWnT+8vlspSy1qjHjJ99znl/evOt88491w+CYz9/nGQSSXzh+eetW7v65C98qdXy9v/IPknsCyE++clPv/THV0455Yu5QuHwIz4bhuHnjz/huT88v2zZsnw2F3p+vVYVPNEo5sAZcMOigBgATx/pI3285w8EHAtBBGgCiAQMEk3008MYh0kcJbFlZxDRqnX3gx/66H4HHPSb//6vI48+6qnf/NfXvn6xlDKKgosvudCy9bPPPvuM08+cMX3mMZ8/No7DY4899thjjj/ppFO+eMoXamNjS5YurlarHGTIYqJrfdP7oyQ2db23tzeXy2GM4zjmIDlPers73Waju6fv8ycc/9yz/33G6afd9/MHjj72uHJXd8zEV75yQRj5X/jSqZd/69u6Ts888/SYx6d++bTPfe6Yk08++ewzz/jTG28uW7J0eHCESyaAlzo6Ozq7KKUY49kzZpcKHRgESCYwn9AQUI4wR/iCc8677957D9hv/2233free+6+4447svncJz9zGAPc1dN78EEHvvTiHy686NJ1G0a+et65Xq3uuq5p2IZucc5POeWUzRcsOOqooy786td/9avHTzjhhJ6+adVq9YwvnzYyPHja6We+8tob5597ThwGI2PjM2fPbTQajXq1VqvZto0xRFFUa7WO+tyx4yOjZ5z+5WbT/dKXTovjpKNQOvLww1gSff64ExYvXX7KySdmHZvzJJfLCZCabdYb7he/cOrq5cu+dMpJY7Xq544/MZft0Kn5+WOOWrJ04Tcv/84LL7x08McOyti6RggSMkmSXC6HEIqSpL+/3/WDMInjOLYd58jPHhaFvmkaSRIdeuihm2222feu+v7njjri5ZdfuvHG6zGGbDY7PLi+q6tcqzZOO+3LCMhxJ550/PEnNKrV0754qm7YAohlWZXx0Z123AGEdH1fs6z58zaL45gnIcJCN42YswsuuODEE0888eRTnnzyqZNOOHGz+QtMO5dwiZF0DH3FylVUNwuFwoplSxFCmUzmoYceuvrqq1evWhtFEWPMNE3l8QEgJVcpUqRI8XcDlVJijBGmUiQAIISQmFiZXLM2ZBga4xLrZqs2mM3ZEkE2m3Wrg4Zll0rTotAPvYrHk0xfae26EQCY3VOMXM/SzY5sh2VneBRmLLO3t7fhtkyNtpp127Q0TRscWD9z1gwASSmVUnIpsI6TJOFSdBW7ud6ojQ/lOzIjIyOWPm1srFLoMtesW7v5vL1fe+XVQolznlhmQdfNwcFhQki5XHYyxfUDYwB82ozesbGKrpmU6p7XMgytu6u0YcNQR5deLhfXrhnWcmZPT8/AmtUSI8CIGrplWaZtEUIsy2o0GrbplAodoecLzkMvjGNWKpdqjXqSJLZtx3EcRVEURQghXdcJIT09PevWrctkMlJK0zRDP8jY1sjIENH0TK5QqzfsjGOblh8G2WxW17XzzzunmM9yFoZhGMdxT3dv99wthMRvvvbm2GjlySefOuiQT26xYKuRdcun9/c888KrlWrzwV8+dsHJ+3bknVzPTCeXXbp0acv3coU80fRtt9uh6XrPP/eCbTuUSA1hijWCCQguBAMkEkg4jVK3miLF+wOKpUY4pUL1M5C8rc2TnHPbsjiXQRSb0iCavs12OyBMH3rkYc75/ff9/KCDP7H33nu//MoL5c7i7bff2mw2oyD++sUX1Rp1gdHsubPqTRl57mFHfCafzV57083r16/DlMSM2Znc9jvs9PTAMwjw3FmzQ88nhhBcIoSEEDtsv+3alat32GGnMIifePSR6thosHDhEZ8/afb8BeMm7u/refyxxwzTGh0f+8pXzxeCW52zhqs1m2iWbh152GEzZ0y7+oa7RkfHwzDwgxbRzLnzNntl5E0e8Dmz5jZq9XJBIGBScoEokhhAcow5IAKkVa1vt/02EviLL76IENnn3z4+bfoMalrFjs7+vp5nn/v90Fj1xZdf3e3A3ebPnrHG1yOEW/UGFvz6a3+gWU498jQDv/rqq/t8eL9Zs2ZZGpk3Z9bzL7xUrTdeeOmPex378Z6ucujSgQ2DeSRmzpxZ89kOO+540nFHfv2yb207f8dMrvD8fz2CpXjq17856FOfndY/oz66bustt3jtjdcZY4888tglZx0/Z9aMN5evbzQaZkbzo+iD+x6Qzxcf/t1vPbf5+BNPfvaw47OZogyj+XPn/PGtVxuN5nO/f/HCs4944pGfW2YiQMZBYuedwEecyYF1G2yrYJkOlmiXXXfafPPNa7UBhBBC6JFHHnnklw9GUTR31sylS5fOmTmrq7O8YcNguVyu1+vTp08///zzGZemndN1ffny5bvuuXdXT1+j5UqBTN2Y2T8tn83UPcgXSnbGqVZHkWAsiZgUjEvDMFzXtQuZhQsX7bvDZrmM4wWjBKH99tnn9ddedhyHaEYSuT3dnXESiQQVi8WVK1d2dHQ4jiOEEJLFSahRAxCa0g5+EqmSMEWKFCneF1DOuUBEAgJEqIYBYapbxXLntVedv3bt2tlbfNDz4+ro4hUrF++0y77lQr66ZuGXTz/D4409d9/tvOMPLZeKK6risd888+tfP+W747tsudVFl1xcRR1rB9ZphGkY+75vGM755507b96cjK2vXbv629+83PM8hBBCICXHmGCMJcK903oP+MhHPr7vh1joShx945tXDKyLM7n8eRectPnmW44N0p6enief+sm9P7vze1d/zzac4z9/3JZbzf/eVdccd/wpu++6r2GSl1976oYbfrjbTnvtttvuhXJ4wQXnf+PiGzOZ7Omnnzh7zvSu8sxFf1p64TlnlTuKF19yCTF1J5cVUkqE77333hdeeEHX9SuuuKKQyydB6Daal158iUG1DRs2fOfKK/r7+zs6Ol566aWbbropm82ec8453d3d2Wx2aGjorrvuOuaYY2bMmIEQWr5sybe/cVkxnz3iqKPnb7HlVddcu3j5imKhRHVtbGxM1/XFi5c+/PAvzj/37JixIIg4DqbPmBOG8YoVq2b2z/SCCBMtSfgHNt9SMrZ6zaq+3v4giFgS77Dd1o8+u7xSrc6ZO//Nt17mEj373O+/cMYF69esHx4eZl7EQ0/TqJ3J1VwvCXysC02jjSTGNJXjp0jxvgBJTARFEqN2SQ9X5aNYAOfMNHOcJ4wxwGDqhkgY55wQMm16f7UabRga6ejq3GabDxQKudGxYSGg1NE5NDSULxdqrcbQ6AjRyh0dnR/dd9/H//PB5cuX9vX1CSnXD26gerT11ls/+6sXecjnzVuw6E8LP7Bzn6ZphFLLcjpLxbvuvmePQ46K47iQz40ObXBdt9ZobrbFlhrz8vnsuoE1I+NjmqF3Ow7Gcsxr5nI5HCVRmGy/3da//e/f/P53v++bPVPTkUSiUqvvuvver76w0La1LTfbcvGfFu+1x1wkE4nU3RdBTNxGpKNUiurF7u7OgYG1SRRIpI2Pj8+aN79Sbc7+0HyNkEV/esvJF70gSeJQx6jVal37o1uf+92jv37iURQbtVbQ2dXp+i1KaRzHiGhUN0CKNWvW5AvFlut5bmvXXXYe+N2bmWyuiMn0ab2XffncMGEXXnjh2sHR+Tt/FBH6x5deqtcqmqYV8iXDsLbYYgudklUrV3R0dNqOEwVBR6moaUNHHn3sllvPO+drFwuJNgwPRZ5r65qkOAaQgn5gyw/ks87rb7yazeZrtTrjcSFnj435MUMIRBAEUiJN0zTTEhw7juM13Y997GM/+tGPjjjiYMZiIUShXIwCn7F4ZGRkwdx5vufGcUwIaTWalpOpVasSQc+0/nrDrdVrSALFZHBobMEWZcMwnnvmpX122/7BRx4zGJ6/xZbrBzfo3O0rUMmjOI6JrkVRZGomB3TggQfGYTg6PFzIF+Mk6u3q/OlPbtN18+JvXD6ngE2KqwE/5oRTS71dtVrNNM0kSRBCaiaGYUQRA4BNmkv9b28dliJFihQp/ioolwKAJBIESB1jLjgQ6uTyCEnT1I84+pgDDjzkEwfs2N/ff+a5Zx93xJG7zJ+2y847vzXoH3vssS//4dfX3/CDE8/79gEHHPjM757TtY6jDj9s0RtvXXjtTz60z76nHXHgupVLpZQnn3xyd3f3+eefX6+OXnbZJaeccsp3v3ulBJ4kCQBomgYIaZo2b8H8Qw455MyTT8AiPuGUoz7+8Y/fftt/7rvfR8vl8mmnn+43ChdecGF3d3cQemeeefZN19/0+OOPPvzIz88555wtttz67LPPrTdGr7r24hNPPP6Gax7s6e7bfJvM9ddfL0T2rLPOmjWr4+tfv8BrwTVXXXfppRdfe81VrtfsL8944P4Hnnn+uZO/8MUjP/e5F1566Qtf+IKU8owzzjCwfulFFx999LF333PX2Wef3dfXd/rpp+fz+UsvvfS444676aabhBCU0quuumrZsmXHHHMM5/yEE04QQlz3g2tOPO64e35217U/uLqj3DU0Vp02YyZnYqxSNSwrW8hfd/31lqn39PSxJMrn84HAQZRohj24fghTVEiSTK7gtkLakclm7KWLF3lB78D6DYHXcGy9Vq889dRTpxz9md322G7l2jUdnd1906bfduNtlfFKXyGHDTLarAZNFxhHUlKJCEUUEcSN/013xRQpUvwNUM0qoN2gVUEKJDCBKAziOOx0OmiMQz9YvGjRvvvuuevuuzzz29/utscHZ8yc7Qb1DYOrVq1ZceTnjujvnNNV7n3xpeeuvfEH5a6OdUMVm6LDjzhqZGj4ySceL+QyYGoRSzjIRX9atOt2Oxi6RYm22fwFP7v32QXblhFCQPCcObMMXYujYO2adTvtsOPuu+5SHR3a6kMf7ih39fa5Q6uWhIH3yU8eeuRZ21qWU1vxyre/dVkgLQ1pZSf7qYMOsUz6wvPPZOwcYxJh7gbN5/7wx0MP+QTGuKujq6en53dPPbzX7rMkMEACIQSAEUIMkERQG68Eno+EbPotjVAuUL3R6CTUcBwpURR4brM52ApCJorZTBz5GStz6hdPMzU3DloOLVJKhwaHi+X8IR87yHXd4dExYCyO41UrljVdf/3gsGMake/VGs0dd931aycfiXn4je9dNTgy6taHenr6642mYTm2bRJAQoixsTHOJSV6zrbWrlo9MDiq246uU9vQAeD222/FJuhWRrNsx85WxsaazXrD8518iXPEExHH4aqVy6o1XUekXqn2dHcuXTqUMGmaZhTFSEcIU8vJ1mutWq120vEnNuuNgYG1VMOWZRm2VavVWo36MZ878lOH/pup0RtvuH54w2A+X+CWlFKapuWHyejoKKFGf3//Xnvu+fqbb3IuNN0MgvD1V1/ef88d5s+dt/y11bvtvufqlQvnT8vphBqahjAmVL/uuuuCpq8Xe1cufuvWW64dWLtmUBQPPHB/09AwoM8cfjil+mmnfUFE3tkXXX7WWWddd81N2Ww2DMMoinK5XBR7lGIuEgBI776VIkWKFH83qBtYISaFFABYMCbCOAGCdV3//XPPEk0fq1cz+dwjj/2CMbZqzepD995+bHhk++0/aFnOr598opjL3nXXXVftvd9nP/vZ5379yPT+vgd+/mC93nzttdcq++5oanoul+vq6X7xxZeCIIjj+JVXXt7nQx/q6emhlErJlfoijKJYyCTmLc/9xCc+8eADd333yivK3TMqFf6ze+777fP3BxGntMuwnM58P0KoUCg0Gi1LNxiPOzs7n3/++bGxsYSFv3ry8Y/su7/jZEulUn9/F+dJkiQZJ7do0Z/q9TqG7L333nvcEZ8plUpREEZR9MwzzwRJnCQJY6xQKKgGGB/aa++nnnz6nHPOsXQjkcnm8zdft24dY2z16tULFy7ceuutC4WCaZphGC5atChJEsMw5syZ89GPfvSJJ54444wzuNekFM+aPj1krFwuiYSN12qd3T1hGLJEIIQ7OjrGx8erlXqr6YKZwZhEYWIaVpxEI2MVxnip1BH5MWcx1TBLRLPZNAytUR/v7u56+r//6/mnH5s5q3fx6mXfv/GGJ3795E677PyV62/2Rgcff/LnD//qsQ3VppbJWKYdCI/FCTGIZGji5mMpUqR4jyFAYCSkxICEQCCREAgIYlJijA1DJyyJdLAM3Xjj5ZeffLz/yCOPOProzy1bMjg6Os45L5byc+bOeuPl188///z+vpmXXHbxZ4844oZbb+zum26aXZ2dnZx7oR8g01i/YYOmaZls9tVXX/3Ihz7c3d3dmetxbHt8dIwQokJX22yzTXW8sn7tuhXVp+bPm3vg/vt/+tBP/Gk0qtVqSZIIIfL5vLdkyWVfv6i3r+/mb517+BGf/fnTfwgYbrr+jjvu+NrrLw8ObsjnOsKISSzCyHvpxT8ecuChvf09W83eDASMDo0iwUFwDmJSgiYBcwn5bG5sXZIkiW1bjMUI67Ztj41Xc/li03VDPyiXy3rC/CDyPK+rVFw/Cn69VZ6W8ZLQNM2IQ6FQ+NSnPjlr1pyf3Xv/qlWrdtlhWwISIZQvllzXTZIEgfjUpz99wMcO+c4VF4wMrB31RCZfyOVyruvmix3j4+OjQ8OOba5dO9DX3080vTI2DgA8iXPZwpqBdUkUO7YZhT7RjGptvCtXFABMyNB3bdMaGRuNEqZrZuRHvucWi8V6nURRYhiG4zhRHCaJbuWyLJaSQMyi8fFxSozNNtts2223vfTir35gq9mRH7Tchuu6SLPL5fKjjz765OP/+W+f+PhZZ3zZNvX/fOyJbL4kJXJd18kWIg4I6Fe/8pXaeOWeu+7WTENKlCRJHEZLFy/Zdrtt/nvJyKxZs1589qktZpbCwMMYY4yjKPrSeWe5tRY3Cwfvt8955533wC9+8eAfV3V1da1evXr5siV7fDpjmObOO+3w5qsvXX311X5MNE3r7OxU7XkppUEoAIDzBIH2j/ovkyJFihT/B0EJIVJiru7phLGUnAkpAUVR5Pl+wjlLOKFISokIEUJEfpDLZubMmadpmtts5ZxMxQ+aTXd8vNrd3V3I5pYtWWpZ1sjI2PjYKJK81WrNmTNnVv/0j3zkwxlb9/0WktJxHISQlBII5iATzhIhl61ccf/99x996Mf32n2HQtF4/Mnf3n7bk/vstedJpx7c9HwTbd5qtVRTwbGxsXw+L4TIZDK9vb1902bt++FPaDrEYgNLhG3bjLEwDNevX9/TsdP06dPnOfnd97gzCrRCrqM5Oji5ge/7ZsaJ45hzHsfxtdde+51vX/GpT33q3z7xye5y58033vTmojd1XV+3bl02mzUMY2RkZIcddqCU1mo1IYRpmtls9sYbb6zX64cddtgRRxzBkuCR++95/vnft9xGrdGycwVsWJZlCSF03axWq93d3bVaAyTu6upyHCfGWhSLzs7unp6+9RvWzpo1R0jk+/60njwhaMG8uX9YPPDZT3/GMg0uWBB42WzREmRgYGDvvffWDev5P7x4zhdPv+r717QG13357OPfXPRWpbmcIKJjAkJKxiXngASg5B99maVI8S8ICViAJiVGmAOAxIJjddsrDEhiArZjhsMtR3cMzarUxu78ye0PPnCz6zURL/7wR7e6rrt8+fIkie67775E8OUrVi1ZsmTu3LmGoYVh6IW1ufMXPHTzFUkcS0qztkM0GsfxypUrpZQ77rhjf+fMJUuWqAY/qsPq/PnzN6x8k2JCKb311lvvrq/IZa261nPZ965fsmxpJpdttZq/+c1vKtVq0/cWLly49dZb/eCun5tOuWTnM5nsiqVLKEFhzAUG06J2xgjjxAvivfbaa8uZc4eGhhBCILkEDiAmElxthGFo6gYgkc8XWRwLJEudZa3cMTQ65uy6a7GQi4LQdorlrm4WhUEQ+L7M5XLNZjVr28C5SOSRnz/uYwftf9ctN7/55puVam18fLyzs3P69P7Xx9YvWDBP13UAePTRx1atW3/RBV/JO8bnTjkLUZqECaG253mlUkd3d7ffqPb09DSbTdd1t5jRGwX+vHnz3hhf/sEPfhBhaDRqjuMIwwJTDg8P72BZlmU5lp0wf9422zTdlpTQ09OXy2Xy+SyCcMaM6ZZhjo+OICEJIXEcY4xjngBAoVAIg+TII498+eWXlfsAgGw2WygUKg2fWlYciobbeuihh/b94N5bbbXV7377bMKibKaINDNKuET4pptuFJF303evHBsbG/WMrbfVs9lstVqtVMenzdti5912rbday1Ys32/PbeI4loJxkIyxNrXW4ZePPnbER3fccccd731u2bx5CxrDr+YzmVtuueVLp59xwQUXDK5Zwe3Sj3/68xeee9l1Xc/zkiRpNpsIA0JI0/QkTm8PkiJFihR/P1CEEEYYgCHMKTEBSwrMoBIwyWYLBgWR+JEXGdTUEMaAgOphItavWRnutH1v/8wXX/xD35a7Y5FkTG0kCEdrzVJnT4dbYYwVOrrcRjWfzw9tGHzljy899NAvWOxjDIVctlKp8JhzLgkghLCh6QYm9fGx/3r6N8//+leWJj756Y/v85EDH3/81X33+eDw8Oill3xTxJ0XX3QJBx8k7uzsqFZHia6FUdJsNn//3EtP//q5gfWre6YZUcQM0u96jSQufGCr7dasaMRJ+LtfP/2ze+5E0iaIGiCqlbEvffm0mDOia0nCAVOJMCFarlC67LLLDE3PWvYpJ5509LGfO+OsF8IknDZt2tjYmGEYvb29QRDU6/XOzk7HcXzfZ4xZlvXggw/+6le/WrZs2W233nLwof/222d+RzRtWv/0aqPZarUyubzrtaKEd3YUa7VaIZ8zDGNsdNgLIyNjtqpjQxvWMpFEcYwF53EQJcHLb6xouvtTwy7lzeGhNX40a2SsniQJCF8DKSQ56OOHvP7Gq36r1t3dLRAeGhs3DTuXyzkZ203iKE6YBbquYwgBpAD+j77MUqT414REXAACKYgUfPL2vhiBrjfGfYqJZugaaPVqo9jZ78deELRMzdplzz11gl964cVCvpTEMohiP4xs02FS+C3PspyG63/kgIM585etWm05uUroG5k8C8NCJmsQbfGbC3fcaftytuu+u+6pu3VCTYQ0ikmxWPzpk09rhskR8txmV74UJ/H0mX2d+Wziu6Mj6zjapdJwLZ3outZoeRpBneVyGOJddtwJU7Jw6bK662Uz2VgkgRsBk4Shysjo3NlzuspdD//mkfF6IwFNgg6SIIkxCCwBS4ElSIQ0yx7YMEJMK1fqCGMWuF41WmdjWR8bDIVgIDUWUB4lWK/6EdFtwZhlmH7oUwlHHXvc/gd97MSTT/Cq44JDR6Hou63B0TFMDQTMIIgj8tqfFhuUrFmx/NQvfXmLBXNvv+PHGwaHv3fFN9aPVLx6VUM8YrzWcufplGCJZLxk6aJmcjAyDCFjr9mghr1m/VCc8OGx9dNm93HQVi1aBIf4mm3Wh/yg3jQxaoWV1xatP7C6bVdnnxCLbFsPmVy5bgRphqlZtWagOxaLYmrQOAhtwy53lubMmvGhvXfRKHMc9MlPf9bJTbvngV/GcSyldBynq7M7Srjrh7plBY2GHwYSaaZpffuKKwfWr7nyW9+IG0OZQmn2jF6/UfE8t9jZ9cqbi3b80AEH7T9jYP2agdWrCKaEOAI0CsjSKOc8k8kEIDBGzSCKuLB1Mm/urOvv+K6fSGrB1Vd975bvjJULubO/dtnhhx++ZOEK3/dt2y6VygBgmmYQBELECLT2zY6n3kMsvc1xihQpUrw/wFwiIRkRvo6FF/E4EUUH4cQNfKZRW2O+BZGtW8BR2GxmbNOX1E3EyqULRexvs8eHULbjkI8fZOviqSceWrlq+UDNO+jwz3mNyvzZ0/NdvbVI1lqNRYvf+uAH9+rt6xYcLr/8W5dc/I05s+c7di704yiIKZCcaYbN5rlnfPm73/lWoVCIOao1wkYzCIO6RnnOLGfN/Ae2mDWtNzdv9gIek7HqiG4TzdZbXrh8xeoDDjggit1c3vnm5d+98oqrnAwxTIgjFPgyTrwlS1/fYfud+3pnEkIuuOC8q35wVf+M/rFqhWpGEMaaldF0K4hYnIivX3TpWeedb1imH3p+5K4ZWO6GzSVLF/X09MyZM8e27W222Wbx4sW6rodhyBhTDvXMM8/8j//4j1ar1d3dvXzFqkrDiyRp+nG95SJMO0qFyuiIY2gZHYs4tHUaB77gieM4CRMUw7qlb9mUf+yTB4NJPvvxg1cvfCMSdRfCVSOtXffaj6L6sUd9bN2Yu3htHSTXkJCS7LDTrh3lwkMP3oNZErZaSNc7Z84UVF83MOAHdWRIz+CRScKYE0mRRDh9pI/08T48AAmJGaBEF0wTUudABOYIh4Q2Gbfz5cATmrQwp5ae7Zs159Y77uzrmaNh/dijDm9UR1575Y+L3lq8auXgl047RwDsuOtO3dP6/vjqa4GbyIRtvWDm6MialcODlTjJdnRXx2t5wzS4AC9565U3ujpLDW/85YUvSgMRmm012MEfPTAKgtXrx3WntPWCOT/8wTWzFmwz3ozOPv3L4wMrn37kwQ2rVtU8edKpZ8rA32benGkz5762cDmKIGq5W24xr1KvvrV0RbbcHYiES4ET04YcDqNnn3xyTt9MKdEf33xNmlpMc6HMUpLFHBI/oDzWoiCjUWo7lSB+beHyQlf/B7bfebPNt9xmq82XvvpSp8GXvfHHgfHa0SedaMfVww/40IrhykAgq61aLmPECQdq7n/IIR/YbuvHHv5ldXRESJQr5DWZDA8OrB6ufvTgT5DYPWT/D68ZqQ6MN2XkocRtRvHCFauPOeaYe3521yWXfbOzo7zqrRfGB1YcdfwJ0nIO/beDVi55ZWRoJTXIwrWD2+69j6mjTxy8//qx6pqROqZ6V7k8uGoA+bGoD/uVoYM+fTh2cocdeNDgktcbbH1gtFYOjOy5x0cIDj+y3x5rBisVD6pBrDkONkiUhJapI6kSmfysM0496sjPHHv80Tfd/KN6K77vvkd+fv8v9tvnw7fecktvdx/j8pBD/y1TKD3922fcMMoWS1GSIAJf/eo5zfrYnT/+YZx4Vr7AALVG1/YUqGnpa4ZGF68frbhBf3fHohefLdpGo9aUWtGNaFE3IXCllEAAiWDPPXfumD77dy+9sueu2zYrQys2jFmd084/+/Srrvw2MTOjrWi8Vm/Uq9XaeCZrJ0kSx8y2nSjkLBGU6AACQAJiKlEJAAAovc1xihQpUrxPwAIkgACZSMEQppTqPPYa1RHbyg1uGI28ZjFj+q4nORSzGbfZIoadLZbfevP1Rx5+aLudd7v9p/fsuefuD//ifsTjNWtW/eCmHwVMPv6fj5591hmr1m3QMzmE0J133jE6NnzppZfefvvt5Y6uH/3otkWLlriuXy6WO4rlbbfe5tvfuLy3s/y973wHgbjiiiu+f/XV++9/yO9++2y9UX3s0V9q2LzxuptP//IXXnzh2Xq10dHRWSjknKy5fOWKbD531VXXLF269L777/7hD28UHN9w/c2DQ+sWL3mrr3fGWWeeN3/B7JtvuWHVqjVXXvH9a6+9dvMtFvzHf1zpBq5hGDFnmWw+CMN6vVnIl2Imvn3Fd7bZettrrr322uuv3XGX7a+99hrTNO64447h4eErr7zy+9///vLlyx944AHP83zfb7Vavb29Y2Nj9957b6vV+sEPfvDTn/507733vvW2Hzea/jnnnn/5N7+dyWRGR0en9/cJnmQzdhh4CERnueR5XhzH5XK5VqlWRgZ/duePN99ysx//5HYdo18+cO/Q6IDU0BXfv27O3M1/fOsNto3v/cXDQ+NN0zRtS+cS9jvgwGeffVajuNmoPPTgz0857bTPn3DK4796YmRslFLCkeAEgU4xpsAmIpeQjumYju/HKDBwLLkmBBWABVbLViGxFKAhSgUQhoiAoXXr//M/n7ji36+492f3DQ9t+M6/X25oenW89qNbfkywcf/PH9j/wAMWLl706mtvMCZmzZhZzFmmQcyMEwnUcv1yuRz7XmVoyNHt1156dXh4cNmKxVzGlmNWa63urv5CLr965SqiZ2p1d+EbbyxduPALp53xi4cft0z9379xSV9nacmixd+/5rowgYcfvP+iC89/7fW37r77gdAPZk7rMy09iPxSZ68bJLVWQ9d1JImjOzyK16xYXq/V1g8MDg5v4AQngjCpU2JpRLctI2MbjqaJOGq6LS+OFy5esWjx8mNPOPHsc89ZvnTZrx57hERe5DV//sgjvdNn3HXbTTjxbv7JT2tRki/mrrziW4ce+skoZpt/YKt5C+ZvucVm999739133/3Nb37z2GOONHR64y232bnivXf9ZPutt7zvF7+su4Fj6wYB3bAMJxfH8euvv37W2ec2Gg0WtB5/9MH+WbO/e811tqPffdePbYuuHVjz0wd+4RQ67vnZXXNmT7/1jp82/BgwOvwzh915x097O7vWLVt8710/2WmPPW6+7Y6iad1/5483DK90k9Ztt99pmdmf3nnbllvNv//Bh9eP1MxMfve997ruhh8Ui/lms+lYtuQiDDyCRaGYKZdLgNHQ8DjBRhQlTz7xq+ee/f3ll1/+yCOPbbvdDr948JfLlq+kmvG1r1/8qc98WtNIJmPvtuuOX7ngnPvuu+eW23983Q3X77jdVuNDA0KwGXPmVlvBq28uHB/esGLxWyKOdN0EYuby5eroWNFxrr322su/9c3777v7uOOOvf+hR15+/a15s6a79Uos8bqhsRtvuE5D4oGHHrnzZ/dtseUHfvjDH0rJm806xjgMw/GxqmVZtp2BNsSUUV3PaVOlFClSpHhfgHbddVeMMeccAFQNgJTSsiwppZSSENJqtSilAKDruu/7qpcx5xwhZNt2kiSO41SrVUqpruucc865KuqVUmbzOU3TxqtVnVJN0yhg0zQFY0EQIIQsy4oZA4wAADCSCFWqY73lLi4SSvSBgYHZs+fXajVDt8IwzGazpmm23IbntSiluk4JRVEUOXa+2WzmcrlqtdrX1zcwMJDJZFQ5VjabBQCEkGEYYRiGYWgamkHQ+vXryuVuzbIbrYAYJiF6wjljjPHY1gyChGChoWENcMQSYlhhlACAqjdrtVpSynw+73keANi2PTo6msvl4jguFApRFKkTqO5urG4P2mw2LctCCEVRlM1m4zhWt/CK41jN0PM8QkgURWoPKnvGOc/lcrVaTdM0SmmxWKzValHITS0HhGMrCCI/iqkOtib0rEYzZjI0vCbbUazysGngpMD3+vTOnu4KTXIs2kKRdEzHdHzvRokEx4IKmY2FxkFgEVDs6kSAbsVGV5B56cdPFV3bjGyJ7BGWlHq6krBer43NnDnT8zzP85RplVLatl2v15WNNQwjSRLTNKvVarFYlFJSSpMkoZRqmqbuARgEgaZpnueFYdjZ2Vmv1wuFgq7r1UqzUCiEYei6LheJruvZrJMkie/7hBAAQQjBBFStUS5bCIIoCAJCiG3bzaabzWY9z8tm8urrqtVaZ2dnrVbTNCOTybhuw9KFpNKNk3y2AM2k5QWb7brz6aef/t0LLg5DvwZx02/kEDIN2ooCM+PEXpzN5oeGhkqlEqWUMRYEQS6XE0LEcWyaJqW00WgoP6Lrum3bIyMjjLFSqdRsNgkhCCElJVANjZJENeSwVfEtQqinpydJktHR0Y6ODt/3lf3Udd00zTiOKaVRFAFAvV6fO3dutVr1PM+2bSEEIYQxpu5khRAql8tqeqqnyODgYDabrdfrs2fPVqf3mGOOufXWWwkhjUZD13XXdTVNQwiFYahp2vTp0zds2JDJZFTLdQCIoqhYLA4PDxeLxTAMS6WS0p9LKev1erlcbjabhUKhXq+rOmSE0PDw8MyZM9UOgyCglCpfKaXs6elRxzg4OOg4TjabbTabURTNnTv3jTfe6O/vF0KEYajaOHmel8vlhABDtwHgpptuevLXj9911x1OxgyCQNeplKp2i4DEABQkBiAAAhBPmxOmSJEixXsOLIRQXY+EEEIIhBDn3Pf9Wq3WarUAgFLabDYBwLIsAMhms5Zl5fN527bVjTVrtVomk8nlcmr1kM1mkyTJ5/N9fX31aq1arXZ2diqSxhhbt26d2ichxHXdVqvleZ4QQtVElYvl8fFx13XjOO7s7Gy1Wup2wPl8fmBgoF6vR1FkmqaiJdVqVdfMWq2GEKrVaoZhqJkoPjNjxoxarVapVJIkqVarrVYLY6xc6axZcxjntVpN13UpRBB4SZJIKQvZHBdMzS2OEsO2kiQJgkB5elVtrM6P53nKqUdRVC6XOzo6AKBWq3me12g0FMsyTVO5/J6eHsuystlsLpfTNC2Tybiu67qu6tgxPj6ez+fVV0gpkyRRjUA6OjpqtZrjOFLKMAzVISh+22g0PDfw/RAAHMeSUjLOx8bGDMMims6YkJzzhCMGOtIAMEgMAOmYjun43o4CYY6BY2AYMwIcYQlYZb1M3WCMCckkEhwSw6S6hhv1iu/7+Xy+VquNj4/btk0prdfrmqYprqVpWqFQMAzDMAyM8fTp0zHGAAAAhmH4vt9sNimlnudFUeT7vrIqKtrl+/7w8HCchLVaLQgCx3H6+voIIYODg+o27mqNzjlXAaBsNjsyMmIYWj6fV/atUMip27VjAnES+r6fz+c0jQJAFAWGoWk6EQgYE2p7AMhkMoViTnKRK+SbzaaUIpfLUUqFkACgEaJpmgokUUrHx8dbrVZHR8fw8LDSCNTrdd/3TdNUmyVJ8tZbb3V0dBiGUalUDMNQHTI457quq4bmKtKn7HmpVHIcZ3Bw0Pf9YrEYx7HjOIZhaJqmbluvLKriYP39/QMDA5TSzs5Oz/OKxaLnedlsttVqlUoly7KGh4eDILBt2/O8oaGh7u5uTdOmTZtWqVQ8z9tzzz09z2s2m+Pj45ZlmaZpmmYmk3EcJ5fLYYzXr18PAK7rAoByW77vx3Gs+KGiUtVqFQAwxoZhTP6m+Xy+WCzquq7omfJTysskSZLL5XRdl1IODw8TQoaHh7u6uhhjjUbDMAyE0Nq1a7u6ujjnURRhjH3fz2Qy06ZNUwHHSqVSLBZrtWqz2eScM8Zt21bfPpGenUTKslKkSJHi/QLaaaedLMtSvZVUKFGlpwqFwsjICOe8VCoBQBAEKt4ppVReLUkSRYSEEACgHJsib5zzJFGB1WzCWRAEhmFIKW3d5JxXKpVCoaByO0KIIArDMLQzjuM4lUqlXCqqDoS2bQuBVaA3SZLe3t5Vq1bNmDGj2ay3Wq2enq56vc4Ys+2MbduVSiWXyzWbzUwmU61WC4XC+vXr58+fL4SoVCqK+TDGTENrVCuObbVcTzMtO1MIEyYkMMYQQvV6va+3JwkDngQZy6xWxnXDzBVLCFPf91XeCQBUT0IACMNQjZMhZxUkrlarKmjaaDQymUx3d/e6des0TcMYe56n+sgrN+y6brlcViQwm82qBQdjTNM0Ff3t6OhQr2CMEUJSkNBn5a5iKJpYgygAvxVn9EzYaORtTKiQOhkNWiyrubr/kcP2ZlnWIgEjaROqFCneezAsAg2wFE4COhcCQULApxQkKeE8GY5fu/u/O1kOu8Q0Cw0JWDdE5FGQSggQRRGlVAhhGIZa6MdxLISQUqo70hqGoRriqQiOZVmEkGq1qgyvynsghCilKhPOOTdN27IsJUNACCVJgnGbrrRaLcsyLMtSBtm27VarFQSB2i3GOEmSarXuOI7qAOQ42Vqtxhib1je91Wq1Wi1MhOBRNp/VLCvyo7gZASJ7HPDRIw//bF+udP899z72zNO1RpXyxDENbNBqrW6bjhCgtBIqN6VyTZqmqfvt+r7faDSUeKFaraqwnQpUJUkShqEipa7rmqbpeZ5hGCoCSCk1DMM0TRUNtG17aGjItu1yuaxYjfJWUspcLqe+FCE0OjpaLpcVy81ms8rSKuLqeV4QBIVCIZvN1mo1pTgghACA53ldXV2VSkXTNMUMVXRMOQKEkKZpaifKquu6ns/nVa5JcSeliWCMVSoV27Ydx6nVavl8vlqt9vf3r1mzhjE2ffp0xli1Wu3o6Gg0GrZtqx9OXScqwKcuGMVRlUdWZMx13Ww2q9JfUspardbV1VWvN485+vN77LFnsVi486d3/O53vxGSUUqGh4dViHCiXktltxCABCRS3pUiRYoU7zk20q0kSVQkVUqpfHNnZydjLI7jIAgUXbEsC2Psuq6Kv6r0C2MsSRLOubp1lVo3WJbleZ7ruk42o8KxvuuZppmxHQBgjDWbTaWmsBzbtu1qvUYptW27Uas6jgMAzabb1dXluj4AGIZRr9dVVDifz8Zx3Go1Zs6c6bpuHDPFzdQ36rpuGIbyykrZOOmowjDUKCYgNU0DQqmmBUHiep7pZKimA4BGiUjiIPR4lOTymaxtua7bcL044YVCQS1TCCGqEbwKP6tjsSyLUso5b7VacRz39/e3Wi0lfVEpr3K57Pt+R0dHvV5X511pOUql0uDgoGmalmVFUeS6bldXl2VZIyMj6uSEYaj6yE/wT8jaxcHhDdQWXHJdsywjFzXDzmIh9hqMhYxARCQU9Bqv7/fZfUI7bOleRNNG8ClSvPfgCEcUI4nsRBIJEiDBEFKCJM0KyxjnrzzwX50sL5qAkNGSyHIy0vNsXVOJCGWd1P9xFWSJokixL13XVcpdLbgJIWqtr1TK/f39k38yxjjnQgjOeblcrtebaoGu67pqXN7RUWw0GgBAKY3jUImrgyBQTEZRC8aYYg6GYeRyhWq1qmxOPp8nhNTrTUJQoVDyfde2NNf3Ii6QkB1OqeW6IUa6bkSVRl9fbz3xAEsL0SjwqKVxIdyml8sVLMsaHx9XeSfXVXe1qqs4XS6Xs2270WgEQZDP55Mk8TxPqbXjONZ13bIslctSygUppRK9AwBjTKkxVSWtYRhjY2PK7CsJRrVaDcOwWCyq6FUQBNOmTavVaiqDpMyvZVmZTKbVainZtlJ9M8bU16kAommaiuFgjBU5BABN09TJV2fPdV1FXNVnXdcdGhqyLKuzs1NxoWXLls2aNatQKKxbt04JNKIoUrp9lStTBp8QomJzmqY1m03lYdXO1VGo3JoQQvk7KWWz2SyVSpxzFVhsNBpKD+n7Ya3amDt3br1Rdd2maZpxHBZLeSF4GIYTXAvat99Uz1O6lSJFihTvA9Cuu+6qarFUhYDy3IZhMMaEEK7r5vN5AFCUScXbFKcql8tRFKkorHKKtVpNeQLl8wqFAkJI8QRFJBhjg4ODSiIPALZt+76v8mCKsClvp0oUms1md3d3tVpV36jinYquqIbs4+PjmUxGFTAUi0XFDBlj2WxWhQYV6VLKEyWeCXzX0rVWq8UkIIw1zaKaxgGpyO7Y2Fi5WDBMTcRMSBZ6biaT0S275fqKXCmhoJq2rusYYxX7VGsXSqnjOKpuTa0GdF1XwUgVnXUcx/M8JfdXdw8zTbNYLAZBoCSLSg8ThqGU0vf9zs5OtVuEkFoMVSo1lsgZM6bHELRajTCMDWpkzFx9rFLIOEkSMSKkiYWNaqz+8aM+Vtcb9Wwj1KJ/4BWWIsW/LKQGwiJCMxgQKQUChiEiBCQ1E5INtBfueyoT6CgkhFi1iNm2TePEQCgIAiWNU0VEAKDkxBhjlWlXiQsAUItvZR6VtLjZbCp7q/qjTtamEkJUDZjv+wCglumqukllzEqlUhAEqqpH0TZNI1EUcZE4jkMprVaruq6DxEoNrmqBpJQIEU0jnhdgDFTDAKAbVhSEWTPnBj6xLIQQCpNisThSG/Mjv5jNCZ7ELAKArs6edevWE0LUtytxhGmakywiiiJFLxVvUUk8paAulUr1eh1jLKVUsS3lbgBA13VV0dRsNqdPnx7H8ejoqOM4ih1NlhlLKZUTUToCZXWVDF6dNMMwFMXNZDLK6irhnyqyVfVdirzVarWOjo4gCJQe0vM8RbGUVMEwDJWENAxjfHwcY9zb26u06Gqetm2rHJT6obu7u4MgUKE6Zf+VklDFPVXoUF0b6iJR7CuKIk3TAMDzvFar1dfXp7Jq6mIcGRmZNWtWo9Hwfb9cLtdqNdO0Dd2K47jeqPb2dpumOTi4HmGpnBcApHQrRYoUKf4+QLvvvrtyAIQQRXuSJFEsRcX5TNNsNBoq7qgianPmzBkaGlKMwnEcpUKcTCURQpSXAgBV9h1FUVdX1/DwcJIk06ZNU5RJ0SSln0EIOY4ThqHiFUq1r5yZKnZS9EZJ7Rljij5NkkNVNq3CfpPuXHVmb7VaruuWSiXl2guFQhh4hCDTtIMwRggBIp4XmI7NOXccy2+5pqElSSJ40lnqGBkfw0TTNE3pBjOZjJIAmaap/KIKKCqln4pGq5WNWkvFcayWSurEqpOj6F+hUGi1WrlcTmn98/m87/tCiCiKVOMNxcoU3VW7SpKkVCoLDtV6TdORYRuUUh4nsc8ylk0EjkUMWHLCPRS0uHvY8YfVRS3MRBFNMIAASMd0TMf3cARJkNCJIDoHJIFjyTCKMQagBiNFYf3m7kdxkxtgOZniuB8YhqFxgQVX5Uyjo6MqHxJFUSaTqdfrpmkSQiZTJUIIZY3V7d0BIIoiRVSUeFupDdU6XjXMUFZRrcI1TVMCvEm5eLswNY6VhAEhxHkigSsxoZLSuS1fTUAxPZUtsW1b2T0JnHNu6Faz6WZsJ4hi07DDOHJMq9ao2ratmxqLE84TFbZrNl3VKGLS4Cu9uioVVkTCdV2VQVLxsskwnMp6qVSebduqNJdSqogQY8xxHJX3y2azQRAo+6wOUzXbGB8fV7ZafW+1Ws1kMorUqRAbxlidMWWflecSQui6XqlUenp6lIpepZsUM1SJREXq6vW64mCTnFa1OFKFVUqoqeqWfd8vFArj4+O9vb2+76vTm81mfd9XgTaVXpuk3OpnUleFor6qOYoSUqqjUEFJFeabzI6q64RzriYAgJUig7HY87x8PqvrerNVVw4aJFYXMgAGiVOulSJFihTvE9Buu+32j57D3xVIuROJJQIAkJM3eWy/CxP+RqieuBIw/DNh43wmXCOWgAUFIAAgkBQ4TkiS0ISRGABjrgFgLEEgSMd0TMf3csRckAgBI1IAAEeYYYgxBsCUEyuhTkyMhCKpSaAxRgIDgn+2zm8CQBmTvw1yYnWujCcAFhO2FABLABASwZRj/Oeyn//3MHn+xZTxne9OPv+nujhTpEiR4l8E9B89gX8M5MT9RZCETftITF0l/FM6HtWHGkAFIxEIiUAgAVIDwBIESIohoQIhqRNBjMShTPtHTzpFin9F4ASwByhRHEPRLYYxSIQl1hk2GKGcKhpCEICYGtv5JwFuT+tvZ1zKbKJ2ig9LEAACKQoqMAAHkAgj+U9pP/+P4p/er6VIkSLFvzTed7qFJtjMxmzSu20wZbO/HI2DjeKHTTb7W/HObBX6M3v4Z8trTUAIBFhSaM+cw+RJk1gCAGAkMAGMAGOJARin7B853xQp/kVBpNAZJRILBBwBQRhhIBgEAJGICAwSc4QB2htI9HfoEfrnDOY7n09urKzHpu+8C/uSAEgiQBJjACFBYhASEAgMwBBIBFgxMXUGACtWluIfir98PfzlLVOkSJEixXuDf3x2SyLA/zM7L/4Zpv0PgVQCQtQWQQK0V0WyHTYnAACAMAAIIJIgiQEJCbLd5Dcd0zEd38NRApIIpAYAKrslAUAChjZb4arXKwiOgSMMIDCIvwfn+lsxVT+2qZbsb853qeyWRCDQez+/FClSpEiR4l8Af48EzjsDqpOPd9/sbYIbJDY+puKfT5fz94BarEmMJJpYt3FAUgAWgAUgtQmWQAUlgiJ1Q5V0TMd0fK9HjnCCaUSwT7FPcUBxRHCMcYIxwzjGOCEiIiLQRERFQmNG2N8jg/Cu1nIqJG4/2sDveExstul+Nz6TgAEEAona1maT55AmtVKkSJEiRYqN+H+R9vKbyhugpQAAAABJRU5ErkJggg==",
            "text/plain": [
              "<PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=1146x29>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            " CommitInfo(commit_url='https://huggingface.co/gopiashokan/whisper-small-mr/commit/6d57dc68223efe0e33923f9753be04039359e302', commit_message='Upload\n",
            " WhisperForConditionalGeneration', commit_description='', oid='6d57dc68223efe0e33923f9753be04039359e302', pr_url=None, pr_revision=None, pr_num=None)\n"
          ]
        }
      ],
      "source": [
        "peft_model_id = \"gopiashokan/whisper-small-mr\"\n",
        "\n",
        "model.push_to_hub(peft_model_id)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wnCpeDPYiRKo"
      },
      "source": [
        "## **Conclusion**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LnHM8eJYjqhk"
      },
      "outputs": [],
      "source": [
        "\"\"\"\n",
        "The Whisper-large-v3 model exhibits superior accuracy when compared to other models. However, a significant challenge arises\n",
        "as training the model with a custom dataset and fine-tuning in Google Colab leads to automatic crashes beyond a certain limit.\n",
        "To overcome this limitation, we opt for the Whisper-small model for training and fine-tuning with a custom dataset.\n",
        "\n",
        "\"\"\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gGmeO6rMlHWw"
      },
      "source": [
        "**Note**: *Performance of the Whisper-small model (WER = 3.38) is comparatively poorer than the Whisper-large-v3 model (WER = 0.74)*"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6vVvDBVeUMf5"
      },
      "outputs": [],
      "source": [
        "\"\"\"\n",
        "We trained the model with a limit of only 100 steps due to the constraints of Google Colab, as it supports training up to this\n",
        "level before encountering GPU limitations and automatic crashes. To further enhance the model's understanding of patterns and\n",
        "improve prediction performance, additional training steps are required. However, this extends beyond the capabilities of the\n",
        "Google Colab, necessitating alternative resources or platforms for extended training.\n",
        "\n",
        "\"\"\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GSys3_K6i_x-"
      },
      "outputs": [],
      "source": [
        "\"\"\"\n",
        "In the future, plan to leverage cloud platforms such as AWS or Azure to train the Whisper-large-v3 model with a custom dataset and\n",
        "conduct fine-tuning. This approach will provide the necessary resources and scalability to train the model extensively, allowing us\n",
        "to reach maximum efficiency and enhance its predictive capabilities.\n",
        "\n",
        "\"\"\""
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [
        "6Yf6o9Q-aLN4",
        "ZEBGrHWampEz"
      ],
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3.11.6"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "01d2362c93ae497b8e8b361aca19d35b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "040ff04866a04a378fe264897c48fe1d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_373ac4ecc7e947c1aca4e370db5c7c9c",
            "max": 1967,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_392c1aa77b7d492c9b3fd56c1c324984",
            "value": 1967
          }
        },
        "07292274632347e2b0b0aeac47d6baaa": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_93ffd5a5d9a041b2895686cb7284eee4",
            "max": 2077,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_71e1fa7c672b4ec8980b3c6607855db4",
            "value": 2077
          }
        },
        "0b847167dc8942358dfaa68cc4ababf6": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0bb723e5d75246159234eee16baaa781": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_e9b634c8aa404ea78b5c2cb9311229bb",
              "IPY_MODEL_cafd4deb033d40dcb3671552df04ef40",
              "IPY_MODEL_ed805508a031401290fa9554c4c44d9c"
            ],
            "layout": "IPY_MODEL_c89343f2708b433094bbbeff1534c5ca"
          }
        },
        "0bdef1463e594e3196041c399fab92d4": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "0ccebef71e774900863a4e6cc915363b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "0f5d0ff6ff8c4d34a594ba9a3c9cc903": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "103a1ed38f424e64aff85d771566a1db": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a2b3b59dc40b4cba9fe996e17875eae0",
            "max": 805,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_87c40b104e834ee5b59685841736f5fa",
            "value": 805
          }
        },
        "11d336127b494cc08bde6bad094b8bcd": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8e0a15c8f01f457bbf46d611627f54e6",
            "placeholder": "​",
            "style": "IPY_MODEL_2ab127ce82ab4155a32de0af06369350",
            "value": "config.json: 100%"
          }
        },
        "14781f7e07ac45c69f0176ea01ee57a1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_1b20a82e4b3a4a899f86ee4b8b7452f1",
              "IPY_MODEL_9e5815c674a0405884371b5892cf4e9c",
              "IPY_MODEL_265f3a837eeb4a4bae58ea6e7cf45f09"
            ],
            "layout": "IPY_MODEL_2eb5683786994286a38ff74ce1f481c9"
          }
        },
        "16b69a1340694468bc031fcd12085bed": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "16fcab05339f42919a3dbd18af305558": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "18eefc5848e14a5daded5c96df890a4b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0b847167dc8942358dfaa68cc4ababf6",
            "placeholder": "​",
            "style": "IPY_MODEL_c1a8ce5566e24ee1ab10b4a3526b548c",
            "value": "vocab.json: 100%"
          }
        },
        "19542a0c07274597b61cb89147edb511": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c655c51b0cc147dba960953631c4b2fe",
            "placeholder": "​",
            "style": "IPY_MODEL_d8a1459942704b0196555bea514c1f0b",
            "value": "preprocessor_config.json: 100%"
          }
        },
        "1b20a82e4b3a4a899f86ee4b8b7452f1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6643d1a3a02a466d9c7b4a40edcebe62",
            "placeholder": "​",
            "style": "IPY_MODEL_cd1beb28bab042f4b0e9ad46a3767b2e",
            "value": "model.safetensors: 100%"
          }
        },
        "1d0545478a3d4760a2933b5729093a49": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8a1d4d65d30d4869953032aed5b12e2d",
            "placeholder": "​",
            "style": "IPY_MODEL_97f48db2927a416b81966668b02899bc",
            "value": "normalizer.json: 100%"
          }
        },
        "237591c827644c3586a1262defe1bad2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_aaef468a9eca46a28f51a07022ecfdbd",
            "max": 52666,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_2b86d64a3f1f43beb8c1dc3f0283abe2",
            "value": 52666
          }
        },
        "24177ad4235f472fb43b779167d17f22": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "265f3a837eeb4a4bae58ea6e7cf45f09": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_752486ac9a55459ea6d3660ac8bc1582",
            "placeholder": "​",
            "style": "IPY_MODEL_81b2a61e619543089143e2938378dab6",
            "value": " 967M/967M [00:08&lt;00:00, 171MB/s]"
          }
        },
        "26a1f8913a9b476b8c5fae4922c8a1ac": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "291f07fe0757470a8810158b5c3cb1ea": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3150ce6fb99a4840b94dba36c6da87d8",
            "max": 835550,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_6da15992e40f44eb9df8b14db572da71",
            "value": 835550
          }
        },
        "2ab127ce82ab4155a32de0af06369350": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "2af26dc88ee340498da24e360494e0da": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "2b86d64a3f1f43beb8c1dc3f0283abe2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "2eb5683786994286a38ff74ce1f481c9": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "30865422703e4506a1c18a2c3944458f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3150ce6fb99a4840b94dba36c6da87d8": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "316895c9699443e6aa23cd14ca5134ae": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_11d336127b494cc08bde6bad094b8bcd",
              "IPY_MODEL_040ff04866a04a378fe264897c48fe1d",
              "IPY_MODEL_e00e388b82064cdfa4100aa247646bb2"
            ],
            "layout": "IPY_MODEL_dcb11b10112f45ea8a0d70967181df4c"
          }
        },
        "339260f42cba4e47808a9eb0f5d29293": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "35d0aaeaea6b4c3f979dd2e5cade400d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "373ac4ecc7e947c1aca4e370db5c7c9c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "377bf3a8ae064682a67e534628c63321": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "392c1aa77b7d492c9b3fd56c1c324984": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "3ba87c4757554de6bf7b395a8dbffaee": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "44c3602d0da24bd0b50b0317dbdae9ee": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0f5d0ff6ff8c4d34a594ba9a3c9cc903",
            "max": 2480452,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_b6637153c079490c9cd1b4cb6fd71576",
            "value": 2480452
          }
        },
        "493a7529bab24b83bcd45ce46ea950cc": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4a98cbabf3864ac5839ed3904d5e9c2e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4c2d0b018a8f40c7ab05fd3ef04358a5": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4ce92585c6834f248494bb66d6add9e0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4d3037992a0943dea1ac8ebc01d607fd": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4e89dc24a66f43dbbe2454cab6a81cc3": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "565e1a7a1e834dc1940871b48ed0cf05": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c496515361be49e19637024cecbecfc5",
            "placeholder": "​",
            "style": "IPY_MODEL_377bf3a8ae064682a67e534628c63321",
            "value": " 3.84k/3.84k [00:00&lt;00:00, 151kB/s]"
          }
        },
        "59269ece94704706a4dbc998994cb26a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_339260f42cba4e47808a9eb0f5d29293",
            "placeholder": "​",
            "style": "IPY_MODEL_0ccebef71e774900863a4e6cc915363b",
            "value": "generation_config.json: 100%"
          }
        },
        "5afc806447034a79831986a44349432e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6643d1a3a02a466d9c7b4a40edcebe62": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "69f0c2459d4145be838e244dccff2c67": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8738e1c8eec440f8a9a17916d0558f78",
            "placeholder": "​",
            "style": "IPY_MODEL_4ce92585c6834f248494bb66d6add9e0",
            "value": " 2.08k/2.08k [00:00&lt;00:00, 131kB/s]"
          }
        },
        "6a6d8cfd55844b58bdb217f663374329": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6da15992e40f44eb9df8b14db572da71": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "6e2d5697283a43c6abbe3a02dd6515ff": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_eb7bd663832f4d37b18c2a989d622ab2",
            "max": 3837,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_2af26dc88ee340498da24e360494e0da",
            "value": 3837
          }
        },
        "71e1fa7c672b4ec8980b3c6607855db4": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "72739990179e402cb4cf8346cb5533e1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a56833bb76d341d0bbbcb464f459b75f",
            "placeholder": "​",
            "style": "IPY_MODEL_3ba87c4757554de6bf7b395a8dbffaee",
            "value": " 185k/185k [00:00&lt;00:00, 1.43MB/s]"
          }
        },
        "72a88a64928048899fee7f685300fae3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_bcacde86517f499cbfc918cbc9776505",
              "IPY_MODEL_103a1ed38f424e64aff85d771566a1db",
              "IPY_MODEL_be94f03f82be421bb59cd70dc7c53d08"
            ],
            "layout": "IPY_MODEL_80281306028f40dfa8f42a111ee4cad4"
          }
        },
        "752486ac9a55459ea6d3660ac8bc1582": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "75da0aa6063f42578f7fddb2b7b17fe0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "7adaf4b6052646bfb4b7d9662d47dc92": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d3966334b63b4763a8999dca9256f76e",
            "placeholder": "​",
            "style": "IPY_MODEL_30865422703e4506a1c18a2c3944458f",
            "value": " 2.48M/2.48M [00:00&lt;00:00, 9.22MB/s]"
          }
        },
        "7b524a30905c418d93a8750c6e509c60": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_18eefc5848e14a5daded5c96df890a4b",
              "IPY_MODEL_291f07fe0757470a8810158b5c3cb1ea",
              "IPY_MODEL_9cce3418066a402dbbd91aec9d80e9ca"
            ],
            "layout": "IPY_MODEL_4d3037992a0943dea1ac8ebc01d607fd"
          }
        },
        "7bc579469cbe463fa9dbfb66132b3077": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7ce7e883fd1d4051ac117e6f9671a192": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_aed0cf8aefb841869179109343d98bb8",
            "max": 184990,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_b6536d6a41bb4cc2b3e598c0d498e6e2",
            "value": 184990
          }
        },
        "7fdf14e114c24af5b8ca4d2662907bf2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "80281306028f40dfa8f42a111ee4cad4": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "807d7a13758c4745905838e1efeed07e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_493a7529bab24b83bcd45ce46ea950cc",
            "placeholder": "​",
            "style": "IPY_MODEL_4a98cbabf3864ac5839ed3904d5e9c2e",
            "value": " 494k/494k [00:00&lt;00:00, 3.79MB/s]"
          }
        },
        "81b2a61e619543089143e2938378dab6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "84e970c8a81c46d9889cfd899ec98a3a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8738e1c8eec440f8a9a17916d0558f78": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "87c40b104e834ee5b59685841736f5fa": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "87c74966afd44040b85e7f5af26bbe6b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8a1d4d65d30d4869953032aed5b12e2d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8b0135f3c8fb4c8fa43865aa7cdbcf94": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8bd1ad8f732241478ce1d117a9aa0901": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_d02b2ae4786246669f7b4b111093709b",
              "IPY_MODEL_07292274632347e2b0b0aeac47d6baaa",
              "IPY_MODEL_69f0c2459d4145be838e244dccff2c67"
            ],
            "layout": "IPY_MODEL_7bc579469cbe463fa9dbfb66132b3077"
          }
        },
        "8e0a15c8f01f457bbf46d611627f54e6": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "93ffd5a5d9a041b2895686cb7284eee4": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "97f48db2927a416b81966668b02899bc": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9cce3418066a402dbbd91aec9d80e9ca": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_35d0aaeaea6b4c3f979dd2e5cade400d",
            "placeholder": "​",
            "style": "IPY_MODEL_7fdf14e114c24af5b8ca4d2662907bf2",
            "value": " 836k/836k [00:00&lt;00:00, 3.14MB/s]"
          }
        },
        "9e5815c674a0405884371b5892cf4e9c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_fd9b8a824e8942308ec3a597333c812c",
            "max": 966995080,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_26a1f8913a9b476b8c5fae4922c8a1ac",
            "value": 966995080
          }
        },
        "a0c9e104643e411c81b80cd6d0a14b6f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a2b3b59dc40b4cba9fe996e17875eae0": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a311edf20529451f926f60164ccd3ca5": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a54724b5746e4e6fa12cec269870dbc1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_87c74966afd44040b85e7f5af26bbe6b",
            "placeholder": "​",
            "style": "IPY_MODEL_16b69a1340694468bc031fcd12085bed",
            "value": "tokenizer.json: 100%"
          }
        },
        "a56833bb76d341d0bbbcb464f459b75f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a6c802a39b4b44a582ca6c5653e9121c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "a81ddd14471d45d0b0d33f6b063dc2b5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f64b28e8777f47159d1573472af9f98a",
            "placeholder": "​",
            "style": "IPY_MODEL_ba2b09fcb49045fbadae79ebb99fbea0",
            "value": " 52.7k/52.7k [00:00&lt;00:00, 3.26MB/s]"
          }
        },
        "aaef468a9eca46a28f51a07022ecfdbd": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "aed0cf8aefb841869179109343d98bb8": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b540827cc46142a58c23a6ebc55b3bbb": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "b6536d6a41bb4cc2b3e598c0d498e6e2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "b6637153c079490c9cd1b4cb6fd71576": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "ba2b09fcb49045fbadae79ebb99fbea0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "bcacde86517f499cbfc918cbc9776505": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_dd47ab93e08b432fa2cbbc4996a7581a",
            "placeholder": "​",
            "style": "IPY_MODEL_d1ad87d0ae2e4c57bdc0bb5df4f7bea7",
            "value": "tokenizer_config.json: 100%"
          }
        },
        "be94f03f82be421bb59cd70dc7c53d08": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c6f576a6b2b243c0a1205e0945aab671",
            "placeholder": "​",
            "style": "IPY_MODEL_0bdef1463e594e3196041c399fab92d4",
            "value": " 805/805 [00:00&lt;00:00, 52.2kB/s]"
          }
        },
        "bf20a70ce0e64228b10e79c4d5ddc092": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c0943b08147346898d4c6ffc792f3006": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_bf20a70ce0e64228b10e79c4d5ddc092",
            "placeholder": "​",
            "style": "IPY_MODEL_df7342a39768493995c322f2bc65ccbc",
            "value": "merges.txt: 100%"
          }
        },
        "c1a8ce5566e24ee1ab10b4a3526b548c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c321be47b3804ac0b9a0ef3066d0f66c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_c0943b08147346898d4c6ffc792f3006",
              "IPY_MODEL_df10da28383e47f99137bf5bcaa6a77a",
              "IPY_MODEL_807d7a13758c4745905838e1efeed07e"
            ],
            "layout": "IPY_MODEL_d23fbc35736f480eac6f3389995738ea"
          }
        },
        "c496515361be49e19637024cecbecfc5": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c655c51b0cc147dba960953631c4b2fe": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c6f576a6b2b243c0a1205e0945aab671": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c89343f2708b433094bbbeff1534c5ca": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "cafd4deb033d40dcb3671552df04ef40": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4e89dc24a66f43dbbe2454cab6a81cc3",
            "max": 34604,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_a6c802a39b4b44a582ca6c5653e9121c",
            "value": 34604
          }
        },
        "cd1beb28bab042f4b0e9ad46a3767b2e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d02b2ae4786246669f7b4b111093709b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e26435ac63e142c197c81c2660d0891f",
            "placeholder": "​",
            "style": "IPY_MODEL_5afc806447034a79831986a44349432e",
            "value": "special_tokens_map.json: 100%"
          }
        },
        "d1ad87d0ae2e4c57bdc0bb5df4f7bea7": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d23fbc35736f480eac6f3389995738ea": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d3966334b63b4763a8999dca9256f76e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d410daa4aee04a98bfb8d5300dc41b77": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_1d0545478a3d4760a2933b5729093a49",
              "IPY_MODEL_237591c827644c3586a1262defe1bad2",
              "IPY_MODEL_a81ddd14471d45d0b0d33f6b063dc2b5"
            ],
            "layout": "IPY_MODEL_01d2362c93ae497b8e8b361aca19d35b"
          }
        },
        "d8a1459942704b0196555bea514c1f0b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "db24725c84744c1983cb8f92c1309f36": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_19542a0c07274597b61cb89147edb511",
              "IPY_MODEL_7ce7e883fd1d4051ac117e6f9671a192",
              "IPY_MODEL_72739990179e402cb4cf8346cb5533e1"
            ],
            "layout": "IPY_MODEL_84e970c8a81c46d9889cfd899ec98a3a"
          }
        },
        "dcb11b10112f45ea8a0d70967181df4c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "dd47ab93e08b432fa2cbbc4996a7581a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "df10da28383e47f99137bf5bcaa6a77a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a311edf20529451f926f60164ccd3ca5",
            "max": 493864,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_b540827cc46142a58c23a6ebc55b3bbb",
            "value": 493864
          }
        },
        "df7342a39768493995c322f2bc65ccbc": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e00e388b82064cdfa4100aa247646bb2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8b0135f3c8fb4c8fa43865aa7cdbcf94",
            "placeholder": "​",
            "style": "IPY_MODEL_fbf956eb2a1848669a3976e584e02260",
            "value": " 1.97k/1.97k [00:00&lt;00:00, 52.4kB/s]"
          }
        },
        "e26435ac63e142c197c81c2660d0891f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e9b634c8aa404ea78b5c2cb9311229bb": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6a6d8cfd55844b58bdb217f663374329",
            "placeholder": "​",
            "style": "IPY_MODEL_75da0aa6063f42578f7fddb2b7b17fe0",
            "value": "added_tokens.json: 100%"
          }
        },
        "eb7bd663832f4d37b18c2a989d622ab2": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ed805508a031401290fa9554c4c44d9c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4c2d0b018a8f40c7ab05fd3ef04358a5",
            "placeholder": "​",
            "style": "IPY_MODEL_a0c9e104643e411c81b80cd6d0a14b6f",
            "value": " 34.6k/34.6k [00:00&lt;00:00, 549kB/s]"
          }
        },
        "f12429ab6f874b65b7bdae0227effb3e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_a54724b5746e4e6fa12cec269870dbc1",
              "IPY_MODEL_44c3602d0da24bd0b50b0317dbdae9ee",
              "IPY_MODEL_7adaf4b6052646bfb4b7d9662d47dc92"
            ],
            "layout": "IPY_MODEL_24177ad4235f472fb43b779167d17f22"
          }
        },
        "f64b28e8777f47159d1573472af9f98a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "fbf956eb2a1848669a3976e584e02260": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "fd9b8a824e8942308ec3a597333c812c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ff27b38667e146758c69131a4f177483": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_59269ece94704706a4dbc998994cb26a",
              "IPY_MODEL_6e2d5697283a43c6abbe3a02dd6515ff",
              "IPY_MODEL_565e1a7a1e834dc1940871b48ed0cf05"
            ],
            "layout": "IPY_MODEL_16fcab05339f42919a3dbd18af305558"
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}