BigQuery_Palm_Redis.ipynb

{
  "cells": [
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "aiHV6ip2f5id"
      },
      "source": [
        "# LLM Reference Architecture using Redis & Google Cloud Platform\n",
        "\n",
        "<a href=\"https://colab.research.google.com/github/RedisVentures/redis-google-llms/blob/main/BigQuery_Palm_Redis.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
        "\n",
        "This notebook serves as a getting started guide for working with LLMs on Google Cloud Platform with Redis Enterprise.\n",
        "\n",
        "## Intro\n",
        "Google's Vertex AI has expanded its capabilities by introducing [Generative AI](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/overview). This advanced technology comes with a specialized [in-console studio experience](https://cloud.google.com/vertex-ai/docs/generative-ai/start/quickstarts/quickstart), a [dedicated API](https://cloud.google.com/vertex-ai/docs/generative-ai/start/quickstarts/api-quickstart) and [Python SDK](https://cloud.google.com/vertex-ai/docs/python-sdk/use-vertex-ai-python-sdk) designed for deploying and managing instances of Google's powerful PaLM language models (more sample code). With a distinct focus on text generation, summarization, chat completion, and embedding creation, PaLM models are reshaping the boundaries of natural language processing and machine learning.\n",
        "\n",
        "Redis Enterprise offers robust vector database features, with an efficient API for vector index creation, management, distance metric selection, similarity search, and hybrid filtering. When coupled with its versatile data structures - including lists, hashes, JSON, and sets - Redis Enterprise shines as the optimal solution for crafting high-quality Large Language Model (LLM)-based applications. It embodies a streamlined architecture and exceptional performance, making it an instrumental tool for production environments.\n",
        "\n",
        "![](https://github.com/RedisVentures/redis-google-llms/blob/main/assets/GCP_RE_GenAI.drawio.png?raw=true)\n",
        "\n",
        "Below we will work through several design patterns with Vertex AI LLMs and Redis Enterprise that will ensure optimal production performance.\n",
        "\n",
        "___\n",
        "## Contents\n",
        "- Setup\n",
        "    1. Prerequisites\n",
        "    2. Create BigQuery Table\n",
        "    3. Generate Embeddings\n",
        "        \n",
        "        a. Embed Text Data\n",
        "\n",
        "    4. Load Embeddings to Redis\n",
        "    5. Create Index\n",
        "- Build LLM Applications\n",
        "- LLM Design Patterns\n",
        "    1. Semantic Search\n",
        "    2. Retrieval Augmented Generation (RAG)\n",
        "    3. Caching\n",
        "    4. Memory\n",
        "- Cleanup\n",
        "\n",
        "___"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rK2rWODkw-kX"
      },
      "source": [
        "# Setup"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "37rbBPKdL09o"
      },
      "source": [
        "## 1. Prerequisites\n",
        "Before we begin, we must install some required libraries, authenticate with Google, create a Redis database, and initialize other required components.\n",
        "\n",
        "### Install required libraries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 903
        },
        "id": "pc-IxYu3wnQm",
        "outputId": "cc60eb72-489a-4b42-9c7b-b2130a70c370"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Collecting redis\n",
            "  Downloading redis-4.6.0-py3-none-any.whl (241 kB)\n",
            "\u001b[?25l     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/241.1 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K     \u001b[91m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[90m╺\u001b[0m \u001b[32m235.5/241.1 kB\u001b[0m \u001b[31m6.9 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m241.1/241.1 kB\u001b[0m \u001b[31m5.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting google-cloud-aiplatform==1.25.0\n",
            "  Downloading google_cloud_aiplatform-1.25.0-py2.py3-none-any.whl (2.6 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.6/2.6 MB\u001b[0m \u001b[31m47.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0 in /usr/local/lib/python3.10/dist-packages (from google-cloud-aiplatform==1.25.0) (2.11.1)\n",
            "Requirement already satisfied: proto-plus<2.0.0dev,>=1.22.0 in /usr/local/lib/python3.10/dist-packages (from google-cloud-aiplatform==1.25.0) (1.22.3)\n",
            "Requirement already satisfied: protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5 in /usr/local/lib/python3.10/dist-packages (from google-cloud-aiplatform==1.25.0) (3.20.3)\n",
            "Requirement already satisfied: packaging>=14.3 in /usr/local/lib/python3.10/dist-packages (from google-cloud-aiplatform==1.25.0) (23.1)\n",
            "Requirement already satisfied: google-cloud-storage<3.0.0dev,>=1.32.0 in /usr/local/lib/python3.10/dist-packages (from google-cloud-aiplatform==1.25.0) (2.8.0)\n",
            "Requirement already satisfied: google-cloud-bigquery<4.0.0dev,>=1.15.0 in /usr/local/lib/python3.10/dist-packages (from google-cloud-aiplatform==1.25.0) (3.10.0)\n",
            "Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform==1.25.0)\n",
            "  Downloading google_cloud_resource_manager-1.10.2-py2.py3-none-any.whl (321 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m321.3/321.3 kB\u001b[0m \u001b[31m38.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting shapely<2.0.0 (from google-cloud-aiplatform==1.25.0)\n",
            "  Downloading Shapely-1.8.5.post1-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.0 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.0/2.0 MB\u001b[0m \u001b[31m64.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: async-timeout>=4.0.2 in /usr/local/lib/python3.10/dist-packages (from redis) (4.0.2)\n",
            "Requirement already satisfied: googleapis-common-protos<2.0.dev0,>=1.56.2 in /usr/local/lib/python3.10/dist-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform==1.25.0) (1.59.1)\n",
            "Requirement already satisfied: google-auth<3.0.dev0,>=2.14.1 in /usr/local/lib/python3.10/dist-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform==1.25.0) (2.17.3)\n",
            "Requirement already satisfied: requests<3.0.0.dev0,>=2.18.0 in /usr/local/lib/python3.10/dist-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform==1.25.0) (2.27.1)\n",
            "Requirement already satisfied: grpcio<2.0dev,>=1.33.2 in /usr/local/lib/python3.10/dist-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform==1.25.0) (1.56.0)\n",
            "Requirement already satisfied: grpcio-status<2.0.dev0,>=1.33.2 in /usr/local/lib/python3.10/dist-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform==1.25.0) (1.48.2)\n",
            "Requirement already satisfied: google-cloud-core<3.0.0dev,>=1.6.0 in /usr/local/lib/python3.10/dist-packages (from google-cloud-bigquery<4.0.0dev,>=1.15.0->google-cloud-aiplatform==1.25.0) (2.3.3)\n",
            "Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in /usr/local/lib/python3.10/dist-packages (from google-cloud-bigquery<4.0.0dev,>=1.15.0->google-cloud-aiplatform==1.25.0) (2.5.0)\n",
            "Requirement already satisfied: python-dateutil<3.0dev,>=2.7.2 in /usr/local/lib/python3.10/dist-packages (from google-cloud-bigquery<4.0.0dev,>=1.15.0->google-cloud-aiplatform==1.25.0) (2.8.2)\n",
            "Requirement already satisfied: grpc-google-iam-v1<1.0.0dev,>=0.12.4 in /usr/local/lib/python3.10/dist-packages (from google-cloud-resource-manager<3.0.0dev,>=1.3.3->google-cloud-aiplatform==1.25.0) (0.12.6)\n",
            "Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from google-auth<3.0.dev0,>=2.14.1->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform==1.25.0) (5.3.1)\n",
            "Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from google-auth<3.0.dev0,>=2.14.1->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform==1.25.0) (0.3.0)\n",
            "Requirement already satisfied: six>=1.9.0 in /usr/local/lib/python3.10/dist-packages (from google-auth<3.0.dev0,>=2.14.1->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform==1.25.0) (1.16.0)\n",
            "Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.10/dist-packages (from google-auth<3.0.dev0,>=2.14.1->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform==1.25.0) (4.9)\n",
            "Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in /usr/local/lib/python3.10/dist-packages (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4.0.0dev,>=1.15.0->google-cloud-aiplatform==1.25.0) (1.5.0)\n",
            "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0.dev0,>=2.18.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform==1.25.0) (1.26.16)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0.dev0,>=2.18.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform==1.25.0) (2023.5.7)\n",
            "Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0.dev0,>=2.18.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform==1.25.0) (2.0.12)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0.dev0,>=2.18.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform==1.25.0) (3.4)\n",
            "Requirement already satisfied: pyasn1<0.6.0,>=0.4.6 in /usr/local/lib/python3.10/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3.0.dev0,>=2.14.1->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform==1.25.0) (0.5.0)\n",
            "Installing collected packages: shapely, redis, google-cloud-resource-manager, google-cloud-aiplatform\n",
            "\u001b[33m  WARNING: The script tb-gcp-uploader is installed in '/root/.local/bin' which is not on PATH.\n",
            "  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.\u001b[0m\u001b[33m\n",
            "\u001b[0mSuccessfully installed google-cloud-aiplatform-1.25.0 google-cloud-resource-manager-1.10.2 redis-4.6.0 shapely-1.8.5.post1\n"
          ]
        },
        {
          "data": {
            "application/vnd.colab-display-data+json": {
              "pip_warning": {
                "packages": [
                  "google"
                ]
              }
            }
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "!pip install redis \"google-cloud-aiplatform==1.25.0\" --upgrade --user"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p5kx9ePDwwp6"
      },
      "source": [
        "^^^ If prompted press the Restart button to restart the kernel. ^^^\n",
        "\n",
        "### Install Redis locally (optional)\n",
        "If you have a Redis db running elsewhere with [Redis Stack](https://redis.io/docs/about/about-stack/) installed, you don't need to run it on this machine. You can skip to the \"Connect to Redis server\" step."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vs4KZURX4XpT",
        "outputId": "82860908-183e-46fa-ce8a-9541ec4ccdf4"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb focal main\n",
            "Starting redis-stack-server, database path /var/lib/redis-stack\n"
          ]
        }
      ],
      "source": [
        "%%sh\n",
        "curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg\n",
        "echo \"deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main\" | sudo tee /etc/apt/sources.list.d/redis.list\n",
        "sudo apt-get update  > /dev/null 2>&1\n",
        "sudo apt-get install redis-stack-server  > /dev/null 2>&1\n",
        "redis-stack-server --daemonize yes"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Using Free Redis Cloud account on GCP\n",
        "You can also use Forever Free instance of Redis Cloud. To activate it:\n",
        "- Head to https://redis.com/try-free/ \n",
        "- Register (using gmail-based registration is the easiest)\n",
        "- Create New Subscription\n",
        "- Use the following options:\n",
        "    - Fixed plan, Gogle Cloud\n",
        "    - New 30Mb Free database\n",
        "- Create new RedisStack DB\n",
        "\n",
        "If you are registering at Redis Cloud for the first time - the last few steps would be performed for you by default. Capture the host, port and default password of the new database. You can use these instead of default `localhost` based in the following code block."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zvDp8WNz4XpU"
      },
      "source": [
        "### Connect to Redis server\n",
        "Replace the connection params below with your own if you are connecting to an external Redis instance."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "duCyNgfZ4XpU",
        "outputId": "2e44d9fb-2ac7-4317-a552-7af1e32a0025"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "execution_count": 2,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "import os\n",
        "import redis\n",
        "\n",
        "# Redis connection params\n",
        "REDIS_HOST = os.getenv(\"REDIS_HOST\", \"localhost\") #\"redis-12110.c82.us-east-1-2.ec2.cloud.redislabs.com\"\n",
        "REDIS_PORT = os.getenv(\"REDIS_PORT\", \"6379\")      #12110\n",
        "REDIS_PASSWORD = os.getenv(\"REDIS_PASSWORD\", \"\")  #\"pobhBJP7Psicp2gV0iqa2ZOc1WdXXXXX\"\n",
        "\n",
        "# Create Redis client\n",
        "redis_client = redis.Redis(\n",
        "  host=REDIS_HOST,\n",
        "  port=REDIS_PORT,\n",
        "  password=REDIS_PASSWORD\n",
        ")\n",
        "\n",
        "# Test connection\n",
        "redis_client.ping()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0Rrz76w96dF3",
        "outputId": "fcd27067-d840-432a-b52b-aba95f34b378"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "execution_count": 3,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Clear Redis database (optional)\n",
        "redis_client.flushdb()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BpjxW-kou-FY"
      },
      "source": [
        "### Authenticate to Google Cloud"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SeTJb51SKs_W",
        "outputId": "d30bc6ba-0f16-433b-e1fc-ef2fbd9bb5be"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Authenticated\n"
          ]
        }
      ],
      "source": [
        "from google.colab import auth\n",
        "auth.authenticate_user()\n",
        "print('Authenticated')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8Yil6twAvIuH",
        "outputId": "a2a0accc-0890-438c-c2fe-7fb8f1ec6252"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "PROJECT_ID:··········\n",
            "REGION:us-central1\n"
          ]
        }
      ],
      "source": [
        "from getpass import getpass\n",
        "\n",
        "# input your GCP project ID and region for Vertex AI\n",
        "PROJECT_ID = getpass(\"PROJECT_ID:\") #'central-beach-194106'\n",
        "REGION = input(\"REGION:\") #'us-central1'"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FoH2gpfOq0WU"
      },
      "source": [
        "### Initialize Vertex AI Components\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "_LibO7HEsGmQ"
      },
      "outputs": [],
      "source": [
        "import vertexai\n",
        "\n",
        "vertexai.init(project=PROJECT_ID, location=REGION)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i2nd1vj2kxvj"
      },
      "source": [
        "## 2. Create BigQuery Table\n",
        "The second step involves preparing the dataset for our LLM applications. We utilize a free (public) hacker news dataset from **Google BigQuery**.\n",
        "\n",
        "*Leveraging BigQuery is a common pattern for building ML applications because of it's powerful query and analytics capabilities.*\n",
        "\n",
        "We will start by creating our own big query table for the dataset. Additionally, if you have a different dataset to work with you can follow a similar pattern, or even load a CSV from a Google Cloud Storage bucket into BigQuery."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ky0pPyMiwDk3"
      },
      "source": [
        "### Create source table\n",
        "First step is to create a new table from the public datasource."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8esN82F2UNph",
        "outputId": "c9e03d12-e77a-45d7-d622-741b7463f64f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Input a Big Query TABLE_NAME:hackernews\n"
          ]
        }
      ],
      "source": [
        "from google.cloud import bigquery\n",
        "\n",
        "# Create bigquery client\n",
        "bq = bigquery.Client(project=PROJECT_ID)\n",
        "\n",
        "TABLE_NAME = input(\"Input a Big Query TABLE_NAME:\") #hackernews\n",
        "DATASET_ID = f\"{PROJECT_ID}.google_redis_llms\"\n",
        "\n",
        "# Create dataset\n",
        "dataset = bigquery.Dataset(DATASET_ID)\n",
        "dataset.location = \"US\"\n",
        "dataset = bq.create_dataset(dataset, timeout=30, exists_ok=True)\n",
        "\n",
        "# Define table ID\n",
        "TABLE_ID = f\"{DATASET_ID}.{TABLE_NAME}\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "Yh7cxtmSvJgq"
      },
      "outputs": [],
      "source": [
        "# Create source table\n",
        "def create_source_table(table_id: str):\n",
        "    create_job = bq.query(f'''\n",
        "      CREATE OR REPLACE TABLE `{table_id}` AS (\n",
        "        SELECT\n",
        "          title, text, time, timestamp, id\n",
        "        FROM `bigquery-public-data.hacker_news.full`\n",
        "        WHERE\n",
        "          type ='story'\n",
        "        LIMIT 1000\n",
        "      )\n",
        "    ''')\n",
        "    res = create_job.result() # Make an API request\n",
        "    table = bq.get_table(table_id)\n",
        "    return table"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JPSjcAkHUnWk",
        "outputId": "bbd48e99-5429-4736-aa0e-0e50d138a7ec"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[SchemaField('title', 'STRING', 'NULLABLE', None, None, (), None),\n",
              " SchemaField('text', 'STRING', 'NULLABLE', None, None, (), None),\n",
              " SchemaField('time', 'INTEGER', 'NULLABLE', None, None, (), None),\n",
              " SchemaField('timestamp', 'TIMESTAMP', 'NULLABLE', None, None, (), None),\n",
              " SchemaField('id', 'INTEGER', 'NULLABLE', None, None, (), None)]"
            ]
          },
          "execution_count": 9,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Create table\n",
        "table = create_source_table(TABLE_ID)\n",
        "\n",
        "# List schema\n",
        "table.schema"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yUpCrznsnh4f"
      },
      "source": [
        "The dataset above contains records of hacker news posts including the **title**, **text**, **time**, **id** and **timestamp**. Let's pull a few sample rows and inspect."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "9C3uLMlBT_c4",
        "outputId": "e586f632-fffc-4b41-df07-2f48d0d4b22a"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-a6d88648-b311-4efd-98f9-4c983734314a\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>title</th>\n",
              "      <th>text</th>\n",
              "      <th>time</th>\n",
              "      <th>timestamp</th>\n",
              "      <th>id</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Parallels – Saigon to Kabul</td>\n",
              "      <td>Did we leave in haste? Should we have left at ...</td>\n",
              "      <td>1625670497</td>\n",
              "      <td>2021-07-07 15:08:17+00:00</td>\n",
              "      <td>27761983</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Mobile Social Network Hits Magic Benchmark</td>\n",
              "      <td>In the New Year, qeep continues to experience ...</td>\n",
              "      <td>1243583061</td>\n",
              "      <td>2009-05-29 07:44:21+00:00</td>\n",
              "      <td>631700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>G</td>\n",
              "      <td>g</td>\n",
              "      <td>1541413708</td>\n",
              "      <td>2018-11-05 10:28:28+00:00</td>\n",
              "      <td>18381237</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Web Browser Bolded Monospace font hack.</td>\n",
              "      <td>If like me you prefer to display code samples ...</td>\n",
              "      <td>1226258822</td>\n",
              "      <td>2008-11-09 19:27:02+00:00</td>\n",
              "      <td>358437</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>I’m afraid because the Inbox app is going away...</td>\n",
              "      <td>I’ve been using inbox for years. I have made t...</td>\n",
              "      <td>1549619456</td>\n",
              "      <td>2019-02-08 09:50:56+00:00</td>\n",
              "      <td>19112798</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-a6d88648-b311-4efd-98f9-4c983734314a')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-a6d88648-b311-4efd-98f9-4c983734314a button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-a6d88648-b311-4efd-98f9-4c983734314a');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "                                               title  \\\n",
              "0                        Parallels – Saigon to Kabul   \n",
              "1         Mobile Social Network Hits Magic Benchmark   \n",
              "2                                                  G   \n",
              "3            Web Browser Bolded Monospace font hack.   \n",
              "4  I’m afraid because the Inbox app is going away...   \n",
              "\n",
              "                                                text        time  \\\n",
              "0  Did we leave in haste? Should we have left at ...  1625670497   \n",
              "1  In the New Year, qeep continues to experience ...  1243583061   \n",
              "2                                                  g  1541413708   \n",
              "3  If like me you prefer to display code samples ...  1226258822   \n",
              "4  I’ve been using inbox for years. I have made t...  1549619456   \n",
              "\n",
              "                  timestamp        id  \n",
              "0 2021-07-07 15:08:17+00:00  27761983  \n",
              "1 2009-05-29 07:44:21+00:00    631700  \n",
              "2 2018-11-05 10:28:28+00:00  18381237  \n",
              "3 2008-11-09 19:27:02+00:00    358437  \n",
              "4 2019-02-08 09:50:56+00:00  19112798  "
            ]
          },
          "execution_count": 10,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Query 5 sample records from BQ\n",
        "query_job = bq.query(f'''\n",
        "SELECT *\n",
        "FROM {TABLE_ID}\n",
        "LIMIT 5\n",
        "''')\n",
        "\n",
        "query_job.result().to_dataframe()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9vDGqjHmVgjB"
      },
      "source": [
        "## 3. Generate Embeddings"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0Kh0ObD4xZtK"
      },
      "source": [
        "### Create text embeddings with Vertex AI embedding model\n",
        "Use the [Vertex AI API for text embeddings](https://cloud.google.com/vertex-ai/docs/generative-ai/embeddings/get-text-embeddings), developed by Google.\n",
        "\n",
        "> Text embeddings are a dense vector representation of a piece of content such that, if two pieces of content are semantically similar, their respective embeddings are located near each other in the embedding vector space. This representation can be used to solve common NLP tasks, such as:\n",
        "> - **Semantic search**: Search text ranked by semantic similarity.\n",
        "> - **Recommendation**: Return items with text attributes similar to the given text.\n",
        "> - **Classification**: Return the class of items whose text attributes are similar to the given text.\n",
        "> - **Clustering**: Cluster items whose text attributes are similar to the given text.\n",
        "> - **Outlier Detection**: Return items where text attributes are least related to the given text.\n",
        "\n",
        "The `textembedding-gecko` model accepts a maximum of 3,072 input tokens (i.e. words) and outputs 768-dimensional vector embeddings."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x3o_9ehYuEpA"
      },
      "source": [
        "### Define embedding helper function\n",
        "We define a helper function, `embedding_model_with_backoff`, to create embeddings from a list of texts while making it resilient to [Vertex AI API quotas](https://cloud.google.com/vertex-ai/docs/quotas) via [exponential backoff](https://en.wikipedia.org/wiki/Exponential_backoff).\n",
        "\n",
        "We also define a method to convert an array of floats to a byte string for efficient storage in Redis (later on).\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "zrpKY4W5yb0M"
      },
      "outputs": [],
      "source": [
        "from typing import Generator, List, Any\n",
        "\n",
        "from tenacity import retry, stop_after_attempt, wait_random_exponential\n",
        "from vertexai.preview.language_models import TextEmbeddingModel\n",
        "\n",
        "# Embedding model definition from VertexAI PaLM API\n",
        "embedding_model = TextEmbeddingModel.from_pretrained(\"textembedding-gecko@001\")\n",
        "VECTOR_DIMENSIONS = 768\n",
        "\n",
        "@retry(wait=wait_random_exponential(min=1, max=20), stop=stop_after_attempt(3))\n",
        "def embed_text(text=[]):\n",
        "    embeddings = embedding_model.get_embeddings(text)\n",
        "    return [each.values for each in embeddings]\n",
        "\n",
        "# Convert embeddings to bytes for Redis storage\n",
        "def convert_embedding(emb: List[float]):\n",
        "  return np.array(emb).astype(np.float32).tobytes()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "peFHIY351eDc"
      },
      "source": [
        "### Embed text data\n",
        "At the moment, our table in BigQuery (created above), contains records of the hacker news posts that we wish to embed and make available for LLMs.\n",
        "\n",
        "In order to conserve RAM usage of this machine, we will iterate over batches of posts from BigQuery, create embeddings, and write them to Redis, which is being used as a [vector database](https://redis.com/solutions/use-cases/vector-database)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "tzkRiknVXOtc"
      },
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "\n",
        "QUERY_TEMPLATE = f\"\"\"\n",
        "SELECT id, title, text\n",
        "FROM {TABLE_ID}\n",
        "LIMIT {{limit}} OFFSET {{offset}};\n",
        "\"\"\"\n",
        "\n",
        "def query_bigquery_batches(\n",
        "    max_rows: int,\n",
        "    rows_per_batch: int,\n",
        "    start_batch: int = 0\n",
        ") -> Generator[pd.DataFrame, Any, None]:\n",
        "    # Generate batches from a table in big query\n",
        "    for offset in range(start_batch, max_rows, rows_per_batch):\n",
        "        query = QUERY_TEMPLATE.format(limit=rows_per_batch, offset=offset)\n",
        "        query_job = bq.query(query)\n",
        "        rows = query_job.result()\n",
        "        df = rows.to_dataframe()\n",
        "        # Join title and text fields\n",
        "        df[\"content\"] = df.apply(lambda r: \"Title: \" + r.title + \". Content: \" + r.text, axis=1)\n",
        "        yield df\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N9jxcBjvNxOt"
      },
      "source": [
        "Below we define a few helper functions for processing a single row of data, writing batches to **Redis**, querying source data from **BigQuery**, and creating text embeddings with **Vertex AI**."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "OETsrYvfuzmX"
      },
      "outputs": [],
      "source": [
        "import math\n",
        "from tqdm.auto import tqdm\n",
        "\n",
        "\n",
        "# Redis key helper function\n",
        "def redis_key(key_prefix: str, id: str) -> str:\n",
        "  return f\"{key_prefix}:{id}\"\n",
        "\n",
        "# Process a single dataset record\n",
        "def process_record(record: dict) -> dict:\n",
        "  return {\n",
        "      'id': record['id'],\n",
        "      'embedding': record['embedding'],\n",
        "      'text': record['text'],\n",
        "      'title': record['title']\n",
        "  }\n",
        "\n",
        "# Load batch of data into Redis as HASH objects\n",
        "def load_redis_batch(\n",
        "    redis_client: redis.Redis,\n",
        "    dataset: list,\n",
        "    key_prefix: str = \"doc\",\n",
        "    id_column: str = \"id\",\n",
        "):\n",
        "    pipe = redis_client.pipeline()\n",
        "    for i, record in enumerate(tqdm(dataset)):\n",
        "        record = process_record(record)\n",
        "        key = redis_key(key_prefix, record[id_column])\n",
        "        pipe.hset(key, mapping=record)\n",
        "    pipe.execute()\n",
        "\n",
        "# Run the entire process\n",
        "def create_embeddings_bigquery_redis(redis_client):\n",
        "    # Create generator from BigQuery\n",
        "    max_rows = 1000\n",
        "    rows_per_batch = 100\n",
        "    bq_content_query = query_bigquery_batches(max_rows, rows_per_batch)\n",
        "\n",
        "    for batch in tqdm(bq_content_query):\n",
        "      # Split batch into smaller chunks for embedding generation\n",
        "      batch_splits = np.array_split(batch, math.ceil(rows_per_batch/5))\n",
        "      # Create embeddings\n",
        "      batch[\"embedding\"] = [\n",
        "          convert_embedding(embedding)\n",
        "          for split in batch_splits\n",
        "          for embedding in embed_text(split.content)\n",
        "      ]\n",
        "      # Write batch to Redis\n",
        "      batch = batch.to_dict(\"records\")\n",
        "      load_redis_batch(redis_client, batch)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ie02v3hfoApC"
      },
      "source": [
        "## 4. Load Embeddings\n",
        "Now that we have a function to generate BigQuery batches, create text embeddings, and write batches to Redis, we can run the single function to process our entire dataset:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 369,
          "referenced_widgets": [
            "d03cde2de63b4be5b0d056051a219fc3",
            "5d78a16175584be397fef3a6ded93a31",
            "e3ea955d38ea4266834d92f4bcae323d",
            "9b224ef5e6074d35afc36b6a5ca6ccad",
            "47c8dd9c84684445be6aca676b57c10b",
            "60dbf3b19aa44ce0a50916b14f223ac8",
            "634ef8a516524c889fd97bc6592a3e77",
            "53560e23ae084a28b114b8046c500c67",
            "13fc33330ef646339cfb215ffaddf5e3",
            "3d28efe8711d48c7a41b6deac843d70d",
            "d25ab4733845475c94ef474b9cb743d1",
            "d1f4e6168ce048f584ead366e880e857",
            "e66a2f6eedee4eba889a7ce8aa4b7043",
            "8e06cd83044144c6bd8d7532051ad041",
            "cc3b7757cf8b4af6bb67cd3c3e9b0c7b",
            "5fbb0fa49b5443b88e74dd91a2beb737",
            "2f09e78298834ae49724b54f97641dae",
            "8bda5f3b7c95412a9f940761740a398d",
            "9aa6e46b56dd4bdfac4debb58da60624",
            "dfbc675c3821466080c20e92f1435641",
            "cc78716187504d2f8f4b5a1a720e9e81",
            "7a2d8e690cc341aeb9182337872c33e8",
            "9b018155074141948c79f692f8789810",
            "a5fa6f0a89cc4cbfa4a042222d19e908",
            "690faf9d30434d15b98dc12efe2530d7",
            "26d052735a154239becf1f6a61edd446",
            "b67159137759433b934c69941159c9d5",
            "2856930aeea74b04a27a9bb6a65888bf",
            "e02593d5b39c49fc8761c5100ce5faeb",
            "1ddcd8ce6eb142e2a2cfd6d7bc4f1468",
            "1ec20a3812fe4952bc87e1daa5b974af",
            "ef6f1d909a544b5a9a3e97d6b4881fb7",
            "a1f95bbb6516429e95265df33d85bd49",
            "71018db51be0484982d494bdcd75c058",
            "36380a55382745ca967109278259cb58",
            "ac4e9f40b45745e2b4f3977e0ef190ad",
            "c00c8dd68ac748969604763a0decbc8c",
            "877f2b283e61492bb736c59e3718a060",
            "dc698bb7bac14221a563b27f3112bfa5",
            "500aa74cd7be4e73924193d6d109c481",
            "d613018ad37b4c028793c990a0c27039",
            "93ab13f7126742a8bd585770e8489344",
            "b421e51a623d4247966069140f55b569",
            "bf7051abdc46495890c1ec0b2b69201a",
            "0b1c14edff784e0aa0f647bdb15508c6",
            "36973dd486f94eef8e095960fbaf4594",
            "76d4e46d3a8c4c178629bff3ce03ac6c",
            "25adf9b83d484e1c98a487f9d9115d6d",
            "e5eb5a5d764f4461a932ac9578ee296f",
            "6bab4d01b47344aaa28c58e5422a0c8e",
            "794553ea60864927957962961c3ae729",
            "df9d6ecb74a441568c94e5fc5526964b",
            "a30987d3511147a4b5f63f89e5f4142d",
            "294e2ec38031449eab4af8531b5c89ab",
            "074bb17ceefa41b390875b24b10600fd",
            "beddfbd5812d455bb1c717a416f70721",
            "d99ff2a10f7e479890f54c285b7a24ef",
            "9f49cc0378f74bc1915a179ea27a775c",
            "6cb177b705184e96aa07afd173f7fe2b",
            "281a72b1a960408f9b80cdbaf9bfd018",
            "7da3fd91f223413589584abbfd1a83ec",
            "269bb8352db54f7a95a32a32fd4a6168",
            "df0845e7ac304662b9708d5931d9ff55",
            "d358eefe15d74f9f83a13ad523f88ed6",
            "fe0a36adc9a848ce855f5761c2ca7223",
            "d835d9a864aa4048bc27bc0b63217d15",
            "e15ea32d9bb84177b226b64b1c253815",
            "9c3d67a4a51e4194a253fb5df203a0dd",
            "9e139fda21fb425385b943762d173280",
            "6836fd622af7416bab90a7a297d09e4b",
            "24790c21d9444d198f16105d6fc9edae",
            "598ff24416f94915bf25228d66383725",
            "848931de937e47b1a19c71890ea4ce81",
            "838d480c8e5040c78898c86da48d98d0",
            "31d5d67b2815452e8ab7ef1a532f0127",
            "976c597448374d9f9e3ca64a31527d25",
            "a5fb3b98fe5c40b3ba213b0668d60329",
            "79dca477a944417cabf7bae71aa1a7ab",
            "f61bd26b951743dc9f5cfc5b1b5035b2",
            "1b76e0470a1641ffa1649de9f6d9c8d9",
            "b33bb8f83ebf43e984a29c5dd0ee23de",
            "6da0517dbae44ebca2f1c691352565cb",
            "81b65549ba2d49fbbef179fc67bdec57",
            "c616e3700d024b528f8737ef2bc8ccc9",
            "cda2d0499b8a4bc0b15f5302f974ada8",
            "05c8192810a04e28bae2a9a092e14a32",
            "6c6d8f5cf65b4a7183e748f76032f326",
            "52874ef92ba048358171016b5e6bb8f3",
            "71daa81bf8d841e28fec06962217550c",
            "fbdcf206b1824c5ba07645c8b9f3ee0f",
            "ec6b45219f624bed8ac56240da8876fa",
            "0ee8f94835254c80bbc754bf5f9e7920",
            "cfc4b18e7544490cbd3e77f6c0911a1c",
            "7706766e3c5f437abf8831183f2aed5c",
            "1afb2fab4daa4643a4464cd84533a5d3",
            "d314d8c4d38a48d5b238ce2e8ac87b4e",
            "49c258cd69634e3c9ae5ac43b4de8ccb",
            "060f3ff726ac449aaae164a34af16a9a",
            "24325a0fa2fe41deac681ebb25ff9124",
            "669ede3b288f4a13b4bd431b6e612269",
            "a9a341478021418bbc113c749bd14d52",
            "5be3e03f5dd842b1b538b3fd4e3d0d1e",
            "12df7f4801ea44ddb2985d1b8cfd5ca2",
            "ba16a3081ab648bb9e3ea7d46bda3933",
            "2b340eef090c4b0885af21681d093d53",
            "c7a576d22b8049319e43eb8339dec0c6",
            "aa428e67455147808b84f81b513be072",
            "c655d8b7a8c74dc6adaac4e7071d65de",
            "f2e35a79172541f6999499344693ead5",
            "479fa71cdee346ef9323ddd2bdef23e8",
            "bce414b1c27b4243868a169b04516b58",
            "21665d8fbda24f87bd73348a354d4c40",
            "b78297245c4743828568cd3c50878e8b",
            "1ce492de9b4e4fb0b9de8e7e7c5c2630",
            "4dc65ac9471942eabba7739b9660c8a8",
            "14bb7d2203b1468883835082d2543a77",
            "964be662ffe948eea311a535eeaad50b",
            "4256261f63f94c78850870d74f791616",
            "f0484fc7733648569a26af7506d87219",
            "969a68beb9fc480fb9bb4c542df50fcc",
            "a3045526915341b38fc4bc21e46ae194"
          ]
        },
        "id": "V9KdsiG_pd3U",
        "outputId": "4a467bd7-1bd3-4df3-e439-4e809a45447e"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "d03cde2de63b4be5b0d056051a219fc3",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "0it [00:00, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "d1f4e6168ce048f584ead366e880e857",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/100 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "9b018155074141948c79f692f8789810",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/100 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "71018db51be0484982d494bdcd75c058",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/100 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "0b1c14edff784e0aa0f647bdb15508c6",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/100 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "beddfbd5812d455bb1c717a416f70721",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/100 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "e15ea32d9bb84177b226b64b1c253815",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/100 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "79dca477a944417cabf7bae71aa1a7ab",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/100 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "71daa81bf8d841e28fec06962217550c",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/100 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "669ede3b288f4a13b4bd431b6e612269",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/100 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "bce414b1c27b4243868a169b04516b58",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/100 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "create_embeddings_bigquery_redis(redis_client)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HvsGxsOvrz5J",
        "outputId": "bf42725b-aa70-4467-8f4d-b7bf8831f486"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "1000"
            ]
          },
          "execution_count": 15,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Validate how many records are stored in Redis\n",
        "redis_client.dbsize()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WGVt7-DNr80e"
      },
      "source": [
        "## 5. Create Vector Index\n",
        "\n",
        "Now that we have created embeddings that represent the text in our dataset and stored them in Redis, we will create a secondary index that enables efficient search over the embeddings. To learn more about the vector similarity features in Redis, [check out these docs](https://redis.io/docs/interact/search-and-query/search/vectors/) and [these Redis AI resources](https://github.com/RedisVentures/redis-ai-resources).\n",
        "\n",
        "**Why do we need to enable search???**\n",
        "Using Redis for vector similarity search allows us to retrieve chunks of text data that are **similar** or **relevant** to an input question or query. This will be extremely helpful for our sample generative ai / LLM application."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "id": "9mNa5LNn4XpX"
      },
      "outputs": [],
      "source": [
        "from redis.commands.search.field import (\n",
        "    NumericField,\n",
        "    TagField,\n",
        "    TextField,\n",
        "    VectorField,\n",
        ")\n",
        "from redis.commands.search.indexDefinition import IndexDefinition, IndexType\n",
        "from redis.commands.search.query import Query\n",
        "\n",
        "\n",
        "INDEX_NAME = \"google:idx\"\n",
        "PREFIX = \"doc:\"\n",
        "VECTOR_FIELD_NAME = \"embedding\"\n",
        "\n",
        "# Store vectors in redis and create index\n",
        "def create_redis_index(\n",
        "    redis_client: redis.Redis,\n",
        "    vector_field_name: str = VECTOR_FIELD_NAME,\n",
        "    index_name: str = INDEX_NAME,\n",
        "    prefix: list = [PREFIX],\n",
        "    dim: int = VECTOR_DIMENSIONS\n",
        "  ):\n",
        "\n",
        "    # Construct index\n",
        "    try:\n",
        "        redis_client.ft(index_name).info()\n",
        "        print(\"Existing index found. Dropping and recreating the index\", flush=True)\n",
        "        redis_client.ft(index_name).dropindex(delete_documents=False)\n",
        "    except:\n",
        "        print(\"Creating new index\", flush=True)\n",
        "\n",
        "    # Create new index\n",
        "    redis_client.ft(index_name).create_index(\n",
        "        (\n",
        "            VectorField(\n",
        "                vector_field_name, \"FLAT\",\n",
        "                {\n",
        "                    \"TYPE\": \"FLOAT32\",\n",
        "                    \"DIM\": dim,\n",
        "                    \"DISTANCE_METRIC\": \"COSINE\",\n",
        "                }\n",
        "            )\n",
        "        ),\n",
        "        definition=IndexDefinition(prefix=prefix, index_type=IndexType.HASH)\n",
        "    )"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "n32J2XikuzpA",
        "outputId": "8f31ea3e-0f82-4ce4-d2d6-9f17793a8ef9"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Creating new index\n"
          ]
        }
      ],
      "source": [
        "# Create index\n",
        "create_redis_index(redis_client)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VOzL5qB-uzrE",
        "outputId": "d66edec2-d78b-4c8b-c746-20ed8af55e60"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'index_name': 'google:idx',\n",
              " 'index_options': [],\n",
              " 'index_definition': [b'key_type',\n",
              "  b'HASH',\n",
              "  b'prefixes',\n",
              "  [b'doc:'],\n",
              "  b'default_score',\n",
              "  b'1'],\n",
              " 'attributes': [[b'identifier',\n",
              "   b'embedding',\n",
              "   b'attribute',\n",
              "   b'embedding',\n",
              "   b'type',\n",
              "   b'VECTOR']],\n",
              " 'num_docs': '37',\n",
              " 'max_doc_id': '37',\n",
              " 'num_terms': '0',\n",
              " 'num_records': '37',\n",
              " 'inverted_sz_mb': '0',\n",
              " 'vector_index_sz_mb': '17592186044416',\n",
              " 'total_inverted_index_blocks': '0',\n",
              " 'offset_vectors_sz_mb': '0',\n",
              " 'doc_table_size_mb': '0.0027265548706054688',\n",
              " 'sortable_values_size_mb': '0',\n",
              " 'key_table_size_mb': '0.001438140869140625',\n",
              " 'records_per_doc_avg': '1',\n",
              " 'bytes_per_record_avg': '0',\n",
              " 'offsets_per_term_avg': '0',\n",
              " 'offset_bits_per_record_avg': '-nan',\n",
              " 'hash_indexing_failures': '0',\n",
              " 'indexing': '1',\n",
              " 'percent_indexed': '0.036999999999999998',\n",
              " 'gc_stats': [b'bytes_collected',\n",
              "  b'0',\n",
              "  b'total_ms_run',\n",
              "  b'0',\n",
              "  b'total_cycles',\n",
              "  b'0',\n",
              "  b'average_cycle_time_ms',\n",
              "  b'-nan',\n",
              "  b'last_run_time_ms',\n",
              "  b'0',\n",
              "  b'gc_numeric_trees_missed',\n",
              "  b'0',\n",
              "  b'gc_blocks_denied',\n",
              "  b'0'],\n",
              " 'cursor_stats': [b'global_idle',\n",
              "  0,\n",
              "  b'global_total',\n",
              "  0,\n",
              "  b'index_capacity',\n",
              "  128,\n",
              "  b'index_total',\n",
              "  0]}"
            ]
          },
          "execution_count": 18,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Inspect index attributes\n",
        "redis_client.ft(INDEX_NAME).info()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4sPe1ZgQkTbX",
        "outputId": "e7b21acb-18c6-4552-ecf3-11410e906a3c"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{b'title': b'What should we name the definitive documentary on the Internet?',\n",
              " b'text': b'If you\\'re doing the definitive documentary on the Internet, doesn\\'t it make sense that you\\'d use the Internet to help pick the name? We\\'ve already begun shooting luminaries such as Brewster Kahle, Steve Wozniak, John Perry Barlow and others. We\\'ll be adding nearly 50 more interviews of the key innovators, thinkers and freedom fighters responsible for this extraordinary creation that has transformed nearly everything about modern life. We\\'ll be asking three kinds of questions: How did we get here, where are we exactly, and what\\'s possible for humanity? Already the moments that we\\'ve captured have been inspirational, as you might expect.<p>But... we still need a name.<p>Our favorite so far, \"Connected\", it turns out is already taken by another project, so we thought we\\'d turn to you. Your suggestion might be a title or a title/subtitle combination, such as \"Connected: The story of the Internet\" etc.<p>Thanks for the assist!<p>@dwhly',\n",
              " b'embedding': b'/T\\x9c\\xbbY\\xa1D<\\x93\\xb9\\xcd;\\xbfW]=rL\\xde<\\xf19\\xbd\\xbc\\x0eh\\x1c=X\\x99\\xe2\\xbb\\xe4\\r\\xa4\\xbcC\\xd2B<1\\xbc\\x92\\xbb\\xb4o\\xe7;\\xb5R\\xe3\\xba\\x7fd\\xba\\xbc_\\x11h\\xbcT\\xdb\\xd0\\xbb\\x88\\x9f\\r\\xbd\\xa4\\x91\\xa2\\xbc\\xe4\\x1f\\x90\\xbb~D\\x8d<\\xcb\\xa0\\xc4\\xbd\\x84\\xd9\\x9a\\xbc\\xde\\xed\\x83<\\x94\\xb4\\x8d;\\xc9w\\xa0<\\x83s\\xab\\xbd\\x97\\x8d\\x14\\xbd\\r\\xc7\\xa6< \\x16\\xbf\\xbc\\xb35\\xef<0f\\xfb;\\xf1\\xd6\\x16=\\x85\\x9e\\x92\\xbdQ\\x9f\\x91;\\x1a&/=h\\xefO=\\xe7I\\x1d\\xbd\\xb2x\\xbf<f\\xba\\x1b<\\x0e\\x03\\t=\\xdd\\x81,\\xb9|\\x9c\\xe3\\xbc7\\x95\\x90=\\xa2\\x85\\xd0<\\x13\\xd4\\xcd:zS]:\\xb6\\xc8\\xa4\\xbczUn=\\x06\\xd5\\'<\\xb8\\xed\\xce\\xbd\\xfd}\\x95\\xbc\\\\N\\xa5\\xbbIo\\xe5;\\x8f[\\x15=\\x06\\x85\\x97\\xbc\\xael\\xc0<iR\\xb5\\xbd(\\xa2\\x9d;\\x19\\xf9\\x12\\xbd\\xf8\\x81i\\xbc\\xbe\\xff\\xc5\\xbb\\xa00\\xe6;\\xfc\\xe6\\xb0<z8\\x89\\xbd\\x8ek\\xfe\\xbc\\x94\\xd4==\\xd7x\\xcd;m?I\\xbcc\\x99\\xeb\\xbc\\xb7V\\x07\\xbd\\xb1ne<\\xa3\\x1fF=T\\x13M\\xbd>\\x80\\x00;\\xb6\\x9f\\xb6;\\xdf\\xca\\xae\\xbc>.C<\\n\"\\xfe<P\\xba\\xa3=\\xc3\\xa1,\\xbd\\x87\\xe9u\\xbbw\\x97\\xf6;\\xdf\\x8a4\\xbd\\x98\\x9e\\xdb\\xbd\\x05ZM\\xbd\\x9d|V=\\xc0n\\x12=\\xc9\\xf3\\xf8<\\x89ts\\xbcx\\xd3\\xac<I\\x08h\\xbc\\xdcs/=\\xf1\\xcc\\x00=]e\\xd2<\\x18[\\xd4\\xbch>]\\xbc\\xdfzm\\xbc\\x05\\xa9\\x04\\xbc\\xd3j\\xb9<p\\x0e\\xa7<\\xf5n\\x99\\xb9H\\x0c\\x95\\xbd\\x91\\xc7D=1\\x02U<\\xff\\n\\x02=h4L\\xbcL\\xfe\\x9d;p\\xefl</\\xc2\\x13\\xbd\\xab\\xd1\\x9e\\xbdT\\xaaD\\xbd\\x1f\\xb4\\x85\\xbc=U6\\xbdhC\\xd1<\\xaa\\xb6\\x95<\\xc6\\xf1m\\xbd\\xfd:\\xa48)Z\\x89<8~[=l\\t\\x04\\xbb\\x97\\xd8D\\xbd\\xea\\xb6\\xcb\\xbc\\xc4\\x84\\xa8\\xbb\\xb1\\xd8M=\\xe9\\xf4(=\\'\\x01\\x8a;\\xfe\\xc0\\x98<\\xb4{\\x16\\xbd\\x89L\\xbf\\xbcCS\\x17=\\xd2\\x18\\x86=\\x89@\\xde;\\xc5\\xc1\\x82=\\x8b\\xf5\\x12\\xbd`\\xde)<\\x01g_\\xbc\\x1f\\xe8\\x87\\xbc.\\xfa\\x85<\\xa9v\\x81\\xbc\\xefg\\x81=\\xc0u.\\xbd[\\x18\\xdb\\xbc\\xb7t\\x19\\xbdX\\xf0c<,t\\x95\\xbc\\xedJI\\xbd^\\x85O<\\xc0]\\x8c\\xbde\\xb4A\\xbdC\\x0bn\\xbc\\xefC\\x14=Hh\\x08\\xbd\\x05\\xdc\\x13\\xbdK\\xcd,=\\xf1\\xf1\\x16=\\t\\\\ <\\x8f\\xf1\\xdb<<\\xc2\\xf2<\\x8d\\x96y\\xbcb.\\x86\\xbdjr\\x1f<,-\\x12\\xbd\\xa3B\\x02\\xbd\\xc0\\xfe\\xa4\\xba#<\\xcf<by\\x80\\xba=\\x96\\xfc\\xbb\\x94\\xb7\\xec<\\xbe\\x047<(\\x1e\"=\\x0f&\\x90=|G\\x02\\xbe\\xd4\\x05\\xd0<h\\x18\\x8d\\xbcW\\xe9\\xb6=0\\xa4d;- A\\xbc\\xd5\\x9a\\x9c\\xbc\\x05\\xb1\\x03=z\\x18\\x9a<\\x06&\\x13\\xbd\\xc3\\xa5\\xa4\\xbdY\\x7f\\x8a<\\xeey8=$|\\x1d\\xbdj\\xc0U<,\\x19d\\xbd\\x80\\xcd\\xfb9\\xd8\\xfb\\xa5\\xbc/\\x0e#=I\\xdd\\xcd\\xbc\\xe4\\xca$\\xbb&@\"=\\x8aG\\x19\\xbd2\\x7f\\xa7\\xbb^b\\'\\xbc\\xbf\\xd8\\x1e=\\xf5\\x95%\\xbe\\xc7Y/\\xbd\\xbc\\x15\\xcf\\xbb\\x01{\\xe7\\xbc\\x1f\\x0f\\x98\\xbd\\x86\\x7f6=oca=%r\\xf1\\xbc\\xbe\\x1f#<K\\xd5*\\xbd>\\xe4\\x0b=\\xe1\\x8b\\x1c=\\xce\\x19K=\\x84k\\xe4\\xbav\\x1d\\xd1\\xbcz\\x99`=\\x1f\\xf8\\x90:\\x13\\xaa\\xe7\\xbbA\\x96z\\xbd\\x10\\x97t<y\\x02\\xa3\\xbdt\\xa5\\x9f\\xbd\\x80i5\\xbcY\\x82\\xb4\\xbce\\x99\\x10<\\xb7\\xb6\\xa8;\\xa9\\xdd\\x94\\xbb\\x9a\\x80\\x0e=}\\x892<-\\x92\\xe4<j\\xf9\\xb4;\\x85<\\xeb<\\xb0\\x8b\\xa2:b\\xb0 =L\\xe6\\xa7\\xbb\\xc9p\\x1a\\xbd\\x83\\x7f\\x14=\\xe5\\xe8w=\\xcf\\xc2\\x03=\\xec%Q<\\xff\\xcb\\xea<\\xf2\\x93l\\xbd\\xab\\xc3\\xeb<\\x99\\x84\\xb0\\xbc\\xa5\\x89\\x03=\\xd2\\xcb\\t\\xbd:\\xe6\\x0b=_\\\\ \\xbd?=L=[\\x07\\x90\\xbc\\x1eP\\xc3<w\\x88\\xad<\\xe1\\xbe\\xaf\\xbc\\x05\\x9b\\x80=\\xdc\\xeb8\\xbd2\\xe3\\xbf\\xba\\xcd\"\\x8e<\\xbc\\xef\\xe6<V\\xc7\\xef\\xbc%\\x08\\xa2<\\x8a\\x04a=\\xa2z\\x87<\\x858_;\"\\xe4\\x11\\xbd\\x18,^<\\xd7\\xb0\\xb0\\xbcv\\xbe1>\\xfd\\xaf@\\xbd\\\\,\\xed\\xbc\\t\\xe1j\\xbd9]\\xbe<\\x92c\\xe6<e\\xa4\\x19\\xbc\\xd9\\xd7\\xe9\\xbc\\xa0\\xa4\\xa7<=p`=?\\xab\\xd6;\\xde\\x99\\x03\\xbc6\\xe3\\xa4\\xbc\\xa1\\xc4\\x99\\xbc\\xef\\x8co=\\xb2\\x9a\\x08=\\x8fVf;\\xc0\\x01\\xde\\xbc\\x98\\x85S\\xbc@o\\x87<\\x87k\\x1b=\\x98Z\\xfe<\\xbb$1\\xbb\\xfcOr<\\x8f\\xfeq<#\\xb7\\x1b\\xbdU\\x12\\x02=\\xb3N/=xM\\xdc<\\'3\\x0c\\xbd7\\x01\\xa3=8\\xf13\\xbb\\x0b\\xa1\\xa0;\\xa6\\x8d&=\\xffK\\x02\\xbc\\xbd~F\\xbdK\\xed\\xb5<?Jm\\xbc\\xfc\\x1d\\xc7\\xbc\\x00CI\\xbd0g]\\xbc\\x82g~\\xbd\\x86\\x16\\x84\\xbdQZ7<?\\xc7\\xfa:\\x92\\xf4P<G\\x84`=\\x92\\xef\\x10=\\xf9\\x84\\x8b\\xbc\\xe8,\\x07\\xbd\\xd9m@\\xbb:\\x92\\x98\\xbd\\xb1V\\x11=\\xc3\\xbd\\x17\\xbd\\x86\\x06\\x0c<\\x0f\\xc4\\x08\\xbd\\xc8\\x1b\\x87<c{\\xc7\\xbc@\\xb3\\xb7\\xbc\\'08=\\x91\\xfd5\\xbc^\\xde\\xc9<\\'Ww\\xbcU\\xf7\\\\\\xbcQ*3<\\x1c\\xa3Q=\\xad\\xc0\\xe0\\xbc\\x15\\t;\\xbd\\xca\\x0f\\xf0<\\xe5$\\xee\\xbcR\\x17\\xa9<\\xb6\\x0b\\xa5<\\x908D\\xbb\\x83,\\xac<\\xd6,\\x00=vP\\xf7\\xbc\\x07M\\x9b\\xbd\\r\\x08\\x03=J\\xa0\\xec\\xbcNl\\x14\\xbdd\\xe7\\x91\\xbd\\xa8\\xb3\\x93\\xbc\\xf0\\x80\\xd0\\xb9\\xad\\n<\\xbd\\xc3\\xf1{\\xbdH.S\\xbd\\xc3\\xb0\\x01\\xbd \\x94\\x14=\\xcb\\\\h=Q\\xb5\\x92<Z48=\\xa1\\x00\\xf6\\xbb\\xd8\\x94>\\xbc6\\xaf\\xb4\\xbc\\x18\\xb5\\'<O\\x1bK=6\\x01~=\\xc3\\x92\\x83\\xbd\\x91\\xa4\\xdd<\\x07\\xc2\\x08\\xbd\\x00\\x07\\xe5;\\xfa\\\\\\xc6;f\\xd3\\x87=\\xb1\\xcc\"\\xbd\\xcf\\x9f\\'=\\x8d\\xbe3=\\xd2\\x85\\x93;\\xbb\\xa6\\x9a\\xbc\\xdd\\xd4\\x99\\xbc\\x87\\xa6\\x9a<\\\\\\xe4\\xb7\\xbc/\\x84!\\xbd\\x0b}\\x8e\\xbcxmy\\xbc4n\\x82<C\\xde\\x00\\xbc\\xf8/\\xad;W\\xed5<\\xc0\\xd7\\x86\\xbc.Q\\x1c\\xbd\\x8e\\x8bv=9\\x8e\\xac\\xbc\\xa7z\\xb3<u\\xb2J=E\\xe1\\xef<#$\\xf1<\\xce\\x08\\xf7\\xbc\\x87\\xf9\\xe2<C\\x17p=U\\xd4\\xdc:\\xff\\x13\\x0f\\xbd\\xd9\\x96\\x01\\xbc>,\\x93=?`\\n\\xbd\\x94\\x91f:\\x02e\\xe0\\xbc\\x06\\xd0\\xad\\xbc\\xca\\x8d\\x0f=\\xf7\\x07\\xad:G\\xcf\\x1e\\xbd9xM\\xbd\\x10T\\x15=x\\xaa\\x83<\\xe7\\x18T\\xbc\\xdf]\\xfc:\\xe8\\xde\\x83=H%\\x01\\xbdGS\\x97<\\xa8\\x03\\x1a\\xbck\\xf7\\xf4<47\\xaa\\xbc\\xf8]\\xe8<\\x1a\\x94R\\xbd\\xca\\xf0\\xd0\\xbb\\xf9\\xf6\\xf8\\xbc\\x84\\xabE;*\\x8d\\xb2\\xbc\\xe5\\'G=\\x19\\x966\\xbd\\xa8\\xd3\\xc2\\xbd.T0\\xbb\\xb2\\x1c]=\\xda\\x1f\\xe5<\\x88\\xd1\\x9f\\xbd\\xadr}<\\x92\\xd3y\\xbc]}\\xab\\xba\\xb6l\\xa2\\xbb\\xcd\\x10\\xa3\\xbc-w[\\xbb~7s\\xbd\\xa4\\xa6\\xd3\\xbb\\xd1c|\\xbdc\\x80\\xa8=\\xf1\\xe6\\x87=\\x8e\\x86\\x07<F\\xe7T\\xbb\"\\x85\\xd8\\xba\\xea/<\\xbaE\\x9c^\\xbcLk\\x97<\\x17\\x04~\\xbc\\x14):\\xbd\\x9a\\xc4\\xd3\\xbbd~Q\\xbd\\xd3UL\\xbd\\x0bX\\n=V\\x9bk\\xbbv~\\xd4\\xbc\\xe9\\xa6X\\xbb\\x90\\x07T=\\x00\\xc4`=3\\x92\\xf4<Q\\xc0\\x04=\\x89\\xc1\\x94\\xbc\\xfcC <\\xfcF\\xed:_~\\xce<i\\x8b\\xdd<\\xb6R<\\xbd\\x94b\\xce<E\\x83\\xb2<:Vx\\xbc\\x1a\\xed8=Wm\\x8c=\\xf0]\\xc1:\\xff*\\x0c=\\x8fOl\\xbd\\xda\\x06\\xde=<\\xdf\\x8a\\xbd\\x14-\\xd8\\xbc\\ryS=\\xe6e\\x03\\xbc%\\x1b\\xb6;\\xa8Kj=pk\\xfc;\\xd2:{\\xbc&R\\x91\\xbd\\xc3\\xf4\\x1d=\\xff\\xde^\\xbc\\x1a|!\\xbdl.&=\\x97\\xc3-\\xbc\\xf6\\x9c9;&\\xbfa\\xbc\\x1fZF<\\xa8\\xda\\x84\\xbb~\\xd9\\xd4\\xba\\xb6t\\xb99Z}~<\\x90K0<\\x95j\\xc8<\\xa4\\xe3\\xa1<F\\xf5\\x9e\\xbb\\x80\\x10\\xe6;\\x87\\x14\\xdc<\\x83\\\\U\\xbc\\x0c\\x01\\xad<\\xc8\\xe9\\x11=\\xd7D\\xf3\\xbb\\x8e\\xc4\\xeb;\\xccU\\x89<R\\xd3\\x05\\xbdI\\x05K:\\x88\\x89o\\xbdw~\\xad\\xbdh\\xad\\x00=\\\\\\xa2\\x13<:\\xb0\\xa3<\\xb9a/\\xbd\\xb5\\xcaR\\xbd\\xf1\\\\\\x17\\xbd#\\xa1g\\xbd\\xde9O\\xbc\\xd0\\x90J;\\xfa\\x0e\\xa5;\\x1c\\x1d\\'\\xbbA\\x11\\xa9<\\x1c\\x0e\\xf7\\xbc\\x1ct4\\xbd\\n\\x06\\xab<)\\xf7\\x95;\\xc6tY=M\\xd6\\x15=\\xf0\\x12$\\xbd\\xe6\\xb2.\\xbd\\xeb\\xb4\\\\=\\x86=p<\\xc6e\\xcb<\\xad\\xa8\\x9a:\\xd5\\xe1\\x0c<\\xf0\\xca\\xeb\\xbaV\\xbaA\\xbd\\x8a`\\xef<<\\xa8\\x86\\xbcp\\x00\\x04\\xbcM\\\\\\xb4=? \\x86\\xbc\\xe8\\xc8^\\xbdS\\x93i=\\xae4A\\xbc\\x9e\\xd3\\x89<\\x03\\x9a\\xfb\\xbcj\\x1e\\x14=X\\xb8\\xf4<\\xdb>\\x00=\\xae\\x9bu\\xbb\\x00\\x90\\xe1<Y\\x08\\x9c\\xbc_\\xc28\\xbd\\x90\\xfd\\x8f=\\x12\\xce\\xc9<\\xddE!\\xbd\\n)\\xe0\\xb9\\x07\\x84\\x9d\\xbc\\xe6_\\n\\xbd\\xf0O=\\xbc;\\xc5\\xcb<\\x93\\xe7\\xe7<\\x07M\\'\\xbb\\xd9\\x87<;\\xdc\\x08\\x9f= %\\x82\\xbcV\\xa5;==5 \\xbc\\xea\\xdd\\xf6<^\\x80\\x85=\\xf2\\xdbk\\xbc\\x82\\xd85\\xbd\\x8a\\x81\\xed\\xbc2\\x93V:/N\\x12\\xba5Di\\xbc_\\xa4\\x03\\xbbXQ<=q\\xe7\\x96\\xbb\\xa6\\xcf\\x81;\\xe0\\x0f\\\\\\xbd\\xba\\xcd\\x01=dN\\'\\xbcRL\\x9a<#p\\x12\\xbd\\x82\\x03*\\xbc\\xef\\xe2D\\xbdy\\xf8\\x05\\xbd\\x83\\xa1\\x15\\xbd\\x04C\\x15\\xbdd\\xb5\\x1a\\xbc\\xc5r\\x98\\xbdO\\xb5\\x15\\xbd\\x12\\x90#\\xba\\xc0\\xba\\x0e\\xbd/\\xd3\\xa5\\xbc^\\xd3I<H\\r\\x8f\\xbcX>\\x9d=\\xe4\\xa6\\x90\\xbc\\x8a\\x8e^\\xbdr\\xd8\\x95\\xbb\\x8e\\x02\\xa8=%Y\\x9b<\\x9f\\x05\\x06<\\xee\\xbdh\\xbb\\xff\\xbb\\x1e=\\x16\\xbc\\x01=\\xd9\\x9b\\x80<y\\x96O<5\\xdd>=\\xa1\\xbd`<\\xf9\\x1dW<@\\xc4\\xf7</\\xb0\\'\\xbdu\\xa4\\x03\\xbc\\xb5\\xe7\\xf1\\xbc;\\x03`=\\xcd\\x829=]Y\\x1b<\\xbd\\r\\x1f=\\xc8\\x1a\\x8d=\\x92=}\\xbb\\x0c\\r\\xdf\\xbc\\xf3X\\x04=fq\\x8c\\xbc\\x1c\\xeer\\xbbi\\xe9n:\\xa3W\\xc29tHb\\xbd\\xc7\\xe9%=\\xf3e\\x16\\xbd#\\xf0\\x99\\xbc\\xeeWb\\xbcz\\xa1\\x95<\\xc4\\xc2\\x13\\xbd4]3<i**<\\r\\x82\\xb1<\\xa3\\xf0\\xc4\\xbc\\xdf\\x8a\\x03\\xbd-6-=\\xccY\\x13\\xbd\\xcb`?=\\x9c\\xde):\\xa4\\x112\\xbc\\xaf\\xc4\\xdf<\\x19\\x93\\xb3:\\xad\\x16h\\xbbn\\x81==K\\xec\\xf1\\xbc\\x9c\\xbc\\x8c\\xbcR\\x1cW<:\\x8f\\xea\\xbc\\xea\\xd8O<\\xf9\\xf4\\xb9\\xbc\\xdb9\\xbc\\xbb\\xeeT\\xa6\\xbc%\\x8f/\\xbd\\x13\\x0e\\xf4<\\x83\\n\\x82<B\\x83\\t:\\xa8\\xf7\\x93<$\\xf03=Y\\xe1\\xff\\xbc\\x18\\x80y=\\\\\\xaa\\xda\\xba\\x03\\xf2`<\\xc3\\xf9<<r\\x14I<L\\x89\\x1f\\xbdv\\x07X\\xbc0[\\xab;G-i\\xbc7\\x00]\\xbc\\xb4\\xd2t\\xbc\\xe9\\xf8\\x9f<\\xd5>o\\xbd\\x103\\xcf<\\x88\\xedx=A\\x9c+\\xbds$\\x04\\xbd8\\x19\\xb1<~\\xc0r\\xbd\\xaft\\x03=\\r\"]\\xbd~\\xfa\\x12\\xbdH\\x9eU\\xbc3ZE\\xbd\\xd1\\x1b3\\xbc\\x1fW\\xff\\xbc\\x8es\\xd0<\\xebk\\xb2\\xbc\\x14\\xf7\\xfc<\\x93\\xa4H\\xbd\\xf3\\xdbi\\xbd\\xdcg\\xaa\\xbc\\xaeZy\\xbc\\r\\xf3\\x1c=\\x9b@\\t=\\xfc\\x03\\xd6<G\\x8e\\xa6<\\x87\\x84\\xbe=\\xb8\\xd5\\x95=Zt\\x18\\xbc\\x83\\x89\\xf5\\xbc\\x9eW*\\xbd\\xc6P\\x06\\xbc`fH<]\\xb4\\xbb\\xbcW\\xb3\\xcc\\xbbm\\xff\\x8e<\\xd8\\x97b=_\\xb7\\x18=\\xc1\\x10\\xd9<\\xd5\\xbd!=]\\xbb/\\xbd\\x9c\\x03r;\\xba\\x93\\x11=\\xabH\\x81<~\\x84Z\\xbc4\\xb2\\xff;\\xa2h\\x8c<\\xa32\\xcc<3\\x94A\\xbbz\\xea\\xb4<\\x82\\x05\\x07=\\x86\\xb5\\x11\\xbb\\x93\\xd3\\x82<\\xccC\\x9a<\\xe2\\xd5/\\xbc7\\xc7\\x00\\xbc\\x0c\\xeb\\xbe\\xbc\\xbf2T=<\\xe7~:(_\\xef\\xbc\\xd06\\x08=vu\\xc3:\\x18w)=\\x99X\\x1b\\xbd\\x02)\\x0b\\xbc\\x06\\x06\\x93\\xbcV\\x8b\\x85;-\\\\\\x8f\\xbc@\\xd6\\xe5<\\xea\\xf1}\\xbc\\xcc-\\xed<\\xef8\\xbc\\xbc\\xdd4\\xdc\\xbch\\x83/\\xbd\\xcc\\x13`\\xbd\\xc1\\xdbe;mZ\\x01\\xbd\\xc9B\\xba<\\x88\\xdd\\x13\\xbd\\xd1\\r\\xcf=/<\\x0e=\\x96\\xb6T=\\x19\\x995\\xbce\\xc0\\x80=*\\x1c\\x8a\\xbc\\xd3u\\xa1<\\x01x\\xdd\\xbc\\xbb78=\\xe7s\\x0c<h\\xd7\\xb59s#\\xc1\\xbc\\xe90\\x83<',\n",
              " b'id': b'2926429'}"
            ]
          },
          "execution_count": 19,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Retreive single HASH from Redis\n",
        "key = redis_client.keys()[1]\n",
        "redis_client.hgetall(key)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G6CmHY3-6wB1"
      },
      "source": [
        "At this point, our **Redis** datastore is completely loaded with a subset of data from **BigQuery** including text embeddings created with **Vertex AI** PaLM APIs."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PiYCEhMqLzvS"
      },
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "82-AbKHxItif"
      },
      "source": [
        "# Build LLM applications\n",
        "With Redis fully loaded as a vector database and powerful PaLM APIs at our disposal, we can build a number of AI applications on this stack. Below we will briefly describe each of these applications and use cases\n",
        "\n",
        "- **Document Retrieval** - search through documents to return only the most relevant to a given query.\n",
        "- **Product Recommendations** - recommend products with similar attributes and descriptions to a product the shopper likes.\n",
        "- **Chatbots** - provide a conversational interface for information retrieval or customer service.\n",
        "- **Text Summarization & Generation** - Generate new copy from sources of relevant information to accelerate team output.\n",
        "- **Fraud/Anomaly Detection** - identify anomalous and potentially fraudulent events, transactions, or items based on attribute similarity of other known entities."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Hey_tOYhMByl"
      },
      "source": [
        "# LLM Design Patterns\n",
        "\n",
        "In order to build these kinds of apps, below we highlight 4 technical design patterns and techniques where Redis Enterprise comes in handy to boost LLM performance:\n",
        "\n",
        "- **Semantic Search**\n",
        "- **Retrieval Augmented Generation (RAG)**\n",
        "- **Caching**\n",
        "- **Memory**\n",
        "\n",
        "Leveraging some combination of these patterns is recommended best practice, derived from enterprise use cases and open source users all over the world."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tm0c981Q7e5k"
      },
      "source": [
        "### Simple Semantic Search\n",
        "\n",
        "\n",
        "**Semantic Search**, in the context of Large Language Models (LLMs), is a sophisticated search technique that goes beyond *literal* keyword matching to understand the contextual meaning and intent behind user queries. Leveraging the power of Google's Vertex AI platform and Redis' vector database capabilities, semantic search can map and extract deep-level knowledge from vast text datasets, including nuanced relationships and hidden patterns.\n",
        "\n",
        "This allows applications to return search results that are contextually relevant, enhancing user experience by offering meaningful responses, even to complex or ambiguous search terms. Thus, semantic search not only boosts the accuracy and relevancy of search results but also empowers applications to interact with users in a more human-like, intuitive manner.\n",
        "\n",
        "The general process of semantic search includes 3 steps:\n",
        "1. Create query vector\n",
        "2. Perform vector search\n",
        "3. Review and return results"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "a6BsbxUG7kVc",
        "outputId": "add7774d-0589-49d1-e2dc-a841eef8a367"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[0.019742609933018684,\n",
              " -0.006285817362368107,\n",
              " 0.057011738419532776,\n",
              " 0.0630318894982338,\n",
              " 0.019907860085368156,\n",
              " -0.045217469334602356,\n",
              " 0.0657828152179718,\n",
              " 0.027572786435484886,\n",
              " -0.056532640010118484,\n",
              " 0.003752157324925065]"
            ]
          },
          "execution_count": 20,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# 1. Create query vector\n",
        "query = \"What is the best computer operating system for software dev?\"\n",
        "query_vector = embed_text([query])[0]\n",
        "\n",
        "# Our query has been converted to a list of floats (this is a truncated view)\n",
        "query_vector[:10]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "id": "zmma7Cjd7kZ9"
      },
      "outputs": [],
      "source": [
        "# Helper method to perform KNN similarity search in Redis\n",
        "def similarity_search(query: str, k: int, return_fields: tuple, index_name: str = INDEX_NAME) -> list:\n",
        "    # create embedding from query text\n",
        "    query_vector = embed_text([query])[0]\n",
        "    # create redis query object\n",
        "    redis_query = (\n",
        "        Query(f\"*=>[KNN {k} @{VECTOR_FIELD_NAME} $embedding AS score]\")\n",
        "            .sort_by(\"score\")\n",
        "            .return_fields(*return_fields)\n",
        "            .paging(0, k)\n",
        "            .dialect(2)\n",
        "    )\n",
        "    # execute the search\n",
        "    results = redis_client.ft(index_name).search(\n",
        "        redis_query, query_params={\"embedding\": convert_embedding(query_vector)}\n",
        "    )\n",
        "    return pd.DataFrame([t.__dict__ for t in results.docs ]).drop(columns=[\"payload\"])\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "id": "wIaYNgNxA4D1"
      },
      "outputs": [],
      "source": [
        "# 2. Perform vector similarity search with given query\n",
        "results = similarity_search(query, k=5, return_fields=(\"score\", \"title\", \"text\"))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "uy7rh-stIIii",
        "outputId": "4131fff8-e6e7-40d5-b4ea-056c3a4f3e08"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-8dd4bde0-581a-4160-a7a8-e2697f73b63a\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>id</th>\n",
              "      <th>score</th>\n",
              "      <th>title</th>\n",
              "      <th>text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>doc:8738518</td>\n",
              "      <td>0.334048986435</td>\n",
              "      <td>Ask HN: Does anybody know any programming lang...</td>\n",
              "      <td>I am looking for something analogous to osdev....</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>doc:25027459</td>\n",
              "      <td>0.334324538708</td>\n",
              "      <td>Ask HN: What is your at-home development setup...</td>\n",
              "      <td>For personal projects, what&amp;#x27;s your physic...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>doc:7638777</td>\n",
              "      <td>0.353065013885</td>\n",
              "      <td>Ask HN: Single Developer Cheap Setup</td>\n",
              "      <td>As a single developer with not too much in ter...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>doc:21206278</td>\n",
              "      <td>0.358504593372</td>\n",
              "      <td>Recommended Coding/Data Science BootCamps in S...</td>\n",
              "      <td>I recently parted ways with tier 2 level broke...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>doc:781794</td>\n",
              "      <td>0.360902309418</td>\n",
              "      <td>Create Your Own Custom Linux Distro Online Fas...</td>\n",
              "      <td>Now, you don't need to be an hardcore programm...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8dd4bde0-581a-4160-a7a8-e2697f73b63a')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-8dd4bde0-581a-4160-a7a8-e2697f73b63a button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-8dd4bde0-581a-4160-a7a8-e2697f73b63a');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "             id           score  \\\n",
              "0   doc:8738518  0.334048986435   \n",
              "1  doc:25027459  0.334324538708   \n",
              "2   doc:7638777  0.353065013885   \n",
              "3  doc:21206278  0.358504593372   \n",
              "4    doc:781794  0.360902309418   \n",
              "\n",
              "                                               title  \\\n",
              "0  Ask HN: Does anybody know any programming lang...   \n",
              "1  Ask HN: What is your at-home development setup...   \n",
              "2               Ask HN: Single Developer Cheap Setup   \n",
              "3  Recommended Coding/Data Science BootCamps in S...   \n",
              "4  Create Your Own Custom Linux Distro Online Fas...   \n",
              "\n",
              "                                                text  \n",
              "0  I am looking for something analogous to osdev....  \n",
              "1  For personal projects, what&#x27;s your physic...  \n",
              "2  As a single developer with not too much in ter...  \n",
              "3  I recently parted ways with tier 2 level broke...  \n",
              "4  Now, you don't need to be an hardcore programm...  "
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "# 3. Review and return the results\n",
        "display(results)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IH04U3ls9Pes"
      },
      "source": [
        "Results above indicate that our search for recommended operating systems for software devs yielded some posts from Hacker News that might be helpful in answering this question.\n",
        "\n",
        "**Interested in tuning the search results?**\n",
        "- Try using a different [Distance Metric](https://redis.io/docs/interact/search-and-query/search/vectors/#creation-attributes-per-algorithm)\n",
        "- Try using a different [Index Type](https://redis.io/docs/interact/search-and-query/search/vectors/#flat)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OKFktSvG9sgN"
      },
      "source": [
        "### Retrieval Augmented Generation (RAG)\n",
        "\n",
        "**Retrieval Augmented Generation** (RAG), within the scope of Large Language Models (LLMs), is a technique that combines the knowledge of domain-specific data and generative models to enhance the production of contextually-rich question responses. In essence, *RAG* functions by retrieving relevant information from a knowledge base of documents or data before proceeding to generate a response. This allows generalized foundation models to gain access to these datasources at runtime, and is NOT the same thing as fine-tuning.\n",
        "\n",
        "RAG exploits the strengths of Redis as a low-latency vector database for efficient retrieval operations and Google's Vertex AI to generate a coherent text response. In LLM applications, RAG enables a deeper comprehension of context, returning highly nuanced responses, even to intricate queries. This pattern enhances the interactive capability of applications, delivering more precise and informative responses, thereby significantly enriching the user experience.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DoZpAD8mFQjW"
      },
      "source": [
        "In order to build a RAG pipeline for question answering, we need to use Vertex PaLM API for text generation (`text-bison@001`)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "EVwj7o5lFKy2",
        "outputId": "f5c9d1dd-5f7b-489d-fd9c-46e32153a5ee"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Example response:\n",
            " A large language model (LLM) is a type of artificial intelligence (AI) model that can understand and generate human language. LLMs are trained on massive datasets of text and code, and they can learn to perform a wide variety of tasks, such as translating languages, writing different kinds of creative content, and answering your questions in an informative way.\n",
            "\n",
            "LLMs are still under development, but they have the potential to revolutionize many industries. For example, LLMs could be used to create more accurate and personalized customer service experiences, to help doctors diagnose and treat diseases, and to even write entire books and movies.\n",
            "\n",
            "\n"
          ]
        }
      ],
      "source": [
        "from vertexai.preview.language_models import TextGenerationModel\n",
        "\n",
        "# Define generation model\n",
        "generation_model = TextGenerationModel.from_pretrained(\"text-bison@001\")\n",
        "\n",
        "response = generation_model.predict(prompt=\"What is a large language model?\")\n",
        "\n",
        "print(\"Example response:\\n\", response.text)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-YqvgImdFiAz"
      },
      "source": [
        "In order to be able to answer questions **while referencing domain-specific sources** (like our sample hackernews dataset), we must build a RAG pipeline:\n",
        "\n",
        "1. First perform **Semantic Search** with the user query on the knowledge base (stored in Redis) to find relevant sources that will help the language model answer and respond intelligently.\n",
        "\n",
        "2. The sources (called context) are \"stuffed\" into the prompt (input).\n",
        "\n",
        "3. Lastly, the full prompt is passed on to the language model for text generation."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "id": "NbMbVRgLGJbt"
      },
      "outputs": [],
      "source": [
        "def create_prompt(prompt_template: str, **kwargs) -> str:\n",
        "  return prompt_template.format(**kwargs)\n",
        "\n",
        "def rag(query: str, prompt: str, verbose: bool = True) -> str:\n",
        "    \"\"\"\n",
        "    Simple pipeline for performing retrieval augmented generation with\n",
        "    Google Vertex PaLM API and Redis Enterprise.\n",
        "    \"\"\"\n",
        "    # Perform a vector similarity search in Redis\n",
        "    if verbose:\n",
        "        print(\"Pulling relevant data sources from Redis\", flush=True)\n",
        "    relevant_sources = similarity_search(query, k=3, return_fields=(\"text\",))\n",
        "    if verbose:\n",
        "        print(\"Relevant sources found!\", flush=True)\n",
        "    # Combine the relevant sources and inject into the prompt\n",
        "    sources_text = \"-\" + \"\\n-\".join([source for source in relevant_sources.text.values])\n",
        "    full_prompt = create_prompt(\n",
        "        prompt_template=prompt,\n",
        "        sources=sources_text,\n",
        "        query=query\n",
        "      )\n",
        "    if verbose:\n",
        "        print(\"\\nFull prompt:\\n\\n\", full_prompt, flush=True)\n",
        "    # Perform text generation to get a response from PaLM API\n",
        "    response = generation_model.predict(prompt=full_prompt)\n",
        "    return response.text\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rHrLLAQcH5_e"
      },
      "source": [
        "Below is an example prompt template. Feel free to edit and tweak the initial sentence that sets the context for the language model to perform the action we are anticipating. The process of tuning and iterating on prompt design is widely refered to as \"*prompt engineering*\"."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "id": "dXP_pzQ0BJl_"
      },
      "outputs": [],
      "source": [
        "PROMPT = \"\"\"You are a helpful virtual technology and IT assistant. Use the hacker news posts below as relevant context and sources to help answer the user question. Don't blindly make things up.\n",
        "\n",
        "SOURCES:\n",
        "{sources}\n",
        "\n",
        "QUESTION:\n",
        "{query}?\n",
        "\n",
        "ANSWER:\"\"\"\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "g3iuL5eb9Wsm",
        "outputId": "52b061a0-34b0-4709-bd70-4b49e2f7e409"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Pulling relevant data sources from Redis\n",
            "Relevant sources found!\n",
            "\n",
            "Full prompt:\n",
            "\n",
            " You are a helpful virtual technology and IT assistant. Use the hacker news posts below as relevant context and sources to help answer the user question. Don't blindly make things up.\n",
            "\n",
            "SOURCES:\n",
            "-I am looking for something analogous to osdev.org but for programming language development.\n",
            "-For personal projects, what&#x27;s your physical and software setup like? And what do you like to code in your spare time?\n",
            "-Now, you don't need to be an hardcore programmer to create your own custom Linux distro - thanks to the team at SUSE Studios anybody is able to rollout their custom Linux distro with own customizations, selection of software's and branding within minutes via a simple to use online interface.\n",
            "\n",
            "QUESTION:\n",
            "What are the best operating systems for software development??\n",
            "\n",
            "ANSWER:\n",
            "There is no one-size-fits-all answer to this question, as the best operating system for software development will vary depending on your individual needs and preferences. However, some of the most popular operating systems for software development include:\n",
            "\n",
            "* **Linux** - Linux is a free and open-source operating system that is known for its stability, security, and flexibility. It is a popular choice for software developers because it offers a wide range of development tools and support options.\n",
            "* **macOS** - macOS is a proprietary operating system developed by Apple. It is known for its user-friendly interface and its integration with other\n"
          ]
        }
      ],
      "source": [
        "query = \"What are the best operating systems for software development?\"\n",
        "response = rag(query=query, prompt=PROMPT)\n",
        "print(response)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GkTUofBRSOCy",
        "outputId": "9172c46a-6c3d-4410-d905-9b77a880261a"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Pulling relevant data sources from Redis\n",
            "Relevant sources found!\n",
            "\n",
            "Full prompt:\n",
            "\n",
            " You are a helpful virtual technology and IT assistant. Use the hacker news posts below as relevant context and sources to help answer the user question. Don't blindly make things up.\n",
            "\n",
            "SOURCES:\n",
            "-Not a licensing question, but a developer experience question. Any pros and cons to call out ? Contributions process, commercial support, if any ?\n",
            "-Most often than not an attempt to introduce an open source product to an \"enterprise\" environment hits this, what I call, \"blame wall\". One of the typical arguments against OSS is \"if it fails, who do we blame?\".<p>What arguments do you use in these situations?\n",
            "-Between organizing and planning new ideas (idea boards), managing current projects, content scheduling, and daily to-do tasks, do you find it difficult to keep track of all of these tasks, or have you found a solution?\n",
            "\n",
            "QUESTION:\n",
            "What are some commonly reported problems with MacBooks and MacOS for software dev purposes??\n",
            "\n",
            "ANSWER:\n",
            "There are a few commonly reported problems with MacBooks and MacOS for software development purposes.\n",
            "\n",
            "* **The price:** MacBooks are generally more expensive than other laptops on the market. This can be a major deterrent for developers who are on a budget.\n",
            "* **The lack of ports:** MacBooks have a limited number of ports, which can make it difficult to connect to external devices.\n",
            "* **The battery life:** MacBooks have a relatively short battery life, which can be a problem for developers who need to work on their laptops for long periods of time.\n",
            "* **The software ecosystem:** MacOS is not as popular as\n"
          ]
        }
      ],
      "source": [
        "query = \"What are some commonly reported problems with MacBooks and MacOS for software dev purposes?\"\n",
        "response = rag(query=query, prompt=PROMPT)\n",
        "print(response)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Rs-PweeYTjuE"
      },
      "source": [
        "Clearly this example dataset (hackernews) is not the only example we could work with and it 's certainly not \"production\" ready out of the gate. This is also only utilizing a subset (1000 records) of the actual data for teaching purposes.\n",
        "\n",
        "However, this example demonstrates how you can combine external sources of data and LLMs to surface more useful information."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FziAlkeg92U4"
      },
      "source": [
        "### LLM Caching\n",
        "\n",
        "**LLM Caching** is an advanced strategy used to optimize the performance of Large Language Model (LLM) applications. Utilizing the ultra-fast, in-memory data store of Redis, LLM Caching enables the storage and quick retrieval of pre-computed responses generated by Google's Vertex AI (PaLM). This means the computationally expensive process of response generation, especially for repetitive queries, is significantly reduced, resulting in faster response times and efficient resource utilization. This pairing of Google's powerful generative AI capabilities with Redis' high-performance caching system thus facilitates a more scalable and performant architecture for LLM applications, improving overall user experience and application reliability.\n",
        "\n",
        "There are primarily two modes of caching for LLMs:\n",
        "- Standard Caching\n",
        "- Semantic Caching"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r6YK-Y_hT4OD"
      },
      "source": [
        "#### Standard Caching\n",
        "\n",
        "Standard caching for LLMs involves simply matching an exact phrase or prompt that has been provided before. We can return the previously used response from the LLM in order to speed up the throughput of the system overall and reduce redundant computation."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "metadata": {
        "id": "FjOvOhJ2T3Wr"
      },
      "outputs": [],
      "source": [
        "# Some boiler plate helper methods\n",
        "import hashlib\n",
        "\n",
        "def hash_input(prefix: str, _input: str):\n",
        "    return prefix + hashlib.sha256(_input.encode(\"utf-8\")).hexdigest()\n",
        "\n",
        "def standard_check(key: str):\n",
        "  # function to perform a standard cache check\n",
        "    res = redis_client.hgetall(key)\n",
        "    if res:\n",
        "      return res[b'response'].decode('utf-8')\n",
        "\n",
        "def cache_response(query: str, response: str):\n",
        "    key = hash_input(\"llmcache:\", query)\n",
        "    redis_client.hset(key, mapping={\"prompt\": query, \"response\": response})\n",
        "\n",
        "# LLM Cache wrapper / decorator function\n",
        "def standard_llmcache(llm_callable):\n",
        "    def wrapper(*args, **kwargs):\n",
        "        # Check LLM Cache first\n",
        "        key = hash_input(\"llmcache:\", *args, **kwargs)\n",
        "        response = standard_check(key)\n",
        "        # Check if we have a cached response we can use\n",
        "        if response:\n",
        "            return response\n",
        "        # Otherwise execute the llm callable here\n",
        "        response = llm_callable(*args, **kwargs)\n",
        "        cache_response(query, response)\n",
        "        return response\n",
        "\n",
        "    return wrapper"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "metadata": {
        "id": "gmjOXe66T_nx"
      },
      "outputs": [],
      "source": [
        "# Define a function that invokes the PaLM API wrapped with a cache check\n",
        "\n",
        "@standard_llmcache\n",
        "def ask_palm(query: str):\n",
        "  prompt = PROMPT\n",
        "  response = rag(query, prompt, verbose=False)\n",
        "  return response"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 128
        },
        "id": "L2enaTNcXCoM",
        "outputId": "289829af-f931-4c67-9b0b-d2753a7bb36a"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "CPU times: user 16.7 ms, sys: 119 µs, total: 16.8 ms\n",
            "Wall time: 1.24 s\n"
          ]
        },
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'There is no one-size-fits-all answer to this question, as the best operating system for software development will vary depending on your individual needs and preferences. However, some of the most popular operating systems for software development include:\\n\\n* **Linux** - Linux is a free and open-source operating system that is known for its stability, security, and flexibility. It is a popular choice for software developers because it offers a wide range of development tools and support options.\\n* **macOS** - macOS is a proprietary operating system developed by Apple. It is known for its user-friendly interface and its integration with other'"
            ]
          },
          "execution_count": 31,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "%%time\n",
        "\n",
        "query = \"What are the best operating systems for software development?\"\n",
        "\n",
        "ask_palm(query)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hGE852RfXutZ"
      },
      "source": [
        "Now if we ask the same question again -- we should get the same response in near real-time."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 128
        },
        "id": "ThheflkTXCzk",
        "outputId": "cc41531c-c90a-49fb-8cee-66583e0446dd"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "CPU times: user 1.12 ms, sys: 0 ns, total: 1.12 ms\n",
            "Wall time: 1.12 ms\n"
          ]
        },
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'There is no one-size-fits-all answer to this question, as the best operating system for software development will vary depending on your individual needs and preferences. However, some of the most popular operating systems for software development include:\\n\\n* **Linux** - Linux is a free and open-source operating system that is known for its stability, security, and flexibility. It is a popular choice for software developers because it offers a wide range of development tools and support options.\\n* **macOS** - macOS is a proprietary operating system developed by Apple. It is known for its user-friendly interface and its integration with other'"
            ]
          },
          "execution_count": 32,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "%%time\n",
        "\n",
        "ask_palm(query)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SZVIhYqNXDAH"
      },
      "source": [
        "#### Semantic Caching\n",
        "Semantic caching builds off of the same concept as above, but takes advantage of Redis' vector database features to lookup smenatically similar prompts that have been applied before -- this makes it possible to cache responses of queries that are very similar in meaning, but not necessarily using the exact same words + phrases. This widens the \"hit rate\", making LLM caching even more effective in practice.\n",
        "\n",
        "**COMING SOON** -- we have a client tool/library in the works to best demonstrate this. But you can build your own using the functions and tools we've defined thus far."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P4Hl5U9O94ad"
      },
      "source": [
        "### Memory\n",
        "\n",
        "Giving your application access to \"memory\" for chat history is a common technique to improve the models ability to reason through recent or past conversations, gain context from previous answers, and thus provide a more accurate and acceptable response."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ud63DzBtfzUX"
      },
      "source": [
        "Below we setup simple helper functions to persist and load conversation history in a Redis List data structure."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 48,
      "metadata": {
        "id": "1ph-e7eZcFsR"
      },
      "outputs": [],
      "source": [
        "import json\n",
        "\n",
        "def add_message(prompt: str, response: str):\n",
        "    msg = {\n",
        "        \"prompt\": prompt,\n",
        "        \"response\": response\n",
        "    }\n",
        "    redis_client.lpush(\"chat-history\", json.dumps(msg))\n",
        "\n",
        "def get_messages(k: int = 5):\n",
        "    return [json.loads(msg) for msg in redis_client.lrange(\"chat-history\", 0, k)]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 44,
      "metadata": {
        "id": "rYhrwdeWcFtz"
      },
      "outputs": [],
      "source": [
        "\n",
        "query = \"Do you have any advice for getting started in the tech field as a software dev?\"\n",
        "response = rag(query, PROMPT, verbose=False)\n",
        "\n",
        "print(response)\n",
        "\n",
        "add_message(query, response)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 46,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VGS8YktgcFwg",
        "outputId": "e47c4943-6e6c-428e-eb98-92bf48691ae7"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "I would recommend starting to apply for jobs as early as possible. This will give you a head start on the competition and allow you to start networking with potential employers. It is also important to start building your skills and experience as early as possible. This can be done by taking on internships, working on personal projects, and contributing to open source projects.\n"
          ]
        }
      ],
      "source": [
        "query = \"What if I am still in college, any tips there?\"\n",
        "response = rag(query, PROMPT, verbose=False)\n",
        "\n",
        "print(response)\n",
        "\n",
        "add_message(query, response)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 49,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BaHZR5htgwTZ",
        "outputId": "d1841647-7d26-4d4d-fad1-37abc58a86ac"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[{'prompt': 'What if I am still in college, any tips there?',\n",
              "  'response': 'I would recommend starting to apply for jobs as early as possible. This will give you a head start on the competition and allow you to start networking with potential employers. It is also important to start building your skills and experience as early as possible. This can be done by taking on internships, working on personal projects, and contributing to open source projects.'},\n",
              " {'prompt': 'Do you have any advice for getting started in the tech field as a software dev?',\n",
              "  'response': 'There are a few things you can do to get started in the tech field as a software developer. First, you should learn the basics of programming. There are many resources available online to help you do this, such as tutorials, courses, and books. Once you have a basic understanding of programming, you can start working on your own projects. This will help you to build your skills and experience. You can also find work as a software developer intern. This will give you the opportunity to work on real projects and learn from experienced developers.'}]"
            ]
          },
          "execution_count": 49,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "get_messages()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "27gJ9J4Gg6R9"
      },
      "source": [
        "# Clean up"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 50,
      "metadata": {
        "id": "IEFG_p2wibLm"
      },
      "outputs": [],
      "source": [
        "# Clean up bigquery\n",
        "bq.delete_table(TABLE_ID, not_found_ok=True)\n",
        "\n",
        "bq.delete_dataset(\n",
        "    DATASET_ID, delete_contents=True, not_found_ok=True\n",
        ")\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 51,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MZ7MxqmViQTF",
        "outputId": "5b5ae70b-d1da-464a-fd48-322cbb063918"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Starting redis-stack-server, database path /var/lib/redis-stack\n",
            "12178:C 13 Jul 2023 18:37:37.554 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo\n",
            "12178:C 13 Jul 2023 18:37:37.555 # Redis version=7.0.0, bits=64, commit=00000000, modified=0, pid=12178, just started\n",
            "12178:C 13 Jul 2023 18:37:37.555 # Configuration loaded\n",
            "12178:M 13 Jul 2023 18:37:37.555 * monotonic clock: POSIX clock_gettime\n",
            "12178:M 13 Jul 2023 18:37:37.555 # Warning: Could not create server TCP listening socket *:6379: bind: Address already in use\n",
            "12178:M 13 Jul 2023 18:37:37.555 # Failed listening on port 6379 (TCP), aborting.\n"
          ]
        }
      ],
      "source": [
        "# Clean up redis\n",
        "!redis-stack-server stop"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "T4",
      "provenance": [],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "05c8192810a04e28bae2a9a092e14a32": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "060f3ff726ac449aaae164a34af16a9a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "074bb17ceefa41b390875b24b10600fd": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "0b1c14edff784e0aa0f647bdb15508c6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_36973dd486f94eef8e095960fbaf4594",
              "IPY_MODEL_76d4e46d3a8c4c178629bff3ce03ac6c",
              "IPY_MODEL_25adf9b83d484e1c98a487f9d9115d6d"
            ],
            "layout": "IPY_MODEL_e5eb5a5d764f4461a932ac9578ee296f"
          }
        },
        "0ee8f94835254c80bbc754bf5f9e7920": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_060f3ff726ac449aaae164a34af16a9a",
            "placeholder": "​",
            "style": "IPY_MODEL_24325a0fa2fe41deac681ebb25ff9124",
            "value": " 100/100 [00:00&lt;00:00, 5820.33it/s]"
          }
        },
        "12df7f4801ea44ddb2985d1b8cfd5ca2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f2e35a79172541f6999499344693ead5",
            "placeholder": "​",
            "style": "IPY_MODEL_479fa71cdee346ef9323ddd2bdef23e8",
            "value": " 100/100 [00:00&lt;00:00, 5006.15it/s]"
          }
        },
        "13fc33330ef646339cfb215ffaddf5e3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "14bb7d2203b1468883835082d2543a77": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1afb2fab4daa4643a4464cd84533a5d3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "1b76e0470a1641ffa1649de9f6d9c8d9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_cda2d0499b8a4bc0b15f5302f974ada8",
            "max": 100,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_05c8192810a04e28bae2a9a092e14a32",
            "value": 100
          }
        },
        "1ce492de9b4e4fb0b9de8e7e7c5c2630": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_969a68beb9fc480fb9bb4c542df50fcc",
            "placeholder": "​",
            "style": "IPY_MODEL_a3045526915341b38fc4bc21e46ae194",
            "value": " 100/100 [00:00&lt;00:00, 4232.69it/s]"
          }
        },
        "1ddcd8ce6eb142e2a2cfd6d7bc4f1468": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1ec20a3812fe4952bc87e1daa5b974af": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "21665d8fbda24f87bd73348a354d4c40": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_14bb7d2203b1468883835082d2543a77",
            "placeholder": "​",
            "style": "IPY_MODEL_964be662ffe948eea311a535eeaad50b",
            "value": "100%"
          }
        },
        "24325a0fa2fe41deac681ebb25ff9124": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "24790c21d9444d198f16105d6fc9edae": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "25adf9b83d484e1c98a487f9d9115d6d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_294e2ec38031449eab4af8531b5c89ab",
            "placeholder": "​",
            "style": "IPY_MODEL_074bb17ceefa41b390875b24b10600fd",
            "value": " 100/100 [00:00&lt;00:00, 6178.73it/s]"
          }
        },
        "269bb8352db54f7a95a32a32fd4a6168": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "26d052735a154239becf1f6a61edd446": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ef6f1d909a544b5a9a3e97d6b4881fb7",
            "placeholder": "​",
            "style": "IPY_MODEL_a1f95bbb6516429e95265df33d85bd49",
            "value": " 100/100 [00:00&lt;00:00, 5412.14it/s]"
          }
        },
        "281a72b1a960408f9b80cdbaf9bfd018": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2856930aeea74b04a27a9bb6a65888bf": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "294e2ec38031449eab4af8531b5c89ab": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2b340eef090c4b0885af21681d093d53": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2f09e78298834ae49724b54f97641dae": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "31d5d67b2815452e8ab7ef1a532f0127": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "36380a55382745ca967109278259cb58": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_dc698bb7bac14221a563b27f3112bfa5",
            "placeholder": "​",
            "style": "IPY_MODEL_500aa74cd7be4e73924193d6d109c481",
            "value": "100%"
          }
        },
        "36973dd486f94eef8e095960fbaf4594": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6bab4d01b47344aaa28c58e5422a0c8e",
            "placeholder": "​",
            "style": "IPY_MODEL_794553ea60864927957962961c3ae729",
            "value": "100%"
          }
        },
        "3d28efe8711d48c7a41b6deac843d70d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4256261f63f94c78850870d74f791616": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "479fa71cdee346ef9323ddd2bdef23e8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "47c8dd9c84684445be6aca676b57c10b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "49c258cd69634e3c9ae5ac43b4de8ccb": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "4dc65ac9471942eabba7739b9660c8a8": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "500aa74cd7be4e73924193d6d109c481": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "52874ef92ba048358171016b5e6bb8f3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "53560e23ae084a28b114b8046c500c67": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": "20px"
          }
        },
        "598ff24416f94915bf25228d66383725": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5be3e03f5dd842b1b538b3fd4e3d0d1e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_aa428e67455147808b84f81b513be072",
            "max": 100,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_c655d8b7a8c74dc6adaac4e7071d65de",
            "value": 100
          }
        },
        "5d78a16175584be397fef3a6ded93a31": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_60dbf3b19aa44ce0a50916b14f223ac8",
            "placeholder": "​",
            "style": "IPY_MODEL_634ef8a516524c889fd97bc6592a3e77",
            "value": ""
          }
        },
        "5fbb0fa49b5443b88e74dd91a2beb737": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "60dbf3b19aa44ce0a50916b14f223ac8": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "634ef8a516524c889fd97bc6592a3e77": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "669ede3b288f4a13b4bd431b6e612269": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_a9a341478021418bbc113c749bd14d52",
              "IPY_MODEL_5be3e03f5dd842b1b538b3fd4e3d0d1e",
              "IPY_MODEL_12df7f4801ea44ddb2985d1b8cfd5ca2"
            ],
            "layout": "IPY_MODEL_ba16a3081ab648bb9e3ea7d46bda3933"
          }
        },
        "6836fd622af7416bab90a7a297d09e4b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_976c597448374d9f9e3ca64a31527d25",
            "placeholder": "​",
            "style": "IPY_MODEL_a5fb3b98fe5c40b3ba213b0668d60329",
            "value": " 100/100 [00:00&lt;00:00, 5368.09it/s]"
          }
        },
        "690faf9d30434d15b98dc12efe2530d7": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1ddcd8ce6eb142e2a2cfd6d7bc4f1468",
            "max": 100,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_1ec20a3812fe4952bc87e1daa5b974af",
            "value": 100
          }
        },
        "6bab4d01b47344aaa28c58e5422a0c8e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6c6d8f5cf65b4a7183e748f76032f326": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6cb177b705184e96aa07afd173f7fe2b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_fe0a36adc9a848ce855f5761c2ca7223",
            "placeholder": "​",
            "style": "IPY_MODEL_d835d9a864aa4048bc27bc0b63217d15",
            "value": " 100/100 [00:00&lt;00:00, 4780.11it/s]"
          }
        },
        "6da0517dbae44ebca2f1c691352565cb": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "71018db51be0484982d494bdcd75c058": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_36380a55382745ca967109278259cb58",
              "IPY_MODEL_ac4e9f40b45745e2b4f3977e0ef190ad",
              "IPY_MODEL_c00c8dd68ac748969604763a0decbc8c"
            ],
            "layout": "IPY_MODEL_877f2b283e61492bb736c59e3718a060"
          }
        },
        "71daa81bf8d841e28fec06962217550c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_fbdcf206b1824c5ba07645c8b9f3ee0f",
              "IPY_MODEL_ec6b45219f624bed8ac56240da8876fa",
              "IPY_MODEL_0ee8f94835254c80bbc754bf5f9e7920"
            ],
            "layout": "IPY_MODEL_cfc4b18e7544490cbd3e77f6c0911a1c"
          }
        },
        "76d4e46d3a8c4c178629bff3ce03ac6c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_df9d6ecb74a441568c94e5fc5526964b",
            "max": 100,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_a30987d3511147a4b5f63f89e5f4142d",
            "value": 100
          }
        },
        "7706766e3c5f437abf8831183f2aed5c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "794553ea60864927957962961c3ae729": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "79dca477a944417cabf7bae71aa1a7ab": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_f61bd26b951743dc9f5cfc5b1b5035b2",
              "IPY_MODEL_1b76e0470a1641ffa1649de9f6d9c8d9",
              "IPY_MODEL_b33bb8f83ebf43e984a29c5dd0ee23de"
            ],
            "layout": "IPY_MODEL_6da0517dbae44ebca2f1c691352565cb"
          }
        },
        "7a2d8e690cc341aeb9182337872c33e8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "7da3fd91f223413589584abbfd1a83ec": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "81b65549ba2d49fbbef179fc67bdec57": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "838d480c8e5040c78898c86da48d98d0": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "848931de937e47b1a19c71890ea4ce81": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "877f2b283e61492bb736c59e3718a060": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8bda5f3b7c95412a9f940761740a398d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "8e06cd83044144c6bd8d7532051ad041": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9aa6e46b56dd4bdfac4debb58da60624",
            "max": 100,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_dfbc675c3821466080c20e92f1435641",
            "value": 100
          }
        },
        "93ab13f7126742a8bd585770e8489344": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "964be662ffe948eea311a535eeaad50b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "969a68beb9fc480fb9bb4c542df50fcc": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "976c597448374d9f9e3ca64a31527d25": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9aa6e46b56dd4bdfac4debb58da60624": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9b018155074141948c79f692f8789810": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_a5fa6f0a89cc4cbfa4a042222d19e908",
              "IPY_MODEL_690faf9d30434d15b98dc12efe2530d7",
              "IPY_MODEL_26d052735a154239becf1f6a61edd446"
            ],
            "layout": "IPY_MODEL_b67159137759433b934c69941159c9d5"
          }
        },
        "9b224ef5e6074d35afc36b6a5ca6ccad": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3d28efe8711d48c7a41b6deac843d70d",
            "placeholder": "​",
            "style": "IPY_MODEL_d25ab4733845475c94ef474b9cb743d1",
            "value": " 10/? [00:41&lt;00:00,  4.10s/it]"
          }
        },
        "9c3d67a4a51e4194a253fb5df203a0dd": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_598ff24416f94915bf25228d66383725",
            "placeholder": "​",
            "style": "IPY_MODEL_848931de937e47b1a19c71890ea4ce81",
            "value": "100%"
          }
        },
        "9e139fda21fb425385b943762d173280": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_838d480c8e5040c78898c86da48d98d0",
            "max": 100,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_31d5d67b2815452e8ab7ef1a532f0127",
            "value": 100
          }
        },
        "9f49cc0378f74bc1915a179ea27a775c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_df0845e7ac304662b9708d5931d9ff55",
            "max": 100,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_d358eefe15d74f9f83a13ad523f88ed6",
            "value": 100
          }
        },
        "a1f95bbb6516429e95265df33d85bd49": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a3045526915341b38fc4bc21e46ae194": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a30987d3511147a4b5f63f89e5f4142d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "a5fa6f0a89cc4cbfa4a042222d19e908": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2856930aeea74b04a27a9bb6a65888bf",
            "placeholder": "​",
            "style": "IPY_MODEL_e02593d5b39c49fc8761c5100ce5faeb",
            "value": "100%"
          }
        },
        "a5fb3b98fe5c40b3ba213b0668d60329": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a9a341478021418bbc113c749bd14d52": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2b340eef090c4b0885af21681d093d53",
            "placeholder": "​",
            "style": "IPY_MODEL_c7a576d22b8049319e43eb8339dec0c6",
            "value": "100%"
          }
        },
        "aa428e67455147808b84f81b513be072": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ac4e9f40b45745e2b4f3977e0ef190ad": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d613018ad37b4c028793c990a0c27039",
            "max": 100,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_93ab13f7126742a8bd585770e8489344",
            "value": 100
          }
        },
        "b33bb8f83ebf43e984a29c5dd0ee23de": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6c6d8f5cf65b4a7183e748f76032f326",
            "placeholder": "​",
            "style": "IPY_MODEL_52874ef92ba048358171016b5e6bb8f3",
            "value": " 100/100 [00:00&lt;00:00, 3315.16it/s]"
          }
        },
        "b421e51a623d4247966069140f55b569": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b67159137759433b934c69941159c9d5": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b78297245c4743828568cd3c50878e8b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4256261f63f94c78850870d74f791616",
            "max": 100,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_f0484fc7733648569a26af7506d87219",
            "value": 100
          }
        },
        "ba16a3081ab648bb9e3ea7d46bda3933": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bce414b1c27b4243868a169b04516b58": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_21665d8fbda24f87bd73348a354d4c40",
              "IPY_MODEL_b78297245c4743828568cd3c50878e8b",
              "IPY_MODEL_1ce492de9b4e4fb0b9de8e7e7c5c2630"
            ],
            "layout": "IPY_MODEL_4dc65ac9471942eabba7739b9660c8a8"
          }
        },
        "beddfbd5812d455bb1c717a416f70721": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_d99ff2a10f7e479890f54c285b7a24ef",
              "IPY_MODEL_9f49cc0378f74bc1915a179ea27a775c",
              "IPY_MODEL_6cb177b705184e96aa07afd173f7fe2b"
            ],
            "layout": "IPY_MODEL_281a72b1a960408f9b80cdbaf9bfd018"
          }
        },
        "bf7051abdc46495890c1ec0b2b69201a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c00c8dd68ac748969604763a0decbc8c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b421e51a623d4247966069140f55b569",
            "placeholder": "​",
            "style": "IPY_MODEL_bf7051abdc46495890c1ec0b2b69201a",
            "value": " 100/100 [00:00&lt;00:00, 7174.41it/s]"
          }
        },
        "c616e3700d024b528f8737ef2bc8ccc9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c655d8b7a8c74dc6adaac4e7071d65de": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "c7a576d22b8049319e43eb8339dec0c6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "cc3b7757cf8b4af6bb67cd3c3e9b0c7b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_cc78716187504d2f8f4b5a1a720e9e81",
            "placeholder": "​",
            "style": "IPY_MODEL_7a2d8e690cc341aeb9182337872c33e8",
            "value": " 100/100 [00:00&lt;00:00, 3352.01it/s]"
          }
        },
        "cc78716187504d2f8f4b5a1a720e9e81": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "cda2d0499b8a4bc0b15f5302f974ada8": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "cfc4b18e7544490cbd3e77f6c0911a1c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d03cde2de63b4be5b0d056051a219fc3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_5d78a16175584be397fef3a6ded93a31",
              "IPY_MODEL_e3ea955d38ea4266834d92f4bcae323d",
              "IPY_MODEL_9b224ef5e6074d35afc36b6a5ca6ccad"
            ],
            "layout": "IPY_MODEL_47c8dd9c84684445be6aca676b57c10b"
          }
        },
        "d1f4e6168ce048f584ead366e880e857": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_e66a2f6eedee4eba889a7ce8aa4b7043",
              "IPY_MODEL_8e06cd83044144c6bd8d7532051ad041",
              "IPY_MODEL_cc3b7757cf8b4af6bb67cd3c3e9b0c7b"
            ],
            "layout": "IPY_MODEL_5fbb0fa49b5443b88e74dd91a2beb737"
          }
        },
        "d25ab4733845475c94ef474b9cb743d1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d314d8c4d38a48d5b238ce2e8ac87b4e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d358eefe15d74f9f83a13ad523f88ed6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "d613018ad37b4c028793c990a0c27039": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d835d9a864aa4048bc27bc0b63217d15": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d99ff2a10f7e479890f54c285b7a24ef": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7da3fd91f223413589584abbfd1a83ec",
            "placeholder": "​",
            "style": "IPY_MODEL_269bb8352db54f7a95a32a32fd4a6168",
            "value": "100%"
          }
        },
        "dc698bb7bac14221a563b27f3112bfa5": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "df0845e7ac304662b9708d5931d9ff55": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "df9d6ecb74a441568c94e5fc5526964b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "dfbc675c3821466080c20e92f1435641": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "e02593d5b39c49fc8761c5100ce5faeb": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e15ea32d9bb84177b226b64b1c253815": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_9c3d67a4a51e4194a253fb5df203a0dd",
              "IPY_MODEL_9e139fda21fb425385b943762d173280",
              "IPY_MODEL_6836fd622af7416bab90a7a297d09e4b"
            ],
            "layout": "IPY_MODEL_24790c21d9444d198f16105d6fc9edae"
          }
        },
        "e3ea955d38ea4266834d92f4bcae323d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_53560e23ae084a28b114b8046c500c67",
            "max": 1,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_13fc33330ef646339cfb215ffaddf5e3",
            "value": 1
          }
        },
        "e5eb5a5d764f4461a932ac9578ee296f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e66a2f6eedee4eba889a7ce8aa4b7043": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2f09e78298834ae49724b54f97641dae",
            "placeholder": "​",
            "style": "IPY_MODEL_8bda5f3b7c95412a9f940761740a398d",
            "value": "100%"
          }
        },
        "ec6b45219f624bed8ac56240da8876fa": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d314d8c4d38a48d5b238ce2e8ac87b4e",
            "max": 100,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_49c258cd69634e3c9ae5ac43b4de8ccb",
            "value": 100
          }
        },
        "ef6f1d909a544b5a9a3e97d6b4881fb7": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f0484fc7733648569a26af7506d87219": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "f2e35a79172541f6999499344693ead5": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f61bd26b951743dc9f5cfc5b1b5035b2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_81b65549ba2d49fbbef179fc67bdec57",
            "placeholder": "​",
            "style": "IPY_MODEL_c616e3700d024b528f8737ef2bc8ccc9",
            "value": "100%"
          }
        },
        "fbdcf206b1824c5ba07645c8b9f3ee0f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7706766e3c5f437abf8831183f2aed5c",
            "placeholder": "​",
            "style": "IPY_MODEL_1afb2fab4daa4643a4464cd84533a5d3",
            "value": "100%"
          }
        },
        "fe0a36adc9a848ce855f5761c2ca7223": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}