05_llama_on_groq.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b163ea3f-3201-407a-85de-9a046113f898",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "# Llama on Groq"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38afa96f-8b1d-42ca-b084-2351f4eb6a9c",
   "metadata": {},
   "source": [
    "[Llama3](https://github.com/meta-llama/llama3) is one of the most advanced open-source LLM models.\n",
    "\n",
    "The original 80 billion parameter model (without quantanization) consumes about [160 Gb of VRAM](https://nlpcloud.com/how-to-install-and-deploy-llama-3-into-production.html). For practical use, especially for smaller organizations or individual developers, using a cloud service like Groq can be a more feasible option."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "577bf5a4-a80b-4509-b768-2915c82bef7f",
   "metadata": {},
   "source": [
    "## Set up the environment and use Llama3"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ccf175c3-5133-43dd-a784-acacbedd3798",
   "metadata": {},
   "source": [
    "To set up and use Llama3 with Groq, you can follow these steps:\n",
    "\n",
    "1. **Log in to Groq**: Visit [groq.com](https://groq.com) and log in with your credentials.\n",
    "\n",
    "2. **Create an API Key**:\n",
    "   - Navigate to the [API section](https://console.groq.com/keys) of the Groq console.\n",
    "   - Generate a new API key. This key will be used to authenticate your requests to the Groq API.\n",
    "\n",
    "3. **Set Up Your Environment**:\n",
    "   - On your local machine, create a `.env` file. This file will store your API key securely.\n",
    "   - Add the following line to your `.env` file:\n",
    "     ```\n",
    "     GROQ_API_KEY=\"PASTE_YOUR_API_KEY_HERE\"\n",
    "     ```\n",
    "   - Replace `\"PASTE_YOUR_API_KEY_HERE\"` with the actual API key you generated.\n",
    "\n",
    "4. **Consult the API Documentation**:\n",
    "   - The API documentation, including details on usage, rate limits, and other configurations, can be found [here](https://console.groq.com/docs/quickstart). This documentation will guide you on how to interact with the API, make requests, and handle responses."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f71661f6-7fa9-4c25-8fa5-0d5919c5c412",
   "metadata": {},
   "source": [
    "Install the Groq library:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b935a55e-2d87-4c56-af12-58ab680a0fa9",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting groq\n",
      "  Downloading groq-0.6.0-py3-none-any.whl.metadata (13 kB)\n",
      "Requirement already satisfied: anyio<5,>=3.5.0 in /Users/lex/opt/miniconda3/envs/env_mc_asr_312/lib/python3.12/site-packages (from groq) (4.3.0)\n",
      "Collecting distro<2,>=1.7.0 (from groq)\n",
      "  Downloading distro-1.9.0-py3-none-any.whl.metadata (6.8 kB)\n",
      "Requirement already satisfied: httpx<1,>=0.23.0 in /Users/lex/opt/miniconda3/envs/env_mc_asr_312/lib/python3.12/site-packages (from groq) (0.27.0)\n",
      "Collecting pydantic<3,>=1.9.0 (from groq)\n",
      "  Downloading pydantic-2.7.1-py3-none-any.whl.metadata (107 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m107.3/107.3 kB\u001b[0m \u001b[31m542.1 kB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: sniffio in /Users/lex/opt/miniconda3/envs/env_mc_asr_312/lib/python3.12/site-packages (from groq) (1.3.1)\n",
      "Requirement already satisfied: typing-extensions<5,>=4.7 in /Users/lex/opt/miniconda3/envs/env_mc_asr_312/lib/python3.12/site-packages (from groq) (4.11.0)\n",
      "Requirement already satisfied: idna>=2.8 in /Users/lex/opt/miniconda3/envs/env_mc_asr_312/lib/python3.12/site-packages (from anyio<5,>=3.5.0->groq) (3.7)\n",
      "Requirement already satisfied: certifi in /Users/lex/opt/miniconda3/envs/env_mc_asr_312/lib/python3.12/site-packages (from httpx<1,>=0.23.0->groq) (2024.2.2)\n",
      "Requirement already satisfied: httpcore==1.* in /Users/lex/opt/miniconda3/envs/env_mc_asr_312/lib/python3.12/site-packages (from httpx<1,>=0.23.0->groq) (1.0.5)\n",
      "Requirement already satisfied: h11<0.15,>=0.13 in /Users/lex/opt/miniconda3/envs/env_mc_asr_312/lib/python3.12/site-packages (from httpcore==1.*->httpx<1,>=0.23.0->groq) (0.14.0)\n",
      "Collecting annotated-types>=0.4.0 (from pydantic<3,>=1.9.0->groq)\n",
      "  Downloading annotated_types-0.6.0-py3-none-any.whl.metadata (12 kB)\n",
      "Collecting pydantic-core==2.18.2 (from pydantic<3,>=1.9.0->groq)\n",
      "  Downloading pydantic_core-2.18.2-cp312-cp312-macosx_11_0_arm64.whl.metadata (6.5 kB)\n",
      "Downloading groq-0.6.0-py3-none-any.whl (84 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m84.9/84.9 kB\u001b[0m \u001b[31m1.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading distro-1.9.0-py3-none-any.whl (20 kB)\n",
      "Downloading pydantic-2.7.1-py3-none-any.whl (409 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m409.3/409.3 kB\u001b[0m \u001b[31m1.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading pydantic_core-2.18.2-cp312-cp312-macosx_11_0_arm64.whl (1.8 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.8/1.8 MB\u001b[0m \u001b[31m3.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0mm\n",
      "\u001b[?25hDownloading annotated_types-0.6.0-py3-none-any.whl (12 kB)\n",
      "Installing collected packages: pydantic-core, distro, annotated-types, pydantic, groq\n",
      "Successfully installed annotated-types-0.6.0 distro-1.9.0 groq-0.6.0 pydantic-2.7.1 pydantic-core-2.18.2\n"
     ]
    }
   ],
   "source": [
    "!pip install groq"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1515e0b6-4932-438d-ae58-a324fc45512c",
   "metadata": {},
   "source": [
    "Install the dotenv library:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "c13ed24a-09f1-48e9-b8a4-8ac2be918ae5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting python-dotenv\n",
      "  Downloading python_dotenv-1.0.1-py3-none-any.whl.metadata (23 kB)\n",
      "Downloading python_dotenv-1.0.1-py3-none-any.whl (19 kB)\n",
      "Installing collected packages: python-dotenv\n",
      "Successfully installed python-dotenv-1.0.1\n"
     ]
    }
   ],
   "source": [
    "!pip install python-dotenv"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6bbd4536-769b-469f-90b6-81f9dbbeffc1",
   "metadata": {},
   "source": [
    "Import the libraries:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "13705912-63e2-4091-bc50-f89802e883cf",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import the standard libraries.\n",
    "import os\n",
    "\n",
    "# Import the third party libraries.\n",
    "from dotenv import load_dotenv\n",
    "from groq import Groq"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e29d6c87-dd28-42fb-bce2-7d9b05fd9e69",
   "metadata": {},
   "source": [
    "Set the environment:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "5a42d247-b6c2-41c2-9f42-8a6d5cc44ad7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "load_dotenv()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1129e60d-9c8d-42e3-850b-1610c459c903",
   "metadata": {},
   "source": [
    "Test the API:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "88d2488b-c7b9-49df-a640-b85dc6f3d205",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fast language models are crucial because they enable real-time natural language understanding and generation, which is essential for applications such as chatbots, virtual assistants, and language translation, allowing for swift and efficient human-machine interaction.\n"
     ]
    }
   ],
   "source": [
    "client = Groq(\n",
    "    api_key=os.environ.get(\"GROQ_API_KEY\"),\n",
    ")\n",
    "\n",
    "chat_completion = client.chat.completions.create(\n",
    "    messages=[\n",
    "        {\n",
    "            \"role\": \"user\",\n",
    "            \"content\": \"Explain the importance of fast language models in one sentence.\",\n",
    "        }\n",
    "    ],\n",
    "    model=\"llama3-70b-8192\",\n",
    ")\n",
    "\n",
    "print(chat_completion.choices[0].message.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d17a0523-20ab-4d9c-9c20-7dfd7325eae9",
   "metadata": {},
   "source": [
    "## Summarize the whisper's transcript"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6dabe629-555b-4b80-9ff1-297309f94f98",
   "metadata": {},
   "source": [
    "Import the cleaned whishper transcript:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "edf4aea3-8813-4768-afe1-c5e25d19cd92",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Welcome to IBM Think 2023. AI-generated art. AI-generated songs. AI, what is that? It sure is a lot of fun. But when foundation models are applied to big business, well, you need to think bigger. Because AI in business needs to be held to a higher standard. Built to be trusted, secured, and adaptable. This isn't simple automation that is only trained to do one thing. This is AI that is built and focused to work across your organization. This isn't committing to a single system. This is hybrid-ready AI that can scale across your systems. This isn't wondering where an answer came from. This is AI that can show its work. When you build AI into the core of your business, you can go so much further. This is more than AI. This is AI for business. Let's create. Please welcome Senior Vice President and Director of Research, IBM, Dr. Dario Gil. Hello. Welcome. Welcome. The last session of Think. And I understand some of you even had a drink. How special. So I hope you've enjoyed the last two da\""
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with open(\"./data/cleaned_whisper_transcript_max_context_64.txt\", \"r\") as file:\n",
    "    transcript = file.read()\n",
    "transcript[0:1000]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eeb2b370-8ae8-48e2-b565-e6e306363b41",
   "metadata": {},
   "source": [
    "<br>\n",
    "Estimate a number of tokens in the transcript (1 token is ~4 characters for English):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "699b77af-7905-40b7-8b22-3cc5107284a2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6876.0"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(transcript) / 4"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b373bffb-54fc-4a4d-8e62-4eae47e110c7",
   "metadata": {},
   "source": [
    "If we try to send a full text, we get an error message like this:\n",
    "```bash\n",
    "Error code: 429 - {'error': {'message': 'Rate limit reached for model `llama3-70b-8192` in organization `org_` on tokens per minute (TPM): Limit 6000, Used 0, Requested 6897. Please try again in 8.97s. Visit https://console.groq.com/docs/rate-limits for more information.', 'type': 'tokens', 'code': 'rate_limit_exceeded'}}\n",
    "```\n",
    "This is because the rate limit for llama3-70b-8192 is [6000 per minute](https://console.groq.com/settings/limits) while we have almost 7000 tokens:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "a9c8243f-e841-4f8c-9183-fc2e204e1a55",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [
    {
     "ename": "RateLimitError",
     "evalue": "Error code: 429 - {'error': {'message': 'Rate limit reached for model `llama3-70b-8192` in organization `org_01hxzrsnf8e94sgaavqxytxpxt` on tokens per minute (TPM): Limit 6000, Used 0, Requested 6897. Please try again in 8.97s. Visit https://console.groq.com/docs/rate-limits for more information.', 'type': 'tokens', 'code': 'rate_limit_exceeded'}}",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mRateLimitError\u001b[0m                            Traceback (most recent call last)",
      "Cell \u001b[0;32mIn[6], line 5\u001b[0m\n\u001b[1;32m      1\u001b[0m client \u001b[38;5;241m=\u001b[39m Groq(\n\u001b[1;32m      2\u001b[0m     api_key\u001b[38;5;241m=\u001b[39mos\u001b[38;5;241m.\u001b[39menviron\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mGROQ_API_KEY\u001b[39m\u001b[38;5;124m\"\u001b[39m),\n\u001b[1;32m      3\u001b[0m )\n\u001b[0;32m----> 5\u001b[0m chat_completion \u001b[38;5;241m=\u001b[39m \u001b[43mclient\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mchat\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mcompletions\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mcreate\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m      6\u001b[0m \u001b[43m    \u001b[49m\u001b[43mmessages\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43m[\u001b[49m\n\u001b[1;32m      7\u001b[0m \u001b[43m        \u001b[49m\u001b[43m{\u001b[49m\n\u001b[1;32m      8\u001b[0m \u001b[43m            \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mrole\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43muser\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[1;32m      9\u001b[0m \u001b[43m            \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mcontent\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43mf\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mPlease summarize the following text in no less than 500 words:\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;132;43;01m{\u001b[39;49;00m\u001b[43mtranscript\u001b[49m\u001b[38;5;132;43;01m}\u001b[39;49;00m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[1;32m     10\u001b[0m \u001b[43m        \u001b[49m\u001b[43m}\u001b[49m\n\u001b[1;32m     11\u001b[0m \u001b[43m    \u001b[49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     12\u001b[0m \u001b[43m    \u001b[49m\u001b[43mmodel\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mllama3-70b-8192\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[1;32m     13\u001b[0m \u001b[43m)\u001b[49m\n\u001b[1;32m     15\u001b[0m \u001b[38;5;28mprint\u001b[39m(chat_completion\u001b[38;5;241m.\u001b[39mchoices[\u001b[38;5;241m0\u001b[39m]\u001b[38;5;241m.\u001b[39mmessage\u001b[38;5;241m.\u001b[39mcontent)\n",
      "File \u001b[0;32m~/opt/miniconda3/envs/env_mc_asr_312/lib/python3.12/site-packages/groq/resources/chat/completions.py:179\u001b[0m, in \u001b[0;36mCompletions.create\u001b[0;34m(self, messages, model, frequency_penalty, logit_bias, logprobs, max_tokens, n, presence_penalty, response_format, seed, stop, stream, temperature, tool_choice, tools, top_logprobs, top_p, user, extra_headers, extra_query, extra_body, timeout)\u001b[0m\n\u001b[1;32m    136\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mcreate\u001b[39m(\n\u001b[1;32m    137\u001b[0m     \u001b[38;5;28mself\u001b[39m,\n\u001b[1;32m    138\u001b[0m     \u001b[38;5;241m*\u001b[39m,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    162\u001b[0m     timeout: \u001b[38;5;28mfloat\u001b[39m \u001b[38;5;241m|\u001b[39m httpx\u001b[38;5;241m.\u001b[39mTimeout \u001b[38;5;241m|\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;241m|\u001b[39m NotGiven \u001b[38;5;241m=\u001b[39m NOT_GIVEN,\n\u001b[1;32m    163\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m ChatCompletion \u001b[38;5;241m|\u001b[39m Stream[ChatCompletionChunk]:\n\u001b[1;32m    164\u001b[0m \u001b[38;5;250m    \u001b[39m\u001b[38;5;124;03m\"\"\"\u001b[39;00m\n\u001b[1;32m    165\u001b[0m \u001b[38;5;124;03m    Creates a completion for a chat prompt\u001b[39;00m\n\u001b[1;32m    166\u001b[0m \n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    177\u001b[0m \u001b[38;5;124;03m      timeout: Override the client-level default timeout for this request, in seconds\u001b[39;00m\n\u001b[1;32m    178\u001b[0m \u001b[38;5;124;03m    \"\"\"\u001b[39;00m\n\u001b[0;32m--> 179\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_post\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    180\u001b[0m \u001b[43m        \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43m/openai/v1/chat/completions\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[1;32m    181\u001b[0m \u001b[43m        \u001b[49m\u001b[43mbody\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmaybe_transform\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    182\u001b[0m \u001b[43m            \u001b[49m\u001b[43m{\u001b[49m\n\u001b[1;32m    183\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmessages\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmessages\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    184\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmodel\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmodel\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    185\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mfrequency_penalty\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mfrequency_penalty\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    186\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mlogit_bias\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mlogit_bias\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    187\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mlogprobs\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mlogprobs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    188\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmax_tokens\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmax_tokens\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    189\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mn\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mn\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    190\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mpresence_penalty\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mpresence_penalty\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    191\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mresponse_format\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mresponse_format\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    192\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mseed\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mseed\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    193\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mstop\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mstop\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    194\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mstream\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mstream\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    195\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mtemperature\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtemperature\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    196\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mtool_choice\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtool_choice\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    197\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mtools\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtools\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    198\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mtop_logprobs\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtop_logprobs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    199\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mtop_p\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtop_p\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    200\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43muser\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43muser\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    201\u001b[0m \u001b[43m            \u001b[49m\u001b[43m}\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    202\u001b[0m \u001b[43m            \u001b[49m\u001b[43mcompletion_create_params\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mCompletionCreateParams\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    203\u001b[0m \u001b[43m        \u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    204\u001b[0m \u001b[43m        \u001b[49m\u001b[43moptions\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmake_request_options\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    205\u001b[0m \u001b[43m            \u001b[49m\u001b[43mextra_headers\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mextra_headers\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mextra_query\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mextra_query\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mextra_body\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mextra_body\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mtimeout\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtimeout\u001b[49m\n\u001b[1;32m    206\u001b[0m \u001b[43m        \u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    207\u001b[0m \u001b[43m        \u001b[49m\u001b[43mcast_to\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mChatCompletion\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    208\u001b[0m \u001b[43m        \u001b[49m\u001b[43mstream\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;129;43;01mor\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[1;32m    209\u001b[0m \u001b[43m        \u001b[49m\u001b[43mstream_cls\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mStream\u001b[49m\u001b[43m[\u001b[49m\u001b[43mChatCompletionChunk\u001b[49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    210\u001b[0m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[0;32m~/opt/miniconda3/envs/env_mc_asr_312/lib/python3.12/site-packages/groq/_base_client.py:1225\u001b[0m, in \u001b[0;36mSyncAPIClient.post\u001b[0;34m(self, path, cast_to, body, options, files, stream, stream_cls)\u001b[0m\n\u001b[1;32m   1211\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mpost\u001b[39m(\n\u001b[1;32m   1212\u001b[0m     \u001b[38;5;28mself\u001b[39m,\n\u001b[1;32m   1213\u001b[0m     path: \u001b[38;5;28mstr\u001b[39m,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m   1220\u001b[0m     stream_cls: \u001b[38;5;28mtype\u001b[39m[_StreamT] \u001b[38;5;241m|\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m,\n\u001b[1;32m   1221\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m ResponseT \u001b[38;5;241m|\u001b[39m _StreamT:\n\u001b[1;32m   1222\u001b[0m     opts \u001b[38;5;241m=\u001b[39m FinalRequestOptions\u001b[38;5;241m.\u001b[39mconstruct(\n\u001b[1;32m   1223\u001b[0m         method\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mpost\u001b[39m\u001b[38;5;124m\"\u001b[39m, url\u001b[38;5;241m=\u001b[39mpath, json_data\u001b[38;5;241m=\u001b[39mbody, files\u001b[38;5;241m=\u001b[39mto_httpx_files(files), \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39moptions\n\u001b[1;32m   1224\u001b[0m     )\n\u001b[0;32m-> 1225\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m cast(ResponseT, \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrequest\u001b[49m\u001b[43m(\u001b[49m\u001b[43mcast_to\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mopts\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstream\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstream_cls\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream_cls\u001b[49m\u001b[43m)\u001b[49m)\n",
      "File \u001b[0;32m~/opt/miniconda3/envs/env_mc_asr_312/lib/python3.12/site-packages/groq/_base_client.py:920\u001b[0m, in \u001b[0;36mSyncAPIClient.request\u001b[0;34m(self, cast_to, options, remaining_retries, stream, stream_cls)\u001b[0m\n\u001b[1;32m    911\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mrequest\u001b[39m(\n\u001b[1;32m    912\u001b[0m     \u001b[38;5;28mself\u001b[39m,\n\u001b[1;32m    913\u001b[0m     cast_to: Type[ResponseT],\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    918\u001b[0m     stream_cls: \u001b[38;5;28mtype\u001b[39m[_StreamT] \u001b[38;5;241m|\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m,\n\u001b[1;32m    919\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m ResponseT \u001b[38;5;241m|\u001b[39m _StreamT:\n\u001b[0;32m--> 920\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_request\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    921\u001b[0m \u001b[43m        \u001b[49m\u001b[43mcast_to\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcast_to\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    922\u001b[0m \u001b[43m        \u001b[49m\u001b[43moptions\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43moptions\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    923\u001b[0m \u001b[43m        \u001b[49m\u001b[43mstream\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    924\u001b[0m \u001b[43m        \u001b[49m\u001b[43mstream_cls\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream_cls\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    925\u001b[0m \u001b[43m        \u001b[49m\u001b[43mremaining_retries\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mremaining_retries\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    926\u001b[0m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[0;32m~/opt/miniconda3/envs/env_mc_asr_312/lib/python3.12/site-packages/groq/_base_client.py:1003\u001b[0m, in \u001b[0;36mSyncAPIClient._request\u001b[0;34m(self, cast_to, options, remaining_retries, stream, stream_cls)\u001b[0m\n\u001b[1;32m   1001\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m retries \u001b[38;5;241m>\u001b[39m \u001b[38;5;241m0\u001b[39m \u001b[38;5;129;01mand\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_should_retry(err\u001b[38;5;241m.\u001b[39mresponse):\n\u001b[1;32m   1002\u001b[0m     err\u001b[38;5;241m.\u001b[39mresponse\u001b[38;5;241m.\u001b[39mclose()\n\u001b[0;32m-> 1003\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_retry_request\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m   1004\u001b[0m \u001b[43m        \u001b[49m\u001b[43moptions\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1005\u001b[0m \u001b[43m        \u001b[49m\u001b[43mcast_to\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1006\u001b[0m \u001b[43m        \u001b[49m\u001b[43mretries\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1007\u001b[0m \u001b[43m        \u001b[49m\u001b[43merr\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mresponse\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mheaders\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1008\u001b[0m \u001b[43m        \u001b[49m\u001b[43mstream\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1009\u001b[0m \u001b[43m        \u001b[49m\u001b[43mstream_cls\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream_cls\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1010\u001b[0m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m   1012\u001b[0m \u001b[38;5;66;03m# If the response is streamed then we need to explicitly read the response\u001b[39;00m\n\u001b[1;32m   1013\u001b[0m \u001b[38;5;66;03m# to completion before attempting to access the response text.\u001b[39;00m\n\u001b[1;32m   1014\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m err\u001b[38;5;241m.\u001b[39mresponse\u001b[38;5;241m.\u001b[39mis_closed:\n",
      "File \u001b[0;32m~/opt/miniconda3/envs/env_mc_asr_312/lib/python3.12/site-packages/groq/_base_client.py:1051\u001b[0m, in \u001b[0;36mSyncAPIClient._retry_request\u001b[0;34m(self, options, cast_to, remaining_retries, response_headers, stream, stream_cls)\u001b[0m\n\u001b[1;32m   1047\u001b[0m \u001b[38;5;66;03m# In a synchronous context we are blocking the entire thread. Up to the library user to run the client in a\u001b[39;00m\n\u001b[1;32m   1048\u001b[0m \u001b[38;5;66;03m# different thread if necessary.\u001b[39;00m\n\u001b[1;32m   1049\u001b[0m time\u001b[38;5;241m.\u001b[39msleep(timeout)\n\u001b[0;32m-> 1051\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_request\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m   1052\u001b[0m \u001b[43m    \u001b[49m\u001b[43moptions\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43moptions\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1053\u001b[0m \u001b[43m    \u001b[49m\u001b[43mcast_to\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcast_to\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1054\u001b[0m \u001b[43m    \u001b[49m\u001b[43mremaining_retries\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mremaining\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1055\u001b[0m \u001b[43m    \u001b[49m\u001b[43mstream\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1056\u001b[0m \u001b[43m    \u001b[49m\u001b[43mstream_cls\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream_cls\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1057\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[0;32m~/opt/miniconda3/envs/env_mc_asr_312/lib/python3.12/site-packages/groq/_base_client.py:1003\u001b[0m, in \u001b[0;36mSyncAPIClient._request\u001b[0;34m(self, cast_to, options, remaining_retries, stream, stream_cls)\u001b[0m\n\u001b[1;32m   1001\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m retries \u001b[38;5;241m>\u001b[39m \u001b[38;5;241m0\u001b[39m \u001b[38;5;129;01mand\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_should_retry(err\u001b[38;5;241m.\u001b[39mresponse):\n\u001b[1;32m   1002\u001b[0m     err\u001b[38;5;241m.\u001b[39mresponse\u001b[38;5;241m.\u001b[39mclose()\n\u001b[0;32m-> 1003\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_retry_request\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m   1004\u001b[0m \u001b[43m        \u001b[49m\u001b[43moptions\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1005\u001b[0m \u001b[43m        \u001b[49m\u001b[43mcast_to\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1006\u001b[0m \u001b[43m        \u001b[49m\u001b[43mretries\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1007\u001b[0m \u001b[43m        \u001b[49m\u001b[43merr\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mresponse\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mheaders\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1008\u001b[0m \u001b[43m        \u001b[49m\u001b[43mstream\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1009\u001b[0m \u001b[43m        \u001b[49m\u001b[43mstream_cls\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream_cls\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1010\u001b[0m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m   1012\u001b[0m \u001b[38;5;66;03m# If the response is streamed then we need to explicitly read the response\u001b[39;00m\n\u001b[1;32m   1013\u001b[0m \u001b[38;5;66;03m# to completion before attempting to access the response text.\u001b[39;00m\n\u001b[1;32m   1014\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m err\u001b[38;5;241m.\u001b[39mresponse\u001b[38;5;241m.\u001b[39mis_closed:\n",
      "File \u001b[0;32m~/opt/miniconda3/envs/env_mc_asr_312/lib/python3.12/site-packages/groq/_base_client.py:1051\u001b[0m, in \u001b[0;36mSyncAPIClient._retry_request\u001b[0;34m(self, options, cast_to, remaining_retries, response_headers, stream, stream_cls)\u001b[0m\n\u001b[1;32m   1047\u001b[0m \u001b[38;5;66;03m# In a synchronous context we are blocking the entire thread. Up to the library user to run the client in a\u001b[39;00m\n\u001b[1;32m   1048\u001b[0m \u001b[38;5;66;03m# different thread if necessary.\u001b[39;00m\n\u001b[1;32m   1049\u001b[0m time\u001b[38;5;241m.\u001b[39msleep(timeout)\n\u001b[0;32m-> 1051\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_request\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m   1052\u001b[0m \u001b[43m    \u001b[49m\u001b[43moptions\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43moptions\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1053\u001b[0m \u001b[43m    \u001b[49m\u001b[43mcast_to\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcast_to\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1054\u001b[0m \u001b[43m    \u001b[49m\u001b[43mremaining_retries\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mremaining\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1055\u001b[0m \u001b[43m    \u001b[49m\u001b[43mstream\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1056\u001b[0m \u001b[43m    \u001b[49m\u001b[43mstream_cls\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream_cls\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1057\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[0;32m~/opt/miniconda3/envs/env_mc_asr_312/lib/python3.12/site-packages/groq/_base_client.py:1018\u001b[0m, in \u001b[0;36mSyncAPIClient._request\u001b[0;34m(self, cast_to, options, remaining_retries, stream, stream_cls)\u001b[0m\n\u001b[1;32m   1015\u001b[0m         err\u001b[38;5;241m.\u001b[39mresponse\u001b[38;5;241m.\u001b[39mread()\n\u001b[1;32m   1017\u001b[0m     log\u001b[38;5;241m.\u001b[39mdebug(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mRe-raising status error\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m-> 1018\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_make_status_error_from_response(err\u001b[38;5;241m.\u001b[39mresponse) \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[1;32m   1020\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_process_response(\n\u001b[1;32m   1021\u001b[0m     cast_to\u001b[38;5;241m=\u001b[39mcast_to,\n\u001b[1;32m   1022\u001b[0m     options\u001b[38;5;241m=\u001b[39moptions,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m   1025\u001b[0m     stream_cls\u001b[38;5;241m=\u001b[39mstream_cls,\n\u001b[1;32m   1026\u001b[0m )\n",
      "\u001b[0;31mRateLimitError\u001b[0m: Error code: 429 - {'error': {'message': 'Rate limit reached for model `llama3-70b-8192` in organization `org_01hxzrsnf8e94sgaavqxytxpxt` on tokens per minute (TPM): Limit 6000, Used 0, Requested 6897. Please try again in 8.97s. Visit https://console.groq.com/docs/rate-limits for more information.', 'type': 'tokens', 'code': 'rate_limit_exceeded'}}"
     ]
    }
   ],
   "source": [
    "client = Groq(\n",
    "    api_key=os.environ.get(\"GROQ_API_KEY\"),\n",
    ")\n",
    "\n",
    "chat_completion = client.chat.completions.create(\n",
    "    messages=[\n",
    "        {\n",
    "            \"role\": \"user\",\n",
    "            \"content\": f\"Please summarize the following text in no less than 500 words:\\n\\n{transcript}\",\n",
    "        }\n",
    "    ],\n",
    "    model=\"llama3-70b-8192\",\n",
    ")\n",
    "\n",
    "print(chat_completion.choices[0].message.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b21b6f71-2fb0-4496-941c-a7d572e8c373",
   "metadata": {},
   "source": [
    "The easiest way to fix it is to use a model with a higher rate limit, such as llama3-groq-70b-8192-tool-use-preview, which has a limit of [15,000 tokens per minute](https://console.groq.com/settings/limits)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1caf5f43-b4a2-45ae-b161-50c93349b832",
   "metadata": {},
   "source": [
    "Create the instruction:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "a3013f61-0fa5-4547-a562-1b391c533248",
   "metadata": {},
   "outputs": [],
   "source": [
    "instruction = (\n",
    "    \"Please divide the following text into several logical parts. \"\n",
    "    \"The division should reflect a logical separation based on \"\n",
    "    \"topics, themes, or any natural breakpoints in the text. \"\n",
    "    \"Label each part with a brief heading that describes the focus or \"\n",
    "    \"content of that section. Summarize every part. The summary every \"\n",
    "    \"part has to be NOT less than 200 tokens.\\n\\n\"\n",
    "    \"Please provide the output in the following format:\\n\\n\"\n",
    "    \"Part 1: [Descriptive Heading]\\n\"\n",
    "    \"[The text from the first section]\\n\\n\"\n",
    "    \"Part 2: [Descriptive Heading]\\n\"\n",
    "    \"[The text from the second section]\\n\"\n",
    ")\n"
   ]
  },
  {
   "cell_type": "raw",
   "id": "829a4288-090e-4340-867e-e567cea69132",
   "metadata": {},
   "source": [
    "We send the request and get the result:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "6f8c4ffa-92fe-4526-b286-b02629632b2a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Part 1: Introduction to IBM Think 2023\n",
      "The text begins with an introduction to IBM Think 2023, highlighting the excitement and potential of AI-generated art and songs. It emphasizes the importance of building AI into the core of businesses and becoming an AI value creator rather than just an AI user. The introduction also highlights the importance of community and the creativity of the open AI community.\n",
      "\n",
      "Summary: The introduction to IBM Think 2023 emphasizes the potential of AI and the importance of becoming an AI value creator. It highlights the role of community and the creativity of the open AI community in driving innovation.\n",
      "\n",
      "Part 2: Watson X Platform\n",
      "The text then moves on to discuss Watson X, IBM's new integrated data and AI platform. Watson X consists of three primary parts: watsonx.data, watsonx.ai, and watsonx.governance. Watson X allows users to create and govern AI in real-time, and it is built on top of Red Hat OpenShift, providing seamless integration of Watson X components and allowing users to access and deploy AI workloads in any IT environment.\n",
      "\n",
      "Summary: Watson X is IBM's new integrated data and AI platform, consisting of three primary parts: watsonx.data, watsonx.ai, and watsonx.governance. It allows users to create and govern AI in real-time and is built on top of Red Hat OpenShift.\n",
      "\n",
      "Part 3: Partnership with Hugging Face\n",
      "The text then discusses the partnership between IBM and Hugging Face, a collaboration that brings the power of the open AI community into Watson X. Hugging Face has over 15,000 companies using the platform, 50,000 data sets, and over 100,000 open demos. The partnership aims to bring the best of both worlds, combining the cutting-edge and community of Hugging Face with the security and support of IBM.\n",
      "\n",
      "Summary: The partnership between IBM and Hugging Face brings the power of the open AI community into Watson X, combining the cutting-edge and community of Hugging Face with the security and support of IBM. This collaboration aims to enable businesses to become AI value creators.\n",
      "\n",
      "Part 4: Foundation Models and Governance\n",
      "The text then discusses the importance of foundation models and governance in AI. It emphasizes that one model will not rule them all and that businesses need to bet on community and the creativity of the open AI community. It also highlights the importance of running AI efficiently, optimizing for performance, latency, and cost by building with open hybrid technologies, and being responsible by building AI responsibly, transparently, and putting governance into the heart of the AI lifecycle.\n",
      "\n",
      "Summary: The text emphasizes the importance of foundation models and governance in AI, highlighting that one model will not rule them all and that businesses need to bet on community and the creativity of the open AI community. It also stresses the importance of running AI efficiently and being responsible in AI development.\n",
      "\n",
      "Part 5: Conclusion\n",
      "The text concludes by summarizing the key points of the presentation, including the importance of becoming an AI value creator, the capabilities of Watson X, the partnership with Hugging Face, and the importance of foundation models and governance. It encourages businesses to act with urgency and capture the moment, and to join IBM on this journey.\n",
      "\n",
      "Summary: The conclusion summarizes the key points of the presentation, emphasizing the importance of becoming an AI value creator, the capabilities of Watson X, the partnership with Hugging Face, and the importance of foundation models and governance. It encourages businesses to act with urgency and capture the moment, and to join IBM on this journey.\n",
      "\n",
      "A number of the completion tokens: 721 \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "client = Groq(\n",
    "    api_key=os.environ.get(\"GROQ_API_KEY\"),\n",
    ")\n",
    "\n",
    "chat_completion = client.chat.completions.create(\n",
    "    messages=[\n",
    "        {\n",
    "            \"role\": \"user\",\n",
    "            \"content\": f\"{instruction}\\n\\nThe text:\\n\\n{transcript}\",\n",
    "        }\n",
    "    ],\n",
    "    model=\"llama3-groq-70b-8192-tool-use-preview\",\n",
    "    temperature=0,\n",
    "    max_tokens=8192   \n",
    ")\n",
    "print(chat_completion.choices[0].message.content)\n",
    "print(f\"\\nA number of the completion tokens: {chat_completion.usage.completion_tokens} \\n\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab8015a1-1abd-4651-b647-bfcaf6ca8e67",
   "metadata": {},
   "source": [
    "In general, it is quite a good result. We see a clear explanation for every section. However, the problem is that it is insensitive to the number of tokens. We asked for no less than 200 tokens for every section but received less than half that amount. This means we can lose important information in our summary. Additionally, this approach (using of a model with a larger token rate limit) has limitations for larger transcriptions."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9c02311-b231-484f-a20c-e26940703b8c",
   "metadata": {},
   "source": [
    "## Alternative ways to summarize the transcript"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "714a65fb-c08b-4e89-ac58-951f530cfe4f",
   "metadata": {},
   "source": [
    "There are a few strategies to address the issue of creating insufficiently detailed summaries and the constraints of a low token limit:\n",
    "1. Chunk and Summarize (Map-Reduce Approach):\n",
    "    * Divide the original transcript into multiple smaller chunks.\n",
    "    * Summarize each chunk individually.\n",
    "    * Concatenate these individual summaries (optionally).\n",
    "    * Create a final summary from the concatenated summaries (optionally).      \n",
    "&nbsp;\n",
    "2. Incremental Summarization (Refine Approach):\n",
    "    * Split the original transcript into smaller chunks.\n",
    "    * Generate a summary for the first chunk.\n",
    "    * Add the next chunk to the previous summary and generate a new summary.\n",
    "    * Repeat this process until all chunks are summarized.   \n",
    "&nbsp;\n",
    "3. Utilize High-Capacity LLMs:\n",
    "    * Use a language model with a very high token limit (e.g., 128k tokens).\n",
    "    * This allows for processing larger portions of the transcript at once, enabling more comprehensive summaries.\n",
    "\n",
    "&nbsp;\n",
    "There is a good video related to this [topic](https://www.youtube.com/watch?v=f9_BWhCI4Zo)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a0ae133-df7e-40d8-b6dc-b384bbc2c65a",
   "metadata": {},
   "source": [
    "### Chunk and Summarize (Map-Reduce Approach)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f6632fa-24bb-46e1-8092-05fb1b9cb9c7",
   "metadata": {},
   "source": [
    "Renew the insruction:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "941d368d-0128-4a8c-bcea-75a991a4186f",
   "metadata": {},
   "outputs": [],
   "source": [
    "instruction = (\n",
    "    \"I give a chunk of a full text. Please summarize this. \"\n",
    "    \"You should divide the chunk into several logical parts. \"\n",
    "    \"The division should reflect a logical separation based on \"\n",
    "    \"topics, themes, or any natural breakpoints in the text. \"\n",
    "    \"Label each part with a brief heading that describes the focus or \"\n",
    "    \"content of that section. Summarize every part.\\n\\n\"\n",
    "    \"Please provide the output in the following format:\\n\\n\"\n",
    "    \"Part 1: [Descriptive Heading]\\n\"\n",
    "    \"[The text from the first section]\\n\\n\"\n",
    "    \"Part 2: [Descriptive Heading]\\n\"\n",
    "    \"[The text from the second section]\\n\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8bf76c2b-f275-4866-9d62-974c44124daf",
   "metadata": {},
   "source": [
    "Divide the original transcript into 4 smaller chunks using [mr_summary_by_groq.py](./utils/mr_summary_by_groq.py) :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "c1959dac-43c3-418c-ac09-52953158357c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Welcome to IBM Think 2023. AI-generated art. AI-generated songs. AI, what is that? It sure is a lot of fun. But when foundation models are applied to big business, well, you need to think bigger. Because AI in business needs to be held to a higher standard. Built to be trusted, secured, and adaptable. This isn't simple automation that is only trained to do one thing. This is AI that is built and focused to work across your organization. This isn't committing to a single system. This is hybrid-re...\n",
      "\n",
      "\n",
      "Once the model passes all the thresholds across the benchmarks, it is packaged and marked as ready for use. For each model, we create a model card that lists all the details of the model. We will have many different models trained on different piles with different target goals. Next, we go to WatsonX.Governance to combine the data card that has the detailed provenance information for the data pile that was used for training with the model card that has the detailed information on how the model w...\n",
      "\n",
      "\n",
      "But the truth is, if you look at how AI is built, when you can build smaller, more specialized customized models for your use cases, they end up being cheaper, they end up being more efficient, and they end up being better for your use case, right? Just the same way every single technology company learned how to write code, right, and to have a different code base than their competitors or than companies in other fields, we're seeing the same thing for AI, right? Every single company needs... Ev...\n",
      "\n",
      "\n",
      "Everything I've mentioned is useless unless you build responsibly, transparently, and put governance into the heart of your AI lifecycle. Continuously govern the data you use and the AI you deploy. And co-create with trusted partners. Trust is your ultimate license to operate. With your AI business strategy against these recommendations, you will be in a prime position to do amazing things with foundation models and generative AI. We have built WatsonX so that you can do just that. And I hope yo...\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from utils.mr_summary_by_groq import split_transcript\n",
    "num_chunks = 4\n",
    "chunks = split_transcript(transcript, num_chunks)\n",
    "for chunk in chunks:\n",
    "    print(f\"{chunk[:500]}...\\n\\n\") "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f009a1d0-e5f9-44e0-8822-37d2f64fa200",
   "metadata": {},
   "source": [
    "Summarize the chunks:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "4de8b018-f116-4d38-8801-8cb9764f3542",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Section #1\n",
      "\n",
      "Part 1: Introduction to IBM Think 2023\n",
      "Dr. Dario Gil, Senior Vice President and Director of Research at IBM, introduces the concept of Watson X, an integrated data and AI platform designed for value creators. Watson X consists of three primary parts: watsonx.data, watsonx.ai, and watsonx.governance. Watson X is built on top of Red Hat OpenShift, providing seamless integration and deployment of AI workloads in any IT environment.\n",
      "\n",
      "Part 2: Watson X Platform Overview\n",
      "Dr. Gil explains the Watson X platform's capabilities, including data preparation, model training, validation, and deployment. Watson X allows data scientists to access and manage data from various sources, filter and process it, and then use it to train models. The platform also provides tools for validation and deployment of the models, ensuring that they are ready for use in applications and solutions. Watson X is designed to work across hybrid cloud architectures and is built on top of Red Hat OpenShift, allowing for easy deployment in any IT environment.\n",
      "\n",
      "\n",
      "Section #2\n",
      "\n",
      "Part 1: Introduction to WatsonX\n",
      "The text introduces WatsonX, a new AI platform that allows users to create, train, and deploy AI models efficiently. WatsonX is designed to help users go from data to a deployed model that is trusted, governed, and ready to serve. The platform includes WatsonX.Data for data preparation, WatsonX.ai for model training, and WatsonX.Governance for model governance.\n",
      "\n",
      "Part 2: Model Training and Validation\n",
      "The text explains the process of training and validating models on WatsonX.ai. The process involves preparing the data, training the model, and validating it against benchmarks. Once the model passes all the thresholds, it is packaged and marked as ready for use. A model card is created that lists all the details of the model, and it is combined with a data card that has detailed provenance information for the data pile used for training. This forms a fact sheet that is cataloged in WatsonX.Governance.\n",
      "\n",
      "Part 3: Data and Model Factory\n",
      "The text discusses the concept of a data and model factory, which is a constant process of monitoring and updating models in WatsonX.Data and WatsonX.ai. This process ensures that the models are always up-to-date and compliant with any changes in regulations or data licensing. The data and model factory is in production today, with over 20 models trained across different modalities like language, code, geospatial, and chemistry.\n",
      "\n",
      "Part 4: Partnership with Hugging Face\n",
      "The text introduces Claym DeLong, co-founder and CEO of Hugging Face, and discusses the partnership between IBM and Hugging Face. Hugging Face is an open source AI community that has shared over 250,000 open models, 50,000 data sets, and over 100,000 open demos on their platform. The partnership aims to bring the creativity and diversity of the open AI community to WatsonX users.\n",
      "\n",
      "\n",
      "Section #3\n",
      "\n",
      "Part 1: Introduction\n",
      "The speaker discusses the importance of creating and governing AI in real time, highlighting the need for companies to build their own models and not outsource them. They emphasize the role of open source and open platforms in enabling this process.\n",
      "\n",
      "Part 2: WatsonX and Hugging Face Partnership\n",
      "The speaker talks about the WatsonX platform, which allows users to create and govern AI in real time. They also discuss the partnership with Hugging Face, which enables companies to build their own chat GPT models and differentiate themselves in the market.\n",
      "\n",
      "Part 3: WatsonX Features\n",
      "The speaker showcases the features of WatsonX, including the ability to prompt models, fine-tune models, and deploy and manage deployed models. They also highlight the use of the prompt lab for summarization tasks and the tuning studio for setting parameters and adding data.\n",
      "\n",
      "Part 4: IBM's AI Achievements\n",
      "The speaker discusses IBM's achievements in AI over the past year, including the launch of WatsonX, the creation of a family of foundation models, and collaborations with Hugging Face, PyTorch, and Ray. They also mention the IBM Vela and IBM Research AIU systems designed for enterprise AI inference and tuning.\n",
      "\n",
      "Part 5: Tips for Harnessing Foundation Models\n",
      "The speaker provides tips for harnessing foundation models, including acting with urgency, being a value creator, betting on community, running everywhere efficiently, and being responsible. They emphasize the importance of building foundation models on one's own data and under one's control.\n",
      "\n",
      "\n",
      "Section #4\n",
      "\n",
      "Part 1: Importance of Responsible AI Development\n",
      "Everything I've mentioned is useless unless you build responsibly, transparently, and put governance into the heart of your AI lifecycle.\n",
      "\n",
      "Part 2: Continuous Governance and Co-Creation\n",
      "Continuously govern the data you use and the AI you deploy. And co-create with trusted partners. Trust is your ultimate license to operate.\n",
      "\n",
      "Part 3: AI Business Strategy and WatsonX\n",
      "With your AI business strategy against these recommendations, you will be in a prime position to do amazing things with foundation models and generative AI. We have built WatsonX so that you can do just that.\n",
      "\n",
      "\n",
      "\n",
      "A number of the completion tokens: 1036 \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from utils.mr_summary_by_groq import execute_tasks\n",
    "responses = await execute_tasks(instruction, chunks)\n",
    "token_num = 0\n",
    "# Print each response message and a number of tokens of all\n",
    "# response messages.\n",
    "for num, response in enumerate(responses, start=1):\n",
    "    print(f\"Section #{num}\\n\\n{response.choices[0].message.content}\\n\\n\")\n",
    "    token_num += response.usage.completion_tokens\n",
    "print(f\"\\nA number of the completion tokens: {token_num} \\n\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5fc64d46-45b1-4bac-b2b6-3ce6f8b7fccc",
   "metadata": {},
   "source": [
    "We can see that this approach works, providing ~30-50% more tokens and a more detailed summary."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6d9d696-bf75-4b78-adf3-fd7b69096521",
   "metadata": {},
   "source": [
    "### Incremental Summarization (Refine Approach)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3fc5096e-9d80-4ed2-89df-606c00f5a23f",
   "metadata": {},
   "source": [
    "Split the original transcript into smaller chunks.\n",
    "Generate a summary for the first chunk.\n",
    "Add the next chunk to the previous summary and generate a new summary.\n",
    "Repeat this process until all chunks are summarized."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "aeb67b9b-1b5a-4e02-b8b1-f7e71f9fcf07",
   "metadata": {},
   "outputs": [],
   "source": [
    "instruction = (\n",
    "    \"I give a chunk of a full text. Please summarize this. \"\n",
    "    \"You should divide the chunk into several logical parts. \"\n",
    "    \"The division should reflect a logical separation based on \"\n",
    "    \"topics, themes, or any natural breakpoints in the text. \"\n",
    "    \"Label each part with a brief heading that describes the focus or \"\n",
    "    \"content of that section. Summarize every part.\\n\\n\"\n",
    "    \"Please provide the output in the following format:\\n\\n\"\n",
    "    \"Part 1: [Descriptive Heading]\\n\"\n",
    "    \"[The text from the first section]\\n\\n\"\n",
    "    \"Part 2: [Descriptive Heading]\\n\"\n",
    "    \"[The text from the second section]\\n\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20958f81-d966-4330-a3c9-cf751170bcee",
   "metadata": {},
   "source": [
    "Divide the original transcript into 4 smaller chunks using [ref_summary_by_groq.py](./utils/ref_summary_by_groq.py) :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "38f271f1-5e92-4f7f-ace7-9cfddb81d8f0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Welcome to IBM Think 2023. AI-generated art. AI-generated songs. AI, what is that? It sure is a lot of fun. But when foundation models are applied to big business, well, you need to think bigger. Because AI in business needs to be held to a higher standard. Built to be trusted, secured, and adaptable. This isn't simple automation that is only trained to do one thing. This is AI that is built and focused to work across your organization. This isn't committing to a single system. This is hybrid-re...\n",
      "\n",
      "\n",
      "Once the model passes all the thresholds across the benchmarks, it is packaged and marked as ready for use. For each model, we create a model card that lists all the details of the model. We will have many different models trained on different piles with different target goals. Next, we go to WatsonX.Governance to combine the data card that has the detailed provenance information for the data pile that was used for training with the model card that has the detailed information on how the model w...\n",
      "\n",
      "\n",
      "But the truth is, if you look at how AI is built, when you can build smaller, more specialized customized models for your use cases, they end up being cheaper, they end up being more efficient, and they end up being better for your use case, right? Just the same way every single technology company learned how to write code, right, and to have a different code base than their competitors or than companies in other fields, we're seeing the same thing for AI, right? Every single company needs... Ev...\n",
      "\n",
      "\n",
      "Everything I've mentioned is useless unless you build responsibly, transparently, and put governance into the heart of your AI lifecycle. Continuously govern the data you use and the AI you deploy. And co-create with trusted partners. Trust is your ultimate license to operate. With your AI business strategy against these recommendations, you will be in a prime position to do amazing things with foundation models and generative AI. We have built WatsonX so that you can do just that. And I hope yo...\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from utils.ref_summary_by_groq import split_transcript\n",
    "num_chunks = 4\n",
    "chunks = split_transcript(transcript, num_chunks)\n",
    "for chunk in chunks:\n",
    "    print(f\"{chunk[:500]}...\\n\\n\") "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ce8f181-98da-46da-9450-9ee242e0c2db",
   "metadata": {},
   "source": [
    "Summarize the chunks:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "0dd47868-f25f-4be2-b1e0-1254c9ccc864",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Final Summary:\n",
      "Part 1: Introduction to IBM Think 2023\n",
      "Dr. Dario Gil introduces Watson X, an integrated data and AI platform designed for value creators. Watson X consists of three primary parts: watsonx.data, watsonx.ai, and watsonx.governance. Watson X is built on top of Red Hat OpenShift, providing seamless integration and deployment of AI workloads in any IT environment.\n",
      "\n",
      "Part 2: Watson X Platform Overview\n",
      "Dr. Gil explains the Watson X platform's capabilities, including data preparation, model training, validation, and deployment. Watson X allows data scientists to access and manage data from various sources, filter and process it, and then use it to train models. The platform also provides tools for validation and deployment of the models, ensuring that they are ready for use in applications and solutions. Watson X is designed to work across hybrid cloud architectures and is built on top of Red Hat OpenShift, allowing for easy deployment in any IT environment.\n",
      "\n",
      "Part 3: Data and Model Factory\n",
      "Dr. Gil discusses the concept of the data and model factory, which continuously monitors and updates models in both .data and .ai. This process is central to solid and proper governance of AI. Watson X has already produced over 20 foundation models across modalities like language, code, geospatial, and chemistry, and spanning different sizes of models -- from hundreds of millions to billions of parameters. These models have been infused into IBM products, Red Hat products, and/or partners products.\n",
      "\n",
      "Part 4: Partnership with Hugging Face\n",
      "Dr. Gil announces a new partnership with Hugging Face, a company that specializes in open source AI. Hugging Face co-founder and CEO, Claym DeLong, shares that since the release of ChatGPT, 100,000 new models have been added on HuggingFace. Companies don't train models just to train models, as it costs money to train models. But the truth is, if you look at how AI is built, when you can build smaller, more specialized customized models for your use cases, they end up being cheaper, they end up being more efficient, and they end up being better for your use case, right? Just the same way every single technology company learned how to write code, and to have a different code base than their competitors or than companies in other fields, we're seeing the same thing for AI, right? Every single company needs... Every single company needs to train their own models, optimize their own models, learn how to run these models at scale. Every single company needs to build their own chat GPT because if they don't, they won't be able to differentiate. They won't be able to create the unique technology value that they've been building for their customers and they'll lose control, right, if they start outsourcing it. So that's what we're seeing on Hugin. Face and in the ecosystem as a whole. It's back to this philosophy of don't just be a prompt tuner user, right, be a value creator with all of this.\n",
      "\n",
      "Part 5: IBM's Achievements and Future Plans\n",
      "Dr. Gil summarizes the achievements of IBM in the past year, including the announcement of WatsonX, a comprehensive platform that allows you to create and govern AI in real time. IBM has also announced a set of foundation models, a data model factory, and important collaborations to advance AI and bring it to the enterprise. They have also launched the IBM Vela or cloud native AI supercomputer to train foundation models with bare metal performance while giving us the flexibility of the cloud. And they announced that they're making it available as a service. Last year, they launched the Telum in C 16. It's an engineering marvel, and IBM's first processor to have on-chip accelerator for AI inferencing. It can process 300 billion inferences per day with one millisecond latencies. This means that now you can infuse AI into every transaction in C 16 for applications like fraud detection and others in real time. Using the same core architecture as Telum, they built the IBM Research AIU. Which is optimized to give superior performance for foundation models and enable with the Red Hat software stack. And at IBM Research, they are incubating powerful AIU systems designed and optimized for enterprise AI inference and tuning. So a truly fantastic year. And this is just the start of all the amazing things that they're building and developing for you and that they will be sharing with you in the coming years.\n",
      "\n",
      "Part 6: Importance of Responsible AI\n",
      "Dr. Gil emphasizes the importance of building responsibly, transparently, and putting governance into the heart of your AI lifecycle. Continuously govern the data you use and the AI you deploy. And co-create with trusted partners. Trust is your ultimate license to operate. With your AI business strategy against these recommendations, you will be in a prime position to do amazing things with foundation models and generative AI. They have built WatsonX so that you can do just that. And I hope you join us, because we cannot wait to get started on this journey with you. Thank you.\n",
      "\n",
      "\n",
      "\n",
      "A number of the completion tokens: 1043\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from utils.ref_summary_by_groq import refine_summary\n",
    "# Generate the refined summary using the refine_summary function.\n",
    "final_summary, token_num = refine_summary(instruction, chunks)\n",
    "# Output the final summary\n",
    "print(f\"\\nFinal Summary:\\n{final_summary}\\n\\n\")\n",
    "print(f\"\\nA number of the completion tokens: {token_num}\\n\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f44a6421-dd18-47d6-8348-faedb4041643",
   "metadata": {},
   "source": [
    "We can see that this approach works. It provides approximately 30-50% more tokens than the connection of separate chunk summaries. It has nearly the same number of tokens as the Map-Reduce approach, and the refined summary is written in a smoother manner. The disadvantage is that it takes more time because we need to make the API requests to Groq consecutively, not concurrently."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb456eb7-2dc0-4edb-af53-69a78dd5d4d8",
   "metadata": {},
   "source": [
    "### Utilize High-Capacity LLMs:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b9a7d34-fb1b-41b6-a759-6beed6c1825b",
   "metadata": {},
   "source": [
    "#### The naive approach (copy and paste the transcript into ChatGPT-4o)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db537689-b901-462f-8ccd-afb30935317c",
   "metadata": {},
   "source": [
    "We send the following request:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f4ef4bf-7c0e-42a9-8689-d79deb5e941b",
   "metadata": {},
   "source": [
    "```text\n",
    "There is a transcript:\n",
    "{HERE OUR TRANSCRIPT}\n",
    "\n",
    "Please, do the following:\n",
    "\n",
    "Please divide this transcript into several logical parts. The division should reflect a logical separation based on topics, themes, the speakers name, or any natural breakpoints in the text. Label each part with a brief heading that describes the focus or content of that section. Summarize every part. The summary every part has to be NOT less than 200 tokens.\n",
    "\n",
    "Please provide the output in the following format:\n",
    "\n",
    "    Part 1: [Descriptive Heading]    \n",
    "    [The summary from the first section]\n",
    "    \n",
    "    Part 2: [Descriptive Heading]\n",
    "    [The summary from the second section]\n",
    "\n",
    "    Etc.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84dd0c11-01d0-44af-ba2d-0a90505d07b5",
   "metadata": {},
   "source": [
    "We get the response:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f710b94-41f3-424c-96f7-c9dedeb2bcc2",
   "metadata": {},
   "source": [
    "```markdown\n",
    "### Part 1: **Introduction and Importance of AI in Business**\n",
    "\n",
    "The event begins with a welcome to IBM Think 2023 and sets the stage by discussing the fun aspects of AI, such as AI-generated art and songs. However, the speaker quickly shifts focus to the more serious applications of AI in business, emphasizing the need for AI to meet higher standards of trust, security, and adaptability. The speaker highlights that AI in business is not about simple automation but about creating AI systems that can work across an organization, scale across systems, and provide transparent results. This introduction underscores the transformative potential of AI when integrated into the core of business operations, setting the tone for the discussions to follow.\n",
    "\n",
    "### Part 2: **The Transformative Impact of AI Across Industries**\n",
    "\n",
    "Dr. Dario Gil, Senior Vice President and Director of Research at IBM, takes the stage and reflects on the rapid advancements in AI over the past year. He expresses excitement about the possibilities AI will bring, noting its impact across various industries such as customer care, logistics, medicine, manufacturing, and energy. Dr. Gil acknowledges that while the pace of technological change can be daunting, the opportunities presented by foundation models and generative AI, when governed correctly, are immense. He advises businesses not to be mere users of AI but to become creators of AI value, emphasizing the importance of owning and controlling AI models and data to maximize their potential.\n",
    "\n",
    "### Part 3: **Introduction to WatsonX and its Components**\n",
    "\n",
    "Dr. Gil introduces WatsonX, IBM’s new integrated data and AI platform designed to empower businesses to become AI value creators. WatsonX is composed of three main parts: watsonx.data, a massive curated data repository; watsonx.ai, an enterprise studio for training, validating, and deploying AI models; and watsonx.governance, a set of tools to ensure responsible AI execution. He explains how these components work together seamlessly across the AI lifecycle, from data preparation and model training to validation, tuning, and deployment. Dr. Gil emphasizes that WatsonX is built on Red Hat OpenShift, enabling hybrid cloud integration and deployment of AI workloads across any IT environment, making it a versatile platform for enterprise AI.\n",
    "\n",
    "### Part 4: **Detailed Workflow of AI Model Creation with WatsonX**\n",
    "\n",
    "This section delves into the detailed workflow of creating AI models using WatsonX. Dr. Gil walks the audience through the process, starting with data preparation, where data scientists can access and filter data from various sources using watsonx.data. He explains the importance of curating data into \"piles\" based on different criteria, such as language or programming language, and filtering out irrelevant or harmful content. The workflow then moves to model training using watsonx.ai, where models are trained on IBM’s Vela AI supercomputer. Dr. Gil describes the process of tokenization, model validation, and the creation of model cards that document the training process. The section concludes with a discussion on tuning models for specific tasks and deploying them across different environments.\n",
    "\n",
    "### Part 5: **WatsonX in Action and Its Value to Enterprises**\n",
    "\n",
    "Dr. Gil showcases how WatsonX enables enterprises to deploy AI models effectively. He describes how businesses can use WatsonX to create and manage foundation models tailored to their specific needs, whether for customer care, scientific research, or industry-specific applications. He highlights several real-world examples, such as SAP using IBM Watson for digital assistants, Moderna applying foundation models for mRNA medicine prediction, and NASA using language models for climate research. This section underscores the practical applications of WatsonX across various industries and the tangible value it brings to businesses.\n",
    "\n",
    "### Part 6: **Partnership with Hugging Face and the Power of Open AI Community**\n",
    "\n",
    "The discussion shifts to IBM's partnership with Hugging Face, a leader in the open AI community. Dr. Gil invites Clem Delangue, co-founder and CEO of Hugging Face, to the stage. Clem shares the origins of Hugging Face and its evolution from a conversational AI company to a platform that hosts over 250,000 open models. He highlights the vibrant and rapidly growing open-source AI community, noting that 4,000 new models were added to Hugging Face in a single week. The partnership between IBM and Hugging Face aims to bring the creativity and diversity of the open AI community into the WatsonX platform, offering enterprises the best of both worlds: cutting-edge AI models from the community and the security and support of IBM.\n",
    "\n",
    "### Part 7: **Future Outlook and Closing Remarks**\n",
    "\n",
    "Dr. Gil concludes the event by reflecting on the significant advancements IBM has made in AI over the past year. He recaps the major announcements, including the launch of WatsonX, the creation of foundation models, and the development of IBM Vela, the AI supercomputer. He stresses the importance of having a business strategy for AI and offers key advice: act with urgency, be a value creator, bet on community, run AI efficiently across hybrid environments, and build responsibly with governance at the core. Dr. Gil’s closing remarks are a call to action for businesses to embrace AI fully, leveraging WatsonX to create and govern AI models that drive value and innovation.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3bd6293-0662-4998-9f8f-481a8274a63e",
   "metadata": {},
   "source": [
    "As we can see, the ChatGPT 4o summary is more detailed, especially at the beginning, and it generally has a smoother manner. However, the Incremental Summarization is more detailed at the end and focuses on the general idea. Both variants can be used depending on the goal."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f6eef01-1463-4e16-8d3c-e80ef21e3a0c",
   "metadata": {},
   "source": [
    "| #   | Incremental Summarization (Llama 3)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        | ChatGPT 4o                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |\n",
    "| --- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n",
    "| 1   | Part 1: Introduction to IBM Think 2023<br>Dr. Dario Gil introduces Watson X, an integrated data and AI platform designed for value creators. Watson X consists of three primary parts: watsonx.data, watsonx.ai, and watsonx.governance. Watson X is built on top of Red Hat OpenShift, providing seamless integration and deployment of AI workloads in any IT environment.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | Part 1: Introduction and Importance of AI in Business  <br>  <br>The event begins with a welcome to IBM Think 2023 and sets the stage by discussing the fun aspects of AI, such as AI-generated art and songs. However, the speaker quickly shifts focus to the more serious applications of AI in business, emphasizing the need for AI to meet higher standards of trust, security, and adaptability. The speaker highlights that AI in business is not about simple automation but about creating AI systems that can work across an organization, scale across systems, and provide transparent results. This introduction underscores the transformative potential of AI when integrated into the core of business operations, setting the tone for the discussions to follow.  <br>  <br>Part 2: The Transformative Impact of AI Across Industries  <br>  <br>Dr. Dario Gil, Senior Vice President and Director of Research at IBM, takes the stage and reflects on the rapid advancements in AI over the past year. He expresses excitement about the possibilities AI will bring, noting its impact across various industries such as customer care, logistics, medicine, manufacturing, and energy. Dr. Gil acknowledges that while the pace of technological change can be daunting, the opportunities presented by foundation models and generative AI, when governed correctly, are immense. He advises businesses not to be mere users of AI but to become creators of AI value, emphasizing the importance of owning and controlling AI models and data to maximize their potential.                                                   |\n",
    "| 2   | Part 2: Watson X Platform Overview<br>Dr. Gil explains the Watson X platform's capabilities, including data preparation, model training, validation, and deployment. Watson X allows data scientists to access and manage data from various sources, filter and process it, and then use it to train models. The platform also provides tools for validation and deployment of the models, ensuring that they are ready for use in applications and solutions. Watson X is designed to work across hybrid cloud architectures and is built on top of Red Hat OpenShift, allowing for easy deployment in any IT environment.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                | Part 3: Introduction to WatsonX and its Components  <br>   <br>Dr. Gil introduces WatsonX, IBM’s new integrated data and AI platform designed to empower businesses to become AI value creators. WatsonX is composed of three main parts: watsonx.data, a massive curated data repository; watsonx.ai, an enterprise studio for training, validating, and deploying AI models; and watsonx.governance, a set of tools to ensure responsible AI execution. He explains how these components work together seamlessly across the AI lifecycle, from data preparation and model training to validation, tuning, and deployment. Dr. Gil emphasizes that WatsonX is built on Red Hat OpenShift, enabling hybrid cloud integration and deployment of AI workloads across any IT environment, making it a versatile platform for enterprise AI.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |\n",
    "| 3   | Part 3: Data and Model Factory<br>Dr. Gil discusses the concept of the data and model factory, which continuously monitors and updates models in both .data and .ai. This process is central to solid and proper governance of AI. Watson X has already produced over 20 foundation models across modalities like language, code, geospatial, and chemistry, and spanning different sizes of models -- from hundreds of millions to billions of parameters. These models have been infused into IBM products, Red Hat products, and/or partners products.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  | Part 4: Detailed Workflow of AI Model Creation with WatsonX  <br>  <br>This section delves into the detailed workflow of creating AI models using WatsonX. Dr. Gil walks the audience through the process, starting with data preparation, where data scientists can access and filter data from various sources using watsonx.data. He explains the importance of curating data into \"piles\" based on different criteria, such as language or programming language, and filtering out irrelevant or harmful content. The workflow then moves to model training using watsonx.ai, where models are trained on IBM’s Vela AI supercomputer. Dr. Gil describes the process of tokenization, model validation, and the creation of model cards that document the training process. The section concludes with a discussion on tuning models for specific tasks and deploying them across different environments.  <br>  <br>Part 5: WatsonX in Action and Its Value to Enterprises  <br>  <br>Dr. Gil showcases how WatsonX enables enterprises to deploy AI models effectively. He describes how businesses can use WatsonX to create and manage foundation models tailored to their specific needs, whether for customer care, scientific research, or industry-specific applications. He highlights several real-world examples, such as SAP using IBM Watson for digital assistants, Moderna applying foundation models for mRNA medicine prediction, and NASA using language models for climate research. This section underscores the practical applications of WatsonX across various industries and the tangible value it brings to businesses. |\n",
    "| 4   | Part 4: Partnership with Hugging Face<br>Dr. Gil announces a new partnership with Hugging Face, a company that specializes in open source AI. Hugging Face co-founder and CEO, Claym DeLong, shares that since the release of ChatGPT, 100,000 new models have been added on HuggingFace. Companies don't train models just to train models, as it costs money to train models. But the truth is, if you look at how AI is built, when you can build smaller, more specialized customized models for your use cases, they end up being cheaper, they end up being more efficient, and they end up being better for your use case, right? Just the same way every single technology company learned how to write code, and to have a different code base than their competitors or than companies in other fields, we're seeing the same thing for AI, right? Every single company needs... Every single company needs to train their own models, optimize their own models, learn how to run these models at scale. Every single company needs to build their own chat GPT because if they don't, they won't be able to differentiate. They won't be able to create the unique technology value that they've been building for their customers and they'll lose control, right, if they start outsourcing it. So that's what we're seeing on Hugin. Face and in the ecosystem as a whole. It's back to this philosophy of don't just be a prompt tuner user, right, be a value creator with all of this.                   | Part 6: Partnership with Hugging Face and the Power of Open AI Community  <br>  <br>The discussion shifts to IBM's partnership with Hugging Face, a leader in the open AI community. Dr. Gil invites Clem Delangue, co-founder and CEO of Hugging Face, to the stage. Clem shares the origins of Hugging Face and its evolution from a conversational AI company to a platform that hosts over 250,000 open models. He highlights the vibrant and rapidly growing open-source AI community, noting that 4,000 new models were added to Hugging Face in a single week. The partnership between IBM and Hugging Face aims to bring the creativity and diversity of the open AI community into the WatsonX platform, offering enterprises the best of both worlds: cutting-edge AI models from the community and the security and support of IBM.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |\n",
    "| 5   | Part 5: IBM's Achievements and Future Plans<br>Dr. Gil summarizes the achievements of IBM in the past year, including the announcement of WatsonX, a comprehensive platform that allows you to create and govern AI in real time. IBM has also announced a set of foundation models, a data model factory, and important collaborations to advance AI and bring it to the enterprise. They have also launched the IBM Vela or cloud native AI supercomputer to train foundation models with bare metal performance while giving us the flexibility of the cloud. And they announced that they're making it available as a service. Last year, they launched the Telum in C 16. It's an engineering marvel, and IBM's first processor to have on-chip accelerator for AI inferencing. It can process 300 billion inferences per day with one millisecond latencies. This means that now you can infuse AI into every transaction in C 16 for applications like fraud detection and others in real time. Using the same core architecture as Telum, they built the IBM Research AIU. Which is optimized to give superior performance for foundation models and enable with the Red Hat software stack. And at IBM Research, they are incubating powerful AIU systems designed and optimized for enterprise AI inference and tuning. So a truly fantastic year. And this is just the start of all the amazing things that they're building and developing for you and that they will be sharing with you in the coming years. | Part 7: Future Outlook and Closing Remarks  <br>  <br>Dr. Gil concludes the event by reflecting on the significant advancements IBM has made in AI over the past year. He recaps the major announcements, including the launch of WatsonX, the creation of foundation models, and the development of IBM Vela, the AI supercomputer. He stresses the importance of having a business strategy for AI and offers key advice: act with urgency, be a value creator, bet on community, run AI efficiently across hybrid environments, and build responsibly with governance at the core. Dr. Gil’s closing remarks are a call to action for businesses to embrace AI fully, leveraging WatsonX to create and govern AI models that drive value and innovation.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |\n",
    "| 6   | Part 6: Importance of Responsible AI<br>Dr. Gil emphasizes the importance of building responsibly, transparently, and putting governance into the heart of your AI lifecycle. Continuously govern the data you use and the AI you deploy. And co-create with trusted partners. Trust is your ultimate license to operate. With your AI business strategy against these recommendations, you will be in a prime position to do amazing things with foundation models and generative AI. They have built WatsonX so that you can do just that. And I hope you join us, because we cannot wait to get started on this journey with you. Thank you.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6aa01989-10d4-4301-a9e1-a8de31bcbfca",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.12 (env_mc_asr_312)",
   "language": "python",
   "name": "env_mc_tf"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}