L1_Router_Engine.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "dd7c4c87",
   "metadata": {},
   "source": [
    "# Lesson 1: Router Engine"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "877df988",
   "metadata": {},
   "source": [
    "Welcome to Lesson 1.\n",
    "\n",
    "To access the `requirements.txt` file, the data/pdf file required for this lesson and the `helper` and `utils` modules, please go to the `File` menu and select`Open...`.\n",
    "\n",
    "I hope you enjoy this course!\n",
    "\n",
    "A `router` is the simplest form of agentic RAG. Given a query, the router will choose the suitable tool among many and return the response."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "357c97c9",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "ab5f4f4a-5890-451c-8869-24606ef9f396",
   "metadata": {
    "height": 64,
    "tags": []
   },
   "outputs": [],
   "source": [
    "from helper import get_openai_api_key\n",
    "\n",
    "OPENAI_API_KEY = get_openai_api_key()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "593971e1",
   "metadata": {},
   "source": [
    "By design asyncio does not allow its event loop to be nested in environments like Jupyter notebooks. This module patches asyncio to allow nested use."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ffc9b4f4-64d4-4266-9889-54db90e00ee9",
   "metadata": {
    "height": 64,
    "tags": []
   },
   "outputs": [],
   "source": [
    "import nest_asyncio\n",
    "\n",
    "nest_asyncio.apply()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "49fca250",
   "metadata": {},
   "source": [
    "## Load Data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3ae2a8c",
   "metadata": {
    "tags": []
   },
   "source": [
    "To download this paper, below is the needed code:\n",
    "\n",
    "#!wget \"https://openreview.net/pdf?id=VtmBAGCN7o\" -O metagpt.pdf\n",
    "\n",
    "**Note**: The pdf file is included with this lesson. To access it, go to the `File` menu and select`Open...`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "5c7f012d-dcd3-4881-a568-72dd27d79159",
   "metadata": {
    "height": 96,
    "tags": []
   },
   "outputs": [],
   "source": [
    "from llama_index.core import SimpleDirectoryReader\n",
    "\n",
    "# load documents\n",
    "documents = SimpleDirectoryReader(input_files=[\"metagpt.pdf\"]).load_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b48a301",
   "metadata": {},
   "source": [
    "## Define LLM and Embedding model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "8a537bc0-78ee-4dda-a43f-60fd80062df6",
   "metadata": {
    "height": 96,
    "tags": []
   },
   "outputs": [],
   "source": [
    "from llama_index.core.node_parser import SentenceSplitter\n",
    "\n",
    "splitter = SentenceSplitter(chunk_size=1024)\n",
    "nodes = splitter.get_nodes_from_documents(documents)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06e64869",
   "metadata": {},
   "source": [
    "### Optional: Define settings to find LLM and embedding model "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "de0660ee-b231-4351-b158-d8ad023e00b5",
   "metadata": {
    "height": 130,
    "tags": []
   },
   "outputs": [],
   "source": [
    "from llama_index.core import Settings\n",
    "from llama_index.llms.openai import OpenAI\n",
    "from llama_index.embeddings.openai import OpenAIEmbedding\n",
    "\n",
    "Settings.llm = OpenAI(model=\"gpt-3.5-turbo\")\n",
    "Settings.embed_model = OpenAIEmbedding(model=\"text-embedding-ada-002\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "997c7559",
   "metadata": {},
   "source": [
    "## Define Summary Index and Vector Index over the Same Data\n",
    "\n",
    "<div><img src=\"images/lesson1-1.png\" style=\"width: 50%\"/></div><div><img src=\"images/lesson1-2.png\" style=\"width: 50%\"/></div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "73d01b01-bc74-432a-8d92-07b9e86498b0",
   "metadata": {
    "height": 96,
    "tags": []
   },
   "outputs": [],
   "source": [
    "from llama_index.core import SummaryIndex, VectorStoreIndex\n",
    "\n",
    "summary_index = SummaryIndex(nodes)\n",
    "vector_index = VectorStoreIndex(nodes)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f9898d3f",
   "metadata": {},
   "source": [
    "## Define Query Engines and Set Metadata"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "44cd7046-c714-4920-b077-b3ded917862f",
   "metadata": {
    "height": 113,
    "tags": []
   },
   "outputs": [],
   "source": [
    "summary_query_engine = summary_index.as_query_engine(\n",
    "    response_mode=\"tree_summarize\",\n",
    "    use_async=True,\n",
    ")\n",
    "vector_query_engine = vector_index.as_query_engine()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "6a1d6d75-247e-426a-8ef4-b49225c24796",
   "metadata": {
    "height": 300,
    "tags": []
   },
   "outputs": [],
   "source": [
    "from llama_index.core.tools import QueryEngineTool\n",
    "\n",
    "\n",
    "summary_tool = QueryEngineTool.from_defaults(\n",
    "    query_engine=summary_query_engine,\n",
    "    description=(\n",
    "        \"Useful for summarization questions related to MetaGPT\"\n",
    "    ),\n",
    ")\n",
    "\n",
    "vector_tool = QueryEngineTool.from_defaults(\n",
    "    query_engine=vector_query_engine,\n",
    "    description=(\n",
    "        \"Useful for retrieving specific context from the MetaGPT paper.\"\n",
    "    ),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98d2c152",
   "metadata": {},
   "source": [
    "## Define Router Query Engine\n",
    "\n",
    "There are several selectors available:\n",
    "\n",
    "- `LLM selectors` use LLM to output a JSON that is parsed, and the corresponding indexes are queried.\n",
    "- `Pydantic selectors` use the OpenAI Function Calling API to produce pydantic selection objects, rather than parsing raw JSON."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "00734d7c-638a-4d63-ab1f-7f5a92a65119",
   "metadata": {
    "height": 232,
    "tags": []
   },
   "outputs": [],
   "source": [
    "from llama_index.core.query_engine.router_query_engine import RouterQueryEngine\n",
    "from llama_index.core.selectors import LLMSingleSelector\n",
    "\n",
    "\n",
    "query_engine = RouterQueryEngine(\n",
    "    selector=LLMSingleSelector.from_defaults(),\n",
    "    query_engine_tools=[\n",
    "        summary_tool,\n",
    "        vector_tool,\n",
    "    ],\n",
    "    verbose=True\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "fe3f0a76-68a8-444d-867f-d084bb3ff112",
   "metadata": {
    "height": 62,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;3;38;5;200mSelecting query engine 0: This choice indicates that the document is useful for summarization questions related to MetaGPT, suggesting that it provides a condensed version or overview of the content..\n",
      "\u001b[0mThe document introduces MetaGPT, a meta-programming framework for LLM-based multi-agent collaboration that enhances problem-solving processes among agents through SOPs, role specialization, structured communication interfaces, and executable feedback mechanisms. It demonstrates superior performance in code generation benchmarks, validates its effectiveness as a meta-programming framework, and sets the stage for future research on human-inspired techniques for artificial multi-agent systems. Additionally, it emphasizes active teamwork, recursive self-improvement mechanisms, learning from past project experiences, and the use of Reinforcement Learning frameworks for societies of agents. The document also details the software development process using MetaGPT for a \"Drawing App,\" discusses the performance of GPT models, addresses limitations and ethical concerns, and highlights transparency, accountability, privacy, and data security aspects of MetaGPT.\n"
     ]
    }
   ],
   "source": [
    "response = query_engine.query(\"What is the summary of the document?\")\n",
    "print(str(response))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "d3fedea0-f2a9-46bb-8aaf-287df65b8fff",
   "metadata": {
    "height": 30,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "34\n"
     ]
    }
   ],
   "source": [
    "print(len(response.source_nodes))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "af8c31b3-8e22-4ad9-9825-b8de21bd03c0",
   "metadata": {
    "height": 96,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;3;38;5;200mSelecting query engine 1: This choice is more relevant as it focuses on retrieving specific context from the MetaGPT paper, which may contain information on how agents share information with other agents..\n",
      "\u001b[0mAgents share information with other agents by utilizing a shared message pool where they can publish structured messages. This shared message pool allows all agents to exchange messages directly, enabling them to both publish their own messages and access messages from other entities transparently. Agents can retrieve required information directly from this shared pool, eliminating the need to inquire about other agents and wait for their responses, thus enhancing communication efficiency.\n"
     ]
    }
   ],
   "source": [
    "response = query_engine.query(\n",
    "    \"How do agents share information with other agents?\"\n",
    ")\n",
    "print(str(response))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aed060ee",
   "metadata": {},
   "source": [
    "## Let's put everything together"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "d8f92e0b-1c54-489b-b8dd-41ebaafb380a",
   "metadata": {
    "height": 79
   },
   "outputs": [],
   "source": [
    "from utils import get_router_query_engine\n",
    "\n",
    "query_engine = get_router_query_engine(\"metagpt.pdf\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "ec1a43f3-77dc-472a-8adc-56551c00a0ff",
   "metadata": {
    "height": 62,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;3;38;5;200mSelecting query engine 1: The ablation study results are specific context from the MetaGPT paper, making choice 2 the most relevant..\n",
      "\u001b[0mThe ablation study results provide insights into the impact of different components or features on the overall performance of a system or model. By systematically removing or disabling specific parts and observing how it affects the system's performance, researchers can understand the individual contributions of each element. This analysis helps in identifying the most critical components and optimizing the system for better efficiency and effectiveness.\n"
     ]
    }
   ],
   "source": [
    "response = query_engine.query(\"Tell me about the ablation study results?\")\n",
    "print(str(response))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}