notebooks/03-rag_from_scratch.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Build a RAG Chatbot from scratch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import importlib\n",
    "import os\n",
    "\n",
    "if not importlib.util.find_spec(\"class_utils\"):\n",
    "    !pip install -qqq git+https://github.com/xtreamsrl/genai-for-engineers-class\n",
    "\n",
    "os.environ[\"TOKENIZERS_PARALLELISM\"] = \"true\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Retrieval-Augmented Generation (RAG) is a method that combines a vector database and generative models for tasks such as open-domain question answering, dialogue systems, and other text generation tasks. The retrieval step consists of finding relevant information from a large corpus, while generative models create new responses based on the input. RAG enhances the performance by first retrieving relevant documents and then generating responses based on the retrieved information.\n",
    "\n",
    "👩‍🍳 Here are the ingredients of a RAG Pipeline:\n",
    "\n",
    "1. Unstructured data (the movie reviews will suffice).\n",
    "2. A pre-trained sentence encoder.\n",
    "3. A vector database and an appropriate similarity metric. Store the encoded vectors in a vector database. Choose an appropriate similarity metric.\n",
    "4. A large language model.\n",
    "5. A system prompt that instructs an LLM to use the context to answer a query.\n",
    "\n",
    "Finally, you should set up a collection and process the data into records:\n",
    "\n",
    "1. Clean and preprocess the data (e.g. by removing irrelevant information, special characters, and applying tokenization, stemming, or lemmatization).\n",
    "2. Use the sentence encoder model to convert text documents into fixed-length dense vectors.\n",
    "\n",
    "We covered these steps in the previous sections. Now we can actually code our own movie assistant! The required steps will be the following:\n",
    "\n",
    "1. For a given input query, encode the query using the same sentence encoder model used for encoding the documents.\n",
    "2. Perform a similarity search in the vector database to retrieve the top-k most relevant documents based on their vector representation.\n",
    "3. Format the retrieved documents and the query to make a single prompt.\n",
    "4. Send the prompt to the generative model.\n",
    "5. Package this into a nice function, or user interface."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Prerequisite: Get the OpenAI Key"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "WWRZeaVJS3LA",
    "outputId": "24292119-8a0f-49de-f137-a18eb0dcdb47"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "os.environ[\"OPENAI_API_KEY\"] = ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Preparation 1: Load Data\n",
    "\n",
    "Load the data using the built-in functions. To make things run faster, you can just take the first 300 records."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "uWeZFFXAVzup"
   },
   "outputs": [],
   "source": [
    "movies = ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Preparation 2: Create the Encoder Model\n",
    "\n",
    "Create an `encoder` with `all-MiniLM-L6-v2`. Use the `sentence-transformers` library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "RKnbu0emVon_",
    "outputId": "e3b0f0ae-24f0-4dc0-b44b-1230ac3f6f8b"
   },
   "outputs": [],
   "source": [
    "encoder = ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Preparation 3: Create the Qdrant Collection\n",
    "\n",
    "Create a `qdrant` collection using the dimensions of the encoder model and cosine similarity as the distance metric."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "RKnbu0emVon_",
    "outputId": "e3b0f0ae-24f0-4dc0-b44b-1230ac3f6f8b"
   },
   "outputs": [],
   "source": [
    "qdrant = ...\n",
    "\n",
    "COLLECTION_NAME = ...\n",
    "\n",
    "qdrant.recreate_collection(...)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Preparation 4: Populate the collection\n",
    "\n",
    "Encode the first 300 movies in the dataset and upload them in the Qdrant collection."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "9X69V57AUGCc",
    "outputId": "c6e123f3-4350-46cf-87e8-8f52592c9677"
   },
   "outputs": [],
   "source": [
    "records = ...\n",
    "\n",
    "qdrant.upload_points(...)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Preparation 5: Write a function to retrieve results\n",
    "\n",
    "Write a function that accepts a query and a number of items to retrieve, and returns that number of items from the vector database."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "kVhY7vVvWXJ6"
   },
   "outputs": [],
   "source": [
    "def get_records(\n",
    "    query,\n",
    "    *,\n",
    "    encoder=encoder,\n",
    "    client=qdrant,\n",
    "    collection=COLLECTION_NAME,\n",
    "    max_results=10,\n",
    "): ...\n",
    "\n",
    "\n",
    "question = ...\n",
    "\n",
    "docs = get_records(...)\n",
    "results = ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we are finally ready to brew our movie assistant! Beware, the exercises will require a bit more coding skill."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 1: Prompt Engineering\n",
    "\n",
    "Most LLMs can be provided with a `system`, i.e. a string that codifies its behavior. This string is prepended to every prompt and is used to give a tone to the model. For example:\n",
    "\n",
    "```python\n",
    "system_1 = \"\"\"You are a helpful assistant.\"\"\"\n",
    "\n",
    "system_2 = \"\"\"You are a distracted poet who always answer in rhyme.\"\"\"\n",
    "```\n",
    "\n",
    "Your first task is to craft a system prompt that will give a unique personality to your movie buddy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SYSTEM = ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "## Intermezzo: Python String Fomatting\n",
    "\n",
    "Before going further, however, we need to review a bit of string formatting. If you already know what that is and how to use the `.format` method, you can move on. For all the others, it's time for an intermezzo.\n",
    "\n",
    "Here is a refresher. There are multiple ways of formatting a string in Python. The most used is this:\n",
    "\n",
    "```python\n",
    "name = \"Rob\"\n",
    "role = \"Movie Buddy\"\n",
    "\n",
    "print(f\"Hi! I am {name} and I am a great {role}!\")\n",
    "```\n",
    "\n",
    "However, there is another, equivalent way: the `.format` method. It works like this:\n",
    "\n",
    "```python\n",
    "print(\"Hi! I am {name} and I am a great {role}!\".format(name=\"Rob\", role=\"Movie Buddy\"))\n",
    "```\n",
    "\n",
    "Note: no more `f` at the beginning of the string! This notation is more verbose and achieves the same result. Generally speaking, you should always prefer the first method. However, this kind of formatting allows you to build a \"lazy\" string. Try running the following code:\n",
    "\n",
    "```python\n",
    "message = \"Hi! I am {name} and I am a great {role}!\"\n",
    "print(message)\n",
    "```\n",
    "\n",
    "No string substitution was performed. We can use this at our advantage. Try running the following code:\n",
    "\n",
    "```python\n",
    "print(message.format(name=\"Rob\", role=\"Movie Buddy\"))\n",
    "```\n",
    "\n",
    "Do you see where we are headed? We can use this `.format` method to \"populate\" the prompt template with the bits we obtain from the user query and the vector database.\n",
    "\n",
    "Now we are finally ready to move on!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2: Prompt Templates\n",
    "\n",
    "You will need to create two *templates* to build a skeleton out of the records retrieved from the vector database and the user query.\n",
    "\n",
    "1. A *prompt template* that will instruct the LLM to use the context to answer the query. This looks a bit like the system prompt. It might look a bit like this:\n",
    "\n",
    "```\n",
    "\"\"\"Given this context: {context}\n",
    "\n",
    "Answer this query: {prompt}\n",
    "\"\"\"\n",
    "```\n",
    "\n",
    "2. A *context template* to format the items retrieved from the vector DB into a textual form. A bit like this:\n",
    "\n",
    "```\n",
    "\"\"\"\n",
    "Movie 1 - some details - some more details\n",
    "Movie 2 - some details - some more details\n",
    "...\n",
    "\"\"\"\n",
    "```\n",
    "\n",
    "Inspect the results of your qdrant query: what information would you like to retain in your context?\n",
    "\n",
    "Beware: this is a bit trickier, because you have to parse every result into a string and then merge every string into a single one. You might want to write this into a function, so that you can use it later more easily."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "GbFzY5PxbITe"
   },
   "outputs": [],
   "source": [
    "prompt_template = ...\n",
    "context_template = ...\n",
    "\n",
    "\n",
    "def format_records_into_context(records, *, template): ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise 3: Use the AI!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's finally time to unleash the artificial intelligence. Create an instance of the OpenAI client. Write a function that follows these guides:\n",
    "\n",
    "1. Takes a `question` and a `max_results` parameters. The first is the movie query, the second is the number of items to retrieve from the vector database.\n",
    "2. Takes a `system`, a `prompt_template` and a `context_template`.\n",
    "3. Calls the `get_records` function to get the `max_results` most similar records in the vector DB.\n",
    "4. Format the records according using the `context_template`.\n",
    "5. Create a prompt using the `prompt_template`.\n",
    "6. Query GPT-3.5 using the resulting template!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "yQncpqpBZljG",
    "outputId": "5207715e-0770-4e14-9310-bf514170355c"
   },
   "outputs": [],
   "source": [
    "import openai\n",
    "\n",
    "client = openai.OpenAI()\n",
    "\n",
    "\n",
    "def ask(\n",
    "    question,\n",
    "    *,\n",
    "    max_results=10,\n",
    "    system=SYSTEM,\n",
    "    prompt_template=prompt_template,\n",
    "    context_template=context_template,\n",
    "    qdrant=qdrant,\n",
    "    collection=COLLECTION_NAME,\n",
    "): ...\n",
    "\n",
    "\n",
    "answer = ask(question=question)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Challenge 1: Use different embeddings models!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ODGc0O8Hg4_S"
   },
   "source": [
    "# Challenge 2: to Production!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's time to get Object Oriented! Refactor the code above into a `VectorDBService` class. It should satisfy the following requirements:\n",
    "\n",
    "1. In the `__init__` method, the function accepts a collection name, an encoder model and a distance metric. In the initialisation, a collection is (re)created using the collection name and distance metric; model weights are also donwloaded.\n",
    "2. A `process` method that accepts a dataset and ingests it into the collection.\n",
    "3. A `get_top_k` method that returns the top K elements by similarity given a `query` vector.\n",
    "\n",
    "Then, write a `MovieBuddy` class:\n",
    "\n",
    "1. The `__init__` function takes a `system`, a `prompt_template` and a `context_template`, plus a `VectorDBService`.\n",
    "2. Has a `apply_template` method that, given a query and a series of records, formats them using the `context_template` and the `prompt_template`, and then returns a `query`.\n",
    "3. It has a `ask` method that takes in a `question` parameter and a `client` object of type `OpenAI`. The implementation uses `VectorDBService.get_top_k` and `apply_template` to ask the client and generate a completion."
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}