From c0800b9653f2984a949a412dd1de63c1573b6f20 Mon Sep 17 00:00:00 2001 From: Kirushikesh Date: Sun, 1 Sep 2024 10:52:26 -0400 Subject: [PATCH 1/5] Added langchain integrated demo notebook --- .../agentchat_huggingface_langchain.ipynb | 749 ++++++++++++++++++ 1 file changed, 749 insertions(+) create mode 100644 notebook/agentchat_huggingface_langchain.ipynb diff --git a/notebook/agentchat_huggingface_langchain.ipynb b/notebook/agentchat_huggingface_langchain.ipynb new file mode 100644 index 00000000000..d9051a5eace --- /dev/null +++ b/notebook/agentchat_huggingface_langchain.ipynb @@ -0,0 +1,749 @@ +{ + "cells": [ + { + "attachments": {}, + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "# Using LangChain with AutoGen and Hugging Face" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Introduction\n", + "\n", + "In this notebook, we'll demonstrate how to integrate LangChain with AutoGen to enable flexible use of various Language Models (LLMs) in agent-based conversations. This combination allows us to leverage LangChain's extensive support for different LLMs within AutoGen's powerful agent framework.\n", + "\n", + "What we'll cover:\n", + "\n", + "1. Creating a custom model client that uses LangChain to load and interact with LLMs\n", + "2. Configuring AutoGen to use our custom LangChain-based model\n", + "3. Setting up AutoGen agents with the custom model\n", + "4. Demonstrating a simple conversation using this setup\n", + "\n", + "While we used a Hugging Face model in this example, the same approach can be applied to any LLM supported by LangChain, including models from OpenAI, Anthropic, or custom models. This integration opens up a wide range of possibilities for creating sophisticated, multi-model conversational agents using AutoGen\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Requirements\n", + "\n", + "````{=mdx}\n", + ":::info Requirements\n", + "Some extra dependencies are needed for this notebook, which can be installed via pip:\n", + "\n", + "```bash\n", + "pip install pyautogen torch transformers sentencepiece langchain-huggingface \n", + "```\n", + "\n", + "For more information, please refer to the [installation guide](/docs/installation/).\n", + ":::\n", + "````\n", + "\n", + "**NOTE: Depending on what model you use, you may need to play with the default prompts of the Agent's**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Setup and Imports\n", + "\n", + "First, let's import the necessary libraries and define our custom model client." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from types import SimpleNamespace\n", + "import os\n", + "import json\n", + "\n", + "from autogen import AssistantAgent, UserProxyAgent, config_list_from_json\n", + "from langchain_core.messages import AIMessage, HumanMessage, SystemMessage\n", + "from langchain_huggingface import ChatHuggingFace, HuggingFacePipeline" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Create and configure the custom model\n", + "\n", + "A custom model class can be created in many ways, but needs to adhere to the `ModelClient` protocol and response structure which is defined in client.py and shown below.\n", + "\n", + "The response protocol has some minimum requirements, but can be extended to include any additional information that is needed.\n", + "Message retrieval therefore can be customized, but needs to return a list of strings or a list of `ModelClientResponseProtocol.Choice.Message` objects.\n", + "\n", + "\n", + "```python\n", + "class ModelClient(Protocol):\n", + " \"\"\"\n", + " A client class must implement the following methods:\n", + " - create must return a response object that implements the ModelClientResponseProtocol\n", + " - cost must return the cost of the response\n", + " - get_usage must return a dict with the following keys:\n", + " - prompt_tokens\n", + " - completion_tokens\n", + " - total_tokens\n", + " - cost\n", + " - model\n", + "\n", + " This class is used to create a client that can be used by OpenAIWrapper.\n", + " The response returned from create must adhere to the ModelClientResponseProtocol but can be extended however needed.\n", + " The message_retrieval method must be implemented to return a list of str or a list of messages from the response.\n", + " \"\"\"\n", + "\n", + " RESPONSE_USAGE_KEYS = [\"prompt_tokens\", \"completion_tokens\", \"total_tokens\", \"cost\", \"model\"]\n", + "\n", + " class ModelClientResponseProtocol(Protocol):\n", + " class Choice(Protocol):\n", + " class Message(Protocol):\n", + " content: Optional[str]\n", + "\n", + " message: Message\n", + "\n", + " choices: List[Choice]\n", + " model: str\n", + "\n", + " def create(self, params) -> ModelClientResponseProtocol:\n", + " ...\n", + "\n", + " def message_retrieval(\n", + " self, response: ModelClientResponseProtocol\n", + " ) -> Union[List[str], List[ModelClient.ModelClientResponseProtocol.Choice.Message]]:\n", + " \"\"\"\n", + " Retrieve and return a list of strings or a list of Choice.Message from the response.\n", + "\n", + " NOTE: if a list of Choice.Message is returned, it currently needs to contain the fields of OpenAI's ChatCompletion Message object,\n", + " since that is expected for function or tool calling in the rest of the codebase at the moment, unless a custom agent is being used.\n", + " \"\"\"\n", + " ...\n", + "\n", + " def cost(self, response: ModelClientResponseProtocol) -> float:\n", + " ...\n", + "\n", + " @staticmethod\n", + " def get_usage(response: ModelClientResponseProtocol) -> Dict:\n", + " \"\"\"Return usage summary of the response using RESPONSE_USAGE_KEYS.\"\"\"\n", + " ...\n", + "```\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example of simple custom client\n", + "\n", + "Following the huggingface example for using [Mistral's Open-Orca](https://huggingface.co/Open-Orca/Mistral-7B-OpenOrca)\n", + "\n", + "For the response object, python's `SimpleNamespace` is used to create a simple object that can be used to store the response data, but any object that follows the `ClientResponseProtocol` can be used.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# custom client with custom model loader\n", + "\n", + "class CustomModelClient:\n", + " \"\"\"Custom model client implementation for LangChain integration with AutoGen.\"\"\"\n", + "\n", + " def __init__(self, config, **kwargs):\n", + " \"\"\"Initialize the CustomModelClient.\"\"\"\n", + " print(f\"CustomModelClient config: {config}\")\n", + " self.device = config.get(\"device\", \"cpu\")\n", + " \n", + " gen_config_params = config.get(\"params\", {})\n", + " self.model_name = config[\"model\"]\n", + " pipeline = HuggingFacePipeline.from_model_id(\n", + " model_id=self.model_name,\n", + " task=\"text-generation\",\n", + " pipeline_kwargs=gen_config_params,\n", + " device=self.device\n", + " )\n", + " self.model = ChatHuggingFace(llm=pipeline)\n", + " print(f\"Loaded model {config['model']} to {self.device}\")\n", + "\n", + " def _to_chatml_format(self, message):\n", + " \"\"\"Convert message to ChatML format.\"\"\"\n", + " if message['role'] == 'system':\n", + " return SystemMessage(content=message[\"content\"])\n", + " if message['role'] == 'assistant':\n", + " return AIMessage(content=message[\"content\"])\n", + " if message['role'] == 'user':\n", + " return HumanMessage(content=message[\"content\"])\n", + " raise ValueError(f\"Unknown message type: {type(message)}\")\n", + " \n", + " def create(self, params):\n", + " \"\"\"Create a response using the model.\"\"\"\n", + " if params.get(\"stream\", False) and \"messages\" in params:\n", + " raise NotImplementedError(\"Local models do not support streaming.\")\n", + " \n", + " num_of_responses = params.get(\"n\", 1)\n", + " response = SimpleNamespace()\n", + " inputs = [self._to_chatml_format(m) for m in params[\"messages\"]]\n", + " response.choices = []\n", + " response.model = self.model_name\n", + "\n", + " for _ in range(num_of_responses):\n", + " outputs = self.model.invoke(inputs)\n", + " text = outputs.content\n", + " choice = SimpleNamespace()\n", + " choice.message = SimpleNamespace()\n", + " choice.message.content = text\n", + " choice.message.function_call = None\n", + " response.choices.append(choice)\n", + "\n", + " return response\n", + "\n", + " def message_retrieval(self, response):\n", + " \"\"\"Retrieve messages from the response.\"\"\"\n", + " return [choice.message.content for choice in response.choices]\n", + "\n", + " def cost(self, response) -> float:\n", + " \"\"\"Calculate the cost of the response.\"\"\"\n", + " response.cost = 0\n", + " return 0\n", + "\n", + " @staticmethod\n", + " def get_usage(response):\n", + " \"\"\"Get usage statistics.\"\"\"\n", + " return {}" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Set your API Endpoint\n", + "\n", + "The [`config_list_from_json`](https://microsoft.github.io/autogen/docs/reference/oai/openai_utils#config_list_from_json) function loads a list of configurations from an environment variable or a json file.\n", + "\n", + "It first looks for an environment variable of a specified name (\"OAI_CONFIG_LIST\" in this example), which needs to be a valid json string. If that variable is not found, it looks for a json file with the same name. It filters the configs by models (you can filter by other keys as well).\n", + "\n", + "The json looks like the following:\n", + "```json\n", + "[\n", + " {\n", + " \"model\": \"gpt-4\",\n", + " \"api_key\": \"\"\n", + " },\n", + " {\n", + " \"model\": \"gpt-4\",\n", + " \"api_key\": \"\",\n", + " \"base_url\": \"\",\n", + " \"api_type\": \"azure\",\n", + " \"api_version\": \"2024-02-01\"\n", + " },\n", + " {\n", + " \"model\": \"gpt-4-32k\",\n", + " \"api_key\": \"\",\n", + " \"base_url\": \"\",\n", + " \"api_type\": \"azure\",\n", + " \"api_version\": \"2024-02-01\"\n", + " }\n", + "]\n", + "```\n", + "\n", + "You can set the value of config_list in any way you prefer. Please refer to this [notebook](https://github.com/microsoft/autogen/blob/main/notebook/oai_openai_utils.ipynb) for full code examples of the different methods." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Set the config for the custom model\n", + "\n", + "You can add any paramteres that are needed for the custom model loading in the same configuration list.\n", + "\n", + "It is important to add the `model_client_cls` field and set it to a string that corresponds to the class name: `\"CustomModelClient\"`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "os.environ[\"OAI_CONFIG_LIST\"] = json.dumps([{\n", + " \"model\": \"microsoft/Phi-3.5-mini-instruct\",\n", + " \"model_client_cls\": \"CustomModelClient\",\n", + " \"device\": 0,\n", + " \"n\": 1,\n", + " \"params\": {\n", + " \"max_new_tokens\": 100,\n", + " \"top_k\": 50,\n", + " \"temperature\": 0.1,\n", + " \"do_sample\": True,\n", + " \"return_full_text\": False\n", + " }\n", + "}])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "config_list_custom = config_list_from_json(\n", + " \"OAI_CONFIG_LIST\",\n", + " filter_dict={\"model_client_cls\": [\"CustomModelClient\"]},\n", + ")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Construct Agents\n", + "\n", + "Consturct a simple conversation between a User proxy and an Assistent agent" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "assistant = AssistantAgent(\"assistant\", llm_config={\"config_list\": config_list_custom})\n", + "user_proxy = UserProxyAgent(\n", + " \"user_proxy\",\n", + " code_execution_config=False\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Register the custom client class to the assistant agent" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "assistant.register_model_client(model_client_cls=CustomModelClient)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "user_proxy.initiate_chat(assistant, message=\"Write python code to print Hello World!\")" + ] + } + ], + "metadata": { + "front_matter": { + "description": "Define and laod a custom model", + "tags": [ + "custom model" + ] + }, + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "vscode": { + "interpreter": { + "hash": "949777d72b0d2535278d3dc13498b2535136f6dfe0678499012e853ee9abcab1" + } + }, + "widgets": { + "application/vnd.jupyter.widget-state+json": { + "state": { + "2d910cfd2d2a4fc49fc30fbbdc5576a7": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "2.0.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "2.0.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "2.0.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border_bottom": null, + "border_left": null, + "border_right": null, + "border_top": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "454146d0f7224f038689031002906e6f": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "2.0.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "2.0.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "2.0.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_e4ae2b6f5a974fd4bafb6abb9d12ff26", + "IPY_MODEL_577e1e3cc4db4942b0883577b3b52755", + "IPY_MODEL_b40bdfb1ac1d4cffb7cefcb870c64d45" + ], + "layout": "IPY_MODEL_dc83c7bff2f241309537a8119dfc7555", + "tabbable": null, + "tooltip": null + } + }, + "577e1e3cc4db4942b0883577b3b52755": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "2.0.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "2.0.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "2.0.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_allow_html": false, + "layout": "IPY_MODEL_2d910cfd2d2a4fc49fc30fbbdc5576a7", + "max": 1, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_74a6ba0c3cbc4051be0a83e152fe1e62", + "tabbable": null, + "tooltip": null, + "value": 1 + } + }, + "6086462a12d54bafa59d3c4566f06cb2": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "2.0.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "2.0.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "2.0.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border_bottom": null, + "border_left": null, + "border_right": null, + "border_top": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "74a6ba0c3cbc4051be0a83e152fe1e62": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "2.0.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "2.0.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "2.0.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "7d3f3d9e15894d05a4d188ff4f466554": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "2.0.0", + "model_name": "HTMLStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "2.0.0", + "_model_name": "HTMLStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "2.0.0", + "_view_name": "StyleView", + "background": null, + "description_width": "", + "font_size": null, + "text_color": null + } + }, + "b40bdfb1ac1d4cffb7cefcb870c64d45": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "2.0.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "2.0.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "2.0.0", + "_view_name": "HTMLView", + "description": "", + "description_allow_html": false, + "layout": "IPY_MODEL_f1355871cc6f4dd4b50d9df5af20e5c8", + "placeholder": "​", + "style": "IPY_MODEL_ca245376fd9f4354af6b2befe4af4466", + "tabbable": null, + "tooltip": null, + "value": " 1/1 [00:00<00:00, 44.69it/s]" + } + }, + "ca245376fd9f4354af6b2befe4af4466": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "2.0.0", + "model_name": "HTMLStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "2.0.0", + "_model_name": "HTMLStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "2.0.0", + "_view_name": "StyleView", + "background": null, + "description_width": "", + "font_size": null, + "text_color": null + } + }, + "dc83c7bff2f241309537a8119dfc7555": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "2.0.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "2.0.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "2.0.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border_bottom": null, + "border_left": null, + "border_right": null, + "border_top": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "e4ae2b6f5a974fd4bafb6abb9d12ff26": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "2.0.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "2.0.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "2.0.0", + "_view_name": "HTMLView", + "description": "", + "description_allow_html": false, + "layout": "IPY_MODEL_6086462a12d54bafa59d3c4566f06cb2", + "placeholder": "​", + "style": "IPY_MODEL_7d3f3d9e15894d05a4d188ff4f466554", + "tabbable": null, + "tooltip": null, + "value": "100%" + } + }, + "f1355871cc6f4dd4b50d9df5af20e5c8": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "2.0.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "2.0.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "2.0.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border_bottom": null, + "border_left": null, + "border_right": null, + "border_top": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + } + }, + "version_major": 2, + "version_minor": 0 + } + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} From 5cb16778294675e84592802cc7c74ed81cd8e3f0 Mon Sep 17 00:00:00 2001 From: Kirushikesh Date: Sun, 1 Sep 2024 12:55:39 -0400 Subject: [PATCH 2/5] Added output --- .../agentchat_huggingface_langchain.ipynb | 120 ++++++++++++++++-- 1 file changed, 110 insertions(+), 10 deletions(-) diff --git a/notebook/agentchat_huggingface_langchain.ipynb b/notebook/agentchat_huggingface_langchain.ipynb index d9051a5eace..7c3e66a6797 100644 --- a/notebook/agentchat_huggingface_langchain.ipynb +++ b/notebook/agentchat_huggingface_langchain.ipynb @@ -63,7 +63,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 2, "metadata": {}, "outputs": [], "source": [ @@ -155,7 +155,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 3, "metadata": {}, "outputs": [], "source": [ @@ -278,7 +278,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 4, "metadata": {}, "outputs": [], "source": [ @@ -299,7 +299,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 5, "metadata": {}, "outputs": [], "source": [ @@ -321,9 +321,17 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 6, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[autogen.oai.client: 09-01 12:53:51] {484} INFO - Detected custom model client in config: CustomModelClient, model client can not be used until register_model_client is called.\n" + ] + } + ], "source": [ "assistant = AssistantAgent(\"assistant\", llm_config={\"config_list\": config_list_custom})\n", "user_proxy = UserProxyAgent(\n", @@ -341,18 +349,98 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 7, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CustomModelClient config: {'model': 'microsoft/Phi-3.5-mini-instruct', 'model_client_cls': 'CustomModelClient', 'device': 0, 'n': 1, 'params': {'max_new_tokens': 100, 'top_k': 50, 'temperature': 0.1, 'do_sample': True, 'return_full_text': False}}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:07<00:00, 3.51s/it]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Loaded model microsoft/Phi-3.5-mini-instruct to 0\n" + ] + } + ], "source": [ "assistant.register_model_client(model_client_cls=CustomModelClient)" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 8, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[33muser_proxy\u001b[0m (to assistant):\n", + "\n", + "Write python code to print Hello World!\n", + "\n", + "--------------------------------------------------------------------------------\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "You are not running the flash-attention implementation, expect numerical differences.\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[33massistant\u001b[0m (to user_proxy):\n", + "\n", + " ```python\n", + "# filename: hello_world.py\n", + "\n", + "print(\"Hello World!\")\n", + "```\n", + "\n", + "To execute this code, save it in a file named `hello_world.py`. Then, open your terminal or command prompt, navigate to the directory containing the file, and run the following command:\n", + "\n", + "```\n", + "python hello_world.py\n", + "```\n", + "\n", + "The output should be:\n", + "\n", + "```\n", + "Hello World!\n", + "```\n", + "\n", + "If you encounter any errors,\n", + "\n", + "--------------------------------------------------------------------------------\n" + ] + }, + { + "data": { + "text/plain": [ + "ChatResult(chat_id=None, chat_history=[{'content': 'Write python code to print Hello World!', 'role': 'assistant', 'name': 'user_proxy'}, {'content': ' ```python\\n# filename: hello_world.py\\n\\nprint(\"Hello World!\")\\n```\\n\\nTo execute this code, save it in a file named `hello_world.py`. Then, open your terminal or command prompt, navigate to the directory containing the file, and run the following command:\\n\\n```\\npython hello_world.py\\n```\\n\\nThe output should be:\\n\\n```\\nHello World!\\n```\\n\\nIf you encounter any errors,', 'role': 'user', 'name': 'assistant'}], summary=' ```python\\n# filename: hello_world.py\\n\\nprint(\"Hello World!\")\\n```\\n\\nTo execute this code, save it in a file named `hello_world.py`. Then, open your terminal or command prompt, navigate to the directory containing the file, and run the following command:\\n\\n```\\npython hello_world.py\\n```\\n\\nThe output should be:\\n\\n```\\nHello World!\\n```\\n\\nIf you encounter any errors,', cost={'usage_including_cached_inference': {'total_cost': 0}, 'usage_excluding_cached_inference': {'total_cost': 0}}, human_input=['exit'])" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "user_proxy.initiate_chat(assistant, message=\"Write python code to print Hello World!\")" ] @@ -370,6 +458,18 @@ "language": "python", "name": "python3" }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.6" + }, "vscode": { "interpreter": { "hash": "949777d72b0d2535278d3dc13498b2535136f6dfe0678499012e853ee9abcab1" From 8b9bbc14d62a5ae11d78431481e260fc8b877d96 Mon Sep 17 00:00:00 2001 From: Kirushikesh DB <49152921+Kirushikesh@users.noreply.github.com> Date: Tue, 3 Sep 2024 22:24:47 +0530 Subject: [PATCH 3/5] Update notebook/agentchat_huggingface_langchain.ipynb Co-authored-by: gagb --- notebook/agentchat_huggingface_langchain.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/notebook/agentchat_huggingface_langchain.ipynb b/notebook/agentchat_huggingface_langchain.ipynb index 7c3e66a6797..a902b0acd19 100644 --- a/notebook/agentchat_huggingface_langchain.ipynb +++ b/notebook/agentchat_huggingface_langchain.ipynb @@ -18,7 +18,7 @@ "source": [ "## Introduction\n", "\n", - "In this notebook, we'll demonstrate how to integrate LangChain with AutoGen to enable flexible use of various Language Models (LLMs) in agent-based conversations. This combination allows us to leverage LangChain's extensive support for different LLMs within AutoGen's powerful agent framework.\n", + "This notebook demonstrates how you can use LangChain's extensive support for LLMs to enable flexible use of various Language Models (LLMs) in agent-based conversations in AutoGen.\n", "\n", "What we'll cover:\n", "\n", From 51713fcc4e591229b175222ab51f4b55b85e0259 Mon Sep 17 00:00:00 2001 From: Kirushikesh Date: Sat, 14 Sep 2024 08:52:00 -0400 Subject: [PATCH 4/5] Changed the underlying LLM and lint fix --- .../agentchat_huggingface_langchain.ipynb | 74 +++++++++++-------- 1 file changed, 43 insertions(+), 31 deletions(-) diff --git a/notebook/agentchat_huggingface_langchain.ipynb b/notebook/agentchat_huggingface_langchain.ipynb index a902b0acd19..000bcbdbcfd 100644 --- a/notebook/agentchat_huggingface_langchain.ipynb +++ b/notebook/agentchat_huggingface_langchain.ipynb @@ -9,7 +9,7 @@ } }, "source": [ - "# Using LangChain with AutoGen and Hugging Face" + "# Using AutoGen AgentChat with LangChain and Hugging Face\"" ] }, { @@ -67,13 +67,14 @@ "metadata": {}, "outputs": [], "source": [ - "from types import SimpleNamespace\n", - "import os\n", "import json\n", + "import os\n", + "from types import SimpleNamespace\n", "\n", - "from autogen import AssistantAgent, UserProxyAgent, config_list_from_json\n", "from langchain_core.messages import AIMessage, HumanMessage, SystemMessage\n", - "from langchain_huggingface import ChatHuggingFace, HuggingFacePipeline" + "from langchain_huggingface import ChatHuggingFace, HuggingFacePipeline\n", + "\n", + "from autogen import AssistantAgent, UserProxyAgent, config_list_from_json" ] }, { @@ -172,29 +173,26 @@ " gen_config_params = config.get(\"params\", {})\n", " self.model_name = config[\"model\"]\n", " pipeline = HuggingFacePipeline.from_model_id(\n", - " model_id=self.model_name,\n", - " task=\"text-generation\",\n", - " pipeline_kwargs=gen_config_params,\n", - " device=self.device\n", + " model_id=self.model_name, task=\"text-generation\", pipeline_kwargs=gen_config_params,device=self.device,\n", " )\n", " self.model = ChatHuggingFace(llm=pipeline)\n", " print(f\"Loaded model {config['model']} to {self.device}\")\n", "\n", " def _to_chatml_format(self, message):\n", " \"\"\"Convert message to ChatML format.\"\"\"\n", - " if message['role'] == 'system':\n", + " if message[\"role\"] == \"system\":\n", " return SystemMessage(content=message[\"content\"])\n", - " if message['role'] == 'assistant':\n", + " if message[\"role\"] == \"assistant\":\n", " return AIMessage(content=message[\"content\"])\n", - " if message['role'] == 'user':\n", + " if message[\"role\"] == \"user\":\n", " return HumanMessage(content=message[\"content\"])\n", " raise ValueError(f\"Unknown message type: {type(message)}\")\n", - " \n", + "\n", " def create(self, params):\n", " \"\"\"Create a response using the model.\"\"\"\n", " if params.get(\"stream\", False) and \"messages\" in params:\n", " raise NotImplementedError(\"Local models do not support streaming.\")\n", - " \n", + "\n", " num_of_responses = params.get(\"n\", 1)\n", " response = SimpleNamespace()\n", " inputs = [self._to_chatml_format(m) for m in params[\"messages\"]]\n", @@ -282,19 +280,23 @@ "metadata": {}, "outputs": [], "source": [ - "os.environ[\"OAI_CONFIG_LIST\"] = json.dumps([{\n", - " \"model\": \"microsoft/Phi-3.5-mini-instruct\",\n", - " \"model_client_cls\": \"CustomModelClient\",\n", - " \"device\": 0,\n", - " \"n\": 1,\n", - " \"params\": {\n", - " \"max_new_tokens\": 100,\n", - " \"top_k\": 50,\n", - " \"temperature\": 0.1,\n", - " \"do_sample\": True,\n", - " \"return_full_text\": False\n", - " }\n", - "}])" + "os.environ[\"OAI_CONFIG_LIST\"] = json.dumps(\n", + " [\n", + " {\n", + " \"model\": \"mistralai/Mistral-7B-Instruct-v0.2\",\n", + " \"model_client_cls\": \"CustomModelClient\",\n", + " \"device\": 0,\n", + " \"n\": 1,\n", + " \"params\": {\n", + " \"max_new_tokens\": 500,\n", + " \"top_k\": 50,\n", + " \"temperature\": 0.1,\n", + " \"do_sample\": True,\n", + " \"return_full_text\": False,\n", + " },\n", + " }\n", + " ]\n", + ")" ] }, { @@ -309,6 +311,19 @@ ")" ] }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "from huggingface_hub import login\n", + "\n", + "# The Mistral-7B-Instruct-v0.2 is a gated model which requires API token to access\n", + "login(token = getpass.getpass(\"Enter your HuggingFace API Token\"))" + ] + }, { "attachments": {}, "cell_type": "markdown", @@ -334,10 +349,7 @@ ], "source": [ "assistant = AssistantAgent(\"assistant\", llm_config={\"config_list\": config_list_custom})\n", - "user_proxy = UserProxyAgent(\n", - " \"user_proxy\",\n", - " code_execution_config=False\n", - ")" + "user_proxy = UserProxyAgent(\"user_proxy\", code_execution_config=False)" ] }, { From b3553f8201f5062b512847fc15d7bc0da1548769 Mon Sep 17 00:00:00 2001 From: Kirushikesh Date: Tue, 1 Oct 2024 03:35:44 -0400 Subject: [PATCH 5/5] lint fix --- notebook/agentchat_huggingface_langchain.ipynb | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/notebook/agentchat_huggingface_langchain.ipynb b/notebook/agentchat_huggingface_langchain.ipynb index 000bcbdbcfd..248ad18f19c 100644 --- a/notebook/agentchat_huggingface_langchain.ipynb +++ b/notebook/agentchat_huggingface_langchain.ipynb @@ -9,7 +9,7 @@ } }, "source": [ - "# Using AutoGen AgentChat with LangChain and Hugging Face\"" + "# Using AutoGen AgentChat with LangChain-based Custom Client and Hugging Face Models" ] }, { @@ -162,6 +162,7 @@ "source": [ "# custom client with custom model loader\n", "\n", + "\n", "class CustomModelClient:\n", " \"\"\"Custom model client implementation for LangChain integration with AutoGen.\"\"\"\n", "\n", @@ -169,11 +170,14 @@ " \"\"\"Initialize the CustomModelClient.\"\"\"\n", " print(f\"CustomModelClient config: {config}\")\n", " self.device = config.get(\"device\", \"cpu\")\n", - " \n", + "\n", " gen_config_params = config.get(\"params\", {})\n", " self.model_name = config[\"model\"]\n", " pipeline = HuggingFacePipeline.from_model_id(\n", - " model_id=self.model_name, task=\"text-generation\", pipeline_kwargs=gen_config_params,device=self.device,\n", + " model_id=self.model_name,\n", + " task=\"text-generation\",\n", + " pipeline_kwargs=gen_config_params,\n", + " device=self.device,\n", " )\n", " self.model = ChatHuggingFace(llm=pipeline)\n", " print(f\"Loaded model {config['model']} to {self.device}\")\n", @@ -318,10 +322,11 @@ "outputs": [], "source": [ "import getpass\n", + "\n", "from huggingface_hub import login\n", "\n", "# The Mistral-7B-Instruct-v0.2 is a gated model which requires API token to access\n", - "login(token = getpass.getpass(\"Enter your HuggingFace API Token\"))" + "login(token=getpass.getpass(\"Enter your HuggingFace API Token\"))" ] }, {