Add a guide doc for GPTAssistantAgent

notebook/agentchat_oai_assistant_function_call.ipynb

@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Chat with OpenAI Assistant using function call in AutoGen: OSS Insights for Advanced GitHub Data Analysis\n",
+    "# Chat with OpenAI Assistant using function call in AutoGen: OSS Insights for Advanced GitHub Data Analysis\n",
     "\n",
     "This Jupyter Notebook demonstrates how to leverage OSS Insight (Open Source Software Insight) for advanced GitHub data analysis by defining `Function calls` in AutoGen for the OpenAI Assistant. \n",
     "\n",
@@ -14,12 +14,19 @@
     "2. Defining an OpenAI Assistant Agent in AutoGen\n",
     "3. Fetching GitHub Insight Data using Function Call\n",
     "\n",
-    "### Requirements\n",
+    "## Requirements\n",
     "\n",
     "AutoGen requires `Python>=3.8`. To run this notebook example, please install:\n",
+    "````{=mdx}\n",
+    ":::info Requirements\n",
+    "Install `pyautogen`:\n",
     "```bash\n",
     "pip install pyautogen\n",
-    "```"
+    "```\n",
+    "\n",
+    "For more information, please refer to the [installation guide](/docs/installation/).\n",
+    ":::\n",
+    "````"
    ]
   },
   {
@@ -36,7 +43,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Function Schema and Implementation\n",
+    "## Function Schema and Implementation\n",
     "\n",
     "This section provides the function schema definition and their implementation details. These functions are tailored to fetch and process data from GitHub, utilizing OSS Insight's capabilities."
    ]
@@ -101,7 +108,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Defining an OpenAI Assistant Agent in AutoGen\n",
+    "## Defining an OpenAI Assistant Agent in AutoGen\n",
     "\n",
     "Here, we explore how to define an OpenAI Assistant Agent within the AutoGen. This includes setting up the agent to make use of the previously defined function calls for data retrieval and analysis."
    ]
@@ -159,7 +166,18 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Fetching GitHub Insight Data using Function Call\n",
+    "````{=mdx}\n",
+    ":::tip\n",
+    "Learn more about configuring LLMs for agents [here](/docs/topics/llm_configuration).\n",
+    ":::\n",
+    "````\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Fetching GitHub Insight Data using Function Call\n",
     "\n",
     "This part of the notebook demonstrates the practical application of the defined functions and the OpenAI Assistant Agent in fetching and interpreting GitHub Insight data."
    ]
@@ -256,6 +274,13 @@
   }
  ],
  "metadata": {
+  "front_matter": {
+   "description": "This Jupyter Notebook demonstrates how to leverage OSS Insight (Open Source Software Insight) for advanced GitHub data analysis by defining `Function calls` in AutoGen for the OpenAI Assistant.",
+   "tags": [
+    "OpenAI Assistant",
+    "function call"
+   ]
+  },
   "kernelspec": {
    "display_name": "autogen",
    "language": "python",

notebook/agentchat_oai_assistant_groupchat.ipynb

@@ -14,9 +14,16 @@
     "## Requirements\n",
     "\n",
     "AutoGen requires `Python>=3.8`. To run this notebook example, please install:\n",
+    "````{=mdx}\n",
+    ":::info Requirements\n",
+    "Install `pyautogen`:\n",
     "```bash\n",
-    "pip install \"pyautogen>=0.2.3\"\n",
-    "```"
+    "pip install pyautogen\n",
+    "```\n",
+    "\n",
+    "For more information, please refer to the [installation guide](/docs/installation/).\n",
+    ":::\n",
+    "````"
    ]
   },
   {
@@ -50,19 +57,11 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "It first looks for environment variable \"OAI_CONFIG_LIST\" which needs to be a valid json string. If that variable is not found, it then looks for a json file named \"OAI_CONFIG_LIST\". It filters the configs by models (you can filter by other keys as well).\n",
-    "\n",
-    "The config list looks like the following:\n",
-    "```python\n",
-    "config_list = [\n",
-    "    {\n",
-    "        \"model\": \"gpt-4\",\n",
-    "        \"api_key\": \"<your OpenAI API key>\",\n",
-    "    },  # OpenAI API endpoint for gpt-4\n",
-    "]\n",
-    "```\n",
-    "\n",
-    "Currently Azure OpenAI does not support assistant api. You can set the value of config_list in any way you prefer. Please refer to this [notebook](https://github.com/microsoft/autogen/blob/main/website/docs/topics/llm_configuration.ipynb) for full code examples of the different methods."
+    "````{=mdx}\n",
+    ":::tip\n",
+    "Learn more about configuring LLMs for agents [here](/docs/topics/llm_configuration).\n",
+    ":::\n",
+    "````"
    ]
   },
   {
@@ -482,6 +481,13 @@
   }
  ],
  "metadata": {
+  "front_matter": {
+   "description": "This Jupyter Notebook demonstrates how to use the GPTAssistantAgent in AutoGen's group chat mode, enabling collaborative task performance through automated chat with agents powered by LLMs, tools, or humans.",
+   "tags": [
+    "OpenAI Assistant",
+    "group chat"
+   ]
+  },
   "kernelspec": {
    "display_name": "Python 3",
    "language": "python",

notebook/agentchat_oai_code_interpreter.ipynb

@@ -10,9 +10,16 @@
     "## Requirements\n",
     "\n",
     "AutoGen requires `Python>=3.8`. To run this notebook example, please install:\n",
+    "````{=mdx}\n",
+    ":::info Requirements\n",
+    "Install `pyautogen`:\n",
     "```bash\n",
     "pip install pyautogen\n",
-    "```"
+    "```\n",
+    "\n",
+    "For more information, please refer to the [installation guide](/docs/installation/).\n",
+    ":::\n",
+    "````"
    ]
   },
   {
@@ -52,19 +59,11 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "It first looks for environment variable \"OAI_CONFIG_LIST\" which needs to be a valid json string. If that variable is not found, it then looks for a json file named \"OAI_CONFIG_LIST\". It filters the configs by models (you can filter by other keys as well).\n",
-    "\n",
-    "The config list looks like the following:\n",
-    "```python\n",
-    "config_list = [\n",
-    "    {\n",
-    "        \"model\": \"gpt-4\",\n",
-    "        \"api_key\": \"<your OpenAI API key>\",\n",
-    "    },  # OpenAI API endpoint for gpt-4\n",
-    "]\n",
-    "```\n",
-    "\n",
-    "Currently Azure OpenAi does not support assistant api. You can set the value of config_list in any way you prefer. Please refer to this [notebook](https://github.com/microsoft/autogen/blob/main/website/docs/llm_endpoint_configuration.ipynb) for full code examples of the different methods."
+    "````{=mdx}\n",
+    ":::tip\n",
+    "Learn more about configuring LLMs for agents [here](/docs/topics/llm_configuration).\n",
+    ":::\n",
+    "````"
    ]
   },
   {
@@ -297,6 +296,13 @@
   }
  ],
  "metadata": {
+  "front_matter": {
+   "description": "This Jupyter Notebook showcases the integration of the Code Interpreter tool which executes Python code dynamically within applications.",
+   "tags": [
+    "OpenAI Assistant",
+    "code interpreter"
+   ]
+  },
   "kernelspec": {
    "display_name": "Python 3",
    "language": "python",

website/docs/topics/openai-assistant/_category_.json

@@ -0,0 +1,5 @@
+{
+    "position": 2,
+    "label": "OpenAI Assistant",
+    "collapsible": true
+}

website/docs/topics/openai-assistant/gpt_assistant_agent.md

@@ -0,0 +1,177 @@
+# Agent Backed by OpenAI Assistant API
+
+The GPTAssistantAgent is a powerful component of the AutoGen framework, utilizing OpenAI's Assistant API to enhance agents with advanced capabilities. This agent enables the integration of multiple tools such as the Code Interpreter, File Search, and Function Calling, allowing for a highly customizable and dynamic interaction model.
+
+Key Features of the GPTAssistantAgent:
+
+- Multi-Tool Mastery:  Agents can leverage a combination of OpenAI's built-in tools, like [Code Interpreter](https://platform.openai.com/docs/assistants/tools/code-interpreter) and [File Search](https://platform.openai.com/docs/assistants/tools/file-search), alongside custom tools you create or integrate via [Function Calling](https://platform.openai.com/docs/assistants/tools/function-calling).
+
+- Streamlined Conversation Management:  Benefit from persistent threads that automatically store message history and adjust based on the model's context length. This simplifies development by allowing you to focus on adding new messages rather than managing conversation flow.
+
+- File Access and Integration:  Enable agents to access and utilize files in various formats. Files can be incorporated during agent creation or throughout conversations via threads. Additionally, agents can generate files (e.g., images, spreadsheets) and cite referenced files within their responses.
+
+For a practical illustration, here are some examples:
+
+- [Chat with OpenAI Assistant using function call](/docs/notebooks/agentchat_oai_assistant_function_call) demonstrates how to leverage function calling to enable intelligent function selection.
+- [GPTAssistant with Code Interpreter](/docs/notebooks/agentchat_oai_code_interpreter) showcases the integration of the  Code Interpreter tool which executes Python code dynamically within applications.
+- [Group Chat with GPTAssistantAgent](/docs/notebooks/agentchat_oai_assistant_groupchat) demonstrates how to use the GPTAssistantAgent in AutoGen's group chat mode, enabling collaborative task performance through automated chat with agents powered by LLMs, tools, or humans.
+
+## Create a OpenAI Assistant in Autogen
+
+```python
+import os
+
+from autogen import config_list_from_json
+from autogen.agentchat.contrib.gpt_assistant_agent import GPTAssistantAgent
+
+assistant_id = os.environ.get("ASSISTANT_ID", None)
+config_list = config_list_from_json("OAI_CONFIG_LIST")
+llm_config = {
+    "config_list": config_list,
+}
+assistant_config = {
+    # define the openai assistant behavior as you need
+}
+oai_agent = GPTAssistantAgent(
+    name="oai_agent",
+    instructions="I'm an openai assistant running in autogen",
+    llm_config=llm_config,
+    assistant_config=assistant_config,
+)
+```
+
+## Use OpenAI Assistant Built-in Tools and Function Calling
+
+### Code Interpreter
+
+The [Code Interpreter](https://platform.openai.com/docs/assistants/tools/code-interpreter) empowers your agents to write and execute Python code in a secure environment provide by OpenAI. This unlocks several capabilities, including but not limited to:
+
+- Process data: Handle various data formats and manipulate data on the fly.
+- Generate outputs: Create new data files or even visualizations like graphs.
+- ...
+
+Using the Code Interpreter with the following configuration.
+```python
+assistant_config = {
+    "tools": [
+        {"type": "code_interpreter"},
+    ],
+    "tool_resources": {
+        "code_interpreter": {
+            "file_ids": ["$file.id"]  # optional. Files that are passed at the Assistant level are accessible by all Runs with this Assistant.
+        }
+    }
+}
+```
+
+To get the `file.id`, you can employ two methods:
+
+1. OpenAI Playground: Leverage the OpenAI Playground, an interactive platform accessible at https://platform.openai.com/playground, to upload your files and obtain the corresponding file IDs.
+
+2. Code-Based Uploading: Alternatively, you can upload files and retrieve their file IDs programmatically using the following code snippet:
+
+    ```python
+    from openai import OpenAI
+    client = OpenAI(
+        # Defaults to os.environ.get("OPENAI_API_KEY")
+    )
+    # Upload a file with an "assistants" purpose
+    file = client.files.create(
+      file=open("mydata.csv", "rb"),
+      purpose='assistants'
+    )
+    ```
+
+### File Search
+
+The [File Search](https://platform.openai.com/docs/assistants/tools/file-search) tool empowers your agents to tap into knowledge beyond its pre-trained model. This allows you to incorporate your own documents and data, such as product information or code files, into your agent's capabilities.
+
+Using the File Search with the following configuration.
+
+```python
+assistant_config = {
+    "tools": [
+        {"type": "file_search"},
+    ],
+    "tool_resources": {
+        "file_search": {
+            "vector_store_ids": ["$vector_store.id"]
+        }
+    }
+}
+```
+
+Here's how to obtain the vector_store.id using two methods:
+
+1. OpenAI Playground: Leverage the OpenAI Playground, an interactive platform accessible at https://platform.openai.com/playground, to create a vector store, upload your files, and add it into your vector store. Once complete, you'll be able to retrieve the associated `vector_store.id`.
+
+2. Code-Based Uploading:Alternatively, you can upload files and retrieve their file IDs programmatically using the following code snippet:
+
+    ```python
+    from openai import OpenAI
+    client = OpenAI(
+        # Defaults to os.environ.get("OPENAI_API_KEY")
+    )
+
+    # Step 1: Create a Vector Store
+    vector_store = client.beta.vector_stores.create(name="Financial Statements")
+    print("Vector Store created:", vector_store.id)  # This is your vector_store.id
+
+    # Step 2: Prepare Files for Upload
+    file_paths = ["edgar/goog-10k.pdf", "edgar/brka-10k.txt"]
+    file_streams = [open(path, "rb") for path in file_paths]
+
+    # Step 3: Upload Files and Add to Vector Store (with status polling)
+    file_batch = client.beta.vector_stores.file_batches.upload_and_poll(
+        vector_store_id=vector_store.id, files=file_streams
+    )
+
+    # Step 4: Verify Completion (Optional)
+    print("File batch status:", file_batch.status)
+    print("Uploaded file count:", file_batch.file_counts.processed)
+    ```
+
+### Function calling
+
+Function Calling empowers you to extend the capabilities of your agents with your pre-defined functionalities, which allows you to describe custom functions to the Assistant, enabling intelligent function selection and argument generation.
+
+Using the Function calling with the following configuration.
+
+```python
+# learn more from https://platform.openai.com/docs/guides/function-calling/function-calling
+from autogen.function_utils import get_function_schema
+
+def get_current_weather(location: str) -> dict:
+    """
+    Retrieves the current weather for a specified location.
+
+    Args:
+    location (str): The location to get the weather for.
+
+    Returns:
+    Union[str, dict]: A dictionary with weather details..
+    """
+
+    # Simulated response
+    return {
+        "location": location,
+        "temperature": 22.5,
+        "description": "Partly cloudy"
+    }
+
+api_schema = get_function_schema(
+    get_current_weather,
+    name=get_current_weather.__name__,
+    description="Returns the current weather data for a specified location."
+)
+
+api_schema = "define your function schema here"
+assistant_config = {
+    "tools": [
+        {
+            "type": "function",
+            "function": api_schema,
+        }
+    ],
+}
+```