From 4ba3d6e1b45ffc7bd53d8bef3991da487973ce57 Mon Sep 17 00:00:00 2001 From: LinxinS97 Date: Mon, 4 Dec 2023 10:33:25 +0800 Subject: [PATCH 1/8] try to fix blog --- website/blog/2023-11-26-Agent-AutoBuild/index.mdx | 1 - 1 file changed, 1 deletion(-) diff --git a/website/blog/2023-11-26-Agent-AutoBuild/index.mdx b/website/blog/2023-11-26-Agent-AutoBuild/index.mdx index 5b9c9804b21..06285ed92eb 100644 --- a/website/blog/2023-11-26-Agent-AutoBuild/index.mdx +++ b/website/blog/2023-11-26-Agent-AutoBuild/index.mdx @@ -1,4 +1,3 @@ - --- title: Agent AutoBuild - Automatically Building Multi-agent Systems authors: From 58c19f4bf0fd5cb30453c589bce7736738f0fc97 Mon Sep 17 00:00:00 2001 From: LinxinS97 Date: Mon, 4 Dec 2023 11:08:30 +0800 Subject: [PATCH 2/8] modify blog --- .../blog/2023-11-26-Agent-AutoBuild/index.mdx | 41 ++++++++++++++----- 1 file changed, 30 insertions(+), 11 deletions(-) diff --git a/website/blog/2023-11-26-Agent-AutoBuild/index.mdx b/website/blog/2023-11-26-Agent-AutoBuild/index.mdx index 06285ed92eb..a5c26cddb55 100644 --- a/website/blog/2023-11-26-Agent-AutoBuild/index.mdx +++ b/website/blog/2023-11-26-Agent-AutoBuild/index.mdx @@ -69,15 +69,29 @@ building_task = "Find a paper on arxiv by programming, and analysis its applicat Use `build()` to let build manager (with a `builder_model` as backbone) complete the group chat agents generation. If you think coding is necessary in your task, you can use `coding=True` to add a user proxy (a local code interpreter) into the agent list as: ```python -builder.build(building_task, default_llm_config, coding=True) +agent_list, agent_configs = builder.build(building_task, default_llm_config, coding=True) ``` If `coding` is not specified, AgentBuilder will determine on its own whether the user proxy should be added or not according to the task. ### Step 5: execute the task Let agents generated in `build()` to complete the task collaboratively in a group chat. ```python -execution_task="Find a latest paper about gpt-4 on arxiv and find its potential applications in software." -builder.start(task=execution_task) +import autogen + +def start_task(execution_task: str, agent_list: list, llm_config: dict): + config_list = autogen.config_list_from_json(config_path, filter_dict={"model": ["gpt-4-1106-preview"]}) + + group_chat = autogen.GroupChat(agents=agent_list, messages=[], max_round=12) + manager = autogen.GroupChatManager( + groupchat=group_chat, llm_config={"config_list": config_list, **llm_config} + ) + agent_list[0].initiate_chat(manager, message=execution_task) + +start_task( + execution_task="Find a recent paper about gpt-4 on arxiv and find its potential applications in software.", + agent_list=agent_list, + llm_config=default_llm_config +) ``` ### Step 6 (Optional): clear all agents and prepare for the next task @@ -117,8 +131,9 @@ You can provide a specific filename, otherwise, AgentBuilder will save config to You can load the saved config and skip the building process. AgentBuilder will create agents with those information without prompting the build manager. ```python -new_builder = AgentBuilder(config_path=config_path).load(saved_path) -new_builder.start() +new_builder = AgentBuilder(config_path=config_path) +agent_list, agent_config = new_builder.load(saved_path) +start_task(...) # skip build() ``` ## Use Open-source LLM @@ -138,14 +153,18 @@ After satisfying the requirements, you can add an open-source LLM's huggingface and specify it when initializing AgentBuilder. AgentBuilder will automatically set up an endpoint server for open-source LLM. Make sure you have sufficient GPUs resources. -## Use GPTs -[GPTs](https://openai.com/blog/introducing-gpts) allow user to create an assistant with a simple instruction of the task. It has plugin support that can let ChatGPT complete some complex instructions, and can optionally update the assistant's instruction to let it adapt to new task or improve on the current task. -AutoBuild also support GPTs api by adding `use_gpts=True` to the `build()` function. +## Use OpenAI Assistant +[Assistants API](https://platform.openai.com/docs/assistants/overview) allows you to build AI assistants within your own applications. +An Assistant has instructions and can leverage models, tools, and knowledge to respond to user queries. +AutoBuild also support assistant api by adding `use_oai_assistant=True` to `build()`. ```python -# Transfer to GPTs API. -new_builder.build(building_task, default_llm_config, use_gpts=True) +# Transfer to OpenAI Assistant API. +agent_list, agent_config = new_builder.build(building_task, default_llm_config, use_oai_assistant=True) +... ``` ## Summary -We propose AutoBuild with a new class `AgentBuilder`. AutoBuild can help user solve their complex task with an automatically built multi-agent system. AutoBuild support open-source LLMs and GPTs api, giving users more flexibility to choose their favorite models. +We propose AutoBuild with a new class `AgentBuilder`. +AutoBuild can help user solve their complex task with an automatically built multi-agent system. +AutoBuild support open-source LLMs and GPTs api, giving users more flexibility to choose their favorite models. More related features coming soon. From 4b856164e11d0251960f3454569d59626e948a98 Mon Sep 17 00:00:00 2001 From: LinxinS97 Date: Tue, 5 Dec 2023 00:23:33 +0800 Subject: [PATCH 3/8] fix test error in #717; fix blog typo in installation; update blogs with output examples. --- test/agentchat/contrib/test_agent_builder.py | 36 ++++++++++++++----- .../blog/2023-11-26-Agent-AutoBuild/index.mdx | 19 ++++++++-- 2 files changed, 45 insertions(+), 10 deletions(-) diff --git a/test/agentchat/contrib/test_agent_builder.py b/test/agentchat/contrib/test_agent_builder.py index 3f0ee7cbbfc..f0184ea1cbb 100644 --- a/test/agentchat/contrib/test_agent_builder.py +++ b/test/agentchat/contrib/test_agent_builder.py @@ -40,8 +40,13 @@ def test_build(): builder.build( building_task=building_task, default_llm_config={"temperature": 0}, - user_proxy_work_dir=f"{here}/test_agent_scripts", - docker="python:3", + code_execution_config={ + "last_n_messages": 2, + "work_dir": f"{here}/test_agent_scripts", + "use_docker": False, + "timeout": 60, + "use_docker": "python:3", + } ) # check number of agents @@ -67,8 +72,13 @@ def test_save(): builder.build( building_task=building_task, default_llm_config={"temperature": 0}, - user_proxy_work_dir=f"{here}/test_agent_scripts", - docker="python:3", + code_execution_config={ + "last_n_messages": 2, + "work_dir": f"{here}/test_agent_scripts", + "use_docker": False, + "timeout": 60, + "use_docker": "python:3", + } ) saved_files = builder.save(f"{here}/example_save_agent_builder_config.json") @@ -99,8 +109,13 @@ def test_load(): agent_list, loaded_agent_configs = builder.load( config_save_path, - user_proxy_work_dir=f"{here}/test_agent_scripts", - docker="python:3", + code_execution_config={ + "last_n_messages": 2, + "work_dir": f"{here}/test_agent_scripts", + "use_docker": False, + "timeout": 60, + "use_docker": "python:3", + } ) # check config loading @@ -125,8 +140,13 @@ def test_clear_agent(): config_save_path = f"{here}/example_test_agent_builder_config.json" builder.load( config_save_path, - user_proxy_work_dir=f"{here}/test_agent_scripts", - docker="python:3", + code_execution_config={ + "last_n_messages": 2, + "work_dir": f"{here}/test_agent_scripts", + "use_docker": False, + "timeout": 60, + "use_docker": "python:3", + } ) builder.clear_all_agents() diff --git a/website/blog/2023-11-26-Agent-AutoBuild/index.mdx b/website/blog/2023-11-26-Agent-AutoBuild/index.mdx index a5c26cddb55..a5bf4baceda 100644 --- a/website/blog/2023-11-26-Agent-AutoBuild/index.mdx +++ b/website/blog/2023-11-26-Agent-AutoBuild/index.mdx @@ -29,7 +29,7 @@ up an endpoint server automatically without any user participant. ## Installation - AutoGen: ```bash -pip install pyautogen==0.2.0b5 +pip install pyautogen==0.2.0 ``` - (Optional: if you want to use open-source LLMs) vLLM and FastChat ```bash @@ -72,6 +72,21 @@ If you think coding is necessary in your task, you can use `coding=True` to add agent_list, agent_configs = builder.build(building_task, default_llm_config, coding=True) ``` If `coding` is not specified, AgentBuilder will determine on its own whether the user proxy should be added or not according to the task. +The generated `agent_list` is a list of `AssistantAgent` instances. +If `coding` is true, a user proxy (a `UserProxyAssistant` instance) will be added as the first element to the `agent_list`. +`agent_configs` is a list of agent configurations including agent name, backbone LLM model, and system message. +For example +``` +// an example of agent_configs. AgentBuilder will generate agents with the following configurations. +[ + { + "name": "Data_scientist", + "model": "gpt-4-1106-preview", + "system_message": "As a Data Scientist, you are tasked with automating the retrieval and analysis of academic papers from arXiv. Utilize your Python programming acumen to develop scripts for gathering necessary information such as searching for relevant papers, downloading them, and processing their contents. Apply your analytical and language skills to interpret the data and deduce the applications of the research within specific domains.\n\n1. To compile information, write and implement Python scripts that search and interact with online resources, download and read files, extract content from documents, and perform other information-gathering tasks. Use the printed output as the foundation for your subsequent analysis.\n\n2. Execute tasks programmatically with Python scripts when possible, ensuring results are directly displayed. Approach each task with efficiency and strategic thinking.\n\nProgress through tasks systematically. In instances where a strategy is not provided, outline your plan before executing. Clearly distinguish between tasks handled via code and those utilizing your analytical expertise.\n\nWhen providing code, include only Python scripts meant to be run without user alterations. Users should execute your script as is, without modifications:\n\n```python\n# filename: \n# Python script\nprint(\"Your output\")\n```\n\nUsers should not perform any actions other than running the scripts you provide. Avoid presenting partial or incomplete scripts that require user adjustments. Refrain from requesting users to copy-paste results; instead, use the 'print' function when suitable to display outputs. Monitor the execution results they share.\n\nIf an error surfaces, supply corrected scripts for a re-run. If the strategy fails to resolve the issue, reassess your assumptions, gather additional details as needed, and explore alternative approaches.\n\nUpon successful completion of a task and verification of the results, confirm the achievement of the stated objective. Ensuring accuracy and validity of the findings is paramount. Evidence supporting your conclusions should be provided when feasible.\n\nUpon satisfying the user's needs and ensuring all tasks are finalized, conclude your assistance with \"TERMINATE\"." + }, + ... +] +``` ### Step 5: execute the task Let agents generated in `build()` to complete the task collaboratively in a group chat. @@ -107,7 +122,7 @@ You can save all necessary information of the built group chat agents by ```python saved_path = builder.save() ``` -Configs will be saved in the JSON format with following content: +Configurations will be saved in JSON format with following content: ```json // FILENAME: save_config_TASK_MD5.json { From 165d1a33896061bf7d1fc6259b93c8767174fda2 Mon Sep 17 00:00:00 2001 From: LinxinS97 Date: Tue, 5 Dec 2023 00:25:27 +0800 Subject: [PATCH 4/8] pre-commit --- test/agentchat/contrib/test_agent_builder.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/test/agentchat/contrib/test_agent_builder.py b/test/agentchat/contrib/test_agent_builder.py index f0184ea1cbb..687cdee5706 100644 --- a/test/agentchat/contrib/test_agent_builder.py +++ b/test/agentchat/contrib/test_agent_builder.py @@ -46,7 +46,7 @@ def test_build(): "use_docker": False, "timeout": 60, "use_docker": "python:3", - } + }, ) # check number of agents @@ -78,7 +78,7 @@ def test_save(): "use_docker": False, "timeout": 60, "use_docker": "python:3", - } + }, ) saved_files = builder.save(f"{here}/example_save_agent_builder_config.json") @@ -115,7 +115,7 @@ def test_load(): "use_docker": False, "timeout": 60, "use_docker": "python:3", - } + }, ) # check config loading @@ -146,7 +146,7 @@ def test_clear_agent(): "use_docker": False, "timeout": 60, "use_docker": "python:3", - } + }, ) builder.clear_all_agents() From f0057be00fddd2c5305508c7283f8af6ae94303c Mon Sep 17 00:00:00 2001 From: LinxinS97 Date: Tue, 5 Dec 2023 00:26:47 +0800 Subject: [PATCH 5/8] pre-commit --- test/agentchat/contrib/test_agent_builder.py | 4 ---- 1 file changed, 4 deletions(-) diff --git a/test/agentchat/contrib/test_agent_builder.py b/test/agentchat/contrib/test_agent_builder.py index 687cdee5706..709ca31f28f 100644 --- a/test/agentchat/contrib/test_agent_builder.py +++ b/test/agentchat/contrib/test_agent_builder.py @@ -43,7 +43,6 @@ def test_build(): code_execution_config={ "last_n_messages": 2, "work_dir": f"{here}/test_agent_scripts", - "use_docker": False, "timeout": 60, "use_docker": "python:3", }, @@ -75,7 +74,6 @@ def test_save(): code_execution_config={ "last_n_messages": 2, "work_dir": f"{here}/test_agent_scripts", - "use_docker": False, "timeout": 60, "use_docker": "python:3", }, @@ -112,7 +110,6 @@ def test_load(): code_execution_config={ "last_n_messages": 2, "work_dir": f"{here}/test_agent_scripts", - "use_docker": False, "timeout": 60, "use_docker": "python:3", }, @@ -143,7 +140,6 @@ def test_clear_agent(): code_execution_config={ "last_n_messages": 2, "work_dir": f"{here}/test_agent_scripts", - "use_docker": False, "timeout": 60, "use_docker": "python:3", }, From 8ceb62b35c89f5afb2d0332b8a7908bda0f29ef0 Mon Sep 17 00:00:00 2001 From: Jieyu Zhang Date: Mon, 4 Dec 2023 15:24:00 -0800 Subject: [PATCH 6/8] Update website/blog/2023-11-26-Agent-AutoBuild/index.mdx Co-authored-by: Qingyun Wu --- website/blog/2023-11-26-Agent-AutoBuild/index.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/blog/2023-11-26-Agent-AutoBuild/index.mdx b/website/blog/2023-11-26-Agent-AutoBuild/index.mdx index a5bf4baceda..611af64e705 100644 --- a/website/blog/2023-11-26-Agent-AutoBuild/index.mdx +++ b/website/blog/2023-11-26-Agent-AutoBuild/index.mdx @@ -29,7 +29,7 @@ up an endpoint server automatically without any user participant. ## Installation - AutoGen: ```bash -pip install pyautogen==0.2.0 +pip install pyautogen~=0.2.0 ``` - (Optional: if you want to use open-source LLMs) vLLM and FastChat ```bash From 1c82a028b63dc635832020e15e283fdb19a6ab3c Mon Sep 17 00:00:00 2001 From: LinxinS97 Date: Wed, 6 Dec 2023 00:27:16 +0800 Subject: [PATCH 7/8] add future work --- website/blog/2023-11-26-Agent-AutoBuild/index.mdx | 3 +++ 1 file changed, 3 insertions(+) diff --git a/website/blog/2023-11-26-Agent-AutoBuild/index.mdx b/website/blog/2023-11-26-Agent-AutoBuild/index.mdx index 611af64e705..40fe8bda162 100644 --- a/website/blog/2023-11-26-Agent-AutoBuild/index.mdx +++ b/website/blog/2023-11-26-Agent-AutoBuild/index.mdx @@ -178,6 +178,9 @@ agent_list, agent_config = new_builder.build(building_task, default_llm_config, ... ``` +## Future work/Roadmap +- Let build manager select the best agents from a given library/database to solve the task. + ## Summary We propose AutoBuild with a new class `AgentBuilder`. AutoBuild can help user solve their complex task with an automatically built multi-agent system. From 8df411350c748a256eb4723fa393a3d19bf15b90 Mon Sep 17 00:00:00 2001 From: Jieyu Zhang Date: Tue, 5 Dec 2023 09:15:42 -0800 Subject: [PATCH 8/8] fix grammar --- .../blog/2023-11-26-Agent-AutoBuild/index.mdx | 50 +++++++++---------- 1 file changed, 25 insertions(+), 25 deletions(-) diff --git a/website/blog/2023-11-26-Agent-AutoBuild/index.mdx b/website/blog/2023-11-26-Agent-AutoBuild/index.mdx index 40fe8bda162..faaeadc77d5 100644 --- a/website/blog/2023-11-26-Agent-AutoBuild/index.mdx +++ b/website/blog/2023-11-26-Agent-AutoBuild/index.mdx @@ -10,21 +10,21 @@ tags: [LLM, research] **TL;DR:** Introducing **AutoBuild**, building multi-agent system automatically, fast, and easily for complex tasks with minimal -user prompt required, powered by a new designed class **AgentBuilder**. AgentBuilder also support open-source LLMs by +user prompt required, powered by a new designed class **AgentBuilder**. AgentBuilder also supports open-source LLMs by leveraging [vLLM](https://docs.vllm.ai/en/latest/index.html) and [FastChat](https://github.com/lm-sys/FastChat). -Checkout example notebooks and file for reference: +Checkout example notebooks and source code for reference: - [AutoBuild Examples](https://github.com/microsoft/autogen/blob/main/notebook/agentchat_autobuild.ipynb) - [AgentBuilder](https://github.com/microsoft/autogen/blob/main/autogen/agentchat/contrib/agent_builder.py) ## Introduction -In this blog, we introduce **AutoBuild**, a pipeline that can automatically build multi-agent system for complex task. +In this blog, we introduce **AutoBuild**, a pipeline that can automatically build multi-agent systems for complex tasks. Specifically, we design a new class called **AgentBuilder**, which will complete the generation of participant expert agents -and the construction of group chat automatically after the user provide descriptions of a building task and a execution task. +and the construction of group chat automatically after the user provides descriptions of a building task and an execution task. -AgentBuilder support open-source models on Hugging Face powered by [vLLM](https://docs.vllm.ai/en/latest/index.html) -and [FastChat](https://github.com/lm-sys/FastChat). Once the user choose to use open-source LLM, AgentBuilder will set -up an endpoint server automatically without any user participant. +AgentBuilder supports open-source models on Hugging Face powered by [vLLM](https://docs.vllm.ai/en/latest/index.html) +and [FastChat](https://github.com/lm-sys/FastChat). Once the user chooses to use open-source LLM, AgentBuilder will set +up an endpoint server automatically without any user participation. ## Installation - AutoGen: @@ -41,7 +41,7 @@ In this section, we provide a step-by-step example of how to use AgentBuilder to ### Step 1: prepare configurations First, we need to prepare the Agent configurations. -Specifically, a config path containing model name and api key, and a default config for each agent, are required. +Specifically, a config path containing the model name and API key, and a default config for each agent, are required. ```python config_path = '/home/elpis_ubuntu/LLM/autogen/OAI_CONFIG_LIST' # modify path default_llm_config = { @@ -49,8 +49,8 @@ default_llm_config = { } ``` -### Step 2: create a AgentBuilder instance -Then, we create a AgentBuilder instance with the config path and default config. +### Step 2: create an AgentBuilder instance +Then, we create an AgentBuilder instance with the config path and default config. You can also specific the builder model and agent model, which are the LLMs used for building and agent respectively. ```python from autogen.agentchat.contrib.agent_builder import AgentBuilder @@ -59,15 +59,15 @@ builder = AgentBuilder(config_path=config_path, builder_model='gpt-4-1106-previe ``` ### Step 3: specify the building task -Specify a building task with a general description. Building task will help build manager (a LLM) decide what agents should be build. +Specify a building task with a general description. Building task will help the build manager (a LLM) decide what agents should be built. Note that your building task should have a general description of the task. Adding some specific examples is better. ```python -building_task = "Find a paper on arxiv by programming, and analysis its application in some domain. For example, find a latest paper about gpt-4 on arxiv and find its potential applications in software." +building_task = "Find a paper on arxiv by programming, and analyze its application in some domain. For example, find a latest paper about gpt-4 on arxiv and find its potential applications in software." ``` ### Step 4: build group chat agents -Use `build()` to let build manager (with a `builder_model` as backbone) complete the group chat agents generation. -If you think coding is necessary in your task, you can use `coding=True` to add a user proxy (a local code interpreter) into the agent list as: +Use `build()` to let the build manager (with a `builder_model` as backbone) complete the group chat agents generation. +If you think coding is necessary for your task, you can use `coding=True` to add a user proxy (a local code interpreter) into the agent list as: ```python agent_list, agent_configs = builder.build(building_task, default_llm_config, coding=True) ``` @@ -89,7 +89,7 @@ For example ``` ### Step 5: execute the task -Let agents generated in `build()` to complete the task collaboratively in a group chat. +Let agents generated in `build()` complete the task collaboratively in a group chat. ```python import autogen @@ -110,11 +110,11 @@ start_task( ``` ### Step 6 (Optional): clear all agents and prepare for the next task -You can clear all agents generated in this task by the following code if your task is completed or the next task is largely different from the current task. +You can clear all agents generated in this task by the following code if your task is completed or if the next task is largely different from the current task. ```python builder.clear_all_agents(recycle_endpoint=True) ``` -If the agent's backbone is an open-source LLM, this process will also shutdown the endpoint server. More details in the next section. +If the agent's backbone is an open-source LLM, this process will also shut down the endpoint server. More details are in the next section. If necessary, you can use `recycle_endpoint=False` to retain the previous open-source LLM's endpoint server. ## Save and Load @@ -122,7 +122,7 @@ You can save all necessary information of the built group chat agents by ```python saved_path = builder.save() ``` -Configurations will be saved in JSON format with following content: +Configurations will be saved in JSON format with the following content: ```json // FILENAME: save_config_TASK_MD5.json { @@ -142,7 +142,7 @@ Configurations will be saved in JSON format with following content: } } ``` -You can provide a specific filename, otherwise, AgentBuilder will save config to the current path with a generated filename `save_config_TASK_MD5.json`. +You can provide a specific filename, otherwise, AgentBuilder will save config to the current path with the generated filename `save_config_TASK_MD5.json`. You can load the saved config and skip the building process. AgentBuilder will create agents with those information without prompting the build manager. ```python @@ -152,7 +152,7 @@ start_task(...) # skip build() ``` ## Use Open-source LLM -AutoBuild support open-source LLM by [vLLM](https://docs.vllm.ai/en/latest/index.html) and [FastChat](https://github.com/lm-sys/FastChat). +AutoBuild supports open-source LLM by [vLLM](https://docs.vllm.ai/en/latest/index.html) and [FastChat](https://github.com/lm-sys/FastChat). Check the supported model list [here](https://docs.vllm.ai/en/latest/models/supported_models.html). After satisfying the requirements, you can add an open-source LLM's huggingface repository to the config file, ```json, @@ -171,18 +171,18 @@ AgentBuilder will automatically set up an endpoint server for open-source LLM. M ## Use OpenAI Assistant [Assistants API](https://platform.openai.com/docs/assistants/overview) allows you to build AI assistants within your own applications. An Assistant has instructions and can leverage models, tools, and knowledge to respond to user queries. -AutoBuild also support assistant api by adding `use_oai_assistant=True` to `build()`. +AutoBuild also supports the assistant API by adding `use_oai_assistant=True` to `build()`. ```python -# Transfer to OpenAI Assistant API. +# Transfer to the OpenAI Assistant API. agent_list, agent_config = new_builder.build(building_task, default_llm_config, use_oai_assistant=True) ... ``` ## Future work/Roadmap -- Let build manager select the best agents from a given library/database to solve the task. +- Let the builder select the best agents from a given library/database to solve the task. ## Summary We propose AutoBuild with a new class `AgentBuilder`. AutoBuild can help user solve their complex task with an automatically built multi-agent system. -AutoBuild support open-source LLMs and GPTs api, giving users more flexibility to choose their favorite models. -More related features coming soon. +AutoBuild supports open-source LLMs and GPTs API, giving users more flexibility to choose their favorite models. +More advanced features are coming soon.