Initiate Agent component

[xuechendi]

Description

add a new component called "Agent", first PR only include agent impl based on 'Langchain'.
Default tools are provided while we also expect user to add their own tools through customer_tool.py file through cmdline.

Issues

n/a.

Type of change

List the type of change like below. Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds new functionality)
• Breaking change (fix or feature that would break existing design and interface)

Dependencies

List the newly introduced 3rd party dependency if exists.

Tests

cd comps/agent/langchain/; python test.py --endpoint_test --ip_addr=${endpoint_ip_addr}

{"query": "What is the weather today in Austin?"}
data: 'The temperature in Austin today is 78°F.</s>'

data: [DONE]

[xuechendi]

Architecture design as below

[xuechendi]

@lvliang-intel @minmin-intel

[hshen14]

Link to RFC: opea-project/docs#26

[xuechendi]

Link to RFC: opea-project/docs#26

Link to RFC: opea-project/docs#29

[xuechendi]

@XuhuiRen @ftian1 @minmin-intel, please help to review this PR.

[xuechendi]

UnitTest added:
Use duckducksearch as only tool.

[xuechendi]

@XuhuiRen , here some update

1. I've update README with instruction for how to add customer_tools:
   https://github.com/opea-project/GenAIComps/blob/22f6b88d721ae58897e3a6eadbe17fadb7b0c8ce/comps/agent/langchain/README.md#4-provide-your-own-tools

2. I also made initial test using RAG as customer_tool. And I'll add it as an AgentExample in genAIExample:
   https://github.com/opea-project/GenAIExamples/pull/413/files

3. I've synced with Minmin and we are planning to support RagAgent in 3 phases. We'll add more advanced strategy for RagAgent to Agent Component later after we ran RAG benchmarks.

4. we hope we can get it merge to provide base codes, so minmin and other can directly contribute new strategies into it.

[xuechendi]

@XuhuiRen , here is the RagAgent Plan - copied from PR413

RagAgent TODOs:

• 1. Agent will help user to decide does this query need to retrieve info from KnowledgeBase or not.
• 2. Agent will help user to decide which knowledge base should be used to retrieve from? (which tool?)
• 3. Agent will help to format/refine retrievel query, so knowledgeGraph, SQL DB can also work as knowledge base.
• 4. Agent will help to decide if retrieved docs are relevant to the query? If yes, continue to answer generation, else rewrite query and retrieve docs again.
• 5. Agent will help to generated answer meet quality standards (factualness, helpfulness, etc.)? If yes, generate again, else output answer.

RAGAgent Roadmap:

with 3 phases along with OPEA release v0.8, v0.9, v1.0

RAG agent v0.1(current version): (base functionality ready)

• support Single VectorDB
• provide agent workflow to close TODO 1, 4, and 5

RAG agent v1.0: (benchmark or real use case)

• Multiple VectorDBs --> retrieve_kb1(query), retrieve_kb2(query), etc.
• provide agent workflow to close TODO 2

RAG agent v2.0: (Support not only vectorDB)

• support structured function_calls, APIs, SQL and KnowledgeGraph query.
• provide agent workflow to close TODO 3

[xuechendi]

Hi, @lkk12014402, there is another comment from @XuhuiRen to align new SPEC API with you.

He told me that you're modifying all current LLM API to align with openAI. Currently, I am using LLMParamDocs, so I guess you will update LLMParamDocs with openAI compatible API, I would like to submit a new PR for UT after your API update completed.

Is it OK that we leave PR using current LLMParamDocs?

[xuechendi]

@lvliang-intel @hshen14 @lkk12014402, please help to review.
Since this PR is getting bigger and bigger, we hope to get this one merged firstly, then Minmin and I can incrementally add more advanced Agent Strategy after.

Per existing comments request:

1. I also added a RAGAgent strategy and RagAgentExample
2. Added how to add customer_tool in README -> link
3. small fixes such as name_fixing, expose llm parameters to users
4. [Fix Later] Xuhui told me that LLM protocols might change to use openAI compatible api, I am now using "LLMParamDocs", so hope this TODO will not block this PR.

[minmin-intel]

looks good

[xuechendi]

New Update - 0719

1. Update Plan Execute

curl http://${ip_address}:9090/v1/chat/completions -X POST -H "Content-Type: application/json" -d '{ "query": "How old is Bill Gate when he built Microsoft?"}'
--- CALL planner ---
input: How old is Bill Gate when he built Microsoft?
plan: ['Step 1: Research the birthdate of Bill Gates', 'Step 2: Calculate the age of Bill Gates at the time he built Microsoft', 'Step 3: Find the date when Bill Gates built Microsoft']
data: {'planner': {'input': 'How old is Bill Gate when he built Microsoft?', 'plan': ['Step 1: Research the birthdate of Bill Gates', 'Step 2: Calculate the age of Bill Gates at the time he built Microsoft', 'Step 3: Find the date when Bill Gates built Microsoft']}}

--- CALL plan_executor ---
past_steps: ['Task is Step 1: Research the birthdate of Bill Gates, Response is The birthdate of Bill Gates is October 28, 1955.  Please let me know if this is correct.  I will wait for your response before proceeding with the next step. ... ]

--- CALL answer_maker ---
output: The final answer is Bill Gates was 20 years old when he built Microsoft in 1975.
data: {'answer_maker': {'output': 'The final answer is Bill Gates was 20 years old when he built Microsoft in 1975.'}}
                                                                                                                                                                                                                                                                                                    data: [DONE]                                                                                                                                                                                                                                                                                                                           

2. Update AgenticRag

curl http://${ip_address}:9090/v1/chat/completions -X POST -H "Content-Type: application/json" -d '{
     "query": "How old is Bill Gate when he built Microsoft?"
    }'
--- CALL agent ---
messages: [AIMessage(content='', additional_kwargs={'tool_calls': [ChatCompletionOutputToolCall(function=ChatCompletionOutputFunctionDefinition(arguments={'query': 'how old is bill gate when he built microsoft'}, name='duckduckgo_search', description=None), id=0, type='function')]}, response_metadata={'token_usage': ChatCompletionOutputUsage(completion_tokens=28, prompt_tokens=5, total_tokens=33), 'model': '', 'finish_reason': 'eos_token'}, id='run-7f1710ba-d7cc-4049-bc81-072fbf08d2d5-0', tool_calls=[{'name': 'duckduckgo_search', 'args': {'query': 'how old is bill gate when he built microsoft'}, 'id': '0', 'type': 'tool_call'}])]
output: content='' additional_kwargs={'tool_calls': [ChatCompletionOutputToolCall(function=ChatCompletionOutputFunctionDefinition(arguments={'query': 'how old is bill gate when he built microsoft'}, name='duckduckgo_search', description=None), id=0, type='function')]} response_metadata={'token_usage': ChatCompletionOutputUsage(completion_tokens=28, prompt_tokens=5, total_tokens=33), 'model': '', 'finish_reason': 'eos_token'} id='run-7f1710ba-d7cc-4049-bc81-072fbf08d2d5-0' tool_calls=[{'name': 'duckduckgo_search', 'args': {'query': 'how old is bill gate when he built microsoft'}, 'id': '0', 'type': 'tool_call'}]
data: {'agent': {'messages': [AIMessage(content='', additional_kwargs={'tool_calls': [ChatCompletionOutputToolCall(function=ChatCompletionOutputFunctionDefinition(arguments={'query': 'how old is bill gate when he built microsoft'}, name='duckduckgo_search', description=None), id=0, type='function')]}, response_metadata={'token_usage': ChatCompletionOutputUsage(completion_tokens=28, prompt_tokens=5, total_tokens=33), 'model': '', 'finish_reason': 'eos_token'}, id='run-7f1710ba-d7cc-4049-bc81-072fbf08d2d5-0', tool_calls=[{'name': 'duckduckgo_search', 'args': {'query': 'how old is bill gate when he built microsoft'}, 'id': '0', 'type': 'tool_call'}])], 'output': AIMessage(content='', additional_kwargs={'tool_calls': [ChatCompletionOutputToolCall(function=ChatCompletionOutputFunctionDefinition(arguments={'query': 'how old is bill gate when he built microsoft'}, name='duckduckgo_search', description=None), id=0, type='function')]}, response_metadata={'token_usage': ChatCompletionOutputUsage(completion_tokens=28, prompt_tokens=5, total_tokens=33), 'model': '', 'finish_reason': 'eos_token'}, id='run-7f1710ba-d7cc-4049-bc81-072fbf08d2d5-0', tool_calls=[{'name': 'duckduckgo_search', 'args': {'query': 'how old is bill gate when he built microsoft'}, 'id': '0', 'type': 'tool_call'}])}}

--- CALL retrieve ---
messages: [ToolMessage(content="Bill Gates. William Henry Gates III (born October 28, 1955) is an American businessman, investor, philanthropist, and writer best known for co-founding the software company Microsoft with his childhood friend Paul Allen. During his career at Microsoft, Gates held the positions of chairman, chief executive officer (CEO), president, and chief ... November 8, 2023. Bill Gates is one of the most iconic business leaders and influential technology pioneers of the modern digital era. As the co-founder of Microsoft, he helped bring the power of personal computing to millions of people worldwide and made computers an indispensable part of everyday life. This article explores Gates ... Microsoft is a multinational computer technology corporation. Microsoft was founded on April 4, 1975, by Bill Gates and Paul Allen in Albuquerque, New Mexico. Its current best-selling products are the Microsoft Windows operating system; Microsoft Office, a suite of productivity software; Xbox, a line of entertainment of games, music, and video; Bing, a line of search engines; and Microsoft ... The Founders of Microsoft. Bill Gates, born on October 28, 1955, in Seattle, Washington, exhibited a keen interest in computer programming from a young age. At just 13, he began exploring the world of programming at Lakeside School. There, he developed his first computer program—a tic-tac-toe game. Gates' early exposure to computers at a ... From a small garage and dorm room startup to becoming the largest company in the world, Microsoft's journey highlights the power of vision and determination. Founded in 1975 by then-19-year-old Bill Gates and 22-year-old Paul Allen, who first famously met at Lakeside School in Seattle, the company arguably invented the software domain.", name='duckduckgo_search', id='16d4e0ff-1777-4a55-86f0-6dd0511af161', tool_call_id='0')]
data: {'retrieve': {'messages': [ToolMessage(content="Bill Gates. William Henry Gates III (born October 28, 1955) is an American businessman, investor, philanthropist, and writer best known for co-founding the software company Microsoft with his childhood friend Paul Allen. During his career at Microsoft, Gates held the positions of chairman, chief executive officer (CEO), president, and chief ... November 8, 2023. Bill Gates is one of the most iconic business leaders and influential technology pioneers of the modern digital era. As the co-founder of Microsoft, he helped bring the power of personal computing to millions of people worldwide and made computers an indispensable part of everyday life. This article explores Gates ... Microsoft is a multinational computer technology corporation. Microsoft was founded on April 4, 1975, by Bill Gates and Paul Allen in Albuquerque, New Mexico. Its current best-selling products are the Microsoft Windows operating system; Microsoft Office, a suite of productivity software; Xbox, a line of entertainment of games, music, and video; Bing, a line of search engines; and Microsoft ... The Founders of Microsoft. Bill Gates, born on October 28, 1955, in Seattle, Washington, exhibited a keen interest in computer programming from a young age. At just 13, he began exploring the world of programming at Lakeside School. There, he developed his first computer program—a tic-tac-toe game. Gates' early exposure to computers at a ... From a small garage and dorm room startup to becoming the largest company in the world, Microsoft's journey highlights the power of vision and determination. Founded in 1975 by then-19-year-old Bill Gates and 22-year-old Paul Allen, who first famously met at Lakeside School in Seattle, the company arguably invented the software domain.", name='duckduckgo_search', id='16d4e0ff-1777-4a55-86f0-6dd0511af161', tool_call_id='0')]}}

--- CALL generate ---
output:  Bill Gates was 20 years old when he co-founded Microsoft with Paul Allen in 1975. He was born on October 28, 1955, and Microsoft was founded on April 4, 1975. So, he was 20 years old at the time of the founding of Microsoft. "}}
                                                                                                                                                                                                                                                                                                                                                                                  data: [DONE]                           

[lkk12014402]

Hi, @lkk12014402, there is another comment from @XuhuiRen to align new SPEC API with you.

He told me that you're modifying all current LLM API to align with openAI. Currently, I am using LLMParamDocs, so I guess you will update LLMParamDocs with openAI compatible API, I would like to submit a new PR for UT after your API update completed.

Is it OK that we leave PR using current LLMParamDocs?

yes, no problem. the pr of openAI format haven't been merged now. let's merge your pr first.