Skip to content

Commit

Permalink
支持 searx 搜索引擎 (#4476)
Browse files Browse the repository at this point in the history 
- 支持 searx 搜索引擎
- /knowledge_base/chat/completions 接口改为 /knowledge_base/{mode}/{param}/chat/completions 形式，更好的支持 openai 兼容的前端框架
- 修复一些bug
- tool_settings.yaml 中添加工具配置说明
  • Loading branch information
@liunux4odoo
liunux4odoo authored Jul 11, 2024
1 parent 40382f5 commit 6bf47c5
Show file tree
Hide file tree
Showing 6 changed files with 160 additions and 67 deletions.
58 changes: 55 additions & 3 deletions docs/contributing/api.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -223,7 +223,7 @@

- 本地知识库问答
```python3
base_url = "http://127.0.0.1:7861/knowledge_base"
base_url = "http://127.0.0.1:7861/knowledge_base/local_kb/samples"
data = {
"model": "qwen2-instruct",
"messages": [
Expand All @@ -234,8 +234,6 @@
"stream": True,
"temperature": 0.7,
"extra_body": {
"mode": "local_kb",
"kb_name": "samples",
"top_k": 3,
"score_threshold": 2.0,
"return_direct": True,
Expand All @@ -253,3 +251,57 @@
```shell
ChatCompletionChunk(id='chat9973e445-8581-45ca-bde5-148fc724b30b', choices=[Choice(delta=None, finish_reason=None, index=None, logprobs=None, message={'role': 'assistant', 'content': '', 'finish_reason': 'stop', 'tool_calls': []})], created=1720592802, model=None, object='chat.completion', service_tier=None, system_fingerprint=None, usage=None, status=None, message_type=1, message_id=None, is_ref=False, docs=['出处 [1] [test_files/test.txt](http://127.0.0.1:7861//knowledge_base/download_doc?knowledge_base_name=samples&file_name=test_files%2Ftest.txt) \n\n[这就是那幅名画]: http://yesaiwen.com/art\nof\nasking\nchatgpt\nfor\nhigh\nquality\nansw\nengineering\ntechniques/#i\n3\t"《如何向ChatGPT提问并获得高质量的答案》"\n\n', '出处 [2] [test_files/test.txt](http://127.0.0.1:7861//knowledge_base/download_doc?knowledge_base_name=samples&file_name=test_files%2Ftest.txt) \n\nChatGPT是OpenAI开发的一个大型语言模型，可以提供各种主题的信息，\n# 如何向 ChatGPT 提问以获得高质量答案：提示技巧工程完全指南\n## 介绍\n我很高兴欢迎您阅读我的最新书籍《The Art of Asking ChatGPT for High-Quality Answers: A complete Guide to Prompt Engineering Techniques》。本书是一本全面指南，介绍了各种提示技术，用于从ChatGPT中生成高质量的答案。\n我们将探讨如何使用不同的提示工 程技术来实现不同的目标。ChatGPT是一款最先进的语言模型，能够生成类似人类的文本。然而，理解如何正确地向ChatGPT提问以获得我们所需的高质量输出非常重要。而这正是 本书的目的。\n无论您是普通人、研究人员、开发人员，还是只是想在自己的领域中将ChatGPT作为个人助手的人，本书都是为您编写的。我使用简单易懂的语言，提供实用的解释，并在每个提示技术中提供了示例和提示公式。通过本书，您将学习如何使用提示工程技术来控制ChatGPT的输出，并生成符合您特定需求的文本。\n在整本书中，我们还提供了如何结合不同的提示技术以实现更具体结果的示例。我希望您能像我写作时一样，享受阅读本书并从中获得知识。\n<div style="page\nbreak\nafter:always;"></div>\n## 第一章：Prompt 工程技术简介\n什么是 Prompt 工程？\nPrompt 工程是创建提示或指导像 ChatGPT 这样的语言模型输出的过程。它允许用户控制模型的输出并生成符合其特定需求的文本。\n\n', '出处 [3] [test_files/test.txt](http://127.0.0.1:7861//knowledge_base/download_doc?knowledge_base_name=samples&file_name=test_files%2Ftest.txt) \n\nPrompt 公式是提示的特定格式，通常由三个主要元素组成：**\n任务：对提示要求模型生成的内容进行清晰而简洁的陈述。\n指令：在生成文本时模型应遵循的指令。\n角色：模型在生成文本时应扮演的角色。\n在本书中，我们将探讨可用于 ChatGPT 的各种 Prompt 工程技术。我们将讨论不同类型的提示，以及如何使用它们实现您想要的特定目标。\n<div style="page\nbreak\nafter:always;"></div>\n## 第二章：指令提示技术\n现在，让我们开始探索“指令提示技术”，以及如何使用它从ChatGPT中生成高质量的文本。\n 指令提示技术是通过为模型提供具体指令来引导ChatGPT的输出的一种方法。这种技术对于确保输出相关和高质量非常有用。\n要使用指令提示技术，您需要为模型提供清晰简洁的任务，以及具体的指令以供模型遵循。\n例如，如果您正在生成客户服务响应，您将提供任务，例如“生成响应客户查询”的指令，例如“响应应该专业且提供准确的信息”。\n 提示公式：“按照以下指示生成[任务]：[指令]”\n示例：\n生成客户服务响应：**\n任务：生成响应客户查询\n指令：响应应该专业且提供准确的信息\n提示公式：“按照以下 指示生成专业且准确的客户查询响应：响应应该专业且提供准确的信息。”\n生成法律文件：**\n任务：生成法律文件\n指令：文件应符合相关法律法规\n提示公式：“按照以下 指示生成符合相关法律法规的法律文件：文件应符合相关法律法规。”\n使用指令提示技术时，重要的是要记住指令应该清晰具体。这将有助于确保输出相关和高质量。可以将指 令提示技术与下一章节中解释的“角色提示”和“种子词提示”相结合，以增强ChatGPT的输出。\n\n'])
```

- 文件对话
```python3
# knowledge_id 为 /knowledge_base/upload_temp_docs 的返回值
base_url = "http://127.0.0.1:7861/knowledge_base/temp_kb/{knowledge_id}"
data = {
"model": "qwen2-instruct",
"messages": [
{"role": "user", "content": "你好"},
{"role": "assistant", "content": "你好，我是人工智能大模型"},
{"role": "user", "content": "如何高质量提问？"},
],
"stream": True,
"temperature": 0.7,
"extra_body": {
"top_k": 3,
"score_threshold": 2.0,
"return_direct": True,
},
}

import openai
client = openai.Client(base_url=base_url, api_key="EMPTY")
resp = client.chat.completions.create(**data)
for r in resp:
print(r)
```

- 搜索引擎问答
```python3
engine_name = "bing" # 可选值：bing, duckduckgo, metaphor, searx
base_url = f"http://127.0.0.1:7861/knowledge_base/search_engine/{engine_name}"
data = {
"model": "qwen2-instruct",
"messages": [
{"role": "user", "content": "你好"},
{"role": "assistant", "content": "你好，我是人工智能大模型"},
{"role": "user", "content": "如何高质量提问？"},
],
"stream": True,
"temperature": 0.7,
"extra_body": {
"top_k": 3,
"score_threshold": 2.0,
"return_direct": True,
},
}

import openai
client = openai.Client(base_url=base_url, api_key="EMPTY")
resp = client.chat.completions.create(**data)
for r in resp:
print(r)
```
16 changes: 14 additions & 2 deletions libs/chatchat-server/chatchat/server/agent/tools_factory/search_internet.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,26 @@
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.utilities.bing_search import BingSearchAPIWrapper
from langchain.utilities.duckduckgo_search import DuckDuckGoSearchAPIWrapper
from langchain.utilities.searx_search import SearxSearchWrapper
from markdownify import markdownify
from strsimpy.normalized_levenshtein import NormalizedLevenshtein

from chatchat.settings import Settings
from chatchat.server.pydantic_v1 import Field
from chatchat.server.utils import get_tool_config

from .tools_registry import BaseToolOutput, regist_tool, format_context


def searx_search(text ,config, top_k: int):
search = SearxSearchWrapper(
searx_host=config["host"],
engines=config["engines"],
categories=config["categories"],
)
return search.results(text, top_k)


def bing_search(text, config, top_k:int):
search = BingSearchAPIWrapper(
bing_subscription_key=config["bing_key"],
Expand Down Expand Up @@ -76,6 +87,7 @@ def metaphor_search(
"bing": bing_search,
"duckduckgo": duckduckgo_search,
"metaphor": metaphor_search,
"searx": searx_search,
}


Expand All @@ -96,7 +108,7 @@ def search_result2docs(search_results) -> List[Document]:
def search_engine(query: str, top_k:int=0, engine_name: str="", config: dict={}):
config = config or get_tool_config("search_internet")
if top_k <= 0:
top_k = config.get("top_k", 3)
top_k = config.get("top_k", Settings.kb_settings.SEARCH_ENGINE_TOP_K)
engine_name = engine_name or config.get("search_engine_name")
search_engine_use = SEARCH_ENGINES[engine_name]
results = search_engine_use(
Expand All @@ -109,4 +121,4 @@ def search_engine(query: str, top_k:int=0, engine_name: str="", config: dict={})
@regist_tool(title="互联网搜索")
def search_internet(query: str = Field(description="query for Internet search")):
"""Use this tool to use bing search engine to search the internet and get information."""
return BaseToolOutput(search_engine(query=query, config=tool_config), format=format_context)
return BaseToolOutput(search_engine(query=query), format=format_context)
15 changes: 10 additions & 5 deletions libs/chatchat-server/chatchat/server/api_server/kb_routes.py
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
from __future__ import annotations

from typing import List
from typing import List, Literal

from fastapi import APIRouter, Request

Expand Down Expand Up @@ -33,17 +33,22 @@


@kb_router.post(
"/chat/completions", summary="知识库对话，openai 兼容，参数与 /chat/kb_chat 一致"
"/{mode}/{param}/chat/completions", summary="知识库对话，openai 兼容，参数与 /chat/kb_chat 一致"
)
async def kb_chat_endpoint(body: OpenAIChatInput, request: Request):
async def kb_chat_endpoint(
mode: Literal["local_kb", "temp_kb", "search_engine"],
param: str,
body: OpenAIChatInput,
request: Request,
):
# import rich
# rich.print(body)

extra = body.model_extra
ret = await kb_chat(
query=body.messages[-1]["content"],
mode=extra.get("mode", "local_kb"),
kb_name=extra.get("kb_name"),
mode=mode,
kb_name=param,
top_k=extra.get("top_k", Settings.kb_settings.VECTOR_SEARCH_TOP_K),
score_threshold=extra.get("score_threshold", Settings.kb_settings.SCORE_THRESHOLD),
history=body.messages[:-1],
Expand Down
14 changes: 6 additions & 8 deletions libs/chatchat-server/chatchat/server/chat/kb_chat.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,7 @@

async def kb_chat(query: str = Body(..., description="用户输入", examples=["你好"]),
mode: Literal["local_kb", "temp_kb", "search_engine"] = Body("local_kb", description="知识来源"),
kb_name: str = Body("", description="知识库名称", examples=["samples"]),
knowledge_id: str = Body("", description="临时知识库ID"),
search_engine_name: str = Body("", description="搜索引擎"),
kb_name: str = Body("", description="mode=local_kb时为知识库名称；temp_kb时为临时知识库ID，search_engine时为搜索引擎名称", examples=["samples"]),
top_k: int = Body(Settings.kb_settings.VECTOR_SEARCH_TOP_K, description="匹配向量数"),
score_threshold: float = Body(
Settings.kb_settings.SCORE_THRESHOLD,
Expand Down Expand Up @@ -80,15 +78,15 @@ async def knowledge_base_chat_iterator() -> AsyncIterable[str]:
source_documents = format_reference(kb_name, docs, request.base_url)
elif mode == "temp_kb":
docs = await run_in_threadpool(search_temp_docs,
knowledge_id,
kb_name,
query=query,
top_k=top_k,
score_threshold=score_threshold)
source_documents = format_reference(knowledge_id, docs, request.base_url)
source_documents = format_reference(kb_name, docs, request.base_url)
elif mode == "search_engine":
docs = await run_in_threadpool(search_engine,query, top_k, search_engine_name)
docs = [x.dict() for x in docs]
source_documents = [f"""出处 [{i + 1}] [{d['filename']}]({d['source']}) \n\n{d['page_content']}\n\n""" for i,d in enumerate(docs)]
result = await run_in_threadpool(search_engine, query, top_k, kb_name)
docs = [x.dict() for x in result.get("docs", [])]
source_documents = [f"""出处 [{i + 1}] [{d['metadata']['filename']}]({d['metadata']['source']}) \n\n{d['page_content']}\n\n""" for i,d in enumerate(docs)]
else:
docs = []
source_documents = []
Expand Down
Loading

0 comments on commit 6bf47c5

Please sign in to comment.