fix readme and manifest

tomasliu-agora · Oct 15, 2024 · 1e36a3d · 1e36a3d
1 parent b0eb9bf
commit 1e36a3d
Show file tree

Hide file tree

Showing 6 changed files with 155 additions and 55 deletions.
diff --git a/agents/ten_packages/extension/azure_vision_python/README.md b/agents/ten_packages/extension/azure_vision_python/README.md
@@ -1,29 +1,44 @@
 # azure_vision_python
 
-<!-- brief introduction for the extension -->
+This is the extension calling azure ai vision.
 
-## Features
-
-<!-- main features introduction -->
+The document is as follow: https://learn.microsoft.com/zh-cn/azure/ai-services/computer-vision/overview
 
-- xxx feature
+## Properties
 
-## API
-
-Refer to `api` definition in [manifest.json] and default values in [property.json](property.json).
+- key 
+- endpoint
 
-<!-- Additional API.md can be referred to if extra introduction needed -->
-
-## Development
+## Features
 
-### Build
+- Only support one frame of image
+- No customization for feature
+- By default will include `TAGS`, `CAPTION`, `READ`, `PEOPLE`, `OBJECTS`
 
-<!-- build dependencies and steps -->
+## API
 
-### Unit test
+Refer to `api` definition in [manifest.json] and default values in [property.json](property.json).
 
-<!-- how to do unit test for the extension -->
+Other extensions can call `analyze_image` cmd and will get all analyze result from result in `response` property, the result will looks like this:
+
+``` json
+{
+    "modelVersion": "2023-10-01",
+    "captionResult": {
+        "text": "a group of toys on a table",
+        "confidence": 0.7558467388153076
+    },
+    "metadata": {
+        "width": 320,
+        "height": 240
+    },
+    "objectsResult": {},
+    "readResult": {},
+    "peopleResult": {}
+}
+```
 
 ## Misc
 
-<!-- others if applicable -->
+- Video analyze
+- Multi-frame analyze
diff --git a/agents/ten_packages/extension/azure_vision_python/manifest.json b/agents/ten_packages/extension/azure_vision_python/manifest.json
@@ -19,5 +19,34 @@
       "README.md"
     ]
   },
-  "api": {}
+  "api": {
+    "property": {
+      "key": {
+        "type": "string"
+      },
+      "endpoint": {
+        "type": "string"
+      }
+    }
+  },
+  "cmd_in": [
+    {
+      "name": "analyze_image",
+      "property": {
+        "image_data": {
+          "type": "buf"
+        }
+      },
+      "required": [
+        "image_data"
+      ],
+      "result": {
+        "property": {
+          "response": {
+            "type": "string"
+          }
+        }
+      }
+    }
+  ]
 }
diff --git a/agents/ten_packages/extension/bingsearch_tool_python/README.md b/agents/ten_packages/extension/bingsearch_tool_python/README.md
@@ -1,29 +1,26 @@
 # bingsearch_tool_python
 
-<!-- brief introduction for the extension -->
+This is tool for bing search, the document link is as follow: https://learn.microsoft.com/en-us/bing/search-apis/bing-web-search/quickstarts/rest/python
+
+It is built using TEN Tool Call Protocol (Beta).
 
 ## Features
 
-<!-- main features introduction -->
+It is the bing search tool that will auto register to any llm extension.
+
+The tool description is as follow: 
 
-- xxx feature
+*Use Bing.com to search for latest information. Call this function if you are not sure about the answer.*
 
 ## API
 
 Refer to `api` definition in [manifest.json] and default values in [property.json](property.json).
 
-<!-- Additional API.md can be referred to if extra introduction needed -->
-
-## Development
-
-### Build
-
-<!-- build dependencies and steps -->
-
-### Unit test
-
-<!-- how to do unit test for the extension -->
+- out: tool_register
+- in: tool_call
 
 ## Misc
 
-<!-- others if applicable -->
+- use Tool Call Protocol Standard
+- support async call
+- apply asyncio template
diff --git a/agents/ten_packages/extension/bingsearch_tool_python/manifest.json b/agents/ten_packages/extension/bingsearch_tool_python/manifest.json
@@ -19,5 +19,55 @@
       "README.md"
     ]
   },
-  "api": {}
+  "api": {
+    "property": {
+      "api_key": {
+        "type": "string"
+      }
+    },
+    "cmd_out": [
+      {
+        "name": "tool_register",
+        "property": {
+          "name": {
+            "type": "string"
+          },
+          "description": {
+            "type": "string"
+          },
+          "parameters": {
+            "type": "string"
+          }
+        },
+        "required": [
+          "name",
+          "description",
+          "parameters"
+        ],
+        "result": {
+          "property": {
+            "response": {
+              "type": "string"
+            }
+          }
+        }
+      }
+    ],
+    "cmd_in": [
+      {
+        "name": "tool_call",
+        "property": {
+          "name": {
+            "type": "string"
+          },
+          "args": {
+            "type": "string"
+          }
+        },
+        "required": [
+          "name"
+        ]
+      }
+    ]
+  }
 }
diff --git a/agents/ten_packages/extension/vision_tool_python/README.md b/agents/ten_packages/extension/vision_tool_python/README.md
@@ -1,29 +1,36 @@
 # vision_tool_python
 
-<!-- brief introduction for the extension -->
+This is tool for vision ability, currently there are two patterns:
+- use triditional model
+- use multimodal llm model
 
-## Features
+The pattern can be switched by `use_llm` pattern to use different cmd protocol.
 
-<!-- main features introduction -->
+Tool description is as follow:
 
-- xxx feature
+*Query to the latest frame from camera. The camera is always on, always use latest frame to answer user's question. Call this whenever you need to understand the input camera image like you have vision capability, for example when user asks 'What can you see?', 'Can you see me?', 'take a look.'*
 
-## API
+It is built using TEN Tool Call Protocol (Beta).
 
-Refer to `api` definition in [manifest.json] and default values in [property.json](property.json).
+## Features
 
-<!-- Additional API.md can be referred to if extra introduction needed -->
+The tool can accept video frame from rtc extension.
 
-## Development
+The tool will only register itself to llm extension as soon as the video frame is received.
 
-### Build
+The tool will cache video frame every `frequency_ms` ms.
 
-<!-- build dependencies and steps -->
+## API
 
-### Unit test
+Refer to `api` definition in [manifest.json] and default values in [property.json](property.json).
 
-<!-- how to do unit test for the extension -->
+- out: `tool_register`
+- in: `tool_call`
+- out(`use_llm=false`): `analyze_image`
+- out(`use_llm=true`): `chat_completion`
 
 ## Misc
 
-<!-- others if applicable -->
+- Multi-frame support
+- Movement detection
+- Prompt Engineering
diff --git a/agents/ten_packages/extension/weatherapi_tool_python/README.md b/agents/ten_packages/extension/weatherapi_tool_python/README.md
@@ -1,21 +1,23 @@
 # weatherapi_tool_python
 
-This is the tool demo for weather query.
+This is the tool for weather query, including current weather, broadcast and history weather check, the document link is as follow: https://www.weatherapi.com/docs/
+
+It is built using TEN Tool Call Protocol (Beta).
 
 ## Features
 
+For free plan:
 - Fetch today's weather.
-- Search for history weather.
+- Search for history weather within 7 days.
 - Forcast weather in 3 days.
 
-## API
-
-Refer to `api` definition in [manifest.json] and default values in [property.json](property.json).
+You can extend by using other plan in your project.
 
-### Out:
+https://www.weatherapi.com/pricing.aspx
 
-- `tool_register`: auto register tool to llm
+## API
 
-### In:
+Refer to `api` definition in [manifest.json] and default values in [property.json](property.json).
 
-- `tool_call`: sync cmd to fetch weather
+- out: tool_register
+- in: tool_call