01.ai Yi Client #3048
01.ai Yi Client #3048
Conversation
|
@microsoft-github-policy-service agree |
| api_key | ||
| ), "Please include the api_key in your config list entry for Yi or set the YI_API_KEY env variable." | ||
|
|
||
| self._oai_client = OpenAI(api_key=api_key, base_url=base_url) |
There was a problem hiding this comment.
I don't understand, why are we initializing an OpenAI client again? can you please explain this.
There was a problem hiding this comment.
According to the official documentation, 01.AI provides an HTTP request interface highly compatible with OpenAI API, but no SDKs for Python or other languages have been found so far.
I'm undecided, so I'll first use the method of the OpenAI API client that I usually use. Other possible implementations include:
- Using requests
- Inheriting the autogen.oai.OpenAIClient
Any suggestions?
There was a problem hiding this comment.
i think this looks good. please let us know the status of the tests using the model. a notebook/ tutorial on this is appreciated. thanks @uukuguy
There was a problem hiding this comment.
Is it possible just to use base_url and api_key on a standard config and use the normal OpenAI client rather than use a specific class for Yi.
There was a problem hiding this comment.
I also agree with directly using the OpenAI client to access the Yi API. The only major difference is that the Yi API can only use some of the parameters from the OpenAI API. So, still need Yi client?
There was a problem hiding this comment.
Yep, that's a good point and I think it's useful to be able to handle client specific parameters. Okay, we move forward with testing then.
@qingyun-wu or @sonichi, let us know if you think this okay to proceed with or if there's another approach.
|
|
||
| # Cost per thousand tokens - Input / Output (NOTE: Convert $/Million to $/K) | ||
| YI_PRICING_1K = { | ||
| "yi-large": (0.003, 0.003), |
There was a problem hiding this comment.
are these models not open sourced? do they have a separate client that could be accessed with the OpenAI python SDK and API key?
There was a problem hiding this comment.
According to the official documentation, 01.AI provides an HTTP request interface highly compatible with OpenAI API, but no SDKs for Python or other languages have been found so far.
There was a problem hiding this comment.
According to the official documentation, 01.AI provides an HTTP request interface highly compatible with OpenAI API, but no SDKs for Python or other languages have been found so far.
How should we run and test the client? Can you provide a code sample?
There was a problem hiding this comment.
# You can get an account with a $5 free credit at https://platform.01.ai/apikeys to create your own YI_API_KEY.
# Use openai client
import os
from openai import OpenAI
client = OpenAI(api_key=os.getenv("YI_API_KEY"), base_url=os.getenv("YI_BASE_URL", "https://api.01.ai/v1"))
# Use autogen.oai client
from autogen.oai.yi import YiClient
client = YiClient(api_key=os.getenv("YI_API_KEY"))There was a problem hiding this comment.
Thanks, are you able to test it with these sample autogen scripts?
https://github.com/marklysze/AutoGenCodeTesting
I use them when developing client classes.
|
Hey @uukuguy, thanks so much for putting this together. In line with @Hk669's questions, can you describe how you use this client? I can see there's an official Yi API but it's limited to Beta Testers - is this designed to use that? I'd love to test it but I'm not sure how to get started and whether I need an account somewhere. |
01.AI's LLM API is located at https://platform.01.ai/, currently featuring only the yi-large model, with plans to deliver multiple models such as yi-large-turbo, yi-medium, and yi-vision. The Yi series of open-source models can be found at https://huggingface.co/01-ai, including several popular models like Yi-1.5-34B. According to the official documentation, 01.AI provides an HTTP request interface highly compatible with OpenAI API, but no SDKs for Python or other languages have been found so far. |
|
Why are these changes needed?
To enhance the support of non-OpenAI models with AutoGen. The 01.ai Yi family of models includes yi-large etc.
Developers can utilize the AI capabilities of the Yi series LLMs to perform a variety of tasks, such as text generation, language translation, content summarization, logical reasoning, mathematical calculation, and code generation.
Checks