Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add listing llm models and embedding models for Azure endpoint #1846

Merged
merged 6 commits into from
Oct 8, 2024

Conversation

mattzh72
Copy link
Collaborator

@mattzh72 mattzh72 commented Oct 8, 2024

Description

Add listing llm models and embedding models for Azure endpoint. We also make some changes to the api_version field for Azure:

  • we remove it from LLMConfig as it is a field only necessary for the Azure endpoint
  • we make it an optional field for the model settings
  • if the env variable is missing, we default to the latest api version (hardcoded)

Testing

Listed out LLMs:

[LLMConfig(model='gpt-35-turbo-0301', model_endpoint_type='azure', model_endpoint='https://letta.openai.azure.com/openai/deployments/gpt-35-turbo-0301/chat/completions?api-version=2024-09-01-preview', model_wrapper=None, context_window=2048), LLMConfig(model='gpt-35-turbo-0613', model_endpoint_type='azure', model_endpoint='https://letta.openai.azure.com/openai/deployments/gpt-35-turbo-0613/chat/completions?api-version=2024-09-01-preview', model_wrapper=None, context_window=2048), LLMConfig(model='gpt-35-turbo-1106', model_endpoint_type='azure', model_endpoint='https://letta.openai.azure.com/openai/deployments/gpt-35-turbo-1106/chat/completions?api-version=2024-09-01-preview', model_wrapper=None, context_window=2048), LLMConfig(model='gpt-35-turbo-0125', model_endpoint_type='azure', model_endpoint='https://letta.openai.azure.com/openai/deployments/gpt-35-turbo-0125/chat/completions?api-version=2024-09-01-preview', model_wrapper=None, context_window=2048), LLMConfig(model='gpt-35-turbo-16k-0613', model_endpoint_type='azure', model_endpoint='https://letta.openai.azure.com/openai/deployments/gpt-35-turbo-16k-0613/chat/completions?api-version=2024-09-01-preview', model_wrapper=None, context_window=2048), LLMConfig(model='gpt-4-0125-Preview', model_endpoint_type='azure', model_endpoint='https://letta.openai.azure.com/openai/deployments/gpt-4-0125-Preview/chat/completions?api-version=2024-09-01-preview', model_wrapper=None, context_window=2048)...

Listed out embeddings:

tests/test_providers.py::test_azure_openai [EmbeddingConfig(embedding_endpoint_type='azure', embedding_endpoint='https://letta.openai.azure.com/openai/deployments/text-embedding-ada-002/embeddings?api-version=2024-09-01-preview', embedding_model='text-embedding-ada-002', embedding_dim=768, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None)...

@mattzh72 mattzh72 changed the title Add listing llm models and embedding models for Azure endpoint feat: Add listing llm models and embedding models for Azure endpoint Oct 8, 2024
letta/providers.py Outdated Show resolved Hide resolved
tests/test_providers.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@sarahwooders sarahwooders left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@mattzh72 mattzh72 merged commit f61ac27 into main Oct 8, 2024
11 checks passed
mattzh72 added a commit that referenced this pull request Oct 9, 2024
…1846)

Co-authored-by: Matt Zhou <mattzhou@Matts-MacBook-Pro.local>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants