Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update Embedding Providers Config #1048

Merged
merged 15 commits into from
May 2, 2024
Merged

Update Embedding Providers Config #1048

merged 15 commits into from
May 2, 2024

Conversation

Hazel-Datastax
Copy link
Contributor

@Hazel-Datastax Hazel-Datastax commented Apr 30, 2024 •
edited
Loading

What this PR does:

  1. Change the config yaml file according to the recent design changes
    1. Change supported-authentication from list to map
    2. vector-dimension can be null and the value could be in parameters
    3. Remove api-key
    4. Lowercase parameter type: STRING -> string
    5. Add more embedding providers (only in config yaml) - azureOpenAI, jinaAI, voyageAI, mistral

TODO in the following PR:

  1. Changes in config file break most verification code in CreateCollectionCommandResolver, fix them in the following PR.
  2. voyageAI properties config in yaml - rethink the design of RequestProperties - more properties in the future?
  3. Fix the uppercase NUMERIC_RANGE in the findEmbeddingProviders return body - cause by enum

Question from #1051

  1. Change stargate.jsonapi to datastax.dataapi?
  2. authentication type should be uppercase or lowercase?

Which issue(s) this PR fixes:
Fixes Jira C2-3488

Checklist

  • Changes manually tested
  • Automated Tests added/updated
  • Documentation added/updated
  • CLA Signed: DataStax CLA

@Hazel-Datastax Hazel-Datastax requested a review from a team as a code owner April 30, 2024 16:52
@Hazel-Datastax Hazel-Datastax changed the title Update findEmbeddingProviders for supported authentication fields [WIP] Update findEmbeddingProviders for supported authentication fields Apr 30, 2024
@Hazel-Datastax Hazel-Datastax changed the base branch from main to feature/embedding-gateway May 1, 2024 16:11
@Hazel-Datastax Hazel-Datastax changed the title [WIP] Update findEmbeddingProviders for supported authentication fields [WIP] Update Embedding Providers Config May 1, 2024
Base automatically changed from feature/embedding-gateway to main May 1, 2024 21:47
@Hazel-Datastax Hazel-Datastax changed the title [WIP] Update Embedding Providers Config Update Embedding Providers Config May 2, 2024
@Hazel-Datastax Hazel-Datastax merged commit 6accbcb into main May 2, 2024
3 checks passed
@Hazel-Datastax Hazel-Datastax deleted the hazel/update_auth branch May 2, 2024 22:43
@Hazel-Datastax Hazel-Datastax mentioned this pull request May 3, 2024
Closed
4 tasks
vkarpov15
vkarpov15 reviewed May 6, 2024
View reviewed changes
Copy link
Collaborator

@vkarpov15 vkarpov15 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vkarpov15 vkarpov15 vkarpov15 left review comments

@tatu-at-datastax tatu-at-datastax tatu-at-datastax approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@Hazel-Datastax @vkarpov15 @tatu-at-datastax