Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[compute-ai-embeddings] Rework support for Hugging Face local models for embeddings #584

Merged
merged 5 commits into from
Oct 13, 2023

Conversation

eolivelli
Copy link
Member

@eolivelli eolivelli commented Oct 13, 2023

Summary:

  • Change Hugging face "modelUrl" parameter to "model-url"
  • Use the "provider" value from the Hugging Face resource instead of picking it from the "agent" (now you can have multiple HF resources), provider can be "api" or "local"
  • Add some tweaks to make HF local computation work again (it used to work but after a couple of refactors we broke the configuration)
  • Add one test that uses HF locally
  • Add a sample application that uses an HF embedding model (intfloat/multilingual-e5-small)
  • Update the example secrets file to allow to configure HF provider and models with ENV variables (at least for the example application)
  • Upgrade DLJ an PyTorch (not strictly needed but as we are touching this part I preferred to pick up the latest and greatest)

Please note that the integration tests shows that when the test is running on CI the precision of the embeddings seems "better", at least it emits one more digit

@eolivelli eolivelli merged commit e3b1567 into main Oct 13, 2023
9 checks passed
@eolivelli eolivelli deleted the impl/hf-local branch October 13, 2023 21:45
benfrank241 pushed a commit to vectorize-io/langstream that referenced this pull request May 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

1 participant