-
Notifications
You must be signed in to change notification settings - Fork 30
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[compute-ai-embeddings] Rework support for Hugging Face local models …
…for embeddings (#584)
- Loading branch information
Showing
12 changed files
with
268 additions
and
72 deletions.
There are no files selected for viewing
35 changes: 35 additions & 0 deletions
35
examples/applications/compute-hugging-face-embeddings/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
# Computing text embeddings with Hugging Face - local execution | ||
|
||
This sample application shows how to use Hugging Face to compute text embeddings without calling the API. | ||
|
||
## Configuring the model | ||
|
||
In order to use an open source model, you need to configure the model in the pipeline.yaml file. | ||
You can do it using environment variables or by editing the file directly. | ||
|
||
``` | ||
export HUGGING_FACE_PROVIDER=local | ||
export HUGGING_FACE_EMBEDDINGS_MODEL=multilingual-e5-small | ||
export HUGGING_FACE_EMBEDDINGS_MODEL_URL=djl://ai.djl.huggingface.pytorch/intfloat/multilingual-e5-small | ||
``` | ||
|
||
if you want to the API set HUGGING_FACE_PROVIDER to "api" and configure your API access key. | ||
``` | ||
export HUGGING_FACE_PROVIDER=api | ||
export HUGGING_FACE_ACCESS_KEY=your_access_key | ||
export HUGGING_FACE_EMBEDDINGS_MODEL=multilingual-e5-small | ||
``` | ||
|
||
## Deploy the LangStream application | ||
|
||
``` | ||
./bin/langstream apps deploy test -app examples/applications/compute-hugging-face-embeddings -i examples/instances/kafka-kubernetes.yaml -s examples/secrets/secrets.yaml | ||
``` | ||
|
||
## Talk with the Chat bot using the CLI | ||
Since the application opens a gateway, we can use the gateway API to send and consume messages. | ||
|
||
``` | ||
./bin/langstream gateway chat test -cg bot-output -pg user-input -p sessionId=$(uuidgen) | ||
``` | ||
|
24 changes: 24 additions & 0 deletions
24
examples/applications/compute-hugging-face-embeddings/configuration.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
# | ||
# | ||
# Copyright DataStax, Inc. | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
# | ||
|
||
configuration: | ||
resources: | ||
- type: "hugging-face-configuration" | ||
name: "Hugging Face AI configuration" | ||
configuration: | ||
access-key: "${ secrets.hugging-face.access-key }" | ||
provider: "${ secrets.hugging-face.provider }" |
38 changes: 38 additions & 0 deletions
38
examples/applications/compute-hugging-face-embeddings/gateways.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
# | ||
# | ||
# Copyright DataStax, Inc. | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
# | ||
|
||
gateways: | ||
- id: produce-input | ||
type: produce | ||
topic: input-topic | ||
parameters: | ||
- sessionId | ||
produce-options: | ||
headers: | ||
- key: langstream-client-session-id | ||
value-from-parameters: sessionId | ||
|
||
- id: consume-output | ||
type: consume | ||
topic: output-topic | ||
parameters: | ||
- sessionId | ||
consume-options: | ||
filters: | ||
headers: | ||
- key: langstream-client-session-id | ||
value-from-parameters: sessionId |
42 changes: 42 additions & 0 deletions
42
examples/applications/compute-hugging-face-embeddings/pipeline.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
# | ||
# Copyright DataStax, Inc. | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
# | ||
|
||
topics: | ||
- name: "input-topic" | ||
creation-mode: create-if-not-exists | ||
- name: "output-topic" | ||
creation-mode: create-if-not-exists | ||
errors: | ||
on-failure: "skip" | ||
pipeline: | ||
- name: "convert-to-structure" | ||
type: "document-to-json" | ||
input: "input-topic" | ||
configuration: | ||
text-field: "question" | ||
- name: "compute-embeddings" | ||
type: "compute-ai-embeddings" | ||
output: "output-topic" | ||
configuration: | ||
model: "${secrets.hugging-face.embeddings-model}" # This is the id of the model | ||
model-url: "${secrets.hugging-face.embeddings-model-url}" # This is the URL of the repository containing the model | ||
embeddings-field: "value.embeddings" | ||
text: "{{ value.question }}" | ||
batch-size: 10 | ||
# this is in milliseconds. It is important to take this value into consideration when using this agent in the chat response pipeline | ||
# in fact this value impacts the latency of the response | ||
# for latency sensitive applications, consider to set batch-size to 1 or flush-interval to 0 | ||
flush-interval: 500 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Binary file removed
BIN
-39.8 MB
langstream-agents/langstream-ai-agents/src/test/resources/ConGen-BERT-Mini.zip
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.