Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix/docs: Python backend dependencies #1360

Merged
merged 2 commits into from
Nov 30, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion backend/python/autogptq/Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PONY: autogptq
.PHONY: autogptq
autogptq:
@echo "Creating virtual environment..."
@conda env create --name autogptq --file autogptq.yml
Expand Down
4 changes: 2 additions & 2 deletions backend/python/bark/Makefile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
.PONY: ttsbark
.PHONY: ttsbark
ttsbark:
@echo "Creating virtual environment..."
@conda env create --name ttsbark --file ttsbark.yml
@echo "Virtual environment created."
@echo "Virtual environment created."
6 changes: 3 additions & 3 deletions backend/python/diffusers/Makefile
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
.PONY: diffusers
.PHONY: diffusers
diffusers:
@echo "Creating virtual environment..."
@conda env create --name diffusers --file diffusers.yml
@echo "Virtual environment created."

.PONY: run
.PHONY: run
run:
@echo "Running diffusers..."
bash run.sh
@echo "Diffusers run."
@echo "Diffusers run."
6 changes: 3 additions & 3 deletions backend/python/exllama/Makefile
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
.PONY: exllama
.PHONY: exllama
exllama:
@echo "Creating virtual environment..."
@conda env create --name exllama --file exllama.yml
@echo "Virtual environment created."

.PONY: run
.PHONY: run
run:
@echo "Running exllama..."
bash run.sh
@echo "exllama run."
@echo "exllama run."
6 changes: 3 additions & 3 deletions backend/python/petals/Makefile
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
.PONY: petals
.PHONY: petals
petals:
@echo "Creating virtual environment..."
@conda env create --name petals --file petals.yml
@echo "Virtual environment created."

.PONY: run
.PHONY: run
run:
@echo "Running petals..."
bash run.sh
@echo "petals run."
@echo "petals run."
8 changes: 4 additions & 4 deletions backend/python/sentencetransformers/Makefile
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
.PONY: sentencetransformers
.PHONY: sentencetransformers
sentencetransformers:
@echo "Creating virtual environment..."
@conda env create --name sentencetransformers --file sentencetransformers.yml
@echo "Virtual environment created."

.PONY: run
.PHONY: run
run:
@echo "Running sentencetransformers..."
bash run.sh
@echo "sentencetransformers run."

# It is not working well by using command line. It only6 works with IDE like VSCode.
.PONY: test
.PHONY: test
test:
@echo "Testing sentencetransformers..."
bash test.sh
@echo "sentencetransformers tested."
@echo "sentencetransformers tested."
8 changes: 4 additions & 4 deletions backend/python/transformers/Makefile
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
.PONY: transformers
.PHONY: transformers
transformers:
@echo "Creating virtual environment..."
@conda env create --name transformers --file transformers.yml
@echo "Virtual environment created."

.PONY: run
.PHONY: run
run:
@echo "Running transformers..."
bash run.sh
@echo "transformers run."

# It is not working well by using command line. It only6 works with IDE like VSCode.
.PONY: test
.PHONY: test
test:
@echo "Testing transformers..."
bash test.sh
@echo "transformers tested."
@echo "transformers tested."
6 changes: 3 additions & 3 deletions backend/python/vall-e-x/Makefile
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
.PONY: ttsvalle
.PHONY: ttsvalle
ttsvalle:
@echo "Creating virtual environment..."
@conda env create --name ttsvalle --file ttsvalle.yml
@echo "Virtual environment created."

.PONY: run
.PHONY: run
run:
@echo "Running ttsvalle..."
bash run.sh
@echo "ttsvalle run."
@echo "ttsvalle run."
6 changes: 3 additions & 3 deletions backend/python/vllm/Makefile
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
.PONY: vllm
.PHONY: vllm
vllm:
@echo "Creating virtual environment..."
@conda env create --name vllm --file vllm.yml
@echo "Virtual environment created."

.PONY: run
.PHONY: run
run:
@echo "Running vllm..."
bash run.sh
@echo "vllm run."
@echo "vllm run."
5 changes: 4 additions & 1 deletion docs/content/features/embeddings.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,10 @@ The `sentencetransformers` backend uses Python [sentence-transformers](https://g

{{% notice note %}}

- The `sentencetransformers` backend is an optional backend of LocalAI and uses Python. If you are running `LocalAI` from the containers you are good to go and should be already configured for use. If you are running `LocalAI` manually you must install the python dependencies (`pip install -r /path/to/LocalAI/extra/requirements`) and specify the extra backend in the `EXTERNAL_GRPC_BACKENDS` environment variable ( `EXTERNAL_GRPC_BACKENDS="sentencetransformers:/path/to/LocalAI/backend/python/sentencetransformers/sentencetransformers.py"` ) .
- The `sentencetransformers` backend is an optional backend of LocalAI and uses Python. If you are running `LocalAI` from the containers you are good to go and should be already configured for use.
- If you are running `LocalAI` manually you must install the python dependencies (`make prepare-extra-conda-environments`). This requires `conda` to be installed.
- For local execution, you also have to specify the extra backend in the `EXTERNAL_GRPC_BACKENDS` environment variable.
- Example: `EXTERNAL_GRPC_BACKENDS="sentencetransformers:/path/to/LocalAI/backend/python/sentencetransformers/sentencetransformers.py"`
- The `sentencetransformers` backend does support only embeddings of text, and not of tokens. If you need to embed tokens you can use the `bert` backend or `llama.cpp`.
- No models are required to be downloaded before using the `sentencetransformers` backend. The models will be downloaded automatically the first time the API is used.

Expand Down