Skip to content

Commit

Permalink
fix/docs: Python backend dependencies (#1360)
Browse files Browse the repository at this point in the history
* Update docs for new requirements.txt path

Signed-off-by: Marcus Köhler <khler.marcus@gmail.com>

* Fix typo (.PONY -> .PHONY) in python backend makefiles

Signed-off-by: Marcus Köhler <khler.marcus@gmail.com>

---------

Signed-off-by: Marcus Köhler <khler.marcus@gmail.com>
  • Loading branch information
B4ckslash authored Nov 30, 2023
1 parent 9b98be1 commit 2d64d8b
Show file tree
Hide file tree
Showing 10 changed files with 30 additions and 27 deletions.
2 changes: 1 addition & 1 deletion backend/python/autogptq/Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PONY: autogptq
.PHONY: autogptq
autogptq:
@echo "Creating virtual environment..."
@conda env create --name autogptq --file autogptq.yml
Expand Down
4 changes: 2 additions & 2 deletions backend/python/bark/Makefile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
.PONY: ttsbark
.PHONY: ttsbark
ttsbark:
@echo "Creating virtual environment..."
@conda env create --name ttsbark --file ttsbark.yml
@echo "Virtual environment created."
@echo "Virtual environment created."
6 changes: 3 additions & 3 deletions backend/python/diffusers/Makefile
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
.PONY: diffusers
.PHONY: diffusers
diffusers:
@echo "Creating virtual environment..."
@conda env create --name diffusers --file diffusers.yml
@echo "Virtual environment created."

.PONY: run
.PHONY: run
run:
@echo "Running diffusers..."
bash run.sh
@echo "Diffusers run."
@echo "Diffusers run."
6 changes: 3 additions & 3 deletions backend/python/exllama/Makefile
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
.PONY: exllama
.PHONY: exllama
exllama:
@echo "Creating virtual environment..."
@conda env create --name exllama --file exllama.yml
@echo "Virtual environment created."

.PONY: run
.PHONY: run
run:
@echo "Running exllama..."
bash run.sh
@echo "exllama run."
@echo "exllama run."
6 changes: 3 additions & 3 deletions backend/python/petals/Makefile
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
.PONY: petals
.PHONY: petals
petals:
@echo "Creating virtual environment..."
@conda env create --name petals --file petals.yml
@echo "Virtual environment created."

.PONY: run
.PHONY: run
run:
@echo "Running petals..."
bash run.sh
@echo "petals run."
@echo "petals run."
8 changes: 4 additions & 4 deletions backend/python/sentencetransformers/Makefile
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
.PONY: sentencetransformers
.PHONY: sentencetransformers
sentencetransformers:
@echo "Creating virtual environment..."
@conda env create --name sentencetransformers --file sentencetransformers.yml
@echo "Virtual environment created."

.PONY: run
.PHONY: run
run:
@echo "Running sentencetransformers..."
bash run.sh
@echo "sentencetransformers run."

# It is not working well by using command line. It only6 works with IDE like VSCode.
.PONY: test
.PHONY: test
test:
@echo "Testing sentencetransformers..."
bash test.sh
@echo "sentencetransformers tested."
@echo "sentencetransformers tested."
8 changes: 4 additions & 4 deletions backend/python/transformers/Makefile
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
.PONY: transformers
.PHONY: transformers
transformers:
@echo "Creating virtual environment..."
@conda env create --name transformers --file transformers.yml
@echo "Virtual environment created."

.PONY: run
.PHONY: run
run:
@echo "Running transformers..."
bash run.sh
@echo "transformers run."

# It is not working well by using command line. It only6 works with IDE like VSCode.
.PONY: test
.PHONY: test
test:
@echo "Testing transformers..."
bash test.sh
@echo "transformers tested."
@echo "transformers tested."
6 changes: 3 additions & 3 deletions backend/python/vall-e-x/Makefile
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
.PONY: ttsvalle
.PHONY: ttsvalle
ttsvalle:
@echo "Creating virtual environment..."
@conda env create --name ttsvalle --file ttsvalle.yml
@echo "Virtual environment created."

.PONY: run
.PHONY: run
run:
@echo "Running ttsvalle..."
bash run.sh
@echo "ttsvalle run."
@echo "ttsvalle run."
6 changes: 3 additions & 3 deletions backend/python/vllm/Makefile
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
.PONY: vllm
.PHONY: vllm
vllm:
@echo "Creating virtual environment..."
@conda env create --name vllm --file vllm.yml
@echo "Virtual environment created."

.PONY: run
.PHONY: run
run:
@echo "Running vllm..."
bash run.sh
@echo "vllm run."
@echo "vllm run."
5 changes: 4 additions & 1 deletion docs/content/features/embeddings.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,10 @@ The `sentencetransformers` backend uses Python [sentence-transformers](https://g

{{% notice note %}}

- The `sentencetransformers` backend is an optional backend of LocalAI and uses Python. If you are running `LocalAI` from the containers you are good to go and should be already configured for use. If you are running `LocalAI` manually you must install the python dependencies (`pip install -r /path/to/LocalAI/extra/requirements`) and specify the extra backend in the `EXTERNAL_GRPC_BACKENDS` environment variable ( `EXTERNAL_GRPC_BACKENDS="sentencetransformers:/path/to/LocalAI/backend/python/sentencetransformers/sentencetransformers.py"` ) .
- The `sentencetransformers` backend is an optional backend of LocalAI and uses Python. If you are running `LocalAI` from the containers you are good to go and should be already configured for use.
- If you are running `LocalAI` manually you must install the python dependencies (`make prepare-extra-conda-environments`). This requires `conda` to be installed.
- For local execution, you also have to specify the extra backend in the `EXTERNAL_GRPC_BACKENDS` environment variable.
- Example: `EXTERNAL_GRPC_BACKENDS="sentencetransformers:/path/to/LocalAI/backend/python/sentencetransformers/sentencetransformers.py"`
- The `sentencetransformers` backend does support only embeddings of text, and not of tokens. If you need to embed tokens you can use the `bert` backend or `llama.cpp`.
- No models are required to be downloaded before using the `sentencetransformers` backend. The models will be downloaded automatically the first time the API is used.

Expand Down

0 comments on commit 2d64d8b

Please sign in to comment.