diff --git a/ChatQnA/README.md b/ChatQnA/README.md index 5d3f93e8f..0574a75d6 100644 --- a/ChatQnA/README.md +++ b/ChatQnA/README.md @@ -101,9 +101,13 @@ This ChatQnA use case performs RAG using LangChain, Redis VectorDB and Text Gene The ChatQnA service can be effortlessly deployed on either Intel Gaudi2 or Intel XEON Scalable Processors. +Two types of ChatQnA pipeline are supported now: `ChatQnA with/without Rerank`. And the `ChatQnA without Rerank` pipeline (including Embedding, Retrieval, and LLM) is offered for Xeon customers who can not run rerank service on HPU yet require high performance and accuracy. + +### Prepare Docker Image + Currently we support two ways of deploying ChatQnA services with docker compose: -1. Start services using the docker image on `docker hub`: +1. Using the docker image on `docker hub`: ```bash docker pull opea/chatqna:latest @@ -119,7 +123,9 @@ Currently we support two ways of deploying ChatQnA services with docker compose: docker pull opea/chatqna-conversation-ui:latest ``` -2. Start services using the docker images `built from source`: [Guide](./docker) +2. Using the docker images `built from source`: [Guide](docker/xeon/README.md) + + > Note: The **opea/chatqna-without-rerank:latest** docker image has not been published yet, users need to build this docker image from source. ### Required Models diff --git a/ChatQnA/docker/gaudi/README.md b/ChatQnA/docker/gaudi/README.md index 717988c6b..0d39b9297 100644 --- a/ChatQnA/docker/gaudi/README.md +++ b/ChatQnA/docker/gaudi/README.md @@ -27,6 +27,8 @@ docker build --no-cache -t opea/retriever-redis:latest --build-arg https_proxy=$ ### 4. Build Rerank Image +> Skip for ChatQnA without Rerank pipeline + ```bash docker build --no-cache -t opea/reranking-tei:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/reranks/tei/docker/Dockerfile . ``` @@ -88,23 +90,38 @@ cd ../.. ### 8. Build MegaService Docker Image -To construct the Mega Service, we utilize the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline within the `chatqna.py` Python script. Build the MegaService Docker image using the command below: +1. MegaService with Rerank -```bash -git clone https://github.com/opea-project/GenAIExamples.git -cd GenAIExamples/ChatQnA/docker -docker build --no-cache -t opea/chatqna:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile . -cd ../../.. -``` + To construct the Mega Service with Rerank, we utilize the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline within the `chatqna.py` Python script. Build the MegaService Docker image using the command below: -If you want to enable guardrails microservice in the pipeline, please use the below command instead: + ```bash + git clone https://github.com/opea-project/GenAIExamples.git + cd GenAIExamples/ChatQnA/docker + docker build --no-cache -t opea/chatqna:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile . + cd ../../.. + ``` -```bash -git clone https://github.com/opea-project/GenAIExamples.git -cd GenAIExamples/ChatQnA/docker -docker build --no-cache -t opea/chatqna-guardrails:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile_guardrails . -cd ../../.. -``` +2. MegaService with Guardrails + + If you want to enable guardrails microservice in the pipeline, please use the below command instead: + + ```bash + git clone https://github.com/opea-project/GenAIExamples.git + cd GenAIExamples/ChatQnA/docker + docker build --no-cache -t opea/chatqna-guardrails:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile_guardrails . + cd ../../.. + ``` + +3. MegaService without Rerank + + To construct the Mega Service without Rerank, we utilize the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline within the `chatqna_without_rerank.py` Python script. Build MegaService Docker image via below command: + + ```bash + git clone https://github.com/opea-project/GenAIExamples.git + cd GenAIExamples/ChatQnA/docker + docker build --no-cache -t opea/chatqna-without-rerank:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile_without_rerank . + cd ../../.. + ``` ### 9. Build UI Docker Image @@ -146,7 +163,7 @@ Then run the command `docker images`, you will have the following 8 Docker Image 4. `opea/llm-tgi:latest` or `opea/llm-vllm:latest` or `opea/llm-vllm-ray:latest` 5. `opea/tei-gaudi:latest` 6. `opea/dataprep-redis:latest` -7. `opea/chatqna:latest` or `opea/chatqna-guardrails:latest` +7. `opea/chatqna:latest` or `opea/chatqna-guardrails:latest` or `opea/chatqna-without-rerank:latest` 8. `opea/chatqna-ui:latest` If Conversation React UI is built, you will find one more image: @@ -223,7 +240,10 @@ cd GenAIExamples/ChatQnA/docker/gaudi/ If use tgi for llm backend. ```bash +# Start ChatQnA with Rerank Pipeline docker compose -f compose.yaml up -d +# Start ChatQnA without Rerank Pipeline +docker compose -f compose_without_rerank.yaml up -d ``` If use vllm for llm backend. @@ -288,6 +308,8 @@ curl http://${host_ip}:7000/v1/retrieval \ 4. TEI Reranking Service +> Skip for ChatQnA without Rerank pipeline + ```bash curl http://${host_ip}:8808/rerank \ -X POST \ @@ -297,6 +319,8 @@ curl http://${host_ip}:8808/rerank \ 5. Reranking Microservice +> Skip for ChatQnA without Rerank pipeline + ```bash curl http://${host_ip}:8000/v1/reranking \ -X POST \ @@ -397,12 +421,31 @@ curl -X POST "http://${host_ip}:6007/v1/dataprep/get_file" \ -H "Content-Type: application/json" ``` +Then you will get the response JSON like this. Notice that the returned `name`/`id` of the uploaded link is `https://xxx.txt`. + +```json +[ + { + "name": "nke-10k-2023.pdf", + "id": "nke-10k-2023.pdf", + "type": "File", + "parent": "" + }, + { + "name": "https://opea.dev.txt", + "id": "https://opea.dev.txt", + "type": "File", + "parent": "" + } +] +``` + To delete the file/link you uploaded: ```bash # delete link curl -X POST "http://${host_ip}:6007/v1/dataprep/delete_file" \ - -d '{"file_path": "https://opea.dev"}' \ + -d '{"file_path": "https://opea.dev.txt"}' \ -H "Content-Type: application/json" # delete file diff --git a/ChatQnA/docker/xeon/README.md b/ChatQnA/docker/xeon/README.md index 675e74cea..c736c674a 100644 --- a/ChatQnA/docker/xeon/README.md +++ b/ChatQnA/docker/xeon/README.md @@ -80,6 +80,8 @@ docker build --no-cache -t opea/retriever-redis:latest --build-arg https_proxy=$ ### 3. Build Rerank Image +> Skip for ChatQnA without Rerank pipeline + ```bash docker build --no-cache -t opea/reranking-tei:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/reranks/tei/docker/Dockerfile . ``` @@ -118,14 +120,27 @@ cd .. ### 6. Build MegaService Docker Image -To construct the Mega Service, we utilize the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline within the `chatqna.py` Python script. Build MegaService Docker image via below command: +1. MegaService with Rerank -```bash -git clone https://github.com/opea-project/GenAIExamples.git -cd GenAIExamples/ChatQnA/docker -docker build --no-cache -t opea/chatqna:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile . -cd ../../.. -``` + To construct the Mega Service with Rerank, we utilize the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline within the `chatqna.py` Python script. Build MegaService Docker image via below command: + + ```bash + git clone https://github.com/opea-project/GenAIExamples.git + cd GenAIExamples/ChatQnA/docker + docker build --no-cache -t opea/chatqna:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile . + cd ../../.. + ``` + +2. MegaService without Rerank + + To construct the Mega Service without Rerank, we utilize the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline within the `chatqna_without_rerank.py` Python script. Build MegaService Docker image via below command: + + ```bash + git clone https://github.com/opea-project/GenAIExamples.git + cd GenAIExamples/ChatQnA/docker + docker build --no-cache -t opea/chatqna-without-rerank:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile_without_rerank . + cd ../../.. + ``` ### 7. Build UI Docker Image @@ -156,7 +171,7 @@ Then run the command `docker images`, you will have the following 7 Docker Image 3. `opea/retriever-redis:latest` 4. `opea/reranking-tei:latest` 5. `opea/llm-tgi:latest` or `opea/llm-vllm:latest` -6. `opea/chatqna:latest` +6. `opea/chatqna:latest` or `opea/chatqna-without-rerank:latest` 7. `opea/chatqna-ui:latest` ## 🚀 Start Microservices @@ -238,7 +253,10 @@ cd GenAIExamples/ChatQnA/docker/xeon/ If use TGI backend. ```bash +# Start ChatQnA with Rerank Pipeline docker compose -f compose.yaml up -d +# Start ChatQnA without Rerank Pipeline +docker compose -f compose_without_rerank.yaml up -d ``` If use vLLM backend. @@ -285,6 +303,8 @@ curl http://${host_ip}:7000/v1/retrieval \ 4. TEI Reranking Service +> Skip for ChatQnA without Rerank pipeline + ```bash curl http://${host_ip}:8808/rerank \ -X POST \ @@ -294,6 +314,8 @@ curl http://${host_ip}:8808/rerank \ 5. Reranking Microservice +> Skip for ChatQnA without Rerank pipeline + ```bash curl http://${host_ip}:8000/v1/reranking\ -X POST \ @@ -384,12 +406,33 @@ curl -X POST "http://${host_ip}:6007/v1/dataprep/get_file" \ -H "Content-Type: application/json" ``` +Then you will get the response JSON like this. Notice that the returned `name`/`id` of the uploaded link is `https://xxx.txt`. + +```json +[ + { + "name": "nke-10k-2023.pdf", + "id": "nke-10k-2023.pdf", + "type": "File", + "parent": "" + }, + { + "name": "https://opea.dev.txt", + "id": "https://opea.dev.txt", + "type": "File", + "parent": "" + } +] +``` + To delete the file/link you uploaded: +The `file_path` here should be the `id` get from `/v1/dataprep/get_file` API. + ```bash # delete link curl -X POST "http://${host_ip}:6007/v1/dataprep/delete_file" \ - -d '{"file_path": "https://opea.dev"}' \ + -d '{"file_path": "https://opea.dev.txt"}' \ -H "Content-Type: application/json" # delete file