Advanced RAG AI Service running in Docker, it can be used as
-
Quick MVP/POCm playground to verify which Index type can exactly address the specific (usually related to accuracy) LLM RAG use case.
-
OpenAPI interface and Gradio REST API interface are supported.
Http Clients, Power platform workflow, and MS Office Word/Outlook Add-In can easily use proper vector index types to search doc info and generate LLM response from the service.
![drawing](https://private-user-images.githubusercontent.com/8623897/350900275-71ca8893-e5f7-4f1b-9306-353cf599f333.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjU5NDAyMjAsIm5iZiI6MTcyNTkzOTkyMCwicGF0aCI6Ii84NjIzODk3LzM1MDkwMDI3NS03MWNhODg5My1lNWY3LTRmMWItOTMwNi0zNTNjZjU5OWYzMzMucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDkxMCUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA5MTBUMDM0NTIwWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9YjVkZDU5MTE4NjVkZmI2ODg5NDlmYTM5NmNhYzBkNWQ1YWU5NmRkM2QxZjcxYzJjMjBhNWY5ZmQ4ZGUwOGI3ZiZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.G8UfyByGVfw-5kdNGVp696s-9cBchu-A6jIreb2nZyk)
The service can help developers quickly verify different RAG indexing techniques (about accuracy and performance) for their own user cases, from Index Generation to verify the output through Chat Mode and Proofreading mode.
It can run as local Docker or put it to Azure Container App, build and perform queries on multiple important Index types. Below is the info about index types and how the project implements them:
-
Azure AI Search : Azure AI Search Python SDK + Hybrid Semantic Search + Sub Question Query
-
MS GraphRAG Local : REST APIs Provided by GraphRAG accelerator
-
MS GraphRAG Global : REST APIs Provided by GraphRAG accelerator
-
Knowledge Graph : LlamaIndex
-
Recursive Retriever : LlamaIndex
-
Summary Index : LlamaIndex
-
CSV Query Engine: LlamaIndex
![drawing](https://private-user-images.githubusercontent.com/8623897/350902284-e5d5da11-7091-4565-95ab-b6a1f0918283.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjU5NDAyMjAsIm5iZiI6MTcyNTkzOTkyMCwicGF0aCI6Ii84NjIzODk3LzM1MDkwMjI4NC1lNWQ1ZGExMS03MDkxLTQ1NjUtOTVhYi1iNmExZjA5MTgyODMucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDkxMCUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA5MTBUMDM0NTIwWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9MzJlOTUyZDcyY2RlNTBkNzhmZDQ3Y2QyYjZmMThlNmQ1ODk1MWE5M2FjZjEwODYxYjRkYzZmN2VmZGI0OWFjNCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.yqcljTmPilLg_QUUiXWXl3ebdcHgDCB2LzDraqXR0fQ)
![drawing](https://private-user-images.githubusercontent.com/8623897/354168392-42b556a2-2862-4dc0-9511-25dd9a01ffc3.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjU5NDAyMjAsIm5iZiI6MTcyNTkzOTkyMCwicGF0aCI6Ii84NjIzODk3LzM1NDE2ODM5Mi00MmI1NTZhMi0yODYyLTRkYzAtOTUxMS0yNWRkOWEwMWZmYzMucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDkxMCUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA5MTBUMDM0NTIwWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9OWRjNTM4ODE3ODE1ZThhNGJmYmFmYmVhYjYwMzJkNjU2OGM4ZTM3YmI4MDIxNThlNWFjMWZjYjVlM2ZhYWY2OSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.gk8ks_oiu9uessCkkDMQaaOmmU8sDJZmCMmt84TEOTo)
Read in Change Log
https://github.com/freistli/AdvancedRAG_SVC
- Download the repo:
git clone https://github.com/freistli/AdvancedRAG.git
-
IMPORTANT: Rename .env.sample to .env, input necessary environment variables.
Azure OpenAI resource and Azure Document Intelligence resource are must required.
Azure AI Search is optional if you don't build Azure AI Search index or use it.
MS GRAPHRAG is optional. If you want to use it please go through these steps to create GraphRAG backend service on Azure: https://github.com/azure-samples/graphrag-accelerator
Check Parameters for more details.
-
Build docker image
docker build -t docaidemo .
- Run it locally
docker run -p 8000:8000 docaidemo
- Access it locally
-
Publish your image to Azure Container Registry or Docker Hub
-
Create Azure Container App, choose the docker image you published, and then deploy the revision pod without any extra command.
Note: If you don't use .env in image, can set environment variables in the Azure Container App.
-
Click one Index Build tab
-
Upload a file to file section
Note: The backend is Azure Document Intelligence to read the content, in general it supports lots of file formats, recommend to use PDF (print it as PDF) if the content is complicated
-
Input an Index name, click Submit
-
Wait till the index building completed, the right pane will update the status in real time.
-
After you see this words, means it is completed:
2024-06-13T10:04:54.120027: Index is persisted in /tmp/index_cache/yourindexname
/tmp/index_cache/yourindexname or yourindexname can be used as your Index name.
- You can download the index files to local by clicking the Download Index button, so that can use it in your own docker image
Developers can use their own indexes folders for the docker image:
To make it work:
- Move to the solution folder which contains the AdvancedRAG dockerfile
- Create a folder to keep the index, for example, index123
- Extract the index zip file you get from the step 6 in the "Build Index" section, save index files you downloaded into ./index123
- Build the docker image again.
After this, you can use index123 as index name in the Chat mode.
NOTE: You can use the same way to store CSV files, build them with docker if you want to try CSV Query Engine without file uploading.
https://{BASEURL}/advchatbot/run/chat
POST
Content-Type: application/json
{
"data": [
"When did the Author convince his farther", <----- Prompt
"", <--- History Object, don't change it
"Azure AI Search", <------ Index Type
"azuresearch_0", <------- Index Name or Folder
"You are a friendly AI Assistant", <----- System Message
false <---- Streaming flag, true or false
]
}
https://{BASEURL}/proofreadaddin/run/predict
NOTE: "rules" Index Name is predefined for Knowledge Graph Index of proofread addin. Please save Default knowledge graph index into ./rules/storage/rules_original
POST
Content-Type: application/json
{
"data": [
"今回は半導体製造装置セクターの最近の動きを分析します。" , <---- Proofread Content
false <--- Streaming flag, true or false
]
}
https://{BASEURL}/csvqueryengine/run/chat
POST
Content-Type: application/json
{
"data": [
"how many records does it have",
"",
"CSV Query Engine",
"./rules/files/sample.csv",
"You are a helpful AI assistant. You can help users with a variety of tasks, such as answering questions, providing recommendations, and assisting with tasks. You can also provide information on a wide range of topics from the retieved documents. If you are unsure about something, you can ask for clarification. Please be polite and professional at all times. If you have any questions, feel free to ask." ,
false
]
}
Check: https://github.com/freistli/ProofreadAddin
Step by Step: Integrate Advanced RAG Service with Your Own Data into Copilot Studio
- Click Chat Mode
- Choose the Index type
- Put the index path or Azure AI Search Index name to Index Name text field
- Now you can try to chat with the document with various system message if need.
This is specific for Proofread scenario, especially for non-English Languages. It needs you generate Knowlege Graph Index.
Generating Knowledge Graph Index steps are the same as others.
- Click the View Knowlege Graph Index tab
- Put you Knowledge Graph Index name, and click Submit
- Wait a while, click Download Knowledge Graph View to see the result.