Skip to content

Latest commit

 

History

History
146 lines (108 loc) · 9.45 KB

README.md

File metadata and controls

146 lines (108 loc) · 9.45 KB
Raw

 English  ｜ 简体中文 


Dify AWS Tool

Introduction

This repo provides some sample code to show how to integrate SageMaker Provider and some AWS Service based tools to Dify.

In addition to the reference code, you can also refer to the Dify official guide for more information.

Prerequisites

  • Dify Environment (It can be deployed on AWS with one click using Cloudformation Template - dify.yaml)

  • AWS Account and AWS Experience

  • Basic experience with Linux environments

Assets

[Attention]：We welcome contributions of more workflows, SageMaker models, and built-in tools. You can fork this repository and submit a merge request, please also update README.md, you need to add a new row to the corresponding table

Workflow(Demo Page)

DSL Name Description Link Dependency Owner
Term_based_translate Translation Workflow with Term mapping Retrieval Tool DSL Tool(Term_mapping) ybalbert
Code_translate Code Transform between different Program Language DSL binc
Basic_RAG_Sample simple basic rag workflow with rerank tool DSL Tool(Rerank) ybalbert
Andrewyng/translation-agent Andrew Ng's translate agent. DSL chuanxie
rag_based_bot_with_tts Rag based bot which can answer with voice DSL Tool(TTS) ybalbert
Marketing-copywriter marketing copywriter DSL Lyson Ober
Simple_Kimi Customized KIMI app DSL ybalbert
SVG_Designer SVG Designer DSL 李继刚
Education_Question_Gen Education Question Generator DSL chuanxie
Apply_guardrails Apply guardrails for chatbot DSL amyli

Builtin_Tools

Tool Name Tool Type Description Deploy_doc Owner
Rerank PAAS Text Similarity Rerank Tool Notebook ybalbert
TTS PAAS Speech synthesis Tool Code ybalbert
Bedrock Guardrails SAAS Text moderation Tool, implemented through the independent assessment API ApplyGuardrail API provided on Amazon Bedrock Guardrail. amyli
Term_multilingual_mapping PAAS Word Segment/ Term mapping Retrieval Tool Repo ybalbert
Image Translation Tool PAAS Translate the text on Image Coming tanqy
Chinese Toxicity Detector PAAS A tool to detect Chinese toxicity Comming ychchen
Transcribe Tool PAAS AWS transcribe service tool (ASR) river xie
Bedrock Retriever PAAS A tool for retrieving relevant information from Amazon Bedrock Knowledge Base Comming ychchen

Model_Provider

Model Name model_type Deploy_doc Owner
Any Model SageMaker\LLM Model_hub ybalbert
Bge-m3-rerank-v2 SageMaker\Rerank Notebook ybalbert
Bge-embedding-m3 SageMaker\Embedding Notebook ybalbert
CosyVoice SageMaker\TTS Code ybalbert
SenseVoice SageMaker\ASR Notebook ybalbert

[Attention]： The Dify provider of SageMaker\LLM can support most open-source models. We recommend you to reploy these models using Model_hub. It's very easy and convenient which supports model fine-tuning and deployment with no-code approach

How to Install

Below Script is only for SageMaker Model_provider and AWS Builtin Tools, you can import workflows from Web Interface. SageMaker Model_provider has already been integrated in Dify v0.6.15

  1. Set Env Variable

    dify_path=/home/ec2-user/dify #Please set the correct dify install path
tag=aws

  2. download code

    cd /home/ec2-user/
git clone https://github.com/aws-samples/dify-aws-tool/

  3. intall code

    # Part of models and tools have been integrated with dify already, no extra installation needed
mv ./dify-aws-tool/builtin_tools/aws ${dify_path}/api/core/tools/provider/builtin/
mv ./dify-aws-tool/model_provider/sagemaker ${dify_path}/api/core/model_runtime/model_providers/

  4. build image

    cd ${dify_path}/api
sudo docker build -t dify-api:${tag} .

  5. Specify the new image for api and worker services

    # Modify docker/docker-compose.yaml, Please refer the below diff
diff --git a/docker/docker-compose.yaml b/docker/docker-compose.yaml
index cffaa5a6a..38538e5ca 100644
--- a/docker/docker-compose.yaml
+++ b/docker/docker-compose.yaml
@@ -177,7 +177,7 @@ x-shared-env: &shared-api-worker-env
 services:
   # API service
   api:
-    image: langgenius/dify-api:0.6.14
+    image: dify-api:aws
     restart: always
     environment:
       # Use the shared environment variables.
@@ -197,7 +197,7 @@ services:
   # worker service
   # The Celery worker for processing the queue.
   worker:
-    image: langgenius/dify-api:0.6.14
+    image: dify-api:aws
     restart: always
     environment:
       # Use the shared environment variables.

  6. restart dify

    cd ${dify_path}/docker/
sudo docker-compose down
sudo docker-compose up -d

How to deploy SageMaker Endpoint

If you want to add your LLM/Embedding/Rerank/ASR/TTS model to Dify Sagemaker Model Provider, you should deploy them by yourself in AWS/SageMaker at first.
Please refer the corresponding notebook to deploy.

Target Audience

  • Dify / AWS User
  • GenAI Developer