-
Notifications
You must be signed in to change notification settings - Fork 9.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support Bedrock cross region inference #8190
Comments
Dify currently supports AWS Bedrock as one of the Model-as-a-Service (MaaS) vendors, including the ability to specify the AWS region for model inference. However, there is no specific documentation or code available in the repository that outlines how to set up cross-region inference for AWS Bedrock models. To support cross-region inference, you would need to configure the from botocore.config import Config
import boto3
client_config = Config(
region_name='us-west-2' # Specify the desired AWS region
)
bedrock_runtime = boto3.client(
service_name='bedrock-runtime',
config=client_config,
aws_access_key_id='your_access_key',
aws_secret_access_key='your_secret_access_key'
) This configuration ensures that the Bedrock runtime client operates in the specified AWS region, which is crucial for cross-region inference [1]. For more detailed steps on deploying and configuring cross-region inference, you can refer to the AWS Bedrock documentation. |
…rent region groups
…to distinguish between different region groups) (#8402) Co-authored-by: Yuanbo Li <ybalbert@amazon.com>
* main: (40 commits) feat: allow users to specify timeout for text generations and workflows by environment variable (#8395) Fix: operation postion of answer in logs (#8411) fix: when the variable does not exist, an error should be prompted (#8413) fix(workflow): the answer node after the iteration node containing the answer was output prematurely (#8419) fix:logs and rm unused codes in CacheEmbedding (#8409) fix: resolve runtime error when self.folder is None (#8401) Fix: Support Bedrock cross region inference #8190 (Update Model name to distinguish between different region groups) (#8402) fix(docker): aliyun oss path env key (#8394) fix: pyproject.toml typo (#8396) fix: o1-mini 65563 -> 65536 (#8388) fix: sandbox issue related httpx and requests (#8397) chore: improve usage of striping prefix or suffix of string with Ruff 0.6.5 (#8392) fix (#8322 followup): resolve the violation of pylint rules (#8391) chore: refurish python code by applying Pylint linter rules (#8322) support hunyuan-turbo (#8372) chore: update firecrawl scrape to V1 api (#8367) fix(workflow): both parallel and single branch errors occur in if-else (#8378) fix: edit load balancing not pass id (#8370) fix: add before send to remove langfuse defaultErrorResponse (#8361) fix: when edit load balancing config not pass the empty filed value hidden (#8366) ...
…rent region groups
* feat/new-login: (27 commits) chore: format feat: add fulfill_login_params_from_env feat: update message feat: update mismatch description feat: remove env example chore: PLR6201 Use a set literal when testing for membership feat: update EmailOrPasswordMismatchError feat: allow users to specify timeout for text generations and workflows by environment variable (#8395) Fix: operation postion of answer in logs (#8411) fix: when the variable does not exist, an error should be prompted (#8413) fix(workflow): the answer node after the iteration node containing the answer was output prematurely (#8419) fix:logs and rm unused codes in CacheEmbedding (#8409) fix: resolve runtime error when self.folder is None (#8401) Fix: Support Bedrock cross region inference #8190 (Update Model name to distinguish between different region groups) (#8402) fix(docker): aliyun oss path env key (#8394) fix: pyproject.toml typo (#8396) fix: o1-mini 65563 -> 65536 (#8388) fix: sandbox issue related httpx and requests (#8397) chore: improve usage of striping prefix or suffix of string with Ruff 0.6.5 (#8392) fix (#8322 followup): resolve the violation of pylint rules (#8391) ...
Co-authored-by: Yuanbo Li <ybalbert@amazon.com>
Co-authored-by: Yuanbo Li <ybalbert@amazon.com>
Co-authored-by: Yuanbo Li <ybalbert@amazon.com>
…odel name to distinguish between different region groups) (langgenius#8402) Co-authored-by: Yuanbo Li <ybalbert@amazon.com>
Co-authored-by: Yuanbo Li <ybalbert@amazon.com>
…odel name to distinguish between different region groups) (langgenius#8402) Co-authored-by: Yuanbo Li <ybalbert@amazon.com>
Co-authored-by: Yuanbo Li <ybalbert@amazon.com>
…odel name to distinguish between different region groups) (langgenius#8402) Co-authored-by: Yuanbo Li <ybalbert@amazon.com>
Co-authored-by: Yuanbo Li <ybalbert@amazon.com>
Self Checks
1. Is this request related to a challenge you're experiencing? Tell me about your story.
Can you consider to support AWS bedrock cross region inference, like Claude 3.5 sonnet model
https://docs.aws.amazon.com/bedrock/latest/userguide/cross-region-inference.html
2. Additional context or comments
https://docs.aws.amazon.com/bedrock/latest/userguide/cross-region-inference.html
3. Can you help us with this feature?
The text was updated successfully, but these errors were encountered: