Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Amazon Bedrock Client for AutoGen #3232

Merged
merged 21 commits into from
Aug 26, 2024
Merged

Amazon Bedrock Client for AutoGen #3232

merged 21 commits into from
Aug 26, 2024

Conversation

Hk669
Copy link
Contributor

@Hk669 Hk669 commented Jul 27, 2024

Why are these changes needed?

Amazon Bedrock also offers a broad set of capabilities to build generative AI applications with security, privacy, and responsible AI. The AWS bedrock has acquired all the latest open-source and closed-source models for inference, making it more flexible for the users to manage the complete infrastructure from AWS. this addition will increase the capability of AutoGen to support the models from bedrock.

API Documentation: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html

For testing:

# Authentication parameters:
# aws_region (mandatory)
# aws_access_key (or environment variable: AWS_ACCESS_KEY)
# aws_secret_key (or environment variable: AWS_SECRET_KEY)
# aws_session_token (or environment variable: AWS_SESSION_TOKEN)
# aws_profile_name

config_list = [
    {
        "api_type": "bedrock",
        "model": "meta.llama3-1-8b-instruct-v1:0",
        "aws_region": "us-west-2",
        "aws_access_key": "",
        "aws_secret_key": "",
        "price" : [0.003, 0.015]
    }
]

Related issue number

Checks

Copy link
Collaborator

@thinkall thinkall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much for the contributions!

@thinkall thinkall added this pull request to the merge queue Aug 26, 2024
Merged via the queue into main with commit 34b34d0 Aug 26, 2024
145 of 157 checks passed
@thinkall thinkall deleted the aws-bedrock branch August 26, 2024 07:34
victordibia pushed a commit that referenced this pull request Aug 28, 2024
* intial commit for aws-bedrock

* format

* converse setup for model req-response

* Renamed to bedrock.py, updated parameter parsing, system message extraction, client class incorporation

* Established Bedrock class based on @astroalek and @ChristianT's code, added ability to disable system prompt separation

* Image parsing and removing access credential checks

* Added tests, added additional parameter support

* Amazon Bedrock documentation

* Moved client parameters to init, align parameter names with Anthropic, spelling fix, remove unnecessary imports, use base and additional parameters, update documentation

* Tidy up comments

* Minor typo fix

* Correct comment re aws_region

---------

Co-authored-by: Mark Sze <mark@sze.family>
Co-authored-by: Mark Sze <66362098+marklysze@users.noreply.github.com>
Co-authored-by: Li Jiang <bnujli@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
models Pertains to using alternate, non-GPT, models (e.g., local models, llama, etc.)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants