Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature/agent_framework] Adding installed plugins validation #290

Merged
merged 5 commits into from
Dec 15, 2023

Conversation

joshpalis
Copy link
Member

@joshpalis joshpalis commented Dec 14, 2023

Description

This PR adds a new field required_plugins to the workflow_step json which is used to determine if the workflow step's corresponding plugin is currently installed within the local node. Additionally, this PR adds validation when creating (with validation) and provisioning to check if the given graph has all the required plugins prior to executing.

The following example uses a create connector node with an intentionally modified required plugin named test-plugin to demonstrate how the validation works :

Creating the workflow :

curl -i -XPOST "localhost:9200/_plugins/_flow_framework/workflow" -H "Content-Type:application/json" --data '{"name":"create-connector-register-deploy-model","description":"test case","use_case":"TEST_CASE","version":{"template":"1.0.0","compatibility":["2.12.0","3.0.0"]},"workflows":{"provision":{"nodes":[{"id":"workflow_step_1","type":"create_connector","user_inputs":{"name":"OpenAI Chat Connector","description":"The connector to public OpenAI model service for GPT 3.5","version":"1","protocol":"http","parameters":{"endpoint":"api.openai.com","model":"gpt-3.5-turbo"},"credential":{"openAI_key":"12345"},"actions":[{"action_type":"predict","method":"POST","url":"https://${parameters.endpoint}/v1/chat/completions"}]}},{"id":"workflow_step_2","type":"register_remote_model","previous_node_inputs":{"workflow_step_1":"connector_id"},"user_inputs":{"name":"openAI-gpt-3.5-turbo","function_name":"remote","description":"test model"}},{"id":"workflow_step_3","type":"deploy_model","previous_node_inputs":{"workflow_step_2":"model_id"}}],"edges":[{"source":"workflow_step_1","dest":"workflow_step_2"},{"source":"workflow_step_2","dest":"workflow_step_3"}]}}}'

HTTP/1.1 201 Created
content-type: application/json; charset=UTF-8
content-length: 38

{"workflow_id":"n-jqaYwBsLRi2-luOv83"}

Provisioning the workflow results in an error that states that the create_connector step requires the test-plugin to be installed

curl -i -XPOST "localhost:9200/_plugins/_flow_framework/workflow/n-jqaYwBsLRi2-luOv83/_provision"
HTTP/1.1 400 Bad Request
content-type: application/json; charset=UTF-8
content-length: 108

{"error":"The workflowStep create_connector requires the following plugins to be installed : [test-plugin]"}

Issues Resolved

#126
Part of #88

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Joshua Palis <jpalis@amazon.com>
Signed-off-by: Joshua Palis <jpalis@amazon.com>
Copy link
Member

@dbwiddis dbwiddis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job! First pass, some suggestions.

Copy link
Member

@dbwiddis dbwiddis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved, suggestions above remain suggestions for consideration but OK as is.

@joshpalis joshpalis changed the title [Feature/agent_framework] Adding installed plugins validation when provisioning [Feature/agent_framework] Adding installed plugins validation Dec 14, 2023
Signed-off-by: Joshua Palis <jpalis@amazon.com>
Signed-off-by: Joshua Palis <jpalis@amazon.com>
Signed-off-by: Joshua Palis <jpalis@amazon.com>
@dbwiddis dbwiddis merged commit 6053594 into opensearch-project:feature/agent_framework Dec 15, 2023
10 checks passed
dbwiddis pushed a commit to dbwiddis/flow-framework that referenced this pull request Dec 15, 2023
…arch-project#290)

* Adding installed plugins validation

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Adding failure success unit tests

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Combining graph and installed plugin validation

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Removing stray comment

Signed-off-by: Joshua Palis <jpalis@amazon.com>

---------

Signed-off-by: Joshua Palis <jpalis@amazon.com>
dbwiddis pushed a commit to dbwiddis/flow-framework that referenced this pull request Dec 15, 2023
…arch-project#290)

* Adding installed plugins validation

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Adding failure success unit tests

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Combining graph and installed plugin validation

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Removing stray comment

Signed-off-by: Joshua Palis <jpalis@amazon.com>

---------

Signed-off-by: Joshua Palis <jpalis@amazon.com>
dbwiddis pushed a commit to dbwiddis/flow-framework that referenced this pull request Dec 15, 2023
…arch-project#290)

* Adding installed plugins validation

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Adding failure success unit tests

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Combining graph and installed plugin validation

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Removing stray comment

Signed-off-by: Joshua Palis <jpalis@amazon.com>

---------

Signed-off-by: Joshua Palis <jpalis@amazon.com>
dbwiddis pushed a commit to dbwiddis/flow-framework that referenced this pull request Dec 15, 2023
…arch-project#290)

* Adding installed plugins validation

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Adding failure success unit tests

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Combining graph and installed plugin validation

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Removing stray comment

Signed-off-by: Joshua Palis <jpalis@amazon.com>

---------

Signed-off-by: Joshua Palis <jpalis@amazon.com>
dbwiddis pushed a commit that referenced this pull request Dec 18, 2023
* Adding installed plugins validation

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Adding failure success unit tests

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Combining graph and installed plugin validation

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Removing stray comment

Signed-off-by: Joshua Palis <jpalis@amazon.com>

---------

Signed-off-by: Joshua Palis <jpalis@amazon.com>
dbwiddis pushed a commit to dbwiddis/flow-framework that referenced this pull request Dec 18, 2023
…arch-project#290)

* Adding installed plugins validation

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Adding failure success unit tests

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Combining graph and installed plugin validation

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Removing stray comment

Signed-off-by: Joshua Palis <jpalis@amazon.com>

---------

Signed-off-by: Joshua Palis <jpalis@amazon.com>
dbwiddis pushed a commit that referenced this pull request Dec 18, 2023
* Adding installed plugins validation

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Adding failure success unit tests

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Combining graph and installed plugin validation

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Removing stray comment

Signed-off-by: Joshua Palis <jpalis@amazon.com>

---------

Signed-off-by: Joshua Palis <jpalis@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants