Modifies use case template format and adds graph validation when provisioning

[joshpalis]

Description

Hoping to merge in #118 first prior to this PR.

This PR achieves multiple things

• Modifies the use case template to separate workflow inputs into previous_node_inputs and user_inputs as described by [DOCS] Use Case Template Format #117
• Introduces a workflow-step.json to define the expected inputs and outputs for each workflow step
• Adds graph validation based on the workflow-step.json. Basically for each ProcessNode in a topologically sorted graph, we retrieve all the expected outputs of its predecessor nodes, combine this with the given user inputs (WorkflowData content) for the current node, and compare this list with the expected inputs of the current node type. If validation fails, we return to the user a list of missing inputs
• Adds tests to ensure that each registered WorkflowStep in the WorkflowStepFactory has a corresponding WorkflowStepValidator in the workflow-steps.json. This is a guard-rail to ensure that each new step that is registered has their inputs/outputs defined for use in validation

Example create workflow request to register and deploy a model (with missing create_connector step that provides the connector_id needed for the register_model_step)

curl -i -XPOST "localhost:9200/_plugins/_flow_framework/workflow" -H "Content-Type:application/json" --data '{"name":"deploy-register-model","description":"test case","use_case":"TEST_USE_CASE","version":{"template":"1.0.0","compatibility":["2.12.0","3.0.0"]},"workflows":{"provision":{"nodes":[{"id":"workflow_step_1","type":"register_model","user_inputs":{"name":"openAI-gpt-3.5-turbo","function_name":"remote","description":"test model"}},{"id":"workflow_step_2","type":"deploy_model","user_inputs":{}}],"edges":[{"source":"workflow_step_1","dest":"workflow_step_2"}]}}}'
HTTP/1.1 200 OK
X-OpenSearch-Version: OpenSearch/3.0.0-SNAPSHOT (opensearch)
content-type: application/json; charset=UTF-8
content-length: 38

{"workflow_id":"ewChbosBllTp7O9ki-A5"}%     

Executing async provisioning for workflow ewChbosBllTp7O9ki-A5

curl -i -XPOST "localhost:9200/_plugins/_flow_framework/workflow/ewChbosBllTp7O9ki-A5/_provision" 
HTTP/1.1 200 OK
X-OpenSearch-Version: OpenSearch/3.0.0-SNAPSHOT (opensearch)
content-type: application/json; charset=UTF-8
content-length: 38

{"workflow_id":"ewChbosBllTp7O9ki-A5"}%     

OpenSearch logs show that graph validation failed and the connector_id field was not provided by user_inputs or predecessor_node_inputs

[2023-10-27T00:56:02,613][INFO ][o.o.f.w.CreateIndexStep  ] [dev-dsk-jpalis-2c-27c8aa11.us-west-2.amazon.com] create index:.plugins-ai-global-context
[2023-10-27T00:56:20,580][ERROR][o.o.f.t.ProvisionWorkflowTransportAction] [dev-dsk-jpalis-2c-27c8aa11.us-west-2.amazon.com] Provisioning failed for workflow ewChbosBllTp7O9ki-A5 : org.opensearch.flowframework.exception.FlowFrameworkException: Invalid graph, missing the following required inputs : [connector_id]

Issues Resolved

Part of #88
#18
#67
#117

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

Merging #119 (8ba9d0b) into main (7d9cd84) will increase coverage by 1.96%.
The diff coverage is 93.45%.

@@             Coverage Diff              @@
##               main     #119      +/-   ##
============================================
+ Coverage     67.09%   69.05%   +1.96%     
- Complexity      276      302      +26     
============================================
  Files            35       37       +2     
  Lines          1319     1393      +74     
  Branches        125      132       +7     
============================================
+ Hits            885      962      +77     
+ Misses          390      386       -4     
- Partials         44       45       +1     

Files	Coverage Δ
...g/opensearch/flowframework/common/CommonValue.java	100.00% <ø> (ø)
...arch/flowframework/indices/FlowFrameworkIndex.java	100.00% <ø> (ø)
...arch/flowframework/model/ProvisioningProgress.java	100.00% <ø> (ø)
...java/org/opensearch/flowframework/model/State.java	100.00% <ø> (ø)
...g/opensearch/flowframework/model/WorkflowNode.java	91.35% <100.00%> (+0.81%)	⬆️
...rch/flowframework/model/WorkflowStepValidator.java	100.00% <100.00%> (ø)
...nsearch/flowframework/model/WorkflowValidator.java	100.00% <100.00%> (ø)
...search/flowframework/workflow/CreateIndexStep.java	85.29% <100.00%> (ø)
...owframework/workflow/CreateIngestPipelineStep.java	90.62% <100.00%> (ø)
.../flowframework/workflow/WorkflowProcessSorter.java	97.80% <100.00%> (+0.70%)	⬆️
... and 3 more

... and 4 files with indirect coverage changes

[owaiskazi19]

Looks good overall with minor comments. Thanks for adding the validation. This is a great addition.
Let's try to break the PR into multiple PRs next time for easy review.

[dbwiddis]

Looks great, a few suggestions.

[dbwiddis]

Sorry, one last change please (but I clicked approve so once that's fixed I don't need to re-review.)