-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add WorkflowStep Factory and implement XContent-based Template Parsing #47
Conversation
Codecov Report
@@ Coverage Diff @@
## main #47 +/- ##
=============================================
+ Coverage 73.87% 86.93% +13.05%
- Complexity 65 163 +98
=============================================
Files 8 13 +5
Lines 245 574 +329
Branches 22 75 +53
=============================================
+ Hits 181 499 +318
+ Misses 54 52 -2
- Partials 10 23 +13
|
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
initial pass, still need to go over the test classes
src/main/java/org/opensearch/flowframework/template/Template.java
Outdated
Show resolved
Hide resolved
src/main/java/org/opensearch/flowframework/template/Template.java
Outdated
Show resolved
Hide resolved
src/main/java/org/opensearch/flowframework/workflow/Workflow.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Daniel Widdis <widdis@gmail.com>
src/main/java/org/opensearch/flowframework/template/WorkflowNode.java
Outdated
Show resolved
Hide resolved
To ease review, a general summary of the main part of this PR:
Additional changes:
|
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Initial Pass! Thanks for adding XContent functionality to the project
src/main/java/org/opensearch/flowframework/template/Template.java
Outdated
Show resolved
Hide resolved
src/main/java/org/opensearch/flowframework/template/Template.java
Outdated
Show resolved
Hide resolved
src/main/java/org/opensearch/flowframework/template/Template.java
Outdated
Show resolved
Hide resolved
src/main/java/org/opensearch/flowframework/template/WorkflowNode.java
Outdated
Show resolved
Hide resolved
src/main/java/org/opensearch/flowframework/workflow/Workflow.java
Outdated
Show resolved
Hide resolved
src/main/java/org/opensearch/flowframework/workflow/Workflow.java
Outdated
Show resolved
Hide resolved
src/main/java/org/opensearch/flowframework/workflow/WorkflowStepFactory.java
Show resolved
Hide resolved
Signed-off-by: Daniel Widdis <widdis@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good overall with a minor comment/question.
Thanks @dbwiddis for adding the Pipeline Processor parsers! |
#47) * Add WorkflowStepFactory class Signed-off-by: Daniel Widdis <widdis@gmail.com> * Add XContent classes representing Template JSON Signed-off-by: Daniel Widdis <widdis@gmail.com> * Add parse methods for the Template XContent Signed-off-by: Daniel Widdis <widdis@gmail.com> * Cleanup parsing, javadocs, and demo output Signed-off-by: Daniel Widdis <widdis@gmail.com> * Refactor to use field name constants, get tests working again Signed-off-by: Daniel Widdis <widdis@gmail.com> * Separate WorkflowNode and ProcessNode functionality Signed-off-by: Daniel Widdis <widdis@gmail.com> * Fix demos to align with template field names Signed-off-by: Daniel Widdis <widdis@gmail.com> * Add workflow, node, and edge tests Signed-off-by: Daniel Widdis <widdis@gmail.com> * Add Template tests Signed-off-by: Daniel Widdis <widdis@gmail.com> * Refactor TemplateParser to WorkflowProcessSorter Signed-off-by: Daniel Widdis <widdis@gmail.com> * Test exceptional cases Signed-off-by: Daniel Widdis <widdis@gmail.com> * Finish up exceptional cases Signed-off-by: Daniel Widdis <widdis@gmail.com> * Fix a template field name bug in demo Signed-off-by: Daniel Widdis <widdis@gmail.com> * Rebase with #34 Signed-off-by: Daniel Widdis <widdis@gmail.com> * Rebase changes from #54 Signed-off-by: Daniel Widdis <widdis@gmail.com> * Integrate thread pool executor service Signed-off-by: Daniel Widdis <widdis@gmail.com> * Fix flaky ProcessNodeTests by removing orTimeout Signed-off-by: Daniel Widdis <widdis@gmail.com> * Rebase and refactor with #44 Signed-off-by: Daniel Widdis <widdis@gmail.com> * Fix demos and remove DataDemo Signed-off-by: Daniel Widdis <widdis@gmail.com> * Use non-deprecated mapping method for CreateIndexStep Signed-off-by: Daniel Widdis <widdis@gmail.com> * Eliminate casting and deprecation warnings on test classes Signed-off-by: Daniel Widdis <widdis@gmail.com> * Remove unused/leftover demo class Signed-off-by: Daniel Widdis <widdis@gmail.com> * Typo Signed-off-by: Daniel Widdis <widdis@gmail.com> * Don't offer steps as an alternative to nodes Signed-off-by: Daniel Widdis <widdis@gmail.com> * Move Workflow into package with all the other parsing classes Signed-off-by: Daniel Widdis <widdis@gmail.com> * Move process sequencing classes into workflow package Signed-off-by: Daniel Widdis <widdis@gmail.com> * Add PipelineProcessor class and XContent parsing, rename package Signed-off-by: Daniel Widdis <widdis@gmail.com> --------- Signed-off-by: Daniel Widdis <widdis@gmail.com> (cherry picked from commit 734f9c2) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…Template Parsing (#60) Add WorkflowStep Factory and implement XContent-based Template Parsing (#47) * Add WorkflowStepFactory class * Add XContent classes representing Template JSON * Add parse methods for the Template XContent * Cleanup parsing, javadocs, and demo output * Refactor to use field name constants, get tests working again * Separate WorkflowNode and ProcessNode functionality * Fix demos to align with template field names * Add workflow, node, and edge tests * Add Template tests * Refactor TemplateParser to WorkflowProcessSorter * Test exceptional cases * Finish up exceptional cases * Fix a template field name bug in demo * Rebase with #34 * Rebase changes from #54 * Integrate thread pool executor service * Fix flaky ProcessNodeTests by removing orTimeout * Rebase and refactor with #44 * Fix demos and remove DataDemo * Use non-deprecated mapping method for CreateIndexStep * Eliminate casting and deprecation warnings on test classes * Remove unused/leftover demo class * Typo * Don't offer steps as an alternative to nodes * Move Workflow into package with all the other parsing classes * Move process sequencing classes into workflow package * Add PipelineProcessor class and XContent parsing, rename package --------- (cherry picked from commit 734f9c2) Signed-off-by: Daniel Widdis <widdis@gmail.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Description
WorkflowStepFactory
class to map a template "step type" to the correct instanceIssues Resolved
Fixes #41
Fixes #42
Fixes #46
Fixes #50
Fixes #55
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.