-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Set up Rest Integration Test framework #251
Conversation
Signed-off-by: Joshua Palis <jpalis@amazon.com>
Signed-off-by: Joshua Palis <jpalis@amazon.com>
src/test/java/org/opensearch/flowframework/FlowFrameworkRestTestCase.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Joshua Palis <jpalis@amazon.com>
Signed-off-by: Joshua Palis <jpalis@amazon.com>
…n updated to PROVISIONING Signed-off-by: Joshua Palis <jpalis@amazon.com>
Signed-off-by: Joshua Palis <jpalis@amazon.com>
@joshpalis we might need to few failure cases which were not tested using unit tests. Overall LGTM! |
Signed-off-by: Joshua Palis <jpalis@amazon.com>
…ntegration test set up to wait for ml config index to become created, fixed settings update to oly occur once Signed-off-by: Joshua Palis <jpalis@amazon.com>
src/test/resources/template/createconnector-registerremotemodel-deploymodel.json
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. One suggestion, mostly for the future.
Locally, all the integration tests pass for me. This error is coming from the resources created update conflict for the state index. This issue has been fixed in this PR that has been merged into the We've mitigated update conflicts by completing workflow step futures only after the update response has been returned, so once everything is merged to main we shouldn't see this error. @dbwiddis due to this flaky test, I would rather we wait on merging this PR until after everything from CC : @amitgalitz |
@owaiskazi19, I will handle in a separate PR for transport integration tests. This PR is just for the rest integration tests |
…istration. Persiting cluster settings between test runs to ensure plugin apis are enabled. Cleaning up resources after all test runs complete, rather than between test runs Signed-off-by: Joshua Palis <jpalis@amazon.com>
As long as we're starting that process now I'm fine with this, but if we want to continue building more tests, not having this merged makes it harder. Does this almost completely cover what we've developed so far... and you can keep it up to date as we build more APIs (delete / unprovision)? |
These tests cover what we have so far in But yes, I'll keep this updated :) |
…ls are encrypted Signed-off-by: Joshua Palis <jpalis@amazon.com>
The comment was for rest integration tests itself, we can have some failure templates (like missing mandatory fields, cyclic graph) which would through a BAD REQUEST. It's fine to handle it in a separate PR. |
Signed-off-by: Joshua Palis <jpalis@amazon.com>
I have added tests for invalid graph/ invalid inputs for this PR |
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
* Setting up rest integration tests Signed-off-by: Joshua Palis <jpalis@amazon.com> * removing stray log Signed-off-by: Joshua Palis <jpalis@amazon.com> * Cleaning up integration test example Signed-off-by: Joshua Palis <jpalis@amazon.com> * Fixing provision transport action to respond only after state has been updated to PROVISIONING Signed-off-by: Joshua Palis <jpalis@amazon.com> * Fixing flaky encryption test Signed-off-by: Joshua Palis <jpalis@amazon.com> * cleaning up old logs Signed-off-by: Joshua Palis <jpalis@amazon.com> * Added helper methods to retrieve state and resources created, fixed integration test set up to wait for ml config index to become created, fixed settings update to oly occur once Signed-off-by: Joshua Palis <jpalis@amazon.com> * Adding another test for update API, input validation, local model registration. Persiting cluster settings between test runs to ensure plugin apis are enabled. Cleaning up resources after all test runs complete, rather than between test runs Signed-off-by: Joshua Palis <jpalis@amazon.com> * Adding test for search workflows API, ensures that returned credentials are encrypted Signed-off-by: Joshua Palis <jpalis@amazon.com> * Update integ test TODOs to match current development progress Signed-off-by: Daniel Widdis <widdis@gmail.com> * Model Group step is not yet implemented Signed-off-by: Daniel Widdis <widdis@gmail.com> * Comment out tests for incomplete register local model implementation Signed-off-by: Daniel Widdis <widdis@gmail.com> * Fix unit tests broken with changes to fix integ tests Signed-off-by: Daniel Widdis <widdis@gmail.com> --------- Signed-off-by: Joshua Palis <jpalis@amazon.com> Signed-off-by: Daniel Widdis <widdis@gmail.com> Co-authored-by: Daniel Widdis <widdis@gmail.com> (cherry picked from commit 7986cbf) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Description
This PR achieves multiple things :
FlowFrameworkRestTestCase
which configures the rest admin client for use in integration tests against a security enabled/disabled clusterCurrently the rest integration test cover the following :
TODO : After the
feature/agent_framework
is merged on both Flow Framework main and ML-Commons main, we can add tests for the sample register agent template outlined hereIssues Resolved
#73
Part of #88
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.