Add integration tests for the remote inference feature #1073

ryanbogan · 2023-07-11T17:31:18Z

Description

Adds integration tests for the remote inference feature

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

ylwu-amzn · 2023-07-11T17:57:23Z

Take a look at the failed cases?

31 tests completed, 4 failed, 12 skipped

Tests with failures:
 - org.opensearch.ml.rest.RestMLRemoteInferenceIT.testOpenAIModerationsModel
 - org.opensearch.ml.rest.RestMLRemoteInferenceIT.testOpenAIEditsModel
 - org.opensearch.ml.rest.RestMLRemoteInferenceIT.testPredictRemoteModel
 - org.opensearch.ml.rest.RestMLRemoteInferenceIT.testCohereGenerateTextModel

ryanbogan · 2023-07-11T17:58:44Z

Those are expected failures until the keys are added to GitHub secrets on the ml-commons repository

ylwu-amzn · 2023-07-11T18:04:22Z

Those are expected failures until the keys are added to GitHub secrets on the ml-commons repository

Got it , thanks. Who will help add the keys to Github secrets?

ryanbogan · 2023-07-11T18:07:57Z

Those are expected failures until the keys are added to GitHub secrets on the ml-commons repository

Got it , thanks. Who will help add the keys to Github secrets?

It has to be someone with admin permissions on the repository.

peterzhuamazon · 2023-07-11T21:06:47Z

I am adding the key sections to the github workflows after talking to @ryanbogan and @ylwu-amzn offline and make required change in their accounts.

peterzhuamazon · 2023-07-11T21:08:01Z

Remove dco.yml as it is replaced by dco apps already.

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

plugin/src/test/java/org/opensearch/ml/rest/RestMLRemoteInferenceIT.java

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Add remote inference integration tests for OpenAI and Cohere Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Change edits model test Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Change loop number in edits test Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Change edits model test again Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Fix spotless Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Refactoring Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Fix typo Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Remove env vars Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> * Removing unnecessary comments Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Rebase with 2.x and remove set up for master key Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Remove print line Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Add throttling check Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Minor change for assert statement Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Test changes Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Uncomment delete connector test Signed-off-by: Ryan Bogan <rbogan@amazon.com> --------- Signed-off-by: Ryan Bogan <rbogan@amazon.com> Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> Co-authored-by: Peter Zhu <zhujiaxi@amazon.com> (cherry picked from commit 3e56ad5)

* Add remote inference integration tests for OpenAI and Cohere Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Change edits model test Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Change loop number in edits test Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Change edits model test again Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Fix spotless Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Refactoring Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Fix typo Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Remove env vars Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> * Removing unnecessary comments Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Rebase with 2.x and remove set up for master key Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Remove print line Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Add throttling check Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Minor change for assert statement Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Test changes Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Uncomment delete connector test Signed-off-by: Ryan Bogan <rbogan@amazon.com> --------- Signed-off-by: Ryan Bogan <rbogan@amazon.com> Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> Co-authored-by: Peter Zhu <zhujiaxi@amazon.com> (cherry picked from commit 3e56ad5) Signed-off-by: Yaliang Wu <ylwu@amazon.com>

* Add remote inference integration tests for OpenAI and Cohere * Change edits model test * Change loop number in edits test * Change edits model test again * Fix spotless * Refactoring * Fix typo * Remove env vars * Removing unnecessary comments * Rebase with 2.x and remove set up for master key * Remove print line * Add throttling check * Minor change for assert statement * Test changes * Uncomment delete connector test --------- (cherry picked from commit 3e56ad5) Signed-off-by: Ryan Bogan <rbogan@amazon.com> Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> Signed-off-by: Yaliang Wu <ylwu@amazon.com> Co-authored-by: Ryan Bogan <10944539+ryanbogan@users.noreply.github.com> Co-authored-by: Peter Zhu <zhujiaxi@amazon.com>

…oject#1073) * Add remote inference integration tests for OpenAI and Cohere Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Change edits model test Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Change loop number in edits test Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Change edits model test again Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Fix spotless Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Refactoring Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Fix typo Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Remove env vars Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> * Removing unnecessary comments Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Rebase with 2.x and remove set up for master key Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Remove print line Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Add throttling check Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Minor change for assert statement Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Test changes Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Uncomment delete connector test Signed-off-by: Ryan Bogan <rbogan@amazon.com> --------- Signed-off-by: Ryan Bogan <rbogan@amazon.com> Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> Co-authored-by: Peter Zhu <zhujiaxi@amazon.com>

* Add remote inference integration tests for OpenAI and Cohere Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Change edits model test Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Change loop number in edits test Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Change edits model test again Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Fix spotless Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Refactoring Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Fix typo Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Remove env vars Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> * Removing unnecessary comments Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Rebase with 2.x and remove set up for master key Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Remove print line Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Add throttling check Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Minor change for assert statement Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Test changes Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Uncomment delete connector test Signed-off-by: Ryan Bogan <rbogan@amazon.com> --------- Signed-off-by: Ryan Bogan <rbogan@amazon.com> Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> Co-authored-by: Peter Zhu <zhujiaxi@amazon.com>

ryanbogan added 7 commits July 11, 2023 07:44

Add remote inference integration tests for OpenAI and Cohere

8ffe4cb

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

Change edits model test

fe21a5f

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

Change loop number in edits test

d10e4d3

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

Change edits model test again

2c4fac6

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

Fix spotless

505087a

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

Refactoring

6ce0f7c

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

Fix typo

e2d28f5

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

ylwu-amzn previously approved these changes Jul 11, 2023

View reviewed changes

peterzhuamazon dismissed ylwu-amzn’s stale review via 38d0195 July 11, 2023 21:08

peterzhuamazon force-pushed the write_remote_inference_integ_tests branch from 51d9f25 to e2d28f5 Compare July 11, 2023 23:37

peterzhuamazon added backport 2.9 release v2.9.0 v2.9.0 test ci infra labels Jul 12, 2023

peterzhuamazon mentioned this pull request Jul 12, 2023

Add test keys to CI related workflows #1099

Merged

5 tasks

peterzhuamazon had a problem deploying to ml-commons-cicd-env July 12, 2023 18:42 — with GitHub Actions Failure

peterzhuamazon had a problem deploying to ml-commons-cicd-env July 12, 2023 18:42 — with GitHub Actions Error

peterzhuamazon had a problem deploying to ml-commons-cicd-env July 12, 2023 18:42 — with GitHub Actions Failure

peterzhuamazon had a problem deploying to ml-commons-cicd-env July 12, 2023 18:48 — with GitHub Actions Failure

peterzhuamazon had a problem deploying to ml-commons-cicd-env July 12, 2023 18:48 — with GitHub Actions Error

ryanbogan temporarily deployed to ml-commons-cicd-env July 12, 2023 23:34 — with GitHub Actions Inactive

Minor change for assert statement

dc027bb

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

ryanbogan temporarily deployed to ml-commons-cicd-env July 12, 2023 23:34 — with GitHub Actions Inactive

peterzhuamazon mentioned this pull request Jul 12, 2023

[RELEASE] Release version 2.9.0 opensearch-project/opensearch-build#3616

Closed

42 tasks

Test changes

0ca24cf

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

ryanbogan temporarily deployed to ml-commons-cicd-env July 12, 2023 23:48 — with GitHub Actions Inactive

Zhangxunmt reviewed Jul 12, 2023

View reviewed changes

plugin/src/test/java/org/opensearch/ml/rest/RestMLRemoteInferenceIT.java Outdated Show resolved Hide resolved

ryanbogan added 2 commits July 13, 2023 00:04

Merge branch '2.x' into write_remote_inference_integ_tests

f913882

Uncomment delete connector test

d43f3a4

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

ryanbogan temporarily deployed to ml-commons-cicd-env July 13, 2023 00:06 — with GitHub Actions Inactive

Zhangxunmt approved these changes Jul 13, 2023

View reviewed changes

ylwu-amzn approved these changes Jul 13, 2023

View reviewed changes

ylwu-amzn merged commit 3e56ad5 into opensearch-project:2.x Jul 13, 2023
9 checks passed

opensearch-trigger-bot bot mentioned this pull request Jul 13, 2023

[Backport 2.9] Add integration tests for the remote inference feature #1109

Merged

peterzhuamazon mentioned this pull request Jul 17, 2023

Migrate Keys from ml-commons AWS secret manager opensearch-project/opensearch-build#3793

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add integration tests for the remote inference feature #1073

Add integration tests for the remote inference feature #1073

ryanbogan commented Jul 11, 2023

ylwu-amzn commented Jul 11, 2023

ryanbogan commented Jul 11, 2023

ylwu-amzn commented Jul 11, 2023

ryanbogan commented Jul 11, 2023

peterzhuamazon commented Jul 11, 2023 •

edited

Loading

peterzhuamazon commented Jul 11, 2023

Add integration tests for the remote inference feature #1073

Add integration tests for the remote inference feature #1073

Conversation

ryanbogan commented Jul 11, 2023

Description

Check List

ylwu-amzn commented Jul 11, 2023

ryanbogan commented Jul 11, 2023

ylwu-amzn commented Jul 11, 2023

ryanbogan commented Jul 11, 2023

peterzhuamazon commented Jul 11, 2023 • edited Loading

peterzhuamazon commented Jul 11, 2023

peterzhuamazon commented Jul 11, 2023 •

edited

Loading