Change httpclient to async #1958
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
zane-neo
requested review from
b4sjoo,
dhrubo-os,
jngz-es,
model-collapse,
rbhavna,
ylwu-amzn,
Zhangxunmt,
austintlee and
HenryL27
as code owners
zane-neo
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Failure
zane-neo
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Failure
zane-neo
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Error
zane-neo
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Failure
zane-neo
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Error
zane-neo
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Error
zane-neo
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Error
zane-neo
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Error
zane-neo
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Failure
zane-neo
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Error
zane-neo
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Error
zane-neo
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Failure
ylwu-amzn
reviewed
Jan 31, 2024
ml-algorithms/src/test/java/org/opensearch/ml/engine/httpclient/MLHttpClientFactoryTests.java
Outdated
Show resolved
Hide resolved
ylwu-amzn
reviewed
Jan 31, 2024
...lgorithms/src/main/java/org/opensearch/ml/engine/algorithms/remote/AwsConnectorExecutor.java
Outdated
Show resolved
Hide resolved
ylwu-amzn
reviewed
Jan 31, 2024
ml-algorithms/src/main/java/org/opensearch/ml/engine/Predictable.java
Outdated
Show resolved
Hide resolved
ylwu-amzn
reviewed
Jan 31, 2024
.../src/main/java/org/opensearch/ml/engine/algorithms/remote/MLSdkAsyncHttpResponseHandler.java
Outdated
Show resolved
Hide resolved
zane-neo
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Failure
zane-neo
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Error
zane-neo
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Failure
zane-neo
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Failure
zane-neo
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Error
zane-neo
temporarily deployed
to
ml-commons-cicd-env
GitHub Actions
Inactive
zane-neo
temporarily deployed
to
ml-commons-cicd-env
GitHub Actions
Inactive
zane-neo
temporarily deployed
to
ml-commons-cicd-env
GitHub Actions
Inactive
Signed-off-by: zane-neo <zaniu@amazon.com>
zane-neo
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Error
zane-neo
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Error
zane-neo
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Failure
zane-neo
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Error
zane-neo
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Failure
zane-neo
had a problem deploying
to
ml-commons-cicd-env
GitHub Actions
Error
Signed-off-by: zane-neo <zaniu@amazon.com>
zane-neo
temporarily deployed
to
ml-commons-cicd-env
GitHub Actions
Inactive
zane-neo
temporarily deployed
to
ml-commons-cicd-env
GitHub Actions
Inactive
zane-neo
temporarily deployed
to
ml-commons-cicd-env
GitHub Actions
Inactive
zane-neo
temporarily deployed
to
ml-commons-cicd-env
GitHub Actions
Inactive
zane-neo
temporarily deployed
to
ml-commons-cicd-env
GitHub Actions
Inactive
zane-neo
temporarily deployed
to
ml-commons-cicd-env
GitHub Actions
Inactive
zane-neo
temporarily deployed
to
ml-commons-cicd-env
GitHub Actions
Inactive
zane-neo
temporarily deployed
to
ml-commons-cicd-env
GitHub Actions
Inactive
zane-neo
temporarily deployed
to
ml-commons-cicd-env
GitHub Actions
Inactive
ylwu-amzn
approved these changes
Apr 30, 2024
xinyual
approved these changes
Apr 30, 2024
|
The backport to To backport manually, run these commands in your terminal: # Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.x 2.x
# Navigate to the new working tree
cd .worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-1958-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 94a113da8a2d1e28a84ba7a422b43287ac00448e
# Push it to GitHub
git push --set-upstream origin backport/backport-1958-to-2.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.xThen, create a pull request where the |
zane-neo
added a commit
to zane-neo/ml-commons
that referenced
this pull request
Apr 30, 2024
* Change httpclient from sync to async Signed-off-by: zane-neo <zaniu@amazon.com> * Change from CRTAsyncHttpClient to NettyAsyncHttpClient Signed-off-by: zane-neo <zaniu@amazon.com> * Add publisher to request Signed-off-by: zane-neo <zaniu@amazon.com> * Change sync httpclient to async Signed-off-by: zane-neo <zaniu@amazon.com> * Handle error case and return error response in actionLListener Signed-off-by: zane-neo <zaniu@amazon.com> * Fix no response when exception Signed-off-by: zane-neo <zaniu@amazon.com> * Add content type header Signed-off-by: zane-neo <zaniu@amazon.com> * Fix issues found in functional test Signed-off-by: zane-neo <zaniu@amazon.com> * Fix no response issue in functional test Signed-off-by: zane-neo <zaniu@amazon.com> * fix default step size error Signed-off-by: zane-neo <zaniu@amazon.com> * Add track inference duration for async httpclient Signed-off-by: zane-neo <zaniu@amazon.com> * Change client appsec highlight issues implementation for async httpclient Signed-off-by: zane-neo <zaniu@amazon.com> * Add UTs Signed-off-by: zane-neo <zaniu@amazon.com> * Add UTs Signed-off-by: zane-neo <zaniu@amazon.com> * Remove unused file Signed-off-by: zane-neo <zaniu@amazon.com> * Add UTs Signed-off-by: zane-neo <zaniu@amazon.com> * format code Signed-off-by: zane-neo <zaniu@amazon.com> * Change error code to honor remote service error code Signed-off-by: zane-neo <zaniu@amazon.com> * Add more UTs Signed-off-by: zane-neo <zaniu@amazon.com> * Change SSRF code to make it correct for return error stattus Signed-off-by: zane-neo <zaniu@amazon.com> * Fix failure UTs and add more UTs Signed-off-by: zane-neo <zaniu@amazon.com> * Fix failure ITs Signed-off-by: zane-neo <zaniu@amazon.com> * format code Signed-off-by: zane-neo <zaniu@amazon.com> * Fix partial success response not correct issue Signed-off-by: zane-neo <zaniu@amazon.com> * format code Signed-off-by: zane-neo <zaniu@amazon.com> * Fix failure ITs Signed-off-by: zane-neo <zaniu@amazon.com> * Add more UTs to increase code coverage Signed-off-by: zane-neo <zaniu@amazon.com> * Change url regex Signed-off-by: zane-neo <zaniu@amazon.com> * Address comments Signed-off-by: zane-neo <zaniu@amazon.com> * format code Signed-off-by: zane-neo <zaniu@amazon.com> * Fix failure UTs Signed-off-by: zane-neo <zaniu@amazon.com> * Add UT for httpclientFactory throw exception when creating httpclient Signed-off-by: zane-neo <zaniu@amazon.com> * format code Signed-off-by: zane-neo <zaniu@amazon.com> * Address comments and add modelTensor status code Signed-off-by: zane-neo <zaniu@amazon.com> * Address comments Signed-off-by: zane-neo <zaniu@amazon.com> * format code Signed-off-by: zane-neo <zaniu@amazon.com> * Add status code to process error response Signed-off-by: zane-neo <zaniu@amazon.com> * format code Signed-off-by: zane-neo <zaniu@amazon.com> * Rebase main after connector level http parameter support Signed-off-by: zane-neo <zaniu@amazon.com> * Fix UT Signed-off-by: zane-neo <zaniu@amazon.com> * Change error message when remote model return empty and chaange the behavior when one of the requests fails Signed-off-by: zane-neo <zaniu@amazon.com> * Add comments\ Signed-off-by: zane-neo <zaniu@amazon.com> * Remove redundant builder and change the error code check Signed-off-by: zane-neo <zaniu@amazon.com> * format code Signed-off-by: zane-neo <zaniu@amazon.com> * Add more UTs for throw exception cases Signed-off-by: zane-neo <zaniu@amazon.com> * fix failure UTs Signed-off-by: zane-neo <zaniu@amazon.com> * format code Signed-off-by: zane-neo <zaniu@amazon.com> * Fix test cases since the error message change Signed-off-by: zane-neo <zaniu@amazon.com> * Rebase code Signed-off-by: zane-neo <zaniu@amazon.com> * fix failure IT Signed-off-by: zane-neo <zaniu@amazon.com> * Add more UTs Signed-off-by: zane-neo <zaniu@amazon.com> * Fix duplicate response to client issue Signed-off-by: zane-neo <zaniu@amazon.com> * fix duplicate response in channel Signed-off-by: zane-neo <zaniu@amazon.com> * change code for all successfully responses case Signed-off-by: zane-neo <zaniu@amazon.com> * Address comments Signed-off-by: zane-neo <zaniu@amazon.com> * format code Signed-off-by: zane-neo <zaniu@amazon.com> * Increase nio httpclient version to fix vulnerbility Signed-off-by: zane-neo <zaniu@amazon.com> * Change validate localhost logic to same with existing code Signed-off-by: zane-neo <zaniu@amazon.com> * change method signature to private Signed-off-by: zane-neo <zaniu@amazon.com> * format code Signed-off-by: zane-neo <zaniu@amazon.com> --------- Signed-off-by: zane-neo <zaniu@amazon.com>
zane-neo
added a commit
that referenced
this pull request
Apr 30, 2024
* Change httpclient from sync to async * Change from CRTAsyncHttpClient to NettyAsyncHttpClient * Add publisher to request * Change sync httpclient to async * Handle error case and return error response in actionLListener * Fix no response when exception * Add content type header * Fix issues found in functional test * Fix no response issue in functional test * fix default step size error * Add track inference duration for async httpclient * Change client appsec highlight issues implementation for async httpclient * Add UTs * Add UTs * Remove unused file * Add UTs * format code * Change error code to honor remote service error code * Add more UTs * Change SSRF code to make it correct for return error stattus * Fix failure UTs and add more UTs * Fix failure ITs * format code * Fix partial success response not correct issue * format code * Fix failure ITs * Add more UTs to increase code coverage * Change url regex * Address comments * format code * Fix failure UTs * Add UT for httpclientFactory throw exception when creating httpclient * format code * Address comments and add modelTensor status code * Address comments * format code * Add status code to process error response * format code * Rebase main after connector level http parameter support * Fix UT * Change error message when remote model return empty and chaange the behavior when one of the requests fails * Add comments\ * Remove redundant builder and change the error code check * format code * Add more UTs for throw exception cases * fix failure UTs * format code * Fix test cases since the error message change * Rebase code * fix failure IT * Add more UTs * Fix duplicate response to client issue * fix duplicate response in channel * change code for all successfully responses case * Address comments * format code * Increase nio httpclient version to fix vulnerbility * Change validate localhost logic to same with existing code * change method signature to private * format code --------- Signed-off-by: zane-neo <zaniu@amazon.com>
dhrubo-os
pushed a commit
to dhrubo-os/ml-commons
that referenced
this pull request
May 17, 2024
…ect#2375) * Change httpclient from sync to async * Change from CRTAsyncHttpClient to NettyAsyncHttpClient * Add publisher to request * Change sync httpclient to async * Handle error case and return error response in actionLListener * Fix no response when exception * Add content type header * Fix issues found in functional test * Fix no response issue in functional test * fix default step size error * Add track inference duration for async httpclient * Change client appsec highlight issues implementation for async httpclient * Add UTs * Add UTs * Remove unused file * Add UTs * format code * Change error code to honor remote service error code * Add more UTs * Change SSRF code to make it correct for return error stattus * Fix failure UTs and add more UTs * Fix failure ITs * format code * Fix partial success response not correct issue * format code * Fix failure ITs * Add more UTs to increase code coverage * Change url regex * Address comments * format code * Fix failure UTs * Add UT for httpclientFactory throw exception when creating httpclient * format code * Address comments and add modelTensor status code * Address comments * format code * Add status code to process error response * format code * Rebase main after connector level http parameter support * Fix UT * Change error message when remote model return empty and chaange the behavior when one of the requests fails * Add comments\ * Remove redundant builder and change the error code check * format code * Add more UTs for throw exception cases * fix failure UTs * format code * Fix test cases since the error message change * Rebase code * fix failure IT * Add more UTs * Fix duplicate response to client issue * fix duplicate response in channel * change code for all successfully responses case * Address comments * format code * Increase nio httpclient version to fix vulnerbility * Change validate localhost logic to same with existing code * change method signature to private * format code --------- Signed-off-by: zane-neo <zaniu@amazon.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Change sync httpclient to async to improve remote inference performance
Issues Resolved
#1839
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.