-
Notifications
You must be signed in to change notification settings - Fork 892
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improves connection pooling support for AWSSigV4 clients in data sources. #6135
Improves connection pooling support for AWSSigV4 clients in data sources. #6135
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #6135 +/- ##
==========================================
+ Coverage 67.22% 67.26% +0.03%
==========================================
Files 3333 3334 +1
Lines 64565 64639 +74
Branches 10391 10399 +8
==========================================
+ Hits 43404 43478 +74
- Misses 18628 18632 +4
+ Partials 2533 2529 -4
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
3e91572
to
9933938
Compare
src/plugins/data_source/server/auth_registry/authentication_methods_registry.ts
Show resolved
Hide resolved
9933938
to
d273c57
Compare
f72838d
to
bbd290e
Compare
Signed-off-by: Bandini Bhopi <bandinib@amazon.com>
Signed-off-by: Bandini Bhopi <bandinib@amazon.com>
Signed-off-by: Bandini Bhopi <bandinib@amazon.com>
Signed-off-by: Bandini Bhopi <bandinib@amazon.com>
Signed-off-by: Bandini Bhopi <bandinib@amazon.com>
Signed-off-by: Bandini Bhopi <bandinib@amazon.com>
bbd290e
to
18af6c9
Compare
* @param [config.protocol=http:] {String} - The HTTP protocol that this connection will use, can be set to https: | ||
* @class HttpConnector | ||
*/ | ||
import { Config, Credentials } from 'aws-sdk'; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seem like that Config
and Credentials
were not used directly in the code. We may remove this. it is not block, we can do it later.
The backport to
To backport manually, run these commands in your terminal: # Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch-Dashboards/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch-Dashboards/backport-2.x
# Create a new branch
git switch --create backport/backport-6135-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 09f55636e4370b003c42d5bcd8ae93535c230c00
# Push it to GitHub
git push --set-upstream origin backport/backport-6135-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch-Dashboards/backport-2.x Then, create a pull request where the |
…rces. (opensearch-project#6135) * Upgrade @opensearch/opensearch@2.6.0 which inherits AWSSigv4 to .child Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Uses client.child for aws sigv4 connection Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Import http-aws-es connector class Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Refactor client caching mechanism Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Fix UT Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Added UT for client pool Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Revert client pool changes from authentication method Signed-off-by: Bandini Bhopi <bandinib@amazon.com> --------- Signed-off-by: Bandini Bhopi <bandinib@amazon.com>
…rces. (opensearch-project#6135) * Upgrade @opensearch/opensearch@2.6.0 which inherits AWSSigv4 to .child Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Uses client.child for aws sigv4 connection Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Import http-aws-es connector class Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Refactor client caching mechanism Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Fix UT Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Added UT for client pool Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Revert client pool changes from authentication method Signed-off-by: Bandini Bhopi <bandinib@amazon.com> --------- Signed-off-by: Bandini Bhopi <bandinib@amazon.com> (cherry picked from commit 09f5563)
…rces. (opensearch-project#6135) * Upgrade @opensearch/opensearch@2.6.0 which inherits AWSSigv4 to .child Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Uses client.child for aws sigv4 connection Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Import http-aws-es connector class Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Refactor client caching mechanism Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Fix UT Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Added UT for client pool Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Revert client pool changes from authentication method Signed-off-by: Bandini Bhopi <bandinib@amazon.com> --------- Signed-off-by: Bandini Bhopi <bandinib@amazon.com> (cherry picked from commit 09f5563)
…rces. (opensearch-project#6135) * Upgrade @opensearch/opensearch@2.6.0 which inherits AWSSigv4 to .child Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Uses client.child for aws sigv4 connection Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Import http-aws-es connector class Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Refactor client caching mechanism Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Fix UT Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Added UT for client pool Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Revert client pool changes from authentication method Signed-off-by: Bandini Bhopi <bandinib@amazon.com> --------- Signed-off-by: Bandini Bhopi <bandinib@amazon.com> (cherry picked from commit 09f5563)
…rces. (#6135) (#6176) * Upgrade @opensearch/opensearch@2.6.0 which inherits AWSSigv4 to .child Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Uses client.child for aws sigv4 connection Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Import http-aws-es connector class Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Refactor client caching mechanism Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Fix UT Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Added UT for client pool Signed-off-by: Bandini Bhopi <bandinib@amazon.com> * Revert client pool changes from authentication method Signed-off-by: Bandini Bhopi <bandinib@amazon.com> --------- Signed-off-by: Bandini Bhopi <bandinib@amazon.com> (cherry picked from commit 09f5563)
Description
As of today, both opensearch-js client and legacy client does not allow updating credentials for existing client. As a result, every HTTP request to AWS services entails the costly process of creating new client.
This PR solves exactly that problem. We have recently released opensearch-js 2.6.0 client to npm. 2.6.0 client has inherited AwsSigV4 in .child. This PR upgrades the opensearch-js 2.6.0 and refactor data source plugin to create child client if endpoint/node is same. Also this PR imports the HttpAmazonESConnector class and it's UT in OpenSearch-Dashboards repo. This will help us to maintain
HttpAmazonESConnector
proactively.This PR
@opensearch/opensearch@2.6.0
which supports AwsSigV4 in .childIssues Resolved
#6114, #5932, partially resolves #5838
Screenshot
Testing the changes
Check List
yarn test:jest
yarn test:jest_integration