Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use async client to delete scroll and pit for OpenSearch as workaroun… #3338

Merged
merged 2 commits into from
Sep 18, 2023

Conversation

graytaylor0
Copy link
Member

@graytaylor0 graytaylor0 commented Sep 14, 2023

…d for bug in client

Description

The OpenSearch Java Client has an issue where delete requests with sigv4 are not signed correctly (opensearch-project/opensearch-java#521).

As a workaround, this change constructs an async OpenSearch client to make only the delete calls for scroll and pit

Check List

  • New functionality includes testing.
  • New functionality has a documentation issue. Please link to it in this PR.
    • New functionality has javadoc added
  • Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

…d for bug in client

Signed-off-by: Taylor Gray <tylgry@amazon.com>
@@ -126,7 +130,7 @@ public SearchWithSearchAfterResults searchWithPit(final SearchPointInTimeRequest
@Override
public void deletePit(final DeletePointInTimeRequest deletePointInTimeRequest) {
try {
final DeletePitResponse deletePitResponse = openSearchClient.deletePit(DeletePitRequest.of(builder -> builder.pitId(Collections.singletonList(deletePointInTimeRequest.getPitId()))));
final DeletePitResponse deletePitResponse = openSearchAsyncClient.deletePit(DeletePitRequest.of(builder -> builder.pitId(Collections.singletonList(deletePointInTimeRequest.getPitId()))));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this is still using the existing synchronous method, but the implementation is the "async" client. Right? Why not use only the async client (but in the synchronous way)?

@dlvenable
Copy link
Member

Also, this comment indicates that the problem may be the Apache HTTP client.

Can we skip using the async client and just change to use Netty underneath?

@graytaylor0
Copy link
Member Author

Also, this comment indicates that the problem may be the Apache HTTP client.

Can we skip using the async client and just change to use Netty underneath?

The only netty client is the NettyNioAsyncHttpClient. Are you suggesting using this one?

@dlvenable
Copy link
Member

The only netty client is the NettyNioAsyncHttpClient. Are you suggesting using this one?

Yes.

As I'm reading this, we have two clients - sync and async. We use them both for synchronous APIs. So why not just change the existing sync client to use NettyNioAsyncHttpClient instead of ApacheHttpClient? Then we could have a single client used everywhere.

Signed-off-by: Taylor Gray <tylgry@amazon.com>
Copy link
Member

@dlvenable dlvenable left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! I'm glad to see that simplification works.

@graytaylor0
Copy link
Member Author

Nice! I'm glad to see that simplification works.

Me as well! Good call

@graytaylor0 graytaylor0 merged commit 6286388 into opensearch-project:main Sep 18, 2023
22 of 24 checks passed
asifsmohammed pushed a commit to asifsmohammed/data-prepper that referenced this pull request Sep 27, 2023
opensearch-project#3338)

Use async client to delete scroll and pit for OpenSearch as workaround for bug in client

Signed-off-by: Taylor Gray <tylgry@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants