Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delete API for weighted round robin search routing #4400

Merged
merged 6 commits into from
Oct 13, 2022

Conversation

anshu1106
Copy link
Contributor

@anshu1106 anshu1106 commented Sep 3, 2022

Description

This pull request adds DELTE api to delete weights for weighted routing policy. This in turn disable the deature.

Issues Resolved

2859
3639

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@anshu1106 anshu1106 requested review from a team and reta as code owners September 3, 2022 16:28
@github-actions
Copy link
Contributor

github-actions bot commented Sep 3, 2022

Gradle Check (Jenkins) Run Completed with:

}

}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we remove the weights and then assert again that requests are equally distributed ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, any pointers on which stat I can assert on for request distribution?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry missed this. You can look for node/stats API call for this.

*
* @opensearch.internal
*/
public class TransportGetWRRWeightsAction extends TransportClusterManagerNodeReadAction<
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we extend HandledTransportAction instead of TransportClusterManagerNodeReadAction ? I do see that TransportClusterManagerNodeReadAction are used for GET actions which needs cluster state . but just thinking about the benefits/cost for the same .

Copy link
Collaborator

@gbbafna gbbafna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delete API LGTM.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 7, 2022

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Oct 6, 2022

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Oct 6, 2022

Gradle Check (Jenkins) Run Completed with:

@codecov-commenter
Copy link

codecov-commenter commented Oct 6, 2022

Codecov Report

Merging #4400 (6a4c8d5) into main (d15795a) will increase coverage by 0.12%.
The diff coverage is 44.81%.

@@             Coverage Diff              @@
##               main    #4400      +/-   ##
============================================
+ Coverage     70.66%   70.78%   +0.12%     
- Complexity    57578    57698     +120     
============================================
  Files          4661     4675      +14     
  Lines        276662   276918     +256     
  Branches      40325    40347      +22     
============================================
+ Hits         195501   196016     +515     
+ Misses        64926    64684     -242     
+ Partials      16235    16218      -17     
Impacted Files Coverage Δ
.../delete/DeleteDecommissionStateRequestBuilder.java 0.00% <0.00%> (ø)
...te/ClusterDeleteWeightedRoutingRequestBuilder.java 0.00% <0.00%> (ø)
...apshots/restore/RestoreSnapshotRequestBuilder.java 15.78% <0.00%> (-0.88%) ⬇️
.../org/opensearch/client/support/AbstractClient.java 32.30% <0.00%> (-0.63%) ⬇️
.../java/org/opensearch/common/util/FeatureFlags.java 50.00% <ø> (ø)
...ateway/TransportNodesListGatewayStartedShards.java 51.00% <0.00%> (-0.68%) ⬇️
...h/index/shard/RemoveCorruptedShardDataCommand.java 84.87% <ø> (+3.36%) ⬆️
...java/org/opensearch/index/shard/StoreRecovery.java 67.88% <ø> (+0.01%) ⬆️
...h/index/store/InMemoryRemoteSnapshotDirectory.java 0.00% <0.00%> (ø)
...in/java/org/opensearch/indices/IndicesService.java 69.65% <0.00%> (-0.25%) ⬇️
... and 482 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@github-actions
Copy link
Contributor

github-actions bot commented Oct 6, 2022

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Oct 6, 2022

Gradle Check (Jenkins) Run Completed with:

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
Comment on lines 86 to 89
ActionListener.delegateFailure(
listener,
(delegatedListener, response) -> {
delegatedListener.onResponse(new ClusterDeleteWeightedRoutingResponse(response.isAcknowledged()));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to delegate this? The service method definition can be changed to ClusterDeleteWRResponse?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed delegatedListener

Comment on lines +117 to +120
logger.info("Deleting weighted routing metadata from the cluster state");
Metadata.Builder mdBuilder = Metadata.builder(currentState.metadata());
mdBuilder.removeCustom(WeightedRoutingMetadata.TYPE);
return ClusterState.builder(currentState).metadata(mdBuilder).build();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we just wipe off the metadata will it internally handle to use ARS strategy? Or what will happen to subsequent requests?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, subsequent requests will use ARS or attribute based routing. For multi-az clusters, currently ARS is not supported.


@Override
public void onFailure(String source, Exception e) {
logger.warn(() -> new ParameterizedMessage("failed to remove weighted routing metadata from cluster state [{}]", e));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the msg looks like is exoecting a cluster state but provided an exception.
We can make this logger.error and pass the exception in it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

modified

@github-actions
Copy link
Contributor

github-actions bot commented Oct 6, 2022

Gradle Check (Jenkins) Run Completed with:

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
@github-actions
Copy link
Contributor

github-actions bot commented Oct 6, 2022

Gradle Check (Jenkins) Run Completed with:

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
@github-actions
Copy link
Contributor

github-actions bot commented Oct 6, 2022

Gradle Check (Jenkins) Run Completed with:

Comment on lines 126 to 131
logger.error(
() -> new ParameterizedMessage(
"failed to remove weighted routing metadata from cluster state with an exception [{}]",
e
)
);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if there's no other parameter we can keep it like - logger.error(msg, e)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed

@github-actions
Copy link
Contributor

github-actions bot commented Oct 6, 2022

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Oct 6, 2022

Gradle Check (Jenkins) Run Completed with:

@anshu1106
Copy link
Contributor Author

Seeing unrelated test failures

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
@github-actions
Copy link
Contributor

github-actions bot commented Oct 7, 2022

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Oct 7, 2022

Gradle Check (Jenkins) Run Completed with:

Copy link
Member

@imRishN imRishN left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @anshu1106 for the changes

Comment on lines +36 to +39
@Override
public List<Route> routes() {
return singletonList(new Route(DELETE, "/_cluster/routing/awareness/weights"));
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to support explicit attribute deletion?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, but this will require few changes in the way data is persisted in cluster state as well. Create an issue to work on this #4747

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@Bukhtawar Bukhtawar merged commit 18f1fa3 into opensearch-project:main Oct 13, 2022
anshu1106 added a commit to anshu1106/OpenSearch that referenced this pull request Nov 1, 2022
…t#4400)

* Delete API for weighted round robin search routing

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
@anshu1106 anshu1106 mentioned this pull request Nov 1, 2022
6 tasks
anshu1106 added a commit to anshu1106/OpenSearch that referenced this pull request Nov 3, 2022
…t#4400)

* Delete API for weighted round robin search routing

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
Bukhtawar pushed a commit that referenced this pull request Nov 3, 2022
* Weighted round-robin scheduling policy for shard coordination traffic… (#4241)

* Add PUT api to update shard routing weights (#4272)

* Add GET api to get shard routing weights (#4275)

* Fix weighted routing metadata deserialization error during node restart (#4691)

* Delete API for weighted round robin search routing (#4400)

* Mark apis experimental

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
ashking94 pushed a commit to ashking94/OpenSearch that referenced this pull request Nov 7, 2022
…t#4400)

* Delete API for weighted round robin search routing

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants