Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactored the RSA to make it more extensible #10254

Merged
merged 8 commits into from
Mar 14, 2024

Conversation

shiv0408
Copy link
Member

@shiv0408 shiv0408 commented Sep 27, 2023

Description

In this PR, we are refactoring the ReplicaShardAllocator code to make it more reusable as we bring in Batch Allocator in #8992.

We have broken this function to multiple functions, so that part of codes can be reused and only a single function need to be overwritten in child class to implement the Batch allocator.

public AllocateUnassignedDecision makeAllocationDecision(
final ShardRouting unassignedShard,
final RoutingAllocation allocation,
final Logger logger
) {

We have also created NodeShardStore and NodeShardStores to encapsulate the data that needs to be passed around in these functions.

Related Issues

Resolves #8960

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Shivansh Arora <shivansh.arora@protonmail.com>
@github-actions github-actions bot added bug Something isn't working Cluster Manager labels Sep 27, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Sep 27, 2023

Compatibility status:

Checks if related components are compatible with change e2e8f38

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/flow-framework.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/performance-analyzer.git]

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@shwetathareja
Copy link
Member

It is failing during build as it has dependency on other PR for merge.

@github-actions github-actions bot added the bug Something isn't working label Mar 12, 2024
# Conflicts:
#	server/src/main/java/org/opensearch/gateway/ReplicaShardAllocator.java
Copy link
Contributor

❌ Gradle check result for c7d1b88: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Shivansh Arora <hishiv@amazon.com>
@shwetathareja shwetathareja removed the bug Something isn't working label Mar 14, 2024
Copy link
Contributor

✅ Gradle check result for e2e8f38: SUCCESS

@github-actions github-actions bot added the bug Something isn't working label Mar 14, 2024
Copy link

codecov bot commented Mar 14, 2024

Codecov Report

Attention: Patch coverage is 69.86301% with 22 lines in your changes are missing coverage. Please review.

Project coverage is 71.41%. Comparing base (b15cb0c) to head (e2e8f38).
Report is 27 commits behind head on main.

Files Patch % Lines
.../org/opensearch/gateway/ReplicaShardAllocator.java 69.86% 8 Missing and 14 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #10254      +/-   ##
============================================
- Coverage     71.42%   71.41%   -0.01%     
- Complexity    59978    60118     +140     
============================================
  Files          4985     4991       +6     
  Lines        282275   282635     +360     
  Branches      40946    40990      +44     
============================================
+ Hits         201603   201846     +243     
- Misses        63999    64067      +68     
- Partials      16673    16722      +49     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@shwetathareja shwetathareja requested review from shwetathareja and removed request for setiah March 14, 2024 12:19
@shiv0408
Copy link
Member Author

Thanks for the approval @shwetathareja!

@shwetathareja shwetathareja merged commit 3a9c20b into opensearch-project:main Mar 14, 2024
36 of 37 checks passed
@shiv0408 shiv0408 deleted the rsa-refactor branch March 14, 2024 12:31
rayshrey pushed a commit to rayshrey/OpenSearch that referenced this pull request Mar 18, 2024
* Refactored the RSA to make it more extensible

Signed-off-by: Shivansh Arora <hishiv@amazon.com>
@amkhar amkhar added the backport 2.x Backport to 2.x branch label Mar 19, 2024
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-10254-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 3a9c20ba394121e73226b48d3e46bf3104a2bde7
# Push it to GitHub
git push --set-upstream origin backport/backport-10254-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-10254-to-2.x.

@shiv0408 shiv0408 added enhancement Enhancement or improvement to existing feature or request backport 2.x Backport to 2.x branch and removed bug Something isn't working backport 2.x Backport to 2.x branch backport-failed labels Mar 19, 2024
opensearch-trigger-bot bot pushed a commit that referenced this pull request Mar 19, 2024
* Refactored the RSA to make it more extensible

Signed-off-by: Shivansh Arora <hishiv@amazon.com>
(cherry picked from commit 3a9c20b)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
shwetathareja pushed a commit that referenced this pull request Mar 20, 2024
* Refactored the RSA to make it more extensible


(cherry picked from commit 3a9c20b)

Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
shiv0408 added a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
* Refactored the RSA to make it more extensible

Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch Cluster Manager enhancement Enhancement or improvement to existing feature or request skip-changelog
Projects
Status: ✅ Done
Development

Successfully merging this pull request may close these issues.

[Enhancement] Add a batch allocator for building allocation decisions for multiple primary & replica shards
5 participants