Reject Resize index requests. (i.e, split, shrink and… #12686

astute-decipher · 2024-03-15T07:34:02Z

… clone), While DocRep to SegRep migration is in progress.

Description

We want to disallow some APIs' which are involved in Index resizing and shard relocation, when Document-Replication type to Segment-Replication type migration is happening.

Always allow if source Index setting/metadata has SETTING_REMOTE_STORE_ENABLED as true.
Disallow If cluster settings has REMOTE_STORE_COMPATIBILITY_MODE_SETTING as MIXED and MIGRATION_DIRECTION_SETTING as REMOTE_STORE and Condition 1 is not met.

Related Issues

Resolves #12976

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
~~Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)~~
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)
~~Public documentation issue/PR created~~

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

… clone, While migration is in progress Signed-off-by: Shubh Sahu <shubhvs@amazon.com>

github-actions · 2024-03-15T07:41:17Z

❌ Gradle check result for c9b0487: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Shubh Sahu <shubhvs@amazon.com>

github-actions · 2024-03-15T07:55:59Z

Compatibility status:

Checks if related components are compatible with change 06ae8c0

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/flow-framework.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/performance-analyzer.git]

github-actions · 2024-03-15T08:10:47Z

❌ Gradle check result for 4548951: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

…tion Signed-off-by: Shubh Sahu <shubhvs@amazon.com>

github-actions · 2024-03-18T08:48:53Z

❌ Gradle check result for 6672cbe:

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Shubh Sahu <shubhvs@amazon.com>

github-actions · 2024-03-19T12:40:07Z

❕ Gradle check result for ace7752: UNSTABLE

TEST FAILURES:

      1 org.opensearch.search.SearchWeightedRoutingIT.testShardRoutingWithNetworkDisruption_FailOpenEnabled

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

codecov · 2024-03-19T12:40:23Z

Codecov Report

Attention: Patch coverage is 46.15385% with 7 lines in your changes are missing coverage. Please review.

Project coverage is 71.47%. Comparing base (b15cb0c) to head (06ae8c0).
Report is 132 commits behind head on main.

Files	Patch %	Lines
...on/admin/indices/shrink/TransportResizeAction.java	46.15%	5 Missing and 2 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #12686      +/-   ##
============================================
+ Coverage     71.42%   71.47%   +0.04%     
- Complexity    59978    60431     +453     
============================================
  Files          4985     5026      +41     
  Lines        282275   284463    +2188     
  Branches      40946    41200     +254     
============================================
+ Hits         201603   203306    +1703     
- Misses        63999    64340     +341     
- Partials      16673    16817     +144

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

CHANGELOG.md

server/src/main/java/org/opensearch/action/admin/indices/shrink/TransportResizeAction.java

Signed-off-by: Shubh Sahu <shubhvs@amazon.com>

github-actions · 2024-04-01T09:42:57Z

❌ Gradle check result for f262296: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Shubh Sahu <shubhvs@amazon.com>

github-actions · 2024-04-01T10:39:30Z

❌ Gradle check result for 872f742: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

...rc/internalClusterTest/java/org/opensearch/remotemigration/ResizeIndexMigrationTestCase.java

server/src/main/java/org/opensearch/action/admin/indices/shrink/TransportResizeAction.java

shourya035

Changes mostly LGTM. Please address the minor comments

github-actions · 2024-04-02T20:51:45Z

❌ Gradle check result for abf1be0: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Shubh Sahu <shubhvs@amazon.com>

github-actions · 2024-04-03T07:54:53Z

❌ Gradle check result for 60c77ea: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

… into serengity_personel

github-actions · 2024-04-03T09:52:35Z

❕ Gradle check result for 2789f8c: UNSTABLE

TEST FAILURES:

      1 org.opensearch.gateway.RecoveryFromGatewayIT.testShardStoreFetchMultiNodeMultiIndexesUsingBatchAction
      1 org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

server/src/main/java/org/opensearch/action/admin/indices/shrink/TransportResizeAction.java

Signed-off-by: Shubh Sahu <shubhvs@amazon.com>

github-actions · 2024-04-04T05:20:30Z

❌ Gradle check result for c4f2530: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

gbbafna · 2024-04-04T05:27:06Z

Changes looks good to me . Please fix spotless issues and fix the build

Signed-off-by: Shubh Sahu <shubhvs@amazon.com>

… into serengity_personel

github-actions · 2024-04-04T06:11:25Z

❌ Gradle check result for c99e50b: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2024-04-04T07:55:07Z

✅ Gradle check result for c99e50b: SUCCESS

Signed-off-by: Shubh Sahu <shubhvs@amazon.com>

github-actions · 2024-04-04T09:36:02Z

✅ Gradle check result for 06ae8c0: SUCCESS

opensearch-trigger-bot · 2024-04-04T10:40:53Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-12686-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 9ab31f55e88cc2d023cad6e60a15a0e0e87dac38
# Push it to GitHub
git push --set-upstream origin backport/backport-12686-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-12686-to-2.x.

…13147) Signed-off-by: Shubh Sahu <shubhvs@amazon.com> (cherry picked from commit 9ab31f5)

…h-project#12686) Signed-off-by: Shubh Sahu <shubhvs@amazon.com> Signed-off-by: Shivansh Arora <hishiv@amazon.com>

Adding checks to reject Resize index requests. i.e, split, shrink and…

c9b0487

… clone, While migration is in progress Signed-off-by: Shubh Sahu <shubhvs@amazon.com>

astute-decipher requested review from Bukhtawar, gbbafna, sachinpkale and shourya035 March 15, 2024 07:45

Adding issue to changelog.md file

4548951

Signed-off-by: Shubh Sahu <shubhvs@amazon.com>

Some structural changes and added UTs to cover all possible configura…

6672cbe

…tion Signed-off-by: Shubh Sahu <shubhvs@amazon.com>

Added IT for ResizeIndex During migration

ace7752

Signed-off-by: Shubh Sahu <shubhvs@amazon.com>

astute-decipher requested a review from linuxpi March 28, 2024 04:56

gbbafna reviewed Mar 28, 2024

View reviewed changes

Addressed comments on PR by @gbbafna

f262296

Signed-off-by: Shubh Sahu <shubhvs@amazon.com>

code refactoring and spotless checks

872f742

Signed-off-by: Shubh Sahu <shubhvs@amazon.com>

shourya035 reviewed Apr 2, 2024

View reviewed changes

...rc/internalClusterTest/java/org/opensearch/remotemigration/ResizeIndexMigrationTestCase.java Show resolved Hide resolved

shourya035 reviewed Apr 2, 2024

View reviewed changes

server/src/main/java/org/opensearch/action/admin/indices/shrink/TransportResizeAction.java Outdated Show resolved Hide resolved

shourya035 suggested changes Apr 2, 2024

View reviewed changes

Added javadoc and test descriptions

60c77ea

Signed-off-by: Shubh Sahu <shubhvs@amazon.com>

astute-decipher force-pushed the serengity_personel branch from abf1be0 to 60c77ea Compare April 3, 2024 07:27

Merge branch 'main' of https://github.com/opensearch-project/OpenSearch…

2789f8c

… into serengity_personel

astute-decipher marked this pull request as ready for review April 3, 2024 10:09

astute-decipher requested review from ryanbogan, saratvemulapalli, shwetathareja, sohami, tlfeng and VachaShah as code owners April 3, 2024 10:09

gbbafna reviewed Apr 3, 2024

View reviewed changes

Refactoring the code

da459ae

Signed-off-by: Shubh Sahu <shubhvs@amazon.com>

Shubh Sahu added 2 commits April 4, 2024 11:00

Spotless apply

df2aa33

Signed-off-by: Shubh Sahu <shubhvs@amazon.com>

Merge branch 'main' of https://github.com/opensearch-project/OpenSearch…

c99e50b

… into serengity_personel

astute-decipher force-pushed the serengity_personel branch from c4f2530 to c99e50b Compare April 4, 2024 05:36

Corrected CHANGELOG.md

06ae8c0

Signed-off-by: Shubh Sahu <shubhvs@amazon.com>

gbbafna approved these changes Apr 4, 2024

View reviewed changes

gbbafna merged commit 9ab31f5 into opensearch-project:main Apr 4, 2024
30 of 31 checks passed

gbbafna added the backport 2.x Backport to 2.x branch label Apr 4, 2024

opensearch-trigger-bot bot added the backport-failed label Apr 4, 2024

gbbafna mentioned this pull request Apr 4, 2024

[META] Remote Store Migration Support #12718

Closed

21 tasks

github-actions bot added enhancement Enhancement or improvement to existing feature or request Storage:Remote labels Apr 8, 2024

astute-decipher mentioned this pull request Apr 10, 2024

[Backport 2.x] Reject Resize index requests during Remote Store Migration #13147

Merged

8 tasks

gbbafna pushed a commit that referenced this pull request Apr 15, 2024

Reject Resize index requests during remote store migration (#12686) (#…

c24a208

…13147) Signed-off-by: Shubh Sahu <shubhvs@amazon.com> (cherry picked from commit 9ab31f5)

astute-decipher self-assigned this Apr 21, 2024

shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024

Reject Resize index requests during remote store migration (opensearc…

5f0c155

…h-project#12686) Signed-off-by: Shubh Sahu <shubhvs@amazon.com> Signed-off-by: Shivansh Arora <hishiv@amazon.com>

BrewTestBot mentioned this pull request May 14, 2024

opensearch 2.14.0 Homebrew/homebrew-core#171722

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reject Resize index requests. (i.e, split, shrink and… #12686

Reject Resize index requests. (i.e, split, shrink and… #12686

astute-decipher commented Mar 15, 2024 •

edited

Loading

github-actions bot commented Mar 15, 2024

github-actions bot commented Mar 15, 2024 •

edited

Loading

github-actions bot commented Mar 15, 2024

github-actions bot commented Mar 18, 2024

github-actions bot commented Mar 19, 2024

codecov bot commented Mar 19, 2024 •

edited

Loading

github-actions bot commented Apr 1, 2024

github-actions bot commented Apr 1, 2024

shourya035 left a comment

github-actions bot commented Apr 2, 2024

github-actions bot commented Apr 3, 2024

github-actions bot commented Apr 3, 2024

github-actions bot commented Apr 4, 2024

gbbafna commented Apr 4, 2024

github-actions bot commented Apr 4, 2024

github-actions bot commented Apr 4, 2024

github-actions bot commented Apr 4, 2024

opensearch-trigger-bot bot commented Apr 4, 2024

Reject Resize index requests. (i.e, split, shrink and… #12686

Reject Resize index requests. (i.e, split, shrink and… #12686

Conversation

astute-decipher commented Mar 15, 2024 • edited Loading

Description

Related Issues

Check List

github-actions bot commented Mar 15, 2024

github-actions bot commented Mar 15, 2024 • edited Loading

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Mar 15, 2024

github-actions bot commented Mar 18, 2024

github-actions bot commented Mar 19, 2024

codecov bot commented Mar 19, 2024 • edited Loading

Codecov Report

github-actions bot commented Apr 1, 2024

github-actions bot commented Apr 1, 2024

shourya035 left a comment

Choose a reason for hiding this comment

github-actions bot commented Apr 2, 2024

github-actions bot commented Apr 3, 2024

github-actions bot commented Apr 3, 2024

github-actions bot commented Apr 4, 2024

gbbafna commented Apr 4, 2024

github-actions bot commented Apr 4, 2024

github-actions bot commented Apr 4, 2024

github-actions bot commented Apr 4, 2024

opensearch-trigger-bot bot commented Apr 4, 2024

astute-decipher commented Mar 15, 2024 •

edited

Loading

github-actions bot commented Mar 15, 2024 •

edited

Loading

codecov bot commented Mar 19, 2024 •

edited

Loading