-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Atomically update cluster state with decommission status and corresponding action #5093
Merged
Bukhtawar
merged 29 commits into
opensearch-project:main
from
imRishN:decommission/init-merge
Dec 13, 2022
Merged
Atomically update cluster state with decommission status and corresponding action #5093
Bukhtawar
merged 29 commits into
opensearch-project:main
from
imRishN:decommission/init-merge
Dec 13, 2022
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
Gradle Check (Jenkins) Run Completed with:
|
Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
Gradle Check (Jenkins) Run Completed with:
|
Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
Gradle Check (Jenkins) Run Completed with:
|
imRishN
changed the title
Atomically update cluster state with INIT and voting config exclusion during decommission
Atomically update cluster state with decommission status and corresponding action
Nov 8, 2022
Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
Gradle Check (Jenkins) Run Completed with:
|
imRishN
requested review from
CEHENKLE,
dblock,
setiah,
kartg,
kotwanikunal,
mch2,
nknize,
owaiskazi19,
adnapibar,
Rishikesh1159,
ryanbogan,
saratvemulapalli,
dreamer-89,
tlfeng,
VachaShah and
xuezhou25
as code owners
December 12, 2022 07:43
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
shwetathareja
approved these changes
Dec 13, 2022
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM thanks @imRishN for the changes.
Gradle Check (Jenkins) Run Completed with:
|
imRishN
added a commit
to imRishN/OpenSearch
that referenced
this pull request
Dec 13, 2022
…nding action (opensearch-project#5093) * Atomically update the cluster state with decommission status and its corresponding action in the same execute call Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
6 tasks
Bukhtawar
pushed a commit
that referenced
this pull request
Dec 13, 2022
…s and corresponding action (#5541) * [Backport-2.x] Atomically update cluster state with decommission status and corresponding action (#5093) * Atomically update the cluster state with decommission status and its corresponding action in the same execute call Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
ryanbogan
pushed a commit
that referenced
this pull request
Dec 14, 2022
…nding action (#5093) * Atomically update the cluster state with decommission status and its corresponding action in the same execute call Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
saratvemulapalli
pushed a commit
that referenced
this pull request
Dec 15, 2022
…ature/identity (#5581) * Fix flaky ShardIndexingPressureConcurrentExecutionTests (#5439) Add conditional check on assertNull to fix flaky tests. Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> * Fix bwc for cluster manager throttling settings (#5305) Signed-off-by: Dhwanil Patel <dhwanip@amazon.com> * Update ingest-attachment plugin dependencies: Apache Tika 3.6.0, Apache Mime4j 0.8.8, Apache Poi 5.2.3, Apache PdfBox 2.0.27 (#5448) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Enhance CheckpointState to support no-op replication (#5282) * CheckpointState enhanced to support no-op replication Signed-off-by: Ashish Singh <ssashish@amazon.com> Co-authored-by: Bukhtawar Khan<bukhtawa@amazon.com> * [BUG] org.opensearch.repositories.s3.RepositoryS3ClientYamlTestSuiteIT/test {yaml=repository_s3/20_repository_permanent_credentials/Snapshot and Restore with repository-s3 using permanent credentials} flaky: randomizing basePath (#5482) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * [Bug] fix case sensitivity for wildcard queries (#5462) Fixes the wildcard query to not normalize the pattern when case_insensitive is set by the user. This is achieved by creating a new normalizedWildcardQuery method so that query_string queries (which do not support case sensitivity) can still normalize the pattern when the default analyzer is used; maintaining existing behavior. Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * Support OpenSSL Provider with default Netty allocator (#5460) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Revert "build no-jdk distributions as part of release build (#4902)" (#5465) This reverts commit 8c9ca4e. It seems that this wasn't entirely the correct way and is currently blocking us from removing the `build.sh` from the `opensearch-build` repository (i.e. this `build.sh` here is not yet being used). See the discussion in opensearch-project/opensearch-build#2835 for further details. Signed-off-by: Ralph Ursprung <Ralph.Ursprung@avaloq.com> Signed-off-by: Ralph Ursprung <Ralph.Ursprung@avaloq.com> * Add max_shard_size parameter for Shrink API (fix supported version after backport) (#5503) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Sync CODEOWNERS with MAINTAINERS. (#5501) Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> * Added jackson dependency to server (#5366) * Added jackson dependency to server Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Updated CHANGELOG Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Update build.gradle files Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Add RuntimePermission to fix errors Signed-off-by: Ryan Bogan <rbogan@amazon.com> Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Fix flaky test BulkIntegrationIT.testDeleteIndexWhileIndexing (#5491) Signed-off-by: Poojita Raj <poojiraj@amazon.com> Signed-off-by: Poojita Raj <poojiraj@amazon.com> * Add release notes for 2.4.1 (#5488) Signed-off-by: Xue Zhou <xuezhou@amazon.com> Signed-off-by: Xue Zhou <xuezhou@amazon.com> * Properly skip OnDemandBlockSnapshotIndexInputTests.testVariousBlockSize on Windows. (#5511) PR #5397 skipped this test in @before block but still frequently throws a TestCouldNotBeSkippedException. This is caused by the after block still executing and throwing an exception while cleaning the directory created at the path in @before. Moving the assumption to the individual test prevents this exception by ensuring the path exists. Signed-off-by: Marc Handalian <handalm@amazon.com> Signed-off-by: Marc Handalian <handalm@amazon.com> * Merge first batch of feature/extensions into main (#5347) * Merge first batch of feature/extensions into main Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Fixed CHANGELOG Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Fixed newline errors Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Renaming and CHANGELOG fixes Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Refactor extension loading into private method Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Removed skipValidation and added connectToExtensionNode method Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Remove unnecessary feature flag calls Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Renaming and exception handling Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Change latches to CompletableFuture Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Removed unnecessary validateSettingKey call Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Fix azure-core dependency Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Update SHAs Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Remove unintended dependency changes Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Removed dynamic settings regitration, removed info() method, and added NoopExtensionsManager Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Add javadoc Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Fixed spotless failure Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Removed NoopExtensionsManager Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Added functioning NoopExtensionsManager Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Added missing javadoc Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Remove forbiddenAPI Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Fix spotless Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Change logger.info to logger.error in handleException Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Fix ExtensionsManagerTests Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Removing unrelated change Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Update SHAs Signed-off-by: Ryan Bogan <rbogan@amazon.com> Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Bump commons-compress from 1.21 to 1.22 (#5520) Bumps commons-compress from 1.21 to 1.22. --- updated-dependencies: - dependency-name: org.apache.commons:commons-compress dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [Segment Replication] Trigger a round of replication for replica shards during peer recovery when segment replication is enabled (#5332) * Fix new added replica shards falling behind primary. Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> * Trigger a round of replication during peer recovery when segment replication is enabled. Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> * Remove unnecessary start replication overloaded method. Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> * Add test for failure case and refactor some code. Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> * Apply spotless check. Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> * Addressing comments on the PR. Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> * Remove unnecessary condition check. Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> * Apply spotless check. Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> * Add step listeners to resolve forcing round of segment replication. Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> * Adding support to register settings dynamically (#5495) * Adding support to register settings dynamically Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Update CHANGELOG Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Removed unnecessary registerSetting methods Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Change setting registration order Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Add unregisterSettings method Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Remove unnecessary feature flag Signed-off-by: Ryan Bogan <rbogan@amazon.com> Signed-off-by: Ryan Bogan <rbogan@amazon.com> * Updated 1.3.7 release notes date (#5536) Signed-off-by: owaiskazi19 <owaiskazi19@gmail.com> Signed-off-by: owaiskazi19 <owaiskazi19@gmail.com> * Pre conditions check before updating weighted routing metadata (#4955) * Pre conditions check to allow weight updates for non decommissioned attribute Signed-off-by: Rishab Nahata <rnnahata@amazon.com> * Atomically update cluster state with decommission status and corresponding action (#5093) * Atomically update the cluster state with decommission status and its corresponding action in the same execute call Signed-off-by: Rishab Nahata <rnnahata@amazon.com> * Update Netty to 4.1.86.Final (#5529) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Update release date in 2.4.1 release notes (#5549) Signed-off-by: Suraj Singh <surajrider@gmail.com> Signed-off-by: Suraj Singh <surajrider@gmail.com> * Update 2.4.1 release notes (#5552) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Refactor fuzziness interface on query builders (#5433) * Refactor Object to Fuzziness type for all query builders Signed-off-by: noCharger <lingzhichu.clz@gmail.com> * Revise on bwc Signed-off-by: noCharger <lingzhichu.clz@gmail.com> * Update change log Signed-off-by: noCharger <lingzhichu.clz@gmail.com> Signed-off-by: noCharger <lingzhichu.clz@gmail.com> Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> * Upgrade lucene version (#5570) * Added bwc version 2.4.2 Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> * Added 2.4.2. Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> * Update Lucene snapshot to 9.5.0-snapshot-d5cef1c Signed-off-by: Suraj Singh <surajrider@gmail.com> * Update changelog entry Signed-off-by: Suraj Singh <surajrider@gmail.com> * Add 2.4.2 bwc version Signed-off-by: Suraj Singh <surajrider@gmail.com> * Internal changes post lucene upgrade Signed-off-by: Suraj Singh <surajrider@gmail.com> Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Signed-off-by: Suraj Singh <surajrider@gmail.com> Co-authored-by: opensearch-ci-bot <opensearch-ci-bot@users.noreply.github.com> Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> * Add CI bundle pattern to distribution download (#5348) * Add CI bundle pattern for ivy repo Signed-off-by: Zelin Hao <zelinhao@amazon.com> * Gradle update Signed-off-by: Zelin Hao <zelinhao@amazon.com> * Extract path Signed-off-by: Zelin Hao <zelinhao@amazon.com> * Change with customDistributionDownloadType Signed-off-by: Zelin Hao <zelinhao@amazon.com> * Add default for exception handle Signed-off-by: Zelin Hao <zelinhao@amazon.com> * Add documentations Signed-off-by: Zelin Hao <zelinhao@amazon.com> Signed-off-by: Zelin Hao <zelinhao@amazon.com> * Bump protobuf-java from 3.21.9 to 3.21.11 in /plugins/repository-hdfs (#5519) * Bump protobuf-java from 3.21.9 to 3.21.11 in /plugins/repository-hdfs Bumps [protobuf-java](https://github.com/protocolbuffers/protobuf) from 3.21.9 to 3.21.11. - [Release notes](https://github.com/protocolbuffers/protobuf/releases) - [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/generate_changelog.py) - [Commits](protocolbuffers/protobuf@v3.21.9...v3.21.11) --- updated-dependencies: - dependency-name: com.google.protobuf:protobuf-java dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Updated changelog Signed-off-by: Owais Kazi <owaiskazi19@gmail.com> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Owais Kazi <owaiskazi19@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: Owais Kazi <owaiskazi19@gmail.com> Co-authored-by: Suraj Singh <surajrider@gmail.com> Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> Signed-off-by: Dhwanil Patel <dhwanip@amazon.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: Nicholas Walter Knize <nknize@apache.org> Signed-off-by: Ralph Ursprung <Ralph.Ursprung@avaloq.com> Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Signed-off-by: Ryan Bogan <rbogan@amazon.com> Signed-off-by: Poojita Raj <poojiraj@amazon.com> Signed-off-by: Xue Zhou <xuezhou@amazon.com> Signed-off-by: Marc Handalian <handalm@amazon.com> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: owaiskazi19 <owaiskazi19@gmail.com> Signed-off-by: Rishab Nahata <rnnahata@amazon.com> Signed-off-by: Suraj Singh <surajrider@gmail.com> Signed-off-by: noCharger <lingzhichu.clz@gmail.com> Signed-off-by: Zelin Hao <zelinhao@amazon.com> Signed-off-by: Owais Kazi <owaiskazi19@gmail.com> Co-authored-by: Rishikesh Pasham <62345295+Rishikesh1159@users.noreply.github.com> Co-authored-by: Dhwanil Patel <dhwanip@amazon.com> Co-authored-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: Ashish <ssashish@amazon.com> Co-authored-by: Nick Knize <nknize@apache.org> Co-authored-by: Ralph Ursprung <39383228+rursprung@users.noreply.github.com> Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com> Co-authored-by: Ryan Bogan <10944539+ryanbogan@users.noreply.github.com> Co-authored-by: Poojita Raj <poojiraj@amazon.com> Co-authored-by: Xue Zhou <85715413+xuezhou25@users.noreply.github.com> Co-authored-by: Marc Handalian <handalm@amazon.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Owais Kazi <owaiskazi19@gmail.com> Co-authored-by: Rishab Nahata <rnnahata@amazon.com> Co-authored-by: Suraj Singh <surajrider@gmail.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: opensearch-ci-bot <opensearch-ci-bot@users.noreply.github.com> Co-authored-by: Zelin Hao <87548827+zelinh@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR attempts to atomically update the cluster state with decommission status and its corresponding action in the same execute call. However, we still need to control the concurrency in the decommission action which will be resolved in the subsequent PRs.
Issues Resolved
#4571
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.