Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Downgrade write lock to read lock before translog upload to remote store #10135

Merged
merged 2 commits into from
Sep 22, 2023

Conversation

ashking94
Copy link
Member

@ashking94 ashking94 commented Sep 20, 2023

Description

#10117

Related Issues

Resolves #10117 and #10133

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

… remote store (opensearch-project#9636)"

This reverts commit b5cc002.

Signed-off-by: Ashish Singh <ssashish@amazon.com>
@github-actions github-actions bot added enhancement Enhancement or improvement to existing feature or request Storage:Durability Issues and PRs related to the durability framework v2.11.0 Issues and PRs related to version 2.11.0 labels Sep 20, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Sep 20, 2023

Compatibility status:

Checks if related components are compatible with change 1bec919

Incompatible components

Incompatible components: [https://github.com/opensearch-project/k-nn.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git]

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

  • RESULT: UNSTABLE ❕
  • TEST FAILURES:
      1 org.opensearch.search.SearchWeightedRoutingIT.testMultiGetWithNetworkDisruption_FailOpenEnabled
      1 org.opensearch.remotestore.RemoteStoreStatsIT.testDownloadStatsCorrectnessSinglePrimaryMultipleReplicaShards

@ashking94
Copy link
Member Author

ashking94 commented Sep 21, 2023

Gradle Check (Jenkins) Run Completed with:

  • RESULT: UNSTABLE ❕
  • TEST FAILURES:
      1 org.opensearch.search.SearchWeightedRoutingIT.testMultiGetWithNetworkDisruption_FailOpenEnabled
      1 org.opensearch.remotestore.RemoteStoreStatsIT.testDownloadStatsCorrectnessSinglePrimaryMultipleReplicaShards

Flaky tests - #10152, #10153

Signed-off-by: Ashish Singh <ssashish@amazon.com>
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@ashking94
Copy link
Member Author

Gradle Check (Jenkins) Run Completed with:

Other sets of flaky tests - #10154, #10155

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@gbbafna gbbafna added the backport 2.x Backport to 2.x branch label Sep 21, 2023
@gbbafna gbbafna merged commit cbff21d into opensearch-project:main Sep 22, 2023
27 of 28 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Sep 22, 2023
…ore (#10135)

* Revert "Optimize read write lock constructs during translog upload to remote store (#9636)"

This reverts commit b5cc002.

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Downgrade writelock to readlock during translog upload

Signed-off-by: Ashish Singh <ssashish@amazon.com>

---------

Signed-off-by: Ashish Singh <ssashish@amazon.com>
(cherry picked from commit cbff21d)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@ashking94 ashking94 deleted the 10117 branch September 22, 2023 05:05
@ashking94 ashking94 restored the 10117 branch September 22, 2023 05:05
@ashking94 ashking94 added backport 2.x Backport to 2.x branch and removed backport 2.x Backport to 2.x branch labels Sep 22, 2023
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-10135-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 cbff21d7ea58d3185b969df43f48de10e32180e2
# Push it to GitHub
git push --set-upstream origin backport/backport-10135-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-10135-to-2.x.

ashking94 added a commit to ashking94/OpenSearch that referenced this pull request Sep 22, 2023
…ore (opensearch-project#10135)

* Revert "Optimize read write lock constructs during translog upload to remote store (opensearch-project#9636)"

This reverts commit b5cc002.

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Downgrade writelock to readlock during translog upload

Signed-off-by: Ashish Singh <ssashish@amazon.com>

---------

Signed-off-by: Ashish Singh <ssashish@amazon.com>
sarthakaggarwal97 pushed a commit to sarthakaggarwal97/OpenSearch that referenced this pull request Sep 24, 2023
…ore (opensearch-project#10135)

* Revert "Optimize read write lock constructs during translog upload to remote store (opensearch-project#9636)"

This reverts commit b5cc002.

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Downgrade writelock to readlock during translog upload

Signed-off-by: Ashish Singh <ssashish@amazon.com>

---------

Signed-off-by: Ashish Singh <ssashish@amazon.com>
sachinpkale pushed a commit that referenced this pull request Sep 25, 2023
…ore (#10135) (#10187)

* Revert "Optimize read write lock constructs during translog upload to remote store (#9636)"

This reverts commit b5cc002.



* Downgrade writelock to readlock during translog upload



---------

Signed-off-by: Ashish Singh <ssashish@amazon.com>
brusic pushed a commit to brusic/OpenSearch that referenced this pull request Sep 25, 2023
…ore (opensearch-project#10135)

* Revert "Optimize read write lock constructs during translog upload to remote store (opensearch-project#9636)"

This reverts commit b5cc002.

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Downgrade writelock to readlock during translog upload

Signed-off-by: Ashish Singh <ssashish@amazon.com>

---------

Signed-off-by: Ashish Singh <ssashish@amazon.com>
Signed-off-by: Ivan Brusic <ivan.brusic@flocksafety.com>
vikasvb90 pushed a commit to vikasvb90/OpenSearch that referenced this pull request Oct 10, 2023
…ore (opensearch-project#10135)

* Revert "Optimize read write lock constructs during translog upload to remote store (opensearch-project#9636)"

This reverts commit b5cc002.

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Downgrade writelock to readlock during translog upload

Signed-off-by: Ashish Singh <ssashish@amazon.com>

---------

Signed-off-by: Ashish Singh <ssashish@amazon.com>
shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
…ore (opensearch-project#10135)

* Revert "Optimize read write lock constructs during translog upload to remote store (opensearch-project#9636)"

This reverts commit b5cc002.

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Downgrade writelock to readlock during translog upload

Signed-off-by: Ashish Singh <ssashish@amazon.com>

---------

Signed-off-by: Ashish Singh <ssashish@amazon.com>
Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch backport-failed enhancement Enhancement or improvement to existing feature or request skip-changelog Storage:Durability Issues and PRs related to the durability framework v2.11.0 Issues and PRs related to version 2.11.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Remote Translog] Restrict higher translog generation upload and possible race conditions
3 participants