[Remote Cluster State] Remote state interfaces #13785

soosinha · 2024-05-22T18:23:21Z

Description

Creating generic interface to upload or download a generic object to a configurable remote storage.

Related Issues

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
API changes companion pull request created.
Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
Commits are signed per the DCO using --signoff
~~Commit changes are listed out in CHANGELOG.md file (See: Changelog)~~
~~Public documentation issue/PR created~~

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2024-05-22T18:33:20Z

❌ Gradle check result for 3f09094: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2024-05-23T05:04:06Z

❌ Gradle check result for e28a949: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2024-05-23T05:16:05Z

✅ Gradle check result for e64941a: SUCCESS

codecov · 2024-05-23T05:16:57Z

Codecov Report

Attention: Patch coverage is 0% with 69 lines in your changes missing coverage. Please review.

Project coverage is 71.61%. Comparing base (b15cb0c) to head (b43d2fe).
Report is 348 commits behind head on main.

Files	Patch %	Lines
...eway/remote/model/RemoteClusterStateBlobStore.java	0.00%	38 Missing ⚠️
...ommon/remote/AbstractRemoteWritableBlobEntity.java	0.00%	23 Missing ⚠️
...g/opensearch/common/remote/BlobPathParameters.java	0.00%	6 Missing ⚠️
...search/gateway/remote/RemoteClusterStateUtils.java	0.00%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #13785      +/-   ##
============================================
+ Coverage     71.42%   71.61%   +0.19%     
- Complexity    59978    61341    +1363     
============================================
  Files          4985     5066      +81     
  Lines        282275   287976    +5701     
  Branches      40946    41697     +751     
============================================
+ Hits         201603   206236    +4633     
- Misses        63999    64782     +783     
- Partials      16673    16958     +285

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

server/src/main/java/org/opensearch/gateway/remote/AbstractRemoteBlobStoreObject.java

server/src/main/java/org/opensearch/gateway/remote/RemoteObject.java

prudhvigodithi · 2024-05-23T19:06:01Z

Hey, If the Gradle check fails with error No such property: post_merge_action , please merge/rebase the PR from upstream to ensure the post_merge_action field is part of the Gradle check setup. More details in the PR #13786.

Bukhtawar

Thanks for the changes @soosinha, looks much better now. The interface changes looks good to me

* Remote Writable Entity interfaces Signed-off-by: Sooraj Sinha <soosinha@amazon.com> (cherry picked from commit 156eca3) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

) * Remote Writable Entity interfaces Signed-off-by: Sooraj Sinha <soosinha@amazon.com>

Bukhtawar · 2024-06-06T18:28:31Z

server/src/main/java/org/opensearch/common/remote/RemoteWritableEntityStore.java

+ * @param <T> The object type which can be uploaded to or downloaded from remote storage.
+ * @param <U> The wrapper entity which provides methods for serializing/deserializing entity T.
+ */
+public interface RemoteWritableEntityStore<T, U extends RemoteWriteableEntity<T>> {


Lets make this ExperimentalApi for the current release

) * Remote Writable Entity interfaces Signed-off-by: Sooraj Sinha <soosinha@amazon.com> (cherry picked from commit 156eca3) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Remote Writable Entity interfaces (cherry picked from commit 156eca3) Signed-off-by: Sooraj Sinha <soosinha@amazon.com>

) (opensearch-project#14065) * Remote Writable Entity interfaces (cherry picked from commit 156eca3) Signed-off-by: Sooraj Sinha <soosinha@amazon.com>

) * Remote Writable Entity interfaces Signed-off-by: Sooraj Sinha <soosinha@amazon.com>

) (opensearch-project#14065) * Remote Writable Entity interfaces (cherry picked from commit 156eca3) Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: kkewwei <kkewwei@163.com>

) * Remote Writable Entity interfaces Signed-off-by: Sooraj Sinha <soosinha@amazon.com>

soosinha requested review from anasalkouz, andrross, Bukhtawar, CEHENKLE, dblock, dbwiddis, dreamer-89, gbbafna, kotwanikunal, mch2, msfroh, nknize, owaiskazi19, reta, Rishikesh1159, sachinpkale, saratvemulapalli, shwetathareja, sohami, tlfeng and VachaShah as code owners May 22, 2024 18:23

soosinha force-pushed the rs-interface branch 2 times, most recently from e28a949 to e64941a Compare May 23, 2024 04:27

shiv0408 reviewed May 23, 2024

View reviewed changes

server/src/main/java/org/opensearch/gateway/remote/AbstractRemoteBlobStoreObject.java Outdated Show resolved Hide resolved

Bukhtawar reviewed May 23, 2024

View reviewed changes

server/src/main/java/org/opensearch/gateway/remote/RemoteObject.java Outdated Show resolved Hide resolved

Bukhtawar approved these changes Jun 5, 2024

View reviewed changes

shwetathareja approved these changes Jun 6, 2024

View reviewed changes

shwetathareja merged commit 156eca3 into opensearch-project:main Jun 6, 2024
32 of 34 checks passed

shwetathareja added the backport 2.x Backport to 2.x branch label Jun 6, 2024

opensearch-trigger-bot bot mentioned this pull request Jun 6, 2024

[Backport 2.x] [Remote Cluster State] Remote state interfaces #14019

Closed

LantaoJin pushed a commit to LantaoJin/OpenSearch that referenced this pull request Jun 6, 2024

[Remote Cluster State] Remote state interfaces (opensearch-project#13785

87a4cc6

) * Remote Writable Entity interfaces Signed-off-by: Sooraj Sinha <soosinha@amazon.com>

This was referenced Jun 6, 2024

[Remote Routing Table]Read remote index routing #13894

Closed

[Remote Routing Table] Add write flow for remote routing table #13870

Merged

Bukhtawar reviewed Jun 6, 2024

View reviewed changes

soosinha mentioned this pull request Jun 7, 2024

[Backport 2.x] Remote state interfaces (#13785) #14065

Merged

himshikha mentioned this pull request Jun 7, 2024

[Remote Routing Table] Refactor Remote routing table interfaces #14067

Closed

shwetathareja pushed a commit that referenced this pull request Jun 8, 2024

[Remote Cluster State] Remote state interfaces (#13785) (#14065)

d6f0ef9

* Remote Writable Entity interfaces (cherry picked from commit 156eca3) Signed-off-by: Sooraj Sinha <soosinha@amazon.com>

parv0201 pushed a commit to parv0201/OpenSearch that referenced this pull request Jun 10, 2024

[Remote Cluster State] Remote state interfaces (opensearch-project#13785

c3ae7b1

) * Remote Writable Entity interfaces Signed-off-by: Sooraj Sinha <soosinha@amazon.com>

soosinha mentioned this pull request Jun 11, 2024

[Remote Cluster State] Create interfaces for remote cluster state #13791

Closed

This was referenced Jun 13, 2024

[AUTOCUT] Gradle Check Flaky Test Report for IndicesRequestCacheIT #14288

Open

[AUTOCUT] Gradle Check Flaky Test Report for MinimumClusterManagerNodesIT #14289

Open

finnegancarroll mentioned this pull request Jun 14, 2024

Remove composeBuild fixture tasks when docker support not found #14357

Merged

3 tasks

prudhvigodithi mentioned this pull request Jun 14, 2024

[AUTOCUT] Gradle Check Flaky Test Report for IndicesRequestCacheIT prudhvigodithi/OpenSearch#27

Closed

soosinha mentioned this pull request Jun 20, 2024

[Remote State] Timing and tracing for RemoteWritableEntityStore #14470

Open

This was referenced Jun 27, 2024

[AUTOCUT] Gradle Check Flaky Test Report for RemoteReplicaRecoveryIT #14303

Open

[AUTOCUT] Gradle Check Flaky Test Report for SimpleClusterStateIT #14568

Open

wdongyu pushed a commit to wdongyu/OpenSearch that referenced this pull request Aug 22, 2024

[Remote Cluster State] Remote state interfaces (opensearch-project#13785

de3fcac

) * Remote Writable Entity interfaces Signed-off-by: Sooraj Sinha <soosinha@amazon.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Remote Cluster State] Remote state interfaces #13785

[Remote Cluster State] Remote state interfaces #13785

soosinha commented May 22, 2024 •

edited

Loading

github-actions bot commented May 22, 2024

github-actions bot commented May 23, 2024

github-actions bot commented May 23, 2024

codecov bot commented May 23, 2024 •

edited

Loading

prudhvigodithi commented May 23, 2024 •

edited

Loading

Bukhtawar left a comment

Bukhtawar Jun 6, 2024

[Remote Cluster State] Remote state interfaces #13785

[Remote Cluster State] Remote state interfaces #13785

Conversation

soosinha commented May 22, 2024 • edited Loading

Description

Related Issues

Check List

github-actions bot commented May 22, 2024

github-actions bot commented May 23, 2024

github-actions bot commented May 23, 2024

codecov bot commented May 23, 2024 • edited Loading

Codecov Report

prudhvigodithi commented May 23, 2024 • edited Loading

Bukhtawar left a comment

Choose a reason for hiding this comment

Bukhtawar Jun 6, 2024

Choose a reason for hiding this comment

soosinha commented May 22, 2024 •

edited

Loading

codecov bot commented May 23, 2024 •

edited

Loading

prudhvigodithi commented May 23, 2024 •

edited

Loading