[Writable Warm] Composite Directory implementation and integrating it with FileCache #12782
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rayshrey
requested review from
abbashus,
adnapibar,
anasalkouz,
andrross,
Bukhtawar,
CEHENKLE,
dblock,
dbwiddis,
dreamer-89,
gbbafna,
kotwanikunal,
mch2,
msfroh,
nknize,
owaiskazi19,
reta,
Rishikesh1159,
ryanbogan,
sachinpkale,
saratvemulapalli,
shwetathareja,
sohami,
tlfeng and
VachaShah
as code owners
Compatibility status:Checks if related components are compatible with change f442533 Incompatible componentsSkipped componentsCompatible componentsCompatible components: [https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/flow-framework.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/sql.git] |
|
❌ Gradle check result for d507327: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
ankitkala
reviewed
Mar 21, 2024
server/src/main/java/org/opensearch/index/store/CompositeDirectory.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/index/store/CompositeDirectoryFactory.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/index/store/remote/utils/filetracker/FileState.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/index/shard/RemoteStoreRefreshListener.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/index/store/CompositeDirectory.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Shreyansh Ray <rayshrey@amazon.com>
Signed-off-by: Shreyansh Ray <rayshrey@amazon.com>
…or ref count via cloneMap in FullFileCachedIndexInput and other review fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com>
Signed-off-by: Shreyansh Ray <rayshrey@amazon.com>
Signed-off-by: Shreyansh Ray <rayshrey@amazon.com>
Signed-off-by: Shreyansh Ray <rayshrey@amazon.com>
rayshrey
force-pushed
the
composite-directory-poc
branch
from
57eca38
to
901849b
Compare
|
❕ Gradle check result for 901849b: UNSTABLE Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure. |
|
sohami
approved these changes
Jun 20, 2024
|
The backport to To backport manually, run these commands in your terminal: # Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-12782-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 b8c78196438897132f6819460ebb7d4222b39297
# Push it to GitHub
git push --set-upstream origin backport/backport-12782-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.xThen, create a pull request where the |
|
@rayshrey Can you please create a manual backport PR for 2.x as above failed |
rayshrey
added a commit
to rayshrey/OpenSearch
that referenced
this pull request
Jun 20, 2024
… with FileCache (opensearch-project#12782) * Composite Directory POC Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Refactor TransferManager interface to RemoteStoreFileTrackerAdapter Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Implement block level fetch for Composite Directory Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Removed CACHE state from FileTracker Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Fixes after latest pull Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add new setting for warm, remove store type setting, FileTracker and RemoteStoreFileTrackerAdapter, CompositeDirectoryFactory and update Composite Directory implementation Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Modify TransferManager - replace BlobContainer with Functional Interface to fetch an InputStream instead Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Reuse OnDemandBlockSnapshotIndexInput instead of OnDemandBlockCompositeIndexInput Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Modify constructors to avoid breaking public api contract and code review fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add experimental annotations for newly created classes and review comment fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Use ref count as a temporary measure to prevent file from eviction until uploaded to Remote Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Remove method level locks Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Handle tmp file deletion Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Nit fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Handle delete and close in Composite Directory, log current state of FileCache and correct it's clear method and modify unit and integration tests as per review comments Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Refactor usages of WRITEABLE_REMOTE_INDEX_SETTING to TIERED_REMOTE_INDEX_SETTING Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add tests for FileCachedIndexInput and review comment fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add additional IT for feature flag disabled Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Move setting for Partial Locality type behind Feature Flag, fix bug for ref count via cloneMap in FullFileCachedIndexInput and other review fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Minor test and nit fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add javadocs for FullFileCachedIndexInput Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Minor precommit fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> --------- Signed-off-by: Shreyansh Ray <rayshrey@amazon.com>
rayshrey
added a commit
to rayshrey/OpenSearch
that referenced
this pull request
Jun 21, 2024
… with FileCache (opensearch-project#12782) * Composite Directory POC Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Refactor TransferManager interface to RemoteStoreFileTrackerAdapter Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Implement block level fetch for Composite Directory Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Removed CACHE state from FileTracker Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Fixes after latest pull Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add new setting for warm, remove store type setting, FileTracker and RemoteStoreFileTrackerAdapter, CompositeDirectoryFactory and update Composite Directory implementation Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Modify TransferManager - replace BlobContainer with Functional Interface to fetch an InputStream instead Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Reuse OnDemandBlockSnapshotIndexInput instead of OnDemandBlockCompositeIndexInput Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Modify constructors to avoid breaking public api contract and code review fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add experimental annotations for newly created classes and review comment fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Use ref count as a temporary measure to prevent file from eviction until uploaded to Remote Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Remove method level locks Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Handle tmp file deletion Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Nit fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Handle delete and close in Composite Directory, log current state of FileCache and correct it's clear method and modify unit and integration tests as per review comments Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Refactor usages of WRITEABLE_REMOTE_INDEX_SETTING to TIERED_REMOTE_INDEX_SETTING Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add tests for FileCachedIndexInput and review comment fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add additional IT for feature flag disabled Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Move setting for Partial Locality type behind Feature Flag, fix bug for ref count via cloneMap in FullFileCachedIndexInput and other review fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Minor test and nit fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add javadocs for FullFileCachedIndexInput Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Minor precommit fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> --------- Signed-off-by: Shreyansh Ray <rayshrey@amazon.com>
3 tasks
sohami
pushed a commit
that referenced
this pull request
Jun 24, 2024
… integrating it with FileCache (#14489) * [Writable Warm] Composite Directory implementation and integrating it with FileCache (#12782) * Composite Directory POC Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Refactor TransferManager interface to RemoteStoreFileTrackerAdapter Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Implement block level fetch for Composite Directory Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Removed CACHE state from FileTracker Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Fixes after latest pull Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add new setting for warm, remove store type setting, FileTracker and RemoteStoreFileTrackerAdapter, CompositeDirectoryFactory and update Composite Directory implementation Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Modify TransferManager - replace BlobContainer with Functional Interface to fetch an InputStream instead Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Reuse OnDemandBlockSnapshotIndexInput instead of OnDemandBlockCompositeIndexInput Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Modify constructors to avoid breaking public api contract and code review fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add experimental annotations for newly created classes and review comment fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Use ref count as a temporary measure to prevent file from eviction until uploaded to Remote Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Remove method level locks Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Handle tmp file deletion Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Nit fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Handle delete and close in Composite Directory, log current state of FileCache and correct it's clear method and modify unit and integration tests as per review comments Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Refactor usages of WRITEABLE_REMOTE_INDEX_SETTING to TIERED_REMOTE_INDEX_SETTING Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add tests for FileCachedIndexInput and review comment fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add additional IT for feature flag disabled Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Move setting for Partial Locality type behind Feature Flag, fix bug for ref count via cloneMap in FullFileCachedIndexInput and other review fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Minor test and nit fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add javadocs for FullFileCachedIndexInput Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Minor precommit fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> --------- Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Fix Writable Warm test for feature flag disabled condition by changing exception type caught Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> --------- Signed-off-by: Shreyansh Ray <rayshrey@amazon.com>
harshavamsi
pushed a commit
to harshavamsi/OpenSearch
that referenced
this pull request
Jul 12, 2024
… with FileCache (opensearch-project#12782) * Composite Directory POC Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Refactor TransferManager interface to RemoteStoreFileTrackerAdapter Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Implement block level fetch for Composite Directory Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Removed CACHE state from FileTracker Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Fixes after latest pull Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add new setting for warm, remove store type setting, FileTracker and RemoteStoreFileTrackerAdapter, CompositeDirectoryFactory and update Composite Directory implementation Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Modify TransferManager - replace BlobContainer with Functional Interface to fetch an InputStream instead Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Reuse OnDemandBlockSnapshotIndexInput instead of OnDemandBlockCompositeIndexInput Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Modify constructors to avoid breaking public api contract and code review fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add experimental annotations for newly created classes and review comment fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Use ref count as a temporary measure to prevent file from eviction until uploaded to Remote Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Remove method level locks Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Handle tmp file deletion Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Nit fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Handle delete and close in Composite Directory, log current state of FileCache and correct it's clear method and modify unit and integration tests as per review comments Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Refactor usages of WRITEABLE_REMOTE_INDEX_SETTING to TIERED_REMOTE_INDEX_SETTING Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add tests for FileCachedIndexInput and review comment fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add additional IT for feature flag disabled Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Move setting for Partial Locality type behind Feature Flag, fix bug for ref count via cloneMap in FullFileCachedIndexInput and other review fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Minor test and nit fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add javadocs for FullFileCachedIndexInput Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Minor precommit fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> --------- Signed-off-by: Shreyansh Ray <rayshrey@amazon.com>
kkewwei
pushed a commit
to kkewwei/OpenSearch
that referenced
this pull request
Jul 24, 2024
… integrating it with FileCache (opensearch-project#14489) * [Writable Warm] Composite Directory implementation and integrating it with FileCache (opensearch-project#12782) * Composite Directory POC Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Refactor TransferManager interface to RemoteStoreFileTrackerAdapter Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Implement block level fetch for Composite Directory Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Removed CACHE state from FileTracker Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Fixes after latest pull Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add new setting for warm, remove store type setting, FileTracker and RemoteStoreFileTrackerAdapter, CompositeDirectoryFactory and update Composite Directory implementation Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Modify TransferManager - replace BlobContainer with Functional Interface to fetch an InputStream instead Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Reuse OnDemandBlockSnapshotIndexInput instead of OnDemandBlockCompositeIndexInput Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Modify constructors to avoid breaking public api contract and code review fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add experimental annotations for newly created classes and review comment fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Use ref count as a temporary measure to prevent file from eviction until uploaded to Remote Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Remove method level locks Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Handle tmp file deletion Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Nit fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Handle delete and close in Composite Directory, log current state of FileCache and correct it's clear method and modify unit and integration tests as per review comments Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Refactor usages of WRITEABLE_REMOTE_INDEX_SETTING to TIERED_REMOTE_INDEX_SETTING Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add tests for FileCachedIndexInput and review comment fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add additional IT for feature flag disabled Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Move setting for Partial Locality type behind Feature Flag, fix bug for ref count via cloneMap in FullFileCachedIndexInput and other review fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Minor test and nit fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add javadocs for FullFileCachedIndexInput Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Minor precommit fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> --------- Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Fix Writable Warm test for feature flag disabled condition by changing exception type caught Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> --------- Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> Signed-off-by: kkewwei <kkewwei@163.com>
wdongyu
pushed a commit
to wdongyu/OpenSearch
that referenced
this pull request
Aug 22, 2024
… with FileCache (opensearch-project#12782) * Composite Directory POC Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Refactor TransferManager interface to RemoteStoreFileTrackerAdapter Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Implement block level fetch for Composite Directory Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Removed CACHE state from FileTracker Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Fixes after latest pull Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add new setting for warm, remove store type setting, FileTracker and RemoteStoreFileTrackerAdapter, CompositeDirectoryFactory and update Composite Directory implementation Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Modify TransferManager - replace BlobContainer with Functional Interface to fetch an InputStream instead Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Reuse OnDemandBlockSnapshotIndexInput instead of OnDemandBlockCompositeIndexInput Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Modify constructors to avoid breaking public api contract and code review fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add experimental annotations for newly created classes and review comment fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Use ref count as a temporary measure to prevent file from eviction until uploaded to Remote Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Remove method level locks Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Handle tmp file deletion Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Nit fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Handle delete and close in Composite Directory, log current state of FileCache and correct it's clear method and modify unit and integration tests as per review comments Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Refactor usages of WRITEABLE_REMOTE_INDEX_SETTING to TIERED_REMOTE_INDEX_SETTING Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add tests for FileCachedIndexInput and review comment fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add additional IT for feature flag disabled Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Move setting for Partial Locality type behind Feature Flag, fix bug for ref count via cloneMap in FullFileCachedIndexInput and other review fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Minor test and nit fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Add javadocs for FullFileCachedIndexInput Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> * Minor precommit fixes Signed-off-by: Shreyansh Ray <rayshrey@amazon.com> --------- Signed-off-by: Shreyansh Ray <rayshrey@amazon.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Implements and design Composite Directory (and integrates it with FileCache) which can fall back to the on-demand fetch when requested data is not present locally.
Opening this PR to get initial review comments on the design and implementation of Composite Directory in this issue - #12781, hence haven't added any tests apart from one IT test for testing the basic functionality. Will include tests in the following revisions to this PR.
Related Issues
Resolves #[12781]
Check List
Public documentation issue/PR createdBy submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.