Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cache of recently seen blobs in CloudBlobStore #1225

Merged
merged 4 commits into from
Aug 6, 2019

Conversation

lightningrob
Copy link
Contributor

To reduce unnecessary Azure round trips when multiple replicas send same data.

@lightningrob lightningrob requested review from cgtz and zzmao July 25, 2019 22:12
@codecov-io
Copy link

codecov-io commented Jul 25, 2019

Codecov Report

Merging #1225 into master will increase coverage by 0.02%.
The diff coverage is 85.71%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #1225      +/-   ##
============================================
+ Coverage     69.51%   69.53%   +0.02%     
- Complexity     5552     5560       +8     
============================================
  Files           432      432              
  Lines         33943    33978      +35     
  Branches       4342     4349       +7     
============================================
+ Hits          23596    23628      +32     
- Misses         9146     9149       +3     
  Partials       1201     1201
Impacted Files Coverage Δ Complexity Δ
...main/java/com.github.ambry/config/CloudConfig.java 100% <100%> (ø) 1 <0> (ø) ⬇️
...c/main/java/com.github.ambry.cloud/VcrMetrics.java 100% <100%> (ø) 2 <0> (ø) ⬇️
...ava/com.github.ambry.cloud/CloudBackupManager.java 46.29% <33.33%> (-0.77%) 4 <1> (ø)
...in/java/com.github.ambry.cloud/CloudBlobStore.java 80% <94.11%> (+2.29%) 35 <1> (+6) ⬆️
...in/java/com.github.ambry.store/BlobStoreStats.java 70.92% <0%> (-0.21%) 103% <0%> (ø)
...src/main/java/com.github.ambry.commons/BlobId.java 93.54% <0%> (+0.35%) 72% <0%> (+1%) ⬆️
...ain/java/com.github.ambry.router/PutOperation.java 91.15% <0%> (+0.53%) 112% <0%> (+1%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2f386ac...76d9eb4. Read the comment docs.

missingKeys = store.findMissingKeys(keys);
assertTrue("Expected no missing keys", missingKeys.isEmpty());
// getBlobMetadata should not have been called a second time.
verify(dest).getBlobMetadata(anyList());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps a unit test to confirm that the LRU nature of the cache works as expected (set the cache limit artificially low, add PUTs over that number, confirm that redoing the PUTs yields (number of PUTs - limit) unskipped cloud PUTs)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done (method a bit different).

Copy link
Contributor

@zzmao zzmao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@zzmao zzmao merged commit 73468e0 into linkedin:master Aug 6, 2019
@lightningrob lightningrob deleted the vcr-blobstore-cache branch January 15, 2020 18:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants