Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache operation support for Prestissimo #10401

Closed
wants to merge 1 commit into from

Conversation

xiaoxmeng
Copy link
Contributor

@xiaoxmeng xiaoxmeng commented Jul 4, 2024

Add checkpoint and ssd write functions support for Prestissimo worker to operate the
cache for performance tuning and online debugging in a test environment
Add metrics about how many ssd cache entries are recovered and how many in-memory
cache entries are evicted without saving to SSD.
Other logs improvements.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 4, 2024
Copy link

netlify bot commented Jul 4, 2024

Deploy Preview for meta-velox canceled.

Name Link
🔨 Latest commit 9a5dce2
🔍 Latest deploy log https://app.netlify.com/sites/meta-velox/deploys/668896ec56030e00081bff4b

@xiaoxmeng xiaoxmeng force-pushed the cache branch 7 times, most recently from a07b603 to 7406f86 Compare July 5, 2024 17:13
@xiaoxmeng xiaoxmeng marked this pull request as ready for review July 5, 2024 17:19
@facebook-github-bot
Copy link
Contributor

@xiaoxmeng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

Copy link
Contributor

@zacw7 zacw7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making the change!

velox/common/caching/tests/AsyncDataCacheTest.cpp Outdated Show resolved Hide resolved
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D59401173

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D59401173

xiaoxmeng added a commit to xiaoxmeng/velox that referenced this pull request Jul 5, 2024
Summary:
Add checkpoint and ssd write functions support for Prestissimo worker to operate the
cache for performance tuning and online debugging in a test environment
Add metrics about how many ssd cache entries are recovered and how many in-memory
cache entries are evicted without saving to SSD.
Other logs improvements.

Pull Request resolved: facebookincubator#10401

Reviewed By: Yuhta, zacw7

Differential Revision: D59401173

Pulled By: xiaoxmeng
xiaoxmeng added a commit to xiaoxmeng/velox that referenced this pull request Jul 5, 2024
Summary:
Add checkpoint and ssd write functions support for Prestissimo worker to operate the
cache for performance tuning and online debugging in a test environment
Add metrics about how many ssd cache entries are recovered and how many in-memory
cache entries are evicted without saving to SSD.
Other logs improvements.

Pull Request resolved: facebookincubator#10401

Reviewed By: Yuhta, zacw7

Differential Revision: D59401173

Pulled By: xiaoxmeng
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D59401173

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D59401173

xiaoxmeng added a commit to xiaoxmeng/velox that referenced this pull request Jul 5, 2024
Summary:
Add checkpoint and ssd write functions support for Prestissimo worker to operate the
cache for performance tuning and online debugging in a test environment
Add metrics about how many ssd cache entries are recovered and how many in-memory
cache entries are evicted without saving to SSD.
Other logs improvements.

Pull Request resolved: facebookincubator#10401

Reviewed By: Yuhta, zacw7, oerling

Differential Revision: D59401173

Pulled By: xiaoxmeng
xiaoxmeng added a commit to xiaoxmeng/velox that referenced this pull request Jul 5, 2024
Summary:
Add checkpoint and ssd write functions support for Prestissimo worker to operate the
cache for performance tuning and online debugging in a test environment
Add metrics about how many ssd cache entries are recovered and how many in-memory
cache entries are evicted without saving to SSD.
Other logs improvements.

Pull Request resolved: facebookincubator#10401

Reviewed By: Yuhta, zacw7, oerling

Differential Revision: D59401173

Pulled By: xiaoxmeng
xiaoxmeng added a commit to xiaoxmeng/velox that referenced this pull request Jul 5, 2024
Summary:
Add checkpoint and ssd write functions support for Prestissimo worker to operate the
cache for performance tuning and online debugging in a test environment
Add metrics about how many ssd cache entries are recovered and how many in-memory
cache entries are evicted without saving to SSD.
Other logs improvements.

Pull Request resolved: facebookincubator#10401

Reviewed By: Yuhta, zacw7, oerling

Differential Revision: D59401173

Pulled By: xiaoxmeng
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D59401173

Summary:
Add checkpoint and ssd write functions support for Prestissimo worker to operate the
cache for performance tuning and online debugging in a test environment
Add metrics about how many ssd cache entries are recovered and how many in-memory
cache entries are evicted without saving to SSD.
Other logs improvements.

Pull Request resolved: facebookincubator#10401

Reviewed By: Yuhta, zacw7, oerling

Differential Revision: D59401173

Pulled By: xiaoxmeng
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D59401173

@facebook-github-bot
Copy link
Contributor

@xiaoxmeng merged this pull request in 07684ea.

Copy link

Conbench analyzed the 3 benchmark runs on commit 07684ea9.

There was 1 benchmark result indicating a performance regression:

The full Conbench report has more details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. fb-exported Merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants