Problem: memiavl background snapshot rewriting panic when shutdown #1292
Conversation
Solution: - gracefully cancel the task when shutdown
yihuang
requested review from
mmsqe and
devashishdxt
and removed request for
a team
|
Warning Rate Limit Exceeded@yihuang has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 5 minutes and 36 seconds before requesting another review. How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. WalkthroughThe codebase has been updated to integrate context-based cancellation into various operations, primarily focusing on snapshot rewriting and import processes. The Changes
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
Signed-off-by: yihuang <huang@crypto.com>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Review Status
Actionable comments generated: 3
Configuration used: CodeRabbit UI
Files selected for processing (5)
- CHANGELOG.md (1 hunks)
- memiavl/db.go (8 hunks)
- memiavl/import.go (2 hunks)
- memiavl/multitree.go (2 hunks)
- memiavl/snapshot.go (8 hunks)
Additional comments: 10
memiavl/multitree.go (1)
- 360-360: The
WriteSnapshotmethod now correctly takes acontext.Contextparameter, which is used to control the flow of the snapshot writing process. This is a good practice for managing long-running operations and allows for cancellation and timeout control.memiavl/snapshot.go (4)
355-356: The
WriteSnapshotfunction has been correctly updated to accept acontext.Contextparameter, which is used to manage the cancellation and timeouts of the snapshot writing process.369-369: The
writeSnapshotfunction has been updated to take acontext.Contextparameter, which is a good practice for managing cancellation and timeouts during the snapshot writing process.468-470: The
snapshotWriterstruct has been correctly augmented with actxfield to store the context object, which is used for cancellation checks in thewriteLeafmethod.514-520: The
writeLeafmethod includes a cancellation check based on the context everyCancelCheckIntervalleaves written, which is a good practice for long-running operations to respond to cancellation requests.memiavl/db.go (5)
54-56: The addition of the
snapshotRewriteCancelfield to theDBstruct is a good implementation for handling graceful shutdowns. This field will be used to cancel the snapshot rewrite operation when the database is closed.421-421: The
snapshotRewriteCancelis set tonilafter the snapshot rewrite goroutine completes. This is a good practice as it helps avoid potential memory leaks by allowing the garbage collector to reclaim the context resources.633-639: > Note: This review was outside the patches, so it was mapped to the patch with the greatest overlap. Original lines [636-647]
The
RewriteSnapshotmethod now accepts acontext.Contextparameter, which is used to support cancellation of the snapshot rewrite operation. This change is crucial for the feature implemented in this PR and aligns with the objective of preventing panics during shutdown.
715-719: The creation of a new context with a cancellation function in the
rewriteSnapshotBackgroundmethod is a key part of the solution to the panic issue. This context is used to control the lifecycle of the snapshot rewrite goroutine.765-777: > Note: This review was outside the patches, so it was mapped to the patch with the greatest overlap. Original lines [757-773]
The
Closemethod has been updated to cancel the snapshot rewrite goroutine if it's running. This is done by calling thesnapshotRewriteCancelfunction and waiting for the goroutine to finish. This change is essential for ensuring a graceful shutdown and preventing the panic issue that this PR aims to solve.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| @@ -133,7 +134,7 @@ | |||
| return errors.New("version overflows uint32") | |||
| } | |||
|
|
|||
| return writeSnapshot(dir, uint32(version), func(w *snapshotWriter) (uint32, error) { | |||
| return writeSnapshot(context.Background(), dir, uint32(version), func(w *snapshotWriter) (uint32, error) { | |||
Check failure
Code scanning / gosec
Potential integer overflow by integer type conversion Error
Signed-off-by: yihuang <huang@crypto.com>
maybe |
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #1292 +/- ##
===========================================
+ Coverage 15.98% 35.88% +19.90%
===========================================
Files 80 116 +36
Lines 6201 10698 +4497
===========================================
+ Hits 991 3839 +2848
- Misses 5130 6482 +1352
- Partials 80 377 +297
|
|
Thanks for fixing this! |
Yes, can't reproduce with smaller IdleTimeout now |
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
|
Confirmed this fixed the panic during shutdown |
c2d8b40
Solution:
👮🏻👮🏻👮🏻 !!!! REFERENCE THE PROBLEM YOUR ARE SOLVING IN THE PR TITLE AND DESCRIBE YOUR SOLUTION HERE !!!! DO NOT FORGET !!!! 👮🏻👮🏻👮🏻
PR Checklist:
make)make test)go fmt)golangci-lint run)go list -json -m all | nancy sleuth)Thank you for your code, it's appreciated! :)
Summary by CodeRabbit