bug(snapshot): Fix unwriten entries in multiple snapshotting Fixes #823 #825
Conversation
…some entries isuee #823 Signed-off-by: adi_holden <adi@dragonflydb.io>
|
Please shorten your commit title and move the details to the body. Also add |
src/server/db_slice.h
Outdated
| @@ -278,6 +278,8 @@ class DbSlice { | |||
| //! at a time of the call. | |||
| uint64_t RegisterOnChange(ChangeCallback cb); | |||
|
|
|||
| void CallChangeOnAllLessThanVersion(DbIndex db_ind, PrimeIterator it, uint64_t version); | |||
There was a problem hiding this comment.
I think the name could be improved :)
void ApplyChangesToPreviousSnapshots(..., uint64_t upper_bound_ver);
| void DbSlice::CallChangeOnAllLessThanVersion(DbIndex db_ind, PrimeIterator it, uint64_t version) { | ||
| uint64_t bucket_version = it.GetVersion(); | ||
| // change_cb_ is ordered by version. | ||
| for (const auto& ccb : change_cb_) { |
There was a problem hiding this comment.
I think there is a bug here if ccb.second preempts and some other snapshot (un)registers to change_cb_.
changing change_cb_ type to map might fix the issue.
There was a problem hiding this comment.
Our callbacks do not preempt. We should not allow for preemptable callbacks as they are getting iterator and it could be invalidated if there is a preempt.
There was a problem hiding this comment.
our callbacks can preempt when writing to the channel, when calling PushBytesToChannel
There was a problem hiding this comment.
@dranikpg you are right. They can preempt when pushing to channel. In this case we should check if we dont have a bug in the flow, as we are using iterator..
Signed-off-by: adi_holden <adi@dragonflydb.io>
adiholden
changed the title
When traversing over buckets for snapshoting we deside to serialize bucket if its version is lower than snapshot version,
after we serialize the bucket we update the snapshot version.
When we have 2 snapshots S1 and S2 running at the same time with versions v1 , v2 where v1 < v2 and
S2 reaches bucket B before S1, then S1 will skip serializing the bucket .
My fix:
In BucketSaveCb called from the snapshot main loop, we first trigger calls of OnDbChange for all snapshots with version less than the current flow version. This way we make sure that the snapshot with higher version will not overide the bucket version before the snapshot with lower version will serialize it
Fixes #823