Allow to get the latest sequence number when creating the backup #1987
Conversation
| @@ -363,7 +363,7 @@ Status Storage::CreateBackup() { | |||
| } | |||
|
|
|||
| std::unique_ptr<rocksdb::Checkpoint> checkpoint_guard(checkpoint); | |||
| s = checkpoint->CreateCheckpoint(tmpdir, config_->rocks_db.write_buffer_size * MiB); | |||
| s = checkpoint->CreateCheckpoint(tmpdir, config_->rocks_db.write_buffer_size * MiB, sequence_number); | |||
There was a problem hiding this comment.
Just a question:
// sequence_number_ptr: if it is not nullptr, the value it points to will be
// set to a sequence number guaranteed to be part of the DB, not necessarily
// the latest. The default value of this parameter is nullptr.
From rocksdb, seems this might a bit less than backup? Or I misunderstanding this?
There was a problem hiding this comment.
Maybe, I didn't investigate when it would be less than the DB.
There was a problem hiding this comment.
It was introduced here: facebook/rocksdb@3ffb3ba
Maybe we should notice user to pay-attention to that, replaying kv might not harm
There was a problem hiding this comment.
@mapleFU I have updated the comment to address this, please take a look again.
There was a problem hiding this comment.
I have confirmed the implementation, it is the latest sequence number while starting to create the checkpoint, so it should be less the current backup DB if has any writes during this process.
There was a problem hiding this comment.
Yeah I guess maybe it's not atomic between:
*sequence_number = db_->GetLatestSequenceNumber();
LiveFilesStorageInfoOptions opts;
opts.include_checksum_info = get_live_table_checksum;
opts.wal_size_for_flush = log_size_for_flush;
std::vector<LiveFileStorageInfo> infos;
{
Status s = db_->GetLiveFilesStorageInfo(opts, &infos);
|
git-hulk
changed the title
As mentioned in #1986, some users want to know the sequence number of the backup
for the increment backup scenario. Fortunately, rocksdb allows us to get the sequence
number when creating the checkpoint. So what we need to do is to pass the receiver.