Skip to content
This repository has been archived by the owner on Jun 23, 2022. It is now read-only.

fix(bulk_load): redownload file if local file damaged #826

Merged
merged 3 commits into from
Apr 27, 2021

Conversation

hycdong
Copy link
Contributor

@hycdong hycdong commented Apr 26, 2021

In bulk load download stage, replica server will download file from remote file provider, if a same-name file exists, block service interface download_file will return ERR_PATH_ALREADY_EXIST. In previous implmentation, if local file doesn't pass verification, it will lead to bulk load failed. This pull request fixs it, if local file doesn't pass verifition, it will remove old file and redownload it.

I test this case in onebox manually in following step:

  1. start bulk load
  2. pause bulk load
  3. modify one file content to simulate downloaded local file is different from remoted file
  4. restart bulk load
  5. I can find redownload log below
W2021-04-26 15:38:04.26 (1619422684026147118 7747) replica.rep_long0.0304000000000420: replica_bulk_loader.cpp:441:operator()(): [2.0@<ip>:<port>] file(<path>/file0.sst) exists, but not verified, try to remove and redownload it
  1. bulk load succeed

@levy5307 levy5307 merged commit 510d7c9 into XiaoMi:master Apr 27, 2021
@hycdong hycdong deleted the bulk_load_redownload branch April 29, 2021 01:42
@hycdong hycdong added the 2.2.0 label Aug 30, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants