Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix logic error in check_integrity #871

Merged
merged 3 commits into from
Apr 25, 2019
Merged

Fix logic error in check_integrity #871

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6c6150e
fixed check integrity
Apr 25, 2019
3b6a8a7
stylistic changes
Apr 25, 2019
fbfa91f
flake8
Apr 25, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 16 additions & 11 deletions torchvision/datasets/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,20 +17,25 @@ def bar_update(count, block_size, total_size):
return bar_update


def calculate_md5(fpath, chunk_size=1024 * 1024):
md5 = hashlib.md5()
with open(fpath, 'rb') as f:
for chunk in iter(lambda: f.read(chunk_size), b''):
md5.update(chunk)
return md5.hexdigest()


def check_md5(fpath, md5, **kwargs):
return md5 == calculate_md5(fpath, **kwargs)


def check_integrity(fpath, md5=None):
if md5 is None:
return True
if not os.path.isfile(fpath):
return False
md5o = hashlib.md5()
with open(fpath, 'rb') as f:
# read in 1MB chunks
for chunk in iter(lambda: f.read(1024 * 1024), b''):
md5o.update(chunk)
md5c = md5o.hexdigest()
if md5c != md5:
return False
return True
if md5 is None:
return True
else:
pmeier marked this conversation as resolved.
Show resolved Hide resolved
return check_md5(fpath, md5)


def makedir_exist_ok(dirpath):
Expand Down