Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better handle corrupted videos #1463

Merged
merged 3 commits into from
Oct 15, 2019

Conversation

fmassa
Copy link
Member

@fmassa fmassa commented Oct 15, 2019

Fixes #1271 and #1327

This PR adds a try: except guards to the locations where we access the file. This should make it more robust to corrupted videos, as before we were only handling errors during seek, and now he handle it both during container creation (i.e., if the header is corrupted, as in #1271) or when parts of the file are corrupted (as in #1327).

cc @bjuncek @stephenyan1231

@fmassa fmassa mentioned this pull request Oct 15, 2019
@fmassa fmassa force-pushed the handle-corrupted-video-header branch from c5acc9d to 1130fd1 Compare October 15, 2019 14:08
@codecov-io
Copy link

Codecov Report

Merging #1463 into master will increase coverage by 0.06%.
The diff coverage is 78.04%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1463      +/-   ##
==========================================
+ Coverage   64.08%   64.14%   +0.06%     
==========================================
  Files          80       80              
  Lines        6328     6342      +14     
  Branches      973      975       +2     
==========================================
+ Hits         4055     4068      +13     
  Misses       1986     1986              
- Partials      287      288       +1
Impacted Files Coverage Δ
torchvision/io/video.py 75.32% <78.04%> (+1.75%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f16b672...1130fd1. Read the comment docs.

@stephenyan1231
Copy link
Contributor

Thanks for adding for exception handling logic. Hopefully the pyav video decoder is most robust.

@fmassa fmassa merged commit da89dad into pytorch:master Oct 15, 2019
@fmassa fmassa deleted the handle-corrupted-video-header branch October 15, 2019 18:50
@fmassa fmassa mentioned this pull request Oct 31, 2019
fmassa added a commit that referenced this pull request Oct 31, 2019
* Handle corrupted video headers in io

* Catch exceptions while decoding partly-corrupted files

* Add more tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support for large video datasets that contain a few corrupted files
3 participants