Integrated base decoder into VideoReader class and video_utils.py #1766
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This pull request was exported from Phabricator. Differential Revision: D19415903 |
1 similar comment
|
This pull request was exported from Phabricator. Differential Revision: D19415903 |
putivsky
pushed a commit
to putivsky/vision
that referenced
this pull request
Jan 21, 2020
…torch#1766) Summary: Pull Request resolved: pytorch#1766 Replaced FfmpegDecoder (incompativle with VUE) by base decoder (compatible with VUE). Modified python utilities video_utils.py for internal simplification. Public interface got preserved. Differential Revision: D19415903 fbshipit-source-id: 4728b6ba8333ef59bb97230f82bbff8724171ad6
putivsky
force-pushed
the
export-D19415903-to-fbsync
branch
from
fb1bb11 to
5454547
Compare
putivsky
pushed a commit
to putivsky/vision
that referenced
this pull request
Jan 22, 2020
…torch#1766) Summary: Pull Request resolved: pytorch#1766 Replaced FfmpegDecoder (incompativle with VUE) by base decoder (compatible with VUE). Modified python utilities video_utils.py for internal simplification. Public interface got preserved. Differential Revision: D19415903 fbshipit-source-id: 2ff12a72404c256d84e6d4d5c46d0b893d0e4c32
putivsky
force-pushed
the
export-D19415903-to-fbsync
branch
from
5454547 to
ed49b07
Compare
Codecov Report
@@ Coverage Diff @@
## fbsync #1766 +/- ##
=========================================
+ Coverage 0.44% 0.5% +0.05%
=========================================
Files 90 90
Lines 7197 7197
Branches 1089 1089
=========================================
+ Hits 32 36 +4
+ Misses 7157 7148 -9
- Partials 8 13 +5
Continue to review full report at Codecov.
|
|
This pull request was exported from Phabricator. Differential Revision: D19415903 |
putivsky
pushed a commit
to putivsky/vision
that referenced
this pull request
Jan 23, 2020
…torch#1766) Summary: Pull Request resolved: pytorch#1766 Replaced FfmpegDecoder (incompativle with VUE) by base decoder (compatible with VUE). Modified python utilities video_utils.py for internal simplification. Public interface got preserved. Differential Revision: D19415903 fbshipit-source-id: c750e2b74573a60f7f6930f1859c577e26245b89
putivsky
force-pushed
the
export-D19415903-to-fbsync
branch
from
ed49b07 to
08d5f42
Compare
|
This pull request was exported from Phabricator. Differential Revision: D19415903 |
1 similar comment
|
This pull request was exported from Phabricator. Differential Revision: D19415903 |
putivsky
pushed a commit
to putivsky/vision
that referenced
this pull request
Jan 25, 2020
…torch#1766) Summary: Pull Request resolved: pytorch#1766 Replaced FfmpegDecoder (incompativle with VUE) by base decoder (compatible with VUE). Modified python utilities video_utils.py for internal simplification. Public interface got preserved. Differential Revision: D19415903 fbshipit-source-id: beffc19a6ada9a1be4a0809e2c29e54281b41bea
putivsky
force-pushed
the
export-D19415903-to-fbsync
branch
from
08d5f42 to
a75d19d
Compare
putivsky
pushed a commit
to putivsky/vision
that referenced
this pull request
Jan 27, 2020
…torch#1766) Summary: Pull Request resolved: pytorch#1766 Replaced FfmpegDecoder (incompativle with VUE) by base decoder (compatible with VUE). Modified python utilities video_utils.py for internal simplification. Public interface got preserved. Differential Revision: D19415903 fbshipit-source-id: 998210ded30e228f3a025d3af5054eef8f073fe1
putivsky
force-pushed
the
export-D19415903-to-fbsync
branch
from
a75d19d to
d7d16a8
Compare
|
This pull request was exported from Phabricator. Differential Revision: D19415903 |
putivsky
force-pushed
the
export-D19415903-to-fbsync
branch
from
d7d16a8 to
8c7006f
Compare
putivsky
pushed a commit
to putivsky/vision
that referenced
this pull request
Jan 28, 2020
…torch#1766) Summary: Pull Request resolved: pytorch#1766 Replaced FfmpegDecoder (incompativle with VUE) by base decoder (compatible with VUE). Modified python utilities video_utils.py for internal simplification. Public interface got preserved. Differential Revision: D19415903 fbshipit-source-id: fcd4a7a6453c1468e578441a55ce424b72fe6778
|
This pull request was exported from Phabricator. Differential Revision: D19415903 |
putivsky
pushed a commit
to putivsky/vision
that referenced
this pull request
Jan 28, 2020
…torch#1766) Summary: Pull Request resolved: pytorch#1766 Replaced FfmpegDecoder (incompativle with VUE) by base decoder (compatible with VUE). Modified python utilities video_utils.py for internal simplification. Public interface got preserved. Differential Revision: D19415903 fbshipit-source-id: 96b2512142f1555bb782ac0cc9738e53a2bb2b8e
putivsky
force-pushed
the
export-D19415903-to-fbsync
branch
from
8c7006f to
e5e8683
Compare
|
This pull request was exported from Phabricator. Differential Revision: D19415903 |
1 similar comment
|
This pull request was exported from Phabricator. Differential Revision: D19415903 |
putivsky
pushed a commit
to putivsky/vision
that referenced
this pull request
Jan 31, 2020
…torch#1766) Summary: Pull Request resolved: pytorch#1766 Replaced FfmpegDecoder (incompativle with VUE) by base decoder (compatible with VUE). Modified python utilities video_utils.py for internal simplification. Public interface got preserved. Differential Revision: D19415903 fbshipit-source-id: c48f939c59e2e1c61f44c92424d26a17ae36ee7f
putivsky
force-pushed
the
export-D19415903-to-fbsync
branch
from
e5e8683 to
7f2fb62
Compare
…torch#1766) Summary: Pull Request resolved: pytorch#1766 Replaced FfmpegDecoder (incompativle with VUE) by base decoder (compatible with VUE). Modified python utilities video_utils.py for internal simplification. Public interface got preserved. Reviewed By: fmassa Differential Revision: D19415903 fbshipit-source-id: f55ac5ecf5ddaa218f48e69317814a5cc274ecf2
|
This pull request was exported from Phabricator. Differential Revision: D19415903 |
putivsky
force-pushed
the
export-D19415903-to-fbsync
branch
from
7f2fb62 to
a7b2287
Compare
facebook-github-bot
pushed a commit
that referenced
this pull request
Feb 3, 2020
) Summary: Pull Request resolved: #1766 Replaced FfmpegDecoder (incompativle with VUE) by base decoder (compatible with VUE). Modified python utilities video_utils.py for internal simplification. Public interface got preserved. Reviewed By: fmassa Differential Revision: D19415903 fbshipit-source-id: 4d7a0158bd77bac0a18732fe4183fdd9a57f6402
|
Merged in f5628e0 |
fmassa
pushed a commit
to fmassa/vision-1
that referenced
this pull request
Mar 13, 2020
…torch#1766) Summary: Pull Request resolved: pytorch#1766 Replaced FfmpegDecoder (incompativle with VUE) by base decoder (compatible with VUE). Modified python utilities video_utils.py for internal simplification. Public interface got preserved. Reviewed By: fmassa Differential Revision: D19415903 fbshipit-source-id: 4d7a0158bd77bac0a18732fe4183fdd9a57f6402
fmassa
pushed a commit
to fmassa/vision-1
that referenced
this pull request
Mar 16, 2020
…torch#1766) Summary: Pull Request resolved: pytorch#1766 Replaced FfmpegDecoder (incompativle with VUE) by base decoder (compatible with VUE). Modified python utilities video_utils.py for internal simplification. Public interface got preserved. Reviewed By: fmassa Differential Revision: D19415903 fbshipit-source-id: 4d7a0158bd77bac0a18732fe4183fdd9a57f6402
fmassa
pushed a commit
to fmassa/vision-1
that referenced
this pull request
Mar 16, 2020
…torch#1766) Summary: Pull Request resolved: pytorch#1766 Replaced FfmpegDecoder (incompativle with VUE) by base decoder (compatible with VUE). Modified python utilities video_utils.py for internal simplification. Public interface got preserved. Reviewed By: fmassa Differential Revision: D19415903 fbshipit-source-id: 4d7a0158bd77bac0a18732fe4183fdd9a57f6402
fmassa
added a commit
that referenced
this pull request
Mar 17, 2020
* Base decoder for video. (#1747) Summary: Pull Request resolved: #1747 Pull Request resolved: #1746 Added the implementation of ffmpeg based decoder with functionality that can be used in VUE and TorchVision. Reviewed By: fmassa Differential Revision: D19358914 fbshipit-source-id: abb672f89bfaca6351dda2354f0d35cf8e47fa0f * Integrated base decoder into VideoReader class and video_utils.py (#1766) Summary: Pull Request resolved: #1766 Replaced FfmpegDecoder (incompativle with VUE) by base decoder (compatible with VUE). Modified python utilities video_utils.py for internal simplification. Public interface got preserved. Reviewed By: fmassa Differential Revision: D19415903 fbshipit-source-id: 4d7a0158bd77bac0a18732fe4183fdd9a57f6402 * Optimizating base decoder performance. (#1852) Summary: Pull Request resolved: #1852 Changed base decoder internals for a faster clip processing. Reviewed By: stephenyan1231 Differential Revision: D19748379 fbshipit-source-id: 58a435f0a0b25545e7bd1a3edb0b1d558176a806 * Minor fix and decoder class members access. Summary: Found and fix a bug in cropping algorithm (simple mistyping). Also derived classes need access to some decoder class members, like initialization parameters - make it protected. Reviewed By: stephenyan1231, fmassa Differential Revision: D19895076 fbshipit-source-id: 691336c8e18526b085ae5792ac3546bc387a6db9 * Added missing header for less dependencies. (#1898) Summary: Pull Request resolved: #1898 Include streams/samplers shouldn't depend on decoder headers. Add dependencies directly to the place where they are required. Reviewed By: stephenyan1231 Differential Revision: D19911404 fbshipit-source-id: ef322a053708405c02cee4562b456b1602fb12fc * Implemented VUE Asynchronous Decoder Summary: For Mothership we have found that asynchronous decoder provides a better performance. Differential Revision: D20026194 fbshipit-source-id: 627b91844b4e3f917002031dd32cb19c239f4ba8 * fix a bug in API read_video_from_memory (#1942) Summary: Pull Request resolved: #1942 In D18720474, it introduces a bug in `read_video_from_memory` API. Thank weiyaowang for reporting it. Reviewed By: weiyaowang Differential Revision: D20270179 fbshipit-source-id: 66348c99a5ad1f9129b90e934524ddfaad59de03 * extend decoder to support new video_max_dimension argument (#1924) Summary: Pull Request resolved: #1924 Extend `video reader` decoder python API in Torchvision to support a new argument `video_max_dimension`. This enables the new video decoding use cases. When setting `video_width=0`, `video_height=0`, `video_min_dimension != 0`, and `video_max_dimension != 0`, we can rescale the video clips so that its spatial resolution (height, width) becomes - (video_min_dimension, video_max_dimension) if original height < original width - (video_max_dimension, video_min_dimension) if original height >= original width This is useful at video model testing stage, where we perform fully convolution evaluation and take entire video frames without cropping as input. Previously, for instance we can only set `video_width=0`, `video_height=0`, `video_min_dimension = 128`, which will preserve aspect ratio. In production dataset, there are a small number of videos where aspect ratio is either extremely large or small, and when the shorter edge is rescaled to 128, the longer edge is still large. This will easily cause GPU memory OOM when we sample multiple video clips, and put them in a single minibatch. Now, we can set (for instance) `video_width=0`, `video_height=0`, `video_min_dimension = 128` and `video_max_dimension = 171` so that the rescale resolution is either (128, 171) or (171, 128) depending on whether original height is larger than original width. Thus, we are less likely to have gpu OOM because the spatial size of video clips is determined. Reviewed By: putivsky Differential Revision: D20182529 fbshipit-source-id: f9c40afb7590e7c45e6908946597141efa35f57c * Fixing samplers initialization (#1967) Summary: Pull Request resolved: #1967 No-ops for torchvision diff, which fixes samplers. Differential Revision: D20397218 fbshipit-source-id: 6dc4d04364f305fbda7ca4f67a25ceecd73d0f20 * Exclude C++ test files Co-authored-by: Yuri Putivsky <yuri@fb.com> Co-authored-by: Zhicheng Yan <zyan3@fb.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Replaced FfmpegDecoder (incompativle with VUE) by base decoder (compatible with VUE).
Modified python utilities video_utils.py for internal simplification. Public interface got preserved.
Differential Revision: D19415903