Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support h.264 codec in re_video #7375

Closed
jprochazk opened this issue Sep 9, 2024 · 1 comment · Fixed by #7511
Closed

Support h.264 codec in re_video #7375

jprochazk opened this issue Sep 9, 2024 · 1 comment · Fixed by #7511

Comments

@jprochazk
Copy link
Member

// TODO(jan): support h.264, h.265, vp8, vp9
let stsd = &video_track.trak(&mp4).mdia.minf.stbl.stsd;
let codec_name = if stsd.avc1.is_some() {
"avc"
} else if stsd.hev1.is_some() {
"hevc"
} else if stsd.vp09.is_some() {
"vp9"
} else {
"unknown"
};
return Err(VideoLoadError::UnsupportedCodec(codec_name.into()));

@emilk emilk changed the title Support more video codecs in re_video Support h.264 h.265 vp8 vp9 video codecs in re_video Sep 10, 2024
@emilk emilk added this to the 0.19 - Dataframe and web video milestone Sep 10, 2024
@emilk emilk changed the title Support h.264 h.265 vp8 vp9 video codecs in re_video Support h.264 codecs in re_video Sep 16, 2024
@emilk emilk changed the title Support h.264 codecs in re_video Support h.264 codec in re_video Sep 20, 2024
@jprochazk
Copy link
Member Author

jprochazk commented Sep 23, 2024

We can load h264 as of #7470, but playback is choppy. Remaining work here is to fix how we pass timestamps to the video decoder, which requires some restructuring of our buffering logic.

jprochazk added a commit that referenced this issue Sep 26, 2024
### What

* Closes #7375

- The seeking logic now searches for the requested segment by the
presentation/composition timestamp passed in by the user. This way if a
user requests frame at 12s they will get the frame which will be
_presented_ at 12s, not _decoded_ at 12s. See [I/P/B frame
terminology](https://en.wikipedia.org/wiki/Video_compression_picture_types)
in video encoding for more information as to why this distinction
matters.
- `EncodedVideoChunk`s are now given the _composition timestamp_ of the
sample instead of its _decode timestamp_, so the output `VideoFrame`
objects are in _composition timestamp_ order.

### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested the web demo (if applicable):
* Using examples from latest `main` build:
[rerun.io/viewer](https://rerun.io/viewer/pr/7511?manifest_url=https://app.rerun.io/version/main/examples_manifest.json)
* Using full set of examples from `nightly` build:
[rerun.io/viewer](https://rerun.io/viewer/pr/7511?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json)
* [x] The PR title and labels are set such as to maximize their
usefulness for the next release's CHANGELOG
* [x] If applicable, add a new check to the [release
checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)!
* [x] If have noted any breaking changes to the log API in
`CHANGELOG.md` and the migration guide

- [PR Build Summary](https://build.rerun.io/pr/7511)
- [Recent benchmark results](https://build.rerun.io/graphs/crates.html)
- [Wasm size tracking](https://build.rerun.io/graphs/sizes.html)

To run all checks from `main`, comment on the PR with `@rerun-bot
full-check`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants