Add video reference docs

crates/store/re_types/definitions/rerun/archetypes/asset_video.fbs

@@ -4,9 +4,10 @@ namespace rerun.archetypes;
 ///
 /// NOTE: Videos can only be viewed in the Rerun web viewer.
 /// Only MP4 containers with a limited number of codecs are currently supported, and not in all browsers.
-/// Follow <https://github.com/rerun-io/rerun/issues/7298> for updates on the native support.
 ///
-/// In order to display a video, you need to log a [archetypes.VideoFrameReference] for each frame.
+/// See <https://rerun.io/docs/reference/video> for details of what is and isn't supported.
+///
+/// In order to display a video, you also need to log a [archetypes.VideoFrameReference] for each frame.
 ///
 /// \example archetypes/video_auto_frames title="Video with automatically determined frames" image="https://static.rerun.io/video_manual_frames/320a44e1e06b8b3a3161ecbbeae3e04d1ccb9589/1200w.png"
 /// \example archetypes/video_manual_frames title="Demonstrates manual use of video frame references" image="https://static.rerun.io/video_manual_frames/9f41c00f84a98cc3f26875fba7c1d2fa2bad7151/1200w.png"

docs/content/reference/video.md

@@ -0,0 +1,73 @@
+---
+title: Video
+order: 2000
+---
+
+## Video in Rerun
+A stream of images (like those produced by a camera) can be logged to Rerun in several different ways:
+
+* Uncompressed, as many [`Image`](../reference/types/archetypes/image.md)s
+* Compressed as many [`EncodedImage`](../reference/types/archetypes/encoded_image.md)s, using e.g. JPEG-encoding
+* Compressed as a single [`AssetVideo`](../reference/types/archetypes/asset_video.md), using e.g. MP4 and AV1 or H.264.
+
+These alternatives range on a scale of "simple, lossless, and big" to "complex, lossy, and small".
+
+If you want lossless encoded images (with no compression artifacts), then you should log each video frame as `Image`.
+This will use up a lot of space and bandwidth. You can also encode them as PNG and log them as `EncodedImage`,
+though it should be noted that PNG encoding usually does very little for the file size of photographic images.
+
+If you want to reduce bandwidth and storage cost, you can encode each frame as a JPEG and log it using `EncodedImage`. This can easily reduce the file sizes by almost two orders of magnitude with minimal perceptiall loss.
+This is also very simple to do, and the Python logging SDK has built-in support for it using [`Image.compress`](https://ref.rerun.io/docs/python/0.18.2/common/archetypes/#rerun.archetypes.Image.compress).
+
+Finally, you can encode the images as a video file, and log it using `AssetVideo`.
+This gives the best compression ratio, reducing file sizes and bandwidth requirements.
+
+## `AssetVideo` limitations
+Video support is new in Rerun, and has several limitations:
+
+* Video playback only works in the web viewer
+* There is no video encoder in the Rerun SDK, so you need to create the video file yourself
+* Only the MP4 container format is supported
+* Only a limited sets of codecs are supported (see below)
+* There is no audio support
+
+## Web viewer support
+As of writing, playback of `AssetVideo` is only supported on the web viewer.
+Native video playback is coming, and can be tracked [in this GitHub issue](https://github.com/rerun-io/rerun/issues/7298).
+
+Video playback is done using the browsers own video decoder, so the supported codecs depends on your browser.
+
+Overall, we recommend using Chrome or another Chromium-based browser, as it seem to have the best video support as of writing.
+
+When choosing codec, we recommend [AV1](https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Video_codecs#av1), as that seems to have the best overall playback support. Since AV1 is patent-free, it is also likely the first codec we will support in the native viewer.
+
+For decoding video in the Web Viewer, we use the [WebCodecs API](https://developer.mozilla.org/en-US/docs/Web/API/WebCodecs_API).
+This API enables us to take advantage of the browser's hardware accelerated video decoding capabilities.
+It is implemented by all modern browsers, but with varying levels of support for different codecs, and varying levels of quality.
+
+With that in mind, here are the browsers which we have tested and verified to work:
+
+|            | Linux  | macOS | Windows |
+| ---------- | ------ | ----- | ------- |
+| Firefox    | ✅[^2] | ✅    | ✅      |
+| Chrome[^1] | ✅     | ✅    | ✅      |
+| Safari     |        | ✅    |         |
+
+When it comes to codecs, we aim to support any codec which the browser supports, but
+we currently cannot guarantee that all of them will work. For more information about
+which codecs are supported by which browser, see [Video codecs on MDN](https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Video_codecs#codec_details).
+
+At the moment, we test the following codecs:
+
+|       | Linux | macOS | Windows |
+| ----- | ----- | ----- | ------- |
+| AV1   | ✅    | 🚧\*  | ✅      |
+| H.264 | ✅    | ✅    | ✅      |
+
+* AV1 on macOS is only supported on [Apple Silicon devices with AV1 hardware decoding support](https://webkit.org/blog/14445/webkit-features-in-safari-17-0/).
+
+[^1]: Any Chromium-based browser should work, but we don't test all of them.
+[^2]: Firefox on Linux has been observed to [stutter when playing back H.264 video](https://github.com/rerun-io/rerun/issues/7532)
+
+## Links
+* [Web video codec guide, by Mozilla](https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Video_codecs)