Skip to content

Commit

Permalink
add info on videoCodec option
Browse files Browse the repository at this point in the history
  • Loading branch information
yume-chan committed Jan 19, 2025
1 parent ac6e140 commit 210b22f
Show file tree
Hide file tree
Showing 6 changed files with 69 additions and 37 deletions.
4 changes: 2 additions & 2 deletions docs/api/bin/pm/install-stream.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@

Install a single APK from a `ReadableStream`.

It will use streaming install if the device supports it. The APK file will be uploaded, parsed, and installed at the same time.
It will use streaming install if supported by the device. This means the APK file will be uploaded, parsed, and installed in parallel.

If the device does not support streaming install, it will first use [`AdbSync.prototype.write`](../../adb/sync/write.mdx) to upload the APK file to a temp folder on the device, then use [`install`](./install.mdx) method to install it.
If streaming install is not supported, `installStream` will first call [`AdbSync.prototype.write`](../../adb/sync/write.mdx) to upload the APK file to a temp folder on the device, then call [`install`](./install.mdx) method with the file path.

```ts showLineNumbers
export interface PackageManagerInstallOptions {
Expand Down
2 changes: 1 addition & 1 deletion docs/api/bin/pm/install.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Install an APK file on device filesystem.

If the APK file is not on the device yet, [`installStream`](./install-stream.mdx) can upload and install it at the same time.
If the APK file is not on the device yet, [`installStream`](./install-stream.mdx) can upload and install it in parallel.

```ts showLineNumbers
export interface PackageManagerInstallOptions {
Expand Down
1 change: 1 addition & 0 deletions docs/can-i-use.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ export default function CanIUse(props: { feature: string }) {
<>
<p
className="ciu_embed"
style={{ marginBottom: 0 }}
data-feature={props.feature}
data-periods="future_1,current,past_1,past_2"
data-accessible-colours="true"
Expand Down
36 changes: 27 additions & 9 deletions docs/scrcpy/options/video-codec.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,24 +4,42 @@ sidebar_position: 1

# Video codec

Scrcpy 2.0 added the option to use H.265 (HEVC) and AV1 video codecs. They can provide better video quality at the same bitrate compared to H.264.
:::info

Added in Scrcpy 2.0

:::

```ts
namespace ScrcpyOptions2_0 {
export interface Init {
videoCodec?: "h264" | "h265" | "av1";
}
}
```

The `videoCodec` option specifies the video codec to use by the server.

The specified `videoCodec` must be supported by the device (has an encoder for it), otherwise an error will be thrown in the startup process.

It's also recommended to use a hardware-based encoder for best experience. From Scrcpy 3.0, the output of `--list-encoders` (or `AdbScrcpyClient.getEncoders` method) will include the hardware type of each encoder.

## H.264

H.264 is the default video codec. It's widely supported and provides good video quality. Most devices since Android 5 has hardware support for H.264 encoding and decoding.
H.264 is the default value, and the only supported video codec in earlier versions of Scrcpy.

Most devices since Android 5 has hardware-based H.264 encoders.

## H.265 (HEVC)

H.265 is a newer video codec that provides better video quality at the same bitrate compared to H.264. However, it requires more processing power to encode and decode.
H.265 can provide better video quality at the same bitrate compared to H.264.

H.265 encoder is not required by Google, but many devices since Android 7 have hardware support for H.265 encoding and decoding.
H.265 encoder is not required by Google, but many devices since Android 7 have hardware-based H.265 encoders.

## AV1

AV1 is a more recent video codec that provides even better video quality at the same bitrate compared to H.265.

Android 14 is the first version to require AV1 encoder and decoder support, and only the most recent SoCs have hardware support for AV1 encoding and decoding.
AV1 can provide even better video quality at the same bitrate compared to H.265.

## VP9
AV1 encoder is mandatory since Android 14, however, the built-in, software-based encoder by Google is too slow to be used in production.

VP9 is a video codec developed by Google. Android Studio's screen mirror feature uses VP9 to encode video, but in my testing, the device support is not as good as H.264.
Support for hardware-based AV1 encoders can be found at https://en.wikipedia.org/wiki/AV1#Hardware.
53 changes: 29 additions & 24 deletions docs/scrcpy/video/web-codecs.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -18,21 +18,7 @@ npm install @yume-chan/scrcpy-decoder-webcodecs

<CanIUse feature="webcodecs" />

### Microsoft Edge on Windows

By default, Chromium browsers uses FFMpeg internally for WebCodecs API. However, Microsoft Edge, when running on Windows, uses Media Foundation decoders instead.

Decoding H.265 requires the [**HEVC Video Extensions**](https://www.microsoft.com/store/productId/9NMZLZ57R3T7) ($0.99) or **HEVC Video Extensions from Device Manufacturer** (free but not available anymore) app from Microsoft Store.

Decoding AV1 requires the [**AV1 Video Extension**](https://www.microsoft.com/store/productId/9MVZQVXJBQ9V) (free) app from Microsoft Store.

### Firefox

Firefox 133 supports playing H.265 videos, but does not support decoding H.265 streams using WebCodecs API.

## Feature detection

Because WebCodecs API is not supported in all browsers, you should check if it is supported before using it.
Because WebCodecs API is still relatively new, you should check if it is supported before using it.

```ts transpile
import { WebCodecsVideoDecoder } from "@yume-chan/scrcpy-decoder-webcodecs";
Expand All @@ -54,11 +40,11 @@ If WebCodecs API is not supported, you can fallback to the [TinyH264 decoder](./

## Codec support

Even when the browser supports WebCodecs API, the video codecs they support may vary.
Scrcpy v2.0 added the [`videoCodec`](../options/video-codec.mdx) option, to specify the video codec to use by the server. H.265 and AV1 codecs can provide better video quality at the same bitrate compared to H.264.

[`VideoDecoder.isConfigSupported`](https://developer.mozilla.org/en-US/docs/Web/API/VideoDecoder/isConfigSupported_static) static method can be used to check if a given codec configuration is supported.
WebCodecs spec supports H.264, H.265 and AV1, but runtime support requires a joint effort from browsers, operating systems, graphics cards and drivers. H.264 should be supported by all browsers that support WebCodecs API, but support for other newer codecs may vary.

It accepts a codec parameter string, for example `"hev1.1.60.L153.B0.0.0.0.0.0"` for H.265 and `"av01.0.05M.08"` for AV1.
[`VideoDecoder.isConfigSupported`](https://developer.mozilla.org/en-US/docs/Web/API/VideoDecoder/isConfigSupported_static) static method can be used to check if a given codec is supported. It takes a codec parameter string, for example `"hev1.1.60.L153.B0.0.0.0.0.0"` for H.265 and `"av01.0.05M.08"` for AV1.

```ts transpile
const result = await VideoDecoder.isConfigSupported({
Expand All @@ -67,7 +53,17 @@ const result = await VideoDecoder.isConfigSupported({
const isHevcSupported = result.supported === true;
```

Scrcpy v2.0 added the `videoCodec` option to specify the video codec. You should use a codec that is supported by both browser and device.
### Microsoft Edge on Windows

By default, Chromium browsers uses FFMpeg internally for WebCodecs API. However, Microsoft Edge, when running on Windows, uses Media Foundation decoders instead.

Decoding H.265 requires the [**HEVC Video Extensions**](https://www.microsoft.com/store/productId/9NMZLZ57R3T7) ($0.99) or **HEVC Video Extensions from Device Manufacturer** (free but not available anymore) app from Microsoft Store.

Decoding AV1 requires the [**AV1 Video Extension**](https://www.microsoft.com/store/productId/9MVZQVXJBQ9V) (free) app from Microsoft Store.

### Firefox

Firefox 133 supports playing H.265 videos, but does not support decoding H.265 streams using WebCodecs API.

## Renderer

Expand Down Expand Up @@ -246,12 +242,21 @@ decoder.sizeChanged(({ width, height }) => {

## Use in Web Worker

Although WebCodecs API already runs in its own thread, and the renderers are very fast, you might still want to run them in a dedicated Web Worker, so other works on the main thread won't affect the performance.
Although WebCodecs API already runs in its own thread, and the renderers are very fast, you might still want to run them in a dedicated Web Worker, so other tasks on the main thread won't affect its responsiveness.

Only `WebGLVideoFrameRenderer` and `BitmapVideoFrameRenderer` are supported in Web Worker. There are two ways to render the frames:

- When their constructors are called without arguments, they will create an `OffscreenCanvas` object. See [Synchronous display of frames produced by an OffscreenCanvas](https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas#synchronous_display_of_frames_produced_by_an_offscreencanvas) on MDN for how to use it.
- Call `transferControlToOffscreen()` on a `<canvas>` element, `postMessage` the `OffscreenCanvas` object to the worker, and use it to create the renderer.
### Create `OffscreenCanvas` directly

When their constructors are called without arguments, they will create an [`OffscreenCanvas`](https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas) object internally.

This `OffscreenCanvas` is not bound to a `<canvas>` element. See [Synchronous display of frames produced by an OffscreenCanvas](https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas#synchronous_display_of_frames_produced_by_an_offscreencanvas) on MDN for how send the image data to a normal `<canvas>` element.

### Create `OffscreenCanvas` from a `<canvas>` element

An [`OffscreenCanvas`](https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas) object can be created by calling [`transferControlToOffscreen()`](https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement/transferControlToOffscreen) method on an existing [`<canvas>`](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/canvas) element.

It can then be [`postMessage`](https://developer.mozilla.org/en-US/docs/Web/API/Worker/postMessage) to the worker, and pass into [renderer constructors](#create-a-renderer). When the renderer draws on the `OffscreenCanvas`, the content will be displayed on the source `<canvas>` element automatically.

```html
<!-- index.html -->
Expand Down Expand Up @@ -322,7 +327,7 @@ Because WebCodecs decoder can render to different types of targets, it's more di
To help with that, the decoder provides the `snapshot` method to easily capture the last rendered frame as a PNG image.

```ts transpile
const blob = await decoder.snapshot();
const blob: Blob | undefined = await decoder.snapshot();
```

Only when no frames has been rendered, the return value will be `undefined`.
If no frames has been rendered, the return value will be `undefined`. Otherwise it will be a [`Blob`](https://developer.mozilla.org/en-US/docs/Web/API/Blob) object with the PNG image data.
10 changes: 9 additions & 1 deletion docs/tango/web-stream/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,19 @@
sidebar_position: 1
---

import CanIUse from "../../can-i-use";

# Web Streams Basics

{/* cspell: ignore interoperating */}

[`ReadableStream`](https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream) and [`WriteableStream`](https://developer.mozilla.org/en-US/docs/Web/API/WritableStream) are from [Web Streams API](https://developer.mozilla.org/en-US/docs/Web/API/Streams_API). They represent a stream of data that can be read from or written to asynchronously.
Tango library uses [Web Streams API](https://developer.mozilla.org/en-US/docs/Web/API/Streams_API) to manage asynchronous I/O. Similar to [Node.js Streams](https://nodejs.org/api/streams.html), it provides a unified API for reading from and writing to data streams.

## Runtime support

<CanIUse feature="streams" />

Node.js also supports Web Streams API since [v18.0.0](https://nodejs.org/api/webstreams.html).

## `@yume-chan/stream-extra` package

Expand Down

0 comments on commit 210b22f

Please sign in to comment.