| uip | title | description | author | status | type | category | created |
|---|---|---|---|---|---|---|---|
0120 |
HTTP Streaming |
Support Media Streaming from Eyre |
~rovnys-ricfer |
Draft |
Standards Track |
Kernel |
2024-01-26 |
As an HTTP server, Urbit should support streaming media, including video and audio, down to clients. Eyre should recognize the standard HTTP headers browsers use to fetch a range of a media file. This should be implemented as a namespace overlay over Eyre's existing interface for scrying over HTTP. Runtime dispatching and caching for this feature are not required but should be supported to allow for performance optimizations.
Now that Urbit supports scrying over HTTP, as defined in UIP-0106, augmenting Urbit with the capability to stream audio and video to web clients should be straightforward to implement. This represents a major jump in user-facing functionality for surprisingly little backend work, and the implementation should be a clean layering over existing functionality, introducing a minimal amount of complexity.
Browsers expect servers to support a certain API to stream video using the HTML5 <video> tag. A browser typically sends HTTP GET requests containing the range header to fetch a byte range within the file. A server's successful response to such a request should have status code 206, meaning partial response, in addition to the content-range header, and in the case of multiple ranges, a content-type header with value multipart/byteranges along with a content-type header for each range.
The server needs to be able to handle requests with range headers of the following forms:
range: bytes=<start>-<stop>
range: bytes=<start>-
range: bytes=-<stop>
range: bytes=<range1>,<range2>,...<rangeN>
This server-side behavior can be implemented in Eyre. If Eyre receives an HTTP request as an Arvo event whose path indicates it is a scry request (see UIP-0106), then Eyre will also check for the presence of the range header. If the header is present, Eyre will perform the scry request, convert the result to the %mime mark, then slice the response body based on the requested byte range(s).
To allow the runtime to optimize this functionality, the runtime can scry into Arvo for a range of bytes rather than the entire noun. This scry endpoint is an overlay namespace over the basic HTTP scry endpoint. The path looks like this:
:: format
/ex/<ship>//<case>/range/<start>/<stop>/<wrapped-path>
:: examples
/ex/~zod//~2024.3.10/range//1023/<wrapped-path> :: start=null, end=1023
/ex/~zod//~2024.3.10/range/1024/2047/<wrapped-path> :: start=1024, end=2047
/ex/~zod//~2024.3.10/range/2048//<wrapped-path> :: start=2048, end=nullThe <wrapped-path> in this example can be any path supported by the usual HTTP scrying interface, namely either a fully qualified scry path or a path containing ='s that can be interpolated by the server to a fully qualified scry path based on the current "beak" (ship, desk, and date).
While Eyre needs to handle multirange requests in its stateful Arvo event handler, to keep the external Arvo scry interface simple, Vere should convert multirange HTTP requests into multiple single-range scry requests before scrying into Arvo.
Note that there are at least three different possible designs for how the system could cache and slice scry results. In one design, when asked for either a whole file or a range, Vere requests the entire file, caches it, and takes byte slices out of the cached value when requested. This minimizes the number of calls to Arvo's +peek arm and IPC overhead, at the expense of potentially requiring a much larger value to be cached in the IO process.
Another option: Vere relays the request faithfully to Arvo, requesting a slice if that was indicated by the range header. Each such request incurs IPC latency and Arvo +peek processing overhead, but the memory requirements in the IO process are smaller.
A third, intermediate, option would be for Vere to request a fixed-size "page" worth of data whenever asked for a slice, no matter how big the slice is. Satisfying a request would be more complex, because it could involve slicing and stitching the cached pages, but the memory use and overhead from IPC and Arvo +peek calls would be much more predictable.
The choice of caching strategy can be left up to the runtime as long as Eyre supports arbitrary single-range queries in its scry interface.
How should the system handle the case where a web browser expects the Urbit to be able to handle a range header in the request, but the Urbit has an old runtime that doesn't know how to handle the request?
One answer is that adding streaming functionality as part of a Kelvin release would at least allow client code distributed by Urbit (i.e. globs) to know whether the server can handle such requests, by knowing its own kernel Kelvin version.
There are cross-origin resource sharing considerations with range headers. Specifically, a request for a single range is a CORS-safelisted request header.
TODO: how to handle CORS for multiple-range requests? Do we even need to support multiple ranges?
Copyright and related rights waived via CC0.