Notes on SASH in 2019 and a v0.2 proposal

[philcluff]

There's 2 pieces to this PR for discussion:

1. My thoughts on the decisions we made 4 years ago, and where I think we're wrong.

2. A proposed set of changes to layout a version 0.2, including a first pass of disco support.

I'd love some feedback!

[bvibber]

A few notes:

Re: startup acceleration / init segments -- embedding init segments in the manifest is probably easier for our tooling at Wikipedia than setting up HTTP/2 server push for lots of little segments. Best to assume the server is 'dumb' or even not under the content creator's control (random CDNs, or say, video help documentation as part of an open source web app that might deploy in many different web environments).

Re: adaptation set grouping -- probably makes sense to group mutually-exclusive options. For text tracks I think you can only have one active so captions and subtitles would be mutually exclusive, although they are semantically distinct from each other?

Ads and DRM aren't in our use cases for Wikipedia but I agree they're important for so many that you need to support them well. :) I'm not familiar enough with discontinuities in HLS to comment on that.

Need something for marking 360/3d/vr but I don't know the state of things there. User data is a great escape. ;)

Progressive sources -- I can also see this being useful for an interchange format to describe both the adaptive sets and the progressive download stub versions. From a static playback perspective we'd usually include those as <source>s on the <video>, with another <source> for the SASH manifest that gets loaded by our player JS. But if you're dynamically instantiating players, then pulling a single manifest and then getting the data right out of it to use in place of MSE chunks might be nicer indeed. Could be left to user_data but I like the idea of having a standard named key.

[OpenToInnovate]

Thanks @geneticgenesis for coming back to this project with an updated proposal! An open transport is definitely needed, CMAF was a big step in this direction however having a community driven alternative would be better. I found this check-in through your "Streaming video on the internet without MPEG" blog post. A very helpful article!

If you don't mind I have some feedback for the next revision:

• Representations -> Renditions. Good call, I feel this term will make it easer to explain what's going on to others.

• SASH is an awesome codename but having OpenABR in the pocket might be a good idea if you need to distance SASH from DASH.

  • OpenABR.com & org are available. I'd suggest you buy them up before MPEG-LA does :)

• Being Container and codec agnostic is extremely important so I 100% agree on the direction you're going there.

• Discontinuity - I vote for the HLS method initially, at least to a point where all discontinuity edge cases that HLS supports are covered.

• Decoder startup acceleration support - I'm a bit worried about this one. If you want to support all HLS/DASH assets without a re-mux then you'll possibly overlap on some of the patent pool. It might be best to come up with a clever way of doing this that no one has done before and publish a white paper for protection.

• HDR, I suggest you treat it as a different codec and don't allow mixing codecs, keep it simple. There is no reason why you'd want lower bitrates to be 8bit and higher to be 10. It's one or the other.

• DRM -> An area I've spent lots of time in. I suggest you take the fields that DASH has in XML and map them to Json, nothing more. None of this is covered in the patent pool as it's already borrowed from the DRM specs heavily. It's important to plug into players that support CENC/PlayReady/Widevine/etc with a 1:1 match when possible since they'll be exposing params to their DRM implementation that require all of these fields and handle everything else internally.

• User data -> Before throwing everything into here I'd suggest you look close at how it's used today. User data is extremely important but makes it hard for the players to integrate if it's used too much.

• 360 / VR / AR -> What is needed here is an easy way to support tiled streams. This is unfortunately not Codec agnostic but it would be great to offer it on one of the free codecs and not just locked to HEVC. Even though there is a spec on how to do this with DASH, it's not really used simply down to the encoding workflow & player support. Everyone I've seen claiming foveated VR playback does it their own proprietary way (aka: an ugly hack). Simplifying this and making it part of the initial version of the spec would make SASH the safe bet for VR/AR/foveated content. ( https://arxiv.org/pdf/1701.06509.pdf & https://gpac.wp.imt.fr/2016/05/25/srdtuto/ )

• Live-streaming support is important. Supporting TS streams and simple, refreshing manifests like HLS would be ideal. An Nginx plug-in that does RTMP->SASH would also be a huge help to increase adoption. Many use this plug-in heavily: https://github.com/arut/nginx-rtmp-module so extending this would be necessary.

Cheers!
Tony

[philcluff]

Thanks for the great feedback, @Brion and @OpenToInnovate

I'm planning on merging this PR so that the new manifests are front and center, and then raising issues for the outstanding pieces I and you have identified. Also, good spot on the domains. cough

Thanks!