Add support for DASH patch manifests (<PatchLocation>)

[joeyparrish]

Have you read the FAQ and checked for duplicate open issues?
Yes

Is your feature request related to a problem? Please describe.

Manifest updates can be expensive in some cases.

Describe the solution you'd like

DASH has now introduced <PatchLocation> and <Patch>, which give a way to update a small part of the existing manifest instead of re-parsing and merging every time.

Additional context

TODO: Get references to the spec on this (AMD.1, landing in 4th edition according to Hulu)

TOTO: Get sample content for this (should be coming soon to the DASH live sim, according to Hulu)

[sr1990]

Hi @joeyparrish, Any idea where I can find spec and sample content for this one? Thanks.

[joeyparrish]

I was told that this is part of Amendment 1 to the 3rd edition of the DASH spec (which has the official designation "ISO/IEC 23009-1:2014/AMD 1:2015", and is also rolled into the 4th edition of the DASH spec (which has the official designation "ISO/IEC 23009-1:2019".

Sadly, I don't have a publicly available PDF for the spec. It would be nice if there were a public doc, but ISO will sell you a PDF for 198 Swiss Francs (currently about 203 USD or 182 EUR). The "preview" that ISO has available only contains the table of contents, and not even the entire table. :-(

[technogeek00]

@joeyparrish Sorry for the long delay here, wanted to get you links on the final text and some other aspects kept iterating keeping it from vote, but you can find the final 4th and amd1 over on https://github.com/cta-wave/Resources#specification-development-2

[acheung-harmonicinc]

Hi, my buddy and I are checking if we can have DASH patch manifest support based on Shaka or have some preliminary implementation. May I know if you have any plan to start it?

[zuzzurro]

We are extremely interested in this functionality because of the traffic reduction it can bring with large DVR windows. Can you share what your plans are for adding support for it?

[TheModMaker]

We have not started any work on this issue. It is currently on the Backlog, so we have no plans to implement it for the next major release. If this is a priority for someone, we'd be happy to review a PR for this. Since this is a larger feature, you should post a design first (e.g. using a Google doc).

[dave-nicholas]

Hi guys @joeyparrish @TheModMaker.
We are currently looking into this as part of an upcoming requirement.
We want to approach this work in a way that would allow us to upstream these changes.
I have made a doc with our proposed implementation.
Your guidance and input would be most welcome.

https://docs.google.com/document/d/1Z3GkKUe9mM4nub1w_WlP3Bva5oNtlywlzRxTuui_Yac/edit?usp=sharing

[technogeek00]

Chiming in to drop a link to the finalized text in the 5th edition which is freely available on ISO: https://standards.iso.org/ittf/PubliclyAvailableStandards/c083314_ISO_IEC%2023009-1_2022(en).zip

I'd defer to Shaka team on the player semantics in the proposal, but left some comments on the path selection understanding as there are explicit path addressing restrictions that should simplify implementations and must be followed for a compliant implementation:

5.15.3.4 Element Addressing Restrictions
The following element addressing restrictions apply for DASH MPD elements:

• Elements with an @id attribute with uniqueness requirement among its siblings present shall only be identified by the value of this attribute. In particular, Period, AdaptationSet, Representation, and SubRepresentation shall be only addressed via their @id.
• DescriptorType elements with an @schemeIdUri attribute shall be identified by the value of this attribute, for example SupplementalProperty.
• S elements shall be addressed in one of the following ways:
  • @t or @n attribute of S elements if the said attribute is present in the S element.
  • by position.
• All other elements shall be addressed by position, e.g. SegmentTemplate[1].

Examples:

/MPD/Period[@id="1"]/AdaptationSet[@id="1"]/SegmentTemplate[1]/SegmentTimeline[1]/S[@t="12345"]

/MPD/Period[@id="1"]/AdaptationSet[@id="1"]/SegmentTemplate[1]/SegmentTimeline[1]/S[42]@r

I'm happy to review a PR for standards compliance semantics when it is ready.

[dave-nicholas]

@technogeek00 Thank you greatly for your input, we are in the process of trying to get an AdapatationSet@id added to our streams. I am in the currently creating a POC that I will present for review once ready.

[zuzzurro]

Just out of curiosity, @dave-nicholas, were you able to make any progress on this?

[dave-nicholas]

Hi @zuzzurro, We are currently waiting on some other work from our organisation to be merged before opening a PR with these changes, as the MPD Patch work is dependent on this.

[acheung-harmonicinc]

Hi, it may be a bit late.

We have a fork with patch manifest support
https://github.com/harmonicinc-com/shaka-player/commits/feature/dash-patch/v4.0.x

Please let me know if you are interested, then we can create a PR.

[zuzzurro]

As I said previously (in 2021, how time flies...) we are extremely interested in this feature. And the sooner we have it in Shaka, the better. How is your implementation different from the one @dave-nicholas is working on, @acheung-harmonicinc ?

[dave-nicholas]

@zuzzurro I hope the open this PR by early next week.

Our approach differs from @acheung-harmonicinc 's implementation in that we do not re-parse the entire manifest each time, some of our full events can have a size up to 11 hours and re-parsing a manifest that size on a low - mid end device would not be a solution for us (hence the dash manifest parser improvements were brought into scope for us first). We instead parse and process the patch only.

[zuzzurro]

Well, I totally agree with your goal, and my question was mostly for @acheung-harmonicinc , in order to understand better the pros and cons of both implementations.. Clearly I expect that using patch manifests should bring down the requirements for low end devices. If not, why bother?

[zuzzurro]

@dave-nicholas , out of curiosity, do you have a list of vendors that generate DASH streams compatible with your patch?

[zuzzurro]

What is the status of this issue? It looks like the pull request from @dave-nicholas went nowhere? Inquiring minds want to know..

[dave-nicholas]

Hi @zuzzurro , @joeyparrish is on leave at the moment but will review the PR when he returns.

[zuzzurro]

We can hardly wait for this to happen.. Also, @dave-nicholas , do you have an answer for my previous question about vendors supporting this stream format?

[dave-nicholas]

@zuzzurro I don't unfortunately. We are testing Patch internally but don't currently have any public streams yet.

[technogeek00]

I found out there is an NGINX module that can generate patches: https://github.com/ab2022/mpddiffs Not quite sure on the setup requirements but maybe a possibility for you.

[tobbee]

There is now MPD Patch support in DASH-IF livesim2. See https://github.com/Dash-Industry-Forum/livesim2/wiki/MPD-Patch for more information. It's intended to work with all live MPDs that can be produced including multi-period live MPDs.

The official DASH-IF instance at https://livesim2.dashif.org has also been updated.