feat: support Dolby Vision profile 8.x signaling for HLS

[XingzhaoYun]

Support Dolby Vision profile 8.1, 8.2, 8.4 signaling in HLS. This PR is based on Dolby Vision Streams Within the ISO Base Media File Format Version 2.4 and Dolby Vision Streams Within the HTTP Live Streaming format version 2.1

The changes mainly include,

1. use SUPPLEMENTAL-CODECS for back compatible together with CODECS, rather than two entries.
2. add Dolby Vision compatible brands, 'db1p', 'db2g', 'db4g', 'db4h', 'dby1' based on https://mp4ra.org/#/brands

[sr1990]

Hi @XingzhaoYun, a few questions/pointers:

1. Which players support this tag?
2. Looks like this PR changes the current flow (2 variants created for DV profile 8.x). Also, there is a similar way to indicate supplemental codecs in DASH using @scte214:supplementalCodecs and @scte214:supplementalProfiles as per https://professionalsupport.dolby.com/s/article/How-to-signal-Dolby-Vision-in-MPEG-DASH?language=en_US
   by create one adaptation set instead of two.
   Another way to add these would be to create a flag to enable/disable supplemental codecs tag/supplemental property indication, keeping the current flow.
   What do you think?
3. As per https://developer.apple.com/av-foundation/High-Dynamic-Range-Metadata-for-Apple-Devices.pdf, we should add colr atom to the output mp4 for DV, HDR, and HLG content.
4. We should also update the unit tests and end to end tests.
5. A separate question:
   In feat(DASH): Add video transfer characteristics. #1210 ( indicating transfer characteristics in DASH mpd), transfer characteristics are parsed out from SPS VUI and for profile 5, transfer characteristics value that is parsed out of SPS VUI is 2 (Unspecified) instead of 16 (PQ). Is this correct?

[XingzhaoYun]

1. Which players support this tag?

So far hls.js supports this tag, exoplayer ignores the tag and plays Dolby Vision profile 8.1 and 8.4 regardless the presence of tag.

Another way to add these would be to create a flag to enable/disable supplemental codecs tag/supplemental property indication, keeping the current flow.
What do you think?

One of the limitations of current flow is that it misleads the users for using two different entries indicating the same content, which might cause track selection issue in player side. Considering Dolby has given up the dual entry solution, the disappearance is just a matter of time.

3. As per https://developer.apple.com/av-foundation/High-Dynamic-Range-Metadata-for-Apple-Devices.pdf, we should add colr atom to the output mp4 for DV, HDR, and HLG content.

Yes, we are planning to add these new atoms in later PR, it's not in the scope of this one, since they are not in Dolby Vision Streams Within the ISO Base Media File Format Version 2.4 and Dolby Vision Streams Within the HTTP Live Streaming format version 2.1 so far.

4. We should also update the unit tests and end to end tests.

Will add it soon, thanks for reminding!

5. A separate question:
   In feat(DASH): Add video transfer characteristics. #1210 ( indicating transfer characteristics in DASH mpd), transfer characteristics are parsed out from SPS VUI and for profile 5, transfer characteristics value that is parsed out of SPS VUI is 2 (Unspecified) instead of 16 (PQ). Is this correct?

Dolby Vision profile 5 must be PQ. The VUI information is optional, usually we don't care the value of transfer_characteristics, both 2(UNSPECIFIED) and 16 (PQ) are valid.

[sr1990]

Thanks for the info.

One of the limitations of current flow is that it misleads the users for using two different 
entries indicating the same content, which might cause track selection issue in player side. 
Considering Dolby has given up the dual entry solution, the disappearance is just a matter of time.

Makes sense. However, considering this change is implemented in only one player, I think it will take some time for all the players to implement parsing of this tag and if the existing logic (creating 2 variants / adaptation sets) is removed, players that have not implemented this will end up selecting the base codec content.

Currently,

1. MuxerListerner is created at
   

   shaka-packager/packager/packager.cc

   Line 704 in cc9a691

   std::unique_ptr<MuxerListener> muxer_listener =

2. Above creates 2 MuxerListeners and both are used if DV P8 is content is present
   

   shaka-packager/packager/media/event/muxer_listener_factory.cc

   Line 104 in cc9a691

   for (int i = 0; i < 2; i++) {

3. While parsing moov box if dolby stream is found
   

   shaka-packager/packager/media/formats/mp4/mp4_media_parser.cc

   Line 679 in cc9a691

   if (!UpdateCodecStringForDolbyVision(

   
   will add ; separated codec string at
   

   shaka-packager/packager/media/formats/mp4/mp4_media_parser.cc

   Line 164 in cc9a691

   *codec_string += ";" + dovi_config.GetCodecString(FOURCC_dvh1);

   
   in case of profile 8
4. In MultiCodecMuxerListener::OnMediaStart, the above codec string is split at
   

   shaka-packager/packager/media/event/multi_codec_muxer_listener.cc

   Line 22 in cc9a691

   base::SplitString(stream_info.codec_string(), ";", base::KEEP_WHITESPACE,

   
   and 2 variants/adaptation sets are created in the manifest files.

This PR changes
3 mentioned above adding supplemental codec and compatible brand
and
4 mentioned above by creating single variant/adapation set in the manifest files.

Even if the generated HLS manifest is correct, the DASH manifest will point only to the base codec stream.
That is why I think DASH manifest creation logic should also be updated and for backward compatibility,
a flag to indicate supplemental codecs tag/supplemental property for DASH and HLS should be added in one PR.

[XingzhaoYun]

players that have not implemented this will end up selecting the base codec content.

This is true if players only rely on manifest level of track selection, I agree with you, but here I have some concerns,

• If the manifest parser fails to recognize stream as Dolby Vision, the playback solution will still allow playing DV as long as it has dvvc/dvwc/dvcc box. The only concern would be from the manifest level of track selection, if new tags are not supported, it only see the base codec for sure (even it plays as DV). However, players may choose not using the manifest solely for track selection, as long as they check deeper into container level, they will find it's Dolby Vision.

• With that said, track selection is a player level of action, we don't see many players align on the dual entry selection logics, and they also don't have to. If the old logic already blocked the progress of track selection, it would be a perfect time to roll out the new logic for simplicity and give up the old one. Old versions of releases are still available for users if they want to try old logic.

Even if the generated HLS manifest is correct, the DASH manifest will point only to the base codec stream. That is why I think DASH manifest creation logic should also be updated and for backward compatibility, a flag to indicate supplemental codecs tag/supplemental property for DASH and HLS should be added in one PR.

I agree we should keep them aligned, I will send a new PR for including both HLS and DASH. Thanks!

[sr1990]

With that said, track selection is a player level of action, we don't see many players align on the dual entry selection logics, and they also don't have to. If the old logic already blocked the progress of track selection, it would be a perfect time to roll out the new logic for simplicity and give up the old one. Old versions of releases are still available for users if they want to try old logic.

1. For players that utilize Media Capabilites API, decodingInfo() will be called for each variant to check which one can be decoded and played. If there are multiple variants that can be played, codec preference can be decided based on decodingInfo's response or application's preference.
   Shaka player has: https://github.com/shaka-project/shaka-player/blob/main/docs/design/codec_preferences.md#configurable-codec-priorities

2. For players that rely on manifest files and not look into the container, what is the expected behavior on the player side when supplemental codec tag is found and how different is it from logic mentioned above?
   In my opinion, the above logic to decide which variant to play won't change even if the SUPPLEMENTAL-CODEC tag is added.

@joeyparrish, any thoughts on above? Please let me know if I missed anything.

[XingzhaoYun]

Also, one of the drawbacks of using two entries for signaling back-compatible content is that, it misleads the player to double buffer the content even though they are the same. Comcast and Apple are not a big fan of the two entries, so they never tried to support it.

FYI, apart from Apple and Comcast, Roku and LG has begun the process for supporting the new scheme of signaling. We're also working on the new scheme on exoplayer.

[vish91]

Hey @XingzhaoYun @sr1990 do we know the status of this PR ? I would like to use this feature for HLS and DASH , could you please help lay out the work needed on this PR please.

[XingzhaoYun]

Hey @XingzhaoYun @sr1990 do we know the status of this PR ? I would like to use this feature for HLS and DASH , could you please help lay out the work needed on this PR please.

Hi @vish91 , we're working on updating this PR, some of the requirements are changed due to the new specs release.

[joeyparrish]

Please merge with main, which has changed a lot in the recent cmake port. Thanks!

[cosmin]

@XingzhaoYun my understanding is that you are planning further changes to this PR, converting it to draft for now to signal it's not yet ready

[cosmin]

Replaced by #1396