Allow adaptive track groups for both audio and video (+allow mixed-sample-rate audio adaptation)

[freaksgit]

Could you please tell, is there simple way to skip chunk on any exception.
For example, I have a DefaultDashChunkSource. There occured 404 http error when I was trying to load chunks. Can I skip the problem chunks and resume playing the stream?

[tonihei]

Generally, we already blacklist a format if it fails to load and try loading other formats instead. Can you describe your specific case in more detail?

[freaksgit]

Generally, we already blacklist a format if it fails to load and try loading other formats instead. Can you describe your specific case in more detail?

In my case, an 404 http error occurs when loading an audio track (Dash streaming) when one or more audio chunks are needed. I decided to skip the missing chunks. As far as I understand, the Blacklist does not work as I need, it switches to another audio track, although it could just switch to another bitrate.

Questions: Is it possible to skip problem chunks? Is it possible to somehow configure the policy of the blacklist to switch the another bitrate (not audiotrack).

I saw that in the new version (2.9.0) of the player the ability to configure the blacklist was added, but there is no possibility to customize it for my needs.

[tonihei]

As far as I understand, the Blacklist does not work as I need, it switches to another audio track, although it could just switch to another bitrate.

I'm not sure I understand your problem correctly. "Another track" and "another bitrate" seem to be the same thing. If not, can you elaborate on your exact use case?

[freaksgit]

So, I have a videotrack and an audiotrack. Audiotrack exists in three different bitrates:

Renderer:0[...]
Renderer:1 [
Group:0, adaptive_supported=YES_NOT_SEAMLESS [
[ ] Track:0, id=p0a1r0, mimeType=audio/mp4a-latm, bitrate=61000, codecs=mp4a.40.2, channels=2, sample_rate=44100, language=eng, supported=YES
[ ] Track:1, id=p0a1r1, mimeType=audio/mp4a-latm, bitrate=93000, codecs=mp4a.40.2, channels=2, sample_rate=44100, language=eng, supported=YES
[X] Track:2, id=p0a1r2, mimeType=audio/mp4a-latm, bitrate=125000, codecs=mp4a.40.2, channels=2, sample_rate=48000, language=eng, supported=YES ] 

Player.DefaultEventListener.onPlayerError(e: ExoPlaybackException) is triggered on 404 HTTP error (chunk resource is not found) without switching to another bitrate...

I can switch to another bitrate manually, but I thought that it works such way out of the box.

Renderer:0[...]
Renderer:1 [
Group:0, adaptive_supported=YES_NOT_SEAMLESS [
[X] Track:0, id=p0a1r0, mimeType=audio/mp4a-latm, bitrate=61000, codecs=mp4a.40.2, channels=2, sample_rate=44100, language=eng, supported=YES
[ ] Track:1, id=p0a1r1, mimeType=audio/mp4a-latm, bitrate=93000, codecs=mp4a.40.2, channels=2, sample_rate=44100, language=eng, supported=YES
[ ] Track:2, id=p0a1r2, mimeType=audio/mp4a-latm, bitrate=125000, codecs=mp4a.40.2, channels=2, sample_rate=48000, language=eng, supported=YES ]

I think my issue is in wrong using of TrackSelector.

[tonihei]

Thanks for the example, I think I understand the problem now.

It looks as if the problem is that we create a fixed selection (with 1 track only) for audio if we already created an adaptive selection (with more than one track) for video. The automatic blacklisting only works when switching to another track in an adaptive selection. The second issue is that we are currently not creating adaptive selections if the switch is not seamless.

Good news is, both issues are almost fixed already and will be part of the next release. I'll get back to you if both parts are in the dev branch so that you can try it out.

[freaksgit]

Thanks for explanation. I'll wait for new release.
But now we will try to switch audiotrack manually when any HTTP error will occur.

[kpandroid]

It seems that the audio adaptation works very strange - I've checked the following two videos using latest 2.11.4 Demo example with

trackSelector.experimental_allowMultipleAdaptiveSelections();

1. https://bitdash-a.akamaihd.net/content/sintel/hls/playlist.m3u8 - for the same 1080 video resolution I see that the audio track is constantly changed (id=stereo:English->id=surround:English->id=stereo:English->id=surround:English) but should be the same and id=surround:English:

2020-04-15 16:30:49.899 29777-29777/com.google.android.exoplayer2.demo D/EventLogger: decoderInputFormat [eventTime=0.62, mediaPos=0.00, window=0, period=0, audio, id=stereo:English, mimeType=audio/mp4a-latm, codecs=mp4a.40.2, channels=2, sample_rate=48000, language=en, label=English]
2020-04-15 16:30:55.675 29777-29777/com.google.android.exoplayer2.demo D/EventLogger: decoderInputFormat [eventTime=6.39, mediaPos=5.24, window=0, period=0, audio, id=surround:English, mimeType=audio/mp4a-latm, codecs=mp4a.40.2, channels=6, sample_rate=48000, language=en, label=English]
2020-04-15 16:30:57.616 29777-29777/com.google.android.exoplayer2.demo D/EventLogger: decoderInputFormat [eventTime=8.33, mediaPos=7.15, window=0, period=0, audio, id=stereo:English, mimeType=audio/mp4a-latm, codecs=mp4a.40.2, channels=2, sample_rate=48000, language=en, label=English]

2. Big Buck Banny video - http://62.210.177.36/video/5727/index.m3u8

The audio track is always the same - id=audio2160:01, ever if you manually select lowest video resolution.

[tonihei]

@kpandroid The problems you describe occur because HLS doesn't provide bitrates for the adaptive audio tracks. See #6257 (comment) for a slightly longer answer.

[kpandroid]

@tonihei actually I don't understand why we need an audio bitrates in order to select right audio track for specified resolution. I see that for the resolution 1921x818 there is the predefined group of audios - AUDIO="surround". In this group there are only two audio tracks

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="surround",LANGUAGE="en",NAME="English",DEFAULT=YES,AUTOSELECT=YES,URI="audio/surround/en/320kbit.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="surround",LANGUAGE="dubbing",NAME="Dubbing",DEFAULT=NO,AUTOSELECT=YES,URI="audio/stereo/none/128kbit.m3u8"

default is NAME="English". It's just impossible that the player can select another group that belongs to another resolution. I mean that group - it's just out of scope

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="stereo",LANGUAGE="en",NAME="English",DEFAULT=YES,AUTOSELECT=YES,URI="audio/stereo/en/128kbit.m3u8"

If we're going to switch resolution equal or bellow RESOLUTION=638x272 we get new audio group AUDIO="stereo" and the same two audio tracks:

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="stereo",LANGUAGE="en",NAME="English",DEFAULT=YES,AUTOSELECT=YES,URI="audio/stereo/en/128kbit.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="stereo",LANGUAGE="dubbing",NAME="Dubbing",DEFAULT=NO,AUTOSELECT=YES,URI="audio/stereo/none/128kbit.m3u8"

and of course If we've selected audiotrack 'English' before we should switch to the corresponded
audio track

ID="stereo",LANGUAGE="en",NAME="English",DEFAULT=YES,AUTOSELECT=YES,URI="audio/stereo/en/128kbit.m3u8"

[tonihei]

It's just impossible that the player can select another group that belongs to another resolution. I mean that group - it's just out of scope

ExoPlayer's track selection is independent of these groups. You can manually select any audio track for any video track. Deducing the order of audio tracks (or even the concrete group mapping) from the HLS playlist is tracked by #6257 and independent of this feature.