ExoPlayer: Only allow hardware-accelerated video codecs in DirectPlay

[programminghoch10]

(everything here relates to ExoPlayer and the experience of selecting "Integrated player" in the client settings)

background

When streaming video, ExoPlayer automatically chooses software decoding if there is no hardware decoder available.
However, depending on the codec this may result in an unpleasant experience.
In my case, streaming AV1 on the device gts4lv resulted in significant stutter when the picture was moving quickly.

Currently there exists an toggle in the player to change between hardware and software encoding.
Software decoding always works (i assume), however hardware encoding will result in an black image if the hardware doesnt support decoding this codec.

best possible solution

The best situation (in my opinion) would be, that video is transcoded by default if the hardware doesn't support it (especially on android, where the device is usually run on battery).
So when choosing software decoding, DirectPlay is fine, but when choosing hardware decoding, transcoding should automatically be applied if the hardware doesn't support this codec.

current solution

Sadly due to limited experience with kotlin and the situation I were not able to implement exactly as I would like it to.
This solution here just filters out all codecs from DirectPlay which can't be hardware decoded.

In my situation this forced transcoding of AV1 video and results in a smooth playback experience.

This solution does also prevent direct-streaming the AV1 stream completely,
but I do think that this should generally lead to a better user experience overall.

code explaination

I do know this solution might not be the best.
The ExoPlayer API seems to give hardware decoder feedback based on MIME-Types.
The function MimeTypes.getMediaMimeType(String) can only interpret av01 but not av1, hence the extra additional rewrite of av1 to av01.
The rest of the codecs seem to be fine, I had some logging in there and it did detect hardware decoding support for vp8 and vp9 correctly. (i think also h264 and h265 if i remember correctly)
But there definitely is a better solution for this container list to MIME-Type conversion.

[programminghoch10]

I fixed the detection of h264 and h265 MIME-Types, so this (temporary) approach is usable now.

[Maxr1998]

Thanks for investigating this and your contribution!

So when choosing software decoding, DirectPlay is fine, but when choosing hardware decoding, transcoding should automatically be applied if the hardware doesn't support this codec.

This would definitely be ideal, but I think I could look into it myself later.

Except for some code style tweaks, this should be mergeable as is. I'll see if I find some time to look into it in the next few days.

[programminghoch10]

mpeg4 might not be the same as h264, so this line might need to go

Suggested change

"mpeg4" -> "avc1"

i weren't really sure about this, there's a lot of conflicting information out there about how mpeg4 and h264 relate 🤔

[Maxr1998]

I refactored your code so that it doesn't have to re-read the supported MediaCodecList list every time, but uses the information already present at earlier stages of the DeviceProfileBuilder.

However, I also talked to @nielsvanvelzen, and we agreed that completely disallowing software decoding by default is quite extreme. There should also exist a few software-only decoders that perform reasonably well. Instead, Niels suggested detecting choppy playback and switching the transcoding if applicable. Thus, I'd like to keep this PR open for now while I think about the feasibility of that suggestion.

[programminghoch10]

Why not just do it straightaway like i proposed intially?

The best situation (in my opinion) would be, that video is transcoded by default if the hardware doesn't support it (especially on android, where the device is usually run on battery).
So when choosing software decoding, DirectPlay is fine, but when choosing hardware decoding, transcoding should automatically be applied if the hardware doesn't support this codec.

So by default chose hardware decoding with transcoding (if its required), but just always use the software decoder in direct play when the user switches to software decoding.

Alternatively maybe an additional setting would work, something like "Force hardware decoding even if it requires transcoding".