Support multiple audio tracks and AV1 for Invidious by using the local API DASH manifest generator #3942

absidue · 2023-08-25T17:49:27Z

Support multiple audio tracks and AV1 for Invidious by using the local API DASH manifest generator

Pull Request Type

Bugfix
Feature Implementation

Related issue

Description

Currently Invidious doesn't support multiple audio tracks in their API response or DASH manifest, they are also missing the important fields width, height for all video streams and fps and qualityLabel for the AV1 formats in their API responses.

This pull request adds multiple audio track support for Invidious, without requiring any changes on their side, a side effect of that ,is that we can also support AV1 formats properly for Invidious.

This pull request takes advantage of YouTube.js exposing its DASH manifest generator and its ability to parse language and role information from the streaming URLs. Unfortunately due to the missing fields in the API responses (mentioned in the first paragraph), this pull request still makes a request to Invidious' manifest endpoint, so that it can extract the height, width and fps from it, for the locally generated manifest. As video.js was already making that request before, this doesn't cause any extra load on Invidious instances.

YouTube.js usually deciphers the formats as part of the manifest generation process, as that requires sending a request to YouTube to check if there is a new player and a second one to fetch it if there is (would use the one from the cache if it's the same one), I decided to make the deciphering a no-op while generating the Invidious manifest (it will still decipher for the local API one).
While we could use the deciphering to unthrottle age-restricted videos on Invidious (Invidious doesn't do its own deciphering and the bypass doesn't use the android endpoint, with the magic params that bypass throttling), I decided against it, to avoid making the extra requests. If unthrottling those is still desirable, I could see if there is a way to detect age restricted videos on Invidious and implement deciphering just for those ones (only if backend fallback is enabled of course) in a follow-up pull request.

Testing

dubbed and descriptive audio tracks: https://youtu.be/Kn56bMZ9OE8 (dubbed at 1:35, descriptive at 1:40)
audio tracks in multiple languages: https://youtu.be/jObOjhUkf50
AV1 formats: https://youtu.be/njX2bu-_Vw4

Also test proxying if you can find an Invidious instance it works on.

Desktop

OS: Windows
OS Version: 10
FreeTube version: 0.19.0

efb4f5ff-1298-471a-8973-3d47447115dc

I noticed that the test videos didnt start in the original track by default. The Local API always starts with the original track.

Is this intended behavior?

absidue · 2023-08-28T19:46:07Z

It should start with the default one, was messing around a lot in FreeTube and YouTube.js while making it, so maybe I broke something along the way or maybe it's dependant on one of the YouTube.js changes I made while making this.

I'll take a look and get back to you.

…l API DASH manifest generator

absidue · 2023-08-29T20:49:51Z

@efb4f5ff-1298-471a-8973-3d47447115dc Should work now, created a pull request to YouTube.js with the bug fix and upgraded to the version with it now.

github-actions · 2023-09-05T05:26:37Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

github-actions · 2023-09-09T21:02:28Z

Conflicts have been resolved. A maintainer will review the pull request shortly.

absidue · 2023-09-09T21:03:41Z

Added language names with the Intl.DisplayNames API, so now the labels match the local API ones. This means they should now look and feel the same.

ChunkyProgrammer · 2023-09-09T22:02:43Z

src/renderer/views/Watch/Watch.js

+
+        if (hasMultipleAudioTracks) {
+          // match YouTube's local API response with English
+          const languageNames = new Intl.DisplayNames('en-US', { type: 'language' })


Should we use the user's current locale instead of using "en-US"?

My goal with this pull request was to match the local API, so that you would have the same, identical user experience with both the local and the Invidious API (maybe I should have made the comment on the line above more clear).

If you want me to switch to using the users locale, I can do that, it'll just be a different user experience than with the local API or the Invidious API when it gets multiple audio track support (ideally we don't want to keep this hack around forever).

efb4f5ff-1298-471a-8973-3d47447115dc · 2023-09-10T15:44:44Z

@efb4f5ff-1298-471a-8973-3d47447115dc Should work now, created a pull request to YouTube.js with the bug fix and upgraded to the version with it now.

So tested both videos again and both didnt start with the original audio

absidue · 2023-09-12T17:43:10Z

@efb4f5ff-1298-471a-8973-3d47447115dc Merged the development branch in again, hopefully that'll help, also make sure you have ran yarn install so that you actually have a youtube.js version installed that works.

src/renderer/helpers/api/invidious.js

Co-authored-by: ChunkyProgrammer <78101139+ChunkyProgrammer@users.noreply.github.com>

absidue · 2023-09-21T21:38:59Z

Fixed

efb4f5ff-1298-471a-8973-3d47447115dc

LGTM

ChunkyProgrammer

LGTM, looked at XHR/Fetch in network tab and no requests to YouTube are made when proxying through invidious is enabled. We should probably be translating the languages to the user's locale but as local api doesn't currently do that, this can come later

* development: (41 commits) Translated using Weblate (Finnish) Mobile/tablet channel page & share button visual improvements (FreeTubeApp#4061) Support multiple audio tracks and AV1 for Invidious by using the local API DASH manifest generator (FreeTubeApp#3942) Normalize ft-list-item grid height (FreeTubeApp#4043) Bump the stylelint group with 1 update (FreeTubeApp#4071) Bump the babel group with 1 update (FreeTubeApp#4069) Bump the eslint group with 1 update (FreeTubeApp#4070) Bump sass from 1.67.0 to 1.68.0 (FreeTubeApp#4072) Bump marked from 9.0.2 to 9.0.3 (FreeTubeApp#4073) Bump lefthook from 1.4.11 to 1.5.0 (FreeTubeApp#4074) Translated using Weblate (Chinese (Traditional)) Translated using Weblate (Greek) Translated using Weblate (German) Translated using Weblate (Portuguese (Brazil)) Translated using Weblate (French) Translated using Weblate (Icelandic) Translated using Weblate (Portuguese) Translated using Weblate (Polish) ! Fix in sub view, restoring last tab is bugged when visible tabs do not include tab to be restored (FreeTubeApp#4064) Translated using Weblate (Ukrainian) ... # Conflicts: # src/renderer/components/ft-prompt/ft-prompt.css

* feature/playlist-2023-05: (176 commits) ! Fix playlist item fetching for local API (FreeTubeApp#4102) ! Fix watch page video published time parsing (FreeTubeApp#4105) Translated using Weblate (Serbian) Bump youtubei.js from 6.4.0 to 6.4.1 (FreeTubeApp#4090) Bump rimraf from 5.0.1 to 5.0.5 (FreeTubeApp#4091) Bump the stylelint group with 2 updates (FreeTubeApp#4088) Bump electron from 22.3.24 to 22.3.25 (FreeTubeApp#4089) Translated using Weblate (Bulgarian) Update FT history import to accept key lastViewedPlaylistId (FreeTubeApp#4038) ! Fix ft-video having different URLs in links Translated using Weblate (Dutch) Translated using Weblate (Lithuanian) Translated using Weblate (Japanese) Translated using Weblate (Croatian) Translated using Weblate (Estonian) Translated using Weblate (Croatian) ! Fix external player handling for video in local playlist Translated using Weblate (Finnish) Mobile/tablet channel page & share button visual improvements (FreeTubeApp#4061) Support multiple audio tracks and AV1 for Invidious by using the local API DASH manifest generator (FreeTubeApp#3942) ...

absidue requested review from PikachuEXE, efb4f5ff-1298-471a-8973-3d47447115dc, ChunkyProgrammer and MarmadileManteater August 25, 2023 17:49

FreeTubeBot enabled auto-merge (squash) August 25, 2023 17:49

github-actions bot added the PR: waiting for review For PRs that are complete, tested, and ready for review label Aug 25, 2023

PikachuEXE previously approved these changes Aug 27, 2023

View reviewed changes

efb4f5ff-1298-471a-8973-3d47447115dc reviewed Aug 28, 2023

View reviewed changes

absidue added 2 commits August 29, 2023 22:46

Support multiple audio tracks and AV1 for Invidious by using the loca…

6e07b2a

…l API DASH manifest generator

Upgrade YouTube.js to 6.2.0 to fix default track selection

9ae58c2

absidue dismissed PikachuEXE’s stale review via 9ae58c2 August 29, 2023 20:48

absidue force-pushed the iv-audio-track-av1 branch from 8c7368d to 9ae58c2 Compare August 29, 2023 20:48

absidue requested a review from kommunarr August 29, 2023 20:49

ChunkyProgrammer requested review from PikachuEXE and efb4f5ff-1298-471a-8973-3d47447115dc September 1, 2023 13:18

PikachuEXE previously approved these changes Sep 2, 2023

View reviewed changes

github-actions bot added PR: merge conflicts / rebase needed and removed PR: waiting for review For PRs that are complete, tested, and ready for review labels Sep 5, 2023

absidue added 3 commits September 9, 2023 18:37

Merge branch 'development' into iv-audio-track-av1

dfad523

Fix audio formats

03b6183

Use Intl.DisplayNames to get the language names

5fd1059

absidue dismissed PikachuEXE’s stale review via 5fd1059 September 9, 2023 21:02

github-actions bot removed the PR: merge conflicts / rebase needed label Sep 9, 2023

absidue added the PR: waiting for review For PRs that are complete, tested, and ready for review label Sep 9, 2023

ChunkyProgrammer reviewed Sep 9, 2023

View reviewed changes

Merge branch 'development' into iv-audio-track-av1

be92aa3

ChunkyProgrammer reviewed Sep 19, 2023

View reviewed changes

src/renderer/helpers/api/invidious.js Outdated Show resolved Hide resolved

src/renderer/helpers/api/invidious.js Outdated Show resolved Hide resolved

Simplify returns

f2f3b9c

Co-authored-by: ChunkyProgrammer <78101139+ChunkyProgrammer@users.noreply.github.com>

PikachuEXE approved these changes Sep 21, 2023

View reviewed changes

efb4f5ff-1298-471a-8973-3d47447115dc approved these changes Sep 25, 2023

View reviewed changes

efb4f5ff-1298-471a-8973-3d47447115dc requested a review from ChunkyProgrammer September 25, 2023 14:15

ChunkyProgrammer approved these changes Sep 25, 2023

View reviewed changes

FreeTubeBot merged commit d91f82f into FreeTubeApp:development Sep 25, 2023
5 checks passed

github-actions bot removed the PR: waiting for review For PRs that are complete, tested, and ready for review label Sep 25, 2023

absidue deleted the iv-audio-track-av1 branch September 25, 2023 14:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support multiple audio tracks and AV1 for Invidious by using the local API DASH manifest generator #3942

Support multiple audio tracks and AV1 for Invidious by using the local API DASH manifest generator #3942

absidue commented Aug 25, 2023 •

edited

Loading

efb4f5ff-1298-471a-8973-3d47447115dc left a comment

absidue commented Aug 28, 2023

absidue commented Aug 29, 2023

github-actions bot commented Sep 5, 2023

github-actions bot commented Sep 9, 2023

absidue commented Sep 9, 2023

ChunkyProgrammer Sep 9, 2023

absidue Sep 9, 2023 •

edited

Loading

efb4f5ff-1298-471a-8973-3d47447115dc commented Sep 10, 2023

absidue commented Sep 12, 2023

absidue commented Sep 21, 2023

efb4f5ff-1298-471a-8973-3d47447115dc left a comment

ChunkyProgrammer left a comment

Support multiple audio tracks and AV1 for Invidious by using the local API DASH manifest generator #3942

Support multiple audio tracks and AV1 for Invidious by using the local API DASH manifest generator #3942

Conversation

absidue commented Aug 25, 2023 • edited Loading

Support multiple audio tracks and AV1 for Invidious by using the local API DASH manifest generator

Pull Request Type

Related issue

Description

Testing

Desktop

efb4f5ff-1298-471a-8973-3d47447115dc left a comment

Choose a reason for hiding this comment

absidue commented Aug 28, 2023

absidue commented Aug 29, 2023

github-actions bot commented Sep 5, 2023

github-actions bot commented Sep 9, 2023

absidue commented Sep 9, 2023

ChunkyProgrammer Sep 9, 2023

Choose a reason for hiding this comment

absidue Sep 9, 2023 • edited Loading

Choose a reason for hiding this comment

efb4f5ff-1298-471a-8973-3d47447115dc commented Sep 10, 2023

absidue commented Sep 12, 2023

absidue commented Sep 21, 2023

efb4f5ff-1298-471a-8973-3d47447115dc left a comment

Choose a reason for hiding this comment

ChunkyProgrammer left a comment

Choose a reason for hiding this comment

absidue commented Aug 25, 2023 •

edited

Loading

absidue Sep 9, 2023 •

edited

Loading