[YouTube] Update clients version and improve extraction of API key and client version of WEB client #890
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
AudricV
added
bug
|
Reviews on the code changes can be made, even if this PR is marked as a draft. |
2 tasks
AudricV
changed the title
Common code in WEB client version HTML extraction has been deduplicated, usage of the Java 8 Stream API has been made and initial data fallback has been used as a last resort. This means that the client version extraction from regexes will be used before this fallback, as it doesn't contain the full client version. This can be used as a way to fingerprint the extractor, even if it seems to be not the case.
…lient The iOS version can be got easily in fact, by looking at the What's New section of the App Store' app page.
Additional parameters have been added to the player requests of ANDROID and IOS clients: - for both clients: osName and osVersion: their respective values are: - for the ANDROID one: Android and 12; - for the IOS one: iOS and 15.6.0.19G71. - for the ANDROID client: androidTargetSdkVersion, with the Android SDK version corresponding to the Android version used in the player requests of this client. This parameter is now required with this client to be sure to get a correct player response, otherwise, the one of a video saying that this content is not available in this app and to watch it with the latest version of YouTube can be returned instead; - for the IOS client: deviceMake, with Apple as its value. The iOS version sent in the IOS client player requests has been also updated to the version 15.6 of the OS. Finally, a comment about the requirement to use the signature timestamp from the player JavaScript base file for HTML5 player requests on videos with obfuscated URLs has been added and replaces a previous one which may be not true.
If YouTube detect that requests come from a third party client, they may replace the real player response by another one of a video saying that this content is not available on this app and to watch it on the latest version of YouTube. We can detect this by checking whether the video ID of the player response returned is the same as the one requested by the extractor.
AudricV
force-pushed
the
yt-clients-update-improvements-and-extraction-fixes
branch
from
16b59af
to
472f5d9
Compare
AudricV
deleted the
yt-clients-update-improvements-and-extraction-fixes
branch
This comment was marked as spam.
This comment was marked as spam.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes the replacement of the player response of a requested video by a one saying that
This content is not available on this app, by adding the new required parameter,androidSdkVersion, for which the value sent is the one of the Android version we use on requests of theANDROIDclient.Detection of a player response redirection has been added on
IOSandANDROIDclients to avoid this behavior in the future.Client versions have been updated to their latest version and more parameters have been added on requests of the clients quoted above.
Extraction of InnerTube API key and client version of the
WEBclient from HTML page has been improvedand requests to its JavaScript service worker and the one of the(seems to be not the case everywhere, so these changes have been reverted).WEB_REMIXclient have been removed, as they do not contain this information anymoreExtractions will now only use HTML pages.Fixes TeamNewPipe/NewPipe#8713.