Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[YouTube] All Age-restricted videos unavailable #8102

Closed
5 tasks done
WesleyTRV opened this issue Mar 25, 2022 · 29 comments · Fixed by TeamNewPipe/NewPipeExtractor#780
Closed
5 tasks done

[YouTube] All Age-restricted videos unavailable #8102

WesleyTRV opened this issue Mar 25, 2022 · 29 comments · Fixed by TeamNewPipe/NewPipeExtractor#780
Labels
bug Issue is related to a bug requires extractor change This issue requires a change to the extractor youtube Service, https://www.youtube.com/

Comments

@WesleyTRV
Copy link

WesleyTRV commented Mar 25, 2022

Checklist

  • I am able to reproduce the bug with the latest version.
  • I made sure that there are no existing issues - open or closed - which I could contribute my information to.
  • I have taken the time to fill in all the required details. I understand that the bug report will be dismissed otherwise.
  • This issue contains only one bug.
  • I have read and understood the contribution guidelines.

Affected version

0.22.1

Steps to reproduce the bug

  1. Enable age-restricted videos in settings.
  2. Look up for an video which is age-restricted.
  3. This message show up.

Expected behavior

Age restricted video will normally load up and proceed.

Actual behavior

This message shows up and it's annoying.

Screenshots/Screen recordings

Screenshot_20220325-035411_NewPipe

Logs

Exception

Crash log

org.schabi.newpipe.extractor.exceptions.AgeRestrictedContentException: This age-restricted video cannot be watched.
	at org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.checkPlayabilityStatus(YoutubeStreamExtractor.java:760)
	at org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.onFetchPage(YoutubeStreamExtractor.java:715)
	at org.schabi.newpipe.extractor.Extractor.fetchPage(Extractor.java:54)
	at org.schabi.newpipe.extractor.stream.StreamInfo.getInfo(StreamInfo.java:68)
	at org.schabi.newpipe.extractor.stream.StreamInfo.getInfo(StreamInfo.java:64)
	at org.schabi.newpipe.util.ExtractorHelper.lambda$getStreamInfo$3(ExtractorHelper.java:116)
	at org.schabi.newpipe.util.ExtractorHelper.$r8$lambda$YTHJjScxCJNO1LTCqs3IKy35iyY(Unknown Source:0)
	at org.schabi.newpipe.util.ExtractorHelper$$ExternalSyntheticLambda6.call(Unknown Source:4)
	at io.reactivex.rxjava3.internal.operators.single.SingleFromCallable.subscribeActual(SingleFromCallable.java:43)
	at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4813)
	at io.reactivex.rxjava3.internal.operators.single.SingleDoOnSuccess.subscribeActual(SingleDoOnSuccess.java:35)
	at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4813)
	at io.reactivex.rxjava3.internal.operators.maybe.MaybeFromSingle.subscribeActual(MaybeFromSingle.java:41)
	at io.reactivex.rxjava3.core.Maybe.subscribe(Maybe.java:5330)
	at io.reactivex.rxjava3.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.drain(MaybeConcatArray.java:153)
	at io.reactivex.rxjava3.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.request(MaybeConcatArray.java:78)
	at io.reactivex.rxjava3.internal.operators.flowable.FlowableElementAtMaybe$ElementAtSubscriber.onSubscribe(FlowableElementAtMaybe.java:66)
	at io.reactivex.rxjava3.internal.operators.maybe.MaybeConcatArray.subscribeActual(MaybeConcatArray.java:42)
	at io.reactivex.rxjava3.core.Flowable.subscribe(Flowable.java:15868)
	at io.reactivex.rxjava3.internal.operators.flowable.FlowableElementAtMaybe.subscribeActual(FlowableElementAtMaybe.java:36)
	at io.reactivex.rxjava3.core.Maybe.subscribe(Maybe.java:5330)
	at io.reactivex.rxjava3.internal.operators.maybe.MaybeToSingle.subscribeActual(MaybeToSingle.java:46)
	at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4813)
	at io.reactivex.rxjava3.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89)
	at io.reactivex.rxjava3.core.Scheduler$DisposeTask.run(Scheduler.java:644)
	at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:65)
	at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:56)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636)
	at java.lang.Thread.run(Thread.java:764)


Affected Android/Custom ROM version

Android 8

Affected device model

Galaxy S7

Additional information

This bug is in requirement of an immediate fix/patch ASAP.

@WesleyTRV WesleyTRV added the bug Issue is related to a bug label Mar 25, 2022
@Amberarch
Copy link

I would like to add that you can still play the audio in the background, and the videos will sometimes be watchable after doing this.

@WesleyTRV

This comment was marked as spam.

@boognish-rising
Copy link

Not sure if it's terribly relevant but I see that Android 8 is the cited Android OS version for this particular issue. I didn't look for additional issues citing the same problem with age-restricted videos suddenly not playing anymore, as I was really only trying to determine if there was already an open issue for it, which this satisfied. The only noteworthy difference is Android version. The issue is also reproducible using Android 12

@Blue166
Copy link

Blue166 commented Mar 25, 2022

I'm having the same issue it's part of why I downloaded this app I would really like to see this fixed please 😕😊

@WesleyTRV WesleyTRV changed the title Age-restricted videos refused to load / Need patch All Age-restricted videos unavailable / Need patch Mar 25, 2022
@WesleyTRV WesleyTRV changed the title All Age-restricted videos unavailable / Need patch [YouTube] All Age-restricted videos unavailable / Need patch Mar 25, 2022
@WesleyTRV

This comment was marked as spam.

@SameenAhnaf SameenAhnaf mentioned this issue Mar 25, 2022
5 tasks
@litetex litetex changed the title [YouTube] All Age-restricted videos unavailable / Need patch [YouTube] All Age-restricted videos unavailable Mar 25, 2022
@litetex
Copy link
Member

litetex commented Mar 25, 2022

Related: TeamNewPipe/NewPipeExtractor#729

@coletdjnz
Copy link

Also related: yt-dlp/yt-dlp#3182

@litetex litetex added the youtube Service, https://www.youtube.com/ label Mar 25, 2022
@WesleyTRV

This comment was marked as spam.

@Blue166

This comment was marked as spam.

@RiedleroD

This comment was marked as resolved.

@videomember

This comment was marked as spam.

@Miteirao

This comment was marked as spam.

@AudricV
Copy link
Member

AudricV commented Mar 28, 2022

For everyone: please stop commenting on the issue unless you have something useful to add.

We know the extraction of YouTube age-restricted videos is broken, so it happens on all devices on every Android version, due to a YouTube change which patched the way we are able to get them.

It seems there is no way to get anonymously streams of age-restricted videos anymore, at least for the same amount.

@zerodytrash
Copy link

I've created a bruteforce script that tries thousands of possible combinations between innertube clients and versions. TVHTML5_SIMPLY_EMBEDDED_PLAYER with version 2.0 has currently no age restrictions. So we can access age-restricted videos anonymously again (at least for now).

POST /youtubei/v1/player?key=AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8 HTTP/2
Host: www.youtube.com

{
    "context": {
        "client": {
            "clientName": "TVHTML5_SIMPLY_EMBEDDED_PLAYER",
            "clientVersion": "2.0"
        },
        "thirdParty": {
            "embedUrl": "https://www.youtube.com/",
        }
    },
    "videoId": "dqRZDebPIGs",
    "racyCheckOk": true,
    "contentCheckOk": true
}

@Blue166

This comment was marked as off-topic.

@ThisIsPaulDaily

This comment was marked as off-topic.

@Daverpe
Copy link

Daverpe commented Mar 29, 2022

I've created a bruteforce script

How to add this script to Newpipe?

Mod note: don't paste a giant wall of text. Only quote the part needed for context for your reply.

@ilwsm

This comment was marked as spam.

@rhjdvsgsgks
Copy link

rhjdvsgsgks commented Apr 1, 2022

I've created a bruteforce script

Mod note: read #8102 (comment)
edit: i will take care about it, thanks

i wrote a simple frida snippet to test that, and it works

Java.perform(function () {
    var JsonBuilder = Java.use("com.grack.nanojson.JsonBuilder");
    JsonBuilder.value.overload("java.lang.String", "java.lang.String").implementation = function (key, s) {
	if (key == "clientName" && s == "ANDROID") {
        	s = "TVHTML5_SIMPLY_EMBEDDED_PLAYER"
	} else if (key == "clientVersion" && s == "16.29.38") {
        	s = "2.0"
	}
	var ret_value = this.value(key, s);
        return ret_value;
    }
});

https://github.com/TeamNewPipe/NewPipeExtractor/blob/5a1873084526a8f67a9924c32457eb4a0a3101c4/extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/YoutubeParsingHelper.java#L1043-L1044
https://github.com/mmastrac/nanojson/blob/959842ccc39256f545a96581e78071a27cb83558/src/main/java/com/grack/nanojson/JsonBuilder.java#L122
(these two lines been hooked)
but pretty slow, only 50-80kb with 1 connection. it means that not very useful to watch video, unless watch only low resolution video or download (multi thread) video before watch it.

@iiSnewoNL
Copy link

but pretty slow, only 50-80kb with 1 connection. it means that not very useful to watch video, unless watch only low resolution video or download (multi thread) video before watch it.

So I guess we are stuck with having no access to watch age-restricted videos? Daim Europe, thanks 😕

@AudricV AudricV self-assigned this Apr 2, 2022
@AudricV
Copy link
Member

AudricV commented Apr 2, 2022

Please test this APK: app-debug.zip

@boognish-rising
Copy link

boognish-rising commented Apr 2, 2022

Please test this APK: app-debug.zip

IT WORKS! I obviously haven't tested it extensively yet, as you only posted it 5 minutes ago but I have confirmed that it works to bypass YT's age restriction script. It's a little slow but it appears that when you configure the app to use an external video player things are sped up a bit.

THANK YOU! You and anyone else that contributed are the dude(s)

I'll continue testing and report back...

Not sure if it's at all helpful but see the hidden comment below for a heap analysis (7 leaks)

@boognish-rising

This comment was marked as duplicate.

@opusforlife2
Copy link
Collaborator

Works. 👍 Now comedians can swear again.

@oddtoddy
Copy link

oddtoddy commented Apr 2, 2022

Can also confirm this is working now.

@jok3rsgr1n
Copy link

Works !

@WesleyTRV
Copy link
Author

Please test this APK: app-debug.zip

Thought it was gonna be a virus or scam but it works! Thanks!

@AudricV
Copy link
Member

AudricV commented Apr 3, 2022

No, it's not at all, I compiled it myself from sources I published that you can find here:

https://github.com/TiA4f8R/NewPipe/tree/yt-more-params-innertube-requests
https://github.com/TiA4f8R/NewPipeExtractor/tree/yt-more-params-innertube-requests (corresponding PR: TeamNewPipe/NewPipeExtractor#780)

Only extractor changes are needed to fix the extraction (the app changes only change the extractor used).

@Minibus93

This comment was marked as spam.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Issue is related to a bug requires extractor change This issue requires a change to the extractor youtube Service, https://www.youtube.com/
Projects
None yet