[Bug] "This helps protect our community." - "Please sign in to confirm you’re not a bot"

[ggtylerr]

Update 21/09/2024: #4734 (comment)

---

EDIT by @unixfox: The Invidious team is aware of this issue. It appears that it affects all the software using YouTube.
Please refrain from commenting if you have nothing new to bring up. Thank you.

---

Describe the bug
This seems to be another block by YouTube - happens on any video, regardless of settings or instance.

Steps to Reproduce

1. Go to any video on Invidious

Logs
No logs on browser. In docker logs:

invidious-invidious-refresh-1  | 2024-06-06 22:35:23 UTC [error] get_video: PrMRuA3pd7g : This helps protect our community. Learn more
invidious-invidious-refresh-1  | 2024-06-06 22:35:23 UTC [warn] i18n: Missing translation key "This helps protect our community. Learn more"

Screenshots

Additional context
This seems to be a global update, done before 22:23 UTC. (10:23 PM.) From my brief testing, this is present throughout all instances, and regardless of IPv6 address.

EDIT: It looks like this is just rolling out throughout all servers, so some may not be affected yet.

[ggtylerr]

Okay, it's clear this isn't global yet and it's a standard YouTube rollout. Please refrain from posting "i'm affected/not affected" comments :)

[Sommerwiesel]

They are probably A/B testing this, I just (programmatically) did 200 requests on my instance and roughly 1/3 of them returned this bug, the other times, the video loaded just fine.
This will most likely get worse the next days, so brace yourself for another downtime of all instances.

[ggtylerr]

Update: Cobalt seems to have fixed this by implementing a token system: imputnet/cobalt#551

I don't recommend this since it'd be easy to overload (ex: nitter) but it is an option, would be good for temporary use.

[m0istn00dl]

This isn't just happening on invidious instances and third party apps, I've been getting this message trying to watch videos directly off of YTs website from the DDG app on my phone.

[ggtylerr]

Oh or maybe @m0istn00dl got their IP banned previously by using Invidious or something so youtube.com won't work until they get unbanned.

Nope. It's pretty much confirmed at this point that basically every client is being blocked and requiring a sign in. Including YouTube's own clients. The only exception found thus far seems to be the TV client (although I could be wrong on this.)

Why exactly this is the case is something only Google can answer - whether it's meant to directly hinder frontends / third-party clients, to prevent unknown (i.e. hard to track) users that don't sign in, or if it's a test to see how much the userbase would be affected by the change. No real way to find out for sure until Google makes some other moves.

(Oh, and also, Google has no way of telling whether an IP visited Invidious, unless you accessed it from their search engine or something. That IP ban claim is ridiculous.)

[unixfox]

@that404nerd that is not a chat discussion. I said to refrain from writing some comments if you have nothing new to bring up.

If you or anyone else want to discuss freely about the issue then join our matrix (https://matrix.to/#/#invidious:matrix.org) or IRC (https://web.libera.chat/?channel=#invidious)

[Raposa-Coltran]

this feels like it needs to be something invidious displays properly, this error should reroute to a "youtube is blocking instances" message or something, especially when the "learn more" just is dead text, atleast untill the problem is fixed properly (tho i imagine it would be good practice to reroute almost any kind of novel error type to such an error in future too)

[unixfox]

this feels like it needs to be something invidious displays properly, this error should reroute to a "youtube is blocking instances" message or something, especially when the "learn more" just is dead text, atleast untill the problem is fixed properly (tho i imagine it would be good practice to reroute almost any kind of novel error type to such an error in future too)

when it's finally implemented, there will be a message explaining what to do in order to solve the issue.

[Raposa-Coltran]

this feels like it needs to be something invidious displays properly, this error should reroute to a "youtube is blocking instances" message or something, especially when the "learn more" just is dead text, atleast untill the problem is fixed properly (tho i imagine it would be good practice to reroute almost any kind of novel error type to such an error in future too)

when it's finally implemented, there will be a message explaining what to do in order to solve the issue.

it feels to me like errors should be updated when the problem is known, not just when theyre fixed, as this leaves users rather confused, before looking it up i assumed my instance had blocked the video itself given the error message, good to know it will be fixed tho

[ggtylerr]

Especially since (when will this be finally fixed?) this is going on for quite an uncomfortable while and who knows how many people we lost due to the error itself and because they couldn't make heads and tails of the error message and just deemed invidious too buggy, given that not everybody is tech-savvy.

I know you're not specifically referring to how long the fix is coming, but I personally know a lot of people are (talking about you, ppl commenting on my site asking when invidious will be fixed >:c ) The patch I was referring to before is fairly complicated, involving multiple stages, decryption, etc. If you know Crystal (which unfortunately I don't), you're more than welcome to help.

[accessiblepixel]

The 'Learn more' text is a link directly from the YouTube error (which links to YouTube's help pages), it's not one that invidious has added. The difference is that some of it gets trimmed and the link gets made non-clickable.... but I understand that it's a little confusing.

If I remember correctly the full text is "Sign in to prove you're not a bot. This helps protect our community. [Learn More]", so I echo @Raposa-Coltran that it should better reflect what's happening and even if there's no solution maybe there's a better way to handle this and any future issues, than just waiting until there's a fix?

I am looking forward to what Invidious's solution to the issue is (much love to the devs working on this, it's much appreciated what you do <3), although I personally found that using different IP endpoints got my local instance back to being useable again (through a public ipv4 VPN endpoint) as well as yt-dlp, so I'd imagine the IPv6 rotator would also help with this too (for those that have it on a VPS/hosting - although I don't host mine publicly or use ipv6).

[unixfox]

Everyone in the Invidious team does not have unlimited free time, we work at our own pace. Like @ggtylerr explained, it's much more complicated to solve than any previous YouTube breakage. Hence, why it takes a lot of long time to get a fix.

It's work in progress. For the more curious, here are the PR for fixing the issue: #4772 and #4789

We have no plan to alter the error message until a solution has been implemented. We know we are probably loosing many users, but it's part of the life of an open source project. Usually an open source project is run by volunteers and users should not expect the same reliability as a product run by a company where many developers are working on the product every day.

[mas1701]

@unixfox
As my instance is also affected after running very smoothly for quite a long time, I want to express my thanks for all the work you and other devs have been putting into Invidious and are still continuing to do so, greatly appreciated! Just take the time it needs. No pressure.

[unixfox]

I'm locking again this issue because some people can't behave sorry.

There will be an announcement when a fix is deployed. It's work in progress, we have solutions, Invidious is not halted.

[unixfox]

Hello,

We have good news to offer.

An official solution has been released. But it's not yet available as a new version/release because we need some testers.

I have personally updated the installation guide to reflect on these two new tools for fixing the error message "This helps protect our community.".

Please consult this pull request to be part of the test phase: iv-org/documentation#581

I won't unlock this GitHub issue in order to have everyone be aware of this announcement.

---

What's new in a few details for fixing the error message `"This helps protect our community."``:

• You will have to run a separate tool called inv_sig_helper which handle the "deciphering" of the video stream fetched from YouTube servers. Thanks @techmetx11 for the development of this tool.
• You will have to generate two identity tokens (po_token and visitor_data) in order to pass all the checks on YouTube side. This will increase the traceability of your Invidious session from YouTube side, but we are going to develop a tool to regenerate those tokens periodically in order to avoid YouTube using them to track you.
  For the moment, we are only providing a tool that generate them once, it's called youtube-trusted-session-generator. You have been warned of the current risk.

[unixfox]

A new fix has been delivered in the master docker image: #4928

Which fixes the latest issue related to This helps protect our community.. Please try if it fixes the issue for you.

Reminder:

image: quay.io/invidious/invidious:master

[unixfox]

Hello,

Sad news for everyone. YouTube/Google has patched the latest workaround that we had in order to restore the video playback functionality.

Right now we have no other solutions/fixes. You may be able to get Invidious working on residential IP addresses (like at home) but on datacenter IP addresses Invidious won't work anymore. (Some datacenter IPs may still work, but that's a matter of time until they don't anymore.)

If you are interested to install Invidious at home, we remind you that we have a guide for that here: https://docs.invidious.io/installation/.

This is not the death of this project. We will still try to find new solutions, but this might take time, months probably.

I have updated the public instance list in order to reflect on the working public instances: https://instances.invidious.io. Please don't abuse them since the number is really low.

Feel free to discuss this politely on Matrix or IRC.

[unixfox]

Hello,

Small update but an important one. We have created a new stable release of Invidious which should improve the support on residential IP addresses: https://github.com/iv-org/invidious/releases/tag/v2.20241110.0

And there is now a page that explains what is this "Please sign in" error and what solutions you may try: https://docs.invidious.io/youtube-errors-explained/
Not guarantee to fully work.

We are still exploring new paths to make Invidious work again on datacenter IP addresses and thus for public instances. And we are also going to add the feature request from #4977

We will keep updated here.