[Bug] "Please sign in" from residential IP, youtube.com works

[markozajc]

Describe the bug
The following is shown after attempting to visit any /watch url:

I am not using a proxy and visiting youtube.com lets me play the video without logging in. yt-dlp 2024.09.27 and mpv also work. This appears to be the same issue as #4972, but I don't believe it's the same as #4734.

Steps to Reproduce

1. Open invidious
2. Try playing any video, for example /watch?v=aqz-KE-bpKQ

Logs

2024-10-03 10:37:46 UTC [trace] SigHelper: Send transaction 0x6b9d453e / opcode GET_SIGNATURE_TIMESTAMP
2024-10-03 10:37:46 UTC [trace] SigHelper: Send transaction 0x6b9d453e - Done
2024-10-03 10:37:46 UTC [trace] SigHelper: Recv transaction 0x6b9d453e / length 8
2024-10-03 10:37:46 UTC [trace] SigHelper: payload = Bytes[0, 0, 0, 0, 0, 0, 78, 29]
2024-10-03 10:37:46 UTC [trace] SigHelper: Recv transaction 0x6b9d453e - Done
2024-10-03 10:37:46 UTC [trace] SigHelper: Transaction unqueued and data sent to channel
2024-10-03 10:37:46 UTC [debug] YoutubeAPI: Using endpoint: "/youtubei/v1/player"
2024-10-03 10:37:46 UTC [trace] YoutubeAPI: ClientConfig: YoutubeAPI::ClientConfig(@client_type=YoutubeAPI::ClientType::WebCreator, @region="US")
2024-10-03 10:37:46 UTC [trace] YoutubeAPI: POST data: {"contentCheckOk" => true, "videoId" => "aqz-KE-bpKQ", "context" => {"client" => {"hl" => "en", "gl" => "US", "clientName" => "WEB_CREATOR", "clientVersion" => "1.20240918.03.00", "osName" => "Windows", "osVersion" => "10.0", "platform" => "DESKTOP", "visitorData" => "SNIP"}}, "racyCheckOk" => true, "user" => {"lockedSafetyMode" => false}, "playbackContext" => {"contentPlaybackContext" => {"html5Preference" => "HTML5_PREF_WANTS", "referer" => "https://www.youtube.com/watch?v=aqz-KE-bpKQ", "signatureTimestamp" => 19997}}, "serviceIntegrityDimensions" => {"poToken" => "SNIP"}, "params" => "2AMB"}
2024-10-03 10:37:46 UTC [error] get_video: aqz-KE-bpKQ : Please sign in
2024-10-03 10:37:46 UTC [warn] i18n: Missing translation key "Please sign in"
2024-10-03 10:37:46 UTC [info] 500 GET /watch?v=aqz-KE-bpKQ 146.11ms

Additional context

• Browser: Firefox
• OS: Debian trixie
• invidious: 53e8a5d
• invidious_sig_helper: iv-org/inv_sig_helper@5025e49

[svpi11]

Same here...

[msinfo32github]

Was encountering the same problem - ran the inv_sig_helper and generated a new visitor_data and po_token. This seems to have solved the issue. Running on a residential IP, with same setup as described by @markozajc

[gr3m1in]

I can confirm that regenerating visitor_data/po_token helps

[markozajc]

I just did the same, and the issue has gone away, but doing so initially didn't help. This sounds like A/B testing, so there's probably more to come. I'll close the issue for now.

[unixfox]

#4734

Please don't create duplicated issues thank you. If you have a constant error please logging, it's expected: #4734 (comment)

[markozajc]

@unixfox apologies if this has been researched already, but this does not seem like the same bug as #4734:

• The error message appears to be different: [Bug] "This helps protect our community." - "Please sign in to confirm you’re not a bot" #4734 uses "Sign in to confirm you’re not a bot", while this issue uses "Please sign in"
• As far as I understand, owners of instances affected by [Bug] "This helps protect our community." - "Please sign in to confirm you’re not a bot" #4734 are unable to use yt-dl(p) on their servers, as the ban applies to the IP address. This is not the case here - only invidious is unable to play videos, but yt-dlp and NewPipe work fine on the same IP address.

As I suspected, the issue has reappeared and I'm once again unable to fix it by regenerating po_token and visitor_data. If the problems are indeed separate, please reopen the issue.

If they are the same issue, it might be worth adding the "Please sign in" message and any other details to #4734 to prevent confusion.

[unixfox]

The issue lies with the fact that we have already done all the research possible with the possible solutions that Invidious can handle. yt-dlp is another tool, which means it may differ from how invidious works.

I said to not create duplicated issue because we already know about these issues, and for the moment we can't do anything more than what we have already done. Something that was explained in #4734 (comment)

Right now, there are many factors that contribute to the "please sign in" errors or to the fact that it works for some people. This makes it very difficult to diagnose the root cause of everyone.

That's why for the moment we have offering what generally works for most people with a residential IP address: https://docs.invidious.io/installation/. If it doesn't, I'm sorry but you can't use Invidious for the moment. I know this is sad but it's the current state of YouTube against 3rd party clients.

[retrocaster]

I know this is sad but it's the current state of YouTube against 3rd party clients.

@unixfox Can you explain why Android apps such as SmartTube continue to work without any issues? It's also a 3rd party client that is open source with ad skipping and Sponsorblock integration: https://github.com/yuliskov/SmartTube

[Caian]

Sadly I'm facing this issue as well at home and no token generation managed to fix it.

[dakabali]

Is there eventually a working docker image with the above patch? Thx

[markozajc]

used the parser.cr file from that fork and replaced the one in master

Make sure you've also replaced src/invidious/yt_backend/youtube_api.cr.

[bbsixzz]

Is there eventually a working docker image with the above patch? Thx

Yes please can someone provide this?

[retrocaster]

Anyone knows the reason this is not in master?

Perhaps @unixfox can answer that.

If it's just a temporary fix then maybe that's the answer. We've had a lot of temporary fixes, but they're ultimately kind of pointless if they quickly stop working.

Also I haven't seen an explanation for why this fixes anything. Given that the issues are inherently intermittent, a fix without an explanation doesn't inspire much confidence.

[unixfox]

We are busy on offering more permanent solutions. Things that get shared here may not work after a few weeks or so.

We prefer to tell the public that they can use whatever solution fits to them instead of saying that we fixed the software but 2 weeks after it's broken again.

Anyway, if people here finds a good solution that works for the moment please create a pull request! By good I mean something that doesn't revert a whole PR just "because it works".

[Caian]

We are busy on offering more permanent solutions. Things that get shared here may not work after a few weeks or so.

We prefer to tell the public that they can use whatever solution fits to them instead of saying that we fixed the software but 2 weeks after it's broken again.

Anyway, if people here finds a good solution that works for the moment please create a pull request! By good I mean something that doesn't revert a whole PR just "because it works".

Perhaps making the Web Creator API optional via environment variable, and making it enabled by default so it won't affect the majority of the users? Unfortunately this solution may be more robust, but clearly broke things for several people.

[unixfox]

We are busy on offering more permanent solutions. Things that get shared here may not work after a few weeks or so.

We prefer to tell the public that they can use whatever solution fits to them instead of saying that we fixed the software but 2 weeks after it's broken again.

Anyway, if people here finds a good solution that works for the moment please create a pull request! By good I mean something that doesn't revert a whole PR just "because it works".

Perhaps making the Web Creator API optional via environment variable, and making it enabled by default so it won't affect the majority of the users? Unfortunately this solution may be more robust, but clearly broke things for several people.

We prefer to avoid this scenario. Because this would make the diagnostic of someone else installation difficult.

What if they configured a client that no longer works, we would have to think about it in the diagnostic.

We prefer that everyone is on the same implementation. This way we can more easily understand why it doesn't work for someone for example.

[unixfox]

Locking this issue as we know that things broke, new code changes will be up soon. Be patient guys, rome wasn't built in a day.