Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Discussion] How to circumvent YouTube blockage - #3822 #4045 related #3915

Closed
unixfox opened this issue Jun 13, 2023 · 33 comments
Closed

[Discussion] How to circumvent YouTube blockage - #3822 #4045 related #3915

unixfox opened this issue Jun 13, 2023 · 33 comments
Labels
discussion type:youtube-change Google changed something (again)

Comments

@unixfox
Copy link
Member

unixfox commented Jun 13, 2023

You can discuss here the different ways to circumvent the blockage from YouTube.

Reminder, the official solution is in #3915 (comment)

@unixfox
Copy link
Member Author

unixfox commented Jun 13, 2023

Posting the comments from the previous issue.


Could it be useful to have a script that automatically changes the IPv6 address and hook it up to a cronjob?

Originally posted by @Ashirg-ch in #3822 (comment)


Could it be useful to have a script that automatically changes the IPv6 address and hook it up to a cronjob?

There is a script just for that :D: https://github.com/ycd/ipv6-rotator

But I did not test if changing the IPv6 address in the same range fixes the issue or not.

Originally posted by @unixfox in #3822 (comment)


This hit me hard. OVH only gives you a /128 block for the IPv6 space, so I could mitigate this for a while using that, but it would be pointless if I'm going to get IP blocked again on IPv6. 🖕 <---- that's for you Google

Originally posted by @trentwiles in #3822 (comment)


If there's some sort of way to set up invidious with a SOCKS5 proxy, I could probably rotate IPs on DigitalOcean/Vultr.

Originally posted by @trentwiles in #3822 (comment)


This hit me hard. OVH only gives you a /128 block for the IPv6 space, so I could mitigate this for a while using that, but it would be pointless if I'm going to get IP blocked again on IPv6. 🖕 <---- that's for you Google

You could try getting a proper IPv6 range from tunnel broker such as Hurricane Electric.

Originally posted by @ddvarpdd in #3822 (comment)


I've had success routing outbound invidious traffic over shared VPNs; blending traffic with 'legitimate' users and connecting from non-Hetzner/OVH/DO address space works well for now.

PostUp = ip route add default dev %i table 5280
PostUp = ip -6 route add default dev %i table 5280

PostUp = ip rule add from (wg v4 address) table 5280
PostUp = ip -6 rule add from (wg v6 address) table 5280
PostUp = ip -4 rule add uidrange 991-991 lookup 5280
PostUp = ip -6 rule add uidrange 991-991 lookup 5280

PostDown = ip rule del from (wg v4 address) lookup 5280
PostDown = ip -6 rule del from (wg v6 address) lookup 5280
PostDown = ip -4 rule del uidrange 991-991 lookup 5280
PostDown = ip -6 rule del uidrange 991-991 lookup 5280

Table = off

reCAPTCHA profiles addresses at the /64 so it probably won't be long until we see that on YT as well, if they aren't already.

Originally posted by @tempname1024 in #3822 (comment)


I'm not smart when it comes to networking, where do you add these lines? Also, how do you authenticate with WG? You can't just connect to the IP without a private key right?

Originally posted by @trentwiles in #3822 (comment)


I'm not smart when it comes to networking, where do you add these lines? Also, how do you authenticate with WG? You can't just connect to the IP without a private key right?

To keep wireguard from updating the system's default routing table we disable that functionality with Table = off, create our own table called 5280, add wireguard's addresses, and create a rule to lookup table 5280 for traffic which originates from the user running invidious, in my case the user ID is 991. These lines are added to the Interface section.

Originally posted by @tempname1024 in #3822 (comment)


If there's some sort of way to set up invidious with a SOCKS5 proxy, I could probably rotate IPs on DigitalOcean/Vultr.

Double that. You could also use a “residential IP” proxy service, or even one of the mobile proxy ones (which use mobile operators networks and usually change IPs every couple minutes for you). This might not work well with proxying video traffic though, and I'm afraid it also might be not too cheap nowadays.

Originally posted by @notpushkin in #3822 (comment)


If there's some sort of way to set up invidious with a SOCKS5 proxy, I could probably rotate IPs on DigitalOcean/Vultr.

Double that. You could also use a “residential IP” proxy service, or even one of the mobile proxy ones (which use mobile operators networks and usually change IPs every couple minutes for you). This might not work well with proxying video traffic though, and I'm afraid it also might be not too cheap nowadays.

Are you sure the video traffic has to be proxied at all?

Originally posted by @ccuser44 in #3822 (comment)

@unixfox unixfox added the type:youtube-change Google changed something (again) label Jun 13, 2023
@unixfox unixfox changed the title [Discussion] How to circumvent YouTube blockage [Discussion] How to circumvent YouTube blockage - #3822 related Jun 13, 2023
@notpushkin
Copy link
Contributor

@ccuser44 Only if you want to get resolutions greater than 720p – those are only available over DASH, and it only works with proxying: https://docs.invidious.io/geoblocking/#video-quality-and-dash

@unixfox
Copy link
Member Author

unixfox commented Jun 13, 2023

@ccuser44 Only if you want to get resolutions greater than 720p – those are only available over DASH, and it only works with proxying: docs.invidious.io/geoblocking/#video-quality-and-dash

You need proxy for "encrypted" videos to load like music videos or so on, even when not using DASH.

@zekefeu
Copy link

zekefeu commented Jun 15, 2023

If the GitHub repo has to go, we probably can make a Gitea instance as a Tor hidden service ? I'm willing to do some tests to see if and how it would work.

@unixfox
Copy link
Member Author

unixfox commented Jun 15, 2023

If the GitHub repo has to go, we probably can make a Gitea instance as a Tor hidden service ? I'm willing to do some tests to see if and how it would work.

That's half related to the original topic which is circumventing the current youtube blockage not a potential future one. Don't worry we are deploying our services if one day the repository is taken down on github.

@gi-yt
Copy link

gi-yt commented Jun 28, 2023

Welp google started ratelimiting IPv6 now (inv.bp.projectsegfau.lt). Surprisingly our main instance (invidious.projectsegfau.lt) is still running fine on v4 :P

@unixfox
Copy link
Member Author

unixfox commented Jun 28, 2023

Welp google started ratelimiting IPv6 now (inv.bp.projectsegfau.lt). Surprisingly our main instance (invidious.projectsegfau.lt) is still running fine on v4 :P

Nice can you try to change the IPv6 address in another address in the same IPv6 range and see if it unblocks the instance?

@tio-trom
Copy link

Well....our https://ytb.trom.tf is also affected. Switching between ipv4 and 6 does not help anymore. My ipv6 ends in /64 so I guess I could change the IP address too? How will this affect my server overall? I have 20+ services hosted on that server.

@Trit34
Copy link

Trit34 commented Jun 28, 2023

https://invidious.fdn.fr/ blocked today, too. 😤

@gi-yt
Copy link

gi-yt commented Jun 28, 2023

Changing IP made the instance unratelimited again :D

@unixfox
Copy link
Member Author

unixfox commented Jun 28, 2023

Well....our ytb.trom.tf is also affected. Switching between ipv4 and 6 does not help anymore. My ipv6 ends in /64 so I guess I could change the IP address too? How will this affect my server overall? I have 20+ services hosted on that server.

yes change your IPv6 address like stated in #3822 (comment)

if you have added your current IPv6 address in your domain record, then you will need to update it too.

@tio-trom
Copy link

Ok. But how long till Google will ban the new IPV6? Is this a solution / worth the effort?

@unixfox
Copy link
Member Author

unixfox commented Jun 28, 2023

Ok. But how long till Google will ban the new IPV6? Is this a solution / worth the effort?

It seems like they ban almost all the invidious instances every 2 weeks. It's not a permanent solution, but it's a temporary one that works for the moment.

I want to reiterate on the temporary, we, the community, are trying to find a solution that works for the long term. Even though it's always a cat and mouse game with Google, there is no permanent solution.

@notpushkin
Copy link
Contributor

notpushkin commented Jun 29, 2023

Anybody gave (SOCKS / http) proxy support any more thought? Should we start an issue for that?

(I'll gladly look into implementing it, although I have absolutely zero experience with Crystal so it may take a while.)

@llebout
Copy link

llebout commented Jun 29, 2023

Hello! In terms of bypassing IPv6 blocks, if you have a dedicated IPv4 you can register for a Free IPv6 Tunnel here: https://tunnelbroker.net/ - it's an Hurricane Electric service (big worldwide ISP)

Those can be as large as /48 however to avoid blocks maybe get a few smaller ones if possible (don't remember exactly how assignment works).

Additionally, you can rotate requests through a whole /48 block so that individual /64 blocks never reach the rate limit. However that would require a bit of advanced sysadmin and network code to achieve.

@unixfox
Copy link
Member Author

unixfox commented Jun 29, 2023

Anybody gave (SOCKS / http) proxy support any more thought? Should we start an issue for that?

(I'll gladly look into implementing it, although I have absolutely zero experience with Crystal so it may take a while.)

There is an open issue for that: #301


By the way useful and fun fact youtube is blocking per IP address even in ipv6. So if you have an ipv6 range of /64 available they will never be able to entirely block you because there are 18,446,744,073,709,551,616 IP addresses in a /64 ipv6 range.


Additionally, you can rotate requests through a whole /48 block so that individual /64 blocks never reach the rate limit. However that would require a bit of advanced sysadmin and network code to achieve.

We officially plan to provide dedicated tools for this.

@tio-trom
Copy link

By the way useful and fun fact youtube is blocking per IP address even in ipv6. So if you have an ipv6 range of /64 available they will never be able to entirely block you because there are 18,446,744,073,709,551,616 IP addresses in a /64 ipv6 range.

Yes but for example in my case, where I have a server with some 20+ services, changing the IPV6 all the time is never going to work since it will affect the other services. So it seems to me like hosting invidious is getting kinda impossible unless you use a VPS only for it.

@unixfox
Copy link
Member Author

unixfox commented Jun 29, 2023

By the way useful and fun fact youtube is blocking per IP address even in ipv6. So if you have an ipv6 range of /64 available they will never be able to entirely block you because there are 18,446,744,073,709,551,616 IP addresses in a /64 ipv6 range.

Yes but for example in my case, where I have a server with some 20+ services, changing the IPV6 all the time is never going to work since it will affect the other services. So it seems to me like hosting invidious is getting kinda impossible unless you use a VPS only for it.

More details from the invidious public room:

we are going to soon share a solution with a tool that will automatically rotate the IPv6 address every day while retaining all your existing networking so that you can keep using a fixed IPv4 and IPv6 address.

@iv-org iv-org deleted a comment from Fish25op Jul 2, 2023
@tio-trom
Copy link

we are going to soon share a solution with a tool that will automatically rotate the IPv6 address every day while retaining all your existing networking so that you can keep using a fixed IPv4 and IPv6 address.

Any update about this?

I tried to keep an eye on the matrix chat but was a bit too busy.

@unixfox
Copy link
Member Author

unixfox commented Jul 12, 2023

we are going to soon share a solution with a tool that will automatically rotate the IPv6 address every day while retaining all your existing networking so that you can keep using a fixed IPv4 and IPv6 address.

Any update about this?

I tried to keep an eye on the matrix chat but was a bit too busy.

No I'll post the tool here so no need to remind us ;). My time is a bit limited lately.

@unixfox unixfox changed the title [Discussion] How to circumvent YouTube blockage - #3822 related [Discussion] How to circumvent YouTube blockage - #3822 #4045 related Aug 17, 2023
@ccuser44

This comment was marked as off-topic.

@unixfox
Copy link
Member Author

unixfox commented Aug 27, 2023

I have not forgotten about the tool, I just had some free time this weekend to think about it.

Basically, I've discovered that in ipv6, it's very easy to force an IP to use a certain IPv6 range. So I'm going to develop a tool that automatically use a new IPv6 address every day and force this new IP address only through Google official IPv6 subnets: https://md5calc.com/google/ip

This way no breakage of current setup, it's compatible with all the Linux systems and very easy to setup. It will just require to use IPv6 on Invidious and have IPv6 connectivity on the server. It's possible to use a tunnelbroker if the server does not have IPv6 connectivity: https://tunnelbroker.services/

In a nutshell, it's just:

ip addr add my:new:ipv6/64 dev eth0
ip -6 route add google:ipv6:range/37 via my:ipv6:gateway src my:new:ipv6

That's it.

EDIT: Have to investigate if this work for programs in docker containers.

@unixfox
Copy link
Member Author

unixfox commented Oct 14, 2023

Hello to all public instances owners or people that are running a private Invidious instance that is blocked!

We have officially released a tool that allow to automatically rotate the IPv6 address of your Invidious server: https://github.com/iv-org/smart-ipv6-rotator and thus escape from the YouTube block!

You can follow the detailed tutorial here: https://docs.invidious.io/ipv6-rotator/

Please give your feedback, it is very important to us!


ping @tio-trom

@joaquinito2070
Copy link

It works.

@SamantazFox SamantazFox pinned this issue Oct 21, 2023
@unixfox
Copy link
Member Author

unixfox commented Nov 2, 2023

We have fixed an issue where invidious would not send all the traffic in ipv6: #4225
Please add this line in your config.yml to be sure that invidious send the traffic in ipv6:

force_resolve: ipv6

If you are using the rotator (https://docs.invidious.io/ipv6-rotator/), please make sure to have the line above and have updated Invidious!
We have reflected the documentation for this new change.

@animegrafmays

This comment was marked as resolved.

@Trit34
Copy link

Trit34 commented Dec 12, 2023

youtube is blocking entire /64's doing this now btw

Not problem on my favorite instance (for now).

@unixfox
Copy link
Member Author

unixfox commented Dec 13, 2023

youtube is blocking entire /64's doing this now btw

Fortunately that is not true.

@animegrafmays
Copy link

youtube is blocking entire /64's doing this now btw

Fortunately that is not true.

fortunate for you maybe. if I disable naked IPv6 and use tunnelbroker then invidious doesn't lock up loading a video. that's a block.

@unixfox
Copy link
Member Author

unixfox commented Dec 13, 2023

youtube is blocking entire /64's doing this now btw

Fortunately that is not true.

fortunate for you maybe. if I disable naked IPv6 and use tunnelbroker then invidious doesn't lock up loading a video. that's a block.

Please join our matrix for discussing more about it: https://matrix.to/#/#invidious:matrix.org

Github issues are not good for that.

@SamantazFox SamantazFox unpinned this issue Mar 30, 2024
@Ghodawalaaman

This comment has been minimized.

@unixfox

This comment has been minimized.

@unixfox
Copy link
Member Author

unixfox commented Aug 15, 2024

Closing in favor of #4734

@unixfox unixfox closed this as completed Aug 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discussion type:youtube-change Google changed something (again)
Projects
None yet
Development

No branches or pull requests