feat(YouTube): DeArrow / Clickbait Remover

[scientiac]

Application

YouTube

Issue

• People being fooled by the creators.
• People spending very long hours on YouTube.

Patch

The feature is this add-on:
Clickbait Remover browser extension

Motivation

• Reduces YouTube addiction and Increases the chances of getting a good YouTube Video.
• Also reduces YouTube consumption by making everything look not as pretty as it should look in the thumbnail.

Acknowledgements

• I have searched the existing issues and this is a new and no duplicate or related to another open issue.
• I have written a short but informative title.
• I filled out all of the requested information in this issue properly.

[ableitung]

I'd recommend integrating DeArrow instead. It allows people to crowdsource an image or a screenshot from a video that actually matters. Plus it can force proper capitalization on video titles.

[LisoUseInAIKyrios]

DeArrow is made by the creator of Sponsorblock.

Because the DeArrown API call would be asynchronous, this would require figuring out how to change Litho components after they are created and displayed.

Changing the thumbnail image might be possible without too much fuss, but changing the text after layout may be difficult.

[fillwithjoy1]

DeArrow is made by the creator of Sponsorblock.

Because the DeArrown API call would be asynchronous, this would require figuring out how to change Litho components after they are created and displayed.

Changing the thumbnail image might be possible without too much fuss, but changing the text after layout may be difficult.

Can you replace a video on the layout by another generated video layout that has the fetched info from DeArrow?

[LisoUseInAIKyrios]

Can you replace a video on the layout by another generated video layout that has the fetched info from DeArrow?

What do you mean change the video? Do you mean change the video thumbnail?

I spent some time looking for the code that handles the thumbnail generation, but I only found dead legacy code ImageView code that isn't used (I would estimate 80% of YouTube is unused legacy code). Layout inspector doesn't help because it only shows the thumbnail is litho component host (and doesn't reveal any more info). More research is needed.

[fillwithjoy1]

Can you replace a video on the layout by another generated video layout that has the fetched info from DeArrow?

What do you mean change the video? Do you mean change the video thumbnail?

I spent some time looking for the code that handles the thumbnail generation, but I only found dead legacy code ImageView code that isn't used (I would estimate 80% of YouTube is unused legacy code). Layout inspector doesn't help because it only shows the thumbnail is litho component host (and doesn't reveal any more info). More research is needed.

Yes, literally that. Is it not possible to load "cards" of each video without their images, and asynchronously fetch images before finally updating the "card" with the right thumbnail?

[LisoUseInAIKyrios]

The YouTube thumbnails first shows up on screen as a black or white empty frame, the thumbnail is then asynchronously fetched, and finally the empty thumbnail is changed to show the image.

I think the images are streamed across a common port or connection and not loaded thru individual urls. Or at least if they are individual urls it's not the imageview doing the loading because the serURL methods don't appear to be ever called.

[oSumAtrIX]

YouTube works almost entirely through ProtoBuf. Everything is streamed through a common port.

[fillwithjoy1]

Hmmm, can we block the main thread temporarily to fetch the DeArrow content, and if no response is occured after 5 secs (customizable), then we show the original video/thumbnail. Yes, this will make the app feel slightly slower, and may even give the impression that the network is bad, but could this be implemented?

[oSumAtrIX]

The problem is not the implementation. The problem is to find the real places that can be modified.

[LisoUseInAIKyrios]

@fillwithjoy1 would be way too laggy. The UI would freeze multiple times when scrolling thru multiple thumbnails.

Doing it asynchronously is not difficult, the issue is figuring out how to change the images and figuring out what video the thumbnail represents.

[oSumAtrIX]

DeArrow probably allows passing a list of ids and receiving a list of thumbnails and titles. Implementation is a second-order issue. The main issue is that we have to find where it loads the thumbnails and titles.

[fillwithjoy1]

Oh my bad I said the wrong things. You know how when you open YouTube, it loads the videos asynchronously, can we modify the loading code to include fetching full data from dearrow within x seconds?

[oSumAtrIX]

Technically possible yes.

[LisoUseInAIKyrios]

I found the thumbnail can be altered by searching thru and modifying the proto buffer. The thumbnails show up as a url in the buffer and it's fairly simple to change it.

This is not ideal and it's much better to find where this url is being used and change the code there, but this strategy does work.

I have these changes working locally, and it functions the same as the Click Bait Remover browser extension: it replaces the thumbnail with an alternate YouTube thumbnail url.

So the video:
https://www.youtube.com/watch?v=YbJOTdZBX1g
Normally displays the thumbnail: https://i.ytimg.com/vi/YbJOTdZBX1g/hq720.jpg

And ReVanced changes the thumbnail in the buffer to one of:
Early screen grab of the video:
https://i.ytimg.com/vi/YbJOTdZBX1g/hq1.jpg
Middle screen grab of the video:
https://i.ytimg.com/vi/YbJOTdZBX1g/hq2.jpg
Late screen grab of the video:
https://i.ytimg.com/vi/YbJOTdZBX1g/hq3.jpg
Unfortunate these screen captures are not high resolution and look a little blurry compared to the original thumbnail.

The ReVanced user settings has an option to pick which to use (early/middle/late), just as the the browser extension does.

More research is needed to find where these thumbnail URL's are being used in the app and make the changes there (instead of searching and modifying a general purpose buffer). But if that is never figured out or if it's not easy to make those changes, then this buffer modifying strategy could be used as a V0 implementation.

[oSumAtrIX]

The buffer is consumed by specific methods that read a number of bytes at a time and parse it. These methods can be hooked so that the index is captured they read from the buffer. Since you know at which index the link appears in you can determine which method reads it from the buffer.

[LisoUseInAIKyrios]

LibreTube is built from the ground up and has the Luxury of designing everything itself. It is not a patch for an obfuscated app released by another company.

While technically anything is possible with patching, it always falls into: "ok but how difficult is it to actually do that?"

[eclipsek20]

LibreTube is built from the ground up and has the Luxury of designing everything itself. It is not a patch for an obfuscated app released by another company.

While technically anything is possible with patching, it always falls into: "ok but how difficult is it to actually do that?"

Yes it's difficult but that's life. If the team works hard enough on it, the end result will work 😃

[LisoUseInAIKyrios]

After much searching, I found a few suitable patch points to modify the thumbnail image url (and the thumbnail url conveniently includes the video id).

I see that DeArrow has an service that provides a replacement thumbnail image simply by fetching a specific image url based on the original video url video id. This is super helpful and dramatically simplifies the patch, and I have a v0 version working locally (it modifies a single URL string parameter of a single method). Except, the image replacement service does not work for all videos, and like any service it may not always be available. Handling these failure cases makes the patch more complicated because additional changes are needed so it can fallback to using the original thumbnail url (I currently do not have any fallback logic implemented).

I see that FFMPEG can generate screen captures of a video, which is what the DeArrow thumbnail cache service does. Perhaps the client could behave the same, and generate the thumbnails locally using FFMPEG (or using some other 3rd party java code)? This would be complicated in it's own right, but it would work for all videos and would always be available. I don't think using Android YouTube to generate thumbnails is a viable option (it's too obfuscated, and way too complicated).

Any thoughts or suggestions?

[oSumAtrIX]

Handling these failure cases makes the patch more complicated because additional changes are needed so it can fallback to using the original thumbnail url (I currently do not have any fallback logic implemented).

If you hook the string parameter, can you not passthrough it?

[oSumAtrIX]

and generate the thumbnails locally using FFMPEG

This is way beyond what is necessary for us. Depending on if you can simply passthrough the hooked string parameter using the existing service, it shouldn't cause any issues. I have no details about your implementation, so I can not say for sure though.

[LisoUseInAIKyrios]

If you hook the string parameter, can you not passthrough it?

What I currently have, modifies a url parameter that's passed into a URL image loader.

So an original String parameter of:
https://i.ytimg.com/vi/R2oD1ZHNMFE/hq720.jpg
is changed to:
https://dearrow-thumb.ajay.app/api/v1/getThumbnail?videoID=R2oD1ZHNMFE
And then YouTube handles loading the modified url the same as the original thumbnail url.

But the thumbnail api does not work for all videos, which causes the client to show an empty thumbnail because no image is returned for videos it does not have cached.

I will try to change the patch point to somewhere in the YouTube URL loader code, so ReVanced can selectively handle fetching the thumbnail image and fallback to loading the original thumbnail URL if needed.

I will also search around a bit and maybe there exist some simple java code to generate a screen grab from a YouTube video (without using something as huge or cumbersome as FFMPEG). But if that's infeasible then it can show non clickbait thumbnails for some (but not all) videos.

[ajayyy]

Generating thumbnails for the first time can take multiple seconds. Doing it server side allows most requests to have been pre-rendered and stored which isn't possible if done on the client side.

The extension does do local rendering though when the server fails, which mostly happens for geoblock reasons. But a simpler fallback is to just show the original thumbnails in these cases instead

[ajayyy]

What if my thumbnail cache returned a 301 redirect on failure to the original thumbnail. Would that get handled properly?

I could also make a new endpoint to do the getBranding call on the server side as well to know what timestamp to use.

Then you'd have one request to get an image or a redirect

[oSumAtrIX]

Otherwise, the patch could download the headers only (to check for 404) for maximum performance before passing the URL to YouTube.

[LisoUseInAIKyrios]

What if my thumbnail cache returned a 301 redirect on failure to the original thumbnail. Would that get handled properly?

That should handle the case of no thumbnail existing. I don't know for sure if the YouTube image loader will follow a 301, but I assume it will.

However ReVanced would still need some extra code to detect if the thumbnail API is down so it can fallback on normal thumbnails. So the 301 behavior would definitely help, but may not be absolutely needed.

[LisoUseInAIKyrios]

After much research, I could not find any simple way to intercept the network call for the thumbnails to handle a fallback behavior. The YouTube code that handles the network call is to complicated to easily patch, and there's a huge potential for introducing bugs by injecting a second fall over http call. Instead, I was able to hook onto an existing callback handler and detect if the url fails to load (such as an unexpected http status response, host not found, connection failure, etc). Then if enough failures happen very quickly (such as 5 failures over a 1 minute period), the client temporarily turns off DeArrow for a fixed amount of time and uses the original thumbnails. I have it set to temporarily turn off for 10 minutes, but the length of time could be set dynamically.

An alternate idea is for ReVanced integrations to run an ultra thin local http "server" that handles fetching the dearrow thumbnail and then falls over to using the original youtube thumbnail. The ReVanced patch would change the thumbnail url to point to itself (ie: https://i.ytimg.com/vi/R2oD1ZHNMFE/hq720.jpg would change to: https://127.0.0.1/vi/R2oD1ZHNMFE/hq720.jpg) The open socket would only accept connections from localhost, and it would simply fetch the DeArrow thumbnail and if that fails then fetch the original youtube thumbnail. This would always work, would allow using the DeArrow API with a 3rd party thumbnail server, and could also allow the client to locally generate the thumbnail if that was ever desired. It appears a thin server could be done with only a few dozen lines of code, but I have not tried implementing this idea just yet.

[LisoUseInAIKyrios]

@ajayyy can you try adding the 301 forward if a thumbnail is not available? I have everything almost finished locally, and it works great (the experience is so much better!), but it still needs this 301 forwarding for it to work with a simple patch.

To handle an overloaded server (there are a lot of ReVanced users), maybe the thumbnail API can return a specific http status code indicating how long the client should temporarily back off. Maybe something like 531 = backoff 1 minute, 535 = backoff 5 minutes, etc.

On a different idea, the DeArrow thumbnail api maybe could support different resolutions.

Android YouTube seems to use only 2 of the different thumbnail sizes:
https://i.ytimg.com/vi/R2oD1ZHNMFE/hqdefault.jpg (480x360) - 25 KB
https://i.ytimg.com/vi/R2oD1ZHNMFE/hq720jpg (1280x720) - 104 KB
And commonly uses webp as well:
https://i.ytimg.com/vi_webp/R2oD1ZHNMFE/hqdefault.webp (480x360) - 16 KB
https://i.ytimg.com/vi_webp/R2oD1ZHNMFE/hq720.webp - (1280x720) - 52 KB

If the client passed in the original file name, then the thumbnail server could hand back a smaller image in some cases.

So an original thumbnail:
https://i.ytimg.com/vi_webp/R2oD1ZHNMFE/hqdefault.webp
would be called with something like:
https://dearrow-thumb.ajay.app/api/v1/getThumbnail?videoID=R2oD1ZHNMFE&size=hqdefault
and it would return a smaller image.

This could speed up the client loading the thumbnails, and possibly save some money on bandwidth hosting.
But of course, the server would need to store at least 2x as many images (or pay processing cost to resize on the fly).

[oSumAtrIX]

@LisoUseInAIKyrios It may be a good idea to open issues regarding DeArrow in their respective repositories so they are correctly trackable and link them here. This way, the maintainer doesn't have to track this here.

[ajayyy]

Do you know the original thumbnail url ahead of time. As in, can it be provided as a parameter, or does the DeArrow server have to craft it

[LisoUseInAIKyrios]

The original url is known ahead of time. So it could be passed as a parameter.

[ajayyy]

Added

https://dearrow-thumb.ajay.app/api/v1/getThumbnail?videoID=R2oD1ZHNMFE&redirectUrl=https://i.ytimg.com/vi_webp/R2oD1ZHNMFE/hq720.webp

https://dearrow-thumb.ajay.app/api/v1/getThumbnail?videoID=SomeWrongID&redirectUrl=https://i.ytimg.com/vi_webp/R2oD1ZHNMFE/hq720.webp

[LisoUseInAIKyrios]

Perfect!
It works as expected 👍

[YT-Advanced]

Perfect! It works as expected

How about the title, it is quite hard to modify, I think

[LisoUseInAIKyrios]

Decided to split DeArrow and the alt images (ie: clickbait remover) into two separate patches.

Alt images (clickbait remover) is first, as it's the simplest and working:
#2834

Will do DeArrow patch later.

When DeArrow does not have a user selected thumbnail, the results are similar to the built in alt images. But where DeArrow really shines, is when it does have user generated thumbnails:

Alt thumbnails:

DeArrow

Obviously the DeArrow thumbnails are way better.

It would be great if the DeArrow thumbnail api had an extra parameter, to indicate the DeArrow thumbnail should be provided only if a user selected thumbnail exist. Then DeArrow could be used as a happy medium of "show me a user selected DeArrow thumbnail, otherwise show me the alt image or show me the original thumbnail".

As for implementation, I tried the lightweight http "server" idea, and it worked very well...except it fails on root installation. Because Android requires HTTPS for all connections, it requires either modifying the app manifest and adding an HTTP exception (which completely fails on root installation since the device always uses the unmodified manifest). Or it requires generating a self signed SSL certificate, and patching Cronet to allow self signed certificates on localhost connections (a mess, and a bunch of code to import including bouncy castle to generate the certificate). I gave up on this idea at that point.

Another alternative idea I have not yet tried, is to save the images to a file, and then alter the image url to a file:// format. I don't know if Cronet will load a file url, but if this does work, it would help with loading latency since it would perform:

• 1 http call if a DeArrow thumbnail exists
• 2 http calls if DeArrow doesn't exist, but an alt image does exist.
• 3 http calls, if both DeArrow and an alt thumbnail does not exist (worse case, and rare).

Because the current strategy of verifying URL's before altering the original url, it always makes a minimum of 2 http calls and if using both DeArrow and alt thumbnails it always makes 2 or 3 network calls.

Update: the file url idea did not work, and cronet refuses to load a file url of a local temporary file. Maybe I was doing something wrong, but it all looked sound and it simply refused to load file:// urls (even after modifying the manifest and adding permission to access SD card). It might work with additional patch changes, but it seems this idea currently is a no go.

Lastly, the YouTube thumbnails have extra parameters that presumably are user and view tracking. If these parameters are left out, the thumbnails load just fine, but YouTube can start acting strange. If scrolling down the home feed to the very end, it sometimes gives a strange error message and is unable to load more videos. I think the recommendations algorithm is confused how the user reached the end of the home feed without viewing any of the feed items it previously recommended. This could be fixed by making a dummy call for the original thumbnail (including it's tracking parameters), if the DeArrow image is used. It could be done in the background, so it would not effectively add any extra latency.

[GeekCornerGH]

@LisoUseInAIKyrios Would titles be included in the DeArrow patch?

[LisoUseInAIKyrios]

@GeekCornerGH At this time, no, the titles will remain unchanged.

[reisxd]

Well, I'm pretty sure this would run very badly for non-flagship phones, like mine. Also, this'd require to add a huge model to the patched application and run it locally, which the performance can vary for each device. Running it on a local PC could also most likely give bad results, and who would want to even keep their PC on just for AI thumbnails? And as you said, a paid API is not something we could do.

[LisoUseInAIKyrios]

That's why I'm calling it an entertaining idea. Right now It's only realistic if someone else implemented a 3rd party paid service, but as I mentioned it's way beyond the scope and purpose of ReVanced.

[enderprism]

DeArrow includes a non-clickbait title feature, that doesn't have a Revanced patch. Also note that the current clickbait replacer can't fetch community DeArrow thumbnails, which is what I use (I don't like the auto-generated thumbnails). Are these features planned?

[fillwithjoy1]

uhh why we using stable diffusion? sounds like a bad idea? battery drain? Anyways, retrieve the server thumbnails and titles if a dearrow title and thumbnail exists instead

…

On Tue Nov 7, 2023, 06:08 PM GMT, EnderPrism ***@***.***> wrote: DeArrow includes a non-clickbait title feature, that doesn't have a Revanced patch. Also note that the current clickbait replacer can't fetch community DeArrow thumbnail, which is what I use (I don't like the auto-generated thumbnails). Are these features planned? — Reply to this email directly, view it on GitHub <https://github.com/ReVanced/revanced-patches-template/issues/1642>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHBFVTSY7ANYZBXU7O4AIYLYDJ2JDAVCNFSM6AAAAAASUOCC56VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJZGM3TINBZGU>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[enderprism]

wdym
i’m just asking if there is an option for this planned (fine-tune options instead of a global dearrow toggle would be nice)
we could put a warning in the app next to the toggle so users know about the eventual battery drain and network usage

[fillwithjoy1]

wdym i’m just asking if there is an option for this planned (fine-tune options instead of a global dearrow toggle would be nice) we could put a warning in the app next to the toggle so users know about the eventual battery drain and network usage

The battery/network impact is completely negligible. After all, this is a video streaming app which in itself consumes a lot of power and network bandwidth, so I don't get your point at all.

[Zero-G-Sys]

This needs an option for only showing the community submitted DeArrow thumbnail, otherwise fallback to the original. The auto generated ones are most of the time quite bad.
Maybe asking the DeArrow dev to put a parameter in the api path to not return a thumb if there isn't an user submitted one

[oSumAtrIX]

@Zero-G-Sys https://github.com/ReVanced/revanced-patches/pull/3378/files#diff-ecead68b5f5e43a0b307e4e1691735553c18edecc876b85df2a9fd61dd88bee5R5

[LisoUseInAIKyrios]

The browser extension has an option to show crowd sourced thumbnails (and not show auto generated). But ReVanced will use the thumbnail api and there currently is not a way to request only crowd sourced images thru that end point.

@Zero-G-Sys maybe create a feature request in the DeArrow repo, requesting to add a 'crowd source only' parameter to the thumbnail end point.

[fillwithjoy1]

Are there any ways to get modified titles working at all? Sent with Shortwave <https://www.shortwave.com?utm_medium=email&utm_content=signature&utm_source=am95c2luZ2guanMxQGdtYWlsLmNvbQ==>

…

On Tue Dec 12, 2023, 12:11 AM GMT, oSumAtrIX ***@***.***> wrote: Closed ReVanced/revanced-patches-template#1642 <https://github.com/ReVanced/revanced-patches-template/issues/1642> as completed via ReVanced/revanced-patches#3358 <#3358>. — Reply to this email directly, view it on GitHub <https://github.com/ReVanced/revanced-patches-template/issues/1642>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHBFVTU3FNTCOSV2J76ETODYI6OLXAVCNFSM6AAAAAASUOCC56VHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJRGIYTOMJYGI2DEOA>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[LisoUseInAIKyrios]

The big requirement is the DeArrow titles would need to be known before the UI is created. So far nobody has figured out how to change litho components after they are created, so it would need to fetch the titles before the UI is created.

Changing the titles in the home feed or search would require:

1. Find the video ids of the feed items (before the UI is created, and off the main thread).
2. Fetch the DeArrow titles.
3. Change the video titles during the UI creation.

Item 1 is the hardest part.

Item 2 is predictable and easy.

Item 3 may or may not be hard.

Changing the title of the video in only the video player might not be that difficult (item 1 is already known for that use case), but it's also not as useful as the feed/search.

[fillwithjoy1]

Yikes, I see the issue as well. I suppose for the time being we are just going to have to be content with what we have. It's good though that YouTube has brought AI topics to comments, so maybe they might do that with titles later on (premium subscription opportunity, youtube plz take notes)