Skip to content
This repository has been archived by the owner on Dec 11, 2019. It is now read-only.

YouTube showing ads #4693

Closed
bsclifton opened this issue Oct 12, 2016 · 109 comments
Closed

YouTube showing ads #4693

bsclifton opened this issue Oct 12, 2016 · 109 comments

Comments

@bsclifton
Copy link
Member

Did you search for similar issues before submitting this one?
Yes

Describe the issue you encountered:
Reported via support:

I’ve been trialing Brave for the last day or so. I was enjoying no ads on youtube until about the 3rd or 4th clip, then straight up I started getting ads. How do I fix this? The ad blocker is switched on in settings.

I tried some of the video links provided and was unable to reproduce. Maybe related to

Expected behavior:

  • Platform (Win7, 8, 10? macOS? Linux distro?):
    Windows 10
  • Brave Version:
  • Screenshot if needed:
  • Any related issues:
@srirambv
Copy link
Collaborator

One more from support

@neeklamy
Copy link
Contributor

Silly question, but worth asking somehow, these are definitely adverts getting through?

I ask because I have been using Brave for many months now, since 0.7, I frequent YouTube, and I’ve yet to see an advert slip through. I haven’t seen any “banner” ads pop up over videos, before the video begins or breaking up a longer video.

I would suggest this could be a few “sponsored by” sections that are part of the video (either at the beginning or end) being mistaken for adverts.

Might be an idea to ask anyone experiencing this to grab a screenshot.

@CliqueBait
Copy link

I can confirm for my install. I see no banner ads or ad breaks in the middle of a video. However, I have been seeing the occasional ad at the beginning of videos which need to be skipped in the usual way.

Ubuntu 16.04.1 LTS
Brave: 0.12.4
Electron: 1.4.12
libchromiumcontent: 53.0.2785.143
V8: 5.3.332.47
Node.js: 6.5.0
Update Channel: dev

image

@neeklamy
Copy link
Contributor

@CliqueBait, I’ve checked out the playlist you’re watching (starts here: Does Political Correctness WORK?) I’m up to the 16th video – but nothing!

  • Brave 0.12.4
  • OS X 10.11.6
  • HTML5
  • Shields up
    • Show Brave ads
    • Block 3rd Party Cookies
    • Block Phishing / Malware
  • Shields down – adverts have to be showing now? Nope!
  • Using a private tab

Under a guest account in OS X, I managed to get the adverts to show immediately by dropping the shields or allowing ads and tracking. Maybe this is a stuck setting somewhere, something from a previous version? It might help to blow away Brave’s config files…

@bsclifton
Copy link
Member Author

bsclifton commented Oct 13, 2016

@neeklamy where are you physically? I see that @CliqueBait is in the UK (thanks for the screenshot, BTW)

I wonder if they are using different code based on the region. I know this is typical because many videos are blocked based on region (and also ads are region aware)

@neeklamy
Copy link
Contributor

@bsclifton, I should have said, I’m in the UK too.

When I was toggling the shields, although it showed that the shields were down, the “Ads and Trackers Blocked” count was still going up, but, no ads were showing. Which as I said, makes me think it seemed like the “ads off” setting was stuck.

At some point the adverts started showing with the shields down, and now I can toggle the shields on and off and get the expected behaviour – not terribly helpful, I know.

@Flo5701
Copy link

Flo5701 commented Oct 18, 2016

I have add on Youtube to (I'm in canada), i try with a VPN in sweden and no add

@CliqueBait
Copy link

@neeklamy "It might help to blow away Brave's config files..."

Um. That doesn't sound too safe/desirable if I'm going to lose anything I need but I'm willing to inspect the files and roll the dice (with backups) if it looks promising. There's a chance it might work because I've been installing each release over the prior release so there might indeed be a stuck setting somewhere.

Main thing I don't want to do is lose all of my bookmarks. I don't care about other settings. Which files do you want me to inspect/delete?

FYI, I upgraded to Ubuntu 16.10 last Friday 14th Oct.

@reesinkf
Copy link

I've always had YouTube ads showing up since I started using Brave (many months ago). I just figured they had not finished the code yet or something (since it's still a beta). It's very very annoying to say the least, and with every update I hoped they started "supporting" YouTube ads...

FYI, I'm in the Netherlands and on Windows 10.

@Dylvn
Copy link

Dylvn commented Oct 20, 2016

I have the same problem for YouTube, the ads at the beginning pop! :)
I'm from France for info !

@tcr
Copy link
Contributor

tcr commented Oct 26, 2016

Using Brave, an ad plays on Youtube for me for nearly every video. Located in the US. Happy to perform any tests to help diagnose the problem.

@luixxiul luixxiul added the needs-investigation A bug not 100% confirmed/fixed that needs QA to better audit. label Oct 27, 2016
@bbondy
Copy link
Member

bbondy commented Oct 28, 2016

Where are you located? Would any of the about:adblock regional lists help?

@tcr
Copy link
Contributor

tcr commented Oct 28, 2016

@bbondy I'm located in Boston, MA. I assume that means none of the regional lists would be useful here?

I wonder if Youtube ads are cosmetic adblock ads (which aren't supported yet?) and I'm just unlucky that I get many ads served to me and others get few. If they're URL-based adblock ads, I should be able to type in custom rules in about:adblock and seem them succeed or fail in the Web Debugger.

@bsclifton
Copy link
Member Author

bsclifton commented Oct 28, 2016

@bbondy I'm curious- does YouTube use Flash if it's available? Is it possible that these are Flash ads getting through? Do you have Flash installed @tcr?

@tcr
Copy link
Contributor

tcr commented Oct 28, 2016

@bsclifton I have Flash installed; but it's not enabled for YouTube, and the ad element being displayed is a <video> element.

I can dump a HTTP log if it's helpful (and if I can figure out how to do it from the Web Console).

@bbondy
Copy link
Member

bbondy commented Oct 31, 2016

If we can find the URLs of the ads that are serving I can check if they are in the adblock list.

@tcr
Copy link
Contributor

tcr commented Oct 31, 2016

Looking into it, it seems to be serving from the same servers as YouTube videos. It skips two doubleclick domain requests successfully, then originates a video request from YouTube's servers. It might be the case that adblock is working fine and that the current adblock rules just don't work if they are routed through the normal video servers.

Also, it only serves ads when I am logged in. Logging out or a new session tab works fine. So I believe the block is working fine. Maybe cosmetic (HTML selector) rule support will fix this.

This video reproduces the issue 80% of the time for me: https://www.youtube.com/watch?v=cAo9o7zV24A

@Flo5701
Copy link

Flo5701 commented Nov 3, 2016

with the new update i have no add on youtube !

@CliqueBait
Copy link

Still an issue for me unfortunately:

Ubuntu 16.10
Brave: 0.12.7
Electron: 1.4.20
libchromiumcontent: 53.0.2785.143
V8: 5.3.332.47
Node.js: 6.5.0
Update Channel: dev

@srirambv
Copy link
Collaborator

srirambv commented Nov 5, 2016

I had not experienced this till today but I suddenly see the Ad on YouTube. Took almost couple hours of running videos for the ad to show up.

image

@bsclifton
Copy link
Member Author

bsclifton commented Nov 7, 2016

Another user reported this issue with #5457

  • Windows 10
  • Brave 0.12.7

Screen cap of issue:
dstctuv

@alexwykoff alexwykoff added this to the 0.12.11 milestone Nov 9, 2016
@KronemeyerJoshua
Copy link

Yep. Still an issue in 0.12.8
Windows 10
Please don't make me watch scientology ads. xD
ad

@BrendanEich
Copy link
Member

BrendanEich commented Nov 15, 2016

A friendly with ad-tech chops has analyzed what's going on, hoping he'll update here shortly.

Cosmetic filters is #344 but this bug has priority and can be fixed without those, we think.

@pixelass
Copy link

I just noticed ads in the search results

screen shot 2016-11-15 at 6 15 36 pm

@cndouglas
Copy link

By the way, to generate a new Brave profile for testing but preserve your personal profile, just rename the Brave profile directory (at ~/Library/Application Support/brave) to something else (e.g., brave1). When you are done testing, delete the new directory (called brave) and rename the old one back to brave.

When I went to set up LastPass, it remembered my email address. I had told it to remember email, so normally that's expected, but having "completely" uninstalled, I would have expected that information to not be there.

@echosa: It is possible that LastPass has your email address stored in the macOS Keychain. You can find out by opening the Keychain Access utility.

@lukemulks
Copy link
Collaborator

@echosa no problem! Going to include them all here for quick reference:

Ads Settings:
For these two settings, try this URL: https://www.google.com/settings/u/0/ads/anonymous?hl=en

  • Ads based on your interests on websites beyond google.com
  • Google Search Ads based on your interests

For the following setting, try this URL: https://www.google.com/settings/u/0/ads/authenticated?hl=en

  • Ads based on your interests (When you are signed in to Google sites)

Related Privacy Settings:
Google Activity Controls: https://myaccount.google.com/activitycontrols

I've been unable to reproduce ads in YT playback with the switches for these settings in the OFF position (gray, with the switch in the left position):

  • Web & App Activity (paused)
  • Location History (paused)
  • Device Information (paused)
  • Voice & Audio Activity (paused)

Note: There are two Youtube specific settings at the bottom of that page, but I have them both ON (blue) and am not getting any ads. You can try shutting those off if you'd like. I have kept them on since I'm not seeing ads w/them on.

Hope this helps!

@echosa
Copy link
Contributor

echosa commented Feb 1, 2017

Neither of those URLs take me to pages that look like your image. :-/ When logged in to Google, both of those first two links take me to the same page. The page they take me to is the last screenshot I posted a couple of comments back; the one with a single toggle, which I turned off.

If I visit the first link in a new session tab (which makes me not logged in), I get this:

screen shot 2017-02-01 at 3 07 03 pm

I turned them off, but when I went back to the page in another new session tab, they were both on again. I guess I have to actually logout and turn those off, because a new session tab's changes aren't permanent or something? It's a bit confusing.

Either way, as you can see, I'm not seeing any pages as what your screenshot showed. Hopefully the pages I am seeing are equivalent, or at least good enough. I wonder why we see different things, though.

@pixelass
Copy link

pixelass commented Feb 1, 2017

@echosa @lukemulks what kind of Google accounts do you have?
I have a free business account from the BETA-era of Google domains.
Maybe it is different with default google accounts (foo@gmail.com vs. foo@bar.com)

@echosa
Copy link
Contributor

echosa commented Feb 1, 2017

I don't know how to answer that. I signed up for a Gmail account many, many years ago, when it was still invite-only. The rest, as they say, is history.

@pixelass
Copy link

pixelass commented Feb 1, 2017

I was actually right.

If you have a business account (use your own domain) you get the pages @lukemulks added as screenshots.

If you have a default gmail account example@gmail.com you get the screen that @echosa posted

screenshot from my test account

screen shot 2017-02-01 at 10 26 32 pm

@echosa
Copy link
Contributor

echosa commented Feb 1, 2017

That makes sense for logged in, but what about logged out?

@lukemulks
Copy link
Collaborator

lukemulks commented Feb 1, 2017 via email

@pixelass
Copy link

pixelass commented Feb 1, 2017

The link is on the page that business accounts get.

screen shot 2017-02-01 at 10 30 53 pm

@pixelass
Copy link

pixelass commented Feb 1, 2017

Maybe it's a feature that has not been ported to either of the two versions. They might not even use the same logic, framework, whatever. Google is big at making a lot of custom stuff that doesn't fit together, it's part of moving along and building new ideas. Just spitballing here though.

EDIT: There are a lot of domain speciffic settings available. You can add and remove different apps to your domain, so the settings page might also take that into account. Just looking at the description of what will or will not be personalized if the switch is toggled.

@echosa
Copy link
Contributor

echosa commented Feb 1, 2017

Well, I turned off what I could find, so hopefully that'll do some good. Sticking to the original plan: ads seem to not be playing, which is great, but I'll keep an eye out and report any that do. Thanks for this Google settings tangent. 👍

I don't understand why Google couldn't just put them all on the same page

If it is easy to do so, more people will turn off the tracking and ads and such. Google doesn't want that but also wants to be able to say they provide a way to do so. So they provide a complicated way people are likely to avoid or miss. It's a win/win for them. :-P

@pixelass
Copy link

pixelass commented Feb 1, 2017

@lukemulks that confused me. Do you have a custom domain? Those were given in the invite and beta phase for free too.
My test account (@gmail.com) is about 1 year old and it has is a vanilla gmail account and I get the same screen as @echosa

My own account (@pixelass.com) has the same screen as you posted

@lukemulks
Copy link
Collaborator

lukemulks commented Feb 1, 2017 via email

@lukemulks lukemulks self-assigned this Feb 2, 2017
@lukemulks
Copy link
Collaborator

With the latest set of settings adjustments (Google ads settings), and the attempts to repro by @srirambv @pixelass, @echosa and me that results in no ads displaying, and the lack of additional reports from users post v0.13.1, I'm going to close out the issue at this time.

If there are repeats of ads displaying for users that are signed in, please let us know and we'll triage those cases into a new issue to specifically investigate further. Thanks again to everyone that helped get us to this point.

@tcr
Copy link
Contributor

tcr commented Feb 3, 2017

For future travelers: following the instructions in this comment by @lukemulks resolved the issue for me in Brave 0.13.0.

@borntohonk
Copy link

borntohonk commented Feb 6, 2017

I've had those google settings off forever @lukemulks

looking in the debugger / developer tools, it's apparent that google forces another video up instead of the video I intend to watch.

note: I have a pi-hole on my network aswell, which has successfully prevented any loading of ads in any browser on any device (including my ipad) until recently, probally february first. So this is a youtube thing. (https://github.com/pi-hole/pi-hole)

A friend links me this:

https://www.youtube.com/watch?v=ye24Qehidkg

and this gets forced before it (as an ad)

https://www.youtube.com/watch?v=Lpbda_NXRL0

"html5-video-player ad-created ad-showing ad-interrupting videoAdUiRedesign ytp-video-ad-learn-more-ui ytp-title-extra-info paused-mode"

I'm guessing this is a testpilot system being rolled out, and that it's being regionally tested.

Might be interesting to note, the videos that get forced as ads, don't have any ads themselves.

mashing F5 on any other video gives ads 100% of the time before loading the intended video I'm clicking (randomly).

but doing the same on the videos designated as ads, it jumps straight to the video. (probally some sort of whitelist to not get an ad ontop of a forced ad?)

here's a recording demonstrating it, brave adblocker enabled, pihole also active on a random video I got to by just clicking random trending videos.

https://a.pomf.cat/vqaxlm.webm

forcedads

ads

adsblocked

just for referance, there are ads on youtube that my pihole do prevent addition to the ones violating my eyes.

pihole

@lukemulks
Copy link
Collaborator

Thanks for the in-depth followup @borntohonk

A few questions, I know they're likely redundant, but have to go thru the motions.

  1. Can you confirm which version of Brave you're running where this was observed?

  2. Can you let me know if you cleared cache/cookies and then restarted the browser prior to this being observed? If not, can you try that and let me know if the issue persists for you?

v0.13.0 and higher w/cache & cookie clearing and the ad settings updated should resolve for the majority of cases.

You're right about this being a YT-specific thing. I suspect that this is likely what you're suspecting as well, a regional rollout test for a player update. I saw the same thing happening last fall prior to the last update.

OK so aside from the questions I asked above, this is why it's such a pain of an issue:

  • DNS blocking for YT ads has been tricky, as Youtube (and Google in general) are packing advertising in youtube.com urls, and others that aren't historically ad-related.

  • What you're observing from the DOM is how Google and YT serve video ads, except that you've caught one of a few of the fallback player configs that are part of the overall player configuration.
    Google and YT use the IMA SDK, which basically overlays a video player on top of the content player for the ad, which destroys after the ad completes. The player below is paused while the SDK is playing the ad. They've been experimenting more with the sdks lately as well. YT tries for flash first, then ultimately fails over into an embedded adsense viral vid player as a last stop.

For the pi-hole setup, can you let me know if there are any overrides applied on your end that would allow for any protocol exceptions over ports?

Thanks again for the details here, really appreciate it.

@borntohonk
Copy link

borntohonk commented Feb 6, 2017

@lukemulks

  1. version 0.13.1

version

  1. clearing cookies (and by that logging out), the ads get cockblocked by my pihole (causing a tiny delay for intended video to load up), otherwise if I log in it returns to same pattern of 100% rate of ads played infront of any video.

note: ads still try to "load" if cookies (logged out) are cleared, but the pihole just sends it to a black-hole.

at this point being logged in at youtube seems to be a disadvantage, but at same time probally find whats coming in near future to "everyone". Considering browser adblocker and my network hardware adblocker have been bypassed/killed by this solution, I'd say that's more likely the case.

the pihole is just a dedicated dns

current setup is my computer (or other devices) have DNS server set as local address 192.168.1.85 (pihole) which in turn communicates with 8.8.8.8 and 8.8.4.4 (google DNS servers)

I query 192.168.1.85 for DNS resolves, if on filter list return 0.0.0.0, if not on filter list query 8.8.8.8/8.8.4.4 then return to device in question.

I have somewhat technical understanding of things, but only on a hobby level.

https://github.com/pi-hole/pi-hole/blob/2f3cf1c1b53514260636f3adea6511a0a86a3c37/adlists.default

logging out it's apparant it still tries to load ads, but nothing shows up visibly because the ads are routed to 0.0.0.0 and return 'nil' instead

only conclusion I can get out of this is to not be signed into youtube to not get ads until this solution finally comes over and annoys the average person who also has adopted an adblock solution.

probally interesting to note that the videos designated as ads have some sort of exception/whitelist to be affected by other ads.

it's obviously paid content / partner solution.

Norwegian ads aren't that bad, most of it seems innocent because of our laws are very strict, but it's still annoying.

loggedout

@lukemulks
Copy link
Collaborator

lukemulks commented Feb 6, 2017

Thanks @borntohonk

  • Yeah what you've noted about being logged in v logged out is similar to what has been reported by other users.

  • Got it re: the pi-hole cockblock. The thing is, Google has penetrated the cookie store so deep in order to keep serving ads that until users cleared cache and cookies, the issue appeared to persist post-update.

  • If you were able to clear cache/cookies, then apologies for the redundancy, just letting you know because the youtube case in general has been really oddball.

  • Reason I asked why there might be any overrides applied are due to the nature of the update that was required in order to block these effectively.

We're still investigating the usage, so just a forewarning there, but we discovered that QUIC over UDP protocol was being used by default by Google in Chromium for ad requests that had been merged into Google non-ad domains (that are now considered fair game for ads by Google's documentation, unfortunately).

For this reason, YT in particular was not blocking ads until we went in and applied some specific blocking to QUIC. We're digging in on this further, as while Google had mentioned QUIC being used in Chromium by default for ~50% of Google's stack, it hadn't been mentioned that ads were involved with this, and until their recent update they would have silo'd the ad domains separately. Once we blocked QUIC this way, the issue was not reproducible on our end in a majority of attempts.

FYI - since you're a bit of a guru on the DNS side, and we can use all the help from good natured people that are DNS-inclined, here's the list of domains, which helps break out why DNS blocking has been such a pain:

  • admob.com

  • adsensecustomsearchads.com

  • adwords.com

  • doubleclick.net

  • google.com

  • googleadservices.com

  • googleapis.com

  • googlesyndication.com

  • googletagmanager.com

  • googletagservices.com

  • googletraveladservices.com

  • googleusercontent.com

  • google-analytics.com

  • gstatic.com

  • urchin.com

  • youtube.com

  • ytimg.com

  • When we tested blocking the highlighted domains above, it blocked core product functionality that wasn't ad related, so we're having to get pretty surgical.

  • The v0.13.0 release resolved the majority of issues reported, but there were some residual cookie/cache cases where until the cookie/cache was cleared, users occasionally observed ads.

  • The ads settings was another outlying item that we discovered needed to be updated for some users, once they did, the ads were blocked.

All that said, what we're both seeing (which is reassuring) is Google really going to some extremes to hack ads into the user experience.

I think that what you reported could also be a prototype test as you mentioned. Google does this in YT with previews, and users aren't aware. I worked closely w/Google for 5+ years and we were often in a spot like this where we'd end up reverse engineering their stuff in the wild and it would only apply to some regions.

Also, thank you again for the info and feedback, we're really lucky to have you as a contributor here. :-)

@borntohonk
Copy link

borntohonk commented Feb 6, 2017

@lukemulks the pi-hole github is probally a mecca for the blocking of dns kind.

I use the default compilation that the list, which the pi-hole installer compiles to a single file, because it breaks minimal amount of stuff so far.

it does prevent some minor things occasionally, mostly related to shopping and using searches where paid contributors get promoted with "ads".

the 98 608 entries in this hosts file are a result of combining all the lists from ( https://github.com/pi-hole/pi-hole/blob/2f3cf1c1b53514260636f3adea6511a0a86a3c37/adlists.default )

I'm just an end-user, so credit obviously would go to the pi-hole github contributors ( https://github.com/pi-hole/pi-hole )

note: this is more than just ads, this is a lot of fradulent/malware/tracker websites too.

hosts.txt

Quick rundown:

admob : 52 entries
adsense : 6 entries (adsensecustomsearchads.com : 1 entry)
adword : 11 entries
doubleclick : 12 551 entries
*.google.com : 23 entries
googleadservices : 0 entries
googleapis : 1 entry
googlesyndication : 12 entries
googletagmanager : 2 entries
googletagservices : 2 entries
googletraveladservices : 0 entries
googleusercontent: 2 entries
google-analytics: 4 entries -> *analytics : 267 entries
gstatic 13 entries
urchin : 5 entries
youtube : 9 entries
ytimg : 0 entries

@lukemulks lukemulks reopened this Feb 27, 2017
@echosa
Copy link
Contributor

echosa commented Mar 1, 2017

Is there a reason this closed? I was just about to comment to say that I've been seeing ads again, lately.

@lukemulks
Copy link
Collaborator

@echosa opening a fresh issue to link to this one - will link here - seeing some additional items that we're running thru, and am linking to a separate repro issue that we're tracking for the blocklists.

@echosa
Copy link
Contributor

echosa commented Mar 1, 2017

Perfect. I'll keep an eye out for the new one. :-) Thanks!

@lukemulks
Copy link
Collaborator

@echosa here is the new one #7432
Breaks out timelines and other related issues. Thanks!

@quesada
Copy link

quesada commented Dec 16, 2017

I still see ads. Linux.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests