Chrome establishes network connection despite filter

[stoically]

Hi,

thanks for your awesome work on uBlock Origin!

How to reproduce

• Start Wireshark
• Set Wireshark Display Filter to ssl.handshake.extensions_server_name contains "api.mixpanel.com" || dns.qry.name == "api.mixpanel.com"
• Start capturing packets with Wireshark
• Navigate to https://www.twitch.tv/, Login/Register
• Open DevTools, Network Tab, Filter for api.mixpanel.com
• Click "Legacy Site" in the Dropdown Menu top right

Actual Behavior

DevTools show Status (failed) | Type xhr for the request to the domain api.mixpanel.com, Wireshark captures DNS and TLSv1.2 packets: actual network connection is made. (refresh site to see xhr/packets again)

Expected Behavior

Wireshark doesn't capture packets: no network connection.

Your settings

• OS/version: Windows10, Ubuntu 16.04.3 LTS & Ubuntu 17.10
• Browser/version: Chrome Version: 62.0.3202.89 (Official Build) (64-bit)
• uBlock Origin version: v1.14.18

Your filter lists

Default

Your custom filters (if any)

None

Notes

Firefox doesn't show this behavior, Wireshark doesn't capture packets.

Tested with fresh Chrome Profile (no other extensions) and uBlock Origin (no custom settings).

Unfortunately I wasn't able to narrow the issue further down. If I can provide further information please let me know.

Best regards

Edit: Couldn't always reproduce while not logged in or on the "new" (not "Legacy") twitch site. Changed the steps to reproduce accordingly.

[gorhill]

Did you disable "Disable pre-fetching" in Settings? (it's enabled by default).

Also, please mind whether the network request originate from behind-the-scene, these are not blocked by default by uBO.

If all seems fine uBO-side, I suggest you report the issue to Chromium devs, uBO can't do more than what the API allows it to do.

[stoically]

Thanks for your fast response!

Tested with default settings, so "Disable pre-fetching" is active.

Checked behind-the-scene requests (the request is not listed there) and even blocked them all, but the behavior is the same, the network connection is still made.

So I guess this is to be considered a problem with Chrome and not uBO. Do you maybe have any tips on how to narrow the issue down, so that it's not needed to install uBO and Login to a specific site to reproduce? I would absolutely understand if explaining that would take to long, tho any pointers are appreciated. I'll try my best to report to Chromium anyway.

[gorhill]

I just looked using chrome://net-internals/#events, here is the result:

This seems to happen only with the "Legacy" option on Twitch. So it does seem that something is happening which escapes the webRequest API. This is not just mixpanel.com, I also see googletagservices.com, googleadservices.com, etc.

[gorhill]

I disabled javascript using the browser's own javascript blocking tool, and I blocked everything on twitch.tv using a global * * * block rule in uBO. There was still a connection to mixpanel.com when I force-reloaded the (blank) twitch.tv page.

Here is what I found finally.

With "Legacy" mode enabled, in view-source:https://www.twitch.tv/ you can see this:

<link href='www.twitch.tv' rel='preconnect'>
<link href='https://player.twitch.tv' rel='preconnect'>
<link href='web-cdn.ttvnw.net' rel='preconnect'>
<link href='https://www.googletagservices.com' rel='preconnect'>
<link href='https://cdn.mxpnl.com' rel='preconnect'>
<link href='https://partner.googleadservices.com' rel='preconnect'>
<link href='https://spade.twitch.tv' rel='preconnect'>
<link href='https://sb.scorecardresearch.com' rel='preconnect'>
<link href='https://cdn.krxd.net' rel='preconnect'>
<link crossorigin='anonymous' href='https://api.mixpanel.com' rel='preconnect'>
<link href='https://static-cdn.jtvnw.net' rel='preconnect'>
<link href='https://pubster.twitch.tv' rel='preconnect'>
<link href='https://client-event-reporter.twitch.tv' rel='preconnect'>

Without "Legacy" mode, it's the following:

<link rel="preconnect" href="https://api.twitch.tv/">
<link rel="preconnect" href="https://gql.twitch.tv/">
<link rel="preconnect" href="https://player.twitch.tv/">
<link rel="preconnect" href="https://static-cdn.jtvnw.net/">
<link rel="preconnect" href="https://static.twitchcdn.net/">
<link rel="preconnect" href="https://cvp.twitch.tv/">
<link rel="preconnect" href="https://custos.twitchcdn.net/">

I think his narrows the issue: In Chromium, a rel="preconnect" directive overrides a user's own browser "Use a prediction service to load pages more quickly" setting.

I consider this a serious issue in the browser, a remote server should never ever override a user's own user agent privacy setting.

[stoically]

Interesting, thanks for digging into this! The only official documentation I was able to find on the "prediction service" setting was in the Chrome Help.

Browsers use an IP address to load a webpage. When you visit a webpage, Chrome
can look up the IP addresses of all the page's links and load the ones you might navigate
to next. If you turn this setting on, websites and any embedded content that are
pre-loaded may set and read their own cookies as if you had visited them, even if you don't.

From how I understand it, this only refers to prefetch, dns-prefetch and prerender (however this works now, officially its deprecated, but internally it's still used somehow according to chrome://net-internals/#prerender). But I might be totally mistaken here.

Do you maybe have other resources that state directly which pre* methods should be deactivated when "prediction service" is disabled? Should I open a ticket on the Bugtracker suggesting to include (or fix) preconnect in that setting?

[stoically]

For reference, I've opened a Chromium Issue here.

[stoically]

Turned out that Chromium 51 "unbundled" the ability to disable preconnect, prefetch and dns-prefetch with the chrome.privacy.network.networkPredictionEnabled Extensions Setting and with the Use a prediction service to load pages more quickly user advanced setting in coordination with the privacy team. This was discussed and changed in Chromium Issue 589369.

To get full functionality for Extensions back I've opened Chromium Issue 786038 as feature request.

Personally I switched back to Firefox because I deem this completely unacceptable.

[sergeevabc]

I am somewhat puzzled on a related issue, could you help? Firefox allows to enable link-prefetching (network.prefetch-next) and dns-prefetching (network.dns.disablePrefetch) in about:config. The latter one could speed things up, but would it work, when “disable prefetching” is turned on in uBlock?

[uBlock-user]

To get full functionality for Extensions back I've opened Chromium Issue 786038 as feature request.

Chrome devs archived the issue.