Provide a Brave-specific User Agent for some sites

[bsclifton]

Updated description

There should be a way to share a Brave-specific User Agent

We currently do this with our "site hacks":
https://github.com/brave/brave-core/blob/66ceaf3491778f7d9a62f215c8fb47b318c9b181/browser/net/brave_site_hacks_network_delegate_helper.cc#L62-L76

User Agent example format:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_2) AppleWebKit/537.36 (KHTML, like Gecko) Brave Chrome/79.0.3945.117 Safari/537.36

This user agent is only shared with a few hard-coded sites, current list found here:
https://github.com/brave/brave-core/blob/4621aa3f71758f3bd890f381f5774d91a0338b2e/common/shield_exceptions.cc#L17-L29

The scope for this issue would be to expand to our list to be dynamic (remove hard-coded entries) and match the list we provide for our supported publishers (already fetched on browser launch).
https://laptop-updates.brave.com/promo/custom-headers

Original issue description

Carried over from brave/browser-laptop#3693
Related #4641

Brave should show some bravery of its own and own up to having its own User-Agent. I'm aware of and acknowledge this issue, but without identifying itself – brave will be counted in online statistics as just Chrome. Who knows? Maybe Brave will be more popular than Chrome one day, but website owners will not be prompted to sit up and take notice of Brave if they can't identify any traffic from the browser and its users.

The original issue report is also misleading. Of course the User-Agent would be unique in that test if no one else yet have run the test. With more users, it's uniqueness for tracking will be reduced over time.

• Original implementation and test plan can be found in Customize UA strings brave-core#2442
• This implementation was reverted with Revert changes to user agent after experiencing webcompat issues in Dev/Nightly brave-core#2583

We may need these initial overrides to start:
UA overrides in Edge: edge://compat/useragent

[tildelowengrimm]

As a general policy, we don't want the user-agent string to make Brave users much more distinctive. Our general ballpark plan is to switch to a Brave UA when users on that platform will have an anonymity set of about a million.

[grahamperrin]

FWIW, a couple of cases where a distinctive string might be problematic:

• Discord
• Slack

Related, in the Waterfox area:

• https://www.reddit.com/r/waterfox/comments/9drjqx/user_agent/e66hdxs/
• https://www.reddit.com/r/waterfox/comments/9drjqx/user_agent/e66h85p/

[sairam]

From the results I see for the browser - https://panopticlick.eff.org/results?aat=1#fingerprintTable

The UserAgent string displays the following:

1. OS with version
2. make of the processor

A simpler way to customize this User Agent string to mask this information could be a starting point for customization.

[bbondy]

Updated the title. We feel like we have a big enough user base now to go ahead with adding Brave to the end of the User-Agent.

So instead of:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Safari/537.36

We'd show:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Safari/537.36 Brave

[da2x]

@bbondy for a slew of historical reasons the Brave component should have a version number to be compatible with existing User-Agent parsing libraries (and regex soups). Some strict profiles of web application firewalls (things like mod_security) block unusually looking User-Agents; like UAs with unversioned components. You can either use Brave’s own marketing version number, or just copy the major version or full version from the Chrome component.

E.g. Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Safari/537.36 Brave/74

[bbondy]

Thanks @da2x we were considering just doing Brave/1.0 in a side discussion but maybe the Chrome version is better.

[Fornax96]

Using Brave/1.0 will probably break a few very badly written web apps. But I think it might be worth it in order to stop all this User Agent madness. Having five browser names and version numbers in your user agent is completely unnecessary nowadays (and humiliating too).

+1 for Brave/1.0 so we can begin to end the user agent war.

[da2x]

@bbondy More reasons to copy the Chr version: HTTP/2 uses header compression and repeated [version] strings compress better than unique ones. 😋 It also doesn't leak any additional data.

@Fornax96 Have a look at the never ending battle over this in the WebCompat database. It's not that easy. Product/Version would be technically nice but it doesn't work. Too many services are completely unmonitored and unmaintained, and require clients to deal with the legacy of the web. You still lose access to portions of the web without the Mozilla/5.0 prefix.

That being said, Brave could probably go with just Brave/Version on HTTP2. At least as an experiment.

[Fornax96]

I like the HTTP/2 idea.

Brave could also add a user agent switcher like Opera has. There will always be some bad seeds left among web developers. For instance a while ago I had to use a user agent switcher in chromium because Whatsapp locked me out.

EDIT: Actually it turns out whatsapp still blocks users based on their user agent. This is what happens when I change my agent to Brave/1.0

I sent them an email about it a year ago but they never replied.