Implement HSTS partitioning

[goodov]

Strict-Transport-Security header handler doesn't support partition logic unlike Expect-CT: the first is not keyed by NIK, the second is keyed (and controlled by PartitionExpectCTStateByNetworkIsolationKey).

Need to figure out if this is intentional or WIP. Chromium doesn't partition HSTS.

[diracdeltas]

cc @pes10k

[pes10k]

https://bravesoftware.slack.com/archives/C01LKMP6X36/p1634540342137000?thread_ts=1632895144.098500&cid=C01LKMP6X36 for context

[goodov]

There are still a couple areas in the network service that still aren't partitioned (cert verifier, auth cache, HSTS, off the top of my head) that we need to figure out, but they should all work fine with the partitioning features enabled.

https://groups.google.com/a/chromium.org/g/net-dev/c/1mDd-OLnHrM/m/stqRX8VKBAAJ

It's not supported and doesn't look like it's going to be implemented in a near future.

There is an unsolved concern about uncontrolled HSTS/Expect-CT storage growth over time which can be increased with NIK-partitioning (a discussion can be found here). Expect-CT-partitioning was still implemented regardless of the concern, but HSTS has kinda bad storage-related issue: its uncontrolled growth can be abused, see https://bugs.chromium.org/p/chromium/issues/detail?id=699461.

[pes10k]

Could we (or they) cap the size of the HSTS store by just caping the size of storage, and evicting rarely used items? I appreciate there is a tension there with the guarantees of HSTS, but it sounds like that already exists if the store gets too big.

In Brave's case though, since we'd be aligning the 3p network cache with the general 3PES DOM Storage policy, it seems like we wouldn't hit the problems upstream is having.

Are either of those routes forward for us?

[goodov]

Could we (or they) cap the size of the HSTS store by just caping the size of storage, and evicting rarely used items? I appreciate there is a tension there with the guarantees of HSTS, but it sounds like that already exists if the store gets too big.

I think it might be seen as a security downside if we will evict old records when they are not expired yet.

In Brave's case though, since we'd be aligning the 3p network cache with the general 3PES DOM Storage policy, it seems like we wouldn't hit the problems upstream is having.

That's not exactly how it works regarding the HSTS/Expect-CT storage. The data stored in a separate JSON file which doesn't know anything about 1PES/3PES, so it won't work automagically.

Nevertheless, we can introduce our ephemeral approach and just keep the HSTS/Expect-CT data in-memory when required, i.e. we can make it to be partitioned and not persisted (to disk) when top_frame/requesting_frame combination require Ephemeral Storage. Although I'm not sure about a cleanup after a website close (can be tricky), but we can work it out.

WDYT?

[pes10k]

I think that sounds great, thanks!

[goodov]

related issues
#3419
#5936

[stephendonner]

@goodov mind putting up a test plan for QA, when you get a chance? Thanks! ❤️

[goodov]

Huh. Not sure if I can come up with a usable test case here. Let's just make sure that HSTS works in general and preload list also works for normal browsing. We know that HSTS partitioning works via privacytests.org (slack discussion: https://bravesoftware.slack.com/archives/C8MP8ME4C/p1653801850048449?thread_ts=1653586057.513899&cid=C8MP8ME4C).

Test cases to ensure HSTS works:

1. Navigate https://hsts.badssl.com/ in a default and incognito profile, ensure it works.
2. Navigate to http://brave.com/, ensure you're redirected to https version and you can see a redirect with Status Code: 307 Internal Redirect in DevTools for the http request (enable "Preserve log" checkbox in DevTools to keep all logs).

@goodov
for test case 2 in Win 11 x64, seeing Status Code: 304

[stephendonner]

Verified PASSED using

Brave	1.41.59 Chromium: 103.0.5060.42 (Official Build) nightly (x86_64)
Revision	de0d840bf9439c31bd86bf74f065c31fdf9b208d-refs/branch-heads/5060@{#667}
OS	macOS Version 12.5 (Build 21G5037d)

Case One: hsts.badssl.com

1. installed 1.41.59
2. launched Brave
3. loaded https://hsts.badssl.com/
4. confirmed it's green with an HSTS is working message

Case Two: brave.com

1. installed 1.41.59
2. launched Brave
3. opened Developer Tools
4. clicked on the Network panel
5. loaded http://brave.com/
6. confirmed it redirected to https://brave.com/ with a 307 Internal Redirect

[MadhaviSeelam]

Verification PASSED using

Brave | 1.41.65 Chromium: 103.0.5060.42 (Official Build) nightly (64-bit)
-- | --
Revision | de0d840bf9439c31bd86bf74f065c31fdf9b208d-refs/branch-heads/5060@{#667}
OS | Windows 11 Version 21H2 (Build 22000.739)

Case One: hsts.badssl.com - PASSED

1. installed 1.41.65
2. launched Brave
3. loaded https://hsts.badssl.com/ in default window
4. confirmed it's green with an HSTS is working message
5. confirmed worked as expected in https://hsts.badssl.com/ in private/incognito window

default_window	private_window

Case Two: brave.com

1. installed 1.41.65
2. launched Brave
3. opened Developer Tools
4. clicked on the Network panel
5. loaded http://brave.com/
6. confirmed it redirected to https://brave.com/ with a 307 Internal Redirect

[stephendonner]

Verified PASSED using

Brave	1.41.77 Chromium: 103.0.5060.53 (Official Build) beta (64-bit)
Revision	a1711811edd74ff1cf2150f36ffa3b0dae40b17f-refs/branch-heads/5060@{#853}
OS	Linux

Case One: hsts.badssl.com

Steps

1. installed 1.41.77
2. launched Brave
3. loaded https://hsts.badssl.com/
4. confirmed it's green with an HSTS is working message

Case Two: brave.com

Steps

1. installed 1.41.77
2. launched Brave
3. opened Developer Tools
4. clicked on the Network panel
5. loaded http://brave.com/
6. confirmed it redirected to https://brave.com/ with a 307 Internal Redirect