Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add cookie consent notice blocking P3A metrics #25568

Closed
DJAndries opened this issue Sep 22, 2022 · 4 comments · Fixed by brave/brave-core#15187
Closed

Add cookie consent notice blocking P3A metrics #25568

DJAndries opened this issue Sep 22, 2022 · 4 comments · Fixed by brave/brave-core#15187
Assignees
@DJAndries
Labels
feature/shields The overall Shields feature in Brave. features/P3A OS/Android Fixes related to Android browser functionality OS/Desktop QA Pass - Android ARM QA Pass-Win64 QA/Yes release-notes/exclude
Milestone
1.45.x - Release

Comments

@DJAndries
Copy link
Collaborator

DJAndries commented Sep 22, 2022 •
edited
Loading

Description

Add two P3A questions for cookie consent notice blocking settings.

"If you have viewed the cookie consent block prompt, how did you react?"
Brave.Shields.CookieListPrompt

  1. Have not seen the prompt
  2. Have seen the prompt, but did not react
  3. Clicked "No thanks" on the prompt
  4. Clicked "Block cookie notices" on the prompt

"Do you have cookie consent notice blocking enabled?"
Brave.Shields.CookieListEnabled

  1. No
  2. Yes
@DJAndries DJAndries added feature/shields The overall Shields feature in Brave. QA/Yes release-notes/exclude OS/Android Fixes related to Android browser functionality features/P3A OS/Desktop labels Sep 22, 2022
@DJAndries DJAndries self-assigned this Sep 22, 2022
@DJAndries DJAndries mentioned this issue Sep 23, 2022
Merged
25 tasks
@brave-builds brave-builds added this to the 1.46.x - Nightly milestone Sep 26, 2022
@kjozwiak kjozwiak mentioned this issue Sep 29, 2022
Closed
@deeppandya deeppandya mentioned this issue Oct 5, 2022
Merged
7 tasks
@kjozwiak
Copy link
Member

The above requires 1.45.104 or higher for 1.45.x verification 👍

@kjozwiak
Copy link
Member

Labelling as QA/Blocked as we'll need to wait for #26078 to be fixed and uplifted into 1.45.x. We'll also need to enable BraveAdblockCookieListOptInStudy on BETA and Release before we can verify the above as we'll be enabling this feature via Grffin on both Desktop & Android.

@Uni-verse @srirambv I can run through this as I have the most context/been working on this for a while.

@GeetaSarvadnya
Copy link

GeetaSarvadnya commented Oct 21, 2022 •
edited
Loading

Verification PASSED on

Brave | 1.45.109 Chromium: 107.0.5304.62 (Official Build) (64-bit)
-- | --
Revision | 1eec40d3a5764881c92085aaee66d25075c159aa-refs/branch-heads/5304@{#942}
OS | Windows 10 Version 21H2 (Build 19044.2130)

Brave.Shields.CookieListPrompt

Brave.Shields.CookieListPrompt = 0_PASSED
  1. Clean profile 1.46.x
  2. Open brave://local-state file
  3. Ensured that the metrics Brave.Shields.CookieListPrompt is NOT shown
Brave.Shields.CookieListPrompt = 1_PASSED
Case 1: Click on close button on cookie consent block prompt
  1. Clean profile 1.45.109
  2. Enable the flag "Show an opt-in bubble for the "Easylist-Cookie List" filter" via brave://flags
  3. Relaunch the browser
  4. Ensure that the cookie consent block prompt is shown
  5. Click on the x or close button in the popup
  6. Open brave://local-state file and ensured that the metrics response value for the metrics Brave.Shields.CookieListPrompt is 1

image

Case 2: Open a new NTP when cookies consent modal popup prompt
  1. Clean profile 1.45.109
  2. Enable the flag "Show an opt-in bubble for the "Easylist-Cookie List" filter" via brave://flags
  3. Relaunch the browser
  4. Ensure that the cookie consent block prompt is shown
  5. Open a new NTP and ensure cookies consent popup is dismissed
  6. Open brave://local-state file and ensured that the metrics response value for the metrics Brave.Shields.CookieListPrompt is 1

image

Case 3: Close the `cookie consent block` modal popup tab
  1. Clean profile 1.45.109
  2. Open any site in NTP
  3. Enable the flag "Show an opt-in bubble for the "Easylist-Cookie List" filter" via brave://flags
  4. Relaunch the browser
  5. Ensure that the cookie consent block prompt is shown
  6. Close the active tab of cookie consent block prompt
  7. Open brave://local-state file and ensured that the metrics response value for the metrics Brave.Shields.CookieListPrompt is 1

image

Case 4: Switch the tab to other tab when cookies consent modal popup prompt
  1. Clean profile 1.45.109
  2. Open any site in NTP
  3. Enable the flag "Show an opt-in bubble for the "Easylist-Cookie List" filter" via brave://flags
  4. Relaunch the browser
  5. Ensure that the cookie consent block prompt is shown
  6. Switch the tab to a site ( opened in step 2)
  7. Open brave://local-state file and ensured that the metrics response value for the metrics Brave.Shields.CookieListPrompt is 1

image

Brave.Shields.CookieListPrompt = 2_PASSED
  • Confirmed that the response value for the metrics Brave.Shields.CookieListPrompt is 2 when user clicks on the "No thanks" button on the cookie consent block prompt

image

Brave.Shields.CookieListPrompt = 3_PASSED
  • Confirmed that the response value for the metrics Brave.Shields.CookieListPrompt is 3 when user clicks on the "Block cookie notices" button on the cookie consent block prompt

image

Brave.Shields.CookieListEnabled

Brave.Shields.CookieListEnabled = 0_PASSED
Case 1: Have not enabled the flag "Easylist-Cookie List"_PASSED
  1. Clean profile 1.45.109
  2. Open brave://local-state file
  3. Ensured that the metrics Brave.Shields.CookieListEnabled response value is 0

image

Case 2: Enabled the flag "Easylist-Cookie List" and clicked on Close button_PASSED
  1. Clean profile 1.45.109
  2. Open brave://flags and enable the flag Show an opt-in bubble for the "Easylist-Cookie List" filter
  3. Relaunch the browser
  4. Confirmed that the cookies consent prompt is shown
  5. Click on close (x) button
  6. Open brave://local-state file
  7. Ensured that the metrics Brave.Shields.CookieListEnabled response value is 0 and Brave.Shields.CookieListPrompt response value is 1

image

Case 3: Enabled the flag "Easylist-Cookie List" and closed the cookies consent tab_PASSED
  1. Clean profile 1.45.109
  2. Open brave://local-state
  3. Open brave://flags and enable the flag Show an opt-in bubble for the "Easylist-Cookie List" filter
  4. Relaunch the browser
  5. Confirmed that the cookies consent prompt is shown
  6. Close the cookies consent tab
  7. Reload brave://local-state file
  8. Ensured that the metrics Brave.Shields.CookieListEnabled response value is 0 and Brave.Shields.CookieListPrompt response value is 1

image

Case 4: Enabled the flag "Easylist-Cookie List" and switch to a tab other than the cookies consent tab_PASSED
  1. Clean profile 1.45.109
  2. Open brave://local-state
  3. Open brave://flags and enable the flag Show an opt-in bubble for the "Easylist-Cookie List" filter
  4. Relaunch the browser
  5. Confirmed that the cookies consent prompt is shown
  6. Switch to a tab other than the cookies consent tab
  7. Reload brave://local-state file
  8. Ensured that the metrics Brave.Shields.CookieListEnabled response value is 0 and Brave.Shields.CookieListPrompt response value is 1

image

Brave.Shields.CookieListEnabled = 1_PASSED
  1. Clean profile 1.45.109
  2. Open brave://flags and enable the flag Show an opt-in bubble for the "Easylist-Cookie List" filter
  3. Relaunch the browser
  4. Confirmed that the cookies consent prompt is shown
  5. Click on the "Block cookie notices" on the prompt
  6. Reload brave://local-state file
  7. Confirmed that the metrics Brave.Shields.CookieListEnabled response value is 1 when user clicks on the "Block cookie notices"on the prompt and the metrics Brave.Shields.CookieListPrompt response value is 3
  8. Confirmed that the Easylist - Cookie List is enabled in brave://settings/shields/filters when user clicks on the "Block cookie notices" on the prompt
Example Example
image image

@kjozwiak
Copy link
Member

kjozwiak commented Oct 24, 2022 •
edited
Loading

Verification PASSED on Samsung S10+ running Android 12 using the following build(s):

Brave | 1.45.111 Chromium: 107.0.5304.62 (Official Build) (32-bit)
--- | ---
Revision | 1eec40d3a5764881c92085aaee66d25075c159aa-refs/branch-heads/5304@{#942}
OS | Android 12; Build/SP1A.210812.016

Brave.Shields.CookieListPrompt

Brave.Shields.CookieListPrompt = 0_PASSED
  1. Clean 1.45.111 Chromium: 107.0.5304.62 profile
  2. Open brave://local-state
  3. Ensured that the Brave.Shields.CookieListPrompt metric is NOT being shown/displayed

Screenshot_20221024-102527_Brave

Brave.Shields.CookieListPrompt = 1_PASSED
Case 1: Click on close button on cookie consent block prompt

This isn't a valid case on Android as there's no X via the cookie consent panel/notification.

Case 2: Open a new NTP when cookies consent modal popup prompt
  1. Clean 1.45.111 Chromium: 107.0.5304.62 profile
  2. Enable brave://flags#brave-adblock-cookie-list-opt-in via brave://flags
  3. Relaunch the browser
  4. Ensure that the cookie consent block prompt is shown after 5 refresh/loads
  5. Open a new NTP and ensure cookies consent popup is dismissed
  6. Open brave://local-state file and ensured that the metrics response value for the metrics Brave.Shields.CookieListPrompt is 1

Screenshot_20221024-102803_Brave

Case 3: Close the `cookie consent block` modal popup tab

This isn't a valid case on mobile/phone as you can't close a tab unless you go into the Tab View which closes the panel/notification. Ran through this particular case on a Samsung Tab S8 Ultra.

  1. Clean 1.45.111 Chromium: 107.0.5304.62 profile
  2. Open any site in NTP
  3. Enable brave://flags#brave-adblock-cookie-list-opt-in via brave://flags
  4. Relaunch the browser
  5. Ensure that the cookie consent block prompt is shown
  6. Close the active tab of cookie consent block prompt
  7. Open brave://local-state file and ensured that the metrics response value for the metrics Brave.Shields.CookieListPrompt is 1

Screenshot_20221024_115454_Brave

Case 4: Switch the tab to other tab when cookies consent modal popup prompt
  1. Clean 1.45.111 Chromium: 107.0.5304.62 profile
  2. Open any site in NTP
  3. Enable brave://flags#brave-adblock-cookie-list-opt-in via brave://flags
  4. Relaunch the browser
  5. Ensure that the cookie consent block prompt is shown after 5 refresh/loads
  6. Switch the tab to a site ( opened in step 2)
  7. Open brave://local-state file and ensured that the metrics response value for the metrics Brave.Shields.CookieListPrompt is 1

Screenshot_20221024-103648_Brave

Brave.Shields.CookieListPrompt = 2_PASSED

Confirmed that the response value for the metrics Brave.Shields.CookieListPrompt is 2 when user clicks on the "No thanks" button on the cookie consent block prompt as per the following:

Screenshot_20221024-104357_Brave

Brave.Shields.CookieListPrompt = 3_PASSED

Confirmed that the response value for the metrics Brave.Shields.CookieListPrompt is 3 when user clicks on the "Block cookie notices" button on the cookie consent block prompt as per the following:

Screenshot_20221024-104619_Brave

Brave.Shields.CookieListEnabled

Brave.Shields.CookieListEnabled = 0_PASSED
Case 1: Have not enabled the flag "Easylist-Cookie List"_PASSED
  1. Clean 1.45.111 Chromium: 107.0.5304.62 profile
  2. Open brave://local-state
  3. Ensured that the metrics Brave.Shields.CookieListEnabled response value is 0

Screenshot_20221024-104951_Brave

Case 2: Enabled the flag "Easylist-Cookie List" and clicked on Close button_PASSED

This isn't a valid case on Android as there's no X via the cookie consent panel/notification.

Case 3: Enabled the flag "Easylist-Cookie List" and closed the cookies consent tab_PASSED
  1. Clean 1.45.111 Chromium: 107.0.5304.62 profile
  2. Open brave://local-state
  3. Enable brave://flags#brave-adblock-cookie-list-opt-in via brave://flags
  4. Relaunch the browser
  5. Confirmed that the cookies consent prompt is shown
  6. Close the cookies consent tab
  7. Reload the brave://local-state tab
  8. Ensured that the metrics Brave.Shields.CookieListEnabled response value is 0 and Brave.Shields.CookieListPrompt response value is 1

Screenshot_20221024-105504_Brave

Case 4: Enabled the flag "Easylist-Cookie List" and switch to a tab other than the cookies consent tab_PASSED
  1. Clean 1.45.111 Chromium: 107.0.5304.62 profile
  2. Open brave://local-state
  3. Enable brave://flags#brave-adblock-cookie-list-opt-in via brave://flags
  4. Relaunch the browser
  5. Confirmed that the cookies consent prompt is shown
  6. Switch to a tab other than the cookies consent tab
  7. Reload the brave://local-state tab
  8. Ensured that the metrics Brave.Shields.CookieListEnabled response value is 0 and Brave.Shields.CookieListPrompt response value is 1

Screenshot_20221024-105504_Brave

Brave.Shields.CookieListEnabled = 1_PASSED
  1. Clean 1.45.111 Chromium: 107.0.5304.62 profile
  2. Enable brave://flags#brave-adblock-cookie-list-opt-in via brave://flags
  3. Relaunch the browser
  4. Confirmed that the cookies consent prompt is shown
  5. Click on the "Block cookie notices" on the prompt
  6. Reload brave://local-state file
  7. Confirmed that the metrics Brave.Shields.CookieListEnabled response value is 1 when user clicks on the "Block cookie notices"on the prompt and the metrics Brave.Shields.CookieListPrompt response value is 3
  8. Confirmed that the Easylist - Cookie List is enabled in brave://settings/shields/filters when user clicks on the "Block cookie notices" on the prompt
Example Example Example
Screenshot_20221024-110127_Brave Screenshot_20221024-110138_Brave Screenshot_20221024-110151_Brave

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@DJAndries DJAndries

Labels
feature/shields The overall Shields feature in Brave. features/P3A OS/Android Fixes related to Android browser functionality OS/Desktop QA Pass - Android ARM QA Pass-Win64 QA/Yes release-notes/exclude
Projects
None yet
Milestone
1.45.x - Release
Development

Successfully merging a pull request may close this issue.

Add P3A metrics for cookie consent notice blocking brave/brave-core
5 participants
@kjozwiak @DJAndries @LaurenWags @GeetaSarvadnya @brave-builds