Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement cosmetic filtering #3303

Merged
merged 9 commits into from
Dec 14, 2019
Merged

Conversation

antonok-edm
Copy link
Collaborator

@antonok-edm antonok-edm commented Aug 29, 2019

Submitter Checklist:

Closes brave/brave-browser#5381
Depends on brave/adblock-rust-ffi#3

Test Plan:

Majority of tests, covering fine-grained API behavior, are implemented in unit tests within https://github.com/brave/adblock-rust.

FFI behavior is validated by tests in https://github.com/brave/adblock-rust-ffi.

End-to-end browser behavior is verified by preloading custom cosmetic rules into the adblock-rust engine, loading a test page, and verifying that DOM elements show as expected using getComputedStyle.

Reviewer Checklist:

  • New files have MPL-2.0 license header.
  • Request a security/privacy review as needed.
  • Adequate test coverage exists to prevent regressions
  • Verify test plan is specified in PR before merging to source

After-merge Checklist:

  • The associated issue milestone is set to the smallest version that the
    changes has landed on.
  • All relevant documentation has been updated.

@antonok-edm antonok-edm self-assigned this Aug 29, 2019
@antonok-edm antonok-edm added feature/shields dependencies Pull requests that update a dependency file enhancement labels Aug 29, 2019
@antonok-edm antonok-edm mentioned this pull request Aug 29, 2019
32 tasks
@antonok-edm antonok-edm force-pushed the cosmetic-filtering-frontend branch 2 times, most recently from 8421e0d to 923c2d4 Compare August 30, 2019 04:16
Copy link
Member

@petemill petemill left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great. On my first review pass, I was attempting to understand the code architecture via the method / API naming and calls. I couldn't do that so I had to read each function implementation to get a basic understanding. I recommend changing the naming to become more understandable. The API method descriptions in the JSON are clearer and could be a good starting point. Here's some suggestions:

  • chrome.braveShields.{get,set}CosmeticFilteredElementsAsync{get,set}CosmeticFilteringControlTypeAsync

  • chrome.braveShields.hostnameCosmeticResourcesgetCosemeticStylesheetForHostname
    Though perhaps ...StyleRules... is less confusing since it returns a string and not a javascript StyleSheet object. Also perhaps create / generate is clearer than get? Behind the scenes is it fetching something or doing some heavy generation work?

  • chrome.braveShields.classIdStylesheetgetCosmeticStyleSheetForElements
    Same considerations as for hostnameCosmeticResources, perhaps ...ForElementSelectors is clearer as it doesn't accept actual DOM Elements, but that's quite long.

cssOrigin: 'user',
runAt: 'document_start'
})
})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

) should be on separate line since { is separate line to (

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, not sure what you mean? The first }) matches to ...insertCSS({, the second matches to ...Stylesheet(... => {.

@antonok-edm
Copy link
Collaborator Author

  • chrome.braveShields.hostnameCosmeticResourcesgetCosemeticStylesheetForHostname
    Though perhaps ...StyleRules... is less confusing since it returns a string and not a javascript StyleSheet object. Also perhaps create / generate is clearer than get? Behind the scenes is it fetching something or doing some heavy generation work?

This actually returns 3 distinct things dependent on the hostname: the stylesheet in string format, a list of exceptions to generic cosmetic rules, and a script to be injected in the page. I figured just calling it a stylesheet might be confusing. I'd lean towards ...StyleRules..., I guess. It is querying saved rules from hashmaps according to the domain and assembling them behind the scenes.

By the way, do you know if there is a convenient way to build a Javascript StyleSheet object as part of the API? That sounds like it would make more sense, although it seems like it would be hard to do on the C++ side / without another layer of wrapping functions.

  • chrome.braveShields.classIdStylesheetgetCosmeticStyleSheetForElements
    Same considerations as for hostnameCosmeticResources, perhaps ...ForElementSelectors is clearer as it doesn't accept actual DOM Elements, but that's quite long.

Perhaps getCosmeticStylesheetForSelectors? Although, again this is "generated/created" in a similar way.

@antonok-edm antonok-edm force-pushed the cosmetic-filtering-frontend branch 2 times, most recently from 93877bb to 25d6c39 Compare September 3, 2019 18:17
@antonok-edm
Copy link
Collaborator Author

Also just added a toggle in settings corresponding to the new shields panel option.

@szaimen
Copy link

szaimen commented Oct 22, 2019

Any update here?

@antonok-edm
Copy link
Collaborator Author

@szaimen Still blocked on code reviews for brave/adblock-rust#50 unfortunately

@szaimen
Copy link

szaimen commented Nov 3, 2019

@antonok-edm any update here now that brave/adblock-rust#50 is merged?

@antonok-edm
Copy link
Collaborator Author

@szaimen #3419 and brave/brave-core-crx-packager#74 are required to fully support brave/adblock-rust#50 in the browser. Cosmetic filtering will be unblocked once that's sorted.

app/brave_generated_resources.grd Outdated Show resolved Hide resolved
exceptions: genericExceptions
})

chrome.tabs.executeScript(tabId, {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we call this only when injectedScript is empty.
Also checking with @diracdeltas about if we can do this and trust the different lists we use. Are these generic scripts or only in a certain guaranteed format to hide things? Could you give an example of an ABP filter syntax rule that uses it?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The scripts come from the same set of resources as for network redirects.
There are a few extra parameterized templated scriptlets, which were included in the same security review.

Here's an example of a scriptlet injection rule with 1 argument, directly from uBlock's filters:

computerbild.de##+js(abort-on-property-read.js, Date.prototype.toUTCString)

The argument (Date.prototype.toUTCString) is interpolated into this function at the location of the first occurrence of {{1}}, and the resulting function would be used in this executeScript call.


function getCurrentURL () {
return window.location.hostname
}

const getClassesAndIds = function (addedNodes: Element[]) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @bridiver is there an observer you know of that we can use in c++ to get the list of IDs and class names on all elements instead of getting them here? It seems a little expensive to do on each page load.

How the API works, is you pass in any class names or IDs that might be used, and then the adblock engine determines the stylesheet to use for blocking.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what uBlock Origin does and I haven't found the performance hit to be noticeable in practice, but it would be nice to take advantage of native APIs if possible.

components/brave_shields/browser/ad_block_base_service.h Outdated Show resolved Hide resolved
classIdBuffer.classes.push(...classes)
classIdBuffer.ids.push(...ids)
} else {
chrome.runtime.sendMessage({
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In practice, do you know how often classIdStylesheet gets hit for pages? Just making sure we're not inserting like 1000 stylesheets. I think maybe we should be debouncing these calls.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It depends entirely on how often the site makes mutations to the page. I can try on a few different ones if you'd like specific examples.
Note that this is already batched on a per-mutation basis - i.e. if a node tree is added, it's only called once for all the classes and ids in the entire tree.

@antonok-edm antonok-edm force-pushed the cosmetic-filtering-frontend branch 2 times, most recently from d3a2595 to cab4209 Compare November 20, 2019 18:58
Copy link
Member

@petemill petemill left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM apart from the small redux point I raised. Let's get CI passing or investigate the issue. Great work!

DEPS Outdated
@@ -1,7 +1,7 @@
use_relative_paths = True

deps = {
"vendor/adblock_rust_ffi": "https://github.com/brave/adblock-rust-ffi.git@89127a30655eaf54cf73794309846084ea8b91b9",
"vendor/adblock_rust_ffi": "https://github.com/brave/adblock-rust-ffi.git@6dad8c1b47c1b866fa1b1c19e36da6eb5797a524",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before this gets merged please merge this:
brave/adblock-rust-ffi#7

And then update this changeset ref. As I only want to commit the ref here for things in master on the ffi repo.

Copy link
Member

@bbondy bbondy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved, just one blocking note about the changeset 👍

@antonok-edm
Copy link
Collaborator Author

antonok-edm commented Dec 12, 2019

@petemill

As an aside, is there anywhere documented as to what the various cosmetic filter properties are? In other words, what 'exceptions' are, as opposed to either selectors for hiding, custom CSS rules, custom JS script and whatever else is contained.

Most of how this works is explained in doc comments from adblock-rust. I just double checked and there are a few important data structures that could use some additional explanation, so I'll follow up there as well.

edit: brave/adblock-rust#61

antonok-edm and others added 9 commits December 13, 2019 13:57
return a single script for cosmetic scriptlet injection
mutation observer applied only if rule exists

notes

change order

mutation observer WIP

browser tests

isIdempotent

isIdempotent

mutation observer

mutationObserver

mutation observer

mutationObserver

mutation observer commented

mutation observer for anton

cleanup comments
Occasionally, chrome.tabs.insertCSS fails when triggered from
chrome.webNavigation events. It cannot work unless some page content is
already loaded.

See https://bugs.chromium.org/p/chromium/issues/detail?id=331654#c15
@bridiver
Copy link
Collaborator

Jenkins is failing because of some glibc error that does not appear to be related to this PR

@bridiver
Copy link
Collaborator

I'm approving this as ok to merge because @antonok-edm has verified the build on linux

@antonok-edm antonok-edm merged commit fe9f891 into master Dec 14, 2019
@antonok-edm antonok-edm deleted the cosmetic-filtering-frontend branch December 14, 2019 18:14
bridiver added a commit that referenced this pull request Dec 18, 2019
bsclifton added a commit that referenced this pull request Dec 18, 2019
This reverts commit fe9f891, reversing
changes made to 4e6ba95.

Was causing problems specifically on Linux because of the Rust dependency.
This should fix brave/brave-browser#7055
This will un-fix brave/brave-browser#5381
@bsclifton bsclifton added this to the 1.4.x - Nightly milestone Dec 18, 2019
@bsclifton
Copy link
Member

Reverted with #4255

Note: when this was merged, milestone should have been set @antonok-edm (ex: set milestone to 1.4, set milestone for issue also). I re-opened the issue and left it without a milestone

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file enhancement feature/shields
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement cosmetic blocking [tracking]
7 participants