Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/brave default rules #1264

Closed
wants to merge 75 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
7b4d93d
feat(mv3): :sparkles: Patching countly-sdk-web
whizzzkid Jul 12, 2023
f3305fe
feat(mv3): :sparkles: Implementing Custom Async Store.
whizzzkid Jul 12, 2023
83af58f
chore(mv3): :adhesive_bandage: Hooking everything up together.
whizzzkid Jul 12, 2023
d504c71
fix(mv3): Countly Patching + ignite-metrics@2.0.0
whizzzkid Jul 13, 2023
4fab16a
fix(mv3): :adhesive_bandage: Patching the Patch
whizzzkid Jul 13, 2023
f02d6a2
fix: tests
whizzzkid Jul 13, 2023
dbab08c
fix: lint
whizzzkid Jul 13, 2023
394b13b
fix(mv3): :recycle: Refactoring `supportsBlock` Checks.
whizzzkid Jul 18, 2023
cbb4ac6
fix(mv3): Regex Bug
whizzzkid Jul 18, 2023
5794aea
feat: Migrating blocking redirection test to observing redirection test
whizzzkid Jul 18, 2023
0b0af76
fix(mv3): :wrench: Fixing the mocha-setup.
whizzzkid Jul 18, 2023
0de2e89
fix(mv3): :recycle: Moving Setup Files.
whizzzkid Jul 19, 2023
1d46a57
fix(mv3): gateway-redirect tests now fixed.
whizzzkid Jul 19, 2023
44653de
fix: :adhesive_bandage: Patching error messages
whizzzkid Jul 19, 2023
64fcde7
fix(patch): countly-web-sdk
whizzzkid Jul 19, 2023
66b13dc
fix(patch): :pin: Pinning countly-web-sdk to 23.2.2
whizzzkid Jul 19, 2023
39af513
Merge branch 'fix/analytics' into feat/redirection-tests
whizzzkid Jul 19, 2023
bf98b8f
fix(mv3): :lipstick: Fixing Lint
whizzzkid Jul 19, 2023
0b8b308
Merge branch 'rc/3.0-mv3' into feat/redirection-tests
whizzzkid Jul 19, 2023
3b80b2a
Merge branch 'rc/3.0-mv3' into feat/redirection-tests
whizzzkid Jul 19, 2023
50797d0
feat: protocol-handler-redirection-tests
whizzzkid Jul 19, 2023
46fca05
feat: more tests fixed
whizzzkid Jul 19, 2023
8e3aee2
fix: More tests
whizzzkid Jul 19, 2023
83ff056
fix: lint fix
whizzzkid Jul 19, 2023
38d7b60
test: merge mocha-setup files (#1246)
SgtPooki Jul 25, 2023
4039f0f
Merge branch 'rc/3.0-mv3' into feat/redirection-tests
whizzzkid Jul 25, 2023
dd86afa
test: fix after merge
SgtPooki Jul 25, 2023
0e316a2
test: fix after merge
SgtPooki Jul 25, 2023
7b94054
fix: typerrors for localstorage
whizzzkid Jul 25, 2023
8d95a1f
fix: Updating test:functional_MV3 command.
whizzzkid Jul 26, 2023
9bbf271
fix: setup
whizzzkid Jul 26, 2023
5ce5de8
fix(test): Fixing tests
whizzzkid Jul 26, 2023
e5d42e0
fix(lint): Becuase Ofcourse
whizzzkid Jul 26, 2023
a260eb9
feat(test): scaffolding mv3 + mv2 calls in a single check.
whizzzkid Jul 27, 2023
eb8fea0
feat(test): unskipping and upgrading dnslink tests to mv3
whizzzkid Jul 27, 2023
cc109b0
feat(test): Upgrading workaround tests to MV3
whizzzkid Jul 27, 2023
441191e
fix(test): removing all skips with better checks.
whizzzkid Jul 27, 2023
5ac8417
fix(test): :recycle: Refactoring tests and removing redundant calls.
whizzzkid Jul 27, 2023
02042cb
fix(test): More Dryer
whizzzkid Jul 27, 2023
f3d4605
fix(test): one more
whizzzkid Jul 27, 2023
47d72f8
fix(test): :pencil2: Renaming isMv3TestingEnabled -> isManifestV3
whizzzkid Jul 27, 2023
239984d
fix(test): refactor expectNoRedirect
whizzzkid Jul 27, 2023
94cb141
fix(test): :recycle: Refactoring more.
whizzzkid Jul 27, 2023
e3e73c9
fix: replacing checks to undefined
whizzzkid Jul 27, 2023
80e7d51
fix: renaming expectNoRedirect -> ensureNoRedirect
whizzzkid Jul 27, 2023
7d1e97c
fix(test): Adding missing JSDoc
whizzzkid Jul 27, 2023
33b8062
fix(test): :shrug: how did this get removed.
whizzzkid Jul 28, 2023
ae23f4c
fix(test): :wastebasket: removed.
whizzzkid Jul 28, 2023
b77ab7b
fix(test): Suggestion
whizzzkid Jul 28, 2023
bb85dbb
Merge branch 'rc/3.0-mv3' into feat/redirection-tests
whizzzkid Jul 28, 2023
6fe39a3
fix(test): :adhesive_bandage: Bad Merge
whizzzkid Jul 28, 2023
e56bb6a
fix(test): sequential expectNoRedirect
lidel Aug 3, 2023
6bd8c3c
Update add-on/src/lib/redirect-handler/blockOrObserve.ts
whizzzkid Aug 4, 2023
7f68ab4
fix(mv3): :wrench: Modifying the default local redirect behaviour.
whizzzkid Aug 4, 2023
d94b9bb
Merge branch 'rc/3.0-mv3' into fix/default-rules
whizzzkid Aug 4, 2023
47ea944
Merge branch 'rc/3.0-mv3' into feat/redirection-tests
whizzzkid Aug 4, 2023
a817045
Merge branch 'rc/3.0-mv3' into fix/default-rules
whizzzkid Aug 4, 2023
045e660
fix(mv3): :wrench: Modifying the default local redirect behaviour.
whizzzkid Aug 4, 2023
f6561b9
Merge remote-tracking branch 'refs/remotes/origin/fix/default-rules' …
whizzzkid Aug 4, 2023
c020638
Merge branch 'feat/redirection-tests' into fix/default-rules
whizzzkid Aug 4, 2023
fc085b5
fix(mv3): :bug: Making rules less greedy
whizzzkid Aug 10, 2023
007f41f
fix(mv3): :sparkles: Dynamic Rules for subdomain gateways.
whizzzkid Aug 11, 2023
f18579b
fix(types): Adding ambient types for is-ipfs.
whizzzkid Aug 11, 2023
3e72d36
fix(test):
whizzzkid Aug 12, 2023
d36a282
fix(test): helper
whizzzkid Aug 12, 2023
6ee2d31
feat(mv3): less greedy rules
whizzzkid Aug 12, 2023
e592755
feat: Adding simpler regex for redirects from similar namespaces.
whizzzkid Aug 12, 2023
5c85d84
fix(lint): :rotating_light: Warnings
whizzzkid Aug 12, 2023
fca5fe2
feat(mv3): Better Default Rules (#1260)
whizzzkid Aug 15, 2023
832679d
Update add-on/src/lib/redirect-handler/blockOrObserve.ts
whizzzkid Aug 16, 2023
5e22cea
fix(docs): :pencil2: Adding comments
whizzzkid Aug 16, 2023
99e2d23
feat(mv3): brave: redirection to protocol handlers.
whizzzkid Aug 18, 2023
70f4824
Merge branch 'rc/3.0-mv3' into feat/brave-default-rules
whizzzkid Aug 27, 2023
83a6eda
Merge branch 'rc/3.0-mv3' into feat/brave-default-rules
whizzzkid Sep 7, 2023
d6fbcb2
fix(mv3): external node type
whizzzkid Sep 8, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 68 additions & 26 deletions add-on/src/lib/redirect-handler/blockOrObserve.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
import debug from 'debug'
import isIPFS from 'is-ipfs'
import browser from 'webextension-polyfill'
import { CompanionState } from '../../types/companion.js'
import { brave } from '../ipfs-client/brave.js'
import { IFilter, IRegexFilter, RegexFilter } from './baseRegexFilter.js'
import { CommonPatternRedirectRegexFilter } from './commonPatternRedirectRegexFilter.js'
import { NamespaceRedirectRegexFilter } from './namespaceRedirectRegexFilter.js'
Expand All @@ -24,13 +26,25 @@ interface regexFilterMap {
regexSubstitution: string
}

interface redirectHandlerInput {
originUrl: string
interface redirectPair {
originUrl: string,
redirectUrl: string
getPort: (state: CompanionState) => string
}

interface redirectRegexPair {
regexSubstitution: string,
regexFilter: string
}

interface redirectHandlerInput extends Omit<redirectPair, 'redirectUrl'> {
priority?: number
getRedirectUrl: (port: string) => string
getPort: (state: CompanionState) => string
}

type messageToSelfType = typeof GLOBAL_STATE_OPTION_CHANGE | typeof DELETE_RULE_REQUEST

interface messageToSelf {
type: messageToSelfType
value?: string | Record<string, unknown>
Expand Down Expand Up @@ -86,19 +100,25 @@ const savedRegexFilters: Map<string, regexFilterMap> = new Map()
const DEFAULT_LOCAL_RULES: redirectHandlerInput[] = [
{
originUrl: 'http://127.0.0.1',
redirectUrl: 'http://localhost',
getRedirectUrl: (port): string => `http://localhost:${port}/\\1/\\2`,
getPort: ({ gwURLString }): string => new URL(gwURLString).port
},
{
originUrl: 'http://[::1]',
redirectUrl: 'http://localhost',
getRedirectUrl: (port): string => `http://localhost:${port}/\\1/\\2`,
getPort: ({ gwURLString }): string => new URL(gwURLString).port
},
{
originUrl: 'http://localhost',
redirectUrl: 'http://127.0.0.1',
getRedirectUrl: (port): string => `http://localhost:${port}/\\1/\\2`,
getPort: ({ apiURL }): string => new URL(apiURL).port
}
},
...(brave ? [{
originUrl: 'http://localhost',
getRedirectUrl: (): string => `\\1://\\2`,
getPort: ({ gwURLString }): string => new URL(gwURLString).port,
priority: 1
}] : []) as redirectHandlerInput[]
]

/**
Expand Down Expand Up @@ -126,14 +146,26 @@ export function escapeURLRegex (str: string): string {
return str.replace(ALLOWED_CHARS_URL_REGEX, '\\$1')
}

/**
* Compute the namespace from the URL. This finds the first path segment.
* e.g. http://<gateway>/<namespace>/path/to/file/or/cid
*
* @param url string
*/
function computeNamespaceFromUrl (url: string): string {
const { pathname } = new URL(url)
// regex to match the first path segment.
return (/\/([^/]+)\//i.exec(pathname)?.[1] ?? '').toLowerCase()
}

/**
* Construct a regex filter and substitution for a redirect.
*
* @param originUrl
* @param redirectUrl
* @returns
*/
function constructRegexFilter ({ originUrl, redirectUrl }: IRegexFilter): IFilter {
function constructRegexFilter ({ originUrl, redirectUrl }: redirectPair): redirectRegexPair {
// the order is very important here, because we want to match the best possible filter.
const filtersToTryInOrder: Array<typeof RegexFilter> = [
SubdomainRedirectRegexFilter,
Expand Down Expand Up @@ -248,19 +280,19 @@ async function reconcileRulesAndRemoveOld (state: CompanionState): Promise<void>
if (rules.length === 0) {
// we need to populate old rules.
for (const [regexFilter, { regexSubstitution, id }] of savedRegexFilters.entries()) {
addRules.push(generateAddRule(id, regexFilter, regexSubstitution))
addRules.push(generateAddRule({ id, regexFilter, regexSubstitution }))
}
}

// make sure that the default rules are added.
for (const { originUrl, redirectUrl, getPort } of DEFAULT_LOCAL_RULES) {
for (const { originUrl, getRedirectUrl, getPort, priority } of DEFAULT_LOCAL_RULES) {
const port = getPort(state)
const regexFilter = `^${escapeURLRegex(`${originUrl}:${port}`)}\\/${defaultNSRegexStr}\\/${RULE_REGEX_ENDING}`
const regexSubstitution = `${redirectUrl}:${port}/\\1/\\2`
const regexSubstitution = getRedirectUrl(port)

if (!savedRegexFilters.has(regexFilter)) {
// We need to add the new rule.
addRules.push(saveAndGenerateRule(regexFilter, regexSubstitution))
addRules.push(saveAndGenerateRule({ regexFilter, regexSubstitution, priority }))
}
}

Expand All @@ -276,16 +308,20 @@ async function reconcileRulesAndRemoveOld (state: CompanionState): Promise<void>
* @param excludedInitiatorDomains - The domains that are excluded from the rule.
* @returns
*/
function saveAndGenerateRule (
regexFilter: string,
regexSubstitution: string,
excludedInitiatorDomains: string[] = []
): browser.DeclarativeNetRequest.Rule {
function saveAndGenerateRule ({
regexFilter,
regexSubstitution,
excludedInitiatorDomains = [],
priority
}: redirectRegexPair & {
excludedInitiatorDomains?: string[],
priority?: number
}): browser.DeclarativeNetRequest.Rule {
// We need to generate a random ID for the rule.
const id = Math.floor(Math.random() * 29999)
// We need to save the regex filter and ID to check if the rule already exists later.
savedRegexFilters.set(regexFilter, { id, regexSubstitution })
return generateAddRule(id, regexFilter, regexSubstitution, excludedInitiatorDomains)
return generateAddRule({ id, regexFilter, regexSubstitution, excludedInitiatorDomains, priority })
}

/**
Expand All @@ -296,15 +332,20 @@ function saveAndGenerateRule (
* @param excludedInitiatorDomains - The domains that are excluded from the rule.
* @returns
*/
export function generateAddRule (
export function generateAddRule ({
id,
priority = 10,
regexFilter,
regexSubstitution,
excludedInitiatorDomains = []
}: redirectRegexPair & {
id: number,
regexFilter: string,
regexSubstitution: string,
excludedInitiatorDomains: string[] = []
): browser.DeclarativeNetRequest.Rule {
priority?: number,
excludedInitiatorDomains?: string[]
}): browser.DeclarativeNetRequest.Rule {
return {
id,
priority: 1,
priority,
action: {
type: 'redirect',
redirect: { regexSubstitution }
Expand Down Expand Up @@ -340,7 +381,8 @@ export function generateAddRule (
* @returns {Promise<void>}
*/
export function addRuleToDynamicRuleSetGenerator (
getState: () => CompanionState): (input: redirectHandlerInput) => Promise<void> {
getState: () => CompanionState
): (input: redirectPair) => Promise<void> {
// setup listeners for the extension.
setupListeners({
[GLOBAL_STATE_OPTION_CHANGE]: async (): Promise<void> => {
Expand All @@ -359,7 +401,7 @@ export function addRuleToDynamicRuleSetGenerator (
})

// returning a closure to avoid passing `getState` as an argument to `addRuleToDynamicRuleSet`.
return async function ({ originUrl, redirectUrl }: redirectHandlerInput): Promise<void> {
return async function ({ originUrl, redirectUrl }: redirectPair): Promise<void> {
// update the rules so that the next request is handled correctly.
const state = getState()
const redirectIsOrigin = originUrl === redirectUrl
Expand Down Expand Up @@ -390,7 +432,7 @@ export function addRuleToDynamicRuleSetGenerator (
await browser.declarativeNetRequest.updateDynamicRules(
{
// We need to add the new rule.
addRules: [saveAndGenerateRule(regexFilter, regexSubstitution)],
addRules: [saveAndGenerateRule({ regexFilter, regexSubstitution })],
// We need to remove the old rules.
removeRuleIds
}
Expand Down
2 changes: 1 addition & 1 deletion add-on/src/lib/state.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
import { isHostname, safeURL } from './options.js'

export const offlinePeerCount = -1
export const POSSIBLE_NODE_TYPES = ['external', 'brave']
export const POSSIBLE_NODE_TYPES = ['external', 'external:brave']

/**
*
Expand Down
9 changes: 5 additions & 4 deletions add-on/src/options/page.js
Original file line number Diff line number Diff line change
Expand Up @@ -115,12 +115,13 @@ export default function optionsPage (state, emit) {
${resetForm({
onOptionsReset
})}
${supportsBlock
${supportsBlock()
? ''
: redirectRuleForm({
redirectRules: state.redirectRules,
emit
})}
redirectRules: state.redirectRules,
emit
})
}
</div>
`
}
10 changes: 5 additions & 5 deletions test/helpers/mv3-test-helper.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,11 @@ export function ensureCallRedirected ({
}): void {
if (isManifestV3) {
const [args] = browser.declarativeNetRequest.updateDynamicRules.firstCall.args
expect(args.addRules[0]).to.deep.equal(generateAddRule(
args.addRules[0].id,
MV3Expectation.origin + RULE_REGEX_ENDING,
MV3Expectation.destination
))
expect(args.addRules[0]).to.deep.equal(generateAddRule({
id: args.addRules[0].id,
regexFilter: MV3Expectation.origin + RULE_REGEX_ENDING,
regexSubstitution: MV3Expectation.destination
}))
} else {
expect(modifiedRequestCallResp.redirectUrl).to.equal(MV2Expectation)
}
Expand Down
Loading