Merge pull request #687 from ipfs-shipyard/feat/redirect-toggle-per-w…

…ebsite Per-site Redirect Opt-out This PR merges reworked redirect controls and adds opt-out per site - menu item in Active Tab section enables user to disable gateway redirect on current website - when clicked on regular site toggles redirect for current FQDN and all its subdomains - when clicked on /ipns/<fqdn>/ (DNSLink) website, toggles redirect for <fqdn> - after redirect preference changes for current website, the tab is reloaded - DNSLink websites are reloaded to with URL change between IPNS path and original URL - Redirect preference applies not only to request with FQDN of the active tab, but also to all subresource requests that have it in `originUrl` (Firefox) or `Referer` header (Chrome). This means toggle is useful to restore functionality of complex websites such as d.tube (which uses IPFS subresources from various subdomains) - reworked UI of browser action menu, labels no longer change, instead we introduce toggle-switch
ipfs · Mar 5, 2019 · 678c714 · 678c714
2 parents 6578886 + f17ae7c
commit 678c714
Show file tree

Hide file tree

Showing 28 changed files with 612 additions and 178 deletions.
diff --git a/README.md b/README.md
@@ -69,9 +69,10 @@ More details: [`x-ipfs-path` Header Support in IPFS Companion](https://github.co
 
 #### Redirect Opt-Out
 
-It is possible to opt-out from redirect by
-a) suspending extension via global toggle
-b) including `x-ipfs-companion-no-redirect` in the URL (as a [hash](https://ipfs.io/ipfs/QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR#x-ipfs-companion-no-redirect) or [query](https://ipfs.io/ipfs/QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR?x-ipfs-companion-no-redirect) parameter).
+It is possible to opt-out from Gateway redirect by:
+- a) suspending redirect via global toggle (see [_Disable All Redirects_](#disable-all-redirects) below)
+- b) suspending redirect for via per website opt-out (in [_Active Tab_ section of _Browser Action_](#disable-gateway-redirect-per-website) or _Preferences_)
+- c) including `x-ipfs-companion-no-redirect` in the URL (as a [hash](https://ipfs.io/ipfs/QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR#x-ipfs-companion-no-redirect) or [query](https://ipfs.io/ipfs/QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR?x-ipfs-companion-no-redirect) parameter).
 
 ### IPFS API as `window.ipfs`
 
@@ -80,11 +81,30 @@ Websites can detect if `window.ipfs` exists and opt-in to use it instead of crea
 It saves system resources and battery (on mobile), avoids the overhead of peer discovery/connection, enables shared repository access and more!
 Make sure to read our [notes on `window.ipfs`](https://github.com/ipfs-shipyard/ipfs-companion/blob/master/docs/window.ipfs.md), where we explain it in-depth and provide examples on how to use it your own dapp.
 
-### Toggle IPFS Integrations
+### Quick Toggles
 
-> ![screenshot of suspend toggle](https://user-images.githubusercontent.com/157609/42685002-18c7cee4-8692-11e8-9171-970866d91ae0.gif)
+The Browser Action pop-up provides handy toggles for often used operations.
 
-The Browser Action pop-up provides a toggle for suspending all active IPFS integrations with a single click.
+#### Disable Gateway Redirect Per Website
+
+> _Active Tab_ actions include option to opt-out current website from Gateway redirect of any IPFS subresources.    
+> Disabling redirect for DNSLink website will restore original URL as well:
+>
+> ![per-site-peek 2019-02-26 00-23](https://user-images.githubusercontent.com/157609/53376094-86557500-395d-11e9-837f-a4712aa19236.gif)    
+
+#### Disable All Redirects
+
+> A handy toggle to disable all gateway redirects while keeping all other features enabled:
+>
+> ![redirect](https://user-images.githubusercontent.com/157609/53376263-0976cb00-395e-11e9-8536-d83d28ffeee9.gif)
+
+#### Suspend IPFS Extension
+
+> The "power" icon can be used to temporarily suspend all IPFS integrations
+> (redirects, API status, content scripts, protocol handlers etc).
+> Useful during testing. Extension can be re-enabled with a single click:
+>
+> ![screenshot of suspend toggle](https://user-images.githubusercontent.com/157609/53376196-d6343c00-395d-11e9-83f2-04c16b3a008f.gif)
 
 ### IPFS Status and Context Actions
 

diff --git a/add-on/_locales/en/messages.json b/add-on/_locales/en/messages.json
@@ -8,7 +8,7 @@
     "description": "A label for IPFS icon (panel_headerIpfsNodeIconLabel)"
   },
   "panel_headerActiveToggleTitle": {
-    "message": "Global toggle: Suspend all IPFS integrations",
+    "message": "Toggle all IPFS integrations",
     "description": "A label for an embedded IPFS node (panel_headerActiveToggleTitle)"
   },
   "panel_statusOffline": {
@@ -43,25 +43,41 @@
     "message": "Open Web UI",
     "description": "A menu item in Browser Action pop-up (panel_openWebui)"
   },
+  "panel_redirectToggle": {
+    "message": "Redirect to Gateway",
+    "description": "A menu item in Browser Action pop-up (panel_redirectToggle)"
+  },
+  "panel_redirectToggleTooltip": {
+    "message": "Click to toggle all gateway redirects",
+    "description": "A menu item in Browser Action pop-up (panel_redirectToggleTooltip)"
+  },
+  "panel_toolsSectionHeader": {
+    "message": "Tools",
+    "description": "A menu item in Browser Action pop-up (panel_toolsSectionHeader)"
+  },
   "panel_openPreferences": {
     "message": "Open Preferences of Browser Extension",
     "description": "A menu item in Browser Action pop-up (panel_openPreferences)"
   },
-  "panel_switchToCustomGateway": {
-    "message": "Switch to Custom Gateway",
-    "description": "A menu item in Browser Action pop-up (panel_switchToCustomGateway)"
+  "panel_activeTabSectionHeader": {
+    "message": "Active Tab",
+    "description": "A menu item in Browser Action pop-up (panel_activeTabSiteRedirectEnable)"
   },
-  "panel_switchToPublicGateway": {
-    "message": "Switch to Public Gateway",
-    "description": "A menu item in Browser Action pop-up (panel_switchToPublicGateway)"
+  "panel_activeTabSiteRedirectToggle": {
+    "message": "Redirect on $1",
+    "description": "A menu item in Browser Action pop-up (panel_activeTabSiteRedirectToggle)"
+  },
+  "panel_activeTabSiteRedirectToggleTooltip": {
+    "message": "Click to toggle gateway redirects on $1",
+    "description": "A menu item tooltip in Browser Action pop-up (panel_activeTabSiteRedirectToggleTooltip)"
   },
   "panel_pinCurrentIpfsAddress": {
     "message": "Pin IPFS Resource",
     "description": "A menu item in Browser Action pop-up (panel_pinCurrentIpfsAddress)"
   },
-  "panel_unpinCurrentIpfsAddress": {
-    "message": "Unpin IPFS Resource",
-    "description": "A menu item in Browser Action pop-up (panel_unpinCurrentIpfsAddress)"
+  "panel_pinCurrentIpfsAddressTooltip": {
+    "message": "Pinning a CID tells your IPFS node that this data is important and mustn’t be thrown away.",
+    "description": "A menu item tooltip in Browser Action pop-up (panel_pinCurrentIpfsAddressTooltip)"
   },
   "panelCopy_currentIpfsAddress": {
     "message": "Copy IPFS Path",
@@ -251,6 +267,14 @@
     "message": "Redirect requests for IPFS resources to the Custom gateway",
     "description": "An option description on the Preferences screen (option_useCustomGateway_description)"
   },
+  "option_noRedirectHostnames_title": {
+    "message": "Redirect Opt-Outs",
+    "description": "An option title on the Preferences screen (option_noRedirectHostnames_title)"
+  },
+  "option_noRedirectHostnames_description": {
+    "message": "List of websites that should not be redirected to the Custom Gateway (includes subresources from other domains). One hostname per line.",
+    "description": "An option description on the Preferences screen (option_noRedirectHostnames_description)"
+  },
   "option_publicGatewayUrl_title": {
     "message": "Default Public Gateway",
     "description": "An option title on the Preferences screen (option_publicGatewayUrl_title)"

diff --git a/add-on/src/lib/dnslink.js b/add-on/src/lib/dnslink.js
@@ -175,6 +175,29 @@ module.exports = function createDnslinkResolver (getState) {
       }
       const fqdn = url.hostname
       return `/ipns/${fqdn}${url.pathname}${url.search}${url.hash}`
+    },
+
+    // Test if URL contains a valid DNSLink FQDN
+    // in url.hostname OR in url.pathname (/ipns/<fqdn>)
+    // and return matching FQDN if present
+    findDNSLinkHostname (url) {
+      const { hostname, pathname } = new URL(url)
+      // check //foo.tld/ipns/<fqdn>
+      if (IsIpfs.ipnsPath(pathname)) {
+        // we may have false-positives here, so we do additional checks below
+        const ipnsRoot = pathname.match(/^\/ipns\/([^/]+)/)[1]
+        // console.log('findDNSLinkHostname ==> inspecting IPNS root', ipnsRoot)
+        // Ignore PeerIDs, match DNSLink only
+        if (!IsIpfs.cid(ipnsRoot) && dnslinkResolver.readAndCacheDnslink(ipnsRoot)) {
+          // console.log('findDNSLinkHostname ==> found DNSLink for FQDN in url.pathname: ', ipnsRoot)
+          return ipnsRoot
+        }
+      }
+      // check //<fqdn>/foo/bar
+      if (dnslinkResolver.readAndCacheDnslink(hostname)) {
+        // console.log('findDNSLinkHostname ==> found DNSLink for url.hostname', hostname)
+        return hostname
+      }
     }
 
   }

diff --git a/add-on/src/lib/ipfs-companion.js b/add-on/src/lib/ipfs-companion.js
@@ -213,13 +213,17 @@ module.exports = async function init () {
 
   async function sendStatusUpdateToBrowserAction () {
     if (!browserActionPort) return
+    const dropSlash = url => url.replace(/\/$/, '')
     const info = {
       active: state.active,
       ipfsNodeType: state.ipfsNodeType,
       peerCount: state.peerCount,
-      gwURLString: state.gwURLString,
-      pubGwURLString: state.pubGwURLString,
+      gwURLString: dropSlash(state.gwURLString),
+      pubGwURLString: dropSlash(state.pubGwURLString),
       webuiRootUrl: state.webuiRootUrl,
+      apiURLString: dropSlash(state.apiURLString),
+      redirect: state.redirect,
+      noRedirectHostnames: state.noRedirectHostnames,
       currentTab: await browser.tabs.query({ active: true, currentWindow: true }).then(tabs => tabs[0])
     }
     try {
@@ -231,7 +235,12 @@ module.exports = async function init () {
       info.gatewayVersion = null
     }
     if (info.currentTab) {
-      info.ipfsPageActionsContext = ipfsPathValidator.isIpfsPageActionsContext(info.currentTab.url)
+      const url = info.currentTab.url
+      info.isIpfsContext = ipfsPathValidator.isIpfsPageActionsContext(url)
+      info.currentDnslinkFqdn = dnslinkResolver.findDNSLinkHostname(url)
+      info.currentFqdn = info.currentDnslinkFqdn || new URL(url).hostname
+      info.currentTabRedirectOptOut = info.noRedirectHostnames && info.noRedirectHostnames.includes(info.currentFqdn)
+      info.isRedirectContext = info.currentFqdn && ipfsPathValidator.isRedirectPageActionsContext(url)
     }
     // Still here?
     if (browserActionPort) {
@@ -641,6 +650,7 @@ module.exports = async function init () {
         case 'automaticMode':
         case 'detectIpfsPathHeader':
         case 'preloadAtPublicGateway':
+        case 'noRedirectHostnames':
           state[key] = change.newValue
           break
       }

diff --git a/add-on/src/lib/ipfs-path.js b/add-on/src/lib/ipfs-path.js
@@ -64,8 +64,19 @@ function createIpfsPathValidator (getState, dnsLink) {
     },
 
     // Test if actions such as 'copy URL', 'pin/unpin' should be enabled for the URL
+    // TODO: include hostname check for DNSLink and display option to copy CID even if no redirect
     isIpfsPageActionsContext (url) {
       return (IsIpfs.url(url) && !url.startsWith(getState().apiURLString)) || IsIpfs.subdomain(url)
+    },
+
+    // Test if actions such as 'per site redirect toggle' should be enabled for the URL
+    isRedirectPageActionsContext (url) {
+      const state = getState()
+      return state.ipfsNodeType === 'external' && // hide with embedded node
+      (IsIpfs.ipnsUrl(url) || // show on /ipns/<fqdn>
+        (url.startsWith('http') && // hide on non-HTTP pages
+         !url.startsWith(state.gwURLString) && // hide on /ipfs/*
+          !url.startsWith(state.apiURLString))) // hide on api port
     }
   }
 
@@ -110,6 +121,7 @@ function validIpnsPath (path, dnsLink) {
       // console.log('==> IPNS is a valid CID', ipnsRoot)
       return true
     }
+    // then see if there is an DNSLink entry for 'ipnsRoot' hostname
     if (dnsLink.readAndCacheDnslink(ipnsRoot)) {
       // console.log('==> IPNS for FQDN with valid dnslink: ', ipnsRoot)
       return true

diff --git a/add-on/src/lib/ipfs-request.js b/add-on/src/lib/ipfs-request.js
@@ -65,6 +65,22 @@ function createRequestModifier (getState, dnslinkResolver, ipfsPathValidator, ru
     if (request.url.startsWith('http://127.0.0.1') || request.url.startsWith('http://localhost') || request.url.startsWith('http://[::1]')) {
       ignore(request.requestId)
     }
+    // skip if a per-site redirect opt-out exists
+    const parentUrl = request.originUrl || request.initiator // FF: originUrl (Referer-like Origin URL), Chrome: initiator (just Origin)
+    const fqdn = new URL(request.url).hostname
+    const parentFqdn = parentUrl && request.url !== parentUrl ? new URL(parentUrl).hostname : null
+    if (state.noRedirectHostnames.some(optout =>
+      fqdn.endsWith(optout) || (parentFqdn && parentFqdn.endsWith(optout)
+      ))) {
+      ignore(request.requestId)
+    }
+    // additional checks limited to requests for root documents
+    if (request.type === 'main_frame') {
+      // lazily trigger DNSLink lookup (will do anything only if status for root domain is not in cache)
+      if (state.dnslinkPolicy && dnslinkResolver.canLookupURL(request.url)) {
+        dnslinkResolver.preloadDnslink(request.url)
+      }
+    }
     return isIgnored(request.requestId)
   }
 

diff --git a/add-on/src/lib/options.js b/add-on/src/lib/options.js
@@ -1,5 +1,7 @@
 'use strict'
 
+const isFQDN = require('is-fqdn')
+
 exports.optionDefaults = Object.freeze({
   active: true, // global ON/OFF switch, overrides everything else
   ipfsNodeType: 'external', // or 'embedded'
@@ -12,6 +14,7 @@ exports.optionDefaults = Object.freeze({
   }, null, 2),
   publicGatewayUrl: 'https://ipfs.io',
   useCustomGateway: true,
+  noRedirectHostnames: [],
   automaticMode: true,
   linkify: false,
   dnslinkPolicy: 'best-effort',
@@ -22,7 +25,7 @@ exports.optionDefaults = Object.freeze({
   customGatewayUrl: 'http://127.0.0.1:8080',
   ipfsApiUrl: 'http://127.0.0.1:5001',
   ipfsApiPollMs: 3000,
-  ipfsProxy: true
+  ipfsProxy: true // window.ipfs
 })
 
 // `storage` should be a browser.storage.local or similar
@@ -64,6 +67,23 @@ function normalizeGatewayURL (url) {
 exports.normalizeGatewayURL = normalizeGatewayURL
 exports.safeURL = (url) => new URL(normalizeGatewayURL(url))
 
+// convert JS array to multiline textarea
+function hostArrayCleanup (array) {
+  array = array.map(host => host.trim().toLowerCase())
+  array = [...new Set(array)] // dedup
+  array = array.filter(Boolean).filter(isFQDN)
+  array.sort()
+  return array
+}
+function hostArrayToText (array) {
+  return hostArrayCleanup(array).join('\n')
+}
+function hostTextToArray (text) {
+  return hostArrayCleanup(text.split('\n'))
+}
+exports.hostArrayToText = hostArrayToText
+exports.hostTextToArray = hostTextToArray
+
 exports.migrateOptions = async (storage) => {
   // <= v2.4.4
   // DNSLINK: convert old on/off 'dnslink' flag to text-based 'dnslinkPolicy'

diff --git a/add-on/src/options/forms/gateways-form.js b/add-on/src/options/forms/gateways-form.js
@@ -3,7 +3,7 @@
 
 const browser = require('webextension-polyfill')
 const html = require('choo/html')
-const { normalizeGatewayURL } = require('../../lib/options')
+const { normalizeGatewayURL, hostTextToArray, hostArrayToText } = require('../../lib/options')
 
 // Warn about mixed content issues when changing the gateway
 // https://github.com/ipfs-shipyard/ipfs-companion/issues/648
@@ -13,19 +13,40 @@ function gatewaysForm ({
   ipfsNodeType,
   customGatewayUrl,
   useCustomGateway,
+  noRedirectHostnames,
   publicGatewayUrl,
   onOptionChange
 }) {
   const onCustomGatewayUrlChange = onOptionChange('customGatewayUrl', normalizeGatewayURL)
   const onUseCustomGatewayChange = onOptionChange('useCustomGateway')
   const onPublicGatewayUrlChange = onOptionChange('publicGatewayUrl', normalizeGatewayURL)
+  const onNoRedirectHostnamesChange = onOptionChange('noRedirectHostnames', hostTextToArray)
   const mixedContentWarning = !secureContextUrl.test(customGatewayUrl)
+  const supportRedirectToCustomGateway = ipfsNodeType === 'external'
 
   return html`
     <form>
       <fieldset>
         <legend>${browser.i18n.getMessage('option_header_gateways')}</legend>
-          ${ipfsNodeType === 'external' ? html`
+          <div>
+            <label for="publicGatewayUrl">
+              <dl>
+                <dt>${browser.i18n.getMessage('option_publicGatewayUrl_title')}</dt>
+                <dd>${browser.i18n.getMessage('option_publicGatewayUrl_description')}</dd>
+              </dl>
+            </label>
+            <input
+              id="publicGatewayUrl"
+              type="url"
+              inputmode="url"
+              required
+              pattern="^https?://[^/]+/?$"
+              spellcheck="false"
+              title="Enter URL without any sub-path"
+              onchange=${onPublicGatewayUrlChange}
+              value=${publicGatewayUrl} />
+          </div>
+          ${supportRedirectToCustomGateway ? html`
             <div>
               <label for="customGatewayUrl">
                 <dl>
@@ -48,7 +69,7 @@ function gatewaysForm ({
 
             </div>
           ` : null}
-          ${ipfsNodeType === 'external' ? html`
+          ${supportRedirectToCustomGateway ? html`
             <div>
               <label for="useCustomGateway">
                 <dl>
@@ -63,24 +84,22 @@ function gatewaysForm ({
                 checked=${useCustomGateway} />
             </div>
           ` : null}
-        <div>
-          <label for="publicGatewayUrl">
-            <dl>
-              <dt>${browser.i18n.getMessage('option_publicGatewayUrl_title')}</dt>
-              <dd>${browser.i18n.getMessage('option_publicGatewayUrl_description')}</dd>
-            </dl>
-          </label>
-          <input
-            id="publicGatewayUrl"
-            type="url"
-            inputmode="url"
-            required
-            pattern="^https?://[^/]+/?$"
-            spellcheck="false"
-            title="Enter URL without any sub-path"
-            onchange=${onPublicGatewayUrlChange}
-            value=${publicGatewayUrl} />
-        </div>
+          ${supportRedirectToCustomGateway ? html`
+            <div>
+              <label for="noRedirectHostnames">
+                <dl>
+                  <dt>${browser.i18n.getMessage('option_noRedirectHostnames_title')}</dt>
+                  <dd>${browser.i18n.getMessage('option_noRedirectHostnames_description')}</dd>
+                </dl>
+              </label>
+              <textarea
+                id="noRedirectHostnames"
+                spellcheck="false"
+                onchange=${onNoRedirectHostnamesChange}
+                rows="4" 
+                >${hostArrayToText(noRedirectHostnames)}</textarea>
+            </div>
+          ` : null}
       </fieldset>
     </form>
   `