Web Extension runs a local go-IPFS node and redirects IPFS urls to the gateway

[flyingzumwalt]

Example: https://github.com/lidel/ipfs-firefox-addon

This will be done when:
We have published a Web Extension that runs a local node and redirects IPFS urls to the gateway

General notes:

• use web extension (works on chrome and firefox)
• There is already an ipfs-station chrome extension that does some of this: https://chrome.google.com/webstore/detail/ipfs-station/kckhgoigikkadogfdiojcblegfhdnjei?brand=CHBD&gclid=CIiYzM7SktICFZORGwodX8MKFw

Acceptance Criteria:

• Web Extension runs a local node and redirects IPFS urls to the gateway
  • downloads and installs go-ipfs
    • runs the go-ipfs daemon (given path to binary)
    • (Expands on the extensions that were already available by downloading the binary)
  • bundles js-ipfs by default
  • resolves IPFS addresses through the gateway

Also see #12.

Background

@Gozala (2017-02-01):

From this point it would make more sense to package up protocol implementations along with ipfs binary as an add-on so users won’t need to install ipfs and an add-on

and from @jbenet

one goal we have for the sprint is to explore putting {go-ipfs or js-ipfs} into {add-on, extension, web extension}, to test out possibilities. ofc, add-ons are going away, so some of this would only be proof-of-concept to demo what it would be like to resolve ipfs content natively.

ALSO

Background

@Gozala (2017-02-01):

It would also great if protocol handlers were able to stream content from the ipfs node without running an http gateway. Although I’m all but certain how that could be pulled off.

@jbenet (2017-02-05):

• Absolutely, that's a long term goal.

• It would be great to look APIs that {addons, extensions, web-extensions} have and how this would work.

• I suspect js-ipfs (or a gopherjs-compiled go-ipfs, or some hybrid of both) will be critical to native resolution. at least last time i looked at it, it did not seem go-ipfs alone could do it for 100% of users, and a full js implementation would have to be involved.

• rust-ipfs is likely going to be a thing by late 2017 or 2018, so there's that (for servo)

• see https://github.com/ipfs/go-ipfs

• see https://github.com/ipfs/js-ipfs

• see https://github.com/ipfs/webui <--- this is a user-facing thing. seriously needs revamp. we can consider ideas about it in this coming sprint (we can offload a lot of design to contractors who work with us, so we can get some meaningful work done in the next 2-3 weeks).

@Gozala (2017-02-06)

In add-on sdk (jetpack) we had node subprocess API implemented. I though of using that to spawn a go process & was hoping some IPC mechanism could be used to do path resolution and other tasks.

and

Rust code actually could also be embedded in Gecko, in fact some of the code in Firefox already is ;)

[flyingzumwalt]

Breaking this up:

• js-ipfs node
• checks if go-ipfs is already available
• get the extension to change IPFS hashes into localhost hashes

[jefft0]

Is it a goal to work in Firefox on Android? The current Firefox extension doesn't. I think there should be an explicit decision, even if it is "not now" or "with a subset of low-resource features".

[lidel]

Old extension does not work on Android, but from what I was able to find WebExtension version should work the same in Android as it does on desktop (+/- some gui elements): ipfs/ipfs-companion#109
So in theory, it should be possible to maintain the same codebase for desktop and mobile.

[Gozala]

@lidel I don't believe web-extensions are ever going to allow bundling binaries like go-ipfs or spawning them processes. I am also skeptical that they would allow implementing custom protocol handlers, in fact with e10s (firefox with multiple processes) intentionally reduced abilities of custom protocol handlers.

[lidel]

@Gozala I see two ways of shipping IPFS with add-on:

1. WebExtensions do not allow for bundling binaries within add-on but I remember conversation about possibility of shipping user-friendly installer that includes both go-ipfs and WebExtension (using Native messaging or HTTP API for communication between both).

2. Alternatively we could just use pure JS node from js-ipfs (when ready) and run it in background page of WebExtensions.

As for custom protocol handlers in WebExtension, there is an open API request at Bugzilla, however it is not clear to me how that implementation will handle origin issues (but that discussion should continue in #6, #3).

[haadcode]

@whyrusleeping made progress on this and put together a test: https://github.com/whyrusleeping/ipfs-webext.

From what I can tell, it's spawning go-ipfs via a shell script. According to his notes, we would:

Extension ships with a small shell script that is able to download an ipfs-manager type program that does a few things:
  - communications with firefox via nativeMessaging (does the length prefixed json message stuff)
  - Is able to download, init, start and stop an ipfs daemon
  - Is able to control the ipfs daemon via messages from the webextension

My immediate question is, how is this sandboxed (if at all)? Can the extensions run just any shell-script?

Alternatively we could just use pure JS node from js-ipfs (when ready) and run it in background page of WebExtensions.

@lidel How do the add-ons background scripts run in terms of isolation? If we have js-ipfs bundled in the add-on and we run it in the background script, is it one per page/tab or one per window? Is it running in its own process or process per page/tab? The concern here is generally resource consumption and more specifically getting CPU cycles if it's in the background.

[victorb]

@haadcode a background script starts running as soon as you install the extension until you uninstall it or close firefox. There is only one instance of the background script running at all times so if we go the js-ipfs route, background scripts would be the best way to start a daemon in the background. There is more information about background scripts here: https://developer.mozilla.org/en-US/Add-ons/WebExtensions/Anatomy_of_a_WebExtension#Background_scripts

[whyrusleeping]

@Gozala @lidel @haadcode

The webextension i've been working on relies on the user installing and ipfs binary and a 'native messaging host' descriptor (Which i've made a handy install script to do). Once thats done, the extension can start the ipfs daemon on its own and redirect requests to it.

@lidel Ideally, We could integrate the work i've been doing into your extension, and then your extension should work for both firefox and chrome (webextensions are cross-browser compatible for the most part, if i understand correctly)

[flyingzumwalt]

This issue will be "done" when:

• the code in https://github.com/whyrusleeping/ipfs-webext satisfies the "Acceptance Criteria” listed in the description of this issue
• there are tests confirming that it satisfies those criteria
• there are docs (ie instructions in the readme) that provide instructions on how someone can confirm that the criteria are met

[flyingzumwalt]

@whyrusleeping your instructions say

For chrome, go to chrome://extensions and select "Load unpacked extension”.

Please explain where to find the option to select “Load unpacked extension”
Also please explain how to install for Firefox, which is our main target browser.

[whyrusleeping]

@flyingzumwalt The sentence right before the one you quote says how to do the same in firefox. Let me adjust the spacing (newlines in markdown don't mean a newline until you have two, i forgot this fact)

[flyingzumwalt]

After installing, my copy of the extension doesn't provide a way to start the daemon. It just says

Oh snap, it looks like your node
is not running yet.

Please start it by running
ipfs daemon
on your terminal.

Thus far, the extension has neither

1. Downloaded the ipfs binary (I had to run a script on the command line to do that)
2. Started the daemon

The other two acceptance criteria are

• bundles js-ipfs by default
• resolves IPFS addresses through the gateway

Does it do those?

[whyrusleeping]

@flyingzumwalt My code does not print any of those messages. Are you using my extension?

Did the install.sh script succeed? Did you try going to ipfs.io/ipns/ipfs.io in your browser?

[whyrusleeping]

The daemon is started automatically when the extension loads

[flyingzumwalt]

Yeah. When I try to start it on the command line I get Error: serveHTTPApi: manet.Listen(/ip4/127.0.0.1/tcp/5001) failed: listen tcp4 127.0.0.1:5001: bind: address already in use but the extension still says it's not running.

[whyrusleeping]

You don't need to start it on the command line. The webextension starts it on its own. Where are you seeing a message saying that the daemon isnt running?

[flyingzumwalt]

I see it when I click on the IPFS cube icon in chrome.

[whyrusleeping]

@flyingzumwalt I think that might be a different extension you have installed.

[whyrusleeping]

Try loading https://ipfs.io/ipns/ipfs.io in your browser

[flyingzumwalt]

How can I test whether the extension is resolving content through the daemon? The readme says

you should be able to visit any site in ipfs via the ipfs.io gateway and be redirected to your locally running ipfs node.

But that prevents offline use. I want to turn off my wifi and read content that I already pinned on my local daemon. I also want to resolve content from daemons on a local network without hitting a remote gateway. Why isn't that possible?

[whyrusleeping]

@flyingzumwalt the requests to ipfs.io get redirected. This works without internet, assuming you have the content in question cached (though the dns resolution of my /ipns/ipfs.io example wouldnt work without internet)

[flyingzumwalt]

Nevermid the bit about Oh snap, it looks like your node is not running yet. That's from the station extension. I forgot to disable it. The new extension doesn't show its icon up in the chrome extension bar -- it's just the default puzzle piece icon.

[whyrusleeping]

@flyingzumwalt Yeah, i didnt put any effort into UI. Others have done that work much better than i will be able to.

[whyrusleeping]

@flyingzumwalt Try pulling latest on the extension and running the install again. I fixed a warning message firefox was giving me about a field in the hosts file.

[lidel]

@whyrusleeping WebExtensions are supposed to be cross-browser compatible, but Firefox ships own "twist" on the topic under browser object. Using it instead of chrome provides Promise support and additional APIs that are not supported by Chrome.

However, running browser-based WebExtension in Chrome should be possible with mozilla/webextension-polyfill (provides browserobject). I did not have time to test it yet, but I hope this helps 👍

As for go-ipfs support, my understanding is that the added value comes from starting/stopping ipfs daemon along with WebExtension. Sure, we can integrate that along with some heuristics that detect already running one 👍

But ipfs installation still needs to be done with external tool (in this case install.sh script), so the most of this effort goes to OS-specific installer, which is outside of WebExtension's scope.

[jefft0]

Test results: The extension https://github.com/whyrusleeping/ipfs-webext works for me in Firefox and Chrome on OS X 10.10, 10.11 and 10.12 and Firefox and in Chromium on Ubuntu 12.04, 14.04, 16.04 and 16.10.