Add WebTorrent support #5342

dcposch · 2016-11-02T03:17:26Z

See #3256

This is still pretty raw. It demonstrates torrent support in Brave.

Visit a magnet URL

Bookmark a magnet link

Play files in a torrent

Here are some of the things we still need to do:

Design nicer UX
Implement nicer UX
Probably move WebTorrent to its own process. Currently it's running in the Brave extension content script, with some of the CSP restrictions disabled.
Add tests
Security review (sort of---cc @diracdeltas )

Feedback appreciated!

@BrendanEich @diracdeltas @ayumi @bbondy

bbondy · 2016-11-02T12:08:28Z

Could you add some sample torrent and magnet links that would demonstrate the added things nicely? Thanks!

dcposch · 2016-11-02T12:37:03Z

@bbondy sure, @feross collected some here: https://codepen.io/ferossity/full/qaezaB/

Right now you have to paste magnet links into the URL bar. Clicking a magnet link still tries to open an external app. I'll try to fix that later today.

dcposch · 2016-11-02T23:31:20Z

@bbondy you can click on magnet links now

do you think this commit is a reasonable way to implement that?

0834c98

dcposch · 2016-11-03T18:33:20Z

app/extensions.js

+  // * Move WebTorrent to its own renderer process, similar to the way it's done in
+  //   WebTorrent Desktop
+  // * Remove this CSP exception:
+  cspDirectives['connect-src'] = '*'


@bbondy do these CSP rules apply only to the braveExtension, or to all extensions?

@dcposch only to the braveExtension, iirc. for security reasons, i would prefer that WebTorrent be its own "extension" similar to how we handle PDF reader (PDF.JS)

diracdeltas · 2016-11-03T20:38:55Z

app/extensions/brave/js/webtorrent.js

@@ -0,0 +1,19 @@
+(function () {
+  var queryString = window.location.search


looks like queryString is not used?

Will remove

diracdeltas · 2016-11-03T20:52:31Z

js/lib/appUrlUtil.js

+module.exports.getSourceMagnetUrl = function (input) {
+  const url = module.exports.getAppUrl('webtorrent.html')
+  if (!input.startsWith(url)) return null
+  return input.substring(input.indexOf('#') + 1)


shouldn't this check that the return value starts with 'magnet:?'

The target URL is internal-only and controlled by us, so I think that we can assume it will always be a magnet link or torrent file. This seems to be the assumption that is made by the analogous getSourceAboutUrl function. Correct me if I'm wrong?

that sounds right, assuming you won't define page-anchor URLs like chrome-extension://.../webtorrent.html#about or something.

diracdeltas · 2016-11-03T21:02:55Z

how should this work for users who already have a magnet: URL protocol handler set up in Brave? ex: i usually set up put.io to handle magnet links

feross · 2016-11-03T22:17:47Z

It's also worth noting that as currently architected in this PR, WebTorrent can only connect to WebRTC peers, and not normal TCP/uTP ones. This is because we're running in a chrome-extension:/// context that has no node integration. So we're using the browser version of WebTorrent (webtorrent.min.js) at the moment.

feross · 2016-11-03T22:19:35Z

@diracdeltas As currently implemented, we ignore any protocol handlers registered by webpages through navigator.registerProtocolHandler().

Does Brave expose an interface for configuring protocol handlers yet?

Actually, here's an idea. What if we just add a new link to the Torrent Viewer page that says "Open in " for users who have a "magnet" protocol handler registered. Here's the current mock @dcposch is working on:

So imagine another link added to that list. @diracdeltas Would that satisfy your use case?

diracdeltas · 2016-11-04T01:15:41Z

What if we just add a new link to the Torrent Viewer page that says "Open in " for users who have a "magnet" protocol handler registered. Here's the current mock @dcposch is working on:

👍

dcposch · 2016-11-04T02:56:16Z

New screenshots

The file list now consists of regular hyperlinks. You can link to individual files within a torrent now, using the ix magnet link parameter.

Right now, following one of those links is slow, because the new page has to connect to the swarm from scratch. That will be fixed once we move WebTorrent to its own process.

dcposch · 2016-11-11T15:08:45Z

WebTorrent now runs in the browser process.

It communicates with the Torrent Viewer extension pages over IPC, and serves content via a streaming torrent-to-HTTP server.

dcposch · 2016-11-11T15:09:53Z

Improved UI/UX coming soon, @feross has a PR! dcposch#7

dcposch · 2016-11-11T23:36:38Z

Almost ready for review!

Work left to do

Merge WebTorrent extension progress dcposch/browser-laptop#7 and rebase on latest master
Fix refresh: hitting F5 should not try to open an external torrent app
Auto stop: once the last tab for a given infohash is closed, remove the torrent after a reasonable amount of time
Auto start: if a torrent is already active, go directly to the file list, don't show the Start button
[Maybe] No extra UI around a file. Just show the file contents. See WebTorrent extension progress dcposch/browser-laptop#7 (comment)
~~[Maybe] Better Incognito support~~
Manual testing. Open 20+ tabs full of magnet links. Hold down the Refresh button. Open the same video across multiple windows. Nothing should crash, glitch, or get slow.

feross · 2016-11-14T20:37:25Z

Also:

Fix the way magnet links show up in the History tab.
Support right-click > "Open in New Tab" in File List.
Play MKVs instead of downloading them

diracdeltas · 2016-11-15T04:46:50Z

js/components/frame.js

@@ -877,10 +887,27 @@ class Frame extends ImmutableComponent {
          method = (currentDetail, originalDetail) =>
            windowActions.setAutofillCreditCardDetail(currentDetail, originalDetail)
          break
+        case messages.TORRENT_MESSAGE:
+          // Relay torrent IPC from the webview to a browser process


I don't think this relaying is necessary; the webview can just call ipc.send instead of ipc.sendToHost to send a message to the browser process. See messages.CERT_ERROR_ACCEPTED for instance.

hmm. that doesn't seem to work

nvm, figured it out!

dcposch · 2016-11-19T02:40:16Z

I think we're ready

Please try it out & let me know if there's anything that needs improvement!

Code review also appreciated. Rebased on the latest master earlier today: 5e36e37, a few commits after Brave 0.12.10

@bbondy @diracdeltas @bradleyrichter @feross @BrendanEich

In this PR

Support for magnet links. View a torrent, stream individual files, bookmark, save .torrents, download files.

Not in this PR

Here are some additional things you could do, probably after Brave 1.0

Support HTTP torrent URLs. Currently, navigating to a .torrent file will just download it. We could treat those the same way we treat magnet links. There are some UX challenges to consider. For example: how do you link to a single file within an HTTP .torrent URL? It's not as straightforward as appending &ix=<n> to a magnet URL. You might use a hash, eg http[s]://<...>.torrent#ix=<n>
History. Currently, magnet: links don't appear in the History tab. Looks like this is the same behavior as PDFs in the PDF Viewer.
Support torrents in the mobile versions of Brave.

Latest screenshots

Regular webpage with magnet links

Navigate https://codepen.io/ferossity/full/qaezaB/. Click one of the links.

Magnet page

Brave doesn't connect to any peers or to the DHT until you give it permission. A short legal warning lets users know they're about to join a P2P network.

Click start.

File list

Right click on one of the named links, then click Open in New Tab.

Magnet page for an individual file

This is a magnet link with the same infohash as the one we were just on, but with an ix attribute specifying a single file inside the torrent.

Since that torrent is already active, there's not disclaimer and no start button, the media starts streaming immediately. WebTorrent downloads pieces from peers in the order that we need them so we can start playing right away.

Download links

Back in the first tab, click Download. This creates a normal download. Under the hood, it uses the same streaming torrent-to-HTTP local server as the

You can also click Download Torrent File to save a .torrent, eg for use with an external app.

Bookmarks

You can bookmark a whole torrent or an individual file.

Preferences

You can disable the Torrent Viewer. In that case, magnet files will prompt the user to open an external app.

LMK if this works for you!

bbondy · 2016-11-22T22:01:56Z

Rebased to get rid of the recent conflict and merged to master
Great work @dcposch @feross @diracdeltas ! 👍 💯
eb004bd...05e1231

feross mentioned this pull request Nov 2, 2016

Feature: WebTorrent Integration #3256

Closed

dcposch force-pushed the master branch 2 times, most recently from 5e11a47 to 709fafd Compare November 2, 2016 03:25

bbondy changed the title ~~Add WebTorrent support (work in progress)~~ [WIP] Add WebTorrent support Nov 2, 2016

luixxiul added the enhancement label Nov 2, 2016

dcposch force-pushed the master branch from f84ed99 to 0834c98 Compare November 2, 2016 23:30

dcposch commented Nov 3, 2016

View reviewed changes

diracdeltas reviewed Nov 3, 2016

View reviewed changes

dcposch force-pushed the master branch 2 times, most recently from a501321 to f9d8d41 Compare November 8, 2016 21:16

dcposch mentioned this pull request Nov 9, 2016

[WIP] Move WebTorrent to its own process #5500

Closed

dcposch force-pushed the master branch from 2566786 to b460ab7 Compare November 9, 2016 02:02

feross force-pushed the master branch from 9df896a to 1c1e488 Compare November 11, 2016 03:01

dcposch force-pushed the master branch from 9f92bf9 to 4fc9511 Compare November 12, 2016 02:02

dcposch force-pushed the master branch 2 times, most recently from e896f82 to 349d13b Compare November 15, 2016 04:11

diracdeltas reviewed Nov 15, 2016

View reviewed changes

dcposch added 17 commits November 18, 2016 16:49

shortcut icon

86d55c4

fix crash when closing a window containing a magnet tab

129da50

webtorrent-remote 0.0.4

e223300

torrent ipc: direct webview-to-browser, don't relay thru frame.js

3f53179

fix Torrent Viewer extension CSP

7e13a2e

Torrent Viewer localization

fcdcbce

webtorrent-remote 0.0.5

61d951f

add torrent and create server simultaneously

50346e6

webtorrent server trace

61d5f28

torrentStatus: always show error message

e12629e

automatically allow openExternal only for PDFJS, not all extensions

f31f9b6

pref to enable or disable Torrent Viewer

a9ed691

webtorrent-remote 0.0.7

3b7f552

appUrlUtilTest

2997c7a

fix web_accessible_resources

2e1d29f

Torrent Viewer: support fullscreen

f4c92b5

Torrent Viewer: add download links

d8a8f55

dcposch force-pushed the master branch from 6ffe619 to d8a8f55 Compare November 19, 2016 01:13

webtorrent-remote 0.0.8

725acea

dcposch changed the title ~~[WIP] Add WebTorrent support~~ Add WebTorrent support Nov 21, 2016

bbondy closed this Nov 22, 2016

This was referenced Dec 12, 2016

Manual tests for Windows 0.13.0 Preview 3 Batch 2 #6165

Closed

Manual tests for OS X 0.13.0 Preview 3 Batch 2 #6166

Closed

This was referenced Dec 21, 2016

Manual tests for OS X 0.13.0 Preview 6 #6332

Closed

Manual tests for Windows ia-32 0.13.0 Preview 6 #6335

Closed

Manual tests for Windows x64 0.13.0 Preview 6 #6334

Closed

Manual tests for Linux Mint 0.13.0 Preview 6 #6337

Closed

feross mentioned this pull request Jun 27, 2017

IPFS Integration - Roadmap and discussion #9556

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add WebTorrent support #5342

Add WebTorrent support #5342

dcposch commented Nov 2, 2016 •

edited

Loading

bbondy commented Nov 2, 2016

dcposch commented Nov 2, 2016 •

edited

Loading

dcposch commented Nov 2, 2016 •

edited

Loading

dcposch Nov 3, 2016

diracdeltas Nov 3, 2016

dcposch Nov 9, 2016

diracdeltas Nov 3, 2016

feross Nov 3, 2016

diracdeltas Nov 3, 2016

feross Nov 3, 2016

diracdeltas Nov 4, 2016

diracdeltas commented Nov 3, 2016

feross commented Nov 3, 2016

feross commented Nov 3, 2016 •

edited

Loading

diracdeltas commented Nov 4, 2016

dcposch commented Nov 4, 2016

dcposch commented Nov 11, 2016

dcposch commented Nov 11, 2016

dcposch commented Nov 11, 2016 •

edited

Loading

feross commented Nov 14, 2016 •

edited

Loading

diracdeltas Nov 15, 2016 •

edited

Loading

dcposch Nov 15, 2016

dcposch Nov 15, 2016

dcposch commented Nov 19, 2016 •

edited

Loading

bbondy commented Nov 22, 2016

		@@ -0,0 +1,19 @@
		(function () {
		var queryString = window.location.search

Add WebTorrent support #5342

Add WebTorrent support #5342

Conversation

dcposch commented Nov 2, 2016 • edited Loading

Visit a magnet URL

Bookmark a magnet link

Play files in a torrent

Here are some of the things we still need to do:

bbondy commented Nov 2, 2016

dcposch commented Nov 2, 2016 • edited Loading

dcposch commented Nov 2, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

diracdeltas commented Nov 3, 2016

feross commented Nov 3, 2016

feross commented Nov 3, 2016 • edited Loading

diracdeltas commented Nov 4, 2016

dcposch commented Nov 4, 2016

New screenshots

dcposch commented Nov 11, 2016

dcposch commented Nov 11, 2016

dcposch commented Nov 11, 2016 • edited Loading

Work left to do

feross commented Nov 14, 2016 • edited Loading

diracdeltas Nov 15, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dcposch commented Nov 19, 2016 • edited Loading

I think we're ready

In this PR

Not in this PR

Latest screenshots

Regular webpage with magnet links

Magnet page

File list

Magnet page for an individual file

Download links

Bookmarks

Preferences

bbondy commented Nov 22, 2016

dcposch commented Nov 2, 2016 •

edited

Loading

dcposch commented Nov 2, 2016 •

edited

Loading

dcposch commented Nov 2, 2016 •

edited

Loading

feross commented Nov 3, 2016 •

edited

Loading

dcposch commented Nov 11, 2016 •

edited

Loading

feross commented Nov 14, 2016 •

edited

Loading

diracdeltas Nov 15, 2016 •

edited

Loading

dcposch commented Nov 19, 2016 •

edited

Loading