Select custom Chunk Sizes

[ya7ya]

• Version: 0.26.0
• Platform: Chrome v57 and Chrome Version 61.0.3163.91 (Official Build) (64-bit) , Ubuntu 16.04, This doesn't happen when testing on firefox
• Subsystem: probably simple-peer or webRTC in chrome or js-libp2p-webrtc

Type: Bug

Severity: High

Description:

browser to browser file exchange through webRTC doesn't work, The packets are sent from the node providing the content. but the peer doesn't get it and disconnects the moment before the block gets there.

Steps to reproduce the error:

1. clone this repo
2. go to examples/exchange-files-in-browser
3. npm install
4. npm start
5. start 2 tabs pointing to http://localhost:12345 and click Start IPFS in both tabs.
6. upload a file in one tab, Make Sure the file doesn't exist on the bootstrapping nodes, generate a random file or something.
7. make sure both tabs list the other one as a connected peer.
8. request the file from the other peer. note how the peer with the file gets disconnected after requesting the file and getting the initial response with file size and path. and then keeps flickering ( connecting and disconnecting with every block )

[ya7ya]

okay , so I did more digging around, and noticed the same issue happens when using js-libp2p-websocket-star, it works fine on firefox but not on Chrome.

[daviddias]

@ya7ya are you sure you are not experiencing #611 ?

[ya7ya]

@diasdavid I'm not sure if it's #611 , It could be the case, but I wanted to make sure it wasn't just me who can produce this issue. and I was hoping it's something we can fix 😄, How would one go about confirming if it's chrome throttling the tab or not ??

[daviddias]

@ya7ya you can test by opening two chrome windows side by side (it is important that you can see both pages at the same time) and load the page. If it works, then it is #611

[ya7ya]

hey @diasdavid , I tested it like specified, data is exchanging but the peer is disconnected with every block. Check out the gif for the exchange
you can notice the flickering of peers when the file transfer starts.

[daviddias]

@ya7ya which version of the exchange files demo are you using? The one in master or the one in the fork?

[ya7ya]

I'm using the one in master branch, With a couple of simple additions so it won't use the Bootstrap nodes, and to log the bitswap ledgerMap every 5 seconds.

function start () {
  if (!node) {
    updateView('starting', node)

    node = new self.Ipfs({
      repo: 'ipfs-' + Math.random(),
      config: {
        Addresses: {
          Swarm: [
            '/dns4/star-signal.cloud.ipfs.team/wss/p2p-webrtc-star'
          ]
        },
        Bootstrap: []
      }
    })

    node.on('start', () => {
      node.id().then((id) => {
        peerInfo = id
        updateView('ready', node)
        setInterval(refreshPeerList, 1000)
        $peers.innerHTML = '<h2>peers</h2><i>waiting for peers...</i>'
      })

      setInterval(() => {
        node._bitswap.engine.ledgerMap.forEach((ledger, peerId, ledgerMap) => {
          console.log(`${peerId} : ${JSON.stringify(ledger.accounting)}\n`)
        })
      }, 5000)
    })
  }
}

[ya7ya]

Hey @diasdavid , I did more investigating regarding this issue. When enabling DEBUG in the browser, I noticed a DHT is not available Error. and simple-peer on the receiver side disconnects with (error : undefined)
The simple-peer error (receiver side)

19:14:27.745 VM85 index.js:108045 simple-peer [0059233] iceStateChange (connection: disconnected) (gathering: complete) +4s
19:14:27.746 VM85 index.js:108045 simple-peer [0059233] destroy (error: undefined) +1ms
19:14:27.753 VM85 index.js:43129 mplex:main:35792 destroy +6s
19:14:27.754 VM85 index.js:43129 mplex:main:35792 destroy +1ms
19:14:27.755 VM85 index.js:43129 mplex:main:35792 already destroyed +1ms
19:14:27.756 VM85 index.js:97708 mss:listener	:error (3ekqkh) Error: underlying socket has been closed
    at http://localhost:12345/js/dist/index.js:97570:37
    at Array.forEach (<anonymous>)
    at Multiplex.destroy (http://localhost:12345/js/dist/index.js:97568:10)
    at http://localhost:12345/js/dist/index.js:102728:45
    at call (http://localhost:12345/js/dist/index.js:102735:3)
    at Array.forEach (<anonymous>)
    at http://localhost:12345/js/dist/index.js:102755:25
    at f (http://localhost:12345/js/dist/index.js:14658:25)
    at Stream.<anonymous> (http://localhost:12345/js/dist/index.js:102714:21)
    at Stream.f (http://localhost:12345/js/dist/index.js:14658:25) +6s
19:14:27.759 VM85 index.js:97708 mss:listener	:error (3ekqkh) Error: underlying socket has been closed
    at http://localhost:12345/js/dist/index.js:97570:37
    at Array.forEach (<anonymous>)
    at Multiplex.destroy (http://localhost:12345/js/dist/index.js:97568:10)
    at http://localhost:12345/js/dist/index.js:102728:45
    at call (http://localhost:12345/js/dist/index.js:102735:3)
    at Array.forEach (<anonymous>)
    at http://localhost:12345/js/dist/index.js:102755:25
    at f (http://localhost:12345/js/dist/index.js:14658:25)
    at Stream.<anonymous> (http://localhost:12345/js/dist/index.js:102714:21)
    at Stream.f (http://localhost:12345/js/dist/index.js:14658:25) +3ms

Uploader Debug log

19:14:23.853 VM98 index.js:3290 bitswap:QmW6GB85:error Failed to provide: DHT is not available +338ms
19:14:23.859 VM98 index.js:3290 bitswap:QmW6GB85 getMany +334ms 1
19:14:23.869 VM98 index.js:3290 bitswap:QmW6GB85:error Error: DHT is not available
    at Object.findProviders (http://localhost:12345/js/dist/index.js:78928:27)
    at Network.findProviders (http://localhost:12345/js/dist/index.js:66746:32)
    at waterfall (http://localhost:12345/js/dist/index.js:66751:20)
    at nextTask (http://localhost:12345/js/dist/index.js:2227:14)
    at exports.default (http://localhost:12345/js/dist/index.js:2237:5)
    at Network.findAndConnect (http://localhost:12345/js/dist/index.js:66750:5)
    at each (http://localhost:12345/js/dist/index.js:66493:20)
    at http://localhost:12345/js/dist/index.js:22080:16
    at iteratorCallback (http://localhost:12345/js/dist/index.js:21978:13)
    at http://localhost:12345/js/dist/index.js:11204:16 +16ms
19:14:26.927 VM98 index.js:108045 simple-peer [4dc4dda] iceStateChange (connection: disconnected) (gathering: complete) +5s
19:14:26.927 VM98 index.js:108045 simple-peer [4dc4dda] destroy (error: undefined) +0ms
19:14:27.128 VM98 index.js:108045 simple-peer [f5d834c] iceStateChange (connection: disconnected) (gathering: complete) +201ms
19:14:27.128 VM98 index.js:108045 simple-peer [f5d834c] destroy (error: undefined) +0ms
19:14:27.136 VM98 index.js:43129 mplex:main:31004 destroy +5s
19:14:27.137 VM98 index.js:43129 mplex:main:31004 destroy +1ms
19:14:27.138 VM98 index.js:43129 mplex:main:31004 already destroyed +1ms
19:14:27.139 VM98 index.js:3290 libp2p:swarm:connection Identify not successful +0ms
19:14:27.140 VM98 index.js:43129 mplex:main:31004 _clear +2ms
19:14:27.142 VM98 index.js:43129 mplex:channel:1 _destroy:remote +2ms
19:14:27.144 VM98 index.js:43129 mplex:main:31004 _remove channel +1ms 1
19:14:27.725 VM98 index.js:43129 mplex:main:64046 _write +582ms 2
19:14:27.727 VM98 index.js:43129 mplex:main:64046 _write +2ms 20
19:14:27.729 VM98 index.js:43129 mplex:main:64046 _push +1ms 20
19:14:27.730 VM98 index.js:97708 mss:listener	 (a4slnd) received ack: /multistream/1.0.0 +6s
19:14:27.731 VM98 index.js:43129 mplex:main:64046 _write +3ms 2
19:14:27.733 VM98 index.js:43129 mplex:main:64046 _write +2ms 17
19:14:27.734 VM98 index.js:43129 mplex:main:64046 _push +1ms 17
19:14:27.735 VM98 index.js:97708 mss:listener	 (a4slnd) received: +5ms
19:14:27.736 VM98 index.js:97708 mss:listener	 (a4slnd) not supported protocol: /floodsub/1.0.0 +1ms
19:14:27.738 VM98 index.js:43129 mplex:channel:3 write:  +4ms 4
19:14:27.740 VM98 index.js:43129 mplex:main:64046 _send +2ms 25 4

When running the same experiment on Firefox, The peers manage to send few blocks before the same issue appears, where in chrome, there is only a single packet (the initial 1079 bytes) that make it through.

So Do you think this has something to do with block size? How do we go from here ?? :D , please advise 🙇 😄

[ya7ya]

So I think I figured this one out, When I used smaller maxChunkSize in ipfs-unixfs-engine builder, and smaller MAX_MESSAGE_SIZE in ipfs-bitswap , Chrome didn't throttle the connection and the file was transferred successfully.

I'm gonna experiment/tweak this and hopefully push some code by tomorrow 👍

[daviddias]

@ya7ya mind sharing an update on this? I'm interested to learn what you found.

[ya7ya]

@diasdavid Hey , Sorry about the late reply. I missed the notification.
I did manage to bypass the throttling issue by reducing the size of the maxChunkSize and MAX_MESSAGE_SIZE for ipfs-unix-engine and ipfs-bitswap. It works 👍

for reference, this PR ipfs/js-ipfs-bitswap#152 has the needed modifications, I got caught up in other tasks so I didn't finish the docs yet, I'll try to wrap this up ASAP 😄

[whyrusleeping]

This just sounds like the stream multiplexer library needs to have an MTU setting

[survirtual]

This bug is a result of Chrome being unable to send messages to Firefox > 16kb through WebRTC, and Chrome being unable to send messages to Chrome > 64kb. Here's an article that led me to that size: http://viblast.com/blog/2015/2/5/webrtc-data-channel-message-size/

I've tested and confirmed this behavior between Firefox and chrome. Changing the maxChunkSize and MAX_MESSAGE_SIZE to 16kb resolve the issue but changes the file hashes, and I didn't test interop to nodes without those changes.

With those two values changed, browser to browser WebRTC works fine.

[achingbrain]

@vasco-santos @jacobheun just going through old issues - is this on your radar?

Does the webrtc-star transport address the inter-browser packet size issue here?

[achingbrain]

Actually, I can't replicate this locally - the exchange files in browser example is happy to exchange data way over the max chunk size via the webrtc star transport so I'm going to close this as stale. Plenty of underlying code has changed since this was opened so the problem may have been fixed.

Please reopen if I've missed how this is not working.