Skip to content
This repository has been archived by the owner on Feb 12, 2024. It is now read-only.

feat: pass libp2pOptions to the bundle function #2591

Merged
merged 2 commits into from
Nov 14, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 26 additions & 11 deletions src/core/components/libp2p.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,20 @@ module.exports = function libp2p (self, config) {
const options = self._options || {}
config = config || {}

// Always create libp2p via a bundle function
const createBundle = typeof options.libp2p === 'function'
? options.libp2p
: defaultBundle

const { datastore } = self._repo
const peerInfo = self._peerInfo
const peerBook = self._peerInfoBook
const libp2p = createBundle({ options, config, datastore, peerInfo, peerBook })

const libp2pOptions = getLibp2pOptions({ options, config, datastore, peerInfo, peerBook })
let libp2p

if (typeof options.libp2p === 'function') {
libp2p = options.libp2p({ libp2pOptions, options, config, datastore, peerInfo, peerBook })
} else {
// Required inline to reduce startup time
const Libp2p = require('libp2p')
libp2p = new Libp2p(mergeOptions(libp2pOptions, get(options, 'libp2p', {})))
}
Comment on lines +20 to +29
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, options is getting passed into getLibp2pOptions, but then it is still passed into libp2p creation. Should getLibp2pOptions just handle merging all of that so options doesn't need to be passed again?

options.libp2p({ libp2pOptions, options, config, datastore, peerInfo, peerBook })

Along those lines, why is options being passed here? It feels like the result of getLibp2pOptions should just be able to get passed directly to both new Libp2p and options.libp2p. No other params should be needed right?

Copy link
Member Author

@alanshaw alanshaw Nov 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should getLibp2pOptions just handle merging all of that so options doesn't need to be passed again?

There may be some other IPFS option you want to use in your custom bundle function. It's "passed again" because that's what currently happens, and I'm trying to avoid a breaking change.

options.libp2p({ libp2pOptions, options, config, datastore, peerInfo, peerBook })

Along those lines, why is options being passed here? It feels like the result of getLibp2pOptions should just be able to get passed directly to both new Libp2p and options.libp2p. No other params should be needed right?

options is IPFS options, passed to the IPFS constructor. It's passed here because that's currently what happens when you provide a custom bundle function. If we removed it, we'd have a breaking change.

getLibp2pOptions is passed IPFS options and returns libp2p options (I would call them defaults, but they're not really because they're based off the IPFS options and the IPFS config from the repo).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, so options and config are for IPFS? If so, would it be easier to understand if in the scope of the custom bundler options were the libp2p options and the IPFS ones were prefixed? If those are going to be exposed I think it might be helpful to make that distinction.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nvm, that would be a breaking change. I think that would be clearer, but since the goal is to avoid the breaking change I think this is fine.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that would be clearer

FWIW I agree!


libp2p.on('stop', () => {
// Clear our addresses so we can start clean
Expand All @@ -39,7 +44,7 @@ module.exports = function libp2p (self, config) {
return libp2p
}

function defaultBundle ({ datastore, peerInfo, peerBook, options, config }) {
function getLibp2pOptions ({ options, config, datastore, peerInfo, peerBook }) {
// Set up Delegate Routing based on the presence of Delegates in the config
let contentRouting
let peerRouting
Expand Down Expand Up @@ -80,7 +85,12 @@ function defaultBundle ({ datastore, peerInfo, peerBook, options, config }) {
peerBook,
modules: {
contentRouting,
peerRouting,
peerRouting
}
}

const libp2pOptions = {
modules: {
pubsub: getPubsubRouter()
},
config: {
Expand Down Expand Up @@ -133,9 +143,14 @@ function defaultBundle ({ datastore, peerInfo, peerBook, options, config }) {
})
}

const libp2pOptions = mergeOptions(libp2pDefaults, get(options, 'libp2p', {}))
// Required inline to reduce startup time
// Note: libp2p-nodejs gets replaced by libp2p-browser when webpacked/browserified
const Node = require('../runtime/libp2p-nodejs')
return new Node(libp2pOptions)
const getEnvLibp2pOptions = require('../runtime/libp2p-nodejs')

// Merge defaults with Node.js/browser/other environments options and configuration
return mergeOptions(
libp2pDefaults,
getEnvLibp2pOptions({ options, config, datastore, peerInfo, peerBook }),
Copy link
Member

@lidel lidel Nov 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: easier to read if options-to-be-merged follow same naming convention (also, runtime is the name of directory, so let's reuse that name)

Suggested change
getEnvLibp2pOptions({ options, config, datastore, peerInfo, peerBook }),
libp2pRuntimeDefaults({ options, config, datastore, peerInfo, peerBook }),

libp2pOptions
)
}
110 changes: 51 additions & 59 deletions src/core/runtime/libp2p-browser.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,73 +8,65 @@ const SECIO = require('libp2p-secio')
const Bootstrap = require('libp2p-bootstrap')
const KadDHT = require('libp2p-kad-dht')
const GossipSub = require('libp2p-gossipsub')
const libp2p = require('libp2p')
const mergeOptions = require('merge-options')
const multiaddr = require('multiaddr')

class Node extends libp2p {
constructor (_options) {
const wrtcstar = new WebRTCStar({ id: _options.peerInfo.id })
module.exports = ({ peerInfo, options }) => {
const wrtcstar = new WebRTCStar({ id: peerInfo.id })

// this can be replaced once optional listening is supported with the below code. ref: https://github.com/libp2p/interface-transport/issues/41
// const wsstar = new WebSocketStar({ id: _options.peerInfo.id })
const wsstarServers = _options.peerInfo.multiaddrs.toArray().map(String).filter(addr => addr.includes('p2p-websocket-star'))
_options.peerInfo.multiaddrs.replace(wsstarServers.map(multiaddr), '/p2p-websocket-star') // the ws-star-multi module will replace this with the chosen ws-star servers
const wsstar = new WebSocketStarMulti({ servers: wsstarServers, id: _options.peerInfo.id, ignore_no_online: !wsstarServers.length || _options.wsStarIgnoreErrors })
// this can be replaced once optional listening is supported with the below code. ref: https://github.com/libp2p/interface-transport/issues/41
// const wsstar = new WebSocketStar({ id: _options.peerInfo.id })
const wsstarServers = peerInfo.multiaddrs.toArray().map(String).filter(addr => addr.includes('p2p-websocket-star'))
peerInfo.multiaddrs.replace(wsstarServers.map(multiaddr), '/p2p-websocket-star') // the ws-star-multi module will replace this with the chosen ws-star servers
const wsstar = new WebSocketStarMulti({ servers: wsstarServers, id: peerInfo.id, ignore_no_online: !wsstarServers.length || options.wsStarIgnoreErrors })

const defaults = {
switch: {
denyTTL: 2 * 60 * 1e3, // 2 minute base
denyAttempts: 5, // back off 5 times
maxParallelDials: 100,
maxColdCalls: 25,
dialTimeout: 20e3
},
modules: {
transport: [
WS,
wrtcstar,
wsstar
],
streamMuxer: [
Multiplex
],
connEncryption: [
SECIO
],
peerDiscovery: [
wrtcstar.discovery,
wsstar.discovery,
Bootstrap
],
dht: KadDHT,
pubsub: GossipSub
},
config: {
peerDiscovery: {
autoDial: true,
bootstrap: {
enabled: true
},
webRTCStar: {
enabled: true
},
websocketStar: {
enabled: true
}
return {
switch: {
denyTTL: 2 * 60 * 1e3, // 2 minute base
denyAttempts: 5, // back off 5 times
maxParallelDials: 100,
maxColdCalls: 25,
dialTimeout: 20e3
},
modules: {
transport: [
WS,
wrtcstar,
wsstar
],
streamMuxer: [
Multiplex
],
connEncryption: [
SECIO
],
peerDiscovery: [
wrtcstar.discovery,
wsstar.discovery,
Bootstrap
],
dht: KadDHT,
pubsub: GossipSub
},
config: {
peerDiscovery: {
autoDial: true,
bootstrap: {
enabled: true
},
dht: {
enabled: false
webRTCStar: {
enabled: true
},
pubsub: {
enabled: true,
emitSelf: true
websocketStar: {
enabled: true
}
},
dht: {
enabled: false
},
pubsub: {
enabled: true,
emitSelf: true
}
}

super(mergeOptions(defaults, _options))
}
}

module.exports = Node
116 changes: 54 additions & 62 deletions src/core/runtime/libp2p-nodejs.js
Original file line number Diff line number Diff line change
Expand Up @@ -9,75 +9,67 @@ const KadDHT = require('libp2p-kad-dht')
const GossipSub = require('libp2p-gossipsub')
const Multiplex = require('pull-mplex')
const SECIO = require('libp2p-secio')
const libp2p = require('libp2p')
const mergeOptions = require('merge-options')
const multiaddr = require('multiaddr')

class Node extends libp2p {
constructor (_options) {
// this can be replaced once optional listening is supported with the below code. ref: https://github.com/libp2p/interface-transport/issues/41
// const wsstar = new WebSocketStar({ id: _options.peerInfo.id })
const wsstarServers = _options.peerInfo.multiaddrs.toArray().map(String).filter(addr => addr.includes('p2p-websocket-star'))
_options.peerInfo.multiaddrs.replace(wsstarServers.map(multiaddr), '/p2p-websocket-star') // the ws-star-multi module will replace this with the chosen ws-star servers
const wsstar = new WebSocketStarMulti({ servers: wsstarServers, id: _options.peerInfo.id, ignore_no_online: !wsstarServers.length || _options.wsStarIgnoreErrors })
module.exports = ({ peerInfo, options }) => {
// this can be replaced once optional listening is supported with the below code. ref: https://github.com/libp2p/interface-transport/issues/41
// const wsstar = new WebSocketStar({ id: _options.peerInfo.id })
const wsstarServers = peerInfo.multiaddrs.toArray().map(String).filter(addr => addr.includes('p2p-websocket-star'))
peerInfo.multiaddrs.replace(wsstarServers.map(multiaddr), '/p2p-websocket-star') // the ws-star-multi module will replace this with the chosen ws-star servers
const wsstar = new WebSocketStarMulti({ servers: wsstarServers, id: peerInfo.id, ignore_no_online: !wsstarServers.length || options.wsStarIgnoreErrors })

const defaults = {
switch: {
denyTTL: 2 * 60 * 1e3, // 2 minute base
denyAttempts: 5, // back off 5 times
maxParallelDials: 150,
maxColdCalls: 50,
dialTimeout: 10e3 // Be strict with dial time
},
modules: {
transport: [
TCP,
WS,
wsstar
],
streamMuxer: [
Multiplex
],
connEncryption: [
SECIO
],
peerDiscovery: [
MulticastDNS,
Bootstrap,
wsstar.discovery
],
dht: KadDHT,
pubsub: GossipSub
},
config: {
peerDiscovery: {
autoDial: true,
mdns: {
enabled: true
},
bootstrap: {
enabled: true
},
websocketStar: {
enabled: true
}
return {
switch: {
denyTTL: 2 * 60 * 1e3, // 2 minute base
denyAttempts: 5, // back off 5 times
maxParallelDials: 150,
maxColdCalls: 50,
dialTimeout: 10e3 // Be strict with dial time
},
modules: {
transport: [
TCP,
WS,
wsstar
],
streamMuxer: [
Multiplex
],
connEncryption: [
SECIO
],
peerDiscovery: [
MulticastDNS,
Bootstrap,
wsstar.discovery
],
dht: KadDHT,
pubsub: GossipSub
},
config: {
peerDiscovery: {
autoDial: true,
mdns: {
enabled: true
},
dht: {
kBucketSize: 20,
enabled: false,
randomWalk: {
enabled: false
}
bootstrap: {
enabled: true
},
pubsub: {
enabled: true,
emitSelf: true
websocketStar: {
enabled: true
}
},
dht: {
kBucketSize: 20,
enabled: false,
randomWalk: {
enabled: false
}
},
pubsub: {
enabled: true,
emitSelf: true
}
}

super(mergeOptions(defaults, _options))
}
}

module.exports = Node
33 changes: 33 additions & 0 deletions test/core/libp2p.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,39 @@ describe('libp2p customization', function () {
done()
})
})

it('should pass libp2p options to libp2p bundle function', (done) => {
class DummyTransport {
filter () {
return []
}
}

const ipfs = {
_repo: {
datastore
},
_peerInfo: peerInfo,
_peerBook: peerBook,
// eslint-disable-next-line no-console
_print: console.log,
_options: {
libp2p: ({ libp2pOptions, peerInfo }) => {
libp2pOptions.modules.transport = [DummyTransport]
return new Libp2p(libp2pOptions)
}
}
}

_libp2p = libp2pComponent(ipfs, testConfig)

_libp2p.start((err) => {
expect(err).to.not.exist()
expect(_libp2p._transport).to.have.length(1)
expect(_libp2p._transport[0] instanceof DummyTransport).to.equal(true)
done()
})
})
})

describe('options', () => {
Expand Down