Skip to content
This repository has been archived by the owner on Feb 12, 2024. It is now read-only.

feat: libp2p async peerstore #4018

Merged
merged 49 commits into from
Jan 27, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
896bb4b
feat: libp2p async peerstore
achingbrain Jan 18, 2022
6c2a4ce
chore: update crypto
achingbrain Jan 18, 2022
c7f5e69
feat: add lru datastore cache for peer store data
achingbrain Jan 19, 2022
0bf01d7
chore: add missing dep
achingbrain Jan 19, 2022
e093168
chore: update bitswap version
achingbrain Jan 20, 2022
4bcfae1
chore: simplify build and fix up exports
achingbrain Jan 20, 2022
330d6fd
chore: fix up exports
achingbrain Jan 20, 2022
c2d0014
Merge remote-tracking branch 'origin/master' into feat/async-peerstore
achingbrain Jan 20, 2022
68d32b8
chore: examples need update
achingbrain Jan 20, 2022
c485105
chore: update readmes
achingbrain Jan 21, 2022
9a9d9b8
chore: but y tho
achingbrain Jan 21, 2022
22a1f7c
chore: but y tho
achingbrain Jan 21, 2022
42c7cf9
chore: disable dht in offline tests
achingbrain Jan 24, 2022
7d0a126
chore: swap gossipsub fork
achingbrain Jan 24, 2022
45f1597
chore: allow disabling dht and make sure we have an ipns datastore av…
achingbrain Jan 24, 2022
f421696
chore: swap message back
achingbrain Jan 24, 2022
24ecda5
chore: use libp2p version
achingbrain Jan 25, 2022
36da9e0
chore: remove ignore
achingbrain Jan 25, 2022
1f96d48
chore: rev libp2p
achingbrain Jan 25, 2022
8e7b0c3
Merge remote-tracking branch 'origin/master' into feat/async-peerstore
achingbrain Jan 25, 2022
8ddc7d5
chore: linting
achingbrain Jan 25, 2022
69520f7
chore: update test
achingbrain Jan 25, 2022
be00cbd
chore: do not abort after successful swarm dial
achingbrain Jan 26, 2022
db5eb2f
chore: update deps
achingbrain Jan 26, 2022
fa6a980
Merge remote-tracking branch 'origin/master' into feat/async-peerstore
achingbrain Jan 26, 2022
088c823
chore: update dep
achingbrain Jan 26, 2022
d12d1ee
chore: try async ipns init
achingbrain Jan 27, 2022
78a5710
chore: linting
achingbrain Jan 27, 2022
7ddf594
chore: fix breakage from murmur3
achingbrain Jan 27, 2022
364aeb2
chore: did something change in our rsa impl
achingbrain Jan 27, 2022
d3f1a5c
chore: revert startup change
achingbrain Jan 27, 2022
48b2092
chore: debug test
achingbrain Jan 27, 2022
b908b69
chore: add logging
achingbrain Jan 27, 2022
65a4c90
chore: run core interface
achingbrain Jan 27, 2022
1f0acf4
chore: run http interface
achingbrain Jan 27, 2022
a0237b2
chore: run directly
achingbrain Jan 27, 2022
aca95cb
chore: revert murmur change
achingbrain Jan 27, 2022
c0b9975
chore: restore build files
achingbrain Jan 27, 2022
d92a903
chore: run playwright install manually
achingbrain Jan 27, 2022
ce771eb
chore: add npx
achingbrain Jan 27, 2022
a0573f6
chore: use action for browser deps
achingbrain Jan 27, 2022
18b3ade
chore: do not run loads of extra builds
achingbrain Jan 27, 2022
99c5250
chore: wat
achingbrain Jan 27, 2022
2b4e195
chore: update build
achingbrain Jan 27, 2022
fd5c9bb
chore: run one test
achingbrain Jan 27, 2022
14de38d
chore: run two jobs
achingbrain Jan 27, 2022
e37cc3e
chore: disable caching
achingbrain Jan 27, 2022
060a90d
chore: update build
achingbrain Jan 27, 2022
ed5da91
chore: update fetch depth
achingbrain Jan 27, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 26 additions & 45 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
with:
node-version: 16
node-version: lts/*
- uses: ipfs/aegir/actions/cache-node-modules@master
with:
build: |
Expand All @@ -30,7 +30,7 @@ jobs:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
with:
node-version: 16
node-version: lts/*
- uses: ipfs/aegir/actions/cache-node-modules@master
with:
build: |
Expand All @@ -41,7 +41,7 @@ jobs:
npm run dep-check -- -- -- -- --unused

test-node:
name: Unit tests ${{ matrix.project }} node ${{ matrix.node }} ${{ matrix.os }}
name: Unit tests node ${{ matrix.node }} ${{ matrix.os }}
needs: build
runs-on: ${{ matrix.os }}
strategy:
Expand All @@ -62,11 +62,10 @@ jobs:
- run: npm run test:node -- --since ${{ github.event.pull_request.base.sha }} --concurrency 1
- uses: codecov/codecov-action@f32b3a3741e1053eb607407145bc9619351dc93b # v2.1.0
with:
directory: '**/.nyc_output'
flags: node

test-chrome:
name: Unit tests ${{ matrix.project }} ${{ matrix.browser }} ${{ matrix.type }}
name: Unit tests chrome
needs: build
runs-on: ubuntu-latest
steps:
Expand All @@ -83,21 +82,12 @@ jobs:
- run: npm run test:chrome -- --since ${{ github.event.pull_request.base.sha }} --concurrency 1
- uses: codecov/codecov-action@f32b3a3741e1053eb607407145bc9619351dc93b # v2.1.0
with:
directory: '**/.nyc_output'
flags: chrome

test-chrome-webworker:
name: Unit tests ${{ matrix.project }} ${{ matrix.browser }} ${{ matrix.type }}
name: Unit tests chrome-webworker
needs: build
runs-on: ubuntu-latest
strategy:
matrix:
browser:
- chromium
- firefox
type:
- browser
- webworker
steps:
- uses: actions/checkout@v2
with:
Expand All @@ -112,11 +102,10 @@ jobs:
- run: npm run test:chrome-webworker -- --since ${{ github.event.pull_request.base.sha }} --concurrency 1
- uses: codecov/codecov-action@f32b3a3741e1053eb607407145bc9619351dc93b # v2.1.0
with:
directory: '**/.nyc_output'
flags: chrome-webworker

test-firefox:
name: Unit tests ${{ matrix.project }} ${{ matrix.browser }} ${{ matrix.type }}
name: Unit tests firefox
needs: build
runs-on: ubuntu-latest
steps:
Expand All @@ -133,21 +122,12 @@ jobs:
- run: npm run test:firefox -- --since ${{ github.event.pull_request.base.sha }} --concurrency 1
- uses: codecov/codecov-action@f32b3a3741e1053eb607407145bc9619351dc93b # v2.1.0
with:
directory: '**/.nyc_output'
flags: firefox

test-firefox-webworker:
name: Unit tests ${{ matrix.project }} ${{ matrix.browser }} ${{ matrix.type }}
name: Unit tests firefox-webworker
needs: build
runs-on: ubuntu-latest
strategy:
matrix:
browser:
- chromium
- firefox
type:
- browser
- webworker
steps:
- uses: actions/checkout@v2
with:
Expand All @@ -159,10 +139,10 @@ jobs:
with:
build: |
npm run link
- run: npx playwright install --with-deps
- run: npm run test:firefox-webworker -- --since ${{ github.event.pull_request.base.sha }} --concurrency 1
- uses: codecov/codecov-action@f32b3a3741e1053eb607407145bc9619351dc93b # v2.1.0
with:
directory: '**/.nyc_output'
flags: firefox-webworker

test-electron-main:
Expand All @@ -175,7 +155,7 @@ jobs:
fetch-depth: 0
- uses: actions/setup-node@v2
with:
node-version: 16
node-version: lts/*
- uses: ipfs/aegir/actions/cache-node-modules@master
with:
build: |
Expand All @@ -185,7 +165,6 @@ jobs:
run: npm run test:electron-main -- --since ${{ github.event.pull_request.base.sha }} --concurrency 1 -- -- --bail
- uses: codecov/codecov-action@f32b3a3741e1053eb607407145bc9619351dc93b # v2.1.0
with:
directory: '**/.nyc_output'
flags: electron-main

test-electron-renderer:
Expand All @@ -198,7 +177,7 @@ jobs:
fetch-depth: 0
- uses: actions/setup-node@v2
with:
node-version: 16
node-version: lts/*
- uses: ipfs/aegir/actions/cache-node-modules@master
with:
build: |
Expand All @@ -208,12 +187,10 @@ jobs:
run: npm run test:electron-renderer -- --since ${{ github.event.pull_request.base.sha }} --concurrency 1 -- -- --bail
- uses: codecov/codecov-action@f32b3a3741e1053eb607407145bc9619351dc93b # v2.1.0
with:
directory: '**/.nyc_output'
flags: electron-renderer

test-interop:
name: Interop tests ${{ matrix.project }} ${{ matrix.type }}
needs: build
runs-on: ubuntu-latest
strategy:
matrix:
Expand All @@ -228,20 +205,22 @@ jobs:
fetch-depth: 0
- uses: actions/setup-node@v2
with:
node-version: 16
node-version: lts/*
- uses: ipfs/aegir/actions/cache-node-modules@master
with:
build: |
npm run link
- run: npm run test:interop -- --since ${{ github.event.pull_request.base.sha }} --concurrency 1 -- -- -- -t ${{ matrix.type }}
- run: |
npm install
npm run build
npm run link
npm run test:interop -- --since ${{ github.event.pull_request.base.sha }} --concurrency 1 -- -- -- -t ${{ matrix.type }}
- uses: codecov/codecov-action@f32b3a3741e1053eb607407145bc9619351dc93b # v2.1.0
with:
directory: '**/.nyc_output'
flags: interop-${{ matrix.type }}

test-interface:
name: Interface tests ${{ matrix.suite }} ${{ matrix.type }}
needs: build
runs-on: ubuntu-latest
strategy:
matrix:
Expand All @@ -261,15 +240,14 @@ jobs:
fetch-depth: 0
- uses: actions/setup-node@v2
with:
node-version: 16
- uses: ipfs/aegir/actions/cache-node-modules@master
with:
build: |
npm run link
- run: npm run ${{ matrix.suite }} -- --since ${{ github.event.pull_request.base.sha }} --concurrency 1 -- -- -t ${{ matrix.type }}
node-version: lts/*
- run: |
npm install
npm run build
npm run link
npm run ${{ matrix.suite }} -- --since ${{ github.event.pull_request.base.sha }} --concurrency 1 -- -- -t ${{ matrix.type }}
- uses: codecov/codecov-action@f32b3a3741e1053eb607407145bc9619351dc93b # v2.1.0
with:
directory: '**/.nyc_output'
flags: interface-${{ matrix.type }}

test-interface-message-port-client:
Expand All @@ -282,11 +260,12 @@ jobs:
fetch-depth: 0
- uses: actions/setup-node@v2
with:
node-version: 16
node-version: lts/*
- uses: ipfs/aegir/actions/cache-node-modules@master
with:
build: |
npm run link
- run: npx playwright install --with-deps
- run: npm run test:interface:message-port-client -- --since ${{ github.event.pull_request.base.sha }} --concurrency 1

release:
Expand Down Expand Up @@ -314,6 +293,8 @@ jobs:
manifest-file: .release-please-manifest.json
config-file: .release-please.json
- uses: actions/checkout@v2
with:
fetch-depth: 0
- uses: actions/setup-node@v2
with:
node-version: lts/*
Expand Down
4 changes: 0 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,10 +36,6 @@ We've come a long way, but this project is still in Alpha, lots of development i
* See https://blog.ipfs.io for news and more
* Need help? Please ask 'How do I?' questions on https://discuss.ipfs.io

## Lead Maintainer <!-- omit in toc -->

[Alex Potsides](http://github.com/achingbrain)

## Table of Contents <!-- omit in toc -->

- [Getting started](#getting-started)
Expand Down
4 changes: 0 additions & 4 deletions packages/interface-ipfs-core/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,6 @@

> A test suite and interface you can use to implement an IPFS core interface.

## Lead Maintainer <!-- omit in toc -->

[Alex Potsides](http://github.com/achingbrain)

## Table of Contents <!-- omit in toc -->

- [Background](#background)
Expand Down
3 changes: 2 additions & 1 deletion packages/interface-ipfs-core/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -89,8 +89,9 @@
"it-pushable": "^1.4.2",
"it-tar": "^4.0.0",
"it-to-buffer": "^2.0.0",
"libp2p-crypto": "^0.21.0",
"libp2p-crypto": "^0.21.1",
"libp2p-websockets": "^0.16.2",
"merge-options": "^3.0.4",
"multiaddr": "^10.0.0",
"multiformats": "^9.4.13",
"nanoid": "^3.1.23",
Expand Down
2 changes: 1 addition & 1 deletion packages/interface-ipfs-core/src/miscellaneous/id.js
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ export function testId (factory, options) {

expect(res).to.have.a.property('protocols').that.is.an('array')

expect(res.protocols).to.have.members([
expect(res.protocols).to.include.members([
'/floodsub/1.0.0',
'/ipfs/bitswap/1.0.0',
'/ipfs/bitswap/1.1.0',
Expand Down
42 changes: 35 additions & 7 deletions packages/interface-ipfs-core/src/miscellaneous/resolve.js
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import { getDescribe, getIt } from '../utils/mocha.js'
import all from 'it-all'
import { isWebWorker } from 'ipfs-utils/src/env.js'
import { ipfsOptionsWebsocketsFilterAll } from '../utils/ipfs-options-websockets-filter-all.js'
import merge from 'merge-options'

/**
* @typedef {import('ipfsd-ctl').Factory} Factory
Expand All @@ -31,7 +32,16 @@ export function testResolve (factory, options) {
let ipfsId

before(async () => {
ipfs = (await factory.spawn({ type: 'proc', ipfsOptions })).api
ipfs = (await factory.spawn({
type: 'proc',
ipfsOptions: merge(ipfsOptions, {
config: {
Routing: {
Type: 'none'
}
}
})
})).api
ipfsId = await ipfs.id()
})

Expand Down Expand Up @@ -97,7 +107,16 @@ export function testResolve (factory, options) {
// @ts-ignore this is mocha
this.timeout(20 * 1000)
// webworkers are not dialable because webrtc is not available
const node = (await factory.spawn({ type: isWebWorker ? 'go' : undefined })).api
const node = (await factory.spawn({
type: isWebWorker ? 'go' : undefined,
ipfsOptions: {
config: {
Routing: {
Type: 'none'
}
}
}
})).api
const nodeId = await node.id()
await ipfs.swarm.connect(nodeId.addresses[0])
const { path } = await ipfs.add(uint8ArrayFromString('should resolve a record recursive === true'))
Expand All @@ -106,15 +125,24 @@ export function testResolve (factory, options) {
await ipfs.name.publish(path, { allowOffline: true })
await ipfs.name.publish(`/ipns/${ipfsId.id}`, { allowOffline: true, key: 'key-name', resolve: false })

return expect(await ipfs.resolve(`/ipns/${keyId}`))
.to.eq(`/ipfs/${path}`)
return expect(ipfs.resolve(`/ipns/${keyId}`))
.to.eventually.equal(`/ipfs/${path}`)
})

it('should resolve IPNS link non-recursively if recursive==false', async function () {
// @ts-ignore this is mocha
this.timeout(20 * 1000)
// webworkers are not dialable because webrtc is not available
const node = (await factory.spawn({ type: isWebWorker ? 'go' : undefined })).api
const node = (await factory.spawn({
type: isWebWorker ? 'go' : undefined,
ipfsOptions: {
config: {
Routing: {
Type: 'none'
}
}
}
})).api
const nodeId = await node.id()
await ipfs.swarm.connect(nodeId.addresses[0])
const { path } = await ipfs.add(uint8ArrayFromString('should resolve an IPNS key if recursive === false'))
Expand All @@ -123,8 +151,8 @@ export function testResolve (factory, options) {
await ipfs.name.publish(path, { allowOffline: true })
await ipfs.name.publish(`/ipns/${ipfsId.id}`, { allowOffline: true, key: 'new-key-name', resolve: false })

return expect(await ipfs.resolve(`/ipns/${keyId}`, { recursive: false }))
.to.eq(`/ipns/${ipfsId.id}`)
return expect(ipfs.resolve(`/ipns/${keyId}`, { recursive: false }))
.to.eventually.equal(`/ipns/${ipfsId.id}`)
})
})
}
10 changes: 9 additions & 1 deletion packages/interface-ipfs-core/src/name/publish.js
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,15 @@ export function testPublish (factory, options) {
let nodeId

before(async () => {
ipfs = (await factory.spawn()).api
ipfs = (await factory.spawn({
ipfsOptions: {
config: {
Routing: {
Type: 'none'
}
}
}
})).api
const peerInfo = await ipfs.id()
nodeId = peerInfo.id
await ipfs.add(fixture.data, { pin: false })
Expand Down
10 changes: 9 additions & 1 deletion packages/interface-ipfs-core/src/name/resolve.js
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,15 @@ export function testResolve (factory, options) {
let nodeId

before(async () => {
ipfs = (await factory.spawn()).api
ipfs = (await factory.spawn({
ipfsOptions: {
config: {
Routing: {
Type: 'none'
}
}
}
})).api
const peerInfo = await ipfs.id()
nodeId = peerInfo.id
})
Expand Down
4 changes: 0 additions & 4 deletions packages/ipfs-cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,6 @@

> The implementation of the js-IPFS CLI

## Lead Maintainer <!-- omit in toc -->

[Alex Potsides](http://github.com/achingbrain)

## Table of Contents <!-- omit in toc -->

- [Install](#install)
Expand Down
Loading