Skip to content
This repository has been archived by the owner on Oct 30, 2023. It is now read-only.

Commit

Permalink
fix!: use @helia/delegated-routing-v1-http-api-client internally (#27)
Browse files Browse the repository at this point in the history 
Refactors the internal implementation to use the [@helia/delegated-routing-v1-http-api-client](https://www.npmjs.com/package/@helia/delegated-routing-v1-http-api-client) instead of a hand-rolled version.

Replaces any older references to "Reframe" with the new name of "Delegated Routing V1 HTTP API".  Phew.

BREAKING CHANGE: the module has been renamed as the spec has been renamed
  • Loading branch information
@achingbrain
achingbrain authored Oct 26, 2023
1 parent c673145 commit 5bd026e
Show file tree
Hide file tree
Showing 4 changed files with 92 additions and 141 deletions.
37 changes: 21 additions & 16 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,44 +1,49 @@
# @libp2p/http-v1-content-routing <!-- omit in toc -->
# @libp2p/delegated-routing-v1-http-api-content-routing <!-- omit in toc -->

[![libp2p.io](https://img.shields.io/badge/project-libp2p-yellow.svg?style=flat-square)](http://libp2p.io/)
[![Discuss](https://img.shields.io/discourse/https/discuss.libp2p.io/posts.svg?style=flat-square)](https://discuss.libp2p.io)
[![codecov](https://img.shields.io/codecov/c/github/libp2p/js-http-v1-content-routing.svg?style=flat-square)](https://codecov.io/gh/libp2p/js-http-v1-content-routing)
[![CI](https://img.shields.io/github/actions/workflow/status/libp2p/js-http-v1-content-routing/js-test-and-release.yml?branch=main\&style=flat-square)](https://github.com/libp2p/js-http-v1-content-routing/actions/workflows/js-test-and-release.yml?query=branch%3Amain)
[![codecov](https://img.shields.io/codecov/c/github/libp2p/js-delegated-routing-v1-http-api-content-routing.svg?style=flat-square)](https://codecov.io/gh/libp2p/js-delegated-routing-v1-http-api-content-routing)
[![CI](https://img.shields.io/github/actions/workflow/status/libp2p/js-delegated-routing-v1-http-api-content-routing/js-test-and-release.yml?branch=main\&style=flat-square)](https://github.com/libp2p/js-delegated-routing-v1-http-api-content-routing/actions/workflows/js-test-and-release.yml?query=branch%3Amain)

> Use a Routing V1 HTTP service to discover content providers
> Use a Delegated Routing V1 HTTP service to discover content providers
This is a [ContentRouting](https://libp2p.github.io/js-libp2p/interfaces/_libp2p_interface.content_routing.ContentRouting.html)
implementation that makes use of the [@helia/delegated-routing-v1-http-api-client](https://www.npmjs.com/package/@helia/delegated-routing-v1-http-api-client)
to use servers that implement the snappily-titled [Delegated Routing V1 HTTP API](Delegated Routing V1 HTTP API)
spec to get/put IPNS records and to resolve providers for CIDs.

## Table of contents <!-- omit in toc -->

- [Install](#install)
- [Browser `<script>` tag](#browser-script-tag)
- - [Install](#install)
- [Browser `<script>` tag](#browser-script-tag)
- [Example](#example)
- [API Docs](#api-docs)
- [License](#license)
- [Contribution](#contribution)
- [API Docs](#api-docs)
- [License](#license)
- [Contribution](#contribution)

## Install

```console
$ npm i @libp2p/http-v1-content-routing
$ npm i @libp2p/delegated-routing-v1-http-api-content-routing
```

### Browser `<script>` tag

Loading this module through a script tag will make it's exports available as `Libp2pHttpV1ContentRouting` in the global namespace.
Loading this module through a script tag will make it's exports available as `Libp2pDelegatedRoutingV1HttpApiContentRouting` in the global namespace.

```html
<script src="https://unpkg.com/@libp2p/http-v1-content-routing/dist/index.min.js"></script>
<script src="https://unpkg.com/@libp2p/delegated-routing-v1-http-api-content-routing/dist/index.min.js"></script>
```

## Example
# Example

```js
import { createLibp2p } from 'libp2p'
import { reframeContentRouting } from '@libp2p/reframe-content-routing'
import { delgatedRoutingV1HTTPAPIContentRouting } from '@libp2p/delegated-routing-http-v1-content-routing'

const node = await createLibp2p({
contentRouters: [
reframeContentRouting('https://cid.contact/reframe')
delgatedRoutingV1HTTPAPIContentRouting('https://example.org')
]
//.. other config
})
Expand All @@ -51,7 +56,7 @@ for await (const provider of node.contentRouting.findProviders('cid')) {

## API Docs

- <https://libp2p.github.io/js-http-v1-content-routing>
- <https://libp2p.github.io/js-delegated-routing-v1-http-api-content-routing>

## License

Expand Down
25 changes: 11 additions & 14 deletions package.json
View file
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
{
"name": "@libp2p/http-v1-content-routing",
"version": "1.0.2",
"description": "Use a Routing V1 HTTP service to discover content providers",
"name": "@libp2p/delegated-routing-v1-http-api-content-routing",
"version": "0.0.0",
"description": "Use a Delegated Routing V1 HTTP service to discover content providers",
"license": "Apache-2.0 OR MIT",
"homepage": "https://github.com/libp2p/js-http-v1-content-routing#readme",
"homepage": "https://github.com/libp2p/js-delegated-routing-v1-http-api-content-routing#readme",
"repository": {
"type": "git",
"url": "git+https://github.com/libp2p/js-http-v1-content-routing.git"
"url": "git+https://github.com/libp2p/js-delegated-routing-v1-http-api-content-routing.git"
},
"bugs": {
"url": "https://github.com/libp2p/js-http-v1-content-routing/issues"
"url": "https://github.com/libp2p/js-delegated-routing-v1-http-api-content-routing/issues"
},
"type": "module",
"types": "./dist/src/index.d.ts",
Expand Down Expand Up @@ -130,20 +130,17 @@
"test:node": "aegir test -t node --cov",
"test:electron-main": "aegir test -t electron-main",
"release": "aegir release",
"docs": "aegir docs"
"docs": "aegir docs -- --includeVersion false"
},
"dependencies": {
"@helia/delegated-routing-v1-http-api-client": "^1.0.1",
"@libp2p/interface": "^0.1.1",
"@libp2p/logger": "^3.0.1",
"@libp2p/peer-id": "^3.0.1",
"@multiformats/multiaddr": "^12.1.2",
"any-signal": "^4.1.1",
"browser-readablestream-to-it": "^2.0.2",
"it-to-buffer": "^4.0.1",
"ipns": "^7.0.1",
"it-map": "^3.0.4",
"multiformats": "^12.0.1",
"p-defer": "^4.0.0",
"p-queue": "^7.3.4",
"uint8arrays": "^4.0.3"
"uint8arrays": "^4.0.6"
},
"devDependencies": {
"@libp2p/peer-id-factory": "^3.0.2",
Expand Down
147 changes: 47 additions & 100 deletions src/index.ts
View file
Original file line number Diff line number Diff line change
@@ -1,34 +1,31 @@
import { type DelegatedRoutingV1HttpApiClient, createDelegatedRoutingV1HttpApiClient } from '@helia/delegated-routing-v1-http-api-client'
import { CodeError } from '@libp2p/interface/errors'
import { logger } from '@libp2p/logger'
import { peerIdFromString } from '@libp2p/peer-id'
import { multiaddr } from '@multiformats/multiaddr'
import { anySignal } from 'any-signal'
import toIt from 'browser-readablestream-to-it'
import toBuffer from 'it-to-buffer'
import defer from 'p-defer'
import PQueue from 'p-queue'
import { toString as uint8ArrayToString } from 'uint8arrays/to-string'
import { peerIdFromBytes } from '@libp2p/peer-id'
import { marshal, unmarshal } from 'ipns'
import map from 'it-map'
import { equals as uint8ArrayEquals } from 'uint8arrays/equals'
import { fromString as uint8ArrayFromString } from 'uint8arrays/from-string'
import type { AbortOptions } from '@libp2p/interface'
import type { ContentRouting } from '@libp2p/interface/content-routing'
import type { PeerId } from '@libp2p/interface/peer-id'
import type { PeerInfo } from '@libp2p/interface/peer-info'
import type { Startable } from '@libp2p/interface/startable'
import type { Multiaddr } from '@multiformats/multiaddr'
import type { CID } from 'multiformats/cid'

const log = logger('reframe-content-routing')
const log = logger('delegated-routing-v1-http-api-content-routing')

export interface ReframeV1Response {
Providers: ReframeV1ResponseItem[]
const IPNS_PREFIX = uint8ArrayFromString('/ipns/')

function isIPNSKey (key: Uint8Array): boolean {
return uint8ArrayEquals(key.subarray(0, IPNS_PREFIX.byteLength), IPNS_PREFIX)
}

export interface ReframeV1ResponseItem {
ID: string
Addrs: string[]
Protocol: string
Schema: string
const peerIdFromRoutingKey = (key: Uint8Array): PeerId => {
return peerIdFromBytes(key.slice(IPNS_PREFIX.length))
}

export interface ReframeContentRoutingInit {
export interface DelegatedRoutingV1HTTPAPIContentRoutingInit {
/**
* A concurrency limit to avoid request flood in web browser (default: 4)
*
Expand All @@ -42,33 +39,21 @@ export interface ReframeContentRoutingInit {
timeout?: number
}

const defaultValues = {
concurrentRequests: 4,
timeout: 30e3
}

/**
* An implementation of content routing, using a delegated peer
*/
class ReframeContentRouting implements ContentRouting, Startable {
class DelegatedRoutingV1HTTPAPIContentRouting implements ContentRouting, Startable {
private started: boolean
private readonly httpQueue: PQueue
private readonly shutDownController: AbortController
private readonly clientUrl: URL
private readonly timeout: number
private readonly client: DelegatedRoutingV1HttpApiClient

/**
* Create a new DelegatedContentRouting instance
*/
constructor (url: string | URL, init: ReframeContentRoutingInit = {}) {
constructor (url: string | URL, init: DelegatedRoutingV1HTTPAPIContentRoutingInit = {}) {
this.started = false
this.shutDownController = new AbortController()
this.httpQueue = new PQueue({
concurrency: init.concurrentRequests ?? defaultValues.concurrentRequests
})
this.clientUrl = url instanceof URL ? url : new URL(url)
this.timeout = init.timeout ?? defaultValues.timeout
log('enabled Reframe routing via', url)
this.client = createDelegatedRoutingV1HttpApiClient(new URL(url), init)

log('enabled Delegated Routing V1 HTTP API Content Routing via', url)
}

isStarted (): boolean {
Expand All @@ -80,85 +65,47 @@ class ReframeContentRouting implements ContentRouting, Startable {
}

stop (): void {
this.httpQueue.clear()
this.shutDownController.abort()
this.client.stop()
this.started = false
}

async * findProviders (key: CID, options: AbortOptions = {}): AsyncIterable<PeerInfo> {
log('findProviders starts: %c', key)

const signal = anySignal([this.shutDownController.signal, options.signal, AbortSignal.timeout(this.timeout)])
const onStart = defer()
const onFinish = defer()

void this.httpQueue.add(async () => {
onStart.resolve()
return onFinish.promise
})

try {
await onStart.promise

// https://github.com/ipfs/specs/blob/main/routing/ROUTING_V1_HTTP.md#api
const resource = `${this.clientUrl}routing/v1/providers/${key.toString()}`
const getOptions = { headers: { Accept: 'application/x-ndjson' }, signal }
const a = await fetch(resource, getOptions)

if (a.body == null) {
throw new CodeError('Reframe response had no body', 'ERR_BAD_RESPONSE')
async * findProviders (cid: CID, options: AbortOptions = {}): AsyncIterable<PeerInfo> {
yield * map(this.client.getProviders(cid, options), (record) => {
return {
id: record.ID,
multiaddrs: record.Addrs ?? [],
protocols: []
}

const body = await toBuffer(toIt(a.body))
const result: ReframeV1Response = JSON.parse(uint8ArrayToString(body))

for await (const event of result.Providers) {
if (event.Protocol !== 'transport-bitswap' || event.Schema !== 'bitswap') {
continue
}

yield this.mapEvent(event)
}
} catch (err) {
log.error('findProviders errored:', err)
} finally {
signal.clear()
onFinish.resolve()
log('findProviders finished: %c', key)
}
})
}

private mapEvent (event: ReframeV1ResponseItem): PeerInfo {
const peer = peerIdFromString(event.ID)
const ma: Multiaddr[] = []
async provide (): Promise<void> {
// noop
}

for (const strAddr of event.Addrs) {
const addr = multiaddr(strAddr)
ma.push(addr)
async put (key: Uint8Array, value: Uint8Array, options?: AbortOptions): Promise<void> {
if (!isIPNSKey(key)) {
return
}

const pi = {
id: peer,
multiaddrs: ma,
protocols: []
}
const peerId = peerIdFromRoutingKey(key)
const record = unmarshal(value)

return pi
await this.client.putIPNS(peerId, record, options)
}

async provide (): Promise<void> {
// noop
}
async get (key: Uint8Array, options?: AbortOptions): Promise<Uint8Array> {
if (!isIPNSKey(key)) {
throw new CodeError('Not found', 'ERR_NOT_FOUND')
}

async put (): Promise<void> {
// noop
}
const peerId = peerIdFromRoutingKey(key)
const record = await this.client.getIPNS(peerId, options)

async get (): Promise<Uint8Array> {
throw new CodeError('Not found', 'ERR_NOT_FOUND')
return marshal(record)
}
}

export function reframeContentRouting (url: string | URL, init: ReframeContentRoutingInit = {}): () => ContentRouting {
return () => new ReframeContentRouting(url, init)
export function delgatedRoutingV1HTTPAPIContentRouting (url: string | URL, init: DelegatedRoutingV1HTTPAPIContentRoutingInit = {}): () => ContentRouting {
return () => new DelegatedRoutingV1HTTPAPIContentRouting(url, init)
}
24 changes: 13 additions & 11 deletions test/index.spec.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import { createEd25519PeerId } from '@libp2p/peer-id-factory'
import { expect } from 'aegir/chai'
import all from 'it-all'
import { CID } from 'multiformats/cid'
import { type ReframeV1ResponseItem, reframeContentRouting } from '../src/index.js'
import { delgatedRoutingV1HTTPAPIContentRouting } from '../src/index.js'

if (process.env.ECHO_SERVER == null) {
throw new Error('Echo server not configured correctly')
Expand All @@ -15,27 +15,29 @@ const cid = CID.parse('QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn')

describe('ReframeContentRouting', function () {
it('should find providers', async () => {
const providers: ReframeV1ResponseItem[] = [{
Protocol: 'transport-bitswap',
Schema: 'bitswap',
const providers = [{
Protocols: ['transport-bitswap'],
Schema: 'peer',
ID: (await createEd25519PeerId()).toString(),
Addrs: ['/ip4/41.41.41.41/tcp/1234']
}, {
Protocol: 'transport-bitswap',
Schema: 'bitswap',
ID: (await createEd25519PeerId()).toString(),
Addrs: ['/ip4/42.42.42.42/tcp/1234']
}, {
Schema: 'unknown',
ID: (await createEd25519PeerId()).toString(),
Addrs: ['/ip4/42.42.42.42/tcp/1234']
}]

// load providers for the router to fetch
await fetch(`${process.env.ECHO_SERVER}/add-providers/${cid.toString()}`, {
method: 'POST',
body: JSON.stringify({
Providers: providers
})
body: providers.map(prov => JSON.stringify(prov)).join('\n')
})

const routing = reframeContentRouting(serverUrl)()
const routing = delgatedRoutingV1HTTPAPIContentRouting(serverUrl)()

const provs = await all(routing.findProviders(cid))
expect(provs.map(prov => ({
Expand All @@ -54,7 +56,7 @@ describe('ReframeContentRouting', function () {
body: 'not json'
})

const routing = reframeContentRouting(serverUrl)()
const routing = delgatedRoutingV1HTTPAPIContentRouting(serverUrl)()

const provs = await all(routing.findProviders(cid))
expect(provs).to.be.empty()
Expand Down Expand Up @@ -84,14 +86,14 @@ describe('ReframeContentRouting', function () {
})
})

const routing = reframeContentRouting(serverUrl)()
const routing = delgatedRoutingV1HTTPAPIContentRouting(serverUrl)()

const provs = await all(routing.findProviders(cid))
expect(provs).to.be.empty()
})

it('should handle empty input', async () => {
const routing = reframeContentRouting(serverUrl)()
const routing = delgatedRoutingV1HTTPAPIContentRouting(serverUrl)()

const provs = await all(routing.findProviders(cid))
expect(provs).to.be.empty()
Expand Down

0 comments on commit 5bd026e

Please sign in to comment.