Skip to content
This repository has been archived by the owner on Aug 11, 2021. It is now read-only.

Awesome IPLD endeavour #60

Merged
merged 15 commits into from
Oct 26, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 36 additions & 54 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,100 +1,82 @@
# IPFS IPLD
# IPLD Resolver

[![](https://img.shields.io/badge/made%20by-Protocol%20Labs-blue.svg?style=flat-square)](http://ipn.io)
[![](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](http://ipfs.io/)
[![](https://img.shields.io/badge/freenode-%23ipfs-blue.svg?style=flat-square)](http://webchat.freenode.net/?channels=%23ipfs)
[![Coverage Status](https://coveralls.io/repos/github/ipld/js-ipld-resolver/badge.svg?branch=master)](https://coveralls.io/github/ipld/js-ipld-resolver?branch=master)
[![Travis CI](https://travis-ci.org/ipld/js-ipld-resolver.svg?branch=master)](https://travis-ci.org/ipld/js-ipld-resolver)
[![Circle CI](https://circleci.com/gh/ipld/js-ipld-resolver.svg?style=svg)](https://circleci.com/gh/ipld/js-ipld-resolver)
[![Dependency Status](https://david-dm.org/ipld/js-ipld-resolver.svg?style=flat-square)](https://david-dm.org/ipld/js-ipld-resolver)
[![js-standard-style](https://img.shields.io/badge/code%20style-standard-brightgreen.svg?style=flat-square)](https://github.com/feross/standard)
[![standard-readme compliant](https://img.shields.io/badge/standard--readme-OK-green.svg?style=flat-square)](https://github.com/RichardLitt/standard-readme)
[![Coverage Status](https://coveralls.io/repos/github/ipfs/js-ipfs-ipld/badge.svg?branch=master)](https://coveralls.io/github/ipfs/js-ipfs-ipld?branch=master)
[![Travis CI](https://travis-ci.org/ipfs/js-ipfs-ipld.svg?branch=master)](https://travis-ci.org/ipfs/js-ipfs-ipld)
[![Circle CI](https://circleci.com/gh/ipfs/js-ipfs-ipld.svg?style=svg)](https://circleci.com/gh/ipfs/js-ipfs-ipld)
[![Dependency Status](https://david-dm.org/ipfs/js-ipfs-ipld.svg?style=flat-square)](https://david-dm.org/ipfs/js-ipfs-ipld) [![js-standard-style](https://img.shields.io/badge/code%20style-standard-brightgreen.svg?style=flat-square)](https://github.com/feross/standard)

> JavaScript implementation of the IPLDService
> JavaScript implementation of the IPLD Resolver

## Table of Contents

* [Install](#install)
* [Usage](#usage)
* [API](#api)
+ [`resolve`](#resolve)
+ [IPLDService](#ipldservice)
- [Install](#install)
- [Usage](#usage)
- [API](#api)
- [IPLD Resolver](#ipldresolver)
- [`.put(node, cb)`](#putnode-cb)
- [`.putStream([cb])`](#putstreamcb)
- [`.get(multihash, cb)`](#getmultihash-cb)
- [`.getStream(multihash)`](#getstreammultihash)
- [`.getRecursive(multihash, cb)`](#getrecursivemultihash-cb)
- [`.getRecursiveStream(multihash)`](#getrecursivestreammultihash)
- [`.remove(multihash, cb)`](#removemultihash-cb)
* [Contribute](#contribute)
* [License](#license)
- [`.get(cid, cb)`](#getcid-cb)
- [`.getStream(cid)`](#getstreamcid)
- [`.remove(cid, cb)`](#removecid-cb)
- [Contribute](#contribute)
- [License](#license)

## Install

```bash
npm install --save ipfs-ipld
> npm install --save ipfs-ipld
```

## Usage

```js
const ipfsIPLD = require('ipfs-ipld')
const IPLDResolver = require('ipld-resolver')

// available components
ipfsIPLD.IPLDService
ipfsIPLD.resolve
// pass an optional blockService, if no blockService is passed,
// one is created in memory.
const ipldResolver = new IPLDResolver(blockService)
```

## API

### `resolve`
### IPLD Resolver

> Resolve IPLD paths against a given IPLDService

```js
const node = {
hello: {
world: 11,
some: 12
}
}
const mh = ipld.multihash(ipld.marshal(node))
ipldService.put(node, (err) => {
resolve(ipldService, `${mh}/hello/world`, (err, res) => {
console.log(res)
// => 11
})
```

### IPLDService

#### `.put(node, cb)`
#### `.put(node, callback)`

> Store the given node (any JavaScript object).

#### `.putStream([cb])`
#### `.putStream([callback])`

Returns a sink pull-stream, to write IPLD objects to.

#### `.get(multihash, cb)`
#### `.get(cid, callback)`

> Retrieve a node by the given `multihash`.

#### `.getStream(multihash)`
#### `.getStream(cid)`

Returns a source pull-stream of the requested IPLD object.

#### `.getRecursive(multihash, cb)`
#### `.remove(cid, callback)`

> Retrieve a node by the given `multihash` and all linked nodes.
> Remove a node by the given `multihash`

#### `.getRecursiveStream(multihash)`
#### `.resolve(cid, path, callback)`

Returns a source pull-stream, which emits the requested node, and
all linked nodes.
> Resolves an IPLD path

#### `.remove(multihash, cb)`
#### `.support.add(multicodec, formatResolver, formatUtil)`

> Remove a node by the given `multihash`
> Add support to another IPLD Format

#### `.support.rm(multicodec)`

> Removes support of an IPLD Format

## Contribute

Expand Down
24 changes: 14 additions & 10 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "ipfs-ipld",
"version": "3.0.0",
"description": "IPLD implementation in JavaScript",
"name": "ipld-resolver",
"version": "0.0.0",
"description": "IPLD Resolver Implementation in JavaScript",
"main": "lib/index.js",
"jsnext:main": "src/index.js",
"pre-commit": [
Expand Down Expand Up @@ -34,23 +34,27 @@
"homepage": "https://github.com/ipfs/js-ipfs-ipld#readme",
"license": "MIT",
"devDependencies": {
"aegir": "^8.0.1",
"aegir": "^8.1.2",
"async": "^2.0.1",
"buffer-loader": "0.0.1",
"chai": "^3.5.0",
"fs-pull-blob-store": "^0.3.0",
"idb-pull-blob-store": "^0.4.0",
"ipfs-block-service": "^0.5.0",
"ipfs-repo": "^0.9.0",
"idb-pull-blob-store": "^0.5.1",
"ipfs-block-service": "^0.6.0",
"ipfs-repo": "^0.10.0",
"lodash": "^4.15.0",
"ncp": "^2.0.0",
"pre-commit": "^1.1.3",
"rimraf": "^2.5.4"
},
"dependencies": {
"async": "^2.1.1",
"babel-runtime": "^6.11.6",
"ipfs-block": "^0.3.0",
"ipld": "^0.6.0",
"cids": "^0.2.0",
"interface-pull-blob-store": "^0.5.0",
"ipfs-block": "^0.4.0",
"ipld-dag-cbor": "^0.7.0",
"ipld-dag-pb": "^0.1.0",
"is-ipfs": "^0.2.0",
"lodash.flatten": "^4.4.0",
"lodash.includes": "^4.3.0",
Expand All @@ -64,4 +68,4 @@
"greenkeeperio-bot <support@greenkeeper.io>",
"nicola <me@nicola.io>"
]
}
}
168 changes: 166 additions & 2 deletions src/index.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,168 @@
'use strict'

exports.IPLDService = require('./ipld-service')
exports.resolve = require('./resolve')
const Block = require('ipfs-block')
const pull = require('pull-stream')
const CID = require('cids')
const until = require('async/until')
const IPFSRepo = require('ipfs-repo')
const MemoryStore = require('../node_modules/interface-pull-blob-store/lib/reference.js')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can do require('interface-pull-blob-store/lib/reference.js') instead, no need for the full path

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right! (I do remember fighting with this one and this was how I got it to reference to the right one, not sure what was the error without node_modules though).

Could you also check ipfs-inactive/interface-pull-blob-store#6, this way we don't even have to point to a path inside.

const BlockService = require('ipfs-block-service')

const dagPB = require('ipld-dag-pb')
const dagCBOR = require('ipld-dag-cbor')

class IPLDResolver {
constructor (blockService) {
// nicola will love this!
if (!blockService) {
const repo = new IPFSRepo('in-memory', { stores: MemoryStore })
blockService = new BlockService(repo)
}

this.bs = blockService
this.resolvers = {}

this.support = {}

// Adds support for an IPLD format
this.support.add = (multicodec, resolver, util) => {
if (this.resolvers[multicodec]) {
throw new Error(multicodec + 'already supported')
}

this.resolvers[multicodec] = {
resolver: resolver,
util: util
}
}

this.support.rm = (multicodec) => {
if (this.resolvers[multicodec]) {
delete this.resolvers[multicodec]
}
}

// Support by default dag-pb and dag-cbor
this.support.add(dagPB.resolver.multicodec, dagPB.resolver, dagPB.util)
this.support.add(dagCBOR.resolver.multicodec, dagCBOR.resolver, dagCBOR.util)
}

resolve (cid, path, callback) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are cid and path two different inputs?
Can we not just pass cid+path?
the "deconcatenation" is something that I will happen to do often

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nicola note that CID is a instance of class CID that has several utility methods https://github.com/ipfs/js-cid/tree/feat/complete

What you are proposing is to use cids always as strings, which would assume that we have to be constantly parsing and mounting them again to get their props.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I see, perfect then.

There must be up in the levels of abstraction a sort of resolver that takes a path and resolve that (by using this, I guess)

if (path === '/') {
return this.get(cid, callback)
}

let value

until(
() => {
if (!path || path === '' || path === '/') {
return true
} else {
// continue traversing
if (value) {
cid = new CID(value['/'])
}
return false
}
},
(cb) => {
// get block
// use local resolver
// update path value
this.bs.get(cid, (err, block) => {
if (err) {
return cb(err)
}
const r = this.resolvers[cid.codec]
r.resolver.resolve(block, path, (err, result) => {
if (err) {
return cb(err)
}
value = result.value
path = result.remainderPath
cb()
})
})
},
(err, results) => {
if (err) {
return callback(err)
}
return callback(null, value)
}
)
}

// Node operations (get and retrieve nodes, not values)

put (nodeAndCID, callback) {
callback = callback || noop
pull(
pull.values([nodeAndCID]),
this.putStream(callback)
)
}

putStream (callback) {
callback = callback || noop

return pull(
pull.asyncMap((nodeAndCID, cb) => {
const cid = nodeAndCID.cid
const r = this.resolvers[cid.codec]

r.util.serialize(nodeAndCID.node, (err, serialized) => {
if (err) {
return cb(err)
}
cb(null, {
block: new Block(serialized),
cid: cid
})
})
}),
this.bs.putStream(),
pull.onEnd(callback)
)
}

get (cid, callback) {
pull(
this.getStream(cid),
pull.collect((err, res) => {
if (err) {
return callback(err)
}
callback(null, res[0])
})
)
}

getStream (cid) {
return pull(
this.bs.getStream(cid),
pull.asyncMap((block, cb) => {
const r = this.resolvers[cid.codec]
if (r) {
r.util.deserialize(block.data, (err, deserialized) => {
if (err) {
return cb(err)
}
cb(null, deserialized)
})
} else { // multicodec unknown, send back raw data
cb(null, block.data)
}
})
)
}

remove (cids, callback) {
this.bs.delete(cids, callback)
}
}

function noop () {}

module.exports = IPLDResolver
Loading