Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WIP: IPFS Internals API #78

Merged
merged 1 commit into from
Feb 27, 2016
Merged

WIP: IPFS Internals API #78

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions api/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ This RFC is organized by chapters described on the *Table of contents* section.

- [%N% Introduction]()
- [%N% APIs]()
- [%N.M% Internals]()
- [%N.M% Core]()
- [%N.M% HTTP]()
- [%N.M% CLI]()
Expand Down
91 changes: 91 additions & 0 deletions api/internals/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# Internals

In addition to the APIs exposed to developers (cli, http and core), the IPFS spec provides a set of specifications for the IPFS subsystems (AKA modules), providing a clean understanding of the components that have to be constructed for a full IPFS implementation or even a clear knowledge of how to make a partial implementation, without compromissing due to the interdependencies of these subsystems.

This spec is a `WIP` and should be treated like so.

## Architecture

In a birds eye view, the Internals of IPFS might look like this:

```
▶ ┌───────────────────────────────────────────────────────────────────────────────┐
│ IPFS Core │
│ └───────────────────────────────────────────────────────────────────────────────┘
│
│ │
│
│ ┌──────────────┬──────────────┼────────────┬─────────────────┐
│ │ │ │ │
│ │ │ │ │ │
▼ │ ▼ │ ▼
│ ┌──────────────────┐ │ ┌──────────────────┐ │ ┌──────────────────┐
│ │ │ │ │ │ │ │
│ │ Block Service │ │ │ DAG Service │ │ │ IPFS Repo │
│ │ │ │ │ │ │ │
│ └──────────────────┘ │ └──────────────────┘ │ └──────────────────┘
│ │ │ │
IPFS Core │ ▼ │ ┌────┴────┐ │
┌────────┐ │ ▼ ▼ │
│ │ Block │ │ ┌────────┐┌────────┐ │
└────────┘ │ │DAG Node││DAG Link│ │
│ │ └────────┘└────────┘ │
┌──────────────────┐ │ │ ┌──────────────────┐
│ │ │ │ │ │ │
│ Bitswap │◀────┤ ├──────▶│ Importer │
│ │ │ │ │ │ │
└──────────────────┘ │ │ └──────────────────┘
│ │ │ │
│ │ ┌────┴────┐
│ │ │ ▼ ▼
│ │ ┌────────┐┌────────┐
│ ┌──────────────────┐ │ │ │ layout ││chunker │
│ │ │ ┌────────────┘ └────────┘└────────┘
│ │ Files │◀────┘ │
│ │ │
│ └──────────────────┘ │
▶ │
▼
┌───────────────────────────────────────────────────────────────────────────────┐
│ │
│ │
│ │
│ libp2p │
│ │
│ │
└───────────────────────────────────────────────────────────────────────────────┘
```

TODO:

- [ ] Add IPLD

## Index

- ipfs-blocks - Block Service and Block
- [spec]()
- [js](https://github.com/ipfs/js-ipfs-blocks)
- [go](https://github.com/ipfs/go-ipfs/tree/master/blocks)
- ipfs-merkle-dag - DAG Service and DAG Node (protobuf data format)
- [spec]()
- [js](https://github.com/vijayee/js-ipfs-merkle-dag)
- [go](https://github.com/ipfs/go-ipfs/tree/master/merkledag)
- ipfs-bitswap
- Defined by the [Exchange Spec](https://github.com/ipfs/specs/pull/53)
- [go](https://github.com/ipfs/go-ipfs/tree/master/exchange/bitswap)
- ipfs-data-importing
- Defined by the [Data Importing Spec](https://github.com/ipfs/specs/pull/57)
- [js](https://github.com/ipfs/js-ipfs-data-importing)
- [go](https://github.com/ipfs/go-ipfs/tree/master/unixfs/io)
- ipfs-repo
- Defined by the [Repo](https://github.com/ipfs/specs/pull/43)
- [js](https://github.com/ipfs/js-ipfs-repo)
- [go](https://github.com/ipfs/go-ipfs/tree/master/repo)
- ipfs-files
- [spec]()
- js - Currently, we support `jsipfs files add`, on the js-ipfs impl, but not the full files API impl
- [go](https://github.com/ipfs/go-ipfs/tree/master/mfs)
- ipfs-unixfs
- [spec]()
- [js](https://github.com/ipfs/js-ipfs-unixfs)
- [go](https://github.com/ipfs/go-ipfs/tree/master/unixfs)
Empty file added api/internals/blocks.md
View file
Open in desktop
Empty file.
97 changes: 97 additions & 0 deletions api/internals/merkle-dag.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
Internal: ipfs-merkle-dag
=========================

> This is the specification for the merkle-dag module used by IPFS.

The merkle-dag module offers an interface to create MerkleDAG nodes and links and in addition implements the DAG Service, offering an interface to store and fetch Nodes that works with ipfs-blocks, and therefore local storage and network storage (through exchange).

## API

### DAG Service

##### add

> stores the node

##### get

> fetches a node by its multihash

##### getRecursive

> fetches a node and all of its links (if possible)

##### remove

> deletes a node

### DAG Node

##### createNode

##### addNodeLink

> creates a link on node A to node B by using node B to get its multihash

##### addRawLink

> creates a link on node A to node B by using direclty node B multihash

##### updateNodeLink

> updates a link on the node. *caution* this method returns a copy of the MerkleDAG node

##### removeNodeLink

> removes a link from the node by name

##### removeNodeLinkByHash

> removes a link from the node by the hash of the linked node

##### copy

> creates a copy of the MerkleDAG Node

##### size

> size of the node. This is a property, not a function

##### multihash

> returns the multihash (default: sha2-256)

##### marshal

> returns a protobuf serialized version, compatible with go-ipfs MerkleDAG

##### unMarshal

> desirializes a probuf serialized node

##### (used internally) getPBNode

> used internally

##### (used internally) makeLink

> used internally

### DAG Link



## Data Structures

### DAG Node

- protobuf schema - https://github.com/ipfs/go-ipfs/blob/master/merkledag/pb/merkledag.proto#L31-L39

### DAG Link

- protobuf schema - https://github.com/ipfs/go-ipfs/blob/master/merkledag/pb/merkledag.proto#L18-L29

## Implementations

- [js](https://github.com/vijayee/js-ipfs-merkle-dag)
- [go](https://github.com/ipfs/go-ipfs/tree/master/merkledag)