Skip to content

Commit

Permalink
fix: fix some mistakes in the spec
Browse files Browse the repository at this point in the history
  • Loading branch information
@daviddias
daviddias committed Sep 25, 2016
1 parent 8c6d915 commit 9ae2670
Showing 1 changed file with 63 additions and 56 deletions.
119 changes: 63 additions & 56 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,16 @@
[![](https://img.shields.io/badge/project-multiformats-blue.svg?style=flat-square)](http://github.com/multiformats/multiformats)
[![](https://img.shields.io/badge/freenode-%23ipfs-blue.svg?style=flat-square)](http://webchat.freenode.net/?channels=%23ipfs)

> self-describing codecs
> Make data and streams self-described by prefixing them with human readable or binary packed codecs. `multicodec` offers a base table, but can also be extended with extra tables by application basis.
## Table of Contents

- [Motivation](#motivation)
- [How does it work? - Protocol Description](#how-does-it-work---protocol-description)
- [Prefix examples](#prefix-examples)
- [prefix - codec - desc](#prefix---codec---desc)
- [The protocol path](#the-protocol-path)
- [How does it work? - Protocol Description](#how-does-it-work---protocol-description)
- [The protocol path](#the-protocol-path)
- [Multicodec table](#multicodec-table)
- [Implementations](#implementations)
- [Implementation details]()
- [FAQ](#faq)
- [Maintainers](#maintainers)
- [Contribute](#contribute)
Expand All @@ -38,52 +38,74 @@ Moreover, this self-description allows straightforward layering of protocols wit
`multicodec` is a _self-describing multiformat_, it wraps other formats with a tiny bit of self-description:

```sh
<multicodec-header><encoded-data>
# or
<varint-len><code>\n<encoded-data>
<varint-len><multicodec><encoded-data>
```

For example, let's encode a json doc:

```node
> // encode some json
> var str = JSON.stringify({"hello":"world"})
> var buf = multicodec.encode('json', str) // prepends multistream.header('/json')
> buf
<Buffer 06 2f 6a 73 6f 6e 2f 7b 22 68 65 6c 6c 6f 22 3a 22 77 6f 72 6c 64 22 7d>
> buf.toString('hex')
062f6a736f6e2f7b2268656c6c6f223a22776f726c64227d
> // decode, and find out what is in buf
> multicodec.decode(buf)
{ "codec": "json", "data": '{"hello": "world"}' }
```JavaScript
// encode some json
const buf = new Buffer(JSON.stringify({ hello: 'world' }))

const prefixedBuf = multicodec.addPrefix('json', str) // prepends multicodec ('json')
console.log(prefixedBuf)
// <Buffer 06 2f 6a 73 6f 6e 2f 7b 22 68 65 6c 6c 6f 22 3a 22 77 6f 72 6c 64 22 7d>

const.log(prefixedBuf.toString('hex'))
// 062f6a736f6e2f7b2268656c6c6f223a22776f726c64227d

// let's get the Multicodec, and get the data back

const codec = multicodec.getMulticodec(prefixedBuf)
console.log(codec)
// json

console.log(multicodec.rmPrefix(prefixedBuf).toString())
// "{ \"hello\": \"world\" }
```

So, `buf` is:

```
hex: 062f6a736f6e2f7b2268656c6c6f223a22776f726c64227d
ascii: json/\n{"hello":"world"}
ascii: /json/"{\"hello\":\"world\"}
```

The more you know! Let's try it again, this time with protobuf:
Note that on the ascii version, the varint at the beginning is not being represented, you should account that.

For a binary packed version of the multicodecs, see [multicodec-packed](./multicodec-packed.md).

## The protocol path

`multicodec` allows us to specify different protocols in a universal namespace, that way being able to recognize, multiplex, and embed them easily. We use the notion of a `path` instead of an `id` because it is meant to be a Unix-friendly URI.

A good path name should be decipherable -- meaning that if some machine or developer -- who has no idea about your protocol -- encounters the path string, they should be able to look it up and resolve how to use it.

An example of a good path name is:

```
cat proto.c
/bittorrent.org/1.0
```

See also: [multicodec-packed](./multicodec-packed.md).
An example of a _great_ path name is:

```
/ipfs/Qmaa4Rw81a3a1VEx4LxB7HADUAXvZFhCoRdBzsMZyZmqHD/ipfs.protocol
/http/w3id.org/ipfs/1.1.0
```

## Prefix examples
These path names happen to be resolvable -- not just in a "multicodec muxer(e.g [multistream](https://github.com/multiformats/multistream))" but -- in the internet as a whole (provided the program (or OS) knows how to use the `/ipfs` and `/http` protocols).

## Multicodec table

| prefix | codec | desc | type | [packed encoding](https://github.com/multiformats/multicodec/blob/master/multicodec-packed.md)|
|----------------|-------|-------------|-------|---------------------------------------|
|0x052f62696e2f | /bin/ |raw binary |binary | 0x00 |
|0x042f62322f | /b2/ |ascii base2 |binary | |
|0x052f6231362f | /b16/ |ascii base16 |hex | |
|0x052f6233322f | /b32/ |ascii base32 | | |
|0x052f6235382f | /b58/ |ascii base58 | | |
|0x052f6236342f | /b64/ |ascii base64 | | |
| prefix | codec | desc | type | [packed encoding](https://github.com/multiformats/multicodec/blob/master/multicodec-packed.md)|
|----------------|-------|--------------|--------|---------------------------------------|
|0x052f62696e2f | /bin/ | raw binary | binary | 0x00 |
|0x042f62322f | /b2/ | ascii base2 | binary | |
|0x052f6231362f | /b16/ | ascii base16 | hex | |
|0x052f6233322f | /b32/ | ascii base32 | | |
|0x052f6235382f | /b58/ | ascii base58 | | |
|0x052f6236342f | /b64/ | ascii base64 | | |
|0x062f6a736f6e2f |/json/ | |json | |
|0x062f63626f722f |/cbor/ | |json | |
|0x062f62736f6e2f |/bson/ | |json | |
Expand All @@ -99,33 +121,18 @@ See also: [multicodec-packed](./multicodec-packed.md).
|0x052f7a69702f |/zip/ | | archive | |
|0x052f706e672f | /png/ | | archive | |
|0x052f726c702f | /rlp/ | recursive length prefix | ethereum | 0x60 |
## The protocol path

`multicodec` allows us to specify different protocols in a universal namespace, that way being able to recognize, multiplex, and embed them easily. We use the notion of a `path` instead of an `id` because it is meant to be a Unix-friendly URI.

A good path name should be decipherable -- meaning that if some machine or developer -- who has no idea about your protocol -- encounters the path string, they should be able to look it up and resolve how to use it.

An example of a good path name is:

```
/bittorrent.org/1.0
```

An example of a _great_ path name is:

```
/ipfs/Qmaa4Rw81a3a1VEx4LxB7HADUAXvZFhCoRdBzsMZyZmqHD/ipfs.protocol
/http/w3id.org/ipfs/ipfs-1.1.0.json
```

These path names happen to be resolvable -- not just in a "multicodec muxer(e.g [multistream]())" but -- in the internet as a whole (provided the program (or OS) knows how to use the `/ipfs` and `/http` protocols).

## Implementations

- [go-multicodec](https://github.com/jbenet/go-multicodec)
- [go-multistream](https://github.com/whyrusleeping/go-multistream) - Implements multistream, which uses multicodec for stream negotiation
- [js-multistream](https://github.com/multiformats/js-multistream) - Implements multistream, which uses multicodec for stream negotiation
- [clj-multicodec](https://github.com/greglook/clj-multicodec)
- multicodec
- [go-multicodec](https://github.com/multiformats/go-multicodec)
- multicodec-packed
- [go-multicodec-packed](https://github.com/multiformats/go-multicodec-packed)
- [js-multicodec-packed](https://github.com/multiformats/js-multicodec-packed)
- multistream
- [go-multistream](https://github.com/multiformats/go-multistream) - Implements multistream, which uses multicodec for stream negotiation
- [js-multistream](https://github.com/multiformats/js-multistream) - Implements multistream, which uses multicodec for stream negotiation
- [clj-multicodec](https://github.com/greglook/clj-multicodec)


## FAQ
Expand Down

0 comments on commit 9ae2670

Please sign in to comment.