Skip to content

Commit

Permalink
initial commit
Browse files Browse the repository at this point in the history
  • Loading branch information
francoispqt committed Apr 25, 2018
0 parents commit d6ba85a
Show file tree
Hide file tree
Showing 58 changed files with 5,709 additions and 0 deletions.
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
*.out
*.log
*.test
.vscode
11 changes: 11 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
language: go

go:
- "1.10.x"
- master

script:
- go test -race -coverprofile=coverage.txt -covermode=atomic

after_success:
- bash <(curl -s https://codecov.io/bash)
9 changes: 9 additions & 0 deletions Gopkg.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

23 changes: 23 additions & 0 deletions Gopkg.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Gopkg.toml example
#
# Refer to https://github.com/golang/dep/blob/master/docs/Gopkg.toml.md
# for detailed Gopkg.toml documentation.
#
# required = ["github.com/user/thing/cmd/thing"]
# ignored = ["github.com/user/project/pkgX", "bitbucket.org/user/project/pkgA/pkgY"]
#
# [[constraint]]
# name = "github.com/user/project"
# version = "1.0.0"
#
# [[constraint]]
# name = "github.com/user/project2"
# branch = "dev"
# source = "github.com/myfork/project2"
#
# [[override]]
# name = "github.com/x/y"
# version = "2.4.0"


ignored = ["github.com/francoispqt/benchmarks*","github.com/stretchr/testify*","github.com/stretchr/testify","github.com/json-iterator/go","github.com/buger/jsonparser"]
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2016 gojay

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
11 changes: 11 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
.PHONY: test
test:
go test -run=^Test -v

.PHONY: cover
cover:
go test -coverprofile=coverage.out

.PHONY: coverhtml
coverhtml:
go tool cover -html=coverage.out
298 changes: 298 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,298 @@
[![Go doc](http://img.shields.io/badge/go-documentation-blue.svg?style=flat-square
)](https://godoc.org/github.com/francoispqt/gojay)
![MIT License](https://img.shields.io/badge/license-mit-blue.svg?style=flat-square)

# GoJay
GoJay is a performant JSON encoder/decoder for Golang (currently the most performant).

It has a simple API and doesn't use reflection. It relies on small interfaces to decode/encode structures and slices.

Gojay also comes with powerful stream decoding features.

# Get started

```bash
go get github.com/francoispqt/gojay
```

## Decoding

Example of basic stucture decoding:
```go
import "github.com/francoispqt/gojay"

type user struct {
id int
name string
email string
}
// implement UnmarshalerObject
func (u *user) UnmarshalObject(dec *gojay.Decoder, key string) {
switch k {
case "id":
return dec.AddInt(&u.id)
case "name":
return dec.AddString(&u.name)
case "email":
return dec.AddString(&u.email)
}
}
func (u *user) NKeys() int {
return 3
}

func main() {
u := &user{}
d := []byte(`{"id":1,"name":"gojay","email":"gojay@email.com"}`)
err := gojay.UnmarshalObject(d, user)
if err != nil {
log.Fatal(err)
}
}
```

### Structs
#### UnmarshalerObject Interface

To unmarshal a JSON object to a structure, the structure must implement the UnmarshalerObject interface:
```go
type UnmarshalerObject interface {
UnmarshalObject(*Decoder, string) error
NKeys() int
}
```
UnmarshalObject method takes two arguments, the first one is a pointer to the Decoder (*gojay.Decoder) and the second one is the string value of the current key being parsed. If the JSON data is not an object, the UnmarshalObject method will never be called.

NKeys method must return the number of keys to Unmarshal in the JSON object.

Example of implementation:
```go
type user struct {
id int
name string
email string
}
// implement UnmarshalerObject
func (u *user) UnmarshalObject(dec *gojay.Decoder, key string) {
switch k {
case "id":
return dec.AddInt(&u.id)
case "name":
return dec.AddString(&u.name)
case "email":
return dec.AddString(&u.email)
}
}
func (u *user) NKeys() int {
return 3
}
```


### Arrays, Slices and Channels

To unmarshal a JSON object to a slice an array or a channel, it must implement the UnmarshalerArray interface:
```go
type UnmarshalerArray interface {
UnmarshalArray(*Decoder) error
}
```
UnmarshalArray method takes one argument, a pointer to the Decoder (*gojay.Decoder). If the JSON data is not an array, the Unmarshal method will never be called.

Example of implementation with a slice:
```go
type testSlice []string

func (t *testStringArr) UnmarshalArray(dec *gojay.Decoder) error {
str := ""
if err := dec.AddString(&str); err != nil {
return err
}
*t = append(*t, str)
return nil
}
```

Example of implementation with a channel:
```go
type ChannelString chan string

func (c *ChannelArray) UnmarshalArray(dec *gojay.Decoder) error {
str := ""
if err := dec.AddString(&str); err != nil {
return err
}
*c <- str
return nil
}
```

### Stream Decoding
GoJay ships with a powerful stream decoder.

It allows to read continuously from an io.Reader stream and do JIT decoding writing unmarshalled JSON to a channel to allow async consuming.

When using the Stream API, the Decoder implements context.Context to provide graceful cancellation.

Example:
```go
type ChannelStream chan *TestObj

func (c *ChannelStream) UnmarshalStream(dec *gojay.StreamDecoder) error {
obj := &TestObj{}
if err := dec.AddObject(obj); err != nil {
return err
}
*c <- obj
return nil
}

func main() {
// create our channel which will receive our objects
streamChan := ChannelStream(make(chan *TestObj))
// get a reader implementing io.Reader
reader := getAnIOReaderStream()
dec := gojay.Stream.NewDecoder(reader)
// start decoding (will block the goroutine until something is written to the ReadWriter)
go dec.DecodeStream(&streamChan)
for {
select {
case v := <-streamChan:
// do something with my TestObj
case <-dec.Done():
os.Exit("finished reading stream")
}
}
}
```

### Other types

## Encoding

Example of basic structure encoding:
```go
import "github.com/francoispqt/gojay"

type user struct {
id int
name string
email string
}
// implement UnmarshalerObject
func (u *user) MarshalObject(dec *gojay.Decoder, key string) {
dec.AddIntKey("id", u.id)
dec.AddStringKey("name", u.name)
dec.AddStringKey("email", u.email)
}
func (u *user) IsNil() bool {
return u == nil
}

func main() {
u := &user{1, "gojay", "gojay@email.com"}
b, _ := gojay.MarshalObject(user)
fmt.Println(string(b)) // {"id":1,"name":"gojay","email":"gojay@email.com"}
}
```

### Structs
### Arrays and Slices
### Other types

# Benchmarks

Benchmarks encode and decode three different data based on size (small, medium, large).

To run benchmark for decoder:
```bash
cd $GOPATH/src/github.com/francoispqt/gojay/benchmarks/decoder && make bench
```

To run benchmark for encoder:
```bash
cd $GOPATH/src/github.com/francoispqt/gojay/benchmarks/encoder && make bench
```

# Benchmark Results
## Decode

<img src="https://images2.imgbox.com/78/01/49OExcPh_o.png" width="500px">

### Small Payload
[benchmark code is here](https://github.com/francoispqt/gojay/blob/master/benchmarks/decoder/decoder_bench_small_test.go)

[benchmark data is here](https://github.com/francoispqt/gojay/blob/master/benchmarks/benchmarks_small.go)

| | ns/op | bytes/op | allocs/op |
|-------------|-------|--------------|-----------|
| Std Library | 4661 | 496 | 12 |
| JsonParser | 1313 | 0 | 0 |
| JsonIter | 899 | 192 | 5 |
| GoJay | 662 | 112 | 1 |

### Medium Payload
[benchmark code is here](https://github.com/francoispqt/gojay/blob/master/benchmarks/decoder/decoder_bench_medium_test.go)

[benchmark data is here](https://github.com/francoispqt/gojay/blob/master/benchmarks/benchmarks_medium.go)

| | ns/op | bytes/op | allocs/op |
|-------------|-------|--------------|-----------|
| Std Library | 30148 | 2152 | 496 |
| JsonParser | 7793 | 0 | 0 |
| JsonIter | 5967 | 496 | 44 |
| GoJay | 3914 | 128 | 12 |

### Large Payload
[benchmark code is here](https://github.com/francoispqt/gojay/blob/master/benchmarks/decoder/decoder_bench_large_test.go)

[benchmark data is here](https://github.com/francoispqt/gojay/blob/master/benchmarks/benchmarks_large.go)

| | ns/op | bytes/op | allocs/op |
|-------------|-------|--------------|-----------|
| JsonParser | 66813 | 0 | 0 |
| JsonIter | 87994 | 6738 | 329 |
| GoJay | 43402 | 1408 | 76 |

## Encode

<img src="https://images2.imgbox.com/e9/cc/pnM8c7Gf_o.png" width="500px">

### Small Struct
[benchmark code is here](https://github.com/francoispqt/gojay/blob/master/benchmarks/encoder/encoder_bench_small_test.go)

[benchmark data is here](https://github.com/francoispqt/gojay/blob/master/benchmarks/benchmarks_small.go)

| | ns/op | bytes/op | allocs/op |
|-------------|-------|--------------|-----------|
| Std Library | 1280 | 464 | 3 |
| JsonIter | 866 | 272 | 3 |
| GoJay | 484 | 320 | 2 |
### Medium Struct
[benchmark code is here](https://github.com/francoispqt/gojay/blob/master/benchmarks/encoder/encoder_bench_medium_test.go)

[benchmark data is here](https://github.com/francoispqt/gojay/blob/master/benchmarks/benchmarks_medium.go)

| | ns/op | bytes/op | allocs/op |
|-------------|-------|--------------|-----------|
| Std Library | 3325 | 1496 | 18 |
| JsonIter | 1939 | 648 | 16 |
| GoJay | 1196 | 936 | 16 |

### Large Struct
[benchmark code is here](https://github.com/francoispqt/gojay/blob/master/benchmarks/encoder/encoder_bench_large_test.go)

[benchmark data is here](https://github.com/francoispqt/gojay/blob/master/benchmarks/benchmarks_large.go)

| | ns/op | bytes/op | allocs/op |
|-------------|-------|--------------|-----------|
| Std Library | 51317 | 28704 | 326 |
| JsonIter | 35247 | 14608 | 320 |
| GoJay | 27847 | 27888 | 326 |

# Contributing

Contributions are welcome :)

If you encounter issues please report it in Github and/or send an email at [francois@parquet.ninja](mailto:francois@parquet.ninja)

Loading

0 comments on commit d6ba85a

Please sign in to comment.