Skip to content
This repository has been archived by the owner on Dec 10, 2021. It is now read-only.

Commit

Permalink
Support Badger (#38)
Browse files Browse the repository at this point in the history
* Support Badger

* Update CHANGES.md
  • Loading branch information
mosuka authored Mar 18, 2019
1 parent e6b42da commit 959f3c3
Showing 15 changed files with 614 additions and 8 deletions.
1 change: 1 addition & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
@@ -10,6 +10,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/).

### Added

- Support Badger #38
- Add index stats #37
- Add Wikipedia example #35
- Support cznicb and leveldb #34
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -45,7 +45,7 @@ RUN apt-get update && \
GOOS=linux \
GOARCH=amd64 \
CGO_ENABLED=1 \
BUILD_TAGS="kagome icu libstemmer cld2 cznicb leveldb" \
BUILD_TAGS="kagome icu libstemmer cld2 cznicb leveldb badger" \
VERSION="${VERSION}" \
make build

9 changes: 9 additions & 0 deletions Gopkg.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 6 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -120,7 +120,7 @@ You can enable supported Bleve Extensions for like following:
```bash
$ make \
GOOS=linux \
BUILD_TAGS="kagome icu libstemmer cld2 cznicb leveldb" \
BUILD_TAGS="kagome icu libstemmer cld2 cznicb leveldb badger" \
CGO_ENABLED=1 \
build
```
@@ -131,14 +131,13 @@ $ make \
```bash
$ make GOOS=darwin \
GOOS=darwin \
BUILD_TAGS="kagome icu libstemmer cld2 cznicb leveldb" \
BUILD_TAGS="kagome icu libstemmer cld2 cznicb leveldb badger" \
CGO_ENABLED=1 \
CGO_LDFLAGS="-L/usr/local/opt/icu4c/lib" \
CGO_CFLAGS="-I/usr/local/opt/icu4c/include" \
build
```


Please refer to the following table for details of Bleve Extensions:

| BUILD_TAGS | CGO_ENABLED | Description |
@@ -149,7 +148,7 @@ Please refer to the following table for details of Bleve Extensions:
| libstemmer | 1 | Enable Language Stemmer (Danish, German, English, Spanish, Finnish, French, Hungarian, Italian, Dutch, Norwegian, Portuguese, Romanian, Russian, Swedish, Turkish) |
| cznicb | 0 | Enable cznicb KV store |
| leveldb | 1 | Enable LevelDB |

| badger | 0 | Enable Badger |

You can see the binary file when build successful like so:

@@ -164,7 +163,7 @@ blast-index
Running a Blast index node is easy. Start Blast data node like so:

```bash
$ ./bin/blast-index start --node-id=index1 --data-dir=/tmp/blast/index1 --bind-addr=:6060 --grpc-addr=:5050 --http-addr=:8080 --index-mapping-file ./example/index_mapping.json
$ ./bin/blast-index start --node-id=index1 --data-dir=/tmp/blast/index1 --bind-addr=:6060 --grpc-addr=:5050 --http-addr=:8080 --index-mapping-file=./example/index_mapping.json
```

Please refer to following document for details of index mapping:
@@ -433,8 +432,8 @@ $ curl -X DELETE 'http://127.0.0.1:8080/documents/enwiki_1'
Blast is easy to bring up the cluster. Blast data node is already running, but that is not fault tolerant. If you need to increase the fault tolerance, bring up 2 more data nodes like so:

```bash
$ ./bin/blast-index start --node-id=index2 --data-dir=/tmp/blast/index2 --bind-addr=:6061 --grpc-addr=:5051 --http-addr=:8081 --index-mapping-file ./example/index_mapping.json --join-addr=:5050
$ ./bin/blast-index start --node-id=index3 --data-dir=/tmp/blast/index3 --bind-addr=:6062 --grpc-addr=:5052 --http-addr=:8082 --index-mapping-file ./example/index_mapping.json --join-addr=:5050
$ ./bin/blast-index start --node-id=index2 --data-dir=/tmp/blast/index2 --bind-addr=:6061 --grpc-addr=:5051 --http-addr=:8081 --index-mapping-file=./example/index_mapping.json --join-addr=:5050
$ ./bin/blast-index start --node-id=index3 --data-dir=/tmp/blast/index3 --bind-addr=:6062 --grpc-addr=:5052 --http-addr=:8082 --index-mapping-file=./example/index_mapping.json --join-addr=:5050
```

_Above example shows each Blast node running on the same host, so each node must listen on different ports. This would not be necessary if each node ran on a different host._
21 changes: 21 additions & 0 deletions config/config_badger.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
// Copyright (c) 2019 Minoru Osuka
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

// +build badger full

package config

import (
_ "github.com/mosuka/bbadger"
)
1 change: 1 addition & 0 deletions index/raft_server.go
Original file line number Diff line number Diff line change
@@ -27,6 +27,7 @@ import (
"github.com/golang/protobuf/ptypes/any"
"github.com/hashicorp/raft"
raftboltdb "github.com/hashicorp/raft-boltdb"
_ "github.com/mosuka/blast/config"
"github.com/mosuka/blast/errors"
"github.com/mosuka/blast/protobuf"
"github.com/mosuka/blast/protobuf/index"
32 changes: 32 additions & 0 deletions vendor/github.com/mosuka/bbadger/README.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

17 changes: 17 additions & 0 deletions vendor/github.com/mosuka/bbadger/bleve.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

22 changes: 22 additions & 0 deletions vendor/github.com/mosuka/bbadger/go.mod

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 959f3c3

Please sign in to comment.