Database Adapters (github.com/moleculer-go/store)

Moleculer framework has an official set of Database adapters. Use them to persist your data in a database.

{% note info Database per service%} Moleculer follows the one database per service pattern. To learn more about this design pattern and its implications check this article. {% endnote %}

Features

default CRUD actions (create, find, count, list, get, update, remove)
cached actions
pagination support
pluggable adapter - There is the default memory adapter for testing & prototyping)
official adapters for MongoDB.
fields filtering
populating
encode/decode IDs
entity lifecycle events for notifications

Memory Adapter

Moleculer's memory adapter uses hashicorp/go-memdb. Use it to quickly set up and test you prototype and for writing test cases.

{% note warn%} Only use this adapter for prototyping and testing. When you are ready to go into production simply swap to Mongo ... adapters as they all implement common Settings, Actions. {% endnote %}

Install

$ go get -u github.com/moleculer-go/store

# Or with go modules
github.com/moleculer-go/store v1.0.1

Usage

package main

import (
  "fmt"
  "time"

  "github.com/moleculer-go/store"

  "github.com/moleculer-go/moleculer"
  "github.com/moleculer-go/moleculer/broker"
)

func main() {
  var bkr = broker.New(&moleculer.Config{LogLevel: "info"})
  bkr.Publish(moleculer.ServiceSchema{
    Name: "users",
    Settings: map[string]interface{}{
      "fields":    []string{"_id", "username", "name"},
      "populates": map[string]interface{}{"friends": "users.get"},
    },
    Mixins: []moleculer.Mixin{store.Mixin(&store.MemoryAdapter{
      Table:        "users",
      SearchFields: []string{"name", "username"},
    })},
  })
  bkr.Start()
  time.Sleep(time.Millisecond * 300)
  user := <-bkr.Call("users.create", map[string]interface{}{
    "username": "john",
    "name":     "John Doe",
    "status":   1,
  })

  id := user.Get("id").String()
  // Get all users
  fmt.Printf("all users: ", <-bkr.Call("users.find", map[string]interface{}{}))

  // List users with pagination
  fmt.Printf("list users: ", <-bkr.Call("users.list", map[string]interface{}{
    "page":     2,
    "pageSize": 10,
  }))

  idParam := map[string]interface{}{"id": id}

  // Get a user
  fmt.Printf("get user: ", <-bkr.Call("users.get", idParam))

  // Update a user
  fmt.Printf("update user: ", <-bkr.Call("users.update", map[string]interface{}{
    "id":   id,
    "name": "Jane Doe",
  }))

  // Print user after update
  fmt.Printf("get user: ", <-bkr.Call("users.get", idParam))

  // Delete a user
  fmt.Printf("remove user: ", <-bkr.Call("users.remove", idParam))

  bkr.Stop()
}

run the example above with:

$ go run github.com/moleculer-go/store/examples/users

More examples can be found on GitHub

Settings

All DB adapters share these common set of settings:

Property	Type	Default	Description
`idField`	`string`	required	Name of ID field.
`fields`	`[]string`	["**"]	Field filtering list. It must be an `Array`. If the value is nil it will assume ["**"] and it will not filter the fields of entities.
`populates`	`map[string]interface{}`		Schema for population. Read more.
`pageSize`	`Number`	required	Default page size in `list` action.
`maxPageSize`	`Number`	required	Maximum page size in `list` action.
`maxLimit`	`Number`	required	Maximum value of limit in `find` action. Default: `-1` (no limit)
`entityValidator`	`Object`, `function`	`null`	Validator schema or a function to validate the incoming entity in `create` action.

Actions

DB adapters also implement CRUD operations. These actions are public methods and can be called by other services.

`find`

Find entities by query.

Parameters

Property	Type	Default	Description
`populate`	`[]string`	-	Populated fields.
`fields`	`[]string`	-	Fields filter.
`limit`	`Number`	required	Max count of rows.
`offset`	`Number`	required	Count of skipped rows.
`sort`	`string`	required	Sorted fields.
`search`	`string`	required	Search text.
`searchFields`	`string`	required	Fields for searching.
`query`	`map[string]interface{}`	required	Query object. Passes to adapter.

Results

Type: moluculer.Paylod - List of found entities.

`count`

Get count of entities by query.

Parameters

Property	Type	Default	Description
`search`	`string`	required	Search text.
`searchFields`	`string`	required	Fields list for searching.
`query`	`Object`	required	Query object. Passes to adapter.

Results

Type: Number - Count of found entities.

`list`

List entities by filters and pagination results.

Parameters

Property	Type	Default	Description
`populate`	`[]string`	-	Populated fields.
`fields`	`[]string`	-	Fields filter.
`page`	`Number`	required	Page number.
`pageSize`	`Number`	required	Size of a page.
`sort`	`string`	required	Sorted fields.
`search`	`string`	required	Search text.
`searchFields`	`string`	required	Fields for searching.
`query`	`map[string]interface{}`	required	Query object. Passes to adapter.

Results

Type: moleculer.Payload - List of found entities and count.

`create`

Create a new entity.

Parameters

Payload with fields to be saved in the new entity record.

Results

Type: moleculer.Payload - Saved entity.

`get`

Get entity by ID.

Parameters

Property	Type	Default	Description
`id`	`string`	required	ID of entity.
`ids`	`[]string`	required	ID(s) of entities.
`populate`	`[]string`	-	Field list for populate.
`fields`	`[]string`	-	Fields filter.
`mapping`	`Bool`	-	Convert the returned `Array` to `Map` where the key is the value of `id`.

Results

Type: moleculer.Payload - Found entity(ies).

`update`

Update an entity by ID.

After update, clear the cache & call lifecycle events.

Parameters

Property	Type	Default	Description
`id`	`string`	-	Id of the records being updated.

Results

Type: moleculer.Payload - Updated entity.

`remove`

Remove an entity by ID.

Parameters

Property	Type	Default	Description
`id`	`string`	required	ID of entity.

Results

Type: Number - Count of removed entities.

Populating

The service allows you to easily populate fields from other services. For exapmle: If you have an author field in post entity, you can populate it with users service by ID of author. If the field is an Array of IDs, it will populate all entities via only one request

Example of populate schema

package main

import (
 "fmt"
 "time"

 "github.com/moleculer-go/store"

 "github.com/moleculer-go/moleculer"
 "github.com/moleculer-go/moleculer/broker"
)

func main() {
 var bkr = broker.New(&moleculer.Config{LogLevel: "info"})
 bkr.Publish(moleculer.Service{
  Name: "users",
  Settings: map[string]interface{}{
   "fields":    []string{"id", "username", "name"},
   "populates": map[string]interface{}{"friends": "users.get"},
  },
  Mixins: []moleculer.Mixin{store.Mixin(&store.MemoryAdapter{
   Table:        "users",
   SearchFields: []string{"name", "username"},
  })},
 })
 bkr.Publish(moleculer.Service{
  Name: "posts",
  Settings: map[string]interface{}{
   "populates": map[string]interface{}{
    //Shorthand populate rule. Resolve the 'voters' values with the users.get action.
    "voters": "users.get",
    // Define the params of action call.
    //It will receive only with username of author.
    "author": map[string]interface{}{
     "action": "users.get",
     "params": map[string]interface{}{
      "fields": []string{"username"},
     },
    },
   },
  },
  Mixins: []moleculer.Mixin{store.Mixin(&store.MemoryAdapter{
   Table: "posts",
  })},
 })
 bkr.Start()
 time.Sleep(time.Millisecond * 300)

 johnSnow := <-bkr.Call("users.create", map[string]interface{}{
  "name":     "John",
  "lastname": "Snow",
  "username": "jsnow",
  "fullname": "John Snow",
 })
 marie := <-bkr.Call("users.create", map[string]interface{}{
  "name":     "Marie",
  "lastname": "Claire",
  "username": "mclaire",
  "fullname": "Marie Claire",
 })

 post := <-bkr.Call("posts.create", map[string]interface{}{
  "content": "Lorem ipsum dolor sit amet, consectetur ...",
  "voters":  []string{marie.Get("id").String()},
  "author":  johnSnow.Get("id").String(),
  "status":  1,
 })

 // List posts with populated author
 fmt.Printf("posts with author: ", <-bkr.Call("posts.find", map[string]interface{}{
  "populate": []string{"author"},
 }))

 // List posts with populated voters
 fmt.Printf("posts with voters: ", <-bkr.Call("posts.find", map[string]interface{}{
  "populate": []string{"voters"},
 }))

 // remove post
 <-bkr.Call("posts.remove", map[string]interface{}{
  "id": post.Get("id").String(),
 })

 //remove users
 <-bkr.Call("users.remove", map[string]interface{}{
  "id": johnSnow.Get("id").String(),
 })
 <-bkr.Call("users.remove", map[string]interface{}{
  "id": marie.Get("id").String(),
 })

 bkr.Stop()
}

run the example above with:
$ go run github.com/moleculer-go/store/examples/populates

The populate parameter is available in find, list and get actions.

Extend with custom actions

Naturally you can extend this service with your custom actions.

package main

import (
 "fmt"
 "time"

 "github.com/moleculer-go/store"

 "github.com/moleculer-go/moleculer"
 "github.com/moleculer-go/moleculer/broker"
)

func main() {
 var bkr = broker.New(&moleculer.Config{LogLevel: "info"})
 bkr.Publish(moleculer.Service{
  Name: "users",
  Settings: map[string]interface{}{
   "fields":    []string{"id", "username", "name"},
   "populates": map[string]interface{}{"friends": "users.get"},
  },
  Mixins: []moleculer.Mixin{store.Mixin(&store.MemoryAdapter{
   Table:        "users",
   SearchFields: []string{"name", "username"},
  })},
 })
 adapter := &store.MemoryAdapter{
  Table: "posts",
 }
 bkr.Publish(moleculer.Service{
  Name: "posts",
  Settings: map[string]interface{}{
   "populates": map[string]interface{}{
    //Shorthand populate rule. Resolve the 'voters' values with the users.get action.
    "voters": "users.get",
    // Define the params of action call. It will receive only with username & full name of author.
    "author": map[string]interface{}{
     "action": "users.get",
     "params": map[string]interface{}{
      "fields": []string{"username", "fullname"},
     },
    },
   },
  },
  Mixins: []moleculer.Mixin{store.Mixin(adapter)},
  Actions: []moleculer.Action{
   {
    Name: "byAuthors",
    Handler: func(ctx moleculer.Context, params moleculer.Payload) interface{} {
     return <-ctx.Call("posts.find", map[string]interface{}{
      "query": map[string]interface{}{
       "author": params.Get("authorId").String(),
      },
      "limit": 10,
      "sort":  "-createdAt",
     })
    },
   },
  },
 })
 bkr.Start()
 time.Sleep(time.Millisecond * 300)

 johnSnow := <-bkr.Call("users.create", map[string]interface{}{
  "name":     "John",
  "lastname": "Snow",
  "username": "jsnow",
  "fullname": "John Snow",
 })
 marie := <-bkr.Call("users.create", map[string]interface{}{
  "name":     "Marie",
  "lastname": "Claire",
  "username": "mclaire",
  "fullname": "Marie Claire",
 })

 post := <-bkr.Call("posts.create", map[string]interface{}{
  "content": "Lorem ipsum dolor sit amet, consectetur ...",
  "voters":  []string{marie.Get("id").String()},
  "author":  johnSnow.Get("id").String(),
  "status":  1,
 })

 // List posts with populated authors
 fmt.Printf("posts by authors: ", <-bkr.Call("posts.byAuthors", map[string]interface{}{
  "authorId": johnSnow.Get("id").String(),
 }))

 // remove post
 <-bkr.Call("posts.remove", map[string]interface{}{
  "id": post.Get("id").String(),
 })

 //remove users
 <-bkr.Call("users.remove", map[string]interface{}{
  "id": johnSnow.Get("id").String(),
 })
 <-bkr.Call("users.remove", map[string]interface{}{
  "id": marie.Get("id").String(),
 })

 bkr.Stop()
}

run the example above with:
$ go run github.com/moleculer-go/store/examples/customActions

Cache

Not Implemented yet!

Mongo Adapter

This adapter is based on MongoDB.

Install

$ go get -u github.com/moleculer-go/store

# Or with go modules
github.com/moleculer-go/store v1.0.1

{% note info Dependencies%} To use this adapter you need to install MongoDB on you system. {% endnote %}

Usage

package main

import (
  "fmt"
  "time"

  "github.com/moleculer-go/store"
  "github.com/moleculer-go/store/mongo"
  "github.com/spf13/cobra"

  "github.com/moleculer-go/moleculer"
  "github.com/moleculer-go/moleculer/broker"
  "github.com/moleculer-go/moleculer/cli"
)

func main() {
  cli.Start(
    &moleculer.Config{LogLevel: "info"},
    func(broker *broker.ServiceBroker, cmd *cobra.Command) {
      broker.Publish(moleculer.ServiceSchema{
        Name: "users",
        Settings: map[string]interface{}{
          "fields":    []string{"_id", "username", "name"},
          "populates": map[string]interface{}{"friends": "users.get"},
        },
        Mixins: []moleculer.Mixin{store.Mixin(&mongo.MongoAdapter{
          MongoURL:   "mongodb://localhost:27017",
          Collection: "users",
          Database:   "test",
          Timeout:    time.Second * 5,
        })},
      })
      broker.Start()
      time.Sleep(time.Millisecond * 300)
      user := <-broker.Call("users.create", map[string]interface{}{
        "username": "john",
        "name":     "John Doe",
        "status":   1,
      })

      id := user.Get("id").String()
      // Get all users
      fmt.Printf("all users: ", <-broker.Call("users.find", map[string]interface{}{}))

      // List users with pagination
      fmt.Printf("list users: ", <-broker.Call("users.list", map[string]interface{}{
        "page":     2,
        "pageSize": 10,
      }))

      idParam := map[string]interface{}{"id": id}

      // Get a user
      fmt.Printf("get user: ", <-broker.Call("users.get", idParam))

      // Update a user
      fmt.Printf("update user: ", <-broker.Call("users.update", map[string]interface{}{
        "id":   id,
        "name": "Jane Doe",
      }))

      // Print user after update
      fmt.Printf("get user: ", <-broker.Call("users.get", idParam))

      // Delete a user
      fmt.Printf("remove user: ", <-broker.Call("users.remove", idParam))

      broker.Stop()

    })
}

run the example above with:

$ go run github.com/moleculer-go/store/examples/usersMongo start

More MongoDB examples can be found on GitHub

SQLite Adapter

This adapter is based on crawshaw/sqlite.

Install

$ go get -u github.com/moleculer-go/store

# Or with go modules
github.com/moleculer-go/store v1.0.1

Usage

package main

import (
  "fmt"
  "time"

  "github.com/moleculer-go/store"
  "github.com/moleculer-go/store/sqlite"
  "github.com/spf13/cobra"

  "github.com/moleculer-go/moleculer"
  "github.com/moleculer-go/moleculer/broker"
  "github.com/moleculer-go/moleculer/cli"
)

func main() {
  cli.Start(
    &moleculer.Config{LogLevel: "debug"},
    func(broker *broker.ServiceBroker, cmd *cobra.Command) {
      broker.Publish(moleculer.ServiceSchema{
        Name: "users",
        Settings: map[string]interface{}{
          "fields":    []string{"id", "username", "name"},
          "populates": map[string]interface{}{"friends": "users.get"},
        },
        Mixins: []moleculer.Mixin{store.Mixin(&sqlite.Adapter{
          URI:   "file:memory:?mode=memory",
          Table: "users",
          Columns: []sqlite.Column{
            {
              Name: "username",
              Type: "string",
            },
            {
              Name: "name",
              Type: "string",
            },
            {
              Name: "status",
              Type: "integer",
            },
          },
        })},
      })
      broker.Start()
      time.Sleep(time.Millisecond * 300)
      user := <-broker.Call("users.create", map[string]interface{}{
        "username": "john",
        "name":     "John Doe",
        "status":   1,
      })

      id := user.Get("id").String()
      // Get all users
      fmt.Printf("all users: ", <-broker.Call("users.find", map[string]interface{}{}))

      // List users with pagination
      fmt.Printf("list users: ", <-broker.Call("users.list", map[string]interface{}{
        "page":     2,
        "pageSize": 10,
      }))

      idParam := map[string]interface{}{"id": id}

      // Get a user
      fmt.Printf("get user: ", <-broker.Call("users.get", idParam))

      // Update a user
      fmt.Printf("update user: ", <-broker.Call("users.update", map[string]interface{}{
        "id":   id,
        "name": "Jane Doe",
      }))

      // Print user after update
      fmt.Printf("get user: ", <-broker.Call("users.get", idParam))

      // Delete a user
      fmt.Printf("remove user: ", <-broker.Call("users.remove", idParam))

      broker.Stop()

    })
}

run the example above with:

$ go run github.com/moleculer-go/store/examples/usersSQLite start

More Database adaptor examples can be found on GitHub

Name		Name	Last commit message	Last commit date
Latest commit History 124 Commits
.snapshots		.snapshots
e2e_tests		e2e_tests
elastic		elastic
examples		examples
mocks		mocks
mongo		mongo
sqlite		sqlite
.drone.yml		.drone.yml
.gitignore		.gitignore
.travis.yml		.travis.yml
LICENSE		LICENSE
README.md		README.md
adapter.go		adapter.go
adapter_suite_test.go		adapter_suite_test.go
adapter_test.go		adapter_test.go
go.mod		go.mod
go.sum		go.sum
memory_adapter.go		memory_adapter.go
memory_adapter_test.go		memory_adapter_test.go
no_adapter.go		no_adapter.go

License

moleculer-go/store

Folders and files

Latest commit

History

Repository files navigation

Database Adapters (github.com/moleculer-go/store)

Features

Memory Adapter

Install

Usage

run the example above with:

Settings

Actions

Parameters

Results

Parameters

Results

Parameters

Results

Parameters

Results

Parameters

Results

Parameters

Results

Parameters

Results

Populating

Extend with custom actions

Cache

Mongo Adapter

Install

Usage

run the example above with:

SQLite Adapter

Install

Usage

run the example above with:

About

Resources

License

Stars

Watchers

Forks

Languages