couchbase-quickinit is a Docker image designed for testing and local development. It starts up a single, standalone Couchbase Server instance within a Docker container and initializes it with buckets, indexes, events, and more.
This can be very useful in reducing developer friction, providing a way to quickly and easily spin up a Couchbase server preinitialized for your application. By including an Dockerfile and associated configuration files within your source control repo, you can version your development data definitions along with your application.
The latest version can be pulled using:
docker pull btburnett3/couchbase-quickinit:latestThe latest tag will be the latest Enterprise edition of Couchbase.
Specific versions may also be available, such as enterprise-7.2.3. This would be the Enterprise edition of Couchbase, version 7.2.3.
To use couchbase-quickinit, create your own Dockerfile that uses couchbase-quickinit as its base image. Then add configuration files to the /startup directory of the new image. You may also override environment variables to change the Couchbase Server configuration.
Alternatively, you may mount a volume to the /startup directory of the container to provide the configuration files.
FROM btburnett3/couchbase-quickinit:enterprise-7.2.3
# Customize environment
ENV CB_DATARAM=256 \
CB_PASSWORD=mypassword
# Copy files
COPY . /startup/The following environment variables can be set to change the Couchbase Server configuration:
| Env Variable | Description |
|---|---|
| CB_CLUSTER_NAME | Specify the name of the cluster |
| CB_DATARAM | Data service RAM in megabytes, default 512 |
| CB_INDEXRAM | Index service RAM in megabytes, default 256 |
| CB_SEARCHRAM | Search (FTS) service RAM in megabytes, default 256 |
| CB_ANALYTICSRAM | Analytics service RAM in megabytes. Only applicable if cbas is added to CB_SERVICES |
| CB_EVENTINGRAM | Eventing service RAM in megabytes. Only applicable if eventing is added to CB_SERVICES |
| CB_SERVICES | Services to enable, default kv,n1ql,index,fts |
| CB_INDEXSTORAGE | Index storage mode, forestdb (default) or memory_optimized |
| CB_USERNAME | Couchbase user name, default Administrator |
| CB_PASSWORD | Couchbase password, default password |
Values for CB_SERVICES and CB_INDEXSTORAGE correspond to parameters for the Couchbase REST API.
NOTE: If you configure CB_SERVICES to create the cbas analytics service, make sure you set CB_ANALYTICSRAM to a minimum of 1024.
To configure your buckets, simply place a buckets.json file in the /startup directory of your image. This file should contain an array of bucket definition objects.
[
{
"name": "sample",
"ramQuotaMB": 100,
"bucketType": "couchbase",
"authType": "sasl",
"saslPassword": "",
"evictionPolicy": "fullEviction",
"replicaNumber": 0,
"flushEnabled": 1
},
{
"name": "default",
"ramQuotaMB": 100,
"bucketType": "couchbase",
"authType": "sasl",
"saslPassword": "",
"evictionPolicy": "fullEviction",
"replicaNumber": 0,
"flushEnabled": 1
}
]Attribute names and values in this file correspond with the Couchbase REST API create bucket endpoint.
If this file is not overridden in your image, it will create a single bucket named default with a RAM quota of 100MB.
NOTE: Only applicable for Couchbase Server 7+
To create scopes and collections, create a file underneath /startup with the name of your bucket, and a file with the following name: collections.json. For example, /startup/sample/collections.json. Note Names are case sensitive.
The format of the collections.json file should be as follows:
{
"scopes": {
"your_scope_name": {
"collections": [
"your_collection_name_1",
"your_collection_name_2",
"your_collection_name_3"
]
}
}
}The values to replace are your_scope_name and the values in the collections array. Note: You can use the _default scope if you'd like by replacing your_scope_name with _default. The _default scope is automatically created in all buckets and cannot be deleted. You can add multiple scopes each having their own collections. For example:
{
"scopes": {
"_default": {
"collections": [
"default_collection_name_1",
"default_collection_name_2"
]
},
"my_scope": {
"collections": [
"default_collection_name_1",
"default_collection_name_2"
]
}
}
}To configure RBAC users for Couchbase Server versions 5+, simply place a rbac-users.json file in the /startup directory of your image. This file should be an array of JSON objects that define the various users and roles that need to be associated with each user. See the following example on how to structure the file:
[
{
"rbacName": "App User",
"rbacUsername": "app-user",
"rbacPassword": "password",
"roles": [
"bucket_full_access[sample]"
]
},
{
...
}
]Information on the available roles can be found here. If you want to limit the role to a specific bucket, place the bucket name in brackets at the end of the name, i.e. bucket_full_access[sample].
To create views, add a directory underneath /startup with the name of your bucket and a text file named views.json. This file should be a JSON object with one or more design document specifications. The name of each attribute should be the name of the design document.
{
"customers": {
"views": {
"CustomersByFirstName": {
"map": "function (doc, meta) {\n if ((doc.type === \"customer\") && doc.firstName) {\n emit(doc.firstName, null);\n }\n}"
}
}
}
}Examples of the syntax for design documents can be found in the Couchbase documentation. Note that views.json has an extra nesting level above the Couchbase examples, as it supports more than one design document in a single file.
To create indexes, add a directory underneath /startup with the name of your bucket and a text file named indexes.n1ql. For example, /startup/default/indexes.n1ql. Note that the names are case sensitive.
Within this file, you can define the CREATE INDEX statements for your bucket, separated by semicolons. It is recommended for performance to use WITH {"defer_build": true} for all indexes, and use a BUILD INDEX statement at the end of the file.
CREATE PRIMARY INDEX `#primary` ON default WITH {"defer_build": true};
CREATE INDEX `Types` ON default (`type`) WITH {"defer_build": true};
BUILD INDEX ON default (`#primary`, `Types`);Alternatively, you may add YAML files with index definitions under the /startup/<bucketname>/indexes folder. This operation uses couchbase-index-manager to create the indexes. See here for an explanation of the YAML file format.
To setup the analytics service datasets, add a directory under the /startup/<bucketname>/analytics folder. Within this folder create a text file named dataset.json. For example, /startup/default/analytics/dataset.json. Note that the names are case sensitive.
Within this file, create a key called statements with an array as the value. Within the array you can define your CREATE DATAVERSE and CREATE DATASET statements for your bucket, separated by semicolons. Below is an example showing what the dataset.json structure should look like. IMPORTANT Make sure to append the proper USE statement to each CREATE DATASET statement so that it's placed in the proper DATAVERSE. Additionaly, always end your file with a CONNECT LINK Local; statement.
{
"statements": [
"CREATE DATAVERSE `sample` IF NOT EXISTS;",
"USE `sample`; CREATE DATASET IF NOT EXISTS users ON `sample` WHERE `type` = 'user';",
"USE `sample`; CONNECT LINK Local;"
]
}To create analytics indexes, add a directory under the /startup/<bucketname>/analytics folder. Within this folder create a text file named indexes.json. For example, /startup/default/analytics/indexes.json. Note that the names are case sensitive.
Within this file, create a key called statements with an array as the value. Within the array you can define your CREATE INDEX statements for your DATASET, separated by semicolons. IMPORTANT Make sure your queries always start with a USE statment otherwise the query engine will have no idea which DATAVERSE to associate the index with.
{
"statements": [
"USE `sample`; CREATE INDEX `idx_users` IF NOT EXISTS ON `users` (id: string);"
]
}To create FTS indexes, add a directory underneath /startup with the name of your bucket, and underneath that a fts directory. Within that, add a json file for each index, with the file name being the index name. For example, /startup/default/fts/my_index.json. Note that names are case sensitive.
Within this file, place the JSON index definition. This can be easily exported from the Couchbase Console.
To create FTS index aliases, add an additional file in the same folder named aliases.json. This file should be an object with each attribute being an alias name, and the value being an array of index names.
{
"my_alias": ["my_index"],
"my_second_alias": ["my_index_2", "my_index_3"]
}Couchbase Eventing allows document mutations from a bucket to be streamed, processed using Javascript, and outputs performed such as storing new documents in another bucket. couchbase-quickinit can create and deploy these events automatically on startup.
First, add a directory within your startup folder named events. Within this directory, add two files for
each event you'd like to deploy.
event-name.json will have configuration, specifically the depcfg configuration for source buckets,
metadata buckets, and buckets which may be referenced by the event. The settings attribute can also be
used to override any settings, defaults apply for any excluded settings. An easy way to get these
settings is to manually configure an event and then get the definition from http://localhost:8096/api/v1/functions
(use HTTP Basic Authentication).
{
"depcfg": {
"buckets": [
{
"alias": "dst",
"bucket_name": "default",
"access": "rw"
}
],
"curl": [],
"metadata_bucket": "default",
"source_bucket": "sample"
},
"settings": {
"worker_count": 3
}
}The second file should have the same name but a .js extension, event-name.js. This file
contains the Javascript for the event.
function OnUpdate(doc, meta) {
// This example event copies all documents from the "example" bucket to the "default" bucket
dst[meta.id] = doc;
}
function OnDelete(meta) {
}The event will be automatically deployed once it is created with the "Everything" feed boundary.
This means that all documents in the source bucket should be processed by the event at startup,
including any documents created from models. However, the /nodestatus/initialized file will be
created before all documents are processed, as events are asynchronous in nature.
Also, note that by default logging will be set to the DEBUG level and each event will use only 1
worker. This is because couchbase-quickinit is intended for local machine development or CI testing.
These settings can be overridden in the JSON file in the settings attribute.
An example image configuration can be found here.
To run the example locally:
- Ensure that Couchbase Server is not currently running on your machine to avoid port conflicts
git clone https://github.com/brantburnett/couchbase-quickinit.gitcd couchbase-quickinit/exampledocker-compose up -d- The server will be accessible at http://localhost:8091 after 15-30 seconds. The username is "Administrator", password is "password".
To shut down and cleanup:
docker-compose down
couchbase-quickinit is designed to be extended by creating a new Docker image that uses it as a base image. This allows you to provide your own additional startup logic that runs after couchbase-quickinit has completed but before the container is marked as initialized. This is done by adding your own scripts to the image and appending them to the /scripts/additional-init.sh file.
FROM btburnett3/couchbase-quickinit:enterprise-7.2.3
COPY ./my-init.sh /scripts/my-init.sh
RUN echo "/scripts/my-init.sh" >> /scripts/additional-init.shIf you are using the Couchbase Server Community images, then note that configuration of enterprise features may cause your settings/configuration to fail.
For example:
- Couchbase Server Community does not have Eventing. Therefore any json configuration files in the
eventsfolder may be ignored or may cause your configuration to fail. - Couchbase Server Community does not support
memory_optimizedindex storage. Setting CB_INDEXSTORAGE tomemory_optimizedmay be ignored or may cause your configuration to fail.
For more information: read about the differences between Community and Enterprise editions